Towards ensuring software interoperability between deep learning frameworks

• Autorzy: Lee Youn Kyu , Park Seong Hee , Lim Min Young , Lee Soo-Hyun , Jeong Jongwook

Identyfikatory

DOI

10.2478/jaiscr-2023-0016

• Języki publikacji: EN

Abstrakty

EN

With the widespread of systems incorporating multiple deep learning models, ensuring interoperability between target models has become essential. However, due to the unreliable performance of existing model conversion solutions, it is still challenging to ensure interoperability between the models developed on different deep learning frameworks. In this paper, we propose a systematic method for verifying interoperability between pre- and post-conversion deep learning models based on the validation and verification approach. Our proposed method ensures interoperability by conducting a series of systematic verifications from multiple perspectives. The case study confirmed that our method successfully discovered the interoperability issues that have been reported in deep learning model conversions.

Słowa kluczowe

EN

deep learning; interoperability; validation; verification; deep learning framework; model conversion

• Wydawca: University of Social Sciences
• Czasopismo: Journal of Artificial Intelligence and Soft Computing Research
• Rocznik: 2023
• Tom: Vol. 13, No. 4
• Strony: 215--228
• Opis fizyczny: Bibliogr.37 poz., rys.

Twórcy

autor

Lee Youn Kyu

• Department of Computer Engineering, Hongik University

• https://orcid.org/0000-0002-4569-2640

autor

Park Seong Hee

• Department of Computer Engineering, Hongik University

• https://orcid.org/0000-0002-6088-0658

autor

Lim Min Young

• Department of Computer Engineering, Hongik University

• https://orcid.org/0000-0001-8016-5945

autor

Lee Soo-Hyun

• Department of Computer Engineering, Hongik University

• https://orcid.org/0000-0001-6696-0387

autor

Jeong Jongwook

• Department of Computer Science and Artificial Intelligence, Jeonbuk National University, 567 Baekje-daero, Deokjin-gu, Jeonju-si (54896), Republic of Korea

• https://orcid.org/0000-0002-3307-0940

Bibliografia

• [1] G. Nguyen, S. Dlugolinsky, M. Bobák, V. Tran, A. Lopez García, I. Heredia, P. Malík, and L. Hluchy, bMachine learning and deep learning frameworks band libraries for large-scale data mining: a survey, Artificial Intelligence Review, 52, 2019, pages 77-124.
• [2] S. Pouyanfar, S. Sadiq, Y. Yan, H. Tian, Y. Tao, M.P. Reyes, M.L. Shyu, S.C. Chen, S.S. and Iyengar, A survey on deep learning: Algorithms, techniques, and applications, ACM Computing Surveys (CSUR), 51(5), 2018, pages 1-36.
• [3] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G.S. Corrado, A. Davis, J. Dean, M. Devin, and S. Ghemawat, Tensorflow: Large-scale machine learning on heterogeneous distributed systems, 2016, arXiv:1603.04467.
• [4] F. Chollet, Keras: The python deep learning library, 2015, https://github.com/fchollet/keras.
• [5] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, and A. Desmaison, Pytorch: An imperative systems, 32, 2019.
• [6] Y. Cheng, D. Wang, P. Zhou, and T. Zhang, A survey of model compression and acceleration for deep neural networks, 2017, arXiv:1710.09282.
• [7] T.G. Dietterich, Ensemble learning, In: The handbook of brain theory and neural networks, The MIT Press, 2002, pages 110-125.
• [8] J. Gao, P. Li, Z. Chen, and J. Zhang, A survey on deep learning for multimodal data fusion, Neural Computation, 32(5), 2020, pages 829-864.
• [9] Facebook and Microsoft, ONNX: Open Neural Network Exchange, 2017, https://github.com/onnx/onnx.
• [10] The Khronos Group, Neural Network Exchange Format (NNEF), 2016, https://www.khronos.org/nnef.
• [11] Y. Liu, C. Chen, R. Zhang, T. Qin, X. Ji, H. Lin, and M. Yang, Enhancing the interoperability between deep learning frameworks by model conversion, In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, pages 1320-1330.
• [12] Hahnyuan, Neural network tools: Converter and analyzer, 2017, https://github.com/hahnyuan/nntools.
• [13] Gmalivenko, pytorch2keras: Pytorch to keras model convertor, 2019, https://github.com/gmalivenko/pytorch2keras.
• [14] Z. Chen, Y. Cao, Y. Liu, H. Wang, T. Xie, and X. Liu, A comprehensive study on challenges in deploying deep learning based software, In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2020, pages 750-762.
• [15] M. Openja, A. Nikanjam, A.H. Yahmed, F. Khomh, and Z.M.J. Jiang, An empirical study of challenges in converting deep learning models, In 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME), 2022, pages 13-23.
• [16] H. Pham, Software reliability, Springer Science& Business Media, 2000.
• [17] J. Schumann, P. Gupta, and S. Nelson, On verification & validation of neural network based controllers, EANN’03, 2003.
• [18] Facebook, CAFFE2, 2017, https://caffe2.ai/.
• [19] Apache Software Foundation, A flexible and oficjant library for Deep Learning, 2017, https://mxnet.apache.org/versions/1.9.1/.Long, R. Girshick, S. Guadarrama, and T. Darrell, Caffe: Convolutional architecture for fast feature embedding, In Proceedings of the 22nd ACM international conference on Multimedia, 2014, pages 675-678.
• [21] Woodsgao, pytorch2caffe, 2010, https://github.com/woodsgao/pytorch2caffe.
• [22] Darshan, Torch to ONNX conversion going wrong, 2021, https://discuss.pytorch.org/t/torch-to-onnxconversion-going-wrong/121596.
• [23] ys.yusaito, Inference result is different between Pytorch and ONNX model, 2022, https://discuss.pytorch.org/t/inference-resultis-different-between-pytorch-and-onnxmodel/147228/1.
• [24] J. Schumann, and K. Goseva-Popstojanova, Verification and validation approaches for modelbased software engineering, In 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), 2019, pages 514-518.
• [25] L. Roeder, Netron, 2020, https://netron.app/.
• [26] T. Fawcett, An introduction to ROC analysis. Pattern recognition letters, 27(8), 2006, pages 861-874.
• [27] Y. Jiang, Prediction of monthly mean daily diffuse solar radiation using artificial neural networks and comparison with other empirical models, Energy policy, 36(10), 2008, pages 3833-3837.
• [28] S. Zeiml, U. Seiler, K. Altendorfer, and T. Felberbauer, Simulation evaluation of automated forecast error correction based on mean percentage error, In 2020 Winter Simulation Conference (WSC), 2020, pages 1572-1583.
• [29] M. Nejadgholi, and J. Yang, A study of Oracle approximations in testing deep learning libraries, In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2019, pages 785-796.
• [30] M.V. Shcherbakov, A. Brebels, N.L. Shcherbakova, A.P. Tyukov, T.A. Janovsky, and V.A.E. Kamaev, A survey of forecast error measures, World applied sciences journal, 24(24),2013, pages 171-176.
• [31] H. Li, W. Ma, Y. Lian, and X. Wang, Estimating daily global solar radiation by day of year in China, Applied Energy, 87(10), 2010, pages 3011-3017.
• [32] K. Simonyan, and A. Zisserman, Very deep convolutional networks for large-scale image recognition, 2014, arXiv:1409.1556.
• [33] L. Deng, The mnist database of handwritten digit images for machine learning research [best of the web], IEEE signal processing magazine, 29(6), 2012, pages 141-142.
• [34] A. Krizhevsky, I. Sutskever, and G.E. Hinton, Imagenet classification with deep convolutional neural networks, Communications of the ACM, 60(6), 2017, pages 84-90.
• [35] Android Google, Keeping your app responsive, 2017, https://developer.android.com/training/articles/perfanr.html.
• [36] G. Kechit, Top 10 deep learning frameworks in 2022 you can’t ignore, 2022, https://www.upgrad.com/blog/top-deep-learningframeworks.
• [37] O.G. Yalc¸ın, Top 5 Deep Learning Frameworks to Watch in 2021 and Why TensorFlow, 2021, https://towardsdatascience.com/top-5-deeplearning-frameworks-to-watch-in-2021-and-whytensorflow-98d8d6667351.

Uwagi

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023).