The High-Level Practical Overview of Open-Source Privacy-Preserving Machine Learning Solutions

• Autorzy: Kuźniewski Konrad , Matusiewicz Krystian , Sapiecha Piotr

Identyfikatory

DOI

10.24425/ijet.2022.143880

• Języki publikacji: EN

Abstrakty

EN

This paper aims to provide a high-level overview of practical approaches to machine-learning respecting the privacy and confidentiality of customer information, which is called Privacy-Preserving Machine Learning. First, the security approaches in offline-learning privacy methods are assessed. Those focused on modern cryptographic methods, such as Homomorphic Encryption and Secure Multi-Party Computation, as well as on dedicated combined hardware and software platforms like Trusted Execution Environment - Intel® Software Guard Extensions (Intel® SGX). Combining the security approaches with different machine learning architectures leads to our Proof of Concept in which the accuracy and speed of the security solutions will be examined. The next step was exploring and comparing the Open-Source Python-based solutions for PPML. Four solutions were selected from almost 40 separate, state-of-the-art systems: SyMPC, TF-Encrypted, TenSEAL, and Gramine. Three different Neural Network architectures were designed to show different libraries’ capabilities. The POC solves the image classification problem based on the MNIST dataset. As the computational results show, the accuracy of all considered secure approaches is similar. The maximum difference between non-secure and secure flow does not exceed 1.2%. In terms of secure computations, the most effective Privacy-Preserving Machine Learning library is based on Trusted Execution Environment, followed by Secure Multi-Party Computation and Homomorphic Encryption. However, most of those are at least 1000 times slower than the nonsecure evaluation. Unfortunately, it is not acceptable for a realworld scenario. Future work could combine different security approaches, explore other new and existing state-of-the-art libraries or implement support for hardware-accelerated secure computation.

Słowa kluczowe

EN

Privacy-Preserving Machine Learning; Homomorphic Encryption; Secure Multiparty Computation; Trusted Execution Environment

• Wydawca: Polish Academy of Sciences, Committee of Electronics and Telecommunication
• Czasopismo: International Journal of Electronics and Telecommunications
• Rocznik: 2022
• Tom: Vol. 68. No. 4
• Strony: 741--747
• Opis fizyczny: Bibliogr. 62 poz., rys.

Twórcy

autor

Kuźniewski Konrad

• Intel, the IPAS division

autor

Matusiewicz Krystian

• Intel, the IPAS division

autor

Sapiecha Piotr

• Intel, the IPAS division

Bibliografia

• [1] F. Newsroom. (2018) Fda permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems. [Online]. Available: https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-based-device-detect-certain-diabetes-related-eye.
• [2] FDA. Artificial intelligence and machine learning (ai/ml)-enabled medical devices. [Online]. Available: https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-aiml-enabled-medical-devices.
• [3] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Y. Bengio and Y. LeCun, Eds., 2014. [Online]. Available: http://arxiv.org/abs/1312.6199.
• [4] A. Shafahi, W. R. Huang, M. Najibi, O. Suciu, C. Studer, T. Dumitras, and T. Goldstein, “Poison frogs! targeted clean-label poisoning attacks on neural networks,” in Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montr ́eal, Canada, S. Bengio, H. M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., 2018, pp. 6106-6116. [Online]. Available: https://proceedings.neurips.cc/paper/ 2018/hash/22722a343513ed45f14905eb07621686-Abstract.html.
• [5] X. Yuan, P. He, Q. Zhu, and X. Li, “Adversarial examples: Attacks and defenses for deep learning,” IEEE Trans. Neural Networks Learn. Syst., vol. 30, no. 9, pp. 2805–2824, 2019. [Online]. Available: https://doi.org/10.1109/TNNLS.2018.2886017.
• [6] D. M. Bamasoud, A. S. Al-Dossary, N. M. Al-Harthy, R. A. Al-Shomrany, G. S. Alghamdi, and R. O. Algahmdi, “Privacy and security issues in cloud computing: A survey paper,” in International Conference on Information Technology, ICIT 2021, Amman, Jordan, July 14-15, 2021. IEEE, 2021, pp. 387–392. [Online]. Available: https://doi.org/10.1109/ICIT52682.2021.9491632.
• [7] Y. Zhang and R. Sion, “Speculative execution attacks and cloud security,” in Proceedings of the 2019 ACM SIGSAC Conference on Cloud Computing Security Workshop, CCSW@CCS 2019, London, UK, November 11, 2019, R. Sion and C. Papamanthou, Eds. ACM, 2019, p. 201. [Online]. Available: https://doi.org/10.1145/3338466.3360287.
• [8] Y. Alghofaili, A. Albattah, N. Alrajeh, M. A. Rassam, and B. A. S. Al-rimy, “Secure cloud infrastructure: A survey on issues, current solutions, and open challenges,” Applied Sciences, vol. 11, no. 19, 2021. [Online]. Available: https://www.mdpi.com/2076-3417/11/19/9005.
• [9] N. Dowlin, R. Gilad-Bachrach, K. Laine, K. Lauter, M. Naehrig, and J. Wernsing, “Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy,” Tech. Rep. MSR-TR-2016-3, 2016. [Online]. Available: https://www.microsoft.com/en-us/research/ publication/cryptonets-applying-neural-networks-to-encrypted-data- with-high-throughput-and-accuracy/.
• [10] J. Alvarez-Valle, P. Bhatu, N. Chandran, D. Gupta, A. Nori, A. Rastogi, M. Rathee, R. Sharma, and S. Ugare, “Secure medical image analysis with cryptflow,” 2020.
• [11] A. Soin, P. Bhatu, R. Takhar, N. Chandran, D. Gupta, J. Alvarez-Valle, R. Sharma, V. Mahajan, and M. P. Lungren, “Multi-institution encrypted medical imaging ai validation without data sharing,” 2021.
• [12] M. H. M. Elham Tabassi (NIST), Kevin Burns (MITRE). A taxonomy and terminology of adversarial machine learning. [Online]. Available: https://csrc.nist.gov/publications/detail/nistir/8269/draft.
• [13] Y. LeCun and C. Cortes, “MNIST handwritten digit database,” 2010. [Online]. Available: http://yann.lecun.com/exdb/mnist/.
• [14] F. Boemer, A. Costache, R. Cammarota, and C. Wierzynski, “ngraphhe2: A high-throughput framework for neural network inference on encrypted data,” 2019.
• [15] F. Boemer, Y. Lao, R. Cammarota, and C. Wierzynski, “ngraph-he: A graph compiler for deep learning on homomorphically encrypted data,” 2019.
• [16] A. Benaissa, B. Retiat, B. Cebere, and A. E. Belfedhal, “Tenseal: A library for encrypted tensor operations using homomorphic encryption,” 2021.
• [17] S. Carpov, P. Dubrulle, and R. Sirdey, “Armadillo: A compilation chain for privacy preserving applications,” in Proceedings of the 3rd International Workshop on Security in Cloud Computing, ser. SCC ’15. Association for Computing Machinery, 2015, p. 13-19. [Online]. Available: https://doi.org/10.1145/2732516.2732520
• [18] I. Chillotti, N. Gama, M. Georgieva, and M. Izabach`ene, “Faster fully homomorphic encryption: Bootstrapping in less than 0.1 seconds,” Cryptology ePrint Archive, Report 2016/870, 2016, https://ia.cr/2016/870.
• [19] S. S. Magara, C. Yildirim, F. Yaman, B. Dilekoglu, F. R. Tutas, E. ̈Ozt ̈urk, K. Kaya, ̈O. Tastan, and E. Savas, “Ml with he: Privacy preserving machine learning inferences for genome studies,” 2021.
• [20] R. Dathathri, O. Saarikivi, H. Chen, K. Laine, K. Lauter, S. Maleki, M. Musuvathi, and T. Mytkowicz, “Chet: an optimizing compiler for fully-homomorphic neural-network inferencing,” in Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, 2019, pp. 142-156.
• [21] E. Hesamifard, H. Takabi, M. Ghasemi, and R. N. Wright, “Privacy-preserving machine learning as a service,” Proc. Priv. Enhancing Technol., vol. 2018, no. 3, pp. 123-142, 2018. [Online]. Available: https://doi.org/10.1515/popets-2018-0024.
• [22] C. Boura, N. Gama, M. Georgieva, and D. Jetchev, “Chimera: Combining ring-lwe-based fully homomorphic encryption schemes,” Cryptology ePrint Archive, Report 2018/758, 2018, https://ia.cr/2018/758.
• [23] Q. Lou, B. Feng, G. C. Fox, and L. Jiang, “Glyph: Fast and accurately training deep neural networks on encrypted data,” 2020.
• [24] OpenMined. (2021) Tenseal library. [Online]. Available: https://github.com/OpenMined/TenSEAL.
• [25] J. H. Cheon, D. Kim, D. Kim, H. H. Lee, and K. Lee, “Numerical method for comparison on homomorphically encrypted numbers,” Cryptology ePrint Archive, Report 2019/417, 2019, https://ia.cr/2019/417.
• [26] J. H. Cheon, D. Kim, and D. Kim, “Efficient homomorphic comparison methods with optimal complexity,” Cryptology ePrint Archive, Report 2019/1234, 2019, https://ia.cr/2019/1234.
• [27] U. Michelucci, Advanced applied deep learning : convolutional neural networks and object detection. Apress, 2019.
• [28] A. Dalskov, D. Escudero, and M. Keller, “Secure evaluation of quantized neural networks,” Cryptology ePrint Archive, Report 2019/131, 2019, https://ia.cr/2019/131.
• [29] OpenMined. (2021) Sympc library. [Online]. Available: https://github.com/OpenMined/SyMPC.
• [30] N. Kumar, M. Rathee, N. Chandran, D. Gupta, A. Rastogi, and R. Sharma, “Cryptflow: Secure tensorflow inference,” 2020.
• [31] D. Rathee, M. Rathee, N. Kumar, N. Chandran, D. Gupta, A. Rastogi, and R. Sharma, “Cryptflow2: Practical 2-party secure inference,” Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, 2020. [Online]. Available: http://dx.doi.org/10.1145/3372297.3417274.
• [32] D. Rathee, M. Rathee, R. K. K. Goli, D. Gupta, R. Sharma, N. Chandran, and A. Rastogi, “Sirnn: A math library for secure rnn inference,” Cryptology ePrint Archive, Report 2021/459, 2021, https://ia.cr/2021/459.
• [33] B. Knott, S. Venkataraman, A. Hannun, S. Sengupta, M. Ibrahim, and L. van der Maaten, “Crypten: Secure multi-party computation meets machine learning,” in arXiv 2109.00984, 2021.
• [34] M. Dahl, J. Mancuso, Y. Dupis, B. Decoste, M. Giraud, I. Livingstone, J. Patriquin, and G. Uhma, “Private machine learning in tensorflow using secure computation,” 2018.
• [35] W. Henecka, S. K ̈ogl, A.-R. Sadeghi, T. Schneider, and I. Wehrenberg, “Tasty: Tool for automating secure two-party computations,” Cryptology ePrint Archive, Report 2010/365, 2010, https://ia.cr/2010/365.
• [36] P. Mohassel and P. Rindal, “Aby¡sup¿3¡/sup¿: A mixed protocol framework for machine learning,” in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’18. Association for Computing Machinery, 2018, p. 35-52. [Online]. Available: https://doi.org/10.1145/3243734.3243760.
• [37] S. Wagh, D. Gupta, and N. Chandran, “Securenn: Efficient and private neural network training,” Cryptology ePrint Archive, Report 2018/442, 2018, https://ia.cr/2018/442.
• [38] W. Zheng, R. Deng, W. Chen, R. A. Popa, A. Panda, and I. Stoica, “Cerebro: A platform for Multi-Party cryptographic collaborative learning,” in 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 2021, pp. 2723-2740. [Online]. Available: https://www.usenix.org/conference/usenixsecurity21/presentation/zheng.
• [39] S. Wagh, S. Tople, F. Benhamouda, E. Kushilevitz, P. Mittal, and T. Rabin, “Falcon: Honest-majority maliciously secure framework for private deep learning,” 2020.
• [40] M. S. Riazi, M. Samragh, H. Chen, K. Laine, K. Lauter, and F. Koushanfar, “Xonn: Xnor-based oblivious deep neural network inference,” 2019.
• [41] M. S. Riazi, C. Weinert, O. Tkachenko, E. M. Songhori, T. Schneider, and F. Koushanfar, “Chameleon: A hybrid secure computation frame-work for machine learning applications,” 2018.
• [42] A.-R. Sadeghi and T. Schneider, “Generalized universal circuits for secure evaluation of private functions with application to data clas-sification,” Cryptology ePrint Archive, Report 2008/453, 2008, https://ia.cr/2008/453.
• [43] M. Barni, P. Failla, R. Lazzeretti, A.-R. Sadeghi, and T. Schneider, “Privacy-preserving ecg classification with branching programs and neural networks,” IEEE Transactions on Information Forensics and Security, vol. 6, no. 2, pp. 452–468, 2011.
• [44] P. Mohassel and Y. Zhang, “Secureml: A system for scalable privacy-preserving machine learning,” in 2017 IEEE Symposium on Security and Privacy (SP), 2017, pp. 19-38.
• [45] N. Koti, A. Patra, R. Rachuri, and A. Suresh, “Tetrad: Actively secure 4pc for secure training and inference,” Cryptology ePrint Archive, Report 2021/755, 2021, https://ia.cr/2021/755.
• [46] A. Patra and A. Suresh, “Blaze: Blazing fast privacy-preserving machine learning,” Proceedings 2020 Network and Distributed System Security Symposium, 2020. [Online]. Available: http://dx.doi.org/10.14722/ndss.2020.24202.
• [47] N. Koti, M. Pancholi, A. Patra, and A. Suresh, “Swift: Super-fast and robust privacy-preserving machine learning,” 2021.
• [48] EzPC. (2021) Ezpc. [Online]. Available: https://github.com/mpc-msri/EzPC.
• [49] PySyft. (2021) Pysyft. [Online]. Available: https://github.com/OpenMined/PySyft.
• [50] T. Ryffel, P. Tholoniat, D. Pointcheval, and F. Bach, “Ariann: Low-interaction privacy-preserving deep learning via function secret sharing,” 2021.
• [51] D. Labs. (2021) tf-encrypted library. [Online]. Available: https://github.com/tf-encrypted/tf-encrypted.
• [52] gramine. (2021) gramine, library. [Online]. Available: https://github.com/gramineproject/gramine.
• [53] D. Labs. (2021) tf-trusted, library. [Online]. Available: https://github.com/capeprivacy/tf-trusted.
• [54] F. Tram`er and D. Boneh, “Slalom: Fast, verifiable and private execution of neural networks in trusted hardware,” 2019.
• [55] F. Mo, H. Haddadi, K. Katevas, E. Marin, D. Perino, and N. Kourtellis, “Ppfl: Privacy-preserving federated learning with trusted execution environments,” 2021.
• [56] F. Mo, A. S. Shamsabadi, K. Katevas, S. Demetriou, I. Leontiadis, A. Cavallaro, and H. Haddadi, “Darknetz,” Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services, 2020. [Online]. Available: http://dx.doi.org/10.1145/3386901.3388946.
• [57] J. J. Dai, Y. Wang, X. Qiu, D. Ding, Y. Zhang, Y. Wang, X. Jia, L. C. Zhang, Y. Wan, Z. Li, J. Wang, S. Huang, Z. Wu, Y. Wang, Y. Yang, B. She, D. Shi, Q. Lu, K. Huang, and G. Song, “Bigdl: A distributed deep learning framework for big data,” in Proceedings of the ACM Symposium on Cloud Computing, ser. SoCC’19. Association for Computing Machinery, 2019, pp. 50-60. [Online]. Available: https://arxiv.org/pdf/1804.05839.pdf.
• [58] M. Orenbach, P. Lifshits, M. Minkin, and M. Silberstein, “Eleos: Exitless os services for sgx enclaves,” in Proceedings of the Twelfth European Conference on Computer Systems, ser. EuroSys ’17. Association for Computing Machinery, 2017, p. 238-253. [Online]. Available: https://doi.org/10.1145/3064176.3064219.
• [59] R. Kunkel, D. L. Quoc, F. Gregor, S. Arnautov, P. Bhatotia, and C. Fetzer, “Tensorscone: A secure tensorflow framework using intel sgx,” 2019.
• [60] W. Ozga, D. L. Quoc, and C. Fetzer, “Perun: Secure multi-stakeholder machine learning framework with gpu support,” 2021.
• [61] A. Mondal, Y. More, R. H. Rooparaghunath, and D. Gupta, “Flatee: Federated learning across trusted execution environments,” 2021.
• [62] LeNET. (2021) Lenet. [Online]. Available: https://en.wikipedia.org/wiki/LeNet.

Uwagi

Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2022-2023).