One-shot federated learning with self-adversarial data

• Autorzy: Danilenka Anastasiya , Bogacka Karolina , Wasielewska-Michniewska Katarzyna

Identyfikatory

DOI

10.15439/2023F1102

• Języki publikacji: EN

Abstrakty

EN

Federated learning (FL) is a decentralized approach that aims at training a global model with the help of multiple devices, without collecting or revealing individual clients' data. The training of a federated model is conducted in communication rounds. Still, in certain scenarios, numerous communication rounds are impossible to perform. In such cases, a one-shot FL is utilized, where the number of communication rounds is limited to one. In this article, the idea of one-shot FL is enhanced with the usage of adversarial data, exploring and illustrating the possibilities to improve the performance of resulting global models, including scenarios with non-IID data, for image classification datasets: MNIST and CIFAR-10.

Słowa kluczowe

EN

performance evaluation; computer science; federated learning; computational modeling; computer architecture; data model

PL

ocena wydajności; informatyka; uczenie federacyjne; modelowanie obliczeniowe; architektura komputerów; model danych

• Wydawca: Polskie Towarzystwo Informatyczne
• Czasopismo: Annals of Computer Science and Information Systems
• Rocznik: 2023
• Tom: Vol. 35
• Strony: 919--924
• Opis fizyczny: Bibliogr. 22 poz., wz., wykr.

Twórcy

autor

Danilenka Anastasiya

• Faculty of Mathematics and Information Science Warsaw University of Technology ul. Koszykowa 75, 00-662 Warsaw, Poland

• https://orcid.org/0000-0002-3080-0303

autor

Bogacka Karolina

• Faculty of Mathematics and Information Science Warsaw University of Technology ul. Koszykowa 75, 00-662 Warsaw, Poland

• https://orcid.org/0000-0002-7109-891X

autor

Wasielewska-Michniewska Katarzyna

• Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

Bibliografia

• 1. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, A. Singh and J. Zhu, Eds., vol. 54. PMLR, 20–22 Apr 2017, pp. 1273–1282. [Online]. Available: https://proceedings.mlr.press/v54/mcmahan17a.html
• 2. O. Shahid, S. Pouriyeh, R. M. Parizi, Q. Z. Sheng, G. Srivastava, and L. Zhao, “Communication efficiency in federated learning: Achievements and challenges,” 2021.
• 3. M. Chen, H. V. Poor, W. Saad, and S. Cui, “Convergence time optimization for federated learning over wireless networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 4, pp. 2457–2471, 2021. http://dx.doi.org/10.1109/TWC.2020.3042530
• 4. Z. Yang, M. Chen, W. Saad, C. S. Hong, and M. Shikh-Bahaei, “Energy efficient federated learning over wireless communication networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 3, pp. 1935–1949, 2021. http://dx.doi.org/10.1109/TWC.2020.3037554
• 5. M. Chen, N. Shlezinger, H. V. Poor, Y. C. Eldar, and S. Cui, “Communication-efficient federated learning,” Proceedings of the National Academy of Sciences, vol. 118, no. 17, p. e2024789118, 2021. http://dx.doi.org/10.1073/pnas.2024789118. [Online]. Available: https://www.pnas.org/doi/abs/10.1073/pnas.2024789118
• 6. L. Fei-Fei, R. Fergus, and P. Perona, “One-shot learning of object categories,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 4, p. 594–611, apr 2006. http://dx.doi.org/10.1109/TPAMI.2006.79. [Online]. Available: https://doi.org/10.1109/TPAMI.2006.79
• 7. N. Guha, A. Talwalkar, and V. Smith, “One-shot federated learning,” 2019.
• 8. Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” CoRR, vol. abs/1806.00582, 2018. [Online]. Available: http://arxiv.org/abs/1806.00582
• 9. H. Zhu, J. Xu, S. Liu, and Y. Jin, “Federated learning on non-iid data: A survey,” CoRR, vol. abs/2106.06843, 2021. [Online]. Available: https://arxiv.org/abs/2106.06843
• 10. C. Xiao and S. Wang, “An experimental study of class imbalance in federated learning,” in 2021 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, dec 2021. http://dx.doi.org/10.1109/ssci50451.2021.9660072. [Online]. Available: https://doi.org/10.1109\%2Fssci50451.2021.9660072
• 11. Y. Zhou, G. Pu, X. Ma, X. Li, and D. Wu, “Distilled one-shot federated learning,” 2021.
• 12. L. Pinheiro Cinelli, M. Araújo Marins, E. A. Barros da Silva, and S. Lima Netto, Variational Autoencoder. Cham: Springer International Publishing, 2021, pp. 111–149. ISBN 978-3-030-70679-1. [Online]. Available: https://doi.org/10.1007/978-3-030-70679-1_5
• 13. C. E. Heinbaugh, E. Luz-Ricca, and H. Shao, “Data-free one-shot federated learning under very high statistical heterogeneity,” in The Eleventh International Conference on Learning Representations, 2023. [Online]. Available: https://openreview.net/forum?id=_hb4vM3jspB
• 14. K. Ren, T. Zheng, Z. Qin, and X. Liu, “Adversarial attacks and defenses in deep learning,” Engineering, vol. 6, no. 3, pp. 346–360, 2020. http://dx.doi.org/https://doi.org/10.1016/j.eng.2019.12.012. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S209580991930503X
• 15. I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” 2014. [Online]. Available: https://arxiv.org/abs/1412.6572
• 16. A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” 2017.
• 17. Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and J. Li, “Boosting adversarial attacks with momentum,” 2018.
• 18. N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in 2017 IEEE Symposium on Security and Privacy (SP), 2017. http://dx.doi.org/10.1109/SP.2017.49 pp. 39–57.
• 19. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” 2014.
• 20. O. Suciu, R. Marginean, Y. Kaya, H. D. III, and T. Dumitras, “When does machine learning FAIL? generalized transferability for evasion and poisoning attacks,” in 27th USENIX Security Symposium (USENIX Security 18). Baltimore, MD: USENIX Association, Aug. 2018. ISBN 978-1-939133-04-5 pp. 1299–1316. [Online]. Available: https://www.usenix.org/conference/usenixsecurity18/presentation/suciu
• 21. A. Danilenka, “Mitigating the effects of non-iid data in federated learning with a self-adversarial balancing method,” 2023, submitted to publication.
• 22. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998. http://dx.doi.org/10.1109/5.726791

Uwagi

1. Thematic Tracks Short Papers

2. Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).