Improving testing of multi-agent systems: An innovative deep learning strategy for automatic, scalable, and dynamic error detection and optimisation

• Autorzy: Dehimi Nour El Houda , Tolba Zakaria , Medkour Mehdi , Hadjadj Anis , Galland Stéphane

Identyfikatory

DOI

10.24425/bpasts.2025.154062

• Języki publikacji: EN

Abstrakty

EN

In this paper, a novel method is introduced for automated, scalable, and dynamic identification of errors in various behavioural versions of a multi-agent system under test, employing deep learning techniques. It is designed to enable accurate error detection, thus opening new possibilities for improving and optimising traditional testing techniques. The approach consists of two phases. The first phase is the training of a deep learning model using randomly generated inputs and predicted outputs generated from the behavioural model of each version. The second phase consists of detecting errors in the multi-agent system under test by replacing the predicted outputs with which the model is trained with execution outputs. The envisioned strategy is put into action through a real case study, which serves to vividly showcase and affirm its practical efficacy.

Słowa kluczowe

EN

multi-agent systems; testing; error detection; artificial intelligence techniques; deep learning

PL

systemy wieloagentowe; testowanie; wykrywanie błędów; techniki sztucznej inteligencji; głębokie uczenie

• Wydawca: Polska Akademia Nauk, Wydział IV Nauk Technicznych
• Czasopismo: Bulletin of the Polish Academy of Sciences. Technical Sciences
• Rocznik: 2025
• Tom: Vol. 73, nr 3
• Strony: art. no. e154062
• Opis fizyczny: Bibliogr. 54 poz., rys., wykr.

Twórcy

autor

Dehimi Nour El Houda

• LIAOA Laboratory, Department of Mathematics and Computer Science, University of Oum El Bouaghi, Algeria

• https://orcid.org/0000-0001-9402-2304

autor

Tolba Zakaria

• LIAOA Laboratory, Department of Mathematics and Computer Science, University of Oum El Bouaghi, Algeria

autor

Medkour Mehdi

• LIAOA Laboratory, Department of Mathematics and Computer Science, University of Oum El Bouaghi, Algeria

autor

Hadjadj Anis

• LIAOA Laboratory, Department of Mathematics and Computer Science, University of Oum El Bouaghi, Algeria

autor

Galland Stéphane

• UTBM, CIAD UR 7533, F-90010 Belfort, France

Bibliografia

• [1] S.T. Goonatilleke and B. Hettige, “Past, present, and future trends in multi-agent system technology,” J. Eur. Syst. Autom., vol. 55, no. 6, pp. 723–739, 2022, doi: 10.18280/jesa.550604.
• [2] S. Bitimanova and A. Shukirova, “Agents and Multi-agent Systems in the Management of Electric Energy Systems,” Manage. Product. Eng. Rev., vol. 14, no. 2, pp. 99–110, Jun. 2023, doi: 10.24425/mper.2023.146027.
• [3] B. Das, B. Subudhi, and B.B. Pati, “Formation control of underwater vehicles using Multi Agent System,” Arch. Control Sci., vol. 30, no. 2, pp. 365–384, Jun. 2020, doi: 10.24425/acs.2020.133503.
• [4] P. Qaderi-Baban, M.B. Menhaj, M. Dosaranian-Moghaddam, and A. Fakharian, “Intelligent multi-agent system for DC microgrid energy coordination control,” Bull. Pol. Acad. Sci. Tech. Sci., vol. 67, no. 4, 2019, doi: 10.24425/bpasts.2019.130183.
• [5] J. Ferber, O. Gutknecht, and F. Michel, “From agents to organizations: An organizational view of multi-agent systems,” in Proc. 4th Int. Workshop Agent-Oriented Softw. Eng., 2003, vol. 2935, pp. 214–230.
• [6] M. Cossentino, N. Gaud, V. Hilaire, S. Galland, and A. Koukam, “ASPECS: An agent-oriented software process for engineering complex systems—How to design agent societies under a holonic perspective,” Int. J. Autonomous Agents Multi-Agent Syst., vol. 2, no. 2, pp. 260–304, 2010.
• [7] M. Wooldridge, N.R. Jennings, and D. Kinny, “The GAIA methodology for agent-oriented analysis and design,” Int. J. Autonomous Agents Multi-Agent Syst., vol. 3, no. 3, pp. 285–312, 2000.
• [8] J. Pavón, J. Gómez-Sanz, and R. Fuentes, “The INGENIAS methodology and tools,” in Agent-Oriented Methodologies, 2005, pp. 236–276.
• [9] M. Hannoun, O. Boissier, J.S. Sichman, and C. Sayettat, “MOISE: An organizational model for multi-agent systems,” in Advances in Artificial Intelligence, IBERAMIA-SBIA, 2000, pp. 156–165.
• [10] B. Putten, V. Dignum, M. Sierhuis, and S. Wolfe, “OperA and Brahms: A symphony?” in Agent-Oriented Software Engineering IX. AOSE 2008, Lecture Notes in Computer Science, 2009.
• [11] A. Kiran, W. H. Butt, M. W. Anwar, F. Azam, and B. Maqbool, “A Comprehensive Investigation of Modern Test Suite Optimization Trends, Tools and Techniques,” IEEE Access, vol. 7, pp. 89093–89117, 2019.
• [12] S. Zardari et al., “A comprehensive bibliometric assessment on software testing (2016–2021),” Electronics, vol. 11, no. 8, p. 1984, 2022, doi: 10.3390/electronics11131984.
• [13] C.D. Nguyen, A. Perini, C. Bernon, J. Pavón, and J. Thangarajah, “Testing in multi-agent systems,” in Proc. Int. Workshop Agent Oriented Software Engineering, Budapest, Hungary, 2009, pp. 180–190.
• [14] Z. Zhang, J. Thangarajah, and L. Padgham, “Automated unit testing intelligent agents in PDT,” in AAMAS (Demos), 2008, pp. 1673–1674.
• [15] E.E. Ekinci, A.M. Tiryaki, O. Cetin, and O. Dikenelli, “Goaloriented agent testing revisited,” in Proc. 9th Int. Workshop Agent-Oriented Software Engineering, 2008, pp. 85–96.
• [16] C.D. Nguyen, A. Perini, and P. Tonella, “Goal-oriented testing for MASs,” Int. J. Agent-Oriented Software Engineering, vol. 4, no. 1, pp. 79–109, 2010.
• [17] D.N. Lam and K.S. Barber, “Debugging agent behaviour in an implemented agent system,” in PROMAS 2004, R.H. Bordini, M.M. Dastani, J. Dix, and A. El Fallah Seghrouchni, Eds., Springer, Heidelberg, vol. 3346, pp. 104–125, 2005.
• [18] C.D. Nguyen, S. Miles, A. Perini, P. Tonella, M. Harman, and M. Luck, “Evolutionary testing of autonomous software agents,” in Proc. 8th Int. Conf. Autonomous Agents and Multiagent Systems (AAMAS 2009), IFAAMAS, 2009, pp. 521–528.
• [19] C.D. Nguyen, A. Perini, and P. Tonella, “Ontology-based test generation for multi-agent systems,” in Proc. Int. Conf. Autonomous Agents and Multiagent Systems, 2008.
• [20] M. Woodward, “Mutation testing: An evolving technique,” in Colloquium Software Testing for Critical Systems, 1990.
• [21] N.E.H. Dehimi, Z. Tolba, and N. Djabelkhir, “Testing inclusive, exclusive, and parallel interactions in multi-agents system: A new model-based approach,” Int. J. Saf. Secur. Eng., vol. 14, no. 4, pp. 1125–1138, 2024.
• [22] D. Guassmi, N.E.H. Dehimi, and M. Derdour, “A state of art review on testing open multi-agent systems,” in Novel and Intelligent Digital Systems Conferences, Athens, Greece, 2023, pp. 262–266, doi: 10.1007/978-3-031-44097-7_28.
• [23] S. Boukeloul, N.E.H. Dehimi, and M. Derdour, “A state-of-theart review of the mutation analysis technique for testing multiagent systems,” in Novel and Intelligent Digital Systems Conferences, Athens, Greece, 2023, pp. 230–235, doi: 10.1007/978-3-031-44146-2_23.
• [24] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
• [25] T. Hall, S. Beecham, D. Bowes, D. Gray, and S. Counsell, “A systematic literature review on fault prediction performance in software engineering,” IEEE Trans. Software Eng., vol. 38, pp. 1276–1304, 2012.
• [26] M. Meiliana, S. Karim, H.L.H.S. Warnars, F.L. Gaol, E. Abdurachman, and B. Soewito, “Software metrics for fault prediction using machine learning approaches: A literature review with PROMISE repository dataset,” in Proc. 2017 IEEE Int. Conf. Cybernetics and Computational Intelligence (CyberneticsCOM 2017), 2018.
• [27] N. Li, M. Shepperd, and Y. Guo, “A systematic review of unsupervised learning techniques for software defect prediction,” Inf. Softw. Technol., vol. 122, p. 106270, 2020, doi: 10.1016/j.infsof.2020.106270.
• [28] R. Pan, M. Bagherzadeh, T. A. Ghaleb, and Others, “Test case selection and prioritization using machine learning: A systematic literature review,” Empirical Softw. Eng., vol. 27, p. 29, 2022, doi: 10.1007/s10664-022-10025-5.
• [29] M. Khatibsyarbini et al., “Trend application of machine learning in test case prioritization: A review on techniques,” IEEE Access, vol. 9, pp. 166262–166282, 2021, doi: 10.1109/ACCESS.2021.3135508.
• [30] I.H. Witten, E. Frank, and M.A. Hall, Data Mining: Practical Machine Learning Tools and Techniques, Elsevier eBooks, 2011. [Online]. Available: https://doi.org/10.1016/c2009-0-19715-5.
• [31] V.H. Durelli, R.S. Durelli, S.S. Borges, A.T. Endo, M.M. Eler, D.R. Dias, and M.P. Guimaraes, “Machine learning applied to software testing: A systematic mapping study,” IEEE Trans. Rel., vol. 68, pp. 1189–1212, 2019.
• [32] N. Jha and R. Popli, “Artificial intelligence for software testing: Perspectives and practices,” in Proc. Fourth Int. Conf. Computational Intelligence and Communication Technologies (CCICT), 2021, pp. 377–382, doi: 10.1109/CCICT53244.2021.00075.
• [33] C. Ioannides and K.I. Eder, “Coverage-directed test generation automated by machine learning – A review,” ACM Trans. Design Autom. Electron. Syst., vol. 17, p. 7, 2012, doi: 10.1145/ 2071356.2071363.
• [34] J.M. Balera and V.A. de Santiago Junior, “A systematic mapping addressing hyper-heuristics within search-based software testing,” Inf. Softw. Technol., vol. 114, pp. 176–189, 2019.
• [35] Z. Zhou, M. Sunkara, Y. Lei, and A. Ramesh, “Machine learning for software testing: A survey,” arXiv:1906.10742, 2019. [Online]. Available: https://arxiv.org/abs/1906.10742.
• [36] M.M. Alam, S. Ali, A. Khan, M. Hamayun, and K.Z. Khan, “Machine learning for improving API testing,” arXiv:2207.13143, 2018. [Online]. Available: https://arxiv.org/abs/2207.13143.
• [37] D. Guassmi, N.E.H. Dehimi, M. Derdour, and A. Kouzou, “Using machine learning techniques for multi-agent systems testing,” in Artificial Intelligence and Its Practical Applications in the Digital Economy (I2COMSAPP 2024), Lecture Notes in Networks and Systems, 2024, vol. 861, pp. 230–235, doi: 10.1007/978-3-031-71426-9_16
• [38] S.U. Rehman and A. Nadeem, “An approach to model-based testing of multiagent systems,” Sci. World J., vol. 2015, p. 925206, 2015, doi: 10.1155/2015/925206.
• [39] N.E.H. Dehimi, F. Mokhati, and M. Badri, “Testing HMASbased applications: An ASPECS-based approach,” Eng. Appl. Artif. Intell., vol. 46, pp. 25–33, 2015, doi: 10.1016/j.engappai.2015.09.013.
• [40] N.A. Bakar and A. Selamat, “Agent systems verification: Systematic literature review and mapping,” Appl. Intell., vol. 48, no. 5, pp. 1251–1274, 2018, doi: 10.1007/s10489-017-1112-z.
• [41] C. Barnier, O.-E.-K. Aktouf, A. Mercier, and J.P. Jamont, “Toward an embedded multi-agent system methodology and positioning on testing,” in Proc. 2017 IEEE Int. Symp. Software Reliability Engineering Workshops (ISSREW), 2017, pp. 239–244.
• [42] M. Winikoff, “BDI agent testability revisited,” Autonomous Agents and Multi-Agent Systems, vol. 31, no. 6, pp. 1094–1132, 2017, doi: 10.1007/s10458-016-9356-2.
• [43] E.M.N. Gonçalves, R. A. Machado, B. C. Rodrigues, and D. Adamatti, “CPN4M: Testing multi-agent systems under organizational model Moise+ using colored Petri nets,” Appl. Sci., vol. 12, no. 12, p. 5857, 2022, doi: 10.3390/app12125857.
• [44] M.S.U. Rehman, A. Nadeem, and M.A. Sindhu, “Towards automated testing of multi-agent systems using Prometheus design models,” Int. Arab J. Inf. Technol., vol. 16, pp. 54–65, 2019. [Online]. Available: https://dblp.uni-trier.de/db/journals/iajit/iajit16.html#RehmanNS19.
• [45] Z. Huang, R. Alexander, and J. Clark, “Mutation testing for Jason agents,” in Proc. Int. Workshop Eng. Multi-Agent Syst. (EMAS 2014), Paris, France, 2014.
• [46] N.E.H. Dehimi, A.H. Benkhalef, and Z. Tolba, “A novel mutation analysis-based approach for testing parallel behavioural scenarios in multi-agent systems,” Electronics, vol. 11, no. 22, p. 3642, 2022, doi: 10.3390/electronics11223642.
• [47] N.E.H. Dehimi, S. Boukelloul, and D. Guassmi, “Towards a new dynamic model-based testing approach for multi-agent systems,” in Proc. 2022 4th Int. Conf. Pattern Analysis and Intelligent Systems (PAIS), IEEE, 2022, pp. 1–6, doi: 10.1007/978-3-031-44146-2_23.
• [48] M. Schuster and K.K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Trans. Signal Process., vol. 45, no. 11, pp. 2673–2681, 1997, doi: 10.1109/78.650093.
• [49] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, 2016. [Online]. Available: http://www.deeplearningbook.org/.
• [50] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, 1998
• [51] G. Hinton et al., “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82–97, 2012.
• [52] A. Krizhevsky, I. Sutskever, and G.E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017.
• [53] H.T. Nguyen, M.W. Berry, and J.D. Kiffe, “Numerical Methods for Partial Differential Equations, 2nd ed., ser. Texts,” in Computational Science and Engineering. Cham: Springer, 2019, doi: 10.1007/978-3-030-38800-3.
• [54] N. El Houda Dehimi and Z. Tolba, “Attention Mechanisms in Deep Learning: Towards Explainable Artificial Intelligence,” in 6th International Conference on Pattern Analysis and Intelligent Systems (PAIS), El Qued, Algeria, 2024, pp. 1-7, doi: 10.1109/PAIS62114.2024.10541203.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr POPUL/SP/0154/2024/02 w ramach programu "Społeczna odpowiedzialność nauki II" - moduł: Popularyzacja nauki (2025).