ABC-CAG: Covering Array Generator for Pair-wise Testing Using Artificial Bee Colony Algorithm

• Autorzy: Bansal P. , Sabharwal S. , Mittal N. , Arora S.

Identyfikatory

DOI

10.5277/e-Inf160101

• Języki publikacji: EN

Abstrakty

EN

Testing is an indispensable part of the software development life cycle. It is performed to improve the performance, quality and reliability of the software. Various types of testing such as functional testing and structural testing are performed on software to uncover the faults caused by an incorrect code, interaction of input parameters, etc. One of the major factors in deciding the quality of testing is the design of relevant test cases which is crucial for the success of testing. In this paper we concentrate on generating test cases to uncover faults caused by the interaction of input parameters. It is advisable to perform thorough testing but the number of test cases grows exponentially with the increase in the number of input parameters, which makes exhaustive testing of interaction of input parameters imprudent. An alternative to exhaustive testing is combinatorial interaction testing (CIT) which requires that every t-way interaction of input parameters be covered by at least one test case. Here, we present a novel strategy ABC-CAG (Artificial Bee Colony-Covering Array Generator) based on the Artificial Bee Colony (ABC) algorithm to generate covering an array and a mixed covering array for pair-wise testing. The proposed ABC-CAG strategy is implemented in a tool and experiments are conducted on various benchmark problems to evaluate the efficacy of the proposed approach. Experimental results show that ABC-CAG generates better/comparable results as compared to the existing state-of-the-art algorithms.

Słowa kluczowe

EN

combinatorial interaction testing; pair-wise testing; covering array; artificial bee colony

• Wydawca: Oficyna Wydawnicza Politechniki Wrocławskiej
• Czasopismo: e-Informatica Software Engineering Journal
• Rocznik: 2016
• Tom: Vol. 10, nr 1
• Strony: 9--29
• Opis fizyczny: Bibliogr. 56 poz., tab., rys.

Twórcy

autor

Bansal P.

• Netaji Subhas Institute of Technology, University of Delhi

autor

Sabharwal S.

• Netaji Subhas Institute of Technology, University of Delhi

autor

Mittal N.

• Netaji Subhas Institute of Technology, University of Delhi

autor

Arora S.

• School of Computer Science and Engineering, Vellore Institute of Technology, Tamil Nadu

Bibliografia

• [1] J.M. Glenford, The art of software testing. John Willey & Sons, 2011.
• [2] D.M. Cohen, S.R. Dalal, A. Kajla, and G.C. Patton, “The automatic efficient test generator (AETG) system,” in Proceedings of the 5th International Symposium on Software Reliability Engineering. IEEE, 1994, pp. 303–309.
• [3] D.M. Cohen, S.R. Dalal, M.L. Fredman, and G.C. Patton, “The AETG system: An approach to testing based on combinatorial design,” IEEE Transactions on Software Engineering, Vol. 23, No. 7, 1997, pp. 437–444.
• [4] K. Burr and W. Young, “Combinatorial test techniques: Table-based automation, test generation and code coverage,” in Proceedings of the International Conference on Software Testing Analysis & Review, San Diego, 1998.
• [5] S.R. Dalal, A. Jain, N. Karunanithi, J. Leaton, C.M. Lott, G.C. Patton, and B.M. Horowitz, “Model-based testing in practice,” in Proceedings of the 21st international conference on Software engineering. ACM, 1999, pp. 285–294.
• [6] D.R. Kuhn, D.R. Wallace, and A.M. Gallo Jr, “Software fault interactions and implications for software testing,” IEEE Transactions on Software Engineering, Vol. 30, No. 6, 2004, pp. 418–421.
• [7] Y. Lei and K.C. Tai, “In-parameter-order: A test generation strategy for pairwise testing,” in Proceedings of the Third IEEE International High-Assurance Systems Engineering Symposium. IEEE, 1998, pp. 254–261.
• [8] Y.W. Tung and W.S. Aldiwan, “Automating test case generation for the new generation mission software system,” in Proceedings of the IEEE Aerospace Conference, Vol. 1. IEEE, 2000, pp. 431–437.
• [9] A. Hartman, T. Klinger, and L. Raskin, “ IBM intelligent test case handler,” Discrete Mathematics, Vol. 284, 2010, pp. 149–156.
• [10] J. Arshem, Test vector generator (TVG), (2010). [Online]. https://sourceforge.net/pro jects/tvg/
• [11] AllPairs, (2009). [Online]. http://sourceforge. net/projects/allpairs/
• [12] J. Czerwonka, “Pairwise testing in the real world: Practical extensions to test-case scenarios,” in Proceedings of the 24th Pacific Northwest Software Quality Conference, 2006, pp. 419–430.
• [13] B. Jenkins, jenny: a pairwise testing tool, (2005). [Online]. http://burtleburtle.net/bob/ math/jenny.html
• [14] Z. Wang, B. Xu, and C. Nie, “Greedy heuristic algorithms to generate variable strength combinatorial test suite,” in The Eighth International Conference on Quality Software. QSIC’08. IEEE, 2008, pp. 155–160.
• [15] Z. Wang and H. He, “Generating variable strength covering array for combinatorial soft- ware testing with greedy strategy,” Journal of Software, Vol. 8, No. 12, 2013, pp. 3173–3181.
• [16] S.A. Abdullah, Z.H. Soh, and K.Z. Zamli, “Variable-strength interaction for t-way test generation strategy,” International Journal of Advances in Soft Computing & Its Applications, Vol. 5, No. 3, 2013.
• [17] M.F. Klaib, K.Z. Zamli, N.A.M. Isa, M.I. Younis, and R. Abdullah, “ G2Way a backtracking strategy for pairwise test data generation,” in 15th Asia-Pacific Software Engineering Conference, APSEC’08. IEEE, 2008, pp. 463–470.
• [18] K.Z. Zamli, M.F. Klaib, M.I. Younis, N.A.M. Isa, and R. Abdullah, “Design and implementation of a t-way test data generation strategy with automated execution tool support,” Information Sciences, Vol. 181, No. 9, 2011, pp. 1741–1758.
• [19] K.F. Rabbi, A.H. Beg, and T. Herawan, “ MT2Way: A novel strategy for pair-wise test data generation,” in Computational Intelligence and Intelligent Systems. Springer, 2012, pp. 180–191.
• [20] K. Rabbi, S. Khatun, C.Y. Yaakub, and M. Klaib, “ EPS2Way: an efficient pairwise test data generation strategy,” International Journal of New Computer Architectures and their Applications (IJNCAA), Vol. 1, No. 4, 2011, pp. 1099–1109.
• [21] Z. Zhang, J. Yan, Y. Zhao, and J. Zhang, “Generating combinatorial test suite using combinatorial optimization,” Journal of Systems and Software, Vol. 98, 2014, pp. 191–207.
• [22] R. Kuhn, Advanced combinatorial testing system (ACTS), National Institute of Standards and Technology, (2011). [Online]. http://csrc.nist.gov/groups/SNS/acts/ documents/comparison-report.html#acts
• [23] T. Shiba, T. Tsuchiya, and T. Kikuno, “Using artificial life techniques to generate test cases for combinatorial testing,” in Proceedings of the 28th Annual International Computer Software and Applications Conference. IEEE, 2004, pp. 72–77.
• [24] X. Chen, Q. Gu, J. Qi, and D. Chen, “Applying particle swarm optimization to pairwise testing,” in IEEEProceedings of the 34th Annual Computer Software and Applications Conference. IEEE, 2010, pp. 107–116.
• [25] L. Gonzalez-Hernandez, N. Rangel-Valdez, and J. Torres-Jimenez, “Construction of mixed covering arrays of variable strength using a tabu search approach,” in Combinatorial Optimization and Applications. Springer, 2010, pp. 51–64.
• [26] L. Gonzalez-Hernandez, N. Rangel-Valdez, and J. Torres-Jimenez, “Construction of mixed covering arrays of strengths 2 through 6 using a tabu search approach,” Discrete Mathematics, Algorithms and Applications, Vol. 4, No. 03, 2012, p. 1250033.
• [27] H. Avila-George, J. Torres-Jimenez, V. Hernández, and L. Gonzalez-Hernandez, “Simulated annealing for constructing mixed covering arrays,” in Distributed Computing and Artificial Intelligence. Springer, 2012, pp. 657–664.
• [28] B.S. Ahmed, K.Z. Zamli, and C. Lim, “The development of a particle swarm based optimization strategy for pairwise testing,” Journal of Artificial Intelligence, Vol. 4, No. 2, 2011, pp. 156–165.
• [29] B.J. Garvin, M.B. Cohen, and M.B. Dwyer, “Evaluating improvements to a meta-heuristic search for constrained interaction testing,” Empirical Software Engineering, Vol. 16, No. 1, 2011, pp. 61–102.
• [30] J.D. McCaffrey, “Generation of pairwise test sets using a genetic algorithm,” in Proceedings of the 33rd Annual IEEE International Computer Software and Applications Conference, Vol. 1. IEEE, 2009, pp. 626–631.
• [31] P. Flores and Y. Cheon, “ PWiseGen: Generating test cases for pairwise testing using genetic algorithms,” in Proceedings of the International Conference on Computer Science and Automation Engineering, Vol. 2. IEEE, 2011, pp. 747–752.
• [32] P. Bansal, S. Sabharwal, S. Malik, V. Arora, and V. Kumar, “An approach to test set generation for pair-wise testing using genetic algorithms,” in Search Based Software Engineering. Springer, 2013, pp. 294–299.
• [33] B.S. Ahmed, T.S. Abdulsamad, and M.Y. Potrus, “Achievement of minimized combinatorial test suite for configuration-aware software functional testing using the cuckoo search algorithm,” Information and Software Technology, Vol. 66, 2015, pp. 13–29.
• [34] T. Mahmoud and B.S. Ahmed, “An efficient strategy for covering array construction with fuzzy logic-based adaptive swarm optimization for software testing use,” Expert Systems with Applications, Vol. 42, No. 22, 2015, pp. 8753–8765.
• [35] D. Karaboga, “An idea based on honey bee swarm for numerical optimization,” Erciyes University, Engineering Faculty, Computer Engineering Department, Tech. Rep. TR-06, 2005.
• [36] D. Karaboga and B. Basturk, “Artificial bee colony (ABC) optimization algorithm for solving constrained optimization problems,” in Foundations of Fuzzy Logic and Soft Computing. Springer, 2007, pp. 789–798.
• [37] A.S. Hedayat, N.J.A. Sloane, and J. Stufken, Orthogonal arrays. Springer Science & Business Media, 2012.
• [38] M.B. Cohen, P.B. Gibbons, W.B. Mugridge, and C.J. Colbourn, “Constructing test suites for interaction testing,” in Proceedings of the 25th International Conference on Software Engineering. IEEE, 2003, pp. 38–48.
• [39] G. Sherwood, Testcover.com, (2006). [Online]. http://testcover.com/
• [40] A.W. Williams, “Determination of test configurations for pair-wise interaction coverage,” in Testing of Communicating Systems. Springer, 2000, pp. 59–74.
• [41] A. Hartman, “Software and hardware testing using combinatorial covering suites,” in Graph theory, combinatorics and algorithms. Springer, 2005, pp. 237–266.
• [42] N. Kobayashi, T. Tsuchiya, and T. Kikuno, “A new method for constructing pair-wise covering designs for software testing,” Information Processing Letters, Vol. 81, No. 2, 2002, pp. 85–91.
• [43] D. Karaboga and B. Basturk, “A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm,” Journal of global optimization, Vol. 39, No. 3, 2007, pp. 459–471.
• [44] E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm intelligence: from natural to artificial systems. Oxford University Press, 1999.
• [45] O.B. Haddad, A. Afshar, and M.A. Mariño, “Honey-bees mating optimization (HBMO) algorithm: a new heuristic approach for water resources optimization,” Water Resources Man- agement, Vol. 20, No. 5, 2006, pp. 661–680.
• [46] D. Teodorović and M. Dell’Orco, “Bee colony optimization – a cooperative learning approach to complex transportation problems,” in Advanced OR and AI Methods in Transportation: Proceedings of 16th Mini–EURO Conference and 10th Meeting of EWGT. Poznań: Publishing House of the Polish Operational and System Research, 2005, pp. 51–60.
• [47] H. Drias, S. Sadeg, and S. Yahi, “Cooperative bees swarm for solving the maximum weighted satisfiability problem,” in Computational Intelligence and Bioinspired Systems. Springer, 2005, pp. 318–325.
• [48] G. Li, P. Niu, and X. Xiao, “Development and investigation of efficient artificial bee colony algorithm for numerical function optimization,” Applied soft computing, Vol. 12, No. 1, 2012, pp. 320–332.
• [49] D. Jeya Mala, V. Mohan, and M. Kamalapriya, “Automated software test optimisation framework – an artificial bee colony optimisation-based approach,” IET Software, Vol. 4, No. 5, 2010, pp. 334–348.
• [50] G. Zhu and S. Kwong, “Gbest-guided artificial bee colony algorithm for numerical function optimization,” Applied Mathematics and Computation, Vol. 217, No. 7, 2010, pp. 3166–3173.
• [51] J. Stardom, “Metaheuristics and the search for covering and packing arrays,” Ph.D. dissertation, Simon Fraser University, 2001.
• [52] B. Kazimipour, X. Li, and A. Qin, “A review of population initialization techniques for evolutionary algorithms,” in IEEE Congress on Evolutionary Computation (CEC). IEEE, 2014, pp. 2585–2592.
• [53] D.R. Kuhn and V. Okun, “Pseudo-exhaustive testing for software,” in Proceedings of the 30th Annual IEEE/NASA Software Engineering Workshop. IEEE, 2006, pp. 153–158.
• [54] Pairwise testing, (2016). [Online]. http://www.pairwise.org/
• [55] P. Flores, PWiseGen, (2010). [Online]. https://code.google.com/p/pwisegen/
• [56] A. Arcuri and L. Briand, “A practical guide for using statistical tests to assess randomized algorithms in software engineering,” in Proceedings of the 33rd International Conference on Software Engineering. IEEE, 2011, pp. 1–10.