A novel strategy for automatic test data generation using soft computing technique

Abstract

Software testing is one of the most crucial and analytical aspect to assure that developed software meets prescribed quality standards. Software development process invests at least 50% of the total cost in software testing process. Optimum and efficacious test data design of software is an important and challenging activity due to the nonlinear structure of software. Moreover, test case type and scope determines the quality of test data. To address this issue, software testing tools should employ intelligence based soft computing techniques like particle swarm optimization (PSO) and genetic algorithm (GA) to generate smart and efficient test data automatically. This paper presents a hybrid PSO and GA based heuristic for automatic generation of test suites. In this paper, we described the design and implementation of the proposed strategy and evaluated our model by performing experiments with ten container classes from the Java standard library. We analyzed our algorithm statistically with test adequacy criterion as branch coverage. The performance adequacy criterion is taken as percentage coverage per unit time and percentage of faults detected by the generated test data. We have compared our work with the heuristic based upon GA, PSO, existing hybrid strategies based on GA and PSO and memetic algorithm. The results showed that the test case generation is efficient in our work.

Keywords

software testing / particle swarm optimization / genetic algorithm / soft computing / test data generation

References

[1] Ramler R, Wolfmaier K. Economic perspectives in test automation: balancing automated and manual testing with opportunity cost. In: Proceedings of the International Workshop on Automation of Software Test. 2006, 85-91

[2] Grottke M, Trivedi K S. 2007. Fighting bugs: remove, retry, replicate, and rejuvenate. IEEE Computer, 2007, 40(2): 107-109

[3] Pargas R P, Harrold M J, Peck R R. Test-data generation using genetic algorithms. Software Testing Verification Reliability, 1999, 9(4): 263-282

[4] Chen X, Gu Q, Qi J X, Chen D X. Applying particle swarm optimization to pairwise testing. In: Proceedings of 34th Annual IEEE Computer Software and Applications Conference, COMPSAC’10. 2010, 107-116

[5] Windisch A, Wappler S, Wegener J. Applying particle swarm optimization to software testing. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO ’07. 2007, 1121-1128

[6] Wegener J, Baresel A, Sthamer H. Evolutionary test environment for automatic structural testing. Information and Software Technology, 2001, 43(14): 841-854

[7] Fraser G, Arcuri A. Evolutionary generation of whole test suites. In: Proceedings of the Quality Software International Conference, QSIC’11. 2011, 31-40

[8] Wappler S, Wegener J. Evolutionary unit testing of object-oriented software using a hybrid evolutionary algorithm. In: Proceedings of IEEE Congress on Evolutionary Computation, CEC’06. 2006, 851-858

[9] Alba E, Chicano F. Observations in using parallel and sequential evolutionary algorithms for automatic software testing. Computers and Operation Research, 2008, 35(10): 3161-3183

[10] Li K, Zhang Z, Kou J. Breeding software test data with genetic-particle swarm mixed algorithms. Journal of Computers, 2010, 5(2): 258-265

[11] Singla S, Kumar D, Rai H M, Singla P. A hybrid PSO approach to automate test data generation for data flow coverage with dominance concepts. International Journal of Advanced Science and Technology, 2011, 37: 15-26

[12] Wu X, Wang Y, Zhang T. An improved GAPSO hybrid programming algorithm. In: Proceedings of International Conference on Information Engineering and Computer Science, ICIECS’09. 2009, 1-4

[13] Zhang S, Ying Z, Hong Z, Qingquan H. Automatic path test data generation based on GA-PSO. In: Proceedings of IEEE International Conference onIntelligent Computing and Intelligent Systems, ICIS’10. 2010, 142-146

[14] Chen S. Particle swarm optimization with pbest crossover. In: Proceedings of IEEE Congress on Evolutionary Computation, CEC’12. 2012, 1-6

[15] Kaur A, Bhatt D. Hybrid particle swarm optimization for regression testing. International Journal on Computer Science and Engineering, 2011, 3 (5): 1815-1824

[16] Goldberg D E, Holland J H. Genetic algorithms and machine learning. Machine Learning, 1988, 3(2): 95-99

[17] Eberhart R C, Kennedy J. A new optimizer using particle swarm theory. In: Proceedings of the 6th International Symposium on Micromachine Human Science, 1995, 39-43

[18] Kennedy J, Eberhart R C. Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks. 1995, 4, 1942-1948

[19] Rabanal P, Rodriguez I, Rubio F. A functional approach to parallelize particle swarm optimization. In: Proceedings of Metaheuristicas, Algoritmos Evolutivos y Bioinspirados, MAEB’12. 2012

[20] Jones B F, Sthamer H, Eyres D E. Automatic test data generation using genetic algorithms. Software Engineering Journal, 1996, 11(5): 299-306

[21] Xanthakis S E, Skourlas C C, LeGall A K. Application of genetic algorithms to software testing. In: Proceedings of the 5th International Conference on Software Engineering and its Applications. 1992, 625-636

[22] Harman M, Jones B. Search-based software engineering. Information and Software Technology, 2001, 43(14): 833-839

[23] Clark J, Dolado J J, Harman M, Hierons R, Jones B, Lumkin M, Mitchell B, Mancoridis S, Rees K, Roper M, Shepperd M. Reformulating software engineering as a search problem. IEE Proceedings- Software, 2003, 150(3): 161-175

[24] Hansen N, Finck S, Ros R, Auger A. Real-parameter black-box optimization benchmarking 2009: noiseless functions definitions. INRIA Technical Report RR-6829, 2009

[25] Younes M, Benhamida F. Genetic algorithm-particle swarm optimization (GA-PSO) for economic load dispatch. PRZEGLAD ELEKTROTECHNICZNY (Electrical Review), 2011, 87(10): 369-372

[26] Juang C F. A hybrid of genetic algorithm and particle swarm optimization for recurrent network design. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2004, 34(2): 997-1006

[27] Saini D K, Sharma Y. Soft computing particle swarm optimization based approach for class responsibility assignment problem. International Journal of Computer Applications, 2012, 40(12): 19-24

[28] Wappler S, Schieferdecker I. Improving evolutionary class testing in the presence of non-public methods. In: Proceedings of the 22nd IEEE/ACM International Conference on Automated Software Engineering, ASE’07. 2007, 381-384

[29] Wilson G C, McIntyre A, Heywood M I. Resource review: three open source systems for evolving programs: Lilgp, ecj and grammatical evolution. Genetic Programming and Evolvable Machines, 2004, 5(1): 103-105

[30] Tonella P. Evolutionary testing of classes. In: Proceedings of the International Symposium on Software Testing and Analysis, ISSTA’04. 2004, 119-128

[31] Wang H C, Jeng B. Structural testing using memetic algorithm. In: Proceedings of the 2nd Taiwan Conference on Software Engineering. 2006

[32] Arcuri A, Yao X. Search based software testing of object-oriented containers. Information Sciences, 2008, 178(15): 3075-3095

[33] Nayak N, Mohapatra D P. Automatic test data generation for data flow testing using particle swarm optimization. Communications in Computer and Information Science, 2010, 95(1): 1-12

[34] Ahmed B S, Zamli K Z, Lim C P. Constructing a T-way interaction test suite using particle swarm optimization approach. International Journal of Innovative Computing Information Control, 2011, 7(11): 1741-1758

[35] Li A, Zhang Y. Automatic generating all-path test data of a program based on PSO. In: Proceedings of World Congress on Software Engineering, 2009, 4: 189-193

[36] Fraser G, Arcuri A. Whole test suite generation. IEEE Transactions on Software Engineering, 2013, 39(2): 276-291

[37] Gong D W, Zhang Y. Generating test data for both path coverage and fault detection using genetic algorithms. Frontiers of Computer Science, 2013, 7(6): 822-837

[38] McMinn P. Search-based software test data generation: a survey. Software Testing, Verification and Reliability, 2004, 14(2): 105-156

[39] McMinn P, Holcombe M. Evolutionary testing of statebased programs. In: Proceedings Conference on Genetic and Evolutionary Computation, GECCO’05. 2005, 1013-1020

[40] Rothermel G, Untch R, Chengyun C, Harrold M J. Prioritizing test cases for regression testing. IEEE Transaction of Software Engineering, 2001, 27(10): 929-948

[41] Arcuri A, Briand L. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: Proceedings of 33rd International Conference on Software Engineering ICSE’11. 2011, 1-10

[42] Ali S, Briand L C, Hemmati H, Panesar-Walawege R K. A systematic review of the application and empirical investigation of searchbased test case generation. IEEE Transactions of Software Engineering, 2010, 36(6): 742-762

[43] Vargha A, Delaney H D. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 2000, 25(2): 101-132

[44] Grissom R, Kim J. Effect sizes for research: a broad practical approach. Lawrence Erlbaum, 2005

RIGHTS & PERMISSIONS

Higher Education Press and Springer-Verlag Berlin Heidelberg