Model-based automated testing of JavaScript Web applications via longer test sequences

Abstract

JavaScript has become one of the most widely used languages for Web development. Its dynamic and event-driven features make it challenging to ensure the correctness of Web applications written in JavaScript. A variety of dynamic analysis techniques have been proposed which are, however, limited in either coverage or scalability. In this paper, we propose a simple, yet effective, model-based automated testing approach to achieve a high code-coverage within the time budget via testing with longer event sequences. We implement our approach as an open-source tool LJS, and perform extensive experiments on 21 publicly available benchmarks. On average, LJS is able to achieve 86.5% line coverage in 10 minutes. Compared with JSDEP, a state-of-the-art breadth-first search based automated testing tool enriched with partial order reduction, the coverage of LJS is 11%–19% higher than that of JSDEP on real-world large Web applications. Our empirical findings support that proper longer test sequences can achieve a higher code coverage in JavaScript Web application testing.

Graphical abstract

Keywords

model-based testing / automated testing / JavaScript Web applications

References

[1] Saxena P, Akhawe D, Hanna S, Mao F, McCamant S, Song D. A symbolic execution framework for JavaScript. In: Proceedings of IEEE Symposium on Security and Privacy. 2010, 513– 528

[2] Artzi S, Dolby J, Jensen S H, Møller A, Tip F. A framework for automated testing of JavaScript web applications. In: Proceedings of International Conference on Software Engineering. 2011, 571–580

[3] Mesbah A , Van Deursen A , Lenselink S . Crawling ajaxbased web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web, 2012, 6( 1): 1– 30

[4] Sen K, Kalasapur S, Brutch T G, Gibbs S. Jalangi: a selective record-replay and dynamic analysis framework for JavaScript. In: Proceedings of Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 2013, 488–498

[5] Mirshokraie S, Mesbah A, Pattabiraman K. Efficient JavaScript mutation testing. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2013, 74–83

[6] Li G, Andreasen E, Ghosh I. SymJS: automatic symbolic testing of JavaScript Web applications. In: Proceedings of ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2014, 449–459

[7] Pradel M, Schuh P, Necula G, Sen K. EventBreak: analyzing the responsiveness of user interfaces through performance-guided test generation. In: Proceedings of ACM International Conference on Object Oriented Programming Systems Languages & Applications. 2014, 33–47

[8] Mirshokraie S , Mesbah A , Pattabiraman K . Guided mutation testing for JavaScript Web Applications. IEEE Transactions on Software Engineering, 2015, 41( 5): 429– 444

[9] Sen K, Necula G C, Gong L, Choi W. MultiSE: multipath symbolic execution using value summaries. In: Proceedings of Joint Meeting on Foundations of Software Engineering. 2015, 842–853

[10] Andreasen E , Gong L , Møller A , Pradel M , Selakovic M , Sen K , Staicu C . A survey of dynamic analysis and test generation for JavaScript. ACM Computing Surveys, 2017, 50( 5): 66:1– 66:36

[11] Sung C, Kusano M, Sinha N, Wang C. Static DOM event dependency analysis for testing Web applications. In: Proceedings of ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2016, 447–459

[12] Arcuri A. Longer is better: On the role of test sequence length in software testing. In: Proofceedings of International Conference on Software Testing, Verification and Validation. 2010, 469–478

[13] Andrews J H, Groce A, Weston M, Xu R G. Random test run length and effectiveness. In: Proofceedings of IEEE/ACM International Conference on Automated Software Engineering. 2008, 19–28

[14] Fraser G, Gargantini A. Experiments on the test case length in specification based test case generation. In: Proceedings of International Workshop on Automation of Software Test. 2009, 18–26

[15] Carino S, Andrews J H. Evaluating the effect of test case length on GUI test suite performance. In: Proceedings of IEEE/ACM International Workshop on Automation of Software Test. 2015, 13–17

[16] Li Y F , Das P K , Dowe D L . Two decades of Web application testing - A survey of recent advances. Information Systems, 2014, 43 : 20– 54

[17] Cheng L, Yang Z, Wang C. Systematic reduction of GUI test sequences. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering. 2017, 849–860

[18] Godefroid P, van Leeuwen J, Hartmanis J, Goos G, Wolper P. Partial-Order Methods for the Verification of Concurrent Systems - An Approach to the StateExplosion Problem. Heidelberg: Springer, 1996

[19] Arcuri A , Briand L C . A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability, 2014, 24( 3): 219– 250

[20] Dias-Neto A C , Travassos G H . A picture from the model-based testing area: Concepts, techniques, and challenges. Advances in Computers, 2010, 80 : 45– 120

[21] Utting M , Pretschner A , Legeard B . A taxonomy of model-based testing approaches. Software Testing, Verification & Reliability, 2012, 22( 5): 297– 312

[22] Li W, Le Gall F, Spaseski N. A survey on model-based testing tools for test case generation. In: Proceedings of International Conference on Tools and Methods of Program Analysis. 2018, 77–89

[23] Jensen C S, Prasad M R, Møller A. Automated testing with targeted event sequence generation. In: Proceedings of International Symposium on Software Testing and Analysis. 2013, 67–77

[24] Takala T, Katara M, Harty J. Experiences of systemlevel model-based GUI testing of an android application. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2011, 377–386

[25] Rau A, Hotzkow J, Zeller A. Efficient GUI test generation by learning from tests of other apps. In: Proceedings of International Conference on Software Engineering: Companion Proceedings. 2018, 370–371

[26] Andrews A A , Offutt J , Alexander R T . Testing Web applications by modeling with fsms. Software and System Modeling, 2005, 4( 3): 326– 345

[27] Ricca F, Tonella P. Analysis and testing of Web applications. In: Proceedings of International Conference on Software Engineering. 2001, 25–34

[28] Dallmeier V, Burger M, Orth T, Zeller A. Webmate: a tool for testing Web 2.0 applications. In: Proceedings of Workshop on JavaScript Tools. 2012, 11–15

[29] Alshahwan N, Harman M. Automated Web application testing using search based software engineering. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering. 2011, 3–12

[30] Nguyen C, Yoshida H, Prasad M R, Ghosh I, Sen K. Generating succinct test cases using don’t care analysis. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2015, 1–10

[31] Rau A, Hotzkow J, Zeller A. Transferring tests across Web applications. In: Proceedings of International Conference on Web Engineering. 2018, 50–64

Acknowledgements

The authors would like to thank Dr. Ting Su for his valuable comments and suggestions. P. Gao, Y. Xu and F. Song were partially supported by the National Natural Science Foundation of China (NSFC) (Grant Nos. 62072309, 61532019, 61761136011). T. Chen is partially supported by the National Natural Science Foundation of China (Grant No. 61872340), Guangdong Science and Technology Department (2018B010107004) and Natural Science Foundation of Guangdong Province (2019A1515011689).

RIGHTS & PERMISSIONS

2022 Higher Education Press