Model-based automated testing of JavaScript Web applications via longer test sequences

Abstract

JavaScript has become one of the most widely used languages for Web development. Its dynamic and event-driven features make it challenging to ensure the correctness of Web applications written in JavaScript. A variety of dynamic analysis techniques have been proposed which are, however, limited in either coverage or scalability. In this paper, we propose a simple, yet effective, model-based automated testing approach to achieve a high code-coverage within the time budget via testing with longer event sequences. We implement our approach as an open-source tool LJS, and perform extensive experiments on 21 publicly available benchmarks. On average, LJS is able to achieve 86.5% line coverage in 10 minutes. Compared with JSDEP, a state-of-the-art breadth-first search based automated testing tool enriched with partial order reduction, the coverage of LJS is 11%–19% higher than that of JSDEP on real-world large Web applications. Our empirical findings support that proper longer test sequences can achieve a higher code coverage in JavaScript Web application testing.

Graphical abstract

Keywords

model-based testing / automated testing / JavaScript Web applications

References

[1] Saxena P, Akhawe D, Hanna S, Mao F, McCamant S, Song D. A symbolic execution framework for JavaScript. In: Proceedings of IEEE Symposium on Security and Privacy. 2010, 513– 528

[2] Artzi S, Dolby J, Jensen S H, Møller A, Tip F. A framework for automated testing of JavaScript web applications. In: Proceedings of International Conference on Software Engineering. 2011, 571–580

[3] Mesbah A , Van Deursen A , Lenselink S . Crawling ajaxbased web applications through dynamic analysis of user interface state changes. ACM Transactions on the Web, 2012, 6( 1): 1– 30

[4] Sen K, Kalasapur S, Brutch T G, Gibbs S. Jalangi: a selective record-replay and dynamic analysis framework for JavaScript. In: Proceedings of Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 2013, 488–498

[5] Mirshokraie S, Mesbah A, Pattabiraman K. Efficient JavaScript mutation testing. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2013, 74–83

[6] Li G, Andreasen E, Ghosh I. SymJS: automatic symbolic testing of JavaScript Web applications. In: Proceedings of ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2014, 449–459

[7] Pradel M, Schuh P, Necula G, Sen K. EventBreak: analyzing the responsiveness of user interfaces through performance-guided test generation. In: Proceedings of ACM International Conference on Object Oriented Programming Systems Languages & Applications. 2014, 33–47

[8] Mirshokraie S , Mesbah A , Pattabiraman K . Guided mutation testing for JavaScript Web Applications. IEEE Transactions on Software Engineering, 2015, 41( 5): 429– 444

[9] Sen K, Necula G C, Gong L, Choi W. MultiSE: multipath symbolic execution using value summaries. In: Proceedings of Joint Meeting on Foundations of Software Engineering. 2015, 842–853

[10] Andreasen E , Gong L , Møller A , Pradel M , Selakovic M , Sen K , Staicu C . A survey of dynamic analysis and test generation for JavaScript. ACM Computing Surveys, 2017, 50( 5): 66:1– 66:36

[11] Sung C, Kusano M, Sinha N, Wang C. Static DOM event dependency analysis for testing Web applications. In: Proceedings of ACM SIGSOFT International Symposium on Foundations of Software Engineering. 2016, 447–459

[12] Arcuri A. Longer is better: On the role of test sequence length in software testing. In: Proofceedings of International Conference on Software Testing, Verification and Validation. 2010, 469–478

[13] Andrews J H, Groce A, Weston M, Xu R G. Random test run length and effectiveness. In: Proofceedings of IEEE/ACM International Conference on Automated Software Engineering. 2008, 19–28

[14] Fraser G, Gargantini A. Experiments on the test case length in specification based test case generation. In: Proceedings of International Workshop on Automation of Software Test. 2009, 18–26

[15] Carino S, Andrews J H. Evaluating the effect of test case length on GUI test suite performance. In: Proceedings of IEEE/ACM International Workshop on Automation of Software Test. 2015, 13–17

[16] Li Y F , Das P K , Dowe D L . Two decades of Web application testing - A survey of recent advances. Information Systems, 2014, 43 : 20– 54

[17] Cheng L, Yang Z, Wang C. Systematic reduction of GUI test sequences. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering. 2017, 849–860

[18] Godefroid P, van Leeuwen J, Hartmanis J, Goos G, Wolper P. Partial-Order Methods for the Verification of Concurrent Systems - An Approach to the StateExplosion Problem. Heidelberg: Springer, 1996

[19] Arcuri A , Briand L C . A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability, 2014, 24( 3): 219– 250

[20] Dias-Neto A C , Travassos G H . A picture from the model-based testing area: Concepts, techniques, and challenges. Advances in Computers, 2010, 80 : 45– 120

[21] Utting M , Pretschner A , Legeard B . A taxonomy of model-based testing approaches. Software Testing, Verification & Reliability, 2012, 22( 5): 297– 312

[22] Li W, Le Gall F, Spaseski N. A survey on model-based testing tools for test case generation. In: Proceedings of International Conference on Tools and Methods of Program Analysis. 2018, 77–89

[23] Jensen C S, Prasad M R, Møller A. Automated testing with targeted event sequence generation. In: Proceedings of International Symposium on Software Testing and Analysis. 2013, 67–77

[24] Takala T, Katara M, Harty J. Experiences of systemlevel model-based GUI testing of an android application. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2011, 377–386

[25] Rau A, Hotzkow J, Zeller A. Efficient GUI test generation by learning from tests of other apps. In: Proceedings of International Conference on Software Engineering: Companion Proceedings. 2018, 370–371

[26] Andrews A A , Offutt J , Alexander R T . Testing Web applications by modeling with fsms. Software and System Modeling, 2005, 4( 3): 326– 345

[27] Ricca F, Tonella P. Analysis and testing of Web applications. In: Proceedings of International Conference on Software Engineering. 2001, 25–34

[28] Dallmeier V, Burger M, Orth T, Zeller A. Webmate: a tool for testing Web 2.0 applications. In: Proceedings of Workshop on JavaScript Tools. 2012, 11–15

[29] Alshahwan N, Harman M. Automated Web application testing using search based software engineering. In: Proceedings of IEEE/ACM International Conference on Automated Software Engineering. 2011, 3–12

[30] Nguyen C, Yoshida H, Prasad M R, Ghosh I, Sen K. Generating succinct test cases using don’t care analysis. In: Proceedings of IEEE International Conference on Software Testing, Verification and Validation. 2015, 1–10

[31] Rau A, Hotzkow J, Zeller A. Transferring tests across Web applications. In: Proceedings of International Conference on Web Engineering. 2018, 50–64

RIGHTS & PERMISSIONS

Higher Education Press