a systematic literature review on test case prioritization techniques

A Systematic Literature Review of Test Case Prioritization Using Genetic Algorithms

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Help | Advanced Search

Computer Science > Software Engineering

Title: test case selection and prioritization using machine learning: a systematic literature review.

Abstract: Regression testing is an essential activity to assure that software code changes do not adversely affect existing functionalities. With the wide adoption of Continuous Integration (CI) in software projects, which increases the frequency of running software builds, running all tests can be time-consuming and resource-intensive. To alleviate that problem, Test case Selection and Prioritization (TSP) techniques have been proposed to improve regression testing by selecting and prioritizing test cases in order to provide early feedback to developers. In recent years, researchers have relied on Machine Learning (ML) techniques to achieve effective TSP (ML-based TSP). Such techniques help combine information about test cases, from partial and imperfect sources, into accurate prediction models. This work conducts a systematic literature review focused on ML-based TSP techniques, aiming to perform an in-depth analysis of the state of the art, thus gaining insights regarding future avenues of research. To that end, we analyze 29 primary studies published from 2006 to 2020, which have been identified through a systematic and documented process. This paper addresses five research questions addressing variations in ML-based TSP techniques and feature sets for training and testing ML models, alternative metrics used for evaluating the techniques, the performance of techniques, and the reproducibility of the published studies. We summarize the results related to our research questions in a high-level summary that can be used as a taxonomy for classifying future TSP studies.

Submission history

Access paper:.

• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

DBLP - CS Bibliography

Bibtex formatted citation.

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

A Systematic Review of Test Case Prioritization Approaches

• Conference paper
• First Online: 24 November 2021
• Cite this conference paper

• Umakanta Dash 17 &
• Arup Abhinna Acharya 17  

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1406))

488 Accesses

1 Citations

Regression testing is a methodology to inspect the behavior product with an accurate arrangement of test plans and test cases. It is required to reduce the time and cost of the developed error-free software. The three key strategies are used to conduct regression testing: reset all, test selection, and test case prioritization (TCP). In regression testing, the prioritization test case is indeed much more utilized due to early error detection capability. This technique is very beneficial in which faults will get detected early in the end-user. It is the most efficient regression testing approach where test cases are re-scheduled for implementation, and this improves the validity of test methods to achieve increasing the rate of fault detection. The TCP is done depending on different attributes like a customer requirement-based priority, coverage, historical information-based model-based, cost-time aware, risk-aware, search-based, etc. The fundamental idea behind the methodical survey in this paper is to provide visibility of advancement in researchers done for TCP. In this paper, a systematic is compiled on the research conducted on TCP between the years from 1999 to 2020. Along with the systematic review, we described the prioritization process in a chronological catalog listing of the reviewed papers and came up with some research motivation, and one of these papers will be beneficial for both learners and prepared experts during their research in the field of TCP.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Prevailing Standards in Requirement-Based Test Case Prioritization: An Overview

Functional Requirement-Based Test Case Prioritization in Regression Testing: A Systematic Literature Review

A Regression Test Case Prioritization Framework for Software Sustainability

Elbaum, S., Malishevsky, A.G., Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Softw. Eng. 28 (2), 159–182 (2002)

Article   Google Scholar  

Rothermel, G., Roland H.: Test case prioritization: an empirical study. IEEE Trans. Softw. Eng. (1999)

Google Scholar  

Srikanth, H., Williams, L., Osborne, J.: System test case prioritization of new and regression test cases. In: International Computer Software and Applications Conference, Chicago, Illinois (2006)

Kim, J., Porter, A.: A history-based test prioritization technique for regression testing in resource constrained environments. In: Proceedings of the 24th international conference on software engineering, pp. 119–129 (2002).  https://doi.org/10.1145/581339.581357

Bo, Q., Changhai N., Baowen X., Zhang, X.: Test case prioritization for black box testing. In: International Computer Software and Applications Conference (COMPSAC 2007) 0-7695-2870-8/07 © 2007

Yoo, S., Harman, M.: Regression testing minimisation, selection, and prioritisation: a survey. Test Verify Reliab. 10 , 1–7 (2007)

Rothermel, G., Untch, R.H., Chengyun, C., Harrold, M.J.: Prioritizing test cases for regression testing. IEEE Trans. Softw. Eng. 27 (10) (2001)

Roongruangsuwan, S., Daengdej, J.: Test case prioritization techniques: an empirical study. J. Theor. Appl. Inf. Technol. © 2005–2010. JATIT & LLS

Srikanth, H., Williams, L.: Requirements-based test case prioritization. IEEE Trans. Softw. Eng. 28 (2010)

Tonella, P., Avesani, P.: Using the case-based ranking methodology for test case prioritization. In: Italy International Symposium on Software Testing and Analysis (ISSTA), pp. 97–106 (2006)

Marijan, D., Gotlieb, A., Sen, S.: Test case prioritization for continuous regression testing: an industrial case study. In: 29th IEEE international conference on Software Maintenance (ICSM), pp. 540–543 (2013)

Korel, B., Koutsogiannakis, G.: Experimental comparison of code-based and model-based test prioritization. In: IEEE International Conference on Software Testing Verification and Validation Workshops (2007)

Zhao, Z.: Test case prioritization ba s ed on analysis of program structure. In: International Conference on Quality Software (2008)

Malangave, P., Kulkarni, D.: Efficient test case prioritization in regression testing. In: International Conference on Software Maintenance (2008)

Engström, E., Runeson, P., Skoglund, M.: A systematic review on regression test selection techniques. Inf. Softw. Technol. 52 , 14–30 © 2009 Elsevier (2010)

Mohanty, S., Acharya, A.A., Mohapatra, D.P.: A model-based prioritization technique for component-based software retesting using UML state chart diagram. In: International Conference on Electronics Computer Technology, © 2011 IEEE

Catal, C.: On the application of genetic algorithms for test case prioritization: a systematic literature review. In: Proceedings of the 2nd International Workshop, © 2012 Springer

Malhotra, R., Tiwari, D.: Development of a framework for test case. Prioritization Using Genetic Algorithm. ACM SIGSOFT Softw. Eng. 38 (3) (2013)

Siddik, M.S., Sakib, K.: An effective test case prioritization framework using software requirements, design and source code collaboration. In: 17th International Conference on Computer and Information Technology (ICCIT) © 2014

Indumathi, C.P., Selvamani, K.: Test case prioritization using open dependency structure algorithm. In: Proceedings of International Conference on Intelligent Computing, Communication and Convergence (ICCC-2015). Procedia Comput. Sci. Elsevier 48 , 250–255 (2015)

Wang, X., Zeng, H.: History-based dynamic test case prioritization for requirement properties in regression testing. In: International Workshop on Continuous Software Evolution and Delivery. ISBN 978-1-4503-4157-8/16/0 ©2016.

Rosero, H., Gómez, S., Rodríguez, G.: 15 years of software regression testing techniques—a survey. Int. J. Softw. Eng. Knowl. Eng. 26 (05), 675–689 (2016)

Srikanth, H., Cashman, M., Cohen, M.: Test case prioritization of build acceptance tests for an enterprise cloud application: an industrial case study. J. Syst. Softw. 119 , 122–135 (2016)

Spieker, H., Gotlieb, A., Marijan, A., Mossige, M.: Reinforcement learning for automatic test case prioritization and selection in continuous integration. In: 26th ACM SIGSOFT International Symposium on software testing and analysis, pp. 12–22 (2017)

Bhuyan, P., Ray, A., Das, M.: Test scenario prioritization using UML use case and activity diagram. In: Computational Intelligence in Data Mining, Advances in Intelligent Systems and Computing, p. 556, © 2017 Springer

Sultan, Z., Bhatti, S.N., Abbas, R., Shah, S.A.A.: Analytical review on test cases prioritization techniques: an empirical study. (IJACSA) Int. J. Adv. Comput. Sci. Appl. 8 (2) (2017)

Panda, N., Acharya, A.A., Bhuyan, P., Mohapatra, D.P.: Test case prioritization using UML state chart diagram and end-user priority computational intelligence in data mining. In: Advances in Intelligent Systems and Computing, p. 556. © Springer Nature Singapore Pte Ltd. (2017). https://doi.org/10.1007/978-981-10-3874-7_54

Panda, N., Acharya, A.A., Mohapatra, D.P.: Test scenario prioritization for object-oriented systems using UML diagram. Int. J. Syst. Assur. Eng. Manage. 10 , 316–325 ©2019 Springer

Lachmann, R.: Machine learning-driven test case prioritization approaches for black-box software testing. In: Test and Telemetry Conference © 2018, Springer

Panwar, D., Tomar, P., Harsh H., Siddique, M.H.: Improved Meta-Heuristic Technique for Test Case Prioritization © 2018 Springer

Dhareula, P., Ganpati, A.: Flower pollination algorithm for test case prioritization in regression testing. In: ICT Analysis and Applications, Lecture Notes in Networks and Systems, © 2020 Springer

Panda, M., Dash, S.: Test-case generation for model-based testing of object-oriented programs. In: Automated Software Testing (pp. 53–77) © 2020 Springer

Panda, N., Acharya, A.A., Mohapatra, D.P.: Regression testing of object-oriented systems using UML state machine diagram and sequence diagram. Int. J. Comput. Sci. Math. (IJCSM) 12 (2) (2020)

Meçe, E.M., Hakik, P., Binjaku, K.: The application of machine learning in test case prioritization—a review. EJECE, Eur. J. Electr. Comput. Eng. 4 (1) (2020)

Raju, S., Uma, G.V.: Factors oriented test case prioritization technique in regression testing us ng genetic algorithm. Eur. J. Sci. Res. 74 (3), 389–402 (2012)

Yoon, M.: A test case prioritization through correlation of requirement and risk. J. Softw. Eng. Appl. 5 (10), 823–836 (2012)

Mirarab, S., Tahvildari, L.: An empirical study on Bayesian network-based approach for test case prioritization. In: International Conference on Software Testing, Verification, and Validation, pp. 278–287 (2008)

Download references

Author information

Authors and affiliations.

School of Computer Engineering, Kalinga Institute of Industrial Technology (Deemed to be University), Bhubaneswar, India

Umakanta Dash & Arup Abhinna Acharya

You can also search for this author in PubMed   Google Scholar

Editor information

Editors and affiliations.

Department of Computer Science and Engineering, University of Kalyani, Kalyani, West Bengal, India

Jyotsna Kumar Mandal

School of Computing and Information Systems, The University of Melbourne, Melbourne, VIC, Australia

Rajkumar Buyya

Department of Computer Science and Engineering, Maulana Abul Kalam Azad University of Technology, West Bengal, Kolkata, West Bengal, India

Debashis De

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper.

Dash, U., Acharya, A.A. (2022). A Systematic Review of Test Case Prioritization Approaches. In: Mandal, J.K., Buyya, R., De, D. (eds) Proceedings of International Conference on Advanced Computing Applications. Advances in Intelligent Systems and Computing, vol 1406. Springer, Singapore. https://doi.org/10.1007/978-981-16-5207-3_55

Download citation

DOI : https://doi.org/10.1007/978-981-16-5207-3_55

Published : 24 November 2021

Publisher Name : Springer, Singapore

Print ISBN : 978-981-16-5206-6

Online ISBN : 978-981-16-5207-3

eBook Packages : Intelligent Technologies and Robotics Intelligent Technologies and Robotics (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research