A systematic literature study of regression test case prioritization approaches

  • Authors

    • Omdev Dahiya Maharshi Dayanand University, Rohtak
    • Kamna Solanki Maharshi Dayanand University, Rohtak
    2018-09-16
    https://doi.org/10.14419/ijet.v7i4.15805
  • Regression Testing, Software Development, Software Testing, Systematic Literature Study, Test Case Prioritization.

  • Abstract

    Regression testing is about running the entire test ensemble again to ensure that amendments do not negatively affect the system. A popular approach in regression testing is test case prioritization which reorders test cases in such a way that those with higher priorities are run earlier than those with lower priorities based on some criterion. Numerous researchers have worked on different aspects of prioritization approaches. This paper presents the result of the study conducted on different prioritization approaches showing mostly stressed areas by researchers and the areas where there is a future scope. For it, studies related to test case prioritization in regression testing from the year 2004 to 2018 are analyzed by dividing this time period into three slots of five years each. 36 studies were selected from 948 studies to answer the research questions framed for this study. The trends followed in TCP along with the approaches evolved are thus documented to find the current trends and future scope for the researchers to work upon.

     

     

  • References

    1. [1] Leung, H. K., & White, L. (1989, October). Insights into regression testing (software testing). In Software Maintenance, 1989, Proceedings, Conference on (pp. 60-69). IEEE.

      [2] Rothermel, G., Untch, R. H., Chu, C., & Harrold, M. J. (1999). Test case prioritization: An empirical study. In Software Maintenance, 1999. (ICSM'99) Proceedings. IEEE International Conference on (pp. 179-188). IEEE. https://doi.org/10.1109/ICSM.1999.792604.

      [3] Rothermel, G., Untch, R. H., Chu, C., & Harrold, M. J. (2001). Prioritizing test cases for regression testing. IEEE Transactions on software engineering, 27(10), 929-948.

      https://doi.org/10.1109/32.962562.

      [4] Yoo, S., & Harman, M. (2012). Regression testing minimization, selection and prioritization: a survey. Software Testing, Verification and Reliability, 22(2), 67-120. https://doi.org/10.1002/stv.430.

      [5] Solanki, K., & Singh, Y. (2014). Importance of Selecting Test Cases for Regression Testing. IOSR Journal of Computer Engineering (IOSRJCE) e-ISSN, 2278-0661. https://doi.org/10.9790/0661-16444351.

      [6] Elbaum, S., Kallakuri, P., Malishevsky, A., Rothermel, G., & Kanduri, S. (2003). Understanding the effects of changes on the costâ€effectiveness of regression testing techniques. Software testing, verification and reliability, 13(2), 65-83.

      https://doi.org/10.1002/stvr.263.

      [7] Elbaum, S., Malishevsky, A. G., & Rothermel, G. (2002). Test case prioritization: A family of empirical studies. IEEE transactions on software engineering, 28(2), 159-182.

      https://doi.org/10.1109/32.988497.

      [8] Raju, S., & Uma, G. V. (2012). Factors oriented test case prioritization technique in regression testing using genetic algorithm. European Journal of Scientific Research, 74(3), 389-402.

      [9] Kumar, A., & Singh, K. (2014). A Literature Survey on test case prioritization. Compusoft, 3(5), 793.

      [10] Kiran, R. S. (2015). A literature survey on TCP-test case prioritization using the RT-regression techniques. Global Journal of Research in Engineering.

      [11] Catal, C., & Mishra, D. (2013). Test case prioritization: a systematic mapping study. Software Quality Journal, 21(3), 445-478. https://doi.org/10.1007/s11219-012-9181-z.

      [12] Singh, Y., Kaur, A., Suri, B., & Singhal, S. (2012). Systematic literature review on regression test prioritization techniques. Informatica, 36(4).

      [13] Khatibsyarbini, M., Isa, M. A., Jawawi, D. N., & Tumeng, R. (2017). Test case prioritization approaches in regression testing: A systematic literature review. Information and Software Technology.

      [14] Omdev, Solanki, K., Dalal, S., A Literature study of various regression testing approaches. InComputing for Sustainable Global Development (INDIACom), 2018 5th International Conference on 2018 Mar 15. IEEE. (In Press).

      [15] Solanki, K., Singh, Y., & Dalal, S. (2016). A Comparative Evaluation of “m-ACO†Technique for Test Suite Prioritization. Indian Journal of science and technology, 9(30).

      https://doi.org/10.17485/ijst/2016/v9i30/86423.

      [16] Solanki, K., Singh, Y., & Dalal, S. (2016). Experimental analysis of m-ACO technique for regression testing. Indian Journal of Science and Technology, 9(30).

      https://doi.org/10.17485/ijst/2016/v9i30/86588.

      [17] Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004), 1-26.

      [18] https://scholar.google.co.in.

      [19] https://dl.acm.org.

      [20] https://ieeexplore.ieee.org.

      [21] https://www.elsevier.com/en-in.

      [22] https://www.springer.com/in.

      [23] Qu, X., Cohen, M. B., & Woolf, K. M. (2007, October). Combinatorial interaction regression testing: A study of test case generation and prioritization. In Software Maintenance, 2007. ICSM 2007. IEEE International Conference on (pp. 255-264). IEEE. https://doi.org/10.1109/ICSM.2007.4362638.

      [24] Yoo, S., & Harman, M. (2007, July). Pareto efficient multi-objective test case selection. In Proceedings of the 2007 international symposium on Software testing and analysis (pp. 140-150). ACM.

      [25] Elbaum, S., Rothermel, G., Kanduri, S., & Malishevsky, A. G. (2004). Selecting a cost-effective test case prioritization technique. Software Quality Journal, 12(3), 185-210. https://doi.org/10.1023/B:SQJO.0000034708.84524.22.

      [26] Malishevsky, A. G., Ruthruff, J. R., Rothermel, G., & Elbaum, S. (2006). Cost-cognizant test case prioritization. Department of Computer Science and Engineering, University of Nebraska-Lincoln, Techical Report.

      [27] Ramasamy, K., & Mary, S. A. (2008, December). Incorporating varying requirement priorities and costs in test case prioritization for new and regression testing. In Computing, Communication and Networking, 2008. ICCCn 2008. International Conference on (pp. 1-9). IEEE.

      [28] Srikanth, H., Williams, L., & Osborne, J. (2005, November). System test case prioritization of new and regression test cases. In Empirical Software Engineering, 2005. 2005 International Symposium on (pp. 10-pp). IEEE.

      [29] Jeffrey, D., & Gupta, N. (2007). Improving fault detection capability by selectively retaining test cases during test suite reduction. IEEE Transactions on software Engineering, 33(2).

      https://doi.org/10.1109/TSE.2007.18.

      [30] Baudry, B., Fleurey, F., Jézéquel, J. M., & Le Traon, Y. (2005). Automatic test case optimization: A bacteriologic algorithm. ieee Software, 22(2), 76-82. https://doi.org/10.1109/MS.2005.30.

      [31] Walcott, K. R., Soffa, M. L., Kapfhammer, G. M., & Roos, R. S. (2006, July). Timeaware test suite prioritization. In Proceedings of the 2006 international symposium on Software testing and analysis (pp. 1-12). ACM. https://doi.org/10.1145/1146238.1146240.

      [32] Hla, K. H. S., Choi, Y., & Park, J. S. (2008, July). Applying particle swarm optimization to prioritizing test cases for embedded real time software retesting. In Computer and Information Technology Workshops, 2008. CIT Workshops 2008. IEEE 8th International Conference on (pp. 527-532). IEEE.

      [33] Arafeen, M. J., & Do, H. (2013, March). Test case prioritization using requirements-based clustering. In Software Testing, Verification and Validation (ICST), 2013 IEEE Sixth International Conference on (pp. 312-321). IEEE.

      [34] Suri, B., & Singhal, S. (2011). Implementing ant colony optimization for test case selection and prioritization. International journal on computer science and engineering, 3(5), 1924-1932.

      [35] Krishnamoorthi, R., & Mary, S. S. A. (2009). Factor oriented requirement coverage based system test case prioritization of new and regression test cases. Information and Software Technology, 51(4), 799-808. https://doi.org/10.1016/j.infsof.2008.08.007.

      [36] Raju, S., & Uma, G. V. (2012). Factors oriented test case prioritization technique in regression testing using genetic algorithm. European Journal of Scientific Research, 74(3), 389-402.

      [37] Kavitha, R., Kavitha, V. R., & Kumar, N. S. (2010, October). Requirement based test case prioritization. In Communication Control and Computing Technologies (ICCCCT), 2010 IEEE International Conference on (pp. 826-829). IEEE.

      [38] Suri, B., Mangal, I., & Srivastava, V. (2011). Regression test suite reduction using an hybrid technique based on BCO and genetic algorithm. Special Issue of International Journal of Computer Science & Informatics (IJCSI), ISSN (PRINT), 2231-5292.

      [39] Suri, B., & Singhal, S. (2011). Analyzing test case selection & prioritization using ACO. ACM SIGSOFT Software Engineering Notes, 36(6), 1-5. https://doi.org/10.1145/2047414.2047431.

      [40] Suri, B., & Singhal, S. (2012, September). Literature survey of ant colony optimization in software testing. In Software Engineering (CONSEG), 2012 CSI Sixth International Conference on (pp. 1-7). IEEE.

      [41] Salehie, M., Li, S., Tahvildari, L., Dara, R., Li, S., & Moore, M. (2011, March). Prioritizing requirements-based regression test cases: A goal-driven practice. In Software Maintenance and Reengineering (CSMR), 2011 15th European Conference on (pp. 329-332). IEEE. https://doi.org/10.1109/CSMR.2011.46.

      [42] Catal, C. (2012, September). On the application of genetic algorithms for test case prioritization: a systematic literature review. In Proceedings of the 2nd international workshop on evidential assessment of software technologies (pp. 9-14). ACM.

      https://doi.org/10.1145/2372233.2372238.

      [43] Gupta, V., & Chauhan, D. S. (2011, December). Hybrid regression testing technique: a multi layered approach. In India Conference (INDICON), 2011 Annual IEEE (pp. 1-5). IEEE.

      https://doi.org/10.1109/INDCON.2011.6139363.

      [44] Kaur, A., & Goyal, S. (2011). A genetic algorithm for faultbased regression test case prioritization. International Journal of Computer Applications, 32(8), 975-8887.

      [45] Di Nardo, D., Alshahwan, N., Briand, L., & Labiche, Y. (2015). Coverageâ€based regression test case selection, minimization and prioritization: A case study on an industrial system. Software Testing, Verification and Reliability, 25(4), 371-396.

      https://doi.org/10.1002/stvr.1572.

      [46] Mayan, J. A., & Ravi, T. (2015). Structural software testing: hybrid algorithm for optimal test sequence selection during regression testing. International Journal of Engineering and Technology (IJET), 7(1).

      [47] Dobuneh, M. R. N., Jawawi, D. N., Ghazali, M., & Malakooti, M. V. (2014, September). Development test case prioritization technique in regression testing based on hybrid criteria. In Software Engineering Conference (MySEC), 2014 8th Malaysian (pp. 301-305). IEEE.

      [48] Wang, X., & Zeng, H. (2016, May). History-based dynamic test case prioritization for requirement properties in regression testing. In Continuous Software Evolution and Delivery (CSED), IEEE/ACM International Workshop on (pp. 41-47). IEEE.

      [49] Hettiarachchi, C., Do, H., & Choi, B. (2014, June). Effective regression testing using requirements and risks. In Software Security and Reliability (SERE), 2014 Eighth International Conference on (pp. 157-166). IEEE.

      [50] Panichella, A., Oliveto, R., Di Penta, M., & De Lucia, A. (2015). Improving multi-objective test case selection by injecting diversity in genetic algorithms. IEEE Transactions on Software Engineering, 41(4), 358-383. https://doi.org/10.1109/TSE.2014.2364175.

      [51] Ma, T., Zeng, H., & Wang, X. (2016, May). Test case prioritization based on requirement correlations. In Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), 2016 17th IEEE/ACIS International Conference on (pp. 419-424). IEEE.

      [52] Srikanth, H., Hettiarachchi, C., & Do, H. (2016). Requirements based test prioritization using risk factors: An industrial study. Information and Software Technology, 69, 71-83.

      https://doi.org/10.1016/j.infsof.2015.09.002.

      [53] Silva, D., Rabelo, R., Campanha, M., Neto, P. S., Oliveira, P. A., & Britto, R. (2016, July). A hybrid approach for test case prioritization and selection. In Evolutionary Computation (CEC), 2016 IEEE Congress on (pp. 4508-4515). IEEE.

      https://doi.org/10.1109/CEC.2016.7744363.

      [54] Achimugu, P., Selamat, A., Ibrahim, R., & Mahrin, M. N. R. (2014). A systematic literature review of software requirements prioritization research. Information and software technology, 56(6), 568-585. https://doi.org/10.1016/j.infsof.2014.02.001.

      [55] Gao D, Guo X, Zhao L. (2015). Test case prioritization for regression testing based on ant colony optimization. InSoftware Engineering and Service Science (ICSESS), 6th IEEE International Conference on (pp. 275-279). IEEE.

      https://doi.org/10.1109/ICSESS.2015.7339054.

      [56] Solanki, K., Singh, Y., Dalal, S., & Srivastava, P. R. (2016). Test case prioritization: An approach based on modified ant colony optimization. In Emerging Research in Computing, Information, Communication and Applications (pp. 213-223). Springer, Singapore.

      [57] Vescan, A., Şerban, C., Chisăliţă-Cretu, C., & Dioşan, L. (2017, September). Requirement dependencies-based formal approach for test case prioritization in regression testing. In Intelligent Computer Communication and Processing (ICCP), 2017 13th IEEE International Conference on (pp. 181-188). IEEE. https://doi.org/10.1109/ICCP.2017.8117002.

      [58] Schwartz, A., & Do, H. (2016). Cost-effective regression testing through Adaptive Test Prioritization strategies. Journal of Systems and Software, 115, 61-81. https://doi.org/10.1016/j.jss.2016.01.018.

  • Downloads

  • How to Cite

    Dahiya, O., & Solanki, K. (2018). A systematic literature study of regression test case prioritization approaches. International Journal of Engineering & Technology, 7(4), 2184-2191. https://doi.org/10.14419/ijet.v7i4.15805

    Received date: 2018-07-19

    Accepted date: 2018-08-30

    Published date: 2018-09-16