Test case prioritization and selection technique in continuous integration development environments: a case study

  • Authors

    • Lei Xiao
    • Huaikou Miao
    • Ying Zhong
    2018-05-16
    https://doi.org/10.14419/ijet.v7i2.28.13207
  • Software Process, Software Development Life Cycle, Traditional Models, Agile Models, Evaluation Metrics.

  • Regression testing is a very important activity in continuous integration development environments. Software engineers frequently integrate new or changed code that involves in a new regression testing. Furthermore, regression testing in continuous integration development environments is together with tight time constraints. It is also impossible to re-run all the test cases in regression testing. Test case prioritization and selection technique are often used to render continuous integration processes more cost-effective. According to multi objective optimization, we present a test case prioritization and selection technique, TCPSCI, to satisfy time constraints and achieve testing goals in continuous integration development environments. Based on historical failure data, testing coverage code size and testing execution time, we order and select test cases. The test cases of the maximize code coverage, the shorter execution time and revealing the latest faults have the higher priority in the same change request. The case study results show that using TCPSCI has a higher cost-effectiveness comparing to the manually prioritization.

     

     

  • References

    1. [1] M. J. Arafeen and H. Do. Test case prioritization using requirements-based clustering. In Proceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, ICST ’13, pages 312–321, Washington, DC, USA, 2013. IEEE Computer Society.

      [2] R. Carlson, H. Do and A. Denton. A clustering approach to improving test case prioritization: An industrial case study. In Proceedings of the 2011 27th IEEE International Conference on Software Maintenance, pages 382–391, Washington, DC, USA, 2011. IEEE Computer Society.

      [3] X. Chen, J.-H. Chen, X.-L. Ju, and Q. Gu. Survey of test case prioritization techniques for regression testing. Journal of Software, 24(8):1695–1712, 2014.

      [4] S. Elbaum, A. Mclaughlin, and J. Penix. The google dataset of testing results. https://code.google.com/p/ google-shared-dataset-of-test-suite-resutls, 2014.

      [5] S. Elbaum, G. Rothermel, and J. Penix. Techniques for improving regression testing in continuous integration development environments. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2014, pages 235–245. ACM, 2014.

      [6] M. Fowler. Continuous integration. https: //www.martinfowler.com/articles/ continuousIntegration.html, 2006.

      [7] Glover, P. M. Duvall, and S. Matyas. Continuous Integration: Improving Software Quality and Reducing Risk. Pearson Education, 2007.

      [8] Henard, M. Papadakis, M. Harman, Y. Jia, and Y. Le Traon. Comparing white-box and black-box test prioritization. In Proceedings of the 38th International Conference on Software Engineering, pages 523–534. ACM, 2016.

      [9] P. Kandil, S. Moussa, and N. Badr. Cluster-based test cases prioritization and selection technique for agile regression testing. Journal of Software Evolution & Process, 29(6), 2017.

      [10] G. Malishevsky, G. Rothermel, and S. Elbaum. Modeling the cost-benefits tradeoffs for regression testing techniques. In Proceedings of the International Conference on Software Maintenance, pages 1–10, 2002.

      [11] D. Marijan, A. Gotlieb, and S. Sen. Test case prioritization for continuous regression testing: An industrial case study. In Proceedings of the 2013 IEEE International Conference on Software Maintenance, ICSM ’13, pages 540–543. IEEE Computer Society, 2013.

      [12] B. Qu, C.-H. Nie, and B.-W. Xu. Case prioritization based on test suite design information. Chinese Journal of Computers, 31(3):431–439, 2008.

      [13] G. Rothermel and M. J. Harrold. Analyzing regression test selection techniques. IEEE Transactions on Software Engineering, 22(8):529–551, 1996.

      [14] G. Rothermel and M. J. Harrold. A safe, efficient regression test selection technique. ACM Transactions on Software Engineeering Methodology, 6(2):173–210, 1997.

      [15] G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Test case prioritization: An empirical study. In Proceedings of the International Conference on Software Maintenance, pages 179–188, 1999.

      [16] G. Rothermel, R. J. Untch, and C. Chu. Prioritizing test cases for regression testing. IEEE Transactions on Software Engineering, 27(10):929–948, 2001.

      [17] H. Spieker, A. Gotlieb, D. Marijan, and M. Mossige. Reinforcement learning for automatic test case prioritization and selection in continuous integration. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 12–22. ACM, 2017.

      [18] H. Spieker, A. Gotlieb, D. Marijan, and M. Mossige. Reinforcement learning for automatic test case prioritization and selection in continuous integration. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2017, pages 12–22. ACM, 2017.

      [19] S. Yoo and M. Harman. Regression testing minimization, selection and prioritization: a survey. Software Testing Verification & Reliability, 22:67–120, 2012.

      [20] S. Yoo, R. Nilsson, and M. Harman. Faster fault finding at google using multi objective regression test optimisation. In Proceedings of ACM SIGSOFT Symposium on the Foundations of Software Engineering, FSE 2011. ACM, 2011.

  • Downloads

  • How to Cite

    Xiao, L., Miao, H., & Zhong, Y. (2018). Test case prioritization and selection technique in continuous integration development environments: a case study. International Journal of Engineering & Technology, 7(2.28), 332-336. https://doi.org/10.14419/ijet.v7i2.28.13207