JISE


  [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]


Journal of Information Science and Engineering, Vol. 38 No. 2, pp. 357-374


Evaluating the Influence on Fault Localization Caused by Test Suite Reduction in Continuous Integration Process


JUTARPORN INTASARA AND CHU-TI LIN+
Department of Computer Science and Information Engineering
National Chiayi University
Chiayi, 600 Taiwan
E-mail: {s1040464; chutilin
+}@mail.ncyu.edu.tw

Correponding author: Chu-Ti Lin


Test suite reduction (TSR) is a frequently adopted approach to improve the efficiency of regression testing while spectrum-based fault localization (SBFL) is a famous approach to shorten the tedious debugging process. Both approaches can be incorporated into the continuous integration (CI) environment. A configuration for a CI is relevant to the settings of three TSR factors (namely the coverage granularity level, the metric to evaluate test cases, and the TSR strategy) and the selection of SBFL technique. Since different TSR techniques will produce different test logs and test logs are the inputs to SBFL techniques, the selections of TSR and SBFL techniques both impact the effectiveness of fault localization (FL). Thus, it is important for software developers to choose a suitable TSR technique and determine which SBFL technique will assort well with it. This paper also aims to investigate how each of the aforementioned parameters in a CI configuration affects FL effectiveness. Our experiment results indicate that applying TSR may be harmful to FL effectiveness and the Jaccard SBFL technique is the most effective to locate faults no matter which of the TSR techniques (including using the original test suite directly) is adopted to run regression testing. Additionally, the experimental results also indicate that it is better to perform TSR based on branch coverage information if TSR must be included in the CI. In comparison with coverage granularity level, the other two TSR factors do not cause statistically significant impact on FL effectiveness. Our findings should be useful for software developers to configure their CI.


Keywords: software testing, debugging, continuous integration, test suite reduction, fault localization

  Retrieve PDF document (JISE_202202_05.pdf)