Evaluating and Improving Unified Debugging
Abstract
Automated debugging techniques, including fault localization and program repair, have been studied for over a decade. However, the only existing connection between fault localization and program repair is that fault localization computes the potential buggy elements for program repair to patch. Recently, a pioneering work, ProFL, explored the idea of unified debugging to unify fault localization and program repair in the other direction for the first time to boost both areas. In this way, ProFL also extends the application scope of automated repair to all possible bugs (not only the small ratio of bugs that repair systems can automatically fix). However, ProFL only considers one program repair system, and it is not clear how other repair systems contribute to unified debugging. In this work, we perform an extensive study of the unified debugging approach on 16 state-of-the-art program repair systems for the first time. Our initial experimental results on the Defects4J benchmark reveal various practical guidelines for unified debugging, such as (1) nearly all 16 studied repair systems positively contribute to unified debugging despite their varying repair capabilities, (2) repair systems targeting multi-edit patches can bring extraneous noise, (3) repair systems with more executed/plausible patches tend to perform better, (4) unified debugging effectiveness does not rely on the availability of correct patches, and (5) we propose a new technique, UniDebug++, which localizes over 20% more bugs within Top-1 than state-of-the-art technique ProFL. Furthermore, we extend the above experiments to make the following additional contributions: we (6) further perform an extensive study on 76.3% additional bugs and confirm that UniDebug++ again outperforms ProFL by localizing 185 (out of 395) bugs within Top-1, (7) investigate the impact of 33 SBFL formulae and observe UniDebug++ consistently improving upon all formulae, (8) demonstrate that UniDebug++ can substantially boost state-of-the-art learning-based method-level fault localization techniques, (9) extend unified debugging to the statement level for first time and observe that UniDebug++ localizes 78 (out of 395) bugs within Top-1 and outperforms state-of-the-art learning-based fault localization techniques by 30%, and finally (10) propose a new technique, UniDebug+*, based on detailed patch statistics, to further improve upon UniDebug++.