Time to Clean Your Test Objectives

Michaël Marcozzi (1, 2) , Mike Papadakis (3) , Sébastien Bardin (2) , Nikolai Kosmatov (2) , Virgile Prévosto (2) , Loic Correnson (2)

1 Department of Computing [Imperial College London]
2 LSL - Laboratoire Sûreté des Logiciels
3 SnT - Interdisciplinary Centre for Security, Reliability and Trust [Luxembourg]

Testing is the primary approach for detecting software defects. A major challenge faced by testers lies in crafting eecient test suites, able to detect a maximum number of bugs with manageable eeort. To do so, they rely on coverage criteria, which deene some precise test objectives to be covered. However, many common criteria specify a signiicant number of objectives that occur to be infeasible or redundant in practice, like covering dead code or semantically equal mutants. Such objectives are well-known to be harmful to the design of test suites, impacting both the eeciency and precision of the tester's eeort. This work introduces a sound and scalable technique to prune out a signiicant part of the infeasible and redundant objectives produced by a panel of white-box criteria. In a nutshell, we reduce this task to proving the validity of logical assertions in the code under test. The technique is implemented in a tool that relies on weakest-precondition calculus and SMT solving for proving the assertions. The tool is built on top of the Frama-C veriication platform, which we carefully tune for our speciic scalability needs. The experiments reveal that the pruning capabilities of the tool can reduce the number of targeted test objectives in a program by up to 27% and scale to real programs of 200K lines, making it possible to automate a painstaking part of their current testing process.

Mots clés: Coverage Criteria; Infeasible Objectives; Redundant Objectives

Domaines: Génie logiciel [cs.SE]

Michaël Marcozzi, Mike Papadakis, Sébastien Bardin, Nikolai Kosmatov, Virgile Prévosto, et al.. Time to Clean Your Test Objectives. International Conference On Software Engineering - ICSE, May 2018, Gothenburg, Sweden. ⟨cea-01835503⟩