Investigating the effect of fitness functions on the performance of automated testing techniques

Testing is technically and economically crucial for ensuring software quality. One of the most challenging testing tasks is to create test suites that will reveal potential defects in software. However, as the size and complexity of software systems increase, the task becomes more labour-intensive and manual test data generation becomes infeasible. To address this issue, researchers have proposed different approaches to automate the process of generating test data using optimisation techniques. Due to the non-linear nature of fitness functions and the large search space, one direction of research has focussed on using metaheuristics, which are approximate methods of finding good solutions in a reasonable amount of time. For many years, researchers have been proposing different definitions of fitness functions for use in the automatic generation of test data, utilising different measurements to evaluate the quality of the test data produced. Investigating the conditions under which a fitness function will either succeed in producing, or fail to produce high software coverage is critical for understanding the suitability and limitations of different fitness function definitions, as well as for automated test data generation. This study examines the performance of a variety of fitness functions across diverse types of software, providing resources from which these conditions may be derived. Experiments show the performance of fitness functions is problem-dependent, such that the coverage obtained by a given function is impacted by the software features being tested. To a large extent, software features which can be captured by software metrics may be seen as an indicator of the possibility of a given fitness function achieving high coverage.