Evidence-Based Software Engineering

EBSE for...

Software Engineering Evidence Map

One of the goals of this website is to document secondary studies of software engineering topics. This page classifies SLRs and mapping studies using the knowledge areas defined by the SWEBOK.

CategoryStudiesStudy Details

[S62] A systematic review of search-based testing for non-functional system properties

2009, Afzal, W., Torkar, R., Feldt, R.

Source: IST Virtual Special Issue on Systematic Reviews in Information and Software Engineering

Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996-2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques.
Software RequirementsS23, S26, S27, S63
Software DesignS1, S25, S31, S38
Software ConstructionS18, S43, S61
Software TestingS10, S17, S28, S62
Software MaintenanceS24, S30
Software Configuration Management -
Software Engineering ManagementM4, S5, S7, S8, S11, S12, S14, S21, S22, S29, S45, S46, S53, S66
Software Engineering ProcessesS3, S41, S47, S49, S50
Software Engineering Tools and MethodsS60, S64, S65
Software QualityM1, S15
Unclassified -