Evidence-Based Software Engineering

EBSE for...

EBSE Bibliography


A systematic review of statistical power in software engineering experiments

Author(s)T. Dyba and V.B. Kampenes and D.I.K. Sjoberg
DetailsArticle: 2006
AbstractStatistical power is an inherent part of empirical studies that employ significance testing and is essential for the planning of studies, for the interpretation of study results, and for the validity of study conclusions. This paper reports a quantitative assessment of the statistical power of empirical software engineering research based on the 103 papers on controlled experiments (of a total of 5453 papers) published in nine major software engineering journals and three conference proceedings in the decade 1993-2002. The results show that the statistical power of software engineering experiments falls substantially below accepted norms as well as the levels found in the related discipline of information systems research. Given this study's findings, additional attention must be directed to the adequacy of sample sizes and research designs to ensure acceptable levels of statistical power. Furthermore, the current reporting of significance tests should be enhanced by also reporting effect sizes and confidence intervals.
DOINo DOI available
BibTexView Citation
TopicsMapping Study, Methodology, Software Engineering

[Return to the bibliography]