Evidence-Based Software Engineering

EBSE for...

Software Engineering Evidence Map

One of the goals of this website is to document secondary studies of software engineering topics. This page classifies SLRs and mapping studies using the knowledge areas defined by the SWEBOK.

CategoryStudiesStudy Details

[S8] Estimation of Software Development Work Effort: Evidence on Expert Judgement and Formal Models

2007, M. Jorgensen

Source: Systematic literature reviews in software engineering - A systematic literature review

The main goal of the review presented in this paper is to examine when to use expert judgment, when to use formal models, and, when to combine these two approaches when estimating software development work effort. Sixteen relevant studies were identified and reviewed. The review found that the average accuracy of expert judgment-based effort estimates was better than the average accuracy of the models in ten of the sixteen studies. Two indicators of higher accuracy of judgment-based effort estimates seem to be that: i) the estimation models are not calibrated to the organization using the model, and, ii) the experts possess important context information not included in the formal estimation models. The use of models, on the other hand, may be particularly useful in estimation situations believed to lead to stronger than usual degree of over-optimism, particularly very large projects. Five of the reviewed studies evaluated estimates based on a combination of expert judgment and model. In all five studies the average accuracy of combination-based effort estimates had similar or more accurate estimates than the average accuracy of the expert estimates and of the best model.
Our Assessment: A good systematic review giving an unexpected result - there is no strong evidence that cost estimates from formal models are better than expert option estimates. 12 of 16 studies comparing model estimates and expert opinion estimates suggested that the expert opinion estimates were better than those of formal models.
Software RequirementsS23, S26, S27, S63
Software DesignS1, S25, S31, S38
Software ConstructionS18, S43, S61
Software TestingS10, S17, S28, S62
Software MaintenanceS24, S30
Software Configuration Management -
Software Engineering ManagementM4, S5, S7, S8, S11, S12, S14, S21, S22, S29, S45, S46, S53, S66
Software Engineering ProcessesS3, S41, S47, S49, S50
Software Engineering Tools and MethodsS60, S64, S65
Software QualityM1, S15
Unclassified -