Assessing the Reliability of Efron’s Bootstrap Statistical Approach for Providing Confidence Limits of Standard Software Metrics

Business decisions are often made based on metrics gathered from software engineering processes. There are two questions that are important to answer. First – if the accuracy of the metrics had been known, would the same business decision have been made? The Efron Bootstrap is a computer intensive approach to determine accuracy of metrics from small data sets. Reference has been made in the literature of several applications of Efron Bootstrap’s Statistical Approach to determine the accuracy of metrics in the context of Software Engineering. However, there is extremely limited reference to whether the Efron Bootstrap actually provides an accurate determination of the errors associated with various common software metrics. We have found accurate Efron Bootstrap estimates for mean and median 90%confidence ranges, gross under estimates for variance errors and consistent overestimates of errors for skewness and kurtosis. The accuracy of estimates of correlation error ranges depends on the data set being analyzed