Optimising Project Feature Weights for Analogy-Based Software Cost Estimation using the Mantel Correlation.
Abstract
Analogy-based systems such as ANGEL [1] do not consider each project feature's influence with respect to target software effort. Auer and Biffl [3] consider the importance of the project feature weights used to determine analogous projects, which will have the overall estimation accuracy and reliability impacts. However their method uses an extensive search approach similar to that of Shepperd and Schofield's brute-force approach [1] for feature subset selection, which inherits similar issues for analogy. In this paper, we suggest that using Mantel's correlation and randomisation test, it is possible to determine a set of project feature weights which can be used to account for the influence of each project feature based on inferential statistics rather than bruteforce. Also, our method can determine whether the dataset is indeed appropriate for analogy-based estimation, and detect extremely outlying cases that will ultimately distort prediction outcomes using a sensitivity analysis strategy. Results show that our method solves some of the fundamental issues of analogy, which feature weights now can be determined statistically, and more importantly, that prediction performance improves. Section 2 presents an overview of related work. Section 3 describes our method and the underlying theory. Section 4 provides the datasets and the analysis procedure. Section 5 presents the results, and section 6 discusses the results and provides further directions. Section 7 concludes the paper. Software cost estimation using analogy is an important area in software engineering research. Previous research has demonstrated that analogy is a viable alternative to other conventional estimation methods in terms of predictive accuracy. One of the important research areas for analogy is how to determine suitable project feature weights. This can be achieved by using an extensive project feature weights search, where the quality measure is optimised. However, this approach suffers similar issues as the brute-force feature selection approach in analogy. We propose a novel method to deal with this issue based upon the use of the Mantel randomisation test. Specifically, we determine proj