Empirical evaluation of analogy-x for software cost estimation.
Abstract
This paper reports on the empirical evaluation of a novel approach called Analogy-X, which is an extension to the classical analogy-based software cost estimation. The Analogy-X approach is a set of procedures that utilize the principles of the Mantel randomization test to provide inferential statistics to Analogy. Our previous studies have clearly demonstrated the novelty and effectiveness of this technique. This paper provides further empirical evaluation of Analogy-X using different kinds of datasets. Our results show that the prediction accuracy of Analogy-X is similar to the one of ANGEL. Analogy-X has the additional advantage of allowing the use of Mantel statistics to select project features and detect abnormal data points, which provides a sound statistical basis for analogy-based systems. 2. THE EXPERIMENT In our previous studies [2][3], we have only examined the effect and result using three large to ultra large datasets, Desharnais, CSC and ISBSG datasets [1][2]. In this study, the intention is to apply Analogy-X to three smaller datasets which have been used in [1], Albrecht, Telecom1, and Kemerer. The purpose is to assess that the Analogy-X approach is also applicable to a wider range of datasets. Especially, the size of these datasets is relatively small compared to the large datasets used in [2][3]. In reality, most of the SME software organizations would have only a small amount of project cases for software cost estimation. This study also has a focus on the application of Analogy-X on more homogenized datasets within the same software organization. Categories and Subject Descriptors D.2.9 [Software Engineering]: Management - Cost Estimation General Terms The evaluation criterion here is that Analogy-X should be capable of statistically testing the above stated basic principle of analogy in section 1. That is to use Analogy-X to check whether or not the dataset under investigation can be successfully applied with analogy-based prediction systems. The empirical assessment of Analogy-X is based on its conformance with the known expected outcomes under various conditions given by different datasets in the study. Management, Measurement, Experi