An Empirical Analysis of Three-Stage Data-Preprocessing for Analogy-Based Software Effort Estimation on the ISBSG Data.
Abstract
Analogy-based software effort estimation is a method to estimate the project cost of an unseen project based on analogies against previous projects sharing selected features. The validity of the selected features depends on many factors, and one of most crucial factors is the effectiveness of the datapreprocessing techniques applied to the datasets of the previous projects. In this paper, we report the first controlled experiment that studies the class of three-stage data-preprocessing techniques with stages of missing data imputation, data normalization, and feature selection for analogy-based effort estimation. We conducted our investigation on the ISBSG data. The experimental results show that three-stage data-preprocessing techniques have significant impacts on the resultant effort estimation accuracy. The results also indicate that the combined use of Z-Score normalization, kNN imputation and mutual information based feature weighting can be an effective choice for analogy-based effort estimation. ABE DP DN FS GA kNNI ML MEI MDI MDT MI SA SBS SEE SFS 2 442 Authorized licensed use limited to: CITY UNIV OF HONG KONG. Downloaded on September 17,2026 at 02:31:47 UTC from IEEE Xplore. Restrictions apply. The present paper extends our previous work [7]. It extends the literature review in [7] and improves the experimental design in terms of the scope, dataset, and metrics. This study confines to the finding on the ISBSG data since we believe its cross-company characteristic could fairly help reflect the effect of multi-stage DP [7]. In particular, we investigate a kind of three-stage DP, which consists of data normalization, missing data imputation and feature selection in chain. After the data is processed via a three-stage DP, a ABE technique is applied to show the corresponding estimation accuracy. Compared to our previous work [7], the marginal effect of more DP techniques is also analysed. The ambiguity in terms of the multi-stage DP in our previous work [7] is further clarified. Apart from only using wrappers for feature selection in SEE, filters and optimization techniques are also frequently applied in recent analogy-based SEE (see Table I for mor