Investigating the Significance of Bellwether Effect to Improve Software Effort Estimation.
Abstract
Bellwether effect refers to the existence of exemplary projects (called the Bellwether) within a historical dataset to be used for improved prediction performance. Recent studies have shown an implicit assumption of using recently completed projects (referred to as moving window) for improved prediction accuracy. In this paper, we investigate the Bellwether effect on software effort estimation accuracy using moving windows. The existence of the Bellwether was empirically proven based on six postulations. We apply statistical stratification and Markov chain methodology to select the Bellwether moving window. The resulting Bellwether moving window is used to predict the software effort of a new project. Empirical results show that Bellwether effect exist in chronological datasets with a set of exemplary and recently completed projects representing the Bellwether moving window. Result from this study has shown that the use of Bellwether moving window with the Gaussian weighting function significantly improve the prediction accuracy. Keywords: Bellwether Effect, Bellwether moving window, Markov chains, Chronological dataset. 1. INTRODUCTION Software effort estimation (SEE) is a core activity in software development life cycle [1] specifically for scheduling and monitoring a new software project development. SEE is defined as the process of predicting the most realistic effort needed to complete the development of a software project. The core objective of SEE is to obtain a robust and accurate predictive model which has proven elusive over the past years. The prediction accuracy of a model depends on the training and validation sets as well as the relative consistency of the nature of dataset considered. According to Mair et al. [2], there exist difficulty in obtaining relevant training data for building effort estimation models since the appropriate 1 ones might be proprietary. This constraint makes researchers rely on using all available historical projects (referred to as the growing portfolio) as the training set which might be very old and will not necessarily yield a better prediction accuracy for new projects. Other researchers rely on using open source an