Kernel methods for software effort estimation - Effects of different kernel functions and bandwidths on estimation accuracy.
Abstract
Analogy based estimation (ABE) generates an effort estimate for a new software project through adaptation of similar past projects (a.k.a. analogies). Majority of ABE methods follow uniform weighting in adaptation procedure. In this research we investigated non-uniform weighting through kernel density estimation. After an extensive experimentation of 19 datasets, 3 evaluation criteria, 5 kernels, 5 bandwidth values and a total of 2090 ABE variants, we found that: (1) nonuniform weighting through kernel methods cannot outperform uniform weighting ABE and (2) kernel type and bandwidth parameters do not produce a definite effect on estimation performance. In summary simple ABE approaches are able to perform better than much more complex approaches. Hence,-provided that similar experimental settings are adopted-we discourage the use of kernel methods as a weighting strategy in ABE. Keywords Effort estimation · Data mining · Kernel function · Bandwidth The work was partially funded by NSF grant CCF:1017330 and the Qatar/West Virginia University research grant NPRP 09-12-5-2-470. E. Kocaguneli (B) · T. Menzies Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26505, USA e-mail: kocaguneli@gmail.com T. Menzies e-mail: tim@menzies.us J. W. Keung Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong e-mail: jacky.keung@comp.polyu.edu.hk 2 Empir Software Eng (2013) 18:1-24 1 Introduction If a researcher or an industrial practitioner reads the literature on software effort estimation (SEE), they will encounter a dauntingly large number of different estimation methods. For example, the following is a partial list of some of the methods currently being used: - - - - - - Boehm uses linear regression (Boehm 1981). Shepperd prefers analogy-based methods (Shepperd et al. 1996). Auer et al. (2006) use extensive search to find weights for project features. Pendharkar et al. use Bayesian Network (BN) for effort estimation and incorporate BN into decision making procedure against risks (Pendharkar et al. 2005). Mendes and Mosley use a data-driven Bayes net for web effort estimation (Mendes and M