Scaling up analogy-based software effort estimation: a comparison of multiple hadoop implementation schemes.
Abstract
1. Analogy-based estimation (ABE) is one of the most time consuming and compute intensive method in software development effort estimation. Optimizing ABE has been a dilemma because simplifying the procedure can reduce the estimation performance, while increasing the procedure complexity with more sophisticated theory may sacrifice an advantage of the unlimited scalability for a large data input. Motivated by an emergence of cloud computing technology in software applications, in this study we present 3 different implementation schemes based on Hadoop MapReduce to optimize the ABE process across multiple computing instances in the cloud-computing environment. We experimentally compared the 3 MapReduce implementation schemes in contrast with our previously proposed GPGPU approach (named ABE-CUDA) over 8 high-performance Amazon EC2 instances. Results present that the Hadoop solution can provide more computational resources that can extend the scalability of the ABE process. We recommend adoption of 2 different Hadoop implementations (Hadoop streaming and RHadoop) for accelerating the computation specifically for compute-intensive software engineering related tasks. Scaling up an effort estimation process is vital for software development, given an increasing amount of information being processed and a need of larger computing resources to facilitate the use of more sophisticated estimation technique. In the past, many important estimation parameters were usually simplified to complete the estimation process, because a sufficiently powerful computing architecture was not available [11, 20]. However, this simplification can produce different solutions and result in inaccurate estimation. To avoid unforeseeable consequences led by the simplification, a sufficiently powerful computing architecture is clearly an interesting alternative approach. We have started discussing this approach and explored a possibility of utilizing general proposed graphic processing unit (GPGPU) technology over one of the most compute intensive methods in software development effort estimation called analogy-based estimation (ABE) [20]. In that work, we proposed ABECUDA algorithm. A use of CUDA tech