Performance Evaluation of Ensemble Methods For Software Fault Prediction: An Experiment.
Abstract
In object-oriented software development, a plethora of studies have been carried out to present the application of machine learning algorithms for fault prediction. Furthermore, it has been empirically validated that an ensemble method can improve classification performance as compared to a single classifier. But, due to the inherent differences among machine learning and data mining approaches, the classification performance of ensemble methods will be varied. In this study, we investigated and evaluated the performance of different ensemble methods with itself and base-level classifiers, in predicting the faults proneness classes. Subsequently, we used three ensemble methods AdaboostM1, Vote and StackingC with five base-level classifiers namely Naivebayes, Logistic, J48, VotedPerceptron and SMO in Weka tool. In order to evaluate the performance of ensemble methods, we retrieved twelve datasets of open source projects from PROMISE repository. In this experiment, we used k-fold (k=10) cross-validation and ROC analysis for validation. Besides, we used recall, precision, accuracy, F-value measures to evaluate the performance of ensemble methods and base-level Classifiers. Finally, we observed significant performance improvement of applying ensemble methods as compared to its base-level classifier, and among ensemble methods we observed StackingC outperformed other selected ensemble methods for software fault prediction. There are many studies which have found association between CK metrics and fault proneness of classes through prediction models. Besides, the behavior of several predicting quantitative models have been observed from a very simple linear analysis to the use of more complex models such as SVM, Logistic regression, neural network and decision tree [ 6,7,8,11,12]. The role of data mining and machine learning approaches in decision making, by extracting and refining the useful knowledge, and reducing the information overload, have been empirically validated, and organization can extensively collect relevant and meaningful data [7,12]. But, due to inherent differences among these data mining and machine learning approaches, questions arise for the eval