Case consistency: a necessary data quality property for software engineering data sets.
Abstract
Data quality is an essential aspect in any empirical study, because the validity of models and/or analysis results derived from an empirical data is inherently influenced by its quality. In this empirical study, we focus on data consistency as a critical factor influencing the accuracy of prediction models in software engineering. We propose a software metric called Cases Inconsistency Level (CIL) for analyzing conflicts within software engineering data sets by leveraging probability statistics on project cases and counting the number of conflicting pairs. The result demonstrated that CIL is able to be used as a metric to identify either consistent data sets or inconsistent data sets, which are valuable for building robust prediction models. In addition to measuring the level of consistency, CIL is proved to be applicable to predict whether or not an effort model built from data set can achieve higher accuracy, an important indicator for empirical experiments in software engineering. Categories and Subject Descriptors D.2.8 [Software Engineering]: Metrics-Performance measures General Terms Management, Measurement, Experimentation Keywords Data quality; Data consistency; Software effort estimation; Heterogeneous distance function; Empirical software engineering 1. INTRODUCTION In quantitative software engineering methodologies, accumulated historical data from previous software projects Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. EASE '15 April 27 - 29, 2015, Nanjing, China Copyright 2015 ACM 978-1-4503-3350-4/15/04 ...$15.00. http://dx.doi.org/10.1145/2745802.2745820. are an important information source supporting new software develo