Keeping the Most Important Data, Not the Most: A Smarter Fix for Imbalanced Defect Prediction
Software defect prediction, or SDP for short, must spot the few buggy modules buried inside a huge pile of clean ones, so most data is boring and only a little is interesting. A team from CityU and Zhengzhou University built AIIUS, a method that keeps the clean modules that actually shape how the model draws its dividing line between good and bad code. Across 18 real projects and 5 classifiers it lifted Balance by 9.50%, AUC by 3.43%, and MCC by 17.02% over strong competitors.
Software defect prediction, or SDP for short, must spot the few buggy modules buried inside a huge pile of clean ones, so most data is boring and only a little is interesting. A team from CityU and Zhengzhou University built AIIUS, a method that keeps the clean modules that actually shape how the model draws its dividing line between good and bad code. Across 18 real projects and 5 classifiers it lifted Balance by 9.50%, AUC by 3.43%, and MCC by 17.02% over strong competitors.
Why it matters
Software defect prediction suffers from a lopsided reality: an ordinary codebase is mostly clean, so only a handful of modules actually carry defects. When a machine learning model trains on that skew, it learns to call almost everything clean, which looks accurate but quietly lets real bugs slip through to production. Older fixes either randomly throw away clean data, which wastes useful signals, or invent synthetic buggy samples that add noise. AIIUS takes a smarter route: it scores every clean module by how much nudging it shifts the model's prediction, treating the ones near the decision boundary as the most valuable to keep.
The method borrows an idea from adversarial machine learning, where small crafted changes test how fragile a model is, and turns it into a positive, constructive tool. Instead of attacking models, AIIUS uses a lightweight stand in model to generate those small nudges and measures the target classifier's reaction, defined as the squared change in predicted defect probability. The bigger the reaction, the more that clean module matters for telling bugs apart from healthy code. It then clusters the clean instances first, chooses representatives, ranks them by their influence scores, and keeps only the most influential ones until the classes are balanced. The result is a training set that preserves the shape of the data while keeping the pieces that really define the boundary.
The payoff goes beyond one benchmark. On 18 public datasets drawn from four authoritative repositories and tested with five common classifiers, AIIUS delivered consistent, often statistically significant gains over six competing resampling methods on the metrics that matter most for imbalanced data. It also proved robust to design choices: a moderate clustering ratio gave the best mix of performance and stability, and a simple neural network stand in was enough, so the method does not depend on heavy machinery. Because it only needs the model's probability outputs and not its internals, it works even when the classifier is a closed black box, which matches how real systems are often deployed.
Key findings
- Datasets Evaluated on 18 public defect datasets spanning four authoritative repositories: AEEEM, NASA, PROMISE, and SOFTLAB.
- Classifiers Tested across five widely used classifiers: KNN, Random Forest, Decision Tree, Logistic Regression, and Naive Bayes.
- Balance gain Improves Balance by 9.50% on average over six representative resampling baselines.
- AUC gain Improves Area Under the ROC Curve by 3.43% on average over the baselines.
- MCC gain Improves Matthews Correlation Coefficient by 17.02% on average, with the largest single gain of 19.4% under Random Forest.
- Imbalance range The studied datasets span imbalance ratios from 3.5 up to 25.08, so the method holds up under extreme skew.
- Black box friendly AIIUS only queries the target classifier's probability outputs, so it works when the model is a closed black box.
- Open source The full AIIUS implementation is released on GitHub to support reproduction and reuse.
Three headline wins, one clear idea
- Balance improves by 9.50% across 18 datasets and 5 classifiers
- MCC rises 17.02%, the strongest of the three headline gains
- AUC gains 3.43% while keeping false alarms controlled
What it means for practitioners
For teams that already run defect prediction pipelines, AIIUS drops in as a data level preprocessing step ahead of training, so it works with nearly any classifier and does not require changing your model. Its main practical advantage is that it keeps the clean modules that actually sculpt the decision boundary rather than discarding clean data at random, which frequently improves balanced measures such as MCC and Balance that reward finding real bugs without crying wolf. Because it only needs the model's predicted probabilities for the original and slightly nudged instances, it also suits setups where the model is served as a closed endpoint, allowing boundary aware selection without access to internal weights.
There is a cost to weigh: AIIUS trains a stand in network and runs iterative perturbation generation plus parameter tuning, so it is heavier than simple random undersampling. The workflow places clustering first to shrink the candidate pool and keep this overhead manageable, and the published results argue the accuracy and stability gains justify the added compute. The method also appears applicable beyond defect prediction, to any tabular imbalance problem with numerical features and a queryable classifier, such as fault diagnosis, risk scoring, smart contract vulnerability detection, or IoT malware detection, and the authors sketch how a federated or privacy preserving version could work when raw data cannot be shared.
Related reading
Get the paper & cite it
Official citation: Shuo Feng, Rongping Li, Jacky Keung, Xiao Yu, Yucheng Shi, Mingliang Xu (2027). Decision-boundary aware undersampling via Adversarial Influence for imbalanced software defect prediction.. Journal of Systems and Software. DOI: 10.1016/j.jss.2026.113105.