Journal Article

Beyond Large Language Models: On the relative value of active learning for code smell detection

Haoxuan Chen, Chenxuan Hu, Chengming Liu, Jacky Wai Keung, Xiaoxue Ma, Wenhua Hu, Chuanxiang Ma
Information and Software Technology 2026 10.1016/j.infsof.2026.108302

Abstract

Context: Machine learning and deep learning approaches for code smell detection typically rely on large-scale labeled datasets. In practice, labeled code smell resources are scarce, while unlabeled code is abundant, making active learning a promising solution to alleviate the data scarcity bottleneck. Objective: To systematically investigate the impact of active learning (query strategies, classifiers, query rates) on code smell detection and to examine whether the findings are sensitive to the choice of pretrained code representation model. Method: A comprehensive empirical study on the MLCQ dataset (4365 code snippets covering four common code smell types), using CodeBERT for semantic features, 5 families of active learning query strategies (11 variants), 8 classifiers, and 10 query rates (5%-50%), with stratified 10-fold cross-validation, four metrics, and the Scott-Knott ESD test. Results: Uncertainty- and committee-based query strategies outperform alternatives (Vote Entropy especially strong); Random Forest achieves the best overall results; increasing the query rate from 5% to 30% yields pronounced gains, while beyond 30% gains diminish or degrade; the optimal configuration (Random Forest + Vote Entropy) outperforms evaluated prompt-based LLM baselines; and CodeBERT remains the best encoder in terms of F1 and MCC. Conclusion: Active learning effectively enhances code smell detection, and practitioners should prioritize pairing CodeBERT feature extraction with Random Forest alongside Vote Entropy, capping the query rate at 30% to balance accuracy and labeling cost.

Keywords