Latest Findings · ASI Lab

When Does Parameter-Efficient Fine-Tuning Beat Full Fine-Tuning in Code-Change Learning?

A systematic study across four PEFT methods and seven code-language models — and PastaK, a probing-guided framework that trains a fraction of the parameters and still wins.

Shuo Liu, Jacky W. Keung, Zhi Jin, Zhen Yang, Fang Liu, Hao Zhang
Agentic Software Intelligence Research Lab · Department of Computer Science, City University of Hong Kong
IEEE TSE · 2026

The first systematic comparison of PEFT (LoRA, adapters, prompt tuning) against full-model fine-tuning across seven code LMs on real code-change tasks — plus PastaK, a probing-guided framework that trains only a fraction of the parameters and still beats full fine-tuning.

An Empirical Study of Parameter-Efficient Fine-Tuning in Code Change Learning and Beyond
Code comprehension tasks come in two flavours: static tasks read a single snippet, code-change tasks read a diff. This study is the first large-scale PEFT evaluation on the dynamic ones.

Why it matters

We examine four prevalent PEFT methods and compare them with full-model fine-tuning across seven PLMs on code-change-related tasks.

Key findings

PastaK

What it means for practitioners

For classification-style code-change tasks and cross-lingual/low-resource settings, PEFT gives better accuracy at a fraction of the cost and should replace FMFT; for demanding generative tasks, use AT or LoRA when compute is limited, and consider PastaK to get full-model quality without the full-model bill.

Get the paper & cite it

Download full PDF ↓ View at DOI

Official citation: Shuo Liu, Jacky W. Keung, Zhi Jin, Zhen Yang, Fang Liu, Hao Zhang (2026). An Empirical Study of Parameter-Efficient Fine-Tuning in Code Change Learning and Beyond. IEEE Transactions on Software Engineering. DOI: 10.1109/TSE.2025.3637335.

Parameter-Efficient Fine-Tuning PEFT LoRA PastaK Code change learning Just-in-Time Defect Prediction Commit Message Generation Code Language Models