AI That Teaches Instead of Fixes: A Three Level Feedback Framework for Learning to Code
When a student's code fails, most AI assistants hand over a corrected solution and the student never learns the concept. This research builds a tutoring framework that first diagnoses the real misunderstanding, retrieves the exact concept from a curated knowledge base, then delivers feedback in three steps from a gentle hint to a targeted repair, so learners keep their role in the fix.
When a student's code fails, most AI assistants hand over a corrected solution and the student never learns the concept. This research builds a tutoring framework that first diagnoses the real misunderstanding, retrieves the exact concept from a curated knowledge base, then delivers feedback in three steps from a gentle hint to a targeted repair, so learners keep their role in the fix.
Why it matters
Learning to program is hard precisely because a wrong answer hides a wrong mental model. A student who gets a corrected solution back learns nothing except that their answer was wrong; the gap in their understanding stays untouched. This work shifts the goal of AI support in programming education away from producing a correct answer and toward helping a learner actually understand why their code fails and how to revise it, which is the part that transfers to the next problem.
The framework solves a practical problem with today's LLM assistants: they are fluent but often not pedagogical. By separating the work into a diagnosis stage, a knowledge retrieval stage, and a staged feedback stage, the system stays honest and inspectable, and it can point at a specific teachable concept such as a sliding window invariant or a dynamic programming transition instead of guessing at generic advice. It keeps the tutor lightweight while still grounding every piece of advice in real instructional material.
The headline is that this approach is more educational without giving away too much accuracy. On a curated set it is far stronger on concept alignment and pedagogical usefulness than a direct LLM baseline, while still preserving most of the second attempt correction power that a pure repair prompt gets. That is the ideal middle ground for a classroom tool: guidance that teaches rather than a solver that replaces the learner.
Key findings
- Curated evaluation set The team built a curated set of 30 LeetCode style programming tasks covering 10 major algorithm categories such as hash maps, sliding windows, linked lists, backtracking, trees, graphs, and dynamic programming.
- Erroneous submissions Each of the 30 tasks was paired with exactly five flawed solutions, producing 150 erroneous submission cases for the evaluation.
- Three stage pipeline The framework works in three connected stages: pedagogical diagnosis of the misconception, knowledge grounded retrieval from a curated concept base, then hierarchical feedback.
- Three levels of feedback Feedback is delivered in three rising levels of explicitness as a hint, a concept explanation, and a targeted repair suggestion, avoiding full solution disclosure by default.
- Concept alignment gain On concept alignment the framework scored 4.20 of 5, a large improvement over a direct LLM baseline at 3.04 and a direct fix baseline at 2.84.
- Pedagogical usefulness gain On pedagogical usefulness it scored 4.28 of 5, ahead of the direct LLM baseline at 3.25 and the repair oriented baseline at 3.64.
- Second attempt correction The framework still supported successful second attempt correction in about 70.4 percent of cases, close to the 79.3 percent of a pure repair prompt while being far more educational.
- Common foundation model All methods including the baselines were powered by the same model, GPT 5.4, so the differences came from scaffolding and retrieval design rather than the underlying model.
- Reviewer validated dataset Three computer science graduate research students reviewed every case for plausibility and concept alignment, and used the same rubric to independently score the feedback quality.
Teach the concept, not just the fix
- Three stage pipeline links a wrong submission to the exact misconception before giving any advice
- Curated knowledge base stores the concept, invariant, common bug, and a ladder of hints for each problem type
- Staged hints, concept explanations, and repairs let feedback guide without replacing the learner
- LLM driven diagnosis, retrieval, and generation is inspectable and auditable at every step
What it means for practitioners
For educators and tool builders, this is a practical blueprint for making LLM based help genuinely instructional rather than answer giving. The key design choice is to insert a diagnostic stage and a concept retrieval stage before generation, so every piece of feedback is anchored to a named teachable idea such as a boundary condition mistake, a pointer update error, or a missing dynamic programming transition.
The three level output is easy to deploy in a real course: start learners with a hint, allow them to open a concept explanation, and only offer a targeted repair if they remain stuck. That graduated structure protects the learner's own reasoning, which is exactly what a classroom tutor does. Because the whole pipeline runs on one LLM with a compact curated knowledge base, it stays lightweight enough to fit into practice or assessment tooling.
Related reading
Get the paper & cite it
Official citation: Yuchen Cao, Yifei Wang, Jacky Keung, Jingyu Zhang (2026). From Fixes to Learning: Knowledge-Augmented Hierarchical Feedback for Programming Practice. Int. Symp. Educational Technology (ISET). DOI: .