A Closed Loop for Clearer Thinking: How Two LLM Agents Tutor Critical Thinking
Case studies are one of the best ways to teach students to reason through messy real world problems, but a teacher can only read so many essays and grade so many arguments. This work pairs case based learning with two specialised AI agents, one that writes rich realistic scenarios and one that grades each student's reasoning and explains exactly how to sharpen it. In a study with 38 postgraduate financial planning students, AI supported closed loop learning produced a statistically significant gain in domain knowledge and strong critical thinking scores, led by evaluation (3.73 of 4) and analysis.
Case studies are one of the best ways to teach students to reason through messy real world problems, but a teacher can only read so many essays and grade so many arguments. This work pairs case based learning with two specialised AI agents, one that writes rich realistic scenarios and one that grades each student's reasoning and explains exactly how to sharpen it. In a study with 38 postgraduate financial planning students, AI supported closed loop learning produced a statistically significant gain in domain knowledge and strong critical thinking scores, led by evaluation (3.73 of 4) and analysis.
](/news/img/fostering-critical-thinking.JPG)
Why it matters
Critical thinking is the skill employers say they want most and the one universities find hardest to prove they develop. It is also hard to teach at scale, because it only grows through repeated cycles of articulating a judgment, having that judgment examined, and revising it. In a normal course the bottleneck is human attention: a lecturer can give deep, dimension specific feedback to a handful of essays per week, not to a cohort. This work shows a practical way around that bottleneck by splitting the job between two LLM agents with clearly separated roles, one that designs and adapts the cases, and one that scores reasoning against a rubric and returns actionable feedback. The pedagogical authority stays where it belongs. The teacher defines the objectives, reviews the generated cases, and verifies the evaluations.
The headline finding is not just that scores went up, it is where they went up. Students scored highest on evaluation (3.73 out of 4, standard deviation 0.41) and inference (3.39), the exact higher order moves that case based learning is supposed to train: weighing competing options, judging under uncertainty, and drawing defensible conclusions. Participants also gave the experience strikingly high ratings, 4.79 out of 5 for both learning effectiveness and critical thinking development, and 4.72 for engagement. Lower and far more variable scores on explanation (3.18, standard deviation 0.92) pinpoint the next research target: many students can reach a sound decision but are inconsistent at articulating why. That is a concrete, actionable diagnosis for the next generation of LLM pedagogy, and a useful reminder that generative AI in education works best as part of a human AI collaboration rather than as a replacement for the instructor.
For the lab, this paper extends agentic research beyond software engineering into reflective learning environments, showing that the same design discipline used to build reliable agents, explicit objectives, structured tooling, and human oversight, transfers directly to higher education.
Key findings
- Cohort size The mixed methods study involved 38 postgraduate students in a Financial Planning programme.
- Domain knowledge gain FPAT scores rose from 0.77 pre test to 0.82 post test, a statistically significant improvement at t(37) = 2.01, p = 0.027.
- Effect size The knowledge gain carried a small to moderate effect size of Cohen's d = 0.326 over a single 60 minute intervention.
- Best critical thinking dimension Evaluation scored highest across the rubric at 3.73 out of 4 (SD 0.41), showing students could assess alternatives and make informed judgments.
- Overall critical thinking Overall critical thinking performance averaged 3.37 out of 4 (SD 0.71) across five dimensions.
- Strong dimensions Inference reached 3.39 and analysis 3.32 out of 4, confirming that the cases engaged higher order reasoning.
- Growth area Explanation scored lowest at 3.18 with the widest spread (SD 0.92), showing inconsistency in how clearly students justified their reasoning.
- Perceived effectiveness Students rated learning effectiveness 4.79 out of 5 (SD 0.52) and critical thinking development 4.79 out of 5 (SD 0.46).
- Engagement Engagement scored 4.72 out of 5 (SD 0.55) in the post intervention questionnaire.
- Agent architecture Two specialised LLM agents split the work: a Content Generation Agent for scenarios and a Feedback Generation Agent for evaluation.
- Rubric scale The Feedback Generation Agent scores responses on a four point rubric across interpretation, analysis, evaluation, inference and explanation.
- Feedback format Feedback is delivered in three proven formative formats: What Went Well, Even Better If, and Situation Behavior Impact.
- Four phase cycle The framework runs a continuous design, development, implementation and evaluation loop that feeds adaptive follow up cases back to each student.
- Teacher oversight Human review is built into the loop, with the instructor approving generated cases and verifying automated evaluations.
- Adaptive materials The Content Generation Agent produces targeted follow up cases aimed at the specific critical thinking dimensions where each student is weak.
- Student voice Qualitative responses most often named the case based scenarios as the most helpful element for applying theory to realistic decisions.
Evaluation and analysis shine in an AI closed loop
- Evaluation led all five critical thinking dimensions at 3.73 out of 4
- Domain knowledge improved significantly (t(37) = 2.01, p = 0.027)
- Students rated learning effectiveness 4.79 out of 5
What it means for practitioners
If you teach ill structured problem solving, the transferable lesson is to separate generation from evaluation. Let one LLM agent build context rich cases against the objectives you set, and give a second agent a fixed rubric so its feedback is dimension specific instead of generic praise. This study used the five dimension Facione rubric, interpretation, analysis, evaluation, inference and explanation, and asked students to submit written case responses that the Feedback Generation Agent then scored on a four point scale.
Two design choices did most of the heavy lifting. First, keep the human in the loop at the decision points that matter, approving AI generated cases before students see them and spot checking the automated evaluations, which is what protected the pedagogical appropriateness and reliability of the feedback. Second, budget for speed. The most common student request was faster access to correct answers and fuller explanations, and the authors note that delayed or opaque feedback weakens the very reflective loop the design is meant to create. If you adopt this pattern, invest in the turn around time of the feedback step and in showing students the reasoning behind each score, not just the score itself.
Related reading
Get the paper & cite it
Official citation: Hi Kuen Yu, Jacky Keung, Man On Wong, Yicheng Sun, Tianqing Xiao (2026). Fostering Critical Thinking by Integrating Case-Based Learning and Generative AI in a Closed-Loop Reflective Learning Environment. Int. Symp. Educational Technology (ISET). DOI: .