Graduates Rate Themselves 4.45 for Job Readiness. Their Employers Say 3.14.
The ASI Lab asked 21 new software engineering graduates and 5 senior engineers to rate how ready those graduates were for real industry work. Graduates gave themselves 4.45 out of 5 on industry readiness, while the engineers who actually hired and worked with new graduates scored their performance at 3.14. The study traces where that confidence gap comes from, how online learning widened it for some students, and why heavy reliance on large language models can boost coding speed while quietly weakening deeper problem solving.
The ASI Lab asked 21 new software engineering graduates and 5 senior engineers to rate how ready those graduates were for real industry work. Graduates gave themselves 4.45 out of 5 on industry readiness, while the engineers who actually hired and worked with new graduates scored their performance at 3.14. The study traces where that confidence gap comes from, how online learning widened it for some students, and why heavy reliance on large language models can boost coding speed while quietly weakening deeper problem solving.
![![Table comparing Likert scores from graduates and senior engineers]](/news/img/expectations-diverge.JPG)
Why it matters
Software engineering graduates are entering the workforce in record numbers, and employers are increasingly impatient about how long it takes a new hire to become productive. This paper puts a number on the problem for the first time from both sides at once: it collects the graduates' own view and the view of the senior engineers who work beside them, using the same measuring stick. That dual perspective turns a vague complaint about the skills gap into a precise map of where expectations diverge, which is exactly what curriculum designers need in order to act.
The study also arrives at a moment when two forces are reshaping software engineering education at the same time. Online learning, accelerated by the pandemic, removed much of the real time debugging help and team collaboration that programming courses depend on. At the same time, large language models have become everyday tools in student coursework. The paper works through the evidence on both fronts, including a controlled comparison of 25 programming problems against historical student scores from 2018 to 2020, and comes out with a balanced and surprisingly constructive message: AI tools help with code completion, debugging and speeding up routine work, but the core skills of independent design and critical thinking still have to be built the old way, through practice.
For the software industry skills debate, the takeaway is encouraging rather than gloomy. Graduates in this study were strong on theory, algorithms and communication, and engineers agreed they are quick learners. The reportable gap is not ability but exposure: large codebases, Agile rituals, code review culture, and the everyday toolchain of Docker, Kubernetes and CI/CD. Every one of those can be taught inside a course if the curriculum is designed around real projects instead of isolated exercises.
Key findings
- Participants The study combined 26 participants: 21 recent software engineering graduates aged 22 to 29 and 5 senior engineers aged 32 to 50, drawn from China, the USA, the UK, Canada and Australia.
- Readiness gap Graduates rated their own industry readiness at 4.45 out of 5, while senior engineers rated graduates' overall performance at 3.14 and their professional knowledge at 3.16 on the same 5 point scale.
- Tools gap Graduates rated their familiarity with industry tools at 3.98, but engineers rated graduates' technical skill for real development tasks at only 3.69, and all five engineers said new hires lacked hands on experience with standard tooling.
- Learning from feedback data The analysis drew on 4 course assessment reports from universities in China, Australia and the USA, covering 42,800 individual student feedback entries and 4,978 distinct faculty comments.
- Online learning sentiment Of 3,961 comments about online versus offline learning, positive sentiment rose from 60.33 percent at bachelor level to 68.62 percent at master level and 71.65 percent at doctoral level, while negative sentiment was highest at bachelor level at 13.22 percent.
- Practical exposure 16 of the 21 graduates said they grasped theory well but had too little experience with large scale software projects, and 12 of 21 said the technologies they learned in class were outdated or insufficient for industry work.
- Debugging and legacy code Four of five senior engineers said graduates write strong algorithms and new code, but struggle to debug unfamiliar codebases, read legacy systems and optimize at scale, gaps that can add roughly six months of training.
- LLM use 19 of 21 graduates reported frequent use of tools such as ChatGPT and Copilot for explanations, code generation, debugging and testing, and on a 25 question programming assessment they outperformed 2018 to 2020 historical students by roughly 7 to 17 percent.
- Where the AI gain is weakest The LLM advantage was smallest on conceptual understanding and open ended design questions, where instructors observed that heavy users produced similar, formulaic answers and showed weaker grasp of design trade offs.
- Career paths A 2024 survey of graduates from seven Chinese universities found that over 80 percent entered front end, back end or software testing roles, with only 12.5 percent leaving software work entirely.
Two Sides, One Scale
- Graduates rate their own industry readiness at 4.45 out of 5
- Senior engineers rate the same graduates' performance at 3.14
- A 1.31 point credibility gap sits at the heart of the skills mismatch
- LLM assisted graduates scored 7 to 17 percent higher than pre 2020 peers, with the smallest gains on open design tasks
What it means for practitioners
For engineering managers and tech leads, the message is that new hires are not short on ability, they are short on shared context. Engineers in this study described new graduates as fast learners with excellent algorithmic skills who simply had never seen a real codebase, a sprint board or a pull request review. Pairing each new hire with a mentor for the first sprints, and assigning a small refactoring task in an existing repository during week one, addresses the exact gap the paper measures. It also replaces the roughly half year of informal ramp up that engineers reported with structured progress.
For educators and curriculum owners, the strongest signal is that students themselves are asking for real projects and professional tools. When the research team mined 42,800 student feedback entries with TF IDF and LDA topic modeling, the top ranked terms were industry tools, real world projects and coding practice. The paper's recommendation follows directly: teach against a real existing codebase, run genuine code reviews in class, and make continuous integration and containerization part of the coursework rather than slides. On LLMs, the balanced route works best: use them openly in class but assess some work without them, especially architecture and algorithm design, so the skill of independent judgment is measured and protected.
Related reading
Get the paper & cite it
Official citation: Yicheng Sun, Jacky W. Keung, Hi Kuen Yu, Yihan Liao, Yishu Li, Xiaoxue Ma (2026). Where Do Expectations Diverge? an Empirical Analysis of Software Engineering Graduate Preparedness in Industry.. COMPSAC. DOI: 10.1109/COMPSAC69091.2026.00213.