Ask students to demonstrate mastery
As schools reconsider assessment in an AI-rich environment, a programming-course study offers a useful design example rather than a test of AI itself. Researchers compared 203 students' reports of completed tasks with oral demonstrations across a 60-task sequence. Of 11,093 reported tasks, 10,885 were ultimately demonstrated: 98.1%. The remaining 208 mismatches clustered in the hardest late tasks and among a minority of students. However, students could try again, and the records retained final outcomes rather than attempt counts. The high agreement therefore describes eventual demonstration within this system, not accurate first-attempt self-assessment. It suggests where targeted conversations could be useful while leaving the staffing and learning benefits of that approach untested.
Why it matters for leaders
Pilot brief explanations or demonstrations at a few consequential milestones. Use a common rubric, accessible response options and moderation between teachers. Treat the conversation as evidence of understanding and a chance for feedback, rather than assuming an unexplained mismatch proves misconduct. Record the staff time needed.
The evidence
The preprint analyzes 2023 course records and 12 examiners. A task was examined only after a student reported it. This one-sided design measures unresolved overclaiming, not unrecognized competence.
Who was studied
First-year engineering programming, 203 students; assessment design evidence adjacent to AI.
What the evidence cannot tell us
One university course; examiner and student groups were not independently crossed. No AI intervention, cheating-detection test or comparison of assessment costs.
Read the original sources
Summary and leadership analysis from the Week ending 2026-10-02 brief. The edition’s selection notes explain source access and evidence limitations.