Check learning after the tool closes
Can general-purpose AI improve what students remember after assistance ends? In an experiment with 211 undergraduates, students studied an unfamiliar technical topic and wrote an essay within a fixed period, with or without permission to use AI. All knowledge tests were unaided. The October 4 revision reports a 7.2-percentage-point immediate test advantage for AI access. A week later, the estimated advantage was 3.9 points, but that delayed result did not meet the conventional 5% significance threshold. Students using AI for explanations showed more promising retention patterns than those delegating writing, although those usage choices were not randomized. The study supports testing retained understanding while leaving the durability and best mode of assistance unsettled.
Why it matters for leaders
Build a brief unaided follow-up into AI pilots, using a new question on the same concept. Compare both immediate work quality and later understanding. Teach students to request explanations, then require their own reasoning; evaluate that routine rather than treating these subgroup patterns as proof.
The evidence
Proctored sessions and conversation logs tracked access and use. Immediate p=.021; delayed p=.068. The revised manuscript changes estimates and expands the analysis of AI-use patterns.
Who was studied
Adult college students; indirect evidence for K-12 policy and assessment design.
What the evidence cannot tell us
One undergraduate setting and short follow-up. Self-selected tutoring versus drafting groups cannot identify the causal effect of a usage strategy.
Read the original sources
Summary and leadership analysis from the Week ending 2026-10-09 brief. The edition’s selection notes explain source access and evidence limitations.