Test what remains after AI help
A systematic review of university physics research found a varied picture behind broad claims about AI-supported learning. Eleven empirical studies met its criteria, covering tutoring, feedback, simulations, collaboration and teacher preparation. Some reported better immediate or task-specific outcomes. Others showed that apparent advantages did not persist on later examinations, depended on close alignment between homework and test questions, or coexisted with difficulty recognizing inaccurate AI explanations. The authors did not calculate a single pooled effect because designs and outcomes differed substantially. For school leaders, the transferable question is whether students can use what they learned independently and later. Successful completion of an AI-supported activity is only one part of that evaluation.
Why it matters for leaders
For a science pilot, agree on the later assessment before launching. Include a new problem that requires transfer, not a repeat of the supported task. Have subject specialists check generated explanations and record misconceptions. Compare teacher preparation time with any correction burden as well as student performance.
The evidence
The journal review searched six sources and used two reviewers for methodological appraisal. The included designs comprised two randomized studies, four nonrandomized studies, two descriptive studies and three mixed-methods studies.
Who was studied
University physics and preservice physics teachers across six countries; indirect K-12 evidence.
What the evidence cannot tell us
University-only evidence with uneven geographic coverage. Only the first 300 relevance-ranked Google Scholar results were screened; short-term and heterogeneous designs constrain conclusions.
Read the original sources
Summary and leadership analysis from the Week ending 2026-10-02 brief. The edition’s selection notes explain source access and evidence limitations.