A brief reflection reduced acceptance of incorrect AI advice
Students accepted incorrect AI recommendations on 62.4% of relevant trials with open ChatGPT support, compared with 39.7% when a reflection prompt preceded their final answer. Reflection also improved awareness of reliance.
Why it matters for leaders
Pilot a short reasoning checkpoint: record an initial answer, identify evidence for changing it, and explain any disagreement with AI. Evaluate whether students can perform the task independently afterward.
The evidence
A randomized three-condition experiment analyzed 342 undergraduates at one Chinese university. Participants recorded initial answers, consulted AI where assigned, and submitted final judgments. The incorrect-advice comparison reported an odds ratio of 0.40, with a 95% confidence interval of 0.28-0.56.
Who was studied
University students; indirect implications for secondary-school instructional design.
What the evidence cannot tell us
The study measured immediate decisions, not durable learning or children's outcomes. The proposed mediation mechanism was exploratory and cannot establish causality; the calibration measure partly overlaps reliance.
Read the original sources
Summary and leadership analysis from the Week ending 2026-09-18 brief. The edition’s selection notes explain source access and evidence limitations.