Student context can change an AI assessment
Can an AI grader judge the same work differently because of what it knows about the learner? Researchers tested six language models on essay scoring, feedback and English-language questions, keeping the task fixed while varying explicit demographic profiles or prior conversations. Scores and response characteristics changed, but the size and direction varied by model and task. One model's essay scores rose by 1.57 points relative to its no-context baseline when conversation history was supplied. The result identifies a reliability issue: personalization can influence an assessment even when the submitted work is unchanged. Removing demographic fields alone may leave signals embedded in chat history.
Why it matters for leaders
Before adopting AI scoring, ask the assessment lead to compare identical work in a fresh session and a personalized session. Record score changes, feedback differences and teacher judgments. Keep final grades with educators, establish an appeal route, and repeat the check after a model or prompt changes.
The evidence
The experiment sampled 40 tasks from each of two datasets and 200 user profiles. The authors report acceptance for EMNLP Findings.
Who was studied
Essay and English-question datasets; no classroom intervention.
What the evidence cannot tell us
Controlled model tests do not measure discrimination in schools. Readability and sentiment changes do not establish educational harm; effects depend on models and prompts.
Read the original sources
Summary and leadership analysis from the Week ending 2026-09-18 brief. The edition’s selection notes explain source access and evidence limitations.