The Sethi Ledger

People · Practice · Possibility

Independent research for education leaders

Research preprint | formative-assessment pilot

Instant feedback still needs an accuracy audit

Edhivo explores whether AI can shorten the gap between a classroom check and useful feedback. In a semester pilot with 36 university students, instructors used generated questions and rubric-based responses for brief formative cycles. Materials were reviewed by the instructor before publication, and these activities did not determine course grades. Thirty students answered the follow-up survey; all agreed that the interface was easy to use and that immediate feedback improved their learning experience. Those responses support usability and perceived usefulness, not measured achievement. The report’s most consequential unfinished task is agreement testing between AI and human grading. A rapid response becomes educationally useful only if its interpretation of the student’s answer is sound.

Why it matters for leaders

Keep an initial pilot formative and sample both accepted and rejected answers for teacher audit. Include unconventional but correct explanations and varied language backgrounds. Measure the full review-and-correction workload before considering any higher-stakes use.

The evidence

The platform combines question generation and feedback models. Reported time savings rely on informal instructor estimates; larger-cohort efficiency figures are projections rather than observed deployment results.

Who was studied

Stony Brook University course; 36 enrolled students and 30 survey responses.

What the evidence cannot tell us

One small course, self-reported experience and no demonstrated causal learning effect. Human–AI grading agreement remains incomplete, and linguistic fairness requires evaluation.

Read the original sources

Summary and leadership analysis from the Week ending 2026-10-30 brief. The edition’s selection notes explain source access and evidence limitations.

Download the complete edition
← Back to this edition