The Sethi Ledger

People · Practice · Possibility

Independent research for education leaders

Preprint; within-participant experiment

A tutor should adjust help to the learner

How can an AI tutor help without taking over? CoMeT increases assistance when a learner struggles with a specific decision, then reduces it when the learner responds. In a study of 131 adults completing three Python tasks, researchers compared it with an unrestricted assistant and a question-only tutor. CoMeT prompted more reported effort than the assistant and less frustration than the question-only design. It surrendered the complete answer in 6.1% of sessions, compared with 16.0% for the question-only tutor. These are promising interaction results, not proof of stronger learning: the analysis included only participants who completed all tasks, in one sitting.

Why it matters for leaders

Editorial recommendation: Add a demonstration to tutor procurement. Ask a vendor to show what happens when a student makes a plausible mistake, repeats it, asks for the answer and then explains a correction. Require a teacher-readable record of the support given. In a pilot, observe whether students can transfer the idea without the tool; completion rates alone cannot answer that question.

The evidence

Each participant used all three tutors; tutor order was balanced. Shared model and task infrastructure helped isolate the support design.

Who was studied

131 adult learners, 393 task sessions; indirect K-12 evidence.

What the evidence cannot tell us

Preprint, adult programming context and completer-only analysis. No delayed unaided assessment establishes retention; findings do not validate a school-age deployment.

Read the original sources

Summary and leadership analysis from the Week ending 2026-09-25 brief. The edition’s selection notes explain source access and evidence limitations.

Download the complete edition

Continue reading

← Back to this edition