The Sethi Ledger

People · Practice · Possibility

Independent research for education leaders

Preprint: randomized pilot with descriptive analysis

A restricted study coach did not show a performance advantage

A small programming-course pilot reported similar assignment performance with a scaffolded AI coach and unrestricted tools. Coach users reported greater confidence, but some exhausted hints early or saved them without a plan.

Why it matters for leaders

Teach and observe how students decide when to ask for help. Include reasoning and tool-use decisions in assessment rather than assuming a restricted interface automatically produces self-regulation.

The evidence

Thirty-three master's students were randomly assigned to groups of 16 and 17 for seven weeks. The coach constrained hints and AI time and prompted reflection. Analyses were descriptive: the paper explicitly reports no inferential tests.

Who was studied

Master's-level data analytics students; indirect K-12 relevance.

What the evidence cannot tell us

This September 15 preprint is a single-course pilot. Similar descriptive scores are not statistical equivalence. Prior literacy and self-chosen strategies were not randomized; their associations with performance cannot establish an instructional effect.

Read the original sources

Summary and leadership analysis from the Week ending 2026-09-18 brief. The edition’s selection notes explain source access and evidence limitations.

Download the complete edition

Continue reading

← Back to this edition