Local scoring still needs teacher judgment
Can smaller, locally hosted language models apply a rubric to student reports? Researchers evaluated four small models on 70 reports produced by 35 teams of three students in an undergraduate cybersecurity course. The work contrasted short executive summaries with longer security advisories containing images, and compared model scores with human reference scores over repeated runs. Performance varied substantially by model, document type and rubric criterion. Some small models approached a much larger baseline on selected tasks, but judging document structure remained difficult. A local model may help with particular scoring tasks while failing on others. Keeping reports on local hardware addresses one data-handling concern; it does not establish fair or defensible grading.
Why it matters for leaders
Test a narrow, low-stakes criterion on held-out work. Examine disagreements and subgroup patterns with teachers. Keep final judgment and appeals human, and budget for local support and review; hosting does not remove those costs.
The evidence
Benchmark of 70 reports from two course offerings, with ten repeated runs and criterion-level exact score agreement. Image captions supported assessment of richer documents; a larger model provided a comparison.
Who was studied
105 undergraduates in 35 teams; cyber incident response reports at Masaryk University, Czech Republic.
What the evidence cannot tell us
One course and imperfect human reference scores limit generalization. Prompts were developed for these tasks. Performance on unfamiliar assignments, adversarial submissions and consequential K-12 grading remains unestablished.
Read the original sources
Summary and leadership analysis from the Week ending 2026-11-06 brief. The edition’s selection notes explain source access and evidence limitations.