Evaluating AI systems: replacing spot checks with a measured baseline

Ask most teams how they know their AI system is working and the answer is some version of "we looked at some outputs". That is not measurement. It does not tell you whether the last change improved things, whether a provider model update quietly changed behaviour, or whether quality is drifting as the corpus ages.

Build the test set first

A maintained test set is a collection of inputs with expected characteristics of a good output — not exact strings, since outputs vary, but criteria: must cite a source, must not mention X, must recommend escalation for Y. Start with fifty to a hundred cases drawn from real usage, weighted toward the failure modes that matter most, and grow it every time something goes wrong in production.

Score with rubrics, calibrate with humans

Scoring can be automated using a model as a judge, but only after calibrating that judge against human raters on a sample. Where the model-judge and humans disagree, the rubric is unclear and needs rewriting. Keep the human calibration sample refreshed.

Gate deployment on the result

Wire the evaluation into the deployment pipeline so that a change which drops the score below threshold cannot promote. This is the point at which quality stops being an opinion and becomes a control — and it is what lets you make changes with confidence rather than fear.

Monitor in production

Run the evaluation suite on a schedule against the production system, not just at deployment, because the world changes: the corpus is updated, the provider changes the model, user behaviour shifts. A regression that shows up in the weekly run is a task; one discovered by a user is an incident.