- Success criteria written before the run protect against fitting the judgment to the result.
- A good criterion is specific, measurable and about your task, not about general model ability.
- A ranking says something about a model, your own example set says something about your application.
When a model answers three questions well, it is easy to believe it works. This essay defends a simple discipline: first write down what “good” means, then run. It is a boring order, but it protects against the most common evaluation mistake, which is fitting the criterion to a result I have already seen.
Why order matters
If you judge after the run, every result can be defended. An answer that is pretty but incomplete seems “almost good”. An answer wrong in a detail looks like a “minor flaw”. Anthropic’s documentation on defining success criteria describes them as specific, measurable and achievable, about the task and not general slogans. Writing them down before the run means the judgment no longer depends on mood.
What a good criterion looks like
A bad criterion says “answers should be good”. A good one says: “at least eighteen of twenty questions get an answer consistent with the documentation, and none invents a feature that does not exist”. In the first you cannot lose; in the second you can both win and lose, and that is why it has value. For tasks without one correct answer, the criterion can concern features: length, included elements, tone. It helps to also write down what is unacceptable, for example invented data or a link that does not exist.
A ranking is not your task
Results on public sets show how a model handles those tasks. They do not say how it will handle your customer questions, your invoice format or the style you consider your own. That is why a set of twenty examples of my own matters more to me than a position in a ranking. When the set grows, a tool is worth using: the Inspect framework from the UK AI Security Institute provides ready building blocks for datasets, agents and scorers. In the article Inspect, lm-evaluation-harness or your own script I compare these routes, and in the text How to evaluate an agent before you trust it I show the shortest one.
What I do when a test comes out poorly
First I check the test, because mistakes happen in the questions and the expected answers too. Then I change one thing: the prompt, the context or the model. I run the same set and compare. I do not change the criteria to make the result look better; if a criterion turned out to be badly worded, I write down the corrected version with a date and run everything again.
Why this is a matter of calm
My rule is that technology should give time back. A test written once saves hours of arguing over whether a new model is better. In a month the next one will appear, and I will run the set and know within minutes. Without a set, only impressions remain, which is the most expensive method of evaluation.