In short
- An evaluation (eval) is a set of tests that checks whether a model-based application works well, before you put it to work and after every change.
Anthropic’s documentation on evaluation begins with success criteria, which should be specific, measurable, achievable and relevant. Then you build test cases, including edge ones, and grade the results with rules, another model against a rubric, or a human. It is worth keeping some cases as a held-out set, to check that fixes work in general and not only on examples you know.
Tools
To begin with a spreadsheet and a script with twenty examples are enough. When sets multiply or you evaluate agents with tools, frameworks such as Inspect from the UK AI Security Institute help.
Where to read next
How to evaluate an agent before you trust it and Inspect, lm-evaluation-harness or your own script. The concept hallucination. The whole topic: the Harness and evals hub.