What it is for
Inspect was created at the UK AI Security Institute. The README describes it as a framework for large language model evaluations with built-in components: support for prompt engineering, tool use, multi-turn dialog and model-graded evaluations. The documentation lists building blocks (datasets, agents, tools, scorers), a collection of over two hundred ready evaluations, a results viewer, a VS Code extension and sandboxes for running untrusted code.
How to start
Install the package with pip, then go through the tutorial on the documentation site, which shows simple evaluations: benchmark questions and a CTF-style task for an agent. The documentation also provides an index in the llms.txt format for coding agents. Start with your own twenty cases, as in the article How to evaluate an agent before you trust it.
pip install inspect-aiWhat to watch out for
A framework will not replace a good set of cases: a result on a ready benchmark says something about the model, not about your application. Run agents in a sandbox, because they execute code. A comparison with other approaches is in the text Inspect, lm-evaluation-harness or your own script.
Further reading
More in the article How to evaluate an agent before you trust it and in the Harness and evals hub. You will find the other repositories on the repository list. Star and release data come from the freshest live data, and the description from the README as of 20 October 2026.