Tom Loosemore

If you’re making an AI-powered thing for real, then start by creating a comprehensive suite of evaluation test cases (aka an ‘eval suite’) with which you can continually evaluate the quality of your LLMs’ outputs. A typical single eval test for an LLM might consist of a very detailed description of a user’s circumstances, together with a description of the outcome the LLM should be providing for that specific user. An eval suite can have hundreds or even thousands of these individual eval tests.

Creating and then applying such eval suites can be a much harder step than you might initially assume. The reasons have nothing to do with AI, and everything to do with the historic lack of professional service ownership inside many large organisations.

If you can’t define and agree what a service is for, then adding LLMs just adds non-deterministic chaos. Many services have never defined success rigorously, in terms of outcomes. Those currently running them can misunderstand, forgot, or have little interest in, the original purpose of their service. For our users, what should be different because this service exists?

You need clarity on and alignment around the purpose of a service if you’re going to incorporate non-deterministic LLMs. Retro-fitting eval suite-ready outcomes to a long-running service often requires highly-charged, highly-political decisions about what the service is actually for, now. If that proves impossible, stop

Creating an eval suite might have helped surface one of the terrible disconnects that wrecked the early years of Universal Credit. The Minister saw the purpose of UC as getting more people into more work, more of the time, while supporting those who couldn’t work. Most of DWP saw it as a more efficient way to pay people their benefits. No service can succeed against such fundamental misalignment of purpose, let alone one powered by non-deterministic LLMs.

Agreeing on what the eval suite should be evaluating against is only half the battle. What often follows is a hazardous debate about what threshold an LLM should reach before it is deemed good enough to go live. Avoiding two straw men are your initial hazards. The desire for 100% success against the eval suite tests, and a desire for the eval suite itself to be 100% comprehensive in the scenarios it tests against. A more realistic threshold is “do these new LLMs do better against the eval suite tests that our current (non-LLM) service? But that assumes those in charge of the current non-LLM service are willing to have it evaluated with the rigour of an eval suite. Some just won’t risk the realities this exposes

Finally, you need an eval suite because LLMs change (‘drift‘) over time. It’s not just that better versions arrive every week; even the results of the same LLMs can drift over time. So you need to be continuously running your eval suite to run LLMs safely. Let’s hope the business case included the required opex…

[Addendum: a friend with much more experience using AI eval suites sent me this correction]

95% agree on the evals post. But! “Even the results of the same LLMs can drift over time” is a common misconception – sometimes a hosted LLM will have its system prompt silently changed without incrementing the version number (see also Grok), or fixing bugs in hosting (see Claude a while back), but that’s a rare and increasingly uncommon phenomenon. Hence, saying that the output of LLMs drift over time is confusing and misleading for people who don’t know or care about the difference between the LLM (frozen weights plus hosting) and the API (LLM plus system prompt and classifiers).