The demo works. You still can't tell if the feature works for its users.

LLM evaluation for developers

Evals, LLM-as-a-judge and regression testing for AI applications, with Python examples and TypeScript notes

The demo went well. What nobody can say yet is whether the feature works for the people using it, or whether it will still work after the next prompt tweak, model swap or change to retrieval.

The book starts from what you already have, unit tests and CI, and shows where they stop helping once a model sits behind the code. From there it works through evaluation on one system. You read traces and name the failures that actually matter, then build a golden set from them. You decide which checks belong in code, which need an LLM judge and which need a person. You design a judge and calibrate it against human labels. A chapter on statistics shows how to tell a real improvement from noise when the set is small. Retrieval is evaluated apart from generation, tool calls and agents get their own chapter, and it all ends up as regression gates in CI, with sampled production traffic and a look at what the whole thing costs.