loading
loading
A test that measures how well an AI model or agent performs on a defined task.
Evals range from simple exact-match checks on model outputs to complex LLM-as-judge pipelines that score nuanced qualities like helpfulness or safety. Running evals in CI lets you catch regressions when you change your prompt, swap models, or update retrieval logic. Good evals are the difference between shipping with confidence and guessing.
Plainly
Think of Eval as the brain part that guesses or decides. A test that measures how well an AI model or agent performs on a defined task.
In practice
Use it when model choice, prompts, latency, cost, or quality affect the product result. In practice, define the owner, input, output, and failure mode before you rely on it.