loading
loading
Using one LLM to grade another model's outputs instead of writing rule-based checks.
When outputs are open-ended (a summary, a rewrite, an agent's plan), you can't assert-equal against a golden string, so you prompt a strong model with a rubric and ask it to score or pick the better of two answers. It's cheap, scales to thousands of cases, and catches quality nuances regex never will — but judges are biased (they favor longer answers, their own family's style) so you calibrate them against human labels first. Example: feed the judge the question, the answer, and 'rate factual accuracy 1-5, cite the failing sentence.'
Plainly
Think of LLM-as-Judge as a simple recipe for doing the work better. Using one LLM to grade another model's outputs instead of writing rule-based checks.
In practice
Use it when you need a repeatable method instead of guessing from vibes. In practice, define the owner, input, output, and failure mode before you rely on it.