AI_SLANG_ENTRY
LLM-as-a-Judge Meaning
Using one language model to grade, compare, or critique another model's output, which is handy until the judge starts sharing the same blind spots as the contestant.
What does LLM-as-a-Judge mean?
Using one language model to grade, compare, or critique another model's output, which is handy until the judge starts sharing the same blind spots as the contestant.
LLM-as-a-judge means asking a model to evaluate answers instead of only generating them. It is common in AI evals because many good or bad answers cannot be scored with a simple string match.
Origin and usage
Grew from evaluation research and benchmark practice as teams needed scalable ways to score open-ended answers that are hard to check with exact-match tests.
Source type: paper. Last checked: 2026-07-09.
Research and evaluation term covered in survey literature; useful as shorthand for model-based grading, but not a replacement for human review on high-stakes decisions.
Examples
- The eval used LLM-as-a-judge for tone, then spot-checked the failures with humans.
- LLM-as-a-judge is not free truth; it still needs rubrics, calibration, and bias checks.