LLM-as-judge uses a capable LLM to score other LLM outputs — cheaper than human annotation at scale.

LLM-as-judge uses a capable LLM to score other LLM outputs — cheaper than human annotation at scale. Risks: self-serving bias (models prefer their own style), position bias (prefers earlier responses), verbosity bias (rewards longer answers). Mitigations: use a different model as judge, swap option order and average scores, define explicit rubrics, calibrate against human labels.