AI & LLM Evals / Quiz
GEval: Write Your Own Judge
Custom criteria, evaluation_params, and chain-of-thought scoring: building a judge for the quality dimension no built-in metric covers.
- Difficulty
- Medium
- Format
- Quiz
- Points
- 150
- Estimate
- 9 min
// MISSION BRIEF
Your Mission
Built-in metrics cover relevancy and faithfulness. But "matches our support tone", "never promises refunds", "explains at a 10-year-old's level"? That is GEval territory: you describe the criteria in plain language, and DeepEval builds an LLM judge around it.
Six questions on authoring GEval metrics that actually measure what you meant. Speed bonus active.
// FIRST CONTACT
Battle teaser
GEval is:
- AA linting rule for prompt files
- BA custom metric you define with natural-language criteria, scored by an LLM judge using chain-of-thought evaluation steps (per the G-Eval paper)
- CA faster, non-LLM version of exact match
- DA Google-hosted evaluation service
Answers, scoring, hints, and the full battle stay sealed.
// SKILL TAGS
deepevalgevalllm-as-judgecustom-metrics