QABattle

AI & LLM Evals / Quiz

Choosing the Right Eval Metric

The wrong metric certifies the wrong thing. Match the metric to the task, or your green dashboard is measuring the incorrect quantity.

Difficulty
Medium
Format
Quiz
Points
150
Estimate
10 min

// MISSION BRIEF

Your Mission

There is no universal LLM metric. A summarizer, a classifier, a RAG bot, and a code generator each need different measures. This quiz drills metric selection, and the traps of using a metric that does not fit the task.

Scoring: equal weight per question.

// FIRST CONTACT

Battle teaser

For a free-form summarization feature, why is exact-match against a reference summary a poor primary metric?

  1. AExact-match is always best
  2. BExact-match is too slow
  3. CMany different phrasings are correct; exact-match punishes valid summaries that differ in wording
  4. DSummaries have no reference
Answers, scoring, hints, and the full battle stay sealed.

// SKILL TAGS

ai-evalsmetricsmetric-selectionragclassification