QABattle

AI Agent Frameworks / Quiz

Datasets, Evaluators, Experiments

evaluate(app, data=dataset, evaluators=[...]) is the unit test of the LLM world, except the assert returns a score and the suite is a dataset.

Difficulty
Medium
Format
Quiz
Points
150
Estimate
10 min

// MISSION BRIEF

Your Mission

LangSmith evals have three parts: a dataset of examples, a target function to test, and evaluators that score each result. Every run of evaluate() becomes an experiment you can compare against the last one. This quiz covers the mechanics.

Scoring: equal weight per question.

// FIRST CONTACT

Battle teaser

A LangSmith dataset is:

  1. AA versioned collection of examples (inputs plus optional reference outputs) that experiments run against
  2. BA SQL database of user accounts
  3. CA fine-tuning corpus that retrains the model
  4. DA cache of previous model responses
Answers, scoring, hints, and the full battle stay sealed.

// SKILL TAGS

langsmithdatasetsevaluatorsexperimentsai-agents