AI & LLM Evals / Quiz
DeepEval: Unit Tests for LLMs
LLMTestCase, assert_test, thresholds, and the deepeval test run CLI: turning 'the answer feels right' into a failing or passing pytest test.
- Difficulty
- Easy
- Format
- Quiz
- Points
- 100
- Estimate
- 8 min
// MISSION BRIEF
Your Mission
DeepEval's pitch is simple: evaluate LLM outputs with the same workflow you already use for code, pytest. You build an LLMTestCase, attach metrics with thresholds, and assert_test makes quality a red/green test result.
This quiz covers the core authoring API: test cases, metric thresholds, and how the CLI wraps pytest. Speed bonus active.
// FIRST CONTACT
Battle teaser
In DeepEval, an LLMTestCase is:
- AA YAML file describing a conversation
- BA subclass of unittest.TestCase that calls the OpenAI API
- CA container for one interaction: input, actual_output, and optionally expected_output, retrieval_context, or context, the unit that metrics evaluate
- DA wrapper that retries flaky LLM calls
Answers, scoring, hints, and the full battle stay sealed.
// SKILL TAGS
deepevalpytestllm-evalsmetrics