AI for QA / Scenario
Prove Which RAG Stage Failed
Use source, chunks, rankings, and controlled context injection to localize a wrong answer.
- Difficulty
- Hard
- Format
- Scenario
- Points
- 200
- Estimate
- 20 min
// MISSION BRIEF
Your Mission
A support RAG answer cites plausible authentication material but states the wrong lockout duration. Trace the source through chunking and retrieval, then design counterfactual runs that distinguish representation, ranking, and generation defects.
// FIRST CONTACT
Battle teaser
First artifact
Canonical source P-9
What failure is best supported by all three runs?
- ACitation ids prove all answers are correct
- BChunking caused a retrieval miss; generation followed the supplied distractor
- CConclude that generation ignores every provided chunk, even though it follows the direct P-9 context in Run B and the repaired context in Run C
- DThe canonical source itself says 30 minutes
Answers, scoring, hints, and the full battle stay sealed.
// SKILL TAGS
rag-builderretrieval-triagechunkingevaluationai-for-qa