AI Agent Frameworks / Scenario
The Graph That Skipped Human Review
A refund agent auto-approved $900 that policy says needs a human. The router did not crash. It did something worse: it failed quietly.
- Difficulty
- Hard
- Format
- Scenario
- Points
- 200
- Estimate
- 15 min
// MISSION BRIEF
Your Mission
Policy: refunds over $200 route to a human_review node before anything is paid out. Last night the graph auto-approved a $900 refund, no crash, no alert. A refactor of the intake node shipped two days ago.
You have the router, the intake diff, and the run's stream trace. Find the silent failure and design the tests that make this class of bug impossible.
// FIRST CONTACT
Battle teaser
First artifact
graph.py (router + wiring)
What is the root cause of the skipped review?
- AThe checkpointer lost the amount between nodes
- BThe LLM hallucinated an approval
- CThe intake refactor renamed amount to amount_usd, and the router's defensive state.get("amount", 0) silently defaulted to 0, so every refund now routes to auto_approve
- DConditional edges are non-deterministic by design
Answers, scoring, hints, and the full battle stay sealed.
// SKILL TAGS
langgraphtrajectory-testingstate-transitionshuman-in-the-loopai-agents