Agentic AI Testing / Scenario
The Trajectory Audit
The agent booked the trip and the customer was happy. Read the actual tool-call trace and decide whether this run should really pass.
- Difficulty
- Hard
- Format
- Scenario
- Points
- 200
- Estimate
- 15 min
// MISSION BRIEF
Your Mission
A travel agent (the AI kind) completed a booking. Outcome eval: pass, the user got a valid itinerary. Your job is to audit the trajectory, the real sequence of tool calls, and decide whether the run is actually healthy or quietly dangerous.
Read the trace, then grade the path.
// FIRST CONTACT
Battle teaser
First artifact
Agent trajectory (tool calls + observations)
Which trajectory problems are present? (Select all that apply.)
- AStep 5 charged the card with no confirmation turn (policy violation)
- BThe final price exceeded $600
- CStep 2 is a redundant duplicate of step 1
- DStep 3 (get_seat_map) was unnecessary for the task
Answers, scoring, hints, and the full battle stay sealed.
// SKILL TAGS
agentictrajectory-evalguardrailstool-callingevaluation