API Testing / Scenario
The Retry Storm That Took Everything Down
A slow dependency plus infinite timeouts plus blind retries equals a cascading outage. Diagnose it and design the resilience.
- Difficulty
- Boss
- Format
- Scenario
- Points
- 300
- Estimate
- 20 min
// MISSION BRIEF
Your Mission
Service A calls payment service B. B got slow. A had no client timeout and retried every failure immediately, including a non-idempotent charge. Threads exhausted, A fell over, and the retries made B's bad day worse.
This is a boss fight in resilience: find the cause, stop the double-charges, and specify timeouts, backoff, and a circuit breaker that hold under pressure.
// FIRST CONTACT
Battle teaser
First artifact
Incident timeline
The primary cause of A's OUTAGE (beyond B being slow):
- AA DNS misconfiguration
- BNo client timeout plus immediate retries against a slow dependency exhausted A's threads/connections, a cascading failure amplified by retries
- CA did not have enough servers
- DB being slow is the only cause; A did nothing wrong
Answers, scoring, hints, and the full battle stay sealed.
// SKILL TAGS
apitimeoutsretriesresiliencecircuit-breaker