Selenium / Scenario
The 14% Flake Budget
A 300-test suite fails 40-odd tests per night, a different set each time. The team stopped reading the reports. You have one week to make green mean something again.
- Difficulty
- Hard
- Format
- Scenario
- Points
- 200
- Estimate
- 18 min
// MISSION BRIEF
Your Mission
You are handed a nightly Selenium suite with a 14% flake rate and zero trust. Reruns are the current 'fix'. The evidence pack: failure clustering data, two representative traces, and the base test class.
Triage the causes, pick the systemic fixes, and decide what green should mean. Decisions are weighted.
// FIRST CONTACT
Battle teaser
First artifact
Failure clustering, last 20 nightly runs
The dashboard TimeoutExceptions cluster in the first 5 minutes, overlapping the staging deploy. The right response:
- ATriple every wait timeout suite-wide
- BMark dashboard tests @Flaky and rerun them automatically
- CFix the pipeline ordering: gate the suite on deploy completion plus an app health check, this cluster is an environment race, not a test bug
Answers, scoring, hints, and the full battle stay sealed.
// SKILL TAGS
seleniumflakinessparallel-executiontest-dataci