QABattle

AI for QA / Scenario

The Risk in AI Regression Selection

An AI picks a 40-test subset to run on each PR. Fast, until you see what it decided to skip.

Difficulty
Hard
Format
Scenario
Points
200
Estimate
15 min

// MISSION BRIEF

Your Mission

To speed up PR feedback, a tool uses AI to select which subset of a 3,000-test suite to run per change, based on the diff. It is fast and mostly right. Your job is to find where 'mostly right' becomes a shipped incident, and how to bound the risk.

// FIRST CONTACT

Battle teaser

First artifact

How the selector works

The core risk in this PR's selection is:

  1. AThere is no risk; utils tests cover everything
  2. BThe nightly full run is unnecessary
  3. CA shared utility has wide blast radius; mapping 'related' by edited files misses the downstream suites (checkout, refunds, tax, payouts) that actually depend on it
  4. D12 tests is too many to run
Answers, scoring, hints, and the full battle stay sealed.

// SKILL TAGS

ai-workflowsregressionriskscenarioai-for-qa