QABattle

Mobile Testing / Scenario

Green on Emulator, Red in the Wild

CI is a wall of green Pixel emulators. The beta channel is a wall of Samsung crash reports. Somebody's test strategy has a blind spot the size of an OEM.

Difficulty
Medium
Format
Scenario
Points
150
Estimate
13 min

// MISSION BRIEF

Your Mission

Your nightly suite passes 100% on Pixel emulators, API 34. Meanwhile beta users on real Samsung and Xiaomi hardware are filing crashes your pipeline has never seen.

Read the CI summary and the field evidence, then make the strategy calls: why the suite is blind, what enters the pipeline, and how the release gate changes. Decisions are weighted.

// FIRST CONTACT

Battle teaser

First artifact

CI vs the field

Why did 412 passing tests miss the photo-attach crash?

  1. AautoGrantPermissions granted every declared permission up front, so the runtime permission logic, including the wrong-permission-for-API-33+ bug, never executed in CI
  2. BEmulators cannot open photo pickers at all
  3. CThe test data lacked photos larger than 1MB
  4. DAppium does not support Samsung devices
Answers, scoring, hints, and the full battle stay sealed.

// SKILL TAGS

mobile-testingreal-devicespermissionsandroid