22 Playwright Retry, Sharding, Parallelism, and CI Interview Scenarios

CI scaling interviews test systems reasoning. A suite can be fast and untrustworthy, or isolated and unnecessarily expensive. Senior candidates must explain process boundaries, data ownership, retry semantics, report completeness, and the resource curve behind a chosen topology.

The scenarios below move from one failing worker to a multi-job pipeline. For every choice, state the unit of isolation, what happens after failure, how artifacts are reunited, and which constraint limits further concurrency.

Draw the Execution Topology

The official Playwright sharding guide defines shards as independent suite partitions that can run on separate machines and explains how parallel configuration affects balance. Interview answers should keep four layers distinct: tests, worker processes, projects, and CI jobs. Confusing them leads to duplicate data, missing reports, and false throughput claims.

Do not optimize from test count alone. File size, runtime distribution, browser projects, setup dependencies, external rate limits, and artifact upload time all shape the useful design.

Retries and Worker Lifecycle

1. Why should CI distinguish passed, flaky, and failed tests?

These outcomes carry different confidence. A first-attempt pass has no observed instability, a retry pass proves at least one attempt failed, and a final failure exhausted the configured attempts. I would report and trend them separately, retain the failing context, and assign recurring flaky identities to owners. Collapsing flaky into passed lets a pipeline appear healthy while consuming time and hiding unresolved risk.

2. What happens to worker-owned state after a test failure?

Playwright discards the failed worker process and continues in a new one, so worker fixtures and hooks may be created again. I would make setup repeatable, cleanup idempotent, and external leases recoverable. A candidate who treats worker scope as a once-per-pipeline singleton can leak accounts or assume state survives a restart. Process lifetime and business-resource lifetime must be designed separately.

3. How should retry-aware code use testInfo.retry?

Use it for diagnostics or safe recovery that is explicitly tied to another attempt, such as clearing a known test cache before retry. I would not change the expected business outcome, use weaker assertions, or silently create different data only on retry. If retry-specific behavior is necessary, the report should expose it. Otherwise the passing attempt no longer validates the same scenario as the failed one.

4. Why are serial groups expensive under retries?

Dependent tests in a serial group are retried together, so an early failure can skip later cases and force already-passed setup scenarios to run again. This enlarges the failure unit and makes diagnosis depend on order. I would prefer isolated tests with API or fixture setup. Serial mode is a deliberate exception for a truly indivisible sequence, not a substitute for modeling independent preconditions.

Parallelism and Resource Ownership

5. How would you decide whether to enable fullyParallel?

I would verify that tests own their data, do not depend on file order, and can tolerate test-level distribution. Then I would compare duration and resource pressure across representative runs. Fully parallel execution can improve utilization and shard balance, but it also exposes hidden shared state and may overload the application. The rollout should start with suitable suites and use failures as isolation evidence.

6. Why should account allocation include both shard and worker identity?

Separate CI jobs can use the same local parallel index, so indexing a shared account pool by worker alone creates cross-shard collisions. I would include a run ID and shard or job identifier in the lease key, then use the worker's stable parallel slot within that partition. The leasing service should reject duplicate ownership and expire abandoned leases. Machine boundaries must be part of the namespace.

import { test as base } from '@playwright/test'

type WorkerFixtures = {
  account: { id: string; email: string }
}

export const test = base.extend<{}, WorkerFixtures>({
  account: [async ({}, use, workerInfo) => {
    const runId = process.env.CI_RUN_ID ?? 'local'
    const shardId = process.env.SHARD_ID ?? 'single'
    const leaseKey = [runId, shardId, workerInfo.parallelIndex].join(':')
    const account = await leaseAccount(leaseKey)

    await use(account)
    await releaseAccount(leaseKey, account.id)
  }, { scope: 'worker' }],
})

7. How would you prevent parallel tests from mutating the same order?

Each test should create or lease its own order, using a run-aware namespace and deterministic cleanup. Read-only tests can share immutable fixtures, but mutation requires an owner. I would avoid selecting the newest database row or a fixed customer because those shortcuts race under concurrency. If data creation is expensive, use a pool with exclusive leases and health checks rather than shared writable state.

8. Why can beforeAll become a scaling bottleneck?

A file-level hook can create expensive shared setup per worker execution and may run again after worker replacement. It can also encourage tests in the file to mutate one object. I would move reusable capabilities into fixtures with explicit scope, prepare independent records through APIs, and measure setup separately. Shared setup is acceptable when immutable and well-owned, but it should not make a file the indivisible scheduling unit by accident.

Sharding and Balance

9. How would you explain the meaning of --shard=2/5?

It selects the second partition from a five-part split for that Playwright invocation. All five shard jobs together represent the intended suite; one shard alone is not a complete result. I would ensure every job uses the same commit, configuration, project selection, and dependency strategy, while giving each a unique data namespace. The merge stage must fail if an expected shard report is absent.

10. Why does file size matter when fullyParallel is disabled?

Without test-level parallel distribution, whole files become important scheduling units. One file containing many slow scenarios can dominate a shard while others finish early. I would split files along coherent, independent behavior rather than manufacture equal line counts, then inspect duration distribution. Alternatively, fully parallel execution can improve balance after isolation is proven. Sharding cannot evenly divide work that the suite packages into large indivisible units.

11. How would you choose a shard count?

I would measure end-to-end pipeline duration across several counts, including queueing, environment startup, dependency setup, browser installation or cache use, artifact upload, and report merge. More shards eventually add overhead or hit service limits. The selected count should meet feedback goals at acceptable cost and flake rate. A theoretical division of test time is not a capacity model.

12. How should browser projects influence sharding design?

Projects multiply executions and may have different duration or infrastructure needs. I would decide whether each shard runs the full project matrix or CI jobs partition projects separately, then preserve unified reporting. Critical smoke coverage might run across engines while deeper suites use selected projects. The topology should make omissions explicit and avoid assigning one slow project to a hidden critical path.

Reports and CI Artifacts

13. Why use blob reports for sharded runs?

Each shard produces structured results that can be uploaded as an artifact and merged later into one report. This preserves test identities, projects, retries, attachments, and outcomes better than concatenating console text. I would give artifact names unique shard coordinates, download them into a clean merge directory, and verify the expected set. The final report is only trustworthy if every partition contributed.

14. How should the pipeline react when one shard report is missing?

The merge stage should fail with a clear list of missing partitions, even if all available reports are green. A missing job is unknown coverage, not a pass. I would also preserve the failed job logs and distinguish test failure from infrastructure or upload failure. Rerunning only the absent job can be acceptable if the commit and environment remain pinned and the final report records the attempt.

15. How would you balance trace retention against CI cost?

I would retain failure-focused traces and expand capture temporarily for a flaky area under investigation. Successful-run traces have lower diagnostic value but high storage and upload cost at scale. The policy must also protect sensitive data and set expiration. Report links should keep trace, retry, project, worker, and shard context together so engineers do not compare an artifact from the wrong attempt.

16. Why should fail-fast behavior be used carefully with shards?

Stopping after a threshold saves resources during broad breakage, but it also leaves parts of the matrix unexecuted and can reduce evidence about whether the issue is localized. I would use a threshold aligned with incident cost, mark unexecuted coverage clearly, and allow diagnostic or critical projects to finish when useful. A fast red pipeline is valuable only if responders can see what remains unknown.

Capacity and External Systems

17. How would you detect too many workers on one runner?

I would compare throughput, action duration, browser crashes, memory pressure, CPU saturation, and service latency as workers increase. If elapsed time stops improving or flakes rise, the runner has crossed a useful concurrency point. I would cap workers based on measured capacity and resource requests, not copy a laptop setting. Scaling out across jobs may be better than oversubscribing one machine.

18. How should external API rate limits affect parallel execution?

Treat the limit as a shared capacity constraint. I would reduce concurrency for affected tests, use approved mocks for behavior outside the integration contract, or provision isolated test capacity. Random backoff inside every test can inflate deadlines and hide an overloaded design. The pipeline should identify rate-limit responses separately and coordinate access at a fixture or service boundary rather than let workers compete blindly.

19. How would you diagnose one consistently slow shard?

I would compare its test and project composition, setup duration, retry count, data region, runner class, and artifact time. Stable slowness suggests uneven work; variable slowness suggests contention or dependency behavior. I would use per-test timing to find large scheduling units and then adjust file boundaries, parallel safety, or topology. Moving tests by name without measuring subsequent runs merely relocates the bottleneck.

Governance and Scaling Decisions

20. How would you choose a retry policy for pull requests and scheduled runs?

I would keep retries limited, preserve flaky classification, and decide from feedback needs and artifact strategy. Pull requests need quick, trustworthy gating; scheduled runs may collect broader diagnostic evidence but should not normalize instability. Different projects can justify different policies when their dependencies differ. Every additional attempt has time and infrastructure cost, so recurring retry passes must enter a repair queue with ownership.

21. Why is permanent quarantine dangerous?

Quarantine can unblock delivery while an owner investigates, but a permanently excluded test creates silent coverage loss and often becomes stale. I would require an issue, risk statement, owner, expiration, and visible reporting. The test should run in a non-gating lane when possible so evidence continues. Reintegration requires a causal fix and repeated proof, not simply several recent green attempts.

22. How would you defend a Playwright CI architecture in a senior interview?

I would show workload measurements, project and shard boundaries, worker capacity, data leasing, retry semantics, dependency setup, artifact flow, and failure policy. Then I would name the limiting system and the next scaling experiment. The design is credible when a missing shard cannot pass, flaky outcomes remain visible, and engineers can trace one result from merged report back to its exact attempt and evidence.

CI Architecture Checklist

• Draw tests, workers, projects, shards, and CI jobs as distinct layers.
• Namespace mutable resources by run, shard, worker, and test where needed.
• Make setup repeatable because workers and jobs can restart.
• Trend flaky outcomes separately from first-attempt passes.
• Merge structured reports and reject incomplete shard sets.
• Tune workers and shards with end-to-end measurements and service limits.
• Give quarantine, retries, and artifact retention explicit owners and policies.

Scale the Signal, Not Just the Process Count

Conclude with an execution topology you can defend. Explain how work is partitioned, how mutable data stays exclusive, what a retry means, and how all shard evidence becomes one complete result. Then identify the resource or dependency that limits the next increase. The best CI design reduces feedback time while preserving isolation, diagnostic depth, and an honest account of unexecuted or unstable coverage.