Shard Playwright Tests and Merge Reports Across CI Jobs

Sharding shortens a Playwright pipeline only when the work is divided cleanly and the evidence is reassembled afterward. Four jobs that finish at very different times create little benefit, and four isolated HTML reports force reviewers to hunt for the real result. A production-ready design treats test distribution and report merging as one workflow.

The contract is straightforward: every shard receives the same source revision, dependency graph, Playwright configuration, and test inventory; each receives a unique partition; every job uploads a blob report even on failure; and a downstream job verifies the artifact set before producing one report. Missing any part can yield a fast but incomplete signal.

Decide whether sharding addresses the actual bottleneck

Measure the existing job before adding machines. If most time is spent installing dependencies, building the application, or waiting for a shared environment, splitting only test execution may multiply setup cost without improving completion time. Sharding is most effective when test execution is a substantial, parallel-safe part of the critical path.

Confirm that tests can run independently across machines. Shared accounts, static database records, global rate limits, and one mutable deployment can turn added concurrency into collisions. Fix resource isolation before increasing shard count, or the pipeline will exchange duration for flakiness.

The official Playwright sharding guide uses --shard=current/total and explains the balance difference between test-level and file-level distribution. Treat the total as a versioned CI decision. Changing it alters which tests land together and can expose hidden dependencies.

Configure the runner for mergeable output

Use the blob reporter in CI and a human-friendly reporter locally. Blob output contains results and attachments in a mergeable format; it is an interchange artifact, not usually the final interface reviewers open.

fullyParallel: true allows Playwright to balance individual tests across shards. Without it, whole files are assigned, so a file containing many slow scenarios can dominate one job. Enabling full parallelism also allows tests from a file to run concurrently, so first remove assumptions about file order and shared mutable state.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests/e2e',
  fullyParallel: true,
  retries: process.env.CI ? 1 : 0,
  reporter: process.env.CI
    ? [['blob']]
    : [['html', { open: 'never' }]],
  use: {
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
  },
});

Keep filters identical across matrix jobs. If one shard adds --grep @smoke while another runs the full inventory, the merged report does not represent a valid partition. Browser projects, environment tags, and feature flags also belong to the stable input contract.

Dispatch one shard per CI matrix entry

A matrix makes the current and total shard values explicit. The shard number is one-based, so a four-job setup runs 1/4, 2/4, 3/4, and 4/4. Set fail-fast: false so one regression does not cancel other shards and erase their coverage.

jobs:
  playwright-tests:
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: lts/*
          cache: npm
      - run: npm ci
      - run: npx playwright install --with-deps
      - name: Run shard
        run: >-
          npx playwright test
          --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}
      - name: Upload blob report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v4
        with:
          name: blob-report-${{ matrix.shardIndex }}
          path: blob-report
          retention-days: 1

The upload condition matters. A test command returns a failure exit code for a regression, but that shard's blob report contains the failure evidence the merged report needs. !cancelled() preserves normal failures while respecting a manually cancelled workflow.

Isolate data and resources across jobs

Machines do not share a Playwright worker index, so a worker-derived account name alone may collide across shards. Include a run identifier, shard index, project, and parallel index in resource namespaces when the external system requires uniqueness. Keep names within database and provider limits.

Provision data through APIs or fixtures that can tolerate retry and partial teardown. A shard may be restarted by CI, and a Playwright worker may restart after a test failure. Cleanup should target the exact namespace rather than a broad shared tenant.

If the application environment has a concurrency ceiling, increasing shards can overload it and lengthen every request. Match shard count to both CI capacity and system-under-test capacity. A stable two-shard run can be more useful than eight competing jobs.

Merge all blob artifacts in a dependent job

The merge job should wait for every shard but run even when one fails. Download blob artifacts into one directory, verify the expected count, and invoke merge-reports. The command can generate HTML or multiple report formats from the same blob inputs.

  merge-reports:
    if: ${{ !cancelled() }}
    needs: [playwright-tests]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v5
        with:
          node-version: lts/*
      - run: npm ci
      - uses: actions/download-artifact@v5
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - name: Merge shard reports
        run: npx playwright merge-reports --reporter=html ./all-blob-reports
      - uses: actions/upload-artifact@v4
        with:
          name: playwright-html-report
          path: playwright-report
          retention-days: 14

For a hardened workflow, add a shell check that the directory contains the expected number of blob archives before merging. Otherwise a missing upload can produce a polished partial report. The test job status still protects the pipeline, but reviewers should never mistake incomplete evidence for full execution.

Preserve identity across environments and projects

Sharding divides one logical inventory. Running the same tests against staging and preview, or Linux and Windows, is a different dimension. Tag those environments in configuration so merged results remain distinguishable. Do not use shard numbers as environment labels; they are distribution details and can change.

When merging output produced on different operating systems, provide an explicit merge configuration if test roots are ambiguous. Keep checkout paths and testDir semantics compatible. A blob file can be valid on its own while the merge cannot map its source location to the current workspace.

Artifact names should include the workflow attempt when reruns can coexist. A rerun of one failed CI job must not accidentally merge three artifacts from the previous attempt with one new artifact. Prefer immutable run-scoped storage or clear the destination before download.

Balance by observed duration, not file count alone

With full parallelism, Playwright distributes individual tests, which usually improves balance. Yet test count is only a proxy for duration. A few authentication or visual tests may still dominate a shard. Inspect per-shard completion times and identify consistent outliers.

Without full parallelism, keep files small enough that one file cannot monopolize a partition. Splitting a large file is reasonable when it reflects independent domains. Do not fracture coherent setup purely to chase equal file counts if the result creates duplication or hidden order dependencies.

Increasing the number of shards eventually reaches diminishing returns because every job repeats checkout, install, application startup, and artifact work. Select the smallest count that meets the pipeline objective with stable resource use, then revisit it as the inventory changes.

Diagnose incomplete or inconsistent merged reports

If tests are absent, compare the discovered test count from each shard and confirm all indexes share the same total. Check filters, project selection, and whether one job failed before producing blob output. A successful merge proves that readable artifacts were combined, not that the inventory was complete.

If attachments are missing, verify trace and screenshot configuration on shard jobs and confirm the artifact upload includes the whole blob-report directory. Do not upload only a report archive selected by a brittle filename pattern.

Duplicate tests usually indicate overlapping shard commands, stale artifacts, or mixed workflow attempts. Clear the download directory, scope artifacts by run, and inspect blob filenames for shard identity. Source-path errors during merge point toward different checkout layouts or an incorrect testDir in the merge context.

Operational sharding checklist

• Measure execution time separately from setup and artifact time.
• Confirm tests and external data are parallel-safe.
• Give every job the same commit, lockfile, config, projects, and filters.
• Use each one-based shard index exactly once.
• Enable full parallelism only after removing file-order coupling.
• Keep matrix fail-fast disabled for complete evidence.
• Upload blob output when tests fail.
• Verify all expected blob artifacts before merging.
• Keep workflow attempts and environments distinguishable.
• Review shard duration and capacity after suite growth.

Roll out sharding as a complete reporting system

Begin with two partitions and record both job duration and test discovery. Add blob output, an always-run upload, and a dependent merge job before expanding the matrix. Inject one controlled failing test to confirm that its trace appears in the unified report and the workflow still returns the correct failure status. Once completeness, isolation, and artifact identity are proven, tune shard count from observed capacity. That sequence delivers faster feedback without sacrificing the single trustworthy result reviewers need.