When one Playwright shard keeps everyone waiting

Four shards start together, three finish, and the last one holds the pipeline open long after the others are idle. The merged report should tell you whether the work was badly distributed. Instead, the report may be incomplete because one blob artifact never reached the merge job.

Those two failures look similar at a glance. Both produce a sparse or lopsided view. Only one is a performance problem, so validate the report inputs before using the timeline to redesign the suite.

How the timeline reaches the HTML report

Each shard is an independent Playwright invocation. It has its own workers, start time, projects, retries, results, and attachments. A normal HTML reporter attached to each shard produces several separate reports. No single one can describe the whole CI run.

The blob reporter is the transport format for this job. It writes a ZIP containing the test-run information needed to produce another Playwright report later. CI uploads the blob from every shard, a merge job downloads all of them into one directory, and merge-reports replays that combined information through the selected output reporter.

The minimal runner configuration is:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  reporter: process.env.CI
    ? [['blob']]
    : [['list'], ['html', { open: 'never' }]],
});

On CI, this produces blob input rather than shard-level HTML output. Locally, the same suite remains convenient to read. You can configure multiple reporters in CI if another system needs JUnit or terminal output, but the blob reporter is the important part for a later Playwright merge.

The merge command converts the accumulated blobs into the standard HTML report:

npx playwright merge-reports --reporter=html ./all-blob-reports
npx playwright show-report ./playwright-report

The Speedboard timeline for merged reports was introduced in Playwright 1.58. A report created directly by playwright test --reporter=html is not equivalent simply because it uses the same HTML UI. If the tab or timeline is absent, check the installed CLI first and identify which command created the folder:

npx playwright --version
find ./all-blob-reports -maxdepth 1 -type f -name '*.zip' -print | sort

Do not infer more from the visualization than the report supports. It helps you see the execution shape captured in the merged results. It does not include CI queue time before a shard command started, dependency installation before Playwright launched, or artifact upload time after the runner exited. Compare the timeline with job timestamps when the pipeline is slow but all tests appear tightly grouped.

The merge also matters to custom reporters. Playwright calls the same Reporter API while processing merged blobs, but projects from different shards remain separate TestProject objects. Five shards of a project named chromium can therefore appear as five project objects with the same name in onBegin. A custom aggregation keyed only by project name can collapse real data even when the built-in HTML report is correct.

Build a merge job that cannot silently lose a shard

Most broken merged reports begin in artifact plumbing, not in the reporter. A shard fails, the upload step is skipped because earlier commands returned non-zero, and the merge job happily processes the files it did receive. The resulting HTML may look polished while omitting the failure that mattered.

Keep matrix fail-fast disabled so one failing shard does not cancel its siblings. Upload each blob when the job has not been cancelled, even if the test step failed. Give artifacts names that expose the shard index. In the merge job, flatten the downloaded artifacts into the directory passed to Playwright.

name: Playwright shards

on:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 45
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: lts/*
      - run: npm ci
      - run: npx playwright install --with-deps
      - name: Run shard
        run: npx playwright test --shard=${{ matrix.shard }}/4
      - name: Upload blob report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v5
        with:
          name: blob-report-${{ matrix.shard }}
          path: blob-report/
          retention-days: 1

  merge:
    if: ${{ !cancelled() }}
    needs: [test]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: lts/*
      - run: npm ci
      - name: Download every blob
        uses: actions/download-artifact@v5
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - name: Validate and merge
        run: |
          blob_count=$(find all-blob-reports -maxdepth 1 -type f -name '*.zip' | wc -l | tr -d ' ')
          if [ "$blob_count" -ne 4 ]; then
            echo "Expected 4 blob reports, found $blob_count" >&2
            find all-blob-reports -maxdepth 2 -type f -print | sort >&2
            exit 1
          fi
          npx playwright merge-reports --reporter=html ./all-blob-reports
      - name: Upload merged HTML report
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@v5
        with:
          name: playwright-html-${{ github.run_attempt }}
          path: playwright-report/
          retention-days: 14

The count check is intentionally tied to this four-shard matrix. If one shard can emit more than one blob because you run distinct filtered commands in the same job, count expected filenames or publish a manifest instead. A generic "at least one ZIP" check protects the merge command from an empty directory but does not protect report completeness.

File names help. The blob reporter's default names include a hash and, for sharded runs, a shard number. The hash can reflect filters such as --grep, selected projects, config tags, and file arguments. Do not rename every download to report.zip; one artifact will overwrite another when the directories are flattened. Preserve the generated names or assign provably unique output names.

All shard jobs should install from the same lockfile and run the same Playwright package version. This is operational discipline rather than permission to infer a particular error message from a mismatch. Record npx playwright --version in the logs when diagnosing a merge problem. If versions differ, align them before trusting subtler report comparisons.

The merge job should also retain its own failure output. An HTML report is not guaranteed when merge-reports fails before the reporter finishes. CI logs, the input file listing, and the untouched blob artifacts are the evidence in that case. Keep blobs briefly enough that an engineer can download and reproduce the command locally.

Tell missing input from uneven execution

Start triage outside the HTML UI. Establish the expected matrix cardinality, list the artifact names CI produced, list the ZIP files downloaded, and compare them. A missing artifact is a pipeline failure even if every visible test passed.

This shell check validates the default shard suffixes, zero padding included, without extracting or rewriting the blobs:

set -euo pipefail

report_dir=all-blob-reports
expected_shards=4
pad_width=${#expected_shards}

for shard in $(seq 1 "$expected_shards"); do
  padded=$(printf "%0${pad_width}d" "$shard")
  if ! find "$report_dir" -maxdepth 1 -type f \
    -name "*-${padded}.zip" -print -quit | grep -q .; then
    echo "No blob report found for shard $padded" >&2
    exit 1
  fi
done

echo "Found a blob ZIP for every shard"

The padding is the part most checks get wrong. Playwright builds the default blob name by padding the shard number to the width of the shard total, so a four-shard matrix produces report-<hash>-1.zip through report-<hash>-4.zip, while a twelve-shard matrix produces report-<hash>-01.zip through report-<hash>-12.zip. A loop that hard-codes unpadded suffixes therefore reports shards 1 to 9 as missing the moment the matrix reaches ten, on a run where every blob arrived correctly. Deriving pad_width from expected_shards keeps the glob aligned with the total at any size, provided the variable matches the x/y total the shard jobs actually passed. Fall back to the count check if an explicit fileName option overrides the default naming. The purpose is to make the expected inputs testable, not to turn a sample glob into a universal contract.

Next, compare test inventory. The HTML report lets you filter and inspect tests, but the strongest check starts before execution with a stable collected list for the same projects and filters. Be careful with shards: each shard intentionally runs only part of the list. If you persist per-shard collection or results, associate it with the same command arguments. A report containing fewer tests than yesterday could reflect a missing blob, a changed filter, a renamed project, or tests that were never collected.

Command drift deserves a direct check. Matrix jobs are supposed to differ only in the x/y shard value, yet conditional shell fragments often add a project or grep to one entry. Print the resolved command in every job. Compare the repository commit, working directory, config path, selected projects, grep expressions, and shard total. Two blobs can merge successfully even when one was created from a smoke selection and another from the full suite. The UI cannot infer that the operator intended identical selections.

Collection is especially useful during a migration. Run the unsharded command with --list in an audit job, save its output, and compare the expected test identities with the combined report. Line and column numbers can move during refactors, so use stable project, file, and title paths rather than treating a source line as identity. Parameterized tests also need their distinct titles preserved. If titles collide, fix the naming before relying on a simple text comparison.

Duplicate input is the mirror image of missing input. An artifact download can contain two blobs for shard 2 and none for shard 3 while still satisfying a raw ZIP count of four. That is why the suffix loop is stronger than the count for default names. If you set custom blob names, publish a small CI manifest alongside each artifact containing the shard index, total, commit, config path, and Playwright version. Validate those manifests in the merge job before invoking Playwright. The manifest belongs to your pipeline, not to a fictional Playwright API.

Do not extract and recompress blob ZIP files to add that metadata. Treat the reporter output as opaque merge input. Upload a neighboring text or JSON file instead, and pass only the directory containing the original ZIPs to merge-reports. This keeps the diagnostic layer from corrupting the evidence it is checking.

Read the pipeline manifest as a set

A useful pipeline-owned manifest has one record for each shard command. Read the shard index and shard total first. A healthy four-way run contains indexes one through four exactly once, and every record says the same total. Next compare run attempt, commit, working directory, config path, selected projects and filters, and Playwright version. Those values should be identical unless the matrix deliberately varies one of them and records that dimension explicitly. Finally, match each record's blob filename to one downloaded ZIP.

This produces two distinct broken shapes. A completeness failure has a missing index, a repeated index, or a manifest whose blob filename is absent. A homogeneity failure has every index but one record names a different commit, config, shard total, or selection. Both can yield four ZIP files and a successful merge. Only the first is fixed by artifact upload logic; the second is fixed where shard commands are assembled.

The raw ZIP count is therefore a misleading healthy-looking value. A count equal to matrix size says only that the directory contains that many files. It does not prove that each expected shard appears once or that every file belongs to the same release attempt. The default filename suffix check improves identity coverage, while the neighboring manifest covers pipeline facts that the filename was never meant to encode.

Treat merge success as a separate integrity check. The manifest does not prove that a ZIP is readable, that Playwright can reconcile its test root, or that every attachment referenced inside it survived production. Preserve the untouched inputs and merge stderr when that later boundary fails. Do not mark a shard transport incident solved merely because its manifest record exists.

Ownership follows the first unequal field. The CI platform team owns a missing artifact, duplicate download, stale run-attempt selection, or archive permission failure. The test-framework team owns inconsistent config paths, filters, project selections, shard totals, and Playwright installations. A suite team owns an execution tail only after the merge job proves that inputs are complete and homogeneous.

The handoff should contain the expected matrix, every manifest record, the downloaded ZIP listing, merge command and status, workflow run and attempt, commit, and the first field that differs. For a genuine long tail, add resolved worker counts, runner class, Playwright command start and finish times, and the tests or retries occupying the edge. That evidence keeps a pipeline defect from becoming a speculative test-refactoring ticket.

Then inspect the CI jobs. If all four test jobs ran and uploaded blobs, but the merge download has only three directories, the fault lies in artifact selection or permissions. If a shard job has no upload step in its log, look for cancellation or an if: success() default. If all blobs exist and the merged test inventory is complete, the sparse timeline is real enough to investigate as scheduling.

A merge error involving test roots is a separate signature. Reports produced on different operating systems can carry paths that need disambiguation. Playwright documents an explicit merge config for that case:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  reporter: [['html', {
    open: 'never',
    outputFolder: 'playwright-report',
  }]],
});

Use it with the CLI rather than hoping the output reporter discovers the intended root:

npx playwright merge-reports \
  --config=merge.config.ts \
  ./all-blob-reports

That config is for producing the merged output. It may differ from the runner config used inside each shard. Keep testDir aligned with the repository layout at the merge checkout, and type-check the config in the same dependency installation used by the merge job.

Read three common timeline shapes without guessing

The timeline becomes useful after input completeness is established. Treat it as a lead generator. Select the tests at the unusual edge, then use their result pages, retries, steps, and attachments to confirm a cause.

Worked example: one oversized file owns the tail

Without fullyParallel, Playwright shards at file granularity. Imagine a repository with many small specs and one admin-regression.spec.ts containing a large serial workflow. One shard receives that file. The other shards exhaust their smaller files and finish while the admin shard continues.

The supporting evidence is concrete:

• The merged report contains all expected tests and all shard blobs were present.
• Most tests near the tail share one source file or serial suite.
• The slow shard ran more work because the assignment unit was a file, not because each individual test became slower.
• Re-running the same shard command preserves the concentration, even when host load is normal.

Two fixes have different costs. Splitting the file into independent, cohesive specs retains file-level execution semantics but requires careful fixture and state ownership. Enabling fullyParallel: true gives Playwright test-level granularity for better distribution, but it permits tests in a file to run concurrently. A suite that relies on shared mutable state can become less stable. Do not flip the switch until those tests are genuinely isolated.

A safe migration starts with the offending file. Remove dependencies between cases, move shared setup into fixtures with appropriate scope, and run that area repeatedly with full parallelism. Expand only after the data and cleanup boundaries survive concurrency.

Worked example: retries, not slow first attempts, extend the shard

Another timeline shows balanced work until the end, where one shard continues with a small set of cases. The test pages reveal first-attempt failures and later retry attempts. The last shard is doing more attempts, not necessarily running slower tests.

The distinction matters because lowering a timeout might shorten each failure while increasing false negatives. Raising a timeout might make the tail longer without fixing the instability. Open the first failed attempt. If it shows a product error, locator race, worker exit, or unavailable dependency, route the defect from that evidence. The passing retry is a comparison, not absolution.

Correlate the runner log with the report's retry labels. A test that passes on attempt zero is not the source of a retry tail even if its ordinary duration is high. A flaky test with a fast second attempt can still add browser and worker restart overhead. The report gives you both the classification and the attempts needed to avoid mixing those cases.

Worked example: CI setup is slow but the timeline looks healthy

The pipeline is slow, while the merged execution view shows all shards densely occupied for only part of the job. Logs show that one runner spent substantial time downloading dependencies or waiting for a service before npx playwright test began.

Nothing in the test report can explain work that happened before the Playwright process. Moving tests between shards will not recover that time. Compare each job's command start with its overall start, inspect setup steps, and measure service readiness in the CI system. This is the near-miss that prevents a test optimization project from chasing infrastructure delay.

The opposite can occur after execution. Artifact compression and upload may dominate a shard with large traces or videos, yet the test timeline ends earlier. Review artifact sizes and upload-step durations. If evidence policy is the cost, change retention modes or split artifact storage deliberately. Do not delete failed-attempt evidence merely to make one graph look shorter.

Second failure mode: one shard has different execution capacity

The test inventory is balanced, but a shard still finishes later. Its runner log says it used fewer Playwright workers, or the CI matrix placed it on a different machine class. The same number of tests spread across fewer concurrent workers creates a longer execution window without making individual tests intrinsically slow.

Compare the resolved worker count printed by each invocation, project selection, host type, CPU limits, and memory pressure. Configuration can be identical while percentage-based workers resolve differently on unequal machines. Container limits can also make the advertised host capacity irrelevant. Keep this diagnosis in CI evidence instead of inferring it from the timeline alone.

The fix is environmental consistency or an explicit worker policy, not moving arbitrary test files until the chart looks balanced. Setting the same worker count across shards makes capacity predictable but may underuse stronger hosts or overload weaker ones. Standardizing runner size costs infrastructure budget. Choose after observing equivalent jobs, and do not publish a speedup figure until those runs exist.

Roll out timeline-based changes with a control run

Performance changes alter scheduling, which can expose data races. Capture a complete report before changing shard count, fullyParallel, workers, or file layout. Apply one variable at a time and compare several equivalent CI runs. A single run can be affected by host contention, retries, or a different selected test set.

Keep commands and filters visible in the report pipeline. Blob filenames include a hash influenced by selections and tags, but humans should not have to reverse-engineer a hash to know what ran. Use job names, artifact names, and testConfig.tag when merging genuinely different environments. Tags help distinguish environment runs in the combined output; they are not a substitute for shard indexes.

If you split a large spec, preserve test titles and behavior where practical so report comparisons remain readable. If the split changes setup scope, watch for new authentication calls, database fixtures, and cleanup load. Better shard balance can increase pressure on a shared dependency because more tests reach the same operation at once.

If you enable full parallelism, begin with suites whose state is already per-test. Use a canary CI job at the target worker and shard counts. Inspect failures for shared identifiers and conflicting writes. The benefit is shorter idle tails. The cost is higher concurrency and potentially more resource consumption in the application under test.

Record the canary's test selection and blob inventory beside its merged report. A faster chart is meaningless if the new job skipped a browser project or filtered out slow cases. Compare passed, failed, skipped, and flaky classifications as well as total duration. Do not combine them into a home-grown performance score. Each count answers a different release question, and a speed improvement caused by early failures is not an improvement.

When you increase shard count, remember that every shard pays fixed costs: process startup, browser setup, authentication dependencies, report creation, artifact upload, and a CI runner allocation. More shards eventually move time from test execution into orchestration. The timeline can show the execution portion, while CI timestamps reveal the fixed costs around it.

Treat the merged report as a release artifact as well as a tuning tool. Give it a run-attempt-specific name so a rerun does not masquerade as the original. Retain it long enough to investigate, and keep access appropriate for screenshots, traces, console output, and request data that may contain sensitive test information.

When not to use the timeline

Do not use it to prove report completeness. Validate blob count, shard identity, command filters, and test inventory first. A visually full chart can still omit a shard whose tests resemble work elsewhere.

Do not use it as an application performance benchmark. Test duration includes automation actions, fixtures, retries, browser work, network conditions, and runner load. Use product telemetry or a purpose-built performance test when the question is response latency under controlled load.

Avoid merging unrelated release runs into one directory. Blob hashes reduce filename collisions, and tags can distinguish environments, but a successful merge is not evidence that the inputs belong to the same commit or CI attempt. Scope artifact downloads to one workflow run and record the commit in the surrounding CI metadata.

Skip timeline-driven parallelization for tests that intentionally share a serial business transaction. Breaking a stateful scenario into concurrent cases can invalidate the test contract. Improve the scenario itself, isolate independent coverage, or accept that this portion forms a tail.

Finally, do not build a custom timeline merely because the built-in view seems absent. Confirm Playwright 1.58 or newer, confirm blob inputs were merged, and open the Speedboard tab in the generated HTML report. If those checks fail, fix the production path. Recreating scheduling logic in a reporter adds maintenance while leaving the missing artifact or wrong command untouched.