Harden Playwright CI with Pinned Containers, Browser Caches, and Artifacts

A Playwright job is reproducible only when its package, browser binaries, operating-system libraries, fonts, runtime settings, and application revision agree. Pinning one layer while allowing another to drift produces failures that look like test instability but are actually environment changes. CI hardening makes those inputs explicit and preserves enough evidence to distinguish runtime faults from product faults.

Containers are one strong strategy because they bundle browser binaries and system dependencies. Hosted runners with browser installation are another valid strategy. Choose one as the source of the browser runtime, define a measured cache policy around it, and keep failure artifacts independent of whether the test command succeeds.

Define the reproducibility contract

Record the inputs that must remain consistent for a comparable run: repository commit, package lockfile, Node runtime, Playwright package, browser build, Linux image or runner image, test configuration, locale, timezone, fonts, worker count, and environment endpoint. A rerun should change only the dimension being investigated.

The official Playwright CI guide recommends a conservative worker count for stability and describes both browser installation and container approaches. It also warns that browser-binary caching is often not beneficial because restore time can rival download time and Linux system dependencies are separate. Treat these as design inputs, then verify them on your own runner.

Pin the package and container as one unit

The Playwright Docker image includes browsers and their operating-system dependencies, but the Playwright npm package is installed by the project. The package version must match the image version so the expected browser executables are present. Avoid floating image tags in release CI.

A Dockerfile can require CI to provide the exact Playwright version read from the lockfile. With no default, an omitted version fails the build instead of silently selecting a moving target.

ARG PLAYWRIGHT_VERSION
FROM mcr.microsoft.com/playwright:v${PLAYWRIGHT_VERSION}-noble

WORKDIR /work
COPY package.json package-lock.json ./
RUN npm ci

COPY . .
CMD ["npx", "playwright", "test"]

Build automation should derive PLAYWRIGHT_VERSION from the locked @playwright/test package and store the complete image reference in run metadata. Upgrade the package, image tag, and browser expectations together in one reviewed change. Test that change against the supported projects before it becomes the default pipeline.

The official image is intended for testing trusted applications. Browser sandbox decisions differ for untrusted crawling. For ordinary end-to-end testing, keep the security model explicit rather than copying flags from an unrelated scraping example.

Configure containers for browser process health

Chromium can exhaust shared memory in constrained containers. Playwright's Docker guidance recommends --ipc=host for Chromium and --init to handle the process tree correctly. CI platforms expose these controls differently, so verify how the selected runner launches containers.

Set CPU and memory limits deliberately. A high worker count inside a small container creates action timeouts, browser crashes, and delayed services that masquerade as flaky tests. Begin with one worker, collect resource telemetry, and increase only when the host and application environment can sustain parallel browsers.

Pin timezone and install the fonts required by visual tests. Containerizing the browser does not guarantee that local and CI text rendering match if the local machine uses fonts absent from the image. Keep screenshot baselines tied to the environment that produces them.

Choose between a pinned image and runtime browser install

With a pinned Playwright image, browsers and Linux dependencies are already present. Running npx playwright install --with-deps again adds avoidable work and can blur which layer owns the runtime. Install the project package with the lockfile and verify compatibility.

On a hosted runner without the image, install dependencies and browsers after npm ci:

npm ci
npx playwright install --with-deps chromium
npx playwright test --project=chromium

Install only the browser projects the job executes. If another matrix job covers WebKit, the Chromium-only job does not need that binary. Keep installation commands next to project selection so future maintainers do not accidentally run a project whose browser was omitted.

These two approaches have different tradeoffs. A container gives a stronger operating-system contract and can be pulled once by the infrastructure. Runtime installation integrates easily with hosted images but depends on the runner's base system and download path. Pick based on repeatability, startup behavior, and platform coverage rather than assuming one is always faster.

Cache package downloads before considering browsers

Cache the npm download store using the package lockfile as a key, then still run npm ci. This preserves lockfile verification and lifecycle scripts while reducing repeated package downloads. Restoring node_modules itself can retain native or generated state from a different runtime image.

Browser caching deserves a separate measurement. On Linux, restoring a large browser archive can be no faster than downloading it, and required system packages are not part of that cache. Record cache restore, browser installation, and total job time over representative runs before accepting the added invalidation logic.

If measurement shows a benefit, key the cache by operating system and the exact Playwright package version, and cache the documented browser directory only. On a miss, run the normal install. On a hit, still ensure operating-system dependencies exist. Never use a broad key that can restore binaries from a different Playwright release.

- name: Restore Playwright browser cache
  uses: actions/cache@v4
  with:
    path: ~/.cache/ms-playwright
    key: playwright-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- name: Install browser and system dependencies
  run: npx playwright install --with-deps chromium

Hashing the lockfile may invalidate more often than a package-only key, but it safely binds the browsers to dependency state. A custom key extractor can narrow invalidation once it is tested and maintained.

Capture traces and media according to failure value

Configure evidence at the Playwright level so every CI provider receives the same output. Traces on the first retry are valuable for comparing a fresh worker attempt; retaining on failure is useful when retries are disabled. Screenshots capture final visual state, while traces preserve chronology and network evidence. Video can help with motion or multi-step context but carries extra storage cost.

import { defineConfig } from '@playwright/test';

export default defineConfig({
  workers: process.env.CI ? 1 : undefined,
  retries: process.env.CI ? 1 : 0,
  outputDir: 'test-results',
  reporter: [['html', { outputFolder: 'playwright-report', open: 'never' }]],
  use: {
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
});

Do not enable every medium for every passing test without a clear use case. The artifact set should make common failures diagnosable while keeping upload, storage, and review manageable.

Upload evidence even when the test command fails

Artifact publication must run after a nonzero test exit. Use an always-run or not-cancelled condition and upload both the HTML report and test output so traces and attachments remain reachable.

- name: Run Playwright tests
  run: npx playwright test
- name: Upload Playwright evidence
  if: ${{ !cancelled() }}
  uses: actions/upload-artifact@v4
  with:
    name: playwright-evidence-${{ github.run_attempt }}
    path: |
      playwright-report
      test-results
    retention-days: 14

Include the workflow attempt in the artifact name so a rerun does not obscure the original failure. For sharded jobs, upload blob reports per shard and merge them downstream instead of publishing unrelated HTML reports.

Apply retention and access as security controls

Traces can contain rendered customer-like data, URLs, console messages, and request bodies. Videos and screenshots can expose account state. Use synthetic test identities, minimize secrets reaching the browser, and limit artifact access to the people who investigate failures.

Set retention according to normal investigation delay and compliance needs. Longer is not automatically safer or more useful. If a defect requires preservation beyond standard expiry, move that specific evidence into an approved incident system rather than extending every test artifact indefinitely.

Avoid printing environment variables or authentication tokens during browser launch debugging. Masking in CI logs may not cover values embedded in a trace attachment created by application code.

Diagnose CI environment failures by layer

"Executable does not exist" usually points to a package and browser mismatch, a missing install step, or an incorrect cache restore. Print the Playwright package version and image identifier, then compare them before reinstalling at random.

Browser crashes under load point toward memory, shared memory, process handling, or worker pressure. Reproduce with one worker and inspect container limits. Rendering-only differences suggest fonts, timezone, locale, GPU behavior, or baseline environment.

Missing artifacts are a workflow problem: check the upload condition, output paths, report cleanup, and whether the job was cancelled. A stale browser cache often appears immediately after a Playwright upgrade; inspect the key and cache source rather than weakening tests.

If tests pass in the container locally but fail on CI, compare container launch options, mounted files, network routing, CPU limits, secrets, and application endpoint. The same image is necessary for reproducibility, but host policy still matters.

Operational CI-hardening checklist

• Pin the Playwright package through the lockfile.
• Pin a matching container image when using Docker.
• Upgrade package, image, and browser expectations together.
• Use container process and shared-memory settings appropriate for browsers.
• Start CI with a conservative worker count.
• Cache package downloads while still running npm ci.
• Cache browser binaries only after measuring net benefit.
• Key any browser cache to OS and exact Playwright dependency state.
• Upload reports and test results after failures.
• Set scoped artifact access, names, and expiry.

Harden one layer at a time

First record the current package, browser, image, worker, and artifact inputs for a known run. Pin the runtime and add a compatibility check. Next stabilize worker capacity and remove browser-cache assumptions until timing data supports them. Finally, inject a controlled failing test and confirm its trace, screenshot, report, and attempt identity survive the pipeline. Reproducible execution plus retained evidence gives failures a stable context, which is the foundation for every later optimization.