Stop Playwright artifacts from becoming a storage bill

Your CI bill jumps after the browser suite is sharded, but the test count barely changed. Each shard is uploading traces, videos, screenshots, and an HTML report, including evidence for tests nobody will investigate. The problem is not that Playwright creates artifacts. The problem is that recording mode, upload scope, and expiration were never designed as one policy.

Know what each recording mode keeps

Playwright makes two separate decisions for traces and videos: when to record, and when to retain the result. Those decisions are easy to blur because one short string controls both.

on records every attempt and keeps every file. It is useful for a narrow reproduction but expensive as a default. retain-on-failure records every attempt, then removes the recording when that attempt passes. It saves persistent storage, although the browser and runner still pay the recording overhead during successful tests.

on-first-retry does something different. The initial attempt is not recorded. If the test retries, Playwright records the first retry and keeps that recording regardless of its outcome. This is a good way to capture flaky behavior without recording the whole green suite, but it cannot show what happened during the original failure.

Current Playwright versions offer additional trace and video modes, including retain-on-first-failure, retain-on-failure-and-retries, and on-all-retries. Use them only after writing down which attempt an investigator needs. More files are not automatically more evidence.

Videos deserve stricter treatment than traces in many suites. A trace contains action timing, DOM snapshots, logs, and network details that often identify the failing boundary. Video is excellent for animation, focus, drag-and-drop, and visual transition failures, but weak for a bad response payload or an incorrect selector. Paying for both on every test rarely improves triage.

Video files are finalized when the browser context closes. The Playwright Test fixtures handle that lifecycle. If a test creates a context manually and forgets to close it, a missing or incomplete video is a lifecycle problem, not proof that retention deleted the file.

Put the default and escape hatch in code

A useful policy has a frugal default and an explicit investigation profile. This configuration retains a trace for failed attempts, records video only on the first retry, and allows a focused run to capture everything.

import { defineConfig } from '@playwright/test';

const investigationRun = process.env.PW_ARTIFACT_PROFILE === 'investigate';

export default defineConfig({
  retries: process.env.CI ? 1 : 0,
  outputDir: 'test-results',
  reporter: [
    ['line'],
    ['html', { outputFolder: 'playwright-report', open: 'never' }],
  ],
  use: {
    trace: investigationRun ? 'on' : 'retain-on-failure',
    video: investigationRun ? 'on' : 'on-first-retry',
    screenshot: 'only-on-failure',
  },
});

The default answers two common questions. A first-attempt failure gets a trace, even if the retry passes. A retry gets a video, which is useful when timing or animation differs between attempts. A test that passes immediately leaves none of those large recordings behind.

The investigation profile is intentionally expensive. Run it against one file, one project, or a small grep selection:

PW_ARTIFACT_PROFILE=investigate npx playwright test tests/checkout.spec.ts --workers=1
du -sh test-results playwright-report

Do not switch that environment variable on for the whole matrix and forget it. Make the profile visible in the CI job name or run summary. A temporary diagnostic setting that silently becomes permanent is a common source of artifact growth.

The cost of this default is recording overhead from retain-on-failure. Every attempt is traced until Playwright knows the outcome. If runtime or ephemeral disk pressure matters more than the first failure, use on-first-retry for traces too. You will spend less during green runs, but a non-retried failure and the original attempt of a flaky test will have no trace.

Measure uploaded bytes, not test-result folders

The runner's output directory is only one part of the bill. CI systems charge or limit what the workflow uploads, often after compression. A job may upload test-results, playwright-report, and a blob report from each shard. The HTML report has a data directory for attachments, so a trace can appear in raw output and in report output depending on the workflow.

Measure the exact directories immediately before the upload step. Record total bytes, file count, test count, failed attempts, retries, projects, and shards. Without those denominators, a 40 percent increase could mean larger videos, more browser projects, or simply twice as many attempts.

A practical monthly estimate is:

storage GB-months = sum(uploaded GB x retained days) / 30

Add download or egress charges if the provider applies them. Compression ratios also differ: text-heavy traces and encoded video do not shrink by the same amount. Use the archive size reported by CI for cost forecasting, and use uncompressed file sizes to find which artifact class is growing.

Look at the distribution, not just the average. One hung 20-minute test can produce a video far larger than hundreds of normal failures. Sort files by size and inspect the top entries. If the same test appears repeatedly, the best cost fix may be a timeout or product synchronization fix rather than a new deletion rule.

Set a budget per useful unit, not only a global quota. Bytes per failed attempt and bytes per thousand executed tests remain comparable when the suite grows. Alerting only on total storage turns planned growth into noise, while a sudden jump in bytes per failure points directly to a recording or duplication change.

Also test the failure-burst case. A policy that looks cheap at a two percent failure rate can overwhelm upload time and storage when a shared outage fails half the suite. Decide whether the workflow should upload every identical failure, a representative sample, or all traces for a short incident window. Sampling saves money but can discard the one browser or shard with a different cause, so preserve result metadata for anything omitted.

Shard count is another multiplier. Each shard normally creates its own output and report data. When reports are merged, preserve shard identity until the merge succeeds, then upload the merged evidence once if raw shard archives have no separate recovery value. Deleting raw reports before a verified merge saves space but makes a broken merge unrecoverable.

Separate retention from recording

Playwright decides which files survive the test run. Your CI or object store decides how many days an uploaded archive survives. Treat those as different controls.

A pull-request failure is usually useful until the change is merged or abandoned. A release-candidate run may need a longer investigation window. A manually triggered diagnostic run can contain much more data but expire sooner because an engineer is already waiting for it. Set expiration by run class rather than using one organization-wide number.

Every longer window has a trade-off. It helps teams investigate late and compare recurring failures, but increases storage, privacy exposure, and the chance that authenticated page content remains accessible after it is useful. Traces can include DOM snapshots, request details, and console output. Videos can capture names, account data, and messages rendered by the test environment. Retention is therefore a security decision as well as a cost decision.

Document deletion behavior too. Expiration often removes the CI archive but not a copy exported to a defect tracker or object store. If evidence is replicated, every destination needs an owner and lifecycle. Otherwise the shortest configured expiration creates false confidence while an untracked copy remains available indefinitely.

Keep the retention rule near the upload configuration and document its owner. The test team owns diagnostic value. The platform team may own quotas and lifecycle rules. Security or compliance may own the maximum allowed window. A policy with no owner tends to drift toward "keep everything."

Diagnose a missing or oversized artifact

When a failure has no trace, start with attempt history. Was the failing run the initial attempt? Did the configured mode only record a retry? Was retries zero in that environment? The HTML report shows attempts and attachments, which makes this faster than searching folders by timestamp.

For a missing video, also check context closure. Built-in fixtures close their context at test end. Manually created contexts need an awaited close() call. If the process is killed by an outer CI timeout, Playwright may never get the chance to finalize the file.

For unexpected volume, compare these quantities between a normal and expensive run:

• retained traces per failed attempt
• videos per retry
• browser projects per test
• retries per failure
• report copies per shard
• the five largest individual files

This evidence distinguishes a mode mistake from a suite change. If every passing test has a trace, trace: 'on' is active somewhere, possibly in a project or command-line override. If only failures have files but the total doubled, the failure rate, duration, project matrix, or duplicate upload scope probably changed.

Do not respond by deleting artifacts inside a custom reporter unless there is no safer layer. Reporters and merge jobs may still need those paths. Prefer built-in recording modes first, then CI expiration, then verified post-merge cleanup.

Know when video or trace is the wrong purchase

Skip video for API-only tests and for assertions where rendered motion cannot explain the result. A response body, server log, or small text attachment will be cheaper and more precise.

Avoid always-on traces in a stable pull-request suite merely because storage is currently cheap. Recording adds runtime work and increases the amount of application data collected. Enable on for a bounded reproduction, then return to the default profile.

Do not shorten retention below the team's actual response time. An artifact that expires over the weekend before anyone can inspect it has zero diagnostic value, regardless of how little it costs.

Finally, do not preserve a video to compensate for weak assertions. A recording may show that the page looked correct while the test still accepted the wrong business result. Keep the artifact that helps explain a failed decision, and make the assertion strong enough to make that decision first.