Open a Playwright HTML report straight from a CI zip

A failed CI job leaves you a file named playwright-report.zip, but npx playwright show-report playwright-report.zip refuses to open it. The test run is not the problem: the archive contains the wrong report type, has an extra directory level, or was damaged during the handoff. All three cases look identical on a download page and need different fixes. Before rerunning a long suite, inspect what is actually inside the file.

Why the archive layout decides whether the report opens

The HTML reporter writes a directory, playwright-report by default. That directory has an index.html entry and everything the page needs to render test results and attachments. The show-report command can serve the directory directly:

npx playwright show-report playwright-report

It can also accept a zip archive. For that path, Playwright expects index.html at the top level of the archive. It extracts the archive to a temporary directory and serves the extracted report. The important contract is the path stored in the zip, not the name printed by the artifact system.

Consider these two packaging commands. They produce files with the same outer name, but only the second one creates the layout expected by show-report:

# Wrong for show-report: the directory name becomes part of every zip entry.
zip -qr playwright-report.zip playwright-report

# Correct: change into the report directory before adding its contents.
archive_path="$PWD/playwright-report.zip"
test ! -e "$archive_path"
(
  cd playwright-report
  zip -qr "$archive_path" .
)

The first archive contains playwright-report/index.html. The second contains index.html. A web browser could open either one after a person extracts it and navigates to the nested file, but the Playwright command has a stricter, documented input shape.

There is another common source of confusion. The blob reporter also produces zip files, with names such as report-<hash>.zip. Those archives are not compressed HTML folders. They hold serialized run data designed for merge-reports, especially when shards execute on separate CI workers. Renaming a blob archive to playwright-report.zip does not convert it. File extensions tell you how bytes were packaged, not which Playwright reporter created the contents.

The distinction matters when a team uploads both formats:

• An HTML report is for a person who wants to investigate a run now.
• A blob report is an intermediate artifact for a job that will combine several runs or generate a different reporter later.
• A trace archive belongs to one test attempt and opens in Trace Viewer. It is not a whole-suite HTML report.

A clean artifact name prevents most mistakes. Use names such as html-report-linux.zip, blob-report-shard-2.zip, and trace-checkout-retry-1.zip. Do not make the reviewer discover the format by trial and error.

The HTML reporter also has an attachmentsBaseURL option. Use it only when report attachments are deliberately hosted somewhere other than the report's data directory. It is not a repair for an archive that omitted local files. If the generated report refers to local attachments and the zip step excluded them, opening index.html successfully can still leave screenshots, traces, or videos unavailable.

Build a local handoff that is hard to package incorrectly

Start by making report generation explicit. CI should never try to open a browser after a failed test run, so configure the reporter with open: 'never'. Give the output directory a stable name that your packaging step can validate.

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['line'],
    ['html', { outputFolder: 'playwright-report', open: 'never' }],
  ],
  use: {
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
  },
});

Multiple reporters serve different audiences here. The line reporter keeps the CI log readable while the HTML reporter creates the investigation artifact. A failing test still makes the test command exit nonzero. Generating an HTML report does not turn a red job green.

Package only after checking the one file that defines the archive contract. This script stops before upload if the reporter never ran, the configured output path changed, or the report was deleted by a later cleanup step.

#!/usr/bin/env bash
set -euo pipefail

report_dir="${1:-playwright-report}"
archive="${2:-playwright-report.zip}"

if [[ ! -f "$report_dir/index.html" ]]; then
  echo "HTML report is missing: $report_dir/index.html" >&2
  exit 1
fi

mkdir -p "$(dirname "$archive")"
archive_path="$(cd "$(dirname "$archive")" && pwd)/$(basename "$archive")"
if [[ -e "$archive_path" ]]; then
  echo "Refusing to update an existing archive: $archive_path" >&2
  exit 1
fi
(
  cd "$report_dir"
  zip -qr "$archive_path" .
)

unzip -t "$archive_path" >/dev/null
if ! unzip -Z1 "$archive_path" | grep -Fxq 'index.html'; then
  echo "Archive root does not contain index.html: $archive_path" >&2
  exit 1
fi

echo "Validated HTML report archive: $archive_path"

The grep -Fxq flags are intentional. -F treats the value as plain text, -x requires the entire stored path to match, and -q avoids printing the archive listing. A loose search for index.html would also accept playwright-report/index.html, which is the mistake this check is supposed to catch.

Run the same validation after downloading the artifact, not just before uploading it. The producer check proves what CI created. The consumer check proves that the file a reviewer received has the same usable shape.

./scripts/package-html-report.sh playwright-report artifacts/playwright-report.zip

unzip -t artifacts/playwright-report.zip
unzip -Z1 artifacts/playwright-report.zip | sed -n '1,25p'
npx playwright show-report artifacts/playwright-report.zip

Keep the terminal running while reviewing. show-report starts a local server, and closing the command stops that server. By default the host is localhost; the documented default port is 9323, or another available port when that one is occupied. Pass --port when a fixed local port helps, but do not bind the server to a network interface merely to share a report. Upload the artifact to an access-controlled location instead.

Three zip failures that require different repairs

The fastest diagnosis is to classify the archive before changing the test configuration. These worked examples cover the failures that repeatedly waste CI reruns.

The report has one extra directory level

A pipeline uploads a directory using an artifact action. A teammate downloads a zip generated by that service and then wraps it in another zip for email. The final archive now contains a path such as downloaded-artifact/playwright-report/index.html.

Run this without extracting anything:

archive="${1:?usage: inspect-report.sh <archive.zip>}"

root_index_count="$(unzip -Z1 "$archive" | awk '$0 == "index.html" { count++ } END { print count + 0 }')"
nested_indexes="$(unzip -Z1 "$archive" | awk '$0 ~ /(^|\/)index\.html$/ && $0 != "index.html" { print }')"

printf 'root index entries: %s\n' "$root_index_count"
if [[ -n "$nested_indexes" ]]; then
  printf 'nested index entries:\n%s\n' "$nested_indexes"
fi

When the root count is zero and a nested path is printed, the report itself may be fine. Extract it into a fresh directory, locate the directory containing index.html, and repackage that directory's contents. Do not solve this by teaching every reviewer a different extraction ritual. Fix the producing pipeline so the next artifact has a stable contract.

The trade-off is a small packaging step and another tool dependency (zip and unzip) on the runner. That cost is usually lower than handling inconsistent artifact wrappers across CI providers. If your artifact service preserves a directory exactly and your team always reviews through its web UI, an explicit zip may be unnecessary.

The file is a blob report with an HTML-sounding name

Suppose four shards each run with reporter: 'blob'. The artifact download directory contains several report-*.zip files. Running show-report on any one of them is the wrong operation because there is no HTML report to serve.

Put all blob archives from the same logical run in one directory, then ask Playwright to merge that directory and generate HTML output:

mkdir -p downloaded-blob-reports
# Copy or download every shard's report-*.zip into this directory first.

npx playwright merge-reports --reporter=html downloaded-blob-reports

test -f playwright-report/index.html
archive_path="$PWD/merged-html-report.zip"
(
  cd playwright-report
  zip -qr "$archive_path" .
)

npx playwright show-report "$archive_path"

Do not unzip each blob file before merging. The merge-reports command expects a directory containing the blob report archives. Also keep the Playwright version and project configuration consistent across shards. A format error during merge is upstream of HTML packaging; changing how you zip the final directory cannot repair incompatible blob inputs.

The cost of this approach is delayed feedback. A shard cannot publish the final cross-shard report by itself. A dedicated merge job has to wait for all shard artifacts, download them, and run even when one shard fails. The benefit is a single report with the whole test matrix instead of several partial stories.

The zip passes a name check but is truncated

An interrupted artifact download can leave a file with the expected name and a nonzero size. Looking only for playwright-report.zip is not evidence that the central directory and compressed entries are readable.

unzip -t reads and checks every archive entry. Its exit status is the useful signal, so keep it in an automated validation step. If that command fails after download but passed before upload, compare checksums from both sides of the handoff.

archive="playwright-report.zip"

unzip -t "$archive"
sha256sum "$archive" > "$archive.sha256"

# On the receiving machine, copy both files into the same directory.
sha256sum --check "$archive.sha256"

On macOS, shasum -a 256 is the commonly available equivalent. Pick one command for the environments your team supports and record the digest as artifact metadata. A mismatch means the bytes changed. It does not implicate the reporter, the tests, or the application.

Checksum management adds ceremony, and some CI artifact services already verify integrity internally. Use an explicit checksum when files cross systems, are copied to long-term storage, or pass through a manual handoff. Skip it when the archive stays inside one provider and the provider already gives a trustworthy integrity guarantee.

Diagnose the report before opening a browser

Opening the report is the last diagnostic step, not the first. A few shell checks separate packaging failures from rendering failures in seconds.

First, confirm the installed CLI version. Zip support and report formats evolve with Playwright, and a globally installed executable can differ from the version in the project. Use the project-local command through npx:

npx playwright --version
npx playwright show-report --help

If the help text on that installed version does not describe the input you intend to use, upgrade the project deliberately and run its normal compatibility checks. Do not silently install the latest package on a CI worker during report review. That makes the viewer version depend on network state and can separate it from the version that generated the artifact.

Record the generating Playwright version beside the archive. A plain text file in the CI artifact is enough:

npx playwright --version > artifacts/playwright-version.txt

When a teammate's older checkout cannot open a report that passes structure and integrity checks, use the project revision and locked Playwright version that produced it before modifying the archive. This costs one small metadata file and avoids guessing about reader compatibility. It also distinguishes an archive created by a different pipeline revision from one corrupted in transit. Do not put tokens, environment dumps, or the full dependency tree in that metadata file; the package version and source revision normally answer the compatibility question.

Next, inspect stored paths:

unzip -Z1 playwright-report.zip | sort | sed -n '1,80p'

Look for these specific signals:

• A line exactly equal to index.html identifies the required root document.
• A single leading directory on every entry identifies an extra wrapper.
• Names dominated by blob-report metadata rather than index.html identify the wrong report type.
• An index.html entry with missing referenced attachment content points to an incomplete packaging include, not a server startup failure.
• More than one report tree in the same archive means the packaging glob was too broad. Choose one report rather than hoping the command selects the right root.

Then test archive integrity with unzip -t. Only after structure and integrity pass should you run show-report. If the server starts but a screenshot or trace link is missing, inspect the browser's network panel and the report's attachment layout. A root-level index.html proves the entry page exists; it does not prove that every optional attachment was uploaded.

The HTML report and Trace Viewer answer different questions. The report shows projects, retries, steps, errors, and attachments across the run. Opening a trace attachment gives a timeline for one recorded attempt. If the report loads and the trace does not, retain and inspect the trace archive separately. Repackaging the outer HTML zip does not reconstruct a trace that CI never retained.

A fourth near-miss appears when index.html loads but every attachment link returns a missing-file response. Check whether the reporter was configured with attachmentsBaseURL. That option is correct when the data content was uploaded to a separate, durable location and the URL still points there. It is wrong when someone copied a configuration from a hosted-report setup into a portable-report job. In the latter case, rebuild the report with local attachments or package the referenced local data directory. Do not rewrite URLs inside generated HTML as a repair. The generated files form one report, and manual edits make later reproduction almost impossible.

Also check whether a cleanup job ran between report generation and packaging. A common sequence uploads traces separately, removes test-results, and then creates the HTML zip. Depending on which attachments the report references and where they were stored, the entry page can survive while evidence disappears. Package the complete HTML output immediately after the reporter finishes, validate that archive, and only then run retention cleanup. This ordering costs temporary disk space on the runner, but it gives the producer one atomic artifact to reason about.

Treat reports downloaded from an unknown source as untrusted files. Open artifacts produced by your own controlled CI in an isolated review environment, particularly when runs execute code from external contributions. A report can contain application text, URLs, screenshots, source snippets, and other material supplied by the tested revision. The fact that show-report serves it on localhost does not turn its contents into trusted input.

Avoid asserting exact archive byte sizes. Test counts, screenshots, traces, source snippets, and videos all change output volume. A fixed minimum size either rejects a valid small report or accepts a large corrupted one. Structural checks and integrity checks express the actual requirement.

Separate an outer artifact wrapper from missing report output

Two downloaded files can produce the same complaint about a missing root index.html even though only one contains a usable report. Artifact services often wrap uploaded files in a download archive. If CI uploaded an already zipped report, the downloaded outer file can contain one member named playwright-report.zip. Passing that outer file to show-report fails because its root document is inside another archive. By contrast, a job that never generated HTML may upload logs and screenshots under an HTML-sounding artifact name. There is no inner report to unwrap.

The archive listing separates them precisely. A portable HTML report has exactly one stored path equal to index.html. A packaging-root mistake has zero exact matches and a path ending in /index.html, such as playwright-report/index.html. A double wrapper has zero exact and nested index matches, but lists another zip file as a member. A missing-report artifact has neither an index path nor an inner report archive; its entries belong to some other output. These cases can share the same outer filename and the same initial show-report error, so the download-page label is a misleading value.

Integrity output answers a different question. A zero exit status from unzip -t says the archive structure and compressed entries passed that tool's checks. It does not say the archive is HTML, that index.html sits at the root, or that attachments describe the intended run. Conversely, a failed integrity check makes all layout interpretation provisional because the listing itself may be incomplete. Run integrity and root-layout checks, and preserve both results in the handoff.

The generating version file is useful only when it travels beside the matching archive. A perfectly valid version string copied from another shard or rerun can send a reviewer to the wrong checkout. Treat run, run attempt, shard, source revision, report kind, and archive digest as one provenance record. A healthy value is one coherent set produced by the same packaging step. A misleading set mixes an older archive with current metadata while every individual file remains readable.

An existing suite should land this path in producer order. First ensure the HTML reporter writes its directory on a normal pass and an ordinary test failure. Next package that directory immediately after the runner finishes, before any cleanup or second invocation can reuse its output location. Add layout and integrity validation in advisory mode on a canary job, then upload exactly the validated archive. After consumers can repeat the validation on download, make producer validation required. Only then remove older directory artifacts or reviewer instructions.

The first rollout failure is often a job that was cancelled or killed before the reporter completed. An unconditional packaging step then reports only that index.html is absent and can distract from the termination that prevented its creation. Keep the original test or executor status visible, and have the packaging log distinguish "report expected but absent" from "runner did not reach report completion." Do not manufacture an empty HTML report to satisfy the artifact step. An empty shell hides the only useful fact about that run.

Packaging also spends measurable resources even when the report is valid. The uncompressed directory and zip coexist temporarily, so peak disk use can approach the size of both representations plus compression workspace. Compression consumes CPU after the test run, and a merge job adds download and generation latency before reviewers see results. Those costs are acceptable when a portable atomic handoff is required. If an authenticated static host already serves the directory reliably, a second archive and its validation pipeline may add maintenance without improving access.

Ownership follows the failed contract. The test-infrastructure owner is responsible for reporter configuration and the presence of a complete HTML directory. The CI owner is responsible for packaging from the correct working directory, artifact wrapping, upload, and retention. The receiving tool or developer-environment owner is responsible for using a compatible project-local viewer after structure and integrity pass. A handoff should include the outer and inner filenames, the first relevant archive entries, integrity exit status, exact root-index count, archive digest from producer and consumer, Playwright version, run and shard identity, and whether the HTML directory existed before packaging. "The zip will not open" is not enough to route the defect.

These checks do not prove that the report's attachments are semantically correct. A screenshot from a previous run can be packaged under a valid data path, pass integrity, and render successfully. Attempt identity, source revision, and report provenance still need review. Zip validation proves transport shape, not evidentiary truth.

Wire the artifact into CI without hiding the test result

A report pipeline has two obligations that can pull in opposite directions. It must upload evidence even when tests fail, and it must preserve the failing exit status. The safest pattern uses separate steps. Let the test step fail, then mark packaging and upload steps to run regardless of earlier failure.

# .github/workflows/e2e.yml (relevant steps)
- name: Run Playwright tests
  run: npx playwright test

- name: Package HTML report
  if: ${{ always() }}
  shell: bash
  run: |
    set -euo pipefail
    test -f playwright-report/index.html
    mkdir -p artifacts
    archive_path="$PWD/artifacts/playwright-report.zip"
    test ! -e "$archive_path"
    (cd playwright-report && zip -qr "$archive_path" .)
    unzip -t "$archive_path" >/dev/null
    unzip -Z1 "$archive_path" | grep -Fxq 'index.html'

- name: Upload HTML report
  if: ${{ always() }}
  uses: actions/upload-artifact@v4
  with:
    name: playwright-html-report
    path: artifacts/playwright-report.zip
    if-no-files-found: warn
    retention-days: 14

The first step's failure remains a job failure. always() allows evidence handling to continue; it does not overwrite the earlier result. Avoid continue-on-error: true on the test step unless another explicit step restores the original exit code. Teams regularly create accidental green pipelines by capturing a failing status and never returning it.

If packaging fails because index.html is absent, that failure is useful. It tells you the expected reporter output was not created. The artifact upload uses a warning for a missing file so it does not obscure the more informative packaging or test failure. Your CI conventions may prefer a hard upload failure, but decide that deliberately.

For sharded runs, upload blob reports from every shard and run a separate merge job with if: always(). Generate HTML once in that job, validate it, then upload the final zip. Do not have all shards write to the same remote archive name. Artifact systems differ in how they handle duplicate names, and overwriting one shard produces a report that looks complete until someone notices missing projects.

Report retention is a product and privacy decision. Traces, screenshots, request data, and attachments can contain personal data or session material. A longer retention window improves historical debugging but expands storage cost and exposure. Set the shortest period that meets your incident and audit needs, and keep access narrower than access to public build logs.

Know when a zip is the wrong delivery format

Use a zip when a reviewer needs a portable, immutable snapshot of one run. It is especially helpful when the CI provider's artifact browser mangles nested files, when reports move into incident storage, or when a teammate must inspect a failed run without access to the original worker.

Do not add a zip merely because the HTML reporter exists. Several alternatives are better in specific situations:

• Keep the directory when a static artifact host can serve it with the correct content types and access controls.
• Keep blob reports when a merge job has not run yet. Converting each shard to HTML early loses the straightforward cross-shard merge workflow.
• Share a trace archive when the question concerns one attempt's action timeline. A whole report adds navigation without adding evidence.
• Use the CI log for a small unit-style failure with a complete stack trace and no browser artifacts. Downloading and serving an HTML report costs more time than reading the error.
• Regenerate the report from retained blob inputs when the presentation format needs to change. Do not hand-edit generated HTML.

Serving a report on 0.0.0.0 from a developer laptop is also a poor sharing mechanism. The report can expose source snippets, URLs, screenshots, and attachment content to anyone who can reach that port. Keep the default local host, or publish through an authenticated artifact system.

Finally, do not treat successful rendering as proof that the test run was sound. The report is evidence emitted by the runner. It cannot tell you that assertions covered the right business risk, that a retry policy is acceptable, or that missing tests should have run. Review project names, expected test counts, skipped tests, retries, and attachments in the report. Archive correctness gets you to that review; it is not a substitute for it.