Stop hidden network calls from escaping HAR replay

A checkout test starts failing with net::ERR_FAILED the day an analytics request is added. Every response the assertion uses is still in the HAR, but the archive has no entry for the new call. The choice between aborting that miss and allowing live traffic now determines whether the test exposes drift or quietly becomes an integration test.

Understand what Playwright means by a HAR miss

The word notFound is easy to misread. It does not mean that the application received an HTTP 404. It means Playwright intercepted a request inside the URL scope registered by page.routeFromHAR() and could not select a matching entry from the archive. With notFound: 'abort', the browser gets a failed request. With notFound: 'fallback', Playwright sends the request to the network and lets the real server decide what happens.

abort is the default, but relying on the default hides an important test policy in setup code. Write it explicitly. A reviewer should be able to tell whether an unexpected call is forbidden or intentionally live without checking a versioned API reference.

HAR replay does more than look for a similar path. Playwright matches the request URL and HTTP method strictly. For a POST request, it also matches the payload strictly. If several archive entries remain eligible, the entry with the most matching request headers is selected. That produces several common misses that look surprising until the actual request is inspected.

That last row causes some of the most misleading green tests. Suppose the route uses url: '**/api/orders/**', while the application starts calling /graphql. Even with notFound: 'abort', the GraphQL request is outside the HAR route and can go to the network. The abort policy is strict only inside the boundary you registered. A narrow filter gives precise ownership, but it is not an offline switch for the whole page.

The opposite mistake is using url: '**/*' and then wondering why a new font, analytics pixel, source map, or browser-specific asset kills the scenario. Broad routing makes every incidental request part of the fixture contract. That can be correct for a genuinely hermetic test, but it creates maintenance work whenever the page shell changes. Most product-flow tests are easier to own when the route covers a deliberate API namespace and the handling of other resources is stated separately.

A miss also says nothing about why the archive lacks a candidate. The fixture may be old. A cache-busting query may have changed. A POST serializer may now emit a different body. The request may now target a different API host after an environment configuration change. An authentication redirect may have introduced a different URL. Treat those as different changes even though abort gives them the same browser-level symptom.

Browser-generated CORS preflight OPTIONS requests are not interceptable by Playwright routing. routeFromHAR() therefore cannot match or miss them, notFound does not govern them, and they do not appear as routed requests in Trace Viewer. Test CORS behavior at a different layer, such as a live browser integration test without routing or a server policy test.

Recording and replaying are separate modes. When update: true is passed to routeFromHAR(), Playwright obtains actual network information and writes the updated archive when the browser context closes. It is not a safe way to make ordinary replay tests learn missing traffic as they run. A test that updates its own fixture can accept the behavior it was supposed to challenge, and parallel workers can compete to write the same file. Keep fixture generation in an explicit, reviewed workflow. Run assertions against a read-only archive.

Service workers create another boundary. Playwright documents that requests intercepted by a service worker are not served from the HAR. If the purpose of the test is API replay rather than service-worker behavior, create the browser context with serviceWorkers: 'block'. If the service worker itself is under test, HAR routing cannot be treated as complete evidence for those intercepted requests.

This is the practical meaning of Playwright HAR notFound behavior: it is a decision about what may happen after a routed request fails archive matching. It is not a retry setting, a server-status assertion, or a general guarantee that no network connection can occur.

Prove abort behavior before it surprises CI

A useful abort test needs evidence from both sides of the boundary. The browser should report a failed request, and the upstream server should show that it was never contacted. The following lab records two calls into a temporary HAR, resets its counters, and exposes a third endpoint that is deliberately absent. It uses only Playwright Test and Node's standard library.

Save this helper as tests/support/har-lab.ts. Each worked example below creates its own server and archive, so it does not depend on a public demo service or a pre-existing fixture.

import { once } from 'node:events';
import { createServer, type ServerResponse } from 'node:http';
import type { AddressInfo } from 'node:net';
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import type { Browser } from '@playwright/test';

type HitCounts = {
  products: number;
  orders: number;
  recommendations: number;
  unknown: number;
};

function sendJson(
  response: ServerResponse,
  status: number,
  body: unknown,
): void {
  response.writeHead(status, {
    'cache-control': 'no-store',
    'content-type': 'application/json',
    'x-har-lab': 'live',
  });
  response.end(JSON.stringify(body));
}

export async function createHarLab(browser: Browser) {
  const hits: HitCounts = {
    products: 0,
    orders: 0,
    recommendations: 0,
    unknown: 0,
  };

  const server = createServer(async (request, response) => {
    const url = new URL(request.url ?? '/', 'http://127.0.0.1');

    if (url.pathname === '/') {
      response.writeHead(200, { 'content-type': 'text/html' });
      response.end('<!doctype html><title>HAR policy lab</title>');
      return;
    }

    if (request.method === 'GET' && url.pathname === '/api/products') {
      hits.products += 1;
      sendJson(response, 200, [{ id: 'p-1', name: 'Keyboard' }]);
      return;
    }

    if (request.method === 'POST' && url.pathname === '/api/orders') {
      hits.orders += 1;
      let rawBody = '';
      for await (const chunk of request) rawBody += chunk.toString();
      const order = JSON.parse(rawBody) as { sku: string; quantity: number };
      sendJson(response, 201, { id: 'o-1', ...order });
      return;
    }

    if (request.method === 'GET' && url.pathname === '/api/recommendations') {
      hits.recommendations += 1;
      sendJson(response, 200, [{ id: 'p-2', name: 'Mouse' }]);
      return;
    }

    if (url.pathname.startsWith('/api/')) {
      hits.unknown += 1;
      sendJson(response, 404, { error: 'No API route for this path' });
      return;
    }

    response.writeHead(404).end();
  });

  server.listen(0, '127.0.0.1');
  await once(server, 'listening');

  const address = server.address() as AddressInfo;
  const origin = `http://127.0.0.1:${address.port}`;
  const directory = await mkdtemp(join(tmpdir(), 'har-policy-'));
  const harPath = join(directory, 'api.har');

  const context = await browser.newContext({ serviceWorkers: 'block' });
  const page = await context.newPage();

  try {
    await page.routeFromHAR(harPath, {
      url: `${origin}/api/**`,
      update: true,
      updateContent: 'embed',
      updateMode: 'minimal',
    });
    await page.goto(origin);

    const statuses = await page.evaluate(async () => {
      const products = await fetch('/api/products');
      await products.text();

      const order = await fetch('/api/orders', {
        method: 'POST',
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ sku: 'A-100', quantity: 1 }),
      });
      await order.text();

      return [products.status, order.status];
    });

    if (statuses[0] !== 200 || statuses[1] !== 201) {
      throw new Error(`Could not record baseline HAR: ${statuses.join(', ')}`);
    }
  } finally {
    await context.close();
  }

  hits.products = 0;
  hits.orders = 0;
  hits.recommendations = 0;
  hits.unknown = 0;

  return {
    harPath,
    hits,
    origin,
    async close(): Promise<void> {
      await new Promise<void>((resolve, reject) => {
        server.close(error => (error ? reject(error) : resolve()));
      });
      await rm(directory, { force: true, recursive: true });
    },
  };
}

Now add tests/har-abort.spec.ts. The target URL is inside the HAR route's scope, but the helper never recorded it.

import { expect, test } from '@playwright/test';
import { createHarLab } from './support/har-lab';

let lab: Awaited<ReturnType<typeof createHarLab>>;

test.beforeAll(async ({ browser }) => {
  lab = await createHarLab(browser);
});

test.afterAll(async () => {
  await lab.close();
});

test('an aborted HAR miss never reaches the upstream server', async ({ page }) => {
  await page.goto(lab.origin);
  await page.routeFromHAR(lab.harPath, {
    url: `${lab.origin}/api/**`,
    notFound: 'abort',
  });

  const target = `${lab.origin}/api/recommendations`;
  const failedRequestPromise = page.waitForEvent('requestfailed', {
    predicate: request => request.url() === target,
  });

  const fetchResult = await page.evaluate(async url => {
    try {
      const response = await fetch(url);
      return { ok: true, status: response.status };
    } catch (error) {
      return {
        ok: false,
        errorName: error instanceof Error ? error.name : 'Unknown',
      };
    }
  }, target);

  const failedRequest = await failedRequestPromise;
  console.info(
    '[requestfailed]',
    failedRequest.method(),
    failedRequest.url(),
    failedRequest.failure()?.errorText,
  );

  expect(fetchResult).toEqual({ ok: false, errorName: 'TypeError' });
  expect(failedRequest.failure()?.errorText).toBeTruthy();
  expect(lab.hits.recommendations).toBe(0);
});

On Chromium, the useful line normally has this shape. The port changes on every run, and the final browser error text can differ across engines, so the test asserts that an error exists instead of freezing a Chromium-only string.

[requestfailed] GET http://127.0.0.1:53142/api/recommendations net::ERR_FAILED

Three assertions carry different information. The rejected fetch() proves the page did not receive an HTTP response. The requestfailed event gives the failed method, URL, and browser error. The zero server-hit count proves fallback did not occur. A locator timeout by itself would prove none of those things; it would only show that the UI never reached its expected state.

The cost of abort appears immediately in this example. Any new request inside ${lab.origin}/api/** breaks the scenario until the HAR or route boundary is reviewed. That is useful pressure when the archive is meant to describe the complete API contract. It is needless churn when optional telemetry, feature-discovery calls, or environment-specific endpoints share the same namespace. The route scope has to be designed with the policy.

Allow fallback only when live traffic is part of the test

Fallback is appropriate when a replayed fixture and a live dependency are intentionally combined. A common case is a stable catalog response from HAR plus a recommendation service being exercised in a test environment. The distinction must be observable. Otherwise a stale archive can shed more and more calls to the network while the test remains green.

Save the next file as tests/har-fallback.spec.ts. It proves that the recorded products call comes from the archive and the unrecorded recommendations call reaches the server.

import { expect, test } from '@playwright/test';
import { createHarLab } from './support/har-lab';

let lab: Awaited<ReturnType<typeof createHarLab>>;

test.beforeAll(async ({ browser }) => {
  lab = await createHarLab(browser);
});

test.afterAll(async () => {
  await lab.close();
});

test('fallback mixes a replayed response with an intentional live call', async ({ page }) => {
  await page.goto(lab.origin);
  await page.routeFromHAR(lab.harPath, {
    url: `${lab.origin}/api/**`,
    notFound: 'fallback',
  });

  const products = await page.evaluate(async () => {
    const response = await fetch('/api/products');
    return { status: response.status, body: await response.json() };
  });

  expect(products).toEqual({
    status: 200,
    body: [{ id: 'p-1', name: 'Keyboard' }],
  });
  expect(lab.hits.products).toBe(0);

  const target = `${lab.origin}/api/recommendations`;
  const liveResponsePromise = page.waitForResponse(target);
  const recommendations = await page.evaluate(async url => {
    const response = await fetch(url);
    return { status: response.status, body: await response.json() };
  }, target);
  const liveResponse = await liveResponsePromise;

  expect(recommendations).toEqual({
    status: 200,
    body: [{ id: 'p-2', name: 'Mouse' }],
  });
  expect(liveResponse.headers()['x-har-lab']).toBe('live');
  expect(lab.hits.recommendations).toBe(1);
});

This test names its mixed-network intent in the title and verifies both halves. In a real system, an upstream access log, request correlation ID, or disposable test-server counter can provide the same proof. Checking only the browser response is insufficient because a HAR response and a live response can have identical status, headers, and body.

Fallback does not append the new request to the archive. The next run will send the same miss to the network again. Only update mode records actual traffic, and that archive is written when the context closes. Keeping those behaviors separate prevents an assertion run from silently changing its own fixture.

The live portion brings normal integration-test costs. It needs credentials, DNS, a reachable environment, seeded data, and an upstream service that can tolerate the call. Its latency is added to every run. Rate limits and deployment incidents can fail the test even when the application code is unchanged. If the request mutates data, a fallback can create duplicate orders or messages during retries. Those are not theoretical trade-offs; they determine whether this policy is safe in parallel CI.

A safer mixed design is often to narrow the HAR route to the endpoints that truly belong to the fixture:

await page.routeFromHAR('hars/catalog.har.zip', {
  url: '**/api/catalog/**',
  notFound: 'abort',
});

Calls outside that pattern are not HAR misses. They follow normal browser networking or another route registered by the test. This makes ownership clearer than applying fallback to **/api/**, but it also means the test no longer prevents an accidental call outside the catalog namespace. Add explicit monitoring for dependencies whose live use matters.

Do not use fallback merely to stop a flaky replay test from failing. First identify the missing request. If it should be stable and deterministic, update the fixture through review. If it is irrelevant to the scenario, move it outside the replay boundary or handle it with a dedicated route. If it is a required live integration, name that dependency in the test and collect evidence that it was contacted.

Tell a replay miss from a real HTTP failure

A backend 404 and an aborted HAR miss can both leave the page showing “Could not load recommendations.” They are different failures at the protocol layer. An HTTP 404 has a Response, a status code, headers, and usually a body. An aborted miss has no response at all. Playwright emits requestfinished for the 404 and requestfailed for the aborted request.

The distinction matters because HTTP error responses are completed requests. A fetch() call resolves for a 404, with response.ok set to false. It rejects for a transport failure such as an aborted HAR miss. Page navigation follows the same principle: valid HTTP statuses do not make page.goto() throw simply because the status is 404 or 500.

The third worked example uses fallback to reach a path that the lab server answers with 404. Save it as tests/har-http-404.spec.ts.

import { expect, test } from '@playwright/test';
import { createHarLab } from './support/har-lab';

let lab: Awaited<ReturnType<typeof createHarLab>>;

test.beforeAll(async ({ browser }) => {
  lab = await createHarLab(browser);
});

test.afterAll(async () => {
  await lab.close();
});

test('a live 404 is a response, not an aborted HAR miss', async ({ page }) => {
  await page.goto(lab.origin);
  await page.routeFromHAR(lab.harPath, {
    url: `${lab.origin}/api/**`,
    notFound: 'fallback',
  });

  const target = `${lab.origin}/api/retired-product`;
  let sawRequestFailure = false;
  page.on('requestfailed', request => {
    if (request.url() === target) sawRequestFailure = true;
  });

  const responsePromise = page.waitForResponse(target);
  const finishedPromise = page.waitForEvent('requestfinished', {
    predicate: request => request.url() === target,
  });

  const result = await page.evaluate(async url => {
    const response = await fetch(url);
    return {
      ok: response.ok,
      status: response.status,
      body: await response.json(),
    };
  }, target);

  const response = await responsePromise;
  await finishedPromise;

  expect(result).toEqual({
    ok: false,
    status: 404,
    body: { error: 'No API route for this path' },
  });
  expect(response.status()).toBe(404);
  expect(sawRequestFailure).toBe(false);
  expect(lab.hits.unknown).toBe(1);
});

The event record gives a reliable first split during triage:

The third column is the important near-miss. With fallback enabled, a genuine network outage can print the same net::ERR_FAILED family of message as an aborted replay miss. The configuration and upstream evidence separate them. If the URL is absent from the HAR and fallback is active, check the environment's DNS, proxy, certificate, and service logs. If abort is active and the server has no request, inspect archive matching before touching timeouts.

Capture a trace on the first attempt rather than relying on a retry:

npx playwright test tests/har-abort.spec.ts --trace on
npx playwright show-trace path/to/trace.zip

In Trace Viewer, select the action that triggered the call, open the Network tab, and filter by the endpoint. Compare the complete URL, method, and POST body with the recording. An aborted miss has no HTTP status or response body to inspect. A live 404 has both. The Network list also shows whether the expected POST itself was sent and whether it completed with an HTTP response.

A trace cannot always prove that a response came from the archive rather than a live service, especially when both return the same bytes. Correlate it with server access logs or a controlled counter, as the examples do. That extra evidence is essential during a migration from fallback to abort because a passing browser assertion may conceal live traffic.

Several near-misses deserve separate checks:

• A changed POST body can miss even when the method and URL look identical in a short log line. Expand the request in Trace Viewer and compare its posted data. JSON fields added by a new client version, a timestamp, or a regenerated idempotency token can change the match.

• A request outside the url filter is not governed by notFound. If it reaches the server while abort is configured, compare the full URL with the route pattern before reporting a Playwright defect.

• A service worker can satisfy or initiate traffic outside HAR interception. Reproduce with a context that blocks service workers. If the symptom disappears, decide whether the test owns the service-worker path or the API path rather than leaving the difference implicit.

• A missing or unreadable HAR file fails during route setup, before the application action. That is fixture packaging or path resolution, not a request miss. Remember that a relative HAR path is resolved from the current working directory, which can differ between a developer shell and a CI job.

• A response already recorded as 404 can replay successfully from the HAR. Its status still says 404, but it is not a miss because Playwright found an entry and fulfilled it. Assert the product's handling of that status just as you would for any other mocked response.

A good failure message should include the route policy, filter, request method, full URL, and whether a response existed. “Recommendations did not load” sends the next engineer toward UI waits. “HAR abort: GET /api/recommendations had no matching entry and emitted requestfailed” sends them toward the archive and the network boundary.

Roll the policy through an existing suite

Changing a mature suite from fallback to abort can expose dozens of calls at once. Flipping the option globally gives a long list of red tests but little information about which misses are defects. Roll it out by endpoint ownership and preserve the first-failure evidence.

1. Inventory every HAR route. Record whether it is installed on a page or browser context, its url filter, its archive path, its current notFound value, and whether any setup uses update: true. An omitted value currently means abort, but make the policy explicit while reviewing each call. Context-level routing deserves extra attention because it covers every page in that context.

2. Observe current live escapes. For routes using fallback, run representative scenarios with tracing and upstream access logging. Build a list of requests that were not served by the archive. A green report does not tell you that list. Include method, complete URL, POST body category, owning team, and whether the call mutates state.

3. Classify each request by purpose. Stable responses required to isolate the UI belong in the HAR. Live services that are part of an integration claim should remain live and be named in assertions. Incidental analytics should not inherit a broad business-API policy. Unknown calls should fail review rather than being waved through as “probably harmless.”

4. Separate recording from verification. Give fixture updates a dedicated command or test project, run it with a single writer, and close the browser context so the HAR is actually written. Review the resulting request and response changes like test data. Do not enable update mode as a conditional rescue path after replay fails. That changes the source of truth during the assertion.

5. Start with a narrow abort boundary. Move one stable API family at a time to an explicit abort policy. Keep traces on for the first CI runs and watch access logs for unexpected calls. Expanding from **/api/catalog/** to **/api/** is a separate change because it transfers ownership of more endpoints to the archive.

6. Exercise variants that affect matching. Include alternate query values, empty results, authentication states, POST payloads, redirects, and request methods that the application can produce. One happy-path HAR may replay perfectly while every boundary case falls through. A test matrix should prove which variants are intentionally recorded and which are rejected.

7. Remove temporary fallback deliberately. Put an owner and expiry on any fallback kept during migration. Before removing it, verify that the server sees no traffic from the replay-only project. A week of green runs with no live hits is stronger evidence than a single successful local run.

Archive updates need their own review discipline. updateMode: 'minimal' keeps only information needed for routing and omits fields such as sizes, timing, cookies, security details, and page data that replay does not use. That usually reduces noisy diffs, but it also means the file is unsuitable when a reviewer expects a full forensic HAR. updateContent: 'attach' stores response resources separately or inside a ZIP archive, while embed stores content in the HAR. Attachments keep the JSON smaller but add files and make manual review less direct. Embedded bodies are convenient to inspect but can make a text HAR large.

Whichever content mode you choose, treat HAR files as captured traffic. They can contain authorization headers, cookies, account identifiers, and response data. Use synthetic accounts, redact secrets through a controlled process, and prevent fixture-generation jobs from pointing at production by accident. A fallback policy increases that risk because an unrecorded URL can contact whatever host the application currently uses.

Parallelism changes the cost profile. Read-only replay can be shared across workers. Updating one archive from several contexts creates a write race and can make the final fixture depend on test completion order. Use a single fixture-generation worker or separate output paths, then promote one reviewed archive. The same rule applies locally when a watch process and a developer command might update the file at the same time.

The policy trade-offs are concrete:

Do not report the migration complete merely because all tests are green under abort. Check that important negative responses are represented, that a changed POST payload fails as intended, and that the application still makes the expected calls. A page that stops requesting the API can also look quiet. Product assertions plus request evidence prevent absence of activity from becoming a false pass.

Know when HAR replay is the wrong boundary

Use a live integration test when the claim is about the deployed service, its authentication, or its current contract. Replaying yesterday's response cannot prove that today's server accepts the request. HAR can still support faster UI coverage elsewhere, but it should not replace the one test whose job is to cross the real boundary.

Avoid fallback for non-idempotent production operations. A missing POST entry must never become permission to create an order, send an email, charge a card, or delete a record on whichever host is configured. Route such calls to an isolated test service, provide a deliberate mock, or abort them. Retry behavior makes this especially dangerous because one visible test attempt can cause several server-side effects.

Do not use replay to measure performance, caching, compression, connection reuse, or server timing. A recorded response removes the network and does not reproduce the live system's queueing or transport behavior. Minimal update mode intentionally omits timing and other full-HAR fields that routing does not need. Measure those qualities against a controlled live environment with suitable telemetry.

A service-worker scenario also needs a different design. Blocking service workers is useful when the worker is incidental to an API test, but it invalidates a test whose purpose is offline caching, update activation, or worker-controlled fetch behavior. Playwright cannot serve worker-intercepted requests from the HAR, so choose a worker-specific fixture strategy or a local server that exercises the real fetch path.

Streaming and long-lived protocols are poor candidates for a static request-response archive. A snapshot can preserve one body, but it does not reproduce incremental delivery, reconnection, heartbeats, or ordering over time. Test those behaviors with a controllable server that can schedule frames and disconnects rather than treating a captured response as the protocol.

Rapidly changing signed URLs and one-time tokens can make strict replay maintenance more expensive than the isolation is worth. Before normalizing or rewriting them, ask whether doing so removes the behavior the test is meant to cover. A small purpose-built route that validates stable request fields and fulfills a known response may communicate the contract better than a large HAR full of volatile data.

Finally, skip HAR replay when nobody can state which requests are fixtures and which are live dependencies. That ambiguity cannot be fixed by choosing abort or fallback. Define the network boundary, identify the owner of each endpoint, and decide what evidence the test must retain before adding archive routing.