Stop random overlays without teaching Playwright to ignore bugs

A survey modal appears on an occasional CI run and covers the Save button. The click times out even though the button is visible, enabled, and stable. A retry passes because the survey does not return, leaving the team with a flaky label instead of a diagnosis.

Locator handlers can remove that kind of obstruction, but they are easy to overuse. A broad handler can silently accept a broken onboarding flow, dismiss a security warning, or steal focus between two keyboard operations. The safe design starts by proving that the overlay is both known and genuinely unpredictable.

Know exactly when the handler can run

page.addLocatorHandler() registers a trigger locator and an asynchronous callback on one Page. Playwright checks the trigger before executing or retrying an action that performs actionability checks, and before an auto-waiting assertion check. If the trigger locator is visible, Playwright runs the callback and then continues with the action or assertion that caused the check.

It is not a DOM event listener. An overlay can sit on the page without invoking anything while the test is waiting on a network promise, sleeping in application code, or performing no qualifying Playwright action. The callback does not fire at the moment a node is inserted. This distinction explains reports that say, “The modal was visible, but the handler did nothing.” Look at what the test did next.

After the callback returns, Playwright normally waits for the triggering overlay to become hidden. That default is a useful contract. A handler that clicks the wrong close button cannot report success while the original obstruction remains visible. The noWaitAfter option disables that wait and should be rare for real overlays.

Handler time counts against the timeout of the action or assertion that invoked it. A Save click with a five-second timeout does not receive five seconds for the handler plus another five seconds for the click. If dismissal involves slow navigation, a server call, or an animation, the original action may expire inside the callback or while waiting for the trigger to hide.

Multiple handlers may be registered, but Playwright runs only one at a time. Actions inside one callback must not rely on another handler being invoked to clear a second obstruction. If one modal is stacked over another, design a single callback that can clear the known stack or fix the application/test environment that creates it.

The smallest useful example registers a handler for one named dialog and scopes the dismissal button to that dialog. Scoping matters because a generic “Close” button elsewhere on the page could otherwise satisfy the locator.

import { test, expect } from '@playwright/test';

test('saves a draft when an optional survey interrupts', async ({ page }) => {
  const survey = page.getByRole('dialog', { name: 'Quick survey' });

  await page.addLocatorHandler(survey, async dialog => {
    await dialog.getByRole('button', { name: 'Not now' }).click();
  }, { times: 1 });

  await page.goto('/editor/42');
  await page.getByLabel('Title').fill('Release notes');
  await page.getByRole('button', { name: 'Save draft' }).click();

  await expect(page.getByRole('status')).toHaveText('Draft saved');
});

times: 1 removes this registration after one invocation. Use it only if a second survey in the same test would be unexpected. Leaving the default unlimited count is appropriate when the known third-party obstruction may return after navigation. That choice is a test contract, not a performance tweak.

Make the dismissal narrow and observable

A handler should identify one overlay, perform the same dismissal a user can perform, and leave evidence. Avoid removing arbitrary DOM nodes with page.evaluate() merely to make the target clickable. Direct removal bypasses the product's dismissal event, persistence behavior, analytics, and accessibility path. It can make a test pass while users remain blocked.

Put a suite-wide policy in a test-scoped fixture rather than copying registrations into every test. Test scope matters because the built-in page is test-scoped. It also keeps invocation counts and evidence attached to the test that experienced the obstruction.

// tests/fixtures.ts
import { test as base, expect } from '@playwright/test';

type OverlayEvent = {
  name: string;
  url: string;
  invocation: number;
};

export const test = base.extend<{ optionalOverlays: void }>({
  optionalOverlays: [async ({ page }, use, testInfo) => {
    const events: OverlayEvent[] = [];
    const survey = page.getByRole('dialog', { name: 'Quick survey' });

    await page.addLocatorHandler(survey, async dialog => {
      events.push({
        name: 'quick-survey',
        url: page.url(),
        invocation: events.length + 1,
      });
      await dialog.getByRole('button', { name: 'Not now' }).click();
    }, { times: 2 });

    await use();

    if (events.length > 0) {
      await testInfo.attach('optional-overlay-events', {
        body: Buffer.from(JSON.stringify(events, null, 2)),
        contentType: 'application/json',
      });
    }
  }, { auto: true }],
});

export { expect };

The fixture records a URL and invocation number, not a guessed reason for the overlay. If the survey is controlled by a remote experiment, add the actual experiment assignment from a response header, cookie, or application test endpoint when available. Do not write “variant B” because the modal looked like variant B.

Automatic registration has a cost. Every actionability or locator assertion check may need to consider the trigger. More importantly, every test now has permission to dismiss that survey. If only editor tests can encounter it, export a non-automatic fixture and request it in those tests, or extend a test object used only by that project. Narrow policy is easier to review.

A self-contained test can prove the handler mechanism without waiting for a production experiment. It inserts a dialog after registration, then performs a click that triggers Playwright's check. The callback clicks the dialog button, the application script hides the dialog, and the original click proceeds.

import { test, expect } from '@playwright/test';

test('handler clears an obstruction before the original click', async ({ page }) => {
  await page.setContent(`
    <button id="save" onclick="document.getElementById('status').textContent = 'saved'">Save</button>
    <div id="status" role="status"></div>
    <div id="survey" role="dialog" aria-label="Quick survey"
         style="position:fixed;inset:0;background:white">
      <button onclick="document.getElementById('survey').hidden = true">Not now</button>
    </div>
  `);

  let calls = 0;
  const survey = page.getByRole('dialog', { name: 'Quick survey' });
  await page.addLocatorHandler(survey, async dialog => {
    calls += 1;
    await dialog.getByRole('button', { name: 'Not now' }).click();
  });

  await page.getByRole('button', { name: 'Save' }).click();

  await expect(page.getByRole('status')).toHaveText('saved');
  expect(calls).toBe(1);
  await expect(survey).toBeHidden();
});

This test verifies three separate facts: the trigger was visible, the callback ran once, and the product action completed after the overlay hid. A handler unit test that asserts only calls === 1 misses whether the original click ever reached the application.

A visible trigger is not proof that it blocks the current action. Playwright checks whether the registered locator is visible around the actionability or assertion check. It does not first calculate that the trigger intercepts the target's pointer events. A broad locator for a promotional banner can therefore run while the banner sits harmlessly below the target.

The next characterization test makes that side effect explicit. The offer is in normal document flow and does not cover Save. Registering it as a trigger still dismisses it before the Save action because it is visible when Playwright checks handlers.

import { test, expect } from '@playwright/test';

test('a visible trigger can run even when it is not the blocker', async ({ page }) => {
  await page.setContent(`
    <aside aria-label="Special offer">
      <p>Save on the next plan</p>
      <button onclick="this.closest('aside').hidden = true">Dismiss offer</button>
    </aside>
    <button onclick="document.getElementById('status').textContent = 'saved'">Save</button>
    <div id="status" role="status"></div>
  `);

  const offer = page.getByRole('complementary', { name: 'Special offer' });
  let calls = 0;
  await page.addLocatorHandler(offer, async banner => {
    calls += 1;
    await banner.getByRole('button', { name: 'Dismiss offer' }).click();
  });

  await page.getByRole('button', { name: 'Save' }).click();

  await expect(page.getByRole('status')).toHaveText('saved');
  expect(calls).toBe(1);
  await expect(offer).toBeHidden();
});

This is valid API behavior and an invalid production policy for that page. The test lost a visible offer even though Save was usable. If the suite is meant to verify layout with the offer present, the handler also changes what screenshots and assertions observe.

Choose a trigger whose visibility means “the known obstruction is active.” A named modal dialog is stronger than generic text copied into a footer. A dedicated test ID on a third-party overlay container can be appropriate when no accessible contract exists, but it should identify that container rather than a phrase used throughout the page. Keep dismissal scoped to the trigger locator so duplicate button names outside it do not create a strictness error.

Hidden templates do not invoke a handler merely because their nodes exist; the trigger must be visible at the check. That helps with applications that keep a dormant dialog in the DOM. It does not solve a template styled offscreen or transparent in a way Playwright still considers visible. When the trace contradicts your visual impression, inspect the element's rendered state and the actionability log instead of adding more text filters.

Invocation frequency is operational evidence. If the policy was approved for a rare survey but the attachment appears on every editor test, stop and inspect the experiment or trigger. Raising times to accommodate the increase hides the changed condition. A maximum count should express how many appearances the journey permits, and exceeding it should expose the unexpected repetition.

Registration belongs to a Page instance. A handler installed before page.goto() can keep using its locator as that page navigates, because locators resolve against the current document when checked. A popup is another Page and does not inherit policy from the opener. If an optional overlay can appear there, register a separately reviewed handler on that popup rather than assuming the original page's callback covers every tab.

Diagnose the blocked action, not just the final timeout

When a click times out, Playwright's call log identifies the actionability check that kept failing. If another element covers the target, the log commonly says that an element intercepts pointer events while Playwright retries the click. If the trigger matched but its callback did not hide the overlay, the timeout can instead point into the callback or the wait after it.

Keep retries off while reproducing so the original attempt is not visually buried. Run one test with a trace and open it locally:

npx playwright test tests/editor.spec.ts:12 --workers=1 --retries=0 --trace=on
npx playwright show-trace test-results/**/trace.zip

The exact trace path depends on the project and test output directory. If the shell does not expand to one archive, pass the path printed in the test result rather than changing the test. In Trace Viewer, select the Save action. Read the call log, then compare the before and after DOM snapshots. The trigger dialog should be visible before the handler and hidden before the Save click completes.

The evidence separates four cases:

1. The overlay covers Save, but the trigger locator matches zero elements. The locator is wrong, perhaps because the accessible name or frame changed.
2. The trigger matches and the callback begins, but the dismissal control is ambiguous or absent. The handler itself has a locator problem.
3. The callback clicks the expected control, but the trigger remains visible. The product did not dismiss, an animation exceeded the remaining timeout, or the trigger describes a container that intentionally stays visible.
4. The handler finishes and the trigger hides, but Save remains blocked. A second obstruction or different actionability condition is responsible.

That fourth case has a close look-alike worth separating before changing the handler. A fixed application header can cover Save after Playwright scrolls the button into view. The click log still reports successful resolution of the Save locator, followed by retries because another element intercepts pointer events. Read only that final message and the failure resembles the survey. The root cause is page geometry, however, not an unpredictable dialog. Dismissing a survey cannot correct a header whose stacking or scroll position places it over the target.

The intercepting element in the call log is the first separating clue. In the survey failure it should identify the dialog, its backdrop, or a descendant of that overlay. In the fixed-header failure it identifies a header control, navigation container, or another element from the application shell. The snapshot immediately before the attempted click supplies the second clue: the survey trigger is visible in one failure, while the header overlaps the target with no active survey in the other. The handler attachment supplies the third clue. A survey invocation at the expected URL supports the overlay diagnosis. No invocation, paired with a header named as the interceptor, supports the geometry diagnosis. Absence of an attachment by itself proves nothing because the overlay may simply not have appeared.

Treat a changing interceptor as a separate signal too. If retries first name the survey backdrop and later name the fixed header, the handler may have successfully cleared the first blocker and exposed the second. Increasing times or broadening the survey locator would only add permission to dismiss more content. The owner of the header layout needs the trace at the point where the second interceptor appears.

The diagnostic output is easiest to read as a sequence rather than as one timeout line. A healthy handled attempt resolves the Save locator, records one overlay event with the expected name and page URL, shows the trigger changing from visible to hidden, and then advances to the completed click and product assertion. A broken dismissal can have the same event name and invocation value, but the trigger stays visible after the callback. The action then spends its remaining timeout waiting or retrying instead of reaching the product result. A different blocker produces no matching event and names a different intercepting element during the click retries.

The values in the JSON attachment are deliberately modest. name should equal the reviewed policy identifier, url should identify the journey where that policy is permitted, and invocation should start at one and rise only when the same test encounters another permitted appearance. An invocation of one at an unrelated checkout URL is broken policy even if the callback works. An invocation of two can be healthy for a journey explicitly reviewed for two appearances, or broken for a journey whose times contract permits only one. The value needs the test's contract beside it.

count() is a useful but misleading value when isolated. A count of one can describe the active modal, a hidden template that remains in the DOM, or a visible banner that does not overlap Save. A healthy post-dismissal page may still report one matching node because hiding does not require removal. Conversely, zero at a later diagnostic point does not prove the trigger was absent when the click first failed. Use the time-aligned snapshot, trigger visibility, intercepting element, and invocation record together. Do not turn a DOM count into a causal claim.

Do not answer all four by raising the action timeout. A larger timeout may help case three when the documented dismissal is slow, but it makes a wrong trigger or second obstruction slower to report.

Add temporary focused diagnostics before editing the handler. locator.count() tells you how many elements currently match, while a screenshot and accessible snapshot in the trace show what the user-facing page contained. Avoid isVisible() as a long poll; it returns the current visibility result. Playwright performs the repeated check when it evaluates the handler around the triggering operation.

The config below retains a trace for the first retry in normal CI and a screenshot for failures. For a dedicated flake investigation, override retries to zero and trace to on at the command line as shown above.

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  retries: process.env.CI ? 1 : 0,
  reporter: [['line'], ['html', { open: 'never' }]],
  use: {
    screenshot: 'only-on-failure',
    trace: 'on-first-retry',
  },
});

An invocation attachment proves the callback ran, but it does not prove the callback caused the overlay to hide. Pair it with the trace and a product assertion. Avoid reporting a handler invocation as a passed test step if the action later fails.

Keep predictable prompts in the test flow

An overlay that appears after a known action is not random. A first-login terms dialog, a required cookie choice, a destructive-action confirmation, or an authentication challenge is part of the product journey. Handling it implicitly removes coverage and makes the test harder to read.

Write that interaction in the test or a named page-object method. Assert the prompt's meaningful content before accepting or rejecting it. The next reader can see the product contract without knowing a global callback exists.

import { test, expect } from '@playwright/test';

test('new users accept the current terms before entering the app', async ({ page }) => {
  await page.goto('/welcome');

  const terms = page.getByRole('dialog', { name: 'Updated terms' });
  await expect(terms).toContainText('Privacy policy');
  await terms.getByRole('checkbox', { name: 'I agree' }).check();
  await terms.getByRole('button', { name: 'Continue' }).click();

  await expect(page).toHaveURL(/\/dashboard$/);
  await expect(terms).toBeHidden();
});

Turning this dialog into a locator handler would let any later action accept terms as a side effect. The handler could even run during an assertion unrelated to onboarding. That is legally and functionally different from a test that deliberately agrees.

Cookie banners sit on the boundary. If your test environment always starts without consent, the banner is predictable and should be handled explicitly or through preconfigured storage state whose provenance is tested. If a third-party platform intermittently forgets consent despite the environment contract, a handler can stabilize unrelated tests while a separate test covers consent behavior. Record every invocation because an increase may reveal a real integration regression.

Authentication prompts should not be auto-dismissed merely because they block a target. A session-expired dialog means the test may no longer be exercising an authenticated operation. Clicking “Sign in later” can turn a security failure into a green UI assertion. Fix authentication setup or make expiry the subject of the test.

Product error dialogs are also out of scope. A generic handler for every element with role=dialog is dangerous because dialogs include validation failures, payment errors, permissions, and confirmations. Name the exact non-product obstruction and the exact user dismissal.

Avoid focus, mouse, and stacked-handler traps

Handler execution changes page state in the middle of the operation that triggered it. Clicking a dismissal button changes focus and moves the pointer. A test that calls locator.focus() and then page.keyboard.press() assumes focus stays unchanged between two Playwright calls. A handler may run before the keyboard-related work and invalidate that assumption.

Prefer an operation that carries its target. locator.press() resolves and acts on the intended locator in one call. The same principle applies to pointer input: prefer locator.click() over a sequence of page.mouse.move(), down(), and up() when an overlay handler can run between those calls.

import { test, expect } from '@playwright/test';

test('submits search without relying on retained focus', async ({ page }) => {
  const survey = page.getByRole('dialog', { name: 'Quick survey' });
  await page.addLocatorHandler(survey, async dialog => {
    await dialog.getByRole('button', { name: 'Not now' }).click();
  });

  await page.goto('/search');
  const search = page.getByRole('searchbox', { name: 'Site search' });
  await search.fill('fixture lifecycle');
  await search.press('Enter');

  await expect(page.getByRole('heading', { name: 'Search results' })).toBeVisible();
});

Stacked overlays require an explicit decision. Because only one handler runs at a time, a survey callback that tries to click through a cookie banner cannot depend on the cookie handler waking up. If both obstructions are known, handle the blocking order inside one callback or configure the test state so only one can appear. Two callbacks that accidentally need each other can consume the triggering action's timeout without clarifying which policy is wrong.

Store the trigger locator if you need to remove its handlers later. page.removeLocatorHandler(locator) removes handlers registered for that locator. This is useful when a test deliberately enters a phase where the same dialog becomes meaningful product behavior.

import { test, expect } from '@playwright/test';

test('limits survey dismissal to the editor phase', async ({ page }) => {
  await page.goto('/editor/42');
  const survey = page.getByRole('dialog', { name: 'Quick survey' });

  await page.addLocatorHandler(survey, async dialog => {
    await dialog.getByRole('button', { name: 'Not now' }).click();
  });

  await page.getByRole('button', { name: 'Open settings' }).click();
  await page.removeLocatorHandler(survey);

  await expect(survey).toBeHidden();
  await expect(page.getByRole('heading', { name: 'Settings' })).toBeVisible();
});

Removal narrows the waiver but adds lifecycle complexity. Prefer registering the handler immediately before the affected journey when only a few actions need it.

Roll out handlers as temporary, reviewed policy

Begin with traces from several original failures. Confirm the same overlay identity, dismissal control, and product impact. If screenshots show different obstructions, do not create one catch-all handler. File separate defects or policies.

For an existing suite, land preserved coverage before landing automatic dismissal. First add or identify the test that intentionally exercises the survey, including its meaningful content and user dismissal. That test must not import the waiver. Next land the narrowly scoped fixture, invocation attachment, and handler characterization test as an explicit opt-in. Only then migrate a small group of affected editor tests. Making the fixture automatic before the coverage test exists creates an interval where the entire suite is allowed to erase the behavior without any test owning it.

The first breakage usually appears in tests that observe intermediate UI rather than in the Save assertion. Screenshot baselines can lose the survey. A test that counts visible dialogs can see one fewer. Split focus and keyboard calls can send input to a dismissal control. A test with a short per-action timeout can expire after the callback consumes part of that budget. Find those cases in the opt-in group before changing a shared test export or an automatic fixture used by unrelated projects.

Run the opt-in group with retries disabled and inspect every overlay attachment, including attachments from passing tests. A falling retry rate is not sufficient evidence because a broad trigger can make the suite green by dismissing legitimate product state. The change is working when the original product assertions complete, each invocation occurs only on an approved journey, traces show the named trigger hiding before the blocked action proceeds, and tests that never encounter the survey produce no event. Keep the old failure traces next to the rollout evidence so reviewers can compare the interceptor identity rather than relying on the final status.

Widen the policy one boundary at a time. Moving from selected tests to an editor project is one boundary. Making it automatic for every project is another. Popups require their own Page policy and should not be included merely because the opener is covered. After each boundary, review invocation locations and repeated counts before moving again. If the handler starts running in tests that previously passed without the survey, stop the rollout and determine whether the experiment population, stored consent, or trigger locator changed.

Add one narrowly scoped handler with an invocation attachment and a maximum count when repeated appearances are invalid. Run the affected tests with retries disabled. The first success proves only that the test continued; inspect the final product assertion and handler evidence as well.

Track actual invocations in CI. A handler that never runs adds maintenance without value. One that runs on every test is no longer handling a rare interruption and may be hiding an environment contract change. Remove dead policies, and promote common predictable flows into explicit setup or test steps.

The trade-offs are concrete. Every callback consumes part of the triggering action's timeout. Every automatic registration expands the set of tests allowed to dismiss the overlay. Every click can alter focus and pointer state. Every hidden prompt removes some coverage unless another test owns it. The stability gain is worthwhile only when those costs are visible.

Those costs show up in specific budgets. A dismissal that waits for a persistence request and an exit animation leaves less time for the Save click that invoked it, so a suite with tight action timeouts can trade an intermittent interception for a handler timeout. Registering the policy automatically also adds trigger evaluation around qualifying actions and assertions that would otherwise have no relationship to the survey. The coverage cost is the missing modal state in screenshots and accessibility assertions. The maintenance cost is keeping trigger names, allowed URLs, maximum appearances, popup registrations, and the separate survey-coverage test aligned as the product changes.

Ownership should be split by the evidence, not assigned wholesale to the test author. The automation infrastructure owner owns fixture scope, timeout accounting, attachments, and removal of the waiver. The feature team owns whether the prompt is optional, whether dismissal preserves user state, and the accessible identity of its controls. If an experimentation platform or third-party script decides when the prompt appears, that integration owner owns the nondeterminism and assignment evidence. A handoff should contain the failing trace, the exact action and interceptor named in its call log, the trigger's visible and hidden snapshots, the page URL, recorded invocation sequence, known experiment or consent state when available, and a test that reproduces the overlay without retries. It should also state which journeys may use the waiver, which test retains prompt coverage, and what condition will remove the handler.

A locator handler does not catch data loss caused by the interruption. If opening or dismissing the survey resets an edited field, the callback can clear the screen and allow Save to click successfully while the wrong content is saved. Only an assertion on the persisted draft exposes that failure. The handler addresses obstruction, not the semantic integrity of the interrupted operation.

Do not use locator handlers to accept required terms, bypass authentication, close product errors, remove arbitrary page nodes, or compensate for an ambiguous target locator. Do not set noWaitAfter on a real modal simply to stop a timeout; that permits the original action to proceed while the obstruction remains. Use the API when a known optional overlay is genuinely nondeterministic, its user dismissal is safe, and the suite retains evidence each time that exception policy activates.