Why tooltip tests flake after hover, and how to make them trustworthy

The tooltip test passes on a laptop, then times out in CI after the pointer reaches the right icon. A screenshot shows no tooltip, while the same build works when a developer moves the mouse by hand. Raising the timeout feels tempting, but it will not tell you whether the pointer action, the application timer, or the assertion is wrong.

Reliable Playwright hover tooltip testing starts by separating those three boundaries. Playwright can prove that it found an actionable target and moved its virtual mouse. Only an application-facing assertion can prove that the expected help content became visible and remained usable.

Know what a completed hover actually proves

locator.hover() is an input action, not a tooltip assertion. Playwright resolves the locator, waits for the actionability checks required by hover, scrolls the element into view when necessary, and moves the mouse over a visible point in the element. The promise resolves after those steps. It does not know that your design system calls the next popup a tooltip, how long the component waits before opening it, or which text should appear.

That distinction explains a common misleading failure. The trace marks the hover action green, but toBeVisible() later times out. Nothing in that result is contradictory. The browser received pointer movement, yet the application's hover handler may not have opened the component. It may also have opened a node that the assertion did not identify.

There are several events between those facts. A real pointer move can cause pointer and mouse boundary events as the hit target changes. Framework code may schedule an open delay, cancel an earlier close timer, update component state, and render the floating element in a portal near the end of body. CSS may then animate opacity or visibility. Playwright does not collapse that chain into the meaning of hover().

Use a locator that names the trigger the way a user or accessibility API does. A button named "Explain risk score" is a stronger target than .icon:nth-child(3). It survives layout changes and it makes a trace legible. If the icon is not a control at all, that is product evidence worth discussing. An element that only reacts to a pointer cannot expose the same explanation to a keyboard user without additional behavior.

The assertion deserves the same care. page.getByRole('tooltip') is appropriate when the product renders a custom ARIA tooltip. A native browser tooltip created only by a title attribute is different. Browser chrome owns its visual presentation, and the popup is not a normal DOM node that getByRole('tooltip') can locate. In that case, assert the title attribute if that is truly the product contract, or ask for a custom accessible component when the content needs richer interaction.

Do not start with force: true. The option tells Playwright to bypass its hover actionability checks. It can make the method complete while another element covers the trigger or while the intended hit target is unstable. That is useful for a narrow DOM-level experiment, but it weakens an end-to-end claim. A user cannot force a pointer through a loading mask.

The same warning applies to dispatchEvent('mouseenter'). Dispatching an event directly can exercise a handler without placing the mouse over the element and without running the normal hit-testing path. It is a valid component-level technique when the event handler itself is the unit under test. It is poor evidence for a regression in which an overlay, animation, or geometry change prevents a user from hovering the control.

Treat the hover action and tooltip expectation as separate checkpoints in reviews. If hover times out, investigate locator identity, visibility, stability, scrolling, and hit testing. If hover completes but the expectation fails, investigate application state, delayed rendering, accessible semantics, duplicate nodes, and premature dismissal. That split prevents a longer assertion timeout from becoming the answer to every symptom.

Assert the user contract, not a timer

A useful test says what help a person can obtain and how they can obtain it. It does not encode the component library's current delay. The delay may move from 300 milliseconds to 500 milliseconds without changing the product promise. An assertion that retries until the tooltip is visible absorbs that internal timing change while still failing if the content never arrives.

The first worked example covers a risk-score explanation available from both pointer and keyboard input. The application contract is explicit: the trigger references the tooltip as an accessible description, the expected text appears, and moving the pointer away closes the extra content. Each assertion can fail when the product changes. None merely verifies a constant created inside the test.

import { expect, test } from '@playwright/test';

test.describe('risk score help', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('/accounts/42/risk');
  });

  test('opens for pointer hover and closes after pointer exit', async ({ page }) => {
    const trigger = page.getByRole('button', { name: 'Explain risk score' });
    const tooltip = page.getByRole('tooltip', { name: /calculated from login history/i });

    await expect(trigger).toHaveAttribute('aria-describedby', 'risk-score-help');
    await trigger.hover();
    await expect(tooltip).toBeVisible();

    await page.getByRole('heading', { name: 'Account risk' }).hover();
    await expect(tooltip).toBeHidden();
  });

  test('opens when the trigger receives keyboard focus', async ({ page }) => {
    const trigger = page.getByRole('button', { name: 'Explain risk score' });
    const tooltip = page.getByRole('tooltip');

    await trigger.focus();
    await expect(trigger).toBeFocused();
    await expect(tooltip).toContainText('Calculated from login history');
  });
});

focus() is used deliberately in the second test. It isolates the tooltip component's focus response. It does not prove that a keyboard user can reach the button in the page's sequential focus order. That wider claim belongs in a separate keyboard-navigation test that presses Tab from a known starting point. Keeping those scopes distinct makes a failure actionable.

The first test also assumes the team chose aria-describedby="risk-score-help" as part of the component contract. ARIA tooltip guidance uses a description relationship from the owning element to the tooltip. If your implementation generates IDs, do not hard-code one merely because it appeared in a snapshot. Assert the accessible description instead, or assert that aria-describedby points to the actual tooltip element after it opens.

For example, an ID-agnostic assertion can read the referenced value, locate that ID, and compare the content. That is more implementation-aware than a role locator, but it catches a broken reference that a visible-text assertion could miss. Use it when stale or duplicate aria-describedby values have caused real regressions.

import { expect, test } from '@playwright/test';

test('the tooltip description points to the visible content', async ({ page }) => {
  await page.goto('/billing');

  const trigger = page.getByRole('button', { name: 'Explain processing fee' });
  await trigger.hover();

  const describedBy = await trigger.getAttribute('aria-describedby');
  expect(describedBy, 'trigger must reference its help content').toBeTruthy();

  const tooltip = page.getByRole('tooltip');
  await expect(tooltip).toHaveAttribute('id', describedBy!);
  await expect(tooltip).toBeVisible();
  await expect(tooltip).toContainText('Charged by the payment provider');
});

Notice what is absent. An earlier version of this test also asserted toHaveAttribute('role', 'tooltip') on a locator that was already page.getByRole('tooltip'). No HTML element carries an implicit tooltip role, so the element has to declare that attribute before the locator can resolve to it at all. The assertion therefore restates its own precondition, and the only input that changes its result makes it wrong: a valid multi-token value such as role="tooltip note" still satisfies the role locator, while the raw attribute reads tooltip note and the equality check fails. A check that cannot fail for a real defect but can fail for a legal one is worse than no check. The remaining assertions all have honest failure modes, because the ID relationship, the visibility, and the text can each break without the others.

This example assumes the application uses a single ID in aria-describedby. The attribute may contain multiple ID references. If your component composes several descriptions, split the value on whitespace and compare the tooltip's ID with that set. Calling out the one-ID assumption in the test name or helper is better than hiding it.

Avoid exact pixel assertions for tooltip placement unless collision handling is the feature under test. A tooltip can move above the trigger near the bottom of the viewport and below it elsewhere. Both positions may be correct. Visibility, content, association, and interaction behavior usually express the stable contract. Screenshot comparison belongs in a focused visual suite with controlled fonts, viewport, and operating system.

Diagnose the first failed attempt before changing timeouts

The most useful question is where the evidence stops. If the locator never becomes actionable, there is no reason to inspect the tooltip's open delay. If pointer events reach the trigger and no tooltip node appears, a selector rewrite is unlikely to help. Build diagnostics that preserve that boundary.

Run the failing test with a trace instead of immediately adding retries. A local command such as npx playwright test tests/tooltips.spec.ts --trace on records the actions and snapshots for that run. Open the resulting archive from the HTML report or with npx playwright show-trace path/to/trace.zip. In the trace, select the hover action and compare its before, action, and after snapshots.

The hover action log tells you which checks Playwright performed and which locator it resolved. The action snapshot shows the input point relative to the page at that moment. The after snapshot can reveal that a menu closed, a skeleton disappeared, or the trigger moved during a layout update. The Errors panel belongs to the failed expectation, while the Console panel may contain an application exception raised by the tooltip component.

Do not expect a trace snapshot to behave exactly like a live page. Use it to inspect recorded DOM state and action chronology. If the component opens and closes entirely between useful snapshots, add a small event or state attachment to the test. The attachment should diagnose the application, not manufacture the condition you want to observe.

The following diagnostic records trusted pointer boundary events for one trigger. It uses the browser's actual events produced by hover(). The final assertion has a real failure mode: it fails if the pointer never enters the intended element, even if a broad page-level listener sees movement elsewhere.

import { expect, test } from '@playwright/test';

test('records pointer evidence for the fee tooltip', async ({ page }, testInfo) => {
  await page.goto('/billing');
  const trigger = page.getByRole('button', { name: 'Explain processing fee' });

  await trigger.evaluate((element) => {
    const target = element as HTMLElement & { dataset: DOMStringMap };
    target.dataset.pointerEvidence = '';

    for (const type of ['pointerover', 'pointerenter', 'mouseover', 'mouseenter']) {
      target.addEventListener(type, () => {
        const previous = target.dataset.pointerEvidence;
        target.dataset.pointerEvidence = previous ? `${previous},${type}` : type;
      });
    }
  });

  await trigger.hover();
  const evidence = (await trigger.getAttribute('data-pointer-evidence')) ?? '';
  await testInfo.attach('pointer-evidence.txt', {
    body: Buffer.from(evidence || 'no recorded events'),
    contentType: 'text/plain',
  });

  expect(evidence.split(',')).toContain('pointerenter');
  await expect(page.getByRole('tooltip')).toBeVisible();
});

Keep instrumentation like this temporary unless it repeatedly pays for itself. It mutates the page under test by adding listeners and a data attribute. That can affect code that observes DOM mutations, and it adds noise to a straightforward scenario. A trace is cheaper for routine coverage. The event recorder is appropriate when the disputed fact is whether the intended element received the boundary event.

Error shape helps with triage. An action timeout whose call log says the element is not stable points toward movement or animation before input. A log that repeatedly reports another element intercepting pointer events points toward hit testing. A passed hover followed by expect(locator).toBeVisible() timing out means the action completed and the expected node never satisfied visibility. A strict-mode locator error means more than one node matched; increasing time cannot choose the correct one.

Duplicate tooltip nodes are a frequent near-miss. Some component libraries retain a hidden tooltip template while rendering a second portal instance. getByRole('tooltip') then resolves to more than one element, or a broad text locator finds the hidden copy. Narrow by a stable relationship or by the exact accessible name. Do not silence strictness with .first(). The first DOM match may be the hidden template today and the active portal after a refactor.

Clipping can look like premature dismissal in a screenshot. Suppose the tooltip node remains visible according to CSS, but a parent with overflow: hidden cuts off everything outside a compact table cell. A role-based visibility assertion may pass because part of the element still has a rendered box, while the sentence a user needs is outside the visible region. Inspect the element in the action snapshot, then compare its bounding box with the clipping ancestor. A screenshot of the tooltip locator can also show how much of the box Playwright can capture. The repair belongs in the component's portal or containment strategy. Extending the wait cannot change clipping geometry.

The opposite near-miss comes from an opacity transition. The DOM node appears immediately with the right role and text, but its first rendered state has zero opacity. Playwright's visibility definition still considers an element with zero opacity visible when its box and CSS visibility otherwise qualify. A simple toBeVisible() can therefore pass before the tooltip is perceptible. If a stuck fade has caused a real regression, poll the computed opacity until it reaches the component's documented final value or use a focused visual assertion. The extra check couples the test to presentation, so do not add it when visible geometry and accessible content are the only product contract.

One more CI-only shape is a font-driven layout shift. A fallback font renders the trigger wider, then the web font loads and moves it while the actionability check is running. The hover may wait for stability or the pointer may end up over a neighboring element after later application layout. The trace's action log and snapshots can distinguish this from a missing event handler. Stabilize font delivery in the test environment or remove the layout dependency. Adding a mouse movement offset is brittle because the offset encodes one transient rendering.

Another near-miss is a viewport-specific target. CI may use a smaller viewport, causing the icon to collapse into an overflow menu. The old locator might still match a hidden copy, or a sticky header may cover the point after scrolling. Check the trace metadata for viewport size and the action snapshot for the responsive layout. That evidence calls for an explicit project or a responsive-path test, not a larger hover timeout.

Cover the failures a green visibility check misses

A tooltip can appear and still be unusable. WCAG's guidance for author-controlled content that appears on hover or focus calls out three behaviors: it should be dismissible under the stated conditions, hoverable when pointer hover reveals it, and persistent until the relevant trigger is removed, the user dismisses it, or the information is no longer valid. Translate only the behaviors that apply to your component into tests.

The second worked example catches a narrow gap between the trigger and the tooltip. The popup appears, so a simple visibility check passes. As the pointer travels toward the content, it crosses the gap and the component starts its close timer. A magnified user trying to move into the popup loses it.

Two details decide whether that test can fail at all, and both are easy to get wrong. The first is the shape of the assertion. expect(locator).toBeVisible() retries until the condition becomes true, so it reports success on its first poll and returns long before any close timer could fire. Point it at a popup that dismisses itself 60 milliseconds after the pointer leaves the trigger and it passes every single time, while the tooltip disappears a moment later and the person who needed the text never gets to read it. The requirement is the opposite shape: the content has to stay present through a window in which dismissal was possible. That needs a hold which outlives the component's documented delay, followed by the visibility check.

The second detail is the pointer path. page.mouse.move() defaults to one step, and the installed type definitions describe that default as emitting a single mousemove event at the destination rather than interpolating travel from the current cursor position. With one step the cursor arrives inside the popup without ever occupying the gap, so the boundary transition the test exists to provoke never happens. Passing an explicit step count sends the intermediate moves a real hand would produce, which is also what exercises any invisible bridge element the component uses to cover the gap.

import { expect, test } from '@playwright/test';

// The design system documents a 300 ms delay between pointer exit and dismissal.
// The hold has to outlive it. On its own, toBeVisible() retries until true and
// passes on its first poll, before any close timer could have fired.
const CLOSE_DELAY_MS = 300;

test('the tooltip survives the pointer trip from trigger to content', async ({ page }) => {
  await page.goto('/analytics');

  const trigger = page.getByRole('button', { name: 'Explain confidence range' });
  const tooltip = page.getByRole('tooltip');

  await trigger.hover();
  await expect(tooltip).toContainText('Expected range based on available samples');

  const box = await tooltip.boundingBox();
  expect(box, 'visible tooltip must have a bounding box').not.toBeNull();
  if (!box) throw new Error('Tooltip disappeared before pointer movement');

  // steps sends interpolated moves, so the pointer travels across the gap.
  // The documented default of 1 emits a single move at the destination.
  await page.mouse.move(box.x + box.width / 2, box.y + box.height / 2, { steps: 24 });

  await page.waitForTimeout(CLOSE_DELAY_MS * 2);
  await expect(tooltip).toBeVisible({ timeout: 1_000 });
});

That wait is a hold, not the sleep this article warns about elsewhere. A sleep inserted before an assertion guesses how long an event needs and hides the timing boundary behind the guess. This hold is the boundary. The promise being tested is that the tooltip is still there after dismissal could already have happened, and no retrying assertion can express a stays-true claim on its own. Keep the constant beside the delay the component documents so a reviewer can see where the number came from, and give the closing check a short local timeout so a genuine dismissal reports quickly instead of consuming the whole global expectation budget.

Confirm that the two versions actually differ before trusting either one. Build two popups with identical geometry, both sitting 24 pixels below their trigger and both starting a 60 millisecond close timer when the pointer leaves it. Give one of them a handler that cancels that timer when the pointer arrives, and give the other nothing. Run each version of the test five times against each popup. The retry-only assertion passes all twenty runs and separates nothing. The held version fails all five runs against the popup that cannot be hovered and passes all five against the one that can. That exercise costs half an hour and it is the only way to know whether a hover test is evidence or decoration.

boundingBox() introduces a trade-off. It observes rendered geometry, so the test belongs in a browser project with a known viewport. It may need separate expectations if the mobile experience does not use hover at all. The assertion remains valuable because geometry is part of this particular requirement: the pointer must physically reach the added content.

Escape dismissal is another product-specific branch. If the tooltip covers adjacent content, the component may offer Escape as its dismissal mechanism. Test the behavior through keyboard input and confirm focus stays on the trigger if that is the intended interaction. Do not claim every tooltip must close on Escape in every circumstance. The WCAG criterion allows exceptions, and native browser tooltips are controlled by the user agent.

Focus-triggered content also deserves an exit test. Move focus to the next meaningful control and assert that the old tooltip closes. This catches global focusin handlers that open help but never clear their state. It also exposes components that rely solely on mouseleave, which never fires during keyboard navigation.

Be careful with negative assertions. await expect(tooltip).toBeHidden() succeeds when no matching node exists or when the matching node is not visible. That is correct for many close behaviors, but it does not prove the application removed the node. If cleanup matters because stale aria-describedby references have caused bugs, assert the relationship and attachment separately. Choose the matcher that describes the defect you want to prevent.

Do not turn every tooltip test into a visual-position test, an accessibility audit, and an event-order test. One concise behavior test should cover the ordinary open path. Add the focus, hoverability, dismissal, or association cases when those are real product requirements or previous regressions. Broad duplication makes the suite slower without making ownership clearer.

Roll the fix through an existing suite without hiding flakes

Start migration with the tests that contain waitForTimeout, forced hover, dispatchEvent used as a user substitute, .first() on a tooltip locator, or retries added only for tooltip cases. Those patterns do not prove a defect by themselves, but they are strong review candidates. Change one behavior at a time and keep the first failing trace.

Replace fixed sleeps with web-first assertions. A wait that stands in for an expectation should go; a hold that a persistence check needs in order to be capable of failing should stay, with the delay it outlives named beside it. Keep the hover and expectation adjacent unless intervening steps are part of the scenario. Name the trigger by role and accessible name. Name the popup by role and content. If the application lacks those semantics, file a product issue instead of burying the deficiency under a test ID.

Run the rewritten case repeatedly before enabling it across all browser projects. Repetition is not proof of correctness, but it is useful for exposing a race you have not removed. Use a command scoped to the spec and project, such as npx playwright test tests/tooltips.spec.ts --project=chromium --repeat-each=20. Treat the count as a stress run chosen by the team, not as a measured reliability guarantee.

Configure evidence according to the suite's retry policy. When retries are disabled, retain-on-failure preserves the original failing attempt and removes traces for passing tests. When retries are enabled, on-first-retry records the retry rather than the first attempt. That retry trace may be useful, but it cannot show the original failure. Teams that need first-attempt evidence should choose a mode that records it and budget for the extra storage and runtime.

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  retries: 0,
  expect: { timeout: 5_000 },
  use: {
    baseURL: process.env.BASE_URL ?? 'http://127.0.0.1:3000',
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
  ],
});

The explicit zero retries in this example is a policy choice, not a universal recommendation. It keeps the first failure visible while the team removes existing flake. A mature suite may restore retries for infrastructure resilience, provided the report still distinguishes flaky tests from clean passes and reviewers can access the relevant attempt.

Cross-browser rollout has a cost. Three projects multiply execution and artifact volume. Start with the browser where the defect occurred, then add coverage where the component relies on browser event or layout behavior. If all projects use the same design-system code and the risk is low, a representative browser on every pull request plus a scheduled matrix can be a defensible compromise.

Keep component delay values out of global Playwright timeouts. Raising expect.timeout for the entire repository slows feedback for every missing element. If the product intentionally delays one tooltip longer, give that assertion a local timeout and document the reason. Better still, question whether a long delay is acceptable to users before enshrining it in automation.

After migration, review failures by class. Locator or actionability failures belong to the trigger path. Missing or wrong text belongs to rendering or data. A tooltip that opens but cannot be hovered belongs to interaction design or geometry. A failure only under one project belongs first to browser-specific reproduction. Those labels help the owning engineer act without reading the entire test history.

Know when hover is the wrong level of test

Do not use hover to verify a native title popup's pixels. The browser owns that rendering outside the normal page DOM. Assert the attribute when its value is the contract, and cover the surrounding feature through a user-visible path that your automation can observe reliably.

Do not use forced hover to get through a consent banner, loading mask, or disabled state. Those elements change what a user can reach. Either dismiss them through supported behavior, wait for the application to become ready, or fix the product bug. Bypassing them turns an end-to-end test into a statement about hidden implementation state.

Skip hover in a mobile-only product path that has no pointing-device hover. Test the tap, long-press, info button, or persistent label that the interface actually provides. Running a desktop mouse abstraction against a touch design produces coverage that no customer can use.

Avoid a full browser test for every delay calculation inside a tooltip hook. Timer cancellation and state transitions are cheaper to cover at component or unit level with controlled clocks. Keep one browser scenario for the integrated pointer, DOM, accessibility, and layout path. This split costs some test-layer coordination, but it reduces slow duplicated cases.

Do not demand that every piece of supplemental text use role="tooltip". Popovers containing links, buttons, or form fields are not tooltips in the ARIA sense. Interactive content needs a component pattern that supports focus and keyboard operation. A role locator can expose that design mistake, but changing the test to call the popup a tooltip will not fix it.

Finally, do not freeze incidental placement. Collision-aware positioning is meant to change with available space. Assert exact coordinates only when an overlap, clipping, or pointer path is the regression. The cost is greater sensitivity to viewport, fonts, operating system, and rendering differences. For ordinary explanatory content, association and behavior provide a stronger contract than a pixel coordinate.