Test the gallery behavior a screenshot cannot see

The lightbox opens, but keyboard focus stays on the thumbnail behind the overlay. Press Tab and the browser walks through controls the user cannot see, while the preview appears perfectly normal in a screenshot. In the same gallery, Next can skip a photo and Download can save a different one from the image on screen. QA teams use accessible gallery interaction testing to separate those failures because each one needs different evidence and a different fix.

Why the screenshot and the accessibility tree disagree

An image gallery is several interfaces stacked on top of each other. The visible grid is only one of them. There is also the accessibility tree exposed by the browser, the keyboard focus path, the lightbox state machine, and the record used by actions such as Favorite or Download. A useful test identifies which interface broke instead of filing one vague "gallery accessibility" bug.

Start with semantics. An informative image needs a text alternative that serves the same purpose as the visual. A decorative image usually needs an empty alternative so assistive technology can ignore it. That choice comes from the image's purpose in context, not from its file type or the tester's preference. The W3C image guidance is useful here because it separates informative, decorative, functional, and complex images. A rule that demands non-empty text for every image will create noisy output and still miss descriptions that are technically present but useless.

A control has a different naming problem. If a thumbnail opens a preview, its accessible name must identify the action well enough for a user to choose it. Native buttons already supply button semantics and keyboard activation. An aria-label can give an icon-only button a name, but it does not turn a decorative div elsewhere in the card into an informative image. Test the control and the visual as separate objects.

PixelVault, the gallery used in QABattle, makes that distinction visible. Each thumbnail is a native button named with text such as "Open Sunset over the ridge." A role locator can find that button. The picture, however, is rendered as a colored div containing an emoji. There is no img element or role="img" for the browser to expose. The button is named, but the visual has no independent image semantics. Whether the title alone is an adequate alternative is a content decision; the seeded exercise says the visual is informative, so the missing image representation is a defect.

Focus is another contract. Adding role="dialog" gives an element a dialog role, but it does not make the background inert, move focus inside, contain the tab sequence, implement Escape, or return focus when the lightbox closes. The WAI-ARIA modal dialog pattern describes those behaviors separately. It also allows initial focus to move to different places depending on the content. A test should require focus to land at an appropriate element inside the dialog, not blindly require the Close button in every product.

The native HTML dialog element can remove part of that implementation burden. When code opens it with showModal(), the browser places it in the top layer and makes the rest of the same document inert. The MDN dialog reference still recommends deliberate initial focus and a visible close mechanism. Native behavior is a better starting point, not a waiver for testing.

State identity is the third contract. The title, selected thumbnail, dialog label, live status, favorite state, and downloaded record should all refer to the same photo after an interaction. Looking only at the pixels misses a common class of bugs where the visible image changes but a handler still closes over an old index. Looking only at the accessibility tree misses the opposite case, where the announced title changes but the displayed or downloaded record does not.

Finally, decide whether the product is a static gallery, a carousel, or both. A grid of linked images does not automatically need arrow-key navigation. A rotating carousel needs controls and announcements that a static collection does not. W3C's carousel tutorial requires keyboard access to functionality, communication of slide changes, and user control over movement. It does not give testers permission to invent an undocumented key map. Write the intended interaction contract before automating it.

For each action, record four facts:

A passing result in one row never proves the other three. That is the mechanism behind most misleading gallery reports.

How to isolate the broken layer before changing code

Reproduce the failure without a mouse first. Begin on a link or control before the gallery, press Tab once at a time, and write down the focused element's role and name. Open a thumbnail with Enter. Do not click the overlay to recover when focus disappears. Continue with Tab, Shift+Tab, and Escape exactly as a keyboard user would.

For the current PixelVault implementation, clicking the first thumbnail leaves focus on that thumbnail. The lightbox is appended later in the document, after the background controls, and the page behind it is not inert. The next Tab can therefore reach a covered control before it reaches a lightbox button. That observation distinguishes missing modal focus management from a CSS-only focus indicator problem.

Now compare focus with the accessibility tree. Playwright's getByRole() follows computed role and accessible-name rules, which makes it valuable for testing the same public interface exposed to assistive technology. The locator documentation is explicit that role locators are not a replacement for an accessibility audit. A locator can tell you that a named button exists. It cannot tell you that every image has a useful description or that a modal interaction is coherent.

The following diagnostic file creates one failure per contract. It uses labels and titles that exist in the PixelVault fixture, so a failure points to behavior rather than a guessed selector.

import { expect, test } from "@playwright/test";

const titles = [
  "Sunset over the ridge",
  "City lights at night",
  "Forest trail in fog",
  "Waves on black sand",
  "Desert dunes at noon",
  "Snowfield silence",
];

test.describe("PixelVault gallery diagnostics", () => {
  test.beforeEach(async ({ page }) => {
    await page.goto("/arenas/image-gallery");
  });

  test("exposes each informative visual as an image", async ({ page }) => {
    await expect(page.getByRole("img")).toHaveCount(titles.length);
  });

  test("moves focus into the lightbox and restores it", async ({ page }) => {
    const opener = page.getByRole("button", {
      name: `Open ${titles[0]}`,
    });

    await opener.click();

    const dialog = page.getByRole("dialog");
    await expect(dialog).toBeVisible();
    await expect(dialog.locator(":focus")).toHaveCount(1);

    await page.keyboard.press("Escape");
    await expect(dialog).toBeHidden();
    await expect(opener).toBeFocused();
  });

  test("visits every photo in order with Next", async ({ page }) => {
    await page.getByRole("button", {
      name: `Open ${titles[0]}`,
    }).click();

    const dialog = page.getByRole("dialog");
    const visited: string[] = [];

    for (let step = 0; step < titles.length; step += 1) {
      visited.push((await dialog.getAttribute("aria-label")) ?? "");
      await dialog.getByRole("button", { name: /next/i }).click();
    }

    expect(visited).toEqual(titles);
  });
});

Those tests expose three different signatures. The image count expects six and receives zero because no image object exists. The focus check expects one focused descendant of the dialog and receives none. The navigation assertion expects photo titles in fixture order, while the +2 index update produces Sunset, Forest, Desert, Sunset, Forest, Desert. That repeated three-item cycle is not a timing symptom. It is direct evidence that the transition graph cannot reach half of an even-sized collection through Next.

Run the narrow file with tracing turned on while investigating. The command below uses Playwright's documented CLI flags and leaves the rest of the suite out of the result.

pnpm exec playwright test e2e/gallery-accessibility.spec.ts --project=chromium --trace on

Open the resulting trace with the command printed by Playwright. In Trace Viewer, inspect the action immediately before the failed assertion. The snapshot should show which dialog label was present, which button received the action, and what the DOM looked like at that moment. The action log also helps reveal an accidental second click or a locator that resolved outside the dialog.

A trace does not record what a screen reader spoke. An ARIA snapshot is not a transcript either. Playwright's ARIA snapshot feature serializes accessible roles, names, states, text, and hierarchy into YAML. It is excellent for proving that a dialog or image node disappeared from the computed tree. It cannot judge whether "Sunset over the ridge" communicates the information the product owner intended the image to convey.

Keep a small evidence ledger beside the automated result:

The entries above come from the seeded implementation's code paths, not from invented timings or success rates. When testing another product, fill the ledger from the run rather than copying these values.

A useful bug report names the divergence. "After opening Sunset with Enter, focus remains on the background thumbnail; expected focus inside the modal" is actionable. "Gallery is inaccessible" forces the developer to repeat your investigation. Include the smallest key sequence, the focused element, the current photo ID, the expected contract, and the trace. Add a screenshot only when it proves visible focus, obscured focus, reflow, or another visual condition.

Three failures worth testing as separate journeys

A single exploratory keyboard journey is valuable. It is a poor regression test when every assertion lives in one function. The first missing image role stops the run before it reaches focus restoration, index movement, or download identity. Split the journeys by failure boundary and keep one broader smoke path for the assembled experience.

The modal looks open but the page behind it still owns focus

Treat opening and closing as paired transitions. Capture the element that invokes the lightbox. After open, require an appropriate focus target inside. While the modal is active, Tab and Shift+Tab must not enter the background. After close, return to the invoker unless that element no longer exists or the workflow has an explicitly documented next step.

A native dialog makes the surrounding document inert when opened with showModal(). This React component also chooses initial focus, handles Escape through the dialog's cancel event, and restores the opener. Its native controls update the current item and bind Download to that record.

"use client";

import {
  type SyntheticEvent,
  useEffect,
  useId,
  useRef,
  useState,
} from "react";

type Photo = {
  id: string;
  title: string;
  alt: string;
  imageUrl: string;
  downloadUrl: string;
  fileName: string;
};

type LightboxProps = {
  photos: Photo[];
  index: number;
  opener: HTMLButtonElement;
  onIndexChange: (index: number) => void;
  onClose: () => void;
};

function Lightbox({
  photos,
  index,
  opener,
  onIndexChange,
  onClose,
}: LightboxProps) {
  const dialogRef = useRef<HTMLDialogElement>(null);
  const closeRef = useRef<HTMLButtonElement>(null);
  const titleId = useId();
  const current = photos[index];

  useEffect(() => {
    const dialog = dialogRef.current;
    if (!dialog) return;

    dialog.showModal();
    closeRef.current?.focus();

    return () => {
      if (dialog.open) dialog.close();
      opener.focus();
    };
  }, [opener]);

  function closeFromKeyboard(event: SyntheticEvent<HTMLDialogElement>) {
    event.preventDefault();
    onClose();
  }

  function move(offset: number) {
    const next = (index + offset + photos.length) % photos.length;
    onIndexChange(next);
  }

  return (
    <dialog
      ref={dialogRef}
      aria-labelledby={titleId}
      onCancel={closeFromKeyboard}
    >
      <h2 id={titleId}>{current.title}</h2>
      <img src={current.imageUrl} alt={current.alt} />
      <p aria-live="polite" aria-atomic="true">
        {current.title}, image {index + 1} of {photos.length}
      </p>
      <div>
        <button type="button" onClick={() => move(-1)}>
          Previous image
        </button>
        <a href={current.downloadUrl} download={current.fileName}>
          Download {current.title}
        </a>
        <button type="button" onClick={() => move(1)}>
          Next image
        </button>
      </div>
      <button ref={closeRef} type="button" onClick={onClose}>
        Close preview
      </button>
    </dialog>
  );
}

export function Gallery({ photos }: { photos: Photo[] }) {
  const [openIndex, setOpenIndex] = useState<number | null>(null);
  const openerRef = useRef<HTMLButtonElement | null>(null);

  return (
    <>
      <ul aria-label="Campaign images">
        {photos.map((photo, index) => (
          <li key={photo.id}>
            <figure>
              <img src={photo.imageUrl} alt={photo.alt} />
              <figcaption>{photo.title}</figcaption>
            </figure>
            <button
              type="button"
              aria-label={`Open preview: ${photo.title}`}
              onClick={(event) => {
                openerRef.current = event.currentTarget;
                setOpenIndex(index);
              }}
            >
              Open preview
            </button>
          </li>
        ))}
      </ul>

      {openIndex !== null && openerRef.current && (
        <Lightbox
          photos={photos}
          index={openIndex}
          opener={openerRef.current}
          onIndexChange={setOpenIndex}
          onClose={() => setOpenIndex(null)}
        />
      )}
    </>
  );
}

The cost is more than extra markup. Someone must decide the correct initial focus target, the wrap behavior at the first and last item, the wording of each text alternative, and whether repeated image descriptions in the grid and dialog are useful. Native dialog support is broad, but a product with older embedded webviews may need a tested fallback. Animation also needs care because unmounting the dialog immediately can conflict with a visual exit transition and focus restoration.

Copying the aria-live line without testing it is risky. The DOM update can be asserted, but assistive technologies differ in how and when they announce changes. Repeating the title through the dialog heading, image alternative, and live region can become noisy. Listen to the finished interaction with the assistive technologies in the product's support policy, then reduce duplication.

Next works, but it never reaches every record

Boundary clicks are not enough. A Next button can move on every press and still visit only a subset of the collection. Record stable IDs for one full cycle and compare the ordered sequence, not just the count of clicks.

PixelVault has six photos and advances by two. In modular arithmetic, starting at zero produces 0, 2, 4, 0. Indices 1, 3, and 5 are unreachable through that control. With five photos, the same defect would eventually visit all records because two and five share no common factor. That is why an odd-sized fixture can hide this bug.

Choose fixture sizes for the fault you need to expose. A one-item gallery tests disabled or no-op controls. Two items reveal duplicate transitions. Six items expose an increment-by-two defect quickly. These are structural properties of the fixture, not measured defect rates.

The fix depends on the navigation contract. A wrapping gallery can use (index + 1) % length. A gallery that stops at the end should disable Next on the final photo and leave the index unchanged. Both can be accessible. Shipping one behavior while the label, help text, or tests assume the other is the actual problem.

Keep index arithmetic below the browser layer. A pure unit test can cover zero items, one item, both edges, and wrapping without opening a page. Retain one Playwright sequence test because filtering, sorting, stale closures, and DOM updates can still connect the correct function to the wrong collection.

Also assert the displayed identity after each move. The dialog's accessible name may update while the image source remains stale, or the image can change while a live status still announces the prior position. Pair one accessibility-facing oracle, such as the dialog name or status text, with one record oracle, such as a stable photo ID supplied by the test fixture.

Download starts, but it belongs to another photo

The browser's download event proves that a download began. It does not prove that the right record produced it. A handler hardcoded to the first array item emits a perfectly legitimate event on every click.

Wait for the event before clicking, then inspect an identity the product actually guarantees. For PixelVault's deterministic fixture, both the suggested filename and text payload identify the selected photo. Playwright documents that suggestedFilename() can be derived from the response's Content-Disposition header or the element's download attribute, and browsers may apply different logic. Use an exact filename only when it is part of the product contract.

import { readFile } from "node:fs/promises";
import { expect, test } from "@playwright/test";

test("Download uses the photo open in the lightbox", async ({
  page,
}, testInfo) => {
  await page.goto("/arenas/image-gallery");

  await page.getByRole("button", {
    name: "Open Snowfield silence",
  }).click();

  const dialog = page.getByRole("dialog", {
    name: "Snowfield silence",
  });
  const downloadPromise = page.waitForEvent("download");

  await dialog.getByRole("button", { name: "Download" }).click();
  const download = await downloadPromise;

  expect(download.suggestedFilename()).toBe("Snowfield silence.txt");

  const savedPath = testInfo.outputPath("snowfield-download.txt");
  await download.saveAs(savedPath);
  const contents = await readFile(savedPath, "utf8");

  expect(contents).toContain("Snowfield silence");
});

The Playwright Download API notes that the event is emitted when the transfer starts and that saveAs() waits for completion. That difference matters. An assertion made immediately after the event can pass before the file is available.

Production downloads may be resized, watermarked, compressed, or served through signed URLs. In that case, a literal payload comparison is too brittle. Seed a fixture with an immutable asset ID in metadata, verify the request or response contract, and reserve byte-level checks for the service that performs the transformation. The trade-off is test-only fixture plumbing, but it produces a failure that names the wrong record rather than a generic file mismatch.

What looks like the same failure but is not

Two gallery failures can produce the same timeout and require opposite changes. Before editing selectors or adding waits, collect the discriminator that separates them.

A role-locator timeout deserves special care. Replacing it with .gallery button:nth-child(2) may make the test green while preserving the user-facing naming defect. First inspect the computed role and name. If the control is correctly named but appears asynchronously, wait for the user-visible loading transition or the control itself. If the name is wrong, fix the public interface and keep the role locator.

Lost focus and invisible focus are separate defects. When document.activeElement is the intended button but no indicator can be seen, changing JavaScript focus management will not help. Capture a screenshot with the focused element and inspect CSS, clipping, overlays, contrast, and WCAG 2.2's focus requirements. When the active element remains behind the overlay, a thicker outline inside the dialog will not help either.

A skipped visual can also be a data problem. Suppose the ordered IDs are complete but two titles and thumbnails are duplicates. The user may reasonably report that Next repeated an image even though the state machine visited different records. Compare stable IDs, image sources, alternatives, and visible captions. Then decide whether the duplication is valid content, a cache key collision, or incorrect rendering.

Announcements are particularly easy to overclaim. An aria-live node changing in the DOM is evidence that the application offered an update through an accessibility mechanism. It is not proof of exact spoken words across every browser and screen-reader pair. Automated checks should assert the relevant tree and state change. Manual checks should record the browser, operating system, assistive technology, navigation mode, and user-observed output.

Automated rule scans sit beside these diagnostics, not above them. A scanner may catch an unnamed control or invalid ARIA relationship. It cannot infer that a six-photo Next loop reaches only three records, and it cannot know that a valid download belongs to the wrong photo. Keep scan findings, keyboard behavior, and business-state assertions as separate report categories so one green badge does not erase another layer's failure.

How to roll these checks into an existing suite

Snapshotting the whole page is the wrong first move. Existing galleries often contain generated captions, personalization, ads, or changing result counts. A broad baseline creates review noise and encourages people to approve diffs without reading them.

Start with an interaction ledger agreed by design, engineering, QA, and content:

• Which visuals are informative, decorative, functional, or complex?
• What accessible name should each control expose?
• Does the grid use ordinary Tab stops, a composite pattern, or simple links?
• Does navigation wrap, stop, or disable controls at the edges?
• Where should focus land on lightbox open and return on close?
• Which changes require a status announcement?
• What stable identity ties the visible image to Favorite and Download?
• Which browsers, zoom levels, and assistive technologies does the product support?

Run the ledger manually against production-like fixtures before freezing assertions. This is where you find disagreement in the product contract. Automation cannot resolve whether a caption is an adequate alternative or whether arrow keys belong in a simple grid.

Add regression coverage in risk order. Put pure index, filter, favorite-membership, and record-selection functions under unit tests. Add browser tests for computed semantics, focus, keyboard activation, the full Next sequence, close-and-return behavior, and download identity. Keep one short manual assistive-technology script for description quality and announcement behavior.

For an already red suite, create one issue per observed failure and land the tests with an explicit ownership plan. Do not approve new ARIA snapshots that merely encode known broken structure. If a temporarily expected failure is unavoidable, include the issue reference and removal condition in the test name or annotation. An unexplained skip becomes permanent surprisingly quickly.

Artifact settings should match the retry policy. If retries are disabled, trace: "on-first-retry" never records the original failure because there is no retry. Use retain-on-failure when the first failing run is the evidence you need. If the team enables retries, keep the first failure and report the retry as flaky rather than silently converting the result to a clean pass.

A focused Playwright configuration can preserve useful evidence without recording every successful run:

import { defineConfig, devices } from "@playwright/test";

const baseURL =
  process.env.PLAYWRIGHT_BASE_URL ?? "http://127.0.0.1:3000";

export default defineConfig({
  testDir: "e2e",
  forbidOnly: Boolean(process.env.CI),
  retries: 0,
  outputDir: "test-results",
  reporter: [
    ["line"],
    ["html", { open: "never", outputFolder: "playwright-report" }],
  ],
  use: {
    baseURL,
    trace: "retain-on-failure",
    screenshot: "only-on-failure",
  },
  projects: [
    {
      name: "chromium",
      use: { ...devices["Desktop Chrome"] },
    },
  ],
  webServer: process.env.PLAYWRIGHT_BASE_URL
    ? undefined
    : {
        command: "pnpm dev",
        url: baseURL,
        reuseExistingServer: !process.env.CI,
      },
});

One browser project is a sensible first gate when the team is introducing the checks. Adding Chromium, Firefox, and WebKit executes every path three times. That extra coverage may be justified for modal focus and keyboard behavior, but it also increases compute, artifacts, and triage. Expand the matrix from the product's support policy and defect history, not from a desire to make the YAML look comprehensive.

The CI job should run the narrow accessibility file on each relevant change and upload evidence even when the test fails. This GitHub Actions example assumes the repository already uses pnpm and has a test:e2e-compatible Playwright installation.

name: Gallery accessibility

on:
  pull_request:
    paths:
      - "src/**"
      - "e2e/gallery-accessibility.spec.ts"
      - "playwright.config.ts"

jobs:
  gallery:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    steps:
      - uses: actions/checkout@v6
      - uses: pnpm/action-setup@v6
        with:
          version: 10
      - uses: actions/setup-node@v6
        with:
          node-version: 22
          cache: pnpm
      - run: pnpm install --frozen-lockfile
      - run: pnpm exec playwright install --with-deps chromium
      - run: pnpm exec playwright test e2e/gallery-accessibility.spec.ts --project=chromium
      - uses: actions/upload-artifact@v7
        if: always()
        with:
          name: gallery-accessibility-evidence
          path: |
            playwright-report/
            test-results/
          retention-days: 14

The timeout and artifact retention in that workflow are policy choices, not performance measurements. The setup step pins pnpm 10 because this repository does not declare a package-manager version; another project should use its own pinned version. Adjust the limits to the repository's normal policy and privacy rules. Traces can contain DOM text, network details, and screenshots of user content. A gallery for medical, legal, or internal assets may require synthetic fixtures or artifact redaction before CI uploads are acceptable.

ARIA snapshots also carry a maintenance cost. They are order-sensitive and can use partial matching, so scope them to the gallery or dialog and include only structure that represents the user contract. A full-page snapshot will churn when navigation or unrelated copy changes. Never run snapshot updates as an automatic repair step after a failure. Review the semantic difference first, then update the baseline only when the product change is intentional.

Rollout is complete when ownership is clear, not when every possible combination runs on every pull request. Keep deterministic semantics and keyboard paths in the fast gate. Schedule broader browser, zoom, forced-colors, and assistive-technology reviews according to release risk. Record which layer each job covers so stakeholders do not read "gallery checks passed" as a complete WCAG conformance statement.

When this approach costs more than it proves

An inline detail panel is not a modal. If opening a thumbnail expands content in the normal document flow and the rest of the page remains usable, focus containment may be the wrong requirement. Test the actual disclosure behavior, reading order, and focus consequences instead.

A genuinely decorative flourish does not need an image role. An empty alt on an img, or a CSS background with no accessibility-tree node, can be correct. The evidence needed is the content decision and the nearby text that already carries the information. Adding a redundant description to satisfy a locator makes the experience noisier.

Carousel arrow keys do not belong in every thumbnail grid. Ordinary links and buttons can follow normal Tab navigation. Composite widgets can use arrow-key movement when the chosen pattern and instructions support it. A test that invents its own keyboard model can reject an accessible implementation.

Pure arithmetic rarely deserves a browser run at every boundary. Index wrapping, case normalization, membership guards, and record selection are faster and easier to diagnose as unit tests. Keep browser coverage for the points where DOM semantics, real focus, event order, and application state meet. The trade-off is maintaining fixtures at two layers, but failures become much more specific.

A giant ARIA snapshot is a change detector, not a conformance certificate. It cannot assess whether an alternative accurately describes the image, whether the focus indicator is visible, whether zoom obscures a control, or what a particular assistive technology announces. It can detect a changed accessible structure, which is a narrower and still valuable claim.

Exact production filenames are brittle when a CDN or server intentionally rewrites them. Verify a stable asset ID, response contract, or fixture payload instead. Conversely, checking only that some file downloaded is too weak when the business requirement is that the open photo downloads. Choose the identity at the boundary the product owns.

Test IDs should carry internal record identity, not conceal a missing accessible name. They are useful when two photos share a title. Roles and names should locate user-facing controls. Using both gives you two independent oracles: what the user can discover and which record the application changed.

Browser automation is not a screen reader surrogate. Use it to catch deterministic semantic, focus, state, and side-effect regressions. Put a human tester with the supported assistive technology on the interaction before release, especially after changes to live announcements, focus order, alternative text, or modal structure.