Old Chrome headless is gone: migrate Selenium without masking regressions

A Chrome update lands in CI, and every session configured with --headless=old keeps starting exactly as it did last quarter. Nothing in the log changes, yet the flag no longer selects the implementation the config file still names. Another team launches successfully too, but a responsive menu now covers its checkout button. Both incidents mention headless mode, and neither one is the failure its label suggests.

Know which browser you are actually comparing

The words old and new used to select two implementations inside Chrome. Old headless was a separate browser implementation bundled with the Chrome binary. Unified headless arrived later and shares Chrome's browser code while creating platform windows without displaying them. That architecture removed many historical differences between headless and headed Chrome, but it did not make test environments identical.

The timeline changes what a command-line flag proves. Chrome 112 introduced the unified implementation under the new-headless selection. Chrome 132 removed old headless from the main Chrome binary. On Chrome 132 and later, --headless and --headless=new run unified headless. --headless=old no longer starts the old implementation. Chrome's old code remains available as the separate chrome-headless-shell binary for users who intentionally need it.

This makes a common comparison invalid. Running current Chrome once with --headless and once with --headless=new does not compare old and new engines. It launches the same headless implementation through two accepted flag forms. Different results between those jobs must come from another changed input, nondeterminism, or wrapper behavior.

Selenium does not implement Chrome's rendering mode. It places browser arguments in Chrome options and asks ChromeDriver to start the browser. Selenium's 2023 guidance matters historically: convenience APIs that set a generic headless boolean were deprecated around Selenium 4.8 and removed in Selenium 4.10 so callers would choose the browser argument explicitly. Current Selenium examples use ChromeOptions.addArguments("--headless=new") or the corresponding binding API.

That older Selenium post described both Chrome modes as they existed at the time. It should not be read as evidence that a current Chrome binary still includes both. Pair it with the current Chrome headless documentation, which records the Chrome 132 removal and the standalone shell.

Record three identities in every comparison. First, record the requested mode and arguments before session creation. Second, record the browser binary or image and its version. Third, record the returned browserVersion capability after creation. Returned capabilities do not have to echo every command-line argument, so the session response alone cannot prove which headless flag your factory requested.

Remote execution adds another identity. A local google-chrome --version command tells you about the CI client host, not necessarily the browser inside a Selenium Grid node. Query the node image through deployment records or collect the capabilities from the created session. If the session never starts, use the node's driver log and image metadata. Do not combine a client-host version with a remote-node error and call it one browser.

Binary paths matter as much as version strings during a migration. A container can contain Chrome for Testing, a system Chrome package, and chrome-headless-shell at once. ChromeDriver may receive an explicit binary path through goog:chromeOptions, while an engineer's shell resolves another executable from PATH. Record the configured path without leaking unrelated environment variables. Resolve symlinks in deployment diagnostics so two names for one binary are not mistaken for two implementations.

Keep ChromeDriver identity separate too. Selenium Manager can discover a compatible driver for local execution, while a Grid image usually supplies its own pair. The version in a developer's Selenium Manager cache does not explain what a remote node used. On remote failures, capture the node image tag and driver startup line. On local failures, capture Selenium Manager output and the resolved executable paths.

The standalone shell deserves an explicit name in reports. Label it chrome-headless-shell, not "Chrome old mode," and record its own artifact version. A test that passes in the shell and fails in current Chrome demonstrates a binary difference. It does not prove current headless Chrome is wrong. Decide which binary represents the product's supported user environment before choosing the expected result.

Reproduce the difference without changing the environment

Mode is one input among many. A useful comparison keeps the browser image, ChromeDriver, Selenium client, operating system, application build, test data, locale, timezone, fonts, window dimensions, and feature flags the same. Run the two cases close enough together that backend state and deployments do not drift. If headed execution needs a virtual display, use it in the same container image that runs headless.

Do not change browser version while changing mode. A local headed run on a developer laptop and a headless run in a Linux container compare operating systems, fonts, graphics stacks, network paths, and browser builds. That pair can reproduce a symptom, but it cannot assign the cause to headless mode.

Use a fresh browser profile for each comparison unless profile persistence is the feature under test. Reusing --user-data-dir can carry cookies, local storage, cached resources, permission choices, extension state, and an unclean-shutdown marker from one run into the next. It can also introduce a profile lock that prevents the second session from launching. If a persistent profile is required, clone the same prepared source into separate run directories and record the preparation step.

Hold execution order still where backend state mutates. If the headed run consumes the only coupon or confirms the only pending order, the headless run no longer sees the same application state. Seed independent records with equivalent values, or reset through a supported test API before each case. Sharing one mutable account saves setup time but turns mode order into an invisible variable.

Treat waits as part of the test contract. A fixed sleep can pass in one mode because that run happens to render faster, then fail in the other under the same product behavior. Wait for the state the user needs, such as an enabled order button or a visible confirmation, and keep the same timeout policy in both runs. If one mode consistently reaches that state later, retain timing traces for a performance investigation instead of lengthening only its sleep.

Represent mode as a typed framework input rather than an arbitrary argument string. The factory below creates either headed or unified-headless Chrome against the same remote endpoint, then requests the same outer window size through WebDriver. The dimensions are a chosen test input, not a claim that one size is universally correct.

package example.browser;

import java.net.MalformedURLException;
import java.net.URI;
import org.openqa.selenium.Dimension;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;

public final class ChromeFactory {
  public enum Mode { HEADED, HEADLESS }

  public static WebDriver create(URI gridUri, Mode mode)
      throws MalformedURLException {
    ChromeOptions options = new ChromeOptions();
    if (mode == Mode.HEADLESS) {
      options.addArguments("--headless=new");
    }

    WebDriver driver = new RemoteWebDriver(gridUri.toURL(), options);
    driver.manage().window().setSize(new Dimension(1440, 900));
    return driver;
  }

  private ChromeFactory() {}
}

An outer window size is not the same as the page's content viewport. Browser decorations and platform behavior can affect window.innerWidth and window.innerHeight. Capture both the WebDriver window rectangle and page-level metrics. If the product changes layout at a breakpoint, the content viewport is the number that explains which component rendered.

Make the product assertion identical in both modes. A screenshot existing is not a product oracle. Click the checkout button and assert that the confirmation appears, then retain the screenshot and metrics to explain a disagreement. If a screenshot comparison is itself the product contract, define the supported environment narrowly and keep separate reviewed baselines where rendering legitimately differs.

The following JUnit test runs the checkout path in the mode requested by the CI invocation. The matrix shown later calls it once per supported mode. It writes observations and a screenshot even when the later assertion fails. Changing the application so the status never reports success will break the assertion, which keeps the test tied to product behavior rather than artifact presence.

package example.browser;

import static org.junit.jupiter.api.Assertions.assertTrue;

import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Locale;
import java.util.Map;
import org.junit.jupiter.api.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.HasCapabilities;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;

class CheckoutModeTest {
  @Test
  void checkoutWorksInTheRequestedMode() throws Exception {
    ChromeFactory.Mode mode = requestedMode();
    Path artifacts = Path.of(
        "target", "headless-comparison", mode.name().toLowerCase(Locale.ROOT));
    Files.createDirectories(artifacts);

    WebDriver driver = ChromeFactory.create(
        java.net.URI.create(System.getenv("SELENIUM_REMOTE_URL")), mode);
    try {
      driver.get(System.getenv("APP_URL") + "/checkout");

      Map<?, ?> metrics = (Map<?, ?>) ((JavascriptExecutor) driver).executeScript(
          "return {innerWidth: window.innerWidth, innerHeight: window.innerHeight," +
          " devicePixelRatio: window.devicePixelRatio, language: navigator.language}");
      String browserVersion = ((HasCapabilities) driver)
          .getCapabilities().getBrowserVersion();
      Files.writeString(
          artifacts.resolve("observation.txt"),
          "requestedMode=" + mode + System.lineSeparator()
              + "browserVersion=" + browserVersion + System.lineSeparator()
              + "outerWindow=" + driver.manage().window().getSize() + System.lineSeparator()
              + "pageMetrics=" + metrics + System.lineSeparator());

      driver.findElement(By.cssSelector("[data-testid='place-order']")).click();
      String status = driver.findElement(By.cssSelector("[role='status']")).getText();
      Files.write(
          artifacts.resolve("result.png"),
          ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES));

      assertTrue(status.toLowerCase(Locale.ROOT).contains("confirmed"), status);
    } finally {
      driver.quit();
    }
  }

  private static ChromeFactory.Mode requestedMode() {
    String raw = System.getProperty("ui.mode");
    if (raw == null) {
      throw new IllegalStateException("Set -Dui.mode=headless or -Dui.mode=headed");
    }
    try {
      return ChromeFactory.Mode.valueOf(raw.toUpperCase(Locale.ROOT));
    } catch (IllegalArgumentException error) {
      throw new IllegalArgumentException(
          "ui.mode must be headless or headed, but was: " + raw, error);
    }
  }
}

This test costs two sessions and writes more artifacts. Use it for a focused set of mode-sensitive flows rather than every locator test. The observation file is diagnostic text, not a golden snapshot. Browser patch upgrades and platform changes can alter capability strings without changing the product.

Separate three failures that look like a headless regression

The first failure never produces a failure. A framework passes --headless=old to a current Chrome binary, the session starts, and the suite goes green. Chrome takes the argument, ignores the value after the equals sign, and runs unified headless anyway. The configuration still advertises the old implementation, the dashboard still shows a legacy lane, and the coverage that lane claims disappeared at a browser version bump nobody connected to it.

Chrome did once object. When Chrome 132 removed old headless from the main binary in January 2025, the release note said the binary would print a helpful error message. Read that as a dated observation, not a standing contract. Measured on Chrome 151.0.7922.170 in August 2026, --headless=old, --headless=new, bare --headless, and an invented value such as --headless=not-a-real-implementation all start a session and render the same page, and stderr carries nothing matching headless, old mode, or removed. A Selenium 4.39 session built with --headless=old against that binary returns browserVersion 151.0.7922.170 and behaves identically to one built with --headless=new. Somewhere between those releases the diagnostic went away, so a suite that survived the removal by ignoring a warning has no warning left to ignore.

That changes what a probe should ask. Whether --headless=old starts is no longer informative, because it starts everywhere. The two useful questions are whether the binary reads the value at all, and which implementation actually rendered the page.

The script below answers both. It prints the Chrome version, then renders one small data URL twice: once with --headless=old, and once with a value no Chrome release has ever implemented. If both render, the binary is not interpreting the value, so --headless=old is evidence of nothing. The page also reports navigator.pdfViewerEnabled and navigator.plugins.length, which do separate the two implementations. The full Chrome binary carries the PDF viewer and reports true and 5; chrome-headless-shell, which is the old implementation, reports false and 0.

#!/usr/bin/env bash
set -uo pipefail

: "${CHROME_BIN:?Set CHROME_BIN to the Chrome executable used by the node}"

timeout_cmd="$(command -v timeout || command -v gtimeout || true)"
: "${timeout_cmd:?Install coreutils so the probe can bound each Chrome run}"
probe_seconds="${PROBE_SECONDS:-30}"

marker='<title>headless-probe</title>'
sentinel='not-a-real-implementation'
page='data:text/html,<title>headless-probe</title><body>x</body>'
page+='<script>document.body.textContent="pdfViewerEnabled="+navigator.pdfViewerEnabled'
page+='+" plugins="+navigator.plugins.length;</script>'

work_dir="$(mktemp -d)"
trap 'rm -rf "$work_dir"' EXIT

render_with() {
  local value="$1" name="$2" profile
  profile="$(mktemp -d "$work_dir/profile.XXXXXX")"
  # Chrome does not always exit after --dump-dom, so bound the run and judge it
  # by the DOM it printed rather than by an exit status that may never arrive.
  "$timeout_cmd" "$probe_seconds" "$CHROME_BIN" \
    "--headless=$value" \
    --user-data-dir="$profile" \
    --dump-dom \
    "$page" >"$work_dir/$name.dom" 2>"$work_dir/$name.err"
  grep -qF "$marker" "$work_dir/$name.dom"
}

"$CHROME_BIN" --version

render_with old old && old_rendered=yes || old_rendered=no
render_with "$sentinel" sentinel && sentinel_rendered=yes || sentinel_rendered=no

printf 'rendered with --headless=old: %s\n' "$old_rendered"
printf 'rendered with --headless=%s: %s\n' "$sentinel" "$sentinel_rendered"

if [ -s "$work_dir/old.dom" ]; then
  printf 'implementation markers under --headless=old: %s\n' \
    "$(sed -n 's/.*<body>\(.*\)<\/body>.*/\1/p' "$work_dir/old.dom")"
fi

if [ "$old_rendered" = yes ] && [ "$sentinel_rendered" = yes ]; then
  printf '%s\n' 'This binary accepts any --headless value, so --headless=old is silently ignored here. Drop the flag and select the binary you actually want.'
elif [ "$old_rendered" = yes ] && [ "$sentinel_rendered" = no ]; then
  printf '%s\n' 'This binary validates the --headless value and still accepts old, so it predates the removal. Confirm its version and plan the migration.'
elif [ "$old_rendered" = no ] && [ "$sentinel_rendered" = no ]; then
  printf '%s\n' 'This binary rejects --headless=old outright. Switch the framework to --headless=new. Chrome stderr follows:'
  sed -n '1,40p' "$work_dir/old.err"
else
  printf '%s\n' 'Unexpected pairing: old failed while an invented value rendered. Keep both stderr logs and inspect the wrapper.'
  sed -n '1,40p' "$work_dir/old.err"
fi

Against Chrome 151 the script reports rendered with --headless=old: yes, the same for the invented value, and implementation markers under --headless=old: pdfViewerEnabled=true plugins=5, which names unified headless. Pointed at chrome-headless-shell it reports the same two yes lines but pdfViewerEnabled=false plugins=0, which names the old implementation. The value of the flag did not decide either answer; the binary did.

Two details in that script matter more than they look. Chrome does not reliably exit after --dump-dom. On macOS, a run with a fresh --user-data-dir printed the DOM and then sat there until a 30 second bound killed it, while the same command without an explicit profile returned in about a second. Every run is therefore wrapped in timeout and judged by the DOM it printed, never by an exit status that may never arrive. A version of this probe that chained the Chrome invocation to grep with && hangs indefinitely on that platform, and its cleanup trap then races a profile directory a live browser is still writing to.

Run this on the browser node, not just on the test client, and use the same operating-system user as the node. A container entrypoint can point ChromeDriver at a different binary from the shell's default chrome. If the node requires documented non-mode arguments, keep those inputs the same in the probe. If direct probing renders the page but WebDriver fails, compare binary paths and ChromeDriver logs before changing application tests.

The same two properties read cleanly from inside a Selenium session, which is where they belong once the probe has told you what to expect. Assert navigator.pdfViewerEnabled in a startup check next to the recorded browserVersion, and the suite will notice the day a container swap replaces full Chrome with the shell. Treat that assertion as an environment contract rather than a product assertion, and keep it in one setup test rather than every scenario.

The fix is to stop claiming an implementation through a flag value the browser discards. Remove --headless=old from every factory, entrypoint, and nightly script. Use --headless=new when you want unified headless in the main binary, and invoke chrome-headless-shell directly when you deliberately want the old one. Keeping the shell preserves old behavior but adds a separate downloaded binary, a driver pairing, and a support burden. Choose it only when old behavior is an intentional requirement, and do not quietly substitute it for current Chrome to make old screenshots pass.

The second failure occurs after navigation at a responsive breakpoint. Headless starts with an implicit window size, the application renders its compact menu, and an overlay intercepts a click that was visible in a larger headed window. The exception mentions an intercepted or non-interactable element, which tempts teams to add a wait.

Capture window.innerWidth, the target's bounding rectangle, the intercepting element from the failure, and the screenshot. Then rerun both modes with the same explicit size. If the difference disappears, the cause was an uncontrolled viewport input, not the existence of unified headless. A longer wait may reduce timing noise but cannot move a permanently covered button.

The fix costs responsive coverage. Forcing every test to a desktop size prevents accidental breakpoint changes, but it also means the suite no longer exercises mobile and narrow layouts. Keep the main flow at an explicit desktop dimension and add intentional viewport cases for supported breakpoints. Label those tests by viewport contract rather than by headless mode.

The third failure is environmental rendering. A container lacks the font used on developer machines, defaults to another locale, or uses a different graphics configuration. Text wraps onto another line and shifts a control. Because CI is headless and local execution is headed, the mode gets blamed even though the operating environments differ.

Run headed Chrome inside the same CI image and virtual display. Check navigator.language, window.devicePixelRatio, computed font-family, and document.fonts.check() for the required face. If headed and headless fail the same way in the container, mode is not the separating variable. Install or bundle the supported font, set locale intentionally, and verify its presence before the UI test.

Font installation increases image size and needs license review. Bundled web fonts reduce host dependency but change application delivery and may introduce loading timing. Choose based on the product contract. A test framework should not smuggle a proprietary desktop font into CI merely to match a developer screenshot.

A stale visual baseline can produce a fourth symptom without any functional regression. The baseline may have been approved under the removed implementation, while the new run comes from unified Chrome. If DOM structure, content viewport, computed styles, loaded fonts, and the user outcome agree, inspect the pixel difference rather than automatically changing layout code. Anti-aliasing and raster output can change between browser implementations or graphics environments even when element geometry does not.

Rebaseline only after a reviewer checks the intended design in a supported browser. Keep the old image beside the proposed baseline during that review, then record the browser image and mode that produced the replacement. The cost is losing direct continuity with historical pixels, but preserving an obsolete renderer as the approval oracle would be worse. Where small raster variation is expected, narrow the comparison to stable regions or assert geometry and behavior instead of applying a generous tolerance to the whole page.

A near-miss is ChromeDriver and Chrome incompatibility. This one does fail before a session exists, and it produces no application evidence at all. Version logs show the driver and browser identities, while stderr talks about an unsupported browser version rather than any headless argument. Let Selenium Manager or a pinned compatible image resolve the pair. Changing headless arguments cannot repair an incompatible executable combination, and a suite that reaches for the mode switch here will chase the wrong variable for a day.

Another near-miss is a headed run without a display server on Linux. Headless succeeds because it does not display windows, while headed Chrome fails to launch. That result proves the headed environment is incomplete, not that the application only works headless. Configure a suitable display such as Xvfb for the comparison and record it as part of the runner image.

Move the suite to unified headless deliberately

Begin with an argument inventory. Search factories, environment variables, Docker entrypoints, Grid node settings, and test-specific overrides for --headless, --headless=new, --headless=old, older aliases, and removed Selenium convenience methods. Record which binary each path launches. A single modern factory does not help if a nightly script still injects the old flag.

Next, reduce the factory to two supported states: current headed Chrome and current unified headless Chrome. Reject an input named legacy unless it points to a separately managed shell adapter. This converts a vague boolean into a reviewable support choice. Unknown strings should fail configuration loading rather than silently selecting headed mode.

Run a shadow lane on a focused contract set. Include navigation, authentication, one responsive page, file behavior your product relies on, and a screenshot-sensitive component if pixels are part of the acceptance criteria. Use the same application deployment and data seed for both modes. Store artifacts under different mode names so one run cannot overwrite the other.

Classify every shadow-lane disagreement before changing the gate. A launch failure belongs to browser configuration or infrastructure. A different viewport belongs to test setup. Matching layout with a pixel-only delta belongs to visual review. A different user-visible outcome belongs to product or browser behavior and deserves the deepest investigation. This routing is useful because each class has different evidence and owners; a blanket "headless failed" label sends all four to the framework team.

Keep the shadow result advisory until the known differences have owners. Advisory does not mean ignored. Publish both outcomes, link each accepted difference to a tracked reason, and prevent new unexplained differences from entering silently. Once the contract set is clean, make unified headless required and keep headed failures visible through the team's chosen scheduled or pull-request policy.

The CI matrix below uses one pinned Selenium standalone image for both cases. That image provides the browser and driver pair, while Maven passes one ui.mode system property to the test. A project using this job needs the remote factory and mode handling from the earlier examples.

name: Chrome mode contract

on:
  pull_request:

jobs:
  chrome-mode:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        mode: [headless, headed]
    services:
      chrome:
        image: selenium/standalone-chrome:4.46.0-20260707
        env:
          SE_START_XVFB: "true"
        ports:
          - 4444:4444
        options: >-
          --shm-size=2g
          --health-cmd "/opt/bin/check-grid.sh --host 0.0.0.0 --port 4444"
          --health-interval 5s
          --health-timeout 3s
          --health-retries 20
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: "21"
          cache: maven
      - name: Run the mode-sensitive contract set
        env:
          SELENIUM_REMOTE_URL: http://localhost:4444
          APP_URL: ${{ vars.TEST_APP_URL }}
        run: >-
          ./mvnw --batch-mode
          -Dui.mode=${{ matrix.mode }}
          -Dtest=CheckoutModeTest
          test
      - name: Keep comparison artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: chrome-${{ matrix.mode }}-${{ github.run_attempt }}
          path: target/headless-comparison/
          if-no-files-found: warn

Pinning costs maintenance. The browser does not update until the image tag changes, so security patches and user-facing regressions can arrive late. Pair the stable pull-request lane with a scheduled upgrade lane. When that lane passes, update the pinned image through review with the version difference visible.

Test the mode parser without opening Chrome. Inputs such as headless and headed should map to the two enum values, while old, an empty value, and misspellings should fail with a configuration error. That unit test catches accidental fallback cheaply. The browser integration lane then proves that the selected enum changes the actual Chrome options and that both supported modes complete the product contract.

Watch resource consumption during the temporary dual run. Headed Chrome under a virtual display and unified headless can have different memory or startup characteristics in a given image, but do not publish one run's figures as a general benchmark. Use your CI history to size Grid slots and timeouts. If capacity queues one matrix leg behind the other, separate queue delay from browser execution time before claiming a mode is slower.

Treat baseline changes as code changes. A new unified-headless screenshot that differs from an old-headless baseline is not automatically wrong, and approving every new image is not migration. Inspect the product behavior, layout metrics, and intended design. If old headless rendered something the real supported Chrome does not, current Chrome should normally define the new expectation.

Remove the old path after the observation period. Leaving a USE_OLD_HEADLESS escape hatch invites teams to flip it during an incident and stop investigating. If a historical defect still requires shell reproduction, keep a separate, manually selected job with an owner and removal condition. It should not participate in the normal green gate.

Keep headed coverage without doubling every build

Unified headless shares Chrome's browser code, which makes it a strong default for CI. Headed coverage still catches environment assumptions around display setup, window management, and workflows that teams inspect manually. Running every test in both modes doubles session demand and can lengthen feedback without doubling useful coverage.

Choose tests by risk. Run the broad functional suite in unified headless. Run a smaller headed contract set on pull requests or a scheduled lane. Include failures previously known to differ, critical purchase or authentication paths, and one representative of each major layout system. Rotate additional tests when browser or image versions change.

Keep responsive coverage orthogonal. A matrix of two modes multiplied by several viewport sizes and browsers grows quickly. Most locator and API-backed assertions do not earn every combination. Select viewport cases based on product breakpoints, browser cases based on supported engines, and mode cases based on a plausible rendering or lifecycle risk.

Screenshot systems need explicit baseline ownership. If headed and headless pixels differ for legitimate platform reasons, store separate baselines with clear labels. Do not apply a large global tolerance that makes both pass, because it can hide meaningful movement. Prefer DOM and behavior assertions for flows where exact pixels are not the contract.

Budget Grid capacity for the comparison lane. Two modes started concurrently consume two slots, and tests that share data may collide. Give each matrix job independent accounts or seeded entities. If the jobs must serialize, record that choice so a longer runtime is not misdiagnosed as browser slowness.

Collect failure artifacts consistently in both modes. A headed run under Xvfb still needs automatic screenshots and logs because nobody is watching its invisible display in CI. A screen recording can help with motion or transient overlays, but it adds storage and encoding work. Enable it for focused cases or failures rather than assuming every run needs video.

Use current headed Chrome as the comparison for current headless Chrome. The standalone old shell answers a historical implementation question, not the ordinary user-browser question. Keeping that distinction in job names prevents a dashboard from showing three Chrome lanes that appear equally supported.

When a legacy comparison no longer earns its cost

Do not compare --headless with --headless=new on current Chrome and call the result old versus new. Both select unified headless. If the jobs differ, inspect the actual command construction, environment, and test state. The labels do not create different browser implementations.

Do not retain chrome-headless-shell only because old screenshots are stable. The shell is a separate artifact with its own upgrade, security, distribution, driver, and baseline concerns. Keep it when the product explicitly embeds it, when a customer environment requires it, or when an active browser bug needs historical reproduction. Attach an owner and a date or condition for reevaluation.

Avoid cargo-cult startup flags during migration. Modern unified headless does not require every flag copied from an old container tutorial. Options such as disabling GPU or sandboxing change more than visibility and can introduce security or rendering differences. Add a flag only for a documented environment requirement, and test its effect separately from the headless switch.

Do not blame mode when both current headed and current headless fail inside the same image. Shared failures point toward the application, browser version, driver, test data, network, font set, or container. Keep the mode artifacts, but move the investigation to the input that separates passing and failing runs.

Do not use a headless comparison for a command rejected before Chrome launches. An invalid WebDriver capability, inaccessible Grid URL, unavailable slot, or browser-driver mismatch has no rendered mode to compare. Read the session response and driver logs first. Adding screenshot capture to a session that never existed only creates misleading missing-artifact noise.

Avoid fixing responsive failures by maximizing the window without recording the resulting viewport. Window manager behavior varies, and maximize may not mean the same dimensions in a virtual display. Set a deliberate size, capture the content viewport, and add separate tests for the narrow layouts the product supports.

The useful exit condition is support scope. Once normal CI runs current unified headless, a smaller lane covers current headed Chrome, and no supported workflow depends on the standalone shell, remove the legacy comparison. Spend the recovered browser minutes on another engine, a meaningful viewport, or a critical flow that currently lacks coverage.