When Chrome tabs die inside Selenium containers

A Chrome test runs for several minutes, then driver.get() or the next element command reports that the tab crashed. The Selenium status endpoint is still green, and rerunning the test alone passes. That combination points toward a browser process losing a container resource, but it does not prove which resource ran out.

Shared memory is a common suspect because Chrome is a multi-process browser and container runtimes often give /dev/shm a much smaller mount than the host. Memory exhaustion, PID limits, a browser defect, and an application-triggered renderer crash can look almost identical from the test client. The useful fix starts with evidence from the browser container at the moment of failure.

Why a healthy Grid can still lose the tab

WebDriver commands travel through several owners. The Python or Java test sends a command to Selenium Server or ChromeDriver. ChromeDriver talks to a browser process, which coordinates renderer, GPU, network, and utility processes. The web application runs inside a renderer. A successful request to the Grid status endpoint proves the server can answer that request; it does not prove every renderer owned by every session is alive.

Linux exposes /dev/shm as a shared-memory filesystem. Chrome uses shared memory as part of its process architecture. A container gives those browser processes their own view of that mount. If several Chrome sessions share one container, they also share that capacity. A page that creates more renderer processes, large canvases, video frames, or other memory-heavy browser work can push the mount harder than a simple login page.

The WebDriver exception arrives after the browser failure, not before it. ChromeDriver may report text containing tab crashed, a disconnected target, or a deleted session. The exact wording varies with the browser and driver versions and with the command that first notices the loss. Selenium then raises a binding exception such as WebDriverException or, after the session is gone, an invalid-session error. None of those client-side labels names /dev/shm as the cause.

This explains a familiar worked example. A suite opens one Chrome session per container and passes. The team raises node concurrency so several sessions now live in the same container. Failures begin on image-heavy routes, while lightweight API-backed pages continue to pass. The change did not make Selenium's element lookup less reliable. It changed how many browser processes compete for the same mount and cgroup limits.

The opposite pattern also matters. If the first session cannot start, inspect SessionNotCreatedException, ChromeDriver startup logs, image contents, permissions, and browser-driver compatibility before blaming shared memory. Startup can fail for resource reasons, but a clean version-mismatch message is stronger evidence than the presence of a small mount. Do not discard that evidence because the article you searched for mentioned /dev/shm.

Chrome's --disable-dev-shm-usage argument is real and appears in Selenium's Chrome options documentation. Its existence does not make it the right first change. The option changes where Chrome backs relevant shared-memory files. It does not increase the container's total memory, repair a PID ceiling, fix a broken browser build, or make a product page safe. Treat it as one controlled intervention, not an explanation.

The same caution applies to a larger shm_size. Selenium's published Docker examples use a 2 GB shared-memory setting, but that is an example configuration, not a measured requirement for every suite. Your browser count and workload determine the useful size. Copying the number without observing the actual job can leave a busy node undersized or reserve far more memory-backed capacity than an isolated smoke job needs.

Prove shared memory is the constrained resource

Collect the first snapshot before changing Chrome arguments or retrying the test. A retry may use a fresh container, a different node, or a quieter period. That passing attempt tells you little about the resource state that killed the first renderer. Preserve the failing container long enough to read its mount, cgroup, process, and browser evidence.

Run the following script inside the Selenium or browser container. It records current values rather than printing a canned sample. The cgroup v2 files are conditional because some environments expose a different hierarchy. Their absence means you need the equivalent data from the container runtime or orchestrator; it does not mean no limit exists.

#!/usr/bin/env bash
set -euo pipefail

snapshot_path="${1:-/tmp/selenium-browser-resources.txt}"

{
  date -u '+timestamp=%Y-%m-%dT%H:%M:%SZ'
  echo 'shared_memory_capacity:'
  df -Pk /dev/shm
  echo 'shared_memory_inodes:'
  df -Pi /dev/shm
  echo 'shared_memory_mount:'
  awk '$2 == "/dev/shm" { print }' /proc/mounts
  echo 'shared_memory_entries:'
  du -sk /dev/shm 2>/dev/null || true

  for metric in \
    memory.current memory.max memory.events \
    pids.current pids.max pids.events; do
    path="/sys/fs/cgroup/${metric}"
    if [ -r "$path" ]; then
      echo "${metric}:"
      cat "$path"
    fi
  done

  echo 'browser_processes:'
  browser_processes=$(
    ps -eo pid,ppid,stat,rss,args | awk '
      NR == 1 || /chrome|chromedriver|selenium-server/ { print }
    '
  )
  printf '%s\n' "$browser_processes"
  if [ -z "$browser_processes" ]; then
    echo 'ERROR: process listing was empty; ps is missing or failed' >&2
    exit 1
  fi
} >"$snapshot_path"

cat "$snapshot_path"

pipefail and the emptiness guard are both load bearing. The process listing is the last section written, and without pipefail a failed ps still leaves the pipeline's exit status at whatever awk returned, which is zero. The script would then write an empty browser_processes: section, exit 0, and hide the one part of the snapshot that identifies the browser process tree. Verify this before trusting the collector: rename ps on a scratch container and confirm the script now exits non-zero instead of producing a cheerful, useless file.

One snapshot after the crash can show that /dev/shm is full, but a time series is better. Sample before the workload, during the heaviest step, and immediately when the command fails. Keep timestamps in UTC and attach the test ID, session ID, container ID, and node name outside the script. Without that join key, a full mount from one session can be mistakenly assigned to another test that failed at the same time.

Look at capacity and usage together. A small mount with plenty of free space at the relevant moment is not proof of pressure. A nearly exhausted mount after Chrome has already exited is suggestive, but the cleanup may also have freed files before your command ran. Continuous container metrics or a short sampling sidecar close that gap. Avoid logging every filename from /dev/shm in a shared environment because names can expose implementation details and the listing rarely improves the diagnosis.

Compare block usage with inode usage as well. A filesystem can refuse a new entry because it has no inodes even when the byte counter shows free capacity. That is uncommon for a normal Chrome session, but df -Pi makes the competing cause cheap to reject. If df reports much more space in use than du can account for, an open file that was unlinked or an unreadable entry may explain the difference. Capture the process list before restarting the container. Installing a new diagnostic package into the failing container can change process and storage state, so rely on preinstalled tools or inspect the container from the runtime.

Next compare cgroup memory evidence. On cgroup v2, memory.events includes counters such as oom and oom_kill. A counter increment during the failing interval is direct evidence that the cgroup hit an out-of-memory path. It does not identify which allocation was responsible by itself. Pair it with container termination details and process logs. If the whole container exits and the runtime reports an OOM kill, increasing only shm_size cannot create more allowed memory.

PID pressure produces another near-match. Chrome creates multiple processes and threads. If pids.current approaches pids.max, and pids.events records that the maximum was reached during the failure, the container could not create more tasks. Shared memory can still look busy because the same workload consumes both resources. Raise or reduce the constrained resource you actually observed, and review session concurrency. A browser flag aimed at /dev/shm does not change the PID limit.

The browser and Selenium logs provide the transition. Save container stdout and stderr, ChromeDriver logs when available, and the original client traceback. Search around the matching timestamp for the browser process exit, renderer disconnect, or session deletion. Do not reduce the artifact to the one line containing tab crashed; the preceding process message and the following session cleanup often separate a renderer loss from a Grid transport problem.

A repeatable probe helps when the full suite is noisy. It must load a route representative of the failure and create the same number of tabs or sessions. The example below requires every important value from CI, so it cannot silently test a placeholder site or an arbitrary concurrency level.

import os
import sys

from selenium import webdriver
from selenium.common.exceptions import WebDriverException
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait


remote_url = os.environ["SELENIUM_REMOTE_URL"]
probe_url = os.environ["SHM_PROBE_URL"]
sentinel_css = os.environ["SHM_PROBE_SENTINEL"]
tab_count = int(os.environ["SHM_PROBE_TABS"])
timeout_seconds = float(os.environ["SHM_PROBE_TIMEOUT_SECONDS"])

if tab_count < 1:
    raise ValueError("SHM_PROBE_TABS must be at least 1")

driver = webdriver.Remote(command_executor=remote_url, options=webdriver.ChromeOptions())
try:
    for index in range(tab_count):
        if index > 0:
            driver.switch_to.new_window("tab")
        driver.get(probe_url)
        WebDriverWait(driver, timeout_seconds).until(
            EC.visibility_of_element_located((By.CSS_SELECTOR, sentinel_css))
        )

    assert len(driver.window_handles) == tab_count
    for handle in driver.window_handles:
        driver.switch_to.window(handle)
        assert driver.find_element(By.CSS_SELECTOR, sentinel_css).is_displayed()
except BaseException:
    try:
        driver.quit()
    except WebDriverException as cleanup_error:
        print(f"cleanup failed after probe error: {cleanup_error}", file=sys.stderr)
    raise
else:
    driver.quit()

This is a reproducer, not a capacity benchmark. Opening a blank tab consumes a different resource profile from loading the application's reporting view. Set the route and tab count from the real failing job. If production CI creates separate sessions rather than tabs in one session, reproduce separate sessions because process ownership and Grid scheduling differ.

Run the probe twice with one controlled difference. Keep the same Selenium image digest or pinned tag, application build, host class, browser arguments, session count, and input. Change only the shared-memory capacity. A failure at the old capacity and repeated success at the new capacity, combined with mount pressure and no competing OOM or PID event, makes a defensible diagnosis. One passing run after several failures is not enough because scheduling noise may have moved the job to a quieter node.

Separate the look-alikes before changing Chrome

An OOM kill is the closest near-miss. The symptom can still be a disconnected browser or missing session because the client learns only that its remote process disappeared. Check the runtime's container status and cgroup event delta. If memory pressure killed Chrome, increasing /dev/shm capacity without increasing the memory limit may make the situation worse by allowing more memory-backed pages before the cgroup intervenes.

A container-wide exit is different from one renderer crash. When the Selenium server process disappears, /status stops responding and every session on that container fails. When one renderer dies, the server can remain reachable and other sessions may continue. Do not make this distinction solely from a health dashboard sampled once a minute. Use process exit data and per-session command timelines.

PID exhaustion often appears after concurrency rises. Evidence includes a reached PID counter, failed process or thread creation in system logs, and improvement when sessions per node are reduced while memory settings remain fixed. A larger shared-memory mount may coincide with a passing rerun because the rerun also changed placement or concurrency. That is why the counterfactual must hold those variables constant.

Browser and driver incompatibility usually presents during new-session creation and names the versions or inability to start Chrome. Selenium's error documentation treats SessionNotCreatedException as a setup class with causes such as incompatible browser and driver versions, missing binaries, permissions, or system restrictions. Capture browserVersion, the effective image tag, and ChromeDriver startup output. Do not infer versions from what the image was supposed to contain.

A product-triggered renderer crash remains possible. A malformed graphics workload, browser bug, or page-specific condition can crash at low parallelism with ample resource headroom. Reproduce the same route in another supported browser and in Chrome outside the constrained container. That comparison does not automatically assign blame to the product, but it separates a cross-browser application failure from one Chrome build or environment. Preserve the smallest content or action that triggers the crash before simplifying it away.

Synchronization failures also create misleading retries. A timeout waiting for an element is not a renderer crash. Verify that a subsequent harmless command such as reading the title or current URL can still reach the session. If it can, inspect the page state and wait condition. Increasing /dev/shm because a test timed out under load treats correlation as cause and may leave the locator or application race untouched.

Grid transport loss has its own evidence. If the test client cannot reach the Router or Node, the exception may wrap a connection failure without any browser exit. Check network and Grid logs at the same session ID. A responsive browser container cannot complete a command that never reaches it. Conversely, a green Router does not prove the Node hosting the affected session remained reachable.

External termination is another case that can leave the in-container counters inconclusive. A node drain, pod eviction, CI cancellation, runtime restart, or host-level failure can remove the browser container from outside its cgroup. The last resource snapshot may look normal because the container did not choose to exit. Check orchestrator events and the job control plane before calling that absence proof of a Chrome defect. Record whether the container was restarted under the same name, because logs from the replacement instance can otherwise appear to show a healthy process immediately after the original failure.

Worked example two illustrates the distinction. Four sessions fail at nearly the same time. The first theory is shared memory because every test uses Chrome in Docker. Container inspection shows the Selenium process disappeared, the runtime marked the container OOM-killed, and the memory event counter changed. /dev/shm had unused capacity in the last sample. The correct action is to reduce browser concurrency, reduce the workload, or raise the memory limit after capacity planning. Adding --disable-dev-shm-usage would change the storage path without addressing the recorded kill.

Worked example three starts with a healthy container and a single failing tab on a dashboard route. The shared-memory series reaches its configured boundary during chart rendering, while memory and PID events do not change. The same pinned workload passes when only shm_size is increased. That evidence supports a mount-capacity fix. It still does not establish a universal size for every route or future browser version, so keep monitoring after rollout.

Choose the fix that matches container ownership

If your team owns the Docker Compose or docker run definition, give Chrome a measured shared-memory mount. The Compose example below also limits the node to one concurrent session. One session is a diagnostic-friendly starting policy, not a throughput commandment. Increase it only after measuring the combined browser process load. The read-only /opt/qa mount is what makes the collector script from the first section reachable inside the container, which the wrapper in the CI section depends on.

services:
  selenium:
    image: "${SELENIUM_CHROME_IMAGE:?Set a pinned Selenium Chrome image}"
    shm_size: "${SELENIUM_SHM_SIZE:?Set a measured shared-memory size}"
    environment:
      SE_NODE_MAX_SESSIONS: "1"
    ports:
      - "127.0.0.1:4444:4444"
    volumes:
      - "./ops/qa:/opt/qa:ro"

Pin the image in the CI environment rather than using latest. That keeps browser, driver, and server changes out of the capacity experiment. A tag is useful, and an immutable digest is stronger when your registry workflow supports it. Record the effective image identifier in artifacts because an environment variable in a YAML file proves only the request, not what the runtime pulled.

The trade-off of a larger mount is resource commitment and weaker guardrails if the container's total memory remains poorly sized. shm_size sets a ceiling, not a reservation that instantly consumes every byte, but applications can now use more of the node's memory through that mount. Account for the browser container's memory limit and the host's aggregate capacity. Several generously sized nodes on one worker can still create host pressure.

Reducing SE_NODE_MAX_SESSIONS lowers contention and makes session ownership clearer. It costs throughput, more containers, or longer queue time. That may be the right exchange for high-value end-to-end tests, but it can be wasteful for light pages. Measure the worst representative route and assign a concurrency policy per node class if one setting cannot serve both workloads.

One session per container also makes post-failure evidence easier to attribute. The mount, browser process tree, and container log all belong to one WebDriver session. Shared nodes need stronger correlation because a neighboring test can fill the mount while the victim test happens to send the next command. Isolation costs startup time and compute, but it can shorten incident diagnosis enough to justify a dedicated node pool for the heaviest scenarios.

Use --disable-dev-shm-usage when the platform does not let you mount or resize /dev/shm, or as a bounded diagnostic comparison. Selenium's Chrome options accept the argument through the normal browser-options API. The example is straightforward:

import os

from selenium import webdriver


options = webdriver.ChromeOptions()
options.add_argument("--disable-dev-shm-usage")

driver = webdriver.Remote(
    command_executor=os.environ["SELENIUM_REMOTE_URL"],
    options=options,
)
try:
    driver.get(os.environ["TEST_URL"])
finally:
    driver.quit()

Provide the URL and endpoint through controlled CI configuration. The code demonstrates the exact option wiring, not proof that the option fixes a given crash. The cost can include more temporary-filesystem I/O and dependence on that filesystem's capacity and performance. If /tmp is also constrained, read-only, or slow, the fallback can introduce a different failure.

Do not add both a larger mount and the flag in the first diagnostic run. If the failure disappears, you will not know which change mattered. Apply one intervention, retain the same probe, and compare resource evidence. After selecting a production fix, remove experimental flags that are no longer part of the chosen policy.

Teams using Kubernetes or a managed Grid need the same ownership conversation at a different layer. Ask the platform owner what backs /dev/shm, what limit applies, whether tmpfs usage counts against pod memory, and how concurrent sessions are placed. Do not paste a volume example that omits the existing security context, resource limit, or chart values. The Selenium client cannot resize a remote container mount through capabilities.

Roll the change through CI without hiding regressions

Start with one reproducible job and preserve the old job as a comparison. Do not turn on retries while testing capacity. Retries schedule a new session and can convert a deterministic boundary into a passing build with one unexplained failed attempt. If the runner must retry for unrelated policy reasons, store artifacts by attempt and classify the first failure independently.

Add three artifact moments: a baseline snapshot after the container becomes ready, samples during the representative browser step, and a final snapshot before teardown. Keep Docker or orchestrator events, Selenium container logs, and the client traceback. Name artifacts with the CI run, shard, attempt, and session ID. Browser logs without the container ID cannot be matched reliably on a multi-node Grid.

The following shell wrapper shows the wiring around a Compose-managed job. It captures real values even when pytest fails and returns pytest's original status. It reaches the diagnostic script through the /opt/qa mount declared in the Compose service above, so keep the collector at ops/qa/collect-browser-resources.sh in the repository.

#!/usr/bin/env bash
set -u

mkdir -p artifacts
trap 'docker compose down' EXIT
docker compose up -d --wait selenium || exit 1

docker compose exec -T selenium \
  /opt/qa/collect-browser-resources.sh \
  /tmp/resources-before.txt || exit 1
docker compose cp \
  selenium:/tmp/resources-before.txt \
  artifacts/resources-before.txt || exit 1

python -m pytest -q tests/ui/test_renderer_probe.py
test_status=$?

docker compose exec -T selenium \
  /opt/qa/collect-browser-resources.sh \
  /tmp/resources-after.txt || true
docker compose cp \
  selenium:/tmp/resources-after.txt \
  artifacts/resources-after.txt || true
docker compose logs --no-color selenium >artifacts/selenium-container.log

exit "$test_status"

This wrapper does not continuously sample the mount, so add your platform metrics or a bounded background sampler when the failure is brief. It does preserve the original test exit code, a detail often lost when artifact commands run last. Artifact collection is allowed to fail after the test because an already-dead container may not accept exec; the container logs still remain valuable.

Roll the fix to a subset of shards or nodes and compare like with like. Watch renderer-crash counts, peak shared-memory use from actual samples, OOM and PID events, session duration, and queue time. These are measurements your system produces. Do not publish an improvement percentage from a handful of non-equivalent runs or turn the Selenium example's 2 GB value into a performance claim.

Keep a capacity regression probe small enough to run on image or infrastructure changes. Its purpose is to catch a large shift in resource behavior, not to certify that every product page will fit forever. Run real workload coverage as well. Chrome versions, site features, fonts, extensions, video recording, and concurrency can all change the boundary.

When not to increase /dev/shm

Leave the mount alone when the resource evidence rejects it. An OOM-killed container needs memory and concurrency work. A reached PID counter needs task capacity or fewer browser processes. A version mismatch needs a compatible image. A locator timeout with a live session needs synchronization or product analysis. A larger mount makes each of those investigations noisier without addressing the recorded failure.

Do not use capacity to conceal an unbounded test. A suite that opens tabs without closing them, leaks sessions after failures, or leaves video and downloads running will eventually consume any finite allocation. Count window handles, ensure quit() runs in teardown, and verify the Grid releases the session. Raising the ceiling can delay the crash until a later shard, where ownership is harder to trace.

Avoid the Chrome fallback flag when the test is specifically measuring production-like browser storage or performance. Moving work away from /dev/shm changes the environment. It may be acceptable for functional coverage on a constrained CI platform, but it is a poor basis for a performance conclusion or a browser-resource regression test.

Do not copy a Docker fix into arena tests that run outside containers. Host Chrome may have a different shared-memory setup, and remote cloud providers hide container ownership entirely. Ask the provider for session logs and supported configuration instead of sending a local filesystem flag on speculation.

Finally, do not classify every crash as infrastructure. If the same pinned page crashes Chrome with ample resource headroom, one session, and no cgroup events, preserve the page-level reproduction. Test another browser and Chrome channel, then report the evidence to the appropriate product or browser owner. Infrastructure should be a conclusion earned from its own records, not a label applied because the failure happened in CI.