Test Selenium Grid External Session Map Failover

Selenium Grid external session map failover testing should create a live session, prove its command path, interrupt only the Session Map service, restore a compatible service against the same external datastore, and retry with the original session ID. Persistence preserves mapping data; it does not itself provide automatic failover or browser migration.

That last boundary is the center of the test design. A Session Map stores the relationship between a WebDriver session ID and the Node URI that owns it. It does not contain a browser checkpoint, duplicate the Node process, control DNS, promote a database replica, or make a Router retry forever. A successful datastore write is necessary for some recovery designs, but it is not a complete availability claim.

Selenium's official external datastore guide documents JDBC-backed and Redis-backed Session Map implementations. The guide shows how to persist currently running session information outside the local process. This article turns that capability into a controlled continuity experiment and keeps unproven infrastructure behavior outside the pass statement.

Define the Exact Failover Claim

Write the claim in one sentence before building the environment: "If the Session Map process is replaced while its external datastore and the session-owning Node remain available, the recovered service can resolve the existing session ID and the Router can deliver a read-only command to the same Node within the recovery objective."

Every noun in that sentence matters. The process is replaced, not magically healed. The datastore remains available, so this case does not also test database failover. The Node remains alive, so the browser can execute the command. The Router remains stable, so service discovery and client connection changes do not confuse the result. A read-only command avoids changing application state while retries occur.

Create separate claims for other failures:

• Session Map process restart: persisted mapping should be readable after the service returns.
• Datastore primary failover: behavior depends on the database or Redis design, driver settings, DNS, timeouts, and the Session Map implementation. Test it as an infrastructure integration.
• Router replacement: the replacement Router should use the restored Session Map service address, but that is a different component lifecycle.
• Node loss: the original browser session is unrecoverable even if the mapping survives.
• Regional outage: traffic policy, datastore placement, Event Bus, queue, Distributor, and Nodes all enter the claim; external mapping alone is insufficient.

This decomposition prevents a narrow green test from becoming an unsupported "Grid is highly available" statement. Use the multi-region Grid architecture guide when the intended boundary crosses network or regional fault domains.

The first observable assertion is an ordinary command through the Router using the new session ID. The first failure boundary is then the Router-to-Session-Map lookup, followed by Session-Map-to-datastore access and Router-to-Node forwarding. Record evidence at each boundary so recovery time is not inferred from one client timeout.

Define retry ownership in the claim. Selenium clients, reverse proxies, service meshes, and test harnesses can each retry, but a hidden retry can make the outage look shorter and duplicate a state-changing command. For the continuity probe, send a read-only command, record every attempt, and let only the harness retry after the replacement service is declared available. Production retry policy should be evaluated independently.

Also define the negative guarantee: during recovery, the original session ID must never be associated with another Node or a newly created browser. A failed lookup is preferable to misrouting. Include owner identity before and after the interruption so matching page content cannot conceal a wrong destination.

Map Session Ownership and Persistence Boundaries

The official Grid components guide explains the routing chain. The Router sends a new request to the queue and Distributor path. After creation, it asks the Session Map for the Node associated with a session ID, then forwards commands to that Node. The Node owns the browser and executes the commands.

The external store changes where the mapping survives. It does not merge the responsibilities of those components. Use this decision table to scope expectations:

Keep Session Map and New Session Queue evidence separate. The queue stores waiting new-session requests, not ownership for established sessions. A queue that is empty after recovery does not prove an existing ID routes. The guide to inspecting Session Maps and queues is useful for maintaining that distinction during incidents.

Persistence also has timing. A client can receive a new-session response only after several distributed actions, but your test should still verify that the mapping is externally visible before injecting failure. If interruption happens during creation, the experiment becomes a write-acknowledgment test and needs its own expected outcomes. Begin with a settled session, then add creation-race cases later.

Build a Reproducible External Session Map Topology

Use a disposable distributed Grid rather than Standalone when the purpose is to interrupt the Session Map independently. Run Event Bus, New Session Queue, Session Map, Distributor, Router, and at least one Node as identifiable workloads. Pin all Selenium component versions and extension artifacts. Use stable internal service addresses, but keep workload instance identity in logs.

The Redis configuration documented by Selenium is compact. Place the password and transport controls in the platform's secret and network configuration rather than publishing them in article examples or command history.

Code 1: Redis-backed Session Map configuration

[sessions]
scheme = "redis"
implementation = "org.openqa.selenium.grid.sessionmap.redis.RedisBackedSessionMap"
hostname = "redis.selenium-grid.svc.cluster.local"
port = 6379

Code 2: Start the Session Map with the matching extension

SE_VERSION=4.46.0
PUBLISH_EVENTS=tcp://event-bus.selenium-grid.svc.cluster.local:4442
SUBSCRIBE_EVENTS=tcp://event-bus.selenium-grid.svc.cluster.local:4443

java -jar "selenium-server-${SE_VERSION}.jar" \
  --ext "$(coursier fetch -p \
    org.seleniumhq.selenium:selenium-session-map-redis:${SE_VERSION})" \
  sessions \
  --publish-events "$PUBLISH_EVENTS" \
  --subscribe-events "$SUBSCRIBE_EVENTS" \
  --port 5556 \
  --config sessions.toml

Use the exact extension version matching selenium-server. Record the resolved class path rather than accepting an unreviewed floating artifact. For JDBC, initialize the documented table and configure JdbcBackedSessionMap with a restricted database user. Schema migration, connection pooling, TLS, credential rotation, and database backup remain platform responsibilities and belong in separate tests.

Do not expose Redis, the database, or the Session Map port to untrusted clients. The Router is the client entry point. The official architecture reference cautions that only the Router is a candidate for wider exposure, and even that exposure should be controlled.

Before fault injection, save the process IDs or workload UIDs, service endpoints, Grid version, extension hash, datastore identity, and Node inventory. A restarted container with the same friendly name is still a new process. Those identifiers make the failover chronology reviewable.

Establish a Settled Session Baseline

Create one canary through the Router with a dedicated se:name and a browser capability supported by exactly one test Node if possible. Record the session ID from the W3C new-session response. Query GraphQL for the session's nodeId and nodeUri, then execute a harmless command such as get title. That establishes client routing, mapping lookup, Node ownership, browser execution, and return routing before disruption.

Avoid using only the Grid UI. It can show a session while the client command path has a different failure. Avoid using only datastore inspection because backend key layout is an implementation detail and a present record can still be unreadable by the live service. The supported Router command is the acceptance signal; GraphQL, logs, and datastore health explain it.

Settle the baseline for several seconds or until a defined observation confirms the session. The test must not interrupt the Session Map before its write completes and then claim persisted data vanished. Capture the successful create response timestamp, GraphQL observation timestamp, and last successful command timestamp.

Use a canary page controlled by the test team or a data URL supported by the browser policy. A public website adds DNS, Internet, certificate, and content dependencies to a control-plane experiment. Keep the browser idle between commands so application behavior cannot terminate the session.

Check timeouts before running. The WebDriver client command timeout should exceed the expected Session Map recovery window only if the Router or proxy can keep or retry that request safely. For clearer evidence, let a command during the outage fail quickly, then issue a new read-only command after recovery. Do not retry clicks, form submissions, or script mutations without an idempotency contract.

The GraphQL capacity dashboard guide documents the session and Node fields useful for this baseline. Save a narrow sanitized query response rather than every capability value.

Run the Controlled Failover Workflow

Execute the experiment in this numbered order. Automate the control actions, but require unique environment identity so a script cannot stop a shared production Session Map accidentally.

1. Confirm Router, Session Map, external datastore, Distributor, queue, and Node health from independent signals.
2. Create the canary session and record its session ID, owning Node, browser process evidence, and successful title command.
3. Confirm the mapping is externally persisted through supported service evidence or a sanitized backend observation owned by the platform team.
4. Stop only the Session Map workload while keeping its service address, datastore, Router, Node, and browser unchanged.
5. Send one read-only command during the outage and record its bounded failure without deleting the session or restarting the Node.
6. Start a compatible Session Map instance using the same datastore and service address.
7. Wait for the Session Map service itself to accept requests, then repeat the read-only command with the original session ID.
8. Query GraphQL and Node ownership evidence to prove the command still reached the original Node.
9. Quit the original session through the Router and confirm the external mapping is removed through normal lifecycle behavior.
10. Restore all monitoring and assert no extra canary sessions, queued requests, or stale test workloads remain.

The stop and start mechanism belongs to the orchestrator: a Kubernetes rollout, systemd operation, container control, or process supervisor. Keep it behind a dedicated test interface with an environment allowlist. Do not embed cloud credentials in the client script.

Measure interruption start, first failed lookup, replacement start, service availability, first recovered lookup, and cleanup completion. Client-visible recovery time is the interval from disruption to the first successful original-session command. Component restart time alone is not the service objective.

If the command never fails during a very short restart, do not call that a failed test. The Router, connection pool, or timing may have avoided the interruption window. Hold the service unavailable long enough to observe one bounded lookup failure, while staying below the Node session timeout and browser idle policy.

Assert the Original Session Through the Router

The following shell harness uses raw W3C WebDriver requests so the original ID remains visible. grid-control is an intentionally abstract, environment-owned command that must stop and start only the test Session Map. The browser session is deleted in a trap when the Router is reachable.

Code 3: Existing-session continuity probe

#!/usr/bin/env bash
set -euo pipefail

: "${GRID_URL:=http://localhost:4444}"
: "${GRID_CONTROL:?Set the isolated environment control command}"

create_response="$(curl --fail --silent --show-error \
  -H 'Content-Type: application/json' \
  -d '{"capabilities":{"alwaysMatch":{"browserName":"chrome","se:name":"session-map-failover-canary"}}}' \
  "${GRID_URL%/}/session")"

session_id="$(jq -er '.value.sessionId' <<<"$create_response")"
cleanup() {
  curl --silent --show-error -X DELETE \
    "${GRID_URL%/}/session/${session_id}" >/dev/null || true
}
trap cleanup EXIT

before_title="$(curl --fail --silent --show-error \
  "${GRID_URL%/}/session/${session_id}/title" | jq -er '.value')"

"$GRID_CONTROL" stop-session-map
if curl --fail --silent --show-error --max-time 5 \
  "${GRID_URL%/}/session/${session_id}/title" >/dev/null; then
  echo "Expected a bounded lookup failure while Session Map was stopped" >&2
  exit 1
fi

"$GRID_CONTROL" start-session-map

recovered=false
for attempt in $(seq 1 30); do
  if after_response="$(curl --fail --silent --show-error --max-time 5 \
    "${GRID_URL%/}/session/${session_id}/title" 2>/dev/null)"; then
    after_title="$(jq -er '.value' <<<"$after_response")"
    recovered=true
    break
  fi
  sleep 2
done

test "$recovered" = true
test "$before_title" = "$after_title"
printf 'recovered session=%s title=%q\n' "$session_id" "$after_title"

Comparing titles is a useful no-mutation check, but equality alone does not prove the same Node served both commands. Pair the script with the session's GraphQL nodeId, direct Node owner endpoint where authorized, or correlated traces. The official Grid endpoints reference documents the Node owner route and warns that internal control endpoints use the registration secret.

Do not print that secret or pass it to the general CI job. A privileged platform-side collector can attach a Boolean ownership result keyed by session ID. The test client needs only Router access.

Test Failure Modes One Boundary at a Time

After the basic restart passes, expand coverage without combining every dependency. Each case should declare held-constant components, injection, expected client state, recovery condition, and cleanup owner.

When testing Node loss, drain and observe ordinary lifecycle first, then run a separate abrupt-kill case. Draining preserves current sessions until completion and is not equivalent to failure. The Node draining upgrade article defines that control path.

For stale records, avoid depending on raw Redis key names or SQL statements in the acceptance test. Those details can change between Selenium versions. Use a disposable backend fixture and a platform-owned setup adapter. The durable assertion is that an unknown or dead owner never causes a command to reach a different live session.

Datastore behavior needs its own evidence. Record connection timeout, command timeout, pool limits, replica promotion time, DNS cache behavior, and the consistency guarantee seen by the Session Map client. A backend can report healthy while a replacement Session Map reads from a lagging replica. Require the original mapping to be visible through the supported service path before declaring recovery, and keep the database platform's health result as supporting evidence rather than a substitute.

Exercise recovery after a longer interruption that remains below the browser and Node session limits. A quick restart tests process replacement; a longer hold reveals idle connection expiry, secret refresh, DNS changes, and browser session timeout interactions. State the maximum supported interruption from measured results and configuration. Do not promise indefinite continuity merely because one thirty-second restart passed.

Test new-session behavior separately. A running Session Map outage can prevent successful ownership recording even when queue and Distributor are healthy. Record queue size, request timeout, and whether a browser process was started before failure. The queue timeout debugging workflow helps identify abandoned creation attempts.

For Kubernetes replacement tests, preserve service identity while replacing only the Session Map Pod, then test service endpoint and DNS recovery independently. The disposable Node Kubernetes guide is about browser worker lifecycle, so do not assume its replacement behavior applies automatically to state-routing components.

Add CI Evidence, Alerts, and Security Controls

Run a fast configuration test on every Grid image change: extension artifact exists, versions match, TOML parses, service endpoints resolve, and no secret appears in rendered configuration. Run the full browser continuity experiment in an isolated staging environment on Session Map, Router, datastore-client, or orchestration changes. Schedule a broader recovery drill at an interval tied to operational risk.

Gate on the original session command recovering within the stated objective, proof of the same owner Node, normal session deletion, and absence of leftover workloads. Report datastore failover failures separately from Session Map process failures so ownership is clear, but do not discard either from release readiness when the changed component owns that path.

Correlate every event with a test run ID and session ID. The Grid trace-correlation guide can connect Router lookup, Session Map access, and Node command spans. Use monotonic durations for thresholds and wall-clock timestamps for cross-system chronology. Keep time synchronization health in the artifact.

Useful alerts include sustained Session Map request failures, datastore connection exhaustion, lookup latency above the client budget, missing mappings for active Nodes, repeated stale-owner errors, and cleanup lag after normal quits. Alert from rates and duration rather than one restart drill's expected error. Tag planned injections so they exercise alert routing without paging an uninformed responder.

Restrict datastore credentials to the Session Map workload, rotate them through the platform secret system, encrypt network traffic where supported, and limit backend inspection to a privileged diagnostic role. Session IDs are bearer-like routing identifiers and should be redacted or access-controlled in broadly visible logs. Capabilities can contain test and environment metadata; store only the fields needed for diagnosis.

Before a Selenium upgrade, repeat the entire workflow with old and new components in separate pools. The Selenium 3 to 4 Grid migration guide is a reminder that topology and capability semantics can change across major transitions. Never assume an external record written by one release is compatible with an arbitrary replacement release unless the project documents and your test proves that path.

Frequently Asked Questions

Does an external Session Map make Selenium Grid fail over automatically?

No. The external datastore can preserve active session-to-Node mappings outside one Session Map process, but Selenium does not promise automatic process, network, datastore, or Router failover from persistence alone. Operators must design the replacement path, test it, and define what clients observe during interruption and recovery.

Can a session continue after its Selenium Node is lost?

No. The mapping only tells the Router which Node owns a session. The Node and its browser process still execute every command. If that owner is terminated or its browser disappears, restarting the Session Map cannot recreate or migrate the live WebDriver session to another Node.

What is the strongest Session Map failover assertion?

Create a session, record its ID and owning Node, execute a harmless command, interrupt only the Session Map service, restore a compatible instance against the same datastore, and repeat the command with the original ID. Success proves routing continuity for that controlled path, not every Grid dependency.

Should failover tests create new sessions during the outage?

Use a separate test case. Existing-session routing and new-session creation exercise different paths and produce different evidence. First hold Node, browser, Router, Distributor, queue, and datastore stable while testing mapping continuity. Then test new-session behavior during and after the outage with its own timeout and cleanup policy.

How should stale external Session Map records be tested?

Create stale state only in an isolated datastore, never by editing a shared production backend. Verify that commands do not silently reach another Node, capture the returned error and routing evidence, then confirm supported session closure or reconciliation removes the record. Treat backend keys and schemas as implementation details.

Which artifacts are needed for a Session Map failover incident?

Keep the session ID, owning Node ID and URI, Grid and extension versions, sanitized datastore configuration, interruption timeline, original and recovered command results, Router and Session Map logs, GraphQL snapshots, and correlated traces. Record datastore health separately, and exclude passwords, connection secrets, and sensitive capabilities.

Practice a Recovery Claim You Can Defend

Build the smallest experiment first: one Router, one Session Map process, one external store, one Node, and one browser session. Prove the baseline command, stop only the map process, restore it against the same data, prove the original command on the original owner, and quit normally. Keep every other component stable until this claim passes repeatedly.

Use QABattle's automation battles to practice separating a durable state assertion from infrastructure activity. Continue with Session Map and queue inspection, then add trace correlation when the boundary is visible. The design is complete when a green result states exactly what recovered, a red result identifies the first failed owner, and no one mistakes persistent mapping data for an automatically failed-over browser session.