Why Selenium Grid leaves a healthy node idle

A browser Node is registered, marked UP, and sitting at zero sessions, yet the next test waits in Selenium Grid's session queue. Adding another identical Node changes nothing because capacity is not the problem. The request and the advertised slot disagree on a capability, so the Distributor never considers them a match.

Why an idle node can still reject your request

Grid does not treat a healthy Node as compatible based on status alone. A Node registers one or more slots with the Distributor, and every slot has a stereotype. Selenium defines that stereotype as the minimum set of capabilities a new session request must match before the Distributor sends it to the Node.

The request follows a specific path. The Router accepts the new-session command and places it in the New Session Queue. The Distributor keeps its own model of registered Nodes and slots. It looks for a free slot whose stereotype is compatible with one of the request's capability candidates. Only then does it reserve the slot and ask the Node to start a browser. The Grid architecture documentation and component description are useful here because they separate matching from browser startup.

That distinction explains the misleading symptom. UP means the Node registered and responds to health checks. A zero session count means it is not busy. Neither fact proves that its slots match the pending request.

The current DefaultSlotMatcher API contract is precise about what it evaluates. Non-extension capabilities in the stereotype must match the candidate. If the request contains browserName, browserVersion, or platformName, the stereotype must contain the same values. A hidden Firefox request sent to a Chrome-only pool is enough to strand the session. An exact browser version or platform can do the same.

Extension capabilities need more care. The default matcher documentation says it does not consider namespaced extension keys because their matching is driver-specific. A difference between gsg:region=west and gsg:region=east therefore does not prove a slot mismatch under the default implementation. If your Grid routes on custom data, confirm which matcher is configured instead of assuming the stereotype key controls scheduling.

Do not compare only the test source with the Node's TOML file. Client bindings build the actual W3C capabilities payload, and deployment tooling can rewrite or replace Node configuration. The two useful facts are the request that reached the queue and the stereotype in the Distributor's live Grid model.

Reproduce the mismatch with two browser requests

This small setup makes the problem visible without a large CI system. It needs Java, a Selenium Server JAR named selenium-server.jar, Chrome, ChromeDriver on PATH, and the Selenium Python package. Firefox does not need to be installed because the Firefox request will never find a compatible slot.

Save this as grid.toml:

[server]
port = 4444

[sessionqueue]
session-request-timeout = 20

[node]
detect-drivers = false

[[node.driver-configuration]]
display-name = "Chrome only"
max-sessions = 1
stereotype = '{"browserName":"chrome"}'

Start a Standalone Grid. Standalone still contains the Router, queue, Distributor, and Node roles, which makes it suitable for this controlled reproduction.

java -jar selenium-server.jar standalone --config grid.toml --log-level FINE

Save the client as request_session.py:

import sys

from selenium import webdriver


browser = sys.argv[1] if len(sys.argv) > 1 else "chrome"

if browser == "chrome":
    options = webdriver.ChromeOptions()
elif browser == "firefox":
    options = webdriver.FirefoxOptions()
else:
    raise SystemExit("Choose chrome or firefox")

driver = webdriver.Remote(
    command_executor="http://localhost:4444",
    options=options,
)

try:
    print(f"session={driver.session_id} browser={browser}")
    driver.get("https://www.selenium.dev/")
    print(driver.title)
finally:
    driver.quit()

Run the matching case first:

python request_session.py chrome

The Chrome request should acquire the configured slot. Now run python request_session.py firefox. The Node remains idle because its only stereotype says browserName=chrome. The client becomes eligible for rejection after 20 seconds, although the queue's periodic timeout check can add a short delay.

This example deliberately changes one value. In a real suite, the difference often comes from a default in a driver factory, an outdated version pin, or a stale ConfigMap mounted into one Node pool. Keep the reproduction equally narrow. Changing browser version, platform, and browser name together destroys the comparison.

Prove matching failed before changing capacity

Capture the live Grid state while the Firefox request is waiting. Selenium exposes the queue and Node stereotypes through its documented GraphQL schema:

curl --fail --silent \
  --header 'Content-Type: application/json' \
  --data '{"query":"{ grid { sessionQueueSize } sessionsInfo { sessionQueueRequests } nodesInfo { nodes { id uri status maxSession slotCount sessionCount stereotypes } } }"}' \
  http://localhost:4444/graphql

The decisive combination is specific:

1. sessionQueueSize is greater than zero, and sessionQueueRequests contains the capability request you are investigating.
2. The relevant Node appears in nodesInfo, its status is UP, and its sessionCount is below its capacity.
3. None of that Node's stereotypes is compatible with the queued candidate. In the example, the queue says Firefox while the slot says Chrome.

Those three observations make a capacity explanation unlikely. They also distinguish the mismatch from several nearby failures.

If the Node is absent from nodesInfo, investigate registration or Event Bus connectivity. If its status is DRAINING or unavailable, scheduling is correctly avoiding it. If every matching slot already has a session, the queue is waiting for capacity. If a slot is selected and ChromeDriver then returns an error, matching succeeded and browser startup failed. Treating all four cases as “Grid could not create a session” wastes time.

The Distributor's model can briefly differ from the Node itself. In Hub-and-Node mode, query the Node's documented status endpoint as a countercheck:

curl --fail --silent http://localhost:5555/status

When the Node reports the expected slot but GraphQL does not, the problem is registration state or a lagging Grid model, not the stereotype in the Node process. When both endpoints advertise the same wrong value, fix the Node's effective configuration.

FINE logging adds request traces to the console. Follow the trace for the POST /session request and confirm that no session ID or slot assignment appears before the timeout. The observability documentation explains the trace ID, span ID, event attributes, and exception fields. Logs support the diagnosis, but the side-by-side queue request and live stereotypes are the stronger evidence.

Fix the contract on the side that is wrong

First decide which browser the test is meant to cover. If it should run on Chrome, then the Firefox request is a client bug. Correct the central driver factory or CI variable that selects the Options class. Do not patch individual tests, because the next caller will recreate the mismatch.

If a Firefox Node was deployed but registered as Chrome, correct its driver configuration and restart that Node so it registers the new stereotype. Confirm the result through GraphQL before returning the pool to service. Editing a ConfigMap or TOML file is not proof that a running process consumed it.

If an exact version or platform does not change the test's meaning, remove that constraint from both the stereotype and the request. Fewer matching dimensions make the Grid easier to operate. The trade-off is less control over the environment, so preserve exact constraints for compatibility checks that truly need them.

Static Grids can also fail faster by setting --reject-unsupported-caps true on the process that owns the Distributor role. Selenium documents this option specifically for environments that do not create Nodes on demand. The benefit is a clear early failure instead of a long queue wait. The cost is that a temporarily unavailable stereotype is rejected rather than given time to register.

Write supported browser, version, and platform values as a small, reviewed vocabulary. Reject misspellings and stale aliases at configuration boundaries. This validation adds deployment work, but it turns a slow scheduling timeout into an immediate configuration error.

The last resort is a custom slot matcher, configured with Selenium's documented --slot-matcher option. It is justified when compatibility depends on namespaced extension capabilities or another rule the default matcher does not evaluate. A matcher becomes Grid-side Java code that must be deployed with the server and checked on every Selenium upgrade.

Keep the same mismatch out of CI

Run one session-allocation smoke test after each Node pool registers. The request should include every routing capability that production tests use, then assert the returned session exists and quit it cleanly. A browser health check that bypasses the Grid does not test the scheduling contract.

Store three artifacts for a failed allocation: the effective client capabilities, the GraphQL queue snapshot, and nodesInfo from the same minute. Redact tokens and proxy credentials before upload. A screenshot of the Grid UI is helpful for humans, but it is harder to compare and often omits the queued payload.

Add a negative check on static infrastructure. Request a browser that no pool advertises and verify that the Grid rejects it within the expected request timeout. This catches an accidental catch-all stereotype. It also proves that the queue policy behaves as the team assumes.

Version exactness deserves a deliberate policy. Request an exact browserVersion only when the test needs it. Otherwise, an old version pinned in a shared options builder can strand work after the browser fleet upgrades. Removing the pin improves availability but gives up exact-version reproducibility, so record the actual returned capabilities with the test result.

Watch session queue time separately from test duration. A suite can report “test timeout” even though no browser ever started. An alert that combines rising queue time, idle matching capacity, and unsupported capability values points to the scheduling boundary much sooner.

When a stereotype change is the wrong move

Do not relax a stereotype merely to empty the queue. Sending a Firefox test to Chrome changes the coverage contract, even when the application happens to behave the same in both. A quick green build can hide a browser-specific defect.

Leave the stereotype alone when the Node is full, draining, unavailable, or missing from the Distributor's model. Those conditions need capacity, lifecycle, health, or registration work. Changing advertised capabilities in response creates a second defect and erases the original evidence.

Avoid adding transient facts such as build numbers, test IDs, or worker names to slot stereotypes. They create high-cardinality contracts that Nodes cannot advertise ahead of each request. Use capabilities for stable properties that determine whether a Node can run a session.

Do not label a namespaced extension-capability difference as the cause while the default matcher is active. Confirm the configured matcher first. If it is DefaultSlotMatcher, continue comparing standard capabilities and Node state rather than forcing custom keys to agree for no scheduling benefit.

Do not add an exact platform or browser version “for clarity” when the test accepts a broader pool. Every extra standard matching constraint is another way for a valid request to stop matching. Precision is useful only when the property changes the test's meaning.

Finally, do not blame matching after the Distributor has reserved a slot. A Chrome binary crash, bad driver path, exhausted shared memory, or Node-side session creation error happens later in the flow. The evidence will show an attempted assignment rather than an idle compatible slot. Fix the browser startup boundary and keep the honest stereotype.