One task, one driver: safe Selenium in Java thread pools

Eight browser tasks finish, Maven reports one failure, and the JVM never exits. One worker threw before quit(), while another task was still using the driver field they shared. Parallelism did not create the defect; it exposed that nobody owned the session lifecycle.

An ExecutorService is useful for bounded concurrency, but a pool does not define WebDriver ownership for you. The clean boundary is one submitted task, one driver, one thread of commands, and one cleanup path. Anything wider needs an explicit reason and much stronger coordination.

Why the task must own the whole browser session

A WebDriver session is a stateful conversation with a remote end. Navigation changes the current document. Window and frame selection change where later commands apply. Cookies, timeouts, alerts, and open windows also belong to that session. Two tasks issuing commands against one driver are not independent consumers of a stateless client.

Java's fixed thread pool adds a second kind of state. Worker threads live longer than individual tasks and are reused. A field on the test class may be visible to every worker. A ThreadLocal<WebDriver> associates a value with a worker thread, not with the logical test that happens to run there. If a task forgets to remove the value, the next task on that worker can inherit a driver that is already authenticated, on the wrong page, or already quit.

The safest starting model aligns three boundaries. The task creates the driver after it begins running. Every WebDriver command happens inside that task. A finally block quits the same session before the task completes. The caller owns the pool and every Future, so it can stop submissions, observe task failures, and decide what to do if termination exceeds its bound.

Selenium's Java-only ThreadGuard helps enforce one part of that model. It records the thread that created the protected driver and rejects calls from a different thread. Selenium's own documentation is explicit that ThreadGuard does not replace ThreadLocal when a framework chooses per-thread storage. More importantly, neither mechanism decides when a session should be created or quit. Detection and storage are not lifecycle policy.

A shared driver protected by a lock is rarely an acceptable substitute. Serializing commands prevents simultaneous method calls, but it does not give tasks independent browser state. Task A can navigate to checkout, release the lock, and Task B can navigate to login before Task A's next assertion. Holding the lock for the entire test effectively reduces the browser work to one task at a time while preserving the complexity of a shared object.

Creating all drivers on the main thread and handing one to each worker also conflicts with the ownership model. Even if no two tasks share the same instance, the creator thread differs from the command thread. Without ThreadGuard, that misuse may produce intermittent behavior. With ThreadGuard, it fails early and identifies both sides of the crossing. Construct the driver inside Callable.call() or Runnable.run() instead.

Session cleanup must cover assertions and Selenium exceptions. Putting quit() after the last assertion handles only the success path. A missing element, failed assertion, interrupted wait, or navigation error skips it. Leftover local driver processes consume memory and ports. Leftover remote sessions consume Grid slots until the remote end reclaims them. The specific reclaim policy varies by endpoint, so client code should not depend on an assumed timeout.

Put construction, commands, and quit in one Callable

The task below receives plain immutable input. It creates a fresh ChromeOptions, creates the driver on the worker, wraps it with ThreadGuard, exercises a product-visible assertion, and quits in finally. It records the session identifier before cleanup so later logs can correlate task events without attempting a command on a closed session.

The assertion can fail if the application changes: it reads the page's heading and compares it with the expected text. This is not a self-confirming check over a hard-coded fixture. The base URI and expected heading are job inputs, so the same task can exercise independent environments or cases.

package example.parallel;

import java.net.URI;
import java.net.URL;
import java.util.Objects;
import java.util.concurrent.Callable;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ThreadGuard;

public final class BrowserTask implements Callable<BrowserTask.Result> {
  public record Job(String id, URI baseUri, URL gridUrl, String expectedHeading) {
    public Job {
      Objects.requireNonNull(id, "id");
      Objects.requireNonNull(baseUri, "baseUri");
      Objects.requireNonNull(gridUrl, "gridUrl");
      Objects.requireNonNull(expectedHeading, "expectedHeading");
    }
  }

  public record Result(String jobId, String sessionId, String heading) {}

  private final Job job;

  public BrowserTask(Job job) {
    this.job = Objects.requireNonNull(job, "job");
  }

  @Override
  public Result call() {
    ChromeOptions options = new ChromeOptions();
    options.addArguments("--headless=new", "--window-size=1440,900");

    RemoteWebDriver rawDriver = new RemoteWebDriver(job.gridUrl(), options);
    WebDriver driver = ThreadGuard.protect(rawDriver);
    String sessionId = String.valueOf(rawDriver.getSessionId());
    Throwable testFailure = null;

    try {
      driver.get(job.baseUri().resolve("/status").toString());
      String heading = driver.findElement(By.cssSelector("main h1")).getText();
      if (!job.expectedHeading().equals(heading)) {
        throw new AssertionError(
            "Expected heading %s but found %s"
                .formatted(job.expectedHeading(), heading));
      }
      return new Result(job.id(), sessionId, heading);
    } catch (RuntimeException | Error failure) {
      testFailure = failure;
      throw failure;
    } finally {
      try {
        driver.quit();
      } catch (RuntimeException quitFailure) {
        if (testFailure != null) {
          testFailure.addSuppressed(quitFailure);
        } else {
          throw quitFailure;
        }
      }
    }
  }
}

The cleanup branch preserves the original failure. A plain finally { driver.quit(); } is better than no cleanup, but a quit() exception can replace the assertion or command exception already in flight. Adding the cleanup error as suppressed evidence keeps the primary product failure visible while retaining proof that session teardown also failed. When only quit() fails, that failure still escapes the task.

Driver construction sits before the try because there is no usable session to quit if the constructor throws. If your driver factory allocates another resource before constructing RemoteWebDriver, give that resource its own cleanup boundary. Do not assign a partly initialized driver to a shared field so another method can “clean it up later.” That creates exactly the ownership ambiguity this design removes.

This task uses RemoteWebDriver because the CI wiring below supplies a Grid URL. A local-only suite can construct ChromeDriver in the same location instead. The ownership rule does not change: the constructor must run inside call(), and the same task must issue commands and quit the session.

ThreadGuard is optional in a design that already confines the driver, but it is a useful tripwire during a migration. If a helper schedules asynchronous Java work and captures the driver, the next WebDriver command runs on a different thread and ThreadGuard rejects it near the misuse. The wrapper cannot detect logical interference when two tasks somehow use the driver on the same thread at different times, which is another reason not to store sessions in a reused worker's ThreadLocal by default.

The task returns facts rather than printing a pass message. The coordinator can attach the job ID, session ID, and observed heading to a test report. A returned result also makes it clear when a task never reached its assertion. If session construction fails, there is no Result; the Future contains the failure.

Make the coordinator observe every failure

Calling submit() transfers an exception into the returned Future. It does not automatically throw that exception on the submitting thread. A loop that submits work, calls shutdown(), and prints “done” has no credible test outcome if it never calls get() on those futures.

The coordinator below closes submission immediately after scheduling the fixed set of jobs. It reads every future against one batch deadline, preserving task failures rather than stopping at the first one. It then waits for worker termination with a separate bound. If the pool does not terminate, shutdownNow() requests interruption and returns tasks that never began. Interruption is cooperative, so the method does not claim that every running browser stopped instantly.

package example.parallel;

import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CancellationException;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public final class BrowserTaskRunner {
  private BrowserTaskRunner() {}

  public static List<BrowserTask.Result> run(
      List<BrowserTask.Job> jobs,
      int workers,
      Duration batchTimeout,
      Duration terminationTimeout) {
    if (workers < 1) throw new IllegalArgumentException("workers must be positive");
    if (batchTimeout.isNegative() || batchTimeout.isZero()) {
      throw new IllegalArgumentException("batchTimeout must be positive");
    }
    if (terminationTimeout.isNegative() || terminationTimeout.isZero()) {
      throw new IllegalArgumentException("terminationTimeout must be positive");
    }

    ExecutorService pool = Executors.newFixedThreadPool(workers);
    List<Future<BrowserTask.Result>> futures = new ArrayList<>();
    for (BrowserTask.Job job : jobs) {
      futures.add(pool.submit(new BrowserTask(job)));
    }
    pool.shutdown();

    List<BrowserTask.Result> results = new ArrayList<>();
    List<Throwable> failures = new ArrayList<>();
    long deadline = System.nanoTime() + batchTimeout.toNanos();

    for (int index = 0; index < futures.size(); index++) {
      Future<BrowserTask.Result> future = futures.get(index);
      String jobId = jobs.get(index).id();
      long remaining = deadline - System.nanoTime();
      try {
        if (future.isDone()) {
          results.add(future.get());
        } else if (remaining <= 0) {
          future.cancel(true);
          failures.add(new TimeoutException(
              "Browser task exceeded the batch deadline: " + jobId));
        } else {
          results.add(future.get(remaining, TimeUnit.NANOSECONDS));
        }
      } catch (CancellationException cancelled) {
        failures.add(cancelled);
      } catch (ExecutionException failure) {
        failures.add(failure.getCause());
      } catch (TimeoutException timeout) {
        future.cancel(true);
        TimeoutException labelled = new TimeoutException(
            "Browser task exceeded the batch deadline: " + jobId);
        labelled.initCause(timeout);
        failures.add(labelled);
      } catch (InterruptedException interrupted) {
        futures.forEach(item -> item.cancel(true));
        Thread.currentThread().interrupt();
        failures.add(interrupted);
        break;
      }
    }

    try {
      if (!pool.awaitTermination(terminationTimeout.toMillis(), TimeUnit.MILLISECONDS)) {
        List<Runnable> neverStarted = pool.shutdownNow();
        failures.add(new IllegalStateException(
            "Executor did not terminate; tasks never started: " + neverStarted.size()));
      }
    } catch (InterruptedException interrupted) {
      pool.shutdownNow();
      Thread.currentThread().interrupt();
      failures.add(interrupted);
    }

    if (!failures.isEmpty()) {
      AssertionError combined = new AssertionError(
          failures.size() + " parallel browser failure(s)");
      failures.forEach(combined::addSuppressed);
      throw combined;
    }

    return List.copyOf(results);
  }
}

There is a trade-off in waiting for futures in submission order. A later future may finish first, but the coordinator will not inspect it until earlier entries complete. That does not prevent the tasks from running concurrently. It can delay reporting of an early failure from a later job. An ExecutorCompletionService reports completions in finish order and is useful for large batches, but it adds another moving part. Choose it when reporting latency matters, not because it makes WebDriver safer.

The batch timeout is not a Selenium command timeout. It bounds how long the coordinator waits for the submitted set, while the termination timeout bounds the later shutdown phase. If either is shorter than legitimate work, the coordinator can request interruption while Selenium is in a blocking command. If the batch wait is unbounded, a deadlocked task can hold the build forever before awaitTermination() is ever reached. Set both from observed suite expectations and infrastructure policy, then report the configured values. Do not publish invented “typical” numbers.

Structured lifecycle events make hangs diagnosable. At minimum record the job ID, worker thread name, phase, and session ID once available. Emit task.started before driver creation, session.started after construction, quit.started before cleanup, quit.completed after it returns, and task.failed with the exception type. Avoid logging remote URLs with embedded credentials.

package example.parallel;

import java.time.Instant;

public final class DriverEvents {
  private DriverEvents() {}

  public static void write(String jobId, String phase, String sessionId) {
    String safeSession = sessionId == null ? "none" : sessionId;
    System.out.printf(
        "time=%s job=%s thread=%s phase=%s session=%s%n",
        Instant.now(),
        jobId,
        Thread.currentThread().getName(),
        phase,
        safeSession);
  }
}

An illustrative event sequence might show task.started, session.started, and quit.started with no quit.completed. Those labels are illustrative because they come from this proposed instrumentation, not from a measurement of your suite. The absence narrows the investigation to teardown or process loss. By contrast, task.started with no session.started points to driver construction, capacity, authentication, or browser startup.

Preserve thread names exactly as the runtime reports them. A ThreadGuard failure includes creator and caller thread information, which you can compare with your lifecycle record. Do not reduce both to a generic “parallel thread” label. The difference between pool-1-thread-2 and a test runner worker can reveal that a helper created its own executor beneath the approved one.

Separate ownership bugs from similar parallel failures

The strongest evidence of a cross-thread ownership bug is a ThreadGuard exception naming different creator and caller threads, or your own command log showing one session ID used by different worker threads. A stale-element exception is not equivalent evidence. Elements can become stale because the page replaced them during a normal single-threaded test. Check the thread and session timeline before editing driver storage.

Data collisions can look almost identical to driver sharing. Two independent sessions create a customer with the same email, then one task deletes it while the other verifies it. Screenshots show “random” pages and retries pass, but session IDs and thread ownership remain clean. The fix is unique test data or coordinated environment setup, not a different WebDriver holder.

Grid scarcity is another near miss. Tasks may spend most of their life waiting for sessions because the executor has more workers than Grid has usable slots. Nothing is crossing threads. Compare task.started to session.started, then inspect Grid capacity. Reducing worker count can improve reliability and sometimes total completion time by lowering contention, but that is a capacity decision rather than a thread-safety repair.

An executor leak has a distinct end-of-run signature. Every browser may have quit successfully, all futures may have results, and the JVM still stays alive. Capture a thread dump from the hanging process. Live pool worker threads after the coordinator should have shut down point at missing or incomplete executor termination. Live driver service or browser threads with no quit.completed point back to session cleanup. The same symptom at the process boundary can therefore have two owners.

The shell can request a Java thread dump without changing test state. On a Unix-like CI runner, locate the test JVM through the job's process metadata and run jcmd against that explicit process ID. Avoid a broad kill command or an unvalidated PID variable.

test_pid="$(jcmd -l | awk '/surefire|junit-platform/ {print $1; exit}')"
if [ -z "$test_pid" ]; then
  echo "No matching test JVM found" >&2
  exit 1
fi

jcmd "$test_pid" Thread.print -l > "artifacts/thread-dump-${test_pid}.txt"

That diagnostic has a limitation: process matching depends on how your build launches tests. Validate the match in your environment and print the selected command line before relying on it. If multiple test JVMs run on one host, the script must use a PID recorded when the process starts instead of selecting the first match.

Interrupted tasks require careful interpretation. shutdownNow() interrupts worker threads, but an underlying driver or network call may not react immediately. A task that later reaches finally should still attempt quit(). Do not add an early return in the cleanup path merely because the interrupt flag is set. Record both the interrupt and cleanup outcome.

ThreadLocal leakage has its own fingerprint. Two sequential jobs run on the same worker thread and report the same stored driver reference or session ID, even though the first task completed. A cross-thread detector may say nothing because both uses occur on one thread. If the framework intentionally uses ThreadLocal, call remove() in the same cleanup boundary as quit(), and test two sequential tasks on one single-thread executor so reuse is guaranteed.

Driver construction failures need a separate row in the incident timeline. When new ChromeDriver(options) throws, no session ID is available and no quit() event should appear. Marking that case as “cleanup missing” sends the investigation in the wrong direction. Preserve the task ID, thread, sanitized browser request, and constructor exception. Then check driver discovery, browser startup, endpoint reachability, and Grid capacity according to whether the run is local or remote.

A failure after session.started but before the first navigation has a different owner. Configuration may have created a valid session while a setup helper threw. The finally block must still produce quit.started and either quit.completed or a cleanup failure. If that sequence is intact, a leftover browser process on a remote node belongs in the node or driver investigation, not in speculation that the task forgot to call quit.

Cancellation adds another trap. Calling future.cancel(true) requests interruption when a task is already running; it does not execute cleanup on the coordinator thread. The browser remains owned by the worker until that task reaches its finally block. Never respond to a cancellation by fetching a driver from a global map and quitting it from the caller. That trades a slow cancellation for a confirmed cross-thread access.

Nested concurrency is worth logging explicitly. A browser task may call a library that uses CompletableFuture, a parallel stream, or another executor. Pure computation can safely happen elsewhere if it does not capture WebDriver, WebElement, or mutable page objects tied to the session. Bring the computed value back to the owner thread before issuing the next browser command. A helper signature that accepts WebDriver and returns a future is a code-review warning because the easiest implementation crosses the boundary.

Page objects can leak ownership even when the driver field itself looks private. A page object normally holds a driver reference, and an element object is also associated with a session. Sending either object through a queue or returning it to the coordinator gives another thread a path to issue commands. Return plain values such as text, identifiers, or immutable domain results from the task. Keep session-bound objects inside it.

Extract the scheduling and observation loop into a package-private generic method that accepts Callable instances, then test that method without paying for browsers. One callable should return normally, one should throw a distinctive exception, and one should block until interrupted. The assertions should prove that the normal result survives, the distinctive cause appears among suppressed failures, and timeout handling requests interruption. Each oracle changes when coordinator behavior regresses. A test that only asserts the hard-coded job list has three entries proves nothing about propagation or shutdown.

There is no requirement that every task failure be an AssertionError. Driver startup and command failures are runtime exceptions, while interruption follows a checked path through Future.get(). Preserve concrete causes in the combined error and let the reporting layer render them. Replacing all causes with “parallel execution failed” removes the class, message, and suppressed cleanup evidence that tell the team where to look.

Roll out bounded parallelism in CI

Start with a worker count of one through the new coordinator. That proves lifecycle and exception propagation without concurrency. Then use two independent jobs that visit different routes or use distinct data. A test that passes only when tasks share a login or ordering dependency is not ready for a larger pool.

Add ThreadGuard during migration and keep the event fields stable. Search for driver fields, static holders, constructors that create drivers on the main thread, and helpers that submit nested asynchronous work. Move one ownership boundary at a time. A wholesale conversion can turn one obvious shared field into several hidden ThreadLocals.

CI should retain the task log and any JVM diagnostic output only as evidence, not treat their existence as success. The workflow below bounds the Maven command, records its combined output, requests a JVM dump signal at the bound, and always uploads the artifact directory. The exact timeout is a team policy; the value shown is configuration, not a claimed benchmark.

name: selenium-java-parallel-contract
on: [pull_request]

jobs:
  ownership:
    runs-on: ubuntu-latest
    services:
      chrome:
        image: selenium/standalone-chrome:4.44.0-20260505
        ports:
          - 4444:4444
        options: >-
          --shm-size=2g
          --health-cmd "/opt/bin/check-grid.sh --host 0.0.0.0 --port 4444"
          --health-interval 5s
          --health-timeout 3s
          --health-retries 20
    env:
      SELENIUM_REMOTE_URL: http://localhost:4444
      BROWSER_WORKERS: "2"
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-java@v5
        with:
          distribution: temurin
          java-version: "21"
          cache: maven
      - run: mkdir -p artifacts
      - shell: bash
        run: |
          set -o pipefail
          timeout --signal=QUIT --kill-after=30s 12m \
            mvn -B -Dtest=ParallelOwnershipTest test 2>&1 \
            | tee artifacts/maven.log
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: selenium-ownership-evidence
          path: artifacts/

GNU timeout sends the configured signal when its bound is reached and returns a nonzero status, so pipefail keeps the test step failed even though tee succeeds. The later kill bound prevents a JVM that only prints on QUIT from continuing forever. Confirm which JVM receives the signal with your Maven fork settings. If the test runs in a child process, use a wrapper that records that child's PID and invokes jcmd explicitly before termination.

Match pool size to the narrower of runner capacity and available browser slots. More Java threads do not create more Grid capacity. Excess tasks wait while consuming memory and making failure timelines harder to read. Keep the number in one configuration source, log it at suite start, and reject zero or negative values.

A practical migration canary uses two jobs with deliberately different assertions and test data. If one job expects the status heading and the other expects the account heading, a shared-session regression is more likely to produce a visible wrong-page failure than two identical probes. Give each job a unique server-side identity as well. This does not manufacture a race; it makes accidental state crossover observable when concurrency is enabled.

Run that canary repeatedly only as a stress diagnostic, not as a substitute for deterministic contract tests. A hundred passing repetitions cannot prove the absence of a race. One failing run with session and thread evidence can prove its presence. Keep the normal pull-request gate small, and schedule heavier stress work separately if the suite's risk justifies the capacity.

During the first rollout, compare three counts at the end of each run: jobs submitted, futures observed, and sessions that reached a cleanup outcome. These are accounting facts from the run, not performance measurements. A mismatch has a concrete interpretation. An unobserved future is a coordinator defect; a started session with no cleanup outcome is a lifecycle defect; a submitted job with no start event points at cancellation or executor shutdown before execution.

The rollout costs infrastructure. Two sessions use more CPU and memory than one, and remote providers may charge by concurrency. It also costs diagnostic discipline because failures can arrive in a different order. The gain is controlled throughput with attributable sessions, not “free speed.” Stop increasing workers when queueing, application contention, or test-data collisions dominate.

Know when not to use ExecutorService

Use the test runner's native parallel execution when it already provides isolated fixtures, failure reporting, cancellation, and worker configuration. Adding a private executor inside each test can multiply concurrency unexpectedly. Four runner workers that each create four pool threads can request sixteen browsers, even if the Grid contract allows far fewer.

Avoid thread pools for tests that must share one browser journey. A checkout flow split into separately submitted login, basket, and payment tasks still has one ordered state machine. Run it in one task or one test. Parallelizing its steps creates coordination code without independent work.

Do not keep a driver in ThreadLocal simply because a blog example calls it a best practice. It is appropriate when a framework deliberately binds one long-lived session to each worker and has reliable reset and removal semantics. Task-scoped creation is easier to reason about when isolation matters more than startup cost. The trade-off is session latency: creating a browser for every task is slower than reusing one, but it prevents cookies, windows, downloads, and application state from crossing task boundaries.

Session reuse can be valid for a narrow, read-only probe where state is fully reset and the runner controls ordering. Treat it as a measured optimization with a documented invalidation rule. Do not start there. First make the one-task, one-session path correct, measure where time is spent, and prove that reuse preserves the coverage you care about.

Finally, do not catch task exceptions merely to print them. Logging followed by a normal return converts a failed assertion into a successful Future. Either let the original exception escape or return an explicit result type that the coordinator must classify. A build result should come from observed task outcomes, not from whether the executor happened to reach shutdown.