Make parallel Selenium failures readable with structured logs

Two checkout tests fail in the same CI minute, and the console output leaves you matching stack traces to browser sessions by eye. One session hit a button covered by a consent panel. The other completed its assertion and failed while closing the browser. Without correlation fields, both failures become anonymous red text in one interleaved stream.

The useful fix is not a louder console.log. Give every record an identity, write events at boundaries the test owns, and preserve the first error even when evidence collection or teardown also fails. The result should let an engineer reconstruct one attempt without guessing which worker, browser, or command produced each line.

Why parallel output loses the story

A CI console is a transport, not a data model. Two Node.js processes can write similar messages within the same second. A test runner can retry a case under the same display name. A Selenium Grid can create a fresh browser session for that retry. Sorting the combined text by wall-clock timestamp does not restore those relationships.

WebDriver gives us one strong join key. A session is created before normal browser commands run, and commands for that session are routed with its session ID. The JavaScript binding exposes the session through driver.getSession(), and Session#getId() returns its ID. That value is the right link between a test-side record and Grid, browser-driver, or vendor-provider records that also include the session. It is not a test ID, though. One test can have several attempts and therefore several sessions, while one poorly isolated session can be reused by several tests. Keep both identities.

The negotiated capabilities provide another useful snapshot. driver.getCapabilities() resolves to the capabilities for the active driver, and methods such as getBrowserName() and getBrowserVersion() return the configured values when present. Record that small allowlist once. Do not serialize the entire capability object. Remote URLs, proxy settings, vendor options, and provider-specific capabilities can contain credentials or account metadata.

Several clocks solve different questions. An ISO timestamp helps an operator search systems that use wall time. A monotonic elapsed value helps order work and calculate duration inside one Node.js process, even if the system clock changes. A sequence number proves emission order from one logger. None of them creates a total order across machines. If worker A writes sequence 18 and worker B writes sequence 11, the numbers are unrelated. That is why each sequence belongs with the logger's test, attempt, worker, and session identities.

Promise boundaries matter too. Selenium's JavaScript methods return promises for commands such as navigation, element interaction, waits, and teardown. A missing await means the test no longer owns the point at which that operation settles. The visible symptom might be a rejection after the test has ended, a command colliding with cleanup, or a runner warning about asynchronous work. A structured wrapper cannot repair the missing await, but an unmatched step.start record gives you a concrete place to inspect.

Keep four evidence streams conceptually separate:

• Test records describe the business step, assertion, attempt, and cleanup path that your code owns.
• Selenium client diagnostics describe what the language binding is doing internally.
• Driver or Grid logs describe protocol handling, routing, queues, and browser process behavior.
• Browser logs describe messages exposed by a supported browser log type, often including page console output.

Those streams can corroborate each other, but they are not interchangeable. A page console error does not prove the test failed because of JavaScript. A TimeoutError from an explicit wait does not prove the Grid was slow. A Grid session that disappears after quit() is expected; the same disappearance during a product action is not.

The WebDriver specification is useful when classifying protocol outcomes. It defines command routing by session ID and standard error codes such as no such element, element click intercepted, and invalid session id. Human-readable error messages and stack traces are implementation-defined, so avoid parsers that depend on one full message. Store the JavaScript error name and a scrubbed message for reading, then use step identity and session evidence for classification.

Structured records also need a narrow purpose. They should answer which attempt did what, which session received it, whether it settled, and what failed first. They should not become a second copy of requests, page HTML, cookies, test data, or screenshots encoded as text. Large payloads make searches slower and privacy reviews harder. Save binary artifacts separately and reference their paths from a bounded record.

Build records around the boundaries you own

Start with a small schema you can explain in a code review. The following logger writes one JSON object per line to a session-specific file. It captures the WebDriver session and an allowlisted capability subset once, assigns a unique ID to every step invocation, and records both wall time and monotonic elapsed time. Event and step names are restricted to short machine labels, which keeps ordinary prose, URLs, and query strings out of those fields.

The snippet assumes a current TypeScript project using Node.js and selenium-webdriver. It relies only on Node's built-in file APIs. Compile it using the same TypeScript configuration as the suite.

import { appendFileSync, mkdirSync } from 'node:fs';
import { createHash } from 'node:crypto';
import { join } from 'node:path';
import { performance } from 'node:perf_hooks';
import type { WebDriver } from 'selenium-webdriver';

type DetailFields = {
  durationMs?: number;
  errorName?: string;
  errorMessage?: string;
  stepName?: string;
  artifactPath?: string;
  availableLogTypes?: string[];
  browserLogLevel?: string;
  browserLogMessage?: string;
  browserLogTimestamp?: number;
};

const SAFE_LABEL = /^[a-z0-9][a-z0-9._:-]{0,79}$/;
const SECRET_ASSIGNMENT =
  /(authorization|cookie|password|passwd|token|secret)\s*[:=]\s*[^\s,;]+/gi;

export function scrubText(value: string): string {
  return value.replace(SECRET_ASSIGNMENT, '$1=[REDACTED]').slice(0, 1_000);
}

function errorFields(error: unknown): Pick<DetailFields, 'errorName' | 'errorMessage'> {
  if (error instanceof Error) {
    return { errorName: error.name, errorMessage: scrubText(error.message) };
  }
  return { errorName: 'NonErrorThrow', errorMessage: scrubText(String(error)) };
}

export async function createWebDriverLog(options: {
  driver: WebDriver;
  runId: string;
  attemptId: string;
  testId: string;
  workerId: string;
  directory: string;
}) {
  for (const value of [options.runId, options.attemptId, options.testId, options.workerId]) {
    if (!SAFE_LABEL.test(value)) throw new TypeError(`Unsafe log identity: ${value}`);
  }

  const session = await options.driver.getSession();
  const sessionId = session.getId();
  const capabilities = await options.driver.getCapabilities();
  const startedAt = performance.now();
  const fileKey = createHash('sha256')
    .update([options.workerId, process.pid, options.testId, options.attemptId, sessionId].join('\0'))
    .digest('hex');
  const file = join(options.directory, `${fileKey}.ndjson`);
  let fileAvailable = true;
  let sequence = 0;
  let stepNumber = 0;
  try {
    mkdirSync(options.directory, { recursive: true });
  } catch (error) {
    fileAvailable = false;
    process.stderr.write(`${JSON.stringify({
      schemaVersion: 1,
      timestamp: new Date().toISOString(),
      elapsedMs: Math.round((performance.now() - startedAt) * 10) / 10,
      sequence: ++sequence,
      event: 'logger.open.fail',
      runId: options.runId,
      attemptId: options.attemptId,
      testId: options.testId,
      workerId: options.workerId,
      sessionId,
      ...errorFields(error),
    })}\n`);
  }

  function write(event: string, stepId?: string, details: DetailFields = {}): void {
    if (!SAFE_LABEL.test(event)) throw new TypeError(`Unsafe event name: ${event}`);
    const record = {
      ...details,
      schemaVersion: 1,
      timestamp: new Date().toISOString(),
      elapsedMs: Math.round((performance.now() - startedAt) * 10) / 10,
      sequence: ++sequence,
      runId: options.runId,
      attemptId: options.attemptId,
      testId: options.testId,
      workerId: options.workerId,
      sessionId,
      browserName: capabilities.getBrowserName() ?? 'unknown',
      browserVersion: capabilities.getBrowserVersion() ?? 'unknown',
      event,
      ...(stepId ? { stepId } : {}),
    };
    const line = `${JSON.stringify(record)}\n`;
    if (!fileAvailable) {
      process.stderr.write(line);
      return;
    }
    try {
      appendFileSync(file, line, 'utf8');
    } catch (error) {
      fileAvailable = false;
      process.stderr.write(line);
      process.stderr.write(`${JSON.stringify({
        schemaVersion: 1,
        timestamp: new Date().toISOString(),
        elapsedMs: Math.round((performance.now() - startedAt) * 10) / 10,
        sequence: ++sequence,
        event: 'logger.write.fail',
        runId: options.runId,
        attemptId: options.attemptId,
        testId: options.testId,
        workerId: options.workerId,
        sessionId,
        ...errorFields(error),
      })}\n`);
    }
  }

  async function step<T>(name: string, operation: () => Promise<T>): Promise<T> {
    if (!SAFE_LABEL.test(name)) throw new TypeError(`Unsafe step name: ${name}`);
    const stepId = `${sessionId}:${++stepNumber}`;
    const stepStartedAt = performance.now();
    write('step.start', stepId, { stepName: name });
    try {
      const value = await operation();
      write('step.pass', stepId, {
        stepName: name,
        durationMs: Math.round((performance.now() - stepStartedAt) * 10) / 10,
      });
      return value;
    } catch (error) {
      write('step.fail', stepId, {
        stepName: name,
        durationMs: Math.round((performance.now() - stepStartedAt) * 10) / 10,
        ...errorFields(error),
      });
      throw error;
    }
  }

  async function quit(driver: WebDriver): Promise<{ ok: true } | { ok: false; error: unknown }> {
    try {
      await driver.quit();
      write('session.quit.pass');
      return { ok: true };
    } catch (error) {
      write('session.quit.fail', undefined, errorFields(error));
      return { ok: false, error };
    }
  }

  write('session.start');
  return { file, write, step, quit };
}

The schema keeps stepName separate from artifactPath, so a query never has to guess whether a value names an action or a file. Do not loosen DetailFields into Record<string, unknown> for convenience. An allowlist forces a reviewer to notice when somebody proposes storing a new kind of data.

Text scrubbing is a last guard, not a secrecy guarantee. An email address without a key named email, a bearer token copied into an exception, or personal data rendered in a locator can still escape a regular expression. Keep step names static. Use synthetic accounts. Limit who can read artifacts. Set retention deliberately. If the application handles regulated data, send the schema through the same security and privacy review as any other telemetry.

The per-session destination matters in parallel runs. Multiple sessions appending to one file introduce a second concurrency problem while trying to diagnose the first. The digest gives each combination of worker, process, test, attempt, and session its own path, while the unhashed identity fields remain inside every record for searching. A synchronous append completes before its own caller continues, but it does not establish an order across concurrent workers or inside the browser.

Wrap business-significant operations, not every findElement() call in every page object. A useful step name stays meaningful when the locator changes. submit.expired-card tells the investigator what the test tried to achieve. webdriver.command.47 forces them back into source code. Driver-level debug logging is available for the rare case where the protocol conversation itself is under suspicion.

Here is a complete Node test that preserves the original test error when teardown also fails. BASE_URL must point to the system under test, and the selectors and expected text represent the application's public test contract. The assertion can fail when the product displays any other message, so it is a real oracle rather than a check against a hard-coded fixture that guarantees itself.

import test from 'node:test';
import assert from 'node:assert/strict';
import { By, until } from 'selenium-webdriver';
import {
  buildChromeWithBrowserLogs,
  captureFailureEvidence,
} from './support/failure-evidence.js';
import { createWebDriverLog } from './support/webdriver-log.js';

test('checkout shows the expired-card decline', async () => {
  const baseUrl = process.env.BASE_URL;
  const expiredCardNumber = process.env.EXPIRED_CARD_NUMBER;
  if (!baseUrl || !expiredCardNumber) {
    throw new Error('BASE_URL and EXPIRED_CARD_NUMBER are required');
  }
  const attemptId = process.env.TEST_ATTEMPT_ID ?? process.env.ATTEMPT_ID ?? 'attempt-1';

  const driver = await buildChromeWithBrowserLogs();
  let log: Awaited<ReturnType<typeof createWebDriverLog>>;
  try {
    log = await createWebDriverLog({
      driver,
      runId: process.env.RUN_ID ?? 'local',
      attemptId,
      testId: 'checkout.expired-card',
      workerId: process.env.TEST_WORKER_ID ?? `pid-${process.pid}`,
      directory: process.env.WEBDRIVER_LOG_DIR ?? 'artifacts/webdriver',
    });
  } catch (error) {
    try {
      await driver.quit();
    } catch {
      // Preserve the logger or session initialization error.
    }
    throw error;
  }

  let testFailed = false;
  let testError: unknown;

  try {
    await log.step('navigate.checkout', () =>
      driver.get(new URL('/checkout', baseUrl).toString()),
    );

    await log.step('submit.expired-card', async () => {
      await driver.findElement(By.css('[data-testid="card-number"]')).sendKeys(expiredCardNumber);
      await driver.findElement(By.css('[data-testid="place-order"]')).click();
    });

    await log.step('assert.decline-message', async () => {
      const alert = await driver.wait(
        until.elementLocated(By.css('[role="alert"]')),
        5_000,
        'Expected a checkout alert',
      );
      assert.match(await alert.getText(), /card has expired/i);
    });
  } catch (error) {
    testFailed = true;
    testError = error;
    await captureFailureEvidence(
      driver,
      log,
      `artifacts/screenshots/checkout.expired-card-${attemptId}-${process.pid}.png`,
    );
  }

  const cleanup = await log.quit(driver);
  if (testFailed) throw testError;
  if (!cleanup.ok) throw cleanup.error;
});

Notice what the test does not log. It never writes the card value, full checkout URL, DOM, cookies, or capabilities map. The test ID and step labels are stable identifiers. The actual input stays in code or an access-controlled data source, where it can be reviewed separately.

Cleanup is outside the main try rethrow path on purpose. If the assertion fails and quit() also fails, the test reports the assertion while the log retains session.quit.fail as secondary evidence. Throwing directly from a finally block can replace the first error with the cleanup error. That replacement turns a product regression into an infrastructure ticket and wastes the evidence you worked to collect.

Session creation needs a separate boundary. createWebDriverLog() cannot include a session ID until Builder#build() succeeds and getSession() resolves. If session creation fails, keep a runner-level event with run, attempt, test, worker, requested browser, and the SessionNotCreatedError, but set sessionId to null. Do not invent a placeholder session ID. The WebDriver specification only gives you an active session identity after successful creation.

Work through three failures that look alike

Parallel CI usually gives you a symptom cluster rather than a clean diagnosis. The following cases can all end with a failed checkout test and red browser-related output, but they demand different fixes.

A click is intercepted by an overlay

Suppose submit.expired-card ends with JavaScript's ElementClickInterceptedError. The standard WebDriver error means another element would receive the click because the requested target is obscured. The Selenium error guide calls out overlapping UI, pop-ups, fixed navigation, and animations as common causes. That category is stronger evidence than a generic screenshot showing the button somewhere on the page.

Start with the record for the failed step. Confirm its session ID, then inspect a screenshot captured before cleanup. Look at the center of the target, not merely whether the target exists. An active consent dialog or loading mask over that point supports the interception diagnosis. Browser console errors might explain why the overlay never closed, but their presence alone does not prove they caused the click failure.

Do not respond by forcing the click through JavaScript. That bypasses the user-facing interaction that WebDriver was checking and can let the test pass while a real user remains blocked. Wait for the known overlay state to clear, close the panel through its supported control, or fix the product condition that leaves it open. The cost is a more explicit page contract and, if the overlay is legitimate, some additional wait time.

The nearest look-alike is ElementNotInteractableError. The W3C definitions distinguish an obscured click target from an element that is not pointer- or keyboard-interactable. A screenshot, element rectangle, and exact error name separate them. Treating every interaction error as an overlay creates the wrong wait and can hide a locator that matched a hidden duplicate.

A wait expires before an alert appears

Now suppose submit.expired-card passes, while assert.decline-message fails with TimeoutError. Selenium's driver.wait() polls its condition until it produces a truthy value or the timeout expires; errors thrown while evaluating the condition propagate. A timeout therefore tells you the condition did not succeed within its budget. It does not identify why.

Use the ordered steps to narrow the search. A completed navigation and completed submit step show that those wrapped promises settled. The failure screenshot can answer whether the browser stayed on checkout, navigated to an error page, or displayed different copy. A browser console entry from the same session may expose a page exception. The application server's request ID, if the product safely renders one, can link to service telemetry without putting the response body in the WebDriver log.

The near-miss is a broken locator. Both a slow response and a selector that no longer matches can exhaust the same wait. Inspect the DOM or use browser developer tools against the captured page to evaluate the selector. If the expected alert is present under a changed attribute, fix the locator. If the alert is absent and the network or service evidence shows unfinished work, fix synchronization or the product. Adding a longer global timeout before making that distinction increases suite duration and preserves the ambiguity.

Another near-miss is a missing await. If a call to log.step('submit.expired-card', ...) is started but not awaited, a later assertion can run before submission settles. The log may show step.start without its matching terminal event at the moment cleanup begins. Fix the promise ownership first. A retry or sleep merely changes the scheduling window.

Teardown loses an already completed result

In the third case, all product steps pass and session.quit.fail is the only failure record. The test should fail as infrastructure or cleanup, not as a checkout regression. Conversely, if assert.decline-message fails first and teardown also fails, report the assertion as primary and attach the teardown failure.

After a successful driver.quit(), the JavaScript driver is invalidated and cannot issue more commands. Code that calls getCurrentUrl(), takes a screenshot, or drains browser logs afterward can produce NoSuchSessionError. The W3C protocol calls the corresponding remote error invalid session id; the JavaScript binding maps that condition to its own error class name. Keep protocol code and language-binding class names in different fields if you collect both. Do not assume the strings are identical.

A Grid node disappearing before quit() completes can look similar from the test process, but the evidence differs. Search Grid or provider records with the captured session ID. A successful session.quit.pass followed by another driver call points to test lifecycle misuse. A transport failure during an otherwise first and only quit, with remote evidence that the node vanished, points toward infrastructure. If the remote logs are unavailable, label the cause unresolved instead of choosing the more convenient owner.

The same discipline applies to browser console noise. A severe browser entry and a failed assertion in one session are correlated, not automatically causal. If the assertion expected a total of 42 and the page rendered 41 after a valid API response, the product state is direct evidence. If an unrelated analytics script also logged an error, fixing or filtering that script may reduce noise but will not repair the total. Product assertions remain the verdict.

Separate the product failure from its near misses

Failure capture must happen while the session is still usable. A screenshot after quit() is too late. Browser log retrieval after the session disappears is too late. Put both in the catch path before cleanup, and treat capture failures as secondary events so they never replace the test error.

Browser log support deserves special caution. Selenium's JavaScript source describes the remote logging API as non-standard and supported by only some browsers. getAvailableLogTypes() tells you what the current driver exposes. Calling get(type) returns the entries available since the start of the session or the previous call, then resets that log buffer. If another helper already drained browser, your failure hook cannot recover those earlier entries.

This helper enables browser log preferences when building a Chrome session, feature-detects the resulting log type, writes severe entries through the allowlisted logger, and saves a PNG. Other browsers and remote providers may expose a different set of log types, so the absence event is evidence about availability, not a test failure.

import { mkdir, writeFile } from 'node:fs/promises';
import { dirname } from 'node:path';
import { Browser, Builder, logging, type WebDriver } from 'selenium-webdriver';
import { scrubText } from './webdriver-log.js';

type EvidenceLog = {
  write: (event: string, stepId?: string, details?: {
    artifactPath?: string;
    availableLogTypes?: string[];
    browserLogLevel?: string;
    browserLogMessage?: string;
    browserLogTimestamp?: number;
    errorName?: string;
    errorMessage?: string;
  }) => void;
};

export async function buildChromeWithBrowserLogs(): Promise<WebDriver> {
  const preferences = new logging.Preferences();
  preferences.setLevel(logging.Type.BROWSER, logging.Level.ALL);
  const driver = await new Builder()
    .forBrowser(Browser.CHROME)
    .setLoggingPrefs(preferences)
    .build();
  return driver;
}

export async function captureFailureEvidence(
  driver: WebDriver,
  log: EvidenceLog,
  screenshotPath: string,
): Promise<void> {
  try {
    const png = await driver.takeScreenshot();
    await mkdir(dirname(screenshotPath), { recursive: true });
    await writeFile(screenshotPath, png, 'base64');
    log.write('evidence.screenshot.pass', undefined, { artifactPath: screenshotPath });
  } catch (error) {
    const value = error instanceof Error ? error : new Error(String(error));
    log.write('evidence.screenshot.fail', undefined, {
      errorName: value.name,
      errorMessage: scrubText(value.message),
    });
  }

  try {
    const logs = driver.manage().logs();
    const available = await logs.getAvailableLogTypes();
    if (!available.includes(logging.Type.BROWSER)) {
      log.write('evidence.browser-log.unavailable', undefined, {
        availableLogTypes: available,
      });
      return;
    }

    for (const entry of await logs.get(logging.Type.BROWSER)) {
      if (entry.level.value < logging.Level.SEVERE.value) continue;
      log.write('evidence.browser-log.entry', undefined, {
        browserLogLevel: entry.level.name,
        browserLogMessage: scrubText(entry.message),
        browserLogTimestamp: entry.timestamp,
      });
    }
  } catch (error) {
    const value = error instanceof Error ? error : new Error(String(error));
    log.write('evidence.browser-log.fail', undefined, {
      errorName: value.name,
      errorMessage: scrubText(value.message),
    });
  }
}

The helper applies the same limited scrubbing rule to browser and capture errors. That still cannot recognize every application-specific secret. Some applications print tokens or personal data to console.error, which is already a product defect and still creates an artifact-handling risk.

Search the artifacts by identity before reading them as a story. The first command below lists failed steps with the fields most likely to classify the boundary. The second shows every event for one session in sequence order. These commands operate on actual records; they do not imply any made-up timing benchmark.

jq -r '
  select(.event == "step.fail") |
  [.runId, .attemptId, .testId, .sessionId, .stepId, .stepName, .errorName, .errorMessage] |
  @tsv
' artifacts/webdriver/*.ndjson

session_id='replace-with-the-session-id-from-the-failure'
jq -s --arg session_id "$session_id" '
  map(select(.sessionId == $session_id)) | sort_by(.workerId, .sequence)
' artifacts/webdriver/*.ndjson

Do not sort a merged run by sequence alone. Each logger starts its own counter, including a new logger created in a reused Node.js process. Interpret sequence with the test, attempt, worker, and session identities. When a single test somehow appears under two session IDs in one attempt, investigate session creation and fixture ownership. When two attempts share a session ID, investigate driver reuse or incorrect IDs. Neither pattern should be normalized away by the report.

Look for the last trustworthy transition. A step.start followed by step.fail proves the wrapper observed settlement as a rejection. A step.start with no terminal record can mean the process crashed, the promise never settled, the command was not awaited by its caller, or the artifact was cut off. The missing event narrows the boundary but does not choose among those causes. Runner exit status, process signals, and Grid logs supply the next evidence.

Selenium WebDriver does not create a Playwright-style trace archive for this pattern. There is no trace viewer to open unless your organization has added another tracing system. Inspect the NDJSON sequence, the pre-quit screenshot, the test runner stack, and driver or Grid records joined by session ID. Saying exactly which artifact does not exist prevents an investigator from searching for imaginary evidence.

Roll the schema into CI without breaking the suite

A logger that lands across hundreds of tests in one change is hard to trust. Begin with one noisy parallel suite and keep the existing reporter. The human-readable runner output remains useful for quick scans, while the JSON Lines files serve correlation and automated checks. Compare both for several real failures before using structured data to route tickets or calculate quality metrics.

Freeze the first schema before expanding it. Require schemaVersion, run, attempt, test, worker, session, timestamp, elapsed time, sequence, and event. Define which fields may be null. Publish an event catalog. A consumer should reject an unknown schema version rather than silently interpreting changed fields. Schema versioning costs coordination, but it is cheaper than a dashboard that mixes incompatible records.

Validate relationships, not merely JSON syntax. Every step.start should have one terminal step.pass or step.fail with the same step ID. Each successfully created session file should contain one session.start and one quit outcome. Sequences should increase within a file. A validator must fail on malformed lines, forbidden keys, missing terminal events, or duplicate terminal events. Those checks can fail when logging code regresses, which makes them meaningful oracles.

Do not turn absence of a session file into an automatic application failure. A browser may fail before a session exists, the runner may be killed, or the logging directory may be unwritable. Capture pre-session events in a separate runner file and retain the CI process status. The structured layer supplements the runner; it cannot observe code that never reached it.

Retries need first-class identity. Reusing only the test name merges the failing first attempt with a passing second attempt and makes a flaky test look clean. Give every attempt its own ID, browser session, log file, screenshot path, and final status. In the example, the workflow-level ATTEMPT_ID distinguishes job reruns; a retry-aware fixture should set the application-defined TEST_ATTEMPT_ID to a unique value for each in-job test attempt. Report the overall result separately. A passed retry does not erase the original failure category.

Adoption works best through a fixture or test hook that owns driver creation and teardown. Page objects should receive a logger or an already wrapped step() function, not instantiate global loggers. A global mutable current-test ID is especially dangerous under concurrency. Pass the identity explicitly or use a runner-supported per-test context whose isolation you have verified.

Keep the rollout reversible. An environment variable can select the artifact directory or disable successful-step detail while leaving failure events enabled. Avoid a switch that changes test behavior, timeout budgets, or browser options at the same time. Otherwise, turning on observability changes the thing being observed and makes before-and-after comparisons weak.

This GitHub Actions job keeps the test command's exit code through pipefail, uploads evidence even when tests fail, and avoids sharing one NDJSON file across workers. It assumes the repository already has an npm run test:e2e script and that tests honor the environment variables used by the logger.

name: selenium-e2e

on:
  workflow_dispatch:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: mkdir -p artifacts/webdriver artifacts/screenshots
      - name: Run Selenium tests
        shell: bash
        env:
          RUN_ID: run-${{ github.run_id }}
          ATTEMPT_ID: attempt-${{ github.run_attempt }}
          WEBDRIVER_LOG_DIR: artifacts/webdriver
          BASE_URL: ${{ vars.E2E_BASE_URL }}
          EXPIRED_CARD_NUMBER: ${{ vars.E2E_EXPIRED_CARD_NUMBER }}
        run: |
          set -o pipefail
          npm run test:e2e 2>&1 | tee artifacts/test-runner.log
      - name: Upload Selenium evidence
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: selenium-evidence-${{ github.run_id }}-${{ github.run_attempt }}
          path: artifacts/
          if-no-files-found: error
          retention-days: 7

setup-node runs before npm ci, so npm is available when dependency installation begins. The artifact upload has if: always(), so a failed test step does not skip evidence retention. if-no-files-found: error catches wiring mistakes, but remember that test-runner.log should exist even when session creation fails. That file keeps the artifact contract valid without pretending a WebDriver session was created.

Choose retention from the data sensitivity and investigation window, not from a copied workflow. Seven days in the example is a policy choice, not an observed optimum. Teams with slower incident triage may need more. Teams testing personal or financial data may need less, stricter access, or no browser messages at all.

During migration, add correlation to the suites with the highest worker count first. Then cover retry-heavy tests, remote Grid runs, and teardown failures. Measure overhead using your own suite before and after the change. Do not publish illustrative timings as if they came from that measurement. If file writes become visible in the critical path, buffer records per process and flush at safe boundaries, accepting that a hard process kill may lose the buffer.

Accept the cost, and know when to stop logging

Structured evidence costs latency, storage, code ownership, and attention. Synchronous file writes make each record durable sooner, but they pause the Node.js process. Screenshots add command time and artifact size. Browser log collection adds another remote command and may drain information another tool expected to read. A broad schema creates long-term compatibility work for every report that consumes it.

The default pattern should therefore be selective. Emit session lifecycle, business-step boundaries, failures, and cleanup. Capture a screenshot on failure. Drain supported browser logs once in a documented hook. Enable verbose Selenium client, browser-driver, or Grid logs for a focused investigation, then turn them back down. High-volume protocol logs can bury the test identity you needed in the first place.

Avoid this wrapper when you need a wire-level audit of every WebDriver request and response. Logical step records deliberately omit protocol payloads. Use supported driver or Grid diagnostics for that investigation, protect their more sensitive contents, and correlate them with the same session ID. Monkey-patching driver.execute() or replacing the command executor ties framework code to binding internals and can break on upgrades.

Do not use browser console collection as a cross-browser gate without checking support for every browser and provider in the matrix. The JavaScript logging module itself warns that remote logging is non-standard and its API is not frozen. Feature detection makes missing support visible, but it cannot make implementations equivalent. If console errors are a product requirement, assert a product-visible consequence or choose a browser event mechanism with a documented contract for your target matrix.

Skip detailed logging in performance measurements until you have quantified its effect. Synchronous writes and screenshots change timing. If the goal is page or command latency, record only the minimum markers needed by the measurement design, run a control, and state exactly how the instrumentation was configured. Numbers collected under one logger mode should not be compared with another as though the setup were identical.

Do not log real secrets to make a flaky authentication test easier to reproduce. Authorization headers, cookies, passwords, one-time codes, reset links, signed URLs, and full capability objects do not belong in general CI artifacts. Use synthetic data and stable non-sensitive identifiers. Where a sensitive value is essential to diagnosis, handle it through an approved secure channel with separate access and retention, not by weakening the shared logger.

Small serial suites may not need this machinery. If one locally run test has one session and a clear stack trace, a stable test name plus a screenshot can be enough. Adding a versioned schema and artifact validator there increases maintenance without resolving ambiguity. Introduce structure when concurrency, retries, remote execution, or multiple evidence producers create a real ownership problem.

Finally, never let log completeness become the test oracle. A run can contain perfect step.pass records while the application shows the wrong result because the wrapped steps were too broad or the assertion was weak. It can also lose its final log line after a process crash while earlier product assertions were valid. Keep product assertions specific, keep logging failures visible, and report those outcomes separately. That separation is what lets a working QA engineer say whether the defect belongs to the product, the automation, the browser session, or the evidence pipeline.