WebDriver listener logs that preserve the real failure

A click fails with ElementClickInterceptedException, and pytest names the failure OSError: disk full from the logging callback. The browser exception is not gone: it sits in the traceback above a During handling of the above exception, another exception occurred: divider. But the short test summary line, the dashboard row, and every triage search see only the disk error, so the listener meant to add evidence has buried the evidence that mattered. A second run passes because the callback also changed the timing around the click.

Know which calls the wrapper can actually observe

Python’s EventFiringWebDriver wraps a real WebDriver and accepts an AbstractEventListener. The listener class exposes named hooks for navigation, back, forward, find, click, value changes, script execution, close, quit, and exceptions. Methods are optional because the abstract base class provides empty implementations.

The wrapper is not a browser event stream. It does not emit HTTP requests, responses, console messages, DOM mutations, paint timings, or JavaScript exceptions simply because they happen in the page. Those require browser logging, WebDriver BiDi, CDP, application instrumentation, or another suitable source. Listener events describe selected Python WebDriver operations.

Explicit wrapper methods call before_<operation>, invoke the underlying driver or element, and then call after_<operation> if the driver call succeeds. If the driver call raises, the wrapper calls on_exception(exception, driver) and then re-raises. The order is useful, but it places callback behavior on the command path.

The placement has concrete consequences. A failure in before_click prevents the real click. A failure in after_click makes a successful click look failed. A failure in on_exception masks the original WebDriver exception because control never reaches Selenium’s bare re-raise. Logging code therefore needs a stronger “do no harm” rule than ordinary test assertions.

Not every WebDriver method has an explicit before and after pair. The wrapper delegates unknown attributes through __getattr__. A delegated callable is wrapped so an exception can reach on_exception, but no generic before or after event is invented. Read the documented hook list before promising command-level coverage in an observability design.

Element wrapping matters too. find_element returns an EventFiringWebElement, so click, clear, send_keys, and nested element finds can fire element hooks. Code that holds the original raw driver, calls wrapped_driver, or retains an unwrapped element can bypass those hooks. Missing records can be a reference-ownership bug rather than a logging failure.

Consider a stale-element incident. A normal sequence might contain before_find, after_find, before_click, and then on_exception naming StaleElementReferenceException. That tells you the element was located before the click failed. It does not tell you which DOM mutation detached it. If before_click is missing, inspect whether the test used an element obtained from the raw driver.

A wait produces a different shape. WebDriverWait can call a wrapped find_element repeatedly while its expected condition returns false or catches NoSuchElementException. Logging every attempt at warning level floods the failure with expected polling noise. The final TimeoutException belongs to the wait, while individual missing-element exceptions are part of its control flow. A listener should put those expected misses at debug level or omit them from its default channel.

One listener instance per driver keeps ownership obvious. Sharing a listener with mutable counters across parallel drivers creates its own races. The standard logger can coordinate writes through its handlers, but your dictionaries, buffers, and sampling state still need safe ownership. Prefer immutable event fields and a test identifier stored in a context variable or fixture-bound object.

Record context without collecting secrets

A useful event record answers four questions: which test issued the operation, which browser session received it, what kind of operation ran, and whether it completed or raised. It does not need passwords, typed values, script bodies, complete URLs, page source, or raw cookies.

Navigation URLs are dangerous because password-reset links, OAuth callbacks, signed downloads, and internal debug pages often place credentials in queries or fragments. Log the origin and, only if reviewed, a normalized route. Strip user information, query, and fragment. Hashing the complete URL is not automatically safe because a small set of known URLs can be guessed.

Locators can also contain data. An XPath may embed a customer email, order number, or access code. Record the strategy and a short, keyed fingerprint of the selector instead of the raw value. A fresh process key lets you correlate repeated attempts within one run without leaving a reusable plain hash that is easy to test against guessed values.

Do not retrieve extra element details from a callback. Calling get_attribute, reading text, executing JavaScript, or taking a screenshot sends another WebDriver command while the original command is in progress or has just failed. That can change page state, block behind a hung browser, and throw a second exception. Record only values already passed to the hook and cheap local properties such as the session ID.

The before_change_value_of hook receives an element and driver, not the characters passed to send_keys. That is a useful privacy boundary. Do not work around it by inspecting the element’s value after typing. A log line that says a value-change operation completed is normally enough.

Exception messages need restraint. Selenium messages can include selector values, URLs, snippets, capabilities, and stack details. The test runner already owns the original exception and traceback. Record the exception class in the listener and preserve the complete exception in the test failure. A generic regular expression cannot reliably sanitize every token that may appear in a path, locator, or application message.

Here is a listener built around those rules. It uses a context variable for the pytest node ID, logs URL origins rather than full URLs, fingerprints locators, treats expected polling misses as debug records, and catches all failures in its own emission path. The broad catch is intentional here because observability must not replace the WebDriver outcome.

# framework/webdriver_listener.py
from __future__ import annotations

import hashlib
import hmac
import logging
import secrets
from contextvars import ContextVar
from urllib.parse import urlsplit

from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.events import AbstractEventListener


CURRENT_TEST_ID: ContextVar[str] = ContextVar("webdriver_test_id", default="unknown")
LOGGER = logging.getLogger("qa.webdriver")
LOCATOR_FINGERPRINT_KEY = secrets.token_bytes(32)


def _fingerprint(value: object) -> str:
    try:
        return hmac.new(
            LOCATOR_FINGERPRINT_KEY,
            str(value).encode("utf-8"),
            hashlib.sha256,
        ).hexdigest()[:12]
    except Exception:
        return "unavailable"


def _origin(url: str) -> str:
    try:
        parsed = urlsplit(url)
        if not parsed.scheme or not parsed.hostname:
            return "unavailable"
        host = f"[{parsed.hostname}]" if ":" in parsed.hostname else parsed.hostname
        port = f":{parsed.port}" if parsed.port is not None else ""
        return f"{parsed.scheme}://{host}{port}"
    except Exception:
        return "unavailable"


class SafeWebDriverListener(AbstractEventListener):
    def _emit(self, level: int, event: str, driver, **fields: object) -> None:
        try:
            LOGGER.log(
                level,
                event,
                extra={
                    "webdriver_event": event,
                    "test_id": CURRENT_TEST_ID.get(),
                    "session_id": getattr(driver, "session_id", None),
                    **fields,
                },
            )
        except Exception:
            # Listener telemetry must not alter the browser command outcome.
            return

    def before_navigate_to(self, url: str, driver) -> None:
        self._emit(logging.INFO, "navigate.before", driver, origin=_origin(url))

    def after_navigate_to(self, url: str, driver) -> None:
        self._emit(logging.INFO, "navigate.after", driver, origin=_origin(url))

    def before_find(self, by, value, driver) -> None:
        self._emit(
            logging.DEBUG,
            "find.before",
            driver,
            by=str(by),
            locator_id=_fingerprint(value),
        )

    def after_find(self, by, value, driver) -> None:
        self._emit(
            logging.DEBUG,
            "find.after",
            driver,
            by=str(by),
            locator_id=_fingerprint(value),
        )

    def before_click(self, element, driver) -> None:
        self._emit(logging.DEBUG, "click.before", driver)

    def after_click(self, element, driver) -> None:
        self._emit(logging.DEBUG, "click.after", driver)

    def before_change_value_of(self, element, driver) -> None:
        self._emit(logging.DEBUG, "value_change.before", driver)

    def after_change_value_of(self, element, driver) -> None:
        self._emit(logging.DEBUG, "value_change.after", driver)

    def before_quit(self, driver) -> None:
        self._emit(logging.INFO, "quit.before", driver)

    def after_quit(self, driver) -> None:
        self._emit(logging.INFO, "quit.after", driver)

    def on_exception(self, exception: Exception, driver) -> None:
        level = logging.DEBUG if isinstance(exception, NoSuchElementException) else logging.WARNING
        self._emit(
            level,
            "webdriver.exception",
            driver,
            exception_type=type(exception).__name__,
        )

This listener deliberately does not serialize records as JSON. Logging format belongs in the configured handler, where the same record can be rendered for a console or structured sink. Keeping transport out of the callback reduces code and keeps handler failures behind the _emit boundary.

The cost of conservative records is less detail at the moment of failure. You trade a full URL and DOM snapshot for safer, more reliable telemetry. Compensate with test-runner artifacts collected after a test fails, server logs correlated by a non-secret test ID, and an explicit reproduction mode that a developer enables in a controlled environment.

Test the listener as carefully as a test helper

Listener tests should prove both content and non-interference. Start with pure unit tests for redaction and fields. They run without a browser, so a regression cannot hide behind a driver outage. Then add one integration test that wraps a real browser and confirms expected hook order.

The redaction oracle must be capable of failing. Hard-coding a sanitized fixture and asserting it contains no secret proves little. Pass a URL and exception that contain known secret strings, invoke the listener, and assert that no captured record field contains either string. If a future edit logs the raw input, the test fails.

Pytest’s caplog fixture exposes LogRecord objects and rendered text. Inspect both because secrets can live in an extra field even when the console formatter does not display it. The following test also verifies that the event retains the useful exception type.

# tests/test_webdriver_listener.py
import logging
from types import SimpleNamespace

from framework.webdriver_listener import SafeWebDriverListener


def _record_text(record: logging.LogRecord) -> str:
    return " ".join(str(value) for value in record.__dict__.values())


def test_listener_redacts_urls_and_exception_fields(caplog):
    listener = SafeWebDriverListener()
    driver = SimpleNamespace(session_id="session-for-test")
    secret_token = "top-secret-token"
    secret_email = "buyer@example.test"

    caplog.set_level(logging.DEBUG, logger="qa.webdriver")
    listener.before_navigate_to(
        f"https://accounts.example.test/callback?token={secret_token}",
        driver,
    )
    listener.on_exception(
        RuntimeError(
            f"request failed for https://app.example.test/?email={secret_email}"
        ),
        driver,
    )

    captured = "\n".join(_record_text(record) for record in caplog.records)
    assert secret_token not in captured
    assert secret_email not in captured
    assert "RuntimeError" in captured
    assert "https://accounts.example.test" in captured

Add a failure-injection test for the handler. Install a custom logging handler whose emit raises, call a listener hook, and confirm the hook returns. Python logging can route internal handler errors differently depending on configuration, so test the exact handler stack used by your suite. The goal is not to hide every programming error during development; it is to ensure a disk, network, serialization, or formatting failure cannot replace the browser failure in CI.

An integration test should observe a sequence that can change. Navigate to an owned static page, find one stable element, and click it. Capture the records in memory. Assert navigate.before precedes navigate.after, find.before precedes find.after, and click.before precedes click.after. Then find a nonexistent element and assert webdriver.exception names NoSuchElementException.

Do not assert a single giant list for a real application page. Browser-driven waits and framework helpers can add finds, and a harmless refactor will make the test brittle. Test a purpose-built page with one interaction or assert relative order around a correlation marker.

Test the masking case explicitly during listener development. Create a temporary bad listener whose on_exception raises a known RuntimeError, trigger a missing element, and observe that the bad listener error reaches the test. That negative test documents why _emit catches callback failures. Keep it out of ordinary product runs if starting a browser solely for the demonstration is too expensive.

There is an uncomfortable trade-off in the broad except Exception inside _emit: a programming bug in telemetry can disappear. Counter that with unit coverage, a metric or stderr fallback outside the browser path if safe, and a debug mode that re-raises during listener tests. In release CI, preserving the product-test outcome takes priority.

Wire one wrapped driver through pytest

Create the raw driver and listener in one fixture, then yield only the wrapper. If page objects receive both references, someone will eventually use the raw one and create holes in the event sequence. Keep the raw driver private to setup and cleanup.

Bind the pytest node ID before driver creation so session-start logs can carry it. Reset the context variable in finally; otherwise a worker that reuses the same Python context may attach the previous test ID to teardown or the next fixture failure.

# conftest.py
import pytest
from selenium import webdriver
from selenium.webdriver.support.events import EventFiringWebDriver

from framework.webdriver_listener import CURRENT_TEST_ID, SafeWebDriverListener


@pytest.fixture
def driver(request):
    token = CURRENT_TEST_ID.set(request.node.nodeid)
    raw_driver = None
    event_driver = None

    try:
        raw_driver = webdriver.Chrome()
        event_driver = EventFiringWebDriver(raw_driver, SafeWebDriverListener())
        yield event_driver
    finally:
        try:
            if event_driver is not None:
                event_driver.quit()
            elif raw_driver is not None:
                raw_driver.quit()
        finally:
            CURRENT_TEST_ID.reset(token)

Keep teardown simple. Calling quit through the wrapper gives the listener its quit hooks. If a listener bug escapes _emit, the nested finally still resets context, but browser cleanup could be interrupted. A fixture can add a last-resort raw-driver quit attempt, although duplicate quit calls can produce their own noise. The better defense is making the listener unable to throw.

When pytest-xdist runs separate worker processes, each process has its own logger and context. Include the worker ID in the configured log filename or use pytest’s supported logging options to prevent several workers from truncating the same file. Do not have every process append uncoordinated JSON to one network file.

Fixture scope affects correlation. A function-scoped driver naturally maps one session to one test and matches Selenium’s isolation guidance. A class- or session-scoped browser saves startup time but makes a single session ID span several tests, carries cookies and page state, and complicates listener context during teardown. If you accept reuse, report both test ID and session ID and reset application state deliberately.

Page objects should type against the wrapper’s WebDriver-like surface and avoid reaching for wrapped_driver. A library that uses isinstance(driver, WebDriver) may reject EventFiringWebDriver because the wrapper is not a subclass of the concrete driver in the same way some utilities expect. Test framework integrations early. If a tool demands a raw driver, decide whether that component is outside listener coverage or choose a decorator compatible with it.

Listeners are synchronous. Even fast formatting on every find can add time to locator-heavy suites. A page with repeated waits may generate thousands of debug records. Start with navigation and unexpected exceptions at INFO or WARNING, keep find and click at DEBUG, and measure log volume from a representative run. If you introduce a queue handler, bound the queue and define what happens when it fills.

Make CI retain evidence without publishing sensitive data

Pytest captures warning and higher records for failed tests by default. INFO and DEBUG records require live or file-level configuration. Decide which audience needs each. Console output should be short enough to read in a failed job. A retained file can contain debug events if redaction is tested and access is limited.

The workflow below writes listener records to a per-job file, runs the listener’s redaction tests first, and uploads the log only when the product run fails. It does not upload screenshots or page source automatically. Artifact retention and access controls still need to follow the organization’s policy.

name: selenium-python

on:
  pull_request:
  push:
    branches: [main]

jobs:
  browser-tests:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: python -m pip install -r requirements.txt
      - name: Prove listener redaction
        run: pytest tests/test_webdriver_listener.py
      - name: Run browser tests with bounded listener logs
        run: |
          pytest tests/browser \
            --log-cli-level=WARNING \
            --log-file=webdriver-events.log \
            --log-file-level=DEBUG \
            --log-file-mode=w
      - name: Retain redacted events for failed runs
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: webdriver-events-${{ github.run_id }}
          path: webdriver-events.log
          retention-days: 7

The upload step assumes the redaction tests cover the fields your formatter emits. A new handler that serializes exception arguments, thread-local context, or the entire record can reintroduce secrets. Review the final artifact, not just the listener source.

Roll out in observation mode. Enable navigation and unexpected-exception records on a small CI shard. Compare failed tests with and without the wrapper, looking for changed timing, different exceptions, or missing methods. Expand after log volume and failure rate remain acceptable. Then enable debug events only where they answer a recurring diagnostic question.

Give the listener a version in the test report. When event fields or filtering change, incident responders need to know which schema produced an artifact. Avoid calling the schema “complete WebDriver command logging,” because the wrapper’s explicit hook set does not support that claim.

Set a retention limit. Session IDs, internal origins, keyed locator fingerprints, and test names are still operational data. They may reveal application structure even after obvious secrets are removed. Delete artifacts when their debugging value expires.

Read event sequences without overclaiming

Start with an intercepted click. The log shows find.after, click.before, then webdriver.exception with ElementClickInterceptedException. There is no click.after, which is correct because the underlying click raised. This establishes that Selenium found an element and the click command failed. It does not establish which overlay received the pointer. Use the original exception, a failure-time screenshot collected outside the listener, and application state to identify the obstruction.

Now change the sequence to click.before, click.after, followed by a product assertion that times out. The driver accepted the click command, but the application did not reach the asserted state. Investigate event handling, request failure, navigation, and the assertion's locator. Retrying the click because “the click failed” can submit a form twice. An after hook is command evidence, not business evidence.

An explicit wait creates many find.before records and debug-level NoSuchElementException records, then the test raises TimeoutException. That sequence is expected when the element never appears. Count and timing can show that polling occurred, but a listener should not invent an exact wait duration from record count because polling can take variable time and other work may occur. Use the timeout configured by the wait and its final exception as the authoritative boundary.

A listener defect has a shorter shape, and a narrower cause than most teams assume. navigate.before is the last record, the test raises from inside the logging call, and the server records no navigation: the before callback ran, the underlying get never did, and the exception belongs to telemetry rather than the browser. Be precise about what can produce that. A Formatter whose format() raises will not, because logging.Handler.emit catches the failure and routes it to handleError, which prints a --- Logging error --- block to stderr and returns normally. The same holds for a real OSError from a full disk behind a stock StreamHandler or FileHandler, since the write happens inside that guarded emit. Only a handler whose own emit raises, typically a custom sink that ships records to a network service without catching, escapes into the browser command path. Check the handler class before you suspect the formatter. If navigation did reach the server and the error arrived afterward, the failure was in an after hook instead. This distinction is why listener tests inject failures into each phase rather than only testing on_exception.

Missing click records need reference tracing. If navigation and finds appear but a later element click has no event, inspect how the element was obtained. A helper may have called event_driver.wrapped_driver, accepted the raw driver through a second fixture, or cached an unwrapped element before wrapping. Add an identity assertion at the framework boundary during migration, then remove the raw reference from page-object constructors. Do not claim the listener dropped a record until the operation is known to pass through it.

Delegated methods produce another legitimate gap. A method not represented by an explicit hook can succeed with no before or after record. If it raises, on_exception may still appear through the delegation wrapper. Document the operation inventory in the telemetry schema. Consumers should interpret absence only for operations the wrapper promises to hook.

Quit deserves special treatment. A quit.before record with no quit.after and a transport exception means browser cleanup did not complete normally. The fixture must still reset test context and release local resources. A quit.after record means the driver's quit call returned, not that a remote Grid instantly reclaimed every operating-system process. Grid capacity and node cleanup need their own evidence.

Parallel log files can reorder records after they leave callbacks. Queue handlers, network sinks, and multi-process aggregation may not preserve global arrival order. Include process or worker identity and a timestamp from the logging system if ordering across workers matters. Within one test, use session ID and test ID to reconstruct the local sequence. Never sort only by message text or assume two wall-clock timestamps from different machines are perfectly aligned.

Redaction failures have a distinct response plan. Stop artifact publication, restrict access to the affected run, rotate any exposed credential according to its owner, and repair the formatter or field selection. Deleting the visible console line is not sufficient if an uploaded artifact or external sink retained the record. The listener's value depends on being safe enough to keep during failures, which is exactly when URLs and exception messages are most likely to contain sensitive state.

Finally, compare listener evidence with the product assertion before assigning an owner. A clean WebDriver command sequence plus a failed backend audit points to the product boundary. An exception before any product request points toward browser interaction or test setup. A missing listener sequence with successful server traffic points toward coverage gaps in the wrapper. These conclusions remain narrow, testable, and less likely to turn logging into a second source of flaky claims.

Choose another source when the listener cannot answer the question

Do not use an event listener to capture network requests or browser console errors. It sees Python calls, not autonomous browser events. Use Selenium’s supported BiDi or logging APIs where the browser and binding provide them, and give those streams their own compatibility tests.

Avoid listener screenshots on exception. The browser may be unreachable, the window may have closed, or an alert may block the command. Capture screenshots in a pytest failure hook after the test outcome is known, guard that capture separately, and never replace the report’s original exception if artifact collection fails.

Do not log every polling miss at warning level. NoSuchElementException is often expected inside an explicit wait. A warning per poll hides the final timeout and can make log ingestion more expensive than the test. Keep expected misses at debug level and preserve the final wait failure.

Skip the wrapper when a framework already decorates WebDriver and the two wrappers are incompatible. Double wrapping can confuse type checks, element conversion, or ownership. Prefer the framework’s supported event or command hook, or add logging at the test-runner boundary. Prove compatibility before stacking decorators.

Avoid using logs as assertions for product behavior. A click.after record says the driver’s click call returned. It does not prove the application processed the click, sent a request, or rendered the next state. Keep product assertions in the test and treat listener records as diagnostic context.

Finally, remove fields that nobody uses. Every field costs formatting time, storage, review effort, and privacy risk. A small sequence of test ID, session ID, event, safe origin, locator fingerprint, and exception type is often more useful than a huge record whose sensitive parts are redacted after the fact.