Selenium Pointer Actions: Viewport, Element, and Offset Coordinates

Selenium pointer action coordinates are deterministic only when the test names the origin. Viewport coordinates begin at the visible browsing area, element coordinates begin at an element's in-view center, and relative offsets begin at the pointer's current remote-end position. The same numbers can therefore identify three different points.

Choose the most stable origin for the behavior under test, calculate against current layout, and verify the element that actually received the event. Random offset changes may suppress a failure temporarily, but they erase the geometry contract the test should explain.

Translate from origin to hit target

The Selenium Actions API provides high-level pointer operations, while the W3C WebDriver specification defines pointer origins and dispatch. An element origin avoids most page-coordinate arithmetic because the remote end resolves the element's current in-view center before applying offsets.

Coordinate validity and hit testing are separate. A point can be inside the viewport yet belong to an overlay, child control, or neighboring element. Always assert the application effect after the move and click.

Prefer an element origin for stable controls

For an ordinary button, use click(element) or moveToElement(element).click() and let WebDriver select the center. Use an element offset only when a meaningful subregion matters, such as a color well or canvas coordinate.

import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.interactions.Actions;

WebElement map = driver.findElement(By.cssSelector("[data-testid='seat-map']"));

new Actions(driver)
    .moveToElement(map, 36, -18)
    .click()
    .perform();

The offsets are relative to the element's in-view center, not its top-left corner. A design specification expressed from the top-left must be converted using current width and height. Avoid hard-coded conversions when responsive layout changes the element dimensions; derive the point from a documented region or expose a stable DOM target.

CSS transforms complicate visual geometry. WebDriver uses rendered, in-view geometry for element origin, while application canvas coordinates may apply their own zoom and pan transform. Keep that domain transform in a named helper and test it separately.

Offset signs should be visible in the requirement: positive x moves right and positive y moves down in the coordinate space. Name expected regions such as legendSwatchPoint rather than passing anonymous integer pairs through page objects. That small modeling step prevents a correct number from being reused against the wrong element origin.

Use viewport coordinates for viewport-owned behavior

An absolute viewport move is appropriate when the behavior is tied to the visible area rather than a particular DOM element, such as dismissing a hover panel by moving to a known empty test region.

new Actions(driver)
    .moveToLocation(12, 12)
    .perform();

Viewport coordinates use the visible browsing area's top-left as zero. They are not document coordinates, so adding the page's scroll offset is usually a category error. They are also not screenshot bitmap pixels. Do not multiply by device pixel ratio.

Reserve a stable empty region in a test fixture if the gesture depends on background space. Production pages with sticky headers, banners, or responsive controls can make a supposedly empty absolute point own a different element across viewports.

Treat moveByOffset as stateful

moveByOffset(x, y) begins at the pointer's current location. WebDriver retains that pointer location across performed actions in the session. A relative move is therefore suitable for a deliberate gesture segment, but fragile as the first operation in a helper.

new Actions(driver)
    .moveToElement(map)
    .moveByOffset(20, 0)
    .clickAndHold()
    .moveByOffset(80, 0)
    .release()
    .perform();

The initial element move establishes the reference point. Without it, a previous hover, drag, or cleanup action changes the destination. Keep the complete relative gesture in one sequence so its coordinate state is locally understandable.

Scrolling changes the relationship between document content and viewport coordinates, but the virtual pointer remains associated with a viewport position until another action moves it. Re-establish an element origin after any scroll that matters.

Express low-level viewport movement precisely

The Java low-level API makes origin and duration visible. It is useful for protocol-focused helpers or coordinated input, while the high-level Actions methods remain clearer for ordinary tests.

import java.time.Duration;
import java.util.List;
import org.openqa.selenium.interactions.PointerInput;
import org.openqa.selenium.interactions.Sequence;
import org.openqa.selenium.remote.RemoteWebDriver;

PointerInput mouse = new PointerInput(PointerInput.Kind.MOUSE, "map-mouse");
Sequence clickAt = new Sequence(mouse, 0);
clickAt.addAction(mouse.createPointerMove(
    Duration.ofMillis(120),
    PointerInput.Origin.viewport(),
    240,
    180
));
clickAt.addAction(mouse.createPointerDown(PointerInput.MouseButton.LEFT.asArg()));
clickAt.addAction(mouse.createPointerUp(PointerInput.MouseButton.LEFT.asArg()));

((RemoteWebDriver) driver).perform(List.of(clickAt));

The movement duration controls the pointer move within the action sequence; it is not an application-readiness wait. Follow the gesture with a condition tied to the resulting state.

Use unique input-source identifiers when composing low-level sequences. Reusing a source name with contradictory assumptions about position or pressed buttons makes session state difficult to diagnose.

Diagnose MoveTargetOutOfBounds at current geometry

An out-of-bounds error means the computed target is outside the remote end's permitted viewport coordinates. Capture the viewport dimensions, scroll position, element rectangle, requested origin, offsets, frame context, and browser window size at the failure.

import java.util.Map;
import org.openqa.selenium.JavascriptExecutor;

@SuppressWarnings("unchecked")
Map<String, Object> geometry = (Map<String, Object>)
    ((JavascriptExecutor) driver).executeScript("""
        const r = arguments[0].getBoundingClientRect();
        return {
          left: r.left, top: r.top, width: r.width, height: r.height,
          viewportWidth: innerWidth, viewportHeight: innerHeight,
          scrollX: scrollX, scrollY: scrollY
        };
        """, map);

This is diagnostic evidence, not a reason to duplicate the browser's complete hit-testing algorithm in Java. Common causes are top-left assumptions applied to center offsets, stale geometry after layout, document coordinates passed as viewport coordinates, a zero-sized element, or a move performed in the wrong frame.

Set an explicit browser window size for geometry-sensitive tests. Responsive breakpoints can legitimately move or replace the target, and a maximized local window is not equivalent to a fixed Grid viewport.

Capture geometry immediately before the action when investigating an intermittent case. A rectangle collected after the failure may already reflect a tooltip, animation, or responsive transition caused by the attempted move. Before-and-after evidence distinguishes a bad calculation from layout that changed during dispatch.

Verify hit testing and overlays

When a move succeeds but the wrong element reacts, inspect the topmost element at the point with document.elementFromPoint. A tooltip, sticky toolbar, transparent layer, or child node may own the location. Preserve its tag and stable attributes in failure output.

Do not immediately replace the action with JavaScript click(). Script dispatch bypasses pointer travel and browser hit testing, so it changes the requirement. Wait for an intentional overlay to close, move to the correct active subregion, or report a real product defect when users would hit the same blocker.

For a canvas, expose the selected logical coordinate or object ID in an accessible status region when possible. Pixel color or event counters alone are weak end-to-end outcomes. The application should confirm which seat, chart point, or drawing object the gesture selected.

Pointer button semantics matter for context menus and alternate buttons. Use the specific low-level button only when the product requires it, and assert the page-owned menu or behavior. A browser-native context menu is outside normal DOM assertions and should not be mistaken for an application result.

Handle frames, scrolling, and responsive layout

Switch to the correct frame before resolving element origins inside it. Element-relative actions let the remote end account for the frame's rendered position; manual addition of every ancestor frame offset is brittle. If an absolute viewport requirement spans frames, reconsider the test design because ordinary user input is targeted within one active browsing context.

Locate the element after layout stabilization, not before a responsive rerender. Scroll it into a known view if the scenario requires a specific visible region, then read geometry and perform the action without unrelated commands between those steps.

Avoid one offset shared across desktop and mobile layouts unless the component guarantees identical internal geometry. A named percentage or logical coordinate can be converted per current rectangle, while a raw pixel constant rarely communicates the product rule.

Apply a pointer coordinate checklist

• Name viewport, element, or current-pointer origin explicitly.
• Prefer direct element actions for ordinary controls.
• Remember that element offsets begin at the in-view center.
• Establish a known origin before any relative move.
• Use CSS pixel geometry, not screenshot or device pixels.
• Recalculate after scrolling, resizing, or rerendering.
• Switch to the correct window and frame first.
• Capture viewport, rectangle, offsets, and topmost element on failure.
• Assert the application object or state that received the gesture.
• Keep JavaScript dispatch out of real pointer-path validation.

Close on explicit geometry

Pointer tests become stable when coordinates express a real product rule. Anchor ordinary interactions to elements, reserve viewport points for viewport behavior, and use relative offsets only inside a sequence with a known start. Then bounds and interception failures describe the page's actual geometry instead of exposing hidden pointer history.