Pointer tools
Drag, scroll, move the pointer, and press or release buttons.
Drag, scroll, move the pointer, and press or release buttons.
| Tool | Description | Platforms |
|---|---|---|
drag | Press-drag-release gesture from (from_x, from_y) to (to_x, to_y) in window-local screenshot pixels: the same space get_window_state returns. | macOS, Linux, Windows |
scroll | Scroll the target pid. | macOS, Linux, Windows |
move_cursor | Move a cursor to (x, y). | macOS, Linux, Windows |
mouse_button_down | Press and hold a mouse button at (x,y) via background X11 delivery. | Linux |
mouse_button_up | Release a previously-held mouse button via background X11 delivery. | Linux |
mouse_drag | Move a previously-held mouse button to a new point via background X11 delivery. | Linux |
parallel_mouse_drag | Run multiple mouse drag gestures concurrently via Linux MPX/XI2 virtual master pointers. | Linux |
Served by cua-driver mcp; see MCP tools for every tool.
drag#Effect: destructive. Platforms: macOS, Linux, Windows.
Press-drag-release gesture from (from_x, from_y) to (to_x, to_y) in window-local screenshot pixels: the same space get_window_state returns. Top-left origin of the target's window.
Use for: marquee/lasso selection, drag-and-drop, resizing via a handle, scrubbing a slider, repositioning a panel.
duration_ms (default 500) is the wall-clock budget for the path between mouse-down and mouse-up; steps (default 20) is the number of intermediate mouseDragged events linearly interpolated along the path. Increase both for slower, more human drags; decrease for snap gestures.
modifier keys (cmd/shift/option/ctrl) are held across the entire gesture.
When from_zoom is true, coordinates are in the last zoom image for this pid; the driver maps them back to window coordinates before dispatching.
macOS has no background drag: a window-scoped drag needs delivery_mode:"foreground" and window_id. It briefly fronts the exact window, moves the physical pointer along the path, then restores the prior frontmost app. Without delivery_mode:"foreground" the call is refused with background_unavailable and nothing is sent.
macOS parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
button | "left" | "right" | "middle" | Mouse button used for the drag. Default: left. | |
delivery_mode | "background" | "foreground" | Window-scoped drag on macOS supports only "foreground": briefly front the exact window (window_id required), drive the physical pointer through the gesture, then restore the prior frontmost app. The default "background" is refused with background_unavailable because macOS has no background drag route. Desktop scope ignores this field. | |
duration_ms | integer | Wall-clock duration of the drag path between mouseDown and mouseUp. Default: 500. Range: 0 to 10000. | |
from_x | number | required | Drag-start X in window-local screenshot pixels. Top-left origin. |
from_y | number | required | Drag-start Y in window-local screenshot pixels. Top-left origin. |
from_zoom | boolean | When true, coordinates are in the last zoom image for this pid; driver maps back to window coordinates. | |
modifier | string[] | Modifier keys held across the entire gesture: cmd/shift/option/ctrl. | |
pid | integer | Target process ID. | |
scope | "window" | "desktop" | "window" | Use desktop with no pid/window_id for native get_desktop_state screenshot coordinates. |
steps | integer | Number of intermediate mouseDragged events linearly interpolated along the path. Default: 20. Range: 1 to 200. | |
to_x | number | required | Drag-end X in window-local screenshot pixels. |
to_y | number | required | Drag-end Y in window-local screenshot pixels. |
window_id | integer | CGWindowID for the window the pixel coordinates were measured against. Optional only when pid owns exactly one eligible top-level window; otherwise the action refuses with ambiguous_window_target. |
Parameters on every platform
| Parameter | Type | Default | Description |
|---|---|---|---|
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. | |
target | object | Preferred per-call target: an exact window (kind="window", pid, window_id) or the primary desktop (kind="desktop", display_id="primary"). | |
target.display_id | string | Display id. Required when kind is desktop; use "primary". | |
target.kind | "window" | "desktop" | window requires pid and window_id; desktop requires display_id. | |
target.pid | integer | Process id. Required when kind is window. Minimum: 0. | |
target.window_id | integer | Window id from list_windows. Required when kind is window. Minimum: 0. |
Example arguments
{"from_x":100,"from_y":200,"to_x":100,"to_y":200}scroll#Effect: mutating. Platforms: macOS, Linux, Windows.
Scroll the target pid.
element_token (preferred) or window-local x, y. Sends a real wheel event at that point, so it scrolls whatever is under it. The only way to scroll nested overflow:auto regions in web views.Electron/Chromium windows drop background wheel events: there a background scroll reveals the content just past the edge of the scroll container under the target through accessibility (about amount notches); delivery_mode:"foreground" is the fallback.
amount is wheel notches (targeted) or key repetitions (keystroke).
macOS parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
amount | integer | Wheel notches or key repetitions, clamped to the maximum. Default 3. Range: 1 to 50. | |
delivery_mode | "background" | "foreground" | Default "background": no fronting or focus steal. "foreground": briefly front the window, act, restore the prior app; last resort when background did not land. | |
pid | integer | Target process ID. Required unless scope is "desktop" or element_token is supplied. | |
scope | "window" | "desktop" | "window" | "desktop" with x,y and no pid/window_id uses get_desktop_state coordinates. |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. | |
window_id | integer | Window ID. Required with x/y; optional with element_token. | |
x | number | Window-local screenshot X (get_window_state PNG). With y, wheel-scrolls at that point; needs window_id. | |
y | number | Window-local screenshot Y. |
Parameters on every platform
| Parameter | Type | Default | Description |
|---|---|---|---|
by | "line" | "page" | Scroll granularity. Default: line. | |
direction | "up" | "down" | "left" | "right" | required | Scroll direction. |
element_token | string | Element handle <snapshot_id>:<row> from get_window_state: row [11] of a read whose snapshot_id is s0000002a is "s0000002a:11" (also in structuredContent.elements[].element_token). Returns an explicit stale error naming the current snapshots once a newer read supersedes it. | |
target | object | Preferred per-call target: an exact window (kind="window", pid, window_id) or the primary desktop (kind="desktop", display_id="primary"). | |
target.display_id | string | Display id. Required when kind is desktop; use "primary". | |
target.kind | "window" | "desktop" | window requires pid and window_id; desktop requires display_id. | |
target.pid | integer | Process id. Required when kind is window. Minimum: 0. | |
target.window_id | integer | Window id from list_windows. Required when kind is window. Minimum: 0. |
Example arguments
{"direction":"up"}move_cursor#Effect: mutating, idempotent. Platforms: macOS, Linux, Windows.
Move a cursor to (x, y). With target {kind: "window", pid, window_id}, x/y are window screenshot pixels (as for click) and only the agent cursor moves there; in a background app it also hovers through a route that leaves the real pointer alone where one exists (AXShowAlternateUI on the element under the point, or an Electron page) and says so. Without a target, moves only the agent cursor overlay, in screen points. Other macOS hover and tooltips follow the real pointer, so most background apps see no hover; for a web page in Chrome, Edge or Brave use browser_pointer hover instead. With scope=desktop, moves the user's real OS pointer in native get_desktop_state screenshot coordinates; over a window of an app that is not in front it refuses unless delivery_mode is "foreground".
macOS parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
delivery_mode | "background" | "foreground" | Desktop scope only. Default "background": refuses, before moving anything, when the point is over a window of an app that is not in front, since that would move the user's pointer. "foreground": move the real pointer anyway; only when nobody is using this screen or the user agreed. | |
scope | "window" | "desktop" | "window" | "window" (default) moves only the agent cursor overlay; "desktop" moves the real OS pointer. |
x | number | required | Destination X. Window target: window screenshot pixels. No target: screen points for the agent cursor overlay. Desktop scope: native get_desktop_state screenshot pixels. |
Parameters on every platform
| Parameter | Type | Default | Description |
|---|---|---|---|
cursor_id | string | Cursor instance to move. Default: 'default'. | |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. | |
target | object | Preferred per-call target: an exact window (kind="window", pid, window_id) or the primary desktop (kind="desktop", display_id="primary"). | |
target.display_id | string | Display id. Required when kind is desktop; use "primary". | |
target.kind | "window" | "desktop" | window requires pid and window_id; desktop requires display_id. | |
target.pid | integer | Process id. Required when kind is window. Minimum: 0. | |
target.window_id | integer | Window id from list_windows. Required when kind is window. Minimum: 0. | |
y | number | required | Destination Y, in the same space as x. |
Example arguments
{"x":100,"y":200}mouse_button_down#Press and hold a mouse button at (x,y) via background X11 delivery. Does not release the button; pair with mouse_drag / mouse_button_up. Returns the current held-button state.
Effect: destructive. Platforms: Linux.
| Parameter | Type | Default | Description |
|---|---|---|---|
button | "left" | "right" | "middle" | Mouse button. Default "left". | |
coordinate_frame | "window" | "desktop" | Frame of x/y (and from_x/from_y/to_x/to_y). Default "window": window-local screenshot pixels as returned by get_window_state. "desktop": full-screen pixels as returned by get_desktop_state, translated to the target window. Passing scope:"desktop" together with pid/window_id means the same thing. | |
cursor_id | string | Optional multi-cursor instance id. Default: 'default'. | |
from_zoom | boolean | Set true after a zoom call to auto-translate zoom-image pixel coordinates back to full-window space. | |
pid | integer | required | Target process ID. |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. When both are present, session takes precedence over cursor_id. | |
window_id | integer | required | Window id from list_windows. |
x | number | required | Window-local pixel X of the target window's own get_window_state screenshot (0..screenshot_width). For get_desktop_state pixels pass scope:"desktop" (or coordinate_frame:"desktop"). |
y | number | required | Window-local pixel Y of the target window's own get_window_state screenshot (0..screenshot_height); see x. |
Example arguments
{"pid":1,"window_id":1,"x":100,"y":200}mouse_button_up#Release a previously-held mouse button via background X11 delivery. If x/y are omitted, releases at the last held position. Returns the current held-button state.
Effect: destructive. Platforms: Linux.
| Parameter | Type | Default | Description |
|---|---|---|---|
coordinate_frame | "window" | "desktop" | Frame of x/y (and from_x/from_y/to_x/to_y). Default "window": window-local screenshot pixels as returned by get_window_state. "desktop": full-screen pixels as returned by get_desktop_state, translated to the target window. Passing scope:"desktop" together with pid/window_id means the same thing. | |
cursor_id | string | Optional multi-cursor instance id. Default: 'default'. | |
from_zoom | boolean | Set true after a zoom call to auto-translate zoom-image pixel coordinates back to full-window space. | |
pid | integer | Optional process ID; must match the held button's target. | |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. When both are present, session takes precedence over cursor_id. | |
window_id | integer | Optional window id; must match the held button's target. | |
x | number | Window-local pixel X of the target window's own get_window_state screenshot (0..screenshot_width). For get_desktop_state pixels pass scope:"desktop" (or coordinate_frame:"desktop"). | |
y | number | Window-local pixel Y of the target window's own get_window_state screenshot (0..screenshot_height); see x. |
mouse_drag#Move a previously-held mouse button to a new point via background X11 delivery. Requires an active mouse_button_down state; does not release the button. Returns the updated held-button state.
Effect: destructive. Platforms: Linux.
| Parameter | Type | Default | Description |
|---|---|---|---|
cursor_id | string | Optional multi-cursor instance id. Default: 'default'. | |
duration_ms | integer | Total drag duration. Default: 500. Range: 0 to 10000. | |
from_zoom | boolean | Set true after a zoom call to auto-translate zoom-image pixel coordinates back to full-window space. | |
pid | integer | Optional process ID; must match the held button's target. | |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. When both are present, session takes precedence over cursor_id. | |
steps | integer | Intermediate MotionNotify events. Default: 20. Range: 1 to 200. | |
window_id | integer | Optional window id; must match the held button's target. | |
x | number | required | Window-local pixel X of the target window's own get_window_state screenshot (0..screenshot_width). For get_desktop_state pixels pass scope:"desktop" (or coordinate_frame:"desktop"). |
y | number | required | Window-local pixel Y of the target window's own get_window_state screenshot (0..screenshot_height); see x. |
Example arguments
{"x":100,"y":200}parallel_mouse_drag#Run multiple mouse drag gestures concurrently via Linux MPX/XI2 virtual master pointers. Each drag item runs on its own session-scoped master pointer (true same-window concurrent draws on X11). Each item presses once, glides continuously through its whole path, and releases once: one smooth held drag, not a chain of clicks. A path is given either as a straight segment (from_x/from_y → to_x/to_y) or as a function fn = y(x) sampled over [x_from, x_to] in window-local pixels (e.g. fn:"x" is a diagonal, fn:"300+120*sin(x/40)" a sine wave). Functions support + - * / ^, sin/cos/tan, sqrt, abs, exp, ln, pi, e.
Effect: destructive. Platforms: Linux.
| Parameter | Type | Default | Description |
|---|---|---|---|
drags | object[] | required | Two or more drag gestures to run concurrently, each on its own virtual master pointer. Items: 2 to any. |
session | string | For multi-call work, prefer a short public session label and repeat it on every call that accepts it. Omit it to use the authenticated transport's implicit lifecycle session. |
Example arguments
{"drags":[{"session":"<session>","window_id":1},{"session":"<session>","window_id":1}]}