Use the SDK

CuaDriver.create() gives you a driver for your Python or TypeScript app. Where the platform allows it, it is the in-process Rust runtime: no daemon, socket, or executable, and your app owns the OS permissions, sessions, and shutdown.

On macOS, create() prefers the CuaDriver.app daemon. An in-process runtime has no AppKit host, so it cannot draw the agent cursor overlay (the visible "the agent is working here" cursor), and macOS attributes the Accessibility and Screen Recording grants to your Python, Node, or IDE process instead of CuaDriver.app. The daemon draws the cursor and owns the grants, so create() connects to a running daemon, or starts the installed app through LaunchServices (the same way cua-driver permissions grant does). Run cua-driver permissions grant once; there is nothing else to configure.

If CuaDriver.app is not installed, create() falls back to the in-process runtime and prints a one-time warning that the agent cursor is unavailable. Pass embedded=True (Python) or { embedded: true } (TypeScript), or set CUA_DRIVER_EMBEDDED=1, to force the in-process runtime on purpose. Other platforms always run in process. To pick the daemon explicitly, use CuaDriver.connect().

python -m pip install cua-driver

Capture the desktop#

The first call creates an implicit session for this runtime; later unnamed calls reuse it. Keep one CuaDriver for the app's lifetime.

import asyncio
 
from cua_driver import CuaDriver, EndSessionInput, GetDesktopStateInput
 
 
async def main() -> None:
    driver = CuaDriver.create()
    try:
        result = await driver.get_desktop_state(
            GetDesktopStateInput(session=None, screenshot_out_file=None)
        )
        if result.is_error:
            raise RuntimeError(result.text)
        print(result.images[0].mime_type)
    finally:
        await driver.end_session(EndSessionInput(session=None))
        await driver.shutdown()
 
 
asyncio.run(main())

In TypeScript, shutdown() stops admitting work and awaits admitted calls; uniffiDestroy() then frees the native handle. Call both, in that order.

Act on a window and verify it#

Find an exact window, snapshot it, click an element by its token, and check the change in a second snapshot. The example expects an app Window Demo with a Counter window, an Increment button, and a Count value; replace them with your own.

import asyncio
 
from cua_driver import (
    ActionTarget, ClickButton, ClickInput, ClickPosition, CuaDriver,
    GetWindowStateInput, InputDeliveryMode, ListAppsInput, ListWindowsInput,
)
 
 
def unique(items, what):
    if len(items) != 1:
        raise RuntimeError(f"Expected one {what}, found {len(items)}")
    return items[0]
 
 
async def main() -> None:
    driver = CuaDriver.create()
    try:
        apps = await driver.list_apps(ListAppsInput())
        app = unique([a for a in apps.apps if a.name == "Window Demo" and a.running], "app")
        windows = await driver.list_windows(ListWindowsInput(pid=app.pid, on_screen_only=True))
        window = unique([w for w in windows.windows if w.title == "Counter"], "window")
        capture = GetWindowStateInput(
            pid=app.pid, window_id=window.window_id, session=None, query=None,
            include_accessibility_tree=True, include_screenshot=True,
            screenshot_out_file=None, max_elements=None, max_depth=None,
            max_dimension=None,
        )
 
        async def snapshot():
            state = await driver.get_window_state(capture)
            if state.degraded or state.truncated:
                raise RuntimeError("Snapshot is degraded or truncated")
            return state
 
        before = await snapshot()
        button = unique([e for e in before.elements or [] if e.label == "Increment"], "button")
        count = unique([e for e in before.elements or [] if e.label == "Count"], "counter")
        await driver.click(ClickInput(
            target=ActionTarget.WINDOW(pid=app.pid, window_id=window.window_id),
            position=ClickPosition.ELEMENT(element_token=button.element_token),
            delivery_mode=InputDeliveryMode.BACKGROUND,
            session=None, button=ClickButton.LEFT, count=1,
        ))
        after = await snapshot()
        updated = unique([e for e in after.elements or [] if e.label == "Count"], "counter")
        if updated.value == count.value:
            raise RuntimeError("Click returned, but the counter did not change")
        print("Verified:", count.value, "->", updated.value)
    finally:
        await driver.shutdown()
 
 
asyncio.run(main())

Version 0.25 does not expose these typed window methods (ActionTarget, ClickPosition); upgrade the bindings and native library together. Rules the example follows:

• click returns ActionResult and raises DriverError.Tool on refusal. A returned action does not prove the UI changed; the second snapshot does.
• Tokens belong to one snapshot. After a stale-token refusal, snapshot again and resolve the element again.
• background delivery never retries in the foreground on its own.
• Window coordinates come from that window's screenshot; do not add the screen origin. See the native-window migration guide.
• An action can reach the OS even if your call times out. Check an independent postcondition before any retry. DriverError.ActionInterrupted reports NotStarted, Completed, or Unknown; never replay Unknown blindly.

A complete runnable version that types into a local browser fixture and confirms the value through a separate /state endpoint lives in libs/cua-driver/examples/agent-sdks (fixture_server.py, then native_driver.py or npm run native).

Constructors#

Python uses snake_case, TypeScript camelCase. Desktop methods mirror the MCP tools (get_window_state, click, type_text, …); list_tools_json() and call_tool() reach the full generic tool surface. Results are ToolResult (text, images, structured_json, is_error, error_code, verified, degraded). On macOS, a direct runtime cannot draw the agent cursor unless the host provides an AppKit main-thread adapter; it returns facility_unavailable. Use a private worker when you need the overlay.

Expose MCP from a desktop app#

When a signed desktop app must let an external agent use its permissions, host a private daemon from the app process and hand its MCP connection to the agent. If only your app calls the driver, use create() instead (on macOS it uses the daemon and draws the agent cursor).

import { CuaDriver, EmbeddedCuaDriverHost } from '@trycua/cua-driver';
 
const host = new EmbeddedCuaDriverHost(
  '/path/inside/YourApp.app/Contents/Resources/cua-driver',
  'com.example.your-app',
);
const connection = await host.start();
const driver = CuaDriver.connect(connection.socketPath);
 
// Your app calls the SDK on `driver`. An external agent launches
// connection.mcp.command with connection.mcp.args and connection.mcp.environment.
 
// Shutdown, in order:
await driver.shutdown();
driver.uniffiDestroy();
await host.stop();
host.uniffiDestroy();

Python exposes the same objects: EmbeddedCuaDriverHost(str(get_binary_path()), "com.yourco.yourapp").

• Call start() from the permission-owning app process. On macOS, a gateway, terminal, open, or NSWorkspace launch changes the TCC responsibility chain and cannot lend the app's grants. A separate backend may launch the returned connection.mcp unchanged, but must not start a second host.

• Ship the executable as a signed app resource, outside Electron's ASAR archive, with its executable bit. The npm package does not bundle it; the Python wheel does.

• Restart on grant changes. restart() gives a new generation, PID, and endpoint: discard old clients and MCP proxies. start() coalesces callers, stop() is idempotent, and waitForExit(generation) reports an unexpected exit.

• Check it. Call check_permissions; on macOS embedded mode returns source.attribution: "host" and never raises its own prompts. Call health_report with {"include":["bundle_identity"]} to confirm the parent app's bundle ID.

• Bound the post-action window wait. After each input action the driver watches the window list to report a menu, dialog, or new window the action opened. For an action that opens nothing, the wait lasts 1000 ms on macOS and 800 ms for Linux X11 foreground delivery. Set CUA_DRIVER_WINDOW_CHANGE_TIMEOUT_MS (0 to 10000; 0 skips it) and CUA_DRIVER_WINDOW_CHANGE_POLL_MS (5 to 1000) in the daemon's launch environment to shorten it; EmbeddedCuaDriverHost admits both in its environment option. On macOS the wait also holds the focus-restore lease, so a shorter wait shortens that protection. Windows ignores both.

• Shorten macOS keystroke pacing. The macOS adapter waits 8 ms between synthesized key events. Set CUA_DRIVER_KEY_GAP_MS (2 to 100) in the same launch environment to change it; EmbeddedCuaDriverHost admits it. Only macOS reads it. Some targets drop key events that arrive too close together, so read the field back before relying on a low value.

Without the SDK, the equivalent low-level launch is:

CUA_DRIVER_EMBEDDED=1 \
CUA_DRIVER_HOST_BUNDLE_ID=com.yourco.yourapp \
cua-driver serve --socket /tmp/yourapp-cua.sock

cua-driver mcp --embedded --socket /tmp/yourapp-cua.sock \
  --host-bundle-id com.yourco.yourapp

Spawn serve directly (for example with posix_spawn or NSTask), never through open(1). A reference macOS host lives at libs/cua-driver/rust/examples/embedded-host-macos.