Batch and script tools

Use run_steps for a fixed list of actions you have already decided. Use run_script when the flow needs logic between actions (a loop over rows, a branch on what the app shows, a retry): it runs a JavaScript program over the driver's own tools in one call and returns only its result. Every driver call either tool makes passes the same session, permission and approval checks as a direct call.

run_script runs in a QuickJS sandbox with no filesystem, network, process, timers or environment. Limits: timeout_ms (default 30 s, max 120 s), max_calls (default 100, max 500), a 32 MiB heap and a 64 KiB script. It is on by default. Whoever runs the driver turns it off with CUA_DRIVER_DISABLE_RUN_SCRIPT=1 or "disable_run_script": true in ~/.cua-driver/config.json (CUA_DRIVER_EXPERIMENTAL_SCRIPT=0 and "experimental_script": false also work); set_config cannot change it.

Served by cua-driver mcp; see MCP tools for every tool.

run_steps#

The default way to act: run one or more action tools in ONE call, in order, stop at the first failure, and (with observe) get what changed in the same response. Use it whenever you know the next action(s) and would otherwise re-read the window after them: fill three fields, click, and see the result in one call instead of five. Even a single step plus observe:{} replaces an action call followed by a get_window_state call. Each step is {tool, args} where tool is one of click, double_click, right_click, set_value, type_text, press_key, hotkey, scroll, drag, move_cursor and args are exactly that tool's arguments. pid and window_id (or app/window) on run_steps itself are the default window of every step that names none. Reads (get_window_state, zoom) do not run between steps: one there is skipped; a get_window_state last is the observe, a zoom last runs after it. Every step goes through the same session, permission and approval checks as a direct call; a batch grants nothing extra. All steps are validated before the first one runs, so a malformed step 4 changes nothing. Element tokens and element_index values come from a get_window_state read before the batch; a step that changes the UI can invalidate later element targets, so put element-targeted actions before the actions that reshuffle the window, or target later steps by role/name. A token that went stale is retried once on the element with the same role and label, if exactly one matches.

Returns per-step status (ok or the error message) and, when observe is given, ONE bounded get_window_state read after the last executed step (after a failure caused by a dialog holding focus, that dialog is read, even without observe). A step refused because its app ignores background input says how to re-send it; foreground_fallback:true retries such steps in the foreground once (the window is activated). That read is a since:"latest" diff by default: only the rows that changed since your last read of the window, plus a new snapshot_id. Without observe nothing is read. The batch uses one session: steps may omit session or repeat the batch's. delay_ms pauses between steps (max 2000). At most 64 steps; delivery_mode on run_steps is the default for its steps.

TARGET BY NAME (no earlier read needed, so one batch can cross screens): a step can be written {"click": {...args}} as well as {tool, args}, and its args may name the target instead of carrying a token: role (button, textfield, link, checkbox, menuitem, ...; AX/UIA/AT-SPI names also work), name (the element's label; exact match wins over substring, case-insensitive), nth (0-based, when several match), and app (app name or bundle id) and/or window (title substring) instead of pid/window_id. The batch reads the window fresh, waits up to 3 s (step timeout_ms, max 10000) for the element to appear, and acts on it; with only app it uses that app's frontmost window, so a dialog that opened on top is found. Later steps inherit the window, so name app once. The first lookup in a window also reads it once for observe, so the end diff shows what the batch changed even in a window you never read. Several matches fail with the candidates listed; add nth or a fuller name.

WAITS AND CHECKS: a step may add wait_for (checked before its action) and expect (one check or up to 4, checked after it); a step may also be only a check. A check is {role?, name?, text?, gone?, value?, value_contains?, enabled?, selected?, app?, window?, timeout_ms?}: text matches any row of the tree, static text included; gone:true waits for it to disappear. Checks poll until they hold or time out (wait_for 5 s, expect 2 s, max 10 s), and a check that does not hold fails the step with what was found instead. The report says which step failed, in which phase (wait_for, find, action, expect) and why. Whole batch: at most 60 s.

Example (fill a form, submit, confirm the dialog, check the result): {"steps":[{"set_value":{"app":"Safari","role":"textfield","name":"Email","value":"ada@example.com"}},{"click":{"role":"button","name":"Submit"},"expect":[{"role":"button","name":"Confirm"}]},{"click":{"role":"button","name":"Confirm"},"expect":[{"name":"Confirm","gone":true},{"text":"Thanks"}]}],"observe":{}}.

Effect: destructive. Platforms: macOS, Linux, Windows.

Example arguments

{"steps":[{}]}

run_script#

Run a JavaScript program that drives apps through Cua Driver, in ONE call: loops, conditions, reads and checks between actions, and only the result comes back. Use it when the flow needs logic (a loop over rows, a branch on what the app shows, a retry); use run_steps for a fixed list of steps.

The script is the body of an async function: use await, and return a JSON value. It runs in a sandbox with no filesystem, network, process, timers or environment; the only API is cua:

• await cua.getApp("TextEdit" | "com.apple.TextEdit" | {pid, windowId}) → app (follows the app's frontmost window unless a windowId is given)
• await cua.launch(nameOrBundleId) → app; cua.listApps(), cua.listWindows({pid?}), cua.sleep(ms), cua.computer.target
• app.getState(opts?) → get_window_state result (tree_markdown or tree_diff, snapshot_id; pass get_window_state options such as {since:"latest"})
• app.query({role?, name?, text?}) → matching elements across the app's windows {element_token, role, label, value, enabled, selected, frame}, then matching static-text rows {role, label, value, text, display_only} (read text such as a status line from their value)
• app.waitFor({role?, name?, text?, gone?, value?, timeout_ms?}) and app.verify(predicate | predicates) (verify_state)
• app.click(t), doubleClick(t), rightClick(t), typeText(text, t?), pressKey("return" | "cmd+s", t?), hotkey(["cmd","s"]), scroll(t, direction, amount?), setValue(t, value), drag([x,y], [x,y]), zoom([x1,y1,x2,y2]) where a target t is {role, name, nth?} (found fresh, waits up to 3 s), an element_token string, a row number from this app's last getState(), or [x, y] window pixels.
• cua.call(tool, args) for any other allowed tool: click, double_click, right_click, set_value, type_text, press_key, hotkey, scroll, drag, launch_app, list_apps, list_windows, get_window_state, verify_state, zoom. A failed driver call throws a CuaError (catch it to recover). Every call goes through the same session, permission and approval checks as a direct call.

Limits: timeout_ms wall time (default 30000, max 120000), max_calls driver calls (default 100, max 500), 32 MiB heap. Returns the return value, console output, and a per-call log; an error names the script line, the call and the reason.

Example: const app = await cua.getApp("Calculator"); for (const key of ["7", "multiply", "6", "equals"]) await app.click({role: "button", name: key}); const [display] = await app.query({role: "statictext"}); return display.value;.

Effect: destructive. Platforms: macOS, Linux, Windows.

Example arguments

{"script":"<script>"}