#766: deliver ScriptUI mouse events to script listeners

Problem: ScriptUI controls never sent mouse events to scripts.
addEventListener("mousedown" | "mouseup" | "mousemove" | "mouseover" |
"mouseout" | "click") listeners never ran, so custom-drawn controls
(onDraw + mouse, as DuScriptUI builds them) could not react to the pointer.
docs/plugins.md listed this as missing.

Fix: scriptui.js builds a ScriptUI MouseEvent (clientX/clientY relative to
the target, button, detail = click count, screenX/Y, modifier keys,
getModifierState, stopPropagation, preventDefault) and delivers it in the
capture, target and bubble phases. addEventListener takes the capture flag.
The parent walk is bounded, so a script-made parent cycle cannot hang it.
Each published control lists the mouse events a listener wants
(Widget::mouse, inherited from containers). The new scriptui.mouse
command dispatches one event with clamped, finite values and rejects
unknown types and disabled controls. The egui ScriptUI view hit-tests the
pointer over script windows and docked panels, skipping covered layers and
windows blocked by a modal dialog. It sends mouseover/mouseout,
mousemove, mousedown/mouseup with click counts, and a click carrying the
pointer details in place of the plain click, so a checkbox still toggles
once. Handlers run under the existing script limits; a throwing listener
reports an error and the window keeps working.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
vangel76
2026-10-11 12:39:23 +02:00
co-authored by Claude Opus 5.5
parent 9a7d3e7699
commit 1c06a793a5
10 changed files with 536 additions and 15 deletions
+4 -1
View File
@@ -512,7 +512,10 @@ Scripts that build ScriptUI windows publish them to the session; agents drive th
3. `scriptui.click {"widget": "ok"}` presses a button / toggles a checkbox / picks a radio button or
tab; `scriptui.set {"widget": "#4", "value": "Shot_"}` types into edit text, moves a slider or
picks a list item (index or text); `scriptui.close {"result": 2}` closes. Controls are addressed
by id, `#id`, `properties.name` or text; `window` can be omitted when one window is open. Each
by id, `#id`, `properties.name` or text; `window` can be omitted when one window is open.
`scriptui.mouse {"widget": "pad", "type": "mousedown", "x": 10, "y": 20}` sends a mouse event
(`mousedown`, `mouseup`, `mousemove`, `mouseover`, `mouseout`, `click`; `x`/`y` relative to the
control, `button`, `detail`, `shiftKey`…) to the script's `addEventListener` listeners. Each
returns the handler run's `{ok, output, error}`.
A dialog's `show()` waits for the user: the `script.run` / `file.runScript` reply carries
+6 -3
View File
@@ -84,9 +84,12 @@ Supported controls: `panel`, `group`, `button`, `iconbutton`, `statictext`, `edi
`tabbedpanel` / `tab` and `image`, with ScriptUI's automatic layout (`orientation`,
`alignChildren`, `alignment`, `margins`, `spacing`, `preferredSize`), resource strings, event
handlers (`onClick`, `onChange`, `onChanging`, `onClose`, `addEventListener`) and `onDraw` with
ScriptUIGraphics. Mouse events (`mousedown`, `mousemove`…) aren't sent to scripts yet. Agents use
a panel like a user: `scriptui.list`, `scriptui.get` (its controls), `scriptui.click`,
`scriptui.set` and `scriptui.close`; every control has the automation id
ScriptUIGraphics. Mouse events (`addEventListener("mousedown" | "mouseup" | "mousemove" |
"mouseover" | "mouseout" | "click", handler, capture?)`) reach the control under the pointer and
its containers (capture, target, bubble; `stopPropagation()`), with a MouseEvent carrying
`clientX`/`clientY` relative to the target, `button`, `detail` (click count) and the modifier
keys. Agents use a panel like a user: `scriptui.list`, `scriptui.get` (its controls),
`scriptui.click`, `scriptui.set`, `scriptui.mouse` and `scriptui.close`; every control has the automation id
`scriptui.<window>.<control id or name>`. The bundled sample `Layer Tools.jsx`
(`crates/engine/scripts/ScriptUI Panels/`) and the Ease Presets extension below are longer
examples.
+1 -1
View File
@@ -19,7 +19,7 @@ our own scripts; no real-world script library has been run against both apps.
| Expression editing | Inline field, error bar, pick whip, Expression Language menu, external editors | Inline field that grows, error bar, pick whips (property and value), language menu | partial | External editor support (#364) |
| Expression performance | Cached, multi-frame | Per-thread boa contexts | unknown | No benchmark |
| Scripting object model | `app`, `Project`, items, layers, properties, keys, eases, render queue, `TextDocument`, `Shape`, markers, `File`/`Folder`, `Socket`, `app.settings` | Same classes, edits run through undoable engine commands; match names table | good | `crates/script` tests (≈ 10 test files) |
| ScriptUI | Dialogs, palettes, dockable panels, resource strings, `onDraw` / ScriptUIGraphics | Same, dockable panels reopen at launch | good | `ui_scriptui`, `tests_ui*` |
| ScriptUI | Dialogs, palettes, dockable panels, resource strings, `onDraw` / ScriptUIGraphics, mouse events | Same, dockable panels reopen at launch | good | `ui_scriptui`, `tests_ui*` |
| Scripts menu, ScriptUI Panels folder | yes | yes | same | [plugins.md](plugins.md) |
| Startup / Shutdown folders | yes | not run | missing | plugins.md |
| `#include`, `#target` directives | yes | a script starting with one doesn't parse | missing | plugins.md; common in real scripts |