mirror of
https://github.com/storytold/effectcraft.git
synced 2026-10-28 12:07:48 -06:00
M13.1: ScriptUI frontend (dialogs, palettes, dockable panels), script menus, example plug-in, docs
- ui-egui: scriptui_view draws script windows at engine layout bounds; PanelKind::ScriptPanel; File ▸ Scripts and Window menu list scripts/panels (egui + native menus) - examples/plugins/posterize-bands (WebAssembly, API v1); host plug-in tests; desktop loads <config>/Plug-ins - MCP: script_ui and history tools; docs/plugins.md, agents.md, parity, architecture, ATTRIBUTION Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
89c9b1b6ee
commit
00eb951711
@@ -220,6 +220,35 @@ In expressions, `text.sourceText.style` / `getStyleAt(i, t)` read styles and the
|
||||
(`setFontSize(v, start?, count?)`, `setFillColor`, `setText`, `setJustification`…) return a
|
||||
styled document.
|
||||
|
||||
### Scripts and ScriptUI windows
|
||||
|
||||
Scripts that build ScriptUI windows publish them to the session; agents drive them like a user:
|
||||
|
||||
1. `scriptui.list` → `[{window, title, kind: dialog|palette|window|panel, script, modal, size}]`.
|
||||
2. `scriptui.get {"window": id}` → the control tree (`type`, `name`, `text`, `value`, `checked`,
|
||||
`items`, `selection`, laid-out `bounds`, `handlers`…).
|
||||
3. `scriptui.click {"widget": "ok"}` presses a button / toggles a checkbox / picks a radio button or
|
||||
tab; `scriptui.set {"widget": "#4", "value": "Shot_"}` types into edit text, moves a slider or
|
||||
picks a list item (index or text); `scriptui.close {"result": 2}` closes. Controls are addressed
|
||||
by id, `#id`, `properties.name` or text; `window` can be omitted when one window is open. Each
|
||||
returns the handler run's `{ok, output, error}`.
|
||||
|
||||
A dialog's `show()` waits for the user: the `script.run` / `file.runScript` reply carries
|
||||
`"waiting": true`, and the script continues (its final output arrives in the reply of the click
|
||||
that closes the dialog). File ▸ Scripts: `file.scripts.list`, `file.runScript {"name": …}`,
|
||||
`file.installScript` / `file.installScriptUIPanel {"path": …}`, `window.scriptPanel {"name": …}`.
|
||||
|
||||
### History, puppet recording, plug-ins
|
||||
|
||||
* `edit.history.list` lists every undo state as a tree (undoing then editing keeps the undone
|
||||
states as a branch); `edit.history.goto {"index": n}` (or `id`, or `steps`) jumps to any of
|
||||
them. MCP: the `history` tool.
|
||||
* `puppet.recordPin {"layer": "#1", "pin": "Puppet Pin 1", "samples": [[t, x, y]…]}` records a
|
||||
drag (t = seconds since it began, layer space) into Position keys at the comp frame rate from the
|
||||
current time; `puppet.recordOptions {speed, smoothing, useDraftDeformation, showMesh}`.
|
||||
* `effect.plugins.load {"path": "x.wasm"}` / `effect.plugins.list`: WebAssembly effect plug-ins
|
||||
([plugins.md](plugins.md)), then `effect.apply` by id like a built-in.
|
||||
|
||||
## CLI
|
||||
|
||||
Each invocation runs a headless engine with no window. It opens the demo project unless you pass
|
||||
|
||||
+9
-5
@@ -22,9 +22,13 @@ Tracking 88%, Effects 85%, Web 65%.
|
||||
What is left, in priority order: the web app's depth (threads, storage, audio, non-blocking
|
||||
renders); stroke taper/wave and multi-segment dashes; variable mask feather points; camera iris/bokeh and focus-link commands; Render Queue field render, crop/resize and templates;
|
||||
approximated effects (Subspace Warp, Key Cleaner) and Liquify's viewer brush; OpenType features and variable
|
||||
font axes; AI/EPS/PDF import; more codecs; 59 preferences not yet wired; ScriptUI;
|
||||
Lumetri Scopes, Footage, Media Browser and Metadata panels; Content-Aware Fill; face tracking; and
|
||||
the "better than After Effects" items (a plug-in API, branching history; GPU particles landed in M12.7).
|
||||
font axes; AI/EPS/PDF import; more codecs; 59 preferences not yet wired;
|
||||
Lumetri Scopes, Footage, Media Browser and Metadata panels; Content-Aware Fill; and face tracking.
|
||||
ScriptUI (dialogs that block `show()`, palettes, dockable ScriptUI panels, File ▸ Scripts install
|
||||
and sample scripts), puppet pin recording and the remaining "better than After Effects" items — a
|
||||
versioned effect plug-in API with sandboxed WebAssembly plug-ins ([plugins.md](plugins.md)),
|
||||
branching undo history in the History panel, and a bit-identical rendering test across runs and
|
||||
thread counts — landed in M13.1 (GPU particles landed in M12.7).
|
||||
|
||||
The sections below are the original audit (morning of 2 October, ≈ 64%) and its updates, kept for
|
||||
history.
|
||||
@@ -112,10 +116,10 @@ the disk cache in the formats wave; the 3D Camera Tracker in M12.6.)
|
||||
| Output | 90% | 3.0 | multiple output modules, pre-render (WebM with VP9 alpha + Opus and WAV/AIFF audio-only landed: VP9 is intra-only, Opus CELT-only) |
|
||||
| Audio | 85% | 0.5 | audio to keyframes |
|
||||
| Import | 88% | 2.0 | AI/EPS/PDF vector footage, PSD smart objects and 3D layers (PSD as footage/composition/retain layer sizes, SVG footage and Create Shapes from Vector Layer landed) |
|
||||
| Automation | ≈ 90% | 0.8 | scripting covers the documented core object model (AUT-2, M14.4: `app`, project items, comps, layers, properties and keyframes, text documents, markers, render queue, Script Console, `effectcraft-cli script`, MCP `run_script`); still missing: ScriptUI panels/dialogs, `.jsxbin`, sockets |
|
||||
| Automation | ≈ 97% | 0.3 | `.jsxbin`, sockets, ScriptUI resource strings and custom `onDraw` graphics (the core object model landed in M14.4; ScriptUI windows/dialogs/dockable panels with `scriptui.*` agent commands, File ▸ Scripts install + sample scripts, and the effect plug-in API (EFF-6, WebAssembly) landed in M13.1) |
|
||||
| Shapes | ≈ 80% | 1.5 | Lottie can't carry stroke taper/wave (stroke Taper and Wave, Dash 2/Gap 2/Dash 3/Gap 3 and radial-gradient Highlight Length/Angle landed in M13.5; pen tool for shape paths and vertex editing in M6.5) |
|
||||
| Compositions | ≈ 80% | 3.5 | Mocha-style planar tracks for templates, Essential Graphics' rare controls (font menus, mirrored properties) (the marker dialog, Composition Flowchart, Essential Graphics with master properties, `.ectemplate` templates and Responsive Design — Time landed: CMP-6, CMP-7) |
|
||||
| Animation | 67% | 10.0 | puppet, Wiggler/Smoother/Motion Sketch (motion-path handles and the graph editor transform box landed in M5.8; keyframe colour labels and Select Keyframe Label Group, Graph Editor snapping to markers / layer ends in M13.5) |
|
||||
| Animation | 70% | 9.0 | puppet depth beyond pins and recording (puppet pin recording with Record Options landed in M13.1), Wiggler/Smoother/Motion Sketch (motion-path handles and the graph editor transform box landed in M5.8; keyframe colour labels and Select Keyframe Label Group, Graph Editor snapping to markers / layer ends in M13.5) |
|
||||
| Text | ≈ 91% | 0.8 | no OpenType feature panel, no extruded strokes, variable-axis animation changes outlines but not advances (vertical Roman / Tate-Chu-Yoko, forced LTR paragraphs, caret on animated and path text, Variable Font Axes and Lottie style runs landed in M13.5; extruded, bevelled text in M7.6; per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API in M9.9–M9.10) |
|
||||
| Web | 85% | 1.0 | viewer frames and Roto Brush propagation still on the page's thread; GPU effects in the browser (browser storage, Web Audio, Web Worker renders/analyses, WebGPU viewer and offline install landed in M15.2) |
|
||||
| 3D | 85% | 6.5 | multi-view layouts, iris shapes in Advanced 3D's depth of field, collapsed 3D precomps in Advanced 3D, the Extended Viewer; stereo rigs, orbit nulls, lights controlled by the camera, cameras/lights from glTF models, environment backgrounds, Advanced 3D motion blur, blend modes and track mattes landed in M7.7; Classic 3D iris-shaped bokeh with highlights, progressive depth of field on tilted layers and the focus-link commands landed in M13.5; Advanced 3D (glTF/OBJ models, primitives, extruded text and shapes, PBR, image-based light, shadow maps, GPU rasteriser) in M7.4–M7.6 |
|
||||
|
||||
+126
@@ -0,0 +1,126 @@
|
||||
# Effect plug-ins
|
||||
|
||||
EffectCraft loads third-party effects through a small, versioned plug-in API (**API version 1**).
|
||||
A plug-in effect behaves exactly like a built-in one: it is listed in Effects & Presets (and the
|
||||
Effect menu) under its category, applied by id (`effect.apply {"effect": "org.example.posterize"}`)
|
||||
or display name, its parameters are ordinary properties (keyframes, expressions, Effect Controls,
|
||||
`prop.set`), and it is saved in projects by id.
|
||||
|
||||
There are two ways to write one:
|
||||
|
||||
| | WebAssembly plug-in | Rust plug-in |
|
||||
|---|---|---|
|
||||
| What | a `.wasm` (or `.wat`) module | a type implementing `effectcraft_effects::plugin::EffectPlugin` |
|
||||
| Loaded by | Effect ▸ Load Effect Plug-in…, `effect.plugins.load {path \| folder}`, or the `Plug-ins` folder next to the settings at start-up | `register_plugin(Arc::new(MyEffect))` in an app that links EffectCraft's crates |
|
||||
| Sandbox | yes: no imports (no files, clock or network), fuel-limited, deterministic floats | none (it is your code) |
|
||||
| Platforms | desktop and CLI (the wasmi interpreter, crate feature `effectcraft-plugin/wasm`); not the web build | everywhere |
|
||||
|
||||
`effect.plugins.list` lists what is loaded: `{api, wasm, plugins: [{id, name, category, version,
|
||||
author, source, params}]}`.
|
||||
|
||||
## The contract
|
||||
|
||||
* **Pixels** are premultiplied RGBA `f32` (0–1; values may exceed 1 in 32 bpc projects), row
|
||||
major, processed **in place**. The buffer is the layer's (after masks and the effects above),
|
||||
possibly at preview resolution (`scale` = buffer pixels per layer pixel) and padded.
|
||||
* **Parameters** arrive flattened to `f64`s in the order they are declared: slider, angle,
|
||||
checkbox (0 / 1) and popup (0-based index) take one value, point two (already in **buffer
|
||||
pixels**), colour four (RGBA 0–1).
|
||||
* **Time** is layer time in seconds.
|
||||
* Rendering must be **deterministic** and may run on several threads at once (a WebAssembly
|
||||
plug-in gets one instance per thread).
|
||||
* A failed render (an error code, a trap, running out of fuel) leaves the frame unchanged.
|
||||
|
||||
## The manifest
|
||||
|
||||
Every plug-in describes itself with a JSON manifest:
|
||||
|
||||
```json
|
||||
{
|
||||
"api": 1,
|
||||
"id": "org.effectcraft.example.posterize-bands",
|
||||
"name": "Posterize Bands",
|
||||
"category": "Stylize",
|
||||
"version": "1.0.0",
|
||||
"author": "EffectCraft contributors",
|
||||
"description": "Quantizes luminance into tinted bands.",
|
||||
"params": [
|
||||
{"id": "levels", "name": "Levels", "type": "slider", "default": 4, "min": 2, "max": 64, "sliderMax": 16, "decimals": 0},
|
||||
{"id": "mix", "name": "Tint Amount", "type": "slider", "default": 50, "min": 0, "max": 100},
|
||||
{"id": "tint", "name": "Tint", "type": "color", "default": [1.0, 0.55, 0.2, 1.0]},
|
||||
{"id": "invert", "name": "Invert Bands", "type": "checkbox", "default": false}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
* `id`: stable, reverse-domain; letters, digits, `.`, `_`, `-`. `ec.` is reserved for built-ins.
|
||||
Ids must be unique (a second plug-in with a taken id is refused).
|
||||
* `category`: one of the Effects & Presets categories (`Blur & Sharpen`, `Stylize`, …) or a new
|
||||
one, which appears after the built-in categories.
|
||||
* Parameter types: `slider` (`default`, `min`, `max`, optional `sliderMin`, `sliderMax`,
|
||||
`decimals`), `angle` (`default` degrees), `checkbox` (`default` bool), `popup` (`options`,
|
||||
`default` index), `point` (`default` as fractions of the layer size, `[0.5, 0.5]` = centre),
|
||||
`color` (`default` RGBA 0–1). Parameter ids are letters, digits and `_`.
|
||||
|
||||
## WebAssembly ABI (v1)
|
||||
|
||||
A module with **no imports** that exports:
|
||||
|
||||
| export | signature | |
|
||||
|---|---|---|
|
||||
| `memory` | memory | linear memory shared with the host |
|
||||
| `ec_api_version` | `() -> i32` | must return `1` |
|
||||
| `ec_manifest_ptr`, `ec_manifest_len` | `() -> i32` | the UTF-8 JSON manifest in memory |
|
||||
| `ec_alloc` | `(bytes: i32) -> i32` | a buffer of at least `bytes` bytes, 8-byte aligned; it may reuse the previous one |
|
||||
| `ec_render` | `(pixels: i32, width: i32, height: i32, params: i32, nparams: i32, time: f64, scale: f64) -> i32` | process the pixels in place; return 0 on success |
|
||||
|
||||
For each frame the host calls `ec_alloc` once for a buffer holding the parameters (`nparams`
|
||||
little-endian `f64`s) followed by the pixels (`width × height × 4` little-endian `f32`s), fills it,
|
||||
calls `ec_render`, and reads the pixels back. Each call gets a fuel budget proportional to the
|
||||
frame size; a module that loops forever stops with an error instead of hanging the app.
|
||||
|
||||
### Example
|
||||
|
||||
[`examples/plugins/posterize-bands`](../examples/plugins/posterize-bands) is a complete plug-in in
|
||||
plain Rust (no dependencies):
|
||||
|
||||
```sh
|
||||
cd examples/plugins/posterize-bands
|
||||
cargo build --release --target wasm32-unknown-unknown
|
||||
# then, in EffectCraft: Effect ▸ Load Effect Plug-in… → target/wasm32-unknown-unknown/release/posterize_bands.wasm
|
||||
# or headless:
|
||||
effectcraft-cli exec effect.plugins.load '{"path": "target/wasm32-unknown-unknown/release/posterize_bands.wasm"}'
|
||||
```
|
||||
|
||||
Any language that compiles to WebAssembly works (C, Zig, AssemblyScript…) as long as the module
|
||||
has no imports. `crates/plugin/src/tests.rs` has a plug-in written directly in WebAssembly text.
|
||||
|
||||
## Rust plug-ins
|
||||
|
||||
```rust
|
||||
use std::sync::Arc;
|
||||
use effectcraft_effects::plugin::*;
|
||||
|
||||
struct Swap(PluginManifest);
|
||||
|
||||
impl EffectPlugin for Swap {
|
||||
fn manifest(&self) -> &PluginManifest { &self.0 }
|
||||
fn render(&self, frame: &mut PluginFrame, params: &PluginParams, _time: f64) -> Result<(), String> {
|
||||
if params.b("on") {
|
||||
for p in frame.pixels.iter_mut() { p.swap(0, 2); }
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
register_plugin(Arc::new(Swap(manifest)))?; // before the UI builds its menus
|
||||
```
|
||||
|
||||
`PluginParams` reads values by parameter id (`f`, `b`, `point`, `color`). At most 64 plug-in
|
||||
effects can be registered per process.
|
||||
|
||||
## Compatibility
|
||||
|
||||
The API version is checked when a plug-in loads: a manifest or module for another version is
|
||||
refused with a message. Within a version the contract above does not change; additions (new
|
||||
parameter types, new optional exports) bump the version.
|
||||
Reference in New Issue
Block a user