Merge main into M13.5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas
2026-10-03 05:37:39 -04:00
co-authored by Claude Opus 5.5
123 changed files with 13948 additions and 1221 deletions
+50 -6
View File
@@ -105,11 +105,18 @@ effectcraft-cli run clip.ecproj track.motion '{"layer":"#2"}' \
`track.mask` follows the pixels inside a mask and keys its Mask Path on every frame (vertices and
tangents move with the fitted motion). `method` is `position`, `positionScale`,
`positionScaleRotation` (the default, also kept as the Tracker panel's Method), `positionScaleRotationSkew`
or `perspective`; `direction` is `forward|backward|frameForward|frameBackward` from the current
`positionScaleRotation` (the default, also kept as the Tracker panel's Method), `positionScaleRotationSkew`,
`perspective`, `faceOutline` (Face Tracking (Outline Only): the mask, drawn around a face, becomes
the face outline on every frame) or `faceDetailed` (Face Tracking (Detailed Features): also keys a
Face Track Points effect with one point per landmark — eyebrows, eyes, pupils, nose tip,
nostrils, mouth corners and lips, chin, jaw); `direction` is `forward|backward|frameForward|frameBackward` from the current
time (or `start` / `end` in seconds). Like `track.analyze` it runs in the background in the app
(progress in `track.status` under `mask`, `track.stop` cancels and keeps what was tracked) and is
one undo step; `"wait": true` blocks.
one undo step; `"wait": true` blocks. `track.extractFaceMeasurements {layer?}` (Extract & Copy Face
Measurements) keys a Face Measurements effect from the Face Track Points (head position, scale and
orientation X/Y/Z, eye openness, eyebrow distance from eye, mouth openness, width and offset), puts
those keys on the keyframe clipboard and returns them per frame plus a tab-separated `clipboard`
text.
`mask.interpolate` (Window ▸ Mask Interpolation ▸ Apply) adds in-between Mask Path keys between
each pair of selected Mask Path keys (or the existing keys at `times`, in seconds), giving both
@@ -138,7 +145,11 @@ the app re-analyses automatically (headless: run `warp.analyze` again). Settings
properties under the instance's groups, e.g. `effects/#1/stabilization/result` (0 Smooth Motion,
1 No Motion), `effects/#1/stabilization/smoothness`, `effects/#1/stabilization/method`,
`effects/#1/borders/framing` (0 Stabilize Only … 3 Stabilize, Synthesize Edges),
`effects/#1/borders/autoScale/maximumScale`, `effects/#1/advanced/showTrackPoints`.
`effects/#1/borders/autoScale/maximumScale`, `effects/#1/advanced/showTrackPoints`. Method 3
(Subspace Warp, the default) bends a mesh per frame to the smoothed feature trajectories, so
scenes with depth stabilise where one perspective transform cannot;
`effects/#1/advanced/rollingShutterRipple` (0 Automatic Reduction, 1 Enhanced Reduction: a finer,
softer mesh) tunes it.
```sh
effectcraft-cli run shaky.ecproj track.warpStabilizer '{"layer":"#1","wait":true}' \
@@ -150,13 +161,17 @@ effectcraft-cli exec warp.status '{"layer":"#1"}' shaky.ecproj --json
### 3D Camera Tracker
Animation ▸ Track Camera and the Tracker panel's button (`track.camera {layer?, shotType?:
fixed|variable|specify, aov?, solveMethod?: auto|typical|flat|tripod, detailed?, wait?}`) apply
fixed|variable|specify, aov?, solveMethod?: auto|typical|flat|tripod, detailed?, lensDistortion?,
undistort?, wait?}`) apply
Effect ▸ Perspective ▸ 3D Camera Tracker (or reuse the layer's) and analyse it in the background:
"Analyzing in background (step 1 of 2)" tracks features, "Solving camera" solves.
`camera.analyze {layer?, effect?, wait?}` re-runs it, `camera.cancel` stops it without writing
anything. `camera.solveStatus` reports `progress`/`banner`, `analyzed`, `solved`, `methodUsed`,
`averageError` (pixels), `focalLength`, `horizontalAngleOfView`, the current frame's `camera`
(position, orientation, zoom) and `groundPlane`. `camera.points {time?}` lists the solved points
(position, orientation, zoom), `groundPlane` and the solved `lensDistortion` (`k1`, `k2`, normalised by
half the frame diagonal) when Solve Lens Distortion (`effects/#1/advanced/lensDistortion`, also on
with Detailed Analysis) is on; `effects/#1/advanced/undistort` renders the footage undistorted so
the solved camera's 3D layers line up. `camera.points {time?}` lists the solved points
visible now with their `id`, `comp` position, `depth`, `world` position and `error`.
Select points with `camera.selectPoints {points, add?, toggle?}` (the viewer's click / Shift-click /
@@ -220,6 +235,35 @@ In expressions, `text.sourceText.style` / `getStyleAt(i, t)` read styles and the
(`setFontSize(v, start?, count?)`, `setFillColor`, `setText`, `setJustification`…) return a
styled document.
### Scripts and ScriptUI windows
Scripts that build ScriptUI windows publish them to the session; agents drive them like a user:
1. `scriptui.list` → `[{window, title, kind: dialog|palette|window|panel, script, modal, size}]`.
2. `scriptui.get {"window": id}` → the control tree (`type`, `name`, `text`, `value`, `checked`,
`items`, `selection`, laid-out `bounds`, `handlers`…).
3. `scriptui.click {"widget": "ok"}` presses a button / toggles a checkbox / picks a radio button or
tab; `scriptui.set {"widget": "#4", "value": "Shot_"}` types into edit text, moves a slider or
picks a list item (index or text); `scriptui.close {"result": 2}` closes. Controls are addressed
by id, `#id`, `properties.name` or text; `window` can be omitted when one window is open. Each
returns the handler run's `{ok, output, error}`.
A dialog's `show()` waits for the user: the `script.run` / `file.runScript` reply carries
`"waiting": true`, and the script continues (its final output arrives in the reply of the click
that closes the dialog). File ▸ Scripts: `file.scripts.list`, `file.runScript {"name": …}`,
`file.installScript` / `file.installScriptUIPanel {"path": …}`, `window.scriptPanel {"name": …}`.
### History, puppet recording, plug-ins
* `edit.history.list` lists every undo state as a tree (undoing then editing keeps the undone
states as a branch); `edit.history.goto {"index": n}` (or `id`, or `steps`) jumps to any of
them. MCP: the `history` tool.
* `puppet.recordPin {"layer": "#1", "pin": "Puppet Pin 1", "samples": [[t, x, y]…]}` records a
drag (t = seconds since it began, layer space) into Position keys at the comp frame rate from the
current time; `puppet.recordOptions {speed, smoothing, useDraftDeformation, showMesh}`.
* `effect.plugins.load {"path": "x.wasm"}` / `effect.plugins.list`: WebAssembly effect plug-ins
([plugins.md](plugins.md)), then `effect.apply` by id like a built-in.
## CLI
Each invocation runs a headless engine with no window. It opens the demo project unless you pass
+19 -5
View File
@@ -35,9 +35,10 @@ cpal or muda. Everything in L0 to L4, the egui UI and the web app also build for
| L3 | `export` | Render queue encoding: H.264, ProRes, WebM (VP9 + alpha, Opus), PNG/JPEG/TIFF/EXR sequences, GIF, WAV/AIFF |
| L3 | `gpu` | The GPU compositor and GPU effects on wgpu compute shaders (Metal, Vulkan, Direct3D 12, WebGPU), checked against the CPU renderer |
| L3 | `lottie` | Lottie JSON / dotLottie import and export (layers, precomps, eased and spatial keyframes, shapes, masks, mattes) with a warnings list for what Lottie cannot express |
| L4 | `engine` | `Session`: project, undo history, editor state, the command registry and menus |
| L4 | `script` | Scripting: JavaScript (boa) with an After Effects-style object model (`app.project`, comps, layers, properties, render queue) whose edits run engine commands; AE match names |
| L4 | `host` | A fully wired `Session` (media, expressions, scripting, exporter) for the frontends |
| L3 | `plugin` | Effect plug-ins: loads sandboxed WebAssembly effect modules (plug-in API v1; the wasmi interpreter behind the `wasm` feature, on for native hosts) into the effect registry ([plugins.md](plugins.md)) |
| L4 | `engine` | `Session`: project, branching undo history, editor state, the command registry and menus, the ScriptUI window model (`scriptui`) |
| L4 | `script` | Scripting: JavaScript (boa) with an After Effects-style object model (`app.project`, comps, layers, properties, render queue) and ScriptUI (dialogs, palettes, dockable panels; script-host threads so dialogs block `show()`), whose edits run engine commands; AE match names |
| L4 | `host` | A fully wired `Session` (media, expressions, scripting, WebAssembly plug-ins, exporter) for the frontends |
| L5 | `ui-egui` | The desktop interface: docking, panels, viewer, timeline, graph editor, dialogs, control channel |
| L5 | `automation` | The MCP server, headless or bridged to the running app |
| L6 | apps `effectcraft`, `effectcraft-cli`, `effectcraft-web` | Desktop app; command-line tool (render, exec, get/set, MCP); the browser app (wasm32, [web.md](web.md)) |
@@ -229,6 +230,13 @@ tracks them with pyramidal Lucas–Kanade and fits the chosen model (position
RANSAC; the motion moves the Mask Path's vertices and tangents and is keyed per frame. **Mask
Interpolation** (`mask.interpolate`) uses `effectcraft-path`'s smart interpolation (arc-length
vertex insertion, shape-context matching, rigid in-betweens) to key in-between shapes.
The two **Face Tracking** methods (`effectcraft_track::face`) fit a face inside the mask instead:
a skin colour model from the first frame, the face outline along rays from the skin component's
centre (an area-moment ellipse gives centre, size and roll), facial features as the non-skin
components inside it, and an active-shape point distribution model trained on synthetic face
shapes (our own generator, no external weights) that fills and checks the landmarks. The outline
keys the Mask Path; Detailed Features keys a Face Track Points effect, and
`track.extractFaceMeasurements` derives a keyed Face Measurements effect (and copies its keys).
**Warp Stabilizer** (`effects::warp_stab`, `engine::warp`): `warp.analyze` renders the layer's
input to the effect (`Renderer::layer_input`: source, masks and the effects above it) for every
@@ -237,7 +245,10 @@ translation / similarity / homography fits) as JSON in the effect's hidden Analy
with a key of the layer's source, In/Out, start, stretch and Time Remap. `Session::edit` clears
analyses whose key no longer matches and queues them; the desktop app re-analyses them in the
background. Rendering derives the stabilization plan (smoothed camera path or No Motion, framing,
auto-scale; cached per analysis and settings) and warps each frame; Synthesize Edges fills the
auto-scale; cached per analysis and settings) and warps each frame. Subspace Warp also stores long
feature trajectories in the analysis, smooths them in a low-rank subspace (Liu et al. 2011) and
warps each frame with a content-preserving mesh (Liu et al. 2009) fitted to the smoothed
positions, rendered as a per-pixel mesh lookup; Synthesize Edges fills the
borders from neighbouring frames read with `EffectHost::self_at`. The effect lives in the
effects crate, hence the `effects → track` edge.
@@ -268,7 +279,10 @@ the essential matrix (normalised 8-point in RANSAC) or a planar homography decom
resection and triangulation; a sparse Levenberg–Marquardt bundle adjustment (Schur complement over
the points, Huber loss) with the focal length fixed, shared, or per frame; a log-spaced and
golden-section focal search for Fixed Angle of View / Variable Zoom; a rotation-only model for tripod
pans, chosen by Auto Detect when it explains the tracks as well. Tracks and solve are stored as JSON
pans, chosen by Auto Detect when it explains the tracks as well; with Solve Lens Distortion (or
Detailed Analysis) radial distortion `k1, k2` is adjusted jointly in a final bundle adjustment on
the raw tracks and the solve repeated on tracks undistorted with that estimate; Undistort Footage
renders the input through the inverse model. Tracks and solve are stored as JSON
in hidden effect parameters with keys: changing the layer's frames clears both, changing Shot Type,
Angle of View, Solve Method or deleting points only re-solves. The solve's canonical frame maps to
comp space so the first frame's camera is the default comp camera (or so a chosen ground plane is
+6 -4
View File
@@ -3,11 +3,11 @@
<!-- Generated from the effect registry (crates/effects/src/catalog.rs); do not edit by hand.
Regenerate: UPDATE_DOCS=1 cargo test -p effectcraft-effects --lib effects_doc_is_current -->
EffectCraft ships 304 effects with 2446 parameters in total, grouped into the same categories as the Effects & Presets panel. Every effect is our own implementation, written from public behaviour descriptions and standard image-processing literature. Parameter names, order, popup options, units, defaults and ranges follow the reference application so that projects, expressions and muscle memory carry over.
EffectCraft ships 306 effects with 2488 parameters in total, grouped into the same categories as the Effects & Presets panel. Every effect is our own implementation, written from public behaviour descriptions and standard image-processing literature. Parameter names, order, popup options, units, defaults and ranges follow the reference application so that projects, expressions and muscle memory carry over.
- **GPU**: 59 effects also run on the GPU compositor with identical results.
- **32**: 304 effects process 32-bit float (HDR, overbright) pixels without clamping.
- **Status**: 262 implemented in full, 42 partial (what is missing is listed).
- **32**: 306 effects process 32-bit float (HDR, overbright) pixels without clamping.
- **Status**: 264 implemented in full, 42 partial (what is missing is listed).
Effects are addressed by id (`ec.<category>.<name>`) or display name in commands, scripts and the control channel, and every parameter by its id (`effects/#1/blurriness`).
@@ -302,7 +302,7 @@ Effects are addressed by id (`ec.<category>.<name>`) or display name in commands
| Effect | Id | Params | GPU | 32 | Status |
|---|---|---:|:-:|:-:|---|
| 3D Camera Tracker | `ec.perspective.cameratracker` | 10 | | 32 | Implemented |
| 3D Camera Tracker | `ec.perspective.cameratracker` | 12 | | 32 | Implemented |
| 3D Glasses | `ec.perspective.3dglasses` | 8 | | 32 | Implemented |
| Bevel Alpha | `ec.perspective.bevelalpha` | 4 | | 32 | Implemented |
| Bevel Edges | `ec.perspective.beveledges` | 4 | | 32 | Implemented |
@@ -416,6 +416,8 @@ Effects are addressed by id (`ec.<category>.<name>`) or display name in commands
| CC Overbrights | `ec.utility.ccoverbrights` | 2 | | 32 | Implemented |
| Cineon Converter | `ec.utility.cineon` | 7 | | 32 | Implemented |
| Color Profile Converter | `ec.utility.colorprofileconverter` | 7 | | 32 | Implemented |
| Face Measurements | `ec.utility.facemeasurements` | 14 | | 32 | Implemented |
| Face Track Points | `ec.utility.facetrackpoints` | 26 | | 32 | Implemented |
| Grow Bounds | `ec.utility.growbounds` | 1 | | 32 | Implemented |
| HDR Compander | `ec.utility.hdrcompander` | 3 | | 32 | Implemented |
| HDR Highlight Compression | `ec.utility.hdrcompression` | 1 | | 32 | Implemented |
+32 -17
View File
@@ -2,27 +2,42 @@
How close EffectCraft is to After Effects 2026, feature by feature, and how much work is left.
## Current status (audit at commit `fa26ad9`, 2 October 2026, late)
## Current status (audit at commit `978e8d7`, 3 October 2026)
| Measure | Value |
|---|---|
| **Feature parity, weighted by tier** (P0 ×3, P1 ×2, P2 ×1) | **≈ 93%** |
| Unweighted | ≈ 91% |
| P0 / P1 / P2 | ≈ 96% / 90% / 65% |
| Features done / partial / missing | 41 / 50 / 1 of 92 (missing: a plug-in API) |
| **Effects** | **298 of 298** After Effects 2026 effects exist (some still simplified) |
| Disabled menu entries left | 26 commands (31 menu items) |
| Remaining work | ≈ 50–55 agent-hours at the pace measured so far (≈ 150 by the original conservative audit scale) |
| **Wall-clock estimate** | **≈ 11–14 hours** with five agents in parallel; ≈ 6–8 hours for 100% of P0 + P1 |
| **Feature parity, weighted by tier** (P0 ×3, P1 ×2, P2 ×1) | **≈ 94%** counting every partial feature as half done; ≈ 98% using per-feature fractions |
| Unweighted | ≈ 94% (half-credit) / 98% (fractions) |
| P0 / P1 / P2 | ≈ 96% / 92% / 85% (half-credit); 99% / 97% / 96% (fractions) |
| Features done / partial / missing | 80 / 12 / 0 of 92 |
| **Effects** | **298 of 298** After Effects 2026 effects exist; 42 still simplified ([effects.md](effects.md)); 58 run on the GPU |
| Disabled menu entries left | 0 (the stub list is empty) |
| Remaining work | ≈ 19–22 agent-hours at the pace measured so far (≈ 58 on the conservative audit scale) |
| **Wall-clock estimate** | **≈ 4–5 hours** with five agents in parallel; ≈ 3 hours for 100% of P0 + P1 |
Partial features: effect implementations (42 simplified effects), the Preview panel's settings
(range, play from, skip, frame rate, full screen), GPU coverage (Advanced 3D, more GPU effects),
the web app (no threads for viewer frames and Roto Brush), the Learn panel and Align to Selection /
Distribute, Advanced 3D (iris shapes, collapsed precomps, extruded strokes), ScriptUI resource
strings and `onDraw`, Essential Graphics font and scale controls, and the learned-model quality of
Roto Brush and face tracking (both are classical). Menu items still absent: Save Frame As ▸
Photoshop Layers / ProEXR, Open in Essential Graphics, Vanishing Point import, Watch Folder.
### Previous audit (commit `fa26ad9`, 2 October 2026, late): ≈ 93%
By area: Layers 98%, Output 95%, Compositions 95%, Automation 96%, Paint 95%, Text 95%, Import 96%,
Animation 94%, Masks 94%, Preview 94%, Interface 93%, Shapes 93%, 3D 91%, Audio 90%, Project 90%,
Tracking 88%, Effects 85%, Web 65%.
Tracking 96%, Effects 85%, Web 65%.
What is left, in priority order: the web app's depth (threads, storage, audio, non-blocking
renders); stroke taper/wave and multi-segment dashes; variable mask feather points; camera iris/bokeh and focus-link commands; Render Queue field render, crop/resize and templates;
approximated effects (Subspace Warp, Key Cleaner) and Liquify's viewer brush; text in imported
PDF/AI files and Illustrator procset EPS; more codecs; 59 preferences not yet wired; ScriptUI; face tracking; and the "better than After Effects" items (a plug-in API, branching history; GPU particles landed in M12.7).
renders); stroke taper/wave and multi-segment dashes; variable mask feather points; camera iris/bokeh and focus-link commands;
approximated effects (Key Cleaner) and Liquify's viewer brush; text in imported
PDF/AI files and Illustrator procset EPS; and more codecs. Face tracking and a true Subspace Warp
landed in M13.3. ScriptUI (dialogs that block `show()`, palettes, dockable ScriptUI panels, File ▸ Scripts install
and sample scripts), puppet pin recording and the remaining "better than After Effects" items — a
versioned effect plug-in API with sandboxed WebAssembly plug-ins ([plugins.md](plugins.md)),
branching undo history in the History panel, and a bit-identical rendering test across runs and
thread counts — landed in M13.1 (GPU particles landed in M12.7). Render Queue field render, crop/resize, templates and logs, and VP9 inter frames landed in M10.2.
The sections below are the original audit (morning of 2 October, ≈ 64%) and its updates, kept for
history.
@@ -107,13 +122,13 @@ the disk cache in the formats wave; the 3D Camera Tracker in M12.6.)
| Area | Weighted parity | Remaining (agent-hours) | Biggest gaps |
|---|---|---|---|
| Layers | 88% | 4.8 | frame blending, collapse transformations, slip edit |
| Output | 90% | 3.0 | multiple output modules, pre-render (WebM with VP9 alpha + Opus and WAV/AIFF audio-only landed: VP9 is intra-only, Opus CELT-only) |
| Output | 97% | 0.8 | Render Settings complete (field render + 3:2 pulldown, effects/solo/guide/depth/blending/blur overrides, time sampling, storage overflow), Output Module crop/ROI/resize, alpha modes, post-render actions, PCM formats, templates with defaults, render logs, Notify (M10.2); WebM VP9 key + inter frames with motion search, loop filter and rate control. Left: Opus is CELT-only, Photoshop sequence output, overflow for movies only checks at file creation |
| Audio | 85% | 0.5 | audio to keyframes |
| Import | 94% | 1.0 | text and images inside PDF/AI files, Illustrator EPS relying on Adobe procsets, smart-object perspective/warps, PSD 3D layers (PDF / PDF-compatible AI / EPS vector footage with Continuously Rasterize, layered composition import and Create Shapes from Vector Layer, and PSD smart objects with embedded files landed in M13.2; PSD as footage/composition/retain layer sizes, SVG footage earlier) |
| Automation | ≈ 90% | 0.8 | scripting covers the documented core object model (AUT-2, M14.4: `app`, project items, comps, layers, properties and keyframes, text documents, markers, render queue, Script Console, `effectcraft-cli script`, MCP `run_script`); still missing: ScriptUI panels/dialogs, `.jsxbin`, sockets |
| Automation | ≈ 97% | 0.3 | `.jsxbin`, sockets, ScriptUI resource strings and custom `onDraw` graphics (the core object model landed in M14.4; ScriptUI windows/dialogs/dockable panels with `scriptui.*` agent commands, File ▸ Scripts install + sample scripts, and the effect plug-in API (EFF-6, WebAssembly) landed in M13.1) |
| Shapes | ≈ 80% | 1.5 | Lottie can't carry stroke taper/wave (stroke Taper and Wave, Dash 2/Gap 2/Dash 3/Gap 3 and radial-gradient Highlight Length/Angle landed in M13.5; pen tool for shape paths and vertex editing in M6.5) |
| Compositions | ≈ 80% | 3.5 | Mocha-style planar tracks for templates, Essential Graphics' rare controls (font menus, mirrored properties) (the marker dialog, Composition Flowchart, Essential Graphics with master properties, `.ectemplate` templates and Responsive Design — Time landed: CMP-6, CMP-7) |
| Animation | 67% | 10.0 | puppet, Wiggler/Smoother/Motion Sketch (motion-path handles and the graph editor transform box landed in M5.8; keyframe colour labels and Select Keyframe Label Group, Graph Editor snapping to markers / layer ends in M13.5) |
| Animation | 70% | 9.0 | puppet depth beyond pins and recording (puppet pin recording with Record Options landed in M13.1), Wiggler/Smoother/Motion Sketch (motion-path handles and the graph editor transform box landed in M5.8; keyframe colour labels and Select Keyframe Label Group, Graph Editor snapping to markers / layer ends in M13.5) |
| Text | ≈ 94% | 0.5 | no extruded strokes, variable-axis animation changes outlines but not advances (OpenType features — stylistic sets, discretionary ligatures, contextual / stylistic alternates, swash, titling, ordinals, fractions, figure styles, true small caps / all small caps and superior / inferior glyphs with faux fallback — per character with the Character panel's OpenType popup and `text.fontFeatures` landed in M13.2; vertical Roman / Tate-Chu-Yoko, forced LTR paragraphs, caret on animated and path text, Variable Font Axes and Lottie style runs landed in M13.5; extruded, bevelled text in M7.6; per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API in M9.9–M9.10) |
| Web | 85% | 1.0 | viewer frames and Roto Brush propagation still on the page's thread; GPU effects in the browser (browser storage, Web Audio, Web Worker renders/analyses, WebGPU viewer and offline install landed in M15.2) |
| 3D | 85% | 6.5 | multi-view layouts, iris shapes in Advanced 3D's depth of field, collapsed 3D precomps in Advanced 3D, the Extended Viewer for Advanced 3D comps (Classic 3D Extended Viewer landed in M13.5 UI completion); collapsed precomps of another size seen through the parent's camera render (fixed in M13.2); stereo rigs, orbit nulls, lights controlled by the camera, cameras/lights from glTF models, environment backgrounds, Advanced 3D motion blur, blend modes and track mattes landed in M7.7; Classic 3D iris-shaped bokeh with highlights, progressive depth of field on tilted layers and the focus-link commands landed in M13.5; Advanced 3D (glTF/OBJ models, primitives, extruded text and shapes, PBR, image-based light, shadow maps, GPU rasteriser) in M7.4–M7.6 |
@@ -122,7 +137,7 @@ the disk cache in the formats wave; the 3D Camera Tracker in M12.6.)
| Project | ≈ 68% | 5.5 | auto-save, folder moves, OCIO displays beyond the built-in tone map (Color Engine with OCIO/ACES working spaces, HDR compand/tone mapping, Rec. 2100 PQ/HLG output, Feet + Frames, display colour management, Simulate Output and the locked viewer landed in M7.7; proxies and Interpret Footage fields / pixel aspect / alpha guess landed: PRJ-8, PRJ-3) |
| Masks & roto | 74% | 5.0 | Roto Brush's learned (3.0) segmentation model (variable-width mask feather points with the Mask Feather tool landed in M13.5; mask tracking and Mask Interpolation landed in M6.6; Roto Brush & Refine Edge with graph-cut segmentation, flow propagation, edge matting, decontamination and Freeze in M6.7) |
| Preview | 66% | 4.0 | GPU bokeh depth of field, wireframes and Advanced 3D compositing (Classic 3D runs and adjustment layers composite on the GPU since M12.7; persistent disk cache with the blue cache bar landed; region of interest, snapshots, exposure and Fast Previews landed in M0.13) |
| Tracking | ≈ 85% | 2.5 | face tracking, Subspace Warp's mesh warp, lens distortion in the camera solve (Rolling Shutter Repair landed in M9.11; point tracker, mask tracking, Warp Stabilizer and the 3D Camera Tracker landed in M6.x / M12.5 / M12.6) |
| Tracking | ≈ 96% | 0.3 | face tracking is a classical (skin model + feature components + shape model) fitter, not a learned detector: profile views and occluded faces are weak; Rolling Shutter Ripple is approximated by Subspace Warp's mesh density (face tracking (Outline Only / Detailed Features with Face Track Points and Extract & Copy Face Measurements), Subspace Warp's content-preserving mesh warp on subspace-smoothed trajectories, and radial lens distortion (k1, k2) in the camera bundle adjustment with Undistort Footage landed in M13.3; Rolling Shutter Repair in M9.11; point tracker, mask tracking, Warp Stabilizer and the 3D Camera Tracker in M6.x / M12.5 / M12.6) |
| Paint | 0% | 4.0 | Brush, Clone Stamp, Eraser |
## Effects still missing
+126
View File
@@ -0,0 +1,126 @@
# Effect plug-ins
EffectCraft loads third-party effects through a small, versioned plug-in API (**API version 1**).
A plug-in effect behaves exactly like a built-in one: it is listed in Effects & Presets (and the
Effect menu) under its category, applied by id (`effect.apply {"effect": "org.example.posterize"}`)
or display name, its parameters are ordinary properties (keyframes, expressions, Effect Controls,
`prop.set`), and it is saved in projects by id.
There are two ways to write one:
| | WebAssembly plug-in | Rust plug-in |
|---|---|---|
| What | a `.wasm` (or `.wat`) module | a type implementing `effectcraft_effects::plugin::EffectPlugin` |
| Loaded by | Effect ▸ Load Effect Plug-in…, `effect.plugins.load {path \| folder}`, or the `Plug-ins` folder next to the settings at start-up | `register_plugin(Arc::new(MyEffect))` in an app that links EffectCraft's crates |
| Sandbox | yes: no imports (no files, clock or network), fuel-limited, deterministic floats | none (it is your code) |
| Platforms | desktop and CLI (the wasmi interpreter, crate feature `effectcraft-plugin/wasm`); not the web build | everywhere |
`effect.plugins.list` lists what is loaded: `{api, wasm, plugins: [{id, name, category, version,
author, source, params}]}`.
## The contract
* **Pixels** are premultiplied RGBA `f32` (0–1; values may exceed 1 in 32 bpc projects), row
major, processed **in place**. The buffer is the layer's (after masks and the effects above),
possibly at preview resolution (`scale` = buffer pixels per layer pixel) and padded.
* **Parameters** arrive flattened to `f64`s in the order they are declared: slider, angle,
checkbox (0 / 1) and popup (0-based index) take one value, point two (already in **buffer
pixels**), colour four (RGBA 0–1).
* **Time** is layer time in seconds.
* Rendering must be **deterministic** and may run on several threads at once (a WebAssembly
plug-in gets one instance per thread).
* A failed render (an error code, a trap, running out of fuel) leaves the frame unchanged.
## The manifest
Every plug-in describes itself with a JSON manifest:
```json
{
"api": 1,
"id": "org.effectcraft.example.posterize-bands",
"name": "Posterize Bands",
"category": "Stylize",
"version": "1.0.0",
"author": "EffectCraft contributors",
"description": "Quantizes luminance into tinted bands.",
"params": [
{"id": "levels", "name": "Levels", "type": "slider", "default": 4, "min": 2, "max": 64, "sliderMax": 16, "decimals": 0},
{"id": "mix", "name": "Tint Amount", "type": "slider", "default": 50, "min": 0, "max": 100},
{"id": "tint", "name": "Tint", "type": "color", "default": [1.0, 0.55, 0.2, 1.0]},
{"id": "invert", "name": "Invert Bands", "type": "checkbox", "default": false}
]
}
```
* `id`: stable, reverse-domain; letters, digits, `.`, `_`, `-`. `ec.` is reserved for built-ins.
Ids must be unique (a second plug-in with a taken id is refused).
* `category`: one of the Effects & Presets categories (`Blur & Sharpen`, `Stylize`, …) or a new
one, which appears after the built-in categories.
* Parameter types: `slider` (`default`, `min`, `max`, optional `sliderMin`, `sliderMax`,
`decimals`), `angle` (`default` degrees), `checkbox` (`default` bool), `popup` (`options`,
`default` index), `point` (`default` as fractions of the layer size, `[0.5, 0.5]` = centre),
`color` (`default` RGBA 0–1). Parameter ids are letters, digits and `_`.
## WebAssembly ABI (v1)
A module with **no imports** that exports:
| export | signature | |
|---|---|---|
| `memory` | memory | linear memory shared with the host |
| `ec_api_version` | `() -> i32` | must return `1` |
| `ec_manifest_ptr`, `ec_manifest_len` | `() -> i32` | the UTF-8 JSON manifest in memory |
| `ec_alloc` | `(bytes: i32) -> i32` | a buffer of at least `bytes` bytes, 8-byte aligned; it may reuse the previous one |
| `ec_render` | `(pixels: i32, width: i32, height: i32, params: i32, nparams: i32, time: f64, scale: f64) -> i32` | process the pixels in place; return 0 on success |
For each frame the host calls `ec_alloc` once for a buffer holding the parameters (`nparams`
little-endian `f64`s) followed by the pixels (`width × height × 4` little-endian `f32`s), fills it,
calls `ec_render`, and reads the pixels back. Each call gets a fuel budget proportional to the
frame size; a module that loops forever stops with an error instead of hanging the app.
### Example
[`examples/plugins/posterize-bands`](../examples/plugins/posterize-bands) is a complete plug-in in
plain Rust (no dependencies):
```sh
cd examples/plugins/posterize-bands
cargo build --release --target wasm32-unknown-unknown
# then, in EffectCraft: Effect ▸ Load Effect Plug-in… → target/wasm32-unknown-unknown/release/posterize_bands.wasm
# or headless:
effectcraft-cli exec effect.plugins.load '{"path": "target/wasm32-unknown-unknown/release/posterize_bands.wasm"}'
```
Any language that compiles to WebAssembly works (C, Zig, AssemblyScript…) as long as the module
has no imports. `crates/plugin/src/tests.rs` has a plug-in written directly in WebAssembly text.
## Rust plug-ins
```rust
use std::sync::Arc;
use effectcraft_effects::plugin::*;
struct Swap(PluginManifest);
impl EffectPlugin for Swap {
fn manifest(&self) -> &PluginManifest { &self.0 }
fn render(&self, frame: &mut PluginFrame, params: &PluginParams, _time: f64) -> Result<(), String> {
if params.b("on") {
for p in frame.pixels.iter_mut() { p.swap(0, 2); }
}
Ok(())
}
}
register_plugin(Arc::new(Swap(manifest)))?; // before the UI builds its menus
```
`PluginParams` reads values by parameter id (`f`, `b`, `point`, `color`). At most 64 plug-in
effects can be registered per process.
## Compatibility
The API version is checked when a plug-in loads: a manifest or module for another version is
refused with a message. Within a version the contract above does not change; additions (new
parameter types, new optional exports) bump the version.
-2
View File
@@ -207,5 +207,3 @@ every request and reply, a render through an in-process offload).
- Cancelling an analysis drops its partial result (the worker is terminated).
- Stored media count against the origin's storage quota; `effectcraft.listStored()` shows usage,
`removeStored` frees it (there is no storage manager in the UI yet).
- `engine.execute` rejects the spread Render Settings / Output Module keys of `renderQueue.add`
(`resolution`, `timeSpan`…) through its strict parameter check; use `ui.menu.invoke` as above.