Merge main into IO-2/SHP-5/OUT-2/PRV-2 work

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas
2026-10-02 22:39:47 -04:00
co-authored by Claude Opus 5.5
104 changed files with 18573 additions and 73 deletions
+14
View File
@@ -49,6 +49,7 @@ The server speaks JSON-RPC 2.0 over stdio, one message per line, and supports MC
|---|---|
| `list_commands {filter?, enabled_only?}` | Discover command ids, their param docs, and whether each can run now. |
| `execute_command {command, params?}` | Run any command (undoable). |
| `run_script {code, name?}` | Run JavaScript with the After Effects-style scripting object model (`app.project`, `comp.layers.addText(…)`, `layer.property("ADBE Transform Group").property("ADBE Position").setValueAtTime(…)`…). Returns `{ok, result, output, error: {message, line, column}}`; edits are undoable. |
| `get_project` / `get_comp {comp?}` | Project items, comp settings and layers. |
| `get_layer {layer, comp?, time?, flat?}` | A layer's property tree. Every node has a `path`. |
| `get_property {layer, path, comp?, time?}` | Value at a time, keyframes and expression. |
@@ -191,6 +192,19 @@ effectcraft-cli exec file.exportLottie '{"comp":"Main","path":"main.json","inclu
effectcraft-cli exec file.importLottie '{"path":"anim.json"}' main.ecproj --save
```
Scripts written against After Effects' documented scripting API run with `script`:
```sh
effectcraft-cli script build.jsx --save-as main.ecproj # empty project unless one is given
effectcraft-cli script --eval 'app.project.item(1).numLayers' main.ecproj --json
effectcraft-cli script tweak.jsx --bridge 9877 # against the live app
```
`writeLn`/`$.writeln`/`alert` output is printed, then the value of the last expression. A script
error exits with status 1 and `file:line:col: message`. Scripts may read files only in the
project's folder and may not write files or use the network unless the user turns on Preferences ▸
Scripting & Expressions ▸ Allow Scripts to Write Files and Access Network.
`file.exportLottie` returns `{path, bytes, warnings}`: the warnings list every feature Lottie
cannot express (most effects, cameras and lights, layer styles, audio, video footage…), so an
agent can check what a player will not show. A `.lottie` path writes a dotLottie archive.
+56 -3
View File
@@ -26,6 +26,7 @@ cpal or muda. Everything in L0 to L4, the egui UI and the web app also build for
| L2 | `text` | Fonts, shaping, layout, per-glyph geometry, text animators and selectors |
| L2 | `effects` | The effect registry (241 effects) and their CPU implementations |
| L2 | `svg` | SVG import (W3C SVG 1.1/2 static subset: shapes, paths, transforms, `use`, CSS, gradients) and rasterisation at any scale |
| L2 | `model` | 3D models for Advanced 3D: glTF 2.0 (`.gltf`/`.glb`) and OBJ/MTL import (meshes, PBR metallic-roughness materials and textures, node hierarchy, skins, animations), parametric primitives, extruded/bevelled outline meshes and polygon triangulation |
| L2 | `track` | Motion tracking: feature/search region point tracking (pyramid normalized cross-correlation, Lucas–Kanade sub-pixel refinement), confidence, homography/affine/similarity solves |
| L3 | `render` | Evaluation and compositing: sources, masks, effects, transforms, 3D, motion blur, mattes, blending, layer cache, audio mixdown |
| L3 | `media` | Footage decoding (FilmCraft's pure-Rust codecs), image sequences, Photoshop and SVG stills, frame cache |
@@ -34,13 +35,15 @@ cpal or muda. Everything in L0 to L4, the egui UI and the web app also build for
| L3 | `gpu` | The GPU compositor and GPU effects on wgpu compute shaders (Metal, Vulkan, Direct3D 12, WebGPU), checked against the CPU renderer |
| L3 | `lottie` | Lottie JSON / dotLottie import and export (layers, precomps, eased and spatial keyframes, shapes, masks, mattes) with a warnings list for what Lottie cannot express |
| L4 | `engine` | `Session`: project, undo history, editor state, the command registry and menus |
| L4 | `host` | A fully wired `Session` (media, expressions, exporter) for the frontends |
| L4 | `script` | Scripting: JavaScript (boa) with an After Effects-style object model (`app.project`, comps, layers, properties, render queue) whose edits run engine commands; AE match names |
| L4 | `host` | A fully wired `Session` (media, expressions, scripting, exporter) for the frontends |
| L5 | `ui-egui` | The desktop interface: docking, panels, viewer, timeline, graph editor, dialogs, control channel |
| L5 | `automation` | The MCP server, headless or bridged to the running app |
| L6 | apps `effectcraft`, `effectcraft-cli`, `effectcraft-web` | Desktop app; command-line tool (render, exec, get/set, MCP); the browser app (wasm32, [web.md](web.md)) |
Allowed same-layer edges: `path → keyframe, raster`, `text → path`, `effects → project, text, path, track`,
`media / expr / export / gpu → render`, `export → media`, `lottie → format`, `host → engine`.
`media / expr / export / gpu → render`, `export → media`, `lottie → format`, `host → engine, script`,
`script → engine`.
## 2. Time
@@ -72,7 +75,35 @@ the **source** (solid, footage frame, text, shape contents, or a nested comp), a
then **effects** in order, then **layer styles**, then the **transform** into comp space (with motion blur sub-samples
when enabled), the **track matte**, and finally **blends** into the accumulator. Adjustment layers
apply their effects to the accumulator. Runs of 3D layers are composited per pixel through the
active camera, with lights, shadows and depth of field.
active camera, with lights, shadows and depth of field (Classic 3D), or rasterised as meshes by
the Advanced 3D renderer (below).
**Advanced 3D** (Composition Settings ▸ 3D Renderer ▸ Advanced 3D, `comp.renderer`;
`crates/render/src/three_d/adv`, `crates/model`). A run of 3D layers becomes one scene of
world-space triangles: 3D model layers (glTF 2.0 / OBJ footage, mapped from +Y-up model units
into the layer by Geometry Options ▸ Model Scale; skins and animation clips played along layer
time), primitive layers (Layer ▸ New ▸ Cube/Sphere/Plane/Torus/Cone/Cylinder, parametric in
Geometry Options), text and shape layers with Geometry Options ▸ Extrusion Depth (glyph and
fill outlines flattened, triangulated with holes and extruded with Angular/Concave/Convex
bevels, per character so per-character 3D carries over), and a textured card for every other
3D layer. Shading is physically based in linear light (glTF metallic-roughness; Cook–Torrance
GGX specular, Smith–Schlick geometry, Schlick Fresnel, Lambert diffuse, after Karis 2013):
parallel, spot and point lights with After Effects' falloffs, ambient lights, image-based
light from an Environment light whose Source (or the comp's Environment Layer, Layer ▸
Environment Layer) is an equirectangular image (cosine irradiance map; roughness-indexed mip
chain with the split-sum environment BRDF fit), and percentage-closer-filtered shadow maps
(spot: perspective, parallel: orthographic over the scene, point: six cube faces). Cards keep
Material Options (Accepts Lights/Shadows, Casts Shadows On/Only, Ambient, Diffuse, Specular);
models and primitives add Base Color, Metallic, Roughness, Emissive. Rasterisation runs at 2×2
supersampling with a reversed-Z depth buffer; opaque triangles resolve visibility first, the
transparent ones are sorted back to front and blended. The software rasteriser
(`adv::raster`) is the reference and the headless/CI path; `effectcraft-gpu` implements
`Accelerator::raster_3d` with a wgpu render pipeline (`advanced3d.wgsl`) that repeats the
shading step for step, and tests compare the two. Resolve, depth of field (a gather blur by
each pixel's circle of confusion from the camera's Depth of Field settings) and the conversion
back to the working encoding run on the CPU for both. Not yet: motion blur, track mattes and
blend modes inside an Advanced 3D run, collapsed 3D precomps (drawn flattened), morph targets,
extruded strokes.
A **layer cache** keeps each layer's finished pixels (source, masks and effects) keyed by a hash of
its evaluated inputs, excluding the transform. Static and transform-only layers render once;
@@ -194,6 +225,28 @@ Expressions are JavaScript, run by boa, with After Effects' object model (`thisC
`time`, `value`, `wiggle`, `loopOut`, vector maths on arrays, and so on). A syntax error keeps the
text, disables the expression and shows a warning, like After Effects.
## 5a. Scripting
`effectcraft-script` runs JavaScript (boa, a fresh context per run; the Script Console keeps one)
with an object model written from the behaviour the After Effects Scripting Guide documents:
`app`, `Project`, `ItemCollection`, `CompItem`/`FootageItem`/`FolderItem`, `LayerCollection`,
`AVLayer`/`TextLayer`/`ShapeLayer`/`CameraLayer`/`LightLayer`, `PropertyGroup`/`Property`
(values, keyframes, eases, expressions), `TextDocument`, `Shape`, `KeyframeEase`, `MarkerValue`,
`RenderQueue`/`RenderQueueItem`/`OutputModule`, `File`/`Folder` and the enums. The model lives in
`prelude.js`; it reads the session through `__query` (JSON snapshots of items, layers, property
nodes, values and keys) and edits only through `__exec`, which runs engine commands, so every
scripted edit is undoable and identical to the UI's. While a script runs the session is moved
into a thread-local the natives borrow (nothing borrowed enters the JS heap); the comp a call
targets is made active for that command only. `app.beginUndoGroup`/`endUndoGroup` fold the undo
steps in between into one named step. Match names (`ADBE Transform Group`, `ADBE Position`,
`ADBE Gaussian Blur 2`, `ADBE Vector Shape - Rect`…) come from a table in `matchnames.rs`; effect
parameters are `<effect match name>-0001…`. `File` reads are limited to the project's folder, and
writes and the network are refused, unless Preferences ▸ Scripting & Expressions ▸ Allow Scripts
to Write Files and Access Network is on. Entry points: the `script.run` command
(`Session::script`, set by the host), File ▸ Scripts ▸ Run Script File… (`.jsx`/`.js`; `.json`
command scripts still run as steps), Window ▸ Script Console, `effectcraft-cli script` and the
MCP `run_script` tool.
## 6. Commands, the UI seam and automation
Every action is a command: an id, a label, a menu path, a shortcut, a parameter description,
+27 -15
View File
@@ -15,7 +15,7 @@ rendered, does not count as done.
| P1 (professional daily use) | ≈ 38% |
| P2 (long tail) | ≈ 17% |
| Features done / partial / missing | 18 / 60 / 14 of 92 |
| **Effects** | **257 of 298** After Effects 2026 effects by name (86%); ≈ 70% allowing for simplified implementations |
| **Effects** | **297 of 298** After Effects 2026 effects by name (99.7%, after M9.11 and Warp Stabilizer; was 257); only the 3D Camera Tracker is left |
| Remaining work | ≈ **167 agent-hours** (139 h of features + ≈ 20% for 1:1 polish against After Effects) |
| Wall-clock estimate | ≈ **42–50 hours** with five agents working in parallel and one integrating; ≈ 31–35 h for P0 + P1 only |
@@ -74,10 +74,10 @@ library groundwork for mask tracking and Warp Stabilizer.
| Wall clock, five parallel agents | 42–50 h | **≈ 8–10 h at today's measured pace; ≈ 25 h by the audit's conservative figures** |
What is left is concentrated in large systems: Roto Brush and Refine Edge, the 3D camera tracker,
Advanced 3D (models, PBR, image-based lighting), the Warp Stabilizer and mask-tracking UI on top of
the new tracking library, a JavaScript scripting object model,
the missing effect categories (3D Channel, Immersive Video, OCIO), native macOS menus, and
wasm threads.
the Warp Stabilizer and mask-tracking UI on top of
the new tracking library,
native macOS menus, and wasm threads. (The missing effect categories, 3D Channel, Immersive Video
and OCIO, landed in M9.11; the scripting object model in M14.4.)
## By area
@@ -87,29 +87,41 @@ wasm threads.
| Output | 90% | 3.0 | multiple output modules, pre-render (WebM with VP9 alpha + Opus and WAV/AIFF audio-only landed: VP9 is intra-only, Opus CELT-only) |
| Audio | 85% | 0.5 | audio to keyframes |
| Import | 88% | 2.0 | AI/EPS/PDF vector footage, PSD smart objects and 3D layers (PSD as footage/composition/retain layer sizes, SVG footage and Create Shapes from Vector Layer landed) |
| Automation | 76% | 3.3 | a JavaScript scripting object model |
| Automation | ≈ 90% | 0.8 | scripting covers the documented core object model (AUT-2, M14.4: `app`, project items, comps, layers, properties and keyframes, text documents, markers, render queue, Script Console, `effectcraft-cli script`, MCP `run_script`); still missing: ScriptUI panels/dialogs, `.jsxbin`, sockets |
| Shapes | 68% | 3.9 | taper and wave strokes (pen tool for shape paths and vertex editing landed in M6.5) |
| Compositions | 67% | 7.4 | marker dialog, flowchart, Essential Graphics |
| Animation | 65% | 10.4 | puppet, Wiggler/Smoother/Motion Sketch (motion-path handles and the graph editor transform box landed in M5.8) |
| Text | ≈ 85% | 1.5 | vertical type is basic (upright characters, no tate-chu-yoko), no OpenType feature panel, no text on 3D bevels; per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API landed (M9.9–M9.10) |
| Text | ≈ 86% | 1.3 | vertical type is basic (upright characters, no tate-chu-yoko), no OpenType feature panel, no extruded strokes (extruded, bevelled text landed in M7.6); per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API landed (M9.9–M9.10) |
| Web | 60% | 2.5 | threads, browser storage, audio |
| 3D | 58% | 20.5 | Advanced 3D (models, PBR, image-based light), 3D camera tracker, multi-view layouts |
| Effects | 58% | 13.6 | Effect Controls widgets (angle dial, point crosshair, eyedropper, curves), GPU versions of more effects, 41 missing effects |
| 3D | 72% | 12.0 | 3D camera tracker, multi-view layouts, Advanced 3D motion blur and blend modes, cameras/lights from models; Advanced 3D (glTF/OBJ models, primitives, extruded text and shapes, PBR, image-based light, shadow maps, GPU rasteriser) landed in M7.4–M7.6 |
| Effects | ≈ 75% | 5.0 | GPU versions of more effects and the 3D Camera Tracker (every other After Effects effect exists since M9.11 and M12.5) |
| Interface | 55% | 10.4 | drag-to-dock and floating panels, preferences, native macOS menus (viewer rulers/snapping/channels/snapshots landed in M0.13) |
| Project | 53% | 8.1 | 8/16/32-bit pipeline and colour management, auto-save, proxies, folder moves |
| Masks & roto | 52% | 10.0 | Roto Brush, variable-width mask feather points (mask tracking and Mask Interpolation landed in M6.6) |
| Preview | 62% | 5.0 | GPU 3D and adjustment layers (persistent disk cache with the blue cache bar landed; region of interest, snapshots, exposure and Fast Previews landed in M0.13) |
| Tracking | ≈ 70% | 4.0 | 3D camera tracker, face tracking, Subspace Warp's mesh warp and rolling-shutter repair (point tracker, mask tracking and Warp Stabilizer landed in M6.x / M12.5) |
| Tracking | ≈ 70% | 4.0 | 3D camera tracker, face tracking and Subspace Warp's mesh warp (Rolling Shutter Repair landed in M9.11; point tracker, mask tracking and Warp Stabilizer landed in M6.x / M12.5) |
| Paint | 0% | 4.0 | Brush, Clone Stamp, Eraser |
## Effects still missing
Immersive Video (all 12), 3D Channel (all 8, including Cryptomatte), Color Stabilizer and the five
OCIO transforms, CC Flo Motion, Liquify, Rolling Shutter Repair, Compressor,
Distortion and Gate (audio), Camera-Shake Deblur, CC Radial Blur, CC Hair, Particle Playground,
Keylight (our Screen Key stands in), Color Profile Converter and the 3D Camera Tracker. Boris FX
The 3D Camera Tracker (built on the tracking library by other work). Boris FX
Mocha and Cineware are third-party and not counted.
### Added in M9.11 (39 effects)
| Category | Effects | Notes |
|---|---|---|
| 3D Channel | 3D Channel Extract, Cryptomatte, Depth Matte, Depth of Field, EXtractoR, Fog 3D, ID Matte, IDentifier | Read auxiliary channels: multi-layer OpenEXR (any `layer.channel`, Cryptomatte manifests from the header) or the compositor's own depth / layer-ID / normal / UV / Cryptomatte pass for precomps of 3D comps. |
| Immersive Video | VR Blur, VR Chromatic Aberrations, VR Color Gradients, VR Converter, VR De-Noise, VR Digital Glitch, VR Fractal Noise, VR Glow, VR Plane to Sphere, VR Rotate Sphere, VR Sharpen, VR Sphere To Plane | Seam-aware equirectangular processing (wrap-around, pole continuation, latitude-scaled filters), mono / over-under / side-by-side layouts; VR Converter between equirectangular, cube maps (4:3, 3:2, 6:1), sphere map and 2D. |
| Color Correction | OCIO CDL / Color Space / Display / File / Look Transform, Color Stabilizer | Built-in minimal OCIO-style config (ACES2065-1, ACEScg, ACEScct, sRGB, Rec.709, Rec.2020, Display P3, linear variants, XYZ) from published matrices and transfer functions; `.cube` / `.3dl` / `.csp` LUTs and ASC `.cc` / `.ccc` / `.cdl`. Custom `.ocio` config files are not read yet. |
| Distort | CC Flo Motion, Liquify, Rolling Shutter Repair | Liquify strokes are effect data replayed into a displacement mesh (`liquify.stroke` / `liquify.clear` commands; no interactive viewer brush yet). Rolling Shutter Repair uses the KLT tracker. |
| Blur & Sharpen | Camera-Shake Deblur, CC Radial Blur | Deblur substitutes aligned patches from sharper neighbouring frames. |
| Audio | Compressor, Distortion, Gate | Applied in the mixdown like the other audio effects. |
| Simulation | CC Hair, Particle Playground | Particle Playground: cannon, grid, layer exploder, layer map, gravity, repel, wall, persistent property mapper (no Particle Exploder, text particles or ephemeral mapper yet). |
| Keying | Key Light | The full Keylight 1.2 control set under a generic name ("Keylight" is a vendor trademark); `lookup("Keylight (1.2)")` finds it. |
| Utility | Color Profile Converter | Our colour spaces and ACES; rendering intents (perceptual gamut compression, relative / absolute colorimetric, saturation). |
| Obsolete | mocha shape | Mocha's export format is not public: reads a documented JSON shape format instead. |
## Highest-value gaps, in order
1. ~~On-canvas text editing and per-character styles~~ (landed: M9.9–M9.10).
@@ -132,4 +144,4 @@ Mocha and Cineware are third-party and not counted.
The engine underneath (keyframes, expressions, shape operators, text animators, Classic 3D, all
38 blend modes, the render queue) is deep. Most of what is missing is interactive tooling in the
viewer, settings that are stored but not yet rendered, about 69 menu entries that are still
disabled, and the large systems: tracking, puppet, paint, roto, Advanced 3D and GPU rendering.
disabled, and the large systems: tracking, puppet, paint and roto (Advanced 3D landed in M7.4–M7.6).