mirror of
https://github.com/storytold/effectcraft.git
synced 2026-10-26 19:18:14 -06:00
Merge main into IO-2/SHP-5/OUT-2/PRV-2 work
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -49,6 +49,7 @@ The server speaks JSON-RPC 2.0 over stdio, one message per line, and supports MC
|
||||
|---|---|
|
||||
| `list_commands {filter?, enabled_only?}` | Discover command ids, their param docs, and whether each can run now. |
|
||||
| `execute_command {command, params?}` | Run any command (undoable). |
|
||||
| `run_script {code, name?}` | Run JavaScript with the After Effects-style scripting object model (`app.project`, `comp.layers.addText(…)`, `layer.property("ADBE Transform Group").property("ADBE Position").setValueAtTime(…)`…). Returns `{ok, result, output, error: {message, line, column}}`; edits are undoable. |
|
||||
| `get_project` / `get_comp {comp?}` | Project items, comp settings and layers. |
|
||||
| `get_layer {layer, comp?, time?, flat?}` | A layer's property tree. Every node has a `path`. |
|
||||
| `get_property {layer, path, comp?, time?}` | Value at a time, keyframes and expression. |
|
||||
@@ -191,6 +192,19 @@ effectcraft-cli exec file.exportLottie '{"comp":"Main","path":"main.json","inclu
|
||||
effectcraft-cli exec file.importLottie '{"path":"anim.json"}' main.ecproj --save
|
||||
```
|
||||
|
||||
Scripts written against After Effects' documented scripting API run with `script`:
|
||||
|
||||
```sh
|
||||
effectcraft-cli script build.jsx --save-as main.ecproj # empty project unless one is given
|
||||
effectcraft-cli script --eval 'app.project.item(1).numLayers' main.ecproj --json
|
||||
effectcraft-cli script tweak.jsx --bridge 9877 # against the live app
|
||||
```
|
||||
|
||||
`writeLn`/`$.writeln`/`alert` output is printed, then the value of the last expression. A script
|
||||
error exits with status 1 and `file:line:col: message`. Scripts may read files only in the
|
||||
project's folder and may not write files or use the network unless the user turns on Preferences ▸
|
||||
Scripting & Expressions ▸ Allow Scripts to Write Files and Access Network.
|
||||
|
||||
`file.exportLottie` returns `{path, bytes, warnings}`: the warnings list every feature Lottie
|
||||
cannot express (most effects, cameras and lights, layer styles, audio, video footage…), so an
|
||||
agent can check what a player will not show. A `.lottie` path writes a dotLottie archive.
|
||||
|
||||
+56
-3
@@ -26,6 +26,7 @@ cpal or muda. Everything in L0 to L4, the egui UI and the web app also build for
|
||||
| L2 | `text` | Fonts, shaping, layout, per-glyph geometry, text animators and selectors |
|
||||
| L2 | `effects` | The effect registry (241 effects) and their CPU implementations |
|
||||
| L2 | `svg` | SVG import (W3C SVG 1.1/2 static subset: shapes, paths, transforms, `use`, CSS, gradients) and rasterisation at any scale |
|
||||
| L2 | `model` | 3D models for Advanced 3D: glTF 2.0 (`.gltf`/`.glb`) and OBJ/MTL import (meshes, PBR metallic-roughness materials and textures, node hierarchy, skins, animations), parametric primitives, extruded/bevelled outline meshes and polygon triangulation |
|
||||
| L2 | `track` | Motion tracking: feature/search region point tracking (pyramid normalized cross-correlation, Lucas–Kanade sub-pixel refinement), confidence, homography/affine/similarity solves |
|
||||
| L3 | `render` | Evaluation and compositing: sources, masks, effects, transforms, 3D, motion blur, mattes, blending, layer cache, audio mixdown |
|
||||
| L3 | `media` | Footage decoding (FilmCraft's pure-Rust codecs), image sequences, Photoshop and SVG stills, frame cache |
|
||||
@@ -34,13 +35,15 @@ cpal or muda. Everything in L0 to L4, the egui UI and the web app also build for
|
||||
| L3 | `gpu` | The GPU compositor and GPU effects on wgpu compute shaders (Metal, Vulkan, Direct3D 12, WebGPU), checked against the CPU renderer |
|
||||
| L3 | `lottie` | Lottie JSON / dotLottie import and export (layers, precomps, eased and spatial keyframes, shapes, masks, mattes) with a warnings list for what Lottie cannot express |
|
||||
| L4 | `engine` | `Session`: project, undo history, editor state, the command registry and menus |
|
||||
| L4 | `host` | A fully wired `Session` (media, expressions, exporter) for the frontends |
|
||||
| L4 | `script` | Scripting: JavaScript (boa) with an After Effects-style object model (`app.project`, comps, layers, properties, render queue) whose edits run engine commands; AE match names |
|
||||
| L4 | `host` | A fully wired `Session` (media, expressions, scripting, exporter) for the frontends |
|
||||
| L5 | `ui-egui` | The desktop interface: docking, panels, viewer, timeline, graph editor, dialogs, control channel |
|
||||
| L5 | `automation` | The MCP server, headless or bridged to the running app |
|
||||
| L6 | apps `effectcraft`, `effectcraft-cli`, `effectcraft-web` | Desktop app; command-line tool (render, exec, get/set, MCP); the browser app (wasm32, [web.md](web.md)) |
|
||||
|
||||
Allowed same-layer edges: `path → keyframe, raster`, `text → path`, `effects → project, text, path, track`,
|
||||
`media / expr / export / gpu → render`, `export → media`, `lottie → format`, `host → engine`.
|
||||
`media / expr / export / gpu → render`, `export → media`, `lottie → format`, `host → engine, script`,
|
||||
`script → engine`.
|
||||
|
||||
## 2. Time
|
||||
|
||||
@@ -72,7 +75,35 @@ the **source** (solid, footage frame, text, shape contents, or a nested comp), a
|
||||
then **effects** in order, then **layer styles**, then the **transform** into comp space (with motion blur sub-samples
|
||||
when enabled), the **track matte**, and finally **blends** into the accumulator. Adjustment layers
|
||||
apply their effects to the accumulator. Runs of 3D layers are composited per pixel through the
|
||||
active camera, with lights, shadows and depth of field.
|
||||
active camera, with lights, shadows and depth of field (Classic 3D), or rasterised as meshes by
|
||||
the Advanced 3D renderer (below).
|
||||
|
||||
**Advanced 3D** (Composition Settings ▸ 3D Renderer ▸ Advanced 3D, `comp.renderer`;
|
||||
`crates/render/src/three_d/adv`, `crates/model`). A run of 3D layers becomes one scene of
|
||||
world-space triangles: 3D model layers (glTF 2.0 / OBJ footage, mapped from +Y-up model units
|
||||
into the layer by Geometry Options ▸ Model Scale; skins and animation clips played along layer
|
||||
time), primitive layers (Layer ▸ New ▸ Cube/Sphere/Plane/Torus/Cone/Cylinder, parametric in
|
||||
Geometry Options), text and shape layers with Geometry Options ▸ Extrusion Depth (glyph and
|
||||
fill outlines flattened, triangulated with holes and extruded with Angular/Concave/Convex
|
||||
bevels, per character so per-character 3D carries over), and a textured card for every other
|
||||
3D layer. Shading is physically based in linear light (glTF metallic-roughness; Cook–Torrance
|
||||
GGX specular, Smith–Schlick geometry, Schlick Fresnel, Lambert diffuse, after Karis 2013):
|
||||
parallel, spot and point lights with After Effects' falloffs, ambient lights, image-based
|
||||
light from an Environment light whose Source (or the comp's Environment Layer, Layer ▸
|
||||
Environment Layer) is an equirectangular image (cosine irradiance map; roughness-indexed mip
|
||||
chain with the split-sum environment BRDF fit), and percentage-closer-filtered shadow maps
|
||||
(spot: perspective, parallel: orthographic over the scene, point: six cube faces). Cards keep
|
||||
Material Options (Accepts Lights/Shadows, Casts Shadows On/Only, Ambient, Diffuse, Specular);
|
||||
models and primitives add Base Color, Metallic, Roughness, Emissive. Rasterisation runs at 2×2
|
||||
supersampling with a reversed-Z depth buffer; opaque triangles resolve visibility first, the
|
||||
transparent ones are sorted back to front and blended. The software rasteriser
|
||||
(`adv::raster`) is the reference and the headless/CI path; `effectcraft-gpu` implements
|
||||
`Accelerator::raster_3d` with a wgpu render pipeline (`advanced3d.wgsl`) that repeats the
|
||||
shading step for step, and tests compare the two. Resolve, depth of field (a gather blur by
|
||||
each pixel's circle of confusion from the camera's Depth of Field settings) and the conversion
|
||||
back to the working encoding run on the CPU for both. Not yet: motion blur, track mattes and
|
||||
blend modes inside an Advanced 3D run, collapsed 3D precomps (drawn flattened), morph targets,
|
||||
extruded strokes.
|
||||
|
||||
A **layer cache** keeps each layer's finished pixels (source, masks and effects) keyed by a hash of
|
||||
its evaluated inputs, excluding the transform. Static and transform-only layers render once;
|
||||
@@ -194,6 +225,28 @@ Expressions are JavaScript, run by boa, with After Effects' object model (`thisC
|
||||
`time`, `value`, `wiggle`, `loopOut`, vector maths on arrays, and so on). A syntax error keeps the
|
||||
text, disables the expression and shows a warning, like After Effects.
|
||||
|
||||
## 5a. Scripting
|
||||
|
||||
`effectcraft-script` runs JavaScript (boa, a fresh context per run; the Script Console keeps one)
|
||||
with an object model written from the behaviour the After Effects Scripting Guide documents:
|
||||
`app`, `Project`, `ItemCollection`, `CompItem`/`FootageItem`/`FolderItem`, `LayerCollection`,
|
||||
`AVLayer`/`TextLayer`/`ShapeLayer`/`CameraLayer`/`LightLayer`, `PropertyGroup`/`Property`
|
||||
(values, keyframes, eases, expressions), `TextDocument`, `Shape`, `KeyframeEase`, `MarkerValue`,
|
||||
`RenderQueue`/`RenderQueueItem`/`OutputModule`, `File`/`Folder` and the enums. The model lives in
|
||||
`prelude.js`; it reads the session through `__query` (JSON snapshots of items, layers, property
|
||||
nodes, values and keys) and edits only through `__exec`, which runs engine commands, so every
|
||||
scripted edit is undoable and identical to the UI's. While a script runs the session is moved
|
||||
into a thread-local the natives borrow (nothing borrowed enters the JS heap); the comp a call
|
||||
targets is made active for that command only. `app.beginUndoGroup`/`endUndoGroup` fold the undo
|
||||
steps in between into one named step. Match names (`ADBE Transform Group`, `ADBE Position`,
|
||||
`ADBE Gaussian Blur 2`, `ADBE Vector Shape - Rect`…) come from a table in `matchnames.rs`; effect
|
||||
parameters are `<effect match name>-0001…`. `File` reads are limited to the project's folder, and
|
||||
writes and the network are refused, unless Preferences ▸ Scripting & Expressions ▸ Allow Scripts
|
||||
to Write Files and Access Network is on. Entry points: the `script.run` command
|
||||
(`Session::script`, set by the host), File ▸ Scripts ▸ Run Script File… (`.jsx`/`.js`; `.json`
|
||||
command scripts still run as steps), Window ▸ Script Console, `effectcraft-cli script` and the
|
||||
MCP `run_script` tool.
|
||||
|
||||
## 6. Commands, the UI seam and automation
|
||||
|
||||
Every action is a command: an id, a label, a menu path, a shortcut, a parameter description,
|
||||
|
||||
+27
-15
@@ -15,7 +15,7 @@ rendered, does not count as done.
|
||||
| P1 (professional daily use) | ≈ 38% |
|
||||
| P2 (long tail) | ≈ 17% |
|
||||
| Features done / partial / missing | 18 / 60 / 14 of 92 |
|
||||
| **Effects** | **257 of 298** After Effects 2026 effects by name (86%); ≈ 70% allowing for simplified implementations |
|
||||
| **Effects** | **297 of 298** After Effects 2026 effects by name (99.7%, after M9.11 and Warp Stabilizer; was 257); only the 3D Camera Tracker is left |
|
||||
| Remaining work | ≈ **167 agent-hours** (139 h of features + ≈ 20% for 1:1 polish against After Effects) |
|
||||
| Wall-clock estimate | ≈ **42–50 hours** with five agents working in parallel and one integrating; ≈ 31–35 h for P0 + P1 only |
|
||||
|
||||
@@ -74,10 +74,10 @@ library groundwork for mask tracking and Warp Stabilizer.
|
||||
| Wall clock, five parallel agents | 42–50 h | **≈ 8–10 h at today's measured pace; ≈ 25 h by the audit's conservative figures** |
|
||||
|
||||
What is left is concentrated in large systems: Roto Brush and Refine Edge, the 3D camera tracker,
|
||||
Advanced 3D (models, PBR, image-based lighting), the Warp Stabilizer and mask-tracking UI on top of
|
||||
the new tracking library, a JavaScript scripting object model,
|
||||
the missing effect categories (3D Channel, Immersive Video, OCIO), native macOS menus, and
|
||||
wasm threads.
|
||||
the Warp Stabilizer and mask-tracking UI on top of
|
||||
the new tracking library,
|
||||
native macOS menus, and wasm threads. (The missing effect categories, 3D Channel, Immersive Video
|
||||
and OCIO, landed in M9.11; the scripting object model in M14.4.)
|
||||
|
||||
## By area
|
||||
|
||||
@@ -87,29 +87,41 @@ wasm threads.
|
||||
| Output | 90% | 3.0 | multiple output modules, pre-render (WebM with VP9 alpha + Opus and WAV/AIFF audio-only landed: VP9 is intra-only, Opus CELT-only) |
|
||||
| Audio | 85% | 0.5 | audio to keyframes |
|
||||
| Import | 88% | 2.0 | AI/EPS/PDF vector footage, PSD smart objects and 3D layers (PSD as footage/composition/retain layer sizes, SVG footage and Create Shapes from Vector Layer landed) |
|
||||
| Automation | 76% | 3.3 | a JavaScript scripting object model |
|
||||
| Automation | ≈ 90% | 0.8 | scripting covers the documented core object model (AUT-2, M14.4: `app`, project items, comps, layers, properties and keyframes, text documents, markers, render queue, Script Console, `effectcraft-cli script`, MCP `run_script`); still missing: ScriptUI panels/dialogs, `.jsxbin`, sockets |
|
||||
| Shapes | 68% | 3.9 | taper and wave strokes (pen tool for shape paths and vertex editing landed in M6.5) |
|
||||
| Compositions | 67% | 7.4 | marker dialog, flowchart, Essential Graphics |
|
||||
| Animation | 65% | 10.4 | puppet, Wiggler/Smoother/Motion Sketch (motion-path handles and the graph editor transform box landed in M5.8) |
|
||||
| Text | ≈ 85% | 1.5 | vertical type is basic (upright characters, no tate-chu-yoko), no OpenType feature panel, no text on 3D bevels; per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API landed (M9.9–M9.10) |
|
||||
| Text | ≈ 86% | 1.3 | vertical type is basic (upright characters, no tate-chu-yoko), no OpenType feature panel, no extruded strokes (extruded, bevelled text landed in M7.6); per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API landed (M9.9–M9.10) |
|
||||
| Web | 60% | 2.5 | threads, browser storage, audio |
|
||||
| 3D | 58% | 20.5 | Advanced 3D (models, PBR, image-based light), 3D camera tracker, multi-view layouts |
|
||||
| Effects | 58% | 13.6 | Effect Controls widgets (angle dial, point crosshair, eyedropper, curves), GPU versions of more effects, 41 missing effects |
|
||||
| 3D | 72% | 12.0 | 3D camera tracker, multi-view layouts, Advanced 3D motion blur and blend modes, cameras/lights from models; Advanced 3D (glTF/OBJ models, primitives, extruded text and shapes, PBR, image-based light, shadow maps, GPU rasteriser) landed in M7.4–M7.6 |
|
||||
| Effects | ≈ 75% | 5.0 | GPU versions of more effects and the 3D Camera Tracker (every other After Effects effect exists since M9.11 and M12.5) |
|
||||
| Interface | 55% | 10.4 | drag-to-dock and floating panels, preferences, native macOS menus (viewer rulers/snapping/channels/snapshots landed in M0.13) |
|
||||
| Project | 53% | 8.1 | 8/16/32-bit pipeline and colour management, auto-save, proxies, folder moves |
|
||||
| Masks & roto | 52% | 10.0 | Roto Brush, variable-width mask feather points (mask tracking and Mask Interpolation landed in M6.6) |
|
||||
| Preview | 62% | 5.0 | GPU 3D and adjustment layers (persistent disk cache with the blue cache bar landed; region of interest, snapshots, exposure and Fast Previews landed in M0.13) |
|
||||
| Tracking | ≈ 70% | 4.0 | 3D camera tracker, face tracking, Subspace Warp's mesh warp and rolling-shutter repair (point tracker, mask tracking and Warp Stabilizer landed in M6.x / M12.5) |
|
||||
| Tracking | ≈ 70% | 4.0 | 3D camera tracker, face tracking and Subspace Warp's mesh warp (Rolling Shutter Repair landed in M9.11; point tracker, mask tracking and Warp Stabilizer landed in M6.x / M12.5) |
|
||||
| Paint | 0% | 4.0 | Brush, Clone Stamp, Eraser |
|
||||
|
||||
## Effects still missing
|
||||
|
||||
Immersive Video (all 12), 3D Channel (all 8, including Cryptomatte), Color Stabilizer and the five
|
||||
OCIO transforms, CC Flo Motion, Liquify, Rolling Shutter Repair, Compressor,
|
||||
Distortion and Gate (audio), Camera-Shake Deblur, CC Radial Blur, CC Hair, Particle Playground,
|
||||
Keylight (our Screen Key stands in), Color Profile Converter and the 3D Camera Tracker. Boris FX
|
||||
The 3D Camera Tracker (built on the tracking library by other work). Boris FX
|
||||
Mocha and Cineware are third-party and not counted.
|
||||
|
||||
### Added in M9.11 (39 effects)
|
||||
|
||||
| Category | Effects | Notes |
|
||||
|---|---|---|
|
||||
| 3D Channel | 3D Channel Extract, Cryptomatte, Depth Matte, Depth of Field, EXtractoR, Fog 3D, ID Matte, IDentifier | Read auxiliary channels: multi-layer OpenEXR (any `layer.channel`, Cryptomatte manifests from the header) or the compositor's own depth / layer-ID / normal / UV / Cryptomatte pass for precomps of 3D comps. |
|
||||
| Immersive Video | VR Blur, VR Chromatic Aberrations, VR Color Gradients, VR Converter, VR De-Noise, VR Digital Glitch, VR Fractal Noise, VR Glow, VR Plane to Sphere, VR Rotate Sphere, VR Sharpen, VR Sphere To Plane | Seam-aware equirectangular processing (wrap-around, pole continuation, latitude-scaled filters), mono / over-under / side-by-side layouts; VR Converter between equirectangular, cube maps (4:3, 3:2, 6:1), sphere map and 2D. |
|
||||
| Color Correction | OCIO CDL / Color Space / Display / File / Look Transform, Color Stabilizer | Built-in minimal OCIO-style config (ACES2065-1, ACEScg, ACEScct, sRGB, Rec.709, Rec.2020, Display P3, linear variants, XYZ) from published matrices and transfer functions; `.cube` / `.3dl` / `.csp` LUTs and ASC `.cc` / `.ccc` / `.cdl`. Custom `.ocio` config files are not read yet. |
|
||||
| Distort | CC Flo Motion, Liquify, Rolling Shutter Repair | Liquify strokes are effect data replayed into a displacement mesh (`liquify.stroke` / `liquify.clear` commands; no interactive viewer brush yet). Rolling Shutter Repair uses the KLT tracker. |
|
||||
| Blur & Sharpen | Camera-Shake Deblur, CC Radial Blur | Deblur substitutes aligned patches from sharper neighbouring frames. |
|
||||
| Audio | Compressor, Distortion, Gate | Applied in the mixdown like the other audio effects. |
|
||||
| Simulation | CC Hair, Particle Playground | Particle Playground: cannon, grid, layer exploder, layer map, gravity, repel, wall, persistent property mapper (no Particle Exploder, text particles or ephemeral mapper yet). |
|
||||
| Keying | Key Light | The full Keylight 1.2 control set under a generic name ("Keylight" is a vendor trademark); `lookup("Keylight (1.2)")` finds it. |
|
||||
| Utility | Color Profile Converter | Our colour spaces and ACES; rendering intents (perceptual gamut compression, relative / absolute colorimetric, saturation). |
|
||||
| Obsolete | mocha shape | Mocha's export format is not public: reads a documented JSON shape format instead. |
|
||||
|
||||
## Highest-value gaps, in order
|
||||
|
||||
1. ~~On-canvas text editing and per-character styles~~ (landed: M9.9–M9.10).
|
||||
@@ -132,4 +144,4 @@ Mocha and Cineware are third-party and not counted.
|
||||
The engine underneath (keyframes, expressions, shape operators, text animators, Classic 3D, all
|
||||
38 blend modes, the render queue) is deep. Most of what is missing is interactive tooling in the
|
||||
viewer, settings that are stored but not yet rendered, about 69 menu entries that are still
|
||||
disabled, and the large systems: tracking, puppet, paint, roto, Advanced 3D and GPU rendering.
|
||||
disabled, and the large systems: tracking, puppet, paint and roto (Advanced 3D landed in M7.4–M7.6).
|
||||
|
||||
Reference in New Issue
Block a user