mirror of
https://github.com/storytold/effectcraft.git
synced 2026-10-31 21:48:20 -06:00
Merge branch 'main' into worktree-agent-a758de7c5b6efcc3d
# Conflicts: # crates/effects/src/lib.rs # crates/render/src/lib.rs # docs/parity.md
This commit is contained in:
@@ -100,6 +100,52 @@ effectcraft-cli run clip.ecproj track.motion '{"layer":"#2"}' \
|
||||
track.analyze '{"wait":true}' track.apply '{}' --save
|
||||
```
|
||||
|
||||
### Mask tracking and Mask Interpolation
|
||||
|
||||
`track.mask` follows the pixels inside a mask and keys its Mask Path on every frame (vertices and
|
||||
tangents move with the fitted motion). `method` is `position`, `positionScale`,
|
||||
`positionScaleRotation` (the default, also kept as the Tracker panel's Method), `positionScaleRotationSkew`
|
||||
or `perspective`; `direction` is `forward|backward|frameForward|frameBackward` from the current
|
||||
time (or `start` / `end` in seconds). Like `track.analyze` it runs in the background in the app
|
||||
(progress in `track.status` under `mask`, `track.stop` cancels and keeps what was tracked) and is
|
||||
one undo step; `"wait": true` blocks.
|
||||
|
||||
`mask.interpolate` (Window ▸ Mask Interpolation ▸ Apply) adds in-between Mask Path keys between
|
||||
each pair of selected Mask Path keys (or the existing keys at `times`, in seconds), giving both
|
||||
ends the same vertex count with a matched correspondence. Options: `keyframeRate` (number or
|
||||
`"auto"` = the comp rate), `keyframeFields`, `linearVertexPaths`, `bendingResistance`, `quality`,
|
||||
`addVertices` (number or `false`) with `addVerticesUnit` `pixels|total|percent`, `matchingMethod`
|
||||
`auto|curve|polyline`, `oneToOne`, `firstVerticesMatch`. `mask.interpolationOptions` sets the
|
||||
panel's defaults.
|
||||
|
||||
```sh
|
||||
effectcraft-cli exec track.mask '{"layer":"#2","mask":1,"method":"perspective","wait":true}' clip.ecproj --save
|
||||
effectcraft-cli exec mask.interpolate '{"layer":"#2","mask":"Mask 1","times":[0,2],"addVertices":10}' clip.ecproj --save --json
|
||||
```
|
||||
|
||||
### Warp Stabilizer
|
||||
|
||||
Effect ▸ Distort ▸ Warp Stabilizer (`effect.apply {"effect":"Warp Stabilizer"}`), Animation ▸ Warp
|
||||
Stabilizer VFX and the Tracker panel's button (`track.warpStabilizer`, which applies the effect and
|
||||
starts analysing) stabilize a layer. `warp.analyze {layer?, effect?, wait?}` analyses every frame
|
||||
between the layer's In and Out points in the background ("Analyzing in background (step 1 of 2)",
|
||||
then "Stabilizing..."; `warp.status` reports `progress`, `banner`, whether the effect is
|
||||
`analyzed`, and the current frame's `warp` matrix, `autoScale` and `crop`); `warp.cancel` stops it
|
||||
without writing anything. The analysis is stored in the effect (saved with the project) and is
|
||||
one undo step; trimming, slipping, time-remapping or replacing the layer's source clears it and
|
||||
the app re-analyses automatically (headless: run `warp.analyze` again). Settings are ordinary
|
||||
properties under the instance's groups, e.g. `effects/#1/stabilization/result` (0 Smooth Motion,
|
||||
1 No Motion), `effects/#1/stabilization/smoothness`, `effects/#1/stabilization/method`,
|
||||
`effects/#1/borders/framing` (0 Stabilize Only … 3 Stabilize, Synthesize Edges),
|
||||
`effects/#1/borders/autoScale/maximumScale`, `effects/#1/advanced/showTrackPoints`.
|
||||
|
||||
```sh
|
||||
effectcraft-cli run shaky.ecproj track.warpStabilizer '{"layer":"#1","wait":true}' \
|
||||
prop.set '{"layer":"#1","path":"effects/#1/stabilization/result","value":1}' --save
|
||||
effectcraft-cli exec warp.analyze '{"layer":"#1","wait":true}' shaky.ecproj --save --json
|
||||
effectcraft-cli exec warp.status '{"layer":"#1"}' shaky.ecproj --json
|
||||
```
|
||||
|
||||
### Text: styles and editing
|
||||
|
||||
Source Text holds character style runs and per-paragraph settings. `layer.setText` changes the
|
||||
|
||||
+48
-1
@@ -22,6 +22,7 @@ cpal or muda. Everything in L0 to L4, the egui UI and the web app also build for
|
||||
| L2 | `project` | The document: items, compositions, layers, the property tree, render queue model, `.ecproj` serde |
|
||||
| L2 | `text` | Fonts, shaping, layout, per-glyph geometry, text animators and selectors |
|
||||
| L2 | `effects` | The effect registry (241 effects) and their CPU implementations |
|
||||
| L2 | `model` | 3D models for Advanced 3D: glTF 2.0 (`.gltf`/`.glb`) and OBJ/MTL import (meshes, PBR metallic-roughness materials and textures, node hierarchy, skins, animations), parametric primitives, extruded/bevelled outline meshes and polygon triangulation |
|
||||
| L2 | `track` | Motion tracking: feature/search region point tracking (pyramid normalized cross-correlation, Lucas–Kanade sub-pixel refinement), confidence, homography/affine/similarity solves |
|
||||
| L3 | `render` | Evaluation and compositing: sources, masks, effects, transforms, 3D, motion blur, mattes, blending, layer cache, audio mixdown |
|
||||
| L3 | `media` | Footage decoding (FilmCraft's pure-Rust codecs), image sequences, frame cache |
|
||||
@@ -68,7 +69,35 @@ the **source** (solid, footage frame, text, shape contents, or a nested comp), a
|
||||
then **effects** in order, then **layer styles**, then the **transform** into comp space (with motion blur sub-samples
|
||||
when enabled), the **track matte**, and finally **blends** into the accumulator. Adjustment layers
|
||||
apply their effects to the accumulator. Runs of 3D layers are composited per pixel through the
|
||||
active camera, with lights, shadows and depth of field.
|
||||
active camera, with lights, shadows and depth of field (Classic 3D), or rasterised as meshes by
|
||||
the Advanced 3D renderer (below).
|
||||
|
||||
**Advanced 3D** (Composition Settings ▸ 3D Renderer ▸ Advanced 3D, `comp.renderer`;
|
||||
`crates/render/src/three_d/adv`, `crates/model`). A run of 3D layers becomes one scene of
|
||||
world-space triangles: 3D model layers (glTF 2.0 / OBJ footage, mapped from +Y-up model units
|
||||
into the layer by Geometry Options ▸ Model Scale; skins and animation clips played along layer
|
||||
time), primitive layers (Layer ▸ New ▸ Cube/Sphere/Plane/Torus/Cone/Cylinder, parametric in
|
||||
Geometry Options), text and shape layers with Geometry Options ▸ Extrusion Depth (glyph and
|
||||
fill outlines flattened, triangulated with holes and extruded with Angular/Concave/Convex
|
||||
bevels, per character so per-character 3D carries over), and a textured card for every other
|
||||
3D layer. Shading is physically based in linear light (glTF metallic-roughness; Cook–Torrance
|
||||
GGX specular, Smith–Schlick geometry, Schlick Fresnel, Lambert diffuse, after Karis 2013):
|
||||
parallel, spot and point lights with After Effects' falloffs, ambient lights, image-based
|
||||
light from an Environment light whose Source (or the comp's Environment Layer, Layer ▸
|
||||
Environment Layer) is an equirectangular image (cosine irradiance map; roughness-indexed mip
|
||||
chain with the split-sum environment BRDF fit), and percentage-closer-filtered shadow maps
|
||||
(spot: perspective, parallel: orthographic over the scene, point: six cube faces). Cards keep
|
||||
Material Options (Accepts Lights/Shadows, Casts Shadows On/Only, Ambient, Diffuse, Specular);
|
||||
models and primitives add Base Color, Metallic, Roughness, Emissive. Rasterisation runs at 2×2
|
||||
supersampling with a reversed-Z depth buffer; opaque triangles resolve visibility first, the
|
||||
transparent ones are sorted back to front and blended. The software rasteriser
|
||||
(`adv::raster`) is the reference and the headless/CI path; `effectcraft-gpu` implements
|
||||
`Accelerator::raster_3d` with a wgpu render pipeline (`advanced3d.wgsl`) that repeats the
|
||||
shading step for step, and tests compare the two. Resolve, depth of field (a gather blur by
|
||||
each pixel's circle of confusion from the camera's Depth of Field settings) and the conversion
|
||||
back to the working encoding run on the CPU for both. Not yet: motion blur, track mattes and
|
||||
blend modes inside an Advanced 3D run, collapsed 3D precomps (drawn flattened), morph targets,
|
||||
extruded strokes.
|
||||
|
||||
A **layer cache** keeps each layer's finished pixels (source, masks and effects) keyed by a hash of
|
||||
its evaluated inputs, excluding the transform. Static and transform-only layers render once;
|
||||
@@ -158,6 +187,24 @@ queue; blocking with `wait`), and `effectcraft-track` matches each point in para
|
||||
covers an analysis. `track.apply` keys the target's Position/Rotation/Scale, the tracked layer's
|
||||
Anchor Point and Position (Stabilize), or a Corner Pin effect (Parallel / Perspective).
|
||||
|
||||
**Mask tracking** (`track.mask`, `engine::mask_track`) runs the same kind of background job over
|
||||
the layer's source frames: `effectcraft-track`'s mask tracker detects features inside the mask,
|
||||
tracks them with pyramidal Lucas–Kanade and fits the chosen model (position … perspective) with
|
||||
RANSAC; the motion moves the Mask Path's vertices and tangents and is keyed per frame. **Mask
|
||||
Interpolation** (`mask.interpolate`) uses `effectcraft-path`'s smart interpolation (arc-length
|
||||
vertex insertion, shape-context matching, rigid in-betweens) to key in-between shapes.
|
||||
|
||||
**Warp Stabilizer** (`effects::warp_stab`, `engine::warp`): `warp.analyze` renders the layer's
|
||||
input to the effect (`Renderer::layer_input`: source, masks and the effects above it) for every
|
||||
frame on a background thread and stores `effectcraft-track`'s `WarpAnalysis` (per-frame
|
||||
translation / similarity / homography fits) as JSON in the effect's hidden Analysis parameter,
|
||||
with a key of the layer's source, In/Out, start, stretch and Time Remap. `Session::edit` clears
|
||||
analyses whose key no longer matches and queues them; the desktop app re-analyses them in the
|
||||
background. Rendering derives the stabilization plan (smoothed camera path or No Motion, framing,
|
||||
auto-scale; cached per analysis and settings) and warps each frame; Synthesize Edges fills the
|
||||
borders from neighbouring frames read with `EffectHost::self_at`. The effect lives in the
|
||||
effects crate, hence the `effects → track` edge.
|
||||
|
||||
## 5. Expressions
|
||||
|
||||
Expressions are JavaScript, run by boa, with After Effects' object model (`thisComp`, `thisLayer`,
|
||||
|
||||
+9
-9
@@ -15,7 +15,7 @@ rendered, does not count as done.
|
||||
| P1 (professional daily use) | ≈ 38% |
|
||||
| P2 (long tail) | ≈ 17% |
|
||||
| Features done / partial / missing | 18 / 60 / 14 of 92 |
|
||||
| **Effects** | **296 of 298** After Effects 2026 effects by name (99%, after M9.11; was 257); the two left are Warp Stabilizer and the 3D Camera Tracker |
|
||||
| **Effects** | **297 of 298** After Effects 2026 effects by name (99.7%, after M9.11 and Warp Stabilizer; was 257); only the 3D Camera Tracker is left |
|
||||
| Remaining work | ≈ **167 agent-hours** (139 h of features + ≈ 20% for 1:1 polish against After Effects) |
|
||||
| Wall-clock estimate | ≈ **42–50 hours** with five agents working in parallel and one integrating; ≈ 31–35 h for P0 + P1 only |
|
||||
|
||||
@@ -74,7 +74,7 @@ library groundwork for mask tracking and Warp Stabilizer.
|
||||
| Wall clock, five parallel agents | 42–50 h | **≈ 8–10 h at today's measured pace; ≈ 25 h by the audit's conservative figures** |
|
||||
|
||||
What is left is concentrated in large systems: Roto Brush and Refine Edge, the 3D camera tracker,
|
||||
Advanced 3D (models, PBR, image-based lighting), the Warp Stabilizer and mask-tracking UI on top of
|
||||
the Warp Stabilizer and mask-tracking UI on top of
|
||||
the new tracking library, a JavaScript scripting object model, SVG/PSD import, WebM, a disk cache,
|
||||
native macOS menus, and wasm threads. (The missing effect categories, 3D Channel, Immersive Video
|
||||
and OCIO, landed in M9.11.)
|
||||
@@ -91,20 +91,20 @@ and OCIO, landed in M9.11.)
|
||||
| Shapes | 68% | 3.9 | taper and wave strokes (pen tool for shape paths and vertex editing landed in M6.5) |
|
||||
| Compositions | 67% | 7.4 | marker dialog, flowchart, Essential Graphics |
|
||||
| Animation | 65% | 10.4 | puppet, Wiggler/Smoother/Motion Sketch (motion-path handles and the graph editor transform box landed in M5.8) |
|
||||
| Text | ≈ 85% | 1.5 | vertical type is basic (upright characters, no tate-chu-yoko), no OpenType feature panel, no text on 3D bevels; per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API landed (M9.9–M9.10) |
|
||||
| Text | ≈ 86% | 1.3 | vertical type is basic (upright characters, no tate-chu-yoko), no OpenType feature panel, no extruded strokes (extruded, bevelled text landed in M7.6); per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API landed (M9.9–M9.10) |
|
||||
| Web | 60% | 2.5 | threads, browser storage, audio |
|
||||
| 3D | 58% | 20.5 | Advanced 3D (models, PBR, image-based light), 3D camera tracker, multi-view layouts |
|
||||
| Effects | ≈ 75% | 6.0 | GPU versions of more effects, Warp Stabilizer and the 3D Camera Tracker (all other After Effects effects exist since M9.11) |
|
||||
| 3D | 72% | 12.0 | 3D camera tracker, multi-view layouts, Advanced 3D motion blur and blend modes, cameras/lights from models; Advanced 3D (glTF/OBJ models, primitives, extruded text and shapes, PBR, image-based light, shadow maps, GPU rasteriser) landed in M7.4–M7.6 |
|
||||
| Effects | ≈ 75% | 5.0 | GPU versions of more effects and the 3D Camera Tracker (every other After Effects effect exists since M9.11 and M12.5) |
|
||||
| Interface | 55% | 10.4 | drag-to-dock and floating panels, preferences, native macOS menus (viewer rulers/snapping/channels/snapshots landed in M0.13) |
|
||||
| Project | 53% | 8.1 | 8/16/32-bit pipeline and colour management, auto-save, proxies, folder moves |
|
||||
| Masks & roto | 44% | 13.0 | mask tracking, Roto Brush, variable-width mask feather points |
|
||||
| Masks & roto | 52% | 10.0 | Roto Brush, variable-width mask feather points (mask tracking and Mask Interpolation landed in M6.6) |
|
||||
| Preview | 55% | 6.1 | GPU 3D and adjustment layers, disk cache (region of interest, snapshots, exposure and Fast Previews landed in M0.13) |
|
||||
| Tracking | 0% | 13.0 | point tracker, Warp Stabilizer, mask and face tracking |
|
||||
| Tracking | ≈ 70% | 4.0 | 3D camera tracker, face tracking and Subspace Warp's mesh warp (Rolling Shutter Repair landed in M9.11; point tracker, mask tracking and Warp Stabilizer landed in M6.x / M12.5) |
|
||||
| Paint | 0% | 4.0 | Brush, Clone Stamp, Eraser |
|
||||
|
||||
## Effects still missing
|
||||
|
||||
Warp Stabilizer and the 3D Camera Tracker (built on the tracking library by other work). Boris FX
|
||||
The 3D Camera Tracker (built on the tracking library by other work). Boris FX
|
||||
Mocha and Cineware are third-party and not counted.
|
||||
|
||||
### Added in M9.11 (39 effects)
|
||||
@@ -144,4 +144,4 @@ Mocha and Cineware are third-party and not counted.
|
||||
The engine underneath (keyframes, expressions, shape operators, text animators, Classic 3D, all
|
||||
38 blend modes, the render queue) is deep. Most of what is missing is interactive tooling in the
|
||||
viewer, settings that are stored but not yet rendered, about 69 menu entries that are still
|
||||
disabled, and the large systems: tracking, puppet, paint, roto, Advanced 3D and GPU rendering.
|
||||
disabled, and the large systems: tracking, puppet, paint and roto (Advanced 3D landed in M7.4–M7.6).
|
||||
|
||||
Reference in New Issue
Block a user