diff --git a/AGENTS.md b/AGENTS.md index e27a83fa..b96c7816 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -107,6 +107,7 @@ a command, everything is agent-drivable, and the quality gates (`cargo xtask ci` ## See also +- [docs/gaps.md](docs/gaps.md): where EffectCraft falls short of After Effects and the prioritised workstreams (G1–G9); read it before choosing work - [CONTRIBUTING.md](CONTRIBUTING.md): setup, gates, commits, how to add things - [docs/architecture.md](docs/architecture.md): layers, data model, commands, pipeline - [docs/testing.md](docs/testing.md): oracle tests, criteria, benchmarks diff --git a/CLAUDE.md b/CLAUDE.md index fa41b323..afa08d3d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,8 @@ EffectCraft is a clean-room, open-source, pure-Rust motion graphics and visual e `plan/` is maintainer-local (gitignored). Public equivalents: [`ROADMAP.md`](ROADMAP.md), [`docs/`](docs/). 1. Read `plan/STATUS.md` (current milestone, next task, running agents, blockers). 2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the crate README/docs you touch. Feature ids: `plan/aftereffects/feature-catalog.md`; effects: `plan/aftereffects/effects-list.md`; UI reference: `plan/aftereffects/README.md`. -3. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask. +3. Read [`docs/gaps.md`](docs/gaps.md): the honest assessment of where we fall short and the prioritised workstreams. Fidelity, reliability and stability (G1–G3) come before new features. +4. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask. ## Non-negotiables **Read [`AGENTS.md`](AGENTS.md) first; its rules override everything here.** No Adobe assets, every asset licensed + attributed (`cargo xtask assets`). diff --git a/README.md b/README.md index a59b29d3..85dab827 100644 --- a/README.md +++ b/README.md @@ -195,14 +195,25 @@ tests, layering, asset attribution and the WebAssembly build). See [CONTRIBUTING ## Where it stands -EffectCraft is young. The core of After Effects is in place and gets deeper every week: what is -done and what comes next is in the [ROADMAP](ROADMAP.md). Roto Brush can use a trained model -(MobileSAM, Apache-2.0, a 40 MB download you opt into in Settings ▸ Roto Brush). Still to come: a -learned face tracker (face tracking uses classical methods today), GPU versions of the remaining -CPU-only effects, the Advanced 3D Extended Viewer, and shared-memory threads in the web build. The -web build ([docs/web.md](docs/web.md)) runs the full app in the browser, with renders, analyses -and viewer frames in Web Workers. Projects are saved as `.ecproj`, readable versioned JSON; After -Effects `.aep` files cannot be opened. +EffectCraft is young: its first commit was on 1 October 2026. Nearly every After Effects feature +exists, including all of its effects, but existing is not the same as behaving exactly like After +Effects, and we haven't measured that yet. Today it's a good place to try things and to tell us +what breaks. It isn't yet a replacement for After Effects on client work. In particular: + +- **After Effects projects can't be opened.** Projects are saved as `.ecproj`, readable versioned + JSON, but EffectCraft can't open `.aep` / `.aepx` files, and After Effects plug-ins don't run. +- **Behaviour still differs from After Effects in places.** Nothing yet compares our renders with + After Effects automatically, so please report differences. +- **macOS is the most tested platform.** Linux and Windows users have hit basic interaction + bugs, which we're fixing first. +- **Face tracking uses classical methods.** Roto Brush can use a trained model (MobileSAM, an + optional 40 MB download in Settings ▸ Roto Brush); it is new and not yet compared with After + Effects. + +The [ROADMAP](ROADMAP.md) has the plan and [docs/gaps.md](docs/gaps.md) the full, honest +assessment. The web build ([docs/web.md](docs/web.md)) runs the full app in the browser. +[Bug reports](https://github.com/storytold/effectcraft/issues) are the most useful thing you can +send us right now. ## How it's made diff --git a/ROADMAP.md b/ROADMAP.md index e1709160..e42f2695 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -25,14 +25,38 @@ once. ## How far from full parity -Measured feature by feature in [docs/parity.md](docs/parity.md) (4 October 2026): +There are two honest answers, and they are far apart. The full assessment, with the evidence, is +in [docs/gaps.md](docs/gaps.md). -- **≈ 99% of After Effects' features, weighted by importance**, counting partial features as half - done (≈ 99.9% with per-feature fractions): 90 of 92 features done, 2 partial, none missing; every - essential (P0) feature done; all 306 effects implemented in full, 280 of them on the GPU. -- **≈ 8.5 agent-hours of work remain**: about **2–3 hours of wall-clock time** with five Claude - Opus 5.5 agents in parallel (estimated before Roto Brush's trained model landed in M13.35). -- What is left: GPU kernels for the remaining visual effects, and a learned face tracker - ([parity.md](docs/parity.md#highest-value-gaps-in-order)). +- **Breadth: ≈ 99%.** Almost every After Effects feature exists: 89 of 92 catalogued features + done and 3 partial, all 306 effects implemented, 280 of them on the GPU + ([docs/parity.md](docs/parity.md)). This is our own checklist, graded by the agents that built + the features. +- **Real use: ≈ 30–50% (estimated, not yet measured).** Whether someone who uses After Effects for + a living can do client work in EffectCraft. What holds it back: + - **Fidelity is unmeasured.** No test compares our output with After Effects itself, and users + are still finding behaviour bugs in features marked done. + - **After Effects projects can't be opened.** EffectCraft can't read `.aep` / `.aepx` files, + and third-party After Effects plug-ins can't run. + - **Reliability is uneven across platforms.** Early Linux users hit basic problems: viewer + panning, panel docking, drag-and-drop import. + - **AI tools are behind.** Face tracking uses classical methods. Roto Brush's trained model + (MobileSAM, optional, M13.35) is new and not yet compared with After Effects. + +## Where we're going + +In priority order (details and "done" criteria in [docs/gaps.md](docs/gaps.md)): + +| | Workstream | Why | +|---|---|---| +| G1 | Measure fidelity against After Effects: a corpus of test projects rendered and sampled in both apps, scored per feature | Turns every estimate into a measurement; a feature that behaves differently is not done | +| G2 | Real-user reliability on macOS, Windows and Linux; fix every open user issue | Basic workflows must work everywhere before more features matter | +| G3 | Stability: a green gate on the current toolchain, fuzzing of every input | A crash loses people's work | +| G4 | Open After Effects projects (`.aep` / `.aepx`, pending an owner decision on clean-room scope); relinking moved footage | The biggest barrier to switching | +| G5 | Performance benchmarks at 1080p and 4K on real-world-sized projects | Unknown today | +| G6 | Media depth: encoder efficiency, camera and phone formats | Files are larger than from mature encoders; camera formats are unverified | +| G7 | Learned models: Roto Brush started (M13.35: MobileSAM, open source, optional download, swappable model module); face tracking still classical | Quality is well below After Effects' and unmeasured | +| G8 | Plug-in ecosystem on our WebAssembly plug-in API | After Effects plug-ins can't run here | +| G9 | Localisation, accessibility, user documentation | Reach beyond English-speaking power users | Come tell us what matters most to you on [Discord](https://discord.gg/artcraft). diff --git a/docs/gaps.md b/docs/gaps.md new file mode 100644 index 00000000..0e0fdc2c --- /dev/null +++ b/docs/gaps.md @@ -0,0 +1,134 @@ +# Where EffectCraft falls short + +An honest assessment of how far EffectCraft is from being a real replacement for After Effects, +and the work that closes the gap. This document is meant for contributors and agents choosing +what to work on. The [ROADMAP](../ROADMAP.md) summarises it; [parity.md](parity.md) is the +feature-by-feature checklist it builds on. + +*Assessed 5 October 2026. Estimates marked "≈" are judgements from the evidence listed, not +measurements. Update this file when the evidence changes.* + +## Two different questions + +[parity.md](parity.md) reports **≈ 99%**. That number answers one question: *does each After +Effects feature exist?* It scores a 92-item catalogue we wrote ourselves, graded by the same agents +that built the features, with partial features counting half. As a measure of breadth it is fair, +and the breadth is real: every one of After Effects' effects exists by name, along with the +menus, panels, 3D, tracking, expressions, scripting and export. + +It does not answer the question users care about: *can someone who uses After Effects for a living +do real work in EffectCraft?* Nobody has measured that yet. Our best estimate is **≈ 30–50%**, +limited mainly by four things: projects we can't open, behaviour nobody has checked against After +Effects, reliability on platforms other than macOS, and the third-party plug-ins professional +projects depend on. + +The checklist is also internally inconsistent: parity.md's per-area table still shows Interface +75%, Project 68%, Animation 70% and Paint 0% next to the 99% headline. Some rows are stale; none +of them have been reconciled. + +## By dimension + +| Dimension | Estimate | Evidence | +|---|---|---| +| Breadth of features | ≈ 95%+ | All 306 effects, the After Effects menus and panels, Classic and Advanced 3D, tracking, expressions, scripting, render queue (parity.md) | +| Behaves like After Effects | unmeasured, ≈ 60–80% | One outside contributor found four bugs in features marked done within a day (PRs #6–#10): Hold keyframes eased the motion into them, `keyInSpatialTangent` / `keyOutSpatialTangent` had the wrong names, a zero frame rate crashed, Find and Enter Full Screen shared Ctrl+F off macOS. There are about 10 After Effects reference captures in total. No test compares our renders or property values with After Effects itself; the tests check what the agents believed After Effects does | +| Opening existing After Effects work | ≈ 0% | `.aep` / `.aepx` projects cannot be opened. Third-party After Effects plug-ins cannot run. Expressions and the scripting object model are strong, so scripts and expressions carry over | +| Stability | improving | 23 never-crash PRs landed on 4 October; [AGENTS.md](../AGENTS.md) "Never crash" now binds every crate. On 5 October `cargo xtask ci` failed on main under Rust 1.99's clippy (a fix is in progress) | +| Real-user experience | ≈ 70%, uneven by platform | Issues #41–#47 (all from the Linux AppImage, 5 October): panning the viewer snaps back, panels can't be resized or rearranged, drag-and-drop and double-click import don't work, layer rename gets stuck, the Layer Settings arrow does nothing, the Project panel clips, the Wayland window icon is generic. parity.md counts docking as done. Most checking happens on macOS. No localisation, no accessibility work | +| Performance | unknown against After Effects | Internal numbers only (e.g. Advanced 3D 290 ms/frame at 1080p on the GPU, an M4 Pro under load). Nothing benchmarked against After Effects; no large real projects (4K footage, hundreds of layers) tested | +| Media formats | ≈ 80% | H.264, ProRes, HEVC, AV1, image sequences and audio exist. The new HEVC / AV1 encoders have no B-frames, multi-reference or SAO / CDEF, so files are larger than from mature encoders. Camera formats (BRAW, R3D, ProRes RAW, variable-frame-rate phone video) are unverified | +| AI-assisted tools | ≈ 50% | Face tracking uses a classical shape-model fitter. Roto Brush 2.0 / 3.0 can use MobileSAM since M13.35 (pure-Rust inference, an optional download): on our synthetic moving-disc test it scores IoU 0.989 on the base frame and ≥ 0.980 over 20 propagated frames (classic: 0.973), but nothing compares it with After Effects yet | +| Maturity | early | First commit 1 October 2026. ≈ 285,000 lines of almost entirely agent-written Rust, ≈ 2,050 tests, 8 external issue reports so far. After Effects has around 30 years of edge cases behind it | + +## Where we're going: workstreams in priority order + +Each workstream lists what "done" means, so progress can be measured rather than self-graded. + +### G1. Measure fidelity against After Effects + +The most important missing piece: it turns every other estimate here into a measurement. + +- Build a corpus of original test projects (our own content, no Adobe assets), one or more per + feature id in `plan/aftereffects/feature-catalog.md`: keyframe interpolation of every kind, + blend modes, track mattes, each effect at default and at non-default settings, text animators, + expressions, 3D, motion blur, time remapping. +- Drive After Effects through ExtendScript (see CLAUDE.md) to record property values at sampled + times and render reference frames; render the same projects in EffectCraft headless. +- Compare values exactly and frames with a perceptual metric; publish a per-feature fidelity + score and a fidelity column in parity.md. +- Clean room: After Effects output stays local in `plan/aftereffects/ref/` (gitignored). Commit + only our projects, the harness and the scores, never After Effects frames. +- Done when: every P0 and P1 feature has at least one corpus project, scores are reproducible from + one command, and parity.md reports breadth and measured fidelity side by side. + +### G2. Real-user reliability on every platform + +- Fix the open user issues (#41–#47), each with a regression test. +- Run the headless snapshot and control-channel checks on Linux (X11 and Wayland) and Windows, not + only macOS. Interactions that only fail with real input (docking drags, viewer pan, drag-and-drop + import, inline rename) need scripted input tests. +- Done when: no open bug blocks a basic workflow (import, arrange, animate, preview, render) on + any of the three desktop platforms, and every bug users report gets triaged within a day. + +### G3. Stability and a green main + +- Keep `cargo xtask ci` green on the current stable toolchain. A toolchain release that breaks the + gate is a P0. +- Fuzz the inputs we don't control: project files, imported media and documents, expressions, + scripts, control-channel and MCP requests. Every crash found becomes a regression test + (AGENTS.md "Never crash"). +- Done when: the gate is green on stable and the fuzzers run regularly without new crashes. + +### G4. Open After Effects projects + +- `.aep` / `.aepx` import is the largest single barrier to switching. It needs an **owner + decision** on clean-room scope (whether and how the format may be studied), like the open + `.prproj` question in `plan/STATUS.md`. +- Also in this area: relative footage paths and relinking when a project moves (parity.md, Project). +- Done when: the decision is recorded and, if approved, a corpus of real-world-shaped projects + opens with its comps, layers, keyframes, effects and expressions intact. + +### G5. Performance at real-world scale + +- Benchmark projects at 1080p and 4K with many layers, heavy effects, long footage and nested + comps; record preview frame times, RAM preview fill and render times on reference machines. +- Compare with After Effects on the same machine where possible. +- Done when: benchmark numbers are tracked over time and regressions fail a check. + +### G6. Media depth + +- Encoder efficiency: B-frames and multi-reference for HEVC / AV1, SAO / CDEF / restoration, Opus + FEC / DTX. +- Verify decode of the camera and phone formats people actually bring, including variable frame + rate; document what is not supported. + +### G7. Learned models for Roto Brush and face tracking + +- Decided on 5 October 2026 for Roto Brush: licensed weights with pure-Rust inference, if the + impact on the system is low, in a composable, swappable module. Done in M13.35: the + `effectcraft-segment` crate (a `MaskModel` interface, a registry that only accepts + open-source licences, MobileSAM under Apache-2.0), weights downloaded on demand and verified, + Settings ▸ Roto Brush to choose. Next: measure it against After Effects (G1); faster encoders + (GPU) for long shots. +- Face tracking: still an owner decision (a learned detector, or the classical fitter as final). + +### G8. Plug-in ecosystem + +- After Effects SDK plug-ins cannot run in EffectCraft. Our own WebAssembly plug-in API exists + ([plugins.md](plugins.md)). Grow it: documentation, examples, and original effects that cover + what the most common third-party plug-ins are used for (particles, glows, 3D objects, sabers). + +### G9. Reach + +- Localisation of the interface, accessibility (keyboard navigation, screen readers, contrast), + user documentation and tutorials. + +## For agents choosing work + +1. Read `plan/STATUS.md` for owner blockers and running work, then this file. +2. Prefer G1–G3 over new features. A feature that exists but behaves differently from After + Effects is not done. +3. When you fix a behaviour bug in a feature parity.md marks as done, add it to the evidence + above. That is how this assessment stays honest. +4. Don't raise parity.md's numbers without evidence: a G1 score, a user-facing check on more than + one platform, or both. diff --git a/docs/parity.md b/docs/parity.md index 5b5fef5b..7df637e3 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -2,17 +2,25 @@ How close EffectCraft is to After Effects 2026, feature by feature, and how much work is left. -## Current status (audit at commit `d39c0e8`, 4 October 2026; updated 5 October for M3.9–M4.12 and M13.35) +> **Read this as a measure of breadth, not of real-world readiness.** It checks whether each +> feature exists, using our own catalogue, graded by the agents that built it. It does not measure +> whether the feature behaves like After Effects (nothing compares the two yet), whether After +> Effects projects open (they don't), or how the app holds up for real users on every platform. +> Those gaps, and the work that closes them, are in [gaps.md](gaps.md). Don't raise the numbers +> here without the evidence gaps.md asks for. The per-area table further down has stale rows that +> have not been reconciled with the headline. + +## Current status (audit at commit `d39c0e8`, 4 October 2026; updated 5 October for M3.9–M4.12) | Measure | Value | |---|---| -| **Feature parity, weighted by tier** (P0 ×3, P1 ×2, P2 ×1) | **≈ 99.4%** counting every partial feature as half done; ≈ 99.9% using per-feature fractions | -| Unweighted | ≈ 98.9% (half-credit) / 99.9% (fractions) | -| P0 / P1 / P2 | 100% / 98% / 95% (half-credit); 100% / 99.8% / 99% (fractions) | -| Features done / partial / missing | 90 / 2 / 0 of 92 (MSK-4 Roto Brush done in M13.35) | +| **Feature parity, weighted by tier** (P0 ×3, P1 ×2, P2 ×1) | **≈ 99%** counting every partial feature as half done; ≈ 99.8% using per-feature fractions | +| Unweighted | ≈ 98.4% (half-credit) / 99.6% (fractions) | +| P0 / P1 / P2 | 100% / 98% / 90% (half-credit); 100% / 99.7% / 97.5% (fractions) | +| Features done / partial / missing | 89 / 3 / 0 of 92 | | **Effects** | **306** effects, every one implemented in full ([effects.md](effects.md)); 280 run on the GPU | | Disabled menu entries left | 1, on purpose: Import ▸ Vanishing Point (.vpe), whose format has no public specification | -| Remaining work | ≈ 8.5 agent-hours at the audit (≈ 26 on the conservative audit scale), before M13.35 landed Roto Brush's trained model; the learned face tracker is the one learned-model item left | +| Remaining work | ≈ 8.5 agent-hours at the pace measured so far (≈ 26 on the conservative audit scale); ≈ 3.5 without the learned-model items | | **Wall-clock estimate** | **≈ 2–3 hours** with five agents in parallel; ≈ 1 hour without the learned-model items | Partial features: @@ -20,6 +28,7 @@ Partial features: | id | Tier | Done | What is missing | |---|---|---|---| | EFF-5 GPU effects | P1 | 0.98 | Fractal (its escape iteration needs f64: WGSL has none and Metal's fast math defeats double-f32 emulation) and the analysis / tool effects (Camera-Shake Deblur, Detail-preserving Upscale, Puppet, Rolling Shutter Repair, Warp Stabilizer, Mocha Shape, Roto Brush, Match Grain, Paint, Camera Tracker, Face Tracker) render on the CPU. M13.28 moved the remaining pixel effects (CC blur / glass family, Color Link, Cineon, HDR, Inner/Outer Key, Basic 3D…) and the geometry and audio generators (Lightning, Advanced Lightning, Beam, Lens Flare, Radio Waves, Vegas, Stroke, Scribble, Write-on, Paint Bucket, Eyedropper Fill, CC Glue Gun, CC Threads, Audio Spectrum / Waveform, Basic / Path Text) onto the GPU; M13.29 the particle effects (CC Particle World / Systems II, Particle Playground, CC Ball Action, CC Pixel Polly, CC Scatterize) and Curl Noise, and removed the Shatter wireframe, Foam extras, Timewarp matte, strong Warp Fisheye / Twist and custom `.ocio` fallbacks. The 13 audio filters stay on the CPU by design | +| MSK-4 Roto Brush | P2 | 0.85 | Roto Brush 2.0 / 3.0 can use a trained model since M13.35 (MobileSAM, optional download); its quality has not been compared with After Effects (G1), so the score stands | | TRK-3 Face tracking | P2 | 0.9 | Classical fitter: weak on profile and occluded faces; Rolling Shutter Ripple is approximated | Since this audit: the project operations (M3.9–M3.14) and preview (M4.5–M4.12) passes fixed @@ -433,7 +442,8 @@ stretch) and shown within its In–Out range. Hovering names them ("beat (marker double-click opens the nested comp at the marker. Agents read the same list with `markers.nested {layer}`. -Since (M13.35): Roto Brush can use a trained segmentation model; MSK-4 is done. +Since (M13.35): Roto Brush can use a trained segmentation model. MSK-4 keeps its score until G1 +measures it against After Effects ([gaps.md](gaps.md)). - The model sits behind a swappable interface: `effectcraft_segment::MaskModel` plus a registry where every entry must have an open-source licence compatible with ours, a source URL, a size and a SHA-256.