aleleba/effectcraft
1
0
Fork 0
mirror of https://github.com/storytold/effectcraft.git synced 2026-10-29 04:26:55 -06:00
Code Issues Packages Projects Releases Wiki Activity

#783: Object Matte effect from a point / box prompt, tracked

Browse Source 
Problem: After Effects 26.2's Object Matte (pick an object with a click,
optionally a box, get a matte that follows it) had no counterpart; only
stroke-driven Roto Brush existed.

Fix: an Object Matte effect (Effect > Matte, ec.matte.objectmatte) turns
the Object Point (and optional box) on a Reference Frame into Roto Brush
strokes and segments them with Roto Brush's engine, content-keyed cache
and the model chosen in Settings > Roto Brush (MobileSAM, classical graph
cut without it). Track Object carries the matte frame by frame up to 600
frames each way; otherwise every frame is segmented on its own. Feather,
Contrast, Shift Edge, Invert and a Matte view. The engine keeps its Input
Key current so upstream edits recompute it. objectMatte.apply creates one
from agents / the CLI. Unusable prompts pass the layer through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
vangel76
2026-10-11 13:32:10 +02:00
co-authored by Claude Opus 5.5
parent 9a7d3e7699
commit 953fbbc730
9 changed files with 674 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/effects.md
+4 -3
View File
@@ -3,11 +3,11 @@
<!-- Generated from the effect registry (crates/effects/src/catalog.rs); do not edit by hand.
Regenerate: UPDATE_DOCS=1 cargo test -p effectcraft-effects --lib effects_doc_is_current -->
EffectCraft ships 306 effects with 3014 parameters in total, grouped into the same categories as the Effects & Presets panel. Every effect is our own implementation, written from public behaviour descriptions and standard image-processing literature. Parameter names, order, popup options, units, defaults and ranges follow the reference application so that projects, expressions and muscle memory carry over.
EffectCraft ships 307 effects with 3028 parameters in total, grouped into the same categories as the Effects & Presets panel. Every effect is our own implementation, written from public behaviour descriptions and standard image-processing literature. Parameter names, order, popup options, units, defaults and ranges follow the reference application so that projects, expressions and muscle memory carry over.
- **GPU**: 280 effects also run on the GPU compositor with identical results.
- **32**: 306 effects process 32-bit float (HDR, overbright) pixels without clamping.
- **Status**: 306 implemented in full, 0 partial (what is missing is listed).
- **32**: 307 effects process 32-bit float (HDR, overbright) pixels without clamping.
- **Status**: 307 implemented in full, 0 partial (what is missing is listed).
Effects are addressed by id (`ec.<category>.<name>`) or display name in commands, scripts and the control channel, and every parameter by its id (`effects/#1/blurriness`).
@@ -255,6 +255,7 @@ Effects are addressed by id (`ec.<category>.<name>`) or display name in commands
|---|---|---:|:-:|:-:|---|
| Matte Choker | `ec.matte.mattechoker` | 7 | GPU | 32 | Implemented |
| Mocha shape | `ec.obsolete.mochashape` | 6 | | 32 | Implemented |
| Object Matte | `ec.matte.objectmatte` | 13 | | 32 | Implemented |
| Refine Hard Matte | `ec.matte.refinehard` | 16 | GPU | 32 | Implemented |
| Refine Soft Matte | `ec.matte.refinesoft` | 18 | GPU | 32 | Implemented |
| Roto Brush & Refine Edge | `ec.matte.rotobrush` | 24 | | 32 | Implemented |
docs/gaps.md
+1 -1
View File
@@ -70,7 +70,7 @@ are grouped by area; the issue tracker has the rest.
| 11 | **Text rendering and layout** | #453 CJK text in comps, #316 paragraph alignment inverted, #483 squeezed at 4:3, #417 soft at Fit, #583 font search, #550 toolbar font options | Text is in almost every motion graphic | 10–20 h | G2 · [ui-parity.md](ui-parity.md) |
| 12 | **3D interaction and depth** | No 3D transform gizmos (#392, #347); Z position (#489); camera aperture (#260); reflections (#261); bevel options (#315); 3D issue reopened (#307) | Advanced 3D is After Effects' growth area since 2023 | 20–35 h | G2 · [ui-parity.md](ui-parity.md) |
| 13 | **3D model formats and materials** | After Effects 26 imports FBX, USD/USDZ, STL besides glTF and OBJ (`usd_plugins`), and Substance 3D materials with mesh displacement (26.0–26.2); we read glTF and OBJ | Product shots and 3D scenes come as FBX/USDZ | 25–45 h | — · [file-format-parity.md](file-format-parity.md) |
| 14 | **Object Matte** (26.2, disk cache in 26.5) | Not present; MobileSAM is already in `effectcraft-segment` and could back it | The new one-click roto; Roto Brush covers part of it | 15–30 h | G7 · [target-app-parity.md](target-app-parity.md) |
| 14 | **Object Matte** (26.2, disk cache in 26.5) | First slice (#783): the Object Matte effect (Effect ▸ Matte) and `objectMatte.apply` turn a point (and optional box) on a reference frame into a matte and follow the object up to 600 frames each way, or segment every frame on its own; Roto Brush's engine, cache and model (MobileSAM from Settings ▸ Roto Brush, classical graph cut without it). Missing: the click / hover tool in the viewer, a background propagation job, the disk cache, comparison with After Effects | The new one-click roto | 10–20 h | G7 · [target-app-parity.md](target-app-parity.md) |
| 15 | **Hardware video decode and encode** | No VideoToolbox, NVENC, Quick Sync or VA-API; #498 asks for it; FilmCraft has VideoToolbox / NVENC / VA-API paths to borrow | 4K playback and export speed | 25–45 h | G5/G6 · [hardware-parity.md](hardware-parity.md) |
| 16 | **Localization depth** | Menus in 6 languages; panels in 4; effect names and ~1,650 effect parameter names untranslated; French, German, Korean, Italian, Russian (After Effects ships them) absent; #599 Hangul boxes on macOS | After Effects is used heavily in Japan, China, Korea, Brazil, Europe | 60–110 h + native review | G9 · [localization-parity.md](localization-parity.md) |
| 17 | **Media Browser lists files import refuses** | `media_browser.rs` lists TGA, DPX, HDR, AVI, MXF; `media::is_importable` has none of them; #431 any image imports as a sequence; #324 MP3 goes silent | Confusing failures on common VFX formats (DPX, TGA) | 15–30 h | G6 · [file-format-parity.md](file-format-parity.md) |
docs/target-app-parity.md
+2 -2
View File
@@ -122,7 +122,7 @@ compositing and effects dominate; audio is light). Hours are to full parity for
| Text and typography | 7% | 94% | 50% | 20–35 | CJK text in comps renders wrongly (#453); paragraph alignment inverted (#316); squeezed text at 4:3 (#483); font search (#583); soft text at Fit (#417); variable-font filter (26.3) |
| Effects and animation presets | 14% | 85% | 40% | 70–130 | All 306 effects exist, but rendered output is unmeasured against After Effects; 8 built-in presets against After Effects' 621; open effect bugs (#580 crash, #514, #526, #463, #369, #473); effect colour labels (26.5); see [effects-parity.md](effects-parity.md) |
| 3D (Classic and Advanced) | 6% | 82% | 35% | 35–65 | No FBX/USD/STL models, no Substance materials, mesh displacement unverified; no 3D transform gizmos (#392, #347); Z position (#489); reflections (#261); camera aperture (#260); bevel options (#315) |
| Tracking and AI-assisted tools | 5% | 80% | 35% | 35–70 | No Object Matte (26.2); Roto Brush (MobileSAM) and face tracking (MediaPipe) unmeasured against After Effects; camera-tracker view (#505) |
| Tracking and AI-assisted tools | 5% | 80% | 35% | 35–70 | Object Matte (26.2) is a first slice: an effect and command from a point / box prompt, no viewer tool or disk cache; Roto Brush (MobileSAM) and face tracking (MediaPipe) unmeasured against After Effects; camera-tracker view (#505) |
| Preview and cache | 7% | 92% | 40% | 25–50 | Playback stutters on cached frames (#595, #304); slow scrubbing (#329); late audio (#554, #504, #335); no benchmark against After Effects |
| Render queue and export | 7% | 85% | 55% | 25–45 | No hardware encoding; ProRes 4444 colour tag (#340); rendering over an existing file (#325); default macOS output location (#446); uncompressed single-file output (#372); HEVC/AV1 without B-frames |
| Audio | 2% | 85% | 45% | 6–12 | Audio sync and crackle (#335); audio to keyframes depth |
@@ -144,7 +144,7 @@ Feature depth (≈ 50%) feeds the overall *ready* number with the other dimensio
| Stability | 45% | 40–80 | Open crash reports #580, #591, #501, #426, #341, #306, #234, #511; no CI runs the tests on pull requests | [gaps.md](gaps.md#g3-stability-and-a-green-main) |
| Platforms | 65% | 30–60 | Releases for macOS, Windows x64/x86/ARM64, Linux, FreeBSD and the web (more than After Effects), but Linux and Windows users hit basic bugs | [gaps.md](gaps.md#g2-real-user-reliability-on-every-platform) |
| Ecosystem and plug-ins | 20% | 40–80 | After Effects scripts and expressions mostly carry over; no After Effects SDK plug-ins, `.ffx` presets or `.mogrt`; our WebAssembly plug-in API has no third-party plug-ins yet | [plugins.md](plugins.md) |
| AI features | 35% | 40–80 (overlaps Tracking) | Roto Brush (MobileSAM) and face tracking (MediaPipe) as optional downloads; no Object Matte; the After Effects bundle ships Whisper, BERT and Marian tokenizers and a shot-cut model we have no equivalent of | [gaps.md](gaps.md) |
| AI features | 35% | 40–80 (overlaps Tracking) | Roto Brush (MobileSAM) and face tracking (MediaPipe) as optional downloads; Object Matte (point / box prompt, same model) without its viewer tool or disk cache; the After Effects bundle ships Whisper, BERT and Marian tokenizers and a shot-cut model we have no equivalent of | [gaps.md](gaps.md) |
| Agent control (beyond After Effects) | ahead | — | Every command is drivable over MCP, the control channel and the CLI | [agents.md](agents.md) |
**Totals.** Summing the rows double-counts: AI overlaps Tracking, performance overlaps Preview,