bench --roto per size: no larger size tracked better on the synthetic clips and
Large lost the object after it re-entered; the hints say what was measured.
The bar shows the current object's stroke count and can remove its strokes.
An env-gated test runs the in-app download of a size.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Strokes carry an object (0 for older projects, nothing written for 0). The
tracker follows every object in one pass and the mattes are united; the
confidence of a frame is the least sure object's. Object picker in the Layer
panel's bar; roto.options/stroke/clearStrokes/status take or report objects.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Hiera configs and size detection from the weights, registry entries pinned to
commits with verified SHA-256, the loader picks the size by checksum. The GPU
encoder submits once per block so Large fits in memory, and uploads no longer
panic on a lost device. Reference tests run per size on the CPU and GPU.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Memory attention (1.8e-5 vs PyTorch) and the memory encoder (2.6e-6) as wgpu kernels,
convolution and layer-norm+GELU kernels, VideoModel::load_auto picks the GPU when
there is one (EFFECTCRAFT_SAM2_DEVICE overrides) and falls back to the CPU on any GPU
error; disable_gpu switches it off mid-clip. One process-wide device (several at once
crashed the Vulkan loader). A tracked frame: 4.3 s CPU, 0.7 s on the RX 7900 GRE.
Clips tracked on the GPU match Sam2VideoModel (worst logit error 0.0032).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
MaskModel::idle_secs (SAM 2 tracks its use); the engine frees a model idle for
ten minutes when no job runs, keeps the choice, and reloads it before a stroke
or a Propagate needs it. Docs: memory and limits.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Frames a tracker followed are coloured by its confidence (green, amber, red;
slate where the object was absent) and drawn taller; roto_trust reports them.
Not checked visually: no headless way to track without weights in a snapshot.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Chain::track tracks the frames of the span without a tracked result, in a window
that restarts at the nearest prompted frame; unchanged spans are not tracked
again. Test with a recording tracker: first pass 0..20, no second pass, then a
correction on frame 8 tracks 8..20 with that prompt only.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TrackRequest takes a list of objects (each with its prompts on its frames) and
the tracker reports (object, frame, result); frame encodings are shared, objects
may start on different frames. VideoTracker::propagate_objects runs a subset.
Limits: 32 objects. Test: two discs, one prompted on frame 0 and one on frame 2.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The tracked frames lived only in memory, so a saved project lost them: a render in another
process (effectcraft-cli render-frame) segmented the frames again with the classic engine.
A propagate with a tracker now saves the frames it followed with the instance, in a hidden
parameter beside Freeze's (TrackCache: each frame's matte, Refine Edge band, colour models and
confidence, keyed by the inputs it was tracked from).
- They apply while the strokes and span they came from are unchanged, whichever model Settings
choose, so a project renders the same when it reopens, without the model loaded or chosen.
- A stroke or span change makes them stale until the next Propagate. A Propagate replaces them;
with the classic engine it leaves none.
- roto.status reports tracker.saved (the model they came from) and a notice when Settings
choose another model.
- The save at job completion is not an undo step (the frames are a cache of the strokes) and is
skipped if the strokes changed while tracking. Decoding checks the saved size, so a corrupt
project is ignored rather than overflowing.
Object Matte saves nothing. Projects saved before this change open unchanged (the parameter is
filled in on open).
Tests: a saved project reopens with the model unloaded and renders the saved frames through
Chain::ensure; choosing the classic engine keeps them until a classic Propagate replaces them; a
stroke change makes them stale. The SAM 2 tests and the failing-tracker test take the Roto test
lock.
Claude-Session: https://claude.ai/code/session_0136kZtXrYo5VUWEFxWduwUD
With SAM 2 (or MobileSAM) chosen, a propagate that started before the model had loaded ran
the whole job on the classic engine and reported nothing. The chosen model is now in use
before a job that segments with it starts (Roto Brush 2.0 / 3.0 and Object Matte):
- roto.propagate and the other Roto jobs wait for a load or download under way when called with
wait: true (bounded to 10 minutes). Without wait they refuse while the model is still loading,
and start the load in the background. A model that is not installed, or failed to load, is
refused either way.
- roto.status reports model {selected, installed, active, loading, error} and a notice while the
chosen model is not the one in use. source is "model" for a frame the model segmented on its own,
instead of "classical".
- Auto-propagate keeps edited instances pending while the model loads, and shows errors as toasts
instead of dropping them.
- Docs: what propagate does now; tracked frames live in the session that ran propagate and are not
saved in the project file, so a separate render process does not show them; roto.model.select
lists sam2.1-hiera-tiny.
Tests: a selected model that is not installed refuses to propagate (no weights needed); SAM 2
selected without wait is loaded before propagate (EFFECTCRAFT_SAM2). The SAM 2 tests and the
failing-tracker test take the Roto test lock, since they share the process-wide model slot.
Claude-Session: https://claude.ai/code/session_0136kZtXrYo5VUWEFxWduwUD
Bit-for-bit test: every frame of the Roto Brush chain equals a direct
segment_with / propagate_with call with no model.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Test with a tracker that always errors: roto.propagate completes, every frame
comes from the classic engine (status source: classical), IoU unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
When the chosen model has a tracker, roto.propagate runs one tracking pass over
the span: prompts from the strokes (a handful of points, which SAM 2 trusts),
the clip tracked both ways, each frame's probability map refined by the same
graph cut with the strokes as hard constraints. Results are cached under their
own keys with a per-frame confidence and an occluded flag; roto.status reports
them (source: tracker|classical). Without a tracker nothing changes.
Real weights: IoU 0.98 on all 21 frames, confidence 0.99.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
maxaeexe