--clip FRAMES_DIR --truth MASKS_DIR runs the trackers on image sequences with true
mattes (strokes made from the truth; nothing is bundled or copied), tested end to end.
Trackers report the most memory their banks held (peakMB column).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Memory attention (1.8e-5 vs PyTorch) and the memory encoder (2.6e-6) as wgpu kernels,
convolution and layer-norm+GELU kernels, VideoModel::load_auto picks the GPU when
there is one (EFFECTCRAFT_SAM2_DEVICE overrides) and falls back to the CPU on any GPU
error; disable_gpu switches it off mid-clip. One process-wide device (several at once
crashed the Vulkan loader). A tracked frame: 4.3 s CPU, 0.7 s on the RX 7900 GRE.
Clips tracked on the GPU match Sam2VideoModel (worst logit error 0.0032).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
VideoTracker::set_budget (default 2 GiB of state, optional seconds per
propagate call) stops with Error::Budget; state_bytes reports what it holds.
Two runs of the same prompts give bit-identical masks.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Antialiased bilinear resize as PyTorch does it, the reference's mask-as-output
path (mask is the frame's output, the decoder makes the object pointer),
VideoTracker::prompt_mask, and mask-only prompts through the tracker interface.
Prompt frame logits exact, tracking forwards and backwards within 0.0019.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
TrackRequest takes a list of objects (each with its prompts on its frames) and
the tracker reports (object, frame, result); frame encodings are shared, objects
may start on different frames. VideoTracker::propagate_objects runs a subset.
Limits: 32 objects. Test: two discs, one prompted on frame 0 and one on frame 2.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ObjectTracker interface in the segment crate (prompts on some frames, the
clip tracked both ways, each frame streamed with confidence and an occluded
flag), implemented by SAM 2. MaskModel::tracker() is None for other models.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Per-object memory banks, prompts on any frame, forward and backward
propagation with streaming results, cancellation, object pointers with
temporal encoding, pruning of memories no later frame can use, and checked
input handling. Hostile-input and cancel tests.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Kristian Myram