S2.41: Roto Brush follows the object with SAM 2

When the chosen model has a tracker, roto.propagate runs one tracking pass over
the span: prompts from the strokes (a handful of points, which SAM 2 trusts),
the clip tracked both ways, each frame's probability map refined by the same
graph cut with the strokes as hard constraints. Results are cached under their
own keys with a per-frame confidence and an occluded flag; roto.status reports
them (source: tracker|classical). Without a tracker nothing changes.
Real weights: IoU 0.98 on all 21 frames, confidence 0.99.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Kristian Myram
2026-10-30 08:43:30 +01:00
co-authored by Claude Sonnet 5.5
parent a62d59da4b
commit c1f5ff9f7d
7 changed files with 268 additions and 5 deletions
+13
View File
@@ -463,6 +463,19 @@ fn status(s: &mut Session, p: &Value) -> Result<Value> {
}
}
}
// Where each frame's matte came from, and how far to trust it: frames a tracker followed
// report its confidence and whether the object was absent; the rest came from the classic engine.
{
let size = rt::layer_size(&s.project, &comp, &layer);
let chain = fx::Chain::new(&params_static(&g), size, comp.frame_rate.as_f64(), 1.0);
let tracked: serde_json::Map<String, Value> = chain
.keys
.iter()
.filter_map(|(f, k)| fx::trust(*k).map(|t| (f.to_string(), json!({"confidence": t.confidence, "occluded": t.occluded}))))
.collect();
out["tracker"] = json!({"model": s.models_roto_active(), "frames": tracked});
out["source"] = json!(if tracked.contains_key(&frame.to_string()) { "tracker" } else { "classical" });
}
let computed = s.roto_computed(cid, lid, uid);
out["computed"] = json!(computed.iter().filter(|(_, c)| **c).map(|(f, _)| *f).collect::<Vec<_>>());
Ok(out)