aleleba/effectcraft
1
0
Fork 0
mirror of https://github.com/storytold/effectcraft.git synced 2026-11-11 16:33:01 -06:00
Code Issues Packages Projects Releases Wiki Activity

S2.63: GPU memory attention and memory encoder; one shared device; CPU fallback

Browse Source 
Memory attention (1.8e-5 vs PyTorch) and the memory encoder (2.6e-6) as wgpu kernels,
convolution and layer-norm+GELU kernels, VideoModel::load_auto picks the GPU when
there is one (EFFECTCRAFT_SAM2_DEVICE overrides) and falls back to the CPU on any GPU
error; disable_gpu switches it off mid-clip. One process-wide device (several at once
crashed the Vulkan loader). A tracked frame: 4.3 s CPU, 0.7 s on the RX 7900 GRE.
Clips tracked on the GPU match Sam2VideoModel (worst logit error 0.0032).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Kristian Myram
2026-10-30 08:43:31 +01:00
co-authored by Claude Sonnet 5.5
parent 5c7eb3c2ae
commit ee3498b24d
16 changed files with 861 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/engine/src/cmd/roto_cmds.rs
+1 -1
View File
@@ -478,7 +478,7 @@ fn status(s: &mut Session, p: &Value) -> Result<Value> {
.keys()
.filter_map(|f| chain.trust_at(*f).map(|t| (f.to_string(), json!({"confidence": t.confidence, "occluded": t.occluded}))))
.collect();
out["tracker"] = json!({"model": s.models_roto_active(), "saved": chain.saved_model(), "frames": tracked});
out["tracker"] = json!({"model": s.models_roto_active(), "device": fx::installed_model().and_then(|m| m.device_name()), "saved": chain.saved_model(), "frames": tracked});
let notice = s.roto_model_notice(s.uses_roto_model(cid, lid, uid)).or_else(|| s.saved_tracking_notice(chain.saved_model()));
out["notice"] = json!(notice);
out["source"] = json!(if tracked.contains_key(&frame.to_string()) {