Memory attention (1.8e-5 vs PyTorch) and the memory encoder (2.6e-6) as wgpu kernels,
convolution and layer-norm+GELU kernels, VideoModel::load_auto picks the GPU when
there is one (EFFECTCRAFT_SAM2_DEVICE overrides) and falls back to the CPU on any GPU
error; disable_gpu switches it off mid-clip. One process-wide device (several at once
crashed the Vulkan loader). A tracked frame: 4.3 s CPU, 0.7 s on the RX 7900 GRE.
Clips tracked on the GPU match Sam2VideoModel (worst logit error 0.0032).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A tracked frame is the encode (2.4 s) plus memory attention (2.4 s with a full
bank), about 5 s on 16 threads, not one encode as the docs said. Docs corrected;
chunked attention tried and reverted.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sine position encodings depend only on the sizes: built once (150 ms of a frame).
do_pool takes the maximum over grouped cells instead of permute + max_pool2d +
permute (tested equal, odd sizes included). Encode 2.6 s -> 2.4 s per frame in
release on 16 threads (3.4 s at the start of the speed work); reference tests
unchanged. Chunked global attention was tried and did not help.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Encode 5.0 s -> 3.4 s per 1024x1024 frame in release on 16 threads (fused
softmax, queries scaled before the matmul, flat 2-D linear layers); a click
on an encoded frame is about 120 ms. Reference tests unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
