Sine position encodings depend only on the sizes: built once (150 ms of a frame).
do_pool takes the maximum over grouped cells instead of permute + max_pool2d +
permute (tested equal, odd sizes included). Encode 2.6 s -> 2.4 s per frame in
release on 16 threads (3.4 s at the start of the speed work); reference tests
unchanged. Chunked global attention was tried and did not help.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Candle's gelu_erf runs on one thread and was a third of the Hiera block time.
A rayon CustomOp1 calls the same erf per element (bit-identical, tested). All
reference tests unchanged. op_bench example times the block's operations.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Encode 5.0 s -> 3.4 s per 1024x1024 frame in release on 16 threads (fused
softmax, queries scaled before the matmul, flat 2-D linear layers); a click
on an encoded frame is about 120 ms. Reference tests unchanged.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
