Pin GOP-cache eviction on the requested pts so a burst past the playhead
does not drop the target and walk the GOP again. Preroll only schedules
the frames that gate clock start. bench_playback no longer meters preroll
jobs as on-clock latency.
This is the seek stall from #848, not the ~3× per-core gap vs ffmpeg.
GPU export rendering was ~20 % slower in wall-clock time than CPU
rendering: one renderer behind a mutex made the export workers queue.
- Export workers share a pool of up to four renderers on one wgpu device
(one per worker was no faster: the read-back then contends for memory
bandwidth).
- render_export_prepared composites into the linear accumulator and
copies it to a reused staging buffer in one submission (no sRGB resolve
pass); read_accumulator shares the read-back, which now returns None
instead of stale pooled values when a row is missing.
- f16_to_f32_fast: same result as f16_to_f32 for all 65536 inputs (test).
- Timing counters (export::export_lock_wait, gpu::export_renderer::timing)
printed by the export bench.
RX 7900, same build, Off / Auto / Off: H.264 + AAC 4.67 / 4.71 / 4.76 s,
ProRes 422 4.20 / 4.21 / 4.21 s; CPU per frame 287-292 -> 277 (H.264),
316-317 -> 300 (ProRes). Was 4.86 -> 5.78 s. Parity, not faster: on this
single-clip scene decoding and encoding dominate (rendering is ~15 of
~290 CPU ms per frame), so GPU rendering stays Off by default.
Gates: fmt, clippy (gpu, export, ui-egui), layers, assets; gpu
export_pipeline, golden and export tests pass. Pre-existing on the
previous commit: fx_tests::gpu_effects_match_cpu_exactly fails on this
AMD GPU, and the gpu unit tests segfault in the driver when run in
parallel (pass serially apart from that test).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add off-screen GPU export renderer (filmcraft-gpu::ExportRenderer) reading back
the linear float accumulator as premultiplied RGBA f32, plug it into the
export pipeline via FrameRenderer hook and GpuRendering setting, register
in desktop app and CLI, and record export.gpu_frames statistics.
🤖 Completed by agy (Antigravity Agent, Gemini 3.8 Flash) based on draft by Merlin
Merge origin/main into the NVENC branch. #183 already provides the
HardwareEncoding enum, ExportSettings.hardware_encoding, the
hardwareEncoding export param and the Export > Video > Hardware Encoding
checkbox, so the duplicates from this branch are dropped and NVENC uses
them. filmcraft-export is now an all-platform dependency of
filmcraft-platform (VideoToolbox and NVENC both register encoders).
Never-crash fixes in the NVENC backend:
- submit() rejects a driver input pitch narrower than a row (and unlocks
the buffer); fill_nv12 uses get_mut instead of slicing rows.
- composition offsets use saturating math and i32::try_from; frame-rate,
size and timestamp casts are checked; sizes above u16::MAX decline.
- Annex B scanning and NAL type checks use get()/first().
- A failed nvEncOpenEncodeSessionEx that returns a handle destroys it.
ffi.rs carries NVIDIA's MIT notice for nvEncodeAPI.h, listed in
ATTRIBUTION.md under a new "Third-party source code" section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NVENC through nvEncodeAPI64.dll (API 12.1, FFI layout checked against the C header by generated
tests), registered as an Export encoder in front of the software one; Export > Hardware encoding
(off by default). Declines two-pass VBR, HDR, MXF, unsupported sizes; B-frames, IDR/GOP, VUI, avcC.
Tests: decode with our decoder (PSNR), pipeline export vs software, decline paths.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
Extends the Windows backend (#141) to VP9 (profiles 0 and 2) and AV1 (main, 8/10-bit), 4:2:0.
- codecs::hw: FrameStreamInfo (vpcC / av1C): profile, bit depth, chroma, picture size and colour as
the software decoders report them (vp9_color / av1_color are now shared with them), random-access
samples, in-band format changes; StreamInfo lets HybridDecoder handle NAL and frame-coded streams.
- platform: media_foundation::stream (what each codec needs: DXVA profile, MFT request, input
format), MfDecoder generalised; VP9 frames / AV1 temporal units go in as they are; the MFT's
declared picture size (display aperture) is checked against the bitstream's.
- Declined up front: VP9 profiles 1/3, AV1 profiles 1/2, 4:2:2/4:4:4, mono, 12-bit, no codec
extension, no DXVA profile. A key frame of another size or a different sequence header falls back
to the software decoder.
- Tests: bit-exact parity (360p / 1080p / 2160p, seeks, resets, flush, hidden frames), forced
failure, damaged samples and av1C, format change fallback, WebM / Matroska through the media stack.
- Bench: AV1 fixtures with libaom when ffmpeg lacks SVT-AV1.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
New backend in crates/platform (media_foundation/): a Direct3D-aware decoder MFT is driven at the
level of single access units (Annex B), given the process's D3D11 video device through an
IMFDXGIDeviceManager so it decodes with DXVA, and its NV12 / P010 textures are read back through a
staging texture into planar Yuv8 / Yuv16. H.264 Baseline/Main/High and HEVC Main / Main 10, 4:2:0.
- Hardware is verified: non-D3D-aware or async MFTs, streams the GPU's DXVA decoder does not list,
and system-memory output (Microsoft's MFT software fallback) are declined or fail the stream so
HybridDecoder continues with our decoder.
- mfplat.dll is loaded at run time so Windows N editions still start.
- register() / registered() / hardware_decoder_for() cover Windows; perf.stats reports the backend.
- Tests (Windows): bit-exact parity with the software decoders at 640x360, 1080p and 2160p
(resets, seeks, flush), forced mid-stream failure, damaged input, concurrency, repeated init.
- Bench: runs on Windows (ffmpeg on PATH, process/thread CPU clocks), HEVC Main 10 2160p row,
hardware counters per row.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
A media clip whose enabled standard effects all have a GPU implementation
plans as a GPU layer carrying them (render::plan::LayerFx): parameters are
evaluated on the CPU at the frame's time (render::gpufx::FxOp::eval, shared
with the CPU path so both clamp hostile values identically), the source is
drawn into an Rgba32Float working image at the size the CPU decodes it, the
effects run as WGSL compute passes (gpu::fx, fx.wgsl) with the CPU
reference's math, and the result is placed with Motion, opacity and blend
mode as before. Layers without effects keep the single draw.
GPU effects: Brightness & Contrast, ProcAmp, Tint, Black & White, Color
Balance, Leave Color, Change to Color, Color Pass, Color Replace, Channel
Mix, ASC CDL, Gamma Correction, Levels, Extract, Invert, Posterize, Alpha
Adjust, Gaussian / Camera / Directional Blur (+ legacy aliases), Sharpen,
Unsharp Mask, Crop, Edge Feather, Transform, H/V Flip, Mirror, Offset.
Masked effects, opacity masks, non-finite parameters and minifying
Transform/Crop zooms stay on the CPU. Devices without compute shaders run
the effects on the CPU inside the compositor.
Also fixes a debug-build overflow in Image::transformed for non-finite maps.
Parity tests (exact f32 checks and composited plans), a golden test that
plans effect chains as GPU layers, and the stack3-fx playback benchmark.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New crates/platform (filmcraft-platform, L5): OS media FFI only. The one
crate allowed `unsafe` (own [lints] with unsafe_code = "deny",
#[allow(unsafe_code)] only on the videotoolbox module, SAFETY comment on
every block, safe Result-returning public API). AGENTS.md §0.3 names the
exception; docs/adr/0001-platform-ffi.md records why and the rules.
- VideoToolbox decoder for avcC / hvcC, 8/10-bit 4:2:0 and 4:2:2: session from
the sample entry's parameter sets with hardware required, asynchronous
decode (2 in flight), biplanar CVPixelBuffer copied to planar Yuv8/Yuv16,
reorder buffer of the stream's own depth, RASL skipped after a starting CRA.
Output callback under catch_unwind.
- HybridDecoder: transparent mid-stream fallback to the software decoder
(replays samples since the last restart point, drops already-returned
pictures). New codecs::software_video_decoder (built-in factories only) and
codecs::hw (switch, counters, NalStreamInfo shared with software decoders;
vui_color / sar_par helpers now used by both).
- Settings ▸ Playback ▸ Hardware decoding: Auto / Off (engine applies it to
the codecs switch); desktop app, CLI/MCP and bench call register().
- perf.stats decode.framesDecoded and decode.hardware counters.
- Tests: bit-exact parity with the software decoders (H.264 High, HEVC Main
open GOP, Main 10; resets, reseeks, flushes), forced-failure fallback,
mutation fuzz of samples and parameter sets, cross-platform fallback with a
stand-in decoder, setting Off, HEVC 4:2:2 vs ffmpeg.
- Bench --hw auto|off; results in docs/performance.md (4K CPU/frame
~86-119 ms -> ~4 ms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All 27 blend modes now composite on the GPU instead of sending the
whole frame to the CPU.
- render::plan: PlanLayer carries its Blend; plan_frame no longer falls
back to a CPU frame for non-Normal modes. Media, graphic clips,
multicam angles and CPU-rendered layer images (standard effects) keep
their clip's mode; inside a transition the mode is Normal, as in the
CPU reference. execute_cpu composites with each layer's mode.
- gpu: Normal keeps the single fixed-function pass. Dissolve also uses
fixed-function "over": the CPU's 64-bit pixel hash is reproduced in
WGSL on u32 pairs (pattern bit-exact). The other 25 modes copy the
accumulator region under the layer's quad into a backdrop texture and
composite in fs_blend with the CPU math (sRGB-encoded straight colour,
W3C formulas, back to linear, same premultiplied mix and clamping).
- Tests: per-mode GPU-vs-CPU parity (gpu crate: overlapping transformed
layers, partial opacity/alpha, empty/partial backdrop; golden: every
mode through plan_frame incl. an effect layer, blended clips in a
cross dissolve), exact-value edge cases (black/white, Divide by 0,
Dodge/Burn at 0 and 1, zero alpha) within 4.9e-4, Dissolve pattern
exact, off-output layers, quad bounds.
- Bench: new stack3-blend playback scenario (V2 Screen, V3 Overlay);
CPU ms/frame 151-214 -> 86-89, Full 19/173 and 138/54 -> 191/1, 192/0.
- Docs: architecture §5, gpu crate docs, testing, performance.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frames are published one superblock row at a time once loop filtered; a post stage per frame
(own thread) assembles bands from the tile strips, filters them in raster order and builds the
output picture, while the next frame's tile columns decode and wait per band for the reference
rows they read. Bit-exact with 1, 3 and all threads. Draft mode skips the loop filter of frames
that refresh no reference slot. av1dec / hevcdec bench knobs and VP9 / AV1 4K playback scenarios
in the bench.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings > Playback > Draft decoding (off by default): the Program monitor
asks for draft frames only while playing at 1/2 resolution or lower
(FrameKey::draft, filmcraft_media::cancel::with_draft). The GOP cache runs
the decoder in draft mode for those requests and serves draft frames to draft
requests only, so paused frames, renders and exports always get exact
pictures (codecs test: exports after draft playback match ffmpeg). Draft
plans hand the GPU box-decimated YUV planes at the drawn size. perf.stats
reports draft frames, H.264 threads and the preference; the bench gains
2160p quarter and draft playback cases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- HEVC inverse transform: sum basis rows of non-zero coefficients with a
const-generic size (bit-exact, tested against the direct product);
SAO edge offset gets a check-free interior loop. HEVC CPU/frame -60%.
- H.264/HEVC/VP9 planes are moved into VideoFrames, not copied.
- Catch-up decoding: frame jobs tell sources which frames are late
(filmcraft_media::cancel::with_catch_up); the GOP cache skips
non-reference samples of late frames (H.264 nal_ref_idc 0, HEVC
sub-layer non-reference pictures). Scrubs keep the last 2 s decoded.
- perf.stats: engine query + UI/control method (decode counters,
playback shown/dropped, frame-worker decode/render ms, cache use).
- Timeline waveform source peak cached; bench seek section and --only.
- docs/performance.md with before/after results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decode fps per codec (H.264, HEVC, VP9, AV1, ProRes at 1080p/2160p),
Program-monitor playback and scrubbing through the real frame scheduler,
timeline UI frame time with a 1000-clip / 20-track sequence under
egui_kittest (+ wgpu render), export throughput (H.264 + AAC, ProRes),
project save/open of 5000 clips, and peak RSS per section (each section
runs in its own process under /usr/bin/time). Writes JSON and Markdown
to target/bench. bench_playback now shares its code (examples/bench/).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
