Merge origin/main into the NVENC branch. #183 already provides the
HardwareEncoding enum, ExportSettings.hardware_encoding, the
hardwareEncoding export param and the Export > Video > Hardware Encoding
checkbox, so the duplicates from this branch are dropped and NVENC uses
them. filmcraft-export is now an all-platform dependency of
filmcraft-platform (VideoToolbox and NVENC both register encoders).
Never-crash fixes in the NVENC backend:
- submit() rejects a driver input pitch narrower than a row (and unlocks
the buffer); fill_nv12 uses get_mut instead of slicing rows.
- composition offsets use saturating math and i32::try_from; frame-rate,
size and timestamp casts are checked; sizes above u16::MAX decline.
- Annex B scanning and NAL type checks use get()/first().
- A failed nvEncOpenEncodeSessionEx that returns a handle destroys it.
ffi.rs carries NVIDIA's MIT notice for nvEncodeAPI.h, listed in
ATTRIBUTION.md under a new "Third-party source code" section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extends the Windows backend (#141) to VP9 (profiles 0 and 2) and AV1 (main, 8/10-bit), 4:2:0.
- codecs::hw: FrameStreamInfo (vpcC / av1C): profile, bit depth, chroma, picture size and colour as
the software decoders report them (vp9_color / av1_color are now shared with them), random-access
samples, in-band format changes; StreamInfo lets HybridDecoder handle NAL and frame-coded streams.
- platform: media_foundation::stream (what each codec needs: DXVA profile, MFT request, input
format), MfDecoder generalised; VP9 frames / AV1 temporal units go in as they are; the MFT's
declared picture size (display aperture) is checked against the bitstream's.
- Declined up front: VP9 profiles 1/3, AV1 profiles 1/2, 4:2:2/4:4:4, mono, 12-bit, no codec
extension, no DXVA profile. A key frame of another size or a different sequence header falls back
to the software decoder.
- Tests: bit-exact parity (360p / 1080p / 2160p, seeks, resets, flush, hidden frames), forced
failure, damaged samples and av1C, format change fallback, WebM / Matroska through the media stack.
- Bench: AV1 fixtures with libaom when ffmpeg lacks SVT-AV1.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
Problem: playback decoded and mixed every ~10 ms device buffer inside the
cpal callback (AAC decode, effects, opening a clip's file at a cut, rayon,
locks). On a loaded Mac any stall missed the device deadline: static and
dropouts, different on every play.
Solution: play_ahead mixes on its own thread into a lock-free ring kept
200 ms ahead; the callback only copies. An underrun plays silence, is
counted, and the mixer skips the same span so sound stays locked to the
picture. Any device width plays the channels it shares with the mix.
perf.stats reports playback.audio (callbacks, underruns, missingFrames,
minLeadMs). The program mix moved to playback_mix so bench_audio can play a
real project through a simulated 48 kHz device and count dropouts.
Context: on a real 22-sequence project, load average 38-58, 15 s per cut:
mixing in the callback overran 8-170 times, mixing ahead 0 (ring never
below 149 ms). The web build keeps mixing in its callback (no threads).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ffmpeg prints "corrupt decoded frame" as a warning, so the oracle's
`-v error -ec 0` decode let a stream with a whole slice missing pass
silently. Switching that decode to `-v warning` is not enough: with
`-ec 0`, ffmpeg 7.1/8.1 flags every picture that has more than one
slice, libx264's multi-slice streams included, while the pixels are
unchanged. That is the 119/120 in #74; the encoder's bitstream is not
at fault (VideoToolbox decodes the same streams bit-identically, and
ffmpeg with strict error detection reports nothing).
The oracle now decodes with ffmpeg's default concealment, `-v warning`
and `-err_detect +crccheck+bitstream+buffer+explode`, fails on any
output or a non-zero exit, and still compares every pixel with the
encoder's reconstruction. A negative control drops and truncates
slices (CAVLC and CABAC, 3 slices) and checks the oracle reports each
one; it fails with the old flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem: a clip with AAC audio in MP4 whose sample rate differs from the
sequence's (48 kHz clip in a 44.1 kHz sequence) exported with loud
low-frequency thumps, several times full scale, recurring through the clip.
Every exported container was affected (AAC clips them into thumps, PCM
clips at full scale); a WAV export had a single one at the clip's start.
Cause: Mp4Source::audio decodes a window starting at s0, the requested
presentation sample minus the edit-list offset, then resamples it. The
resampler derived its sub-sample phase from s0, so the edit-list offset
(-1024 samples of AAC priming) was applied a second time: the read
position started about 1024 samples before the window, was clamped to
index 0 with a large negative fraction, and linear interpolation
extrapolated a ramp over the first ~940 samples of every request. Export
asks for audio one batch of video frames at a time, so the ramp repeated
at each request. Sources at the sequence rate take the copy path and
were unaffected, as were tracks without an edit list.
Fix: take the phase from the presentation sample the window starts at
(before the edit-list offset is subtracted).
Test: aac_mp4_resampled_reads_follow_the_edit_list
(crates/codecs/tests/mp4_oracle.rs, with `require_ffmpeg!` and the
shared fixture helper as docs/testing.md asks of new oracle tests) reads
an ffmpeg AAC fixture at 44.1 kHz in frame-sized pieces and compares it
with the 48 kHz decode interpolated at the same times (peak was 7.35
against a source level of 0.15 before the fix). docs/testing.md lists it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New crates/platform (filmcraft-platform, L5): OS media FFI only. The one
crate allowed `unsafe` (own [lints] with unsafe_code = "deny",
#[allow(unsafe_code)] only on the videotoolbox module, SAFETY comment on
every block, safe Result-returning public API). AGENTS.md §0.3 names the
exception; docs/adr/0001-platform-ffi.md records why and the rules.
- VideoToolbox decoder for avcC / hvcC, 8/10-bit 4:2:0 and 4:2:2: session from
the sample entry's parameter sets with hardware required, asynchronous
decode (2 in flight), biplanar CVPixelBuffer copied to planar Yuv8/Yuv16,
reorder buffer of the stream's own depth, RASL skipped after a starting CRA.
Output callback under catch_unwind.
- HybridDecoder: transparent mid-stream fallback to the software decoder
(replays samples since the last restart point, drops already-returned
pictures). New codecs::software_video_decoder (built-in factories only) and
codecs::hw (switch, counters, NalStreamInfo shared with software decoders;
vui_color / sar_par helpers now used by both).
- Settings ▸ Playback ▸ Hardware decoding: Auto / Off (engine applies it to
the codecs switch); desktop app, CLI/MCP and bench call register().
- perf.stats decode.framesDecoded and decode.hardware counters.
- Tests: bit-exact parity with the software decoders (H.264 High, HEVC Main
open GOP, Main 10; resets, reseeks, flushes), forced-failure fallback,
mutation fuzz of samples and parameter sets, cross-platform fallback with a
stand-in decoder, setting Off, HEVC 4:2:2 vs ffmpeg.
- Bench --hw auto|off; results in docs/performance.md (4K CPU/frame
~86-119 ms -> ~4 ms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All 27 blend modes now composite on the GPU instead of sending the
whole frame to the CPU.
- render::plan: PlanLayer carries its Blend; plan_frame no longer falls
back to a CPU frame for non-Normal modes. Media, graphic clips,
multicam angles and CPU-rendered layer images (standard effects) keep
their clip's mode; inside a transition the mode is Normal, as in the
CPU reference. execute_cpu composites with each layer's mode.
- gpu: Normal keeps the single fixed-function pass. Dissolve also uses
fixed-function "over": the CPU's 64-bit pixel hash is reproduced in
WGSL on u32 pairs (pattern bit-exact). The other 25 modes copy the
accumulator region under the layer's quad into a backdrop texture and
composite in fs_blend with the CPU math (sRGB-encoded straight colour,
W3C formulas, back to linear, same premultiplied mix and clamping).
- Tests: per-mode GPU-vs-CPU parity (gpu crate: overlapping transformed
layers, partial opacity/alpha, empty/partial backdrop; golden: every
mode through plan_frame incl. an effect layer, blended clips in a
cross dissolve), exact-value edge cases (black/white, Divide by 0,
Dodge/Burn at 0 and 1, zero alpha) within 4.9e-4, Dissolve pattern
exact, off-output layers, quad bounds.
- Bench: new stack3-blend playback scenario (V2 Screen, V3 Overlay);
CPU ms/frame 151-214 -> 86-89, Full 19/173 and 138/54 -> 191/1, 192/0.
- Docs: architecture §5, gpu crate docs, testing, performance.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Lumetri grades PQ / HLG working spaces in their own signal normalised to
HDR White (Basic Correction) / HDR Range (Curves) in cd/m²
(filmcraft_color::GradeSpace): exposure in stops of light, highlights above
HDR White kept, HDR Specular scales them; Rec. 709 unchanged.
- HDR static metadata (mdcv/clli, Matroska MasteringMetadata/MaxCLL/MaxFALL)
on VideoStreamInfo::hdr; PQ media use MaxCLL / mastering peak as the
BT.2390 source peak; media.colorInfo reports it.
- HSL Secondary Refine: Denoise (median of the key) and Blur (Gaussian).
- Lumetri Presets (18 original looks in 4 folders) with thumbnails rendered by
our Lumetri on a procedural preview picture; Effects panel rows and a
thumbnail grid when wide; lumetri.presets / applyPreset / presetThumbnails.
- Color Match: Comparison View toggle; Apply Match uses its reference frame.
- Goldens lumetri_hdr_pq and lumetri_presets_grid; D-Log M stays unsupported
(no published formula).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
List / Icon / Freeform views, Metadata Display, view presets, font size,
Preview Area, bin tabs and windows, hover-scrub In/Out; Media Browser tree,
Favorites, history, file types, thumbnails, Edit Columns, import and drag.
Engine: project_panel and media_browser modules, Services::list_entries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Interleaved before/after bench (decode, playback incl. 1/4 and draft cases,
4K cold seeks) and decoder cycle counts in docs/performance.md; threading,
draft mode and 4K numbers in crates/h264/README.md; bench cases in
docs/testing.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5.1 / multichannel:
- audio-dsp::channels: layouts (L R C LFE Ls Rs), ITU-R BS.775 up/downmix, 5.1 Mixdown Types,
equal-power 5.1 panner (puck, Center %, LFE).
- Mixer graph buses are 2 or 6 channels wide; 5.1 tracks/submixes/Mix; strips feeding a 5.1 bus
use the 5.1 panner (lanes pan51.x/y/center/lfe stored in MixerStrip::lanes, no schema change);
5.1 strips fold into stereo buses with BS.775; per-channel meters; clip effects on 6 channels.
- file.newSequence {mix, trackType}, sequence.settings {mix}; playback honours the 5.1 Mixdown
Type preference on stereo devices and plays 6 channels on 5.1 devices.
- Export Audio Channels 5.1: WAV WAVE_FORMAT_EXTENSIBLE (mask 0x3F), QuickTime PCM with a chan
atom, AAC channel configuration 6 (reordered to C L R Ls Rs LFE).
- Track Mixer: 5.1 panner puck with Center/LFE, N-channel meters, Show/Hide Tracks, Meter
Input(s) Only; transport record button runs voice-over.
Voice-over: engine AudioInput trait + SyntheticInput, cpal input in the app, audio.voiceover.*
commands (punch-in, pre/post-roll, countdown cues, WAV next to project / scratch disk, one undo
step), track header mic button and Voice-Over Record Settings dialog.
Remix: beat tracking + chroma/cepstra self-similarity planner in audio-dsp, hidden `remix` clip
effect holding the plan, render::remix playback with equal-power crossfades, clip.remix.*
commands, Remix tool and Remix Properties dialog.
Clip Mixer automation: per-track Latch/Touch/Write (clipMixer.setMode/touch/release) heard live
and written as thinned clip Volume/Panner keyframes at the end of the pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- HEVC inverse transform: sum basis rows of non-zero coefficients with a
const-generic size (bit-exact, tested against the direct product);
SAO edge offset gets a check-free interior loop. HEVC CPU/frame -60%.
- H.264/HEVC/VP9 planes are moved into VideoFrames, not copied.
- Catch-up decoding: frame jobs tell sources which frames are late
(filmcraft_media::cancel::with_catch_up); the GOP cache skips
non-reference samples of late frames (H.264 nal_ref_idc 0, HEVC
sub-layer non-reference pictures). Scrubs keep the last 2 s decoded.
- perf.stats: engine query + UI/control method (decode counters,
playback shown/dropped, frame-worker decode/render ms, cache use).
- Timeline waveform source peak cached; bench seek section and --only.
- docs/performance.md with before/after results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- project: vtransition.rs registry. Animation, Dissolve, Grunge & Distort,
Immersive Video, Lights & Blurs, Slide, Smart Tools, Text, Transformers
and Wipe folders, plus Legacy > Video Transitions. Old ids are kept:
modern versions reuse push/slide/split/whip/clock_wipe/radial_wipe/
cross_zoom/page_peel, the old behaviour lives under *_legacy ids, and
removed transitions move to a hidden Obsolete folder. Parameters are
not animatable; find_transition looks up by kind (Mosaic/Mirror/
Directional Blur names are shared with video effects).
- render: transitions/ module with procedural CPU implementations, a
field-wipe engine (feather, border width/colour, anti-aliasing, centre),
a 3D card renderer (flips, cube, folds, spins), page curl, blur/light/
grunge families and flat VR approximations. Endpoints are exact,
pixel params scale with the render resolution, and Reverse now plays
B->A backwards. There is no GPU transition path, so the plan keeps
only the symmetric dissolves on the GPU.
- engine: applyVideoTransition takes params/reverse; new
sequence.setTransition; effects.list adds folder/kind filters,
badges and detail paramInfo; sequence.inspect shows transition params.
- ui: the Effects panel shows the Legacy top folder and all badges;
ui.set effectsSearch.
- tests: per-transition endpoint, continuity, midpoint, determinism,
NaN, parameter-wiring and direction properties; golden fingerprints;
tree counts; apply-every-transition via command; UI panel test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One Preferences window (app menu ▸ Settings ▸ <category>, Edit ▸ Preferences in-window, General =
Cmd+,): category list, schema-driven pages, Help / Reset… / Cancel / OK. Settings live in the
per-user preferences.json (version 2, v1 migrated, bad values repaired), reachable via
prefs.get/set/reset{category} and the new prefs.schema; app.settings.<category> opens a page;
every control has a settings.<key> automation id.
Wired: still/transition default durations, auto-scroll, snap playhead, return to beginning, play
after rendering, step many, selection-tool roll/ripple, theme/highlight/contrast, tooltips, at
startup + recent projects, label names/colours/defaults, indeterminate timebase, default media
scaling, media cache location/policy/clean, frame cache budget, cpal device/buffer/rate/output
mapping, transcription defaults + auto-transcribe on import, text smart quotes/ligatures/font.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MonitorView state is serde and settable with ui.set {program|source}; view.* UI commands with
menu checkmarks; wrench and zoom menus with automation ids; headless kittest tests (view_ui.rs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New L0 crate filmcraft-dnx, written from SMPTE ST 2019-1:2016 + Amd 1:2023
(tables extracted from the standard's text by tools/extract_tables.py).
Decoder: every compression ID (DNxHD 1235-1260 incl. field-coded and
macroblock-adaptive interlace, thin rasters, RGB 4:4:4; DNxHR 1270-1274
LB/SQ/HQ/HQX/444), 8/10/12-bit, 4:2:0/4:2:2/4:4:4, DCT and lossless alpha,
macroblock rows in parallel. Matches ffmpeg within IDCT precision (max 2 LSB,
<=50 per million beyond 1, 87-99% identical) on 22 ffmpeg fixtures; follows
ffmpeg where it departs from the standard's text (reconstruction offset,
CID 1271 weights, 4:4:4 signalling) - documented in the README.
Encoder: DNxHR LB/SQ/HQ/HQX(10/12-bit)/444, CBR at the eq. 7.1 frame sizes,
uniform-qsf binary search + per-macroblock refinement; ffmpeg decodes every
profile with -xerror (HQ luma PSNR 58.8-66.3 dB).
Wiring: MOV AVdn/AVdh import (codec label from the CID), export
Format::DnxHr (MOV AVdh, dnxProfile lb|sq|hq|hqx) in the export dialog and
file.exportMedia.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- render::color_match: non-ML statistics matching — Oklab means per tonal range (smoothstep
shadow/midtone/highlight weights) plus mean chroma; "Face Detection" down-weights skin-tone
pixels (Oklab hue 20–75°) in the colour statistics and then limits the skin hue shift to 8°;
the grade is solved in Lumetri's own parameters (3 wheels, their lightness, saturation) by
damped Gauss–Newton through the real Lumetri implementation, so the result stays editable
- render::render_clip: one clip's layer at a time (the shot as Lumetri sees it)
- lumetri.applyMatch {clip?, referenceTime|Frame|Timecode, faceDetection} sets the grade in one
undo step and reports the statistics distance before/after; refuses a reference inside the
clip itself
- Lumetri panel: Color Match row (reference timecode, Face Detection, Apply Match; automation
ids lumetri.match.*)
- tests: Oklab reference values, self-match is a no-op, a known grade is recovered (residual
−86 %), skin protection holds the skin hue shift to ≤ 8° (51° without), engine and headless
UI tests
- docs: control protocol (colour commands and ids), testing; ROADMAP M8 row and log
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
