Commit Graph
73 Commits
Author SHA1 Message Date
Brandon Thomas fcf1c45583 Merge pull request #141 from SaymonMaster14/feat/windows-media-foundation-hwdecode
Windows: Media Foundation/D3D11 hardware H.264 + HEVC decode

# Conflicts:
#	README.md
#	ROADMAP.md
#	crates/platform/Cargo.toml
#	crates/platform/src/lib.rs
#	docs/architecture.md
#	docs/performance.md
2026-10-07 17:48:06 -04:00
Brandon Thomas b5f197567b Merge pull request #186 from dcarnelutti/platform/videotoolbox-hevc-encode
platform: H.265 (HEVC) export with VideoToolbox

# Conflicts:
#	crates/export/src/settings_tests.rs
#	crates/ui-egui/tests/export_ui.rs
2026-10-07 17:47:10 -04:00
Brandon Thomas 84fcbeb1b9 Merge pull request #183 from dcarnelutti/platform/videotoolbox-h264-encode
platform: opt-in hardware H.264 encoding with VideoToolbox

# Conflicts:
#	crates/engine/src/export_tests.rs
2026-10-07 17:46:57 -04:00
Dario CarneluttiandClaude Sonnet 5.5 790b3a0697 platform: H.265 (HEVC) export with VideoToolbox
Adds `Format::Hevc` (Export > Format > H.265 (HEVC); `"format": "hevc"` in
`file.exportMedia`): HEVC Main, 8-bit 4:2:0, in MP4 (`hvc1`) or QuickTime, with AAC audio,
encoded by the macOS hardware encoder. There is no built-in H.265 encoder (one in pure Rust
is a project of its own, and x265 is GPL), so the format exists only where the OS has one.

- `videotoolbox_encode`: the session wrapper creates HEVC sessions too (`VtProfile::HevcMain`;
  the profile also says which codec). Keyframes are the HEVC random access points (NAL types
  16 to 21); the parameter sets are VPS / SPS / PPS; the `hvcC` record in the MP4 sample
  entry is the one VideoToolbox writes (sample description extension atoms), so profile,
  level and flags are the encoder's own. `VtEncoder::uses_hardware()` asks VideoToolbox
  whether the session runs on a hardware encoder (a test checks H.264 and HEVC), so a
  requirement dropped by accident could never become a silent software encode.
- `hardware_encode`: one adapter for both codecs. Choosing the format is the opt-in (the
  `hardware_encoding` setting is for H.264). `hevc_available()` (one small hardware session,
  created once) is registered with the new `filmcraft_export::register_format_probe`, so
  `available(Format::Hevc)`, the format list of Export mode and `exportFormats` follow the
  machine. `Format::has_builtin_encoder` says which formats need such a probe.
- `export`: `Format::is_h26x` where H.264 and H.265 behave alike (MP4 or QuickTime, AAC, size
  estimate, summary line). Two-pass VBR is refused up front ("H.265 export has no two-pass
  mode") and HDR sequences export as SDR, because there is no encoder to fall back to.
- Export mode: H.265 shows H.264's multiplexer, audio, metadata and bitrate controls, without
  profile, level, two-pass or the hardware checkbox, and says "Encoder: Hardware".

Apple M1 (8 cores), 1080p25, a 9:29 timeline (camera clip, ProRes 4444 overlays, AAC,
loudness): hardware H.265 at 10 Mb/s 195 s, hardware H.264 at 20 Mb/s 204 s (CPU 1029 s and
1042 s; the same H.264 export took 193 s and 200 s on other runs). It is not faster: the codec
that compresses further is the more expensive one (raw encoders: 166 and 189 frames/s), and
the export is bound by the CPU compositor, not by the media engine.

Quality per bit, 30 s of the same timeline against a ProRes 422 HQ export (average PSNR over
the three planes, SSIM): 20 Mb/s 46.88 dB / 0.9902 (H.264) and 46.91 dB / 0.9903 (H.265);
10 Mb/s 44.96 / 0.9863 and 45.09 / 0.9863; 6 Mb/s 43.72 / 0.9833 and 44.31 / 0.9843; 4 Mb/s
42.91 / 0.9807 and 43.64 / 0.9825; 3 Mb/s 42.37 / 0.9786 and 43.05 / 0.9807. The same picture
at 10 Mb/s and above; at 3 to 6 Mb/s H.265 needs 22 to 31 % less bitrate for the same PSNR.
Both formats therefore keep the same default bitrate. Details in docs/performance.md (HW3).

Tests: tests/hardware_encode.rs gets the HEVC counterparts of the H.264 ones (what is taken
and declined, a round trip through our HEVC decoder with luma PSNR above 30 dB, keyframes, MP4
and QuickTime exports with AAC, ffmpeg / ffprobe reading `hevc` / `Main` / `hvc1` /
`yuv420p` / BT.709, exact sizes, hostile configurations, encoders dropped at any point, the
format list agreeing with the probe, sessions on the hardware encoder); settings_tests.rs
checks names, ids, defaults, the summary, two-pass refusal and the probe; export_ui.rs the
controls H.265 shows and hides.

Stacked on the hardware H.264 encoder (the first commit of this branch).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:00:08 -06:00
Dario CarneluttiandClaude Sonnet 5.5 d2ab33ba76 render: composite only what overlays cover, and reuse the float images
The CPU compositor, which is what export runs, works on premultiplied linear f32 images,
33 MB each at 1080p, and every layer was one of them. A profile of a 9:29 export with a
hardware H.264 encoder (so the encoder is no longer the bottleneck) put 61-64 % of the busy
CPU in compositing, 12 % in zeroing and copying memory, 8-11 % each in the float -> 8-bit
and RGBA -> Y'CbCr conversions and 4-7 % in decoding the ProRes overlays: a banner that is
transparent except for its bottom 100 rows still cost a whole conversion and a whole mix,
and the bottom layer paid for a zero-filled canvas and a pass that mixes it over that canvas.

Three changes, each of which gives the bits the general path gives:

- The canvas is allocated when a layer needs it, and an opaque first layer (a Y'CbCr frame
  without an alpha plane, opacity 100 %, Normal blend, the size of the output) becomes the
  canvas instead of being mixed over zeros.
- A clip of plain media whose frame has an alpha plane (a ProRes 4444 banner) is converted
  and mixed only inside the rectangle where its alpha is not zero: `VideoFrame::alpha_region`,
  `to_linear_f32_region` and `blend::composite_at` (`composite` is `composite_at` at the
  origin; Dissolve's pattern follows the pixel's place in the output). A clip with effects,
  opacity masks, frame blending, colour management or a picture that is not placed one to
  one on the output takes the general path, as before.
- `filmcraft_frame::pool` keeps idle f32 images (320 MiB at most). `take_f32_overwritten`
  hands one back with its old contents, which the conversions overwrite whole (so no
  zero-fill), and the export pipeline gives each frame's image back after the 8-bit
  conversion.

Apple M1 (8 cores), a machine that was not idle, so the two builds alternate and the
median is reported:

- Synthetic 1080p frame (`cargo run --release -p filmcraft-render --example frame_bench`,
  5 rounds): camera only 8.9 -> 6.5 ms, camera + banner 16.2 -> 7.8 ms, same picture checksum.
- A real project (1080p25 camera clip, ProRes 4444 banner and subscribe bar), 30 s with the
  banner exported with the built-in encoder, which is deterministic: 46.4 -> 36.7 s, and
  both exports are the same file (same MD5), so every frame of the real pipeline matches.
- The same project with a hardware encoder, 3 rounds: 30 s with the banner 17.4 -> 11.2 s
  (CPU 99 -> 56 s), 30 s of the camera alone 11.4 -> 8.8 s, 20 s with the subscribe bar
  11.1 -> 7.0 s.

Tests: `render/src/region_tests.rs` compares each shortcut with the general path bit for bit
(an opaque bottom layer, banners over and under a camera, an 8-bit sticker, every blend mode,
several opacities, moved, scaled and effect clips, a bigger source, reduced-resolution
rendering), each render twice so the second starts from recycled buffers, and checks which
layers take a shortcut; the frame tests check that a region is the crop of the whole
conversion, that `alpha_region` is the box around what is not transparent, and that a buffer
full of NaN never leaks into a conversion, for every pixel format. Breaking a shortcut on
purpose (a region one pixel off, an opaque layer taken as the canvas at half opacity, a
conversion that skips the alpha channel) makes them fail.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:35:20 -06:00
Dario CarneluttiandClaude Sonnet 5.5 4cab6bc5da platform: opt-in hardware H.264 encoding with VideoToolbox
Export encodes H.264 on the CPU with the built-in encoder, which is most of the time
of an export (about two thirds of the busy CPU in a profile of a 20 s export). On
macOS the media engine can do it: a VideoToolbox compression session behind
`filmcraft_export::VideoEncoder`, registered by `filmcraft_platform::register()`.

- `ExportSettings::hardware_encoding` (`HardwareEncoding::Off` | `Auto`; Export > Video >
  Hardware Encoding, `"hardwareEncoding": "off|auto"` in `file.exportMedia`, a boolean is
  understood too), off by default: the built-in encoder's output is byte-identical on
  every machine, a hardware encoder's is not.
- `videotoolbox_encode` (FFI, same containment as the decoder: SAFETY comments, no
  panic across the callback, safe API) and `hardware_encode` (safe adapter and factory).
- The factory declines, so the built-in encoder is used, for other formats, MXF, two-pass
  VBR, HDR, odd sizes, non-square pixels and any configuration VideoToolbox cannot create a
  hardware session for. An encoder that fails mid-export stops it with an error: unlike a
  decoder, it cannot hand a half-written stream to another one.
- No B-frames: on real 1080p footage the quality is the same with and without them
  (+-0.3 dB at equal bitrate) and the bitrate lands closer to the target, and without
  them every frame is presented at its decode time (no composition offsets, no edit list).
- `export::register_encoder` ignores a factory registered twice; `encoder_registered`.

Apple M1, 8 cores, single runs: a 9:29 timeline (camera clip, ProRes 4444 overlays with
alpha, AAC, loudness) exports in 311 s instead of 793 s (2.5x faster); its densest 134 s
in 84 s instead of 395 s (4.7x). The outputs compare at SSIM 0.990 / PSNR 46.5 dB. The encoder
alone runs 200 fps at 1080p (docs/performance.md).

Tests (tests/hardware_encode.rs): what is taken and declined, a round trip through our
software decoder, an export that decodes in our decoder and in ffmpeg, the fallback for
everything hardware declines, exact output sizes, an empty range, hostile configurations
and encoders dropped at any point; the engine's `hardwareEncoding` parameter.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:34:51 -06:00
Davi AndreiandClaude Sonnet 5.5 932cdb77f5 Docs: Windows hardware decoding (platform README, ADR 0001, architecture, performance, roadmap)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MMorBHwct72udS2LiC3ZHp
2026-10-07 10:07:40 -03:00
Brandon Thomas cd3fe20ef9 Merge remote-tracking branch 'origin/main' into apv-105 2026-10-07 07:09:54 -04:00
justjohn12345andClaude Opus 5.5 1ebc1440e7 NEST11: the architecture doc says a plain nest's layers go into the frame plan
docs/architecture.md still listed every nested sequence among the things
rendered on the CPU and handed to the GPU as one image. Since NEST11 that
holds only for nests that are not plain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:53:53 -04:00
justjohn12345andClaude Opus 5.5 9b31f4eb84 NEST15: nested sequences in AAF and OMF, the way Premiere Pro writes them
A nested sequence was left out of both formats as a gap, with a report entry
telling the user to "render and replace" it first (a command FilmCraft does
not have). An AAF with nested compositions was read with gaps in their place.

What Premiere Pro 26.5.2 does, read off files it exported for a sequence with
a nest:
- AAF: the nested sequence is a composition mob of its own, and its clips are
  source clips that point at it.
- OMF: the nested sequence's sound is rendered into the document as one clip
  named after the sequence.

Export
- AAF: a nested sequence is written as a composition mob (no usage code; the
  exported sequence stays the top-level one). A clip of it is a source clip
  at that mob's first slot of the clip's kind, its start in that slot's edit
  units. Nests in nests are followed; a nest used twice is written once. With
  embedded or consolidated audio, the media of the clips inside a nest is
  prepared with the rest (`NestNeeds::Inside`).
- OMF: `audio_needs` lists the range of the nested sequence each nest clip
  plays (`NestNeeds::Render`); the engine mixes it through the export audio
  pipeline with what is on the clip, and it is written as media. Without that
  essence the nest is a gap and the report names the sequence.

Import (AAF)
- A source clip that points at a composition becomes a clip of a nested
  sequence; the composition is read once.
- Premiere Pro's layout is read: all video tracks in one slot (a nested scope
  with a segment per track; before, only the last segment was read, so a
  Premiere AAF came in with one video track), and "top-level" tagged on nested
  compositions too (top-level are those no other composition uses).
- A composition without a frame size takes that of the first picture it
  shows, not of the first picture in the file.

A sequence that claims to be inside itself is a gap with a report entry in
both directions; the walks are bounded (16 levels).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 20:53:34 -04:00
mweinbach 15787f7fe8 Codecs: add clean-room APV (RFC 9924) decoder, encoder, container support, and export presets 2026-10-06 17:24:34 -04:00
Brandon Thomas a407099019 Merge pull request #46 from punkhop/fix/export-interchange-unknown-format
engine: file.exportInterchange refuses an unknown format
2026-10-06 11:36:36 -04:00
Brandon Thomas 5012fbd598 Merge pull request #19 from storybored379/pr/deterministic-export
Export: same file on every machine (fixed mux interleave, H.264 slices by frame size)
2026-10-06 07:27:44 -04:00
punkhopandClaude Opus 5.5 285da5c66b engine: file.exportInterchange refuses an unknown format
Problem: `file.exportInterchange {"format": "<anything unknown>"}` wrote
FCP7 XML and returned ok, so a typo or an unsupported format produced a
file of the wrong kind under the name that was asked for.

Solution: only a missing `format` means FCP7 XML. A given format must be
one of the six documented names (edl, xml, fcpxml, otio, aaf, omf: each
format's extension, in either case). Anything else, including a value
that is not a string, is a parameter error that lists exactly those
names, and nothing is written. No other spelling is accepted (".xml",
"fcpxmld", "omfi"): none was accepted before either, they silently
became XML. The parameter doc now lists aaf and omf, which the command
already took, and the default; docs/architecture.md says the same.

Test: `export_interchange_refuses_an_unknown_format`, next to the
interchange round-trip test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 19:20:02 -07:00
Brandon ThomasandClaude Opus 5.5 4aea06e2ae GPU2: standard effects on the GPU compositor (#30)
A media clip whose enabled standard effects all have a GPU implementation
plans as a GPU layer carrying them (render::plan::LayerFx): parameters are
evaluated on the CPU at the frame's time (render::gpufx::FxOp::eval, shared
with the CPU path so both clamp hostile values identically), the source is
drawn into an Rgba32Float working image at the size the CPU decodes it, the
effects run as WGSL compute passes (gpu::fx, fx.wgsl) with the CPU
reference's math, and the result is placed with Motion, opacity and blend
mode as before. Layers without effects keep the single draw.

GPU effects: Brightness & Contrast, ProcAmp, Tint, Black & White, Color
Balance, Leave Color, Change to Color, Color Pass, Color Replace, Channel
Mix, ASC CDL, Gamma Correction, Levels, Extract, Invert, Posterize, Alpha
Adjust, Gaussian / Camera / Directional Blur (+ legacy aliases), Sharpen,
Unsharp Mask, Crop, Edge Feather, Transform, H/V Flip, Mirror, Offset.
Masked effects, opacity masks, non-finite parameters and minifying
Transform/Crop zooms stay on the CPU. Devices without compute shaders run
the effects on the CPU inside the compositor.

Also fixes a debug-build overflow in Image::transformed for non-finite maps.
Parity tests (exact f32 checks and composited plans), a golden test that
plans effect chains as GPU layers, and the stack3-fx playback benchmark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 07:50:20 -04:00
Brandon ThomasandClaude Opus 5.5 c3ec0e85d3 HW1: platform crate + VideoToolbox H.264/HEVC hardware decode (#30)
- New crates/platform (filmcraft-platform, L5): OS media FFI only. The one
  crate allowed `unsafe` (own [lints] with unsafe_code = "deny",
  #[allow(unsafe_code)] only on the videotoolbox module, SAFETY comment on
  every block, safe Result-returning public API). AGENTS.md §0.3 names the
  exception; docs/adr/0001-platform-ffi.md records why and the rules.
- VideoToolbox decoder for avcC / hvcC, 8/10-bit 4:2:0 and 4:2:2: session from
  the sample entry's parameter sets with hardware required, asynchronous
  decode (2 in flight), biplanar CVPixelBuffer copied to planar Yuv8/Yuv16,
  reorder buffer of the stream's own depth, RASL skipped after a starting CRA.
  Output callback under catch_unwind.
- HybridDecoder: transparent mid-stream fallback to the software decoder
  (replays samples since the last restart point, drops already-returned
  pictures). New codecs::software_video_decoder (built-in factories only) and
  codecs::hw (switch, counters, NalStreamInfo shared with software decoders;
  vui_color / sar_par helpers now used by both).
- Settings ▸ Playback ▸ Hardware decoding: Auto / Off (engine applies it to
  the codecs switch); desktop app, CLI/MCP and bench call register().
- perf.stats decode.framesDecoded and decode.hardware counters.
- Tests: bit-exact parity with the software decoders (H.264 High, HEVC Main
  open GOP, Main 10; resets, reseeks, flushes), forced-failure fallback,
  mutation fuzz of samples and parameter sets, cross-platform fallback with a
  stand-in decoder, setting Off, HEVC 4:2:2 vs ffmpeg.
- Bench --hw auto|off; results in docs/performance.md (4K CPU/frame
  ~86-119 ms -> ~4 ms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 04:53:28 -04:00
Brandon ThomasandClaude Opus 5.5 6102a3794a GPU1: blend modes on the GPU compositor (#30)
All 27 blend modes now composite on the GPU instead of sending the
whole frame to the CPU.

- render::plan: PlanLayer carries its Blend; plan_frame no longer falls
  back to a CPU frame for non-Normal modes. Media, graphic clips,
  multicam angles and CPU-rendered layer images (standard effects) keep
  their clip's mode; inside a transition the mode is Normal, as in the
  CPU reference. execute_cpu composites with each layer's mode.
- gpu: Normal keeps the single fixed-function pass. Dissolve also uses
  fixed-function "over": the CPU's 64-bit pixel hash is reproduced in
  WGSL on u32 pairs (pattern bit-exact). The other 25 modes copy the
  accumulator region under the layer's quad into a backdrop texture and
  composite in fs_blend with the CPU math (sRGB-encoded straight colour,
  W3C formulas, back to linear, same premultiplied mix and clamping).
- Tests: per-mode GPU-vs-CPU parity (gpu crate: overlapping transformed
  layers, partial opacity/alpha, empty/partial backdrop; golden: every
  mode through plan_frame incl. an effect layer, blended clips in a
  cross dissolve), exact-value edge cases (black/white, Divide by 0,
  Dodge/Burn at 0 and 1, zero alpha) within 4.9e-4, Dissolve pattern
  exact, off-output layers, quad bounds.
- Bench: new stack3-blend playback scenario (V2 Screen, V3 Overlay);
  CPU ms/frame 151-214 -> 86-89, Full 19/173 and 138/54 -> 191/1, 192/0.
- Docs: architecture §5, gpu crate docs, testing, performance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 03:25:16 -04:00
storybored379andClaude Opus 5.5 a55d20c454 export: same file on every machine (fixed mux interleave, H.264 slices by frame size)
Exporting the same project twice could give different files, because two
export parameters followed the number of cores the process sees:

- The muxer got one chunk per render batch, and a batch is one frame per
  rayon thread. The chunk layout (stsc/stco, PCM packet sizes) changed with
  the core count; the samples themselves did not.
- The H.264 encoder used one slice per core (capped at one per four MB rows),
  so the bitstream and every decoded picture changed with the core count.

On a host whose CPU quota changes (containers, power limits), two exports in a
row differed. Now packets are muxed in fixed groups of 16 output frames
(INTERLEAVE), each followed by its audio, whatever the batch size, and H.264
exports use one slice per four macroblock rows. Hosts with 17 or more threads
already used these values, so their 1080p output does not change.

Tests: determinism_tests (fail on main): same bytes for ProRes, H.264 and
Motion-JPEG at 2/3/5/16 cores and 1/7 frames per step; 64 slices for a
4096-line H.264 export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 08:32:49 -07:00
Brandon Thomas 0948dcf682 Merge branch 'main' into M3.13/M8.10 2026-10-03 11:50:01 -04:00
Brandon ThomasandClaude Opus 5.5 2a199425d0 M8.10: HDR-aware Lumetri, mastering metadata tone-map peak, HSL Refine, Lumetri Presets grid, Comparison View match
- Lumetri grades PQ / HLG working spaces in their own signal normalised to
  HDR White (Basic Correction) / HDR Range (Curves) in cd/m²
  (filmcraft_color::GradeSpace): exposure in stops of light, highlights above
  HDR White kept, HDR Specular scales them; Rec. 709 unchanged.
- HDR static metadata (mdcv/clli, Matroska MasteringMetadata/MaxCLL/MaxFALL)
  on VideoStreamInfo::hdr; PQ media use MaxCLL / mastering peak as the
  BT.2390 source peak; media.colorInfo reports it.
- HSL Secondary Refine: Denoise (median of the key) and Blur (Gaussian).
- Lumetri Presets (18 original looks in 4 folders) with thumbnails rendered by
  our Lumetri on a procedural preview picture; Effects panel rows and a
  thumbnail grid when wide; lumetri.presets / applyPreset / presetThumbnails.
- Color Match: Comparison View toggle; Apply Match uses its reference frame.
- Goldens lumetri_hdr_pq and lumetri_presets_grid; D-Log M stays unsupported
  (no published formula).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 10:31:00 -04:00
Brandon Thomas 81873deedb Merge branch 'main' into worktree-agent-a861ca53f77717930
# Conflicts:
#	Cargo.toml
#	docs/architecture.md
#	xtask/src/main.rs
2026-10-03 08:55:10 -04:00
Brandon ThomasandClaude Opus 5.5 8e7eebce41 M6.6: fix the architecture crate table after merging the MXF writer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 07:12:07 -04:00
Brandon Thomas 9ec0c7213b Merge branch 'worktree-agent-a1a2bf10c26331f84' into worktree-agent-a861ca53f77717930
# Conflicts:
#	docs/architecture.md
2026-10-03 07:11:48 -04:00
Brandon Thomas 41942e17d8 Merge branch 'main' into worktree-agent-a3c3d94e29f8131bb 2026-10-03 07:11:09 -04:00
Brandon Thomas 167f971e05 Merge branch 'main' into worktree-agent-a861ca53f77717930
# Conflicts:
#	crates/ui-egui/src/panels/dialogs.rs
2026-10-03 05:04:28 -04:00
Brandon ThomasandClaude Opus 5.5 6cddaf08aa M11.4: document AAF / OMF interchange and the cfb crate
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 04:55:23 -04:00
Brandon ThomasandClaude Opus 5.5 09947712d0 M6.6: MXF OP1a / OP-Atom writer and MXF export formats
filmcraft-mxf gains a streaming writer (ST 377-1 partitions, primer, header
metadata, VBR index segments with temporal offsets, RIP; ST 378 OP1a
frame wrapping, ST 390 OP-Atom clip wrapping) for VC-3, ProRes, AVC byte
stream and PCM with start timecode and caller-supplied package UMIDs, plus
write_opatom_pcm. filmcraft-export adds mxf-op1a / mxf-opatom formats
(DNxHR, ProRes or H.264 + PCM; OP-Atom writes one mono file per channel),
presets, the engine mxfVideoCodec parameter and CLI .mxf inference.
Tests read our files back and check them against ffprobe / ffmpeg.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 04:45:27 -04:00
Brandon Thomas 72e0dda167 Merge main into M9.15 2026-10-03 04:16:41 -04:00
Brandon ThomasandClaude Opus 5.5 263a39132f M9.15: READMEs for mpeg2v / mpegts / ac3 (specs, accuracy, benchmark); architecture and testing docs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 04:14:23 -04:00
Brandon Thomas d5e3d7e2ef Merge branch 'main' into worktree-agent-a8349582751c0ec31 2026-10-03 03:51:43 -04:00
Brandon ThomasandClaude Opus 5.5 02a7039ac5 M11.4: filmcraft-cfb, a Compound File Binary ([MS-CFB]) reader and writer
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 03:51:18 -04:00
Brandon ThomasandClaude Opus 5.5 c1c846aaad M12.7: Project panel and Media Browser parity
List / Icon / Freeform views, Metadata Display, view presets, font size,
Preview Area, bin tabs and windows, hover-scrub In/Out; Media Browser tree,
Favorites, history, file types, thumbnails, Edit Columns, import and drag.
Engine: project_panel and media_browser modules, Services::list_entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 02:14:31 -04:00
Brandon ThomasandClaude Opus 5.5 0bc1d4e8b8 M4.9: merge main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 00:04:16 -04:00
Brandon ThomasandClaude Opus 5.5 aa3ad1d950 M4.9: H.264 inverse transforms reconstruct across columns at once
The column pass and the add/clip of idct4/idct8 work on whole rows so they
vectorise (same integer steps in the spec's row-then-column order; checked
against the per-column formulation on random blocks). Docs: draft decoding in
architecture.md and perf.stats fields in control-protocol.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 23:46:52 -04:00
Brandon ThomasandClaude Opus 5.5 1fa560c1c7 M7.8: audio parity — 5.1 multichannel, voice-over record, Remix, Clip Mixer automation
5.1 / multichannel:
- audio-dsp::channels: layouts (L R C LFE Ls Rs), ITU-R BS.775 up/downmix, 5.1 Mixdown Types,
  equal-power 5.1 panner (puck, Center %, LFE).
- Mixer graph buses are 2 or 6 channels wide; 5.1 tracks/submixes/Mix; strips feeding a 5.1 bus
  use the 5.1 panner (lanes pan51.x/y/center/lfe stored in MixerStrip::lanes, no schema change);
  5.1 strips fold into stereo buses with BS.775; per-channel meters; clip effects on 6 channels.
- file.newSequence {mix, trackType}, sequence.settings {mix}; playback honours the 5.1 Mixdown
  Type preference on stereo devices and plays 6 channels on 5.1 devices.
- Export Audio Channels 5.1: WAV WAVE_FORMAT_EXTENSIBLE (mask 0x3F), QuickTime PCM with a chan
  atom, AAC channel configuration 6 (reordered to C L R Ls Rs LFE).
- Track Mixer: 5.1 panner puck with Center/LFE, N-channel meters, Show/Hide Tracks, Meter
  Input(s) Only; transport record button runs voice-over.

Voice-over: engine AudioInput trait + SyntheticInput, cpal input in the app, audio.voiceover.*
commands (punch-in, pre/post-roll, countdown cues, WAV next to project / scratch disk, one undo
step), track header mic button and Voice-Over Record Settings dialog.

Remix: beat tracking + chroma/cepstra self-similarity planner in audio-dsp, hidden `remix` clip
effect holding the plan, render::remix playback with equal-power crossfades, clip.remix.*
commands, Remix tool and Remix Properties dialog.

Clip Mixer automation: per-track Latch/Touch/Write (clipMixer.setMode/touch/release) heard live
and written as thinned clip Volume/Panner keyframes at the end of the pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 21:40:35 -04:00
Brandon Thomas 84f7df6479 Merge branch 'main' into worktree-agent-a7998c724fdfa5d6b 2026-10-02 20:09:15 -04:00
Brandon Thomas 9dde8107ad Merge main into M6.5 export parity
# Conflicts:
#	crates/engine/src/lib.rs
#	crates/ui-egui/src/control.rs
#	docs/control-protocol.md
2026-10-02 19:59:52 -04:00
Brandon ThomasandClaude Opus 5.5 c59d21181f M6.5: export parity — presets, Export mode settings, queue, Quick Export, image sequences
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 18:27:19 -04:00
Brandon Thomas c98f5b76d3 Merge branch 'main' into worktree-agent-a1e148ad04b0018ef
# Conflicts:
#	ROADMAP.md
2026-10-02 17:33:45 -04:00
Brandon ThomasandClaude Opus 5.5 efa8167962 M9.14: MXF (OP1a/OP-Atom) and Ogg demuxers, image sequences, BWF timecode
- crates/mxf (L0): KLV walk, partitions, primer, header metadata, index tables, material ->
  file package resolution, AVC/VC-3/ProRes/MPEG-2 identification, PCM/AES3 (ST 331) sound,
  timecode; OP1a and OP-Atom (clip wrapping).
- crates/ogg (L0): RFC 3533 pages/CRC/packets, RFC 7845 Opus timing (granules, pre-skip, end trim).
- codecs: MxfSource (GOP cache, POC ordering for AVC without temporal offsets, MPEG-2 reported
  unsupported), OggSource (Opus with pre-roll seeking, Vorbis), Annex B H.264 decoder.
- media: image sequences (numbered stills, missing frames held, frame cache); BWF bext/iXML
  TimeReference -> start timecode.
- engine/UI: file.import {imageSequence}, auto-detect setting, file.importImageSequence,
  Media Browser "Import as Image Sequence"; CLI probe details for MXF/Ogg/BWF/sequences.
- Oracle tests vs ffmpeg for MXF, Ogg, image sequences, BWF; truncation/mutation tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 17:33:29 -04:00
Brandon ThomasandClaude Opus 5.5 45b8c0c2a0 M8.9 + M12.6: Lumetri Scopes complete; Metadata, Timecode, Events, Progress, Reference Monitor panels
M8.9 (scopes):
- New L3 crate filmcraft-scopes (wasm-clean): Waveform RGB/Luma/YC/YC no Chroma,
  Parade RGB/YUV/RGB-White, Histogram, Vectorscope YUV (75/100 % targets, skin-tone
  line) and HLS; Rec. 601/709/2100 matrices, Clamp Signal, 8-bit/float/HDR (PQ cd/m2)
  scales; numeric summaries; density painting. < 1 ms per scope at 1080p.
- Lumetri Scopes panel rewritten: multi-scope grid, wrench / right-click menu (presets,
  scopes, parade/waveform type, targets, colour space, brightness, scale), footer
  (colour space, Clamp Signal, 8 Bit/Float/HDR). Settings in UiState::panels.
- scopes.read engine query returns levels, histogram bins, waveform/parade columns and
  vectorscope peaks.

M12.6 (panels):
- Metadata panel with editable Name/Label/log fields (metadata.get / metadata.set,
  one undo step, stored in ProjectItem::metadata; no schema change, stays v11).
- Timecode panel (rows: current/media/duration/in-out/remaining, any time display).
- Events panel on a new engine event log (failed/disabled commands, toasts, auto-save
  errors, jobs started/finished/failed/cancelled; events.list / events.clear).
- Progress panel (jobs with progress and Cancel).
- Reference Monitor (parked or ganged to Program; picture or scopes).
- All open from Window > menu, have automation ids; ui.set {"panels": ...} merges state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 17:32:22 -04:00
Brandon Thomas f5aa00ff7d Merge branch 'main' into worktree-agent-aacf40607c4b686e0
# Conflicts:
#	xtask/src/main.rs
2026-10-02 13:01:34 -04:00
Brandon ThomasandClaude Opus 5.5 1f6562c08a M4.8: HEVC transform/SAO speed-up, catch-up decoding, perf.stats
- HEVC inverse transform: sum basis rows of non-zero coefficients with a
  const-generic size (bit-exact, tested against the direct product);
  SAO edge offset gets a check-free interior loop. HEVC CPU/frame -60%.
- H.264/HEVC/VP9 planes are moved into VideoFrames, not copied.
- Catch-up decoding: frame jobs tell sources which frames are late
  (filmcraft_media::cancel::with_catch_up); the GOP cache skips
  non-reference samples of late frames (H.264 nal_ref_idc 0, HEVC
  sub-layer non-reference pictures). Scrubs keep the last 2 s decoded.
- perf.stats: engine query + UI/control method (decode counters,
  playback shown/dropped, frame-worker decode/render ms, cache use).
- Timeline waveform source peak cached; bench seek section and --only.
- docs/performance.md with before/after results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 13:01:16 -04:00
Brandon ThomasandClaude Opus 5.5 3139e63870 M5.11: Effects panel Legacy bin and alphabetical folders, Text param editing, vfx goldens and docs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 11:04:45 -04:00
Brandon Thomas f5ac5989f0 Merge branch 'worktree-agent-a7d751de0d72abab7' into worktree-agent-a6d2cbb6d6b3e5d7a
# Conflicts:
#	crates/edit/src/lib.rs
2026-10-01 23:31:39 -04:00
Brandon ThomasandClaude Opus 5.5 fc18a6962e M10.5: Text-based editing / transcripts
- crates/speech (L2): Transcriber trait, FixedTranscriber, Whisper model catalogue (pinned,
  SHA-256 verified) with optional downloader (feature `download`: ureq + rustls + RustCrypto),
  pure-Rust Whisper inference on candle with word timestamps (feature `whisper`), log-mel, VAD
  word tightening, MFCC speaker clustering. Both features off by default and for wasm.
- project: per-media-item transcripts (Project::transcripts); schema v9 with a no-op v8->v9
  migration and a v8 fixture.
- edit::transcript: sequence transcript derived from clip transcripts, search, paragraphs,
  word ranges, pause/filler detection, ripple delete of many ranges, caption layout.
- engine: transcript.* commands (generate, set, delete, inspect, search, select, extract, lift,
  renameSpeaker, removeFillers, removePauses, createCaptions, models, downloadModel);
  Session::transcriber for hosts/tests; features `whisper` / `speech-download` (also on the
  desktop and CLI apps). Without whisper, transcript.generate fails with a clear error.
- ui: Text panel Transcript tab (Transcribe button, speaker paragraphs of clickable words,
  Shift+click selection -> In/Out, extract/lift/remove fillers/create captions, search) with
  text.transcript.* automation ids; headless UI test.
- docs/transcripts.md, crates/speech/README.md, architecture and project-files updates.
  whisper-tiny measures 12.2% WER over 797 words on local samples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 23:25:49 -04:00
Brandon ThomasandClaude Opus 5.5 2d5267a7df M12.4: agent-friendly CLI: exec/describe/inspect/import/export, run from stdin with JSON lines and --keep-going, --save/--save-as, --bridge to the live app, exit codes; tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 21:26:44 -04:00
Brandon ThomasandClaude Opus 5.5 5f613aff25 M13.2: FilmCraft web app (apps/filmcraft-web) with Blob media, OPFS recovery, WebAudio, WebCodecs
- eframe web runner on WebGPU (WebGL2 fallback; GPU compositor on WebGPU, CPU on WebGL2).
- fs: virtual file table of File/Blob handles, 1 MiB chunked async range reads with an
  LRU cache and read-ahead (pending + retry instead of blocking), downloads for written
  files, /opfs/ paths written to OPFS.
- import: File System Access picker / <input type=file>, drag-and-drop anywhere,
  container index prefetch (moov wherever it is), retried imports; project open/save.
- recovery: OPFS auto-save snapshot while dirty, media copies, reopened on next visit.
- audio: AudioWorklet output fed ahead from the UI thread; played-frame audio clock.
- webcodecs: H.264/HEVC/VP9/AV1 MP4/MOV decoded by the browser VideoDecoder via a
  reader opener (fallback to our decoders on any error), probed with isConfigSupported.
- api: window.filmcraft (execute/request/commands/screenshot/importFiles/importUrl/
  openProject/files/readFile/info).
- cargo xtask web [--dev] [--serve PORT] (wasm-bindgen CLI); xtask wasm also checks
  filmcraft-ui-egui and filmcraft-web; layer L6 for the web app.
- Original web icon (favicon.svg) with attribution sidecar; docs/web.md; headless-Chrome
  smoke test (tests/smoke.mjs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 14:52:35 -04:00
Brandon Thomas dc9c5fd32c Merge branch 'main' into worktree-agent-a98dbfec8c9b949c7 2026-10-01 13:06:51 -04:00
Brandon ThomasandClaude Opus 5.5 f0c705493a M9.7: AV1 inter-GOP seek tests in MP4/WebM, decode perf bench, docs and roadmap
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 12:03:34 -04:00