mirror of
https://github.com/storytold/filmcraft.git
synced 2026-10-24 10:10:05 -06:00
platform: opt-in hardware H.264 encoding with VideoToolbox
Export encodes H.264 on the CPU with the built-in encoder, which is most of the time of an export (about two thirds of the busy CPU in a profile of a 20 s export). On macOS the media engine can do it: a VideoToolbox compression session behind `filmcraft_export::VideoEncoder`, registered by `filmcraft_platform::register()`. - `ExportSettings::hardware_encoding` (`HardwareEncoding::Off` | `Auto`; Export > Video > Hardware Encoding, `"hardwareEncoding": "off|auto"` in `file.exportMedia`, a boolean is understood too), off by default: the built-in encoder's output is byte-identical on every machine, a hardware encoder's is not. - `videotoolbox_encode` (FFI, same containment as the decoder: SAFETY comments, no panic across the callback, safe API) and `hardware_encode` (safe adapter and factory). - The factory declines, so the built-in encoder is used, for other formats, MXF, two-pass VBR, HDR, odd sizes, non-square pixels and any configuration VideoToolbox cannot create a hardware session for. An encoder that fails mid-export stops it with an error: unlike a decoder, it cannot hand a half-written stream to another one. - No B-frames: on real 1080p footage the quality is the same with and without them (+-0.3 dB at equal bitrate) and the bitrate lands closer to the target, and without them every frame is presented at its decode time (no composition offsets, no edit list). - `export::register_encoder` ignores a factory registered twice; `encoder_registered`. Apple M1, 8 cores, single runs: a 9:29 timeline (camera clip, ProRes 4444 overlays with alpha, AAC, loudness) exports in 311 s instead of 793 s (2.5x faster); its densest 134 s in 84 s instead of 395 s (4.7x). The outputs compare at SSIM 0.990 / PSNR 46.5 dB. The encoder alone runs 200 fps at 1080p (docs/performance.md). Tests (tests/hardware_encode.rs): what is taken and declined, a round trip through our software decoder, an export that decodes in our decoder and in ffmpeg, the fallback for everything hardware declines, exact output sizes, an empty range, hostile configurations and encoders dropped at any point; the engine's `hardwareEncoding` parameter. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5.5
parent
86f3375775
commit
4cab6bc5da
+2
-2
@@ -36,7 +36,7 @@ before choosing work.
|
||||
| **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. |
|
||||
| **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. |
|
||||
| **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. |
|
||||
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS only (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects, export and encoding are CPU. Linux and Windows have no hardware path. 8K and AV1 aren't real-time. The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
|
||||
| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS only (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software). The GPU does compositing, blend modes (#32) and 31 common effects (GPU2): colour adjustments, crop/flip/transform, blurs, sharpen. Lumetri, keys, masks, the remaining effects and export rendering are CPU; H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames). Linux and Windows have no hardware path. 8K and AV1 aren't real-time. The effect list shows Premiere's "GPU accelerated" badge, but only 31 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. |
|
||||
| **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. |
|
||||
| **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. |
|
||||
| **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. |
|
||||
@@ -51,7 +51,7 @@ In priority order. Agents should prefer this work over adding more checklist ite
|
||||
- hardware decode on Linux (VA-API) and Windows (Media Foundation);
|
||||
- zero-copy decoded frames into wgpu;
|
||||
- the remaining effects on the GPU: Lumetri, keys, Vignette, Video Limiter, masks (31 common effects are done);
|
||||
- GPU export, then hardware encode (H.264 / HEVC).
|
||||
- GPU export, then hardware encode: H.264 on macOS is in (opt-in); HEVC and other systems remain.
|
||||
2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus,
|
||||
commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in
|
||||
`plan/premiere/`. It should report presence and fidelity separately (exact, approximate, stub).
|
||||
|
||||
Reference in New Issue
Block a user