diff --git a/ROADMAP.md b/ROADMAP.md index 8eaf78ff9..06688b9af 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -36,7 +36,7 @@ before choosing work. | **Correctness in depth** | not tracked | **~60–70%** | A first-time contributor found four real bugs in core paths within hours (#5–#8: AIFF import failed, Slide left linked audio behind, transitions didn't follow ripple trims, export ignored start/end times). User reports: Color Matte always grey with no picker (#29); ALSA audio error on Linux (#23). Settings are "most wired", not all. | | **Codecs and media** | ~90% | **~75%** | Our decoders are bit-exact on conformance streams. Real camera and phone media (variable frame rate, damaged files, unusual containers) is far less tested. No camera RAW (RED, BRAW, ARRIRAW), no E-AC-3. | | **Export** | ~92% | **~70%** | Delivery: H.264 only (ProRes, DNxHR and image sequences cover mastering). No HEVC or AV1 export. AAF and OMF have never been validated in Avid Media Composer or Pro Tools. Export renders and encodes on the CPU only. | -| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264, HEVC Main / Main 10, VP9 and AV1 4:2:0 8/10-bit, #30). The GPU does compositing, blend modes (#32) and 32 common effects (GPU2, GPU4): colour adjustments, crop/flip/transform, blurs, sharpen, vignette. Lumetri, keys, masks, the remaining effects and export rendering are CPU; H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames) and NVENC on Windows with an NVIDIA GPU (opt-in per export), and H.265 (HEVC) export exists there only, through the same hardware encoder. Linux has no hardware path. 8K isn't real-time (AV1 is on Windows with a hardware decoder). The effect list shows Premiere's "GPU accelerated" badge, but only 32 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. | +| **Performance and hardware** | ~62% | **~35–40%** | Premiere runs effects, decoding and encoding on the GPU and the hardware media engines. Here, hardware decode works on macOS (VideoToolbox, #33; active in the desktop app only since #41, so releases up to 0.2.1 decoded in software) and Windows (Media Foundation / Direct3D 11 DXVA, H.264, HEVC Main / Main 10, VP9 and AV1 4:2:0 8/10-bit, #30). The GPU does compositing, blend modes (#32) and 33 common effects (GPU2, GPU4, GPU5): colour adjustments, crop/flip/transform, blurs, sharpen, vignette, video limiter. Lumetri, keys, masks, the remaining effects and export rendering are CPU; H.264 encoding can use VideoToolbox on macOS (opt-in per export, no B-frames) and NVENC on Windows with an NVIDIA GPU (opt-in per export), and H.265 (HEVC) export exists there only, through the same hardware encoder. Linux has no hardware path. 8K isn't real-time (AV1 is on Windows with a hardware decoder). The effect list shows Premiere's "GPU accelerated" badge, but only 33 effect ids run on the GPU here; the rest, Lumetri included, run on the CPU. | | **Stability** | not tracked | **improving, unproven** | The never-crash pass (no panics in product code, last-resort guards) landed on 2026-10-04. There's no field record from real users' projects yet. | | **Plugins and pro ecosystem** | not tracked | **~0–10%** | No audio plugin hosting (VST3 / Audio Units) and no third-party video effects (OpenFX). For many professional editors this alone rules FilmCraft out. Team Projects and Productions are out of scope by design. | | **AI features** | partial | **~25–35%** | Speech to text exists but is off in default builds (the `whisper` feature). Enhance Speech is a DSP chain, not a model. Auto Reframe is approximate. No Generative Extend, and no media-intelligence search or auto colour. | @@ -50,7 +50,7 @@ In priority order. Agents should prefer this work over adding more checklist ite 1. **Hardware acceleration** (#30), the most visible gap to users: - hardware decode on Linux (VA-API); - zero-copy decoded frames into wgpu; - - the remaining effects on the GPU: Lumetri, keys, Video Limiter, masks (32 common effects are done); + - the remaining effects on the GPU: Lumetri, keys, masks (33 common effects are done); - GPU export, then hardware encode: H.264 and H.265 on macOS are in (H.264 opt-in; H.265 has no other encoder), H.264 on NVENC (Windows, opt-in); HEVC on Windows and Linux remain. 2. **A measured parity number.** Add `cargo xtask parity`, run in CI. It should cover menus, commands, effects, transitions, panels, preferences and formats against the Premiere snapshot in @@ -143,7 +143,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining | M2 | H.264 decoder | ✅ | Own decoder, bit-exact on 37+ streams, 500–600 fps 1080p | — | — | | M3 | Editing core | 🟡 | Edit algebra (insert/overwrite/razor/lift/extract/ripple/roll/slip/slide/rate-stretch/nest/paste), tools, markers, trim mode + Trim Monitor + dynamic J/K/L trimming, Keyboard Shortcuts editor with FilmCraft/Premiere/FCP/Avid presets; Edit/Clip/File menu commands (Label colours and Select Label Group, Paste/Remove Attributes, Select All Matching, Remove Unused, Consolidate Duplicates, Sequence From Clip, Bin From Selection, Offline File, Close Project, Make/Edit Subclip, Modify Audio Channels/Timecode, Frame Hold Options/Add Frame Hold/Insert Frame Hold Segment, Time Interpolation with frame blending, Fit/Fill frame, Breakout to Mono, Extract Audio, Replace With Clip; M3.11: Scene Edit Detection (pure-Rust cut detection, background job), Normalize Mix Track, Simplify Sequence, Transcribe Sequence, Find/Find Next and search bins, Automate to Sequence, Edit Original, Edit Offline, Source Settings, Update Metadata (XMP), Generate Audio Waveform, Project Settings General/Scratch Disks, Get Media File Properties, Save as Template, Selection as FilmCraft Project, Avid Log Exchange export, Flash Cue markers, Dynamic Audio Waveforms, Reveal Log Files, System Compatibility Report); M3.12 keyboard parity: Premiere's default keyboard (~115 keyboard-only commands: edit-point navigation on targeted/any track, select clip at playhead/next/previous, extend edit to playhead, nudge/slip/slide selection, target and source-patch toggles, clip volume ±1 dB/many, frame maximize/full screen/panel cycling, monitor zoom, track heights, Project and Text panel keyboard navigation, text size/leading/alignment, Export Frame, poster frames; see docs/keyboard.md); multicam (Create Multi-Camera Source Sequence, Multi-Camera view with live switching on 1–9, angle switching, Enable/Flatten, Edit Cameras) and sync (Synchronize, Merge Clips; In/Out/timecode/marker/audio — GCC-PHAT, sample-accurate) | Multicam paging >16 angles and grid thumbnails, optical flow (renders as frame blending), 39 Premiere default shortcuts skipped with reasons in docs/keyboard.md (Productions, AI/cloud tools, work area bar, Production/Search panels) | 6–10 | | M4 | Playback | 🟡 | Audio-clock master, prefetch with cancellation, J/K/L, correct dropped-frame stats, playback resolution, render bar + content-hashed render previews, App Nap opt-out; 1080p H.264 and 3 stacked 1080p streams play with 0 dropped frames | 4K under load, 8K, frame-threaded AV1 decode, reduced-resolution decode for multicam grids | 6–10 | -| M5 | Effects, keyframes, GPU | 🟡 | All 93 Premiere 26 video effects and 84 transitions (CPU), blend modes and 32 common effects on the GPU (#32, GPU2, GPU4), keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Effects on the GPU (in progress, #30), GPU masks in the live path, GPU export, exact Warp Stabilizer (3-D) and Morph Cut | 20–30 | +| M5 | Effects, keyframes, GPU | 🟡 | All 93 Premiere 26 video effects and 84 transitions (CPU), blend modes and 33 common effects on the GPU (#32, GPU2, GPU4, GPU5), keyframes + value/velocity graphs, wgpu compositor, effect + opacity masks (ellipse/polygon/pen, feather, expansion, modes; CPU/WGSL parity), mask tracking (Lucas–Kanade + RANSAC), adjustment layers, effect presets (built-in + user, JSON import/export) | Effects on the GPU (in progress, #30), GPU masks in the live path, GPU export, exact Warp Stabilizer (3-D) and Morph Cut | 20–30 | | M6 | Export | ✅ | Own H.264 encoder (High/Main/Baseline, B-frames, VBR/CBR/2-pass) → MP4 or QuickTime + own AAC; ProRes, DNxHR, MJPEG, PNG/TIFF/BMP sequences (Premiere-style numbering), GIF, WAV, AIFF; background jobs; M6.5: Export mode parity (frame size / rate / scaling / pixel aspect, profile / level, CBR / VBR 1- and 2-pass, keyframe distance, audio codec / rate / channels / bitrate / sample size, multiplexer, burn-in or sidecar captions, image / name / timecode overlays, video limiter, loudness normalization with true-peak limiter, metadata, ranges, estimated size, summary), 24 built-in presets + user presets with favourites and import/export (Preset Manager), export queue (reorder, cancel, retry, several sequences / ranges), Quick Export; `export.*` commands and `filmcraft-cli export --preset` | Interlaced encoding, 5.1 audio, smart render, publishing destinations | 6–8 | | M7 | Audio | 🟡 | Mixer graph (tracks → submixes → Mix, pre/post-fader inserts and sends, latency-compensated, sample-accurate, ~6× realtime for 24 tracks × 3 effects on one core), Audio Track Mixer + Audio Clip Mixer panels, track automation (Off/Read/Latch/Touch/Write, recorded live while playing, thinned to keyframes, timeline lanes with pen editing), solo/solo-safe, channel mapping basics, peak + BS.1770 loudness meters (match ffmpeg), DSP crate with 16 clip/track effects, Audio Gain (set/adjust/normalize), Constant Power / Constant Gain / Exponential Fade, Essential Sound (types, Loudness auto-match, Repair incl. DeEss/DeReverb, Clarity, Creative, Ducking keyframes, presets; full Dialogue chain 22× realtime) | Music duration remix, ML speech enhancement, 5.1 panner and multichannel buses, voice-over record, effect editor windows (EQ curve), remaining effects (multiband, convolution reverb), clip-mixer automation recording | 6–9 | | M8 | Colour | 🟡 | Lumetri: basic, creative + looks, RGB & hue curves, wheels, HSL secondary, vignette, section bypass; Input LUT / Look LUT (.cube 1D/3D/shaper, .3dl; tetrahedral CPU + WGSL; project LUT library; built-in camera conversions); Colour Match (Oklab tonal-range statistics, skin protection, solved in Lumetri wheels); colour management: Rec. 709 / Rec. 2100 PQ / HLG working spaces + wide gamut, Interpret Footage colour space (S-Log3, V-Log, Canon Log 2/3, LogC3/4, Apple Log, D-Log from published specs), metadata auto-detect, BT.2390 tone mapping, gamut mapping, HDR export signalling (VUI/colr/mdcv/clli/SEI, ffprobe-verified); Lumetri Scopes: Vectorscope YUV (75/100 % targets, skin-tone line) and HLS, Histogram, Parade (RGB/YUV/RGB-White), Waveform (RGB/Luma/YC/YC no Chroma), presets, Rec. 601/709/2100, 8-bit/float/HDR (cd/m²) scales, multi-scope grid, `scopes.read` | HDR-aware Lumetri maths, macOS EDR monitors, mastering metadata → tone-map peak, D-Log M (no published formula), HSL Secondary refine | 3–5 | @@ -154,7 +154,7 @@ Status: ✅ done · 🟡 in progress · ⬜ not started. Estimates are remaining ## Running now -- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 32 common effects on the GPU (GPU2, GPU4). Windows: Media Foundation / Direct3D 11 H.264 + HEVC decode (HW2) and VP9 + AV1 (HW3), NVENC H.264 export (HW4, opt-in). Next: VA-API (Linux) decode, zero-copy upload, Lumetri on the GPU. +- **Hardware acceleration (#30):** landed: blend modes on the GPU (#32); VideoToolbox hardware decode + the `platform` FFI crate (#33); 33 common effects on the GPU (GPU2, GPU4, GPU5). Windows: Media Foundation / Direct3D 11 H.264 + HEVC decode (HW2) and VP9 + AV1 (HW3), NVENC H.264 export (HW4, opt-in). Next: VA-API (Linux) decode, zero-copy upload, Lumetri on the GPU. - **Next:** [Where we are lacking](#where-we-are-lacking) items 1–5. GPU export waits for the export-crate work (#19 and the never-crash follow-up) to land. ## Log diff --git a/crates/gpu/src/fx.rs b/crates/gpu/src/fx.rs index 96c386c77..424457e21 100644 --- a/crates/gpu/src/fx.rs +++ b/crates/gpu/src/fx.rs @@ -50,6 +50,7 @@ const OP_HFLIP: u32 = 26; const OP_VFLIP: u32 = 27; const OP_MIRROR: u32 = 28; const OP_OFFSET: u32 = 29; +const OP_VIDEO_LIMITER: u32 = 30; type Target = (wgpu::Texture, wgpu::TextureView); @@ -181,6 +182,9 @@ fn steps(op: &FxOp) -> Vec { out.push(step(OP_VIGNETTE, [0; 4], &[*amount, *midpoint, *roundness, *feather, target[0], target[1], target[2]])); } } + FxOp::VideoLimiter { max, comp, axis, warn, warning_color } => { + out.push(step(OP_VIDEO_LIMITER, [*axis, *warn as u32, 0, 0], &[*max, *comp, warning_color[0], warning_color[1], warning_color[2]])); + } } out } diff --git a/crates/gpu/src/fx.wgsl b/crates/gpu/src/fx.wgsl index 80c8e5206..e74dfe63c 100644 --- a/crates/gpu/src/fx.wgsl +++ b/crates/gpu/src/fx.wgsl @@ -46,6 +46,7 @@ const OP_HFLIP: u32 = 26u; const OP_VFLIP: u32 = 27u; const OP_MIRROR: u32 = 28u; const OP_OFFSET: u32 = 29u; +const OP_VIDEO_LIMITER: u32 = 30u; fn size() -> vec2 { return vec2(i32(u.i0.y), i32(u.i0.z)); @@ -219,6 +220,88 @@ fn hsl_to_rgb(h: f32, s: f32, l: f32) -> vec3 { return vec3(hsl_channel(p, q, h + 1.0 / 3.0), hsl_channel(p, q, h), hsl_channel(p, q, h - 1.0 / 3.0)); } +fn linear_to_enc_limiter(v: f32) -> f32 { + if v <= 1.0 { + return linear_to_srgb1(max(v, 0.0)); + } + return 1.0 + (v - 1.0) / 2.4; +} + +fn enc_to_linear_limiter(v: f32) -> f32 { + if v <= 1.0 { + return srgb_to_linear1(max(v, 0.0)); + } + return 1.0 + (v - 1.0) * 2.4; +} + +fn knee_limiter(v: f32, max_val: f32, comp: f32) -> f32 { + let k = max_val * (1.0 - comp); + if comp <= 0.0 || v <= k { + return min(v, max_val); + } + let r = max_val - k; + return k + r * (1.0 - exp(-(v - k) / r)); +} + +fn to_rgb_709(y: f32, cb: f32, cr: f32) -> vec3 { + let kr = 0.2126; + let kb = 0.0722; + let kg = 1.0 - kr - kb; + let r = y + 2.0 * (1.0 - kr) * cr; + let b = y + 2.0 * (1.0 - kb) * cb; + let g = (y - kr * r - kb * b) / kg; + return vec3(r, g, b); +} + +fn limit_video(v: vec3, max_val: f32, comp: f32, axis: u32) -> vec3 { + let kr = 0.2126; + let kb = 0.0722; + let kg = 1.0 - kr - kb; + var y = kr * v.x + kg * v.y + kb * v.z; + var cb = (v.z - y) / (2.0 * (1.0 - kb)); + var cr = (v.x - y) / (2.0 * (1.0 - kr)); + + if axis != 1u { + y = knee_limiter(max(y, 0.0), max_val, comp); + } + + if axis != 0u { + let rgb = to_rgb_709(y, cb, cr); + var s = 1.0; + let d = rgb - vec3(y); + + if rgb.x > max_val && d.x > 1e-6 { + s = min(s, max((max_val - y) / d.x, 0.0)); + } + if rgb.x < 0.0 && d.x < -1e-6 { + s = min(s, max(y / (-d.x), 0.0)); + } + + if rgb.y > max_val && d.y > 1e-6 { + s = min(s, max((max_val - y) / d.y, 0.0)); + } + if rgb.y < 0.0 && d.y < -1e-6 { + s = min(s, max(y / (-d.y), 0.0)); + } + + if rgb.z > max_val && d.z > 1e-6 { + s = min(s, max((max_val - y) / d.z, 0.0)); + } + if rgb.z < 0.0 && d.z < -1e-6 { + s = min(s, max(y / (-d.z), 0.0)); + } + + if axis == 3u && s < 1.0 - 1e-4 { + s = min(knee_limiter(s, 1.0, max(comp, 0.03)), 1.0); + } + cb *= s; + cr *= s; + } + + let out_rgb = to_rgb_709(y, cb, cr); + return clamp(out_rgb, vec3(0.0), vec3(max_val)); +} + // ---- per-pixel colour effects on straight colour (`Image::map_rgb`) fn color_op(op: u32, c: vec3) -> vec3 { @@ -498,6 +581,29 @@ fn pixel(op: u32, p: vec2) -> vec4 { let res = dec(ec + (tgt - ec) * (edge * abs(amt))); return vec4(res * a, a); } + case OP_VIDEO_LIMITER: { + let o = ld(p); + let a = o.a; + if a <= 1e-6 { + return o; + } + let c = o.rgb / a; + let v = vec3(linear_to_enc_limiter(c.x), linear_to_enc_limiter(c.y), linear_to_enc_limiter(c.z)); + let max_val = u.p0.x; + let comp = u.p0.y; + let axis = u.i1.x; + let warn = u.i1.y != 0u; + let wc = u.p0.zw; + let wc_b = u.p1.x; + let warning_col = vec3(wc.x, wc.y, wc_b); + let out_enc = limit_video(v, max_val, comp, axis); + let diff = abs(v - out_enc); + if warn && (diff.x > 1e-4 || diff.y > 1e-4 || diff.z > 1e-4) { + return vec4(warning_col * a, a); + } + let res = vec3(enc_to_linear_limiter(out_enc.x), enc_to_linear_limiter(out_enc.y), enc_to_linear_limiter(out_enc.z)); + return vec4(res * a, a); + } default: { let o = ld(p); let a = o.a; diff --git a/crates/gpu/src/fx_tests.rs b/crates/gpu/src/fx_tests.rs index 37da6f047..02d8e7114 100644 --- a/crates/gpu/src/fx_tests.rs +++ b/crates/gpu/src/fx_tests.rs @@ -41,6 +41,9 @@ fn fl(v: f64) -> ParamValue { fn col(r: f32, g: f32, b: f32) -> ParamValue { ParamValue::Color([r, g, b, 1.0]) } +fn ch(c: u32) -> ParamValue { + ParamValue::Choice(c) +} fn pt(x: f64, y: f64) -> ParamValue { ParamValue::Vec2(Vec2::new(x, y)) } @@ -195,6 +198,22 @@ fn cases() -> Vec<(&'static str, Vec<(&'static str, ParamValue)>, bool)> { false, ), ("vignette", vec![("amount", fl(40.0)), ("midpoint", fl(90.0)), ("roundness", fl(100.0)), ("feather", fl(100.0))], false), + ("video_limiter", vec![], false), + ("video_limiter", vec![("axis", ch(0)), ("clip_level", ch(0)), ("compression", ch(0))], false), + ("video_limiter", vec![("axis", ch(1)), ("clip_level", ch(9)), ("compression", ch(4))], false), + ("video_limiter", vec![("axis", ch(2)), ("clip_level", ch(5)), ("compression", ch(2))], false), + ("video_limiter", vec![("axis", ch(3)), ("clip_level", ch(0)), ("compression", ch(1))], false), + ( + "video_limiter", + vec![ + ("axis", ch(2)), + ("clip_level", ch(0)), + ("compression", ch(0)), + ("gamut_warning", ParamValue::Bool(true)), + ("warning_color", col(0.0, 1.0, 0.0)), + ], + false, + ), ] } @@ -331,6 +350,10 @@ fn hostile_parameters_are_bounded() { effect("vignette", &[("midpoint", fl(f64::INFINITY))]), effect("vignette", &[("roundness", fl(f64::NAN))]), effect("vignette", &[("feather", fl(f64::NAN))]), + effect("video_limiter", &[("clip_level", fl(f64::NAN))]), + effect("video_limiter", &[("compression", fl(f64::INFINITY))]), + effect("video_limiter", &[("axis", fl(f64::NAN))]), + effect("video_limiter", &[("warning_color", ParamValue::Color([f32::NAN, 0.0, 0.0, 1.0]))]), ]; for e in &nan { let op = FxOp::eval(e, &cx(Tick::ZERO, 1.0), w, h); diff --git a/crates/render/src/gpufx.rs b/crates/render/src/gpufx.rs index cc5b6afcd..a718438a8 100644 --- a/crates/render/src/gpufx.rs +++ b/crates/render/src/gpufx.rs @@ -168,6 +168,13 @@ pub enum FxOp { feather: f32, target: [f32; 3], }, + VideoLimiter { + max: f32, + comp: f32, + axis: u32, + warn: bool, + warning_color: [f32; 3], + }, } /// Effect ids [`FxOp::eval`] understands (the GPU-capable standard effects). @@ -204,6 +211,7 @@ pub const GPU_EFFECTS: &[&str] = &[ "color_replace", "alpha_adjust", "vignette", + "video_limiter", ]; /// `Image::transformed` without the mip path: the destination rectangle it writes and the inverse. @@ -434,6 +442,35 @@ impl FxOp { let target = if amt < 0.0 { [col[0], col[1], col[2]] } else { [1.0; 3] }; FxOp::Vignette { amount: amt, midpoint: mid, roundness: round, feather, target } } + "video_limiter" => { + use crate::vfx::{bv, cv}; + use filmcraft_project::ParamValue; + let clip_level = match e.param("clip_level").map(|p| p.value.clone()).or_else(|| crate::vfx::def_value(e, "clip_level")) { + Some(ParamValue::Choice(c)) => c as f32, + Some(ParamValue::Float(f)) => f as f32, + _ => 0.0, + }; + let comp_val = match e.param("compression").map(|p| p.value.clone()).or_else(|| crate::vfx::def_value(e, "compression")) { + Some(ParamValue::Choice(c)) => [0.0, 0.03, 0.05, 0.10, 0.20][(c as usize).min(4)], + Some(ParamValue::Float(f)) => f as f32, + _ => 0.03, + }; + let axis = match e.param("axis").map(|p| p.value.clone()).or_else(|| crate::vfx::def_value(e, "axis")) { + Some(ParamValue::Choice(c)) => c, + Some(ParamValue::Float(f)) => { + if f.is_nan() { + u32::MAX + } else { + f as u32 + } + } + _ => 3, + }; + let warn = bv(e, "gamut_warning"); + let c = cv(e, "warning_color", cx); + let wc = dec([c[0], c[1], c[2]]); + FxOp::VideoLimiter { max: 1.0 + clip_level / 100.0, comp: comp_val, axis, warn, warning_color: wc } + } _ => return None, }) } @@ -478,6 +515,7 @@ impl FxOp { FxOp::ColorReplace { sim, target, replace, replace_hsl, .. } => sim.is_finite() && fin(target) && fin(replace) && fin(replace_hsl), FxOp::AlphaAdjust { opacity, .. } => opacity.is_finite(), FxOp::Vignette { amount, midpoint, roundness, feather, target } => fin(&[*amount, *midpoint, *roundness, *feather]) && fin(target), + FxOp::VideoLimiter { max, comp, axis, warning_color, .. } => *axis <= 3 && max.is_finite() && comp.is_finite() && fin(warning_color), } } @@ -753,10 +791,81 @@ impl FxOp { dec(crate::vfx::lerp3(enc(c), target, edge * amt.abs())) }); } + FxOp::VideoLimiter { max, comp, axis, warn, warning_color } => { + let (max, comp, axis, warn, wc) = (*max, *comp, *axis, *warn, *warning_color); + if !max.is_finite() || !comp.is_finite() || !wc.iter().all(|c| c.is_finite()) { + return; + } + img.map_rgb(|c, _, _| { + let v = [linear_to_enc(c[0]), linear_to_enc(c[1]), linear_to_enc(c[2])]; + let out = limit(v, max, comp, axis); + if warn && v.iter().zip(&out).any(|(a, b)| (a - b).abs() > 1e-4) { + return wc; + } + out.map(enc_to_linear) + }); + } } } } +/// sRGB encoding that keeps values above 1 (super-whites) instead of clamping them. +#[inline] +fn linear_to_enc(v: f32) -> f32 { + if v <= 1.0 { filmcraft_color::linear_to_srgb(v.max(0.0)) } else { 1.0 + (v - 1.0) / 2.4 } +} +#[inline] +fn enc_to_linear(v: f32) -> f32 { + if v <= 1.0 { filmcraft_color::srgb_to_linear(v.max(0.0)) } else { 1.0 + (v - 1.0) * 2.4 } +} + +#[inline] +fn knee(v: f32, max: f32, comp: f32) -> f32 { + let k = max * (1.0 - comp); + if comp <= 0.0 || v <= k { + return v.min(max); + } + let r = max - k; + k + r * (1.0 - (-(v - k) / r).exp()) +} + +pub(crate) fn limit(v: [f32; 3], max: f32, comp: f32, axis: u32) -> [f32; 3] { + let m = filmcraft_color::Matrix::Bt709; + let ycc = filmcraft_color::rgb_to_ycbcr(v[0], v[1], v[2], m); + let (kr, kb) = (0.2126f32, 0.0722f32); + let to_rgb = |y: f32, cb: f32, cr: f32| { + let r = y + 2.0 * (1.0 - kr) * cr; + let b = y + 2.0 * (1.0 - kb) * cb; + let g = (y - kr * r - kb * b) / (1.0 - kr - kb); + [r, g, b] + }; + let (mut y, mut cb, mut cr) = (ycc[0], ycc[1], ycc[2]); + if axis != 1 { + y = knee(y.max(0.0), max, comp); + } + if axis != 0 { + // scale chroma so every channel fits 0…max (the gamut of legal R′G′B′) + let rgb = to_rgb(y, cb, cr); + let mut s = 1.0f32; + for &ch in &rgb { + let d = ch - y; + if ch > max && d > 1e-6 { + s = s.min(((max - y) / d).max(0.0)); + } + if ch < 0.0 && d < -1e-6 { + s = s.min((y / -d).max(0.0)); + } + } + if axis == 3 && s < 1.0 - 1e-4 { + // smart limit: soft chroma compression rather than a hard cut + s = knee(s, 1.0, comp.max(0.03)).min(1.0); + } + cb *= s; + cr *= s; + } + to_rgb(y, cb, cr).map(|q| q.clamp(0.0, max)) +} + /// The body of `Image::transformed` (no mip path) for a precomputed inverse and rectangle. fn resample_cpu(src: &Image, inv: &Affine, rect: [u32; 4]) -> Image { let (w, h) = (src.w, src.h); diff --git a/crates/render/src/vfx/color.rs b/crates/render/src/vfx/color.rs index b6bfb1ea6..35f4e464a 100644 --- a/crates/render/src/vfx/color.rs +++ b/crates/render/src/vfx/color.rs @@ -110,84 +110,8 @@ pub(crate) fn cdl(c: [f32; 3], s: [f32; 3], o: [f32; 3], p: [f32; 3], sat: f32) v.map(|q| (l + sat * (q - l)).clamp(0.0, 1.0)) } -/// Video Limiter: keeps Y′CbCr (Rec. 709, display-encoded) within 0…Clip Level with an -/// exponential soft knee that starts `compression` below the limit. -pub fn video_limiter(img: &mut Image, e: &EffectInstance, cx: &FxCtx) { - let max = 1.0 + chv(e, "clip_level") as f32 / 100.0; - let comp = [0.0, 0.03, 0.05, 0.10, 0.20][chv(e, "compression").min(4) as usize]; - let axis = chv(e, "axis"); - let warn = bv(e, "gamut_warning"); - let wc = lin(cv(e, "warning_color", cx)); - img.map_rgb(|c, _, _| { - let v = [linear_to_enc(c[0]), linear_to_enc(c[1]), linear_to_enc(c[2])]; - let out = limit(v, max, comp, axis); - if warn && v.iter().zip(&out).any(|(a, b)| (a - b).abs() > 1e-4) { - return wc; - } - dec_unclamped(out) - }); - let _ = cx; -} - -/// sRGB encoding that keeps values above 1 (super-whites) instead of clamping them. -#[inline] -fn linear_to_enc(v: f32) -> f32 { - if v <= 1.0 { filmcraft_color::linear_to_srgb(v.max(0.0)) } else { 1.0 + (v - 1.0) / 2.4 } -} -#[inline] -fn enc_to_linear(v: f32) -> f32 { - if v <= 1.0 { filmcraft_color::srgb_to_linear(v.max(0.0)) } else { 1.0 + (v - 1.0) * 2.4 } -} -fn dec_unclamped(c: [f32; 3]) -> [f32; 3] { - c.map(enc_to_linear) -} - -#[inline] -fn knee(v: f32, max: f32, comp: f32) -> f32 { - let k = max * (1.0 - comp); - if comp <= 0.0 || v <= k { - return v.min(max); - } - let r = max - k; - k + r * (1.0 - (-(v - k) / r).exp()) -} - -pub(crate) fn limit(v: [f32; 3], max: f32, comp: f32, axis: u32) -> [f32; 3] { - let m = Matrix::Bt709; - let ycc = rgb_to_ycbcr(v[0], v[1], v[2], m); - let (kr, kb) = (0.2126f32, 0.0722f32); - let to_rgb = |y: f32, cb: f32, cr: f32| { - let r = y + 2.0 * (1.0 - kr) * cr; - let b = y + 2.0 * (1.0 - kb) * cb; - let g = (y - kr * r - kb * b) / (1.0 - kr - kb); - [r, g, b] - }; - let (mut y, mut cb, mut cr) = (ycc[0], ycc[1], ycc[2]); - if axis != 1 { - y = knee(y.max(0.0), max, comp); - } - if axis != 0 { - // scale chroma so every channel fits 0…max (the gamut of legal R′G′B′) - let rgb = to_rgb(y, cb, cr); - let mut s = 1.0f32; - for &ch in &rgb { - let d = ch - y; - if ch > max && d > 1e-6 { - s = s.min(((max - y) / d).max(0.0)); - } - if ch < 0.0 && d < -1e-6 { - s = s.min((y / -d).max(0.0)); - } - } - if axis == 3 && s < 1.0 { - // smart limit: soft chroma compression rather than a hard cut - s = knee(s, 1.0, comp.max(0.03)).min(1.0); - } - cb *= s; - cr *= s; - } - to_rgb(y, cb, cr).map(|q| q.clamp(0.0, max)) -} +#[allow(unused_imports)] +pub(crate) use crate::gpufx::limit; /// Logo Cutout: removes a flat white/black/custom background by un-multiplying it — the alpha of /// a pixel is how far it is from the background towards the gamut edge, and the colour is diff --git a/crates/render/src/vfx/mod.rs b/crates/render/src/vfx/mod.rs index 70c69ce20..bdca8e52d 100644 --- a/crates/render/src/vfx/mod.rs +++ b/crates/render/src/vfx/mod.rs @@ -460,7 +460,6 @@ pub fn apply(img: &mut Image, e: &EffectInstance, cx: &FxCtx) -> bool { match e.effect.as_str() { // Adjust / Color / Image Control / Keying / Utility colour "lighting_effects" => color::lighting(img, e, cx), - "video_limiter" => color::video_limiter(img, e, cx), "logo_cutout" => color::logo_cutout(img, e, cx), "ultra_key" => color::ultra_key(img, e, cx), "track_matte" => color::track_matte(img, e, cx), diff --git a/docs/architecture.md b/docs/architecture.md index 72e351dba..abeb32d30 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -283,7 +283,7 @@ file ──► codecs (MP4/MOV, MKV, audio) demux + decode, GOP-aware see Balance, Leave Color, Change to Color, Color Pass, Color Replace, Channel Mix, ASC CDL, Gamma Correction, Levels, Extract, Invert, Posterize, Alpha Adjust, Gaussian Blur and Directional Blur (and their legacy aliases), Camera Blur, Sharpen, Unsharp Mask, Crop, Edge Feather, Transform, - Horizontal / Vertical Flip, Mirror, Offset and Vignette. When every enabled effect of a media clip is in + Horizontal / Vertical Flip, Mirror, Offset, Vignette and Video Limiter. When every enabled effect of a media clip is in that set (unmasked, with finite parameters, and no Transform shrinking the picture below half size, which the CPU pre-filters), the plan hands the GPU the clip's source with the effects' parameters evaluated at that time: the source is drawn into an `Rgba32Float`