diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index abe06308d..394012f27 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -23,6 +23,24 @@ iconography, images or other Adobe assets; open licences only; every asset attri | `docs/images/filmcraft-effects.png` | FilmCraft contributors | Original work: FilmCraft screenshot crop; NASA Earth Views (US Government work, public domain) | MIT OR Apache-2.0; footage public domain | | `docs/images/filmcraft-timeline.png` | FilmCraft contributors | Original work: FilmCraft UI crop; thumbnails from Night of the Living Dead (1968) and Carnival of Souls (1962), CC0 Chopin waveform | MIT OR Apache-2.0; film frames public domain | +## Golden test images + +Reference frames for the renderer golden tests (`crates/golden`), rendered by FilmCraft itself from +procedural demo footage (`crates/media/src/generators.rs`); no external media. Regenerate with +`FILMCRAFT_BLESS=1 cargo test -p filmcraft-golden`. + +| Asset | Author | Source | Licence | +|---|---|---|---| +| `crates/golden/goldens/blend_modes.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `blend_modes`) from procedurally generated media | MIT OR Apache-2.0 | +| `crates/golden/goldens/crop.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `crop`) from procedurally generated media | MIT OR Apache-2.0 | +| `crates/golden/goldens/gaussian_blur.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `gaussian_blur`) from procedurally generated media | MIT OR Apache-2.0 | +| `crates/golden/goldens/lumetri_basic.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `lumetri_basic`) from procedurally generated media | MIT OR Apache-2.0 | +| `crates/golden/goldens/text_burnin.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `text_burnin`) from procedurally generated media | MIT OR Apache-2.0 | +| `crates/golden/goldens/transform_opacity.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transform_opacity`) from procedurally generated media | MIT OR Apache-2.0 | +| `crates/golden/goldens/transition_cross_dissolve.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transition_cross_dissolve`) from procedurally generated media | MIT OR Apache-2.0 | +| `crates/golden/goldens/transition_dip_to_black.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transition_dip_to_black`) from procedurally generated media | MIT OR Apache-2.0 | +| `crates/golden/goldens/transition_wipe.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `transition_wipe`) from procedurally generated media | MIT OR Apache-2.0 | + ## Third-party media shown in screenshots The README screenshots show frames from these works. The media files themselves are not in the diff --git a/Cargo.lock b/Cargo.lock index c0c0ca828..9bc39411d 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1451,6 +1451,21 @@ dependencies = [ "serde", ] +[[package]] +name = "filmcraft-golden" +version = "0.1.0" +dependencies = [ + "filmcraft-geom", + "filmcraft-gpu", + "filmcraft-media", + "filmcraft-project", + "filmcraft-render", + "filmcraft-testkit", + "filmcraft-time", + "pollster 0.4.0", + "wgpu", +] + [[package]] name = "filmcraft-gpu" version = "0.1.0" diff --git a/crates/golden/Cargo.toml b/crates/golden/Cargo.toml new file mode 100644 index 000000000..8ed401cb5 --- /dev/null +++ b/crates/golden/Cargo.toml @@ -0,0 +1,26 @@ +[package] +name = "filmcraft-golden" +description = "Golden-image tests for the CPU renderer/compositor and GPU-vs-CPU parity (test-only crate)" +version.workspace = true +edition.workspace = true +license.workspace = true +rust-version.workspace = true +publish = false + +[lints] +workspace = true + +# Test-only: the library is empty and everything is a dev-dependency, so the crate adds no edges +# to the layer graph. Goldens live in `goldens/` (see docs/testing.md §3). +[dependencies] + +[dev-dependencies] +filmcraft-testkit = { workspace = true } +filmcraft-time = { workspace = true } +filmcraft-geom = { workspace = true } +filmcraft-media = { workspace = true } +filmcraft-project = { workspace = true } +filmcraft-render = { workspace = true } +filmcraft-gpu = { workspace = true } +pollster = "0.4" +wgpu = { version = "30", features = ["wgsl", "metal", "vulkan", "dx12", "gles"] } diff --git a/crates/golden/goldens/blend_modes.png b/crates/golden/goldens/blend_modes.png new file mode 100644 index 000000000..97ec12057 Binary files /dev/null and b/crates/golden/goldens/blend_modes.png differ diff --git a/crates/golden/goldens/blend_modes.png.attribution b/crates/golden/goldens/blend_modes.png.attribution new file mode 100644 index 000000000..89c40f8e0 --- /dev/null +++ b/crates/golden/goldens/blend_modes.png.attribution @@ -0,0 +1,8 @@ +asset: blend_modes.png +title: Blend modes (Multiply, Screen, Overlay, Difference) +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (blend_modes). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/goldens/crop.png b/crates/golden/goldens/crop.png new file mode 100644 index 000000000..cdc7b10ab Binary files /dev/null and b/crates/golden/goldens/crop.png differ diff --git a/crates/golden/goldens/crop.png.attribution b/crates/golden/goldens/crop.png.attribution new file mode 100644 index 000000000..811cf9951 --- /dev/null +++ b/crates/golden/goldens/crop.png.attribution @@ -0,0 +1,8 @@ +asset: crop.png +title: Crop effect +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (crop). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/goldens/gaussian_blur.png b/crates/golden/goldens/gaussian_blur.png new file mode 100644 index 000000000..ecf3fdfdd Binary files /dev/null and b/crates/golden/goldens/gaussian_blur.png differ diff --git a/crates/golden/goldens/gaussian_blur.png.attribution b/crates/golden/goldens/gaussian_blur.png.attribution new file mode 100644 index 000000000..1e0c7bf7c --- /dev/null +++ b/crates/golden/goldens/gaussian_blur.png.attribution @@ -0,0 +1,8 @@ +asset: gaussian_blur.png +title: Gaussian Blur effect +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (gaussian_blur). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/goldens/lumetri_basic.png b/crates/golden/goldens/lumetri_basic.png new file mode 100644 index 000000000..7ae95cb90 Binary files /dev/null and b/crates/golden/goldens/lumetri_basic.png differ diff --git a/crates/golden/goldens/lumetri_basic.png.attribution b/crates/golden/goldens/lumetri_basic.png.attribution new file mode 100644 index 000000000..29d170f6e --- /dev/null +++ b/crates/golden/goldens/lumetri_basic.png.attribution @@ -0,0 +1,8 @@ +asset: lumetri_basic.png +title: Lumetri Color basic correction +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (lumetri_basic). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/goldens/text_burnin.png b/crates/golden/goldens/text_burnin.png new file mode 100644 index 000000000..e447d3dd0 Binary files /dev/null and b/crates/golden/goldens/text_burnin.png differ diff --git a/crates/golden/goldens/text_burnin.png.attribution b/crates/golden/goldens/text_burnin.png.attribution new file mode 100644 index 000000000..b15c28c36 --- /dev/null +++ b/crates/golden/goldens/text_burnin.png.attribution @@ -0,0 +1,8 @@ +asset: text_burnin.png +title: Timecode and Clip Name burn-in text +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (text_burnin). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/goldens/transform_opacity.png b/crates/golden/goldens/transform_opacity.png new file mode 100644 index 000000000..f59dff28a Binary files /dev/null and b/crates/golden/goldens/transform_opacity.png differ diff --git a/crates/golden/goldens/transform_opacity.png.attribution b/crates/golden/goldens/transform_opacity.png.attribution new file mode 100644 index 000000000..7d0e12b74 --- /dev/null +++ b/crates/golden/goldens/transform_opacity.png.attribution @@ -0,0 +1,8 @@ +asset: transform_opacity.png +title: Motion transform and opacity +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (transform_opacity). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/goldens/transition_cross_dissolve.png b/crates/golden/goldens/transition_cross_dissolve.png new file mode 100644 index 000000000..2ef34ac41 Binary files /dev/null and b/crates/golden/goldens/transition_cross_dissolve.png differ diff --git a/crates/golden/goldens/transition_cross_dissolve.png.attribution b/crates/golden/goldens/transition_cross_dissolve.png.attribution new file mode 100644 index 000000000..f8d4c0254 --- /dev/null +++ b/crates/golden/goldens/transition_cross_dissolve.png.attribution @@ -0,0 +1,8 @@ +asset: transition_cross_dissolve.png +title: Cross Dissolve at 50% +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (transition_cross_dissolve). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/goldens/transition_dip_to_black.png b/crates/golden/goldens/transition_dip_to_black.png new file mode 100644 index 000000000..0a40896b8 Binary files /dev/null and b/crates/golden/goldens/transition_dip_to_black.png differ diff --git a/crates/golden/goldens/transition_dip_to_black.png.attribution b/crates/golden/goldens/transition_dip_to_black.png.attribution new file mode 100644 index 000000000..57ff8d19a --- /dev/null +++ b/crates/golden/goldens/transition_dip_to_black.png.attribution @@ -0,0 +1,8 @@ +asset: transition_dip_to_black.png +title: Dip to Black at 25% +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (transition_dip_to_black). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/goldens/transition_wipe.png b/crates/golden/goldens/transition_wipe.png new file mode 100644 index 000000000..2548e9ce9 Binary files /dev/null and b/crates/golden/goldens/transition_wipe.png differ diff --git a/crates/golden/goldens/transition_wipe.png.attribution b/crates/golden/goldens/transition_wipe.png.attribution new file mode 100644 index 000000000..8ea1f4dff --- /dev/null +++ b/crates/golden/goldens/transition_wipe.png.attribution @@ -0,0 +1,8 @@ +asset: transition_wipe.png +title: Wipe at 50% +author: FilmCraft contributors +source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media) +license: MIT OR Apache-2.0 +license-file: +added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1) +notes: Golden reference image generated by crates/golden/tests/golden.rs (transition_wipe). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork. diff --git a/crates/golden/src/lib.rs b/crates/golden/src/lib.rs new file mode 100644 index 000000000..71c8cad0e --- /dev/null +++ b/crates/golden/src/lib.rs @@ -0,0 +1,8 @@ +//! Golden-image tests for FilmCraft's renderer (test-only crate; the library is empty). +//! +//! `tests/golden.rs` builds small procedural projects (no media files), renders one frame of each +//! through the CPU reference compositor (`filmcraft-render`) and compares it with a committed PNG +//! in `goldens/` using `filmcraft_testkit::golden`. The same scenes are composited on the GPU +//! (`filmcraft-gpu`) when an adapter is available and compared with the CPU result. +//! +//! Regenerate references with `FILMCRAFT_BLESS=1 cargo test -p filmcraft-golden`. diff --git a/crates/golden/tests/golden.rs b/crates/golden/tests/golden.rs new file mode 100644 index 000000000..ddc3da30b --- /dev/null +++ b/crates/golden/tests/golden.rs @@ -0,0 +1,345 @@ +//! Golden-image tests: procedural projects rendered through the CPU compositor and compared with +//! committed reference PNGs (`goldens/*.png`, made by this test with `FILMCRAFT_BLESS=1`), plus +//! GPU-vs-CPU parity on the same scenes when a GPU adapter is available. +//! +//! Tolerances (8-bit sRGB levels, see `filmcraft_testkit::golden`): +//! - CPU vs golden: `Tolerance::RENDER` — PSNR ≥ 45 dB, max abs ≤ 12, 99th percentile ≤ 2. +//! - GPU vs CPU: 99th percentile of the per-pixel max channel difference ≤ 6 and mean < 1.5 +//! (antialiased edges and half-float textures differ slightly; the CPU is the reference). + +use std::path::PathBuf; +use std::sync::Arc; + +use filmcraft_geom::Vec2; +use filmcraft_media::generators::GeneratorSource; +use filmcraft_media::{DemoScene, Generator, MediaSource, SharedSource}; +use filmcraft_project::{ + ClipId, EffectInstance, ItemId, ItemKind, Label, MediaClip, MediaRef, ParamValue, Project, SequenceSettings, TrackItem, TrackKind, Transition, + TransitionId, find_effect, +}; +use filmcraft_render::{RenderOptions, SourceMap, render_sequence}; +use filmcraft_testkit::golden::{Rgba8, Tolerance, assert_golden, diff}; +use filmcraft_time::{FrameRate, TICKS_PER_SECOND, Tick, TimeRange}; + +const W: u32 = 320; +const H: u32 = 180; +const RATE: FrameRate = FrameRate::FPS_24; + +/// A procedurally generated project and the frame to render. +struct Scene { + project: Project, + seq: ItemId, + sources: SourceMap, + frame: i64, +} + +struct Builder { + p: Project, + map: SourceMap, + seq: ItemId, + next_transition: u64, +} + +impl Builder { + fn new(video_tracks: usize) -> Self { + let mut p = Project::new("golden"); + let seq = p.new_sequence("golden", SequenceSettings { width: W, height: H, frame_rate: RATE, ..Default::default() }, video_tracks, 0, None); + Builder { p, map: SourceMap::default(), seq, next_transition: 1 } + } + + /// A generated media item (640×360, 24 fps, 10 s). + fn media(&mut self, g: Generator) -> ItemId { + let src = GeneratorSource::new(g, 640, 360, RATE, Tick(10 * TICKS_PER_SECOND)); + let info = src.info().clone(); + let id = self.p.add_item( + &info.name.clone(), + Label::Iris, + ItemKind::Media(MediaClip { + media: MediaRef::Generator(src.generator.clone()), + info, + interpret: Default::default(), + mark_in: None, + mark_out: None, + markers: vec![], + offline: false, + proxy: None, + }), + None, + ); + self.map.0.insert(id, Arc::new(src) as SharedSource); + id + } + + fn demo(&mut self, scene: DemoScene) -> ItemId { + self.media(Generator::Demo(scene)) + } + + /// Place `item` on video track `track` (frames), scaled to the frame size. + fn place(&mut self, track: usize, item: ItemId, start: i64, dur: i64) -> ClipId { + let mut ti = self.p.make_track_item(item, TrackKind::Video, RATE.tick_of(start), TimeRange::new(Tick::ZERO, RATE.tick_of(dur)), RATE).unwrap(); + ti.scale_to_frame = true; + let id = ti.id; + let s = self.p.sequence_mut(self.seq).unwrap(); + s.video_tracks[track].items.push(ti); + s.video_tracks[track].sort(); + id + } + + fn clip(&mut self, id: ClipId) -> &mut TrackItem { + self.p.sequence_mut(self.seq).unwrap().find_item_mut(id).unwrap().1 + } + + /// Set intrinsic-effect parameters (`motion`, `opacity`). + fn fixed(&mut self, id: ClipId, effect: &str, params: &[(&str, ParamValue)]) { + let e = self.clip(id).effect_mut(effect).unwrap(); + set(e, params); + } + + /// Append a standard effect with parameters. + fn effect(&mut self, id: ClipId, effect: &str, params: &[(&str, ParamValue)]) { + let mut e = find_effect(effect).unwrap_or_else(|| panic!("no effect {effect}")).instance(); + set(&mut e, params); + self.clip(id).effects.push(e); + } + + /// A transition of `dur` frames centred on the cut at `cut` between `a` and `b` on `track`. + fn transition(&mut self, track: usize, effect: &str, a: ClipId, b: ClipId, cut: i64, dur: i64) { + let id = TransitionId(self.next_transition); + self.next_transition += 1; + let tr = Transition { + id, + effect: find_effect(effect).unwrap_or_else(|| panic!("no transition {effect}")).instance(), + start: RATE.tick_of(cut - dur / 2), + duration: RATE.tick_of(dur), + from: Some(a), + to: Some(b), + align: Default::default(), + reverse: false, + }; + self.p.sequence_mut(self.seq).unwrap().video_tracks[track].transitions.push(tr); + } + + fn at(self, frame: i64) -> Scene { + Scene { project: self.p, seq: self.seq, sources: self.map, frame } + } +} + +fn set(e: &mut EffectInstance, params: &[(&str, ParamValue)]) { + let id = e.effect.clone(); + for (k, v) in params { + e.param_mut(k).unwrap_or_else(|| panic!("{id}: no param {k}")).value = v.clone(); + } +} + +fn fl(v: f64) -> ParamValue { + ParamValue::Float(v) +} + +fn pt(x: f64, y: f64) -> ParamValue { + ParamValue::Vec2(Vec2::new(x, y)) +} + +fn blend(name: &str) -> ParamValue { + ParamValue::Choice(filmcraft_project::effect::BLEND_MODES.iter().position(|b| *b == name).unwrap_or_else(|| panic!("blend {name}")) as u32) +} + +// ---- scenes ---- + +/// Motion (scale, rotation, position) and 70 % opacity over a full-frame background. +fn transform_opacity() -> Scene { + let mut b = Builder::new(2); + let bg = b.demo(DemoScene::OceanSunset); + let fg = b.demo(DemoScene::Aurora); + b.place(0, bg, 0, 48); + let top = b.place(1, fg, 0, 48); + b.fixed(top, "motion", &[("scale", fl(45.0)), ("rotation", fl(20.0)), ("position", pt(215.0, 70.0))]); + b.fixed(top, "opacity", &[("opacity", fl(70.0))]); + b.at(12) +} + +/// Four quadrant layers over a background in Multiply, Screen, Overlay and Difference. +fn blend_modes() -> Scene { + let mut b = Builder::new(5); + let bg = b.demo(DemoScene::OceanSunset); + let fg = b.demo(DemoScene::Dunes); + b.place(0, bg, 0, 48); + for (i, (mode, x, y)) in [("Multiply", 80.0, 45.0), ("Screen", 240.0, 45.0), ("Overlay", 80.0, 135.0), ("Difference", 240.0, 135.0)].into_iter().enumerate() + { + let c = b.place(i + 1, fg, 0, 48); + b.fixed(c, "motion", &[("scale", fl(50.0)), ("position", pt(x, y))]); + b.fixed(c, "opacity", &[("blend", blend(mode))]); + } + b.at(12) +} + +/// Gaussian Blur on colour bars (sharp edges show the kernel). +fn gaussian_blur() -> Scene { + let mut b = Builder::new(1); + let bars = b.media(Generator::BarsAndTone); + let c = b.place(0, bars, 0, 48); + b.effect(c, "gaussian_blur", &[("blurriness", fl(12.0))]); + b.at(12) +} + +/// Lumetri Color basic correction: temperature, exposure, contrast, highlights/shadows, saturation. +fn lumetri_basic() -> Scene { + let mut b = Builder::new(1); + let bg = b.demo(DemoScene::OceanSunset); + let c = b.place(0, bg, 0, 48); + b.effect( + c, + "lumetri", + &[ + ("temperature", fl(25.0)), + ("exposure", fl(0.6)), + ("contrast", fl(30.0)), + ("highlights", fl(-30.0)), + ("shadows", fl(25.0)), + ("saturation", fl(135.0)), + ], + ); + b.at(12) +} + +/// Crop (left/top/right/bottom) of a top layer revealing the background. +fn crop() -> Scene { + let mut b = Builder::new(2); + let bg = b.demo(DemoScene::OceanSunset); + let bars = b.media(Generator::BarsAndTone); + b.place(0, bg, 0, 48); + let c = b.place(1, bars, 0, 48); + b.effect(c, "crop", &[("left", fl(15.0)), ("top", fl(10.0)), ("right", fl(25.0)), ("bottom", fl(20.0))]); + b.at(12) +} + +/// Two clips with a 12-frame transition centred on the cut at frame 24, rendered at `frame`. +fn transition(effect: &str, frame: i64) -> Scene { + let mut b = Builder::new(1); + let a = b.demo(DemoScene::OceanSunset); + let c = b.demo(DemoScene::Aurora); + let ca = b.place(0, a, 0, 24); + let cb = b.place(0, c, 24, 24); + b.transition(0, effect, ca, cb, 24, 12); + b.at(frame) +} + +/// Timecode and Clip Name burn-ins (FilmCraft's built-in bitmap text). +fn text_burnin() -> Scene { + let mut b = Builder::new(1); + let bg = b.demo(DemoScene::CityNight); + let c = b.place(0, bg, 0, 48); + b.effect(c, "timecode", &[("position", pt(160.0, 40.0)), ("size", fl(20.0))]); + b.effect(c, "clip_name", &[("position", pt(160.0, 145.0)), ("size", fl(12.0))]); + b.at(30) +} + +/// (name, title, scene). +fn scenes() -> Vec<(&'static str, &'static str, fn() -> Scene)> { + vec![ + ("transform_opacity", "Motion transform and opacity", transform_opacity), + ("blend_modes", "Blend modes (Multiply, Screen, Overlay, Difference)", blend_modes), + ("gaussian_blur", "Gaussian Blur effect", gaussian_blur), + ("lumetri_basic", "Lumetri Color basic correction", lumetri_basic), + ("crop", "Crop effect", crop), + ("transition_cross_dissolve", "Cross Dissolve at 50%", || transition("cross_dissolve", 24)), + ("transition_dip_to_black", "Dip to Black at 25%", || transition("dip_to_black", 21)), + ("transition_wipe", "Wipe at 50%", || transition("wipe", 24)), + ("text_burnin", "Timecode and Clip Name burn-in text", text_burnin), + ] +} + +fn render_cpu(s: &Scene) -> Rgba8 { + let img = render_sequence(&s.project, s.seq, RATE.tick_of(s.frame), RenderOptions::default(), &s.sources); + Rgba8::new(img.w as u32, img.h as u32, img.over_black_rgba8()) +} + +fn golden_path(name: &str) -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("goldens").join(format!("{name}.png")) +} + +fn check(name: &str) { + let (_, title, make) = scenes().into_iter().find(|(n, _, _)| *n == name).unwrap(); + let img = render_cpu(&make()); + assert_eq!((img.w, img.h), (W, H)); + let d = assert_golden(&golden_path(name), &img, Tolerance::RENDER, title, &format!("crates/golden/tests/golden.rs ({name})")); + eprintln!("{name}: {d}"); +} + +macro_rules! goldens { + ($($name:ident),* $(,)?) => { + mod cpu { + $( + #[test] + fn $name() { + super::check(stringify!($name)); + } + )* + } + + #[test] + fn every_scene_has_a_test() { + let tested = [$(stringify!($name)),*]; + for (n, _, _) in scenes() { + assert!(tested.contains(&n), "scene {n} has no golden test"); + } + } + }; +} + +goldens!(transform_opacity, blend_modes, gaussian_blur, lumetri_basic, crop, transition_cross_dissolve, transition_dip_to_black, transition_wipe, text_burnin); + +/// Sanity: the scenes are not trivially empty or identical to each other. +#[test] +fn scenes_are_distinct() { + let imgs: Vec<(&str, Rgba8)> = scenes().into_iter().map(|(n, _, f)| (n, render_cpu(&f()))).collect(); + for (n, img) in &imgs { + let lit = img.px.chunks_exact(4).filter(|p| p[0] as u32 + p[1] as u32 + p[2] as u32 > 30).count(); + assert!(lit > img.px.len() / 4 / 4, "{n}: mostly black"); + } + for i in 0..imgs.len() { + for j in i + 1..imgs.len() { + let d = diff(&imgs[i].1, &imgs[j].1).unwrap(); + assert!(d.psnr < 30.0, "{} and {} are near-identical ({d})", imgs[i].0, imgs[j].0); + } + } +} + +// ---- GPU parity ---- + +fn device() -> Option<(wgpu::Device, wgpu::Queue)> { + let instance = wgpu::Instance::default(); + let adapter = pollster::block_on(instance.request_adapter(&wgpu::RequestAdapterOptions::default())).ok()?; + pollster::block_on(adapter.request_device(&wgpu::DeviceDescriptor::default())).ok() +} + +#[test] +fn gpu_matches_cpu_on_golden_scenes() { + let Some((dev, q)) = device() else { + eprintln!("SKIPPED (gpu parity): no GPU adapter"); + return; + }; + let mut c = filmcraft_gpu::GpuCompositor::new(&dev, &q); + let mut failures = Vec::new(); + for (name, _, make) in scenes() { + let s = make(); + let t = RATE.tick_of(s.frame); + let cpu = render_cpu(&s); + let plan = filmcraft_render::plan::plan_frame(&s.project, s.seq, t, RenderOptions::default(), &s.sources); + let kind = if matches!(plan, filmcraft_render::plan::FramePlan::Layers { .. }) { "layers" } else { "cpu image" }; + c.composite(&plan); + let (gw, gh, mut px) = c.read_output().expect("GPU readback"); + px.chunks_exact_mut(4).for_each(|p| p[3] = 255); + let gpu = Rgba8::new(gw, gh, px); + let d = diff(&cpu, &gpu).unwrap(); + // mean over RGB only (alpha is equal by construction) + let mean_rgb = d.mean_abs * 4.0 / 3.0; + eprintln!("{name} ({kind}): GPU vs CPU {d}"); + if d.p99 > 6 || mean_rgb >= 1.5 { + let dir = filmcraft_testkit::golden::failures_dir(); + let _ = filmcraft_testkit::golden::write_png(&dir.join(format!("{name}.gpu.png")), &gpu); + let _ = filmcraft_testkit::golden::write_png(&dir.join(format!("{name}.cpu.png")), &cpu); + failures.push(format!("{name} ({kind}): {d}; images in {}", dir.display())); + } + } + assert!(failures.is_empty(), "GPU differs from the CPU reference:\n{}", failures.join("\n")); +} diff --git a/crates/gpu/src/lib.rs b/crates/gpu/src/lib.rs index 83e46f795..790861692 100644 --- a/crates/gpu/src/lib.rs +++ b/crates/gpu/src/lib.rs @@ -27,6 +27,9 @@ struct Uploaded { code_scale: f32, chroma: (u32, u32), last_used: u64, + /// The uploaded pixel buffers, kept alive while cached: the cache key is the buffer address, + /// and a freed buffer's address can be reused by a different frame (stale texture). + _pixels: PixelData, } pub struct GpuCompositor { @@ -245,12 +248,12 @@ impl GpuCompositor { let up = match &f.data { PixelData::Rgba8(d) => { let v = self.plane_texture(w, h, wgpu::TextureFormat::Rgba8UnormSrgb, d, 4); - Uploaded { views: [v, dummy(), dummy()], kind: 0, code_scale: 1.0, chroma: (w, h), last_used: self.clock } + Uploaded { views: [v, dummy(), dummy()], kind: 0, code_scale: 1.0, chroma: (w, h), last_used: self.clock, _pixels: f.data.clone() } } PixelData::RgbaF32(d) => { let half: Vec = d.iter().flat_map(|v| f32_to_f16(*v).to_le_bytes()).collect(); let v = self.plane_texture(w, h, wgpu::TextureFormat::Rgba16Float, &half, 8); - Uploaded { views: [v, dummy(), dummy()], kind: 1, code_scale: 1.0, chroma: (w, h), last_used: self.clock } + Uploaded { views: [v, dummy(), dummy()], kind: 1, code_scale: 1.0, chroma: (w, h), last_used: self.clock, _pixels: f.data.clone() } } PixelData::Yuv8 { planes, chroma, .. } => { let (sx, sy) = chroma.shifts(); @@ -258,7 +261,7 @@ impl GpuCompositor { let y = self.plane_texture(w, h, wgpu::TextureFormat::R8Unorm, &planes[0], 1); let u = self.plane_texture(cw, ch, wgpu::TextureFormat::R8Unorm, &planes[1], 1); let v = self.plane_texture(cw, ch, wgpu::TextureFormat::R8Unorm, &planes[2], 1); - Uploaded { views: [y, u, v], kind: 2, code_scale: 255.0, chroma: (cw, ch), last_used: self.clock } + Uploaded { views: [y, u, v], kind: 2, code_scale: 255.0, chroma: (cw, ch), last_used: self.clock, _pixels: f.data.clone() } } PixelData::Yuv16 { planes, chroma, bits, .. } => { let (sx, sy) = chroma.shifts(); @@ -270,7 +273,7 @@ impl GpuCompositor { let u = self.plane_texture(cw, ch, wgpu::TextureFormat::R16Float, &pu, 2); let v = self.plane_texture(cw, ch, wgpu::TextureFormat::R16Float, &pv, 2); let _ = Chroma::C420; - Uploaded { views: [y, u, v], kind: 2, code_scale: scale, chroma: (cw, ch), last_used: self.clock } + Uploaded { views: [y, u, v], kind: 2, code_scale: scale, chroma: (cw, ch), last_used: self.clock, _pixels: f.data.clone() } } }; self.uploads.insert(key, up); diff --git a/crates/gpu/src/tests.rs b/crates/gpu/src/tests.rs index 9aacbde97..f787d7056 100644 --- a/crates/gpu/src/tests.rs +++ b/crates/gpu/src/tests.rs @@ -77,3 +77,21 @@ fn half_float_conversion() { assert!((d - v).abs() <= v.abs() * 1e-3 + 1e-6, "{v} → {d}"); } } + +/// The upload cache is keyed by pixel-buffer address, so it must keep the buffer alive: otherwise +/// a new frame allocated at a freed frame's address is drawn with the stale texture (seen as +/// whole frames from a previous composite in `crates/golden` GPU parity). +#[test] +fn upload_cache_keeps_buffers_alive() { + let Some((dev, q)) = device() else { + eprintln!("no GPU adapter; skipping"); + return; + }; + let mut c = GpuCompositor::new(&dev, &q); + let px = Arc::new(vec![200u8; 16 * 8 * 4]); + let frame = Arc::new(VideoFrame { width: 16, height: 8, data: PixelData::Rgba8(px.clone()), ..(*yuv_frame(16, 8)).clone() }); + let plan = FramePlan::Layers { width: 16, height: 8, layers: vec![PlanLayer { frame, matrix: Affine::IDENTITY, opacity: 1.0 }] }; + c.composite(&plan); + drop(plan); + assert!(Arc::strong_count(&px) > 1, "cached upload must own its pixel buffer"); +} diff --git a/crates/testkit/src/golden.rs b/crates/testkit/src/golden.rs new file mode 100644 index 000000000..d79fbad02 --- /dev/null +++ b/crates/testkit/src/golden.rs @@ -0,0 +1,344 @@ +//! Golden-image testing: RGBA8 images, PNG I/O, difference metrics and reference checks. +//! +//! A golden test renders an image, then calls [`assert_golden`] with the path of a committed +//! reference PNG and a [`Tolerance`]. On mismatch the actual image and an amplified difference +//! image are written to `/target/golden-failures/` for inspection. +//! +//! `FILMCRAFT_BLESS=1` (re)writes the reference instead of comparing, plus its `.attribution` +//! sidecar when missing (AGENTS.md §1). References must be produced by FilmCraft itself (original +//! work), be small (< [`MAX_GOLDEN_BYTES`]) and be listed in `ATTRIBUTION.md`. + +use std::io::{BufReader, Cursor}; +use std::path::{Path, PathBuf}; + +/// Size limit for a committed golden PNG. +pub const MAX_GOLDEN_BYTES: u64 = 50 * 1024; + +/// An 8-bit RGBA image, row-major, no padding. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Rgba8 { + pub w: u32, + pub h: u32, + pub px: Vec, +} + +impl Rgba8 { + pub fn new(w: u32, h: u32, px: Vec) -> Self { + assert_eq!(px.len(), w as usize * h as usize * 4, "pixel buffer size"); + Self { w, h, px } + } + pub fn get(&self, x: u32, y: u32) -> [u8; 4] { + let i = (y as usize * self.w as usize + x as usize) * 4; + [self.px[i], self.px[i + 1], self.px[i + 2], self.px[i + 3]] + } + fn opaque(&self) -> bool { + self.px.chunks_exact(4).all(|p| p[3] == 255) + } +} + +/// Encode as PNG (RGB when fully opaque, else RGBA), maximum compression. +pub fn encode_png(img: &Rgba8) -> Result, String> { + let mut out = Vec::new(); + { + let mut enc = png::Encoder::new(&mut out, img.w, img.h); + let opaque = img.opaque(); + enc.set_color(if opaque { png::ColorType::Rgb } else { png::ColorType::Rgba }); + enc.set_depth(png::BitDepth::Eight); + enc.set_compression(png::Compression::High); + let mut w = enc.write_header().map_err(|e| e.to_string())?; + if opaque { + let rgb: Vec = img.px.chunks_exact(4).flat_map(|p| [p[0], p[1], p[2]]).collect(); + w.write_image_data(&rgb).map_err(|e| e.to_string())?; + } else { + w.write_image_data(&img.px).map_err(|e| e.to_string())?; + } + w.finish().map_err(|e| e.to_string())?; + } + Ok(out) +} + +/// Decode a PNG (8-bit gray/RGB/RGBA, palette expanded) to RGBA8. +pub fn decode_png(bytes: &[u8]) -> Result { + let mut dec = png::Decoder::new(BufReader::new(Cursor::new(bytes))); + dec.set_transformations(png::Transformations::EXPAND | png::Transformations::STRIP_16); + let mut r = dec.read_info().map_err(|e| e.to_string())?; + let mut buf = vec![0; r.output_buffer_size().ok_or("PNG too large")?]; + let info = r.next_frame(&mut buf).map_err(|e| e.to_string())?; + let (w, h) = (info.width, info.height); + let n = w as usize * h as usize; + let px = match info.color_type { + png::ColorType::Rgba => buf[..n * 4].to_vec(), + png::ColorType::Rgb => buf[..n * 3].chunks_exact(3).flat_map(|p| [p[0], p[1], p[2], 255]).collect(), + png::ColorType::GrayscaleAlpha => buf[..n * 2].chunks_exact(2).flat_map(|p| [p[0], p[0], p[0], p[1]]).collect(), + png::ColorType::Grayscale => buf[..n].iter().flat_map(|&g| [g, g, g, 255]).collect(), + png::ColorType::Indexed => return Err("unexpanded palette PNG".into()), + }; + Ok(Rgba8 { w, h, px }) +} + +pub fn read_png(path: &Path) -> Result { + let bytes = std::fs::read(path).map_err(|e| format!("{}: {e}", path.display()))?; + decode_png(&bytes).map_err(|e| format!("{}: {e}", path.display())) +} + +pub fn write_png(path: &Path, img: &Rgba8) -> Result { + let bytes = encode_png(img)?; + if let Some(d) = path.parent() { + std::fs::create_dir_all(d).map_err(|e| e.to_string())?; + } + std::fs::write(path, &bytes).map_err(|e| format!("{}: {e}", path.display()))?; + Ok(bytes.len() as u64) +} + +/// Per-pixel difference statistics over all four channels (8-bit levels). +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct Diff { + /// Largest absolute channel difference. + pub max_abs: u8, + /// Mean absolute channel difference. + pub mean_abs: f64, + /// 99th percentile of the per-pixel maximum channel difference. + pub p99: u8, + /// PSNR in dB over all channels (`f64::INFINITY` when identical). + pub psnr: f64, + /// Pixels with any channel different. + pub differing: usize, +} + +impl std::fmt::Display for Diff { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "PSNR {:.2} dB, max {}, p99 {}, mean {:.3}, {} pixels differ", self.psnr, self.max_abs, self.p99, self.mean_abs, self.differing) + } +} + +/// Compare two images of equal size. +pub fn diff(a: &Rgba8, b: &Rgba8) -> Result { + if (a.w, a.h) != (b.w, b.h) { + return Err(format!("size {}x{} vs {}x{}", a.w, a.h, b.w, b.h)); + } + let mut hist = [0usize; 256]; + let (mut sum, mut sq, mut max, mut differing) = (0u64, 0u64, 0u8, 0usize); + for (p, q) in a.px.chunks_exact(4).zip(b.px.chunks_exact(4)) { + let mut m = 0u8; + for k in 0..4 { + let d = p[k].abs_diff(q[k]); + sum += d as u64; + sq += d as u64 * d as u64; + m = m.max(d); + } + hist[m as usize] += 1; + max = max.max(m); + differing += (m > 0) as usize; + } + let pixels = (a.w as usize * a.h as usize).max(1); + let target = (pixels * 99).div_ceil(100); + let mut acc = 0; + let mut p99 = 0u8; + for (v, n) in hist.iter().enumerate() { + acc += n; + if acc >= target { + p99 = v as u8; + break; + } + } + let n = (pixels * 4) as f64; + let mse = sq as f64 / n; + let psnr = if mse == 0.0 { f64::INFINITY } else { 10.0 * (255.0f64 * 255.0 / mse).log10() }; + Ok(Diff { max_abs: max, mean_abs: sum as f64 / n, p99, psnr, differing }) +} + +/// Acceptance thresholds; every set limit must hold. +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct Tolerance { + pub min_psnr: f64, + pub max_abs: u8, + pub p99: u8, +} + +impl Tolerance { + /// Bit-exact. + pub const EXACT: Tolerance = Tolerance { min_psnr: f64::INFINITY, max_abs: 0, p99: 0 }; + /// Default for CPU renders compared with CPU-made references: absorbs float differences + /// between platforms (FMA, SIMD widths, thread scheduling of reductions) but not real changes. + pub const RENDER: Tolerance = Tolerance { min_psnr: 45.0, max_abs: 12, p99: 2 }; + + pub fn accepts(&self, d: &Diff) -> bool { + d.psnr >= self.min_psnr && d.max_abs <= self.max_abs && d.p99 <= self.p99 + } +} + +/// Whether `FILMCRAFT_BLESS` asks to regenerate references. +pub fn blessing() -> bool { + std::env::var("FILMCRAFT_BLESS").is_ok_and(|v| matches!(v.trim().to_ascii_lowercase().as_str(), "1" | "true" | "yes" | "on")) +} + +/// Where failing comparisons write `.actual.png` / `.diff.png`. +pub fn failures_dir() -> PathBuf { + crate::workspace_root().join("target").join("golden-failures") +} + +/// Difference image: |a − b| per channel ×8, opaque. +pub fn diff_image(a: &Rgba8, b: &Rgba8) -> Rgba8 { + let px = + a.px.chunks_exact(4) + .zip(b.px.chunks_exact(4)) + .flat_map(|(p, q)| { + let d = |k: usize| (p[k].abs_diff(q[k]) as u32 * 8).min(255) as u8; + let al = d(3); + [d(0).max(al), d(1).max(al), d(2).max(al), 255] + }) + .collect(); + Rgba8 { w: a.w, h: a.h, px } +} + +/// Today's date (UTC) as YYYY-MM-DD, for attribution sidecars. +fn today() -> String { + let secs = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0); + // civil-from-days (Howard Hinnant) + let z = (secs / 86_400) as i64 + 719_468; + let era = z.div_euclid(146_097); + let doe = z - era * 146_097; + let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365; + let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); + let mp = (5 * doy + 2) / 153; + let d = doy - (153 * mp + 2) / 5 + 1; + let m = if mp < 10 { mp + 3 } else { mp - 9 }; + let y = yoe + era * 400 + (m <= 2) as i64; + format!("{y:04}-{m:02}-{d:02}") +} + +/// The attribution sidecar text for a generated golden image. +pub fn sidecar_text(file_name: &str, title: &str, generated_by: &str) -> String { + format!( + "asset: {file_name}\n\ + title: {title}\n\ + author: FilmCraft contributors\n\ + source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)\n\ + license: MIT OR Apache-2.0\n\ + license-file:\n\ + added: {} by FilmCraft contributors (FILMCRAFT_BLESS=1)\n\ + notes: Golden reference image generated by {generated_by}. Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.\n", + today() + ) +} + +/// The `ATTRIBUTION.md` table row for a golden image at workspace-relative `rel`. +pub fn attribution_row(rel: &str, generated_by: &str) -> String { + format!("| `{rel}` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft ({generated_by}) | MIT OR Apache-2.0 |") +} + +/// Compare `actual` with the golden PNG at `golden` (or bless it). `title` and `generated_by` +/// fill the attribution sidecar written on bless. Returns the difference on success. +pub fn check_golden(golden: &Path, actual: &Rgba8, tol: Tolerance, title: &str, generated_by: &str) -> Result { + let name = golden.file_stem().map(|s| s.to_string_lossy().to_string()).unwrap_or_default(); + if blessing() { + let size = write_png(golden, actual)?; + if size > MAX_GOLDEN_BYTES { + return Err(format!("{}: {size} bytes exceeds the {MAX_GOLDEN_BYTES}-byte golden limit; render smaller", golden.display())); + } + let side = PathBuf::from(format!("{}.attribution", golden.display())); + if !side.exists() { + let file = golden.file_name().map(|s| s.to_string_lossy().to_string()).unwrap_or_default(); + std::fs::write(&side, sidecar_text(&file, title, generated_by)).map_err(|e| e.to_string())?; + } + let root = crate::workspace_root(); + let rel = golden.strip_prefix(&root).unwrap_or(golden).to_string_lossy().replace('\\', "/"); + let index = std::fs::read_to_string(root.join("ATTRIBUTION.md")).unwrap_or_default(); + if !index.contains(&format!("`{rel}`")) { + eprintln!("golden {rel}: add this row to ATTRIBUTION.md:\n{}", attribution_row(&rel, generated_by)); + } + eprintln!("blessed {rel} ({size} bytes)"); + return Ok(Diff { max_abs: 0, mean_abs: 0.0, p99: 0, psnr: f64::INFINITY, differing: 0 }); + } + let expected = read_png(golden).map_err(|e| format!("{e}\n(missing or unreadable golden; generate it with FILMCRAFT_BLESS=1)"))?; + let d = diff(&expected, actual).map_err(|e| format!("{}: {e}", golden.display()))?; + if tol.accepts(&d) { + return Ok(d); + } + let dir = failures_dir(); + let a = dir.join(format!("{name}.actual.png")); + let _ = write_png(&a, actual); + let _ = write_png(&dir.join(format!("{name}.diff.png")), &diff_image(&expected, actual)); + Err(format!( + "{}: {d} (tolerance: PSNR ≥ {} dB, max ≤ {}, p99 ≤ {}); actual and diff written to {}", + golden.display(), + tol.min_psnr, + tol.max_abs, + tol.p99, + dir.display() + )) +} + +/// [`check_golden`] that panics on failure. +pub fn assert_golden(golden: &Path, actual: &Rgba8, tol: Tolerance, title: &str, generated_by: &str) -> Diff { + match check_golden(golden, actual, tol, title, generated_by) { + Ok(d) => d, + Err(e) => panic!("golden mismatch: {e}"), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn ramp(w: u32, h: u32, alpha: u8) -> Rgba8 { + let px = (0..w * h).flat_map(|i| [(i % w * 255 / w) as u8, (i / w * 255 / h) as u8, 77, alpha]).collect(); + Rgba8::new(w, h, px) + } + + #[test] + fn png_round_trip_rgb_and_rgba() { + for a in [255, 128] { + let img = ramp(37, 21, a); + let png = encode_png(&img).unwrap(); + assert_eq!(decode_png(&png).unwrap(), img); + } + } + + #[test] + fn diff_metrics() { + let a = ramp(10, 10, 255); + assert_eq!(diff(&a, &a).unwrap().psnr, f64::INFINITY); + let mut b = a.clone(); + b.px[0] = b.px[0].wrapping_add(10); // one channel of one pixel + let d = diff(&a, &b).unwrap(); + assert_eq!((d.max_abs, d.p99, d.differing), (10, 0, 1)); + assert!((d.mean_abs - 10.0 / 400.0).abs() < 1e-12); + let mse: f64 = 100.0 / 400.0; + assert!((d.psnr - 10.0 * (255.0f64 * 255.0 / mse).log10()).abs() < 1e-9); + assert!(Tolerance::RENDER.accepts(&d)); + assert!(!Tolerance::EXACT.accepts(&d)); + assert!(diff(&a, &ramp(10, 11, 255)).is_err()); + } + + #[test] + fn check_golden_accepts_within_tolerance_and_reports_failures() { + if blessing() { + return; + } + let dir = std::env::temp_dir().join(format!("filmcraft-testkit-golden-{}", std::process::id())); + let path = dir.join("ramp.png"); + let img = ramp(32, 16, 255); + write_png(&path, &img).unwrap(); + assert!(check_golden(&path, &img, Tolerance::EXACT, "t", "unit test").is_ok()); + let mut near = img.clone(); + near.px[5] += 1; + assert!(check_golden(&path, &near, Tolerance::RENDER, "t", "unit test").is_ok()); + let mut far = img.clone(); + far.px.iter_mut().step_by(4).for_each(|v| *v = 255 - *v); + let e = check_golden(&path, &far, Tolerance::RENDER, "t", "unit test").unwrap_err(); + assert!(e.contains("PSNR"), "{e}"); + assert!(check_golden(&dir.join("missing.png"), &img, Tolerance::RENDER, "t", "unit test").unwrap_err().contains("FILMCRAFT_BLESS")); + } + + #[test] + fn date_and_sidecar_fields() { + let t = today(); + assert_eq!(t.len(), 10); + assert!(t.starts_with("20")); + let s = sidecar_text("x.png", "X", "test y"); + for f in ["asset:", "title:", "author:", "source:", "license:", "added:"] { + assert!(s.lines().any(|l| l.starts_with(f) && l.len() > f.len() + 2), "{f}"); + } + } +} diff --git a/crates/testkit/src/lib.rs b/crates/testkit/src/lib.rs index f617e9221..4abcd3a36 100644 --- a/crates/testkit/src/lib.rs +++ b/crates/testkit/src/lib.rs @@ -4,11 +4,14 @@ //! `FILMCRAFT_REQUIRE_ORACLES=1`, fail — when they are missing ([`require_ffmpeg!`] & co.). //! - [`fixtures`]: the shared fixture directory (`/target/fixtures/`) and //! collision-free temporary names for concurrent generators. +//! - [`golden`]: RGBA8 images, PNG I/O, image-difference metrics and golden-file checks +//! (`FILMCRAFT_BLESS=1` regenerates references). //! //! ffmpeg/ffprobe are external test oracles and fixture generators only; they are never linked //! or shipped (AGENTS.md §2). pub mod fixtures; +pub mod golden; pub mod oracle; pub use fixtures::{fixtures_dir, temp_path, workspace_root}; diff --git a/docs/architecture.md b/docs/architecture.md index 5a488be55..c225ff120 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -23,7 +23,7 @@ Design principles: L6 apps/filmcraft · apps/filmcraft-cli L5 ui-egui · automation L4 engine - L3 render · gpu · export + L3 render · gpu · export · golden (test-only) L2 edit · codecs · interchange L1 frame · media · project · audio-dsp L0 foundation: time · geom · color · bitstream · testkit (dev-dependency only) @@ -55,6 +55,7 @@ and `filmcraft-cli`. | `interchange` | L2 | EDL, FCP7 XML, FCPXML and OTIO import/export (no file I/O) | | `render` | L3 | sequence evaluation, CPU compositor, video effects, transitions, audio mix | | `gpu` | L3 | wgpu compositor (WGSL) | +| `golden` | L3 | test-only: golden-image tests of the CPU renderer and GPU-vs-CPU parity; empty library, dev-dependencies only | | `export` | L3 | render → encode → mux pipeline, progress/cancel | | `engine` | L4 | `Session`, command registry, undo history, media pool, jobs, interchange glue | | `ui-egui` | L5 | the egui frontend: docking, panels, timeline, monitors, playback, control-channel handlers | diff --git a/docs/testing.md b/docs/testing.md index 6153af5e5..56fca2664 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -82,9 +82,26 @@ Each codec README has the full fixture matrix and the measured results. difference of ≤ 6 and a mean of < 1.5 (8-bit levels). It skips when no GPU adapter is available. - Effects implemented only on the CPU need no GPU test: the plan pre-renders those layers on the CPU. -- Golden-image files (rendered PNGs compared against committed references) are not used yet. If you - add them, generate the references from FilmCraft itself, give each one an attribution sidecar, - and document the tolerance (PSNR in dB, or max/percentile error). +- **Golden images** (`crates/golden`, a test-only crate): `tests/golden.rs` builds small procedural + projects (demo-generator footage, bars, no media files) at 320×180 and renders one frame of each + through the CPU compositor: Motion transform + opacity, four blend modes (Multiply, Screen, + Overlay, Difference), Gaussian Blur, Lumetri basic correction, Crop, Cross Dissolve at 50 %, Dip + to Black at 25 %, Wipe at 50 %, and Timecode / Clip Name burn-in text. Each frame is compared + with `crates/golden/goldens/.png` by `filmcraft_testkit::golden`: + **PSNR ≥ 45 dB, max abs ≤ 12, 99th-percentile per-pixel max channel difference ≤ 2** (8-bit + sRGB levels; `Tolerance::RENDER`). On failure the actual frame and a ×8 difference image go to + `/target/golden-failures/`. `scenes_are_distinct` guards against blank or duplicate + scenes. +- **Blessing:** `FILMCRAFT_BLESS=1 cargo test -p filmcraft-golden` rewrites the references (and + writes a `.attribution` sidecar for any new one; it prints the `ATTRIBUTION.md` row to add). + References are original work rendered by FilmCraft, must stay under 50 KB (enforced on bless), + and every one needs its sidecar and index row (`cargo xtask assets`). Look at a re-blessed PNG + before committing it. +- The same scenes run through `plan_frame` + `GpuCompositor` when a GPU adapter exists + (`gpu_matches_cpu_on_golden_scenes`, same p99 ≤ 6 / mean < 1.5 criterion; skipped otherwise). + This found a real bug: the GPU upload cache was keyed by pixel-buffer address without keeping + the buffer alive, so a new frame allocated at a freed frame's address was drawn with the stale + texture (fixed; regression test `upload_cache_keeps_buffers_alive`). ## 4. UI: control channel and screenshots diff --git a/xtask/src/main.rs b/xtask/src/main.rs index 03e265913..ee24ea1e5 100644 --- a/xtask/src/main.rs +++ b/xtask/src/main.rs @@ -46,6 +46,7 @@ const LAYERS: &[(&str, u8)] = &[ ("interchange", 2), ("render", 3), ("gpu", 3), + ("golden", 3), ("scopes", 3), ("playback", 3), ("export", 3),