mirror of
https://github.com/storytold/filmcraft.git
synced 2026-10-24 10:10:05 -06:00
M8.8: colour management — sequence working space (Rec. 709 / Rec. 2100 PQ / HLG, wide gamut), Interpret Footage colour space, log/HDR decode with BT.2390 tone mapping, HDR export signalling, HDR scopes
- model: SequenceSettings::color (ColorPipeline: working space, wide gamut, Auto Tone Map Media; serde default Rec. 709) and Interpretation::color_space (override; None = metadata) - render::colorman: source colour space from the override or VUI/colr/MKV metadata, decode tables (camera log with super-whites, PQ, HLG + OOTF), gamut matrix, tone mapping into SDR working spaces, gamut mapping; working → SDR monitor transform at the top level (RenderOptions::working_output keeps working values for HDR export and scopes). Plain Rec. 709 media in a Rec. 709 sequence keeps the old fast path and stays on the GPU; managed layers and HDR/wide-gamut sequences composite on the CPU - export: H.264 and ProRes of a PQ/HLG sequence encode BT.2020 PQ/HLG and signal it: VUI / ProRes header, colr (nclx/nclc), and for PQ mdcv + clli boxes and H.264 SEI (payloadTypes 137/144); `sdr: true` (and render previews) export tone-mapped SDR. isobmff reads/writes mdcv and clli - commands: sequence.colorSettings (Sequence ▸ Color Management…), clip.interpretFootage (Clip ▸ Modify ▸ Interpret Footage…, Project panel context menu), color.spaces, media.colorInfo; dialogs with automation ids colorDialog.* - scopes: HDR sequences show a cd/m² waveform on a PQ scale (203 marked) and a BT.2020 vectorscope from the working-space render (scopes.hdrWaveform) - display-referred wide-gamut input is desaturated into gamut keeping in-gamut colours exactly; the soft compression is reserved for camera gamuts - tests: colorman unit tests, engine (interpret/undo, PQ working space values and display), goldens log_to_rec709 and hdr_tone_map (own renders + sidecars), export PQ/HLG × ProRes/H.264 read back by our demuxer and ffprobe (color_transfer/primaries/space, mastering display and content light side data), headless UI (dialogs, HDR scopes) - docs: architecture §5.2 (pipeline, HDR export, display colour management: no macOS EDR yet) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
a7c1cfd956
commit
1653430c42
@@ -38,6 +38,8 @@ procedural demo footage (`crates/media/src/generators.rs`); no external media. R
|
||||
| `crates/golden/goldens/crop.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `crop`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/gaussian_blur.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `gaussian_blur`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/graphic_shapes.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `graphic_shapes`) from procedurally generated media and bundled OFL fonts | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/log_to_rec709.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `log_to_rec709`) from procedurally generated media re-encoded as S-Log3 | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/hdr_tone_map.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `hdr_tone_map`) from procedurally generated media re-encoded as Rec. 2100 PQ | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/graphic_title.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `graphic_title`) from procedurally generated media and bundled OFL fonts | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/lumetri_basic.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `lumetri_basic`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
| `crates/golden/goldens/text_burnin.png` | FilmCraft contributors | Original work: golden reference rendered by FilmCraft (crates/golden/tests/golden.rs, scene `text_burnin`) from procedurally generated media | MIT OR Apache-2.0 |
|
||||
|
||||
Generated
+3
@@ -1455,6 +1455,7 @@ version = "0.1.0"
|
||||
dependencies = [
|
||||
"filmcraft-aac",
|
||||
"filmcraft-codecs",
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
"filmcraft-h264enc",
|
||||
"filmcraft-isobmff",
|
||||
@@ -1505,6 +1506,8 @@ dependencies = [
|
||||
name = "filmcraft-golden"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"filmcraft-color",
|
||||
"filmcraft-frame",
|
||||
"filmcraft-geom",
|
||||
"filmcraft-gpu",
|
||||
"filmcraft-media",
|
||||
|
||||
@@ -90,6 +90,8 @@ on the sRGB curve).
|
||||
the achromatic axis, after the ACES Reference Gamut Compression: for each channel
|
||||
d = (max − c)/max; distances below 0.8 are unchanged, distances from 0.8 to 1.25 are compressed
|
||||
into 0.8–1.0 with a power-1.2 curve. Colours within 80 % of the boundary are untouched.
|
||||
* **Display-referred wide-gamut input** (BT.2020 / P3 / PQ / HLG media into a narrower working
|
||||
gamut) and
|
||||
* **Output** (BT.2020 working → BT.709 monitor/export): colours with a negative component are
|
||||
desaturated towards their BT.709 luminance just enough to fit (`desaturate_into_gamut`); in-gamut
|
||||
colours, such as BT.709 media in a wide-gamut sequence, come back exactly.
|
||||
|
||||
@@ -195,6 +195,9 @@ pub struct InputTransform {
|
||||
hlg: Option<([f32; 3], f32)>,
|
||||
tone: Option<ToneMap>,
|
||||
compress: bool,
|
||||
/// Display-referred wide-gamut media (BT.2020 / P3 / HDR): desaturate into the working gamut
|
||||
/// with these luma weights instead of the camera-gamut compression.
|
||||
desaturate: Option<[f32; 3]>,
|
||||
}
|
||||
|
||||
impl InputTransform {
|
||||
@@ -215,13 +218,18 @@ impl InputTransform {
|
||||
_ => REFERENCE_WHITE_NITS,
|
||||
};
|
||||
let tone = (src.is_hdr() && !pipe.working.is_hdr() && pipe.auto_tone_map).then(|| ToneMap::new(peak, REFERENCE_WHITE_NITS));
|
||||
let compress = !identity_matrix && !gamut_within(src.gamut(), wg);
|
||||
InputTransform { source: src, table: DecodeTable::for_curve(curve, range), matrix: to_f32(&m), identity_matrix, hlg, tone, compress }
|
||||
let wider = !identity_matrix && !gamut_within(src.gamut(), wg);
|
||||
// camera gamuts (log media) get the soft compression; display gamuts keep their in-gamut
|
||||
// colours exactly (BT.709 content in a BT.2020 file comes back unchanged)
|
||||
let compress = wider && src.is_log();
|
||||
let l = wg.luma();
|
||||
let desaturate = (wider && !src.is_log()).then(|| [l[0] as f32, l[1] as f32, l[2] as f32]);
|
||||
InputTransform { source: src, table: DecodeTable::for_curve(curve, range), matrix: to_f32(&m), identity_matrix, hlg, tone, compress, desaturate }
|
||||
}
|
||||
|
||||
/// Whether anything happens after the per-channel table.
|
||||
pub fn has_pixel_stage(&self) -> bool {
|
||||
!self.identity_matrix || self.hlg.is_some() || self.tone.is_some() || self.compress
|
||||
!self.identity_matrix || self.hlg.is_some() || self.tone.is_some() || self.compress || self.desaturate.is_some()
|
||||
}
|
||||
|
||||
/// The per-pixel stage (input: the table's output).
|
||||
@@ -241,6 +249,9 @@ impl InputTransform {
|
||||
if self.compress {
|
||||
c = gamut_compress(c);
|
||||
}
|
||||
if let Some(l) = self.desaturate {
|
||||
c = desaturate_into_gamut(c, l);
|
||||
}
|
||||
c
|
||||
}
|
||||
|
||||
|
||||
+127
-5
@@ -1,14 +1,15 @@
|
||||
//! Colour commands: the project LUT library (`lut.*`), Lumetri LUT slots and section switches
|
||||
//! (`lumetri.*`).
|
||||
//! Colour commands: colour management (`sequence.colorSettings`, `clip.interpretFootage`,
|
||||
//! `color.spaces`, `media.colorInfo`), the project LUT library (`lut.*`), Lumetri LUT slots and
|
||||
//! section switches (`lumetri.*`).
|
||||
//!
|
||||
//! Lumetri commands address a clip by `clip` (default: the first selected video clip with
|
||||
//! Lumetri Color, else the first selected video clip — Lumetri is added if missing).
|
||||
|
||||
use filmcraft_color::{Lut, LutFormat};
|
||||
use filmcraft_project::{ClipId, ParamValue, ProjectLut, TrackKind};
|
||||
use filmcraft_color::{ColorSpace, Lut, LutFormat, WorkingSpace};
|
||||
use filmcraft_project::{ClipId, ItemId, ItemKind, ParamValue, ProjectLut, TrackKind};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::commands::{CommandSpec, always, bad, bool_p, clip_p, has_seq, str_p};
|
||||
use crate::commands::{CommandSpec, always, bad, bool_p, clip_p, has_seq, item_p, str_p};
|
||||
use crate::{Result, Session};
|
||||
|
||||
type Run = fn(&mut Session, &Value) -> Result<Value>;
|
||||
@@ -23,6 +24,24 @@ fn query(id: &'static str, label: &'static str, params: &'static str, run: Run)
|
||||
|
||||
pub(crate) fn commands() -> Vec<CommandSpec> {
|
||||
vec![
|
||||
spec(
|
||||
"sequence.colorSettings",
|
||||
"Color Management…",
|
||||
&["Sequence"],
|
||||
r#"{"workingSpace":"rec709"|"rec2100-pq"|"rec2100-hlg"?,"wideGamut":bool?,"autoToneMap":bool?}"#,
|
||||
has_seq,
|
||||
color_settings,
|
||||
),
|
||||
spec(
|
||||
"clip.interpretFootage",
|
||||
"Interpret Footage…",
|
||||
&["Clip", "Modify"],
|
||||
r#"{"items":[id]?,"colorSpace":"auto"|"<color space id>"}"#,
|
||||
has_footage,
|
||||
interpret,
|
||||
),
|
||||
query("color.spaces", "List Colour Spaces", "{}", spaces),
|
||||
query("media.colorInfo", "Media Colour Info", r#"{"item":id}"#, color_info),
|
||||
spec("lut.import", "Import LUT…", &[], r#"{"path":str,"name":str?}"#, always, import),
|
||||
query("lut.list", "List LUTs", "{}", list),
|
||||
spec("lut.remove", "Remove LUT", &[], r#"{"id":str}"#, always, remove),
|
||||
@@ -195,3 +214,106 @@ fn set_section(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
set_lumetri_param(s, clip, idx, param, ParamValue::Bool(v), "Lumetri Section")?;
|
||||
Ok(json!({"clip": clip.0, "param": param, "on": v}))
|
||||
}
|
||||
|
||||
fn color_settings(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let id = s.state.active_sequence.ok_or_else(|| bad("sequence.colorSettings", "no active sequence"))?;
|
||||
let ws = match str_p(p, "workingSpace") {
|
||||
Some(w) => Some(WorkingSpace::parse(w).ok_or_else(|| bad("sequence.colorSettings", format!("unknown working space `{w}`")))?),
|
||||
None => None,
|
||||
};
|
||||
let (wide, tone) = (bool_p(p, "wideGamut"), bool_p(p, "autoToneMap"));
|
||||
let cur = s.project.sequence(id).map(|q| q.settings.color).unwrap_or_default();
|
||||
let changed = ws.is_some_and(|w| w != cur.working) || wide.is_some_and(|v| v != cur.wide_gamut) || tone.is_some_and(|v| v != cur.auto_tone_map);
|
||||
if changed {
|
||||
s.edit("Sequence Color Settings", |pr, _| {
|
||||
let q = pr.sequence_mut(id).ok_or_else(|| bad("sequence.colorSettings", "no sequence"))?;
|
||||
let c = &mut q.settings.color;
|
||||
if let Some(w) = ws {
|
||||
c.working = w;
|
||||
}
|
||||
if let Some(v) = wide {
|
||||
c.wide_gamut = v;
|
||||
}
|
||||
if let Some(v) = tone {
|
||||
c.auto_tone_map = v;
|
||||
}
|
||||
q.settings.working_space = c.working.label().into();
|
||||
Ok(())
|
||||
})?;
|
||||
}
|
||||
let c = s.project.sequence(id).map(|q| q.settings.color).unwrap_or_default();
|
||||
Ok(json!({"workingSpace": c.working.id(), "label": c.working.label(), "wideGamut": c.wide_gamut, "autoToneMap": c.auto_tone_map}))
|
||||
}
|
||||
|
||||
fn has_footage(s: &Session) -> std::result::Result<(), String> {
|
||||
if footage_targets(s, &Value::Null).is_empty() { Err("select footage in the Project panel or clips in the timeline".into()) } else { Ok(()) }
|
||||
}
|
||||
|
||||
/// Media items addressed by `items`, else the Project panel selection, else the media of the
|
||||
/// selected timeline clips.
|
||||
pub(crate) fn footage_targets(s: &Session, p: &Value) -> Vec<ItemId> {
|
||||
let media = |i: &ItemId| s.project.item(*i).is_some_and(|it| matches!(it.kind, ItemKind::Media(_)));
|
||||
if let Some(a) = p.get("items").and_then(Value::as_array) {
|
||||
return a.iter().filter_map(Value::as_u64).map(ItemId).filter(media).collect();
|
||||
}
|
||||
let mut v: Vec<ItemId> = s.state.project_selection.iter().copied().filter(media).collect();
|
||||
if v.is_empty()
|
||||
&& let Some(q) = s.active_sequence()
|
||||
{
|
||||
for c in &s.state.selection {
|
||||
if let Some((_, it)) = q.find_item(*c)
|
||||
&& media(&it.item)
|
||||
&& !v.contains(&it.item)
|
||||
{
|
||||
v.push(it.item);
|
||||
}
|
||||
}
|
||||
}
|
||||
v
|
||||
}
|
||||
|
||||
fn interpret(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let items = footage_targets(s, p);
|
||||
if items.is_empty() {
|
||||
return Err(bad("clip.interpretFootage", "no footage selected"));
|
||||
}
|
||||
let cs = match str_p(p, "colorSpace") {
|
||||
None => return Err(bad("clip.interpretFootage", "need `colorSpace` (auto or a colour space id)")),
|
||||
Some("auto") | Some("") => None,
|
||||
Some(c) => Some(ColorSpace::parse(c).ok_or_else(|| bad("clip.interpretFootage", format!("unknown colour space `{c}`")))?),
|
||||
};
|
||||
s.edit("Interpret Footage", |pr, _| {
|
||||
for i in &items {
|
||||
if let Some(ItemKind::Media(m)) = pr.item_mut(*i).map(|it| &mut it.kind) {
|
||||
m.interpret.color_space = cs;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
Ok(json!({"items": items.iter().map(|i| i.0).collect::<Vec<_>>(), "colorSpace": cs.map(|c| c.id()).unwrap_or("auto")}))
|
||||
}
|
||||
|
||||
fn spaces(_: &mut Session, _: &Value) -> Result<Value> {
|
||||
let cs: Vec<Value> = ColorSpace::ALL
|
||||
.iter()
|
||||
.map(|c| json!({"id": c.id(), "label": c.label(), "hdr": c.is_hdr(), "log": c.is_log(), "gamut": format!("{:?}", c.gamut())}))
|
||||
.collect();
|
||||
let ws: Vec<Value> = WorkingSpace::ALL.iter().map(|w| json!({"id": w.id(), "label": w.label(), "hdr": w.is_hdr()})).collect();
|
||||
Ok(json!({"colorSpaces": cs, "workingSpaces": ws}))
|
||||
}
|
||||
|
||||
fn color_info(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let id = item_p(p, "item").ok_or_else(|| bad("media.colorInfo", "need `item`"))?;
|
||||
let Some(ItemKind::Media(m)) = s.project.item(id).map(|i| &i.kind) else { return Err(bad("media.colorInfo", "not a media item")) };
|
||||
let detected = m.info.video.as_ref().map(|v| ColorSpace::from_info(&v.color));
|
||||
let effective = m.interpret.color_space.or(detected);
|
||||
Ok(json!({
|
||||
"detected": detected.map(|c| c.id()),
|
||||
"detectedLabel": detected.map(|c| c.label()),
|
||||
"metadata": m.info.video.as_ref().map(|v| json!({"transfer": format!("{:?}", v.color.transfer), "primaries": format!("{:?}", v.color.primaries), "matrix": format!("{:?}", v.color.matrix), "range": format!("{:?}", v.color.range)})),
|
||||
"override": m.interpret.color_space.map(|c| c.id()),
|
||||
"effective": effective.map(|c| c.id()),
|
||||
"effectiveLabel": effective.map(|c| c.label()),
|
||||
"hdr": effective.is_some_and(|c| c.is_hdr()),
|
||||
}))
|
||||
}
|
||||
|
||||
@@ -14,9 +14,8 @@ fn demo() -> Session {
|
||||
fn pick_clip(s: &mut Session) -> ClipId {
|
||||
let q = s.active_sequence().unwrap();
|
||||
let t = s.playhead();
|
||||
let c = q.video_tracks[0].item_at(t).or(q.video_tracks[0].items.first()).unwrap();
|
||||
let id = c.id;
|
||||
s.set_playhead(c.start + filmcraft_time::Tick(1000));
|
||||
// the demo opens with the playhead inside the first V1 clip (its first frames fade from black)
|
||||
let id = q.video_tracks[0].item_at(t).unwrap().id;
|
||||
s.state.selection = vec![id];
|
||||
id
|
||||
}
|
||||
@@ -32,6 +31,7 @@ fn lumetri_text(s: &Session, clip: ClipId, p: &str) -> String {
|
||||
}
|
||||
|
||||
fn mean(img: &filmcraft_render::Image) -> [f32; 3] {
|
||||
assert!(img.px.iter().any(|v| *v > 0.01), "blank render");
|
||||
let mut m = [0f64; 3];
|
||||
for p in img.px.chunks_exact(4) {
|
||||
for k in 0..3 {
|
||||
@@ -130,3 +130,39 @@ fn set_param_fills_in_parameters_missing_from_old_instances() {
|
||||
.unwrap();
|
||||
s.execute("effects.setParam", json!({"clip": clip.0, "effect": "lumetri", "param": "vignette_on", "value": false})).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn interpret_footage_and_sequence_colour_settings() {
|
||||
let mut s = demo();
|
||||
let clip = pick_clip(&mut s);
|
||||
let item = s.active_sequence().unwrap().find_item(clip).unwrap().1.item;
|
||||
let sdr = mean(&s.render_program(0.125).unwrap());
|
||||
// metadata: the demo footage is SDR
|
||||
let info = s.execute("media.colorInfo", json!({"item": item.0})).unwrap();
|
||||
assert_eq!(info["override"], serde_json::Value::Null);
|
||||
assert_eq!(info["hdr"], false);
|
||||
// interpret it as S-Log3: decoded as log (lifted signal → brighter scene light), tone mapped
|
||||
let r = s.execute("clip.interpretFootage", json!({"colorSpace": "slog3-sgamut3cine"})).unwrap();
|
||||
assert_eq!(r["items"][0], item.0);
|
||||
let info = s.execute("media.colorInfo", json!({"item": item.0})).unwrap();
|
||||
assert_eq!(info["effective"], "slog3-sgamut3cine");
|
||||
assert_eq!(info["hdr"], true);
|
||||
let log = mean(&s.render_program(0.125).unwrap());
|
||||
assert!((log[0] - sdr[0]).abs() + (log[1] - sdr[1]).abs() > 0.02, "{sdr:?} vs {log:?}");
|
||||
assert!(log.iter().all(|v| *v <= 1.0 + 1e-3), "tone mapped into SDR: {log:?}");
|
||||
assert!(s.execute("clip.interpretFootage", json!({"colorSpace": "nope"})).is_err());
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
let back = mean(&s.render_program(0.125).unwrap());
|
||||
assert!((back[0] - sdr[0]).abs() < 1e-4, "undo restores the metadata interpretation");
|
||||
// a PQ sequence: SDR media keeps its values in working space; the monitor shows the same SDR
|
||||
let r = s.execute("sequence.colorSettings", json!({"workingSpace": "Rec. 2100 PQ"})).unwrap();
|
||||
assert_eq!(r["workingSpace"], "rec2100-pq");
|
||||
assert_eq!(s.active_sequence().unwrap().settings.working_space, "Rec. 2100 PQ");
|
||||
let work = mean(&s.render_program_working(0.125).unwrap());
|
||||
let disp = mean(&s.render_program(0.125).unwrap());
|
||||
assert!(work.iter().zip(&sdr).all(|(a, b)| (a - b).abs() < 0.08), "BT.2020 working values ≈ BT.709 SDR values: {work:?} vs {sdr:?}");
|
||||
assert!(disp.iter().zip(&sdr).all(|(a, b)| (a - b).abs() < 0.08), "display transform: {disp:?} vs {sdr:?}");
|
||||
let spaces = s.execute("color.spaces", json!({})).unwrap();
|
||||
assert_eq!(spaces["colorSpaces"].as_array().unwrap().len(), filmcraft_color::ColorSpace::ALL.len());
|
||||
assert!(s.execute("sequence.colorSettings", json!({"workingSpace": "xyz"})).is_err());
|
||||
}
|
||||
|
||||
@@ -171,7 +171,7 @@ fn track_p(s: &Session, p: &Value, k: &str) -> Option<TrackId> {
|
||||
let tracks = if kind.eq_ignore_ascii_case("v") { &seq.video_tracks } else { &seq.audio_tracks };
|
||||
tracks.get(i.checked_sub(1)?).map(|t| t.id)
|
||||
}
|
||||
fn item_p(p: &Value, k: &str) -> Option<ItemId> {
|
||||
pub(crate) fn item_p(p: &Value, k: &str) -> Option<ItemId> {
|
||||
u64_p(p, k).map(ItemId)
|
||||
}
|
||||
|
||||
@@ -1883,6 +1883,8 @@ fn export_media(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
bitrate_kbps: u64_p(p, "bitrateKbps").unwrap_or(20_000) as u32,
|
||||
burn_captions: bool_p(p, "burnCaptions").unwrap_or(false),
|
||||
part_of_batch: false,
|
||||
sdr: bool_p(p, "sdr").unwrap_or(false),
|
||||
..Default::default()
|
||||
};
|
||||
let id = s.jobs.len() as u64 + 1;
|
||||
let job = crate::Job {
|
||||
|
||||
@@ -530,6 +530,15 @@ impl Session {
|
||||
self.media.source_for(&self.project, item, &*self.services)
|
||||
}
|
||||
|
||||
/// Render the active sequence at the playhead in its working colour space (HDR values kept;
|
||||
/// for scopes and analysis).
|
||||
pub fn render_program_working(&self, scale: f32) -> Option<filmcraft_render::Image> {
|
||||
let seq = self.state.active_sequence?;
|
||||
let provider = self.media.provider(self.project.clone(), self.services.clone());
|
||||
let opts = filmcraft_render::RenderOptions { scale, working_output: true, ..Default::default() };
|
||||
Some(filmcraft_render::render_sequence(&self.project, seq, self.playhead(), opts, &provider))
|
||||
}
|
||||
|
||||
/// Render the active sequence at the playhead (CPU reference path).
|
||||
pub fn render_program(&self, scale: f32) -> Option<filmcraft_render::Image> {
|
||||
let seq = self.state.active_sequence?;
|
||||
|
||||
@@ -459,6 +459,9 @@ pub fn render(s: &mut Session, mode: RenderMode, p: &Value) -> Result<Value> {
|
||||
// Captions stay live over previews and are not part of the preview hash.
|
||||
burn_captions: false,
|
||||
part_of_batch: true,
|
||||
// previews stand in for the monitor picture: display-referred SDR
|
||||
sdr: true,
|
||||
..Default::default()
|
||||
};
|
||||
match filmcraft_export::export(&project, seq, &settings, &provider, &prog) {
|
||||
Ok(r) => {
|
||||
|
||||
@@ -12,6 +12,7 @@ workspace = true
|
||||
[dependencies]
|
||||
filmcraft-time = { workspace = true }
|
||||
filmcraft-frame = { workspace = true }
|
||||
filmcraft-color = { workspace = true }
|
||||
filmcraft-media = { workspace = true }
|
||||
filmcraft-project = { workspace = true }
|
||||
filmcraft-render = { workspace = true }
|
||||
|
||||
+199
-16
@@ -101,6 +101,59 @@ pub struct ExportSettings {
|
||||
/// `progress.total`, `finished` and the final status; this call only adds to `done`.
|
||||
#[serde(default)]
|
||||
pub part_of_batch: bool,
|
||||
/// Encode display-referred SDR (Rec. 709, tone mapped) even when the sequence works in
|
||||
/// Rec. 2100 PQ/HLG. Otherwise H.264 and ProRes exports of an HDR sequence are encoded in the
|
||||
/// sequence's HDR space and signal it (VUI / `colr` / `mdcv` / `clli` / SEI).
|
||||
#[serde(default)]
|
||||
pub sdr: bool,
|
||||
/// Colour signalling chosen by [`export`] for the encoders (not set by callers).
|
||||
#[serde(skip)]
|
||||
pub signal: ColorSignal,
|
||||
}
|
||||
|
||||
/// Colour description of the encoded stream (ITU-T H.273 code points).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct ColorSignal {
|
||||
pub primaries: u8,
|
||||
pub transfer: u8,
|
||||
pub matrix: u8,
|
||||
}
|
||||
|
||||
impl Default for ColorSignal {
|
||||
fn default() -> Self {
|
||||
ColorSignal { primaries: 1, transfer: 1, matrix: 1 }
|
||||
}
|
||||
}
|
||||
|
||||
impl ColorSignal {
|
||||
pub const PQ: ColorSignal = ColorSignal { primaries: 9, transfer: 16, matrix: 9 };
|
||||
pub const HLG: ColorSignal = ColorSignal { primaries: 9, transfer: 18, matrix: 9 };
|
||||
pub fn is_hdr(&self) -> bool {
|
||||
matches!(self.transfer, 16 | 18)
|
||||
}
|
||||
/// (Kr, Kb) of the matrix.
|
||||
pub fn kr_kb(&self) -> (f32, f32) {
|
||||
if self.matrix == 9 { (0.2627, 0.0593) } else { (0.2126, 0.0722) }
|
||||
}
|
||||
/// Sample-entry boxes: `colr` (nclx for MP4, nclc for MOV) plus `mdcv`/`clli` for PQ
|
||||
/// (mastering display BT.2020/D65, 1000/0.0001 cd/m²; MaxCLL/MaxFALL 0 = unknown).
|
||||
pub fn apply_to(&self, e: &mut SampleEntry, mov: bool) {
|
||||
if !self.is_hdr() {
|
||||
return;
|
||||
}
|
||||
if let Some(v) = e.video.as_mut() {
|
||||
let (p, t, m) = (self.primaries as u16, self.transfer as u16, self.matrix as u16);
|
||||
v.color = Some(if mov {
|
||||
filmcraft_isobmff::ColorInfo::Nclc { primaries: p, transfer: t, matrix: m }
|
||||
} else {
|
||||
filmcraft_isobmff::ColorInfo::Nclx { primaries: p, transfer: t, matrix: m, full_range: false }
|
||||
});
|
||||
if self.transfer == 16 {
|
||||
v.mastering_display = Some(filmcraft_isobmff::MasteringDisplay::bt2020(1000.0, 0.0001));
|
||||
v.content_light = Some((0, 0));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for ExportSettings {
|
||||
@@ -115,6 +168,8 @@ impl Default for ExportSettings {
|
||||
bitrate_kbps: 20_000,
|
||||
burn_captions: false,
|
||||
part_of_batch: false,
|
||||
sdr: false,
|
||||
signal: ColorSignal::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -159,11 +214,13 @@ pub struct EncodedPacket {
|
||||
pub composition_offset: i32,
|
||||
}
|
||||
|
||||
/// Input picture for encoders: straight sRGB RGBA8 (encoders convert to their own YUV).
|
||||
/// Input picture for encoders: straight sRGB RGBA8 (encoders convert to their own YUV), or for
|
||||
/// HDR exports the encoded R'G'B' (PQ/HLG, BT.2020; 3 floats per pixel, 0..1) in `hdr`.
|
||||
pub struct EncoderFrame<'a> {
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub rgba: &'a [u8],
|
||||
pub hdr: Option<&'a [f32]>,
|
||||
pub index: u64,
|
||||
}
|
||||
|
||||
@@ -285,18 +342,27 @@ struct ProResEncoder {
|
||||
w: u32,
|
||||
h: u32,
|
||||
rate: FrameRate,
|
||||
signal: ColorSignal,
|
||||
}
|
||||
|
||||
impl VideoEncoder for ProResEncoder {
|
||||
fn sample_entry(&self) -> SampleEntry {
|
||||
SampleEntry::prores(filmcraft_isobmff::FourCc(*b"apch"), self.w as u16, self.h as u16)
|
||||
let mut e = SampleEntry::prores(filmcraft_isobmff::FourCc(*b"apch"), self.w as u16, self.h as u16);
|
||||
self.signal.apply_to(&mut e, true);
|
||||
e
|
||||
}
|
||||
fn timescale(&self) -> u32 {
|
||||
self.rate.num as u32
|
||||
}
|
||||
fn encode(&mut self, f: &EncoderFrame) -> Result<Vec<EncodedPacket>> {
|
||||
let mut fr = filmcraft_prores::Frame::new(f.width, f.height, filmcraft_prores::ChromaFormat::Yuv422, 10, false);
|
||||
rgba_to_yuv422_10(f.rgba, f.width as usize, f.height as usize, &mut fr.y, &mut fr.cb, &mut fr.cr);
|
||||
match f.hdr {
|
||||
Some(rgb) => {
|
||||
let (kr, kb) = self.signal.kr_kb();
|
||||
rgbf_to_yuv422_10(rgb, f.width as usize, f.height as usize, kr, kb, &mut fr.y, &mut fr.cb, &mut fr.cr)
|
||||
}
|
||||
None => rgba_to_yuv422_10(f.rgba, f.width as usize, f.height as usize, &mut fr.y, &mut fr.cb, &mut fr.cr),
|
||||
}
|
||||
let data = self.enc.encode(&fr).map_err(|e| ExportError::Encode(e.to_string()))?;
|
||||
Ok(vec![EncodedPacket { data, key: true, duration: self.rate.den as u32, composition_offset: 0 }])
|
||||
}
|
||||
@@ -331,9 +397,41 @@ pub fn rgba_to_yuv422_10(rgba: &[u8], w: usize, h: usize, y: &mut [u16], cb: &mu
|
||||
});
|
||||
}
|
||||
|
||||
fn prores_factory(format: Format, w: u32, h: u32, rate: FrameRate, _s: &ExportSettings) -> Option<Result<Box<dyn VideoEncoder>>> {
|
||||
(format == Format::ProRes)
|
||||
.then(|| Ok(Box::new(ProResEncoder { enc: filmcraft_prores::Encoder::new(filmcraft_prores::Profile::Hq, w, h), w, h, rate }) as Box<dyn VideoEncoder>))
|
||||
/// Limited-range 10-bit 4:2:2 from encoded R'G'B' floats with matrix (Kr, Kb).
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn rgbf_to_yuv422_10(rgb: &[f32], w: usize, h: usize, kr: f32, kb: f32, y: &mut [u16], cb: &mut [u16], cr: &mut [u16]) {
|
||||
let cw = w.div_ceil(2);
|
||||
debug_assert!(rgb.len() >= w * h * 3 && y.len() >= w * h);
|
||||
let kg = 1.0 - kr - kb;
|
||||
let (sb, sr) = (2.0 * (1.0 - kb), 2.0 * (1.0 - kr));
|
||||
y.par_chunks_mut(w).zip(cb.par_chunks_mut(cw).zip(cr.par_chunks_mut(cw))).enumerate().for_each(|(row, (yr, (cbr, crr)))| {
|
||||
let src = &rgb[row * w * 3..(row + 1) * w * 3];
|
||||
let mut us = vec![0f32; w];
|
||||
let mut vs = vec![0f32; w];
|
||||
for x in 0..w {
|
||||
let (r, g, b) = (src[x * 3], src[x * 3 + 1], src[x * 3 + 2]);
|
||||
let yy = kr * r + kg * g + kb * b;
|
||||
yr[x] = (64.0 + 876.0 * yy).round().clamp(4.0, 1019.0) as u16;
|
||||
us[x] = (b - yy) / sb;
|
||||
vs[x] = (r - yy) / sr;
|
||||
}
|
||||
for cx in 0..cw {
|
||||
let a = cx * 2;
|
||||
let b2 = (a + 1).min(w - 1);
|
||||
cbr[cx] = (512.0 + 896.0 * (us[a] + us[b2]) * 0.5).round().clamp(4.0, 1019.0) as u16;
|
||||
crr[cx] = (512.0 + 896.0 * (vs[a] + vs[b2]) * 0.5).round().clamp(4.0, 1019.0) as u16;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
fn prores_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Option<Result<Box<dyn VideoEncoder>>> {
|
||||
(format == Format::ProRes).then(|| {
|
||||
let mut cfg = filmcraft_prores::EncoderConfig::new(filmcraft_prores::Profile::Hq, w, h);
|
||||
if s.signal.is_hdr() {
|
||||
cfg.color = filmcraft_prores::ColorInfo { primaries: s.signal.primaries, transfer: s.signal.transfer, matrix: s.signal.matrix };
|
||||
}
|
||||
Ok(Box::new(ProResEncoder { enc: filmcraft_prores::Encoder::with_config(cfg), w, h, rate, signal: s.signal }) as Box<dyn VideoEncoder>)
|
||||
})
|
||||
}
|
||||
|
||||
/// H.264 High (our encoder): sRGB/709 RGBA8 → 8-bit limited-range BT.709 4:2:0, VBR at the
|
||||
@@ -346,6 +444,7 @@ struct H264Encoder {
|
||||
y: Vec<u8>,
|
||||
u: Vec<u8>,
|
||||
v: Vec<u8>,
|
||||
signal: ColorSignal,
|
||||
}
|
||||
|
||||
impl H264Encoder {
|
||||
@@ -362,13 +461,21 @@ impl VideoEncoder for H264Encoder {
|
||||
let (sps, pps) = self.enc.sps_pps();
|
||||
filmcraft_isobmff::AvcConfig::new(vec![sps], vec![pps], 4)
|
||||
});
|
||||
SampleEntry::avc(cfg, self.w as u16, self.h as u16)
|
||||
let mut e = SampleEntry::avc(cfg, self.w as u16, self.h as u16);
|
||||
self.signal.apply_to(&mut e, false);
|
||||
e
|
||||
}
|
||||
fn timescale(&self) -> u32 {
|
||||
self.rate.num as u32
|
||||
}
|
||||
fn encode(&mut self, f: &EncoderFrame) -> Result<Vec<EncodedPacket>> {
|
||||
rgba_to_yuv420_8(f.rgba, f.width as usize, f.height as usize, &mut self.y, &mut self.u, &mut self.v);
|
||||
match f.hdr {
|
||||
Some(rgb) => {
|
||||
let (kr, kb) = self.signal.kr_kb();
|
||||
rgbf_to_yuv420_8(rgb, f.width as usize, f.height as usize, kr, kb, &mut self.y, &mut self.u, &mut self.v)
|
||||
}
|
||||
None => rgba_to_yuv420_8(f.rgba, f.width as usize, f.height as usize, &mut self.y, &mut self.u, &mut self.v),
|
||||
}
|
||||
let cw = (f.width as usize).div_ceil(2);
|
||||
let frame = filmcraft_h264enc::YuvFrame { y: &self.y, u: &self.u, v: &self.v, y_stride: f.width as usize, uv_stride: cw };
|
||||
let ps = self.enc.try_encode(&frame, f.index as i64 * self.rate.den).map_err(|e| ExportError::Encode(e.to_string()))?;
|
||||
@@ -414,6 +521,39 @@ pub fn rgba_to_yuv420_8(rgba: &[u8], w: usize, h: usize, y: &mut Vec<u8>, u: &mu
|
||||
});
|
||||
}
|
||||
|
||||
/// Limited-range 8-bit 4:2:0 from encoded R'G'B' floats with matrix (Kr, Kb).
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn rgbf_to_yuv420_8(rgb: &[f32], w: usize, h: usize, kr: f32, kb: f32, y: &mut Vec<u8>, u: &mut Vec<u8>, v: &mut Vec<u8>) {
|
||||
let (cw, ch) = (w.div_ceil(2), h.div_ceil(2));
|
||||
let kg = 1.0 - kr - kb;
|
||||
let (sb, sr) = (2.0 * (1.0 - kb), 2.0 * (1.0 - kr));
|
||||
y.resize(w * h, 0);
|
||||
u.resize(cw * ch, 0);
|
||||
v.resize(cw * ch, 0);
|
||||
y.par_chunks_mut(w * 2).zip(u.par_chunks_mut(cw).zip(v.par_chunks_mut(cw))).enumerate().for_each(|(cy, (yr, (ur, vr)))| {
|
||||
let rows = yr.len() / w;
|
||||
let mut us = vec![0f32; cw];
|
||||
let mut vs = vec![0f32; cw];
|
||||
let mut cnt = vec![0f32; cw];
|
||||
for dy in 0..rows {
|
||||
let row = cy * 2 + dy;
|
||||
let src = &rgb[row * w * 3..(row + 1) * w * 3];
|
||||
for x in 0..w {
|
||||
let (r, g, b) = (src[x * 3], src[x * 3 + 1], src[x * 3 + 2]);
|
||||
let yy = kr * r + kg * g + kb * b;
|
||||
yr[dy * w + x] = (16.0 + 219.0 * yy).round().clamp(1.0, 254.0) as u8;
|
||||
us[x / 2] += (b - yy) / sb;
|
||||
vs[x / 2] += (r - yy) / sr;
|
||||
cnt[x / 2] += 1.0;
|
||||
}
|
||||
}
|
||||
for cx in 0..cw {
|
||||
ur[cx] = (128.0 + 224.0 * us[cx] / cnt[cx]).round().clamp(1.0, 254.0) as u8;
|
||||
vr[cx] = (128.0 + 224.0 * vs[cx] / cnt[cx]).round().clamp(1.0, 254.0) as u8;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
fn h264_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSettings) -> Option<Result<Box<dyn VideoEncoder>>> {
|
||||
if format != Format::H264 {
|
||||
return None;
|
||||
@@ -424,9 +564,19 @@ fn h264_factory(format: Format, w: u32, h: u32, rate: FrameRate, s: &ExportSetti
|
||||
cfg.keyint = (rate.num as f64 / rate.den as f64 * 2.0).round().max(1.0) as u32;
|
||||
let kbps = s.bitrate_kbps.max(100);
|
||||
cfg.rate = filmcraft_h264enc::RateControl::Vbr { target_kbps: kbps, max_kbps: kbps * 3 / 2 };
|
||||
if s.signal.is_hdr() {
|
||||
cfg.color = filmcraft_h264enc::ColorConfig { primaries: s.signal.primaries, transfer: s.signal.transfer, matrix: s.signal.matrix, full_range: false };
|
||||
if s.signal.transfer == 16 {
|
||||
let md = filmcraft_isobmff::MasteringDisplay::bt2020(1000.0, 0.0001).to_bytes();
|
||||
let mut b = [0u8; 24];
|
||||
b.copy_from_slice(&md);
|
||||
cfg.mastering_display = Some(b);
|
||||
cfg.content_light = Some((0, 0));
|
||||
}
|
||||
}
|
||||
Some(
|
||||
filmcraft_h264enc::Encoder::new(cfg)
|
||||
.map(|enc| Box::new(H264Encoder { enc, w, h, rate, y: Vec::new(), u: Vec::new(), v: Vec::new() }) as Box<dyn VideoEncoder>)
|
||||
.map(|enc| Box::new(H264Encoder { enc, w, h, rate, y: Vec::new(), u: Vec::new(), v: Vec::new(), signal: s.signal }) as Box<dyn VideoEncoder>)
|
||||
.map_err(|e| ExportError::Encode(e.to_string())),
|
||||
)
|
||||
}
|
||||
@@ -447,6 +597,17 @@ pub fn export_range(project: &Project, seq: ItemId, settings: &ExportSettings) -
|
||||
pub fn export(project: &Arc<Project>, seq: ItemId, settings: &ExportSettings, sources: &dyn SourceProvider, progress: &Progress) -> Result<Report> {
|
||||
let t0 = std::time::Instant::now();
|
||||
let q = project.sequence(seq).ok_or(ExportError::NoSequence)?;
|
||||
// HDR sequences export HDR (H.264 / ProRes) unless SDR is asked for
|
||||
let pipe = q.settings.color;
|
||||
let hdr_out = pipe.working.is_hdr() && !settings.sdr && matches!(settings.format, Format::H264 | Format::ProRes);
|
||||
let mut settings = settings.clone();
|
||||
settings.signal = match (hdr_out, pipe.working) {
|
||||
(true, filmcraft_color::WorkingSpace::Rec2100Pq) => ColorSignal::PQ,
|
||||
(true, _) => ColorSignal::HLG,
|
||||
_ => ColorSignal::default(),
|
||||
};
|
||||
let settings = &settings;
|
||||
let out_tf = hdr_out.then(|| filmcraft_color::OutputTransform::new(&pipe, pipe.working.output_space()));
|
||||
let rate = q.settings.frame_rate;
|
||||
let range = export_range(project, seq, settings)?;
|
||||
let f0 = rate.frame_at(range.start);
|
||||
@@ -458,7 +619,26 @@ pub fn export(project: &Arc<Project>, seq: ItemId, settings: &ExportSettings, so
|
||||
progress.total.store(if settings.format == Format::Wav { 1 } else { nframes }, Ordering::Relaxed);
|
||||
progress.set_status(format!("Exporting {} frames ({})", nframes, settings.format.label()));
|
||||
}
|
||||
let opts = RenderOptions { scale: w as f32 / q.settings.width as f32, captions: settings.burn_captions, ..Default::default() };
|
||||
let opts = RenderOptions { scale: w as f32 / q.settings.width as f32, captions: settings.burn_captions, working_output: hdr_out, ..Default::default() };
|
||||
// HDR: encoded R'G'B' floats over black, padded/cropped to the even output size
|
||||
let render_hdr = |f: i64| -> Vec<f32> {
|
||||
let img = filmcraft_render::render_sequence(project, seq, rate.tick_of(f), opts, sources);
|
||||
let tf = out_tf.as_ref().expect("hdr transform");
|
||||
let mut out = vec![0f32; (w * h * 3) as usize];
|
||||
let black = tf.encode([0.0; 3]);
|
||||
for (y, row) in out.chunks_exact_mut(w as usize * 3).enumerate() {
|
||||
for (x, o) in row.chunks_exact_mut(3).enumerate() {
|
||||
let c = if x < img.w && y < img.h {
|
||||
let i = (y * img.w + x) * 4;
|
||||
tf.encode([img.px[i], img.px[i + 1], img.px[i + 2]])
|
||||
} else {
|
||||
black
|
||||
};
|
||||
o.copy_from_slice(&c);
|
||||
}
|
||||
}
|
||||
out
|
||||
};
|
||||
let render = |f: i64| -> Vec<u8> {
|
||||
let img = filmcraft_render::render_sequence(project, seq, rate.tick_of(f), opts, sources);
|
||||
let mut rgba = img.over_black_rgba8();
|
||||
@@ -544,9 +724,10 @@ pub fn export(project: &Arc<Project>, seq: ItemId, settings: &ExportSettings, so
|
||||
// Encode the first batch before creating tracks (encoders may finalise codec config then).
|
||||
let mut pending: Vec<EncodedPacket> = Vec::new();
|
||||
let first_end = (f0 + batch).min(f1);
|
||||
let first: Vec<Vec<u8>> = (f0..first_end).into_par_iter().map(render).collect();
|
||||
for (k, rgba) in first.iter().enumerate() {
|
||||
pending.extend(venc.encode(&EncoderFrame { width: w, height: h, rgba, index: k as u64 })?);
|
||||
let frame = |fi: i64| -> (Vec<u8>, Vec<f32>) { if hdr_out { (Vec::new(), render_hdr(fi)) } else { (render(fi), Vec::new()) } };
|
||||
let first: Vec<(Vec<u8>, Vec<f32>)> = (f0..first_end).into_par_iter().map(frame).collect();
|
||||
for (k, (rgba, hdr)) in first.iter().enumerate() {
|
||||
pending.extend(venc.encode(&EncoderFrame { width: w, height: h, rgba, hdr: hdr_out.then_some(hdr.as_slice()), index: k as u64 })?);
|
||||
}
|
||||
progress.done.fetch_add((first_end - f0) as u64, Ordering::Relaxed);
|
||||
let mut mux = Mp4Writer::new(std::io::BufWriter::new(file), WriterOptions::new(brand)).map_err(|e| ExportError::Io(e.to_string()))?;
|
||||
@@ -614,9 +795,11 @@ pub fn export(project: &Arc<Project>, seq: ItemId, settings: &ExportSettings, so
|
||||
return Err(ExportError::Cancelled);
|
||||
}
|
||||
let end = (f + batch).min(f1);
|
||||
let frames: Vec<Vec<u8>> = (f..end).into_par_iter().map(render).collect();
|
||||
for (k, rgba) in frames.iter().enumerate() {
|
||||
for p in venc.encode(&EncoderFrame { width: w, height: h, rgba, index: (f - f0) as u64 + k as u64 })? {
|
||||
let frames: Vec<(Vec<u8>, Vec<f32>)> = (f..end).into_par_iter().map(frame).collect();
|
||||
for (k, (rgba, hdr)) in frames.iter().enumerate() {
|
||||
for p in
|
||||
venc.encode(&EncoderFrame { width: w, height: h, rgba, hdr: hdr_out.then_some(hdr.as_slice()), index: (f - f0) as u64 + k as u64 })?
|
||||
{
|
||||
mux.write_sample(vt, WriteSample { data: &p.data, duration: p.duration, composition_offset: p.composition_offset, is_sync: p.key })
|
||||
.map_err(|e| ExportError::Io(e.to_string()))?;
|
||||
}
|
||||
|
||||
@@ -175,3 +175,62 @@ fn caption_burn_in_h264_ffmpeg_oracle() {
|
||||
let (left, right) = (bright(&with, h * 2 / 3, h, 0, w / 2), bright(&with, h * 2 / 3, h, w / 2, w));
|
||||
assert!(left > 100 && right > 100, "{left} / {right}");
|
||||
}
|
||||
|
||||
/// HDR sequences export PQ / HLG with colour signalling in the bitstream (VUI, ProRes header) and
|
||||
/// the container (`colr`, `mdcv`, `clli`); our demuxer and ffprobe both read it back, and the
|
||||
/// picture survives the round trip.
|
||||
#[test]
|
||||
fn hdr_exports_signal_pq_and_hlg() {
|
||||
use filmcraft_color::{ColorPipeline, Transfer, WorkingSpace};
|
||||
for (ws, fmt, ext, transfer, ff_transfer) in [
|
||||
(WorkingSpace::Rec2100Pq, Format::ProRes, "mov", Transfer::Pq, "smpte2084"),
|
||||
(WorkingSpace::Rec2100Pq, Format::H264, "mp4", Transfer::Pq, "smpte2084"),
|
||||
(WorkingSpace::Rec2100Hlg, Format::ProRes, "mov", Transfer::Hlg, "arib-std-b67"),
|
||||
(WorkingSpace::Rec2100Hlg, Format::H264, "mp4", Transfer::Hlg, "arib-std-b67"),
|
||||
] {
|
||||
let (p, seq, m) = project();
|
||||
let mut p = (*p).clone();
|
||||
p.sequence_mut(seq).unwrap().settings.color = ColorPipeline { working: ws, ..ColorPipeline::REC709 };
|
||||
let p = Arc::new(p);
|
||||
let path = tmp(&format!("hdr-{}.{ext}", ws.id()));
|
||||
let s = ExportSettings { format: fmt, path: path.clone(), include_audio: false, ..Default::default() };
|
||||
export(&p, seq, &s, &m, &Progress::default()).unwrap();
|
||||
// our demuxer/decoder sees the signalling
|
||||
let bytes: Arc<[u8]> = std::fs::read(&path).unwrap().into();
|
||||
let src = filmcraft_codecs::open_bytes(&path, bytes).unwrap();
|
||||
let v = src.info().video.clone().unwrap();
|
||||
assert_eq!(v.color.transfer, transfer, "{path}");
|
||||
assert_eq!(v.color.primaries, filmcraft_color::Primaries::Bt2020, "{path}");
|
||||
assert_eq!(v.color.matrix, filmcraft_color::Matrix::Bt2020Ncl, "{path}");
|
||||
// 709 red matte → BT.2020 HDR → decoded back into Rec. 709 (tone mapped) stays red
|
||||
let f = src.video_frame(FrameRequest::full(Tick::ZERO)).unwrap();
|
||||
let back = filmcraft_render::colorman::decode(&Project::new("x"), ItemId(0), &f, 1, &ColorPipeline::REC709);
|
||||
let c = filmcraft_render::Image::unpremul(back.get(160, 90));
|
||||
assert!(c[0] > 0.6 && c[1] < 0.05 && c[2] < 0.05, "{path}: {c:?}");
|
||||
// ffprobe agrees
|
||||
let Some(ffprobe) = filmcraft_testkit::ffprobe_or_skip("hdr export signalling") else { continue };
|
||||
let out = std::process::Command::new(&ffprobe)
|
||||
.args([
|
||||
"-v",
|
||||
"error",
|
||||
"-select_streams",
|
||||
"v:0",
|
||||
"-show_entries",
|
||||
"stream=color_transfer,color_primaries,color_space:stream_side_data",
|
||||
"-of",
|
||||
"default=nw=1",
|
||||
&path,
|
||||
])
|
||||
.output()
|
||||
.unwrap();
|
||||
let txt = String::from_utf8_lossy(&out.stdout).to_string();
|
||||
assert!(txt.contains(&format!("color_transfer={ff_transfer}")), "{path}: {txt}");
|
||||
assert!(txt.contains("color_primaries=bt2020"), "{path}: {txt}");
|
||||
assert!(txt.contains("color_space=bt2020nc"), "{path}: {txt}");
|
||||
if transfer == Transfer::Pq {
|
||||
assert!(txt.contains("Mastering display metadata") && txt.contains("max_luminance=10000000/10000"), "{path}: {txt}");
|
||||
assert!(txt.contains("Content light level metadata"), "{path}: {txt}");
|
||||
}
|
||||
eprintln!("{path}:\n{txt}");
|
||||
}
|
||||
}
|
||||
|
||||
+21
-4
@@ -7,7 +7,7 @@
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use filmcraft_color::{ColorInfo, Matrix, Range, linear_to_srgb_u8, normalize_c, normalize_y, srgb_u8_to_linear_table, to_linear, ycbcr_to_rgb};
|
||||
use filmcraft_color::{ColorInfo, DecodeTable, Matrix, Range, linear_to_srgb_u8, normalize_c, normalize_y, srgb_u8_to_linear_table, to_linear, ycbcr_to_rgb};
|
||||
use filmcraft_time::Tick;
|
||||
use rayon::prelude::*;
|
||||
use serde::{Deserialize, Serialize};
|
||||
@@ -107,6 +107,13 @@ impl VideoFrame {
|
||||
/// in linear light. Reduced-resolution playback uses this so it never builds full-size float
|
||||
/// buffers. Returns (width, height, pixels).
|
||||
pub fn to_linear_f32_decimated(&self, n: usize) -> (usize, usize, Vec<f32>) {
|
||||
self.to_linear_f32_decimated_with(n, None)
|
||||
}
|
||||
|
||||
/// Like [`VideoFrame::to_linear_f32_decimated`], decoding each channel's signal through
|
||||
/// `decode` (a colour-managed curve: log, PQ, HLG scene light…) instead of the frame's
|
||||
/// transfer. Float frames are already linear and ignore it.
|
||||
pub fn to_linear_f32_decimated_with(&self, n: usize, decode: Option<&DecodeTable>) -> (usize, usize, Vec<f32>) {
|
||||
let n = n.max(1);
|
||||
let (w, h) = (self.width as usize, self.height as usize);
|
||||
let (ow, oh) = ((w / n).max(1), (h / n).max(1));
|
||||
@@ -119,8 +126,11 @@ impl VideoFrame {
|
||||
}
|
||||
// Encoded (0..1, quantised to 12 bits) → linear lookup for this frame's transfer.
|
||||
let info = self.color;
|
||||
let lin: Vec<f32> = (0..4096).map(|i| to_linear(i as f32 / 4095.0, info.transfer)).collect();
|
||||
let q = |v: f32| lin[(v.clamp(0.0, 1.0) * 4095.0 + 0.5) as usize];
|
||||
let lin: Vec<f32> = if decode.is_some() { Vec::new() } else { (0..4096).map(|i| to_linear(i as f32 / 4095.0, info.transfer)).collect() };
|
||||
let q = |v: f32| match decode {
|
||||
Some(t) => t.lookup(v),
|
||||
None => lin[(v.clamp(0.0, 1.0) * 4095.0 + 0.5) as usize],
|
||||
};
|
||||
let inv = 1.0 / (n * n) as f32;
|
||||
match &self.data {
|
||||
PixelData::RgbaF32(d) => {
|
||||
@@ -144,7 +154,14 @@ impl VideoFrame {
|
||||
});
|
||||
}
|
||||
PixelData::Rgba8(d) => {
|
||||
let lut = srgb_u8_to_linear_table();
|
||||
let table: Vec<f32>;
|
||||
let lut: &[f32] = match decode {
|
||||
Some(t) => {
|
||||
table = (0..256).map(|i| t.lookup(i as f32 / 255.0)).collect();
|
||||
&table
|
||||
}
|
||||
None => srgb_u8_to_linear_table(),
|
||||
};
|
||||
out.par_chunks_mut(ow * 4).enumerate().for_each(|(oy, row)| {
|
||||
for ox in 0..ow {
|
||||
let mut acc = [0f32; 4];
|
||||
|
||||
@@ -16,6 +16,8 @@ workspace = true
|
||||
|
||||
[dev-dependencies]
|
||||
filmcraft-testkit = { workspace = true }
|
||||
filmcraft-color = { workspace = true }
|
||||
filmcraft-frame = { workspace = true }
|
||||
filmcraft-time = { workspace = true }
|
||||
filmcraft-geom = { workspace = true }
|
||||
filmcraft-media = { workspace = true }
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 4.6 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: hdr_tone_map.png
|
||||
title: Colour management: Rec. 2100 PQ footage tone mapped into a Rec. 709 sequence
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (hdr_tone_map). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
@@ -0,0 +1,8 @@
|
||||
asset: log_to_rec709.png
|
||||
title: Colour management: S-Log3/S-Gamut3.Cine footage to Rec. 709 (tone mapped)
|
||||
author: FilmCraft contributors
|
||||
source: original work (rendered by FilmCraft's own renderer from a procedurally generated project; no external media)
|
||||
license: MIT OR Apache-2.0
|
||||
license-file:
|
||||
added: 2026-10-01 by FilmCraft contributors (FILMCRAFT_BLESS=1)
|
||||
notes: Golden reference image generated by crates/golden/tests/golden.rs (log_to_rec709). Regenerate with FILMCRAFT_BLESS=1. Contains no Adobe artwork.
|
||||
@@ -10,6 +10,7 @@
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
|
||||
use filmcraft_color::{ColorInfo, ColorSpace, Gamut, Primaries, Transfer};
|
||||
use filmcraft_geom::Vec2;
|
||||
use filmcraft_media::generators::GeneratorSource;
|
||||
use filmcraft_media::{DemoScene, Generator, MediaSource, SharedSource};
|
||||
@@ -74,6 +75,18 @@ impl Builder {
|
||||
self.media(Generator::Demo(scene))
|
||||
}
|
||||
|
||||
/// Demo footage re-encoded as camera log / HDR (see [`Encoded`]), with an optional Interpret
|
||||
/// Footage colour-space override.
|
||||
fn encoded(&mut self, scene: DemoScene, target: ColorSpace, gain: f32, flag: Option<ColorInfo>, interpret: Option<ColorSpace>) -> ItemId {
|
||||
let id = self.demo(scene);
|
||||
let inner = self.map.0.remove(&id).unwrap();
|
||||
self.map.0.insert(id, Arc::new(Encoded { inner, target, gain, flag }) as SharedSource);
|
||||
if let ItemKind::Media(m) = &mut self.p.item_mut(id).unwrap().kind {
|
||||
m.interpret.color_space = interpret;
|
||||
}
|
||||
id
|
||||
}
|
||||
|
||||
/// Place `item` on video track `track` (frames), scaled to the frame size.
|
||||
fn place(&mut self, track: usize, item: ItemId, start: i64, dur: i64) -> ClipId {
|
||||
let mut ti = self.p.make_track_item(item, TrackKind::Video, RATE.tick_of(start), TimeRange::new(Tick::ZERO, RATE.tick_of(dur)), RATE).unwrap();
|
||||
@@ -143,6 +156,46 @@ fn blend(name: &str) -> ParamValue {
|
||||
ParamValue::Choice(filmcraft_project::effect::BLEND_MODES.iter().position(|b| *b == name).unwrap_or_else(|| panic!("blend {name}")) as u32)
|
||||
}
|
||||
|
||||
/// A source that re-encodes another source's (sRGB) frames into a camera log or HDR colour space:
|
||||
/// linear BT.709 × `gain` → `target` gamut → `target` curve, stored as full-range RGBA8. `flag`
|
||||
/// replaces the frame's colour metadata (e.g. PQ/BT.2020, so auto-detection kicks in).
|
||||
struct Encoded {
|
||||
inner: SharedSource,
|
||||
target: ColorSpace,
|
||||
gain: f32,
|
||||
flag: Option<ColorInfo>,
|
||||
}
|
||||
|
||||
impl MediaSource for Encoded {
|
||||
fn info(&self) -> &filmcraft_media::MediaInfo {
|
||||
self.inner.info()
|
||||
}
|
||||
fn video_frame(&self, req: filmcraft_media::FrameRequest) -> filmcraft_media::Result<Arc<filmcraft_frame::VideoFrame>> {
|
||||
let f = self.inner.video_frame(req)?;
|
||||
let rgba = f.to_rgba8();
|
||||
let m = filmcraft_color::spaces::to_f32(&filmcraft_color::spaces::gamut_matrix(Gamut::Bt709, self.target.gamut()));
|
||||
let curve = self.target.curve();
|
||||
let out: Vec<u8> = rgba
|
||||
.chunks_exact(4)
|
||||
.flat_map(|p| {
|
||||
let lin = [0, 1, 2].map(|k| filmcraft_color::srgb_to_linear(p[k] as f32 / 255.0) * self.gain);
|
||||
let c = filmcraft_color::spaces::apply3(&m, lin);
|
||||
let e = c
|
||||
.map(|v| (filmcraft_color::transform::encode_channel(v as f64, curve, filmcraft_color::Range::Full).clamp(0.0, 1.0) * 255.0).round() as u8);
|
||||
[e[0], e[1], e[2], p[3]]
|
||||
})
|
||||
.collect();
|
||||
let mut nf = filmcraft_frame::VideoFrame::rgba8(f.width, f.height, out);
|
||||
if let Some(c) = self.flag {
|
||||
nf.color = c;
|
||||
}
|
||||
Ok(Arc::new(nf))
|
||||
}
|
||||
fn audio(&self, start: i64, frames: usize, sample_rate: u32) -> filmcraft_media::Result<filmcraft_frame::AudioBuffer> {
|
||||
self.inner.audio(start, frames, sample_rate)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- scenes ----
|
||||
|
||||
/// Motion (scale, rotation, position) and 70 % opacity over a full-frame background.
|
||||
@@ -327,6 +380,25 @@ fn graphic_shapes() -> Scene {
|
||||
graphic_scene(DemoScene::Aurora, vec![ell, poly, path, txt])
|
||||
}
|
||||
|
||||
/// Camera log → Rec. 709: Ocean Sunset encoded as S-Log3 / S-Gamut3.Cine one stop over, then
|
||||
/// interpreted as such in a Rec. 709 sequence (decode, gamut compression, BT.2390 tone mapping).
|
||||
fn log_to_rec709() -> Scene {
|
||||
let mut b = Builder::new(1);
|
||||
let src = b.encoded(DemoScene::OceanSunset, ColorSpace::SLog3SGamut3Cine, 2.0, None, Some(ColorSpace::SLog3SGamut3Cine));
|
||||
b.place(0, src, 0, 48);
|
||||
b.at(12)
|
||||
}
|
||||
|
||||
/// HDR → SDR: City Night as Rec. 2100 PQ with highlights up to ~1200 cd/m² (×6 over reference
|
||||
/// white), detected from the frame's metadata and tone mapped into a Rec. 709 sequence.
|
||||
fn hdr_tone_map() -> Scene {
|
||||
let mut b = Builder::new(1);
|
||||
let pq = ColorInfo { transfer: Transfer::Pq, primaries: Primaries::Bt2020, ..ColorInfo::SRGB_FULL };
|
||||
let src = b.encoded(DemoScene::CityNight, ColorSpace::Rec2100Pq, 6.0, Some(pq), None);
|
||||
b.place(0, src, 0, 48);
|
||||
b.at(12)
|
||||
}
|
||||
|
||||
/// (name, title, scene).
|
||||
fn scenes() -> Vec<(&'static str, &'static str, fn() -> Scene)> {
|
||||
vec![
|
||||
@@ -341,6 +413,8 @@ fn scenes() -> Vec<(&'static str, &'static str, fn() -> Scene)> {
|
||||
("text_burnin", "Timecode and Clip Name burn-in text", text_burnin),
|
||||
("graphic_title", "Graphic clip: title text with stroke and shadow, lower-third bar", graphic_title),
|
||||
("graphic_shapes", "Graphic clip: ellipse, polygon, path and rotated text with background", graphic_shapes),
|
||||
("log_to_rec709", "Colour management: S-Log3/S-Gamut3.Cine footage to Rec. 709 (tone mapped)", log_to_rec709),
|
||||
("hdr_tone_map", "Colour management: Rec. 2100 PQ footage tone mapped into a Rec. 709 sequence", hdr_tone_map),
|
||||
]
|
||||
}
|
||||
|
||||
@@ -393,7 +467,9 @@ goldens!(
|
||||
transition_wipe,
|
||||
text_burnin,
|
||||
graphic_title,
|
||||
graphic_shapes
|
||||
graphic_shapes,
|
||||
log_to_rec709,
|
||||
hdr_tone_map
|
||||
);
|
||||
|
||||
/// Sanity: the scenes are not trivially empty or identical to each other.
|
||||
|
||||
@@ -156,6 +156,11 @@ pub struct EncoderConfig {
|
||||
pub slices: usize,
|
||||
/// Access unit delimiters in Annex-B output.
|
||||
pub aud: bool,
|
||||
/// HDR static metadata sent in an SEI with the first access unit: the 24-byte SMPTE ST 2086
|
||||
/// mastering display payload (payloadType 137) …
|
||||
pub mastering_display: Option<[u8; 24]>,
|
||||
/// … and (MaxCLL, MaxFALL) content light level (payloadType 144).
|
||||
pub content_light: Option<(u16, u16)>,
|
||||
}
|
||||
|
||||
impl EncoderConfig {
|
||||
@@ -178,6 +183,8 @@ impl EncoderConfig {
|
||||
sar: (1, 1),
|
||||
scenecut: true,
|
||||
slices: 0,
|
||||
mastering_display: None,
|
||||
content_light: None,
|
||||
aud: true,
|
||||
}
|
||||
}
|
||||
@@ -818,6 +825,9 @@ impl Encoder {
|
||||
self.sei_sent = true;
|
||||
let text = format!("filmcraft-h264enc {} - clean-room H.264 encoder", env!("CARGO_PKG_VERSION"));
|
||||
nals.push(nal::nal(0, NAL_SEI, &nal::sei_user_data_rbsp(&text)));
|
||||
if self.cfg.mastering_display.is_some() || self.cfg.content_light.is_some() {
|
||||
nals.push(nal::nal(0, NAL_SEI, &nal::sei_hdr_rbsp(self.cfg.mastering_display.as_ref(), self.cfg.content_light)));
|
||||
}
|
||||
}
|
||||
for r in results {
|
||||
nals.push(r.nal);
|
||||
|
||||
@@ -181,6 +181,25 @@ pub fn sei_user_data_rbsp(text: &str) -> Vec<u8> {
|
||||
w.finish()
|
||||
}
|
||||
|
||||
/// SEI with HDR static metadata: mastering_display_colour_volume (payloadType 137, the 24-byte
|
||||
/// ST 2086 payload) and/or content_light_level_info (payloadType 144), H.264 Annex D.
|
||||
pub fn sei_hdr_rbsp(mastering: Option<&[u8; 24]>, cll: Option<(u16, u16)>) -> Vec<u8> {
|
||||
let mut w = BitWriter::new();
|
||||
if let Some(m) = mastering {
|
||||
w.write_bits(137, 8);
|
||||
w.write_bits(24, 8);
|
||||
w.write_bytes(m);
|
||||
}
|
||||
if let Some((a, b)) = cll {
|
||||
w.write_bits(144, 8);
|
||||
w.write_bits(4, 8);
|
||||
w.write_bits(a as u32, 16);
|
||||
w.write_bits(b as u32, 16);
|
||||
}
|
||||
w.rbsp_trailing();
|
||||
w.finish()
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
|
||||
pub enum SliceType {
|
||||
P = 0,
|
||||
|
||||
@@ -41,6 +41,62 @@ pub struct VideoParams {
|
||||
pub field_info: Option<FieldInfo>,
|
||||
/// QuickTime `gama` (16.16 fixed point).
|
||||
pub gamma: Option<u32>,
|
||||
/// `mdcv`: mastering display colour volume (SMPTE ST 2086).
|
||||
pub mastering_display: Option<MasteringDisplay>,
|
||||
/// `clli`: content light level (MaxCLL, MaxFALL in cd/m²; 0 = unknown).
|
||||
pub content_light: Option<(u16, u16)>,
|
||||
}
|
||||
|
||||
/// SMPTE ST 2086 mastering display metadata, in the units of the `mdcv` box and the H.264/HEVC
|
||||
/// SEI: chromaticities in 0.00002 steps (order G, B, R as the SEI), luminance in 0.0001 cd/m².
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct MasteringDisplay {
|
||||
pub primaries: [(u16, u16); 3],
|
||||
pub white_point: (u16, u16),
|
||||
pub max_luminance: u32,
|
||||
pub min_luminance: u32,
|
||||
}
|
||||
|
||||
impl MasteringDisplay {
|
||||
/// BT.2020 primaries, D65, with the given peak and black in cd/m².
|
||||
pub fn bt2020(max_nits: f64, min_nits: f64) -> Self {
|
||||
let q = |v: f64| (v / 0.00002).round() as u16;
|
||||
MasteringDisplay {
|
||||
primaries: [(q(0.170), q(0.797)), (q(0.131), q(0.046)), (q(0.708), q(0.292))],
|
||||
white_point: (q(0.3127), q(0.3290)),
|
||||
max_luminance: (max_nits * 10_000.0).round() as u32,
|
||||
min_luminance: (min_nits * 10_000.0).round() as u32,
|
||||
}
|
||||
}
|
||||
pub fn max_nits(&self) -> f64 {
|
||||
self.max_luminance as f64 / 10_000.0
|
||||
}
|
||||
/// The 24-byte payload shared by `mdcv` and the SEI message.
|
||||
pub fn to_bytes(&self) -> Vec<u8> {
|
||||
let mut v = Vec::with_capacity(24);
|
||||
for (x, y) in self.primaries {
|
||||
v.extend_from_slice(&x.to_be_bytes());
|
||||
v.extend_from_slice(&y.to_be_bytes());
|
||||
}
|
||||
v.extend_from_slice(&self.white_point.0.to_be_bytes());
|
||||
v.extend_from_slice(&self.white_point.1.to_be_bytes());
|
||||
v.extend_from_slice(&self.max_luminance.to_be_bytes());
|
||||
v.extend_from_slice(&self.min_luminance.to_be_bytes());
|
||||
v
|
||||
}
|
||||
pub fn parse(p: &[u8]) -> Option<Self> {
|
||||
if p.len() < 24 {
|
||||
return None;
|
||||
}
|
||||
let u16_at = |i: usize| u16::from_be_bytes([p[i], p[i + 1]]);
|
||||
let u32_at = |i: usize| u32::from_be_bytes([p[i], p[i + 1], p[i + 2], p[i + 3]]);
|
||||
Some(MasteringDisplay {
|
||||
primaries: [(u16_at(0), u16_at(2)), (u16_at(4), u16_at(6)), (u16_at(8), u16_at(10))],
|
||||
white_point: (u16_at(12), u16_at(14)),
|
||||
max_luminance: u32_at(16),
|
||||
min_luminance: u32_at(20),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// `colr` box contents.
|
||||
@@ -779,6 +835,8 @@ pub(crate) fn parse_sample_entry(format: FourCc, payload: &[u8], handler: FourCc
|
||||
b"av1C" if f == b"av01" => codec = Some(CodecConfig::Av1(Av1Config::parse(p)?)),
|
||||
b"vpcC" if f == b"vp09" => codec = Some(CodecConfig::Vp9(VpcConfig::parse(p)?)),
|
||||
b"colr" => v.color = parse_colr(p).ok(),
|
||||
b"mdcv" => v.mastering_display = MasteringDisplay::parse(p),
|
||||
b"clli" if p.len() >= 4 => v.content_light = Some((u16::from_be_bytes([p[0], p[1]]), u16::from_be_bytes([p[2], p[3]]))),
|
||||
b"pasp" => {
|
||||
let mut c = Cur::new(p, "pasp");
|
||||
v.pixel_aspect = Some((c.u32()?, c.u32()?));
|
||||
@@ -1216,6 +1274,14 @@ impl SampleEntry {
|
||||
}
|
||||
b.end(m);
|
||||
}
|
||||
if let Some(md) = &v.mastering_display {
|
||||
b.leaf(b"mdcv", &md.to_bytes());
|
||||
}
|
||||
if let Some((cll, fall)) = v.content_light {
|
||||
let mut d = cll.to_be_bytes().to_vec();
|
||||
d.extend_from_slice(&fall.to_be_bytes());
|
||||
b.leaf(b"clli", &d);
|
||||
}
|
||||
if let Some((h, vv)) = v.pixel_aspect {
|
||||
let m = b.start(b"pasp");
|
||||
b.u32(h);
|
||||
@@ -1373,8 +1439,11 @@ mod tests {
|
||||
e.video.as_mut().unwrap().pixel_aspect = Some((4, 3));
|
||||
e.video.as_mut().unwrap().field_info = Some(FieldInfo { fields: 2, detail: 9 });
|
||||
e.video.as_mut().unwrap().clean_aperture = Some(CleanAperture { width: (1920, 1), height: (1080, 1), horiz_offset: (-2, 1), vert_offset: (0, 1) });
|
||||
e.video.as_mut().unwrap().mastering_display = Some(MasteringDisplay::bt2020(1000.0, 0.0001));
|
||||
e.video.as_mut().unwrap().content_light = Some((1000, 400));
|
||||
e.bitrate = Some(BitRate { buffer_size: 1, max_bitrate: 2, avg_bitrate: 3 });
|
||||
assert_eq!(roundtrip(&e, false, b"vide"), e);
|
||||
assert_eq!(MasteringDisplay::bt2020(1000.0, 0.0001).max_nits(), 1000.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -17,8 +17,8 @@ mod source;
|
||||
|
||||
pub use bytes::FourCc;
|
||||
pub use codec::{
|
||||
AacConfig, AudioParams, Av1Config, AvcConfig, BitRate, CleanAperture, CodecConfig, ColorInfo, FieldInfo, FlacConfig, HevcConfig, HevcNalArray, OpusConfig,
|
||||
PcmConfig, SampleEntry, TimecodeConfig, VideoParams, VpcConfig, parse_asc,
|
||||
AacConfig, AudioParams, Av1Config, AvcConfig, BitRate, CleanAperture, CodecConfig, ColorInfo, FieldInfo, FlacConfig, HevcConfig, HevcNalArray,
|
||||
MasteringDisplay, OpusConfig, PcmConfig, SampleEntry, TimecodeConfig, VideoParams, VpcConfig, parse_asc,
|
||||
};
|
||||
pub use demux::{Edit, Metadata, Mp4File, Sample, SampleGroup, Track, TrackKind, open};
|
||||
pub use error::{Error, Result};
|
||||
|
||||
@@ -153,6 +153,9 @@ pub struct Interpretation {
|
||||
pub par: Option<(u32, u32)>,
|
||||
pub ignore_alpha: bool,
|
||||
pub invert_alpha: bool,
|
||||
/// Color Management ▸ override the colour space detected from the file's metadata.
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub color_space: Option<filmcraft_color::ColorSpace>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
@@ -623,8 +626,16 @@ pub struct SequenceSettings {
|
||||
pub preview_codec: String,
|
||||
pub max_bit_depth: bool,
|
||||
pub max_render_quality: bool,
|
||||
/// Working colour space.
|
||||
/// Working colour space (display label; [`SequenceSettings::color`] is authoritative).
|
||||
pub working_space: String,
|
||||
/// Colour pipeline: working space (Rec. 709 / Rec. 2100 PQ / HLG), wide-gamut compositing,
|
||||
/// Auto Tone Map Media.
|
||||
#[serde(default = "default_color_pipeline")]
|
||||
pub color: filmcraft_color::ColorPipeline,
|
||||
}
|
||||
|
||||
fn default_color_pipeline() -> filmcraft_color::ColorPipeline {
|
||||
filmcraft_color::ColorPipeline::REC709
|
||||
}
|
||||
|
||||
impl Default for SequenceSettings {
|
||||
@@ -642,6 +653,7 @@ impl Default for SequenceSettings {
|
||||
max_bit_depth: false,
|
||||
max_render_quality: false,
|
||||
working_space: "Rec. 709".into(),
|
||||
color: filmcraft_color::ColorPipeline::REC709,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
//! Colour management in the renderer: source colour space resolution (Interpret Footage override
|
||||
//! or stream metadata), source → working conversion of decoded frames, and the working → monitor
|
||||
//! transform. The maths lives in `filmcraft_color::transform`.
|
||||
//!
|
||||
//! A sequence whose pipeline is "plain" (Rec. 709 working space without wide gamut) and media
|
||||
//! whose metadata says Rec. 709 / sRGB / linear take the original fast path unchanged.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
|
||||
use filmcraft_color::{ColorPipeline, ColorSpace, InputTransform, OutputTransform, Range};
|
||||
use filmcraft_frame::VideoFrame;
|
||||
use filmcraft_project::{ItemId, ItemKind, Project};
|
||||
|
||||
use crate::Image;
|
||||
|
||||
/// The Interpret Footage colour-space override of an item (following subclips to their media).
|
||||
pub fn override_of(project: &Project, item: ItemId) -> Option<ColorSpace> {
|
||||
match &project.item(item)?.kind {
|
||||
ItemKind::Media(m) => m.interpret.color_space,
|
||||
ItemKind::Subclip { parent, .. } => override_of(project, *parent),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// The colour space a frame of `item` is interpreted in.
|
||||
pub fn source_space(project: &Project, item: ItemId, frame: &VideoFrame) -> ColorSpace {
|
||||
override_of(project, item).unwrap_or_else(|| ColorSpace::from_info(&frame.color))
|
||||
}
|
||||
|
||||
/// Whether the frame needs more than the default (transfer-only) decode.
|
||||
pub fn needs_management(pipe: &ColorPipeline, cs: ColorSpace, frame: &VideoFrame) -> bool {
|
||||
let default_decode = cs == ColorSpace::from_info(&frame.color) && matches!(cs, ColorSpace::Rec709 | ColorSpace::Srgb | ColorSpace::Linear709);
|
||||
!(pipe.is_plain() && default_decode)
|
||||
}
|
||||
|
||||
type Key = (ColorSpace, Range, ColorPipeline);
|
||||
|
||||
/// A cached input transform.
|
||||
pub fn input_transform(cs: ColorSpace, range: Range, pipe: &ColorPipeline) -> Arc<InputTransform> {
|
||||
static C: OnceLock<Mutex<HashMap<Key, Arc<InputTransform>>>> = OnceLock::new();
|
||||
let c = C.get_or_init(Default::default);
|
||||
let key = (cs, range, *pipe);
|
||||
if let Some(t) = c.lock().unwrap_or_else(|e| e.into_inner()).get(&key) {
|
||||
return t.clone();
|
||||
}
|
||||
let t = Arc::new(InputTransform::new(cs, range, pipe, None));
|
||||
c.lock().unwrap_or_else(|e| e.into_inner()).insert(key, t.clone());
|
||||
t
|
||||
}
|
||||
|
||||
/// Decode a frame of `item` (box-decimated by `n`) into the working space of `pipe`.
|
||||
pub fn decode(project: &Project, item: ItemId, frame: &VideoFrame, n: usize, pipe: &ColorPipeline) -> Image {
|
||||
let cs = source_space(project, item, frame);
|
||||
if !needs_management(pipe, cs, frame) {
|
||||
let (w, h, px) = frame.to_linear_f32_decimated(n);
|
||||
return Image { w, h, px };
|
||||
}
|
||||
// RGB stills/generators carry full-range code values; YUV carries its own range flag
|
||||
let range = match frame.data {
|
||||
filmcraft_frame::PixelData::Rgba8(_) | filmcraft_frame::PixelData::RgbaF32(_) => Range::Full,
|
||||
_ => frame.color.range,
|
||||
};
|
||||
let t = input_transform(cs, range, pipe);
|
||||
let float_src = matches!(frame.data, filmcraft_frame::PixelData::RgbaF32(_));
|
||||
// float frames are linear already: only the gamut/tone stages apply to them
|
||||
let (w, h, px) = frame.to_linear_f32_decimated_with(n, if float_src { None } else { Some(&t.table) });
|
||||
let mut img = Image { w, h, px };
|
||||
if t.has_pixel_stage() {
|
||||
img.map_rgb(|c, _, _| t.apply(c));
|
||||
}
|
||||
img
|
||||
}
|
||||
|
||||
/// Working space → monitor (SDR BT.709) for a non-plain pipeline; no-op otherwise.
|
||||
pub fn to_display(img: &mut Image, pipe: &ColorPipeline) {
|
||||
if pipe.is_plain() {
|
||||
return;
|
||||
}
|
||||
let o = display_transform(pipe);
|
||||
if o.is_identity() {
|
||||
return;
|
||||
}
|
||||
img.map_rgb(|c, _, _| o.apply(c));
|
||||
}
|
||||
|
||||
pub fn display_transform(pipe: &ColorPipeline) -> Arc<OutputTransform> {
|
||||
static C: OnceLock<Mutex<HashMap<ColorPipeline, Arc<OutputTransform>>>> = OnceLock::new();
|
||||
let c = C.get_or_init(Default::default);
|
||||
c.lock().unwrap_or_else(|e| e.into_inner()).entry(*pipe).or_insert_with(|| Arc::new(OutputTransform::display(pipe))).clone()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use filmcraft_color::{ColorInfo, Primaries, Transfer, WorkingSpace};
|
||||
|
||||
fn frame(rgb: [u8; 3], color: ColorInfo) -> VideoFrame {
|
||||
let mut f = VideoFrame::rgba8(2, 2, [rgb[0], rgb[1], rgb[2], 255].repeat(4));
|
||||
f.color = color;
|
||||
f
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn plain_media_keeps_the_fast_path() {
|
||||
let f = frame([200, 100, 50], ColorInfo::SRGB_FULL);
|
||||
assert!(!needs_management(&ColorPipeline::REC709, ColorSpace::Srgb, &f));
|
||||
let hdr = ColorPipeline { working: WorkingSpace::Rec2100Pq, ..ColorPipeline::REC709 };
|
||||
assert!(needs_management(&hdr, ColorSpace::Srgb, &f));
|
||||
let pq = ColorInfo { transfer: Transfer::Pq, primaries: Primaries::Bt2020, ..ColorInfo::SRGB_FULL };
|
||||
assert!(needs_management(&ColorPipeline::REC709, ColorSpace::from_info(&pq), &frame([1, 2, 3], pq)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pq_frame_is_tone_mapped_into_rec709() {
|
||||
let p = Project::new("t");
|
||||
let pq = ColorInfo { transfer: Transfer::Pq, primaries: Primaries::Bt2020, ..ColorInfo::SRGB_FULL };
|
||||
// ~1000 cd/m² white (PQ 0.75) → SDR peak after BT.2390; 203 cd/m² (PQ 0.58) → ≈ 0.8
|
||||
let hi = decode(&p, ItemId(99), &frame([192, 192, 192], pq), 1, &ColorPipeline::REC709);
|
||||
assert!((hi.px[0] - 1.0).abs() < 0.03, "{:?}", &hi.px[..4]);
|
||||
let rw = decode(&p, ItemId(99), &frame([148, 148, 148], pq), 1, &ColorPipeline::REC709);
|
||||
assert!((0.7..0.9).contains(&rw.px[0]), "{:?}", &rw.px[..4]);
|
||||
// the same frame in a PQ sequence keeps its HDR values (working units, 1.0 = 203 cd/m²)
|
||||
let hdr = ColorPipeline { working: WorkingSpace::Rec2100Pq, ..ColorPipeline::REC709 };
|
||||
let w = decode(&p, ItemId(99), &frame([192, 192, 192], pq), 1, &hdr);
|
||||
assert!(w.px[0] > 4.0, "{:?}", &w.px[..4]);
|
||||
let mut d = w.clone();
|
||||
to_display(&mut d, &hdr);
|
||||
assert!((d.px[0] - 1.0).abs() < 0.03);
|
||||
}
|
||||
}
|
||||
@@ -10,6 +10,7 @@
|
||||
pub mod audio;
|
||||
pub mod audio_fx;
|
||||
pub mod blend;
|
||||
pub mod colorman;
|
||||
pub mod effects;
|
||||
pub mod graphic_clip;
|
||||
pub mod graphics;
|
||||
@@ -52,11 +53,14 @@ pub struct RenderOptions {
|
||||
/// Draw the sequence's visible caption tracks over the picture (Program monitor, burn-in on
|
||||
/// export). Never applies to nested sequences.
|
||||
pub captions: bool,
|
||||
/// Return the image in the sequence's working space (HDR exports, scopes) instead of
|
||||
/// converting it for an SDR monitor. Only matters for HDR / wide-gamut sequences.
|
||||
pub working_output: bool,
|
||||
}
|
||||
|
||||
impl Default for RenderOptions {
|
||||
fn default() -> Self {
|
||||
Self { scale: 1.0, effects: true, depth: 0, captions: false }
|
||||
Self { scale: 1.0, effects: true, depth: 0, captions: false, working_output: false }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -139,6 +143,9 @@ fn render_seq(project: &Project, seq: &Sequence, t: Tick, opts: RenderOptions, s
|
||||
blend::composite(&mut canvas, &layer, op, bl);
|
||||
}
|
||||
}
|
||||
if opts.depth == 0 && !opts.working_output {
|
||||
colorman::to_display(&mut canvas, &seq.settings.color);
|
||||
}
|
||||
if opts.captions && opts.depth == 0 {
|
||||
for o in caption_overlays(seq, t, w, h) {
|
||||
o.composite_onto(&mut canvas.px, w, h);
|
||||
@@ -238,11 +245,10 @@ pub(crate) fn item_layer(
|
||||
let src = sources.source(item.item)?;
|
||||
let frame = src.video_frame(FrameRequest { time: mt, scale: want }).ok()?;
|
||||
let n = decimation(frame.width as f32, src_size.0 as f32 * want);
|
||||
let (w, h, px) = frame.to_linear_f32_decimated(n);
|
||||
Image { w, h, px }
|
||||
colorman::decode(project, item.item, &frame, n, &seq.settings.color)
|
||||
}
|
||||
ItemKind::Sequence(nested) => {
|
||||
let sub = RenderOptions { scale: want, effects: opts.effects, depth: opts.depth + 1, captions: false };
|
||||
let sub = RenderOptions { scale: want, effects: opts.effects, depth: opts.depth + 1, captions: false, working_output: true };
|
||||
render_seq(project, nested, mt, sub, sources)
|
||||
}
|
||||
ItemKind::AdjustmentLayer { .. } => return None,
|
||||
@@ -307,8 +313,8 @@ pub fn render_item(project: &Project, item: ItemId, t: Tick, scale: f32, sources
|
||||
let f = src.video_frame(FrameRequest { time: t, scale }).ok()?;
|
||||
let full_w = src.info().video.as_ref().map_or(f.width, |v| v.width) as f32;
|
||||
let n = decimation(f.width as f32, full_w * scale);
|
||||
let (w, h, px) = f.to_linear_f32_decimated(n);
|
||||
Some(Image { w, h, px })
|
||||
// the Source monitor shows media as SDR Rec. 709 (log/HDR tone mapped per its colour space)
|
||||
Some(colorman::decode(project, item, &f, n, &filmcraft_color::ColorPipeline::REC709))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -56,6 +56,10 @@ fn gpu_simple(project: &Project, item: &TrackItem, mt: Tick) -> bool {
|
||||
pub fn plan_frame(project: &Project, seq_id: ItemId, t: Tick, opts: RenderOptions, sources: &dyn SourceProvider) -> FramePlan {
|
||||
let Some(seq) = project.sequence(seq_id) else { return FramePlan::Image(crate::Image::new(1, 1)) };
|
||||
let (w, h) = output_size(seq, opts.scale);
|
||||
// HDR / wide-gamut sequences composite and convert on the CPU.
|
||||
if !seq.settings.color.is_plain() {
|
||||
return FramePlan::Image(crate::render_sequence(project, seq_id, t, opts, sources));
|
||||
}
|
||||
// Whole-frame fallback: adjustment layers or complex transitions anywhere at t.
|
||||
for tr in &seq.video_tracks {
|
||||
if !tr.enabled {
|
||||
@@ -162,10 +166,14 @@ fn push_item(
|
||||
let lin = ((motion.a * motion.a + motion.b * motion.b).sqrt()).max((motion.c * motion.c + motion.d * motion.d).sqrt());
|
||||
let want = (lin * opts.scale as f64).clamp(1.0 / 64.0, 1.0) as f32;
|
||||
let Ok(frame) = src.video_frame(FrameRequest { time: mt, scale: want }) else { return };
|
||||
let px_scale = frame.width as f64 / size.0.max(1) as f64;
|
||||
let m = Affine::scale(opts.scale as f64, opts.scale as f64).then_apply(&motion).then_apply(&Affine::scale(1.0 / px_scale, 1.0 / px_scale));
|
||||
out.push(PlanLayer { frame, matrix: m, opacity: op * extra_opacity });
|
||||
return;
|
||||
let cs = crate::colorman::source_space(project, item.item, &frame);
|
||||
// log / HDR / wide-gamut media is converted on the CPU (below)
|
||||
if !crate::colorman::needs_management(&seq.settings.color, cs, &frame) {
|
||||
let px_scale = frame.width as f64 / size.0.max(1) as f64;
|
||||
let m = Affine::scale(opts.scale as f64, opts.scale as f64).then_apply(&motion).then_apply(&Affine::scale(1.0 / px_scale, 1.0 / px_scale));
|
||||
out.push(PlanLayer { frame, matrix: m, opacity: op * extra_opacity });
|
||||
return;
|
||||
}
|
||||
}
|
||||
// CPU-rendered layer (standard effects): drawn by the GPU as a pre-rendered canvas image.
|
||||
let tc = filmcraft_time::format_time(t, seq.settings.frame_rate, seq.settings.drop_frame, filmcraft_time::TimeDisplay::Timecode, 48_000);
|
||||
|
||||
@@ -182,6 +182,17 @@ pub fn invoke(app: &mut FilmcraftApp, ctx: &egui::Context, id: &str, params: Val
|
||||
app.dialog = Some(crate::Dialog::AudioGain);
|
||||
return Ok(json!({"dialog": "audioGain"}));
|
||||
}
|
||||
// Colour dialogs from the menus; with params the engine command applies directly.
|
||||
"clip.interpretFootage" if params.get("colorSpace").is_none() => {
|
||||
filmcraft_engine::find_command(id).map_or(Ok(()), |c| (c.enabled)(&app.session))?;
|
||||
crate::panels::color_dialogs::open_interpret(app, ¶ms);
|
||||
return Ok(json!({"dialog": "interpretFootage"}));
|
||||
}
|
||||
"sequence.colorSettings" if params.as_object().is_none_or(|m| m.is_empty()) => {
|
||||
filmcraft_engine::find_command(id).map_or(Ok(()), |c| (c.enabled)(&app.session))?;
|
||||
crate::panels::color_dialogs::open_sequence(app);
|
||||
return Ok(json!({"dialog": "sequenceColor"}));
|
||||
}
|
||||
// From menus/shortcuts (no params) these ask first; agents pass params to act directly.
|
||||
"file.revert" if params.as_object().is_none_or(|m| m.is_empty()) && app.session.is_dirty() => {
|
||||
if app.session.path.is_none() {
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
//! Colour dialogs: Clip ▸ Modify ▸ Interpret Footage… (Color Management) and Sequence ▸ Color
|
||||
//! Management… (working space, wide gamut, Auto Tone Map Media). Each applies one engine command
|
||||
//! on OK, so it is a single undo step. Automation ids: `colorDialog.space.<id>`,
|
||||
//! `colorDialog.working.<id>`, `colorDialog.wideGamut`, `colorDialog.autoToneMap`,
|
||||
//! `colorDialog.ok`, `colorDialog.cancel`.
|
||||
|
||||
use filmcraft_color::{ColorSpace, WorkingSpace};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::FilmcraftApp;
|
||||
use crate::state::ColorDialog;
|
||||
|
||||
pub fn open_interpret(app: &mut FilmcraftApp, params: &Value) {
|
||||
let items: Vec<u64> = match params.get("items").and_then(Value::as_array) {
|
||||
Some(a) => a.iter().filter_map(Value::as_u64).collect(),
|
||||
None => {
|
||||
let s = &app.session;
|
||||
let mut v: Vec<u64> = s.state.project_selection.iter().map(|i| i.0).collect();
|
||||
if v.is_empty()
|
||||
&& let Some(q) = s.active_sequence()
|
||||
{
|
||||
v = s.state.selection.iter().filter_map(|c| q.find_item(*c).map(|(_, it)| it.item.0)).collect();
|
||||
v.dedup();
|
||||
}
|
||||
v
|
||||
}
|
||||
};
|
||||
let current = items
|
||||
.first()
|
||||
.and_then(|i| app.session.execute("media.colorInfo", json!({"item": i})).ok())
|
||||
.and_then(|v| v["override"].as_str().map(str::to_string))
|
||||
.unwrap_or_else(|| "auto".into());
|
||||
app.ui.color_dialog = Some(ColorDialog::Interpret { items, color_space: current });
|
||||
}
|
||||
|
||||
pub fn open_sequence(app: &mut FilmcraftApp) {
|
||||
let c = app.session.active_sequence().map(|q| q.settings.color).unwrap_or(filmcraft_color::ColorPipeline::REC709);
|
||||
app.ui.color_dialog = Some(ColorDialog::Sequence { working_space: c.working.id().into(), wide_gamut: c.wide_gamut, auto_tone_map: c.auto_tone_map });
|
||||
}
|
||||
|
||||
fn radio(app: &mut FilmcraftApp, ui: &mut egui::Ui, id: &str, selected: bool, label: &str) -> bool {
|
||||
let r = ui.radio(selected, label);
|
||||
app.auto.add(id, r.rect, label);
|
||||
r.clicked()
|
||||
}
|
||||
|
||||
fn buttons(app: &mut FilmcraftApp, ui: &mut egui::Ui) -> (bool, bool) {
|
||||
let mut out = (false, false);
|
||||
ui.add_space(6.0);
|
||||
ui.horizontal(|ui| {
|
||||
let c = ui.button("Cancel");
|
||||
app.auto.add("colorDialog.cancel", c.rect, "Cancel");
|
||||
let o = ui.button("OK");
|
||||
app.auto.add("colorDialog.ok", o.rect, "OK");
|
||||
out = (o.clicked(), c.clicked());
|
||||
});
|
||||
out
|
||||
}
|
||||
|
||||
pub fn show(app: &mut FilmcraftApp, ctx: &egui::Context) {
|
||||
let Some(mut d) = app.ui.color_dialog.clone() else { return };
|
||||
let mut open = true;
|
||||
let mut apply: Option<(&'static str, Value)> = None;
|
||||
let mut close = false;
|
||||
match &mut d {
|
||||
ColorDialog::Interpret { items, color_space } => {
|
||||
let detected = items
|
||||
.first()
|
||||
.and_then(|i| app.session.execute("media.colorInfo", json!({"item": i})).ok())
|
||||
.and_then(|v| v["detectedLabel"].as_str().map(str::to_string))
|
||||
.unwrap_or_else(|| "—".into());
|
||||
let n = items.len();
|
||||
egui::Window::new("Interpret Footage").open(&mut open).collapsible(false).resizable(false).default_width(380.0).show(ctx, |ui| {
|
||||
ui.label(egui::RichText::new(format!("Color Management · {n} item{}", if n == 1 { "" } else { "s" })).strong());
|
||||
ui.label(format!("Media colour space (from file metadata): {detected}"));
|
||||
ui.add_space(4.0);
|
||||
ui.label("Override media colour space:");
|
||||
if radio(app, ui, "colorDialog.space.auto", color_space == "auto", "Use file metadata") {
|
||||
*color_space = "auto".into();
|
||||
}
|
||||
egui::ScrollArea::vertical().max_height(300.0).show(ui, |ui| {
|
||||
for cs in ColorSpace::ALL {
|
||||
if radio(app, ui, &format!("colorDialog.space.{}", cs.id()), color_space == cs.id(), cs.label()) {
|
||||
*color_space = cs.id().into();
|
||||
}
|
||||
}
|
||||
});
|
||||
let (ok, cancel) = buttons(app, ui);
|
||||
if ok {
|
||||
apply = Some(("clip.interpretFootage", json!({"items": items, "colorSpace": color_space})));
|
||||
}
|
||||
close = ok || cancel;
|
||||
});
|
||||
}
|
||||
ColorDialog::Sequence { working_space, wide_gamut, auto_tone_map } => {
|
||||
egui::Window::new("Sequence Color Management").open(&mut open).collapsible(false).resizable(false).default_width(340.0).show(ctx, |ui| {
|
||||
ui.label(egui::RichText::new("Working Color Space").strong());
|
||||
for w in WorkingSpace::ALL {
|
||||
if radio(app, ui, &format!("colorDialog.working.{}", w.id()), working_space == w.id(), w.label()) {
|
||||
*working_space = w.id().into();
|
||||
}
|
||||
}
|
||||
ui.add_space(4.0);
|
||||
let r = ui.checkbox(wide_gamut, "Wide gamut color (composite in BT.2020)");
|
||||
app.auto.add("colorDialog.wideGamut", r.rect, "Wide gamut");
|
||||
let r = ui.checkbox(auto_tone_map, "Auto tone map media (HDR and log into SDR)");
|
||||
app.auto.add("colorDialog.autoToneMap", r.rect, "Auto tone map");
|
||||
let (ok, cancel) = buttons(app, ui);
|
||||
if ok {
|
||||
apply = Some(("sequence.colorSettings", json!({"workingSpace": working_space, "wideGamut": *wide_gamut, "autoToneMap": *auto_tone_map})));
|
||||
}
|
||||
close = ok || cancel;
|
||||
});
|
||||
}
|
||||
}
|
||||
app.ui.color_dialog = if close || !open { None } else { Some(d) };
|
||||
if let Some((cmd, p)) = apply
|
||||
&& let Err(e) = app.session.execute(cmd, p)
|
||||
{
|
||||
app.ui.status = e.to_string();
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@
|
||||
use crate::{Dialog, FilmcraftApp};
|
||||
|
||||
pub fn show(app: &mut FilmcraftApp, ctx: &egui::Context) {
|
||||
crate::panels::color_dialogs::show(app, ctx);
|
||||
let Some(d) = app.dialog else { return };
|
||||
if let Some(still_open) = crate::panels::file_dialogs::show(app, ctx, d) {
|
||||
if !still_open && app.dialog == Some(d) {
|
||||
|
||||
@@ -206,6 +206,11 @@ pub fn scopes(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) {
|
||||
let key = FrameKey { target: Target::Sequence(seq_id), frame, size: 250, revision: app.session.revision };
|
||||
let project = app.session.project.clone();
|
||||
app.frames.request(key, rate.tick_of(frame), 0.25, &project, 2);
|
||||
let pipe = app.session.active_sequence().map(|q| q.settings.color).unwrap_or(filmcraft_color::ColorPipeline::REC709);
|
||||
if pipe.working.is_hdr() {
|
||||
hdr_scopes(app, ui, rect, (seq_id.0, frame, app.session.revision));
|
||||
return;
|
||||
}
|
||||
let Some(img) = app.frames.get(&key) else {
|
||||
crate::dock::placeholder(ui, rect, &t, "…");
|
||||
return;
|
||||
@@ -243,3 +248,68 @@ pub fn scopes(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect) {
|
||||
}
|
||||
ui.painter().circle_stroke(vs.center(), vs_size / 2.0, egui::Stroke::new(1.0, Color32::from_white_alpha(40)));
|
||||
}
|
||||
|
||||
/// Nits → waveform height (0..1) on the PQ scale used by HDR waveforms.
|
||||
fn pq_axis(nits: f32) -> f32 {
|
||||
filmcraft_color::pq_inverse_eotf((nits / 10_000.0).clamp(0.0, 1.0))
|
||||
}
|
||||
|
||||
/// Scopes of an HDR (Rec. 2100 PQ/HLG) sequence: luminance waveform in cd/m² on a PQ-scaled
|
||||
/// axis (0–10 000 nits, reference white 203 marked) and a BT.2020 vectorscope. The frame is the
|
||||
/// sequence rendered in its working space (HDR values kept), small and cached per frame.
|
||||
fn hdr_scopes(app: &mut FilmcraftApp, ui: &mut egui::Ui, rect: Rect, key: (u64, i64, u64)) {
|
||||
use std::sync::Mutex;
|
||||
static CACHE: Mutex<Option<((u64, i64, u64), filmcraft_render::Image)>> = Mutex::new(None);
|
||||
let t = app.tokens;
|
||||
let img = {
|
||||
let mut c = CACHE.lock().unwrap_or_else(|e| e.into_inner());
|
||||
if c.as_ref().is_none_or(|(k, _)| *k != key)
|
||||
&& let Some(img) = app.session.render_program_working(0.125)
|
||||
{
|
||||
*c = Some((key, img));
|
||||
}
|
||||
c.as_ref().map(|(_, i)| i.clone())
|
||||
};
|
||||
let Some(img) = img else {
|
||||
crate::dock::placeholder(ui, rect, &t, "…");
|
||||
return;
|
||||
};
|
||||
let half = rect.width() / 2.0;
|
||||
let wf = Rect::from_min_size(rect.min + vec2(34.0, 8.0), vec2(half - 38.0, rect.height() - 16.0));
|
||||
let vs_size = (rect.height() - 16.0).min(half - 12.0);
|
||||
let vs = Rect::from_center_size(pos2(rect.min.x + half + half / 2.0, rect.center().y), vec2(vs_size, vs_size));
|
||||
ui.painter().rect_filled(wf, 0.0, Color32::BLACK);
|
||||
ui.painter().rect_filled(vs, vs_size / 2.0, Color32::BLACK);
|
||||
let luma = filmcraft_color::Gamut::Bt2020.luma();
|
||||
let (w, h) = (img.w, img.h);
|
||||
let mut mesh = egui::Mesh::default();
|
||||
let mut peak = 0f32;
|
||||
for y in 0..h {
|
||||
for x in 0..w {
|
||||
let p = img.get(x, y);
|
||||
let a = p[3].max(1e-6);
|
||||
let c = [p[0] / a, p[1] / a, p[2] / a];
|
||||
let nits = (luma[0] as f32 * c[0] + luma[1] as f32 * c[1] + luma[2] as f32 * c[2]).max(0.0) * filmcraft_color::REFERENCE_WHITE_NITS as f32;
|
||||
peak = peak.max(nits);
|
||||
let px = wf.min.x + x as f32 / w as f32 * wf.width();
|
||||
let py = wf.max.y - pq_axis(nits) * wf.height();
|
||||
mesh.add_colored_rect(Rect::from_min_size(pos2(px, py), vec2(1.5, 1.0)), Color32::from_rgba_unmultiplied(120, 255, 140, 70));
|
||||
// vectorscope on PQ-encoded BT.2020 Y'CbCr
|
||||
let e = c.map(|v| pq_axis(v.max(0.0) * filmcraft_color::REFERENCE_WHITE_NITS as f32));
|
||||
let ycc = filmcraft_color::rgb_to_ycbcr(e[0], e[1], e[2], filmcraft_color::Matrix::Bt2020Ncl);
|
||||
let vx = vs.center().x + ycc[1] * vs_size;
|
||||
let vy = vs.center().y - ycc[2] * vs_size;
|
||||
mesh.add_colored_rect(Rect::from_min_size(pos2(vx, vy), vec2(1.0, 1.0)), Color32::from_rgba_unmultiplied(200, 255, 200, 60));
|
||||
}
|
||||
}
|
||||
ui.painter().add(mesh);
|
||||
for (nits, label) in [(0.0, "0"), (10.0, "10"), (100.0, "100"), (203.0, "203"), (1000.0, "1000"), (4000.0, "4000"), (10_000.0, "10000")] {
|
||||
let y = wf.max.y - pq_axis(nits) * wf.height();
|
||||
let col = if nits == 203.0 { Color32::from_rgba_unmultiplied(255, 200, 80, 90) } else { Color32::from_white_alpha(30) };
|
||||
ui.painter().line_segment([pos2(wf.min.x, y), pos2(wf.max.x, y)], egui::Stroke::new(0.5, col));
|
||||
ui.painter().text(pos2(wf.min.x - 3.0, y), Align2::RIGHT_CENTER, label, Tokens::ui(8.0), t.text_faint);
|
||||
}
|
||||
ui.painter().text(wf.left_top() + vec2(4.0, 4.0), Align2::LEFT_TOP, format!("HDR · cd/m² · peak {peak:.0}"), Tokens::ui(9.0), t.text_dim);
|
||||
app.auto.add("scopes.hdrWaveform", wf, &format!("HDR waveform, peak {peak:.0} nits"));
|
||||
ui.painter().circle_stroke(vs.center(), vs_size / 2.0, egui::Stroke::new(1.0, Color32::from_white_alpha(40)));
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! Panel bodies. `show` dispatches on [`PanelKind`]; drag-and-drop between panels (project items,
|
||||
//! effects) is carried in egui temp data so the timeline/monitors can accept drops.
|
||||
|
||||
pub mod color_dialogs;
|
||||
pub mod dialogs;
|
||||
pub mod effect_controls;
|
||||
pub mod effects;
|
||||
|
||||
@@ -301,6 +301,10 @@ fn item_interactions(
|
||||
actions.push(("edit.duplicate".into(), json!({})));
|
||||
ui.close();
|
||||
}
|
||||
if ui.button("Interpret Footage…").clicked() {
|
||||
actions.push(("clip.interpretFootage".into(), json!({"items": [id.0]})));
|
||||
ui.close();
|
||||
}
|
||||
ui.menu_button("Label", |ui| {
|
||||
for l in filmcraft_project::Label::ALL {
|
||||
if ui.button(l.name()).clicked() {
|
||||
|
||||
@@ -292,6 +292,18 @@ pub struct UiState {
|
||||
/// Pen tool: path points placed so far (sequence pixels).
|
||||
#[serde(default)]
|
||||
pub pen_points: Vec<[f64; 2]>,
|
||||
/// Open colour dialog (Interpret Footage ▸ Color Management, Sequence ▸ Color Management).
|
||||
#[serde(default)]
|
||||
pub color_dialog: Option<ColorDialog>,
|
||||
}
|
||||
|
||||
/// Colour dialogs: drafts are applied with one engine command on OK.
|
||||
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
|
||||
pub enum ColorDialog {
|
||||
/// `clip.interpretFootage`: the media items and the chosen colour space id ("auto" = metadata).
|
||||
Interpret { items: Vec<u64>, color_space: String },
|
||||
/// `sequence.colorSettings`.
|
||||
Sequence { working_space: String, wide_gamut: bool, auto_tone_map: bool },
|
||||
}
|
||||
|
||||
/// The Audio Gain dialog (Clip ▸ Audio Gain…, G).
|
||||
@@ -356,6 +368,7 @@ impl Default for UiState {
|
||||
audio_gain: AudioGainDraft::default(),
|
||||
gfx_edit: None,
|
||||
pen_points: vec![],
|
||||
color_dialog: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -165,3 +165,41 @@ fn lumetri_lut_menus_and_section_switches() {
|
||||
let e = lumetri(&mut d, clip);
|
||||
assert_eq!(e["params"]["creative_on"]["value"], json!("Bool(true)"), "{e}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn interpret_footage_and_sequence_color_dialogs_and_hdr_scopes() {
|
||||
let mut d = Driver::demo();
|
||||
d.ok("ui.set", json!({"workspace": "Color"}));
|
||||
d.frames(3);
|
||||
let clip = select_first_clip(&mut d);
|
||||
let seq = d.exec("sequence.inspect", json!({}));
|
||||
let item = seq["video"][0]["items"].as_array().unwrap().iter().find(|i| i["clip"] == clip).unwrap()["item"].clone();
|
||||
// Clip ▸ Modify ▸ Interpret Footage… opens the dialog (no params)
|
||||
let r = d.ok("ui.menu.invoke", json!({"id": "clip.interpretFootage"}));
|
||||
assert_eq!(r["dialog"], "interpretFootage");
|
||||
d.frames(3);
|
||||
d.click("colorDialog.space.slog3-sgamut3cine");
|
||||
d.snapshot("color-interpret-footage", None);
|
||||
d.click("colorDialog.ok");
|
||||
let info = d.exec("media.colorInfo", json!({"item": item}));
|
||||
assert_eq!(info["override"], "slog3-sgamut3cine", "{info}");
|
||||
// Sequence ▸ Color Management…: Rec. 2100 PQ
|
||||
let r = d.ok("ui.menu.invoke", json!({"id": "sequence.colorSettings"}));
|
||||
assert_eq!(r["dialog"], "sequenceColor");
|
||||
d.frames(3);
|
||||
d.click("colorDialog.working.rec2100-pq");
|
||||
d.snapshot("color-sequence-settings", None);
|
||||
d.click("colorDialog.ok");
|
||||
let st = d.exec("sequence.colorSettings", json!({"autoToneMap": true}));
|
||||
assert_eq!(st["workingSpace"], "rec2100-pq");
|
||||
// the scopes switch to the HDR waveform (cd/m², PQ scale)
|
||||
d.ok("ui.panel.show", json!({"panel": "LumetriScopes"}));
|
||||
d.frames(6);
|
||||
assert!(d.ids("scopes.").iter().any(|i| i == "scopes.hdrWaveform"), "{:?}", d.ids("scopes."));
|
||||
d.snapshot("color-hdr-scopes", None);
|
||||
// undo the working space and the interpretation
|
||||
d.exec("edit.undo", json!({}));
|
||||
d.exec("edit.undo", json!({}));
|
||||
let info = d.exec("media.colorInfo", json!({"item": item}));
|
||||
assert_eq!(info["override"], Value::Null, "{info}");
|
||||
}
|
||||
|
||||
@@ -259,6 +259,46 @@ Mix: bus sum → pre-fader inserts → fader → meter → post-fader inserts
|
||||
mix (Track Mixer and Audio Meters read them) and the newest project snapshot, which the audio
|
||||
callback uses, so edits made during playback are heard.
|
||||
|
||||
### 5.2 Colour management
|
||||
|
||||
```text
|
||||
frame (Y'CbCr/RGB + metadata) ─► source colour space: Interpret Footage override, else VUI/colr/MKV Colour
|
||||
─► decode table (sRGB/BT.709, PQ, HLG scene light, camera log) ─► HLG OOTF ─► 3×3 to working gamut
|
||||
─► BT.2390 tone map (HDR/log into an SDR sequence, Auto Tone Map Media) ─► gamut compression
|
||||
─► effects + compositing in working linear (1.0 = reference white = SDR white = 203 cd/m²)
|
||||
─► monitors: working → SDR BT.709 (tone map from HDR, gamut map from BT.2020)
|
||||
─► HDR export: working → PQ/HLG BT.2020 R'G'B' → Y'CbCr (BT.2020 NCL) + VUI/colr/mdcv/clli/SEI
|
||||
```
|
||||
|
||||
- **Model.** `SequenceSettings::color` (`ColorPipeline`: working space Rec. 709 / Rec. 2100 PQ /
|
||||
Rec. 2100 HLG, wide gamut, auto tone map) and `Interpretation::color_space` (per media item;
|
||||
`None` = from metadata). Commands: `sequence.colorSettings`, `clip.interpretFootage`,
|
||||
`color.spaces`, `media.colorInfo`.
|
||||
- **Maths** in `filmcraft-color` (`transform`, `log`, `spaces`; formulas and sources in
|
||||
[crates/color/README.md](../crates/color/README.md)); the renderer side is `render::colorman`.
|
||||
- **Fast path.** A Rec. 709 sequence without wide gamut and media whose metadata says Rec. 709 /
|
||||
sRGB decodes exactly as before, and its layers stay on the GPU. Log, HDR or wide-gamut media,
|
||||
and every layer of an HDR/wide-gamut sequence, are converted on the CPU (the GPU path draws them
|
||||
as pre-rendered images, like layers with effects).
|
||||
- **Outputs.** `RenderOptions::working_output` returns working-space pixels (HDR exports,
|
||||
scopes); otherwise the top-level render is converted for an SDR monitor. Lumetri and the other
|
||||
colour effects work on display-encoded values clamped to 0..1, so in an HDR sequence they clip
|
||||
highlights above reference white (HDR-aware grading is future work).
|
||||
- **HDR export.** H.264 and ProRes exports of a PQ/HLG sequence encode BT.2020 PQ/HLG (ProRes
|
||||
10-bit; our H.264 encoder is 8-bit, so H.264 HDR is 8-bit) and signal it in the VUI / ProRes
|
||||
frame header, `colr`, and for PQ `mdcv` (BT.2020 / D65, 1000 / 0.0001 cd/m²) and `clli`
|
||||
(MaxCLL/MaxFALL 0 = unknown, they are not measured) plus the matching H.264 SEI. `sdr: true`
|
||||
exports the tone-mapped SDR picture instead; render previews always do.
|
||||
- **Monitors / display colour management.** The monitors are SDR (sRGB-encoded RGBA8 textures):
|
||||
HDR sequences are shown tone mapped. macOS EDR (extended-range `CAMetalLayer` output) is not
|
||||
wired up: there is no `platform` crate yet and eframe/wgpu do not expose an EDR surface, so HDR
|
||||
values above SDR white are never sent to the display. The scopes of an HDR sequence show the
|
||||
working-space values (waveform in cd/m² on a PQ scale, BT.2020 vectorscope).
|
||||
- **LUTs.** Lumetri Input LUT and Creative Look reference `lib:<id>` (the project's LUT library,
|
||||
`Project::luts`, embedded `.cube`/`.3dl` text) or `builtin:<id>` (code-generated camera
|
||||
conversions and looks). `filmcraft-gpu::GpuLut` is the WGSL tetrahedral counterpart, tested for
|
||||
parity.
|
||||
|
||||
## 6. Export jobs (`filmcraft-export`)
|
||||
|
||||
```text
|
||||
|
||||
Reference in New Issue
Block a user