From 05f242c44526c8bacb9dcd1ac2c939b498558a86 Mon Sep 17 00:00:00 2001 From: arata1972 Date: Tue, 6 Oct 2026 05:36:57 +0900 Subject: [PATCH 1/5] M11: estimate missing ARW colour from guarded camera-preview correspondences Co-authored-by: Codex --- ROADMAP.md | 10 +- crates/engine/src/camera_preview.rs | 207 ++++++++++++++++++++++++++++ crates/engine/src/files.rs | 7 +- crates/engine/src/lib.rs | 1 + crates/engine/src/media.rs | 2 +- docs/camera-preview-colour.md | 31 +++++ docs/parity.md | 6 +- 7 files changed, 251 insertions(+), 13 deletions(-) create mode 100644 crates/engine/src/camera_preview.rs create mode 100644 docs/camera-preview-colour.md diff --git a/ROADMAP.md b/ROADMAP.md index 54c86e812..e36c4f3fb 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -24,7 +24,7 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78). |---|---:|---|---| | **Feature checklist** | 79% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim | | **RAW coverage** (formats people shoot) | ~50% | DNG (all kinds), CR2, ARW, NEF (uncompressed + Huffman lossless/lossy), uncompressed RAF/ORF, packed RW2, PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed RAF/ORF, RW2 v4, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. Per-model verification is thin (~40 corpus files vs >1,000 models) | -| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No camera colour calibration**: non-DNG raws use a neutral matrix, so colour is muted/inaccurate. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) | +| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW has a guarded per-file embedded-JPEG colour estimate (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) | | **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) | | **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering | | **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ | @@ -36,7 +36,7 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78). | User | Readiness | What blocks them | |---|---:|---| | JPEG / DNG shooter, single machine | ~85% | Fidelity polish, AI masks | -| Nikon / Sony / older-Canon raw shooter | ~65% | Camera colour (muted colour on every photo) | +| Nikon / Sony / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW preview estimates are only a starting point) | | Canon CR3 / Fujifilm / Olympus shooter | ~35% | Their raws open as embedded previews only | | Lightroom Classic power user | ~45% | Print, Book, Map, publish, tethering | | Relies on AI (masks, denoise) | ~25% | No segmentation or AI denoise models | @@ -46,7 +46,7 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78). Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*. 1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use - matrices the files carry themselves, then chart shots. Raises colour quality for every non-DNG raw at once. + matrices the files carry themselves, then chart shots. ARW preview-derived fitting is a guarded first step; expand coverage and validate fidelity. 2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): **CR3** first, then compressed RAF / ORF, RW2 v4, NEF lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source, no LGPL dependency); compressed NEF (#86) is the template. @@ -81,7 +81,7 @@ hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help | M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) | | M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ | | M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs ✅; JXL encode, HDR export ⬜) | -| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; **camera colour calibration** ⬜ (top priority), CR3, compressed ORF/RAF ⬜) | +| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW preview fitting; measured database still missing), CR3, compressed ORF/RAF ⬜) | | M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ | | M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ | | M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ | @@ -175,7 +175,7 @@ Not decoded yet — preview only (no permissively licensed description; black-bo from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established. - **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG. -**Camera colour matrices:** non-DNG raws use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged +**Camera colour matrices:** ARW files can use a guarded correction fitted from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate, not measured calibration. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged `matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our own chart-based calibration (M11.4). Adobe matrices are never used. diff --git a/crates/engine/src/camera_preview.rs b/crates/engine/src/camera_preview.rs new file mode 100644 index 000000000..97b33864d --- /dev/null +++ b/crates/engine/src/camera_preview.rs @@ -0,0 +1,207 @@ +//! Fit missing camera colour from a file's own colour JPEG, never from another vendor's profiles. +//! The JPEG supplies colour correspondences only: all output pixels still come from the RAW mosaic. +//! Use a fixed-size sensor proxy so thumbnails and full exports obtain the same transform. +use lightcraft_color::{Mat3, luminance_2020}; +use lightcraft_pipeline::tone::ToneMap; +use lightcraft_raster::{ + Rgb32f, + resample::{Filter, fit}, +}; +use lightcraft_raw::{RawImage, color::CameraTransform}; + +pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option { + if !transform.matrix_is_fallback || raw.format != lightcraft_raw::RawFormat::Arw { + return None; + } + let jpeg = lightcraft_raw::embedded_preview(bytes)?; + let decoded = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions { max_size: Some((384, 384)), max_pixels: 64_000_000 }).ok()?; + let reference = decoded.to_working(); + // Reject reference aspect mismatches; same-aspect crops/warps also face the holdout gate. + let crop = raw.crop.clipped(raw.active_area.width, raw.active_area.height); + if crop.width == 0 || crop.height == 0 || reference.width == 0 || reference.height == 0 { + return None; + } + if lightcraft_pipeline::profiling() { + eprintln!("[profile] camera-preview reference {}x{} raw {}x{}", reference.width, reference.height, crop.width, crop.height); + } + let aspect = crop.width as f64 / crop.height as f64; + if (reference.width as f64 / reference.height as f64 / aspect - 1.0).abs() > 0.02 { + return None; + } + let k = (crop.width.max(crop.height).div_ceil(384).max(2)).div_ceil(2) * 2; + let sensor = raw.develop_binned(k, 0.99).ok()??; + let mut sensor = fit(&sensor, 96, 96, Filter::Box); + let reference = fit(&reference, sensor.width, sensor.height, Filter::Box); + if (sensor.width, sensor.height) != (reference.width, reference.height) { + return None; + } + let gain = 2f32.powf(transform.baseline_exposure as f32); + sensor.map_in_place(|p| transform.matrix.apply_f32(std::array::from_fn(|i| p[i] * transform.wb[i] * gain))); + let matrix = fit_pairs(&sensor, &reference)?; + if lightcraft_pipeline::profiling() { + eprintln!("[profile] camera-preview colour fitted: {:?}", matrix.0); + } + Some(matrix) +} + +fn scene_reference(display: [f32; 3], tone: &ToneMap) -> Option<[f64; 3]> { + let y = luminance_2020(display); + if !display.iter().all(|v| v.is_finite() && *v > 0.004 && *v < 0.98) || !(0.015..0.85).contains(&y) { + return None; + } + // Undo the default RAW shoulder before fitting; it is applied exactly once by the normal pipeline. + let (mut lo, mut hi) = (0.0, 4.0); + for _ in 0..24 { + let mid = (lo + hi) * 0.5; + if tone.apply(mid) < y { + lo = mid; + } else { + hi = mid; + } + } + let scale = ((lo + hi) * 0.5 / y) as f64; + Some(display.map(|v| v as f64 * scale)) +} + +// Validate in linear display light, where a fixed error bound is meaningful regardless of exposure. +fn display_reference(scene: [f64; 3], tone: &ToneMap) -> Option<[f64; 3]> { + if !scene.iter().all(|v| v.is_finite()) { + return None; + } + let y = luminance_2020(scene.map(|v| v as f32)); + if y <= 0.0 || !y.is_finite() { + return None; + } + Some(scene.map(|v| v * (tone.apply(y) / y) as f64)) +} + +fn fit_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option { + if sensor.data.len() != reference.data.len() { + return None; + } + let tone = ToneMap::new(0.0, 0.0, 0.0); + let mut pairs = Vec::new(); + let mut colour = 0; + for (input, output) in sensor.data.iter().zip(&reference.data) { + if !input.iter().all(|v| v.is_finite() && *v > 0.001 && *v < 1.5) { + continue; + } + let Some(target) = scene_reference(*output, &tone) else { + continue; + }; + let min = output.iter().copied().fold(f32::INFINITY, f32::min); + let max = output.iter().copied().fold(0.0, f32::max); + colour += usize::from(max - min > 0.05); + pairs.push((input.map(f64::from), target)); + } + if lightcraft_pipeline::profiling() { + eprintln!("[profile] calibration pairs {} colour {}", pairs.len(), colour); + } + if pairs.len() < 256 || colour < pairs.len() / 20 { + return None; + } + let mut gram = [[0.0; 3]; 3]; + let mut cross = [[0.0; 3]; 3]; + for (i, (x, y)) in pairs.iter().enumerate() { + if i % 3 == 0 { + continue; + } + for row in 0..3 { + for col in 0..3 { + gram[row][col] += x[row] * x[col]; + cross[row][col] += y[row] * x[col]; + } + } + } + let trace: f64 = (0..3).map(|i| gram[i][i]).sum(); + // Reject insufficient colour diversity; regularization must not invent a calibration. + let g = Mat3(gram); + let inverse = g.inverse()?; + let condition = trace * (0..3).map(|i| inverse.0[i][i].abs()).sum::(); + if !inverse.0.iter().flatten().all(|v| v.is_finite()) || trace <= 0.0 || !condition.is_finite() || condition > 1e6 { + return None; + } + let regularization = trace * 1e-3; + let exposure = (0..3).map(|i| cross[i][i]).sum::() / trace; + for i in 0..3 { + gram[i][i] += regularization; + cross[i][i] += regularization * exposure; + } + let matrix = Mat3(cross).mul(&Mat3(gram).inverse()?); + if lightcraft_pipeline::profiling() { + eprintln!("[profile] calibration candidate {:?}", matrix.0); + } + if !matrix.0.iter().flatten().all(|v| v.is_finite() && v.abs() < 8.0) { + return None; + } + let mut before = 0.0; + let mut after = 0.0; + let mut samples = 0; + for (i, (x, y)) in pairs.iter().enumerate() { + if i % 3 != 0 { + continue; + } + let corrected = display_reference(matrix.apply(*x), &tone)?; + let original = display_reference(*x, &tone)?; + let target = display_reference(*y, &tone)?; + for c in 0..3 { + before += (original[c] - target[c]).powi(2); + after += (corrected[c] - target[c]).powi(2); + samples += 1; + } + } + // Independent held-out pixels must materially improve, and match within a bounded error. + if lightcraft_pipeline::profiling() { + eprintln!("[profile] calibration holdout before {} after {} mean {}", before, after, after / samples as f64); + } + if !after.is_finite() || after >= before * 0.7 || after / samples as f64 > 0.075f64.powi(2) { + return None; + } + Some(matrix) +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn recovers_missing_colour_and_exposure_without_replacing_sensor_pixels() { + let tone = ToneMap::new(0.0, 0.0, 0.0); + let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]); + let mut sensor = Rgb32f::new(32, 32); + let mut reference = sensor.clone(); + for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() { + *src = [0.05 + (i % 11) as f32 * 0.018, 0.05 + (i % 17) as f32 * 0.011, 0.06 + (i % 23) as f32 * 0.008]; + let p = known.apply_f32(*src); + let y = luminance_2020(p); + *dst = p.map(|v| v * tone.apply(y) / y); + } + let original = sensor.clone(); + let fitted = fit_pairs(&sensor, &reference).unwrap(); + assert_eq!(sensor.data, original.data); + for (a, b) in fitted.0.iter().flatten().zip(known.0.iter().flatten()) { + assert!((a - b).abs() < 0.15, "{fitted:?}"); + } + } + #[test] + fn rejects_monochrome_invalid_and_unrelated_previews() { + let mut sensor = Rgb32f::new(32, 32); + let mut reference = sensor.clone(); + for (i, p) in sensor.data.iter_mut().enumerate() { + *p = [0.1 + (i % 13) as f32 * 0.02, 0.15, 0.1]; + } + reference.data.fill([0.2; 3]); + assert!(fit_pairs(&sensor, &reference).is_none()); + reference.data.fill([f32::NAN; 3]); + assert!(fit_pairs(&sensor, &reference).is_none()); + for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() { + *src = [0.04 + (i % 11) as f32 * 0.02, 0.05 + (i % 17) as f32 * 0.01, 0.03 + (i % 23) as f32 * 0.01]; + *dst = [0.05 + (i % 7) as f32 * 0.07, 0.05 + (i % 19) as f32 * 0.02, 0.05 + (i % 29) as f32 * 0.01]; + } + assert!(fit_pairs(&sensor, &reference).is_none()); + // A constant coloured sensor cannot determine a three-dimensional colour transform. + sensor.data.fill([0.1, 0.15, 0.12]); + assert!(fit_pairs(&sensor, &reference).is_none()); + reference.data.truncate(8); + assert!(fit_pairs(&sensor, &reference).is_none()); + } +} diff --git a/crates/engine/src/files.rs b/crates/engine/src/files.rs index 32296c5af..3a9f42c8a 100644 --- a/crates/engine/src/files.rs +++ b/crates/engine/src/files.rs @@ -221,6 +221,9 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result< } Err(e) => return Err(e.to_string()), }; + let xy = lightcraft_raw::color::as_shot_white_xy(&raw); + let t = lightcraft_raw::color::camera_transform(&raw, xy); + let preview_matrix = crate::camera_preview::fit_preview(&raw, &bytes, &t); drop(bytes); // Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in. let lens = embedded_lens(&raw.info()); @@ -242,12 +245,10 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result< // the samples aren't needed any more (the colour model below reads only the tags) raw.data = lightcraft_raw::RawData::U16(Vec::new()); let mut stages = vec![("develop", t0.elapsed())]; - let xy = lightcraft_raw::color::as_shot_white_xy(&raw); - let t = lightcraft_raw::color::camera_transform(&raw, xy); stages.push(("transform", t0.elapsed())); lightcraft_raw::highlight::reconstruct(&mut img, t.wb, HIGHLIGHT_CLIP); stages.push(("highlights", t0.elapsed())); - let m = t.matrix.to_f32(); + let m = preview_matrix.map(|p| p.mul(&t.matrix)).unwrap_or(t.matrix).to_f32(); let gain = 2f32.powf(t.baseline_exposure as f32); let wb = t.wb; img.map_in_place(|p| { diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index dcbfd9edc..9d05b855e 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -10,6 +10,7 @@ //! run them off the UI thread. #![forbid(unsafe_code)] +mod camera_preview; pub mod cmd; pub mod crs; pub mod crs_masks; diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rs index 4841533b6..96d25932c 100644 --- a/crates/engine/src/media.rs +++ b/crates/engine/src/media.rs @@ -26,7 +26,7 @@ use lightcraft_raster::{Histogram, Rgb32f, Rgba8}; use serde::{Deserialize, Serialize}; /// Bump when the pipeline's output changes, to invalidate cached thumbnails. -pub const RENDER_CACHE_VERSION: u64 = 4; +pub const RENDER_CACHE_VERSION: u64 = 5; /// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache). pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512]; diff --git a/docs/camera-preview-colour.md b/docs/camera-preview-colour.md new file mode 100644 index 000000000..f10a53820 --- /dev/null +++ b/docs/camera-preview-colour.md @@ -0,0 +1,31 @@ +# ARW preview-derived colour starting point + +Sony ARW files with no usable camera colour matrix previously used the generic camera-RGB≈sRGB fallback. This can produce visibly muted, dark photographs. The engine now estimates a bounded 3×3 correction from a file's own embedded camera JPEG. The JPEG provides colour correspondences, not output pixels: decoding, demosaicing, white balance, highlight reconstruction and RAW editing still operate on the sensor data. + +A fixed sensor proxy (at most 96×96) is used at every requested output size. The reference is decoded with DCT scaling and a 64-million-pixel input limit. Reference and sensor aspect ratios must agree within 2%; references with a different aspect ratio are rejected. Near-black, clipped and nonfinite samples are excluded. At least 256 usable pairs, enough coloured samples and a nonsingular, bounded-condition input covariance are required. Ridge regression is biased towards an exposure-only correction, rather than unconstrained colour amplification. Matrix coefficients are bounded. + +One third of the usable pixels are held out from fitting. After the normal RAW tone shoulder, the correction must reduce linear-display squared error by at least 30%, with per-channel RMS error at most 0.075 (a provisional camera-look tolerance, not a colour-accuracy certification). Unusable, monochrome, mismatched, ill-conditioned or poor-fitting references retain the existing fallback. DNG matrices and other RAW formats are unchanged. Render cache version 5 invalidates previously rendered thumbnails and views. + +This is a per-file camera-look estimate, not a measured per-camera spectral calibration. The embedded JPEG includes camera tone curves, picture styles, noise reduction and possibly local/lens adjustments; a global matrix cannot reproduce all of them. Similar-sized cropped or warped references are not guaranteed to be detected. Skin colour and highlights can still differ. A camera-matrix database and chart-based verification remain necessary. Do not advertise this as Lightroom colour parity or complete camera coverage. + +Validation uses synthetic coloured scenes with a known matrix, monochrome/nonfinite/singular inputs, unrelated references and mismatched lengths. Five private Sony ILCE-6700 ARWs are checked locally; photographs and embedded JPEGs are not committed or uploaded. See the PR for the observed acceptance count and remaining differences. + +## Local observations (Sony ILCE-6700) + +The guarded estimate was accepted on all five private samples. Brightness and yellow clothing improved; skin still differs from the camera JPEG. This is why the change is proposed as a draft for colour-science review and broader camera verification. + +A camera JPEG from the same capture, the previous RAW renderer and the fitted RAW renderer were exported at 1200 px with neutral settings and separately with Exposure +1 EV, Saturation +100 and Contrast +50. All three controls changed the previous RAW output. Mean absolute encoded-RGB differences from each source’s neutral rendering were: + +| Source | Exposure +1 | Saturation +100 | Contrast +50 | +|---|---:|---:|---:| +| Camera JPEG | 0.1930 | 0.0642 | 0.0373 | +| RAW, generic fallback | 0.1399 | 0.0318 | 0.0315 | +| RAW, preview-derived estimate | 0.1628 | 0.0509 | 0.0706 | + +These are response diagnostics on one image, not colour-accuracy or Lightroom-parity scores. The weaker saturation response starts from an already muted rendering; increasing saturation does not replace missing camera colour characterization. JPEG uses an identity tone map at neutral settings; RAW uses the scene-to-display shoulder, so identical slider values are not expected to produce identical changes. Edited RAW was also checked in the headless UI with the actual GPU renderer, after waiting for pending jobs: the loupe reported `source: render`, the histogram and displayed pixels changed, and Exposure +1 / Saturation +100 remained in the controls. No slider-no-op was reproduced in this sample. + +A second comparison used five separately supplied same-capture JPEGs, including four with embedded Display P3 profiles. RAW-only changes left all five JPEG exports pixel-identical. Two supplied JPEGs have a narrower crop than the RAW; no pixel-by-pixel colour-accuracy claim is made across those different frames. On the supplied matching JPEG for the slider-response sample, the corresponding response figures were exposure 0.1952, saturation 0.0517 and contrast 0.0260. RAW saturation response increased from 0.0318 to 0.0509 with the estimate. + +**Unresolved white-balance limitation:** the generic camera model also derives the RAW as-shot temperature/tint without a measured camera matrix. The sample reports roughly 6829 K / −127 tint. Changing temperature alone to 9000 K causes a strong pink cast, even after preview-derived colour fitting. The estimate does not establish a physically calibrated illuminant or repair absolute-Kelvin WB behaviour. It must not be presented as a complete RAW-development fix; measured camera characterization and white-balance validation are still required. + +For the colour-fit sample, independent CLI renders at 400 px and full resolution selected exactly the same fitted matrix. Full cold render/export took 3.96 s; 400 px took 0.48 s on this local machine (includes process/import/render/encode, not just the adjustment pipeline). diff --git a/docs/parity.md b/docs/parity.md index 2f24d5995..fda380c4c 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -59,9 +59,7 @@ Ordered by user impact, then tier, then effort. The checklist above counts featu that decide whether a photographer can switch (see the honest assessment in [ROADMAP.md](../ROADMAP.md#where-we-stand)). Take the first one nobody is working on. -1. **LR-PROF-CAMERACOLOR** (P0): our own camera colour calibration. Every non-DNG raw is developed with a neutral - matrix today, so colours are muted. Start by fitting each camera to its own embedded JPEG and using matrices the - files carry themselves; never Adobe data. +1. **LR-PROF-CAMERACOLOR** (P0): our own camera colour calibration. ARW now has guarded per-file embedded-JPEG fitting; measured calibration and fidelity remain missing. Other non-DNG raws and rejected fits still use a neutral matrix. Expand validated preview fitting and use matrices the files carry themselves; never Adobe data. 2. **LR-IMP-FORMATS** (P0): **CR3** first (every Canon body since ~2018), then compressed RAF / ORF, RW2 v4, Nikon "lossy after split" NEF, Canon sRAW; HEIC/AVIF decode. Clean-room, from prose descriptions only (see `crates/raw/src/vendor/nefc.rs` for how compressed NEF was done). Until decoded, such photos are `preview_only`. @@ -276,7 +274,7 @@ under the UI command that wraps it (a no-op error); the UI command now wins. | LR-PROF-ADOBE | Standard raw looks (own equivalents) | P0 | ✅ | `crates/engine/src/presets.rs` (`PROFILES`), `crates/pipeline/src/profiles.rs` | six own looks: Color, Neutral, Vivid, Landscape, Portrait, Monochrome | | LR-PROF-ADAPTIVE | Adaptive profiles | P2 | ⬜ | | | | LR-PROF-CAMERA | Camera-matching looks | P2 | ⬜ | | | -| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs` | DNG files use the colour matrices they carry; every other raw uses a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | +| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices they carry; ARW has guarded per-file embedded-JPEG colour fitting (see docs/camera-preview-colour.md), not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | | LR-PROF-CREATIVE | Creative profiles (own) | P2 | ✅ | `cmd:develop.profile`, `crates/pipeline/src/profiles.rs` | 16 own looks in Film / Cinematic / Muted / B&W (tone + point-curve fades, colour grading, mixer / B&W mix); scale with `ctl:profile.amount`; sliders untouched | | LR-PROF-LEGACY | Legacy profiles | P2 | ⬜ | | | | LR-PROF-NONRAW | Profiles for non-raw files | P0 | ✅ | `cmd:develop.profile` | same looks apply to JPEG/TIFF | From 43f4e24355b596a2402c48d6a0a00e98174ee439 Mon Sep 17 00:00:00 2001 From: arata1972 Date: Tue, 6 Oct 2026 07:16:15 +0900 Subject: [PATCH 2/5] M11: separate Sony ARW colour and tone and preserve decoded source facts Co-authored-by: Codex --- ROADMAP.md | 2 +- apps/lightcraft-web/src/wire.rs | 34 +++-- crates/catalog/src/model.rs | 8 +- crates/engine/src/camera_preview.rs | 197 ++++++++++++++++------------ crates/engine/src/cmd/color.rs | 2 +- crates/engine/src/cmd/develop.rs | 50 +++++-- crates/engine/src/cmd/masks.rs | 4 +- crates/engine/src/files.rs | 15 ++- crates/engine/src/media.rs | 143 +++++++++++++++----- crates/engine/src/tests.rs | 24 ++++ crates/gpu/tests/equivalence.rs | 21 +++ crates/pipeline/src/finish.rs | 4 +- crates/pipeline/src/lib.rs | 5 +- crates/pipeline/src/local.rs | 2 +- crates/pipeline/src/tone.rs | 89 +++++++++++++ crates/ui-egui/src/panels/edit.rs | 8 +- docs/camera-preview-colour.md | 36 ++--- docs/parity.md | 4 +- 18 files changed, 479 insertions(+), 169 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index e36c4f3fb..bb0011efa 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -175,7 +175,7 @@ Not decoded yet — preview only (no permissively licensed description; black-bo from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established. - **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG. -**Camera colour matrices:** ARW files can use a guarded correction fitted from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate, not measured calibration. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged +**Camera colour matrices:** ARW files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged `matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our own chart-based calibration (M11.4). Adobe matrices are never used. diff --git a/apps/lightcraft-web/src/wire.rs b/apps/lightcraft-web/src/wire.rs index 2e546eb23..71d9283a4 100644 --- a/apps/lightcraft-web/src/wire.rs +++ b/apps/lightcraft-web/src/wire.rs @@ -15,10 +15,9 @@ use std::sync::Arc; use lightcraft_engine::catalog::Source; use lightcraft_engine::develop::{DevelopSettings, EmbeddedLens}; -use lightcraft_engine::media::{MediaCache, RenderJob, SourceLevel, SourceRef}; +use lightcraft_engine::media::{DecodedSource, MediaCache, RenderJob, SourceLevel, SourceRef}; use lightcraft_engine::pipeline::{Quality, RenderRequest, Rendered, SourceInfo, StageCache}; use lightcraft_preview::Lru; -use lightcraft_raster::Rgb32f; use serde::{Deserialize, Serialize}; use crate::store::hash_of_path; @@ -31,6 +30,8 @@ pub struct WireJob { /// Decode the source at most this long. pub max_edge: usize, pub raw: bool, + #[serde(default)] + pub relative_wb: bool, pub as_shot_temp: f64, pub as_shot_tint: f64, pub lens: Option, @@ -55,7 +56,7 @@ impl WireJob { pub fn from_job(job: &RenderJob, thumb_cached: bool, view: &str) -> WireJob { let max_edge = match &job.source { SourceRef::Demo { max_edge, .. } | SourceRef::File { max_edge, .. } => *max_edge, - SourceRef::Loaded(img) => img.width.max(img.height), + SourceRef::Loaded(img) => img.image.width.max(img.image.height), SourceRef::Smart { .. } => 2560, }; WireJob { @@ -63,6 +64,7 @@ impl WireJob { origin: job.origin.clone(), max_edge, raw: job.info.raw, + relative_wb: job.info.relative_wb, as_shot_temp: job.info.as_shot_temp, as_shot_tint: job.info.as_shot_tint, lens: job.info.lens, @@ -79,7 +81,14 @@ impl WireJob { } pub fn info(&self) -> SourceInfo { - SourceInfo { raw: self.raw, as_shot_temp: self.as_shot_temp, as_shot_tint: self.as_shot_tint, lens: self.lens } + SourceInfo { + raw: self.raw, + as_shot_temp: self.as_shot_temp, + as_shot_tint: self.as_shot_tint, + lens: self.lens, + relative_wb: self.relative_wb, + ..Default::default() + } } pub fn request(&self) -> RenderRequest { @@ -119,7 +128,7 @@ const WORKER_SOURCE_BYTES: usize = 192 << 20; /// The synchronous part of a worker: decoded-source cache + rendering. pub struct WorkerCore { - sources: Lru>, + sources: Lru, media: MediaCache, stages: HashMap>, } @@ -150,21 +159,25 @@ impl WorkerCore { Some(s) => s, None => { let src = match (&job.origin, original) { - (Source::File { .. }, Some(bytes)) => Arc::new(lightcraft_engine::files::load_bytes(bytes, job.max_edge)?.0), + (Source::File { .. }, Some(bytes)) => { + let (image, info) = lightcraft_engine::files::load_bytes(bytes, job.max_edge)?; + DecodedSource { image: Arc::new(image), info: Some(info) } + } (Source::File { path }, None) => return Err(format!("{path}: original not found in browser storage")), - (origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load()?, + (origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load_source()?, }; - let cost = src.width * src.height * 12 + 64; + let cost = src.image.width * src.image.height * 12 + 64; self.sources.insert(key, src.clone(), cost); src } }; + let info = src.info.unwrap_or(job.info()); Ok(match &job.stages { Some(view) => { let st = self.stages.entry(view.clone()).or_default().clone(); - lightcraft_engine::pipeline::render_cached(&src, &job.info(), &job.settings, &job.request(), &st) + lightcraft_engine::pipeline::render_cached(&src.image, &info, &job.settings, &job.request(), &st) } - None => lightcraft_engine::pipeline::render(&src, &job.info(), &job.settings, &job.request()), + None => lightcraft_engine::pipeline::render(&src.image, &info, &job.settings, &job.request()), }) } } @@ -295,6 +308,7 @@ mod tests { origin, max_edge: 512, raw: false, + relative_wb: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None, diff --git a/crates/catalog/src/model.rs b/crates/catalog/src/model.rs index f8e518464..8b400bb52 100644 --- a/crates/catalog/src/model.rs +++ b/crates/catalog/src/model.rs @@ -331,6 +331,11 @@ impl Photo { pub fn develops_raw(&self) -> bool { self.kind == MediaKind::Raw && self.preview_only.is_none() } + /// The current ARW reader has vendor WB multipliers but no measured camera illuminant. + /// Use adjustments relative to the camera's as-shot look, as for rendered photographs. + pub fn relative_wb(&self) -> bool { + self.develops_raw() && self.format.eq_ignore_ascii_case("ARW") + } /// The develop settings import gave this photo: [`Photo::camera_defaults`], or the user's /// default preset applied on top of them ([`Photo::import_look`]). pub fn import_defaults(&self) -> DevelopSettings { @@ -342,7 +347,8 @@ impl Photo { /// The built-in defaults for this photo, before any user default preset: raws start from /// their as-shot white balance; embedded lens corrections on when the file has them. pub fn camera_defaults(&self) -> DevelopSettings { - let mut d = match self.as_shot_wb { + let wb = if self.relative_wb() { Some((6500.0, 0.0)) } else { self.as_shot_wb }; + let mut d = match wb { Some((t, tint)) => DevelopSettings::for_raw(t, tint), None => DevelopSettings::default(), }; diff --git a/crates/engine/src/camera_preview.rs b/crates/engine/src/camera_preview.rs index 97b33864d..4a550630b 100644 --- a/crates/engine/src/camera_preview.rs +++ b/crates/engine/src/camera_preview.rs @@ -1,101 +1,79 @@ -//! Fit missing camera colour from a file's own colour JPEG, never from another vendor's profiles. -//! The JPEG supplies colour correspondences only: all output pixels still come from the RAW mosaic. -//! Use a fixed-size sensor proxy so thumbnails and full exports obtain the same transform. +//! Estimate an ARW starting look from its own JPEG. Colour and luminance are fitted separately; +//! the JPEG supplies correspondences only, never output pixels or a replacement for RAW editing. use lightcraft_color::{Mat3, luminance_2020}; -use lightcraft_pipeline::tone::ToneMap; +use lightcraft_pipeline::tone::{CameraTone, ToneMap}; use lightcraft_raster::{ Rgb32f, resample::{Filter, fit}, }; use lightcraft_raw::{RawImage, color::CameraTransform}; -pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option { +#[derive(Clone, Copy, Debug)] +pub(crate) struct CameraLook { + pub matrix: Mat3, + pub tone: CameraTone, +} + +pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option { if !transform.matrix_is_fallback || raw.format != lightcraft_raw::RawFormat::Arw { return None; } let jpeg = lightcraft_raw::embedded_preview(bytes)?; let decoded = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions { max_size: Some((384, 384)), max_pixels: 64_000_000 }).ok()?; let reference = decoded.to_working(); - // Reject reference aspect mismatches; same-aspect crops/warps also face the holdout gate. let crop = raw.crop.clipped(raw.active_area.width, raw.active_area.height); if crop.width == 0 || crop.height == 0 || reference.width == 0 || reference.height == 0 { return None; } - if lightcraft_pipeline::profiling() { - eprintln!("[profile] camera-preview reference {}x{} raw {}x{}", reference.width, reference.height, crop.width, crop.height); - } let aspect = crop.width as f64 / crop.height as f64; if (reference.width as f64 / reference.height as f64 / aspect - 1.0).abs() > 0.02 { return None; } + // Fixed, bounded proxy: the selected look cannot depend on thumbnail/export resolution. let k = (crop.width.max(crop.height).div_ceil(384).max(2)).div_ceil(2) * 2; let sensor = raw.develop_binned(k, 0.99).ok()??; let mut sensor = fit(&sensor, 96, 96, Filter::Box); let reference = fit(&reference, sensor.width, sensor.height, Filter::Box); - if (sensor.width, sensor.height) != (reference.width, reference.height) { - return None; - } let gain = 2f32.powf(transform.baseline_exposure as f32); sensor.map_in_place(|p| transform.matrix.apply_f32(std::array::from_fn(|i| p[i] * transform.wb[i] * gain))); - let matrix = fit_pairs(&sensor, &reference)?; + let look = fit_pairs(&sensor, &reference)?; if lightcraft_pipeline::profiling() { - eprintln!("[profile] camera-preview colour fitted: {:?}", matrix.0); + eprintln!("[profile] ARW camera look: {:?}, {:?}", look.matrix.0, look.tone); } - Some(matrix) + Some(look) } -fn scene_reference(display: [f32; 3], tone: &ToneMap) -> Option<[f64; 3]> { - let y = luminance_2020(display); - if !display.iter().all(|v| v.is_finite() && *v > 0.004 && *v < 0.98) || !(0.015..0.85).contains(&y) { - return None; - } - // Undo the default RAW shoulder before fitting; it is applied exactly once by the normal pipeline. - let (mut lo, mut hi) = (0.0, 4.0); - for _ in 0..24 { - let mid = (lo + hi) * 0.5; - if tone.apply(mid) < y { - lo = mid; - } else { - hi = mid; - } - } - let scale = ((lo + hi) * 0.5 / y) as f64; - Some(display.map(|v| v as f64 * scale)) +fn luma(p: [f64; 3]) -> f64 { + p[0] * 0.2627 + p[1] * 0.6780 + p[2] * 0.0593 } -// Validate in linear display light, where a fixed error bound is meaningful regardless of exposure. -fn display_reference(scene: [f64; 3], tone: &ToneMap) -> Option<[f64; 3]> { - if !scene.iter().all(|v| v.is_finite()) { - return None; +fn displayed(scene: [f64; 3], tone: &ToneMap) -> [f64; 3] { + let scene = scene.map(|v| v.max(0.0)); + let y = luma(scene); + if y <= 0.0 { + return [0.0; 3]; } - let y = luminance_2020(scene.map(|v| v as f32)); - if y <= 0.0 || !y.is_finite() { - return None; - } - Some(scene.map(|v| v * (tone.apply(y) / y) as f64)) + scene.map(|v| v * f64::from(tone.apply(y as f32)) / y) } -fn fit_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option { - if sensor.data.len() != reference.data.len() { +fn fit_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option { + if (sensor.width, sensor.height) != (reference.width, reference.height) || sensor.data.len() != reference.data.len() { return None; } - let tone = ToneMap::new(0.0, 0.0, 0.0); let mut pairs = Vec::new(); let mut colour = 0; for (input, output) in sensor.data.iter().zip(&reference.data) { - if !input.iter().all(|v| v.is_finite() && *v > 0.001 && *v < 1.5) { + let y = luminance_2020(*output); + if !input.iter().all(|v| v.is_finite() && *v > 0.001 && *v < 1.5) + || !output.iter().all(|v| v.is_finite() && *v > 0.004 && *v < 0.98) + || !(0.015..0.85).contains(&y) + { continue; } - let Some(target) = scene_reference(*output, &tone) else { - continue; - }; let min = output.iter().copied().fold(f32::INFINITY, f32::min); let max = output.iter().copied().fold(0.0, f32::max); colour += usize::from(max - min > 0.05); - pairs.push((input.map(f64::from), target)); - } - if lightcraft_pipeline::profiling() { - eprintln!("[profile] calibration pairs {} colour {}", pairs.len(), colour); + pairs.push((input.map(f64::from), output.map(f64::from))); } if pairs.len() < 256 || colour < pairs.len() / 20 { return None; @@ -106,81 +84,137 @@ fn fit_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option { if i % 3 == 0 { continue; } + let (lx, ly) = (luma(*x), luma(*y)); for row in 0..3 { for col in 0..3 { - gram[row][col] += x[row] * x[col]; - cross[row][col] += y[row] * x[col]; + // Normalising by luminance prevents a camera S-curve from corrupting colour. + gram[row][col] += x[row] * x[col] / (lx * lx); + cross[row][col] += y[row] * x[col] / (ly * lx); } } } let trace: f64 = (0..3).map(|i| gram[i][i]).sum(); - // Reject insufficient colour diversity; regularization must not invent a calibration. - let g = Mat3(gram); - let inverse = g.inverse()?; + let inverse = Mat3(gram).inverse()?; let condition = trace * (0..3).map(|i| inverse.0[i][i].abs()).sum::(); - if !inverse.0.iter().flatten().all(|v| v.is_finite()) || trace <= 0.0 || !condition.is_finite() || condition > 1e6 { + if trace <= 0.0 || !condition.is_finite() || condition > 1e6 { return None; } - let regularization = trace * 1e-3; - let exposure = (0..3).map(|i| cross[i][i]).sum::() / trace; + let regularization = trace * 1e-4; for i in 0..3 { gram[i][i] += regularization; - cross[i][i] += regularization * exposure; + cross[i][i] += regularization; } let matrix = Mat3(cross).mul(&Mat3(gram).inverse()?); - if lightcraft_pipeline::profiling() { - eprintln!("[profile] calibration candidate {:?}", matrix.0); - } if !matrix.0.iter().flatten().all(|v| v.is_finite() && v.abs() < 8.0) { return None; } + let tone_pairs: Vec<_> = + pairs.iter().enumerate().filter(|(i, _)| i % 3 != 0).map(|(_, (x, y))| (luma(matrix.apply(*x).map(|v| v.max(0.0))), luma(*y))).collect(); + let curve = fit_tone(tone_pairs)?; + let tone = ToneMap::camera(&curve, 0.0, 0.0, 0.0); + let original_tone = ToneMap::new(0.0, 0.0, 0.0); let mut before = 0.0; let mut after = 0.0; let mut samples = 0; - for (i, (x, y)) in pairs.iter().enumerate() { + for (i, (x, target)) in pairs.iter().enumerate() { if i % 3 != 0 { continue; } - let corrected = display_reference(matrix.apply(*x), &tone)?; - let original = display_reference(*x, &tone)?; - let target = display_reference(*y, &tone)?; + let corrected = displayed(matrix.apply(*x), &tone); + let original = displayed(*x, &original_tone); for c in 0..3 { before += (original[c] - target[c]).powi(2); after += (corrected[c] - target[c]).powi(2); samples += 1; } } - // Independent held-out pixels must materially improve, and match within a bounded error. if lightcraft_pipeline::profiling() { - eprintln!("[profile] calibration holdout before {} after {} mean {}", before, after, after / samples as f64); + eprintln!("[profile] ARW holdout RMS {:.5} -> {:.5} ({samples} channels)", (before / samples as f64).sqrt(), (after / samples as f64).sqrt()); } - if !after.is_finite() || after >= before * 0.7 || after / samples as f64 > 0.075f64.powi(2) { + if !after.is_finite() || samples == 0 || after >= before * 0.7 || after / samples as f64 > 0.055f64.powi(2) { return None; } - Some(matrix) + Some(CameraLook { matrix, tone: curve }) +} + +fn median(values: &mut [f64]) -> Option { + values.sort_by(f64::total_cmp); + values.get(values.len() / 2).copied() +} + +fn fit_tone(mut pairs: Vec<(f64, f64)>) -> Option { + if pairs.len() < 128 || !pairs.iter().all(|(x, y)| x.is_finite() && *x > 0.0 && y.is_finite()) { + return None; + } + pairs.sort_by(|a, b| a.0.total_cmp(&b.0)); + let mut knots = [[0.0; 2]; 32]; + for (i, knot) in knots.iter_mut().enumerate() { + let bin = pairs.get(i * pairs.len() / 32..(i + 1) * pairs.len() / 32)?; + let mut xs: Vec<_> = bin.iter().map(|p| p.0).collect(); + let mut ys: Vec<_> = bin.iter().map(|p| p.1).collect(); + *knot = [median(&mut xs)? as f32, median(&mut ys)? as f32]; + } + if knots[31][0] < knots[0][0] * 1.5 { + return None; + } + // Pool adjacent violating bins (isotonic regression): no reversals or arbitrary polynomial. + let mut blocks: Vec<(f32, usize)> = Vec::new(); + for knot in knots { + blocks.push((knot[1], 1)); + while blocks.len() >= 2 { + let (a, an) = *blocks.get(blocks.len() - 2)?; + let (b, bn) = *blocks.last()?; + if a <= b { + break; + } + blocks.truncate(blocks.len() - 2); + blocks.push(((a * an as f32 + b * bn as f32) / (an + bn) as f32, an + bn)); + } + } + let mut i = 0; + for (y, n) in blocks { + for knot in knots.get_mut(i..i + n)? { + knot[1] = y; + } + i += n; + } + CameraTone::new(knots) } #[cfg(test)] mod tests { use super::*; #[test] - fn recovers_missing_colour_and_exposure_without_replacing_sensor_pixels() { - let tone = ToneMap::new(0.0, 0.0, 0.0); + fn separates_nonlinear_tone_from_colour_and_keeps_sensor_headroom() { let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]); - let mut sensor = Rgb32f::new(32, 32); + let mut sensor = Rgb32f::new(64, 64); let mut reference = sensor.clone(); for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() { - *src = [0.05 + (i % 11) as f32 * 0.018, 0.05 + (i % 17) as f32 * 0.011, 0.06 + (i % 23) as f32 * 0.008]; + let ev = 0.05 + (i % 31) as f32 * 0.017; + *src = [ev * (0.8 + (i % 11) as f32 * 0.025), ev, ev * (0.8 + (i % 17) as f32 * 0.014)]; let p = known.apply_f32(*src); let y = luminance_2020(p); - *dst = p.map(|v| v * tone.apply(y) / y); + *dst = p.map(|v| v * (1.0 - (-2.5 * y).exp()) / y); } let original = sensor.clone(); - let fitted = fit_pairs(&sensor, &reference).unwrap(); + let fit = fit_pairs(&sensor, &reference).unwrap(); assert_eq!(sensor.data, original.data); - for (a, b) in fitted.0.iter().flatten().zip(known.0.iter().flatten()) { - assert!((a - b).abs() < 0.15, "{fitted:?}"); - } + let tone = ToneMap::camera(&fit.tone, 0.0, 0.0, 0.0); + let error: f64 = sensor + .data + .iter() + .zip(&reference.data) + .map(|(x, y)| { + let p = displayed(fit.matrix.apply(x.map(f64::from)), &tone); + (0..3).map(|c| (p[c] - f64::from(y[c])).powi(2)).sum::() / 3.0 + }) + .sum::() + / sensor.data.len() as f64; + assert!(error.sqrt() < 0.025, "{error}"); + // The colour transform is homogeneous; tone mapping happens only after exposure. + let p = fit.matrix.apply([2.0, 2.0, 2.0]); + assert!(luma(p) > 1.0); + assert!(tone.apply(0.2) < tone.apply(0.4)); } #[test] fn rejects_monochrome_invalid_and_unrelated_previews() { @@ -198,7 +232,6 @@ mod tests { *dst = [0.05 + (i % 7) as f32 * 0.07, 0.05 + (i % 19) as f32 * 0.02, 0.05 + (i % 29) as f32 * 0.01]; } assert!(fit_pairs(&sensor, &reference).is_none()); - // A constant coloured sensor cannot determine a three-dimensional colour transform. sensor.data.fill([0.1, 0.15, 0.12]); assert!(fit_pairs(&sensor, &reference).is_none()); reference.data.truncate(8); diff --git a/crates/engine/src/cmd/color.rs b/crates/engine/src/cmd/color.rs index 65a9a709e..55ea8b5ab 100644 --- a/crates/engine/src/cmd/color.rs +++ b/crates/engine/src/cmd/color.rs @@ -26,7 +26,7 @@ const PROBE_EDGE: usize = 384; pub(crate) fn probe(s: &mut Session, c: &str, x: f64, y: f64, neutral: impl FnOnce(&mut DevelopSettings)) -> Result<[f32; 3]> { let id = s.active().ok_or_else(|| bad(c, "no active photo"))?; let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad(c, e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let mut d = (*s.develop_of(id).unwrap_or_default()).clone(); neutral(&mut d); let req = lightcraft_pipeline::RenderRequest::fit(PROBE_EDGE, PROBE_EDGE); diff --git a/crates/engine/src/cmd/develop.rs b/crates/engine/src/cmd/develop.rs index 4d955ac5b..8f197429c 100644 --- a/crates/engine/src/cmd/develop.rs +++ b/crates/engine/src/cmd/develop.rs @@ -98,7 +98,11 @@ pub fn specs() -> Vec { } } let label = if vals.len() == 1 { controls::find(&vals[0].0).map(|c| c.label).unwrap_or("Edit").to_string() } else { "Edit".into() }; - let apply = |d: &mut DevelopSettings| { + let apply = |d: &mut DevelopSettings, info: &lightcraft_pipeline::SourceInfo| { + if d.wb.mode == WbMode::AsShot && vals.iter().any(|(k, _)| k == "wb.temp" || k == "wb.tint") { + d.wb.temp = info.as_shot_temp; + d.wb.tint = info.as_shot_tint; + } for (k, v) in &vals { controls::set(d, k, *v); if k == "wb.temp" || k == "wb.tint" { @@ -113,15 +117,18 @@ pub fn specs() -> Vec { .filter_map(|id| s.develop_of(*id).map(|d| (*id, d))) .filter_map(|(id, d)| { let mut d = (*d).clone(); - apply(&mut d); + let info = s.source_info(id); + apply(&mut d, &info); s.develop_op(id, d, &label) }) .collect(); s.commit(&label, Op::Batch { ops })?; return ok(); } + let id = active(s, "develop.set")?; + let info = s.source_info(id); edit(s, "develop.set", &label, |d| { - apply(d); + apply(d, &info); Ok(()) }) } @@ -130,7 +137,15 @@ pub fn specs() -> Vec { let c = str_param(p, "control").ok_or_else(|| bad("develop.adjust", "missing control"))?.to_string(); let spec = controls::find(&c).ok_or_else(|| bad("develop.adjust", "unknown control"))?; let delta = f64_req(p, "delta", "develop.adjust")?; + let info = s.source_info(active(s, "develop.adjust")?); edit(s, "develop.adjust", spec.label, |d| { + if c == "wb.temp" || c == "wb.tint" { + if d.wb.mode == WbMode::AsShot { + d.wb.temp = info.as_shot_temp; + d.wb.tint = info.as_shot_tint; + } + d.wb.mode = WbMode::Custom; + } let v = controls::get(d, &c).unwrap_or(spec.default); controls::set(d, &c, v + delta); Ok(()) @@ -149,12 +164,18 @@ pub fn specs() -> Vec { let ops = ids .iter() .filter_map(|id| s.develop_of(*id).map(|d| (*id, d))) - .filter_map(|(id, d)| { + .filter_map(|(id, _d)| { // back to the photo's import defaults when a default preset gave it its look let look = s.catalog.photo(id).and_then(|p| p.import_look.clone()); let fresh = match look { Some(l) => (*l).clone(), - None => DevelopSettings { wb: lightcraft_develop::WhiteBalance { mode: WbMode::AsShot, ..d.wb }, ..Default::default() }, + None => { + let info = s.source_info(id); + DevelopSettings { + wb: lightcraft_develop::WhiteBalance { mode: WbMode::AsShot, temp: info.as_shot_temp, tint: info.as_shot_tint }, + ..Default::default() + } + } }; s.develop_op(id, fresh, "Reset") }) @@ -203,7 +224,7 @@ pub fn specs() -> Vec { |s, _| { let id = active(s, "develop.autoBwMix")?; let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.autoBwMix", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); let m = lightcraft_pipeline::auto::auto_bw_mix(&src, &info, &d); edit(s, "develop.autoBwMix", "Auto B&W Mix", |d| { @@ -218,7 +239,7 @@ pub fn specs() -> Vec { cmd!("develop.auto", "Auto Settings", ["Photo"], Some("Shift+A"), "{}", has_active, |s, _| { let id = active(s, "develop.auto")?; let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.auto", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); let a = lightcraft_pipeline::auto::auto_tone(&src, &info, &d); edit(s, "develop.auto", "Auto", |d| { @@ -245,7 +266,7 @@ pub fn specs() -> Vec { let mode: WbMode = serde_json::from_value(p.get("mode").cloned().unwrap_or(json!("custom"))).map_err(|e| bad("develop.wb", e.to_string()))?; let id = active(s, "develop.wb")?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let (mut t, mut tint) = match mode { WbMode::AsShot => (info.as_shot_temp, info.as_shot_tint), WbMode::Auto => { @@ -256,7 +277,7 @@ pub fn specs() -> Vec { .preset() .map(|(t, ti)| { // presets are relative to daylight for rendered files - if info.raw { (t, ti) } else { (6500.0 * t / 5500.0, ti) } + if info.raw && !info.relative_wb { (t, ti) } else { (6500.0 * t / 5500.0, ti) } }) .unwrap_or((info.as_shot_temp, info.as_shot_tint)), }; @@ -277,7 +298,7 @@ pub fn specs() -> Vec { let (x, y) = (f64_req(p, "x", "develop.wbPick")?, f64_req(p, "y", "develop.wbPick")?); let id = active(s, "develop.wbPick")?; let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.wbPick", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); // Sample a small neighbourhood in the oriented source. let o = src.oriented(d.orientation); @@ -405,7 +426,7 @@ pub fn specs() -> Vec { |s, _| { let id = active(s, "crop.autoStraighten")?; let src = s.source_now(id, SourceLevel::Preview).map_err(|e| bad("crop.autoStraighten", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); let Some(level) = lightcraft_pipeline::upright::level_degrees(&src, &info, &d) else { return Ok(json!({"changed": false, "reason": "no dominant horizontal or vertical lines"})); @@ -501,7 +522,7 @@ pub fn specs() -> Vec { Upright::Off | Upright::Guided => None, m => { let src = s.source_now(id, SourceLevel::Preview).map_err(|e| bad("geometry.upright", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); Some(lightcraft_pipeline::upright::auto_transform(&src, &info, &d, m)) } }; @@ -657,6 +678,11 @@ pub fn specs() -> Vec { .filter_map(|id| s.develop_of(id).map(|d| (id, d))) .filter_map(|(id, d)| { let mut nd = (*d).clone(); + if nd.wb.mode == WbMode::AsShot && (ctl == "wb.temp" || ctl == "wb.tint") { + let info = s.source_info(id); + nd.wb.temp = info.as_shot_temp; + nd.wb.tint = info.as_shot_tint; + } let v = controls::get(&nd, &ctl).unwrap_or(spec.default); controls::set(&mut nd, &ctl, v + delta); if ctl == "wb.temp" || ctl == "wb.tint" { diff --git a/crates/engine/src/cmd/masks.rs b/crates/engine/src/cmd/masks.rs index fcf918c47..ec3ffac07 100644 --- a/crates/engine/src/cmd/masks.rs +++ b/crates/engine/src/cmd/masks.rs @@ -185,7 +185,7 @@ pub fn specs() -> Vec { let add = p.get("add").and_then(Value::as_bool).unwrap_or(false); let id = s.active().ok_or_else(|| bad(c, "no active photo"))?; let src = s.source_now(id, crate::media::SourceLevel::Thumb).map_err(|e| bad(c, e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); let req = lightcraft_pipeline::RenderRequest::fit(384, 384); let lab = lightcraft_pipeline::color_range_sample(&src, &info, &d, &req, lightcraft_geom::Point::new(x, y)) @@ -646,7 +646,7 @@ fn spots_edit(s: &mut Session, c: &str, label: &str, f: impl FnOnce(&mut Vec) -> Option { let src = s.source_now(id, crate::media::SourceLevel::Thumb).ok()?; - let info = crate::media::source_info(s.catalog.photo(id)?); + let info = s.source_info(id); lightcraft_pipeline::spots::pick_source(&src, &info, d, spot, avoid) } diff --git a/crates/engine/src/files.rs b/crates/engine/src/files.rs index 3a9f42c8a..b4c9f863e 100644 --- a/crates/engine/src/files.rs +++ b/crates/engine/src/files.rs @@ -131,7 +131,9 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result { std::mem::swap(&mut w, &mut h); } let (t, tint) = xy_to_temp_tint(lightcraft_raw::color::as_shot_white_xy_of(&raw)); - let as_shot_wb = Some((t.round(), tint.round())); + // Vendor RGB multipliers do not identify an absolute illuminant without camera calibration. + let relative = raw.format == lightcraft_raw::RawFormat::Arw && !lightcraft_raw::color::has_matrix(&raw.color); + let as_shot_wb = Some(if relative { (6500.0, 0.0) } else { (t.round(), tint.round()) }); let embedded_lens = embedded_lens(&raw); return Ok(ProbeInfo { embedded_lens, @@ -223,7 +225,7 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result< }; let xy = lightcraft_raw::color::as_shot_white_xy(&raw); let t = lightcraft_raw::color::camera_transform(&raw, xy); - let preview_matrix = crate::camera_preview::fit_preview(&raw, &bytes, &t); + let camera_look = crate::camera_preview::fit_preview(&raw, &bytes, &t); drop(bytes); // Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in. let lens = embedded_lens(&raw.info()); @@ -248,7 +250,7 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result< stages.push(("transform", t0.elapsed())); lightcraft_raw::highlight::reconstruct(&mut img, t.wb, HIGHLIGHT_CLIP); stages.push(("highlights", t0.elapsed())); - let m = preview_matrix.map(|p| p.mul(&t.matrix)).unwrap_or(t.matrix).to_f32(); + let m = camera_look.map(|p| p.matrix.mul(&t.matrix)).unwrap_or(t.matrix).to_f32(); let gain = 2f32.powf(t.baseline_exposure as f32); let wb = t.wb; img.map_in_place(|p| { @@ -277,7 +279,12 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result< eprintln!("[profile] raw source {}×{} (max {max_edge}, ms after decode): {}", img.width, img.height, parts.join(", ")); } let (temp, tint) = xy_to_temp_tint(xy); - return Ok((img, SourceInfo { raw: true, as_shot_temp: temp.round(), as_shot_tint: tint.round(), lens })); + let relative = raw.format == lightcraft_raw::RawFormat::Arw && t.matrix_is_fallback; + let (temp, tint) = if relative { (6500.0, 0.0) } else { (temp.round(), tint.round()) }; + return Ok(( + img, + SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens, relative_wb: relative, camera_tone: camera_look.map(|p| p.tone) }, + )); } let d = lightcraft_codecs::decode(&bytes, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).map_err(|e| e.to_string())?; drop(bytes); diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rs index 96d25932c..0492ffe66 100644 --- a/crates/engine/src/media.rs +++ b/crates/engine/src/media.rs @@ -26,7 +26,7 @@ use lightcraft_raster::{Histogram, Rgb32f, Rgba8}; use serde::{Deserialize, Serialize}; /// Bump when the pipeline's output changes, to invalidate cached thumbnails. -pub const RENDER_CACHE_VERSION: u64 = 5; +pub const RENDER_CACHE_VERSION: u64 = 6; /// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache). pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512]; @@ -71,9 +71,16 @@ pub type FileLoader = Arc Result<(Rgb32f, SourceInfo), St /// `max_edge` (set by the app). `None`: no usable preview. pub type PreviewLoader = Arc Option + Send + Sync>; +/// Pixels and the source interpretation learned while decoding; they must be cached together. +#[derive(Clone)] +pub struct DecodedSource { + pub image: Arc, + pub info: Option, +} + #[derive(Clone)] pub enum SourceRef { - Loaded(Arc), + Loaded(Box), Demo { scene: Box, max_edge: usize, @@ -91,28 +98,33 @@ pub enum SourceRef { impl SourceRef { pub fn load(&self) -> Result, String> { - match self { - SourceRef::Loaded(a) => Ok(a.clone()), - SourceRef::Demo { scene, max_edge } => Ok(Arc::new(scene.render_fit(*max_edge))), + self.load_source().map(|s| s.image) + } + + pub fn load_source(&self) -> Result { + let image = match self { + SourceRef::Loaded(a) => return Ok((**a).clone()), + SourceRef::Demo { scene, max_edge } => Arc::new(scene.render_fit(*max_edge)), SourceRef::File { path, max_edge, loader } => match loader { - Some(l) => l(path, *max_edge).map(|(img, _)| Arc::new(img)), - None => Err(format!("no decoder available for {path}")), + Some(l) => return l(path, *max_edge).map(|(image, info)| DecodedSource { image: Arc::new(image), info: Some(info) }), + None => return Err(format!("no decoder available for {path}")), }, #[cfg(not(target_arch = "wasm32"))] - SourceRef::Smart { path } => crate::smart::load(path), + SourceRef::Smart { path } => crate::smart::load(path)?, #[cfg(target_arch = "wasm32")] - SourceRef::Smart { .. } => Err("smart previews are not available here".into()), - } + SourceRef::Smart { .. } => return Err("smart previews are not available here".into()), + }; + Ok(DecodedSource { image, info: None }) } } pub struct MediaCache { /// Decoded thumbnail-level sources (LRU by bytes). - thumbs: Lru>, + thumbs: Lru, /// Decoded preview-level sources with their last use ([`lightcraft_preview::next_tick`]). - previews: Vec<(PhotoId, Arc, u64)>, + previews: Vec<(PhotoId, DecodedSource, u64)>, /// The last full-resolution original (exports; one at a time: ~300 MB at 24 MP). - full: Option<(PhotoId, Arc, u64)>, + full: Option<(PhotoId, DecodedSource, u64)>, /// How many previews to keep (LRU). pub preview_capacity: usize, /// Bytes all decoded sources and rendered previews in memory may take together (the cache @@ -156,7 +168,7 @@ fn rendered_budget(share: usize) -> usize { } fn source_bytes(img: &Rgb32f) -> usize { - img.data.len() * 12 + 64 + img.data.len() * 12 + 64 + std::mem::size_of::() } impl MediaCache { @@ -207,8 +219,8 @@ impl MediaCache { let Some((tick, which)) = candidates.into_iter().flatten().min() else { break }; let freed = match which { 0 => self.thumbs.pop_oldest(), - 1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1)), - 2 => self.full.take().map(|e| source_bytes(&e.1)), + 1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1.image)), + 2 => self.full.take().map(|e| source_bytes(&e.1.image)), _ => self.rendered.evict_oldest(), }; match freed { @@ -219,6 +231,10 @@ impl MediaCache { } pub fn get(&mut self, id: PhotoId, level: SourceLevel) -> Option> { + self.get_source(id, level).map(|s| s.image) + } + + fn get_source(&mut self, id: PhotoId, level: SourceLevel) -> Option { match level { SourceLevel::Thumb => self.thumbs.get(&id).cloned(), SourceLevel::Preview => self.previews.iter_mut().find(|e| e.0 == id).map(|e| { @@ -232,11 +248,23 @@ impl MediaCache { } } + fn source_facts(&self, id: PhotoId) -> Option { + self.thumbs + .peek(&id) + .and_then(|s| s.info) + .or_else(|| self.previews.iter().find(|e| e.0 == id).and_then(|e| e.1.info)) + .or_else(|| self.full.as_ref().filter(|e| e.0 == id).and_then(|e| e.1.info)) + } + pub fn insert(&mut self, id: PhotoId, level: SourceLevel, img: Arc) { + self.insert_source(id, level, DecodedSource { image: img, info: None }); + } + + fn insert_source(&mut self, id: PhotoId, level: SourceLevel, img: DecodedSource) { let tick = lightcraft_preview::next_tick(); match level { SourceLevel::Thumb => { - let cost = source_bytes(&img); + let cost = source_bytes(&img.image); self.thumbs.insert(id, img, cost); } SourceLevel::Preview => { @@ -270,14 +298,14 @@ impl MediaCache { /// Decoded sources held: (thumbnail level, preview level, full size). pub fn usage(&self) -> (crate::memory::Usage, crate::memory::Usage, crate::memory::Usage) { use crate::memory::Usage; - let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1)).sum()); - let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1))).unwrap_or_default(); + let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1.image)).sum()); + let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1.image))).unwrap_or_default(); (Usage::new(self.thumbs.len(), self.thumbs.cost()), previews, full) } pub fn source_ref(&mut self, p: &Photo, level: SourceLevel) -> SourceRef { - if let Some(a) = self.get(p.id, level) { - return SourceRef::Loaded(a); + if let Some(a) = self.get_source(p.id, level) { + return SourceRef::Loaded(Box::new(a)); } // Procedural scenes have a nominal size: "full" is that size, not unbounded. let max_edge = level.max_edge().min(match (&p.source, level) { @@ -344,7 +372,7 @@ pub struct RenderResult { pub key: u64, pub rendered: Result, /// A source that was loaded by this job (to be inserted into the cache). - pub loaded: Option>, + pub loaded: Option, /// Set for a [`QuickJob`]'s stand-in: where the image came from. pub quick: Option, } @@ -410,12 +438,14 @@ impl RenderJob { }; } let was_loaded = matches!(self.source, SourceRef::Loaded(_)); - match self.source.load() { - Ok(src) => { + match self.source.load_source() { + Ok(source) => { + let src = &source.image; + let info = source.info.unwrap_or(self.info); // Thumbnails (many small jobs side by side) stay on the CPU; views and exports use // the GPU when there is one. let gpu = self.cache.is_none(); - let rendered = develop(&src, &self.info, &self.settings, &self.request, self.stages.as_deref(), gpu); + let rendered = develop(src, &info, &self.settings, &self.request, self.stages.as_deref(), gpu); if let Some((cache, key)) = &self.cache { cache.put(*key, Arc::new(rendered.image.clone())); } @@ -429,7 +459,7 @@ impl RenderJob { level: self.level, key: self.key, rendered: Ok(rendered), - loaded: (!was_loaded).then_some(src), + loaded: (!was_loaded).then_some(source), quick: None, } } @@ -521,11 +551,16 @@ pub fn content_key(p: &Photo) -> String { pub fn source_info(p: &Photo) -> SourceInfo { // Procedural demo scenes are scene-referred HDR (like raw files): use the filmic tone map. if matches!(p.source, Source::Demo { .. }) { - return SourceInfo { raw: true, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None }; + return SourceInfo { raw: true, ..Default::default() }; } // A raw shown from its embedded preview is a rendered (display-referred) JPEG: relative white // balance and the display tone curve, like any other rendered file. - if p.develops_raw() { SourceInfo { raw: true, as_shot_temp: 5500.0, as_shot_tint: 0.0, lens: p.embedded_lens } } else { SourceInfo::default() } + if p.develops_raw() { + let (temp, tint) = if p.relative_wb() { (6500.0, 0.0) } else { p.as_shot_wb.unwrap_or((5500.0, 0.0)) }; + SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens: p.embedded_lens, relative_wb: p.relative_wb(), ..Default::default() } + } else { + SourceInfo::default() + } } impl crate::Session { @@ -680,7 +715,7 @@ impl crate::Session { /// Accept a finished job's loaded source into the cache. pub fn accept(&mut self, r: &RenderResult) { if let Some(src) = &r.loaded { - self.media.insert(r.photo, r.level, src.clone()); + self.media.insert_source(r.photo, r.level, src.clone()); } } @@ -723,12 +758,21 @@ impl crate::Session { Ok(job) } + /// Prefer decoder facts to header-only metadata for pixel-statistics commands. + pub fn source_info(&self, id: PhotoId) -> SourceInfo { + if let Some(info) = self.media.source_facts(id) { + return info; + } + self.catalog.photo(id).map(|p| source_info(p)).unwrap_or_default() + } + /// The source proxy for pixel-statistics commands (auto tone/WB), loading synchronously. pub fn source_now(&mut self, id: PhotoId, level: SourceLevel) -> Result, String> { let p = self.catalog.photo(id).ok_or("no such photo")?.clone(); - let r = self.media.source_ref(&p, level).load()?; - self.media.insert(id, level, r.clone()); - Ok(r) + let r = self.media.source_ref(&p, level).load_source()?; + let image = r.image.clone(); + self.media.insert_source(id, level, r); + Ok(image) } } @@ -760,6 +804,41 @@ pub type FileProbe = Arc Result + Send + Sync mod tests { use super::*; + #[test] + fn decoder_info_survives_render_jobs_cache_and_eviction() { + let tone = lightcraft_pipeline::tone::CameraTone::new(std::array::from_fn(|i| { + let x = 0.01 * (i + 1) as f32; + [x, (x * 2.0).min(0.9)] + })) + .unwrap(); + let info = SourceInfo { raw: true, relative_wb: true, camera_tone: Some(tone), ..Default::default() }; + let mut s = crate::Session::with_demo(); + let id = s.active().unwrap(); + let mut job = s.render_job(id, 64, 64, false, true).unwrap(); + job.source = SourceRef::File { + path: "synthetic.arw".into(), + max_edge: 64, + loader: Some(Arc::new(move |_, _| { + let mut image = Rgb32f::new(64, 64); + image.data.fill([0.1; 3]); + Ok((image, info)) + })), + }; + job.info = SourceInfo::default(); // Header facts cannot override decoder facts. + job.settings = Arc::new(DevelopSettings::default()); + let r = job.clone().run(); + assert_eq!(r.loaded.as_ref().unwrap().info, Some(info)); + let expected = lightcraft_pipeline::render(&r.loaded.as_ref().unwrap().image, &info, &job.settings, &job.request); + assert_eq!(r.rendered.as_ref().unwrap().image.data, expected.image.data); + s.accept(&r); + job.source = s.media.source_ref(s.catalog.photo(id).unwrap(), job.level); + let again = job.run(); + assert!(again.loaded.is_none()); + assert_eq!(r.rendered.unwrap().image.data, again.rendered.unwrap().image.data); + s.media.forget(id); + assert!(s.media.get_source(id, r.level).is_none()); + } + #[test] fn budget_evicts_least_recently_used_across_caches() { let img = |w: usize| Arc::new(Rgb32f::new(w, w)); diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 281a7a5f6..58b7e643e 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -10,6 +10,30 @@ fn active_dev(s: &Session) -> lightcraft_develop::DevelopSettings { (*s.develop_of(s.active().unwrap()).unwrap()).clone() } +#[test] +fn changing_one_wb_control_resolves_as_shot_without_stale_tint() { + use lightcraft_catalog::{Photo, PhotoId, Source}; + use lightcraft_develop::{DevelopSettings, WbMode}; + let mut s = demo(); + let id = PhotoId(100); + let mut p = Photo::new(id, Source::File { path: "synthetic.arw".into() }, "synthetic.arw", "ARW", 16, 16, ""); + // A catalog made by the old generic-matrix Kelvin inference. + p.as_shot_wb = Some((6829.0, -127.0)); + p.develop = std::sync::Arc::new(DevelopSettings::for_raw(6829.0, -127.0)); + s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(p) }).unwrap(); + s.execute("library.select", &json!({"ids": [100], "active": 100})).unwrap(); + s.execute("develop.set", &json!({"control": "wb.temp", "value": 8000})).unwrap(); + let d = active_dev(&s); + assert_eq!(d.wb.mode, WbMode::Custom); + assert_eq!(d.wb.temp, 8000.0); + assert_eq!(d.wb.tint, 0.0, "untouched tint comes from the current as-shot reference"); + s.execute("develop.wb", &json!({"mode": "asShot"})).unwrap(); + s.execute("develop.set", &json!({"control": "wb.tint", "value": 10})).unwrap(); + assert_eq!(active_dev(&s).wb.temp, 6500.0); + s.execute("develop.reset", &json!({})).unwrap(); + assert_eq!((active_dev(&s).wb.temp, active_dev(&s).wb.tint), (6500.0, 0.0)); +} + #[test] fn demo_library_loads() { let mut s = demo(); diff --git a/crates/gpu/tests/equivalence.rs b/crates/gpu/tests/equivalence.rs index a83c7beb1..cf7566e0d 100644 --- a/crates/gpu/tests/equivalence.rs +++ b/crates/gpu/tests/equivalence.rs @@ -22,6 +22,27 @@ fn gpu() -> bool { ok } +#[test] +fn camera_tone_and_relative_wb() { + if !gpu() { + return; + } + let src = scene(2, 320, 240); + let curve = lightcraft_pipeline::tone::CameraTone::new(std::array::from_fn(|i| { + let x = 0.004 * 1.18f32.powi(i as i32); + [x, 1.0 - (-2.0 * x).exp()] + })) + .unwrap(); + let info = SourceInfo { raw: true, relative_wb: true, camera_tone: Some(curve), ..Default::default() }; + let mut s = DevelopSettings::default(); + check("camera tone neutral", &src, &info, &s, &RenderRequest::fit(320, 240)); + s.light.exposure = 1.0; + s.light.contrast = 40.0; + s.wb.mode = WbMode::Custom; + s.wb.temp = 8000.0; + check("camera tone edited", &src, &info, &s, &RenderRequest::fit(320, 240)); +} + fn scene(i: usize, w: usize, h: usize) -> Arc { Arc::new(lightcraft_scenes::demo_library()[i].render(w, h)) } diff --git a/crates/pipeline/src/finish.rs b/crates/pipeline/src/finish.rs index aeb0adf81..dd1e2eea5 100644 --- a/crates/pipeline/src/finish.rs +++ b/crates/pipeline/src/finish.rs @@ -221,7 +221,9 @@ impl FinishParams { FinishParams { calib: if calibration { crate::colorops::calibration_matrix(&s.calibration) } else { None }, shadow_tint: if calibration { (s.calibration.shadows_tint / 100.0) as f32 } else { 0.0 }, - tone: if info.raw { + tone: if let Some(curve) = info.camera_tone.as_ref().filter(|_| info.raw) { + ToneMap::camera(curve, s.light.contrast, s.light.whites, s.light.blacks) + } else if info.raw { ToneMap::new(s.light.contrast, s.light.whites, s.light.blacks) } else { ToneMap::display(s.light.contrast, s.light.whites, s.light.blacks) diff --git a/crates/pipeline/src/lib.rs b/crates/pipeline/src/lib.rs index 875aca11c..15fbd7f33 100644 --- a/crates/pipeline/src/lib.rs +++ b/crates/pipeline/src/lib.rs @@ -64,11 +64,14 @@ pub struct SourceInfo { pub raw: bool, pub as_shot_temp: f64, pub as_shot_tint: f64, + /// No measured camera illuminant: WB adjustments are relative to the camera's rendered look. + pub relative_wb: bool, + pub camera_tone: Option, } impl Default for SourceInfo { fn default() -> Self { - Self { raw: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None } + Self { raw: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None, relative_wb: false, camera_tone: None } } } diff --git a/crates/pipeline/src/local.rs b/crates/pipeline/src/local.rs index ae26888c7..cedb7cd1a 100644 --- a/crates/pipeline/src/local.rs +++ b/crates/pipeline/src/local.rs @@ -61,7 +61,7 @@ pub fn effective_wb(info: &SourceInfo, s: &DevelopSettings) -> (f64, f64) { use lightcraft_develop::WbMode; match s.wb.mode { WbMode::AsShot => (info.as_shot_temp, info.as_shot_tint), - m if info.raw => m.preset().unwrap_or((s.wb.temp, s.wb.tint)), + m if info.raw && !info.relative_wb => m.preset().unwrap_or((s.wb.temp, s.wb.tint)), _ => (s.wb.temp, s.wb.tint), } } diff --git a/crates/pipeline/src/tone.rs b/crates/pipeline/src/tone.rs index 63d23abc1..232bb8da4 100644 --- a/crates/pipeline/src/tone.rs +++ b/crates/pipeline/src/tone.rs @@ -14,12 +14,75 @@ pub const LUT_MIN_EV: f32 = -14.0; pub const LUT_MAX_EV: f32 = 10.0; pub const LUT_N: usize = 4096; +/// A file-local camera look, fitted independently of the scene-linear colour transform. +/// Knots are scene/display-linear luminance pairs. Keeping this in the finish stage preserves +/// RAW exposure and highlight headroom; it is never baked into the decoded sensor pixels. +#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize)] +pub struct CameraTone { + knots: [[f32; 2]; 32], +} + +impl<'de> serde::Deserialize<'de> for CameraTone { + fn deserialize>(d: D) -> Result { + #[derive(serde::Deserialize)] + struct Wire { + knots: [[f32; 2]; 32], + } + let w = Wire::deserialize(d)?; + Self::new(w.knots).ok_or_else(|| serde::de::Error::custom("invalid camera tone curve")) + } +} + +impl CameraTone { + pub fn new(knots: [[f32; 2]; 32]) -> Option { + let mut previous = [0.0, 0.0]; + for p in knots { + if !p.iter().all(|v| v.is_finite()) || p[0] <= previous[0] || p[1] < previous[1] || p[1] >= 1.0 { + return None; + } + previous = p; + } + Some(Self { knots }) + } + + pub fn apply(&self, y: f32) -> f32 { + if !y.is_finite() || y <= 0.0 { + return 0.0; + } + let mut previous = [0.0, 0.0]; + for p in self.knots { + if y <= p[0] { + let t = (y - previous[0]) / (p[0] - previous[0]); + return previous[1] + t * (p[1] - previous[1]); + } + previous = p; + } + let a = self.knots[30]; + let b = self.knots[31]; + // Extend beyond observed (unclipped) highlights with a continuous, bounded shoulder. + let slope = ((b[1] - a[1]) / (b[0] - a[0])).clamp(0.1, 16.0); + 1.0 - (1.0 - b[1]) * (-(y - b[0]) * slope / (1.0 - b[1]).max(0.01)).exp() + } +} + #[derive(Clone, Debug)] pub struct ToneMap { lut: Vec, } impl ToneMap { + pub fn camera(curve: &CameraTone, contrast: f64, whites: f64, blacks: f64) -> ToneMap { + let adjustment = Self::display(contrast, whites, blacks); + let neutral = contrast == 0.0 && whites == 0.0 && blacks == 0.0; + let lut = (0..LUT_N) + .map(|i| { + let ev = LUT_MIN_EV + (LUT_MAX_EV - LUT_MIN_EV) * i as f32 / (LUT_N - 1) as f32; + let y = curve.apply(GREY * 2f32.powf(ev)); + if neutral { y } else { adjustment.apply(y) } + }) + .collect(); + ToneMap { lut } + } /// `contrast`, `whites`, `blacks` in −100..100 (Lightroom slider units). pub fn new(contrast: f64, whites: f64, blacks: f64) -> ToneMap { let c = (contrast / 100.0) as f32; @@ -116,6 +179,32 @@ fn smooth(e0: f32, e1: f32, x: f32) -> f32 { mod tests { use super::*; + #[test] + fn camera_curve_preserves_black_and_extends_headroom() { + let knots = std::array::from_fn(|i| { + let x = 0.005 * 1.15f32.powi(i as i32); + [x, 1.0 - (-3.0 * x).exp()] + }); + let curve = CameraTone::new(knots).unwrap(); + let base = ToneMap::camera(&curve, 0.0, 0.0, 0.0); + assert_eq!(base.apply(0.0), 0.0); + assert!(base.apply(0.1) < base.apply(0.2)); + let mut previous = 0.0; + for i in 0..2000 { + let y = 1e-6 * 1.01f32.powi(i); + let v = base.apply(y); + assert!((0.0..=1.0).contains(&v)); + assert!(v >= previous - 1e-6); + previous = v; + } + assert!((base.apply(0.1) - curve.apply(0.1)).abs() < 0.001); + assert!(ToneMap::camera(&curve, 50.0, 0.0, 0.0).apply(0.3) > base.apply(0.3)); + let mut invalid = knots; + invalid[1][0] = invalid[0][0]; + assert!(CameraTone::new(invalid).is_none()); + assert!(serde_json::from_value::(serde_json::json!({"knots": invalid})).is_err()); + } + #[test] fn monotone_and_bounded() { for (c, w, b) in [(0.0, 0.0, 0.0), (100.0, 100.0, -100.0), (-100.0, -100.0, 100.0), (50.0, -30.0, -40.0)] { diff --git a/crates/ui-egui/src/panels/edit.rs b/crates/ui-egui/src/panels/edit.rs index e2ecca268..2d0b452d5 100644 --- a/crates/ui-egui/src/panels/edit.rs +++ b/crates/ui-egui/src/panels/edit.rs @@ -76,7 +76,13 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui, id: PhotoId) { let t = Tokens::get(ui.ctx()); let d = app.session.develop_of(id).unwrap_or_default(); // a raw shown from its embedded JPEG (preview only) gets the rendered-file white balance scale - let raw = app.session.catalog.photo(id).is_some_and(|p| p.develops_raw()); + let raw = app.session.catalog.photo(id).is_some_and(|p| p.develops_raw() && !p.relative_wb()); + let mut d = d; + if d.wb.mode == WbMode::AsShot && app.session.catalog.photo(id).is_some_and(|p| p.relative_wb()) { + let wb = &mut std::sync::Arc::make_mut(&mut d).wb; + wb.temp = 6500.0; + wb.tint = 0.0; + } let preview_only = app.session.catalog.photo(id).and_then(|p| p.preview_only.clone()); if app.ui.histogram { diff --git a/docs/camera-preview-colour.md b/docs/camera-preview-colour.md index f10a53820..7d20687db 100644 --- a/docs/camera-preview-colour.md +++ b/docs/camera-preview-colour.md @@ -1,31 +1,31 @@ -# ARW preview-derived colour starting point +# Sony ARW starting look -Sony ARW files with no usable camera colour matrix previously used the generic camera-RGB≈sRGB fallback. This can produce visibly muted, dark photographs. The engine now estimates a bounded 3×3 correction from a file's own embedded camera JPEG. The JPEG provides colour correspondences, not output pixels: decoding, demosaicing, white balance, highlight reconstruction and RAW editing still operate on the sensor data. +ARW decoding supplies a Bayer mosaic and camera white-balance multipliers, but currently no measured camera colour matrix. Treating camera RGB as linear sRGB produces a dark, muted default rendering. The ARW loader now estimates a file-local starting look from that file's embedded camera JPEG, only when a usable camera matrix is absent. All output pixels still come from the RAW mosaic; there is no JPEG replacement or uniform saturation boost. -A fixed sensor proxy (at most 96×96) is used at every requested output size. The reference is decoded with DCT scaling and a 64-million-pixel input limit. Reference and sensor aspect ratios must agree within 2%; references with a different aspect ratio are rejected. Near-black, clipped and nonfinite samples are excluded. At least 256 usable pairs, enough coloured samples and a nonsingular, bounded-condition input covariance are required. Ridge regression is biased towards an exposure-only correction, rather than unconstrained colour amplification. Matrix coefficients are bounded. +## Colour and tone are separate -One third of the usable pixels are held out from fitting. After the normal RAW tone shoulder, the correction must reduce linear-display squared error by at least 30%, with per-channel RMS error at most 0.075 (a provisional camera-look tolerance, not a colour-accuracy certification). Unusable, monochrome, mismatched, ill-conditioned or poor-fitting references retain the existing fallback. DNG matrices and other RAW formats are unchanged. Render cache version 5 invalidates previously rendered thumbnails and views. +A fixed sensor proxy of at most 96×96 is developed with the file's WB, black/white levels and active crop. The embedded JPEG is decoded with DCT scaling and a 64-million-pixel input limit, converted through the codec's colour management, and reduced to the same size. Aspect ratios must agree within 2%. Near-black, clipped and nonfinite samples are excluded. At least 256 paired pixels, 5% coloured samples and a nonsingular input covariance with bounded condition are required. -This is a per-file camera-look estimate, not a measured per-camera spectral calibration. The embedded JPEG includes camera tone curves, picture styles, noise reduction and possibly local/lens adjustments; a global matrix cannot reproduce all of them. Similar-sized cropped or warped references are not guaranteed to be detected. Skin colour and highlights can still differ. A camera-matrix database and chart-based verification remain necessary. Do not advertise this as Lightroom colour parity or complete camera coverage. +Two thirds of the usable pixels train a ridge-regularised 3×3 **chromaticity** correction (luminance-normalised RGB). This avoids trying to encode the camera's nonlinear tone curve in a matrix, which left skin grey in the earlier matrix-only estimate. Coefficients are bounded by 8. Luminance is fitted separately: 32 equal-population bins with robust median knots and pooled-adjacent-violators isotonic regression. The curve preserves black, cannot reverse, and extends beyond observed highlights with a bounded shoulder. -Validation uses synthetic coloured scenes with a known matrix, monochrome/nonfinite/singular inputs, unrelated references and mismatched lengths. Five private Sony ILCE-6700 ARWs are checked locally; photographs and embedded JPEGs are not committed or uploaded. See the PR for the observed acceptance count and remaining differences. +The matrix is applied to scene-linear sensor pixels. The curve is carried in `SourceInfo` and applied in the pipeline finish stage, after exposure/WB/local edits. It is never baked into decoded luminance or clipped to JPEG precision. CPU and GPU use the same tone LUT. Contrast, whites and blacks adjust the starting look through the display-domain controls. JPEG neutral rendering and the existing DNG colour model are unchanged. -## Local observations (Sony ILCE-6700) +One third of the pixels are held out from both fits. The combined model must reduce linear-display squared error by at least 30% and have per-channel RMS error ≤0.055. These are provisional acceptance gates, not a colour-accuracy certification. Monochrome, singular, unrelated, aspect-mismatched and poor-fitting references retain the documented fallback. Both fits use the same fixed proxy at every decode resolution. -The guarded estimate was accepted on all five private samples. Brightness and yellow clothing improved; skin still differs from the camera JPEG. This is why the change is proposed as a draft for colour-science review and broader camera verification. +## Source metadata and white balance -A camera JPEG from the same capture, the previous RAW renderer and the fitted RAW renderer were exported at 1200 px with neutral settings and separately with Exposure +1 EV, Saturation +100 and Contrast +50. All three controls changed the previous RAW output. Mean absolute encoded-RGB differences from each source’s neutral rendering were: +Previously `SourceRef::load()` discarded the decoder's `SourceInfo`, and render jobs used header-only facts (including a hard-coded RAW WB). Decoded pixels and source facts now travel and remain cached together, including across web-worker renders. Pixel-statistics/colour-sampling commands use decoded facts too. Render cache version 6 invalidates old thumbnails and loupe renders. -| Source | Exposure +1 | Saturation +100 | Contrast +50 | -|---|---:|---:|---:| -| Camera JPEG | 0.1930 | 0.0642 | 0.0373 | -| RAW, generic fallback | 0.1399 | 0.0318 | 0.0315 | -| RAW, preview-derived estimate | 0.1628 | 0.0509 | 0.0706 | +Vendor RGB WB multipliers alone do **not** identify an absolute illuminant without camera calibration. The generic model inferred approximately 6829 K / −127 tint for one Sony sample; changing only temperature retained that spurious tint and caused a pink cast. Uncalibrated ARWs now keep the file's vendor WB at decode and expose **relative** WB controls around the as-shot look. 6500/0 is the internal neutral reference of that relative scale, not a claim about the capture's measured Kelvin temperature. Presets, picker/auto and manual controls use this reference consistently. Switching from As Shot to Custom first resolves both controls; changing one cannot inherit stale catalog tint. Reset restores the current as-shot reference. Measured/absolute-Kelvin Sony WB remains future camera-calibration work. -These are response diagnostics on one image, not colour-accuracy or Lightroom-parity scores. The weaker saturation response starts from an already muted rendering; increasing saturation does not replace missing camera colour characterization. JPEG uses an identity tone map at neutral settings; RAW uses the scene-to-display shoulder, so identical slider values are not expected to produce identical changes. Edited RAW was also checked in the headless UI with the actual GPU renderer, after waiting for pending jobs: the loupe reported `source: render`, the histogram and displayed pixels changed, and Exposure +1 / Saturation +100 remained in the controls. No slider-no-op was reproduced in this sample. +Sony's proprietary `ColorMatrix` is not assumed to be a DNG XYZ-to-camera matrix. No Adobe profile/matrix or GPL decoder/calibration source is used. The product remains pure Rust. -A second comparison used five separately supplied same-capture JPEGs, including four with embedded Display P3 profiles. RAW-only changes left all five JPEG exports pixel-identical. Two supplied JPEGs have a narrower crop than the RAW; no pixel-by-pixel colour-accuracy claim is made across those different frames. On the supplied matching JPEG for the slider-response sample, the corresponding response figures were exposure 0.1952, saturation 0.0517 and contrast 0.0260. RAW saturation response increased from 0.0318 to 0.0509 with the estimate. +## Validation and scope -**Unresolved white-balance limitation:** the generic camera model also derives the RAW as-shot temperature/tint without a measured camera matrix. The sample reports roughly 6829 K / −127 tint. Changing temperature alone to 9000 K causes a strong pink cast, even after preview-derived colour fitting. The estimate does not establish a physically calibrated illuminant or repair absolute-Kelvin WB behaviour. It must not be presented as a complete RAW-development fix; measured camera characterization and white-balance validation are still required. +Synthetic tests cover colour with a nonlinear camera tone, preserved scene-linear headroom, invalid/singular/unrelated references, monotone black-preserving curves, hostile serialized knots, decode-to-render/cache metadata propagation, old-catalog WB transitions and CPU/GPU tone equivalence. Private photos are never committed or uploaded. -For the colour-fit sample, independent CLI renders at 400 px and full resolution selected exactly the same fitted matrix. Full cold render/export took 3.96 s; 400 px took 0.48 s on this local machine (includes process/import/render/encode, not just the adjustment pipeline). +Five local Sony ILCE-6700 ARWs were accepted. Held-out per-channel linear-display RMS values were 0.04244, 0.03349, 0.04552, 0.02871 and 0.03100. All five unedited supplied JPEG exports stayed pixel-identical to the installed pre-change renderer at equal size/quality. Four supplied JPEGs use Display P3 and two have a narrower crop than the RAW, so they are visual references rather than aligned pixel ground truth. + +On one sample, Exposure +1 EV, Saturation +100, Contrast +50, warm/cool WB, tint, highlights and shadows all changed the output. The same matrix and curve were selected for 400-pixel and full-resolution renders. Headless UI checks wait until the loupe reports `source: render` and no jobs remain; the camera JPEG stand-in does not count as verification. + +This is a per-file camera-look estimate, **not measured spectral calibration or Lightroom parity**. Camera local tone, picture styles, noise reduction and lens warps cannot all be reproduced by a global matrix/curve; same-aspect crops may evade the aspect check. Skin, mixed-light scenes and saturated highlights can still differ from a supplied JPEG. Broader Sony model/lighting coverage, chart-based calibration, absolute WB and other manufacturers remain unverified. Rejected fits and other RAW formats retain their existing behaviour. diff --git a/docs/parity.md b/docs/parity.md index fda380c4c..c970969e5 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -59,7 +59,7 @@ Ordered by user impact, then tier, then effort. The checklist above counts featu that decide whether a photographer can switch (see the honest assessment in [ROADMAP.md](../ROADMAP.md#where-we-stand)). Take the first one nobody is working on. -1. **LR-PROF-CAMERACOLOR** (P0): our own camera colour calibration. ARW now has guarded per-file embedded-JPEG fitting; measured calibration and fidelity remain missing. Other non-DNG raws and rejected fits still use a neutral matrix. Expand validated preview fitting and use matrices the files carry themselves; never Adobe data. +1. **LR-PROF-CAMERACOLOR** (P0): our own camera colour calibration. ARW now has guarded per-file embedded-JPEG colour/tone fitting and relative WB; measured calibration and fidelity remain missing. Other non-DNG raws and rejected fits still use a neutral matrix. Expand validated preview fitting and use matrices the files carry themselves; never Adobe data. 2. **LR-IMP-FORMATS** (P0): **CR3** first (every Canon body since ~2018), then compressed RAF / ORF, RW2 v4, Nikon "lossy after split" NEF, Canon sRAW; HEIC/AVIF decode. Clean-room, from prose descriptions only (see `crates/raw/src/vendor/nefc.rs` for how compressed NEF was done). Until decoded, such photos are `preview_only`. @@ -274,7 +274,7 @@ under the UI command that wraps it (a no-op error); the UI command now wins. | LR-PROF-ADOBE | Standard raw looks (own equivalents) | P0 | ✅ | `crates/engine/src/presets.rs` (`PROFILES`), `crates/pipeline/src/profiles.rs` | six own looks: Color, Neutral, Vivid, Landscape, Portrait, Monochrome | | LR-PROF-ADAPTIVE | Adaptive profiles | P2 | ⬜ | | | | LR-PROF-CAMERA | Camera-matching looks | P2 | ⬜ | | | -| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices they carry; ARW has guarded per-file embedded-JPEG colour fitting (see docs/camera-preview-colour.md), not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | +| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices they carry; ARW has guarded per-file embedded-JPEG chromaticity/tone fitting and relative WB (see docs/camera-preview-colour.md), not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | | LR-PROF-CREATIVE | Creative profiles (own) | P2 | ✅ | `cmd:develop.profile`, `crates/pipeline/src/profiles.rs` | 16 own looks in Film / Cinematic / Muted / B&W (tone + point-curve fades, colour grading, mixer / B&W mix); scale with `ctl:profile.amount`; sliders untouched | | LR-PROF-LEGACY | Legacy profiles | P2 | ⬜ | | | | LR-PROF-NONRAW | Profiles for non-raw files | P0 | ✅ | `cmd:develop.profile` | same looks apply to JPEG/TIFF | From 4b3048c48f2f19255be5ac25ecdb0245a965c1e9 Mon Sep 17 00:00:00 2001 From: Brandon Thomas Date: Mon, 5 Oct 2026 19:28:22 -0400 Subject: [PATCH 3/5] LR-PROF-CAMERACOLOR: accept ARW camera-look fits up to 0.10 held-out RMS The 0.055 cap rejected half of the public Sony samples tried (ILCE-6400, ILCE-6700, ILCE-7M4) although their fits were 1.5-3x closer to the camera JPEG than the fallback and visibly better (mean dE vs the embedded preview 19-26 -> 5-10). Unrelated previews stay rejected (synthetic test lands at 0.115). Co-Authored-By: Claude Opus 5.5 --- crates/engine/src/camera_preview.rs | 34 ++++++++++++++++++++++++++++- docs/camera-preview-colour.md | 2 +- 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/crates/engine/src/camera_preview.rs b/crates/engine/src/camera_preview.rs index 4a550630b..54b308b13 100644 --- a/crates/engine/src/camera_preview.rs +++ b/crates/engine/src/camera_preview.rs @@ -43,6 +43,14 @@ pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransf Some(look) } +/// A fit must cut the held-out squared error to below this share of the fallback's. +const MIN_IMPROVEMENT: f64 = 0.7; +/// ...and stay within this per-channel RMS of the camera JPEG (linear display values). On public +/// raw.pixls.us samples (eight Sony bodies) good fits that still beat the fallback 1.5–3× landed +/// at 0.065–0.093 (camera local tone, vignetting and lens processing that a global matrix + curve +/// cannot follow): visibly better renders that 0.055 rejected. +const MAX_HOLDOUT_RMS: f64 = 0.10; + fn luma(p: [f64; 3]) -> f64 { p[0] * 0.2627 + p[1] * 0.6780 + p[2] * 0.0593 } @@ -131,7 +139,7 @@ fn fit_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option { if lightcraft_pipeline::profiling() { eprintln!("[profile] ARW holdout RMS {:.5} -> {:.5} ({samples} channels)", (before / samples as f64).sqrt(), (after / samples as f64).sqrt()); } - if !after.is_finite() || samples == 0 || after >= before * 0.7 || after / samples as f64 > 0.055f64.powi(2) { + if !after.is_finite() || samples == 0 || after >= before * MIN_IMPROVEMENT || after / samples as f64 > MAX_HOLDOUT_RMS.powi(2) { return None; } Some(CameraLook { matrix, tone: curve }) @@ -216,6 +224,30 @@ mod tests { assert!(luma(p) > 1.0); assert!(tone.apply(0.2) < tone.apply(0.4)); } + #[test] + fn accepts_a_much_better_fit_despite_local_camera_processing() { + // The camera JPEG departs from any global matrix + curve (local tone, vignetting): ±0.12 + // per-pixel deviations, ~0.07 RMS. The fit is still far closer than the fallback. + let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]); + let mut sensor = Rgb32f::new(64, 64); + let mut reference = sensor.clone(); + let mut seed = 0x2545_f491_u32; + for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() { + let ev = 0.05 + (i % 31) as f32 * 0.017; + *src = [ev * (0.8 + (i % 11) as f32 * 0.025), ev, ev * (0.8 + (i % 17) as f32 * 0.014)]; + let p = known.apply_f32(*src); + let y = luminance_2020(p); + *dst = p.map(|v| { + seed ^= seed << 13; + seed ^= seed >> 17; + seed ^= seed << 5; + let noise = (seed % 2001) as f32 / 1000.0 - 1.0; + (v * (1.0 - (-2.5 * y).exp()) / y + 0.12 * noise).clamp(0.005, 0.97) + }); + } + assert!(fit_pairs(&sensor, &reference).is_some()); + } + #[test] fn rejects_monochrome_invalid_and_unrelated_previews() { let mut sensor = Rgb32f::new(32, 32); diff --git a/docs/camera-preview-colour.md b/docs/camera-preview-colour.md index 7d20687db..0aef3452e 100644 --- a/docs/camera-preview-colour.md +++ b/docs/camera-preview-colour.md @@ -10,7 +10,7 @@ Two thirds of the usable pixels train a ridge-regularised 3×3 **chromaticity** The matrix is applied to scene-linear sensor pixels. The curve is carried in `SourceInfo` and applied in the pipeline finish stage, after exposure/WB/local edits. It is never baked into decoded luminance or clipped to JPEG precision. CPU and GPU use the same tone LUT. Contrast, whites and blacks adjust the starting look through the display-domain controls. JPEG neutral rendering and the existing DNG colour model are unchanged. -One third of the pixels are held out from both fits. The combined model must reduce linear-display squared error by at least 30% and have per-channel RMS error ≤0.055. These are provisional acceptance gates, not a colour-accuracy certification. Monochrome, singular, unrelated, aspect-mismatched and poor-fitting references retain the documented fallback. Both fits use the same fixed proxy at every decode resolution. +One third of the pixels are held out from both fits. The combined model must reduce linear-display squared error by at least 30% and have per-channel RMS error ≤0.10 (0.055 rejected half of the public raw.pixls.us samples tried — ILCE-6400, -6700, -7M4 — whose fits were still 1.5–3× closer to the camera JPEG than the fallback and visibly better). These are provisional acceptance gates, not a colour-accuracy certification. Monochrome, singular, unrelated, aspect-mismatched and poor-fitting references retain the documented fallback. Both fits use the same fixed proxy at every decode resolution. ## Source metadata and white balance From f7a78313808ba9d93b9ec91efb6e7fad63494b09 Mon Sep 17 00:00:00 2001 From: Brandon Thomas Date: Mon, 5 Oct 2026 19:30:43 -0400 Subject: [PATCH 4/5] LR-PROF-CAMERACOLOR: keep the ARW camera tone curve with smart previews Smart previews store the decoded source with the fitted colour matrix baked in, but the file-local tone curve lives in the decoder's SourceInfo, which a proxy did not keep: an offline ARW rendered and exported with the generic filmic curve instead (darker, a different look). The proxy header now carries the curve (validated on load, ignored when invalid), DecodedSource::info_or merges it into the header facts, and is_valid reads the longer header. Co-Authored-By: Claude Opus 5.5 --- apps/lightcraft-web/src/wire.rs | 4 +- crates/engine/src/cmd/previews.rs | 4 +- crates/engine/src/media.rs | 55 +++++++++++++++-------- crates/engine/src/smart.rs | 73 +++++++++++++++++++++++++------ 4 files changed, 101 insertions(+), 35 deletions(-) diff --git a/apps/lightcraft-web/src/wire.rs b/apps/lightcraft-web/src/wire.rs index 71d9283a4..05960f7ef 100644 --- a/apps/lightcraft-web/src/wire.rs +++ b/apps/lightcraft-web/src/wire.rs @@ -161,7 +161,7 @@ impl WorkerCore { let src = match (&job.origin, original) { (Source::File { .. }, Some(bytes)) => { let (image, info) = lightcraft_engine::files::load_bytes(bytes, job.max_edge)?; - DecodedSource { image: Arc::new(image), info: Some(info) } + DecodedSource::new(Arc::new(image), Some(info)) } (Source::File { path }, None) => return Err(format!("{path}: original not found in browser storage")), (origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load_source()?, @@ -171,7 +171,7 @@ impl WorkerCore { src } }; - let info = src.info.unwrap_or(job.info()); + let info = src.info_or(job.info()); Ok(match &job.stages { Some(view) => { let st = self.stages.entry(view.clone()).or_default().clone(); diff --git a/crates/engine/src/cmd/previews.rs b/crates/engine/src/cmd/previews.rs index 8a2bf7e53..d3b633895 100644 --- a/crates/engine/src/cmd/previews.rs +++ b/crates/engine/src/cmd/previews.rs @@ -163,8 +163,8 @@ fn smart_run( // atomic: a failed write leaves no partial proxy that would pass for a built one let r = source .ok_or_else(|| "nothing to build from".to_string()) - .and_then(|s| s.load()) - .and_then(|src| crate::smart::encode(&src)) + .and_then(|s| s.load_source()) + .and_then(|src| crate::smart::encode(&src.image, src.info_or(Default::default()).camera_tone.as_ref())) .and_then(|b| lightcraft_catalog::safe_file::write_atomic(&path, &b).map_err(|e| format!("{}: {e}", path.display()))); match r { Ok(()) => { diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rs index 023c37801..7fd8a5962 100644 --- a/crates/engine/src/media.rs +++ b/crates/engine/src/media.rs @@ -76,6 +76,21 @@ pub type PreviewLoader = Arc Option + Send + Sync> pub struct DecodedSource { pub image: Arc, pub info: Option, + /// A smart preview's stored camera tone curve: the one decoder fact its pixels need that the + /// catalog's header facts lack (used when `info` is `None`). + pub camera_tone: Option, +} + +impl DecodedSource { + pub fn new(image: Arc, info: Option) -> Self { + DecodedSource { image, info, camera_tone: None } + } + + /// What to render these pixels against: the decoder's facts, else `header` (the catalog's) + /// with any stored camera tone curve. + pub fn info_or(&self, header: SourceInfo) -> SourceInfo { + self.info.unwrap_or(SourceInfo { camera_tone: self.camera_tone.or(header.camera_tone), ..header }) + } } #[derive(Clone)] @@ -110,24 +125,22 @@ impl SourceRef { SourceRef::Demo { scene, max_edge } => Arc::new(scene.render_fit(*max_edge)), SourceRef::File { path, max_edge, loader, fallback } => { let r = match loader { - Some(l) => l(path, *max_edge).map(|(image, info)| DecodedSource { image: Arc::new(image), info: Some(info) }), + Some(l) => l(path, *max_edge).map(|(image, info)| DecodedSource::new(Arc::new(image), Some(info))), None => Err(format!("no decoder available for {path}")), }; // An offline original renders from its smart preview (no decoder facts: header ones). return match (r, fallback) { #[cfg(not(target_arch = "wasm32"))] - (Err(e), Some(sp)) if crate::smart::is_valid(sp) => { - crate::smart::load(sp).map(|image| DecodedSource { image, info: None }).map_err(|e2| format!("{e}; smart preview: {e2}")) - } + (Err(e), Some(sp)) if crate::smart::is_valid(sp) => crate::smart::load(sp).map_err(|e2| format!("{e}; smart preview: {e2}")), (r, _) => r, }; } #[cfg(not(target_arch = "wasm32"))] - SourceRef::Smart { path } => crate::smart::load(path)?, + SourceRef::Smart { path } => return crate::smart::load(path), #[cfg(target_arch = "wasm32")] SourceRef::Smart { .. } => return Err("smart previews are not available here".into()), }; - Ok(DecodedSource { image, info: None }) + Ok(DecodedSource::new(image, None)) } } @@ -265,16 +278,24 @@ impl MediaCache { } } - fn source_facts(&self, id: PhotoId) -> Option { - self.thumbs - .peek(&id) - .and_then(|s| s.info) - .or_else(|| self.previews.iter().find(|e| e.0 == id).and_then(|e| e.1.info)) - .or_else(|| self.full.as_ref().filter(|e| e.0 == id).and_then(|e| e.1.info)) + /// The facts of photo `id`'s cached sources: the decoder's when one has them, else `header` + /// (with a smart preview's stored camera tone curve). + fn source_facts(&self, id: PhotoId, header: SourceInfo) -> SourceInfo { + let cached = || { + self.thumbs + .peek(&id) + .into_iter() + .chain(self.previews.iter().filter(|e| e.0 == id).map(|e| &e.1)) + .chain(self.full.iter().filter(|e| e.0 == id).map(|e| &e.1)) + }; + match cached().find(|s| s.info.is_some()).or_else(|| cached().next()) { + Some(s) => s.info_or(header), + None => header, + } } pub fn insert(&mut self, id: PhotoId, level: SourceLevel, img: Arc) { - self.insert_source(id, level, DecodedSource { image: img, info: None }); + self.insert_source(id, level, DecodedSource::new(img, None)); } fn insert_source(&mut self, id: PhotoId, level: SourceLevel, img: DecodedSource) { @@ -465,7 +486,7 @@ impl RenderJob { match self.source.load_source() { Ok(source) => { let src = &source.image; - let info = source.info.unwrap_or(self.info); + let info = source.info_or(self.info); // Thumbnails (many small jobs side by side) stay on the CPU; views and exports use // the GPU when there is one. let gpu = self.cache.is_none(); @@ -784,10 +805,8 @@ impl crate::Session { /// Prefer decoder facts to header-only metadata for pixel-statistics commands. pub fn source_info(&self, id: PhotoId) -> SourceInfo { - if let Some(info) = self.media.source_facts(id) { - return info; - } - self.catalog.photo(id).map(|p| source_info(p)).unwrap_or_default() + let header = self.catalog.photo(id).map(|p| source_info(p)).unwrap_or_default(); + self.media.source_facts(id, header) } /// The source proxy for pixel-statistics commands (auto tone/WB), loading synchronously. diff --git a/crates/engine/src/smart.rs b/crates/engine/src/smart.rs index 038f33304..8deff7bde 100644 --- a/crates/engine/src/smart.rs +++ b/crates/engine/src/smart.rs @@ -1,13 +1,16 @@ //! Smart previews: a compact proxy of each photo (its decoded preview-size source, about 1 MB) //! kept in the library, so photos stay editable — and exportable at proxy size — while their //! originals are offline (an unplugged drive). The proxy is the scene-linear source scaled into -//! 0..1 by a stored factor, sRGB-encoded and saved as a JPEG after a one-line header. +//! 0..1 by a stored factor, sRGB-encoded and saved as a JPEG after a one-line header. The header +//! also keeps the decoder's file-local camera tone curve (Sony ARW camera look), which is applied +//! at render time and is not baked into the pixels. use std::path::{Path, PathBuf}; use std::sync::Arc; use lightcraft_catalog::Photo; use lightcraft_color::transfer::linear_to_srgb; +use lightcraft_pipeline::tone::CameraTone; use lightcraft_raster::{Rgb32f, Rgba8}; const MAGIC: &[u8] = b"LCSP1\n"; @@ -18,8 +21,11 @@ pub fn file_name(p: &Photo) -> String { format!("{:032x}.lcsp", h.0) } -/// Encode a source image as a smart preview. -pub fn encode(img: &Rgb32f) -> Result, String> { +/// The most header [`is_valid`] reads (the JSON line before the JPEG). +const HEADER_MAX: u64 = 4096; + +/// Encode a source image (and the decoder's camera tone curve, if any) as a smart preview. +pub fn encode(img: &Rgb32f, tone: Option<&CameraTone>) -> Result, String> { // scale so all but the brightest 0.05 % fit into 0..1 let mut lum: Vec = img.data.iter().map(|c| c[0].max(c[1]).max(c[2])).filter(|v| v.is_finite()).collect(); let scale = if lum.is_empty() { @@ -47,22 +53,29 @@ pub fn encode(img: &Rgb32f) -> Result, String> { ) .map_err(|e| e.to_string())?; let mut out = MAGIC.to_vec(); - out.extend_from_slice(format!("{{\"w\":{},\"h\":{},\"scale\":{scale}}}\n", img.width, img.height).as_bytes()); + let mut head = serde_json::json!({"w": img.width, "h": img.height, "scale": scale}); + if let Some(t) = tone { + head["tone"] = serde_json::to_value(t).map_err(|e| e.to_string())?; + } + out.extend_from_slice(head.to_string().as_bytes()); + out.push(b'\n'); out.extend_from_slice(&jpg); Ok(out) } -/// Decode a smart preview back into a source image. -pub fn decode(bytes: &[u8]) -> Result { +/// Decode a smart preview back into a source image and its stored camera tone curve (an invalid +/// curve is ignored, like a missing one). +pub fn decode(bytes: &[u8]) -> Result<(Rgb32f, Option), String> { let rest = bytes.strip_prefix(MAGIC).ok_or("not a smart preview")?; let nl = rest.iter().position(|b| *b == b'\n').ok_or("bad smart preview")?; let head: serde_json::Value = serde_json::from_slice(&rest[..nl]).map_err(|e| e.to_string())?; let scale = head["scale"].as_f64().unwrap_or(1.0) as f32; + let tone = head.get("tone").and_then(|t| serde_json::from_value::(t.clone()).ok()); let d = lightcraft_codecs::decode(&rest[nl + 1..], Default::default()).map_err(|e| e.to_string())?; // the decoder undoes the sRGB encoding; the values are the source's own primaries let mut img = d.image; img.data.iter_mut().for_each(|c| *c = c.map(|v| v * scale)); - Ok(img) + Ok((img, tone)) } /// Where a library keeps its smart previews. @@ -157,8 +170,8 @@ pub fn migrate(from: &Path, to: &Path, what: Existing) -> (usize, Vec) { pub fn is_valid(path: &Path) -> bool { use std::io::{Read, Seek, SeekFrom}; let Ok(mut f) = std::fs::File::open(path) else { return false }; - let mut head = Vec::with_capacity(256); - if (&mut f).take(256).read_to_end(&mut head).is_err() { + let mut head = Vec::with_capacity(1024); + if (&mut f).take(HEADER_MAX).read_to_end(&mut head).is_err() { return false; } let Some(rest) = head.strip_prefix(MAGIC) else { return false }; @@ -175,9 +188,9 @@ pub fn is_valid(path: &Path) -> bool { } /// Load the proxy at `path`. -pub fn load(path: &Path) -> Result, String> { +pub fn load(path: &Path) -> Result { let b = std::fs::read(path).map_err(|e| format!("{}: {e}", path.display()))?; - decode(&b).map(Arc::new) + decode(&b).map(|(image, camera_tone)| crate::media::DecodedSource { image: Arc::new(image), info: None, camera_tone }) } #[cfg(test)] @@ -228,7 +241,7 @@ mod tests { std::fs::create_dir_all(&a).unwrap(); std::fs::create_dir_all(&b).unwrap(); let img = lightcraft_scenes::demo_library()[0].render(64, 40); - let good = encode(&img).unwrap(); + let good = encode(&img, None).unwrap(); std::fs::write(a.join("1.lcsp"), &good).unwrap(); assert!(is_valid(&a.join("1.lcsp"))); for cut in [good.len() - 1, good.len() / 2, 10, 0] { @@ -242,11 +255,45 @@ mod tests { let _ = std::fs::remove_dir_all(&base); } + #[test] + fn camera_tone_travels_with_the_proxy() { + let tone = CameraTone::new(std::array::from_fn(|i| { + let x = 0.004 * 1.17f32.powi(i as i32); + [x, 0.95 * (1.0 - (-2.7 * x).exp())] + })) + .unwrap(); + let img = lightcraft_scenes::demo_library()[0].render(64, 40); + let bytes = encode(&img, Some(&tone)).unwrap(); + assert_eq!(decode(&bytes).unwrap().1, Some(tone)); + let dir = temp("tone"); + let path = dir.join("t.lcsp"); + std::fs::write(&path, &bytes).unwrap(); + // the longer header is still read by the validity check, and the curve by `load` + assert!(is_valid(&path)); + let loaded = load(&path).unwrap(); + assert_eq!(loaded.camera_tone, Some(tone)); + let header = lightcraft_pipeline::SourceInfo { raw: true, ..Default::default() }; + assert_eq!(loaded.info_or(header).camera_tone, Some(tone)); + // a hostile curve (reversing knots) is ignored, not trusted + let nl = MAGIC.len() + bytes[MAGIC.len()..].iter().position(|b| *b == b'\n').unwrap(); + let mut head: serde_json::Value = serde_json::from_slice(&bytes[MAGIC.len()..nl]).unwrap(); + let mut knots: Vec<[f32; 2]> = serde_json::from_value(head["tone"]["knots"].clone()).unwrap(); + knots.swap(3, 4); + head["tone"]["knots"] = serde_json::json!(knots); + let mut hostile = MAGIC.to_vec(); + hostile.extend_from_slice(head.to_string().as_bytes()); + hostile.extend_from_slice(&bytes[nl..]); + let (pixels, tone) = decode(&hostile).unwrap(); + assert_eq!((pixels.width, tone), (img.width, None)); + let _ = std::fs::remove_dir_all(&dir); + } + #[test] fn roundtrip_keeps_the_picture() { let img = lightcraft_scenes::demo_library()[0].render(160, 100); - let back = decode(&encode(&img).unwrap()).unwrap(); + let (back, tone) = decode(&encode(&img, None).unwrap()).unwrap(); assert_eq!((back.width, back.height), (img.width, img.height)); + assert!(tone.is_none()); let mut err = 0.0f64; for (a, b) in img.data.iter().zip(&back.data) { for k in 0..3 { From 5b414ffb8e64246962c8b2ba424ee3a34fbc6531 Mon Sep 17 00:00:00 2001 From: Brandon Thomas Date: Mon, 5 Oct 2026 19:34:01 -0400 Subject: [PATCH 5/5] LR-PROF-CAMERACOLOR: record the ARW file-local fit's public-sample results Co-Authored-By: Claude Opus 5.5 --- ROADMAP.md | 3 ++- docs/camera-preview-colour.md | 4 +++- docs/parity.md | 4 ++-- 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index bb0011efa..6af9d726a 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -46,7 +46,8 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78). Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*. 1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use - matrices the files carry themselves, then chart shots. ARW preview-derived fitting is a guarded first step; expand coverage and validate fidelity. + matrices the files carry themselves, then chart shots. Sony ARW's file-local fit (matrix + tone curve from its own + JPEG) is the first step; generalise it to the other makes' raws (NEF, RW2, PEF, ORF…), then validate fidelity. 2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): **CR3** first, then compressed RAF / ORF, RW2 v4, NEF lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source, no LGPL dependency); compressed NEF (#86) is the template. diff --git a/docs/camera-preview-colour.md b/docs/camera-preview-colour.md index 0aef3452e..c47546640 100644 --- a/docs/camera-preview-colour.md +++ b/docs/camera-preview-colour.md @@ -14,7 +14,7 @@ One third of the pixels are held out from both fits. The combined model must red ## Source metadata and white balance -Previously `SourceRef::load()` discarded the decoder's `SourceInfo`, and render jobs used header-only facts (including a hard-coded RAW WB). Decoded pixels and source facts now travel and remain cached together, including across web-worker renders. Pixel-statistics/colour-sampling commands use decoded facts too. Render cache version 6 invalidates old thumbnails and loupe renders. +Previously `SourceRef::load()` discarded the decoder's `SourceInfo`, and render jobs used header-only facts (including a hard-coded RAW WB). Decoded pixels and source facts now travel and remain cached together, including across web-worker renders. Smart previews keep the tone curve in their header (the matrix is already in their pixels), so an offline original renders and exports with the same look; proxies built before this fall back to the generic curve until rebuilt. Pixel-statistics/colour-sampling commands use decoded facts too. Render cache version 6 invalidates old thumbnails and loupe renders. Vendor RGB WB multipliers alone do **not** identify an absolute illuminant without camera calibration. The generic model inferred approximately 6829 K / −127 tint for one Sony sample; changing only temperature retained that spurious tint and caused a pink cast. Uncalibrated ARWs now keep the file's vendor WB at decode and expose **relative** WB controls around the as-shot look. 6500/0 is the internal neutral reference of that relative scale, not a claim about the capture's measured Kelvin temperature. Presets, picker/auto and manual controls use this reference consistently. Switching from As Shot to Custom first resolves both controls; changing one cannot inherit stale catalog tint. Reset restores the current as-shot reference. Measured/absolute-Kelvin Sony WB remains future camera-calibration work. @@ -26,6 +26,8 @@ Synthetic tests cover colour with a nonlinear camera tone, preserved scene-linea Five local Sony ILCE-6700 ARWs were accepted. Held-out per-channel linear-display RMS values were 0.04244, 0.03349, 0.04552, 0.02871 and 0.03100. All five unedited supplied JPEG exports stayed pixel-identical to the installed pre-change renderer at equal size/quality. Four supplied JPEGs use Display P3 and two have a narrower crop than the RAW, so they are visual references rather than aligned pixel ground truth. +Public CC0 samples (raw.pixls.us; 10 files, ILCE-6000, -6400 ×2, -6700, -7M3, -7M4 ×2, -7RM4, -7SM3, DSC-RX100M3), rendered at 1200 px and compared with each file's embedded JPEG (mean CIE76 ΔE at 200 px): 7 fits accepted, ΔE 17–26 → 3–10 with matching mean lightness. Rejected and unchanged: the RX100M3 (held-out RMS 0.103; its fallback render already has a strong green cast), an ILCE-7RM4 dusk scene whose fit was worse than the fallback (ΔE 7.6), and a reduced-size lossless ARW (no fit attempted). Saturated hues can still differ (an orange-red hull renders yellow). + On one sample, Exposure +1 EV, Saturation +100, Contrast +50, warm/cool WB, tint, highlights and shadows all changed the output. The same matrix and curve were selected for 400-pixel and full-resolution renders. Headless UI checks wait until the loupe reports `source: render` and no jobs remain; the camera JPEG stand-in does not count as verification. This is a per-file camera-look estimate, **not measured spectral calibration or Lightroom parity**. Camera local tone, picture styles, noise reduction and lens warps cannot all be reproduced by a global matrix/curve; same-aspect crops may evade the aspect check. Skin, mixed-light scenes and saturated highlights can still differ from a supplied JPEG. Broader Sony model/lighting coverage, chart-based calibration, absolute WB and other manufacturers remain unverified. Rejected fits and other RAW formats retain their existing behaviour. diff --git a/docs/parity.md b/docs/parity.md index 4af6fd823..6a00a1a14 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -59,7 +59,7 @@ Ordered by user impact, then tier, then effort. The checklist above counts featu that decide whether a photographer can switch (see the honest assessment in [ROADMAP.md](../ROADMAP.md#where-we-stand)). Take the first one nobody is working on. -1. **LR-PROF-CAMERACOLOR** (P0): our own camera colour calibration. ARW now has guarded per-file embedded-JPEG colour/tone fitting and relative WB; measured calibration and fidelity remain missing. Other non-DNG raws and rejected fits still use a neutral matrix. Expand validated preview fitting and use matrices the files carry themselves; never Adobe data. +1. **LR-PROF-CAMERACOLOR** (P0): our own camera colour calibration. Sony ARW now gets a guarded file-local fit to its own embedded JPEG (colour matrix + tone curve, relative WB; 7 of 10 public samples accepted, mean ΔE vs the camera JPEG 17–26 → 3–10); measured calibration and fidelity remain missing. Other non-DNG raws and rejected fits still use a neutral matrix. Expand validated preview fitting and use matrices the files carry themselves; never Adobe data. 2. **LR-IMP-FORMATS** (P0): **CR3** first (every Canon body since ~2018), then compressed RAF / ORF, RW2 v4, Nikon "lossy after split" NEF, Canon sRAW; HEIC/AVIF decode. Clean-room, from prose descriptions only (see `crates/raw/src/vendor/nefc.rs` for how compressed NEF was done). Until decoded, such photos are `preview_only`. @@ -274,7 +274,7 @@ under the UI command that wraps it (a no-op error); the UI command now wins. | LR-PROF-ADOBE | Standard raw looks (own equivalents) | P0 | ✅ | `crates/engine/src/presets.rs` (`PROFILES`), `crates/pipeline/src/profiles.rs` | six own looks: Color, Neutral, Vivid, Landscape, Portrait, Monochrome | | LR-PROF-ADAPTIVE | Adaptive profiles | P2 | ⬜ | | | | LR-PROF-CAMERA | Camera-matching looks | P2 | ⬜ | | | -| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices they carry; ARW has guarded per-file embedded-JPEG chromaticity/tone fitting and relative WB (see docs/camera-preview-colour.md), not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | +| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices they carry; Sony ARW gets a guarded file-local fit to its own embedded JPEG — chromaticity matrix + tone curve, relative WB, kept with smart previews (see docs/camera-preview-colour.md); on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | | LR-PROF-CREATIVE | Creative profiles (own) | P2 | ✅ | `cmd:develop.profile`, `crates/pipeline/src/profiles.rs` | 16 own looks in Film / Cinematic / Muted / B&W (tone + point-curve fades, colour grading, mixer / B&W mix); scale with `ctl:profile.amount`; sliders untouched | | LR-PROF-LEGACY | Legacy profiles | P2 | ⬜ | | | | LR-PROF-NONRAW | Profiles for non-raw files | P0 | ✅ | `cmd:develop.profile` | same looks apply to JPEG/TIFF |