diff --git a/ROADMAP.md b/ROADMAP.md index c790c5662..a0836d771 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -24,7 +24,7 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78). |---|---:|---|---| | **Feature checklist** | 79% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim | | **RAW coverage** (formats people shoot) | ~50% | DNG (all kinds), CR2, ARW, NEF (uncompressed + Huffman lossless/lossy), uncompressed RAF/ORF, packed RW2, PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed RAF/ORF, RW2 v4, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. Per-model verification is thin (~40 corpus files vs >1,000 models) | -| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No camera colour calibration**: non-DNG raws use a neutral matrix, so colour is muted/inaccurate. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) | +| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW has a guarded per-file embedded-JPEG colour estimate (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) | | **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) | | **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering | | **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic) | **Map view, Book, Print, Slideshow module, Web, publish services**: ~40 tracker rows ⬜ | @@ -36,7 +36,7 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78). | User | Readiness | What blocks them | |---|---:|---| | JPEG / DNG shooter, single machine | ~85% | Fidelity polish, AI masks | -| Nikon / Sony / older-Canon raw shooter | ~65% | Camera colour (muted colour on every photo) | +| Nikon / Sony / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW preview estimates are only a starting point) | | Canon CR3 / Fujifilm / Olympus shooter | ~35% | Their raws open as embedded previews only | | Lightroom Classic power user | ~45% | Print, Book, Map, publish, tethering | | Relies on AI (masks, denoise) | ~25% | No segmentation or AI denoise models | @@ -46,7 +46,8 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78). Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*. 1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use - matrices the files carry themselves, then chart shots. Raises colour quality for every non-DNG raw at once. + matrices the files carry themselves, then chart shots. Sony ARW's file-local fit (matrix + tone curve from its own + JPEG) is the first step; generalise it to the other makes' raws (NEF, RW2, PEF, ORF…), then validate fidelity. 2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): **CR3** first, then compressed RAF / ORF, RW2 v4, NEF lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source, no LGPL dependency); compressed NEF (#86) is the template. @@ -81,7 +82,7 @@ hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help | M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) | | M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ | | M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs ✅; JXL encode, HDR export ⬜) | -| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; **camera colour calibration** ⬜ (top priority), CR3, compressed ORF/RAF ⬜) | +| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW preview fitting; measured database still missing), CR3, compressed ORF/RAF ⬜) | | M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ | | M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ | | M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ | @@ -175,7 +176,7 @@ Not decoded yet — preview only (no permissively licensed description; black-bo from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established. - **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG. -**Camera colour matrices:** non-DNG raws use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged +**Camera colour matrices:** ARW files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged `matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our own chart-based calibration (M11.4). Adobe matrices are never used. diff --git a/apps/lightcraft-web/src/wire.rs b/apps/lightcraft-web/src/wire.rs index 2e546eb23..05960f7ef 100644 --- a/apps/lightcraft-web/src/wire.rs +++ b/apps/lightcraft-web/src/wire.rs @@ -15,10 +15,9 @@ use std::sync::Arc; use lightcraft_engine::catalog::Source; use lightcraft_engine::develop::{DevelopSettings, EmbeddedLens}; -use lightcraft_engine::media::{MediaCache, RenderJob, SourceLevel, SourceRef}; +use lightcraft_engine::media::{DecodedSource, MediaCache, RenderJob, SourceLevel, SourceRef}; use lightcraft_engine::pipeline::{Quality, RenderRequest, Rendered, SourceInfo, StageCache}; use lightcraft_preview::Lru; -use lightcraft_raster::Rgb32f; use serde::{Deserialize, Serialize}; use crate::store::hash_of_path; @@ -31,6 +30,8 @@ pub struct WireJob { /// Decode the source at most this long. pub max_edge: usize, pub raw: bool, + #[serde(default)] + pub relative_wb: bool, pub as_shot_temp: f64, pub as_shot_tint: f64, pub lens: Option, @@ -55,7 +56,7 @@ impl WireJob { pub fn from_job(job: &RenderJob, thumb_cached: bool, view: &str) -> WireJob { let max_edge = match &job.source { SourceRef::Demo { max_edge, .. } | SourceRef::File { max_edge, .. } => *max_edge, - SourceRef::Loaded(img) => img.width.max(img.height), + SourceRef::Loaded(img) => img.image.width.max(img.image.height), SourceRef::Smart { .. } => 2560, }; WireJob { @@ -63,6 +64,7 @@ impl WireJob { origin: job.origin.clone(), max_edge, raw: job.info.raw, + relative_wb: job.info.relative_wb, as_shot_temp: job.info.as_shot_temp, as_shot_tint: job.info.as_shot_tint, lens: job.info.lens, @@ -79,7 +81,14 @@ impl WireJob { } pub fn info(&self) -> SourceInfo { - SourceInfo { raw: self.raw, as_shot_temp: self.as_shot_temp, as_shot_tint: self.as_shot_tint, lens: self.lens } + SourceInfo { + raw: self.raw, + as_shot_temp: self.as_shot_temp, + as_shot_tint: self.as_shot_tint, + lens: self.lens, + relative_wb: self.relative_wb, + ..Default::default() + } } pub fn request(&self) -> RenderRequest { @@ -119,7 +128,7 @@ const WORKER_SOURCE_BYTES: usize = 192 << 20; /// The synchronous part of a worker: decoded-source cache + rendering. pub struct WorkerCore { - sources: Lru>, + sources: Lru, media: MediaCache, stages: HashMap>, } @@ -150,21 +159,25 @@ impl WorkerCore { Some(s) => s, None => { let src = match (&job.origin, original) { - (Source::File { .. }, Some(bytes)) => Arc::new(lightcraft_engine::files::load_bytes(bytes, job.max_edge)?.0), + (Source::File { .. }, Some(bytes)) => { + let (image, info) = lightcraft_engine::files::load_bytes(bytes, job.max_edge)?; + DecodedSource::new(Arc::new(image), Some(info)) + } (Source::File { path }, None) => return Err(format!("{path}: original not found in browser storage")), - (origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load()?, + (origin @ Source::Demo { .. }, _) => self.media.origin_ref(origin, job.max_edge).load_source()?, }; - let cost = src.width * src.height * 12 + 64; + let cost = src.image.width * src.image.height * 12 + 64; self.sources.insert(key, src.clone(), cost); src } }; + let info = src.info_or(job.info()); Ok(match &job.stages { Some(view) => { let st = self.stages.entry(view.clone()).or_default().clone(); - lightcraft_engine::pipeline::render_cached(&src, &job.info(), &job.settings, &job.request(), &st) + lightcraft_engine::pipeline::render_cached(&src.image, &info, &job.settings, &job.request(), &st) } - None => lightcraft_engine::pipeline::render(&src, &job.info(), &job.settings, &job.request()), + None => lightcraft_engine::pipeline::render(&src.image, &info, &job.settings, &job.request()), }) } } @@ -295,6 +308,7 @@ mod tests { origin, max_edge: 512, raw: false, + relative_wb: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None, diff --git a/crates/catalog/src/model.rs b/crates/catalog/src/model.rs index f8e518464..8b400bb52 100644 --- a/crates/catalog/src/model.rs +++ b/crates/catalog/src/model.rs @@ -331,6 +331,11 @@ impl Photo { pub fn develops_raw(&self) -> bool { self.kind == MediaKind::Raw && self.preview_only.is_none() } + /// The current ARW reader has vendor WB multipliers but no measured camera illuminant. + /// Use adjustments relative to the camera's as-shot look, as for rendered photographs. + pub fn relative_wb(&self) -> bool { + self.develops_raw() && self.format.eq_ignore_ascii_case("ARW") + } /// The develop settings import gave this photo: [`Photo::camera_defaults`], or the user's /// default preset applied on top of them ([`Photo::import_look`]). pub fn import_defaults(&self) -> DevelopSettings { @@ -342,7 +347,8 @@ impl Photo { /// The built-in defaults for this photo, before any user default preset: raws start from /// their as-shot white balance; embedded lens corrections on when the file has them. pub fn camera_defaults(&self) -> DevelopSettings { - let mut d = match self.as_shot_wb { + let wb = if self.relative_wb() { Some((6500.0, 0.0)) } else { self.as_shot_wb }; + let mut d = match wb { Some((t, tint)) => DevelopSettings::for_raw(t, tint), None => DevelopSettings::default(), }; diff --git a/crates/engine/src/camera_preview.rs b/crates/engine/src/camera_preview.rs new file mode 100644 index 000000000..54b308b13 --- /dev/null +++ b/crates/engine/src/camera_preview.rs @@ -0,0 +1,272 @@ +//! Estimate an ARW starting look from its own JPEG. Colour and luminance are fitted separately; +//! the JPEG supplies correspondences only, never output pixels or a replacement for RAW editing. +use lightcraft_color::{Mat3, luminance_2020}; +use lightcraft_pipeline::tone::{CameraTone, ToneMap}; +use lightcraft_raster::{ + Rgb32f, + resample::{Filter, fit}, +}; +use lightcraft_raw::{RawImage, color::CameraTransform}; + +#[derive(Clone, Copy, Debug)] +pub(crate) struct CameraLook { + pub matrix: Mat3, + pub tone: CameraTone, +} + +pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option { + if !transform.matrix_is_fallback || raw.format != lightcraft_raw::RawFormat::Arw { + return None; + } + let jpeg = lightcraft_raw::embedded_preview(bytes)?; + let decoded = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions { max_size: Some((384, 384)), max_pixels: 64_000_000 }).ok()?; + let reference = decoded.to_working(); + let crop = raw.crop.clipped(raw.active_area.width, raw.active_area.height); + if crop.width == 0 || crop.height == 0 || reference.width == 0 || reference.height == 0 { + return None; + } + let aspect = crop.width as f64 / crop.height as f64; + if (reference.width as f64 / reference.height as f64 / aspect - 1.0).abs() > 0.02 { + return None; + } + // Fixed, bounded proxy: the selected look cannot depend on thumbnail/export resolution. + let k = (crop.width.max(crop.height).div_ceil(384).max(2)).div_ceil(2) * 2; + let sensor = raw.develop_binned(k, 0.99).ok()??; + let mut sensor = fit(&sensor, 96, 96, Filter::Box); + let reference = fit(&reference, sensor.width, sensor.height, Filter::Box); + let gain = 2f32.powf(transform.baseline_exposure as f32); + sensor.map_in_place(|p| transform.matrix.apply_f32(std::array::from_fn(|i| p[i] * transform.wb[i] * gain))); + let look = fit_pairs(&sensor, &reference)?; + if lightcraft_pipeline::profiling() { + eprintln!("[profile] ARW camera look: {:?}, {:?}", look.matrix.0, look.tone); + } + Some(look) +} + +/// A fit must cut the held-out squared error to below this share of the fallback's. +const MIN_IMPROVEMENT: f64 = 0.7; +/// ...and stay within this per-channel RMS of the camera JPEG (linear display values). On public +/// raw.pixls.us samples (eight Sony bodies) good fits that still beat the fallback 1.5–3× landed +/// at 0.065–0.093 (camera local tone, vignetting and lens processing that a global matrix + curve +/// cannot follow): visibly better renders that 0.055 rejected. +const MAX_HOLDOUT_RMS: f64 = 0.10; + +fn luma(p: [f64; 3]) -> f64 { + p[0] * 0.2627 + p[1] * 0.6780 + p[2] * 0.0593 +} + +fn displayed(scene: [f64; 3], tone: &ToneMap) -> [f64; 3] { + let scene = scene.map(|v| v.max(0.0)); + let y = luma(scene); + if y <= 0.0 { + return [0.0; 3]; + } + scene.map(|v| v * f64::from(tone.apply(y as f32)) / y) +} + +fn fit_pairs(sensor: &Rgb32f, reference: &Rgb32f) -> Option { + if (sensor.width, sensor.height) != (reference.width, reference.height) || sensor.data.len() != reference.data.len() { + return None; + } + let mut pairs = Vec::new(); + let mut colour = 0; + for (input, output) in sensor.data.iter().zip(&reference.data) { + let y = luminance_2020(*output); + if !input.iter().all(|v| v.is_finite() && *v > 0.001 && *v < 1.5) + || !output.iter().all(|v| v.is_finite() && *v > 0.004 && *v < 0.98) + || !(0.015..0.85).contains(&y) + { + continue; + } + let min = output.iter().copied().fold(f32::INFINITY, f32::min); + let max = output.iter().copied().fold(0.0, f32::max); + colour += usize::from(max - min > 0.05); + pairs.push((input.map(f64::from), output.map(f64::from))); + } + if pairs.len() < 256 || colour < pairs.len() / 20 { + return None; + } + let mut gram = [[0.0; 3]; 3]; + let mut cross = [[0.0; 3]; 3]; + for (i, (x, y)) in pairs.iter().enumerate() { + if i % 3 == 0 { + continue; + } + let (lx, ly) = (luma(*x), luma(*y)); + for row in 0..3 { + for col in 0..3 { + // Normalising by luminance prevents a camera S-curve from corrupting colour. + gram[row][col] += x[row] * x[col] / (lx * lx); + cross[row][col] += y[row] * x[col] / (ly * lx); + } + } + } + let trace: f64 = (0..3).map(|i| gram[i][i]).sum(); + let inverse = Mat3(gram).inverse()?; + let condition = trace * (0..3).map(|i| inverse.0[i][i].abs()).sum::(); + if trace <= 0.0 || !condition.is_finite() || condition > 1e6 { + return None; + } + let regularization = trace * 1e-4; + for i in 0..3 { + gram[i][i] += regularization; + cross[i][i] += regularization; + } + let matrix = Mat3(cross).mul(&Mat3(gram).inverse()?); + if !matrix.0.iter().flatten().all(|v| v.is_finite() && v.abs() < 8.0) { + return None; + } + let tone_pairs: Vec<_> = + pairs.iter().enumerate().filter(|(i, _)| i % 3 != 0).map(|(_, (x, y))| (luma(matrix.apply(*x).map(|v| v.max(0.0))), luma(*y))).collect(); + let curve = fit_tone(tone_pairs)?; + let tone = ToneMap::camera(&curve, 0.0, 0.0, 0.0); + let original_tone = ToneMap::new(0.0, 0.0, 0.0); + let mut before = 0.0; + let mut after = 0.0; + let mut samples = 0; + for (i, (x, target)) in pairs.iter().enumerate() { + if i % 3 != 0 { + continue; + } + let corrected = displayed(matrix.apply(*x), &tone); + let original = displayed(*x, &original_tone); + for c in 0..3 { + before += (original[c] - target[c]).powi(2); + after += (corrected[c] - target[c]).powi(2); + samples += 1; + } + } + if lightcraft_pipeline::profiling() { + eprintln!("[profile] ARW holdout RMS {:.5} -> {:.5} ({samples} channels)", (before / samples as f64).sqrt(), (after / samples as f64).sqrt()); + } + if !after.is_finite() || samples == 0 || after >= before * MIN_IMPROVEMENT || after / samples as f64 > MAX_HOLDOUT_RMS.powi(2) { + return None; + } + Some(CameraLook { matrix, tone: curve }) +} + +fn median(values: &mut [f64]) -> Option { + values.sort_by(f64::total_cmp); + values.get(values.len() / 2).copied() +} + +fn fit_tone(mut pairs: Vec<(f64, f64)>) -> Option { + if pairs.len() < 128 || !pairs.iter().all(|(x, y)| x.is_finite() && *x > 0.0 && y.is_finite()) { + return None; + } + pairs.sort_by(|a, b| a.0.total_cmp(&b.0)); + let mut knots = [[0.0; 2]; 32]; + for (i, knot) in knots.iter_mut().enumerate() { + let bin = pairs.get(i * pairs.len() / 32..(i + 1) * pairs.len() / 32)?; + let mut xs: Vec<_> = bin.iter().map(|p| p.0).collect(); + let mut ys: Vec<_> = bin.iter().map(|p| p.1).collect(); + *knot = [median(&mut xs)? as f32, median(&mut ys)? as f32]; + } + if knots[31][0] < knots[0][0] * 1.5 { + return None; + } + // Pool adjacent violating bins (isotonic regression): no reversals or arbitrary polynomial. + let mut blocks: Vec<(f32, usize)> = Vec::new(); + for knot in knots { + blocks.push((knot[1], 1)); + while blocks.len() >= 2 { + let (a, an) = *blocks.get(blocks.len() - 2)?; + let (b, bn) = *blocks.last()?; + if a <= b { + break; + } + blocks.truncate(blocks.len() - 2); + blocks.push(((a * an as f32 + b * bn as f32) / (an + bn) as f32, an + bn)); + } + } + let mut i = 0; + for (y, n) in blocks { + for knot in knots.get_mut(i..i + n)? { + knot[1] = y; + } + i += n; + } + CameraTone::new(knots) +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn separates_nonlinear_tone_from_colour_and_keeps_sensor_headroom() { + let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]); + let mut sensor = Rgb32f::new(64, 64); + let mut reference = sensor.clone(); + for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() { + let ev = 0.05 + (i % 31) as f32 * 0.017; + *src = [ev * (0.8 + (i % 11) as f32 * 0.025), ev, ev * (0.8 + (i % 17) as f32 * 0.014)]; + let p = known.apply_f32(*src); + let y = luminance_2020(p); + *dst = p.map(|v| v * (1.0 - (-2.5 * y).exp()) / y); + } + let original = sensor.clone(); + let fit = fit_pairs(&sensor, &reference).unwrap(); + assert_eq!(sensor.data, original.data); + let tone = ToneMap::camera(&fit.tone, 0.0, 0.0, 0.0); + let error: f64 = sensor + .data + .iter() + .zip(&reference.data) + .map(|(x, y)| { + let p = displayed(fit.matrix.apply(x.map(f64::from)), &tone); + (0..3).map(|c| (p[c] - f64::from(y[c])).powi(2)).sum::() / 3.0 + }) + .sum::() + / sensor.data.len() as f64; + assert!(error.sqrt() < 0.025, "{error}"); + // The colour transform is homogeneous; tone mapping happens only after exposure. + let p = fit.matrix.apply([2.0, 2.0, 2.0]); + assert!(luma(p) > 1.0); + assert!(tone.apply(0.2) < tone.apply(0.4)); + } + #[test] + fn accepts_a_much_better_fit_despite_local_camera_processing() { + // The camera JPEG departs from any global matrix + curve (local tone, vignetting): ±0.12 + // per-pixel deviations, ~0.07 RMS. The fit is still far closer than the fallback. + let known = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]); + let mut sensor = Rgb32f::new(64, 64); + let mut reference = sensor.clone(); + let mut seed = 0x2545_f491_u32; + for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() { + let ev = 0.05 + (i % 31) as f32 * 0.017; + *src = [ev * (0.8 + (i % 11) as f32 * 0.025), ev, ev * (0.8 + (i % 17) as f32 * 0.014)]; + let p = known.apply_f32(*src); + let y = luminance_2020(p); + *dst = p.map(|v| { + seed ^= seed << 13; + seed ^= seed >> 17; + seed ^= seed << 5; + let noise = (seed % 2001) as f32 / 1000.0 - 1.0; + (v * (1.0 - (-2.5 * y).exp()) / y + 0.12 * noise).clamp(0.005, 0.97) + }); + } + assert!(fit_pairs(&sensor, &reference).is_some()); + } + + #[test] + fn rejects_monochrome_invalid_and_unrelated_previews() { + let mut sensor = Rgb32f::new(32, 32); + let mut reference = sensor.clone(); + for (i, p) in sensor.data.iter_mut().enumerate() { + *p = [0.1 + (i % 13) as f32 * 0.02, 0.15, 0.1]; + } + reference.data.fill([0.2; 3]); + assert!(fit_pairs(&sensor, &reference).is_none()); + reference.data.fill([f32::NAN; 3]); + assert!(fit_pairs(&sensor, &reference).is_none()); + for (i, (src, dst)) in sensor.data.iter_mut().zip(&mut reference.data).enumerate() { + *src = [0.04 + (i % 11) as f32 * 0.02, 0.05 + (i % 17) as f32 * 0.01, 0.03 + (i % 23) as f32 * 0.01]; + *dst = [0.05 + (i % 7) as f32 * 0.07, 0.05 + (i % 19) as f32 * 0.02, 0.05 + (i % 29) as f32 * 0.01]; + } + assert!(fit_pairs(&sensor, &reference).is_none()); + sensor.data.fill([0.1, 0.15, 0.12]); + assert!(fit_pairs(&sensor, &reference).is_none()); + reference.data.truncate(8); + assert!(fit_pairs(&sensor, &reference).is_none()); + } +} diff --git a/crates/engine/src/cmd/color.rs b/crates/engine/src/cmd/color.rs index 65a9a709e..55ea8b5ab 100644 --- a/crates/engine/src/cmd/color.rs +++ b/crates/engine/src/cmd/color.rs @@ -26,7 +26,7 @@ const PROBE_EDGE: usize = 384; pub(crate) fn probe(s: &mut Session, c: &str, x: f64, y: f64, neutral: impl FnOnce(&mut DevelopSettings)) -> Result<[f32; 3]> { let id = s.active().ok_or_else(|| bad(c, "no active photo"))?; let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad(c, e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let mut d = (*s.develop_of(id).unwrap_or_default()).clone(); neutral(&mut d); let req = lightcraft_pipeline::RenderRequest::fit(PROBE_EDGE, PROBE_EDGE); diff --git a/crates/engine/src/cmd/develop.rs b/crates/engine/src/cmd/develop.rs index 4d955ac5b..8f197429c 100644 --- a/crates/engine/src/cmd/develop.rs +++ b/crates/engine/src/cmd/develop.rs @@ -98,7 +98,11 @@ pub fn specs() -> Vec { } } let label = if vals.len() == 1 { controls::find(&vals[0].0).map(|c| c.label).unwrap_or("Edit").to_string() } else { "Edit".into() }; - let apply = |d: &mut DevelopSettings| { + let apply = |d: &mut DevelopSettings, info: &lightcraft_pipeline::SourceInfo| { + if d.wb.mode == WbMode::AsShot && vals.iter().any(|(k, _)| k == "wb.temp" || k == "wb.tint") { + d.wb.temp = info.as_shot_temp; + d.wb.tint = info.as_shot_tint; + } for (k, v) in &vals { controls::set(d, k, *v); if k == "wb.temp" || k == "wb.tint" { @@ -113,15 +117,18 @@ pub fn specs() -> Vec { .filter_map(|id| s.develop_of(*id).map(|d| (*id, d))) .filter_map(|(id, d)| { let mut d = (*d).clone(); - apply(&mut d); + let info = s.source_info(id); + apply(&mut d, &info); s.develop_op(id, d, &label) }) .collect(); s.commit(&label, Op::Batch { ops })?; return ok(); } + let id = active(s, "develop.set")?; + let info = s.source_info(id); edit(s, "develop.set", &label, |d| { - apply(d); + apply(d, &info); Ok(()) }) } @@ -130,7 +137,15 @@ pub fn specs() -> Vec { let c = str_param(p, "control").ok_or_else(|| bad("develop.adjust", "missing control"))?.to_string(); let spec = controls::find(&c).ok_or_else(|| bad("develop.adjust", "unknown control"))?; let delta = f64_req(p, "delta", "develop.adjust")?; + let info = s.source_info(active(s, "develop.adjust")?); edit(s, "develop.adjust", spec.label, |d| { + if c == "wb.temp" || c == "wb.tint" { + if d.wb.mode == WbMode::AsShot { + d.wb.temp = info.as_shot_temp; + d.wb.tint = info.as_shot_tint; + } + d.wb.mode = WbMode::Custom; + } let v = controls::get(d, &c).unwrap_or(spec.default); controls::set(d, &c, v + delta); Ok(()) @@ -149,12 +164,18 @@ pub fn specs() -> Vec { let ops = ids .iter() .filter_map(|id| s.develop_of(*id).map(|d| (*id, d))) - .filter_map(|(id, d)| { + .filter_map(|(id, _d)| { // back to the photo's import defaults when a default preset gave it its look let look = s.catalog.photo(id).and_then(|p| p.import_look.clone()); let fresh = match look { Some(l) => (*l).clone(), - None => DevelopSettings { wb: lightcraft_develop::WhiteBalance { mode: WbMode::AsShot, ..d.wb }, ..Default::default() }, + None => { + let info = s.source_info(id); + DevelopSettings { + wb: lightcraft_develop::WhiteBalance { mode: WbMode::AsShot, temp: info.as_shot_temp, tint: info.as_shot_tint }, + ..Default::default() + } + } }; s.develop_op(id, fresh, "Reset") }) @@ -203,7 +224,7 @@ pub fn specs() -> Vec { |s, _| { let id = active(s, "develop.autoBwMix")?; let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.autoBwMix", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); let m = lightcraft_pipeline::auto::auto_bw_mix(&src, &info, &d); edit(s, "develop.autoBwMix", "Auto B&W Mix", |d| { @@ -218,7 +239,7 @@ pub fn specs() -> Vec { cmd!("develop.auto", "Auto Settings", ["Photo"], Some("Shift+A"), "{}", has_active, |s, _| { let id = active(s, "develop.auto")?; let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.auto", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); let a = lightcraft_pipeline::auto::auto_tone(&src, &info, &d); edit(s, "develop.auto", "Auto", |d| { @@ -245,7 +266,7 @@ pub fn specs() -> Vec { let mode: WbMode = serde_json::from_value(p.get("mode").cloned().unwrap_or(json!("custom"))).map_err(|e| bad("develop.wb", e.to_string()))?; let id = active(s, "develop.wb")?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let (mut t, mut tint) = match mode { WbMode::AsShot => (info.as_shot_temp, info.as_shot_tint), WbMode::Auto => { @@ -256,7 +277,7 @@ pub fn specs() -> Vec { .preset() .map(|(t, ti)| { // presets are relative to daylight for rendered files - if info.raw { (t, ti) } else { (6500.0 * t / 5500.0, ti) } + if info.raw && !info.relative_wb { (t, ti) } else { (6500.0 * t / 5500.0, ti) } }) .unwrap_or((info.as_shot_temp, info.as_shot_tint)), }; @@ -277,7 +298,7 @@ pub fn specs() -> Vec { let (x, y) = (f64_req(p, "x", "develop.wbPick")?, f64_req(p, "y", "develop.wbPick")?); let id = active(s, "develop.wbPick")?; let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad("develop.wbPick", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); // Sample a small neighbourhood in the oriented source. let o = src.oriented(d.orientation); @@ -405,7 +426,7 @@ pub fn specs() -> Vec { |s, _| { let id = active(s, "crop.autoStraighten")?; let src = s.source_now(id, SourceLevel::Preview).map_err(|e| bad("crop.autoStraighten", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); let Some(level) = lightcraft_pipeline::upright::level_degrees(&src, &info, &d) else { return Ok(json!({"changed": false, "reason": "no dominant horizontal or vertical lines"})); @@ -501,7 +522,7 @@ pub fn specs() -> Vec { Upright::Off | Upright::Guided => None, m => { let src = s.source_now(id, SourceLevel::Preview).map_err(|e| bad("geometry.upright", e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); Some(lightcraft_pipeline::upright::auto_transform(&src, &info, &d, m)) } }; @@ -657,6 +678,11 @@ pub fn specs() -> Vec { .filter_map(|id| s.develop_of(id).map(|d| (id, d))) .filter_map(|(id, d)| { let mut nd = (*d).clone(); + if nd.wb.mode == WbMode::AsShot && (ctl == "wb.temp" || ctl == "wb.tint") { + let info = s.source_info(id); + nd.wb.temp = info.as_shot_temp; + nd.wb.tint = info.as_shot_tint; + } let v = controls::get(&nd, &ctl).unwrap_or(spec.default); controls::set(&mut nd, &ctl, v + delta); if ctl == "wb.temp" || ctl == "wb.tint" { diff --git a/crates/engine/src/cmd/masks.rs b/crates/engine/src/cmd/masks.rs index fcf918c47..ec3ffac07 100644 --- a/crates/engine/src/cmd/masks.rs +++ b/crates/engine/src/cmd/masks.rs @@ -185,7 +185,7 @@ pub fn specs() -> Vec { let add = p.get("add").and_then(Value::as_bool).unwrap_or(false); let id = s.active().ok_or_else(|| bad(c, "no active photo"))?; let src = s.source_now(id, crate::media::SourceLevel::Thumb).map_err(|e| bad(c, e))?; - let info = s.catalog.photo(id).map(|p| crate::media::source_info(p)).unwrap_or_default(); + let info = s.source_info(id); let d = s.develop_of(id).unwrap_or_default(); let req = lightcraft_pipeline::RenderRequest::fit(384, 384); let lab = lightcraft_pipeline::color_range_sample(&src, &info, &d, &req, lightcraft_geom::Point::new(x, y)) @@ -646,7 +646,7 @@ fn spots_edit(s: &mut Session, c: &str, label: &str, f: impl FnOnce(&mut Vec) -> Option { let src = s.source_now(id, crate::media::SourceLevel::Thumb).ok()?; - let info = crate::media::source_info(s.catalog.photo(id)?); + let info = s.source_info(id); lightcraft_pipeline::spots::pick_source(&src, &info, d, spot, avoid) } diff --git a/crates/engine/src/cmd/previews.rs b/crates/engine/src/cmd/previews.rs index 8a2bf7e53..d3b633895 100644 --- a/crates/engine/src/cmd/previews.rs +++ b/crates/engine/src/cmd/previews.rs @@ -163,8 +163,8 @@ fn smart_run( // atomic: a failed write leaves no partial proxy that would pass for a built one let r = source .ok_or_else(|| "nothing to build from".to_string()) - .and_then(|s| s.load()) - .and_then(|src| crate::smart::encode(&src)) + .and_then(|s| s.load_source()) + .and_then(|src| crate::smart::encode(&src.image, src.info_or(Default::default()).camera_tone.as_ref())) .and_then(|b| lightcraft_catalog::safe_file::write_atomic(&path, &b).map_err(|e| format!("{}: {e}", path.display()))); match r { Ok(()) => { diff --git a/crates/engine/src/files.rs b/crates/engine/src/files.rs index 32296c5af..b4c9f863e 100644 --- a/crates/engine/src/files.rs +++ b/crates/engine/src/files.rs @@ -131,7 +131,9 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result { std::mem::swap(&mut w, &mut h); } let (t, tint) = xy_to_temp_tint(lightcraft_raw::color::as_shot_white_xy_of(&raw)); - let as_shot_wb = Some((t.round(), tint.round())); + // Vendor RGB multipliers do not identify an absolute illuminant without camera calibration. + let relative = raw.format == lightcraft_raw::RawFormat::Arw && !lightcraft_raw::color::has_matrix(&raw.color); + let as_shot_wb = Some(if relative { (6500.0, 0.0) } else { (t.round(), tint.round()) }); let embedded_lens = embedded_lens(&raw); return Ok(ProbeInfo { embedded_lens, @@ -221,6 +223,9 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result< } Err(e) => return Err(e.to_string()), }; + let xy = lightcraft_raw::color::as_shot_white_xy(&raw); + let t = lightcraft_raw::color::camera_transform(&raw, xy); + let camera_look = crate::camera_preview::fit_preview(&raw, &bytes, &t); drop(bytes); // Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in. let lens = embedded_lens(&raw.info()); @@ -242,12 +247,10 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result< // the samples aren't needed any more (the colour model below reads only the tags) raw.data = lightcraft_raw::RawData::U16(Vec::new()); let mut stages = vec![("develop", t0.elapsed())]; - let xy = lightcraft_raw::color::as_shot_white_xy(&raw); - let t = lightcraft_raw::color::camera_transform(&raw, xy); stages.push(("transform", t0.elapsed())); lightcraft_raw::highlight::reconstruct(&mut img, t.wb, HIGHLIGHT_CLIP); stages.push(("highlights", t0.elapsed())); - let m = t.matrix.to_f32(); + let m = camera_look.map(|p| p.matrix.mul(&t.matrix)).unwrap_or(t.matrix).to_f32(); let gain = 2f32.powf(t.baseline_exposure as f32); let wb = t.wb; img.map_in_place(|p| { @@ -276,7 +279,12 @@ fn load_bytes_now(bytes: std::borrow::Cow<'_, [u8]>, max_edge: usize) -> Result< eprintln!("[profile] raw source {}×{} (max {max_edge}, ms after decode): {}", img.width, img.height, parts.join(", ")); } let (temp, tint) = xy_to_temp_tint(xy); - return Ok((img, SourceInfo { raw: true, as_shot_temp: temp.round(), as_shot_tint: tint.round(), lens })); + let relative = raw.format == lightcraft_raw::RawFormat::Arw && t.matrix_is_fallback; + let (temp, tint) = if relative { (6500.0, 0.0) } else { (temp.round(), tint.round()) }; + return Ok(( + img, + SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens, relative_wb: relative, camera_tone: camera_look.map(|p| p.tone) }, + )); } let d = lightcraft_codecs::decode(&bytes, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).map_err(|e| e.to_string())?; drop(bytes); diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index ecdb078d5..b76d7a153 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -11,6 +11,7 @@ #![forbid(unsafe_code)] pub mod availability; +mod camera_preview; pub mod cmd; pub mod crs; pub mod crs_masks; diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rs index 561b7b24d..7fd8a5962 100644 --- a/crates/engine/src/media.rs +++ b/crates/engine/src/media.rs @@ -26,7 +26,7 @@ use lightcraft_raster::{Histogram, Rgb32f, Rgba8}; use serde::{Deserialize, Serialize}; /// Bump when the pipeline's output changes, to invalidate cached thumbnails. -pub const RENDER_CACHE_VERSION: u64 = 4; +pub const RENDER_CACHE_VERSION: u64 = 6; /// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache). pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512]; @@ -71,9 +71,31 @@ pub type FileLoader = Arc Result<(Rgb32f, SourceInfo), St /// `max_edge` (set by the app). `None`: no usable preview. pub type PreviewLoader = Arc Option + Send + Sync>; +/// Pixels and the source interpretation learned while decoding; they must be cached together. +#[derive(Clone)] +pub struct DecodedSource { + pub image: Arc, + pub info: Option, + /// A smart preview's stored camera tone curve: the one decoder fact its pixels need that the + /// catalog's header facts lack (used when `info` is `None`). + pub camera_tone: Option, +} + +impl DecodedSource { + pub fn new(image: Arc, info: Option) -> Self { + DecodedSource { image, info, camera_tone: None } + } + + /// What to render these pixels against: the decoder's facts, else `header` (the catalog's) + /// with any stored camera tone curve. + pub fn info_or(&self, header: SourceInfo) -> SourceInfo { + self.info.unwrap_or(SourceInfo { camera_tone: self.camera_tone.or(header.camera_tone), ..header }) + } +} + #[derive(Clone)] pub enum SourceRef { - Loaded(Arc), + Loaded(Box), Demo { scene: Box, max_edge: usize, @@ -94,35 +116,41 @@ pub enum SourceRef { impl SourceRef { pub fn load(&self) -> Result, String> { - match self { - SourceRef::Loaded(a) => Ok(a.clone()), - SourceRef::Demo { scene, max_edge } => Ok(Arc::new(scene.render_fit(*max_edge))), + self.load_source().map(|s| s.image) + } + + pub fn load_source(&self) -> Result { + let image = match self { + SourceRef::Loaded(a) => return Ok((**a).clone()), + SourceRef::Demo { scene, max_edge } => Arc::new(scene.render_fit(*max_edge)), SourceRef::File { path, max_edge, loader, fallback } => { let r = match loader { - Some(l) => l(path, *max_edge).map(|(img, _)| Arc::new(img)), + Some(l) => l(path, *max_edge).map(|(image, info)| DecodedSource::new(Arc::new(image), Some(info))), None => Err(format!("no decoder available for {path}")), }; - match (r, fallback) { + // An offline original renders from its smart preview (no decoder facts: header ones). + return match (r, fallback) { #[cfg(not(target_arch = "wasm32"))] (Err(e), Some(sp)) if crate::smart::is_valid(sp) => crate::smart::load(sp).map_err(|e2| format!("{e}; smart preview: {e2}")), (r, _) => r, - } + }; } #[cfg(not(target_arch = "wasm32"))] - SourceRef::Smart { path } => crate::smart::load(path), + SourceRef::Smart { path } => return crate::smart::load(path), #[cfg(target_arch = "wasm32")] - SourceRef::Smart { .. } => Err("smart previews are not available here".into()), - } + SourceRef::Smart { .. } => return Err("smart previews are not available here".into()), + }; + Ok(DecodedSource::new(image, None)) } } pub struct MediaCache { /// Decoded thumbnail-level sources (LRU by bytes). - thumbs: Lru>, + thumbs: Lru, /// Decoded preview-level sources with their last use ([`lightcraft_preview::next_tick`]). - previews: Vec<(PhotoId, Arc, u64)>, + previews: Vec<(PhotoId, DecodedSource, u64)>, /// The last full-resolution original (exports; one at a time: ~300 MB at 24 MP). - full: Option<(PhotoId, Arc, u64)>, + full: Option<(PhotoId, DecodedSource, u64)>, /// How many previews to keep (LRU). pub preview_capacity: usize, /// Bytes all decoded sources and rendered previews in memory may take together (the cache @@ -170,7 +198,7 @@ fn rendered_budget(share: usize) -> usize { } fn source_bytes(img: &Rgb32f) -> usize { - img.data.len() * 12 + 64 + img.data.len() * 12 + 64 + std::mem::size_of::() } impl MediaCache { @@ -221,8 +249,8 @@ impl MediaCache { let Some((tick, which)) = candidates.into_iter().flatten().min() else { break }; let freed = match which { 0 => self.thumbs.pop_oldest(), - 1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1)), - 2 => self.full.take().map(|e| source_bytes(&e.1)), + 1 => self.previews.iter().position(|e| e.2 == tick).map(|i| source_bytes(&self.previews.remove(i).1.image)), + 2 => self.full.take().map(|e| source_bytes(&e.1.image)), _ => self.rendered.evict_oldest(), }; match freed { @@ -233,6 +261,10 @@ impl MediaCache { } pub fn get(&mut self, id: PhotoId, level: SourceLevel) -> Option> { + self.get_source(id, level).map(|s| s.image) + } + + fn get_source(&mut self, id: PhotoId, level: SourceLevel) -> Option { match level { SourceLevel::Thumb => self.thumbs.get(&id).cloned(), SourceLevel::Preview => self.previews.iter_mut().find(|e| e.0 == id).map(|e| { @@ -246,11 +278,31 @@ impl MediaCache { } } + /// The facts of photo `id`'s cached sources: the decoder's when one has them, else `header` + /// (with a smart preview's stored camera tone curve). + fn source_facts(&self, id: PhotoId, header: SourceInfo) -> SourceInfo { + let cached = || { + self.thumbs + .peek(&id) + .into_iter() + .chain(self.previews.iter().filter(|e| e.0 == id).map(|e| &e.1)) + .chain(self.full.iter().filter(|e| e.0 == id).map(|e| &e.1)) + }; + match cached().find(|s| s.info.is_some()).or_else(|| cached().next()) { + Some(s) => s.info_or(header), + None => header, + } + } + pub fn insert(&mut self, id: PhotoId, level: SourceLevel, img: Arc) { + self.insert_source(id, level, DecodedSource::new(img, None)); + } + + fn insert_source(&mut self, id: PhotoId, level: SourceLevel, img: DecodedSource) { let tick = lightcraft_preview::next_tick(); match level { SourceLevel::Thumb => { - let cost = source_bytes(&img); + let cost = source_bytes(&img.image); self.thumbs.insert(id, img, cost); } SourceLevel::Preview => { @@ -284,14 +336,14 @@ impl MediaCache { /// Decoded sources held: (thumbnail level, preview level, full size). pub fn usage(&self) -> (crate::memory::Usage, crate::memory::Usage, crate::memory::Usage) { use crate::memory::Usage; - let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1)).sum()); - let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1))).unwrap_or_default(); + let previews = Usage::new(self.previews.len(), self.previews.iter().map(|e| source_bytes(&e.1.image)).sum()); + let full = self.full.as_ref().map(|e| Usage::new(1, source_bytes(&e.1.image))).unwrap_or_default(); (Usage::new(self.thumbs.len(), self.thumbs.cost()), previews, full) } pub fn source_ref(&mut self, p: &Photo, level: SourceLevel) -> SourceRef { - if let Some(a) = self.get(p.id, level) { - return SourceRef::Loaded(a); + if let Some(a) = self.get_source(p.id, level) { + return SourceRef::Loaded(Box::new(a)); } // Procedural scenes have a nominal size: "full" is that size, not unbounded. let max_edge = level.max_edge().min(match (&p.source, level) { @@ -365,7 +417,7 @@ pub struct RenderResult { pub key: u64, pub rendered: Result, /// A source that was loaded by this job (to be inserted into the cache). - pub loaded: Option>, + pub loaded: Option, /// Set for a [`QuickJob`]'s stand-in: where the image came from. pub quick: Option, } @@ -431,12 +483,14 @@ impl RenderJob { }; } let was_loaded = matches!(self.source, SourceRef::Loaded(_)); - match self.source.load() { - Ok(src) => { + match self.source.load_source() { + Ok(source) => { + let src = &source.image; + let info = source.info_or(self.info); // Thumbnails (many small jobs side by side) stay on the CPU; views and exports use // the GPU when there is one. let gpu = self.cache.is_none(); - let rendered = develop(&src, &self.info, &self.settings, &self.request, self.stages.as_deref(), gpu); + let rendered = develop(src, &info, &self.settings, &self.request, self.stages.as_deref(), gpu); if let Some((cache, key)) = &self.cache { cache.put(*key, Arc::new(rendered.image.clone())); } @@ -450,7 +504,7 @@ impl RenderJob { level: self.level, key: self.key, rendered: Ok(rendered), - loaded: (!was_loaded).then_some(src), + loaded: (!was_loaded).then_some(source), quick: None, } } @@ -542,11 +596,16 @@ pub fn content_key(p: &Photo) -> String { pub fn source_info(p: &Photo) -> SourceInfo { // Procedural demo scenes are scene-referred HDR (like raw files): use the filmic tone map. if matches!(p.source, Source::Demo { .. }) { - return SourceInfo { raw: true, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None }; + return SourceInfo { raw: true, ..Default::default() }; } // A raw shown from its embedded preview is a rendered (display-referred) JPEG: relative white // balance and the display tone curve, like any other rendered file. - if p.develops_raw() { SourceInfo { raw: true, as_shot_temp: 5500.0, as_shot_tint: 0.0, lens: p.embedded_lens } } else { SourceInfo::default() } + if p.develops_raw() { + let (temp, tint) = if p.relative_wb() { (6500.0, 0.0) } else { p.as_shot_wb.unwrap_or((5500.0, 0.0)) }; + SourceInfo { raw: true, as_shot_temp: temp, as_shot_tint: tint, lens: p.embedded_lens, relative_wb: p.relative_wb(), ..Default::default() } + } else { + SourceInfo::default() + } } impl crate::Session { @@ -701,7 +760,7 @@ impl crate::Session { /// Accept a finished job's loaded source into the cache. pub fn accept(&mut self, r: &RenderResult) { if let Some(src) = &r.loaded { - self.media.insert(r.photo, r.level, src.clone()); + self.media.insert_source(r.photo, r.level, src.clone()); } } @@ -744,12 +803,19 @@ impl crate::Session { Ok(job) } + /// Prefer decoder facts to header-only metadata for pixel-statistics commands. + pub fn source_info(&self, id: PhotoId) -> SourceInfo { + let header = self.catalog.photo(id).map(|p| source_info(p)).unwrap_or_default(); + self.media.source_facts(id, header) + } + /// The source proxy for pixel-statistics commands (auto tone/WB), loading synchronously. pub fn source_now(&mut self, id: PhotoId, level: SourceLevel) -> Result, String> { let p = self.catalog.photo(id).ok_or("no such photo")?.clone(); - let r = self.media.source_ref(&p, level).load()?; - self.media.insert(id, level, r.clone()); - Ok(r) + let r = self.media.source_ref(&p, level).load_source()?; + let image = r.image.clone(); + self.media.insert_source(id, level, r); + Ok(image) } } @@ -781,6 +847,42 @@ pub type FileProbe = Arc Result + Send + Sync mod tests { use super::*; + #[test] + fn decoder_info_survives_render_jobs_cache_and_eviction() { + let tone = lightcraft_pipeline::tone::CameraTone::new(std::array::from_fn(|i| { + let x = 0.01 * (i + 1) as f32; + [x, (x * 2.0).min(0.9)] + })) + .unwrap(); + let info = SourceInfo { raw: true, relative_wb: true, camera_tone: Some(tone), ..Default::default() }; + let mut s = crate::Session::with_demo(); + let id = s.active().unwrap(); + let mut job = s.render_job(id, 64, 64, false, true).unwrap(); + job.source = SourceRef::File { + path: "synthetic.arw".into(), + max_edge: 64, + loader: Some(Arc::new(move |_, _| { + let mut image = Rgb32f::new(64, 64); + image.data.fill([0.1; 3]); + Ok((image, info)) + })), + fallback: None, + }; + job.info = SourceInfo::default(); // Header facts cannot override decoder facts. + job.settings = Arc::new(DevelopSettings::default()); + let r = job.clone().run(); + assert_eq!(r.loaded.as_ref().unwrap().info, Some(info)); + let expected = lightcraft_pipeline::render(&r.loaded.as_ref().unwrap().image, &info, &job.settings, &job.request); + assert_eq!(r.rendered.as_ref().unwrap().image.data, expected.image.data); + s.accept(&r); + job.source = s.media.source_ref(s.catalog.photo(id).unwrap(), job.level); + let again = job.run(); + assert!(again.loaded.is_none()); + assert_eq!(r.rendered.unwrap().image.data, again.rendered.unwrap().image.data); + s.media.forget(id); + assert!(s.media.get_source(id, r.level).is_none()); + } + #[test] fn budget_evicts_least_recently_used_across_caches() { let img = |w: usize| Arc::new(Rgb32f::new(w, w)); diff --git a/crates/engine/src/smart.rs b/crates/engine/src/smart.rs index 038f33304..8deff7bde 100644 --- a/crates/engine/src/smart.rs +++ b/crates/engine/src/smart.rs @@ -1,13 +1,16 @@ //! Smart previews: a compact proxy of each photo (its decoded preview-size source, about 1 MB) //! kept in the library, so photos stay editable — and exportable at proxy size — while their //! originals are offline (an unplugged drive). The proxy is the scene-linear source scaled into -//! 0..1 by a stored factor, sRGB-encoded and saved as a JPEG after a one-line header. +//! 0..1 by a stored factor, sRGB-encoded and saved as a JPEG after a one-line header. The header +//! also keeps the decoder's file-local camera tone curve (Sony ARW camera look), which is applied +//! at render time and is not baked into the pixels. use std::path::{Path, PathBuf}; use std::sync::Arc; use lightcraft_catalog::Photo; use lightcraft_color::transfer::linear_to_srgb; +use lightcraft_pipeline::tone::CameraTone; use lightcraft_raster::{Rgb32f, Rgba8}; const MAGIC: &[u8] = b"LCSP1\n"; @@ -18,8 +21,11 @@ pub fn file_name(p: &Photo) -> String { format!("{:032x}.lcsp", h.0) } -/// Encode a source image as a smart preview. -pub fn encode(img: &Rgb32f) -> Result, String> { +/// The most header [`is_valid`] reads (the JSON line before the JPEG). +const HEADER_MAX: u64 = 4096; + +/// Encode a source image (and the decoder's camera tone curve, if any) as a smart preview. +pub fn encode(img: &Rgb32f, tone: Option<&CameraTone>) -> Result, String> { // scale so all but the brightest 0.05 % fit into 0..1 let mut lum: Vec = img.data.iter().map(|c| c[0].max(c[1]).max(c[2])).filter(|v| v.is_finite()).collect(); let scale = if lum.is_empty() { @@ -47,22 +53,29 @@ pub fn encode(img: &Rgb32f) -> Result, String> { ) .map_err(|e| e.to_string())?; let mut out = MAGIC.to_vec(); - out.extend_from_slice(format!("{{\"w\":{},\"h\":{},\"scale\":{scale}}}\n", img.width, img.height).as_bytes()); + let mut head = serde_json::json!({"w": img.width, "h": img.height, "scale": scale}); + if let Some(t) = tone { + head["tone"] = serde_json::to_value(t).map_err(|e| e.to_string())?; + } + out.extend_from_slice(head.to_string().as_bytes()); + out.push(b'\n'); out.extend_from_slice(&jpg); Ok(out) } -/// Decode a smart preview back into a source image. -pub fn decode(bytes: &[u8]) -> Result { +/// Decode a smart preview back into a source image and its stored camera tone curve (an invalid +/// curve is ignored, like a missing one). +pub fn decode(bytes: &[u8]) -> Result<(Rgb32f, Option), String> { let rest = bytes.strip_prefix(MAGIC).ok_or("not a smart preview")?; let nl = rest.iter().position(|b| *b == b'\n').ok_or("bad smart preview")?; let head: serde_json::Value = serde_json::from_slice(&rest[..nl]).map_err(|e| e.to_string())?; let scale = head["scale"].as_f64().unwrap_or(1.0) as f32; + let tone = head.get("tone").and_then(|t| serde_json::from_value::(t.clone()).ok()); let d = lightcraft_codecs::decode(&rest[nl + 1..], Default::default()).map_err(|e| e.to_string())?; // the decoder undoes the sRGB encoding; the values are the source's own primaries let mut img = d.image; img.data.iter_mut().for_each(|c| *c = c.map(|v| v * scale)); - Ok(img) + Ok((img, tone)) } /// Where a library keeps its smart previews. @@ -157,8 +170,8 @@ pub fn migrate(from: &Path, to: &Path, what: Existing) -> (usize, Vec) { pub fn is_valid(path: &Path) -> bool { use std::io::{Read, Seek, SeekFrom}; let Ok(mut f) = std::fs::File::open(path) else { return false }; - let mut head = Vec::with_capacity(256); - if (&mut f).take(256).read_to_end(&mut head).is_err() { + let mut head = Vec::with_capacity(1024); + if (&mut f).take(HEADER_MAX).read_to_end(&mut head).is_err() { return false; } let Some(rest) = head.strip_prefix(MAGIC) else { return false }; @@ -175,9 +188,9 @@ pub fn is_valid(path: &Path) -> bool { } /// Load the proxy at `path`. -pub fn load(path: &Path) -> Result, String> { +pub fn load(path: &Path) -> Result { let b = std::fs::read(path).map_err(|e| format!("{}: {e}", path.display()))?; - decode(&b).map(Arc::new) + decode(&b).map(|(image, camera_tone)| crate::media::DecodedSource { image: Arc::new(image), info: None, camera_tone }) } #[cfg(test)] @@ -228,7 +241,7 @@ mod tests { std::fs::create_dir_all(&a).unwrap(); std::fs::create_dir_all(&b).unwrap(); let img = lightcraft_scenes::demo_library()[0].render(64, 40); - let good = encode(&img).unwrap(); + let good = encode(&img, None).unwrap(); std::fs::write(a.join("1.lcsp"), &good).unwrap(); assert!(is_valid(&a.join("1.lcsp"))); for cut in [good.len() - 1, good.len() / 2, 10, 0] { @@ -242,11 +255,45 @@ mod tests { let _ = std::fs::remove_dir_all(&base); } + #[test] + fn camera_tone_travels_with_the_proxy() { + let tone = CameraTone::new(std::array::from_fn(|i| { + let x = 0.004 * 1.17f32.powi(i as i32); + [x, 0.95 * (1.0 - (-2.7 * x).exp())] + })) + .unwrap(); + let img = lightcraft_scenes::demo_library()[0].render(64, 40); + let bytes = encode(&img, Some(&tone)).unwrap(); + assert_eq!(decode(&bytes).unwrap().1, Some(tone)); + let dir = temp("tone"); + let path = dir.join("t.lcsp"); + std::fs::write(&path, &bytes).unwrap(); + // the longer header is still read by the validity check, and the curve by `load` + assert!(is_valid(&path)); + let loaded = load(&path).unwrap(); + assert_eq!(loaded.camera_tone, Some(tone)); + let header = lightcraft_pipeline::SourceInfo { raw: true, ..Default::default() }; + assert_eq!(loaded.info_or(header).camera_tone, Some(tone)); + // a hostile curve (reversing knots) is ignored, not trusted + let nl = MAGIC.len() + bytes[MAGIC.len()..].iter().position(|b| *b == b'\n').unwrap(); + let mut head: serde_json::Value = serde_json::from_slice(&bytes[MAGIC.len()..nl]).unwrap(); + let mut knots: Vec<[f32; 2]> = serde_json::from_value(head["tone"]["knots"].clone()).unwrap(); + knots.swap(3, 4); + head["tone"]["knots"] = serde_json::json!(knots); + let mut hostile = MAGIC.to_vec(); + hostile.extend_from_slice(head.to_string().as_bytes()); + hostile.extend_from_slice(&bytes[nl..]); + let (pixels, tone) = decode(&hostile).unwrap(); + assert_eq!((pixels.width, tone), (img.width, None)); + let _ = std::fs::remove_dir_all(&dir); + } + #[test] fn roundtrip_keeps_the_picture() { let img = lightcraft_scenes::demo_library()[0].render(160, 100); - let back = decode(&encode(&img).unwrap()).unwrap(); + let (back, tone) = decode(&encode(&img, None).unwrap()).unwrap(); assert_eq!((back.width, back.height), (img.width, img.height)); + assert!(tone.is_none()); let mut err = 0.0f64; for (a, b) in img.data.iter().zip(&back.data) { for k in 0..3 { diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 281a7a5f6..58b7e643e 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -10,6 +10,30 @@ fn active_dev(s: &Session) -> lightcraft_develop::DevelopSettings { (*s.develop_of(s.active().unwrap()).unwrap()).clone() } +#[test] +fn changing_one_wb_control_resolves_as_shot_without_stale_tint() { + use lightcraft_catalog::{Photo, PhotoId, Source}; + use lightcraft_develop::{DevelopSettings, WbMode}; + let mut s = demo(); + let id = PhotoId(100); + let mut p = Photo::new(id, Source::File { path: "synthetic.arw".into() }, "synthetic.arw", "ARW", 16, 16, ""); + // A catalog made by the old generic-matrix Kelvin inference. + p.as_shot_wb = Some((6829.0, -127.0)); + p.develop = std::sync::Arc::new(DevelopSettings::for_raw(6829.0, -127.0)); + s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(p) }).unwrap(); + s.execute("library.select", &json!({"ids": [100], "active": 100})).unwrap(); + s.execute("develop.set", &json!({"control": "wb.temp", "value": 8000})).unwrap(); + let d = active_dev(&s); + assert_eq!(d.wb.mode, WbMode::Custom); + assert_eq!(d.wb.temp, 8000.0); + assert_eq!(d.wb.tint, 0.0, "untouched tint comes from the current as-shot reference"); + s.execute("develop.wb", &json!({"mode": "asShot"})).unwrap(); + s.execute("develop.set", &json!({"control": "wb.tint", "value": 10})).unwrap(); + assert_eq!(active_dev(&s).wb.temp, 6500.0); + s.execute("develop.reset", &json!({})).unwrap(); + assert_eq!((active_dev(&s).wb.temp, active_dev(&s).wb.tint), (6500.0, 0.0)); +} + #[test] fn demo_library_loads() { let mut s = demo(); diff --git a/crates/gpu/tests/equivalence.rs b/crates/gpu/tests/equivalence.rs index a83c7beb1..cf7566e0d 100644 --- a/crates/gpu/tests/equivalence.rs +++ b/crates/gpu/tests/equivalence.rs @@ -22,6 +22,27 @@ fn gpu() -> bool { ok } +#[test] +fn camera_tone_and_relative_wb() { + if !gpu() { + return; + } + let src = scene(2, 320, 240); + let curve = lightcraft_pipeline::tone::CameraTone::new(std::array::from_fn(|i| { + let x = 0.004 * 1.18f32.powi(i as i32); + [x, 1.0 - (-2.0 * x).exp()] + })) + .unwrap(); + let info = SourceInfo { raw: true, relative_wb: true, camera_tone: Some(curve), ..Default::default() }; + let mut s = DevelopSettings::default(); + check("camera tone neutral", &src, &info, &s, &RenderRequest::fit(320, 240)); + s.light.exposure = 1.0; + s.light.contrast = 40.0; + s.wb.mode = WbMode::Custom; + s.wb.temp = 8000.0; + check("camera tone edited", &src, &info, &s, &RenderRequest::fit(320, 240)); +} + fn scene(i: usize, w: usize, h: usize) -> Arc { Arc::new(lightcraft_scenes::demo_library()[i].render(w, h)) } diff --git a/crates/pipeline/src/finish.rs b/crates/pipeline/src/finish.rs index aeb0adf81..dd1e2eea5 100644 --- a/crates/pipeline/src/finish.rs +++ b/crates/pipeline/src/finish.rs @@ -221,7 +221,9 @@ impl FinishParams { FinishParams { calib: if calibration { crate::colorops::calibration_matrix(&s.calibration) } else { None }, shadow_tint: if calibration { (s.calibration.shadows_tint / 100.0) as f32 } else { 0.0 }, - tone: if info.raw { + tone: if let Some(curve) = info.camera_tone.as_ref().filter(|_| info.raw) { + ToneMap::camera(curve, s.light.contrast, s.light.whites, s.light.blacks) + } else if info.raw { ToneMap::new(s.light.contrast, s.light.whites, s.light.blacks) } else { ToneMap::display(s.light.contrast, s.light.whites, s.light.blacks) diff --git a/crates/pipeline/src/lib.rs b/crates/pipeline/src/lib.rs index 875aca11c..15fbd7f33 100644 --- a/crates/pipeline/src/lib.rs +++ b/crates/pipeline/src/lib.rs @@ -64,11 +64,14 @@ pub struct SourceInfo { pub raw: bool, pub as_shot_temp: f64, pub as_shot_tint: f64, + /// No measured camera illuminant: WB adjustments are relative to the camera's rendered look. + pub relative_wb: bool, + pub camera_tone: Option, } impl Default for SourceInfo { fn default() -> Self { - Self { raw: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None } + Self { raw: false, as_shot_temp: 6500.0, as_shot_tint: 0.0, lens: None, relative_wb: false, camera_tone: None } } } diff --git a/crates/pipeline/src/local.rs b/crates/pipeline/src/local.rs index ae26888c7..cedb7cd1a 100644 --- a/crates/pipeline/src/local.rs +++ b/crates/pipeline/src/local.rs @@ -61,7 +61,7 @@ pub fn effective_wb(info: &SourceInfo, s: &DevelopSettings) -> (f64, f64) { use lightcraft_develop::WbMode; match s.wb.mode { WbMode::AsShot => (info.as_shot_temp, info.as_shot_tint), - m if info.raw => m.preset().unwrap_or((s.wb.temp, s.wb.tint)), + m if info.raw && !info.relative_wb => m.preset().unwrap_or((s.wb.temp, s.wb.tint)), _ => (s.wb.temp, s.wb.tint), } } diff --git a/crates/pipeline/src/tone.rs b/crates/pipeline/src/tone.rs index 63d23abc1..232bb8da4 100644 --- a/crates/pipeline/src/tone.rs +++ b/crates/pipeline/src/tone.rs @@ -14,12 +14,75 @@ pub const LUT_MIN_EV: f32 = -14.0; pub const LUT_MAX_EV: f32 = 10.0; pub const LUT_N: usize = 4096; +/// A file-local camera look, fitted independently of the scene-linear colour transform. +/// Knots are scene/display-linear luminance pairs. Keeping this in the finish stage preserves +/// RAW exposure and highlight headroom; it is never baked into the decoded sensor pixels. +#[derive(Clone, Copy, Debug, PartialEq, serde::Serialize)] +pub struct CameraTone { + knots: [[f32; 2]; 32], +} + +impl<'de> serde::Deserialize<'de> for CameraTone { + fn deserialize>(d: D) -> Result { + #[derive(serde::Deserialize)] + struct Wire { + knots: [[f32; 2]; 32], + } + let w = Wire::deserialize(d)?; + Self::new(w.knots).ok_or_else(|| serde::de::Error::custom("invalid camera tone curve")) + } +} + +impl CameraTone { + pub fn new(knots: [[f32; 2]; 32]) -> Option { + let mut previous = [0.0, 0.0]; + for p in knots { + if !p.iter().all(|v| v.is_finite()) || p[0] <= previous[0] || p[1] < previous[1] || p[1] >= 1.0 { + return None; + } + previous = p; + } + Some(Self { knots }) + } + + pub fn apply(&self, y: f32) -> f32 { + if !y.is_finite() || y <= 0.0 { + return 0.0; + } + let mut previous = [0.0, 0.0]; + for p in self.knots { + if y <= p[0] { + let t = (y - previous[0]) / (p[0] - previous[0]); + return previous[1] + t * (p[1] - previous[1]); + } + previous = p; + } + let a = self.knots[30]; + let b = self.knots[31]; + // Extend beyond observed (unclipped) highlights with a continuous, bounded shoulder. + let slope = ((b[1] - a[1]) / (b[0] - a[0])).clamp(0.1, 16.0); + 1.0 - (1.0 - b[1]) * (-(y - b[0]) * slope / (1.0 - b[1]).max(0.01)).exp() + } +} + #[derive(Clone, Debug)] pub struct ToneMap { lut: Vec, } impl ToneMap { + pub fn camera(curve: &CameraTone, contrast: f64, whites: f64, blacks: f64) -> ToneMap { + let adjustment = Self::display(contrast, whites, blacks); + let neutral = contrast == 0.0 && whites == 0.0 && blacks == 0.0; + let lut = (0..LUT_N) + .map(|i| { + let ev = LUT_MIN_EV + (LUT_MAX_EV - LUT_MIN_EV) * i as f32 / (LUT_N - 1) as f32; + let y = curve.apply(GREY * 2f32.powf(ev)); + if neutral { y } else { adjustment.apply(y) } + }) + .collect(); + ToneMap { lut } + } /// `contrast`, `whites`, `blacks` in −100..100 (Lightroom slider units). pub fn new(contrast: f64, whites: f64, blacks: f64) -> ToneMap { let c = (contrast / 100.0) as f32; @@ -116,6 +179,32 @@ fn smooth(e0: f32, e1: f32, x: f32) -> f32 { mod tests { use super::*; + #[test] + fn camera_curve_preserves_black_and_extends_headroom() { + let knots = std::array::from_fn(|i| { + let x = 0.005 * 1.15f32.powi(i as i32); + [x, 1.0 - (-3.0 * x).exp()] + }); + let curve = CameraTone::new(knots).unwrap(); + let base = ToneMap::camera(&curve, 0.0, 0.0, 0.0); + assert_eq!(base.apply(0.0), 0.0); + assert!(base.apply(0.1) < base.apply(0.2)); + let mut previous = 0.0; + for i in 0..2000 { + let y = 1e-6 * 1.01f32.powi(i); + let v = base.apply(y); + assert!((0.0..=1.0).contains(&v)); + assert!(v >= previous - 1e-6); + previous = v; + } + assert!((base.apply(0.1) - curve.apply(0.1)).abs() < 0.001); + assert!(ToneMap::camera(&curve, 50.0, 0.0, 0.0).apply(0.3) > base.apply(0.3)); + let mut invalid = knots; + invalid[1][0] = invalid[0][0]; + assert!(CameraTone::new(invalid).is_none()); + assert!(serde_json::from_value::(serde_json::json!({"knots": invalid})).is_err()); + } + #[test] fn monotone_and_bounded() { for (c, w, b) in [(0.0, 0.0, 0.0), (100.0, 100.0, -100.0), (-100.0, -100.0, 100.0), (50.0, -30.0, -40.0)] { diff --git a/crates/ui-egui/src/panels/edit.rs b/crates/ui-egui/src/panels/edit.rs index 188f3ea58..ff9d9a261 100644 --- a/crates/ui-egui/src/panels/edit.rs +++ b/crates/ui-egui/src/panels/edit.rs @@ -76,7 +76,13 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui, id: PhotoId) { let t = Tokens::get(ui.ctx()); let d = app.session.develop_of(id).unwrap_or_default(); // a raw shown from its embedded JPEG (preview only) gets the rendered-file white balance scale - let raw = app.session.catalog.photo(id).is_some_and(|p| p.develops_raw()); + let raw = app.session.catalog.photo(id).is_some_and(|p| p.develops_raw() && !p.relative_wb()); + let mut d = d; + if d.wb.mode == WbMode::AsShot && app.session.catalog.photo(id).is_some_and(|p| p.relative_wb()) { + let wb = &mut std::sync::Arc::make_mut(&mut d).wb; + wb.temp = 6500.0; + wb.tint = 0.0; + } let preview_only = app.session.catalog.photo(id).and_then(|p| p.preview_only.clone()); if app.ui.histogram { diff --git a/docs/camera-preview-colour.md b/docs/camera-preview-colour.md new file mode 100644 index 000000000..c47546640 --- /dev/null +++ b/docs/camera-preview-colour.md @@ -0,0 +1,33 @@ +# Sony ARW starting look + +ARW decoding supplies a Bayer mosaic and camera white-balance multipliers, but currently no measured camera colour matrix. Treating camera RGB as linear sRGB produces a dark, muted default rendering. The ARW loader now estimates a file-local starting look from that file's embedded camera JPEG, only when a usable camera matrix is absent. All output pixels still come from the RAW mosaic; there is no JPEG replacement or uniform saturation boost. + +## Colour and tone are separate + +A fixed sensor proxy of at most 96×96 is developed with the file's WB, black/white levels and active crop. The embedded JPEG is decoded with DCT scaling and a 64-million-pixel input limit, converted through the codec's colour management, and reduced to the same size. Aspect ratios must agree within 2%. Near-black, clipped and nonfinite samples are excluded. At least 256 paired pixels, 5% coloured samples and a nonsingular input covariance with bounded condition are required. + +Two thirds of the usable pixels train a ridge-regularised 3×3 **chromaticity** correction (luminance-normalised RGB). This avoids trying to encode the camera's nonlinear tone curve in a matrix, which left skin grey in the earlier matrix-only estimate. Coefficients are bounded by 8. Luminance is fitted separately: 32 equal-population bins with robust median knots and pooled-adjacent-violators isotonic regression. The curve preserves black, cannot reverse, and extends beyond observed highlights with a bounded shoulder. + +The matrix is applied to scene-linear sensor pixels. The curve is carried in `SourceInfo` and applied in the pipeline finish stage, after exposure/WB/local edits. It is never baked into decoded luminance or clipped to JPEG precision. CPU and GPU use the same tone LUT. Contrast, whites and blacks adjust the starting look through the display-domain controls. JPEG neutral rendering and the existing DNG colour model are unchanged. + +One third of the pixels are held out from both fits. The combined model must reduce linear-display squared error by at least 30% and have per-channel RMS error ≤0.10 (0.055 rejected half of the public raw.pixls.us samples tried — ILCE-6400, -6700, -7M4 — whose fits were still 1.5–3× closer to the camera JPEG than the fallback and visibly better). These are provisional acceptance gates, not a colour-accuracy certification. Monochrome, singular, unrelated, aspect-mismatched and poor-fitting references retain the documented fallback. Both fits use the same fixed proxy at every decode resolution. + +## Source metadata and white balance + +Previously `SourceRef::load()` discarded the decoder's `SourceInfo`, and render jobs used header-only facts (including a hard-coded RAW WB). Decoded pixels and source facts now travel and remain cached together, including across web-worker renders. Smart previews keep the tone curve in their header (the matrix is already in their pixels), so an offline original renders and exports with the same look; proxies built before this fall back to the generic curve until rebuilt. Pixel-statistics/colour-sampling commands use decoded facts too. Render cache version 6 invalidates old thumbnails and loupe renders. + +Vendor RGB WB multipliers alone do **not** identify an absolute illuminant without camera calibration. The generic model inferred approximately 6829 K / −127 tint for one Sony sample; changing only temperature retained that spurious tint and caused a pink cast. Uncalibrated ARWs now keep the file's vendor WB at decode and expose **relative** WB controls around the as-shot look. 6500/0 is the internal neutral reference of that relative scale, not a claim about the capture's measured Kelvin temperature. Presets, picker/auto and manual controls use this reference consistently. Switching from As Shot to Custom first resolves both controls; changing one cannot inherit stale catalog tint. Reset restores the current as-shot reference. Measured/absolute-Kelvin Sony WB remains future camera-calibration work. + +Sony's proprietary `ColorMatrix` is not assumed to be a DNG XYZ-to-camera matrix. No Adobe profile/matrix or GPL decoder/calibration source is used. The product remains pure Rust. + +## Validation and scope + +Synthetic tests cover colour with a nonlinear camera tone, preserved scene-linear headroom, invalid/singular/unrelated references, monotone black-preserving curves, hostile serialized knots, decode-to-render/cache metadata propagation, old-catalog WB transitions and CPU/GPU tone equivalence. Private photos are never committed or uploaded. + +Five local Sony ILCE-6700 ARWs were accepted. Held-out per-channel linear-display RMS values were 0.04244, 0.03349, 0.04552, 0.02871 and 0.03100. All five unedited supplied JPEG exports stayed pixel-identical to the installed pre-change renderer at equal size/quality. Four supplied JPEGs use Display P3 and two have a narrower crop than the RAW, so they are visual references rather than aligned pixel ground truth. + +Public CC0 samples (raw.pixls.us; 10 files, ILCE-6000, -6400 ×2, -6700, -7M3, -7M4 ×2, -7RM4, -7SM3, DSC-RX100M3), rendered at 1200 px and compared with each file's embedded JPEG (mean CIE76 ΔE at 200 px): 7 fits accepted, ΔE 17–26 → 3–10 with matching mean lightness. Rejected and unchanged: the RX100M3 (held-out RMS 0.103; its fallback render already has a strong green cast), an ILCE-7RM4 dusk scene whose fit was worse than the fallback (ΔE 7.6), and a reduced-size lossless ARW (no fit attempted). Saturated hues can still differ (an orange-red hull renders yellow). + +On one sample, Exposure +1 EV, Saturation +100, Contrast +50, warm/cool WB, tint, highlights and shadows all changed the output. The same matrix and curve were selected for 400-pixel and full-resolution renders. Headless UI checks wait until the loupe reports `source: render` and no jobs remain; the camera JPEG stand-in does not count as verification. + +This is a per-file camera-look estimate, **not measured spectral calibration or Lightroom parity**. Camera local tone, picture styles, noise reduction and lens warps cannot all be reproduced by a global matrix/curve; same-aspect crops may evade the aspect check. Skin, mixed-light scenes and saturated highlights can still differ from a supplied JPEG. Broader Sony model/lighting coverage, chart-based calibration, absolute WB and other manufacturers remain unverified. Rejected fits and other RAW formats retain their existing behaviour. diff --git a/docs/parity.md b/docs/parity.md index ed4b2f27f..cfc61c962 100644 --- a/docs/parity.md +++ b/docs/parity.md @@ -59,9 +59,7 @@ Ordered by user impact, then tier, then effort. The checklist above counts featu that decide whether a photographer can switch (see the honest assessment in [ROADMAP.md](../ROADMAP.md#where-we-stand)). Take the first one nobody is working on. -1. **LR-PROF-CAMERACOLOR** (P0): our own camera colour calibration. Every non-DNG raw is developed with a neutral - matrix today, so colours are muted. Start by fitting each camera to its own embedded JPEG and using matrices the - files carry themselves; never Adobe data. +1. **LR-PROF-CAMERACOLOR** (P0): our own camera colour calibration. Sony ARW now gets a guarded file-local fit to its own embedded JPEG (colour matrix + tone curve, relative WB; 7 of 10 public samples accepted, mean ΔE vs the camera JPEG 17–26 → 3–10); measured calibration and fidelity remain missing. Other non-DNG raws and rejected fits still use a neutral matrix. Expand validated preview fitting and use matrices the files carry themselves; never Adobe data. 2. **LR-IMP-FORMATS** (P0): **CR3** first (every Canon body since ~2018), then compressed RAF / ORF, RW2 v4, Nikon "lossy after split" NEF, Canon sRAW; HEIC/AVIF decode. Clean-room, from prose descriptions only (see `crates/raw/src/vendor/nefc.rs` for how compressed NEF was done). Until decoded, such photos are `preview_only`. @@ -276,7 +274,7 @@ under the UI command that wraps it (a no-op error); the UI command now wins. | LR-PROF-ADOBE | Standard raw looks (own equivalents) | P0 | ✅ | `crates/engine/src/presets.rs` (`PROFILES`), `crates/pipeline/src/profiles.rs` | six own looks: Color, Neutral, Vivid, Landscape, Portrait, Monochrome | | LR-PROF-ADAPTIVE | Adaptive profiles | P2 | ⬜ | | | | LR-PROF-CAMERA | Camera-matching looks | P2 | ⬜ | | | -| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs` | DNG files use the colour matrices they carry; every other raw uses a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | +| LR-PROF-CAMERACOLOR | Camera colour calibration (own) | P0 | 🟡 | `crates/raw/src/color.rs`, `crates/engine/src/camera_preview.rs` | DNG files use the colour matrices they carry; Sony ARW gets a guarded file-local fit to its own embedded JPEG — chromaticity matrix + tone curve, relative WB, kept with smart previews (see docs/camera-preview-colour.md); on 10 public raw.pixls.us samples from 8 bodies 7 fits were accepted, mean ΔE vs the camera JPEG 17–26 → 3–10; this is not measured camera calibration; other raws and rejected fits use a neutral fallback (camera RGB ≈ linear sRGB, `matrix_is_fallback`) with as-shot white balance, so colours are muted and not accurate. Needs our own per-camera calibration: matrices the files carry themselves (e.g. Olympus `ColorMatrix`), fitting each camera to its own embedded JPEG, then chart shots. Adobe matrices / DCPs are never used. Biggest image-quality gap today | | LR-PROF-CREATIVE | Creative profiles (own) | P2 | ✅ | `cmd:develop.profile`, `crates/pipeline/src/profiles.rs` | 16 own looks in Film / Cinematic / Muted / B&W (tone + point-curve fades, colour grading, mixer / B&W mix); scale with `ctl:profile.amount`; sliders untouched | | LR-PROF-LEGACY | Legacy profiles | P2 | ⬜ | | | | LR-PROF-NONRAW | Profiles for non-raw files | P0 | ✅ | `cmd:develop.profile` | same looks apply to JPEG/TIFF |