M11.4: harden RAF validation and camera profile fallback

This commit is contained in:
sebastian pfluegelmeier
2026-10-08 11:23:55 +02:00
parent cfcc169ff1
commit b4ca9f7439
12 changed files with 3736 additions and 17725 deletions
Generated
+1
View File
@@ -3028,6 +3028,7 @@ dependencies = [
"proptest",
"rayon",
"serde",
"sha2",
"thiserror 2.0.21",
"zune-core",
"zune-jpeg",
+6 -6
View File
@@ -10,7 +10,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
[`docs/parity.md`](docs/parity.md) counts features that **exist**; this section is about whether a photographer can
**switch** from Lightroom. Update it when a gap below closes.*
**In one line:** the checklist says **79.7%** (P0 98.5%, P1 95.6%, P2 41.2%), but measured by whether a working
**In one line:** the checklist says **79.8%** (P0 98.5%, P1 95.6%, P2 41.6%), but measured by whether a working
photographer could replace Lightroom without noticing, we are at roughly **60–70%**. The remaining gap is mostly
**quality of results and camera coverage**, not missing buttons.
@@ -22,7 +22,7 @@ duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
| Dimension | Estimate | What's true today | Biggest gaps |
|---|---:|---|---|
| **Feature checklist** | 79% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **Feature checklist** | 79.8% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, ARW, NEF (uncompressed + Huffman lossless/lossy), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans), uncompressed ORF, RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed ORF, Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Per-model verification is thin (~55 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2 and RAF have guarded camera-JPEG colour estimates, with bundled ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics | Real segmentation masks (subject, sky, people, objects, landscape, depth), AI denoise, super resolution, lens blur, generative remove, faces/people, natural-language search. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
@@ -92,7 +92,7 @@ hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help
| M15 | Classic modules | Map, Book, Slideshow, Print, Web; smart collections, stacks, virtual copies, publish services, tethering | 25–40 | 🚧 (smart albums, stacks, virtual copies, compare/survey ✅; Map/Book/Slideshow/Print/Web ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; English/Chinese (Simplified, Traditional)/Japanese localisation 🟡; accessibility and further locales ⬜) |
## Parity estimate (feature count updated 2026-10-05; effort estimate from 2026-10-02)
## Parity estimate (feature count updated 2026-10-08; effort estimate from 2026-10-02)
**By feature count** — from `docs/parity.md` (one row per Lightroom feature, menu item and shortcut; `cargo xtask
parity` prints this line on every run, so it stays current):
@@ -100,9 +100,9 @@ parity` prints this line on every run, so it stays current):
| Scope | Weighted completion | Rows | 2026-10-02 |
|---|---:|---:|---:|
| P0 (core) | **98.5%** | 200 | 98.7% |
| P1 (important parity) | **95.9%** | 147 | 94.4% |
| P2 (later / AI / niche, incl. Classic modules) | **39.2%** | 158 | 29.4% |
| **All in-scope rows** | **79.2%** | 505 | 75.6% |
| P1 (important parity) | **95.6%** | 149 | 94.4% |
| P2 (later / AI / niche, incl. Classic modules) | **41.6%** | 160 | 29.4% |
| **All in-scope rows** | **79.8%** | 509 | 75.6% |
The 2026-10-05 count includes three new 🟡 rows that make quality gaps visible (camera colour calibration, verified
camera coverage, render fidelity), which is why P0 dipped slightly.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+33 -2
View File
@@ -40,13 +40,13 @@ pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransf
}
let (sensor, reference) = proxies(raw, bytes, transform, PROXY)?;
// A camera profile pooled from many photos knows colours this photo shows too little of;
// only the tone and chroma curves are fitted per photo (DRO and picture styles vary).
// try its colour first and fit tone/chroma per photo (DRO and picture styles vary).
let profile = raw.metadata.model.as_deref().and_then(crate::camera_profiles::get);
let colour = profile.as_ref().and_then(|p| Some((p.matrix().mul(&transform.matrix.inverse()?), p.hue_sat.clone())));
let look = fit_pairs_with(&sensor, &reference, colour)?;
if lightcraft_pipeline::profiling() {
eprintln!(
"[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile {}",
"[profile] {:?} camera look: {:?}, {:?}, hue/sat table {}, camera profile available {}",
raw.format,
look.matrix.0,
look.tone,
@@ -219,6 +219,13 @@ fn fit_matrix(pairs: &[([f64; 3], [f64; 3])]) -> Option<Mat3> {
/// colour model (a camera profile's, in the sensor proxy's space), each completed with a tone
/// and chroma curve fitted to this photo.
fn fit_pairs_with(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Option<HsvTable>)>) -> Option<CameraLook> {
let has_profile = colour.is_some();
// A different picture style can make a camera profile fail the gates. Preserve the
// photo's own colour fit before resorting to the neutral fallback.
fit_candidate(sensor, reference, colour).or_else(|| has_profile.then(|| fit_candidate(sensor, reference, None)).flatten())
}
fn fit_candidate(sensor: &Rgb32f, reference: &Rgb32f, colour: Option<(Mat3, Option<HsvTable>)>) -> Option<CameraLook> {
// A known colour model needs enough signal for tone fitting, not a scene rich enough to
// learn a new colour matrix. Still reject monochrome references.
let (pairs, bright) = collect_pairs(sensor, reference, if colour.is_some() { 0.005 } else { 0.05 })?;
@@ -787,6 +794,30 @@ mod tests {
assert!(fit_pairs_with(&sensor, &reference, Some((matrix, None))).is_none(), "monochrome JPEGs remain rejected");
}
#[test]
fn rejected_profile_preserves_a_usable_per_photo_fit() {
let matrix = Mat3([[1.8, -0.4, -0.1], [-0.2, 1.5, -0.1], [-0.05, -0.3, 1.7]]);
let wrong = Mat3([[0.0, 0.0, 1.0], [0.0, 1.0, 0.0], [1.0, 0.0, 0.0]]);
let mut sensor = Rgb32f::new(64, 64);
let mut reference = sensor.clone();
let mut seed = 0x2545_f491_u32;
for (src, dst) in sensor.data.iter_mut().zip(&mut reference.data) {
*src = std::array::from_fn(|_| {
seed ^= seed << 13;
seed ^= seed >> 17;
seed ^= seed << 5;
0.05 + (seed % 1000) as f32 * 0.0003
});
let p = matrix.apply_f32(*src);
let y = luminance_2020(p);
*dst = p.map(|v| (v * (1.0 - (-2.5 * y).exp()) / y).clamp(0.0, 0.999));
}
let per_photo = fit_pairs(&sensor, &reference).unwrap();
let look = fit_pairs_with(&sensor, &reference, Some((wrong, None))).unwrap();
assert_eq!(look.matrix, per_photo.matrix, "a rejected camera profile must not suppress the per-photo fit");
assert_eq!(look.tone, per_photo.tone);
}
#[test]
fn hue_sat_passes_black_and_non_finite_pixels_through() {
let table = HsvTable { hue_divisions: 4, sat_divisions: 2, val_divisions: 1, data: vec![[10.0, 1.5, 1.0]; 8], srgb_value: false };
+2
View File
@@ -24,3 +24,5 @@ zune-core = { workspace = true }
[dev-dependencies]
jpeg-encoder = { workspace = true }
proptest = { workspace = true }
# Pinned CC0 corpus identities; not a product dependency.
sha2 = { version = "0.10", default-features = false, features = ["std"] }
+12 -1
View File
@@ -131,9 +131,20 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
.ok_or_else(|| RawError::Corrupt("RAF strip outside file".into()))?;
let cfa = cfa(&h);
let compression = be32(bytes, 108).unwrap_or(0);
let compressed = matches!(compression, 2 | 3);
let compressed = match compression {
0 => false,
2 | 3 => true,
_ => return Err(RawError::Unsupported(format!("Fujifilm RAF compression mode {compression}"))),
};
let mut data = Vec::new();
if compressed {
if cfa.width == 6 && cfa != Cfa::xtrans() {
return Err(RawError::Unsupported("Fujifilm compressed RAF with an unknown X-Trans layout".into()));
}
let flag = if compression == 2 { 1 } else { 0 };
if src.get(2) != Some(&flag) {
return Err(RawError::Corrupt("RAF container and stream compression modes disagree".into()));
}
data = super::rafc::decode(src, w, hgt, bits, cfa.width == 6, mode)?;
} else {
// Samples may use 16-bit words (e.g. 14-bit X-T20); otherwise they are packed at `bits`.
+28
View File
@@ -337,6 +337,7 @@ fn decode_stripe(stripe: &Stripe<'_>, height: usize, bits: u32, xtrans: bool) ->
#[cfg(test)]
mod tests {
use super::*;
use crate::Cfa;
use proptest::prelude::*;
/// Procedural zero residuals: no camera media or encoder dependency.
@@ -445,6 +446,33 @@ mod tests {
assert!(reader.residual(0, 14, 16384, 41).is_err());
}
#[test]
fn unsupported_container_modes_and_cfa_are_not_silently_decoded() {
let layout = std::array::from_fn(|i| Cfa::xtrans().pattern[35 - i]);
let src = compressed(14, true, None);
let raf = super::super::raf::tests::raf(780, 18, 14, src, Some(layout), &[]);
let mut mismatched = raf.clone();
mismatched[108..112].copy_from_slice(&3u32.to_be_bytes());
assert!(crate::decode(&mismatched).is_err(), "declared lossy mode disagrees with lossless stream");
let bad_layout = layout.map(|c| {
if c == 0 {
2
} else if c == 2 {
0
} else {
c
}
});
let raf = super::super::raf::tests::raf(780, 18, 14, compressed(14, true, None), Some(bad_layout), &[]);
assert!(matches!(crate::decode(&raf), Err(RawError::Unsupported(_))), "compression uses a fixed X-Trans arrangement");
let mut raf = super::super::raf::tests::raf(780, 18, 16, vec![0; 780 * 18 * 2], None, &[]);
for mode in [1u32, 4, u32::MAX] {
raf[108..112].copy_from_slice(&mode.to_be_bytes());
assert!(matches!(crate::decode(&raf), Err(RawError::Unsupported(_))), "unknown mode {mode}");
assert!(matches!(crate::probe_info(&raf), Err(RawError::Unsupported(_))), "header-only mode {mode}");
}
}
#[test]
fn residual_sign_wrap_and_escape() {
// Ordinary codes 0, 1, 2, 3 with k=2; then the escape for -8192.
+7
View File
@@ -432,6 +432,8 @@ fn corpus_panasonic_encodings() {
/// lossy block quantizer changes; the source files are CC0, fetched by `xtask corpus`.
#[test]
fn corpus_fujifilm_compressed_samples() {
use sha2::{Digest, Sha256};
let checksums = include_str!("../../../docs/raf-corpus.sha256");
let cases: &[(&str, usize, usize, u64, u64)] = &[
("raf-fuji-gfx100-3773.raf", 11808, 8754, 483668733090, 6453395759968151080),
("raf-fuji-gfx100-3775.raf", 11808, 8754, 120434414850, 6144091676930008551),
@@ -454,7 +456,12 @@ fn corpus_fujifilm_compressed_samples() {
let dir = corpus_root().join("raw");
let mut seen = 0;
for &(name, width, height, sum, weighted) in cases {
let path = format!("corpus/raw/{name}");
let checksum = checksums.lines().filter_map(|s| s.split_once(" ")).find(|(_, p)| *p == path).unwrap().0;
assert_eq!(checksum.len(), 64, "{name}: missing published SHA-256");
let Ok(bytes) = std::fs::read(dir.join(name)) else { continue };
let actual_sha: String = Sha256::digest(&bytes).iter().map(|b| format!("{b:02x}")).collect();
assert_eq!(actual_sha, checksum, "{name}: corpus file differs from its pinned identity");
let img = decode(&bytes).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!((img.width, img.height), (width, height), "{name}");
let lightcraft_raw::RawData::U16(data) = img.data else { panic!("{name}: float data") };
+5 -5
View File
@@ -1,6 +1,6 @@
# Sony ARW, Nikon NEF, Panasonic RW2 and Fujifilm RAF starting look
ARW, NEF, RW2 and RAF decoding supply a Bayer or X-Trans mosaic and camera white-balance multipliers, but currently no measured camera colour matrix. Treating camera RGB as linear sRGB produces a dark, muted default rendering. The loader now estimates a file-local starting look from that file's embedded camera JPEG, only when a usable camera matrix is absent. All output pixels still come from the RAW mosaic; there is no JPEG replacement or uniform saturation boost.
ARW, NEF, RW2 and RAF decoding supply sensor data and camera white-balance multipliers, but currently no measured camera colour matrix. Treating camera RGB as linear sRGB produces a dark, muted default rendering. The loader now estimates a file-local starting look from that file's embedded camera JPEG, only when a usable camera matrix is absent. All output pixels still come from the RAW sensor data; there is no JPEG replacement or uniform saturation boost.
## Colour and tone are separate
@@ -50,7 +50,7 @@ Public CC0 samples from raw.pixls.us (178 files from 118 Panasonic and Leica bod
The same guarded fit and relative WB now cover uncompressed, lossless compressed and lossy compressed RAFs, both X-Trans and Bayer. The shared loader serves desktop, CLI, web, export and smart-preview generation. Render cache version 12 refreshes older neutral-colour renders after an updated app is started; existing preview-only catalog entries still need Photo → Reload from Disk.
Verification used the seven supplied X-H2S/X-T4 files and 17 CC0 compressed RAFs (13 camera bodies, including 14/16-bit GFX). Renders at 1200 pixels were compared with each file's embedded JPEG at 200 pixels, in matching orientation, using mean CIE76 ΔE. Twenty fits were accepted and every accepted render improved: mean ΔE 16.70 → 4.13. The 16 accepted public files averaged 18.30 → 4.41; the four supplied X-T4 files averaged 10.30 → 3.02. The most difficult accepted public example, GFX100RF, remains at ΔE 11.54, so this is not a parity claim.
Verification used the seven supplied X-H2S/X-T4 files and 17 CC0 compressed RAFs (13 camera bodies, including 14/16-bit GFX). Renders at 1200 pixels were compared with each file's embedded JPEG at 200 pixels, in matching orientation, using mean CIE76 ΔE. Without pooled profiles, twenty fits were accepted and every accepted render improved: mean ΔE 16.70 → 4.13. The 16 accepted public files averaged 18.30 → 4.41; the four supplied X-T4 files averaged 10.30 → 3.02. The most difficult accepted public example, GFX100RF, remains at ΔE 11.54, so this is not a parity claim.
The additional supplied folder contains 748 RAFs. Profiles were fitted separately from 201 usable X-H2S and 545 X-T4 photos (722,133 and 1,951,939 colour pairs); one X-T20 file is insufficient for its own profile, and one reference was unusable. The two aggregate profiles are bundled in desktop, CLI and web builds; no private photographs are shipped. Other Fujifilm models use the same guarded per-file fit without a camera-name allowlist or borrowed matrix.
@@ -58,12 +58,12 @@ For validation, every fifth usable photo of each model in sorted path order was
The three original dark X-H2S wrench files have too little colour variation to learn a matrix, but now use the known profile to fit tone: ΔE 8.74 → 2.25, all accepted. The four original X-T4 files and two public X-T4 files remain accepted with the final bundled profile; they regress slightly against their own per-file fits (six-file mean 4.72 → 4.94), while remaining much closer than the neutral fallback (13.47). One public X-T20 reference still fails the gates and keeps the fallback.
RAF inputs also work with `lightcraft-cli calibrate`. Tests cover real X-Trans/16-bit Bayer fits, profile use on dull scenes, monochrome rejection, resolution independence, header/decoded relative WB, old-catalog WB transitions and invalid RAF calibration inputs. A full-size 6240×4160 X-T4 JPEG export also succeeded.
RAF inputs also work with `lightcraft-cli calibrate`. Tests cover real X-Trans/16-bit Bayer fits, profile use on dull scenes, per-file fitting after profile rejection, monochrome rejection, resolution independence, header/decoded relative WB, old-catalog WB transitions and invalid RAF calibration inputs. A full-size 6240×4160 X-T4 JPEG export also succeeded.
These are camera-JPEG look estimates, **not measured spectral calibration or Lightroom parity**. Camera local tone, picture styles, noise reduction and lens warps cannot all be reproduced by a global matrix/curve; same-aspect crops may evade the aspect check. Skin, mixed-light scenes and saturated highlights can still differ from a supplied JPEG. Broader Sony, Nikon and Fujifilm model/lighting coverage, chart-based calibration, absolute WB and other manufacturers remain unverified. Rejected fits and other RAW formats retain their existing behaviour.
These are camera-JPEG look estimates, **not measured spectral calibration or Lightroom parity**. Camera local tone, picture styles, noise reduction and lens warps cannot all be reproduced by a global matrix/curve; same-aspect crops may evade the aspect check. Skin, mixed-light scenes and saturated highlights can still differ from a supplied JPEG. Broader Sony, Nikon, Panasonic and Fujifilm model/lighting coverage, chart-based calibration, absolute WB and other manufacturers remain unverified. Rejected fits and other RAW formats retain their existing behaviour.
## Camera profiles (pooled per model)
One photo can show too little of a colour for its own fit to learn it: in a second shot of the lime shirt only a few dozen proxy pixels show it, next to a hillside of foliage at the same hue that the camera renders differently, and the shirt stayed yellow (hue 68° against the JPEG's 82°; a 192-pixel proxy only reached 74°). `lightcraft-cli calibrate [--max N] [--out DIR] FOLDERS…` therefore pools the colour pairs of many ARWs, NEFs or RAFs per camera model — each against its own embedded JPEG, at a 192-pixel proxy, at most 4000 pairs per photo, files spread evenly over the folders — and fits one matrix (from white-balanced camera RGB, so no single photo's white point is baked in) and one hue/saturation/value table. Models need at least 5 usable photos.
Profiles are JSON files `<model>.json` in `$LIGHTCRAFT_CAMERA_PROFILES`, else `<config>/camera-profiles` (macOS `~/Library/Application Support/LightCraft/camera-profiles`); a local profile replaces a built-in one. Built-in profiles live in `assets/camera-profiles/` (listed in `assets/ATTRIBUTION.md`) and are compiled in, so the app, CLI and web build share them: ILCE-7M4, fitted to 597 photos (2.2 million colour pairs) shot in 2026, mostly with the Standard creative style and DRO Auto; X-H2S, 201 photos; X-T4, 545 photos. They hold aggregate colour statistics only. A photo of a profiled model takes its colour from the profile and fits only its own tone and chroma curves (DRO and picture styles vary per shot); the acceptance gates still apply, and a rejected fit falls back as before. Files are read once per process and validated (version, bounded invertible matrix, table shape and finite data); a damaged file is ignored with a warning. The profiles folder's contents are part of the render cache keys, so thumbnails rendered before a profile existed are redone; smart previews built before keep their colour until rebuilt.
Profiles are JSON files `<model>.json` in `$LIGHTCRAFT_CAMERA_PROFILES`, else `<config>/camera-profiles` (macOS `~/Library/Application Support/LightCraft/camera-profiles`); a local profile replaces a built-in one. Built-in profiles live in `assets/camera-profiles/` (listed in `assets/ATTRIBUTION.md`) and are compiled in, so the app, CLI and web build share them: ILCE-7M4, fitted to 597 photos (2.2 million colour pairs) shot in 2026, mostly with the Standard creative style and DRO Auto; X-H2S, 201 photos; X-T4, 545 photos. They hold aggregate colour statistics only. A photo of a profiled model takes its colour from the profile and fits only its own tone and chroma curves (DRO and picture styles vary per shot); the acceptance gates still apply. A rejected profile fit retries the photo's own colour and tone fit before falling back to neutral rendering. Files are read once per process and validated (version, bounded invertible matrix, table shape and finite data); a damaged file is ignored with a warning. The profiles folder's contents are part of the render cache keys, so thumbnails rendered before a profile existed are redone; smart previews built before keep their colour until rebuilt.
+4 -4
View File
@@ -2,7 +2,7 @@
LightCraft decodes uncompressed, lossless compressed and lossy compressed RAF sensor data in pure Rust. The shared RAW decoder serves import, desktop previews, editing, CLI rendering, web and export. These files no longer enter the embedded-JPEG-only fallback. After restarting with the updated build, existing imported preview-only photos need **Photo → Reload from Disk** to refresh their status; edits are retained.
Fujifilm now uses a guarded colour/tone estimate from each file’s own embedded JPEG when the reference is usable, plus relative as-shot WB and optional pooled per-model profiles. Output pixels still come from the sensor. Rejected fits retain the neutral fallback; measured sensor calibration and Lightroom fidelity remain separate gaps. See [camera colour verification](camera-preview-colour.md#fujifilm-raf).
Fujifilm now uses a guarded colour/tone estimate from each file’s own embedded JPEG when the reference is usable, plus relative as-shot WB and bundled pooled profiles for X-H2S and X-T4. Output pixels still come from the sensor. A rejected profile fit retries the per-file colour fit; if both fail, the neutral fallback remains. Measured sensor calibration and Lightroom fidelity remain separate gaps. See [camera colour verification](camera-preview-colour.md#fujifilm-raf).
## Format and implementation
@@ -17,7 +17,7 @@ Fujifilm now uses a guarded colour/tone estimate from each file’s own embedded
- Signed prediction errors use adaptive Rice coding: a unary prefix followed by a context-dependent number of bits, with a bounded escape for large errors. Three pair phases have separate even/odd statistics. The 81 signed gradient contexts fold into 41 states.
- Lossy quantizers change the error range and reconstruction step. Low-gradient sites retain finer precision (0, 1 or 2), with five folded contexts per precision and statistics that survive main-quantizer changes. The main context statistics reset when its quantizer changes. Their initial sum is `max(2, (range + 32) / 64)`; using 16 as the minimum breaks more heavily quantized files.
Stripes decode in parallel using the existing Rayon dependency. Offsets, sizes, dimensions and entropy reads are checked; truncated data, impossible codes and inconsistent headers return errors. Header-only probing validates the tables and stripe ranges without decompressing the image.
Stripes decode in parallel using the existing Rayon dependency. Offsets, sizes, dimensions and entropy reads are checked; truncated data, impossible codes and inconsistent headers return errors. Unknown container compression modes, disagreements between container and stream modes, and compressed X-Trans layouts other than the supported CFA return errors. Header-only probing validates the tables and stripe ranges without decompressing the image.
## Verification
@@ -32,9 +32,9 @@ Complete sensor arrays were compared sample by sample with an external decoder u
All 24 files matched the reference sensor arrays exactly. This covers 13 camera bodies and both older and current compression variants. Synthetic tests also cover 12-bit streams; a real compressed 12-bit camera file has not been verified. Support is selected from the file's layout, without a camera-name allowlist. Other bodies using this layout should decode, but this is not a claim that every Fujifilm model has been tested. Older FinePix layouts without the supported raw TIFF IFD remain unsupported.
The 17 CC0 files live in the gitignored corpus, with published SHA-256 and licence records. `cargo xtask corpus --download` fetches them. `crates/raw/tests/corpus.rs` records reference sensor sums and position-weighted sums as regression checks; the full-array comparisons were performed locally. No supplied media or large binary fixtures are committed.
The 17 CC0 files live in the gitignored corpus. Their published SHA-256 identities are pinned in [raf-corpus.sha256](raf-corpus.sha256); their source licence records were checked as CC0 1.0. `cargo xtask corpus --download` fetches them. `cargo test -p lightcraft-raw corpus_fujifilm_compressed_samples -- --nocapture` verifies each available file's checksum before checking its reference sensor sum and position-weighted sum. The test skips absent corpus files; the full-array comparisons were performed locally. No supplied media or large binary fixtures are committed.
Unconditional procedural tests cover Bayer/X-Trans, 12/14/16 bits, partial final stripes, quantizer changes, residual signs and escapes, malformed headers and truncated/random entropy streams. A regression also checks overflowing uncompressed row strides.
Unconditional procedural tests cover Bayer/X-Trans, 12/14/16 bits, partial final stripes, quantizer changes, residual signs and escapes, malformed headers and truncated/random entropy streams. Regressions also check overflowing uncompressed row strides, unsupported modes and CFA layouts, and container/stream mode mismatches.
Local release decode measurements: about 0.16–0.24 seconds for 24–40 MP X-Trans files, 0.29 seconds for 50 MP Bayer, and 0.5–0.6 seconds for 100 MP Bayer. These measure sensor decoding, not the full develop/export pipeline.
+19
View File
@@ -0,0 +1,19 @@
# Fujifilm RAF regression files; CC0 1.0, raw.pixls.us.
# Published SHA-256 values retrieved 2026-10-07; see docs/raf-compression.md.
018781e126842596fcf3876b3fb6cd8973dc3936ccc753ee8648618be8889287 corpus/raw/raf-fuji-gfx100-3773.raf
0e7dad9819a748bb3d3be26bbf5770d4968e2d80b9cb41d97036bf7c27f7bfe7 corpus/raw/raf-fuji-gfx100-3775.raf
14bed5ac8191d6f2a7c82984e3bf6470ad4740df329293eac42b8018cc6a668a corpus/raw/raf-fuji-gfx100rf-8091.raf
a7bf8f061b603f7ef5fd29276eb6b2ac57355ca2f589a340117bfff0d03c065c corpus/raw/raf-fuji-gfx100s-4495.raf
1bb4608b61861a3d9e85b0ca6c00689b8ba4620cbe8579374d958c21ed08edf0 corpus/raw/raf-fuji-gfx100s-4503.raf
a7124d13910e0241a6b035154bc1fcdb7be4439371d043b1e1950a0251feb51e corpus/raw/raf-fuji-gfx50s-1435.raf
28f56cc9784164209e92fae209bdcc78267bd692aa2fb964595485f7e93692ab corpus/raw/raf-fuji-xe5-8509.raf
caeadb2666f6c5d9bbe32fbe692c2079b6a7e028f8514a13173f360656d08507 corpus/raw/raf-fuji-xh2-6001.raf
109be38f59963c9b8ff289ebe1f5ec9168c52f609e5b601432bd194520b4605b corpus/raw/raf-fuji-xh2-6002.raf
c76ce546e15727abb4ba201471010c554403176963a1a6b8cc017462606429e5 corpus/raw/raf-fuji-xm5-7748.raf
f756ad5be551a8b3e1b503a8a6f7e8e637fba519c0b0f8c79dd58e94b0287602 corpus/raw/raf-fuji-xt2-865.raf
a23045101b2912e64f154100d646c228c0d16585d21e95e0d1a006e537f7784c corpus/raw/raf-fuji-xt20-compressed.raf
07ea642656074834ffe5c661870a612ffa2e80fc1218a05166ee966bbe5f6fae corpus/raw/raf-fuji-xt4-3914.raf
871836938ffa695f4c41765ddb107c2703ce755f7f3ced7e10f9e04746f1734e corpus/raw/raf-fuji-xt4-3918.raf
b2a067e7fda16338f15c2532caa12b238b2415dc1c75a96a6d24d3445cfd0ae3 corpus/raw/raf-fuji-xt5-6122.raf
cddd7ae0c43f9280876e5fdcbdf8878718972ebd3d034d8affc8cfc6752d33dc corpus/raw/raf-fuji-xt5-6123.raf
3a8a9ac6d92274969aaa1f55ea669191b98bb8f588d00ec8ec17afd8b3963dea corpus/raw/raf-fuji-xt50-7807.raf