Samsung SRW raws whose raw IFD (private compression 32769 or 32770) stores the samples uncompressed are decoded from the sensor data: 12-bit samples packed two in three bytes (bit order told apart from the samples; NX5, NX10, NX11 and NX20, NX200, NX210, NX1000, NX1100) and 16-bit little-endian words (EX1, WB2000). CFA layout from the raw IFD's CFAPattern; framing, white balance, black and saturation levels from the maker note (value offsets relative to the note; the vectors are stored offset by its EncryptionKey); a one-direction Exif picture size crops to the centre of the framing; black from dark rows or columns when the file states none; saturation plateau below the stated level wins. Compressed codings (32770 with fewer bytes, 32772, 32773) stay preview only with that reason. srw is imported from folders; SRW takes the file-local camera look. RENDER_CACHE_VERSION 17.
Decode every JPEG XL tile with wide (32-bit) modular buffers: the
modular_16bit_buffers header flag does not hold on real lossy files (68
of 108 tiles failed with narrow buffers).
Lossy (XYB) tiles are taken from the decoder as linear sRGB primaries
and restored to the declared primaries with a plain 3x3 matrix, instead
of the decoder's gamut-mapped, [0, 1]-clipped conversion. Lossy tiles
whose declared encoding cannot be restored faithfully (ICC profile,
non-linear transfer function, non-D65 white point, intensity target
above 255 nits, single plane) are unsupported. Lossless output is
unchanged.
A JPEG XL tile that fails to decode is now an error for the image
(the file falls back to its embedded preview) instead of a zero tile.
Render cache version 17.
The Canon maker note's AspectInfo (tag 0x009a: aspect code, cropped width,
height, left, top) holds the rectangle of the shot, measured from the
recommended image crop. Use it as the CR3 default crop, so 3:2, 4:3, 16:9 and
1:1 shots develop in the shape the photographer chose. The tag is ignored when
it is missing, does not fit, or the recommended crop itself is inconsistent.
RENDER_CACHE_VERSION 15 -> 16. The M50 C-RAW corpus sample (raw.pixls.us 2663)
was shot at 1:1 and now develops at 4000 x 4000.
ColorData::baseline_sharpness holds the file's BaselineSharpness (raw IFD
or IFD 0; zero, negative, non-finite or above 16 is ignored) and the DNG
writer writes it back, so a conversion or smart preview keeps it. Merge
outputs leave it unset. Rendering doesn't use it yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read DNG 1.6 semantic masks (SemanticName, SemanticInstanceID,
MaskSubArea; any coding the tile/strip reader decodes, e.g. ProRAW's
lossy-JPEG matte) with lightcraft_raw::semantic_masks, map them onto the
developed image, and carry Apple's sky / portrait / skin / hair / teeth /
glasses mattes in SourceInfo::mattes (SourceInfo is no longer Copy).
Sky, Subject, Background and People components sample them through
orientation, warp (Warp::frame, as geometry.rs) and crop like the image,
so preview = export, instead of the heuristics; files without mattes
render as before. Malformed mask IFDs are skipped, and only the first
MAX_MASKS mask IFDs are decoded, so a file full of them costs bounded
work. The render cache version is bumped once, in the white-balance commit.
The corpus test reads corpus/raw/dng-apple-iphone12pro-proraw.dng, the
name `cargo xtask corpus` downloads it under.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parse ProfileGainTableMap (DNG 1.6) and ProfileGainTableMap2 (DNG 1.7):
both versions, DNG 1.7 precedence (version 2 in IFD 0, then the raw IFD;
version 1 in the raw IFD, then IFD 0), u8/u16/f16/f32 gains, gamma.
Malformed tags (wrong size, zero or huge grids, non-finite or negative
gains, bad spacing or gamma) are ignored, never allocated for.
Byte order: the specification gives this tag no byte order of its own,
unlike opcode lists, so the payload is read in the file's byte order with
a big-endian fallback; a wrong-order read can't pass the size checks.
The module docs quote the DNG 1.6 "Byte Order" rule.
The map is kept in ColorData, written back by the DNG writer (version 1,
or version 2 when its gamma isn't 1) and dropped from merge outputs, whose
geometry no longer matches the capture's active area. It is not applied
to the render: Lightroom Classic renders Apple ProRAW without it.
Supersedes the parser of #273.
The CC0 iPhone 12 Pro ProRAW from raw.pixls.us joins the corpus.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compression 52546 tiles and strips decode through jxl-oxide (new
default-on `jxl` feature of lightcraft-raw): bare codestreams and
ISO-BMFF containers, 1 or 3 planes, 8-16-bit integers scaled back
exactly from the codestream's own bit depth, 16-bit floats. Edge tiles
may be coded at their nominal or in-image size. Headers are checked
against the IFD in header-only probing too. A single-chunk JPEG XL
preview IFD (RGB or grey) is offered as the embedded preview.
The decoder's memory is bounded by the tile: 16 bytes per sample (three
times that for a lossy grey tile, which the decoder expands to three
channels) plus 64 KiB; jxl-oxide 0.12 was measured at <= 10.3 bytes per
sample for lossless and <= 13.5 per colour sample for lossy tiles. A
codestream needing more (e.g. extra channels) is an error.
Lossless tiles are checked bit-exact on synthetic CFA and LinearRaw
files (zune-jpegxl as a test-only encoder); lossy tiles are unverified
on a real file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase One / Leaf IIQ, Canon EOS-1D / 1Ds and Kodak DCS TIFFs keep their raw data in
a private block. IFD0 is only a 160-640 px RGB copy and no IFD is marked as raw, so
they opened as an ordinary TIFF of that size (a 150 MP file rendered 296x220) with
no preview-only label.
probe() now recognises such a file from its structure (OtherTiff): a picture at
least 4x larger than IFD0 in both directions is declared by a SubIFDs child, or by
Exif PixelXDimension/PixelYDimension together with enough bytes outside every IFD's
strips and tiles to hold it at one bit per pixel. decode() reports the sizes as the
reason. A raw we cannot decode now falls back to its embedded JPEG, else to the TIFF
reader's view of its first image, so it imports preview-only with that reason.
RENDER_CACHE_VERSION 12: these files render differently.
Main already had Fujifilm RAF in relative WB, the file-local look, the
docs and the corpus lists; CR3 is added next to it. RENDER_CACHE_VERSION
was already 12 on main (from #255/#258/#282), so CR3 thumbnails rendered
by main from embedded previews would not be redone: 13.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
Decode lossless CRX and version 0x100/0x200 C-RAW in portable Rust,
including sensor geometry, timed white balance and black/white levels.
Verify all sensor samples against six independent M50/R100/R8 references.
Any CR3 decode error falls back to the embedded JPEG, as CR3 files did
before this decoder. The full sensor buffer is allocated only after the
first plane decodes, and plane buffers grow row by row. Tile planes
decode in parallel with a windowed bit reader (24 MP: 424 -> 89 ms
lossless, 519 -> 138 ms C-RAW on an M4).
Add guarded CR3 camera-JPEG colour fitting with independent framing
validation, relative white balance and cache invalidation.
Validation: cargo xtask ci.
Maker note tag 0x4001 is UNDEFINED bytes (ColorData versions -3 and -4) on
PowerShot and EOS M bodies. It was read as one number per byte, so no plausible
as-shot levels were found and the white balance was 1, 1, 1 (green cast, a* about
-13 on neutrals). Byte and undefined data is now paired into 16-bit words in the
maker note's byte order before the existing probe (word 71). No per-model table.
RENDER_CACHE_VERSION 11 -> 12.
Split out of #240 without the compressed decoder (crates/raw/src/vendor/orfc.rs
is not part of this change; compressed ORF stays preview-only).
- The layout is read from Exif CFAPattern (0xa302, repeat counts in either
byte order) when it describes a 2x2 Bayer cell. BGGR and GBRG were not
reachable before.
- Without a usable tag the green diagonal is still found from the samples,
now from totals of 32x32 blocks instead of single-pixel differences, so
fine texture no longer outweighs a small red/blue difference. The old
version indexed the samples directly; this one reads them with get().
- probe_info no longer unpacks 12-bit packed ORFs that state their layout
(XZ-2: under 0.1 ms; it ran the same unpack as a decode before, which
takes 11 ms). 16-bit files still read their samples to find the bit depth.
Tests: stated layouts in both byte orders, unusable tags falling back to the
samples, the texture case, hostile areas for the fallback, probe_info equal
to decode; a corpus test comparing the E-1, E-400 and XZ-2 mosaics with
their embedded JPEGs under every layout.
No user-visible change yet for the bodies whose layout the old guess got
wrong (E-M1 and E-620 are BGGR): they write compressed ORF.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NEF (and other vendor raws without a colour matrix) carry their as-shot
white balance as vendor multipliers, not as a DNG AsShotNeutral. The DNG
writer only copied AsShotNeutral/AsShotWhiteXY, so Convert to DNG and
Copy as DNG produced files with no white balance at all, which develop
dark and green. Write the reciprocal of the vendor multipliers as
AsShotNeutral when the source has no DNG white of its own.
Decode every raw encoding Panasonic and Leica bodies write, from the DMC-LX1 (2005) to the DC-S1RM2 (2025):
compressed formats 4 (most bodies, 2008-2022) and 6 (S1 / S1R / S1H / S5, GH5M2), the prefix-coded strips of
format 8 (GH6 and later), packed formats 2, 5 and 7, and the 16-bit words of the 2005-2007 .RAW files. Until now
only format 5 decoded; every other Panasonic file opened as preview only.
Clean-room: no raw-decoder source was read. Tag names come from tag documentation; every encoding was recovered
by black-box analysis of 178 CC0 files from raw.pixls.us (118 bodies), and the module docs note the measurement
that settled each rule. Format-8 tags of unknown meaning must carry the values every sample has, otherwise the file
is reported as unsupported rather than guessed at.
Levels and geometry: formats 4 and 6 store samples 15 above the recorded black level (files without one use 15);
white is the active area's saturation plateau (the linearity limits are wrong on extended low-ISO shots, the GF1
and the Digilux 3); the in-camera aspect ratio is the default crop; mapped-out defect pixels are filled from
same-colour neighbours; the CFA pattern applies at sample (0, 0).
.rwl and .raw files are imported (engine, MCP folder expansion, file dialog, Linux MIME types). One file per
encoding joins the never-panic proptest; 11 CC0 files join `cargo xtask corpus`, and corpus_panasonic_encodings
checks each encoding's bit depth, black level, crop and CFA layout against the camera's own JPEG.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QtQmZ6peZXGn3wiqeowV2v
LightCraft has no colour matrix for Nikon cameras. Because of this, NEF
files showed colours that were too gray and too green.
- Colour: LightCraft now calculates a colour matrix and a tone curve from
the JPEG preview in each NEF file. Sony ARW files use the same
procedure and the same quality checks. White balance is relative to
the as-shot look, as for ARW files.
- One function, camera_preview::file_local_look, selects the formats for
the fit and for the relative white balance. Photo::relative_wb in the
catalog uses the same formats.
- Black level: Nikon stores the black level (maker note 0x003d) in
14-bit units, also in 12-bit files. The decoder read it as a 12-bit
value. Because of this, 12-bit files from the D750, D780, D850, D7500
and Z 50 were too dark. Now the decoder divides the value by 4 for
12-bit files.
- The corpus gets the D7500 12-bit and 14-bit lossless samples. A test
compares their levels. The render cache version increases to 9.
Results on 13 NEF files that LightCraft can decode: LightCraft accepted
the fit for all 13 files. The mean colour difference from the camera
JPEG (CIE76 dE) decreased from 13-46 to 2.9-7.8. One file with mixed
light improved by a small quantity only (12.7 to 12.0).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sony bodies from before ~2017 (RX100–RX100M5, RX10, NEX, SLT, ILCE-6000,
A7/A7II/A7RII, ...) write no plain 0x7313/0x7310/0x74c7 tags in the raw IFD,
so the decoder used unit WB multipliers (strong green cast, e.g. the CC0
DSC-RX100M3 sample), black 512 (800 on 1-inch sensors), and kept padding
columns at the right edge.
- White balance: WB_RGBLevels from the maker note's enciphered Tag2010
block (layouts per ExifTool's Tag2010a..i tables; byte substitution
p^3 mod 249 checked black-box on 15 CC0 files: SonyISO matches Exif ISO,
ILCE-7M3 levels match its plain 0x7313).
- Black level: recovered from the encrypted SR2SubIFD by known-plaintext
analysis (constant key in every file; RX100M6/ILCE-7M3 plain 0x7310 as
plaintext; 8 keystream bytes at offset 2510, plausibility-checked).
- Crop: DNG DefaultCrop or the maker note's FullImageSize.
- "12-bit uncompressed" ARWs hold 14-bit-scale data: black/white follow it.
- Render cache version bumped; corpus manifest + corpus test, unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lightroom-converted DNGs carry their camera profile's ProfileHueSatMapData1/2
and ProfileLookTableData (and sometimes ProfileToneCurve). We used only the
colour matrices, so such DNGs rendered flat and muted.
- raw: new `profile` module reads the tags (DNG spec ch. 6: value-major,
hue-middle, saturation-minor tables, wrap-around hue, tri-linear lookups,
sRGB value encoding) from the raw IFD or IFD 0; malformed tags are ignored.
- engine: the raw loader applies the hue/sat map (interpolated by the same
illuminant weight as the matrices) before BaselineExposure and the look
table after it, in linear ProPhoto; a ProfileToneCurve becomes the source's
camera tone. Without one the standard base tone is kept.
- dngwrite keeps the look with the data (conversions, smart previews, merges).
- Render cache version 7.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Bayer layout of Canon CR2 files differs by model (GBRG on the 7D, 50D,
60D, 550D, 600D, 1200D, 1300D, 5D Mark II, 1D Mark IV; RGGB on the 40D,
5D Mark III, 6D, 5DS R and others). Read it from raw IFD tag 0xc5e0
(CR2CFAPattern: 1 RGGB, 2 BGGR, 3 GBRG, 4 GRBG), anchored at the full
decoded sensor; fall back to RGGB when the tag is missing. No model table.
Checked on 35 decoded CC0 raw.pixls.us CR2s (35 models): the tag agrees
with the mosaic's green-diagonal statistic on every file; the SensorInfo
border parity does not predict the layout. Corpus manifest gains the
reporter's samples; a corpus test asserts each layout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decode Nikon Huffman-compressed NEF (compression 34713): lossless and lossy
(type 1 and 2) at 12 and 14 bits, with the maker-note 0x96 linearization
curve, predictor seeds and either byte order. Clean-room: 0x96 layout and
encoding-table sizes from prose write-ups (Clevy, Claff); the fixed Huffman
tables were recovered by black-box analysis of CC0 raw.pixls.us samples
(compressed vs uncompressed shots of the same scene) and verified on ~45
files from ~35 bodies. Lossy-after-split files stay preview only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ljpeg: a scan naming an undefined Huffman table is an error, not an expect.
- ARW2/RW2 block reads use get() + try_from instead of slice + expect.
- Vendor fallback CFAs use Cfa::bayer_static (literal names, tested)
instead of Cfa::bayer(..).expect("static").
- dngwrite matches (data, compression) instead of a let-else unreachable!.
- lightcraft-raw and lightcraft-tiff now #![deny] unwrap/expect/panic/
unimplemented/todo/unreachable; clippy.toml (same as the steering PR)
keeps them allowed in tests.
- tests/vendor_robust.rs: synthetic NEF, ARW, PEF, SRW, ORF, RW2, CR2 and
RAF files, truncated at every length and randomly mutated, must decode
to an error, never panic (no new crash found).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- tiffraw: Mode::Header parses the first LJ92 frame header (Compression 7),
so probe_info reports Unsupported for subsampled frames (Sony lossless
M/S) as the full decode does, instead of claiming the file decodes.
- corpus: CC0 ILCE-7M4 lossless L/M/S samples from raw.pixls.us in
`cargo xtask corpus`; M/S listed as known unsupported.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DNG compression 34892: each tile / strip is a baseline JPEG of 8-bit samples (decoded with
zune-jpeg, pure Rust), then the usual linearization table, levels and crop apply. Per the DNG
specification. Lossy DNGs (Smart Previews, "lossy compression" DNG exports, many phone DNGs)
now open as raw files instead of falling back to their embedded preview. Unit test with
synthetic JPEG tiles (edge tile cut short); the corpus sample decodes (5760×3840 in 144 ms).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decoders take a Mode: Header reads everything but the samples (DNG, CR2, ARW,
RAF, RW2; PEF when it has crop tags). NEF/ORF (and PEF without crop tags)
derive geometry/CFA from the samples and still unpack them. decode() stays
bit-exact (debug-hash of every corpus/raw decode identical before/after); the
corpus test asserts probe_info == decode().info() for all 37 files.
Import of 14 corpus raws: 3.4 s -> 1.4-2.0 s; peak RSS through import
1.41-1.49 GB -> 1.12-1.22 GB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lightcraft-raw: RawImage::develop_binned(k) — k×k superpixel binning of the CFA
(Bayer k even, X-Trans k multiple of 3), black/white normalised row by row, default
crop only, clipping preserved for highlight reconstruction. 2560 px previews of a
24 MP raw bin 2×, thumbnails 4–8×; exports / 1:1 still demosaic (AHD) at full size.
- highlight::reconstruct and its pyramid fill are rayon-parallel (bit-exact).
- X-Trans directional demosaic: precomputed per-phase neighbour offsets and no
reflection in the interior, no per-pixel allocation (bit-exact, tested against the
reference formulation): 2.9 s → 0.75 s CPU at 24 MP.
- files::load_bytes runs on a rayon worker so its short parallel loops don't each
wait for the pool to wake (dominant cost on a loaded machine); LIGHTCRAFT_PROFILE
prints per-stage source timings; files::bin_factor picks the block size.
- render_bench ONLY=source: per-file decode / normalise / demosaic / binning / load.
- Fix: u64 overflow on hostile IFD sizes in DNG/ARW/NEF raw-IFD selection (proptest).
load_bytes(2560) CPU time, ARW a7 III 24 MP: 2347 → 240 ms; X-T20: 3120 → 1220 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- embedded_preview: CR3 PRVW/THMB boxes (plus RAF, RW2, ORF, PEF from the
format commits); stops at the first valid box of each kind
- engine: raw variants that report Unsupported are probed and shown from
their embedded JPEG (orientation from the JPEG or the raw), not an error
- crates/raw/tests/corpus.rs: every corpus/raw file is recognised, has a
JPEG preview (DNG optional) and decodes or is a known-unsupported
variant; prints decode timings
- xtask corpus --download: 37 CC0 raw.pixls.us samples (matched by SHA-256)
- ROADMAP/crate docs: format coverage, NEF Huffman / RW2 format 4 /
compressed ORF+RAF gaps, neutral colour-matrix fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
