Canon's SensorInfo left border is not where light starts: on every corpus
body 10-36 columns left of it already hold image (EOS 6D: masked columns
0-70, border 84), and the black level was averaged over all of them. The
6D read 2591 instead of 2047 (4 % of the range), so CR2s rendered dark with
crushed shadows; 40D +141, 5DS R +131, 5D Mark II +73. The masked run is
now measured from the data: columns whose mean stays within 1/256 of the
range of the leftmost columns', minus a two-column margin.
Against Lightroom Classic (EOS 6D still life, 2000 px, ΔE00): base 7.63 ->
5.23 (p95 24.7 -> 11.3); mean over 86 edit variants abs 9.13 -> 6.19,
effect 3.09 -> 2.75.
RENDER_CACHE_VERSION 23.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
High Efficiency NEFs (HE / HE★, Z 8, Z 9, Z 6III, Z f) keep compression
34713 but carry a JPEG XS-based codestream (SOC + CAP, FF10 FF50) and no
maker-note 0x0096 table. They fell through to "Nikon compressed NEF
without a linearization table", so the Preview only notice gave Z f users
no hint why their raw wasn't decoded or that Lossless compressed NEFs work.
Detect the codestream before asking for the table and report "Nikon High
Efficiency NEF" with the workaround. Add a synthetic regression test, CC0
Z f HE + Lossless samples to the corpus (raw.pixls.us) with a corpus test,
and list HE NEF among the format gaps in parity.md and ROADMAP.md.
No decoder: the format has no public specification and the existing
decoders (LibRaw, rawler) are LGPL and derived from Nikon's binaries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bbmJTxYJyYqKfNH8uXRC6
C5060WZ, C7070WZ, SP-510UZ, SP-550UZ, SP-565UZ and SP-570UZ files say
uncompressed 12-bit with many strips whose byte counts sum to exactly
width x height x 1.5. main refused them because the 12-bit branch wanted a
single strip, so they fell to "Olympus compressed ORF" and opened
preview-only. The samples are packed MSB-first, the strips hold two fields
(even output rows, then odd ones) with one gap at the seam, and the last row
of the second field sits right after the last strip. Accepted only when all of
that holds exactly. CFA comes from the Exif tag, white balance from maker-note
entries 0x1017/0x1018, black 0. RENDER_CACHE_VERSION 27 (adjust at merge).
The YCbCr M coding's pedestal is corrected separately (#535, item 3), so
this test no longer depends on which change lands first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three CC0 raw.pixls.us files of one ILCE-7CR (61 MP) scene: lossless
compressed L and M (subsampled YCbCr) and compressed ARW2. A raw corpus
test pins model, colour-filter layout, black level, white level, as-shot
gains and default crop, and checks that the lossless and compressed
mosaics agree inside the crop (green block correlation 0.9976). An engine
corpus test checks each gets a fitted camera look, not the neutral
fallback. All nine public codings decode and fit (mean dE76 1.59 against
their camera JPEGs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The one-strip Sony ARW layout was accepted when the raw strip was 1.00-1.25x width x height bytes. The DSLR-A200, A230 and A350 sample files carry strips of 1.06x, 1.23x and 1.04x, passed that test and decoded as colour noise. Every file that decodes correctly (108 in the test archive) has a strip of exactly width x height bytes, so the layout is now accepted only then; the rest report "Sony ARW version 1 / packed compressed variant" and open from the embedded preview like the A290/A380. Part of #461 (not a decoder). RENDER_CACHE_VERSION 16 -> 19.
Renders a raw's DNG ProfileGainTableMap (Apple ProRAW's local tone mapping)
when the photo's Profile option "Camera local tone mapping" is on
(`profile.camera_local_tone`, off by default: Lightroom Classic renders
ProRAW without it). Built on #427's parser; supersedes the decode-time
applier of the first version of #273.
- raw/gaintable: GainTableMap::gain / evaluator (GainEval) per DNG 1.7.1:
bilinear between tables with edges replicated, input clamp(weights .
(R,G,B,min,max))^gamma in linear ProPhoto after the baseline exposure,
index input x MapPointsN clamped to the last point. Saturating cell
math; maps whose sizes don't match their gains aren't evaluated.
SourcePlacement: where a developed source sits in the active area.
- pipeline/local_tone: the gain per output pixel in the scene-linear
stage, before the white balance, mapped back through the frame (crop,
user orientation, warp) and the EXIF orientation. lin_needs_cpu routes
the stage to the CPU when it's on (CPU and GPU match); lin_key includes
it. SourceInfo::local_tone carries the map from the decoder.
- develop: Profile::camera_local_tone (omitted from JSON when off, so
existing settings and their hashes are unchanged); Profile row checkbox
when the source has a map; translations.
- RENDER_CACHE_VERSION 17. Docs: parity row and What's New as an opt-in.
Off renders pixel-identical to main on two iPhone 15 Pro ProRAWs (12 and
48 MP). On adds ~60-75 ms to the scene-linear stage at 1536x2048 on a
2-core VM, the same for both files (cost follows output pixels).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VFqow2P1EtDxt6nRjjQoTA
The 512 floor and its ColorDataVersion 3 keying rested on one file.
Versions without a known black-level position now fall back to black 0
for the white level, so the 40D sRAW renders with slightly raised blacks.
The Huffman-compressed Pentax files (compression 65535) assumed BGGR from
the array origin. The K10D, K200D, K2000 and every body from the K-3 on are
RGGB, so red and blue were exchanged before demosaicing. They now use the
Exif CFAPattern from the first sample of the stored array, like the other
compressions; BGGR stays the fallback for files without the tag. Extends the
CFA test to compression 65535. RENDER_CACHE_VERSION 17 to 18.
An odd-height adaptive QP map (EOS R7: 4732 sensor rows, 1183 QP rows)
was rejected as unsupported, so R7 C-RAW files opened from their embedded
JPEG. The map decodes to the exact end of its entropy stream; the last
row of the second-level high-frequency bands has no partner row and its
QP is used alone. Even-height maps never reach this rule, so the six
verified M50/R100/R8 fixtures are unchanged.
Checked on paired R7 RAW and C-RAW captures (ISO 100 and 6400): all
six C-RAW files decode, developed images match their RAW counterparts
visually, and ISO 6400 sensor means agree within 0.1%.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gs6roPby7THvteijZwptQt
The pattern was shifted by the image-area origin for compression 1 and
32773. The *ist D (area origin 19, 11, tag RGGB) fits RGGB from the array
origin in a mosaic-level fit against its camera JPEG, so the shifted reading
exchanged red and blue. The six 32773 files have an even origin and keep
their layout. Adds a hand-made-bytes test with an odd area origin for both
compressions.
Samsung SRW raws whose raw IFD (private compression 32769 or 32770) stores the samples uncompressed are decoded from the sensor data: 12-bit samples packed two in three bytes (bit order told apart from the samples; NX5, NX10, NX11 and NX20, NX200, NX210, NX1000, NX1100) and 16-bit little-endian words (EX1, WB2000). CFA layout from the raw IFD's CFAPattern; framing, white balance, black and saturation levels from the maker note (value offsets relative to the note; the vectors are stored offset by its EncryptionKey); a one-direction Exif picture size crops to the centre of the framing; black from dark rows or columns when the file states none; saturation plateau below the stated level wins. Compressed codings (32770 with fewer bytes, 32772, 32773) stay preview only with that reason. srw is imported from folders; SRW takes the file-local camera look. RENDER_CACHE_VERSION 17.
sRAW and mRAW store subsampled YCbCr in a lossless-JPEG frame cut into
slices of whole MCUs. Reuse the subsampled lossless decoder added for
Sony M/S ARW with Canon's per-component sequential prediction, re-chop
the stream into image planes, and convert to linear white-balanced RGB
with chroma gains measured per file against the camera's own IFD2 RGB.
Render cache version 17.
Compression 32773 is accepted in the PEF reader only when every strip has exactly the packed size of its rows at the declared bit depth, and is then read as uncompressed MSB-first samples. Any other 32773 size stays unsupported. Uncompressed PEFs take their colour filter from the Exif CFAPattern as seen from the image area origin. A Samsung-branded TIFF whose maker note has the Pentax layout is probed as PEF. RENDER_CACHE_VERSION 16 to 17.
One config-dir helper (engine::config::config_dir; camera_profiles re-exports it) with both
the face-models and denoise-models defaults; hosts set up both model folders; Settings has
Faces and AI Denoise tabs; both command sets registered; both model dialogs; locale entries
from both; one [features] table per app manifest; Cargo.lock regenerated by cargo; the
DecodedSource cache cost counts both segmentation mattes and the denoised twin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TIFF raws that describe a full-resolution CFA IFD with DNG's tags (levels, crop, AsShotNeutral) but carry no DNG version tag are read by the DNG reader (RawFormat::CfaTiff, dng::decode_as). RGGB when the file states no CFAPattern. Lossless JPEG scans with a predictor selection value above 7 are rejected with that reason and stay preview-only. .3fr and .fff are imported from folders. RENDER_CACHE_VERSION 17.
Decode every JPEG XL tile with wide (32-bit) modular buffers: the
modular_16bit_buffers header flag does not hold on real lossy files (68
of 108 tiles failed with narrow buffers).
Lossy (XYB) tiles are taken from the decoder as linear sRGB primaries
and restored to the declared primaries with a plain 3x3 matrix, instead
of the decoder's gamut-mapped, [0, 1]-clipped conversion. Lossy tiles
whose declared encoding cannot be restored faithfully (ICC profile,
non-linear transfer function, non-D65 white point, intensity target
above 255 nits, single plane) are unsupported. Lossless output is
unchanged.
A JPEG XL tile that fails to decode is now an error for the image
(the file falls back to its embedded preview) instead of a zero tile.
Render cache version 17.
The Canon maker note's AspectInfo (tag 0x009a: aspect code, cropped width,
height, left, top) holds the rectangle of the shot, measured from the
recommended image crop. Use it as the CR3 default crop, so 3:2, 4:3, 16:9 and
1:1 shots develop in the shape the photographer chose. The tag is ignored when
it is missing, does not fit, or the recommended crop itself is inconsistent.
RENDER_CACHE_VERSION 15 -> 16. The M50 C-RAW corpus sample (raw.pixls.us 2663)
was shot at 1:1 and now develops at 4000 x 4000.
Recover the unpublished denoise work against current upstream. Preserve later opt-in model setup, shared Rust HTTPS downloads, pending-photo resume and an Amount-independent switch. Keep original RAWs unchanged and share the current color finish between ordinary and denoised sources.
Bound public tiling inputs, refuse corrupt-cache exports, and retain synthetic/real-model coverage. Include two Windows path spelling fixes in existing tests. No model weights, face code, media or Tract runtime are included.
Validation: cargo xtask ci (all seven gates); installed RawNIND CPU/GPU agreement 1.38e-6; native CPU preview/export on a public 24 MP Sony RAW, unchanged original SHA-256.
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Generated with Codex.
In NRW data version 0104 the u32 at byte 0x20 is the black level in sample units: 200 where the darkest samples of the B700, P950, P1000 and P1100 sit, 0 for the A1000 whose samples reach 0. Used when the maker note has no 0x003d.
Raw-IFD flag 0xf006 = 1 marks RAF samples packed MSB-first into 32-bit little-endian words at any depth; 12-bit files with it (FinePix S1, X-A10, SL1000) were read as an LSB-first stream. Nikon uncompressed 12-bit strips of exactly 1.5 bytes per pixel in a file whose maker note has an NRW data block (COOLPIX B700, P950, P1000, P1100) use the same words in file byte order; without the block (E5700, E8400, E8800) they stay an MSB-first byte stream. Shared unpack_words32_msb and Packing::Words32Msb. RENDER_CACHE_VERSION 15 to 16.
A custom white balance on a raw with a colour matrix (DNGs, and other raws
without a file-local look) re-develops the pixels for the chosen white
through the file's own colour model: lightcraft_raw::color::rebalance
re-evaluates the camera transform at the new white and maps the as-shot
development onto it, so a neutral lit by that white renders neutral and
saturated colours move as the camera's matrices say (DNG spec ch. 6, as
Lightroom does). The decoder passes the colour tags and the white the
pixels were developed for in SourceInfo::camera_color; other photos, and
relative white balances, keep the Bradford adaptation. The eyedropper and
Auto invert the same model (color::neutral_white), so a picked grey card
renders neutral.
The Tint sign is main's (#349): unchanged here. Render cache version 15.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ColorData::baseline_sharpness holds the file's BaselineSharpness (raw IFD
or IFD 0; zero, negative, non-finite or above 16 is ignored) and the DNG
writer writes it back, so a conversion or smart preview keeps it. Merge
outputs leave it unset. Rendering doesn't use it yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read DNG 1.6 semantic masks (SemanticName, SemanticInstanceID,
MaskSubArea; any coding the tile/strip reader decodes, e.g. ProRAW's
lossy-JPEG matte) with lightcraft_raw::semantic_masks, map them onto the
developed image, and carry Apple's sky / portrait / skin / hair / teeth /
glasses mattes in SourceInfo::mattes (SourceInfo is no longer Copy).
Sky, Subject, Background and People components sample them through
orientation, warp (Warp::frame, as geometry.rs) and crop like the image,
so preview = export, instead of the heuristics; files without mattes
render as before. Malformed mask IFDs are skipped, and only the first
MAX_MASKS mask IFDs are decoded, so a file full of them costs bounded
work. The render cache version is bumped once, in the white-balance commit.
The corpus test reads corpus/raw/dng-apple-iphone12pro-proraw.dng, the
name `cargo xtask corpus` downloads it under.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parse ProfileGainTableMap (DNG 1.6) and ProfileGainTableMap2 (DNG 1.7):
both versions, DNG 1.7 precedence (version 2 in IFD 0, then the raw IFD;
version 1 in the raw IFD, then IFD 0), u8/u16/f16/f32 gains, gamma.
Malformed tags (wrong size, zero or huge grids, non-finite or negative
gains, bad spacing or gamma) are ignored, never allocated for.
Byte order: the specification gives this tag no byte order of its own,
unlike opcode lists, so the payload is read in the file's byte order with
a big-endian fallback; a wrong-order read can't pass the size checks.
The module docs quote the DNG 1.6 "Byte Order" rule.
The map is kept in ColorData, written back by the DNG writer (version 1,
or version 2 when its gamma isn't 1) and dropped from merge outputs, whose
geometry no longer matches the capture's active area. It is not applied
to the render: Lightroom Classic renders Apple ProRAW without it.
Supersedes the parser of #273.
The CC0 iPhone 12 Pro ProRAW from raw.pixls.us joins the corpus.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Compression 52546 tiles and strips decode through jxl-oxide (new
default-on `jxl` feature of lightcraft-raw): bare codestreams and
ISO-BMFF containers, 1 or 3 planes, 8-16-bit integers scaled back
exactly from the codestream's own bit depth, 16-bit floats. Edge tiles
may be coded at their nominal or in-image size. Headers are checked
against the IFD in header-only probing too. A single-chunk JPEG XL
preview IFD (RGB or grey) is offered as the embedded preview.
The decoder's memory is bounded by the tile: 16 bytes per sample (three
times that for a lossy grey tile, which the decoder expands to three
channels) plus 64 KiB; jxl-oxide 0.12 was measured at <= 10.3 bytes per
sample for lossless and <= 13.5 per colour sample for lossy tiles. A
codestream needing more (e.g. extra channels) is an error.
Lossless tiles are checked bit-exact on synthetic CFA and LinearRaw
files (zune-jpegxl as a test-only encoder); lossy tiles are unverified
on a real file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas