Canon's SensorInfo left border is not where light starts: on every corpus
body 10-36 columns left of it already hold image (EOS 6D: masked columns
0-70, border 84), and the black level was averaged over all of them. The
6D read 2591 instead of 2047 (4 % of the range), so CR2s rendered dark with
crushed shadows; 40D +141, 5DS R +131, 5D Mark II +73. The masked run is
now measured from the data: columns whose mean stays within 1/256 of the
range of the leftmost columns', minus a two-column margin.
Against Lightroom Classic (EOS 6D still life, 2000 px, ΔE00): base 7.63 ->
5.23 (p95 24.7 -> 11.3); mean over 86 edit variants abs 9.13 -> 6.19,
effect 3.09 -> 2.75.
RENDER_CACHE_VERSION 23.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ORF decoder supplies the as-shot white balance (ImageProcessing
WB_RBLevels) and no colour matrix, like ARW, NEF, RW2, RAF, CR2/CR3, PEF
and SRW, but ORF was left out of camera_preview::file_local_look and the
catalog's relative_wb, so it rendered with the neutral fallback matrix:
flat, grey and dark. It now gets the same guarded fit to its camera JPEG
and white balance relative to the as-shot look. Corpus (held-out RMS
against the fallback, all accepted): E-1 0.126 -> 0.014, E-400 0.087 ->
0.022, XZ-2 0.061 -> 0.023; compressed ORFs still open from their preview.
Against Lightroom Classic (CC0 E-1 kit, 2000 px, ΔE00): unedited render
11.25 -> 8.30; mean over 86 variants abs 12.10 -> 9.48, effect 3.44 ->
2.80. Every other kit's unedited render is byte-identical.
RENDER_CACHE_VERSION 23.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sensor-clipped pixels are rebuilt at raw decode. The tone map rolls them off to
white at Neutral, so any colour they carry shows only when Highlights, a negative
Exposure or Whites darkens them. They took colour from the nearest unclipped pixel
(dark foliage, lens fringes along branches, pixels demosaiced from clipped samples),
left two clipped channels at their white-balance-scaled clip levels (red or pink),
and were neutral only in camera RGB, not after the colour model (a look fitted to
the camera's JPEG, a camera profile).
- highlight::reconstruct_with: chromaticity only from bright unclipped content away
from edges and from the clip, through a normalised push-pull pyramid; a neutral
estimate blended in where reliable content is scarce at every scale up to a
quarter of the long edge, applied only as far as the pixel lies past the nearest
near-clip unclipped pixels (so a surface still runs into clipping without a
step); clipped pairs keep the estimate's ratio at their clip floors; partly
clipped pixels fade into the clipped neutral on their way from the first channel
clipping to the last; fully clipped pixels become the colour model's neutral.
Pixels without a clipped channel are left exactly as they were.
- engine: the decode colour step as CameraColourModel, with clipped_white, which
keeps the camera's neutral for nearly singular or implausible models.
- Tests in raw (continuity along a ramp through every clip, also as lines 1 to 16
rows thick with no reliable colour, on canvases 32 to 1024 rows), engine
(pipeline renders at Highlights -100, #523's edit and Exposure -2 through the
first clip, over the same sweep; a CC0 ILCE-7RM4 corpus file added to
RAW_SAMPLES) and the GPU equivalence suite.
- Render cache version 27.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
High Efficiency NEFs (HE / HE★, Z 8, Z 9, Z 6III, Z f) keep compression
34713 but carry a JPEG XS-based codestream (SOC + CAP, FF10 FF50) and no
maker-note 0x0096 table. They fell through to "Nikon compressed NEF
without a linearization table", so the Preview only notice gave Z f users
no hint why their raw wasn't decoded or that Lossless compressed NEFs work.
Detect the codestream before asking for the table and report "Nikon High
Efficiency NEF" with the workaround. Add a synthetic regression test, CC0
Z f HE + Lossless samples to the corpus (raw.pixls.us) with a corpus test,
and list HE NEF among the format gaps in parity.md and ROADMAP.md.
No decoder: the format has no public specification and the existing
decoders (LibRaw, rawler) are LGPL and derived from Nikon's binaries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011bbmJTxYJyYqKfNH8uXRC6
A DNG can state a white level the sensor never reaches (an iPhone 15 Pro Max Bayer DNG states 4095 but clips at 3567), so clipped highlights were not recognised and white balance tinted them magenta. On decode, samples that form a plateau clearly below the white level, with nothing above it, are raised to the white level. RENDER_CACHE_VERSION 25 (needs adjusting at merge).
Lightroom Classic 15.6 imports an XMP packet's channel curves only when it holds
ToneCurvePV2012 and all three of ToneCurvePV2012Red/Green/Blue; any partial set
(master + one or two channels, channels without the master) imports without
channel curves. #416 read them whenever the master curve was present.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #535 (items 4 and 5).
- Pre-2017 bodies without Sony crop tags shot in the camera's 16:9 mode now get
the 16:9 crop of their camera JPEG: width from FullImageSize, anchored left as
before (drops the ILCE-7SM2's 32 padding columns); height from FullImageSize
(ILCE-7SM2) or from an Exif image size of the same width (DSLR-A580, whose
FullImageSize still says 3:2); centred vertically at an even offset, as
measured by registering each camera JPEG on its raw. The aspect now matches
the JPEG, so the camera-look fit runs: ILCE-7SM2 dE 15.2 -> 4.8,
DSLR-A580 dE 26.7 -> 3.9 (vs main 88ea1a8).
- The DSC-R1's SR2 (maker-note FileFormat 1 0 0 0) stores encrypted samples and
decoded to noise; it is now Unsupported in full and header mode and imports
preview-only with that reason.
- Only these 3 of 140 Sony decodes change; render cache version 26 (23-25 are
taken by #568, #542, #575).
- Tests: crop helper (3:2 unchanged, both 16:9 sources, even offsets, hostile
sizes), synthetic Sony file with FileFormat SR2 vs ARW, corpus test of the
two crops and the SR2 refusal; three samples added to the corpus list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #535 (item 1). Stacked on #542 (SR2 black levels); supersedes its
positional keystream tables.
- White balance is WB_RGGBLevels (0x7313), the gains the camera applied:
the raw IFD's plain copy when present, else the copy in the encrypted
SR2SubIFD, which every ARW from the DSLR-A200 on carries (identical to
the plain one on all 57 CC0 files that have both). The maker note's
Tag2010 gains stay only as a last resort.
- 14 bodies had no white balance and rendered green whenever the camera
look was rejected (A450 ... A700, NEX-3/5/C3, SLT-A33/A35/A55,
ILCE-6001, UMC-R10C). On others Tag2010 missed the applied gains by up
to 100 % (ILCE-3500 at 5600 K: R -31 / B +101 % against the camera
JPEG's neutral pixels, now +4 / +0 %; ILCE-7S on Shade -9 / +15 ->
-1 / +1 %; SLT-A57 -25 / +26 -> -6 / -3 %).
- New vendor/sr2.rs decrypts the whole SR2SubIFD, found black-box:
exiftool -v5's decrypted dump as known plaintext gives the keystream
(key 11 22 33 44, the same at every position in 126 CC0 ARWs from 97
bodies, 15 layouts); an exhaustive search over lag pairs found
k[n] = k[n-63] ^ k[n-127] on 32-bit words, so the first 127 words
generate the rest, matching all 126 files (5.2 MB of keystream).
- The SR2 black level (0x7310) is now read through the same decryptor,
replacing #542's head and per-position keystream tables: one SR2
decryption path. Identical results on all 131 Sony files (every
value #542 decoded; the 51 it could not are the files' plain 0x7310).
Unit tests pin the keystream against answers recorded without the
generator: every byte #542 and #148 had recovered, the SHA-256 of the
whole 62112-byte keystream recovered from the CC0 DSLR-A700 sample, and
the encrypted WB_RGGBLevels of four CC0 files (one per layout) with the
levels ExifTool decrypts.
- Sweep (4f350c4 -> this): gains equal ExifTool's decrypted
WB_RGGBLevels on all 100 decoded CC0 CFA files; on 83 files with
>= 200 JPEG-neutral pixels, within 10 % of the camera's applied gains
71 -> 83, over 10 % 5 -> 0. Accepted fits unchanged (max 0.15 dE);
forced fallback median dE 18.50 -> 18.06, neutral-area cast on the 22
changed files 8.8 -> 1.2.
- Corpus test for five CC0 samples (A500, A33, A700, ILCE-3500 5600 K,
ILCE-7S Shade; three added to RAW_SAMPLES); the pre-2017 test's gains
move to WB_RGGBLevels (RX100 III, RX100, ILCE-7RM2: within 2.6 % of
the old Tag2010 values in Auto). Synthetic tests for decryption,
directory parsing, source precedence, black level through the full
DNGPrivateData chain, and hostile input; #542's layout tests (incl.
the A700 trap) now run through the decryptor.
- Render cache version 25.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #535 (items 2 and 3).
- SR2SubIFD black level: decrypt the directory's first entry (always the
black level, 0x7310) to find where its value sits, and decrypt it with the
keystream known for that position (1638, 2166, 2418, 2510). The fixed read at
2510 gave 354-365 on the DSLR-A450/A500/A550 and 975 on the DSLR-A700, where
ExifTool and the data floor say 512 (A700 render: dE 20.1 -> 8.8,
dL* -9.8 -> 0.0). Keystreams recovered black-box (plain 0x7310 and exiftool -v3
as known plaintext), consistent over 115 ARWs from 97 bodies, 15 layouts.
- Downsized lossless M/S ARWs (linear YCbCr): black 1024 and white 16895, i.e.
the recorded 512 plus 512, measured against same-scene full-size codings on
12 files from 4 bodies (offset 1015-1052, scale 0.98-1.02). The 512 pedestal
lifted shadows by ~3 % of white; M/S renders now match their full-size twins
(7RM5 M dE 6.7 -> 3.1).
- Only these 16 of 140 Sony files change; render cache version 24.
- Tests: synthetic SR2 blocks for every known layout (incl. the A700 trap),
YCbCr black/white; corpus test of black levels against the data floor;
A500 and A700 samples added to the corpus list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fitted with `lightcraft-cli calibrate` to 125 usable ILCE-7CR ARWs (of 131,
September 2025 to September 2026, Standard creative style) and their
in-camera JPEGs; aggregate colour statistics only.
Without it, 8 of the 131 photos fail the fit gates and open with the
neutral matrix (dE76 6-26 against the camera JPEG). On 26 withheld photos
(profile fitted to the other 105) mean dE76 goes 4.07 -> 3.29 and both
withheld rejections are accepted (26.0 -> 6.1, 6.2 -> 5.5); the 24
withheld photos whose own fit was accepted are about unchanged (12
improve, mean 3.06 -> 3.08, worst regression 1.8). With the final profile
all 131 are accepted. The nine public raw.pixls.us ILCE-7CR codings stay
accepted (mean 1.59 -> 1.99).
A corpus test checks the public lossless-L sample's fit keeps exactly the
built-in profile's matrix and table.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three CC0 raw.pixls.us files of one ILCE-7CR (61 MP) scene: lossless
compressed L and M (subsampled YCbCr) and compressed ARW2. A raw corpus
test pins model, colour-filter layout, black level, white level, as-shot
gains and default crop, and checks that the lossless and compressed
mosaics agree inside the crop (green block correlation 0.9976). An engine
corpus test checks each gets a fitted camera look, not the neutral
fallback. All nine public codings decode and fit (mean dE76 1.59 against
their camera JPEGs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reuse the landed current-view deletion selection logic for photo.deletePermanently, retaining survivors or choosing the next neighbour and the previous at the end. Preserve catalog-only deletion and existing undo/persistence behavior.
Refs #441, #451, #545.
Sony's Dynamic Range Optimizer (factory default Auto) brightens the darker
regions of the camera JPEG, not the raw. The tone curve fitted to that JPEG
carried the brightening into the raw's starting look: on 8 ILCE-7CR photos
with DRO on, LightCraft rendered +6.4 L* from Lightroom's default (median
dE76 7.5), against -0.2 L* (3.6) for the 16 with DRO off.
- lightcraft_raw::embedded_preview_dynamic_range_optimized reads Sony
maker-note tag 0xb025 (every documented value: 0 off; 1-3, 8-12 and
16-23, i.e. up to Lv8, on). It was present on all 264 Sony raws checked
(98 bodies); 0xb04f only says on/off and is missing on older bodies.
- With DRO on, the look keeps the colour fitted to the JPEG and its tone
curve is lowered, knot by knot, to Sony's curve without DRO wherever that
is darker (assets/camera-tone/sony-dro-off.json, pooled from 12 CC0
raw.pixls.us ARWs of 10 interchangeable-lens bodies; sources and
derivation in the file and assets/ATTRIBUTION.md). The second-last knot
is lowered only as far as keeps the shoulder past the last knot at or
below the fitted curve, so the result is never brighter anywhere.
Compacts that map raw values darker keep their own curve.
- DRO off, unknown and other makers: unchanged (byte-identical renders on
71 + 35 photos). Render cache version 23.
DRO-on photos now land at +0.8 L* from Lightroom (median dE76 3.3), band by
band like the DRO-off ones. Docs: camera-preview-colour.md, parity row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A catalog written before the lens data was read at import (RW2/RWL from
v0.4.0) has none stored, and the lens correction stays off. Reload now
stores the fresh probe's lens data (new journaled, undoable
Op::SetEmbeddedLens; catalog format 4) and, for photos still at import
settings, switches the correction on as import does. Edited photos and
photos with a default-preset look keep their settings.
Brandon Thomas