The ORF decoder supplies the as-shot white balance (ImageProcessing
WB_RBLevels) and no colour matrix, like ARW, NEF, RW2, RAF, CR2/CR3, PEF
and SRW, but ORF was left out of camera_preview::file_local_look and the
catalog's relative_wb, so it rendered with the neutral fallback matrix:
flat, grey and dark. It now gets the same guarded fit to its camera JPEG
and white balance relative to the as-shot look. Corpus (held-out RMS
against the fallback, all accepted): E-1 0.126 -> 0.014, E-400 0.087 ->
0.022, XZ-2 0.061 -> 0.023; compressed ORFs still open from their preview.
Against Lightroom Classic (CC0 E-1 kit, 2000 px, ΔE00): unedited render
11.25 -> 8.30; mean over 86 variants abs 12.10 -> 9.48, effect 3.44 ->
2.80. Every other kit's unedited render is byte-identical.
RENDER_CACHE_VERSION 23.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sensor-clipped pixels are rebuilt at raw decode. The tone map rolls them off to
white at Neutral, so any colour they carry shows only when Highlights, a negative
Exposure or Whites darkens them. They took colour from the nearest unclipped pixel
(dark foliage, lens fringes along branches, pixels demosaiced from clipped samples),
left two clipped channels at their white-balance-scaled clip levels (red or pink),
and were neutral only in camera RGB, not after the colour model (a look fitted to
the camera's JPEG, a camera profile).
- highlight::reconstruct_with: chromaticity only from bright unclipped content away
from edges and from the clip, through a normalised push-pull pyramid; a neutral
estimate blended in where reliable content is scarce at every scale up to a
quarter of the long edge, applied only as far as the pixel lies past the nearest
near-clip unclipped pixels (so a surface still runs into clipping without a
step); clipped pairs keep the estimate's ratio at their clip floors; partly
clipped pixels fade into the clipped neutral on their way from the first channel
clipping to the last; fully clipped pixels become the colour model's neutral.
Pixels without a clipped channel are left exactly as they were.
- engine: the decode colour step as CameraColourModel, with clipped_white, which
keeps the camera's neutral for nearly singular or implausible models.
- Tests in raw (continuity along a ramp through every clip, also as lines 1 to 16
rows thick with no reliable colour, on canvases 32 to 1024 rows), engine
(pipeline renders at Highlights -100, #523's edit and Exposure -2 through the
first clip, over the same sweep; a CC0 ILCE-7RM4 corpus file added to
RAW_SAMPLES) and the GPU equivalence suite.
- Render cache version 27.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
C5060WZ, C7070WZ, SP-510UZ, SP-550UZ, SP-565UZ and SP-570UZ files say
uncompressed 12-bit with many strips whose byte counts sum to exactly
width x height x 1.5. main refused them because the 12-bit branch wanted a
single strip, so they fell to "Olympus compressed ORF" and opened
preview-only. The samples are packed MSB-first, the strips hold two fields
(even output rows, then odd ones) with one gap at the seam, and the last row
of the second field sits right after the last strip. Accepted only when all of
that holds exactly. CFA comes from the Exif tag, white balance from maker-note
entries 0x1017/0x1018, black 0. RENDER_CACHE_VERSION 27 (adjust at merge).
A DNG can state a white level the sensor never reaches (an iPhone 15 Pro Max Bayer DNG states 4095 but clips at 3567), so clipped highlights were not recognised and white balance tinted them magenta. On decode, samples that form a plateau clearly below the white level, with nothing above it, are raised to the white level. RENDER_CACHE_VERSION 25 (needs adjusting at merge).
Lightroom Classic 15.6 imports an XMP packet's channel curves only when it holds
ToneCurvePV2012 and all three of ToneCurvePV2012Red/Green/Blue; any partial set
(master + one or two channels, channels without the master) imports without
channel curves. #416 read them whenever the master curve was present.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fitted with `lightcraft-cli calibrate` to 125 usable ILCE-7CR ARWs (of 131,
September 2025 to September 2026, Standard creative style) and their
in-camera JPEGs; aggregate colour statistics only.
Without it, 8 of the 131 photos fail the fit gates and open with the
neutral matrix (dE76 6-26 against the camera JPEG). On 26 withheld photos
(profile fitted to the other 105) mean dE76 goes 4.07 -> 3.29 and both
withheld rejections are accepted (26.0 -> 6.1, 6.2 -> 5.5); the 24
withheld photos whose own fit was accepted are about unchanged (12
improve, mean 3.06 -> 3.08, worst regression 1.8). With the final profile
all 131 are accepted. The nine public raw.pixls.us ILCE-7CR codings stay
accepted (mean 1.59 -> 1.99).
A corpus test checks the public lossless-L sample's fit keeps exactly the
built-in profile's matrix and table.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three CC0 raw.pixls.us files of one ILCE-7CR (61 MP) scene: lossless
compressed L and M (subsampled YCbCr) and compressed ARW2. A raw corpus
test pins model, colour-filter layout, black level, white level, as-shot
gains and default crop, and checks that the lossless and compressed
mosaics agree inside the crop (green block correlation 0.9976). An engine
corpus test checks each gets a fitted camera look, not the neutral
fallback. All nine public codings decode and fit (mean dE76 1.59 against
their camera JPEGs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reuse the landed current-view deletion selection logic for photo.deletePermanently, retaining survivors or choosing the next neighbour and the previous at the end. Preserve catalog-only deletion and existing undo/persistence behavior.
Refs #441, #451, #545.
Sony's Dynamic Range Optimizer (factory default Auto) brightens the darker
regions of the camera JPEG, not the raw. The tone curve fitted to that JPEG
carried the brightening into the raw's starting look: on 8 ILCE-7CR photos
with DRO on, LightCraft rendered +6.4 L* from Lightroom's default (median
dE76 7.5), against -0.2 L* (3.6) for the 16 with DRO off.
- lightcraft_raw::embedded_preview_dynamic_range_optimized reads Sony
maker-note tag 0xb025 (every documented value: 0 off; 1-3, 8-12 and
16-23, i.e. up to Lv8, on). It was present on all 264 Sony raws checked
(98 bodies); 0xb04f only says on/off and is missing on older bodies.
- With DRO on, the look keeps the colour fitted to the JPEG and its tone
curve is lowered, knot by knot, to Sony's curve without DRO wherever that
is darker (assets/camera-tone/sony-dro-off.json, pooled from 12 CC0
raw.pixls.us ARWs of 10 interchangeable-lens bodies; sources and
derivation in the file and assets/ATTRIBUTION.md). The second-last knot
is lowered only as far as keeps the shoulder past the last knot at or
below the fitted curve, so the result is never brighter anywhere.
Compacts that map raw values darker keep their own curve.
- DRO off, unknown and other makers: unchanged (byte-identical renders on
71 + 35 photos). Render cache version 23.
DRO-on photos now land at +0.8 L* from Lightroom (median dE76 3.3), band by
band like the DRO-off ones. Docs: camera-preview-colour.md, parity row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A raw whose camera look fails the acceptance gates fell back to the neutral
rendering: dark (9-22 L* below the camera JPEG), about a third of its
colourfulness, hues from the generic camera = sRGB matrix. Before falling
back, fit a partial look: the quantile tone curve (registration-free), the
chroma curve and, on a scene with enough colour, the photo's own matrix at
half weight (no hue/saturation table). It is used only when it cuts the
held-out error to below 0.7 of the neutral fallback's (linear and encoded)
and the JPEG's luminance ranks the pixels like the scene's (Spearman >= 0.5).
The full look's gates and search are unchanged. Render cache version 24.
Demo scene pixels were generated inside `OnceLock::get_or_init` on the
caller's thread, and the render runs on rayon. A rayon worker that waits
for rows another worker took runs queued jobs on top of itself; when one
of those asked for the same scene, it waited in `get_or_init` on the
generation further down its own stack, and every other request for that
scene waited behind it, forever. The faces scan's pool threads hit it (two
tests_face_recognize tests sat in `Once::wait` for 30 minutes); any rayon
worker loading demo photos could.
Each (scene, size) slot now has one generation at a time: the first caller
renders, with no lock held, and later callers wait on a condvar for that
generation's outcome. The render never needs a thread that waits:
- a rayon worker (the faces scan, merges) renders on its own thread alone,
without rayon (`Scene::render_fit_serial`, the same pixels), so it never
runs other jobs mid-render, and the work stays within its pool and that
pool's limits (scan pace, LIGHTCRAFT_FACE_THREADS);
- other callers (the render workers, plain threads) render in parallel on a
pool of rayon's default size (RAYON_NUM_THREADS, else the available
parallelism), whose threads only render; where it can't start, they
render alone too (no rayon, so no deadlock). Generating a scene never
touches the global rayon pool (asking its size would start it, which
panics where its threads can't start).
The scan's first render of a demo scene now takes 1 to 2.3 s on one
thread instead of 0.5 to 1.3 s on its light pace's two (M3 Max, 2560 px
preview source); interactive loads are unchanged.
A generation number gives each waiter the outcome of the generation it
waited for, even when a retry starts first. A render that panics is an
error for its callers (`scene_pixels` returns a Result) and the next
request tries again. No duplicates, no unbounded threads.
Tests (each waits with a timeout, so a deadlock fails instead of hanging):
rayon workers keep asking a 4-thread pool for a scene while it is generated
(24 replies counted, at least 4 overlapping; times out at 120 s on the
OnceLock code); a scene is rendered once however many ask (the first
version of this fix rendered 5 copies at once); a rayon worker renders
alone on its own thread; callers wait for the generation running; a failed
generation is an error and the next request retries; a retry never changes
the outcome waited for (state-level, and a racing retrier: 60 of 60
waiters got the retry's pixels on the previous version); in a process
whose global pool failed to start, a scene is still generated; the serial
render is the parallel one, pixel for pixel.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A catalog written before the lens data was read at import (RW2/RWL from
v0.4.0) has none stored, and the lens correction stays off. Reload now
stores the fresh probe's lens data (new journaled, undoable
Op::SetEmbeddedLens; catalog format 4) and, for photos still at import
settings, switches the correction on as import does. Edited photos and
photos with a default-preset look keep their settings.
A Custom white balance from a Lightroom catalog or XMP sidecar was taken as absolute Kelvin for every
raw. Raws LightCraft develops relative to their as-shot look (ARW, NEF, RW2, RAF, CR3, CR2, PEF; 6500 K
means as shot) then read a small tweak as a large blue shift: ΔE 22 against Lightroom's export on the
photos of issue #510.
The crs mapper now takes a `Target` (preset, rendered, raw with a measured illuminant, relative-WB raw)
instead of a bool. For relative-WB raws, and rendered files with only `Temperature`, a Kelvin value is
converted to the same mired shift from the packet's `AsShotTemperature` / `AsShotTint`. Without that
reference (Lightroom catalogs keep none for Custom photos) the photo stays As Shot and the import
report lists `Temperature, Tint` as not carried over. Import, sidecar read and catalog import decide
the target the same way through `lightcraft_catalog::relative_wb_format`.
Tests: shift from the issue's numbers (3650 → 3578 K lands at ~6275 K), the As Shot fallback with its
report entry, target selection per file, and the catalog mapper's Custom-WB case.
Each photo's develop settings record the rendering process they are
interpreted by (`DevelopSettings::process`). There is one process, V1, the
pipeline as it is; nothing renders differently (180 CPU 8/16-bit and GPU
renders of procedural scenes, and the saved settings JSON and hash64, are
byte-identical to main).
Settings saved without the field load as V1 (a field-level serde default),
new settings get the latest (`Default`), and V1 is left out when saving, so
files and preview-cache keys stay as they were. A damaged value (anything
but a whole number up to 2^32 - 1) reads as V1 too, so it never costs the
rest of the settings (an unparsable `lc:settings` sidecar is ignored). Reset moves a photo to the
latest process, section and slider resets don't; virtual copies inherit
it; copy / paste / sync / Auto Sync / presets don't carry it; a number from
a newer LightCraft is kept and renders with the newest known process. The
process is not an edit (Edited badges, import-look checks). The base tone
map is chosen per process in one place (`finish::base_tone`, shared by CPU
and GPU). Photo > Update to Current Process (`develop.updateProcess`) moves
selected photos on an older process to the latest as one undo step. Its
menu enablement follows the selection; a call that names photos (`ids` /
`id`) is validated on those instead, through a new opt-in
`CommandSpec::explicit_targets` (execute checks enablement before the
handler sees `ids`). `develop.get` reports the process. The rule and how to add V2:
docs/process-versions.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
proxies() fitted the decoded camera JPEG into the sensor proxy's box,
which keeps the JPEG's own aspect and can round a pixel short: an
ILCE-7RM4's 1616x1080 preview, decoded at 384x257, fits a 192x128
profile proxy as 191x128, and collect_pairs rejected the mismatch. So
`lightcraft-cli calibrate` skipped every file from this camera and no
profile could be written.
Resize the JPEG to exactly the sensor proxy's size instead; proxies()
has already rejected references whose aspect differs by more than 2%.
Where fitting already landed on the proxy's size the result is
identical (fit is resize at a rounded size), so working renders don't
change: 34 real ILCE-7RM4 ARWs render bit-identically. On 923 files
from one trip calibrate goes from 0 to 843 accepted.
Refs #515
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas