Nikon bodies from before ~2016 record no CropArea, so the decoder kept
their whole sensor data, less trailing dark columns. Where that data
holds masked lines above black or optical black, they were developed as
a coloured or dark bar on that edge: the D7000 and D5100's 36 columns at
~600 (#976), the D800's 14 (#914, the top in portrait), the D600 / D610's
8 (#1236), the D3200's 46, and others.
MEASURED_BODIES (crates/raw/src/vendor/nef.rs) now gives 44 more model /
data size / depth groups the frame of their camera JPEG inside the data,
next to the D4, D7100 and D90 entries already there: D7000, D5100, D800,
D800E, Df, the D4's 12-bit mode, D3, D3S, D700, D300, D300S, D600, D610,
D810, D4S, D5600, Coolpix A, D3000, D40X, D60, D80, D2Xs, D200, D3200
and Nikon 1 V1; and, on best-guess frames short of the evidence rule,
each with a comment naming its evidence, the D3100 (3 of 4 clean files
agree), D5000 (4 of 5, and the D90's sensor), D40 (2 of 2), D50 and D70
(one file each, agreeing) and D70s (next to them, between pixels).
Every best-guess frame keeps the masked lines out by several pixels.
An entry matches model, data size and depth exactly. Bodies whose data
holds only picture beyond the camera frame get no entry and render as
before.
`cargo xtask raw-frame FILE...` (crates/engine/examples/raw_frame.rs)
measures a body: it aligns each raw's embedded camera JPEG with the
sensor data (half-size luminance, gradient correlation, sub-pixel peak;
weak, ambiguous or between-pixel peaks are flagged), counts the edge
lines that hold no picture and reports what the decoder's crop keeps
beyond the camera frame. A masked line is flat and its two CFA phases
agree within the noise or within 0.4% of white, so filler with a small
fixed offset between them (D90, D5000) counts. It prints entries only
with evidence: at least 3 clean files agreeing exactly, of the model
(12- and 14-bit pooled) or, with at least one of its own, with sibling
models of the same readout (data and JPEG size) that agree on the same
frame. On the D7500, which records its CropArea, it finds that area
exactly; it declines the Z f, whose JPEG is distortion-corrected. It
works for any make whose raw embeds a full-size camera JPEG: on Sony,
Canon CR2 / CR3, Pentax and Samsung files it finds the decoder's own
crop (relative to the active area).
Every Nikon raw on raw.pixls.us without CropArea was surveyed this way
(its 138 CC0 samples, plus 124 other publicly posted sample files; only
the measurements are committed). Bodies whose data holds only picture
beyond the frame need no entry (D750, D5300, D5500, D3300, D2X, D2Hs,
D3X, D5200, Coolpix P6000, Nikon 1 AW1 / J3 / V2 / V3, the D800's crop
modes); raw-parity.md lists them, and no longer claims every NEF
coding but HE decodes. measured_bodies_are_consistent checks every entry lies
inside its data and is listed once; measured_older_dx_bodies_image_area
covers the D200, D3200, Nikon 1 V1 and best-guess entries.
Fixes#976Fixes#914Fixes#1236
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Setting up Synchronize Folder, an import review or an import made a table
of every photo in the library on the session's thread before the first
file was looked at, and naming one photo's sidecar worked out who owns
every shared sidecar name in the library. In a library of 3.5 million
photos (`scan_setup_bench`, release, a folder of 500): 5.8 s before a
Synchronize scan, 2.6 s before an import review, 1.7 s before an import,
2.4 s for each sidecar written.
- A scan keeps the library as it is (photos are shared, not copied) and
looks up the files it finds (`library_files`) and the contents it probes
(`library_hashes`) on its own thread.
- An import looks the library up with its first files
(`ImportJob::look_up_library`), from the library and the options as they
were when it was set up.
- The owners of shared sidecar names are worked out for the photos asked
about (`StemOwners::among`): all five callers name only those photos.
- Synchronize Folder names its photos' sidecars in the scan, after it has
seen that the folder is there.
Now 0.9 s, 0.05 s, 0.05 s and under 1 s. What is found, and how sidecars
are named, is unchanged: the tests hold the owners against the table of
the whole library as it was made before, and the key made without
allocating against the key, for every string of up to six of the
characters that decide it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- drag_latency (ui-egui example): the whole app driven headless at 60 frames a second; per develop control,
the latency of each drag tick until its draft is the loupe (or, with ZOOM, the zoom window) texture, the
ticks never shown, the UI thread's frame time and the time to the full-quality render after release.
- drag_tick_bench (engine example): the UI-thread work of one drag tick outside the render.
- render_bench: ONLY=window (a 1:1 zoom window cut from the original: first window, pan, slider changes;
GPU_ONLY=1, DUMP=file for pixel A/B), ONLY=load / load1 / jpeg (+ LOADONLY=1): the steps of opening a photo.
The window rows carry no colon in their names, so `cargo xtask bench` records each of them (parser test).
- open_timeline (ui-egui example): opening raws in Detail through the real app logic, headless with the GPU.
Co-Authored-By: Claude <noreply@anthropic.com>
A batch ran its photos strictly one after another: each render is
row-parallel inside, but a photo's decode, its PNG/TIFF/WebP encode and
the wait on a GPU render are mostly serial, so a 32-core machine averaged
~9.5 busy cores on the CPU path and 1-2 with the GPU.
run_batch now keeps export_parallelism() photos in flight (cores / 4,
1..=4; one on wasm32) on scoped lane threads plus the calling thread.
Each holds an estimate of its working memory from a new export_gate
(twice the memory budget, separate from the decode gate it nests over),
so full-size 24 MP exports (~1.2 GB each) run two at a time by default.
Lanes run at most `lanes` photos past the last one placed, which also
bounds the finished files waiting for their turn. Paths, conflict
handling, original protection, writes, results and progress stay in
batch order (Placer), so the files are the same as one at a time; a
panic in a lane becomes that photo's error.
render_bench's batch scenario now goes through export_batch. 8 x 24 MP
JPEGs, 32 cores + RTX 4080 Laptop: CPU 3.87 s -> 2.35 s, GPU 4.21 s ->
1.82 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The RW2 reader reads PanasonicRaw tag 0x0119 (16 LE words) and expresses the
camera's own correction as the equivalent DNG WarpRectilinear in OpcodeList3,
so embedded_lens() turns it into the existing EmbeddedLens/EmbeddedWarp and
"Enable Profile Corrections" applies it like a DNG's warp (no new warp path;
Convert to DNG keeps it). Rule, from our own measurements against the
embedded camera JPEGs (clean-room; ExifTool tag names only):
corrected = rho*(1 + (w8 rho^2 + w4 rho^4 + w11 rho^6)/32768), rho = r/w12
(w12 = active-area half diagonal), output = corrected / (1 + w5/32768),
on when (w7 & 0xf) == 1. Chosen on 38 GX85 photos, held out on 35 photos
from 12 other bodies / 9 other lenses; corrected renders line up with the
camera JPEG within 0.75 px at 1440 px at the corners (up to 13 % of the
half diagonal before). Implausible values leave the photo uncorrected.
The loader now removes lens opcodes before the RW2 camera-look fit (its
binned proxy needs an empty OpcodeList3); Opcode::is_lens_correction()
replaces the duplicated matches. RENDER_CACHE_VERSION 11 -> 12.
Fix-ups: diagnostic overlays never populate the view-preview cache; Photo Merge's RenderRequest gets
the new overlay field.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Up to 8 OkLCh samples (pointColor.pick samples the colour after the mixer on a proxy render);
per sample hue/saturation/luminance shift, variance (compress/expand the spread around the
sample) and an overall + per-axis range with soft edges, applied after the colour mixer in
ColorOps and its WGSL twin (equivalence case). Indexed control specs
(pointColor.<i>.hueShift …) so develop.set/controls/MCP address each sample. Visualize range
is a render overlay (RenderRequest::overlay) drawn after either renderer: the sample's own
edit is left out and colours outside the range turn grey. Point Color flyout in the Color
section with swatches, eyedropper, sliders, visualize and delete.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lightcraft-raw: RawImage::develop_binned(k) — k×k superpixel binning of the CFA
(Bayer k even, X-Trans k multiple of 3), black/white normalised row by row, default
crop only, clipping preserved for highlight reconstruction. 2560 px previews of a
24 MP raw bin 2×, thumbnails 4–8×; exports / 1:1 still demosaic (AHD) at full size.
- highlight::reconstruct and its pyramid fill are rayon-parallel (bit-exact).
- X-Trans directional demosaic: precomputed per-phase neighbour offsets and no
reflection in the interior, no per-pixel allocation (bit-exact, tested against the
reference formulation): 2.9 s → 0.75 s CPU at 24 MP.
- files::load_bytes runs on a rayon worker so its short parallel loops don't each
wait for the pool to wake (dominant cost on a loaded machine); LIGHTCRAFT_PROFILE
prints per-stage source timings; files::bin_factor picks the block size.
- render_bench ONLY=source: per-file decode / normalise / demosaic / binning / load.
- Fix: u64 overflow on hostile IFD sizes in DNG/ARW/NEF raw-IFD selection (proptest).
load_bytes(2560) CPU time, ARW a7 III 24 MP: 2347 → 240 ms; X-T20: 3120 → 1220 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`media::develop` renders on the GPU when available (views, render_now, exports;
thumbnails stay on the CPU) and falls back to the CPU otherwise. `app.gpu
{enabled?}` toggles it at runtime, `LIGHTCRAFT_GPU=0` disables it for the process,
`ui.inspect` reports the adapter under perf.gpu.
Performance: dropped device buffers are recycled once their thread has submitted
(fresh 100-300 MB allocations per pass cost as much as the passes), Gaussian
passes ping-pong between two buffers and read all channels of a pixel at once,
and long box radii slide over longer chunks. LIGHTCRAFT_PROFILE prints real GPU
stage timings. render_bench prints CPU and GPU columns and the CPU/GPU difference.
24 MP ARW, loaded machine, min of 7: cold loupe 415 -> 32 ms, exposure drag
22-42 -> 3.9 ms, NR drag (1920) 301+ -> 18 ms, export render 1037+ -> 291 ms;
max 1 LSB difference. docs/gpu-pipeline.md describes the implementation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The guided base, clarity band, texture band and dark channel (and the paired blurs inside the
guided filter) are independent; each alone scales poorly (subsampled grids, short rows), so they now
run under raster::par_join. Same arithmetic, identical pixels. render_bench gains a per-feature
breakdown of the per-pixel stage (ONLY=finish).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Exports above 2560 px were rendered from the 2560 px preview proxy and upsampled. SourceLevel::Full
(native size; procedural demo scenes at their nominal size) is now selected for outputs larger than
a preview; the media cache keeps only the last full-size original. render_bench gains a batch-export
scenario (decode + render + JPEG per file).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- pipeline: StageCache + render_cached: resampled source, white-balanced/retouched/denoised image
and each spatial plane (log L, guided base, clarity, texture, dark channel + airlight) are keyed
by exactly their inputs; keeps a draft-size and a full-size entry warm.
- Exposure moved after the spatial filters (applied per pixel; planes shifted by EV): identical up
to rounding (test: <= 2 LSB vs. exposing the source), so exposure drags hit the cache too.
- engine: RenderJob::draft / with_stages; ui: loupe and before views own a StageCache, loupe
renders at Draft quality while interacting.
- engine example render_bench (min-of-N wall + process CPU time).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
