Claude
951d52788a
Measured against the darkest frame on real ARW brackets, each frame is linear to 1 % up to the clip far from clipped pixels, but within 2 px of them the +2 EV frame's green reads 3–5 % low: demosaicing interpolates it from clipped photosites. That frame carries the most weight (k = 4) and those pixels were unclipped themselves, so they kept it, and bright windows took a magenta tinge. A frame's weight now goes by its brightest channel within 1 px (3 × 3; wider margins changed nothing more). On two real interior brackets (ILCE-7M3, 0 / −2 / +2 EV), bright pixels whose green against red and blue is over 10 % below the darkest frame's, in camera RGB: dining room 0.32 % → 0.08 %, kitchen 2.86 % → 0.28 %. Rendered at −3 EV with main's colour tags (#1123), window pixels with G < 0.85·min(R, B) go 50 → 3 and 16 → 10 (the darkest frame alone: 2 and 205); most of the visible magenta those brackets showed before came from the missing colour tags. Test: a bracket whose +2 EV frame has 5 % less green next to its clipped pixels merges without the deficit (fails before: −1.96 %). Docs: docs/merge.md, the LR-MERGE-HDR row, docs/whats-new.md (also for the frame-order fix). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015gRtdWF1E9358FfzhzYqCS
4.9 KiB
Photo Merge (HDR, Panorama, HDR Panorama)
crates/merge (layer L3, no UI dependencies) implements the algorithms; crates/engine/src/merge.rs
plans, runs and finishes merges in the library; the desktop UI shows a dialog with a live preview.
All algorithms are our own implementations from published papers (cited in the module docs).
Using it
- App: select the photos, then Photo Merge ▸ HDR… (
Ctrl+H), Panorama… (Ctrl+M) or HDR Panorama… from the photo context menu (or thedialog.merge*commands). The dialog previews the merge at ≤ 1024 px and re-runs the preview when an option changes; Merge runs the full merge in the background (a row in the activity stack shows its progress, with ✕ to stop it). The result is written next to the first photo as<name>-HDR.dng,-Pano.dngor-HDR-Pano.dng, imported and selected. - Commands (CLI, MCP, control channel):
merge.hdr {ids?, align=true, deghost=none|low|medium|high, autoSettings=true, stack=false, preview=false, showOverlay=false, previewPath?}merge.panorama {ids?, projection=auto|spherical|cylindrical|perspective, boundaryWarp=0..100, autoCrop=false, fillEdges=false, autoSettings=true, maxMegapixels=40, preview, previewPath?}merge.hdrPanorama {ids?, bracket=0 (from EXIF), …both sets of options}
- CLI:
lightcraft-cli merge hdr|panorama|hdr-panorama [options] FILES…;lightcraft-cli synth-merge hdr|panorama -o DIRwrites synthetic inputs (procedural scene).
What it does
HDR (hdr.rs): frames ordered by EXIF exposure (or median luminance); the middle one is the
reference. Auto Align: features on exposure-normalised luminance, RANSAC translation / similarity /
homography (the simplest model that explains ≥ 97 % of the matches). Exposure ratios are measured
(median log ratio of well-exposed pixels between brightness neighbours), EXIF is the fallback.
Radiance is a weighted mean in linear light, weight k · (1 − smoothstep(0.8·clip, 0.96·clip, max channel within 1 px)): a pixel next to clipped photosites was demosaiced from them and reads low in the clipped
channel, so it goes with them (on real ARW brackets the brightest frame's green was 3–5 % low there, and bright
windows took a magenta tinge). The frames are taken in brightness order (ties by content), so the result doesn't
depend on the order they come in. Deghost: a per-pixel consistency test against the reference radiance (with a noise
floor), dilated and feathered masks; the union is the deghost overlay. Output: 16-bit float DNG.
Panorama (pano/): MOPS-style features, pairwise RANSAC homographies with Brown & Lowe's
verification, focal from homographies (or EXIF 35 mm), rotations chained along a maximum spanning
tree, Levenberg–Marquardt bundle adjustment of rotations + focal lengths, straightening.
Projections: spherical, cylindrical, perspective; Auto picks perspective for compact fields of
view < 110°, cylindrical when the vertical field is < 90°, else spherical. Exposure: EXIF, then
gain compensation from the overlaps. Seams: each pixel goes to the image with the highest
centre-weighted feather weight; multi-band blending (Laplacian pyramids) in log space. Boundary
Warp: a separable stretch of the smoothed valid-area edges to the canvas edges. Auto Crop: the
largest rectangle of image data (set as the develop crop, so it stays adjustable). Fill Edges:
push–pull diffusion. Output: 16-bit integer DNG (16-bit float for HDR panoramas).
DNG output (output.rs): LinearRaw, 3 samples, Deflate tiles; values scaled below 1.0 with the
scale in BaselineExposure; raw sources keep their DNG colour tags (camera RGB), other sources get
ColorMatrix1/ForwardMatrix1 of their RGB space.
Measured (synthetic tests, cargo test -p lightcraft-merge)
- HDR, 3 brackets ±2 EV with 14-bit noise and handheld shifts: alignment error < 0.3 px, exposure error < 0.02 EV, radiance median log error < 0.03 (noise-limited, bias < 0.01), recovered highlights < 0.05; moving object: ghost error 0.2–2.0 without deghosting, < 0.06 with High.
- Panorama, 5 views over 182° with a 1.3× exposure drift: relative rotations < 0.1°, focal < 1 %, BA RMS ≈ 0.35 px; median log error vs ground truth < 0.02; no seam steps.
Limits (known gaps)
- No lens-distortion model in the panorama bundle adjustment (rectilinear lenses assumed); no 360° wrap-around; flat (translating-camera) scans are fitted with the rotation model.
- No camera response calibration for JPEG brackets (sRGB decoding is taken as linear light).
- Boundary Warp is a separable stretch, not a content-preserving mesh warp; Fill Edges is diffusion, not patch synthesis.
- Create Stack: the result goes on top of a collapsed stack of its sources (
Catalog::stack_with_ops); its caption also lists them. - Panorama output is limited to
maxMegapixels(default 40) — blending memory is ~25 bytes/pixel. - The browser build has no file access for merges yet.