Widens the preview-only fallback. A recognised raw whose decode fails now falls back
to its embedded JPEG for any decode error (unsupported, corrupt, limit, TIFF
structure), not only an unsupported variant, and the preview search covers more
containers:
- Canon CRW, Minolta MRW and Sigma X3F are recognised by their magic (RawFormat
Crw, Mrw, X3f) instead of failing as "RawOther files are not supported yet".
- A TIFF whose IFD0 has no ImageWidth/ImageLength but a JPEG pointer or SubIFDs
(Kodak KDC) is a raw container, not a malformed image. TIFF compression 99
(Leaf MOS) counts as a raw compression.
- When no tag points to a preview, a recognised raw we cannot decode is scanned
for the largest complete three-component DCT JPEG. A preview whose first byte
was overwritten (Minolta, Epson) is restored. A one-component JPEG is skipped
(a PowerShot CRW stores its mosaic that way).
Files with no JPEG at all still fail with "without an embedded preview".
Phase One / Leaf IIQ, Canon EOS-1D / 1Ds and Kodak DCS TIFFs keep their raw data in
a private block. IFD0 is only a 160-640 px RGB copy and no IFD is marked as raw, so
they opened as an ordinary TIFF of that size (a 150 MP file rendered 296x220) with
no preview-only label.
probe() now recognises such a file from its structure (OtherTiff): a picture at
least 4x larger than IFD0 in both directions is declared by a SubIFDs child, or by
Exif PixelXDimension/PixelYDimension together with enough bytes outside every IFD's
strips and tiles to hold it at one bit per pixel. decode() reports the sizes as the
reason. A raw we cannot decode now falls back to its embedded JPEG, else to the TIFF
reader's view of its first image, so it imports preview-only with that reason.
RENDER_CACHE_VERSION 12: these files render differently.
Decode lossless CRX and version 0x100/0x200 C-RAW in portable Rust,
including sensor geometry, timed white balance and black/white levels.
Verify all sensor samples against six independent M50/R100/R8 references.
Any CR3 decode error falls back to the embedded JPEG, as CR3 files did
before this decoder. The full sensor buffer is allocated only after the
first plane decodes, and plane buffers grow row by row. Tile planes
decode in parallel with a windowed bit reader (24 MP: 424 -> 89 ms
lossless, 519 -> 138 ms C-RAW on an M4).
Add guarded CR3 camera-JPEG colour fitting with independent framing
validation, relative white balance and cache invalidation.
Validation: cargo xtask ci.
The RW2 reader reads PanasonicRaw tag 0x0119 (16 LE words) and expresses the
camera's own correction as the equivalent DNG WarpRectilinear in OpcodeList3,
so embedded_lens() turns it into the existing EmbeddedLens/EmbeddedWarp and
"Enable Profile Corrections" applies it like a DNG's warp (no new warp path;
Convert to DNG keeps it). Rule, from our own measurements against the
embedded camera JPEGs (clean-room; ExifTool tag names only):
corrected = rho*(1 + (w8 rho^2 + w4 rho^4 + w11 rho^6)/32768), rho = r/w12
(w12 = active-area half diagonal), output = corrected / (1 + w5/32768),
on when (w7 & 0xf) == 1. Chosen on 38 GX85 photos, held out on 35 photos
from 12 other bodies / 9 other lenses; corrected renders line up with the
camera JPEG within 0.75 px at 1440 px at the corners (up to 13 % of the
half diagonal before). Implausible values leave the photo uncorrected.
The loader now removes lens opcodes before the RW2 camera-look fit (its
binned proxy needs an empty OpcodeList3); Opcode::is_lens_correction()
replaces the duplicated matches. RENDER_CACHE_VERSION 11 -> 12.
load_bytes_now timed the binned demosaic with std::time::Instant, which
panics on wasm32-unknown-unknown ("time not implemented on this
platform"). Opening a raw in the web build aborted the decode. Use
web_time::Instant, as library.rs already does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Conflicts with issue #150 (NEF colour from the camera JPEG): the per-file
look, its hue/saturation/value table and chroma curve, and camera
profiles now apply to every `file_local_look` format (ARW, NEF, NRW);
`calibrate` also collects NEF/NRW. Render cache version 11 (main 9,
branch 10).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A sunny ILCE-7M4 waterfall shot rendered yellow: sunlit rock came out
cream, +4 b* in the brightest fifth of the picture. Camera JPEGs make
shadows up to 1.4x more colourful and bleach highlights to 0.1-0.3x
(a per-channel tone curve), which a luminance-only curve can't do.
- CameraTone gains an 8-node chroma-by-display-luminance curve, fitted
per file (bright pixels included) and kept only when it lowers the
held-out error; applied after the tone map on CPU and GPU, keeping
luminance. Old smart-preview headers load with an identity curve.
- The hue/saturation table gets an sRGB-encoded value axis: dark yellows
had to turn toward orange while bright yellow-greens turned toward
green. It is applied after the baseline exposure, as it is fitted.
- Matrix-only and matrix+table looks are completed (tone, chroma) and
compared in gamma-encoded values; acceptance gates stay linear.
29 local ARWs vs their camera JPEGs: mean dE 4.23 -> 3.54, every file
closer, saturated areas 7.65 -> 6.41. Render cache v10.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A global 3x3 matrix can't follow the camera's hue-dependent rendering:
on an ILCE-7M4 photo of a lime shirt among grey rocks the best matrix
rendered the shirt olive (16 deg hue error), since the few saturated
pixels carry little weight and a matrix fitted to them breaks the rest.
After the matrix, fit a 72x5 hue/saturation table (DNG ProfileHueSatMap
layout, linear ProPhoto) to the residual hue shifts and saturation
ratios: kernel-weighted per node, shrunk to identity where samples are
sparse, identity at saturation 0. It preserves luminance (the tone curve
owns it) and is kept only when it lowers the held-out error.
28 local ILCE-7M4 ARWs: mean dE vs camera JPEG 4.74 -> 4.27, none worse
overall; the shirt's hue error drops to 6-8 deg. Render cache v9.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
LightCraft has no colour matrix for Nikon cameras. Because of this, NEF
files showed colours that were too gray and too green.
- Colour: LightCraft now calculates a colour matrix and a tone curve from
the JPEG preview in each NEF file. Sony ARW files use the same
procedure and the same quality checks. White balance is relative to
the as-shot look, as for ARW files.
- One function, camera_preview::file_local_look, selects the formats for
the fit and for the relative white balance. Photo::relative_wb in the
catalog uses the same formats.
- Black level: Nikon stores the black level (maker note 0x003d) in
14-bit units, also in 12-bit files. The decoder read it as a 12-bit
value. Because of this, 12-bit files from the D750, D780, D850, D7500
and Z 50 were too dark. Now the decoder divides the value by 4 for
12-bit files.
- The corpus gets the D7500 12-bit and 14-bit lossless samples. A test
compares their levels. The render cache version increases to 9.
Results on 13 NEF files that LightCraft can decode: LightCraft accepted
the fit for all 13 files. The mean colour difference from the camera
JPEG (CIE76 dE) decreased from 13-46 to 2.9-7.8. One file with mixed
light improved by a small quantity only (12.7 to 12.0).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Threads the Vec<Region> that lightcraft-meta already reads from XMP
(MWG-RS) through to the screen, so it's no longer inert:
- catalog::Meta gains a `regions` field (catalog now depends on
lightcraft-meta, which doesn't depend back on catalog, so no cycle).
- crates/engine/src/files.rs (meta_of) and src/sidecar.rs (parse_sidecar
/ merge_into) populate it for both embedded-XMP and external-sidecar
photos, mirroring how `keywords` is already handled.
- ui-egui's loupe view (panels/detail.rs) draws each region as a
rectangle over the photo via the existing CanvasMap, color-coded by
kind (Face/Pet/other) with its name label if present, right alongside
the existing info overlay. Read-only, no new interaction.
New end-to-end test (regions_from_sidecar_are_read_on_import) covers a
region written to a sidecar reaching Photo.meta.regions through real
import, not just the parser. cargo xtask ci green (fmt, clippy -D warnings, test, parity, layers, assets, wasm).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Lightroom-converted DNGs carry their camera profile's ProfileHueSatMapData1/2
and ProfileLookTableData (and sometimes ProfileToneCurve). We used only the
colour matrices, so such DNGs rendered flat and muted.
- raw: new `profile` module reads the tags (DNG spec ch. 6: value-major,
hue-middle, saturation-minor tables, wrap-around hue, tri-linear lookups,
sRGB value encoding) from the raw IFD or IFD 0; malformed tags are ignored.
- engine: the raw loader applies the hue/sat map (interpolated by the same
illuminant weight as the matrices) before BaselineExposure and the look
table after it, in linear ProPhoto; a ProfileToneCurve becomes the source's
camera tone. Without one the standard base tone is kept.
- dngwrite keeps the look with the data (conversions, smart previews, merges).
- Render cache version 7.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Catalog Meta gets copyright_status (unknown / copyrighted / publicDomain), usage_terms and
copyright_url, read and written as XMP Rights Management fields (xmpRights:Marked, UsageTerms,
WebStatement) on import, sidecars and exports (kept by "copyright only"). photo.setMeta,
copy/paste metadata, metadata presets, a Copyright Status smart-album rule and the Info panel.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raw variants the decoder can't read yet (e.g. Nikon Huffman-compressed NEF
from a D3200/D5100) silently fell back to the camera's embedded JPEG, so
users edited a JPEG with the camera's picture style (e.g. Monochrome) baked
in without being told.
- catalog: Photo.preview_only (serde default; old catalogs load), set /
cleared by Op::SetContent with an exact inverse; Photo::develops_raw().
- engine: ProbeInfo.preview_only carries the decoder's Unsupported reason
into import, the import review (previewOnly) and catalog.query
(previewOnly); photo.inspect includes it. photo.reload and photo.relink
re-probe and clear/set it when the file decodes (or not) now.
- source_info: a preview-only raw is a rendered source (relative WB scale,
display tone curve) instead of raw:true/5500 K + filmic tone map on a
display-referred JPEG; import uses the JPEG default preset for it; the
Edit panel shows the rendered-file WB sliders.
- UI: "Preview" badge on grid thumbnails, a pill on the loupe, a notice in
Edit and Info (amber caution token, Info icon drawn in code).
- Tests: catalog serde/undo, engine import -> reopen -> reload/relink,
headless UI badge/notice widgets.
Refs #10
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Metadata gains alt text, extended description, sublocation, city, state/province, country:
catalog, photo.setMeta, copy/paste metadata, XMP read/write (Iptc4xmpCore AltText /
ExtDescrAccessibility / Location, photoshop City / State / Country), sidecars and export
(place fields dropped with "remove location").
- Info panel: camera card (camera, lens, size · file size, format badge, focal length,
shutter, aperture, ISO), then title, caption, alt text, extended description, copyright,
creator, file name (rename), file path (reveal), captured (edit), place fields, GPS.
- Text fields keep what is typed across frames and save on focus loss (they were rebound to
the stored value every frame). New pencil icon (drawn in code); dates::display_time.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- memory::budget (default min(25% RAM, 1.5 GiB); LIGHTCRAFT_MEMORY_MB, app.memoryBudget):
half for engine caches (sources + rendered previews, LRU across them via a global
use tick; the photo on screen stays), a quarter for decodes in flight (WorkGate:
thumbnails, prefetch and import probes wait; loupe/export loads never wait), an
eighth for the GPU free pool.
- GPU: best-fit buffer reuse (<= 25% slack), buffers released outside a render are
pooled at once (no stranded retired lists), pool trimmed after 3 s idle.
- Decodes drop file bytes and raw samples early, colour/crop/orient in place.
- macOS: apps install malloc_zone_pressure_relief as the release hook (after each
decode, import and idle trim): freed large blocks no longer stay resident.
14 raws + loupe + 12 steps (M4 Pro): peak RSS 1.9-2.3 GB -> 0.83-0.99 GB, footprint
2.5-2.6 GB -> 0.91-0.96 GB, heap peak 930 -> 656 MB; stepping median ~13 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decoders take a Mode: Header reads everything but the samples (DNG, CR2, ARW,
RAF, RW2; PEF when it has crop tags). NEF/ORF (and PEF without crop tags)
derive geometry/CFA from the samples and still unpack them. decode() stays
bit-exact (debug-hash of every corpus/raw decode identical before/after); the
corpus test asserts probe_info == decode().info() for all 37 files.
Import of 14 corpus raws: 3.4 s -> 1.4-2.0 s; peak RSS through import
1.41-1.49 GB -> 1.12-1.22 GB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- engine: QuickJob — what to show the moment a photo opens: the loupe's last render for the
current settings (kept by Session::loupe_job under a size-independent "view" key in the
preview cache, memory + disk with a library; the disk JPEG is written off the render
thread via PreviewCache::put_deferred), else the embedded camera JPEG of a raw that still
has its import look (import::has_import_look), else a thumbnail-level render.
- Grid / filmstrip: an unedited raw without a thumbnail shows its cached thumbnail, else its
embedded preview (Slot::ThumbQuick) at once; the real rendered thumbnail follows at
background priority (same cache keys, so it still ends up matching the develop render;
edited photos render directly).
- Loupe (detail, compare, filmstrip selection): Slot::Preview holds the stand-in until
Slot::Main has the photo; the full render then replaces it in place (no blank frame).
- files::embedded_preview_srgb + PreviewLoader hook (desktop/CLI/MCP and web).
- ui.inspect → loupe {photo, source: render|cached|embedded|small|thumb|none, pending}.
Real app (24 MP raws, loaded machine): first image 15–90 ms (thumbnail, then embedded
preview), full render 0.2–0.5 s (was 1.0–2.4 s with nothing shown before it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lightcraft-raw: RawImage::develop_binned(k) — k×k superpixel binning of the CFA
(Bayer k even, X-Trans k multiple of 3), black/white normalised row by row, default
crop only, clipping preserved for highlight reconstruction. 2560 px previews of a
24 MP raw bin 2×, thumbnails 4–8×; exports / 1:1 still demosaic (AHD) at full size.
- highlight::reconstruct and its pyramid fill are rayon-parallel (bit-exact).
- X-Trans directional demosaic: precomputed per-phase neighbour offsets and no
reflection in the interior, no per-pixel allocation (bit-exact, tested against the
reference formulation): 2.9 s → 0.75 s CPU at 24 MP.
- files::load_bytes runs on a rayon worker so its short parallel loops don't each
wait for the pool to wake (dominant cost on a loaded machine); LIGHTCRAFT_PROFILE
prints per-stage source timings; files::bin_factor picks the block size.
- render_bench ONLY=source: per-file decode / normalise / demosaic / binning / load.
- Fix: u64 overflow on hostile IFD sizes in DNG/ARW/NEF raw-IFD selection (proptest).
load_bytes(2560) CPU time, ARW a7 III 24 MP: 2347 → 240 ms; X-T20: 3120 → 1220 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- pipeline::optics::Warp: one inverse map (lens-corrected → source) inside the single geometry resample:
manual radial distortion, embedded DNG WarpRectilinear (per plane), per-channel lateral CA scale,
radial vignetting gain (manual + embedded FixVignetteRadial); all in normalized radii
- automatic lateral CA estimate: radial edge profiles matched sub-pixel per plane, robust LSQ fit of
displacement = α·r (cached per source)
- defringe: purple/green hue ranges (OkLab hue) desaturated next to high-contrast edges, width relative
to the image long edge
- "Enable Lens Corrections" applies only lens data embedded in DNGs (opcodes are no longer baked at decode;
stored on the photo record, on by default at import when present). No LCP data anywhere.
- new controls: optics.caRed/caBlue, defringe hue ranges; Optics panel: Manual sub-section, Defringe flyout
- tests: barrel grid straightened, CA removal (auto + manual), embedded corrections toggle/scale, preview vs
export resolution independence, CA estimator accuracy, orientation of embedded lens data
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- embedded_preview: CR3 PRVW/THMB boxes (plus RAF, RW2, ORF, PEF from the
format commits); stops at the first valid box of each kind
- engine: raw variants that report Unsupported are probed and shown from
their embedded JPEG (orientation from the JPEG or the raw), not an error
- crates/raw/tests/corpus.rs: every corpus/raw file is recognised, has a
JPEG preview (DNG optional) and decodes or is a known-unsupported
variant; prints decode timings
- xtask corpus --download: 37 CC0 raw.pixls.us samples (matched by SHA-256)
- ROADMAP/crate docs: format coverage, NEF Huffman / RW2 format 4 /
compressed ORF+RAF gaps, neutral colour-matrix fallback
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- engine::import: expand files/folders recursively (sorted; hidden entries
and the library folder skipped), skip known paths, probe + hash in
parallel (std::thread::scope), skip duplicates by content hash (against
the library and within the batch), add in place or copy into
<library>/Originals/YYYY/YYYY-MM-DD/ with unique names; one undoable op.
- library.import {paths, mode: add|copy, album?} returns
{imported, duplicates[{path, existing, reason}], failed, scanned}.
- probe_bytes records the content hash (SipHash-1-3 128) on every photo.
- system clock for import/edit times on native (Session::with_system_clock).
- app: command-line folders are expanded by the engine (re-importing the
same folder is a no-op); example gen_photos writes procedural test JPEGs
(never committed).
- Measured (200 procedural 2000 px JPEGs, M-series): import 2.25 s; reopen:
library load 1.7 ms, control ready 0.12 s, first thumbnail 0.17 s,
visible grid drawn 0.48 s (thumbnails from the disk cache).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
