Commit Graph
372 Commits
Author SHA1 Message Date
Brandon Thomas 89a51be77f Merge PR #1029
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/target-app-parity.md
2026-10-14 21:59:06 -04:00
Brandon Thomas 58dc540cc0 Merge PR #1010
# Conflicts:
#	docs/gaps.md
2026-10-14 21:57:58 -04:00
Brandon Thomas cbf3664e4f Merge PR #824
# Conflicts:
#	crates/engine/src/files.rs
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
2026-10-14 21:56:53 -04:00
midasmatterhorn 3bb13afdea raw: check maker-note crop bounds with checked_add
A PEF or ORF maker-note crop with an origin of `u64::MAX` and a width of
1 made `x + width` wrap to 0, so the guard read it as inside the image and
`RawImage::check`'s identical comparison also wrapped: `validate()` passed
and `normalized()` then indexed past the sample buffer. Debug builds
panicked at the guard itself ("attempt to add with overflow").

The active-area check and both vendor crop guards (PEF AOC, ORF
ImageProcessing) now compare with `checked_add`, so a wrapping rectangle
is a `Corrupt` error at validate time. Exact fits still pass.

Regression test `an_active_area_that_wraps_past_the_image_is_corrupt`
builds such an image and asserts `validate()` and `normalized()` return
`Err` (fails before the fix with the overflow panic above).
2026-10-14 23:12:32 +03:00
Michael Anderson 6d080de447 Never crash: Check raw data size before allocating in PEF, TIFF tiles and ARW6 (#1512)
A compressed PEF too short for its size, an uncompressed tile claiming far more samples than it holds, and an ARW6 with a failed tile now get refused before the big buffer is allocated.
2026-10-14 13:56:34 -05:00
Michael Anderson ef3642c39f LR-IMP-CAMERA-COVERAGE: Nikon D600, D610 and D5000 NEFs drop the columns past the image (#1627) 2026-10-14 13:44:58 -05:00
dev-null-undefinedandClaude Opus 5.5 1e55d36e36 Perf: DNG Deflate tiles at level 1, 4.8–7× faster writes for 3–12 % more bytes
Deflate-compressed DNG tiles are written at zlib level 1 instead of 6. Measured on
Sony A7R VI files (67 MP), 4 threads, best of 3:
- Photo Merge DNG (half floats, 3 channels): 3.88 → 0.55 s, 265.7 → 274.8 MB (+3.4 %)
- raw export with Deflate (16-bit CFA): 0.89 → 0.18 s, 96.7 → 103.0 MB (+6.5 %)
- rendered DNG: 4.60 → 0.69 s, 239.0 → 267.1 MB (+11.8 %)
Lossless either way; both written files decode back bit-exactly. Lossless-JPEG
DNGs (the default for convert/import) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:58:37 +02:00
dev-null-undefinedandClaude Opus 5.5 6fb0d22326 LR-MERGE-HDR: decode full-size merge frames one at a time; AHD writes its tiles straight into the picture
A 7-frame bracket of 67 MP raws ran out of memory: the merge decoded the frames four at a time, and the AHD demosaic
collected every tile before copying them into a second full-size picture. Full-size frames now decode one after the
other (the demosaic is parallel inside; previews stay parallel), and AHD fills the picture by rows of tiles. Peak memory
of 67 MP HDR merges: 3 frames 4.8 → 3.8 GB, 4 frames 6.3 → 4.7 GB, about the same time; AHD output is pixel-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:56:59 +02:00
Aleksandr Batyrev aa62ba05f9 fix(rawinfo): return failure for unreadable or invalid inputs 2026-10-14 14:16:35 +05:00
Claude 3ae306ecb3 LR-IMP-FORMATS: decode EOS R6 Mark II C-RAW with adaptive QP up to 175
EOS R6 Mark II C-RAW QP maps reach 172 and 175; crx.rs accepted 131-168,
so those files opened as their preview ("Canon CRX adaptive QP value 172").
Adaptive QP now goes to 175 (quantization value 51, step 228), the highest
seen in any file; uniform subband values still stop at 44.

Checked black-box against the camera JPEGs on private files: 144 R6 II /
R7 C-RAW files decode to identical hashes where they decoded before, the 3
failing ones now decode (green follows the JPEG at r 0.98), fine detail per
QP shows no jump at 172/175, 100 % crops are clean, and 1,492 files sampled
from a 35,796-file library decode with max QP 175. No third-party decoder
source consulted. Bumps RENDER_CACHE_VERSION to 63; cr3.md and What's New.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSGFEf87ba1zKJZb9uzddK
2026-10-14 08:41:52 +00:00
Andrew FrahnandClaude Opus 5.5 e5efaea6a8 Merge upstream/main into pr/file-writes
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 19:23:00 +11:00
Andrew FrahnandClaude Opus 5.5 61117e53e1 Merge upstream/main into pr/more-verified-bodies-2
Docs conflicts re-applied on upstream's text; coverage numbers regenerated
(unchanged: 334 verified, 844 unverified, 232 preview); duplicate rows in
the raw-parity summary table removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 19:21:03 +11:00
Andrew FrahnandClaude Opus 5.5 065818f524 Merge upstream/main into pr/more-verified-bodies
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 19:19:31 +11:00
Andrew FrahnandClaude Opus 5.5 bd2ad63b26 Merge upstream/main into pr/more-verified-bodies-3
Resolve doc conflicts in gaps, parity-checklist and raw-parity; coverage
numbers regenerated with docs/raw/coverage.py, parity summary refreshed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 19:02:15 +11:00
Andrew FrahnandClaude Opus 5.5 befd25f2a3 Merge pr/raw-erf into pr/raw-mef
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 19:00:57 +11:00
Andrew FrahnandClaude Opus 5.5 50cb797872 Merge upstream/main into pr/raw-erf
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 18:48:38 +11:00
Andrew FrahnandClaude Opus 5.5 5f1464170e Merge upstream/main into pr/more-verified-bodies-4
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 18:35:33 +11:00
Andrew FrahnandClaude Opus 5.5 1181b89048 Merge upstream/main into pr/cr3-variants
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 18:12:20 +11:00
Alex ChambersandClaude Opus 5.5 8feda7a997 LR-IMP-FORMATS: the TIFF reader's next-IFD pointer no longer overflows on huge BigTIFF entry counts
Pre-existing on main. The reader computed the position of a directory's next-IFD pointer before it
checked that every declared entry had been read: a saturating multiplication of the declared count,
then an unchecked addition. A 44-byte BigTIFF declaring u64::MAX entries therefore panicked with
overflow checks on. The position is now computed only for a directory read whole, with checked
arithmetic.

32-bit note: the same guard compared the counts after narrowing the declared one to usize, so on
wasm32 a BigTIFF declaring 4,294,967,297 entries but holding one passed for a whole directory and its
next pointer was followed. The counts now compare as 64-bit numbers (reads_every_entry). On wasm32 the
old comparison followed all six malformed test chains and the new one follows none; 64-bit results
are unchanged for every directory whose count is read whole.

Regression tests (crates/tiff/src/reader.rs):
a_bigtiff_directory_declaring_u64_max_entries_parses_without_overflow, which panicked before;
a_bigtiff_count_past_u32_max_never_follows_the_next_pointer (the whole chain, on every target); and
only_a_directory_read_whole_has_a_next_pointer (the comparison itself, on every target).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:40:42 +02:00
Alex ChambersandCodex c03eb8222f LR-IMP-CAMERA-COVERAGE: verify 30 public camera models
Recheck all 54 files in the 41-model shortlist on main 25ef65c8.
Pin 40 accepted files by SHA-256 and full decoded sensor FNV-1a,
and record their camera-image correlation. Exclude eleven candidate
models after checking identity, camera module, provenance and references.

Update generated coverage and tracking documents to 265 verified models.
Keep Nikon coding limits and the own-matrix DNG rendering issue explicit.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-14 08:21:44 +02:00
Alex ChambersandClaude Opus 5.5 ac677a2624 LR-EDIT-OPTICS-PROFILE: decode Sony shading, CA and distortion tables with settings and provenance; apply nothing new
Sony ARWs record shading, lateral chromatic aberration and distortion as a camera setting plus a radial
table in the raw image IFD (0x7031-0x7037), with copies in the encrypted SR2SubIFD (0x797c-0x7982).
They are now decoded for every body that records them and carried through RawInfo and RawImage as
`lens_metadata.sony` (types in crates/raw/src/lens_metadata.rs, decoder in
crates/raw/src/vendor/arw_lens.rs):

- Raw-IFD fields are the ones the TIFF reader parsed, which is what the image itself uses. A scan of
  the directory adds diagnostics only: repeated tags, and why an entry the reader skipped was dropped
  (values outside the file, an unknown type, past the reader's limits). SR2SubIFD fields are read
  from their first entry. Only the exact layouts are accepted: SHORT[1] or non-negative SSHORT[1]
  settings, SSHORT[17] headed by 16 for shading and distortion, SSHORT[33] headed by 32 for CA.
  Nothing is allocated from a count in the file.
- The raw IFD's copy wins. The SR2SubIFD copy fills in only for a tag the raw IFD lacks, never for a
  malformed one, and the two copies' agreement is recorded.
- Settings keep their number and their documented meaning, Unknown(value) otherwise (never Off).
- Framing evidence (stored frame, Sony and default crops, reference rectangle, coding, 3:2) and a
  per-component status. Only Applied components become opcodes.

Applied corrections are unchanged: the ILCE-7RM4A distortion conversion receives exactly the table it
received before, and no other body or component gains a correction. The ILCE-7CR distortion misses its
camera JPEG at 24 mm, and shading and CA semantics are not validated on any body; see
docs/sony-lens-corrections.md.

Locating the SR2SubIFD, now done for every ARW that has one, selects the SR2Private offset, length and
key with the TIFF reader's own rules, which lightcraft-tiff now exposes and builds its reader on
(directory_entries, entry_span, ParseOptions::budget_for, decode_value). Only each field's first value
is decoded, so the directory's other entries cost no memory sized by the file: a 16 MiB file whose
SR2Private holds eight fields aliasing it peaks at 37 kB. DirectoryEntries keeps its layout private,
and its positions() iterator uses checked arithmetic.

Tests: synthetic decoder tests; a differential test of the SR2Private selection against the TIFF
reader's in both byte orders (0 to 8,193 filler entries, unreadable and unusual first occurrences of
each field, a used-up value budget); a dhat peak-heap test; public-corpus checks against the values
ExifTool prints, with pinned and anchored SHA-256s (LIGHTCRAFT_CORPUS_REQUIRED=1 makes absent files
fail); and an opt-in local inventory check (crates/raw/tests/sony_lens_inventory.rs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 08:19:21 +02:00
Brandon Thomas 5fe332c7b4 Merge PR #1570 (updated head)
# Conflicts:
#	README.md
#	crates/raw/src/vendor/nef_lens.rs
#	docs/gaps.md
#	docs/nikon-lens-corrections.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/whats-new.md
2026-10-14 01:54:50 -04:00
Brandon Thomas 4fe0466942 Merge PR #1666
# Conflicts:
#	docs/whats-new.md
2026-10-14 01:39:02 -04:00
Brandon Thomas 35c4290d14 Merge PR #1477 2026-10-14 01:38:59 -04:00
Brandon Thomas 0a68a21e03 Merge PR #1459
# Conflicts:
#	crates/raw/tests/dng.rs
2026-10-14 01:38:59 -04:00
Brandon Thomas b54a2e9793 Merge PR #1541 2026-10-14 01:38:52 -04:00
Brandon Thomas 992e67188b Merge PR #1539
# Conflicts:
#	docs/whats-new.md
2026-10-14 01:38:51 -04:00
Brandon Thomas b18ce72b61 Merge PR #1535
# Conflicts:
#	crates/raw/src/vendor/pef.rs
2026-10-14 01:38:49 -04:00
Brandon Thomas b4d5b47617 Merge PR #1453 (rebased onto main) 2026-10-14 01:27:32 -04:00
skybluu bcfda31678 Merge current main into DNG identity fix 2026-10-13 21:34:33 -05:00
skybluu ce607d5a49 LR-LIB-CONVERT: avoid duplicated DNG camera make (#840) 2026-10-13 21:13:51 -05:00
Andrew FrahnandClaude Opus 5.5 363a1e2a67 Merge upstream/main into pr/file-writes
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:08:53 +11:00
Andrew FrahnandClaude Opus 5.5 9b9676ca85 Merge upstream/main into pr/more-verified-bodies
Coverage regenerated with docs/raw/coverage.py: 354 verified, 824 unverified,
230 preview, 38 unsupported; ~86% / ~58% use-weighted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:06:27 +11:00
Andrew FrahnandClaude Opus 5.5 506566d436 Merge pr/raw-erf (with upstream/main) into pr/raw-mef
Docs recounted with docs/raw/coverage.py: 241 verified, 35 unsupported. The MCP
headless extension list now comes from engine::import::EXTENSIONS upstream, which
already carries mef.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:56:09 +11:00
Andrew FrahnandClaude Opus 5.5 a3071fe0f3 Merge upstream/main into pr/raw-erf
Docs conflicts resolved on upstream's newer state (Leica Q3, Hasselblad Lunar);
ERF counts recomputed with docs/raw/coverage.py: 240 verified, 229 preview.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:55:22 +11:00
Andrew FrahnandClaude Opus 5.5 37deb70da9 Merge upstream/main into pr/more-verified-bodies-2
Conflicts in ROADMAP.md, docs/gaps.md, docs/parity-checklist.md and docs/raw-parity.md resolved by
taking upstream's text and re-applying this PR's rows; coverage re-measured with docs/raw/coverage.py
(334 verified, 844 unverified, 50% of photographers on a verified body).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:53:45 +11:00
Andrew FrahnandClaude Opus 5.5 853f3ddafe Merge upstream/main into pr/more-verified-bodies-3
Coverage regenerated with docs/raw/coverage.py (258 verified, 920 unverified);
doc conflicts resolved on upstream's text with this PR's numbers re-applied.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:53:29 +11:00
Andrew FrahnandClaude Opus 5.5 0cb32e384e Merge upstream/main into pr/more-verified-bodies-4
Coverage regenerated with docs/raw/coverage.py: 254 verified (18%), 932 unverified, 224 preview.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:43:46 +11:00
Andrew FrahnandClaude Opus 5.5 f2f6f1386f Merge upstream/main into pr/cr3-variants
Keeps upstream's ColorData 66 handling (white level from the data); recounts
coverage (259 verified, 919 unverified, 232 preview); drops the duplicate
raw-parity rows; notes high-QP R6 Mark II C-RAW and unverified R7 C-RAW.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:27:04 +11:00
Michael Anderson 1ee751ebdf LR-EDIT-OPTICS-PROFILE: Apply Nikon's in-file distortion correction for the Z 6II, Z 8 and Z f
Read the polynomial, zoom and optical center offset from NikonNEFInfo.
2026-10-13 11:10:46 -05:00
Brandon Thomas 13c32ef1bb Merge PR #1577 2026-10-13 11:49:21 -04:00
Brandon Thomas 05dc7374d8 Merge PR #1576 2026-10-13 11:49:20 -04:00
Brandon Thomas 642662d90f Merge PR #1570
# Conflicts:
#	docs/parity-checklist.md
#	docs/whats-new.md
2026-10-13 11:49:17 -04:00
Brandon Thomas c6b2686ba0 Merge PR #1410
# Conflicts:
#	docs/whats-new.md
2026-10-13 11:49:05 -04:00
Michael Anderson 853a92e2de LR-EDIT-OPTICS-PROFILE: Apply Nikon's in-file distortion correction for the Z 6II and Z f
Read the polynomial, zoom and optical center offset from NikonNEFInfo.
2026-10-13 10:47:57 -05:00
Michael Anderson 3624a50423 Never crash: Refuse raw data with two samples per pixel (#1511)
The demosaic read past each pixel. DNG decode and validation refuse it now.
2026-10-13 10:45:29 -05:00
Michael Anderson cef6f2d4b8 Never crash: Fix demosaic panic on a NaN black level
Read non-finite DNG black levels as 0 and map non-finite normalized samples to black.
2026-10-13 10:45:18 -05:00
Michael Anderson b62b51378e Never crash: Fix i32 overflow in the NEF and PEF Huffman predictors (#1515)
Use checked adds and return a corrupt data error instead of panicking.
2026-10-13 10:45:00 -05:00
MarekSpodBiedry 3946688f49 Reject RAW TIFFs with missing or undecodable image chunks (#1513) 2026-10-13 16:38:39 +02:00
MarekSpodBiedry df914dca41 Reject PEF streams that end before the pixels do instead of decoding zero padding (#1514) 2026-10-13 16:35:19 +02:00