Resolve doc conflicts in gaps, parity-checklist and raw-parity; coverage
numbers regenerated with docs/raw/coverage.py, parity summary refreshed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Docs recounted with docs/raw/coverage.py: 241 verified, 35 unsupported. The MCP
headless extension list now comes from engine::import::EXTENSIONS upstream, which
already carries mef.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Conflicts in ROADMAP.md, docs/gaps.md, docs/parity-checklist.md and docs/raw-parity.md resolved by
taking upstream's text and re-applying this PR's rows; coverage re-measured with docs/raw/coverage.py
(334 verified, 844 unverified, 50% of photographers on a verified body).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Coverage regenerated with docs/raw/coverage.py (258 verified, 920 unverified);
doc conflicts resolved on upstream's text with this PR's numbers re-applied.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Keeps upstream's ColorData 66 handling (white level from the data); recounts
coverage (259 verified, 919 unverified, 232 preview); drops the duplicate
raw-parity rows; notes high-QP R6 Mark II C-RAW and unverified R7 C-RAW.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Verified on 54 ILCE-7RM6 photos, the CA planes made the render worse: our
demosaic already hides about 70 % of the lens CA (the render shows 0.29x
the raw-CFA CA) while the post-demosaic warp moves R and B by 0.84x the
table, so the corner R/G and B/G CA went from rms 0.032 / max 0.125 px
(off) to 0.052 / 0.219 px (on) at 1440 px; only the 16-35 mm improved.
warp_opcode takes a lateral_ca flag; lens_opcodes passes false, so a Sony
ARW gets the one-plane distortion warp. The CA conversion stays live and
tested (FF and APS-C examples, both lossless corpus files' tables). The
follow-up is to warp the CFA colour planes before demosaic. The doc,
tracker (docs/parity-checklist.md), docs/gaps.md, docs/raw-parity.md and
whats-new say distortion applied, CA parsed and off
by default, vignetting not applied; the one-plane default costs 24 -> 45 ms
in the sample stage at 1920 px (three planes: 112 ms).
Co-Authored-By: Claude <noreply@anthropic.com>
vendor::arw_lens_rm6::warp_opcode (was distortion_opcode) turns the CA table
(0x7035: red half, blue half, on the distortion knots) into the R and B
planes of the WarpRectilinear: a colour's source radius is
g (1 + c(g) / 3.36e6) at the green source radius g. The unit is provisional
(borderline measurement, about +-30 % scene bias). Each colour plane goes
through the same fit gate as G, and a table with a value beyond 2e-3 relative
(about 12 px at the full-frame corner; the 54 photos' tables stay below
5.1e-4, i.e. 1724) is refused; without a CA table, or when it is refused, the
warp keeps its single G plane, and CA alone is never applied. fit_radial_warp
now rounds its gate up to the next sample, capped at 1 (a NaN gates every
sample), so the band between the last sample inside the crop and the crop
corner is checked too (RW2 passes 1.0: unchanged).
Co-Authored-By: Claude <noreply@anthropic.com>
New vendor::arw_lens_rm6 reads the raw IFD's lens tables (0x7037 distortion,
0x7032 vignetting, 0x7035 lateral CA, their flags) and turns the distortion
table into a one-plane WarpRectilinear in OpcodeList3, with the conventions
measured on 54 ILCE-7RM6 photos: knots at i/15 of the full-frame crop half
diagonal, S(r) = r (1 + v(r)/15000), frame-filling zoom, centred on the
default crop. Only the ILCE-7RM6 is enabled; vignetting is read, not applied.
The radial least-squares fit is split out as vendor::fit_radial_warp (RW2
results unchanged); its error gate can be limited to the radii the crop
shows, so a padded active area (lossless files) doesn't refuse a fit over
pixels that are never rendered.
It sits next to main's vendor::arw_lens (#498), which holds the ILCE-7RM4A
conventions measured against Sony's exports (unit 16384, table indexed at the
output radius); the two are selected by camera model (disjoint allowlists).
arw::decode chains this module's opcodes after arw_lens::distortion, with the
same active area and the same crop (the DNG-style default crop when the file
has one, which on the ILCE-7RM6 equals Sony's crop tags), and like it only
for CFA data (not the downsized linear-RGB lossless sizes, where the knot
spacing in full-frame pixels would not hold).
The corpus test checks the conversion on the two lossless RawDB files (CC0).
The APS-C one is pinned in xtask RAW_SAMPLES with its sha256; the full-frame
one is not (at most two pins per PR, and #625 pins two Compressed ARW 6.0
files), so its case runs only where the file is present.
RENDER_CACHE_VERSION 62 -> 63.
Co-Authored-By: Claude <noreply@anthropic.com>
docs/cr3.md gains a variant-status table: every supported and rejected CRX
coding variant, the function that gates it and the exact Unsupported text.
crates/raw/tests/cr3_variants.rs enforces the rejections (19 tests). Each
rejected variant is rebuilt as a small synthetic CR3 (container, CMP1
descriptor, CRX stream) and the test asserts the exact string, and that
probe_info (the import path) and decode return the same error. Control
fixtures decode through the same container, including both edges of the
accepted uniform quantization range (4..=44) and adaptive QP range
(131..=168), so a rejection can only come from the documented gate. The
table itself is prose and is not checked against the tests.
The doc said only QP values 131-167 were accepted; the decoder also accepts
168 (quantization value 44, step 102), so that sentence now says so.
Tests and docs only; no decoder change.
crates/raw/tests/robust.rs only mutated DNGs. Add two small valid CR3
fixtures (a 4x4 lossless CRX and a 16x16 three-level C-RAW CRX) and a
property that flips bytes in them or truncates them, then runs the existing
exercise() helper (probe, embedded_preview, decode, develop and the colour
transform) plus probe_info, the header-only description used on import:
errors or clean decodes, never a panic. Unlike random TIFF-like bytes, a
structurally valid file gets past the container walk, so the mutations land
in the CRX marker walker and entropy readers. A fixture test asserts that
both files decode as built, so the mutations hit live decode paths.
Tests only; the existing DNG property keeps its 400 cases and the new one
uses proptest's default (256, PROPTEST_CASES overrides it).
crates/raw/tests/crx_properties.rs drives lightcraft_raw::decode with
synthetic CR3 containers built around CRX streams that the test encodes
itself: reference entropy coders for the MED, Rice, MELCODE and QP-map
layouts, and the exact algebraic inverse of one 5/3 lifting line. The
tile-halo coefficients are free parameters of that inverse, so synthesis
must reconstruct any encoded plane exactly. Tests only; no decoder change.
- Wavelet round trip: random planes decode exactly across odd plane sizes
and multi-tile halo configurations (eight geometries, arbitrary halo
values).
- Quantization: every uniform quant value 4..=44 and every adaptive QP value
131..=168 dequantizes within the 14-bit range. Overshoot is clamped, and
an in-range plane lands exactly on level * step, so the step sequence
itself is observable. Quant 3 and 45 and QP 130 and 169 are rejected by
decode and probe_info alike. Random QP maps, including odd map heights,
with random per-band base and gain stay in range.
- Lossless samples outside the bit depth are Corrupt errors instead of being
clamped; in-range planes decode exactly.
- Adversarial entropy payloads planted in well-formed lossless, C-RAW,
version 0x200 and QP-map streams either error or decode deterministically
and never panic. A QP Rice prefix longer than the 23-zero escape is
Corrupt.
- Random IAD1 geometry either decodes with the active area inside the
sensor and the crop inside the active area, or is rejected.
- Decoding gives identical results across repeated and four-way parallel
invocations.
- Odd QP-map heights decode: the last QP row has no partner and is used
alone, and QP values are averaged before the step lookup. Large QP jumps
take the escape code.
The case budget is proptest's default (256 per property); set
PROPTEST_CASES to change it.
Four CC0 EOS RP samples from raw.pixls.us (lossless RAW and C-RAW, full
frame and 1.6x crop mode) now have exact full-sensor regressions against
the DNGLab 0.8.0 black-box oracle (Windows binary, checked first against
the documented M50 RAW hash). The decoder already matched; no decoder
change was needed. Crop mode is a single 4032x2656 tile, full frame two
3192-wide tiles.
docs/cr3.md also records that the white level is Canon's specular white,
which at the -1/3-stop ISOs (e.g. 11062 at ISO 5000) is well below the
sensor's saturation point.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The DNG CFAPattern describes the top-left of the ActiveArea, but RawImage::cfa is sensor-anchored (the crop to the active area shifts it by the area's offset). The DNG reader now moves the tag's pattern out to the sensor origin, and the DNG writer states the pattern back at the ActiveArea. A DNG whose area starts on an odd row or column (the Canon EOS 60D DNG of #1086, top 51) no longer reads RGGB as GBRG.
Cause: a floating-point CFA DNG is read as stored (no check of its float
samples). AHD and PPG clamp each green estimate between its two neighbouring
greens with `est.clamp(a.min(b), a.max(b))`; when both greens are NaN the
bounds are NaN and `f32::clamp` panics (ppg.rs:45, ahd.rs green pass). Any
float DNG with two adjacent NaN greens crashed decode + develop. Infinities
alone did not panic but turned into NaN in the demosaic arithmetic.
Fix, at two layers:
- demosaic: a `clamp_between` helper replaces the three clamps. With finite
or infinite bounds it is the same `f32::clamp` call (bit-identical
results); with NaN bounds it returns the estimate unclamped, so the panic
is gone whatever puts NaN into the mosaic.
- DNG decode: non-finite float samples are replaced where the white level is
known. NaN and -inf read as black (0), +inf as the white level (a clipped
highlight). No sensor records NaN or infinity, so a valid file is never
touched; finite samples, negative or above white included, stay as stored.
Tests: `demosaic::tests::non_finite_samples_do_not_panic` (NaN block, +/-inf
and an all-NaN mosaic through Bilinear, PPG and AHD; plus `clamp_between`
equal to the old clamp bit for bit over finite, signed-zero and infinite
values) and `non_finite_float_samples_decode_and_develop` (a float CFA DNG,
uncompressed and Deflate, with NaN and +/-inf samples: decoded values
checked, develop with all three methods gives a finite image). Both panicked
before the fix at ppg.rs:45 / ahd.rs.
Co-Authored-By: Claude <noreply@anthropic.com>
Brandon Thomas