One CC0 raw.pixls.us file per Sony and Leica body that no earlier bulk
check covered (22 files, 21 Adobe models) is added to RAW_SAMPLES and
pinned in crates/raw/tests/sony_leica_corpus.rs: own sensor hash plus a
black-box check against the camera's embedded JPEG where it has a colour
one (five files pinned only, inspected by eye).
Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.
Docs: evidence.tsv + coverage re-measure (256 verified), gap 10,
checklist, raw-parity, ROADMAP. Headline unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sources: TIFF 6.0 and own black-box analysis of the three CC0 samples;
white-balance estimate from the embedded preview: own design.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
One CC0 raw.pixls.us file per Canon CR2, Pentax and Ricoh body that
decodes (100 files, 97 Adobe models) is added to RAW_SAMPLES and pinned
in crates/raw/tests/canon_pentax_corpus.rs: own sensor hash plus a
black-box check that the decoded green follows the camera's embedded
JPEG (r 0.69-1.00; four files pinned only, inspected by eye).
Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.
Docs: evidence.tsv + coverage re-measure (332 verified, 50% on verified
bodies), gap 10, checklist, raw-parity, ROADMAP. Headline unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Version 66 keeps version 64's as-shot white balance and black level
offsets; its word 661 is below where the sensor clips, so the white
level stays measured from the data. Checked on 33 R6 Mark III files
from 12 shoots: all now open with their as-shot white balance, black
levels match the masked columns.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017JKdR4oZ3jRSPPwMLLTa2U
Nikon bodies from before ~2016 record no CropArea, so the decoder kept
their whole sensor data, less trailing dark columns. Where that data
holds masked lines above black or optical black, they were developed as
a coloured or dark bar on that edge: the D7000 and D5100's 36 columns at
~600 (#976), the D800's 14 (#914, the top in portrait), and others.
MEASURED_BODIES (crates/raw/src/vendor/nef.rs) now gives 36 model /
data size / depth groups the frame of their camera JPEG inside the data:
D7000, D5100, D800, D800E, Df, the D4's 12-bit mode, D3, D3S, D700,
D300, D300S, D600, D610, D810, D4S, D5600, Coolpix A, D3000, D40X, D60,
D80 and D2Xs. An entry matches model, data size and depth exactly; the
measured white level becomes optional (only the D4's 14-bit mode has
one). Bodies whose data holds only picture beyond the camera frame get
no entry and render as before.
`cargo xtask raw-frame FILE...` (crates/engine/examples/raw_frame.rs)
measures a body: it aligns each raw's embedded camera JPEG with the
sensor data (half-size luminance, gradient correlation, sub-pixel peak;
weak, ambiguous or between-pixel peaks are flagged), counts the edge
lines that hold no picture and reports what the decoder's crop keeps
beyond the camera frame. It prints entries only with evidence: at least
3 clean files agreeing exactly, of the model (12- and 14-bit pooled) or,
with at least one of its own, with sibling models of the same readout
(data and JPEG size) that agree on the same frame. On the D7500, which
records its CropArea, it finds that area exactly; it declines the Z f,
whose JPEG is distortion-corrected.
It works for any make whose raw embeds a full-size camera JPEG: on
Sony, Canon CR2 / CR3, Pentax and Samsung files it finds the decoder's
own crop (relative to the active area).
docs/nikon-bodies.md surveys every Nikon raw on raw.pixls.us (98
models, 291 files; 138 CC0 samples measured): 22 models record
CropArea; of the 88 groups without it, 36 have entries, 13 are fine, 25
could not be aligned and 7 do not decode. The D3200, D3100, D200, D50,
D750 and Nikon 1 V1 / V2 are pending, each listed with the files it
needs; the doc also describes how to add a body. raw-parity.md no
longer claims every NEF coding but HE decodes.
measured_bodies_are_consistent checks every entry lies inside its data
and is listed once.
The evidence is committed in docs/nikon-bodies/: headers.json (every
file's raw size, depth and maker-note crop tags), samples.json (the 138
measured CC0 files with URL and sha256) and frames.jsonl (the tool's
per-file results). `cargo xtask corpus --download nikon` fetches the
samples (sha256 checked) so `cargo xtask raw-frame` reproduces every
count.
Fixes#976Fixes#914
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New export format `rendereddng` (export dialog: Format DNG, "DNG of: Rendered
photo"; `app.export`; MCP export tool): the rendered photo, with the edit,
crop, size, output sharpening and watermark baked in, written by the new
`lightcraft_raw::write_rendered_dng` as a LinearRaw DNG of 16-bit float Deflate
tiles in the export colour space. ColorMatrix1 (XYZ -> space), ForwardMatrix1
(space -> XYZ D50), a neutral AsShotNeutral and an identity ProfileToneCurve
make raw converters show it as rendered. HDR edits keep highlights above white.
Bad input (wrong sample count, zero or huge sizes, NaN/inf, out of half-float
range) gives an error or is clamped, never a panic.
Also fixes the Rust 1.99 clippy lint chunks_exact_to_as_chunks in
pipeline/spots.rs that fails `cargo xtask ci` on main.
Sources: Adobe DNG Specification 1.4+ (LinearRaw, ColorMatrix/ForwardMatrix,
ProfileToneCurve); own design.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
One CC0 raw.pixls.us file per Nikon and Fujifilm body that decodes is
added to RAW_SAMPLES and pinned in crates/raw/tests/body_corpus.rs: own
sensor hash plus a black-box check that the decoded green follows the
camera's embedded JPEG (r 0.77-1.00).
The sweep found COOLPIX 5700 / 8400 / 8800 decoding wrongly (two stacked
half-height fields, CMYG read as RGB): recorded as unsupported. #702 is
fixed, so the A100 evidence row is preview, not unsupported.
Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.
Docs: evidence.tsv + coverage re-measure (352 verified, 86% use-weighted,
58% on verified bodies), gap 10, checklist, raw-parity, target-app-parity,
ROADMAP. Headline numbers unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`preview_plan`'s assertions sat inside the `header_plan == Ready` branch, so a NEF, an ORF or a PEF
— whose header probe reads samples and whose header plan is therefore `Whole` — never had its
preview plan looked at, and neither did any file the header plan declines. The display path wants
exactly that answer, so both plans are now asked for every corpus file, and the preview the plan
hands on must be the whole file's with the skipped bytes `0` and again `0xFF`; `Whole` stays a
legitimate answer (that format reads whole for a preview too) and is counted. The test prints what
each format reads for both plans.
Co-Authored-By: DeepSeek V3.2 <noreply@deepseek.com>
All 110 CC0 CR3s on raw.pixls.us decode from sensor data; none uses a
refused CRX variant. One CC0 file per further body (1D X III, 90D, 250D,
850D, M200, M50 II, M6 II, R, R3, R5, R6, R6 II, R6 III, R7, R10, RP, R50,
R50 V, G5 X II, G7 X III, SX70, V1) is added to RAW_SAMPLES and pinned in
cr3_corpus.rs, checked against the camera's embedded JPEG.
The R6 Mark III writes ColorData version 66, laid out as 64/65; it lost
its as-shot WB and white level. Render cache 61 -> 62.
Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.
Docs: evidence.tsv + coverage re-measure (257 verified, 92% use-weighted),
gaps 2/10, checklist rows, raw-parity, target-app-parity, ROADMAP, README.
Sources: own black-box analysis of CC0 files; ExifTool Canon tag tables.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DNG export gains a "Lossy" compression: the raw data is demosaiced to linear
camera RGB (OpcodeList3 and the default crop applied), scaled to the resize long
edge when one is set, and stored as 8-bit baseline JPEG tiles at the export
quality. The codes follow a 1/2.2 power curve that a LinearizationTable maps
back to linear 16-bit values; colour matrices, white balance and the profile
look are kept. Available in the export dialog, `app.export`
(`dngCompression: "lossy"`) and the MCP export tool.
Also fixes a new clippy lint (chunks_exact_to_as_chunks) in pipeline/spots.rs
that fails `cargo xtask ci` on Rust 1.99.
Sources: Adobe DNG Specification 1.4+ (Compression 34892, LinearizationTable);
own design for the code curve.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On merge: the quick preview path orients like the whole-file path (preview_orientation with the
raw's header prefix, so a portrait raw whose embedded JPEG carries no orientation is not sideways),
and its doc links point at preview_window.
`lightcraft_raw::sparse::header_plan` says which byte ranges of a raw file the header
probe (`probe_info`, `lightcraft_meta::extract`) reads, given a buffer of the file's length
in which only some ranges are loaded: everything but the bulk sample data (large TIFF
strips/tiles minus the first chunk's coding headers, a CR3's image tracks minus the raw
track's CRX headers and QP maps, a RAF's CFA strip, an RW2's raw data). `preview_plan`
also reads what `embedded_preview` needs, for raws shown from their preview. NEF, ORF and
PEF (their header probe looks at samples) and anything else are read whole.
The corpus test checks, for every raw.pixls.us sample the plan describes, that probing the
partly loaded buffer gives exactly what the whole file gives, with the skipped bytes zero and
again 0xFF: CR3 2.7 %, DNG 2.4 %, CR2 3 %, RAF 6 %, RW2 7 %, ARW 8 % of their bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HsvTable::apply took the value through linear -> sRGB -> linear even when the table's value
scale is 1 (every hue/saturation-only table, as fitted camera looks are), and wrapped hues with
rem_euclid (an fmodf call) three times per pixel. The value is now kept as it is for a unit
scale and the sRGB-encoded value is reused otherwise; hues are wrapped by a helper that equals
rem_euclid bit for bit (tested) without the call for values within one period.
2048 px render of a 24 MP ARW, single thread: 16.4 -> 15.1 G instructions, 4.94 -> 4.76 G cycles.
Sources: own design.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The CRX 0x200 QP map opens with its first symbol as a 32-bit literal; every
later symbol is Rice coded with a limited-length escape: 23 zeros, the
terminating 1 and the symbol in 8 bits (the ITU-T T.87 layout for a 32-bit
limit and 8-bit symbols, without its minus-one offset). The old rule (an
escape after 16 zeros with a 16-bit literal) matched only the first symbol,
which is all the R8 and R7 maps need. R5 Mark II maps (QP 150-162) jump by up
to 12 where the Rice parameter is 0, giving 18- to 22-zero prefixes that
misparsed ("Canon CRX adaptive QP value -21087"): 174 of 306 sampled files
from a private R5 Mark II library opened as previews. Every QP map of its
33,665 C-RAW files now decodes to the exact end of its entropy stream; the six
oracle fixtures are unchanged. Established by black-box analysis of the
bitstream; no third-party decoder source consulted.
Adds the four CC0 R5 Mark II samples (raw.pixls.us 7881-7884) to the corpus
with pinned sensor hashes (the crop-mode framing is left to #754), bumps
RENDER_CACHE_VERSION to 43, and updates cr3.md, raw-parity (235 verified
models), coverage, gaps, the checklist rows, ROADMAP, target-app-parity and
What's New.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Sony body shot in Adobe RGB writes ColorSpace Uncalibrated with InteropIndex R03 in the raw's EXIF and leaves the
embedded JPEG untagged, so the look was fitted against the JPEG decoded as sRGB. The EXIF fallback now applies to
all makes (the Nikon maker note still wins); DNGs are skipped, since their previews come from the converter.
Canon EOS R50 C-RAW files reach QP 168 (quant 44, step 102, measured about 100). The quantization step and the adaptive QP map range accept it; anything above stays rejected. Bumps RENDER_CACHE_VERSION since such files change from the embedded preview to a raw decode.
The 2668-entry table read off the black-box oracle's output is gone.
arw6_curve.rs now builds the 4096-entry table at first use from our own
measurement: piecewise linear through knots u_k = 128 k, c_k = u_k for
k <= 11 and 356.89 + 655.646 ln(k - 6) above, rounded half up, fitted to
the 7 same-exposure lossless (LJ92) / ARW6 pairs of the CC0 ILCE-7RM6 set.
A unit test pins the whole table (sum 27776336, weighted 92792980288),
its anchors, the identity through 1408 and strict monotonicity.
The corpus sums become regression values of the new output; the 12-bit
codes before the curve stay verified against the oracle (Phase 0).
docs/arw6-compression.md describes the curve, how it was measured, its
uncertainty, the structure and x2-scale evidence, why it is not in the
file, and the after-the-fact comparison with the oracle's curve; the
tracker and docs no longer call the output bit-exact against LibRaw.
Co-Authored-By: Claude <noreply@anthropic.com>
docs/superpowers/ held the working spec and plan, which pointed at the
local plan/ folder. The ARW6 module docs now cite the sections of
docs/arw6-compression.md, which gains the facts they relied on from the
spec: the NP field of word 0, the stream header word layout and how the
first half's residual lines are counted.
Co-Authored-By: Claude <noreply@anthropic.com>
`cargo xtask corpus --download` now fetches one full-frame Compressed and
one APS-C Compressed HQ file (about 94 MB) instead of all 12 (about
630 MB); the corpus test checks those two and the lossless-sibling
regression test is gone, since nothing would download those files.
docs/arw6-compression.md keeps the full table with source paths and
SHA-256, says which two xtask fetches and how to fetch the rest with
`curl -L`, and records the CC0 1.0 statement of RawDB's set record
(accessed 2026-10-10).
Co-Authored-By: Claude <noreply@anthropic.com>
The RawDB entries in xtask carry their SHA-256 as the third field, which
`cargo xtask corpus --download` checks; the corpus test keeps only the
sensor sum checks.
Co-Authored-By: Claude <noreply@anthropic.com>
