33 Commits
Author SHA1 Message Date
aiongg f3bb7a9f30 CI: one test binary per crate, line tables only, one WASM check (#2710)
* CI: dev profile keeps line tables for workspace crates and no debug info for dependencies

* Control server tests: persistent_requests_and_slow_replies_outlive_the_idle_read_timeout no longer races the idle timeout

Each request (and the closing half-shutdown) reaches the socket before the
worker reads, so on a slow machine only the reply waits outlast the 100 ms
idle timeout, as the test intends. The idle test sends its input the same
way, and the no-frames test allows 5 s instead of 2 s to give up.

* CI: one integration-test binary per crate

Each crate's tests/*.rs become modules of tests/it/main.rs (51 test files
into 13 binaries, plus 6 that stay their own). Inner #![cfg] attributes
move to the mod lines. Files that change state for the whole process keep
a binary of their own: codecs jxl_tail_heap and raw sr2_heap (dhat global
allocator), gpu shutdown, shutdown_first and memory, ui-egui shutdown.
Docs, comments and the GPU probe name the new paths and targets
(--test it <module>::); the CI report names an it binary's skips after
the crate.

* CI: tests under nextest, doctests apart, one wasm check

xtask ci runs the tests with cargo nextest run --lib --bins --tests when
cargo-nextest is installed, else cargo test with the same targets and a
one-line notice; doctests run after them with cargo test --workspace --doc.
The heif and hardware GPU steps use the same runner. Examples stay out of
the test build (clippy --all-targets still checks them). Concurrent tests
under nextest follow the RUST_TEST_THREADS cap (NEXTEST_TEST_THREADS).
The wasm check is one cargo check --keep-going over the same crates, and
clippy runs with --keep-going.

* CI: tests run under cargo test by default, nextest with LIGHTCRAFT_NEXTEST=1 (per-process scene cache)
2026-10-23 13:34:30 -04:00
aiongg 3300e6dad8 Merge track r2-t01 into next
# Conflicts:
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw/coverage.tsv
#	docs/whats-new.md
2026-10-22 02:49:52 +00:00
Andrew FrahnandClaude Sonnet 5.5 5e1bac1264 Merge upstream/main into pr/longtail-raws
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 09:58:17 +11:00
Alex ChambersandCodex 484215f009 LR-IMP-FORMATS: discover Sony SR2 and SRF files in folders
Reuse the engine's import extensions in MCP and CLI discovery. Add
recursive, case-insensitive discovery tests and record that DSC-F828
SRF import remains preview-only.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-15 10:15:53 +02:00
Brandon Thomas f27ca795c7 Merge PR #1327
# Conflicts:
#	ROADMAP.md
#	crates/raw/tests/corpus.rs
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	docs/raw/evidence.tsv
#	docs/target-app-parity.md
2026-10-15 00:44:42 -04:00
Brandon Thomas d8b52d9405 Merge PR #1105
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
2026-10-14 21:59:32 -04:00
Brandon Thomas aba04aaf5c Merge PR #1087
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	docs/raw/evidence.tsv
#	xtask/src/main.rs
2026-10-14 21:59:31 -04:00
Brandon Thomas d3edca8000 Merge PR #1081
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/coverage.tsv
#	xtask/src/main.rs
2026-10-14 21:59:29 -04:00
Brandon Thomas 6ab5e2b0d4 Merge PR #1108
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/raw/evidence.tsv
#	xtask/src/main.rs
2026-10-14 21:59:28 -04:00
Brandon Thomas 381b57dd0b Merge PR #1096
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	xtask/src/main.rs
2026-10-14 21:59:10 -04:00
Brandon Thomas a19cbdb17b Merge PR #1045
# Conflicts:
#	ROADMAP.md
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/target-app-parity.md
2026-10-14 21:59:07 -04:00
Alex ChambersandCodex c03eb8222f LR-IMP-CAMERA-COVERAGE: verify 30 public camera models
Recheck all 54 files in the 41-model shortlist on main 25ef65c8.
Pin 40 accepted files by SHA-256 and full decoded sensor FNV-1a,
and record their camera-image correlation. Exclude eleven candidate
models after checking identity, camera module, provenance and references.

Update generated coverage and tracking documents to 265 verified models.
Keep Nikon coding limits and the own-matrix DNG rendering issue explicit.

Co-Authored-By: Codex <noreply@openai.com>
2026-10-14 08:21:44 +02:00
Andrew FrahnandClaude Opus 5.5 6696a92005 Merge upstream/main into pr/longtail-raws
Docs conflicts resolved on upstream's text; MRW edits reapplied with
coverage re-measured by docs/raw/coverage.py (243 verified, 946
unverified, 221 preview). Drops stale duplicate rows upstream's docs
carried in raw-parity.md and gaps.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:07:58 +11:00
Andrew FrahnandClaude Opus 5.5 9b9676ca85 Merge upstream/main into pr/more-verified-bodies
Coverage regenerated with docs/raw/coverage.py: 354 verified, 824 unverified,
230 preview, 38 unsupported; ~86% / ~58% use-weighted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:06:27 +11:00
Andrew FrahnandClaude Opus 5.5 506566d436 Merge pr/raw-erf (with upstream/main) into pr/raw-mef
Docs recounted with docs/raw/coverage.py: 241 verified, 35 unsupported. The MCP
headless extension list now comes from engine::import::EXTENSIONS upstream, which
already carries mef.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:56:09 +11:00
Andrew FrahnandClaude Opus 5.5 a3071fe0f3 Merge upstream/main into pr/raw-erf
Docs conflicts resolved on upstream's newer state (Leica Q3, Hasselblad Lunar);
ERF counts recomputed with docs/raw/coverage.py: 240 verified, 229 preview.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:55:22 +11:00
Andrew FrahnandClaude Opus 5.5 37deb70da9 Merge upstream/main into pr/more-verified-bodies-2
Conflicts in ROADMAP.md, docs/gaps.md, docs/parity-checklist.md and docs/raw-parity.md resolved by
taking upstream's text and re-applying this PR's rows; coverage re-measured with docs/raw/coverage.py
(334 verified, 844 unverified, 50% of photographers on a verified body).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:53:45 +11:00
Andrew FrahnandClaude Opus 5.5 853f3ddafe Merge upstream/main into pr/more-verified-bodies-3
Coverage regenerated with docs/raw/coverage.py (258 verified, 920 unverified);
doc conflicts resolved on upstream's text with this PR's numbers re-applied.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:53:29 +11:00
Andrew FrahnandClaude Opus 5.5 0cb32e384e Merge upstream/main into pr/more-verified-bodies-4
Coverage regenerated with docs/raw/coverage.py: 254 verified (18%), 932 unverified, 224 preview.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:43:46 +11:00
Andrew FrahnandClaude Opus 5.5 f2f6f1386f Merge upstream/main into pr/cr3-variants
Keeps upstream's ColorData 66 handling (white level from the data); recounts
coverage (259 verified, 919 unverified, 232 preview); drops the duplicate
raw-parity rows; notes high-QP R6 Mark II C-RAW and unverified R7 C-RAW.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 07:27:04 +11:00
bflatastic 4e78514948 Merge PR #1324
# Conflicts:
#	docs/gaps.md
#	docs/parity-checklist.md
#	docs/raw-parity.md
#	docs/whats-new.md
2026-10-13 11:05:53 +07:00
Alex ChambersandClaude Opus 5.5 ecd6618f6f LR-IMP-FORMATS: uncompressed Leaf MOS is a raw, the Hasselblad Lunar decodes as an ARW (#1275)
An uncompressed Leaf Aptus 22 MOS keeps its 16-bit mosaic in IFD0
(compression 1, BlackIsZero, one sample per pixel) with no Make tag and
no CFA photometric, so `probe` returned None and the file imported as a
greyscale image of the mosaic. Leaf's data block, tag 0x8606 (LeafData
in the ExifTool EXIF tag-name documentation), now marks such an IFD0 as
a raw: OtherTiff, preview only with the reason, as the lossless-JPEG MOS
already is. A rendered RGB image that kept the block stays an image.

The Hasselblad Lunar is a Sony NEX-7 that writes an ARW under Make
HASSELBLAD; only SONY files reached the ARW reader, so it opened as its
preview. A HASSELBLAD file with Sony's raw coding (an IFD with
compression 32767 or the tone-curve tag 0x7010, which no 3FR / FFF has)
now goes there and decodes like a NEX-7: black 512, the SR2SubIFD white
balance. The Tag2010 white-balance fallback, the one table arw.rs keys
by model, takes the body from the maker note's SonyModelID (0xb001)
when the file is not Sony-branded.

Docs: the Lunar is verified in docs/raw/evidence.tsv (236 verified),
raw-parity.md, parity-checklist.md, gaps.md and whats-new.md.
RENDER_CACHE_VERSION 61 -> 62: a Lunar's thumbnails were rendered from
its preview.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:02:19 +02:00
Andrew FrahnandClaude Opus 5.5 b7d77f16b8 LR-IMP-CAMERA-COVERAGE: verify 17 uncompressed Olympus and Samsung models on CC0 files
One CC0 raw.pixls.us file per uncompressed Olympus ORF and Samsung SRW
body not yet pinned (23 files, 17 Adobe models) is added to RAW_SAMPLES
and pinned in crates/raw/tests/olympus_samsung_corpus.rs: own sensor
hash plus a black-box check against the camera's embedded JPEG (the
E-10 pinned only, inspected by eye).

Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.

Docs: evidence.tsv + coverage re-measure (252 verified), gap 10,
checklist, raw-parity, ROADMAP. Headline unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 06:38:36 +11:00
Andrew FrahnandClaude Opus 5.5 9181aa0b6c LR-IMP-FORMATS: import and decode Mamiya MEF (uncompressed 12-bit; WB from the RGB preview)
Sources: TIFF 6.0 and own black-box analysis of the CC0 ZD sample.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 06:27:05 +11:00
Andrew FrahnandClaude Opus 5.5 f63522a234 LR-IMP-CAMERA-COVERAGE: verify 21 Sony and Leica models on CC0 files
One CC0 raw.pixls.us file per Sony and Leica body that no earlier bulk
check covered (22 files, 21 Adobe models) is added to RAW_SAMPLES and
pinned in crates/raw/tests/sony_leica_corpus.rs: own sensor hash plus a
black-box check against the camera's embedded JPEG where it has a colour
one (five files pinned only, inspected by eye).

Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.

Docs: evidence.tsv + coverage re-measure (256 verified), gap 10,
checklist, raw-parity, ROADMAP. Headline unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 06:05:11 +11:00
Andrew FrahnandClaude Opus 5.5 a6d048be8d LR-IMP-FORMATS: decode Epson ERF (12-bit 16-byte blocks; WB from the preview's neutrals)
Sources: TIFF 6.0 and own black-box analysis of the three CC0 samples;
white-balance estimate from the embedded preview: own design.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 05:43:30 +11:00
Andrew FrahnandClaude Opus 5.5 bbd2e8cbd4 LR-IMP-CAMERA-COVERAGE: verify 97 Canon CR2, Pentax and Ricoh models on CC0 files
One CC0 raw.pixls.us file per Canon CR2, Pentax and Ricoh body that
decodes (100 files, 97 Adobe models) is added to RAW_SAMPLES and pinned
in crates/raw/tests/canon_pentax_corpus.rs: own sensor hash plus a
black-box check that the decoded green follows the camera's embedded
JPEG (r 0.69-1.00; four files pinned only, inspected by eye).

Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.

Docs: evidence.tsv + coverage re-measure (332 verified, 50% on verified
bodies), gap 10, checklist, raw-parity, ROADMAP. Headline unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 05:27:26 +11:00
Andrew FrahnandClaude Opus 5.5 a44a4baa31 LR-IMP-CAMERA-COVERAGE: verify 117 Nikon and Fujifilm bodies on CC0 files
One CC0 raw.pixls.us file per Nikon and Fujifilm body that decodes is
added to RAW_SAMPLES and pinned in crates/raw/tests/body_corpus.rs: own
sensor hash plus a black-box check that the decoded green follows the
camera's embedded JPEG (r 0.77-1.00).

The sweep found COOLPIX 5700 / 8400 / 8800 decoding wrongly (two stacked
half-height fields, CMYG read as RGB): recorded as unsupported. #702 is
fixed, so the A100 evidence row is preview, not unsupported.

Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.

Docs: evidence.tsv + coverage re-measure (352 verified, 86% use-weighted,
58% on verified bodies), gap 10, checklist, raw-parity, target-app-parity,
ROADMAP. Headline numbers unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 04:28:43 +11:00
Andrew FrahnandClaude Opus 5.5 d6c29aaf25 LR-IMP-FORMATS: decode Minolta MRW (packed 12-bit and 16-bit words, RGGB / GBRG)
The \0MRM block container: PRD sensor size, storage and Bayer cell; WBG
as-shot gains in cell order; TTW TIFF for Exif and orientation. Black 0,
white from the saturation plateau. 4 CC0 samples pinned in RAW_SAMPLES;
DiMAGE 7, 7Hi, A1, A2, A200 and Dynax / Maxxum 7D verified. Also fixes
clippy::chunks_exact_to_as_chunks in pipeline/spots.rs.

Sources: public prose description of the MRW layout (Dalibor Jelinek's
MRW notes) and own black-box analysis of the CC0 samples.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 04:18:59 +11:00
Andrew FrahnandClaude Opus 5.5 1477f892b5 LR-IMP-FORMATS: verify 22 more CR3 bodies; EOS R6 Mark III white balance
All 110 CC0 CR3s on raw.pixls.us decode from sensor data; none uses a
refused CRX variant. One CC0 file per further body (1D X III, 90D, 250D,
850D, M200, M50 II, M6 II, R, R3, R5, R6, R6 II, R6 III, R7, R10, RP, R50,
R50 V, G5 X II, G7 X III, SX70, V1) is added to RAW_SAMPLES and pinned in
cr3_corpus.rs, checked against the camera's embedded JPEG.

The R6 Mark III writes ColorData version 66, laid out as 64/65; it lost
its as-shot WB and white level. Render cache 61 -> 62.

Also: as_chunks in spots.rs for the chunks_exact_to_as_chunks lint.

Docs: evidence.tsv + coverage re-measure (257 verified, 92% use-weighted),
gaps 2/10, checklist rows, raw-parity, target-app-parity, ROADMAP, README.

Sources: own black-box analysis of CC0 files; ExifTool Canon tag tables.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 03:45:14 +11:00
Alfons HoogervorstandClaude Sonnet 5.5 8c19c9f60a LR-IMP-CAMERA-COVERAGE: Leica Q3 verified by a recorded check of 38 private DNGs (236 models verified)
All 38 files decode from sensor data and develop at full size (9520x6336 and
6336x9520) without error; mean CIE76 dE to the camera JPEG is 6.3-13.9 at 400 px.
The files are not committed and the evidence entry carries no path. Leica is now
14 verified / 47 unverified, the Leica bucket on a verified body 5% -> 21%; the
weighted shares (85% decode, 42% on a verified body) do not move.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-11 16:54:06 +02:00
ingolfteetzandClaude Opus 5.5 4700c8a74e LR-IMP-FORMATS: decode Canon EOS R5 Mark II C-RAW (CRX QP-map first literal and escape)
The CRX 0x200 QP map opens with its first symbol as a 32-bit literal; every
later symbol is Rice coded with a limited-length escape: 23 zeros, the
terminating 1 and the symbol in 8 bits (the ITU-T T.87 layout for a 32-bit
limit and 8-bit symbols, without its minus-one offset). The old rule (an
escape after 16 zeros with a 16-bit literal) matched only the first symbol,
which is all the R8 and R7 maps need. R5 Mark II maps (QP 150-162) jump by up
to 12 where the Rice parameter is 0, giving 18- to 22-zero prefixes that
misparsed ("Canon CRX adaptive QP value -21087"): 174 of 306 sampled files
from a private R5 Mark II library opened as previews. Every QP map of its
33,665 C-RAW files now decodes to the exact end of its entropy stream; the six
oracle fixtures are unchanged. Established by black-box analysis of the
bitstream; no third-party decoder source consulted.

Adds the four CC0 R5 Mark II samples (raw.pixls.us 7881-7884) to the corpus
with pinned sensor hashes (the crop-mode framing is left to #754), bumps
RENDER_CACHE_VERSION to 43, and updates cr3.md, raw-parity (235 verified
models), coverage, gaps, the checklist rows, ROADMAP, target-app-parity and
What's New.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 09:20:49 +02:00
Brandon ThomasandClaude Opus 5.5 964d9338ba Progress docs: target-app parity, RAW coverage vs Adobe, gaps, alpha gate, readiness by audience (#788)
Progress docs per craftrules standards/progress-docs.md: target-app parity, gaps, roadmap with alpha gate, readiness by audience (full / mainstream / essentials) with Opus 5.5 hours, model-by-model RAW coverage against Adobe's camera list (docs/raw/), and format, hardware, UI and localization parity docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 14:25:15 -04:00