aleleba/lightcraft
1
0
Fork 0
mirror of https://github.com/storytold/lightcraft.git synced 2026-10-28 03:57:57 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge origin/main into menu-scroll-clips-shortcuts (docs restructure; no conflicts)

Browse Source
This commit is contained in:
Jason Baldwin
2026-10-10 21:18:32 +02:00
parent 94fed226df 964d9338ba
commit 2c2f28952f
110 changed files with 9403 additions and 786 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+12 -6
View File
@@ -5,12 +5,18 @@ LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructi
## Start every session here
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers).
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Behaviour/visual reference: `plan/lightroom/` (incl. `10-observed-ui.md` + screenshots).
3. **Know where we stand:** read [`ROADMAP.md`](ROADMAP.md) → *Where we stand* (honest assessment by dimension and
by kind of user) and *Where we're going* (ordered priorities). The checklist counts features that *exist*; the real
gaps are quality and coverage: camera colour calibration, CR3 / compressed raws, per-model verification, render
fidelity against Lightroom, AI models, HDR / video / Classic modules. A ✅ row is not proof of parity — if you find a
✅ feature that is wrong or incomplete, downgrade it to 🟡 with a note.
4. **Pick work** from [`docs/parity.md`](docs/parity.md) → *Top gaps* (ordered by user impact; the tracker has one row per feature, menu item and shortcut). When you land a feature, update its row(s) and the gap list in the same commit, and the *Where we stand* / *Where we're going* sections of `ROADMAP.md` when a listed gap closes; `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
3. **Know where we stand:** read [`ROADMAP.md`](ROADMAP.md) (stage, headline numbers, every dimension) and, for
method and evidence, [`docs/target-app-parity.md`](docs/target-app-parity.md). The checklist counts features that
*exist*; the real gaps are quality and coverage: camera colour calibration, CR3 / compressed raws, per-model
verification, render fidelity against Lightroom, AI models, HDR / video / Classic modules (raw detail:
[`docs/raw-parity.md`](docs/raw-parity.md)). A ✅ row is not proof of parity — if you find a ✅ feature that is
wrong or incomplete, downgrade it to 🟡 with a note.
4. **Pick work** from [`docs/gaps.md`](docs/gaps.md) (ranked by user impact; `B` marks beta blockers). The row-by-row
tracker is [`docs/parity-checklist.md`](docs/parity-checklist.md) (one row per feature, menu item and shortcut).
When you land a feature, update its row(s) and the gap entry in the same commit, and the numbers in `ROADMAP.md` /
`docs/target-app-parity.md` when a listed gap closes (bump each doc's *Last updated* line and revision history, per
craftrules `standards/progress-docs.md`); `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and
path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
5. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
`plan/` is gitignored (local-only).
CLAUDE.md
+16 -6
View File
@@ -5,12 +5,18 @@ LightCraft is a clean-room, open-source, pure-Rust photo library + non-destructi
## Start every session here
1. Read `plan/STATUS.md` (current milestone, next unchecked task, blockers).
2. Read the task in `plan/execution-plan.md` §3, the relevant section of `plan/architecture.md`, and the README/docs of the crate you touch. Behaviour/visual reference: `plan/lightroom/` (incl. `10-observed-ui.md` + screenshots).
3. **Know where we stand:** read [`ROADMAP.md`](ROADMAP.md) → *Where we stand* (honest assessment by dimension and
by kind of user) and *Where we're going* (ordered priorities). The checklist counts features that *exist*; the real
gaps are quality and coverage: camera colour calibration, CR3 / compressed raws, per-model verification, render
fidelity against Lightroom, AI models, HDR / video / Classic modules. A ✅ row is not proof of parity — if you find a
✅ feature that is wrong or incomplete, downgrade it to 🟡 with a note.
4. **Pick work** from [`docs/parity.md`](docs/parity.md) → *Top gaps* (ordered by user impact; the tracker has one row per feature, menu item and shortcut). When you land a feature, update its row(s) and the gap list in the same commit, and the *Where we stand* / *Where we're going* sections of `ROADMAP.md` when a listed gap closes; `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
3. **Know where we stand:** read [`ROADMAP.md`](ROADMAP.md) (stage, headline numbers, every dimension) and, for
method and evidence, [`docs/target-app-parity.md`](docs/target-app-parity.md). The checklist counts features that
*exist*; the real gaps are quality and coverage: camera colour calibration, CR3 / compressed raws, per-model
verification, render fidelity against Lightroom, AI models, HDR / video / Classic modules (raw detail:
[`docs/raw-parity.md`](docs/raw-parity.md)). A ✅ row is not proof of parity — if you find a ✅ feature that is
wrong or incomplete, downgrade it to 🟡 with a note.
4. **Pick work** from [`docs/gaps.md`](docs/gaps.md) (ranked by user impact; `B` marks beta blockers). The row-by-row
tracker is [`docs/parity-checklist.md`](docs/parity-checklist.md) (one row per feature, menu item and shortcut).
When you land a feature, update its row(s) and the gap entry in the same commit, and the numbers in `ROADMAP.md` /
`docs/target-app-parity.md` when a listed gap closes (bump each doc's *Last updated* line and revision history, per
craftrules `standards/progress-docs.md`); `cargo xtask parity` (part of `ci`) checks that every `cmd:`/`ctl:` id and
path the tracker cites still exists, and `cargo xtask parity --write` refreshes its summary.
5. Follow the autonomous operation protocol (`plan/execution-plan.md` §7). Don't stop to ask unless §7 lists the decision as the user's.
`plan/` is gitignored (local-only).
@@ -55,6 +61,10 @@ of it. Full standard: `../craftrules/standards/never-crash.md`
- **Pure Rust** in the product. No C/C++ dependencies.
- **Layering** (`plan/architecture.md` §3, enforced by `cargo xtask layers`): nothing below L5 depends on egui/eframe/winit/rfd.
- **Everything is a command** (`crates/engine`): id, label, menu path, shortcut, params, enabled(), run(). UI, CLI, control channel and MCP all dispatch by id. Every slider is a `develop` control spec.
- **Long-running work shows in the activity stack** ([`docs/background-tasks.md`](docs/background-tasks.md)): anything
that runs off the UI thread and can take more than a moment holds a `TaskGuard` from `session.activity.start(kind,
label, cancel)` for its whole run. It adopts the cancel flag the job already checks (`Cancel::Flag`), or uses
`Cancel::No` when the job can't stop. No progress window, panel or progress toast of its own; its tests assert the row.
- **Resolution independence:** settings use normalized image coordinates and relative radii; previews and exports must match.
- **Quality gates** before every commit: `cargo xtask ci` (fmt, clippy -D warnings, tests, layers, assets, wasm).
- **Commits:** one task id per commit (`M2.3: local Laplacian highlights/shadows`). Only green states. End messages with the attribution line required by the environment.
Cargo.lock
Generated
+1
View File
@@ -3352,6 +3352,7 @@ dependencies = [
"lightcraft-engine",
"lightcraft-faces",
"lightcraft-geom",
"lightcraft-merge",
"lightcraft-meta",
"lightcraft-pipeline",
"lightcraft-preview",
NOTICE
+7
View File
@@ -15,6 +15,13 @@ list (author, source, license) is in assets/ATTRIBUTION.md.
Inter (assets/fonts/Inter-*.ttf) — SIL Open Font License 1.1. See assets/fonts/OFL-Inter.txt.
Showcase photographs by Ansel Adams, Dorothea Lange and NASA (docs/images/) — public domain. See assets/ATTRIBUTION.md.
crates/raw/src/spectral_table.rs (lightcraft-raw) holds camera colour matrices derived from the
camera spectral sensitivities, training reflectances and model alias list of rawtoaces-data
(https://github.com/AcademySoftwareFoundation/rawtoaces-data, v1.1.0), an Academy Software
Foundation project licensed under the Apache License 2.0, and from the CIE 1931 colour-matching
functions it ships (CIE, CC-BY-4.0). The matrices were fitted by the LightCraft contributors
(2026) with tools/spectral_camera_table.py. See assets/ATTRIBUTION.md.
crates/segment (lightcraft-segment) ports the SAM 3 model code of Hugging Face Transformers
(https://github.com/huggingface/transformers, src/transformers/models/sam3 and sam3_tracker),
Copyright The HuggingFace Team and Meta Platforms, Inc., licensed under the Apache License 2.0.
README.md
+12 -10
View File
@@ -297,18 +297,20 @@ lightcraft --control 7980 ~/Pictures/trip
## Feature status
LightCraft is young and moving fast. **Where we honestly stand** (details in the [roadmap](ROADMAP.md#where-we-stand)):
LightCraft is young and moving fast: **alpha**. **Where we honestly stand** (details in the [roadmap](ROADMAP.md) and
[docs/target-app-parity.md](docs/target-app-parity.md)):
- **By feature count we're at ~79%** of Lightroom (core features 98%), tracked row by row in
[docs/parity.md](docs/parity.md).
- **As a day-to-day Lightroom replacement we're nearer 60–70%.** It's great for JPEG/DNG and most Nikon / Sony /
older-Canon raws on one machine.
- **By feature count we're at ~81%** of Lightroom (core features 98%), tracked row by row in
[docs/parity-checklist.md](docs/parity-checklist.md).
- **As a day-to-day Lightroom replacement we're nearer 61% for the whole product, ~49% for a typical working photographer and ~64% for
casual use.** It's great for JPEG/DNG and most Sony / Panasonic /
Fujifilm / Nikon / older-Canon raws on one machine; see [docs/raw-parity.md](docs/raw-parity.md) for your camera.
- **The biggest gaps:**
- **camera colour calibration:** Sony, Nikon, Panasonic, Fujifilm and Canon CR3 raws have guarded estimates from their camera JPEGs, with built-in ILCE-7CR, ILCE-7M4, X-H2S and X-T4 profiles; measured calibration is missing, and other raws or rejected fits retain a neutral matrix;
- **compressed Olympus raws and unsupported CR3 variants:** these use embedded JPEG previews when present. Fujifilm lossless/lossy compressed RAF now decodes sensor data; [verification and existing-library reload instructions](docs/raf-compression.md);
- **AI masks and denoise:** subject and sky selection are classical heuristics;
- **on-screen HDR display, video and the Classic Print / Book / Map modules.**
- **What's next:** see [where we're going](ROADMAP.md#where-were-going).
- **What's next:** see [the ranked gaps](docs/gaps.md) and [the roadmap](docs/roadmap.md).
| Area | Status |
|---|---|
@@ -318,10 +320,10 @@ LightCraft is young and moving fast. **Where we honestly stand** (details in the
| Masking: brush, linear, radial, luminance/colour range, add/subtract/intersect | ✅ (AI subject/sky use classical heuristics for now) |
| Crop, straighten tool + auto straighten, flip, rotate, aspect ratios, overlays | ✅ |
| Profiles (Color, Neutral, Vivid, Landscape, Portrait, Monochrome: our own looks), presets, versions, history, copy/paste/sync settings | ✅ |
| Camera colour: DNG files use their own matrices | ✅ DNG · 🟡 own Sony/Fujifilm profiles; measured calibration database missing |
| Camera colour: DNG files use their own matrices | ✅ DNG · 🚧 own Sony/Fujifilm profiles; measured calibration database missing |
| Native macOS menu bar (generated from the command registry), control channel + every widget addressable, headless UI snapshots | ✅ |
| RAW: DNG, CR2, CR3 (lossless CRX Bayer and version 0x100/0x200 C-RAW), ARW, NEF (uncompressed + lossless/lossy compressed), Fujifilm RAF (uncompressed + lossless/lossy compressed, Bayer + X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format, DMC-LX1 to DC-S1RM2), Pentax PEF, Olympus ORF (uncompressed), Samsung SRW (uncompressed); embedded previews for every format incl. CR3 | 🟡 · CR3, compressed ORF and compressed SRW decode ⬜ |
| Detail: sharpening, luminance + colour noise reduction | ✅ · AI Denoise, Super Resolution ⬜ |
| RAW: DNG, CR2, CR3 (lossless CRX Bayer and version 0x100/0x200 C-RAW), ARW, NEF (uncompressed + lossless/lossy compressed, incl. lossy after split), Fujifilm RAF (uncompressed + lossless/lossy compressed, Bayer + X-Trans), Panasonic RW2 / Leica RWL / Panasonic RAW (every raw format, DMC-LX1 to DC-S1RM2), Pentax PEF, Olympus ORF (uncompressed, 12.8 bpp), Samsung SRW (uncompressed); embedded previews for every format incl. CR3 | 🚧 · other CR3 variants, compressed ORF / SRW and Nikon HE NEF decode ⬜ ([raw parity](docs/raw-parity.md)) |
| Detail: sharpening, luminance + colour noise reduction | ✅ · AI Denoise 🚧 (Bayer, user-installed weights), Super Resolution ⬜ |
| Remove / Heal / Clone spots (auto source), Visualize Spots, Red Eye and Pet Eye (auto pupil detection, catchlight) | ✅ · content-aware fill, spot pin editing 🚧 |
| Export: JPEG / PNG / TIFF / WebP / AVIF / DNG / original, sizing, file-size limit, output sharpening, naming templates, batch, metadata policy, text or image watermark | ✅ |
| HDR: HDR editing with a headroom limit, SDR rendition, Visualize HDR; export as ISO 21496-1 gain map JPEG, PQ AVIF or 32-bit float TIFF | ✅ · HDR display ⬜ |
@@ -332,7 +334,7 @@ LightCraft is young and moving fast. **Where we honestly stand** (details in the
| Optics (distortion, vignetting, auto + manual CA, defringe, lens corrections embedded in DNG files and Panasonic / Leica RW2 / RWL distortion data), Geometry (transforms, Constrain Crop), Upright (Auto/Level/Vertical/Full/Guided) | ✅ · camera lens profiles (our own) ⬜ |
| Photo Merge: HDR (auto-align, deghost), Panorama (spherical/cylindrical/perspective, boundary warp, auto crop), HDR Panorama → DNG | ✅ |
| GPU pipeline (wgpu compute, CPU-exact within 1/255), CPU fallback on device limits / errors | ✅ · WebGPU in the browser 🚧 |
| AI: segmentation masks, AI denoise, super resolution, faces; video | ⬜ (see [roadmap](ROADMAP.md#where-were-going)) |
| AI: segmentation masks, AI denoise, super resolution, faces; video | 🚧 (see [gaps](docs/gaps.md)) |
| Web build (same UI in the browser via WASM): persistent library in OPFS/IndexedDB, Web Worker rendering, export downloads | ✅ · WebGPU, Safari/Firefox testing 🚧 |
<sub>✅ works today · 🚧 in progress · ⬜ not started</sub>
ROADMAP.md
+138 -212
View File
@@ -1,240 +1,166 @@
# LightCraft roadmap
Milestones toward full Adobe Lightroom parity (cloud Lightroom first, then every Lightroom Classic module), with wall-clock
estimates for continuous (24/7) agent-driven development with 4–6 parallel agents. Estimates are calibrated on the sibling
projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revised as milestones land.
**Stage: alpha** · next: beta, ~14 points and ~340–600 h away
## Where we stand
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (full re-measure against Lightroom Classic 15.5 and the installed Lightroom 9.6; restructured to the progress-docs standard) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
*Honest assessment, 2026-10-08. Agents: read this before picking work. The checklist in
[`docs/parity.md`](docs/parity.md) counts features that **exist**; this section is about whether a photographer can
**switch** from Lightroom. Update it when a gap below closes.*
One-page summary of how close LightCraft is to Adobe Lightroom. Agents: read this, then pick work
from [docs/gaps.md](docs/gaps.md). The full assessment, method and evidence are in
[docs/target-app-parity.md](docs/target-app-parity.md).
**In one line:** the checklist says **79.7%** (P0 98.2%, P1 95.6%, P2 41.6%), but measured by whether a working
photographer could replace Lightroom without noticing, we are at roughly **60–70%**. The remaining gap is mostly
**quality of results and camera coverage**, not missing buttons.
## Headline numbers
Caveat on the checklist: ✅ is set by whoever lands a feature, and nobody has systematically checked rows against
Lightroom's behaviour or output. Bugs keep turning up in ✅ areas (CR2 colour-filter phase on some Canon models #85,
duplicate Local entries #22, black GPU exports on an Intel iGPU #78).
### By dimension
| Dimension | Estimate | What's true today | Biggest gaps |
|---|---:|---|---|
| **Feature checklist** | 79.7% | P0 core and P1 nearly complete: import (Add / Copy / Move, templates, devices), library, grid/loupe/compare/survey, every Edit slider, curves, colour grading, masking tools, crop/Upright, heal/clone, presets/profiles, versions/history, sync, export, menus, shortcuts | P0: full-resolution zoom detail; P1: lens-profile database, content-aware fill (patch synthesis), video playback/trim |
| **RAW coverage** (formats people shoot) | ~55% | DNG (all kinds), CR2, CR3 lossless Bayer and version 0x100/0x200 C-RAW (M50/R100/R8 pixel-exact corpus), ARW, NEF (uncompressed + Huffman lossless/lossy), RAF (uncompressed, lossless/lossy compressed Bayer and X-Trans), uncompressed ORF, uncompressed Samsung SRW (NX5 to NX1100, EX1, WB2000), RW2 / RWL / Panasonic RAW (every raw format, checked on 178 files from 118 Panasonic and Leica bodies), PEF; every container's embedded preview (incl. CR3) | **CR3** (every Canon since ~2018), compressed ORF, compressed Samsung SRW (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini), Nikon lossy-after-split, Canon sRAW, HEIC/AVIF. NEFs labelled compressed but stored uncompressed (Z 6 packed 14-bit, D850 12-bit uncompressed). Nikon High Efficiency NEF (HE / HE★: Z 8, Z 9, Z 6III, Z f; JPEG XS-based, preview only, #193). Per-model verification is thin (~55 corpus files vs >1,000 models) |
| **Colour & image quality** | ~55–65% | Pipeline is complete and fast; GPU path CPU-exact within 1/255 | **No measured camera calibration database**: ARW, NEF, RW2, RAF, CR3, CR2 and PEF have guarded camera-JPEG colour estimates, with bundled ILCE-7CR, ILCE-7M4, X-H2S and X-T4 profiles (docs/camera-preview-colour.md); other non-DNG raws and rejected estimates use a neutral matrix. Colour fidelity remains incomplete. No lens-profile database. No measured fidelity against Lightroom (tone, highlights, texture/clarity, NR, sharpening are tuned by eye) |
| **AI & computational** | ~15–20% | Assisted culling (focus, bursts), auto tone, HDR/panorama merge; subject/sky/background masks as classical heuristics, or the photo's own segmentation mattes when it ships them (DNG semantic masks, e.g. iPhone ProRAW's sky matte); non-destructive Bayer RAW denoise with opt-in weights on pure-Rust CPU/GPU; **faces (MVP, [docs/faces.md](docs/faces.md))**: a pure-Rust detector (a 232 KB opt-in download), opt-in recognition models (one-click download or add a file; they run on the shared pure-Rust CPU interpreter; pinned SFace/AuraFace embeddings agree with an independent reference runtime on 40 public crops; one-thread medians 34–36 / 233–252 ms (AuraFace remains expensive)), a background scan that finds and embeds faces, name suggestions, a person's page with look-alike faces to confirm, an Unnamed faces section (look-alikes together, select a group and name it at once) | Real segmentation masks (subject, sky, people, objects, landscape, depth), denoise model policy / linear RGB / quality verification, super resolution, lens blur, generative remove, natural-language search; for faces: clusters with separators between groups, small faces in large photos (the detector looks at 640 px), thresholds calibrated on live faces, writing face regions back to XMP. **Blocked on a model strategy** (licensable weights or our own training; pure-Rust inference is feasible) |
| **Workflow & library** | ~85% (single machine) | Robust catalog (journal + snapshots, background compaction, crash-tested), 85k-photo libraries stay responsive, Local browsing with automatic cleanup, XMP interop, keywords, smart albums, Move import | Opening an 85k library takes 1.7–4.7 s; no cloud sync (out of scope), no tablet companion (#74, roadmap), shared albums, publish services, tethering |
| **Classic modules** | ~30% | Geotagging from GPX track logs, soft proofing (partial), slideshow (basic), paginated contact-sheet PDF export (A4/Letter, layout/captions, current edits, 150 dpi sRGB) | **Map view, Book, full Print module, Slideshow module, Web, publish services**: contact sheets work, custom print packages and printer colour management remain missing |
| **HDR & video** | ~25% | HDR editing (headroom limit, SDR rendition, Visualize HDR); gain map JPEG, PQ AVIF and float TIFF export | On-screen HDR display, HLG, HDR settings from crs XMP; video play/trim/edit/export |
| **Platform & robustness** | ~70% | macOS native; Windows/Linux builds; web via WASM; no-panic lints workspace-wide, `unsafe` confined to `crates/sysmem`; failed saves are reported; GPU errors fall back to CPU | Windows installer UI unverified on Windows (PR #79); GPU path proven only on Apple + user reports; English, Simplified/Traditional Chinese, Japanese, Brazilian Portuguese, Spanish, German, Russian and Ukrainian UI (see docs/localization.md); remaining technical errors; Traditional Chinese uses the Simplified Chinese font until craft-fonts has a TC face; accessibility partial; headless UI tests time out under machine load |
### By kind of user
| User | Readiness | What blocks them |
| | Value | Kind |
|---|---:|---|
| JPEG / DNG shooter, single machine | ~85% | Fidelity polish, AI masks |
| Nikon / Sony / Panasonic / older-Canon raw shooter | ~65% | Camera colour fidelity and coverage (ARW, NEF and RW2 preview estimates are only a starting point); Panasonic compacts and kit zooms also miss their embedded distortion correction |
| Fujifilm raw shooter | ~65% | Uncompressed and lossless/lossy compressed RAF decoded; camera colour calibration remains missing. Verified on 13 bodies, including 14/16-bit GFX; see `docs/raf-compression.md` |
| Canon CR3 / Olympus compressed-raw shooter | ~45% | CR3 lossless Bayer and version 0x100/0x200 C-RAW develop from sensor data (M50, R100, R8 verified pixel for pixel); other CRX variants and compressed ORF open as embedded JPEG previews. Camera colour and broader model verification remain |
| Lightroom Classic power user | ~45% | Print, Book, Map, publish, tethering |
| Relies on AI (masks, denoise) | ~25% | Object / Describe masks via optional SAM 3 (download not yet hosted); Bayer AI denoise needs separately installed weights; X-Trans/linear RGB, model policy and quality parity remain open |
| **Feature breadth** (checklist) | **81.1%** of 517 rows (P0 98.0%, P1 95.1%, P2 46.9%) | measured: [parity-checklist.md](docs/parity-checklist.md) recount |
| **Feature breadth** (weighted by use) | **~77%** | estimated |
| **Ready for real work** (full target) | **~61%** (56–66%) | estimated: weighted sum over the dimensions |
| **Mainstream practitioner** | **~49%** (44–54%) | estimated |
| **Essentials user** | **~64%** (59–69%) | estimated |
| Remaining to **beta** | **~340–600 h** (Opus 5.5 agent hours, ~60% parallel) | estimated |
| Remaining to **full parity** | **~1,000–1,800 h** (~70% parallel) | estimated |
## Where we're going
Each audience with its own hours (Opus 5.5 agent wall-clock hours to ~95%):
Priorities, in order. Each points at tracker rows in [`docs/parity.md`](docs/parity.md) → *Top gaps*.
| Audience | Ready | Opus 5.5 agent wall-clock hours to ~95% | Work that dominates |
|---|---:|---:|---|
| Full target (ready for real work) | **~61%** | **1,000–1,800 h** (~70% parallelizes) | raw colour, lens and camera data; AI models; Classic output modules; localization; plug-in ecosystem |
| Mainstream practitioner | **~49%** | **450–840 h** (~60% parallelizes) | camera colour and render fidelity; CR3 / ORF / HE NEF; lens database; subject / sky masks; Lightroom XMP exchange; stability |
| Essentials user | **~64%** | **160–300 h** (~50% parallelizes) | default raw look for popular cameras; launch stability; basic-slider feel; opening CR3 / HEIC files people send |
1. **Camera colour calibration of our own** (LR-PROF-CAMERACOLOR, P0): fit each camera to its own embedded JPEG, use
matrices the files carry themselves, then chart shots. Sony ARW, Nikon NEF, Panasonic RW2, Fujifilm RAF, Canon CR3/CR2,
Pentax PEF, Samsung SRW and Olympus ORF have the file-local fit (matrix + tone curve from their own JPEG; CR2 and PEF
since issue #310, not yet checked on the corpus); then validate fidelity.
2. **Raw formats, clean-room** (LR-IMP-FORMATS, P0): Remaining **unverified CR3 CRX** variants, compressed ORF, NEF
lossy-after-split, sRAW. Decided 2026-10-05: write our own decoders from prose descriptions (never decoder source,
no LGPL dependency); compressed NEF (#86) is the template.
3. **Verified camera coverage** (LR-IMP-CAMERA-COVERAGE, P0): a CC0 sample per model in the corpus, each decoded and
checked for plausible colour; fix per-model bugs (#85).
4. **Render fidelity suite** (LR-BEHAV-RENDER-FIDELITY, P1): measure our output against Lightroom on the same CC0 raws
(references stay in the local `plan/`), then tune against the numbers.
5. **Lens profiles of our own** (LR-EDIT-OPTICS-PROFILE, P1).
6. **AI model strategy** (maintainer decision): which permissively licensed models (or our own training) for
segmentation masks and denoise. Bayer denoise inference is implemented in pure Rust with an opt-in GPL model; release policy and independently licensed weights remain open. Unblocks M12 and Enhance. Face detection and recognition run on our own checked Rust engine. Object / Describe masks now
run Meta's SAM 3 in pure Rust (optional, user-consented download under the non-OSI SAM License; CDN mirrors still
to be set up, docs/ai-masks.md); Subject / Sky / People, Raw Details and Super Resolution are still open.
7. **Then:** HDR (Q), the Classic output modules (expand contact-sheet PDF export into full Print, then Map view, Book, Slideshow), video (R), localisation
and accessibility.
The gap between breadth and readiness is quality and coverage, not missing buttons: camera colour is
estimated rather than measured, CR3 decodes on three verified Canon bodies, compressed ORF and HE NEF
open as previews, rendering is tuned by eye, AI masks are heuristics, there is no lens database, and
tests run in CI on FreeBSD only. **Why alpha:** core workflows run end to end, but readiness is below
the ~75% beta bar (it is above the ~40% alpha bar) and Lightroom's main inputs, raw files, still have blocking gaps (Canon, ~30% of
raw users, decodes CR3 on three verified bodies). It passes the alpha gate: all six core workflows
(import, cull, develop, save and reopen, batch looks, export) work end to end on macOS
([docs/roadmap.md](docs/roadmap.md#alpha-gate)). The three readiness numbers and their weights and discounts are in
[target-app-parity.md](docs/target-app-parity.md#ready-for-real-work-mainstream-practitioner-and-essentials-user).
Hours are calibrated on this repo's PR history; see
[target-app-parity.md](docs/target-app-parity.md#how-hours-are-calibrated).
Already closed in the week of 2026-10-03: compressed NEF (#10), Move import (#29), per-library smart-preview folder,
Local roots / cleanup, grid and catalog performance at 85k photos (#35, #37), failed-save reporting, GPU export
hardening (#78), copyright metadata (#51), GPX geotagging (#60), import tag help and folder templates (#31, #32).
## By dimension
Library folders now have records of their own in the catalog (`Catalog.folder_records`, keyed by folder path; colour
labels first). Photos still store full paths, so moving a whole disk or folder relinks each photo. If that becomes a
problem (an "Update Folder Location" for a renamed or replaced drive, journal growth on large moves), the next step is
the Classic model: photos point to a folder id (root + relative path), and the records keyed by path become those rows.
| Dimension | Ready | Breadth | Remaining | Doc |
|---|---:|---:|---:|---|
| Features | ~61% | ~77% (checklist 81.1%) | 600–1,100 h | [target-app-parity](docs/target-app-parity.md#features) |
| ↳ RAW cameras, colour, lens profiles | ~56% | 81% of 1,446 Adobe models decode, 16% verified | 255–480 h | [raw-parity](docs/raw-parity.md) |
| UI/UX fidelity | ~70% | ~85% | 40–70 h | [ui-parity](docs/ui-parity.md) |
| File formats (non-raw) | ~70% | ~85% | 30–55 h | [file-format-parity](docs/file-format-parity.md) |
| Hardware | ~40% | ~50% | 40–80 h | [hardware-parity](docs/hardware-parity.md) |
| Localization | ~52% of the 12 key languages | 7 of 12 present | 130–210 h (+145–255 h new scripts) | [localization-parity](docs/localization-parity.md) |
| Performance | ~70% | ~80% | 30–50 h | [target-app-parity](docs/target-app-parity.md#by-dimension) |
| Stability | ~60% | — | 30–60 h | [target-app-parity](docs/target-app-parity.md#by-dimension) |
| Platforms | ~60% | macOS, Windows, Linux, FreeBSD, web | 25–45 h | [target-app-parity](docs/target-app-parity.md#by-dimension) |
| Ecosystem / plug-ins | ~15% | ~25% | 40–80 h | [target-app-parity](docs/target-app-parity.md#by-dimension) |
| AI features | ~20% | ~30% | 120–220 h (overlaps features) | [target-app-parity](docs/target-app-parity.md#ai) |
Image navigation now supports native pinch zoom and two-finger pan in Detail, Compare and Reference views.
The whole-image preview still has a configurable size cap (2,560 px default), so 100% on larger photos does not
yet guarantee original-pixel detail; visible-region rendering remains a gap (LR-VIEW-ZOOM).
## Features
## Milestones
**Status legend:** ✅ done · 🚧 in progress · ⬜ not started
| # | Milestone | Scope (summary) | Estimate (h) | Status |
|---|---|---|---|---|
| M0 | Skeleton + visual shell | workspace, xtask CI + layering, geom/color/raster, develop model, pipeline v0, catalog v0, engine commands, Lightroom-look UI (grid, loupe, filmstrip, Edit panel), control channel, MCP, web build | 3–5 | ✅ |
| M1 | Library core | import (JPEG/PNG/TIFF/WebP), EXIF/XMP, persistent catalog (op log + snapshots), albums, ratings/flags/labels, filter/search/sort, thumbnail cache, 100k-photo grid | 6–10 | ✅ (100k-photo grid scale test pending) |
| M2 | Pipeline v1 (quality) | WB temp/tint, profiles, local tone mapping (highlights/shadows), curves, HSL, point colour, colour grading, texture/clarity/dehaze, vignette, grain, B&W, auto tone/WB, histogram, before/after | 10–15 | ✅ (look tuning vs our references ongoing) |
| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW, NEF uncompressed + Huffman lossless/lossy, embedded previews ✅; NEF lossy-after-split ⬜) |
| M4 | Crop, geometry, optics | crop tool + overlays, straighten, Upright (auto/level/vertical/full/guided), manual transforms, CA, defringe, manual lens corrections | 6–10 | ✅ |
| M5 | Performance | source pyramids, wgpu compute pipeline (CPU oracle), draft/full renders, prefetch, budgets (16 ms slider updates on 24 MP) | 10–15 | 🚧 (stage cache, source pyramid, wgpu pipeline, prefetch, memory budget ✅; colour NR at half resolution, GPU histogram ⬜) |
| M6 | Masking | brush, linear/radial gradients, colour/luminance/depth range, add/subtract/intersect/invert, all local adjustments, masks panel | 8–12 | 🚧 (brush/linear/radial/colour/luminance range, add/subtract/intersect, masks panel ✅; depth range, AI masks ⬜) |
| M7 | Detail | sharpening + masking preview, luminance/colour NR, Denoise, Raw Details, Super Resolution | 6–10 | 🚧 (sharpening, luminance/colour NR ✅; AI Denoise 🟡 (Bayer, optional weights; policy/fidelity open); Raw Details, Super Resolution ⬜) |
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs, HDR gain map JPEG / PQ AVIF ✅; JXL encode ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (CR3 lossless CRX and C-RAW, RAF uncompressed and lossless/lossy compressed, RW2 (every raw format), PEF, ORF uncompressed ✅; **camera colour calibration** 🚧 (guarded ARW, NEF, RW2 and RAF preview fitting; measured database still missing), CR3, compressed ORF ⬜) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
| M15 | Classic modules | Map, Book, Slideshow, Print, Web; smart collections, stacks, virtual copies, publish services, tethering | 25–40 | 🚧 (smart albums, stacks, virtual copies, compare/survey ✅; Map/Book/Slideshow/Print/Web ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; English/Chinese (Simplified, Traditional)/Japanese/Brazilian Portuguese/Spanish/German/Russian/Ukrainian localisation 🟡; accessibility and further locales ⬜) |
## Parity estimate (feature count updated 2026-10-08; effort estimate from 2026-10-02)
**By feature count** — from `docs/parity.md` (one row per Lightroom feature, menu item and shortcut; `cargo xtask
parity` prints this line on every run, so it stays current):
| Scope | Weighted completion | Rows | 2026-10-02 |
| Area | Ready | Breadth | Remaining |
|---|---:|---:|---:|
| P0 (core) | **98.2%** | 200 | 98.7% |
| P1 (important parity) | **95.6%** | 149 | 94.4% |
| P2 (later / AI / niche, incl. Classic modules) | **41.6%** | 160 | 29.4% |
| **All in-scope rows** | **79.7%** | 509 | 75.6% |
| Import and library | 80% | 90% | 25–45 h |
| RAW decoding and camera coverage | 76% | 81% | 120–220 h |
| Colour, camera profiles, render fidelity | 35% | 65% | 80–150 h |
| Global develop adjustments | 70% | 93% | 30–60 h |
| Optics, lens profiles, crop and geometry | 50% | 80% | 40–80 h |
| Masking | 45% | 67% | 60–110 h |
| Remove, heal, Enhance | 30% | 45% | 70–130 h |
| Presets, profiles, versions, sync | 85% | 95% | 8–15 h |
| Export | 75% | 89% | 15–30 h |
| Merge and HDR | 60% | 85% | 20–35 h |
| Views, compare, zoom, culling | 70% | 90% | 15–30 h |
| Classic output modules (Map, Book, Slideshow, Print, publish, tethering) | 10% | 15% | 90–150 h |
| Video | 0% | 0% | 25–45 h |
The 2026-10-05 count includes three new 🟡 rows that make quality gaps visible (camera colour calibration, verified
camera coverage, render fidelity), which is why P0 dipped slightly.
Weights and evidence: [target-app-parity.md](docs/target-app-parity.md#features).
✅ counts 1, 🟡 ½, ⬜ 0; out-of-scope rows (cloud sharing, Adobe accounts…) are left out.
## Languages
**By remaining effort** — rows are not equal: a shortcut and the whole Book module are one row each, and what is
left is the heavy part (AI, video, Classic output modules, undocumented raw codecs). Remaining work in **Opus agent-hours**
(one agent working continuously; calibrated on this project — a single lead agent closed ≈ 45 tracker rows of
UI/feature work in ≈ 5 h on 2026-10-01, and ≈ 60 more (preset import incl. `.lrtemplate` / DNG / zip and masks, smart-album
rule editor, auto sync, keyword and label sets, import options and DNG conversion, smart previews, external-editor round
trip, slideshow, auto import…) in ≈ 12 h on 2026-10-02; four to six parallel agents built M0–M13 in ≈ 25 active hours):
Measured from `crates/ui-egui/locales` (share of 2,304 catalog messages translated); none is `full`
while lower-layer errors stay English. Detail: [localization-parity.md](docs/localization-parity.md).
| Work package | Tracker rows | Agent-hours | Risk |
| Language | Code | Translated | Status |
|---|---|---:|---|
| Remaining P0/P1 UI and library features (folder rename/move, keyword painter, people view…) | ≈ 8 | 5–10 | low |
| Raw codecs: CR3 (CRX), compressed ORF, NEF lossy-after-split, HEIC/AVIF decode | LR-IMP-FORMATS | 40–80 | **high** — clean-room black-box analysis, no permissive specs |
| Lens-profile database of our own (calibration targets, fitting, data) | LR-EDIT-OPTICS-PROFILE | 15–30 | data collection |
| Video: playback, trim, edits, export (pure-Rust decode, ideally shared with FilmCraft) | R. Video | 20–40 | medium |
| AI: subject / sky / background / people / object masks, object-aware remove, AI denoise, super resolution, lens blur, people & faces, natural-language search, culling | ≈ 30 | 80–150 | **high** — permissively licensed weights, pure-Rust inference, maybe training |
| HDR editing, display, visualisation and export | Q. HDR, LR-EXP-HDR | 15–25 | medium |
| Classic modules: Map, Book, Slideshow module, Print, publish services, tethering, soft proofing | ≈ 50 | 60–100 | medium (large, well-understood) |
| Smaller P2 items: Enhance dialog, export to Photos, help / what's new, accessibility, localisation, sidecar variants | ≈ 15 | 12–25 | low |
| Look tuning, performance budgets (colour NR at half resolution on CPU + GPU, 100k-photo library), packaging, hardening | — | 25–45 | medium |
| **Total remaining** | | **≈ 270–505** | |
| English | en | 100% | full (source) |
| Simplified Chinese | zh-hans | 87% | partial |
| Spanish | es | 86% | partial |
| Hindi | hi | 0% | none (no Devanagari shaping) |
| Arabic | ar | 0% | none (no RTL / shaping) |
| French | fr | 88% | partial |
| Portuguese (Brazil) | pt-br | 85% | partial |
| Indonesian | id | 0% | none |
| Japanese | ja | 87% | partial |
| German | de | 91% | partial |
| Korean | ko | 0% | none (no Hangul face) |
| Vietnamese | vi | 0% | none |
Spent so far ≈ 105–145 agent-hours, so **by effort the project is ≈ 20–35% of the way to complete Lightroom + Classic
parity**, and ≈ 45–60% of the way for cloud-Lightroom parity without AI and the Classic modules (remaining ≈ 130–255 h,
most of it the raw codecs, lens data, video and HDR).
Also shipped: Traditional Chinese (87%), Russian (74%), Ukrainian (99%).
**Wall clock:** ≈ 270–505 h for one agent working alone; with 4–6 parallel agents (≈ 70% parallel efficiency, merges and
CI under load cost the rest) **≈ 65–125 h of continuous work**. The AI package and the raw codecs carry most of the
uncertainty: they can finish faster if suitable permissive models / documentation turn up, or stall on licensing.
## Upcoming
## Totals
Ranked toward beta; detail in [docs/roadmap.md](docs/roadmap.md), gaps in [docs/gaps.md](docs/gaps.md).
Remaining from 2026-10-02 evening (see *Parity estimate* above for the breakdown):
1. Camera colour of our own: measured calibration for the popular models (50–90 h)
2. CR3 for every Canon body (25–45 h)
3. Render-fidelity suite against Lightroom, then tuning (30–60 h)
4. Full-resolution zoom (8–15 h)
5. AI masks: Subject / Sky / People (40–70 h + model decision)
6. Compressed ORF and HE NEF (28–55 h)
7. Lens profile database of our own (40–80 h)
8. Tests in CI on macOS, Windows and Linux (6–12 h)
9. Per-model raw verification (20–35 h)
| Target | Remaining agent-hours | Wall clock, 4–6 parallel agents |
|---|---:|---:|
| Cloud-Lightroom parity without AI or Classic modules | ≈ 130–255 h | ≈ 35–65 h |
| Full parity incl. AI, video and the Classic modules | ≈ 270–505 h | ≈ 65–125 h |
## Progress log
The milestone estimates in the table above were made before work started and are kept for calibration (M0–M13 took
≈ 25 active hours with parallel agents against an estimate of ≈ 110–170 h for those milestones).
- 2026-10-10 (RAW): RAW measured model by model against Adobe's 1,446-model list (234 verified, 943
unverified, 233 preview-only, 36 unsupported; ~84% of photographers' cameras decode, ~40% on a
verified body). Full ~61%, mainstream ~49%, essentials ~64%; beta ~14 points away.
- 2026-10-10 (final): full ready is a weighted sum over the dimensions (~59%; method aligned with
the standard); mainstream practitioner ~51% (450–840 h), essentials user ~63% (160–300 h);
alpha gate passes.
- 2026-10-10: full re-measure for the progress-docs standard: stage alpha, ready ~55%, checklist
81.1%; new docs (target-app-parity, gaps, roadmap, architecture, raw / file-format / hardware / UI
/ localization parity); `docs/parity.md` renamed `docs/parity-checklist.md`. Landed today: Nikon
lossy-after-split NEF (#633), ORF 12.8 bpp (#651), HDR gain map JPEG / PQ AVIF / float TIFF export
(#646), Lightroom-like Auto (#628), activity stack.
- 2026-10-09: Sony ILCE-7CR camera colour: bundled profile from 125 photos; on 26 withheld photos mean
ΔE vs the camera JPEG 4.07 → 3.29. HEIC in release builds (#609), Samsung uncompressed SRW,
contact-sheet PDF export, Ukrainian and French UI, release v0.5.0.
- 2026-10-08: independent Rust CR3 lossless and version 0x100/0x200 C-RAW decoding, six exact
full-sensor regressions on M50 / R100 / R8; guarded CR3 camera-JPEG colour fitting
([cr3.md](docs/cr3.md)).
- 2026-10-08: Fujifilm lossless / lossy compressed RAF: pure-Rust striped decoder, full sensor arrays
verified on 17 CC0 and 7 supplied files (13 bodies). Fujifilm camera colour: bundled X-H2S / X-T4
profiles from 201 / 545 photos; on 150 withheld files mean ΔE X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34.
AI RAW denoise (#384), Lightroom catalog import, German, Spanish and Russian UI.
- 2026-10-07: Panasonic RW2 / Leica RWL / Panasonic RAW decode in every raw format, established on
178 CC0 files from 118 bodies; RW2 file-local camera look (140 of 174 files). Faces: recognition in
pure Rust and People view.
- 2026-10-03 – 10-05: community PRs (#42–#51, #60) and 25+ issues: compressed NEF, Move import,
folder templates, Local roots and cleanup, 85k-photo grid and catalog performance, background
compaction, failed-save errors, GPU export hardening. Added the honest *Where we stand* assessment
and tracker rows for camera colour, camera coverage and render fidelity.
- 2026-10-02: parity estimate added (`cargo xtask parity` prints weighted completion); milestones
refreshed.
- 2026-10-01: RAW II formats: RAF (uncompressed Bayer + X-Trans), RW2 (packed), PEF (incl. Huffman),
ORF (uncompressed); embedded previews for every container incl. CR3; raw corpus test with 37 CC0
samples.
- 2026-09-30 (later): app running with the full Lightroom-style UI; pipeline v0; DNG / CR2 / ARW;
README showcase. ≈ 8 h elapsed.
- 2026-09-30: roadmap created; M0 in progress; research docs (Lightroom reference, Rust imaging
ecosystem) complete.
## Risks that coding hours alone don't retire
## Revision history
- **AI features** (subject/sky/people masks, generative remove) need model weights with licences we can ship; classical
fallbacks first. No permissively licensed sky-segmentation or raw-denoise model was found — we may need to train our own.
- **Camera colour and lens data** is a data problem: we never use Adobe's matrices, DCPs or LCPs. DNG-embedded data first,
then our own calibration; long-tail camera/lens coverage grows over time.
- **Raw-format sources:** decided 2026-10-05: decoders are written from *prose* format descriptions (even ones
published alongside GPL code); decoder source is never read. Still open: freedom-to-operate review for local
Laplacian filters, PatchMatch and HEVC (HEIC).
- **Look parity** with Adobe's default rendering is tuned by eye today; the planned fidelity suite (LR-BEHAV-RENDER-FIDELITY)
turns it into measured comparisons against local-only Lightroom references. Retuning changes existing edits, so it
ships as a new process version (`docs/process-versions.md`): photos keep their look until they are updated.
## Raw format coverage and known gaps
Sony ILCE-7M4 downsized lossless ARW (subsampled YCbCr 4:2:0 / 4:2:2 tiles) now decodes to linear RGB;
private 3:2 & 4:3 examples verified through native rendering & full-resolution 16-bit export.
CR3 lossless Bayer and version 0x100/0x200 C-RAW sensor decoding is verified pixel-exactly on Canon M50, R100 and R8; any other CR3, or one that fails to decode, opens from its embedded JPEG as before. See [`docs/cr3.md`](docs/cr3.md) for coding limits, reference provenance and colour limitations.
Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
LJ92, lossy JPEG / Smart Previews, Deflate, float, linear, DNG 1.7 JPEG XL tiles: lossless ones bit-exact on synthetic files,
lossy ones unclipped in the declared linear primaries, checked on one real file), CR2, ARW (uncompressed, ARW2, LJ92; as-shot white balance and black level of pre-2017 bodies from the encrypted
`SR2SubIFD`, decrypted with a keystream recovered by black-box analysis, `crates/raw/src/vendor/sr2.rs`, `crates/raw/src/vendor/arw.rs`), NEF/NRW uncompressed and Huffman-compressed (lossless, lossy type 1/2, 12/14-bit), RAF uncompressed and lossless/lossy compressed (Bayer and
X-Trans, 14/16-bit real-file verification), RW2 / Leica RWL / Panasonic RAW in every raw format (compressed formats 4 and 6, the prefix-coded strips of format 8,
packed formats 2, 5 and 7, the 16-bit words of the 2005–2007 bodies; all recovered by black-box analysis of 178 CC0 files from
118 bodies, `crates/raw/src/vendor/rw2.rs`), PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Supported containers yield their embedded JPEG preview when present (including CR3). The engine uses it for RAW variants it cannot decode yet; HEVC-only CR3 previews are not decoded.
Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):
- **Olympus compressed ORF**, **Canon CR3 unverified CRX variants** (M11.1), **Canon sRAW/mRAW**.
**Camera colour matrices:** ARW, NEF, RW2, RAF, CR3, CR2, PEF, Samsung SRW and Olympus ORF files can use guarded, separate chromaticity and tone estimates from their own embedded JPEG (see `docs/camera-preview-colour.md`); this is a per-file camera-look estimate with relative WB, not measured calibration or absolute-Kelvin WB. Other non-DNG raws and rejected fits use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax equivalents; Panasonic RW2 files carry none) and our
own chart-based calibration (M11.4). Adobe matrices are never used.
## Lightroom Classic catalog migration
File → Import Lightroom Catalog… reads `.lrcat` plus committed WAL pages directly through a pure-Rust
SQLite reader. Originals stay in place; ratings, flags, labels, XMP metadata, hierarchical keywords,
collections/sets, virtual copies and supported develop settings migrate into LightCraft. Existing edits
are preserved by default, reimport identities are persistent, and source settings/history/snapshots are
archived before catalog mutation. Missing originals remain available for relinking.
Rendering is approximate: unsupported Adobe profiles/AI/process settings are reported and archived;
history/snapshots remain source data, and smart collections import current membership. The Lightroom
database is read-only; LightCraft owns subsequent edits. See `docs/lightroom-catalog-import.md`.
## Log
- 2026-09-30: roadmap created; M0 in progress; research docs (Lightroom reference, Rust imaging ecosystem) complete.
- 2026-09-30 (later): app running with the full Lightroom-style UI; pipeline v0; DNG/CR2/ARW; README showcase. ≈ 8 h elapsed.
- 2026-10-02: parity estimate added (xtask parity prints weighted completion); milestone statuses refreshed.
- 2026-10-08: Fujifilm camera colour: guarded JPEG matching and relative WB, plus bundled X-H2S/X-T4 profiles from 201/545 photos. On 150 withheld files the pooled profiles improve mean ΔE slightly over per-file fitting (X-H2S 1.70 → 1.62, X-T4 2.43 → 2.34); dark X-H2S samples 8.74 → 2.25. Measured calibration remains open.
- 2026-10-08: Fujifilm lossless/lossy compressed RAF: pure-Rust striped decoder; full sensor arrays verified against a black-box reference on 17 CC0 files and 7 supplied X-H2S / X-T4 files (13 camera bodies, 14/16-bit Bayer and X-Trans). Camera colour calibration remains a separate gap.
- 2026-10-01: RAW II formats: RAF (uncompressed Bayer + X-Trans), RW2 (packed), PEF (incl. Huffman), ORF (uncompressed); embedded previews for every container incl. CR3; raw corpus test with 37 CC0 samples.
- 2026-10-03 – 10-05: community PRs (#42–#51, #60) and 25+ issues landed: compressed NEF, Move import, folder templates,
Local roots and cleanup, 85k-photo grid and catalog performance, background compaction, failed-save errors, GPU
export hardening, clippy 1.99. Added the honest *Where we stand* assessment and *Where we're going* priorities; added
tracker rows for camera colour, camera coverage and render fidelity.
- 2026-10-07: Panasonic RW2 / Leica RWL / Panasonic RAW decode in every raw format (compressed 4/6/8, packed 2/5/7,
16-bit words), black levels, in-camera aspect crops and mapped-out defects established on 178 CC0 files from 118
bodies; RW2 gets the file-local camera look (accepted on 140 of 174 files with a preview).
- 2026-10-08: independent Rust CR3 lossless and version 0x100/0x200 C-RAW decoding, six exact full-sensor regressions on M50/R100/R8; any CR3 decode error falls back to the embedded JPEG; guarded CR3 camera-JPEG colour fitting with independent framing validation (`docs/cr3.md`). Unverified CRX variants and broader model verification remain.
- 2026-10-09: Sony ILCE-7CR camera colour: bundled profile from 125 photos. On 26 withheld photos mean ΔE vs the camera JPEG 4.07 → 3.29; all 131 photos accepted, where 8 fell back to the neutral matrix without it. Measured calibration remains open.
## Chinese and Japanese interfaces, and text watermarks
English, Simplified Chinese, Traditional Chinese (Taiwan) and Japanese interface languages are persisted in UI state, and the
language table (`crates/ui-egui/src/i18n.rs`) drives the menus, settings and fonts, so another
language is a table entry plus two catalogs (docs/localization.md). Core menus and panels are
translated; untranslated text retains English. Ukrainian covers all existing catalog keys, command/control/rule labels
and format messages, with count-neutral labels and numeric dates; technical errors from lower layers remain English.
CJK glyphs (Chinese: Noto Sans CJK SC; Japanese UI:
BIZ UDPGothic; watermarks: BIZ UDMincho) come from storytold/craft-fonts, embedded by builds made
with the optional `CRAFT_FONTS_DIR` input (all releases), so no system fonts are needed. Text watermarks now accept
`vertical: true` in export JSON/presets and expose an orientation selector. Japanese
characters stay upright in top-to-bottom columns, with newlines starting columns to the
left. This is basic lettering, without tate-chu-yoko, ruby, kinsoku, or general vertical
OpenType shaping. The same coverage renderer serves 8/16/32-bit exports.
| Date | Change | Summary |
|---|---|---|
| 2026-10-10 | minor | RAW measured model by model; RAW decode and colour as explicit rows: full 59% → 61%, mainstream 51% → 49%, essentials 63% → 64%; beta ~14 points |
| 2026-10-10 | minor | Full ready restored to the additive weighted sum (~59%; method aligned with the standard, no new evidence); hours per audience; beta ~16 points away |
| 2026-10-10 | minor | Mainstream practitioner and essentials user numbers; full ready 55% → 46% (method written down); beta 340–600 h |
| 2026-10-10 | minor | Alpha gate added (core-workflow check): passes, stays alpha |
| 2026-10-10 | major | Restructured to the progress-docs standard; re-measured; *Where we stand*, *Parity estimate* and *Totals* moved to docs/target-app-parity.md, milestones and risks to docs/roadmap.md, raw coverage to docs/raw-parity.md, catalog migration to docs/file-format-parity.md, CJK notes to docs/localization-parity.md |
| 2026-10-08 | minor | *Where we stand* by dimension and by kind of user |
| 2026-10-02 | major | Parity estimate and effort table |
| 2026-09-30 | major | Created |
assets/ATTRIBUTION.md
+1
View File
@@ -14,6 +14,7 @@ covers a whole directory), or if a licence file referenced in the Licence column
| `packaging/macos/dmg/` (`background.svg`, the `background.tiff` rendered from it by `generate.py`, and the `dmg-layout.DS_Store` layout file) | LightCraft macOS DMG window background | @XusBadia | original work for this repository, derived from the LightCraft app icon (`assets/app-icon/lightcraft.svg`, referenced via `<image>`, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0 | 2026-10-08 | none (original); used for the Finder window of the macOS DMG, `packaging/macos/dmg/generate.py` renders it to `background.tiff` |
| `assets/camera-profiles/ILCE-7CR.json` | Sony ILCE-7CR camera colour profile (matrix + hue/saturation/value table), built into LightCraft | LightCraft contributors; source photographs supplied by @AlexChambers | original work: fitted by `lightcraft-cli calibrate` to 125 usable ILCE-7CR ARWs (of 131, September 2025 to September 2026, Standard creative style) and their in-camera JPEGs; aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-09 | none |
| `assets/camera-profiles/X-H2S.json`, `assets/camera-profiles/X-T4.json` | Fujifilm camera colour profiles (matrix + hue/saturation/value table), built into LightCraft | LightCraft contributors; source photographs supplied by Sebastian Pfluegelmeier | original work: fitted by `lightcraft-cli calibrate` to 201 X-H2S and 545 X-T4 RAFs and their in-camera JPEGs; aggregate colour statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 | 2026-10-08 | none |
| `crates/raw/src/spectral_table.rs` | Camera colour matrices (DNG ColorMatrix1/2 and ForwardMatrix1/2 for Standard Illuminant A and D65) for 52 camera models, and the model alias list | Derived by LightCraft contributors from rawtoaces-data (Academy Software Foundation): camera spectral sensitivities measured by Weta Digital (44 cameras), scitech (7) and Jonathon Irons at the National Physical Laboratory (Nikon D5100); training reflectances; CIE 1931 colour-matching functions (CIE, CC-BY-4.0, as shipped in rawtoaces-data) | https://github.com/AcademySoftwareFoundation/rawtoaces-data, tag v1.1.0, commit 5cfa53707b053594b2d0d48e8019fb56e6582d6c | Apache-2.0 (`LICENSE-APACHE`); the colour-matching functions CC-BY-4.0 | 2026-10-08 | generated by `tools/spectral_camera_table.py`: a 3×3 matrix per illuminant fitted to the spectral data (no measurement copied verbatim); aliases copied from `data/aliases.json`; no Adobe or third-party camera matrix |
| `assets/camera-tone/sony-dro-off.json` | Sony global tone curve with the Dynamic Range Optimizer off (32 knots), built into LightCraft for ARWs shot with DRO on | LightCraft contributors; source photographs by raw.pixls.us contributors | original work: tone curve quantile-matched over the pooled pixel pairs of 12 CC0 raw.pixls.us ARWs (DRO off, Standard creative style; ILCE-3500, -5000, -6600, -7CR, -7M3 ×3, -7S, -9M2, ILCA-77M2, SLT-A57, NEX-F3) and their embedded camera JPEGs; the file lists each source URL and SHA-256 and how the curve was derived (see also `docs/camera-preview-colour.md`); aggregate tone statistics only, no image content; no Adobe or third-party profile data | MIT OR Apache-2.0 (sources CC0) | 2026-10-09 | none |
| `crates/scenes/` (generated images) | Procedural demo photographs | LightCraft contributors | original work (generated at runtime, no source imagery) | MIT OR Apache-2.0 | 2026-09-30 | n/a |
| `crates/ui-egui/src/icons.rs` | UI icons drawn as vector paths in code | LightCraft contributors | original work | MIT OR Apache-2.0 | 2026-09-30 | n/a |
crates/engine/src/activity.rs
+391
View File
@@ -0,0 +1,391 @@
//! Background tasks in flight (issue #345): imports, exports, preview builds, downloads, the face scan… Each one
//! registers here while it runs, so every frontend can show them in one place (the desktop app's activity stack) and
//! cancel the ones that can be cancelled (`activity.list`, `activity.cancel`).
//!
//! A job holds a [`TaskGuard`] for as long as it runs; dropping it — when the job ends, fails or its thread unwinds
//! from a panic — removes the row, so a dead worker never leaves a stuck bar. Jobs keep the cancel flag they already
//! check between files or photos: the guard adopts it ([`Cancel::Flag`]), so the stack's ✕, `activity.cancel` and the
//! job's own Cancel all set the same atomic. Numbers are worded by the frontend from `done`, `total` and [`Unit`]; a
//! detail set here is a name (a file, a model), never a sentence, so the frontend can translate everything else.
use std::sync::atomic::{AtomicBool, AtomicU8, AtomicU64, Ordering};
use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
use serde::Serialize;
/// The longest label or detail kept, in characters (a file name can be anything).
const MAX_TEXT: usize = 200;
/// The background tasks in flight. Cheap to clone (one `Arc`), so worker threads can hold it.
#[derive(Clone, Default)]
pub struct Activity {
inner: Arc<Inner>,
}
#[derive(Default)]
struct Inner {
/// Oldest first.
tasks: Mutex<Vec<Arc<Entry>>>,
next: AtomicU64,
}
/// How a task can be cancelled.
pub enum Cancel {
/// Not at all (no ✕): the face scan, AI Denoise.
No,
/// With a flag of its own ([`TaskGuard::cancel_flag`]).
Yes,
/// With the job's existing flag.
Flag(Arc<AtomicBool>),
}
/// How the frontend words `done` / `total`: "3 of 25", "12 of 340 MB", "40 %".
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize)]
#[serde(rename_all = "lowercase")]
pub enum Unit {
#[default]
Count,
Bytes,
Percent,
}
impl Unit {
fn from_u8(v: u8) -> Unit {
match v {
1 => Unit::Bytes,
2 => Unit::Percent,
_ => Unit::Count,
}
}
fn to_u8(self) -> u8 {
match self {
Unit::Count => 0,
Unit::Bytes => 1,
Unit::Percent => 2,
}
}
}
/// One task as the frontends see it (`activity.list`).
#[derive(Clone, Debug, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct TaskInfo {
pub id: u64,
/// A stable id: `import`, `export`, `scan`, `previews`, `smartPreviews`, `lightroom`, `merge`, `download`,
/// `faces`, `denoise`, `findMissing`.
pub kind: &'static str,
/// English; the frontend translates it.
pub label: String,
pub done: u64,
/// 0 while the amount of work isn't known yet (an indeterminate bar).
pub total: u64,
pub unit: Unit,
/// The file, photo or model being worked on, if any.
pub detail: String,
pub cancellable: bool,
/// Cancel was asked for; the job stops at its next check.
pub cancelling: bool,
pub age_ms: u64,
}
struct Entry {
id: u64,
kind: &'static str,
label: String,
started: web_time::Instant,
done: AtomicU64,
total: AtomicU64,
unit: AtomicU8,
detail: Mutex<String>,
cancel: Option<Arc<AtomicBool>>,
cancellable: AtomicBool,
}
fn lock<T>(m: &Mutex<T>) -> MutexGuard<'_, T> {
m.lock().unwrap_or_else(PoisonError::into_inner)
}
fn capped(text: &str) -> String {
text.chars().take(MAX_TEXT).collect()
}
impl Entry {
fn is_cancellable(&self) -> bool {
self.cancel.is_some() && self.cancellable.load(Ordering::Relaxed)
}
fn is_cancelled(&self) -> bool {
self.cancel.as_ref().is_some_and(|c| c.load(Ordering::Relaxed))
}
fn progress(&self, done: u64, total: u64) {
self.total.store(total, Ordering::Relaxed);
self.done.store(if total > 0 { done.min(total) } else { done }, Ordering::Relaxed);
}
fn detail(&self, text: &str) {
*lock(&self.detail) = capped(text);
}
fn info(&self) -> TaskInfo {
TaskInfo {
id: self.id,
kind: self.kind,
label: self.label.clone(),
done: self.done.load(Ordering::Relaxed),
total: self.total.load(Ordering::Relaxed),
unit: Unit::from_u8(self.unit.load(Ordering::Relaxed)),
detail: lock(&self.detail).clone(),
cancellable: self.is_cancellable(),
cancelling: self.is_cancelled(),
age_ms: u64::try_from(self.started.elapsed().as_millis()).unwrap_or(u64::MAX),
}
}
}
impl Activity {
/// Register a task; it is listed until the returned guard is dropped. `label` is English (the frontend
/// translates it).
pub fn start(&self, kind: &'static str, label: &str, cancel: Cancel) -> TaskGuard {
let id = self.inner.next.fetch_add(1, Ordering::Relaxed).saturating_add(1);
let cancel = match cancel {
Cancel::No => None,
Cancel::Yes => Some(Arc::new(AtomicBool::new(false))),
Cancel::Flag(flag) => Some(flag),
};
let entry = Arc::new(Entry {
id,
kind,
label: capped(label),
started: web_time::Instant::now(),
done: AtomicU64::new(0),
total: AtomicU64::new(0),
unit: AtomicU8::new(Unit::Count.to_u8()),
detail: Mutex::new(String::new()),
cancellable: AtomicBool::new(cancel.is_some()),
cancel,
});
lock(&self.inner.tasks).push(entry.clone());
TaskGuard { activity: self.clone(), entry }
}
/// Every task in flight, oldest first.
pub fn list(&self) -> Vec<TaskInfo> {
lock(&self.inner.tasks).iter().map(|e| e.info()).collect()
}
/// Ask task `id` to stop (it stops at its next check). An unknown id (the task may have just ended) or a task
/// that can't be cancelled is an error.
pub fn cancel(&self, id: u64) -> Result<(), String> {
let tasks = lock(&self.inner.tasks);
let entry = tasks.iter().find(|e| e.id == id).ok_or_else(|| format!("no task {id}"))?;
match (&entry.cancel, entry.is_cancellable()) {
(Some(flag), true) => {
flag.store(true, Ordering::Relaxed);
Ok(())
}
_ => Err(format!("{} can't be cancelled", entry.label)),
}
}
/// Ask every cancellable task to stop; returns how many were asked.
pub fn cancel_all(&self) -> usize {
let tasks = lock(&self.inner.tasks);
let mut n = 0;
for e in tasks.iter().filter(|e| e.is_cancellable()) {
if let Some(flag) = &e.cancel {
flag.store(true, Ordering::Relaxed);
n += 1;
}
}
n
}
/// Tasks that quitting would cut short: cancellable ones not already stopping.
pub fn running_cancellable(&self) -> Vec<TaskInfo> {
lock(&self.inner.tasks).iter().filter(|e| e.is_cancellable() && !e.is_cancelled()).map(|e| e.info()).collect()
}
}
/// A task's place in the list, held by whoever owns the job; dropping it removes the row.
pub struct TaskGuard {
activity: Activity,
entry: Arc<Entry>,
}
impl TaskGuard {
pub fn id(&self) -> u64 {
self.entry.id
}
/// `done` of `total` (`total` 0 = not known yet). `done` is kept at most `total`.
pub fn progress(&self, done: u64, total: u64) {
self.entry.progress(done, total);
}
/// The file, photo or model being worked on (a name, not a sentence).
pub fn detail(&self, text: &str) {
self.entry.detail(text);
}
pub fn set_unit(&self, unit: Unit) {
self.entry.unit.store(unit.to_u8(), Ordering::Relaxed);
}
/// A step that must not be cut short (the Lightroom import's commit) turns cancelling off for its duration.
pub fn set_cancellable(&self, yes: bool) {
self.entry.cancellable.store(yes, Ordering::Relaxed);
}
pub fn cancel_flag(&self) -> Option<Arc<AtomicBool>> {
self.entry.cancel.clone()
}
pub fn is_cancelled(&self) -> bool {
self.entry.is_cancelled()
}
/// For progress from another thread; it does not keep the row.
pub fn handle(&self) -> TaskHandle {
TaskHandle { entry: self.entry.clone() }
}
}
impl Drop for TaskGuard {
fn drop(&mut self) {
let id = self.entry.id;
lock(&self.activity.inner.tasks).retain(|e| e.id != id);
}
}
impl std::fmt::Debug for TaskGuard {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("TaskGuard").field("id", &self.entry.id).field("kind", &self.entry.kind).finish()
}
}
/// Progress for a task from another thread (see [`TaskGuard::handle`]).
#[derive(Clone)]
pub struct TaskHandle {
entry: Arc<Entry>,
}
impl TaskHandle {
pub fn progress(&self, done: u64, total: u64) {
self.entry.progress(done, total);
}
pub fn detail(&self, text: &str) {
self.entry.detail(text);
}
pub fn is_cancelled(&self) -> bool {
self.entry.is_cancelled()
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
#[test]
fn guard_drop_removes_the_row() {
let a = Activity::default();
let g = a.start("export", "Exporting", Cancel::Yes);
assert_eq!(a.list().len(), 1);
drop(g);
assert!(a.list().is_empty());
}
#[test]
fn guard_drop_on_panic_unwind_removes_the_row() {
let a = Activity::default();
let a2 = a.clone();
let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(move || {
let _g = a2.start("import", "Importing", Cancel::Yes);
panic!("synthetic worker panic");
}));
assert!(r.is_err());
assert!(a.list().is_empty());
}
#[test]
fn cancel_sets_the_adopted_flag() {
let a = Activity::default();
let flag = Arc::new(AtomicBool::new(false));
let g = a.start("export", "Exporting", Cancel::Flag(flag.clone()));
a.cancel(g.id()).unwrap();
assert!(flag.load(Ordering::Relaxed) && g.is_cancelled());
assert!(a.list()[0].cancelling);
}
#[test]
fn cancel_errors_instead_of_panicking() {
let a = Activity::default();
assert!(a.cancel(42).is_err());
let g = a.start("faces", "Finding faces", Cancel::No);
assert!(a.cancel(g.id()).is_err());
let h = a.start("lightroom", "Importing Lightroom catalog", Cancel::Yes);
h.set_cancellable(false);
assert!(a.cancel(h.id()).is_err());
assert!(!h.is_cancelled());
}
#[test]
fn cancel_all_counts_only_cancellable() {
let a = Activity::default();
let (_x, _y, _z) = (a.start("export", "E", Cancel::Yes), a.start("import", "I", Cancel::Yes), a.start("faces", "F", Cancel::No));
assert_eq!(a.cancel_all(), 2);
assert!(a.running_cancellable().is_empty());
}
#[test]
fn progress_is_clamped_and_detail_capped() {
let a = Activity::default();
let g = a.start("export", "E", Cancel::Yes);
g.progress(12, 10);
g.detail(&"é".repeat(300));
let t = &a.list()[0];
assert_eq!((t.done, t.total), (10, 10));
assert_eq!(t.detail.chars().count(), 200);
g.progress(5, 0); // indeterminate keeps done as given
assert_eq!(a.list()[0].total, 0);
assert_eq!(a.list()[0].unit, Unit::Count);
g.set_unit(Unit::Bytes);
assert_eq!(a.list()[0].unit, Unit::Bytes);
}
#[test]
fn ids_are_unique_and_list_is_oldest_first() {
let a = Activity::default();
let (g1, g2) = (a.start("export", "A", Cancel::Yes), a.start("import", "B", Cancel::Yes));
assert!(g2.id() > g1.id());
assert_eq!(a.list().iter().map(|t| t.id).collect::<Vec<_>>(), vec![g1.id(), g2.id()]);
}
#[test]
fn poisoned_lock_still_works() {
let a = Activity::default();
let a2 = a.clone();
let _ = std::thread::spawn(move || {
let _held = a2.inner.tasks.lock().unwrap();
panic!("synthetic panic while holding the task list");
})
.join();
assert!(a.inner.tasks.is_poisoned());
let _g = a.start("export", "E", Cancel::Yes);
assert_eq!(a.list().len(), 1);
}
#[test]
fn handle_updates_from_another_thread() {
let a = Activity::default();
let g = a.start("previews", "Building previews", Cancel::Yes);
let h = g.handle();
std::thread::spawn(move || h.progress(3, 7)).join().unwrap();
assert_eq!((a.list()[0].done, a.list()[0].total), (3, 7));
}
}
crates/engine/src/camera_preview.rs
+175 -1
View File
@@ -4,7 +4,7 @@
//! A global matrix can't follow the camera's hue-dependent rendering (the best matrix rendered a
//! lime shirt olive that the camera kept lime): a hue/saturation table fitted to the residuals
//! (applied like a DNG `ProfileHueSatMap`) corrects that when it also improves the held-out pixels.
use lightcraft_color::{D50, D65, Mat3, PROPHOTO, REC2020, bradford, luminance_2020};
use lightcraft_color::{D50, D65, Mat3, PROPHOTO, REC2020, Xy, bradford, luminance_2020};
use lightcraft_pipeline::tone::{CameraTone, ToneMap};
use lightcraft_raster::{
Rgb32f,
@@ -12,6 +12,19 @@ use lightcraft_raster::{
};
use lightcraft_raw::{RawFormat, RawImage, color::CameraTransform, profile::HsvTable};
/// The version of the camera-look fit: how a raw's colour matrix, hue/saturation table and tone
/// and chroma curves are chosen. Smart previews record it in their header (`look_version`), because
/// they keep the curve and the colour the fit produced; one written by an older version is rebuilt
/// from its original when that is online, and kept as it is while it is offline.
///
/// **Bump this whenever a change makes the fit give a different result for the same file** (the
/// fit, its gates, the fallbacks, the camera profiles' effect on it). Do not bump it for changes
/// that leave the fitted look alone. `RENDER_CACHE_VERSION` covers renders; this covers proxies.
///
/// 1: the fit as of #499's follow-up; 2: Sony DRO (tone curve lowered to Sony's curve without DRO)
/// and the ILCE-7CR profile (#528, #583, #568, #616).
pub const LOOK_VERSION: u32 = 2;
#[derive(Clone, Debug)]
pub(crate) struct CameraLook {
pub matrix: Mat3,
@@ -46,6 +59,43 @@ pub(crate) fn file_local_look(format: RawFormat) -> bool {
)
}
/// The colour a raw starts from, first available first: the file's own colour matrices (DNG); a look fitted to the
/// file's JPEG ([`fit_preview`]: with the camera profile for its model when there is one, else with colour fitted to
/// this photo alone); the model's colour matrices from measured spectral sensitivities (`lightcraft_raw::spectral`);
/// the neutral fallback. The spectral matrices only stand in for the neutral fallback, so a photo whose JPEG fit is
/// accepted renders as without them; when used they are written into `raw.color`. Returns the as-shot white, the
/// camera transform at it and the fitted look.
pub(crate) fn starting_colour(raw: &mut RawImage, bytes: &[u8]) -> (Xy, CameraTransform, Option<CameraLook>) {
starting_colour_with(raw, |raw, t| fit_preview(raw, bytes, t))
}
/// [`starting_colour`] with the JPEG fit passed in (tests pin the order with it).
fn starting_colour_with(
raw: &mut RawImage,
fit: impl FnOnce(&RawImage, &CameraTransform) -> Option<CameraLook>,
) -> (Xy, CameraTransform, Option<CameraLook>) {
let start = |raw: &RawImage| {
let xy = lightcraft_raw::color::as_shot_white_xy(raw);
(xy, lightcraft_raw::color::camera_transform(raw, xy))
};
let (xy, t) = start(raw);
if !t.matrix_is_fallback {
return (xy, t, None);
}
if let Some(look) = fit(raw, &t) {
return (xy, t, Some(look));
}
let Some(camera) = raw.metadata.model.as_deref().and_then(|model| lightcraft_raw::spectral::find(raw.metadata.make.as_deref(), model)) else {
return (xy, t, None);
};
camera.fill(&mut raw.color);
if lightcraft_pipeline::profiling() {
eprintln!("[profile] {:?} camera colour from spectral sensitivities: {} {}", raw.format, camera.make, camera.model);
}
let (xy, t) = start(raw);
(xy, t, None)
}
pub(crate) fn fit_preview(raw: &RawImage, bytes: &[u8], transform: &CameraTransform) -> Option<CameraLook> {
if !transform.matrix_is_fallback || !file_local_look(raw.format) {
return None;
@@ -1389,6 +1439,130 @@ mod tests {
assert!(fit_hue_sat(&[], &Mat3::IDENTITY).is_none(), "too few samples");
}
fn raw_of(format: RawFormat, make: &str, model: &str, color: lightcraft_raw::ColorData) -> RawImage {
let area = lightcraft_raw::Rect::new(0, 0, 2, 2);
RawImage {
format,
width: 2,
height: 2,
cpp: 1,
data: lightcraft_raw::RawData::U16(vec![0; 4]),
cfa: None,
bits: 14,
black: Default::default(),
white: vec![16383.0],
active_area: area,
crop: area,
orientation: Default::default(),
color,
wb_multipliers: Some([2.0, 1.0, 1.5]),
linearized: false,
opcodes: Default::default(),
metadata: lightcraft_raw::Metadata { make: Some(make.to_owned()), model: Some(model.to_owned()), ..Default::default() },
}
}
/// Step 1: a file's own colour matrices win over everything, including a spectral row for its model; no JPEG fit
/// is tried.
#[test]
fn a_files_own_matrices_come_first() {
let own = Mat3([[0.6, -0.1, -0.05], [-0.4, 1.3, 0.1], [-0.1, 0.2, 0.7]]);
let color = lightcraft_raw::ColorData { illuminant: [21, 0], color_matrix: [Some(own), None], ..Default::default() };
let mut raw = raw_of(RawFormat::Cr2, "Canon", "Canon EOS 600D", color.clone());
let (_, t, look) = starting_colour_with(&mut raw, |_, _| panic!("no JPEG fit for a raw with its own matrices"));
assert_eq!(raw.color, color);
assert!(!t.matrix_is_fallback && look.is_none());
}
/// Steps 2 and 3 before step 4: an accepted JPEG fit (with or without a camera profile) is kept for a model that
/// has spectral matrices, and the spectral matrices are not written. Photos that had a fitted look before the
/// spectral table render exactly as they did.
#[test]
fn an_accepted_jpeg_fit_comes_before_the_spectral_matrices() {
for (make, model) in [("SONY", "ILCE-7M4"), ("Canon", "Canon EOS 600D")] {
assert!(lightcraft_raw::spectral::find(Some(make), model).is_some(), "{model}");
let mut raw = raw_of(RawFormat::Cr2, make, model, Default::default());
let fitted = dro_test_look(&std::array::from_fn(|i| 0.006 * 1.13f32.powi(i as i32)), &|x| x.sqrt());
let (xy, t, look) = starting_colour_with(&mut raw, |raw, t| {
assert!(t.matrix_is_fallback && !lightcraft_raw::color::has_matrix(&raw.color), "the fit sees the raw without spectral matrices");
Some(fitted.clone())
});
assert_eq!(look.map(|l| l.matrix), Some(fitted.matrix), "{model}");
assert_eq!(raw.color, lightcraft_raw::ColorData::default(), "{model}: spectral matrices left out");
assert!(t.matrix_is_fallback, "{model}");
assert_eq!(xy, lightcraft_raw::color::as_shot_white_xy(&raw));
}
}
/// Step 4: when the JPEG fit fails (here: no camera JPEG at all), a model with a spectral row takes its matrices,
/// whether or not it has a camera profile (the ILCE-7M4 has a bundled one).
#[test]
fn a_failed_jpeg_fit_falls_back_to_the_spectral_matrices() {
assert!(crate::camera_profiles::get("ILCE-7M4").is_some());
let camera = lightcraft_raw::spectral::find(Some("SONY"), "ILCE-7M4").unwrap();
let mut raw = raw_of(RawFormat::Arw, "SONY", "ILCE-7M4", Default::default());
let (_, t, look) = starting_colour(&mut raw, &[]);
assert!(look.is_none());
assert_eq!(raw.color.color_matrix, camera.color_matrix.map(Some));
assert!(!t.matrix_is_fallback);
let camera = lightcraft_raw::spectral::find(Some("Canon"), "Canon EOS 600D").unwrap();
let mut raw = raw_of(RawFormat::Cr2, "Canon", "Canon EOS 600D", Default::default());
let (xy, t, look) = starting_colour_with(&mut raw, |_, _| None);
assert!(look.is_none());
assert_eq!(raw.color.illuminant, [17, 21]);
assert_eq!(raw.color.forward_matrix, camera.forward_matrix.map(Some));
assert!(!t.matrix_is_fallback);
assert_eq!(xy, lightcraft_raw::color::as_shot_white_xy(&raw));
// white-balanced white stays white
let w = t.matrix.apply([1.0; 3]);
assert!(w.iter().all(|v| (v - 1.0).abs() < 1e-4), "{w:?}");
}
/// Step 5: a camera outside the table whose fit fails keeps the neutral fallback.
#[test]
fn a_camera_outside_the_table_keeps_the_fallback() {
let mut raw = raw_of(RawFormat::Cr2, "Canon", "Canon EOS 7D", Default::default());
let (_, t, look) = starting_colour(&mut raw, &[]);
assert!(look.is_none() && t.matrix_is_fallback);
assert_eq!(raw.color, lightcraft_raw::ColorData::default());
// without a model there is no table row
let mut raw = raw_of(RawFormat::Cr2, "Canon", "", Default::default());
raw.metadata.model = None;
assert!(starting_colour(&mut raw, &[]).1.matrix_is_fallback);
}
/// The colour precedence on public raws (each skipped without its corpus file): a photo whose JPEG fit is
/// accepted keeps it and its colour data; one whose fit fails takes its spectral matrices when its model has
/// them; a DNG keeps its own matrices. White balance stays relative to the as-shot look for every raw without
/// matrices of its own.
#[test]
fn corpus_colour_precedence() {
let dir = std::env::var_os("LIGHTCRAFT_CORPUS")
.map(std::path::PathBuf::from)
.unwrap_or_else(|| std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
.join("raw");
for name in ["arw-sony-a7m4-14bit.arw", "cr2-canon-5d3.cr2", "nef-nikon-d5100-lossless.nef", "cr2-canon-7d.cr2", "dng-canon-5d3-16bit.dng"] {
let Ok(bytes) = std::fs::read(dir.join(name)) else {
eprintln!("skip: {name} absent");
continue;
};
let mut raw = lightcraft_raw::decode(&bytes).unwrap();
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
let own = lightcraft_raw::color::has_matrix(&raw.color);
let covered = raw.metadata.model.as_deref().and_then(|m| lightcraft_raw::spectral::find(raw.metadata.make.as_deref(), m)).is_some();
let before = raw.color.clone();
let (_, t, fitted) = starting_colour(&mut raw, &bytes);
let spectral = !own && fitted.is_none() && covered;
eprintln!("{name}: own matrices {own}, JPEG fit {}, spectral {spectral}", fitted.is_some());
assert_eq!(raw.color != before, spectral, "{name}");
assert_eq!(t.matrix_is_fallback, !own && !spectral, "{name}");
let (_, info) = crate::files::load_bytes(&bytes, 400).unwrap();
assert_eq!(info.camera_tone.is_some(), fitted.is_some(), "{name}");
assert_eq!(info.relative_wb, !own, "{name}");
}
}
#[test]
fn supported_raws_get_a_file_local_look() {
// every decoded raw without a colour matrix of its own (issue #310: CR2 rendered flat with the fallback)
crates/engine/src/cmd/activity.rs
+65
View File
@@ -0,0 +1,65 @@
//! `activity.list` / `activity.cancel`: the background tasks in flight (see [`crate::activity`]).
use serde_json::{Value, json};
use super::{CommandSpec, always, bad, cmd};
pub fn specs() -> Vec<CommandSpec> {
vec![
cmd!(
query "activity.list",
"Background Tasks",
[],
None,
"{} → {tasks: [{id, kind, label, done, total, unit: count|bytes|percent, detail, cancellable, cancelling, ageMs}]} — the long-running tasks in flight (imports, exports, preview builds, downloads, the face scan…), oldest first; total 0 = not known yet",
always,
|s, _| Ok(json!({"tasks": s.activity.list()}))
),
cmd!(
"activity.cancel",
"Cancel Background Task",
[],
None,
"{id} | {all: true} → {cancelled: n} — ask a task (or every cancellable task) to stop; it stops at its next file or photo. An unknown id (it may have just finished) or a task that can't be cancelled is an error",
always,
|s, p| {
const C: &str = "activity.cancel";
if p.get("all").and_then(Value::as_bool) == Some(true) {
return Ok(json!({"cancelled": s.activity.cancel_all()}));
}
let id = p.get("id").and_then(Value::as_u64).ok_or_else(|| bad(C, "id or all"))?;
s.activity.cancel(id).map_err(|e| bad(C, e))?;
Ok(json!({"cancelled": 1}))
}
),
]
}
#[cfg(test)]
mod tests {
use serde_json::json;
use crate::Session;
use crate::activity::Cancel;
#[test]
fn list_and_cancel_through_execute() {
let mut s = Session::new();
let g = s.activity.start("export", "Exporting", Cancel::Yes);
let r = s.execute("activity.list", &json!({})).unwrap();
assert_eq!(r["tasks"][0]["kind"], "export");
assert_eq!(r["tasks"][0]["cancellable"], true);
let r = s.execute("activity.cancel", &json!({"id": g.id()})).unwrap();
assert_eq!(r["cancelled"], 1);
assert!(g.is_cancelled());
}
#[test]
fn cancel_all_and_errors() {
let mut s = Session::new();
let (_a, _b) = (s.activity.start("export", "E", Cancel::Yes), s.activity.start("faces", "F", Cancel::No));
assert_eq!(s.execute("activity.cancel", &json!({"all": true})).unwrap()["cancelled"], 1);
assert!(s.execute("activity.cancel", &json!({"id": 999})).is_err());
assert!(s.execute("activity.cancel", &json!({})).is_err());
}
}
crates/engine/src/cmd/denoise.rs
+2 -1
View File
@@ -368,7 +368,7 @@ fn download(s: &mut Session, p: &Value) -> Result<Value> {
return Err(bad(C, "that model is already installed"));
}
let host = lightcraft_denoise::known::host(&spec.url).to_string();
s.denoise.downloads.start(spec, &dir).map_err(|e| bad(C, e))?;
s.denoise.downloads.start(spec, &dir, &s.activity, &k.manifest.name).map_err(|e| bad(C, e))?;
Ok(json!({"started": id, "from": host}))
}
@@ -614,6 +614,7 @@ fn status(s: &mut Session, p: &Value) -> Result<Value> {
/// `denoise.pump {pace?}`: cheap to call every frame. Takes in what the background job has finished and keeps one going.
fn pump(s: &mut Session, p: &Value) -> Result<Value> {
s.denoise_pump(Pace::parse(p.get("pace").and_then(Value::as_str)));
s.denoise_track();
Ok(json!({
"active": s.denoise.active.is_some(),
"running": s.denoise_running_json(),
crates/engine/src/cmd/face_models.rs
+7 -1
View File
@@ -356,7 +356,13 @@ fn download(s: &mut Session, p: &Value) -> Result<Value> {
return Err(bad(C, "that model is already installed"));
}
let host = spec.host().to_string();
s.face_downloads.start(spec, &dir).map_err(|e| bad(C, e))?;
let name = known::all()
.into_iter()
.find(|m| m.id == spec.id)
.map(|m| m.name)
.or_else(|| s.face_catalog.entries.iter().find(|e| e.manifest.id == spec.id).map(|e| e.manifest.name.clone()))
.unwrap_or_else(|| spec.id.clone());
s.face_downloads.start(spec, &dir, &s.activity, &name).map_err(|e| bad(C, e))?;
Ok(json!({"started": id, "from": host}))
}
crates/engine/src/cmd/face_recognize.rs
+22 -1
View File
@@ -515,6 +515,21 @@ mod imp {
}))
}
/// The scan's row in the activity stack: there while `pending` photos are left, its whole the most that were left at
/// once since the scan last finished (which it returns; 0 when none are left). It can't be stopped from there: the
/// scan is switched off in Settings.
fn track(s: &mut Session, pending: u64) -> u64 {
if pending == 0 {
s.faces.task = None;
s.faces.peak = 0;
return 0;
}
s.faces.peak = s.faces.peak.max(pending);
let task = s.faces.task.get_or_insert_with(|| s.activity.start("faces", "Finding faces", crate::activity::Cancel::No));
task.progress(s.faces.peak.saturating_sub(pending), s.faces.peak);
s.faces.peak
}
/// `faces.pump`: cheap to call every frame. Takes in what the background worker has finished and, when it is idle,
/// hands it the next photo with faces to embed. Does nothing (and says so) until recognition is switched on in
/// Settings with a model chosen.
@@ -528,14 +543,17 @@ mod imp {
finish_downloads(s);
let enabled = s.face_models_dir.clone().is_some_and(|d| enabled_cached(s, &d));
if !enabled {
track(s, 0);
return Ok(json!({"active": false}));
}
if s.faces.retry_at.is_some_and(|t| Instant::now() < t) {
track(s, 0);
return Ok(json!({"active": false}));
}
let Ok(embedder) = current_with(s, C, false) else {
// the chosen model is gone or broken: look again in a few seconds, not every frame
s.faces.retry_at = Some(Instant::now() + Duration::from_secs(3));
track(s, 0);
return Ok(json!({"active": false}));
};
s.faces.retry_at = None;
@@ -573,10 +591,13 @@ mod imp {
break;
}
}
let pending = s.faces.queue.len() + s.faces.in_flight.len();
let peak = track(s, pending as u64);
Ok(json!({
"active": true,
"inFlight": s.faces.in_flight.len(),
"pendingPhotos": s.faces.queue.len() + s.faces.in_flight.len(),
"pendingPhotos": pending,
"peak": peak,
"indexedFaces": s.faces.index.len(),
"searchedPhotos": s.faces.scanned.len(),
"workers": target,
crates/engine/src/cmd/masks.rs
+1 -1
View File
@@ -464,7 +464,7 @@ pub fn specs() -> Vec<CommandSpec> {
),
));
}
let started = s.segmenter.start_download().map_err(ai_err)?;
let started = s.segmenter.start_download(&s.activity).map_err(ai_err)?;
Ok(json!({"started": started, "installed": s.segmenter.installed(), "downloading": s.segmenter.download_status().running}))
}
),
crates/engine/src/cmd/mod.rs
+2
View File
@@ -4,6 +4,7 @@
//! delete), `album.*`, `develop.*` (settings), `crop.*`, `mask.*`, `preset.*`, `version.*`,
//! `edit.*` (undo/redo), and queries (`catalog.query`, `photo.inspect`, `develop.get`…).
mod activity;
mod before;
pub(crate) mod browse;
mod color;
@@ -154,6 +155,7 @@ pub fn command_specs() -> &'static [CommandSpec] {
v.extend(face_models::specs());
v.extend(face_detect::specs());
v.extend(face_recognize::specs());
v.extend(activity::specs());
v
})
}
crates/engine/src/cmd/previews.rs
+128 -23
View File
@@ -1,7 +1,7 @@
//! Build previews ahead of time: each photo's grid thumbnail and its loupe view (standard size,
//! or 1:1), rendered into the memory + disk preview cache so browsing and the loupe are instant.
//! Runs on a background thread (the app keeps working; progress / cancel by command), or inline
//! with `wait` (CLI, MCP, tests).
//! Runs on a background thread (the app keeps working; progress / cancel by command, and a row in
//! the activity stack, issue #345), or inline with `wait` (CLI, MCP, tests).
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
@@ -9,6 +9,7 @@ use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use serde_json::{Value, json};
use super::{CommandSpec, always, bad, cmd, str_param};
use crate::activity::{Cancel, TaskGuard};
use crate::media::{RenderJob, THUMB_SIZES};
use crate::{Result, Session};
@@ -21,7 +22,8 @@ pub struct PreviewBuild {
pub total: usize,
pub done: AtomicUsize,
pub failed: AtomicUsize,
pub cancel: AtomicBool,
/// Shared with the build's row in the activity stack (its ✕ sets it).
pub cancel: Arc<AtomicBool>,
pub finished: AtomicBool,
/// Damaged smart previews rebuilt (Build Smart Previews only).
pub repaired: AtomicUsize,
@@ -62,7 +64,19 @@ fn jobs_for(s: &mut Session, id: lightcraft_catalog::PhotoId, edge: Option<usize
v
}
fn run_all(jobs: Vec<Vec<RenderJob>>, state: &PreviewBuild) {
/// The build's row in the activity stack (`what` as in [`PreviewBuild::what`]).
fn task_for(s: &Session, state: &PreviewBuild) -> TaskGuard {
let (kind, label) = match state.what {
"" => ("previews", "Building previews"),
"smart previews" => ("smartPreviews", "Building smart previews"),
_ => ("smartPreviews", "Discarding smart previews"),
};
let task = s.activity.start(kind, label, Cancel::Flag(state.cancel.clone()));
task.progress(0, state.total as u64);
task
}
fn run_all(jobs: Vec<Vec<RenderJob>>, state: &PreviewBuild, task: &TaskGuard) {
for photo_jobs in jobs {
if state.cancel.load(Ordering::Relaxed) {
break;
@@ -71,7 +85,8 @@ fn run_all(jobs: Vec<Vec<RenderJob>>, state: &PreviewBuild) {
if !ok {
state.failed.fetch_add(1, Ordering::Relaxed);
}
state.done.fetch_add(1, Ordering::Relaxed);
let done = state.done.fetch_add(1, Ordering::Relaxed).saturating_add(1);
task.progress(done as u64, state.total as u64);
}
state.finished.store(true, Ordering::Relaxed);
}
@@ -91,26 +106,35 @@ fn build(s: &mut Session, p: &Value) -> Result<Value> {
let jobs: Vec<Vec<RenderJob>> = ids.iter().map(|id| jobs_for(s, *id, edge)).filter(|j| !j.is_empty()).collect();
let state = Arc::new(PreviewBuild { total: jobs.len(), ..Default::default() });
s.preview_build = Some(state.clone());
let task = task_for(s, &state);
let wait = p.get("wait").and_then(Value::as_bool).unwrap_or(false) || cfg!(target_arch = "wasm32");
if wait {
run_all(jobs, &state);
run_all(jobs, &state, &task);
return Ok(state.json());
}
#[cfg(not(target_arch = "wasm32"))]
{
// the row stays while the worker runs (it is dropped with the closure, however that ends)
let st = state.clone();
std::thread::Builder::new()
.name("lc-build-previews".into())
.spawn(move || run_all(jobs, &st))
.spawn(move || run_all(jobs, &st, &task))
.map_err(|e| bad(C, format!("could not start: {e}")))?;
}
Ok(state.json())
}
/// One photo's smart preview to build or discard: (photo, its file in the smart previews folder,
/// how to load the original at preview size).
/// how to load the original at preview size, whether the photo takes the per-file camera look —
/// `camera_preview::file_local_look`, [`lightcraft_catalog::Photo::relative_wb`]: only those
/// proxies go stale when the fit changes).
#[cfg(not(target_arch = "wasm32"))]
type SmartJob = (lightcraft_catalog::PhotoId, std::path::PathBuf, Option<crate::media::SourceRef>);
type SmartJob = (lightcraft_catalog::PhotoId, std::path::PathBuf, Option<crate::media::SourceRef>, bool);
/// Held while one proxy is checked and (re)written, so Build Smart Previews and the opening look
/// refresh ([`refresh_stale_smart_previews`]), on their own threads, never write the same file at once.
#[cfg(not(target_arch = "wasm32"))]
static SMART_WRITE: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// What Build / Discard Smart Previews did.
#[cfg(not(target_arch = "wasm32"))]
@@ -120,11 +144,29 @@ struct SmartCounts {
built: usize,
/// Of those, damaged ones (cut short by a crash or a full drive) that were rebuilt.
repaired: usize,
/// Of those, proxies written by an older camera-look fit that were rebuilt from their online originals.
refreshed: usize,
/// Of those kept as they were: out of date, but the original can't be read now (offline), so
/// the stored look stays and the next run with the original online rebuilds them.
stale_kept: usize,
removed: usize,
/// `[[id, why]]`
failed: Vec<Value>,
}
/// `source` if it reads the photo's original, else `None`: a cached source decoded from a smart
/// preview (it has no decoder facts), a proxy standing in for an offline original, and the proxy
/// fallback of a file source are not the original.
#[cfg(not(target_arch = "wasm32"))]
fn original_only(source: crate::media::SourceRef) -> Option<crate::media::SourceRef> {
use crate::media::SourceRef;
match source {
SourceRef::File { path, max_edge, loader, denoise, .. } => Some(SourceRef::File { path, max_edge, loader, fallback: None, denoise }),
SourceRef::Loaded(d) if d.info.is_some() => Some(SourceRef::Loaded(d)),
_ => None,
}
}
/// The file-system half of Build / Discard Smart Previews (runs on a worker thread in the app).
/// `custom`: the folder was chosen (on another drive, perhaps): it must be there, it is never
/// recreated. With `state`, progress and cancel.
@@ -134,7 +176,7 @@ fn smart_run(
custom: bool,
discard: bool,
jobs: Vec<SmartJob>,
state: Option<&PreviewBuild>,
state: Option<(&PreviewBuild, &TaskGuard)>,
) -> std::result::Result<SmartCounts, String> {
let mut n = SmartCounts::default();
if !discard {
@@ -145,21 +187,27 @@ fn smart_run(
std::fs::create_dir_all(dir).map_err(|e| format!("{}: {e}", dir.display()))?;
}
}
for (id, path, source) in jobs {
if state.is_some_and(|st| st.cancel.load(Ordering::Relaxed)) {
for (id, path, source, look) in jobs {
if state.is_some_and(|(st, _)| st.cancel.load(Ordering::Relaxed)) {
break;
}
let _one = SMART_WRITE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if discard {
if std::fs::remove_file(&path).is_ok() {
n.removed += 1;
}
} else {
// a complete proxy is kept; a missing or damaged one (cut short by a crash or a full
// drive) is (re)built
// a complete, current proxy is kept; a missing or damaged one (cut short by a crash or a
// full drive) is (re)built, and so is one written by an older camera-look fit (see
// `camera_preview::LOOK_VERSION`) when its original can be read
let damaged = path.exists();
if damaged && crate::smart::is_valid(&path) {
let stale = look && damaged && crate::smart::is_stale(&path);
if damaged && !stale && crate::smart::is_valid(&path) {
n.built += 1;
} else {
// a stale proxy is rebuilt from the original only: never from itself (the file
// fallback or a source decoded from a proxy would stamp the old look as current)
let source = if stale { source.and_then(original_only) } else { source };
// atomic: a failed write leaves no partial proxy that would pass for a built one;
// and synced (issue #134): built so the photo can be edited while its original
// is offline, when it is the only copy — the sync is small next to the decode
@@ -171,24 +219,33 @@ fn smart_run(
match r {
Ok(()) => {
n.built += 1;
if damaged {
if stale {
n.refreshed += 1;
} else if damaged {
n.repaired += 1;
if let Some(st) = state {
if let Some((st, _)) = state {
st.repaired.fetch_add(1, Ordering::Relaxed);
}
}
}
Err(e) if stale => {
// offline (or unreadable) original: the proxy is all there is, keep its look
log::info!("smart preview {} keeps its older look: {e}", path.display());
n.built += 1;
n.stale_kept += 1;
}
Err(e) => {
n.failed.push(json!([id.0, e]));
if let Some(st) = state {
if let Some((st, _)) = state {
st.failed.fetch_add(1, Ordering::Relaxed);
}
}
}
}
}
if let Some(st) = state {
st.done.fetch_add(1, Ordering::Relaxed);
if let Some((st, task)) = state {
let done = st.done.fetch_add(1, Ordering::Relaxed).saturating_add(1);
task.progress(done as u64, st.total as u64);
}
}
Ok(n)
@@ -215,18 +272,19 @@ fn smart(s: &mut Session, p: &Value) -> Result<Value> {
}
let path = dir.join(crate::smart::file_name(&ph));
let source = (!discard).then(|| s.media.source_ref(&ph, crate::media::SourceLevel::Preview));
jobs.push((id, path, source));
jobs.push((id, path, source, ph.relative_wb()));
}
let custom = s.smart_previews_dir.is_some();
if background {
let what = if discard { "discarding smart previews" } else { "smart previews" };
let state = Arc::new(PreviewBuild { total: jobs.len(), what, ..Default::default() });
s.preview_build = Some(state.clone());
let task = task_for(s, &state);
let st = state.clone();
std::thread::Builder::new()
.name("lc-smart-previews".into())
.spawn(move || {
if let Err(e) = smart_run(&dir, custom, discard, jobs, Some(&st)) {
if let Err(e) = smart_run(&dir, custom, discard, jobs, Some((&st, &task))) {
log::warn!("{C}: {e}");
*st.error.lock().unwrap_or_else(std::sync::PoisonError::into_inner) = Some(e);
}
@@ -236,7 +294,54 @@ fn smart(s: &mut Session, p: &Value) -> Result<Value> {
return Ok(state.json());
}
let r = smart_run(&dir, custom, discard, jobs, None).map_err(|e| bad(C, e))?;
Ok(json!({"built": r.built, "repaired": r.repaired, "removed": r.removed, "failed": r.failed}))
Ok(
json!({"built": r.built, "repaired": r.repaired, "refreshed": r.refreshed, "staleKept": r.stale_kept, "removed": r.removed, "failed": r.failed}),
)
}
/// Bring smart previews written by an older camera-look fit up to date, once per fit version: when
/// the smart previews folder has not been checked at this `LOOK_VERSION` (its marker,
/// [`crate::smart::look_checked`]), a worker thread (the UI never waits on the drives) looks at the
/// proxies of the raws that take the per-file camera look, the only ones the fit reaches (JPEG,
/// PNG… proxies are never rebuilt), and rebuilds the stale ones from their originals. One whose
/// original can't be read (offline) is kept as it is, still marked stale in its header: Build
/// Smart Previews rebuilds it once the original is back. Only existing proxies are touched: none
/// is built, and a missing folder is never created. Returns the thread, if started.
#[cfg(not(target_arch = "wasm32"))]
pub(crate) fn refresh_stale_smart_previews(s: &mut Session) -> Option<std::thread::JoinHandle<()>> {
let dir = s.media.smart_dir.clone()?;
if !dir.is_dir() || crate::smart::look_checked(&dir) {
return None;
}
let max_edge = crate::media::SourceLevel::Preview.max_edge();
let photos: Vec<_> = s.catalog.photos().filter(|p| matches!(p.source, lightcraft_catalog::Source::File { .. }) && p.relative_wb()).collect();
let jobs: Vec<SmartJob> =
photos.iter().map(|p| (p.id, dir.join(crate::smart::file_name(p)), Some(s.media.origin_ref(&p.source, max_edge)), true)).collect();
std::thread::Builder::new()
.name("lc-smart-look".into())
.spawn(move || refresh_stale(&dir, jobs))
.map_err(|e| log::warn!("smart previews: could not start the look refresh: {e}"))
.ok()
}
/// The body of [`refresh_stale_smart_previews`]: rebuild the stale proxies among `jobs`.
#[cfg(not(target_arch = "wasm32"))]
fn refresh_stale(dir: &std::path::Path, jobs: Vec<SmartJob>) {
let stale: Vec<SmartJob> = jobs.into_iter().filter(|(_, path, _, _)| crate::smart::is_stale(path)).collect();
if !stale.is_empty() {
// `custom`: the folder is there already, and must not be recreated if it went away since
match smart_run(dir, true, false, stale, None) {
Ok(n) => log::info!("smart previews: {} refreshed to the current camera look, {} kept (original offline)", n.refreshed, n.stale_kept),
Err(e) => {
// not marked: the next opening tries again
log::warn!("smart previews: look refresh: {e}");
return;
}
}
}
if let Err(e) = crate::smart::mark_look_checked(dir) {
log::warn!("smart previews: {e}");
}
}
/// The smart previews folder: where it is, whether it is custom, what it holds, whether it can be
crates/engine/src/denoise.rs
+28
View File
@@ -890,6 +890,8 @@ pub(crate) struct State {
pub generation: u64,
/// Photos made since the session started.
made: u64,
/// AI Denoise's row in the activity stack, while a photo is being made or waits its turn.
task: Option<crate::activity::TaskGuard>,
}
impl Default for State {
@@ -909,6 +911,7 @@ impl Default for State {
failed: HashMap::new(),
generation: 0,
made: 0,
task: None,
}
}
}
@@ -1354,6 +1357,31 @@ impl Session {
self.denoise.running.is_some() || (!self.denoise.queue.is_empty() && self.denoise.active.is_some())
}
/// Keep AI Denoise's row in the activity stack up to date (each `denoise.pump`): there while a photo is being made or
/// waits its turn, with the running photo's tiles and its file name; gone when the queue is idle. It has no ✕: what is
/// made follows the photos' Denoise amounts and Settings ▸ AI Denoise.
pub(crate) fn denoise_track(&mut self) {
if !self.denoise_busy() {
self.denoise.task = None;
return;
}
let task = self.denoise.task.get_or_insert_with(|| {
let t = self.activity.start("denoise", "AI Denoise", crate::activity::Cancel::No);
t.set_unit(crate::activity::Unit::Percent);
t
});
match &self.denoise.running {
Some(r) => {
task.progress(r.progress.done.load(Ordering::Relaxed) as u64, r.progress.total.load(Ordering::Relaxed) as u64);
task.detail(self.catalog.photo(r.photo).map_or("", |p| p.file_name.as_str()));
}
None => {
task.progress(0, 0);
task.detail("");
}
}
}
/// Seconds the running job has been going, and its progress.
pub(crate) fn denoise_running_json(&self) -> Value {
match &self.denoise.running {
crates/engine/src/face_download.rs
+76 -4
View File
@@ -15,6 +15,8 @@ use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex, PoisonError};
use std::time::Duration;
use crate::activity::{Activity, Cancel, TaskHandle, Unit};
use lightcraft_faces::known::Download;
/// The folder inside the models folder where a downloaded file waits to be installed. Its name starts with a dot, so it
@@ -42,15 +44,20 @@ pub enum State {
struct Shared {
state: Mutex<State>,
cancel: AtomicBool,
cancel: Arc<AtomicBool>,
/// The download's row in the activity stack.
task: Option<TaskHandle>,
}
impl Shared {
fn new(total: u64) -> Self {
Shared { state: Mutex::new(State::Running { bytes: 0, total }), cancel: AtomicBool::new(false) }
Shared { state: Mutex::new(State::Running { bytes: 0, total }), cancel: Arc::new(AtomicBool::new(false)), task: None }
}
fn set(&self, s: State) {
if let (State::Running { bytes, total }, Some(task)) = (&s, &self.task) {
task.progress(*bytes, *total);
}
*self.state.lock().unwrap_or_else(PoisonError::into_inner) = s;
}
@@ -68,18 +75,27 @@ pub struct Downloads {
impl Downloads {
/// Start fetching `spec` into `<models dir>/.downloads/`. Different models can download side by side; the same
/// one is started once.
pub fn start(&mut self, spec: Download, models_dir: &Path) -> Result<(), String> {
/// The activity stack shows it (as `name`) until the thread ends; its ✕ stops it like [`Downloads::discard`].
pub fn start(&mut self, spec: Download, models_dir: &Path, activity: &Activity, name: &str) -> Result<(), String> {
if self.jobs.get(spec.id.as_str()).is_some_and(|j| matches!(j.get(), State::Running { .. })) {
return Err("that model is already downloading".into());
}
let staging = models_dir.join(STAGING);
std::fs::create_dir_all(&staging).map_err(|e| format!("could not create the download folder: {e}"))?;
let shared = Arc::new(Shared::new(spec.size_bytes));
let cancel = Arc::new(AtomicBool::new(false));
let task = activity.start("download", "Downloading model", Cancel::Flag(cancel.clone()));
task.set_unit(Unit::Bytes);
task.detail(name);
task.progress(0, spec.size_bytes);
let total = spec.size_bytes;
let shared = Arc::new(Shared { state: Mutex::new(State::Running { bytes: 0, total }), cancel, task: Some(task.handle()) });
let worker = shared.clone();
let id = spec.id.clone();
std::thread::Builder::new()
.name("face-model-download".into())
.spawn(move || {
// the row goes when the thread ends, however it ends
let _task = task;
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| run(&spec, spec.base(), &staging, &worker)));
if outcome.is_err() {
worker.set(State::Failed("the download stopped unexpectedly".into()));
@@ -267,6 +283,62 @@ mod tests {
base
}
/// A local server that sends `body` a few kilobytes at a time: a download that keeps running for a while.
fn serve_slowly(body: Vec<u8>) -> String {
let l = TcpListener::bind("127.0.0.1:0").unwrap();
let base = format!("http://{}/m", l.local_addr().unwrap());
std::thread::spawn(move || {
for mut s in l.incoming().flatten() {
let mut r = BufReader::new(s.try_clone().unwrap());
loop {
let mut line = String::new();
if r.read_line(&mut line).unwrap_or(0) == 0 || line.trim().is_empty() {
break;
}
}
let _ = s.write_all(format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", body.len()).as_bytes());
for chunk in body.chunks(4096) {
if s.write_all(chunk).is_err() {
break;
}
std::thread::sleep(Duration::from_millis(20));
}
}
});
base
}
/// Poll until `done` holds (or fail after 10 s).
fn wait(what: &str, done: impl Fn() -> bool) {
let t = std::time::Instant::now();
while !done() {
assert!(t.elapsed() < Duration::from_secs(10), "{what}");
std::thread::sleep(Duration::from_millis(10));
}
}
#[test]
fn a_running_download_shows_in_activity_and_cancels() {
use crate::activity::{Activity, Unit};
let dir = scratch("activity");
let bytes: Vec<u8> = (0..400_000u32).map(|i| (i % 251) as u8).collect();
let mut spec = spec_for(&bytes);
spec.url = format!("{}/{}", serve_slowly(bytes), spec.file_name);
let activity = Activity::default();
let mut d = Downloads::default();
d.start(spec, &dir, &activity, "Test Model").unwrap();
let rows = activity.list();
assert_eq!(rows.len(), 1);
assert_eq!((rows[0].kind, rows[0].label.as_str(), rows[0].detail.as_str()), ("download", "Downloading model", "Test Model"));
assert_eq!((rows[0].unit, rows[0].total), (Unit::Bytes, 400_000));
assert!(rows[0].cancellable);
activity.cancel(rows[0].id).unwrap();
wait("the download never stopped", || !d.running());
assert_eq!(d.jobs["test-model"].get(), State::Cancelled);
wait("the row stayed", || activity.list().is_empty());
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn fetches_checks_and_stages_a_file() {
let dir = scratch("ok");
crates/engine/src/faces_index.rs
+4
View File
@@ -434,6 +434,10 @@ pub(crate) struct FacesState {
pub scanned: Scanned,
/// Photos that could not be searched this session (unreadable files): left alone until the next launch.
pub scan_failed: std::collections::HashSet<u64>,
/// The scan's row in the activity stack, while photos are left.
pub task: Option<crate::activity::TaskGuard>,
/// The most photos left at once since the scan last finished (the row's whole).
pub peak: u64,
}
impl FacesState {
crates/engine/src/files.rs
+78 -15
View File
@@ -223,6 +223,12 @@ impl CameraColourModel<'_> {
lightcraft_raw::highlight::reconstruct_with(img, self.wb, HIGHLIGHT_CLIP, clipped_white(&self.matrix));
}
/// [`Self::rebuild_highlights`] for a binned image whose clipped channels `mask` tells
/// ([`lightcraft_raw::RawImage::develop_binned_masked`]).
pub fn rebuild_highlights_masked(&self, img: &mut Rgb32f, mask: &[u8]) {
lightcraft_raw::highlight::reconstruct_masked(img, self.wb, HIGHLIGHT_CLIP, clipped_white(&self.matrix), mask);
}
/// White balance, matrix, tables and baseline exposure, in place.
pub fn apply(&self, img: &mut Rgb32f) {
let (wb, m, gain) = (self.wb, self.matrix.to_f32(), self.gain);
@@ -319,8 +325,9 @@ pub fn load_vec_pair(bytes: Vec<u8>, max_edge: usize, spec: &DenoiseSpec) -> Res
}
/// The denoised camera RGB of `raw` from its cached product, as the plain development would have it: the default
/// crop, and `factor` × `factor` blocks averaged into one pixel when the plain one was binned. `None` when the product
/// is missing, damaged, made from something else or not the size of this raw's active area.
/// crop, and `factor` × `factor` blocks averaged into one pixel when the plain one was binned (plain means: the
/// plain development's clip mask tells highlight reconstruction which of them held clipped samples). `None` when
/// the product is missing, damaged, made from something else or not the size of this raw's active area.
fn denoised_camera_rgb(raw: &lightcraft_raw::RawImage, spec: &DenoiseSpec, factor: usize) -> Option<Rgb32f> {
use lightcraft_denoise::product::{self, Window};
let a = raw.active_area;
@@ -331,7 +338,7 @@ fn denoised_camera_rgb(raw: &lightcraft_raw::RawImage, spec: &DenoiseSpec, facto
if raw.opcodes.list3.is_empty() {
let c = raw.develop_crop(a.width, a.height);
let window = Window { x: c.x, y: c.y, width: c.width, height: c.height };
product::read_window(&spec.product, &spec.key, Some(window), factor, Some(HIGHLIGHT_CLIP)).ok()
product::read_window(&spec.product, &spec.key, Some(window), factor, None).ok()
} else {
// (the plain development is not binned either when the file has opcodes of this kind: `factor` is 1)
let whole = product::read(&spec.product, &spec.key, 1).ok()?;
@@ -357,27 +364,29 @@ fn load_bytes_now(
};
// Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in
// (removed before the camera look's binned sensor proxy too, which needs an empty `OpcodeList3`).
let lens = embedded_lens(&raw.info());
let info = raw.info();
let lens = embedded_lens(&info);
raw.opcodes.list3.retain(|op| !op.is_lens_correction());
let xy = lightcraft_raw::color::as_shot_white_xy(&raw);
let t = lightcraft_raw::color::camera_transform(&raw, xy);
// the source's segmentation mattes (DNG semantic masks), read while the preview is fitted
let (camera_look, mattes) = rayon::join(|| crate::camera_preview::fit_preview(&raw, &bytes, &t), || dng_mattes(&bytes, &raw));
// Without colour matrices of its own, white balance is relative to the as-shot look (`Photo::relative_wb`).
let own_matrix = lightcraft_raw::color::has_matrix(&raw.color);
// the source's segmentation mattes (DNG semantic masks), read while the starting colour is fitted
let ((xy, t, camera_look), mattes) = rayon::join(|| crate::camera_preview::starting_colour(&mut raw, &bytes), || dng_mattes(&bytes, &info));
drop(bytes);
// Previews and thumbnails bin the mosaic straight to (about) the size they need; only
// larger levels (exports, 1:1) demosaic the whole sensor.
let t0 = web_time::Instant::now();
let bin = bin_factor(&raw, max_edge);
// (binned: block means, and which colours had a clipped sample in each block)
let binned = match bin {
Some(k) => raw.develop_binned(k, HIGHLIGHT_CLIP).map_err(|e| e.to_string())?,
Some(k) => raw.develop_binned_masked(k, HIGHLIGHT_CLIP).map_err(|e| e.to_string())?,
None => None,
};
let factor = if binned.is_some() { bin.unwrap_or(1) } else { 1 };
let img = match binned {
Some(img) => img,
let (img, clip_mask) = match binned {
Some((img, mask)) => (img, Some(mask)),
None => {
let method = if max_edge <= 600 { lightcraft_raw::Method::Bilinear } else { lightcraft_raw::Method::Ahd };
raw.develop(method).map_err(|e| e.to_string())?
(raw.develop(method).map_err(|e| e.to_string())?, None)
}
};
// the same development from the denoised mosaic, when the photo has been denoised
@@ -399,7 +408,10 @@ fn load_bytes_now(
let finish = |mut img: Rgb32f| {
let mut stages = vec![("develop", t0.elapsed())];
stages.push(("transform", t0.elapsed()));
colour.rebuild_highlights(&mut img);
match &clip_mask {
Some(mask) => colour.rebuild_highlights_masked(&mut img, mask),
None => colour.rebuild_highlights(&mut img),
}
stages.push(("highlights", t0.elapsed()));
colour.apply(&mut img);
stages.push(("colour", t0.elapsed()));
@@ -424,7 +436,7 @@ fn load_bytes_now(
}
let twin = twin.map(|picture| finish(picture).0);
let (temp, tint) = xy_to_temp_tint(xy);
let relative = crate::camera_preview::file_local_look(raw.format) && t.matrix_is_fallback;
let relative = crate::camera_preview::file_local_look(raw.format) && !own_matrix;
// White balance re-evaluates the file's own colour model (when it has one and no
// file-local look matrix sits on top of it)
let camera_color = (!t.matrix_is_fallback && camera_look.is_none()).then(|| {
@@ -472,7 +484,7 @@ fn local_tone(raw: &lightcraft_raw::RawImage) -> Option<Arc<lightcraft_pipeline:
/// The semantic masks of a DNG that AI masks understand, over the developed image (default crop,
/// oriented like it). Person mattes without a confidently selected pixel are left out (the
/// Subject mask then falls back to its heuristic); a sky matte counts even when empty (no sky).
fn dng_mattes(bytes: &[u8], raw: &lightcraft_raw::RawImage) -> Option<Arc<lightcraft_pipeline::masks::Mattes>> {
fn dng_mattes(bytes: &[u8], raw: &lightcraft_raw::RawInfo) -> Option<Arc<lightcraft_pipeline::masks::Mattes>> {
use lightcraft_pipeline::masks::{MatteKind, Mattes};
if raw.format != lightcraft_raw::RawFormat::Dng {
return None;
@@ -708,6 +720,37 @@ mod tests {
assert!(probe_bytes("x.jpg", b"not an image").is_err());
}
/// Issue #548: a small preview bins the mosaic by 8; specular points that clip a single green sample
/// of a block on a grey surface must not become green speckles (the block's green used to take the
/// clip level, which highlight reconstruction then kept).
#[test]
fn coarse_binning_keeps_specular_points_neutral() {
let (w, h) = (256usize, 192usize);
// grey after the file's white balance (as-shot neutral 0.5 : 1 : 0.7), brighter to the right
let neutral = [0.5f32, 1.0, 0.7];
let cfa = lightcraft_raw::Cfa::bayer("RGGB").unwrap();
let mut data: Vec<u16> = (0..w * h)
.map(|i| {
let (x, y) = (i % w, i / w);
let v = (0.3 + 0.2 * x as f32 / w as f32) * neutral[cfa.color_at(x, y).min(2) as usize];
(256.0 + v * (16000.0 - 256.0)).round() as u16
})
.collect();
// one clipped green sample (RGGB: even row, odd column) in every other 8 × 8 block
for by in 0..h / 8 {
for bx in (by % 2..w / 8).step_by(2) {
data[(by * 8 + 4) * w + bx * 8 + 5] = 16000;
}
}
let bytes = crate::tests_xmp::synthetic_dng_of(w, h, "RGGB", data, None, Default::default());
let raw = lightcraft_raw::decode(&bytes).unwrap();
assert_eq!(bin_factor(&raw, 32), Some(8));
let (img, _) = load_bytes(&bytes, 32).unwrap();
assert_eq!((img.width, img.height), (32, 24));
let worst = img.data.iter().map(|p| p[1] / ((p[0] + p[2]) / 2.0).max(1e-6)).fold(0.0f32, f32::max);
assert!(worst < 1.12, "green speckles: green {worst:.2}× the other channels");
}
#[test]
fn cached_denoise_uses_the_same_white_balance_profile_and_orientation_as_plain_raw() {
use lightcraft_denoise::product;
@@ -1464,6 +1507,26 @@ mod tests {
let plain = probe_bytes("small.tif", &tiff_shell(24, 16, false)).unwrap();
assert_eq!((plain.kind, plain.preview_only, plain.width, plain.height), (MediaKind::Image, None, 24, 16));
assert!(lightcraft_raw::probe(&tiff_shell(24, 16, false)).is_none());
// TIFFs preserving camera Make tags (SONY, NIKON, etc.) are ordinary images, not raws (#281)
for make in ["SONY", "NIKON CORPORATION", "PENTAX", "RICOH", "SAMSUNG"] {
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value, tags as t};
let mut ifd0 = IfdBuilder::new();
ifd0.set(t::IMAGE_WIDTH, Value::Long(vec![24]));
ifd0.set(t::IMAGE_LENGTH, Value::Long(vec![16]));
ifd0.set(t::BITS_PER_SAMPLE, Value::Short(vec![8, 8, 8]));
ifd0.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![3]));
ifd0.set(t::PHOTOMETRIC, Value::Short(vec![2]));
ifd0.set(t::COMPRESSION, Value::Short(vec![1]));
ifd0.set(t::MAKE, Value::Ascii(make.into()));
let px: Vec<u8> = (0..24 * 16).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
ifd0.set_image(ImageData::Strips { rows_per_strip: 16, strips: vec![px] });
let bytes = TiffWriter::default().write(&[ifd0]).unwrap();
let p = probe_bytes("photo.tif", &bytes).unwrap();
assert_eq!((p.kind, p.preview_only, p.width, p.height), (MediaKind::Image, None, 24, 16), "make: {make}");
let (img, src) = load_bytes(&bytes, 24).unwrap();
assert_eq!((img.width, img.height, src.raw), (24, 16, false), "make: {make}");
}
}
/// Recognised raw containers we don't decode (Minolta MRW here: the preview's first byte is
crates/engine/src/import.rs
+2 -1
View File
@@ -501,7 +501,8 @@ pub struct ScanProgress {
/// Files to probe, set once the folders are expanded.
pub total: std::sync::atomic::AtomicUsize,
pub done: std::sync::atomic::AtomicUsize,
pub cancel: std::sync::atomic::AtomicBool,
/// Shared with the scan's row in the activity stack (its ✕ sets it).
pub cancel: std::sync::Arc<std::sync::atomic::AtomicBool>,
}
/// The result of [`scan_with`]: the candidates, and the probes to keep for the import that follows.
crates/engine/src/lib.rs
+6 -1
View File
@@ -14,8 +14,9 @@
#[cfg(not(feature = "denoise"))]
extern crate lightcraft_denoise_core as lightcraft_denoise;
pub mod activity;
pub mod availability;
mod camera_preview;
pub(crate) mod camera_preview;
pub mod camera_profiles;
pub mod cmd;
pub mod config;
@@ -261,6 +262,9 @@ pub struct Session {
/// The last Synchronize Folder scan, which `folder.synchronize` acts on while it is current
/// (see [`Session::take_folder_changes`]).
pub folder_changes: Option<sync::FolderChanges>,
/// Background tasks in flight (imports, exports, preview builds…), for the activity stack and
/// `activity.list` / `activity.cancel`.
pub activity: activity::Activity,
/// The last (or running) Build Previews.
pub preview_build: Option<std::sync::Arc<cmd::previews::PreviewBuild>>,
/// Develop defaults applied on import (persisted in prefs.json).
@@ -345,6 +349,7 @@ impl Session {
before: Default::default(),
import_probes: Default::default(),
folder_changes: None,
activity: Default::default(),
preview_build: None,
import_defaults: import::ImportDefaults::default(),
cache_mb: 0,
crates/engine/src/library.rs
+5
View File
@@ -419,6 +419,11 @@ impl Session {
lib.forgot_local = Some(plan);
}
}
// smart previews written by an older camera-look fit are brought up to date in the background
#[cfg(not(target_arch = "wasm32"))]
if on_disk {
crate::cmd::previews::refresh_stale_smart_previews(self);
}
self.library_identity = std::sync::Arc::new(());
Ok(())
}
crates/engine/src/media.rs
+1 -1
View File
@@ -62,7 +62,7 @@ impl SettingsHashes {
}
/// Bump when the pipeline's output changes, to invalidate cached thumbnails.
pub const RENDER_CACHE_VERSION: u64 = 37;
pub const RENDER_CACHE_VERSION: u64 = 42;
/// Thumbnails render at one of these long edges (so window/cell size changes reuse the cache).
pub const THUMB_SIZES: [usize; 4] = [128, 256, 384, 512];
crates/engine/src/model_download.rs
+77 -5
View File
@@ -15,6 +15,8 @@ use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex, PoisonError};
use std::time::Duration;
use crate::activity::{Activity, Cancel, TaskHandle, Unit};
use lightcraft_denoise::known::Download;
/// The folder inside the models folder where a downloaded file waits to be installed. Its name starts with a dot, so it
@@ -43,15 +45,20 @@ pub enum State {
struct Shared {
state: Mutex<State>,
cancel: AtomicBool,
cancel: Arc<AtomicBool>,
/// The download's row in the activity stack.
task: Option<TaskHandle>,
}
impl Shared {
fn new(total: u64) -> Self {
Shared { state: Mutex::new(State::Running { bytes: 0, total }), cancel: AtomicBool::new(false) }
Shared { state: Mutex::new(State::Running { bytes: 0, total }), cancel: Arc::new(AtomicBool::new(false)), task: None }
}
fn set(&self, s: State) {
if let (State::Running { bytes, total }, Some(task)) = (&s, &self.task) {
task.progress(*bytes, *total);
}
*self.state.lock().unwrap_or_else(PoisonError::into_inner) = s;
}
@@ -69,18 +76,27 @@ pub struct Downloads {
impl Downloads {
/// Start fetching `spec` into `<models dir>/.downloads/`. Different models can download side by side; the same
/// one is started once.
pub fn start(&mut self, spec: Download, models_dir: &Path) -> Result<(), String> {
/// The activity stack shows it (as `name`) until the thread ends; its ✕ stops it like [`Downloads::discard`].
pub fn start(&mut self, spec: Download, models_dir: &Path, activity: &Activity, name: &str) -> Result<(), String> {
if self.jobs.get(&spec.id).is_some_and(|j| matches!(j.get(), State::Running { .. })) {
return Err("that model is already downloading".into());
}
let staging = models_dir.join(STAGING);
std::fs::create_dir_all(&staging).map_err(|e| format!("could not create the download folder: {e}"))?;
let shared = Arc::new(Shared::new(spec.size_bytes));
let cancel = Arc::new(AtomicBool::new(false));
let task = activity.start("download", "Downloading model", Cancel::Flag(cancel.clone()));
task.set_unit(Unit::Bytes);
task.detail(name);
task.progress(0, spec.size_bytes);
let total = spec.size_bytes;
let shared = Arc::new(Shared { state: Mutex::new(State::Running { bytes: 0, total }), cancel, task: Some(task.handle()) });
let worker = shared.clone();
let id = spec.id.clone();
std::thread::Builder::new()
.name("denoise-model-download".into())
.spawn(move || {
// the row goes when the thread ends, however it ends
let _task = task;
let outcome = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| run(&spec, &spec.url, &staging, &worker)));
if outcome.is_err() {
worker.set(State::Failed("the download stopped unexpectedly".into()));
@@ -218,7 +234,7 @@ mod tests {
use std::net::TcpListener;
fn scratch(name: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("lc-face-dl-{name}-{}", std::process::id()));
let d = std::env::temp_dir().join(format!("lc-denoise-dl-{name}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
@@ -254,6 +270,62 @@ mod tests {
base
}
/// A local server that sends `body` a few kilobytes at a time: a download that keeps running for a while.
fn serve_slowly(body: Vec<u8>) -> String {
let l = TcpListener::bind("127.0.0.1:0").unwrap();
let base = format!("http://{}/m", l.local_addr().unwrap());
std::thread::spawn(move || {
for mut s in l.incoming().flatten() {
let mut r = BufReader::new(s.try_clone().unwrap());
loop {
let mut line = String::new();
if r.read_line(&mut line).unwrap_or(0) == 0 || line.trim().is_empty() {
break;
}
}
let _ = s.write_all(format!("HTTP/1.1 200 OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n", body.len()).as_bytes());
for chunk in body.chunks(4096) {
if s.write_all(chunk).is_err() {
break;
}
std::thread::sleep(Duration::from_millis(20));
}
}
});
base
}
/// Poll until `done` holds (or fail after 10 s).
fn wait(what: &str, done: impl Fn() -> bool) {
let t = std::time::Instant::now();
while !done() {
assert!(t.elapsed() < Duration::from_secs(10), "{what}");
std::thread::sleep(Duration::from_millis(10));
}
}
#[test]
fn a_running_download_shows_in_activity_and_cancels() {
use crate::activity::{Activity, Unit};
let dir = scratch("activity");
let bytes: Vec<u8> = (0..400_000u32).map(|i| (i % 251) as u8).collect();
let mut spec = spec_for(&bytes);
spec.url = format!("{}/{}", serve_slowly(bytes), spec.file_name);
let activity = Activity::default();
let mut d = Downloads::default();
d.start(spec, &dir, &activity, "Test Model").unwrap();
let rows = activity.list();
assert_eq!(rows.len(), 1);
assert_eq!((rows[0].kind, rows[0].label.as_str(), rows[0].detail.as_str()), ("download", "Downloading model", "Test Model"));
assert_eq!((rows[0].unit, rows[0].total), (Unit::Bytes, 400_000));
assert!(rows[0].cancellable);
activity.cancel(rows[0].id).unwrap();
wait("the download never stopped", || !d.running());
assert_eq!(d.jobs["test-model"].get(), State::Cancelled);
wait("the row stayed", || activity.list().is_empty());
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn fetches_checks_and_stages_a_file() {
let dir = scratch("ok");
crates/engine/src/segment/download.rs
+19 -4
View File
@@ -8,6 +8,8 @@ use std::sync::{Arc, Mutex};
use lightcraft_segment::fetch::{self, Options, Progress};
use crate::activity::{Activity, Cancel, Unit};
/// What the session sees of a download.
#[derive(Clone, Debug, Default, PartialEq, Eq, serde::Serialize)]
pub struct Status {
@@ -31,7 +33,7 @@ struct Text {
#[derive(Default)]
struct Shared {
running: AtomicBool,
cancel: AtomicBool,
cancel: Arc<AtomicBool>,
finished: AtomicBool,
done: AtomicU64,
total: AtomicU64,
@@ -92,9 +94,17 @@ impl Downloader {
running
}
/// Start downloading `files` from `mirrors` into `dir` on a background thread. False when
/// one is already running.
pub fn start(&self, files: &'static [fetch::FileSpec], mirrors: Vec<String>, dir: PathBuf, opts: Options) -> Result<bool, String> {
/// Start downloading `files` from `mirrors` into `dir` on a background thread, shown in the activity stack as
/// `name` (its ✕ cancels like [`Downloader::cancel`]). False when one is already running.
pub fn start(
&self,
files: &'static [fetch::FileSpec],
mirrors: Vec<String>,
dir: PathBuf,
opts: Options,
activity: &Activity,
name: &str,
) -> Result<bool, String> {
let s = self.shared.clone();
if s.running.swap(true, Ordering::SeqCst) {
return Ok(false);
@@ -106,6 +116,10 @@ impl Downloader {
// nothing else runs while `running` was false
*lock(&s.text) = Text::default();
*lock(&self.last) = Text::default();
let task = activity.start("download", "Downloading model", Cancel::Flag(s.cancel.clone()));
task.set_unit(Unit::Bytes);
task.detail(name);
task.progress(0, s.total.load(Ordering::SeqCst));
let guard = Running(s.clone());
let spawned = std::thread::Builder::new().name("sam3-download".into()).spawn(move || {
let s = guard.0.clone();
@@ -113,6 +127,7 @@ impl Downloader {
let mut on_progress = |p: &Progress| {
s.done.store(p.done, Ordering::SeqCst);
s.total.store(p.total, Ordering::SeqCst);
task.progress(p.done, p.total);
if p.file != last_file {
last_file.clone_from(&p.file);
lock(&s.text).file.clone_from(&p.file);
crates/engine/src/segment/mod.rs
+3 -3
View File
@@ -259,7 +259,7 @@ impl Segmenter {
/// Start downloading the model (on a background thread; see `segment.model.download`).
/// `Ok(false)` when it is installed or downloading already.
pub fn start_download(&self) -> Result<bool, String> {
pub fn start_download(&self, activity: &crate::activity::Activity) -> Result<bool, String> {
if !Self::AVAILABLE {
return Err("AI masks are not available in this build".into());
}
@@ -273,11 +273,11 @@ impl Segmenter {
if mirrors.is_empty() {
return Err(lightcraft_segment::fetch::no_mirrors_message());
}
self.download.start(lightcraft_segment::fetch::SAM3_FILES, mirrors, dir, lightcraft_segment::fetch::options())
self.download.start(lightcraft_segment::fetch::SAM3_FILES, mirrors, dir, lightcraft_segment::fetch::options(), activity, "SAM 3")
}
#[cfg(not(feature = "sam"))]
{
let _ = dir;
let _ = (dir, activity);
Ok(false)
}
}
crates/engine/src/smart.rs
+55 -1
View File
@@ -24,6 +24,10 @@ pub fn file_name(p: &Photo) -> String {
/// The most header [`is_valid`] reads (the JSON line before the JPEG).
const HEADER_MAX: u64 = 4096;
/// The header field that records which version of the camera-look fit (`camera_preview::LOOK_VERSION`)
/// wrote the proxy's pixels (colour matrix and profile are baked into them) and stored tone curve.
const LOOK_FIELD: &str = "look_version";
/// Encode a source image (and the decoder's camera tone curve, if any) as a smart preview.
pub fn encode(img: &Rgb32f, tone: Option<&CameraTone>) -> Result<Vec<u8>, String> {
// scale so all but the brightest 0.05 % fit into 0..1
@@ -53,7 +57,7 @@ pub fn encode(img: &Rgb32f, tone: Option<&CameraTone>) -> Result<Vec<u8>, String
)
.map_err(|e| e.to_string())?;
let mut out = MAGIC.to_vec();
let mut head = serde_json::json!({"w": img.width, "h": img.height, "scale": scale});
let mut head = serde_json::json!({"w": img.width, "h": img.height, "scale": scale, LOOK_FIELD: crate::camera_preview::LOOK_VERSION});
if let Some(t) = tone {
head["tone"] = serde_json::to_value(t).map_err(|e| e.to_string())?;
}
@@ -187,6 +191,56 @@ pub fn is_valid(path: &Path) -> bool {
f.seek(SeekFrom::End(-2)).is_ok() && f.read_exact(&mut end).is_ok() && end == [0xFF, 0xD9]
}
/// The fit version a header records: a missing, negative, fractional or otherwise malformed value
/// counts as 0 (older than any real version).
fn header_look_version(head: &serde_json::Value) -> u64 {
head.get(LOOK_FIELD).and_then(serde_json::Value::as_u64).unwrap_or(0)
}
/// Whether the complete proxy at `path` was written by an older camera-look fit than this build's
/// (so its stored curve, and the colour baked into its pixels, are out of date). A proxy from a
/// newer build is left alone (rebuilding it here would only undo that build's work), as is one
/// that can't be read: those are [`is_valid`]'s business.
pub fn is_stale(path: &Path) -> bool {
use std::io::Read;
if !is_valid(path) {
return false;
}
let Ok(f) = std::fs::File::open(path) else { return false };
let mut head = Vec::with_capacity(1024);
if f.take(HEADER_MAX).read_to_end(&mut head).is_err() {
return false;
}
let Some(rest) = head.strip_prefix(MAGIC) else { return false };
let Some(nl) = rest.iter().position(|b| *b == b'\n') else { return false };
rest.get(..nl)
.and_then(|h| serde_json::from_slice::<serde_json::Value>(h).ok())
.is_some_and(|h| header_look_version(&h) < u64::from(crate::camera_preview::LOOK_VERSION))
}
/// The marker file in a smart previews folder: the `LOOK_VERSION` its proxies were last checked
/// at ([`look_checked`]).
const LOOK_MARKER: &str = "look-version";
/// Whether the smart previews folder `dir` has been checked for stale proxies at this build's
/// camera-look fit version (or a newer one). A missing or unreadable marker counts as not.
pub fn look_checked(dir: &Path) -> bool {
use std::io::Read;
let Ok(f) = std::fs::File::open(dir.join(LOOK_MARKER)) else { return false };
let mut text = String::new();
if f.take(64).read_to_string(&mut text).is_err() {
return false;
}
text.trim().parse::<u64>().is_ok_and(|v| v >= u64::from(crate::camera_preview::LOOK_VERSION))
}
/// Record that the proxies in `dir` were checked at this build's camera-look fit version.
pub fn mark_look_checked(dir: &Path) -> Result<(), String> {
let path = dir.join(LOOK_MARKER);
lightcraft_catalog::safe_file::write_atomic(&path, format!("{}\n", crate::camera_preview::LOOK_VERSION).as_bytes())
.map_err(|e| format!("{}: {e}", path.display()))
}
/// Load the proxy at `path`.
pub fn load(path: &Path) -> Result<crate::media::DecodedSource, String> {
let b = std::fs::read(path).map_err(|e| format!("{}: {e}", path.display()))?;
crates/engine/src/tests.rs
+36
View File
@@ -721,6 +721,42 @@ fn build_previews_fills_the_cache() {
assert!(s.execute("library.buildPreviews", &json!({"size": "huge"})).is_err());
}
/// Wait (at most a minute) until no background task is listed.
fn wait_activity_empty(s: &Session) {
let t0 = std::time::Instant::now();
while !s.activity.list().is_empty() {
assert!(t0.elapsed().as_secs() < 60, "tasks still listed: {:?}", s.activity.list());
std::thread::sleep(std::time::Duration::from_millis(20));
}
}
/// Issue #345: a background preview build is a row in the activity stack while it runs.
#[test]
fn preview_build_shows_in_activity_until_it_finishes() {
let mut s = demo();
let ids: Vec<u64> = s.catalog.photos().take(4).map(|p| p.id.0).collect();
let r = s.execute("library.buildPreviews", &json!({"ids": ids, "size": "standard", "edge": 256})).unwrap();
let tasks = s.activity.list();
assert_eq!(tasks.first().map(|t| (t.kind, t.total, t.label.as_str())), Some(("previews", 4, "Building previews")), "{tasks:?} {r}");
assert!(tasks[0].cancellable);
wait_activity_empty(&s);
assert_eq!(s.execute("library.previewProgress", &json!({})).unwrap()["running"], false);
}
/// Issue #345: `activity.cancel` stops a preview build like `library.cancelPreviews`.
#[test]
fn activity_cancel_stops_a_preview_build() {
let mut s = demo();
let ids: Vec<u64> = s.catalog.photos().map(|p| p.id.0).collect();
s.execute("library.buildPreviews", &json!({"ids": ids, "size": "full"})).unwrap();
let id = s.activity.list().first().map(|t| t.id).expect("a previews row");
s.execute("activity.cancel", &json!({"id": id})).unwrap();
wait_activity_empty(&s);
let p = s.execute("library.previewProgress", &json!({})).unwrap();
assert_eq!(p["cancelled"], true, "{p}");
assert!(p["done"].as_u64() < p["total"].as_u64(), "stopped part-way: {p}");
}
/// Colour range: a click samples the colour under it, so the mask selects that colour (white
/// in the mask view) and not a different one; ⇧ adds samples, up to five.
#[test]
crates/engine/src/tests_denoise.rs
+32
View File
@@ -757,3 +757,35 @@ fn the_denoise_switch_validates_arguments_and_accepts_explicit_targets_without_s
drop(x.s);
let _ = std::fs::remove_dir_all(x.dir);
}
#[test]
fn denoise_row_while_queue_is_busy() {
let mut x = setup("activity", true);
let rows = |s: &Session| s.activity.list().into_iter().filter(|t| t.kind == "denoise").collect::<Vec<_>>();
let r = x.s.execute("denoise.pump", &json!({"pace": "full"})).unwrap();
assert!(rows(&x.s).is_empty(), "nothing to make, no row: {r}");
set_amount(&mut x.s, 100.0);
let mut shown = false;
for _ in 0..600 {
let r = x.s.execute("denoise.pump", &json!({"pace": "full"})).unwrap();
let rows = rows(&x.s);
// (what the pump left running or queued is what the row reflects until the next pump)
if x.s.denoise_busy() {
assert_eq!(rows.len(), 1, "{r}");
assert_eq!((rows[0].label.as_str(), rows[0].cancellable, rows[0].unit), ("AI Denoise", false, crate::activity::Unit::Percent), "{r}");
if !r["running"].is_null() {
assert_eq!(rows[0].detail, "ramp.dng", "{r}");
}
shown = true;
} else {
assert!(rows.is_empty(), "{r}");
}
if r["ready"].as_u64() >= Some(1) {
break;
}
std::thread::sleep(std::time::Duration::from_millis(50));
}
assert!(shown, "the photo never showed a row");
assert!(rows(&x.s).is_empty(), "the row goes once the photo is made");
let _ = std::fs::remove_dir_all(&x.dir);
}
crates/engine/src/tests_face_recognize.rs
+36
View File
@@ -487,3 +487,39 @@ fn every_scanned_face_is_shown_by_the_same_kind_of_box() {
assert!((v.x0 - tight.x0).abs() < 1e-3 && (v.y0 - tight.y0).abs() < 1e-3, "{v:?}");
let _ = std::fs::remove_dir_all(&d);
}
#[test]
fn face_scan_row_follows_pending_photos() {
let d = temp("activity");
let (mut s, _) = setup(&d, 64);
// the last two demo photos, not the first two the other scan tests use: two sessions scanning the same demo scene
// at once can deadlock (issue #630), and this test is about the row, not that
let mut ids = s.catalog.photos().map(|p| p.id).collect::<Vec<_>>();
ids.sort();
let (a, b) = (ids[ids.len() - 2], ids[ids.len() - 1]);
set_regions(&mut s, a, vec![region(0.1, None), region(0.5, None)]);
set_regions(&mut s, b, vec![region(0.3, None)]);
let faces_rows = |s: &Session| s.activity.list().into_iter().filter(|t| t.kind == "faces").collect::<Vec<_>>();
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(60);
let (mut last, mut shown) = (json!(null), false);
while std::time::Instant::now() < deadline {
last = s.execute("faces.pump", &json!({})).unwrap();
let rows = faces_rows(&s);
if last["pendingPhotos"].as_u64().unwrap_or(0) > 0 {
// while photos are left: one row, not cancellable, its whole the most photos left at once
assert_eq!(rows.len(), 1, "{last}");
assert_eq!((rows[0].label.as_str(), rows[0].cancellable), ("Finding faces", false));
assert_eq!(rows[0].total, last["peak"].as_u64().unwrap(), "{last}");
assert_eq!(rows[0].done, rows[0].total - last["pendingPhotos"].as_u64().unwrap());
shown = true;
}
if last["active"] == true && last["pendingPhotos"] == 0 && last["indexedFaces"] == 3 {
break;
}
std::thread::sleep(std::time::Duration::from_millis(50));
}
assert!(shown, "the scan never showed a row: {last}");
assert_eq!((last["pendingPhotos"].clone(), last["peak"].clone()), (json!(0), json!(0)), "{last}");
assert!(faces_rows(&s).is_empty(), "the row goes when the scan is done");
let _ = std::fs::remove_dir_all(&d);
}
crates/engine/src/tests_import.rs
+212
View File
@@ -497,6 +497,12 @@ fn damaged_smart_previews_are_rebuilt_and_failed_writes_leave_none() {
};
assert_eq!((r["done"].as_u64(), r["failed"].as_u64(), r["repaired"].as_u64()), (Some(1), Some(0), Some(1)), "{r}");
assert_eq!(std::fs::read(&file).unwrap(), full, "rebuilt whole in the background");
// its row in the activity stack (issue #345) goes with it
let t0 = std::time::Instant::now();
while !s.activity.list().is_empty() && t0.elapsed() < std::time::Duration::from_secs(10) {
std::thread::sleep(std::time::Duration::from_millis(5));
}
assert!(s.activity.list().is_empty(), "{:?}", s.activity.list());
let _ = std::fs::remove_dir_all(&src);
let _ = std::fs::remove_dir_all(&lib);
}
@@ -1125,6 +1131,212 @@ fn folder_import_picks_up_undecodable_raw_containers_as_preview_only() {
let _ = std::fs::remove_dir_all(&src);
}
/// Rewrite the JSON header line of the smart preview at `file`.
fn edit_smart_header(file: &Path, edit: impl FnOnce(&mut Value)) {
let bytes = std::fs::read(file).unwrap();
let start = b"LCSP1\n".len();
let nl = start + bytes[start..].iter().position(|b| *b == b'\n').unwrap();
let mut head: Value = serde_json::from_slice(&bytes[start..nl]).unwrap();
edit(&mut head);
let mut out = bytes[..start].to_vec();
out.extend_from_slice(head.to_string().as_bytes());
out.extend_from_slice(&bytes[nl..]);
std::fs::write(file, out).unwrap();
}
fn smart_header(file: &Path) -> Value {
let bytes = std::fs::read(file).unwrap();
let start = b"LCSP1\n".len();
let nl = start + bytes[start..].iter().position(|b| *b == b'\n').unwrap();
serde_json::from_slice(&bytes[start..nl]).unwrap()
}
/// A camera tone curve with a recognisable shape (`k` changes it).
fn test_tone(k: f32) -> lightcraft_pipeline::tone::CameraTone {
lightcraft_pipeline::tone::CameraTone::new(std::array::from_fn(|i| {
let x = 0.004 * 1.17f32.powi(i as i32);
[x, 0.95 * (1.0 - (-k * x).exp())]
}))
.unwrap()
}
/// `plain`, but reporting `tone` as the camera look the (current) fit gives the file; counts its calls.
fn look_loader(
plain: crate::media::FileLoader,
tone: lightcraft_pipeline::tone::CameraTone,
calls: std::sync::Arc<std::sync::atomic::AtomicUsize>,
) -> crate::media::FileLoader {
std::sync::Arc::new(move |path, max_edge| {
calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
let (img, mut info) = plain(path, max_edge)?;
info.camera_tone = Some(tone);
Ok((img, info))
})
}
/// The follow-up to the camera tone curve change (#499): a smart preview keeps the look the fit
/// gave it, so one written by an older fit is rebuilt when its original is online, kept while the
/// original is offline, and not rebuilt again once current. Only raws that take the per-file
/// camera look go stale (a PNG's proxy never does), and the opening scan runs once per fit version.
#[test]
fn smart_previews_from_an_older_look_fit_are_refreshed() {
use std::sync::atomic::{AtomicUsize, Ordering};
let src = temp_dir("smartlook-src");
let lib = temp_dir("smartlook-lib");
write_png(&src.join("a.png"), 3);
write_png(&src.join("b.png"), 9);
let mut s = Session::new().with_fs();
s.open_library(&lib, false).unwrap();
s.execute("library.import", &json!({"paths": [src.join("a.png").to_string_lossy(), src.join("b.png").to_string_lossy()]})).unwrap();
let find = |s: &Session, name: &str| s.catalog.photos().find(|p| p.file_name == name).unwrap().id;
let (id, png) = (find(&s, "a.png"), find(&s, "b.png"));
// `a` stands for a raw that takes the per-file camera look (the test loader decodes its pixels)
let mut raw = (**s.catalog.photo(id).unwrap()).clone();
raw.format = "ARW".into();
raw.kind = lightcraft_catalog::MediaKind::Raw;
s.catalog.apply(lightcraft_catalog::Op::RemovePhoto { id }).unwrap();
s.catalog.apply(lightcraft_catalog::Op::AddPhoto { photo: Box::new(raw) }).unwrap();
assert!(s.catalog.photo(id).unwrap().relative_wb() && !s.catalog.photo(png).unwrap().relative_wb());
s.execute("library.select", &json!({"ids": [id.0, png.0], "active": id.0})).unwrap();
let dir = s.media.smart_dir.clone().unwrap();
let file = dir.join(crate::smart::file_name(s.catalog.photo(id).unwrap()));
let png_file = dir.join(crate::smart::file_name(s.catalog.photo(png).unwrap()));
let (old, new) = (test_tone(1.5), test_tone(3.5));
let calls = std::sync::Arc::new(AtomicUsize::new(0));
let plain = s.media.file_loader.clone().unwrap();
s.media.file_loader = Some(look_loader(plain.clone(), old, calls.clone()));
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["built"].as_u64(), r["refreshed"].as_u64()), (Some(2), Some(0)), "{r}");
assert_eq!(smart_header(&file)["look_version"], json!(crate::camera_preview::LOOK_VERSION), "the writer stamps the fit version");
assert!(!crate::smart::is_stale(&file));
// current header: kept as it is, the original is not even read
let before = std::fs::read(&file).unwrap();
calls.store(0, Ordering::SeqCst);
s.media.file_loader = Some(look_loader(plain.clone(), new, calls.clone()));
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["built"].as_u64(), r["refreshed"].as_u64()), (Some(2), Some(0)), "{r}");
assert!(std::fs::read(&file).unwrap() == before, "a current proxy is not rebuilt");
assert_eq!(calls.load(Ordering::SeqCst), 0);
// old headers (no field), originals offline: kept unchanged, and still marked stale
for f in [&file, &png_file] {
edit_smart_header(f, |h| {
h.as_object_mut().unwrap().remove("look_version");
});
}
assert!(crate::smart::is_stale(&file), "no field = version 0");
let old_bytes = std::fs::read(&file).unwrap();
let png_bytes = std::fs::read(&png_file).unwrap();
std::fs::rename(&src, src.with_extension("offline")).unwrap();
s.media.forget(id);
s.media.forget(png);
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["built"].as_u64(), r["refreshed"].as_u64(), r["staleKept"].as_u64()), (Some(2), Some(0), Some(1)), "{r}");
assert!(r["failed"].as_array().unwrap().is_empty(), "{r}");
assert!(std::fs::read(&file).unwrap() == old_bytes, "offline: the stored preview and curve are untouched");
assert!(crate::smart::is_stale(&file), "and it stays marked for the next time");
assert_eq!(crate::smart::decode(&old_bytes).unwrap().1, Some(old), "the old look is still there");
// ... and the opening refresh keeps it too, then marks the folder checked
assert!(!crate::smart::look_checked(&dir));
crate::cmd::previews::refresh_stale_smart_previews(&mut s).unwrap().join().unwrap();
assert!(std::fs::read(&file).unwrap() == old_bytes);
assert!(crate::smart::look_checked(&dir));
// the original comes back: the scan has run at this version, so the next opening leaves it;
// Build Smart Previews rebuilds it with the current look
std::fs::rename(src.with_extension("offline"), &src).unwrap();
calls.store(0, Ordering::SeqCst);
assert!(crate::cmd::previews::refresh_stale_smart_previews(&mut s).is_none(), "checked once per version");
assert!(std::fs::read(&file).unwrap() == old_bytes);
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["refreshed"].as_u64(), calls.load(Ordering::SeqCst)), (Some(1), 1), "{r}");
assert_eq!(smart_header(&file)["look_version"], json!(crate::camera_preview::LOOK_VERSION));
assert_eq!(crate::smart::decode(&std::fs::read(&file).unwrap()).unwrap().1, Some(new), "rebuilt with the current look");
assert!(!crate::smart::is_stale(&file));
assert!(std::fs::read(&png_file).unwrap() == png_bytes, "a PNG's proxy is never rebuilt for the look");
// a folder checked at an older version (or not at all) is scanned once at the next opening
edit_smart_header(&file, |h| h["look_version"] = json!(0));
std::fs::write(dir.join("look-version"), b"0\n").unwrap();
assert!(!crate::smart::look_checked(&dir));
s.media.forget(id);
calls.store(0, Ordering::SeqCst);
crate::cmd::previews::refresh_stale_smart_previews(&mut s).unwrap().join().unwrap();
assert_eq!(calls.load(Ordering::SeqCst), 1, "only the raw's original is read");
assert!(!crate::smart::is_stale(&file));
assert!(std::fs::read(&png_file).unwrap() == png_bytes);
assert!(crate::smart::look_checked(&dir));
// once only
let rebuilt = std::fs::read(&file).unwrap();
assert!(crate::cmd::previews::refresh_stale_smart_previews(&mut s).is_none());
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!((r["refreshed"].as_u64(), calls.load(Ordering::SeqCst)), (Some(0), 1), "{r}");
assert!(std::fs::read(&file).unwrap() == rebuilt);
// Build Smart Previews refreshes a stale one the same way
edit_smart_header(&file, |h| h["look_version"] = json!(0));
s.media.file_loader = Some(plain);
let r = s.execute("library.smartPreviews", &json!({})).unwrap();
assert_eq!(r["refreshed"], 1, "{r}");
assert_eq!(crate::smart::decode(&std::fs::read(&file).unwrap()).unwrap().1, None, "this fit gives no curve: none is kept");
let _ = std::fs::remove_dir_all(&src);
let _ = std::fs::remove_dir_all(&lib);
}
/// The look marker: missing, damaged or older than this build's fit version counts as unchecked.
#[test]
fn smart_preview_look_marker_is_read_safely() {
let dir = temp_dir("smartlook-marker");
assert!(!crate::smart::look_checked(&dir), "no marker");
let cur = crate::camera_preview::LOOK_VERSION;
for (text, checked) in [
(String::new(), false),
("banana".into(), false),
("-1".into(), false),
(format!("{}", cur.saturating_sub(1)), false),
("9".repeat(200), false),
(format!("{cur}\n"), true),
(format!(" {}\n", cur + 1), true),
] {
std::fs::write(dir.join("look-version"), &text).unwrap();
assert_eq!(crate::smart::look_checked(&dir), checked, "{text:?}");
}
std::fs::remove_file(dir.join("look-version")).unwrap();
crate::smart::mark_look_checked(&dir).unwrap();
assert!(crate::smart::look_checked(&dir));
assert!(crate::smart::mark_look_checked(&dir.join("missing")).is_err());
let _ = std::fs::remove_dir_all(&dir);
}
/// A hostile or damaged `look_version` never panics: anything that is not a non-negative integer
/// counts as 0 (rebuilt when the original is online); a version from a newer build is left alone.
#[test]
fn malformed_smart_preview_look_versions_are_handled() {
let dir = temp_dir("smartlook-bad");
let img = lightcraft_scenes::demo_library()[0].render(32, 20);
let file = dir.join("a.lcsp");
std::fs::write(&file, crate::smart::encode(&img, None).unwrap()).unwrap();
let cur = u64::from(crate::camera_preview::LOOK_VERSION);
for bad in [json!("1"), json!(-1), json!(1.5), json!(null), json!([]), json!({}), json!(true), json!(1e30), json!(-0.0)] {
let shown = bad.to_string();
edit_smart_header(&file, |h| h["look_version"] = bad);
assert!(crate::smart::is_valid(&file), "{shown}");
assert!(crate::smart::is_stale(&file), "{shown} counts as version 0");
assert!(crate::smart::decode(&std::fs::read(&file).unwrap()).is_ok(), "{shown}");
}
for fine in [json!(cur), json!(cur + 1), json!(u64::MAX)] {
let shown = fine.to_string();
edit_smart_header(&file, |h| h["look_version"] = fine);
assert!(!crate::smart::is_stale(&file), "{shown}");
}
// not a header at all, or missing
assert!(!crate::smart::is_stale(&dir.join("missing.lcsp")));
std::fs::write(&file, b"LCSP1\n{not json\n\xFF\xD8").unwrap();
assert!(!crate::smart::is_stale(&file));
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn a_file_gone_since_the_import_review_is_not_imported() {
let src = temp_dir("gone-since-review");
crates/engine/src/tests_segment.rs
+48
View File
@@ -230,3 +230,51 @@ fn object_clicks_are_capped() {
assert!(!s.segmenter.busy());
let _ = std::fs::remove_dir_all(&dir);
}
#[cfg(feature = "sam")]
#[test]
fn a_running_model_download_shows_in_activity_and_cancels() {
use std::io::{BufRead, BufReader, Write};
// a local mirror that sends its files slowly, so the download is still running when it is cancelled
let l = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let base = format!("http://{}/sam3", l.local_addr().unwrap());
std::thread::spawn(move || {
for mut c in l.incoming().flatten() {
let mut r = BufReader::new(c.try_clone().unwrap());
let mut line = String::new();
while r.read_line(&mut line).unwrap_or(0) > 2 {
line.clear();
}
let _ = c.write_all(b"HTTP/1.1 200 OK\r\nContent-Length: 1000000\r\nConnection: close\r\n\r\n");
for _ in 0..250 {
if c.write_all(&[b' '; 4000]).is_err() {
break;
}
std::thread::sleep(Duration::from_millis(20));
}
}
});
let dir = tmp("download-activity");
let mirrors = dir.with_extension("activity-mirrors.txt");
std::fs::write(&mirrors, format!("# test mirror\n{base}\n")).unwrap();
let mut s = Session::with_demo();
s.segmenter.dir = Some(dir.clone());
s.segmenter.mirrors_file = Some(mirrors.clone());
assert_eq!(s.execute("segment.model.download", &json!({"acknowledged": true})).unwrap()["started"], true);
let rows = s.activity.list();
assert_eq!(rows.len(), 1);
assert_eq!((rows[0].kind, rows[0].label.as_str(), rows[0].detail.as_str()), ("download", "Downloading model", "SAM 3"));
assert_eq!(rows[0].unit, crate::activity::Unit::Bytes);
assert!(rows[0].cancellable);
s.execute("activity.cancel", &json!({"id": rows[0].id})).unwrap();
let t = Instant::now();
while s.segmenter.download_status().running || !s.activity.list().is_empty() {
assert!(t.elapsed() < Duration::from_secs(10), "the download never stopped");
std::thread::sleep(Duration::from_millis(10));
}
let e = s.segmenter.download_status().error.unwrap_or_default();
assert!(e.contains("cancelled"), "{e}");
assert!(!s.segmenter.installed());
let _ = std::fs::remove_dir_all(&dir);
let _ = std::fs::remove_file(&mirrors);
}
crates/meta/src/xmp.rs
+1 -1
View File
@@ -418,7 +418,7 @@ fn parse_one_region(item: &XmpValue, px_dims: Option<(f64, f64)>) -> Option<Regi
// frame and writes the orientation as the region's `mwg-rs:Rotation` (π for 3, −π/2 for 6, +π/2 for
// 8; 0 for 1) while `AppliedToDimensions` names the upright size. Turn it into the upright frame.
// MWG has no way to say "mirrored", so orientations 2/4/5/7 can't be told apart from their unmirrored
// twins here; boxes on mirrored photos are taken as written (see LR-LIB-PEOPLE in docs/parity.md).
// twins here; boxes on mirrored photos are taken as written (see LR-LIB-PEOPLE in docs/parity-checklist.md).
let rotation = item.field("mwg-rs:Rotation").and_then(XmpValue::text).and_then(parse_number).filter(|r| r.is_finite());
let (quarter, half) = (std::f64::consts::FRAC_PI_2, std::f64::consts::PI);
let (cx, cy, w, h) = match rotation {
crates/raw/src/binned.rs
+65 -7
View File
@@ -8,9 +8,13 @@
//! interpolation, so there are no demosaicing artefacts; the colour planes are offset by less than
//! one output pixel, which is invisible at these scales.
//!
//! Clipping is preserved for [`crate::highlight::reconstruct`]: a channel whose block contains a
//! sample at or above `clip` takes that block's maximum sample of the colour (instead of a mean that
//! would fall just below the clip level and escape reconstruction).
//! Clipping is decided per sample, before averaging. [`RawImage::develop_binned_masked`] keeps every
//! channel the mean of its samples and says, per output pixel, which colours had a sample at or above
//! `clip` (for [`crate::highlight::reconstruct_masked`], which rebuilds them with the mean as a lower
//! bound). A specular point that clips one green sample of the 32 in an 8 × 8 block thus counts as
//! partly clipped without the block's green jumping to the clip level, which would read as a green
//! speckle once reconstructed. [`RawImage::develop_binned`] instead gives such a channel the block's
//! maximum sample of the colour, so that plain value thresholds see it as clipped.
use crate::{Normalized, RawData, RawImage, Result, Rgb32f};
use rayon::prelude::*;
@@ -31,6 +35,19 @@ impl RawImage {
/// not oriented). `None` when the data cannot be binned ([`RawImage::can_bin`]) or carries an
/// `OpcodeList3` (whose operations are defined at full resolution); callers then demosaic.
pub fn develop_binned(&self, k: usize, clip: f32) -> Result<Option<Rgb32f>> {
Ok(self.bin(k, clip, true)?.map(|(img, _)| img))
}
/// Like [`RawImage::develop_binned`], with every channel the mean of its block's samples, and the
/// clip mask: per output pixel, bits [`crate::highlight::CLIPPED_R`] / `_G` / `_B` for the colours
/// with a sample at or above `clip` in the block. Hand both to [`crate::highlight::reconstruct_masked`].
pub fn develop_binned_masked(&self, k: usize, clip: f32) -> Result<Option<(Rgb32f, Vec<u8>)>> {
self.bin(k, clip, false)
}
/// The binned image and its clip mask; `keep_max`: a channel with a clipped sample takes the
/// block's maximum sample of the colour instead of the mean.
fn bin(&self, k: usize, clip: f32, keep_max: bool) -> Result<Option<(Rgb32f, Vec<u8>)>> {
self.validate()?;
if !self.can_bin(k) || !self.opcodes.list3.is_empty() {
return Ok(None);
@@ -63,7 +80,8 @@ impl RawImage {
.collect();
let wlen = bw * k;
let mut out = Rgb32f::new(bw, bh);
out.data.par_chunks_mut(bw).enumerate().for_each(|(by, row)| {
let mut mask = vec![0u8; bw * bh];
out.data.par_chunks_mut(bw).zip(mask.par_chunks_mut(bw)).enumerate().for_each(|(by, (row, mrow))| {
// the block row's k sample rows, normalised
let mut rows = vec![0f32; k * wlen];
for dy in 0..k {
@@ -86,7 +104,7 @@ impl RawImage {
}
}
let py = (c.y + by * k) % ph;
for (bx, px) in row.iter_mut().enumerate() {
for (bx, (px, m)) in row.iter_mut().zip(mrow.iter_mut()).enumerate() {
let (layout, inv) = &layouts[py * pw + (c.x + bx * k) % pw];
let (mut sum, mut max) = ([0f32; 3], [f32::MIN; 3]);
for dy in 0..k {
@@ -98,11 +116,13 @@ impl RawImage {
}
}
for ch in 0..3 {
px[ch] = if max[ch] >= clip { max[ch] } else { sum[ch] * inv[ch] };
let clipped = max[ch] >= clip;
px[ch] = if clipped && keep_max { max[ch] } else { sum[ch] * inv[ch] };
*m |= u8::from(clipped) << ch;
}
}
});
Ok(Some(out))
Ok(Some((out, mask)))
}
}
@@ -177,6 +197,44 @@ mod tests {
assert!((xb.get(4, 3)[2] - 0.3).abs() < 2e-3);
}
/// Issue #548: specular points that clip one green sample in some 8 × 8 blocks of a grey surface.
/// Taking the block's maximum makes those blocks' green the clip level, which reconstruction then
/// keeps (a green speckle); the masked binning keeps the mean and marks the green as clipped.
#[test]
fn clipping_is_decided_before_averaging() {
let img = Rgb32f::filled(64, 64, [0.3, 0.3, 0.3]);
let mut raw = raw_from(&mosaic_from_rgb(&img, &Cfa::bayer("RGGB").unwrap()), 0.0, 4095.0);
if let RawData::U16(d) = &mut raw.data {
// one green sample (RGGB: odd column of an even row) in every other block
for by in 0..8 {
for bx in (by % 2..8).step_by(2) {
d[(by * 8 + 2) * 64 + bx * 8 + 3] = 4095;
}
}
}
let wb = [1.0f32; 3];
let (mut masked, mask) = raw.develop_binned_masked(8, 0.99).unwrap().unwrap();
assert_eq!(mask.len(), 64);
assert_eq!(mask.iter().filter(|&&m| m == highlight::CLIPPED_G).count(), 32);
assert!(mask.iter().all(|&m| m & (highlight::CLIPPED_R | highlight::CLIPPED_B) == 0));
// the clipped block's green is its mean: (31 · 0.3 + 1) / 32
assert!((masked.get(0, 0)[1] - (31.0 * 0.3 + 1.0) / 32.0).abs() < 1e-3, "{:?}", masked.get(0, 0));
assert_eq!(highlight::reconstruct_masked(&mut masked, wb, 0.99, [1.0; 3], &mask), 32);
let green = |img: &Rgb32f| img.data.iter().map(|p| p[1] / p[0]).fold(0.0f32, f32::max);
// (no more than the mean itself: the clipped sample counts at the clip level)
assert!(green(&masked) < 1.08, "green excess {}", green(&masked));
// the maximum rule: a green speckle at the clip level
let mut max = raw.develop_binned(8, 0.99).unwrap().unwrap();
assert!(max.get(0, 0)[1] >= 0.99);
highlight::reconstruct(&mut max, wb, 0.99);
assert!(green(&max) > 3.0, "{}", green(&max));
// without clipped samples both are the same plain means, and the mask is empty
let clean = raw_from(&mosaic_from_rgb(&img, &Cfa::bayer("RGGB").unwrap()), 0.0, 4095.0);
let (a, m) = clean.develop_binned_masked(8, 0.99).unwrap().unwrap();
assert!(m.iter().all(|&v| v == 0));
assert!(a.data == clean.develop_binned(8, 0.99).unwrap().unwrap().data);
}
#[test]
fn binning_matches_demosaic_downscaled_on_smooth_images() {
let img = Rgb32f::from_fn(64, 48, |x, y| [0.2 + 0.3 * (x as f32 / 64.0), 0.4 - 0.2 * (y as f32 / 48.0), 0.25]);
crates/raw/src/highlight.rs
+75 -10
View File
@@ -9,6 +9,8 @@
//! there is none within reach. Right past the rim, the rim's colour carries on. Fully clipped pixels become
//! neutral at the brightest plausible level; partly clipped ones fade into it on their way from the first channel
//! clipping to the last, pale ones from the start.
//! - [`reconstruct_masked`]: the same for a binned image, whose clipped channels a mask tells (block means with a
//! clipped sample among them).
//!
//! Why the care: the white-balance gains push clipped channels apart (green clips first, red and blue keep
//! rising), so whatever colour the rebuilt pixels carry is hidden at Neutral only by the tone map's roll-off to
@@ -116,7 +118,7 @@ fn sample<const N: usize>(l: &Level<N>, scale: f32, x: usize, y: usize) -> [f32;
/// The half-resolution source level of `img`: every `2 × 2` block's source-weighted chromaticity sums, with
/// blocks near clipped pixels or on strong edges left out. Also returns, per cell, whether a clipped pixel lies
/// in its row within [`MARGIN`] cells (the vertical half of that test is left to the readers).
fn sources(img: &Rgb32f, wb: [f32; 3], clip: f32) -> (Level<3>, Vec<bool>) {
fn sources(img: &Rgb32f, wb: [f32; 3], clip: f32, clipped: &impl Clipped) -> (Level<3>, Vec<bool>) {
let (w, h) = (img.width, img.height);
let (cw, ch) = (w.div_ceil(2), h.div_ceil(2));
// per cell: [red sum, blue sum, weight] densities, luminance, whether it holds a clipped pixel
@@ -131,13 +133,15 @@ fn sources(img: &Rgb32f, wb: [f32; 3], clip: f32) -> (Level<3>, Vec<bool>) {
for x in 0..cw {
let (mut a, mut sum, mut n, mut hit) = ([0f32; 3], 0f32, 0usize, false);
let span = 2 * x..(2 * x + 2).min(w);
for p in r0.get(span.clone()).unwrap_or(&[]).iter().chain(r1.get(span).unwrap_or(&[])) {
let (i0, i1) = (2 * y * w + 2 * x, (2 * y + 1) * w + 2 * x);
let pixels = r0.get(span.clone()).unwrap_or(&[]).iter().zip(i0..).chain(r1.get(span).unwrap_or(&[]).iter().zip(i1..));
for (p, i) in pixels {
let q = [p[0] * wb[0], p[1] * wb[1], p[2] * wb[2]];
let s = q[0].max(0.0) + q[1].max(0.0) + q[2].max(0.0);
sum += s;
n += 1;
let top = p[0].max(p[1]).max(p[2]);
if top >= clip {
if clipped(i, p).iter().any(|&c| c) {
hit = true;
continue;
}
@@ -191,7 +195,7 @@ fn sources(img: &Rgb32f, wb: [f32; 3], clip: f32) -> (Level<3>, Vec<bool>) {
/// above [`RIM_LO`] of the clip level, weighted (steeply) towards the clip, as densities. Pixels next to clipped
/// ones (`margin`, by half-resolution cell) are left out, like the colour sources: they were demosaiced partly from
/// clipped samples, and lens fringes along clipped content would pass for its colour.
fn rim(img: &Rgb32f, wb: [f32; 3], clip: f32, margin: &(dyn Fn(usize, usize) -> bool + Sync)) -> Level<4> {
fn rim(img: &Rgb32f, wb: [f32; 3], clip: f32, clipped: &impl Clipped, margin: &(dyn Fn(usize, usize) -> bool + Sync)) -> Level<4> {
let (w, h) = (img.width, img.height);
let (cw, ch) = (w.div_ceil(4), h.div_ceil(4));
let mut cells = vec![[0f32; 4]; cw * ch];
@@ -202,7 +206,7 @@ fn rim(img: &Rgb32f, wb: [f32; 3], clip: f32, margin: &(dyn Fn(usize, usize) ->
for (i, p) in img.data.get(sy * w + 4 * x..sy * w + (4 * x + 4).min(w)).unwrap_or(&[]).iter().enumerate() {
n += 1;
let top = p[0].max(p[1]).max(p[2]) / clip;
if !(RIM_LO..1.0).contains(&top) || margin((4 * x + i) / 2, sy / 2) {
if !(RIM_LO..1.0).contains(&top) || margin((4 * x + i) / 2, sy / 2) || clipped(sy * w + 4 * x + i, p).iter().any(|&c| c) {
continue;
}
let k = ((top - RIM_LO) / (1.0 - RIM_LO)).powi(4);
@@ -341,9 +345,44 @@ pub fn reconstruct(img: &mut Rgb32f, wb: [f32; 3], clip: f32) -> usize {
/// chromaticity of reliable unclipped content nearby, and that of `white` where there is none.
/// Returns the number of pixels that had at least one clipped channel; the others are left exactly as they were.
pub fn reconstruct_with(img: &mut Rgb32f, wb: [f32; 3], clip: f32, white: [f32; 3]) -> usize {
rebuild_clipped(img, wb, clip, white, &|_, p| by_value(p, clip))
}
/// Bits of a [`reconstruct_masked`] mask entry: which channels of the pixel had a clipped sample.
pub const CLIPPED_R: u8 = 1;
pub const CLIPPED_G: u8 = 2;
pub const CLIPPED_B: u8 = 4;
/// [`reconstruct_with`] for a binned image ([`crate::RawImage::develop_binned_masked`]): its values are block
/// means, and `mask` (one entry per pixel, bits [`CLIPPED_R`], [`CLIPPED_G`], [`CLIPPED_B`]) says which colours
/// had a clipped sample in the block. Those channels count as clipped with their mean as the lower bound of their
/// true value, so one clipped specular sample among many unclipped ones neither escapes reconstruction nor
/// (as the block's maximum would) lifts the channel to the clip level, which reads as a coloured speckle (#548).
/// A mask of the wrong length is ignored (clipping is then told by the values alone).
pub fn reconstruct_masked(img: &mut Rgb32f, wb: [f32; 3], clip: f32, white: [f32; 3], mask: &[u8]) -> usize {
if mask.len() != img.data.len() {
return reconstruct_with(img, wb, clip, white);
}
rebuild_clipped(img, wb, clip, white, &|i, p| {
let m = mask.get(i).copied().unwrap_or(0);
let v = by_value(p, clip);
[v[0] || m & CLIPPED_R != 0, v[1] || m & CLIPPED_G != 0, v[2] || m & CLIPPED_B != 0]
})
}
/// Which channels of pixel `i` (values `p`) are clipped.
trait Clipped: Fn(usize, &[f32; 3]) -> [bool; 3] + Sync {}
impl<F: Fn(usize, &[f32; 3]) -> [bool; 3] + Sync> Clipped for F {}
#[inline]
fn by_value(p: &[f32; 3], clip: f32) -> [bool; 3] {
[p[0] >= clip, p[1] >= clip, p[2] >= clip]
}
/// [`reconstruct_with`], with `clipped` telling the clipped channels.
fn rebuild_clipped(img: &mut Rgb32f, wb: [f32; 3], clip: f32, white: [f32; 3], clipped: &impl Clipped) -> usize {
let (w, h) = (img.width, img.height);
let is_clipped = |p: &[f32; 3]| p[0] >= clip || p[1] >= clip || p[2] >= clip;
let count = img.data.par_iter().filter(|p| is_clipped(p)).count();
let count = img.data.par_iter().enumerate().filter(|(i, p)| clipped(*i, p).iter().any(|&c| c)).count();
if count == 0 || w == 0 || h == 0 {
return 0;
}
@@ -354,7 +393,7 @@ pub fn reconstruct_with(img: &mut Rgb32f, wb: [f32; 3], clip: f32, white: [f32;
let neutral = [white[0] / total, white[1] / total, white[2] / total];
// A clipped pixel is never a source, so its chromaticity is always the bilinear sample of the
// half-resolution estimate: the pyramid starts there, straight from the image.
let (first, near) = sources(img, wb, clip);
let (first, near) = sources(img, wb, clip, clipped);
let mut levels = vec![first];
while let Some(l) = levels.last().filter(|l| l.w > 1 || l.h > 1) {
let next = pull(l);
@@ -392,7 +431,7 @@ pub fn reconstruct_with(img: &mut Rgb32f, wb: [f32; 3], clip: f32, white: [f32;
let Some(field) = levels.first() else { return count };
// the rim (level 0 of its pyramid holds the estimate), if there is one anywhere at all
let rims = {
let mut levels = vec![rim(img, wb, clip, &needed)];
let mut levels = vec![rim(img, wb, clip, clipped, &needed)];
while let Some(l) = levels.last().filter(|l| l.w > 1 || l.h > 1) {
let next = pull(l);
levels.push(next);
@@ -405,7 +444,7 @@ pub fn reconstruct_with(img: &mut Rgb32f, wb: [f32; 3], clip: f32, white: [f32;
img.data.par_chunks_mut(w).enumerate().for_each(|(y, row)| {
for (x, px) in row.iter_mut().enumerate() {
let p = *px;
let cl = [p[0] >= clip, p[1] >= clip, p[2] >= clip];
let cl = clipped(y * w + x, &p);
if !cl.iter().any(|&b| b) {
continue;
}
@@ -813,4 +852,30 @@ mod tests {
.fold(f64::MAX, f64::min);
eprintln!("reconstruct 24 MP: {ms:.0} ms");
}
/// A mask decides which channels are clipped (their values are lower bounds), whatever their values; the
/// unmarked pixels are left as they were; a mask of another size leaves clipping to the values.
#[test]
fn a_mask_marks_the_clipped_channels() {
let base = Rgb32f::from_fn(32, 32, |x, _| if x >= 16 { [0.2, 0.5, 0.25] } else { [0.5, 0.5, 0.5] });
let mask: Vec<u8> = (0..32 * 32).map(|i| if i % 32 >= 16 { CLIPPED_R } else { 0 }).collect();
let mut img = base.clone();
assert_eq!(reconstruct_masked(&mut img, [1.0; 3], 0.99, [1.0; 3], &mask), 512);
for (p, b) in img.data.iter().zip(&base.data) {
assert!(p.iter().all(|v| v.is_finite()), "{p:?}");
if b[0] == 0.5 {
assert_eq!(p, b);
} else {
// red rebuilt from at least its mean, the others kept
assert!(p[0] >= 0.2 && p[1] == 0.5 && p[2] == 0.25, "{p:?}");
}
}
// red is rebuilt above its mean, towards the neighbours' neutral (as far as the pixel's own blue agrees)
assert!(img.get(24, 16)[0] > 0.3, "{:?}", img.get(24, 16));
// the values alone see nothing clipped here, nor does a mask of another size
let mut plain = base.clone();
assert_eq!(reconstruct(&mut plain, [1.0; 3], 0.99), 0);
assert_eq!(reconstruct_masked(&mut plain, [1.0; 3], 0.99, [1.0; 3], &mask[1..]), 0);
assert!(plain.data == base.data);
}
}
crates/raw/src/lib.rs
+71 -20
View File
@@ -7,7 +7,8 @@
//! - [`RawImage::normalized`] subtracts black, scales white to 1.0 and crops to the active area (applying DNG
//! `OpcodeList1`/`OpcodeList2`); [`demosaic`] turns CFA data into camera-RGB [`Rgb32f`];
//! [`RawImage::develop`] does all of it plus `OpcodeList3` and the default crop; [`RawImage::develop_binned`]
//! produces the same at 1/k of the size straight from the mosaic (previews, thumbnails).
//! produces the same at 1/k of the size straight from the mosaic (previews, thumbnails;
//! [`RawImage::develop_binned_masked`] with the mask of blocks holding clipped samples, for [`highlight`]).
//! - [`color`] implements the DNG colour model (dual-illuminant interpolation, forward matrices, white balance)
//! and produces camera → linear Rec.2020 D65 matrices; [`profile`] reads and applies a DNG's own profile
//! look tables and tone curve, and [`gaintable`] its gain table map (Apple ProRAW's local tone mapping).
@@ -19,7 +20,8 @@
//! packed 2/5/7, the 16-bit words of the oldest bodies), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed: 16-bit words, 12-bit XZ-2 words, 12-bit 16-byte blocks of the E-300/E-330/E-500).
//! [`embedded_preview`] covers these containers' JPEG previews. Variants we can't decode yet (compressed ORF, CR3 unverified marker families / C-RAW configurations) return [`RawError::Unsupported`]; each vendor module documents its sources
//! (public specifications, tag-name documentation, black-box analysis of CC0 samples) and gaps. Non-DNG files carry no
//! colour matrix: [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
//! colour matrix: [`spectral`] has matrices fitted to measured spectral sensitivities for 52 models, and
//! otherwise [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
#![forbid(unsafe_code)]
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
@@ -38,6 +40,7 @@ mod preview;
pub mod profile;
mod saturation;
pub mod semantic;
pub mod spectral;
mod tiffraw;
mod unpack;
mod vendor;
@@ -159,23 +162,28 @@ pub fn probe(bytes: &[u8]) -> Option<RawFormat> {
}
let make = t.find(lightcraft_tiff::tags::MAKE).and_then(|e| e.value.as_str()).unwrap_or_default().to_ascii_uppercase();
let has_cfa = has_raw_ifd(&t);
// Pentax's reader takes the layout from the Exif `CFAPattern` when no IFD has a CFA photometric (uncompressed
// and PackBits PEFs), so for the Pentax family that tag counts as raw evidence too
let pentax_raw = has_cfa || t.exif().is_some_and(|e| e.contains(vendor::EXIF_CFA_PATTERN));
// Samsung's raw IFD uses private compressions 32769..=32773 (`vendor::srw`), 32773 being PackBits elsewhere
let samsung_raw = has_cfa || t.all_ifds().iter().any(|i| i.u16(lightcraft_tiff::tags::COMPRESSION).is_some_and(|c| (32769..=32773).contains(&c)));
if make.starts_with("CANON") && t.ifds.len() >= 4 && t.ifds[3].u16(lightcraft_tiff::tags::COMPRESSION) == Some(6) {
return Some(RawFormat::Cr2);
}
if make.starts_with("NIKON") {
return Some(if has_cfa || t.all_ifds().len() > 1 { RawFormat::Nef } else { RawFormat::Nrw });
if make.starts_with("NIKON") && has_cfa {
return Some(if t.all_ifds().len() > 1 { RawFormat::Nef } else { RawFormat::Nrw });
}
if make.starts_with("SONY") {
if make.starts_with("SONY") && has_cfa {
return Some(RawFormat::Arw);
}
if make.starts_with("PENTAX") || make.starts_with("RICOH") {
if (make.starts_with("PENTAX") || make.starts_with("RICOH")) && pentax_raw {
return Some(RawFormat::Pef);
}
// a Samsung-branded body built on a Pentax design writes a Pentax-style maker note (the note's own magic)
if make.starts_with("SAMSUNG") && has_pentax_maker_note(&t, bytes) {
if make.starts_with("SAMSUNG") && pentax_raw && has_pentax_maker_note(&t, bytes) {
return Some(RawFormat::Pef);
}
if make.starts_with("SAMSUNG") {
if make.starts_with("SAMSUNG") && samsung_raw {
return Some(RawFormat::Srw);
}
if dng::is_plain_cfa_tiff(&t, bytes) {
@@ -194,11 +202,15 @@ fn has_pentax_maker_note(t: &Tiff, bytes: &[u8]) -> bool {
}
/// Whether some IFD is marked as raw: CFA photometric, or a raw-only compression value (99 is not a
/// registered TIFF compression; Leaf MOS files use it for their tiled 16-bit lossless-JPEG raw).
/// registered TIFF compression; Leaf MOS files use it for their tiled 16-bit lossless-JPEG raw),
/// or vendor-specific raw tags (Sony tone curve, CFA pattern, or Pentax compression 65535).
fn has_raw_ifd(t: &Tiff) -> bool {
t.all_ifds().iter().any(|i| {
i.u16(lightcraft_tiff::tags::PHOTOMETRIC) == Some(lightcraft_tiff::tags::photometric::CFA)
|| i.u16(lightcraft_tiff::tags::COMPRESSION).is_some_and(|c| c == 34713 || c == 32767 || c == 32769 || c == 32770 || c == 99)
|| i.u16(lightcraft_tiff::tags::COMPRESSION).is_some_and(|c| matches!(c, 34713 | 32767 | 32769 | 32770 | 99 | 65535))
|| i.contains(0x7010)
|| i.contains(lightcraft_tiff::tags::CFA_PATTERN_EP)
|| i.contains(lightcraft_tiff::tags::CFA_REPEAT_PATTERN_DIM)
})
}
@@ -910,6 +922,25 @@ mod tests {
assert_eq!(probe(&padded(&[rgb_ifd(16, 12), rgb_ifd(400, 300)])), None);
// no dimensions at all
assert_eq!(probe(&padded(&[IfdBuilder::new().with(t::MAKE, Value::Ascii("X".into()))])), None);
// raw evidence without a CFA photometric: Pentax's Exif `CFAPattern` (uncompressed / PackBits PEFs) and
// Samsung's private compressions (here 32773, PackBits elsewhere: NX1, NX500)
let mut pef = rgb_ifd(400, 300);
pef.set(t::MAKE, Value::Ascii("PENTAX Corporation".into()));
let mut exif = exif_size(400, 300);
exif.set(vendor::EXIF_CFA_PATTERN, Value::Undefined(vec![0, 2, 0, 2, 0, 1, 1, 2]));
pef.set_child(t::EXIF_IFD, exif);
assert_eq!(probe(&padded(&[pef])), Some(RawFormat::Pef));
let mut srw = rgb_ifd(400, 300);
srw.set(t::MAKE, Value::Ascii("SAMSUNG".into()));
srw.set(t::COMPRESSION, Value::Short(vec![32773]));
assert_eq!(probe(&padded(&[srw])), Some(RawFormat::Srw));
// camera-authored or exported TIFFs preserving camera Make tags remain ordinary images
for make in ["SONY", "NIKON CORPORATION", "PENTAX", "RICOH", "SAMSUNG"] {
let mut cam = rgb_ifd(400, 300);
cam.set(t::MAKE, Value::Ascii(make.into()));
cam.set_child(t::EXIF_IFD, exif_size(400, 300));
assert_eq!(probe(&padded(&[cam])), None, "{make} TIFF should not be probed as raw");
}
}
// --- containers that are recognised but not decoded, with a preview ---
@@ -1077,22 +1108,36 @@ mod tests {
assert_eq!(r.cfa.unwrap().name(), "RGGB");
}
/// A lossless-JPEG scan whose predictor selection value is outside 0 to 7 (the 3FR/FFF files use 8) is
/// not a coding the lossless decoder knows: the file stays an undecodable raw, with that as the reason,
/// and keeps the stand-in preview an unsupported container gets.
/// Hasselblad's own lossless-JPEG coding (selection value 8, as in FFF and most compressed 3FR files) of the
/// same layout decodes: the file is read like the uncompressed one.
#[test]
fn selection_value_8_cfa_tiff_decodes() {
let strip = ljpeg::tests::encode_hasselblad(&cfa_samples(), 8, 6);
let bytes = plain_cfa_tiff(7, strip, None);
assert_eq!(probe(&bytes), Some(RawFormat::CfaTiff));
let r = decode(&bytes).unwrap();
assert_eq!((r.format, r.width, r.height), (RawFormat::CfaTiff, 8, 6));
assert_eq!(r.data, RawData::U16(cfa_samples()));
assert_eq!(r.cfa.unwrap().name(), "RGGB");
assert_eq!(probe_info(&bytes).unwrap().format, RawFormat::CfaTiff);
}
/// A lossless-JPEG scan whose predictor selection value is neither 0 to 7 (T.81) nor 8 (Hasselblad) is not a
/// coding the lossless decoder knows: the file stays an undecodable raw, with that as the reason, and keeps the
/// stand-in preview an unsupported container gets.
#[test]
fn cfa_tiff_with_an_unknown_predictor_stays_undecodable() {
let mut strip = ljpeg::encode(&cfa_samples(), 8, 6, 1, 16, 1, 0);
let sos = strip.windows(2).position(|w| w == [0xff, 0xda]).unwrap();
let ss = sos + 2 + 2 + 1 + 2;
assert_eq!(strip[ss], 1, "selection value byte");
strip[ss] = 8;
assert!(matches!(ljpeg::decode(&strip, 1 << 20), Err(RawError::Unsupported(w)) if w.contains("selection value 8")));
strip[ss] = 9;
assert!(matches!(ljpeg::decode(&strip, 1 << 20), Err(RawError::Unsupported(w)) if w.contains("selection value 9")));
let bytes = plain_cfa_tiff(7, strip, None);
assert_eq!(probe(&bytes), Some(RawFormat::OtherTiff));
assert!(!RawFormat::OtherTiff.is_supported());
let Err(RawError::Unsupported(why)) = decode(&bytes) else { panic!("expected Unsupported") };
assert!(why.contains("lossless JPEG") && why.contains("selection value 8"), "{why}");
assert!(why.contains("lossless JPEG") && why.contains("selection value 9"), "{why}");
}
/// Other codings of a CFA IFD (here Deflate) are not claimed by the plain reader.
@@ -1106,17 +1151,23 @@ mod tests {
/// Samsung file stays with the Samsung format.
#[test]
fn samsung_make_with_a_pentax_maker_note_is_pef() {
let with_note = |note: &[u8]| {
let with_note = |note: &[u8], cfa: bool| {
let mut ifd = rgb_ifd(16, 12);
if cfa {
ifd.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::CFA]));
}
ifd.set(t::MAKE, Value::Ascii("SAMSUNG TECHWIN".into()));
let mut exif = IfdBuilder::new();
exif.set(t::MAKER_NOTE, Value::Undefined(note.to_vec()));
ifd.set_child(t::EXIF_IFD, exif);
write(&[ifd])
};
assert_eq!(probe(&with_note(b"AOC\0MM\0\x01\0\0\0\0\0\0")), Some(RawFormat::Pef));
assert_eq!(probe(&with_note(b"PENTAX \0MM\0\0\0\0\0\0\0")), Some(RawFormat::Pef));
assert_eq!(probe(&with_note(b"STMN100\0\0\0\0\0\0\0\0\0")), Some(RawFormat::Srw));
assert_eq!(probe(&with_note(b"AOC\0MM\0\x01\0\0\0\0\0\0", true)), Some(RawFormat::Pef));
assert_eq!(probe(&with_note(b"PENTAX \0MM\0\0\0\0\0\0\0", true)), Some(RawFormat::Pef));
assert_eq!(probe(&with_note(b"STMN100\0\0\0\0\0\0\0\0\0", true)), Some(RawFormat::Srw));
// without raw evidence (an RGB image from such a body) neither reader claims it (#281)
assert_eq!(probe(&with_note(b"AOC\0MM\0\x01\0\0\0\0\0\0", false)), None);
assert_eq!(probe(&with_note(b"STMN100\0\0\0\0\0\0\0\0\0", false)), None);
}
#[test]
crates/raw/src/ljpeg.rs
+316 -4
View File
@@ -5,6 +5,8 @@
//! restart intervals, `SSSS = 16` differences.
//! - [`decode`] returns the frame as interleaved samples in raster order (`width × height × components`),
//! which callers re-tile (DNG tiles, CR2 slices).
//! - Selection value 8, which T.81 leaves undefined, is Hasselblad's coding in 3FR and FFF files
//! ([`decode`] reads it; [`decode_hasselblad`] documents the rule and how it was established).
//! - [`encode`] writes a conforming stream with per-component optimal Huffman tables built with the
//! T.81 Annex K.2 procedure — used by tests (bit-exact round trips) and later by DNG export.
@@ -81,7 +83,7 @@ impl Huffman {
}
#[inline]
pub fn decode(&self, br: &mut BitReader) -> Result<u8, RawError> {
pub fn decode<B: Bits>(&self, br: &mut B) -> Result<u8, RawError> {
let peek = br.peek(LOOKUP);
let (len, sym) = self.fast[peek as usize];
if len > 0 {
@@ -103,6 +105,33 @@ impl Huffman {
}
}
/// A source of entropy-coded bits, first bit most significant.
pub trait Bits {
/// The next `n` (1..=32) bits without consuming them; zeros past the end.
fn peek(&mut self, n: u32) -> u32;
fn consume(&mut self, n: u32);
#[inline]
fn get(&mut self, n: u32) -> u32 {
if n == 0 {
return 0;
}
let v = self.peek(n);
self.consume(n);
v
}
}
impl Bits for BitReader<'_> {
#[inline]
fn peek(&mut self, n: u32) -> u32 {
BitReader::peek(self, n)
}
#[inline]
fn consume(&mut self, n: u32) {
BitReader::consume(self, n)
}
}
/// MSB-first bit reader over JPEG entropy-coded data (handles `FF 00` stuffing; stops at markers, feeding zeros).
pub struct BitReader<'a> {
data: &'a [u8],
@@ -190,7 +219,7 @@ impl<'a> BitReader<'a> {
/// Difference value for category `ssss` (T.81 Table H.2 / F.12 EXTEND).
#[inline]
pub fn diff_value(br: &mut BitReader, ssss: u8) -> i32 {
pub fn diff_value<B: Bits>(br: &mut B, ssss: u8) -> i32 {
match ssss {
0 => 0,
16 => 32768,
@@ -325,9 +354,9 @@ fn parse_header(d: &[u8]) -> Result<Header, RawError> {
}
h.predictor = seg[1 + 2 * ns];
h.pt = seg[3 + 2 * ns] & 15;
if h.predictor > 7 {
if h.predictor > 8 {
return Err(RawError::Unsupported(format!(
"lossless JPEG scan with predictor selection value {} (T.81 defines 0 to 7)",
"lossless JPEG scan with predictor selection value {} (T.81 defines 0 to 7, Hasselblad uses 8)",
h.predictor
)));
}
@@ -347,6 +376,9 @@ fn parse_header(d: &[u8]) -> Result<Header, RawError> {
pub fn frame_info(d: &[u8]) -> Result<(usize, usize, usize, u8), RawError> {
let h = parse_header(d)?;
check_full_resolution(&h)?;
if h.predictor == 8 {
check_hasselblad(&h)?;
}
Ok((h.width, h.height, h.comps.len(), h.precision))
}
@@ -357,6 +389,139 @@ fn check_full_resolution(h: &Header) -> Result<(), RawError> {
Ok(())
}
/// The layout of every Hasselblad selection-value-8 scan measured; other layouts are not known to follow the rule.
fn check_hasselblad(h: &Header) -> Result<(), RawError> {
if h.comps.len() != 1 || h.precision != 16 || h.pt != 0 || h.restart != 0 || h.width == 0 || !h.width.is_multiple_of(2) {
return Err(RawError::Unsupported(format!(
"lossless JPEG with selection value 8 and {} components, {} bits, point transform {}, restart interval {}, width {}",
h.comps.len(),
h.precision,
h.pt,
h.restart,
h.width
)));
}
Ok(())
}
/// Bits of Hasselblad's entropy data: 32-bit little-endian words, each read from its most significant bit, with
/// no byte stuffing (every byte is data; `FF` bytes are not markers).
struct WordBits<'a> {
data: &'a [u8],
pos: usize,
buf: u64,
bits: u32,
/// Zero bits fed past the last whole word (consumption reached them once `overrun > bits`).
overrun: u32,
}
impl<'a> WordBits<'a> {
fn new(data: &'a [u8]) -> Self {
Self { data, pos: 0, buf: 0, bits: 0, overrun: 0 }
}
#[inline]
fn fill(&mut self) {
while self.bits <= 32 {
let word = match self.data.get(self.pos..self.pos + 4) {
Some(w) => {
self.pos += 4;
u32::from_le_bytes([w[0], w[1], w[2], w[3]])
}
None => {
self.overrun = self.overrun.saturating_add(32);
0
}
};
self.buf |= (word as u64) << (32 - self.bits);
self.bits += 32;
}
}
}
impl Bits for WordBits<'_> {
#[inline]
fn peek(&mut self, n: u32) -> u32 {
if self.bits < n {
self.fill();
}
(self.buf >> (64 - n)) as u32
}
#[inline]
fn consume(&mut self, n: u32) {
self.buf <<= n;
self.bits -= n;
}
}
/// Hasselblad's lossless JPEG with selection value 8 (3FR and FFF raws).
///
/// The rule was established by measurement on 14 sample files with the `ljpeg-explorer` tool, which tested it
/// against alternative bit orders, stuffing, sample groupings, category-16 treatments and predictors:
/// - the entropy data are 32-bit little-endian words, each read from its most significant bit, without byte
/// stuffing;
/// - each row is coded in pairs of samples: both Huffman categories (`SSSS`, T.81 table) come first, then both
/// differences;
/// - a difference has `SSSS` extra bits, extended as in T.81 Table H.2; after `SSSS = 16` come sixteen bits that
/// take no part, and the difference is -32768 (equal to T.81's +32768 modulo 2^16);
/// - a sample is the previous sample of the same column parity in its row plus the difference, modulo 2^16; before
/// the first pair of every row both are 32768 (half the 16-bit range).
///
/// Provenance: the first hypothesis for this rule was recalled by the AI model that wrote this code from its training
/// data, which includes other raw converters; it was treated as one candidate, corrected where the files disagreed
/// (the byte order of the words) and confirmed by an exhaustive black-box search over 17,280 layout hypotheses on the
/// CC0 sample files. No other converter's source was consulted, and the code is original.
pub fn decode_hasselblad(d: &[u8], max_samples: usize) -> Result<Frame, RawError> {
let h = parse_header(d)?;
if h.predictor != 8 {
return Err(err("not a selection value 8 scan"));
}
hasselblad_frame(d, &h, max_samples)
}
fn hasselblad_frame(d: &[u8], h: &Header, max_samples: usize) -> Result<Frame, RawError> {
check_hasselblad(h)?;
let (w, ht) = (h.width, h.height);
let total = w.checked_mul(ht).ok_or_else(|| err("frame too large"))?;
if total > max_samples {
return Err(RawError::Limit("lossless JPEG frame larger than expected"));
}
let entropy = &d[h.scan_start..];
if (entropy.len() as u64 + 64) * 8 < total as u64 {
return Err(err("entropy data too short for frame"));
}
let table =
h.table_for.first().and_then(|&t| h.tables.get(t)).and_then(Option::as_ref).ok_or_else(|| err("scan uses an undefined Huffman table"))?;
let mut out = vec![0u16; total];
let mut br = WordBits::new(entropy);
let start = 1u32 << (h.precision - 1);
for row in out.chunks_exact_mut(w) {
let mut last = [start, start];
// the width is even (checked), so every sample is in a pair
for pair in row.as_chunks_mut::<2>().0 {
let ssss = [table.decode(&mut br)?, table.decode(&mut br)?];
for (k, s) in ssss.into_iter().enumerate() {
let diff = match s {
0..=15 => diff_value(&mut br, s),
16 => {
br.get(16);
-32768
}
_ => return Err(err("invalid difference category")),
};
let v = (last[k] as i32 + diff) as u32 & 0xffff;
last[k] = v;
pair[k] = v as u16;
}
}
// bits consumed beyond the last whole word
if br.overrun > br.bits {
return Err(err("entropy data exhausted"));
}
}
Ok(Frame { width: w, height: ht, components: 1, precision: h.precision, predictor: 8, point_transform: 0, data: out })
}
fn check_subsampled(h: &Header) -> Result<usize, RawError> {
let vertical = match h.sampling.as_slice() {
[0x22, 0x11, 0x11] => 2,
@@ -481,6 +646,9 @@ pub(crate) fn decode_subsampled(d: &[u8], max_samples: usize, prediction: Predic
pub fn decode(d: &[u8], max_samples: usize) -> Result<Frame, RawError> {
let h = parse_header(d)?;
check_full_resolution(&h)?;
if h.predictor == 8 {
return hasselblad_frame(d, &h, max_samples);
}
let (w, ht, nc) = (h.width, h.height, h.comps.len());
if w == 0 || ht == 0 {
return Err(err("zero frame size"));
@@ -988,6 +1156,150 @@ pub(crate) mod tests {
let _ = decode(&bad, usize::MAX);
}
// --- selection value 8 (Hasselblad 3FR / FFF) ---
/// SOI, SOF3 (16 bits, `height` x `width`, one component), a DHT and an SOS with selection value 8, then
/// `entropy` and EOI.
fn hasselblad_stream(width: u16, height: u16, counts: &[u8; 16], values: &[u8], entropy: &[u8]) -> Vec<u8> {
let mut out = vec![0xff, 0xd8, 0xff, 0xc3, 0, 11, 16];
out.extend_from_slice(&height.to_be_bytes());
out.extend_from_slice(&width.to_be_bytes());
out.extend_from_slice(&[1, 0, 0x11, 0]);
out.extend_from_slice(&[0xff, 0xc4]);
out.extend_from_slice(&((2 + 17 + values.len()) as u16).to_be_bytes());
out.push(0);
out.extend_from_slice(counts);
out.extend_from_slice(values);
out.extend_from_slice(&[0xff, 0xda, 0, 8, 1, 0, 0, 8, 0, 0]);
out.extend_from_slice(entropy);
out.extend_from_slice(&[0xff, 0xd9]);
out
}
/// Codes: SSSS 0 = `00`, 1 = `01`, 2 = `100`, 15 = `101`, 16 = `110`.
const HAND_COUNTS: [u8; 16] = [0, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0];
const HAND_VALUES: [u8; 5] = [0, 1, 2, 15, 16];
/// A 4 x 2 frame written out bit by bit:
/// - row 0, pair 1: codes `110` `110` (SSSS 16 twice), then 16 bits each (`FFFF`, `7FFF`) that do not count:
/// both samples are 32768 - 32768 = 0;
/// - row 0, pair 2: codes `01` `100`, bits `1` (+1) and `01` (-2): 0 + 1 = 1 and 0 - 2 = 65534 (modulo 2^16);
/// - row 1, pair 1: codes `00` `01`, bit `0` (-1): both predictors start again at 32768, so 32768 and 32767;
/// - row 1, pair 2: codes `100` `00`, bits `11` (+3): 32771 and 32767.
///
/// 58 bits, padded with zeros to two 32-bit words `DBFFFDFF` `FD9450C0`, stored little-endian. The `FF FD`
/// they start with would be a marker to a T.81 reader: this coding has no byte stuffing.
const HAND_WORDS: [u8; 8] = [0xff, 0xfd, 0xff, 0xdb, 0xc0, 0x50, 0x94, 0xfd];
const HAND_SAMPLES: [u16; 8] = [0, 0, 1, 65534, 32768, 32767, 32771, 32767];
#[test]
fn selection_value_8_known_answer() {
let enc = hasselblad_stream(4, 2, &HAND_COUNTS, &HAND_VALUES, &HAND_WORDS);
assert_eq!(frame_info(&enc).unwrap(), (4, 2, 1, 16));
let f = decode(&enc, 8).unwrap();
assert_eq!((f.width, f.height, f.components, f.predictor), (4, 2, 1, 8));
assert_eq!(f.data, HAND_SAMPLES);
assert_eq!(decode_hasselblad(&enc, 8).unwrap().data, HAND_SAMPLES);
assert!(matches!(decode(&enc, 7), Err(RawError::Limit(_))));
}
/// The same bits in plain byte order (not 32-bit little-endian words) are a different stream.
#[test]
fn selection_value_8_reads_little_endian_words() {
let mut plain = HAND_WORDS;
plain[..4].reverse();
plain[4..].reverse();
let enc = hasselblad_stream(4, 2, &HAND_COUNTS, &HAND_VALUES, &plain);
assert_ne!(decode(&enc, 8).map(|f| f.data).ok(), Some(HAND_SAMPLES.to_vec()));
}
#[test]
fn selection_value_8_rejects_bad_streams() {
// one word is not enough for the second row
let short = hasselblad_stream(4, 2, &HAND_COUNTS, &HAND_VALUES, &HAND_WORDS[..4]);
assert!(decode(&short, 8).is_err());
// a category above 16
let enc = hasselblad_stream(4, 2, &HAND_COUNTS, &[0, 1, 2, 15, 17], &HAND_WORDS);
assert!(decode(&enc, 8).is_err());
// layouts never seen with selection value 8 are not guessed at
let mut odd = hasselblad_stream(3, 2, &HAND_COUNTS, &HAND_VALUES, &HAND_WORDS);
assert!(matches!(frame_info(&odd), Err(RawError::Unsupported(_))));
assert!(matches!(decode(&odd, 8), Err(RawError::Unsupported(_))));
odd = hasselblad_stream(4, 2, &HAND_COUNTS, &HAND_VALUES, &HAND_WORDS);
odd[6] = 14; // precision
assert!(matches!(decode(&odd, 8), Err(RawError::Unsupported(_))));
// selection values above 8 stay unknown
let mut nine = hasselblad_stream(4, 2, &HAND_COUNTS, &HAND_VALUES, &HAND_WORDS);
let sos = nine.windows(2).position(|w| w == [0xff, 0xda]).unwrap();
nine[sos + 7] = 9;
assert!(matches!(decode(&nine, 8), Err(RawError::Unsupported(w)) if w.contains("selection value 9")));
// every prefix fails cleanly
let enc = hasselblad_stream(4, 2, &HAND_COUNTS, &HAND_VALUES, &HAND_WORDS);
for n in 0..enc.len() {
let _ = decode(&enc[..n], 8);
}
}
/// Test-only writer of the selection-value-8 coding (the inverse of the rule in [`decode_hasselblad`]), with
/// a 5-bit code for every category. Category-16 differences carry `FFFF` as their ignored bits.
pub(crate) fn encode_hasselblad(data: &[u16], width: usize, height: usize) -> Vec<u8> {
assert!(width.is_multiple_of(2) && data.len() == width * height);
let mut bits: Vec<bool> = vec![];
let put = |bits: &mut Vec<bool>, v: u32, n: u32| {
for i in (0..n).rev() {
bits.push((v >> i) & 1 == 1);
}
};
for row in data.chunks_exact(width) {
let mut last = [32768i32, 32768];
for pair in row.as_chunks::<2>().0 {
let mut d = [0i32; 2];
for k in 0..2 {
let mut v = (pair[k] as i32 - last[k]) & 0xffff;
if v >= 32768 {
v -= 65536;
}
d[k] = v;
last[k] = pair[k] as i32;
}
let s = [ssss_of(d[0]), ssss_of(d[1])];
put(&mut bits, s[0] as u32, 5);
put(&mut bits, s[1] as u32, 5);
for k in 0..2 {
match s[k] {
0 => {}
16 => put(&mut bits, 0xffff, 16),
n => {
let v = if d[k] < 0 { d[k] - 1 } else { d[k] };
put(&mut bits, v as u32 & ((1 << n) - 1), n as u32);
}
}
}
}
}
let mut entropy = vec![];
for word in bits.chunks(32) {
let w = word.iter().enumerate().fold(0u32, |a, (i, &b)| a | ((b as u32) << (31 - i)));
entropy.extend_from_slice(&w.to_le_bytes());
}
let mut counts = [0u8; 16];
counts[4] = 17;
let values: Vec<u8> = (0..=16).collect();
hasselblad_stream(width as u16, height as u16, &counts, &values, &entropy)
}
#[test]
fn selection_value_8_round_trips_through_the_test_writer() {
// extremes force SSSS 16 and wrap-around; noise exercises every other category
let mut data = noise(10 * 7, 16, 9);
data[0] = 0;
data[1] = 65535;
data[2] = 65535;
data[3] = 0;
let enc = encode_hasselblad(&data, 10, 7);
assert_eq!(decode(&enc, 70).unwrap().data, data);
}
#[test]
fn code_lengths_are_limited_to_16() {
// Fibonacci-like frequencies produce very long codes before adjustment
crates/raw/src/spectral.rs
+171
View File
@@ -0,0 +1,171 @@
//! Camera colour matrices from measured spectral sensitivities, for raws that carry no colour matrix.
//!
//! [`find`] looks a camera up by its Exif make and model in a table generated by
//! `tools/spectral_camera_table.py` from rawtoaces-data (Academy Software Foundation, Apache-2.0): per model, DNG
//! `ColorMatrix1/2` and `ForwardMatrix1/2` for Standard Illuminant A and D65, fitted to the camera's measured spectral
//! sensitivities. [`Camera::fill`] writes them into a file's [`ColorData`], and [`crate::color`]'s DNG model does the
//! rest. Other names of a covered camera (Rebel T3i for EOS 600D) come from rawtoaces-data's alias list. No number
//! comes from Adobe or from a GPL raw decoder. Which colour a raw starts from is the engine's choice
//! (`docs/camera-preview-colour.md`).
use crate::ColorData;
use lightcraft_color::Mat3;
#[path = "spectral_table.rs"]
mod table;
/// DNG `CalibrationIlluminant` codes of the two matrices: Standard Illuminant A, D65.
const ILLUMINANTS: [u16; 2] = [17, 21];
/// One camera's colour matrices.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct Camera {
pub make: &'static str,
pub model: &'static str,
/// XYZ → camera under Standard Illuminant A and D65 (`ColorMatrix1/2`).
pub color_matrix: [Mat3; 2],
/// White-balanced camera → XYZ D50 (`ForwardMatrix1/2`).
pub forward_matrix: [Mat3; 2],
}
impl Camera {
/// Give `color` this camera's matrices, when it has no colour matrix of its own. Returns whether it did.
pub fn fill(&self, color: &mut ColorData) -> bool {
if crate::color::has_matrix(color) {
return false;
}
color.illuminant = ILLUMINANTS;
color.color_matrix = self.color_matrix.map(Some);
color.forward_matrix = self.forward_matrix.map(Some);
true
}
}
/// Model names compared without case, the brand, a leading `EOS` or a trailing `DIGITAL`
/// (Exif "Canon EOS 400D DIGITAL" is the table's "EOS 400D").
fn key(brand: &str, model: &str) -> String {
let mut words: Vec<String> = model.split_whitespace().map(str::to_uppercase).collect();
if words.first().is_some_and(|w| *w == brand) {
words.remove(0);
}
if words.first().is_some_and(|w| w == "EOS") {
words.remove(0);
}
if words.len() > 1 && words.last().is_some_and(|w| w == "DIGITAL") {
words.pop();
}
words.join(" ")
}
/// First word of a make, upper case ("NIKON CORPORATION" → "NIKON").
fn brand(make: &str) -> String {
make.split_whitespace().next().unwrap_or_default().to_uppercase()
}
/// The table's camera for a file's Exif make and model, directly or through an alias. Without a make, the model
/// alone must match.
pub fn find(make: Option<&str>, model: &str) -> Option<&'static Camera> {
let make = make.map(brand).filter(|b| !b.is_empty());
let same = |row_make: &str, row_model: &str| {
let row_brand = brand(row_make);
let wanted = key(&row_brand, model);
make.as_ref().is_none_or(|m| *m == row_brand) && !wanted.is_empty() && key(&row_brand, row_model) == wanted
};
match table::ALIASES.iter().find(|(m, alias, _)| same(m, alias)) {
Some((m, _, target)) => table::CAMERAS.iter().find(|c| brand(c.make) == brand(m) && c.model == *target),
None => table::CAMERAS.iter().find(|c| same(c.make, c.model)),
}
}
/// Every camera in the table.
pub fn cameras() -> &'static [Camera] {
table::CAMERAS
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn exif_names_find_their_rows() {
for (make, model, row) in [
("Canon", "Canon EOS 5D Mark III", "EOS 5D Mark III"),
("Canon", "Canon EOS 600D", "EOS 600D"),
("Canon", "Canon EOS-1Ds Mark III", "EOS-1Ds Mark III"),
("Canon", "Canon EOS DIGITAL REBEL XTi", "Digital Rebel XTi"),
("Canon", "Canon EOS R5m2", "EOS R5m2"),
("Canon", "Canon PowerShot S90", "PowerShot S90"),
("NIKON CORPORATION", "NIKON D850", "D850"),
("NIKON CORPORATION", "NIKON Z f", "Z f"),
("SONY", "ILCE-7M3", "ILCE-7M3"),
("FUJIFILM", "X-T4", "X-T4"),
("Panasonic", "DC-GX9", "DC-GX9"),
] {
assert_eq!(find(Some(make), model).map(|c| c.model), Some(row), "{make} {model}");
}
assert_eq!(find(None, "NIKON D850").map(|c| c.model), Some("D850"));
assert_eq!(find(None, "ILCE-7M3").map(|c| c.model), Some("ILCE-7M3"));
}
#[test]
fn aliases_reach_the_camera_they_name() {
for (make, model, row) in [
("Canon", "Canon EOS 400D DIGITAL", "Digital Rebel XTi"),
("Canon", "Canon EOS Kiss Digital X", "Digital Rebel XTi"),
("Canon", "Canon EOS REBEL T3i", "EOS 600D"),
("Canon", "Canon EOS Kiss X5", "EOS 600D"),
("Canon", "Canon EOS R5 Mark II", "EOS R5m2"),
("NIKON CORPORATION", "NIKON D800", "D800E"),
("NIKON CORPORATION", "NIKON Z 6_2", "Z f"),
("SONY", "ILCE-7RM4A", "ILCE-7RM4"),
("FUJIFILM", "GFX100S", "GFX 100"),
] {
assert_eq!(find(Some(make), model).map(|c| c.model), Some(row), "{make} {model}");
}
}
#[test]
fn other_cameras_and_other_makes_find_nothing() {
assert_eq!(find(Some("Canon"), "Canon EOS 7D"), None);
assert_eq!(find(Some("SONY"), "ILCE-7M2"), None);
assert_eq!(find(Some("OLYMPUS IMAGING CORP."), "E-M5"), None);
// the model of another make
assert_eq!(find(Some("Canon"), "D850"), None);
assert_eq!(find(Some("SONY"), "X-T4"), None);
assert_eq!(find(None, ""), None);
assert_eq!(find(Some(""), ""), None);
}
#[test]
fn every_row_is_a_usable_dng_profile() {
assert_eq!(cameras().len(), 52);
for c in cameras() {
assert_eq!(find(Some(c.make), c.model), Some(c), "{} {}", c.make, c.model);
for m in c.color_matrix.iter().chain(&c.forward_matrix) {
assert!(m.0.iter().flatten().all(|v| v.is_finite() && v.abs() < 4.0), "{}", c.model);
assert!(m.inverse().is_some(), "{}", c.model);
}
// a forward matrix maps white-balanced white to D50 (DNG spec, chapter 6)
for fm in &c.forward_matrix {
let w = fm.apply([1.0; 3]);
let d50 = lightcraft_color::D50.to_xyz();
assert!((0..3).all(|i| (w[i] - d50[i]).abs() < 1e-4), "{} {w:?}", c.model);
}
}
}
#[test]
fn fill_keeps_a_files_own_matrices() {
let c = find(Some("Canon"), "Canon EOS 600D").unwrap();
let mut color = ColorData::default();
assert!(c.fill(&mut color));
assert_eq!(color.illuminant, [17, 21]);
assert_eq!(color.color_matrix, [Some(c.color_matrix[0]), Some(c.color_matrix[1])]);
assert_eq!(color.forward_matrix, [Some(c.forward_matrix[0]), Some(c.forward_matrix[1])]);
let own = Mat3([[0.5, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 0.7]]);
let mut color = ColorData { illuminant: [21, 0], color_matrix: [Some(own), None], ..Default::default() };
let before = color.clone();
assert!(!c.fill(&mut color));
assert_eq!(color, before);
}
}
crates/raw/src/spectral_table.rs
+349
View File
@@ -0,0 +1,349 @@
// Generated by tools/spectral_camera_table.py from rawtoaces-data (https://github.com/AcademySoftwareFoundation/rawtoaces-data, commit 5cfa53707b053594b2d0d48e8019fb56e6582d6c).
// Derived from the camera spectral sensitivities of rawtoaces-data (Academy Software Foundation), licensed under
// the Apache License 2.0 (LICENSE-APACHE); see NOTICE and assets/ATTRIBUTION.md. Do not edit by hand.
use super::Camera;
use lightcraft_color::Mat3;
#[rustfmt::skip]
pub(super) const CAMERAS: &[Camera] = &[
Camera {
make: "ARRI",
model: "D21",
color_matrix: [Mat3([[0.933065, -0.03997, 0.042177], [-0.321253, 0.916394, 0.250116], [-0.047929, 0.258088, 0.770888]]), Mat3([[0.841118, 0.11685, -0.05716], [-0.416513, 1.06087, 0.169491], [-0.121062, 0.361236, 0.692345]])],
forward_matrix: [Mat3([[0.913988, -0.102353, 0.152661], [0.315506, 0.849769, -0.165276], [-0.134707, -0.698389, 1.6582]]), Mat3([[1.01361, -0.108757, 0.059444], [0.405694, 0.811177, -0.216871], [-0.027612, -0.326433, 1.17915]])],
},
Camera {
make: "Arashi Vision",
model: "Insta360 X5",
color_matrix: [Mat3([[1.0899, -0.436896, 0.023525], [-0.482767, 1.39879, 0.369626], [-0.056659, 0.169421, 0.72407]]), Mat3([[0.700283, -0.158934, -0.077847], [-0.543955, 1.30291, 0.196622], [-0.148964, 0.25165, 0.516509]])],
forward_matrix: [Mat3([[0.710808, 0.160362, 0.093126], [0.231838, 0.891164, -0.123002], [0.020454, -0.498248, 1.3029]]), Mat3([[0.706099, 0.209581, 0.048616], [0.287186, 0.8971, -0.184286], [0.044158, -0.279999, 1.06095]])],
},
Camera {
make: "Canon",
model: "Digital Rebel XTi",
color_matrix: [Mat3([[0.984721, -0.317618, -0.03823], [-0.536205, 1.41139, 0.499196], [-0.019493, 0.0898, 0.993219]]), Mat3([[0.654378, -0.152128, -0.077165], [-0.504141, 1.17532, 0.279049], [-0.071499, 0.132862, 0.672873]])],
forward_matrix: [Mat3([[0.735947, 0.130604, 0.097745], [0.271991, 0.870186, -0.142177], [-0.013289, -0.229681, 1.06808]]), Mat3([[0.693623, 0.247958, 0.022715], [0.295094, 0.978904, -0.273998], [0.00951, -0.117049, 0.932644]])],
},
Camera {
make: "Canon",
model: "EOS-1D X Mark II",
color_matrix: [Mat3([[0.841821, -0.109388, -0.029826], [-0.469467, 1.34341, 0.484215], [-0.074495, 0.204633, 0.843881]]), Mat3([[0.559829, 0.012974, -0.076841], [-0.474068, 1.17573, 0.252424], [-0.137973, 0.263217, 0.557436]])],
forward_matrix: [Mat3([[0.867342, -0.069209, 0.166163], [0.282942, 0.879178, -0.162119], [0.042308, -0.560464, 1.34326]]), Mat3([[0.866644, -0.0379, 0.135551], [0.340507, 0.913959, -0.254466], [0.035971, -0.324047, 1.11318]])],
},
Camera {
make: "Canon",
model: "EOS-1Ds Mark III",
color_matrix: [Mat3([[0.933464, -0.245919, -0.026807], [-0.571496, 1.49286, 0.379177], [-0.048229, 0.100118, 0.927528]]), Mat3([[0.623037, -0.073582, -0.085177], [-0.54689, 1.28786, 0.213006], [-0.105037, 0.156191, 0.659429]])],
forward_matrix: [Mat3([[0.785005, 0.07525, 0.10404], [0.285685, 0.799015, -0.084701], [0.044611, -0.231741, 1.01223]]), Mat3([[0.764142, 0.106772, 0.093381], [0.318478, 0.835999, -0.154477], [0.03106, -0.128764, 0.922809]])],
},
Camera {
make: "Canon",
model: "EOS-1Ds Mark II",
color_matrix: [Mat3([[0.945891, -0.245875, -0.023592], [-0.58518, 1.49586, 0.412994], [-0.049647, 0.124624, 1.04996]]), Mat3([[0.633076, -0.080316, -0.073882], [-0.552992, 1.27833, 0.227086], [-0.10768, 0.180208, 0.730732]])],
forward_matrix: [Mat3([[0.78531, 0.073316, 0.10567], [0.29637, 0.804671, -0.101041], [0.021036, -0.254656, 1.05872]]), Mat3([[0.780605, 0.119757, 0.063934], [0.332784, 0.851961, -0.184745], [0.021321, -0.137163, 0.940946]])],
},
Camera {
make: "Canon",
model: "EOS 100D",
color_matrix: [Mat3([[0.823379, -0.153284, -0.011827], [-0.451822, 1.34611, 0.422153], [-0.063154, 0.17914, 0.866791]]), Mat3([[0.537713, -0.001778, -0.071702], [-0.479729, 1.20511, 0.230384], [-0.135862, 0.245077, 0.608091]])],
forward_matrix: [Mat3([[0.839709, -0.013024, 0.137611], [0.260264, 0.869968, -0.130232], [0.040567, -0.473274, 1.25781]]), Mat3([[0.852444, -0.005311, 0.117164], [0.33135, 0.880249, -0.211599], [0.038232, -0.260159, 1.04703]])],
},
Camera {
make: "Canon",
model: "EOS 200D",
color_matrix: [Mat3([[0.882798, -0.11491, -0.016841], [-0.443662, 1.31235, 0.491834], [-0.062151, 0.206192, 0.798346]]), Mat3([[0.570466, 0.029473, -0.079371], [-0.461712, 1.15876, 0.257222], [-0.118587, 0.24972, 0.538517]])],
forward_matrix: [Mat3([[0.867592, -0.074026, 0.17073], [0.27684, 0.904905, -0.181745], [0.007997, -0.615366, 1.43247]]), Mat3([[0.884048, -0.067045, 0.147293], [0.347584, 0.917256, -0.26484], [0.021454, -0.322825, 1.12647]])],
},
Camera {
make: "Canon",
model: "EOS 200D II",
color_matrix: [Mat3([[0.820814, -0.08112, -0.025232], [-0.449049, 1.31524, 0.500348], [-0.063522, 0.20839, 0.807016]]), Mat3([[0.534782, 0.039491, -0.075304], [-0.461016, 1.15287, 0.26203], [-0.117011, 0.248633, 0.539767]])],
forward_matrix: [Mat3([[0.879874, -0.096427, 0.180849], [0.283378, 0.900453, -0.183831], [0.006798, -0.61219, 1.4305]]), Mat3([[0.897644, -0.089573, 0.156225], [0.354938, 0.914467, -0.269405], [0.020307, -0.323555, 1.12835]])],
},
Camera {
make: "Canon",
model: "EOS 5DS",
color_matrix: [Mat3([[0.784071, -0.14278, -0.018109], [-0.493528, 1.36672, 0.49299], [-0.067845, 0.178082, 0.792129]]), Mat3([[0.501948, -0.005161, -0.069927], [-0.500698, 1.19504, 0.25793], [-0.125608, 0.228657, 0.525639]])],
forward_matrix: [Mat3([[0.838414, -0.019978, 0.145859], [0.285256, 0.874752, -0.160008], [0.038705, -0.507361, 1.29376]]), Mat3([[0.840997, -0.003166, 0.126464], [0.342942, 0.904091, -0.247033], [0.034419, -0.289165, 1.07985]])],
},
Camera {
make: "Canon",
model: "EOS 5D",
color_matrix: [Mat3([[0.961758, -0.26807, -0.006664], [-0.565407, 1.47679, 0.405541], [-0.067833, 0.12912, 0.951998]]), Mat3([[0.616895, -0.064661, -0.085183], [-0.554321, 1.28625, 0.22097], [-0.137616, 0.197613, 0.656267]])],
forward_matrix: [Mat3([[0.79062, 0.078103, 0.095573], [0.287453, 0.812941, -0.100394], [0.068474, -0.279714, 1.03635]]), Mat3([[0.77434, 0.091394, 0.098562], [0.324069, 0.841224, -0.165293], [0.044245, -0.169979, 0.950838]])],
},
Camera {
make: "Canon",
model: "EOS 5D Mark III",
color_matrix: [Mat3([[0.798227, -0.156522, -0.024001], [-0.479014, 1.37953, 0.412187], [-0.067815, 0.178094, 0.795854]]), Mat3([[0.536462, -0.020759, -0.074508], [-0.493492, 1.22724, 0.222071], [-0.136505, 0.242956, 0.547031]])],
forward_matrix: [Mat3([[0.829015, -0.007278, 0.142558], [0.266702, 0.857593, -0.124295], [0.048011, -0.488746, 1.26584]]), Mat3([[0.818929, 0.023372, 0.121994], [0.319742, 0.879191, -0.198933], [0.040465, -0.281513, 1.06615]])],
},
Camera {
make: "Canon",
model: "EOS 5D Mark II",
color_matrix: [Mat3([[0.765281, -0.126067, -0.028016], [-0.539891, 1.44802, 0.407637], [-0.058008, 0.128461, 0.861097]]), Mat3([[0.508976, -0.002163, -0.072663], [-0.524812, 1.25489, 0.224018], [-0.11344, 0.183978, 0.599963]])],
forward_matrix: [Mat3([[0.84396, -0.009524, 0.12986], [0.296711, 0.799464, -0.096175], [0.052873, -0.321603, 1.09383]]), Mat3([[0.8407, 0.001511, 0.122085], [0.343565, 0.82753, -0.171095], [0.035331, -0.182192, 0.971966]])],
},
Camera {
make: "Canon",
model: "EOS 5D Mark IV",
color_matrix: [Mat3([[0.763394, -0.085035, -0.016567], [-0.442907, 1.31347, 0.486351], [-0.080725, 0.247335, 0.884784]]), Mat3([[0.49672, 0.034424, -0.068025], [-0.461551, 1.16055, 0.255439], [-0.154521, 0.307914, 0.579555]])],
forward_matrix: [Mat3([[0.880649, -0.094084, 0.177732], [0.27666, 0.906137, -0.182797], [0.024429, -0.659213, 1.45989]]), Mat3([[0.898336, -0.087734, 0.153694], [0.347611, 0.924078, -0.271689], [0.035769, -0.378779, 1.16812]])],
},
Camera {
make: "Canon",
model: "EOS 600D",
color_matrix: [Mat3([[0.800214, -0.150575, -0.010842], [-0.462694, 1.36091, 0.41413], [-0.054076, 0.17327, 0.853835]]), Mat3([[0.528239, -0.007001, -0.067784], [-0.485979, 1.21691, 0.225002], [-0.13144, 0.244539, 0.592949]])],
forward_matrix: [Mat3([[0.836952, -0.011114, 0.138458], [0.267123, 0.859868, -0.12699], [0.017395, -0.463764, 1.27147]]), Mat3([[0.849461, 0.005947, 0.108888], [0.332065, 0.875561, -0.207627], [0.033552, -0.262503, 1.05405]])],
},
Camera {
make: "Canon",
model: "EOS M",
color_matrix: [Mat3([[0.858133, -0.188655, -0.015316], [-0.477476, 1.37657, 0.415737], [-0.066593, 0.173487, 0.870331]]), Mat3([[0.559337, -0.024286, -0.076489], [-0.498439, 1.22646, 0.227107], [-0.13663, 0.237788, 0.612213]])],
forward_matrix: [Mat3([[0.818185, 0.015661, 0.130449], [0.263684, 0.85776, -0.121444], [0.046738, -0.442459, 1.22083]]), Mat3([[0.820106, 0.031315, 0.112875], [0.32549, 0.874009, -0.199499], [0.03847, -0.243276, 1.02991]])],
},
Camera {
make: "Canon",
model: "EOS R10",
color_matrix: [Mat3([[0.900395, -0.120204, -0.023257], [-0.450784, 1.32079, 0.490112], [-0.060508, 0.207987, 0.757971]]), Mat3([[0.586274, 0.022629, -0.081629], [-0.465757, 1.16444, 0.255537], [-0.114546, 0.247787, 0.505639]])],
forward_matrix: [Mat3([[0.860416, -0.07424, 0.178119], [0.279245, 0.90614, -0.185385], [-0.005252, -0.64772, 1.47808]]), Mat3([[0.874599, -0.057155, 0.146852], [0.346428, 0.920831, -0.267259], [0.018292, -0.341711, 1.14852]])],
},
Camera {
make: "Canon",
model: "EOS R5",
color_matrix: [Mat3([[0.906792, -0.181028, -0.039982], [-0.497007, 1.37995, 0.46653], [-0.056558, 0.135904, 0.790883]]), Mat3([[0.599617, -0.03146, -0.086837], [-0.500836, 1.20715, 0.246929], [-0.122485, 0.204473, 0.529195]])],
forward_matrix: [Mat3([[0.825544, 0.000874, 0.137878], [0.27921, 0.851443, -0.130653], [0.053855, -0.393303, 1.16455]]), Mat3([[0.80269, 0.040879, 0.120727], [0.321664, 0.899552, -0.221215], [0.039913, -0.245426, 1.03062]])],
},
Camera {
make: "Canon",
model: "EOS R5m2",
color_matrix: [Mat3([[0.925997, -0.201878, -0.03247], [-0.514263, 1.39694, 0.472057], [-0.067213, 0.159605, 0.932813]]), Mat3([[0.603973, -0.037874, -0.08705], [-0.520173, 1.22523, 0.247204], [-0.147012, 0.240988, 0.619754]])],
forward_matrix: [Mat3([[0.816853, 0.016859, 0.130583], [0.284699, 0.847793, -0.132492], [0.049678, -0.384186, 1.15961]]), Mat3([[0.796684, 0.049994, 0.117618], [0.326624, 0.8894, -0.216024], [0.040655, -0.243309, 1.02776]])],
},
Camera {
make: "Canon",
model: "EOS R6",
color_matrix: [Mat3([[0.887235, -0.176902, -0.018642], [-0.462429, 1.33965, 0.473061], [-0.070097, 0.19962, 0.862666]]), Mat3([[0.571385, -0.014759, -0.079163], [-0.485367, 1.18708, 0.251862], [-0.145333, 0.265796, 0.580543]])],
forward_matrix: [Mat3([[0.830203, -0.011614, 0.145706], [0.268315, 0.891706, -0.16002], [0.03391, -0.533608, 1.3248]]), Mat3([[0.829826, 0.011021, 0.123449], [0.329107, 0.920067, -0.249174], [0.037653, -0.307101, 1.09455]])],
},
Camera {
make: "Canon",
model: "EOS R6m2",
color_matrix: [Mat3([[0.904651, -0.185945, -0.032292], [-0.490303, 1.37329, 0.46456], [-0.076482, 0.19622, 0.939437]]), Mat3([[0.595351, -0.032062, -0.086354], [-0.499687, 1.20659, 0.246445], [-0.151602, 0.266204, 0.633694]])],
forward_matrix: [Mat3([[0.822515, -0.0028, 0.14458], [0.275534, 0.86495, -0.140483], [0.04532, -0.468236, 1.24802]]), Mat3([[0.80207, 0.038555, 0.123671], [0.322259, 0.905003, -0.227262], [0.03728, -0.271382, 1.05921]])],
},
Camera {
make: "Canon",
model: "EOS RP",
color_matrix: [Mat3([[0.843269, -0.158656, -0.026198], [-0.497933, 1.37758, 0.476056], [-0.070882, 0.179269, 0.839769]]), Mat3([[0.548578, -0.014251, -0.077893], [-0.504763, 1.20691, 0.250579], [-0.139008, 0.243991, 0.560854]])],
forward_matrix: [Mat3([[0.836529, -0.017241, 0.145008], [0.285362, 0.860694, -0.146057], [0.049163, -0.481077, 1.25702]]), Mat3([[0.825922, 0.011858, 0.126516], [0.334357, 0.898585, -0.232942], [0.038621, -0.283664, 1.07015]])],
},
Camera {
make: "Canon",
model: "EOS R",
color_matrix: [Mat3([[0.732886, -0.080769, -0.010303], [-0.454757, 1.32829, 0.481287], [-0.083062, 0.251827, 0.895746]]), Mat3([[0.473635, 0.038459, -0.063761], [-0.474141, 1.17757, 0.250797], [-0.157456, 0.313293, 0.589901]])],
forward_matrix: [Mat3([[0.884185, -0.093978, 0.174089], [0.284166, 0.893986, -0.178152], [0.023242, -0.654769, 1.45663]]), Mat3([[0.908841, -0.095987, 0.151442], [0.356864, 0.902145, -0.259009], [0.034581, -0.371735, 1.16226]])],
},
Camera {
make: "Canon",
model: "PowerShot S90",
color_matrix: [Mat3([[1.37542, -0.592063, 0.037432], [-0.219062, 1.12456, 0.326212], [-0.021707, 0.09788, 0.58587]]), Mat3([[0.90732, -0.244506, -0.087152], [-0.314903, 1.06475, 0.215416], [-0.11223, 0.179775, 0.439924]])],
forward_matrix: [Mat3([[0.67794, 0.203129, 0.083227], [0.105177, 1.01192, -0.1171], [0.055684, -0.464622, 1.23404]]), Mat3([[0.655349, 0.321862, -0.012915], [0.180096, 1.10102, -0.281111], [0.064028, -0.26917, 1.03025]])],
},
Camera {
make: "Fujifilm",
model: "GFX 100",
color_matrix: [Mat3([[1.37208, -0.633889, 0.081365], [-0.430548, 1.32101, 0.427017], [-0.039908, 0.114944, 0.782345]]), Mat3([[0.834968, -0.233255, -0.07589], [-0.475844, 1.18948, 0.241344], [-0.126239, 0.194812, 0.541708]])],
forward_matrix: [Mat3([[0.673446, 0.25104, 0.039809], [0.200798, 0.950696, -0.151494], [0.031013, -0.340478, 1.13457]]), Mat3([[0.679917, 0.304542, -0.020163], [0.261808, 1.00662, -0.268426], [0.044444, -0.214012, 0.994673]])],
},
Camera {
make: "Fujifilm",
model: "X-T3",
color_matrix: [Mat3([[1.29933, -0.598524, 0.075033], [-0.403354, 1.28265, 0.450876], [-0.037817, 0.112326, 0.810909]]), Mat3([[0.785701, -0.221043, -0.071522], [-0.456611, 1.15627, 0.255052], [-0.124341, 0.191243, 0.562458]])],
forward_matrix: [Mat3([[0.671025, 0.255703, 0.037568], [0.190434, 0.97306, -0.163493], [0.03182, -0.334086, 1.12737]]), Mat3([[0.676828, 0.316449, -0.028982], [0.256753, 1.03712, -0.293871], [0.04319, -0.207191, 0.989106]])],
},
Camera {
make: "Fujifilm",
model: "X-T4",
color_matrix: [Mat3([[1.27406, -0.587718, 0.074639], [-0.371617, 1.24808, 0.450058], [-0.038746, 0.113215, 0.780014]]), Mat3([[0.771728, -0.215706, -0.070731], [-0.439625, 1.14183, 0.253491], [-0.125229, 0.192512, 0.542394]])],
forward_matrix: [Mat3([[0.668116, 0.258486, 0.037694], [0.175163, 0.992049, -0.167213], [0.040145, -0.355105, 1.14006]]), Mat3([[0.675162, 0.317733, -0.028599], [0.248581, 1.05006, -0.298641], [0.04702, -0.219918, 0.998002]])],
},
Camera {
make: "Hasselblad",
model: "L1D-20c",
color_matrix: [Mat3([[0.697908, -0.225515, 0.018865], [-0.403519, 1.29608, 0.413637], [-0.048508, 0.167167, 0.74727]]), Mat3([[0.466971, -0.076456, -0.040872], [-0.471572, 1.21777, 0.21163], [-0.133006, 0.246905, 0.516655]])],
forward_matrix: [Mat3([[0.754419, 0.110524, 0.099352], [0.21061, 0.943846, -0.154456], [0.016488, -0.523808, 1.33242]]), Mat3([[0.780313, 0.168783, 0.0152], [0.295267, 0.942833, -0.2381], [0.041, -0.301041, 1.08515]])],
},
Camera {
make: "Hasselblad",
model: "L2D-20c",
color_matrix: [Mat3([[0.787283, -0.249155, 0.044752], [-0.415734, 1.29586, 0.451949], [-0.060535, 0.144901, 0.709626]]), Mat3([[0.506449, -0.071884, -0.038243], [-0.435817, 1.14056, 0.251334], [-0.129637, 0.209508, 0.474062]])],
forward_matrix: [Mat3([[0.776836, 0.120396, 0.067064], [0.218196, 0.951802, -0.169998], [0.071832, -0.4598, 1.21307]]), Mat3([[0.807201, 0.167509, -0.010414], [0.291699, 1.01528, -0.306978], [0.062694, -0.296911, 1.05932]])],
},
Camera {
make: "Nikon",
model: "D200",
color_matrix: [Mat3([[1.37569, -0.560621, 0.01677], [-0.606594, 1.52996, 0.38327], [-0.024329, 0.069398, 0.99135]]), Mat3([[0.898945, -0.249527, -0.086342], [-0.568281, 1.29839, 0.222009], [-0.098369, 0.139456, 0.687094]])],
forward_matrix: [Mat3([[0.708545, 0.18748, 0.068271], [0.274935, 0.818921, -0.093855], [0.012224, -0.169279, 0.98216]]), Mat3([[0.685079, 0.274473, 0.004744], [0.294469, 0.898619, -0.193087], [0.026107, -0.10149, 0.900488]])],
},
Camera {
make: "Nikon",
model: "D3300",
color_matrix: [Mat3([[1.01047, -0.365965, 0.032301], [-0.473579, 1.3513, 0.474736], [-0.075188, 0.164522, 1.02012]]), Mat3([[0.644589, -0.136221, -0.056505], [-0.47509, 1.16499, 0.26318], [-0.15352, 0.237111, 0.681108]])],
forward_matrix: [Mat3([[0.738138, 0.162349, 0.063809], [0.234836, 0.925124, -0.15996], [0.059989, -0.351996, 1.11711]]), Mat3([[0.742449, 0.23561, -0.013763], [0.290358, 1.00587, -0.29623], [0.04507, -0.217376, 0.99741]])],
},
Camera {
make: "Nikon",
model: "D5100",
color_matrix: [Mat3([[1.34772, -0.48474, 0.012024], [-0.466856, 1.29899, 0.44042], [-0.076248, 0.161652, 0.981853]]), Mat3([[0.929942, -0.213704, -0.081538], [-0.481178, 1.17186, 0.262186], [-0.14942, 0.233631, 0.699016]])],
forward_matrix: [Mat3([[0.732398, 0.159354, 0.072543], [0.240376, 0.912366, -0.152741], [0.06111, -0.349037, 1.11303]]), Mat3([[0.727572, 0.254229, -0.017505], [0.287762, 1.00467, -0.292432], [0.040205, -0.206009, 0.990908]])],
},
Camera {
make: "Nikon",
model: "D5300",
color_matrix: [Mat3([[0.943297, -0.302977, 0.012055], [-0.489398, 1.36679, 0.480036], [-0.079912, 0.172098, 1.02217]]), Mat3([[0.613661, -0.114638, -0.052652], [-0.475392, 1.16153, 0.266621], [-0.147109, 0.232899, 0.680179]])],
forward_matrix: [Mat3([[0.75961, 0.127618, 0.077067], [0.24786, 0.911605, -0.159465], [0.062139, -0.362458, 1.12542]]), Mat3([[0.765013, 0.208344, -0.009061], [0.301196, 0.998195, -0.299391], [0.042143, -0.217084, 1.00005]])],
},
Camera {
make: "Nikon",
model: "D7000",
color_matrix: [Mat3([[1.0579, -0.370378, 0.024968], [-0.50908, 1.38998, 0.475636], [-0.075079, 0.158561, 1.00866]]), Mat3([[0.675661, -0.143226, -0.053077], [-0.488456, 1.16985, 0.270384], [-0.137868, 0.21334, 0.681191]])],
forward_matrix: [Mat3([[0.744976, 0.150244, 0.069076], [0.251735, 0.900835, -0.15257], [0.058406, -0.336086, 1.10278]]), Mat3([[0.755556, 0.237153, -0.028413], [0.304411, 0.998446, -0.302857], [0.039148, -0.192875, 0.978832]])],
},
Camera {
make: "Nikon",
model: "D700",
color_matrix: [Mat3([[1.25876, -0.491229, 0.017162], [-0.619342, 1.54791, 0.372191], [-0.050936, 0.098145, 1.01122]]), Mat3([[0.835285, -0.217528, -0.074624], [-0.583051, 1.32613, 0.209425], [-0.121368, 0.168895, 0.708639]])],
forward_matrix: [Mat3([[0.722062, 0.17127, 0.070964], [0.279306, 0.809472, -0.088779], [0.042476, -0.200528, 0.983157]]), Mat3([[0.707886, 0.252255, 0.004155], [0.304416, 0.875438, -0.179854], [0.032977, -0.118624, 0.910752]])],
},
Camera {
make: "Nikon",
model: "D70",
color_matrix: [Mat3([[0.95752, -0.326533, -0.000865], [-0.654123, 1.5628, 0.437699], [-0.01723, 0.05324, 0.899754]]), Mat3([[0.621434, -0.137207, -0.067353], [-0.59081, 1.28888, 0.250406], [-0.066923, 0.098053, 0.633702]])],
forward_matrix: [Mat3([[0.74958, 0.134109, 0.080606], [0.313007, 0.787492, -0.100498], [0.005453, -0.15183, 0.971482]]), Mat3([[0.722318, 0.215673, 0.026305], [0.326263, 0.879351, -0.205614], [0.017282, -0.077471, 0.885294]])],
},
Camera {
make: "Nikon",
model: "D800E",
color_matrix: [Mat3([[0.981723, -0.320813, 0.025417], [-0.500428, 1.37669, 0.486261], [-0.07946, 0.169419, 1.04548]]), Mat3([[0.629114, -0.117451, -0.046422], [-0.483155, 1.16337, 0.27171], [-0.147278, 0.230496, 0.696276]])],
forward_matrix: [Mat3([[0.760723, 0.133423, 0.07015], [0.25382, 0.906695, -0.160515], [0.059962, -0.34837, 1.11351]]), Mat3([[0.780492, 0.212338, -0.028534], [0.311949, 0.998057, -0.310006], [0.041771, -0.208304, 0.991637]])],
},
Camera {
make: "Nikon",
model: "D810",
color_matrix: [Mat3([[0.983274, -0.318832, 0.030267], [-0.51358, 1.39278, 0.481653], [-0.076666, 0.176366, 1.07197]]), Mat3([[0.625688, -0.110113, -0.045912], [-0.497828, 1.18171, 0.267671], [-0.149244, 0.239807, 0.71615]])],
forward_matrix: [Mat3([[0.763991, 0.129679, 0.070625], [0.262526, 0.897486, -0.160012], [0.045849, -0.35404, 1.13329]]), Mat3([[0.791151, 0.196615, -0.023471], [0.322368, 0.977225, -0.299593], [0.038389, -0.209045, 0.995761]])],
},
Camera {
make: "Nikon",
model: "D850",
color_matrix: [Mat3([[1.17359, -0.461087, -0.011934], [-0.607581, 1.52285, 0.4063], [-0.096127, 0.196063, 1.00005]]), Mat3([[0.773836, -0.196466, -0.097677], [-0.609077, 1.35526, 0.205388], [-0.190117, 0.290252, 0.676129]])],
forward_matrix: [Mat3([[0.713588, 0.158895, 0.091813], [0.273041, 0.838318, -0.111359], [0.055293, -0.367196, 1.13701]]), Mat3([[0.669707, 0.227388, 0.0672], [0.291703, 0.871502, -0.163205], [0.042827, -0.229884, 1.01216]])],
},
Camera {
make: "Nikon",
model: "Z f",
color_matrix: [Mat3([[1.16951, -0.415422, 0.033906], [-0.516721, 1.41065, 0.441119], [-0.080466, 0.180954, 1.07672]]), Mat3([[0.749962, -0.151755, -0.063228], [-0.504317, 1.20913, 0.248149], [-0.162888, 0.256933, 0.729432]])],
forward_matrix: [Mat3([[0.744024, 0.151119, 0.069153], [0.253649, 0.888649, -0.142297], [0.049891, -0.351314, 1.12653]]), Mat3([[0.756117, 0.216534, -0.008355], [0.304737, 0.961046, -0.265784], [0.041615, -0.212408, 0.995897]])],
},
Camera {
make: "Panasonic",
model: "DC-GX9",
color_matrix: [Mat3([[0.820911, -0.254983, 0.035979], [-0.372553, 1.25333, 0.438192], [-0.054294, 0.146795, 0.747319]]), Mat3([[0.534195, -0.07728, -0.038219], [-0.400358, 1.10647, 0.251688], [-0.12982, 0.216241, 0.50214]])],
forward_matrix: [Mat3([[0.772645, 0.116685, 0.074966], [0.196365, 0.972505, -0.16887], [0.06062, -0.463162, 1.22765]]), Mat3([[0.805529, 0.178696, -0.01993], [0.275218, 1.04725, -0.322465], [0.060973, -0.29745, 1.06158]])],
},
Camera {
make: "Sony",
model: "DSC-RX100M4",
color_matrix: [Mat3([[0.783848, -0.284096, 0.031033], [-0.411895, 1.30629, 0.410807], [-0.047532, 0.165091, 0.725564]]), Mat3([[0.507376, -0.094837, -0.042497], [-0.488786, 1.23246, 0.213166], [-0.137082, 0.247475, 0.503834]])],
forward_matrix: [Mat3([[0.733625, 0.143965, 0.086705], [0.210364, 0.945779, -0.156144], [0.012886, -0.528963, 1.34118]]), Mat3([[0.767183, 0.193058, 0.004054], [0.296288, 0.94455, -0.240838], [0.043397, -0.304922, 1.08663]])],
},
Camera {
make: "Sony",
model: "ILCE-6400",
color_matrix: [Mat3([[0.686447, -0.204525, 0.026311], [-0.43013, 1.31387, 0.445779], [-0.061279, 0.132669, 0.791917]]), Mat3([[0.44557, -0.064452, -0.027734], [-0.437513, 1.13079, 0.261784], [-0.122323, 0.190304, 0.552427]])],
forward_matrix: [Mat3([[0.782555, 0.106676, 0.075064], [0.224677, 0.924591, -0.149268], [0.078407, -0.377222, 1.12392]]), Mat3([[0.821757, 0.17479, -0.032251], [0.30346, 1.00862, -0.31208], [0.052261, -0.224433, 0.997277]])],
},
Camera {
make: "Sony",
model: "ILCE-7CM2",
color_matrix: [Mat3([[0.758033, -0.221503, 0.036455], [-0.492549, 1.36195, 0.503357], [-0.065665, 0.148645, 0.906293]]), Mat3([[0.482991, -0.070139, -0.027429], [-0.478804, 1.14702, 0.282926], [-0.119226, 0.194381, 0.616628]])],
forward_matrix: [Mat3([[0.788283, 0.10759, 0.068422], [0.261923, 0.909809, -0.171732], [0.053026, -0.362705, 1.13478]]), Mat3([[0.83437, 0.173346, -0.043421], [0.336938, 0.996333, -0.333271], [0.037073, -0.203664, 0.991696]])],
},
Camera {
make: "Sony",
model: "ILCE-7M3",
color_matrix: [Mat3([[0.759047, -0.22787, 0.042531], [-0.502563, 1.37732, 0.491076], [-0.066877, 0.150947, 0.900094]]), Mat3([[0.482867, -0.070899, -0.024574], [-0.489531, 1.16667, 0.274244], [-0.123988, 0.201218, 0.610148]])],
forward_matrix: [Mat3([[0.787201, 0.113043, 0.064052], [0.265374, 0.902211, -0.167585], [0.052744, -0.364541, 1.1369]]), Mat3([[0.840001, 0.174345, -0.05005], [0.340881, 0.981246, -0.322127], [0.039232, -0.209817, 0.995689]])],
},
Camera {
make: "Sony",
model: "ILCE-7M4",
color_matrix: [Mat3([[0.763892, -0.227658, 0.033049], [-0.512687, 1.38412, 0.503225], [-0.063774, 0.141899, 0.91715]]), Mat3([[0.485445, -0.074907, -0.026973], [-0.491937, 1.15934, 0.283071], [-0.115422, 0.185993, 0.622862]])],
forward_matrix: [Mat3([[0.784064, 0.112194, 0.068037], [0.2698, 0.896428, -0.166229], [0.050008, -0.338826, 1.11392]]), Mat3([[0.830062, 0.181159, -0.046925], [0.341401, 0.985843, -0.327244], [0.034749, -0.188826, 0.979182]])],
},
Camera {
make: "Sony",
model: "ILCE-7RM2",
color_matrix: [Mat3([[0.775308, -0.25358, 0.026979], [-0.524545, 1.40154, 0.490876], [-0.060608, 0.132591, 0.920838]]), Mat3([[0.492753, -0.089554, -0.036484], [-0.505928, 1.18284, 0.273705], [-0.116121, 0.182519, 0.626106]])],
forward_matrix: [Mat3([[0.76369, 0.133034, 0.067571], [0.267238, 0.888036, -0.155274], [0.048003, -0.313698, 1.0908]]), Mat3([[0.787687, 0.201628, -0.025019], [0.326839, 0.971594, -0.298433], [0.034173, -0.177854, 0.968786]])],
},
Camera {
make: "Sony",
model: "ILCE-7RM3",
color_matrix: [Mat3([[0.742519, -0.22996, 0.032883], [-0.500197, 1.37662, 0.485744], [-0.063378, 0.142467, 0.898759]]), Mat3([[0.473388, -0.075142, -0.031025], [-0.487453, 1.16732, 0.271829], [-0.119917, 0.192619, 0.610537]])],
forward_matrix: [Mat3([[0.776802, 0.119664, 0.06783], [0.260471, 0.900308, -0.160779], [0.051869, -0.347156, 1.12039]]), Mat3([[0.813753, 0.181916, -0.031373], [0.32902, 0.979656, -0.308676], [0.037764, -0.198487, 0.985828]])],
},
Camera {
make: "Sony",
model: "ILCE-7RM4",
color_matrix: [Mat3([[0.732028, -0.220808, 0.020118], [-0.501296, 1.37502, 0.49365], [-0.061615, 0.13838, 0.885461]]), Mat3([[0.472151, -0.07908, -0.031322], [-0.483014, 1.15528, 0.279019], [-0.111192, 0.18072, 0.604588]])],
forward_matrix: [Mat3([[0.775623, 0.114594, 0.074079], [0.260863, 0.901608, -0.162471], [0.050735, -0.342927, 1.1173]]), Mat3([[0.806038, 0.193021, -0.034763], [0.32686, 0.992356, -0.319216], [0.033831, -0.189102, 0.980375]])],
},
Camera {
make: "Sony",
model: "ILCE-7SM2",
color_matrix: [Mat3([[0.615775, -0.137144, 0.06099], [-0.408209, 1.28402, 0.462018], [-0.084205, 0.196387, 0.85663]]), Mat3([[0.395982, -0.016098, -0.005517], [-0.420931, 1.11058, 0.265879], [-0.162165, 0.267589, 0.576945]])],
forward_matrix: [Mat3([[0.854583, 0.041834, 0.067878], [0.234125, 0.948414, -0.182539], [0.097857, -0.530195, 1.25744]]), Mat3([[0.948357, 0.092307, -0.076369], [0.33668, 1.02973, -0.366408], [0.073834, -0.330955, 1.08223]])],
},
Camera {
make: "Sony",
model: "ILCE-7SM3",
color_matrix: [Mat3([[0.749715, -0.228299, 0.030188], [-0.512062, 1.38442, 0.500457], [-0.072776, 0.169469, 0.933545]]), Mat3([[0.476058, -0.07383, -0.031403], [-0.495874, 1.16904, 0.277606], [-0.134473, 0.222664, 0.625349]])],
forward_matrix: [Mat3([[0.779691, 0.113899, 0.070706], [0.268632, 0.905927, -0.174559], [0.04639, -0.390929, 1.16964]]), Mat3([[0.816745, 0.179015, -0.031464], [0.334824, 0.988848, -0.323672], [0.037938, -0.229363, 1.01653]])],
},
Camera {
make: "Sony",
model: "ILCE-9",
color_matrix: [Mat3([[0.747492, -0.216385, 0.049086], [-0.499768, 1.37142, 0.499037], [-0.071324, 0.164705, 0.951496]]), Mat3([[0.473639, -0.063703, -0.016557], [-0.488372, 1.1591, 0.280181], [-0.129154, 0.213406, 0.651]])],
forward_matrix: [Mat3([[0.796409, 0.106147, 0.061739], [0.26822, 0.906732, -0.174952], [0.050574, -0.378333, 1.15286]]), Mat3([[0.868551, 0.167059, -0.071314], [0.354491, 0.985971, -0.340462], [0.037779, -0.211149, 0.998474]])],
},
];
/// `(make, model as sold, table model)`: other names of a camera in [`CAMERAS`], or of a camera with the same sensor.
#[rustfmt::skip]
pub(super) const ALIASES: &[(&str, &str, &str)] = &[
("Canon", "EOS 400D", "Digital Rebel XTi"), // same model, alternative name
("Canon", "Kiss Digital X", "Digital Rebel XTi"), // same model, alternative name
("Canon", "EOS Rebel SL1", "EOS 100D"), // same model, alternative name
("Canon", "Kiss X7", "EOS 100D"), // same model, alternative name
("Canon", "EOS Rebel SL2", "EOS 200D"), // same model, alternative name
("Canon", "Kiss X9", "EOS 200D"), // same model, alternative name
("Canon", "EOS 250D", "EOS 200D II"), // same model, alternative name
("Canon", "EOS Rebel SL3", "EOS 200D II"), // same model, alternative name
("Canon", "Kiss X10", "EOS 200D II"), // same model, alternative name
("Canon", "EOS Rebel T3i", "EOS 600D"), // same model, alternative name
("Canon", "Kiss X5", "EOS 600D"), // same model, alternative name
("Canon", "EOS R5 Mark II", "EOS R5m2"), // same model, alternative name
("Canon", "EOS 5DS R", "EOS 5DS"), // different model, same sensor
("Canon", "EOS 6D Mark II", "EOS RP"), // different model, same sensor
("Canon", "EOS R5 C", "EOS R5"), // different model, same sensor
("Canon", "EOS R50", "EOS R10"), // different model, same sensor
("Canon", "EOS R8", "EOS R6m2"), // different model, same sensor
("Nikon", "D800", "D800E"), // different model, same sensor
("Nikon", "Z 6_2", "Z f"), // different model, same sensor
("Sony", "ILCE-7RM5", "ILCE-7RM4"), // different model, same sensor
("Sony", "ILCE-7RM4A", "ILCE-7RM4"), // updated model, same sensor
("Sony", "ILCE-7RM3A", "ILCE-7RM3"), // updated model, same sensor
("Fujifilm", "GFX100S", "GFX 100"), // different model, same sensor
];
crates/raw/src/vendor/arw.rs
Vendored
+152 -25
View File
@@ -32,6 +32,10 @@
//! comes from the data. The crop and white balance are NOT addressed here: the file has no maker-note `FullImageSize`
//! and no plain WB tag (they live in its `SR2SubIFD`), so it opens uncropped with unit multipliers.
//!
//! Packed 12-bit ARW (DSLR-A900; Compression 32767 but BitsPerSample 12 and a strip of exactly width × height × 1.5
//! bytes): plain 12-bit samples, two per three bytes, least-significant-bit first ([`unpack_row12`]); linear, black 128
//! (512 on the 14-bit scale), CFA from the file's own pattern tag, crop centred on the frame (see [`default_crop`]).
//!
//! Also: uncompressed 16-bit ARW, and lossless-compressed ARW (Compression 7, ILCE-7M4 and later): LJ92 tiles whose
//! frames hold one 2×2 CFA cell per four-component sample ([`read_quad_tiles`]). Other lossless-JPEG layouts go
//! through the generic TIFF path.
@@ -203,6 +207,25 @@ fn tag2010_wb(model: &str, block: &[u8], order: lightcraft_tiff::ByteOrder) -> O
((0.2..=8.0).contains(&r) && (0.2..=8.0).contains(&b)).then_some([r, 1.0, b])
}
/// Whether a 12-bit, single-strip, Sony-compressed (32767) image is stored as plain packed 12-bit samples: the strip
/// is exactly 1.5 bytes per pixel (width even), so every row is `1.5 · width` bytes with no padding. On the one
/// DSLR-A900 sample this is the whole strip (36 917 760 = 6080 × 4048 × 1.5) and it ends at the end of the file.
fn is_packed12(info: &ImageInfo, strip_count: usize, strip_len: u64) -> bool {
let (w, h) = (info.width as u64, info.height as u64);
info.bits() == 12 && strip_count == 1 && w % 2 == 0 && w > 0 && h > 0 && strip_len == w * h * 3 / 2
}
/// Unpack plain 12-bit samples stored least-significant-bit first: two pixels per three bytes, the first being
/// `b0 | (b1 & 0xf) << 8` and the second `b1 >> 4 | b2 << 4` (measured: same-colour neighbours differ by ~16 codes
/// in this order against ~200 in the most-significant-first order). `row` is `1.5 · out.len()` bytes.
pub(crate) fn unpack_row12(row: &[u8], out: &mut [u16]) {
for (i, b) in row.windows(3).step_by(3).enumerate() {
let Some(px) = out.get_mut(2 * i..2 * i + 2) else { break };
px[0] = u16::from(b[0]) | (u16::from(b[1] & 0xf) << 8);
px[1] = u16::from(b[1] >> 4) | (u16::from(b[2]) << 4);
}
}
/// White-balance gains `[R/G, 1, B/G]` from `WB_RGGBLevels` (R, G, G, B); `None` unless there are four levels
/// with a positive green and plausible ratios.
fn rggb_gains(levels: &[f64]) -> Option<[f32; 3]> {
@@ -462,18 +485,30 @@ fn word16_order(strip: &[u8], bits: u32, file: lightcraft_tiff::ByteOrder) -> li
if n > 0 && over_other == 0 && over_file * 100 > n { other } else { file }
}
/// The image area for files without Sony's crop tags (`0x74c7/0x74c8`, written since about 2017): the DNG-style
/// default crop when the raw IFD has one, else the image size the camera records (see [`recorded_size_crop`]).
fn default_crop(raw: &Ifd, mn: Option<&makernote::MakerNote>, exif_size: Option<(u64, u64)>, w: usize, h: usize) -> Rect {
let full = Rect::new(0, 0, w, h);
/// The raw IFD's DNG-style `DefaultCropOrigin` / `DefaultCropSize`, clipped to the `w × h` frame.
fn dng_default_crop(raw: &Ifd, w: usize, h: usize) -> Option<Rect> {
if let (Some([x, y]), Some([cw, ch])) = (raw.u64s(t::DEFAULT_CROP_ORIGIN).as_deref(), raw.u64s(t::DEFAULT_CROP_SIZE).as_deref())
&& *cw > 0
&& *ch > 0
{
return Rect::new(*x as usize, *y as usize, *cw as usize, *ch as usize).clipped(w, h);
return Some(Rect::new(*x as usize, *y as usize, *cw as usize, *ch as usize).clipped(w, h));
}
None
}
/// The image area for files without Sony's crop tags (`0x74c7/0x74c8`, written since about 2017): the DNG-style
/// default crop when the raw IFD has one, else the image size the camera records (see [`recorded_size_crop`]).
///
/// `centred` places that recorded-size window in the middle of the frame (on even offsets, keeping the CFA phase)
/// instead of at the left: the packed 12-bit frame (DSLR-A900) has no padding at either edge, and its camera JPEG
/// is centred on the frame.
fn default_crop(raw: &Ifd, mn: Option<&makernote::MakerNote>, exif_size: Option<(u64, u64)>, w: usize, h: usize, centred: bool) -> Rect {
let full = Rect::new(0, 0, w, h);
if let Some(crop) = dng_default_crop(raw, w, h) {
return crop;
}
match mn.and_then(|m| m.ifd.u64s(MN_FULL_IMAGE_SIZE)).as_deref() {
Some([fh, fw]) => recorded_size_crop((*fw, *fh), exif_size, w, h).unwrap_or(full),
Some([fh, fw]) => recorded_size_crop((*fw, *fh), exif_size, w, h, centred).unwrap_or(full),
_ => full,
}
}
@@ -486,8 +521,9 @@ fn default_crop(raw: &Ifd, mn: Option<&makernote::MakerNote>, exif_size: Option<
/// top. An in-camera aspect ratio narrower than the frame (16:9) is centred vertically, as measured by registering
/// each camera JPEG on its raw (the ILCE-7SM2's 4240 × 2384 at y 228–232 of 2848; the DSLR-A580's 4912 × 2760 at
/// y 258–262 of 3280; issue #535): its height comes from `FullImageSize` (ILCE-7SM2) or, when that still says
/// 3:2, from an Exif image size of the same width (DSLR-A580).
fn recorded_size_crop(full: (u64, u64), exif: Option<(u64, u64)>, w: usize, h: usize) -> Option<Rect> {
/// 3:2, from an Exif image size of the same width (DSLR-A580). `centred` centres the window in both directions
/// instead (see [`default_crop`]).
fn recorded_size_crop(full: (u64, u64), exif: Option<(u64, u64)>, w: usize, h: usize, centred: bool) -> Option<Rect> {
let (fw, fh) = (usize::try_from(full.0).ok()?, usize::try_from(full.1).ok()?);
// only a plausible trim of the width: never more than 64 pixels, never an enlargement
if fw == 0 || fh == 0 || fw > w || fw.saturating_add(64) < w || fh > h {
@@ -497,11 +533,13 @@ fn recorded_size_crop(full: (u64, u64), exif: Option<(u64, u64)>, w: usize, h: u
Some((ew, eh)) if ew == fw && eh > 0 && eh < fh => eh,
_ => fh,
};
// even offsets keep the CFA phase
let x = if centred { ((w - fw) / 2) & !1 } else { 0 };
if height.saturating_add(64) >= h {
return Some(Rect::new(0, 0, fw, height));
return Some(Rect::new(x, if centred { ((h - height) / 2) & !1 } else { 0 }, fw, height));
}
// an in-camera aspect ratio: at least half the frame's height, centred (an even offset keeps the CFA phase)
(height * 2 >= h).then(|| Rect::new(0, ((h - height) / 2) & !1, fw, height))
// an in-camera aspect ratio: at least half the frame's height, centred
(height * 2 >= h).then(|| Rect::new(x, ((h - height) / 2) & !1, fw, height))
}
fn raw_ifd(tiff: &Tiff) -> Option<&Ifd> {
@@ -528,6 +566,7 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
// archive measure exactly width x height. A200/A230/A350 files carry a strip 4-23 % larger than that (and A290/A390 one
// smaller), which is a different packing, so they are refused rather than read with the wrong layout.
let one_byte_per_sample = chunks.len() == 1 && strip_len == (w * h) as u64;
let packed12 = info.compression == 32767 && is_packed12(&info, chunks.len(), strip_len);
let (data, out_bits) = match info.compression {
7 if linear_rgb => (RawData::U16(read_ycbcr_tiles(bytes, &info, raw, mode)?), 14),
32767 if one_byte_per_sample && mode == Mode::Header => {
@@ -547,6 +586,21 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
});
(RawData::U16(data), 14)
}
32767 if packed12 && mode == Mode::Header => {
chunk_bytes(bytes, &chunks[0]).ok_or_else(|| RawError::Corrupt("raw strip outside file".into()))?;
(RawData::U16(Vec::new()), 12)
}
32767 if packed12 => {
let src = chunk_bytes(bytes, &chunks[0]).ok_or_else(|| RawError::Corrupt("raw strip outside file".into()))?;
let row_bytes = w / 2 * 3;
let mut data = vec![0u16; w * h];
data.par_chunks_mut(w).enumerate().for_each(|(y, out)| {
if let Some(row) = src.get(y * row_bytes..(y + 1) * row_bytes) {
unpack_row12(row, out);
}
});
(RawData::U16(data), 12)
}
32767 => return Err(RawError::Unsupported("Sony ARW version 1 / packed compressed variant (raw strip is not one byte per pixel)".into())),
7 if is_quad_tiled(bytes, &info) => match mode {
Mode::Full => (RawData::U16(read_quad_tiles(bytes, &info)?), bits),
@@ -611,7 +665,7 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
(Some([x, y]), Some([cw, ch])) if *cw > 0 && *ch > 0 => Rect::new(*x as usize, *y as usize, *cw as usize, *ch as usize).clipped(w, h),
_ => {
let exif_size = tiff.exif().and_then(|e| Some((e.u64(t::PIXEL_X_DIMENSION)?, e.u64(t::PIXEL_Y_DIMENSION)?)));
default_crop(raw, mn.as_ref(), exif_size, w, h)
default_crop(raw, mn.as_ref(), exif_size, w, h, packed12)
}
};
let mut metadata = lightcraft_meta::from_tiff(&tiff);
@@ -635,7 +689,19 @@ pub(crate) fn decode(bytes: &[u8], mode: Mode) -> Result<RawImage> {
// makes a neutral surface magenta. WB edits remain relative to this as-shot RGB.
wb_multipliers: if linear_rgb { Some([1.0; 3]) } else { wb },
linearized: false,
opcodes: OpcodeLists::default(),
// The distortion table is centred on the DNG-style default crop when the file has one: on the ILCE-7RM4A
// it starts at x = 32 where Sony's crop tags (the image crop, unchanged) start at 0, and the warp was
// validated against Sony's exports with that centre. The opcode's centre is in active-area coordinates, so
// it stays on the optical centre whichever crop frames the image.
opcodes: OpcodeLists {
list3: if linear_rgb {
Vec::new()
} else {
let geometry = dng_default_crop(raw, w, h).unwrap_or(crop);
super::arw_lens::distortion(&model, raw, Rect::new(0, 0, w, h), geometry).into_iter().collect()
},
..Default::default()
},
metadata,
};
img.validate_for(mode)?;
@@ -684,21 +750,24 @@ mod tests {
#[test]
fn recorded_size_crops_in_camera_aspect_ratios() {
// 3:2: anchored at the top-left, as before (ILCE-7S, DSLR-A700 whose Exif size is the whole frame)
assert_eq!(recorded_size_crop((4240, 2832), Some((4240, 2832)), 4288, 2848), Some(Rect::new(0, 0, 4240, 2832)));
assert_eq!(recorded_size_crop((4272, 2848), Some((4288, 2856)), 4288, 2856), Some(Rect::new(0, 0, 4272, 2848)));
assert_eq!(recorded_size_crop((4240, 2832), None, 4288, 2848), Some(Rect::new(0, 0, 4240, 2832)));
assert_eq!(recorded_size_crop((4240, 2832), Some((4240, 2832)), 4288, 2848, false), Some(Rect::new(0, 0, 4240, 2832)));
assert_eq!(recorded_size_crop((4272, 2848), Some((4288, 2856)), 4288, 2856, false), Some(Rect::new(0, 0, 4272, 2848)));
assert_eq!(recorded_size_crop((4240, 2832), None, 4288, 2848, false), Some(Rect::new(0, 0, 4240, 2832)));
// 16:9 from FullImageSize (ILCE-7SM2) and from the Exif size when FullImageSize says 3:2 (DSLR-A580): centred
assert_eq!(recorded_size_crop((4240, 2384), Some((4240, 2384)), 4288, 2848), Some(Rect::new(0, 232, 4240, 2384)));
assert_eq!(recorded_size_crop((4912, 3264), Some((4912, 2760)), 4928, 3280), Some(Rect::new(0, 260, 4912, 2760)));
assert_eq!(recorded_size_crop((4240, 2384), Some((4240, 2384)), 4288, 2848, false), Some(Rect::new(0, 232, 4240, 2384)));
assert_eq!(recorded_size_crop((4912, 3264), Some((4912, 2760)), 4928, 3280, false), Some(Rect::new(0, 260, 4912, 2760)));
// offsets stay even (CFA phase)
assert_eq!(recorded_size_crop((4240, 2386), None, 4288, 2848).map(|r| r.y), Some(230));
assert_eq!(recorded_size_crop((4240, 2386), None, 4288, 2848, false).map(|r| r.y), Some(230));
// not a crop of this frame: wider than it, far narrower, under half its height, empty, hostile values
assert_eq!(recorded_size_crop((4300, 2832), None, 4288, 2848), None);
assert_eq!(recorded_size_crop((4000, 2832), None, 4288, 2848), None);
assert_eq!(recorded_size_crop((4240, 1000), None, 4288, 2848), None);
assert_eq!(recorded_size_crop((0, 2832), None, 4288, 2848), None);
assert_eq!(recorded_size_crop((u64::MAX, u64::MAX), Some((u64::MAX, 1)), 4288, 2848), None);
assert_eq!(recorded_size_crop((4240, 2832), Some((4240, 0)), 4288, 2848), Some(Rect::new(0, 0, 4240, 2832)));
assert_eq!(recorded_size_crop((4300, 2832), None, 4288, 2848, false), None);
assert_eq!(recorded_size_crop((4000, 2832), None, 4288, 2848, false), None);
assert_eq!(recorded_size_crop((4240, 1000), None, 4288, 2848, false), None);
assert_eq!(recorded_size_crop((0, 2832), None, 4288, 2848, false), None);
assert_eq!(recorded_size_crop((u64::MAX, u64::MAX), Some((u64::MAX, 1)), 4288, 2848, false), None);
assert_eq!(recorded_size_crop((4240, 2832), Some((4240, 0)), 4288, 2848, false), Some(Rect::new(0, 0, 4240, 2832)));
// centred (packed 12-bit DSLR-A900: 6048 x 4032 in the middle of 6080 x 4048)
assert_eq!(recorded_size_crop((6048, 4032), Some((6048, 4032)), 6080, 4048, true), Some(Rect::new(16, 8, 6048, 4032)));
assert_eq!(recorded_size_crop((6046, 4030), None, 6080, 4048, true), Some(Rect::new(16, 8, 6046, 4030)));
}
/// A 32767-compressed ARW whose single strip is `strip` bytes for a 32 x 4 image.
@@ -732,6 +801,64 @@ mod tests {
}
}
/// A 32767-compressed, 12-bit ARW of `w` x `h` whose single strip is `strip`.
fn packed_arw(w: u32, h: u32, strip: Vec<u8>) -> Vec<u8> {
use lightcraft_tiff::{IfdBuilder, ImageData, TiffWriter, Value};
let mut raw = IfdBuilder::new();
raw.set(t::MAKE, Value::Ascii("SONY".into()));
raw.set(t::MODEL, Value::Ascii("DSLR-A900".into()));
raw.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
raw.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![12]));
raw.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![1]));
raw.set(t::PHOTOMETRIC, Value::Short(vec![photometric::CFA]));
raw.set(t::COMPRESSION, Value::Short(vec![32767]));
raw.set(t::CFA_REPEAT_PATTERN_DIM, Value::Short(vec![2, 2]));
raw.set(t::CFA_PATTERN_EP, Value::Byte(vec![0, 1, 1, 2]));
raw.set(TONE_CURVE, Value::Short(vec![8000, 10400, 12900, 14100]));
raw.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![strip] });
TiffWriter::default().write(&[raw]).unwrap()
}
/// Pack 12-bit samples as the file does: `b0 = a & 0xff`, `b1 = a >> 8 | b << 4 & 0xf0`, `b2 = b >> 4`.
fn pack12(samples: &[u16]) -> Vec<u8> {
samples.chunks(2).flat_map(|p| [p[0] as u8, (p[0] >> 8) as u8 | ((p[1] & 0xf) << 4) as u8, (p[1] >> 4) as u8]).collect()
}
#[test]
fn packed_12_bit_strip_of_one_and_a_half_bytes_per_pixel_decodes() {
let (w, h) = (8usize, 4usize);
let samples: Vec<u16> = (0..w * h).map(|i| (i as u16 * 129 + 130) & 0xfff).collect();
let file = packed_arw(w as u32, h as u32, pack12(&samples));
let full = decode(&file, Mode::Full).unwrap();
let RawData::U16(ref got) = full.data else { panic!("integer ARW") };
assert_eq!(got, &samples);
assert_eq!((full.width, full.height, full.bits), (w, h, 12));
assert_eq!(full.cfa.as_ref().map(|c| c.pattern.clone()), Some(vec![0, 1, 1, 2]));
assert_eq!(full.black.values.first().copied().unwrap_or(0.0), 128.0);
let header = decode(&file, Mode::Header).unwrap();
assert_eq!(header.info(), full.info());
// the first three bytes 0x12 0xA3 0x45 are the samples 0x312 and 0x45A (least-significant-bit first)
let mut row = vec![0u16; 2];
unpack_row12(&[0x12, 0xa3, 0x45], &mut row);
assert_eq!(row, [0x312, 0x45a]);
}
#[test]
fn packed_12_bit_needs_the_exact_strip_size() {
let (w, h) = (8usize, 4usize);
let exact = w * h * 3 / 2;
for strip in [exact - 1, exact + 1, exact + 3 * h] {
for mode in [Mode::Header, Mode::Full] {
let err = decode(&packed_arw(w as u32, h as u32, vec![0u8; strip]), mode).unwrap_err();
assert!(matches!(err, RawError::Unsupported(ref m) if m.contains("packed compressed variant")), "{strip} {mode:?}: {err:?}");
}
}
// an odd width cannot be packed in pairs
let err = decode(&packed_arw(7, 4, vec![0u8; 7 * 4 * 3 / 2]), Mode::Full).unwrap_err();
assert!(matches!(err, RawError::Unsupported(_)), "{err:?}");
}
/// Encode one 16-value set as an ARW2 block (the paper's scheme, used here to test the decoder).
pub(crate) fn encode_block(v: &[u16; 16]) -> [u8; 16] {
let (imax, &max) = v.iter().enumerate().max_by_key(|(i, x)| (**x, usize::MAX - *i)).unwrap();
crates/raw/src/vendor/arw_lens.rs
Vendored
+230
View File
@@ -0,0 +1,230 @@
//! Sony's plain raw-IFD distortion table, expressed as a DNG warp for the shared optics pipeline.
//!
//! Tag identity/layout: https://exiftool.org/TagNames/EXIF.html (`0x7037`, signed shorts).
//! Geometry inferred independently from files and Sony Imaging Edge exports; see docs/sony-lens-corrections.md.
//! No lens database or external decoder code. The unrelated MakerNote table has a different scale.
use crate::{Opcode, Rect};
use lightcraft_tiff::{Ifd, Value};
const DISTORTION: u16 = 0x7037;
const SAMPLES: usize = 1024;
const MAX_ERROR: f64 = 0.0005;
/// Only independently validated camera models and the 16-entry layout. Unknown models/layouts
/// and implausible/non-smooth maps are left uncorrected. See docs/sony-lens-corrections.md.
/// A camera setting of Off does not remove the lens data: the user can enable it in the developer.
pub(super) fn distortion(model: &str, ifd: &Ifd, active: Rect, crop: Rect) -> Option<Opcode> {
// A matching table shape does not establish its scale or radial normalization on another body.
// Keep this exact: ILCE-7RM4 and ILCE-7RM4A are distinct models, not interchangeable aliases.
if model != "ILCE-7RM4A" {
return None;
}
let Value::SShort(values) = &ifd.get(DISTORTION)?.value else { return None };
let [16, table @ ..] = values.as_slice() else { return None };
let table: &[i16; 16] = table.try_into().ok()?;
if table.iter().all(|v| *v == 0) || table.iter().any(|v| !(-4096..=4096).contains(v)) {
return None;
}
let crop = crop.clipped(active.width, active.height);
if active.width < 2 || active.height < 2 || crop.width < 2 || crop.height < 2 {
return None;
}
// Only native 3:2 framing has been validated. An in-camera aspect crop may retain a table
// normalized to the full sensor frame; do not silently reinterpret it as the smaller crop.
if (crop.width as f64 / crop.height as f64 - 1.5).abs() > 0.005 {
return None;
}
// Interpolate fractional radial scale at evenly spaced radii from centre to cropped corner.
let scale = |r: f64| {
let at = r.clamp(0.0, 1.0) * 15.0;
let i = (at.floor() as usize).min(14);
let a = f64::from(table[i]);
let b = f64::from(table[i + 1]);
1.0 + (a + (b - a) * (at - i as f64)) / 16384.0
};
let (cw, ch) = (crop.width as f64 - 1.0, crop.height as f64 - 1.0);
let diagonal = cw.hypot(ch);
// Zoom until the entire rectangular output boundary is inside the source. For a piecewise-linear
// scale its maximum on that boundary occurs at its nearest radius or at one of the remaining knots.
let nearest = cw.min(ch) / diagonal;
let max_scale = (0..16).map(|i| i as f64 / 15.0).filter(|&r| r >= nearest).map(scale).fold(scale(nearest), f64::max);
let zoom = 1.0 / max_scale;
let basis = |r: f64| {
let r2 = r * r;
[r, r * r2, r * r2 * r2, r * r2 * r2 * r2]
};
let mut matrix = [[0.0; 4]; 4];
let mut rhs = [0.0; 4];
let mut previous = -1.0;
for j in 0..=SAMPLES {
let r = j as f64 / SAMPLES as f64;
let source = r * scale(r) * zoom;
if source <= previous {
return None;
}
previous = source;
let terms = basis(r);
for (i, &a) in terms.iter().enumerate() {
rhs[i] += a * source;
for (k, &b) in terms.iter().enumerate() {
matrix[i][k] += a * b;
}
}
}
let mut k = solve4(matrix, rhs)?;
// Keep the polynomial within a fraction of a preview pixel of the file's table, and forbid folds.
for j in 0..=SAMPLES {
let r = j as f64 / SAMPLES as f64;
let fit = basis(r).iter().zip(k).map(|(a, b)| a * b).sum::<f64>();
let r2 = r * r;
let derivative = k[0] + r2 * (3.0 * k[1] + r2 * (5.0 * k[2] + r2 * 7.0 * k[3]));
if (fit - r * scale(r) * zoom).abs() > MAX_ERROR || derivative <= 0.0 {
return None;
}
}
// DNG uses the farthest *active-area* corner, not the cropped diagonal. Preserve the same mapping
// after the engine translates the opcode into default-cropped and EXIF-oriented coordinates.
let (ax, ay) = (active.width as f64 - 1.0, active.height as f64 - 1.0);
let (cx, cy) = (crop.x as f64 + cw / 2.0, crop.y as f64 + ch / 2.0);
let farthest = cx.max(ax - cx).hypot(cy.max(ay - cy));
let ratio2 = (2.0 * farthest / diagonal).powi(2);
for (i, coefficient) in k.iter_mut().enumerate() {
*coefficient *= ratio2.powi(i as i32);
}
Some(Opcode::WarpRectilinear { planes: vec![[k[0], k[1], k[2], k[3], 0.0, 0.0]], center: [cx / ax, cy / ay] })
}
/// Small fixed-size least-squares system; partial pivoting, no input-sized allocations.
fn solve4(mut a: [[f64; 4]; 4], mut b: [f64; 4]) -> Option<[f64; 4]> {
for col in 0..4 {
let pivot = (col..4).max_by(|&i, &j| a[i][col].abs().total_cmp(&a[j][col].abs()))?;
if !a[pivot][col].is_finite() || a[pivot][col].abs() < 1e-12 {
return None;
}
a.swap(col, pivot);
b.swap(col, pivot);
for row in col + 1..4 {
let factor = a[row][col] / a[col][col];
for j in col..4 {
a[row][j] -= factor * a[col][j];
}
b[row] -= factor * b[col];
}
}
let mut x = [0.0; 4];
for i in (0..4).rev() {
x[i] = (b[i] - (i + 1..4).map(|j| a[i][j] * x[j]).sum::<f64>()) / a[i][i];
}
x.iter().all(|v| v.is_finite()).then_some(x)
}
#[cfg(test)]
mod tests {
use super::*;
use lightcraft_tiff::Entry;
fn ifd(value: Value) -> Ifd {
Ifd { entries: vec![Entry { tag: DISTORTION, value, offset: 0 }], ..Default::default() }
}
fn table(values: &[i16]) -> Ifd {
ifd(Value::SShort(std::iter::once(16).chain(values.iter().copied()).collect()))
}
const TELE: [i16; 16] = [-2, 1, 7, 17, 31, 48, 68, 93, 120, 152, 187, 225, 265, 309, 356, 403];
const WIDE: [i16; 16] = [14, 0, -27, -59, -106, -158, -223, -291, -367, -440, -519, -593, -667, -731, -792, -839];
const AREA: Rect = Rect { x: 0, y: 0, width: 9504, height: 6336 };
#[test]
fn matches_independent_sony_reference_geometry() {
// On/off feature correspondences from Imaging Edge 4.1 at 600 mm, independently fitted.
let Opcode::WarpRectilinear { planes, center } = distortion("ILCE-7RM4A", &table(&TELE), AREA, AREA).unwrap() else { panic!("warp") };
assert_eq!(center, [0.5, 0.5]);
let reference = [0.9760159044, 0.0254891644, -0.0005984764, -0.0010120638];
for i in 0..=100 {
let r = i as f64 / 100.0;
let evaluate = |k: &[f64]| r * (k[0] + r * r * (k[1] + r * r * (k[2] + r * r * k[3])));
assert!((evaluate(&planes[0]) - evaluate(&reference)).abs() < 0.00015);
}
}
#[test]
fn barrel_framing_and_offset_crop_keep_the_same_geometry() {
let crop = Rect::new(32, 20, 9504, 6336);
let active = Rect::new(0, 0, 9600, 6376);
let Opcode::WarpRectilinear { planes: a, .. } = distortion("ILCE-7RM4A", &table(&WIDE), AREA, AREA).unwrap() else { panic!("warp") };
let Opcode::WarpRectilinear { planes: b, center } = distortion("ILCE-7RM4A", &table(&WIDE), active, crop).unwrap() else { panic!("warp") };
assert!(a[0][0] > 1.02 && a[0][0] < 1.03, "barrel correction expands the middle");
assert!((center[0] * 9599.0 - 4783.5).abs() < 1e-10);
assert!((center[1] * 6375.0 - 3187.5).abs() < 1e-10);
let ratio = (4815.5f64.hypot(3187.5)) / (4751.5f64.hypot(3167.5));
for i in 0..4 {
assert!((a[0][i] - b[0][i] / ratio.powi(2 * i as i32)).abs() < 1e-10);
}
}
#[test]
fn refuses_missing_unknown_malformed_or_unusable_tables() {
for bad in [
Ifd::default(),
table(&[]),
table(&TELE[..15]),
table(&[0; 16]),
table(&[i16::MIN; 16]),
table(&[0, 4096, -4096, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]),
ifd(Value::SShort(vec![15; 17])),
ifd(Value::Short(vec![16; 17])),
ifd(Value::Double(vec![f64::NAN; 17])),
] {
assert!(distortion("ILCE-7RM4A", &bad, AREA, AREA).is_none());
}
assert!(distortion("ILCE-7RM4A", &table(&TELE), Rect::new(0, 0, 1, 1), AREA).is_none());
assert!(distortion("ILCE-7RM4A", &table(&TELE), AREA, Rect::new(0, 0, 9504, 5346)).is_none());
assert!(distortion("ILCE-7RM4A", &table(&TELE), AREA, Rect::new(99999, 0, 10, 10)).is_none());
}
#[test]
fn refuses_unvalidated_models_even_with_a_valid_table() {
let valid = table(&TELE);
assert!(distortion("ILCE-7RM4A", &valid, AREA, AREA).is_some());
for model in ["", "ILCE-7M3", "ILCE-7M4", "ILCE-7RM4", "ILCE-7RM5", "ILCE-9M2", "DSC-RX100M3", "ILCE-7RM4A unknown"] {
assert!(distortion(model, &valid, AREA, AREA).is_none(), "{model}");
}
}
#[test]
fn header_and_full_decode_preserve_correction_through_dng() {
use lightcraft_tiff::{ByteOrder, IfdBuilder, ImageData, TiffWriter, tags as t};
for (model, expected) in [("ILCE-7RM4A", 1), ("ILCE-7RM4", 0), ("ILCE-7M3", 0), ("", 0)] {
for order in [ByteOrder::Little, ByteOrder::Big] {
let mut raw = IfdBuilder::new();
raw.set(t::MAKE, Value::Ascii("SONY".into()));
raw.set(t::MODEL, Value::Ascii(model.into()));
raw.set(t::IMAGE_WIDTH, Value::Long(vec![32]));
raw.set(t::IMAGE_LENGTH, Value::Long(vec![24]));
raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![16]));
raw.set(t::SAMPLES_PER_PIXEL, Value::Short(vec![1]));
raw.set(t::PHOTOMETRIC, Value::Short(vec![t::photometric::CFA]));
raw.set(t::COMPRESSION, Value::Short(vec![1]));
raw.set(t::WHITE_LEVEL, Value::Long(vec![16383]));
raw.set(0x74c7, Value::Long(vec![0, 2]));
raw.set(t::DEFAULT_CROP_ORIGIN, Value::Long(vec![2, 2]));
raw.set(t::DEFAULT_CROP_SIZE, Value::Long(vec![30, 20]));
raw.set(0x74c8, Value::Long(vec![30, 20]));
raw.set(DISTORTION, Value::SShort(std::iter::once(16).chain(TELE).collect()));
// Zero pixels have the same byte representation in both orders.
raw.set_image(ImageData::Strips { rows_per_strip: 24, strips: vec![vec![0; 32 * 24 * 2]] });
let file = TiffWriter { order, ..Default::default() }.write(&[raw]).unwrap();
let header = super::super::arw::decode(&file, crate::Mode::Header).unwrap();
let full = super::super::arw::decode(&file, crate::Mode::Full).unwrap();
assert_eq!(header.info(), full.info());
assert_eq!(full.opcodes.list3.len(), expected, "{model} {order:?}");
// the image keeps Sony's crop tags; the warp is centred on the DNG default crop (x 2..32 of 32)
assert_eq!(full.crop, Rect::new(0, 2, 30, 20));
if let Some(Opcode::WarpRectilinear { center, .. }) = full.opcodes.list3.first() {
assert!((center[0] - 16.5 / 31.0).abs() < 1e-12 && (center[1] - 11.5 / 23.0).abs() < 1e-12, "{center:?}");
}
let dng = crate::write_dng(&full, &Default::default()).unwrap();
assert_eq!(crate::decode(&dng).unwrap().opcodes.list3, full.opcodes.list3);
}
}
}
}
crates/raw/src/vendor/mod.rs
Vendored
+1
View File
@@ -1,6 +1,7 @@
//! Vendor raw formats (TIFF-based) and helpers shared by them.
pub mod arw;
mod arw_lens;
pub mod cr2;
pub mod cr3;
pub mod crx;
crates/raw/tests/corpus.rs
+57 -5
View File
@@ -516,11 +516,12 @@ fn corpus_sony_sr2_white_balance() {
let dir = corpus_root().join("raw");
// (file, R and B levels)
let cases = [
("arw-sony-a500.arw", [2212, 1416]), // 27152-byte SR2SubIFD layout (2.04 / 1.35)
("arw-sony-a33.arw", [2344, 1508]), // 29000-byte layout (2.29 / 1.48)
("arw-sony-a700.arw", [2128, 1564]), // 62112-byte layout (too few neutral pixels)
("arw-sony-a3500-5600k.arw", [2932, 1576]), // 5600 K (2.76 / 1.53; Tag2010 1.89 / 3.08)
("arw-sony-a7s-shade.arw", [2932, 1372]), // Shade (2.90 / 1.32; Tag2010 2.63 / 1.52)
("arw-sony-a500.arw", [2212, 1416]), // 27152-byte SR2SubIFD layout (2.04 / 1.35)
("arw-sony-a33.arw", [2344, 1508]), // 29000-byte layout (2.29 / 1.48)
("arw-sony-a700.arw", [2128, 1564]), // 62112-byte layout (too few neutral pixels)
("arw-sony-a3500-5600k.arw", [2932, 1576]), // 5600 K (2.76 / 1.53; Tag2010 1.89 / 3.08)
("arw-sony-a7s-shade.arw", [2932, 1372]), // Shade (2.90 / 1.32; Tag2010 2.63 / 1.52)
("arw-sony-a900-packed12.arw", [2688, 1516]), // packed 12-bit (the maker note's plain 0x0020 block agrees)
];
let mut seen = 0;
for (name, [r, b]) in cases {
@@ -537,6 +538,22 @@ fn corpus_sony_sr2_white_balance() {
eprintln!("Sony SR2SubIFD white balance checked on {seen} files");
}
/// The DSLR-A900's packed 12-bit strip decodes (issue #535, item 6): 12 bits on a black of 128, and the camera
/// JPEG's `FullImageSize` window centred in the frame, which has no padding at either edge.
#[test]
fn corpus_sony_a900_packed12() {
let name = "arw-sony-a900-packed12.arw";
let Ok(bytes) = std::fs::read(corpus_root().join("raw").join(name)) else {
eprintln!("skip: {name} absent");
return;
};
let img = decode(&bytes).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!((img.width, img.height, img.bits), (6080, 4048, 12));
assert_eq!(img.black.values, vec![128.0]);
let c = img.crop;
assert_eq!((c.x, c.y, c.width, c.height), (16, 8, 6048, 4032));
}
/// Issue #535: pre-2017 Sony bodies without crop tags shot in the camera's 16:9 mode. The ILCE-7SM2 records the
/// 16:9 size in `FullImageSize`, the DSLR-A580 only in the Exif image size; both crops are centred vertically (each
/// camera JPEG registered on its raw) and drop the right-edge padding, so the image matches its JPEG's aspect ratio
@@ -879,3 +896,38 @@ fn corpus_raws_keep_their_container_and_are_not_thumbnail_shells() {
}
eprintln!("{checked} corpus raws keep their container");
}
/// Matching Sony table dimensions are not evidence of a shared correction model. Exercise the
/// actual corpus headers and decoded files as well as the synthetic model-boundary unit tests.
#[test]
fn sony_embedded_distortion_is_limited_to_validated_models() {
let root = corpus_root().join("raw");
let mut checked = 0;
for (name, model, expected) in [
("arw-sony-a7rm4a-compressed.arw", "ILCE-7RM4A", 1),
("arw-sony-a9m2-compressed.arw", "ILCE-9M2", 0),
("arw-sony-a7m3-compressed.arw", "ILCE-7M3", 0),
("arw-sony-a7m3-uncompressed.arw", "ILCE-7M3", 0),
("arw-sony-a7m4-14bit.arw", "ILCE-7M4", 0),
("arw-sony-a7m4-lossless-l.arw", "ILCE-7M4", 0),
("arw-sony-a7m4-lossless-m.arw", "ILCE-7M4", 0),
("arw-sony-a7m4-lossless-s.arw", "ILCE-7M4", 0),
("arw-sony-a7rm2-12bit-uncompressed.arw", "ILCE-7RM2", 0),
("arw-sony-rx100.arw", "DSC-RX100", 0),
("arw-sony-rx100m3.arw", "DSC-RX100M3", 0),
] {
let path = root.join(name);
if !path.exists() {
eprintln!("skip: {} absent", path.display());
continue;
}
let bytes = std::fs::read(&path).unwrap();
let full = decode(&bytes).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(probe_info(&bytes).unwrap(), full.info(), "{name}");
assert_eq!(full.metadata.model.as_deref(), Some(model), "{name}");
assert_eq!(full.opcodes.list3.len(), expected, "{name}: only the validated Sony model gets a distortion warp");
eprintln!("{name}: {expected} warp(s); header and full decode agree");
checked += 1;
}
eprintln!("Sony distortion model boundary: {checked} corpus files checked");
}
crates/ui-egui/Cargo.toml
+1
View File
@@ -28,6 +28,7 @@ log = { workspace = true }
lightcraft-denoise = { workspace = true, features = ["runtime"] }
lightcraft-codecs = { workspace = true }
lightcraft-faces = { workspace = true }
lightcraft-merge = { workspace = true }
[build-dependencies]
serde_json = { workspace = true }
crates/ui-egui/locales/de-formats.json
+8 -1
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": "; mit neuen Synonymen: {n}",
"Exported {n} keyword{}": "Stichwörter exportiert: {n}{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — Capture One importiert keine Liste mit ; , < oder > in einem Stichwort: {names}",
" — left out, as a keyword list can't hold them: {names}": " — ausgelassen, weil eine Stichwortliste sie nicht aufnehmen kann: {names}"
" — left out, as a keyword list can't hold them: {names}": " — ausgelassen, weil eine Stichwortliste sie nicht aufnehmen kann: {names}",
"{done} of {total}": "{done} von {total}",
"{done} of {total} MB": "{done} von {total} MB",
"{pct} %": "{pct} %",
"+{n} more": "+{n} weitere",
"1 task is still running: {names}. Quitting stops it.": "1 Aufgabe läuft noch: {names}. Beenden bricht sie ab.",
"{n} tasks are still running: {names}. Quitting stops them.": "{n} Aufgaben laufen noch: {names}. Beenden bricht sie ab.",
"{what} merge cancelled": "{what}-Zusammenfügung abgebrochen"
}
crates/ui-egui/locales/de.json
+16 -1
View File
@@ -2016,5 +2016,20 @@
"Stopped by an error (see the log)": "Durch einen Fehler angehalten (siehe Protokoll)",
"Not saved": "Nicht gespeichert",
"A folder is being synchronized": "Ein Ordner wird gerade synchronisiert",
"Wait for the import to finish": "Warten Sie, bis der Import abgeschlossen ist"
"Wait for the import to finish": "Warten Sie, bis der Import abgeschlossen ist",
"Show less": "Weniger anzeigen",
"Quit while tasks are running?": "Beenden, während Aufgaben laufen?",
"Exporting contact sheet": "Kontaktabzug wird exportiert",
"Reading folder": "Ordner wird gelesen",
"Scanning folder": "Ordner wird durchsucht",
"Building previews": "Vorschauen werden erstellt",
"Building smart previews": "Smart-Vorschauen werden erstellt",
"Discarding smart previews": "Smart-Vorschauen werden verworfen",
"Importing Lightroom catalog": "Lightroom-Katalog wird importiert",
"Reading Lightroom catalog": "Lightroom-Katalog wird gelesen",
"Merging": "Zusammenfügen",
"Downloading model": "Modell wird heruntergeladen",
"Finding faces": "Gesichter werden gesucht",
"Synchronizing folder": "Ordner wird synchronisiert",
"Looking for changes": "Änderungen werden gesucht"
}
crates/ui-egui/locales/es-formats.json
+8 -1
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": "; con sinónimos nuevos: {n}",
"Exported {n} keyword{}": "Palabras clave exportadas: {n}{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — Capture One no importa una lista con ; , < o > en una palabra clave: {names}",
" — left out, as a keyword list can't hold them: {names}": " — omitidas, porque una lista de palabras clave no puede contenerlas: {names}"
" — left out, as a keyword list can't hold them: {names}": " — omitidas, porque una lista de palabras clave no puede contenerlas: {names}",
"{done} of {total}": "{done} de {total}",
"{done} of {total} MB": "{done} de {total} MB",
"{pct} %": "{pct} %",
"+{n} more": "+{n} más",
"1 task is still running: {names}. Quitting stops it.": "1 tarea sigue en curso: {names}. Salir la detiene.",
"{n} tasks are still running: {names}. Quitting stops them.": "{n} tareas siguen en curso: {names}. Salir las detiene.",
"{what} merge cancelled": "Combinación {what} cancelada"
}
crates/ui-egui/locales/es.json
+16 -1
View File
@@ -1861,5 +1861,20 @@
"Stopped by an error (see the log)": "Detenido por un error (ver el registro)",
"Not saved": "Sin guardar",
"A folder is being synchronized": "Se está sincronizando una carpeta",
"Wait for the import to finish": "Espera a que termine la importación"
"Wait for the import to finish": "Espera a que termine la importación",
"Show less": "Mostrar menos",
"Quit while tasks are running?": "¿Salir con tareas en curso?",
"Exporting contact sheet": "Exportando hoja de contactos",
"Reading folder": "Leyendo carpeta",
"Scanning folder": "Explorando carpeta",
"Building previews": "Generando vistas previas",
"Building smart previews": "Generando vistas previas inteligentes",
"Discarding smart previews": "Descartando vistas previas inteligentes",
"Importing Lightroom catalog": "Importando catálogo de Lightroom",
"Reading Lightroom catalog": "Leyendo catálogo de Lightroom",
"Merging": "Combinando",
"Downloading model": "Descargando modelo",
"Finding faces": "Buscando caras",
"Synchronizing folder": "Sincronizando carpeta",
"Looking for changes": "Buscando cambios"
}
crates/ui-egui/locales/fr-formats.json
+9 -2
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": " ; avec de nouveaux synonymes : {n}",
"Exported {n} keyword{}": "Mots-clés exportés : {n}{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — Capture One n’importe pas une liste dont un mot-clé contient ; , < ou > : {names}",
" — left out, as a keyword list can't hold them: {names}": " — omis, car une liste de mots-clés ne peut pas les contenir : {names}"
}
" — left out, as a keyword list can't hold them: {names}": " — omis, car une liste de mots-clés ne peut pas les contenir : {names}",
"{done} of {total}": "{done} sur {total}",
"{done} of {total} MB": "{done} sur {total} Mo",
"{pct} %": "{pct} %",
"+{n} more": "+{n} autres",
"1 task is still running: {names}. Quitting stops it.": "1 tâche est encore en cours : {names}. Quitter l'arrête.",
"{n} tasks are still running: {names}. Quitting stops them.": "{n} tâches sont encore en cours : {names}. Quitter les arrête.",
"{what} merge cancelled": "Fusion {what} annulée"
}
crates/ui-egui/locales/fr.json
+16 -1
View File
@@ -1861,5 +1861,20 @@
"Stopped by an error (see the log)": "Arrêté par une erreur (voir le journal)",
"Not saved": "Non enregistré",
"A folder is being synchronized": "Un dossier est en cours de synchronisation",
"Wait for the import to finish": "Attendez la fin de l'importation"
"Wait for the import to finish": "Attendez la fin de l'importation",
"Show less": "Afficher moins",
"Quit while tasks are running?": "Quitter pendant que des tâches sont en cours ?",
"Exporting contact sheet": "Exportation de la planche-contact",
"Reading folder": "Lecture du dossier",
"Scanning folder": "Analyse du dossier",
"Building previews": "Génération des aperçus",
"Building smart previews": "Génération des aperçus dynamiques",
"Discarding smart previews": "Suppression des aperçus dynamiques",
"Importing Lightroom catalog": "Importation du catalogue Lightroom",
"Reading Lightroom catalog": "Lecture du catalogue Lightroom",
"Merging": "Fusion en cours",
"Downloading model": "Téléchargement du modèle",
"Finding faces": "Recherche des visages",
"Synchronizing folder": "Synchronisation du dossier",
"Looking for changes": "Recherche des modifications"
}
crates/ui-egui/locales/ja-formats.json
+8 -1
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": "。{n}件に同義語を追加",
"Exported {n} keyword{}": "{n}件のキーワードを書き出しました{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — Capture One は ; , < > を含むキーワードがあるリストを読み込みません: {names}",
" — left out, as a keyword list can't hold them: {names}": " — キーワードリストに書けないため省略: {names}"
" — left out, as a keyword list can't hold them: {names}": " — キーワードリストに書けないため省略: {names}",
"{done} of {total}": "{total}件中{done}件",
"{done} of {total} MB": "{done} / {total} MB",
"{pct} %": "{pct}%",
"+{n} more": "ほか{n}件",
"1 task is still running: {names}. Quitting stops it.": "1件のタスクが実行中です：{names}。終了すると停止します。",
"{n} tasks are still running: {names}. Quitting stops them.": "{n}件のタスクが実行中です：{names}。終了すると停止します。",
"{what} merge cancelled": "{what}合成をキャンセルしました"
}
crates/ui-egui/locales/ja.json
+16 -1
View File
@@ -1863,5 +1863,20 @@
"Stopped by an error (see the log)": "エラーで停止しました（ログを参照）",
"Not saved": "保存されていません",
"A folder is being synchronized": "フォルダを同期中です",
"Wait for the import to finish": "読み込みが終わるまでお待ちください"
"Wait for the import to finish": "読み込みが終わるまでお待ちください",
"Show less": "表示を減らす",
"Quit while tasks are running?": "タスクの実行中に終了しますか？",
"Exporting contact sheet": "コンタクトシートを書き出し中",
"Reading folder": "フォルダーを読み込み中",
"Scanning folder": "フォルダーをスキャン中",
"Building previews": "プレビューを作成中",
"Building smart previews": "スマートプレビューを作成中",
"Discarding smart previews": "スマートプレビューを破棄中",
"Importing Lightroom catalog": "Lightroom カタログを読み込み中",
"Reading Lightroom catalog": "Lightroom カタログを読み取り中",
"Merging": "合成中",
"Downloading model": "モデルをダウンロード中",
"Finding faces": "顔を検索中",
"Synchronizing folder": "フォルダを同期中",
"Looking for changes": "変更を確認中"
}
crates/ui-egui/locales/pt-br-formats.json
+8 -1
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": "; {n} receberam sinônimos",
"Exported {n} keyword{}": "Palavras-chave exportadas: {n}{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — O Capture One não importa uma lista com ; , < ou > em uma palavra-chave: {names}",
" — left out, as a keyword list can't hold them: {names}": " — omitidas, pois uma lista de palavras-chave não pode contê-las: {names}"
" — left out, as a keyword list can't hold them: {names}": " — omitidas, pois uma lista de palavras-chave não pode contê-las: {names}",
"{done} of {total}": "{done} de {total}",
"{done} of {total} MB": "{done} de {total} MB",
"{pct} %": "{pct}%",
"+{n} more": "+{n} mais",
"1 task is still running: {names}. Quitting stops it.": "1 tarefa ainda está em andamento: {names}. Sair a interrompe.",
"{n} tasks are still running: {names}. Quitting stops them.": "{n} tarefas ainda estão em andamento: {names}. Sair as interrompe.",
"{what} merge cancelled": "{what} mesclagem cancelada"
}
crates/ui-egui/locales/pt-br.json
+16 -1
View File
@@ -1861,5 +1861,20 @@
"Stopped by an error (see the log)": "Interrompido por um erro (veja o log)",
"Not saved": "Não salvo",
"A folder is being synchronized": "Uma pasta está sendo sincronizada",
"Wait for the import to finish": "Aguarde a importação terminar"
"Wait for the import to finish": "Aguarde a importação terminar",
"Show less": "Mostrar menos",
"Quit while tasks are running?": "Sair com tarefas em andamento?",
"Exporting contact sheet": "Exportando folha de contato",
"Reading folder": "Lendo pasta",
"Scanning folder": "Verificando pasta",
"Building previews": "Gerando prévias",
"Building smart previews": "Gerando prévias inteligentes",
"Discarding smart previews": "Descartando prévias inteligentes",
"Importing Lightroom catalog": "Importando catálogo do Lightroom",
"Reading Lightroom catalog": "Lendo catálogo do Lightroom",
"Merging": "Mesclando",
"Downloading model": "Baixando modelo",
"Finding faces": "Procurando rostos",
"Synchronizing folder": "Sincronizando pasta",
"Looking for changes": "Procurando alterações"
}
crates/ui-egui/locales/ru-formats.json
+8 -1
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": "; синонимы добавлены к {n}",
"Exported {n} keyword{}": "Экспортировано ключевых слов: {n}{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — Capture One не импортирует список, где в ключевом слове есть ; , < или >: {names}",
" — left out, as a keyword list can't hold them: {names}": " — пропущены: список ключевых слов не может их содержать: {names}"
" — left out, as a keyword list can't hold them: {names}": " — пропущены: список ключевых слов не может их содержать: {names}",
"{done} of {total}": "{done} из {total}",
"{done} of {total} MB": "{done} из {total} МБ",
"{pct} %": "{pct} %",
"+{n} more": "ещё {n}",
"1 task is still running: {names}. Quitting stops it.": "Ещё выполняется задача: {names}. Выход прервёт её.",
"{n} tasks are still running: {names}. Quitting stops them.": "Ещё выполняется задач: {n} — {names}. Выход прервёт их.",
"{what} merge cancelled": "{what}: слияние отменено"
}
crates/ui-egui/locales/ru.json
+17 -1
View File
@@ -1574,5 +1574,21 @@
"Stopped by an error (see the log)": "Остановлено из-за ошибки (см. журнал)",
"Not saved": "Не сохранено",
"A folder is being synchronized": "Идёт синхронизация папки",
"Wait for the import to finish": "Дождитесь окончания импорта"
"Wait for the import to finish": "Дождитесь окончания импорта",
"Show less": "Свернуть",
"Quit while tasks are running?": "Выйти, пока выполняются задачи?",
"Exporting contact sheet": "Экспорт контрольного листа",
"Reading folder": "Чтение папки",
"Scanning folder": "Сканирование папки",
"Building previews": "Создание просмотров",
"Building smart previews": "Создание смарт-просмотров",
"Discarding smart previews": "Удаление смарт-просмотров",
"Importing Lightroom catalog": "Импорт каталога Lightroom",
"Reading Lightroom catalog": "Чтение каталога Lightroom",
"Merging": "Слияние",
"Downloading model": "Загрузка модели",
"Finding faces": "Поиск лиц",
"Quit Anyway": "Всё равно выйти",
"Synchronizing folder": "Синхронизация папки",
"Looking for changes": "Поиск изменений"
}
crates/ui-egui/locales/uk-formats.json
+8 -1
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": "; отримали синоніми: {n}",
"Exported {n} keyword{}": "Експортовано ключових слів: {n}{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — Capture One не імпортує список, у якому ключове слово містить ; , < або >: {names}",
" — left out, as a keyword list can't hold them: {names}": " — пропущено, бо список ключових слів не може їх містити: {names}"
" — left out, as a keyword list can't hold them: {names}": " — пропущено, бо список ключових слів не може їх містити: {names}",
"{done} of {total}": "{done} з {total}",
"{done} of {total} MB": "{done} з {total} МБ",
"{pct} %": "{pct} %",
"+{n} more": "+{n} ще",
"1 task is still running: {names}. Quitting stops it.": "1 завдання ще виконується: {names}. Вихід зупинить його.",
"{n} tasks are still running: {names}. Quitting stops them.": "Ще виконуються завдання ({n}): {names}. Вихід зупинить їх.",
"{what} merge cancelled": "Об’єднання {what} скасовано"
}
crates/ui-egui/locales/uk.json
+16 -1
View File
@@ -2055,5 +2055,20 @@
"Stopped by an error (see the log)": "Зупинено через помилку (див. журнал)",
"Not saved": "Не збережено",
"A folder is being synchronized": "Тека синхронізується",
"Wait for the import to finish": "Дочекайтеся завершення імпорту"
"Wait for the import to finish": "Дочекайтеся завершення імпорту",
"Show less": "Показати менше",
"Quit while tasks are running?": "Вийти, поки виконуються завдання?",
"Exporting contact sheet": "Експорт аркуша мініатюр",
"Reading folder": "Читання папки",
"Scanning folder": "Сканування папки",
"Building previews": "Створення попередніх переглядів",
"Building smart previews": "Створення смарт-переглядів",
"Discarding smart previews": "Видалення смарт-переглядів",
"Importing Lightroom catalog": "Імпорт каталогу Lightroom",
"Reading Lightroom catalog": "Читання каталогу Lightroom",
"Merging": "Об’єднання",
"Downloading model": "Завантаження моделі",
"Finding faces": "Пошук облич",
"Synchronizing folder": "Синхронізація папки",
"Looking for changes": "Пошук змін"
}
crates/ui-egui/locales/zh-hans-formats.json
+8 -1
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": "；{n} 个获得了同义词",
"Exported {n} keyword{}": "已导出 {n} 个关键字{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — Capture One 不会导入关键字中含有 ; , < 或 > 的列表：{names}",
" — left out, as a keyword list can't hold them: {names}": " — 已省略，关键字列表无法容纳: {names}"
" — left out, as a keyword list can't hold them: {names}": " — 已省略，关键字列表无法容纳: {names}",
"{done} of {total}": "{done}／{total}",
"{done} of {total} MB": "{done}／{total} MB",
"{pct} %": "{pct}%",
"+{n} more": "还有 {n} 项",
"1 task is still running: {names}. Quitting stops it.": "仍有 1 个任务在运行：{names}。退出将停止它。",
"{n} tasks are still running: {names}. Quitting stops them.": "仍有 {n} 个任务在运行：{names}。退出将停止它们。",
"{what} merge cancelled": "已取消 {what} 合并"
}
crates/ui-egui/locales/zh-hans.json
+16 -1
View File
@@ -1861,5 +1861,20 @@
"Stopped by an error (see the log)": "因错误而停止（见日志）",
"Not saved": "未保存",
"A folder is being synchronized": "正在同步文件夹",
"Wait for the import to finish": "请等待导入完成"
"Wait for the import to finish": "请等待导入完成",
"Show less": "收起",
"Quit while tasks are running?": "要在任务运行时退出吗？",
"Exporting contact sheet": "正在导出联系表",
"Reading folder": "正在读取文件夹",
"Scanning folder": "正在扫描文件夹",
"Building previews": "正在构建预览",
"Building smart previews": "正在构建智能预览",
"Discarding smart previews": "正在丢弃智能预览",
"Importing Lightroom catalog": "正在导入 Lightroom 目录",
"Reading Lightroom catalog": "正在读取 Lightroom 目录",
"Merging": "正在合并",
"Downloading model": "正在下载模型",
"Finding faces": "正在查找人脸",
"Synchronizing folder": "正在同步文件夹",
"Looking for changes": "正在查找更改"
}
crates/ui-egui/locales/zh-hant-formats.json
+8 -1
View File
@@ -221,5 +221,12 @@
"; {n} gained synonyms": "；{n} 個獲得了同義詞",
"Exported {n} keyword{}": "已匯出 {n} 個關鍵字{:.0}",
" — Capture One won't import a list with ; , < or > in a keyword: {names}": " — Capture One 不會匯入關鍵字中含有 ; , < 或 > 的清單：{names}",
" — left out, as a keyword list can't hold them: {names}": " — 已省略，關鍵字清單無法容納: {names}"
" — left out, as a keyword list can't hold them: {names}": " — 已省略，關鍵字清單無法容納: {names}",
"{done} of {total}": "{done}／{total}",
"{done} of {total} MB": "{done}／{total} MB",
"{pct} %": "{pct}%",
"+{n} more": "還有 {n} 項",
"1 task is still running: {names}. Quitting stops it.": "仍有 1 個工作在執行：{names}。結束將停止它。",
"{n} tasks are still running: {names}. Quitting stops them.": "仍有 {n} 個工作在執行：{names}。結束將停止它們。",
"{what} merge cancelled": "已取消{what}合併"
}
crates/ui-egui/locales/zh-hant.json
+16 -1
View File
@@ -1861,5 +1861,20 @@
"Stopped by an error (see the log)": "因錯誤而停止（請見記錄）",
"Not saved": "未儲存",
"A folder is being synchronized": "正在同步資料夾",
"Wait for the import to finish": "請等待匯入完成"
"Wait for the import to finish": "請等待匯入完成",
"Show less": "收合",
"Quit while tasks are running?": "要在工作執行時結束嗎？",
"Exporting contact sheet": "正在匯出聯絡表",
"Reading folder": "正在讀取檔案夾",
"Scanning folder": "正在掃描檔案夾",
"Building previews": "正在建立預覽",
"Building smart previews": "正在建立智慧型預覽",
"Discarding smart previews": "正在捨棄智慧型預覽",
"Importing Lightroom catalog": "正在匯入 Lightroom 目錄",
"Reading Lightroom catalog": "正在讀取 Lightroom 目錄",
"Merging": "正在合併",
"Downloading model": "正在下載模型",
"Finding faces": "正在尋找臉孔",
"Synchronizing folder": "正在同步資料夾",
"Looking for changes": "正在尋找變更"
}
crates/ui-egui/src/control.rs
+3 -1
View File
@@ -105,7 +105,7 @@ pub fn inspect(app: &LightcraftApp, ctx: &egui::Context) -> Value {
"status": app.ui.status,
"copied": app.copied,
"notices": app.notices,
"quitPrompt": app.quit_prompt,
"quitPrompt": app.quit_prompt.as_ref().map(crate::QuitPrompt::text),
// how hard the background face scan may work right now, and why it is judged so (see `panels::faces::scan_pace`)
"faceScan": {"pace": app.caches.faces_pace, "focused": app.caches.faces_in_front, "pending": app.caches.faces_pending, "indexed": app.caches.faces_indexed},
"unsaved": app.session.unsaved().map(|(n, e)| json!({"ops": n, "error": e})),
@@ -114,6 +114,8 @@ pub fn inspect(app: &LightcraftApp, ctx: &egui::Context) -> Value {
"export": {"running": app.export.as_ref().map(crate::export_task::ExportTask::status), "last": app.last_export_result},
"import": app.import.as_ref().map(crate::import::ImportTask::status),
"tasks": app.tasks.labels(),
// every long-running task in flight, as the activity stack shows them (`activity.list`)
"activity": app.session.activity.list(),
// commands waiting on a native file dialog shown off the UI thread (`pick`)
"fileDialogs": app.pending_picks.iter().map(|p| p.command.clone()).collect::<Vec<_>>(),
"memory": memory(app),
crates/ui-egui/src/export_task.rs
+36 -33
View File
@@ -1,8 +1,8 @@
//! Background export: the Export dialog, File → Export with Preset and Export with Previous hand
//! their batch to a worker thread so the window stays responsive. Photos are prepared on the UI
//! thread ([`lightcraft_engine::export::prepare_export`]: cheap, needs the session) and rendered,
//! encoded and written on the worker (several side by side: [`run_batch`]); a progress panel shows
//! the count and a Cancel button.
//! encoded and written on the worker (several side by side: [`run_batch`]); its row in the
//! activity stack shows the count, the file in progress and ✕ (issue #345).
//!
//! Needs [`crate::Services::write_shared`] (a thread-safe writer); without it (web) the batch runs
//! synchronously as before.
@@ -11,12 +11,11 @@ use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{Receiver, channel};
use std::sync::{Arc, Mutex};
use egui::Align2;
use lightcraft_engine::activity::{Cancel, TaskGuard};
use lightcraft_engine::export::{Destination, ExportOptions, PreparedExport, run_batch};
use serde_json::{Value, json};
use crate::LightcraftApp;
use crate::theme::Tokens;
pub struct ExportTask {
pub total: usize,
@@ -24,6 +23,8 @@ pub struct ExportTask {
pub progress: Arc<Mutex<(usize, String)>>,
pub cancel: Arc<AtomicBool>,
rx: Receiver<Result<Vec<Value>, String>>,
/// The export's row in the activity stack (`activity.cancel` sets `cancel`).
guard: TaskGuard,
}
impl ExportTask {
@@ -59,11 +60,13 @@ pub fn start(app: &mut LightcraftApp, items: Vec<PreparedExport>, opts: ExportOp
std::thread::Builder::new().name("export".into()).spawn(work).map_err(|e| e.to_string())?;
#[cfg(target_arch = "wasm32")]
work();
app.export = Some(ExportTask { total, progress, cancel, rx });
let guard = app.session.activity.start("export", "Exporting", Cancel::Flag(cancel.clone()));
guard.progress(0, total as u64);
app.export = Some(ExportTask { total, progress, cancel, rx, guard });
Ok(json!({"background": true, "total": total}))
}
/// Export a single paginated PDF using the same progress and cancellation UI as image exports.
/// Export a single paginated PDF with the same activity row and cancellation as image exports.
pub fn start_contact_sheet(app: &mut LightcraftApp, params: &Value) -> Result<Value, String> {
if app.export.is_some() {
return Err("an export is already running".into());
@@ -103,11 +106,13 @@ pub fn start_contact_sheet(app: &mut LightcraftApp, params: &Value) -> Result<Va
std::thread::Builder::new().name("contact-sheet".into()).spawn(work).map_err(|e| e.to_string())?;
#[cfg(target_arch = "wasm32")]
work();
app.export = Some(ExportTask { total, progress, cancel, rx });
let row = app.session.activity.start("export", "Exporting contact sheet", Cancel::Flag(cancel.clone()));
row.progress(0, total as u64);
app.export = Some(ExportTask { total, progress, cancel, rx, guard: row });
Ok(json!({"background": true, "total": total}))
}
/// Per frame: draw the progress panel; when the batch finishes, report it.
/// Per frame: keep the activity row up to date; when the batch finishes, report it.
pub fn poll(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(task) = &app.export else { return };
match task.rx.try_recv() {
@@ -144,30 +149,8 @@ pub fn poll(app: &mut LightcraftApp, ctx: &egui::Context) {
}
Err(std::sync::mpsc::TryRecvError::Empty) => {
let (done, current) = task.progress.lock().map(|g| g.clone()).unwrap_or_default();
let (total, cancel) = (task.total, task.cancel.clone());
let t = Tokens::get(ctx);
egui::Window::new(crate::i18n::tr("Exporting"))
.title_bar(false)
.resizable(false)
.anchor(Align2::LEFT_BOTTOM, [16.0, -56.0])
.fixed_size([300.0, 64.0])
.show(ctx, |ui| {
ui.horizontal(|ui| {
ui.label(
egui::RichText::new(crate::i18n::tr_format!("Exporting {} of {total}", (done + 1).min(total), total = total))
.color(t.text),
);
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
let stopping = cancel.load(Ordering::Relaxed);
if crate::widgets::text_button(ui, "exportCancel", if stopping { "Stopping…" } else { "Cancel" }, false).clicked() {
cancel.store(true, Ordering::Relaxed);
}
});
});
ui.add(egui::ProgressBar::new(done as f32 / total.max(1) as f32).desired_width(280.0));
ui.label(egui::RichText::new(current).size(11.0).color(t.text_dim));
});
ctx.request_repaint_after(std::time::Duration::from_millis(100));
task.guard.progress(done as u64, task.total as u64);
task.guard.detail(&current);
}
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
app.export = None;
@@ -214,7 +197,7 @@ mod contact_sheet_tests {
assert_eq!(h.app.pending_picks.len(), 1);
let deadline = Instant::now() + Duration::from_secs(20);
loop {
// The progress window lays out text, which egui only allows inside a frame.
// The activity stack lays out text, which egui only allows inside a frame.
h.step();
if h.app.last_export_result.is_some() {
break;
@@ -232,6 +215,26 @@ mod contact_sheet_tests {
assert_eq!(h.app.last_export_result.as_ref().unwrap()["files"][0]["photos"], 1);
}
#[test]
fn contact_sheet_export_shows_a_row_and_its_cross_stops_it() {
let mut app = LightcraftApp::new(
lightcraft_engine::Session::with_demo(),
crate::Services { write_shared: Some(Arc::new(|_, _| Ok(()))), ..Default::default() },
);
let ids: Vec<_> = app.session.catalog.photos().take(24).map(|p| p.id.0).collect();
start_contact_sheet(&mut app, &json!({"path": "Sheet.pdf", "ids": ids})).unwrap();
let rows = app.session.activity.list();
assert_eq!(rows.len(), 1, "{rows:?}");
assert_eq!((rows[0].kind, rows[0].label.as_str(), rows[0].total), ("export", "Exporting contact sheet", 24));
assert!(rows[0].cancellable);
app.session.activity.cancel(rows[0].id).unwrap();
assert!(app.export.as_ref().unwrap().cancel.load(Ordering::Relaxed), "the stack's ✕ stops the sheet");
let task = app.export.take().unwrap();
assert!(task.rx.recv_timeout(Duration::from_secs(20)).unwrap().is_err(), "a stopped sheet is not written");
drop(task);
assert!(app.session.activity.list().is_empty(), "the row goes with the export");
}
#[test]
fn cancelled_contact_sheet_never_calls_writer() {
let mut app = LightcraftApp::new(
crates/ui-egui/src/headless.rs
+12 -3
View File
@@ -1800,6 +1800,8 @@ mod tests {
assert!(h.app.export.is_some(), "running in the background");
let running = h.request("ui.inspect", json!({}), t);
assert_eq!(running["result"]["export"]["running"]["total"], 3, "{}", running["result"]["export"]);
// …and as a row in the activity stack (issue #345)
assert_eq!(running["result"]["activity"][0]["kind"], "export", "{}", running["result"]["activity"]);
let t0 = Instant::now();
while h.app.export.is_some() && t0.elapsed() < Duration::from_secs(60) {
h.step();
@@ -1808,7 +1810,9 @@ mod tests {
let w = written.lock().unwrap().clone();
assert_eq!(w.len(), 3, "{w:?}");
assert!(w.iter().all(|p| p.starts_with("/lc-test-out/")));
let last = h.request("ui.inspect", json!({}), t)["result"]["export"]["last"].clone();
let after = h.request("ui.inspect", json!({}), t)["result"].clone();
assert_eq!(after["activity"], json!([]), "the row is gone");
let last = after["export"]["last"].clone();
assert_eq!(last["files"].as_array().map(Vec::len), Some(3), "{last}");
assert!(last["files"][0]["width"].as_u64().is_some_and(|w| w <= 64));
}
@@ -2447,9 +2451,14 @@ mod tests {
assert!(!h.app.scan.as_ref().unwrap().copy, "the running scan keeps its options");
let r = h.request("ui.inspect", json!({}), t);
assert!(r["result"]["scan"].is_object(), "{r}");
let r = h.request("ui.clickWidget", json!({"id": "button:scanCancel"}), t);
// Cancel is the ✕ of the scan's row in the activity stack (shown once it is half a second old)
let id = h.app.session.activity.list().first().map(|t| t.id).expect("a scan row");
let cross = format!("activity:cancel:{id}");
assert!(h.step_until(t, |h| h.app.widgets.iter().any(|(w, _)| *w == cross)), "the row shows");
let r = h.request("ui.clickWidget", json!({"id": cross}), t);
assert_eq!(r["ok"], true, "{r}");
assert!(h.app.scan.is_none(), "Cancel closes the scan at once");
h.step();
assert!(h.app.scan.is_none(), "✕ closes the scan at once");
gate.store(true, std::sync::atomic::Ordering::Relaxed);
h.settle(SETTLE);
assert!(h.app.ui.dialog.is_none(), "a cancelled scan opens no review");
crates/ui-egui/src/import.rs
+40 -70
View File
@@ -5,7 +5,8 @@
//! chosen folder, filed by day, by month, into one folder or by a custom folder template,
//! optionally renamed), an album (existing or new), a preset and keywords to apply. Importing
//! runs on a worker thread (files are probed, copied or moved there) and its batches join the
//! catalog between frames, with a progress window and Cancel; the whole import is one undo step.
//! catalog between frames, with a row and ✕ in the activity stack (issue #345); the whole import is
//! one undo step.
use egui::{Align2, Color32, Rect, Sense, Stroke, StrokeKind, pos2, vec2};
use lightcraft_engine::import::{ImportCandidate, ScanInput, ScanOutput, ScanProgress, scan_with};
@@ -235,6 +236,8 @@ pub struct ImportTask {
/// Auto Import: the selection to keep.
keep_selection: Option<lightcraft_engine::Selection>,
run: Option<ImportRun>,
/// The import's row in the activity stack (its ✕ and `activity.cancel` set the run's cancel flag).
guard: Option<lightcraft_engine::activity::TaskGuard>,
}
impl ImportTask {
@@ -391,6 +394,14 @@ pub struct ScanTask {
browse: bool,
/// What is being scanned (the review's source).
sources: Vec<String>,
/// The scan's row in the activity stack (its ✕ and `activity.cancel` set `progress.cancel`).
guard: lightcraft_engine::activity::TaskGuard,
}
/// The activity row of a scan: the review's ("Scanning folder") or the Local view's ("Reading folder").
fn scan_guard(app: &LightcraftApp, progress: &ScanProgress, browse: bool) -> lightcraft_engine::activity::TaskGuard {
let label = if browse { "Reading folder" } else { "Scanning folder" };
app.session.activity.start("scan", label, lightcraft_engine::activity::Cancel::Flag(progress.cancel.clone()))
}
/// An import and a Synchronize Folder never run at once: each readies its files against the
@@ -416,7 +427,8 @@ pub fn open(app: &mut LightcraftApp, paths: Vec<String>) -> Result<Value, String
std::thread::spawn(job);
#[cfg(target_arch = "wasm32")]
job();
app.scan = Some(ScanTask { progress, rx, copy: false, browse: false, sources });
let guard = scan_guard(app, &progress, false);
app.scan = Some(ScanTask { progress, rx, copy: false, browse: false, sources, guard });
Ok(json!({"scanning": true}))
}
@@ -477,7 +489,8 @@ pub fn browse(app: &mut LightcraftApp, path: &str, subfolders: Option<bool>) ->
std::thread::spawn(job);
#[cfg(target_arch = "wasm32")]
job();
app.scan = Some(ScanTask { progress, rx, copy: false, browse: true, sources: Vec::new() });
let guard = scan_guard(app, &progress, true);
app.scan = Some(ScanTask { progress, rx, copy: false, browse: true, sources: Vec::new(), guard });
app.renderer.forget_imports();
Ok(json!({"path": dir_s, "subfolders": subfolders, "scanning": true}))
}
@@ -488,7 +501,18 @@ pub fn poll_scan(app: &mut LightcraftApp, ctx: &egui::Context) {
ctx.request_repaint_after(std::time::Duration::from_millis(100));
let out = match task.rx.try_recv() {
Ok(o) => o,
Err(std::sync::mpsc::TryRecvError::Empty) => return,
Err(std::sync::mpsc::TryRecvError::Empty) => {
if task.guard.is_cancelled() {
// ✕: the worker stops at its next file; don't wait for it (a NAS read can take a while)
app.scan = None;
return;
}
let (total, done) = (task.progress.total.load(Ordering::Relaxed), task.progress.done.load(Ordering::Relaxed));
task.guard.progress(done as u64, total as u64);
// while the folders are listed the total isn't known: say what is happening instead
task.guard.detail(if total == 0 { crate::i18n::tr("Looking for photos…") } else { "" });
return;
}
Err(_) => {
app.scan = None;
app.toast(ctx, crate::i18n::tr("Scan failed"));
@@ -531,37 +555,6 @@ pub fn poll_scan(app: &mut LightcraftApp, ctx: &egui::Context) {
app.ui.dialog = Some(crate::state::Dialog::Import { opts: Box::new(d) });
}
/// The progress window while a folder is being scanned.
pub fn scan_progress(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(task) = &app.scan else { return };
let t = Tokens::get(ctx);
let total = task.progress.total.load(Ordering::Relaxed);
let done = task.progress.done.load(Ordering::Relaxed);
let text = if total == 0 {
if task.browse { "Reading folder…" } else { "Looking for photos…" }.to_string()
} else {
crate::i18n::tr_format!("Reading photos… {done} of {total}", done = done, total = total)
};
let mut cancel = false;
egui::Window::new(crate::i18n::tr("Scanning"))
.title_bar(false)
.resizable(false)
.anchor(Align2::CENTER_BOTTOM, [0.0, -80.0])
.fixed_size([340.0, 80.0])
.show(ctx, |ui| {
ui.label(egui::RichText::new(text).color(t.text));
ui.add(egui::ProgressBar::new(done as f32 / total.max(1) as f32).desired_width(320.0));
let r = ui.button(crate::i18n::tr("Cancel"));
register(ui.ctx(), "button:scanCancel", r.rect);
cancel = r.clicked();
});
if cancel {
// the worker stops at its next file; don't wait for it (a NAS read can take a while)
task.progress.cancel.store(true, Ordering::Relaxed);
app.scan = None;
}
}
impl ScanTask {
/// `{done, total}` for `ui.inspect` (total is 0 while the folders are still being listed).
pub fn status(&self) -> Value {
@@ -648,7 +641,11 @@ pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
task.keep_selection = Some(app.session.selection.clone());
}
match started {
Ok(run) => task.run = Some(run),
Ok(run) => {
let label = if task.browse { "Reading folder" } else { "Importing" };
task.guard = Some(app.session.activity.start("import", label, lightcraft_engine::activity::Cancel::Flag(run.cancel.clone())));
task.run = Some(run);
}
Err(e) => {
log::warn!("import: {e}");
app.toast(ctx, crate::i18n::tr_format!("Import failed: {e}", e = e));
@@ -671,6 +668,13 @@ pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
task.total = run.total.load(Ordering::Relaxed).max(task.total);
task.done = run.done.load(Ordering::Relaxed);
}
if let Some(guard) = &task.guard {
guard.progress(task.done as u64, task.total as u64);
// ✕ in the stack (or `activity.cancel`): no further files are started
if guard.is_cancelled() {
task.cancelled = true;
}
}
if !finished {
app.import = Some(task);
return;
@@ -812,40 +816,6 @@ fn show_existing(app: &mut LightcraftApp, ctx: &egui::Context, existing: &[u64])
true
}
/// The progress window while an import runs, with Cancel (files already copied or added stay;
/// nothing new is started).
pub fn progress(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(task) = &app.import else { return };
let t = Tokens::get(ctx);
let frac = task.done as f32 / task.total.max(1) as f32;
let text = if task.cancelled {
crate::i18n::tr("Stopping…").to_string()
} else if task.browse {
crate::i18n::tr_format!("Reading photos… {} of {}", task.done, task.total)
} else {
crate::i18n::tr_format!("Adding photos… {} of {}", task.done, task.total)
};
let mut cancel = false;
egui::Window::new(crate::i18n::tr("Importing"))
.title_bar(false)
.resizable(false)
.anchor(Align2::CENTER_BOTTOM, [0.0, -80.0])
.fixed_size([340.0, 80.0])
.show(ctx, |ui| {
ui.label(egui::RichText::new(text).color(t.text));
ui.add(egui::ProgressBar::new(frac).desired_width(320.0));
let r = ui.add_enabled(!task.cancelled, egui::Button::new(crate::i18n::tr("Cancel")));
register(ui.ctx(), "button:importCancel", r.rect);
cancel = r.clicked();
});
if cancel && let Some(task) = app.import.as_mut() {
task.cancelled = true;
if let Some(run) = &task.run {
run.cancel.store(true, Ordering::Relaxed);
}
}
}
/// The dialog body: options, then the candidate grid.
pub fn body(app: &mut LightcraftApp, ui: &mut egui::Ui, d: &mut ImportDialog) {
let t = Tokens::get(ui.ctx());
crates/ui-egui/src/lib.rs
+30 -16
View File
@@ -36,6 +36,8 @@ pub mod theme;
pub mod titlebar;
pub mod widgets;
#[cfg(test)]
mod tests_activity;
#[cfg(test)]
mod tests_album_picker;
#[cfg(test)]
@@ -189,6 +191,24 @@ pub struct Perf {
pub fps: f64,
}
/// Why quitting stopped to ask first (`panels::notices`).
#[derive(Clone, Debug, PartialEq)]
pub enum QuitPrompt {
/// Changes couldn't be written to disk (issue #103).
Unsaved(String),
/// Tasks that quitting would cut short are running: an import, an export… (issue #345).
Tasks(String),
}
impl QuitPrompt {
/// The prompt's message.
pub fn text(&self) -> &str {
match self {
QuitPrompt::Unsaved(text) | QuitPrompt::Tasks(text) => text,
}
}
}
pub struct LightcraftApp {
pub(crate) model_setup: model_setup::Pending,
/// Per-catalog-revision caches of library-wide results the panels show every frame
@@ -213,8 +233,8 @@ pub struct LightcraftApp {
pub headless_host: bool,
/// Warnings to show one at a time (damaged settings files…, issue #103).
pub notices: Vec<String>,
/// Quitting was stopped because changes couldn't be saved: the prompt's text.
pub quit_prompt: Option<String>,
/// Quitting was stopped to ask first: changes couldn't be saved, or tasks are running.
pub quit_prompt: Option<QuitPrompt>,
/// Quit Anyway was chosen: the window may close with unsaved changes.
pub quit_confirmed: bool,
control_rx: Option<Receiver<ControlRequest>>,
@@ -302,6 +322,8 @@ pub struct LightcraftApp {
/// The library failed to open at launch: the blocking window, then the temporary-session
/// banner (issue #100). Cleared once a library opens.
pub library_problem: Option<panels::library_problem::LibraryProblem>,
/// The activity stack shows every task, not just the first few ("+N more" was clicked).
pub activity_expanded: bool,
}
impl LightcraftApp {
@@ -367,6 +389,7 @@ impl LightcraftApp {
display_applied: None,
display_error: None,
library_problem: None,
activity_expanded: false,
model_setup: Default::default(),
}
}
@@ -454,7 +477,7 @@ impl LightcraftApp {
ctx.request_repaint_after(std::time::Duration::from_secs_f64((due - now).clamp(0.05, interval)));
}
/// Announce the start and end of a Build Previews run.
/// Announce the end of a Build Previews run.
fn preview_build_status(&mut self, ctx: &egui::Context) {
use std::sync::atomic::Ordering;
let Some(b) = self.session.preview_build.clone() else { return };
@@ -477,14 +500,7 @@ impl LightcraftApp {
self.toast(ctx, msg);
}
} else {
if self.ui.preview_build_seen != Some((key, false)) {
self.ui.preview_build_seen = Some((key, false));
let msg = match b.what {
"" => crate::i18n::tr_format!("Building previews for {} photos…", b.total),
what => crate::i18n::tr_format!("Working on {what} for {} photos…", b.total, what = crate::i18n::tr(what)),
};
self.toast(ctx, msg);
}
// The activity stack shows the run while it works; poll for the end.
ctx.request_repaint_after(std::time::Duration::from_millis(250));
}
}
@@ -852,7 +868,8 @@ impl LightcraftApp {
let undo0 = app.session.undo.len();
app.import = Some(import::ImportTask::new(paths, params, undo0, false).auto());
};
if let Err(e) = tasks::spawn(self, LABEL, work, done) {
// listed every few seconds: quiet, no row in the activity stack
if let Err(e) = tasks::spawn(self, LABEL, None, work, done) {
log::warn!("{e}");
}
}
@@ -1049,11 +1066,8 @@ impl LightcraftApp {
panels::notices::show(self, &ctx);
panels::dialogs::show(self, &ctx);
panels::library_problem::show(self, &ctx);
import::progress(self, &ctx);
sync::progress_window(self, &ctx);
import::scan_progress(self, &ctx);
lightroom_import::progress(self, &ctx);
export_task::poll(self, &ctx);
panels::activity::show(self, &ctx);
pick::poll(self, &ctx);
panels::grid::drag_feedback(self, &ctx);
panels::left::album_drag_feedback(self, &ctx);
crates/ui-egui/src/lightroom_import.rs
+108 -48
View File
@@ -8,12 +8,10 @@ use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
use std::sync::{Arc, mpsc};
use egui::Align2;
use lightcraft_engine::activity::{Cancel, TaskGuard};
use serde_json::{Value, json};
use crate::LightcraftApp;
use crate::theme::Tokens;
use crate::widgets::register;
#[derive(Clone, Copy)]
enum Kind {
@@ -34,7 +32,7 @@ enum Phase {
Finalizing,
}
/// A Lightroom task in flight. The receiver is polled from the egui owner thread.
/// A Lightroom task in flight. The receiver is polled from the egui owner thread; the activity stack shows it.
pub struct LightroomTask {
kind: Kind,
path: PathBuf,
@@ -44,9 +42,53 @@ pub struct LightroomTask {
rx: mpsc::Receiver<Message>,
phase: Phase,
cancelled: bool,
guard: TaskGuard,
}
impl LightroomTask {
#[cfg(not(target_arch = "wasm32"))]
fn new(
s: &lightcraft_engine::Session,
kind: Kind,
path: PathBuf,
cancel: Arc<AtomicBool>,
total: Arc<AtomicUsize>,
done: Arc<AtomicUsize>,
rx: mpsc::Receiver<Message>,
) -> Self {
let label = match kind {
Kind::Inspect => "Reading Lightroom catalog",
Kind::Import => "Importing Lightroom catalog",
};
let guard = s.activity.start("lightroom", label, Cancel::Flag(cancel.clone()));
let task = Self { kind, path, cancel, total, done, rx, phase: Phase::Reading, cancelled: false, guard };
task.refresh();
task
}
/// Move to `phase`; only reading the catalog can stop, adding the photos runs to the end.
fn enter(&mut self, phase: Phase) {
self.phase = phase;
self.guard.set_cancellable(phase == Phase::Reading);
self.refresh();
}
/// Bring the activity row up to date: the count while reading, then what is being done.
fn refresh(&self) {
let detail = match (self.phase, self.kind) {
(Phase::Reading, Kind::Inspect) => "",
(Phase::Reading, Kind::Import) => crate::i18n::tr("Reading Lightroom catalog…"),
(Phase::Committing, _) => crate::i18n::tr("Adding Lightroom photos…"),
(Phase::Finalizing, _) => crate::i18n::tr("Saving Lightroom import index…"),
};
if self.phase == Phase::Reading {
self.guard.progress(self.done.load(Ordering::Relaxed) as u64, self.total.load(Ordering::Relaxed) as u64);
} else {
self.guard.progress(0, 0);
}
self.guard.detail(detail);
}
#[cfg(not(target_arch = "wasm32"))]
fn status(&self) -> Value {
let kind = match self.kind {
@@ -236,7 +278,7 @@ fn start_inspect(app: &mut LightcraftApp, path: PathBuf, ctx: &egui::Context) ->
let (tx, rx) = mpsc::channel();
spawn_inspect(path.clone(), cancel.clone(), total.clone(), done.clone(), tx, ctx.clone())?;
app.lightroom_last = None;
app.lightroom = Some(LightroomTask { kind: Kind::Inspect, path: path.clone(), cancel, total, done, rx, phase: Phase::Reading, cancelled: false });
app.lightroom = Some(LightroomTask::new(&app.session, Kind::Inspect, path.clone(), cancel, total, done, rx));
Ok(json!({"running": true, "kind": "inspect", "path": path}))
}
@@ -250,7 +292,7 @@ fn start_import(app: &mut LightcraftApp, path: PathBuf, update_existing: bool, c
let (tx, rx) = mpsc::channel();
spawn_prepare(job, cancel.clone(), tx, ctx.clone())?;
app.lightroom_last = None;
app.lightroom = Some(LightroomTask { kind: Kind::Import, path: path.clone(), cancel, total, done, rx, phase: Phase::Reading, cancelled: false });
app.lightroom = Some(LightroomTask::new(&app.session, Kind::Import, path.clone(), cancel, total, done, rx));
Ok(json!({"running": true, "kind": "import", "path": path}))
}
@@ -296,6 +338,7 @@ pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
let message = match task.rx.try_recv() {
Ok(m) => m,
Err(mpsc::TryRecvError::Empty) => {
task.refresh();
app.lightroom = Some(task);
return;
}
@@ -337,7 +380,7 @@ pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
);
return;
}
task.phase = Phase::Committing;
task.enter(Phase::Committing);
let mut finalization = None;
let mut report = None;
let committed = app.session.execute_fn("library.importLightroom", |s| {
@@ -347,7 +390,7 @@ pub fn tick(app: &mut LightcraftApp, ctx: &egui::Context) {
Ok(completion.report)
});
let report = report.unwrap_or(Value::Null);
task.phase = Phase::Finalizing;
task.enter(Phase::Finalizing);
let (tx, rx) = mpsc::channel();
let commit_error = committed.as_ref().err().map(ToString::to_string);
let Some(finalization) = finalization else {
@@ -409,50 +452,67 @@ pub fn wait_for(_app: &mut LightcraftApp, _ctx: &egui::Context, _timeout: std::t
unsupported_wasm()
}
/// The progress window for inspect/import, with cancellation.
pub fn progress(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(task) = app.lightroom.as_ref() else { return };
let total = task.total.load(Ordering::Relaxed);
let done = task.done.load(Ordering::Relaxed);
let title = match task.kind {
Kind::Inspect => crate::i18n::tr("Inspecting Lightroom Catalog"),
Kind::Import => crate::i18n::tr("Importing Lightroom Catalog"),
};
let text = match task.phase {
Phase::Reading if total == 0 => crate::i18n::tr("Reading Lightroom catalog…").into(),
Phase::Reading => crate::i18n::tr_format!("Reading Lightroom catalog… {done} of {total}", done = done, total = total),
Phase::Committing => crate::i18n::tr("Adding Lightroom photos…").into(),
Phase::Finalizing => crate::i18n::tr("Saving Lightroom import index…").into(),
};
let mut cancel = false;
let t = Tokens::get(ctx);
egui::Window::new(title).title_bar(false).resizable(false).anchor(Align2::CENTER_BOTTOM, [0.0, -80.0]).fixed_size([360.0, 92.0]).show(
ctx,
|ui| {
ui.label(egui::RichText::new(text).color(t.text));
ui.add(egui::ProgressBar::new(done as f32 / total.max(1) as f32).desired_width(340.0));
let r =
ui.add_enabled(task.phase == Phase::Reading && !task.cancel.load(Ordering::Relaxed), egui::Button::new(crate::i18n::tr("Cancel")));
register(ui.ctx(), "button:lightroomCancel", r.rect);
cancel = r.clicked();
},
);
if cancel {
if let Some(task) = app.lightroom.as_mut() {
task.cancel();
}
ctx.request_repaint();
}
if app.lightroom.is_some() {
ctx.request_repaint_after(std::time::Duration::from_millis(100));
}
}
#[cfg(test)]
mod tests {
use super::inspect_report;
use super::*;
use lightcraft_engine::lightroom_catalog::CatalogImport;
/// A task as `start_import` makes it, fed by a channel the test holds instead of a worker.
fn fake_import(h: &crate::headless::Headless) -> (LightroomTask, mpsc::Sender<Message>, Arc<AtomicBool>, Arc<AtomicUsize>, Arc<AtomicUsize>) {
let (tx, rx) = mpsc::channel();
let (cancel, total, done) = (Arc::new(AtomicBool::new(false)), Arc::new(AtomicUsize::new(0)), Arc::new(AtomicUsize::new(0)));
let task = LightroomTask::new(&h.app.session, Kind::Import, "fixture.lrcat".into(), cancel.clone(), total.clone(), done.clone(), rx);
(task, tx, cancel, total, done)
}
fn demo() -> crate::headless::Headless {
let app = crate::LightcraftApp::new(lightcraft_engine::Session::with_demo(), crate::Services { png: None, ..Default::default() });
let mut h = crate::headless::Headless::new(app, [1200.0, 800.0], 1.0);
// no render left running at exit (see tests_activity::demo)
h.settle(std::time::Duration::from_secs(20));
h
}
#[test]
fn lightroom_row_is_not_cancellable_while_committing() {
let mut h = demo();
let (task, tx, cancel, total, done) = fake_import(&h);
h.app.lightroom = Some(task);
total.store(10, Ordering::Relaxed);
done.store(3, Ordering::Relaxed);
h.step();
let rows = h.app.session.activity.list();
assert_eq!(rows.len(), 1);
assert_eq!((rows[0].kind, rows[0].label.as_str()), ("lightroom", "Importing Lightroom catalog"));
assert_eq!((rows[0].done, rows[0].total), (3, 10));
assert!(rows[0].cancellable, "reading the catalog can stop");
h.app.lightroom.as_mut().unwrap().enter(Phase::Committing);
let row = h.app.session.activity.list().remove(0);
assert!(!row.cancellable, "once photos are being added it runs to the end");
assert!(h.app.session.activity.cancel(row.id).is_err());
assert!(!cancel.load(Ordering::Relaxed));
tx.send(Message::Finalized(Ok(()))).unwrap();
h.step();
assert!(h.app.lightroom.is_none());
assert!(h.app.session.activity.list().is_empty(), "the row goes with the task");
}
#[test]
fn lightroom_cancel_from_the_stack_stops_the_read() {
let mut h = demo();
let (task, tx, cancel, _, _) = fake_import(&h);
h.app.lightroom = Some(task);
h.step();
let id = h.app.session.activity.list()[0].id;
h.app.session.activity.cancel(id).unwrap();
assert!(cancel.load(Ordering::Relaxed), "the worker sees the stack's ✕");
tx.send(Message::Prepared(Err("cancelled".into()))).unwrap();
h.step();
assert!(h.app.lightroom.is_none());
assert_eq!(h.app.lightroom_last, Some(serde_json::json!({"cancelled": true})));
assert!(h.app.session.activity.list().is_empty());
}
#[test]
fn inspect_report_keeps_empty_catalog_shape() {
let report = inspect_report(CatalogImport {
crates/ui-egui/src/menus.rs
+2 -2
View File
@@ -27,7 +27,7 @@ pub const LANGUAGE_COMMANDS: &[UiCommand] = &[
];
/// Every UI command: the languages, then everything else. `xtask parity` reads both tables from
/// this file, so an id listed in `docs/parity.md` is checked wherever it is declared.
/// this file, so an id listed in `docs/parity-checklist.md` is checked wherever it is declared.
pub fn ui_commands() -> impl Iterator<Item = &'static UiCommand> {
LANGUAGE_COMMANDS.iter().chain(UI_COMMANDS)
}
@@ -1212,7 +1212,7 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option<Re
}
};
app.ui.last_find_missing = None;
if let Err(e) = crate::tasks::spawn(app, LABEL, work, done) {
if let Err(e) = crate::tasks::spawn(app, LABEL, Some("findMissing"), work, done) {
return Some(Err(e));
}
if p.get("wait").and_then(Value::as_bool).unwrap_or(false) {
crates/ui-egui/src/merge.rs
+16 -5
View File
@@ -4,7 +4,7 @@
//! Previews and the final merge run [`lightcraft_engine::merge::MergeJob`]s on worker threads with
//! progress and cancellation; the window stays responsive. Changing an option cancels the running
//! preview and starts a new one. The final merge keeps running after the dialog closes (progress
//! in a toast); when it finishes, the result is written, imported and selected
//! in the activity stack); when it finishes, the result is written, imported and selected
//! ([`lightcraft_engine::Session::finish_merge`]).
use std::sync::atomic::{AtomicBool, Ordering};
@@ -12,6 +12,7 @@ use std::sync::mpsc::{Receiver, channel};
use std::sync::{Arc, Mutex};
use lightcraft_catalog::PhotoId;
use lightcraft_engine::activity::{Cancel, TaskGuard, Unit};
use lightcraft_engine::merge::{MergeJob, MergeOutput};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
@@ -88,6 +89,8 @@ pub struct MergeTask {
pub progress: Arc<Mutex<(f32, String)>>,
pub cancel: Arc<AtomicBool>,
rx: Receiver<Result<MergeOutput, String>>,
/// The final merge's row in the activity stack (previews have none).
guard: Option<TaskGuard>,
}
/// Merge state of the app.
@@ -140,7 +143,7 @@ fn spawn(app: &LightcraftApp, options: &MergeDialog, ids: &[PhotoId], preview: b
std::thread::Builder::new().name("photo-merge".into()).spawn(work).map_err(|e| e.to_string())?;
#[cfg(target_arch = "wasm32")]
work();
Ok(MergeTask { preview, job, options: options.clone(), progress, cancel, rx })
Ok(MergeTask { preview, job, options: options.clone(), progress, cancel, rx, guard: None })
}
/// Open the merge dialog for the selection.
@@ -169,7 +172,11 @@ pub fn start_final(app: &mut LightcraftApp, opts: &MergeDialog) -> Result<Value,
t.cancel.store(true, Ordering::Relaxed);
}
let ids = app.merge.ids.clone();
let task = spawn(app, opts, &ids, false)?;
let mut task = spawn(app, opts, &ids, false)?;
let guard = app.session.activity.start("merge", "Merging", Cancel::Flag(task.cancel.clone()));
guard.set_unit(Unit::Percent);
guard.progress(0, 1000);
task.guard = Some(guard);
app.merge.final_task = Some(task);
app.merge.last.insert(opts.command.clone(), opts.clone());
Ok(json!({"started": true, "photos": ids.len()}))
@@ -265,6 +272,7 @@ pub fn poll(app: &mut LightcraftApp, ctx: &egui::Context) {
app.merge.last_result = Some(v);
app.toast(ctx, crate::i18n::tr_format!("{what} merge added", what = what));
}
Err(_) if t.cancel.load(Ordering::Relaxed) => app.toast(ctx, crate::i18n::tr_format!("{what} merge cancelled", what = what)),
Err(e) => {
app.ui.status = e.to_string();
app.toast(ctx, crate::i18n::tr_format!("{what} merge failed: {e}", e = e, what = what));
@@ -273,8 +281,11 @@ pub fn poll(app: &mut LightcraftApp, ctx: &egui::Context) {
}
Err(_) => {
let (f, stage) = t.progress.lock().map(|g| g.clone()).unwrap_or_default();
let now = ctx.input(|i| i.time);
app.ui.toast = Some((crate::i18n::tr_format!("Merging… {stage} {:.0}%", f * 100.0, stage = stage), now + 0.5, None));
if let Some(guard) = &t.guard {
let f = if f.is_finite() { f.clamp(0.0, 1.0) } else { 0.0 };
guard.progress((f * 1000.0) as u64, 1000);
guard.detail(crate::i18n::tr(&stage));
}
ctx.request_repaint_after(std::time::Duration::from_millis(100));
}
}
crates/ui-egui/src/panels/activity.rs
+165
View File
@@ -0,0 +1,165 @@
//! The activity stack (issue #345): every long-running task in one place, floating in the top-left corner under the
//! top bar while something runs — a row each with its name, a progress bar, what it is working on, and ✕ for the ones
//! that can be cancelled. The tasks come from the engine's registry ([`lightcraft_engine::activity`]); ✕ runs
//! `activity.cancel` like any other frontend would.
use egui::{Align, Layout, Margin, Rect, RichText, Sense, Stroke, pos2, vec2};
use lightcraft_engine::activity::{TaskInfo, Unit};
use serde_json::json;
use crate::LightcraftApp;
use crate::i18n::{tr, tr_format};
use crate::icons::{Icon, paint};
use crate::theme::Tokens;
use crate::widgets::register;
/// A task shows once it has run this long, so quick ones don't flash.
pub const SHOW_AFTER_MS: u64 = 500;
/// Rows shown before "+N more".
pub const MAX_ROWS: usize = 3;
/// The stack's width (points).
pub const WIDTH: f32 = 280.0;
const MARGIN: i8 = 8;
const CROSS: f32 = 16.0;
/// "3 of 25", "12 of 340 MB", "40 %"; empty while the amount of work isn't known.
pub fn count_text(t: &TaskInfo) -> String {
if t.total == 0 {
return String::new();
}
match t.unit {
Unit::Count => tr_format!("{done} of {total}", done = t.done, total = t.total),
Unit::Bytes => tr_format!("{done} of {total} MB", done = megabytes(t.done), total = megabytes(t.total)),
Unit::Percent => tr_format!("{pct} %", pct = t.done.saturating_mul(100) / t.total),
}
}
fn megabytes(bytes: u64) -> String {
let mb = bytes as f64 / 1_048_576.0;
if mb < 10.0 { format!("{mb:.1}") } else { format!("{:.0}", mb.floor()) }
}
/// The row's second line: "Stopping…" once cancelled, else the count and what is being worked on.
fn detail_line(t: &TaskInfo) -> String {
if t.cancelling {
return tr("Stopping…").to_string();
}
let count = count_text(t);
match (count.is_empty(), t.detail.is_empty()) {
(false, false) => format!("{count} · {}", t.detail),
(false, true) => count,
(true, _) => t.detail.clone(),
}
}
/// Draw the stack (called every frame, after the panels and dialogs).
pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) {
let all = app.session.activity.list();
if all.is_empty() {
app.activity_expanded = false;
return;
}
// look again soon: a young task becomes due, bars move
ctx.request_repaint_after(std::time::Duration::from_millis(100));
let tasks: Vec<&TaskInfo> = all.iter().filter(|t| t.age_ms >= SHOW_AFTER_MS).collect();
if tasks.is_empty() {
return;
}
let t = Tokens::get(ctx);
let shown = if app.activity_expanded { tasks.len() } else { tasks.len().min(MAX_ROWS) };
let hidden = tasks.len().saturating_sub(shown);
let max_height = (ctx.content_rect().height() - t.top_bar_h - 60.0).max(80.0);
let mut cancel = None;
let mut toggle = false;
let frame = egui::Frame::NONE.fill(t.chrome).stroke(Stroke::new(1.0, t.field_border)).corner_radius(6).inner_margin(Margin::same(MARGIN));
egui::Area::new(egui::Id::new("activity-stack")).order(egui::Order::Middle).fixed_pos(pos2(8.0, t.top_bar_h + 8.0)).show(ctx, |ui| {
// An area lays out in the size it had last frame, and the scroll area would take that as all the room there
// is: the stack would never grow when rows are added. Give it the whole height below the top bar instead
// (the scroll area keeps to `max_height`).
ui.set_max_height((ctx.content_rect().height() - t.top_bar_h - 16.0).max(80.0));
frame.show(ui, |ui| {
let inner = WIDTH - 2.0 * (f32::from(MARGIN) + 1.0);
ui.set_width(inner);
ui.spacing_mut().item_spacing.y = 3.0;
egui::ScrollArea::vertical().max_height(max_height).auto_shrink([false, true]).show(ui, |ui| {
for (i, task) in tasks.iter().take(shown).enumerate() {
if i > 0 {
ui.add_space(6.0);
}
if row(ui, &t, task, inner) {
cancel = Some(task.id);
}
}
});
if hidden > 0 || (app.activity_expanded && tasks.len() > MAX_ROWS) {
ui.add_space(4.0);
let text = if hidden > 0 { tr_format!("+{n} more", n = hidden) } else { tr("Show less").to_string() };
let r = ui.add(egui::Label::new(RichText::new(text).font(t.font(11.0)).color(t.accent)).sense(Sense::click()));
register(ui.ctx(), "activity:more", r.rect);
toggle = r.clicked();
}
});
});
if let Some(id) = cancel {
self::cancel(app, id);
}
if toggle {
app.activity_expanded = !app.activity_expanded;
}
}
/// ✕ on task `id`'s row. The task may have just finished or stopped being cancellable: nothing to tell the user then,
/// so the command goes to the engine directly (`LightcraftApp::run` would put its error in the status bar).
pub(crate) fn cancel(app: &mut LightcraftApp, id: u64) {
let _ = app.session.execute("activity.cancel", &json!({"id": id}));
}
/// One task: name and ✕, the bar, the detail line. True when ✕ was clicked.
fn row(ui: &mut egui::Ui, t: &Tokens, task: &TaskInfo, width: f32) -> bool {
let mut clicked = false;
let r = ui.vertical(|ui| {
ui.set_width(width);
ui.horizontal(|ui| {
let cross = task.cancellable && !task.cancelling;
let label_w = if cross { width - CROSS - 6.0 } else { width };
ui.allocate_ui_with_layout(vec2(label_w, 18.0), Layout::left_to_right(Align::Center), |ui| {
// the whole width, so the cross lands at the right edge however short the name
ui.set_width(label_w);
ui.add(egui::Label::new(RichText::new(tr(&task.label)).font(t.font(12.5)).color(t.text)).truncate());
});
if cross {
let (rect, resp) = ui.allocate_exact_size(vec2(CROSS, CROSS), Sense::click());
paint(ui.painter(), rect.shrink(2.0), Icon::Close, if resp.hovered() { t.text } else { t.icon });
register(ui.ctx(), format!("activity:cancel:{}", task.id), rect);
clicked = resp.on_hover_text(tr("Cancel")).clicked();
}
});
bar(ui, t, task, width);
let line = detail_line(task);
if !line.is_empty() {
ui.add(egui::Label::new(RichText::new(line).font(t.font(11.0)).color(t.text_dim)).truncate());
}
});
register(ui.ctx(), format!("activity:row:{}", task.id), r.response.rect);
clicked
}
/// The progress bar; while the amount of work isn't known, a segment sweeps across.
fn bar(ui: &mut egui::Ui, t: &Tokens, task: &TaskInfo, width: f32) {
let (rect, _) = ui.allocate_exact_size(vec2(width, 5.0), Sense::hover());
let p = ui.painter();
p.rect_filled(rect, 2.5, t.field);
let fill = if task.total > 0 {
let frac = (task.done as f64 / task.total as f64).clamp(0.0, 1.0) as f32;
Rect::from_min_size(rect.min, vec2(rect.width() * frac, rect.height()))
} else {
let phase = (ui.input(|i| i.time) * 0.7).fract() as f32;
let w = rect.width() * 0.3;
let x = rect.left() + (rect.width() + w) * phase - w;
Rect::from_min_max(pos2(x.max(rect.left()), rect.top()), pos2((x + w).min(rect.right()), rect.bottom()))
};
if fill.width() > 0.0 {
p.rect_filled(fill, 2.5, if task.cancelling { t.text_dim } else { t.accent });
}
}
crates/ui-egui/src/panels/faces.rs
+3 -2
View File
@@ -669,8 +669,9 @@ pub fn pump(app: &mut LightcraftApp, ctx: &egui::Context) {
app.caches.faces_active = v["active"] == true;
app.caches.faces_indexed = v["indexedFaces"].as_u64().unwrap_or(0);
app.caches.faces_pending = v["pendingPhotos"].as_u64().unwrap_or(0);
// the progress bar's whole: the most photos that were left at once since the scan last finished
app.caches.faces_peak = if app.caches.faces_pending == 0 { 0 } else { app.caches.faces_peak.max(app.caches.faces_pending) };
// the progress bar's whole: the most photos that were left at once since the scan last finished (the engine keeps
// it, so Settings and the activity stack agree)
app.caches.faces_peak = v["peak"].as_u64().unwrap_or(0);
// Frames are drawn only when something asks for one, so nothing here wakes the window needlessly: while there is work it
// asks to be called again in 50 ms (the progress in Settings, the next photos for the workers); with recognition on and
// nothing to do it looks again every few seconds (a few wake-ups a minute); with recognition off it asks for nothing
crates/ui-egui/src/panels/mod.rs
+1
View File
@@ -1,5 +1,6 @@
//! Window regions and panels.
pub mod activity;
pub mod bottombar;
pub mod chips;
pub mod compare;
crates/ui-egui/src/panels/notices.rs
+36 -8
View File
@@ -1,12 +1,12 @@
//! Things the user must not miss (issue #103): settings files that were damaged or unreadable when
//! the library opened (and host warnings such as a damaged `ui.json`), and quitting while changes
//! are only in memory.
//! are only in memory or while tasks that would be cut short are running (issue #345).
use egui::{Align2, RichText, vec2};
use crate::LightcraftApp;
use crate::theme::Tokens;
use crate::widgets::register;
use crate::{LightcraftApp, QuitPrompt};
/// Collect the library's settings-file warnings (each is shown once).
pub fn logic(app: &mut LightcraftApp) {
@@ -14,10 +14,25 @@ pub fn logic(app: &mut LightcraftApp) {
app.notices.extend(new);
}
/// The window may close now (`true`), or the quit prompt is shown (`false`): changes that
/// couldn't be written are retried first — the log, then a snapshot of everything in memory.
/// The window may close now (`true`), or the quit prompt is shown (`false`). Running tasks that
/// can be cancelled (an import, an export…) are asked about first; then changes that couldn't be
/// written are retried — the log, then a snapshot of everything in memory.
pub fn may_close(app: &mut LightcraftApp) -> bool {
if app.quit_confirmed || app.session.unsaved().is_none() {
if app.quit_confirmed {
return true;
}
// before the library is touched: a Cancel here keeps working with it
let running = app.session.activity.running_cancellable();
if !running.is_empty() {
let names = running.iter().map(|t| crate::i18n::tr(&t.label)).collect::<Vec<_>>().join(", ");
app.quit_prompt = Some(QuitPrompt::Tasks(if running.len() == 1 {
crate::i18n::tr_format!("1 task is still running: {names}. Quitting stops it.", names = names)
} else {
crate::i18n::tr_format!("{n} tasks are still running: {names}. Quitting stops them.", n = running.len(), names = names)
}));
return false;
}
if app.session.unsaved().is_none() {
return true;
}
let _ = app.session.persist();
@@ -29,13 +44,13 @@ pub fn may_close(app: &mut LightcraftApp) -> bool {
match app.session.unsaved() {
None => true,
Some((n, e)) => {
app.quit_prompt = Some(crate::i18n::tr_format!(
app.quit_prompt = Some(QuitPrompt::Unsaved(crate::i18n::tr_format!(
"{n} change{} couldn't be written to disk: {e}\nIf you quit now, {} lost.",
if n == 1 { "" } else { "s" },
if n == 1 { "it is" } else { "they are" },
n = n,
e = e
));
)));
false
}
}
@@ -85,7 +100,20 @@ fn window(ctx: &egui::Context, id: &str, title: &str, text: &str, buttons: &[(&s
}
pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) {
if let Some(text) = app.quit_prompt.clone() {
if let Some(QuitPrompt::Tasks(text)) = app.quit_prompt.clone() {
let buttons = [("quitAnyway", "Quit Anyway", Choice::QuitAnyway), ("quitCancel", "Cancel", Choice::Cancel)];
let Some(i) = window(ctx, "quit-tasks", "Quit while tasks are running?", &text, &buttons) else { return };
app.quit_prompt = None;
if matches!(buttons[i].2, Choice::QuitAnyway) {
app.session.activity.cancel_all();
// unsaved changes still get their own question
if may_close(app) {
app.ui.quit = true;
}
}
return;
}
if let Some(QuitPrompt::Unsaved(text)) = app.quit_prompt.clone() {
let buttons = [
("quitRetry", "Try Saving Again", Choice::Retry),
("quitAnyway", "Quit Anyway", Choice::QuitAnyway),
crates/ui-egui/src/shortcuts.rs
+1 -1
View File
@@ -192,7 +192,7 @@ pub fn format(m: Modifiers, k: Key) -> Option<String> {
/// Secondary key bindings for commands that already exist: `(shortcut, command id, params JSON)`.
/// They complement the primary shortcut declared on the command (Lightroom-desktop keys that our
/// primary keymap assigns elsewhere, see docs/parity.md → Shortcuts). Shown in Help → Keyboard Shortcuts.
/// primary keymap assigns elsewhere, see docs/ui-parity.md → Shortcuts). Shown in Help → Keyboard Shortcuts.
pub const ALIASES: &[(&str, &str, &str)] = &[
("Cmd+D", "library.selectNone", "{}"),
("Shift+E", "dialog.export", "{}"),
crates/ui-egui/src/sync.rs
+26 -32
View File
@@ -1,11 +1,13 @@
//! Synchronize Folder in the app: the folder is scanned on a worker thread (a network share can
//! take minutes; no frame waits for it) while its dialog is open, and the dialog then says what
//! changed and what to do about it. Synchronize runs `folder.synchronize`, which acts on exactly
//! the scan the dialog showed (see `lightcraft_engine::sync`).
//! the scan the dialog showed (see `lightcraft_engine::sync`). Both the scan and the run show a row in the activity
//! stack (issue #345); its ✕ stops them like the dialog's Cancel.
use std::sync::Arc;
use std::sync::atomic::Ordering;
use lightcraft_engine::activity::{Cancel, TaskGuard};
use lightcraft_engine::sync::{FolderChanges, SyncChoice, SyncCommit, SyncInput, SyncJob, SyncProgress, SyncStep, scan_with};
use serde_json::{Value, json};
@@ -18,6 +20,8 @@ pub struct SyncTask {
path: String,
progress: Arc<SyncProgress>,
rx: std::sync::mpsc::Receiver<FolderChanges>,
/// Its row in the activity stack; ✕ cancels the scan and closes the dialog.
guard: TaskGuard,
}
/// Open the dialog for folder `path` (called `name` in it) and start scanning.
@@ -34,7 +38,9 @@ pub fn open(app: &mut LightcraftApp, path: &str, name: &str, disk: bool) -> Resu
std::thread::spawn(job);
#[cfg(target_arch = "wasm32")]
job();
app.sync = Some(SyncTask { path: path.to_string(), progress, rx });
let guard = app.session.activity.start("sync", "Looking for changes", Cancel::Flag(progress.files.cancel.clone()));
guard.detail(name);
app.sync = Some(SyncTask { path: path.to_string(), progress, rx, guard });
let d = SyncChoice::default();
app.ui.dialog = Some(Dialog::SynchronizeFolder {
path: path.to_string(),
@@ -78,6 +84,14 @@ pub fn poll(app: &mut LightcraftApp, ctx: &egui::Context) {
cancel(app);
return;
}
if task.guard.is_cancelled() {
// ✕ on its row: a scan stopped part-way has nothing to show
cancel(app);
app.ui.dialog = None;
return;
}
let (done, total) = task.progress.counts();
task.guard.progress(done as u64, total as u64);
ctx.request_repaint_after(std::time::Duration::from_millis(100));
let changes = match task.rx.try_recv() {
Ok(c) => c,
@@ -210,7 +224,9 @@ pub fn confirm(app: &mut LightcraftApp, dlg: &Dialog) -> Result<Value, String> {
std::thread::spawn(job);
#[cfg(target_arch = "wasm32")]
job();
app.sync_run = Some(SyncRun { name: name.clone(), progress, crashed, rx, commit: Some(commit), stopping: false, not_saved: Vec::new() });
let guard = app.session.activity.start("sync", "Synchronizing folder", Cancel::Flag(progress.files.cancel.clone()));
guard.detail(name);
app.sync_run = Some(SyncRun { name: name.clone(), progress, crashed, rx, commit: Some(commit), stopping: false, not_saved: Vec::new(), guard });
Ok(json!({"started": true}))
}
@@ -227,6 +243,8 @@ pub struct SyncRun {
commit: Option<SyncCommit>,
/// Cancel was pressed: nothing new is started; what was readied is still committed.
stopping: bool,
/// Its row in the activity stack (its ✕ is Cancel).
guard: TaskGuard,
}
impl SyncRun {
@@ -253,6 +271,11 @@ impl Drop for SyncRun {
pub fn poll_run(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(run) = app.sync_run.as_mut() else { return };
ctx.request_repaint_after(std::time::Duration::from_millis(100));
if run.guard.is_cancelled() {
run.stopping = true;
}
let (done, total) = run.counts();
run.guard.progress(done as u64, total as u64);
let mut steps = Vec::new();
let finished = loop {
match run.rx.try_recv() {
@@ -317,32 +340,3 @@ pub fn poll_run(app: &mut LightcraftApp, ctx: &egui::Context) {
app.toast_for(ctx, format!("{name}: {text}"), 12.0);
}
}
/// The progress window while a synchronize runs, with Cancel (what was done stays, as one undo
/// step; nothing new is started).
pub fn progress_window(app: &mut LightcraftApp, ctx: &egui::Context) {
let Some(run) = &app.sync_run else { return };
let t = Tokens::get(ctx);
let (done, total) = run.counts();
let text =
if run.stopping { crate::i18n::tr("Stopping…").to_string() } else { format!("{} {done} / {total}", crate::i18n::tr("Synchronizing…")) };
let frac = if total == 0 { 0.0 } else { done as f32 / total as f32 };
let stopping = run.stopping;
let mut cancel = false;
egui::Window::new(crate::i18n::tr("Synchronize Folder"))
.id(egui::Id::new("sync-progress"))
.title_bar(false)
.resizable(false)
.anchor(egui::Align2::CENTER_BOTTOM, [0.0, -80.0])
.fixed_size([340.0, 80.0])
.show(ctx, |ui| {
ui.label(egui::RichText::new(text).color(t.text));
ui.add(egui::ProgressBar::new(frac).desired_width(320.0));
let r = ui.add_enabled(!stopping, egui::Button::new(crate::i18n::tr("Cancel")));
crate::widgets::register(ui.ctx(), "button:syncCancel", r.rect);
cancel = r.clicked();
});
if cancel && let Some(run) = app.sync_run.as_mut() {
run.cancel();
}
}
crates/ui-egui/src/tasks.rs
+9 -3
View File
@@ -1,7 +1,8 @@
//! Background tasks for commands whose file-system work can take long on a slow or offline drive
//! (Find Missing Photos walking a folder, listing the auto-import folder…): the work runs on a
//! worker thread and its result is applied on the UI thread between frames, so the window keeps
//! answering. Imports, scans, exports and preview builds have their own tasks with progress.
//! answering. Imports, scans, exports and preview builds have their own tasks with progress. A task the user started
//! (Find Missing Photos) shows in the activity stack while it runs; quiet ones (the auto-import listing) don't.
use std::sync::mpsc::{Receiver, TryRecvError, channel};
@@ -13,6 +14,8 @@ type Finish = Box<dyn FnOnce(&mut LightcraftApp, &egui::Context) + Send>;
struct Task {
label: String,
rx: Receiver<Finish>,
/// Its row in the activity stack (none for quiet tasks); goes with the task.
_guard: Option<lightcraft_engine::activity::TaskGuard>,
}
/// The background tasks in flight.
@@ -39,10 +42,12 @@ impl Tasks {
}
/// Run `work` on a worker thread, then `done(app, ctx, result)` on the UI thread (in the browser
/// build, which has no threads, `work` runs at once and `done` on the next frame).
/// build, which has no threads, `work` runs at once and `done` on the next frame). With `kind`, the
/// activity stack shows `label` meanwhile, without a count and without ✕ (the work can't be stopped).
pub fn spawn<T: Send + 'static>(
app: &mut LightcraftApp,
label: &str,
kind: Option<&'static str>,
work: impl FnOnce() -> T + Send + 'static,
done: impl FnOnce(&mut LightcraftApp, &egui::Context, T) + Send + 'static,
) -> Result<(), String> {
@@ -59,7 +64,8 @@ pub fn spawn<T: Send + 'static>(
std::thread::Builder::new().name(format!("lc-task-{label}")).spawn(job).map_err(|e| format!("{label}: could not start: {e}"))?;
#[cfg(target_arch = "wasm32")]
job();
app.tasks.running.push(Task { label: label.to_string(), rx });
let guard = kind.map(|k| app.session.activity.start(k, label, lightcraft_engine::activity::Cancel::No));
app.tasks.running.push(Task { label: label.to_string(), rx, _guard: guard });
Ok(())
}
crates/ui-egui/src/tests_activity.rs
+395
View File
@@ -0,0 +1,395 @@
//! Issue #345: the activity stack — one place, top-left, for every long-running task, with ✕ to cancel.
use std::time::Duration;
use lightcraft_engine::activity::{Cancel, TaskInfo, Unit};
use serde_json::json;
use crate::headless::Headless;
use crate::panels::activity::{WIDTH, count_text};
use crate::{LightcraftApp, Services};
const T: Duration = Duration::from_secs(20);
fn demo() -> Headless {
let app = LightcraftApp::new(lightcraft_engine::Session::with_demo(), Services { png: None, ..Default::default() });
let mut h = Headless::new(app, [1200.0, 800.0], 1.0);
// let the first renders finish: a render still running when the test binary exits can crash in the
// GPU driver's teardown (the preview pool doesn't join its workers)
h.settle(T);
h
}
fn has(h: &Headless, id: &str) -> bool {
h.app.widgets.iter().any(|(w, _)| w == id)
}
fn rows(h: &Headless) -> usize {
h.app.widgets.iter().filter(|(w, _)| w.starts_with("activity:row:")).count()
}
fn rect(h: &Headless, id: &str) -> egui::Rect {
h.app.widgets.iter().find(|(w, _)| w == id).map(|(_, r)| *r).unwrap_or_else(|| panic!("no widget {id}"))
}
/// Rows show once their task is half a second old.
fn wait_visible(h: &mut Headless) {
std::thread::sleep(Duration::from_millis(600));
h.step();
h.step();
}
/// Click a widget and let the synthetic events (move, press, release) play out.
fn click(h: &mut Headless, id: &str) {
h.step();
h.step();
assert_eq!(h.request("ui.clickWidget", json!({"id": id}), T)["ok"], true, "{id}");
for _ in 0..4 {
h.step();
}
}
#[test]
fn rows_appear_after_half_a_second_and_cancel_by_click() {
let mut h = demo();
let g = h.app.session.activity.start("export", "Exporting", Cancel::Yes);
g.progress(2, 5);
h.step();
// a frame on a loaded machine can outlast the half second: only judge a task still young after it
if h.app.session.activity.list()[0].age_ms < crate::panels::activity::SHOW_AFTER_MS {
assert!(!has(&h, &format!("activity:row:{}", g.id())), "not before 0.5 s");
}
wait_visible(&mut h);
assert!(has(&h, &format!("activity:row:{}", g.id())));
click(&mut h, &format!("activity:cancel:{}", g.id()));
assert!(g.is_cancelled());
let inspect = h.request("ui.inspect", json!({}), T);
assert_eq!(inspect["result"]["activity"][0]["cancelling"], true, "{}", inspect["result"]["activity"]);
}
#[test]
fn more_than_three_rows_overflow_and_expand() {
let mut h = demo();
let guards: Vec<_> = (0..5).map(|i| h.app.session.activity.start("export", &format!("Task {i}"), Cancel::Yes)).collect();
wait_visible(&mut h);
assert_eq!(rows(&h), 3);
assert!(has(&h, "activity:more"));
click(&mut h, "activity:more");
assert_eq!(rows(&h), 5);
// expanded, the stack grows to show them all
let stack = h.view.ctx.memory(|m| m.area_rect(egui::Id::new("activity-stack"))).expect("the stack is drawn");
for g in &guards {
let r = rect(&h, &format!("activity:row:{}", g.id()));
assert!(stack.contains_rect(r), "row {} at {r:?} is cut off by the stack at {stack:?}", g.id());
}
drop(guards);
h.step();
assert_eq!(rows(&h), 0);
}
#[test]
fn not_cancellable_rows_have_no_cross() {
let mut h = demo();
let g = h.app.session.activity.start("faces", "Finding faces", Cancel::No);
wait_visible(&mut h);
assert!(has(&h, &format!("activity:row:{}", g.id())));
assert!(!has(&h, &format!("activity:cancel:{}", g.id())));
}
#[test]
fn count_text_by_unit() {
let mut t = TaskInfo {
id: 1,
kind: "export",
label: "Exporting".into(),
done: 3,
total: 25,
unit: Unit::Count,
detail: String::new(),
cancellable: true,
cancelling: false,
age_ms: 600,
};
assert_eq!(count_text(&t), "3 of 25");
(t.done, t.total, t.unit) = (12 * 1_048_576, 340 * 1_048_576, Unit::Bytes);
assert_eq!(count_text(&t), "12 of 340 MB");
(t.done, t.total, t.unit) = (400, 1000, Unit::Percent);
assert_eq!(count_text(&t), "40 %");
t.total = 0;
assert_eq!(count_text(&t), "");
}
#[test]
fn stale_cancel_is_harmless() {
let mut h = demo();
let g = h.app.session.activity.start("export", "Exporting", Cancel::Yes);
wait_visible(&mut h);
let id = g.id();
drop(g);
let r = h.request("engine.execute", json!({"command": "activity.cancel", "params": {"id": id}}), T);
assert_eq!(r["ok"], false, "an error result, not a panic: {r}");
h.step();
assert!(h.app.ui.toast.is_none(), "{:?}", h.app.ui.toast);
// the same through the row's ✕ (clicked in the frame the task ended): no message in the status bar either
h.app.ui.status = "Ready".into();
crate::panels::activity::cancel(&mut h.app, id);
h.step();
assert_eq!(h.app.ui.status, "Ready");
assert!(h.app.ui.toast.is_none(), "{:?}", h.app.ui.toast);
}
#[test]
fn long_detail_stays_inside_the_stack() {
let mut h = demo();
let g = h.app.session.activity.start("export", "Exportieren", Cancel::Yes);
g.detail(&"IMG_".repeat(75));
wait_visible(&mut h);
let row = rect(&h, &format!("activity:row:{}", g.id()));
assert!(row.width() <= WIDTH + 1.0, "{row:?}");
}
#[test]
fn cross_sits_at_the_right_edge_of_its_row() {
let mut h = demo();
let g = h.app.session.activity.start("export", "Exporting", Cancel::Yes);
wait_visible(&mut h);
let row = rect(&h, &format!("activity:row:{}", g.id()));
let cross = rect(&h, &format!("activity:cancel:{}", g.id()));
assert!(cross.right() >= row.right() - 2.0, "the cross is at the right, not after the label: {cross:?} in {row:?}");
}
#[test]
fn quit_with_a_running_task_asks_and_quit_anyway_cancels() {
let mut h = demo();
let g = h.app.session.activity.start("export", "Exporting", Cancel::Yes);
wait_visible(&mut h);
assert!(!crate::panels::notices::may_close(&mut h.app));
assert!(matches!(h.app.quit_prompt, Some(crate::QuitPrompt::Tasks(_))), "{:?}", h.app.quit_prompt);
h.step();
assert!(!has(&h, "button:quitRetry"));
click(&mut h, "button:quitAnyway");
assert!(g.is_cancelled() && h.quit_requested());
}
#[test]
fn quit_cancel_keeps_the_task_running() {
let mut h = demo();
let g = h.app.session.activity.start("import", "Importing", Cancel::Yes);
assert!(!crate::panels::notices::may_close(&mut h.app));
click(&mut h, "button:quitCancel");
assert!(!g.is_cancelled() && h.app.quit_prompt.is_none() && !h.app.quit_confirmed);
}
#[test]
fn cancelling_task_does_not_block_quit() {
let mut h = demo();
let g = h.app.session.activity.start("export", "Exporting", Cancel::Yes);
h.app.session.activity.cancel(g.id()).unwrap();
let _f = h.app.session.activity.start("faces", "Finding faces", Cancel::No);
assert!(crate::panels::notices::may_close(&mut h.app));
}
/// A slow disk: every file takes `ms` to write (the UI must not wait for it).
fn slow_writer(ms: u64) -> crate::SharedWrite {
std::sync::Arc::new(move |_: &str, _: &[u8]| {
std::thread::sleep(Duration::from_millis(ms));
Ok(())
})
}
/// Export the first `n` photos in view through the Export dialog (small JPEGs, a made-up folder).
fn start_export(h: &mut Headless, n: usize) {
let ids: Vec<u64> = h.app.session.visible_cloned().iter().take(n).map(|p| p.0).collect();
assert_eq!(ids.len(), n, "the demo library has {n} photos");
h.request("engine.execute", json!({"command": "library.select", "params": {"ids": ids}}), T);
h.request("engine.execute", json!({"command": "dialog.export", "params": {}}), T);
h.step();
if let Some(crate::state::Dialog::Export { full_size, resize, dir, .. }) = &mut h.app.ui.dialog {
*full_size = false;
*resize = lightcraft_engine::export::Resize::long_edge(64);
*dir = "/lc-test-out".into();
}
let r = h.request("ui.dialog.confirm", json!({}), T);
assert_eq!(r["ok"], true, "{r}");
}
#[test]
fn export_shows_a_row_and_stops_on_activity_cancel() {
let mut h = demo();
h.app.services.write_shared = Some(slow_writer(200));
start_export(&mut h, 5);
assert!(h.app.export.is_some(), "running in the background");
let tasks = h.request("ui.inspect", json!({}), T)["result"]["activity"].clone();
assert_eq!(tasks[0]["kind"], "export", "{tasks}");
assert_eq!(tasks[0]["total"], 5, "{tasks}");
let id = tasks[0]["id"].as_u64().unwrap();
assert_eq!(h.request("engine.execute", json!({"command": "activity.cancel", "params": {"id": id}}), T)["ok"], true);
assert!(h.step_until(Duration::from_secs(60), |h| h.app.export.is_none()));
assert!(h.app.session.activity.list().is_empty());
assert!(h.app.last_export_result.as_ref().is_some_and(|r| r["cancelled"] == true), "{:?}", h.app.last_export_result);
assert!(!has(&h, "button:exportCancel"), "the old window is gone");
}
#[test]
fn dead_export_worker_leaves_no_row() {
let mut h = demo();
h.app.services.write_shared = Some(std::sync::Arc::new(|_: &str, _: &[u8]| -> Result<(), String> { panic!("synthetic writer panic") }));
start_export(&mut h, 2);
assert!(h.step_until(Duration::from_secs(60), |h| h.app.export.is_none()));
assert!(h.app.session.activity.list().is_empty());
}
/// A folder of `n` stand-in JPEGs and an app whose file probe takes 100 ms each (a slow drive).
fn slow_import(tag: &str, n: usize) -> (Headless, std::path::PathBuf) {
let dir = std::env::temp_dir().join(format!("lc-activity-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
for i in 0..n {
std::fs::write(dir.join(format!("IMG_{i:03}.jpg")), format!("not really a jpeg {i}")).unwrap();
}
let mut s = lightcraft_engine::Session::new();
s.media.file_probe = Some(std::sync::Arc::new(|p: &str| {
std::thread::sleep(Duration::from_millis(100));
Ok(lightcraft_engine::media::ProbeInfo {
width: 60,
height: 40,
format: "JPEG".into(),
content_hash: Some(p.to_string()),
..Default::default()
})
}));
let mut h = Headless::new(LightcraftApp::new(s, Services { png: None, ..Default::default() }), [1200.0, 800.0], 1.0);
h.settle(T);
(h, dir)
}
#[test]
fn import_cancelled_by_command_reports_cancelled() {
let n = 40;
let (mut h, dir) = slow_import("cancel", n);
let undo0 = h.app.session.undo.len();
crate::import::start_paths(&mut h.app, vec![dir.to_string_lossy().to_string()]).unwrap();
assert!(h.step_until(Duration::from_secs(60), |h| h.app.import.as_ref().is_some_and(|t| t.imported >= 2)));
let tasks = h.app.session.activity.list();
assert_eq!(tasks.first().map(|t| t.kind), Some("import"), "{tasks:?}");
h.app.session.activity.cancel(tasks[0].id).unwrap();
assert!(h.step_until(Duration::from_secs(60), |h| h.app.import.is_none()));
assert!(h.app.session.activity.list().is_empty());
assert!(h.app.ui.toast.as_ref().is_some_and(|t| t.0.starts_with("Import cancelled")), "{:?}", h.app.ui.toast);
assert!(h.app.session.catalog.len() < n, "stopped part-way");
assert_eq!(h.app.session.undo.len(), undo0 + 1, "one undo step");
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn scan_shows_a_row_and_cancel_closes_it() {
let (mut h, dir) = slow_import("scan", 120);
crate::import::open(&mut h.app, vec![dir.to_string_lossy().to_string()]).unwrap();
h.step();
let tasks = h.app.session.activity.list();
assert_eq!(tasks.first().map(|t| t.kind), Some("scan"), "{tasks:?}");
assert!(tasks[0].cancellable);
h.app.session.activity.cancel(tasks[0].id).unwrap();
// like the old Cancel: the scan is dropped at once, without waiting for the file being read
assert!(h.step_until(Duration::from_secs(10), |h| h.app.scan.is_none()));
assert!(h.app.session.activity.list().is_empty());
for _ in 0..3 {
h.step();
}
assert!(h.app.ui.dialog.is_none(), "no review opened");
let _ = std::fs::remove_dir_all(&dir);
}
/// A library of three bracketed DNGs (−2, 0, +2 EV) on disk, all selected.
fn bracket(tag: &str) -> (Headless, std::path::PathBuf) {
let dir = std::env::temp_dir().join(format!("lc-activity-{tag}-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let mut paths = Vec::new();
for (i, bytes) in lightcraft_merge::synth::bracket_dngs(1600, 1200, &[-2.0, 0.0, 2.0]).unwrap().into_iter().enumerate() {
let p = dir.join(format!("IMG_{i}.dng"));
std::fs::write(&p, bytes).unwrap();
paths.push(p.to_string_lossy().to_string());
}
let mut session = lightcraft_engine::Session::new().with_fs();
let r = session.execute("library.import", &json!({"paths": paths})).unwrap();
let ids: Vec<u64> = r["imported"].as_array().unwrap().iter().map(|v| v.as_u64().unwrap()).collect();
session.execute("library.select", &json!({"ids": ids})).unwrap();
let app = LightcraftApp::new(session, Services { png: None, ..Default::default() });
let mut h = Headless::new(app, [1200.0, 800.0], 1.0);
h.settle(T);
(h, dir)
}
#[test]
fn final_merge_shows_a_row_and_cancels() {
let (mut h, dir) = bracket("merge");
let before = h.app.session.catalog.len();
crate::merge::start_last(&mut h.app, "merge.hdr").unwrap();
h.step();
let rows = h.app.session.activity.list();
assert_eq!(rows.len(), 1);
assert_eq!((rows[0].kind, rows[0].label.as_str(), rows[0].unit), ("merge", "Merging", Unit::Percent));
assert!(rows[0].cancellable);
assert_eq!(rows[0].total, 1000);
h.app.session.activity.cancel(rows[0].id).unwrap();
assert!(h.step_until(T, |h| h.app.merge.final_task.is_none()));
assert!(h.app.merge.last_result.is_none(), "nothing merged");
assert_eq!(h.app.session.catalog.len(), before, "no photo added");
assert!(h.app.session.activity.list().is_empty());
let toast = h.app.ui.toast.as_ref().map(|t| t.0.clone()).unwrap_or_default();
assert_eq!(toast, "HDR merge cancelled");
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn find_missing_shows_an_indeterminate_row() {
let mut h = demo();
let dir = std::env::temp_dir().join(format!("lc-activity-findmissing-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
// run directly, not through a request (which runs frames): the search of an empty folder is over at once, but its
// result is applied between frames, and the row is there until then
let r = crate::menus::run_ui_command(&mut h.app, "file.findMissing", &json!({"folder": dir.to_string_lossy()})).unwrap().unwrap();
assert_eq!(r["background"], true, "{r}");
let rows = h.app.session.activity.list();
assert_eq!(rows.len(), 1, "{rows:?}");
assert_eq!((rows[0].kind, rows[0].label.as_str()), ("findMissing", "Find Missing Photos"));
assert_eq!((rows[0].total, rows[0].cancellable), (0, false), "indeterminate, no ✕");
assert!(h.step_until(T, |h| h.app.tasks.is_empty()));
assert!(h.app.session.activity.list().is_empty(), "the row goes with the task");
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn auto_import_listing_has_no_row() {
let mut h = demo();
let dir = std::env::temp_dir().join(format!("lc-activity-autoimport-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
h.app.session.import_defaults.auto_folder = Some(dir.to_string_lossy().to_string());
// the watched folder is listed every few seconds, quietly
assert!(h.step_until(Duration::from_secs(10), |h| h.app.tasks.is_running("Auto Import")));
assert!(h.app.session.activity.list().is_empty(), "{:?}", h.app.session.activity.list());
h.app.session.import_defaults.auto_folder = None;
let _ = std::fs::remove_dir_all(dir);
}
#[test]
fn rows_added_later_grow_the_stack() {
let mut h = demo();
let a = h.app.session.activity.start("export", "Exporting", Cancel::Yes);
a.progress(1, 24);
a.detail("LC01347.jpg");
wait_visible(&mut h);
// two more tasks while the stack already shows one row: it grows to show them, nothing is cut off
let b = h.app.session.activity.start("previews", "Building previews", Cancel::Yes);
let c = h.app.session.activity.start("lightroom", "Reading Lightroom catalog", Cancel::Yes);
wait_visible(&mut h);
for _ in 0..5 {
h.step();
}
let stack = h.view.ctx.memory(|m| m.area_rect(egui::Id::new("activity-stack"))).expect("the stack is drawn");
for g in [&a, &b, &c] {
let r = rect(&h, &format!("activity:row:{}", g.id()));
assert!(stack.contains_rect(r), "row {} at {r:?} is cut off by the stack at {stack:?}", g.id());
}
}
crates/ui-egui/src/tests_offline.rs
+7 -3
View File
@@ -123,7 +123,8 @@ fn import_reads_files_off_the_ui_thread_and_can_be_cancelled() {
let dir = std::env::temp_dir().join(format!("lc-ui-slow-import-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let n = 40;
// enough files that the import is still running when its activity row shows (after 0.5 s)
let n = 120;
for i in 0..n {
std::fs::write(dir.join(format!("IMG_{i:03}.jpg")), format!("not really a jpeg {i}")).unwrap();
}
@@ -163,8 +164,11 @@ fn import_reads_files_off_the_ui_thread_and_can_be_cancelled() {
assert!(h.app.import.is_some(), "still importing");
let status = h.app.import.as_ref().map(crate::import::ImportTask::status).unwrap_or_default();
assert!(status["total"].as_u64().unwrap_or(0) >= 8, "{status}");
// Cancel: the files being read finish, nothing new starts
let r = h.request("ui.clickWidget", json!({"id": "button:importCancel"}), Duration::from_secs(10));
// Cancel (the ✕ of its row in the activity stack): the files being read finish, nothing new starts
let id = h.app.session.activity.list().first().map(|t| t.id).expect("an import row");
let cross = format!("activity:cancel:{id}");
assert!(h.step_until(Duration::from_secs(10), |h| h.app.widgets.iter().any(|(w, _)| *w == cross)), "the row shows");
let r = h.request("ui.clickWidget", json!({"id": cross}), Duration::from_secs(10));
assert_eq!(r["ok"], true, "{r}");
let t0 = Instant::now();
while h.app.import.is_some() && t0.elapsed() < Duration::from_secs(60) {
crates/ui-egui/src/tests_quit_unsaved.rs
+1 -1
View File
@@ -79,7 +79,7 @@ fn quitting_with_unsaved_changes_asks_first() {
assert_eq!(r["ok"], false, "{r}");
assert!(!crate::panels::notices::may_close(&mut h.app), "retried, still failing: ask");
h.step();
let prompt = h.app.quit_prompt.clone().unwrap();
let prompt = h.app.quit_prompt.clone().unwrap().text().to_string();
assert!(prompt.contains("1 change couldn't be written to disk") && prompt.contains("drive unplugged"), "{prompt}");
for b in ["button:quitRetry", "button:quitAnyway", "button:quitCancel"] {
assert!(has(&h, b), "{b}");
crates/ui-egui/src/tests_sync.rs
+50 -2
View File
@@ -9,7 +9,7 @@
//! * Ticking "Remove missing photos" also moves those to Recently Deleted.
//! * Cancel leaves the library as it was.
//! * Synchronize hands the work to the background at once: the dialog closes, the app keeps
//! answering, and the photos arrive over the next frames under a progress window, as one undo
//! answering, and the photos arrive over the next frames under a row in the activity stack, as one undo
//! step.
use std::time::Duration;
@@ -218,7 +218,7 @@ fn a_stopped_synchronize_says_it_was_stopped_and_stays_at_most_one_step() {
let steps = h.app.session.undo.len();
let dlg = h.app.ui.dialog.take().expect("the dialog is open");
assert!(crate::panels::dialogs::confirm_dialog(&mut h.app, &dlg).is_ok());
// the progress window's Cancel, before any frame committed anything
// Cancel (what the row's ✕ does), before any frame committed anything
h.app.sync_run.as_mut().expect("running").cancel();
assert!(h.step_until(T, |h| h.app.sync_run.is_none()), "it ends");
let toast = h.app.ui.toast.as_ref().map(|t| t.0.clone()).unwrap_or_default();
@@ -254,3 +254,51 @@ fn a_synchronize_and_an_import_never_run_at_once() {
let r = crate::panels::dialogs::confirm_dialog(&mut h.app, &d);
assert!(r.is_err() && h.app.sync_run.is_none(), "no synchronize while importing: {r:?}");
}
#[test]
fn a_synchronize_shows_a_row_and_its_cross_stops_it() {
let dir = Scratch::new("row");
let mut h = changed_folder(&dir);
open_and_scan(&mut h, &dir.path("trip"));
let dlg = h.app.ui.dialog.take().expect("the dialog is open");
assert!(crate::panels::dialogs::confirm_dialog(&mut h.app, &dlg).is_ok());
let rows = h.app.session.activity.list();
assert_eq!(rows.len(), 1, "{rows:?}");
assert_eq!((rows[0].kind, rows[0].label.as_str()), ("sync", "Synchronizing folder"));
assert!(rows[0].detail.ends_with("trip"), "the folder as the dialog named it: {:?}", rows[0].detail);
assert!(rows[0].cancellable);
// the row's ✕ (as `activity.cancel`): what was done stays, and the end says it was stopped
h.app.session.activity.cancel(rows[0].id).unwrap();
assert!(h.step_until(T, |h| h.app.sync_run.is_none()), "it ends");
let toast = h.app.ui.toast.as_ref().map(|t| t.0.clone()).unwrap_or_default();
assert!(toast.contains("Stopped"), "{toast:?}");
assert!(h.app.session.activity.list().is_empty(), "the row goes with the run");
}
#[test]
fn the_folders_scan_shows_a_row_and_its_cross_closes_the_dialog() {
let dir = Scratch::new("scanrow");
let mut h = changed_folder(&dir);
// a scan that can't finish until the test lets it: reading the new file waits on a flag
let gate = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false));
let g = gate.clone();
h.app.session.media.file_probe = Some(std::sync::Arc::new(move |_: &str| {
while !g.load(std::sync::atomic::Ordering::Relaxed) {
std::thread::sleep(Duration::from_millis(5));
}
Ok(lightcraft_engine::media::ProbeInfo { format: "PNG".into(), ..Default::default() })
}));
crate::sync::open(&mut h.app, &dir.path("trip"), "trip", false).unwrap();
let rows = h.app.session.activity.list();
assert_eq!(rows.len(), 1, "{rows:?}");
assert_eq!((rows[0].kind, rows[0].label.as_str(), rows[0].detail.as_str()), ("sync", "Looking for changes", "trip"));
assert!(rows[0].cancellable);
h.app.session.activity.cancel(rows[0].id).unwrap();
h.step();
assert!(h.app.sync.is_none(), "✕ stops the scan");
assert!(h.app.ui.dialog.is_none(), "and closes its dialog: there is nothing to show");
gate.store(true, std::sync::atomic::Ordering::Relaxed);
assert!(h.app.session.activity.list().is_empty());
h.settle(SETTLE);
assert!(h.app.ui.dialog.is_none(), "a stopped scan opens nothing later");
}
docs/architecture.md
+70
View File
@@ -0,0 +1,70 @@
# Architecture
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (created from the code; the full design notes stay in the local `plan/architecture.md`) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
How LightCraft is built today. The longer design document (`plan/architecture.md`) is local-only
(`plan/` is gitignored); this page is the committed, current summary. Sizes are Rust lines counted
on 2026-10-10 (≈ 202,000 tracked lines in 505 files: 24 library crates, 3 apps and `xtask`).
## Layers
`cargo xtask layers` (part of `cargo xtask ci`, `xtask/src/layers.rs`) enforces that nothing below
L5 depends on egui, eframe, winit or rfd, and that each crate only uses lower layers.
| Layer | Crates (lines) | Role |
|---|---|---|
| L0 | `geom` (1.4k), `color` (0.9k), `raster` (0.8k), `tiff` (2.0k), `fetch` (1.4k), `sysmem` (0.1k) | Matrices and warps, colour spaces and transfer functions, float image buffers, TIFF/IFD read and write, pinned HTTPS model downloads, the one `unsafe` FFI call (`malloc_zone_pressure_relief`) |
| L1 | `raw` (22.3k), `codecs` (7.3k), `meta` (4.3k), `develop` (2.5k), `denoise-core` (2.2k), `denoise` (2.7k), `faces` (5.3k), `scenes` (1.0k), `heif` (2.1k, optional) | Raw decoders and demosaic, DNG colour model; standard codecs and encoders; EXIF/XMP/IPTC; the non-destructive edit model and control specs; AI denoise; face detection and recognition; procedural demo photos; HEIC |
| L2 | `pipeline` (9.8k) | The develop pipeline: scene-referred float, resolution independent; the CPU reference (oracle) for everything else |
| L3 | `catalog` (11.0k), `gpu` (5.1k), `preview` (1.3k), `merge` (3.8k), `segment` (2.3k) | Library model and persistence; wgpu compute twin of the pipeline; thumbnail/preview cache and job pool; HDR and panorama merge; SAM 3 segmentation |
| L4 | `engine` (53.2k) | Session, command registry (everything is a command), history, render and export jobs, import, Lightroom catalog and preset import, faces index, view-models |
| L5 | `ui-egui` (46.9k), `mcp` (2.3k) | The Lightroom-style egui interface (swappable); the MCP server |
| L6 | `apps/lightcraft` (2.8k), `apps/lightcraft-cli` (1.5k), `apps/lightcraft-web` (3.4k), `xtask` (2.7k) | Desktop app (eframe + wgpu, native menus, control server), CLI (render, commands, calibrate, snapshot, MCP), WASM build, tooling |
## Data model
- **Photo** records in `catalog` hold the original's path, content hash, metadata, ratings, flags,
labels, keywords, faces and the current `develop::Settings`. Albums, smart albums (rule trees),
stacks, virtual copies, versions and folder records live alongside.
- **Persistence** (`crates/catalog/src/journal.rs`): an append-only journal of human-readable
operations plus periodic snapshots, compacted in the background; torn appends and crashes are
tested (`tests_torn_append.rs`, `tests_journal.rs`). A lock file guards against two writers.
- **Edits** are a `develop::Settings` value (every slider is a control spec with id, range and
default), normalized to image coordinates so previews and exports match. Process versions
(`docs/process-versions.md`) keep old edits rendering the same after a retune.
- **Sidecars**: XMP read and write with Adobe `crs:` interop (`docs/xmp-interop.md`); Lightroom
Classic `.lrcat` import through a pure-Rust SQLite reader (`docs/lightroom-catalog-import.md`).
## Rendering
1. **Decode** (`raw`, `codecs`): raw files to linear camera RGB (demosaic, black/white levels,
opcodes, highlight handling) and a camera-to-XYZ colour model; standard images to linear light
through their ICC profile.
2. **Develop** (`pipeline`): white balance, profile, exposure and tone (local Laplacian
highlights/shadows), curves, HSL / colour mixer, point colour, colour grading, texture, clarity,
dehaze, detail (sharpening, NR, AI denoise result), optics (lens profile, CA, defringe,
vignette), geometry (crop, Upright, transforms), masks with local adjustments, heal/clone, grain,
effects. Stage caching and a source pyramid make slider drags cheap.
3. **GPU** (`gpu`): the same stages in WGSL on Metal / DX12 / Vulkan, checked against the CPU path
(within 1/255); GPU errors fall back to the CPU (`docs/gpu-pipeline.md`).
4. **Display and export**: display-profile conversion (`docs/display-profiles.md`); export through
one encoder for app, CLI, MCP and web (`lightcraft_engine::export`).
## Agent control
Every action is an engine command (id, label, menu path, shortcut, params, enabled, run). The UI,
the CLI, the JSON-lines control channel (`docs/control-protocol.md`) and MCP (`docs/mcp.md`) all
dispatch by id, and headless snapshots render the UI without a window.
## Quality gates
`cargo xtask ci`: fmt, clippy `-D warnings` with the never-crash lints, the HEIF build, tests,
`parity` (every id and path cited in [parity-checklist.md](parity-checklist.md) exists), `layers`,
`assets` (every asset attributed, no Adobe formats) and the WASM build. Other workflows build and
test FreeBSD and Windows ARM64, lint packaging, and build releases.
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-10 | major | Created from the code (crate sizes, layers, data model, rendering, control, gates) |
docs/background-tasks.md
+65
View File
@@ -0,0 +1,65 @@
# Long-running tasks and the activity stack
Everything that takes a while runs off the UI thread and shows up in **one** place: the activity stack in the
top-left corner under the top bar (issue #345). Each task has a row with its name, a progress bar (a sweep while
the amount of work isn't known), the count and what it's working on, and ✕ if it can stop. Quitting asks first while
a task that can stop is running. Agents see the same list through `activity.list` / `activity.cancel`, and
`ui.inspect` reports it as `activity`.
The registry is `lightcraft_engine::activity` (`crates/engine/src/activity.rs`, `Session::activity`). The stack is
`crates/ui-egui/src/panels/activity.rs`. Every job that can last more than a moment uses it; **don't add a progress
window, panel or progress toast of your own**. A dialog may still show progress inline (the Synchronize Folder
dialog counts files while it scans), and a toast at the end saying what happened is fine.
## Adding a task
```rust
use lightcraft_engine::activity::{Cancel, Unit};
// when the job starts, with the cancel flag the worker already checks between files
let guard = app.session.activity.start("export", "Exporting", Cancel::Flag(cancel.clone()));
guard.progress(0, total as u64);
// every frame (or from the worker, through `guard.handle()`)
guard.progress(done as u64, total as u64);
guard.detail(&file_name);
```
- **Keep the guard for exactly as long as the job runs.** Put it in the struct the UI polls (`ExportTask`, `ScanTask`,
`SyncRun`…), or move it into the worker thread. Dropping it removes the row, also when the worker panics, so a
dead job never leaves a stuck bar. A worker that only reports progress takes a `TaskHandle` (`guard.handle()`,
`Clone + Send`).
- **`kind`** is a stable camelCase id that agents filter by (`export`, `import`, `scan`, `sync`, `previews`,
`download`, `faces`…). **`label`** is a short English name in sentence case ("Building previews"). The stack shows it
translated, so add it to every catalog in `crates/ui-egui/locales/`.
- **Progress:** `progress(done, total)`. A `total` of 0 means the amount of work isn't known yet, and the bar sweeps.
`set_unit(Unit::Bytes)` makes the count read "12 of 340 MB"; `Unit::Percent` reads "40 %". The default `Unit::Count`
reads "3 of 25".
- **Detail:** what the job is on (a file name, a model's name). Detail set by engine code is a name, never a sentence.
UI code may set translated text.
- **Cancel:**
- `Cancel::Flag(Arc<AtomicBool>)` adopts the flag the job already checks. The row's ✕, `activity.cancel` and the
job's own Cancel then all set the same atomic.
- `Cancel::Yes`: the job polls `guard.is_cancelled()` itself.
- `Cancel::No`: the job can't stop; it gets no ✕ and doesn't make Quit ask.
- `set_cancellable(false)` for a phase that has to finish once started (a Lightroom import adding its photos).
- **Quiet periodic work** shows no row: the auto-import folder listing runs every few seconds through
`tasks::spawn(.., None, ..)`.
## Examples in the code
| Job | Where | Shows how to |
|---|---|---|
| Export, contact sheet | `crates/ui-egui/src/export_task.rs` | adopt the job's flag; update the row each frame |
| Build Previews | `crates/engine/src/cmd/previews.rs` | move the guard into the worker thread |
| Model downloads | `crates/engine/src/face_download.rs` | report progress through a `TaskHandle` in shared state |
| Lightroom catalog import | `crates/ui-egui/src/lightroom_import.rs` | stop being cancellable for a phase that must finish |
| Face scan, AI Denoise | `crates/engine/src/cmd/face_recognize.rs`, `crates/engine/src/denoise.rs` | a continuous job with no ✕, kept in session state |
| Find Missing Photos | `crates/ui-egui/src/tasks.rs` | a generic background task with a row (`kind: Some(..)`) |
## Testing
Assert the row (`session.activity.list()`: kind, label, total, `cancellable`), that `activity.cancel(id)` stops the
job, and that the row is gone once the job ends. `crates/ui-egui/src/tests_activity.rs` has the helpers, and the
job-specific tests show the pattern (`tests_sync.rs`, `export_task.rs`, `tests_denoise.rs`…). A row only appears
on screen once its task is 0.5 s old, so tests that click its ✕ wait for `activity:cancel:<id>` first.
docs/camera-preview-colour.md
+43
View File
@@ -99,6 +99,49 @@ Measured before → after (mean L* and CIE76 ΔE at 200 px): on the 24 ILCE-7CR
Picture styles other than Standard are lowered to the Standard curve too where it is darker; Lightroom ignores them as well. Nikon Active D-Lighting, Canon Auto Lighting Optimizer and Fujifilm D-Range settings are likely to have the same effect and are not handled yet. Render cache version 23 redraws thumbnails; smart previews built before keep the brightened tone curve in their header until rebuilt.
## Spectral camera matrices and the order of precedence
For 52 camera models LightCraft has colour matrices fitted to the camera's measured spectral sensitivities
(`crates/raw/src/spectral.rs`, generated into `spectral_table.rs` by `tools/spectral_camera_table.py` from
[rawtoaces-data](https://github.com/AcademySoftwareFoundation/rawtoaces-data) v1.1.0, Apache-2.0): 22 Canon, 11 Nikon,
11 Sony, 3 Fujifilm and 5 others, plus the other names rawtoaces-data lists for them (EOS Rebel T3i and Kiss X5 for
the EOS 600D; bodies it lists as sharing a sensor, such as the GFX100S with the GFX 100). Per model the generator fits,
under CIE Standard Illuminant A and under D65, a white-preserving 3×3 from white-balanced camera RGB to XYZ by the
least mean CIE76 error over rawtoaces-data's 190 training reflectances, and writes DNG `ColorMatrix1/2` and
`ForwardMatrix1/2`. The existing DNG colour model then interpolates them for the file's as-shot white, which the
vendor white-balance multipliers give through the same matrices. No number comes from Adobe or from a raw decoder.
A raw starts from the first of these that is available (`camera_preview::starting_colour`):
1. the file's own colour matrices (DNG);
2. a camera profile for its model (bundled or local), with tone and chroma fitted to the file's JPEG;
3. the file-local fit to its own JPEG described above;
4. the model's spectral matrices, with LightCraft's default tone curve and no JPEG fit;
5. the neutral fallback.
The spectral matrices only replace the neutral fallback: a photo whose JPEG fit is accepted (with or without a
profile) renders exactly as before, and a covered model takes its spectral matrices only when that fit fails the
gates or the file has no usable JPEG. White balance stays relative to the as-shot look for every raw without matrices
of its own, as before, so the catalog's `Photo::relative_wb` and the controls are unchanged. `lightcraft-cli
calibrate` still pools the uncorrected camera colour of covered models, so a profile can be fitted for them.
No colour-difference measurement of the spectral matrices against the camera JPEG, or against the file-local fit,
is recorded here yet. Whether they should come before the file-local fit for these models is open until one is.
## Smart previews and the look version
A smart preview keeps the result of the fit twice: the colour matrix is baked into its pixels and the tone and chroma curves are stored in its header. When the fit changes, a preview built earlier keeps the old look until it is rebuilt, and the render cache version does not reach it (renders are keyed on the library's files, proxies are files of their own).
The fit therefore has a version, `LOOK_VERSION` in `crates/engine/src/camera_preview.rs`, and every smart preview records the version that wrote it in its header line (`look_version`). **Bump the constant in any change that makes the fit give a different result for the same file**: the matrix, hue/saturation, tone or chroma fit, their gates and fallbacks, or how a camera profile feeds them. Changes that leave the fitted look alone don't need it. A header without the field (everything built before it existed) counts as version 0, and so does a value that is not a non-negative integer; a value from a newer build is left alone.
A proxy older than the constant is brought up to date by the existing Build Smart Previews path (`smart_run`), which also repairs damaged proxies:
- only proxies of raws that take the per-file look (`file_local_look`: ARW, NEF, RW2, RAF, CR3, CR2, PEF, SRW, ORF…) go stale; the fit never reaches JPEG, PNG, TIFF or DNG proxies, which are never rebuilt for it;
- when the library is opened and its smart previews folder has not been checked at this `LOOK_VERSION` (the `look-version` marker file in the folder), a background thread looks at those proxies, rebuilds the stale ones from their originals and then writes the marker, so the scan runs once per version bump, not at every opening (no proxy is created, nothing is shown, the UI never waits on a drive);
- Build Smart Previews does the same for the photos it is run on (`refreshed` in its result), whatever the marker says;
- when the original can't be read (offline drive), the proxy and its curve are kept exactly as they are (`staleKept`). The old version in its header is the mark: the next Build Smart Previews with the original online rebuilds it. A rebuilt proxy carries the current version, so nothing is rebuilt twice. A proxy is only ever rebuilt from the original, never from another proxy;
- the two never write the same proxy at once: each proxy is checked and written under one lock.
## Nikon crop and preview colour metadata
Nikon maker-note `CropArea` (0x0045) supplies the default `[left, top, width, height]` crop. The decoder validates the rectangle against the active sensor area and falls back to that area for missing or invalid values. The CFA origin is unchanged; cropping follows demosaicing.
docs/control-protocol.md
+3 -1
View File
@@ -30,7 +30,7 @@ mode ([mcp.md](mcp.md)) is a thin layer over this channel. Implementation:
| `engine.commands` | — | Engine + UI commands: id, label, menu, shortcut, params doc, enabled |
| `ui.menu.list` | — | Menu entries (flat: id, label, menu path, shortcut, enabled) |
| `ui.menu.tree` | — | The menu bar as shown (File … Help): items `{id, params?, label, shortcut?, enabled, checked?}`, separators, submenus — the model behind the native macOS menu bar and the in-window menus |
| `ui.inspect` | — | UI state, window, `loupe` (`source`, and `region: {full, window, pending}` when a zoomed view also renders the window on screen at no more than 100 % — `full` is the frame it is cut from, `window` its `[x, y, w, h]` — and `regionBefore` for the Before side of a Before/After view), canvas/image rects, `scroll: {grid, filmstrip}` (scroll offsets in points, `null` until drawn), active photo, selection, perf (`frameMs` = layout, `logicMs` = per-frame logic before it, `updateMs` = both, `maxUpdateMs`, `fps`, render queue …, `gpu` = adapter in use, `gpuReason` = why renders don't use the GPU, `gpuFallback` = latest render redone on the CPU and why — see `docs/gpu-pipeline.md`), status, memory (bytes per cache, see `library.memory`; plus stage caches — `budgetBytes`, `trimmed`, `sharedSourceBytes` (the one device copy of a big original) — and textures), `export: {running: {total, done, current} \| null, last}`, `notices` (warnings waiting to be shown, e.g. a damaged settings file; OK = `button:noticeOk`), `quitPrompt` (why quitting was stopped: unsaved changes; `button:quitRetry` / `button:quitAnyway` / `button:quitCancel`), `import: {done, total, imported, cancelled} \| null` (an import runs on a worker thread), `tasks` (other background work: `Find Missing Photos`, `Auto Import`), `fileDialogs` (commands waiting on a native file dialog, which runs off the UI thread; they run again with the answer when it closes), `copied` (the text the UI last put on the clipboard, `null` until something is copied) |
| `ui.inspect` | — | UI state, window, `loupe` (`source`, and `region: {full, window, pending}` when a zoomed view also renders the window on screen at no more than 100 % — `full` is the frame it is cut from, `window` its `[x, y, w, h]` — and `regionBefore` for the Before side of a Before/After view), canvas/image rects, `scroll: {grid, filmstrip}` (scroll offsets in points, `null` until drawn), active photo, selection, perf (`frameMs` = layout, `logicMs` = per-frame logic before it, `updateMs` = both, `maxUpdateMs`, `fps`, render queue …, `gpu` = adapter in use, `gpuReason` = why renders don't use the GPU, `gpuFallback` = latest render redone on the CPU and why — see `docs/gpu-pipeline.md`), status, memory (bytes per cache, see `library.memory`; plus stage caches — `budgetBytes`, `trimmed`, `sharedSourceBytes` (the one device copy of a big original) — and textures), `export: {running: {total, done, current} \| null, last}`, `notices` (warnings waiting to be shown, e.g. a damaged settings file; OK = `button:noticeOk`), `quitPrompt` (why quitting was stopped: tasks that would be cut short are running — `button:quitAnyway` cancels them and quits, `button:quitCancel` keeps working — or unsaved changes: `button:quitRetry` / `button:quitAnyway` / `button:quitCancel`), `import: {done, total, imported, cancelled} \| null` (an import runs on a worker thread), `tasks` (other background work: `Find Missing Photos`, `Auto Import`), `activity` (every long-running task in flight as the activity stack in the top-left corner shows it, same shape as `activity.list`; a row appears once its task is 0.5 s old, ids `activity:row:<id>`, ✕ = `activity:cancel:<id>`, `activity:more` expands past 3 rows), `fileDialogs` (commands waiting on a native file dialog, which runs off the UI thread; they run again with the answer when it closes), `copied` (the text the UI last put on the clipboard, `null` until something is copied) |
| `ui.widgets` | `{filter?}` | On-screen widgets `{id, rect: [x, y, w, h]}` (screen points) |
| `ui.clickWidget` / `ui.dragWidget` / `ui.hoverWidget` | `{id, count?, button?, fx?, fy?}` (`button: "right"` right-clicks) / `{id, toX?, toY?, dx?, dy?, steps?}` / `{id, fx?, fy?}` | Real egui input on a widget (hover: the pointer rests on it, e.g. for preset/profile previews) |
| `ui.move` / `ui.click` / `ui.drag` | `{x, y, count?, button?}` / `{x, y, toX, toY, steps?}` | Raw pointer input, screen points |
@@ -46,6 +46,8 @@ mode ([mcp.md](mcp.md)) is a thin layer over this channel. Implementation:
| `ui.zoomFactor` | `{factor}` | Set the interface scale (0.5–3; egui's zoom keys are off, Cmd+= / Cmd+- zoom the photo) |
| `ui.screenshot` | `{path?, headless?}` | `{path, width, height}` once the frame (with finished renders) is captured. `headless: true` draws the UI on the CPU (no compositor needed); a windowed capture that gets no frame within 2 s falls back to headless automatically |
| `engine.execute {command: "app.export", params}` | export params (see `docs/mcp.md`), plus `preset`, `dir` / `path`, `ids`, `background` | Writes the files and returns `{files}`; with `background: true` (what the Export dialog and menus use) it returns `{background: true, total}` at once and the batch runs on a worker thread — poll `ui.inspect` → `export` |
| `engine.execute {command: "activity.list"}` | — | The long-running tasks in flight (imports, exports, preview builds, downloads, the face scan…), oldest first: `{tasks: [{id, kind, label, done, total, unit, detail, cancellable, cancelling, ageMs}]}`; `total` 0 = not known yet; `unit` is `count`, `bytes` or `percent`; `label` is English (the app shows it translated) |
| `engine.execute {command: "activity.cancel"}` | `{id}` or `{all: true}` | Asks the task (or every cancellable one) to stop at its next file or photo → `{cancelled: n}`; an unknown id (it may have just finished) or a task that can't be cancelled is an error |
| `ui.render` | `{id?, size?, path?}` | Render a photo (PNG to `path`), `{width, height}` |
| `app.quit` | — | Close the app |
docs/faces.md
+1 -1
View File
@@ -248,7 +248,7 @@ Turning recognition on starts a scan of the whole library, once, in the backgrou
the running ones, so a worker that finishes has its next photo at once.
- **Earlier scan speed (a different runner).** A mixed raw and JPEG library of 184 photos (the raws read through their embedded previews, 73 photos searched
for faces) took about 29 seconds on that machine at full pace: roughly 6 photos a second, so the first scan of 10,000 photos
is a matter of half an hour; after that only new photos are looked at. Settings ▸ Faces shows how many are left, with a progress bar. After you accept a model's terms
is a matter of half an hour; after that only new photos are looked at. Settings ▸ Faces shows how many are left, with a progress bar, and the activity stack shows "Finding faces" meanwhile. After you accept a model's terms
you are returned to that tab, so the download and then the scan can be watched there; the main window has no status bar.
- Opening another library starts a fresh scan state; nothing learned about one library is used in another.
docs/file-format-parity.md
+79
View File
@@ -0,0 +1,79 @@
# File format parity
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (created; formats measured from the code and the target's Info.plist) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
Every format Lightroom reads or writes, with our support, fidelity and tests. Camera raw formats are
deep enough to have their own document: [raw-parity.md](raw-parity.md).
**How measured:** the target's document types come from the installed Lightroom 9.6
`Info.plist` (`CFBundleDocumentTypes`: catalog `mcat`, `xmp`, toolkit / module / library bundles, and
the raw-and-image list: jpg, tif, dng, heic and 27 raw extensions) plus Lightroom Classic's documented
import and export formats. Ours come from `crates/codecs/src/sniff.rs`, `crates/codecs/src/encode.rs`,
`crates/engine/src/export.rs`, `crates/engine/src/import.rs` and the tests named below.
## Images (non-raw)
| Format | Target reads | Target writes | LightCraft reads | LightCraft writes | Notes / tests |
|---|---|---|---|---|---|
| JPEG | ✅ | ✅ | ✅ (parallel decoder, ICC, EXIF orientation) | ✅ quality, size limit, ICC, metadata | `crates/codecs/src/jpeg.rs`, export tests |
| TIFF | ✅ 8/16/32-bit | ✅ 8/16-bit, LZW/ZIP | ✅ 1–64-bit integer, 16–64-bit float, bilevel, CCITT fax, WhiteIsZero | ✅ 8/16-bit, 32-bit float linear, none / LZW / ZIP | `crates/tiff`, `crates/codecs/src/tiff_codec.rs` |
| PNG | ✅ | ✅ | ✅ | ✅ | |
| PSD / PSB | ✅ (maximized-compatibility composite) | ✅ (Classic: PSD export, edit-in round trip) | ✅ merged composite | ⬜ (external-editor round trip writes 16-bit TIFF) | `crates/codecs/src/psd.rs` |
| HEIC / HEIF | ✅ | ⬜ (Classic) / HDR only | ✅ in every release build (`crates/heif`, heic-rs) | ⬜ | `heif` feature on by packaging scripts |
| AVIF | ✅ | ✅ (HDR) | ⬜ recognised, not decoded (no permissive pure-Rust AV1 decoder) | ✅ 8/10-bit, PQ HDR | |
| JPEG XL | ✅ | ✅ | ✅ (`jxl-oxide`) incl. DNG 1.7 JXL tiles | ⬜ | |
| WebP | not in its document types | ⬜ | ✅ | ✅ | beyond target |
| GIF, BMP | ⬜ | ⬜ | ✅ | ⬜ | beyond target |
| DNG (from raw) | ✅ | ✅ incl. lossy, embedded original | ✅ | ✅ uncompressed or LJ92; ⬜ lossy DNG, embedded original | `crates/raw/src/dngwrite.rs` |
| Render to DNG (15.5) | — | ✅ | — | ⬜ | new in Classic 15.5 |
| Original + sidecar | — | ✅ | — | ✅ | |
| HDR gain map JPEG (ISO 21496-1) | ✅ | ✅ | ✅ (gain map read) | ✅ | `crates/codecs/src/gainmap.rs` |
| PQ / HLG HDR | ✅ | ✅ AVIF, JXL, TIFF | — | 🟡 PQ AVIF, float TIFF; no HLG | |
| Contact sheet / print PDF | — | ✅ (Print module: PDF / JPEG / printer) | — | 🟡 paginated contact-sheet PDF (A4 / Letter, 150 dpi, sRGB) | `crates/engine/src/contact_sheet.rs` |
| Video (MP4, MOV, AVCHD…) | ✅ playback, trim, edit | ✅ export | 🟡 catalogued (kind, duration) | ⬜ | see video gap |
## Colour spaces and profiles
| Feature | Target | LightCraft |
|---|---|---|
| Export colour spaces | sRGB, Adobe RGB, Display P3, ProPhoto, Rec.2020, custom ICC | ✅ sRGB, Display P3, Adobe RGB-compatible, ProPhoto, Rec.2020 with gamut mapping (CPU and GPU), embedded ICC; ⬜ custom ICC output profiles; AVIF stays sRGB |
| Input ICC (JPEG / TIFF / PNG) | ✅ | ✅ matrix / TRC and LUT profiles (moxcms) |
| Soft proofing with printer profiles | ✅ | 🟡 |
## Metadata, sidecars, presets and catalogs
| Format | Target | LightCraft | Notes |
|---|---|---|---|
| XMP sidecars (`crs:` develop settings, ratings, keywords, regions) | read / write | ✅ read / write, manual (⌘S) and automatic | [xmp-interop.md](xmp-interop.md) |
| EXIF / IPTC / XMP embedded on export | ✅ | ✅ with metadata filters | `crates/meta` |
| Develop presets `.xmp` | ✅ | ✅ import (mapping of supported settings) | |
| Legacy presets `.lrtemplate` | ✅ (converted) | ✅ import | |
| Camera / creative profiles (`.dcp`, `.xmp` profiles) | ✅ | ⬜ deliberately (Adobe profile formats are never used); `.cube` LUT profiles ✅ | |
| Lens profiles `.lcp` | ✅ | ⬜ deliberately | |
| Other apps' presets | — | ✅ Luminar `.lmp` / `.mplumpack`, `.zip` bundles | beyond target |
| Lightroom Classic catalog `.lrcat` | native | 🟡 import (ratings, flags, labels, keywords, collections, virtual copies, supported develop settings; read-only pure-Rust SQLite incl. WAL) | [lightroom-catalog-import.md](lightroom-catalog-import.md) |
| Lightroom (cloud) library `.lrlibrary` | native | ⬜ | cloud library format, local cache only |
| Write back to `.lrcat` | native | ⬜ (one-way migration by design) | |
| Keyword lists (text) | import / export | ✅ | |
| GPX track logs | ✅ | ✅ | `crates/meta/src/gpx.rs` |
| Export presets, metadata presets, filename templates | ✅ (`.lrtemplate`) | ✅ own JSON | not interchangeable |
| Lightroom plug-ins (`.lrplugin`, `.lrdevplugin`) | ✅ | ⬜ | see ecosystem in [target-app-parity.md](target-app-parity.md) |
## Scores
| Part | Weight | Ready for real work | Kind |
|---|---:|---:|---|
| Standard image import / export | 35% | ~85% | estimated (table above; missing: AVIF read, JXL / PSD write, lossy DNG) |
| Sidecars, metadata, presets | 25% | ~85% | estimated |
| Lightroom catalog migration | 25% | ~50% | estimated (one-way, approximate rendering of unsupported settings) |
| HDR, print, video output | 15% | ~35% | estimated |
| **Non-raw file formats** | 100% | **~70%** | estimated; raw is in [raw-parity.md](raw-parity.md) (~50%) |
Remaining: **30–55 h** (AVIF decode 8–15 h if a permissive decoder is written or found; JXL encode
4–8 h; PSD write 4–8 h; lossy DNG and Render to DNG 4–8 h; HLG 3–5 h; catalog import depth 8–12 h).
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-10 | major | Created from the code and the installed Lightroom's document types |
docs/gaps.md
+77
View File
@@ -0,0 +1,77 @@
# Gaps: where LightCraft falls short of Lightroom
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (created; absorbs the checklist's Top gaps and the ROADMAP's honest assessment, re-ranked after a full re-measure) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
Every known shortfall, one entry each, ranked by user impact (how many Lightroom users hit it, and
how hard it blocks them), then by effort. **Agents pick work from the top** (the first entry nobody is
working on). When you close a gap, delete or shrink its entry, update its rows in
[parity-checklist.md](parity-checklist.md), and the numbers in [target-app-parity.md](target-app-parity.md)
and [ROADMAP.md](../ROADMAP.md) if they move.
Hours are Opus 5.5 agent wall-clock hours (see [target-app-parity.md](target-app-parity.md#how-hours-are-calibrated)).
`B` marks a gap that blocks **beta**.
## Ranked
| # | Gap | Evidence | User impact | Hours | Doc |
|---:|---|---|---|---:|---|
| 1 B | **Camera colour is estimated, not measured** (LR-PROF-CAMERACOLOR). Raws start from a per-file fit to the camera JPEG, 4 bundled profiles or 52 spectral matrices; others use a neutral matrix | ΔE vs camera JPEG 2.9–10 after fitting (raw-parity.md); 0 chart-calibrated models vs 1,500 Adobe Standard profiles | every raw shooter sees colour that is close but not Lightroom's; some get neutral-matrix colour | 50–90 | [raw-parity](raw-parity.md#colour) |
| 2 B | **CR3 beyond three verified bodies** (LR-IMP-FORMATS): CRX variants other than those verified on M50 / R100 / R8 fall back to the preview; CR3 sRAW / mRAW and HEIF CR3 too | measured: Canon is the largest bucket (33% of photographers) and its share whose camera fully works is the lowest of the big four, 72%; 17 of 22 popular Canon bodies write CR3, 2 verified (raw-parity.md) | +6.5 points of photographers whose camera works, the biggest single raw gain | 25–45 | [raw-parity](raw-parity.md#remaining-work-ranked-by-photographers-unlocked-per-hour) |
| 3 B | **Render fidelity unmeasured** (LR-BEHAV-RENDER-FIDELITY): tone, highlights, texture / clarity / dehaze, NR and sharpening are tuned by eye | no side-by-side suite; a retune needs a new process version (only V1 exists) | edits imported from Lightroom look different; the look "isn't Lightroom's" | 30–60 | [raw-parity](raw-parity.md), [process-versions](process-versions.md) |
| 4 B | **Original-pixel detail at 100%** (LR-VIEW-ZOOM, P0): the whole-image preview is capped (2,560 px default); compare, overlays and soft proofing lack sharp zoom windows (#323) | checklist row 🟡 | checking focus, the first thing culling needs, isn't reliable on large files | 8–15 | [ui-parity](ui-parity.md) |
| 5 B | **AI masks: Subject, Sky, People, Background, Landscape** are heuristics; Object / Describe run SAM 3 but its download has no mirror configured | `crates/segment`; ai-masks.md; MASK rows 67% | masking is how most Lightroom users edit today | 40–70 + model decision | [target-app-parity](target-app-parity.md#ai) |
| 6 B | **Compressed ORF** (every Olympus / OM body since ~2008) | measured: 66 of 78 Olympus / OM models preview-only; `orf.rs` refuses; OM bucket 3% of photographers, 3% of them fully work | +2.6 points; best gain per hour of any raw gap (~0.2) | 8–15 | [raw-parity](raw-parity.md) |
| 7 | **Nikon High Efficiency NEF** (HE / HE★ on Z 8, Z 9, Z 6III, Z f, Z50II, Z5II when set to HE) | named and preview only (#193); needs a pure-Rust JPEG XS decoder | +1.1 points (only shots taken in HE; lossless works on the same bodies); low gain per hour (~0.04), so not a beta blocker | 20–40 | [raw-parity](raw-parity.md) |
| 8 B | **No lens profile database** (LR-EDIT-OPTICS-PROFILE): only DNG opcodes, RW2 in-file distortion and Sony ILCE-7RM4A distortion | Adobe ships 3,635 lens profiles; we never use them | distortion / vignetting correction is a default step for many | 40–80 + calibration shots | [raw-parity](raw-parity.md#lens-profiles) |
| 9 B | **Tests don't run in CI on macOS, Windows or Linux x64**: only FreeBSD runs `cargo test`; the others build packages | `.github/workflows/` (freebsd, packaging-lint, release, windows-arm64) | regressions reach users on the main platforms; `cargo xtask ci` runs only locally | 6–12 | [target-app-parity](target-app-parity.md#by-dimension) |
| 10 B | **Per-model verification thin, plus cheap per-body decode fixes**: 234 of 1,446 Adobe models verified (16%), ~40% of photographers on a verified body; Nikon 5 bodies verified, Canon 11. Cheap fixes: Z 6II packed 14-bit NEF (#769, PR #671), Sony A100 / F828 regression (#702), DSLR-A200…A390 / A850 12-bit ARW (#535) | measured: [raw/coverage.tsv](raw/coverage.tsv) | per-body bugs (levels, crops, WB) surface only from users | 25–45 | [raw-parity](raw-parity.md#by-maker-measured) |
| 11 | **Enhance: Raw Details, Super Resolution; AI denoise without shippable weights or X-Trans** | ENH rows 0%; denoise Bayer only, RawNIND weights GPL and opt-in | Fujifilm users get no AI denoise; nobody gets it out of the box | 30–60 + model decision | [raw-parity](raw-parity.md#demosaic-levels-crops-and-lens-metadata) |
| 12 | **Content-aware / generative remove, detect distractions, reflections removal** | REM rows 75%; patch synthesis missing | cleanup work goes back to Photoshop | 25–45 (+ generative model) | [target-app-parity](target-app-parity.md#ai) |
| 13 | **Lightroom catalog migration is one-way and approximate**: unsupported Adobe profiles / AI settings archived, history kept as data | lightroom-catalog-import.md | switching users lose some looks | 8–12 | [file-format-parity](file-format-parity.md) |
| 14 | **HDR display** (EDR / Windows HDR); HLG; HDR settings from `crs` XMP | HDR rows 70%; LR-VIEW-HDR-DISPLAY ⬜ | HDR editing works blind on HDR monitors | 12–25 | [hardware-parity](hardware-parity.md) |
| 15 | **Print module**: custom packages, printer colour management (contact-sheet PDF only) | LRC-PRINT rows | Classic users who print | 25–40 | [target-app-parity](target-app-parity.md#features) |
| 16 | **Map view, Book, Slideshow module, publish services** | LRC-MAP / BOOK / SS rows ⬜ | Classic users; publish services matter for Flickr / SmugMug workflows | 60–100 | [target-app-parity](target-app-parity.md#features) |
| 17 | **Tethered capture** | ⬜ | studio photographers | 25–45 + cameras | [hardware-parity](hardware-parity.md) |
| 18 | **Video**: playback, trim, global edits, export | VID rows 0% | users who mix stills and clips | 25–45 | [target-app-parity](target-app-parity.md#features) |
| 19 | **Plug-in ecosystem**: no Lightroom SDK equivalent (export / publish plug-ins, metadata plug-ins) | ⬜ | pro workflows built on plug-ins (LR/Enfuse, Jeffrey's exporters, Negative Lab Pro) | 40–80 + owner decision | [target-app-parity](target-app-parity.md#by-dimension) |
| 20 | **Long-tail raw formats**: compressed SRW, ARQ pixel shift, IIQ, Hasselblad compression, X3F, CRW, MRW, ERF, KDC, DCR, MEF, GPR, old FinePix | raw-parity.md | small user share each | 45–85 | [raw-parity](raw-parity.md) |
| 21 | **Localization**: 9 of 15 shipped languages partial (74–99% of catalog messages), lower-layer errors English; Hindi, Arabic, Indonesian, Korean, Vietnamese absent; no RTL or complex shaping | measured from catalogs | non-English users see mixed English | 55–90 (shipped to full); 145–255 (five new) | [localization-parity](localization-parity.md) |
| 22 | **Pen pressure**, OS monitor profiles, follow-window colour | hardware-parity.md | brush users with tablets; multi-monitor setups | 11–22 | [hardware-parity](hardware-parity.md) |
| 23 | **Accessibility** audit with screen readers; canvas tools keyboard-reachable | LR-BEHAV-ACCESS 🟡 | blind and low-vision users | 10–20 | [ui-parity](ui-parity.md) |
| 24 | **Brush live preview and masking feel** (#517, #518); Feather / Edge mask refinement (15.5) | issues | masking feels slower than Lightroom | 8–12 | [ui-parity](ui-parity.md) |
| 25 | **File writes**: JXL encode, PSD export, lossy DNG, Render to DNG; AVIF import | file-format-parity.md | interchange edge cases | 25–45 | [file-format-parity](file-format-parity.md) |
| 26 | **Performance at scale**: 85k-photo library opens in 1.7–4.7 s; 100k grid test pending; colour NR at half resolution | ROADMAP log, gpu-pipeline.md | large-catalog users | 15–30 | [target-app-parity](target-app-parity.md#by-dimension) |
| 27 | **Windows / Linux runtime coverage**: Windows installer UI unverified on Windows (#79); GPU path proven on Apple only | ROADMAP, issues | non-Mac users find bugs first | 15–30 | [hardware-parity](hardware-parity.md) |
| 28 | **Small Classic shortcuts and keys**: rating `[` `]`, flag cycling, filter-bar keys, F1 help, panel keys (Tab, F5–F8) | [ui-parity](ui-parity.md#shortcuts) | Classic muscle memory | 4–8 | [ui-parity](ui-parity.md) |
Beta-blocking gaps (B) total **~230–430 h**; with the stability and platform work they need around them,
**~340–600 h** to beta (incl. 40–80 h for the ~92 open core-path user issues) (see [target-app-parity.md](target-app-parity.md#stage)).
## Gaps by kind
- **Feature:** 5, 8, 11, 12, 13, 15, 16, 18, 19
- **UI / UX:** 4, 23, 24, 28
- **File format:** 2, 6, 7, 10, 13, 20, 25 (raw ones in [raw-parity.md](raw-parity.md))
- **Hardware:** 14, 17, 22, 27
- **Localization:** 21
- **Quality / fidelity:** 1, 3
- **Stability and platforms:** 9, 26, 27
## Closed recently (kept for calibration)
- 2026-10-10: Nikon lossy-after-split NEF (#633), ORF 12.8 bpp (#651), HDR gain map / PQ AVIF / float
TIFF export (#646), activity stack (#345).
- 2026-10-09: HEIC in release builds (#609), Samsung uncompressed SRW (#459), contact-sheet PDF (#563),
Ukrainian UI (#564), Sony ILCE-7CR profile.
- 2026-10-08: CR3 CRX lossless and C-RAW (#279), Fujifilm compressed RAF (#282), AI RAW denoise (#384),
Lightroom catalog import (#218), German, Spanish, Russian UI.
- 2026-10-07: RW2 every format (#215), faces recognition in pure Rust (#174).
- 2026-10-05: compressed NEF (#86). The older history is in [ROADMAP.md](../ROADMAP.md#progress-log).
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-10 | minor | RAW gaps re-measured model by model (raw-parity.md): CR3, compressed ORF and verification entries carry measured shares; HE NEF no longer a beta blocker (+1.1 points for 20–40 h); verification entry absorbs the cheap per-body fixes (#769, #702, A200 series) |
| 2026-10-10 | minor | Beta total raised to 340–600 h to include the ~92 open core-path user issues |
| 2026-10-10 | major | Created from the checklist's Top gaps (whose items 1–2 had been duplicated by a merge) and ROADMAP's Where we stand; re-ranked; added evidence, impact, hours and beta flags; added platform-CI, lens-database, HE NEF and localization gaps |
docs/hardware-parity.md
+50
View File
@@ -0,0 +1,50 @@
# Hardware parity
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (created from the code, gpu-pipeline.md and the parity checklist) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
What Lightroom does with the machine's hardware, per platform, and what LightCraft does. Sources:
[gpu-pipeline.md](gpu-pipeline.md), [display-profiles.md](display-profiles.md), [denoise.md](denoise.md),
[ai-masks.md](ai-masks.md), `crates/gpu`, `crates/engine/src/devices.rs`, `crates/ui-egui/src/panels/second.rs`.
| Feature | Target | LightCraft: macOS | Windows | Linux / BSD | Web | Status |
|---|---|---|---|---|---|---|
| GPU for display and editing | Metal / DX12, "Use GPU for display / image processing / export" | wgpu Metal: geometry, resize, blur, WB, NR, guided filter, dehaze, masks, finish; CPU fallback per render | DX12 only (Vulkan opt-in, #136) | Vulkan | CPU only (no WebGPU yet) | 🟡 defringe, spot removal, heuristic masks, long brushes and high-bit exports stay on CPU |
| GPU export | ✅ | ✅ (6000 × 4000 in ~0.3 s vs ~1.2 s CPU) | ✅ | ✅ | — | ✅ (black-export fix on an Intel iGPU, #78) |
| AI on GPU / neural engine | Metal / DirectML / Core ML, model per feature | AI denoise on GPU or CPU (setting); SAM 3 on Metal via candle; faces on CPU | CPU / GPU denoise | CPU / GPU denoise | — | 🟡 no Core ML / Neural Engine / DirectML |
| HDR (EDR) display | ✅ (macOS EDR, Windows HDR) | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ (LR-VIEW-HDR-DISPLAY) |
| Monitor profile | automatic per display, follows the window | 🟡 picked by hand | 🟡 | 🟡 | — | 🟡 OS profile not read; no follow across monitors |
| Second window / second monitor | ✅ (Classic) | ✅ (⌘F11) | ✅ | ✅ | — | ✅ (Classic secondary-window keys missing) |
| Pen pressure / tilt (brush masks) | ✅ pressure for size / flow | ⬜ | ⬜ | ⬜ | ⬜ | ⬜ |
| Trackpad gestures | pinch zoom, pan | ✅ pinch, two-finger pan | 🟡 (winit) | 🟡 | 🟡 | ✅ on macOS |
| Card readers / cameras as import sources | ✅ (devices panel, eject after import) | ✅ mounted volumes with DCIM | ✅ | ✅ | — | 🟡 no eject-after-import check, no MTP / PTP phones |
| Tethered capture | ✅ Classic (Canon, Nikon, Sony, Leica) | ⬜ | ⬜ | ⬜ | — | ⬜ |
| Printers | ✅ Print module, printer colour management | ⬜ (PDF contact sheets only) | ⬜ | ⬜ | — | ⬜ |
| Control surfaces (MIDI / Loupedeck via plug-ins) | via SDK plug-ins | control channel and MCP can drive every command | same | same | — | 🟡 different mechanism; no MIDI mapping |
| Multi-core CPU | ✅ | ✅ worker pools, parallel JPEG decode, parallel import copy | ✅ | ✅ | single thread | ✅ |
| Memory pressure | managed | ✅ memory budget, `crates/sysmem` returns allocator pages on macOS | ✅ budget | ✅ budget | — | ✅ |
## Score
**~40% ready** (estimated): the GPU editing path is real and fast (measured: 24 MP exposure drag
3.9 ms on GPU vs 22–42 ms CPU; NR drag 18 ms vs 301–471 ms), but HDR display, pen pressure,
tethering, printing, automatic monitor profiles and neural-engine inference are missing. GPU
correctness is proven on Apple GPUs; other vendors rely on user reports.
## Remaining (Opus 5.5 agent hours)
| Work | Hours | Needs |
|---|---:|---|
| HDR display (EDR on macOS, Windows HDR, Linux where possible) | 10–20 | an HDR display to verify (human) |
| OS monitor profile, follow window across displays | 5–10 | multi-monitor setup |
| Pen pressure for brush masks (winit tablet events or a pure-Rust tablet layer) | 6–12 | a tablet |
| Remaining stages on GPU (defringe, spot removal, masks) | 10–20 | |
| Tethered capture (PTP over USB in pure Rust, Canon / Nikon / Sony first) | 25–45 | cameras (human) |
| Printing with colour management | 15–25 | printers (human) |
| Neural-engine / DirectML inference for AI features | 10–20 | depends on the AI model decision |
| **Total** | **80–150 h** (the core set for beta, HDR display + monitor profile + pen pressure: 20–40 h) | |
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-10 | major | Created |
docs/localization-parity.md
+92
View File
@@ -0,0 +1,92 @@
# Localization parity
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (first per-language measurement from the catalogs) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
How far each interface language is from `full`. How to add or maintain a language is in
[localization.md](localization.md); per-language notes are in `localization-<code>.md`
([zh-hans](localization-zh-hans.md), [zh-hant](localization-zh-hant.md), [ja](localization-ja.md),
[pt-br](localization-pt-br.md), [es](localization-es.md), [de](localization-de.md),
[ru](localization-ru.md)).
## The target
Lightroom 9.6 (installed on the owner's Mac) ships 16 interface languages, listed from its bundle's
`.lproj` folder names: English, German, Spanish, French, Italian, Japanese, Korean, Norwegian Bokmål,
Dutch, Polish, Portuguese (Brazil), Russian, Swedish, Thai, Simplified Chinese and Traditional
Chinese. It does **not** ship Hindi, Arabic, Indonesian or Vietnamese, so for those LightCraft would
go beyond the target.
## How the numbers are measured
**Measured** (a Python count over `crates/ui-egui/locales/*.json`, 2026-10-10):
- The denominator is every message any catalog knows: the union of the plain catalogs' keys (2,074
English source strings: menus, commands, controls, panels, dialogs, tooltips, presets, What's New
lines) plus the 230 messages of the formats catalogs (`*-formats.json`: text with runtime values and
date patterns), 2,304 messages in all.
- A message counts as translated when the catalog has a non-empty value that differs from the English
key. Some identical values are correct (`ISO`, `OK` in German), so the figure slightly understates
coverage; on the other hand, strings that never reach a catalog (technical errors raised in lower
crates, some `format!` text outside `tr_format!`) are not in the denominator at all. That is why no
language is `full` even at 99%.
- The UI has 563 distinct `tr("…")` literals and 198 `tr_format!` messages in `crates/ui-egui` and
`apps/`; command, control, menu and preset labels reach the catalogs through `display_label`.
Script support is checked against the code: `language_table!` in `crates/ui-egui/src/i18n.rs` lists
scripts `Latn`, `Cyrl`, `Hans`, `Hant`, `Jpan` only. The UI is egui, which has no bidirectional layout
and no complex-script shaping, so Arabic (RTL, joining) and Devanagari (conjuncts, reordering) cannot
render correctly today.
Native-speaker review: no catalog has had a formal review. Several catalogs have edits from community
contributors whose commits only touch that language (ja, zh-hans, zh-hant, fr, de, ru, uk, pt-br);
that is promising but is not recorded as review, so the column says `no (community edits)`.
## The twelve key languages
| # | Language | Code | UI strings translated | Dialogs / tooltips / help | Script support | Native review | Status | To `full` |
|---|---|---|---:|---|---|---|---|---:|
| 1 | English | `en` | 2,304 / 2,304 (100%, source) | all; Help ▸ What's New and the shortcut sheet; no full user manual | Latin (Inter) | n/a | **full** | 0 h |
| 2 | Simplified Chinese | `zh-hans` | 1,994 / 2,304 (87%) | menus, panels, dialogs, presets, profiles, dates; lower-layer errors English | Hans via craft-fonts (Noto Sans CJK SC); IME through winit; no vertical text in UI (not in target either) | no (community edits) | partial | 6–10 h |
| 3 | Spanish | `es` | 1,972 / 2,304 (86%) | as zh-hans | Latin | no (community edits) | partial | 6–10 h |
| 4 | Hindi | `hi` | 0 (0%) | none | **Devanagari shaping missing** (egui has no complex shaping) | no | none (target: not shipped) | 50–90 h (shaping 25–50 h, shared with Arabic) |
| 5 | Arabic | `ar` | 0 (0%) | none | **RTL layout and Arabic shaping missing** | no | none (target: not shipped) | 60–110 h (RTL mirroring 30–50 h + shaping, shared with Hindi) |
| 6 | French | `fr` | 2,021 / 2,304 (88%) | as zh-hans | Latin | no (community edits) | partial | 5–9 h |
| 7 | Portuguese (Brazil) | `pt-br` | 1,966 / 2,304 (85%) | as zh-hans | Latin | no (community edits) | partial | 6–10 h |
| 8 | Indonesian | `id` | 0 (0%) | none | Latin (Inter covers it) | no | none (target: not shipped) | 10–16 h |
| 9 | Japanese | `ja` | 1,995 / 2,304 (87%) | as zh-hans; vertical export watermarks | Jpan via craft-fonts (BIZ UDPGothic / UDMincho); IME through winit | no (community edits) | partial | 6–10 h |
| 10 | German | `de` | 2,104 / 2,304 (91%) | as zh-hans; What's New translated line by line | Latin | no (community edits) | partial | 4–8 h |
| 11 | Korean | `ko` | 0 (0%) | none | **no Korean (Hangul) face**: needs a non-Adobe OFL face in craft-fonts (Noto CJK is allowed only for Chinese) | no | none | 14–22 h |
| 12 | Vietnamese | `vi` | 0 (0%) | none | Latin with stacked diacritics (Inter covers it; check combining marks) | no | none (target: not shipped) | 10–16 h |
## Other languages LightCraft ships (3)
| Language | Code | UI strings translated | Script | Native review | Status | To `full` |
|---|---|---:|---|---|---|---:|
| Traditional Chinese (Taiwan) | `zh-hant` | 1,993 / 2,304 (87%) | Hant; uses the Simplified Chinese face until craft-fonts has a TC face | no (community edits) | partial | 8–14 h (incl. the font) |
| Russian | `ru` | 1,704 / 2,304 (74%) | Cyrillic (Inter) | no (community edits) | partial | 8–12 h |
| Ukrainian | `uk` | 2,287 / 2,304 (99%) | Cyrillic (Inter) | no (community edits) | partial (lower-layer errors English) | 2–4 h |
Target languages LightCraft lacks: Italian, Korean, Dutch, Polish, Swedish, Norwegian Bokmål, Thai
(Thai needs shaping and line breaking without spaces). Each Latin-script one is ~10–16 h; Thai
~20–35 h on top of the shaping work.
## Shared work before any language is `full` (counted once, ~20–35 h)
- Route lower-layer error messages (catalog, raw, codecs, engine) through translatable message ids
instead of English `format!` strings (~12–20 h).
- A check in `cargo xtask ci` that every user-visible literal reaches a catalog (~4–8 h).
- A user manual / help pages beyond What's New and the shortcut sheet (Lightroom links to online
help; ~4–8 h for a translatable in-app help index).
- Native-speaker review passes (human; cannot be done by agents).
## Totals
- Bring the 9 shipped non-English catalogs to `full`: ~55–90 h (incl. the shared work).
- Add the 5 missing key languages: ~145–255 h, most of it RTL and complex shaping in egui.
- Reach the target's 16 languages: + ~60–100 h (Italian, Dutch, Polish, Swedish, Norwegian, Thai).
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-10 | major | Created: per-language counts measured from the catalogs, the twelve key languages, script support, estimates |
docs/merge.md
+1 -1
View File
@@ -9,7 +9,7 @@ All algorithms are our own implementations from published papers (cited in the m
- **App:** select the photos, then *Photo Merge ▸ HDR…* (`Ctrl+H`), *Panorama…* (`Ctrl+M`) or
*HDR Panorama…* from the photo context menu (or the `dialog.merge*` commands). The dialog previews
the merge at ≤ 1024 px and re-runs the preview when an option changes; *Merge* runs the full merge
in the background (progress in a toast). The result is written next to the first photo as
in the background (a row in the activity stack shows its progress, with ✕ to stop it). The result is written next to the first photo as
`<name>-HDR.dng`, `-Pano.dng` or `-HDR-Pano.dng`, imported and selected.
- **Commands** (CLI, MCP, control channel):
- `merge.hdr {ids?, align=true, deghost=none|low|medium|high, autoSettings=true, stack=false, preview=false, showOverlay=false, previewPath?}`
docs/parity.md → docs/parity-checklist.md
+35 -92
View File
File diff suppressed because one or more lines are too long
docs/process-versions.md
+1 -1
View File
@@ -32,7 +32,7 @@ Today there is one process, **V1**: LightCraft's rendering from before process v
passing: settings saved without the field must stay V1, not become the new latest. That is the
field's `#[serde(default = "ProcessVersion::legacy")]`; the struct-level `Default` gives new
settings the latest instead.
4. Render a few procedural scenes on both processes and compare, and update `docs/parity.md`.
4. Render a few procedural scenes on both processes and compare, and update `docs/parity-checklist.md`.
## What gets which process
docs/raw-parity.md
+253
View File
@@ -0,0 +1,253 @@
# RAW parity: cameras, formats, demosaic, colour and lens profiles
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (model-by-model measurement against Adobe's published camera list; use-weighted coverage; variant matrix; colour and lens sections; ranked next cameras) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6 (Camera Raw 18.7)
Raw processing is the deepest single dimension of Lightroom parity: a photographer whose camera
does not decode, or decodes with the wrong colour, cannot switch. This document measures it model by
model. The numbers feed [target-app-parity.md](target-app-parity.md); individual shortfalls are
ranked in [gaps.md](gaps.md). Details per format: [cr3.md](cr3.md),
[raf-compression.md](raf-compression.md), [sony-lens-corrections.md](sony-lens-corrections.md),
[camera-preview-colour.md](camera-preview-colour.md), [denoise.md](denoise.md).
## Headline
| Measure | Value | Kind |
|---|---:|---|
| Adobe-listed camera models | **1,446** | measured: [raw/adobe-cameras.tsv](raw/adobe-cameras.tsv) |
| Decoded from sensor data and **verified** on a real file | **234** (16%) | measured: [raw/evidence.tsv](raw/evidence.tsv) |
| Decoder path should handle it, **unverified** | **943** (65%) | measured from the format rules (632 of them DNG, mostly phones) |
| **Preview-only** (embedded JPEG) | **233** (16%) | measured |
| **Unsupported** (not imported, refused or decodes wrongly) | **36** (2%) | measured |
| Models added since 2017 (Camera Raw 10+): verified / unverified / preview / unsupported | 81 / 641 / 24 / 8 of 754 | measured |
| **Share of photographers whose camera fully works** (decodes from sensor data), use-weighted | **~84%** | estimated: measured classes × written usage weights × per-format confidence |
| …of which on a body we have verified | **~40%** | estimated, same weights |
| Camera colour: Adobe profiles vs ours | 1,500 Adobe Standard profiles, 481 models with Camera Matching profiles vs **4 bundled profiles** + per-file fits + 52 spectral matrices | measured (bundle listing, Adobe list, `assets/camera-profiles`) |
| Lens profiles: Adobe vs ours | **3,635 vs 0** (in-file corrections only) | measured |
| **RAW readiness** (decoding 60%, colour 25%, lens 15%) | **~56%** (range 50–60%) | estimated, see [Scores](#scores) |
Re-measure with `python3 docs/raw/coverage.py` (standard library only): it classifies every Adobe
model from the format rules in the script (which mirror `crates/raw/src`) plus the per-model evidence,
rewrites [raw/coverage.tsv](raw/coverage.tsv) and prints the tables below; `--check` fails when the
committed table is stale.
## How it was measured
- **Adobe's list**: helpx "Camera Raw supported cameras" (the live page refuses scripted requests, so
the Wayback Machine capture of 2026-10-03 was parsed): 1,446 camera rows from 40 makers after
dropping profile-only rows; maker, model, extension(s), Camera Matching profile, minimum Camera Raw
version. The year a model was added is approximated from that Camera Raw major version.
- **What we decode**: the decoders and their refusals in `crates/raw/src` (`lib.rs` routing, `vendor/*.rs`
coding variants, `crx.rs` CRX limits, `orf.rs` / `srw.rs` compressed refusals), the import list
`lightcraft_engine::import::EXTENSIONS`, and `KNOWN_UNSUPPORTED` in `crates/raw/tests/corpus.rs`. Doc
claims were not used.
- **Verified** means a real file from that body decoded from sensor data in: the corpus test over the
`RAW_SAMPLES` manifest (`xtask/src/main.rs`, 101 CC0 files; bodies whose sample is in
`KNOWN_UNSUPPORTED` excluded), `crates/raw/tests/cr3_corpus.rs` (3 bodies, exact), the RAF reference
check ([raf-compression.md](raf-compression.md)), and the raw.pixls.us bulk checks recorded in issue
#535 (all 100 Sony bodies there; 87 decode) and PR #215 (every Panasonic / Leica RW2 / RWL / RAW body
there, 178 files from 118 bodies). Bulk-check body lists were matched to Adobe names through the
raw.pixls.us folder names. Checks without a body list (≈ 60 NEFs, 35 CR2 bodies for the CFA layout)
are not counted as verified.
- **Usage weights** ([raw/usage-weights.tsv](raw/usage-weights.tsv)): maker buckets from 2025 market
data (Nikkei production: Canon ~36%, Sony ~29%, Nikon ~14%, Fujifilm ~12% of mirrorless; BCN retail:
Sony 29.9%, Canon 27.4%, Nikon 15.1%), adjusted for the DSLR installed base and for the makers named
in this repo's 325 user issues (Sony 33, Nikon 31, Canon 26, phones 23, Fujifilm 9, Panasonic / Leica
8, OM / Olympus 6, Pentax / Ricoh 6). Inside each bucket, 80% goes to the popular bodies of the last
~8 years (listed in the file), 20% to the bucket's other Adobe models.
- **Per-format confidence** for unverified bodies (the chance their files decode correctly): DNG, CR2,
RAF, RW2 0.95; NEF, ARW, PEF 0.9; ORF, SRW, SR2 0.8; **CR3 0.6** (the CRX decoder refuses several
coding variants and only three bodies are verified; maintainers report the R7 working).
## By maker (measured)
| Maker | Adobe models | verified | unverified | preview | unsupported | Main reason for preview / unsupported |
|---|---:|---:|---:|---:|---:|---|
| Sony | 149 | 86 | 51 | 9 | 3 | DSLR-A200…A390 and A850 12-bit compressed ARW (preview, #535); A100 and DSC-F828 refused (regression #702); SRF not imported |
| Canon | 134 | 11 | 100 | 21 | 2 | CRW (2000–2004 bodies) preview; EOS-1D / 1Ds TIFF raws not imported |
| Samsung | 128 | 1 | 118 | 9 | 0 | compressed SRW (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini, Galaxy NX) |
| Apple | 113 | 1 | 112 | 0 | 0 | — (DNG / ProRAW) |
| Panasonic | 104 | 93 | 11 | 0 | 0 | — |
| Nikon | 99 | 5 | 94 | 0 | 0 | — (HE / HE★ is a per-shot setting, preview-only) |
| Fujifilm | 97 | 16 | 55 | 26 | 0 | pre-X FinePix RAF without a raw IFD (estimated split) |
| Google | 88 | 1 | 87 | 0 | 0 | — |
| Olympus | 71 | 3 | 9 | 59 | 0 | **compressed ORF** |
| Leica | 61 | 13 | 48 | 0 | 0 | — |
| Xiaomi, Oppo, LG, Huawei, OnePlus, Motorola, Nokia | 139 | 0 | 139 | 0 | 0 | — (DNG) |
| Pentax / Ricoh | 60 | 4 | 56 | 0 | 0 | — |
| Hasselblad | 34 | 0 | 2 | 32 | 0 | Hasselblad-compressed 3FR / FFF |
| Phase One / Leaf / Mamiya | 59 | 0 | 0 | 43 | 16 | IIQ, MOS preview; TIFF raws and MEF not imported |
| Casio | 26 | 0 | 26 | 0 | 0 | — (DNG) |
| DJI | 21 | 0 | 21 | 0 | 0 | — (DNG) |
| OM Digital Solutions | 7 | 0 | 0 | 7 | 0 | **compressed ORF** |
| Kodak, Konica Minolta, Epson, Sigma, GoPro, Contax, others | 56 | 0 | 14 | 27 | 15 | DCR, GPR not imported; KDC, MRW, ERF, X3F preview |
| **All** | **1,446** | **234 (16%)** | **943 (65%)** | **233 (16%)** | **36 (2%)** | |
## By use (estimated)
| Bucket | Weight | Camera fully works | On a verified body | Why |
|---|---:|---:|---:|---|
| Canon | 33% | 72% | 16% | 17 of the 22 popular bodies write CR3; 2 of them verified, the rest at 0.6 |
| Sony | 27% | 98% | 91% | bulk-checked |
| Nikon | 16% | 91% | 12% | decoders cover every NEF coding but HE; few bodies verified |
| Fujifilm | 10% | 91% | 45% | |
| Panasonic | 4% | 100% | 98% | bulk-checked |
| OM / Olympus | 3% | 3% | 1% | compressed ORF |
| Phones | 3% | 93% | 1% | DNG |
| Pentax / Ricoh | 1.5% | 94% | 14% | |
| Leica | 1% | 95% | 5% | |
| DJI | 1% | 95% | 0% | |
| Medium format and others | 0.5% | 23% | 0% | IIQ, 3FR, X3F |
| **Weighted** | 100% | **84%** | **40%** | |
About 5 in 6 raw photographers' cameras decode, but only about 2 in 5 use a body we have actually
checked, and Canon, the largest bucket, is the weakest of the big four.
## Variant matrix (depth per format)
✅ decoded and checked on real files · 🟡 decoded, partial or unverified · ⬜ preview-only or refused.
| Format | Variant | Status | Evidence |
|---|---|---|---|
| CR2 | lossless JPEG slices | ✅ | 8 corpus bodies; CFA phase from `CR2CFAPattern` (#85, 35 bodies); masked-column black level |
| CR2 | sRAW / mRAW (YCbCr) | ✅ | `vendor/cr2/sraw.rs`, 31 samples from 16 models |
| CR3 | CRX lossless, version 0x100 | ✅ | M50, R100, R8 exact against a reference |
| CR3 | C-RAW 0x100 (horizontal tiles), 0x200 (single 14-bit tile, adaptive QP) | ✅ | same 3 bodies exact |
| CR3 | other CRX variants (vertical tiles, partial subbands, other plane flags, 0x200 multi-tile) | ⬜ | refused in `crx.rs`, falls back to the preview; which bodies write them is unmeasured |
| CR3 | sRAW / mRAW | ⬜ | 4-plane Bayer only |
| CR3 | HEIF / HDR PQ | ⬜ | HEVC tracks recognised, not decoded |
| CR3 | dual-pixel raw | ⬜ | never substituted |
| CRW | all | ⬜ | preview |
| NEF | uncompressed, lossless, lossy type 1 / 2, lossy after split | ✅ | D5100, D7000, D7500, Z f corpus; ~60 NEFs by hand; #633 |
| NEF | packed 14-bit stored under a compressed tag (Z 6II uncompressed) | ⬜ | #769; fix in PR #671 (not merged) |
| NEF | High Efficiency / HE★ (JPEG XS) | ⬜ | named, preview (#193); needs a JPEG XS decoder |
| ARW | uncompressed, cRAW (ARW2), lossless LJ92 (L / M / S) | ✅ | 86 bodies (#535); M/S black level 1024 |
| ARW | packed 12-bit (A900) | ✅ | one CC0 file; A850 unmeasured |
| ARW | 12-bit compressed of DSLR-A200…A390 | ⬜ | preview (#535) |
| ARW | A100 ARW; SRF (F828, V3) | ⬜ | refused (#702); SRF not imported |
| ARW | ARQ pixel shift | ⬜ | no code |
| SR2 | DSC-R1 | ✅ | CC0 3221 sample |
| RAF | uncompressed, lossless and lossy compressed; Bayer and X-Trans; GFX 14/16-bit | ✅ | 24 files from 13 bodies exact |
| RAF | pre-X FinePix without a raw IFD | ⬜ | `raf.rs` refuses (26 Adobe models, estimated split) |
| ORF | uncompressed 16-bit, 12-bit packed, two-field 12-bit, 12.8 bpp | ✅ | E-1, E-400, XZ-2 corpus; #651 |
| ORF | **compressed** (every Olympus / OM body since ~2008) | ⬜ | `orf.rs` refuses; the E-M1 and E-M10 III corpus files are `KNOWN_UNSUPPORTED` |
| RW2 / RWL / RAW | formats 2, 4, 5, 6, 7, 8 and 16-bit words | ✅ | 118 bodies (#215) |
| PEF | uncompressed, Huffman, packed 12-bit | ✅ | K10D, K-3, K-5 IIs |
| PEF / DNG | Pentax pixel shift | ⬜ | frames not combined |
| SRW | uncompressed, 12-bit packed | ✅ | EX1 corpus; NX10, NX20 |
| SRW | compressed 32770 / 32772 / 32773 | ⬜ | refused |
| DNG | uncompressed, LJ92, lossy JPEG / Smart Preview, Deflate incl. float, LinearRaw, tiles / strips, row / column interleave | ✅ | 6 corpus DNGs; 4 ILCE-6400 JXL DNGs |
| DNG | JPEG XL (DNG 1.7, lossless and lossy) | ✅ | synthetic bit-exact; real lossless and lossy files |
| DNG | Apple ProRAW (incl. semantic mattes) | ✅ | iPhone 12 Pro corpus; ProfileGainTableMap read but off (ΔE00 19 against Lightroom with it on) |
| DNG | phone DNGs (Pixel, Galaxy, Xiaomi…) | 🟡 | Pixel 2 XL only; white-level pile-up fix #548 |
| DNG | GoPro GPR (VC-5) | ⬜ | refused |
| 3FR / FFF | DNG-like layouts | 🟡 | decoded when uncompressed or LJ92 |
| 3FR / FFF | Hasselblad compression | ⬜ | preview |
| IIQ, MOS, MRW, ERF, KDC, X3F | all | ⬜ | preview |
| DCR, MEF, MFW, DXO, GPR, TIFF raws | all | ⬜ | not imported |
## Demosaic, levels, crops and lens metadata
| Stage | Target | LightCraft | Status |
|---|---|---|---|
| Bayer demosaic | Adobe's own, plus AI "Raw Details" | AHD (default), PPG, bilinear (`crates/raw/src/demosaic`) | 🟡 quality unmeasured against Lightroom; no Raw Details |
| X-Trans demosaic | ✅ | in-house edge-weighted directional | 🟡 unmeasured |
| Foveon | ✅ (older bodies) | — | ⬜ |
| Monochrome / LinearRaw | ✅ | handled, no demosaic | ✅ |
| Pixel shift (Sony ARQ, Pentax, OM high-res, Nikon) | ✅ (Sony, Pentax, Nikon) | — | ⬜ |
| Black and white levels | per model, calibrated | read from each file (masked columns, maker notes, SR2SubIFD, CMP1) | 🟡 per-body bugs keep surfacing: A450–A700 SR2 black level and M/S ARW pedestal (#535, fixed); 5D Mark IV CR2 renders dark (#455, open); white-level pile-up in phone DNGs (#548, fixed) |
| Crop masks and aspect | ✅ | DefaultCrop, Sony recorded size, Canon AspectInfo, Panasonic in-camera crops, Nikon CropArea | 🟡 7RM3 / 7RM4 crop origin 8–32 px off (cosmetic) |
| Lens metadata | ✅ read for profile selection | EXIF lens fields read; used only for the in-file corrections below | 🟡 |
| Highlight reconstruction | ✅ | neutral clip, chromaticity diffusion (`highlight.rs`) | 🟡 tuned by eye |
| AI denoise | ✅ Bayer and X-Trans, model bundled | Bayer only, user-installed weights, pure-Rust CPU / GPU | 🟡 no X-Trans, no shippable weights |
| Super Resolution | ✅ | — | ⬜ |
| HDR / panorama merge to raw DNG | ✅ | linear DNG (float16 HDR) with the source's colour tags | ✅ |
| Convert to DNG | ✅ | uncompressed or LJ92 tiles (`dngwrite.rs`) | ✅ (no lossy DNG) |
## Colour
| Item | Adobe | LightCraft |
|---|---|---|
| Default profile per model | Adobe Standard for every model (1,500 profile names in the bundle) | the DNG's own matrices; else a pooled profile (**4 bundled**: ILCE-7CR, ILCE-7M4, X-H2S, X-T4); else a guarded per-file fit to the camera's embedded JPEG; else spectral matrices (**52 models**, rawtoaces-data); else a neutral matrix |
| Camera Matching looks | 481 models (plus phone-maker looks) | none (the per-file JPEG fit imitates the camera's own look) |
| Creative profiles | Adobe Color, Portrait, Landscape, Vivid, Monochrome… | our own looks (Color, Neutral, Vivid, Landscape, Portrait, Monochrome) |
| DCP / dual-illuminant model | ✅ | dual-illuminant DNG colour (ColorMatrix / ForwardMatrix 1-2, CameraCalibration, AnalogBalance), HueSatMap, LookTable, ProfileToneCurve; external `.dcp` never loaded (clean-room) |
| White balance in Kelvin | ✅ for every model | absolute Kelvin only where matrices exist; per-file fits give relative WB (#296, #730) |
**How far the default rendering is from Adobe's** (CIE ΔE; measured where stated):
- Against the camera JPEG (what the fit targets): the Sony bulk check (#535) accepted fits on 81 of 87
bodies, median ΔE 5.1, ΔL\* +0.2; rejected fits fall back to a neutral matrix at median ΔE 18.5,
ΔL\* −11, chroma ×0.39. RW2 140 of 174 files accepted, median 5.0. NEF 13 of 13, 2.9–7.8. RAF 23 of
24, pooled X-H2S 1.62 / X-T4 2.34. SRW 2.2–7.6.
- Against Lightroom itself (few): Sony DRO median ΔE76 7.5 → 3.3; Olympus E-1 vs Lightroom Classic
ΔE00 11.25 → 8.30; ProRAW with its gain map ΔE00 19 (why the map is off). No measurement against
Adobe Standard across models exists.
- User evidence: 33 open issues about raw colour or rendering (wrong colours, dark, flat, green,
desaturated), the largest group of open core-path issues.
**Colour score ~35%** (estimated): most decoded files start close to the camera's own look (ΔE ~5),
about 10% start far off (ΔE ~18), nothing is calibrated, Kelvin WB is missing for fitted cameras, and
the distance to Adobe Standard is unmeasured. In the readiness numbers this becomes a colour factor
of ×0.85 for a working photographer (they correct colour by hand, at a cost) and ×0.75 for the
essentials user (the default look is what they keep).
## Lens profiles
| Item | Adobe | LightCraft |
|---|---|---|
| Profile database | 3,635 profile files, 56 makers, auto-selected | **0** (Adobe LCPs never used; no database of our own) |
| In-file corrections | ✅ | DNG opcodes (WarpRectilinear, FixVignetteRadial, GainMap…) applied; Panasonic / Leica RW2 distortion (79 photos, 13 bodies, 11 lenses); Sony tag 0x7037 distortion for the ILCE-7RM4A (2 lenses) |
| Vignetting / lateral CA from maker notes | ✅ | ⬜ |
| Manual distortion, vignetting, CA, defringe | ✅ | ✅ |
**Lens score ~10%** (estimated): automatic correction reaches roughly Panasonic / Leica, phone and
drone DNG users (~9% of photographers by the weights above); Adobe reaches nearly everyone.
## Scores
| Part | Weight | Score | Basis |
|---|---:|---:|---|
| Decoding | 60% | **76%** | use-weighted 84% × 0.9 for per-body decode bugs still surfacing (levels, crops, WB: #535, #455, #461, #702) |
| Colour | 25% | **35%** | see [Colour](#colour) |
| Lens profiles | 15% | **10%** | see [Lens profiles](#lens-profiles) |
| **RAW readiness** | 100% | **~56%** (range 50–60%) | 45.6 + 8.75 + 1.5; demosaic and highlight quality unmeasured |
The previous estimate (earlier on 2026-10-10) was ~50%, with decoding at ~78% weighted by maker
from the format list alone. Measured model by model, decoding comes out about the same; the change is
the written decomposition, not new decoding.
## Remaining work, ranked by photographers unlocked per hour
Hours are Opus 5.5 agent hours, calibrated on this repo's raw PRs: RW2 every format 6.5 h (#215),
compressed RAF 8.6 h (#282), CR3 CRX 10 h (#279), NEF lossy after split 4.7 h (#633), ORF 12.8 bpp
1.7 h (#651), HEIC 7 h (#609). "Gain" is the change in the use-weighted share whose camera fully
works, from [raw/usage-weights.tsv](raw/usage-weights.tsv).
| # | Camera / variant | Gain (points of photographers) | Hours | Gain per hour | Needs |
|---:|---|---:|---:|---:|---|
| 1 | Compressed ORF (every Olympus / OM body since ~2008) | +2.6 | 8–15 | ~0.20 | CC0 samples (have 2) |
| 2 | CR3: remaining CRX variants, then verify the 17 popular CR3 bodies | +6.5 | 25–45 | ~0.19 | samples per body (raw.pixls.us has most) |
| 3 | Verify popular Nikon, Fujifilm, Canon CR2 and Pentax bodies (one CC0 file each, fix what breaks) | +2.0 | 8–15 | ~0.17 | public samples |
| 4 | Nikon packed 14-bit under a compressed tag (Z 6II uncompressed, #769) | +0.2 | 1–2 | ~0.13 | PR #671 exists |
| 5 | Sony DSLR-A200…A390 / A850 12-bit compressed ARW | +0.4 | 3–6 | ~0.08 | samples on raw.pixls.us |
| 6 | Pre-X FinePix RAF | +0.6 | 4–8 | ~0.08 | samples |
| 7 | Sony A100 and DSC-F828 regression (#702) | +0.1 | 1–2 | ~0.07 | samples on raw.pixls.us |
| 8 | Canon CRW (2000–2004 bodies) | +1.2 by the uniform tail weight, far less by real use | 6–12 | ≤ 0.1 | samples |
| 9 | Nikon HE / HE★ NEF (Z 8, Z 9, Z 6III, Z f, Z50II, Z5II set to HE) | +1.1 | 20–40 | ~0.04 | a pure-Rust JPEG XS decoder |
| 10 | Compressed SRW, Hasselblad 3FR, IIQ, X3F, MRW, KDC, ERF, DCR, MEF, GPR | +0.6 together | 45–85 | ~0.01 | samples; low use |
Beyond decoding:
| Gap | Hours | Needs |
|---|---:|---|
| Measured colour calibration for the ~60 popular bodies, Kelvin WB, render-fidelity suite against Lightroom | 80–150 | chart shots per camera (a human with cameras) or licensable measured data |
| Lens profile database of our own (fitting tools, data format, top 100 lenses), maker-note vignetting / CA | 40–80 | calibration shots (a human with lenses) |
| Demosaic and highlight quality measured against Lightroom; X-Trans AI denoise | 15–30 | |
| **Total RAW** | **255–480 h** (decoding list above 120–220 h; ~60% parallelizes) | |
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-10 (later) | major | Model-by-model measurement against Adobe's published list (1,446 models: 234 verified, 943 unverified, 233 preview, 36 unsupported); `docs/raw/` data and re-measure script; use-weighted share 84% (40% on verified bodies); variant matrix; colour and lens sections with scores; ranked next cameras; RAW readiness ~56% from written parts |
| 2026-10-10 | major | Created: formats, makers, demosaic, colour and lens checklists; target counts from the installed Lightroom 9.6 bundle listing |
docs/raw/adobe-cameras.tsv
+1450
View File
File diff suppressed because it is too large Load Diff
docs/raw/coverage.py
+172
View File
@@ -0,0 +1,172 @@
#!/usr/bin/env python3
"""Measure LightCraft's raw camera coverage against Adobe's supported-camera list.
python3 docs/raw/coverage.py # print the summary, rewrite docs/raw/coverage.tsv
python3 docs/raw/coverage.py --check # exit 1 if coverage.tsv is stale
Inputs (all in docs/raw/):
adobe-cameras.tsv Adobe's published list (maker, camera, extensions, ...).
evidence.tsv per-model evidence: verified decodes, known preview-only or refused bodies.
usage-weights.tsv estimated share of photographers per maker bucket and popular bodies.
Every Adobe-listed model gets one class:
verified decoded from sensor data, and a real file from this body was checked (evidence.tsv)
unverified the decoder path for its format and coding should handle it; no sample checked
preview opens as the embedded JPEG (format or coding not decoded)
unsupported not imported, refused, or decodes wrongly
The format rules below mirror the decoders in crates/raw/src (see docs/raw-parity.md). When a decoder
changes, update the rules and evidence, rerun, and commit coverage.tsv with the docs.
Standard library only; no third-party packages.
"""
import csv
import os
import re
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
# Olympus ORF bodies that write uncompressed / packed data (crates/raw/src/vendor/orf.rs). Everything else
# writes compressed ORF, which is preview-only (the E-M5 II / PEN-F high-res mode alone is decoded).
ORF_DECODED = re.compile(r"^(E-1|E-10|E-20|EVOLT E-300|EVOLT E-330|EVOLT E-400|EVOLT E-500|E-300|E-330|E-500|Camedia C-\S+.*|Stylus XZ-2 iHS)$")
# Samsung SRW bodies with compressed raw data (crates/raw/src/vendor/srw.rs refuses 32770 / 32772 / 32773).
SRW_COMPRESSED = re.compile(r"(^|\W)(NX1|NX30|NX300|NX300M|NX500|NX2000|NX3000|NX3300|NX mini|EK-GN\d+|EK-KN\d+)(\W|$)")
# Fujifilm bodies before the X series whose RAF has no raw IFD (crates/raw/src/vendor/raf.rs).
RAF_OLD = re.compile(r"^(FinePix (E|F|S\d|S\d{3,4}|IS)|IS Pro|S5Pro|S3Pro|S2Pro)", re.I)
# Probability that an unverified body of a format decodes correctly, used for the use-weighted share only.
CONFIDENCE = {"DNG": 0.95, "CR2": 0.95, "CR3": 0.6, "NEF": 0.9, "NRW": 0.9, "ARW": 0.9, "SR2": 0.8, "RAF": 0.95,
"RW2": 0.95, "RWL": 0.95, "RAW": 0.95, "PEF": 0.9, "ORF": 0.8, "SRW": 0.8}
def read_tsv(name):
with open(os.path.join(HERE, name), newline="", encoding="utf-8") as f:
lines = [line for line in f if not line.startswith("#")]
return list(csv.DictReader(lines, delimiter="\t"))
def exts(field):
return [e.strip().upper() for e in re.split(r"[,/+]", re.sub(r"\(.*?\)", "", field)) if e.strip()]
def classify(maker, camera, ext_field):
"""(class, format, reason) from the format rules alone."""
es = exts(ext_field)
for e in es:
if e == "DNG":
return "unverified", "DNG", "DNG decoder (uncompressed, LJ92, lossy, Deflate, JPEG XL, LinearRaw)"
if e == "CR2":
return "unverified", "CR2", "CR2 lossless JPEG decoder"
if e == "CR3":
return "unverified", "CR3", "CRX decoder: lossless and C-RAW 0x100 / 0x200 single-tile; other variants fall back to the preview"
if e in ("NEF", "NRW"):
return "unverified", e, "NEF decoder (uncompressed, lossless, lossy, lossy after split); HE / HE* setting preview-only"
if e == "ARW":
return "unverified", "ARW", "ARW decoder (uncompressed, cRAW, packed 12-bit, lossless, downsized YCbCr)"
if e == "SR2":
return "unverified", "SR2", "SR2 decoder"
if e == "RAF":
if RAF_OLD.search(camera):
return "preview", "RAF", "pre-X FinePix RAF without a raw IFD"
return "unverified", "RAF", "RAF decoder (uncompressed, lossless / lossy compressed, Bayer and X-Trans)"
if e in ("RW2", "RWL") or (e == "RAW" and maker in ("Panasonic", "Leica")):
return "unverified", e, "Panasonic decoder, every raw format"
if e == "PEF":
return "unverified", "PEF", "PEF decoder (uncompressed, Huffman, packed 12-bit)"
if e == "ORF":
if ORF_DECODED.match(camera):
return "unverified", "ORF", "uncompressed / packed ORF"
return "preview", "ORF", "compressed ORF is not decoded"
if e == "SRW":
if SRW_COMPRESSED.search(camera):
return "preview", "SRW", "compressed SRW is not decoded"
return "unverified", "SRW", "uncompressed / packed SRW"
e = es[0] if es else ""
if e in ("CRW", "3FR", "FFF", "IIQ", "MOS", "ERF", "KDC", "MRW", "X3F", "RAW", "ORI"):
return "preview", e, f"{e} is opened from its embedded JPEG only"
return "unsupported", e, f"{e or 'unknown'} is not imported"
def main():
adobe = read_tsv("adobe-cameras.tsv")
evidence = {(r["maker"], r["camera"]): r for r in read_tsv("evidence.tsv")}
rows = []
for a in adobe:
cls, fmt, reason = classify(a["maker"], a["camera"], a["extensions"])
ev = evidence.get((a["maker"], a["camera"]))
if ev:
cls, reason = ev["status"], ev["source"]
rows.append({"maker": a["maker"], "camera": a["camera"], "format": fmt, "year_added": a["year_added"], "class": cls, "reason": reason})
out = os.path.join(HERE, "coverage.tsv")
header = ["maker", "camera", "format", "year_added", "class", "reason"]
text = "# Generated by docs/raw/coverage.py; do not edit.\n" + "\t".join(header) + "\n"
text += "".join("\t".join(r[k] for k in header) + "\n" for r in rows)
if "--check" in sys.argv:
with open(out, encoding="utf-8") as f:
if f.read() != text:
print("docs/raw/coverage.tsv is stale: run python3 docs/raw/coverage.py")
return 1
else:
with open(out, "w", encoding="utf-8") as f:
f.write(text)
classes = ["verified", "unverified", "preview", "unsupported"]
by_maker = {}
for r in rows:
by_maker.setdefault(r["maker"], {c: 0 for c in classes})[r["class"]] += 1
print("| Maker | Adobe models | verified | unverified | preview | unsupported |")
print("|---|---:|---:|---:|---:|---:|")
for m, c in sorted(by_maker.items(), key=lambda kv: -sum(kv[1].values())):
print(f"| {m} | {sum(c.values())} | " + " | ".join(str(c[k]) for k in classes) + " |")
tot = {k: sum(c[k] for c in by_maker.values()) for k in classes}
n = sum(tot.values())
print(f"| **All** | **{n}** | " + " | ".join(f"**{tot[k]}** ({100 * tot[k] / n:.0f}%)" for k in classes) + " |")
recent = [r for r in rows if r["year_added"] and int(r["year_added"]) >= 2017]
rc = {k: sum(1 for r in recent if r["class"] == k) for k in classes}
print(f"\nModels added since 2017 (ACR 10+): {len(recent)}: " + ", ".join(f"{k} {rc[k]}" for k in classes))
# Use-weighted share of photographers whose camera fully works (decodes from sensor data).
weights = read_tsv("usage-weights.tsv")
by_cam = {(r["maker"], r["camera"]): r for r in rows}
total = works = verified_share = 0.0
print("\n| Bucket | Weight | Camera fully works | of which verified bodies |")
print("|---|---:|---:|---:|")
for b in [w for w in weights if w["kind"] == "maker"]:
makers = b["makers_or_camera"].split(",")
popular = [w["makers_or_camera"] for w in weights if w["kind"] == "body" and w["bucket"] == b["bucket"]]
members = [r for r in rows if r["maker"] in makers]
pop_rows = [r for m in makers for (mk, cam), r in by_cam.items() if mk == m and cam in popular]
missing = set(popular) - {r["camera"] for r in pop_rows}
if missing:
print(f"usage-weights.tsv: unknown cameras {sorted(missing)}", file=sys.stderr)
return 1
rest = [r for r in members if r["camera"] not in popular]
def score(r):
if r["class"] == "verified":
return 1.0
if r["class"] == "unverified":
return CONFIDENCE.get(r["format"], 0.7)
return 0.0
def mean(rs, f):
return sum(f(r) for r in rs) / len(rs) if rs else 0.0
if pop_rows:
s = 0.8 * mean(pop_rows, score) + 0.2 * mean(rest, score)
v = 0.8 * mean(pop_rows, lambda r: r["class"] == "verified") + 0.2 * mean(rest, lambda r: r["class"] == "verified")
else:
s, v = mean(members, score), mean(members, lambda r: r["class"] == "verified")
w = float(b["weight"])
total += w
works += w * s
verified_share += w * v
print(f"| {b['bucket']} | {w:g}% | {100 * s:.0f}% | {100 * v:.0f}% |")
print(f"| **Weighted** | {total:g}% | **{100 * works / total:.0f}%** | **{100 * verified_share / total:.0f}%** |")
return 0
if __name__ == "__main__":
sys.exit(main())
docs/raw/coverage.tsv
+1448
View File
File diff suppressed because it is too large Load Diff
docs/raw/evidence.tsv
+249
View File
@@ -0,0 +1,249 @@
# Per-model evidence that overrides the format rules in coverage.py. status: verified | preview | unsupported.
# "verified" means a real file from this body decoded from sensor data in a corpus test or a recorded bulk check (source column).
maker camera status source
Apple iPhone 12 Pro Rear Wide Camera verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Canon EOS 40D verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Canon EOS 550D (EOS Digital Rebel T2i, EOS Kiss X4 Digital) verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Canon EOS 5D Mark II verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Canon EOS 5DS R verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Canon EOS 6D verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Canon EOS 7D verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Canon EOS 80D verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Canon EOS M50 (EOS Kiss M) verified corpus: crates/raw/tests/cr3_corpus.rs (exact sensor hashes, docs/cr3.md)
Canon EOS R100 verified corpus: crates/raw/tests/cr3_corpus.rs (exact sensor hashes, docs/cr3.md)
Canon EOS R8 verified corpus: crates/raw/tests/cr3_corpus.rs (exact sensor hashes, docs/cr3.md)
Canon EOS-5D Mark III verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm GFX 100 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm GFX 100RF (Final Support) verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm GFX 100S verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm GFX 50S verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-A2 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-A5 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-E1 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-E5 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-H2 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-H2S verified docs/raf-compression.md (supplied files, exact sensor arrays)
Fujifilm X-M5 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-T2 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-T20 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-T4** verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-T5 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Fujifilm X-T50 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Google Pixel 2 XL verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Leica C (Typ 112) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica D-Lux (Typ 109) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica D-Lux 3 verified PR #215: raw.pixls.us bulk check of every RW2 / RWL / RAW body
Leica D-Lux 4 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica D-Lux 5 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica D-Lux 6 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica D-Lux 7 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica Digilux 2 verified PR #215: raw.pixls.us bulk check of every RW2 / RWL / RAW body
Leica Digilux 3 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica V-LUX (Typ 114) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica V-LUX 1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica V-LUX 4 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Leica V-LUX 5 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Nikon COOLPIX B700 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Nikon D5100 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Nikon D7000 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Nikon D7500 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Nikon Z f verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Olympus E-1 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Olympus EVOLT E-400 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Olympus Stylus XZ-2 iHS verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Panasonic LUMIX DC - G97 (DC-G99M2) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-FZ1000M2 (DC-FZ10002) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-FZ80 (DC-FZ82, DC-FZ85) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-G100 (DC-G110) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-G100D verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-G9 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-G95D verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-G99 (DC-G90, DC-G91, DC-G95) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GF10 (DC-GF90) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GF9 (DC-GX850, DC-GX800) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GH5 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GH5M2 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GH5s verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GH6 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GH7 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GX880 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-GX9 (DC-GX7MK3) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-LX100 II verified PR #215: raw.pixls.us bulk check of every RW2 / RWL / RAW body
Panasonic LUMIX DC-S1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S1H verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S1M2 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S1M2ES (Final Support) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S1R verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S1RM2 (Final Support) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S5 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S5M2 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S5M2X verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-S9 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-TX2D (DC-TZ200D, DC-TZ202D, DC-TZ220D, DC-ZS200D, DC-ZS220D) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-ZS70 (DC-TZ90, DC-TZ91, DC-TZ92, DC-TZ93) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-ZS80 (DC-TZ95, DC-TZ96, DC-TZ97) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DC-ZS80D (DC-TZ95D, DC-TZ96D) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DCG9M2 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-CM1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FX150 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ100 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ1000 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ150 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ18 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ200 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ2500 (DMC-FZ2000, DMC-FZH1) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ28 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ30 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ300 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ330 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ35 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ38 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ40 (DMC-FZ45) verified PR #215: raw.pixls.us bulk check of every RW2 / RWL / RAW body
Panasonic LUMIX DMC-FZ50 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ70 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ72 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-FZ8 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-G1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-G10 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-G2 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-G3 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-G5 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-G6 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-G7 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-G8 (DMC-G80, DMC-G81, DMC-G85) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GF1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GF2 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GF3 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GF5 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GF6 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GF7 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GF8 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GH1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GH2 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GH3 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GH4 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GM1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GM1S verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GM5 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GX1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GX7 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GX8 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-GX85 (DMC-GX80, DMC-GX7MK2) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-L1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-L10 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-LF1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-LX1 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-LX100 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-LX2 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-LX3 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-LX5 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-LX7 verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-LX9 (DMC-LX10, DMC-LX15) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-ZS100 (DMC-ZS110, DMC-TZ100, DMC-TZ101, DMC-TZ110, DMC-TX1) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-ZS40 (DMC-TZ60, DMC-TZ61) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-ZS50 (DMC-TZ70, DMC-TZ71) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX DMC-ZS60 (DMC-TZ80, DMC-TZ81, DMC-TZ85) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Panasonic LUMIX ZS200 (DC-TX2, DC-TZ200, DC-TZ202, DC-TZ220, DC-ZS220) verified PR #215: 178 raw.pixls.us files from 118 Panasonic / Leica bodies, every RW2 format
Pentax K-3 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Pentax K-5 IIs verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Pentax K10D verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Ricoh GR III verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Samsung TL500 (EX1) verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
Sony A100 (DSLR-A100) unsupported issue #702: regression, refused as not a readable photo
Sony A200 (DSLR-A200) preview issue #535: preview-only on the raw.pixls.us sample
Sony A230 (DSLR-A230) preview issue #535: preview-only on the raw.pixls.us sample
Sony A290 (DSLR-A290) preview issue #535: preview-only on the raw.pixls.us sample
Sony A300 (DSLR-A300) preview issue #535: preview-only on the raw.pixls.us sample
Sony A3000 (ILCE-3000) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A33 (SLT-A33) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A330 (DSLR A330) preview issue #535: preview-only on the raw.pixls.us sample
Sony A35 (SLT-A35) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A350 (DSLR-A350) preview issue #535: preview-only on the raw.pixls.us sample
Sony A3500 (ILCE-3500) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A37 (SLT-A37) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A380 (DSLR-A380) preview issue #535: preview-only on the raw.pixls.us sample
Sony A390 (DSLR-A390) preview issue #535: preview-only on the raw.pixls.us sample
Sony A450 (DSLR-A450) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A500 (DSLR-A500) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A5000 (ILCE-5000) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A5100 (ILCE-5100) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A550 (DSLR-A550) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A55V (SLT-A55V) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies
Sony A560 (DSLR-A560) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A57 (SLT-A57) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A58 (SLT-A58) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A580 (DSLR-A580) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A6000 (ILCE-6000) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A6100 (ILCE-6100) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A6300 (ILCE-6300) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A6400 (ILCE-6400) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A65 (SLT-A65) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A6500 (ILCE-6500) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A6600 (ILCE-6600) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A6700 (ILCE-6700) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A68 (ILCA-68) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7 (ILCE-7) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7 II (ILCE-A7M2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies
Sony A7 III (ILCE-7M3) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7 IV (ILCE-7M4) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A700 (DSLR-A700) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A77 (SLT-A77) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A77 II (ILCA-77M2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7C (ILCE-7C) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7C II (ILCE-7CM2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7CR (ILCE-7CR) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7R (ILCE-7R) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7R II (ILCE-7RM2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7R III (ILCE-7RM3) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7R IIIA (ILCE-7RM3A) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7R IV (ILCE-7RM4) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7R IVA (ILCE-7RM4A) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7R V (ILCE-7RM5) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7S (ILCE-7S) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7S II (ILCE-7SM2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A7s III (ILCE-7SM3) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A850 (DSLR-A850) preview issue #535: preview-only on the raw.pixls.us sample
Sony A9 (ILCE-9) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A9 II (ILCE-9M2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A9 III (ILCE-9M3) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A900 (DSLR-A900) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony A99 (SLT-A99V) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies
Sony A99 II (ILCA-99M2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony Alpha 1 (ILCE-1)* verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony DSC-F828 unsupported issue #702 (SRF refused); SRF not in the import list
Sony DSC-HX95 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony DSC-HX99 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony DSC-V3 unsupported SRF not in the import list
Sony FX3 (ILME-FX3) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony FX30 (ILME-FX30) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-3 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-3N verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-5 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-5N verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-5R verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-5T verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-6 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-7 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-C3 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony NEX-F3 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony QX1 (ILCE-QX1) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX0 (DSC-RX0) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX0 II (DSC-RX0M2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX1 (DSC-RX1) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX10 (DSC-RX10) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX10 II (DSC-RX10M2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX10 III (DSC-RX10M3) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX10 IV (DSC-RX10M4) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX100 (DSC-RX100) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX100 II (DSC-RX100M2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX100 III (DSC-RX100M3) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX100 IV (DSC-RX100M4) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX100 V (DSC-RX100M5) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX100 VA (DSC-RX100M5A) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX100 VI (DSC-RX100M6) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX100 VII (DSC-RX100M7) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX1R (DSC-RX1R) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony RX1R II (DSC-RX1RM2) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony ZV-1 (DCZV1/B) verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony ZV-E1 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Sony ZV-E10 verified issue #535: raw.pixls.us bulk check of all 100 Sony bodies (decodes; BL/WL/crop/WB vs ExifTool)
Can't render this file because it contains an unexpected character in line 2 and column 3.
docs/raw/usage-weights.tsv
+114
View File
@@ -0,0 +1,114 @@
# Estimated share of Lightroom raw photographers by camera maker, and the popular bodies of the last ~8 years inside each.
# Sources (2026-10-10): Nikkei 2025 mirrorless production via PhotoRumors (Canon 2.3 M, Sony 1.8 M, Nikon 0.86 M, Fujifilm 0.75 M
# units, i.e. ~36 / 29 / 14 / 12 % of CIPA's 6.31 M); BCN 2025 Japanese retail via PetaPixel (Sony 29.9 %, Canon 27.4 %, Nikon 15.1 %);
# camera makers named in this repo's user issues (325 issues: Sony 33, Nikon 31, Canon 26, phones 23, Fujifilm 9, Panasonic/Leica 8,
# OM/Olympus 6, Pentax/Ricoh 6, Samsung NX 5). Canon and Nikon get extra weight for their installed base of DSLRs.
# Rows "maker" give the maker bucket's weight (percent of photographers); rows "body" give a popular body inside a bucket.
# Inside a bucket, the listed bodies share 80 % equally and every other Adobe-listed model of the bucket's makers shares 20 %.
kind bucket makers_or_camera weight
maker Canon Canon 33
maker Sony Sony 27
maker Nikon Nikon 16
maker Fujifilm Fujifilm 10
maker Panasonic Panasonic 4
maker OM / Olympus Olympus,OM Digital Solutions 3
maker Phones Apple,Google,Samsung,Xiaomi,Huawei,Oppo,OnePlus,LG,Motorola,Nokia 3
maker Pentax / Ricoh Pentax,Ricoh 1.5
maker Leica Leica 1
maker DJI DJI 1
maker Medium format and others Hasselblad,Phase One,Leaf,Mamiya,Sigma,Kodak,Konica Minolta,Epson,Casio,GoPro,Insta360,Zeiss,Pixii,Light,Skydio,Parrot,Yuneec,Contax,DxO 0.5
body Canon EOS R6 Mark II 1
body Canon EOS R5 1
body Canon EOS R6 1
body Canon EOS R7 1
body Canon EOS R8 1
body Canon EOS R10 1
body Canon EOS R50 1
body Canon EOS RP 1
body Canon EOS R (** see note) 1
body Canon EOS R5 Mark II 1
body Canon EOS R6 Mark III 1
body Canon EOS R3 1
body Canon EOS R100 1
body Canon EOS 90D 1
body Canon EOS M50 (EOS Kiss M) 1
body Canon EOS M50 Mark II 1
body Canon EOS M6 Mark II 1
body Canon EOS-5D Mark IV (** see note) 1
body Canon EOS 6D Mark II 1
body Canon EOS 80D 1
body Canon EOS 250D (EOS Rebel SL3, EOS 200D II, EOS Kiss X10) 1
body Canon EOS 1500D (EOS Rebel T7, EOS Kiss X90, EOS 2000D) 1
body Sony A7 IV (ILCE-7M4) 1
body Sony A7 III (ILCE-7M3) 1
body Sony A7R V (ILCE-7RM5) 1
body Sony A7R IV (ILCE-7RM4) 1
body Sony A7C (ILCE-7C) 1
body Sony A7C II (ILCE-7CM2) 1
body Sony A7CR (ILCE-7CR) 1
body Sony A6400 (ILCE-6400) 1
body Sony A6700 (ILCE-6700) 1
body Sony A6600 (ILCE-6600) 1
body Sony A6100 (ILCE-6100) 1
body Sony Alpha 1 (ILCE-1)* 1
body Sony A9 III (ILCE-9M3) 1
body Sony ZV-E10 1
body Sony A7s III (ILCE-7SM3) 1
body Sony RX100 VII (DSC-RX100M7) 1
body Nikon Z 6 II 1
body Nikon Z 6 III 1
body Nikon Z 8 1
body Nikon Z 9 (*See note) 1
body Nikon Z f 1
body Nikon Z 5 1
body Nikon Z 50 1
body Nikon Z 50 II 1
body Nikon Z fc 1
body Nikon Z 7 II 1
body Nikon D850 1
body Nikon D750 1
body Nikon D7500 1
body Nikon D780 1
body Fujifilm X-T5 1
body Fujifilm X-T4** 1
body Fujifilm X-S20 1
body Fujifilm X-S10 1
body Fujifilm X-H2S 1
body Fujifilm X-H2 1
body Fujifilm X100V** 1
body Fujifilm X100VI 1
body Fujifilm X-T30 II 1
body Fujifilm X-E4 1
body Fujifilm X-T50 1
body Fujifilm GFX 100S 1
body Fujifilm X-M5 1
body Panasonic LUMIX DC-S5 1
body Panasonic LUMIX DC-S5M2 1
body Panasonic LUMIX DC-GH6 1
body Panasonic LUMIX DC-GH5M2 1
body Panasonic LUMIX DCG9M2 1
body Panasonic LUMIX DC-G9 1
body Panasonic LUMIX DC-S1 1
body Panasonic LUMIX DC-GX9 (DC-GX7MK3) 1
body Panasonic LUMIX DMC-G8 (DMC-G80, DMC-G81, DMC-G85) 1
body Panasonic LUMIX DC-LX100 II 1
body OM / Olympus OM-D E-M1 Mark III 1
body OM / Olympus OM-D E-M1 Mark II 1
body OM / Olympus OM-D E-M10 Mark IV 1
body OM / Olympus OM-D E-M5 Mark III 1
body OM / Olympus PEN E-P7 1
body OM / Olympus Stylus Tough TG-6 1
body OM / Olympus OM System OM-1 1
body OM / Olympus OM System OM-1 MARK II 1
body OM / Olympus OM System OM-5 1
body Pentax / Ricoh K-1 II (* see note) 1
body Pentax / Ricoh K-3 Mark III 1
body Pentax / Ricoh KF 1
body Pentax / Ricoh K-70 1
body Pentax / Ricoh GR III 1
body Pentax / Ricoh GR IIIx 1
body Leica Q3 1
body Leica Q2 1
body Leica M11 1
body Leica SL2 1
body Leica M10 1
Can't render this file because it contains an unexpected character in line 6 and column 8.

Some files were not shown because too many files have changed in this diff Show More