Files
lightcraft/docs/roadmap.md
T
8972565249 LR-PROF-IMPORT: import the user's own DNG camera profiles (.dcp) (#1063)
* LR-PROF-IMPORT: import the user's own DNG camera profiles (.dcp)

profile.import (File > Import Profiles & Presets, drag & drop) now also
reads DNG Camera Profiles: the user's own .dcp files, e.g. made with a
colour-checker tool. They appear in the profile browser and menu under
Camera Profiles with dcp:<camera>-<file> ids; profiles.camera lists them
and profile.deleteCamera / profile.deleteImported remove them. A library
on disk keeps a copy in Profiles/. No profile ships with the app and the
repository holds none: tests build theirs in memory (write_dcp).

lightcraft_raw::dcp parses the file per DNG spec ch. 6 (magic RC in
place of 42; IFD 0 read through the existing DNG colour builders) and
DcpProfile::applied_to puts its matrices, illuminants and look in place
of a raw file's, keeping the shot's white balance, analog balance and
baseline exposure.

Choosing one (develop.profile) replaces the raw file's colour model when
it is decoded, before the starting colour and the white-balance scale
(WbScaleOf) are worked out, so the matrix path takes over from the fitted
JPEG look (CR3/CR2/ARW...) and white balance follows the profile. A
profile without a tone curve of its own (most chart-made ones) keeps the
photo's contrast: the DNG's tone curve, else the tone of the camera-JPEG
fit. The profile travels with SourceRef::File into FileLoader /
PairLoader (new Option<&DcpProfile> argument); a decoded source records
the profile it was made with and is not reused for another. Smart
previews and the web render worker keep the file's own colour; rendered
(non-raw) photos ignore it.

This reverses the documented "Adobe profile formats deliberately
unsupported" stance for user-supplied files only; docs updated.

Split out of #1012.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: build against upstream main (kelvin_presets, 3-arg loaders)

DcpProfile::applied_to keeps the file's white-balance presets (new
ColorData::kelvin_presets upstream): they are camera facts like as-shot
neutral. Upstream's new FileLoader test closures take the profile argument.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: hover previews and browser thumbnails develop the hovered camera profile

The loupe's hover preview (preview_job) and the profile browser's thumbnails (variant_job) took
the photo's own decoded source, so hovering an imported camera profile showed the built-in colour
(and hovering a built-in profile while one was applied kept the camera profile's). Both now decode
as the hovered settings develop: the source follows the settings' camera profile, as the main
render does, and a variant's cache key already follows its settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: build against upstream main (face-tile test loader and render_bench take the camera profile)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* CI: the catalog's staging-link test module is Unix-only (unused import broke clippy on Windows)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* CI: same-folder export test expects the '/'-joined subfolder (failed on Windows)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* CI: folder-drop tests build widget ids with the Folders tree's '/' paths (failed on Windows)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: rustfmt after merging main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a real D750 camera profile as a pinned test corpus, checked against Lightroom

dieli's CC0 Nikon D750 ColorChecker set (Calibrite PROFILER dual-illuminant DCP,
matrix-only; the two chart NEFs; Lightroom Classic 15.6 renders with the profile)
is fetched by cargo xtask corpus --download into corpus/d750-colorchecker/, pinned
to commit f37f4054 and by sha256; nothing is committed.

Tests (skip without the corpus): the profile parses as matrix-only with
Cool White Fluorescent + D55; the matrices blend by inverse temperature; the
matrix-only path inverts the colour matrix and adapts to D50; it takes over the
chart NEFs' colour; imported and applied end to end, the charts develop within
mean dE2000 3.84 (flash) / 3.35 (indoor) of Lightroom's renders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a single .dcp file is a camera profile only, not also an unreadable LUT

profile.import read every plain file that was neither a folder nor a .zip as a .cube LUT, so importing
one .dcp imported the camera profile and also reported it as failed ("stream did not contain valid
UTF-8"). Only .cube files are read as LUTs now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: review fixes for imported camera profiles

- A library moved or restored elsewhere finds its camera profiles in its own Profiles/ folder.
- A grid thumbnail is cut from the decoded preview only when that preview used the same camera profile.
- Render and source cache keys carry a fingerprint of the profile file, so a deleted or replaced profile never
  has its old renders stand in.
- Camera profiles can be favourites, and favourites/recent entries survive reopening the library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: smart previews without the camera profile, current-profile source facts, unsupported files reported

CodeRabbit review of 4ef025d9:
- Build Smart Previews develops the proxy without the photo's imported camera profile and never takes a cached
  source decoded with it (the denoised picture is kept); before, only the stale-proxy refresh left it out.
- Session::source_info reports decoder facts only from a cached source developed with the camera profile the
  photo uses now, else the catalog's.
- profile.import reports a file that is neither .cube, .dcp nor .zip as failed instead of passing over it.

Each with a regression test in tests_dcp.rs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: the Before side decodes with its own camera profile; a failed profile copy fails that file only

- build_job (and region_job_before through it) picks the Before render's source with the Before
  settings, so it never reuses or decodes a source developed with the edit's camera profile.
- profile.import: a .dcp whose copy into the library fails is reported in `failed`; the profiles
  imported with it stay imported and are saved, and nothing half-written is left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a cached smart preview stands in while the original is offline

A smart preview carries no camera profile, so with an imported profile the profile check
rejected the cached proxy and every render decoded it again (it gives the same picture). It
now stands in, for its own level and for the thumbnail cut from it, while the original is
known to be offline; otherwise the profile check is as before, so once the original is back
the photo is decoded from it with its profile. Photos without an imported profile keep main's
behaviour. Test `tests_dcp::a_cached_smart_preview_stands_in_while_the_original_is_offline`
(fails without the fix).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: deleting a camera profile removes the library's copy before forgetting it

profile.deleteCamera now removes the library's copy of the profile file
first. A copy that is already gone is fine; any other error (file in use,
no access) fails the command and keeps the profile listed, so the library
never holds a profile file it no longer lists. Test:
a_profile_whose_copy_cannot_be_deleted_stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: deleting a camera profile never follows `..` out of the profiles folder

A listed path with a `..` component can start with the profiles folder and
still lead out of it; such a path is not treated as the library's copy, so
deleting the profile leaves that file alone. Test:
deleting_a_profile_never_follows_dot_dot_out_of_the_profiles_folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: both tone fits of a DNG see its lens opcodes; no side effect in the profile-tag comparison

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a hover preview's key carries the camera profile file's fingerprint

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: unusable camera profiles not offered; file's WB scale and the profile's tone kept

Review of #1063 (CodeRabbit, outside-diff findings):
- a camera profile whose file is missing or damaged when the library opens
  is no longer offered in the profile browser or accepted by develop.profile
  (it silently developed with the raw's own colour); the entry stays, so it is
  offered again once the file is back.
- a raw on the relative white-balance scale stays relative with a camera
  profile (its matrices turned it to Kelvin, so existing edits were read as
  absolute temperatures and the catalog and quick previews disagreed).
- with a camera profile its tone curve, then the tone kept for a curve-less
  profile, come before the fitted camera look's tone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: one Camera Profiles group; the web loader keeps a raw's own colour

Review of #1063 (CodeRabbit):
- a LUT group named "Camera Profiles" that didn't sort last made the profile
  menu list that group twice; it is now one group, last, with both kinds.
- without web workers a render ran inline and the web loader applied the
  selected camera profile, unlike the worker (the documented web behaviour
  keeps the raw's own colour); the loader now leaves it out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: camera profile files are read at most to the 64 MiB limit

A .dcp larger than the parser accepts was read whole into memory before
being refused. One bounded read (lightcraft_raw::dcp::read_dcp) stops one
byte past the limit and is used for import, reopening a library and the
duplicate check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* LR-PROF-IMPORT: a test that a hostile hue/saturation map header in a camera profile drops the map (FredApps, #1063)

Dimensions far past the data, past u32 products or zero, an entry count of 4 billion values and a
data offset past the end each drop the map while the profile's matrix still reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Docs: parity checklist keeps both sides of the merge

* LR-PROF-IMPORT: tests added on main take the file loader's camera-profile argument

* Merge main; a thumbnail test takes the file loader's camera-profile argument

* Docs: one camera colour row, with main's notes and the import note

* Process key tests take the profile salt argument

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-05 07:09:30 +01:00

116 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Roadmap: milestones, current focus and what's next
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-24 · **Change:** major (LR-PROF-IMPORT: the user's own `.dcp` camera profiles import; gap 9: roadmap item 8 no longer lists Linux, where CI runs the tests since #2667, and lists the GPU tests; created; milestones moved here from ROADMAP.md, focus re-ranked after the 2026-10-10 re-measure) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
Forward-looking plan. The summary with the headline numbers is [ROADMAP.md](../ROADMAP.md); the
ranked shortfalls are [gaps.md](gaps.md); the numbers come from [target-app-parity.md](target-app-parity.md).
LightCraft has no `cargo xtask scorecard`; the measured checklist is
[parity-checklist.md](parity-checklist.md) (`cargo xtask parity`).
## Current focus
Toward **beta** (~340–600 Opus 5.5 agent hours; ranked, each a gap in [gaps.md](gaps.md)):
1. **Camera colour of our own** (gap 1, 50–90 h): chart-based calibration for the most-used models,
building on the per-file JPEG fits and pooled profiles (`lightcraft-cli calibrate`); never Adobe
data.
2. **CR3 for every Canon body** (gap 2, 25–45 h): the remaining CRX variants, sRAW / mRAW, HEIF CR3;
clean-room from prose descriptions (decided 2026-10-05; compressed NEF is the template).
3. **Render-fidelity suite** (gap 3, 30–60 h): score our output against Lightroom's on the same CC0
raws (references local in `plan/`), then tune; a retune ships as a new process version.
4. **Full-resolution zoom** (gap 4, 8–15 h): visible-region rendering so 100% shows original pixels.
5. **AI masks** (gap 5, 40–70 h): Subject / Sky / People on licensable weights; configure the SAM 3
mirror. Needs the maintainer's model decision.
6. **Compressed ORF and HE NEF** (gaps 6–7, 28–55 h).
7. **Lens profiles of our own** (gap 8, 40–80 h).
8. **Tests in CI on macOS and Windows, and the GPU tests in CI** (gap 9, 6–12 h).
9. **Per-model verification** (gap 10, 20–35 h).
## Alpha gate
The target's core workflows: what a typical Lightroom user does every day. Alpha requires each to
work end to end on the main platform (macOS) and the work to save and reopen. Checked on 2026-10-10
against the code, tests, issues and [gaps.md](gaps.md).
| Workflow | Works end to end? | Evidence | Hours to pass |
|---|---|---|---:|
| 1. Import a shoot (raw + JPEG) from a card or folder into the library | yes | Add / Copy / Move import with templates, duplicates, devices with DCIM, background import with cancel (`crates/engine/src/import.rs`, `tests_import.rs`, `tests_import_move.rs`); checklist A 83% | 0 |
| 2. Cull: rate, flag, label, compare / survey, filter, check focus | yes (partial: focus check) | ratings / flags / labels with advance, Compare, Survey ≤ 48, filter bar, smart albums (checklist B–D 88–92%); 100% zoom is capped by the preview size (2,560 px default, adjustable in Settings), gap 4 — degrades focus checks on large files but does not block culling | 0 (8–15 h to close gap 4, beta) |
| 3. Develop raws non-destructively (global sliders, crop, masks, heal) | yes for most cameras (partial: Canon CR3 outside the verified variants, compressed ORF, HE NEF open as embedded previews) | Sony, Nikon (except HE), Fujifilm, Panasonic, Pentax, CR2, DNG decode from sensor data (raw-parity.md); every Edit slider, crop / Upright, brush / gradient / range masks, heal / clone (checklist F, H, K); colour estimated per file rather than measured, gaps 1–3 | 0 (coverage and colour gaps are beta blockers 1–3, 6–7) |
| 4. Save, quit and reopen the work | yes | crash-safe journal + snapshots, torn-append and crash tests (`crates/catalog/src/journal.rs`, `tests_journal.rs`, `tests_torn_append.rs`); XMP sidecars read / write; failed saves reported | 0 |
| 5. Batch-apply a look: presets, copy / paste / sync settings | yes | presets (incl. `.xmp` / `.lrtemplate` import), copy / paste / sync, auto sync (checklist L 86%, N 100%) | 0 |
| 6. Export deliverables (JPEG / TIFF, size, sharpening, metadata, watermark, naming) | yes | one encoder for app, CLI and MCP (`crates/engine/src/export.rs`, `tests_export.rs`); checklist S 89% | 0 |
**Result: passes.** Every core workflow runs end to end on macOS and the work survives quit and
reopen; the partial rows degrade fidelity for some cameras and large files but don't stop the
workflow, so LightCraft stays **alpha** (ready for real work ~61%, above the ~40% bar).
## After beta
Enhance (Raw Details, Super Resolution, X-Trans denoise), content-aware remove, HDR display, the
Classic output modules (Print, then Map, Book, Slideshow, publish services), tethering, video,
plug-in ecosystem, localization to `full` and new scripts (RTL, shaping). Estimates in
[gaps.md](gaps.md) entries 11–28; ~500–1,300 h after beta.
## Milestones
Original wall-clock estimates (made before work started, 4–6 parallel agents) are kept for
calibration: M0–M13 took ≈ 25 active hours against an estimate of ≈ 110–170 h.
**Status legend:** ✅ done · 🚧 in progress · ⬜ not started
| # | Milestone | Scope (summary) | Estimate (h) | Status |
|---|---|---|---|---|
| M0 | Skeleton + visual shell | workspace, xtask CI + layering, geom/color/raster, develop model, pipeline v0, catalog v0, engine commands, Lightroom-look UI (grid, loupe, filmstrip, Edit panel), control channel, MCP, web build | 3–5 | ✅ |
| M1 | Library core | import (JPEG/PNG/TIFF/WebP), EXIF/XMP, persistent catalog (op log + snapshots), albums, ratings/flags/labels, filter/search/sort, thumbnail cache, 100k-photo grid | 6–10 | ✅ (100k-photo grid scale test pending) |
| M2 | Pipeline v1 (quality) | WB temp/tint, profiles, local tone mapping (highlights/shadows), curves, HSL, point colour, colour grading, texture/clarity/dehaze, vignette, grain, B&W, auto tone/WB, histogram, before/after | 10–15 | ✅ (look tuning vs our references ongoing) |
| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | ✅ (DNG, CR2 incl. sRAW, ARW, NEF uncompressed / lossless / lossy / lossy after split, embedded previews) |
| M4 | Crop, geometry, optics | crop tool + overlays, straighten, Upright (auto/level/vertical/full/guided), manual transforms, CA, defringe, manual lens corrections | 6–10 | ✅ |
| M5 | Performance | source pyramids, wgpu compute pipeline (CPU oracle), draft/full renders, prefetch, budgets (16 ms slider updates on 24 MP) | 10–15 | 🚧 (stage cache, source pyramid, wgpu pipeline, prefetch, memory budget ✅; colour NR at half resolution, GPU histogram ⬜) |
| M6 | Masking | brush, linear/radial gradients, colour/luminance/depth range, add/subtract/intersect/invert, all local adjustments, masks panel | 8–12 | 🚧 (brush/linear/radial/colour/luminance range, add/subtract/intersect, masks panel ✅; depth range, AI masks ⬜) |
| M7 | Detail | sharpening + masking preview, luminance/colour NR, Denoise, Raw Details, Super Resolution | 6–10 | 🚧 (sharpening, luminance/colour NR ✅; AI Denoise 🟡 (Bayer, optional weights; policy/fidelity open); Raw Details, Super Resolution ⬜) |
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | 🚧 (heal, clone, auto source, visualize spots, red/pet eye ✅; PatchMatch remove ⬜) |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ✅ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | 🚧 (all formats incl. DNG/original, sizing, presets, background jobs, HDR gain map JPEG / PQ AVIF ✅; JXL encode ⬜) |
| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF all, RW2 every format, PEF, ORF uncompressed and 12.8 bpp, SRW uncompressed, HEIC ✅; CR3 🟡 (3 bodies verified); **camera colour calibration** 🚧 (per-file JPEG fits, 5 bundled profiles, 52 spectral matrices; measured database missing); compressed ORF / SRW, HE NEF, IIQ, X3F ⬜) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | 🚧 (faces: detection, recognition, People view ✅; Object / Describe masks via SAM 3 🟡; Subject / Sky / People masks, semantic search ⬜) |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ✅ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
| M15 | Classic modules | Map, Book, Slideshow, Print, Web; smart collections, stacks, virtual copies, publish services, tethering | 25–40 | 🚧 (smart albums, stacks, virtual copies, compare/survey ✅; contact-sheet PDF 🟡; Map/Book/Slideshow/Print/Web ⬜) |
| M16 | 1.0 polish | preferences, shortcut editor, accessibility, localization, packaging (dmg/msi/AppImage/web), hardening | 10–20 | 🚧 (settings, keyboard shortcuts sheet, packaging basics ✅; English/Chinese (Simplified, Traditional)/Japanese/Brazilian Portuguese/Spanish/German/Russian/Ukrainian/Georgian localisation 🟡; accessibility and further locales ⬜) |
## Risks that coding hours alone don't retire
- **AI features** (subject / sky / people masks, generative remove) need model weights with
licences we can ship; classical fallbacks first. No permissively licensed sky-segmentation or
raw-denoise model was found; we may need to train our own.
- **Camera colour and lens data** is a data problem: we never use Adobe's matrices, DCPs or LCPs (users may import their own `.dcp` camera profiles).
DNG-embedded data first, then our own calibration; long-tail coverage grows over time and needs
people with cameras, lenses and charts.
- **Raw-format sources:** decoders are written from prose format descriptions (even ones published
alongside GPL code); decoder source is never read (decided 2026-10-05). Still open:
freedom-to-operate review for local Laplacian filters, PatchMatch and HEVC (HEIC).
- **Look parity** with Adobe's default rendering is tuned by eye until the fidelity suite exists.
Retuning changes existing edits, so it ships as a new process version
([process-versions.md](process-versions.md)).
## Design notes kept from the old ROADMAP
- **Folder records:** library folders have records of their own in the catalog
(`Catalog.folder_records`, keyed by folder path; colour labels first). Photos still store full
paths, so moving a whole disk relinks each photo. If that becomes a problem (an "Update Folder
Location" for a renamed drive, journal growth on large moves), the next step is the Classic model:
photos point to a folder id (root + relative path).
- **Zoom:** native pinch zoom and two-finger pan work in Detail, Compare and Reference views; the
whole-image preview still has a configurable cap (2,560 px default), see gap 4.
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-24 | patch | Item 8 (gap 9): Linux dropped from "Tests in CI" and "the GPU tests in CI" added, because the CI gate has run the tests there since #2667; hours unchanged |
| 2026-10-12 | minor | LR-PROF-IMPORT: the user's own DNG camera profiles (`.dcp`) import as camera profiles; none ship |
| 2026-10-10 | minor | Beta hours 340–600; full ready ~59% in the gate note |
| 2026-10-10 | minor | Added the Alpha gate table (core-workflow gate from the revised standard): passes, stays alpha |
| 2026-10-10 | major | Created: milestones moved from ROADMAP.md (statuses refreshed: M3 ✅, M11 / M12 / M15 detail), current focus re-ranked toward beta, risks and design notes kept |