mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-31 05:28:25 -06:00
Merge origin/main into slideshow-playback (icons: Play/Pause beside grading and appearance icons; settings: Slideshow tab beside appearance cards; render: Slide and CompareRegion slots; tracker summary and whats-new)
This commit is contained in:
Jason Baldwin
@@ -0,0 +1,174 @@
|
||||
# Sony ARW6 (Compressed RAW 2)
|
||||
|
||||
LightCraft decodes the raw sensor data of Sony's *Compressed* and *Compressed (HQ)* ARW files (TIFF Compression 32766, SonyRawFileType 6, "ARW 6.0") written by the A7R VI (ILCE-7RM6) in pure Rust. The shared RAW decoder serves import, desktop previews, editing, CLI rendering, web and export. Before this decoder these files failed the full decode and silently rendered from their embedded JPEG. The ILCE-7M5 (A7 V) writes the same codec; no sample of it was available, so it is unverified.
|
||||
|
||||
Decode timings: see the end of this document (measured in the performance pass).
|
||||
|
||||
## Format and implementation
|
||||
|
||||
The codec is Sony's use of SMPTE RDD 34 (LLVC, Low Latency Video Codec). `crates/raw/src/llvc.rs` is the codec core (bit reader, variable-length decoder, dequantiser, inverse 5/3 wavelet); `crates/raw/src/vendor/arw6.rs` is the Sony container, geometry and colour reconstruction; `crates/raw/src/vendor/arw6_curve.rs` is the companding curve; `arw.rs` dispatches Compression 32766 to it. A test-only encoder (`arw6_testenc.rs`) builds synthetic tiles for round-trip tests.
|
||||
|
||||
### Container
|
||||
|
||||
- Raw SubIFD: Compression 32766, 14 bits per sample, BlackLevel 512 x4 (`0x7310`), WB `0x7313`, crop `0x74c7/0x74c8` (full frame: origin 12,8, size 9984x6656; APS-C: origin 16,10, size 6528x4352), CFA RGGB. The Sony tag-based white level (15360) is not read; the DNG `WhiteLevel` is (16383).
|
||||
- The single strip starts with a tile table: `u32` count, `u32` 0, then per tile a `u64` offset (relative to the strip) and `u32` x, y, width, height. Seen: full frame 4 tiles of 5008x3336 (HQ) or 5024 / 4992 wide (Compressed); APS-C (6592x4372) one tile. Bytes up to offset 512 follow the table (480 on APS-C, 408 on full frame; not interpreted, not needed).
|
||||
- A tile is a sequence of 16-byte words. Word 0 is RDD 34 `Picture_info` (HS = tile width, VS = tile height / 2, BBD = 16, NC = 3, NW = 3, NP = `1000000000b`; other values are refused as unsupported). Word 1 holds five `u24` group byte-sums, word 2 is zero, words 3-7 describe the groups (a count byte, then `u24` stream sizes in words). Groups: g0 = LL3 (3 streams, one per component), g1 / g2 / g3 = level-3 / 2 / 1 bands (3 streams each), g4 = green residual (1 stream). Streams follow in that order from word 8.
|
||||
- A stream is a header word (a `u16` count of index words, a `u24` count of data words, then 11 bytes that are not interpreted), an index table and data, each padded to a word. The index has one entry per transform unit (TU): `A(16) X B(16) X`, where A and B are the byte lengths of the TU's two halves and X are quantiser-index (QI) nibbles (one for g0 and g4, three for g1-g3: HL, LH, HH). Halves per tile = ceil((VS - 2 - s) / 8) + 1 with s the first-half shift (below), TUs = ceil(halves / 2): 105 and 137 for the sample sizes. Halves are numbered n = 2 * TU + h; each is byte-aligned and consecutive halves are contiguous.
|
||||
- QI values seen, identical in Compressed and HQ files: level 1 component 0 (1,1,2), components 1-2 (2,2,3); level 2 (0,0,1); level 3 (0,0,0); LL3 0; residual 2. Compressed and HQ differ only in the coefficient data.
|
||||
|
||||
### Entropy coding and dequantisation
|
||||
|
||||
- Each line of a band is an RDD 34 section 6.3 coefficient-set stream: ceil(width / 4) sets of four coefficients driven by the DPT / ABS / SGN state machine, with zero runs and the "0^lim" end code. Values past the width are discarded. Lines of a half follow one another bit-contiguously; the state resets per line.
|
||||
- Two deviations from the RDD 34 text were established by black-box comparison: in state DPTs = 0 with DPT = 0, a "0" code is a zero set that also enters DPTs = 1; and a run code of value V means V - 1 further zero sets before the post-run "0^n 1" depth code.
|
||||
- Dequantisation is `recon(q) = sign(q) * (((2|q| + 1) << (QI - 1)) - (|q| & 1))`, `recon(0) = 0`, and QI = 0 leaves q unchanged. This replaces RDD 34's plain shift.
|
||||
- LL3 lines are DPCM: a running sum along the line. The 2048 base (codes are unsigned 12-bit) applies to component 0 (the green mean) only; the chroma planes start at 0.
|
||||
- Check: every half's lines end inside its last byte. The Phase 0 measurement found budget - 8 < consumed <= budget on every half of all 8 files; the decoder rejects a half that does not.
|
||||
|
||||
### Frame geometry
|
||||
|
||||
- Each tile holds three W/2 x H/2 planes (W, H = tile size): component 0 = M, the green mean; components 1-2 = chroma c1, c2; plus a W/2 x H/2 residual plane from g4. A three-level 5/3 wavelet is inverted (vertical, then horizontal) with whole-sample symmetric extension at every edge and integer arithmetic. Tiles are independent.
|
||||
- The TU grid and the lifting phases are anchored to the full sensor, not to the tile. Let s = (lines in g4's first half) - 2, read from the stream (residual lines are decoded while at least 8 bits of the half remain, at most 8): 0 on full frame, 3 on the APS-C crop (the crop's sensor-row offset 573 is 5 mod 8). Half n then covers plane rows 8n - 6 + s .. 8n + 1 + s, clipped to the frame. The lifting phases follow from s: full frame (0,1,1), APS-C (1,0,0). Each half holds, in clipped natural order, its rows of HL1, LH1, HH1, HL2, LH2, HH2, LL3, HL3, LH3, HH3 and, in g4, 8 residual rows; edge halves are shorter. With the right s every line of every file is used.
|
||||
|
||||
### Colour reconstruction
|
||||
|
||||
With plane row j, column x (x +- 1 clamped at the edges), in 12-bit code units:
|
||||
|
||||
- `G1[j][x] = M[j][x] - ((res[j-1][x] + res[j-1][x+1] + res[j][x] + res[j][x+1] + 4) >> 3)`, with `res[-1] := res[0]`.
|
||||
- `G2[j][x] = ((G1[j][x-1] + G1[j][x] + G1[j+1][x-1] + G1[j+1][x]) >> 2) + res[j][x]` (unclipped G1; the last row uses G1[j] for G1[j+1]).
|
||||
- `R = 2*c1 + ((clamp(G1, 0, 4095) + clamp(G2, 0, 4095)) >> 1)`, `B = 2*c2 + (same)` (a negative green, as in deep high-ISO shadows, counts as 0).
|
||||
- Every code is clipped to 0..4095. Per 2x2 cell: R (0,0), G1 (0,1), G2 (1,0), B (1,1).
|
||||
|
||||
### Output range (companding curve)
|
||||
|
||||
Codes map through one monotone curve, the same for every file and both modes. It is not stored in the file (below); it is our own measurement. Output values are in 2 x 14-bit DN: black 2 x 512 = 1024, the 14-bit white 16383 maps to 32766.
|
||||
|
||||
- The curve is piecewise linear through knots (c_k, u_k), k = 0..306: u_k = 128 k (one knot per 64 DN of the 14-bit scale); c_k = u_k for k <= 11 (the identity through code 1408); c_k = 356.89 + 655.646 ln(k - 6) for k >= 12 (that is, c = -2824.32 + 655.646 ln(u - 768)). F(code) is the linear interpolation of u over c, rounded half up (floor(x + 0.5)). Values: F(1408) = 1408, F(1536) = 1542, F(2048) = 2457, F(3000) = 7979, F(3500) = 16227, F(3977) = 32767, F(4095) = 39077.
|
||||
- `arw6_curve.rs` builds the 4096-entry table from the formula at first use; a unit test pins the whole table (its sum and position-weighted sum), so a platform whose `ln` rounds differently fails it.
|
||||
- The decoder reports black 1024 and white = 2 x the file's DNG `WhiteLevel` (16383), i.e. 32766; a file without a usable white level gets 32766 too (2 x 16383), not a data-derived value. Codes above 3977 sit above white and clip as usual.
|
||||
|
||||
#### How it was measured
|
||||
|
||||
Each of the four scenes of the CC0 set (APS-C and full frame, landscape and portrait) has a lossless (LJ92) file and a Compressed and an HQ file, shot handheld one after another. Seven Compressed / HQ files share shutter, aperture and ISO with their lossless sibling. The eighth, APSC_land RAW-C (1/500 s against 1/1000 s), was left out: its brightness ratio to the lossless file is 1.37 rather than 2 and drifts with level, so the light changed between the shots.
|
||||
|
||||
- Data: the ARW6 12-bit codes from LightCraft's decoder with the curve replaced by the identity, and the lossless 14-bit DN from LightCraft's LJ92 path. Both modes share the sensor origin (their crop tags are equal); the lossless frame is the ARW6 frame padded with zeros at the right and bottom.
|
||||
- Registration: the handheld shifts are large (median 3 to 102 green cells). Patches were registered locally on the green planes with a curve-independent rank transform, normalised cross-correlation and a neighbour-consistency filter.
|
||||
- Two estimators per pair and CFA channel, each capture with its own gain and offset (the gains' geometric mean fixed to 1, the offsets' mean to 0):
|
||||
- quantile (CDF) matching over all overlapping pixels: for each code, the mean lossless value over the slice of the distribution the code occupies (precise, slightly smoothed by compression noise);
|
||||
- block pairs: smooth 8 x 8-cell blocks (16 x 16 sensor pixels) away from clipping, a model linear in F, fitted robustly (unbiased).
|
||||
- Fitting c = C + A ln(u - B) through the knots gives A 655.88, B 766.9, C -2826.5 on the blocks and A 653.47, B 770.1 on the quantile curves. Holding B at 768 changes the curve by at most 0.08 %; the formula above (A 655.646, C -2824.32) uses it.
|
||||
|
||||
Uncertainty: 95 % intervals of a cluster bootstrap over the 7 captures (40 replicates), as a share of F - 1024 and in codes, and the largest deviation of any subset or method fit (Compressed only, HQ only, greens, red + blue, APS-C, full frame, quantile method):
|
||||
|
||||
| Codes | Bootstrap 95 % | In codes | Largest subset / method deviation |
|
||||
|---|---|---|---|
|
||||
| 1408-1536 | -0.42..+0.35 % | -1.6..+2.0 | 0.75 % |
|
||||
| 1536-2048 | -0.65..+0.58 % | -3.0..+2.8 | 0.80 % |
|
||||
| 2048-2560 | -0.55..+0.57 % | -3.1..+3.3 | 0.78 % |
|
||||
| 2560-3072 | -0.59..+0.50 % | -3.1..+3.7 | 0.64 % |
|
||||
| 3072-3584 | -0.76..+0.54 % | -3.5..+4.9 | 0.77 % (quantile) |
|
||||
| 3584-3840 | -0.84..+0.57 % | -3.7..+5.5 | 0.90 % (quantile) |
|
||||
| 3840-3977 | -0.88..+0.59 % | -3.8..+5.8 | 0.97 % (quantile) |
|
||||
| 3977-4095 | -0.91..+0.62 % | -4.0..+5.9 | 1.03 %, extrapolated |
|
||||
|
||||
The lossless files clip at the 14-bit white, so codes above about 3977 rest on the formula alone. The camera's exact integer table can't be recovered: integer knot codes, the encoder's rounding and a few codes around the knee are below the measurement's resolution.
|
||||
|
||||
#### Evidence for the structure
|
||||
|
||||
- Knots every 128 u: the measured slope dF/dc is flat within segments and jumps at codes 1531, 1632, 1719, 1796, 1865, 1928, 1984, 2037, 2085…, where u = 1536, 1664, 1792…. On codes 1380-2200 a spacing of 128 leaves 0.021 RMS relative slope error; the next best, 126, 64 and a smooth log, leave 0.037, 0.041 and 0.068. Above code 2200 the segments are too short to resolve, but there the piecewise-linear and smooth forms differ by less than 1.
|
||||
- Log spacing: the measured knot codes follow c = C + A ln(u - B) with chi-squared per degree of freedom 0.90 (bootstrap 95 %: A 654.3-658.2, B 762.8-771.1).
|
||||
- The knee: the log branch at u = 1408 gives code 1413 (95 % 1410-1415), so the identity runs through knot 11 and the log knots start at k = 12 (segment slope 1.035 between them). Starting them at k = 11 fits as well but would need a slope below 1 right after the identity; a smooth log leaving the identity near code 1340 fits the blocks clearly worse (cost 25.4 against 22.9).
|
||||
- One curve for everything: Compressed and HQ, all four CFA channels, APS-C and full frame agree within 0.8 %, and the ratio is flat with frame radius to +-0.5 % (no gain map).
|
||||
|
||||
#### Scale
|
||||
|
||||
The x2 unit is measured, not assumed:
|
||||
|
||||
- Below the knee F(c) = c: the measured slope is 1.000 +- 0.01 for codes 1150-1408, with offsets within +-2.
|
||||
- The overall level is fixed by the captures' average exposure to +-0.4 % (per-capture gains 0.99-1.02). Independently, flat clipped areas of the Compressed files sit at code 3977 +- 1 in all four scenes, and the formula, not fitted to that, puts the 14-bit white (32766) at code 3977.0 (bootstrap 95 % 3973-3983).
|
||||
- HQ files carry data above the 14-bit clip: their clipped highlights sit at codes 3996-4095 depending on the scene (about 16.8k-19.5k DN), so the curve keeps its values above 32766. The reported white level is unaffected.
|
||||
|
||||
#### Not stored in the file
|
||||
|
||||
Every IFD of all 12 files was dumped and compared across modes:
|
||||
|
||||
- Raw SubIFD: `0x7010` (SonyToneCurve) and `0x7011` are zero. The monotone arrays `0x7032`, `0x7035` and `0x7037` are lens corrections, the same in every mode. The only mode-dependent tag is `0x7000` SonyRawFileType (4 against 6).
|
||||
- Maker note (`0x9050`, `0x94xx`, deciphered): the Compressed and HQ bytes differ from the lossless ones only in 1-2 bytes (the quality setting), and hold no curve-shaped monotone run.
|
||||
- SR2SubIFD (encrypted, 42478 bytes): every file uses the same keystream, so XOR-ing two files compares their plaintexts. The lossless and HQ files of a scene differ in only 34 bytes, so there is no ARW6-only region. A curve stored identically in every mode can't be ruled out, but none is known there. `0x7250` likewise has no mode-specific region.
|
||||
- The bytes after the tile table (480 on APS-C, 408 on full frame; the last 120 high-entropy) are identical in the Compressed and HQ files of a scene, even at different exposure, and differ between scenes: not a curve.
|
||||
|
||||
#### Verification against the oracle
|
||||
|
||||
After the derivation was frozen, the derived table was compared with the black-box oracle's output curve (below), as a check only, with no tuning. They are identical through code 1422 (both are the identity through 1408), then within -1.0 % to +0.25 % of (value - black); the largest difference is 75 (0.2 %) at code 4093. Ours minus the oracle's: -5..+1 for codes 1408-1536, -13..-2 for 1536-2048, -33..-12 for 2048-3072, -34..+40 for 3072-3977, +1..+75 for 3977-4095.
|
||||
|
||||
## Verification
|
||||
|
||||
Method: LibRaw master (commit `d1f0dd95`, 2026-10-08, run as `unprocessed_raw`, 16-bit output before any processing) was used only as a black-box oracle: it was built and run, and its source was never read. A Python reference model of the format (the Phase 0 reference model, local, not in the repo) was written from RDD 34 and from the oracle's behaviour on real and synthetic input, then reproduced the oracle exactly on every tile of the 8 Compressed / HQ files below. A test-only encoder, validated byte for byte against real tiles, probed the oracle with synthetic coefficients (dequantiser, phases, edge mapping). The Rust decoder reproduced the Phase 0 reference sums on all 8 files. Its output then went through a table read off the oracle's output; that table was strictly increasing, so these exact matches prove the 12-bit codes before the curve identical to the oracle's. The table has since been replaced by our own measured curve (*Output range*), so the output values now differ from the oracle's by the amounts in *Verification against the oracle*.
|
||||
|
||||
Geometry was checked against each file's embedded full-resolution JPEG at the DefaultCropOrigin: our output aligns at offset 0 on full frame and APS-C.
|
||||
|
||||
### LibRaw's APS-C bug
|
||||
|
||||
LibRaw assumes the full-frame grid (s = 0) for every tile. On the APS-C crop that drops the first three plane rows, leaves the last three unfed, and shifts its whole output up by 6 sensor rows; its first rows and last rows are also wrong. Our decoder differs from LibRaw there on purpose. To compare against LibRaw on APS-C files, our row y+6 equals LibRaw's row y for 12 <= y < h - 24 (LibRaw's rows 0-11 and the last 18 rows of the shifted overlap differ); on full frame compare directly: the 12-bit codes match over the whole frame.
|
||||
|
||||
### Corpus
|
||||
|
||||
| File | Source path (RawDB, ILCE-7RM6) | Size (bytes) | SHA-256 | Fetched by |
|
||||
|---|---|---|---|---|
|
||||
| `arw6-sony-ilce7rm6-apsc-land-hq.arw` | `arw6/ILCE-7RM6_APSC_land_RAW-C-HD.ARW` | 29 720 576 | `b60a7e752d2a8102ccbe6eebcfdbf533c6b0f2545739090873e9e8eb32cc0637` | `cargo xtask corpus --download` |
|
||||
| `arw6-sony-ilce7rm6-apsc-land-c.arw` | `arw6/ILCE-7RM6_APSC_land_RAW-C.ARW` | 30 900 224 | `1c803304e63ec508b2b5f9d59d4a7e30ed279315b35fa4c5d75f959bed62198b` | `curl -L` |
|
||||
| `arw6-sony-ilce7rm6-apsc-port-hq.arw` | `arw6/ILCE-7RM6_APSC_port_RAW-C-HD.ARW` | 28 758 016 | `0b6d782571e86868670f613de9fa1f1364623b24fbfd21b704b2c516d7a1c1cb` | `curl -L` |
|
||||
| `arw6-sony-ilce7rm6-apsc-port-c.arw` | `arw6/ILCE-7RM6_APSC_port_RAW-C.ARW` | 29 044 736 | `c41674a485ffcf6d873e20e545ae62936192d94e95be3d53548e93bc543ecfd4` | `curl -L` |
|
||||
| `arw6-sony-ilce7rm6-ff-land-hq.arw` | `arw6/ILCE-7RM6_FF_land_RAW-C-HD.ARW` | 63 451 136 | `559fa26501d7813e56afecde2aea3d9965a99228162b194752d986193abb172c` | `curl -L` |
|
||||
| `arw6-sony-ilce7rm6-ff-land-c.arw` | `arw6/ILCE-7RM6_FF_land_RAW-C.ARW` | 63 844 352 | `d900d6cbe838be864269ad54de4a33fcac7eb3a3d4f91009739f770b8f85a23b` | `cargo xtask corpus --download` |
|
||||
| `arw6-sony-ilce7rm6-ff-port-hq.arw` | `arw6/ILCE-7RM6_FF_port_RAW-C-HD.ARW` | 62 640 128 | `81c709633cb8f6340e612c3f53bb6ec6b0cccfe5b713c67efb1e977ece9e266d` | `curl -L` |
|
||||
| `arw6-sony-ilce7rm6-ff-port-c.arw` | `arw6/ILCE-7RM6_FF_port_RAW-C.ARW` | 63 168 512 | `f4448485629c3f82ff1955fd046404350e9a1f09b346038574b89c074d7c50fd` | `curl -L` |
|
||||
| `arw-sony-ilce7rm6-apsc-land-lossless.arw` | `raw_modes/ILCE-7RM6_APSC_land_RAW.ARW` | 44 167 168 | `7aaf0b3941509d7364a6f03844783155ebc912d246d1c456e5cdb4693ec09fbc` | `curl -L` |
|
||||
| `arw-sony-ilce7rm6-apsc-port-lossless.arw` | `raw_modes/ILCE-7RM6_APSC_port_RAW.ARW` | 43 692 032 | `0f011933ee96525a39710daa50b59d39702b4ec5934fa0206742e2128c55f366` | `curl -L` |
|
||||
| `arw-sony-ilce7rm6-ff-land-lossless.arw` | `raw_modes/ILCE-7RM6_FF_land_RAW.ARW` | 100 851 712 | `5cb3c870bc961cf6b079d56c860d0cdf11acd1afd04e5fb8dbe041bcfd2d6201` | `curl -L` |
|
||||
| `arw-sony-ilce7rm6-ff-port-lossless.arw` | `raw_modes/ILCE-7RM6_FF_port_RAW.ARW` | 100 884 480 | `b1a4026aa6adbaa27973b8b3426d8b4aa0c1cae1845cf5102359327d3b069691` | `curl -L` |
|
||||
|
||||
These 12 files are CC0 1.0. RawDB's record of the set, <https://rawdb.dnglab.org/api/sets/Sony/ILCE-7RM6> (the data behind <https://rawdb.dnglab.org/sets/Sony/ILCE-7RM6>, accessed 2026-10-10), says `"license":"CC0 1.0"` for the set (uploaded by abbradar, 2026-08-29) and again for each of the 12 files, next to the SHA-256 values in the table.
|
||||
|
||||
They live in the gitignored corpus. `cargo xtask corpus --download` fetches two of them, one per geometry and compression mode (full frame Compressed, APS-C HQ), and checks each against its SHA-256, pinned in `RAW_SAMPLES` (`xtask/src/main.rs`). Fetch the other ten by hand: the server answers 303 to a signed object URL, so `curl -L -o corpus/raw/<file> https://rawdb.dnglab.org/api/download/Sony/ILCE-7RM6/<source path>`, then compare with the SHA-256 above. `cargo test -p lightcraft-raw corpus_sony -- --nocapture` checks the decoder's sensor sum and position-weighted sum of the two pinned files and skips absent files. The sums are regression values from LightCraft's decoder with the derived curve; the 12-bit codes behind them were verified against the oracle in Phase 0.
|
||||
|
||||
Nine further full-frame HQ photos from the maintainer's A7R VI were compared with the oracle locally; they are private and never committed or published. On all 9 (ISO 100-12800, incl. two near-black night shots) the 12-bit codes matched the oracle over the full frame (0 mismatching pixels of 66.8 MP each).
|
||||
|
||||
Unconditional synthetic tests (encode -> decode round trips for both geometries, the RDD 34 worked example, the dequantiser, hostile and truncated input) run without the corpus.
|
||||
|
||||
## Coverage
|
||||
|
||||
| Coverage | Status |
|
||||
|---|---|
|
||||
| ILCE-7RM6 Compressed and HQ, full frame, landscape and portrait | 4 files, 12-bit codes exact against the oracle; output through our measured curve |
|
||||
| ILCE-7RM6 Compressed and HQ, APS-C crop, landscape and portrait | 4 files, 12-bit codes exact against the Phase 0 model (interior-exact against LibRaw after the 6-row shift; LibRaw's own edge rows differ) |
|
||||
| ILCE-7RM6 lossless (existing LJ92 path) | 4 files decode; not in the corpus test |
|
||||
| Private A7R VI HQ photos | 9 files, 12-bit codes exact over the full frame against the oracle (local) |
|
||||
|
||||
## Not covered
|
||||
|
||||
- The 12-bit Compressed mode (electronic shutter with continuous drive): no sample exists, so it returns `Unsupported` unless its headers prove identical.
|
||||
- The ILCE-7M5: same codec, no sample; unverified.
|
||||
- Tiles whose plane width is not a multiple of 8.
|
||||
- First-half shifts other than 0 and 3 (other crops / bodies): the geometry is generic and round-trip tested, but no real sample exists.
|
||||
- Memory: one FF decode peaks at about 896 MB RSS (about 6.7 x the 134 MB output; the lossless path peaks at 249 MB). Follow-ups: dequantise straight into bands, place tiles as they arrive.
|
||||
- Colour: the ILCE-7RM6 bundled colour profile and Sony's embedded lens corrections are separate work.
|
||||
|
||||
## Sources
|
||||
|
||||
- SMPTE RDD 34:2015 "LLVC (Low Latency Video Codec) for Network Transfer", <https://pub.smpte.org/latest/rdd34/rdd34-2015.pdf> (accessed 2026-10-09): picture / TU structure, VLD, inverse quantisation, 5/3 lifting with symmetric extension.
|
||||
- LibRaw master `d1f0dd95`, run only as a black-box oracle (not a product, test or build dependency) for the 12-bit codes; the companding curve was measured without it and compared with it only afterwards.
|
||||
- The embedded JPEG of each file, for the geometry check.
|
||||
- Not read, by licence: LibRaw, rawspeed, dnglab, RawTherapee and other ARW6 decoder sources. Exact byte layouts, deviations and geometry were established through sample analysis and black-box comparison; the curve was measured from the CC0 lossless / ARW6 pairs.
|
||||
|
||||
## Timings
|
||||
|
||||
Raw decode only, 4 CPUs, release build, best of 3 wall-clock (2026-10-10). Method: a small throwaway Rust program linking `lightcraft-raw` reads the file into memory, then times `lightcraft_raw::decode` alone with `std::time::Instant` (no render, no JPEG encode), three decodes in one process; the best is reported. Files were already in the page cache.
|
||||
|
||||
| File | Size | Sensor | Decode |
|
||||
|---|---|---|---|
|
||||
| ILCE-7RM6 FF, Compressed (HQ) | 63.5 MB | 10016x6672 | 0.45 s |
|
||||
| ILCE-7RM6 FF, Compressed | 63.8 MB | 10016x6672 | 0.51 s |
|
||||
| ILCE-7RM6 APS-C, Compressed (HQ) | 29.7 MB | 6592x4372 | 0.35 s |
|
||||
| ILCE-7RM6 APS-C, Compressed | 30.9 MB | 6592x4372 | 0.34 s |
|
||||
|
||||
The 1.0 s budget for the full-frame file is met. Parallelised: rayon over the tiles, over the 13 streams of each tile and over its 3 components.
|
||||
@@ -16,15 +16,21 @@ Matrix-only and matrix + table looks are each completed (tone curve, then chroma
|
||||
|
||||
The matrix is applied to scene-linear sensor pixels. The curve is carried in `SourceInfo` and applied in the pipeline finish stage, after exposure/WB/local edits. It is never baked into decoded luminance or clipped to JPEG precision. CPU and GPU use the same tone LUT. Contrast, whites and blacks adjust the starting look through the display-domain controls. JPEG neutral rendering and the existing DNG colour model are unchanged.
|
||||
|
||||
One third of the pixels are held out from both fits. The combined model must reduce linear-display squared error by at least 30% and have per-channel RMS error ≤0.10 (0.055 rejected half of the public raw.pixls.us samples tried — ILCE-6400, -6700, -7M4 — whose fits were still 1.5–3× closer to the camera JPEG than the fallback and visibly better). When the fit on all pixels fails these gates, it is tried once more, with the same gates, on the pixels away from edges (where the 3×3 neighbourhood in either proxy spans a luminance ratio above 3). Many camera JPEGs are lens-corrected while the raw is not (Sony "Distortion Comp.: Auto", compacts and kit zooms), and on a detailed scene the misaligned edges alone fail the gate although the colours agree everywhere else (issue #232). The public ILCE-7RM2 samples (raw.pixls.us, one façade shot in seven compression/correction variants, 24–70 mm at 51 mm) are up to 15 px of 1440 off their JPEG; the six with distortion correction on missed the gate at held-out RMS 0.108–0.115 and opened in the grey neutral fallback, 13 L* too dark (ΔE76 16.6–16.9 against the JPEG after registering its geometry). Away from edges they pass at 0.049–0.052 and land at ΔE 3.7–3.8 with lightness within 0.6 L*, like the variant shot with distortion correction off (0.059 on all pixels, ΔE 3.9). The corpus GX80 kit-zoom sample (up to 63 px of 1440 off) passes too: 0.107 → 0.081, ΔE 19.1 → 7.9. Fits accepted on all pixels are unchanged. These are provisional acceptance gates, not a colour-accuracy certification. Monochrome, singular, unrelated and aspect-mismatched references retain the documented fallback; poor-fitting ones get the partial look below when it is clearly closer to the camera JPEG. Both fits use the same fixed proxy at every decode resolution.
|
||||
One third of the pixels are held out from both fits. The combined model must reduce linear-display squared error by at least 30% and have per-channel RMS error ≤0.10 (0.055 rejected half of the public raw.pixls.us samples tried — ILCE-6400, -6700, -7M4 — whose fits were still 1.5–3× closer to the camera JPEG than the fallback and visibly better). When the fit on all pixels fails these gates, it is tried once more, with the same gates, on the pixels away from edges (where the 3×3 neighbourhood in either proxy spans a luminance ratio above 3). Many camera JPEGs are lens-corrected while the raw is not (Sony "Distortion Comp.: Auto", compacts and kit zooms), and on a detailed scene the misaligned edges alone fail the gate although the colours agree everywhere else (issue #232). The public ILCE-7RM2 samples (raw.pixls.us, one façade shot in seven compression/correction variants, 24–70 mm at 51 mm) are up to 15 px of 1440 off their JPEG; the six with distortion correction on missed the gate at held-out RMS 0.108–0.115 and opened in the grey neutral fallback, 13 L* too dark (ΔE76 16.6–16.9 against the JPEG after registering its geometry). Away from edges they pass at 0.049–0.052 and land at ΔE 3.7–3.8 with lightness within 0.6 L*, like the variant shot with distortion correction off (0.059 on all pixels, ΔE 3.9). The corpus GX80 kit-zoom sample (up to 63 px of 1440 off) passes too: 0.107 → 0.081, ΔE 19.1 → 7.9. Fits accepted on all pixels are unchanged. These are provisional acceptance gates, not a colour-accuracy certification. Singular, unrelated and aspect-mismatched references retain the documented fallback; poor-fitting ones get the partial look below when it is clearly closer to the camera JPEG, and black-and-white ones only their tone curve (below). Both fits use the same fixed proxy at every decode resolution.
|
||||
|
||||
**A partial look when the full look misses the gates.** The neutral fallback (camera RGB taken as sRGB, the default tone map) is far from the camera JPEG. Rendered for every photo on this path in the CC0 archive (1,150 files with a camera JPEG), it is 10.7 L\* too dark (median; about one stop), keeps 0.36 of the camera's colourfulness and has hues 7° off on the coloured pixels (median CIEDE2000 13.6). Its white balance is right: on the pixels the camera renders near-neutral the cast is under 1 a\*/b\* (median, every maker). The "blue" fallback on newer Sony bodies (Δb\* −22 over the whole frame on the ILCE-7RM5 sample) is a warm scene losing its colour, not a cast: its near-neutral pixels are within −1.6 b\*. So before falling back, a partial look is fitted (`fit_partial`): the quantile tone curve (registration-free, the largest part of the error), the chroma curve, and, when the scene has enough colour to learn a matrix (the full look's condition), the photo's own colour matrix at half weight, the other half the identity (`PARTIAL_MATRIX_WEIGHT`). No hue/saturation table. Half weight because, on the held-out proxy pixels of the 23 colourful photos without a look, the full matrix left 2 of them more than 2° further from the camera's hues than the neutral fallback and half weight none (median hue error 9.6° with the generic colour, 5.1° at half weight, 4.2° at full weight; on the 1,080 photos with a look 3.9° and 2.7°). Without a matrix (tone and chroma curve alone, or with one global saturation scale) about a third of all photos ended up more than 2° further from the camera's hues than the fallback, as the added colour makes the generic matrix's hue errors visible. The partial look is tried on all pixels, then away from edges, and used only when it cuts the held-out squared error below 0.7 of the neutral fallback's in linear and in gamma-encoded display values, and when the JPEG's luminance ranks the pixels like the scene's (Spearman ≥ 0.5, `PARTIAL_MIN_RANK_CORRELATION`): a curve matched to the brightness of an unrelated picture also beats the dark fallback pixel for pixel. The accepted full looks have 0.55 or more; three old Coolpix files whose raws decode wrongly have 0.07–0.27 and keep the fallback. There is no limit on the partial look's own error. On the archive (1,783 renders): all 1,087 full looks are byte-identical; of the 55 photos that reach the partial fit, 49 get it (40 on all pixels, 9 away from edges; 13 with the damped matrix, 41 with a chroma curve) and 6 keep the fallback (lens-corrected compacts: RX10, TZ80 1:1, D-LUX 4, G10 16:9, GF2 ×2). On the 49, median CIEDE2000 to the camera JPEG 14.0 → 2.1, ΔL\* −11.5 → +0.6, colourfulness 0.36 → 0.87, hue error 10.1° → 7.1°; every one is closer, by at least 1.3. Most of the others without a look never reach a fit: on 48 files (Nikon D4, Df, D300/D300S, D7000, D5100 and others) the camera JPEG proxy comes out a pixel narrower than the sensor proxy and no pairs are collected (the resize in PR #531 fixes this; with it 46 of them pass the full look's gates, CIEDE2000 17.9 → 2.9).
|
||||
|
||||
**A tone curve from a black-and-white JPEG.** A raw shot in a monochrome picture style (Canon Monochrome, Sony B/W, Nikon Monochrome) records full colour, but its camera JPEG is grey, so neither the full look nor the partial look has colour to fit. Such a photo started from the spectral matrices or the neutral fallback with the default tone curve: on a Canon EOS RP, about 1.3 EV darker and flatter than the same body's photos shot in colour (issue #577). The grey JPEG still shows the camera's tone curve. When no look was fitted and every pixel of the JPEG proxy but one in a thousand has its channels within 0.002 of each other (`monochrome`; a sepia-toned JPEG has colour and isn't one), the quantile tone curve is fitted alone, luminance against luminance, on top of the colour the photo gets otherwise: the model's camera profile when it has one (in `fit_preview`, like the profile's colour looks), else its spectral matrices or the neutral fallback's (`monochrome_look`). The chroma curve stays neutral. It has the partial look's checks: the JPEG must rank the pixels like the scene (Spearman ≥ 0.5, with tied values sharing their rank, so a flat grey JPEG relates to nothing), and the curve must cut the held-out squared luminance error below 0.7 of the default curve's, in linear and gamma-encoded values, on all pixels or away from edges. Sony's DRO is handled as for colour JPEGs, and white balance stays relative to the as-shot look. The check reads pixels, not the picture style: a colour JPEG of a scene so colourless that every pixel is grey within 0.002 counts too (such a photo gets no look today either). Photos whose JPEG has colour render as before.
|
||||
|
||||
## Source metadata and white balance
|
||||
|
||||
Previously `SourceRef::load()` discarded the decoder's `SourceInfo`, and render jobs used header-only facts (including a hard-coded RAW WB). Decoded pixels and source facts now travel and remain cached together, including across web-worker renders. Smart previews keep the tone curve in their header (the matrix is already in their pixels), so an offline original renders and exports with the same look; proxies built before this fall back to the generic curve until rebuilt. Pixel-statistics/colour-sampling commands use decoded facts too. Render cache version 6 invalidates old thumbnails and loupe renders.
|
||||
|
||||
Vendor RGB WB multipliers alone do **not** identify an absolute illuminant without camera calibration. The generic model inferred approximately 6829 K / −127 tint for one Sony sample; changing only temperature retained that spurious tint and caused a pink cast. Uncalibrated ARWs, NEFs, RW2s and RAFs keep the file's vendor WB at decode and expose **relative** WB controls around the as-shot look. 6500/0 is the internal neutral reference of that relative scale, not a claim about the capture's measured Kelvin temperature. Presets, picker/auto and manual controls use this reference consistently. Switching from As Shot to Custom first resolves both controls; changing one cannot inherit stale catalog tint. Reset restores the current as-shot reference. Old As Shot numbers in a catalog do not mark an otherwise unchanged photo as edited; Custom WB remains an edit. Measured/absolute-Kelvin WB remains future camera-calibration work.
|
||||
Vendor RGB WB multipliers alone do **not** identify an absolute illuminant without camera calibration. The generic model inferred approximately 6829 K / −127 tint for one Sony sample; changing only temperature retained that spurious tint and caused a pink cast. Uncalibrated ARWs, NEFs, RW2s and RAFs keep the file's vendor WB at decode and expose **relative** WB controls around the as-shot look. 6500/0 is the internal neutral reference of that relative scale, not a claim about the capture's measured Kelvin temperature. Presets, picker/auto and manual controls use this reference consistently. Switching from As Shot to Custom first resolves both controls; changing one cannot inherit stale catalog tint. Reset restores the current as-shot reference. Old As Shot numbers in a catalog do not mark an otherwise unchanged photo as edited; Custom WB remains an edit.
|
||||
|
||||
**Kelvin through the camera's measured matrices (issue #730).** A raw of a camera in `lightcraft_raw::spectral` (52 bodies: ILCE-7M3, ILCE-7M4, ILCE-7RM3, D850, EOS 5D Mark III…) edits white balance in Kelvin like a DNG, whichever look it got: the matrices read its as-shot multipliers as a temperature and tint (`spectral::as_shot_white_through`, the DNG `AsShotNeutral` reading with the model's `ColorMatrix1/2`) and the Temp slider runs 2000–50000 K with As Shot at that reading. The matrices do nothing else: the colour look stays the file's JPEG fit (or the profile, or the spectral fallback), and the picture is adapted from the as-shot white to the chosen one with Bradford, as every fitted look is — not re-developed in camera space as a DNG is. That keeps one adaptation model per camera whether or not a photo's fit was accepted, and it is what makes the upgrade exact: white balance written on the relative scale before this (no `scale` in the settings' `wb`, `WbScale::Legacy`) renders as the composition of the two Bradford adaptations, which is again one Bradford adaptation (`lightcraft_pipeline::local::legacy_white`), so existing edits are pixel-identical; they are converted to Custom Kelvin values the first time they are edited (`normalize_wb`). The decision is one function on both sides — `lightcraft_catalog::relative_wb_camera(format, camera)` from the catalog's camera string and `files::WbScaleOf` from the file — and a test keeps them in step. A photo imported before this keeps the relative scale's 6500 / 0 as its recorded as-shot white until its file is next decoded for display or reloaded (`Op::SetAsShotWb`, bookkeeping); only the readout waits, not the look. Raws of cameras without measured matrices stay relative.
|
||||
|
||||
What the matrices say is not verified against Lightroom's Kelvin. On the ILCE-7M3 a Color Temp. series reads 7–10 % below Sony's dial through them, smoothly and monotonically, with tint near the locus (issue #730); whether the matrices or the dial are off is open (a grey card under an incandescent bulb would tell, since the Illuminant A matrix is exact there by construction). Public samples: ILCE-7M3 AWB 4268 K / +7, ILCE-7M4 AWB 4344 K / +26, GFX100S on its Daylight preset 4659 K / +9 (Fujifilm's nominal daylight is about 5300 K: the same direction and size of gap).
|
||||
|
||||
Sony's proprietary `ColorMatrix` is not assumed to be a DNG XYZ-to-camera matrix. No Adobe profile/matrix or GPL decoder/calibration source is used. The product remains pure Rust.
|
||||
|
||||
@@ -119,11 +125,14 @@ A raw starts from the first of these that is available (`camera_preview::startin
|
||||
4. the model's spectral matrices, with LightCraft's default tone curve and no JPEG fit;
|
||||
5. the neutral fallback.
|
||||
|
||||
When the file's JPEG is black and white, step 2 fits the tone curve alone with the profile's colour; without a profile, or when that fit fails, 4 and 5 take the tone curve fitted to it (see the black-and-white JPEG paragraph above).
|
||||
|
||||
The spectral matrices only replace the neutral fallback: a photo whose JPEG fit is accepted (with or without a
|
||||
profile) renders exactly as before, and a covered model takes its spectral matrices only when that fit fails the
|
||||
gates or the file has no usable JPEG. White balance stays relative to the as-shot look for every raw without matrices
|
||||
of its own, as before, so the catalog's `Photo::relative_wb` and the controls are unchanged. `lightcraft-cli
|
||||
calibrate` still pools the uncorrected camera colour of covered models, so a profile can be fitted for them.
|
||||
gates or the file has no usable JPEG. White balance of a covered model is in Kelvin through its matrices whichever
|
||||
look it got (above, issue #730); for every other raw without matrices of its own it stays relative to the as-shot
|
||||
look (`Photo::relative_wb`). `lightcraft-cli calibrate` still pools the uncorrected camera colour of covered
|
||||
models, so a profile can be fitted for them.
|
||||
|
||||
No colour-difference measurement of the spectral matrices against the camera JPEG, or against the file-local fit,
|
||||
is recorded here yet. Whether they should come before the file-local fit for these models is open until one is.
|
||||
|
||||
@@ -73,12 +73,20 @@ filmstrip is visible.
|
||||
| `ui.resize` | `{width, height}` | Resize the window |
|
||||
| `ui.zoomFactor` | `{factor}` | Set the interface scale (0.5–3; egui's zoom keys are off, Cmd+= / Cmd+- zoom the photo) |
|
||||
| `ui.screenshot` | `{path?, headless?}` | `{path, width, height}` once the frame (with finished renders) is captured. `headless: true` draws the UI on the CPU (no compositor needed); a windowed capture that gets no frame within 2 s falls back to headless automatically |
|
||||
| `engine.execute {command: "app.export", params}` | export params (see `docs/mcp.md`), plus `preset`, `dir` / `path`, `ids`, `background` | Writes the files and returns `{files}`; with `background: true` (what the Export dialog and menus use) it returns `{background: true, total}` at once and the batch runs on a worker thread — poll `ui.inspect` → `export` |
|
||||
| `engine.execute {command: "app.export", params}` | export params (see `docs/mcp.md`), plus `preset`, `dir` / `path`, `ids`, `background` | Writes the files and returns `{files}` (one `{path, width, height, bytes, sidecars}` per file, plus `gpuFallback` — why that render fell back from the GPU to the CPU — when it did; see `docs/gpu-pipeline.md`); with `background: true` (what the Export dialog and menus use) it returns `{background: true, total}` at once and the batch runs on a worker thread — poll `ui.inspect` → `export` |
|
||||
| `engine.execute {command: "activity.list"}` | — | The long-running tasks in flight (imports, exports, preview builds, downloads, the face scan…), oldest first: `{tasks: [{id, kind, label, done, total, unit, detail, cancellable, cancelling, ageMs}]}`; `total` 0 = not known yet; `unit` is `count`, `bytes` or `percent`; `label` is English (the app shows it translated) |
|
||||
| `engine.execute {command: "activity.cancel"}` | `{id}` or `{all: true}` | Asks the task (or every cancellable one) to stop at its next file or photo → `{cancelled: n}`; an unknown id (it may have just finished) or a task that can't be cancelled is an error |
|
||||
| `ui.render` | `{id?, size?, path?}` | Render a photo (PNG to `path`), `{width, height}` |
|
||||
| `app.quit` | — | Close the app |
|
||||
|
||||
White balance (`develop.set` with `wb.temp` / `wb.tint`, `develop.wb`, `develop.wbPick`) is in Kelvin and tint on
|
||||
raws with colour matrices — in the file (DNG) or measured for the camera model (`lightcraft_raw::spectral`; issue
|
||||
#730) — where As Shot reads the file's as-shot white; rendered files and raws without matrices use a relative scale
|
||||
whose 6500 K / 0 is the as-shot look (the Edit panel shows it as −100..+100). The stored `wb` of such a Kelvin raw
|
||||
says `"scale": "kelvin"`; a partial settings JSON given to `develop.merge` or a preset whose `wb` sets `temp` / `tint`
|
||||
**without** `scale` keeps the relative meaning it always had (6500 = as shot) on every raw, so agents that mean
|
||||
Kelvin on a Kelvin raw pass `"scale": "kelvin"` or use `develop.set` / `develop.wb`.
|
||||
|
||||
Image navigation is also available directly as the UI command `view.navigate`, with
|
||||
`{zoom?: "fit" | "fill" | {"percent": number}, pan?: [x, y]}`. Percentage zoom accepts fractional
|
||||
values greater than 0 and at most 800; pan is the normalized image centre, with coordinates from 0 to 1.
|
||||
@@ -87,6 +95,12 @@ scrolling pans in both axes and respects the operating system's scrolling direct
|
||||
These gestures work in Detail (including editing tools and full-screen preview), Compare and Reference
|
||||
views, and only apply over their image areas. Panning stops at the image edges.
|
||||
|
||||
The interface theme is `view.appearance {mode?: "auto" | "light" | "dark"}` (no mode: the next one,
|
||||
Auto → Light → Dark → Auto, like the top bar's `icon:appearance` button) and `view.theme {theme:
|
||||
"charcoal" | "midnight" | "silver" | "paper"}`, which picks a theme and fixes the mode to its family.
|
||||
Both reply `{mode, darkTheme, lightTheme}`; the same values are `settings.appearanceMode`,
|
||||
`settings.darkTheme` and `settings.lightTheme` in `ui.set`. Auto follows the system (on Linux the
|
||||
XDG desktop portal, watched for changes rather than polled) and is dark when the system says nothing.
|
||||
Mask brushing uses the UI command `tool.brush` through `engine.execute`. With `{}` it activates
|
||||
painting on the current mask. `{"new": true}` creates a separate brush mask; `{"op": "add"}` or
|
||||
`{"op": "subtract"}` appends a brush component to the selected mask. Creating either starts in
|
||||
@@ -114,6 +128,18 @@ are `button:libraryRetry`, `button:libraryChoose`, `button:libraryTemporary` (Co
|
||||
`button:libraryQuit`. A temporary session shows a banner (`indicator:temporarySession`, `button:libraryReopen`) and
|
||||
writes nothing. `app.openLibrary` opening a library ends it.
|
||||
|
||||
### Open Library on a Lightroom catalog
|
||||
|
||||
`app.openLibrary` never makes a LightCraft library inside a Lightroom Classic catalog (issue #891): given the catalog
|
||||
folder (it holds a `*.lrcat`), one of its `*.lrdata` / `*.lrcat-data` folders or `Backups/`, the `.lrcat` itself or
|
||||
its `-wal` / `-shm` / `.lock` files, it keeps the library open now and returns `{opened: false, lightroomCatalog:
|
||||
<the .lrcat found, or null>, path}`; the window shows a dialog whose `button:dialogOk` (Import Lightroom Catalog…, or
|
||||
`ui.dialog.confirm`) runs `file.importLightroom` with that catalog (no catalog found: it asks for one) and
|
||||
`button:dialogCancel` (`ui.dialog.cancel`) does nothing. A folder that is a LightCraft library (`catalog.snap` / `catalog.log`) always
|
||||
opens as one. With no library open (the library problem window) it is an error instead. The engine refuses the same
|
||||
places (`EngineError::LightroomCatalog`, so `lightcraft-cli --library` and `mcp --library` too). `library.importLightroom
|
||||
{status: true}` reports the catalog's `path` after the task ends as well as while it runs.
|
||||
|
||||
## Headless rendering (no window, no GPU)
|
||||
|
||||
The egui UI can be rasterized on the CPU (`crates/ui-egui/src/softpaint.rs`, driven by
|
||||
|
||||
+39
-1
@@ -24,7 +24,7 @@ are the public prose descriptions and patents cited in
|
||||
embedded-JPEG fallback and its visible warning when a JPEG is present. So does
|
||||
any other CR3 decode error (a corrupt file, or a body whose CRX stream the
|
||||
decoder misreads): every CR3 opened from its embedded JPEG before this decoder,
|
||||
and only three bodies are verified, so a CR3 never fails to import or load
|
||||
and only four bodies are verified, so a CR3 never fails to import or load
|
||||
where a preview exists. HEVC-only preview tracks are recognized but are not
|
||||
decoded.
|
||||
|
||||
@@ -55,6 +55,16 @@ control. The exact inputs and reference hashes are:
|
||||
| Canon R8 RAW, raw.pixls.us 6585 | `7d5c6dbb11ff7e6ee58715d90a20f5d801e8b103f4711ff6ea329ceafcef254c` | `63c1c6d8d1eb2312` |
|
||||
| Canon R8 C-RAW, raw.pixls.us 6587 | `df33cf394573645ce03dea1b2e9f0b5cc2b7734e9e3391ee7114151414cf2812` | `f836a663a795a375` |
|
||||
|
||||
The EOS R5 Mark II files (raw.pixls.us 7881–7884, CC0) are pinned to LightCraft's own decode; no external
|
||||
decoder was run on them:
|
||||
|
||||
| File | Input SHA-256 | Sensor FNV-1a (u16 little-endian) |
|
||||
|---|---|---|
|
||||
| Canon R5 Mark II RAW, raw.pixls.us 7881 | `d6e6aa3b493ea8936dff0efb0cc17c87471437e7317bce37c8c82ba08b9cab7b` | `ff716fb30248496f` |
|
||||
| Canon R5 Mark II C-RAW, raw.pixls.us 7882 | `b3c08a5c7a97299dfd58896a12b939e7b43ec78682761812524825e6e5d087be` | `af32b45357b534e5` |
|
||||
| Canon R5 Mark II crop-mode RAW, raw.pixls.us 7883 | `9f88c0ff012924193b7f4497559d133dfe626766d600b5d67689156721739ead` | `31cf630afe868958` |
|
||||
| Canon R5 Mark II crop-mode C-RAW, raw.pixls.us 7884 | `6a597497776a6abbfab8fcf12d0086ac7a3f8afea2879d61accf6bbf48d357d7` | `4721605b20339332` |
|
||||
|
||||
Reference pixels were observed from the DNGLab 0.8.0 macOS arm64 release
|
||||
binary; no decoder source was read or copied, and it is not shipped or linked.
|
||||
The release ZIP SHA-256 was
|
||||
@@ -69,6 +79,34 @@ the nonlinear integer table, then multiplication, division by eight, addition
|
||||
of the subband base and a minimum step of one. QP values 131–167 are verified;
|
||||
unmeasured values are rejected.
|
||||
|
||||
The QP map is coded as follows (established on the EOS R5 Mark II). Its first symbol is a 32-bit
|
||||
big-endian literal, mapped like every other symbol and added to a prediction of zero. Every later
|
||||
symbol is Rice coded with a limited-length escape: a quotient of 23 or more is written as 23 zeros,
|
||||
the terminating 1 and the symbol in 8 bits. This is the escape layout of ITU-T T.87 for a 32-bit
|
||||
code-length limit and 8-bit symbols, without that standard's minus-one offset. In the R8 maps only
|
||||
the first symbol needs more than 15 leading zeros, and its bits read the same whether taken as a
|
||||
32-bit literal or as an escape after 16 zeros with a 16-bit literal (the rule LightCraft used before).
|
||||
The R7 maps decoded to their exact ends under that rule, which a later long prefix would have broken;
|
||||
no R7 file was at hand to recheck them. R5 Mark II maps (QP 150–162) also jump by up to 12 where the Rice parameter is 0. That
|
||||
gives 18- to 22-zero prefixes, which the 16-zero rule misread as escapes ("Canon CRX adaptive QP value
|
||||
−21087" and other out-of-range values; the file then opened as a preview). It also gives real escapes,
|
||||
symbols 23 and 24. Escapes were seen only where the Rice parameter is 0 (in 100 of 612 maps; all 351 inspected, up to 10 per map); at a larger
|
||||
parameter they would use the same layout, which is unobserved. With this coding the QP maps of all
|
||||
33,665 R5 Mark II C-RAW files in a private library (version `0x200`, 8480 × 5650) decode to the exact
|
||||
end of their entropy streams, with values 150–162. So do the reduced-size tracks of 306 of those
|
||||
files, the two CC0 R5 Mark II C-RAW files above and the R8 C-RAW fixture; all six fixture hashes are
|
||||
unchanged. Under the 16-zero rule, 174 of those 306 files (57%) failed and opened as previews, all
|
||||
with out-of-range values (none at QP 168). The two CC0 C-RAW files are low-ISO shots whose maps need no
|
||||
long prefix: they decoded under the old rule too, so the escape is covered by synthetic unit tests in
|
||||
`crx.rs` rather than by a public file. Render cache version 43 redraws thumbnails of files whose decode
|
||||
changed; photos imported as preview-only switch over with Photo ▸ Reload from Disk.
|
||||
|
||||
The R5 Mark II sensor has 5650 rows (3574 in crop mode), so its Bayer planes have odd heights (2825,
|
||||
1787). Lossless planes decode as before. In C-RAW every wavelet level has an odd height (2825 → 1413 →
|
||||
707), and each subband's entropy stream ends exactly where the subband geometry says. The synthesis
|
||||
uses the same boundary rules that are verified on the R8's odd lengths. Decoded rows show no step
|
||||
down to the last sensor row, and the developed images match the camera's embedded JPEGs at 100 %.
|
||||
|
||||
An odd QP-map height (EOS R7: 4732 sensor rows give 1183 QP rows) leaves the
|
||||
last row of the second-level high-frequency bands without a partner row; that
|
||||
row's QP is used on its own. The odd-height map decodes to the exact end of its
|
||||
|
||||
@@ -121,6 +121,51 @@ LC_DENOISE_MODEL=<model.onnx> LC_DENOISE_RAW=<corpus/raw> cargo test --release -
|
||||
LC_DENOISE_MODEL=<model.onnx> LC_DENOISE_RAW=<corpus/raw> cargo test --release -p lightcraft-engine --features rawnind-model --test denoise_real -- --ignored --nocapture
|
||||
```
|
||||
|
||||
## Another computer's graphics card
|
||||
|
||||
A laptop can hand tiles to a desktop with a bigger graphics card over a LAN or a VPN. The server runs the installed,
|
||||
selected model the way the app does (the card when it can run it, checked against the CPU first; else the CPU):
|
||||
|
||||
```text
|
||||
export LIGHTCRAFT_DENOISE_TOKEN='a long random string, at least 16 characters'
|
||||
lightcraft-cli denoise-serve --listen 100.101.102.103:7990 # e.g. the machine's Tailscale address
|
||||
```
|
||||
|
||||
On the laptop, before LightCraft starts:
|
||||
|
||||
```text
|
||||
export LIGHTCRAFT_DENOISE_URL=100.101.102.103:7990
|
||||
export LIGHTCRAFT_DENOISE_TOKEN='…the same token…'
|
||||
```
|
||||
|
||||
When the laptop first uses the model it greets the server and times a check tile there and back. If that is faster
|
||||
than a tile on the laptop (its graphics card, or its processor with the tiles it runs at once), the server's slots
|
||||
(four tiles in flight) are added to the laptop's own: a tile goes to the server when one of its slots is free and runs
|
||||
locally otherwise. If it is slower, nothing is sent, so a photo never waits for the slower machine:
|
||||
`denoise.status` → `device.remote.state` is `slower`, with both times (`remoteMs`, `localMs`). Measured with
|
||||
RawNIND: an Apple M3 laptop does a tile in 116 ms, an RTX 3060 Ti reached over Tailscale 345 ms there and back,
|
||||
so that laptop keeps its tiles; one without a fast card hands them over. A tile the server fails runs locally; after
|
||||
three failures the server is no longer used for that model. Both ends must
|
||||
run the same model file: the client compares the manifest's SHA-256 (or the file's) when it first greets the server,
|
||||
and the server checks it again for every tile. `denoise.status` → `device.remote` reports the server's address, its
|
||||
device and whether it is used, or why not.
|
||||
|
||||
**The protocol is not encrypted.** Listen on a VPN address (Tailscale, WireGuard) or a LAN, or bind every interface
|
||||
behind a firewall rule that admits only the VPN; never expose the port to the internet. Requests carry the shared
|
||||
token (compared in constant time); the server refuses other models, other tile sizes, oversized headers and payloads
|
||||
(also ones that inflate past their tile) and caps concurrent connections (`--max-connections`, default 8).
|
||||
|
||||
Protocol version 1 (`crates/denoise/src/remote.rs`): one JSON line per message, then a payload of `bytes` bytes —
|
||||
the tile's samples as little-endian half floats, byte-shuffled (low bytes, then high bytes) and deflated:
|
||||
|
||||
```text
|
||||
→ {"v":1,"token":"…","op":"hello"}
|
||||
← {"ok":true,"v":1,"device":"GPU: …","model":{"id":"…","version":"…","sha256":"…"},"tile":T}
|
||||
→ {"v":1,"token":"…","op":"run","model":"<sha256>","tile":T,"bytes":N} + N bytes (4 planes of T × T cells)
|
||||
← {"ok":true,"tile":T,"bytes":N} + N bytes (3 planes of 2T × 2T pixels)
|
||||
← {"ok":false,"error":"…"}
|
||||
```
|
||||
|
||||
## Commands and limits
|
||||
|
||||
UI, CLI, control channel and MCP dispatch `denoise.toggle`, `denoise.settings`,
|
||||
|
||||
@@ -3,15 +3,22 @@
|
||||
LightCraft can show photos through the monitor's ICC profile, so colours are right on wide-gamut and calibrated
|
||||
displays. Without one, previews are sRGB, which a wide-gamut panel shows oversaturated.
|
||||
|
||||
**macOS uses the screen's own profile** (the one System Settings ▸ Displays assigns) while none is chosen. macOS doesn't
|
||||
colour-manage LightCraft's window (wgpu leaves its Metal layer without a colour space, which Core Animation shows
|
||||
unmatched), so on a P3 screen previews used to look more saturated than the exported files in Preview or Photos (issue
|
||||
#787). The profile is read again about once a second, so moving the window to another screen is followed.
|
||||
|
||||
## Using it
|
||||
|
||||
Settings ▸ Display ▸ **Choose Profile…** picks an `.icc` / `.icm` file (the dialog opens where the system keeps
|
||||
them: `~/.local/share/icc` on Linux, ColorSync's folders on macOS, `…\spool\drivers\color` on Windows). **Use sRGB**
|
||||
goes back to no profile. The choice is an app setting (`ui.json`, `displayProfile`) and applies at once; a file that
|
||||
can't be used is reported there and previews stay sRGB.
|
||||
(on macOS **Use Screen Profile**) goes back to no profile. The choice is an app setting (`ui.json`, `displayProfile`)
|
||||
and applies at once; a file that can't be used is reported there and previews stay sRGB.
|
||||
|
||||
Agents and scripts: `app.displayProfile {"path": "/path/to/monitor.icc"}` (`""` or `null`: none; no `path`: report
|
||||
only) returns `{path, description, kind: "matrix" | "lut", primaries: {red, green, blue: [x, y]}}`.
|
||||
only) returns `{path, description, kind: "matrix" | "lut", primaries: {red, green, blue: [x, y]}}`; the screen's
|
||||
profile (macOS, none chosen) reports `path: null, screen: true`. Hosts supply it as `Services::screen_profile`
|
||||
(`apps/lightcraft/src/screen_profile.rs`); headless sessions have none and stay sRGB.
|
||||
|
||||
## How it works
|
||||
|
||||
@@ -40,5 +47,5 @@ LUT-based profile's correction about 65 ms. Drafts during slider drags skip the
|
||||
|
||||
## Not yet
|
||||
|
||||
- Picking up the profile the operating system assigns (X11 `_ICC_PROFILE`, colord, ColorSync, WCS) and following
|
||||
the window across monitors with different profiles.
|
||||
- Picking up the profile the operating system assigns on Linux and Windows (X11 `_ICC_PROFILE`, colord, WCS) and
|
||||
following the window across monitors there (macOS does both, above).
|
||||
|
||||
+11
-1
@@ -200,7 +200,8 @@ photos in the set.
|
||||
|
||||
## Suggesting who is in a photo
|
||||
|
||||
With **Recognise faces** switched on in Settings ▸ Faces and a recognition model chosen (**Use**), LightCraft works out, in
|
||||
With **Recognise faces** switched on in Settings ▸ Faces and a recognition model chosen (**Use**; the switch cannot be
|
||||
turned on before one is, and turns itself off when the last recogniser is removed), LightCraft works out, in
|
||||
the background, what each face in your library looks like to the model, and uses the faces you have already named to
|
||||
suggest names for the ones you have not:
|
||||
|
||||
@@ -296,3 +297,12 @@ suggestions it would make and how many were right.
|
||||
All of this is reachable from the control channel, the CLI and MCP: `faces.models.list`, `faces.models.inspect {path}`,
|
||||
`faces.models.install {path, acknowledged: true, activate?}`, `faces.models.download {id, acknowledged: true}` (then `faces.models.downloads`, which also installs what has arrived, and `faces.models.downloadCancel {id}`), `faces.models.remove {id}`, `faces.models.select {id}`,
|
||||
`faces.enable {enabled?}`, `faces.detect {ids?, apply?}`, `faces.index {budgetMs?, ids?}`, `faces.pump` (what the app calls every frame), `faces.suggest {ids?, threshold?, margin?}`, `faces.person {name, more?}` (a person's faces and the unnamed faces that look like them), `faces.unnamed {limit?}` (every unnamed face, look-alikes together, with suggested names), `faces.setName {id?, index, name}`, `faces.nameFaces {faces: [{photo, index}], name}` (name many at once, one undo step) and `faces.evaluate`. `acknowledged` must be `true`: the caller has shown the user the terms and the user agreed. Installing makes the model the one in use and switches recognition on unless `activate` is `false`.
|
||||
|
||||
## Find Blemishes
|
||||
|
||||
Photo ▸ Find Blemishes (`spot.findBlemishes`) uses the installed detector's boxes and landmarks to look for small,
|
||||
round, darker-or-redder spots on each face's skin and adds a heal spot on each, the way Find Dust Spots does for
|
||||
sensor dust. Eyes, brows, nostrils and the mouth are cut out around the landmarks; faces narrower than 160 px in
|
||||
the 1600 px analysis image get nothing, as do faces with more than 12 candidates (stubble, freckles and pores are
|
||||
texture, not blemishes); at most six per face. It needs no model beyond the detector and never changes anything
|
||||
without a spot you can see and delete (`crates/pipeline/src/blemish.rs`).
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
# Fujifilm embedded lens corrections
|
||||
|
||||
LightCraft reads the distortion and vignetting tables that Fujifilm cameras store in the raw IFD of every RAF
|
||||
(`0xf00b` GeometricDistortionParams, `0xf010` VignettingParams, signed rationals) and turns them into the existing
|
||||
`OpcodeList3` pair, `WarpRectilinear` and `FixVignetteRadial`. The profile-correction control, the CPU / GPU optics
|
||||
path, EXIF orientation handling and DNG export use them like any other embedded correction.
|
||||
|
||||
Corrected: **distortion** (with the camera's frame-filling zoom) and **vignetting**, for the 59 bodies listed in
|
||||
`crates/raw/src/vendor/raf_lens.rs` (`VALIDATED`). **Not corrected: lateral chromatic aberration** (the third table,
|
||||
`0xf00f`); see "Chromatic aberration" below. Bodies without a checked sample, and tables in a layout or normalisation
|
||||
that was not understood, stay uncorrected.
|
||||
|
||||
## Clean-room evidence
|
||||
|
||||
The tag names come from the [ExifTool FujiFilm tag-name documentation](https://exiftool.org/TagNames/FujiFilm.html);
|
||||
ExifTool was only run as a black box (`exiftool -j -n`, `-v3`) to print the arrays. Everything else was measured:
|
||||
the 129 CC0 RAFs with these tables under `raw.pixls.us` (Fujifilm, 65 bodies) against the camera's own embedded JPEG, which the
|
||||
cameras correct in-camera, and against LightCraft's uncorrected raw render. No other decoder's code or data,
|
||||
lens database or Adobe profile was consulted. Scripts and per-file results: `data/testing/raf-lens/` (local).
|
||||
|
||||
## Layout and units
|
||||
|
||||
Every table is an array of signed rationals with `n` knots:
|
||||
|
||||
```
|
||||
[scale, knot_1 … knot_n, value_1 … value_n]
|
||||
```
|
||||
|
||||
- `scale × n` is the half diagonal of the recorded image (the RAF crop) in pixels: 3749 for 6240×4160, 4643 for
|
||||
7728×5152, 2942 for 4896×3264, 7280 for 11648×8736, on 61 of the 65 bodies. Files that store another number (X-S20,
|
||||
X-T30 III and X-M5 write the 40 MP value, 4643; an X-T5 in an in-camera crop) are left uncorrected.
|
||||
- Knots are positions along the radius. The **last knot is the farthest corner**: `ρ = knot_last · r`, `r` the
|
||||
distance from the centre in half diagonals. Older bodies use 11 knots `0, 0.1 … 1`, newer ones 9 knots evenly
|
||||
spaced in `r²` (`√(k/8)`, last `√(9/8) = 1.0607`), GFX 50 bodies 8 (`√(k/8)`, last 1). The distortion zero and the 100 %
|
||||
illumination at the axis are anchored at `ρ = 0` when the first knot lies above it.
|
||||
- Distortion values are percent: the bracket `1 + D/100` multiplies the radius.
|
||||
- Vignetting values are percent of the axial illumination at the (source) radius.
|
||||
- The chromatic-aberration table has the layout `[scale, knots, red × n, blue × n, scale]` (fractions of the radius,
|
||||
the displacement of the red and blue planes relative to green).
|
||||
|
||||
## Geometry
|
||||
|
||||
The camera's JPEG samples the raw at
|
||||
|
||||
```
|
||||
source = z · r · (1 + D(ρ(source)) / 100) ρ = knot_last · source
|
||||
z = 1 / max over the output border radii of (1 + D(ρ(source)) / 100)
|
||||
```
|
||||
|
||||
The table is indexed by the **source** (sensor) radius, not the output radius, and the zoom `z` keeps the output
|
||||
rectangle inside the source (as for the Sony table, the bracket is largest at the nearest border radius). Source
|
||||
radius is solved per radius by fixed-point iteration, then `source(r) = r (k0 + k1 r² + k2 r⁴ + k3 r⁶)` is fitted
|
||||
(minimax, 1025 samples) for the DNG warp; a fit that strays more than 0.0008 half diagonals (largest over the 118 sample
|
||||
tables 0.0007, median 0.00012) or folds is rejected. The correction itself is the warp's bracket; the camera's rounding
|
||||
and interpolation between knots are not reproduced beyond linear interpolation between knots.
|
||||
|
||||
Vignetting is the gain `100 / V(ρ(r))` at the source radius, fitted by a 5-term minimax polynomial in `r²`
|
||||
(`FixVignetteRadial`); a fit more than 0.025 off the table is rejected.
|
||||
|
||||
## Validation
|
||||
|
||||
Method: LightCraft's uncorrected render of each RAF (1200 px long edge) against the camera JPEG (`camera-jpeg/`,
|
||||
resized to the same size). Per file the high-passed luminance is correlated (NCC) after a warp with a free global scale and
|
||||
shift; the warp is (a) similarity only, (b) the table with its sign flipped, (c) the table, lookup at the output
|
||||
radius, (d) the table, lookup at the source radius with the computed zoom (adopted), (e) a free three-term radial
|
||||
polynomial (the upper bound). 67 body/lens/focal-length combinations from 65 bodies, 26 named lenses and the fixed-lens cameras (129 files).
|
||||
|
||||
| Warp | median NCC | mean NCC |
|
||||
|---|---:|---:|
|
||||
| similarity only (nothing corrected) | 0.941 | 0.865 |
|
||||
| table, opposite sign | 0.493 | 0.530 |
|
||||
| table, lookup at the output radius | 0.989 | 0.869 |
|
||||
| **table, lookup at the source radius, computed zoom (adopted)** | **0.991** | 0.872 |
|
||||
| free radial polynomial | 0.994 | 0.985 |
|
||||
|
||||
The sign is determined (the 48 combinations with more than 1 % distortion reach a median of 0.29, at most 0.91, with the wrong sign), the
|
||||
source-radius lookup is slightly better where distortion is large (GFX100RF 0.985 vs 0.971, X-S20 0.989 vs 0.971, X-S1
|
||||
0.977 vs 0.961). The table reaches the free fit's correlation within 0.02 for 51 of 67 combinations including the
|
||||
computed zoom, and for 60 of 67 when the zoom is free (shape right). The zoom `z` agrees with the free fit's scale
|
||||
within 0.0047 on all 51.
|
||||
|
||||
**Where the framing differs:** on five corrected combinations the camera did not zoom (XC16–50 at 16 mm on the X-A10,
|
||||
XC15–45 on the X-A5 and X-T100, the F770EXR and HS50EXR) and on two more it zoomed about 1 % differently (X-T30 III, XQ2):
|
||||
the shape matches but the JPEG shows 4 % more of the sensor than the computed zoom does. The same body and table
|
||||
zoom on other lenses (the X-A1 and X-A2 with the XC16–50), so the rule is not in the table. LightCraft uses the computed zoom
|
||||
everywhere (no empty border, as for the Sony table). Six bodies whose shape did not match well enough (F550EXR, S1,
|
||||
SL1000, X10, XF1, XQ1) are not corrected.
|
||||
|
||||
**Through the shipped code path** (`lightcraft-cli render` of the same 67 combinations with the embedded
|
||||
correction applied, then the free fit of the corrected render against the camera JPEG, global scale removed so
|
||||
the framing difference does not count): the residual radial shape error falls from a median of
|
||||
0.72 % of the half diagonal (largest 2.60 %) to a median of 0.05 % (largest 0.42 %), 54 of the 57 corrected combinations land within 0.25 %
|
||||
(2 px at 1440 px), and the similarity-only correlation rises from a median of 0.957 to 0.993. Per-lens rows are in
|
||||
`data/testing/raf-lens/out/after_table.md`.
|
||||
|
||||
### Vignetting
|
||||
|
||||
Brightness of the camera JPEG against the aligned uncorrected render, in linear light, modelled per file as
|
||||
`log Y_jpeg = poly3(log Y_render) + β(radius bin)` (the polynomial absorbs the tone curve): the JPEG's gain β against
|
||||
`-ln(V/100)` of the table over 42 combinations whose corner gain exceeds 5 %: correlation median 0.987 (37 of 42 above
|
||||
0.9), proportionality factor 0.98 (interquartile 0.73–1.31), against the tone curve's own slope of about 1.05; flat
|
||||
tables (X-M5, X-T2, X-T3) give a gain of zero in the JPEG. Through the shipped code path the mean |β| over the outer three radius bins of the corrected render falls from a median of
|
||||
0.184 to 0.031 (log units; largest 0.81 to 0.15) on the 38 corrected combinations with a corner gain above 5 %, and the corner
|
||||
β at least halves on 30 of them. Seven combinations show only a fraction of the amplitude
|
||||
(S1, SL1000, X-A2, X-A3, X-S1, X-T1, X-Pro3); the camera setting that governs them is not recorded.
|
||||
|
||||
### Chromatic aberration (not applied)
|
||||
|
||||
The CA table is real: on the raw mosaics themselves (uncompressed RAFs, colour planes built by normalised convolution so
|
||||
no demosaic is involved) the radial shift of the red and blue sites against green correlates with the table over 46
|
||||
curves from 30 files at a median 0.93 (31 above 0.8), with the table's sign, at about 0.7 of its amplitude (the same
|
||||
estimator reads 1.0–1.15 on a synthetic mosaic with the table applied). The camera JPEGs show no measurable red/blue
|
||||
shift, but 4:2:0 chroma shares the luminance edges, so that proves nothing.
|
||||
|
||||
LightCraft's own demosaic, however, already registers the colour planes: its render shows red/blue shifts of only
|
||||
0.05–0.2 px where the table has 1.5–5 px. Applying the table as per-channel warp planes (tried; the opcode supports
|
||||
them) shifted the planes the wrong way: the median measured shift rose from 0.10 to 0.13 px and grew by more than 1.5×
|
||||
on 11 of 28 curves (X-T4 blue 0.16 → 0.58 px, X-T3 red 0.06 → 0.59 px). The table is therefore not applied; a correction
|
||||
before demosaicing would be needed.
|
||||
|
||||
## Tests
|
||||
|
||||
- `crates/raw/src/vendor/raf_lens.rs`: layout, normalisation, the model list, malformed tables, the zoom and the
|
||||
vignette gain on the X-T4's real arrays, and the X-T4 geometry against its measured free fit.
|
||||
- `crates/raw/src/vendor/raf.rs`: a synthetic RAF through header and full decode, only for validated models, and a DNG
|
||||
round trip.
|
||||
- `crates/raw/tests/corpus.rs` `fuji_embedded_lens_tables_follow_the_validated_models`: the corpus' `raf-fuji-*` files,
|
||||
or any folder of RAFs in `LIGHTCRAFT_FUJI` (the pixls set: 129 decoded, 88 warps, 98 vignette gains).
|
||||
+16
-10
@@ -1,6 +1,6 @@
|
||||
# Gaps: where LightCraft falls short of Lightroom
|
||||
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (created; absorbs the checklist's Top gaps and the ROADMAP's honest assessment, re-ranked after a full re-measure) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
|
||||
> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-11 · **Change:** minor (CR3 entry names the EOS R7 C-RAW fix and the refused CRX variants; CI and Windows / Linux entries note the 0.6.0 riscv64 build; camera colour: tone curve from black-and-white camera JPEGs; Compare and Reference get sharp zoom windows; gap 4 shrinks; gap 14: the HDR switch is read from crs XMP; gaps 2 and 10: EOS R5 Mark II CR3 verified) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
|
||||
|
||||
Every known shortfall, one entry each, ranked by user impact (how many Lightroom users hit it, and
|
||||
how hard it blocks them), then by effort. **Agents pick work from the top** (the first entry nobody is
|
||||
@@ -15,20 +15,20 @@ Hours are Opus 5.5 agent wall-clock hours (see [target-app-parity.md](target-app
|
||||
|
||||
| # | Gap | Evidence | User impact | Hours | Doc |
|
||||
|---:|---|---|---|---:|---|
|
||||
| 1 B | **Camera colour is estimated, not measured** (LR-PROF-CAMERACOLOR). Raws start from a per-file fit to the camera JPEG, 4 bundled profiles or 52 spectral matrices; others use a neutral matrix | ΔE vs camera JPEG 2.9–10 after fitting (raw-parity.md); 0 chart-calibrated models vs 1,500 Adobe Standard profiles | every raw shooter sees colour that is close but not Lightroom's; some get neutral-matrix colour | 50–90 | [raw-parity](raw-parity.md#colour) |
|
||||
| 2 B | **CR3 beyond three verified bodies** (LR-IMP-FORMATS): CRX variants other than those verified on M50 / R100 / R8 fall back to the preview; CR3 sRAW / mRAW and HEIF CR3 too | measured: Canon is the largest bucket (33% of photographers) and its share whose camera fully works is the lowest of the big four, 72%; 17 of 22 popular Canon bodies write CR3, 2 verified (raw-parity.md) | +6.5 points of photographers whose camera works, the biggest single raw gain | 25–45 | [raw-parity](raw-parity.md#remaining-work-ranked-by-photographers-unlocked-per-hour) |
|
||||
| 1 B | **Camera colour is estimated, not measured** (LR-PROF-CAMERACOLOR). Raws start from a per-file fit to the camera JPEG, 4 bundled profiles or 52 spectral matrices; others use a neutral matrix (a black-and-white camera JPEG gives its tone curve only, #577) | ΔE vs camera JPEG 2.9–10 after fitting (raw-parity.md); 0 chart-calibrated models vs 1,500 Adobe Standard profiles | every raw shooter sees colour that is close but not Lightroom's; some get neutral-matrix colour | 50–90 | [raw-parity](raw-parity.md#colour) |
|
||||
| 2 B | **CR3 beyond four verified bodies** (LR-IMP-FORMATS): `crx.rs` refuses vertical-tile C-RAW, partial-subband and other plane coding, multi-tile 0x200 and adaptive low-frequency gains, and those files fall back to the preview; CR3 sRAW / mRAW and HEIF CR3 too. Verified: M50 / R100 / R8 / R5 Mark II. EOS R7 C-RAW (odd-height QP map) decodes since a45c2b66 and R6 Mark II C-RAW is reported working on 0.6.0, but neither is verified yet (no corpus test or recorded check) | measured: Canon is the largest bucket (33% of photographers) and its share whose camera fully works is the lowest of the big four, 73%; 18 of 22 popular Canon bodies write CR3, 4 verified (raw-parity.md); R7 rule checked only on paired RAW / C-RAW captures ([cr3.md](cr3.md)) | +6.2 points of photographers whose camera works, the biggest single raw gain | 25–45 | [raw-parity](raw-parity.md#remaining-work-ranked-by-photographers-unlocked-per-hour) |
|
||||
| 3 B | **Render fidelity unmeasured** (LR-BEHAV-RENDER-FIDELITY): tone, highlights, texture / clarity / dehaze, NR and sharpening are tuned by eye | no side-by-side suite; a retune needs a new process version (only V1 exists) | edits imported from Lightroom look different; the look "isn't Lightroom's" | 30–60 | [raw-parity](raw-parity.md), [process-versions](process-versions.md) |
|
||||
| 4 B | **Original-pixel detail at 100%** (LR-VIEW-ZOOM, P0): the whole-image preview is capped (2,560 px default); compare, overlays and soft proofing lack sharp zoom windows (#323) | checklist row 🟡 | checking focus, the first thing culling needs, isn't reliable on large files | 8–15 | [ui-parity](ui-parity.md) |
|
||||
| 4 B | **Original-pixel detail at 100%** (LR-VIEW-ZOOM, P0): Detail, Before/After, Compare and Reference are sharp at 1:1–8:1 (zoom windows, #323). Mask overlays and soft proofing still render the whole frame at the drawn size instead of a window: sharp up to the GPU texture side, but a full-size render per edit at 1:1 | checklist row 🟡 | editing with a mask overlay or soft proof on at 1:1 is slow on large files, and blurry past the texture side | 4–8 | [ui-parity](ui-parity.md) |
|
||||
| 5 B | **AI masks: Subject, Sky, People, Background, Landscape** are heuristics; Object / Describe run SAM 3 but its download has no mirror configured | `crates/segment`; ai-masks.md; MASK rows 67% | masking is how most Lightroom users edit today | 40–70 + model decision | [target-app-parity](target-app-parity.md#ai) |
|
||||
| 6 B | **Compressed ORF** (every Olympus / OM body since ~2008) | measured: 66 of 78 Olympus / OM models preview-only; `orf.rs` refuses; OM bucket 3% of photographers, 3% of them fully work | +2.6 points; best gain per hour of any raw gap (~0.2) | 8–15 | [raw-parity](raw-parity.md) |
|
||||
| 6 B | **Compressed ORF** (every Olympus / OM body since ~2008) | measured: 66 of 78 Olympus / OM models preview-only; `orf.rs` refuses; OM bucket 3% of photographers, 3% of them fully work | +2.6 points; best gain per hour of any raw gap (~0.2) | 8–15 | [raw-parity](raw-parity.md#compressed-orf-a-clean-room-starting-point) (the two Olympus patents that describe the method) |
|
||||
| 7 | **Nikon High Efficiency NEF** (HE / HE★ on Z 8, Z 9, Z 6III, Z f, Z50II, Z5II when set to HE) | named and preview only (#193); needs a pure-Rust JPEG XS decoder | +1.1 points (only shots taken in HE; lossless works on the same bodies); low gain per hour (~0.04), so not a beta blocker | 20–40 | [raw-parity](raw-parity.md) |
|
||||
| 8 B | **No lens profile database** (LR-EDIT-OPTICS-PROFILE): only DNG opcodes, RW2 in-file distortion and Sony ILCE-7RM4A distortion | Adobe ships 3,635 lens profiles; we never use them | distortion / vignetting correction is a default step for many | 40–80 + calibration shots | [raw-parity](raw-parity.md#lens-profiles) |
|
||||
| 9 B | **Tests don't run in CI on macOS, Windows or Linux x64**: only FreeBSD runs `cargo test`; the others build packages | `.github/workflows/` (freebsd, packaging-lint, release, windows-arm64) | regressions reach users on the main platforms; `cargo xtask ci` runs only locally | 6–12 | [target-app-parity](target-app-parity.md#by-dimension) |
|
||||
| 10 B | **Per-model verification thin, plus cheap per-body decode fixes**: 234 of 1,446 Adobe models verified (16%), ~40% of photographers on a verified body; Nikon 5 bodies verified, Canon 11. Cheap fixes: Z 6II packed 14-bit NEF (#769, PR #671), Sony A100 / F828 regression (#702), DSLR-A200…A390 / A850 12-bit ARW (#535) | measured: [raw/coverage.tsv](raw/coverage.tsv) | per-body bugs (levels, crops, WB) surface only from users | 25–45 | [raw-parity](raw-parity.md#by-maker-measured) |
|
||||
| 9 B | **Tests don't run in CI on macOS, Windows or Linux x64**: only FreeBSD runs `cargo test`; the others build packages | `.github/workflows/` (freebsd, packaging-lint, release, windows-arm64); the linux-riscv64 release job added in 0.6.0 (#844) runs only `lightcraft-cli --version` under QEMU | regressions reach users on the main platforms; `cargo xtask ci` runs only locally | 6–12 | [target-app-parity](target-app-parity.md#by-dimension) |
|
||||
| 10 B | **Per-model verification thin, plus cheap per-body decode fixes**: 235 of 1,446 Adobe models verified (16%), ~42% of photographers on a verified body; Nikon 5 bodies verified, Canon 12. Cheap fixes: Z 6II packed 14-bit NEF (#769, PR #671), Sony A100 / F828 regression (#702), DSLR-A200…A390 / A850 12-bit ARW (#535) | measured: [raw/coverage.tsv](raw/coverage.tsv) | per-body bugs (levels, crops, WB) surface only from users | 25–45 | [raw-parity](raw-parity.md#by-maker-measured) |
|
||||
| 11 | **Enhance: Raw Details, Super Resolution; AI denoise without shippable weights or X-Trans** | ENH rows 0%; denoise Bayer only, RawNIND weights GPL and opt-in | Fujifilm users get no AI denoise; nobody gets it out of the box | 30–60 + model decision | [raw-parity](raw-parity.md#demosaic-levels-crops-and-lens-metadata) |
|
||||
| 12 | **Content-aware / generative remove, detect distractions, reflections removal** | REM rows 75%; patch synthesis missing | cleanup work goes back to Photoshop | 25–45 (+ generative model) | [target-app-parity](target-app-parity.md#ai) |
|
||||
| 13 | **Lightroom catalog migration is one-way and approximate**: unsupported Adobe profiles / AI settings archived, history kept as data | lightroom-catalog-import.md | switching users lose some looks | 8–12 | [file-format-parity](file-format-parity.md) |
|
||||
| 14 | **HDR display** (EDR / Windows HDR); HLG; HDR settings from `crs` XMP | HDR rows 70%; LR-VIEW-HDR-DISPLAY ⬜ | HDR editing works blind on HDR monitors | 12–25 | [hardware-parity](hardware-parity.md) |
|
||||
| 14 | **HDR display** (EDR / Windows HDR); HLG; the headroom limit and SDR rendition from `crs` XMP (the HDR switch, `HDREditMode`, is read) | HDR rows 70%; LR-VIEW-HDR-DISPLAY ⬜ | HDR editing works blind on HDR monitors | 12–25 | [hardware-parity](hardware-parity.md) |
|
||||
| 15 | **Print module**: custom packages, printer colour management (contact-sheet PDF only) | LRC-PRINT rows | Classic users who print | 25–40 | [target-app-parity](target-app-parity.md#features) |
|
||||
| 16 | **Map view, Book, Slideshow module, publish services** | LRC-MAP / BOOK / SS rows ⬜ | Classic users; publish services matter for Flickr / SmugMug workflows | 60–100 | [target-app-parity](target-app-parity.md#features) |
|
||||
| 17 | **Tethered capture** | ⬜ | studio photographers | 25–45 + cameras | [hardware-parity](hardware-parity.md) |
|
||||
@@ -41,7 +41,7 @@ Hours are Opus 5.5 agent wall-clock hours (see [target-app-parity.md](target-app
|
||||
| 24 | **Brush live preview and masking feel** (#517, #518); Feather / Edge mask refinement (15.5) | issues | masking feels slower than Lightroom | 8–12 | [ui-parity](ui-parity.md) |
|
||||
| 25 | **File writes**: JXL encode, PSD export, lossy DNG, Render to DNG; AVIF import | file-format-parity.md | interchange edge cases | 25–45 | [file-format-parity](file-format-parity.md) |
|
||||
| 26 | **Performance at scale**: 85k-photo library opens in 1.7–4.7 s; 100k grid test pending; colour NR at half resolution | ROADMAP log, gpu-pipeline.md | large-catalog users | 15–30 | [target-app-parity](target-app-parity.md#by-dimension) |
|
||||
| 27 | **Windows / Linux runtime coverage**: Windows installer UI unverified on Windows (#79); GPU path proven on Apple only | ROADMAP, issues | non-Mac users find bugs first | 15–30 | [hardware-parity](hardware-parity.md) |
|
||||
| 27 | **Windows / Linux runtime coverage**: Windows installer UI unverified on Windows (#79); GPU path proven on Apple only; the Linux riscv64 tarball (new in 0.6.0, #844) is cross-compiled and never run beyond a QEMU smoke test | ROADMAP, issues, `release.yml` | non-Mac users find bugs first | 15–30 | [hardware-parity](hardware-parity.md) |
|
||||
| 28 | **Small Classic shortcuts and keys**: rating `[` `]`, flag cycling, filter-bar keys, F1 help, panel keys (Tab, F5–F8) | [ui-parity](ui-parity.md#shortcuts) | Classic muscle memory | 4–8 | [ui-parity](ui-parity.md) |
|
||||
|
||||
Beta-blocking gaps (B) total **~230–430 h**; with the stability and platform work they need around them,
|
||||
@@ -61,7 +61,7 @@ Beta-blocking gaps (B) total **~230–430 h**; with the stability and platform w
|
||||
|
||||
- 2026-10-10: Nikon lossy-after-split NEF (#633), ORF 12.8 bpp (#651), HDR gain map / PQ AVIF / float
|
||||
TIFF export (#646), activity stack (#345).
|
||||
- 2026-10-09: HEIC in release builds (#609), Samsung uncompressed SRW (#459), contact-sheet PDF (#563),
|
||||
- 2026-10-09: EOS R7 C-RAW with an odd-height QP map (a45c2b66; part of gap 2), HEIC in release builds (#609), Samsung uncompressed SRW (#459), contact-sheet PDF (#563),
|
||||
Ukrainian UI (#564), Sony ILCE-7CR profile.
|
||||
- 2026-10-08: CR3 CRX lossless and C-RAW (#279), Fujifilm compressed RAF (#282), AI RAW denoise (#384),
|
||||
Lightroom catalog import (#218), German, Spanish, Russian UI.
|
||||
@@ -72,6 +72,12 @@ Beta-blocking gaps (B) total **~230–430 h**; with the stability and platform w
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-11 | minor | Reviewed against main after 0.6.0: CR3 entry names the CRX variants `crx.rs` still refuses and the R7 / R6 Mark II C-RAW state (decoding, not verified, so the measured numbers don't move); CI and Windows / Linux entries note the riscv64 release build (#844); R7 fix added to Closed recently. Panel docking (UI.9, #869) closes no listed gap |
|
||||
| 2026-10-11 | minor | Camera colour entry: raws with a black-and-white camera JPEG take its tone curve (#577); gap not closed |
|
||||
| 2026-10-11 | minor | Gap 4: Compare and Reference views get zoom windows per pane (1:1 was a 2,560 px render magnified); overlays and soft proofing remain, 8–15 h → 4–8 h |
|
||||
| 2026-10-10 | minor | Compressed ORF entry links the two Olympus patents that describe the method |
|
||||
| 2026-10-11 | minor | Gap 14: `crs:HDREditMode` is read from XMP (#663); the headroom limit and SDR rendition still aren't |
|
||||
| 2026-10-11 | minor | Gaps 2 and 10 shrink: EOS R5 Mark II CR3 decodes and is verified (fourth CR3 body, Canon 12 verified, 235 models, ~42% on a verified body); CR3 gain +6.5 → +6.2 points |
|
||||
| 2026-10-10 | minor | RAW gaps re-measured model by model (raw-parity.md): CR3, compressed ORF and verification entries carry measured shares; HE NEF no longer a beta blocker (+1.1 points for 20–40 h); verification entry absorbs the cheap per-body fixes (#769, #702, A200 series) |
|
||||
| 2026-10-10 | minor | Beta total raised to 340–600 h to include the ~92 open core-path user issues |
|
||||
| 2026-10-10 | major | Created from the checklist's Top gaps (whose items 1–2 had been duplicated by a merge) and ROADMAP's Where we stand; re-ranked; added evidence, impact, hours and beta flags; added platform-CI, lens-database, HE NEF and localization gaps |
|
||||
|
||||
+31
-6
@@ -73,18 +73,32 @@ hosts, tests) waits for its running jobs the same way (`JobPool::shutdown`, `cra
|
||||
|
||||
**Crash sentinel.** The desktop app writes `gpu-init.marker` into its settings folder (next to
|
||||
`ui.json`: `%APPDATA%\LightCraft`, `~/Library/Application Support/LightCraft`,
|
||||
`~/.config/lightcraft`) just before the compute device is created and removes it as soon as creation
|
||||
returns, successfully or not. If the marker is still there at the next launch, the process died inside
|
||||
`~/.config/lightcraft`) just before the compute device is created and its kernels are compiled (and, on the first
|
||||
AI denoise, before that network's kernels are) and removes it as soon as that returns, successfully or
|
||||
not (issue #250: Apple's Metal compiler, `MTLCompilerService`, aborting on a kernel on an older Intel
|
||||
Mac). A kernel the driver refuses without crashing is caught by wgpu error scopes: the log says why
|
||||
(`gpu: rendering on the CPU: …`) and renders stay on the CPU. `LIGHTCRAFT_GPU=0` (or Settings ▸
|
||||
uncheck *Use the GPU for rendering*) forces the CPU path from the start. If the marker is still there at the next launch, the process died inside
|
||||
the driver: LightCraft starts with GPU rendering off (the preference is saved unchecked), removes the
|
||||
marker and says so in a notice. Checking *Use the GPU for rendering* again tries the GPU once more
|
||||
(and re-arms the sentinel). Not with `LIGHTCRAFT_NO_PREFS` (tests, scripts), and not in a `--memory` session,
|
||||
which writes nothing: it neither arms the sentinel nor removes a marker it finds (it still starts with GPU
|
||||
rendering off when one is there, and the next ordinary launch reports and clears it). Killing the app during
|
||||
the ~0.3 s of device creation, or two instances starting at the same moment, can trip it falsely —
|
||||
harmless: rendering is then on the CPU until the box is checked again. The sentinel only covers the
|
||||
compute device; a crash while the window's renderer starts is avoided by the backend defaults above
|
||||
harmless: rendering is then on the CPU until the box is checked again. The sentinel covers the compute
|
||||
device and every kernel build; a crash while the window's renderer starts is avoided by the backend defaults above
|
||||
or worked around with `LIGHTCRAFT_GPU_BACKEND`.
|
||||
|
||||
**The window's renderer and its fallback (Windows).** Before the window opens, each renderer is probed in
|
||||
order: DX12, Vulkan, GL, then eframe's OpenGL renderer (glow). A probe creates a device and builds egui's
|
||||
pipeline inside wgpu error scopes, so a driver that cannot compile egui's shaders ("The parameter is
|
||||
incorrect. (0x80070057)" in `egui_pipeline`) is logged and skipped instead of killing the app. The renderer
|
||||
that worked is remembered in the settings folder (`window-renderer`); a start that dies before the window is up
|
||||
leaves `window-init.marker`, and the next start (relaunched automatically, at most 4 times) tries that
|
||||
renderer last. `LIGHTCRAFT_GPU_BACKEND=dx12 | vulkan | gl | glow` (or `WGPU_BACKEND`) forces one; DX12 stays
|
||||
first because a broken Vulkan driver can crash the process uncatchably (#136). Compute keeps its own
|
||||
device and falls back to the CPU as before.
|
||||
|
||||
**Troubleshooting a crash at startup (Windows).** Start LightCraft from a `.cmd` file or a terminal
|
||||
with `set LIGHTCRAFT_GPU_BACKEND=dx12` (the default since #136), or `=off` to keep the GPU out of
|
||||
rendering; `set VK_LOADER_DRIVERS_DISABLE=*igvk64*` (Vulkan loader) hides a specific Vulkan driver
|
||||
@@ -190,10 +204,19 @@ redone on the CPU and why, e.g. `"6000×4000: the GPU returned an incomplete ima
|
||||
24000000 pixels unwritten); the GPU is not used again"`). The same in `app.gpu` (`reason`,
|
||||
`lastFallback`), Help ▸ System Info and Settings ▸ Performance. `LIGHTCRAFT_PROFILE=1` prints the
|
||||
fallback with the stage timings; `RUST_LOG=warn` logs it.
|
||||
`perf.gpuFallback` is one value for the whole process, overwritten by the latest render on any
|
||||
thread. A render's *own* reason comes from `lightcraft_gpu::try_render` (`Err(Fallback::Failed(why))`;
|
||||
`Fallback::NotTried` when the GPU was not used at all), which `lightcraft_engine::media::develop_reporting`
|
||||
passes on as `RenderResult::gpu_fallback`; an export carries it as `Exported::gpu_fallback`, and
|
||||
each file of an `app.export` batch result has `gpuFallback` when its render fell back (issue #675).
|
||||
**Reproducing a smaller GPU:** `LIGHTCRAFT_GPU_LIMITS=webgpu` (or `downlevel`) creates the device
|
||||
with 128 MiB storage bindings / 256 MiB buffers, `LIGHTCRAFT_GPU_LIMITS=<n>` with n MiB / 2n MiB.
|
||||
Tests inject failures with `lightcraft_gpu::inject_fault` (`crates/gpu/tests/fallback.rs`,
|
||||
`export_falls_back_to_the_cpu_when_gpu_work_is_lost`).
|
||||
`export_falls_back_to_the_cpu_when_gpu_work_is_lost`). The fault is thread-local, and so are its
|
||||
consequences: a simulated fatal failure stops GPU use on the injecting thread only (`available()`
|
||||
false, `unavailable_reason()` set there), until `reset_failures()`, which undoes nothing else — a
|
||||
real failure still stops the whole process. Such a test asserts on its own render's reason, not on
|
||||
`last_fallback()`, so it can run beside other tests rendering on the GPU (issue #675).
|
||||
|
||||
## Correctness: CPU oracle and equivalence tests
|
||||
`crates/gpu/tests/equivalence.rs` renders the same settings on both and compares 8-bit sRGB:
|
||||
@@ -256,7 +279,9 @@ A view's stages keep an uploaded source only up to 96 MB (a Preview-level source
|
||||
`/usr/bin/time -l lightcraft-cli snapshot <files> --script steps.jsonl -o out.png` → "maximum
|
||||
resident set size" and "peak memory footprint". Run it several times: the high-water mark is
|
||||
noisy (allocator caching, scheduling). On Apple silicon GPU buffers count in the footprint.
|
||||
- Imports read raw headers only (`lightcraft_raw::probe_info`): no pixel data is decompressed.
|
||||
- Imports read raw headers only (`lightcraft_raw::probe_info`): no pixel data is decompressed,
|
||||
except the first rows a few vendor formats need for facts their headers lack (Nikon NEF: the
|
||||
first 128 rows for the masked trailing columns, issue #708).
|
||||
- One budget (`memory::budget`, default min(25 % of RAM, 1.5 GiB); `LIGHTCRAFT_MEMORY_MB` or
|
||||
`app.memoryBudget {mb}`): half for the engine caches (decoded thumbnail / preview / full
|
||||
sources and rendered previews, evicted least recently used *across* them; the photo on screen
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# Lens profiles from a lens database
|
||||
|
||||
Raws that carry no lens correction of their own (DNG `OpcodeList3`, or a camera correction a raw reader converts,
|
||||
see [sony-lens-corrections.md](sony-lens-corrections.md)) can be corrected from a **lens database in the lensfun
|
||||
XML format** that you supply. The app does not ship any lens database: lensfun's data is licensed CC-BY-SA 3.0 and
|
||||
stays a separate download. Adobe LCP profiles are never used.
|
||||
|
||||
## Setup
|
||||
|
||||
1. Get the lensfun database (the `data/db/*.xml` files of <https://github.com/lensfun/lensfun>, or the copy a Linux
|
||||
distribution installs under `/usr/share/lensfun/version_1/`).
|
||||
2. Point `LIGHTCRAFT_LENSFUN_DIR` at that directory and start the app (or the CLI).
|
||||
|
||||
Files are read once per process. Invalid files are skipped with a warning in the log. Without the variable nothing
|
||||
changes.
|
||||
|
||||
## What is applied
|
||||
|
||||
The lens is identified from EXIF: camera make / model (for the mount and crop factor; the 35 mm equivalent focal
|
||||
length is used when the camera isn't listed) and the lens model string. Lens names match on their words; every
|
||||
number (focal lengths, apertures) in the database name must appear in the EXIF name, and the lens maker must agree
|
||||
when the file names one.
|
||||
|
||||
| Correction | Models | Interpolated over |
|
||||
|---|---|---|
|
||||
| Distortion | `poly3`, `poly5`, `ptlens` | focal length |
|
||||
| Lateral CA (TCA) | `linear`, `poly3` | focal length |
|
||||
| Vignetting | `pa` | focal length, aperture (in stops), focus distance (nearest calibrated, by 1/d; infinity when unknown) |
|
||||
|
||||
Fisheye and other non-rectilinear lenses get no distortion correction. The profile is converted by least squares
|
||||
into the DNG `WarpRectilinear` / `FixVignetteRadial` polynomials, so it goes through the same path as embedded
|
||||
corrections: **Enable Profile Corrections** and its Distortion / Vignetting amounts, the CPU and GPU geometry
|
||||
stage (one resample), orientation handling and DNG export. A conversion whose radial error exceeds 0.001 of the
|
||||
half diagonal, or that would fold the image, is not applied.
|
||||
|
||||
Normalisation follows lensfun's documentation: distortion and TCA radii are 1 at half the shorter side of the
|
||||
calibration frame (its `cropfactor` and `aspect-ratio`), vignetting radii 1 at its corner; the optical centre is
|
||||
the image centre (lensfun's `<center>` offsets are not read).
|
||||
|
||||
## Limits
|
||||
|
||||
- Only raws: rendered images (JPEG, HEIF, TIFF) are often already corrected in camera.
|
||||
- The focus distance is not read from maker notes yet: vignetting uses the farthest calibrated distance.
|
||||
- No manual make / model picker yet; matching is automatic.
|
||||
- Not yet validated against real photos and another renderer; the tests use synthetic profiles and grids.
|
||||
@@ -4,6 +4,10 @@ File → Import Lightroom Catalog… opens `.lrcat` directly. Lightroom, Python,
|
||||
and a C SQLite runtime are not required. The source database is read-only, including committed
|
||||
`-wal` pages; a catalog that changes during the read is rejected. LightCraft owns subsequent edits.
|
||||
|
||||
File → Open Library… given a Lightroom catalog folder (or an `.lrdata` / `.lrcat-data` folder, or the `.lrcat`)
|
||||
doesn't make a LightCraft library there: it offers this import instead, with the catalog found
|
||||
(`crates/engine/src/lightroom_folder.rs`, issue #891). `lightcraft-cli --library` refuses those places too.
|
||||
|
||||
Equivalent commands:
|
||||
|
||||
```text
|
||||
@@ -47,7 +51,11 @@ A Custom white balance on a raw LightCraft develops relative to its as-shot look
|
||||
RAF, CR3, CR2, PEF, SRW; Lightroom names them all `RAW`) can't be taken as Kelvin: the catalog keeps no
|
||||
as-shot white for Custom photos, so the photo stays As Shot and `Temperature, Tint` are reported
|
||||
(issue #510; a DNG keeps the Kelvin value, and an XMP packet with `crs:AsShotTemperature` is
|
||||
shifted by the same mired difference, see [xmp-interop.md](xmp-interop.md)).
|
||||
shifted by the same mired difference, see [xmp-interop.md](xmp-interop.md)). A raw whose camera has
|
||||
measured colour matrices (`lightcraft_raw::spectral`: ILCE-7M3, D850, EOS 5D Mark III…) edits in Kelvin
|
||||
and keeps the Kelvin value too (issue #730) — when its file was found and probed, since the camera is
|
||||
read from the file, not from the Lightroom catalog; a missing file is treated as a relative-scale raw
|
||||
until Reload reads it.
|
||||
Lightroom's `-999999` deferred-adjustment sentinel is omitted from both catalog and XMP mappings;
|
||||
it is reported, included in any saved archive, and never clamped into a real slider value. Deferred Adobe Auto Tone
|
||||
is not evaluated by the importer; LightCraft's Auto control remains available after migration.
|
||||
@@ -62,5 +70,9 @@ Archive version 1 begins with the 12 bytes `LC-LRARCH\0\x01Z`, followed by zlib-
|
||||
Recovery readers must cap decompression at 32 MiB. JSON retains source identities, photo paths,
|
||||
settings, decoded XMP, keywords, history, snapshots, collection definitions and membership.
|
||||
|
||||
The importer uses shared Rust code on macOS, Windows and Linux. If originals move between
|
||||
computers or volumes, use LightCraft's missing-photo relinking tools to update their paths.
|
||||
The importer uses shared Rust code on macOS, Windows and Linux. When a root folder's recorded
|
||||
path isn't a folder on this computer (a catalog copied from another machine), the importer uses
|
||||
the path relative to the catalog that Lightroom records for root folders on the catalog's volume
|
||||
(`relativePathFromCatalog`), when that is a folder, and reports the remapping as a warning. Copying
|
||||
a project folder with its catalog inside it therefore keeps its originals linked. Other moved
|
||||
originals stay missing; use LightCraft's missing-photo relinking tools to update their paths.
|
||||
|
||||
+2
-2
@@ -164,9 +164,9 @@ per-request `_meta` receive `resultType: complete` and list/read cache hints; re
|
||||
| `select_photos {ids, active?, mode?}` | Set the selection / active photo. Every id (and `active`) must be in the library: an unknown id is a tool error (`no such photo 9999`) and the selection and active photo are left as they were |
|
||||
| `list_controls {section?}` | Every develop slider: id (`light.exposure`…), range, default, current value |
|
||||
| `get_develop {id?}` | Full develop-settings JSON, with `process`: the rendering process the photo is on ([process-versions.md](process-versions.md)) |
|
||||
| `set_develop {id?, values?, settings?, label?}` | `values`: `{controlId: number}`; `settings`: partial develop JSON deep-merged. Undoable |
|
||||
| `set_develop {id?, values?, settings?, label?}` | `values`: `{controlId: number}`; `settings`: partial develop JSON deep-merged. Undoable. A `crop.angle` value turns the crop as `crop.straighten` does: kept if it still fits, else shrunk to the largest crop of its aspect inside the photo (`settings` is merged as given, unfitted). White balance: `values` with `wb.temp` / `wb.tint` is Kelvin on raws with colour matrices (DNG, and cameras LightCraft has matrices for) and relative (6500 = as shot) elsewhere; a `settings.wb` with `temp` / `tint` and no `"scale": "kelvin"` is read on the relative scale on every raw (see [control-protocol.md](control-protocol.md)) |
|
||||
| `apply_preset {preset, amount?, ids?}` | Apply a preset (ids from `cmd_presets_list`) |
|
||||
| `crop {id?, rect?, angle?, reset?}` | Normalized crop rect `[x0,y0,x1,y1]` and straighten angle; at least one of `rect`, `angle`, `reset: true` |
|
||||
| `crop {id?, rect?, angle?, reset?}` | Normalized crop rect `[x0,y0,x1,y1]` and straighten angle; at least one of `rect`, `angle`, `reset: true`. The crop always ends inside the photo: an `angle` alone keeps the crop if it still fits, else shrinks it to the largest of its aspect that does |
|
||||
| `render_photo {id?, size?, format?, path?}` | Render with current settings → **image content** (PNG, or JPEG with `format: "jpeg"`), long edge `size` (default 1024) |
|
||||
| `export {path \| dir, id? \| ids?, format?, longEdge? \| shortEdge? \| width?/height? \| megapixels? \| percent?, dontEnlarge?, ppi?, quality?, colorSpace?, bitDepth?, …}` | Full-quality render to `.png` / `.jpg` / `.tif` / `.webp` / `.avif`; `format: "original"` copies the file + an XMP sidecar with the edits, `format: "dng"` writes raw photos as DNG with the edits embedded. No size param = 3000 px long edge; `longEdge: 0` = full size (cropped, native resolution); `width` + `height` fit either orientation; `dontEnlarge` defaults to true |
|
||||
|
||||
|
||||
+74
-59
File diff suppressed because one or more lines are too long
@@ -5,7 +5,10 @@ Every photo's develop settings record the **rendering process** they are interpr
|
||||
this the process version. It lets the look of LightCraft improve (a new default tone curve, issue #146;
|
||||
stronger Light sliders, issue #196) without changing a single photo someone has already edited.
|
||||
|
||||
Today there is one process, **V1**: LightCraft's rendering from before process versions existed.
|
||||
| Process | What it changes |
|
||||
|---|---|
|
||||
| **V1** | LightCraft's rendering from before process versions existed. Frozen. |
|
||||
| **V2** (the latest) | Highlights and Shadows keep fine detail (issue #632): their base layer (`local::tone_base`) is the edge-aware filter of a separable median of log luminance (radius 0.15 % of the long edge, at most 24 px, `local::BASE_DETAIL`) instead of log luminance itself, so twigs, lines and texture take their surroundings' shift and keep their contrast; region edges stay sharp. CPU (`local::prepare`) and GPU prepare branch on `local::tone_base_detail` (0 = no median under V1); the base plane is cached per process (`local::BaseKey`). Renders without Highlights or Shadows (global or local) are the same as V1's. Tone map unchanged. |
|
||||
|
||||
## The rule
|
||||
|
||||
|
||||
+39
-20
@@ -1,6 +1,6 @@
|
||||
# RAW parity: cameras, formats, demosaic, colour and lens profiles
|
||||
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (model-by-model measurement against Adobe's published camera list; use-weighted coverage; variant matrix; colour and lens sections; ranked next cameras) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6 (Camera Raw 18.7)
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** minor (EOS R5 Mark II decoded and verified: CRX QP-map escape, odd-height planes) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6 (Camera Raw 18.7)
|
||||
|
||||
Raw processing is the deepest single dimension of Lightroom parity: a photographer whose camera
|
||||
does not decode, or decodes with the wrong colour, cannot switch. This document measures it model by
|
||||
@@ -14,13 +14,13 @@ ranked in [gaps.md](gaps.md). Details per format: [cr3.md](cr3.md),
|
||||
| Measure | Value | Kind |
|
||||
|---|---:|---|
|
||||
| Adobe-listed camera models | **1,446** | measured: [raw/adobe-cameras.tsv](raw/adobe-cameras.tsv) |
|
||||
| Decoded from sensor data and **verified** on a real file | **234** (16%) | measured: [raw/evidence.tsv](raw/evidence.tsv) |
|
||||
| Decoder path should handle it, **unverified** | **943** (65%) | measured from the format rules (632 of them DNG, mostly phones) |
|
||||
| Decoded from sensor data and **verified** on a real file | **235** (16%) | measured: [raw/evidence.tsv](raw/evidence.tsv) |
|
||||
| Decoder path should handle it, **unverified** | **942** (65%) | measured from the format rules (632 of them DNG, mostly phones) |
|
||||
| **Preview-only** (embedded JPEG) | **233** (16%) | measured |
|
||||
| **Unsupported** (not imported, refused or decodes wrongly) | **36** (2%) | measured |
|
||||
| Models added since 2017 (Camera Raw 10+): verified / unverified / preview / unsupported | 81 / 641 / 24 / 8 of 754 | measured |
|
||||
| **Share of photographers whose camera fully works** (decodes from sensor data), use-weighted | **~84%** | estimated: measured classes × written usage weights × per-format confidence |
|
||||
| …of which on a body we have verified | **~40%** | estimated, same weights |
|
||||
| Models added since 2017 (Camera Raw 10+): verified / unverified / preview / unsupported | 82 / 640 / 24 / 8 of 754 | measured |
|
||||
| **Share of photographers whose camera fully works** (decodes from sensor data), use-weighted | **~85%** | estimated: measured classes × written usage weights × per-format confidence |
|
||||
| …of which on a body we have verified | **~42%** | estimated, same weights |
|
||||
| Camera colour: Adobe profiles vs ours | 1,500 Adobe Standard profiles, 481 models with Camera Matching profiles vs **4 bundled profiles** + per-file fits + 52 spectral matrices | measured (bundle listing, Adobe list, `assets/camera-profiles`) |
|
||||
| Lens profiles: Adobe vs ours | **3,635 vs 0** (in-file corrections only) | measured |
|
||||
| **RAW readiness** (decoding 60%, colour 25%, lens 15%) | **~56%** (range 50–60%) | estimated, see [Scores](#scores) |
|
||||
@@ -42,7 +42,8 @@ committed table is stale.
|
||||
claims were not used.
|
||||
- **Verified** means a real file from that body decoded from sensor data in: the corpus test over the
|
||||
`RAW_SAMPLES` manifest (`xtask/src/main.rs`, 101 CC0 files; bodies whose sample is in
|
||||
`KNOWN_UNSUPPORTED` excluded), `crates/raw/tests/cr3_corpus.rs` (3 bodies, exact), the RAF reference
|
||||
`KNOWN_UNSUPPORTED` excluded), `crates/raw/tests/cr3_corpus.rs` (3 bodies exact against an external decoder;
|
||||
the EOS R5 Mark II pinned, with a QP-map check of 33,665 private files), the RAF reference
|
||||
check ([raf-compression.md](raf-compression.md)), and the raw.pixls.us bulk checks recorded in issue
|
||||
#535 (all 100 Sony bodies there; 87 decode) and PR #215 (every Panasonic / Leica RW2 / RWL / RAW body
|
||||
there, 178 files from 118 bodies). Bulk-check body lists were matched to Adobe names through the
|
||||
@@ -56,14 +57,14 @@ committed table is stale.
|
||||
~8 years (listed in the file), 20% to the bucket's other Adobe models.
|
||||
- **Per-format confidence** for unverified bodies (the chance their files decode correctly): DNG, CR2,
|
||||
RAF, RW2 0.95; NEF, ARW, PEF 0.9; ORF, SRW, SR2 0.8; **CR3 0.6** (the CRX decoder refuses several
|
||||
coding variants and only three bodies are verified; maintainers report the R7 working).
|
||||
coding variants and only four bodies are verified; maintainers report the R7 working).
|
||||
|
||||
## By maker (measured)
|
||||
|
||||
| Maker | Adobe models | verified | unverified | preview | unsupported | Main reason for preview / unsupported |
|
||||
|---|---:|---:|---:|---:|---:|---|
|
||||
| Sony | 149 | 86 | 51 | 9 | 3 | DSLR-A200…A390 and A850 12-bit compressed ARW (preview, #535); A100 and DSC-F828 refused (regression #702); SRF not imported |
|
||||
| Canon | 134 | 11 | 100 | 21 | 2 | CRW (2000–2004 bodies) preview; EOS-1D / 1Ds TIFF raws not imported |
|
||||
| Canon | 134 | 12 | 99 | 21 | 2 | CRW (2000–2004 bodies) preview; EOS-1D / 1Ds TIFF raws not imported |
|
||||
| Samsung | 128 | 1 | 118 | 9 | 0 | compressed SRW (NX1, NX30, NX300, NX500, NX2000, NX3000, NX3300, NX mini, Galaxy NX) |
|
||||
| Apple | 113 | 1 | 112 | 0 | 0 | — (DNG / ProRAW) |
|
||||
| Panasonic | 104 | 93 | 11 | 0 | 0 | — |
|
||||
@@ -80,13 +81,13 @@ committed table is stale.
|
||||
| DJI | 21 | 0 | 21 | 0 | 0 | — (DNG) |
|
||||
| OM Digital Solutions | 7 | 0 | 0 | 7 | 0 | **compressed ORF** |
|
||||
| Kodak, Konica Minolta, Epson, Sigma, GoPro, Contax, others | 56 | 0 | 14 | 27 | 15 | DCR, GPR not imported; KDC, MRW, ERF, X3F preview |
|
||||
| **All** | **1,446** | **234 (16%)** | **943 (65%)** | **233 (16%)** | **36 (2%)** | |
|
||||
| **All** | **1,446** | **235 (16%)** | **942 (65%)** | **233 (16%)** | **36 (2%)** | |
|
||||
|
||||
## By use (estimated)
|
||||
|
||||
| Bucket | Weight | Camera fully works | On a verified body | Why |
|
||||
|---|---:|---:|---:|---|
|
||||
| Canon | 33% | 72% | 16% | 17 of the 22 popular bodies write CR3; 2 of them verified, the rest at 0.6 |
|
||||
| Canon | 33% | 73% | 19% | 18 of the 22 popular bodies write CR3; 4 of them verified (M50, R8, R100, R5 Mark II), the rest at 0.6 |
|
||||
| Sony | 27% | 98% | 91% | bulk-checked |
|
||||
| Nikon | 16% | 91% | 12% | decoders cover every NEF coding but HE; few bodies verified |
|
||||
| Fujifilm | 10% | 91% | 45% | |
|
||||
@@ -97,7 +98,7 @@ committed table is stale.
|
||||
| Leica | 1% | 95% | 5% | |
|
||||
| DJI | 1% | 95% | 0% | |
|
||||
| Medium format and others | 0.5% | 23% | 0% | IIQ, 3FR, X3F |
|
||||
| **Weighted** | 100% | **84%** | **40%** | |
|
||||
| **Weighted** | 100% | **85%** | **42%** | |
|
||||
|
||||
About 5 in 6 raw photographers' cameras decode, but only about 2 in 5 use a body we have actually
|
||||
checked, and Canon, the largest bucket, is the weakest of the big four.
|
||||
@@ -110,8 +111,8 @@ checked, and Canon, the largest bucket, is the weakest of the big four.
|
||||
|---|---|---|---|
|
||||
| CR2 | lossless JPEG slices | ✅ | 8 corpus bodies; CFA phase from `CR2CFAPattern` (#85, 35 bodies); masked-column black level |
|
||||
| CR2 | sRAW / mRAW (YCbCr) | ✅ | `vendor/cr2/sraw.rs`, 31 samples from 16 models |
|
||||
| CR3 | CRX lossless, version 0x100 | ✅ | M50, R100, R8 exact against a reference |
|
||||
| CR3 | C-RAW 0x100 (horizontal tiles), 0x200 (single 14-bit tile, adaptive QP) | ✅ | same 3 bodies exact |
|
||||
| CR3 | CRX lossless, version 0x100 | ✅ | M50, R100, R8 exact against a reference; R5 Mark II (odd-height planes) pinned |
|
||||
| CR3 | C-RAW 0x100 (horizontal tiles), 0x200 (single 14-bit tile, adaptive QP incl. the QP-map escape) | ✅ | same 3 bodies exact; R5 Mark II pinned, QP maps of 33,665 files decode exactly, matches the camera JPEG |
|
||||
| CR3 | other CRX variants (vertical tiles, partial subbands, other plane flags, 0x200 multi-tile) | ⬜ | refused in `crx.rs`, falls back to the preview; which bodies write them is unmeasured |
|
||||
| CR3 | sRAW / mRAW | ⬜ | 4-plane Bayer only |
|
||||
| CR3 | HEIF / HDR PQ | ⬜ | HEVC tracks recognised, not decoded |
|
||||
@@ -122,6 +123,7 @@ checked, and Canon, the largest bucket, is the weakest of the big four.
|
||||
| NEF | High Efficiency / HE★ (JPEG XS) | ⬜ | named, preview (#193); needs a JPEG XS decoder |
|
||||
| ARW | uncompressed, cRAW (ARW2), lossless LJ92 (L / M / S) | ✅ | 86 bodies (#535); M/S black level 1024 |
|
||||
| ARW | packed 12-bit (A900) | ✅ | one CC0 file; A850 unmeasured |
|
||||
| ARW | ARW6 Compressed / HQ (compression 32766; ILCE-7RM6) | ✅ | 8 CC0 files; 12-bit mode unsupported; `docs/arw6-compression.md` |
|
||||
| ARW | 12-bit compressed of DSLR-A200…A390 | ⬜ | preview (#535) |
|
||||
| ARW | A100 ARW; SRF (F828, V3) | ⬜ | refused (#702); SRF not imported |
|
||||
| ARW | ARQ pixel shift | ⬜ | no code |
|
||||
@@ -145,6 +147,21 @@ checked, and Canon, the largest bucket, is the weakest of the big four.
|
||||
| IIQ, MOS, MRW, ERF, KDC, X3F | all | ⬜ | preview |
|
||||
| DCR, MEF, MFW, DXO, GPR, TIFF raws | all | ⬜ | not imported |
|
||||
|
||||
### Compressed ORF: a clean-room starting point
|
||||
|
||||
Two Olympus patents by Takashi Ishikawa describe the *method* of lossless RAW compression in prose; neither names ORF
|
||||
and neither gives the file layout. See the closed #240 and the open #360 for why a decoder has to be derived from the
|
||||
files and prose rather than from dcraw-family code.
|
||||
|
||||
- [US7333035B2](https://patents.google.com/patent/US7333035): lossless RAW compression for SLR cameras:
|
||||
median-edge-detector prediction from the left, upper and upper-left neighbours, a separately coded sign, Golomb-Rice
|
||||
codes whose parameter `k` comes from flatness run counters, counters reset per line.
|
||||
- [US7715639B2](https://patents.google.com/patent/US7715639B2/en): a sibling with predictor variants, `k` derived from
|
||||
neighbouring positions, raw low-order bits and a correction filter.
|
||||
|
||||
They give the algorithm's shape; constants, the bit layout and the stream header still have to be established from CC0
|
||||
sample files. A pointer for contributors, not legal advice.
|
||||
|
||||
## Demosaic, levels, crops and lens metadata
|
||||
|
||||
| Stage | Target | LightCraft | Status |
|
||||
@@ -171,7 +188,7 @@ checked, and Canon, the largest bucket, is the weakest of the big four.
|
||||
| Camera Matching looks | 481 models (plus phone-maker looks) | none (the per-file JPEG fit imitates the camera's own look) |
|
||||
| Creative profiles | Adobe Color, Portrait, Landscape, Vivid, Monochrome… | our own looks (Color, Neutral, Vivid, Landscape, Portrait, Monochrome) |
|
||||
| DCP / dual-illuminant model | ✅ | dual-illuminant DNG colour (ColorMatrix / ForwardMatrix 1-2, CameraCalibration, AnalogBalance), HueSatMap, LookTable, ProfileToneCurve; external `.dcp` never loaded (clean-room) |
|
||||
| White balance in Kelvin | ✅ for every model | absolute Kelvin only where matrices exist; per-file fits give relative WB (#296, #730) |
|
||||
| White balance in Kelvin | ✅ for every model | absolute Kelvin where the camera has colour matrices (its own, or one of the 52 spectral rows), also for files whose look is a per-file fit (#730); other raws keep the relative scale (#296) |
|
||||
|
||||
**How far the default rendering is from Adobe's** (CIE ΔE; measured where stated):
|
||||
|
||||
@@ -186,7 +203,7 @@ checked, and Canon, the largest bucket, is the weakest of the big four.
|
||||
desaturated), the largest group of open core-path issues.
|
||||
|
||||
**Colour score ~35%** (estimated): most decoded files start close to the camera's own look (ΔE ~5),
|
||||
about 10% start far off (ΔE ~18), nothing is calibrated, Kelvin WB is missing for fitted cameras, and
|
||||
about 10% start far off (ΔE ~18), nothing is calibrated, Kelvin WB is missing for fitted cameras without colour matrices, and
|
||||
the distance to Adobe Standard is unmeasured. In the readiness numbers this becomes a colour factor
|
||||
of ×0.85 for a working photographer (they correct colour by hand, at a cost) and ×0.75 for the
|
||||
essentials user (the default look is what they keep).
|
||||
@@ -207,10 +224,10 @@ drone DNG users (~9% of photographers by the weights above); Adobe reaches nearl
|
||||
|
||||
| Part | Weight | Score | Basis |
|
||||
|---|---:|---:|---|
|
||||
| Decoding | 60% | **76%** | use-weighted 84% × 0.9 for per-body decode bugs still surfacing (levels, crops, WB: #535, #455, #461, #702) |
|
||||
| Decoding | 60% | **77%** | use-weighted 85% × 0.9 (76.5%) for per-body decode bugs still surfacing (levels, crops, WB: #535, #455, #461, #702) |
|
||||
| Colour | 25% | **35%** | see [Colour](#colour) |
|
||||
| Lens profiles | 15% | **10%** | see [Lens profiles](#lens-profiles) |
|
||||
| **RAW readiness** | 100% | **~56%** (range 50–60%) | 45.6 + 8.75 + 1.5; demosaic and highlight quality unmeasured |
|
||||
| **RAW readiness** | 100% | **~56%** (range 50–60%) | 45.9 + 8.75 + 1.5; demosaic and highlight quality unmeasured |
|
||||
|
||||
The previous estimate (earlier on 2026-10-10) was ~50%, with decoding at ~78% weighted by maker
|
||||
from the format list alone. Measured model by model, decoding comes out about the same; the change is
|
||||
@@ -226,7 +243,7 @@ works, from [raw/usage-weights.tsv](raw/usage-weights.tsv).
|
||||
| # | Camera / variant | Gain (points of photographers) | Hours | Gain per hour | Needs |
|
||||
|---:|---|---:|---:|---:|---|
|
||||
| 1 | Compressed ORF (every Olympus / OM body since ~2008) | +2.6 | 8–15 | ~0.20 | CC0 samples (have 2) |
|
||||
| 2 | CR3: remaining CRX variants, then verify the 17 popular CR3 bodies | +6.5 | 25–45 | ~0.19 | samples per body (raw.pixls.us has most) |
|
||||
| 2 | CR3: remaining CRX variants, then verify the other 14 popular CR3 bodies | +6.2 | 25–45 | ~0.19 | samples per body (raw.pixls.us has most) |
|
||||
| 3 | Verify popular Nikon, Fujifilm, Canon CR2 and Pentax bodies (one CC0 file each, fix what breaks) | +2.0 | 8–15 | ~0.17 | public samples |
|
||||
| 4 | Nikon packed 14-bit under a compressed tag (Z 6II uncompressed, #769) | +0.2 | 1–2 | ~0.13 | PR #671 exists |
|
||||
| 5 | Sony DSLR-A200…A390 / A850 12-bit compressed ARW | +0.4 | 3–6 | ~0.08 | samples on raw.pixls.us |
|
||||
@@ -240,7 +257,7 @@ Beyond decoding:
|
||||
|
||||
| Gap | Hours | Needs |
|
||||
|---|---:|---|
|
||||
| Measured colour calibration for the ~60 popular bodies, Kelvin WB, render-fidelity suite against Lightroom | 80–150 | chart shots per camera (a human with cameras) or licensable measured data |
|
||||
| Measured colour calibration for the ~60 popular bodies, Kelvin WB for cameras without matrices, render-fidelity suite against Lightroom | 80–150 | chart shots per camera (a human with cameras) or licensable measured data |
|
||||
| Lens profile database of our own (fitting tools, data format, top 100 lenses), maker-note vignetting / CA | 40–80 | calibration shots (a human with lenses) |
|
||||
| Demosaic and highlight quality measured against Lightroom; X-Trans AI denoise | 15–30 | |
|
||||
| **Total RAW** | **255–480 h** (decoding list above 120–220 h; ~60% parallelizes) | |
|
||||
@@ -249,5 +266,7 @@ Beyond decoding:
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-10 | minor | Compressed ORF: pointer to the two Olympus patents that describe the method, as a clean-room starting point |
|
||||
| 2026-10-11 | minor | EOS R5 Mark II verified (CRX QP-map first literal and limited-length escape, odd-height Bayer planes; 4 CC0 files pinned, QP maps of 33,665 private files): 235 verified, 942 unverified; use-weighted 85% (42% on verified bodies); decoding 77%, RAW readiness unchanged at ~56% |
|
||||
| 2026-10-10 (later) | major | Model-by-model measurement against Adobe's published list (1,446 models: 234 verified, 943 unverified, 233 preview, 36 unsupported); `docs/raw/` data and re-measure script; use-weighted share 84% (40% on verified bodies); variant matrix; colour and lens sections with scores; ranked next cameras; RAW readiness ~56% from written parts |
|
||||
| 2026-10-10 | major | Created: formats, makers, demosaic, colour and lens checklists; target counts from the installed Lightroom 9.6 bundle listing |
|
||||
|
||||
@@ -188,7 +188,7 @@ Canon EOS R1 CR3 2023 unverified CRX decoder: lossless and C-RAW 0x100 / 0x200 s
|
||||
Canon EOS R3 CR3 2021 unverified CRX decoder: lossless and C-RAW 0x100 / 0x200 single-tile; other variants fall back to the preview
|
||||
Canon EOS R5 CR3 2019 unverified CRX decoder: lossless and C-RAW 0x100 / 0x200 single-tile; other variants fall back to the preview
|
||||
Canon EOS R5 C CR3 2021 unverified CRX decoder: lossless and C-RAW 0x100 / 0x200 single-tile; other variants fall back to the preview
|
||||
Canon EOS R5 Mark II CR3 2023 unverified CRX decoder: lossless and C-RAW 0x100 / 0x200 single-tile; other variants fall back to the preview
|
||||
Canon EOS R5 Mark II CR3 2023 verified corpus: crates/raw/tests/cr3_corpus.rs (pinned sensor hashes, docs/cr3.md); bulk: QP maps of 33,665 C-RAW files
|
||||
Canon EOS R6 CR3 2019 unverified CRX decoder: lossless and C-RAW 0x100 / 0x200 single-tile; other variants fall back to the preview
|
||||
Canon EOS R6 V CR3 2025 unverified CRX decoder: lossless and C-RAW 0x100 / 0x200 single-tile; other variants fall back to the preview
|
||||
Canon EOS R6 Mark II CR3 2022 unverified CRX decoder: lossless and C-RAW 0x100 / 0x200 single-tile; other variants fall back to the preview
|
||||
|
||||
|
Can't render this file because it contains an unexpected character in line 86 and column 19.
|
@@ -11,6 +11,7 @@ Canon EOS 7D verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
|
||||
Canon EOS 80D verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
|
||||
Canon EOS M50 (EOS Kiss M) verified corpus: crates/raw/tests/cr3_corpus.rs (exact sensor hashes, docs/cr3.md)
|
||||
Canon EOS R100 verified corpus: crates/raw/tests/cr3_corpus.rs (exact sensor hashes, docs/cr3.md)
|
||||
Canon EOS R5 Mark II verified corpus: crates/raw/tests/cr3_corpus.rs (pinned sensor hashes, docs/cr3.md); bulk: QP maps of 33,665 C-RAW files
|
||||
Canon EOS R8 verified corpus: crates/raw/tests/cr3_corpus.rs (exact sensor hashes, docs/cr3.md)
|
||||
Canon EOS-5D Mark III verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
|
||||
Fujifilm GFX 100 verified corpus: crates/raw/tests/corpus.rs (xtask RAW_SAMPLES)
|
||||
|
||||
|
Can't render this file because it contains an unexpected character in line 2 and column 3.
|
@@ -64,7 +64,8 @@ from the interpolated file table was below 0.12 px at 1440 px. That measures app
|
||||
not agreement with an independent renderer. Camera JPEG and Apple Core Image comparisons support
|
||||
the wide-lens interpretation but are weaker evidence than same-renderer On/Off pairs.
|
||||
|
||||
Actual LightCraft exports after crop normalization were also compared directly with Sony's corrected
|
||||
Historical validation before restoring main's image crop precedence: LightCraft exports with the
|
||||
standard default crop framing were compared directly with Sony's corrected
|
||||
TIFFs. At 2376 px wide, median / 95th-percentile feature residuals were 0.35 / 1.04 px at 24 mm,
|
||||
0.15 / 0.40 px at 200 mm, and 0.13 / 0.56 px at 600 mm (median translation below 0.024 px).
|
||||
The uncorrected medians at 200 and 600 mm were 8.63 and 10.82 px respectively. Different colour,
|
||||
@@ -73,6 +74,20 @@ An isolated desktop run on Apple M1 Metal detected the profile, rendered without
|
||||
warnings, and exposed the existing Optics controls. Existing library edits are not reset: once the
|
||||
source has loaded, enable lens corrections if they were previously disabled.
|
||||
|
||||
After restoring main's crop precedence, fresh 2376×1584 exports on 2026-10-10
|
||||
(main `b70ae174` plus the aspect-framing guard) measured:
|
||||
|
||||
| Focal length | Corrected median / p95 residual | Uncorrected median | Removed translation x / y | Matches / maximum radius |
|
||||
|---|---|---|---|---|
|
||||
| 24 mm | 0.36 / 1.12 px | 5.62 px | 7.98 / −0.01 px | 1,442 / 0.95 |
|
||||
| 200 mm | 0.13 / 0.35 px | 8.65 px | 8.01 / 0.02 px | 11,606 / 0.98 |
|
||||
| 600 mm | 0.10 / 0.37 px | 10.33 px | 8.02 / 0.00 px | 12,268 / 0.97 |
|
||||
|
||||
Residuals subtract only the median feature translation; no scale or lens coefficients are fitted to
|
||||
these reference pairs. The roughly eight-pixel horizontal translation is the retained crop-origin
|
||||
difference described above, not a claim of exact Sony framing. Maximum radius is in half diagonals.
|
||||
The distortion geometry remains aligned while every body's image framing follows main's crop order.
|
||||
|
||||
Tests cover signedness/length rejection, zero and malformed tables, independent 600 mm reference
|
||||
geometry, barrel framing, offset crop normalization, both TIFF byte orders, header/full consistency,
|
||||
and preservation through DNG export. No private photographs or reference TIFFs are committed.
|
||||
@@ -84,10 +99,9 @@ model names, and every other model remain unsupported by this distortion decoder
|
||||
numbers and table lengths do not establish the scale, interpolation or crop normalization on another
|
||||
body. Support can expand after independent geometry validation; do not add model-name prefix matching.
|
||||
|
||||
The original nine Sony samples listed by `cargo xtask corpus --download` were downloaded from its
|
||||
CC0 URLs. The manifest now also includes CC0 samples 4822 (A7R IVA / 24–105 mm) and 3989
|
||||
(A9 II / 200–600 mm), with their SHA-256 hashes recorded alongside the URLs. All eleven files
|
||||
are checked with `sony_embedded_distortion_is_limited_to_validated_models`:
|
||||
The current `cargo xtask corpus --download` manifest contains 23 Sony CC0 samples across 17 bodies,
|
||||
including samples 4822 (A7R IVA / 24–105 mm) and 3989 (A9 II / 200–600 mm). All 23 files have
|
||||
pinned SHA-256 hashes and are checked with `sony_embedded_distortion_is_limited_to_validated_models`:
|
||||
|
||||
| Camera | Corpus samples | Raw-IFD table observed | Expected correction |
|
||||
|---|---|---|---|
|
||||
@@ -98,15 +112,23 @@ are checked with `sony_embedded_distortion_is_limited_to_validated_models`:
|
||||
| ILCE-7RM2 | 12-bit uncompressed | 16 samples | None |
|
||||
| DSC-RX100 | One | No raw-IFD table | None |
|
||||
| DSC-RX100M3 | One | 11 samples | None |
|
||||
| ILCE-7CR | Compressed, lossless L/M | Not used; model excluded | None |
|
||||
| ILCE-7RM4 | 14-bit compressed | Not used; model excluded | None |
|
||||
| DSLR-A500, DSLR-A700, SLT-A33 | One each | Not used; model excluded | None |
|
||||
| ILCE-3500, ILCE-7S | 5600 K, Shade respectively | Not used; model excluded | None |
|
||||
| ILCE-7SM2, DSLR-A580 | 16:9 framing, one each | Not used; model excluded | None |
|
||||
| DSLR-A900 | Packed 12-bit | Not used; model excluded | None |
|
||||
|
||||
These are **model-boundary regression checks, not new geometric validation of those models**. In particular,
|
||||
refusing the A7 III tables resolves the unverified corner warp identified in review. The lossless M/S
|
||||
files are linear YCbCr and remain excluded independently of the model check. Header and full decode
|
||||
must agree for every sample. Synthetic tests also verify both TIFF byte orders, exact model matching,
|
||||
that the warp is centred on the standard default crop while the image keeps Sony's crop, and that DNG export preserves an accepted warp without adding one to rejected
|
||||
models. The existing independent Sony geometry tests continue to cover the enabled A7R IVA path.
|
||||
Re-exporting the 24, 200 and 600 mm reference files after the model restriction produced byte-identical
|
||||
corrected PNGs to the earlier validated exports. Isolated headless app checks showed the A7R IVA profile
|
||||
models. A non-3:2 Sony image crop is also rejected when the standard default crop still describes
|
||||
3:2 geometry; the regression fails without the image-framing guard. The existing independent Sony
|
||||
geometry tests continue to cover the enabled A7R IVA path.
|
||||
Before restoring main's crop precedence, re-exporting the 24, 200 and 600 mm reference files after
|
||||
the model restriction produced byte-identical corrected PNGs to the earlier validated exports. Isolated headless app checks showed the A7R IVA profile
|
||||
controls present and the A7 III profile controls absent, with no notices and completed RAW renders.
|
||||
|
||||
Before enabling another body, compare corrected and uncorrected renders against independent references
|
||||
|
||||
+11
-10
@@ -1,6 +1,6 @@
|
||||
# Target-app parity: LightCraft vs Adobe Lightroom
|
||||
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (full re-measure against Lightroom Classic 15.5 and the installed Lightroom 9.6; merged ROADMAP's parity estimate and honest assessment into this file) · **Target:** Adobe Lightroom Classic 15.5 (August 2026) and Lightroom 9.6 (desktop)
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** minor (EOS R5 Mark II CR3 verified: RAW decoding 76% → 77%; headline numbers unchanged) · **Target:** Adobe Lightroom Classic 15.5 (August 2026) and Lightroom 9.6 (desktop)
|
||||
|
||||
The authoritative assessment of how close LightCraft is to Lightroom. The row-by-row checklist is
|
||||
[parity-checklist.md](parity-checklist.md); the ranked work list is [gaps.md](gaps.md); deep areas
|
||||
@@ -67,7 +67,7 @@ from the checklist sections and the deep-dive documents.
|
||||
| Area | Weight | Breadth | Ready | Remaining | Notes |
|
||||
|---|---:|---:|---:|---:|---|
|
||||
| Import and library (import, folders, albums, smart albums, keywords, metadata, search, faces) | 15% | 90% | 80% | 25–45 h | checklist A–E 83–92%; robust journal; 85k-photo libraries |
|
||||
| RAW decoding and camera coverage | 15% | 81% | 76% | 120–220 h | [raw-parity.md](raw-parity.md), measured model by model: 84% of photographers' cameras decode (use-weighted), × 0.9 for per-body bugs; CR3 variants, compressed ORF, HE NEF |
|
||||
| RAW decoding and camera coverage | 15% | 81% | 77% | 120–220 h | [raw-parity.md](raw-parity.md), measured model by model: 85% of photographers' cameras decode (use-weighted), × 0.9 for per-body bugs; CR3 variants, compressed ORF, HE NEF |
|
||||
| Colour, camera profiles, render fidelity | 10% | 65% | 35% | 80–150 h | 4 bundled profiles vs Adobe's 1,500; per-file fits median ΔE ~5 vs the camera JPEG, ~10% far off; no fidelity suite ([raw-parity.md](raw-parity.md#colour)) |
|
||||
| Global develop adjustments (light, colour, curves, grading, effects, detail) | 15% | 93% | 70% | 30–60 h | checklist F 91.7%; character tuned by eye |
|
||||
| Optics, lens profiles, crop and geometry | 6% | 80% | 50% | 40–80 h | no lens database; crop / Upright complete |
|
||||
@@ -129,7 +129,7 @@ valued from the model-by-model measurement in [raw-parity.md](raw-parity.md):
|
||||
| Dimension | Weight | Ready |
|
||||
|---|---:|---:|
|
||||
| Features: the other 11 areas, weighted depth from [Features](#features) | 45% | 60.9% |
|
||||
| RAW decoding and camera coverage | 9% | 76% (84% use-weighted × 0.9) |
|
||||
| RAW decoding and camera coverage | 9% | 77% (85% use-weighted × 0.9) |
|
||||
| Camera colour and default rendering | 6% | 35% |
|
||||
| UI/UX fidelity | 8% | 70% |
|
||||
| File formats (non-raw) | 6% | 70% |
|
||||
@@ -139,7 +139,7 @@ valued from the model-by-model measurement in [raw-parity.md](raw-parity.md):
|
||||
| Stability | 8% | 60% |
|
||||
| Platforms | 4% | 60% |
|
||||
| Ecosystem / plug-ins | 2% | 15% |
|
||||
| **Weighted sum** | 100% | **60.6% ≈ 61%** (range 56–66%) |
|
||||
| **Weighted sum** | 100% | **60.7% ≈ 61%** (range 56–66%) |
|
||||
|
||||
Mainstream comes out lower than full: its scope is the raw-develop core, where LightCraft is weakest
|
||||
(camera colour, CR3 coverage), and its discounts are multiplicative, while the full sum is lifted by
|
||||
@@ -171,18 +171,18 @@ than the user's own.
|
||||
|---|---:|---:|
|
||||
| Import and library | 18% | 80% |
|
||||
| Cull: views, compare, zoom, ratings | 12% | 70% |
|
||||
| Raw decoding for their camera | 27% | 84% (measured use-weighted share whose camera fully works, [raw-parity.md](raw-parity.md#by-use-estimated)) |
|
||||
| Raw decoding for their camera | 27% | 85% (measured use-weighted share whose camera fully works, [raw-parity.md](raw-parity.md#by-use-estimated)) |
|
||||
| Global develop adjustments | 15% | 70% |
|
||||
| Crop, geometry, lens corrections | 6% | 50% |
|
||||
| Masking (manual, subject / sky; no generative) | 8% | 45% |
|
||||
| Heal / remove (no generative), denoise | 5% | 40% |
|
||||
| Presets, copy / paste / sync | 5% | 85% |
|
||||
| Export | 4% | 75% |
|
||||
| **Weighted depth** | 100% | **71.8%** |
|
||||
| **Weighted depth** | 100% | **72.1%** |
|
||||
|
||||
Colour fidelity is a written discount, **×0.85**: most raws start near the camera's own look (median
|
||||
ΔE ~5), about 10% far off (ΔE ~18), none calibrated to Adobe Standard, so a working photographer
|
||||
corrects colour by hand (33 open colour issues). 71.8% × 0.804 × 0.85 = **~49%** (range 44–54%).
|
||||
corrects colour by hand (33 open colour issues). 72.1% × 0.804 × 0.85 = **~49%** (range 44–54%).
|
||||
|
||||
### Essentials user: ~64%
|
||||
|
||||
@@ -199,11 +199,11 @@ basic sliders, crop, apply a preset, undo, export JPEGs.
|
||||
| Crop and straighten | 10% | 80% |
|
||||
| Presets and profiles | 8% | 80% |
|
||||
| Undo and history | 5% | 90% |
|
||||
| Their camera's raws: decode × default look | 10% | 63% (84% whose camera fully works × colour factor 0.75) |
|
||||
| Their camera's raws: decode × default look | 10% | 64% (85% whose camera fully works × colour factor 0.75) |
|
||||
| Export JPEG | 10% | 85% |
|
||||
| **Weighted depth** | 100% | **77.4%** |
|
||||
| **Weighted depth** | 100% | **77.5%** |
|
||||
|
||||
77.4% × 0.830 = **~64%** (range 59–69%).
|
||||
77.5% × 0.830 = **~64%** (range 59–69%).
|
||||
|
||||
### User evidence (GitHub issues, excluding the four maintainers)
|
||||
|
||||
@@ -297,6 +297,7 @@ P2 items 12–25 h; look tuning, performance, packaging, hardening 25–45 h.
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-11 | minor | EOS R5 Mark II CR3 verified (raw-parity.md): use-weighted share whose camera decodes 84% → 85%, RAW decoding 76% → 77%; full 60.6% → 60.7%, mainstream depth 71.8% → 72.1%, essentials depth 77.4% → 77.5%; the rounded numbers (~61%, ~49%, ~64%) are unchanged |
|
||||
| 2026-10-10 | minor | RAW measured model by model (raw-parity.md): RAW decoding (76%) and colour (35%) are explicit rows of the full sum (59% → 61%); mainstream uses the use-weighted 84% plus a ×0.85 colour discount (51% → 49%); essentials uses 84% × colour 0.75 (63% → 64%); beta ~14 points |
|
||||
| 2026-10-10 | minor | Full number restored to an additive weighted sum over the dimensions (~59%; method aligned with the standard, no new evidence); hours to ~95% for each of the three audiences; beta distance ~16 points |
|
||||
| 2026-10-10 | minor | Added mainstream practitioner (~51%) and essentials user (~63%) numbers; full ready re-derived from written weights × discounts (55% → 46%); user-issue evidence; beta hours 340–600 |
|
||||
|
||||
+3
-2
@@ -1,6 +1,6 @@
|
||||
# UI parity
|
||||
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (created; shortcut notes moved here from the checklist) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** minor (zoom row: Compare and Reference sharp at 1:1) · **Target:** Adobe Lightroom Classic 15.5 / Lightroom 9.6
|
||||
|
||||
How LightCraft's interaction compares with Lightroom's: views, panels, tools and handles, modifier
|
||||
keys, shortcuts, zoom, context menus and feel. Rows for individual menu items and shortcuts live in
|
||||
@@ -27,7 +27,7 @@ assessment and the notes that don't fit a row. Visual reference for agents: `pla
|
||||
| Crop, Upright, transform handles | handles, aspect lock, angle, overlays (O / ⇧O), guided Upright | ✅ all of them; overlay cycle on ⇧O | 85% | crop opacity of cut-out area (15.5) |
|
||||
| Masking tools | brush (A / B, flow, density, auto mask, pressure), linear / radial, range masks, AI masks, pins | brush, linear, radial, colour / luminance range, add / subtract / intersect / invert, masks panel, pins | 60% | brush live preview (#517 / #518), no pen pressure, no depth range, AI masks, Feather / Edge controls (15.5) |
|
||||
| Remove / heal | content-aware remove, heal, clone, visualize spots | heal, clone, auto source, visualize spots, red / pet eye | 65% | content-aware (patch synthesis) remove, generative remove, detect distractions |
|
||||
| Zoom and navigation | Fit / Fill / 1:1 / 2:1 … 11:1, Navigator, spacebar pan | Fit to 800%, pinch zoom, pan, Navigator-like | 60% | **original-pixel detail at 100% (preview capped at 2,560 px; P0 gap LR-VIEW-ZOOM)** |
|
||||
| Zoom and navigation | Fit / Fill / 1:1 / 2:1 … 11:1, Navigator, spacebar pan | Fit to 800%, pinch zoom, pan, Navigator-like | 60% | **original-pixel detail at 100% with a mask overlay or soft proofing** (no zoom window there; Detail, Before/After, Compare and Reference have them; P0 gap LR-VIEW-ZOOM) |
|
||||
| Library culling and metadata | ratings, flags, labels with keys, painter, keywording, metadata panel, filter bar | ratings / flags / labels and advance, keyword list and sets, keywording box, metadata fields, smart albums, filter bar | 85% | painter tool, quick collection, some Classic keys |
|
||||
| Shortcuts | Lightroom desktop and Classic keymaps | Lightroom-desktop keymap with Classic additions, editable | 85% | deliberate differences below; no Classic keymap layer |
|
||||
| Context menus | everywhere | 26 sites (grid, filmstrip, albums, folders, keywords, masks, presets…) | 70% | unaudited against Lightroom's menus item by item |
|
||||
@@ -78,4 +78,5 @@ under the UI command that wraps it (a no-op error); the UI command now wins.
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-10-11 | minor | Zoom row: Compare and Reference get zoom windows; what remains of LR-VIEW-ZOOM is overlays and soft proofing |
|
||||
| 2026-10-10 | major | Created: measured command and shortcut counts, assessment by area, estimates; moved the shortcut conflicts section here from the checklist |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Vertical text watermarks
|
||||
|
||||
Set `watermark.vertical` to `true` in export options, or choose Vertical in the export dialog. Text runs down each column; a newline starts the next column to its left. The size, nine anchors, inset, opacity, colour and shadow use the same controls as horizontal watermarks. Existing presets without `vertical` remain horizontal.
|
||||
Set `watermark.vertical` to `true` in export options, or choose Vertical in the export dialog. Text runs down each column; a newline starts the next column to its left. The size, nine anchors, horizontal and vertical insets, opacity, colour and shadow use the same controls as horizontal watermarks. Existing presets without `vertical` remain horizontal.
|
||||
|
||||
## Character cells
|
||||
|
||||
|
||||
+129
-1
@@ -1,6 +1,36 @@
|
||||
# What's new in LightCraft
|
||||
|
||||
## 2026-10-10
|
||||
## 2026-10-11
|
||||
|
||||
### Open Library knows a Lightroom catalog
|
||||
- Choosing a Lightroom Classic catalog folder in File → Open Library… (or one of its `Previews.lrdata` /
|
||||
`.lrcat-data` folders, or the `.lrcat`) no longer makes an empty LightCraft library inside it (issue #891). The
|
||||
library open now stays, and LightCraft says "This is a Lightroom catalog, not a LightCraft library" and offers
|
||||
**Import Lightroom Catalog…** with that catalog, or Cancel. `lightcraft-cli --library` refuses those folders too.
|
||||
|
||||
### Canon EOS R5 Mark II
|
||||
- Canon EOS R5 Mark II C-RAW files that opened as "preview only" (more than half of them, depending on the
|
||||
scene) now develop from their sensor data with every editing tool, like the camera's other raws. Photos
|
||||
imported earlier switch over with Photo ▸ Reload from Disk.
|
||||
|
||||
### Switching libraries
|
||||
- File ▸ New Library… asks for a name, then where to put it, and opens the new library (issue #902).
|
||||
- File ▸ Open Recent Library lists the last 8 libraries; ones that are gone (moved, drive disconnected) are greyed
|
||||
out. Clear Recent Libraries empties the list.
|
||||
- The open library's name shows under My Photos at the top of the left panel and in the window title.
|
||||
- `lightcraft-cli --library` given a library's own file (`catalog.snap`) opens its folder instead of failing with
|
||||
"File exists".
|
||||
- Agents use `app.newLibrary {path, name}`, `app.recentLibraries` and `app.clearRecentLibraries`.
|
||||
|
||||
### Edit in PhotoCraft
|
||||
- Right-click a photo ▸ Edit In ▸ Edit in PhotoCraft (or Photo ▸ Edit In, ⇧⌘E) renders a copy with your edits, stacks it
|
||||
with the original and opens it in PhotoCraft. LightCraft finds PhotoCraft where its installer put it (Windows,
|
||||
macOS, Linux packages, AppImage, Flatpak), and says so if it isn't installed, before making a copy.
|
||||
- The copy is a 16-bit PSD by default, which PhotoCraft saves straight back to with File ▸ Save; when you come back to
|
||||
LightCraft the copy shows what you saved.
|
||||
- Settings ▸ General ▸ External Editing: the format (PSD or TIFF), colour space and bit depth for PhotoCraft and for
|
||||
the other editor (Edit in Other Editor, now ⌥⌘E), and where PhotoCraft is if you use a portable copy.
|
||||
- Export can write PSD too (flat, 8- or 16-bit).
|
||||
|
||||
### Slideshow playback
|
||||
- View ▸ Slideshow (⌥⌘↩) now plays with Lightroom Classic's Playback settings: how long each slide stays up, a fade
|
||||
@@ -12,6 +42,61 @@
|
||||
- While it plays, moving the pointer shows a small control bar: previous, play / pause, next, the slide number and
|
||||
end. Space pauses, ← / → step (a cut), Esc ends. A slide's time starts once it is shown sharp.
|
||||
|
||||
## 2026-10-10
|
||||
|
||||
### Dialogs fit the window
|
||||
- A dialog taller than the window no longer runs off it (issue #781): on a 1366×768 screen the Export dialog's title
|
||||
and its Cancel / OK buttons were off screen, so it couldn't be confirmed. Now its options scroll and the title and
|
||||
buttons stay visible. Long lists in dialogs (What's New, All Metadata, Keyboard Shortcuts) get shorter to fit.
|
||||
Dialogs that fit look as before.
|
||||
|
||||
### Previews match exports on Mac screens
|
||||
- On macOS, photos are shown through the profile of the screen the window is on while no monitor profile is chosen in
|
||||
Settings ▸ Display (issue #787). On P3 screens previews used to look more saturated than the exported files in
|
||||
Preview or Photos. Moving the window to another screen is followed; **Use Screen Profile** goes back to it after
|
||||
choosing a profile file.
|
||||
|
||||
### Sidebar sections
|
||||
- Arrange the left sidebar your way: drag Albums, Local, By Date, Folders or Keywords by its header to move it (a line
|
||||
shows where it will go), or right-click a header for Move Up / Move Down.
|
||||
- Hide the sections you don't use: View ▸ Sidebar Sections, or right-click a header ▸ Hide This Section. Right-click
|
||||
the My Photos title to bring one back, or choose Reset Sidebar Sections. The arrangement is remembered.
|
||||
- In a long section, the header stays at the top of the sidebar while you scroll through its rows, so you always see
|
||||
which section you are in and can fold it from there.
|
||||
- Sections are easier to tell apart: a hairline between them, the fold chevron in the same column as the rows'
|
||||
triangles, a highlight under the pointer. A folded section shows how many rows it holds, counts of 0 are fainter,
|
||||
and a section with nothing in it says so instead of disappearing.
|
||||
- Recently Deleted is listed with All Photos, Recently Added and Picks under My Photos.
|
||||
- Keywords has a + in its header, like Albums: Create Keyword Tag without opening the Keyword List.
|
||||
- Agents: `view.sidebarSection`, `view.sidebarMoveSection`, `view.sidebarReset`; `ui.set` takes `sidebar`.
|
||||
|
||||
### Kelvin white balance for raws whose camera has colour matrices
|
||||
- Raws from the 52 camera models whose colour matrices LightCraft has measured (Sony ILCE-7M3, ILCE-7M4, ILCE-7RM3,
|
||||
Nikon D850, Canon EOS 5D Mark III and others) now show white balance in Kelvin, as a DNG does and as Lightroom
|
||||
does: As Shot reads the file's white (an ILCE-7M3 sample: 4268 K / +7), the Temp slider runs 2000–50000 K and the
|
||||
Daylight, Cloudy, Shade, Tungsten, Fluorescent and Flash presets set real temperatures (issue #730). The matrices
|
||||
only convert between the camera's multipliers and Kelvin; the colour look is what it was.
|
||||
- Nothing already edited changes: a Custom white balance set on the old relative scale renders pixel for pixel as
|
||||
before, and becomes the Kelvin value with the same look the next time it is edited. A photo imported earlier shows
|
||||
its real As Shot temperature once the file is opened (or Reload reads it).
|
||||
- Lightroom's `crs:Temperature` for these raws is read as the Kelvin it is (sidecars, Lightroom catalogs, presets);
|
||||
raws of other cameras keep the as-shot shift of issue #510. The two Kelvin scales come from different matrices and
|
||||
differ by about 7–10 % on the ILCE-7M3; which one is off is still open.
|
||||
|
||||
### Fujifilm lens corrections
|
||||
- Fujifilm RAFs from 59 camera models now get the lens correction the camera stores in the file: distortion (with the
|
||||
camera's own zoom to fill the frame) and vignetting, as with Panasonic and Sony files. The Optics profile controls
|
||||
adjust or switch them off, and DNG export keeps them. Lateral chromatic aberration data is not applied: our
|
||||
demosaic already lines the colour planes up. Details and measurements: `docs/fuji-lens-corrections.md`.
|
||||
|
||||
### Appearance
|
||||
- Settings ▸ Interface ▸ Appearance Mode: Sync with System, Dark Mode or Light Mode, above a light
|
||||
and a dark theme card, each previewing its theme (dark: Charcoal, the look so far, or Midnight;
|
||||
light: Silver or Paper). New and existing installs stay dark; following the system is opt-in.
|
||||
- The top bar's appearance button cycles Auto, Light, Dark (a monitor, sun or moon), and View ▸
|
||||
Appearance lists the modes and themes. On Linux, Auto follows the desktop portal's colour scheme
|
||||
as it changes, without polling.
|
||||
|
||||
### Activity stack
|
||||
- Long-running tasks show in one place, top left under the top bar, as in Lightroom Classic (issue #345): imports and
|
||||
folder scans, exports (and the contact sheet PDF), Synchronize Folder, Build / Smart Previews, Lightroom catalog import, Photo Merge, model downloads, the face scan,
|
||||
@@ -21,8 +106,39 @@
|
||||
- Quitting while a task that can be stopped is running asks first ("Quit Anyway" stops it).
|
||||
- Agents list and stop tasks with `activity.list` and `activity.cancel`; `ui.inspect` reports them as `activity`.
|
||||
|
||||
### Importing a deleted photo again
|
||||
- Importing a file whose photo is in Recently Deleted brings the photo back, with its edits, rating and albums
|
||||
(issue #706; before, the import skipped it as "already in the library" while the grid showed it deleted). The
|
||||
import review checks such files, marks them "Will be restored" and counts them as "in Recently Deleted"; the
|
||||
choice next to that (Restore them / Import as new / Leave them) now starts on Restore. Files dropped on the window
|
||||
restore the photo too and say "Restored 1 photo from Recently Deleted". A deleted photo whose file was moved is
|
||||
relinked to the file it is imported from. Browsing a folder in Local, Auto Import and a Lightroom catalog import
|
||||
never restore anything. Agents: `library.import` → `onDeleted` defaults to `restore` (`skip` keeps the old
|
||||
behaviour, `fresh` imports a clean copy and deletes the old record).
|
||||
|
||||
### Formats
|
||||
- File extensions match whatever their letter case: `IMG_0001.JPG` and `DSC_0002.NEF` as Nikon writes them, `.Jpg`
|
||||
after a rename, an `.XMP` sidecar. Adding photos or a folder, Browse Folder (Local), drag-and-drop, the CLI and the
|
||||
sidecar lookup always did; the Add Photos dialog on Linux did not until 0.5.0 (GTK and the desktop portal match
|
||||
`*.jpg` case-sensitively, so `.JPG` and `.NEF` files were hidden — issues #342 and #705).
|
||||
|
||||
### Reliability
|
||||
- Closing a library releases its lock explicitly (issue #680). On Linux and macOS the lock used to go with the file
|
||||
handle, and a program being started by another thread at that moment held a copy of the handle until it was fully
|
||||
launched, so reopening the library straight away could be refused with "This library is already open in another
|
||||
LightCraft program" although nothing had it open. Nothing in the shipped app is known to hit the window today; it
|
||||
made a lock test flaky, and the fix closes the window for good.
|
||||
|
||||
## 2026-10-09
|
||||
|
||||
### HDR PNG export
|
||||
- With **HDR output** on, a photo edited in HDR can now be exported as a **16-bit PNG** with the PQ curve in Rec. 2020,
|
||||
like the HDR AVIF: the same brightness, with SDR white at 203 cd/m², and content light levels recorded.
|
||||
- The colour is described by the PNG `cICP` chunk, not an ICC profile. HDR-aware viewers (current Chromium-based
|
||||
browsers, for example) show it in HDR; **viewers that don't read `cICP` show the PQ values as if they were sRGB**,
|
||||
so the picture looks flat and washed out there. That's how PQ PNG works, not a broken file: for a picture that
|
||||
looks right everywhere, export a JPEG with HDR output on (an SDR image plus a gain map).
|
||||
|
||||
### Smart album rules
|
||||
- The rule editor's field menu is grouped: Rating, Pick Flag, Color Label and Any Searchable Text at the top, then
|
||||
submenus for Source, File, Date, Keywords & People, Description, Camera Info, Location, Size, Develop and Assisted
|
||||
@@ -135,6 +251,13 @@
|
||||
into the display's own gamut; thumbnails and other previews are converted to it. Histograms, the preview caches
|
||||
and exports don't change. Agents use `app.displayProfile`. See [display profiles](display-profiles.md).
|
||||
|
||||
### Highlights and Shadows (process 2)
|
||||
- Highlights and Shadows keep the fine detail of the tones they move (issue #632): pulling Highlights down on a bright
|
||||
sky no longer flattens the twigs, lines and texture in it, and lifting Shadows keeps the texture of dark areas, much
|
||||
as Exposure would; the edges between bright and dark regions stay as sharp as before. This is rendering process 2:
|
||||
new photos (and Reset) get it, photos you already edited keep their look until you choose Photo ▸ Update to Current
|
||||
Process. See [process versions](process-versions.md).
|
||||
|
||||
### Masking
|
||||
- Drag anywhere inside the selected radial gradient's ellipse to move it, including rotated
|
||||
ellipses and radial components in compound masks. Edge and rotation handles keep their existing
|
||||
@@ -297,6 +420,11 @@
|
||||
show no dialogs and take `INSTALLFOLDER=...`.
|
||||
- If the desktop app can't open its window (for example when no graphics device can be used), it now says so in a
|
||||
message box that names the log file, instead of quitting without a trace (issue #260).
|
||||
- On Linux and FreeBSD, file dialogs (Import Photos…, Open Library, Export, presets, keyword lists…) are children of
|
||||
the LightCraft window. The xdg-desktop-portal picker used to open as a separate top-level window on Wayland
|
||||
("Failed to associate portal window with parent window" in the portal's log): tiling compositors put it beside the
|
||||
app, and with LightCraft fullscreen it was hidden altogether, so Import Photos… looked like a hang until the
|
||||
dialog was found and closed.
|
||||
- On macOS, single-key shortcuts that appear in the menu bar now work: E, C, H, M, ⇧P, I, K, D, ratings 0–5,
|
||||
labels 6–9, P / U and the rest did nothing, because macOS only passes ⌘ / ⌃ combinations and function keys
|
||||
to the menu bar and the app ignored those keys, assuming the menu bar would handle them. Keys outside the menus
|
||||
|
||||
+24
-4
@@ -108,8 +108,9 @@ application's renderer and is ignored: the photo keeps LightCraft's own process,
|
||||
|---|---|---|
|
||||
| `Exposure2012` | `light.exposure` | EV, 1:1 |
|
||||
| `Contrast2012`, `Highlights2012`, `Shadows2012`, `Whites2012`, `Blacks2012` | `light.contrast` … `light.blacks` | −100..100, 1:1 |
|
||||
| `HDREditMode` | `hdr.enabled` | `1` / `True` = HDR editing on, `0` / `False` off |
|
||||
| `WhiteBalance` | `wb.mode` | `As Shot`, `Auto`, `Daylight`, `Cloudy`, `Shade`, `Tungsten`, `Fluorescent`, `Flash`; other names → custom |
|
||||
| `Temperature`, `Tint` | `wb.temp`, `wb.tint` | Kelvin / tint as written for raws with a measured illuminant (DNG) and for presets. Raws LightCraft develops relative to their as-shot look (ARW, NEF, RW2, RAF, CR3, CR2, PEF, SRW: 6500 K / 0 means as shot) take the same mired shift from the packet's `AsShotTemperature` / `AsShotTint` instead; without that reference a custom Kelvin stays As Shot and is reported (`Temperature, Tint (custom white balance without AsShotTemperature: kept As Shot)`), because read on the relative scale it would be a large colour cast (issue #510) |
|
||||
| `Temperature`, `Tint` | `wb.temp`, `wb.tint` (+ `wb.scale: kelvin`) | Kelvin / tint as written for raws with colour matrices — in the file (DNG) or measured for the camera (`lightcraft_raw::spectral`: ILCE-7M3, D850, EOS 5D Mark III…, issue #730) — and for presets; the mapped settings say `"scale": "kelvin"` so the value stays Kelvin wherever it lands. Raws LightCraft develops relative to their as-shot look (ARW, NEF, RW2, RAF, CR3, CR2, PEF, SRW of other cameras: 6500 K / 0 means as shot) take the same mired shift from the packet's `AsShotTemperature` / `AsShotTint` instead; without that reference a custom Kelvin stays As Shot and is reported (`Temperature, Tint (custom white balance without AsShotTemperature: kept As Shot)`), because read on the relative scale it would be a large colour cast (issue #510). Note that LightCraft's Kelvin for a camera comes from its measured matrices, Lightroom's from Adobe's: the two scales differ (about 7–10 % on the ILCE-7M3), so the number is carried over, not the exact look |
|
||||
| `IncrementalTemperature`, `IncrementalTint` | `wb.temp`, `wb.tint` | rendered files: −100..100 on our relative scale (mired shift around 6500 K, same as the Temp slider); a rendered file with only `Temperature` shifts from `AsShotTemperature` like the raws above when it is present |
|
||||
| `Vibrance`, `Saturation` | `color.vibrance`, `color.saturation` | 1:1 |
|
||||
| `Texture`, `Clarity2012`, `Dehaze` | `effects.texture`, `effects.clarity`, `effects.dehaze` | 1:1 |
|
||||
@@ -140,8 +141,27 @@ application's renderer and is ignored: the photo keeps LightCraft's own process,
|
||||
|
||||
Values pass through our control specs, so anything outside our slider ranges gets clamped.
|
||||
|
||||
**Not mapped:** camera profiles and looks (`CameraProfile`, `Look`; we have our own profile set), local adjustments
|
||||
(masks, gradients, brushes), spot removal, red eye, lens blur, process-version 2010 field names, and AI features.
|
||||
**Not mapped:** camera profiles and looks (`CameraProfile`, `Look`; we have our own profile set), red eye, lens blur,
|
||||
process-version 2010 field names, and AI features. Local adjustments and spot removal are read as described below.
|
||||
|
||||
## Spot removal
|
||||
|
||||
Heal and clone spots stored as `crs:RetouchAreas` (one structure per spot, a mask component inside) and the older
|
||||
`crs:RetouchInfo` list (one `key = value, …` line per spot; read on its own when it is all a packet has) become our
|
||||
spots (`crates/engine/src/crs_masks.rs`, `spots`). Implemented from the public data model and by observing what a
|
||||
sidecar written for a healed photo contains.
|
||||
|
||||
| Field | Ours | Notes |
|
||||
|---|---|---|
|
||||
| `SpotType` | `mode` | `heal` (default) or `clone`; the removal method (`Method`) is ours |
|
||||
| `Masks` → `Mask/Ellipse` (`X`, `Y`, `SizeX`) | a circle spot | centre in image fractions; `SizeX` is a fraction of the width, our size of the long edge |
|
||||
| `Masks` → `Mask/CircularGradient` (`Top/Left/Bottom/Right`) | a circle spot | the box's centre and half-width |
|
||||
| `Masks` → `Mask/Paint` (`Dabs`, `Radius`) | a brushed spot | the dabs are the path |
|
||||
| `SourceX`, `OffsetY` | `source_offset` | the source's **absolute** x and y (despite the name); ours is the offset from the target. Only with `SourceState="sourceSetExplicitly"` (or no state); anything else leaves the source automatic |
|
||||
| `Opacity`, `Feather` | `opacity`, `feather` | 0..1 → 0..100 |
|
||||
|
||||
A spot whose mask is any other kind is reported as `Spot: <kind>`. Sizes are clamped to our spot range. A sidecar's
|
||||
spots replace the photo's, as its masks do.
|
||||
|
||||
## Local corrections (masks)
|
||||
|
||||
@@ -172,7 +192,7 @@ carry over are listed in `preset.import`'s `unmapped` as `Mask: <kind>`.
|
||||
|---|---|
|
||||
| Ours: `.lcpreset` | JSON `{"format": "lightcraft.preset", "version": 1, "presets": [{id, name, group, settings}]}`, where `settings` is a partial develop-settings object (only the groups the preset includes). A file can hold one preset or many, and every preset keeps its group. Import also accepts a bare preset object or an array of them. |
|
||||
| Export | `preset.export {path, ids?, group?}`: all user presets by default, or the given ids or one group. In the app: File ▸ Export Presets…, Presets panel ▸ ⋯ ▸ Export User Presets…, or right-click a group ▸ Export Group…. |
|
||||
| Import | `preset.import {paths, group?, dryRun?}`: files or folders (recursive): `.lcpreset`, `.xmp`, classic `.lrtemplate` (a Lua table: `value.settings` holds the same field names as `crs:`), photos that carry their edits in XMP ("DNG presets" from mobile apps; their crop, geometry and custom white balance are left out) and `.zip` bundles of any of these. Presets in a folder (or a folder inside a zip) go to a group named after it, unless the file names its own group. The result lists, per preset, the settings that couldn't be carried over (`unmapped`, e.g. `CameraProfile`, `Look`, local masks), and the app's toast names them; fields that only describe the preset or the photo (`Cluster`, `SortName`, `Description`, `SupportsAmount2`, `RequiresRGBTables`, `CropConstrainToWarp`, `OverrideLookVignette`, `AsShotTemperature` / `AsShotTint`, an off `HDREditMode`, empty `PointColors` slots) aren't listed. Older (process version 2010) fields — `Exposure`, `Contrast`, `FillLight`, `HighlightRecovery`, `Shadows`, `Brightness`, `Clarity`, `ToneCurve` — are approximated with today's sliders when a preset has no 2012-era fields. Dropping preset files on the window imports them too. In the app: File ▸ Import Presets…, or Presets panel ▸ ⋯ ▸ Import Presets…. A preset that's already there (same name, group and settings) is skipped. If an id clashes, the import gets a fresh `user.*` id, and built-in presets are never replaced. |
|
||||
| Import | `preset.import {paths, group?, dryRun?}`: files or folders (recursive): `.lcpreset`, `.xmp`, classic `.lrtemplate` (a Lua table: `value.settings` holds the same field names as `crs:`), photos that carry their edits in XMP ("DNG presets" from mobile apps; their crop, geometry and custom white balance are left out) and `.zip` bundles of any of these. Presets in a folder (or a folder inside a zip) go to a group named after it, unless the file names its own group. The result lists, per preset, the settings that couldn't be carried over (`unmapped`, e.g. `CameraProfile`, `Look`, local masks), and the app's toast names them; fields that only describe the preset or the photo (`Cluster`, `SortName`, `Description`, `SupportsAmount2`, `RequiresRGBTables`, `CropConstrainToWarp`, `OverrideLookVignette`, `AsShotTemperature` / `AsShotTint`, empty `PointColors` slots) aren't listed. Older (process version 2010) fields — `Exposure`, `Contrast`, `FillLight`, `HighlightRecovery`, `Shadows`, `Brightness`, `Clarity`, `ToneCurve` — are approximated with today's sliders when a preset has no 2012-era fields. Dropping preset files on the window imports them too. In the app: File ▸ Import Presets…, or Presets panel ▸ ⋯ ▸ Import Presets…. A preset that's already there (same name, group and settings) is skipped. If an id clashes, the import gets a fresh `user.*` id, and built-in presets are never replaced. |
|
||||
| XMP presets | Read with the `crs:` table above, with `crs:Name` as the name (falling back to the file name) and `crs:Group` as the group (falling back to "Imported Presets"). Only the fields the preset sets are included, so applying it leaves everything else alone and the Amount slider scales it like any other preset. We only read XMP presets; we don't write them. |
|
||||
|
||||
LightCraft ships no third-party presets. Its built-in presets are its own values (`crates/engine/src/presets.rs`).
|
||||
|
||||
Reference in New Issue
Block a user