Copy import used exists() then std::fs::copy: no sync, no comparison, and
a file appearing at the chosen name in between was replaced. It now goes
through import_move::copy_new, which reuses Move's copy_verified: a new
file (create_new, retrying -1, -2… on AlreadyExists), synced and compared
byte for byte with the source; a copy that fails or differs is removed and
reported as a failed import. Copy as DNG's copy step uses the same path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Convert to DNG / Copy as DNG: the encoded DNG must decode to the same raw
data, then goes through safe_file::write_new_unique (temp file, sync,
read-back, hard-linked to the first free <stem>.dng / -2 / -3 name, never
replacing a file). On failure no DNG is left, the photo is not relinked
and the raw copy is kept.
- Photo Merge writes its DNG the same way.
- Smart previews are written atomically; smart::is_valid (header + complete
JPEG) decides "built", has_smart_preview and the offline fallback, so a
proxy cut short is rebuilt (reported as `repaired`); moving proxies to a
new folder never deletes a good one in favour of a damaged copy and
copies across drives atomically.
Stacked on the #93 branch (shared lightcraft_catalog::safe_file helper).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Renaming IMG_1.CR3 alone took the shared IMG_1.xmp along and left
IMG_1.JPG without it. A Stem-named sidecar that a sibling photo file
still uses is now copied (independently, never linked) instead of moved;
the last file renamed takes it. A rollback removes the copy.
Stacked on fix/issue-105-rename-rollback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saving (Cmd+S or auto-write) now merges LightCraft's properties into an
existing sidecar (lightcraft_meta::merge_xmp): owned properties (rating,
label, dc/IPTC/rights fields, lc:*) are replaced or removed, capture time
and GPS only when LightCraft has a value; everything else (crs:,
xmpMM:History, unknown namespaces, packet wrapper) is kept byte for byte.
The result is re-parsed and checked before it is written (temp + fsync +
rename). A sidecar that isn't valid XMP is copied to .xmp.bak-<time>
(never overwriting) before it is replaced; one that can't be read at all
is left alone and the save fails.
Shared stems: when several catalogued files map to one stem sidecar
(IMG.CR3 + IMG.JPG), it belongs to the raw (else the first by name) and
the others write and read <file>.xmp.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lightcraft_catalog::safe_file: one tested helper for crash-safe writes
(temp file in the same folder, sync, rename, folder sync; temp removed on
failure), write_new/write_new_unique that never replace a file and read
the result back, same_file, and a per-thread fault injection for tests.
- engine::originals::OriginalGuard: refuses a write whose target is a
catalogued photo's original or XMP sidecar (by file identity), whatever
the conflict policy or exact path. Used by run_batch (export, Export with
Previous, control/MCP exact paths), ui.render/ui.screenshot paths, merge
previewPath, MCP render_photo/screenshot paths (new query command
export.checkTarget) and lightcraft-cli render.
- The conflict policy now covers an export's sidecars together with the
file (Unique gives both the same free name, Skip skips when either is
taken).
- export::write_file replaces the plain std::fs::write in the desktop app,
CLI, MCP and engine writers; XMP sidecar writes use the shared helper.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rollbacks in move_file, apply_rename and undo/redo no longer ignore
errors: every move that can't be undone is reported (old -> new), and the
catalog follows the files that kept their new names (an undoable partial
rename; a logged change for undo/redo), so nothing shows as missing.
A file that can't go back keeps the sidecars that moved with it.
library.findMissing: same name + size confirmed by the stored content
hash first, then renamed files by size + content hash, then a unique
name + size match; several look-alikes are skipped and reported in
'ambiguous' instead of guessed. Files already used by a photo are skipped.
Stacked on fix/issue-95-case-only-rename (same functions).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Whether an existing target is the photo itself is decided by file identity
(dev+inode on Unix; an exact directory entry check elsewhere), not by a
lower-case name comparison, so on case-sensitive volumes img_1.JPG next to
IMG_1.JPG is a collision (-1 suffix). Moves use a no-replace rename (hard
link + unlink where supported) and case changes go through a unique temp
name. Tests model case-sensitive and case-insensitive volumes and check
the real disk.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scan() now accepts only <2 hex>/<32 hex>.jpg (matching shard) and the
cache's own <32 hex>.tmp<id> temp files, so prune/clear never delete and
size() never counts foreign files in a shared thumbs/ folder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Shippori Mincho's name table credits 'FONTDASU / Google Inc. / but / Adobe' — its kanji
come from Adobe's Source Han Serif, which our asset rules exclude even under the OFL
(CLAUDE.md: no Source Han fonts). BIZ UDMincho (Morisawa Inc., design TypeBank Co., Ltd.;
OFL-1.1, from google/fonts at 63833b7) is a Mincho of its own with no Adobe glyphs, and
smaller (6.2 MB vs 8.7 MB). Attribution, licence text and NOTICE updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Bayer layout of Canon CR2 files differs by model (GBRG on the 7D, 50D,
60D, 550D, 600D, 1200D, 1300D, 5D Mark II, 1D Mark IV; RGGB on the 40D,
5D Mark III, 6D, 5DS R and others). Read it from raw IFD tag 0xc5e0
(CR2CFAPattern: 1 RGGB, 2 BGGR, 3 GBRG, 4 GRBG), anchored at the full
decoded sensor; fall back to RGGB when the tag is missing. No model table.
Checked on 35 decoded CC0 raw.pixls.us CR2s (35 models): the tag agrees
with the mosaic's green-diagonal statistic on every file; the SensorInfo
border parity does not predict the layout. Corpus manifest gains the
reporter's samples; a corpus test asserts each layout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Decode Nikon Huffman-compressed NEF (compression 34713): lossless and lossy
(type 1 and 2) at 12 and 14 bits, with the maker-note 0x96 linearization
curve, predictor seeds and either byte order. Clean-room: 0x96 layout and
encoding-table sizes from prose write-ups (Clevy, Claff); the fixed Huffman
tables were recovered by black-box analysis of CC0 raw.pixls.us samples
(compressed vs uncompressed shots of the same scene) and verified on ~45
files from ~35 bodies. Lossy-after-split files stay preview only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A GPU render no longer hands back an image it cannot vouch for:
- set the wgpu device-lost callback (wgpu never reports a lost device as an
error, so a driver reset went unnoticed); readback waits with a timeout and
checks the poll result and map callback instead of blocking forever
- each render runs in OutOfMemory/Validation/Internal error scopes; errors
fail that render (OOM and limit failures for that render only, device
errors for the process) and it is redone on the CPU
- every buffer is checked against the device limit when created (no
validation error, no process-wide disable); LIGHTCRAFT_GPU_LIMITS=webgpu|<MiB>
creates the device with lower limits to reproduce small adapters
- the output buffer is cleared and the per-pixel kernel writes alpha 255:
any unwritten pixel (work silently dropped, the issue's all-black export)
fails the render; an all-black result is re-rendered on the CPU
- renders submit per stage and every 16 M invocations, the per-pixel stage
runs in ~4 MP bands: no long submission for a slow iGPU's watchdog to reset
- ui.inspect perf.gpuReason / perf.gpuFallback, app.gpu reason/lastFallback,
System Info and Settings say why the GPU is off (e.g. llvmpipe skipped) and
why the last render fell back
Tests: crates/gpu/tests/fallback.rs (limit, mid-render limit, validation
error, dropped work, black photo), engine export falls back to the CPU image,
banded 3000x2000 GPU/CPU equivalence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Local browse records can be most of a large catalog (85k photos, 90 %
Local: 313.5 MB snapshot). Untouched Local records of folders whose last
browse is older than library.preferences forgetLocalDays (default 30,
0 = never) are now removed from the catalog when the library opens (and by
library.forgetLocal {dryRun?, days?}), as journaled, non-undoable ops, then
compacted in the background. Files and XMP sidecars are never touched;
browsing the folder again catalogues them afresh.
Untouched = a Local file record, not deleted, not a virtual copy, whose
state still hashes to the fingerprint taken when the browse catalogued it
(any change keeps it); records from before fingerprints need every
user-editable field empty/default. Kept too: records in albums (or as
cover), stacks, sources of virtual copies, visible/selected/edited ones,
ones referenced by undo/redo, ones with a smart preview.
Per-folder last-browsed times live in the catalog (Op::SetBrowsed,
normalised with folder_key; re-stamped at most hourly). Old catalogs load
unchanged; folders without a time are stamped now, so nothing is forgotten
on the first open after the upgrade.
persist_bench (85k photos, 76.5k Local, every 7th folder recent, 1 in 50
rated): 63.8k forgotten, plan 162 ms, snapshot 313.5 MB -> 94.9 MB.
Settings -> Performance -> Local folders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A command whose journal append fails now returns EngineError::NotSaved
("saved in memory but not written to disk: <reason>; LightCraft will retry")
instead of OK. The change stays applied and undoable, its ops stay queued and
are written by the next save (any command, or the frame loop with a 2 s
backoff). Queries and commands that change nothing are unaffected; a failed
compaction is reported in lastError, not as a failed command (and a failed
synchronous snapshot now backs off like a background one).
Session::unsaved(), library.info unsavedOps/unsavedError, ui.inspect unsaved;
the desktop UI toasts the failure and shows a warning on the top bar's cloud
icon until a save succeeds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The macOS malloc_zone_pressure_relief call (returns freed allocator pages
after raw decodes) lived in identical copies inside the desktop app and the
CLI. It now lives in lightcraft-sysmem, an isolated L0 helper crate and the
workspace's only crate allowed unsafe: one #[allow(unsafe_code)] module, a
SAFETY comment, a safe API that returns bytes released (0 off macOS), and
tests (repeat calls, release after large frees, concurrent calls). Both apps'
alloc_release.rs now just install the safe hook, and their crate roots gain
#![forbid(unsafe_code)], so every production crate except sysmem forbids it.
AGENTS.md/CLAUDE.md name the exception, per craftrules standards/never-crash.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At the 2,000-record threshold the UI thread serialised and fsynced the whole catalog (1-2 s
at 85k photos after streaming). Now `persist()` starts `Journal::snapshot_in_background`:
- the worker writes `catalog.snap` for op N from a clone of the catalog (photos are Arc-shared,
so the clone is cheap) through `Store::background_writer` (a second FsStore handle; MemStore
opt-in; none on wasm/web, which stays synchronous);
- appends go on as before (fsynced before every command returns) and are also kept in memory
as the "tail" (records > N);
- `Journal::poll` (from `persist` and the per-frame `persist_if_dirty`) sees the worker done and
atomically replaces the log with exactly the tail.
Crash points: before the snapshot rename -> old snapshot + whole log; between the rename and the
log rewrite -> new snapshot + whole log (records <= N skipped as stale, existing rule); after ->
new snapshot + tail. A failed snapshot keeps the log whole and retries after another
max_records records (no retry storm). At most one snapshot in flight; a synchronous snapshot
(close, Optimize Library), reopening a library and dropping the Journal first wait for it, so an
older snapshot can never replace a newer one. On-disk format unchanged.
Tests (crates/catalog/src/tests_background.rs, a gated store holds the worker mid-flight and
"crashes" by copying the files): edits during a background snapshot survive crashes at each
point and the trimmed log holds exactly seq N+1.. in order; failure loses nothing and backs
off; sync snapshot waits (no regression); drop waits; real FsStore with a leftover partial
temp file; a proptest over interleavings of edits / start / land / finish / sync snapshot /
crash. Mutations checked: trimming the log to empty, or not waiting in `snapshot`, fail them.
Engine: `threshold_compaction_runs_in_the_background` (crash copy mid-run, reopen without close).
`library.info` -> persistence gains snapshotRunning, failedSnapshots, maxBlockingMs and
lastSnapshot.{background, blockingMs}.
persist_bench, same session (load avg ~160), UI-thread time per threshold compaction:
85k photos: synchronous 0.85-1.7 s (median 1.25 / 1.34 s) -> background: blocking median
45 / 74 ms (clone 1.3-154 ms + log rewrite 9-23 ms); worker 1.2-2.4 s; 230-380
single-op appends during it at median 4-6 ms (same as without)
10k photos: synchronous 72-233 ms -> blocking median 12.5 / 14 ms
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Journal::snapshot serialised the catalog into a String and then copied it into a second
`format!`-ed String before writing (2x the snapshot size in memory, ~620 MB at 85k photos).
It now streams `{"format":…,"seq":N,"catalog":` + serde_json::to_writer + `}\n` through a
1 MiB BufWriter into the store (new `Store::write_atomic_with`; FsStore writes the temp file
directly, then flush + sync_all + rename + dir fsync as before; other stores buffer, as before).
Durability and format are unchanged: same temp + fsync + atomic rename; a failed write leaves
the old snapshot (and now removes the temp file); the bytes are identical to the old format
(test `streamed_snapshot_keeps_the_on_disk_format`, which also loads a legacy snapshot), so
no version bump. Also tested: `failed_streamed_write_keeps_the_old_file`.
persist_bench, 85k photos (310 MB snapshot), compaction at the 2,000-record threshold,
before/after binaries alternated in the same session (load avg ~130-170), building+compacting
only (PERSIST_BENCH_NO_LOAD):
compaction median: before 3.5 s / 4.6 s, after 1.9 s / 2.5 s
peak RSS: before 926 MB / 1263 MB, after 308 MB / 366 MB
10k photos peak RSS: 149-235 MB -> 61-68 MB (times too noisy to compare)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An unchanged grid frame used to copy every visible id, hash them twice
(date-run and layout cache keys), clone the date runs, build a full
Vec<Cell>, rebuild the stack index, loop over every cell for the
viewport test and, in Local folder views, look up every photo to count
the browsed ones for "Add N to My Photos".
- engine: Session keeps the visible list as Arc<[PhotoId]> with a
process-unique generation that changes whenever the list is
recomputed; `visible_shared()` hands both out without copying.
- grid: date runs, layout, stack index and the Local count are cached
by that generation (GridCache; replaces the id-hashing date_runs and
grid_layout caches). The layout gains a row index (GridRow), so the
visible + prefetch rows and the date headers are found by binary
search; the sticky header too.
- GridStats counters (frames, cells visited, view/layout/stack-index
builds, last frame time) for the bench and tests.
Measured with the new `cargo run --release -p lightcraft-ui-egui
--example grid_bench` (synthetic library, 1400x900, thumbs 160, missing
files so no decoding is timed, scrolled to the middle, 120 unchanged
frames; machine at load ~200-250, so medians of the whole app frame,
before/after interleaved in the same session):
photos layout source median frame ms allocs/frame cells visited
before -> after before -> after before -> after
200 photo all 1.29 -> 1.28 4747 -> 4729 200 -> 93
200 square all 1.09 -> 1.09 4771 -> 4753 200 -> 74
10 000 photo all 1.52 -> 1.41 5425 -> 4911 10000 -> 100
10 000 photo local 1.90 -> 1.42 5475 -> 4948 10000 -> 100
10 000 square local 1.62 -> 1.07 5365 -> 4838 10000 -> 76
85 000 photo all 2.41 -> 1.29 8996 -> 4729 85000 -> 93
85 000 square all 2.29 -> 1.08 9020 -> 4753 85000 -> 74
85 000 photo local 17.6-34.0 -> 1.32 9049 -> 4766 85000 -> 93
85 000 square local 16.2-25.0 -> 1.10 9073 -> 4790 85000 -> 74
Frame cost no longer depends on the number of results; what is left
(~1 ms, ~4.7k allocs) is drawing the ~75-100 cells near the screen.
Headless snapshots of the demo grid (justified with a sticky header,
after keyboard navigation, square) are byte-identical before/after.
Tests: layout rows cover every cell in order and the binary search
matches a linear scan (square + justified, groups); a 3000-photo
headless library checks that unchanged frames build nothing and visit
< 300 cells, plus date headers, scroll-to-selection, stack badges,
keyboard navigation and a rotation re-laying the grid.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
library.import mode `move` (UI: Transfer ▸ Move, MCP import tool): reuses
Copy's destination, folder organisation (incl. custom templates) and rename
template. Each photo and its XMP sidecars (both namings) are placed without
touching the source (hard link on the same volume, else a new file that is
copied, synced and compared byte for byte; taken names get -1, -2…, never
overwritten). Sources are removed only after the catalog records are
committed and saved, and only when the destination is still intact; failed,
duplicate, unchecked and in-library files keep their sources; sources that
can't be removed are kept and reported (`kept`). Undo removes the catalog
records and leaves the moved files at the destination.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brandon Thomas