The revision field and Catalog::apply said an error changes nothing and that the revision counts applied ops; a batch whose rollback fails changes the catalog and advances it.
* security: harden dev-tool scripts and the testkit skip-log test (CodeRabbit)
- tools/install-sam3.sh: the HF bearer token went through curl's argv, where
`ps` exposes it to other local accounts (CWE-214); it now travels through
a private 0600 temp config file removed on exit.
- packaging/macos/import-cert.sh: the decoded .p12 was written to a
predictable path in a shared TMPDIR, where a precreated attacker-readable
file would receive it (CWE-377); both cert and keychain are created inside
a mktemp -d 0700 directory now.
- docs/showcase/run.py: the reply reader copied whole responses with no size
cap (obvious quadratic-copy and unbounded-memory DoS). It accumulates in a
bytearray now and closes the connection past 128 MiB.
- tools/spectral_camera_table.py: alias/make/model strings and comments from
the rawtoaces-data checkout moved directly into Rust source; one newline in
a comment became Rust code. External text is now escaped as Rust literal
content and comments are normalized to one printable line.
- crates/testkit: appended_lines_accumulate reused a pip-predictable temp
dir by creating it with create_dir_all and carried on after failing to
remove its log path, so the append would follow an attacker-planted
symlink (CWE-59); it now fails the test if the directory exists instead of
testing through it, and removes any existing log path only after clearly
checking it, always failing the test instead of proceeding when removal fails.
Findings #10080/#10078 PR (rows 2, 6, 8, 9, 15 of the
storytold-lightcraft-agent-findings report).
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* security: bounded catalog keyword, repair, journal recovery work (CodeRabbit)
- keywords: from_file searched every hierarchical level for each flat keyword
and `same` lower-cased two strings per comparison — one lowered level set
instead (CWE-407, row 97).
- keywords: parse_keyword_list scanned each keyword's growing synonym list per
line — a lowered-synonym set per keyword instead (CWE-407, row 100).
- keywords: keyword_tree stored every level's prefix twice, so a photo XMP
keyword nested deeper than a list file accepts made stored bytes quadratic
in the level count; the tree caps at MAX_LEVELS, and keyword_list_text names
whatever lies deeper from the photos' and list's own spellings, so the file
still reads back (CWE-400, row 103).
- catalog: repair validated a folder chain once per album — memoized by chain
now, so a valid chain in a flat snapshot costs one walk overall (CWE-407,
row 101).
- store: journal append and truncate follow a planted catalog.log symlink;
both refuse any non-regular file at the name before writing through it
(CWE-59, row 107), like the staging writes since #1502.
- journal: resync re-validated the whole remaining suffix UTF-8 for every
candidate record start; a whole line is validated once (covering every
candidate in it), and an invalid line's re-checks stop at a 64 MiB budget,
with the tail then staying a torn one the damaged path keeps and reports
(CWE-407, row 109).
New tests: from_file case-insensitive level match, synonyms kept once, a
4096-level keyword cut at MAX_LEVELS, a link at catalog.log refused, and
resync over a fragment of 1024 record starts.
Findings rows 97, 100, 101, 103, 107, 109 of the
storytold-lightcraft-agent-findings report.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
---------
Co-authored-by: Codebuff <noreply@codebuff.com>
Catalog::apply noted the photos an op touches before checking they exist, and a batch that rolled back kept what its first ops and the rollback had noted. touched_since and aspects_since then named photos nothing changed, ids that do not exist among them, and a failed album op made earlier revisions unknown. apply now drops the notes made for the revision a failed op would have made and puts back the broad-change mark. A batch whose rollback itself fails keeps its notes and makes earlier revisions unknown, since the catalog did change.
Two catalog additions and one internal change, with nothing calling the new ones yet:
- Catalog::folder_records_located: folder_records_follow for a folder located where it
already is on disk (records of library folders at the destination stay).
- Catalog::local_records_of: the Local records of some files, split into those that can
give way and those that hold something of the person's.
- Find Missing Photos hashes a file in 64 KiB pieces instead of reading it whole (the same
hash).
A full count (AlbumTally::of, albums.list) keeps each loop-free smart album's photos for the pass and
answers "album is #N" from them, so an album tested by 51 others is worked out once instead of 52 times
per photo. Albums in or reaching a loop, or in a chain too deep for the depth limit, are evaluated by
their rules as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The left panel shows a count beside every album. Each catalog revision had
every album counted again, and a smart album's count asks every photo of
the library whether its rules hold: one pass over the library for each smart
album after each keyword, rating or slider tick (about 25 ms an album in a
library of 67,000 photos).
`AlbumTally` (`crates/catalog/src/tally.rs`) keeps the counts and, for each
smart album, which photos it holds. After a small edit each smart album asks
only the photos the edit touched (`Catalog::touched_since`); an album of
chosen photos is counted again only when a photo came, went or was deleted.
Counted from every photo as before: new rules, a change to an album, a stack
or the keyword list, an edit to more than 256 photos, another library, and
each new minute while an album asks about "the last N days".
What an album remembers is in proportion to the photos it holds: a bit a
photo id while ids are close together, a set when they are far apart.
The frame after a keyword in that library, both side panels open: about
54 ms -> 27 ms (156 ms before #1660).
Tests: 900 random edits against eight smart albums (keywords, flags,
ratings, labels, sliders, capture dates, file names, an album of chosen
photos) and an ordinary one, the followed counts compared with
`album_count` after each; a delete followed, three hundred photos counted
whole; the members' bits at the edges and for ids far apart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Everything derived from the catalog was keyed on its revision, which every
op raises. So each edit (a keyword, a rating, every tick of a slider) had
the grid's list queried and sorted again from every photo, the source
counted by a second full query, and the side panels tally the whole library
again. In a library of 67,000 photos the frame after one keyword took about
156 ms with both side panels open (an idle frame: 2 ms).
The catalog now remembers which photos its latest ops touched and what of
them they changed (`Catalog::touched_since`, `aspects_since`, `Aspects`). It
answers "not known" for an op that changes more than photos of its own (an
album, a stack, the keyword list), for changes too long ago, and for a
revision it did not make itself (a library just loaded).
- The view (`Session::visible`): when the same view is asked again after a
small edit, the photos the edit touched are taken out of the list it had,
asked about again, and put back at their place in the order
(`Catalog::follow`). 37 ms -> 0.6 ms. Worked out whole as before: more
than 256 photos touched, a change that is not photos' own, an album in its
own order, Recently Added, Missing Photos, and a view that asks about "the
last N days" (it changes with the clock too).
- The source's count: taken from the list when no filter is set, else
counted without sorting (`Catalog::count`). 37 ms -> 0.
- The side panels (`Caches`): the folder tree, keyword tree, date groups,
people's names and library counts are kept through an edit that changed
nothing they read. A rating or a slider changes none; a keyword leaves the
folders, dates and counts.
The frame after a keyword: about 156 ms -> 54 ms with both side panels
open, 71 ms -> 2 ms with the grid alone (`tests_view_cost.rs`, a synthetic
library, run with --ignored --nocapture).
Still paid on each edit: the count beside every album and smart album in the
left panel (about 25 ms for a smart album there), and the filter bar's lists.
Tests: 1,440 random edits of every photo-local kind across sources, sorts,
filters and the hold, each compared with the list worked out whole; 600
random edits comparing every kept tally with a fresh one; the clock; the
record itself, with its failure paths. `ui.inspect` -> `perf` reports
`viewFollowed` and `viewWorkedOut`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Catalog format v9 adds Photo.preview_checked and Op::SetContent.preview_checked:
the app version (CARGO_PKG_VERSION) whose decoder set preview_only, empty when
it isn't set. Import and content_op (Reload, Relink) stamp it; undo swaps it
back. Catalogs from before v9 read an empty stamp, so their preview-only photos
count as stale (Photo::preview_only_stale) and will be rechecked once.
The version guard test now also accepts a format version that only adds fields.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Go to History Step (and new Previous/Next History Step commands) loads the
step's settings and moves a per-photo cursor (`Photo.history_cursor`,
`Op::SetHistoryCursor`, catalog format v8; older catalogs open at the latest
step) instead of appending "History: …". Later steps stay listed, greyed in
the History panel, and can be returned to; a new edit made behind the end
drops them first. Each move is one undo step; undoing an edit restores the
dropped steps and the cursor. `history.list` returns `cursor`.
Also: spots.rs uses as_chunks (clippy chunks_exact_to_as_chunks).
Sources: own design (Lightroom Classic / Photoshop History panel behaviour, observed).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
stack_of, validate_stack and regular-album membership looked a photo up by
scanning every stack or the album's list, so an "In a Stack" or "Album"
smart album, or a query filtered to an album, cost photos x list length
(2.7 s and 0.6-0.8 s at 100k photos). Derived indexes (never saved) answer
them now: built on first use, kept current by the six ops that change
stacks or album photos in apply_inner, so undo, failed-batch rollback and
journal replay follow. A file with a photo in two stacks falls back to the
scan.
query() works out each sort key once (File Name was lower-cased twice per
comparison) and tests `only` as a set; smart-album passes prepare the
filter once; rules name flag, label and kind without formatting per photo.
A property test checks the indexes against a scan through random op
sequences, rollbacks, snapshots, replay, clones and undo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raws without matrices in the file (ARW, NEF, RW2, RAF, CR2, CR3, PEF, SRW,
ORF) were all edited on the relative white-balance scale (6500 K / 0 = as
shot), even when lightcraft_raw::spectral has measured matrices for the
camera. Those 52 bodies (ILCE-7M3, ILCE-7M4, D850, EOS 5D Mark III...) now
edit in Kelvin like a DNG and like Lightroom.
Design:
- One decision on both sides: spectral::find_camera("Make Model") backs
catalog::relative_wb_camera (Photo::relative_wb); files::WbScaleOf decides
the same from the decoded file. A corpus test asserts they agree.
Photo::fitted_look keeps the old predicate for smart previews.
- The matrices only convert: As Shot is the file's multipliers read through
the model's ColorMatrix1/2 (spectral::as_shot_white_through). The look
(JPEG fit / profile / spectral fallback) is unchanged, and the slider
adapts from that white with Bradford. The camera-space rebalance stays
reserved for files with their own matrices.
- Presets resolve to their Kelvin values (WbMode::preset, sources cited).
- The Color panel shows the white balance in effect (effective_wb).
- XMP: crs::Target::for_file takes the camera; matrix cameras read
crs:Temperature as Kelvin (marked "scale": "kelvin"), others keep the
#510 as-shot shift.
Migration (no look changes):
- WhiteBalance.scale: WbScale { Legacy (absent in JSON), Kelvin }. A
white balance without a scale keeps its old meaning. On a spectral-Kelvin
raw (SourceInfo.legacy_relative_wb) the pipeline composes the old
adaptation (6500 -> value) with the new one (as shot -> ...) in Bradford
cone space (local::legacy_white). Von Kries scalings compose exactly, so
the matrix matches to f32 and the pixels are identical
(pipeline legacy_relative_white_balance_renders_as_before_on_a_kelvin_raw,
media old_relative_white_balance_on_a_kelvin_camera_keeps_its_look...).
- Every white-balance writer (develop.set/adjust/quickAdjust/wb/wbPick/
reset) first runs local::normalize_wb: a legacy value becomes the Custom
Kelvin value with the same look and is marked Kelvin. Partials without a
scale (old presets, develop.merge) stay legacy.
- Catalog format 7: Op::SetAsShotWb replaces the 6500/0 as-shot recorded
by older imports when the file is next decoded or reloaded (system op,
not an undo step). Format-version test updated.
- The web wire carries legacy_relative_wb.
Docs: parity (LR-EDIT-COLOR-TEMP), camera-preview-colour, xmp-interop,
lightroom-catalog-import, control-protocol, mcp, whats-new.
Library folders can now carry what the library knows about them beyond
their photos: FolderRecord (for now a colour label), kept in
Catalog.folder_records under the folder's identity (folder_key) and
changed through Op::SetFolderRecord, one undo step. An empty record is
never kept; a path that names no folder is refused.
folder_records_follow builds the ops that carry a folder's records and
those of its subfolders to its new place on rename or move (dropping
stale records left at the destination), so they can be committed with
the photos' relinks. FolderNode reports the label.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A photo listed twice in the selection counted twice, and an id the
library doesn't have counted in "of": the chips, Will Export and the
Keyword List's tick boxes now count the distinct selected photos.
- The chips looked each keyword's spelling up by scanning every photo
(178 ms with 20,000 photos and 2,000 keywords selected, at every
change while selected): they now take the keyword list's spelling
(`Catalog::listed_path`) or the first one seen, 26 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The keyword list's own actions, each one undo step:
- create a keyword with its attributes, and tag photos with it at once;
one that exists already (listed or on a photo) is refused;
- move a keyword (with those below it) inside another or to the top
level; never inside itself or its children, and onto a name that is
taken only when merging is allowed (`CatalogError::KeywordExists`
otherwise, so the UI can ask);
- edit: rename its last level and set its attributes together; a taken
name is refused, merging being its own action;
- purge: take off the list the keywords no photo has.
Typed in any case, keywords keep the spelling the library has
(`keyword_path`).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keywords existed only as `a|b|c` strings on photos: one couldn't be
created before a photo had it, nor a parent made to organize under.
The catalog now has a keyword list (`Op::SetKeyword`, format version
4): a keyword listed on its own is in the tree with no photos, its
parents too, and carries Lightroom Classic's keyword tag options
(`KeywordInfo`: synonyms, include on export, export containing
keywords, export synonyms, person), exported by default. Undo restores
what was listed, as it was written. Version 3 libraries open and are
rewritten as 4.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A catalog written before the lens data was read at import (RW2/RWL from
v0.4.0) has none stored, and the lens correction stays off. Reload now
stores the fresh probe's lens data (new journaled, undoable
Op::SetEmbeddedLens; catalog format 4) and, for photos still at import
settings, switches the correction on as import does. Edited photos and
photos with a default-preset look keep their settings.
From final review: loop detection only followed Album rules. A smart
album's own album field (Filter.album, as set by the filter bar or
album.setRules {rules: {album}}) was ignored, so
"album.setRules {id: A, rules: {album: A}}" was accepted and A became
silently empty. On main it would have recursed to a stack overflow; the
guard already stops that.
Filter::albums_tested covers both, and album_reaches follows it.
Catalog::album_filter_problem reports a loop through the album field;
smart_album_problems (sidebar ⚠, albums.list) and album.setRules
(including fromView) use it. A problem with no rule position displays
without the "rule …:" / "#…" prefix.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From final review: the loop guard bounded the depth of smart albums
testing smart albums, but not repeated work. Each album tested by the
next one twice ("all of: album is X(k-1), album is X(k-1)") doubled the
cost per level, so 25 levels meant 2^25 evaluations per photo, enough
to freeze the sidebar's counts. Any agent or catalog file could build
such a chain.
rules::smart_album_holds remembers each smart album's answer for each
photo during one outermost question, and forgets them when it ends (a
drop guard, however it ends). A regression test counts a 26-deep
doubled chain in well under its 2 s limit; before the fix it ran for
minutes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review: the walk through smart-album references kept its seen-set
in a Vec, which is quadratic over long chains. The loop test's comment
now says what a saved loop does: inside it, the album being evaluated
counts as holding no photos.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An Album rule may now name a smart album: "Keywords contain travel"
and "Album isn't Excluded Photos" (red or rejected) leaves out whatever
Excluded Photos matches, as its rules change. Smart albums used to be
refused for this.
Loops:
- RuleSet::check_for(cat, owner) refuses an Album rule that would make
the owner include itself, directly or through other smart albums
(Issue::AlbumLoop). Catalog::album_reaches walks the references with
a seen-set; RuleSet::albums_tested lists them. album.setRules checks
for the album being edited, and smart_album_problems reports a loop
saved anyway.
- Matching never recurses without end: album_contains evaluates a smart
album's rules through rules::evaluating, a thread-local stack of
albums being evaluated with a 32-deep cap. An album already on it (a
saved loop) holds nothing, and the stack is restored by a drop guard
however evaluation ends. A test builds A → B → C → A and checks that
every album returns and reports the loop.
The issue "can't test a smart album" is replaced by the loop issue in
all seven locales. The sidebar counts already refresh while any smart
album uses "in the last…", so an album testing one is covered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rules are checked when they are made, but a saved smart album can go
bad later. The usual case is a rule testing an album that is then
deleted; it silently matches nothing.
Catalog::smart_album_problems(id) checks a saved album's rules as they
are now (RuleSet::upgrade, then RuleSet::check). It is empty for a sound
smart album, a plain album or no album. albums.list reports `problems`
for every smart album, e.g. ["rule 1: no album 8"].
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review of the value checks (false rejections and one silent
mismatch):
- In-the-last units: one table (unit_secs) for check and matcher. Any
case, singular or plural; no unit means days. An unknown unit is
refused and now matches nothing, where the matcher used to read it as
days, so "week" meant one day.
- Rating: refused only when no rating 0–5 could meet the rule, tested
with the matcher's own comparison ("is 9", "≥ 7", "between 6 and 9").
"< 6" is broad but valid. Message: "no rating 0–5 is ≥ 7".
- Catalog::label_from_name compares with Unicode lowercase, as rules
do, so a custom label "Été" is found as "été".
- Empty text hints at the operator that fits: "isn't empty" for
isn't / doesn't contain, "is empty" otherwise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Album.order and Op::SetAlbumOrder; album_children lists folders first, then
by place, then by name. Moving an album drops its place and undo restores it.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Review of the onDeleted work found that purging photos one at a time built each
op list against the unchanged catalog, so photos sharing an album or a stack made
Empty Recently Deleted (and Fresh imports) fail with NoStack or leave dangling
album entries. Catalog::delete_photos_permanently_ops builds one list for all of
them; Empty, Delete Permanently and the Fresh purge use it. Photos an import
restores now take part in album/albumName/selection, and undo labels say Restore.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Another way to explore the library next to By Date and Keywords: a tree of
the folders that hold imported photos, with counts (subfolders included),
under the disk or network share each is on. Local stays the way to browse
any folder on disk.
- catalog: Catalog::folder_tree and Filter::library_folder. Each disk is a
volume row (drive letter, Windows share, /Volumes/<share>, /media, /mnt,
gvfs, /net…), found from the stored paths alone, so an offline share costs
nothing. Levels that hold nothing and lead to one folder fold into one row;
depth is bounded. The tree and the filter read paths through one splitter
(separators, `.`, `..`, verbatim UNC prefix, repeated separators), and a
row that would cover other disks' photos is not choosable
- engine: library.folders, library.removeFolder (to Recently Deleted, one
undo step, no file touched; a whole disk or a folder spanning several
needs `disk: true`, the startup disk never), libraryFolder in
library.filter, chip; a chosen folder follows folder.rename / folder.move
and their undo
- ui: collapsible Folders section; the chosen folder's row is always opened
into view; right-click (row or triangle) Rename, Move, Show in Finder,
Remove from Library…; disk rows Remove Disk from Library…; both ask first
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
SortKey::Random orders photos by a splitmix64 rank of (Sort.seed, id): same seed gives
the same order, and added/edited photos never reshuffle the others. library.sort takes
an optional seed; library.shuffle picks a new one (below 2^53 for JSON clients). Menus
gain Random and Reshuffle; Ascending/Descending are disabled for Random. No effect in
Recently Added (always newest import first).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A dedicated view mode (View > People, and a toolbar button) shows a card per
person named on faces: a close-up of their largest face, the name and how many
photos they are in. A click shows that person's photos in the grid. Only the
cards on screen render their face. The sidebar section is gone (it would get
crowded).
`Catalog::people()` now returns `Person { name, count, photo, face }`, the
photo and face being the person's largest. `Session::face_job` renders a
square close-up through the engine (the photo's own look, crop replaced, the
smallest source level that keeps it sharp, cached like variant thumbnails); it
clamps hostile rectangles, with a test for corners, oversized, degenerate and
non-finite faces.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
main added Op::SetBrowsed, Catalog.browsed and Photo.local_baseline while
the snapshot still said version 1, so v0.2.0 read such a library in part
(unknown ops as log damage, unknown fields dropped on its next snapshot).
- journal::VERSION = 2 (now public), with a format table and rules in the
journal docs; a test with an exhaustive match over Op makes adding a
variant fail until it is assigned a version.
- Journal::open reads the snapshot header first and refuses a newer
version with CatalogError::Newer ("written by a newer version of
LightCraft"), even when its catalog wouldn't parse.
- A log record whose CRC matches but whose op doesn't parse is "newer",
not damage or a torn tail: refused before any file is modified.
- Opening an older (or snapshot-less) library writes a snapshot in this
format before anything is appended (LoadReport::upgraded_from); a new
library gets one at once, so v0.2.0 refuses it instead of reading a bare
log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed append (disk full, dropped share) could leave part of its batch in
catalog.log; the retry (same seqs) then shared a line with the fragment and
the next load treated it as mid-log damage, setting later edits aside.
- Journal::append cuts the log back to the last whole record on error (or,
if that fails too, before the next append).
- FsStore drops a failed append handle so the next attempt reopens the file;
truncate of a never-created file is a no-op.
- The loader recognises a CRC-valid record glued after a fragment (logs
written by older builds) and skips only the fragment.
- log_bytes now counts the newline added to an unterminated last record.
- close_library writes the closing snapshot even when appending fails; the
snapshot absorbs the queued ops (Journal::snapshot_with_unlogged).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two processes on one library each kept their own catalog and seq counter;
the last snapshot silently dropped the other's edits.
- lightcraft_catalog:🔒 LibraryLock holds an exclusive OS lock
(std File::try_lock: flock / LockFileEx) on <library>/catalog.lock for
the session; who holds it goes to catalog.lock.owner (separate file: a
Windows lock also blocks reads). Released on drop and by the OS when the
process dies, so leftover files from a crash never lock anyone out. File
systems without lock support open unprotected with a warning.
- Session::open_library takes the lock before reading anything and fails
with EngineError::LibraryInUse ("this library is already open in
LightCraft 0.x (process N on host)"); reopening the session's own
library hands its lock over.
- lightcraft-cli (mcp/run/snapshot --library) adds how to drive the
running app instead (--control + mcp --connect).
- Tests: in-process, leftover files, a killed child process, two engine
sessions, CLI e2e (mcp --library holds it, run --library is refused).
Existing tests that reopened a library while it was open now drop the
first session.
- docs: parity row LRC-LIB-CATALOG-LOCK, mcp.md, web.md (no guard across
browser tabs yet), whats-new.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- lightcraft_catalog::safe_file: one tested helper for crash-safe writes
(temp file in the same folder, sync, rename, folder sync; temp removed on
failure), write_new/write_new_unique that never replace a file and read
the result back, same_file, and a per-thread fault injection for tests.
- engine::originals::OriginalGuard: refuses a write whose target is a
catalogued photo's original or XMP sidecar (by file identity), whatever
the conflict policy or exact path. Used by run_batch (export, Export with
Previous, control/MCP exact paths), ui.render/ui.screenshot paths, merge
previewPath, MCP render_photo/screenshot paths (new query command
export.checkTarget) and lightcraft-cli render.
- The conflict policy now covers an export's sidecars together with the
file (Unique gives both the same free name, Skip skips when either is
taken).
- export::write_file replaces the plain std::fs::write in the desktop app,
CLI, MCP and engine writers; XMP sidecar writes use the shared helper.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Local browse records can be most of a large catalog (85k photos, 90 %
Local: 313.5 MB snapshot). Untouched Local records of folders whose last
browse is older than library.preferences forgetLocalDays (default 30,
0 = never) are now removed from the catalog when the library opens (and by
library.forgetLocal {dryRun?, days?}), as journaled, non-undoable ops, then
compacted in the background. Files and XMP sidecars are never touched;
browsing the folder again catalogues them afresh.
Untouched = a Local file record, not deleted, not a virtual copy, whose
state still hashes to the fingerprint taken when the browse catalogued it
(any change keeps it); records from before fingerprints need every
user-editable field empty/default. Kept too: records in albums (or as
cover), stacks, sources of virtual copies, visible/selected/edited ones,
ones referenced by undo/redo, ones with a smart preview.
Per-folder last-browsed times live in the catalog (Op::SetBrowsed,
normalised with folder_key; re-stamped at most hourly). Old catalogs load
unchanged; folders without a time are stamped now, so nothing is forgotten
on the first open after the upgrade.
persist_bench (85k photos, 76.5k Local, every 7th folder recent, 1 in 50
rated): 63.8k forgotten, plan 162 ms, snapshot 313.5 MB -> 94.9 MB.
Settings -> Performance -> Local folders.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At the 2,000-record threshold the UI thread serialised and fsynced the whole catalog (1-2 s
at 85k photos after streaming). Now `persist()` starts `Journal::snapshot_in_background`:
- the worker writes `catalog.snap` for op N from a clone of the catalog (photos are Arc-shared,
so the clone is cheap) through `Store::background_writer` (a second FsStore handle; MemStore
opt-in; none on wasm/web, which stays synchronous);
- appends go on as before (fsynced before every command returns) and are also kept in memory
as the "tail" (records > N);
- `Journal::poll` (from `persist` and the per-frame `persist_if_dirty`) sees the worker done and
atomically replaces the log with exactly the tail.
Crash points: before the snapshot rename -> old snapshot + whole log; between the rename and the
log rewrite -> new snapshot + whole log (records <= N skipped as stale, existing rule); after ->
new snapshot + tail. A failed snapshot keeps the log whole and retries after another
max_records records (no retry storm). At most one snapshot in flight; a synchronous snapshot
(close, Optimize Library), reopening a library and dropping the Journal first wait for it, so an
older snapshot can never replace a newer one. On-disk format unchanged.
Tests (crates/catalog/src/tests_background.rs, a gated store holds the worker mid-flight and
"crashes" by copying the files): edits during a background snapshot survive crashes at each
point and the trimmed log holds exactly seq N+1.. in order; failure loses nothing and backs
off; sync snapshot waits (no regression); drop waits; real FsStore with a leftover partial
temp file; a proptest over interleavings of edits / start / land / finish / sync snapshot /
crash. Mutations checked: trimming the log to empty, or not waiting in `snapshot`, fail them.
Engine: `threshold_compaction_runs_in_the_background` (crash copy mid-run, reopen without close).
`library.info` -> persistence gains snapshotRunning, failedSnapshots, maxBlockingMs and
lastSnapshot.{background, blockingMs}.
persist_bench, same session (load avg ~160), UI-thread time per threshold compaction:
85k photos: synchronous 0.85-1.7 s (median 1.25 / 1.34 s) -> background: blocking median
45 / 74 ms (clone 1.3-154 ms + log rewrite 9-23 ms); worker 1.2-2.4 s; 230-380
single-op appends during it at median 4-6 ms (same as without)
10k photos: synchronous 72-233 ms -> blocking median 12.5 / 14 ms
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
