* security: harden dev-tool scripts and the testkit skip-log test (CodeRabbit)
- tools/install-sam3.sh: the HF bearer token went through curl's argv, where
`ps` exposes it to other local accounts (CWE-214); it now travels through
a private 0600 temp config file removed on exit.
- packaging/macos/import-cert.sh: the decoded .p12 was written to a
predictable path in a shared TMPDIR, where a precreated attacker-readable
file would receive it (CWE-377); both cert and keychain are created inside
a mktemp -d 0700 directory now.
- docs/showcase/run.py: the reply reader copied whole responses with no size
cap (obvious quadratic-copy and unbounded-memory DoS). It accumulates in a
bytearray now and closes the connection past 128 MiB.
- tools/spectral_camera_table.py: alias/make/model strings and comments from
the rawtoaces-data checkout moved directly into Rust source; one newline in
a comment became Rust code. External text is now escaped as Rust literal
content and comments are normalized to one printable line.
- crates/testkit: appended_lines_accumulate reused a pip-predictable temp
dir by creating it with create_dir_all and carried on after failing to
remove its log path, so the append would follow an attacker-planted
symlink (CWE-59); it now fails the test if the directory exists instead of
testing through it, and removes any existing log path only after clearly
checking it, always failing the test instead of proceeding when removal fails.
Findings #10080/#10078 PR (rows 2, 6, 8, 9, 15 of the
storytold-lightcraft-agent-findings report).
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* security: bounded catalog keyword, repair, journal recovery work (CodeRabbit)
- keywords: from_file searched every hierarchical level for each flat keyword
and `same` lower-cased two strings per comparison — one lowered level set
instead (CWE-407, row 97).
- keywords: parse_keyword_list scanned each keyword's growing synonym list per
line — a lowered-synonym set per keyword instead (CWE-407, row 100).
- keywords: keyword_tree stored every level's prefix twice, so a photo XMP
keyword nested deeper than a list file accepts made stored bytes quadratic
in the level count; the tree caps at MAX_LEVELS, and keyword_list_text names
whatever lies deeper from the photos' and list's own spellings, so the file
still reads back (CWE-400, row 103).
- catalog: repair validated a folder chain once per album — memoized by chain
now, so a valid chain in a flat snapshot costs one walk overall (CWE-407,
row 101).
- store: journal append and truncate follow a planted catalog.log symlink;
both refuse any non-regular file at the name before writing through it
(CWE-59, row 107), like the staging writes since #1502.
- journal: resync re-validated the whole remaining suffix UTF-8 for every
candidate record start; a whole line is validated once (covering every
candidate in it), and an invalid line's re-checks stop at a 64 MiB budget,
with the tail then staying a torn one the damaged path keeps and reports
(CWE-407, row 109).
New tests: from_file case-insensitive level match, synonyms kept once, a
4096-level keyword cut at MAX_LEVELS, a link at catalog.log refused, and
resync over a fragment of 1024 record starts.
Findings rows 97, 100, 101, 103, 107, 109 of the
storytold-lightcraft-agent-findings report.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
---------
Co-authored-by: Codebuff <noreply@codebuff.com>
Catalog format v9 adds Photo.preview_checked and Op::SetContent.preview_checked:
the app version (CARGO_PKG_VERSION) whose decoder set preview_only, empty when
it isn't set. Import and content_op (Reload, Relink) stamp it; undo swaps it
back. Catalogs from before v9 read an empty stamp, so their preview-only photos
count as stale (Photo::preview_only_stale) and will be rechecked once.
The version guard test now also accepts a format version that only adds fields.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Go to History Step (and new Previous/Next History Step commands) loads the
step's settings and moves a per-photo cursor (`Photo.history_cursor`,
`Op::SetHistoryCursor`, catalog format v8; older catalogs open at the latest
step) instead of appending "History: …". Later steps stay listed, greyed in
the History panel, and can be returned to; a new edit made behind the end
drops them first. Each move is one undo step; undoing an edit restores the
dropped steps and the cursor. `history.list` returns `cursor`.
Also: spots.rs uses as_chunks (clippy chunks_exact_to_as_chunks).
Sources: own design (Lightroom Classic / Photoshop History panel behaviour, observed).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Raws without matrices in the file (ARW, NEF, RW2, RAF, CR2, CR3, PEF, SRW,
ORF) were all edited on the relative white-balance scale (6500 K / 0 = as
shot), even when lightcraft_raw::spectral has measured matrices for the
camera. Those 52 bodies (ILCE-7M3, ILCE-7M4, D850, EOS 5D Mark III...) now
edit in Kelvin like a DNG and like Lightroom.
Design:
- One decision on both sides: spectral::find_camera("Make Model") backs
catalog::relative_wb_camera (Photo::relative_wb); files::WbScaleOf decides
the same from the decoded file. A corpus test asserts they agree.
Photo::fitted_look keeps the old predicate for smart previews.
- The matrices only convert: As Shot is the file's multipliers read through
the model's ColorMatrix1/2 (spectral::as_shot_white_through). The look
(JPEG fit / profile / spectral fallback) is unchanged, and the slider
adapts from that white with Bradford. The camera-space rebalance stays
reserved for files with their own matrices.
- Presets resolve to their Kelvin values (WbMode::preset, sources cited).
- The Color panel shows the white balance in effect (effective_wb).
- XMP: crs::Target::for_file takes the camera; matrix cameras read
crs:Temperature as Kelvin (marked "scale": "kelvin"), others keep the
#510 as-shot shift.
Migration (no look changes):
- WhiteBalance.scale: WbScale { Legacy (absent in JSON), Kelvin }. A
white balance without a scale keeps its old meaning. On a spectral-Kelvin
raw (SourceInfo.legacy_relative_wb) the pipeline composes the old
adaptation (6500 -> value) with the new one (as shot -> ...) in Bradford
cone space (local::legacy_white). Von Kries scalings compose exactly, so
the matrix matches to f32 and the pixels are identical
(pipeline legacy_relative_white_balance_renders_as_before_on_a_kelvin_raw,
media old_relative_white_balance_on_a_kelvin_camera_keeps_its_look...).
- Every white-balance writer (develop.set/adjust/quickAdjust/wb/wbPick/
reset) first runs local::normalize_wb: a legacy value becomes the Custom
Kelvin value with the same look and is marked Kelvin. Partials without a
scale (old presets, develop.merge) stay legacy.
- Catalog format 7: Op::SetAsShotWb replaces the 6500/0 as-shot recorded
by older imports when the file is next decoded or reloaded (system op,
not an undo step). Format-version test updated.
- The web wire carries legacy_relative_wb.
Docs: parity (LR-EDIT-COLOR-TEMP), camera-preview-colour, xmp-interop,
lightroom-catalog-import, control-protocol, mcp, whats-new.
Library folders can now carry what the library knows about them beyond
their photos: FolderRecord (for now a colour label), kept in
Catalog.folder_records under the folder's identity (folder_key) and
changed through Op::SetFolderRecord, one undo step. An empty record is
never kept; a path that names no folder is refused.
folder_records_follow builds the ops that carry a folder's records and
those of its subfolders to its new place on rename or move (dropping
stale records left at the destination), so they can be committed with
the photos' relinks. FolderNode reports the label.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keywords existed only as `a|b|c` strings on photos: one couldn't be
created before a photo had it, nor a parent made to organize under.
The catalog now has a keyword list (`Op::SetKeyword`, format version
4): a keyword listed on its own is in the tree with no photos, its
parents too, and carries Lightroom Classic's keyword tag options
(`KeywordInfo`: synonyms, include on export, export containing
keywords, export synonyms, person), exported by default. Undo restores
what was listed, as it was written. Version 3 libraries open and are
rewritten as 4.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A catalog written before the lens data was read at import (RW2/RWL from
v0.4.0) has none stored, and the lens correction stays off. Reload now
stores the fresh probe's lens data (new journaled, undoable
Op::SetEmbeddedLens; catalog format 4) and, for photos still at import
settings, switches the correction on as import does. Edited photos and
photos with a default-preset look keep their settings.
Album.order and Op::SetAlbumOrder; album_children lists folders first, then
by place, then by name. Moving an album drops its place and undo restores it.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
main added Op::SetBrowsed, Catalog.browsed and Photo.local_baseline while
the snapshot still said version 1, so v0.2.0 read such a library in part
(unknown ops as log damage, unknown fields dropped on its next snapshot).
- journal::VERSION = 2 (now public), with a format table and rules in the
journal docs; a test with an exhaustive match over Op makes adding a
variant fail until it is assigned a version.
- Journal::open reads the snapshot header first and refuses a newer
version with CatalogError::Newer ("written by a newer version of
LightCraft"), even when its catalog wouldn't parse.
- A log record whose CRC matches but whose op doesn't parse is "newer",
not damage or a torn tail: refused before any file is modified.
- Opening an older (or snapshot-less) library writes a snapshot in this
format before anything is appended (LoadReport::upgraded_from); a new
library gets one at once, so v0.2.0 refuses it instead of reading a bare
log.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed append (disk full, dropped share) could leave part of its batch in
catalog.log; the retry (same seqs) then shared a line with the fragment and
the next load treated it as mid-log damage, setting later edits aside.
- Journal::append cuts the log back to the last whole record on error (or,
if that fails too, before the next append).
- FsStore drops a failed append handle so the next attempt reopens the file;
truncate of a never-created file is a no-op.
- The loader recognises a CRC-valid record glued after a fragment (logs
written by older builds) and skips only the fragment.
- log_bytes now counts the newline added to an unterminated last record.
- close_library writes the closing snapshot even when appending fails; the
snapshot absorbs the queued ops (Journal::snapshot_with_unlogged).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A command whose journal append fails now returns EngineError::NotSaved
("saved in memory but not written to disk: <reason>; LightCraft will retry")
instead of OK. The change stays applied and undoable, its ops stay queued and
are written by the next save (any command, or the frame loop with a 2 s
backoff). Queries and commands that change nothing are unaffected; a failed
compaction is reported in lastError, not as a failed command (and a failed
synchronous snapshot now backs off like a background one).
Session::unsaved(), library.info unsavedOps/unsavedError, ui.inspect unsaved;
the desktop UI toasts the failure and shows a warning on the top bar's cloud
icon until a save succeeds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At the 2,000-record threshold the UI thread serialised and fsynced the whole catalog (1-2 s
at 85k photos after streaming). Now `persist()` starts `Journal::snapshot_in_background`:
- the worker writes `catalog.snap` for op N from a clone of the catalog (photos are Arc-shared,
so the clone is cheap) through `Store::background_writer` (a second FsStore handle; MemStore
opt-in; none on wasm/web, which stays synchronous);
- appends go on as before (fsynced before every command returns) and are also kept in memory
as the "tail" (records > N);
- `Journal::poll` (from `persist` and the per-frame `persist_if_dirty`) sees the worker done and
atomically replaces the log with exactly the tail.
Crash points: before the snapshot rename -> old snapshot + whole log; between the rename and the
log rewrite -> new snapshot + whole log (records <= N skipped as stale, existing rule); after ->
new snapshot + tail. A failed snapshot keeps the log whole and retries after another
max_records records (no retry storm). At most one snapshot in flight; a synchronous snapshot
(close, Optimize Library), reopening a library and dropping the Journal first wait for it, so an
older snapshot can never replace a newer one. On-disk format unchanged.
Tests (crates/catalog/src/tests_background.rs, a gated store holds the worker mid-flight and
"crashes" by copying the files): edits during a background snapshot survive crashes at each
point and the trimmed log holds exactly seq N+1.. in order; failure loses nothing and backs
off; sync snapshot waits (no regression); drop waits; real FsStore with a leftover partial
temp file; a proptest over interleavings of edits / start / land / finish / sync snapshot /
crash. Mutations checked: trimming the log to empty, or not waiting in `snapshot`, fail them.
Engine: `threshold_compaction_runs_in_the_background` (crash copy mid-run, reopen without close).
`library.info` -> persistence gains snapshotRunning, failedSnapshots, maxBlockingMs and
lastSnapshot.{background, blockingMs}.
persist_bench, same session (load avg ~160), UI-thread time per threshold compaction:
85k photos: synchronous 0.85-1.7 s (median 1.25 / 1.34 s) -> background: blocking median
45 / 74 ms (clone 1.3-154 ms + log rewrite 9-23 ms); worker 1.2-2.4 s; 230-380
single-op appends during it at median 4-6 ms (same as without)
10k photos: synchronous 72-233 ms -> blocking median 12.5 / 14 ms
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Journal::snapshot serialised the catalog into a String and then copied it into a second
`format!`-ed String before writing (2x the snapshot size in memory, ~620 MB at 85k photos).
It now streams `{"format":…,"seq":N,"catalog":` + serde_json::to_writer + `}\n` through a
1 MiB BufWriter into the store (new `Store::write_atomic_with`; FsStore writes the temp file
directly, then flush + sync_all + rename + dir fsync as before; other stores buffer, as before).
Durability and format are unchanged: same temp + fsync + atomic rename; a failed write leaves
the old snapshot (and now removes the temp file); the bytes are identical to the old format
(test `streamed_snapshot_keeps_the_on_disk_format`, which also loads a legacy snapshot), so
no version bump. Also tested: `failed_streamed_write_keeps_the_old_file`.
persist_bench, 85k photos (310 MB snapshot), compaction at the 2,000-record threshold,
before/after binaries alternated in the same session (load avg ~130-170), building+compacting
only (PERSIST_BENCH_NO_LOAD):
compaction median: before 3.5 s / 4.6 s, after 1.9 s / 2.5 s
peak RSS: before 926 MB / 1263 MB, after 308 MB / 366 MB
10k photos peak RSS: 149-235 MB -> 61-68 MB (times too noisy to compare)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- catalog: Store trait (FsStore: temp+fsync+rename+dir fsync; MemStore),
Journal: CRC-checked JSON-lines op log with sequence numbers + atomic
snapshots; torn final record dropped and truncated, stale records after a
snapshot skipped, mid-log damage preserved as catalog.log.damaged-N.
- PushHistory op so each edit logs one history step (not the full history).
- engine: Session::open_library/persist/close_library; every top-level
command is fsynced before returning; presets.json + view.json;
library.info / library.compact commands.
- app: default library ~/Pictures/LightCraft Library (LIGHTCRAFT_LIBRARY or
--library DIR override), --memory for throwaway sessions, snapshot on quit.
- tests: proptest replay(log)==state with random snapshots/reopens, every
truncation of the last record, stale log, mid-log damage, FsStore, engine
restart without close.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
