mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-26 19:17:02 -06:00
Claude
86deb5b004
Import -> Copy (and Move, and Copy as DNG) handled one file after another on the import worker: copy, fsync, read back and hash, optionally decode and re-encode the raw as DNG. Every other CPU-heavy step of an import was already parallel (the probes, up to 8 threads), so with the probes cached by the review dialog's scan the transfer phase ran on one core. ImportJob::prepare_files now plans each file in order, as before (duplicates by path and content, sidecars, rename sequence numbers, folders, Kept notes) and collects the copies/moves, which then run on up to 8 threads (std::thread::scope; sequential on wasm32). Outcomes keep their places, so results, order, names and failure messages are those of a one-by-one import: - a file whose content matches a transfer still pending, or whose name could collide with one (same folder, same name family: IMG.jpg, IMG-1.jpg, img.dng), waits until that one has finished; - cancel stops before the next file; transfers not started leave no item; - a panic in one transfer fails that file instead of the batch; - Copy as DNG holds a memory-gate permit (6x the file size) while it runs. Copy verification and crash safety are unchanged (same copy_new / place). The desktop's import worker prepares batches of 2 x workers files (8..16, lightcraft_engine::import::batch_size) instead of 8, so a batch's slowest file holds back fewer threads; the browser build keeps 8. Measured (4-core shared VM, 200 procedural 3.7 MB JPEGs = 754 MB, warm page cache, throwaway benchmark not committed): - probe: ~80 ms CPU per file is the JPEG decode (DCT-scaled to 64 px), 1.2 ms the content hash; the dialog's scan: 4.4 s, 3.6 cores busy. - Copy, probes cached: copy+fsync+verify 1.59 s at 0.43 cores before, 0.94 s at 0.81 cores after. Copy, not cached: 6.45 s -> 5.51 s. - Add, probes cached: 2 ms prepare; nothing to parallelize. - commit of a batch on the UI thread (journal append + fsync): ~4 ms, flat for libraries of 0, 5k, 20k and 50k photos (~0.7 s for 1300). Copy as DNG could not be measured (no raw corpus here); it is the most CPU-heavy per-file step and was sequential. Test: copies_run_side_by_side_with_the_same_outcome (copies under the test fault Slow run on more than one thread; names, order and in-batch duplicates match a one-by-one copy; the outcome assertions also pass on the old code, the thread assertion fails there). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8