8 Commits

Include renames

Author	SHA1	Message	Date
5 people	a220d3eb68	LR-EDIT-DETAIL-DENOISE: built-in BM3D denoising with GPU acceleration (#2982) ... * LR-EDIT-DETAIL-DENOISE: AI Denoise works out of the box with a built-in denoiser Turning AI Denoise on no longer needs an ONNX model and a denoise-model.json. Without a model of the user's own, the engine runs a built-in pure-Rust denoiser (noise measured per tile, non-local means on the Bayer planes, edge-directed demosaic) through the same tiling, cache, queue, preview and export path as a model. Installing a model stays optional; Stop using returns to the built-in one, and the id `builtin` is reserved. The set-up prompt and its resume-after-install flow (model_setup.rs, PhotoState::NoModel) are removed. Tests updated for the new behaviour, with new tests for the built-in path; docs, trackers and locale catalogs updated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: built-in denoiser cleans the colour planes together and adapts to the noise The built-in denoiser now measures the photo's noise once from the whole mosaic (the model prepares the photo before its tiles run) instead of per tile, stabilises the variance (generalised Anscombe), rotates the four Bayer planes with a Hadamard transform and runs a non-local-means filter whose patch distance spans all planes, so colour blotches are cleaned with the brightness. It cleans gently at low noise to keep fine detail, and harder, with a second pass that matches on the first pass's cleaner picture, as the noise grows. Cached pictures made by the first version are made again (builtin version 2). Adds an opt-in benchmark (tests/denoise_builtin_quality.rs, public CC0 raw plus noise of known level, scored against the original) and tests for the whole-photo noise estimate, strong noise, bad noise values and the strength ramp. On the Sony A7 III sample the result is 3.6, 7.0, 9.1 and 9.7 dB above the noisy picture at light, medium, heavy and extreme noise, against 2.9, 6.0, 6.5 and 5.8 dB for the first version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * contributors: add esli1283 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: docs say how to run the built-in denoiser's quality benchmark Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: built-in denoiser cleans colour further, keeps more texture and leaves no isolated specks Colour-difference planes get a wider pass guided by the cleaned brightness plane, the first-pass threshold is lower (more texture kept), and cells with no matching patch lean on a looser average in heavy noise. Skips negligible weights for speed; bumps builtin::VERSION to 3. Benchmark adds a texture figure; on the public A7 III raw the gain over the noisy picture is +3.7 / +7.2 / +9.7 / +10.4 dB (was +3.6 / +7.0 / +9.1 / +9.7). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Windows: tests build paths one component at a time and doc_path has no unused variable On Windows `cargo xtask ci` failed before it reached any of the work under test: - doc_path.rs bound the gvfs path outside the cfg(not(windows)) block that uses it, an unused variable under -D warnings. - tests_import and the MCP headless discovery test joined "trip/day2" onto a temp folder, giving mixed separators, and compared them with the normalised paths the folder walk returns. They now join each component. Test and lint changes only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: built-in denoiser filters with block-matching collaborative filtering, on the GPU The built-in denoiser's filter is now block matching with collaborative filtering (BM3D / CBM3D, written from the paper) instead of non-local means. On the public Sony A7 III benchmark with added noise it scores 0.25 to 0.6 dB higher at every noise level and keeps more texture at medium to extreme noise (light noise keeps slightly less: 0.58 against 0.62), as fast as before on the processor. docs/denoise.md records the comparison with non-local means and weighted nuclear norm minimisation (tried, not kept: about +0.03 texture for ten times the time) and what the tuning showed. - denoise-core: bm3d.rs (matching on the brightness plane, 2-D DCT plus Haar across the group, hard threshold then Wiener, hard threshold rising with the noise). Builtin::run_with takes the filter stage, so every other step is shared between devices. Tuning.bm3d selects it; non-local means stays as the fallback. builtin::VERSION is 4, so cached pictures are made again. - gpu: bm3d.rs and wgsl/bm3d_*.wgsl run the grouping, the filtering and the putting back on the graphics card, in the same order as the reference (the results agree to rounding: at most 0.1 noise deviation at a few tie blocks, 4e-6 on average, identical every run). Noise measurement, stabilising and the demosaic stay on the processor. FXC (DX12) needs the working arrays at module scope and no dynamically indexed vector: documented in docs/gpu-pipeline.md. - engine: the card is set up once per process for the first photo that may use it, checked against the processor on a check tile (in the filter's own domain: the demosaic's direction choice can flip a pixel on a tiny difference), timed, and used where it is faster. A tile it gets wrong runs on the processor and three stop it. Settings' Run on choice, device line and "Try the graphics card again" now work for the built-in denoiser. Its set-up marker lives in the models folder. - The benchmark takes LC_GPU=1, LC_NLM=1 and LC_B_* for sweeps. On an RTX 4080 SUPER a 256-cell tile takes 22 ms on the card against 408 ms on one processor thread, and the 1536 x 1024 benchmark crop 0.13 to 0.24 s against 1.0 to 2.1 s on four threads, with identical scores. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: the built-in denoiser measures the noise per colour plane and sets its strength from it The first noise estimator read a picture's texture as noise: it never reported less than about 0.003 at mid-grey whatever the real noise, so the denoiser always believed a photo was about as noisy as ISO 12800. The threshold curve was tuned on that over-estimate, and its levels (0.024 to 0.14) are far noisier than real Canon bodies produce (about 0.0025 at ISO 400 up to 0.038 above ISO 25600, measured on a private library; only aggregates are recorded in the docs). - estimate_noise: each colour plane is read on its own (they sit at different levels in a raw), a second-difference operator cancels flat and sloping signal, 12 x 12 blocks, the quietest 5 % at each brightness divided by the bias of that quantile of pure noise (0.67, measured), a constrained line fit. The first version stays as estimate_noise_v1 to compare with. On 30 public pictures with known added noise the 10th to 90th percentile of estimated / true deviation at mid-grey is 0.97 to 1.29 at the ISO 6400 level against 0.94 to 1.96. - bm3d::Params::lambda is a curve of the noise (1.2 at 0.0028 up to 3.5 at 0.14, straight against its logarithm) instead of two end values and a strength, and plan / filter take the noise level; the GPU path shares plan, so CPU and GPU agree as before. Fitted on clean low-ISO photos of a Canon EOS R and R6 Mark III with noise of their measured levels added: the lowest threshold within about a quarter of a decibel of the best at each level. - Result (PSNR, EOS R / R6 Mark III, 10 + 5 photos): ISO 25600 level 38.36 / 38.86 from 36.18 / 36.21, ISO 51200 level 33.83 / 34.45 from 28.42 / 28.78, ISO 6400 39.93 / 40.69 from 39.30 / 39.74, quiet photos no longer over-smoothed; the noise left in the result falls from 1.3 to 3.1 down to 0.73 to 0.99. - Tests: threshold curve, flat versus textured pictures, no-flat-area pictures, clipped / NaN mosaics; an opt-in accuracy harness on the public corpus (denoise_noise_estimate) with a calibration of the quantile bias; the benchmark takes LC_REAL, LC_BIN, LC_FULL, LC_EST and LC_B_LOFF. - builtin::VERSION is 5, so cached pictures are made again. docs/denoise.md has "How the noise is measured" and "Fitted on real cameras". Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: GPU workspace pool cannot miss a wake-up; benchmark path and parity date fixed Review findings (CodeRabbit) on the built-in denoiser pull request: - bm3d::Bm3dGpu::forget and nn::NetRunner::forget changed the workspace count without the free-list lock. A take() that had found no free workspace and read made == WORKSPACES, but had not gone to sleep yet, could miss the wake-up and wait for good on a slot that was already free. forget now changes the count under the free lock, in the same order as take(), then notifies. - A test for each runner holds the free lock and checks that forget waits for it; both fail on the old code and pass now. - docs/denoise.md: the benchmark command resolves the raw file from the repository root, since the test runs from the crate's folder. - docs/target-app-parity.md: Last updated matches its newest revision-history entry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: doc comments for the functions this branch adds CodeRabbit's pre-merge docstring check counted 49.72 % of the functions this pull request touches as documented (threshold 80 %). Every function the branch adds, the built-in denoiser's block-matching and non-local-means filters, its noise estimator, the GPU runner and their tests, now says what it does or what the test proves. Older functions the branch only touched are left as they were. Comments only: no code line changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: make the GPU set-up marker's folder; fix the benchmark's default path The crash guard was lost when the models folder did not exist yet. The opt-in quality benchmark resolved its default raw file against crates/engine. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: enforce the cache limit without a models folder The built-in denoiser makes products when there is no models folder, but eviction returned early in that case, so the cache could outgrow its limit. It now applies the default limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * LR-EDIT-DETAIL-DENOISE: the cache-limit test needs no 20 GiB file off Unix The no-models-folder test now checks the limit itself on every platform and keeps the oversized sparse file for Unix only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * S0002: review fixes for the built-in denoiser Tell the runner a photo's noise only when it could be measured, run prepare() after the work gate with a cancel check, reword the Settings hint for builds without a models folder, drop untranslated placeholder locale entries (they fall back to English) and restore the cs.json spacing; rustfmt. Co-Authored-By: Claude Code <noreply@anthropic.com> * S0002: fix review findings: command descriptions, stale gpu test, self-test noise The denoise.models.select and denoise.toggle descriptions now describe the built-in fallback. Drop our workspace-forget test, which read fields that upstream's Workspaces pool (#2631, same fix) replaced and so did not compile; upstream's own tests cover it. The built-in self-test is told a noise instead of measuring a 16-cell tile too small to measure. Co-Authored-By: Claude Code <noreply@anthropic.com> --------- Co-authored-by: esli1283 <esli.aaron@outlook.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Claude Code <noreply@anthropic.com> Co-authored-by: FredApps <FredApps@users.noreply.github.com>	2026-11-05 09:36:22 +01:00
aiongg	f3bb7a9f30	CI: one test binary per crate, line tables only, one WASM check (#2710) ... * CI: dev profile keeps line tables for workspace crates and no debug info for dependencies * Control server tests: persistent_requests_and_slow_replies_outlive_the_idle_read_timeout no longer races the idle timeout Each request (and the closing half-shutdown) reaches the socket before the worker reads, so on a slow machine only the reply waits outlast the 100 ms idle timeout, as the test intends. The idle test sends its input the same way, and the no-frames test allows 5 s instead of 2 s to give up. * CI: one integration-test binary per crate Each crate's tests/*.rs become modules of tests/it/main.rs (51 test files into 13 binaries, plus 6 that stay their own). Inner #![cfg] attributes move to the mod lines. Files that change state for the whole process keep a binary of their own: codecs jxl_tail_heap and raw sr2_heap (dhat global allocator), gpu shutdown, shutdown_first and memory, ui-egui shutdown. Docs, comments and the GPU probe name the new paths and targets (--test it <module>::); the CI report names an it binary's skips after the crate. * CI: tests under nextest, doctests apart, one wasm check xtask ci runs the tests with cargo nextest run --lib --bins --tests when cargo-nextest is installed, else cargo test with the same targets and a one-line notice; doctests run after them with cargo test --workspace --doc. The heif and hardware GPU steps use the same runner. Examples stay out of the test build (clippy --all-targets still checks them). Concurrent tests under nextest follow the RUST_TEST_THREADS cap (NEXTEST_TEST_THREADS). The wasm check is one cargo check --keep-going over the same crates, and clippy runs with --keep-going. * CI: tests run under cargo test by default, nextest with LIGHTCRAFT_NEXTEST=1 (per-process scene cache)	2026-10-23 13:34:30 -04:00
justinjohn0306	de038ca3d4	#1880: allow 102 MP AI denoise inputs	2026-10-15 16:31:06 +05:30
akvaithiandClaude Opus 5.5	88f424e7e7	LR-EDIT-DETAIL-DENOISE: share tiles with a server only when it is faster ... Measured with RawNIND on a 20.8 MP raw: an Apple M3 does a tile in 116 ms, an RTX 3060 Ti over Tailscale 345 ms there and back; sharing blindly made the export take 34 s instead of 6.4 s while the photo waited for the server's last tiles. The greeting now times a check tile there and back, and tiles are shared only when that beats this computer's own time per tile (the card's, or the CPU's with its threads counted as `card_is_faster` does). Otherwise `denoise.status` → `device.remote` says `slower` with both times. With the server configured but slower the export takes 7.2 s (greeting + check tile). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 15:53:39 -05:00
akvaithiandClaude Opus 5.5	84b8995d7b	LR-EDIT-DETAIL-DENOISE: run tiles on another computer's graphics card ... - `lightcraft_denoise::remote`: protocol v1 over TCP (JSON lines + byte-shuffled, deflated half-float tiles), a `Remote` client (connection pool, four slots, greeting checks the model's SHA-256 and tile size, gives up after three failed tiles) and `serve` (shared token compared in constant time, size caps including payloads that inflate past their tile, connection limit, timeouts). - Engine: with LIGHTCRAFT_DENOISE_URL / LIGHTCRAFT_DENOISE_TOKEN set, the loaded model greets the server on first use and adds its slots to this computer's tiles at once; a tile goes to a free server slot, else runs here, and a tile the server fails runs here too. `denoise.status` → `device.remote` says where. - `lightcraft-cli denoise-serve` serves the installed, selected model on this computer's card (checked against the CPU, as in the app) or CPU. - docs/denoise.md: setup, safety (unencrypted: VPN/LAN only) and the protocol. Tested: protocol round trips, refusals (token, model, tile size, hostile payloads), fallback and give-up; the engine model with a live, a mismatched and a dead server; the CLI binary serving a synthetic model end to end. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 15:21:48 -05:00
Elliot Graham	bec1841561	LR-EDIT-DETAIL-DENOISE: avoid shared vector write races (#479)	2026-10-09 13:12:47 +01:00
Daniel Voshart	fd4019300a	LR-EDIT-DETAIL-DENOISE: address installation and model-policy review ... Generated with Codex.	2026-10-08 22:07:23 -04:00
3 people	7935999d94	LR-EDIT-DETAIL-DENOISE: non-destructive AI RAW denoise with pure-Rust CPU/GPU ... Recover the unpublished denoise work against current upstream. Preserve later opt-in model setup, shared Rust HTTPS downloads, pending-photo resume and an Amount-independent switch. Keep original RAWs unchanged and share the current color finish between ordinary and denoised sources. Bound public tiling inputs, refuse corrupt-cache exports, and retain synthetic/real-model coverage. Include two Windows path spelling fixes in existing tests. No model weights, face code, media or Tract runtime are included. Validation: cargo xtask ci (all seven gates); installed RawNIND CPU/GPU agreement 1.38e-6; native CPU preview/export on a public 24 MP Sony RAW, unchanged original SHA-256. Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Generated with Codex.	2026-10-08 22:07:20 -04:00