lightcraft/docs/gpu-pipeline.md

# GPU develop pipeline (M5.2)

The CPU pipeline (`crates/pipeline`) is the reference ("oracle"). `lightcraft-gpu` (`crates/gpu`, L3)
evaluates the same stages with wgpu compute shaders (WGSL) on Metal / Vulkan / DX12. Everything is
pure Rust (wgpu, naga); the drivers are the system's. No GL backend is compiled in.

## Where it is used
- `lightcraft_engine::media::develop` (called by every `RenderJob`): loupe / before / compare views,
  `render_now` (CLI, MCP, control channel renders) and exports render on the GPU when one is
  available; grid/filmstrip thumbnails (many small jobs in parallel) stay on the CPU.
- Anything the GPU path cannot do returns `None` and the CPU renders instead: no adapter (CI
  machines, software-only adapters), `LIGHTCRAFT_GPU=0` (whole process), the `app.gpu {enabled}`
  command (runtime preference; `ui.inspect` → `perf.gpu` shows the adapter), an output larger than
  the device's storage-buffer limit, or a device error / panic during a render (the process then
  stays on the CPU).
- wasm32: the crate compiles to the CPU fallback (WebGPU needs asynchronous device creation and a
  device shared with the canvas — open item below).

## Structure
- Shared parameters, so the two implementations cannot drift apart: `pipeline::plan` (effective
  settings, frame, output size, stage-cache keys), `Frame::sample_plan`, `local::{wb_matrix_for,
  nr_params, plane_sigmas, guided_fast_step, airlight_of}`, `finish::{FinishParams, mask_terms}`,
  `masks::brush_dabs`; exact tables (tone LUT, sRGB LUT, curve LUTs, resample taps) and the OkLab
  matrices are uploaded / generated into the WGSL prelude from the CPU values.
- Buffers, not textures: images are `array<f32>` with the CPU's interleaved layout (RGB, 1–3
  channels), so upload / readback are plain copies of `Rgb32f` / `Plane` / `Rgba8` data.
- Kernels (`crates/gpu/src/wgsl/`): `geom` (orientation pixel map, bilinear through the
  crop/straighten/flip affine or the full lens + perspective warp), `resize` (separable resample
  with the CPU's taps; Mitchell prefilter, box/bilinear for the fast guided filter), `blur` (box
  passes with running sums over pixel chunks; three each way = the CPU's Gaussian), `map`
  (log luminance, dark channel, guided-filter steps, white balance, luminance / colour NR, airlight
  sampling), `mask` (linear / radial / luminance / colour range / brush shapes, combine, finalize),
  `finish` (the whole per-pixel stage) — each mirrors a named CPU function.
- Per-stage hybrid: defringe and spot removal (rare, CPU-only for now) download the resampled image,
  run on the CPU and upload; heuristic mask shapes (Sky, Subject, Background, …) and brushes of more
  than 4096 dabs are evaluated on the CPU and uploaded; sources over the buffer limit are resampled
  on the CPU.
- Stage cache: `GpuStages` mirrors `StageCache` (same keys: source identity + `geo`, `lin_key`,
  per-plane radii) and lives in it as an extension, so the UI needs no change. The uploaded source is
  kept per view. A tone / colour / exposure drag re-runs one dispatch of `finish` plus the readback.
- One command encoder per render; the only mid-render readback is the airlight samples (dehaze, when
  the dark channel is recomputed). Dropped buffers are recycled (exact size, ≤ 2 GB pool) once the
  thread that dropped them has submitted — allocating and zero-filling fresh 100–300 MB buffers
  per pass cost as much as the passes.
- `LIGHTCRAFT_PROFILE=1` prints GPU stage timings (each stage is then submitted and waited for).

## Correctness: CPU oracle and equivalence tests
`crates/gpu/tests/equivalence.rs` renders the same settings on both and compares 8-bit sRGB:
bounds **mean |Δ| < 0.5 LSB and max |Δ| ≤ 3 LSB** per channel. Measured (Apple M4 Pro, Metal):
**max 1 LSB, mean ≤ 0.0003 LSB** for every case — 26 settings cases (all tone/colour/effects tools,
3 vignette styles, grain, curves, grading, B&W, WB, NR, crop/straighten/flip, lens + perspective,
three mask sets incl. CPU-evaluated shapes, spots + defringe), all 8 orientations ± crop, embedded
DNG lens data, display-referred sources, draft quality, full-size renders; the 24 MP export in
`render_bench` also differs by max 1 LSB. Cached (slider-drag) GPU renders are bit-identical to
fresh ones. The tests skip (pass with a note) when no adapter exists.

Remaining differences come from f32 vs f64 coordinate math, fast-math transcendental functions on
Metal and running-sum order in the box filters — all far below one 8-bit step.

## Measurements
`render_bench` on `corpus/raw/arw-sony-a7m3-compressed.arw` (24 MP; loupe = 2560 px preview →
1920×1280), min of 7 runs, wall clock, on a heavily shared machine (load average 60–110 on 14 cores,
so CPU numbers are pessimistic; the GPU numbers are less affected):

| scenario | CPU | GPU |
|---|---|---|
| loupe 1920×1280 cold (new photo, incl. upload) | 415 ms | 32 ms |
| loupe draft 1152×768 cold | 169 ms | 21 ms |
| exposure drag 1920×1280 (warm) | 22–42 ms | 3.9 ms |
| highlights drag, draft (warm) | 14 ms | 3.2 ms |
| clarity drag, draft (warm) | 16–23 ms | 3.0 ms |
| NR drag, draft / 1920×1280 (warm) | 186 / 301–471 ms | 10 / 18 ms |
| per-pixel stage 1920×1280, + vibrance/saturation | 53–65 ms | 5 ms |
| export render 6000×4000 | 1037–1414 ms | 291–330 ms |

GPU timings include the readback of the 8-bit result and the histogram. Device creation + kernel
compilation: ~0.4 s once per process — the desktop app starts it on a background thread at launch
(`lightcraft_gpu::warm_up`), so the first loupe render doesn't wait for it; other processes create
the device on their first GPU render. `lightcraft_gpu::ready()` asks without blocking.

## Opening a photo (M5.4)
- The loupe shows a stand-in at once (`media::QuickJob`): the photo's cached view render for its
  current settings, else (raws with their import look) the embedded camera JPEG, else a
  thumbnail-level render; the full render replaces it in place.
- Raw sources for previews (2560 px) and thumbnails are binned straight from the mosaic
  (`RawImage::develop_binned`, 2× for a 24 MP preview); only exports / 1:1 demosaic at full size.
- The next and previous photos in filmstrip order are prepared in the background once the current
  one is rendered (decoded source kept, view render cached): stepping through photos shows the
  developed image in ~50 ms.

## Memory (M5.6)
- `library.memory` reports what the engine's caches hold (decoded thumbnail / preview / full-size
  sources, rendered previews) and the GPU renderer's device buffers (allocated, of which pooled
  and retired); `ui.inspect` → `memory` adds the loupe's stage caches (CPU images, GPU buffers)
  and the textures.
- Heap profile: build `lightcraft-cli` with `--features dhat-heap`; `library.memory` then also
  reports live/peak heap bytes and the run writes `dhat-heap.json` (`LIGHTCRAFT_DHAT_FILE`), whose
  allocation sites at the peak (`t-gmax`) show who holds the memory.
- Measuring the scenario (import 14 raws from `corpus/raw`, open the loupe, step 12 times):
  `/usr/bin/time -l lightcraft-cli snapshot <files> --script steps.jsonl -o out.png` → "maximum
  resident set size" and "peak memory footprint". Run it several times: the high-water mark is
  noisy (allocator caching, scheduling). On Apple silicon GPU buffers count in the footprint.
- Imports read raw headers only (`lightcraft_raw::probe_info`): no pixel data is decompressed.
- One budget (`memory::budget`, default min(25 % of RAM, 1.5 GiB); `LIGHTCRAFT_MEMORY_MB` or
  `app.memoryBudget {mb}`): half for the engine caches (decoded thumbnail / preview / full
  sources and rendered previews, evicted least recently used *across* them; the photo on screen
  stays), a quarter for decodes in flight (`memory::work_gate`: grid thumbnails, neighbour
  prefetch and import probes wait for room, loupe and export loads never wait), an eighth for the
  GPU buffer pool (reuse by best fit within 25 %; freed after 3 s idle).
- Decodes free the file's bytes and the raw samples as soon as they are consumed and colour /
  crop / orient in place. macOS keeps freed large blocks resident ("reusable") until the system
  is short of memory, so the resident size climbed with every photo decoded; the apps install
  `malloc_zone_pressure_relief` as `memory::set_release_hook`, called after each decode, import
  and idle trim.
- Result (M4 Pro, 14 raws, loupe, 12 steps): peak RSS 1.9–2.3 GB → 0.83–0.99 GB, peak footprint
  2.5–2.6 GB → 0.91–0.96 GB, heap peak 930 → ~600 MB; stepping median ~13 ms (unchanged).

## Open items
- Keep the loupe texture on the GPU (render into an `egui-wgpu` texture on eframe's device instead of
  reading back); share the device with eframe.
- WebGPU in the browser build (async init, single device per canvas).
- Defringe and spot removal kernels; Sky/Subject heuristics (replaced by the segmenter in M12).
- Tiling for sources / outputs beyond the buffer limit (60 MP+ on smaller adapters): tile with
  overlap equal to the largest filter radius.
- Large-radius blurs at full size (dehaze dark channel at 24 MP) dominate the export: a summed-area
  table or a downsampled dark channel would cut them further.
- GPU histogram (atomics) to skip the CPU pass over the readback.