mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-26 02:59:17 -06:00
Brandon ThomasandClaude Opus 5.5
30aac43fab
`media::develop` renders on the GPU when available (views, render_now, exports;
thumbnails stay on the CPU) and falls back to the CPU otherwise. `app.gpu
{enabled?}` toggles it at runtime, `LIGHTCRAFT_GPU=0` disables it for the process,
`ui.inspect` reports the adapter under perf.gpu.
Performance: dropped device buffers are recycled once their thread has submitted
(fresh 100-300 MB allocations per pass cost as much as the passes), Gaussian
passes ping-pong between two buffers and read all channels of a pixel at once,
and long box radii slide over longer chunks. LIGHTCRAFT_PROFILE prints real GPU
stage timings. render_bench prints CPU and GPU columns and the CPU/GPU difference.
24 MP ARW, loaded machine, min of 7: cold loupe 415 -> 32 ms, exposure drag
22-42 -> 3.9 ms, NR drag (1920) 301+ -> 18 ms, export render 1037+ -> 291 ms;
max 1 LSB difference. docs/gpu-pipeline.md describes the implementation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
89 lines
6.1 KiB
Markdown
89 lines
6.1 KiB
Markdown
# GPU develop pipeline (M5.2)
|
||
|
||
The CPU pipeline (`crates/pipeline`) is the reference ("oracle"). `lightcraft-gpu` (`crates/gpu`, L3)
|
||
evaluates the same stages with wgpu compute shaders (WGSL) on Metal / Vulkan / DX12. Everything is
|
||
pure Rust (wgpu, naga); the drivers are the system's. No GL backend is compiled in.
|
||
|
||
## Where it is used
|
||
- `lightcraft_engine::media::develop` (called by every `RenderJob`): loupe / before / compare views,
|
||
`render_now` (CLI, MCP, control channel renders) and exports render on the GPU when one is
|
||
available; grid/filmstrip thumbnails (many small jobs in parallel) stay on the CPU.
|
||
- Anything the GPU path cannot do returns `None` and the CPU renders instead: no adapter (CI
|
||
machines, software-only adapters), `LIGHTCRAFT_GPU=0` (whole process), the `app.gpu {enabled}`
|
||
command (runtime preference; `ui.inspect` → `perf.gpu` shows the adapter), an output larger than
|
||
the device's storage-buffer limit, or a device error / panic during a render (the process then
|
||
stays on the CPU).
|
||
- wasm32: the crate compiles to the CPU fallback (WebGPU needs asynchronous device creation and a
|
||
device shared with the canvas — open item below).
|
||
|
||
## Structure
|
||
- Shared parameters, so the two implementations cannot drift apart: `pipeline::plan` (effective
|
||
settings, frame, output size, stage-cache keys), `Frame::sample_plan`, `local::{wb_matrix_for,
|
||
nr_params, plane_sigmas, guided_fast_step, airlight_of}`, `finish::{FinishParams, mask_terms}`,
|
||
`masks::brush_dabs`; exact tables (tone LUT, sRGB LUT, curve LUTs, resample taps) and the OkLab
|
||
matrices are uploaded / generated into the WGSL prelude from the CPU values.
|
||
- Buffers, not textures: images are `array<f32>` with the CPU's interleaved layout (RGB, 1–3
|
||
channels), so upload / readback are plain copies of `Rgb32f` / `Plane` / `Rgba8` data.
|
||
- Kernels (`crates/gpu/src/wgsl/`): `geom` (orientation pixel map, bilinear through the
|
||
crop/straighten/flip affine or the full lens + perspective warp), `resize` (separable resample
|
||
with the CPU's taps; Mitchell prefilter, box/bilinear for the fast guided filter), `blur` (box
|
||
passes with running sums over pixel chunks; three each way = the CPU's Gaussian), `map`
|
||
(log luminance, dark channel, guided-filter steps, white balance, luminance / colour NR, airlight
|
||
sampling), `mask` (linear / radial / luminance / colour range / brush shapes, combine, finalize),
|
||
`finish` (the whole per-pixel stage) — each mirrors a named CPU function.
|
||
- Per-stage hybrid: defringe and spot removal (rare, CPU-only for now) download the resampled image,
|
||
run on the CPU and upload; heuristic mask shapes (Sky, Subject, Background, …) and brushes of more
|
||
than 4096 dabs are evaluated on the CPU and uploaded; sources over the buffer limit are resampled
|
||
on the CPU.
|
||
- Stage cache: `GpuStages` mirrors `StageCache` (same keys: source identity + `geo`, `lin_key`,
|
||
per-plane radii) and lives in it as an extension, so the UI needs no change. The uploaded source is
|
||
kept per view. A tone / colour / exposure drag re-runs one dispatch of `finish` plus the readback.
|
||
- One command encoder per render; the only mid-render readback is the airlight samples (dehaze, when
|
||
the dark channel is recomputed). Dropped buffers are recycled (exact size, ≤ 2 GB pool) once the
|
||
thread that dropped them has submitted — allocating and zero-filling fresh 100–300 MB buffers
|
||
per pass cost as much as the passes.
|
||
- `LIGHTCRAFT_PROFILE=1` prints GPU stage timings (each stage is then submitted and waited for).
|
||
|
||
## Correctness: CPU oracle and equivalence tests
|
||
`crates/gpu/tests/equivalence.rs` renders the same settings on both and compares 8-bit sRGB:
|
||
bounds **mean |Δ| < 0.5 LSB and max |Δ| ≤ 3 LSB** per channel. Measured (Apple M4 Pro, Metal):
|
||
**max 1 LSB, mean ≤ 0.0003 LSB** for every case — 26 settings cases (all tone/colour/effects tools,
|
||
3 vignette styles, grain, curves, grading, B&W, WB, NR, crop/straighten/flip, lens + perspective,
|
||
three mask sets incl. CPU-evaluated shapes, spots + defringe), all 8 orientations ± crop, embedded
|
||
DNG lens data, display-referred sources, draft quality, full-size renders; the 24 MP export in
|
||
`render_bench` also differs by max 1 LSB. Cached (slider-drag) GPU renders are bit-identical to
|
||
fresh ones. The tests skip (pass with a note) when no adapter exists.
|
||
|
||
Remaining differences come from f32 vs f64 coordinate math, fast-math transcendental functions on
|
||
Metal and running-sum order in the box filters — all far below one 8-bit step.
|
||
|
||
## Measurements
|
||
`render_bench` on `corpus/raw/arw-sony-a7m3-compressed.arw` (24 MP; loupe = 2560 px preview →
|
||
1920×1280), min of 7 runs, wall clock, on a heavily shared machine (load average 60–110 on 14 cores,
|
||
so CPU numbers are pessimistic; the GPU numbers are less affected):
|
||
|
||
| scenario | CPU | GPU |
|
||
|---|---|---|
|
||
| loupe 1920×1280 cold (new photo, incl. upload) | 415 ms | 32 ms |
|
||
| loupe draft 1152×768 cold | 169 ms | 21 ms |
|
||
| exposure drag 1920×1280 (warm) | 22–42 ms | 3.9 ms |
|
||
| highlights drag, draft (warm) | 14 ms | 3.2 ms |
|
||
| clarity drag, draft (warm) | 16–23 ms | 3.0 ms |
|
||
| NR drag, draft / 1920×1280 (warm) | 186 / 301–471 ms | 10 / 18 ms |
|
||
| per-pixel stage 1920×1280, + vibrance/saturation | 53–65 ms | 5 ms |
|
||
| export render 6000×4000 | 1037–1414 ms | 291–330 ms |
|
||
|
||
GPU timings include the readback of the 8-bit result and the histogram. Device creation + kernel
|
||
compilation: ~0.4 s once per process (lazily, on the first GPU render).
|
||
|
||
## Open items
|
||
- Keep the loupe texture on the GPU (render into an `egui-wgpu` texture on eframe's device instead of
|
||
reading back); share the device with eframe.
|
||
- WebGPU in the browser build (async init, single device per canvas).
|
||
- Defringe and spot removal kernels; Sky/Subject heuristics (replaced by the segmenter in M12).
|
||
- Tiling for sources / outputs beyond the buffer limit (60 MP+ on smaller adapters): tile with
|
||
overlap equal to the largest filter radius.
|
||
- Large-radius blurs at full size (dehaze dark channel at 24 MP) dominate the export: a summed-area
|
||
table or a downsampled dark channel would cut them further.
|
||
- GPU histogram (atomics) to skip the CPU pass over the readback.
|