From 37ab239b7b6a3000a77c4ca43017be2e583fe1a7 Mon Sep 17 00:00:00 2001 From: Brandon Thomas Date: Thu, 1 Oct 2026 01:48:48 -0400 Subject: [PATCH] =?UTF-8?q?M0.10:=20headless=20UI=20rendering=20=E2=80=94?= =?UTF-8?q?=20CPU=20rasterizer=20for=20egui,=20ui.screenshot=20headless=20?= =?UTF-8?q?+=20fallback,=20lightcraft-cli=20snapshot?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - ui-egui/softpaint: pure-Rust rasterizer for tessellated egui output (gamma-space premultiplied blending like egui-wgpu, bilinear texel-centre sampling, scissor clips, tie-break rule so shared edges blend once, parallel row bands, deterministic) + CPU texture mirror from TexturesDelta. - ui-egui/headless: HeadlessView (offscreen context with our fonts/theme) and Headless (windowless app session: logic+ui per frame, answers viewport Screenshot/InnerSize/Close, settle loop, control requests through the shared handler). - ui.screenshot {headless: true} in the running app renders from the logic tick (works while the window is occluded / display asleep); windowed captures fall back to headless after 2 s. Screenshots now wait for in-flight renders, not just queued ones. - lightcraft-cli snapshot [--demo|--library DIR] [--script f.jsonl] -o out.png --size WxH --scale S. - Fixes found with it: dialog backdrop was painted over the dialog itself (now an area below the window); dialog content padding; crop overlay options use our buttons (registered widget ids). - Docs: control-protocol.md (headless section), AGENTS.md (prefer headless when unattended). Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 4 + Cargo.lock | 1 + apps/lightcraft-cli/Cargo.toml | 5 +- apps/lightcraft-cli/src/main.rs | 147 +++++++++ apps/lightcraft-cli/tests/cli.rs | 37 +++ crates/ui-egui/src/control.rs | 6 +- crates/ui-egui/src/headless.rs | 298 +++++++++++++++++ crates/ui-egui/src/lib.rs | 122 ++++++- crates/ui-egui/src/panels/dialogs.rs | 23 +- crates/ui-egui/src/panels/right.rs | 2 +- crates/ui-egui/src/render.rs | 25 +- crates/ui-egui/src/softpaint.rs | 471 +++++++++++++++++++++++++++ docs/control-protocol.md | 27 +- 13 files changed, 1133 insertions(+), 35 deletions(-) create mode 100644 crates/ui-egui/src/headless.rs create mode 100644 crates/ui-egui/src/softpaint.rs diff --git a/AGENTS.md b/AGENTS.md index 40f492c7e..24e9f1444 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -42,6 +42,10 @@ and often so a crash loses minutes, not hours. ## Running and looking at the app - `cargo run --release -p lightcraft -- --control 7980` opens the desktop app with the JSON-lines control server (see `docs/control-protocol.md`). - For UI work, **look at the result**: drive via the control channel and take `ui.screenshot`, compare with `plan/lightroom/screenshots/`. +- **Unattended (display asleep/locked, CI, nightly runs): prefer headless snapshots** — no window needed: + `lightcraft-cli snapshot --demo --script tour.jsonl -o out.png --size 1600x1000` (control-protocol requests, + one per line; see `docs/control-protocol.md` → Headless rendering). In a running app use + `ui.screenshot {"path": …, "headless": true}`; windowed screenshots fall back to headless after 2 s. - MCP: `lightcraft-cli mcp` (headless, `--demo` for the procedural library) or `lightcraft-cli mcp --connect` (drives the running app). See `docs/mcp.md`. Quick non-UI checks: `lightcraft-cli render in.jpg -o out.jpg --set light.exposure=1`. - Export goes through `lightcraft_engine::export` (one encoder for app, CLI, MCP and web); UI-only commands live in diff --git a/Cargo.lock b/Cargo.lock index d8f36eab1..8ff6fc3ef 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2025,6 +2025,7 @@ dependencies = [ "lightcraft-engine", "lightcraft-mcp", "lightcraft-raster", + "lightcraft-ui-egui", "serde_json", ] diff --git a/apps/lightcraft-cli/Cargo.toml b/apps/lightcraft-cli/Cargo.toml index 6a733f31d..939bdefef 100644 --- a/apps/lightcraft-cli/Cargo.toml +++ b/apps/lightcraft-cli/Cargo.toml @@ -16,8 +16,7 @@ workspace = true [dependencies] lightcraft-engine = { workspace = true } lightcraft-mcp = { workspace = true } -serde_json = { workspace = true } - -[dev-dependencies] +lightcraft-ui-egui = { workspace = true } lightcraft-codecs = { workspace = true } lightcraft-raster = { workspace = true } +serde_json = { workspace = true } diff --git a/apps/lightcraft-cli/src/main.rs b/apps/lightcraft-cli/src/main.rs index 11c042654..46186fb4e 100644 --- a/apps/lightcraft-cli/src/main.rs +++ b/apps/lightcraft-cli/src/main.rs @@ -3,6 +3,7 @@ //! ```text //! lightcraft-cli mcp [--connect [ADDR]] [--demo] [--compact] [FILES/FOLDERS…] //! lightcraft-cli render -o [--set control=value]… [--settings FILE.json] [--preset ID] [--size N] [--quality Q] +//! lightcraft-cli snapshot [--library DIR | --demo] [--script FILE.jsonl] [-o OUT.png] [--size WxH] [--scale S] [FILES…] //! lightcraft-cli commands [--json] //! lightcraft-cli controls [--json] //! ``` @@ -33,6 +34,18 @@ USAGE: --preset ID apply a preset (see `commands`/presets.list) --size N long edge in pixels (default: full size) --quality Q JPEG quality 1..100 (default 92) + lightcraft-cli snapshot [OPTIONS] [FILES/FOLDERS…] + Run the full app UI headlessly (no window, no GPU: CPU-rasterized egui) and write PNGs. + Options: + --demo the procedural demo library (default unless --library or FILES) + --library DIR open (or create) a LightCraft library + --script FILE JSON-lines control-protocol requests (docs/control-protocol.md), one + per line: {\"method\": \"ui.set\", \"params\": {\"view\": \"detail\"}}. + Replies go to stdout. `ui.screenshot` without a path writes -o (then + OUT-2.png, OUT-3.png…); `ui.settle {timeoutMs?}` waits for renders. + -o, --output OUT PNG path (a final screenshot is written here if the script took none) + --size WxH window size in points (default 1600x1000) + --scale S pixels per point (default 1) lightcraft-cli commands [--json] list every command id with its parameters lightcraft-cli controls [--json] list every develop control id with its range lightcraft-cli --version | --help @@ -43,6 +56,7 @@ fn main() -> ExitCode { let r = match args.first().map(String::as_str) { Some("mcp") => mcp(&args[1..]), Some("render") => render(&args[1..]), + Some("snapshot") => snapshot(&args[1..]), Some("commands") => commands(&args[1..]), Some("controls") => controls(&args[1..]), Some("--version" | "-V" | "version") => { @@ -197,6 +211,139 @@ fn render(args: &[String]) -> Result<(), String> { Ok(()) } +fn snapshot(args: &[String]) -> Result<(), String> { + use lightcraft_ui_egui::headless::Headless; + use std::time::{Duration, Instant}; + let mut library: Option = None; + let mut script: Option = None; + let mut output: Option = None; + let mut size = [1600.0f32, 1000.0]; + let mut scale = 1.0f32; + let mut files = Vec::new(); + let mut i = 0; + while i < args.len() { + match args[i].as_str() { + "--demo" => {} + "--library" => library = Some(take_value(args, &mut i, "--library")?.to_string()), + "--script" => script = Some(take_value(args, &mut i, "--script")?.to_string()), + "-o" | "--output" => output = Some(take_value(args, &mut i, "-o")?.to_string()), + "--size" => { + let v = take_value(args, &mut i, "--size")?; + let (w, h) = v.split_once(['x', 'X']).ok_or("--size expects WxH, e.g. 1600x1000")?; + size = [w.trim().parse().map_err(|_| "--size: bad width")?, h.trim().parse().map_err(|_| "--size: bad height")?]; + } + "--scale" => scale = take_value(args, &mut i, "--scale")?.parse().map_err(|_| "--scale expects a number")?, + a if a.starts_with('-') => return Err(format!("unknown option `{a}`")), + f => files.push(f.to_string()), + } + i += 1; + } + if script.is_none() && output.is_none() { + return Err("snapshot: give -o OUT.png and/or --script FILE".into()); + } + if !(scale > 0.0 && size[0] >= 1.0 && size[1] >= 1.0) { + return Err("snapshot: bad --size/--scale".into()); + } + let t0 = Instant::now(); + let mut session = match &library { + Some(dir) => { + let mut s = Session::new().with_fs(); + s.open_library(dir, false).map_err(|e| format!("{dir}: {e}"))?; + s + } + None if files.is_empty() => Session::with_demo().with_fs(), + None => Session::new().with_fs(), + }; + if !files.is_empty() { + session.execute("library.import", &json!({"paths": expand_paths(&files)})).map_err(|e| e.to_string())?; + } + let services = lightcraft_ui_egui::Services { + write: Some(Box::new(|p: &str, b: &[u8]| { + if let Some(dir) = Path::new(p).parent().filter(|d| !d.as_os_str().is_empty()) { + std::fs::create_dir_all(dir).map_err(|e| e.to_string())?; + } + std::fs::write(p, b).map_err(|e| format!("{p}: {e}")) + })), + png: Some(Box::new(|img: &lightcraft_raster::Rgba8| { + lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(img), &lightcraft_codecs::EncodeMeta::default()).unwrap_or_default() + })), + ..Default::default() + }; + let app = lightcraft_ui_egui::LightcraftApp::new(session, services); + let mut h = Headless::new(app, size, scale); + let timeout = Duration::from_secs(60); + let mut shots = 0usize; + let next_path = |shots: &mut usize| -> Option { + let out = output.as_deref()?; + *shots += 1; + if *shots == 1 { + return Some(out.to_string()); + } + let p = Path::new(out); + let stem = p.file_stem().map(|s| s.to_string_lossy().to_string()).unwrap_or_default(); + let ext = p.extension().map(|e| e.to_string_lossy().to_string()).unwrap_or_else(|| "png".into()); + Some(p.with_file_name(format!("{stem}-{shots}.{ext}")).to_string_lossy().to_string()) + }; + let mut wrote_output = false; + if let Some(script) = &script { + let text = std::fs::read_to_string(script).map_err(|e| format!("{script}: {e}"))?; + let mut out = std::io::stdout().lock(); + for (n, line) in text.lines().enumerate() { + let line = line.trim(); + if line.is_empty() || line.starts_with('#') || line.starts_with("//") { + continue; + } + let msg: Value = serde_json::from_str(line).map_err(|e| format!("{script}:{}: {e}", n + 1))?; + let id = msg.get("id").cloned().unwrap_or(json!(n + 1)); + let method = msg.get("method").and_then(Value::as_str).unwrap_or("").to_string(); + let mut params = msg.get("params").cloned().unwrap_or(json!({})); + let ts = Instant::now(); + let mut reply = match method.as_str() { + "ui.settle" => { + let ms = params.get("timeoutMs").and_then(Value::as_u64).unwrap_or(20_000); + json!({"ok": true, "result": {"settled": h.settle(Duration::from_millis(ms))}}) + } + "ui.screenshot" => { + if params.get("path").and_then(Value::as_str).is_none() { + match next_path(&mut shots) { + Some(p) => { + wrote_output = true; + params["path"] = json!(p); + } + None => return Err(format!("{script}:{}: ui.screenshot needs a `path` (or give -o)", n + 1)), + } + } + h.request(&method, params, timeout) + } + _ => h.request(&method, params, timeout), + }; + if let Some(o) = reply.as_object_mut() { + o.insert("id".into(), id); + } + writeln!(out, "{reply}").map_err(|e| e.to_string())?; + if method == "ui.screenshot" { + eprintln!( + "lightcraft-cli snapshot: {} ({:.0} ms)", + reply["result"]["path"].as_str().unwrap_or("?"), + ts.elapsed().as_secs_f64() * 1000.0 + ); + } + if h.quit_requested() { + break; + } + } + } + if !wrote_output && let Some(path) = next_path(&mut shots) { + let r = h.request("ui.screenshot", json!({"path": path}), timeout); + if r["ok"] != true { + return Err(format!("screenshot failed: {}", r["error"])); + } + eprintln!("lightcraft-cli snapshot: wrote {path} ({}×{})", r["result"]["width"], r["result"]["height"]); + } + eprintln!("lightcraft-cli snapshot: done in {:.2} s ({} frames)", t0.elapsed().as_secs_f64(), h.frames()); + Ok(()) +} + fn commands(args: &[String]) -> Result<(), String> { let mut h = Headless::demo(); let cmds = h.call("engine.commands", json!({}))?; diff --git a/apps/lightcraft-cli/tests/cli.rs b/apps/lightcraft-cli/tests/cli.rs index bf26e8ffb..9b3991c25 100644 --- a/apps/lightcraft-cli/tests/cli.rs +++ b/apps/lightcraft-cli/tests/cli.rs @@ -98,3 +98,40 @@ fn commands_subcommand_lists_registry() { let o = Command::new(BIN).arg("controls").output().unwrap(); assert!(String::from_utf8_lossy(&o.stdout).contains("light.exposure")); } + +#[test] +fn snapshot_subcommand_renders_the_ui_headlessly() { + let script = tmp("snap.jsonl"); + let a = tmp("snap-grid.png"); + let b = tmp("snap-export.png"); + std::fs::write( + &script, + format!( + "# comment\n{}\n{}\n{}\n{}\n", + json!({"method": "ui.set", "params": {"view": "photoGrid"}}), + json!({"method": "ui.screenshot"}), + json!({"method": "engine.execute", "params": {"command": "dialog.export"}}), + json!({"method": "ui.screenshot", "params": {"path": b.to_str().unwrap()}}), + ), + ) + .unwrap(); + let o = Command::new(BIN) + .args(["snapshot", "--demo", "--script", script.to_str().unwrap(), "-o", a.to_str().unwrap(), "--size", "640x400"]) + .output() + .unwrap(); + assert!(o.status.success(), "{}", String::from_utf8_lossy(&o.stderr)); + let replies: Vec = String::from_utf8_lossy(&o.stdout).lines().map(|l| serde_json::from_str(l).unwrap()).collect(); + assert_eq!(replies.len(), 4); + assert!(replies.iter().all(|r| r["ok"] == true), "{replies:?}"); + for (p, dimmed) in [(&a, false), (&b, true)] { + let d = lightcraft_codecs::decode(&std::fs::read(p).unwrap(), Default::default()).unwrap(); + assert_eq!((d.width, d.height), (640, 400)); + // the export dialog dims everything around it + assert_eq!(mean(p) < mean(&a) - 2.0, dimmed, "{}", p.display()); + } + // no script: one settled screenshot, at 2× scale + let o = Command::new(BIN).args(["snapshot", "-o", a.to_str().unwrap(), "--size", "320x240", "--scale", "2"]).output().unwrap(); + assert!(o.status.success(), "{}", String::from_utf8_lossy(&o.stderr)); + let d = lightcraft_codecs::decode(&std::fs::read(&a).unwrap(), Default::default()).unwrap(); + assert_eq!((d.width, d.height), (640, 480)); +} diff --git a/crates/ui-egui/src/control.rs b/crates/ui-egui/src/control.rs index 97e130f9d..3b6885c4a 100644 --- a/crates/ui-egui/src/control.rs +++ b/crates/ui-egui/src/control.rs @@ -12,7 +12,7 @@ //! image coordinates (brush strokes, crop handles…) — mapped to screen and injected as real input //! - `ui.key {key, shift?, alt?, cmd?}` / `ui.text {text}` / `ui.scroll {dx, dy}` //! - `ui.set {view?, panel?, filmstrip?, leftPanel?, zoom?, …}` / `ui.dialog.confirm|cancel` -//! - `ui.screenshot {path?}`, `ui.resize {width, height}`, `ui.render {id?, size?, path?}`, `app.quit` +//! - `ui.screenshot {path?, headless?}` (headless: CPU-rendered, no compositor needed), `ui.resize {width, height}`, `ui.render {id?, size?, path?}`, `app.quit` use std::sync::mpsc::Sender; @@ -37,7 +37,7 @@ impl ControlRequest { pub enum Outcome { Done(Value), - Screenshot { path: Option }, + Screenshot { path: Option, headless: bool }, } fn ok(v: Value) -> Outcome { @@ -261,7 +261,7 @@ pub fn handle(app: &mut LightcraftApp, ctx: &egui::Context, req: &ControlRequest } "ui.screenshot" => { ctx.request_repaint(); - Outcome::Screenshot { path: s("path").map(str::to_string) } + Outcome::Screenshot { path: s("path").map(str::to_string), headless: p.get("headless").and_then(Value::as_bool).unwrap_or(false) } } "ui.render" => { let id = p.get("id").and_then(Value::as_u64).map(lightcraft_catalog::PhotoId).or(app.session.active()); diff --git a/crates/ui-egui/src/headless.rs b/crates/ui-egui/src/headless.rs new file mode 100644 index 000000000..ccac06f21 --- /dev/null +++ b/crates/ui-egui/src/headless.rs @@ -0,0 +1,298 @@ +//! Headless UI rendering: run the app's UI in an offscreen [`egui::Context`] and rasterize it on +//! the CPU ([`crate::softpaint`]) — no window, no GPU, no compositor. +//! +//! Two users: +//! - [`Headless`]: a complete windowless app session (`lightcraft-cli snapshot`, tests). It plays +//! the role eframe plays for the desktop app: builds [`egui::RawInput`], runs `logic` + `ui`, +//! keeps a CPU mirror of the textures, executes viewport commands (`Screenshot` is answered with +//! a CPU-rendered frame, `InnerSize` resizes, `Close` quits). Control-protocol requests go +//! through the very same handler as the desktop app's control server. +//! - [`HeadlessView`] inside the desktop app: `ui.screenshot {"headless": true}` (and the +//! automatic fallback when the compositor delivers no frame, e.g. while the display sleeps) +//! draws the app's UI into a shadow context from `logic`, which keeps ticking when the window +//! is occluded. + +use std::collections::HashMap; +use std::sync::Arc; +use std::sync::mpsc::Sender; +use std::time::{Duration, Instant}; + +use egui::{Color32, ColorImage, RawInput, TextureId, ViewportCommand, ViewportId}; +use serde_json::{Value, json}; + +use crate::softpaint::{self, CpuTexture, Layered, TextureStore}; +use crate::{ControlRequest, LightcraftApp}; + +/// Background behind the panels (eframe's default clear colour, made opaque). +const CLEAR: Color32 = Color32::from_rgb(12, 12, 12); +/// Simulated frame interval (deterministic animation time). +const FRAME_DT: f64 = 1.0 / 60.0; + +/// An offscreen egui context with our fonts and theme, and a CPU mirror of its textures. +pub struct HeadlessView { + pub ctx: egui::Context, + pub textures: TextureStore, + shapes: Vec, + pixels_per_point: f32, + size: egui::Vec2, + frames: u64, +} + +impl Default for HeadlessView { + fn default() -> Self { + Self::new() + } +} + +impl HeadlessView { + pub fn new() -> Self { + let ctx = egui::Context::default(); + crate::theme::install_fonts(&ctx); + crate::theme::apply(&ctx); + HeadlessView { ctx, textures: TextureStore::default(), shapes: vec![], pixels_per_point: 1.0, size: egui::vec2(1600.0, 1000.0), frames: 0 } + } + + /// Input for one frame of a `size` (points) viewport at `pixels_per_point`. + pub fn raw_input(size: egui::Vec2, pixels_per_point: f32, time: f64, events: Vec) -> RawInput { + let rect = egui::Rect::from_min_size(egui::Pos2::ZERO, size); + let mut raw = RawInput { + screen_rect: Some(rect), + time: Some(time), + predicted_dt: FRAME_DT as f32, + focused: true, + events, + max_texture_side: Some(16384), + ..Default::default() + }; + let info = raw.viewports.entry(ViewportId::ROOT).or_default(); + info.native_pixels_per_point = Some(pixels_per_point); + info.inner_rect = Some(rect); + info.outer_rect = Some(rect); + info.focused = Some(true); + raw + } + + /// Run one frame; keeps the shapes for [`Self::paint`]. Returns the root viewport's commands. + pub fn run(&mut self, raw: RawInput, run_ui: impl FnMut(&mut egui::Ui)) -> Vec { + if let Some(r) = raw.screen_rect { + self.size = r.size(); + } + let mut out = self.ctx.run_ui(raw, run_ui); + self.frames += 1; + self.textures.apply(std::mem::take(&mut out.textures_delta)); + self.shapes = std::mem::take(&mut out.shapes); + self.pixels_per_point = out.pixels_per_point; + out.viewport_output.remove(&ViewportId::ROOT).map(|v| v.commands).unwrap_or_default() + } + + /// Frames run so far. + pub fn frames(&self) -> u64 { + self.frames + } + + /// Size in pixels of the last frame. + pub fn size_px(&self) -> [usize; 2] { + [(self.size.x * self.pixels_per_point).round() as usize, (self.size.y * self.pixels_per_point).round() as usize] + } + + /// Rasterize the last frame. `extra` textures (by id) take precedence over the context's own. + pub fn paint(&self, extra: &HashMap) -> ColorImage { + let prims = self.ctx.tessellate(self.shapes.clone(), self.pixels_per_point); + softpaint::paint(&prims, &Layered { over: extra, base: &self.textures }, self.size_px(), self.pixels_per_point, CLEAR) + } +} + +/// A windowless app session: the app's `logic` + `ui` driven frame by frame into a +/// [`HeadlessView`], with control-protocol requests answered by the shared handler. +pub struct Headless { + pub app: LightcraftApp, + pub view: HeadlessView, + /// Logical size (points) and scale. + pub size: egui::Vec2, + pub pixels_per_point: f32, + time: f64, + frames: u64, + events: Vec, + control: Sender, + quit: bool, +} + +impl Headless { + /// Wrap `app` (its control channel is replaced by the driver's). + pub fn new(app: LightcraftApp, size: [f32; 2], pixels_per_point: f32) -> Self { + let (tx, rx) = std::sync::mpsc::channel(); + let mut app = app.with_control(rx); + app.headless_host = true; + Headless { + app, + view: HeadlessView::new(), + size: egui::vec2(size[0], size[1]), + pixels_per_point, + time: 0.0, + frames: 0, + events: vec![], + control: tx, + quit: false, + } + } + + pub fn frames(&self) -> u64 { + self.frames + } + + /// `app.quit` was requested. + pub fn quit_requested(&self) -> bool { + self.quit + } + + /// Run one frame. + pub fn step(&mut self) { + let mut raw = HeadlessView::raw_input(self.size, self.pixels_per_point, self.time, std::mem::take(&mut self.events)); + self.app.raw_input_hook(&mut raw); + let app = &mut self.app; + let commands = self.view.run(raw, |ui| { + app.logic(ui.ctx()); + app.ui(ui); + }); + self.time += FRAME_DT; + self.frames += 1; + for c in commands { + match c { + ViewportCommand::Screenshot(user_data) => { + let image = Arc::new(self.paint()); + self.events.push(egui::Event::Screenshot { viewport_id: ViewportId::ROOT, user_data, image }); + } + ViewportCommand::InnerSize(s) if s.x >= 1.0 && s.y >= 1.0 => self.size = s, + ViewportCommand::Close => self.quit = true, + _ => {} + } + } + } + + /// Is anything still in progress (renders, queued input)? + pub fn busy(&self) -> bool { + self.app.renderer.in_flight() > 0 || !self.app.synthetic.is_empty() || !self.events.is_empty() + } + + /// Run frames until nothing is pending (renders finished, input consumed) for a few frames in + /// a row, or `timeout` passes. Returns whether it settled. + pub fn settle(&mut self, timeout: Duration) -> bool { + let t0 = Instant::now(); + let mut quiet = 0; + loop { + self.step(); + if self.busy() { + quiet = 0; + std::thread::sleep(Duration::from_millis(1)); + } else { + quiet += 1; + } + // a few quiet frames let layout and short UI animations finish + if quiet >= 12 { + return true; + } + if t0.elapsed() > timeout { + return false; + } + } + } + + /// Rasterize the last frame (with the photo textures). + pub fn paint(&self) -> ColorImage { + self.view.paint(&HashMap::new()) + } + + /// Settle, then render the current UI. + pub fn snapshot(&mut self, timeout: Duration) -> ColorImage { + self.settle(timeout); + self.paint() + } + + /// Send a control-protocol request (see [`crate::control`]) and run frames until it is + /// answered, then until its input has been consumed. Returns `{"ok": …, "result"|"error": …}`. + pub fn request(&mut self, method: &str, params: Value, timeout: Duration) -> Value { + let (req, rx) = ControlRequest::new(method, params); + if self.control.send(req).is_err() { + return json!({"ok": false, "error": "control channel closed"}); + } + let t0 = Instant::now(); + let reply = loop { + self.step(); + if let Ok(v) = rx.try_recv() { + break v; + } + if t0.elapsed() > timeout { + return json!({"ok": false, "error": "timeout"}); + } + if self.busy() { + std::thread::sleep(Duration::from_millis(1)); + } + }; + // let injected input (clicks, keys, drags) play out before the next request + let t1 = Instant::now(); + while (!self.app.synthetic.is_empty() || !self.events.is_empty()) && t1.elapsed() < timeout { + self.step(); + } + self.step(); + reply + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn demo(size: [f32; 2]) -> Headless { + let services = crate::Services { png: None, ..Default::default() }; + let mut app = LightcraftApp::new(lightcraft_engine::Session::with_demo(), services); + app.ui.view = crate::state::ViewMode::PhotoGrid; + Headless::new(app, size, 1.0) + } + + #[test] + fn demo_grid_snapshot_has_ui_pixels() { + let t0 = Instant::now(); + let mut h = demo([1200.0, 760.0]); + let img = h.snapshot(Duration::from_secs(20)); + eprintln!("headless snapshot: {:?} in {:?} ({} frames)", img.size, t0.elapsed(), h.frames()); + assert_eq!(img.size, [1200, 760]); + // not blank: many distinct colours + let mut colours: Vec = img.pixels.iter().map(|c| u32::from_le_bytes(c.to_array())).collect(); + colours.sort_unstable(); + colours.dedup(); + assert!(colours.len() > 200, "only {} colours", colours.len()); + // text in the top bar: bright pixels on the dark chrome + let top_h = crate::theme::Tokens::default().top_bar_h as usize; + let bright = img.pixels[..top_h * 1200].iter().filter(|c| c.r() > 150 && c.g() > 150 && c.b() > 150).count(); + assert!(bright > 30, "no text in the top bar ({bright} bright px)"); + // thumbnails arrived (photo textures were drawn) + assert!(h.app.renderer.thumb_textures() > 0); + } + + #[test] + fn snapshots_are_deterministic_and_control_requests_work() { + let shot = || { + let mut h = demo([900.0, 600.0]); + let r = h.request("ui.set", json!({"view": "detail"}), Duration::from_secs(10)); + assert_eq!(r["ok"], true, "{r}"); + h.snapshot(Duration::from_secs(20)) + }; + let (a, b) = (shot(), shot()); + assert_eq!(a.size, b.size); + let diff = a.pixels.iter().zip(&b.pixels).filter(|(x, y)| x != y).count(); + assert_eq!(diff, 0, "{diff} pixels differ between two runs"); + } + + #[test] + fn screenshot_request_is_answered_without_a_window() { + let mut h = demo([800.0, 500.0]); + let r = h.request("ui.screenshot", json!({}), Duration::from_secs(20)); + assert_eq!(r["ok"], true, "{r}"); + assert_eq!(r["result"]["width"], 800); + assert_eq!(r["result"]["height"], 500); + let r = h.request("ui.resize", json!({"width": 640, "height": 400}), Duration::from_secs(5)); + assert_eq!(r["ok"], true); + let r = h.request("ui.screenshot", json!({"headless": true}), Duration::from_secs(20)); + assert_eq!(r["result"]["width"], 640, "{r}"); + } +} diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs index 42f58fcc9..05013ce15 100644 --- a/crates/ui-egui/src/lib.rs +++ b/crates/ui-egui/src/lib.rs @@ -6,11 +6,13 @@ #![forbid(unsafe_code)] pub mod control; +pub mod headless; pub mod icons; pub mod menus; pub mod panels; pub mod render; pub mod shortcuts; +pub mod softpaint; pub mod state; pub mod theme; pub mod widgets; @@ -59,10 +61,13 @@ pub struct LightcraftApp { pub integrated_titlebar: bool, /// The host installed a native menu bar. pub native_menu: bool, + /// The host is [`headless::Headless`] (it answers viewport screenshot commands itself). + pub headless_host: bool, control_rx: Option>, - pending_screenshots: Vec<(u64, Option, Sender)>, - queued_screenshots: Vec<(u64, f64, u32)>, + pending_screenshots: Vec, screenshot_token: u64, + /// Offscreen context for headless screenshots of the windowed app. + shadow: Option, /// Synthetic input events (from the control channel) injected one step per frame. pub synthetic: Vec, styled: bool, @@ -87,10 +92,11 @@ impl LightcraftApp { perf: Perf::default(), integrated_titlebar: false, native_menu: false, + headless_host: false, control_rx: None, pending_screenshots: vec![], - queued_screenshots: vec![], screenshot_token: 0, + shadow: None, synthetic: vec![], styled: false, fonts_ready: false, @@ -104,6 +110,8 @@ impl LightcraftApp { pub fn with_control(mut self, rx: Receiver) -> Self { self.control_rx = Some(rx); + // keep CPU copies of photo textures so `ui.screenshot {"headless": true}` can draw them + self.renderer.keep_pixels = true; self } @@ -133,35 +141,95 @@ impl LightcraftApp { control::Outcome::Done(v) => { let _ = reply.send(v); } - control::Outcome::Screenshot { path } => { + control::Outcome::Screenshot { path, headless } => { self.screenshot_token += 1; - let token = self.screenshot_token; - self.queued_screenshots.push((token, now_ms() + 120.0, 0)); - self.pending_screenshots.push((token, path, reply)); + let now = now_ms(); + self.pending_screenshots.push(PendingShot { + token: self.screenshot_token, + path, + reply, + headless: headless && !self.headless_host, + not_before: now + 120.0, + deadline: now + SCREENSHOT_SETTLE_MS, + frames: 0, + sent_at: None, + }); } } } self.control_rx = Some(rx); } + /// Advance pending screenshots: wait (≥ 3 frames, ≥ 120 ms) until no renders are in flight + /// (or a timeout), then capture — via the host's compositor, or headlessly + /// ([`Self::headless_screenshot`]) when asked or when the compositor delivers nothing. fn issue_screenshots(&mut self, ctx: &egui::Context) { + if self.pending_screenshots.is_empty() { + return; + } let now = now_ms(); - let busy = self.renderer.queued() > 0; - self.queued_screenshots.retain_mut(|(token, at, frames)| { - *frames += 1; - // wait for pending renders (up to ~3 s) so screenshots show finished pixels - if now >= *at && *frames >= 3 && (!busy || *frames > 180) { - ctx.send_viewport_cmd(egui::ViewportCommand::Screenshot(egui::UserData::new(*token))); - false + let busy = self.renderer.in_flight() > 0; + let mut shots = std::mem::take(&mut self.pending_screenshots); + let mut shadow_ticked = false; + shots.retain_mut(|s| { + if let Some(sent) = s.sent_at { + if now - sent < SCREENSHOT_FALLBACK_MS || self.headless_host { + return true; + } + // The compositor delivered nothing (display asleep, window occluded): go headless. + log::warn!("ui.screenshot: no frame from the compositor after {SCREENSHOT_FALLBACK_MS} ms; rendering headlessly"); + s.sent_at = None; + s.headless = true; + s.deadline = now + SCREENSHOT_SETTLE_MS; + } + s.frames += 1; + let ready = now >= s.not_before && s.frames >= 3 && (!busy || now > s.deadline); + if s.headless { + // The shadow frame requests the renders the UI needs, even while the window shows nothing. + let img = if ready || !shadow_ticked { self.headless_screenshot(ctx, ready) } else { None }; + shadow_ticked = true; + if let Some(img) = img { + let _ = s.reply.send(control::save_screenshot(self, &img, s.path.as_deref())); + return false; + } + true } else { + if ready { + ctx.send_viewport_cmd(egui::ViewportCommand::Screenshot(egui::UserData::new(s.token))); + s.sent_at = Some(now); + } true } }); - if !self.queued_screenshots.is_empty() || !self.pending_screenshots.is_empty() { + shots.append(&mut self.pending_screenshots); + self.pending_screenshots = shots; + if !self.pending_screenshots.is_empty() { ctx.request_repaint_after(std::time::Duration::from_millis(16)); } } + /// Draw the UI into an offscreen context at the window's size; with `capture`, rasterize it + /// on the CPU (photo textures come from the renderer's CPU copies). + pub fn headless_screenshot(&mut self, main: &egui::Context, capture: bool) -> Option { + let mut view = self.shadow.take().unwrap_or_default(); + let size = main.input(|i| i.content_rect()).size(); + let size = if size.x >= 1.0 && size.y >= 1.0 { size } else { egui::vec2(1600.0, 1000.0) }; + let ppp = main.pixels_per_point(); + let time = main.input(|i| i.time); + if view.frames() == 0 { + // warm-up pass: activates our fonts (pending font definitions live in `Memory`, which + // is replaced below) + view.run(headless::HeadlessView::raw_input(size, ppp, time, vec![]), |_| {}); + } + // same scroll offsets, open sections, style… as the window + let memory = main.memory(|m| m.clone()); + view.ctx.memory_mut(|m| *m = memory); + view.run(headless::HeadlessView::raw_input(size, ppp, time, vec![]), |ui| self.ui(ui)); + let img = capture.then(|| view.paint(&self.renderer.cpu_textures())); + self.shadow = Some(view); + img + } + fn collect_screenshots(&mut self, ctx: &egui::Context) { if self.pending_screenshots.is_empty() { return; @@ -180,9 +248,9 @@ impl LightcraftApp { .collect() }); for (token, image) in events { - if let Some(i) = self.pending_screenshots.iter().position(|(t, _, _)| *t == token) { - let (_, path, reply) = self.pending_screenshots.remove(i); - let _ = reply.send(control::save_screenshot(self, &image, path.as_deref())); + if let Some(i) = self.pending_screenshots.iter().position(|s| s.token == token) { + let s = self.pending_screenshots.remove(i); + let _ = s.reply.send(control::save_screenshot(self, &image, s.path.as_deref())); } } } @@ -273,6 +341,24 @@ impl LightcraftApp { } } +/// How long a screenshot waits for in-flight renders. +const SCREENSHOT_SETTLE_MS: f64 = 3000.0; +/// How long a windowed screenshot waits for the compositor before falling back to headless. +const SCREENSHOT_FALLBACK_MS: f64 = 2000.0; + +/// A `ui.screenshot` request in progress. +struct PendingShot { + token: u64, + path: Option, + reply: Sender, + headless: bool, + not_before: f64, + deadline: f64, + frames: u32, + /// When the viewport screenshot command went out. + sent_at: Option, +} + /// Wall-clock milliseconds since the Unix epoch (`web-time` maps to `Date.now()` on the web). pub fn now_ms() -> f64 { use web_time::{SystemTime, UNIX_EPOCH}; diff --git a/crates/ui-egui/src/panels/dialogs.rs b/crates/ui-egui/src/panels/dialogs.rs index 05eda2d45..254f7ad5f 100644 --- a/crates/ui-egui/src/panels/dialogs.rs +++ b/crates/ui-egui/src/panels/dialogs.rs @@ -11,7 +11,11 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) { let Some(mut dlg) = app.ui.dialog.clone() else { return }; let t = Tokens::get(ctx); let screen = ctx.content_rect(); - ctx.layer_painter(egui::LayerId::new(egui::Order::Middle, egui::Id::new("dim"))).rect_filled(screen, 0.0, egui::Color32::from_black_alpha(140)); + // The backdrop is an area below the dialog window (a bare `Middle` layer painter would be + // painted after every area — i.e. over the dialog too). + egui::Area::new(egui::Id::new("dialog-dim")).order(egui::Order::Middle).fixed_pos(screen.min).interactable(false).show(ctx, |ui| { + ui.painter().rect_filled(screen, 0.0, egui::Color32::from_black_alpha(140)); + }); let mut close = false; let mut confirm = false; let title = match &dlg { @@ -24,9 +28,14 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) { Dialog::About => "About LightCraft", Dialog::Shortcuts => "Keyboard Shortcuts", }; - egui::Window::new(title).collapsible(false).resizable(false).anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0]).default_width(380.0).show( - ctx, - |ui| { + let frame = egui::Frame::window(&ctx.global_style()).inner_margin(egui::Margin::symmetric(16, 12)); + let shown = egui::Window::new(title) + .collapsible(false) + .resizable(false) + .frame(frame) + .anchor(egui::Align2::CENTER_CENTER, [0.0, 0.0]) + .default_width(380.0) + .show(ctx, |ui| { ui.spacing_mut().item_spacing.y = 8.0; match &mut dlg { Dialog::NewAlbum { name, .. } | Dialog::RenameAlbum { name, .. } => { @@ -177,8 +186,10 @@ pub fn show(app: &mut LightcraftApp, ctx: &egui::Context) { } } }); - }, - ); + }); + if let Some(w) = shown { + ctx.move_to_top(w.response.layer_id); + } if ctx.input(|i| i.key_pressed(egui::Key::Escape)) { close = true; } diff --git a/crates/ui-egui/src/panels/right.rs b/crates/ui-egui/src/panels/right.rs index d112c8b3c..e797ecfba 100644 --- a/crates/ui-egui/src/panels/right.rs +++ b/crates/ui-egui/src/panels/right.rs @@ -125,7 +125,7 @@ fn crop(app: &mut LightcraftApp, ui: &mut egui::Ui, id: PhotoId) { ("Golden", crate::state::CropOverlay::Golden), ("None", crate::state::CropOverlay::None), ] { - if ui.selectable_label(app.ui.crop_overlay == o, label).clicked() { + if text_button(ui, &format!("cropOverlay-{}", label.to_lowercase()), label, app.ui.crop_overlay == o).clicked() { app.ui.crop_overlay = o; } } diff --git a/crates/ui-egui/src/render.rs b/crates/ui-egui/src/render.rs index ecfe694b3..b84a4d5ec 100644 --- a/crates/ui-egui/src/render.rs +++ b/crates/ui-egui/src/render.rs @@ -29,6 +29,8 @@ pub struct Tex { pub size: [usize; 2], pub histogram: Option, pub ms: f64, + /// CPU copy of the pixels (only with [`Renderer::keep_pixels`]; for headless screenshots). + pub pixels: Option>, } pub struct Renderer { @@ -39,12 +41,22 @@ pub struct Renderer { pub last_main_ms: f64, /// Jobs finished since start (for inspect/perf). pub completed: u64, + /// Keep a CPU copy of every texture so the UI can be rasterized headlessly (set when the app + /// is driven by the control channel). + pub keep_pixels: bool, } impl Default for Renderer { fn default() -> Self { let threads = if cfg!(target_arch = "wasm32") { 0 } else { JobPool::::default_threads().min(6) }; - Renderer { pool: JobPool::new(threads), pending: HashMap::new(), textures: HashMap::new(), last_main_ms: 0.0, completed: 0 } + Renderer { + pool: JobPool::new(threads), + pending: HashMap::new(), + textures: HashMap::new(), + last_main_ms: 0.0, + completed: 0, + keep_pixels: false, + } } } @@ -107,11 +119,13 @@ impl Renderer { continue; }; let img = &rendered.image; - let color = egui::ColorImage::from_rgba_unmultiplied([img.width, img.height], &img.as_bytes()); + let color = std::sync::Arc::new(egui::ColorImage::from_rgba_unmultiplied([img.width, img.height], &img.as_bytes())); + let pixels = self.keep_pixels.then(|| color.clone()); let name = format!("{slot:?}"); match self.textures.get_mut(&slot) { Some(t) => { t.tex.set(color, egui::TextureOptions::LINEAR); + t.pixels = pixels; t.key = r.key; t.photo = r.photo; t.size = [img.width, img.height]; @@ -122,7 +136,7 @@ impl Renderer { let tex = ctx.load_texture(name, color, egui::TextureOptions::LINEAR); self.textures.insert( slot, - Tex { key: r.key, photo: r.photo, tex, size: [img.width, img.height], histogram: Some(rendered.histogram), ms }, + Tex { key: r.key, photo: r.photo, tex, size: [img.width, img.height], histogram: Some(rendered.histogram), ms, pixels }, ); } } @@ -137,6 +151,11 @@ impl Renderer { changed } + /// CPU copies of the current textures by id (see [`Self::keep_pixels`]). + pub fn cpu_textures(&self) -> HashMap { + self.textures.values().filter_map(|t| Some((t.tex.id(), crate::softpaint::CpuTexture::linear(t.pixels.clone()?)))).collect() + } + /// Drop thumbnails that aren't in `keep` (bounded memory for huge libraries), and queued /// thumbnail jobs for photos that scrolled out of `keep`. pub fn evict_thumbs(&mut self, keep: &std::collections::HashSet, max: usize) { diff --git a/crates/ui-egui/src/softpaint.rs b/crates/ui-egui/src/softpaint.rs new file mode 100644 index 000000000..fb7632d7c --- /dev/null +++ b/crates/ui-egui/src/softpaint.rs @@ -0,0 +1,471 @@ +//! A CPU rasterizer for egui's tessellated output, so the exact UI can be drawn into an image +//! without a window or a GPU (headless screenshots, CI, agents running while the display sleeps). +//! +//! It mirrors what egui's GPU backends do (`egui-wgpu`'s `fs_main_gamma_framebuffer`): +//! - vertex colours are sRGB-encoded, premultiplied [`Color32`]s, interpolated in gamma space; +//! - textures are sampled in gamma space (bilinear on texel centres, clamp to edge — like the +//! "predictable texture filtering" path), and multiplied with the vertex colour; +//! - blending is premultiplied `src + dst·(1 − src.a)` into an 8-bit gamma framebuffer; +//! - clip rects become integer scissor rects (`round(points · pixels_per_point)`). +//! +//! Triangles are rasterized at pixel centres with a consistent tie-break rule, so shared edges are +//! covered exactly once (no double-blended seams in anti-aliasing fringes). Paint callbacks are +//! skipped (the app draws everything — including photos — as textured meshes). Bands of rows are +//! rasterized in parallel; each pixel is touched by one thread in primitive order, so the output is +//! deterministic. + +use std::collections::HashMap; +use std::sync::Arc; + +use egui::epaint::{ClippedPrimitive, Primitive}; +use egui::{Color32, ColorImage, ImageData, TextureFilter, TextureId, TexturesDelta}; + +/// One texture's pixels (sRGB, premultiplied) and sampling filters. +#[derive(Clone)] +pub struct CpuTexture { + pub image: Arc, + pub magnification: TextureFilter, + pub minification: TextureFilter, +} + +impl CpuTexture { + pub fn linear(image: Arc) -> Self { + CpuTexture { image, magnification: TextureFilter::Linear, minification: TextureFilter::Linear } + } +} + +/// Looks up textures by id while painting. +pub trait TextureSource: Sync { + fn texture(&self, id: TextureId) -> Option<&CpuTexture>; +} + +/// CPU mirror of a context's textures, kept up to date from [`TexturesDelta`]s. +#[derive(Default)] +pub struct TextureStore { + map: HashMap, + /// Frees from the last delta; applied before the next one (the frame using them may still be + /// painted until then). + deferred_free: Vec, +} + +impl TextureStore { + /// Apply a frame's texture updates (call once per frame, before painting that frame). + pub fn apply(&mut self, mut delta: TexturesDelta) { + for id in self.deferred_free.drain(..) { + self.map.remove(&id); + } + for (id, d) in delta.set.iter().flat_map(|(id, ds)| ds.iter().map(move |d| (id, d))) { + let ImageData::Color(img) = &d.image; + match d.pos { + None => { + self.map + .insert(*id, CpuTexture { image: img.clone(), magnification: d.options.magnification, minification: d.options.minification }); + } + Some([x0, y0]) => { + let Some(t) = self.map.get_mut(id) else { continue }; + let dst = Arc::make_mut(&mut t.image); + let [dw, dh] = dst.size; + let [w, h] = img.size; + for y in 0..h.min(dh.saturating_sub(y0)) { + let n = w.min(dw.saturating_sub(x0)); + let s = &img.pixels[y * w..y * w + n]; + dst.pixels[(y0 + y) * dw + x0..(y0 + y) * dw + x0 + n].copy_from_slice(s); + } + t.magnification = d.options.magnification; + t.minification = d.options.minification; + } + } + } + self.deferred_free.extend(delta.free.iter().copied()); + delta.clear(); + } + + pub fn insert(&mut self, id: TextureId, t: CpuTexture) { + self.map.insert(id, t); + } + + pub fn len(&self) -> usize { + self.map.len() + } + + pub fn is_empty(&self) -> bool { + self.map.is_empty() + } +} + +impl TextureSource for TextureStore { + fn texture(&self, id: TextureId) -> Option<&CpuTexture> { + self.map.get(&id) + } +} + +/// Textures from `over` take precedence over `base` (e.g. the app's photo textures over an +/// offscreen context's font atlas). +pub struct Layered<'a> { + pub over: &'a HashMap, + pub base: &'a TextureStore, +} + +impl TextureSource for Layered<'_> { + fn texture(&self, id: TextureId) -> Option<&CpuTexture> { + self.over.get(&id).or_else(|| self.base.texture(id)) + } +} + +#[derive(Clone, Copy)] +struct PVert { + x: f64, + y: f64, + c: [f32; 4], + u: f32, + v: f32, +} + +struct Prepared<'a> { + /// Scissor rect in pixels: [x0, y0, x1, y1) . + clip: [usize; 4], + tex: Option<&'a CpuTexture>, + verts: Vec, + indices: &'a [u32], +} + +const BAND: usize = 32; + +/// Rasterize `primitives` into a `size` (pixels) image cleared to `clear`. The result is opaque if +/// `clear` is. +pub fn paint(primitives: &[ClippedPrimitive], textures: &dyn TextureSource, size: [usize; 2], pixels_per_point: f32, clear: Color32) -> ColorImage { + let [w, h] = size; + let ppp = pixels_per_point as f64; + let prepared: Vec> = primitives + .iter() + .filter_map(|cp| { + let Primitive::Mesh(mesh) = &cp.primitive else { return None }; + if mesh.indices.is_empty() { + return None; + } + let r = cp.clip_rect; + let px = |v: f32, max: usize| ((v * pixels_per_point).round().max(0.0) as usize).min(max); + let clip = [px(r.min.x, w), px(r.min.y, h), px(r.max.x, w), px(r.max.y, h)]; + if clip[0] >= clip[2] || clip[1] >= clip[3] { + return None; + } + let verts = mesh + .vertices + .iter() + .map(|v| { + let [r, g, b, a] = v.color.to_array(); + PVert { + x: v.pos.x as f64 * ppp, + y: v.pos.y as f64 * ppp, + c: [r as f32 / 255.0, g as f32 / 255.0, b as f32 / 255.0, a as f32 / 255.0], + u: v.uv.x, + v: v.uv.y, + } + }) + .collect(); + Some(Prepared { clip, tex: textures.texture(mesh.texture_id), verts, indices: &mesh.indices }) + }) + .collect(); + + let mut pixels = vec![clear; w * h]; + let bands: Vec<(usize, &mut [Color32])> = pixels.chunks_mut((BAND * w).max(1)).enumerate().map(|(i, c)| (i * BAND, c)).collect(); + let threads = thread_count().min(bands.len()).max(1); + if threads <= 1 { + for (y0, band) in bands { + paint_band(&prepared, band, y0, w); + } + } else { + let mut groups: Vec> = (0..threads).map(|_| Vec::new()).collect(); + for (i, b) in bands.into_iter().enumerate() { + groups[i % threads].push(b); + } + #[cfg(not(target_arch = "wasm32"))] + std::thread::scope(|s| { + for g in groups { + let prepared = &prepared; + s.spawn(move || { + for (y0, band) in g { + paint_band(prepared, band, y0, w); + } + }); + } + }); + #[cfg(target_arch = "wasm32")] + for g in groups { + for (y0, band) in g { + paint_band(&prepared, band, y0, w); + } + } + } + ColorImage::new(size, pixels) +} + +fn thread_count() -> usize { + if cfg!(target_arch = "wasm32") { 1 } else { std::thread::available_parallelism().map_or(1, |n| n.get()).min(16) } +} + +fn paint_band(prepared: &[Prepared<'_>], band: &mut [Color32], y0: usize, w: usize) { + if w == 0 { + return; + } + let y1 = y0 + band.len() / w; + for p in prepared { + let cy0 = p.clip[1].max(y0); + let cy1 = p.clip[3].min(y1); + if cy0 >= cy1 { + continue; + } + let clip = [p.clip[0], cy0, p.clip[2], cy1]; + for tri in p.indices.chunks_exact(3) { + let (Some(a), Some(b), Some(c)) = (p.verts.get(tri[0] as usize), p.verts.get(tri[1] as usize), p.verts.get(tri[2] as usize)) else { + continue; + }; + raster_triangle(band, y0, w, clip, p.tex, *a, *b, *c); + } + } +} + +/// Edge function `E(p) = A·x + B·y + C` of the directed edge a→b (positive on the inside of a +/// counter-clockwise-normalized triangle), plus whether this edge owns pixels exactly on it. +#[derive(Clone, Copy)] +struct Edge { + a: f64, + b: f64, + c: f64, + owns_ties: bool, +} + +impl Edge { + fn new(p: &PVert, q: &PVert) -> Self { + let a = p.y - q.y; + let b = q.x - p.x; + let c = -(a * p.x + b * p.y); + // Of two triangles sharing this edge, exactly one sees (A > 0) || (A == 0 && B > 0). + Edge { a, b, c, owns_ties: a > 0.0 || (a == 0.0 && b > 0.0) } + } + #[inline] + fn eval(&self, x: f64, y: f64) -> f64 { + self.a * x + self.b * y + self.c + } + #[inline] + fn inside(&self, w: f64) -> bool { + w > 0.0 || (w == 0.0 && self.owns_ties) + } +} + +#[allow(clippy::too_many_arguments)] +fn raster_triangle( + band: &mut [Color32], + band_y0: usize, + w: usize, + clip: [usize; 4], + tex: Option<&CpuTexture>, + v0: PVert, + mut v1: PVert, + mut v2: PVert, +) { + let area = (v1.x - v0.x) * (v2.y - v0.y) - (v1.y - v0.y) * (v2.x - v0.x); + if area == 0.0 || !area.is_finite() { + return; + } + if area < 0.0 { + std::mem::swap(&mut v1, &mut v2); + } + let area = area.abs(); + // Pixel-centre bounding box, clipped. + let min_x = v0.x.min(v1.x).min(v2.x); + let max_x = v0.x.max(v1.x).max(v2.x); + let min_y = v0.y.min(v1.y).min(v2.y); + let max_y = v0.y.max(v1.y).max(v2.y); + let x0 = ((min_x - 0.5).ceil().max(clip[0] as f64)) as usize; + let x1 = ((max_x - 0.5).floor() + 1.0).min(clip[2] as f64).max(0.0) as usize; + let y0 = ((min_y - 0.5).ceil().max(clip[1] as f64)) as usize; + let y1 = ((max_y - 0.5).floor() + 1.0).min(clip[3] as f64).max(0.0) as usize; + if x0 >= x1 || y0 >= y1 { + return; + } + // Barycentric weight of v0 comes from the edge opposite it (v1→v2), etc. + let e0 = Edge::new(&v1, &v2); + let e1 = Edge::new(&v2, &v0); + let e2 = Edge::new(&v0, &v1); + let inv = 1.0 / area; + + let white = Color32::WHITE; + let uniform_color = v0.c == v1.c && v1.c == v2.c; + let uniform_uv = v0.u == v1.u && v1.u == v2.u && v0.v == v1.v && v1.v == v2.v; + // Texture filter: magnify if the triangle covers more pixels than texels. + let filter = tex.map(|t| { + if t.magnification == t.minification { + return t.magnification; + } + let [tw, th] = t.image.size; + let uv_area = ((v1.u - v0.u) * (v2.v - v0.v) - (v1.v - v0.v) * (v2.u - v0.u)).abs() as f64 * tw as f64 * th as f64; + if uv_area <= area { t.magnification } else { t.minification } + }); + let sample = |u: f32, v: f32| -> [f32; 4] { + match (tex, filter) { + (Some(t), Some(f)) => sample_texture(&t.image, u, v, f), + _ => to_f(white), + } + }; + let const_tex = if uniform_uv { Some(sample(v0.u, v0.v)) } else { None }; + let const_src = match (uniform_color, const_tex) { + (true, Some(t)) => Some(mul(v0.c, t)), + _ => None, + }; + if let Some(s) = const_src + && s == [0.0; 4] + { + return; + } + + for y in y0..y1 { + let py = y as f64 + 0.5; + let row = &mut band[(y - band_y0) * w..(y - band_y0 + 1) * w]; + let px0 = x0 as f64 + 0.5; + let mut w0 = e0.eval(px0, py); + let mut w1 = e1.eval(px0, py); + let mut w2 = e2.eval(px0, py); + for dst in &mut row[x0..x1] { + if e0.inside(w0) && e1.inside(w1) && e2.inside(w2) { + let src = match const_src { + Some(s) => s, + None => { + let (l0, l1, l2) = ((w0 * inv) as f32, (w1 * inv) as f32, (w2 * inv) as f32); + let c = if uniform_color { v0.c } else { std::array::from_fn(|i| l0 * v0.c[i] + l1 * v1.c[i] + l2 * v2.c[i]) }; + let t = match const_tex { + Some(t) => t, + None => sample(l0 * v0.u + l1 * v1.u + l2 * v2.u, l0 * v0.v + l1 * v1.v + l2 * v2.v), + }; + mul(c, t) + } + }; + *dst = blend(*dst, src); + } + w0 += e0.a; + w1 += e1.a; + w2 += e2.a; + } + } +} + +#[inline] +fn to_f(c: Color32) -> [f32; 4] { + let [r, g, b, a] = c.to_array(); + [r as f32 / 255.0, g as f32 / 255.0, b as f32 / 255.0, a as f32 / 255.0] +} + +#[inline] +fn mul(a: [f32; 4], b: [f32; 4]) -> [f32; 4] { + [a[0] * b[0], a[1] * b[1], a[2] * b[2], a[3] * b[3]] +} + +/// Premultiplied "over" in gamma space, rounded to 8 bits (like a `Rgba8Unorm` framebuffer). +#[inline] +fn blend(dst: Color32, src: [f32; 4]) -> Color32 { + let k = 1.0 - src[3].clamp(0.0, 1.0); + let d = dst.to_array(); + let ch = |i: usize| ((src[i] + d[i] as f32 / 255.0 * k).clamp(0.0, 1.0) * 255.0 + 0.5) as u8; + Color32::from_rgba_premultiplied(ch(0), ch(1), ch(2), ch(3)) +} + +fn sample_texture(img: &ColorImage, u: f32, v: f32, filter: TextureFilter) -> [f32; 4] { + let [w, h] = img.size; + if w == 0 || h == 0 { + return [0.0; 4]; + } + let at = |x: i64, y: i64| to_f(img.pixels[y.clamp(0, h as i64 - 1) as usize * w + x.clamp(0, w as i64 - 1) as usize]); + match filter { + TextureFilter::Nearest => at((u * w as f32).floor() as i64, (v * h as f32).floor() as i64), + TextureFilter::Linear => { + let fx = u * w as f32 - 0.5; + let fy = v * h as f32 - 0.5; + let (x, y) = (fx.floor(), fy.floor()); + let (tx, ty) = (fx - x, fy - y); + let (x, y) = (x as i64, y as i64); + let (a, b, c, d) = (at(x, y), at(x + 1, y), at(x, y + 1), at(x + 1, y + 1)); + std::array::from_fn(|i| { + let top = a[i] + (b[i] - a[i]) * tx; + let bot = c[i] + (d[i] - c[i]) * tx; + top + (bot - top) * ty + }) + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use egui::epaint::{Mesh, Vertex}; + use egui::{Rect, pos2}; + + fn quad(rect: Rect, color: Color32) -> ClippedPrimitive { + let mut m = Mesh::default(); + m.add_colored_rect(rect, color); + ClippedPrimitive { clip_rect: Rect::EVERYTHING, primitive: Primitive::Mesh(m) } + } + + fn white_tex() -> TextureStore { + let mut s = TextureStore::default(); + s.insert(TextureId::default(), CpuTexture::linear(Arc::new(ColorImage::new([1, 1], vec![Color32::WHITE])))); + s + } + + #[test] + fn opaque_rect_fills_exactly_its_pixels() { + let img = paint(&[quad(Rect::from_min_max(pos2(2.0, 3.0), pos2(6.0, 5.0)), Color32::RED)], &white_tex(), [10, 8], 1.0, Color32::BLACK); + let n = img.pixels.iter().filter(|p| **p == Color32::RED).count(); + assert_eq!(n, 4 * 2); + assert_eq!(img.pixels[3 * 10 + 2], Color32::RED); + assert_eq!(img.pixels[2 * 10 + 2], Color32::BLACK); + } + + #[test] + fn shared_diagonal_is_not_blended_twice() { + // half-transparent quad: every covered pixel gets exactly one blend + let c = Color32::from_rgba_premultiplied(64, 64, 64, 128); + let img = paint(&[quad(Rect::from_min_max(pos2(0.0, 0.0), pos2(8.0, 8.0)), c)], &white_tex(), [8, 8], 1.0, Color32::BLACK); + let first = img.pixels[0]; + assert!(img.pixels.iter().all(|p| *p == first), "seam on the diagonal"); + assert_eq!(first, Color32::from_rgba_premultiplied(64, 64, 64, 255)); + } + + #[test] + fn clip_rect_and_scale_apply() { + let mut p = quad(Rect::from_min_max(pos2(0.0, 0.0), pos2(10.0, 10.0)), Color32::GREEN); + p.clip_rect = Rect::from_min_max(pos2(1.0, 1.0), pos2(3.0, 2.0)); + let img = paint(&[p], &white_tex(), [20, 20], 2.0, Color32::BLACK); + assert_eq!(img.pixels.iter().filter(|p| **p == Color32::GREEN).count(), 4 * 2); + assert_eq!(img.pixels[2 * 20 + 2], Color32::GREEN); + } + + #[test] + fn textured_triangle_samples_texels() { + let mut s = TextureStore::default(); + let tex = ColorImage::new([2, 1], vec![Color32::RED, Color32::BLUE]); + let id = TextureId::User(7); + s.insert(id, CpuTexture { image: Arc::new(tex), magnification: TextureFilter::Nearest, minification: TextureFilter::Nearest }); + let mut m = Mesh::with_texture(id); + m.add_rect_with_uv(Rect::from_min_max(pos2(0.0, 0.0), pos2(4.0, 2.0)), Rect::from_min_max(pos2(0.0, 0.0), pos2(1.0, 1.0)), Color32::WHITE); + let img = paint(&[ClippedPrimitive { clip_rect: Rect::EVERYTHING, primitive: Primitive::Mesh(m) }], &s, [4, 2], 1.0, Color32::BLACK); + assert_eq!(img.pixels[0], Color32::RED); + assert_eq!(img.pixels[3], Color32::BLUE); + let _ = Vertex::default(); + } + + #[test] + fn partial_texture_updates_patch_the_mirror() { + let mut s = TextureStore::default(); + let id = TextureId::Managed(0); + let mut d = TexturesDelta::default(); + d.push(id, egui::epaint::ImageDelta::full(ColorImage::new([4, 4], vec![Color32::BLACK; 16]), egui::TextureOptions::LINEAR)); + s.apply(d); + let mut d = TexturesDelta::default(); + d.push(id, egui::epaint::ImageDelta::partial([1, 2], ColorImage::new([2, 1], vec![Color32::WHITE; 2]), egui::TextureOptions::LINEAR)); + s.apply(d); + let t = s.texture(id).unwrap(); + assert_eq!(t.image.pixels[2 * 4 + 1], Color32::WHITE); + assert_eq!(t.image.pixels[2 * 4 + 2], Color32::WHITE); + assert_eq!(t.image.pixels[2 * 4 + 3], Color32::BLACK); + } +} diff --git a/docs/control-protocol.md b/docs/control-protocol.md index 875d2bcec..c3cb81035 100644 --- a/docs/control-protocol.md +++ b/docs/control-protocol.md @@ -31,6 +31,31 @@ mode ([mcp.md](mcp.md)) is a thin layer over this channel. Implementation: | `ui.set` | partial UI state, e.g. `{"view": "detail"}` | Resulting UI state | | `ui.dialog.confirm` / `ui.dialog.cancel` | — | Close the open dialog | | `ui.resize` | `{width, height}` | Resize the window | -| `ui.screenshot` | `{path?}` | `{path, width, height}` once the frame (with finished renders) is captured | +| `ui.screenshot` | `{path?, headless?}` | `{path, width, height}` once the frame (with finished renders) is captured. `headless: true` draws the UI on the CPU (no compositor needed); a windowed capture that gets no frame within 2 s falls back to headless automatically | | `ui.render` | `{id?, size?, path?}` | Render a photo (PNG to `path`), `{width, height}` | | `app.quit` | — | Close the app | + +## Headless rendering (no window, no GPU) + +The egui UI can be rasterized on the CPU (`crates/ui-egui/src/softpaint.rs`, driven by +`crates/ui-egui/src/headless.rs`): same tessellation, gamma-space premultiplied blending and +scissor clipping as egui's GPU backends, so the image matches the window (minus GPU dithering). + +- **In the running app:** `{"method": "ui.screenshot", "params": {"path": "a.png", "headless": true}}`. + The UI is drawn into an offscreen context from the app's logic tick, which keeps running when the + window is occluded or the display sleeps/locks. Windowed screenshots fall back to this after 2 s. +- **Without any app window:** `lightcraft-cli snapshot` runs a whole app session headlessly and + answers the same control requests (same handler) from a JSON-lines script: + +```text +lightcraft-cli snapshot --demo -o grid.png --size 1600x1000 [--scale 2] +lightcraft-cli snapshot --library DIR --script tour.jsonl -o shot.png +``` + + `tour.jsonl` holds one request per line (`#` comments allowed), e.g. + `{"method": "ui.set", "params": {"view": "detail", "right": "edit", "openSections": ["optics"]}}` + then `{"method": "ui.screenshot"}`. Replies are printed to stdout. A `ui.screenshot` without + `path` writes `-o` (then `OUT-2.png`, `OUT-3.png`, …); `ui.settle {timeoutMs?}` waits until no + renders are in flight. Each request runs frames until it is answered and its injected input + (clicks, keys, drags) has played out. Widget ids for `ui.clickWidget` come from `ui.widgets` + (e.g. `button:upright-auto`). A 1600×1000 demo snapshot takes ~1–2 s (debug build).