M8: the control channel answers while the window is minimized (#720) (#751)

- A minimised or covered window now answers every control request that acts on the app's state
  (`engine.execute`, `document.inspect`, `node.get`, `ui.inspect`, `ui.pointer`, `ui.key`, `ui.type`,
  `ui.set`, `ui.menu.invoke`, `ui.render`, `window.list`, `window.focus`, `app.quit`), in that window's
  file, from the app's `logic` (no frame drawn). Before, only six methods were answered there.
- What only a drawn frame can answer (`ui.screenshot`, `ui.input`, `window.new`) is an error at once
  for a hidden window: `window 0 is minimised or hidden: `ui.screenshot` needs it on screen
  (`ui.render` draws the document without it)`. Before, these waited up to 60 s.
- `crates/ui-egui/src/control.rs`: the `NO_FRAME` allow-list becomes a short `NEEDS_FRAME` list;
  `drain_hidden` answers the rest and fails the frame-only ones at once.
- Docs: `docs/control-protocol.md` (hidden windows, `ui.screenshot`) and `docs/mcp.md` (attached
  mode while the window is minimised). ROADMAP progress log and revision history.

Part of #720.
This commit is contained in:
BFlat
2026-10-17 20:29:32 +07:00
committed by GitHub
parent ac4b98dbd7
commit b8f97a5f4f
6 changed files with 53 additions and 25 deletions
+2
View File
@@ -131,6 +131,7 @@ Ranked; detail and milestones in [`docs/roadmap.md`](docs/roadmap.md).
## Progress log
- **2026-10-17** · Control channel while the window is minimised (M8, #720): a minimised or covered window answers every request that acts on the app's state (edits, `ui.inspect`, `ui.pointer`, `ui.key`, `ui.type`, `ui.set`, `app.quit`…) without drawing, and what needs a drawn frame (`ui.screenshot`, `ui.input`, `window.new`) is an error at once; before, those waited up to 60 s and held up every later request on the connection (an attached agent's `inspect_ui` or window screenshot stalled until the window came back). Measured on Windows 11, minimised: requests ~100 ms each (eframe wakes a minimised window at most ten times a second; ~2 ms on screen), a window screenshot fails in ~100 ms instead of 60 s, idle CPU 0. Numbers unchanged.
- **2026-10-17** · Number variables on text content (M3, #629): the Typography section's variable picker also lists number variables for a text layer's content, as the engine already accepted (the text shows the number, e.g. 42.5, and follows the variable). Other text fields keep their own types. Numbers unchanged.
- **2026-10-17** · Frames and sections convert into each other (M1, #719): `object.convertToSection` turns a frame on the page or in a section into a section in place (same box, name, fill and layers, which stay where they are; what only a frame has goes: auto layout, clipping, layout grids, guides, scrolling), and `object.convertToFrame` now also turns a section into a frame that keeps its fill and doesn't clip, so the look stays. Nested frames, instances, main components and sections holding sections are refused with the reason; one undo step each. Both are in the canvas context menu (Convert to section / Convert to frame) and in Spanish. Numbers unchanged.
- **2026-10-14** · Corner handles win over the corner-radius handles (M1, #292): a press on the inner half of a corner handle's square picked the radius handle 10 px in, so the drag neither resized nor (dragging outward) rounded anything. Inside the drawn square the corner now resizes both sides (in ratio with Shift or the link); further in the radius handle still answers. Regression test in the transform behaviour suite. Numbers unchanged.
@@ -315,6 +316,7 @@ Ranked; detail and milestones in [`docs/roadmap.md`](docs/roadmap.md).
| Date | Change | Summary |
|---|---|---|
| 2026-10-17 | minor | Control channel while the window is minimised (M8, #720): a minimised or covered window answers every request that acts on the app's state (edits, `ui.inspect`, `ui.pointer`, `ui.key`, `ui.type`, `ui.set`, `app.quit`…) without drawing, and what needs a drawn frame (`ui.screenshot`, `ui.input`, `window.new`) is an error at once; before, those waited up to 60 s and held up every later request on the connection (an attached agent's `inspect_ui` or window screenshot stalled until the window came back). Measured on Windows 11, minimised: requests ~100 ms each (eframe wakes a minimised window at most ten times a second; ~2 ms on screen), a window screenshot fails in ~100 ms instead of 60 s, idle CPU 0. Numbers unchanged. |
| 2026-10-17 | minor | Community fixes from #524 up (batches #727, #728, #733, #735, #736 and #729; 55 PRs) and a crash guard. `.fig` import: explicit instance identity, overrides by key, text style precedence, grid track sizes, slot content, connector labels, skip ink, paragraph options and hanging lists, list start numbers, vertical trim, per-point stroke joins, text on a path, image adjustments and embedded images; large files open within budgets from the file's own size; an import report after opening; imported bytes stay out of the command journal. Files and paste: save a copy as `.fig`; Cmd/Ctrl+V pastes images copied in other apps and layers copied from another design app (`edit.pasteDesignLayers`); paste over and paste properties; clipboard reads on Linux and the BSDs time out after 2 s. Editor: live font preview on hover; Shift+I and Resources open Assets; Shift-drag moves the selection; thin strokes take a nearby double-click; dropped points merge; Flatten keeps open paths; the Actions palette gets tabs, ranked search, a keyboard highlight and recents; the colour picker opens on a new fill or stroke; text strokes follow their alignment; selection highlights after line breaks; a draggable pages/layers divider; variant properties reorder by drag; submenus keep their hover with a safe triangle; menu actions that need a value ask for it; File ▸ Rename edits the name in place. Engine: bad paint indexes, non-finite geometry and pages in copy/cut are refused; a param naming a layer says so when it isn't one; a CLI script is one undo step and resolves `$step`. Server and collaboration: message limits follow `max_file_mb` (client capped at 1 GiB); a damaged generation field no longer drops flushed edits; the web build joins a session from a `#join=` link. Also Swedish command labels, Nix CI through the shared workflow, paint bounds once per render, and a guard that turns a panic inside the boolean-op library into bad input (the stroke draws unclipped). Numbers unchanged. |
| 2026-10-17 | minor | Number variables on text content (M3, #629): the Typography section's variable picker also lists number variables for a text layer's content, as the engine already accepted (the text shows the number, e.g. 42.5, and follows the variable). Other text fields keep their own types. Numbers unchanged. |
| 2026-10-17 | minor | Frames and sections convert into each other (M1, #719): `object.convertToSection` turns a frame on the page or in a section into a section in place (same box, name, fill and layers, which stay where they are; what only a frame has goes: auto layout, clipping, layout grids, guides, scrolling), and `object.convertToFrame` now also turns a section into a frame that keeps its fill and doesn't clip, so the look stays. Nested frames, instances, main components and sections holding sections are refused with the reason; one undo step each. Both are in the canvas context menu (Convert to section / Convert to frame) and in Spanish. Numbers unchanged. |
+17 -8
View File
@@ -139,8 +139,10 @@ pub fn drain(app: &mut FigApp, ctx: &Context) {
}
}
/// Methods that need no frame drawn (#269): a hidden window answers them too.
const NO_FRAME: &[&str] = &["engine.execute", "engine.commands", "document.inspect", "node.get", "window.list", "ui.render"];
/// Methods only a drawn frame can answer: a capture of the window, raw window input and a new
/// window (the main window's frame opens it). Everything else acts on the app's state, which a
/// hidden window answers without a frame (#269, #720).
const NEEDS_FRAME: &[&str] = &["ui.screenshot", "ui.input", "window.new"];
/// Whether window `id` draws no frames now: minimised, or covered by other windows (on macOS also
/// on another Space).
@@ -150,10 +152,11 @@ fn hidden(ctx: &Context, id: u64) -> bool {
/// A hidden window (minimised, or covered by other windows) draws no frames, but the desktop app's
/// `logic` still runs, for the main window even while every window is hidden: it answers the
/// requests for any hidden window that need no frame, in that window's file, so an attached agent
/// can edit, inspect and save. The rest (screenshots, input, the interface's own state) wait for the
/// window to show; another window on screen is asked to draw, which the main window's frame would
/// do but doesn't while it is hidden. A request for a window that doesn't exist is an error at once.
/// requests for any hidden window, in that window's file, so an attached agent can edit, inspect,
/// type and save while the app is out of sight. What needs a frame ([`NEEDS_FRAME`]) is an error at
/// once instead of waiting for the window to show. A request for a window on screen waits for its
/// frame; another window on screen is asked to draw, which the main window's frame would do but
/// doesn't while it is hidden. A request for a window that doesn't exist is an error at once.
pub fn drain_hidden(app: &mut FigApp, ctx: &Context) {
for r in std::mem::take(&mut app.control_queue) {
if r.is_cancelled() {
@@ -164,15 +167,21 @@ pub fn drain_hidden(app: &mut FigApp, ctx: &Context) {
let _ = r.reply.send(ControlReply { id: r.id, result: Err(format!("no window {target}")) });
continue;
}
if !hidden(ctx, target) || !NO_FRAME.contains(&r.method.as_str()) {
if !hidden(ctx, target) {
// Another window on screen draws only when asked: while the main window is hidden no
// frame of it does that (`drain`), so ask here.
if target != app.windows.current && !hidden(ctx, target) {
if target != app.windows.current {
ctx.request_repaint_of(crate::windows::viewport(target));
}
app.control_queue.push(r);
continue;
}
if NEEDS_FRAME.contains(&r.method.as_str()) {
let error =
format!("window {target} is minimised or hidden: `{}` needs it on screen (`ui.render` draws the document without it)", r.method);
let _ = r.reply.send(ControlReply { id: r.id, result: Err(error) });
continue;
}
// Outside a frame, so outside the frame's guard: within the window's swap, a panic still
// ends in recovery copies and leaves every window its own state.
let res = crate::windows::as_window(app, target, |app| {
+12 -7
View File
@@ -332,9 +332,10 @@ fn cjk_names_are_looked_for_in_edits_not_on_every_pan() {
}
#[test]
fn a_hidden_window_answers_requests_that_need_no_frame() {
// #269: minimised or covered, the window draws no frames, but the app's logic still runs: it
// answers edits, inspection and saves; what needs a frame waits for the window to show.
fn a_hidden_window_answers_at_once() {
// #269, #720: minimised or covered, the window draws no frames, but the app's logic still runs:
// it answers edits, inspection, saves and the interface's state; what needs a frame (a capture
// of the window) is an error at once instead of waiting for the window to show.
let dir = std::env::temp_dir().join(format!("figcraft-hidden-save-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("hidden.figcraft");
@@ -350,6 +351,7 @@ fn a_hidden_window_answers_requests_that_need_no_frame() {
};
h.app.control_queue.push(request(1, "engine.execute", json!({"command": "file.save", "params": {"path": path.to_string_lossy()}})));
h.app.control_queue.push(request(2, "ui.inspect", json!({})));
h.app.control_queue.push(request(3, "ui.screenshot", json!({"path": dir.join("shot.png").to_string_lossy()})));
let mut hidden = RawInput::default();
hidden.viewports.insert(ViewportId::ROOT, egui::ViewportInfo { minimized: Some(true), ..Default::default() });
let app = &mut h.app;
@@ -357,10 +359,13 @@ fn a_hidden_window_answers_requests_that_need_no_frame() {
let saved = rx.try_recv().unwrap();
assert_eq!(saved.id, json!(1));
assert!(saved.result.is_ok() && path.exists(), "saved while hidden: {:?}", saved.result);
assert!(rx.try_recv().is_err(), "the interface's state waits for a frame");
// Shown again: the next frame answers it.
h.frame(vec![]);
assert_eq!(rx.try_recv().unwrap().id, json!(2));
let inspected = rx.try_recv().unwrap();
assert_eq!(inspected.id, json!(2));
assert!(inspected.result.is_ok(), "the interface's state: {:?}", inspected.result);
let shot = rx.try_recv().unwrap();
assert_eq!(shot.id, json!(3));
assert!(shot.result.unwrap_err().contains("minimised or hidden"), "a capture needs the window on screen");
assert!(h.app.control_queue.is_empty(), "nothing waits for the window");
let _ = std::fs::remove_dir_all(&dir);
}
+5 -4
View File
@@ -116,13 +116,14 @@ fn another_hidden_window_answers_requests_that_need_no_frame_in_its_own_file() {
assert_eq!(inspect.try_recv().expect("answered").result.unwrap()["name"], json!("A"));
assert_eq!(app.session.ds().unwrap().doc.name, "B");
// What needs a frame keeps waiting for the other window; an unknown window is an error at once.
let ui = request(&mut app, "ui.inspect", json!({"window": w}));
// What needs a frame, and an unknown window, are errors at once: nothing waits for a window
// that may stay hidden.
let shot = request(&mut app, "ui.screenshot", json!({"window": w, "path": "unused.png"}));
let unknown = request(&mut app, "window.list", json!({"window": 99}));
logic(&ctx, &mut app, &[ViewportId::ROOT, other]);
assert!(ui.try_recv().is_err(), "waits for the window to show");
assert!(shot.try_recv().expect("answered").result.unwrap_err().contains(&format!("window {w} is minimised or hidden")));
assert!(unknown.try_recv().expect("answered").result.unwrap_err().contains("no window 99"));
assert_eq!(app.control_queue.len(), 1);
assert!(app.control_queue.is_empty());
}
#[test]
+11 -6
View File
@@ -37,11 +37,15 @@ requests still queued after the UI timeout are discarded, including requests def
window. A command already running is not interrupted.
A minimised or covered window (the main one or another, see Windows below) draws no frames, but
still answers `engine.execute`, `engine.commands`, `document.inspect`, `node.get`, `window.list`
and `ui.render`, acting on the file that window shows, so an attached agent can edit, inspect and
save while the app is out of sight. Screenshots, input (`ui.input`,
`ui.pointer`, `ui.key`, `ui.type`) and the interface's own state wait for the window to show, and
time out after 60 s if it doesn't.
still answers every method that acts on the app's state (`engine.execute`, `document.inspect`,
`node.get`, `ui.inspect`, `ui.pointer`, `ui.key`, `ui.type`, `ui.set`, `ui.render`, `window.list`,
…), in the file that window shows, so an attached agent can edit, inspect and save while the app is
out of sight. The app is not drawn meanwhile and only wakes for requests, at most ten times a
second while minimised, so a request there takes up to about 0.1 s. What only a drawn frame can
answer (`ui.screenshot`, `ui.input`, `window.new`) is an error at once (`window 0 is minimised or
hidden: …`) instead of waiting for the window to show; `ui.render` draws the document without it.
Under Wayland the app can't tell that its window is minimised: where the compositor then stops
drawing it, a request waits for the window and gets the 60 s timeout error.
Coordinates are page units, y down. `x`/`y` in `node.set`/`node.create` are relative to the parent
frame (groups have no coordinate space of their own).
@@ -292,7 +296,8 @@ focus view shows, or `null`) and `annotationFilter`).
PNG of the whole window (`window` picks one), UI included (needs a visible window; an idle one draws the frame to
capture). Returns `{path, width, height}`, without `path` `{format: "png", width, height, data}`
(base64), or an error after 60 s when the window doesn't deliver.
(base64); an error at once when the window is minimised or hidden, or after 60 s when the window
doesn't deliver.
### `ui.render {path?, scale?, ids?}`
+6
View File
@@ -26,6 +26,12 @@ If that connection cannot be established, the error says the request was not sen
A connection the app closed while it sat idle is noticed before the next request is written and
replaced then, so a pause between calls is not an error.
The app keeps answering while its window is minimised or covered by other windows (see
`docs/control-protocol.md`): edits, inspection (`inspect_ui` too), pointer, key and text input and
renders work as usual (minimised, each request to the app takes up to about 0.1 s). Only a capture
of the window itself (`screenshot` with `window`) needs it on screen: it is then an error at once;
`screenshot` without `window` renders the document instead.
### Confining file access
```sh