M5: prototyping tail: gamepad triggers, widgets in prototypes, mode lookups

The part of gap 15 that doesn't need video, from the public documentation of prototype triggers, plugins/widgets and prototype expressions.

- **Controller buttons on key/gamepad triggers.** A key trigger's key can be a controller button: `Gamepad A` ... `Gamepad Home`, the standard gamepad layout's 17 buttons (face buttons by position, shoulders, triggers, View/Menu, stick presses, d-pad, Home). The engine keeps its input-agnostic model (a key trigger with a key name; `figcraft_doc::proto::GAMEPAD_BUTTONS`, `gamepad_button`, `is_gamepad_button`, `uses_gamepad`). The interaction editor gets a Controller menu beside the key field; the trigger list shows the button's own name.
- **Reading controllers.** A new `Services::gamepad_held()` (bitmask of held buttons; default none). The player polls it only while the presented prototype has a controller trigger and the window has the focus (about 30 times a second), plays each press once (held buttons don't repeat), and fires the first matching key/gamepad trigger on the screen like a key.
  - Desktop, Windows: Windows.Gaming.Input (Xbox-compatible controllers) through the `windows` crate the app already depends on (one more feature flag, `Gaming_Input`; no new crate).
  - Browser: the Gamepad API through `web-sys` (features `Gamepad`, `GamepadButton`, `GamepadMappingType`; standard-mapped controllers only).
  - macOS, Linux, FreeBSD: no controllers yet (documented gap).
  - Control channel: `ui.input {events: [{gamepad: "A" | "Gamepad A" | 0}]}` presses a button in the prototype playing (presenter, else preview pane); an error when nothing plays.
- **REST import** maps controller triggers (`device` other than `KEYBOARD`) to the button at the code's standard-layout index instead of skipping them; two buttons at once or an unknown code is skipped with a warning (`.fig` goes through the same path).
- **Widgets react to clicks in a presented prototype.** A click on a widget's clickable part in the presenter or the preview pane (when no interaction on the layer takes it) calls the plugin back through the canvas's own callback with the same sandbox limits, on the player's copy of the file (`widgets::click_on`, a scratch session holding the copy and the installed plugins): the widget changes in the prototype only, the file and its undo history stay untouched, restarting starts the widget over. Failures (plugin missing or failing) toast; toasts now also show over the full-window presenter.
- **Variable mode lookups.** Lookups (`accent[Dark]`, `mode(Theme)`), set-variable-mode affecting later expressions in the same interaction, and every operator the documentation lists already held; they are now behaviour cases. New `proto.evaluate {expression | expr, id?}` (MCP and agents) computes an expression where a layer's interaction would run, without changing anything. No other cheap expression gap found: every documented operator (numbers, text join, booleans, comparisons, negation, mode lookup) already existed.
- Docs: prototyping-parity (new rule rows, features, revision), target-app-parity area 13 and totals, ROADMAP, gaps.md gap 15, roadmap.md M5, mcp.md, plugins.md, control-protocol.md.

Dependencies: none added. Windows uses the existing `windows` 0.62 dependency with the `Gaming_Input` feature; the browser uses the existing `web-sys` with three more features. The usual Rust controller library (gilrs) was not taken: its Linux backend links the system's libudev (a C library, needing headers on CI and the FreeBSD job), which breaks the pure-Rust rule.
This commit is contained in:
bflatastic
2026-10-12 05:28:53 +07:00
committed by BFlat
parent def6f84b22
commit 119b8c7441
34 changed files with 945 additions and 59 deletions
+12 -4
View File
@@ -358,7 +358,12 @@ Interactions are `proto.interaction.add` / `set` / `remove` / `clear` (a navigat
the scroll position with `preserveScroll: true`; `{type: "scrollTo", dest, offset?: [x, y]}` stops
the layer that far from the top-left of the innermost frame scrolling it, e.g. below a fixed
header). A key trigger takes a key or a combination (`{type: "keyPress", key: "Shift+K"}`; Ctrl,
Alt, Shift and Cmd, all held and no others). In the player, an "on drag" trigger with a move, push
Alt, Shift and Cmd, all held and no others) or a controller button (`key: "Gamepad A"`; `Gamepad`
followed by `A`, `B`, `X`, `Y`, `LB`, `RB`, `LT`, `RT`, `View`, `Menu`, `LS`, `RS`, `Up`, `Down`,
`Left`, `Right` or `Home`: the standard layout, face buttons by position), which the player reads
from controllers on Windows and in the browser and which the control channel presses with
`ui.input {events: [{gamepad: "A"}]}`. Widgets react to clicks in the player, on its own copy of
the file (see [plugins.md](plugins.md)). In the player, an "on drag" trigger with a move, push
or slide animation starts only for a drag that way, and the animation follows the pointer until it
lets go; spring easings play until they settle (their duration follows from the spring, not
`durationMs`); an overlay's after-delay timer starts when it opens; instances of a component play
@@ -374,7 +379,10 @@ a variable) or `{op: {op, args}}` with `add` (text joins when a string is involv
`and`, `or`, `not` and `negate`; `{inMode: {variable, mode}}` reads a variable's value in one mode
of its collection whichever mode is chosen, and `{mode: {collection}}` gives the name of the mode a
collection is in where the interaction runs (as text; it follows set variable mode actions).
Variables, collections and modes may be given by name; they are stored by id. Expressions can also
Variables, collections and modes may be given by name; they are stored by id.
`proto.evaluate {expression | expr, id?}` computes an expression without changing anything, with
variables in the modes where layer `id`'s interaction would run (default: the default modes) →
`{value, type, text}`; it fails with what's wrong, like the interaction commands. Expressions can also
be written as text: `expression: "count + 1"`
on a set variable action, or `expression: "count >= 3 and not done"` on a conditional block, in
place of `expr` / `condition` (variables by name, `{name}` for any name; numbers, `"text"`,
@@ -678,13 +686,13 @@ The JSON returned by the public REST API's file endpoint (`GET /v1/files/:key`,
`run_command {command: "file.import", params: {path}}` (pages appended to the current document,
with their flows, prototype settings and variables).
The result's `warnings` list what was skipped or approximated (unsupported layer, paint and effect
types, video and gamepad triggers, expression functions other than a variable mode lookup over a
types, video triggers, controller triggers on more than one button, expression functions other than a variable mode lookup over a
variable and one of its modes, bindings to library
variables, instances of components outside the file, which come in as frames). Pages, the layer
tree, geometry, paints, strokes, effects, corners, constraints, auto layout, layout grids, exports,
text with styled runs, vectors, components, component sets, instances and component properties are
mapped, and so is prototyping: `interactions` (on click, hover, press, drag, after delay, mouse
enter/leave/down/up and key triggers; navigate, overlays, back, scroll to, change to, open link,
enter/leave/down/up and key triggers, controller triggers as controller buttons; navigate, overlays, back, scroll to, change to, open link,
set variable (values, variable references and expressions), set variable mode and conditional
(if / else if / else over expressions) actions; dissolve, smart animate, move, push and slide transitions with their easing,
custom bezier and spring curves included), flow starting points, scroll overflow and fixed/sticky