mirror of
https://github.com/storytold/soundcraft.git
synced 2026-10-24 10:36:46 -06:00
coygeek
1f5bc965fe
Resolve System appearance from egui on every frame and fall back to Dark when the integration cannot report a theme. Keep receiving OS changes during manual modes and apply the selected palette to both widget style branches and the current root UI. Remove the cached detector and its unused dependency. Cover startup, live transitions, manual overrides, saved System preferences, and user colors with an application-frame regression. Add reproducible offscreen system-theme inputs and document the fallback.
75 lines
4.4 KiB
Markdown
75 lines
4.4 KiB
Markdown
# Control protocol
|
|
|
|
SoundCraft can be driven completely by other programs: scripts, test harnesses and AI agents.
|
|
Every menu item, button, mouse gesture and keystroke maps to a **command** (or to synthetic
|
|
input), and every command is reachable from:
|
|
|
|
| Surface | How |
|
|
|---|---|
|
|
| Desktop app control channel | `soundcraft --control <port>` (`0` = pick a free port; printed as `SOUNDCRAFT_CONTROL_PORT=…`) |
|
|
| CLI against a running app | `soundcraft-cli app --port <port> <method or command id> [JSON]` |
|
|
| Headless CLI | `soundcraft-cli run --demo --cmd 'mix.volume={"track":"Kick","db":-6}' --bounce out.wav` |
|
|
| MCP (headless) | `soundcraft-cli mcp [--demo]` |
|
|
| MCP (bridged to the app) | `soundcraft-cli mcp --connect <port>` |
|
|
|
|
## Wire format
|
|
|
|
Newline-delimited JSON over TCP on `127.0.0.1`. One request per line, one reply per line:
|
|
|
|
```json
|
|
{"id": 1, "method": "engine.execute", "params": {"command": "track.new", "params": {"count": 2, "format": "Stereo"}}}
|
|
{"id": 1, "ok": true, "result": {"tracks": [12, 13]}}
|
|
```
|
|
|
|
Errors: `{"id": 1, "ok": false, "error": "bad parameters for `mix.volume`: no track named `Nope`"}`.
|
|
|
|
## Methods
|
|
|
|
| Method | Params | What it does |
|
|
|---|---|---|
|
|
| `engine.execute` (alias `command`) | `{command, params}` | Run a command programmatically. **Never opens a dialog**; empty params use defaults or the current selection. |
|
|
| `engine.commands` | `{filter?}` | Every command: id, label, menu path, shortcut, params doc, enabled + reason. Includes UI-layer commands (`window.*`). |
|
|
| `engine.parity` | `{}` | Parity report against the incumbent's menu catalog. |
|
|
| `session.inspect` | `{detail?: "summary"|"full"}` | The document: tracks, clips (with times), inserts (+params when full), sends, routing, automation, markers, groups, busses, sources (`loaded` true when decoded audio is in the pool), selection, transport, undo label. |
|
|
| `ui.inspect` | `{}` | UI state (`window`, panels, dialogs), window size, playing/position, audio device, and `edit_layout` (timeline rect and every track row's rect, for clicking). |
|
|
| `ui.set` | any `UiState` fields | e.g. `{"window": "Mix", "narrow_mix": true}`. |
|
|
| `ui.menu.list` | `{}` | Every catalog menu path and the command that implements it (or null). |
|
|
| `ui.menu.invoke` | `{path}` or `{id}` | Like clicking the menu item: may open its dialog (e.g. `"Track > New..."`). |
|
|
| `ui.click` | `{x, y, button?, count?, cmd?, shift?, alt?, ctrl?}` | Synthetic click (window points). |
|
|
| `ui.drag` | `{x, y, to_x, to_y, steps?}` | Synthetic drag with the primary button. |
|
|
| `ui.move` | `{x, y}` | Move the pointer. |
|
|
| `ui.key` | `{key, cmd?, shift?, alt?, ctrl?}` | Key press (`Space`, `Enter`, `A`, `F7`, `=`…). |
|
|
| `ui.text` | `{text}` | Type text into the focused field. |
|
|
| `ui.screenshot` | `{path?}` | PNG of the window (base64 in `png_base64` when no path). |
|
|
| `app.quit` | `{}` | Quit. |
|
|
|
|
## Positions and references
|
|
|
|
- Tracks: id (number) or name (`"Kick"`).
|
|
- Clips: id (from `session.inspect`).
|
|
- Positions: samples (`96000`), seconds (`{"seconds": 2.5}`), or a time string in any timebase,
|
|
optionally prefixed: `"bars_beats:5|1|000"`, `"min_secs:0:12.500"`, `"timecode:00:00:10:00"`,
|
|
`"samples:48000"`.
|
|
- Ranges: `start` + `end` (or `start` + `length`); omitted = the current edit selection.
|
|
|
|
## Examples
|
|
|
|
```sh
|
|
P=$(soundcraft --demo --control 0 | sed -n 's/SOUNDCRAFT_CONTROL_PORT=//p') # or read stderr
|
|
soundcraft-cli app --port $P mix.solo '{"track":"Bass"}'
|
|
soundcraft-cli app --port $P transport.play
|
|
soundcraft-cli app --port $P ui.screenshot '{"path":"/tmp/shot.png"}'
|
|
soundcraft-cli app --port $P ui.menu.invoke '{"path":"Track > New..."}'
|
|
```
|
|
|
|
Offscreen (no window): `cargo run -p soundcraft-ui-egui --example ui_shot -- out.png script.jsonl`,
|
|
where each script line is a request (`{"method": …, "params": …}`), `{"shot": "path.png"}` or
|
|
`{"steps": n}`. Use `--system-theme light|dark|none` to supply the initial OS appearance, and
|
|
script events such as `{"system_theme": "dark"}` to change it on subsequent frames.
|
|
|
|
Appearance defaults to Dark. `ui.theme {"mode":"system"}` follows the appearance supplied by
|
|
the native or web integration at launch and on later frames, while the saved choice stays System.
|
|
When the integration cannot report an appearance, System uses Dark. Manual Light and Dark stay
|
|
fixed when the OS appearance changes. An offscreen render without `--system-theme` supplies no
|
|
OS appearance and therefore resolves System to Dark.
|