mirror of
https://github.com/storytold/soundcraft.git
synced 2026-10-24 10:36:46 -06:00
Brandon ThomasandClaude Opus 5.5
a28e15210c
- mix: independent strips process in parallel (rayon, native), parameter pushes cached, plugin sync/routing order cached per session structure; plugin and render benchmarks - UI: Playback Engine, I/O Setup (bus rename/delete), Keyboard Shortcuts windows; clip gain line + info; folder open/close; tempo changes from the ruler; Inserts/Sends F-J, EQ curve and comments sections in the Mix window - docs: control protocol and MCP guides Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
68 lines
3.7 KiB
Markdown
68 lines
3.7 KiB
Markdown
# Control protocol
|
|
|
|
SoundCraft can be driven completely by other programs: scripts, test harnesses and AI agents.
|
|
Every menu item, button, mouse gesture and keystroke maps to a **command** (or to synthetic
|
|
input), and every command is reachable from:
|
|
|
|
| Surface | How |
|
|
|---|---|
|
|
| Desktop app control channel | `soundcraft --control <port>` (`0` = pick a free port; printed as `SOUNDCRAFT_CONTROL_PORT=…`) |
|
|
| CLI against a running app | `soundcraft-cli app --port <port> <method or command id> [JSON]` |
|
|
| Headless CLI | `soundcraft-cli run --demo --cmd 'mix.volume={"track":"Kick","db":-6}' --bounce out.wav` |
|
|
| MCP (headless) | `soundcraft-cli mcp [--demo]` |
|
|
| MCP (bridged to the app) | `soundcraft-cli mcp --connect <port>` |
|
|
|
|
## Wire format
|
|
|
|
Newline-delimited JSON over TCP on `127.0.0.1`. One request per line, one reply per line:
|
|
|
|
```json
|
|
{"id": 1, "method": "engine.execute", "params": {"command": "track.new", "params": {"count": 2, "format": "Stereo"}}}
|
|
{"id": 1, "ok": true, "result": {"tracks": [12, 13]}}
|
|
```
|
|
|
|
Errors: `{"id": 1, "ok": false, "error": "bad parameters for `mix.volume`: no track named `Nope`"}`.
|
|
|
|
## Methods
|
|
|
|
| Method | Params | What it does |
|
|
|---|---|---|
|
|
| `engine.execute` (alias `command`) | `{command, params}` | Run a command programmatically. **Never opens a dialog**; empty params use defaults or the current selection. |
|
|
| `engine.commands` | `{filter?}` | Every command: id, label, menu path, shortcut, params doc, enabled + reason. Includes UI-layer commands (`window.*`). |
|
|
| `engine.parity` | `{}` | Parity report against the incumbent's menu catalog. |
|
|
| `session.inspect` | `{detail?: "summary"|"full"}` | The document: tracks, clips (with times), inserts (+params when full), sends, routing, automation, markers, groups, busses, sources, selection, transport, undo label. |
|
|
| `ui.inspect` | `{}` | UI state (`window`, panels, dialogs), window size, playing/position, audio device, and `edit_layout` (timeline rect and every track row's rect, for clicking). |
|
|
| `ui.set` | any `UiState` fields | e.g. `{"window": "Mix", "narrow_mix": true}`. |
|
|
| `ui.menu.list` | `{}` | Every catalog menu path and the command that implements it (or null). |
|
|
| `ui.menu.invoke` | `{path}` or `{id}` | Like clicking the menu item: may open its dialog (e.g. `"Track > New..."`). |
|
|
| `ui.click` | `{x, y, button?, count?, cmd?, shift?, alt?, ctrl?}` | Synthetic click (window points). |
|
|
| `ui.drag` | `{x, y, to_x, to_y, steps?}` | Synthetic drag with the primary button. |
|
|
| `ui.move` | `{x, y}` | Move the pointer. |
|
|
| `ui.key` | `{key, cmd?, shift?, alt?, ctrl?}` | Key press (`Space`, `Enter`, `A`, `F7`, `=`…). |
|
|
| `ui.text` | `{text}` | Type text into the focused field. |
|
|
| `ui.screenshot` | `{path?}` | PNG of the window (base64 in `png_base64` when no path). |
|
|
| `app.quit` | `{}` | Quit. |
|
|
|
|
## Positions and references
|
|
|
|
- Tracks: id (number) or name (`"Kick"`).
|
|
- Clips: id (from `session.inspect`).
|
|
- Positions: samples (`96000`), seconds (`{"seconds": 2.5}`), or a time string in any timebase,
|
|
optionally prefixed: `"bars_beats:5|1|000"`, `"min_secs:0:12.500"`, `"timecode:00:00:10:00"`,
|
|
`"samples:48000"`.
|
|
- Ranges: `start` + `end` (or `start` + `length`); omitted = the current edit selection.
|
|
|
|
## Examples
|
|
|
|
```sh
|
|
P=$(soundcraft --demo --control 0 | sed -n 's/SOUNDCRAFT_CONTROL_PORT=//p') # or read stderr
|
|
soundcraft-cli app --port $P mix.solo '{"track":"Bass"}'
|
|
soundcraft-cli app --port $P transport.play
|
|
soundcraft-cli app --port $P ui.screenshot '{"path":"/tmp/shot.png"}'
|
|
soundcraft-cli app --port $P ui.menu.invoke '{"path":"Track > New..."}'
|
|
```
|
|
|
|
Offscreen (no window): `cargo run -p soundcraft-ui-egui --example ui_shot -- out.png script.jsonl`,
|
|
where each script line is a request (`{"method": …, "params": …}`), `{"shot": "path.png"}` or
|
|
`{"steps": n}`.
|