mirror of
https://github.com/storytold/filmcraft.git
synced 2026-10-25 10:37:34 -06:00
74 lines
5.1 KiB
Markdown
74 lines
5.1 KiB
Markdown
# Control protocol & MCP
|
||
|
||
## Desktop control channel
|
||
`filmcraft --control 9876` (or `FILMCRAFT_CONTROL_PORT`) listens on `127.0.0.1:<port>` (loopback only).
|
||
One JSON request per line → one JSON reply per line. The app opts out of macOS App Nap
|
||
(`apps/filmcraft/src/app_nap.rs`): a hidden window would otherwise drop the whole process to
|
||
background priority, and on a busy machine it would stop answering.
|
||
|
||
```json
|
||
{"id": 1, "method": "engine.execute", "params": {"command": "sequence.addEdit", "params": {"seconds": 3}}}
|
||
{"id": 1, "ok": true, "result": {"cuts": 2}}
|
||
```
|
||
|
||
Methods (handlers in `crates/ui-egui/src/control.rs`):
|
||
|
||
| Method | Params | |
|
||
|---|---|---|
|
||
| `engine.execute` / `ui.menu.invoke` | `{command, params}` | any engine or UI command id |
|
||
| `engine.commands` / `ui.menu.list` | – | command registry / menu tree |
|
||
| `ui.inspect` | – | UI state (tool, workspace, dock, timeline view, playback, fps…) |
|
||
| `ui.elements` | `{prefix?}` | every on-screen interactive element: id, label, rect |
|
||
| `ui.set` | `{tool, workspace, mode, theme, focused, playbackRes, timeline:{pps,scroll,fit}}` | |
|
||
| `ui.panel.show` / `ui.panel.close` | `{panel}` | |
|
||
| `ui.click` / `ui.move` | `{id}` or `{x,y}`, `button`, `count`, `modifiers` | synthetic pointer input |
|
||
| `ui.drag` | `{from, to, steps, modifiers}` | press–move–release |
|
||
| `ui.scroll` | `{id|x,y, dx, dy, modifiers}` | wheel / trackpad |
|
||
| `ui.key` / `ui.type` | `{key}` (`Cmd+K`, `Space`…) / `{text}` | keyboard |
|
||
| `ui.timeline.hit` / `ui.timeline.locate` | `{x,y}` / `{clip, edge?}` | timeline hit-testing |
|
||
| `ui.playback` | `{action: play|stop|toggle, speed?}` | |
|
||
| `ui.screenshot` | `{path?, panel?}` | PNG of the window or one panel; fails after 10 s when no frame is presented (window hidden, display asleep) |
|
||
| `ui.resize`, `ui.focus`, `app.quit` | | |
|
||
|
||
Element ids are stable, e.g. `timeline.clip.<id>`, `timeline.track.V1.lock`, `tools.Razor`,
|
||
`project.item.<id>`, `effects.item.gaussian_blur`, `panel.tab.Timeline`, `program.transport.playback.toggle`.
|
||
|
||
Audio mixing: `mixer.*` commands (strips `"A1"`, `"S1"`, `"Mix"` or ids; lanes `volume`, `pan`,
|
||
`mute`, `send.<i>.level`, `fx.<slot>.<param>`): `mixer.inspect`, `mixer.setStrip` (volume, pan, mute,
|
||
solo, record arm, solo safe, mode Off/Read/Latch/Touch/Write, output, input map, channels),
|
||
`mixer.setValue`, `mixer.touch` / `mixer.release` (a fader gesture; recorded during an automation pass),
|
||
`mixer.recordStart` / `mixer.recordStop` (playback runs them), `mixer.addSubmix`, `mixer.deleteSubmix`,
|
||
`mixer.addInsert` / `removeInsert` / `setInsert`, `mixer.addSend` / `setSend` / `removeSend`,
|
||
`mixer.setKeyframe` / `deleteKeyframe` / `moveKeyframe` / `clearLane`, `mixer.writeAutomation`;
|
||
`clipMixer.set`; `clip.audioGain {mode: set|adjust|normalizeMax|normalizeAll, db}`, `clip.audioPeak`;
|
||
`effects.setDefaultTransition`. UI ids: `mixer.<A1|S1|Mix>.<fader|value|pan|panValue|mode|mute|solo|
|
||
record|soloSafe|output|input|fx.<n>|send.<n>|meter|name>` (popup entries below them, e.g.
|
||
`mixer.A1.mode.Touch`, `mixer.A1.fx.0.studio_reverb`), `mixer.showEffects`, `mixer.transport.<cmd>`,
|
||
`clipMixer.A1.<fader|pan|mute|solo|keyframe|value>`, `timeline.track.A1.keyframes[.<lane>|.clip]`,
|
||
`timeline.track.A1.lane[.kf.<n>]`, `audioGain.<set|adjust|normalizeMax|normalizeAll|ok|cancel|peak>`.
|
||
|
||
Project files, auto-save, crash recovery and preferences commands (`file.recover`, `prefs.set`, …) and their
|
||
automation ids are listed in [project-files.md](project-files.md).
|
||
|
||
**Trim mode** (`crates/engine/src/trim.rs`): `trim.selectEditPoint` / `trim.selectNearest` enter trim
|
||
mode (the Program monitor becomes the Trim Monitor, ids `trimMonitor.*`); `trim.monitor` returns what
|
||
it shows. Dynamic trimming takes an explicit clock (seconds) so it is deterministic: `trim.shuttle
|
||
{direction, slow?, clock}` (J/L), `trim.tick {clock}`, `trim.shuttleStop {clock?}` (K, one undo step),
|
||
`trim.cancelDynamic` (Esc), `trim.playAround {clock, loop?}` (Space / Shift+K).
|
||
|
||
**Keyboard shortcuts** (`crates/engine/src/shortcuts.rs`): `shortcuts.list {query?, panel?}`,
|
||
`shortcuts.get`, `shortcuts.set {command, keys, panel?, add?, keepConflicts?}`, `shortcuts.clear`,
|
||
`shortcuts.undo` / `shortcuts.redo`, `shortcuts.conflicts {platform?}`, `shortcuts.forKey {key}`,
|
||
`shortcuts.resolve {keys, panel?}`, `shortcuts.presets`, `shortcuts.loadPreset` / `savePreset` /
|
||
`deletePreset {name}`, `shortcuts.export` / `import {path}`, `shortcuts.audit`. Keys use `Cmd` (⌘ /
|
||
Ctrl), `Ctrl` (macOS ⌃), `Alt`, `Shift`. The dialog (Edit ▸ Keyboard Shortcuts…, ⌥⌘K) uses ids
|
||
`shortcuts.*` (`shortcuts.key.K`, `shortcuts.cell.<command>`, `shortcuts.ok`, …).
|
||
|
||
## MCP
|
||
`filmcraft-cli mcp` serves MCP on stdio: headless (in-process session; `--demo` / `--project p.fcproj`)
|
||
or `--bridge 127.0.0.1:9876` to drive the running app. Tools: `command_list`, `command_run`,
|
||
`project_inspect`, `sequence_inspect`, `media_import`, `render_frame`, `ui_inspect`, `ui_elements`,
|
||
`ui_click`, `ui_drag`, `ui_key`, `ui_type`, `ui_screenshot`, `ui_control`. `.mcp.json` registers both.
|
||
|
||
Time is in ticks (254 016 000 000 per second); commands also accept `seconds`, `frame` or `timecode`.
|