mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-10 10:20:47 -06:00
* Plug-ins: API 2 (PluginHost, padding, layer_span) Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI stays at 1. - point3 / layer / text / hidden parameters, integer sliders, parameter groups. - EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side). - EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters at other times, frame rate, per-instance key, the layer's source range. - EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024. - plugin_error: a plug-in effect's last render error, for the UI. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Plug-ins: external effect helpers (crates/helper-host) Native plug-in formats run in separate helper programs, never in EffectCraft's process (docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3, native only, unsafe forbidden): it finds helper programs, starts each as a child process and registers its effects through plug-in API 2, so the UI and command surface need nothing new. - Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on Linux), never through the pipe. Render-time callbacks for the input at other times, other layers and parameters at other times, with nested renders through the same helper. - A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails with a message, the frame is left as it was, and the helper restarts for the next render. An effect that stops its helper 3 times is turned off (helper_blocklist.txt). - Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing to the settings folder alone can't run code. effect.helpers.scan / load are refused for confined agent sessions (confine.rs), like file.executeFile. - Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings folder, EFFECTCRAFT_HELPER_PATH. - Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render error of an effect shows as a warning sign after its name in Effect Controls. - serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render, padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper, with nothing to install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: address review (start failures, frame file mode, nextest listing) - A helper that can't start (not installed any more, restarting too often, no frame exchange file) is reported but no longer charged to the effect being rendered: it is Failure::Refused, not Failure::Broken, so it never counts toward turning that effect off. New check in tests/it: once the helper is out of restarts, renders of an effect fail without blocking it (fails without the fix: the effect was turned off after 3 such renders). - The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and /dev/shm is shared. Asserted in the frames unit test. - tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses: `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and `helper_protocol --exact` runs the checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS * Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step - Writes to a helper go through a writer thread with the request timeout: a helper that stops reading its stdin (hung with a large callback answer or render request pending) no longer blocks the render thread forever while holding the helper. - stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE / SIGPIPE on its next write); lines were unbounded. - Callback images that can't be written are answered with `null` instead of failing the render while the helper waits (which left the protocol out of step); a frame that can't be written before the request is sent is not charged to the effect. - Mid-render calls use the render's own process: no respawn or heartbeat ping while the helper is waiting for an answer. - Waiting for a busy helper is bounded (load timeout) instead of forever; status() no longer blocks on a running render. - Reap the child when start-up fails after spawning; cap the block list read. - Scripts start helpers only with Allow Scripts to Write Files and Access Network on (they run native code), as for File.execute(). - helper-host declares the workspace's no-panic crate lints; the desktop main() keeps its doc comment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again - WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the library's compatibility check use it, so it never installs an API 2 release that can't load. - with_running: a process that is already gone was stopped by what broke it (a nested render, charged there), so this render fails without charging its own effect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * frames: plug-in effects count as reading other frames (plug-in API 2) With main's narrowed frame invalidation (#2795), an edit inside a precomp under an effect that reads its layer at other times must reach the layer's whole span. Plug-in API 2 gives every plug-in effect the input at other times, so any effect outside ec. counts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
166 lines
8.2 KiB
Rust
166 lines
8.2 KiB
Rust
//! Effect plug-ins in a full session: a Rust plug-in registered through the trait and the
|
|
//! example WebAssembly plug-in (examples/plugins/posterize-bands) loaded with
|
|
//! `effect.plugins.load`, both applied by id like built-ins and rendered.
|
|
|
|
use std::sync::Arc;
|
|
|
|
use effectcraft_engine::Session;
|
|
use effectcraft_engine::effects::plugin::{EffectPlugin, PluginFrame, PluginManifest, PluginParam, PluginParamKind, PluginParams, register_plugin};
|
|
use effectcraft_engine::time::Tick;
|
|
use serde_json::json;
|
|
|
|
struct Swap {
|
|
m: PluginManifest,
|
|
}
|
|
|
|
impl EffectPlugin for Swap {
|
|
fn manifest(&self) -> &PluginManifest {
|
|
&self.m
|
|
}
|
|
fn render(&self, frame: &mut PluginFrame, params: &PluginParams, _time: f64) -> Result<(), String> {
|
|
if params.b("on") {
|
|
for p in frame.pixels.iter_mut() {
|
|
p.swap(0, 2);
|
|
}
|
|
}
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
fn setup() -> (Session, u64) {
|
|
let mut s = effectcraft_host::session();
|
|
s.execute("comp.new", json!({"name": "P", "width": 40, "height": 20, "duration": 1})).unwrap();
|
|
let l = s.execute("layer.newSolid", json!({"color": [1.0, 0.5, 0.0], "width": 40, "height": 20})).unwrap()["layer"].as_u64().unwrap();
|
|
(s, l)
|
|
}
|
|
|
|
fn px(s: &Session) -> [f32; 4] {
|
|
let cid = s.active_comp_id().unwrap();
|
|
s.render(cid, Tick::ZERO, Default::default()).get(20, 10)
|
|
}
|
|
|
|
#[test]
|
|
fn rust_plugins_apply_by_id() {
|
|
let m = PluginManifest {
|
|
api: 1,
|
|
id: "org.test.swap".into(),
|
|
name: "Swap Red Blue".into(),
|
|
category: "Channel".into(),
|
|
version: "1".into(),
|
|
author: String::new(),
|
|
description: String::new(),
|
|
params: vec![PluginParam { id: "on".into(), name: "On".into(), kind: PluginParamKind::Checkbox { default: true }, group: String::new() }],
|
|
};
|
|
register_plugin(Arc::new(Swap { m })).unwrap();
|
|
let (mut s, l) = setup();
|
|
s.execute("effect.apply", json!({"layer": l, "effect": "org.test.swap"})).unwrap();
|
|
let p = px(&s);
|
|
assert!(p[0] < 0.01 && (p[1] - 0.5).abs() < 0.01 && p[2] > 0.99, "{p:?}");
|
|
// Its parameters are ordinary properties: set (and animate) them like any effect's.
|
|
s.execute("prop.set", json!({"layer": l, "path": "effects/#1/on", "value": false})).unwrap();
|
|
let p = px(&s);
|
|
assert!(p[0] > 0.99 && p[2] < 0.01, "{p:?}");
|
|
let list = s.execute("effect.plugins.list", json!({})).unwrap();
|
|
assert!(list["plugins"].as_array().unwrap().iter().any(|p| p["id"] == "org.test.swap" && p["category"] == "Channel"));
|
|
assert_eq!(list["api"], 2);
|
|
assert!(s.execute("effect.list", json!({"filter": "swap red"})).unwrap().as_array().unwrap().len() == 1);
|
|
}
|
|
|
|
/// Build the example plug-in (skipped when the wasm32 target isn't installed).
|
|
fn build_example() -> Option<std::path::PathBuf> {
|
|
let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../../examples/plugins/posterize-bands");
|
|
let workspace = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
|
|
let assigned = std::env::var_os("CARGO_TARGET_DIR");
|
|
let target = assigned.as_ref().map(std::path::PathBuf::from).unwrap_or_else(|| workspace.join("target"));
|
|
let target = if target.is_absolute() { target } else { workspace.join(target) };
|
|
// The plug-in is part of this test's build. Inherit the assigned target and
|
|
// compiler guards rather than creating a second target at its root.
|
|
let mut command = std::process::Command::new(env!("CARGO"));
|
|
command.current_dir(&workspace).args(["build", "--release", "--target", "wasm32-unknown-unknown", "--manifest-path"]).arg(root.join("Cargo.toml"));
|
|
if assigned.is_none() {
|
|
command.env("CARGO_TARGET_DIR", &target);
|
|
}
|
|
let st = command.status().ok()?;
|
|
if !st.success() {
|
|
eprintln!("skipping: the example plug-in didn't build (rustup target add wasm32-unknown-unknown)");
|
|
return None;
|
|
}
|
|
Some(target.join("wasm32-unknown-unknown/release/posterize_bands.wasm"))
|
|
}
|
|
|
|
#[test]
|
|
fn example_wasm_plugin_loads_and_renders() {
|
|
let Some(wasm) = build_example() else { return };
|
|
let (mut s, l) = setup();
|
|
let r = s.execute("effect.plugins.load", json!({"path": wasm.to_string_lossy()})).unwrap();
|
|
assert_eq!(r["loaded"][0]["id"], "org.effectcraft.example.posterize-bands", "{r}");
|
|
s.execute("effect.apply", json!({"layer": l, "effect": "Posterize Bands"})).unwrap();
|
|
s.execute("prop.set", json!({"layer": l, "path": "effects/#1/mix", "value": 0})).unwrap();
|
|
s.execute("prop.set", json!({"layer": l, "path": "effects/#1/levels", "value": 2})).unwrap();
|
|
// Luminance of (1, .5, 0) is ~0.57: with 2 levels it bands to 1, scaling the colour by 1/y
|
|
// (8 bpc: red clips at 1).
|
|
let p = px(&s);
|
|
let y = 0.2126 + 0.7152 * 0.5;
|
|
assert!((p[1] - 0.5 / y as f32).abs() < 0.01 && p[0] > 0.99 && p[2] < 0.01, "{p:?}");
|
|
// Invert: band 0.
|
|
s.execute("prop.set", json!({"layer": l, "path": "effects/#1/invert", "value": true})).unwrap();
|
|
assert_eq!(px(&s)[..3], [0.0, 0.0, 0.0]);
|
|
let list = s.execute("effect.plugins.list", json!({})).unwrap();
|
|
assert!(list["plugins"].as_array().unwrap().iter().any(|p| p["source"].as_str().unwrap_or("").ends_with(".wasm")));
|
|
}
|
|
|
|
/// A parameterless plug-in module (WebAssembly text): `render` is `ec_render`'s body.
|
|
pub(super) fn wat_plugin(id: &str, api: i32, render: &str) -> String {
|
|
let manifest = format!(r#"{{"api":1,"id":"{id}","name":"Host {id}","category":"Test Plug-ins","version":"1","params":[]}}"#);
|
|
let len = manifest.len();
|
|
let escaped = manifest.replace('"', "\\\"");
|
|
format!(
|
|
r#"(module
|
|
(memory (export "memory") 2)
|
|
(data (i32.const 16) "{escaped}")
|
|
(func (export "ec_api_version") (result i32) i32.const {api})
|
|
(func (export "ec_manifest_ptr") (result i32) i32.const 16)
|
|
(func (export "ec_manifest_len") (result i32) i32.const {len})
|
|
(func (export "ec_alloc") (param i32) (result i32) i32.const 1024)
|
|
(func (export "ec_render") (param i32 i32 i32 i32 i32 f64 f64) (result i32) {render}))"#
|
|
)
|
|
}
|
|
|
|
#[test]
|
|
fn plugin_loading_reports_bad_modules_and_survives_failing_renders() {
|
|
let dir = std::env::temp_dir().join(format!("effectcraft-host-plugins-{}", std::process::id()));
|
|
std::fs::create_dir_all(&dir).unwrap();
|
|
let files = [
|
|
("a-good.wasm", wat_plugin("org.test.host.good", 1, "i32.const 0").into_bytes()),
|
|
("b-garbage.wasm", b"\0asm\x01\0\0\0garbage".to_vec()),
|
|
("c-v2.wasm", wat_plugin("org.test.host.v2", 2, "i32.const 0").into_bytes()),
|
|
("d-empty.wasm", vec![]),
|
|
("e-trap.wasm", wat_plugin("org.test.host.trap", 1, "unreachable").into_bytes()),
|
|
("notes.txt", b"not a plug-in".to_vec()),
|
|
];
|
|
for (name, bytes) in &files {
|
|
std::fs::write(dir.join(name), bytes).unwrap();
|
|
}
|
|
let (mut s, l) = setup();
|
|
let r = s.execute("effect.plugins.load", json!({"folder": dir.to_string_lossy()})).unwrap();
|
|
let ids: Vec<&str> = r["loaded"].as_array().unwrap().iter().filter_map(|v| v["id"].as_str()).collect();
|
|
assert_eq!(ids, ["org.test.host.good", "org.test.host.trap"], "{r}");
|
|
let errs: Vec<String> = r["errors"].as_array().unwrap().iter().map(|e| e["path"].as_str().unwrap().to_string()).collect();
|
|
assert_eq!(errs.len(), 3, "{r}");
|
|
assert!(errs.iter().all(|p| !p.ends_with(".txt")));
|
|
assert!(r["errors"].to_string().contains("API 2"), "{r}");
|
|
// A single file: errors come back as command errors; loading twice is refused.
|
|
let good = dir.join("a-good.wasm").to_string_lossy().to_string();
|
|
assert!(s.execute("effect.plugins.load", json!({"path": good})).unwrap_err().to_string().contains("already registered"));
|
|
assert!(s.execute("effect.plugins.load", json!({"path": dir.join("missing.wasm").to_string_lossy()})).is_err());
|
|
assert!(s.execute("effect.plugins.load", json!({"folder": dir.join("missing").to_string_lossy()})).is_err());
|
|
assert!(s.execute("effect.plugins.load", json!({})).is_err());
|
|
// A plug-in that traps on every frame leaves the layer as it was.
|
|
let before = px(&s);
|
|
s.execute("effect.apply", json!({"layer": l, "effect": "org.test.host.trap"})).unwrap();
|
|
assert_eq!(px(&s), before);
|
|
s.execute("effect.apply", json!({"layer": l, "effect": "org.test.host.good"})).unwrap();
|
|
assert_eq!(px(&s), before);
|
|
let _ = std::fs::remove_dir_all(&dir);
|
|
}
|