Files
effectcraft/crates/engine/src/system/library.rs
T
611a282bea Plug-ins: external effect helpers (safe, out-of-process native effects) + plug-in API 2 (#889)
* Plug-ins: API 2 (PluginHost, padding, layer_span)

Split out of the OpenFX work so it can be reviewed on its own. Additive; the WebAssembly ABI
stays at 1.

- point3 / layer / text / hidden parameters, integer sliders, parameter groups.
- EffectPlugin::padding (transparent pixels around the buffer, at most 4096 per side).
- EffectPlugin::render_v2 with a PluginHost: the input at other times, other layers, parameters
  at other times, frame rate, per-instance key, the layer's source range.
- EffectEnv::layer_span from the renderer; MAX_PLUGINS 1024.
- plugin_error: a plug-in effect's last render error, for the UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Plug-ins: external effect helpers (crates/helper-host)

Native plug-in formats run in separate helper programs, never in EffectCraft's process
(docs/adr/0001-external-effect-helpers.md). New safe-Rust crate effectcraft-helper-host (L3,
native only, unsafe forbidden): it finds helper programs, starts each as a child process and
registers its effects through plug-in API 2, so the UI and command surface need nothing new.

- Protocol v1 (docs/external-helpers.md): JSON lines over stdin / stdout (helper replies are
  prefixed so plug-in chatter is ignored); pixels go through a frame exchange file (/dev/shm on
  Linux), never through the pipe. Render-time callbacks for the input at other times, other layers
  and parameters at other times, with nested renders through the same helper.
- A helper that exits, hangs past its timeout or breaks the protocol is stopped. The render fails
  with a message, the frame is left as it was, and the helper restarts for the next render. An
  effect that stops its helper 3 times is turned off (helper_blocklist.txt).
- Opt-in: nothing starts helpers unless the user asks (Effect > Scan for Effect Helpers), or opts
  in to starting them at launch with EFFECTCRAFT_HELPERS=1, an environment variable, so writing
  to the settings folder alone can't run code. effect.helpers.scan / load are refused for
  confined agent sessions (confine.rs), like file.executeFile.
- Discovery: helpers/ next to the app (Contents/Helpers on macOS), helpers/ in the settings
  folder, EFFECTCRAFT_HELPER_PATH.
- Commands effect.helpers.list / scan / load; Effect menu entries (translated); the last render
  error of an effect shows as a warning sign after its name in Effect Controls.
- serve(): the helper side for Rust helpers. tests/it/main.rs runs the whole protocol (render,
  padding, callbacks, nesting, threads, crash, restart, block list, hang) against a fake helper,
  with nothing to install.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Effect helpers: address review (start failures, frame file mode, nextest listing)

- A helper that can't start (not installed any more, restarting too often, no frame exchange
  file) is reported but no longer charged to the effect being rendered: it is Failure::Refused,
  not Failure::Broken, so it never counts toward turning that effect off. New check in
  tests/it: once the helper is out of restarts, renders of an effect fail without blocking it
  (fails without the fix: the effect was turned off after 3 such renders).
- The frame exchange file is created owner-only (0o600) on Unix: it holds the user's frames and
  /dev/shm is shared. Asserted in the frames unit test.
- tests/it (harness = false) speaks the libtest listing protocol cargo-nextest uses:
  `--list --format terse` prints `helper_protocol: test` (nothing with `--ignored`), and
  `helper_protocol --exact` runs the checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KxacwTpUwqyfpcLgKW8yjS

* Effect helpers: bound writes to a helper, drain stderr, keep callbacks in step

- Writes to a helper go through a writer thread with the request timeout: a
  helper that stops reading its stdin (hung with a large callback answer or
  render request pending) no longer blocks the render thread forever while
  holding the helper.
- stderr is drained as bytes (lossy, in 64 KiB pieces) to the end: output that
  isn't UTF-8 stopped the reader, closing the pipe under the helper (EPIPE /
  SIGPIPE on its next write); lines were unbounded.
- Callback images that can't be written are answered with `null` instead of
  failing the render while the helper waits (which left the protocol out of
  step); a frame that can't be written before the request is sent is not
  charged to the effect.
- Mid-render calls use the render's own process: no respawn or heartbeat ping
  while the helper is waiting for an answer.
- Waiting for a busy helper is bounded (load timeout) instead of forever;
  status() no longer blocks on a running render.
- Reap the child when start-up fails after spawning; cap the block list read.
- Scripts start helpers only with Allow Scripts to Write Files and Access
  Network on (they run native code), as for File.execute().
- helper-host declares the workspace's no-panic crate lints; the desktop
  main() keeps its doc comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Effect helpers: rustfmt the fake provider, build the test session in its initializer (clippy)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Plug-in Library: offer WebAssembly API 1 releases only; a helper already stopped mid-render isn't charged again

- WasmPlugin loads only modules at the WebAssembly ABI version, which stays 1 while
  PLUGIN_API_VERSION is 2: share it as effects::plugin::WASM_API_VERSION and have the
  library's compatibility check use it, so it never installs an API 2 release that
  can't load.
- with_running: a process that is already gone was stopped by what broke it (a nested
  render, charged there), so this render fails without charging its own effect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* frames: plug-in effects count as reading other frames (plug-in API 2)

With main's narrowed frame invalidation (#2795), an edit inside a precomp under an
effect that reads its layer at other times must reach the layer's whole span.
Plug-in API 2 gives every plug-in effect the input at other times, so any effect
outside ec. counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-05 11:11:57 +01:00

1204 lines
50 KiB
Rust

//! The Plug-in Library (Settings ▸ Plug-in Library; `library.*` commands): WebAssembly effect
//! plug-ins published by the community, listed in one reviewed catalog (an `index.json` in a Git
//! repository, see `docs/plugins.md#plug-in-library`), installed, updated and removed without
//! bundling them into the app.
//!
//! * The catalog names, per package, its releases: version, plug-in API, oldest EffectCraft it
//! needs, the `.wasm` file's URL (a GitHub release asset) and its SHA-256. Nothing is compiled
//! here: the published file is downloaded, checked against the catalog's SHA-256 and loaded by
//! the host's [`PluginLoader`](crate::PluginLoader), so it runs in the same sandbox as any
//! plug-in (no imports, fuel and memory limits).
//! * Installed packages live in `<Plug-ins>/Library/<id>/<version>/`, listed in
//! `<Plug-ins>/Library/installed.json` with the SHA-256 they were installed with; the app loads
//! them at start-up ([`load_installed`], `library.loadInstalled`). The catalog last fetched is
//! kept as `catalog.json` there, so the list works offline.
//! * Effects can't be unregistered: removing or updating a package that is loaded takes effect
//! at the next start, and the list says so (`restartNeeded`).
//! * Downloads use the system's `curl`, as trained models do ([`crate::models`]), so no
//! networking code is linked into EffectCraft. Only `https://` URLs on GitHub's hosts are
//! fetched, and a download whose redirects end anywhere else is refused; a local catalog (`file://`, for plug-in authors testing their entry) may also name
//! local files.
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, PoisonError};
use effectcraft_segment::sha256;
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
use crate::commands::{CommandSpec, always, b_p, bad, str_p};
use crate::{PluginLoader, PluginPreviewer, Result, Session, cmd, query};
/// The community catalog.
pub const CATALOG_URL: &str = "https://raw.githubusercontent.com/storytold/effectcraft-plugins/main/index.json";
/// The catalog format this build reads.
pub const CATALOG_SCHEMA: u32 = 1;
/// Hosts plug-ins and catalogs may be downloaded from (release assets redirect to the last two).
const HOSTS: &[&str] = &["github.com", "raw.githubusercontent.com", "objects.githubusercontent.com", "release-assets.githubusercontent.com"];
const MAX_CATALOG: u64 = 4 << 20;
const MAX_MODULE: u64 = 64 << 20;
const MAX_PACKAGES: usize = 10_000;
const LIBRARY_DIR: &str = "Library";
const INSTALLED_FILE: &str = "installed.json";
const CACHE_FILE: &str = "catalog.json";
/// The catalog (`index.json`).
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct Catalog {
pub schema: u32,
#[serde(default)]
pub packages: Vec<Package>,
}
/// One plug-in package: one `.wasm` module per release.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct Package {
pub id: String,
pub name: String,
#[serde(default)]
pub author: String,
#[serde(default)]
pub description: String,
#[serde(default)]
pub license: String,
/// The source repository.
#[serde(default)]
pub repo: String,
#[serde(default)]
pub tags: Vec<String>,
#[serde(default)]
pub versions: Vec<Release>,
/// The author's demo media.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub preview: Option<PackagePreview>,
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct PackagePreview {
/// An `https://` link to a short demo video (YouTube, Vimeo, GitHub…), opened in the browser.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub video: Option<String>,
}
/// A package's effect rendered on the sample frame (`library.preview`), as straight RGBA8.
#[derive(Debug)]
pub struct Preview {
pub id: String,
pub version: String,
pub width: u32,
pub height: u32,
pub fps: f64,
/// The sample frame.
pub before: Vec<u8>,
/// The effect at default settings, one frame per `1 / fps` from time 0.
pub frames: Vec<Vec<u8>>,
/// The module's manifest (`{id, name, category, description, params…}`).
pub manifest: Value,
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct Release {
/// Dotted numbers (`1.2.0`).
pub version: String,
/// The plug-in API it implements.
pub api: u32,
/// The oldest EffectCraft version it runs in.
#[serde(default, rename = "minApp", skip_serializing_if = "Option::is_none")]
pub min_app: Option<String>,
pub url: String,
/// Lower-case hex SHA-256 of the file at `url`.
pub sha256: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub size: Option<u64>,
/// The effect ids the module registers (`library.missing` finds packages by them).
#[serde(default)]
pub effects: Vec<String>,
}
/// `installed.json`: package id → what is installed.
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct Installed {
#[serde(default)]
pub packages: BTreeMap<String, InstalledPackage>,
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
pub struct InstalledPackage {
pub version: String,
/// The module, relative to the library folder (`<id>/<version>/<id>.wasm`).
pub file: String,
pub sha256: String,
#[serde(default)]
pub effects: Vec<String>,
}
/// `catalog.json`: the catalog last fetched, and from where.
#[derive(Clone, Debug, Default, Serialize, Deserialize)]
struct Cache {
url: String,
fetched: String,
catalog: Catalog,
}
/// What the library's worker reports, and what this session loaded.
#[derive(Clone, Debug, Default)]
pub struct Status {
/// What is happening now ("Downloading Glow Pack 1.2.0…").
pub busy: Option<String>,
/// The last failure.
pub error: Option<String>,
/// A finished job's message and whether it failed, for a toast ([`Session::poll_library`]).
pub done: Option<(String, bool)>,
/// The catalog in use and where it came from.
pub catalog: Option<(String, Arc<Catalog>)>,
/// When it was fetched (RFC 3339).
pub fetched: Option<String>,
/// Packages whose module this session loaded: id → version.
pub loaded: BTreeMap<String, String>,
/// Packages removed or updated after they loaded: the change applies at the next start.
pub restart: BTreeSet<String>,
/// The last package previewed.
pub preview: Option<Arc<Preview>>,
}
/// The session's Plug-in Library state (shared with its worker thread).
#[derive(Default)]
pub struct Library {
status: Arc<Mutex<Status>>,
}
impl Library {
/// A copy of the current status.
pub fn status(&self) -> Status {
self.status.lock().unwrap_or_else(PoisonError::into_inner).clone()
}
}
/// Change the shared status (a poisoned lock is still used, not a panic).
fn set(status: &Arc<Mutex<Status>>, f: impl FnOnce(&mut Status)) {
f(&mut status.lock().unwrap_or_else(PoisonError::into_inner));
}
impl Session {
/// The effect plug-ins folder: the host's, else `Plug-ins` next to the settings.
pub fn plugins_dir(&self) -> Option<PathBuf> {
self.plugins_dir.clone().or_else(|| Some(self.config.as_ref()?.dir()?.join("Plug-ins")))
}
/// Where Plug-in Library packages are installed.
pub fn library_dir(&self) -> Option<PathBuf> {
self.plugins_dir().map(|d| d.join(LIBRARY_DIR))
}
/// Finish what the library's worker did: a toast.
pub fn poll_library(&mut self) {
let done = self.library.status.lock().ok().and_then(|mut s| s.done.take());
if let Some((message, error)) = done {
self.events.push(crate::Event::Toast { message, error });
}
}
}
// ---------------------------------------------------------------- catalog
/// Dotted numbers as a comparable list (`None`: not a version).
fn version_key(v: &str) -> Option<Vec<u64>> {
if v.is_empty() || v.len() > 32 {
return None;
}
let parts: Option<Vec<u64>> = v.split('.').map(|p| if p.is_empty() || !p.bytes().all(|b| b.is_ascii_digit()) { None } else { p.parse().ok() }).collect();
let mut parts = parts?;
while parts.len() > 1 && parts.last() == Some(&0) {
parts.pop();
}
Some(parts)
}
/// `a` is newer than `b`.
fn newer(a: &str, b: &str) -> bool {
match (version_key(a), version_key(b)) {
(Some(a), Some(b)) => a > b,
_ => false,
}
}
/// A package id that is also a safe folder name: letters, digits, `.`, `-`, `_`; no leading or
/// trailing dot (Windows drops a trailing one, so `a.` would share `a`'s folder).
pub fn valid_id(id: &str) -> bool {
(1..=100).contains(&id.len())
&& !id.starts_with('.')
&& !id.ends_with('.')
&& !id.contains("..")
&& id.bytes().all(|b| b.is_ascii_alphanumeric() || b"._-".contains(&b))
}
/// 64 hex digits.
fn valid_sha(s: &str) -> bool {
s.len() == 64 && s.bytes().all(|b| b.is_ascii_hexdigit())
}
/// Why a release can't be used, if it can't.
fn release_problem(r: &Release) -> Option<String> {
if version_key(&r.version).is_none() {
return Some(format!("bad version `{}`", r.version));
}
if r.min_app.as_deref().is_some_and(|m| version_key(m).is_none()) {
return Some(format!("{}: bad minApp", r.version));
}
if !valid_sha(&r.sha256) {
return Some(format!("{}: bad sha256", r.version));
}
if r.url.is_empty() {
return Some(format!("{}: no url", r.version));
}
None
}
/// Read a catalog: packages and releases that are malformed are left out (and reported), so one
/// bad entry doesn't hide the others.
pub fn parse_catalog(bytes: &[u8]) -> std::result::Result<(Catalog, Vec<String>), String> {
if bytes.len() as u64 > MAX_CATALOG {
return Err(format!("the catalog is larger than {} MB", MAX_CATALOG >> 20));
}
let raw: Catalog = serde_json::from_slice(bytes).map_err(|e| format!("the catalog isn't valid: {e}"))?;
if raw.schema != CATALOG_SCHEMA {
return Err(format!("the catalog uses format {} and this EffectCraft reads format {CATALOG_SCHEMA}: update EffectCraft", raw.schema));
}
let mut warnings = vec![];
let mut seen = BTreeSet::new();
let mut packages = vec![];
for mut p in raw.packages.into_iter().take(MAX_PACKAGES) {
if !valid_id(&p.id) || p.name.trim().is_empty() {
warnings.push(format!("left out a package with a bad id or no name (`{}`)", p.id));
continue;
}
// Windows and macOS folders ignore case: `Org.X` and `org.x` would share one.
if !seen.insert(p.id.to_ascii_lowercase()) {
warnings.push(format!("left out a second package `{}`", p.id));
continue;
}
p.versions.retain(|r| match release_problem(r) {
Some(why) => {
warnings.push(format!("{}: {why}", p.id));
false
}
None => true,
});
for r in &mut p.versions {
r.sha256.make_ascii_lowercase();
}
if let Some(video) = p.preview.as_ref().and_then(|v| v.video.as_deref())
&& !(video.starts_with("https://") && video.len() <= 500)
{
warnings.push(format!("{}: left out a preview video that isn't an https:// link", p.id));
p.preview = None;
}
packages.push(p);
}
Ok((Catalog { schema: CATALOG_SCHEMA, packages }, warnings))
}
/// This build loads the release's WebAssembly plug-in API ([`WASM_API_VERSION`](crate::effects::plugin::WASM_API_VERSION):
/// the library lists `.wasm` modules only) and is at least its `minApp`.
fn runs_here(r: &Release) -> bool {
// Without a pre-release suffix (`0.18.0-dev` is 0.18.0).
let app = env!("CARGO_PKG_VERSION").split(['-', '+']).next().unwrap_or_default();
r.api == crate::effects::plugin::WASM_API_VERSION && r.min_app.as_deref().is_none_or(|m| !newer(m, app))
}
/// The newest release this build can run (plug-in API and minimum app version).
pub fn latest_compatible(p: &Package) -> Option<&Release> {
p.versions.iter().filter(|r| runs_here(r)).fold(None, |best: Option<&Release>, r| match best {
Some(b) if !newer(&r.version, &b.version) => Some(b),
_ => Some(r),
})
}
// ---------------------------------------------------------------- fetching
/// Where a URL may be read from.
#[derive(Clone, Debug, PartialEq)]
enum Source {
Https(String),
File(PathBuf),
}
/// `file:///C:/x` or `file:///home/x` as a path (percent-decoded; no remote hosts).
fn file_url_path(url: &str) -> Option<PathBuf> {
let rest = url.strip_prefix("file://")?;
let rest = rest.strip_prefix("localhost").unwrap_or(rest);
if !rest.starts_with('/') {
return None;
}
let b = rest.as_bytes();
// `/C:/…` on Windows.
let rest = if b.len() > 2 && b.get(2) == Some(&b':') { rest.get(1..)? } else { rest };
let mut out = Vec::with_capacity(rest.len());
let mut bytes = rest.bytes();
while let Some(c) = bytes.next() {
if c == b'%' {
let hex = [bytes.next()?, bytes.next()?];
out.push(u8::from_str_radix(std::str::from_utf8(&hex).ok()?, 16).ok()?);
} else {
out.push(c);
}
}
String::from_utf8(out).ok().map(PathBuf::from)
}
/// Check `url` against the rules: `https://` on GitHub's hosts; `file://` when the catalog
/// (`catalog`, or `url` itself when it is the catalog) is local.
fn source(url: &str, catalog: Option<&str>) -> std::result::Result<Source, String> {
if let Some(rest) = url.strip_prefix("https://") {
let host = rest.split(['/', '?', '#']).next().unwrap_or_default().to_ascii_lowercase();
if host.contains('@') || host.contains(':') || !HOSTS.contains(&host.as_str()) {
return Err(format!("`{url}` isn't on GitHub: the Plug-in Library only downloads from {}", HOSTS.join(", ")));
}
return Ok(Source::Https(url.to_string()));
}
let local_catalog = catalog.is_none_or(|c| c.starts_with("file://"));
if url.starts_with("file://") && local_catalog {
return file_url_path(url).map(Source::File).ok_or_else(|| format!("bad file URL `{url}`"));
}
Err(format!("`{url}` isn't an https:// URL"))
}
/// Read `src` (at most `max` bytes), downloading into `tmp_dir`.
fn fetch(src: &Source, max: u64, tmp_dir: &Path) -> std::result::Result<Vec<u8>, String> {
match src {
Source::File(p) => {
let len = std::fs::metadata(p).map_err(|e| format!("cannot read {}: {e}", p.display()))?.len();
if len > max {
return Err(format!("{} is larger than {} MB", p.display(), max >> 20));
}
std::fs::read(p).map_err(|e| format!("cannot read {}: {e}", p.display()))
}
Source::Https(url) => {
std::fs::create_dir_all(tmp_dir).map_err(|e| format!("cannot create {}: {e}", tmp_dir.display()))?;
// Unique per download: sessions in one process may download at the same time.
static NEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
let n = NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let tmp = tmp_dir.join(format!(".download-{}-{n}.part", std::process::id()));
// `-q` first: no ~/.curlrc; `--globoff`: `[]` and `{}` in a URL are literal.
let out = std::process::Command::new("curl")
.args([
"-q",
"--globoff",
"-fsSL",
"--proto",
"=https",
"--proto-redir",
"=https",
"--max-redirs",
"5",
"-w",
"%{url_effective}",
"--retry",
"2",
"--connect-timeout",
"20",
"--max-time",
"900",
])
.args(["--max-filesize", &max.to_string(), "-o"])
.arg(&tmp)
.arg(url)
.output()
.map_err(|e| format!("downloading needs curl ({e})"))?;
// The size is checked before reading: `--max-filesize` can't stop a stream of unknown length.
let too_big = std::fs::metadata(&tmp).is_ok_and(|m| m.len() > max);
let bytes = if too_big {
Err(format!("the download is larger than {} MB", max >> 20))
} else if out.status.success() {
// Redirects are followed, so check where the download ended up.
let end = String::from_utf8_lossy(&out.stdout).trim().to_string();
match source(&end, None) {
Ok(Source::Https(_)) => std::fs::read(&tmp).map_err(|e| format!("cannot read the download: {e}")),
_ => Err(format!("`{url}` redirected to `{end}`, which isn't on GitHub")),
}
} else {
Err(format!("download failed: {}", String::from_utf8_lossy(&out.stderr).trim()))
};
let _ = std::fs::remove_file(&tmp);
let bytes = bytes?;
if bytes.len() as u64 > max {
return Err(format!("the download is larger than {} MB", max >> 20));
}
Ok(bytes)
}
}
}
// ---------------------------------------------------------------- installed packages
/// `installed.json`; missing or unreadable is an empty library.
fn read_installed(dir: &Path) -> Installed {
std::fs::read(dir.join(INSTALLED_FILE)).ok().and_then(|b| serde_json::from_slice(&b).ok()).unwrap_or_default()
}
/// Write `installed.json`.
fn write_installed(dir: &Path, inst: &Installed) -> std::result::Result<(), String> {
use crate::config::FileOps;
let text = serde_json::to_vec_pretty(inst).map_err(|e| e.to_string())?;
crate::config::StdFiles.write(&dir.join(INSTALLED_FILE), &text).map_err(|e| format!("cannot write {INSTALLED_FILE}: {e}"))
}
/// An installed module's path, if `file` stays inside the library folder.
fn module_path(dir: &Path, file: &str) -> Option<PathBuf> {
let parts: Vec<&str> = file.split('/').collect();
let safe = parts.len() == 3 && parts.iter().all(|p| !p.is_empty() && *p != "." && *p != ".." && !p.contains(['\\', ':']));
safe.then(|| parts.iter().fold(dir.to_path_buf(), |d, p| d.join(p)))
}
/// Load every installed package's module (at start-up): each is checked against the SHA-256 it
/// was installed with. Returns `{loaded: [{package, version, effect…}], errors: [{package, error}]}`.
pub fn load_installed(s: &mut Session) -> Result<Value> {
let c = "library.loadInstalled";
let loader = s.plugin_loader.ok_or_else(|| bad(c, "WebAssembly effect plug-ins are not available in this build"))?;
let Some(dir) = s.library_dir() else { return Ok(json!({"loaded": [], "errors": []})) };
let inst = read_installed(&dir);
let (mut loaded, mut errors) = (vec![], vec![]);
for (id, p) in &inst.packages {
if s.library.status().loaded.contains_key(id) {
continue;
}
let r = module_path(&dir, &p.file)
.ok_or_else(|| format!("bad module path `{}`", p.file))
.and_then(|path| std::fs::read(&path).map_err(|e| format!("cannot read {}: {e}", path.display())).map(|b| (path, b)))
.and_then(|(path, bytes)| {
if sha256::hex(&bytes) != p.sha256 {
return Err("the installed file changed since it was installed (SHA-256 differs): install it again".into());
}
loader(&bytes, &path.to_string_lossy())
});
match r {
Ok(mut v) => {
set(&s.library.status, |st| {
st.loaded.insert(id.clone(), p.version.clone());
});
v["package"] = json!(id);
v["packageVersion"] = json!(p.version);
loaded.push(v);
}
Err(e) => {
log::warn!("plug-in library: {id} not loaded: {e}");
errors.push(json!({"package": id, "error": e}));
}
}
}
Ok(json!({"loaded": loaded, "errors": errors}))
}
/// Download `rel` of `pkg` (temporary files in `dir`) and check it against the catalog.
fn download(dir: &Path, catalog_url: &str, pkg: &Package, rel: &Release) -> std::result::Result<Vec<u8>, String> {
let src = source(&rel.url, Some(catalog_url))?;
let bytes = fetch(&src, MAX_MODULE, dir)?;
if rel.size.is_some_and(|n| n != bytes.len() as u64) || sha256::hex(&bytes) != rel.sha256 {
return Err(format!("the downloaded file isn't the one the catalog lists for {} {} (size or SHA-256 differs)", pkg.name, rel.version));
}
Ok(bytes)
}
/// Download, check, write and load `rel` of `pkg` (the worker's job).
fn install_job(
dir: &Path,
catalog_url: &str,
pkg: &Package,
rel: &Release,
loader: PluginLoader,
previewer: Option<PluginPreviewer>,
status: &Arc<Mutex<Status>>,
) -> std::result::Result<String, String> {
let bytes = download(dir, catalog_url, pkg, rel)?;
let file = format!("{}/{}/{}.wasm", pkg.id, rel.version, pkg.id);
let path = module_path(dir, &file).ok_or_else(|| format!("bad package id `{}`", pkg.id))?;
let previous = status.lock().ok().and_then(|s| s.loaded.get(&pkg.id).cloned());
let restart = match previous {
// Its effects are registered already: the new version loads at the next start. It is
// still checked now (without registering it), so the version it replaces isn't deleted
// for one that won't load.
Some(v) => {
if let Some(check) = previewer {
check(&bytes, 1, 1, &[[0.0; 4]], &[]).map_err(|e| format!("{} {} didn't load: {e}", pkg.name, rel.version))?;
}
v != rel.version
}
// Loaded before anything is written, so a module that fails leaves the disk as it was.
None => {
loader(&bytes, &path.to_string_lossy()).map_err(|e| format!("{} {} didn't load: {e}", pkg.name, rel.version))?;
set(status, |s| {
s.loaded.insert(pkg.id.clone(), rel.version.clone());
});
false
}
};
{
use crate::config::FileOps;
crate::config::StdFiles.write(&path, &bytes).map_err(|e| format!("cannot write {}: {e}", path.display()))?;
}
let mut inst = read_installed(dir);
let old =
inst.packages.insert(pkg.id.clone(), InstalledPackage { version: rel.version.clone(), file, sha256: rel.sha256.clone(), effects: rel.effects.clone() });
write_installed(dir, &inst)?;
// `installed.json` may be hand-edited: only a real version names a folder to delete.
if let Some(old) = old.filter(|o| o.version != rel.version && version_key(&o.version).is_some()) {
let _ = std::fs::remove_dir_all(dir.join(&pkg.id).join(&old.version));
}
set(status, |s| {
if restart {
s.restart.insert(pkg.id.clone());
} else {
s.restart.remove(&pkg.id);
}
});
Ok(if restart {
format!("{} {} installed: restart EffectCraft to use it", pkg.name, rel.version)
} else {
format!("{} {} installed", pkg.name, rel.version)
})
}
/// Run `job` on a worker thread (`wait`: right here), reporting through `status`.
fn spawn(status: &Arc<Mutex<Status>>, what: String, wait: bool, job: impl FnOnce(&Arc<Mutex<Status>>) -> std::result::Result<String, String> + Send + 'static) {
set(status, |s| {
s.busy = Some(what);
s.error = None;
});
let st = status.clone();
let run = move || {
// A panic in a job (a hostile module, say) must not leave the library busy for good.
let r = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| job(&st))).unwrap_or_else(|_| Err("the job stopped unexpectedly".into()));
set(&st, |s| {
s.busy = None;
match r {
// An empty message: nothing to toast (the result shows where it was asked for).
Ok(msg) if msg.is_empty() => {}
Ok(msg) => s.done = Some((msg, false)),
Err(e) => {
s.done = Some((format!("Plug-in Library: {e}"), true));
s.error = Some(e);
}
}
});
};
if wait {
run();
} else if let Err(e) = std::thread::Builder::new().name("ec-library".into()).spawn(run) {
set(status, |s| {
s.busy = None;
s.error = Some(format!("cannot start: {e}"));
});
}
}
// ---------------------------------------------------------------- commands
/// The library needs files and `curl`: not in the web build.
fn desktop_only(c: &str) -> Result<()> {
if cfg!(target_arch = "wasm32") {
return Err(bad(c, "the Plug-in Library is available in the desktop app"));
}
Ok(())
}
/// Fails while the worker runs a job (it takes one at a time).
fn idle(s: &Session, c: &str) -> Result<()> {
match s.library.status().busy {
Some(b) => Err(bad(c, format!("the Plug-in Library is busy ({b})"))),
None => Ok(()),
}
}
/// After a `wait`: the job's failure, as the command's.
fn waited(s: &Session, c: &str) -> Result<()> {
match s.library.status().error {
Some(e) => Err(bad(c, e)),
None => Ok(()),
}
}
/// The catalog in memory, else the one cached on disk.
fn catalog(s: &Session) -> Option<(String, Arc<Catalog>)> {
if let Some(c) = s.library.status().catalog {
return Some(c);
}
let dir = s.library_dir()?;
let cache: Cache = serde_json::from_slice(&std::fs::read(dir.join(CACHE_FILE)).ok()?).ok()?;
let (cat, _) = parse_catalog(&serde_json::to_vec(&cache.catalog).ok()?).ok()?;
let c = (cache.url, Arc::new(cat));
set(&s.library.status, |st| {
st.catalog = Some(c.clone());
st.fetched = Some(cache.fetched);
});
Some(c)
}
/// `library.refresh`: fetch the catalog (`catalog`, else the community one) and keep a copy for offline use.
fn refresh(s: &mut Session, p: &Value) -> Result<Value> {
let c = "library.refresh";
desktop_only(c)?;
idle(s, c)?;
let url = str_p(p, "catalog").unwrap_or(CATALOG_URL).to_string();
let src = source(&url, None).map_err(|e| bad(c, e))?;
let dir = s.library_dir().ok_or_else(|| bad(c, "no settings folder for the Plug-in Library"))?;
let wait = b_p(p, "wait").unwrap_or(false);
spawn(&s.library.status, "Fetching the plug-in catalog…".into(), wait, move |st| {
let bytes = fetch(&src, MAX_CATALOG, &dir)?;
let (cat, warnings) = parse_catalog(&bytes)?;
for w in &warnings {
log::warn!("plug-in catalog: {w}");
}
let secs = web_time::SystemTime::now().duration_since(web_time::UNIX_EPOCH).map(|d| d.as_secs()).unwrap_or(0);
let fetched = chrono::DateTime::from_timestamp(i64::try_from(secs).unwrap_or(0), 0).map(|t| t.to_rfc3339()).unwrap_or_default();
let cache = Cache { url: url.clone(), fetched: fetched.clone(), catalog: cat.clone() };
if let Ok(text) = serde_json::to_vec(&cache) {
use crate::config::FileOps;
if let Err(e) = crate::config::StdFiles.write(&dir.join(CACHE_FILE), &text) {
log::warn!("plug-in catalog: cannot keep a copy: {e}");
}
}
let n = cat.packages.len();
set(st, |s| {
s.catalog = Some((url, Arc::new(cat)));
s.fetched = Some(fetched);
});
Ok(format!("Plug-in catalog: {n} package(s)"))
});
if wait {
waited(s, c)?;
let n = s.library.status().catalog.map_or(0, |(_, c)| c.packages.len());
return Ok(json!({"packages": n}));
}
Ok(json!({"refreshing": true}))
}
/// `q` (case-insensitive) is in the package's id, name, author, description, tags or effect ids.
fn matches(p: &Package, q: &str) -> bool {
let q = q.to_lowercase();
let latest = p.versions.iter().flat_map(|r| r.effects.iter());
[&p.id, &p.name, &p.author, &p.description].into_iter().chain(p.tags.iter()).chain(latest).any(|f| f.to_lowercase().contains(&q))
}
/// One package as `library.list` reports it.
fn package_json(p: &Package, inst: Option<&InstalledPackage>, st: &Status) -> Value {
let latest = latest_compatible(p);
let newest_any = p.versions.iter().map(|r| r.version.as_str()).fold(None, |b: Option<&str>, v| if b.is_none_or(|b| newer(v, b)) { Some(v) } else { b });
json!({
"id": p.id,
"name": p.name,
"author": p.author,
"description": p.description,
"license": p.license,
"repo": p.repo,
"tags": p.tags,
"latest": latest.map(|r| r.version.clone()),
"size": latest.and_then(|r| r.size),
"effects": latest.map(|r| r.effects.clone()).unwrap_or_default(),
// A newer release this build can't run (it needs a newer EffectCraft or plug-in API).
"needsNewerApp": newest_any.is_some_and(|v| latest.is_none_or(|l| newer(v, &l.version))),
"installed": inst.map(|i| i.version.clone()),
"updateAvailable": matches!((inst, latest), (Some(i), Some(l)) if newer(&l.version, &i.version)),
"loaded": st.loaded.get(&p.id),
"restartNeeded": st.restart.contains(&p.id),
"video": p.preview.as_ref().and_then(|v| v.video.clone()),
// `library.preview` rendered it (this session).
"previewed": st.preview.as_ref().is_some_and(|v| v.id == p.id),
})
}
/// `library.list`: the catalog packages with their install state, filtered, and installed ones it no longer lists.
fn list(s: &mut Session, p: &Value) -> Result<Value> {
let cat = catalog(s);
let st = s.library.status();
let dir = s.library_dir();
let inst = dir.as_deref().map(read_installed).unwrap_or_default();
let (query, tag, only_installed) = (str_p(p, "query").unwrap_or("").trim(), str_p(p, "tag"), b_p(p, "installed").unwrap_or(false));
let mut v = vec![];
let mut known = BTreeSet::new();
let mut tags = BTreeSet::new();
if let Some((_, c)) = &cat {
for pkg in &c.packages {
known.insert(pkg.id.as_str());
tags.extend(pkg.tags.iter().cloned());
let i = inst.packages.get(&pkg.id);
if (only_installed && i.is_none()) || (!query.is_empty() && !matches(pkg, query)) || tag.is_some_and(|t| !pkg.tags.iter().any(|x| x == t)) {
continue;
}
v.push(package_json(pkg, i, &st));
}
}
// Installed packages the catalog no longer lists (or no catalog yet).
for (id, i) in &inst.packages {
if known.contains(id.as_str()) || (!query.is_empty() && !id.to_lowercase().contains(&query.to_lowercase())) || tag.is_some() {
continue;
}
let pkg = Package { id: id.clone(), name: id.clone(), ..Default::default() };
let mut j = package_json(&pkg, Some(i), &st);
j["effects"] = json!(i.effects);
j["unlisted"] = json!(true);
v.push(j);
}
Ok(json!({
"catalog": cat.as_ref().map(|(u, _)| u.clone()),
"fetched": st.fetched,
"packages": v,
"tags": tags,
"busy": st.busy,
"error": st.error,
"folder": dir.map(|d| d.display().to_string()),
"available": !cfg!(target_arch = "wasm32") && s.plugin_loader.is_some(),
}))
}
/// `library.install`: download, check and load a release (`version`, else the newest compatible).
fn install(s: &mut Session, p: &Value) -> Result<Value> {
let c = "library.install";
desktop_only(c)?;
idle(s, c)?;
let loader = s.plugin_loader.ok_or_else(|| bad(c, "WebAssembly effect plug-ins are not available in this build"))?;
let previewer = s.plugin_previewer;
let (url, pkg, rel) = pick(s, c, p)?;
let dir = s.library_dir().ok_or_else(|| bad(c, "no settings folder to install into"))?;
let wait = b_p(p, "wait").unwrap_or(false);
let what = format!("Installing {} {}…", pkg.name, rel.version);
let (pid, version) = (pkg.id.clone(), rel.version.clone());
spawn(&s.library.status, what, wait, move |st| install_job(&dir, &url, &pkg, &rel, loader, previewer, st));
if wait {
waited(s, c)?;
let st = s.library.status();
return Ok(json!({"id": pid, "version": version, "installed": true, "restartNeeded": st.restart.contains(&pid)}));
}
Ok(json!({"id": pid, "version": version, "installing": true}))
}
/// The catalog's URL, the package `id` names and its release: `version`, else the newest this
/// build runs. The release's URL is checked against the rules here, so the command fails rather
/// than the worker.
fn pick(s: &Session, c: &str, p: &Value) -> Result<(String, Package, Release)> {
let id = str_p(p, "id").ok_or_else(|| bad(c, "missing `id` (a package in library.list)"))?;
let (url, cat) = catalog(s).ok_or_else(|| bad(c, "no plug-in catalog yet: run library.refresh"))?;
let pkg = cat.packages.iter().find(|x| x.id == id).cloned().ok_or_else(|| bad(c, format!("the catalog has no package `{id}`")))?;
let rel = match str_p(p, "version") {
Some(v) => pkg.versions.iter().find(|r| r.version == v).cloned().ok_or_else(|| bad(c, format!("{} has no version {v}", pkg.name)))?,
None => {
latest_compatible(&pkg).cloned().ok_or_else(|| bad(c, format!("{} has no release for this EffectCraft (plug-in API or app version)", pkg.name)))?
}
};
if !runs_here(&rel) {
return Err(bad(
c,
format!(
"{} {} needs plug-in API {} and EffectCraft {} or later; this is EffectCraft {} with WebAssembly plug-in API {}",
pkg.name,
rel.version,
rel.api,
rel.min_app.as_deref().unwrap_or("(any)"),
env!("CARGO_PKG_VERSION"),
crate::effects::plugin::WASM_API_VERSION
),
));
}
source(&rel.url, Some(&url)).map_err(|e| bad(c, e))?;
Ok((url, pkg, rel))
}
const PREVIEW_W: u32 = 320;
const PREVIEW_H: u32 = 180;
const PREVIEW_FPS: f64 = 12.0;
const PREVIEW_FRAMES: usize = 24;
/// The preview's sample frame (opaque, premultiplied RGBA): a colour gradient with two discs and
/// a band of fine stripes, so colour, blur, edge and distortion effects all show.
fn sample_frame() -> Vec<[f32; 4]> {
let (w, h) = (PREVIEW_W as f32, PREVIEW_H as f32);
let mut px = Vec::with_capacity((PREVIEW_W * PREVIEW_H) as usize);
for y in 0..PREVIEW_H {
for x in 0..PREVIEW_W {
let (u, v) = (x as f32 / w, y as f32 / h);
let mut c = [0.15 + 0.7 * u, 0.2 + 0.5 * v, 0.85 - 0.6 * u];
for (cx, cy, r, col) in [(0.3, 0.42, 0.24, [1.0, 0.85, 0.2]), (0.68, 0.5, 0.18, [0.95, 0.25, 0.35])] {
// Coverage with a one-pixel soft edge.
let a = (r * h - ((u - cx) * w).hypot((v - cy) * h)).clamp(0.0, 1.0);
for (ch, k) in c.iter_mut().zip(col) {
*ch += (k - *ch) * a;
}
}
if v > 0.82 {
let s = if (x / 4) % 2 == 0 { 0.95 } else { 0.05 };
c = [s; 3];
}
px.push([c[0], c[1], c[2], 1.0]);
}
}
px
}
/// Premultiplied float pixels as straight RGBA8.
fn to_rgba8(px: &[[f32; 4]]) -> Vec<u8> {
let byte = |v: f32| (v.clamp(0.0, 1.0) * 255.0).round() as u8;
px.iter()
.flat_map(|p| {
let a = p[3].clamp(0.0, 1.0);
let un = |c: f32| if a > 0.0 { c / a } else { 0.0 };
[byte(un(p[0])), byte(un(p[1])), byte(un(p[2])), byte(a)]
})
.collect()
}
/// `library.preview`'s reply: size, frame count, whether the effect changes anything, and the manifest.
fn preview_json(v: &Preview) -> Value {
json!({
"id": v.id,
"version": v.version,
"width": v.width,
"height": v.height,
"fps": v.fps,
"frames": v.frames.len(),
// Whether the effect changes the sample frame at all, and over time.
"changes": v.frames.iter().any(|f| *f != v.before),
"animated": v.frames.windows(2).any(|w| w.first() != w.last()),
"effect": v.manifest,
})
}
/// Render a package's effect on the sample frame without installing it.
fn preview(s: &mut Session, p: &Value) -> Result<Value> {
let c = "library.preview";
desktop_only(c)?;
let previewer = s.plugin_previewer.ok_or_else(|| bad(c, "WebAssembly effect plug-ins are not available in this build"))?;
let (url, pkg, rel) = pick(s, c, p)?;
let same = |v: &Preview| v.id == pkg.id && v.version == rel.version;
if let Some(v) = s.library.status().preview.filter(|v| same(v)) {
return Ok(preview_json(&v));
}
idle(s, c)?;
let dir = s.library_dir().ok_or_else(|| bad(c, "no settings folder for the Plug-in Library"))?;
let wait = b_p(p, "wait").unwrap_or(false);
let (pid, version) = (pkg.id.clone(), rel.version.clone());
spawn(&s.library.status, format!("Previewing {} {}…", pkg.name, rel.version), wait, move |st| {
let bytes = download(&dir, &url, &pkg, &rel)?;
let before = sample_frame();
let times: Vec<f64> = (0..PREVIEW_FRAMES).map(|i| i as f64 / PREVIEW_FPS).collect();
let (manifest, frames) =
previewer(&bytes, PREVIEW_W, PREVIEW_H, &before, &times).map_err(|e| format!("{} {} didn't render: {e}", pkg.name, rel.version))?;
let v = Preview {
id: pkg.id,
version: rel.version,
width: PREVIEW_W,
height: PREVIEW_H,
fps: PREVIEW_FPS,
before: to_rgba8(&before),
frames: frames.iter().map(|f| to_rgba8(f)).collect(),
manifest,
};
set(st, |s| s.preview = Some(Arc::new(v)));
Ok(String::new())
});
if wait {
waited(s, c)?;
if let Some(v) = s.library.status().preview.filter(|v| v.id == pid && v.version == version) {
return Ok(preview_json(&v));
}
}
Ok(json!({"id": pid, "version": version, "previewing": true}))
}
/// `library.update`: install the newest compatible release of every package (or `id`) with an update.
fn update(s: &mut Session, p: &Value) -> Result<Value> {
let c = "library.update";
let list = list(s, &json!({"installed": true}))?;
let due: Vec<String> = list["packages"]
.as_array()
.into_iter()
.flatten()
.filter(|x| x["updateAvailable"].as_bool().unwrap_or(false))
.filter_map(|x| x["id"].as_str().map(str::to_string))
.filter(|x| str_p(p, "id").is_none_or(|want| want == x))
.collect();
if due.is_empty() {
return Ok(json!({"updating": []}));
}
desktop_only(c)?;
idle(s, c)?;
let loader = s.plugin_loader.ok_or_else(|| bad(c, "WebAssembly effect plug-ins are not available in this build"))?;
let previewer = s.plugin_previewer;
// A package that can't be updated doesn't hold back the others.
let (mut jobs, mut failed) = (vec![], vec![]);
for id in &due {
match pick(s, c, &json!({"id": id})) {
Ok(job) => jobs.push(job),
Err(e) => failed.push(e.to_string()),
}
}
let dir = s.library_dir().ok_or_else(|| bad(c, "no settings folder to install into"))?;
let wait = b_p(p, "wait").unwrap_or(false);
// One job for all of them: the worker runs one job at a time.
spawn(&s.library.status, format!("Updating {} plug-in(s)…", due.len()), wait, move |st| {
let mut done = vec![];
for (url, pkg, rel) in &jobs {
match install_job(&dir, url, pkg, rel, loader, previewer, st) {
Ok(msg) => done.push(msg),
Err(e) => failed.push(e),
}
}
match (done.is_empty(), failed.is_empty()) {
(_, true) => Ok(done.join("; ")),
(true, false) => Err(failed.join("; ")),
(false, false) => Err(format!("{}; not updated: {}", done.join("; "), failed.join("; "))),
}
});
if wait {
waited(s, c)?;
}
Ok(json!({"updating": due}))
}
/// `library.uninstall`: forget the package and delete its folder; loaded effects stay until restart.
fn uninstall(s: &mut Session, p: &Value) -> Result<Value> {
let c = "library.uninstall";
idle(s, c)?;
let id = str_p(p, "id").ok_or_else(|| bad(c, "missing `id`"))?;
let dir = s.library_dir().ok_or_else(|| bad(c, "no settings folder"))?;
let mut inst = read_installed(&dir);
if inst.packages.remove(id).is_none() {
return Ok(json!({"removed": false}));
}
write_installed(&dir, &inst).map_err(|e| bad(c, e))?;
// It is uninstalled either way (it no longer loads); a folder left behind is only disk space.
if valid_id(id)
&& let Err(e) = std::fs::remove_dir_all(dir.join(id))
{
log::warn!("plug-in library: cannot remove the folder of {id}: {e}");
}
let restart = s.library.status().loaded.contains_key(id);
set(&s.library.status, |st| {
if restart {
st.restart.insert(id.to_string());
}
});
s.toast(if restart { format!("Removed {id}: its effects stay until you restart EffectCraft") } else { format!("Removed {id}") });
Ok(json!({"removed": true, "restartNeeded": restart}))
}
/// Effects used in the project that aren't registered, with the catalog packages that provide them.
fn missing(s: &mut Session, _: &Value) -> Result<Value> {
let mut effects = BTreeSet::new();
for (_, comp) in s.project.comps() {
for l in &comp.layers {
for g in l.effects().into_iter().flat_map(|fx| fx.groups()) {
if let effectcraft_project::GroupKind::Effect { effect } = &g.kind
&& crate::effects::find(effect).is_none()
{
effects.insert(effect.clone());
}
}
}
}
let cat = catalog(s);
let found: Vec<Value> = effects
.iter()
.map(|e| {
let pkgs: Vec<&str> = cat
.iter()
.flat_map(|(_, c)| c.packages.iter())
.filter(|p| latest_compatible(p).is_some_and(|r| r.effects.iter().any(|x| x == e)))
.map(|p| p.id.as_str())
.collect();
json!({"effect": e, "packages": pkgs})
})
.collect();
Ok(json!({"effects": found}))
}
/// The `library.*` commands.
pub fn specs() -> Vec<CommandSpec> {
vec![
query!(
"library.list",
"Plug-in Library",
"{query?, tag?, installed?: bool} → {catalog, fetched, packages: [{id, name, author, description, license, repo, tags, latest, effects, installed, updateAvailable, needsNewerApp, loaded, restartNeeded, video, previewed, unlisted?}], tags, busy, error, folder, available}",
list
),
query!(
"library.missing",
"Missing Effects in the Plug-in Library",
"{} → {effects: [{effect, packages: [package id]}]} — effects the project uses that aren't installed, and the catalog packages that provide them",
missing
),
cmd!(
"library.refresh",
"Refresh Plug-in Catalog",
[],
None,
"{catalog?: https:// or file:// URL of an index.json (default: the community catalog), wait?}",
always,
refresh
),
cmd!(
"library.preview",
"Preview Plug-in",
[],
None,
"{id, version?, wait?} → {id, version, width, height, fps, frames, changes, animated, effect: manifest} — downloads the release (checked like an install) and renders its effect on a sample frame at default settings, without installing it",
always,
preview
),
cmd!(
"library.install",
"Install Plug-in",
[],
None,
"{id, version?, wait?} — downloads the release, checks its SHA-256 against the catalog, loads it",
always,
install
),
cmd!("library.update", "Update Plug-ins", [], None, "{id? (default: every package with an update), wait?}", always, update),
cmd!("library.uninstall", "Remove Plug-in", [], None, "{id} — its effects stay registered until the next start", always, uninstall),
cmd!("library.loadInstalled", "Load Installed Plug-ins", [], None, "{} → {loaded, errors} (the app runs it at start-up)", always, |s, _| {
load_installed(s)
}),
]
}
#[cfg(test)]
mod tests {
use super::*;
/// A release of `version` for `api`, with a valid GitHub URL and checksum.
fn rel(version: &str, api: u32, min_app: Option<&str>) -> Release {
Release {
version: version.into(),
api,
min_app: min_app.map(Into::into),
url: "https://github.com/a/b/x.wasm".into(),
sha256: "a".repeat(64),
size: None,
effects: vec![],
}
}
#[test]
fn versions_compare_numerically() {
assert!(newer("1.10.0", "1.9.9"));
assert!(!newer("1.0", "1.0.0"));
assert!(!newer("1.0.0", "1.0"));
assert!(newer("2", "1.99"));
assert!(!newer("x", "1.0"));
assert_eq!(version_key("1..2"), None);
assert_eq!(version_key("-1"), None);
assert_eq!(version_key(&"9".repeat(40)), None);
}
#[test]
fn ids_are_safe_folder_names() {
for ok in ["org.example.glow", "a", "glow-pack_2"] {
assert!(valid_id(ok), "{ok}");
}
for no in ["", ".hidden", "a.", "a/../b", "a\\b", "..", "a..b", "c:", "x y", &"a".repeat(101)] {
assert!(!valid_id(no), "{no}");
}
}
#[test]
fn ids_that_differ_only_in_case_are_one_package() {
let pkg = |id: &str| json!({"id": id, "name": "Glow", "versions": []});
let bytes = serde_json::to_vec(&json!({"schema": 1, "packages": [pkg("Org.X.Glow"), pkg("org.x.glow")]})).unwrap();
let (cat, warnings) = parse_catalog(&bytes).unwrap();
assert_eq!(cat.packages.len(), 1);
assert_eq!(cat.packages[0].id, "Org.X.Glow");
assert!(warnings.iter().any(|w| w.contains("second package `org.x.glow`")), "{warnings:?}");
}
#[test]
fn the_newest_compatible_release_wins() {
let api = crate::effects::plugin::WASM_API_VERSION;
let p = Package {
versions: vec![
rel("1.0.0", api, None),
rel("1.2.0", api, Some("0.1")),
rel("1.1.0", api, None),
rel("9.0.0", api, Some("999.0")),
rel("3.0.0", api + 1, None),
rel("4.0.0", 0, None),
],
..Default::default()
};
assert_eq!(latest_compatible(&p).map(|r| r.version.as_str()), Some("1.2.0"));
assert!(runs_here(&rel("1.0.0", 1, None)), "API 1 modules still run on a build with a newer plug-in API");
assert!(!runs_here(&rel("1.0.0", 2, None)), "WebAssembly modules are API 1: an API 2 release can't load");
let j = package_json(&p, None, &Status::default());
assert_eq!(j["needsNewerApp"], true);
assert!(latest_compatible(&Package::default()).is_none());
}
#[test]
fn catalogs_leave_out_bad_entries() {
let good = "a".repeat(64);
let text = json!({"schema": 1, "packages": [
{"id": "org.ok", "name": "Ok", "versions": [
{"version": "1.0.0", "api": 1, "url": "https://github.com/x", "sha256": good.to_uppercase()},
{"version": "nope", "api": 1, "url": "https://github.com/x", "sha256": good},
{"version": "1.1.0", "api": 1, "url": "https://github.com/x", "sha256": "short"}
]},
{"id": "org.ok", "name": "Second"},
{"id": "../evil", "name": "Evil"},
{"id": "org.noname", "name": " "},
{"id": "org.extra", "name": "Extra", "unknownField": true}
]});
let (c, warnings) = parse_catalog(text.to_string().as_bytes()).unwrap();
let ids: Vec<&str> = c.packages.iter().map(|p| p.id.as_str()).collect();
assert_eq!(ids, ["org.ok", "org.extra"]);
assert_eq!(c.packages[0].versions.len(), 1);
assert_eq!(c.packages[0].versions[0].sha256, good, "checksums are kept in lower case");
assert_eq!(warnings.len(), 5, "{warnings:?}");
assert!(parse_catalog(br#"{"schema": 2, "packages": []}"#).unwrap_err().contains("update EffectCraft"));
assert!(parse_catalog(b"not json").is_err());
assert!(parse_catalog(&vec![b' '; (MAX_CATALOG + 1) as usize]).unwrap_err().contains("larger"));
}
#[test]
fn only_github_https_and_local_catalog_files_are_fetched() {
assert!(matches!(source("https://github.com/a/b/releases/download/v1/x.wasm", Some(CATALOG_URL)), Ok(Source::Https(_))));
assert!(matches!(source("https://GitHub.com/a", Some(CATALOG_URL)), Ok(Source::Https(_))));
for no in [
"http://github.com/a",
"https://example.com/x.wasm",
"https://github.com.evil.com/x",
"https://user@github.com/x",
"https://github.com:8443/x",
"ftp://github.com/x",
] {
assert!(source(no, Some(CATALOG_URL)).is_err(), "{no}");
}
// file:// only from a local catalog (or as the catalog).
assert!(source("file:///tmp/x.wasm", Some(CATALOG_URL)).is_err());
assert_eq!(source("file:///tmp/x.wasm", Some("file:///tmp/index.json")), Ok(Source::File(PathBuf::from("/tmp/x.wasm"))));
assert_eq!(source("file:///C:/p/x.wasm", None), Ok(Source::File(PathBuf::from("C:/p/x.wasm"))));
assert!(source("file://", None).is_err());
// Percent-decoded; no remote hosts.
assert_eq!(file_url_path("file:///C:/My%20Plug-ins/Jos%C3%A9%23.wasm"), Some(PathBuf::from("C:/My Plug-ins/José#.wasm")));
assert_eq!(file_url_path("file://localhost/home/x.wasm"), Some(PathBuf::from("/home/x.wasm")));
for no in ["file://server/share/x.wasm", "file:///bad%2", "file:///bad%zz"] {
assert_eq!(file_url_path(no), None, "{no}");
}
}
#[test]
fn the_sample_frame_round_trips_to_rgba8() {
let f = sample_frame();
assert_eq!(f.len(), (PREVIEW_W * PREVIEW_H) as usize);
assert!(f.iter().all(|p| p[3] == 1.0));
assert_eq!(to_rgba8(&[[0.5, 0.25, 0.0, 0.5], [1.0, 2.0, -1.0, 0.0]]), [255, 128, 0, 128, 0, 0, 0, 0]);
}
#[test]
fn module_paths_stay_in_the_library() {
let d = Path::new("lib");
assert_eq!(module_path(d, "a/1.0/a.wasm"), Some(d.join("a").join("1.0").join("a.wasm")));
for no in ["../x/a.wasm", "a/../a.wasm", "a/1/..", "a.wasm", "a/1/b/c.wasm", "a//a.wasm", "C:/x/a.wasm", "a\\..\\b/1/x"] {
assert_eq!(module_path(d, no), None, "{no}");
}
}
}