Merge pull request #6223 from dotCooCoo/feat/plugin-filter-inputs

Plug-ins: filters can ask to read type, images and symbol instances

docs/mcp.md

@@ -6642,7 +6642,17 @@ WebAssembly plug-ins ([plugins.md](plugins.md)) are reached through commands; th
 them again at startup (`plugin.remove` deletes them there):
 `plugin.install {path | dataBase64}` installs one, `plugin.list` / `plugin.info {id}` describe them (parameters,
 schema, defaults). An object filter runs on the selected paths and compound paths with `plugin.run {id, params}` (one
-undo step; it may change, delete or add objects, and the result is selected). A live-effect plug-in is an effect with
+undo step; it may change, delete or add objects, and the result is selected). A filter whose manifest lists `inputs`
+(`type`, `image`, `symbolInstance`; `inputs` in its `plugin.list` entry) also lists `doc.read` in its `permissions`, so
+`plugin.install` needs `grant`. It is also given the selected objects of those kinds to read, with at most 16 MiB of
+pixels and 16 MiB of type runs over one run (later ones come with `pixelsOmitted` or `truncated`). Every object counts
+toward the input's half of the plug-in's memory cap, ids, bounds, transforms and names included: pixels and runs that
+don't fit come the same way, and an object whose other fields don't fit makes `plugin.run` return a memory-budget error
+naming the object and its size, with nothing changed. A path or compound path that doesn't fit makes it return the
+memory-budget error `the selection is too large to hand over`, and more than 100,000 of these objects make it return
+an object-budget error. Its output returns
+each of them as `{"id": …}` or exactly as given, otherwise `plugin.run` returns an error naming the object and the
+field, and the document and undo history are unchanged. A live-effect plug-in is an effect with
 id `plugin.<id>`: `effect.apply`, `effect.setParams` and `effect.expandAppearance` work as for built-in effects, and
 `effect.list` lists it. One whose manifest declares `paintOutput` (at plug-in API level 2; `plugin.list` gives `api`
 and `paintOutput: true`) draws the pieces it returns with paints of their own, and `effect.expandAppearance` writes them

docs/plugins.md

@@ -224,9 +224,12 @@ capability model proposed for PhotoCraft plug-ins
   `documentData` is `null` and an output that sets it is refused; without `doc.write` an output that changes the
   document is refused. A refused output leaves the document as it was.
 - `api` is the plug-in API level the manifest is written for: `1` (the default) or `2`, which adds the [host
-  imports](#host-imports-api-level-2) and a live effect's [`paintOutput`](#painted-live-effects). A higher one is
-  refused. `plugin.list` and `plugin.info` give it as `api`. A level 2 effect that declares `paintOutput` needs no
-  other permission than `doc.read` and `effect.render`.
+  imports](#host-imports-api-level-2), a filter's `inputs` ([reading type, images and
+  symbols](#reading-type-images-and-symbols)) and a live effect's [`paintOutput`](#painted-live-effects). A higher one
+  is refused. `plugin.list` and `plugin.info` give it as `api`. `inputs` is not a permission: the objects it asks for
+  are read under `doc.read`, which a filter that lists `inputs` must list in `permissions`, so installing it asks the
+  user to grant them. A level 2 effect that declares `paintOutput` needs no other permission than `doc.read` and
+  `effect.render`.
 - Parameter declarations are unchanged by permissions (`params`, below).
 
 Every run gets a **fresh instance**, so no state survives between calls except what a plug-in writes to its
@@ -379,6 +382,11 @@ A live effect runs while the art draws, so keep the entries it uses small.
   filters](#heavy-filters)). Hosts from before it ignore the key, so there the filter gets a light run's budget.
 - `api`: the plug-in API level, `1` when left out. `permissions`: what the plug-in may do, a list of names
   ([permissions](#permissions)); without it, what its kind implies.
+- `inputs` (filters at `api` 2 only): the kinds of object besides paths and compound paths the filter reads, any of
+  `"type"`, `"image"` and `"symbolInstance"`, each at most once (`"inputs": ["type", "image"]`). Left out or `[]`, the
+  filter gets paths and compound paths alone, as before. An unknown, repeated or wrongly typed entry, `inputs` on
+  another kind, below `api` 2, or without a `permissions` list that holds `doc.read`, is refused when the plug-in is
+  installed, and so is `api` 2 on hosts from before it.
 - `paintOutput` (live effects at `api` 2 only): `true` makes the effect return [painted pieces](#painted-live-effects)
   instead of geometry alone. `true` is the only value the key takes; `false`, `null`, another type, another kind or
   `api` 1 is refused when the module is installed, and the error names `paintOutput`. Without the key the effect returns
@@ -429,8 +437,9 @@ parameters uses `label`, `unit`, `help` and `visibleWhen`, and lists the paramet
 
 `input` is UTF-8 JSON: `{"objects": [...]}`. An object filter gets the selected paths and compound paths in paint
 order (bottom first): the selected ones and those inside selected groups and layers; hidden, locked, guide (compound
-guides too) and clipping paths are left out, and so are other kinds of object (type, images, symbols, live blends…), which a filter never
-touches. With nothing selected the list is empty (a generator then adds its own objects). A live effect gets one
+guides too) and clipping paths are left out, and so are other kinds of object (type, images, symbols, live blends…),
+which a filter never touches unless its manifest lists them in `inputs` ([reading type, images and
+symbols](#reading-type-images-and-symbols)). With nothing selected the list is empty (a generator then adds its own objects). A live effect gets one
 object: its geometry (`type`, `path`, `fillRule`, `bounds`) and its `fills` and `strokes`, as a filter gets them (hosts
 from before these give geometry only). The paints are read only: an effect may draw as wide as a stroke or use a
 fill's colour to decide its shape, but it changes only the geometry, unless it declares `paintOutput` ([painted live
@@ -479,6 +488,65 @@ An **importer** gets the bytes of the file being opened, as they are (not JSON),
   `plugin.<id>` for a live-effect plug-in), `params` and `visible`. Its fills' and strokes' own effects aren't listed.
 - `bounds`: `[x0, y0, x1, y1]`, the geometric bounds (read only).
 
+### Reading type, images and symbols
+
+A filter whose manifest lists `inputs` also gets the selected objects of those kinds, in paint order among the paths
+and under the same rules (hidden and locked ones, objects that clip a clip group, alone or inside a group that clips
+it, and anything inside a symbol, a placed document or a live object are left out). They are never turned into
+outlines, rectangles or the symbol's art. Each has `id`, `type`, `name` (when it has one), `bounds` (`null` until the
+object has been laid out or drawn) and
+`transform`, `[a, b, c, d, e, f]` mapping its own space (text space, pixel space, the symbol's art) into the document.
+In this example the character and paragraph styles are cut short; a filter gets every attribute:
+
+```json
+{"objects": [
+  {"id": 17, "type": "type", "bounds": [10, 38, 52, 52], "transform": [1, 0, 0, 1, 10, 50],
+   "text": {"kind": "point", "vertical": false,
+            "runs": [{"text": "Hello\nthere", "style": {"font_family": "Source Sans 3", "size": 12}}],
+            "paragraphs": [{"justify": "Left"}, {"justify": "Center"}]},
+   "truncated": false},
+  {"id": 18, "type": "image", "bounds": [0, 0, 4, 2], "transform": [2, 0, 0, 2, 0, 0], "pixelSize": [2, 1],
+   "colorMode": "rgb", "linked": false, "pixelsOmitted": false, "pixelsBase64": "AAAegGQAHoA="},
+  {"id": 19, "type": "symbolInstance", "bounds": [100, 0, 110, 10], "transform": [1, 0, 0, 1, 100, 0],
+   "symbolId": "Star", "overrides": {}}
+]}
+```
+
+- Type (`"type"`): `text.kind` is `"point"`, `"area"` or `"path"`; `runs` are its text runs with their character
+  attributes as the native file stores them (`inline` when a run is an inline graphic), and `paragraphs` holds one
+  paragraph style per paragraph (the paragraphs are split at `\n`). No font data or glyph outlines are given. Type
+  whose runs hold more than 256 KiB of UTF-8 text, or more than 4,096 runs or paragraphs, comes with `truncated:
+  true` and empty `runs` and `paragraphs`. The runs and paragraphs of all the type in one input take at most 16 MiB
+  of JSON: type whose runs and paragraphs don't fit what the type before it in paint order left comes with
+  `truncated: true` too.
+- An image (`"image"`): `pixelSize` is its width and height in pixels; `colorMode` is `"rgb"`, `"cmyk"`, `"gray"` or
+  `"unknown"`, read from the image itself. An embedded image whose pixels take at most 1 MiB in base64 (443 × 443
+  pixels fit) has them in `pixelsBase64`: straight RGBA, 8 bits a channel, row by row from the top, and
+  `pixelsOmitted: false`. A CMYK image's pixels are its inks through the working CMYK profile. The pixels of all the
+  images in one input take at most 16 MiB in base64. A larger image, one whose pixels don't fit what the images before
+  it in paint order left, and a linked image (`linked: true`) have
+  `pixelsOmitted: true` and no pixels; the host checks an image's size before it decodes the image. The host never
+  reads a linked image's file for a filter and never gives its path.
+- A symbol instance (`"symbolInstance"`): `symbolId` is the name of the symbol it shows, with `missing: true` when the
+  document has no symbol of that name. `overrides` is `{}`.
+
+These objects are read only. The output must list each of them exactly once, as `{"id": 17}` alone or exactly as it
+was given (numbers compare by value, so `1` and `1.0` are the same). A changed or added field, one left out, an object
+listed twice or left out, and a new object with one of these types are refused, and the document and undo history
+stay as they were: `objects[1] (id 17): read-only field 'text.runs[0].text' changed`, `objects (id 17): a read-only
+object was left out`. The field is named with control and bidirectional characters escaped (`\u{202E}`), cut to 64
+characters. New objects listed after one of them go just above it.
+
+Every object in the input counts toward the half of the memory cap that the input and the parameters share, in paint
+order, with its id, bounds, transform and name. Type whose runs and paragraphs don't fit what is left comes with
+`truncated: true`, and an image whose pixels don't fit comes with `pixelsOmitted: true`. When the other fields of one
+of these objects don't fit what is left, the run fails before the plug-in starts, naming the object and the bytes it
+needs: `plug-in exceeded its memory budget (object 17 needs 4200 bytes of input, 1000 left)`. A path or compound path
+that doesn't fit fails it with `plug-in exceeded its memory budget (the selection is too large to hand over)` before
+the objects after it are written. An output lists at most 100,000 objects, so a run given more than 100,000 of these
+objects fails with `plug-in exceeded its object budget (more than 100000 read-only objects)`. An echo of them counts
+toward the 64 MiB output cap.
+
 ### Parameters
 
 `params` points at `params_len` bytes of UTF-8 JSON: the validated parameters, plus `_context`:
@@ -632,5 +700,6 @@ per change of their input, parameters or paints (results are cached), not per fr
   importers that place files (File › Place reads only the built-in formats) or make type, images, layers or
   artboards.
 - Live effects that change opacity, blending or effects (at `api` 2 one may [paint](#painted-live-effects) the pieces
-  it returns), and filters on type, images or symbols.
+  it returns), and filters that change type, images or symbol instances (a filter at `api` 2 reads them, see
+  [`inputs`](#reading-type-images-and-symbols)).
 - Apply Last Effect and Last Effect work for plug-in effects; there is no "last filter" for object filters yet.

docs/target-app-parity.md

@@ -1571,6 +1571,7 @@ Moved from the ROADMAP's "Shipped so far" on 2026-10-10. Update it when a featur
 - **Safe Mode for plug-ins (G17):** `vectorcraft --safe-mode` and `vectorcraft-cli --safe-mode <subcommand>` start without plug-ins. When a launch stopped while a plug-in was loading or drawing its first live effect, the next launch names that plug-in, without saying it caused the stop, and offers Disable and Continue, Start Normally or Start in Safe Mode. Help › Safe Mode lists the disabled plug-ins with Enable on Re-launch, which takes effect at the next launch, and the app bar and the window title show Safe Mode. The help page (https://web.archive.org/web/20210601162814/https://helpx.adobe.com/pdf/illustrator_reference.pdf, pages 65 to 69) describes the same choice between diagnosing and starting normally, a Safe Mode with the problem item off and the rest of the app working, and turning the item back on at the next launch. In Safe Mode no plug-in loads or runs, and a live-effect plug-in draws its object's geometry unchanged while the object keeps the effect, which the Appearance panel marks "(Safe Mode)" with the plug-in's id in its tooltip and `effect.list` marks `skipped`; Expand Appearance refuses such an object. The web build keeps no plug-ins between page loads and has no marker. Each desktop launch with preferences keeps a marker in the `Launch` folder next to them; `vectorcraft-cli` and a desktop launch without preferences keep none, so a stop there leads to no choice at the next launch. Disabled plug-ins are listed in `Plug-ins/disabled.json`. `plugin.safeMode` reports Safe Mode and the disabled plug-ins, and `plugin.setEnabled` changes the list. Regressions: `crates/plugins/src/safe_mode.rs`, `crates/plugins/tests/safe_mode.rs`, `crates/engine/tests/plugin_safe_mode.rs`, `crates/ui-egui/tests/safe_mode.rs`. Scores and hours unchanged.
 - **Plug-in instruction budget at a bulk memory instruction:** a run that pauses where one slice of its instruction budget ends keeps the instructions the slice still held. A `memory.fill` or `memory.copy` that the slice can't pay for in full runs on the next slice, so a run whose budget covers its work finishes; a run ends with `Limit("instruction budget")` only when the instructions it has left can't pay for the next instruction. Regressions: `crates/plugins/tests/abi.rs`. Scores and hours unchanged.
 - **Plug-in permissions (G17):** a manifest may list `permissions` (`doc.read`, `doc.write`, `state`, `effect.render`, `export.write`; the first three follow storytold/photocraft#662) and an `api` level; without the list a plug-in gets what its kind implies and installs as before. Install Plug-in… and File › Open show the name, id, version, file SHA-256 and each permission in plain words, with control and bidirectional characters escaped, and install nothing on Cancel; for a plug-in that lists permissions, `plugin.install` needs `grant` and folders install it only when it is granted. The grant is kept with the module's hash, a module asking for more asks again, and a filter run refuses output its permissions don't allow. Regressions: `crates/plugins/src/permissions.rs`, `crates/plugins/src/manifest.rs`, `crates/engine/src/tests/plugins.rs`, `crates/engine/tests/plugin_fuzz.rs`, `crates/ui-egui/src/tests/plugins.rs`. Scores and hours unchanged.
+- **Filter plug-ins read type, images and symbol instances (G17, #4333):** a filter at plug-in API level 2 lists `inputs` (`type`, `image`, `symbolInstance`) with `doc.read` in its `permissions`, so installing it asks for approval, and is also given the selected objects of those kinds, read only: type with its runs, character and paragraph attributes (no text past 256 KiB or 4,096 runs or paragraphs, or 16 MiB of runs over the input), images with their size, color mode and, when embedded and at most 1 MiB in base64 (16 MiB over the input), their RGBA pixels (a CMYK image's through the working CMYK profile; never a linked file's), and symbol instances with their symbol and transform. Every object counts toward the input's half of the memory cap; one whose fields besides its pixels and runs don't fit fails the run naming it and its size, a path or compound path that doesn't fit fails it too, and so do more than 100,000 of these objects. Its output returns each as `{id}` or exactly as given; any change or omission is refused before the document changes. Filters without `inputs` get the same input as before, except that compound paths that clip a clip group and paths inside a group that clips one are left out for every filter. Regressions: `crates/plugins/src/objects/tests.rs`, `crates/plugins/src/manifest.rs`, `crates/engine/src/tests/plugins.rs`, `crates/engine/tests/plugin_fuzz.rs`. Scores and hours unchanged.
 - **Plug-in grants cover one module (G17):** a grant counts only for the module with the id and SHA-256 it records. A module that lists permissions and has other bytes under the same id needs a new approval, even with the same permissions: `plugin.inspect` reports `needsApproval`, `plugin.install` refuses it without `grant`, Install Plug-in… and File › Open ask again, and the plug-in store and plug-in folders skip it. A module with a missing or empty permissions list needs no grant; without the list, it gets its kind's implied permissions. A grant with a malformed hash covers nothing. Regressions: `crates/plugins/src/permissions.rs`, `crates/engine/src/tests/plugins.rs`, `crates/ui-egui/src/tests/plugins.rs`. Scores and hours unchanged.
 - **Plug-in commands (G17):** a filter or live effect may declare up to 32 `commands` (`{id, label, params?}`); each is the command `plugin.<id>.<command>` while the plug-in is installed, in a submenu named after the plug-in under Object › Plug-ins (Effect › Plug-ins for a live effect) whatever menu the manifest names, with its label escaped. Commands are recorded in actions, listed in Keyboard Shortcuts for the user to give a shortcut, and run through `run` and MCP with their parameters checked first (a command with parameters opens a dialog from its menu item or shortcut). A run uses the plug-in's permissions (a live effect with commands may also hold `doc.write` and `state`, and gets them without a `permissions` list), is one undo step named after the command, and on failure (a trap or an exhausted instruction budget) leaves the document and the undo history as they were with an error naming the plug-in and the command. For a command and for `plugin.run`, an `ids` that is not a list of object ids is an error rather than the selection. A malformed entry is refused by name when the module is installed. Regressions: `crates/plugins/src/manifest.rs`, `crates/plugins/src/permissions.rs`, `crates/engine/src/tests/plugins.rs`, `crates/engine/tests/plugin_fuzz.rs`, `crates/ui-egui/src/tests/plugins.rs`. Scores and hours unchanged.
 - **Plug-in progress, Cancel and log lines (G17):** at plug-in API level 2 a plug-in with the `progress` permission imports `env.vc_progress`, `env.vc_cancelled` and `env.vc_log`. A heavy filter's and an export's progress windows show its progress as a percentage bar, their Cancel reaches it through `vc_cancelled`, and its log lines go to the app's log with its id, capped per line and per run. Installing checks every import and names the one it refuses. Regressions: `crates/plugins/src/host.rs`, `crates/plugins/tests/abi.rs`, `crates/engine/src/tests/plugins.rs`, `crates/engine/tests/plugin_fuzz.rs`, `crates/ui-egui/src/tests/plugin_runs.rs`. Scores and hours unchanged.