From 2d40d85450653702ecbf0c65c8a2166abd57564e Mon Sep 17 00:00:00 2001 From: Tanish Manku Date: Fri, 9 Oct 2026 11:44:37 -0700 Subject: [PATCH] D5: READMEs for cos and engine (#332) * D5: README for cos and engine, and an honest cos status line Thirteen crates have no README, although the session protocol tells every session to read the README of the crate it is touching. These are the two biggest: cos is where every hostile file lands first, and engine is the whole API surface a frontend gets. Each states its layer and the rules that bind it, sketches the API against the real signatures, and ends with what is deliberately not done. The cos README spells out the clone-is-a-snapshot property and why both writers return bytes rather than taking a sink; the engine README spells out the four-step editing model that makes what you see what Save writes. Also corrects the module doc on cos/src/lib.rs, which still said encryption and object-stream output for full saves were not done. Both shipped: security.rs handles R2-R6 with crypt filters and /EFF, and SaveOptions::object_streams defaults to true. Co-Authored-By: Claude Opus 5 * M14: let neutral Windows locales fall through to display language Co-authored-by: Codex * pdfcraft-cli --help lists `mcp` only when the build has it Follow-up to #540: a build without the `mcp` feature (`--no-default-features`) still listed `pdfcraft-cli mcp` in its usage, and `pdfcraft-cli mcp` then said "unknown command". USAGE is now put together with `concat!`, and its `mcp` lines come from a macro that is empty without the feature. In such a build `pdfcraft-cli mcp` says the MCP server was left out instead of calling it an unknown command. - tests/help.rs: usage_lists_mcp_only_when_built_with_it (runs in both feature sets) and mcp_without_the_feature_says_it_was_left_out (without the feature) Co-Authored-By: Claude Opus 5.5 * engine README: the execution plan is local-only, so don't link a path that doesn't exist --------- Co-authored-by: Claude Opus 5 Co-authored-by: Streetblock Co-authored-by: Codex Co-authored-by: lloyd Co-authored-by: Brandon Thomas --- crates/cos/README.md | 91 +++++++++++++++++++++++++++++++++++++++++ crates/cos/src/lib.rs | 8 ++-- crates/engine/README.md | 80 ++++++++++++++++++++++++++++++++++++ 3 files changed, 176 insertions(+), 3 deletions(-) create mode 100644 crates/cos/README.md create mode 100644 crates/engine/README.md diff --git a/crates/cos/README.md b/crates/cos/README.md new file mode 100644 index 000000000..b3ac964d5 --- /dev/null +++ b/crates/cos/README.md @@ -0,0 +1,91 @@ +# pdfcraft-cos + +The PDF object layer (ISO 32000-2 §7): the COS object graph, parsed lazily and tolerantly, edited +through a copy-on-write overlay, and written back incrementally or as a full rewrite. + +- **Layer:** L1, standalone. The layering table lets it use only `pdfcraft-filters`, + `pdfcraft-crypt` and `pdfcraft-geom` (`xtask/src/layers.rs::STANDALONE_DEPS`); it currently uses + the first two. That keeps the parser and the crypto independently auditable and reusable. +- `#![forbid(unsafe_code)]`, and the crate denies `unwrap`/`expect`/`panic!`/`todo!`. Builds for + `wasm32-unknown-unknown`. + +**The object graph is the document model.** Nothing is lifted into a tidier representation and +written back out, because that is how data gets silently dropped. Typed views live above, in +`pdfcraft-model`. + +## API + +```rust +let doc = Document::open(bytes)?; // Arc> +let doc = Document::open_with_password(bytes, Some("owner"))?; // encrypted files + +let root = doc.root(); // Option — the catalog +let obj = doc.get(ObjRef { num: 12, gen: 0 }); // load (and cache): Arc +let real = doc.resolve(&obj); // follow one indirect reference +let d = doc.dict(&obj); // Option, resolving first +let data = stream.decoded()?; // filters applied on demand + +doc.set(r, Object::Dict(d)); // edit: lands in the overlay +let r = doc.add(obj); // append a new object +doc.free(r); + +let bytes = write_incremental(&doc, &SaveOptions::default())?; // append a revision +let bytes = write_full(&doc, &SaveOptions::default())?; // garbage-collected rewrite +``` + +Both writers **return the bytes**; the caller decides where they go, which is what lets the engine +produce a working file in memory after every edit and keep saving atomic. + +Useful alongside: `doc.repair_log()` (what was reconstructed), `doc.revisions()` and +`doc.revision_ends()` (the incremental history), `doc.is_edited(num)` and `doc.modified_objects()`, +`doc.permissions()` and `doc.security()`, `doc.set_encryption()` / `doc.remove_encryption()`, and +`doc.require_full_save()` for edits that cannot be expressed incrementally. + +`Document` is **cheap to clone**: the original bytes, the xref index and the parse caches are +shared, so a clone is a snapshot. Undo/redo and background jobs hold clones while the UI edits +another. Edits never touch the original bytes. + +## Reading + +- **Every cross-reference form:** classic tables with their quirky subsections, cross-reference + streams, hybrid-reference files (`/XRefStm`), and object streams. +- **Repair.** A file whose xref is wrong or missing is reconstructed by scanning for `obj` + headers. Repairs are *recorded*, never silent, so the UI can say the file was damaged. +- **Tolerant, never fatal.** The lexer works on a byte slice and a position; it allocates nothing + for skipped data, bounds-checks every index, and caps nesting depth. Malformed input yields + `CosError`, never a panic — this crate is the first thing a hostile file meets. +- **Lazy.** Objects are parsed on first use and cached. (Whole-file buffering is still the case; + `ByteSource` streaming is `core.lazy-loading`, not yet done.) +- **Encryption** is handled on load and save through `pdfcraft-crypt`: every standard-security + revision R2–R6 (RC4, AES-128/256), crypt filters, `/EFF` embedded-file encryption, and + `/EncryptMetadata`. `security.rs` knows what the spec exempts — the `/Encrypt` dictionary, xref + streams, the trailer `/ID`, and objects inside an object stream (the stream is encrypted whole). + +## Writing + +`write_incremental` appends only the edited objects, a cross-reference section **in the same +style as the file's last one**, and a trailer with `/Prev`. The original bytes are never modified, +which is what keeps a signature over an earlier revision valid. + +`write_full` writes only what the trailer can reach, renumbered densely — the Save As and +garbage-collecting path. By default (`SaveOptions::object_streams`) it packs non-stream objects +into compressed object streams with a cross-reference stream (PDF 1.5+); turning that off writes a +classic table any reader accepts. + +Dictionaries keep their key order and streams keep their *encoded* bytes, so rewritten objects stay +recognisable in a diff and byte-stable when re-serialized. + +## Errors + +`CosError`: `NotPdf`, `Syntax { offset, detail }`, `MissingObject`, `NotADictionary`, `Filter`, +`NeedsPassword`, `WrongPassword`, `Security`, `Poisoned`. + +## Not done here + +- Streaming very large files within a memory budget (`ByteSource`, `core.lazy-loading`); files are + read whole today. +- Linearized output, Fast Web View (`core.linearization`, M11). +- Structural validation against the Arlington model (`core.arlington-validation`), which will live + in the reserved `arlington` crate. +- Image codecs. `pdfcraft-filters` carries the non-image filters; DCT, JPX, JBIG2 and CCITT are + still decoded through the bootstrap renderer (`core.image-filters`, M2.6). diff --git a/crates/cos/src/lib.rs b/crates/cos/src/lib.rs index 2a534eca4..7aa560795 100644 --- a/crates/cos/src/lib.rs +++ b/crates/cos/src/lib.rs @@ -5,9 +5,11 @@ //! untouched) or as a full, garbage-collected rewrite. The PDF object graph *is* PdfCraft's //! document model (plan/architecture.md §5, ADR-0010). //! -//! Status (M1 in progress): parsing of all xref forms, object streams, repair by scanning, -//! incremental and full writing. Not yet: encryption (M1.6), object-stream / xref-stream output -//! for full saves, linearization (M11). +//! Done: every cross-reference form (tables, streams, hybrid `/XRefStm`), object streams, repair +//! by scanning, encryption on load and save (R2–R6, crypt filters, `/EFF`), incremental writing, +//! and full saves that pack objects into compressed object streams with a cross-reference stream. +//! Not yet: streaming very large files within a memory budget (`ByteSource`, +//! `core.lazy-loading`), and linearized output (`core.linearization`, M11). See the README. #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] diff --git a/crates/engine/README.md b/crates/engine/README.md new file mode 100644 index 000000000..c0553769d --- /dev/null +++ b/crates/engine/README.md @@ -0,0 +1,80 @@ +# pdfcraft-engine + +The façade every frontend talks to. It holds the open documents, their edit history and the tool +catalogue; frontends never reach past it into the parsing, rendering or editing crates. + +- **Layer:** L6. It depends on most of L1–L4 and on nothing above itself. **No UI toolkit** — + `egui`, `winit`, `eframe` and `rfd` are forbidden below L7, and `xtask layers` enforces it. +- The crate denies `unwrap`/`expect`/`panic!`/`todo!`. Entry points that run untrusted-document + code keep a last-resort `catch_unwind` guard so an escaped panic becomes an error and the user + keeps their document. + +Two frontends exist on top of it today: `pdfcraft-ui-egui` (desktop and web) and +`pdfcraft-automation` (headless tools, the CLI and MCP). Anything one can do, the other must be +able to do — see AGENTS.md §3. + +## The editing model + +Each open document holds a `pdfcraft_cos::Document`: the object graph plus a copy-on-write overlay. + +1. An edit runs **on a clone**. On success the previous state is pushed onto the undo stack. Clones + share all unchanged data, so a snapshot is cheap — this is what makes 100 levels of undo + affordable on a 200 MB file. +2. After every edit the **working file** is produced by an incremental write: the original bytes + plus one appended revision. +3. The view is refreshed from those bytes, so **what you see is exactly what Save will write**. +4. Saving rebases onto the written bytes, so the next save appends only new edits. + +Consequences worth preserving: the original bytes of an opened file are never rewritten in place; a +full rewrite happens only where it must (redaction, PDF/A conversion, print imposition, a changed +password); and unknown data survives a round trip because nothing is reconstructed that wasn't +understood. + +## API + +```rust +let mut s = Session::new(); +let id = s.open("report.pdf", Some(path), bytes, None)?; // -> DocId + +s.apply(id, Edit::RotatePages { pages: vec![0], degrees: 90 })?; +let label = s.undo(id)?; // "Rotate pages" +s.redo(id)?; + +let doc = s.get(id).ok_or("gone")?; // &Document: queries, never edits +doc.can_undo(); doc.editable(); doc.allows_printing(); +doc.text_blocks(page); doc.page_boxes(page); doc.security_summary(); + +let bytes = s.save_bytes(id)?; // incremental: keeps signatures over earlier revisions valid +let bytes = s.save_full_bytes(id)?; // full rewrite, garbage-collected +s.mark_saved(id, bytes.clone(), path)?; +``` + +`Edit` is one enum of 75 variants — every undoable change in the product, from +`RotatePages` to `AddAnnotation`, `SetFieldValue`, `ApplyRedactions` and `AddWatermark`. Adding a +feature means adding a variant and handling it in `apply`, which is what keeps undo, the dirty +flag, autosave and the automation tools consistent for free. + +`Session` also owns the operations that produce a *new* document rather than editing one: +`create_blank`, `create_from_images`, `create_from_text`, `combine`, `extract`, `split`, +`split_by_size`, `reduced_bytes`, `optimized_bytes`, `print_pdf`, `export_data`, and the signing +family (`sign`, `sign_with_timestamp`, `timestamp_document`, `embed_ltv`). + +## Modules + +| Module | What it holds | +|---|---| +| `commands` | The command registry: id, label, menu, shortcut, and a `Needs` precondition encoding document security. Menus, the ⌘K palette, shortcuts and the control channel all read it, so every surface agrees on what exists and when it is enabled | +| `catalog` | Acrobat's "All tools" information architecture as data — groups, sections, icons, hues, and an `Availability` of `Ready`, `Planned(milestone)` or `Provider`. The UI renders what is *not* built yet, honestly | +| `actions` | PDF action dispatch (go-to, URI, named, set-layer-state, JavaScript) | +| `links` | Link creation, properties, and finding URLs in page text | +| `js` | The bridge to `pdfcraft-js`. `forms` never depends on `js` directly; the engine injects it, so a build without JavaScript is possible | +| `xfa`, `ocr`, `compare`, `export` | Façades over the L3/L4 crates of the same names | + +## Not done here + +- **Rendering is still borrowed.** `pdfcraft-render` wraps `hayro` and inspects through `lopdf` + (ADR-0004). The engine only sees `inspect() -> DocInfo`, `RenderPool` and `text::PageText`, and + that API stays frozen while M2 replaces what is behind it. +- Performance budgets (`misc.performance-budgets`) are not set or enforced. +- Some shipped features are still reachable only from the UI, not through an `automation` tool; + `cargo xtask parity` lists them; closing that gap is tracked in the execution plan (D4).