Files
pdfcraft/crates/cos
Nick Nyssen 2ec501fa74 test(cos): property tests for save fidelity, plus credit entry
Adds crates/cos/tests/proptests.rs (464 lines) - three properties over generated catalogs (edit sequence
vs a plain ordered model; full save in both output styles; incremental save over bases built BOTH ways) -
and a self-credit entry in contributors/people.toml committed under the author's own identity. No product
code changes; total +466 lines across two files: people.toml +2; proptests.rs +464.

Two design decisions that came out of adversarial review, each measured rather than asserted:
  * catalog keys insert in a scrambled order ([5,0,7,2,6,1,4,3] filtered to the drawn keys), because with
    ascending insertion "preserves key order" and "sorted by key" are the same observable.
  * incremental expectations derive from the values written, not from reading the writer's own output back,
    so a consistently-wrong implementation cannot pass by agreeing with itself.

Measured against one-line breaks in crates/cos/src/writer.rs, each reverted and byte-compared afterwards
(verdicts from cargo exit codes; "pre-existing" = lib 48 + writer 10 + encryption 8 + shared_streams 3 +
bounded_writer 4, i.e. every target that existed before this PR):

  sort every dictionary by key              CAUGHT (props 1+2)      pre-existing suite FULLY GREEN
  reverse dictionary emission              CAUGHT (all three props)   pre-existing suite FULLY GREEN
  move /Type to the end of every dict     CAUGHT (prop 1)           pre-existing suite FULLY GREEN
                                            - caught only after catalog_keys() was added to pin the full key
                                              sequence; without it this break was green everywhere
  classic-table incremental startxref +1   CAUGHT (prop 2 alone; props 1+3 green), failure message names
                                            "classic-table", confirming the incremental table path executes
  drop null-valued dictionary entries      CAUGHT (all three)        at the pre-fix commit b51381c this same
                                            break left property 2 green - the self-reference it removed

Known limits, stated not hidden: incremental saves are asserted to add bytes and leave the original prefix
untouched - minimality is NOT claimed; catalog key order pins one fixed scramble rather than drawn
permutations, so 25.4% of cases draw too few keys for order to be meaningful; nested dictionary key order
is deliberately not pinned (canon sorts inside values); /Perms and permissions semantics are out of scope
here.

Gates at this tip (upstream 24f93e6, drift 0): cargo test -p pdfcraft-cos 85 passed / 0 failed / 2
ignored; cargo fmt -p pdfcraft-cos --check clean; clippy exit 0 for this crate with
-D warnings -A clippy::chunks_exact_to_as_chunks (that lint fires only in upstream's own
crates/oracle/src/pixels.rs:189 from f988cf8, tracked by #2342/#2324; flags after `--` forward into every
crate in the build graph, and crates/cos itself emits zero warnings under a forced recompile).
cargo xtask layers OK: 37 crates.

Scope: part of #1187 (items 1-3 partly; the reference-following comparison, undo state machine, password
property and damaged-file splices remain open). The encryption-side properties are PR #2358 by the same
author; earlier drafts of this text mis-cited #2014, which is certificate encryption and still open.
2026-10-15 16:41:32 -07:00
..

pdfcraft-cos

The PDF object layer (ISO 32000-2 §7): the COS object graph, parsed lazily and tolerantly, edited through a copy-on-write overlay, and written back incrementally or as a full rewrite.

  • Layer: L1, standalone. The layering table lets it use only pdfcraft-filters, pdfcraft-crypt and pdfcraft-geom (xtask/src/layers.rs::STANDALONE_DEPS); it currently uses the first two. That keeps the parser and the crypto independently auditable and reusable.
  • #![forbid(unsafe_code)], and the crate denies unwrap/expect/panic!/todo!. Builds for wasm32-unknown-unknown.

The object graph is the document model. Nothing is lifted into a tidier representation and written back out, because that is how data gets silently dropped. Typed views live above, in pdfcraft-model.

API

let doc = Document::open(bytes)?;                               // Arc<Vec<u8>>
let doc = Document::open_with_password(bytes, Some("owner"))?;  // encrypted files

let root = doc.root();                               // Option<ObjRef> — the catalog
let obj  = doc.get(ObjRef { num: 12, gen: 0 });      // load (and cache): Arc<Object>
let real = doc.resolve(&obj);                        // follow one indirect reference
let d    = doc.dict(&obj);                           // Option<Dict>, resolving first
let data = stream.decoded()?;                        // filters applied on demand

doc.set(r, Object::Dict(d));                         // edit: lands in the overlay
let r = doc.add(obj);                                // append a new object
doc.free(r);

let bytes = write_incremental(&doc, &SaveOptions::default())?;  // append a revision
let bytes = write_full(&doc, &SaveOptions::default())?;         // garbage-collected rewrite

Both writers return the bytes; the caller decides where they go, which is what lets the engine produce a working file in memory after every edit and keep saving atomic.

Useful alongside: doc.repair_log() (what was reconstructed), doc.revisions() and doc.revision_ends() (the incremental history), doc.is_edited(num) and doc.modified_objects(), doc.permissions() and doc.security(), doc.set_encryption() / doc.remove_encryption(), and doc.require_full_save() for edits that cannot be expressed incrementally.

Document is cheap to clone: the original bytes, the xref index and the parse caches are shared, so a clone is a snapshot. Undo/redo and background jobs hold clones while the UI edits another. Edits never touch the original bytes.

Reading

  • Every cross-reference form: classic tables with their quirky subsections, cross-reference streams, hybrid-reference files (/XRefStm), and object streams.
  • Repair. A file whose xref is wrong or missing is reconstructed by scanning for obj headers. Repairs are recorded, never silent, so the UI can say the file was damaged.
  • Tolerant, never fatal. The lexer works on a byte slice and a position; it allocates nothing for skipped data, bounds-checks every index, and caps nesting depth. Malformed input yields CosError, never a panic — this crate is the first thing a hostile file meets. Exhausted xref entries are skipped with a repair log; exhausted /Size hints allocate unused numbers without overflowing or replacing existing objects.
  • Lazy. Objects are parsed on first use and cached. (Whole-file buffering is still the case; ByteSource streaming is core.lazy-loading, not yet done.)
  • Encryption is handled on load and save through pdfcraft-crypt: every standard-security revision R2–R6 (RC4, AES-128/256), crypt filters, /EFF embedded-file encryption, and /EncryptMetadata. security.rs knows what the spec exempts — the /Encrypt dictionary, xref streams, the trailer /ID, and objects inside an object stream (the stream is encrypted whole).

Writing

write_incremental appends only the edited objects, a cross-reference section in the same style as the file's last one, and a trailer with /Prev. The original bytes are never modified, which is what keeps a signature over an earlier revision valid.

write_full writes only what the trailer can reach, renumbered densely — the Save As and garbage-collecting path. By default (SaveOptions::object_streams) it packs non-stream objects into compressed object streams with a cross-reference stream (PDF 1.5+); turning that off writes a classic table any reader accepts.

Dictionaries keep their key order and streams keep their encoded bytes, so rewritten objects stay recognisable in a diff and byte-stable when re-serialized.

Errors

CosError: NotPdf, Syntax { offset, detail }, MissingObject, NotADictionary, Filter, NeedsPassword, WrongPassword, Security, Poisoned.

Not done here

  • Streaming very large files within a memory budget (ByteSource, core.lazy-loading); files are read whole today.
  • Linearized output, Fast Web View (core.linearization, M11).
  • Structural validation against the Arlington model (core.arlington-validation), which will live in the reserved arlington crate.
  • Image codecs. pdfcraft-filters carries the non-image filters; DCT, JPX, JBIG2 and CCITT are still decoded through the bootstrap renderer (core.image-filters, M2.6).