A stream whose /Length is wrong or missing is ended at the next
endstream, which the parser found by searching the rest of the file.
Many such streams before one late endstream (a damaged file being
reconstructed reads every one) cost the file size per stream. The
document lists where endstream occurs the first time it needs it, and
each stream looks its end up there.
To read the longest valid prefix of a token like 1.2.3 the parser
retried the conversion on each shorter prefix, so a long digit run
followed by long junk was converted once per junk byte: quadratic in
the token. The prefix is found in one scan and converted once. A
prefix that spells infinity or NaN no longer reads as a number.
The parser pushed every array element and dictionary value with only a
nesting limit, so a small compressed object stream decoding to a long run
of '0 ' became tens of millions of Object values. Each top-level parse now
counts the objects it builds and fails past 4M, including inside
tolerated junk.
* fix(parse): accept spaces between stream and its line break
Microsoft Reporting Services writes `stream \r\n`, a space before the line break.
hayro-syntax and pdfcraft-cos took the stream data to start at the space, so the
content stream failed to inflate: the page rendered blank (white, or black with
inverted colours) and an edited save corrupted the stream. Spaces and tabs are now
skipped, but only when a line break follows.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Number the stream end-of-line patch (20), as vendor/README does
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The PDF/A device-colour scan split content streams only on spaces and
line feeds, so `0 0 0 1 k` written with carriage returns, tabs, form
feeds or NULs between the tokens went unseen: verify reported no
DeviceCMYK, and convert gave the CMYK page an sRGB output intent.
Split on all six whitespace characters of ISO 32000-2 §7.2.3, using the
lexer's own `is_whitespace` (now exported from pdfcraft-cos) instead of
a second list.
Adds tests::device_colour_is_found_after_any_whitespace, which fails
before this change; its gray and RGB controls check that those pages
still get the sRGB output intent.
* M1: stream data read from a file points into the file instead of being copied
Every stream COS loaded was copied out of the document's bytes
(`data[start..end].to_vec()`) and kept in the object cache until the document
closed. Anything that reads every object therefore left a second copy of all
stream data behind. That includes a full save, which edits needing a rewrite
(repaired files, security changes, redaction) run on every change. A full save
of a 208 MB document left 208 MB cached on it.
`Stream::raw` is now `Bytes`: owned bytes, or a range of a larger shared
buffer. Streams parsed from a document's own bytes are ranges of that buffer,
which the document keeps anyway. Decrypted and newly built streams own their
bytes as before. A stream added to a document from another document's file gets
its own copy of just its bytes (`Document::set`/`add`), so it never keeps
another whole file alive.
Results, measured with an exact heap counter:
- A full save of a 208 MB document leaves 0.2 MB cached instead of 208.2 MB.
- The first full save of a freshly opened document is about twice as fast:
10 → 5 ms at 40 MB, 38 → 21 ms at 200 MB.
Output is identical: 68 incremental and full saves of five originals,
encrypted ones included, hash the same before and after.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M1: the full-save capacity test (#523) builds its stream from Bytes
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Dylan M. Taylor