cyfung1031andClaude Haiku 5.5 dd046a25b2 M2: stream page content past the 256 MiB cap (item 1 of storytold/pdfcraft#623) (#797) ...

* M2: stream page content past the 256 MiB cap (item 1 of storytold/pdfcraft#623)

Page content was decoded whole, and the decode stopped at
MAX_DECODED_STREAM (256 MiB), so the last ~10 MB of the #623 sample
was dropped without a warning. Top-level page content is now read a
window at a time, so the decoded content is never all in memory:

- the content tokenizer reads windows and holds back an instruction
  that runs past one; Reader records reads that touch the end of the
  data, including reads made through clones, so an instruction is
  only decided once it has been seen whole;
- the Flate decoder is resumable and keeps its own 32 KiB history, so
  it pauses at any output size and back-references do not depend on
  what the caller has kept;
- page content may decode to max(256 MiB, 64 x its compressed size),
  counting each distinct stream once; content that reaches that
  budget is cut and the page reports ContentTruncated, as it did not
  before;
- forms, tiling patterns, Type 3 glyph procedures and generic streams
  keep their existing budgets and caps.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>

* M2: Avoid duplicate page content raw reads

* M2: Document page content byte reuse

---------

Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>

2026-10-11 12:17:09 -04:00

..

src

M2: stream page content past the 256 MiB cap (item 1 of storytold/pdfcraft#623) (#797)

2026-10-11 12:17:09 -04:00

Cargo.toml

M2: bound page-label traversal and output (#480)

2026-10-09 14:12:16 -04:00