mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-11-03 06:51:50 -06:00
Grzegorz MartensandClaude Opus 5.5
85c1005148
Each JBIG2 bitmap was bounded on its own, but a few kilobytes could still exhaust memory: nothing bounded what a page's segments hold for later ones, or how far one segment could multiply them. Measured on black-box streams, peak memory before and after: - intermediate regions, 16 in 774 bytes: 514 MiB, now 98 MiB; - an 8 MiB symbol exported 1,200 times, a 2,989-byte PDF: killed at 8 GiB, now 79 MB; - dictionaries of zero-width symbols: 781 MB, now 72 MB; - dictionaries retaining their coding contexts, 1.1 MB: 2.89 GB, now 71 MB; - 65,535 references to one dictionary, 78 KB: about 17 GB, now 6 MB; - a halftone grid of 16,384 cells a side, 98 bytes: 1.1 GB, now 2.4 MB; - a pattern dictionary of 65,535 patterns, 48 bytes: 539 MB, now 15 MB, and it still draws; - a custom Huffman table behind nested Flate, a 1,297-byte PDF: killed at 8 GiB, now 278 MB, which is mostly the decoded stream; - 11-byte segments behind nested Flate in a stream and its globals, a 4,765-byte PDF: 5 GB, now 602 MB, which is mostly the two decoded streams; - an Indexed image whose page header claims 2^20 a side, a 743-byte PDF: aborted asking for 137 GB, now 23 MB; - a chain of 4,000 /JBIG2Globals streams that are JBIG2 themselves, on a 2 MiB stack: overflowed it and aborted, now 24 MB. So PageState counts what it stores: regions, pattern and symbol dictionaries and Huffman tables, with bitmap headers, retained contexts and a charge per item, up to 64 MiB, two square bitmaps of 16,384 a side. A dictionary's new and exported symbols count against what is left as they are made, not once it is stored. A segment may refer to at most 2^20 symbols, checked before any are collected. A halftone grid has at most 2^25 cells. A pattern dictionary keeps at most 16 MiB of shifted copies, which only speed up drawing, and none that wouldn't fit what the page has left. A custom Huffman table has at most 2^16 lines, and a stream with its globals at most 2^20 segments and 2^22 references, checked as they are read. A page past the bounds on a bitmap is refused when the stream is opened, since callers size their output from its header before the page bitmap is made. And in hayro-syntax, a /JBIG2Globals stream that is itself JBIG2 isn't decoded: globals are raw segments, so it reads as globals that can't be decoded. An image past a bound isn't drawn, as with the other caps, and the rest of the page is. This bounds memory, not time: decoding work has no total yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY