Review of #3924: in a flat area of ` <<(` (CMYK or RGBA 32, 60, 60, 40) in an uncompressed image, each `<<` after white space was read as a dictionary, and each read its string, which never ends, to 64 KiB past it: about 16 K bytes a pixel, so a few hundred bytes of it spent the budget and the file was refused.
Whether a byte in a literal string is escaped doesn't depend on where reading began (the run of `\` before it lies after any `(` before it), so the string from any `(` ends after the first `)` that takes the file-wide count of `(` less `)` below the count at that `(`. Those ends are found for the whole file in three linear passes the first time a string read in skipped text runs past 64 bytes: forward for the final count, back for the `(` whose string never ends, forward pairing the rest. Ends further than 64 bytes are kept (at most one for every 16 bytes of the file; a file built to nest past that is refused); nearer ones are read again. So entering a string costs at most 64 bytes, charged as one. A comment's end of line is kept for every comment read, not only the last.
Review of #3924: every `<<` in a string, comment or stream's data was read as a dictionary of its own, and in a run of `<` (an uncompressed image's flat area of grey 60) every position is one, each read to past the run, so about N²/2 bytes for N `<` and a 1 MiB run refused the file.
The reader starts a dictionary there only where its repair scan tries one (after white space or a dictionary's `>>`) or after an object's `N G obj`: so only those `<<` are read, in a run of `<` only the first. A reading goes on as far as 64 KiB past the last `<<` opened on the way, looks at each nested dictionary's `>>` for a stream and marks each `<<`, so it is not read again; a comment's end of line is kept, as many readings cross the same comment; the `<<<` second-`<` reading in the main pass goes (the reader never starts there). The dictionary text a stream check parses is charged to the budget.
Review of #3919: the scan raised its walk-back floor at every `>>stream`, so one inside a literal string moved the floor past the `<<` of the cross-reference stream around it, and that stream was never looked at. The backward walk also counted a `>>` inside a string, and missed a comment between `>>` and `stream`.
The scan now reads the file forward once, as the reader does: literal strings (nested, escaped), hex strings and comments are skipped, `<<` and `>>` pair on a stack, and a dictionary followed by `stream` is a stream's, whose data is skipped. Because the reader can still start at a `<<` inside skipped text (a cross-reference offset or its repair scan), each such `<<` is read as a dictionary of its own, up to 64 KiB, within a budget of four times the file plus 1 MiB; a crafted file that spends it is refused. The scan stays linear.
Each `stream` keyword walked back up to 64 KiB through every dictionary before it, so 1 MiB of `<</A <</B 1>> >>stream` took minutes. A walk back now stops where the previous keyword's walk began, and a stream without a Length shares the search for the next `endstream`.
hayro (0.7.2 and 0.8.0 alike) inflates Flate and LZW data with no ceiling, so a few megabytes
of zeros ask for gigabytes, and applies a predictor before checking its parameters: Columns 0
divides by zero (a panic, an abort on the web), a huge Columns allocates a row that large (an
abort everywhere). Both happen inside the reader, before any of our limits.
A file is refused ("a stream in it would decode to more than 256 MB, or its decode parameters
can't be applied") when a stream's decode parameters ask for a predictor the reader can't apply,
or when it decodes past 256 MB through Flate, LZW, run-length and the ASCII encodings before
them, decoded here with a ceiling (only streams large enough to get there are decoded). Every
object's stream is looked at once the file is open (decrypted, references followed), with the
inline images of anything that can be content; object and cross-reference streams, which the
reader decodes while it opens the file, are looked at first in the file's text.
VoiceLessQ