A region's location, a refinement's offset, a symbol's corner, a Huffman
range line and a decoded integer of 2^31 could each overflow: a panic
wherever overflow is checked. The last four wrap, as a release build did.
Bitmap::combine saturates, which comes to the same: a side is at most
65,535, so a location that overflows is past the end of any page. What is
drawn doesn't change. Patch (12), after the numbers #5491 and #5569 take.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
`IndirectObject<f32>` read an object holding `6 0 R` as the number 6, so `/Decode [11 0 R 0]`, with
11 naming 6 and 6 holding 1, read as 6, and `/W` gave glyphs such widths. As anything but a number,
such an object was lost.
`XRef::get_with` is how every reference is read, to an object stored on its own or in an object
stream, so it now follows such objects in a loop, up to 32 references as `Document::resolve` does in
#5548. A loop takes no stack however long the chain, and a cycle runs out of references and reads as
a missing object, as a reference to nothing does: the image with such a `/Decode` isn't drawn, as
with `/Decode [99 0 R 0]`.
An object stream isn't reached that way: it has to be a stream stored as that object, and following
a reference to one stored inside another let a 1 KB file run out of stack. `XRef::objects` leaves
such objects out, or a page came out once more for each one holding a reference to it, and a comment
inside a reference counts as white space, as it does everywhere else.
Reading an object stream's index now counts against the 64 nested loads, and reads its dictionary
alone. That is older than this branch, which only made it slightly worse: a chain of object streams,
each with a member of the next for its `/N`, `/First` or `/Length`, recursed with no limit, holding a
copy of the chain at each, and aborted on 750 KB with an 8 MiB stack, or 250 KB with 2 MiB, in a test
build. Parsing the stream read `/Length` as well as `/N`, two ways to the next stream's index, and a
failed index isn't remembered, so the work doubled at each stream: 20 streams in a ring took 20 s on
`main`. ISO 32000-2 7.5.7 keeps an object stream's `/Length` out of object streams, so no legal file
has a `/Length` chain. Nothing forbids `/N` or `/First` by reference, though no writer I know of makes
such a chain. Past the limit it reads as missing unless a stream within 64 of it has been read.
`/W` and `/W2` with their list by reference are tested too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Colour spaces already had tests of this through their bases. `/Decode`, in the renderer and in image
export, `/W`, `/W2` and an Indexed space's table hadn't: each read now has such a reference in its
test, and ends.
These hold that the read ends, not what value it comes to. A reference to a reference in a number
array reads as the first number of that reference in the renderer (`/Decode [11 0 R 0]`, with 11
naming 6 and 6 holding 1, reads as 6). That is older than this branch, in `IndirectObject::read` for
an object stored on its own and `ObjectStream::get` for one in an object stream.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
The depth limit didn't hold. An ICC profile read was kept whole, with the alternate it fell back
to, and a later read took it without counting its levels. A page selecting the last of a chain
first built each space on the one before, and no read was deep: 20,000 levels, and the stack
overflowed painting with it. In the other direction, a profile met at the limit kept the fallback
it got there, and a read nearer the top got that instead of its alternate.
Both come from keeping a result that depends on the depth. What's kept now is what the profile
itself reads as, one level whatever it is, and the alternate is read where it's met.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Reading an object already being read is refused, which ends a cycle but not a chain. 5,000 spaces
each naming the next, 180 KB of file, overflowed a 2 MiB stack and aborted the process, through a
Pattern's underlying space, an Indexed base, or a Separation's or DeviceN's alternate. An ICC
profile's alternate is the same recursion. Indexed, Separation and DeviceN could already do this;
the fix for Pattern added a way in.
A legal file goes five levels at most. All five ways through count against the one limit, in
ColorSpaceType::new, so what comes in from outside is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Rows are charged by the symbol, so an empty class decoded its collective bitmap for nothing: 65,535
rows a class, a thousand classes a dictionary. 806 KB took 196 s. It steps over the bytes instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From a review of the work count. An item with no pixels was charged a step, though reading it can take
hundreds of decisions, so a 22,515-byte PDF of refined text instances of no size stayed inside the
limit and took 730 s to render. Empty height classes and export runs weren't charged at all, and 19 KB
of them took 47 s. Each integer an item reads now costs what one takes at worst, 38, so a decision
never costs more than a pixel. A full region of 10,000 instances is 0.14% of the limit, so a real file
doesn't notice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From CodeRabbit on the work count.
A symbol with no rows cost no work, though it is decoded all the same, so a stream of thousands of
dictionaries of such symbols, exporting none, went uncounted. Each symbol now costs a step more than
its pixels.
A refined text instance was charged after it was decoded, so one could still be 2^28 pixels past what
was left before the decoder refused. It's charged once its size is read, before anything is allocated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
The Pattern fix opened a recursion path: a Pattern's underlying space is now followed, as an Indexed
space's base always was. Self-references and cycles between the two end as a space that can't be
read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Since the renderer now draws an image with /Decode [7 0 R 0], export disagreed with it. Its check of
which way round a stencil mask or a CMYK JPEG is compared the two numbers as given, and a reference
reads as none, so the file came out a negative of what the page draws.
Tests for the function arrays the previous commit's tuple fix also covers: a function's /Domain,
/Range, /Encode and /Decode are read a pair at a time too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Two more ways a reference was lost in the same readers. A Pattern space read its family name twice,
which took the underlying space when that was a name or a reference, so an uncoloured pattern took
its components as RGB: [/Pattern /DeviceCMYK] drew a cyan as red, direct names included (patch 72).
Tuples of array items, which /Decode, /W and /W2 are read as, took a reference at their start for the
whole tuple. A tuple can't be given by reference, so it was lost and the array ended there: an image
with /Decode [7 0 R 0] wasn't drawn, and CID widths fell back to /DW. A tuple now reads its items
directly, and each follows its own reference (hayro-syntax patch 32).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
A colour space [/Indexed /DeviceRGB 1 5 0 R] whose table is a string in object 5 couldn't be read:
the table was tried as a stream first, and a failed read uses up a reference, though not a direct
object, so nothing was left to read it as a string. The space then drew in grey from the index, so
a palette's colours were lost. The table is now read once, as whichever it is (hayro-interpret
patch 71).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Each JBIG2 bitmap is bounded, but nothing bounded how many a stream asks for, so a few hundred
bytes could tie up a render worker for minutes, and CodeRabbit raised the same on #5461. Measured
through pdfcraft-cli, before and after:
- one halftone segment, a 795-byte PDF: 265 s, now refused at once;
- 66 halftones over patterns 1 wide, a 3,894-byte PDF: about 330 s, now refused at once;
- about 226,000 generic regions behind nested Flate, a 923-byte PDF: unfinished at 40 s, some
88 hours by extrapolation, now 4.8 s;
- 8 dictionaries of symbols of no width, a 983-byte PDF: 81 s, now 0.7 s;
- about 1,048,000 generic regions of no width, a 34 KB PDF: unfinished at 90 s, now 2.6 s;
- an aggregated dictionary over 2^20 symbols, a 713-byte PDF: unfinished at 90 s, now 9.1 s;
- 10,000 instances of an 8192 × 8192 symbol, an 18 KB PDF: 64 s, now 3.2 s;
- 30 generic regions of no data: 48 s, now 4.7 s.
A stream may now ask for 2^30 units of work: four of the largest pages, about 6 s on data the
decoder predicts and about 17 s on data it can't, inside the viewer's 20 s watchdog. Decoding a
bitmap costs a unit a pixel and 4 a row, since a row costs time at any width. Copying one costs a
unit a word and 1 a row, so a tall thin bitmap costs its rows. Each item is charged before it is
decoded, apart from what is known only as it goes: new symbols and text instances are charged in
their loops. An aggregated symbol is charged for the symbol table it rebuilds. Past the bound the
image isn't drawn, as with the other caps, and the rest of the page is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Each JBIG2 bitmap was bounded on its own, but a few kilobytes could still exhaust memory:
nothing bounded what a page's segments hold for later ones, or how far one segment could
multiply them. Measured on black-box streams, peak memory before and after:
- intermediate regions, 16 in 774 bytes: 514 MiB, now 98 MiB;
- an 8 MiB symbol exported 1,200 times, a 2,989-byte PDF: killed at 8 GiB, now 79 MB;
- dictionaries of zero-width symbols: 781 MB, now 72 MB;
- dictionaries retaining their coding contexts, 1.1 MB: 2.89 GB, now 71 MB;
- 65,535 references to one dictionary, 78 KB: about 17 GB, now 6 MB;
- a halftone grid of 16,384 cells a side, 98 bytes: 1.1 GB, now 2.4 MB;
- a pattern dictionary of 65,535 patterns, 48 bytes: 539 MB, now 15 MB, and it still draws;
- a custom Huffman table behind nested Flate, a 1,297-byte PDF: killed at 8 GiB, now 278 MB,
which is mostly the decoded stream;
- 11-byte segments behind nested Flate in a stream and its globals, a 4,765-byte PDF: 5 GB, now
602 MB, which is mostly the two decoded streams;
- an Indexed image whose page header claims 2^20 a side, a 743-byte PDF: aborted asking for
137 GB, now 23 MB;
- a chain of 4,000 /JBIG2Globals streams that are JBIG2 themselves, on a 2 MiB stack: overflowed
it and aborted, now 24 MB.
So PageState counts what it stores: regions, pattern and symbol dictionaries and Huffman
tables, with bitmap headers, retained contexts and a charge per item, up to 64 MiB, two square
bitmaps of 16,384 a side. A dictionary's new and exported symbols count against what is left as they are
made, not once it is stored. A segment may refer to at most 2^20 symbols, checked before any are
collected. A halftone grid has at most 2^25 cells. A pattern dictionary keeps at most 16 MiB of
shifted copies, which only speed up drawing, and none that wouldn't fit what the page has left. A custom Huffman table has at most 2^16 lines, and a
stream with its globals at most 2^20 segments and 2^22 references, checked as they are read. A
page past the bounds on a bitmap is refused when the stream is opened, since callers size their
output from its header before the page bitmap is made. And in hayro-syntax, a /JBIG2Globals
stream that is itself JBIG2 isn't decoded: globals are raw segments, so it reads as globals that
can't be decoded.
An image past a bound isn't drawn, as with the other caps, and the rest of the page is. This
bounds memory, not time: decoding work has no total yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Review is right that `Stack::push` always returned `Some(())`, so `overflowed`
never became true and the whole-instruction discard was dead code.
Making it live breaks the page. `an_operator_with_hundreds_of_operands_does_
not_end_the_content` (#4260) puts 300 stray operands before `0 1 0 rg` and
asserts the `rg` still reads its own three: with the discard alive the fill
stays the previous red, measured as `left: [255, 0, 0]` against
`right: [0, 255, 0]`.
The ring buffer already gives the right answer. It keeps the newest operands,
which are the ones nearest the operator -- its own -- and no operator takes
anywhere near 256 of them: `scn` takes one per colourant and ISO 32000 allows a
DeviceN space 32. So the strays fall off the back and the operator is applied to
its own operands, which is what Acrobat and Chromium do. Dropping the
instruction would lose content that is there to be read.
The dead branch and the flag are gone, `push` no longer returns a result it
cannot produce, and a test pins what the buffer does: the strays are `1` and the
operator's own operand is `0`, so the two answers differ. Keeping the oldest
operands instead fails it at 256 strays.
From the review of #5228. The finished CMYK group was drawn through
whatever paint transform a preceding shading pattern left on the
context, so its cyan half showed magenta. Hayro patch (27) now resets
the paint transform before drawing the group's pixels.
Two regression tests, both reported in that review:
- a pattern painted before the group (fails without the reset);
- the Color blend mode over an empty CMYK group, swept over all 256 K
levels (failed on nine levels before #5199's vello_cpu fix was merged
in; passes with it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
main's knockout patch (hayro (22)) draws each element of a knockout group
once in erase mode, to clear what the group drew under the element's shape.
An isolated DeviceCMYK group nested in such an element still took the CMYK
path in that pass, and drew its finished pixels with Normal blending over
what should have been cleared. draw_isolated_cmyk_group now declines while
erasing, so the group's contents are interpreted in erase mode as before.
The new test fails without the fix (the centre shows red through the
magenta: [240, 63, 168] against [245, 127, 197] for the DeviceRGB control).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
main has taken hayro-interpret patches up to (66) and hayro up to (25)
since #3777 was opened, and open PRs claim hayro-interpret (67)-(69) and
hayro (26). The isolated DeviceCMYK group patches become hayro-interpret
(70) and hayro (27). vendor/README.md names the tests by their new path,
raster::tests::isolated_cmyk_groups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Review of #3777 (test report by mtlcl410).
- A group wider than about 512 device pixels lost its blend mode in the
tiles that a paint covers completely: on the bare pass canvas, vello_cpu
fills such a tile without the blend mode. At 300 dpi, magenta then black
with Difference drew black in the middle of the group, not green. Each
ink pass now draws inside a layer of its own.
- A nested isolated group that names another blending space (for example
/CS /DeviceRGB) blends in that space (ISO 32000-2 §11.3.4). The
interpreter now tells the device about such a group
(Device::group_blends_outside_device_cmyk), and a pass that meets one
declines, so the enclosing group is drawn as before. A non-isolated
group's /CS is ignored (ISO 32000-2 §11.6.6, Table 145, CS: it takes its parent's space), so
it stays in the enclosing group's CMYK.
Test: raster::tests::isolated_cmyk_groups_blend_at_any_size_and_respect_nested_spaces.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ISO 32000-2 §11.3.4: the elements of a transparency group blend in the
group's colour space. For a subtractive space, each component is
complemented before the blend function and again after it. §11.3.5.3:
the nonseparable modes blend C, M and Y as their complements R, G and B,
and K comes from the backdrop (Hue, Saturation, Color) or from the source
(Luminosity). PdfCraft converted each colour to RGB and blended in RGB.
A print test file's blend-mode patch then showed an X in almost every
cell: magenta under black with Difference stayed magenta, not green.
hayro-interpret patch (45): CmykGroup and Device::draw_isolated_cmyk_group
(default false) for isolated, non-knockout groups with /CS /DeviceCMYK.
interpret_form_contents is now a function of its own. Color::ink_tints
and color::cmyk_to_srgb8 are new.
hayro patch (21): the renderer draws such a group as a CMY pass and a K
pass of complemented tints. It converts the tints to RGB with the CMYK
profile and composites the result through the group's layer. The group
is drawn as before with reading colours, for contents without process
tints (RGB, spot colours, patterns, images) and above 2^24 pixels.
Test: raster::tests::isolated_groups_blend_in_device_cmyk.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit's message and this header said the Luminosity ramp
passes because pages use the u8 pipeline. That wasn't established; the
case simply renders correctly here, so the test is a guard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
vello_cpu 0.0.8's clip_color, shared by the nonseparable blend modes,
divided 0 by 0 when a gray's equal components were rounded just below 0
or above 1 after set_lum. The NaN drew the pixel black at the source's
alpha whatever the backdrop: nine gray levels under /BM /Color in an
empty isolated group (0.96 g among them) rendered (0, 0, 0), and two
more at half opacity came out wrong.
Vendor vello_cpu 0.0.8 (Apache-2.0 OR MIT) with a backport of
linebender/vello#2000, which clips only when the denominator is
positive. No vello_cpu release carries the fix yet; drop the copy when
one does.
Tests in it::nonseparable_gray run every gray level. The two Color
ramps fail with the unpatched blend.rs. The Luminosity ramp (black over
each opaque gray, upstream's other case) passes either way here: pages
use the u8 pipeline, and that case only failed in the f32 one; it stays
as a guard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
A wrapper without a `CFF ` table must draw exactly what the same font
with no embedded program draws, and not the embedded program. The old
check (some ink on the page) also passed if the program were read
despite the missing table; with the lookup changed to take the first
table whatever its tag, the new assertion fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
A Type1 font whose FontFile3 has subtype /OpenType (ISO 32000-2 §9.9)
keeps its CFF program in the `CFF ` table of an OpenType wrapper.
CffFontBlob::new rejected the wrapper, so the font fell back to the
standard-font stand-in and the text drew in the wrong face and weight.
The CFF table is now taken from the wrapper, as composite fonts already
do (hayro-interpret patch (69)).
The test wraps the renderer's bundled CFF program in a one-table OTTO
directory at run time (no font file added) and requires the wrapped and
bare programs to give identical pixels; it fails with the fix reverted.
A wrapper without a `CFF ` table still draws, with the stand-in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
A malformed token ended the content stream where it stood, so the renderer,
text extraction, Find and search-based redaction all lost the rest of the
page -- while the edit reader, MuPDF and pdftotext read it (#3565, #3568). A
redaction that cannot see the text below the bad token cannot remove it,
which is the part that matters.
Each malformed shape is stepped over instead: an unterminated string, a
number that is only a sign or a dot, a name with a bad hex escape, an
unbalanced dictionary or array. Content after an inline image whose data
holds `EI` is still read, under a bounded lookahead.
Rebased onto main, 1,898 commits on: the tokenizer is still
vendor/hayro-syntax/src/content/mod.rs and applied as it was, and render's
raster.rs is split, so the two tests went to render/raster/tests.rs.
Review of #4898 (CodeRabbit): a shading clearing in another blend-mode
pass cleared its whole drawing path, so a small mesh triangle erased the
earlier elements of its pass around it. The same happened in the
shading's own pass, and in any knockout group, because a knockout paint
is copied over the path and a mesh texture is transparent outside its
triangles.
A shading in a knockout group now goes through knockout_shading: the
shading at full opacity, composited with Compose::DestOut, clears its
own shape (mesh triangles, /BBox), then in its own pass it is painted
with SrcOver. A shading that paints nothing still clears nothing. The
clear uses the shading without the fill opacity it carries, so a
half-transparent shading still replaces all of what is under it.
Test: it::knockout_backdrop::a_small_mesh_replaces_only_what_is_under_its_triangle
(own pass, another pass, isolated group). Revert-proven per half; keeping
the opacity in the clear fails
raster::tests::knockout_groups_replace_what_their_elements_cover.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Review of #4898 (FredApps):
- An image or nested group of another blend mode than the pass's composited
its layer at opacity 0, on the assumption that its shape was cleared first.
`knockout_image` skips images with an `/SMask` and stencil masks, so a
white Difference square stayed under a blue Normal masked image in the
Difference pass and turned it yellow. The layer now composites with
`Compose::DestOut` at full opacity and no soft mask, clearing exactly its
coverage.
- In another pass (and an erase pass), a shading cleared under its path
before the mesh-extent check that makes an off-page mesh paint nothing,
so a Difference `sh` of such a mesh erased the blue under it. The clear
now makes the same decision (`shading_bbox`, `mesh_area`, shared with the
paint path).
Tests: `it::knockout_backdrop` gains a masked image, a stencil and an
off-page mesh case; each fails with its half of the fix reverted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
kurbo 0.13.1's stroker can give a cubic that is nearly a line (all control points within a
rounding error of its chord) a NaN join, and vello_common's `flatten::fill` drops any path
containing NaN, so the whole outline vanished. An Illustrator outline ending in
`73.763 -0.536 l 73.764 -0.536 73.764 -0.536 73.764 -0.536 c` did (#4907). kurbo tracks the
bug class in linebender/kurbo#617 and #519; #610 is unreleased.
Vendored vello_common patch (2): when the expanded stroke is not finite, `flatten::stroke`
strokes the path again with every quadratic or cubic whose control points lie within the
tolerance of its chord replaced by a line, one element out per element in. A finite outline
is unchanged, and a path with non-finite input is still dropped. `stroke` and
`StripGenerator::generate_stroked_path` take a `Clone` path; vello_cpu's callers pass
`&BezPath` and a copied slice iterator.
Tests: `flatten::tests::a_tiny_degenerate_cubic_does_not_drop_the_stroke` and the render
crate's `stroke_degenerate_cubic` pixel test fail with the fallback disabled;
`a_finite_outline_is_stroked_as_before` and `a_path_with_nan_is_still_dropped` are guards.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
