What a page has left is shared by all of it, so it says nothing about a stream that isn't JBIG2: a bad
Flate or JPEG mask dropped its picture on a page that had spent its allowance, and a failing image of
another kind raised a false "content skipped". Both ask about the stream now.
The allowance also outlived the page on its thread, and a Context used again kept it. interpret_page
sets it, and a guard ends it with the page.
Found by CodeRabbit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Rows are charged by the symbol, so an empty class decoded its collective bitmap for nothing: 65,535
rows a class, a thousand classes a dictionary. 806 KB took 196 s. It steps over the bytes instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Without its mask a picture shows what the mask hides, so the page leaves it out, as export does. Any
other unreadable mask still leaves the picture, which is the renderer's tolerance for bad masks.
What an export may hold is a function of its own, tested down to the step and at the top of the range.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From a review of who reads a refusal.
The rule is to leave it out and say so, and it held on some paths only. An image whose JBIG2 mask is
refused no longer comes out opaque. The first image past the allowance reads as that and not as
corrupt. A page reports what it left out. The Office warning stopped blaming forms. Redaction refuses
the run, leaving the document as it was: removing a readable image whole would take what no mark
covers with it, unreported.
The 2 GiB an export holds is twice the document if that is more, so a large honest file loses nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
One image has its own limit, about 6 s to 17 s, but nothing bounded a file of thousands of images
made to reach it: an operation took that long for each, and an image sharing /JBIG2Globals decodes
them again.
Image::decode_within (hayro-jbig2 patch 11) spends from what a run may still do, starting with its
page's pixels, and charge_reading charges reading a stream and its globals, 4 steps a byte, before
anything is parsed, so an image that fails has paid too. pdfcraft_filters::decode_jbig2_within takes
it, and ImageBudget in cos is that allowance: 2^34 steps, about 1,000 scanned pages, and 2^13 more for
each byte of the file, so a long scan is read whole. An image past it spends all that's left, and
later ones aren't read, globals included.
Optimize leaves such an image as it is and counts it in Report::images_refused. Image export leaves it
out, with the reason, and stops at 2 GiB of files held. Office export shares one allowance, leaves the
picture out and says the page was truncated. Redaction removes it whole, as any image it can't read.
Rendering gives a page's images 2^30 steps between them, inside the watchdog, and each page starts
afresh in Context::new as its other budgets do (hayro-syntax patch 31, hayro-interpret patch 73).
It's a thread-local, unbounded until a page sets it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From a review of the work count. An item with no pixels was charged a step, though reading it can take
hundreds of decisions, so a 22,515-byte PDF of refined text instances of no size stayed inside the
limit and took 730 s to render. Empty height classes and export runs weren't charged at all, and 19 KB
of them took 47 s. Each integer an item reads now costs what one takes at worst, 38, so a decision
never costs more than a pixel. A full region of 10,000 instances is 0.14% of the limit, so a real file
doesn't notice.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From CodeRabbit on the work count.
A symbol with no rows cost no work, though it is decoded all the same, so a stream of thousands of
dictionaries of such symbols, exporting none, went uncounted. Each symbol now costs a step more than
its pixels.
A refined text instance was charged after it was decoded, so one could still be 2^28 pixels past what
was left before the decoder refused. It's charged once its size is read, before anything is allocated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Each JBIG2 bitmap is bounded, but nothing bounded how many a stream asks for, so a few hundred
bytes could tie up a render worker for minutes, and CodeRabbit raised the same on #5461. Measured
through pdfcraft-cli, before and after:
- one halftone segment, a 795-byte PDF: 265 s, now refused at once;
- 66 halftones over patterns 1 wide, a 3,894-byte PDF: about 330 s, now refused at once;
- about 226,000 generic regions behind nested Flate, a 923-byte PDF: unfinished at 40 s, some
88 hours by extrapolation, now 4.8 s;
- 8 dictionaries of symbols of no width, a 983-byte PDF: 81 s, now 0.7 s;
- about 1,048,000 generic regions of no width, a 34 KB PDF: unfinished at 90 s, now 2.6 s;
- an aggregated dictionary over 2^20 symbols, a 713-byte PDF: unfinished at 90 s, now 9.1 s;
- 10,000 instances of an 8192 × 8192 symbol, an 18 KB PDF: 64 s, now 3.2 s;
- 30 generic regions of no data: 48 s, now 4.7 s.
A stream may now ask for 2^30 units of work: four of the largest pages, about 6 s on data the
decoder predicts and about 17 s on data it can't, inside the viewer's 20 s watchdog. Decoding a
bitmap costs a unit a pixel and 4 a row, since a row costs time at any width. Copying one costs a
unit a word and 1 a row, so a tall thin bitmap costs its rows. Each item is charged before it is
decoded, apart from what is known only as it goes: new symbols and text instances are charged in
their loops. An aggregated symbol is charged for the symbol table it rebuilds. Past the bound the
image isn't drawn, as with the other caps, and the rest of the page is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Each JBIG2 bitmap was bounded on its own, but a few kilobytes could still exhaust memory:
nothing bounded what a page's segments hold for later ones, or how far one segment could
multiply them. Measured on black-box streams, peak memory before and after:
- intermediate regions, 16 in 774 bytes: 514 MiB, now 98 MiB;
- an 8 MiB symbol exported 1,200 times, a 2,989-byte PDF: killed at 8 GiB, now 79 MB;
- dictionaries of zero-width symbols: 781 MB, now 72 MB;
- dictionaries retaining their coding contexts, 1.1 MB: 2.89 GB, now 71 MB;
- 65,535 references to one dictionary, 78 KB: about 17 GB, now 6 MB;
- a halftone grid of 16,384 cells a side, 98 bytes: 1.1 GB, now 2.4 MB;
- a pattern dictionary of 65,535 patterns, 48 bytes: 539 MB, now 15 MB, and it still draws;
- a custom Huffman table behind nested Flate, a 1,297-byte PDF: killed at 8 GiB, now 278 MB,
which is mostly the decoded stream;
- 11-byte segments behind nested Flate in a stream and its globals, a 4,765-byte PDF: 5 GB, now
602 MB, which is mostly the two decoded streams;
- an Indexed image whose page header claims 2^20 a side, a 743-byte PDF: aborted asking for
137 GB, now 23 MB;
- a chain of 4,000 /JBIG2Globals streams that are JBIG2 themselves, on a 2 MiB stack: overflowed
it and aborted, now 24 MB.
So PageState counts what it stores: regions, pattern and symbol dictionaries and Huffman
tables, with bitmap headers, retained contexts and a charge per item, up to 64 MiB, two square
bitmaps of 16,384 a side. A dictionary's new and exported symbols count against what is left as they are
made, not once it is stored. A segment may refer to at most 2^20 symbols, checked before any are
collected. A halftone grid has at most 2^25 cells. A pattern dictionary keeps at most 16 MiB of
shifted copies, which only speed up drawing, and none that wouldn't fit what the page has left. A custom Huffman table has at most 2^16 lines, and a
stream with its globals at most 2^20 segments and 2^22 references, checked as they are read. A
page past the bounds on a bitmap is refused when the stream is opened, since callers size their
output from its header before the page bitmap is made. And in hayro-syntax, a /JBIG2Globals
stream that is itself JBIG2 isn't decoded: globals are raw segments, so it reads as globals that
can't be decoded.
An image past a bound isn't drawn, as with the other caps, and the rest of the page is. This
bounds memory, not time: decoding work has no total yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
Review is right that `Stack::push` always returned `Some(())`, so `overflowed`
never became true and the whole-instruction discard was dead code.
Making it live breaks the page. `an_operator_with_hundreds_of_operands_does_
not_end_the_content` (#4260) puts 300 stray operands before `0 1 0 rg` and
asserts the `rg` still reads its own three: with the discard alive the fill
stays the previous red, measured as `left: [255, 0, 0]` against
`right: [0, 255, 0]`.
The ring buffer already gives the right answer. It keeps the newest operands,
which are the ones nearest the operator -- its own -- and no operator takes
anywhere near 256 of them: `scn` takes one per colourant and ISO 32000 allows a
DeviceN space 32. So the strays fall off the back and the operator is applied to
its own operands, which is what Acrobat and Chromium do. Dropping the
instruction would lose content that is there to be read.
The dead branch and the flag are gone, `push` no longer returns a result it
cannot produce, and a test pins what the buffer does: the strays are `1` and the
operator's own operand is `0`, so the two answers differ. Keeping the oldest
operands instead fails it at 256 strays.
From the review of #5228. The finished CMYK group was drawn through
whatever paint transform a preceding shading pattern left on the
context, so its cyan half showed magenta. Hayro patch (27) now resets
the paint transform before drawing the group's pixels.
Two regression tests, both reported in that review:
- a pattern painted before the group (fails without the reset);
- the Color blend mode over an empty CMYK group, swept over all 256 K
levels (failed on nine levels before #5199's vello_cpu fix was merged
in; passes with it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
main's knockout patch (hayro (22)) draws each element of a knockout group
once in erase mode, to clear what the group drew under the element's shape.
An isolated DeviceCMYK group nested in such an element still took the CMYK
path in that pass, and drew its finished pixels with Normal blending over
what should have been cleared. draw_isolated_cmyk_group now declines while
erasing, so the group's contents are interpreted in erase mode as before.
The new test fails without the fix (the centre shows red through the
magenta: [240, 63, 168] against [245, 127, 197] for the DeviceRGB control).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
main has taken hayro-interpret patches up to (66) and hayro up to (25)
since #3777 was opened, and open PRs claim hayro-interpret (67)-(69) and
hayro (26). The isolated DeviceCMYK group patches become hayro-interpret
(70) and hayro (27). vendor/README.md names the tests by their new path,
raster::tests::isolated_cmyk_groups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp9
Review of #3777 (test report by mtlcl410).
- A group wider than about 512 device pixels lost its blend mode in the
tiles that a paint covers completely: on the bare pass canvas, vello_cpu
fills such a tile without the blend mode. At 300 dpi, magenta then black
with Difference drew black in the middle of the group, not green. Each
ink pass now draws inside a layer of its own.
- A nested isolated group that names another blending space (for example
/CS /DeviceRGB) blends in that space (ISO 32000-2 §11.3.4). The
interpreter now tells the device about such a group
(Device::group_blends_outside_device_cmyk), and a pass that meets one
declines, so the enclosing group is drawn as before. A non-isolated
group's /CS is ignored (ISO 32000-2 §11.6.6, Table 145, CS: it takes its parent's space), so
it stays in the enclosing group's CMYK.
Test: raster::tests::isolated_cmyk_groups_blend_at_any_size_and_respect_nested_spaces.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ISO 32000-2 §11.3.4: the elements of a transparency group blend in the
group's colour space. For a subtractive space, each component is
complemented before the blend function and again after it. §11.3.5.3:
the nonseparable modes blend C, M and Y as their complements R, G and B,
and K comes from the backdrop (Hue, Saturation, Color) or from the source
(Luminosity). PdfCraft converted each colour to RGB and blended in RGB.
A print test file's blend-mode patch then showed an X in almost every
cell: magenta under black with Difference stayed magenta, not green.
hayro-interpret patch (45): CmykGroup and Device::draw_isolated_cmyk_group
(default false) for isolated, non-knockout groups with /CS /DeviceCMYK.
interpret_form_contents is now a function of its own. Color::ink_tints
and color::cmyk_to_srgb8 are new.
hayro patch (21): the renderer draws such a group as a CMY pass and a K
pass of complemented tints. It converts the tints to RGB with the CMYK
profile and composites the result through the group's layer. The group
is drawn as before with reading colours, for contents without process
tints (RGB, spot colours, patterns, images) and above 2^24 pixels.
Test: raster::tests::isolated_groups_blend_in_device_cmyk.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A malformed token ended the content stream where it stood, so the renderer,
text extraction, Find and search-based redaction all lost the rest of the
page -- while the edit reader, MuPDF and pdftotext read it (#3565, #3568). A
redaction that cannot see the text below the bad token cannot remove it,
which is the part that matters.
Each malformed shape is stepped over instead: an unterminated string, a
number that is only a sign or a dot, a name with a bad hex escape, an
unbalanced dictionary or array. Content after an inline image whose data
holds `EI` is still read, under a bounded lookahead.
Rebased onto main, 1,898 commits on: the tokenizer is still
vendor/hayro-syntax/src/content/mod.rs and applied as it was, and render's
raster.rs is split, so the two tests went to render/raster/tests.rs.
New modules: core, annotations, resources, navigation, model. The small test modules
(version_tests, open_action_javascript_tests, substitution_tests, open_dest_tests) moved with the code they test.
Conflicts:
- crates/render/src/inspect.rs: this branch's reply-linking methods and main's NoRotate
rectangles (#3016) both go in the Annotation impl; the annotation reader takes this
branch's loop with main's no_rotate field.
- crates/automation/src/comments.rs: this branch's HashMap import, main's import list.
- ATTRIBUTION.toml: each catalogue's source note keeps main's secured-document clause and
this branch's reply-count clause; ATTRIBUTION.md regenerated (cargo xtask assets --write).
- The 33 catalogues: main's new rows, then this branch's reply-count row.
main's #3016 tests build Annotation field by field; they now set parent and grouped.
ISO 32000-2 9.3.6: for a Type 3 font, modes 3 and 7 shall not be
rendered, the other modes render the glyph descriptions, and modes
4 to 7 add nothing to the clipping path. Vendored hayro-interpret
patch 42 drew every mode but 3 filled, so 7 Tr painted the glyphs
in the fill colour (black by default) wherever a following fill
did not cover them.
Patch 64: show_glyph keeps mode 7 for Type 3 glyphs and reports
them as clipping text (GlyphDrawMode::Clip, painting nothing, kept
by extraction), and clip_glyph's Type 3 arm is a deliberate no-op.
Modes 0-2 and 4-6 still render filled.
Claude-Session: https://claude.ai/code/session_0175FWj4xFhMhSH3UpMU8DSC
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The Bookmarks panel opened every top-level bookmark whatever its /Count
said, so a bookmark with a negative /Count showed its children. Every
level now starts as its /Count says (positive open; negative or missing
closed). The reader's toggles and a search still override it.
Moving a bookmark into a different parent (Indent) now opens that
parent, as adding one already did; the forced top-level open used to
hide that it stayed closed.
Claude-Session: https://claude.ai/code/session_0175FWj4xFhMhSH3UpMU8DSC
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
- render: DocInfo::open_dest resolves /OpenAction (an explicit destination,
a /GoTo action's /D, or a name or string through /Names /Dests or the
catalog /Dests) with the existing bounded dest() resolver.
- ui-egui: apply_initial_view goes there with DocView::go_to_dest, so the
/XYZ point lands at the window's top-left and /FitH top at its top edge.
With "Always use my default zoom and page layout", only the page.
- organize, ui-egui: /ViewerPreferences /NonFullScreenPageMode (Table 147)
is read for /PageMode /FullScreen documents and names the panel shown
behind the full-screen prompt; such a document no longer gets the outline
panel only because it has an outline.
- organize: set_initial_view keeps an opening action whose page and
magnification are unchanged, instead of dropping its position or deleting
a named go-to.
- automation: doc_info reports open_destination.
Claude-Session: https://claude.ai/code/session_0175FWj4xFhMhSH3UpMU8DSC
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
main's #4284 stopped hashing translation catalogues in ATTRIBUTION.toml; the
conflict took main's side (no sha256 lines) and kept this branch's source notes.
