* M14: bound large-PDF memory use (#307)
Inspect large documents lazily, share parser/resource metadata, bound full-save
and raster retention, and cache signature candidate discovery safely.
Preserve upstream rendering deduplication, pixel alignment, signature previews,
and organizer insertion behavior, with regression coverage for their interaction.
Validated with 1,336 workspace tests, 193 parser tests, strict workspace Clippy,
30 WASM crate checks, 1x/2x GUI protocols, and same-PDF workflow comparisons.
* cos: drop the writer cache-policy probe and its test-only hooks in production paths
* cos: the temporary reader keeps parsed objects until a count or owned-bytes budget
Since #552 a stream read from the file is a view into the input, so clearing the whole parse
cache after every read saved little memory and cost most of the full-save slowdown (every
object parsed twice, indirect /Length objects re-parsed per stream). Clear only past
OBJECTS entries or STREAM_BYTES of data of the cache's own (decrypted streams, strings).
* cos: the writers refuse a document opened with a stream limit (read-only inspection)
* cos: a full save fails rather than writing null when an object reads differently the second time
* render: parsing is not timed by the watchdog; timing tests with wide margins
A slow first parse (a huge or damaged file) made the whole pool fail every page for its
lifetime. Parsing, shared or private, is now its own untimed stage, as on main before the
shared parser; rendering is timed from when parsing ends. The init-timeout tests become one
deterministic test (a 2 s parse against a 10 ms limit, shared and private). Watchdog tests
use limits and injected delays ~10x apart instead of 25-50 ms against real renders. Also:
saturating retired-worker count, main's stop-flag test runs on private parsers (the first
shared timeout is a private retry), set_queue documents that unlisted results are dropped,
and retire() documents the permanent per-worker-parser fallback.
* render: page numbers in the lazy page tree use checked conversion
* render: the lazy inspector is compared with lopdf on named destinations and object streams too
* ui: on-screen pages always render; pages turned back to stay cached; stable thumbnail tags
- Visible pages and tiles are always admitted and uploaded (the viewport bounds them); the
byte and count limits decide only how much prefetch fits. A HiDPI two-up view or a 5K
viewport left pages on 'Rendering...' for good.
- Pages no longer demanded stay cached, nearest the current page first, while the 128 MiB
allowance and the 24-page count have room: paging back no longer re-renders.
- Thumbnail density follows the demand count rounded up to a power of two, so rows scrolling
in and out of an organize grid don't change every thumbnail's tag and re-render it.
- A suspended (background) tab drains its pool's finished results instead of holding them.
- Organize drag-and-drop ends on the primary button only.
- Tests: the stale-result test uses valid pixels and checks nothing was uploaded; the
organizer wheel test runs 40 steps; regressions for the first three fixes.
* vendor: mark the hayro-syntax lock and SegmentList patches and list them as (8) and (9); clippy 1.99 as_chunks in a raster test
* render: clippy 1.99 — contains over iter().any() in a raster test
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* hayro-syntax: fix float mantissa overflow on high-precision coordinates (#199)
jsPDF outputs double-precision floating-point coordinates with 16+ decimal
places in /MediaBox. In hayro-syntax, read_inner accumulated float mantissa
digits into u64 using wrapping arithmetic. Integer mantissas exceeding
u64::MAX wrapped modulo 2^64, severely corrupting page dimensions
(e.g., 3874.96 pt became 185.61 pt, rendering as a warped ribbon).
Accumulation now uses checked arithmetic: trailing fractional digits
exceeding u64 capacity are safely ignored without incrementing decimal_shift.
This preserves full IEEE-754 f64 precision while preventing overflow for
numbers with arbitrary numbers of decimal places.
Includes:
- Unit test in hayro-syntax with reported values and a 50-digit fraction.
- Synthetic PDF regression test in pdfcraft-render verifying inspect().
- Vendored patch (6) documentation in vendor/README.md.
Fixes#199
* M2: bound page-label work and output before applying labels
Share bounded PDF Roman and repeated-letter formatters between inspection
and editing. Bound tree traversal, raw and decoded prefixes, individual
labels and aggregate label output, with checked numbering arithmetic.
Stage inspection labels before applying them and retain physical page
numbers with a warning when optional labels cannot be processed. Reject
unsupported label edits before changing the document tree.
Add synthetic limit, cycle, Unicode, numbering and atomic-edit regressions.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* Never crash: mesh shadings can't exhaust memory or run for minutes
`cargo xtask fuzz` (with the dash, JBIG2 and decompression fixes in
place) found a page whose Coons patch shading (type 6), a 147-byte
stream with 1-bit coordinates decoded onto +-40000, took the renderer
past 3 GB. hayro-interpret samples mesh shadings (types 4-7) pixel by
pixel over each triangle's device-space bounding box into a map, with
nothing but the u16 cast bounding it: up to 65536 x 65536 entries for
one triangle. A synthetic single patch passed 6 GB in six seconds; ten
thousand triangles that each cover half a 1000 x 1000 page ran for
minutes. Two rounds of review of earlier versions found more: every
patch was cut into 722 triangles (about 150 KB) before anything limited
them; a lattice mesh with `/VerticesPerRow 0` pushed empty rows forever,
and one with 4294967295 read the whole stream into one row; a mesh
painted a thousand times was triangulated a thousand times; and the
first caps cut content without a word.
Vendored hayro-interpret patch (18):
- `ShadingPattern::encode_within(area)`: the vendored hayro renderer
passes the area it will actually draw (the path's bounding box on the
canvas, which it already computes) plus a pixel for texels at a
fractional edge, and triangles are sampled only inside it, so the map
holds at most that area's pixels.
- `TriangleSampler`: at most 8 visits per pixel of that area plus 4 per
triangle in it. A triangle's bounding box is about twice its area and a
sub-pixel triangle's is at most 2 x 2 pixels, so tiling meshes and
small dense ones (a patch in a few pixels is 722 triangles) still
shade completely; overlapping ones stop.
- Coons and tensor-product patches are triangulated one at a time.
- Meshes hold at most `MAX_MESH_PATCHES` (2^16; a 200 x 200 gradient
mesh is 40 thousand) patches and `MAX_MESH_TRIANGLES` (2^20)
triangles, and a lattice needs two (ISO 32000-2 8.7.4.5.6) to 2^19
vertices per row.
- A page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25; the 200 x 200
mesh is 29 million) mesh triangles in all, reset per page like the
existing paint budget.
- Each of these cuts logs a warning.
`encode()` keeps its signature; with no area it samples within a fixed
2^24 visits.
Synthetic regression tests in render:
- `mesh_shadings_far_off_the_page_render`: a Coons patch and a triangle
mesh decoded onto +-40000 render and the rest of the page draws; a
patch on the page, the part of the huge one that covers it, and a
patch filling a 4 x 4 pt rectangle as a pattern are all shaded.
- `many_overlapping_mesh_triangles_finish`: 10,000 triangles each
covering half of a 1000 x 1000 page finish, and are drawn.
- `mesh_shadings_with_many_patches_or_no_rows_finish`: 40,000 patches
beside the page, and a lattice with `/VerticesPerRow 0`.
Red on main (out of memory, killed at 3.9 GB). Each part is pinned by a
mutation: no visit budget stalls past 60 s; no clip leaves the covering
patch unshaded; no per-triangle allowance leaves the small patch
unshaded; triangulating all patches first runs out of memory (killed at
5.8 GB); no rows check runs out of memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Never crash: decompression bombs can't exhaust the renderer's memory
The renderer's parser (vendored hayro-syntax) decoded Flate, LZW and
RunLength streams with no output limit. A 2 MB page whose content
stream inflates to 2 GB, or whose 8 x 8 image does, took `pdfcraft-cli
render` and `text` past 3 GB in under a second, and a few more MB of
file scale it further. On the desktop that is the first-page render of
an emailed PDF, and the allocation-failure abort takes every open
document with it. Our own layers already bound single streams (lopdf's
load-time decoding at 256 MiB, cos at 1 GiB), so `info` stays at a few
MB; the renderer path didn't. Random mutation almost never makes a
high-ratio stream, so `cargo xtask fuzz` doesn't find this; found by
hand while triaging the fuzz run.
Vendored hayro-syntax patch (6):
- `decode_limit` gives each filter an output limit, and Flate (both
flate2 and the fallback inflater), LZW and RunLength never write past
it (the buffer can still grow to under twice it):
`MAX_DECODED_STREAM`, 256 MiB, lopdf's figure; or, for the last filter
of an image whose size, bits per component and components are known,
twice its pixel bytes (a PNG predictor tag byte per byte at most),
capped at 1 GiB. Earlier filters in a chain produce compressed data
and get the general limit.
- `Stream::decoded_within(max)` for callers that budget.
- A page's `/Contents` array shares one `MAX_DECODED_STREAM` (naming one
bomb twenty times decoded 5 GB), reserving exactly what it appends.
Vendored hayro-interpret patch (18):
- The content a page holds at once (its contents, and the forms, tiling
patterns and Type 3 glyph procedures being painted) stays within
`MAX_PAGE_CONTENT`, 512 MiB, reset per page like the existing paint
budget. Forms nest fifty deep and each level kept its content while
the next ran: a form painting itself around a 256 MiB stream would
have held over 12 GB. Each level gives its bytes back when it is
dropped, so content painted again and again keeps drawing, and a
stream without filters (borrowed from the file) costs nothing.
- Type 0 sampled functions read every sample into a 4-byte value and
then a table entry: 256 MiB of 1-bit samples asked for over 8 GB. They
now take at most 2^21 table points and 2^23 values (the largest real
tables, DeviceN tint transforms such as 33^4 points x 3 outputs, are
1.2 million and 3.6 million), counted with checked arithmetic, and
read no further than `/Size` asks. `/Size 0` no longer underflows
building the default `/Encode`.
Three rounds of independent review found the aggregate cases, buffer
doubling past the limit, a chain/predictor shape the first image limit
would have truncated, the Type 0 amplifier, and a first, cumulative
budget that would have dropped legitimately repeated content, and a
first Type 0 cap low enough to refuse real tint transforms; all fixed
here.
Measured in a release build (render; OS peak commit):
- content stream inflating to 2 GB: 389 MB, 0.3 s
- 8 x 8 image inflating to 2 GB: 4 MB, 0.2 s
- /Contents naming it twenty times: 516 MB, 0.3 s
- a form painting itself around it: 644 MB, 0.4 s
Synthetic regression tests in render (flate2 is a new dev-dependency
there, already in the workspace):
- `decompression_bombs_stop_at_the_stream_limit`: 288 MiB bombs through
flate2, the fallback inflater (a stored block with a wrong length
check, which flate2 refuses and the fallback accepts), LZW and
RunLength each decode exactly 256 MiB, and at most 128 bytes as an
8 x 8 image.
- `page_contents_and_nested_forms_share_one_budget`: three /Contents
references decode one stream's worth; a self-painting form finishes.
- `content_painted_again_and_again_keeps_drawing`: a 2 MiB form painted
300 times draws the 300th time.
- `sampled_functions_with_billions_of_samples_are_refused`.
Ten mutations each turn them red: no limit at all; no limit in flate2,
the fallback, LZW, RunLength or for images; an unshared /Contents limit;
no page budget; no refund on drop; no Type 0 sample cap. (Taking a form
out of the budget no longer compiles: its content is the budget guard.)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M2: normalize JPEG2000 samples without overflowing
Use bounded integer sample scaling for 1-32-bit JPX images and reject empty dimensions before chunking. Cover public decoding and continued page rendering with synthetic regressions.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* Never crash: decompression cut at its limit is logged, not silent; vendor README renumbered after #575
---------
Co-authored-by: philkwek <philkwek03@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: James Locke <336218001+jcl-bit@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* hayro-syntax: fix float mantissa overflow on high-precision coordinates (#199)
jsPDF outputs double-precision floating-point coordinates with 16+ decimal
places in /MediaBox. In hayro-syntax, read_inner accumulated float mantissa
digits into u64 using wrapping arithmetic. Integer mantissas exceeding
u64::MAX wrapped modulo 2^64, severely corrupting page dimensions
(e.g., 3874.96 pt became 185.61 pt, rendering as a warped ribbon).
Accumulation now uses checked arithmetic: trailing fractional digits
exceeding u64 capacity are safely ignored without incrementing decimal_shift.
This preserves full IEEE-754 f64 precision while preventing overflow for
numbers with arbitrary numbers of decimal places.
Includes:
- Unit test in hayro-syntax with reported values and a 50-digit fraction.
- Synthetic PDF regression test in pdfcraft-render verifying inspect().
- Vendored patch (6) documentation in vendor/README.md.
Fixes#199
* Never crash: mesh shadings can't exhaust memory or run for minutes
`cargo xtask fuzz` (with the dash, JBIG2 and decompression fixes in
place) found a page whose Coons patch shading (type 6), a 147-byte
stream with 1-bit coordinates decoded onto +-40000, took the renderer
past 3 GB. hayro-interpret samples mesh shadings (types 4-7) pixel by
pixel over each triangle's device-space bounding box into a map, with
nothing but the u16 cast bounding it: up to 65536 x 65536 entries for
one triangle. A synthetic single patch passed 6 GB in six seconds; ten
thousand triangles that each cover half a 1000 x 1000 page ran for
minutes. Two rounds of review of earlier versions found more: every
patch was cut into 722 triangles (about 150 KB) before anything limited
them; a lattice mesh with `/VerticesPerRow 0` pushed empty rows forever,
and one with 4294967295 read the whole stream into one row; a mesh
painted a thousand times was triangulated a thousand times; and the
first caps cut content without a word.
Vendored hayro-interpret patch (18):
- `ShadingPattern::encode_within(area)`: the vendored hayro renderer
passes the area it will actually draw (the path's bounding box on the
canvas, which it already computes) plus a pixel for texels at a
fractional edge, and triangles are sampled only inside it, so the map
holds at most that area's pixels.
- `TriangleSampler`: at most 8 visits per pixel of that area plus 4 per
triangle in it. A triangle's bounding box is about twice its area and a
sub-pixel triangle's is at most 2 x 2 pixels, so tiling meshes and
small dense ones (a patch in a few pixels is 722 triangles) still
shade completely; overlapping ones stop.
- Coons and tensor-product patches are triangulated one at a time.
- Meshes hold at most `MAX_MESH_PATCHES` (2^16; a 200 x 200 gradient
mesh is 40 thousand) patches and `MAX_MESH_TRIANGLES` (2^20)
triangles, and a lattice needs two (ISO 32000-2 8.7.4.5.6) to 2^19
vertices per row.
- A page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25; the 200 x 200
mesh is 29 million) mesh triangles in all, reset per page like the
existing paint budget.
- Each of these cuts logs a warning.
`encode()` keeps its signature; with no area it samples within a fixed
2^24 visits.
Synthetic regression tests in render:
- `mesh_shadings_far_off_the_page_render`: a Coons patch and a triangle
mesh decoded onto +-40000 render and the rest of the page draws; a
patch on the page, the part of the huge one that covers it, and a
patch filling a 4 x 4 pt rectangle as a pattern are all shaded.
- `many_overlapping_mesh_triangles_finish`: 10,000 triangles each
covering half of a 1000 x 1000 page finish, and are drawn.
- `mesh_shadings_with_many_patches_or_no_rows_finish`: 40,000 patches
beside the page, and a lattice with `/VerticesPerRow 0`.
Red on main (out of memory, killed at 3.9 GB). Each part is pinned by a
mutation: no visit budget stalls past 60 s; no clip leaves the covering
patch unshaded; no per-triangle allowance leaves the small patch
unshaded; triangulating all patches first runs out of memory (killed at
5.8 GB); no rows check runs out of memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Never crash: decompression bombs can't exhaust the renderer's memory
The renderer's parser (vendored hayro-syntax) decoded Flate, LZW and
RunLength streams with no output limit. A 2 MB page whose content
stream inflates to 2 GB, or whose 8 x 8 image does, took `pdfcraft-cli
render` and `text` past 3 GB in under a second, and a few more MB of
file scale it further. On the desktop that is the first-page render of
an emailed PDF, and the allocation-failure abort takes every open
document with it. Our own layers already bound single streams (lopdf's
load-time decoding at 256 MiB, cos at 1 GiB), so `info` stays at a few
MB; the renderer path didn't. Random mutation almost never makes a
high-ratio stream, so `cargo xtask fuzz` doesn't find this; found by
hand while triaging the fuzz run.
Vendored hayro-syntax patch (6):
- `decode_limit` gives each filter an output limit, and Flate (both
flate2 and the fallback inflater), LZW and RunLength never write past
it (the buffer can still grow to under twice it):
`MAX_DECODED_STREAM`, 256 MiB, lopdf's figure; or, for the last filter
of an image whose size, bits per component and components are known,
twice its pixel bytes (a PNG predictor tag byte per byte at most),
capped at 1 GiB. Earlier filters in a chain produce compressed data
and get the general limit.
- `Stream::decoded_within(max)` for callers that budget.
- A page's `/Contents` array shares one `MAX_DECODED_STREAM` (naming one
bomb twenty times decoded 5 GB), reserving exactly what it appends.
Vendored hayro-interpret patch (18):
- The content a page holds at once (its contents, and the forms, tiling
patterns and Type 3 glyph procedures being painted) stays within
`MAX_PAGE_CONTENT`, 512 MiB, reset per page like the existing paint
budget. Forms nest fifty deep and each level kept its content while
the next ran: a form painting itself around a 256 MiB stream would
have held over 12 GB. Each level gives its bytes back when it is
dropped, so content painted again and again keeps drawing, and a
stream without filters (borrowed from the file) costs nothing.
- Type 0 sampled functions read every sample into a 4-byte value and
then a table entry: 256 MiB of 1-bit samples asked for over 8 GB. They
now take at most 2^21 table points and 2^23 values (the largest real
tables, DeviceN tint transforms such as 33^4 points x 3 outputs, are
1.2 million and 3.6 million), counted with checked arithmetic, and
read no further than `/Size` asks. `/Size 0` no longer underflows
building the default `/Encode`.
Three rounds of independent review found the aggregate cases, buffer
doubling past the limit, a chain/predictor shape the first image limit
would have truncated, the Type 0 amplifier, and a first, cumulative
budget that would have dropped legitimately repeated content, and a
first Type 0 cap low enough to refuse real tint transforms; all fixed
here.
Measured in a release build (render; OS peak commit):
- content stream inflating to 2 GB: 389 MB, 0.3 s
- 8 x 8 image inflating to 2 GB: 4 MB, 0.2 s
- /Contents naming it twenty times: 516 MB, 0.3 s
- a form painting itself around it: 644 MB, 0.4 s
Synthetic regression tests in render (flate2 is a new dev-dependency
there, already in the workspace):
- `decompression_bombs_stop_at_the_stream_limit`: 288 MiB bombs through
flate2, the fallback inflater (a stored block with a wrong length
check, which flate2 refuses and the fallback accepts), LZW and
RunLength each decode exactly 256 MiB, and at most 128 bytes as an
8 x 8 image.
- `page_contents_and_nested_forms_share_one_budget`: three /Contents
references decode one stream's worth; a self-painting form finishes.
- `content_painted_again_and_again_keeps_drawing`: a 2 MiB form painted
300 times draws the 300th time.
- `sampled_functions_with_billions_of_samples_are_refused`.
Ten mutations each turn them red: no limit at all; no limit in flate2,
the fallback, LZW, RunLength or for images; an unshared /Contents limit;
no page budget; no refund on drop; no Type 0 sample cap. (Taking a form
out of the budget no longer compiles: its content is the budget guard.)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M2: normalize JPEG2000 samples without overflowing
Use bounded integer sample scaling for 1-32-bit JPX images and reject empty dimensions before chunking. Cover public decoding and continued page rendering with synthetic regressions.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* Never crash: decompression cut at its limit is logged, not silent; vendor README renumbered after #575
---------
Co-authored-by: James Locke <336218001+jcl-bit@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Streetblock <github@streetblock.de>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Never crash: mesh shadings can't exhaust memory or run for minutes
`cargo xtask fuzz` (with the dash, JBIG2 and decompression fixes in
place) found a page whose Coons patch shading (type 6), a 147-byte
stream with 1-bit coordinates decoded onto +-40000, took the renderer
past 3 GB. hayro-interpret samples mesh shadings (types 4-7) pixel by
pixel over each triangle's device-space bounding box into a map, with
nothing but the u16 cast bounding it: up to 65536 x 65536 entries for
one triangle. A synthetic single patch passed 6 GB in six seconds; ten
thousand triangles that each cover half a 1000 x 1000 page ran for
minutes. Two rounds of review of earlier versions found more: every
patch was cut into 722 triangles (about 150 KB) before anything limited
them; a lattice mesh with `/VerticesPerRow 0` pushed empty rows forever,
and one with 4294967295 read the whole stream into one row; a mesh
painted a thousand times was triangulated a thousand times; and the
first caps cut content without a word.
Vendored hayro-interpret patch (18):
- `ShadingPattern::encode_within(area)`: the vendored hayro renderer
passes the area it will actually draw (the path's bounding box on the
canvas, which it already computes) plus a pixel for texels at a
fractional edge, and triangles are sampled only inside it, so the map
holds at most that area's pixels.
- `TriangleSampler`: at most 8 visits per pixel of that area plus 4 per
triangle in it. A triangle's bounding box is about twice its area and a
sub-pixel triangle's is at most 2 x 2 pixels, so tiling meshes and
small dense ones (a patch in a few pixels is 722 triangles) still
shade completely; overlapping ones stop.
- Coons and tensor-product patches are triangulated one at a time.
- Meshes hold at most `MAX_MESH_PATCHES` (2^16; a 200 x 200 gradient
mesh is 40 thousand) patches and `MAX_MESH_TRIANGLES` (2^20)
triangles, and a lattice needs two (ISO 32000-2 8.7.4.5.6) to 2^19
vertices per row.
- A page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25; the 200 x 200
mesh is 29 million) mesh triangles in all, reset per page like the
existing paint budget.
- Each of these cuts logs a warning.
`encode()` keeps its signature; with no area it samples within a fixed
2^24 visits.
Synthetic regression tests in render:
- `mesh_shadings_far_off_the_page_render`: a Coons patch and a triangle
mesh decoded onto +-40000 render and the rest of the page draws; a
patch on the page, the part of the huge one that covers it, and a
patch filling a 4 x 4 pt rectangle as a pattern are all shaded.
- `many_overlapping_mesh_triangles_finish`: 10,000 triangles each
covering half of a 1000 x 1000 page finish, and are drawn.
- `mesh_shadings_with_many_patches_or_no_rows_finish`: 40,000 patches
beside the page, and a lattice with `/VerticesPerRow 0`.
Red on main (out of memory, killed at 3.9 GB). Each part is pinned by a
mutation: no visit budget stalls past 60 s; no clip leaves the covering
patch unshaded; no per-triangle allowance leaves the small patch
unshaded; triangulating all patches first runs out of memory (killed at
5.8 GB); no rows check runs out of memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Never crash: decompression bombs can't exhaust the renderer's memory
The renderer's parser (vendored hayro-syntax) decoded Flate, LZW and
RunLength streams with no output limit. A 2 MB page whose content
stream inflates to 2 GB, or whose 8 x 8 image does, took `pdfcraft-cli
render` and `text` past 3 GB in under a second, and a few more MB of
file scale it further. On the desktop that is the first-page render of
an emailed PDF, and the allocation-failure abort takes every open
document with it. Our own layers already bound single streams (lopdf's
load-time decoding at 256 MiB, cos at 1 GiB), so `info` stays at a few
MB; the renderer path didn't. Random mutation almost never makes a
high-ratio stream, so `cargo xtask fuzz` doesn't find this; found by
hand while triaging the fuzz run.
Vendored hayro-syntax patch (6):
- `decode_limit` gives each filter an output limit, and Flate (both
flate2 and the fallback inflater), LZW and RunLength never write past
it (the buffer can still grow to under twice it):
`MAX_DECODED_STREAM`, 256 MiB, lopdf's figure; or, for the last filter
of an image whose size, bits per component and components are known,
twice its pixel bytes (a PNG predictor tag byte per byte at most),
capped at 1 GiB. Earlier filters in a chain produce compressed data
and get the general limit.
- `Stream::decoded_within(max)` for callers that budget.
- A page's `/Contents` array shares one `MAX_DECODED_STREAM` (naming one
bomb twenty times decoded 5 GB), reserving exactly what it appends.
Vendored hayro-interpret patch (18):
- The content a page holds at once (its contents, and the forms, tiling
patterns and Type 3 glyph procedures being painted) stays within
`MAX_PAGE_CONTENT`, 512 MiB, reset per page like the existing paint
budget. Forms nest fifty deep and each level kept its content while
the next ran: a form painting itself around a 256 MiB stream would
have held over 12 GB. Each level gives its bytes back when it is
dropped, so content painted again and again keeps drawing, and a
stream without filters (borrowed from the file) costs nothing.
- Type 0 sampled functions read every sample into a 4-byte value and
then a table entry: 256 MiB of 1-bit samples asked for over 8 GB. They
now take at most 2^21 table points and 2^23 values (the largest real
tables, DeviceN tint transforms such as 33^4 points x 3 outputs, are
1.2 million and 3.6 million), counted with checked arithmetic, and
read no further than `/Size` asks. `/Size 0` no longer underflows
building the default `/Encode`.
Three rounds of independent review found the aggregate cases, buffer
doubling past the limit, a chain/predictor shape the first image limit
would have truncated, the Type 0 amplifier, and a first, cumulative
budget that would have dropped legitimately repeated content, and a
first Type 0 cap low enough to refuse real tint transforms; all fixed
here.
Measured in a release build (render; OS peak commit):
- content stream inflating to 2 GB: 389 MB, 0.3 s
- 8 x 8 image inflating to 2 GB: 4 MB, 0.2 s
- /Contents naming it twenty times: 516 MB, 0.3 s
- a form painting itself around it: 644 MB, 0.4 s
Synthetic regression tests in render (flate2 is a new dev-dependency
there, already in the workspace):
- `decompression_bombs_stop_at_the_stream_limit`: 288 MiB bombs through
flate2, the fallback inflater (a stored block with a wrong length
check, which flate2 refuses and the fallback accepts), LZW and
RunLength each decode exactly 256 MiB, and at most 128 bytes as an
8 x 8 image.
- `page_contents_and_nested_forms_share_one_budget`: three /Contents
references decode one stream's worth; a self-painting form finishes.
- `content_painted_again_and_again_keeps_drawing`: a 2 MiB form painted
300 times draws the 300th time.
- `sampled_functions_with_billions_of_samples_are_refused`.
Ten mutations each turn them red: no limit at all; no limit in flate2,
the fallback, LZW, RunLength or for images; an unshared /Contents limit;
no page budget; no refund on drop; no Type 0 sample cap. (Taking a form
out of the budget no longer compiles: its content is the budget guard.)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M2: normalize JPEG2000 samples without overflowing
Use bounded integer sample scaling for 1-32-bit JPX images and reject empty dimensions before chunking. Cover public decoding and continued page rendering with synthetic regressions.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* Never crash: decompression cut at its limit is logged, not silent; vendor README renumbered after #575
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Streetblock <github@streetblock.de>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Never crash: decompression bombs can't exhaust the renderer's memory
The renderer's parser (vendored hayro-syntax) decoded Flate, LZW and
RunLength streams with no output limit. A 2 MB page whose content
stream inflates to 2 GB, or whose 8 x 8 image does, took `pdfcraft-cli
render` and `text` past 3 GB in under a second, and a few more MB of
file scale it further. On the desktop that is the first-page render of
an emailed PDF, and the allocation-failure abort takes every open
document with it. Our own layers already bound single streams (lopdf's
load-time decoding at 256 MiB, cos at 1 GiB), so `info` stays at a few
MB; the renderer path didn't. Random mutation almost never makes a
high-ratio stream, so `cargo xtask fuzz` doesn't find this; found by
hand while triaging the fuzz run.
Vendored hayro-syntax patch (6):
- `decode_limit` gives each filter an output limit, and Flate (both
flate2 and the fallback inflater), LZW and RunLength never write past
it (the buffer can still grow to under twice it):
`MAX_DECODED_STREAM`, 256 MiB, lopdf's figure; or, for the last filter
of an image whose size, bits per component and components are known,
twice its pixel bytes (a PNG predictor tag byte per byte at most),
capped at 1 GiB. Earlier filters in a chain produce compressed data
and get the general limit.
- `Stream::decoded_within(max)` for callers that budget.
- A page's `/Contents` array shares one `MAX_DECODED_STREAM` (naming one
bomb twenty times decoded 5 GB), reserving exactly what it appends.
Vendored hayro-interpret patch (18):
- The content a page holds at once (its contents, and the forms, tiling
patterns and Type 3 glyph procedures being painted) stays within
`MAX_PAGE_CONTENT`, 512 MiB, reset per page like the existing paint
budget. Forms nest fifty deep and each level kept its content while
the next ran: a form painting itself around a 256 MiB stream would
have held over 12 GB. Each level gives its bytes back when it is
dropped, so content painted again and again keeps drawing, and a
stream without filters (borrowed from the file) costs nothing.
- Type 0 sampled functions read every sample into a 4-byte value and
then a table entry: 256 MiB of 1-bit samples asked for over 8 GB. They
now take at most 2^21 table points and 2^23 values (the largest real
tables, DeviceN tint transforms such as 33^4 points x 3 outputs, are
1.2 million and 3.6 million), counted with checked arithmetic, and
read no further than `/Size` asks. `/Size 0` no longer underflows
building the default `/Encode`.
Three rounds of independent review found the aggregate cases, buffer
doubling past the limit, a chain/predictor shape the first image limit
would have truncated, the Type 0 amplifier, and a first, cumulative
budget that would have dropped legitimately repeated content, and a
first Type 0 cap low enough to refuse real tint transforms; all fixed
here.
Measured in a release build (render; OS peak commit):
- content stream inflating to 2 GB: 389 MB, 0.3 s
- 8 x 8 image inflating to 2 GB: 4 MB, 0.2 s
- /Contents naming it twenty times: 516 MB, 0.3 s
- a form painting itself around it: 644 MB, 0.4 s
Synthetic regression tests in render (flate2 is a new dev-dependency
there, already in the workspace):
- `decompression_bombs_stop_at_the_stream_limit`: 288 MiB bombs through
flate2, the fallback inflater (a stored block with a wrong length
check, which flate2 refuses and the fallback accepts), LZW and
RunLength each decode exactly 256 MiB, and at most 128 bytes as an
8 x 8 image.
- `page_contents_and_nested_forms_share_one_budget`: three /Contents
references decode one stream's worth; a self-painting form finishes.
- `content_painted_again_and_again_keeps_drawing`: a 2 MiB form painted
300 times draws the 300th time.
- `sampled_functions_with_billions_of_samples_are_refused`.
Ten mutations each turn them red: no limit at all; no limit in flate2,
the fallback, LZW, RunLength or for images; an unshared /Contents limit;
no page budget; no refund on drop; no Type 0 sample cap. (Taking a form
out of the budget no longer compiles: its content is the budget guard.)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M2: normalize JPEG2000 samples without overflowing
Use bounded integer sample scaling for 1-32-bit JPX images and reject empty dimensions before chunking. Cover public decoding and continued page rendering with synthetic regressions.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* Never crash: decompression cut at its limit is logged, not silent; vendor README renumbered after #575
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Streetblock <github@streetblock.de>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M2: normalize JPEG2000 samples without overflowing
Use bounded integer sample scaling for 1-32-bit JPX images and reject empty dimensions before chunking. Cover public decoding and continued page rendering with synthetic regressions.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* ROADMAP: keep the JPX entry with this session's entries
---------
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M3: rasters reach egui without a copy on the UI thread
Every page, tile and thumbnail raster was copied pixel by pixel on the UI
thread (`ColorImage::from_rgba_premultiplied`), because egui's `Color32` is
4-byte aligned and a `Vec<u8>` isn't. That is 0.5 ms per 1024² tile, 5.4 ms
for a 33 MB page and 36 ms for a 197 MB page, against a 16.7 ms frame.
hayro, vendored patch 7: `render_size` and `render_into` render into a buffer
the caller allocates. `render` is `render_into` on a new pixmap, so its output
is unchanged. pdfcraft-render renders straight into `Pixels`, the same
premultiplied RGBA bytes kept in one 4-byte word per pixel. `Pixels`
dereferences to `[u8]`, so every reader of `RenderedPage::rgba` compiles
unchanged. The canvas reinterprets the words as `Color32`s and hands them to
egui as they are.
Pixels are identical: all 33 renders of #260's test file hash the same. The
UI-thread copy is gone, and rendering a 6182×8000 page takes 67 ms instead of
71–76 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M3: render_into leaves a buffer of the wrong length untouched instead of reaching vello's assert
* M3: regression test for render_into with a wrong-length buffer
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Every destination was reduced to a page index (`dest_page` in the
inspector), so a bookmark written as [p1 /XYZ 0 420 0] went to the top
of page 1 instead of putting y 420 at the top of the window, and /FitH,
/FitV, /FitR and an /XYZ zoom were ignored.
Destinations now keep their view (ISO 32000-2 §12.3.2.2):
- render: `DestView` (Top, Xyz, Fit, FitH, FitV, FitR) parsed from
explicit arrays, /D dictionaries and named destinations (/Names
/Dests and /Dests). `OutlineItem` gains `view`; `LinkTarget::Page`
carries it. Missing, wrong-typed or non-finite operands are null, a
zoom that isn't positive keeps the zoom, and an unknown type or a
/FitR without four numbers falls back to the page top.
`PageInfo::dest_fraction` maps a user-space point to the displayed
page through the crop box origin, /Rotate and the view rotation,
clamped to the page.
- ui: `DocView::go_to_dest` applies it for bookmark and link clicks
(and so for `ui.click` on the control channel): /XYZ puts (left, top)
at the window's top-left (beside the quick-action gutter) at its
zoom; /Fit, /FitH and /FitV switch to fit page, width and height with
top/left at the edge; /FitR zooms so the rectangle fits, centred. The
/FitB* content box is read as the whole page.
Tests: operand parsing incl. hostile values, a fixture with every form
(named, GoTo, wrong types, huge numbers, no page), point mapping under
crop offsets and rotations, go_to_dest state, and kittest bookmark and
link clicks checking the point lands at the window's top.
Co-authored-by: lloyd <lloyd@stackworx.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
When a render runs past `stuck_after`, the watchdog answers with an error,
starts a replacement worker and drops the late result. The abandoned worker
still rendered the page to the end: #409's stop flag belonged to the whole
pool and was only set when the pool was dropped.
Each worker now has its own stop flag. Dropping the pool sets every worker's
flag, as before. The watchdog sets the flag of the worker it gives up on, so
that render stops at its next content operator. Its answer was already thrown
away, so nothing the user sees changes. Live workers are never stopped.
After giving up on 4 heavy renders, the process used 11.2–11.6 s of CPU in
the next 3 s with nothing left to do; now it uses none.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`cargo xtask fuzz` found a 1.1 KB page whose JBIG2 image never finished
decoding (over two minutes, flat memory). It differs from a valid file by
one byte inside a symbol dictionary's arithmetic-coded data. Past the end
of its data the arithmetic decoder keeps returning values, and two loops
in symbol.rs only move on when those values make progress:
- Export runs (6.5.10): a run of length 0 doesn't advance, and the
decoder returned 0 for ever. This was the fuzz finding.
- Height classes (6.5.5): a class whose first width is already the
end-of-class marker decodes no symbol. A read-only sweep of the crate
found this one. In a synthetic dictionary every class after the first
symbol adds 13 to the height and decodes nothing, so it only fails
when the height overflows u32, after 330 million classes (about five
seconds per image, logged); a delta of 0 would never end.
The sweep found every other loop in hayro-jbig2 bounded by a symbol,
instance or pixel cap or by the end of the data. Bounded is not cheap
everywhere: refinement bitmaps in text regions and aggregate symbols
(up to 10,000 instances of up to 2^28 pixels each) are not changed
here.
Vendored hayro-jbig2 patch (3): a valid dictionary needs at most one
height class per new symbol and `2 x symbols + 1` export runs (they
alternate, so at most one zero-length run before each symbol's). Both
loops get that budget plus `EXTRA_EMPTY_STEPS` (1024), then fail the
dictionary as invalid. Encoders don't write empty steps back to back,
so the slack only tolerates oddities.
The fuzz finding now renders in 0.2 s. Synthetic regression test in
render: two dictionaries of a few arbitrary bytes each, found by
searching for streams that decode the same way (no corpus bytes). The
height-class one is drawn as twenty images so that it stalls well past
the 20 s timeout without the patch on any machine. Each case is red
with its own budget removed and green with it.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M2: bound lopdf predictor scratch to available samples
Return empty PNG predictor frames before allocating row buffers. Reject
overflowing packed TIFF row dimensions and bound component scratch to the
samples present in the input, aligning with the existing lopdf#589 fix.
Cover hostile dimensions and valid packed rows with synthetic regressions
and record the vendored patch and its upstream proposal.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* Keep main's M1 roadmap row after merging main
---------
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
`render_page` copied every raster out of vello_cpu's pixmap with
`data_as_u8_slice().to_vec()`. That is a second buffer the size of the raster
(up to 256 MB for a 64 MP page), alive alongside the first for as long as the
copy runs. `Pixmap::take` hands over the pixel vector, and `bytemuck`
reinterprets its four one-byte channels as bytes in place: the same bytes, in
the same buffer.
On a 6182×8000 px page, peak RSS goes from about 425 MB to 246 MB and render
time from 102 ms to 81 ms. Pixels are identical: every page of #260's test
file, at two scales and as a tile, hashes the same before and after.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The engine replaces a document's render pool after every edit, undo, redo,
layer toggle and comment-visibility change, and drops it on close. Workers in
the middle of a page kept rendering it to the end before finding that nobody
could receive the answer. On heavy pages that is seconds of CPU per worker,
competing with the new pool re-rendering the same pages.
hayro-interpret, vendored patch 17: `InterpreterSettings::cancelled`, checked
before each content operator. The pool sets it only in `Drop`, when its result
channel is gone, so a partly drawn page can never be delivered. Live pools,
`PageRenderer` and inline pools never set it.
7 heavy pages on 7 workers re-rendered after replacing a busy pool: 2.4–2.5 s
before, 1.1 s after (1.0–1.3 s without a replacement, unchanged).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`cargo xtask fuzz` found a 392-byte page whose ExtGState sets
`/D [[11] 0]` on a line that starts at x = 92234775807. Stroke expansion
keeps every dash, so it cut the line into billions of them and allocated
more than 5 GB before the run was stopped. Triage found the same class
in ordinary-looking geometry, and a second bug next to it:
- A tiny dash on an ordinary line (`[0.000001] 0 d` on 20 pt) passed
10 GB in under two seconds; so did `[1] 0 d` under a shrinking matrix,
and a long pattern on many short subpaths, because kurbo restarts the
pattern at every subpath. The render watchdog can skip a stuck page but
can't stop it allocating, so on a desktop this ends in an abort that
takes every open document with it.
- A dash array whose entries sum to less than zero (`[-1 -1] 0 d`) made
kurbo's search for the starting dash loop forever. ISO 32000-2
§8.4.3.6 requires nonnegative entries.
Vendored hayro patch (5, 6): `usable_dash_pattern` bounds the pieces a
pattern can cut the path into, in user space where kurbo dashes:
(control-polygon length / period + restarts) x entries per period, where
restarts are kurbo's: every MoveTo and ClosePath. Over 250,000 the stroke
is drawn solid, as it is for a pattern with a negative or non-finite
entry or a period that isn't positive. Measured in release builds: a
250,000-piece stroke takes about 16 ms and 75 MB at its peak (a million
took 80 ms and 290 MB). A non-finite phase becomes 0.
Every case now renders in about 0.2 s and a few MB. Synthetic regression
tests in render. Both were red before the fix (the first killed at
2.7 GB, the second stalled past 20 s), and four mutations each turn them
red: the first estimate (one piece per subpath) with a 100k cap runs out
of memory, a 100k cap draws a fine legitimate dash solid, dropping the
negative-entry check draws `[-1 3]` dashed, and dropping it together
with the period check hangs. (The period check alone is defence in
depth: hayro-interpret turns 0 entries into 0.01, so a pattern without
negative entries always has a positive period.)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Since #368, a non-embedded Adobe-Japan1 font with a Mincho name
(HeiseiMin-W3, MS-Mincho, ...) is drawn in document_japanese_font(),
i.e. Shippori Mincho. Shippori Mincho has none of the 63 half-width
katakana (U+FF61-U+FF9F), which are common in Japanese documents.
hayro maps the CID through Adobe-Japan1-UCS2 to Unicode, finds no
glyph in the face's cmap, and falls back to drawing the CID as a glyph
index: half-width katakana came out as unrelated glyphs (ligatures
such as fi and fl).
The Mincho substitute is now BIZ UDMincho Regular, which covers
half-width katakana (and was designed as a metric-compatible
replacement for MS Mincho), falling back to document_japanese_font()
when it isn't built in. The face used for Japanese text written into
PDFs is unchanged. Without craft-fonts nothing changes.
Tests: raster::tests::mincho_substitute_covers_half_width_katakana
(with CRAFT_FONTS_DIR: the Mincho substitute's cmap maps U+FF61-U+FF9F
and 日本; it lacked all 63 half-width katakana before; skips without
craft-fonts). skrifa 0.44 is a dev-dependency of pdfcraft-render for
it, the version pdfcraft-fonts already uses. The parity entry
view.render-cid-fonts cites the test.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Read the date prefix at a checked UTF-8 boundary and leave malformed dates
unchanged. Commit auxiliary metadata only after successful inspection so
an escaped inspector panic cannot discard the renderer's page geometry.
Add synthetic date-boundary, UTF16BE annotation and independent failure
regressions.
Co-authored-by: OpenAI Codex <codex@openai.com>
The canvas rebuilds its render queue from every page, tile and text layer it
is still missing, and calls `set_queue` whenever that list changes, which is
whenever any answer arrives. Requests a worker had already popped were still
missing, so they went back into the queue and an idle worker rendered them a
second time. With one worker every other page was rendered twice; with seven
workers about half of all renders were duplicates. A slow page could also take
every idle worker in turn until the watchdog fired.
The pool now remembers the requests its workers have taken until `try_recv`
hands out their answer. A queued request equal to one of them is skipped, and
an answer removes queued copies of its request. Equal requests to one pool
always produce the same answer (its bytes and settings never change), so
nothing the caller sees changes except that answers arrive sooner.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Text: find the word and the line around a glyph
PageText gets word_at(i) and line_at(i). Each returns the first and
last glyph of the word or the line around glyph i, or None when i is
past the end of the page's text. A word stops at a word gap, a blank
glyph or a line end. A line is the run of glyphs with the same line
number.
The canvas's double-click used its own loop for this, which indexed the
glyph, line and gap arrays directly. It now calls word_at, which reads
them with get(). Double-clicking selects the same words as before.
Test: render words_and_lines_around_a_glyph, on two lines with a blank
glyph, a word gap and a line break.
* Issue #295: three clicks select the line, four clicks the page
With the Select tool, quick clicks on text widen the selection like
Acrobat: two clicks select a word (as before), three the line, four or
more all the text on the page. A third click used to clear the
selection.
egui counts only up to triple clicks and reports a fourth quick click
as another triple, so the view keeps its own count of the current run.
A single click resets it and a pause ends it.
With the markup tools (Highlight and the like) and Redact, a
double-click already turns the word into a mark at once. They stop at
the word, so more clicks don't stack line or page marks.
Tests: ui-egui quick_clicks_select_a_word_then_the_line_then_the_page
and quick_clicks_with_the_highlighter_mark_only_the_word.
parity: view.text-select cites the tests. The notes say what is
missing: paragraph selection, and Ctrl/⌘+A across the whole document
(it selects the current page).
* Clippy: as_chunks in the signature preview test (follow-up to #355)
Clippy 1.99 rejects chunks_exact with a constant chunk size
(chunks_exact_to_as_chunks), so `cargo clippy --workspace --all-targets
-- -D warnings` fails on main since #355. Use as_chunks, as #360 did
for the test above it.
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
#260 (page 2): text in a Japanese CID font that the PDF doesn't embed
(HeiseiMin-W3 with UniJIS-UCS2-H, Adobe-Japan1) drew nothing, while
MuPDF, Poppler, PDFium and Acrobat draw it in a Mincho face. hayro asks
its font resolver for a substitute with the CID collection in the
query, but the renderer never installed one, so hayro's default
answered with a Latin standard-14 font that has no Japanese glyphs.
Release builds have embedded Japanese faces from craft-fonts all along;
the renderer just didn't use them.
RenderConfig::settings now installs a resolver: for an Adobe-Japan1
fallback query it returns a craft-fonts face (Shippori Mincho for
Mincho names, BIZ UDPGothic Regular or Bold for Gothic names, the
descriptor's serif flag otherwise); every other query keeps hayro's
default. hayro already maps such fonts through the Adobe-Japan1-UCS2
CMap to Unicode and the face's cmap. Without craft-fonts nothing
changes. Other CJK collections are untouched: the pinned craft-fonts
has only Japanese faces, and its later Chinese face is Noto CJK, which
AGENTS.md §1.1 rules out. pdfcraft-render now depends on pdfcraft-fonts
(L3 to L2).
Tests: raster::tests::non_embedded_japanese_cid_fonts_draw_with_a_craft_fonts_face
(with CRAFT_FONTS_DIR: 日本 in HeiseiMin-W3 and HeiseiKakuGo-W5 now
draws; it drew 0 dark pixels before; skips without craft-fonts, as the
Linux CI job has them) and raster::tests::japanese_font_names_pick_mincho_or_gothic.
The parity entry view.render-cid-fonts cites the test and notes what is
still missing.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
ISO 32000-2 §8.5.4: W and W* mark the current path to become the clip
once the path-painting operator that ends it has painted it with the old
clip, and n is only one such operator. The vendored interpreter applied
the clip in n alone. After S, s, f, F, f*, B, B*, b or b* it was never
applied, so everything painted afterwards ignored it, and it stayed
pending until some later, unrelated n (even after Q) clipped content
that should show. Issue #260's test file uses `W* S` on pages 3, 5 and
9: a shading meant for a circle flooded page 3, an image meant for a
rounded rectangle and nested clips on 5 and 9 spilled out.
Every painting operator now applies a pending clip after painting and
clears it (apply_pending_clip, shared with n). The now unused
Context::clip getter is removed.
Test: raster::tests::clipping_path_applies_after_any_painting_operator.
W and W* with each of n, S, s, f, F, f*, B, B*, b, b*: the later red fill
stays inside the square, and a stroking operator's outer half still
shows outside it (painted under the old clip). A clip used by S doesn't
linger for a later n, inside or after q/Q. On main it fails at `W S`.
MuPDF, Poppler and PDFium agree on every case. Page 3 of #260's file now
differs from PDFium by 0.84/255 on average (was 164). Vendored patch
(16).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Layers: run set-layer-visibility actions, keeping radio-button groups
The engine can apply a set-layer-visibility action (SetOCGState,
ISO 32000-2 §12.6.4.13) to a document's layers: each ON, OFF or Toggle
in order, naming the layer by its optional content group, so a toggle
sees the state the changes before it left. With /PreserveRB (the
default), a layer turned on turns off the other layers of its
radio-button groups, now read from /OCProperties /D /RBGroups; turning
one off leaves them alone. The renderer is rebuilt once per action, not
once per layer.
Like the Layers panel, this changes the view, not the document.
Tests: engine set_layer_actions_toggle_in_order_and_keep_radio_groups
and set_layer_actions_change_what_renders (pixels before and after),
render radio_button_layer_groups_are_read.
* Links: run set-layer-visibility actions
Clicking a link whose action is SetOCGState showed "SetOCGState actions
run in the JavaScript engine (M6)" and did nothing. Links now read the
action (/State, also when indirect, and /PreserveRB) and clicking runs
it; the Layers panel follows. Hovering shows "Set layer visibility"
(translated in the ja, es, zh-hans and zh-hant catalogs), and doc_info
describes the action in its link list.
Tests: render links_read_set_layer_actions, automation
doc_info_describes_set_layer_links and ui-egui
links_set_layer_visibility.
* Forms: push buttons run set-layer-visibility actions
A push button whose mouse-up action is SetOCGState counted as having no
action, so clicking it did nothing. Buttons now read the action as
links do, and clicking runs it.
Tests: forms push_buttons_read_set_layer_actions and ui-egui
push_buttons_set_layer_visibility.
parity: edit.action-set-ocg-state is partial. The notes say what is
missing: creating the action, bookmarks and other triggers, /Next
chains, JavaScript ocg.state, and the Layers panel keeping radio-button
groups itself.
* French: Set layer visibility
The French catalog is coverage-checked since #291.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ISO 32000-2 §12.5.5 maps an appearance onto its annotation with a matrix
A that takes the lower-left and upper-right corners of the transformed
appearance box to those of /Rect. The vendored interpreter scaled by
/Rect size over box size but translated by /Rect.x0 - box.x0, which is
only right when the box starts at the origin or the scale is 1. Any
other appearance was drawn shifted by (scale - 1) × box origin, partly
outside its /Rect: /BBox [10 10 30 30] in /Rect [10 10 60 40] painted
x 25..74, y 55..84 instead of x 10..59, y 60..89 (device pixels).
MuPDF, Poppler, PDFium and pdf.js all fill /Rect exactly.
The translation is now /Rect.x0 - scale × box.x0 (and likewise for y).
Boxes at the origin, and boxes as large as /Rect (PdfCraft's own
appearances use /BBox = /Rect), map exactly as before. A box that
collapses to a line still yields a non-finite matrix and is skipped.
Test: raster::tests::annotation_appearance_box_maps_onto_rect. An offset
/BBox, an offset made by /Matrix and a negative-origin /BBox now fill
exactly /Rect (on main the first is drawn at (25, 55)-(74, 84)); a box at
the origin and an offset box as large as /Rect are unchanged. Vendored
patch (15) in vendor/README.md (after (14) in #276).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The vendored hayro-interpret reads /FirstChar, /LastChar and /Widths of
every simple font (TrueType, Type 1, Type 3, standard 14) in read_widths.
Three defects there, each checked against MuPDF, Poppler, PDFium and
pdf.js on synthetic files:
- One entry was pushed per code below /FirstChar before any width was
read. 50,000,000 already took 276 MB; 4294967295 asks for 34 GB, and
a larger value aborts on allocation (AGENTS.md §4: cap allocations
sized by the file). Simple-font codes are single bytes, so only codes
0 to 255 now get an entry.
- An entry that isn't a number (null, a name) ended the array, so every
later code lost its width and glyphs piled up. It now leaves just that
code missing (/MissingWidth); the four renderers do the same.
- A /LastChar below /FirstChar, or a /FirstChar so large the count
overflowed, made read_widths fail, so the font didn't load and a
standard font stood in for the embedded program. The font now loads
with no widths from the array; the four renderers keep the font too.
Valid /Widths read exactly as before, including explicit zero widths,
which all four renderers and Apple Quartz (sips) honour literally.
Tests (a Type 3 font, so no font program is needed):
raster::tests::malformed_simple_font_widths_keep_later_widths_and_the_font
fails on main for [null 600], [/W 600], /LastChar < /FirstChar and
/FirstChar 4294967295; raster::tests::huge_simple_font_first_char_terminates
renders /FirstChar 4294967295 /LastChar 4294967295 quickly (not run
against the old code: it would exhaust memory). Vendored patch (14).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A Highlight annotation (ISO 32000-2 §12.5.6.10) whose /AP is missing, as
many older and generated files have, was not drawn at all: the vendored
interpreter only draws annotations through their normal appearance.
MuPDF, Poppler and PDFium all draw such highlights.
When there is no usable /AP /N, the interpreter now fills the
/QuadPoints quadrilaterals with /C at /CA opacity, blended with
Multiply, the way pdfcraft-annot draws PdfCraft's own highlights. It
uses the interpreter's own fill path, so clipping, layers and the device
see an ordinary fill. pdfcraft-annot itself can't be reused here: render
and annot are both L3, and it builds pdfcraft-cos objects while hayro
reads hayro-syntax ones. Nothing changes when an /AP exists. Malformed
/QuadPoints (not a non-empty multiple of eight numbers) and an absent,
empty or malformed /C draw nothing, as pdfcraft-annot does. Hidden,
NoView and "Hide all comments" are checked before this point and still
apply. Only Highlight changes; the source PDF is not modified.
Test: raster::tests::highlights_without_an_appearance_are_drawn. On
main the quad stays white; now it is yellow over white and black over
black (Multiply), two quads leave their gap white, /CA 0.5 and gray /C
match MuPDF, Poppler and PDFium. An /AP still wins, and ten malformed or
hidden variants draw nothing. Vendored patch (13) in vendor/README.md;
the parity entry view.render-annotation-appearances cites the test and
notes that other comment types without /AP are still not drawn.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ISO 32000-2 §12.5.5 maps an appearance onto its annotation by transforming
/BBox with /Matrix and scaling the resulting box onto /Rect. When /BBox has
zero width or height, or /Matrix has a zero row or column, that box is a
line or a point, and the vendored interpreter divided the /Rect size by
zero. The infinite or NaN matrix then reached the device with every path,
clip and glyph the appearance drew (4 of 4 drawing calls in the test).
hayro's rasteriser happens to drop such geometry, so pages looked right,
but nothing guaranteed it; AGENTS.md §4 asks for division by zero and NaN
to be guarded where they arise. Clipped to a collapsed box the appearance
shows nothing (MuPDF and Poppler draw nothing for these fixtures either),
so the appearance is now skipped, as is one whose mapping overflows. The
mapping of valid appearances is unchanged.
Test: raster::tests::degenerate_annotation_appearances_are_skipped. A
recording device sees non-finite geometry on main for a zero-width,
zero-height and point /BBox and for a /Matrix with a zero row or column;
none after. Each page renders blank; a valid appearance still fills its
/Rect and nothing outside it. Vendored patch (12) in vendor/README.md.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
`comment_add` converted a note's or attachment's `at` (the icon's top-left
as displayed) as a single point through `PageInfo::view_to_user`, and the
engine's `rect_for` extends the icon square from `at` toward +x/-y in user
space. On an unrotated page the displayed and user-space top-left are the
same corner; under `/Rotate` they are different corners of the square, so
the icon landed one icon width (20 pt) away from the requested point.
`icon_anchor` converts the whole displayed square to user space and takes
its user-space top-left, so the `/Rect` covers the requested square under
any rotation. `PageInfo::view_rect_to_user` is the render-side counterpart
of `view_rect_to_quad`, and `NOTE_SIZE` is re-exported from the engine.
Test: `note_icons_anchor_at_the_requested_corner_on_rotated_pages` adds a
note and an attachment on pages rotated 0/90/180/270 and asserts the
`comment_list` rectangles; before the fix the 90° page reports
[52, 72, 72, 92] for a note asked for at [72, 72].
Co-authored-by: wstock <1394687+wstock@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
An A4 page (595.28×841.89 pt) rendered 595×841 at 72 dpi, where
pdftoppm gives 596×842: the whole-page render passed hayro no size, and
hayro floors the scaled page size. Content touching the right or bottom
edge lost its last partial pixel at every resolution.
- render: new device_pixels(pt, scale) rounds up (ignoring float noise
up to 1/100 px, so Letter at 150 dpi stays 1275 px). The untiled
render passes it to hayro as the width and height, still within
MAX_SIDE. The partial edge pixels are anti-aliased against the white
page, as the clip to the crop box already does.
- ui-egui: the canvas tile grid uses the same helper instead of
round(), so tiled pages get tiles for the partial row and column too.
- No change to vendored hayro.
Regression test: a fractional MediaBox renders 596×842 with partly
covered edge pixels, 2481 px wide at 300 dpi, and a corner tile matches
the whole-page render. It failed (595×841) before the fix.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
hayro-syntax recovered a stream with a wrong /Length by scanning ahead
for the bytes `stream`. An indirect appearance-state dictionary
(`<< /Yes … /Off … >>`, as Quartz writes a checkbox's `/AP /N`) was
read as whatever stream came later in the file. That dictionary has no
`/BBox`, so the appearance was dropped and the widget was skipped.
`stream` now has to be the next token. A wrong `/Length` still
recovers, and the renderer already selects the `/AS` state once `/N`
is a dictionary.
Fixes#180
Co-authored-by: Eri Bastos <eb@fluxmail.xyz>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The app, crates, binaries, bundle id, packaging, icons, screenshots and docs are now
PdfCraft / pdfcraft (repo storytold/pdfcraft). On first launch the desktop app moves the
PrintCraft settings and crash-recovery folders to the new name, and both apps read settings
saved under the old storage key, so upgrades keep recent files, preferences and unsaved work.
The signing test fixtures keep their PrintCraft subject names (they are binary).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
XFA forms aren't read yet, and opening one gave no hint: a dynamic XFA form
showed only its "Please wait…" placeholder page, and a static one looked like
an ordinary form though Acrobat shows its XFA data in preference to the
fields that get filled.
- render: DocInfo.xfa (Static / Dynamic) from /AcroForm /XFA; dynamic when
/NeedsRendering is true or there are no fields. Exported for doc_info
(automation and CLI) as "xfa".
- ui: the notice bar explains each case (above the form-fields notice); a
static form keeps its Highlight fields button.
- Tests: tests/xfa.rs (dynamic with and without /NeedsRendering, static,
ordinary forms, a malformed /XFA). Parity notes for form.xfa-static and
form.xfa-dynamic (still planned: this explains XFA, it doesn't support it).
Reported by @Sudachi-dev.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nightly fuzz job found a Type 1 font that hung hayro: read-fonts 0.39
(through skrifa 0.42) looked ahead after every integer by parsing the next
token, which looked ahead again, recursing through a whole run of numbers.
A long run hung; 20,000 numbers overflowed the stack and aborted the
process. Upstream fixed the tokenizer; the vendored hayro-interpret now uses
skrifa 0.47 (read-fonts 0.44), with its one API change. The finding finishes
in 0.5 s. Synthetic regression test (it aborts with the old skrifa).
Corpus sweep: 0 regressions; text oracle unchanged (median F1 0.980, mean
0.866, every per-file score identical).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nightly fuzz job found a JBIG2 symbol dictionary that decoded for
minutes: each bitmap side may be 65535, and up to 65535 symbols are decoded
pixel by pixel. Vendored hayro-jbig2 0.3.0 with two bounds: at most 2^28
pixels per bitmap (also the page bitmap, halftone grid and grayscale image),
and at most 2^26 symbol pixels per dictionary. The finding now finishes in
about a second. Synthetic regression test (a 65535 x 65535 page and region
behind a small /Width and /Height); the corpus sweep shows 0 regressions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every hang the nightly fuzz job found (11 findings, 8 source files) now
finishes in 0.1-2 s; each cause has a synthetic regression test that times
out without its fix.
- cos (7 findings, our own parser): a stream whose /Length pointed at
itself, with no endstream, made every parse attempt fail and each attempt
reload the same object: ~2^32 parses. Objects being loaded are a seen-set,
/Length values are memoised per load, and the shifted-header retry only
runs when the header is shifted.
- hayro-syntax (3): deciding where an inline image's data ends re-parsed the
rest of the stream, nesting through every inline image; 50 images ran for
over ten minutes. The look-ahead no longer nests and is bounded (256).
- hayro-interpret (1): a Type 3 glyph showing eight glyphs of its own font,
sixteen deep. Nested paints share a per-page budget (50,000).
The pdf.js corpus sweep matches its baseline (963 ok, same 20 known).
xtask check writes its report under CARGO_TARGET_DIR too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second full nightly fuzz run on main found two more:
- hayro-syntax computed a Flate/LZW predictor row as columns × bits in
wrapping arithmetic: /Columns 9223372036854775807 asked for 2^61 bytes and
aborted on any machine. Row widths are now saturating usize (colors × bits
also overflowed u8), a zero width is refused (it divided by zero), and a
row longer than the data is not allocated (empty output, as before).
- hayro stroked /LW 9223372036854775807 as given, and stroke expansion
allocated 10 GB. Widths are clamped to 4 × (canvas width + height) device
pixels, which covers the same pixels.
Both findings now run in 8 MB. Synthetic regression test in render.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first full nightly fuzz run on main (now that the runner survives) found
two 4 GiB allocations, caught by the 4 GiB child cap:
- an inline stencil mask claiming /W 4294967295 was decoded in hayro-interpret
before the renderer's MAX_IMAGE_PIXELS check could skip it (locally 9.6 GB,
42 s). Vendored patch image_size_ok: images and masks over 2^28 pixels are
not decoded.
- a CCITT image whose /DecodeParms claimed /Columns 4294967295 (with a small
/Width) sized the decoder's line buffers (locally 4.3 GB). Vendored patch
ccitt_size_ok in hayro-syntax: at most 2^20 columns and 2^28 pixels.
Both findings now run in 7 MB. Synthetic regression test in render.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fuzzed inputs drove one check-one process to 2-6 GB (the 16 GB CI runner
running two of them was killed every night). Three causes, each with a test:
- lopdf (bootstrap inspector) sized PNG predictor rows from /Columns before
reading data: /Columns 4294967295 allocated two 4 GiB rows. Vendored lopdf
0.45.0 refuses rows longer than the data (and uses saturating row widths);
its load-time stream decoding also gets a 256 MiB limit (it had none).
- hayro sized a tiling pattern's cell pixmap from /XStep x scale, bounded only
by u16: a step far beyond /BBox allocated 2 GB. The vendored patch keeps
the cell within 3000 px a side, lowering the resolution instead.
- cos decoded cross-reference streams up to the general 1 GiB limit; they now
stop at what 8,388,607 objects' rows can hold (~200 MB at most), and the
section is reconstructed (logged) instead.
On the nightly's seed (41) the worst captured input fell from 6.2 GB to 6 MB,
and a 300 s fuzz run had no process above 300 MB (before: many at 2-5 GB).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The text reading order uses `let … else`, `while let` and `get_mut` instead of `expect`, and
named destinations bind their key with an or-pattern instead of `unreachable!`. The crate now
denies the panic lints.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stable clippy 1.99 denies chunks_exact/chunks_exact_mut with a constant
size (32 sites), which failed the test job on main and every open PR.
Behaviour unchanged; MSRV 1.90 has as_chunks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
