Patch 38 now also replaces the /ColorSpace of a shading that is
DeviceGray, DeviceRGB or DeviceCMYK with the default for it, whether the
shading is painted by sh or as a shading pattern. Poppler and MuPDF draw
these in the default too. The shading's cache key includes the default,
as the image's does.
The forms test also covers a form whose /ColorSpace has no default: it
draws in its page's, as in Poppler and MuPDF.
hayro-interpret cached the colour space built from an /ICCBased profile
stream under the hash of the stream's dictionary bytes alone. Two
different profiles with the same /N and /Length (any two uncompressed
RGB profiles of one size) shared whichever was read first, so colours in
the second space were drawn through the first profile for the rest of
the document. Found while fixing #1201.
The space is now keyed by what its profile is read from (IccProfileKey):
the stream object a space names, looked up without reading the stream,
and on a miss the old dictionary key plus the stream's own bytes
(decrypted, not decoded), so byte-for-byte copies in other objects still
share one parsed profile and a lookup never decodes. A stream written
directly in the colour space array reports the object holding the array,
as every other stream written there does; it is recognised by its
dictionary lying within the array's bytes and keyed by content only.
InterpreterCache::icc_profile_reads counts the profiles read, so the
tests can check that the cache still hits. Tests: two synthetic RGB
profiles of one length (linear sRGB, and red and blue swapped) as
objects, a reused object, a copy and direct streams on one page.
A resource dictionary's DefaultGray, DefaultRGB and DefaultCMYK colour
spaces replace the device space content selects (ISO 32000-2 §8.6.5.6).
hayro ignored them: g/G, rg/RG and k/K set the device space directly,
cs/CS resolved the device names before looking at the resources, and
images took their /ColorSpace as written. Pages that calibrate device
colour this way, common in PDF/A, came out in the wrong colours.
The interpreter now looks the defaults up once per content window
(DefaultColorSpaces), in the resources and those they inherit, so a form
XObject without its own draws in its page's. They replace DeviceGray,
DeviceRGB and DeviceCMYK selected by the colour operators, by cs/CS (by
name or through a /ColorSpace resource that is a device space), and as
the /ColorSpace of image XObjects and inline images. A default that is a
Lab, Indexed, Pattern or device space, has another number of components
or is not a colour space is ignored, and device colour is drawn as
before. Device spaces nested in other spaces, shading dictionaries,
masks and transparency group spaces are unchanged. The decoded-image
cache keys a decode by the default it was drawn in, since one image can
be drawn with and without one.
Part of #1201.
* Heavy pages: cull off-tile images, cache decodes, preview backdrops, calmer queues
A large CAD sheet (page 24 of the benchmark file: ~5050 pt wide, 2104
images, 9 MB of content) cost a full page interpret per tile: 4.3 s for a
1024 px tile at 1000%, 10.5 s for the whole page, and a storm of renders on
every pan frame. Tiles now cost 0.6 s, the sheet is covered by a handful of
large tiles, and gestures no longer queue work faster than workers finish it.
- Vendored hayro patch (10): skip images whose stream-size box misses the
canvas before decoding them, so a tile no longer decodes, resamples and
rasterizes every image on its page. Pixel-identical; the skip regression
test now covers images too.
- Vendored hayro-interpret patch (23): decoded image pixels shared across
renders through the interpreter cache, keyed by stream and decode target
size within a 64 MiB budget.
- Preview renders (hayro patch 11, RenderRequest::preview through to the
device): backdrops, thumbnails, grid cells and signature-drag backgrounds
skip images under 16 device px. Tiles, whole pages, print and export always
draw everything. Previews may differ by those pixels; finals don't.
- Viewer: zoom/pan settle holds apply only to pages whose raster proved slow
(slow_pages, 500 ms), so fast pages render with no added latency; tile sides
grow with zoom and page size (giant sheets need ~8 tiles, not ~28), queued
center-first and keyed by exact coordinates; Edit-text outlines capped.
- vendor/README.md records the new patches with their tests.
Co-Authored-By: Muse Spark <noreply@meta.com>
* Decoded-image cache: key every decode input, survive a poisoned lock
The key was the dictionary's bytes and the target size, so two images with byte-identical
dictionaries (uncompressed, one size) shared pixels, and an image drawn under a transfer
function reused pixels decoded without it. DecodedImageKey now holds the dictionary's place
in the document (its data follows it), its bytes and object, mask/stencil/raster output,
interpolation, the target size and preview or final quality; images under a transfer
function and inline images (named colour spaces, dictionaries in content streams) are not
cached, and an image over the budget is never kept. A poisoned lock decodes uncached
instead of panicking. Soft masks: the per-renderer cache can't carry a preview mask into a
final render (documented). Regression tests for identical dictionaries, preview then final
through one cache, and transfer functions.
---------
Co-authored-by: Muse Spark <noreply@meta.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* render: cancel obsolete in-flight requests
* #623: recheck cancellation after busy publication
* #623: record rendering session in roadmap
* M2: a request cancelled during the parse doesn't leave its worker unable to render
The abort was cached as the worker's parse result, so every later request on that worker failed as
unparsable. Also drops ROADMAP edits that described other PRs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* M3: harden render and CLI failure surfaces
Make content truncation observable, reject unknown CLI options, expose deterministic render work counters, and scope render failures to exact requests. Refuse silently reduced whole-page export rasters while preserving tiled rendering, and add heavy-vector and CLI regression coverage.
Co-authored-by: Codex <noreply@openai.com>
* Render: remember a failed render by page and kind again
A key with scale and tag made every zoom level and tile a new key: a
pathological page tied up a worker for the watchdog limit per request
(exhausting worker replacements) and was re-parsed each time. The
repeated-parsing check asks for a new generation and scale again.
* Export: pages too large at the chosen dpi export at the most the renderer allows, and say so
#669 made every Exporter refuse oversized whole-page renders, which
reached File > Export to image, doc_export_* and OCR: at 600 dpi any page
side over 13.65 in (Legal, A3, Tabloid) failed instead of exporting. The
exporter clamps again; Exporter::dpi_used reports the resolution a page
is drawn at, which the Export dialog's message and doc_export_png's
lower_dpi list mention. A source with reject_oversize still refuses
(CLI render keeps its strict config).
* doc_export_images: describe lower_dpi
* Render: ContentTruncated is reported once per page; the hayro-interpret patch is recorded
Nested forms, Type 3 glyphs and patterns are interpreted again and each
reported the exhausted budget; pdfcraft-render now keeps one. The warning
is part of hayro-interpret patch (18) (the page content budget): marked
in the code and described in vendor/README.md.
* CLI render: print render warnings (skipped content) to stderr
* CLI: the option check refuses unknown options only; commands report missing values themselves
text's own --page parsing (and its test on main) says what a page value
must be; the generic "needs a value" pre-empted it.
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M14: bound large-PDF memory use (#307)
Inspect large documents lazily, share parser/resource metadata, bound full-save
and raster retention, and cache signature candidate discovery safely.
Preserve upstream rendering deduplication, pixel alignment, signature previews,
and organizer insertion behavior, with regression coverage for their interaction.
Validated with 1,336 workspace tests, 193 parser tests, strict workspace Clippy,
30 WASM crate checks, 1x/2x GUI protocols, and same-PDF workflow comparisons.
* cos: drop the writer cache-policy probe and its test-only hooks in production paths
* cos: the temporary reader keeps parsed objects until a count or owned-bytes budget
Since #552 a stream read from the file is a view into the input, so clearing the whole parse
cache after every read saved little memory and cost most of the full-save slowdown (every
object parsed twice, indirect /Length objects re-parsed per stream). Clear only past
OBJECTS entries or STREAM_BYTES of data of the cache's own (decrypted streams, strings).
* cos: the writers refuse a document opened with a stream limit (read-only inspection)
* cos: a full save fails rather than writing null when an object reads differently the second time
* render: parsing is not timed by the watchdog; timing tests with wide margins
A slow first parse (a huge or damaged file) made the whole pool fail every page for its
lifetime. Parsing, shared or private, is now its own untimed stage, as on main before the
shared parser; rendering is timed from when parsing ends. The init-timeout tests become one
deterministic test (a 2 s parse against a 10 ms limit, shared and private). Watchdog tests
use limits and injected delays ~10x apart instead of 25-50 ms against real renders. Also:
saturating retired-worker count, main's stop-flag test runs on private parsers (the first
shared timeout is a private retry), set_queue documents that unlisted results are dropped,
and retire() documents the permanent per-worker-parser fallback.
* render: page numbers in the lazy page tree use checked conversion
* render: the lazy inspector is compared with lopdf on named destinations and object streams too
* ui: on-screen pages always render; pages turned back to stay cached; stable thumbnail tags
- Visible pages and tiles are always admitted and uploaded (the viewport bounds them); the
byte and count limits decide only how much prefetch fits. A HiDPI two-up view or a 5K
viewport left pages on 'Rendering...' for good.
- Pages no longer demanded stay cached, nearest the current page first, while the 128 MiB
allowance and the 24-page count have room: paging back no longer re-renders.
- Thumbnail density follows the demand count rounded up to a power of two, so rows scrolling
in and out of an organize grid don't change every thumbnail's tag and re-render it.
- A suspended (background) tab drains its pool's finished results instead of holding them.
- Organize drag-and-drop ends on the primary button only.
- Tests: the stale-result test uses valid pixels and checks nothing was uploaded; the
organizer wheel test runs 40 steps; regressions for the first three fixes.
* vendor: mark the hayro-syntax lock and SegmentList patches and list them as (8) and (9); clippy 1.99 as_chunks in a raster test
* render: clippy 1.99 — contains over iter().any() in a raster test
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
aiongg