Files
pdfcraft/vendor/hayro-syntax
skylar taylor-barrickandBrandon Thomas dbd0748177 Reduce large-PDF memory growth (#307) (#419)
* M14: bound large-PDF memory use (#307)

Inspect large documents lazily, share parser/resource metadata, bound full-save
and raster retention, and cache signature candidate discovery safely.

Preserve upstream rendering deduplication, pixel alignment, signature previews,
and organizer insertion behavior, with regression coverage for their interaction.

Validated with 1,336 workspace tests, 193 parser tests, strict workspace Clippy,
30 WASM crate checks, 1x/2x GUI protocols, and same-PDF workflow comparisons.

* cos: drop the writer cache-policy probe and its test-only hooks in production paths

* cos: the temporary reader keeps parsed objects until a count or owned-bytes budget

Since #552 a stream read from the file is a view into the input, so clearing the whole parse
cache after every read saved little memory and cost most of the full-save slowdown (every
object parsed twice, indirect /Length objects re-parsed per stream). Clear only past
OBJECTS entries or STREAM_BYTES of data of the cache's own (decrypted streams, strings).

* cos: the writers refuse a document opened with a stream limit (read-only inspection)

* cos: a full save fails rather than writing null when an object reads differently the second time

* render: parsing is not timed by the watchdog; timing tests with wide margins

A slow first parse (a huge or damaged file) made the whole pool fail every page for its
lifetime. Parsing, shared or private, is now its own untimed stage, as on main before the
shared parser; rendering is timed from when parsing ends. The init-timeout tests become one
deterministic test (a 2 s parse against a 10 ms limit, shared and private). Watchdog tests
use limits and injected delays ~10x apart instead of 25-50 ms against real renders. Also:
saturating retired-worker count, main's stop-flag test runs on private parsers (the first
shared timeout is a private retry), set_queue documents that unlisted results are dropped,
and retire() documents the permanent per-worker-parser fallback.

* render: page numbers in the lazy page tree use checked conversion

* render: the lazy inspector is compared with lopdf on named destinations and object streams too

* ui: on-screen pages always render; pages turned back to stay cached; stable thumbnail tags

- Visible pages and tiles are always admitted and uploaded (the viewport bounds them); the
  byte and count limits decide only how much prefetch fits. A HiDPI two-up view or a 5K
  viewport left pages on 'Rendering...' for good.
- Pages no longer demanded stay cached, nearest the current page first, while the 128 MiB
  allowance and the 24-page count have room: paging back no longer re-renders.
- Thumbnail density follows the demand count rounded up to a power of two, so rows scrolling
  in and out of an organize grid don't change every thumbnail's tag and re-render it.
- A suspended (background) tab drains its pool's finished results instead of holding them.
- Organize drag-and-drop ends on the primary button only.
- Tests: the stale-result test uses valid pixels and checks nothing was uploaded; the
  organizer wheel test runs 40 steps; regressions for the first three fixes.

* vendor: mark the hayro-syntax lock and SegmentList patches and list them as (8) and (9); clippy 1.99 as_chunks in a raster test

* render: clippy 1.99 — contains over iter().any() in a raster test

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-09 14:38:01 -04:00
..

hayro-syntax

Crates.io Documentation

A low-level library for reading PDF files.

This crate implements the Syntax chapter of the PDF reference, and therefore serves as a very good basis for building various abstractions on top of it, without having to reimplement the PDF parsing logic.

This crate does not provide more high-level functionality, such as parsing fonts or color spaces. Such functionality is out-of-scope for hayro-syntax, since this crate is supposed to be as light-weight and application-agnostic as possible.

Functionality-wise, this crate is therefore close to feature-complete. The main missing feature is support for password-protected documents. In addition to that, more low-level APIs might be added in the future.

The crate is no_std compatible but requires an allocator to be available.

Example

This short example shows you how to load a PDF file and iterate over the content streams of all pages.

use hayro_syntax::Pdf;
use std::path::PathBuf;

// First load the data that constitutes the PDF file.
let data = std::fs::read(
    PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../hayro-tests/pdfs/custom/text_with_rise.pdf"),
)
.unwrap();

// Then create a new PDF file from it.
//
// Here we are just unwrapping in case reading the file failed, but you
// might instead want to apply proper error handling.
let pdf = Pdf::new(data).unwrap();

// First access all pages, and then iterate over the operators of each page's
// content stream and print them.
let pages = pdf.pages();
for page in pages.iter() {
    let mut ops = page.typed_operations();

    while let Some(op) = ops.next() {
        println!("{op:?}");
    }
}

Safety

When the unsafe feature is disabled, there is only use of unsafe in hayro-syntax. We also unconditionally use the smallvec crate which uses unsafe internally, but that's it.

For better performance, it is strongly recommended to enable the unsafe feature, which does result in slightly more unsafe code being, but does give better performance:

  • We will use the flate2 crate for decoding flate streams.
  • SIMD will be enabled to accelerate decoding of images.
  • memchr will be used to accelerate some of the fallback parsing code.

Features

The supported features include:

  • Parsing xref tables in all its possible formats, including xref streams.
  • Best-effort attempt at repairing PDF files with broken xref tables.
  • Parsing of all objects types (also in object streams).
  • Parsing and decoding PDF streams.
  • Iterating over pages as well as their content streams in a typed or untyped fashion.
  • The crate is very lightweight, especially in comparison to other PDF crates.

Limitations

  • There are still a few features missing, for example, support for password-protected PDFs. In addition to that, many properties (like page annotations) are currently not exposed.
  • This crate is for read-only processing, you cannot directly use it to manipulate PDF files. If you need to do that, there are other crates in the Rust ecosystem that are suitable for this.

License

Licensed under either of

at your option.