* M2: stream page content past the 256 MiB cap (item 1 of storytold/pdfcraft#623) Page content was decoded whole, and the decode stopped at MAX_DECODED_STREAM (256 MiB), so the last ~10 MB of the #623 sample was dropped without a warning. Top-level page content is now read a window at a time, so the decoded content is never all in memory: - the content tokenizer reads windows and holds back an instruction that runs past one; Reader records reads that touch the end of the data, including reads made through clones, so an instruction is only decided once it has been seen whole; - the Flate decoder is resumable and keeps its own 32 KiB history, so it pauses at any output size and back-references do not depend on what the caller has kept; - page content may decode to max(256 MiB, 64 x its compressed size), counting each distinct stream once; content that reaches that budget is cut and the page reports ContentTruncated, as it did not before; - forms, tiling patterns, Type 3 glyph procedures and generic streams keep their existing budgets and caps. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> * M2: Avoid duplicate page content raw reads * M2: Document page content byte reuse --------- Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
hayro-syntax
A low-level library for reading PDF files.
This crate implements the Syntax chapter of the PDF reference, and therefore
serves as a very good basis for building various abstractions on top of it, without having to reimplement
the PDF parsing logic.
This crate does not provide more high-level functionality, such as parsing fonts or color spaces.
Such functionality is out-of-scope for hayro-syntax, since this crate is supposed to be
as light-weight and application-agnostic as possible.
Functionality-wise, this crate is therefore close to feature-complete. The main missing feature is support for password-protected documents. In addition to that, more low-level APIs might be added in the future.
The crate is no_std compatible but requires an allocator to be available.
Example
This short example shows you how to load a PDF file and iterate over the content streams of all pages.
use hayro_syntax::Pdf;
use std::path::PathBuf;
// First load the data that constitutes the PDF file.
let data = std::fs::read(
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../hayro-tests/pdfs/custom/text_with_rise.pdf"),
)
.unwrap();
// Then create a new PDF file from it.
//
// Here we are just unwrapping in case reading the file failed, but you
// might instead want to apply proper error handling.
let pdf = Pdf::new(data).unwrap();
// First access all pages, and then iterate over the operators of each page's
// content stream and print them.
let pages = pdf.pages();
for page in pages.iter() {
let mut ops = page.typed_operations();
while let Some(op) = ops.next() {
println!("{op:?}");
}
}
Safety
When the unsafe feature is disabled, there is only use of unsafe in hayro-syntax.
We also unconditionally use the smallvec crate which uses unsafe internally, but that's
it.
For better performance, it is strongly recommended to enable the unsafe feature,
which does result in slightly more unsafe code being, but does give better performance:
- We will use the
flate2crate for decoding flate streams. - SIMD will be enabled to accelerate decoding of images.
memchrwill be used to accelerate some of the fallback parsing code.
Features
The supported features include:
- Parsing xref tables in all its possible formats, including xref streams.
- Best-effort attempt at repairing PDF files with broken xref tables.
- Parsing of all objects types (also in object streams).
- Parsing and decoding PDF streams.
- Iterating over pages as well as their content streams in a typed or untyped fashion.
- The crate is very lightweight, especially in comparison to other PDF crates.
Limitations
- There are still a few features missing, for example, support for password-protected PDFs. In addition to that, many properties (like page annotations) are currently not exposed.
- This crate is for read-only processing, you cannot directly use it to manipulate PDF files. If you need to do that, there are other crates in the Rust ecosystem that are suitable for this.
License
Licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
- MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)
at your option.