* D5: README for cos and engine, and an honest cos status line
Thirteen crates have no README, although the session protocol tells every session to read the README of the crate it is touching. These are the two biggest: cos is where every hostile file lands first, and engine is the whole API surface a frontend gets.
Each states its layer and the rules that bind it, sketches the API against the real signatures, and ends with what is deliberately not done. The cos README spells out the clone-is-a-snapshot property and why both writers return bytes rather than taking a sink; the engine README spells out the four-step editing model that makes what you see what Save writes.
Also corrects the module doc on cos/src/lib.rs, which still said encryption and object-stream output for full saves were not done. Both shipped: security.rs handles R2-R6 with crypt filters and /EFF, and SaveOptions::object_streams defaults to true.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* M14: let neutral Windows locales fall through to display language
Co-authored-by: Codex <codex@openai.com>
* pdfcraft-cli --help lists `mcp` only when the build has it
Follow-up to #540: a build without the `mcp` feature (`--no-default-features`) still listed
`pdfcraft-cli mcp` in its usage, and `pdfcraft-cli mcp` then said "unknown command".
USAGE is now put together with `concat!`, and its `mcp` lines come from a macro that is empty
without the feature. In such a build `pdfcraft-cli mcp` says the MCP server was left out instead
of calling it an unknown command.
- tests/help.rs: usage_lists_mcp_only_when_built_with_it (runs in both feature sets) and
mcp_without_the_feature_says_it_was_left_out (without the feature)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* engine README: the execution plan is local-only, so don't link a path that doesn't exist
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Streetblock <github@streetblock.de>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: lloyd <lloyd@stackworx.io>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M14: bound large-PDF memory use (#307)
Inspect large documents lazily, share parser/resource metadata, bound full-save
and raster retention, and cache signature candidate discovery safely.
Preserve upstream rendering deduplication, pixel alignment, signature previews,
and organizer insertion behavior, with regression coverage for their interaction.
Validated with 1,336 workspace tests, 193 parser tests, strict workspace Clippy,
30 WASM crate checks, 1x/2x GUI protocols, and same-PDF workflow comparisons.
* cos: drop the writer cache-policy probe and its test-only hooks in production paths
* cos: the temporary reader keeps parsed objects until a count or owned-bytes budget
Since #552 a stream read from the file is a view into the input, so clearing the whole parse
cache after every read saved little memory and cost most of the full-save slowdown (every
object parsed twice, indirect /Length objects re-parsed per stream). Clear only past
OBJECTS entries or STREAM_BYTES of data of the cache's own (decrypted streams, strings).
* cos: the writers refuse a document opened with a stream limit (read-only inspection)
* cos: a full save fails rather than writing null when an object reads differently the second time
* render: parsing is not timed by the watchdog; timing tests with wide margins
A slow first parse (a huge or damaged file) made the whole pool fail every page for its
lifetime. Parsing, shared or private, is now its own untimed stage, as on main before the
shared parser; rendering is timed from when parsing ends. The init-timeout tests become one
deterministic test (a 2 s parse against a 10 ms limit, shared and private). Watchdog tests
use limits and injected delays ~10x apart instead of 25-50 ms against real renders. Also:
saturating retired-worker count, main's stop-flag test runs on private parsers (the first
shared timeout is a private retry), set_queue documents that unlisted results are dropped,
and retire() documents the permanent per-worker-parser fallback.
* render: page numbers in the lazy page tree use checked conversion
* render: the lazy inspector is compared with lopdf on named destinations and object streams too
* ui: on-screen pages always render; pages turned back to stay cached; stable thumbnail tags
- Visible pages and tiles are always admitted and uploaded (the viewport bounds them); the
byte and count limits decide only how much prefetch fits. A HiDPI two-up view or a 5K
viewport left pages on 'Rendering...' for good.
- Pages no longer demanded stay cached, nearest the current page first, while the 128 MiB
allowance and the 24-page count have room: paging back no longer re-renders.
- Thumbnail density follows the demand count rounded up to a power of two, so rows scrolling
in and out of an organize grid don't change every thumbnail's tag and re-render it.
- A suspended (background) tab drains its pool's finished results instead of holding them.
- Organize drag-and-drop ends on the primary button only.
- Tests: the stale-result test uses valid pixels and checks nothing was uploaded; the
organizer wheel test runs 40 steps; regressions for the first three fixes.
* vendor: mark the hayro-syntax lock and SegmentList patches and list them as (8) and (9); clippy 1.99 as_chunks in a raster test
* render: clippy 1.99 — contains over iter().any() in a raster test
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* hayro-syntax: fix float mantissa overflow on high-precision coordinates (#199)
jsPDF outputs double-precision floating-point coordinates with 16+ decimal
places in /MediaBox. In hayro-syntax, read_inner accumulated float mantissa
digits into u64 using wrapping arithmetic. Integer mantissas exceeding
u64::MAX wrapped modulo 2^64, severely corrupting page dimensions
(e.g., 3874.96 pt became 185.61 pt, rendering as a warped ribbon).
Accumulation now uses checked arithmetic: trailing fractional digits
exceeding u64 capacity are safely ignored without incrementing decimal_shift.
This preserves full IEEE-754 f64 precision while preventing overflow for
numbers with arbitrary numbers of decimal places.
Includes:
- Unit test in hayro-syntax with reported values and a 50-digit fraction.
- Synthetic PDF regression test in pdfcraft-render verifying inspect().
- Vendored patch (6) documentation in vendor/README.md.
Fixes#199
* M2: bound page-label work and output before applying labels
Share bounded PDF Roman and repeated-letter formatters between inspection
and editing. Bound tree traversal, raw and decoded prefixes, individual
labels and aggregate label output, with checked numbering arithmetic.
Stage inspection labels before applying them and retain physical page
numbers with a warning when optional labels cannot be processed. Reject
unsupported label edits before changing the document tree.
Add synthetic limit, cycle, Unicode, numbering and atomic-edit regressions.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* Never crash: mesh shadings can't exhaust memory or run for minutes
`cargo xtask fuzz` (with the dash, JBIG2 and decompression fixes in
place) found a page whose Coons patch shading (type 6), a 147-byte
stream with 1-bit coordinates decoded onto +-40000, took the renderer
past 3 GB. hayro-interpret samples mesh shadings (types 4-7) pixel by
pixel over each triangle's device-space bounding box into a map, with
nothing but the u16 cast bounding it: up to 65536 x 65536 entries for
one triangle. A synthetic single patch passed 6 GB in six seconds; ten
thousand triangles that each cover half a 1000 x 1000 page ran for
minutes. Two rounds of review of earlier versions found more: every
patch was cut into 722 triangles (about 150 KB) before anything limited
them; a lattice mesh with `/VerticesPerRow 0` pushed empty rows forever,
and one with 4294967295 read the whole stream into one row; a mesh
painted a thousand times was triangulated a thousand times; and the
first caps cut content without a word.
Vendored hayro-interpret patch (18):
- `ShadingPattern::encode_within(area)`: the vendored hayro renderer
passes the area it will actually draw (the path's bounding box on the
canvas, which it already computes) plus a pixel for texels at a
fractional edge, and triangles are sampled only inside it, so the map
holds at most that area's pixels.
- `TriangleSampler`: at most 8 visits per pixel of that area plus 4 per
triangle in it. A triangle's bounding box is about twice its area and a
sub-pixel triangle's is at most 2 x 2 pixels, so tiling meshes and
small dense ones (a patch in a few pixels is 722 triangles) still
shade completely; overlapping ones stop.
- Coons and tensor-product patches are triangulated one at a time.
- Meshes hold at most `MAX_MESH_PATCHES` (2^16; a 200 x 200 gradient
mesh is 40 thousand) patches and `MAX_MESH_TRIANGLES` (2^20)
triangles, and a lattice needs two (ISO 32000-2 8.7.4.5.6) to 2^19
vertices per row.
- A page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25; the 200 x 200
mesh is 29 million) mesh triangles in all, reset per page like the
existing paint budget.
- Each of these cuts logs a warning.
`encode()` keeps its signature; with no area it samples within a fixed
2^24 visits.
Synthetic regression tests in render:
- `mesh_shadings_far_off_the_page_render`: a Coons patch and a triangle
mesh decoded onto +-40000 render and the rest of the page draws; a
patch on the page, the part of the huge one that covers it, and a
patch filling a 4 x 4 pt rectangle as a pattern are all shaded.
- `many_overlapping_mesh_triangles_finish`: 10,000 triangles each
covering half of a 1000 x 1000 page finish, and are drawn.
- `mesh_shadings_with_many_patches_or_no_rows_finish`: 40,000 patches
beside the page, and a lattice with `/VerticesPerRow 0`.
Red on main (out of memory, killed at 3.9 GB). Each part is pinned by a
mutation: no visit budget stalls past 60 s; no clip leaves the covering
patch unshaded; no per-triangle allowance leaves the small patch
unshaded; triangulating all patches first runs out of memory (killed at
5.8 GB); no rows check runs out of memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Never crash: decompression bombs can't exhaust the renderer's memory
The renderer's parser (vendored hayro-syntax) decoded Flate, LZW and
RunLength streams with no output limit. A 2 MB page whose content
stream inflates to 2 GB, or whose 8 x 8 image does, took `pdfcraft-cli
render` and `text` past 3 GB in under a second, and a few more MB of
file scale it further. On the desktop that is the first-page render of
an emailed PDF, and the allocation-failure abort takes every open
document with it. Our own layers already bound single streams (lopdf's
load-time decoding at 256 MiB, cos at 1 GiB), so `info` stays at a few
MB; the renderer path didn't. Random mutation almost never makes a
high-ratio stream, so `cargo xtask fuzz` doesn't find this; found by
hand while triaging the fuzz run.
Vendored hayro-syntax patch (6):
- `decode_limit` gives each filter an output limit, and Flate (both
flate2 and the fallback inflater), LZW and RunLength never write past
it (the buffer can still grow to under twice it):
`MAX_DECODED_STREAM`, 256 MiB, lopdf's figure; or, for the last filter
of an image whose size, bits per component and components are known,
twice its pixel bytes (a PNG predictor tag byte per byte at most),
capped at 1 GiB. Earlier filters in a chain produce compressed data
and get the general limit.
- `Stream::decoded_within(max)` for callers that budget.
- A page's `/Contents` array shares one `MAX_DECODED_STREAM` (naming one
bomb twenty times decoded 5 GB), reserving exactly what it appends.
Vendored hayro-interpret patch (18):
- The content a page holds at once (its contents, and the forms, tiling
patterns and Type 3 glyph procedures being painted) stays within
`MAX_PAGE_CONTENT`, 512 MiB, reset per page like the existing paint
budget. Forms nest fifty deep and each level kept its content while
the next ran: a form painting itself around a 256 MiB stream would
have held over 12 GB. Each level gives its bytes back when it is
dropped, so content painted again and again keeps drawing, and a
stream without filters (borrowed from the file) costs nothing.
- Type 0 sampled functions read every sample into a 4-byte value and
then a table entry: 256 MiB of 1-bit samples asked for over 8 GB. They
now take at most 2^21 table points and 2^23 values (the largest real
tables, DeviceN tint transforms such as 33^4 points x 3 outputs, are
1.2 million and 3.6 million), counted with checked arithmetic, and
read no further than `/Size` asks. `/Size 0` no longer underflows
building the default `/Encode`.
Three rounds of independent review found the aggregate cases, buffer
doubling past the limit, a chain/predictor shape the first image limit
would have truncated, the Type 0 amplifier, and a first, cumulative
budget that would have dropped legitimately repeated content, and a
first Type 0 cap low enough to refuse real tint transforms; all fixed
here.
Measured in a release build (render; OS peak commit):
- content stream inflating to 2 GB: 389 MB, 0.3 s
- 8 x 8 image inflating to 2 GB: 4 MB, 0.2 s
- /Contents naming it twenty times: 516 MB, 0.3 s
- a form painting itself around it: 644 MB, 0.4 s
Synthetic regression tests in render (flate2 is a new dev-dependency
there, already in the workspace):
- `decompression_bombs_stop_at_the_stream_limit`: 288 MiB bombs through
flate2, the fallback inflater (a stored block with a wrong length
check, which flate2 refuses and the fallback accepts), LZW and
RunLength each decode exactly 256 MiB, and at most 128 bytes as an
8 x 8 image.
- `page_contents_and_nested_forms_share_one_budget`: three /Contents
references decode one stream's worth; a self-painting form finishes.
- `content_painted_again_and_again_keeps_drawing`: a 2 MiB form painted
300 times draws the 300th time.
- `sampled_functions_with_billions_of_samples_are_refused`.
Ten mutations each turn them red: no limit at all; no limit in flate2,
the fallback, LZW, RunLength or for images; an unshared /Contents limit;
no page budget; no refund on drop; no Type 0 sample cap. (Taking a form
out of the budget no longer compiles: its content is the budget guard.)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M2: normalize JPEG2000 samples without overflowing
Use bounded integer sample scaling for 1-32-bit JPX images and reject empty dimensions before chunking. Cover public decoding and continued page rendering with synthetic regressions.
Co-Authored-By: OpenAI Codex <codex@openai.com>
* Never crash: decompression cut at its limit is logged, not silent; vendor README renumbered after #575
---------
Co-authored-by: philkwek <philkwek03@gmail.com>
Co-authored-by: OpenAI Codex <codex@openai.com>
Co-authored-by: James Locke <336218001+jcl-bit@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M1: stream data read from a file points into the file instead of being copied
Every stream COS loaded was copied out of the document's bytes
(`data[start..end].to_vec()`) and kept in the object cache until the document
closed. Anything that reads every object therefore left a second copy of all
stream data behind. That includes a full save, which edits needing a rewrite
(repaired files, security changes, redaction) run on every change. A full save
of a 208 MB document left 208 MB cached on it.
`Stream::raw` is now `Bytes`: owned bytes, or a range of a larger shared
buffer. Streams parsed from a document's own bytes are ranges of that buffer,
which the document keeps anyway. Decrypted and newly built streams own their
bytes as before. A stream added to a document from another document's file gets
its own copy of just its bytes (`Document::set`/`add`), so it never keeps
another whole file alive.
Results, measured with an exact heap counter:
- A full save of a 208 MB document leaves 0.2 MB cached instead of 208.2 MB.
- The first full save of a freshly opened document is about twice as fast:
10 → 5 ms at 40 MB, 38 → 21 ms at 200 MB.
Output is identical: 68 incremental and full saves of five originals,
encrypted ones included, hash the same before and after.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M1: the full-save capacity test (#523) builds its stream from Bytes
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
`write_full` can't know the output size in advance, so its buffer grows by
doubling and the result kept up to as much again unused: 28 % for a 100 MB
save, 99 % for a 5 MB one. The result is often kept for as long as a document
is open. It becomes the working bytes after edits that need a full rewrite
(repaired files, security changes, redaction) through `write_incremental`, and
it backs Save As ▸ optimized and every new document from Create, Combine,
Extract and Split.
`shrink_to_fit` at the end drops the unused capacity. The bytes are the same.
On macOS the shrink happens in place (2 µs for 100 MB, the buffer doesn't
move), and full-save times are unchanged.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`write_incremental` copied the original into a buffer with 4 KB to spare,
then appended the revision. A revision above 4 KB (rotating a few dozen pages,
an image, a long comment thread) outgrew it. The buffer was then reallocated
at twice the size: the whole original was copied a second time, and the
result kept up to the original's size again unused. The engine keeps that
result as the document's working bytes, shared with the renderer and handed
to Save, so the unused half stayed alive. A 100 MB document held a 200 MB
allocation after one such edit, and still did after saving.
The revision is now written on its own, with offsets counted from where it
will start. The result is allocated once at exactly the original's size plus
the revision's, and filled in one pass. The output is byte for byte what it
was before.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Check the effective output limit before cloning a stream with an empty
decoder chain. Share the raw guard between tolerant and strict decoding,
including empty filter arrays and already-applied Crypt filters.
Add small regressions for exceeded, exact and zero-byte limits and keep
filtered decoding behavior unchanged.
Co-authored-by: OpenAI Codex <codex@openai.com>
Reject invalid xref ranges before arithmetic or narrowing casts, allowing the existing reconstruction path to preserve intact objects and record the repair reason. Cover both tables and streams through Document::open.
Co-authored-by: storybored379 <storybored379@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The app, crates, binaries, bundle id, packaging, icons, screenshots and docs are now
PdfCraft / pdfcraft (repo storytold/pdfcraft). On first launch the desktop app moves the
PrintCraft settings and crash-recovery folders to the new name, and both apps read settings
saved under the old storage key, so upgrades keep recent files, preferences and unsaved work.
The signing test fixtures keep their PrintCraft subject names (they are binary).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every hang the nightly fuzz job found (11 findings, 8 source files) now
finishes in 0.1-2 s; each cause has a synthetic regression test that times
out without its fix.
- cos (7 findings, our own parser): a stream whose /Length pointed at
itself, with no endstream, made every parse attempt fail and each attempt
reload the same object: ~2^32 parses. Objects being loaded are a seen-set,
/Length values are memoised per load, and the shifted-header retry only
runs when the header is shifted.
- hayro-syntax (3): deciding where an inline image's data ends re-parsed the
rest of the stream, nesting through every inline image; 50 images ran for
over ten minutes. The look-ahead no longer nests and is bounded (256).
- hayro-interpret (1): a Type 3 glyph showing eight glyphs of its own font,
sixteen deep. Nested paints share a per-page budget (50,000).
The pdf.js corpus sweep matches its baseline (963 ok, same 20 known).
xtask check writes its report under CARGO_TARGET_DIR too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fuzzed inputs drove one check-one process to 2-6 GB (the 16 GB CI runner
running two of them was killed every night). Three causes, each with a test:
- lopdf (bootstrap inspector) sized PNG predictor rows from /Columns before
reading data: /Columns 4294967295 allocated two 4 GiB rows. Vendored lopdf
0.45.0 refuses rows longer than the data (and uses saturating row widths);
its load-time stream decoding also gets a 256 MiB limit (it had none).
- hayro sized a tiling pattern's cell pixmap from /XStep x scale, bounded only
by u16: a step far beyond /BBox allocated 2 GB. The vendored patch keeps
the cell within 3000 px a side, lowering the resolution instead.
- cos decoded cross-reference streams up to the general 1 GiB limit; they now
stop at what 8,388,607 objects' rows can hold (~200 MB at most), and the
section is reconstructed (logged) instead.
On the nightly's seed (41) the worst captured input fell from 6.2 GB to 6 MB,
and a 300 s fuzz run had no process above 300 MB (before: many at 2-5 GB).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
annot, compare, content, cos, export, geom, model, ocr, preflight, print and redact have no
production `unwrap`/`expect`/`panic!`/`unreachable!`/`todo!`/`unimplemented!`. The attribute
keeps it that way.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stable clippy 1.99 denies chunks_exact/chunks_exact_mut with a constant
size (32 sites), which failed the test job on main and every open PR.
Behaviour unchanged; MSRV 1.90 has as_chunks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
