Commit Graph
96 Commits
Author SHA1 Message Date
aiongg 58dc10c227 fix(cos): read a damaged real's numeric prefix in one pass
To read the longest valid prefix of a token like 1.2.3 the parser
retried the conversion on each shorter prefix, so a long digit run
followed by long junk was converted once per junk byte: quadratic in
the token. The prefix is found in one scan and converted once. A
prefix that spells infinity or NaN no longer reads as a number.
2026-10-22 17:43:03 +00:00
aiongg a374d16515 fix(cos): index the keys of a large dictionary
Dict::set searched every existing entry before inserting a key, so a
parsed dictionary of n distinct keys cost n^2/2 key comparisons: a
few megabytes of keys stalled parsing for minutes. From 32 entries a
dictionary keeps a key index, so each insert and lookup is one hash
probe. Equality and debug output still compare and show the entries
alone.
2026-10-22 17:43:03 +00:00
aiongg d5047aa4ac test(cos): build the private temp directory without an unused mut on Windows
The only mutable use of the DirBuilder is the Unix mode call, so the binding
is mutable only where that call exists.
2026-10-22 12:13:16 -04:00
aiongg b72c730a95 fix(cos): give the protect example a fresh key on every run
The example passed a constant seed to set_encryption. AES-256 derives the
file key from the seed alone, so anyone could rebuild the key and read a
file it protected without either password. The seed now comes from the
operating system's randomness through RandomState, as in the engine.
2026-10-22 12:13:16 -04:00
aiongg 7fb6ab0f1f test(cos): write the qpdf oracle files into a private new directory
The qpdf oracle test wrote into temp_dir()/pdfcraft-enc-<pid>, reusing the
directory if it existed and following any symlink inside it, so another
local account could prepare it and have the test overwrite a file of the
developer's. The directory now has an unpredictable name, is created by
the test (mode 0700 on Unix) and the files are created new.
2026-10-22 12:13:16 -04:00
aiongg 60d2d1e3f2 fix(cos): put inherited crypt-filter selectors under the new protection
A stream that named its own crypt filter (/Filter /Crypt with
/DecodeParms /Name) kept that name when the document was given new
password protection. The new /Encrypt defines only /StdCF, so the
inherited name resolved to /Identity and the stream was written in
clear into a file the user asked to protect. When saving with protection
applied by set_encryption, a selector the new handler does not define is
removed with its parameters and the stream follows the new policy.
Saving with the original handler keeps the source's selectors.
2026-10-22 12:13:16 -04:00
aiongg a08d974ac1 fix(cos): encrypt the /Contents of an annotation that carries a /ByteRange
Saving with encryption left a dictionary's /Contents in clear whenever it
had a string /Contents and an array /ByteRange, whatever else it was. A
source file could add a /ByteRange to an ordinary annotation and its
comment text was then written unencrypted. A dictionary with a /Subtype or
a /Type other than Sig or DocTimeStamp is never a signature dictionary, so
its /Contents is encrypted like any other string.
2026-10-22 12:13:16 -04:00
aiongg 9526363152 Build each crate's integration tests as one binary (#4087)
* Build each crate's integration tests as one binary

Every tests/*.rs file is its own executable that links the whole crate
graph. Move each crate's test files into tests/it/ as modules of one
tests/it/main.rs, for apps/pdfcraft, apps/pdfcraft-cli, automation, cos,
crypt, engine, render, sign and ui-egui. ui-egui's ui_group binary and
the support modules move into tests/it/ too. Crates with a single test
file keep it. The test executables go from 150 to 52, and every crate
runs the same number of tests as before.

ui-egui perf keeps its own binary (own process): its frame-time budgets
would charge each measurement for the other tests running beside it.

What the move needs:
- include_str!/include_bytes! and #[path] paths gain one "../".
- ui-egui: one GPU lock in tests/it/main.rs replaces the five per-file
  ones (content, document_links, editing, fill_sign, view), so one
  process creates one wgpu device at a time. The drop-line test in ui.rs
  and the screenshot blocks behind PDFCRAFT_*_SHOT(S) and
  CRAFT_UI_DOCKING_DIR, which render without holding it, take it too.
- pdfcraft winit_wayland_scroll: the vendored winit source it compiles
  names crate::event, so that shim lives in main.rs.
- parity evidence names the new paths.

* Name the one-binary test paths in docs and comments

- READMEs, docs, ATTRIBUTION and source comments name tests/it/<file>.rs.
- The oracle README's example filters with --test it encryption::qpdf_agrees.
- pdfcraft's GPU lock comment points at the UI tests' shared lock in
  crates/ui-egui/tests/it/main.rs.
2026-10-22 11:28:58 -04:00
aiongg 2c13fd9373 Merge r2 track t16: mixed: automation-mcp, crate:misc, crypt-security, export-convert, optimize, preflight-pdfa, save-io
# Conflicts:
#	ATTRIBUTION.toml
#	contributors/people.toml
#	crates/automation/src/comments.rs
#	crates/edit/src/tests.rs
#	crates/engine/src/tests.rs
#	parity/acrobat-features.toml
2026-10-22 02:23:58 +00:00
aiongg 7bcf25991d Merge PR #2397: Issue #2365: a rebuilt file keeps the /Info and /ID of its cross-reference stream
crates/cos/src/document.rs: #2220 moved the header scan into object_headers; it also returns the position after `obj`, and reconstruct applies this PR's /XRef candidate check (same 1024-byte window) to each header there. Tests: kept both sides.
2026-10-22 01:47:39 +00:00
aiongg 0edf07969c Merge PR #2400: Issue #2366: a trailer whose /Prev is a reference still reaches the earlier sections
crates/cos/src/document.rs: kept #2220's shifted_subsection_end and object_headers and this PR's prev_offset.
vendor/README.md: hayro-syntax row already has (22) and (23); this PR's patch and tests are numbered (24).
2026-10-22 01:46:50 +00:00
aiongg 2f224ecedf Merge PR #3006: fix(cos): write whole reals past the 32-bit integer range as reals 2026-10-22 01:46:42 +00:00
aiongg 2c4dc9158b Merge PR #2073: cos: property tests for save fidelity (part of #1187)
contributors/people.toml: union of the new entries
2026-10-22 01:46:42 +00:00
aiongg 79d843524c Integration: #2220 look up set-aside object numbers in a set
fill_xref_gaps scanned every xref oddity for each object header; a file with many offset-0 rows and many headers took quadratic time.
2026-10-22 01:46:03 +00:00
aiongg 57283a52fe Merge PR #2220: fix(cos): find objects a broken cross-reference table leaves out 2026-10-22 01:45:53 +00:00
aiongg 898a57ddc6 Merge PR #1850: fix(cos): check object-stream member numbers and offsets (#1815) 2026-10-22 01:45:49 +00:00
Dylan M. Taylor 606c560d24 Use chrono instead of hand-rolled UTC conversion (#3824)
* Use chrono for UTC/calendar conversion instead of hand-rolled copies

Replaces the hand-rolled civil_from_days/days_from_civil copies and the
leap-year/month math around them (log timestamp, PDF /Info dates, ASN.1
UTCTime/GeneralizedTime, FormCalc Date/Num2Date, the Combine modified-time
stamp, the form date picker, and the demo-PDF timestamps) with chrono as a
workspace dependency. The engine's existing chrono (clock) is lifted to the
workspace. Every output format is unchanged and pinned by tests. CLAUDE.md now
says time goes through chrono.

Follow-up to storytold/cadcraft#1785

* Guard chrono date paths: no panic on overflow, no '+' on 5-digit years

Systemic chrono follow-ups to the UTC/calendar PR:

- logging::timestamp and combine_ui::utc_stamp dropped chrono's
  From<SystemTime> (which unwraps internally and panics on a clock past
  year 262143) for the fallible DateTime::from_timestamp, reading an
  out-of-range clock as the epoch, as the old clamping code did.
- timestamp, utc_stamp, pdf_date and the xtask Stamp now write the year
  with {:04} instead of %Y, which prefixes a '+' before years past 9999;
  the old hand-rolled code wrote the plain zero-padded year.
- forms_ui calendar navigation and length, and formcalc days_in, use
  checked_add for the December year successor so y + 1 can't overflow at
  i32::MAX / i64::MAX (reachable from a parsed or persisted date).

Docstrings added to the chrono-touched civil_from_days and Time::from_unix
for the docstring-coverage gate. Tests cover the out-of-range-epoch and
10000-01-01 cases for each.

* Don't build an unrepresentable SystemTime in the far-future test

UNIX_EPOCH + Duration::from_secs(u64::MAX / 2) overflows Windows'
SystemTime and panicked before timestamp() ran, failing the Windows
test job. Build it with checked_add and only assert where it exists.

* Keep FormCalc dates exact past chrono's range

MAX_DAYS accepts day numbers up to 1e8 (years past 270000), but NaiveDate
stops near ±262143, so Num2Date(99000000) and Format fell back to
1900-01-01. days_from_civil, civil_from_days and days_in now have chrono
convert within the date's 400-year Gregorian cycle and add whole cycles
back (via i128), which gives the pre-chrono results for every year. Tests
cover accepted inputs past chrono's range on both sides of the epoch,
through Num2Date and Format, with values from the old formula.

* Bump rustls-webpki and webpki-root-certs for RUSTSEC-2026-0382/0383

rustls-webpki 0.103.15 -> 0.103.16 and webpki-root-certs 1.0.9 -> 1.0.10
(both semver-compatible, lockfile only) drop the distrusted emSign
intermediates and root. Only these two entries change; cargo deny check
advisories passes. Upstream main fails the same check.

* logging/cos/sign/combine/demo: keep extended-year dates, no epoch fallback

For consistency with the stricter no-epoch-fallback rule now agreed across the sibling
chrono PRs (wordcraft#3427, filmcraft#2662): `DateTime::from_timestamp(..).unwrap_or_default()`
changes a timestamp the old code converted past chrono's year 262143 range — it collapses
to 1970-01-01 instead of the real proleptic-Gregorian date. The FormCalc Num2Date path was
already fixed (FredApps approved); this applies the same fix to every remaining conversion.
Each site uses the 400-year Gregorian cycle (146_097 days) with chrono as the only calendar
logic: chrono converts the day within one cycle, whole cycles carry in the year (i128,
saturated into i64). PDF/ASN.1 format-specific year handling is kept exactly as the old code
had it. Verified the cycle result equals the old formula across a wide range and at the extremes.

- apps/pdfcraft/src/logging.rs `timestamp`: cycle `civil_from_days(u64)`; pre-1970 epoch clamp
  and ms/time-of-day format kept. Pins: +10_000_000_000_000 s -> 318857-05-20T17:46:40.000Z
  (was 1970); checked_add u64::MAX/2 s -> 292277026596-12-04T15:30:07.000Z (was epoch).
- crates/cos/src/writer.rs `pdf_date` (i64): cycle converter, no year clamp (old code wrote `y`
  as-is, so a negative year keeps its sign). Pins: pdf_date(10_000_000_000_000)="D:3188570520174640Z",
  pdf_date(-10_000_000_000_000)="D:-3149180813061320Z".
- crates/sign/src/der.rs `Time::from_unix` (i64): cycle converter; the `year` field is a u32, so
  the year is cast `as u32` (wrapping) exactly as the old code did — ASN.1 year width preserved
  byte-for-byte. Pins: from_unix(10e12).year=318857, from_unix(-10e12).year=4_294_652_378,
  from_unix(i64::MAX).year=219_250_468, from_unix(i64::MIN).year=4_075_720_767.
- crates/ui-egui/src/combine_ui.rs `utc_stamp` (u64): cycle converter; pre-1970 epoch clamp kept.
  Pins: utc_stamp(10e12)="318857-05-20 17:46 UTC", u64::MAX/2 s="292277026596-12-04 15:30 UTC".
- xtask/src/demo_pdf/pdf.rs `Stamp::from_unix` (i64): cycle converter, no year clamp. Pins:
  from_unix(10e12).iso="318857-05-20T17:46:40Z"/.pdf="D:3188570520174640Z",
  from_unix(-10e12).iso="-314918-08-13T06:13:20Z".

cos/sign/xtask built, tested (pdf_dates_are_utc_and_byte_identical, der::tests::times,
stamps_round_trip), fmt and clippy clean. pdfcraft-ui-egui and the pdfcraft app can't link
here (unrelated LLVM intrinsic bug in the rten-gemm dependency, 1.98.1), so their logging and
combine_ui functions + tests were copied verbatim into a scratch crate with chrono 0.4.45 and
pass there; fmt --check clean on all five packages. CI builds these on all platforms.

* Don't build a year-318857 SystemTime unchecked in the logging test

Windows' SystemTime only reaches about year 30828, so
UNIX_EPOCH + Duration::from_secs(10_000_000_000_000) panicked before
timestamp() ran and failed the Windows test job. Build it with
checked_add and assert where the platform can hold it. The year-10000
case fits everywhere and stays unconditional.

* Name the 400-year range extension as the one exception to the time rule

The rule said never to hand-roll calendar math, but the range fix
(shifting by whole 400-year Gregorian cycles so dates past chrono's year
range keep their old output) is cycle arithmetic, so the rule and the code
disagreed. The rule now names that exception: the library still computes
month, day and time, and only whole cycles are carried in the year.

* Say the library computes month and day; time-of-day is plain arithmetic

Narrows the range-extension wording: the date library computes the
calendar fields, and splitting seconds into hours, minutes and seconds
is ordinary arithmetic, which is what the code does.
2026-10-21 19:36:46 -04:00
Marco Arnulfo 8444a08c00 fix(cos): a repaired xref table fills only the numbers the damage leaves out (refs #1628) 2026-10-21 10:53:02 +02:00
Marco Arnulfo 57ab6fcb2a fix(cos): the renumbering check follows a shifted header (refs #1628)
With junk before %PDF the table's offsets are relative to the header, so the "1 G obj" check is also tried there, as load does; otherwise object 1 stayed free and a full save dropped the image.
2026-10-21 10:22:02 +02:00
Marco Arnulfo 062270da50 fix(cos): fill xref gaps on the compact cross-reference index (refs #1628)
main now keeps valid tables as a compact index; the gap filling reads entries through it and materializes them only when it repairs.
2026-10-21 10:18:13 +02:00
Marco Arnulfo 4e94aa1cbb fix(cos): find objects a broken cross-reference table leaves out (fixes #1628) 2026-10-21 10:18:13 +02:00
wstock 801980ba35 cos: the /Prev repair note names the reference's own generation (#2366) 2026-10-20 12:43:21 +00:00
wstockandClaude Opus 5.5 3af350026f Issue #2366: a /Prev reference to an object holding the offset reads that offset
iText's indirectPrev.pdf writes /Prev 6 0 R, and object 6, listed in the
same update, holds 470, the offset of the first section. Both readers took
the object number as the offset, so cos found no section at 6, rebuilt the
file from object headers and lost its revision list, and hayro's chain
stopped there.

When /Prev is a reference, the sections read so far (cos) or the section's
own entries (hayro, which now reads them before following /Prev) are asked
for that object first. If it holds a non-negative integer, that is the
offset; otherwise the object number still is, as the tiff2pdf files need.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NH3itd85jPuM6wNjgcGfEH
2026-10-20 11:32:17 +00:00
wstockandClaude Fable 5.1 0e8ebf00e6 Issue #2366: a trailer whose /Prev is a reference still reaches the earlier sections
The JHOVE PDF-HUL-157 files (tiff2pdf output) end their first trailer with
`/Prev 297298 0 R`: the offset written as a reference. The chain reader took
/Prev as an integer only, so the chain stopped after the first section without
a word, and every object of the earlier sections (here pages 2-4 and their
images) read as null. The document showed one page where Poppler and MuPDF
show four, and a full save wrote the missing kids as null, so pages 2-4 were
unreadable afterwards ("cycle in page tree", 1x1 renders).

A reference-valued /Prev is now read as the offset it names, with a repair-log
note; any other non-offset value is noted and ends the chain as before.

Regression test: document::tests::a_prev_written_as_a_reference_still_reaches_the_earlier_sections.
Verified on JA3-1998-PDF.pdf: four pages read, and after a Subject edit and a
full save Poppler renders all four pages without a warning (before: "Kid
object (page 2) is not an indirect reference (null)").

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011K1peyrh5MdZ6ttHkVq8Gh
2026-10-20 11:29:42 +00:00
cbbeb61650 fix(parse): accept spaces between stream and its line break (#3378)
* fix(parse): accept spaces between stream and its line break

Microsoft Reporting Services writes `stream \r\n`, a space before the line break.
hayro-syntax and pdfcraft-cos took the stream data to start at the space, so the
content stream failed to inflate: the page rendered blank (white, or black with
inverted colours) and an edited save corrupted the stream. Spaces and tabs are now
skipped, but only when a line break follows.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Number the stream end-of-line patch (20), as vendor/README does

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-19 02:50:21 -04:00
104411dc7a Read doubled-sign integers like --280 in the COS lexer (#3299)
* Read doubled-sign integers like --280 in the COS lexer

Fixes #3111

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* COS lexer: strip the doubled sign without allocating

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-18 17:06:53 -04:00
wstock e336b48a35 cos: a rebuild prefers a trailer whose /Root is the catalog (#2365 review)
A later cross-reference stream (or trailer) whose /Root points at another
dictionary, such as /Info, replaced an earlier good trailer, and a full save
then wrote that dictionary as /Root. A candidate whose /Root passes the
root_is_catalog rule (/Type /Catalog or /Pages) now beats one whose /Root is
only some dictionary, wherever it is in the file. The looser rule still
applies when nothing better is there, which FDF files (whose /Root has
neither key) and the repair path for a non-catalog /Root rely on.

Test: reconstruction_skips_a_trailer_whose_root_is_not_the_catalog
2026-10-18 15:06:31 +00:00
aiongg eedf250b4c Merge upstream main ecffce69e into next
crates/cos/src/document.rs: both new accessors, header_offset (#2525) and reconstructed (#2029).
parity/acrobat-features.toml: the malformed-xref entry has both sides' evidence and notes (#2029 full save of a repaired file, #2717 table numbered from 1).
2026-10-18 14:50:20 +00:00
mtlcl410andClaude Opus 5.5 ecffce69ee fix(engine): Save writes a repaired file in full even without edits (#1937) (#2029)
* fix(engine): Save writes a repaired file in full even without edits (#1937)

A damaged file is repaired on open (its cross-reference index rebuilt from
object headers, as Document Properties reports), but Save wrote it back as it
was: `Session::save_bytes` returned the bytes as opened whenever nothing was
edited, and a repair is not an edit. The "repaired" file on disk was still the
damaged one.

`save_bytes` now writes a reconstructed document in full even with no edits.
`Document::reconstructed()` in cos names the condition (no revision chain:
rebuilt, or built from scratch), and `write_incremental` uses it too, so the
writer and the engine share one definition.

Tests: `engine::tests::a_repaired_file_is_rewritten_by_save_even_without_edits`
(table offsets shifted; no table at all) and
`automation::saving_a_repaired_file_puts_the_repair_on_disk` (doc_open,
doc_info, doc_save over the same path, reopen). Both fail with the condition
removed: the saved bytes equal the damaged file. parity: core.repair-broken-files
cites them and notes that a repair still does not count as an unsaved change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz

* fix(engine): a signed repaired file keeps its bytes on Save without edits (#1937)

Review follow-up: writing a reconstructed file in full on Save puts the
repair on disk, but for a signed file it would also invalidate its
signatures without warning. Save now skips the early return only when the
document is reconstructed and not signed. A regression test signs a
document, appends a `startxref` that points nowhere (the signed bytes stay
intact, the cross-reference data must be rebuilt) and checks that Save
without edits returns those bytes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-18 10:47:28 -04:00
aiongg 7f718ecc62 Merge upstream main 50f261d70 into next
ROADMAP.md, docs/roadmap.md: union of both sides' log lines and milestone rows, each line once.
parity/acrobat-features.toml, crates/xfdf/README.md: next's text, a superset of main's (#1917, #2686 plus #2688).
crates/edit/src/tests.rs: both sides' new tests (#2558 and #3084).
crates/ui-egui/src/editing.rs: the picked picture is cleared on every edit, and a new drawing commits its raster (#3149).
2026-10-18 13:12:21 +00:00
wstockandClaude Fable 5.1 227b4bfcf0 Issue #2365: a rebuilt file keeps the /Info and /ID of its cross-reference stream
When the cross-reference chain is unusable (here: a second, truncated PDF
appended after the first, so no startxref is in reach), the index is rebuilt
from object headers and the trailer taken from the last `trailer` dictionary
with a usable /Root. A file written with cross-reference streams has no
`trailer` keyword, so the rebuild fell through to the catalog alone and /Info
(and /ID, /Encrypt) were lost: Document Properties showed nothing, and a full
save after editing one Info field wrote only that field and /ModDate.

reconstruct() now also considers every /Type /XRef stream dictionary with a
usable /Root, taking whichever trailer comes last in the file.

Regression test: document::tests::reconstruction_takes_the_trailer_from_a_cross_reference_stream.
Verified on the issue's fixture (openpreserve PDF-HUL-138/6.2017-0960.pdf):
Creator, Producer, CreationDate and ModDate are read and kept by a full save.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011K1peyrh5MdZ6ttHkVq8Gh
2026-10-18 11:25:48 +00:00
wstockandBrandon Thomas 72dac33f8e Issue #2367: an object stream whose entry cannot place it is found by its header (#2398)
* Issue #2367: an object stream whose entry cannot place it is found by its header

qpdf's xref-compressed-in-compressed.pdf marks object stream 69 as itself
compressed inside stream 81, which the format forbids and which does not
exist. objstm() loaded the container through its cross-reference entry only,
so every object inside (the outlines, the page labels) read as null and a
full save wrote `/Outlines null /PageLabels null`, while an incremental save
(which copies the original bytes) kept them, and MuPDF recovers them.

The container is now found by its own `N G obj` header when its entry cannot
load it (scan_for, the same last resort a plain object with a wrong offset has).

Regression test: document::tests::an_object_stream_marked_as_compressed_is_found_by_its_header.
Verified on the fixture: objects 70 and 82 load (Count 6; 12 page-label
ranges) and a full save keeps both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011K1peyrh5MdZ6ttHkVq8Gh

The fallback is for file entries only: once the stream is freed or replaced in the overlay,
its objects must not come back from the file's copy (sign's
object_stream_changes_cannot_reuse_old_candidate_membership caught this after the rebase);
a cos test covers both cases.

* cos: a container the saved table marks free is not found by its header either

Freeing an object stream, saving incrementally and reopening left the
overlay empty and the table saying the stream is free; the header
fallback then rescanned the file and the stream's objects came back on
the next full save. The fallback now also stops at an on-disk free
entry.

* M1: put the #2367 test's description on the test, not its fixture

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-18 03:48:23 -04:00
7c8ca0bde8 M7: keep the text blocks cache after the document is edited (#2504)
The engine caches a page's text blocks for opening a paragraph and
applying its edit, but only while the document is unmodified: it was
keyed by the source identity, which edits keep. After the first edit the
cache never hit again, and the page was grouped again on every call (the
inline editor and the panel each call it after an apply).

pdfcraft_cos::Document::read_state() returns what the document reads as:
its source identity, its trailer and every changed object, compared as
allocations that the state keeps alive. A clone or a snapshot restored by
undo has the same state; any set, add, free or trailer change makes a
different one. The cache is keyed by it, so it hits after edits and never
returns another state's blocks.

On a page of 120 rows, text_blocks after an edit took 0.215 ms per call
and now 0.006 ms (release build, best of three; a throwaway timing test).

Tests: a read state matches a clone and not a document changed by set,
free or a trailer change, nor another document; after an edit the engine
groups the page once and undo and redo get their own blocks. The engine
test fails without this change.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-18 03:46:56 -04:00
aiongg 0b3b63b22a Integration: #2717 the header-relative retry keeps its flag after the merge with the new head 2026-10-17 12:23:05 +00:00
aiongg cc809ae2ed Merge PR #2717 (new head aea1d3d12)
# Conflicts:
#	crates/cos/src/document.rs
2026-10-17 12:18:46 +00:00
mtlcl410andClaude Opus 5.5 4a7d556a32 fix(cos): write whole reals past the 32-bit integer range as reals
write_real printed a whole real below 1e15 as an integer, and a larger one
through Display, which has no decimal point either. So 4e9 or 1e30 was
written as an integer token outside the range ISO 32000-2 Annex C gives
integers (±2^31-1), and read back as Int (or, in readers with 32-bit
integers, overflowed). Whole reals now use the integer form only within
that range; past it they keep a trailing '.' (§7.3.3 allows "4.").

Raised in the review of #2877.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
2026-10-17 17:46:05 +08:00
Tiago KielandClaude Opus 5.5 2c4645de41 fix(cos): check object-stream member numbers and offsets (#1815)
Object-stream members were indexed with unchecked casts: `/First` and each
offset went through `as usize` and an unchecked addition, which truncate or
wrap on 32-bit targets such as the wasm32 build, and the member number went
through `as u32` on every target, so a member numbered 2^32 + 3 was read as
object 3. A negative offset was clamped to zero and read the stream's first
member instead.

Each member is now converted and added with checks, and kept as an unusable
slot (preserving positions, which cross-reference entries refer to) when its
number is not an object number or its start falls outside the decoded data.
Looking such a member up is a MissingObject error, as for any other absent
object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 11:43:25 -04:00
mtlcl410andClaude Opus 5.5 aea1d3d127 M1: put valid_xref_range's doc comment back, and no trailing spaces in the tests (#2676)
Review nits on #2717: the new doc comment had been inserted inside
valid_xref_range's; the tests' xref tables are now one-line literals with
\n escapes, so git diff --check is clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
2026-10-16 22:30:55 +08:00
mtlcl410andClaude Opus 5.5 c6acaefd35 M1: read an xref table numbered from 1 that begins with object 0 (#2676)
Some producers head a classic xref table "1 n" and then write object 0's
free entry first, so every row is one number off and object 1 reads as
free. The renderer coped, but cos did not: printing a scanned invoice
built from cos dropped object 1 (page 1's image) and printed the page
blank. Such a subsection is now read from 0 when its second row locates
an object that is numbered 1, in both xref readers, and the repair is
logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
2026-10-16 22:30:55 +08:00
aiongg 70abceba0d Merge PR #2267: fix(filters): preserve output-limit errors in raw Flate fallback
ROADMAP.md: next's revision history plus this PR's row; the #2267 refresh progress line dropped
docs/roadmap.md: next's revision history plus this PR's row; refresh wording dropped from the status line and the M1 row
2026-10-16 14:03:27 +00:00
aiongg a463e32eef Integration: #2717 move the valid_xref_range doc comment back above its function 2026-10-16 14:02:33 +00:00
aiongg 1a773e8098 Merge PR #2717: M1: read an xref table numbered from 1 that begins with object 0 (#2676)
crates/cos/src/document.rs: union with #2525: header_relative revisions, the renumbering log line, and log passed to read_section on the first try and the header-relative retry
2026-10-16 14:02:26 +00:00
aiongg bb695240f0 Merge PR #2525: fix(cos): an incremental save of a file with bytes before its header counts from the header 2026-10-16 14:02:08 +00:00
Christoph OrtenburgandClaude Opus 5.5 c0a8a1a10f test(cos): qpdf can't check a prefixed file whose offsets include the prefix
qpdf counts every offset from the header in a file with bytes before it, so
it already reports the input of
`incremental_saves_of_a_file_whose_offsets_count_from_its_first_byte_keep_counting_so`
as damaged ("xref not found") and reconstructs it, before anything is
appended. The qpdf check there could never pass; the test's own reader checks
stay. The header-relative test keeps its qpdf check, which passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 21:46:45 -04:00
mtlcl410andClaude Opus 5.5 271cb6161f M1: read an xref table numbered from 1 that begins with object 0 (#2676)
Some producers head a classic xref table "1 n" and then write object 0's
free entry first, so every row is one number off and object 1 reads as
free. The renderer coped, but cos did not: printing a scanned invoice
built from cos dropped object 1 (page 1's image) and printed the page
blank. Such a subsection is now read from 0 when its second row locates
an object that is numbered 1, in both xref readers, and the repair is
logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
2026-10-16 08:35:35 +08:00
Claude 33fe488061 fix(cos): an incremental save of a file with bytes before its header counts from the header
Some files carry bytes before `%PDF-`, and their offsets count from the header rather than from
the first byte. We read those, retrying a section relative to the header when the absolute
offset misses, but the incremental writer then appended a section whose offsets and `startxref`
were absolute while its `/Prev` kept the header-relative value it was given. qpdf reported "xref
not found" and reconstructed the file, so a save that reported success left a damaged-looking
update behind (#2364, found with KillerPDF's reserve3.pdf and a Subject edit).

The reader now records on each `Revision` whether it was found counting from the header, and the
writer counts the new section the same way as the one its `/Prev` points to. A file whose
offsets already include the prefix keeps absolute offsets, which the second test pins so the fix
cannot overcorrect.

Seen red with the writer change reverted: the appended section's `startxref` missed its `xref`.

Fixes storytold/pdfcraft#2364

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DsT1y8xBvkXeBxEUQqdDtj
2026-10-15 19:54:17 -04:00
Nick Nyssen 2ec501fa74 test(cos): property tests for save fidelity, plus credit entry
Adds crates/cos/tests/proptests.rs (464 lines) - three properties over generated catalogs (edit sequence
vs a plain ordered model; full save in both output styles; incremental save over bases built BOTH ways) -
and a self-credit entry in contributors/people.toml committed under the author's own identity. No product
code changes; total +466 lines across two files: people.toml +2; proptests.rs +464.

Two design decisions that came out of adversarial review, each measured rather than asserted:
  * catalog keys insert in a scrambled order ([5,0,7,2,6,1,4,3] filtered to the drawn keys), because with
    ascending insertion "preserves key order" and "sorted by key" are the same observable.
  * incremental expectations derive from the values written, not from reading the writer's own output back,
    so a consistently-wrong implementation cannot pass by agreeing with itself.

Measured against one-line breaks in crates/cos/src/writer.rs, each reverted and byte-compared afterwards
(verdicts from cargo exit codes; "pre-existing" = lib 48 + writer 10 + encryption 8 + shared_streams 3 +
bounded_writer 4, i.e. every target that existed before this PR):

  sort every dictionary by key              CAUGHT (props 1+2)      pre-existing suite FULLY GREEN
  reverse dictionary emission              CAUGHT (all three props)   pre-existing suite FULLY GREEN
  move /Type to the end of every dict     CAUGHT (prop 1)           pre-existing suite FULLY GREEN
                                            - caught only after catalog_keys() was added to pin the full key
                                              sequence; without it this break was green everywhere
  classic-table incremental startxref +1   CAUGHT (prop 2 alone; props 1+3 green), failure message names
                                            "classic-table", confirming the incremental table path executes
  drop null-valued dictionary entries      CAUGHT (all three)        at the pre-fix commit b51381c this same
                                            break left property 2 green - the self-reference it removed

Known limits, stated not hidden: incremental saves are asserted to add bytes and leave the original prefix
untouched - minimality is NOT claimed; catalog key order pins one fixed scramble rather than drawn
permutations, so 25.4% of cases draw too few keys for order to be meaningful; nested dictionary key order
is deliberately not pinned (canon sorts inside values); /Perms and permissions semantics are out of scope
here.

Gates at this tip (upstream 24f93e6, drift 0): cargo test -p pdfcraft-cos 85 passed / 0 failed / 2
ignored; cargo fmt -p pdfcraft-cos --check clean; clippy exit 0 for this crate with
-D warnings -A clippy::chunks_exact_to_as_chunks (that lint fires only in upstream's own
crates/oracle/src/pixels.rs:189 from f988cf8, tracked by #2342/#2324; flags after `--` forward into every
crate in the build graph, and crates/cos itself emits zero warnings under a forced recompile).
cargo xtask layers OK: 37 crates.

Scope: part of #1187 (items 1-3 partly; the reference-following comparison, undo state machine, password
property and damaged-file splices remain open). The encryption-side properties are PR #2358 by the same
author; earlier drafts of this text mis-cited #2014, which is certificate encryption and still open.
2026-10-15 16:41:32 -07:00
Marco Arnulfo bb651792a9 Remove security also from files with /Encrypt inline in the trailer (fixes #2661) 2026-10-16 00:33:39 +02:00
Mex 7bb2d49467 M1: merge current upstream into Flate fallback fix 2026-10-15 21:40:21 +02:00
cccd55fab2 M1: edit finds the pages a damaged file's readers find (#1664)
* M1: edit finds the pages a damaged file's readers find (fixes #1656)

* M1: rebuild a lost page tree only when the reader shows the same pages, and refuse edits to signed rebuilt files (#1656)

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 04:01:45 -04:00