Commit Graph
73 Commits
Author SHA1 Message Date
aiongg 241c5512d6 fix(redact): charge repeated name-tree kids to the node budget
The name-tree count cloned each node's /Kids array and skipped repeated
references without charging the budget, so a node listing the same kid
many times cost work proportional to every entry while the budget saw one
step. The walk now borrows the array, charges every entry it does not
descend into, and stops once the budget is spent.
2026-10-22 13:28:40 -04:00
aiongg 4d64ca8e2d docs(redact): say which objects hidden-information removal reaches 2026-10-22 13:28:40 -04:00
aiongg 94563649e6 fix(redact): name the owner sweep's holder list for clippy 2026-10-22 13:28:40 -04:00
aiongg 822179ccf2 fix(redact): remove hidden data from every object that owns it
Metadata, private application data, associated files and additional
actions were removed from the catalog and pages only. The same entries on
an image or font (XMP), a Form XObject (PieceInfo), a page or annotation
(/AF with its embedded file) or a form field (/AA scripts) survived the
full rewrite, and the scan did not count them, so removing only such a
category reported nothing to remove. A walk of every object reachable from
the catalog now counts and strips these entries wherever they are, leaving
name maps such as resource dictionaries alone (their keys are names a
producer chose), and refuses a graph nested too deeply to walk in full.
2026-10-22 13:28:40 -04:00
aiongg 750bbe27f1 fix(redact): clean the structure tree after removing hidden content
Removing hidden text or hidden layers rewrote page and Form content but
left the structure tree alone, unlike applying redactions. An element's
ActualText, Alt or expansion text still described the removed content,
and a marked-content reference into a Form kept the original Form, with
the removed text, in the full rewrite. The hidden-content pass now cleans
the tags of what each page no longer shows and moves Form references to
the rewritten copy, refusing when the tree can't be cleaned completely.
2026-10-22 13:28:40 -04:00
aiongg 3fe9fe218e fix(redact): remove the widgets flattening keeps when removing form fields
Removing form fields flattens them and then drops the interactive form,
but flattening keeps hidden and no-view widgets (they draw nothing) and
widgets with no appearance. The pages still listed them, so their values
and their parent fields' values survived the full rewrite. Every widget
left after flattening now leaves the pages and the structure tree, inline
annotations included.
2026-10-22 13:28:40 -04:00
aiongg ee010fbbe5 fix(redact): read content strictly before sanitizing or redacting it
Page and Form content was decoded tolerantly, keeping what came before a
corrupt spot and dropping the rest without saying so. A page whose content
broke at its first bytes therefore read as empty: the hidden-text scan
counted nothing, the removal was skipped and the original stream, which
other viewers still draw, stayed in the file. Content is now decoded
strictly; an unreadable page counts as something to remove, so the removal
runs and refuses the page, and a Form read only in part is treated like an
undecodable one.
2026-10-22 13:28:40 -04:00
aiongg f1c56ef865 fix(redact): count a name tree's shared nodes once
The hidden-information count followed every /Kids entry of the attachment
and document-script name trees with only a depth limit, so a node listed
twice by each level was read once per path: a few hundred bytes made the
scan take 2^32 steps. Indirect nodes are now read once each, under a total
node budget.
2026-10-22 13:28:40 -04:00
aiongg 15f558aaa8 Merge r2 track t11: redaction, mixed: combine, compare, crate:edit, measure, ocr
# Conflicts:
#	ATTRIBUTION.toml
#	crates/automation/src/tools.rs
#	crates/edit/src/lib.rs
#	crates/ui-egui/tests/compare.rs
#	crates/ui-egui/tests/ui_group/marks.rs
2026-10-22 02:21:38 +00:00
aiongg 75fa0d0109 Merge PR #3571: fix: read the last array of a surplus-operand TJ, not the first (#3569) 2026-10-22 01:45:17 +00:00
aiongg c58f1a0ff7 Merge PR #3723: Redact: count shared outline items once so the walk ends 2026-10-22 01:44:56 +00:00
aiongg ee0b147a61 Merge PR #3728: Refuse redaction marks with no readable area instead of skipping them 2026-10-22 01:44:51 +00:00
aiongg 69ef1c18e6 Merge PR #3674: M8: applying redactions drops structure-tree references to the comments and fields it removes (#2874) 2026-10-22 01:44:49 +00:00
aiongg 06ab5aa275 Merge PR #3237: M8: phone pattern finds slashed and labelled unseparated numbers (#3205) 2026-10-22 01:44:44 +00:00
skylar taylor-barrick d695bc5654 fix(redact): evaluate OCMD visibility during sanitization (#3522)
* fix(redact): evaluate OCMD visibility during sanitization

Fixes #3499 by evaluating OCMD policies and bounded visibility expressions against the default optional-content configuration before removing hidden groups. Unknown or malformed visibility data is handled conservatively.

* fix(redact): preserve OCMD visibility during sanitization

* fix(redact): count hidden XObjects during verification

* test(redact): cover hidden form depth limit
2026-10-20 18:17:34 -06:00
eliets2 b97b383232 redact: count inline outline items against the total budget
The total budget of the outline count (100,000 items) was checked against the
number of indirect items the walk had reached. Items written inline, as a
direct dictionary, were counted in the result but never entered that set, so
an outline of 60,000 indirect siblings, each with one inline child, counted
120,000 items.

The walk now keeps a separate running total of every item it counts, inline
items included, and stops when that total reaches the budget. The set of
indirect items is still used to count shared items once. Ordinary outlines
count as before.

The doc comment of the walk now states the depth limit in one sentence: an
item first reached through a deeper path is not visited again from a shallower
one, so items nested past depth 64 may stay uncounted. The depth behaviour is
unchanged.

Test added in crates/redact/src/tests.rs:
inline_outline_children_count_towards_the_total_budget. It reads 120000 on the
previous head and 100000 now. Reverting the budget check to the indirect-only
count fails it; reverting the increment fails it and the existing total budget
test. The other outline tests pass unchanged.

Checked with cargo fmt --all -- --check, clippy with -D warnings on
pdfcraft-redact, and cargo test for pdfcraft-redact (85 passed, 0 failed).
2026-10-20 21:04:06 +03:00
eliets2 71e0346186 docs(redact): say that a mark with no readable area fails apply
The apply section of the redact README described the fail-closed rules but not the new refusal of a Redact annotation with neither a usable QuadPoints array nor a usable Rect. Documented here with the error name and the page scope of the count. No behaviour change in this commit.
2026-10-20 21:03:24 +03:00
eliets2 92c979c5b2 redact: refuse marks with no readable area instead of skipping them
A Redact annotation with neither a usable QuadPoints array nor a usable
Rect was dropped silently by marks(). Applying the readable marks beside
it then reported success, and the text under the dropped mark stayed in
the file.

apply() now counts such marks on the pages being applied and returns
RedactError::UnreadableMark(n) before anything changes. The message goes
out through the existing error text: the engine edit error, the
automation reply and the UI failure line all print it. Marks that were
readable through the existing fallbacks (a Rect alone, a usable
QuadPoints array) work as before.

Tests: three in crates/redact/src/tests.rs (refusal with the count and
no change, Rect-only still redacts, an unreadable mark on another page
does not block the selected page) and one end to end in
crates/automation/tests/automation.rs (the reply carries the reason and
the page keeps its text). On upstream main with only the variant added,
the two refusal tests and the automation test fail; the Rect test passes.
Reverting each pinned production line turns the tests red again.
Gates: cargo fmt --check, clippy -D warnings for pdfcraft-redact and
pdfcraft-automation, and the redact test suite pass.
2026-10-20 21:03:24 +03:00
eliets2 c2c4db7eca redact: count shared outline items once so the walk ends
An outline whose items name shared children (/First and /Next pointing at
nodes that other items also name) was counted by following every path. With
60 levels of two items each, a walk that follows every path takes 2^60
steps. The outline count never finished, and Remove Hidden Information and
Sanitize both run that count first.

The count now keeps the set of items it has reached. An item reached a
second time is not counted again, and the level stops there: the walk that
reached the item first goes on to count the rest of that level. A total
budget of 100,000 distinct items bounds the work for an outline with very
many items. The per-level cap is unchanged.

Ordinary outlines count as before. The existing hidden-information test
(two bookmarks) passes unchanged, and a new nested outline test (five
items) pins the count for nesting and siblings.

Tests added in crates/redact/src/tests.rs:
- an_outline_whose_items_share_children_is_counted_without_spinning
- removing_bookmarks_from_an_outline_whose_items_share_children_finishes
- an_outline_counts_no_more_items_than_its_total_budget
- a_nested_outline_is_counted_item_by_item

The first, second and third were seen to fail on the code before this
change: the first two timed out after 10 seconds, and the third read 120000
where 100000 is expected. The fourth passes before and after the change and
guards the ordinary count. The spin tests run the walk on a thread with a
10 second bound, so a regression fails the test instead of hanging the run.
Reverting only the shared-item check, or only the budget check, fails the
tests they pin.

Checked with cargo fmt --all -- --check, clippy with -D warnings on
pdfcraft-redact, and cargo test for pdfcraft-redact.
2026-10-20 19:09:44 +03:00
Jayesh-Prajapati9 e9fc4cadcf M8: applying redactions drops structure-tree references to the comments and fields it removes (#2874)
Redaction removed annotations and form fields under a mark from the pages
but left the structure tree's /OBJR references to them, so a full rewrite of
a tagged PDF kept the comment's /Contents and the field's /V. apply now
drops references to every annotation no page lists afterwards (widgets of a
removed field on other pages included), as Sanitize does since #2532, and
fails closed with RedactError::Tags when an element can't be edited.
2026-10-20 20:05:01 +05:30
MichaelandClaude Sonnet 5 1ce4965c27 fix: read the last array of a surplus-operand TJ, not the first (#3569)
`[(FIRSTARR)] [(LASTARR)] TJ` is malformed (TJ takes one array
operand), but the operator still shows the one immediately before it,
as the renderer, MuPDF and pdftotext agree. Three readers here counted
operands from the front instead and showed the discarded first array:

- crates/edit/src/text.rs (text_lines/text_paragraphs, and the Office
  export through reading_lines)
- crates/redact/src/interp.rs (search-based redaction's content
  interpreter)
- crates/a11y/src/headings.rs (autotagged heading detection)

So the editor could offer a word the page never draws, a redaction
search could miss it, and a heading could read as text nobody sees.
Take the last array for TJ in all three, matching the `_ =>
op.operands.last()` arm already used for Tj/'/" alongside it.

Added a regression test per crate (`a_tj_with_a_surplus_array_reads_
the_last_one`, `a_tj_surplus_arrays_undrawn_glyphs_are_not_seen_
under_a_mark`, `a_heading_with_a_tj_surplus_array_reads_the_one_the_
page_draws`), confirmed each fails without its fix and passes with it.
cargo test -p pdfcraft-edit (196 passed) -p pdfcraft-redact (81
passed) -p pdfcraft-a11y (44 passed), fmt, clippy -D warnings, xtask
layers and xtask assets all clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-20 08:09:59 +08:00
skylar taylor-barrick e397efa854 Merge pull request #3285 from mtlcl410/fix/struct-refs-parent-tree-2532
M8: redaction's structure-tree walks start from the parent and ID trees too (#2447, #3302)
2026-10-19 10:05:04 -06:00
Earl Co 236ce9a064 Clear image pixels intersected by redaction marks (#3368) 2026-10-19 02:48:29 -04:00
skylar taylor-barrick cece9f5ccc M8: document discard-on-error for public redaction calls (#3302) 2026-10-18 16:45:35 -06:00
mtlcl410andClaude Opus 5.5 3ef3c9efa9 M8: tag cleanup refuses what it can't finish instead of removing the tree (#3302)
Maintainer review on #3285: an element written in place that owns redacted
marked content or tags a removed annotation, an inherited page the lookup gave
up on (over 256 ancestors or a loop), and a walk past its limit are now
RedactError::Tags, so nothing is changed. Removing only /StructTreeRoot left
outline /SE and GoTo /SD reaching the elements. Inherited pages are cached per
ancestor. clear_every_element and drop_tree are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
2026-10-19 03:20:34 +08:00
mtlcl410andClaude Opus 5.5 f62bf71a77 M8: tag cleanup keeps inherited pages, fails closed over replaced streams and in-place elements (#3302)
Review follow-ups on #3285: walk /K before the parent and ID trees and look up
an element's inherited /Pg through /P; drop the structure tree when a cut-short
walk could leave MCRs naming a replaced stream; drop it when an element written
in place tags a removed annotation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
2026-10-19 01:33:42 +08:00
mtlcl410andClaude Opus 5.5 ce3c0e46c9 M8: redaction clears tag text of elements reached only through the parent or ID tree, and fails closed past the walk limit (#3302)
tags::clean, which drops /Alt, /ActualText and /E that repeat redacted
text, walked the structure tree from the root's /K only, so an element
listed only in /ParentTree or /IDTree kept the redacted words. It now
starts from both trees too, as drop_object_refs does since 3f543ca1.

Past its 2,000,000-reference cap the walk used to skip the rest and
report success. It now fails closed: every structure element in the
document, found by reading each object rather than through the tree,
loses its alternate and actual text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
2026-10-19 00:52:18 +08:00
mtlcl410andClaude Opus 5.5 e6de108972 M8: note that structure elements written in place keep their references (#2447)
Review note from Losminek on #3285.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
2026-10-19 00:44:58 +08:00
mtlcl410andClaude Opus 5.5 3f543ca17f M8: redaction drops OBJRs of elements reached only through the parent or ID tree (#2447)
Follow-up to #2532 (review by aiongg): the structure-tree walk started
only at the root's /K, so an element listed only in /ParentTree or
/IDTree kept its object reference to a removed link or comment, and the
root kept that annotation reachable in a full rewrite. The walk now
also starts from both trees and follows their nodes (/Nums, /Names,
/Kids), written in place or as their own objects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
2026-10-19 00:09:17 +08:00
aiongg b6b02a1b27 Merge upstream main 1952a8880 into next
ROADMAP.md, docs/roadmap.md: union of both sides' lines, each once.
parity/acrobat-features.toml: the XFA entry has both sides' evidence and notes (#2604 page areas, #2813 CLI layout).
2026-10-18 14:22:30 +00:00
2eb1639097 Find credit card numbers followed by an expiry, year or CVV (#2969)
* Find credit card numbers followed by an expiry, year or CVV

Found while testing crates/redact/src/patterns.rs:180-190 (credit_card, via digit_run at :53)

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Redact: a card followed by a number drops at most four trailing digits

A longer grouped number (an IBAN's digits) no longer matches on a Luhn-valid prefix; the run is
read far enough to see the tail and refused when it goes on past that.

---------

Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-18 09:49:13 -04:00
NetterSilverandClaude Haiku 5.5 7e3f413f52 Redact email addresses whose local part starts with punctuation (#2948)
Found while testing crates/redact/src/patterns.rs:141-146 (email)

Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
2026-10-18 09:48:02 -04:00
aiongg 7f718ecc62 Merge upstream main 50f261d70 into next
ROADMAP.md, docs/roadmap.md: union of both sides' log lines and milestone rows, each line once.
parity/acrobat-features.toml, crates/xfdf/README.md: next's text, a superset of main's (#1917, #2686 plus #2688).
crates/edit/src/tests.rs: both sides' new tests (#2558 and #3084).
crates/ui-egui/src/editing.rs: the picked picture is cleared on every edit, and a new drawing commits its raster (#3149).
2026-10-18 13:12:21 +00:00
mtlcl410andClaude Opus 5.5 1c338b3ea6 M8: a phone number takes one slash, right after its area code (#3205)
Review follow-up: "0426/12/3456" and "0426 12/3456" matched, since any
slash between digits counted. Only one slash is allowed now, and only
right after the first (area code) group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
2026-10-18 19:48:53 +08:00
mtlcl410andClaude Opus 5.5 73aea6e0ac M8: phone pattern finds slashed and labelled unseparated numbers (#3205)
Italian documents write landlines with a slash between area code and
number (0426/123456), and mobiles without separators (3451234567,
+393451234567). The phone pattern needed a separator other than a slash,
so these stayed readable after redaction.

- A slash joins two digits in a number with a trunk-0 area code; a
  North American or + number with a slash still doesn't match, so
  reference numbers like 2024/123456 stay unmarked.
- A number without separators matches only right after a phone label
  (Tel., cell., Fax:, telefono, mobile, ...). Unlabelled runs of digits
  (ids, VAT numbers) still don't match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
2026-10-18 19:37:27 +08:00
Ridwan 5827afdcde M8: measure fonts without /Widths with their embedded program (#2130) (#2232)
Find and the renderer place glyphs of a simple font that has no /Widths
with the embedded font program's own advances, but redaction guessed the
widths from the font's name (a standard-14 face, Helvetica by default).
When the guess was wider than the real font, the end of a line under a
mark seemed to lie past it and was kept in the content stream, and the
verify pass, using the same guess, agreed.

Metrics now fills the width table from an embedded sfnt program
(FontFile2, or OpenType FontFile3) when /Widths is missing, looking the
glyph up as the renderer does. A subset's unmapped codes get width 0 so
has_glyph stays false for them. Fonts with /Widths, and bare CFF or
Type 1 programs, are unchanged.

Fixes #2130
2026-10-18 03:48:00 -04:00
mtlcl410andClaude Opus 5.5 2c4b84f6af M8: redaction removes the replies of the annotations it removes (#2659) (#2697)
* M8: redaction removes the replies of the annotations it removes (#2659)

A reply (/IRT) placed away from the mark survived, and its /IRT kept the
removed annotation - its comment text and the QuadPoints of the redacted
words - in the saved file. Replies to a removed annotation, and replies to
those, now go with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz

* M8: test replies listed before their parent, with a pop-up on a reply (#2659)

Review of #2697: the replies precede the annotation they answer in
/Annots, deepest first, and a reply has a pop-up. A single pass in
/Annots order would keep them; the test fails with the loop cut to one
pass, which the first test does not notice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 03:45:46 -04:00
d8b2705825 Clear replacement text (#1350)
* M8: Clear marked-content replacement text

* M8: Move roadmap note

* M8: Keep inherited resource guard clean under Rust 1.96

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: skylar taylor-barrick <hi@skylar.cc>
2026-10-17 22:10:22 -06:00
skylar taylor-barrick 392406eb4a fix(content): read a comment at the end of a content stream as the renderer does (#3081)
Pieces::join separated a page's content streams with a newline, which ended
a comment left open at the end of one stream. hayro, MuPDF and Poppler join
them with a space, so the comment runs on into the next stream and hides its
first line. Edit text, object_list, the redaction audit, redaction and the
Office/HTML export therefore saw text and boxes that nothing draws, and
redacting unrelated text on the page made the hidden content visible: the
rewritten stream dropped the comment.

- Pieces::join uses a space, so every reader of the joined streams agrees
  with rendering.
- The parser records, per piece, whether a top-level comment runs into it
  and out of it (boundaries only, not every comment). A redaction rewrite
  of a piece ends an incoming comment before its first operator and leaves
  an outgoing one open, so hidden content stays hidden.
- Reading for export parses the streams joined, instead of one by one.
- The redaction tag check compares the joined streams the same way.
- Pieces::piece_of uses a binary search.
2026-10-17 21:50:58 +02:00
mtlcl410andClaude Opus 5.5 f5a2376c86 fix(redact): a structure tree too large to walk goes whole (#2447)
Review follow-up on #2532: drop_object_refs stopped visiting at 2,000,000
references and still reported success, so a reference beyond that could
keep a removed annotation reachable. A tree the walk can't finish is now
removed whole, with /MarkInfo: fail-closed, at the cost of the tags. The
limit is a parameter of drop_object_refs_within so the test can reach it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
2026-10-17 02:02:23 +08:00
mtlcl410andClaude Opus 5.5 4d6812503d fix(redact): flattened widgets and far pop-ups leave the structure tree too (#2447)
Review follow-up on #2532:
- the structure-tree references dropped are those of every annotation no
  page lists any more after Sanitize, so widgets flattened with "form
  fields" go too, with their field value;
- the annotations to remove are collected on every page first, so a
  pop-up listed on another page than its parent goes with it;
- an element left with nothing to tag drops /Alt, /ActualText, /T and /E,
  which can repeat the URL or comment;
- the test's byte search saves without object streams, so it can see the
  strings it looks for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
2026-10-17 02:02:23 +08:00
mtlcl410andClaude Opus 5.5 b7b6e6fe09 fix(redact): removed links and comments leave the structure tree too (#2447)
Remove Hidden Information and Sanitize took a link or comment off its
page's /Annots, but a tagged PDF's structure tree still referenced it
through an /OBJR, so the full rewrite kept the annotation, its URL or
text included, while link_list reported none. The annotations removed
from every page are now collected, and tags::drop_object_refs drops the
structure elements' object references to them (direct or indirect,
array or single kid), with the same seen-set bound as tags::clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
2026-10-17 02:02:23 +08:00
6e04f6eed7 fix(redact): size image clearing with checked arithmetic (#2177) (#2221)
* fix(redact): size image clearing with checked arithmetic (#2177)

An ICCBased image's /N comes from the file and was cast with `as usize`, then multiplied
unchecked into the row size. /N 2^60 overflowed it (debug: overflow panic; release: a zero row
emptied the buffer and the write loop indexed out of bounds), and a negative /N wrapped.

/N is now limited to 1, 3 or 4 (ISO 32000-2 §8.6.5.5), every size product is checked, and the
pixel writes use `get_mut`. An image that fails any of these is refused, and the caller already
removes it whole (fail-closed). Regression test covers /N 2^60, -1 and 2.

* redact tests: pass the pixel array by value (clippy needless_borrows_for_generic_args)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 02:50:02 -04:00
ysrdevs e8d1980ded XFA: flatten a form into a plain PDF, or convert it to a static form (#1948)
Flattening the fields of a laid-out XFA form used to leave the XFA in
the file: Adobe's viewers then laid the form out again from the
template over the baked pages, so the "flattened" PDF was still a live
form there and a plain one everywhere else.

pdfcraft_xfa::strip takes the XFA out of a document: the packets
(template, data and all), /NeedsRendering on the catalog and the
AcroForm, the layout mark, the overrides, and the private keys on
every field down the tree. The engine calls it after Flatten form
fields on a document with XFA packets, so the result is a plain PDF
every viewer shows the same, and for the new Edit::XfaToStatic
("Convert to a static form"; tool xfa_to_static), which keeps the
laid-out pages and the fields with their values as an ordinary
AcroForm that no viewer lays out again and whose scripts no longer
run. Both undoable: the engine's post-edit refresh re-reads the
layout mark to drop or restore its XFA state and the cached template.

What the packets held beyond the fields is reported (packets by
name, data no field showed, buttons left inert, the layout's own
warnings, Reader usage rights) as document warnings, and a dynamic
form that was never laid out is refused rather than left with
nothing. Applying redactions shares the same strip. Closes the last
planned XFA parity entry (form.xfa-flatten shipped).
2026-10-15 02:32:04 -04:00
mtlcl410andClaude Fable 5.1 1ae12090d0 fix(redact): the tail of a longer number is not a phone number (#1884) (#1911)
The phone pattern tried every position, so the last groups of a French
social security number, an IBAN or a SIRET matched as a trunk-0 or NANP
number once the groups before them were skipped, and the start of the
identifier stayed visible. A phone number is now a whole run of digit groups:
one reached through separators from a digit before it, or going on past the
28-character cap, is not one.


Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-15 02:29:06 -04:00
Sir Clangington III.andLabDennis 8a04b6d5c2 M8: Bound and reuse repeated redaction overlays (#1426)
Consolidate the reviewed change and its follow-ups on current upstream.

Original change notes:
M8: Bound and reuse repeated redaction overlays

Prepare appearances within a shared 8 MiB budget before changing the document. Reuse one measured and encoded row, reject excessive geometry, and preserve document and undo state on refusal.

Fixes #1326

M8: Preserve upstream font metrics in bounded redaction overlays

M8: reuse ordered redaction mark groups (#1326)

Co-authored-by: LabDennis <341377186+LabDennis@users.noreply.github.com>
2026-10-15 02:16:56 -04:00
Jules Kreuer 039706c437 Clear image leftovers (#1151)
* M8: Remove image redaction leftovers

* M8: Move roadmap note
2026-10-15 02:16:35 -04:00
Tanish MankuandClaude Opus 5 50629eb6bf feat(testkit): one way to write a synthetic PDF fixture
55 files hand-rolled the same serialiser -- number the objects, record each one's offset, emit a
classic cross-reference table and a trailer -- 111 times over. The object bodies are properly the
business of the crate under test; the envelope around them is not, and every copy is a place for a
fixture to be quietly wrong in a way that makes a test prove less than it appears to.

`pdfcraft-testkit` holds that envelope. It has no workspace dependencies on purpose, so the crates
at the bottom of the layering can use it without a cycle, and it returns bytes rather than a parsed
document for the same reason.

The builder is byte-identical to the serialiser it replaces, asserted against a verbatim copy of
the old code across versions and trailer variations, because a migration that changes a fixture's
bytes silently changes what the test using it proves. Watched failing by breaking the free-entry
generation: three tests caught it. The one intended difference is that an empty stream dictionary
gives `<< /Length N >>` rather than a doubled space.

Migrates cos, measure and redact: 111 copies in 55 files becomes 108 in 53. The rest are
follow-ups rather than one sweep -- ui-egui alone is 27 of those files and is the most contested
area in the repository. Some copies should stay: a test about a malformed cross-reference table
needs to write its own bytes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-14 13:57:56 -07:00
Márk BartalosandBrandon Thomas 9ed3abe490 Keep undrawn comments and accurate search hits (#1317, #1327, #1302, #1315, #1340) (#1442)
Flattening deleted a cloudy comment it could not draw, and a refused text-box restyle or resize left the new properties in the file. Arrow defaults were stored on plain lines. A custom match ending on a separator included the next glyph, and the vertical-text redaction error claimed an overlap it had not checked.

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-10-13 01:09:36 -04:00
Jeremy YaoandClaude Opus 5.5 759cfa65f4 Security: fix three audit findings in redaction and signature validation (#1631)
* M8: redaction unlinks removed images and forms from inherited and shared resources

Apply Redactions and Remove Hidden Information give a rewritten page its own
pruned /Resources, but the dictionary the page drew through before could still
name the image or form that was removed:

- a /Resources inherited from a /Pages node;
- one resource dictionary object shared by several pages;
- the page's own dictionary, when a form without /Resources drew the image
  through it.

A full-rewrite save keeps whatever is still referenced, so the removed image
samples and form text stayed in the saved file while the operation reported
success. Per-page resources (the case fixed in #1099) were not affected.

The interpreter now records the original object behind every XObject it drops
or replaces. After the page loop, each page-tree resource dictionary that still
names one loses that entry unless a page drawing through that dictionary still
draws the name (the same image on another page, or again outside a mark). A
dictionary that loses an entry is replaced by a private copy on its node, so a
dictionary object shared with other nodes is left alone for them. If a page's
use of a name cannot be told (unreadable content, a form that will not decode),
the operation fails closed.

Tests: redacted_xobjects_do_not_survive_through_inherited_or_shared_resources,
an_image_another_page_still_draws_through_the_same_resources_is_kept,
an_image_a_form_drew_through_the_pages_resources_does_not_survive,
a_hidden_layers_image_does_not_survive_through_inherited_resources.

Not covered here: originals named by a form XObject's own /Resources, by an
annotation appearance or by a pattern.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M9: a cross-reference entry the reader sets aside is never an unchanged signed object

The reader that judges post-signing changes and the renderer that draws the
page resolve some cross-reference entries differently. A revision appended
after signing could redefine a page's content stream and give the object an
entry the reader sets aside while the renderer follows it:

- an in-use entry at offset 0 (the reader skips it; the renderer fails to read
  the object there, rebuilds its index and takes the last definition);
- a second entry for the same object in one section (the reader keeps the
  first, the renderer the last).

The classifier compared the signed object with itself, found nothing changed
and reported the signature Valid while the page showed the new content.

pdfcraft-cos now records the entries it does not take at face value
(Document::xref_oddities: offset-0 in-use entries other than object 0,
duplicate entries that differ, reserved cross-reference stream types, and a
hybrid /XRefStm entry that disagrees with the table), and can say whether an
entry locates the object itself (Document::xref_entry_is_exact). What it
resolves is unchanged.

classify_changes treats any such entry added after the signed revision as a
change that is never permitted (page content when the object was a signed page
or its content, other changes otherwise). An object whose content compares
equal at a new place is unchanged only when its entry is exact: where the
reader had to search for the object, another reader may arrive elsewhere.

Tests: a_cross_reference_entry_the_reader_sets_aside_does_not_hide_new_page_content,
a_cross_reference_entry_that_misses_its_object_is_not_an_unchanged_object,
cross_reference_entries_that_are_set_aside_are_recorded,
an_entry_that_misses_its_object_is_found_by_searching_but_is_not_exact.

This closes the two demonstrated triggers and their near variants. It does not
prove that the two parsers agree on every file: other differences in what each
tolerates may remain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M8: Apply Redactions is refused when the page still shows text under a mark

Redaction decides what lies under a mark from the positions its own
interpreter computes, and its verify pass reuses them. Where those differ from
the renderer's, a mark drawn over the text as displayed lies over glyphs the
interpreter places elsewhere, or does not see at all: they stayed in the file
and the operation reported success.

The case the test pins: a font and size set through an ExtGState /Font entry
(`gs`), which the renderer honours and the interpreter does not model. The same
kind of gap was observed for a font a form XObject takes from its caller, for
some predefined CMaps and for a rotated Type 3 font matrix. The standard-14
width error that first showed it is fixed at its root by d0d1ad81.

The engine now asks the renderer, which is what the user saw when placing the
marks. After redaction runs, the result is written as a save would write it
(without the marks' own overlay), its text is extracted, and a glyph still at
least half inside a mark fails the edit with the existing "could not be
verified" error. The edit runs on a copy, so the document is unchanged. A page
whose text cannot be read back fails the same way.

This makes such mismatches fail closed; it does not make those documents
redactable. Teaching the interpreter each of them is separate work.

Test: redaction_is_refused_when_the_page_still_shows_text_under_a_mark (the
line whose font is set by `gs` is refused and left unchanged; the same line set
with `Tf` is redacted).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 00:14:40 -04:00