A /Kids array listing one widget many times, or shared by several fields,
built a Widget for every listing, so a small file expanded into fields times
listings widgets. A widget now belongs to the first field that lists it, and a
page widget already owned is not adopted again.
The tests each branch appends to the audit, edit and organize test files are
kept side by side; prune.rs merges cleanly, with the form memo on top of the
budgeted page content.
The JPEG decoder re-sorts and rescans every pending Extended XMP chunk after
each marker, so a file of many small chunks cost time quadratic in their
number before any pixel limit applied. Turning a photo upright now counts
the chunks first and leaves a JPEG with more than 1024 as it is.
A one-pixel GIF on a huge logical screen decodes to a canvas that size,
which the conversion then copied before splitting it into colour and alpha,
so a few bytes could hold over a gigabyte. Screens past 16 Mpx are refused
before decoding, and the decoded canvas is converted without a copy.
A TIFF's pages were each capped at 2 GiB but all kept in memory until
embedded, so many small directories sharing one compressed strip could ask
for many times that; embedding one image also decoded every page. The pages
of one TIFF are now refused past 4 GiB together, and image_xobject reads
only the first page.
The redaction audit read each page's text through text_lines, which walked
the whole page tree again for every page (quadratic in the page count) and
let /ActualText replace the glyphs, so an empty or substitute ActualText hid
covered text from the audit. Edit gains glyph_text_lines, which reads the
page dictionary it is given and ignores ActualText; the audit uses it.
Text extraction copies a ToUnicode destination for every glyph showing its
code, and the page's total text was not bounded, so a small page could expand
into gigabytes. One page's extraction (its forms included) stops with an error
past 16 MiB of text.
Past the marked-content depth budget the number of open sequences is unknown,
and the mark on top was appended with none of them closed, so the page's own
optional-content scopes (possibly hidden) could hide a redaction box, a
header or a watermark. Placing on top of such a page is now refused before
anything changes; behind it is still possible.
Removing flattened widgets from the field tree recursed into every /Kids
entry with only a depth limit, so a node listing its child twice per level
(or itself) made the walk exponential. Each node's answer is now remembered;
a node met again while it is still being visited survives, as it did when
the depth limit stopped a cycle.
Writing an added text item re-wrapped the whole text inside the loop over its
lines to learn the line count, so multiline text cost time and temporary
memory quadratic in its length. The wrapped lines are computed once.
Each candidate attribute match searched the whole packet before it for the
last < and >, so a long text node repeating the attribute name cost time
quadratic in the packet size (up to the 16 MiB decode cap). The scan now
keeps the last < and > as it moves forward.
The pruner decoded and walked a form XObject again on every Do that drew
it, so a form drawing itself a few times per level was analysed
exponentially often within the depth limit. Forms are now remembered with
the resources owner they were drawn against, and a repeat records nothing
new.
Looking up a named destination and listing attachments in a source's name
trees followed every /Kids entry with only a depth limit, so a node listing
the same child twice per level made the walk (and the attachment list)
exponential in the depth. Both walks now skip a node already read.
Fixing up copied annotations cloned the whole source-to-destination map for
every non-GoTo action and dropped the copies it made, so many annotations
cost quadratic memory and time and an object shared by their actions was
copied once per action. The copier now lends its map to the action copy and
takes it back with the new entries.
Renaming a colliding field tried Name-1, Name-2, ... from 1 every time, so
many fields of one name cost a quadratic number of attempts during import,
combine and their conflict previews. Taken names only grow, so the search
now resumes after the last suffix handed out for that name; the result is
still the first free variant.
Each imported top-level field was checked against every field already in
/AcroForm /Fields before it was appended, so a page with many distinct
widget fields cost time quadratic in their number. Membership is now kept
in a set beside the list.
Adds a test-only work counter to the crate, so tests can bound the work a
hostile document causes without timing it.
Redaction reads page content and form XObjects strictly (upstream) and
within one page's decode budget (this branch): both now go through
DecodeBudget::decode_strict.
Filtered inline image data has no length, so its end is an EI chosen
among candidates, and the content parser chose by a different rule
than the renderer's tokenizer (hayro-syntax): the renderer also takes an
EI with no whitespace before it, and checks a candidate by reading what
follows as operators. With image data such as "7FEI 1 0 0 1 200 0 cm",
the renderer ran a cm that redaction read as image data, so text was
drawn 200 pt from where removal and its verify pass looked.
The parser now takes each inline image's end from the renderer's
tokenizer (already in the tree through pdfcraft-cos), so both read the
same operators after the image; a test runs both on the same fixtures.
Only where that tokenizer reads no image (no EI it accepts, or
parameters it can't read) does the parser fall back to its own rules,
counting the image in Parsed::ambiguous_images, and applying redactions
refuses such a page as unreadable.
Structure cleanup cleared alternate text, actual text and expansions
only on the element that directly owned changed marked content, and an
element that lost an annotation reference kept its descriptions unless
it lost every kid. An ancestor's /ActualText covering a redacted span,
or a link element's /Alt repeating the address of a removed link, stayed
in the saved file. Elements that lose an annotation now always lose
their descriptions, and every ancestor of an affected element (found
through both /P and the tree's /K, which may disagree) loses its own.
An element written in place above one with a description to clear
refuses the operation, as other in-place cases already do.
Finding which marked content a redaction changed copied every operator
into the buffer of each MCID enclosing it, so content nested n deep was
held n times: a modest stream of nested BDCs could exhaust memory.
Each operator inside marked content is now serialized once, and each
MCID keeps the ranges its sequences span. The bytes compared are capped;
MCIDs past the cap count as changed, so their descriptions are cleared
rather than kept.
Clearing the pixels of a partly covered image decoded its stream with
the global 1 GiB allowance and only then cut it to the raster length its
Width, Height and bit depth declare. A small Flate stream could make
every such image allocate and inflate far more than it can use. Decoding
now stops at twice the raster plus a byte per row (filter-chain and
predictor overhead) and a little slack; an image past that is removed
whole, as an undecodable one already is.
After applying redactions, removed images and forms were unlinked from
page-tree resource dictionaries but not from the resources of forms the
page still draws. A form sharing the page's original resource dictionary
(a common producer layout) kept the removed image reachable, so a full
save wrote it. The walk over surviving forms now drops XObject entries
for removed objects that the form's own content does not draw, and fails
closed when that cannot be determined.
The redaction interpreter started every form XObject with an empty text
state (no font, size 0, no spacing) although a form inherits its caller's
graphics state, and it ignored the /Font entry of an ExtGState set by gs.
Text shown that way was placed at the wrong positions, so removal and
the verify pass could both miss glyphs under a mark. Forms now start
from the caller's state, and gs sets the font and size as Tf does.
The engine's renderer cross-check test used the gs case as its example
of interpreter/renderer disagreement; it now runs the check directly on
an unredacted page, which also covers an input stream labelled as a
redaction overlay.
Both checks for text left under applied redaction marks (redaction's own
verify pass and the engine's renderer cross-check) skipped any page
content stream whose dictionary said /PCMark /Redaction, so that the
overlay's label would not count. An input file could put that label on
its own content stream and keep it out of both checks. Applying
redactions now records the overlay streams it adds (stamp returns the
new stream), and only those are skipped.
Deleting a field took it off its pages and out of the field tree but left
its entry in the form's /CO array. That reference kept the field
dictionary, its value and its widgets reachable, so a full save after
applying redactions over the field still wrote the redacted value.
A run of images that is not one picture was searched again from each of
its groups, collecting the rest of the run every time, so many images drawn
over one another made a page quadratic. The search reads at most 16 groups
per content operator (at least 65 536) and leaves the rest of the page as
it is once that is spent.
The pixel reader capped the image size the dictionary gives, then decoded
the JPEG at the size its header gives and compared afterwards, so a small
image dictionary over a huge JPEG allocated the huge one first. The reader
takes the size from the JPEG header and refuses a mismatch before decoding.
The body-size tally and the RTF colour table searched every slot seen so
far for each block, and the RTF writer searched the colour table again for
every paragraph and cell, so a document with many distinct sizes or colours
made export quadratic. Both keep their first-seen order and look slots up
in a hash map.
Line wrapping built and measured the whole candidate line for every word,
so a long field value or annotation text at a tiny font size, with many
words on one line, took quadratic time and copying. The wrapper keeps the
line's width as it goes and measures each word, space and broken
character once.
Code-to-CID and /W range lookups scanned every record in declaration
order, once per glyph and once per explicit ToUnicode entry while a font
was read, so a few thousand records multiplied by as many codes took
billions of steps. Both tables are indexed once into stretches whose
first-declared covering record wins, the precedence the scan had, and the
/W ranges stop at the CMap entry limit.
A sequential bfrange stores one destination and builds every covered
glyph's text from all of it, so one huge destination and many repeats of
its codes made text extraction copy megabytes per glyph. Destinations keep
at most 256 UTF-16 units (the format's 512 bytes), and marked-text
extraction stops asking for a glyph's text once its id has all the text it
keeps.
Figure listing decoded and parsed a page's content once per figure on it,
so many small Figure elements pointing at one page multiplied the work.
The figures are grouped by page and one pass gathers the bounds of every
marked-content id they own; what a sequence draws also counts for the
sequences around it, as before.
Marked-text extraction and figure bounds searched the whole marked-content
stack before every operator, so deeply nested unmarked BMC blocks made a page
quadratic. The text stack now holds each level's effective id (its own, else
the enclosing one's), and the bounds pass counts the open sequences that
match.
The name-tree count cloned each node's /Kids array and skipped repeated
references without charging the budget, so a node listing the same kid
many times cost work proportional to every entry while the budget saw one
step. The walk now borrows the array, charges every entry it does not
descend into, and stops once the budget is spent.
Metadata, private application data, associated files and additional
actions were removed from the catalog and pages only. The same entries on
an image or font (XMP), a Form XObject (PieceInfo), a page or annotation
(/AF with its embedded file) or a form field (/AA scripts) survived the
full rewrite, and the scan did not count them, so removing only such a
category reported nothing to remove. A walk of every object reachable from
the catalog now counts and strips these entries wherever they are, leaving
name maps such as resource dictionaries alone (their keys are names a
producer chose), and refuses a graph nested too deeply to walk in full.
Removing hidden text or hidden layers rewrote page and Form content but
left the structure tree alone, unlike applying redactions. An element's
ActualText, Alt or expansion text still described the removed content,
and a marked-content reference into a Form kept the original Form, with
the removed text, in the full rewrite. The hidden-content pass now cleans
the tags of what each page no longer shows and moves Form references to
the rewritten copy, refusing when the tree can't be cleaned completely.
Removing form fields flattens them and then drops the interactive form,
but flattening keeps hidden and no-view widgets (they draw nothing) and
widgets with no appearance. The pages still listed them, so their values
and their parent fields' values survived the full rewrite. Every widget
left after flattening now leaves the pages and the structure tree, inline
annotations included.
Page and Form content was decoded tolerantly, keeping what came before a
corrupt spot and dropping the rest without saying so. A page whose content
broke at its first bytes therefore read as empty: the hidden-text scan
counted nothing, the removal was skipped and the original stream, which
other viewers still draw, stayed in the file. Content is now decoded
strictly; an unreadable page counts as something to remove, so the removal
runs and refuses the page, and a Form read only in part is treated like an
undecodable one.
The hidden-information count followed every /Kids entry of the attachment
and document-script name trees with only a depth limit, so a node listed
twice by each level was read once per path: a few hundred bytes made the
scan take 2^32 steps. Indirect nodes are now read once each, under a total
node budget.
The /Rect fallback dropped non-numbers before checking the length, so a
malformed /Rect such as [0 0 /Bad 100 100] read as a valid rectangle and
gave the mark quads. The array itself must now hold exactly four finite
numbers.
A quad whose corners all lie on one line (or on one point) encloses
nothing, yet such /QuadPoints counted as drawable, replaced the mark's
/Rect and left an imported redaction that covers nothing. Any quad without
area now makes the mark's quads unusable, so its /Rect is used instead.
Merge Data Files into Spreadsheet writes field names and values from other
people's files as CSV cells, escaping only the delimiters. A cell starting
with =, +, - or @ is evaluated as a formula when the CSV is opened in a
spreadsheet program. Such cells, after any leading whitespace, are now
written with a leading apostrophe; finite numbers such as -12.5 are left
as they are.
A redaction mark's outline is drawn from its /QuadPoints, but applying the
mark falls back to its /Rect when it has none. An imported mark with only a
/Rect, or with quads the outline can't draw, therefore had no visible
outline while still removing content when applied, and an imported opacity
of 0 hid the outline too. Imported marks now get quads from their /Rect
when their own are unusable (or none, so they remove nothing), and their
/CA and /ca are dropped.
Comment import replaces an existing annotation with the same /NM in place.
The check for annotations it must keep covered locked comments and
redaction marks but not widgets or field dictionaries, so an imported
comment could overwrite a field's widget object, setting its value and
validation entries without the form value setter while the AcroForm field
list still pointed at it. Widgets and field dictionaries are now kept, and
the imported comment is added beside them under no name.
The fragment merge multiplied the merged width by a height cast from a
float without checking either fit. A tiny fragment rectangle gives a huge
height: the multiplication overflowed (a panic in debug builds, a wrap past
the pixel limit in release builds) and the u32 cast saturated, sizing an
enormous allocation. The height must now fit u32 before the pixel limit is
checked, and the buffer size uses checked multiplication.
Image export parsed an Indexed colour space's base before checking whether
that base was itself Indexed. A base that refers back to the same array
recursed until the stack overflowed. The base is now checked first; since
Indexed is the only colour space that recurses, this also ends any cycle.
Pages printed as pictures were drawn with everything on them, so form fields only
printed the page content, and the document with stamps printed every comment. The
pictures are drawn from a copy holding only what the vector print draws.
The atomic writer gave the new file the replaced file's mode but left it in
the saver's own group, so a file readable by one group became readable by
another. The staging file joins the replaced file's group before it gets the
mode and before anything is written; where it can't, group and others get no
access.
A step that applies an edit logged any error and carried on, which is right
for a flatten with nothing to flatten but not for removing hidden
information or sanitizing: on a signed file the removal was refused, and the
unchanged file was written out as the action's successful result. Those
steps now fail the action unless there was nothing to remove.
Printing checked the print permission but not the separate high-quality one
(Table 22, bit 12), so a document limited to low-resolution printing came out
as its original vector pages, text included. Such a document now always
prints as pictures of at most 150 dpi, whatever the settings ask for.
Compare read both documents' words out with no check of the copying
permission, and its result, report and difference comments carry that text,
so a copy-restricted document's text could be taken out through a comparison.
Both documents must now allow copying.
text_lines returned a copy-restricted document's lines, ocr_recognize returned
the words it recognised, and text_edit on a paragraph echoed the paragraph's
own text, all without checking the copying permission that the other text
tools check. text_lines is refused like them; recognising and restyling still
work with permission to change the document, and leave the text out.
eval and the Function constructors compiled code built at run time, which never
passed the size and nesting checks a script passes before it runs, so a short
form script could hand boa's recursive parser a deeply nested string and
overflow the stack. Form scripts now refuse to compile code from strings. XFA
scripts, which the runner itself starts through eval, hold such code to the
same checks as the script.
text_editor_fonts counted only the programs it kept against its 32 MiB
budget, so up to 128 aliases of one unusable program were each inflated
in full. Every decode now spends a shared 64 MiB allowance.
embedded_sfnt decoded each program with a fresh 16 MiB allowance and
charged nothing when the program turned out unusable, so a page whose
font resources all named one bad program inflated it once per alias.
embedded_sfnt_within charges every decode to a caller's DecodeBudget.
Callers with a tighter limit for one stream (an embedded font program)
need it alongside the shared allowance. A decode that runs past that
bound charges the bound, the work it took, so repeated failures still
spend the allowance.
The editor's page readers decoded every /Contents entry and every form
entry with a fresh 1 GiB allowance, and the reading-order image
inventory added a record per image draw in every form entered, so a few
shared forms drawing each other produced millions of records. Content
streams and form entries are decoded against one per-page DecodeBudget
(content past it is an error, so no edit rewrites a page from part of
its content; forms past it are reported truncated), and an inventory
past 100,000 image draws is refused.
The cover walk decoded every /Contents entry and every form draw with
a fresh 1 GiB allowance; the operator bound does not count whitespace,
so a large blank form drawn many times meant hours of decoding. The
walk charges a per-page DecodeBudget with the strict decoder, and
spending it fails the audit.
The interpreter entered a form again at every Do with only a nesting
limit, so a few shared forms drawing each other produced exponential
work in apply, sanitize and verify. Scope carries a per-page
DecodeBudget for form entries and form bytes; past it, apply refuses
the page and verification counts the form as possible residue. Page
content streams are decoded against a page budget too.
Each field action and document-script entry decoded its /JS stream
with a fresh 1 GiB allowance and kept its own copy, so many fields or
name-tree pairs naming one compressed stream retained gigabytes. A
reading of the fields, one field's actions, and the document scripts
each share a 64 MiB DecodeBudget for their script streams.
Pruning and page_as_form joined every /Contents entry with a fresh
1 GiB allowance each, so a page naming one compressed stream many
times became gigabytes. Both share one helper that decodes against a
per-page DecodeBudget and fails when it runs out.
Printing and the label text-direction check decoded every /Contents
entry with a fresh 1 GiB allowance, and the direction check re-entered
forms at every Do with only a depth limit, so a few hundred bytes of
nested repeated draws meant hundreds of thousands of form walks. Both
charge a per-page DecodeBudget: printing refuses a page whose content
runs past it, and the direction check stops reading there.
Image measurement and fragment merging decoded every /Contents entry
and every form occurrence with a fresh 1 GiB allowance, so repeated
references to one compressed stream multiplied memory. Both charge a
per-page DecodeBudget. A page whose content does not decode within it
is left unmerged, and its images are not resampled, since a use may be
missed; fragment merging no longer rewrites a page from content it
could not fully decode.
The checker decoded every /Contents entry and every form a page drew
with a fresh 1 GiB allowance each, so a page repeating one compressed
stream reference produced gigabytes. The two copies of the page-content
helper are one now, and it and the form walk charge a per-page
DecodeBudget.
MAX_DECODED bounds one decode, so a page whose /Contents repeats a
reference to one compressed stream, or a form drawn many times, gets a
fresh 1 GiB allowance per occurrence. DecodeBudget is one cumulative
decoded-byte allowance plus a form-entry count and a seen-set of stream
references, shared by a whole page walk. A decode that runs past the
remaining allowance spends all of it.
A timestamp counted as trusted time whenever its signer's certificate
chained to a trusted certificate, whatever the certificate was issued
for. Any key from a trusted CA could then vouch for an arbitrary time.
Trusted time, for a document timestamp and for a signature's embedded
one, now also needs a certificate whose extended key usage includes
timeStamping, whose key usage allows signing, and which has no critical
extension PdfCraft doesn't process (RFC 3161 §2.3). The test timestamp
authority gets such a certificate.
Chain building accepted an intermediate CA without looking at its critical
extensions, which were checked only on the signer's certificate. A
subordinate CA limited by critical name constraints could vouch for a
signer outside its namespace, because PdfCraft does not enforce those
constraints. A CA certificate (other than a self-signed root, whose
limits are the trust store's business) with a critical extension
PdfCraft doesn't process is refused as an issuer, and the reason is
reported.
Revision checks compared objects only. A later revision that pointed the
trailer at another catalog already present in the signed file changed no
object, so the signature stayed valid while the reader showed a different
document. A /Root that differs from the signed revision's is now a
disallowed change to the document structure.
An OCSP response's signer counted as the issuer itself whenever its
certificate carried the issuer's subject name. A response naming its
responder by key hash could embed a certificate for any key with the
issuer's name copied in, skip the delegation checks, and mark a valid
certificate revoked. Whether the responder is the issuer now follows from
how it was selected (the issuer's own name or key hash); every embedded
certificate must be signed by the issuer and carry the OCSP-signing
purpose.
The example passed a constant seed to set_encryption. AES-256 derives the
file key from the seed alone, so anyone could rebuild the key and read a
file it protected without either password. The seed now comes from the
operating system's randomness through RandomState, as in the engine.
The qpdf oracle test wrote into temp_dir()/pdfcraft-enc-<pid>, reusing the
directory if it existed and following any symlink inside it, so another
local account could prepare it and have the test overwrite a file of the
developer's. The directory now has an unpredictable name, is created by
the test (mode 0700 on Unix) and the files are created new.
A stream that named its own crypt filter (/Filter /Crypt with
/DecodeParms /Name) kept that name when the document was given new
password protection. The new /Encrypt defines only /StdCF, so the
inherited name resolved to /Identity and the stream was written in
clear into a file the user asked to protect. When saving with protection
applied by set_encryption, a selector the new handler does not define is
removed with its parameters and the stream follows the new policy.
Saving with the original handler keeps the source's selectors.
aiongg