Files
pdfcraft/crates/xfa/src/packets.rs
T
42ddb2d546 XFA: lay dynamic forms out from their template and keep their data in step (#60) (#121)
* M12: lay dynamic XFA forms out from their template (#60)

A dynamic XFA form is a PDF shell around an XML template: one placeholder
page ("requires Adobe Reader") and /NeedsRendering true. Every viewer
without an XFA engine shows the placeholder, and so did PrintCraft.

New crate printcraft-xfa (L3, already reserved in the layering table). It
reads the XDP packets, parses the template (XFA 3.3: subforms, areas,
fields, draws, exclusion groups, page sets, fonts, paragraphs, borders,
captions, rich text, inline JPEGs, occurrences, breaks, overflow leaders,
column widths), lays it out (positioned, tb, lr-tb, table/row; containers
without a width size to their content; flow across content areas and page
masters; header rows repeat after a break; hidden takes no space; page
number and count fields resolve) and writes real pages and AcroForm
widgets into the document: text, date (AFDate format and keystroke
actions from the picture clause), check boxes, radio groups, push buttons
with reset/print/save/URI actions. Standard-14 fonts stand in for Arial,
Courier New and friends; nothing is embedded.

The engine lays a dynamic form out when it opens it. The original bytes
stay: the layout is one appended revision, the widgets get appearances
from the forms crate, and the XFA packets remain so Adobe's viewers keep
rendering the form their way. A marker on the AcroForm (/PCXfaLayout)
stops a saved form from being laid out twice; each field keeps its SOM
path (/PCSom) for the data round trip to come. doc_info, the CLI's info
and the notice bar report the layout.

Checked on IRCC's IMM 5669 (Schedule A, Designer 6.2, 464 KB template,
AES-encrypted, usage rights): four pages and 308 fields in 27 ms, the
same page structure as the form, filled through form_fill, saved in
place with the original bytes intact and reopened with the values.

Not done, and said so in the parity notes: the datasets packet is neither
read nor written, so Adobe's viewers don't show values filled in here;
no FormCalc or XFA JavaScript (rows don't grow, nothing calculates);
choice lists are text fields; signature, image and barcode fields are
blank; metrics are approximate and fidelity is eyeballed, not measured.

Tests: crates/xfa (parsing, sizing, pagination with repeated headers,
PDF writing, hostile input), the engine (open, fill, incremental save,
reopen without a second layout), the automation tools (open, info,
render, fill, save, reopen) and the UI notice (kittest).

* M12: XFA forms read and write their datasets packet; static forms too (#60)

With the layout alone, a form filled in PrintCraft opened blank in Adobe
Reader, and one filled in Acrobat opened blank here: Adobe's viewers keep
the values in the XFA datasets packet, not in the AcroForm fields.

The xfa crate's new data module reads the datasets packet (normal binding:
a field's node is named after it under its named ancestor subforms,
repeated instances as repeated siblings) and writes it back from the
fields. Laying a form out now takes each field's value from its data node
(dates in ISO form shown in the field's pattern, check boxes and radio
groups by their on values) and gives a repeating subform or table row as
many instances as the data has, so rows another viewer added appear.
Every laid-out field carries its SOM path and its on/off items, so the
packet can be rebuilt from the AcroForm at any time.

The engine writes the datasets after every form edit (one revision with
the field change), and on open applies the datasets to the fields of a
static XFA form, or of a dynamic one laid out earlier, when another
viewer changed them since. The static-form notice now says the two are
kept in step instead of warning that Acrobat may disagree.

Checked on IRCC's IMM 5669: fill, save in place (original bytes intact),
and the file holds <IMM_5669><page1><familyName>SINGH</familyName>…, the
structure Acrobat writes for this form; reopening shows the values.

Not done: explicit bind refs, global binding and data descriptions are
not followed; no import or export of standalone XML or XDP data files.

* M12: XFA layout memoizes container sizes, bounding measurement work (#60)

A container without a width measured its content for its width and again for its
height at every level, so nested width-less subforms took 2^depth measurements, all
synchronously in Session::open. Container sizes are now cached per (node, width,
depth), the root and page-area items keep stable addresses instead of being cloned
per measurement, and a measurement budget turns anything left into TooLarge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12: rustfmt the merged XFA notice (#60)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12: XFA datasets are merged into, not replaced; capped; UTF-16 kept; rewrites recorded (#60)

- write_datasets merged nothing: it replaced <xfa:data> with the AcroForm-bound values
  alone, dropping unbound, namespaced and deep data. It now edits the existing packet
  text in place: a bound node's text is replaced only when it differs, missing nodes are
  added below the deepest existing one, and everything else (unbound data, other
  namespaces, attributes, comments, layout) stays byte for byte. A packet of an XFA array
  that relies on the envelope's prefixes is read inside a wrapper declaring them.
- What can't be written (a node holding structured content, too-deep paths or absurd
  SOM indices, conflicting fields, an unreadable or non-UTF packet) comes back as
  DatasetsWrite.warnings; the engine keeps them in Document.xfa_warnings and
  info.warnings (surviving re-reads), and doc_info reports xfa_warnings.
- Building data counted same-name siblings for every one it added (quadratic in an
  untrusted index) with no total cap. Children are now indexed by name, indices above
  10,000 and paths deeper than 64 are refused, and one write creates at most 20,000 nodes.
- Packets are decoded exactly (UTF-8, UTF-16 LE/BE with or without BOM) and written back
  in the encoding they came in; anything else is reported and left untouched instead of
  being rewritten from a lossy UTF-8 reading.
- A static XFA form whose fields are rewritten from the datasets on open now says so in
  info.warnings, naming the fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 23:24:54 -04:00

160 lines
6.8 KiB
Rust

//! The XDP packets inside a PDF (`/AcroForm /XFA`, ISO 32000-1 §12.7.8): either one stream
//! holding the whole XDP, or an array of (name, stream) pairs.
use pdfcraft_cos::{Document, Object};
use crate::XfaError;
/// Most decoded packet bytes read in total.
const MAX_XDP: usize = 64 << 20;
/// What was found in the XFA entry.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct Packets {
/// The XDP as one XML document (packets concatenated in order), or the template packet
/// alone when there is no XDP envelope. Empty when the entry holds no template.
pub xdp: String,
/// `/NeedsRendering true`: the form expects to be laid out from the template.
pub needs_rendering: bool,
/// Whether the AcroForm has any fields of its own.
pub has_fields: bool,
}
/// How a packet's bytes encode its text.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Encoding {
/// UTF-8, with or without a byte order mark.
Utf8 { bom: bool },
/// UTF-16, little-endian, with or without a byte order mark.
Utf16Le { bom: bool },
/// UTF-16, big-endian, with or without a byte order mark.
Utf16Be { bom: bool },
}
fn utf16(bytes: &[u8], from: fn([u8; 2]) -> u16) -> Option<String> {
let (chunks, rest) = bytes.as_chunks::<2>();
if !rest.is_empty() {
return None;
}
char::decode_utf16(chunks.iter().map(|c| from(*c))).collect::<Result<String, _>>().ok()
}
/// A packet's text and how it was encoded, when it decodes without loss: UTF-8 or UTF-16,
/// told apart by a byte order mark or (without one) by how `<` is written. `None` for
/// anything else (invalid UTF-8, a legacy encoding, a broken surrogate), which can be read
/// only approximately and so must not be written back.
pub fn decode(bytes: &[u8]) -> Option<(String, Encoding)> {
match bytes {
[0xFF, 0xFE, rest @ ..] => utf16(rest, u16::from_le_bytes).map(|t| (t, Encoding::Utf16Le { bom: true })),
[0xFE, 0xFF, rest @ ..] => utf16(rest, u16::from_be_bytes).map(|t| (t, Encoding::Utf16Be { bom: true })),
[0xEF, 0xBB, 0xBF, rest @ ..] => String::from_utf8(rest.to_vec()).ok().map(|t| (t, Encoding::Utf8 { bom: true })),
[b'<', 0, ..] => utf16(bytes, u16::from_le_bytes).map(|t| (t, Encoding::Utf16Le { bom: false })),
[0, b'<', ..] => utf16(bytes, u16::from_be_bytes).map(|t| (t, Encoding::Utf16Be { bom: false })),
_ => String::from_utf8(bytes.to_vec()).ok().map(|t| (t, Encoding::Utf8 { bom: false })),
}
}
/// `text` in `encoding` (the inverse of [`decode`]).
pub fn encode(text: &str, encoding: Encoding) -> Vec<u8> {
let units = |bom: bool, to: fn(u16) -> [u8; 2]| {
let mut out = Vec::with_capacity(text.len().saturating_mul(2).saturating_add(2));
if bom {
out.extend_from_slice(&to(0xFEFF));
}
for u in text.encode_utf16() {
out.extend_from_slice(&to(u));
}
out
};
match encoding {
Encoding::Utf8 { bom } => {
let mut out = Vec::with_capacity(text.len() + 3);
if bom {
out.extend_from_slice(&[0xEF, 0xBB, 0xBF]);
}
out.extend_from_slice(text.as_bytes());
out
}
Encoding::Utf16Le { bom } => units(bom, u16::to_le_bytes),
Encoding::Utf16Be { bom } => units(bom, u16::to_be_bytes),
}
}
/// A packet's text for reading: exact when it decodes, else UTF-8 with replacement characters.
fn to_text(bytes: &[u8]) -> String {
if let Some((text, _)) = decode(bytes) {
return text;
}
let lossy16 = |rest: &[u8], from: fn([u8; 2]) -> u16| -> String {
char::decode_utf16(rest.as_chunks::<2>().0.iter().map(|c| from(*c))).map(|c| c.unwrap_or('\u{FFFD}')).collect()
};
match bytes {
[0xFF, 0xFE, rest @ ..] => lossy16(rest, u16::from_le_bytes),
[0xFE, 0xFF, rest @ ..] => lossy16(rest, u16::from_be_bytes),
[b'<', 0, ..] => lossy16(bytes, u16::from_le_bytes),
[0, b'<', ..] => lossy16(bytes, u16::from_be_bytes),
_ => String::from_utf8_lossy(bytes.strip_prefix(&[0xEF, 0xBB, 0xBF]).unwrap_or(bytes)).into_owned(),
}
}
/// Read the XFA packets, if the document has any.
pub fn read_packets(doc: &Document) -> Result<Option<Packets>, XfaError> {
let Some(root) = doc.root() else { return Ok(None) };
let catalog = doc.get(root);
let Some(acro) = catalog.as_dict().and_then(|c| c.get(b"AcroForm")).map(|a| doc.resolve(a)) else { return Ok(None) };
let Some(acro) = acro.as_dict() else { return Ok(None) };
let Some(xfa) = acro.get(b"XFA").map(|x| doc.resolve(x)) else { return Ok(None) };
// `/NeedsRendering` is a catalog entry (ISO 32000-1 Table 28); some producers put it on
// the form dictionary, so look there too.
let bool_of = |v: Option<&Object>| match v.map(|v| doc.resolve(v)).as_deref() {
Some(Object::Bool(b)) => Some(*b),
_ => None,
};
let needs_rendering = bool_of(catalog.as_dict().and_then(|c| c.get(b"NeedsRendering"))).or_else(|| bool_of(acro.get(b"NeedsRendering")));
let has_fields = acro.get(b"Fields").map(|f| doc.resolve(f)).and_then(|f| f.as_array().map(|a| !a.is_empty())).unwrap_or(false);
let mut total = 0usize;
let mut read = |o: &Object| -> Result<Option<String>, XfaError> {
let s = doc.resolve(o);
let Object::Stream(s) = &*s else { return Ok(None) };
let bytes = s.decoded_within(MAX_XDP)?;
total = total.saturating_add(bytes.len());
if total > MAX_XDP {
return Err(XfaError::TooLarge("the XFA packets are larger than 64 MB".into()));
}
Ok(Some(to_text(&bytes)))
};
let mut xdp = String::new();
match &*xfa {
Object::Array(items) => {
let mut template = None;
let mut i = 0;
while i < items.len() {
let name = items.get(i).and_then(|n| n.as_string()).map(|s| s.to_text());
let stream = if name.is_some() { items.get(i + 1) } else { items.get(i) };
if let Some(o) = stream
&& let Some(text) = read(o)?
{
if name.as_deref() == Some("template") {
template = Some(text.clone());
}
xdp.push_str(&text);
xdp.push('\n');
}
i += if name.is_some() { 2 } else { 1 };
}
if !xdp.contains("<template") {
xdp = template.unwrap_or_default();
}
}
Object::Stream(_) => {
xdp = read(&xfa)?.unwrap_or_default();
}
// Not a stream or an array: an XFA entry all the same, with nothing to lay out.
_ => {}
}
if !xdp.contains("<template") {
xdp.clear();
}
Ok(Some(Packets { xdp, needs_rendering: needs_rendering.unwrap_or(false), has_fields }))
}