* M12: preserve document properties when importing legacy DOC and DOT
M12: preserve document properties when importing legacy DOC and DOT
Integrate current upstream while retaining import behavior and embedded-font loading.
* docbin: as_chunks for fixed-size records (clippy::chunks_exact_to_as_chunks on Rust 1.99); rustfmt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: LabDennis <341377186+LabDennis@users.noreply.github.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The story tables of footnotes and endnotes (PlcffndTxt, PlcfendTxt),
headers and footers (Plcfhdd) and text boxes (PlcftxbxTxt) were read
one range at a time, and each range only had to run forwards: a table
stepping back to an earlier CP after every story pointed many notes,
header slots or text boxes at the same text, and each was read into a
part of its own (2,000 note references on one story made 1,001 copies
of it). The CPs of a well-formed PLC ascend, so a story that starts
before the end of the last one kept is now dropped (a header or footer
slot is left empty, which links it to the previous section's): every
subdocument CP becomes at most one story, and the stories together
never hold more than the file's text.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
docbin decrypts XOR-obfuscated, RC4 and RC4 CryptoAPI files after verifying the
password against the stored verifier; the engine reuses the .docx password
prompt errors. Written from [MS-DOC] and [MS-OFFCRYPTO]; synthetic tests only.
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
ODT: every frame inflated its picture entry again, and the byte budget
was checked only after inflating, so frames pointing at an entry that
didn't fit (or at any entry once the budget was used up) inflated it
again each time. Each entry is now inflated at most once, and only when
the size it declares fits in what is left; the bytes inflated and every
later copy are charged, so an entry that inflates past its declared
size uses the budget up and nothing is inflated after it.
.doc: inline pictures remembered only successes, so a picture that
couldn't be read or didn't fit was located, decoded and copied again
for every anchor on it, and an inline shape's BLIP was decoded against
the whole media cap rather than what was left. Failures are remembered
too, bytes are copied and decoded only within what is left of the cap,
and nothing is read once it is used up.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Found by comparing EPA's .doc templates with Word for Windows.
Before: custom tab stops in .doc files were skipped, so centred titles,
right-aligned dates and dot leaders set with tabs lost their alignment;
and a section mark (0x0C) at a section's end became a page break inside
a paragraph, so the file's sections (continuous breaks, footers) were
lost and every section break started a new page.
After: sprmPChgTabsPapx and sprmPChgTabs are read in styles and
paragraphs (removed stops clear inherited ones; each added stop keeps
its alignment and leader), and sprmPChgTabs' one-byte size is decoded
per [MS-DOC] (it was read as two bytes, swallowing the sprms after it).
A section mark at a section's end now ends that section's last
paragraph, so the section's properties and footers are kept. Both EPA
.doc templates in the comparison set now have Word's page count.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* .doc import: inline pictures, floating pictures and basic shapes (#551)
Word 97-2003 files kept their text but dropped every picture and drawing.
Pictures and drawings are now read from the OfficeArt records ([MS-DOC],
[MS-ODRAW]):
- Inline pictures: the PICF's OfficeArtInlineSpContainer (MM_SHAPE /
MM_SHAPEFILE) and its FBSE/BLIP, with crop and alt text from the shape
properties; the older raw-image path stays as a fallback.
- Floating pictures and shapes: PlcSpaMom/PlcSpaHdr anchors joined with
the shapes of the OfficeArtContent drawings (fcDggInfo), placed by the
SPA bounds, bx/by or posrelh/posrelv/posh/posv, wrap type and
fBelowText; rotation and flips; pictures from the BLIP store.
- Shapes: rectangle, rounded rectangle, ellipse, diamond, triangles, star,
block arrows, heart, lines/connectors, with fill/line colour and width;
groups flattened into one group space; text boxes linked to their
PlcftxbxTxt story parts by spid.
- BLIPs: PNG, JPEG, TIFF as-is, DIB converted to PNG, EMF/WMF inflated
(the metafile renderer draws them), PICT kept as a sized placeholder.
- Hostile input: record lengths clamped to their parent, nesting depth
and record counts capped, decompression size-limited, picture bytes
capped per document as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #558: miniz_oxide as a workspace dependency
Restores the change @echelon pushed in 04b59287, which a rebase dropped: miniz_oxide is declared once in the root [workspace.dependencies] and docbin uses it with workspace = true.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
- Indents in characters (w:firstLineChars, w:hangingChars, w:leftChars,
w:rightChars; sprmPDxc* in .doc) are kept in the model, read and written,
and laid out in the paragraph mark's font size, winning over the points
as in Word. Indent commands cancel an inherited character indent.
- .doc: sprmPDxaLeft1 has a two-byte operand (was read as four, so every
first-line indent was lost); sprmPDxaLeft1_80 is read; LSPD line spacing
is decoded as signed dyaLine + fMultLinespace (exact 20pt read as 799pt,
a page per empty paragraph).
- A floating table placed on the page or margin that would cover the text
already on the page and doesn't fit below it moves to the next page.
- A line holding only a page break stays at the bottom of a full page
instead of leaving a blank page.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Follow [MS-DOC] in the Word 97-2003 reader:
- Prm: bit 0 is fComplex; Prm0 isprm in bits 1-7, Prm1 Prc index in bits 1-15 (#458)
- sprmSBkc operands 0-4 map to continuous/new column/new page/even/odd (#459)
- sprmSBOrientation 1 = portrait, 2 = landscape (#460)
- negative sprmPIlfo (0xF802-0xFFFF) references the negated LFO index (#461)
- StdfBase: istdBase shares the word with stk and is zero-based; the
whole base chain is applied, bounded and cycle-safe (#462)
- SttbfFfn entries advance by cchData, covering the optional xszAlt (#463)
One regression test per issue, built in code from the spec.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* D0+D1: open Word 97-2003 .doc — container, FIB, piece table text (#28)
New wordcraft-docbin crate (L2) reads the OLE2 container via the cfb
crate (MIT) and parses the FIB and piece table from the [MS-DOC]
specification: text pieces decode as UTF-16LE or Windows-1252 (table
moved to wordcraft-doc::encoding), subdocument CP ranges, and Word's
special characters map to the model's (tabs, breaks, hyphens). .doc/.dot
registered in OPEN_EXTS and the I/O dispatch. Encrypted and Word 6/95
files give clear errors; every length is checked and proptest
truncation/flips never panic.
* D2: .doc formatting — stylesheet, fonts, CHPX/PAPX FKPs, sprms (#28)
Sprm decoder with operand-size rules (incl. sprmTDefTable/sprmPChgTabs
exceptions and the Prm0 isprm table), bin tables + ChpxFkp/PapxFkp page
lookup, STSH styles (StdfBase layout validated against real Word files:
bchUpe == cbStd, sti 0 = Normal) and the SttbfFfn font table in both
extended and self-delimiting flavours. The CP walk now splits runs on
formatting changes and resolves paragraph styles: bold/italic/strike/
underline/caps/hidden, colours (Ico palette + COLORREF), half-point
sizes, super/subscript, fonts, highlights; alignment, indents, spacing,
keep options, outline levels. Unknown sprms are skipped leniently;
an env-gated corpus test opens real files (styles + text asserted).
* D3: .doc structure — sections, headers/footers, tables, lists (#28)
- Sections: PlcfSed/Sepx with 16-bit measurement sprms (sprmSXaPage & co),
explicit sprmSBOrientation with a width/height fallback; even/odd/first
header and footer stories from Plcfhdd become document parts.
- Tables: sprmTDefTable grids (operand length fixed to cb+1 per the spec),
horizontal merges (sprmTMerge), vertical merges (sprmTVertMerge), cell
shading (sprmTSetShd / sprmTDefTableShd80), cell widths, valign, row
height/header/cant-split, and table borders (sprmTTableBorders80 Brc80s).
- Lists: PlfLst (LSTF + appended LVLs) and PlfLfo (LFO + LFOLVL overrides)
resolved into the numbering model; sprmPIlvl/sprmPIlfo mark paragraphs.
- Combined fixture (intro + table + list + 2 sections + header) round-trips
through Document JSON; hostile proptests now mutate the structured fixture.
* D4: .doc notes, fields, bookmarks and inline pictures (#28)
- Footnotes/endnotes: PlcffndRef/PlcfEndRef reference CPs paired with the
PlcffndTxt/PlcfEndTxt story ranges; each story walks into a Footnote or
Endnote part and the 0x02 anchor becomes a NoteRef object.
- Fields: 0x13/0x14/0x15 state machine in the walker — instruction text is
hidden, the cached result kept; HYPERLINK fields keep their result as
linked runs, everything else becomes one Field object.
- Bookmarks: SttbfBkmk names + Plcfbkf/Plcfbkl CPs become BookmarkStart/
BookmarkEnd objects at their anchors.
- Pictures: sprmCPicLocation points into the Data stream; the PICF header
gives the display size (dxaGoal/dyaGoal scaled by mx/my) and the payload
is format-sniffed (PNG/JPEG/GIF/BMP; metafiles are dropped with a warn).
- Hostile proptests now truncate/flip a fixture with all of the above.
* D5: .doc import hardening, caps and roadmap (#28)
- Cap inline pictures at 5 000 per document.
- Engine test: .doc/.dot bytes dispatch through the Word 97-2003 reader
(garbage must error, never panic or open empty).
- ROADMAP: Word 97-2003 import row (60% — text, formatting, styles,
sections, headers/footers, tables, lists, notes, fields, bookmarks,
raster pictures).
Real-corpus validation stays opt-in: WORDCRAFT_DOCBIN_FIXTURES=<dir>
cargo test -p wordcraft-docbin corpus_files_open.
* Corpus fixes: compressed-piece FC space, PAPX query point, none-level labels (#28)
Validated against two real files (kept locally under plan/word/fixtures,
gitignored): a 1 MB Lorem-ipsum sample and a 2003 styles/lists/table
sample.
- FKP bin tables and pages address the WordDocument stream in true byte
offsets; a compressed piece stores its base fc doubled (byte = fc/2),
so fc_of_cp now unwinds the doubling. Before, every PAPX/CHPX lookup in
compressed (cp1252) files landed on the wrong entry — tables collapsed
to one row, lists and heading styles were lost.
- The paragraph-properties query now uses the paragraph mark's own FC
(inside the paragraph) instead of the FC after it, which picked up the
next paragraph's FKP entry.
- Fixture papx entries re-keyed to paragraph-start CPs, the real rgfc
convention.
- Counters::next_label returns nothing for NumFormat::None levels (Word's
numFmt none shows no label; also fixes a latent docx-reader case).
- Symbol/Wingdings bullet chars map to Unicode glyphs (• ▪ ○ ●).
With this, both corpus files render headings, styles, nested and numbered
lists, the tables and the PAGE field correctly; metafile-wrapped images
and OLE objects stay dropped by policy (logged).
* docbin: bound the text walk by real text; dedupe and cap pictures (#28)
Hostile CP numbers no longer drive the work:
- header/footnote/endnote story ranges from the Table stream are clamped to their
subdocument (ccpHdd/ccpFtn/ccpEdn) with checked base additions, and every walk stops at
the last piece's CP; a story spanning [0, 4e9) used to push billions of U+FFFD;
- characters are found by binary search over the pieces (PieceTable::char_at), and an
unreadable stretch (gap, truncated or out-of-stream piece) is skipped in one step rather
than one U+FFFD per CP; the old per-character scan was O(chars x pieces);
- the piece CPs are made non-decreasing at parse, so inverted pieces are empty and binary
search stays sound; surrogate pairs now decode;
- note references are looked up by binary search; the style's CharProps is borrowed, not
cloned, per character;
- pictures are interned by fc_pic (5000 anchors on one picture share one media entry and
one copy) and the bytes copied per document are capped at 256 MB.
Each fix has a crafted-bytes regression test; against the previous code the story,
piece-claim, last-piece and many-pieces tests hang (killed at 60 s) and the shared
picture test copies the picture once per anchor (80 GB).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* docbin: checked offset arithmetic in lists, sections and table rows (#28)
- list.rs: PlfLst/PlfLfo/LVL offsets (at + 2 + cLst*28, at + 4 + lfoMac*16, off += len,
...) saturate, which matters on 32-bit targets (wasm32) where fc + n can overflow usize;
the LFO override path no longer indexes abstracts directly.
- sections.rs: the Sepx range (fc_sepx + 2 + cb) is checked; Plcfhdd is read only as far
as the stories a capped section count can use (it used to reserve lcb/4 entries).
- table.rs: sprmTDyaRowHeight = i16::MIN negated in i16 (overflow panic in debug builds);
now -(v as f32).
Regression tests: the row height test failed before (overflow panic with debug assertions, a 0 pt height without); the list test pins the saturating
offsets (it only overflowed on 32-bit targets).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Format the merged OPEN_EXTS list
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
divyanshu2074