mirror of
https://github.com/storytold/wordcraft.git
synced 2026-10-31 22:07:06 -06:00
* D0+D1: open Word 97-2003 .doc — container, FIB, piece table text (#28) New wordcraft-docbin crate (L2) reads the OLE2 container via the cfb crate (MIT) and parses the FIB and piece table from the [MS-DOC] specification: text pieces decode as UTF-16LE or Windows-1252 (table moved to wordcraft-doc::encoding), subdocument CP ranges, and Word's special characters map to the model's (tabs, breaks, hyphens). .doc/.dot registered in OPEN_EXTS and the I/O dispatch. Encrypted and Word 6/95 files give clear errors; every length is checked and proptest truncation/flips never panic. * D2: .doc formatting — stylesheet, fonts, CHPX/PAPX FKPs, sprms (#28) Sprm decoder with operand-size rules (incl. sprmTDefTable/sprmPChgTabs exceptions and the Prm0 isprm table), bin tables + ChpxFkp/PapxFkp page lookup, STSH styles (StdfBase layout validated against real Word files: bchUpe == cbStd, sti 0 = Normal) and the SttbfFfn font table in both extended and self-delimiting flavours. The CP walk now splits runs on formatting changes and resolves paragraph styles: bold/italic/strike/ underline/caps/hidden, colours (Ico palette + COLORREF), half-point sizes, super/subscript, fonts, highlights; alignment, indents, spacing, keep options, outline levels. Unknown sprms are skipped leniently; an env-gated corpus test opens real files (styles + text asserted). * D3: .doc structure — sections, headers/footers, tables, lists (#28) - Sections: PlcfSed/Sepx with 16-bit measurement sprms (sprmSXaPage & co), explicit sprmSBOrientation with a width/height fallback; even/odd/first header and footer stories from Plcfhdd become document parts. - Tables: sprmTDefTable grids (operand length fixed to cb+1 per the spec), horizontal merges (sprmTMerge), vertical merges (sprmTVertMerge), cell shading (sprmTSetShd / sprmTDefTableShd80), cell widths, valign, row height/header/cant-split, and table borders (sprmTTableBorders80 Brc80s). - Lists: PlfLst (LSTF + appended LVLs) and PlfLfo (LFO + LFOLVL overrides) resolved into the numbering model; sprmPIlvl/sprmPIlfo mark paragraphs. - Combined fixture (intro + table + list + 2 sections + header) round-trips through Document JSON; hostile proptests now mutate the structured fixture. * D4: .doc notes, fields, bookmarks and inline pictures (#28) - Footnotes/endnotes: PlcffndRef/PlcfEndRef reference CPs paired with the PlcffndTxt/PlcfEndTxt story ranges; each story walks into a Footnote or Endnote part and the 0x02 anchor becomes a NoteRef object. - Fields: 0x13/0x14/0x15 state machine in the walker — instruction text is hidden, the cached result kept; HYPERLINK fields keep their result as linked runs, everything else becomes one Field object. - Bookmarks: SttbfBkmk names + Plcfbkf/Plcfbkl CPs become BookmarkStart/ BookmarkEnd objects at their anchors. - Pictures: sprmCPicLocation points into the Data stream; the PICF header gives the display size (dxaGoal/dyaGoal scaled by mx/my) and the payload is format-sniffed (PNG/JPEG/GIF/BMP; metafiles are dropped with a warn). - Hostile proptests now truncate/flip a fixture with all of the above. * D5: .doc import hardening, caps and roadmap (#28) - Cap inline pictures at 5 000 per document. - Engine test: .doc/.dot bytes dispatch through the Word 97-2003 reader (garbage must error, never panic or open empty). - ROADMAP: Word 97-2003 import row (60% — text, formatting, styles, sections, headers/footers, tables, lists, notes, fields, bookmarks, raster pictures). Real-corpus validation stays opt-in: WORDCRAFT_DOCBIN_FIXTURES=<dir> cargo test -p wordcraft-docbin corpus_files_open. * Corpus fixes: compressed-piece FC space, PAPX query point, none-level labels (#28) Validated against two real files (kept locally under plan/word/fixtures, gitignored): a 1 MB Lorem-ipsum sample and a 2003 styles/lists/table sample. - FKP bin tables and pages address the WordDocument stream in true byte offsets; a compressed piece stores its base fc doubled (byte = fc/2), so fc_of_cp now unwinds the doubling. Before, every PAPX/CHPX lookup in compressed (cp1252) files landed on the wrong entry — tables collapsed to one row, lists and heading styles were lost. - The paragraph-properties query now uses the paragraph mark's own FC (inside the paragraph) instead of the FC after it, which picked up the next paragraph's FKP entry. - Fixture papx entries re-keyed to paragraph-start CPs, the real rgfc convention. - Counters::next_label returns nothing for NumFormat::None levels (Word's numFmt none shows no label; also fixes a latent docx-reader case). - Symbol/Wingdings bullet chars map to Unicode glyphs (• ▪ ○ ●). With this, both corpus files render headings, styles, nested and numbered lists, the tables and the PAGE field correctly; metafile-wrapped images and OLE objects stay dropped by policy (logged). * docbin: bound the text walk by real text; dedupe and cap pictures (#28) Hostile CP numbers no longer drive the work: - header/footnote/endnote story ranges from the Table stream are clamped to their subdocument (ccpHdd/ccpFtn/ccpEdn) with checked base additions, and every walk stops at the last piece's CP; a story spanning [0, 4e9) used to push billions of U+FFFD; - characters are found by binary search over the pieces (PieceTable::char_at), and an unreadable stretch (gap, truncated or out-of-stream piece) is skipped in one step rather than one U+FFFD per CP; the old per-character scan was O(chars x pieces); - the piece CPs are made non-decreasing at parse, so inverted pieces are empty and binary search stays sound; surrogate pairs now decode; - note references are looked up by binary search; the style's CharProps is borrowed, not cloned, per character; - pictures are interned by fc_pic (5000 anchors on one picture share one media entry and one copy) and the bytes copied per document are capped at 256 MB. Each fix has a crafted-bytes regression test; against the previous code the story, piece-claim, last-piece and many-pieces tests hang (killed at 60 s) and the shared picture test copies the picture once per anchor (80 GB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docbin: checked offset arithmetic in lists, sections and table rows (#28) - list.rs: PlfLst/PlfLfo/LVL offsets (at + 2 + cLst*28, at + 4 + lfoMac*16, off += len, ...) saturate, which matters on 32-bit targets (wasm32) where fc + n can overflow usize; the LFO override path no longer indexes abstracts directly. - sections.rs: the Sepx range (fc_sepx + 2 + cb) is checked; Plcfhdd is read only as far as the stories a capped section count can use (it used to reserve lcb/4 entries). - table.rs: sprmTDyaRowHeight = i16::MIN negated in i16 (overflow panic in debug builds); now -(v as f32). Regression tests: the row height test failed before (overflow panic with debug assertions, a 0 pt height without); the list test pins the saturating offsets (it only overflowed on 32-bit targets). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Format the merged OPEN_EXTS list Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Brandon Thomas <echelon@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
110 lines
4.2 KiB
Rust
110 lines
4.2 KiB
Rust
//! Footnotes, endnotes and bookmarks: `PlcffndRef`/`PlcffndTxt`, `PlcfendRef`/`PlcfendTxt`,
|
|
//! and `SttbfBkmk` + `Plcfbkf`/`Plcfbkl` ([MS-DOC] §2.9).
|
|
|
|
/// Cap on notes and bookmarks per document.
|
|
const MAX: usize = 10_000;
|
|
|
|
/// Parse the note reference/story PLCs. `ref_pair`/`txt_pair` are the FIB pair indices
|
|
/// (`(2, 3)` for footnotes, `(46, 47)` for endnotes); story ranges are relative to the
|
|
/// note subdocument.
|
|
pub(crate) fn parse_notes(table: &[u8], fib: &crate::fib::Fib, ref_pair: usize, txt_pair: usize) -> Vec<(u32, (u32, u32))> {
|
|
let refs = fib.pair(ref_pair).and_then(|(fc, lcb)| plc_of(table, fc, lcb, 2)).unwrap_or_default();
|
|
let txts = fib.pair(txt_pair).and_then(|(fc, lcb)| plc_of(table, fc, lcb, 0)).unwrap_or_default();
|
|
if txts.len() < 2 {
|
|
return Vec::new();
|
|
}
|
|
let mut out = Vec::new();
|
|
// The reference PLC's aFtnIdx order matches the story order; pair them by index.
|
|
for (i, &cp) in refs.iter().enumerate() {
|
|
if let (Some(a), Some(b)) = (txts.get(i), txts.get(i + 1))
|
|
&& b > a
|
|
{
|
|
out.push((cp, (*a, *b)));
|
|
}
|
|
}
|
|
out
|
|
}
|
|
|
|
/// The CPs of a PLC (the trailing element is a limit and is kept so callers can pair
|
|
/// range ends) plus the data-element size check. Returns the CPs only.
|
|
fn plc_of(table: &[u8], fc: u32, lcb: u32, data_size: usize) -> Option<Vec<u32>> {
|
|
if lcb == 0 {
|
|
return None;
|
|
}
|
|
let plc = table.get(fc as usize..(fc as usize).saturating_add(lcb as usize))?;
|
|
if plc.len() < 4 {
|
|
return None;
|
|
}
|
|
// aCP has one more element than aData: n = (len - data_size) / (4 + data_size).
|
|
let n = (plc.len() - data_size) / (4 + data_size);
|
|
if n == 0 || n > MAX {
|
|
return None;
|
|
}
|
|
let mut cps = Vec::with_capacity(n.min(MAX));
|
|
for i in 0..=n.min(MAX) {
|
|
match plc.get(i * 4..i * 4 + 4) {
|
|
Some(b) => cps.push(u32::from_le_bytes([b[0], b[1], b[2], b[3]])),
|
|
None => break,
|
|
}
|
|
}
|
|
Some(cps)
|
|
}
|
|
|
|
/// A bookmark boundary to insert while walking.
|
|
pub(crate) enum Mark {
|
|
Start(String),
|
|
End(String),
|
|
}
|
|
|
|
/// Parse `SttbfBkmk` + `Plcfbkf` + `Plcfbkl` into (cp, event) pairs sorted for a sequential
|
|
/// walk. Hidden bookmarks (leading `_`) are kept: the engine can decide.
|
|
pub(crate) fn parse_bookmarks(table: &[u8], fib: &crate::fib::Fib) -> Vec<(u32, Mark)> {
|
|
let names = fib.pair(crate::fib::pair::STTBF_BKMK).and_then(|(fc, lcb)| sttb_strings(table, fc, lcb)).unwrap_or_default();
|
|
let starts = fib.pair(crate::fib::pair::PLCF_BKF).and_then(|(fc, lcb)| plc_of(table, fc, lcb, 4)).unwrap_or_default();
|
|
let ends = fib.pair(crate::fib::pair::PLCF_BKL).and_then(|(fc, lcb)| plc_of(table, fc, lcb, 2)).unwrap_or_default();
|
|
let n = starts.len().saturating_sub(1).min(ends.len().saturating_sub(1)).min(names.len()).min(MAX);
|
|
let mut out = Vec::with_capacity(n * 2);
|
|
for i in 0..n {
|
|
let (Some(&a), Some(&b)) = (starts.get(i), ends.get(i)) else { break };
|
|
if b < a {
|
|
continue;
|
|
}
|
|
if let Some(name) = names.get(i) {
|
|
if name.is_empty() {
|
|
continue;
|
|
}
|
|
out.push((a, Mark::Start(name.clone())));
|
|
out.push((b, Mark::End(name.clone())));
|
|
}
|
|
}
|
|
out.sort_by_key(|(cp, m)| (*cp, matches!(m, Mark::End(_))));
|
|
out
|
|
}
|
|
|
|
/// Strings of an extended STTB (fExtend = 0xFFFF, 2-byte chars, no extra data).
|
|
fn sttb_strings(table: &[u8], fc: u32, lcb: u32) -> Option<Vec<String>> {
|
|
let b = table.get(fc as usize..(fc as usize).saturating_add(lcb as usize))?;
|
|
if b.len() < 6 {
|
|
return None;
|
|
}
|
|
let c_data = u16::from_le_bytes([b[2], b[3]]) as usize;
|
|
if c_data > MAX {
|
|
return None;
|
|
}
|
|
let mut at = 6usize;
|
|
let mut out = Vec::with_capacity(c_data);
|
|
for _ in 0..c_data {
|
|
let cch = match b.get(at..at + 2) {
|
|
Some(x) => u16::from_le_bytes([x[0], x[1]]) as usize,
|
|
None => break,
|
|
};
|
|
at += 2;
|
|
let cch = cch.min(40);
|
|
let chars = b.get(at..at + cch * 2)?;
|
|
at += cch * 2;
|
|
let s: String = chars.as_chunks::<2>().0.iter().map(|c| char::from_u32(u16::from_le_bytes(*c) as u32).unwrap_or('\u{FFFD}')).collect();
|
|
out.push(s);
|
|
}
|
|
Some(out)
|
|
}
|