mirror of
https://github.com/storytold/wordcraft.git
synced 2026-11-10 12:20:12 -06:00
* D0+D1: open Word 97-2003 .doc — container, FIB, piece table text (#28) New wordcraft-docbin crate (L2) reads the OLE2 container via the cfb crate (MIT) and parses the FIB and piece table from the [MS-DOC] specification: text pieces decode as UTF-16LE or Windows-1252 (table moved to wordcraft-doc::encoding), subdocument CP ranges, and Word's special characters map to the model's (tabs, breaks, hyphens). .doc/.dot registered in OPEN_EXTS and the I/O dispatch. Encrypted and Word 6/95 files give clear errors; every length is checked and proptest truncation/flips never panic. * D2: .doc formatting — stylesheet, fonts, CHPX/PAPX FKPs, sprms (#28) Sprm decoder with operand-size rules (incl. sprmTDefTable/sprmPChgTabs exceptions and the Prm0 isprm table), bin tables + ChpxFkp/PapxFkp page lookup, STSH styles (StdfBase layout validated against real Word files: bchUpe == cbStd, sti 0 = Normal) and the SttbfFfn font table in both extended and self-delimiting flavours. The CP walk now splits runs on formatting changes and resolves paragraph styles: bold/italic/strike/ underline/caps/hidden, colours (Ico palette + COLORREF), half-point sizes, super/subscript, fonts, highlights; alignment, indents, spacing, keep options, outline levels. Unknown sprms are skipped leniently; an env-gated corpus test opens real files (styles + text asserted). * D3: .doc structure — sections, headers/footers, tables, lists (#28) - Sections: PlcfSed/Sepx with 16-bit measurement sprms (sprmSXaPage & co), explicit sprmSBOrientation with a width/height fallback; even/odd/first header and footer stories from Plcfhdd become document parts. - Tables: sprmTDefTable grids (operand length fixed to cb+1 per the spec), horizontal merges (sprmTMerge), vertical merges (sprmTVertMerge), cell shading (sprmTSetShd / sprmTDefTableShd80), cell widths, valign, row height/header/cant-split, and table borders (sprmTTableBorders80 Brc80s). - Lists: PlfLst (LSTF + appended LVLs) and PlfLfo (LFO + LFOLVL overrides) resolved into the numbering model; sprmPIlvl/sprmPIlfo mark paragraphs. - Combined fixture (intro + table + list + 2 sections + header) round-trips through Document JSON; hostile proptests now mutate the structured fixture. * D4: .doc notes, fields, bookmarks and inline pictures (#28) - Footnotes/endnotes: PlcffndRef/PlcfEndRef reference CPs paired with the PlcffndTxt/PlcfEndTxt story ranges; each story walks into a Footnote or Endnote part and the 0x02 anchor becomes a NoteRef object. - Fields: 0x13/0x14/0x15 state machine in the walker — instruction text is hidden, the cached result kept; HYPERLINK fields keep their result as linked runs, everything else becomes one Field object. - Bookmarks: SttbfBkmk names + Plcfbkf/Plcfbkl CPs become BookmarkStart/ BookmarkEnd objects at their anchors. - Pictures: sprmCPicLocation points into the Data stream; the PICF header gives the display size (dxaGoal/dyaGoal scaled by mx/my) and the payload is format-sniffed (PNG/JPEG/GIF/BMP; metafiles are dropped with a warn). - Hostile proptests now truncate/flip a fixture with all of the above. * D5: .doc import hardening, caps and roadmap (#28) - Cap inline pictures at 5 000 per document. - Engine test: .doc/.dot bytes dispatch through the Word 97-2003 reader (garbage must error, never panic or open empty). - ROADMAP: Word 97-2003 import row (60% — text, formatting, styles, sections, headers/footers, tables, lists, notes, fields, bookmarks, raster pictures). Real-corpus validation stays opt-in: WORDCRAFT_DOCBIN_FIXTURES=<dir> cargo test -p wordcraft-docbin corpus_files_open. * Corpus fixes: compressed-piece FC space, PAPX query point, none-level labels (#28) Validated against two real files (kept locally under plan/word/fixtures, gitignored): a 1 MB Lorem-ipsum sample and a 2003 styles/lists/table sample. - FKP bin tables and pages address the WordDocument stream in true byte offsets; a compressed piece stores its base fc doubled (byte = fc/2), so fc_of_cp now unwinds the doubling. Before, every PAPX/CHPX lookup in compressed (cp1252) files landed on the wrong entry — tables collapsed to one row, lists and heading styles were lost. - The paragraph-properties query now uses the paragraph mark's own FC (inside the paragraph) instead of the FC after it, which picked up the next paragraph's FKP entry. - Fixture papx entries re-keyed to paragraph-start CPs, the real rgfc convention. - Counters::next_label returns nothing for NumFormat::None levels (Word's numFmt none shows no label; also fixes a latent docx-reader case). - Symbol/Wingdings bullet chars map to Unicode glyphs (• ▪ ○ ●). With this, both corpus files render headings, styles, nested and numbered lists, the tables and the PAGE field correctly; metafile-wrapped images and OLE objects stay dropped by policy (logged). * docbin: bound the text walk by real text; dedupe and cap pictures (#28) Hostile CP numbers no longer drive the work: - header/footnote/endnote story ranges from the Table stream are clamped to their subdocument (ccpHdd/ccpFtn/ccpEdn) with checked base additions, and every walk stops at the last piece's CP; a story spanning [0, 4e9) used to push billions of U+FFFD; - characters are found by binary search over the pieces (PieceTable::char_at), and an unreadable stretch (gap, truncated or out-of-stream piece) is skipped in one step rather than one U+FFFD per CP; the old per-character scan was O(chars x pieces); - the piece CPs are made non-decreasing at parse, so inverted pieces are empty and binary search stays sound; surrogate pairs now decode; - note references are looked up by binary search; the style's CharProps is borrowed, not cloned, per character; - pictures are interned by fc_pic (5000 anchors on one picture share one media entry and one copy) and the bytes copied per document are capped at 256 MB. Each fix has a crafted-bytes regression test; against the previous code the story, piece-claim, last-piece and many-pieces tests hang (killed at 60 s) and the shared picture test copies the picture once per anchor (80 GB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docbin: checked offset arithmetic in lists, sections and table rows (#28) - list.rs: PlfLst/PlfLfo/LVL offsets (at + 2 + cLst*28, at + 4 + lfoMac*16, off += len, ...) saturate, which matters on 32-bit targets (wasm32) where fc + n can overflow usize; the LFO override path no longer indexes abstracts directly. - sections.rs: the Sepx range (fc_sepx + 2 + cb) is checked; Plcfhdd is read only as far as the stories a capped section count can use (it used to reserve lcb/4 entries). - table.rs: sprmTDyaRowHeight = i16::MIN negated in i16 (overflow panic in debug builds); now -(v as f32). Regression tests: the row height test failed before (overflow panic with debug assertions, a 0 pt height without); the list test pins the saturating offsets (it only overflowed on 32-bit targets). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Format the merged OPEN_EXTS list Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Brandon Thomas <echelon@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
109 lines
5.3 KiB
Rust
109 lines
5.3 KiB
Rust
//! Character and paragraph formatting lookup: the bin tables (`PlcBteChpx`/`PlcBtePapx`) in
|
||
//! the Table stream point at 512-byte FKP pages in the WordDocument stream, which map FC
|
||
//! ranges to CHPX (grpprls) and PAPX (istd + grpprl) ([MS-DOC] §2.4.6.1/2.4.6.2).
|
||
|
||
use crate::DocbinError;
|
||
|
||
/// Largest bin table (entries) we accept.
|
||
const MAX_BINS: usize = 100_000;
|
||
|
||
/// A parsed `PlcBte*`: FC breakpoints and the FKP page each range lives in.
|
||
pub(crate) struct Bins {
|
||
fcs: Vec<u32>,
|
||
/// Page number (× 512 = byte offset in the WordDocument stream).
|
||
pns: Vec<u32>,
|
||
}
|
||
|
||
impl Bins {
|
||
/// Parse the Plc at `table[fc..fc+lcb]` (aFc[n+1] u32s + aPn[n] u32s).
|
||
pub(crate) fn parse(table: &[u8], fc: u32, lcb: u32) -> Result<Bins, DocbinError> {
|
||
let fc = fc as usize;
|
||
let lcb = lcb as usize;
|
||
if lcb < 4 || !(lcb - 4).is_multiple_of(8) {
|
||
return Ok(Bins { fcs: Vec::new(), pns: Vec::new() });
|
||
}
|
||
let end = fc.checked_add(lcb).ok_or_else(|| DocbinError::Malformed("bin table overflow".into()))?;
|
||
let plc = table.get(fc..end).ok_or_else(|| DocbinError::Malformed("bin table outside the Table stream".into()))?;
|
||
let n = (lcb - 4) / 8;
|
||
if n > MAX_BINS {
|
||
return Err(DocbinError::Limit(format!("bin table has {n} entries")));
|
||
}
|
||
let at = |i: usize| -> Option<u32> { plc.get(i * 4..i * 4 + 4).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]])) };
|
||
let mut fcs = Vec::with_capacity(n + 1);
|
||
for i in 0..=n {
|
||
fcs.push(at(i).ok_or_else(|| DocbinError::Malformed("truncated bin aFc".into()))?);
|
||
}
|
||
let mut pns = Vec::with_capacity(n);
|
||
for i in 0..n {
|
||
// PnBte* keeps the page number in its low bits; the rest must be zero.
|
||
pns.push(at(n + 1 + i).ok_or_else(|| DocbinError::Malformed("truncated bin aPn".into()))? & 0x003F_FFFF);
|
||
}
|
||
Ok(Bins { fcs, pns })
|
||
}
|
||
|
||
/// The page containing `fc`, as a slice of the WordDocument stream.
|
||
fn page<'w>(&self, word: &'w [u8], fc: u32) -> Option<&'w [u8]> {
|
||
let i = self.fcs.partition_point(|&f| f <= fc).checked_sub(1)?;
|
||
let pn = *self.pns.get(i)?;
|
||
let at = pn.checked_mul(512)? as usize;
|
||
word.get(at..at.checked_add(512)?)
|
||
}
|
||
|
||
/// The grpprl of the Chpx covering `fc` ([MS-DOC] ChpxFkp: rgfc u32s, rgb offsets ×2,
|
||
/// crun in the last byte).
|
||
pub(crate) fn chpx<'w>(&self, word: &'w [u8], fc: u32) -> &'w [u8] {
|
||
let Some(page) = self.page(word, fc) else { return &[] };
|
||
let Some(&crun) = page.get(511) else { return &[] };
|
||
let crun = crun.min(0x65) as usize;
|
||
let rgfc = |i: usize| -> Option<u32> { page.get(i * 4..i * 4 + 4).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]])) };
|
||
// rgb array sits right after rgfc (crun+1 u32s); each byte is a half-offset into the
|
||
// region between rgb's end and crun.
|
||
let rgb_at = (crun + 1) * 4;
|
||
let j = (0..crun).rev().find(|&j| rgfc(j).is_some_and(|f| f <= fc));
|
||
let Some(j) = j else { return &[] };
|
||
let Some(&off) = page.get(rgb_at + j) else { return &[] };
|
||
if off == 0 {
|
||
return &[];
|
||
}
|
||
let at = off as usize * 2;
|
||
let Some(&cb) = page.get(at) else { return &[] };
|
||
page.get(at + 1..at + 1 + cb as usize).unwrap_or(&[])
|
||
}
|
||
|
||
/// The (istd, grpprl) of the Papx of the paragraph whose mark is just before `fc`
|
||
/// ([MS-DOC] PapxFkp: rgfc u32s — each the FC *following* a paragraph mark —, rgbx BX
|
||
/// entries (1 offset byte + 12 PHE bytes), crun in the last byte; the PAPX itself is a
|
||
/// PapxInFkp with the cb/cb' size quirk). Query with the FC of the character after the mark.
|
||
pub(crate) fn papx<'w>(&self, word: &'w [u8], fc: u32) -> (u16, &'w [u8]) {
|
||
let Some(page) = self.page(word, fc) else { return (0, &[]) };
|
||
let Some(&crun) = page.get(511) else { return (0, &[]) };
|
||
// BX is 13 bytes; cap crun at what fits before byte 511.
|
||
let max_run = (511usize.saturating_sub((crun as usize + 1) * 4)) / 13;
|
||
let crun = (crun as usize).min(max_run);
|
||
let rgfc = |i: usize| -> Option<u32> { page.get(i * 4..i * 4 + 4).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]])) };
|
||
let rgbx_at = (crun + 1) * 4;
|
||
let j = (0..crun).rev().find(|&j| rgfc(j).is_some_and(|f| f <= fc));
|
||
let Some(j) = j else { return (0, &[]) };
|
||
let Some(&off) = page.get(rgbx_at + j * 13) else { return (0, &[]) };
|
||
if off == 0 {
|
||
return (0, &[]);
|
||
}
|
||
let at = off as usize * 2;
|
||
let u16at = |o: usize| -> Option<u16> { page.get(o..o + 2).map(|b| u16::from_le_bytes([b[0], b[1]])) };
|
||
let Some(cb) = page.get(at).copied() else { return (0, &[]) };
|
||
// PapxInFkp: if cb is nonzero the GrpPrlAndIstd (istd u16 + grpprl) is 2×cb-1 bytes;
|
||
// if cb is zero, the next byte cb' (1 byte) gives 2×cb' bytes of it.
|
||
let (data_at, data_len) = if cb != 0 {
|
||
(at + 1, 2 * cb as usize - 1)
|
||
} else {
|
||
let cb2 = page.get(at + 1).copied().unwrap_or(0) as usize;
|
||
(at + 2, 2 * cb2)
|
||
};
|
||
if data_len < 2 {
|
||
return (0, &[]);
|
||
}
|
||
let istd = u16at(data_at).unwrap_or(0);
|
||
(istd, page.get(data_at + 2..data_at + data_len).unwrap_or(&[]))
|
||
}
|
||
}
|