5 Commits

Include renames

Author	SHA1	Message	Date
Martin BallandClaude Opus 5.5	17d19b42c5	#3642: review — the DIFAT sample is test-only, valid .doc/.xls files pass the guard, and budget drops are logged ... - repeating_difat_sample is built only for tests: this crate's, and the readers' through a `test-support` feature they turn on as a dev-dependency; it is no longer in the production API. - Tests write valid compound files with the cfb crate in Word's and Excel's shapes (WordDocument + table stream with Data, summary and ObjectPool streams; Workbook with summary streams), in 512- and 4096-byte sectors, plus an 8 MB one whose FAT sectors spill into a DIFAT chain; each passes cfb_guard::check and opens (no Word- or Excel-made files are added). - The .doc work budget notes what it leaves out (floating objects, link targets, alt text, Prm formatting) and logs one warning per kind once the document is read (#3692). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CX1hMKX28764Lrx5moxs1o	2026-10-25 23:47:39 +00:00
Dylan M. Taylor	7f38f41d8c	Use chrono instead of hand-rolled UTC conversion (#3427) ... * Use chrono for UTC/calendar conversion instead of hand-rolled copies Replaces the hand-rolled civil_from_days / days_from_civil copies and the Excel serial-date, FILETIME and logging/PDF date conversions with chrono as a workspace dependency (default-features = false, std only, so wasm is unaffected). Output formats are unchanged and pinned by the existing (and one new) test. Excel's 1900 leap-year bug and the 1900/1904 systems keep their explicit special cases; a small proleptic-Gregorian fallback covers years outside chrono's NaiveDate range so no input changes behaviour or panics. AGENTS.md now says time goes through chrono. Follow-up to storytold/cadcraft#1785 * Never panic or reshape date output for clocks outside chrono's range chrono's From<SystemTime> unwraps, so a clock past year 262143 panicked inside the logger. logging::timestamp now uses the fallible DateTime::from_timestamp and reads such a clock as the epoch, as the old saturating code did. iso_from_unix_secs fell back to the Unix epoch for timestamps chrono rejects; it now keeps the proleptic Gregorian date civil_from_days gives, restoring the pre-chrono result for supported out-of-range u64 inputs. Years past 9999 are written with {:04} rather than %Y (which prepends a '+') in logging::timestamp, iso_from_unix_secs, docbin::filetime_iso and odt::now_iso, so byte-for-byte output is preserved. filetime_iso gained a docstring; AGENTS.md's chrono bullet now notes the narrow out-of-range fallback. Tests cover each boundary. * Don't build an unrepresentable SystemTime in the far-future test UNIX_EPOCH + Duration::from_secs(u64::MAX / 2) overflows Windows' SystemTime and panicked before timestamp() ran, failing the Windows test job. Build it with checked_add and only assert where it exists. * Route out-of-range dates through chrono's 400-year cycle, not hand-rolled copies The PR kept hand-rolled Hinnant `civil_fallback`/`days_from_civil_fallback` copies for dates outside chrono's ±262143-year range, and AGENTS.md carved a "narrow fallback" exception for them. That left calendar arithmetic outside chrono after all. Replace every copy with the Gregorian 400-year cycle (146_097 days): chrono converts the day/year within one cycle and whole cycles are added back via i128 (so i64::MIN/MAX can't overflow). This keeps chrono as the only calendar logic and reproduces the pre-chrono proleptic Gregorian result for every value: - engine cmd/mod.rs `civil_from_days` (reached by the system clock and by `iso_from_unix_secs` for a u64 seconds count past chrono's range). - engine cmd/controls.rs `days_from_civil` and `days_in_month` (`days_in_month` now uses `rem_euclid(400)` since month lengths repeat every 400 years). - formats sheet.rs `civil` and docx chart.rs `civil` (Excel serials). Outputs are unchanged: verified the cycle result equals the old formula over a wide range and at the extremes with a scratch chrono program. New tests pin accepted inputs past chrono's range on both sides of the epoch, through the public/lib entry points and the private converters, with values from the old formula. Removed the AGENTS.md "narrow fallback" exception. pdf/lib.rs `civil` is left as-is: its only caller is the live system clock and its year is clamped to 0..9999 by the PDF DateTime, so the epoch fallback past year 262143 is the accepted live-clock behavior, not a hand-rolled copy. Tested: cargo test -p wordcraft-engine -p wordcraft-formats -p wordcraft-docx; fmt --check; clippy --all-targets -D warnings. * logging/docbin/odt/pdf: keep extended-year dates, no epoch fallback FredApps (CHANGES_REQUESTED) showed `DateTime::from_timestamp(..).unwrap_or_default()` changes output: a timestamp the old code converted past chrono's year 262143 range now collapsed to 1970-01-01 instead of its real proleptic-Gregorian date. The same regression exists wherever the PR swapped a hand-rolled converter for the epoch fallback. Fix every such site to keep chrono as the only calendar logic via the 400-year Gregorian cycle (146_097 days): chrono converts the day within one cycle, whole cycles carry in the year (i128, saturated into i64). Verified the cycle result equals the old formula across a wide range and at the extremes. - apps/wordcraft/src/logging.rs `timestamp`: added `civil_from_days(u64)` (cycle), kept the pre-1970 epoch clamp (`duration_since`) and the milliseconds/time-of-day format exactly. Pins: UNIX_EPOCH + 10_000_000_000_000 s -> 318857-05-20T17:46:40.000Z (FredApps' example, was 1970-...); the checked_add u64::MAX/2 s fixture now asserts 292277026596-12-04T15:30:07.000Z instead of the epoch. - crates/docbin/src/metadata.rs `filetime_iso`: restored a cycle-based `civil(i64)` instead of `from_timestamp(..)?` (which dropped the field past chrono's range). A u64 FILETIME caps at ~1.83e12 unix s (year 60056), inside chrono's range, so no reachable value changes; pinned the largest FILETIME (u64::MAX) -> 60056-05-28T05:36:10Z to lock the extended-year path. - crates/formats/src/odt.rs `now_iso`: reuse the crate's existing cycle converter `sheet::civil` (made pub(crate)) rather than the epoch fallback. - crates/pdf/src/lib.rs `civil`: cycle converter; year returned unclamped and the caller's existing PDF DateTime 0..=9999 year clamp is kept exactly, so month/day/time of a far-future clock survive. Pins: civil(10_000_000_000_000)=(318857,5,20,17,46,40), civil(-10_000_000_000_000)=(-314918,8,13,6,13,20), civil(253_402_300_800)=(10000,1,1,0,0,0). engine `cmd/mod.rs::iso_from_unix_secs` already uses the cycle fallback (last round); unchanged. * Don't build a year-318857 SystemTime unchecked in the logging test Windows' SystemTime only reaches about year 30828, so UNIX_EPOCH + Duration::from_secs(10_000_000_000_000) panicked before timestamp() ran and failed the Windows test job. Build it with checked_add and assert where the platform can hold it. The year-10000 case fits everywhere and stays unconditional. * Name the 400-year range extension as the one exception to the time rule The rule said never to hand-roll calendar math, but the range fix (shifting by whole 400-year Gregorian cycles so dates past chrono's year range keep their old output) is cycle arithmetic, so the rule and the code disagreed. The rule now names that exception: the library still computes month, day and time, and only whole cycles are carried in the year. * Say the library computes month and day; time-of-day is plain arithmetic Narrows the range-extension wording: the date library computes the calendar fields, and splitting seconds into hours, minutes and seconds is ordinary arithmetic, which is what the code does.	2026-10-21 19:36:41 -04:00
3 people	575c5f98df	.doc: open password-protected Word 97–2003 files (RC4, RC4 CryptoAPI, XOR) (#1079) (#1087) ... docbin decrypts XOR-obfuscated, RC4 and RC4 CryptoAPI files after verifying the password against the stored verifier; the engine reuses the .docx password prompt errors. Written from [MS-DOC] and [MS-OFFCRYPTO]; synthetic tests only. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Brandon Thomas <echelon@gmail.com>	2026-10-14 17:21:08 -04:00
3 people	6801d89065	.doc import: inline pictures, floating pictures and basic shapes (#551) (#558) ... * .doc import: inline pictures, floating pictures and basic shapes (#551) Word 97-2003 files kept their text but dropped every picture and drawing. Pictures and drawings are now read from the OfficeArt records ([MS-DOC], [MS-ODRAW]): - Inline pictures: the PICF's OfficeArtInlineSpContainer (MM_SHAPE / MM_SHAPEFILE) and its FBSE/BLIP, with crop and alt text from the shape properties; the older raw-image path stays as a fallback. - Floating pictures and shapes: PlcSpaMom/PlcSpaHdr anchors joined with the shapes of the OfficeArtContent drawings (fcDggInfo), placed by the SPA bounds, bx/by or posrelh/posrelv/posh/posv, wrap type and fBelowText; rotation and flips; pictures from the BLIP store. - Shapes: rectangle, rounded rectangle, ellipse, diamond, triangles, star, block arrows, heart, lines/connectors, with fill/line colour and width; groups flattened into one group space; text boxes linked to their PlcftxbxTxt story parts by spid. - BLIPs: PNG, JPEG, TIFF as-is, DIB converted to PNG, EMF/WMF inflated (the metafile renderer draws them), PICT kept as a sized placeholder. - Hostile input: record lengths clamped to their parent, nesting depth and record counts capped, decompression size-limited, picture bytes capped per document as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Review fixes for #558: miniz_oxide as a workspace dependency Restores the change @echelon pushed in 04b59287, which a rebase dropped: miniz_oxide is declared once in the root [workspace.dependencies] and docbin uses it with workspace = true. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Brandon Thomas <echelon@gmail.com>	2026-10-12 03:42:18 -04:00
3 people	663904a2e2	Import Word 97-2003 .doc files (#28) (#145) ... * D0+D1: open Word 97-2003 .doc — container, FIB, piece table text (#28) New wordcraft-docbin crate (L2) reads the OLE2 container via the cfb crate (MIT) and parses the FIB and piece table from the [MS-DOC] specification: text pieces decode as UTF-16LE or Windows-1252 (table moved to wordcraft-doc::encoding), subdocument CP ranges, and Word's special characters map to the model's (tabs, breaks, hyphens). .doc/.dot registered in OPEN_EXTS and the I/O dispatch. Encrypted and Word 6/95 files give clear errors; every length is checked and proptest truncation/flips never panic. * D2: .doc formatting — stylesheet, fonts, CHPX/PAPX FKPs, sprms (#28) Sprm decoder with operand-size rules (incl. sprmTDefTable/sprmPChgTabs exceptions and the Prm0 isprm table), bin tables + ChpxFkp/PapxFkp page lookup, STSH styles (StdfBase layout validated against real Word files: bchUpe == cbStd, sti 0 = Normal) and the SttbfFfn font table in both extended and self-delimiting flavours. The CP walk now splits runs on formatting changes and resolves paragraph styles: bold/italic/strike/ underline/caps/hidden, colours (Ico palette + COLORREF), half-point sizes, super/subscript, fonts, highlights; alignment, indents, spacing, keep options, outline levels. Unknown sprms are skipped leniently; an env-gated corpus test opens real files (styles + text asserted). * D3: .doc structure — sections, headers/footers, tables, lists (#28) - Sections: PlcfSed/Sepx with 16-bit measurement sprms (sprmSXaPage & co), explicit sprmSBOrientation with a width/height fallback; even/odd/first header and footer stories from Plcfhdd become document parts. - Tables: sprmTDefTable grids (operand length fixed to cb+1 per the spec), horizontal merges (sprmTMerge), vertical merges (sprmTVertMerge), cell shading (sprmTSetShd / sprmTDefTableShd80), cell widths, valign, row height/header/cant-split, and table borders (sprmTTableBorders80 Brc80s). - Lists: PlfLst (LSTF + appended LVLs) and PlfLfo (LFO + LFOLVL overrides) resolved into the numbering model; sprmPIlvl/sprmPIlfo mark paragraphs. - Combined fixture (intro + table + list + 2 sections + header) round-trips through Document JSON; hostile proptests now mutate the structured fixture. * D4: .doc notes, fields, bookmarks and inline pictures (#28) - Footnotes/endnotes: PlcffndRef/PlcfEndRef reference CPs paired with the PlcffndTxt/PlcfEndTxt story ranges; each story walks into a Footnote or Endnote part and the 0x02 anchor becomes a NoteRef object. - Fields: 0x13/0x14/0x15 state machine in the walker — instruction text is hidden, the cached result kept; HYPERLINK fields keep their result as linked runs, everything else becomes one Field object. - Bookmarks: SttbfBkmk names + Plcfbkf/Plcfbkl CPs become BookmarkStart/ BookmarkEnd objects at their anchors. - Pictures: sprmCPicLocation points into the Data stream; the PICF header gives the display size (dxaGoal/dyaGoal scaled by mx/my) and the payload is format-sniffed (PNG/JPEG/GIF/BMP; metafiles are dropped with a warn). - Hostile proptests now truncate/flip a fixture with all of the above. * D5: .doc import hardening, caps and roadmap (#28) - Cap inline pictures at 5 000 per document. - Engine test: .doc/.dot bytes dispatch through the Word 97-2003 reader (garbage must error, never panic or open empty). - ROADMAP: Word 97-2003 import row (60% — text, formatting, styles, sections, headers/footers, tables, lists, notes, fields, bookmarks, raster pictures). Real-corpus validation stays opt-in: WORDCRAFT_DOCBIN_FIXTURES=<dir> cargo test -p wordcraft-docbin corpus_files_open. * Corpus fixes: compressed-piece FC space, PAPX query point, none-level labels (#28) Validated against two real files (kept locally under plan/word/fixtures, gitignored): a 1 MB Lorem-ipsum sample and a 2003 styles/lists/table sample. - FKP bin tables and pages address the WordDocument stream in true byte offsets; a compressed piece stores its base fc doubled (byte = fc/2), so fc_of_cp now unwinds the doubling. Before, every PAPX/CHPX lookup in compressed (cp1252) files landed on the wrong entry — tables collapsed to one row, lists and heading styles were lost. - The paragraph-properties query now uses the paragraph mark's own FC (inside the paragraph) instead of the FC after it, which picked up the next paragraph's FKP entry. - Fixture papx entries re-keyed to paragraph-start CPs, the real rgfc convention. - Counters::next_label returns nothing for NumFormat::None levels (Word's numFmt none shows no label; also fixes a latent docx-reader case). - Symbol/Wingdings bullet chars map to Unicode glyphs (• ▪ ○ ●). With this, both corpus files render headings, styles, nested and numbered lists, the tables and the PAGE field correctly; metafile-wrapped images and OLE objects stay dropped by policy (logged). * docbin: bound the text walk by real text; dedupe and cap pictures (#28) Hostile CP numbers no longer drive the work: - header/footnote/endnote story ranges from the Table stream are clamped to their subdocument (ccpHdd/ccpFtn/ccpEdn) with checked base additions, and every walk stops at the last piece's CP; a story spanning [0, 4e9) used to push billions of U+FFFD; - characters are found by binary search over the pieces (PieceTable::char_at), and an unreadable stretch (gap, truncated or out-of-stream piece) is skipped in one step rather than one U+FFFD per CP; the old per-character scan was O(chars x pieces); - the piece CPs are made non-decreasing at parse, so inverted pieces are empty and binary search stays sound; surrogate pairs now decode; - note references are looked up by binary search; the style's CharProps is borrowed, not cloned, per character; - pictures are interned by fc_pic (5000 anchors on one picture share one media entry and one copy) and the bytes copied per document are capped at 256 MB. Each fix has a crafted-bytes regression test; against the previous code the story, piece-claim, last-piece and many-pieces tests hang (killed at 60 s) and the shared picture test copies the picture once per anchor (80 GB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docbin: checked offset arithmetic in lists, sections and table rows (#28) - list.rs: PlfLst/PlfLfo/LVL offsets (at + 2 + cLst*28, at + 4 + lfoMac*16, off += len, ...) saturate, which matters on 32-bit targets (wasm32) where fc + n can overflow usize; the LFO override path no longer indexes abstracts directly. - sections.rs: the Sepx range (fc_sepx + 2 + cb) is checked; Plcfhdd is read only as far as the stories a capped section count can use (it used to reserve lcb/4 entries). - table.rs: sprmTDyaRowHeight = i16::MIN negated in i16 (overflow panic in debug builds); now -(v as f32). Regression tests: the row height test failed before (overflow panic with debug assertions, a 0 pt height without); the list test pins the saturating offsets (it only overflowed on 32-bit targets). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Format the merged OPEN_EXTS list Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Brandon Thomas <echelon@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-10 08:36:38 -04:00