mirror of
https://github.com/storytold/wordcraft.git
synced 2026-10-25 10:57:14 -06:00
8a2a8bb60dfd46ddceee26a4aa91aeb5cbe14532
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
e9a38a88ca |
Pictures: draw WMF and EMF pictures instead of a grey box (#293)
Windows metafiles (WMF, EMF), common for logos, crests and diagrams in older documents, couldn't be decoded and showed as grey boxes, on screen and in PDF. The new crate wordcraft-metafile (layer 1) reads them from the public [MS-WMF] and [MS-EMF] specifications into paths and bitmaps, and places them in a picture's box; render and pdf paint the result as vectors. It covers the drawing records Office pictures use (lines, polygons, curves, rectangles, ellipses, arcs, path brackets), pens and brushes, the mapping modes and world transforms, and bitmaps in DIB form with the usual raster operations. Input is treated as hostile: every read is bounded, and a byte budget caps what one picture may allocate. Not yet: text records (ExtTextOut) and EMF+ records are skipped, so a metafile whose content is mostly text shows only its lines and fills. docxide-pdf bench (365 documents, vs main + #135): the twelve documents with WMF/EMF pictures show them; the pictures are a small part of their pages, so the scores move little (Jaccard passes 208 -> 209, five documents up to 3 SSIM points better, none worse). Co-authored-by: Oliver Zieschang <4963183+naxvog@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
eb31860917 |
Progress docs: target-app parity, gaps, alpha gate, readiness by audience (#277)
Progress docs per craftrules standards/progress-docs.md: target-app parity, gaps, roadmap with alpha gate, readiness by audience (full / mainstream / essentials) with Opus 5.5 hours, and feature, format, layout, hardware, UI and localization parity docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
663904a2e2 |
Import Word 97-2003 .doc files (#28) (#145)
* D0+D1: open Word 97-2003 .doc — container, FIB, piece table text (#28) New wordcraft-docbin crate (L2) reads the OLE2 container via the cfb crate (MIT) and parses the FIB and piece table from the [MS-DOC] specification: text pieces decode as UTF-16LE or Windows-1252 (table moved to wordcraft-doc::encoding), subdocument CP ranges, and Word's special characters map to the model's (tabs, breaks, hyphens). .doc/.dot registered in OPEN_EXTS and the I/O dispatch. Encrypted and Word 6/95 files give clear errors; every length is checked and proptest truncation/flips never panic. * D2: .doc formatting — stylesheet, fonts, CHPX/PAPX FKPs, sprms (#28) Sprm decoder with operand-size rules (incl. sprmTDefTable/sprmPChgTabs exceptions and the Prm0 isprm table), bin tables + ChpxFkp/PapxFkp page lookup, STSH styles (StdfBase layout validated against real Word files: bchUpe == cbStd, sti 0 = Normal) and the SttbfFfn font table in both extended and self-delimiting flavours. The CP walk now splits runs on formatting changes and resolves paragraph styles: bold/italic/strike/ underline/caps/hidden, colours (Ico palette + COLORREF), half-point sizes, super/subscript, fonts, highlights; alignment, indents, spacing, keep options, outline levels. Unknown sprms are skipped leniently; an env-gated corpus test opens real files (styles + text asserted). * D3: .doc structure — sections, headers/footers, tables, lists (#28) - Sections: PlcfSed/Sepx with 16-bit measurement sprms (sprmSXaPage & co), explicit sprmSBOrientation with a width/height fallback; even/odd/first header and footer stories from Plcfhdd become document parts. - Tables: sprmTDefTable grids (operand length fixed to cb+1 per the spec), horizontal merges (sprmTMerge), vertical merges (sprmTVertMerge), cell shading (sprmTSetShd / sprmTDefTableShd80), cell widths, valign, row height/header/cant-split, and table borders (sprmTTableBorders80 Brc80s). - Lists: PlfLst (LSTF + appended LVLs) and PlfLfo (LFO + LFOLVL overrides) resolved into the numbering model; sprmPIlvl/sprmPIlfo mark paragraphs. - Combined fixture (intro + table + list + 2 sections + header) round-trips through Document JSON; hostile proptests now mutate the structured fixture. * D4: .doc notes, fields, bookmarks and inline pictures (#28) - Footnotes/endnotes: PlcffndRef/PlcfEndRef reference CPs paired with the PlcffndTxt/PlcfEndTxt story ranges; each story walks into a Footnote or Endnote part and the 0x02 anchor becomes a NoteRef object. - Fields: 0x13/0x14/0x15 state machine in the walker — instruction text is hidden, the cached result kept; HYPERLINK fields keep their result as linked runs, everything else becomes one Field object. - Bookmarks: SttbfBkmk names + Plcfbkf/Plcfbkl CPs become BookmarkStart/ BookmarkEnd objects at their anchors. - Pictures: sprmCPicLocation points into the Data stream; the PICF header gives the display size (dxaGoal/dyaGoal scaled by mx/my) and the payload is format-sniffed (PNG/JPEG/GIF/BMP; metafiles are dropped with a warn). - Hostile proptests now truncate/flip a fixture with all of the above. * D5: .doc import hardening, caps and roadmap (#28) - Cap inline pictures at 5 000 per document. - Engine test: .doc/.dot bytes dispatch through the Word 97-2003 reader (garbage must error, never panic or open empty). - ROADMAP: Word 97-2003 import row (60% — text, formatting, styles, sections, headers/footers, tables, lists, notes, fields, bookmarks, raster pictures). Real-corpus validation stays opt-in: WORDCRAFT_DOCBIN_FIXTURES=<dir> cargo test -p wordcraft-docbin corpus_files_open. * Corpus fixes: compressed-piece FC space, PAPX query point, none-level labels (#28) Validated against two real files (kept locally under plan/word/fixtures, gitignored): a 1 MB Lorem-ipsum sample and a 2003 styles/lists/table sample. - FKP bin tables and pages address the WordDocument stream in true byte offsets; a compressed piece stores its base fc doubled (byte = fc/2), so fc_of_cp now unwinds the doubling. Before, every PAPX/CHPX lookup in compressed (cp1252) files landed on the wrong entry — tables collapsed to one row, lists and heading styles were lost. - The paragraph-properties query now uses the paragraph mark's own FC (inside the paragraph) instead of the FC after it, which picked up the next paragraph's FKP entry. - Fixture papx entries re-keyed to paragraph-start CPs, the real rgfc convention. - Counters::next_label returns nothing for NumFormat::None levels (Word's numFmt none shows no label; also fixes a latent docx-reader case). - Symbol/Wingdings bullet chars map to Unicode glyphs (• ▪ ○ ●). With this, both corpus files render headings, styles, nested and numbered lists, the tables and the PAGE field correctly; metafile-wrapped images and OLE objects stay dropped by policy (logged). * docbin: bound the text walk by real text; dedupe and cap pictures (#28) Hostile CP numbers no longer drive the work: - header/footnote/endnote story ranges from the Table stream are clamped to their subdocument (ccpHdd/ccpFtn/ccpEdn) with checked base additions, and every walk stops at the last piece's CP; a story spanning [0, 4e9) used to push billions of U+FFFD; - characters are found by binary search over the pieces (PieceTable::char_at), and an unreadable stretch (gap, truncated or out-of-stream piece) is skipped in one step rather than one U+FFFD per CP; the old per-character scan was O(chars x pieces); - the piece CPs are made non-decreasing at parse, so inverted pieces are empty and binary search stays sound; surrogate pairs now decode; - note references are looked up by binary search; the style's CharProps is borrowed, not cloned, per character; - pictures are interned by fc_pic (5000 anchors on one picture share one media entry and one copy) and the bytes copied per document are capped at 256 MB. Each fix has a crafted-bytes regression test; against the previous code the story, piece-claim, last-piece and many-pieces tests hang (killed at 60 s) and the shared picture test copies the picture once per anchor (80 GB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docbin: checked offset arithmetic in lists, sections and table rows (#28) - list.rs: PlfLst/PlfLfo/LVL offsets (at + 2 + cLst*28, at + 4 + lfoMac*16, off += len, ...) saturate, which matters on 32-bit targets (wasm32) where fc + n can overflow usize; the LFO override path no longer indexes abstracts directly. - sections.rs: the Sepx range (fc_sepx + 2 + cb) is checked; Plcfhdd is read only as far as the stories a capped section count can use (it used to reserve lcb/4 entries). - table.rs: sprmTDyaRowHeight = i16::MIN negated in i16 (overflow panic in debug builds); now -(v as f32). Regression tests: the row height test failed before (overflow panic with debug assertions, a 0 pt height without); the list test pins the saturating offsets (it only overflowed on 32-bit targets). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Format the merged OPEN_EXTS list Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Brandon Thomas <echelon@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
43c0be5735 |
Zotero integration: cite while you write, with a Zotero tab (#189)
* Zotero 1: range fields and custom document properties round-trip Citation managers' ADDIN fields (Zotero, Mendeley, EndNote) are now kept as range fields: FieldStart/FieldEnd markers with the result as ordinary, formatted content between them, possibly spanning paragraphs (a Zotero bibliography). Before, the result was flattened to plain text and a multi-paragraph field lost its end. - doc: InlineObject::FieldStart/FieldEnd, Document::field_ranges, balance_field_ranges; CustomProp and custom_props with get/set/remove. - docx: read ADDIN fields as ranges (field codes up to 2 MiB), write them back as begin/instr/separate…end; orphan markers are dropped on save so files always hold whole fields. docProps/custom.xml is parsed and regenerated (simple types as text, vectors/blobs verbatim) instead of passed through, so Zotero's ZOTERO_PREF_n settings are editable. - engine: file.properties reads and sets custom properties. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero 2: wire-protocol client and document bridge New crate wordcraft-zotero (L5) speaks Zotero's published word-processor protocol (port 23116; 32-bit transaction id + big-endian length frames, JSON payloads, ERR: replies): - client: sends an integration command (addEditCitation, addEditBibliography, addNote, refresh, removeCodes, setDocPrefs) and answers Zotero's calls until Document_complete or the connection closes (cancelled dialog). Answers go through a callback so the app can apply them on its UI thread; run_on_session answers directly. - Bridge: implements Zotero's Document_*/Field_* calls on a Session. Fields are stored as Word's plugin stores them (ADDIN ZOTERO_ range fields; prefs in ZOTERO_PREF_n, with Zotero's ReferenceMark field type kept as Word's Field in the file), so documents stay Word-compatible. Rich text (RTF or HTML) keeps citation formatting; bibliographies get a Bibliography style from setBibliographyStyle; note styles create footnotes/endnotes, and deleting a lone note citation removes its note; placeholder links convert to fields. One transaction = one undo step. Field ids stay stable through inserts and deletes. - Tests: a fake Zotero over a real socket, every call, docx round trip, hostile arguments for every method, random frames (proptest). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero 3: Zotero in the app and the CLI, tested with the real Zotero - ui.zotero.<command> (addEditCitation, addEditBibliography, addNote, refresh, removeCodes, setDocPrefs), ui.zotero.status and ui.zotero.answer. The socket runs on a background thread; each of Zotero's calls is answered on the UI thread against the session, and Zotero's questions become a dialog the session waits on. Web builds report that Zotero needs the desktop app. - wordcraft-cli zotero <command> <file> [--save] [--trace] [--port]. - Bridge: a selection Zotero makes to show a citation is put back when the session ends. - ui_shot: {"wait": ms} keeps frames running for background work. - docs/zotero.md. Checked against Zotero 10.0.6: refresh of a citation and bibliography (APA), its "modified citation" question answered in the app's dialog, RTF results with italics, Bibliography style, prefs round trip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero 4: switching between in-text and note styles moves citations Document_convert and Field_convert now move a citation between the text, footnotes and endnotes, as Zotero asks when the document's style changes between an in-text and a note style: into a new note at the same place, or back into the text where the note mark was (removing a note that held only the citation; keeping one with other text). Field ids survive the move, formatting is kept, and each Zotero session stays one undo step. Checked against Zotero 10.0.6: unlink citations (removeCodes, with its confirmation question) and a refresh after switching the document to Chicago notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero 5: Zotero tab in the ribbon A Zotero tab (before Help) with Add/Edit Citation, Add Note, Add/Edit Bibliography, Refresh, Document Preferences and Unlink Citations. The buttons are disabled while a Zotero command is running and in the web build. Three new icons drawn in code (document preferences, unlink, add note); the others reuse the References tab's. Labels and status messages translated (zh-hans, zh-hant, ja). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero: Add/Edit Bibliography inserted nothing Zotero reads a newly inserted bibliography field back before filling it, and unlinks a field whose text is empty, taking it for a bibliography the user deleted. Our new fields were empty, so Add/Edit Bibliography ended with Field_removeCode and nothing in the document. New fields now hold "{Citation}" until Zotero writes the result. Found by tracing the session with Zotero 10.0.6; regression test added. wordcraft-cli zotero takes --cmd (run before the Zotero command, e.g. caret.docEnd) to reproduce sessions from a given caret position. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero: log each command's outcome and Zotero's calls to the app log Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cargo.lock: wordcraft-zotero at the workspace version (0.4.0) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Brandon Thomas <echelon@gmail.com> |
||
|
|
bded38dd23 |
Control port: require a per-window key (split from #54) (#124)
* MCP bridge: send the control key, to loopback addresses only
`wordcraft-cli mcp --connect` now sends the app's control key with every
request. It takes the key from WORDCRAFT_CONTROL_KEY, or else from the
key file the app writes for that port, and reads it again on every new
connection, also after an `unauthorized` reply (the app restarted with
a new key).
The key goes only to loopback addresses: 127.0.0.0/8, ::1 and
`localhost`, and only when the connection really leads to a loopback
address. For any other host the bridge sends no key, prints why at
start, and turns the app's `unauthorized` into an error that says the
key was not sent and how to reach the app (through 127.0.0.1, for
example an SSH tunnel). Other host names are never resolved for this.
The new `wordcraft_mcp::control_key` module holds the rule the app and
the bridge share: the settings folder (the one with ui.json), the
Flatpak instance id and `<settings>/control-key.<instance>`
(`local-<port>` outside Flatpak). No new dependency; wasm-safe.
* Control port: every request needs the window's key
Any local program, and possibly a web page, could drive the app through
its loopback control port. Each window started with --control now has
a key:
- once the port is bound, the app writes a random key (32 bytes from
the OS, as hex) to <settings>/control-key.<instance>, mode 0600 on
Unix, replacing a stale file; it removes the file on a clean exit. A
failed bind writes nothing. <instance> is the Flatpak instance id, or
local-<port> outside Flatpak (wordcraft_mcp::control_key, the rule
the MCP bridge reads it with). The settings folder is the one that
holds ui.json; prefs_path now uses the same helper.
- every request needs the key: top-level "key", or
{"method":"auth","key":...} once per connection. Only auth reads
params.key; elsewhere it stays an ordinary parameter (ui.key). The key
is checked on every request, in time that does not depend on where a
wrong key differs. A missing or wrong key gets
{"ok":false,"error":"unauthorized"} and the connection is closed.
- a line over 1 MiB gets "line too long" and the connection is closed
(the rest of the line is drained first, so the client reads the
error).
- a connection that sends no authorised request within 30 s of being
accepted is closed; blank lines and bad JSON do not extend that
limit. After that it may stay idle.
* Docs: control-port keys
docs/control-protocol.md gets a Keys section: where the key file is
(per platform and per instance), top-level "key" or auth once per
connection, `unauthorized` and the closed connection, the 1 MiB line
limit, the 30 s limit for the first authorised request, and how the MCP
bridge finds the key and sends it only to loopback addresses (an SSH
tunnel for an app on another machine). docs/mcp.md points there, and
the agent guides (AGENTS.md, CLAUDE.md) show the key in their control
channel example.
* Control key: the key-file helpers in their own small crate
wordcraft-control-key (standard library only, Standalone in the layering
table) holds where the key file lives and how to read it. The app and the
MCP bridge both use it, so the app no longer depends on wordcraft-mcp.
* Cargo.lock: wordcraft-control-key follows the workspace version (0.4.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: lpgn <2556951+lpgn@users.noreply.github.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Brandon ThomasandClaude Opus 5.5
|
d95e09ba25 |
Clippy/fmt clean workspace, ROADMAP with honest parity estimate; xtask ci green
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
d1a0ed39c0 |
M7.1: proofing — spelling with suggestions, grammar checks, squiggles, F7, right-click menu, thesaurus; web app; hyphenation dictionary
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
435194f7c2 |
M0.9–M0.12: CLI, MCP server, xtask (layers/assets/parity/ci), icon placeholder, AGENTS/ATTRIBUTION/NOTICE, protocol docs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
d1e80df4da |
M0.1–M0.6: workspace, geom, document model, fonts, layout, render
Clean-room foundations for WordCraft: Arc-shared document model with paragraph/run editing, styles, lists, sections and tables; Word-style font substitution; first-fit line breaking with tabs, lists, alignment, pagination, tables, headers/footers and hit testing; vello_cpu page rasteriser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |