mirror of
https://github.com/storytold/wordcraft.git
synced 2026-10-25 10:57:14 -06:00
2f8f304fc9e10540887c3e49e2d75d8ee7baac2a
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
663904a2e2 |
Import Word 97-2003 .doc files (#28) (#145)
* D0+D1: open Word 97-2003 .doc — container, FIB, piece table text (#28) New wordcraft-docbin crate (L2) reads the OLE2 container via the cfb crate (MIT) and parses the FIB and piece table from the [MS-DOC] specification: text pieces decode as UTF-16LE or Windows-1252 (table moved to wordcraft-doc::encoding), subdocument CP ranges, and Word's special characters map to the model's (tabs, breaks, hyphens). .doc/.dot registered in OPEN_EXTS and the I/O dispatch. Encrypted and Word 6/95 files give clear errors; every length is checked and proptest truncation/flips never panic. * D2: .doc formatting — stylesheet, fonts, CHPX/PAPX FKPs, sprms (#28) Sprm decoder with operand-size rules (incl. sprmTDefTable/sprmPChgTabs exceptions and the Prm0 isprm table), bin tables + ChpxFkp/PapxFkp page lookup, STSH styles (StdfBase layout validated against real Word files: bchUpe == cbStd, sti 0 = Normal) and the SttbfFfn font table in both extended and self-delimiting flavours. The CP walk now splits runs on formatting changes and resolves paragraph styles: bold/italic/strike/ underline/caps/hidden, colours (Ico palette + COLORREF), half-point sizes, super/subscript, fonts, highlights; alignment, indents, spacing, keep options, outline levels. Unknown sprms are skipped leniently; an env-gated corpus test opens real files (styles + text asserted). * D3: .doc structure — sections, headers/footers, tables, lists (#28) - Sections: PlcfSed/Sepx with 16-bit measurement sprms (sprmSXaPage & co), explicit sprmSBOrientation with a width/height fallback; even/odd/first header and footer stories from Plcfhdd become document parts. - Tables: sprmTDefTable grids (operand length fixed to cb+1 per the spec), horizontal merges (sprmTMerge), vertical merges (sprmTVertMerge), cell shading (sprmTSetShd / sprmTDefTableShd80), cell widths, valign, row height/header/cant-split, and table borders (sprmTTableBorders80 Brc80s). - Lists: PlfLst (LSTF + appended LVLs) and PlfLfo (LFO + LFOLVL overrides) resolved into the numbering model; sprmPIlvl/sprmPIlfo mark paragraphs. - Combined fixture (intro + table + list + 2 sections + header) round-trips through Document JSON; hostile proptests now mutate the structured fixture. * D4: .doc notes, fields, bookmarks and inline pictures (#28) - Footnotes/endnotes: PlcffndRef/PlcfEndRef reference CPs paired with the PlcffndTxt/PlcfEndTxt story ranges; each story walks into a Footnote or Endnote part and the 0x02 anchor becomes a NoteRef object. - Fields: 0x13/0x14/0x15 state machine in the walker — instruction text is hidden, the cached result kept; HYPERLINK fields keep their result as linked runs, everything else becomes one Field object. - Bookmarks: SttbfBkmk names + Plcfbkf/Plcfbkl CPs become BookmarkStart/ BookmarkEnd objects at their anchors. - Pictures: sprmCPicLocation points into the Data stream; the PICF header gives the display size (dxaGoal/dyaGoal scaled by mx/my) and the payload is format-sniffed (PNG/JPEG/GIF/BMP; metafiles are dropped with a warn). - Hostile proptests now truncate/flip a fixture with all of the above. * D5: .doc import hardening, caps and roadmap (#28) - Cap inline pictures at 5 000 per document. - Engine test: .doc/.dot bytes dispatch through the Word 97-2003 reader (garbage must error, never panic or open empty). - ROADMAP: Word 97-2003 import row (60% — text, formatting, styles, sections, headers/footers, tables, lists, notes, fields, bookmarks, raster pictures). Real-corpus validation stays opt-in: WORDCRAFT_DOCBIN_FIXTURES=<dir> cargo test -p wordcraft-docbin corpus_files_open. * Corpus fixes: compressed-piece FC space, PAPX query point, none-level labels (#28) Validated against two real files (kept locally under plan/word/fixtures, gitignored): a 1 MB Lorem-ipsum sample and a 2003 styles/lists/table sample. - FKP bin tables and pages address the WordDocument stream in true byte offsets; a compressed piece stores its base fc doubled (byte = fc/2), so fc_of_cp now unwinds the doubling. Before, every PAPX/CHPX lookup in compressed (cp1252) files landed on the wrong entry — tables collapsed to one row, lists and heading styles were lost. - The paragraph-properties query now uses the paragraph mark's own FC (inside the paragraph) instead of the FC after it, which picked up the next paragraph's FKP entry. - Fixture papx entries re-keyed to paragraph-start CPs, the real rgfc convention. - Counters::next_label returns nothing for NumFormat::None levels (Word's numFmt none shows no label; also fixes a latent docx-reader case). - Symbol/Wingdings bullet chars map to Unicode glyphs (• ▪ ○ ●). With this, both corpus files render headings, styles, nested and numbered lists, the tables and the PAGE field correctly; metafile-wrapped images and OLE objects stay dropped by policy (logged). * docbin: bound the text walk by real text; dedupe and cap pictures (#28) Hostile CP numbers no longer drive the work: - header/footnote/endnote story ranges from the Table stream are clamped to their subdocument (ccpHdd/ccpFtn/ccpEdn) with checked base additions, and every walk stops at the last piece's CP; a story spanning [0, 4e9) used to push billions of U+FFFD; - characters are found by binary search over the pieces (PieceTable::char_at), and an unreadable stretch (gap, truncated or out-of-stream piece) is skipped in one step rather than one U+FFFD per CP; the old per-character scan was O(chars x pieces); - the piece CPs are made non-decreasing at parse, so inverted pieces are empty and binary search stays sound; surrogate pairs now decode; - note references are looked up by binary search; the style's CharProps is borrowed, not cloned, per character; - pictures are interned by fc_pic (5000 anchors on one picture share one media entry and one copy) and the bytes copied per document are capped at 256 MB. Each fix has a crafted-bytes regression test; against the previous code the story, piece-claim, last-piece and many-pieces tests hang (killed at 60 s) and the shared picture test copies the picture once per anchor (80 GB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docbin: checked offset arithmetic in lists, sections and table rows (#28) - list.rs: PlfLst/PlfLfo/LVL offsets (at + 2 + cLst*28, at + 4 + lfoMac*16, off += len, ...) saturate, which matters on 32-bit targets (wasm32) where fc + n can overflow usize; the LFO override path no longer indexes abstracts directly. - sections.rs: the Sepx range (fc_sepx + 2 + cb) is checked; Plcfhdd is read only as far as the stories a capped section count can use (it used to reserve lcb/4 entries). - table.rs: sprmTDyaRowHeight = i16::MIN negated in i16 (overflow panic in debug builds); now -(v as f32). Regression tests: the row height test failed before (overflow panic with debug assertions, a 0 pt height without); the list test pins the saturating offsets (it only overflowed on 32-bit targets). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Format the merged OPEN_EXTS list Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Brandon Thomas <echelon@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
43c0be5735 |
Zotero integration: cite while you write, with a Zotero tab (#189)
* Zotero 1: range fields and custom document properties round-trip Citation managers' ADDIN fields (Zotero, Mendeley, EndNote) are now kept as range fields: FieldStart/FieldEnd markers with the result as ordinary, formatted content between them, possibly spanning paragraphs (a Zotero bibliography). Before, the result was flattened to plain text and a multi-paragraph field lost its end. - doc: InlineObject::FieldStart/FieldEnd, Document::field_ranges, balance_field_ranges; CustomProp and custom_props with get/set/remove. - docx: read ADDIN fields as ranges (field codes up to 2 MiB), write them back as begin/instr/separate…end; orphan markers are dropped on save so files always hold whole fields. docProps/custom.xml is parsed and regenerated (simple types as text, vectors/blobs verbatim) instead of passed through, so Zotero's ZOTERO_PREF_n settings are editable. - engine: file.properties reads and sets custom properties. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero 2: wire-protocol client and document bridge New crate wordcraft-zotero (L5) speaks Zotero's published word-processor protocol (port 23116; 32-bit transaction id + big-endian length frames, JSON payloads, ERR: replies): - client: sends an integration command (addEditCitation, addEditBibliography, addNote, refresh, removeCodes, setDocPrefs) and answers Zotero's calls until Document_complete or the connection closes (cancelled dialog). Answers go through a callback so the app can apply them on its UI thread; run_on_session answers directly. - Bridge: implements Zotero's Document_*/Field_* calls on a Session. Fields are stored as Word's plugin stores them (ADDIN ZOTERO_ range fields; prefs in ZOTERO_PREF_n, with Zotero's ReferenceMark field type kept as Word's Field in the file), so documents stay Word-compatible. Rich text (RTF or HTML) keeps citation formatting; bibliographies get a Bibliography style from setBibliographyStyle; note styles create footnotes/endnotes, and deleting a lone note citation removes its note; placeholder links convert to fields. One transaction = one undo step. Field ids stay stable through inserts and deletes. - Tests: a fake Zotero over a real socket, every call, docx round trip, hostile arguments for every method, random frames (proptest). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero 3: Zotero in the app and the CLI, tested with the real Zotero - ui.zotero.<command> (addEditCitation, addEditBibliography, addNote, refresh, removeCodes, setDocPrefs), ui.zotero.status and ui.zotero.answer. The socket runs on a background thread; each of Zotero's calls is answered on the UI thread against the session, and Zotero's questions become a dialog the session waits on. Web builds report that Zotero needs the desktop app. - wordcraft-cli zotero <command> <file> [--save] [--trace] [--port]. - Bridge: a selection Zotero makes to show a citation is put back when the session ends. - ui_shot: {"wait": ms} keeps frames running for background work. - docs/zotero.md. Checked against Zotero 10.0.6: refresh of a citation and bibliography (APA), its "modified citation" question answered in the app's dialog, RTF results with italics, Bibliography style, prefs round trip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero 4: switching between in-text and note styles moves citations Document_convert and Field_convert now move a citation between the text, footnotes and endnotes, as Zotero asks when the document's style changes between an in-text and a note style: into a new note at the same place, or back into the text where the note mark was (removing a note that held only the citation; keeping one with other text). Field ids survive the move, formatting is kept, and each Zotero session stays one undo step. Checked against Zotero 10.0.6: unlink citations (removeCodes, with its confirmation question) and a refresh after switching the document to Chicago notes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero 5: Zotero tab in the ribbon A Zotero tab (before Help) with Add/Edit Citation, Add Note, Add/Edit Bibliography, Refresh, Document Preferences and Unlink Citations. The buttons are disabled while a Zotero command is running and in the web build. Three new icons drawn in code (document preferences, unlink, add note); the others reuse the References tab's. Labels and status messages translated (zh-hans, zh-hant, ja). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero: Add/Edit Bibliography inserted nothing Zotero reads a newly inserted bibliography field back before filling it, and unlinks a field whose text is empty, taking it for a bibliography the user deleted. Our new fields were empty, so Add/Edit Bibliography ended with Field_removeCode and nothing in the document. New fields now hold "{Citation}" until Zotero writes the result. Found by tracing the session with Zotero 10.0.6; regression test added. wordcraft-cli zotero takes --cmd (run before the Zotero command, e.g. caret.docEnd) to reproduce sessions from a given caret position. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Zotero: log each command's outcome and Zotero's calls to the app log Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Cargo.lock: wordcraft-zotero at the workspace version (0.4.0) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Brandon Thomas <echelon@gmail.com> |
||
|
|
bded38dd23 |
Control port: require a per-window key (split from #54) (#124)
* MCP bridge: send the control key, to loopback addresses only
`wordcraft-cli mcp --connect` now sends the app's control key with every
request. It takes the key from WORDCRAFT_CONTROL_KEY, or else from the
key file the app writes for that port, and reads it again on every new
connection, also after an `unauthorized` reply (the app restarted with
a new key).
The key goes only to loopback addresses: 127.0.0.0/8, ::1 and
`localhost`, and only when the connection really leads to a loopback
address. For any other host the bridge sends no key, prints why at
start, and turns the app's `unauthorized` into an error that says the
key was not sent and how to reach the app (through 127.0.0.1, for
example an SSH tunnel). Other host names are never resolved for this.
The new `wordcraft_mcp::control_key` module holds the rule the app and
the bridge share: the settings folder (the one with ui.json), the
Flatpak instance id and `<settings>/control-key.<instance>`
(`local-<port>` outside Flatpak). No new dependency; wasm-safe.
* Control port: every request needs the window's key
Any local program, and possibly a web page, could drive the app through
its loopback control port. Each window started with --control now has
a key:
- once the port is bound, the app writes a random key (32 bytes from
the OS, as hex) to <settings>/control-key.<instance>, mode 0600 on
Unix, replacing a stale file; it removes the file on a clean exit. A
failed bind writes nothing. <instance> is the Flatpak instance id, or
local-<port> outside Flatpak (wordcraft_mcp::control_key, the rule
the MCP bridge reads it with). The settings folder is the one that
holds ui.json; prefs_path now uses the same helper.
- every request needs the key: top-level "key", or
{"method":"auth","key":...} once per connection. Only auth reads
params.key; elsewhere it stays an ordinary parameter (ui.key). The key
is checked on every request, in time that does not depend on where a
wrong key differs. A missing or wrong key gets
{"ok":false,"error":"unauthorized"} and the connection is closed.
- a line over 1 MiB gets "line too long" and the connection is closed
(the rest of the line is drained first, so the client reads the
error).
- a connection that sends no authorised request within 30 s of being
accepted is closed; blank lines and bad JSON do not extend that
limit. After that it may stay idle.
* Docs: control-port keys
docs/control-protocol.md gets a Keys section: where the key file is
(per platform and per instance), top-level "key" or auth once per
connection, `unauthorized` and the closed connection, the 1 MiB line
limit, the 30 s limit for the first authorised request, and how the MCP
bridge finds the key and sends it only to loopback addresses (an SSH
tunnel for an app on another machine). docs/mcp.md points there, and
the agent guides (AGENTS.md, CLAUDE.md) show the key in their control
channel example.
* Control key: the key-file helpers in their own small crate
wordcraft-control-key (standard library only, Standalone in the layering
table) holds where the key file lives and how to read it. The app and the
MCP bridge both use it, so the app no longer depends on wordcraft-mcp.
* Cargo.lock: wordcraft-control-key follows the workspace version (0.4.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: lpgn <2556951+lpgn@users.noreply.github.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
c0ccda288c |
Find/Replace skips text deleted under Track Changes (#89); a failed command keeps undo history (#153)
* Find/Replace: skip text deleted under Track Changes With Track Changes on, Replace marks the match deleted and leaves it in the paragraph text, then inserts the replacement in front of it. The search helper matched the raw paragraph text, so the next step found that same deleted "cat" right after the caret and replaced it again: four Replaces on "one cat two cat" gave "one dogdogdogdog two cat" and `remaining` never dropped. `search` (used by Find, Find Next/Previous, Replace, its `remaining` count and Replace All) now skips any match that overlaps a run marked deleted, so deleted text is never found, as in Word. A match that would run across deleted text is skipped too: that string is not in the document as it now reads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Undo: a failed command keeps the redo stack and the oldest undo step `run` checkpoints a mutating command before running it, and the checkpoint clears the redo stack (and, once the history is at its limit, drops the oldest step). When the command then failed, the error path put back the document, selection and typing group and truncated the history, but the cleared redo stack stayed cleared. Insert, Undo, then any command that fails on its parameters (`text.insert {}`, `para.align {value:"bogus"}`) made the undone work unrecoverable. At the limit, truncating to the old length also kept the failed command's empty step and lost the oldest real one. The checkpoint now hands back what it removed, and the error path restores both. join_next is deliberately left consumed: the drag that set it re-arms it every frame, and restoring it would merge the next unrelated command into the drag's undo step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Find/Replace: find live text next to a tracked deletion The previous fix rejected any regex match that overlapped tracked-deleted text, but `find_iter` had already consumed those positions, so a valid match overlapping the rejected one was never tried. "aaa" with the first "a" deleted reads "aa", yet Find "aa" found nothing (raw 0..2 was rejected and the scan resumed at 2) and Replace All replaced nothing. Searching the raw text also let deleted characters decide word boundaries: deleting the space in "cat fish" still matched whole-word "fish". `search` now builds each paragraph as it reads, without deleted runs, and searches each stretch of live text separately with `regex_automata::Input::range`. A match stays within one stretch, as before, and `\b`, `^` and `$` see the neighbouring live text, so a deletion neither creates nor hides a word boundary. `regex::Regex` cannot bound a search's end without slicing the haystack, which turns the slice end into a word boundary; regex-automata is the engine `regex` already uses (0.4.18, already in the lockfile), so no new code is built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Undo: a failed command restores history changed by nested commands The previous fix had the checkpoint report what it evicted and put that back on failure, but it only saw the outer command's own checkpoint. Commands run nested commands, and a nested one that succeeds checkpoints too. `file.inspect {"remove":["comments","revisions"]}` in a comments-only document runs `review.deleteComment` (allowed) and then `review.acceptAll` (refused). With 499 undo steps, the nested checkpoint evicted the oldest step and the rollback, truncating to 499, kept the outer command's empty step instead. At 500, only one of the two evictions came back. Nested undo, redo and version restore change the stacks in ways a length cannot describe either. Undo steps are now `Arc<Undo>`, and `run` snapshots both stacks whole before a mutating command and puts them back on failure, so the history is exact whatever ran inside. The snapshot copies one pointer per step, under 1,000 refcount bumps and an 8 KB allocation at MAX_UNDO, beside the two document clones every mutating command already makes. Undo and Redo are not mutating commands, so a top-level Undo still moves its document out of the step without cloning it. Only an undo nested inside a mutating command (a macro) clones, because the snapshot still holds the step. This replaces the eviction-return plumbing, so `checkpoint` is back to its original form. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Tests: compare undo timelines without comment timestamps failed_command_restores_history_changed_by_nested_commands builds the same session twice and compares their documents. Comments are stamped with the time to the second, so when the clock crossed a second between the two builds the documents differed and the test failed (about 1 run in 12). Clear the stamp in the compared copies; the test still checks every other part of each document in the timeline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Undo: wrap the typing-parity undo step in Arc after merging main Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Brandon Thomas <echelon@gmail.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
a1caf22a0b |
Bump version to 0.4.0 (from 0.3.0) for release; refresh contributors
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
7584b9b293 |
Bump version to 0.3.0 (from 0.2.0) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
2ed3895276 |
Bump version to 0.2.0 (from 0.1.0) for release
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Sirsyorrz
|
a9ce05d2db |
fix first-page header layout and tiff header images (#1)
- body on a titlePg first page now starts below the first-page header - decode tiff pictures - fade images through an opacity layer; vello_cpu panics on image sampler alpha |
||
|
Brandon ThomasandClaude Opus 5.5
|
435194f7c2 |
M0.9–M0.12: CLI, MCP server, xtask (layers/assets/parity/ci), icon placeholder, AGENTS/ATTRIBUTION/NOTICE, protocol docs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
98bbfbed4e |
M0.8: Word-style egui UI — ribbon (all tabs), page canvas, rulers, panes, dialogs, Backstage, control channel; desktop app
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
d1e80df4da |
M0.1–M0.6: workspace, geom, document model, fonts, layout, render
Clean-room foundations for WordCraft: Arc-shared document model with paragraph/run editing, styles, lists, sections and tables; Word-style font substitution; first-fit line breaking with tabs, lists, alignment, pagination, tables, headers/footers and hit testing; vello_cpu page rasteriser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |