Files
wordcraft/apps/wordcraft-cli/src/main.rs
T
663904a2e2 Import Word 97-2003 .doc files (#28) (#145)
* D0+D1: open Word 97-2003 .doc — container, FIB, piece table text (#28)

New wordcraft-docbin crate (L2) reads the OLE2 container via the cfb
crate (MIT) and parses the FIB and piece table from the [MS-DOC]
specification: text pieces decode as UTF-16LE or Windows-1252 (table
moved to wordcraft-doc::encoding), subdocument CP ranges, and Word's
special characters map to the model's (tabs, breaks, hyphens). .doc/.dot
registered in OPEN_EXTS and the I/O dispatch. Encrypted and Word 6/95
files give clear errors; every length is checked and proptest
truncation/flips never panic.

* D2: .doc formatting — stylesheet, fonts, CHPX/PAPX FKPs, sprms (#28)

Sprm decoder with operand-size rules (incl. sprmTDefTable/sprmPChgTabs
exceptions and the Prm0 isprm table), bin tables + ChpxFkp/PapxFkp page
lookup, STSH styles (StdfBase layout validated against real Word files:
bchUpe == cbStd, sti 0 = Normal) and the SttbfFfn font table in both
extended and self-delimiting flavours. The CP walk now splits runs on
formatting changes and resolves paragraph styles: bold/italic/strike/
underline/caps/hidden, colours (Ico palette + COLORREF), half-point
sizes, super/subscript, fonts, highlights; alignment, indents, spacing,
keep options, outline levels. Unknown sprms are skipped leniently;
an env-gated corpus test opens real files (styles + text asserted).

* D3: .doc structure — sections, headers/footers, tables, lists (#28)

- Sections: PlcfSed/Sepx with 16-bit measurement sprms (sprmSXaPage & co),
  explicit sprmSBOrientation with a width/height fallback; even/odd/first
  header and footer stories from Plcfhdd become document parts.
- Tables: sprmTDefTable grids (operand length fixed to cb+1 per the spec),
  horizontal merges (sprmTMerge), vertical merges (sprmTVertMerge), cell
  shading (sprmTSetShd / sprmTDefTableShd80), cell widths, valign, row
  height/header/cant-split, and table borders (sprmTTableBorders80 Brc80s).
- Lists: PlfLst (LSTF + appended LVLs) and PlfLfo (LFO + LFOLVL overrides)
  resolved into the numbering model; sprmPIlvl/sprmPIlfo mark paragraphs.
- Combined fixture (intro + table + list + 2 sections + header) round-trips
  through Document JSON; hostile proptests now mutate the structured fixture.

* D4: .doc notes, fields, bookmarks and inline pictures (#28)

- Footnotes/endnotes: PlcffndRef/PlcfEndRef reference CPs paired with the
  PlcffndTxt/PlcfEndTxt story ranges; each story walks into a Footnote or
  Endnote part and the 0x02 anchor becomes a NoteRef object.
- Fields: 0x13/0x14/0x15 state machine in the walker — instruction text is
  hidden, the cached result kept; HYPERLINK fields keep their result as
  linked runs, everything else becomes one Field object.
- Bookmarks: SttbfBkmk names + Plcfbkf/Plcfbkl CPs become BookmarkStart/
  BookmarkEnd objects at their anchors.
- Pictures: sprmCPicLocation points into the Data stream; the PICF header
  gives the display size (dxaGoal/dyaGoal scaled by mx/my) and the payload
  is format-sniffed (PNG/JPEG/GIF/BMP; metafiles are dropped with a warn).
- Hostile proptests now truncate/flip a fixture with all of the above.

* D5: .doc import hardening, caps and roadmap (#28)

- Cap inline pictures at 5 000 per document.
- Engine test: .doc/.dot bytes dispatch through the Word 97-2003 reader
  (garbage must error, never panic or open empty).
- ROADMAP: Word 97-2003 import row (60% — text, formatting, styles,
  sections, headers/footers, tables, lists, notes, fields, bookmarks,
  raster pictures).

Real-corpus validation stays opt-in: WORDCRAFT_DOCBIN_FIXTURES=<dir>
cargo test -p wordcraft-docbin corpus_files_open.

* Corpus fixes: compressed-piece FC space, PAPX query point, none-level labels (#28)

Validated against two real files (kept locally under plan/word/fixtures,
gitignored): a 1 MB Lorem-ipsum sample and a 2003 styles/lists/table
sample.

- FKP bin tables and pages address the WordDocument stream in true byte
  offsets; a compressed piece stores its base fc doubled (byte = fc/2),
  so fc_of_cp now unwinds the doubling. Before, every PAPX/CHPX lookup in
  compressed (cp1252) files landed on the wrong entry — tables collapsed
  to one row, lists and heading styles were lost.
- The paragraph-properties query now uses the paragraph mark's own FC
  (inside the paragraph) instead of the FC after it, which picked up the
  next paragraph's FKP entry.
- Fixture papx entries re-keyed to paragraph-start CPs, the real rgfc
  convention.
- Counters::next_label returns nothing for NumFormat::None levels (Word's
  numFmt none shows no label; also fixes a latent docx-reader case).
- Symbol/Wingdings bullet chars map to Unicode glyphs (• ▪ ○ ●).

With this, both corpus files render headings, styles, nested and numbered
lists, the tables and the PAGE field correctly; metafile-wrapped images
and OLE objects stay dropped by policy (logged).

* docbin: bound the text walk by real text; dedupe and cap pictures (#28)

Hostile CP numbers no longer drive the work:
- header/footnote/endnote story ranges from the Table stream are clamped to their
  subdocument (ccpHdd/ccpFtn/ccpEdn) with checked base additions, and every walk stops at
  the last piece's CP; a story spanning [0, 4e9) used to push billions of U+FFFD;
- characters are found by binary search over the pieces (PieceTable::char_at), and an
  unreadable stretch (gap, truncated or out-of-stream piece) is skipped in one step rather
  than one U+FFFD per CP; the old per-character scan was O(chars x pieces);
- the piece CPs are made non-decreasing at parse, so inverted pieces are empty and binary
  search stays sound; surrogate pairs now decode;
- note references are looked up by binary search; the style's CharProps is borrowed, not
  cloned, per character;
- pictures are interned by fc_pic (5000 anchors on one picture share one media entry and
  one copy) and the bytes copied per document are capped at 256 MB.

Each fix has a crafted-bytes regression test; against the previous code the story,
piece-claim, last-piece and many-pieces tests hang (killed at 60 s) and the shared
picture test copies the picture once per anchor (80 GB).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docbin: checked offset arithmetic in lists, sections and table rows (#28)

- list.rs: PlfLst/PlfLfo/LVL offsets (at + 2 + cLst*28, at + 4 + lfoMac*16, off += len,
  ...) saturate, which matters on 32-bit targets (wasm32) where fc + n can overflow usize;
  the LFO override path no longer indexes abstracts directly.
- sections.rs: the Sepx range (fc_sepx + 2 + cb) is checked; Plcfhdd is read only as far
  as the stories a capped section count can use (it used to reserve lcb/4 entries).
- table.rs: sprmTDyaRowHeight = i16::MIN negated in i16 (overflow panic in debug builds);
  now -(v as f32).

Regression tests: the row height test failed before (overflow panic with debug assertions, a 0 pt height without); the list test pins the saturating
offsets (it only overflowed on 32-bit targets).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Format the merged OPEN_EXTS list

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 08:36:38 -04:00

349 lines
16 KiB
Rust

//! `wordcraft-cli`: WordCraft from the command line.
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
use std::process::ExitCode;
use serde_json::{Value, json};
use wordcraft_engine::Session;
const USAGE: &str = "\
wordcraft-cli — WordCraft from the command line
USAGE:
wordcraft-cli convert <in> <out> convert between formats (docx, doc [read], pdf, odt, rtf, html, md, tex, txt, json, png)
wordcraft-cli info <file> pages, words, paragraphs, properties (JSON)
wordcraft-cli text <file> plain text
wordcraft-cli inspect <file> document structure (JSON)
wordcraft-cli render <file> <out.png> [--page N] [--scale S]
wordcraft-cli run [--file F | --template T] --cmd 'id={json}' [--cmd …] [--save OUT] [--print]
run commands headlessly, then save
wordcraft-cli commands [--json] list every command
wordcraft-cli parity [--markdown] feature-catalog parity
wordcraft-cli mcp [--connect HOST:PORT] MCP server on stdio (headless, or bridged to the app)
wordcraft-cli zotero <command> <file> [--cmd 'id={json}' …] [--save OUT] [--trace] [--port P]
run a Zotero command on a document (Zotero must be
running): addEditCitation, addEditBibliography,
addNote, refresh, removeCodes, setDocPrefs
(--cmd runs first, e.g. --cmd caret.docEnd)
wordcraft-cli --version
";
fn open(path: &str) -> Result<Session, String> {
let doc = wordcraft_engine::io::open_path(std::path::Path::new(path))?;
let mut s = Session::new(doc);
s.path = Some(path.into());
Ok(s)
}
fn arg_value(args: &[String], name: &str) -> Option<String> {
args.iter().position(|a| a == name).and_then(|i| args.get(i + 1)).cloned()
}
#[derive(Clone, Copy)]
struct OptionSpec {
name: &'static str,
takes_value: bool,
}
fn validate_options(command: &str, args: &[String], specs: &[OptionSpec]) -> Result<(), String> {
let mut i = 0;
while let Some(arg) = args.get(i) {
if let Some(name) = arg.strip_prefix("--") {
let Some(spec) = specs.iter().find(|spec| spec.name == arg) else {
return Err(format!("{command}: unknown option --{name}"));
};
if spec.takes_value && args.get(i + 1).is_some_and(|value| !value.starts_with("--")) {
i += 2;
} else {
i += 1;
}
} else {
i += 1;
}
}
Ok(())
}
const NO_OPTIONS: &[OptionSpec] = &[];
const RENDER_OPTIONS: &[OptionSpec] = &[OptionSpec { name: "--page", takes_value: true }, OptionSpec { name: "--scale", takes_value: true }];
const RUN_OPTIONS: &[OptionSpec] = &[
OptionSpec { name: "--file", takes_value: true },
OptionSpec { name: "--template", takes_value: true },
OptionSpec { name: "--cmd", takes_value: true },
OptionSpec { name: "--save", takes_value: true },
OptionSpec { name: "--page", takes_value: true },
OptionSpec { name: "--scale", takes_value: true },
OptionSpec { name: "--print", takes_value: false },
];
const COMMANDS_OPTIONS: &[OptionSpec] = &[OptionSpec { name: "--json", takes_value: false }];
const PARITY_OPTIONS: &[OptionSpec] = &[OptionSpec { name: "--markdown", takes_value: false }];
const MCP_OPTIONS: &[OptionSpec] = &[OptionSpec { name: "--connect", takes_value: true }];
const ZOTERO_OPTIONS: &[OptionSpec] = &[
OptionSpec { name: "--cmd", takes_value: true },
OptionSpec { name: "--save", takes_value: true },
OptionSpec { name: "--trace", takes_value: false },
OptionSpec { name: "--port", takes_value: true },
];
fn run(args: &[String]) -> Result<(), String> {
let cmd = args.first().map(String::as_str).unwrap_or("");
let rest: Vec<String> = args.iter().skip(1).cloned().collect();
let pos = |i: usize| rest.iter().filter(|a| !a.starts_with("--")).nth(i).cloned().ok_or_else(|| format!("missing argument\n\n{USAGE}"));
match cmd {
"convert" => {
validate_options("convert", &rest, NO_OPTIONS)?;
let (i, o) = (pos(0)?, pos(1)?);
let mut s = open(&i)?;
let r = if o.to_ascii_lowercase().ends_with(".png") {
s.run("file.exportPng", &json!({"path": o, "page": 1}))
} else {
s.run("file.save", &json!({"path": o}))
};
r.map_err(|e| e.to_string())?;
eprintln!("wrote {o}");
Ok(())
}
"info" => {
validate_options("info", &rest, NO_OPTIONS)?;
let mut s = open(&pos(0)?)?;
println!("{}", serde_json::to_string_pretty(&s.run("file.info", &json!({})).map_err(|e| e.to_string())?).unwrap_or_default());
Ok(())
}
"text" => {
validate_options("text", &rest, NO_OPTIONS)?;
let s = open(&pos(0)?)?;
println!("{}", s.doc.plain_text(wordcraft_doc::StoryRef::Body));
Ok(())
}
"inspect" => {
validate_options("inspect", &rest, NO_OPTIONS)?;
let mut s = open(&pos(0)?)?;
println!("{}", serde_json::to_string_pretty(&s.run("document.inspect", &json!({})).map_err(|e| e.to_string())?).unwrap_or_default());
Ok(())
}
"render" => {
validate_options("render", &rest, RENDER_OPTIONS)?;
let (i, o) = (pos(0)?, pos(1)?);
let mut s = open(&i)?;
let page = arg_value(&rest, "--page").and_then(|p| p.parse::<u64>().ok()).unwrap_or(1);
let scale = arg_value(&rest, "--scale").and_then(|p| p.parse::<f64>().ok()).unwrap_or(2.0);
let r = s.run("file.exportPng", &json!({"path": o, "page": page, "scale": scale})).map_err(|e| e.to_string())?;
println!("{r}");
Ok(())
}
"run" => {
validate_options("run", &rest, RUN_OPTIONS)?;
let mut s = match (arg_value(&rest, "--file"), arg_value(&rest, "--template")) {
(Some(f), _) => open(&f)?,
(None, Some(t)) => {
let mut s = Session::new(wordcraft_doc::Document::new());
s.run("file.new", &json!({"template": t})).map_err(|e| e.to_string())?;
s
}
_ => Session::new(wordcraft_doc::Document::new()),
};
let mut i = 0;
while i < rest.len() {
if rest.get(i).map(String::as_str) == Some("--cmd") {
let spec = rest.get(i + 1).ok_or("--cmd needs a value")?;
let (id, params) = match spec.split_once('=') {
Some((id, p)) => (id.to_string(), serde_json::from_str::<Value>(p).map_err(|e| format!("{id}: bad JSON params: {e}"))?),
None => (spec.clone(), json!({})),
};
let r = s.run(&id, &params).map_err(|e| format!("{id}: {e}"))?;
if rest.iter().any(|a| a == "--print") {
println!("{id}: {r}");
}
i += 2;
} else {
i += 1;
}
}
if let Some(out) = arg_value(&rest, "--save") {
let r = if out.to_ascii_lowercase().ends_with(".png") {
let mut v = json!({"path": out});
if let Some(n) = arg_value(&rest, "--page").and_then(|p| p.parse::<u64>().ok()) {
v["page"] = json!(n);
}
if let Some(k) = arg_value(&rest, "--scale").and_then(|p| p.parse::<f64>().ok()) {
v["scale"] = json!(k);
}
s.run("file.exportPng", &v)
} else {
s.run("file.save", &json!({"path": out}))
};
r.map_err(|e| e.to_string())?;
eprintln!("wrote {out}");
}
Ok(())
}
"commands" => {
validate_options("commands", &rest, COMMANDS_OPTIONS)?;
let s = Session::new(wordcraft_doc::Document::new());
if rest.iter().any(|a| a == "--json") {
println!("{}", serde_json::to_string_pretty(&s.registry.describe()).unwrap_or_default());
} else {
for c in s.registry.all() {
println!("{:<32} {:<36} {:<24} {}", c.id, c.label, c.shortcut, c.location);
}
}
Ok(())
}
"parity" => {
validate_options("parity", &rest, PARITY_OPTIONS)?;
let s = Session::new(wordcraft_doc::Document::new());
let p = wordcraft_engine::catalog::parity(&s.registry);
if rest.iter().any(|a| a == "--markdown") {
print!("{}", parity_markdown(&p));
} else {
println!("{}", serde_json::to_string_pretty(&p).unwrap_or_default());
}
Ok(())
}
"mcp" => {
validate_options("mcp", &rest, MCP_OPTIONS)?;
let backend: Box<dyn wordcraft_mcp::Backend> = match arg_value(&rest, "--connect") {
Some(addr) => {
let remote = wordcraft_mcp::Remote::connect(&addr).map_err(|e| format!("can't reach the WordCraft app at {addr}: {e}"))?;
if let Some(w) = remote.key_warning() {
eprintln!("wordcraft-cli: {w}");
}
Box::new(remote)
}
None => Box::new(wordcraft_mcp::Headless::default()),
};
let mut server = wordcraft_mcp::Server::new(backend);
let stdin = std::io::stdin();
server.serve(stdin.lock(), std::io::stdout()).map_err(|e| e.to_string())
}
"zotero" => {
validate_options("zotero", &rest, ZOTERO_OPTIONS)?;
let name = pos(0)?;
let command = wordcraft_zotero::Command::from_name(&name).ok_or_else(|| format!("unknown Zotero command `{name}`\n\n{USAGE}"))?;
let file = pos(1)?;
let mut s = open(&file)?;
for spec in rest.iter().zip(rest.iter().skip(1)).filter(|(a, _)| *a == "--cmd").map(|(_, v)| v) {
let (id, params) = match spec.split_once('=') {
Some((id, p)) => (id.to_string(), serde_json::from_str::<Value>(p).map_err(|e| format!("{id}: bad JSON params: {e}"))?),
None => (spec.clone(), json!({})),
};
s.run(&id, &params).map_err(|e| format!("{id}: {e}"))?;
}
let mut opts = wordcraft_zotero::client::Options::default();
if let Some(p) = arg_value(&rest, "--port") {
opts.addr.set_port(p.parse().map_err(|_| format!("bad port `{p}`"))?);
}
let trace = rest.iter().any(|a| a == "--trace");
let mut bridge = wordcraft_zotero::Bridge::new();
let mut host = wordcraft_zotero::Headless::default();
let out = wordcraft_zotero::client::run_command(&opts, command, &mut |call| {
let r = bridge.handle(&mut s, &mut host, call);
if trace {
eprintln!("← {} {}", call.method, clip(&Value::Array(call.args.clone()).to_string()));
eprintln!("→ {}", clip(&String::from_utf8_lossy(&wordcraft_zotero::wire::encode_reply(&r))));
}
r
})
.map_err(|e| e.to_string())?;
for a in &host.alerts {
eprintln!("Zotero: {a}");
}
eprintln!("{} calls, {}", out.calls, if out.completed { "completed" } else { "ended without completing (cancelled?)" });
if let Some(o) = arg_value(&rest, "--save") {
s.run("file.save", &json!({"path": o})).map_err(|e| e.to_string())?;
eprintln!("wrote {o}");
}
Ok(())
}
"--version" | "-V" => {
println!("wordcraft-cli {}", env!("CARGO_PKG_VERSION"));
Ok(())
}
"" | "help" | "--help" | "-h" => {
print!("{USAGE}");
Ok(())
}
other => Err(format!("unknown command `{other}`\n\n{USAGE}")),
}
}
/// At most 400 characters of `s`, for traces.
fn clip(s: &str) -> String {
match s.char_indices().nth(400) {
Some((i, _)) => format!("{}…", s.get(..i).unwrap_or(s)),
None => s.to_string(),
}
}
fn parity_markdown(p: &Value) -> String {
let mut s = String::from(
"# WordCraft feature parity\n\nGenerated by `cargo xtask parity` (`wordcraft-cli parity --markdown`): the word-processor feature catalog (`crates/engine/src/catalog.rs`) compared with the live command registry.\n\n",
);
s += &format!(
"**{} of {} catalog features have commands ({}%).**\n\n| Tab | Live | Total |\n|---|---|---|\n",
p["live"], p["total"], p["percent"]
);
for t in p["tabs"].as_array().cloned().unwrap_or_default() {
s += &format!("| {} | {} | {} |\n", t["tab"].as_str().unwrap_or(""), t["live"], t["total"]);
}
s += "\n## Missing\n\n";
for t in p["tabs"].as_array().cloned().unwrap_or_default() {
let m = t["missing"].as_array().cloned().unwrap_or_default();
if m.is_empty() {
continue;
}
s += &format!("### {}\n\n", t["tab"].as_str().unwrap_or(""));
for x in m {
s += &format!("- {}\n", x.as_str().unwrap_or(""));
}
s += "\n";
}
s
}
fn main() -> ExitCode {
let args: Vec<String> = std::env::args().skip(1).collect();
match run(&args) {
Ok(()) => ExitCode::SUCCESS,
Err(e) => {
eprintln!("wordcraft-cli: {e}");
ExitCode::FAILURE
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn args(values: &[&str]) -> Vec<String> {
values.iter().map(|value| (*value).into()).collect()
}
#[test]
fn validators_accept_only_the_documented_options() {
assert!(validate_options("info", &args(&["file.docx"]), NO_OPTIONS).is_ok());
assert!(validate_options("render", &args(&["in.docx", "out.png", "--page", "2", "--scale", "1.5"]), RENDER_OPTIONS).is_ok());
assert!(validate_options("run", &args(&["--cmd", "text.insert={}", "--cmd", "edit.undo", "--print"]), RUN_OPTIONS).is_ok());
assert!(validate_options("commands", &args(&["--json"]), COMMANDS_OPTIONS).is_ok());
assert!(validate_options("parity", &args(&["--markdown"]), PARITY_OPTIONS).is_ok());
assert!(validate_options("mcp", &args(&["--connect", "127.0.0.1:9000"]), MCP_OPTIONS).is_ok());
assert!(validate_options("render", &args(&["in.docx", "out.png", "--page"]), RENDER_OPTIONS).is_ok());
assert!(
validate_options(
"zotero",
&args(&["refresh", "in.docx", "--cmd", "caret.docEnd", "--save", "out.docx", "--trace", "--port", "23119"]),
ZOTERO_OPTIONS
)
.is_ok()
);
}
#[test]
fn unknown_options_are_rejected_before_file_access() {
assert_eq!(run(&args(&["info", "missing.docx", "--passwrod", "x"])), Err("info: unknown option --passwrod".into()));
assert_eq!(run(&args(&["render", "missing.docx", "out.png", "--sclae", "2"])), Err("render: unknown option --sclae".into()));
assert_eq!(run(&args(&["zotero", "refresh", "missing.docx", "--prot", "1"])), Err("zotero: unknown option --prot".into()));
}
}