diff --git a/Cargo.lock b/Cargo.lock index 503ba4373..9c4db4969 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -5238,6 +5238,7 @@ name = "wordcraft-formats" version = "0.7.0" dependencies = [ "image", + "js-sys", "log", "proptest", "quick-xml", diff --git a/ROADMAP.md b/ROADMAP.md index df85ead13..1bed44d9c 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~15 points (60% → 75% ready for real work) and ~150–230 h away -> **Last reviewed:** 2026-10-14 · **Last updated:** 2026-10-14 · **Change:** trivial (progress log: All Caps text in PDFs, #1473; previously trivial: progress log: Ink to Shape arrows; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** trivial (progress log: All Caps text in PDFs, #1473; previously trivial: progress log: ODT tracked changes, gap 19; previously trivial: progress log: RTF tracked changes, gap 19; previously trivial: progress log: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 > **Printing update:** 2026-10-12 · direct desktop printing through craft-print; details below. @@ -156,6 +156,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm | 2026-10-14 | Web: Save in the "save changes?" prompt before New or Open now waits for the browser's save picker: the new or opened document replaces yours only once the file is written and closed. Cancelling the picker, or a write that fails, keeps your document open with its edits and undo, still unsaved, and the prompt comes back with the reason; a save that ends after another document has replaced it no longer runs the New/Open on that one or marks it unsaved, and New/Open while a picker save is still in flight ask first (#1315) | | 2026-10-13 | Closing with unsaved changes → Save that can't write the file (no permission, locked by another app, folder gone) now says so plainly instead of looking like Cancel: the prompt comes back with the reason in it ("Couldn't save … so it is still open and nothing was lost") for Save again, Don't Save or Cancel, the status bar says it too, and `ui.saveChanges` returns that error; the same for a Save As from the prompt; the document stays open with its edits (#1445) | | 2026-10-14 | File › Info › Inspect Document finds hidden text the way Word does (#1344): hidden by direct formatting, a character style or a paragraph style, in the body, headers, footers, footnotes and endnotes, comments and text boxes, and Remove All deletes all of it (and nothing visible) in one undo step, whole hidden paragraphs and the notes of hidden references included. The result lists anything it couldn't reach (`residual`), so a sanitised file is never reported clean by mistake | +| 2026-10-14 | ODT files keep their tracked changes (gap 19): insertions and deletions made in LibreOffice open as revisions with their author and date, deleted paragraph breaks and text inserted then deleted included, so Accept All and Reject All give the right text; saving to ODT writes them back, and the Track Changes setting goes both ways | | 2026-10-14 | RTF keeps tracked changes (gap 19): insertions and deletions open as tracked changes with their authors and dates (`\revtbl`, `\revised`, `\deleted`, `\revdttm`) instead of plain text, so deleted text no longer reappears as normal text; saving to RTF writes them back, and Accept All / Reject All work after reopening | | 2026-10-14 | Alignment guides: moving a picture, shape or text box snaps its edges or centre to the page's edges and centre, the margins and the other floating objects on the page, with a thin line across the page for each guide while dragging; Alt moves freely; File › Options › Advanced › Editing turns them off (`ui.alignmentGuides`). Our own design: Word for Mac shows no guides | | 2026-10-14 | ODT files open with their sections (#1379): a paragraph that switches to another page style starts a new section on a new page, with that page's size, orientation, margins and columns and, when the file asks, page numbering restarting; a two-column `text:section` becomes a continuous two-column section. Before, the whole document was one portrait section, so landscape pages, columns and restarts were lost on open and on save to DOCX | @@ -337,6 +338,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm | 2026-10-14 | trivial | Progress log: the web's Save before New/Open waits for the save picker (#1315) | | 2026-10-13 | trivial | Progress log: a failed save from the close prompt says so (#1445) | | 2026-10-14 | trivial | Progress log: Inspect Document removes hidden text in every story (#1344) | +| 2026-10-14 | trivial | Progress log: ODT tracked changes (gap 19) | | 2026-10-14 | trivial | Progress log: RTF tracked insertions and deletions (gap 19) | | 2026-10-14 | trivial | Progress log: alignment guides while dragging pictures and shapes | | 2026-10-14 | trivial | Progress log: ODT sections on import (#1379) | diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index db442f84d..136d52cdb 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -5903,3 +5903,89 @@ fn exported_page_images_draw_headers_at_full_strength() { assert!(darkest < 60, "header drawn dimmed: darkest pixel {darkest}"); let _ = std::fs::remove_dir_all(&dir); } + +/// Gap 19, review: text inserted by one author and deleted by another keeps both marks through +/// an ODT save, so Reject All (undo the deletion, then the insertion) and Accept All both remove it. +#[test] +fn odt_inserted_then_deleted_text_round_trips() { + use std::io::Write; + let ns = r#"xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" xmlns:dc="http://purl.org/dc/elements/1.1/""#; + let info = |who: &str| format!("{who}2024-03-01T10:00:00"); + let content = format!( + r#"{}{}goneKeep this."#, + info("Alice"), + info("Bob") + ); + let mut z = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); + let o = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored); + for (name, data) in [("mimetype", "application/vnd.oasis.opendocument.text"), ("content.xml", content.as_str())] { + z.start_file(name, o).unwrap(); + z.write_all(data.as_bytes()).unwrap(); + } + let doc = crate::io::open_bytes("t.odt", &z.finish().unwrap().into_inner()).unwrap(); + let doc = crate::io::open_bytes("t.odt", &wordcraft_formats::odt::export(&doc).unwrap()).unwrap(); + for cmd in ["review.rejectAll", "review.acceptAll"] { + let mut s = Session::new(doc.clone()); + run(&mut s, cmd, json!({})); + assert_eq!(text(&s), "Keep this.", "{cmd}"); + } +} + +/// Gap 19: an ODT's tracked insertion and multi-paragraph deletion come in as revisions that +/// Reject All turns back into the original and Accept All into the edited text. +#[test] +fn odt_tracked_changes_accept_and_reject_all() { + use std::io::Write; + let ns = r#"xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" xmlns:dc="http://purl.org/dc/elements/1.1/""#; + let info = |who: &str| format!("{who}2024-03-01T10:00:00"); + let content = format!( + r#"{}{}tailheadFirst rest added.End."#, + info("Alice"), + info("Bob") + ); + let mut z = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); + let o = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored); + for (name, data) in [("mimetype", "application/vnd.oasis.opendocument.text"), ("content.xml", content.as_str())] { + z.start_file(name, o).unwrap(); + z.write_all(data.as_bytes()).unwrap(); + } + let bytes = z.finish().unwrap().into_inner(); + let doc = crate::io::open_bytes("t.odt", &bytes).unwrap(); + + let mut sr = Session::new(doc.clone()); + run(&mut sr, "review.rejectAll", json!({})); + assert_eq!(text(&sr), "First tail\nhead rest .\nEnd."); + assert!(sr.doc.revisions.is_empty() || run(&mut sr, "review.changes", json!({})).as_array().unwrap().is_empty()); + + let mut sa = Session::new(doc); + run(&mut sa, "review.acceptAll", json!({})); + assert_eq!(text(&sa), "First rest added.\nEnd."); + assert!(run(&mut sa, "review.changes", json!({})).as_array().unwrap().is_empty()); +} + +/// Gap 19, review 2: a paragraph break deleted inside an insertion range is not itself +/// inserted, so Reject All keeps the two paragraphs (after an ODT save, too). +#[test] +fn odt_deleted_mark_inside_an_insertion_survives_reject_all() { + use std::io::Write; + let ns = r#"xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" xmlns:dc="http://purl.org/dc/elements/1.1/""#; + let info = |who: &str| format!("{who}2024-03-01T10:00:00"); + let content = format!( + r#"{}{}One AC two"#, + info("Alice"), + info("Bob") + ); + let mut z = zip::ZipWriter::new(std::io::Cursor::new(Vec::new())); + let o = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored); + for (name, data) in [("mimetype", "application/vnd.oasis.opendocument.text"), ("content.xml", content.as_str())] { + z.start_file(name, o).unwrap(); + z.write_all(data.as_bytes()).unwrap(); + } + let doc = crate::io::open_bytes("t.odt", &z.finish().unwrap().into_inner()).unwrap(); + let saved = crate::io::open_bytes("t.odt", &wordcraft_formats::odt::export(&doc).unwrap()).unwrap(); + for d in [doc, saved] { + let mut s = Session::new(d); + run(&mut s, "review.rejectAll", json!({})); + assert_eq!(text(&s), "One \n two"); + } +} diff --git a/crates/formats/Cargo.toml b/crates/formats/Cargo.toml index 393599097..a892251ed 100644 --- a/crates/formats/Cargo.toml +++ b/crates/formats/Cargo.toml @@ -16,5 +16,9 @@ quick-xml = { workspace = true } zip = { workspace = true } log = { workspace = true } +[target.'cfg(target_arch = "wasm32")'.dependencies] +# The browser clock (`SystemTime::now()` panics on wasm32): ODT change dates. +js-sys = "0.3" + [dev-dependencies] proptest = { workspace = true } diff --git a/crates/formats/src/model.rs b/crates/formats/src/model.rs index feb78a0b4..8ab44b70c 100644 --- a/crates/formats/src/model.rs +++ b/crates/formats/src/model.rs @@ -48,6 +48,10 @@ pub struct Fmt { pub struct RevMark { pub author: String, pub date: String, + /// The source's identity of the change, when it has one (an ODT changed region; the + /// revision index in [`from_doc_tracked`]): marks with different ids stay separate + /// revisions. `None` (RTF) = one revision per kind, author and date. + pub id: Option, } /// An embedded picture. @@ -125,6 +129,11 @@ pub struct Para { pub borders: Option, /// The section this paragraph ends (RTF `\sect`), as the document's last paragraph of it. pub section: Option>, + /// Tracked insertion of the paragraph mark. + pub mark_ins: Option>, + /// Tracked deletion of the paragraph mark (only [`from_doc_tracked`] and formats that keep + /// revisions set it). + pub mark_del: Option>, } impl Para { @@ -414,7 +423,7 @@ impl<'a> Builder<'a> { let mut revs = std::collections::HashMap::new(); for (i, r) in doc.revisions.iter().enumerate() { if r.move_name.is_none() { - let mark = RevMark { author: r.author.clone(), date: r.date.clone() }; + let mark = RevMark { author: r.author.clone(), date: r.date.clone(), id: None }; revs.entry((rev_key(r.kind), mark)).or_insert(i as u32); } } @@ -500,6 +509,8 @@ impl<'a> Builder<'a> { } } } + out.mark.ins = p.mark_ins.as_deref().map(|m| self.revision(RevisionKind::Insert, m)); + out.mark.del = p.mark_del.as_deref().map(|m| self.revision(RevisionKind::Delete, m)); out.normalize(); out.touch(); out @@ -744,7 +755,7 @@ fn fmt_of(doc: &Document, cp: &CharProps, in_code: bool) -> Fmt { /// The insertion and deletion `cp` tracks, as flow marks. fn rev_marks(doc: &Document, cp: &CharProps) -> (Option, Option) { let mark = |i: Option| { - i.map(|i| doc.revisions.get(i as usize).map(|r| RevMark { author: r.author.clone(), date: r.date.clone() }).unwrap_or_default()) + i.map(|i| doc.revisions.get(i as usize).map(|r| RevMark { author: r.author.clone(), date: r.date.clone(), id: Some(i) }).unwrap_or_default()) }; (mark(cp.ins), mark(cp.del)) } @@ -807,6 +818,8 @@ fn flow_paras_in(doc: &Document, p: &Paragraph, tracked: bool) -> Vec { spacing: Spacing::default(), borders: None, section: None, + mark_ins: None, + mark_del: None, }; let in_code = kind == Kind::Code; let mut k = 0usize; @@ -833,6 +846,8 @@ fn flow_paras_in(doc: &Document, p: &Paragraph, tracked: bool) -> Vec { spacing: Spacing::default(), borders: None, section: None, + mark_ins: None, + mark_del: None, }; let prev = std::mem::replace(&mut out, next); if prev.inlines.is_empty() { @@ -898,6 +913,10 @@ fn flow_paras_in(doc: &Document, p: &Paragraph, tracked: bool) -> Vec { out.push_text(&buf, &f); } } + if tracked { + let (ins, del) = rev_marks(doc, &p.mark); + (out.mark_ins, out.mark_del) = (ins.map(Box::new), del.map(Box::new)); + } done.push(out); done } diff --git a/crates/formats/src/odt.rs b/crates/formats/src/odt.rs index fca92ab99..b13ca181a 100644 --- a/crates/formats/src/odt.rs +++ b/crates/formats/src/odt.rs @@ -8,8 +8,14 @@ //! Quotations, Preformatted Text, page layout) and `meta.xml`. //! //! Import reads the same subset back from `content.xml` / `styles.xml` (style inheritance -//! resolved, both automatic and common styles), skipping notes, annotations, text boxes and -//! tracked-change records. +//! resolved, both automatic and common styles), skipping notes, annotations and text boxes. +//! +//! Tracked changes (ODF 1.3 Part 3 §5.5) go both ways: `text:tracked-changes` holds one +//! `text:changed-region` per change; an insertion is the body text between +//! `text:change-start`/`text:change-end`, a deletion keeps the deleted paragraphs in its region +//! and leaves a `text:change` point in the body. They map to `Document::revisions` and the +//! `ins`/`del` marks of runs and paragraph marks; a deleted paragraph mark is a region of +//! several paragraphs. Format changes are read as no change. use std::collections::HashMap; use std::io::{Cursor, Read, Write}; @@ -17,9 +23,9 @@ use std::io::{Cursor, Read, Write}; use quick_xml::XmlVersion; use quick_xml::events::{BytesStart, Event}; use wordcraft_doc::section::{Columns, SectionStart}; -use wordcraft_doc::{Align, Document, Rgb, SectionProps}; +use wordcraft_doc::{Align, Document, RevisionKind, Rgb, SectionProps}; -use crate::model::{self, Cell, FBlock, FTable, Flow, Fmt, Inline, Kind, ListInfo, Meta, Para, make_img, mime_of}; +use crate::model::{self, Cell, FBlock, FTable, Flow, Fmt, Inline, Kind, ListInfo, Meta, Para, RevMark, make_img, mime_of}; /// Largest zip entry we inflate. const MAX_ENTRY: u64 = 256 << 20; @@ -27,6 +33,12 @@ const MAX_ENTRY: u64 = 256 << 20; const MAX_XML_DEPTH: usize = 512; /// Most sections an import makes (page-style changes and column sections); later ones are ignored. const MAX_SECTIONS: usize = 10_000; +/// Most `text:changed-region`s read (hostile input). +const MAX_REGIONS: usize = 10_000; +/// Most insertions open at once (`text:change-start` without its end yet). +const MAX_OPEN_INSERTIONS: usize = 64; +/// Longest change author or date kept, in characters. +const MAX_INFO_CHARS: usize = 256; // --------------------------------------------------------------------------------------------- // Export @@ -79,6 +91,16 @@ struct Writer { /// Cell styles by shading. cell_styles: Vec>, pictures: Vec<(String, std::sync::Arc>, String)>, + /// `text:changed-region` elements written so far. + regions: Vec, + /// The insertion open in the body, and its region id. + open_ins: Option<(RevMark, String)>, + /// Style of the paragraph being written. + cur_style: String, + /// Styles of the paragraphs joined into the one being written (deleted paragraph marks), + /// and which of them the text being written belongs to. + chain: Vec, + part: usize, } fn text_props(f: &Fmt) -> String { @@ -181,7 +203,119 @@ impl Writer { name } - fn inlines(&mut self, inl: &[Inline], out: &mut String) { + /// A new `text:changed-region` of `kind` by `m` with `body` after its change info; its id. + fn region(&mut self, kind: &str, m: &RevMark, body: &str) -> String { + let id = format!("ct{}", self.regions.len() + 1); + // `dc:date` is required: a change without a date is dated now. + let date = if m.date.trim().is_empty() { now_iso() } else { m.date.clone() }; + self.regions.push(format!( + "{}{}{body}", + x(&m.author), + x(&date) + )); + id + } + + /// Make `m` the insertion open in `st` (closing another one first). + fn ins_open(&mut self, st: &mut Option<(RevMark, String)>, m: &RevMark, out: &mut String) { + if st.as_ref().is_some_and(|(o, _)| o == m) { + return; + } + ins_close(st, out); + let ct = self.region("insertion", m, ""); + out.push_str(&format!("")); + *st = Some((m.clone(), ct)); + } + + fn open_ins(&mut self, m: &RevMark, out: &mut String) { + let mut st = self.open_ins.take(); + self.ins_open(&mut st, m, out); + self.open_ins = st; + } + + fn close_ins(&mut self, out: &mut String) { + ins_close(&mut self.open_ins, out); + } + + /// Inlines with their insertions as change ranges kept in `st`. + fn ins_inlines(&mut self, items: &[Item<'_>], st: &mut Option<(RevMark, String)>, out: &mut String) { + let mut k = 0; + while k < items.len() { + let ins = items.get(k).and_then(|i| i.marks().0); + let mut j = k + 1; + while j < items.len() && items.get(j).and_then(|i| i.marks().0) == ins { + j += 1; + } + match ins { + Some(m) => self.ins_open(st, m, out), + None => ins_close(st, out), + } + let seg: Vec = items.get(k..j).unwrap_or(&[]).iter().filter_map(|i| i.inline().cloned()).collect(); + self.plain_inlines(&seg, out); + k = j; + } + } + + /// Deleted content: a region holding it (one paragraph per deleted paragraph mark plus + /// one; text inserted before it was deleted as change ranges inside) and a `text:change` + /// point in the body. + fn deletion(&mut self, seg: &[Item<'_>], m: &RevMark, out: &mut String) { + let mut body = String::new(); + let mut st: Option<(RevMark, String)> = None; + let mut start = 0; + let mut first = true; + loop { + let end = seg.iter().skip(start).position(|i| matches!(i, Item::Mark(_))).map(|e| start + e); + if !first { + self.part += 1; + } + first = false; + let style = self.chain.get(self.part).cloned().unwrap_or_else(|| self.cur_style.clone()); + body.push_str(&format!("")); + self.ins_inlines(seg.get(start..end.unwrap_or(seg.len())).unwrap_or(&[]), &mut st, &mut body); + // A deleted paragraph mark that had been inserted: the range runs on past it. + match end.and_then(|e| seg.get(e)).and_then(|i| i.marks().0) { + Some(ins) => self.ins_open(&mut st, ins, &mut body), + None => ins_close(&mut st, &mut body), + } + body.push_str(""); + match end { + Some(e) => start = e + 1, + None => break, + } + } + let ct = self.region("deletion", m, &body); + out.push_str(&format!("")); + } + + /// Paragraph content, with tracked insertions as change ranges and deletions as change points. + fn items(&mut self, items: &[Item<'_>], out: &mut String) { + let mut k = 0; + while k < items.len() { + let key = items.get(k).map(Item::segment).unwrap_or_default(); + let mut j = k + 1; + while j < items.len() && items.get(j).map(Item::segment).unwrap_or_default() == key { + j += 1; + } + let seg = items.get(k..j).unwrap_or(&[]); + match key { + (_, Some(del)) => self.deletion(seg, del, out), + (Some(ins), None) => { + self.open_ins(ins, out); + let inl: Vec = seg.iter().filter_map(|i| i.inline().cloned()).collect(); + self.plain_inlines(&inl, out); + } + (None, None) => { + self.close_ins(out); + let inl: Vec = seg.iter().filter_map(|i| i.inline().cloned()).collect(); + self.plain_inlines(&inl, out); + } + } + k = j; + } + } + + fn plain_inlines(&mut self, inl: &[Inline], out: &mut String) { let mut prev_space = true; let inl = model::equations_as_text(inl); for i in inl.iter() { @@ -256,23 +390,54 @@ impl Writer { } } - fn para(&mut self, p: &Para, out: &mut String) { + /// Paragraph `p`, or with `chain` the paragraphs joined at deleted paragraph marks (`p` + /// is the first). + fn para_in(&mut self, p: &Para, chain: Option<&Vec>, out: &mut String) { + let paras: &[Para] = chain.map(|c| c.as_slice()).unwrap_or(std::slice::from_ref(p)); + self.chain = chain.map(|c| c.iter().map(|q| self.para_style(q)).collect()).unwrap_or_default(); + self.part = 0; let style = self.para_style(p); - match p.kind { + self.cur_style = style.clone(); + let tag = match p.kind { Kind::Heading(n) => { out.push_str(&format!("", n.clamp(1, 6))); - self.inlines(&p.inlines, out); - out.push_str(""); + "" } _ => { out.push_str(&format!("")); - self.inlines(&p.inlines, out); - out.push_str(""); + "" } + }; + let tracked = paras.len() > 1 || p.inlines.iter().any(|i| matches!(i, Inline::Text(_, f) if f.ins.is_some() || f.del.is_some())); + if tracked { + let mut items = Vec::new(); + for (k, q) in paras.iter().enumerate() { + items.extend(q.inlines.iter().map(Item::Inl)); + if k + 1 < paras.len() { + items.push(Item::Mark(q)); + } + } + self.items(&items, out); + } else { + self.close_ins(out); + self.plain_inlines(&p.inlines, out); } + // An inserted paragraph mark: the insertion runs on past the paragraph's end. + match paras.last().and_then(|q| q.mark_ins.as_deref()) { + Some(m) => self.open_ins(m, out), + None => self.close_ins(out), + } + out.push_str(tag); + self.chain.clear(); } fn blocks(&mut self, blocks: &[FBlock], out: &mut String, depth: usize) { + let joined = join_deleted_marks(blocks); + let (blocks, chains) = match &joined { + Some((b, c)) => (b.as_slice(), Some(c)), + None => (blocks, None), + }; + let chain = |i: usize| chains.and_then(|c| c.get(&i)); let mut i = 0; while let Some(b) = blocks.get(i) { match b { @@ -305,7 +470,7 @@ impl Writer { let lname = format!("L{}", self.list_styles.len()); // Open lists: per level, whether a list-item is open. let mut stack: Vec = Vec::new(); - for q in blocks.get(start..i).unwrap_or(&[]) { + for (qi, q) in blocks.get(start..i).unwrap_or(&[]).iter().enumerate() { let FBlock::Para(q) = q else { continue }; let lv = q.list.map(|l| l.level.min(8) as usize).unwrap_or(0); while stack.len() > lv + 1 { @@ -335,7 +500,7 @@ impl Writer { *top = true; } out.push_str(""); - self.para(q, out); + self.para_in(q, chain(start + qi), out); } while let Some(open) = stack.pop() { if open { @@ -346,7 +511,7 @@ impl Writer { out.push('\n'); } FBlock::Para(p) => { - self.para(p, out); + self.para_in(p, chain(i), out); out.push('\n'); i += 1; } @@ -366,6 +531,7 @@ impl Writer { } fn table(&mut self, t: &FTable, out: &mut String, depth: usize) { + self.close_ins(out); let cols = t.cols().max(1); let widths: Vec = if t.widths.len() == cols && t.widths.iter().all(|w| w.is_finite() && *w > 0.0) { t.widths.clone() @@ -407,6 +573,7 @@ impl Writer { out.push_str(""); } self.blocks(&c.blocks, out, depth + 1); + self.close_ins(out); out.push_str(""); for _ in 1..span { out.push_str(""); @@ -481,6 +648,112 @@ impl Writer { } } +/// One piece of a paragraph being written: an inline, or the deleted mark of a paragraph +/// joined to the next one. +enum Item<'a> { + Inl(&'a Inline), + Mark(&'a Para), +} + +impl Item<'_> { + /// Its tracked insertion and deletion. + fn marks(&self) -> (Option<&RevMark>, Option<&RevMark>) { + match self { + Item::Inl(Inline::Text(_, f)) => (f.ins.as_ref(), f.del.as_ref()), + Item::Inl(_) => (None, None), + Item::Mark(p) => (p.mark_ins.as_deref(), p.mark_del.as_deref()), + } + } + + /// What a run of content is written as: deleted content is one region whatever was + /// inserted inside it, so only its deletion counts. + fn segment(&self) -> (Option<&RevMark>, Option<&RevMark>) { + match self.marks() { + (_, Some(del)) => (None, Some(del)), + k => k, + } + } + + fn inline(&self) -> Option<&Inline> { + match self { + Item::Inl(i) => Some(i), + Item::Mark(_) => None, + } + } +} + +/// Close the insertion open in `st`. +fn ins_close(st: &mut Option<(RevMark, String)>, out: &mut String) { + if let Some((_, ct)) = st.take() { + out.push_str(&format!("")); + } +} + +/// `blocks` with each run of paragraphs joined at tracked deleted paragraph marks (ODF keeps +/// such a deletion inside one body paragraph): the first paragraph stands for the run, and the +/// whole run is returned by its index. `None` when nothing is joined. A deleted mark with no +/// paragraph after it (before a table, at the end) is written as a plain mark. +#[allow(clippy::type_complexity)] +fn join_deleted_marks(blocks: &[FBlock]) -> Option<(Vec, HashMap>)> { + if !blocks.iter().any(|b| matches!(b, FBlock::Para(p) if p.mark_del.is_some())) { + return None; + } + let mut out = Vec::with_capacity(blocks.len()); + let mut chains = HashMap::new(); + let mut i = 0; + while let Some(b) = blocks.get(i) { + i += 1; + let FBlock::Para(p) = b else { + out.push(b.clone()); + continue; + }; + let mut chain = vec![p.clone()]; + while chain.last().is_some_and(|q| q.mark_del.is_some()) + && let Some(FBlock::Para(q)) = blocks.get(i) + { + chain.push(q.clone()); + i += 1; + } + if chain.len() > 1 { + chains.insert(out.len(), chain); + } + out.push(FBlock::Para(p.clone())); + } + Some((out, chains)) +} + +/// Now, as an `xsd:dateTime` (UTC). +fn now_iso() -> String { + let t = now_unix(); + let (days, secs) = (t.div_euclid(86_400), t.rem_euclid(86_400)); + // Days since 1970-01-01 → civil date (Howard Hinnant's algorithm). + let z = days + 719_468; + let era = z.div_euclid(146_097); + let doe = z - era * 146_097; + let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365; + let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); + let mp = (5 * doy + 2) / 153; + let d = doy - (153 * mp + 2) / 5 + 1; + let m = if mp < 10 { mp + 3 } else { mp - 9 }; + let y = yoe + era * 400 + i64::from(m <= 2); + format!("{y:04}-{m:02}-{d:02}T{:02}:{:02}:{:02}Z", secs / 3600, secs / 60 % 60, secs % 60) +} + +/// Seconds since the Unix epoch: the system clock, or the browser's on the web. +fn now_unix() -> i64 { + #[cfg(not(target_arch = "wasm32"))] + { + use std::time::{SystemTime, UNIX_EPOCH}; + SystemTime::now().duration_since(UNIX_EPOCH).ok().and_then(|d| i64::try_from(d.as_secs()).ok()).unwrap_or(0) + } + // `SystemTime::now()` panics on wasm32-unknown-unknown, so ask the browser's clock. + #[cfg(target_arch = "wasm32")] + { + let ms = js_sys::Date::now(); + if ms.is_finite() && ms > 0.0 { (ms / 1000.0) as i64 } else { 0 } + } +} + fn styles_xml(sec: &wordcraft_doc::SectionProps, body_font: &str) -> String { let mut s = format!("\n"); s.push_str(&format!( @@ -542,12 +815,18 @@ fn meta_xml(m: &Meta) -> String { /// Write an `.odt` package. pub fn export(doc: &Document) -> Result, String> { - let flow = model::from_doc(doc); + let flow = model::from_doc_tracked(doc); let mut w = Writer::default(); let mut body = String::new(); w.blocks(&flow.blocks, &mut body, 0); + w.close_ins(&mut body); + let tracked = if w.regions.is_empty() && !doc.settings.track_changes { + String::new() + } else { + format!("{}\n", doc.settings.track_changes, w.regions.concat()) + }; let content = format!( - "\n{}\n{body}\n", + "\n{}\n{tracked}{body}\n", w.automatic_styles() ); let body_font = doc.styles.default_chr.font.clone().unwrap_or_else(|| wordcraft_doc::styles::BODY_FONT.to_string()); @@ -991,6 +1270,37 @@ struct TableB { hcover: usize, } +/// A `text:changed-region` read from `text:tracked-changes`. +#[derive(Default)] +struct Region { + kind: Option, + /// Sequence number: tells regions by the same author at the same time apart. + id: u32, + author: String, + date: String, + /// A deletion's deleted paragraphs. + blocks: Vec, +} + +impl Region { + fn mark(&self) -> RevMark { + RevMark { author: self.author.trim().to_string(), date: self.date.trim().to_string(), id: Some(self.id) } + } +} + +/// Reader state saved while a deletion's content is read into its own container. +struct DelState { + containers: usize, + tables: usize, + cells: usize, + lists_saved: usize, + fmt: usize, + lists: Vec<(Option, bool)>, + item_used: Vec, + /// Insertions open inside the deleted content (text inserted, then deleted). + ins: Vec<(String, RevMark)>, +} + struct Body<'a> { styles: &'a Styles, zip: &'a mut dyn FnMut(&str) -> Option>, @@ -1020,6 +1330,23 @@ struct Body<'a> { sections: usize, /// Element depth of the open columned `text:section`. cols_at: Option, + /// Inside `text:tracked-changes`. + in_tc: bool, + /// `text:track-changes` of the document's tracked changes (default on), when it has them. + track: Option, + /// Changed regions read, by id. + regions: HashMap, + /// Regions seen (capped at [`MAX_REGIONS`]). + region_count: usize, + /// The region being read. + region: Option<(String, Region)>, + /// Depth of the `office:change-info` being read, and the field its text goes to. + info_at: Option, + info_field: Option, + /// Reading a deletion's content. + del: Option, + /// Insertions open in the body: (region id, revision). + active_ins: Vec<(String, RevMark)>, } impl Body<'_> { @@ -1123,7 +1450,31 @@ impl Body<'_> { fn end_para(&mut self) { let Some(mut p) = self.para.take() else { return }; self.fmt.pop(); - if p.kind != Kind::Code { + // A paragraph ending inside an open insertion has an inserted mark. (Paragraphs that + // `place_deletion` splits off keep only the marks read inside their region.) + if p.mark_ins.is_none() { + p.mark_ins = self.open_insertion().map(Box::new); + } + self.finish_para(p); + } + + /// The innermost insertion open where text is being read (the body, or deleted content). + fn open_insertion(&self) -> Option { + let open = match &self.del { + Some(d) => &d.ins, + None => &self.active_ins, + }; + open.last().map(|(_, m)| m.clone()) + } + + /// Add a finished paragraph to the current container. + fn finish_para(&mut self, mut p: Para) { + // Spaces next to tracked text are part of a change. + let tracked = self.del.is_some() + || p.mark_ins.is_some() + || p.mark_del.is_some() + || p.inlines.iter().any(|i| matches!(i, Inline::Text(_, f) if f.ins.is_some() || f.del.is_some())); + if p.kind != Kind::Code && !tracked { p.trim(); } if p.kind == Kind::Rule && !p.is_empty() { @@ -1156,7 +1507,8 @@ impl Body<'_> { } fn push_text(&mut self, t: &str) { - let f = self.cur_fmt(); + let mut f = self.cur_fmt(); + f.ins = self.open_insertion(); if self.para.is_none() { if t.trim().is_empty() { return; @@ -1172,6 +1524,17 @@ impl Body<'_> { if self.skip > 0 || !self.in_text { return; } + if self.info_at.is_some() { + if let (Some(field), Some((_, r))) = (self.info_field, &mut self.region) { + let s = if field { &mut r.author } else { &mut r.date }; + let room = MAX_INFO_CHARS.saturating_sub(s.chars().count()); + s.extend(t.chars().take(room)); + } + return; + } + if self.in_tc && self.del.is_none() { + return; + } if self.in_desc { if let Some(fr) = &mut self.frame { fr.3.push_str(t); @@ -1217,10 +1580,57 @@ impl Body<'_> { if !self.in_text { return; } + if self.info_at.is_some() { + self.info_field = match local(q) { + "creator" => Some(true), + "date" => Some(false), + _ => None, + }; + return; + } + if self.in_tc { + if self.del.is_none() { + self.tracked_start(q, a); + return; + } + // Inside deleted content: only insertions (text inserted, then deleted) nest. + if matches!(local(q), "change-start" | "change-end") { + let regions = &self.regions; + if let Some(d) = &mut self.del { + change_mark(regions, &mut d.ins, local(q) == "change-start", a); + } + return; + } + if matches!(local(q), "tracked-changes" | "changed-region" | "insertion" | "deletion" | "format-change" | "change") { + self.skip = 1; + return; + } + if local(q) == "change-info" && self.region.is_some() { + self.info_at = Some(self.depth); + return; + } + } match local(q) { + "tracked-changes" => { + if self.para.is_none() && self.tables.is_empty() && self.cells.is_empty() && self.frame.is_none() { + self.in_tc = true; + self.track = Some(get(a, "text:track-changes") != Some("false")); + } else { + self.skip = 1; + } + } + "change" => { + if let Some(id) = get(a, "text:change-id") + && self.regions.get(id).is_some_and(|r| r.kind == Some(RevisionKind::Delete)) + && let Some(r) = self.regions.remove(id) + { + let mark = r.mark(); + self.place_deletion(r.blocks, mark); + } + } + "change-start" | "change-end" => change_mark(&self.regions, &mut self.active_ins, local(q) == "change-start", a), "note" | "annotation" - | "tracked-changes" | "text-box" | "sequence-decls" | "forms" @@ -1355,9 +1765,141 @@ impl Body<'_> { } } + /// An element inside `text:tracked-changes`, outside deleted content. + fn tracked_start(&mut self, q: &str, a: &Attrs) { + match local(q) { + "changed-region" if self.region.is_none() && self.region_count < MAX_REGIONS => { + let id = get(a, "text:id").unwrap_or_default().to_string(); + let r = Region { id: self.region_count as u32, ..Default::default() }; + self.region_count += 1; + self.region = Some((id, r)); + } + "insertion" | "deletion" | "format-change" if self.region.as_ref().is_some_and(|(_, r)| r.kind.is_none()) => { + let kind = match local(q) { + "insertion" => RevisionKind::Insert, + "deletion" => RevisionKind::Delete, + _ => RevisionKind::Format, + }; + if let Some((_, r)) = &mut self.region { + r.kind = Some(kind); + } + if kind == RevisionKind::Delete { + self.containers.push(Vec::new()); + self.del = Some(DelState { + containers: self.containers.len(), + tables: self.tables.len(), + cells: self.cells.len(), + lists_saved: self.lists_saved.len(), + fmt: self.fmt.len(), + lists: std::mem::take(&mut self.lists), + item_used: std::mem::take(&mut self.item_used), + ins: Vec::new(), + }); + } + } + "change-info" if self.region.is_some() => self.info_at = Some(self.depth), + _ => self.skip = 1, + } + } + + /// End a deletion's content: its paragraphs go to the region being read. + fn close_deletion(&mut self) { + if self.del.is_none() { + return; + } + self.end_para(); + let Some(d) = self.del.take() else { return }; + let base = d.containers.saturating_sub(1).min(self.containers.len()); + let blocks: Vec = self.containers.split_off(base).into_iter().flatten().filter(|b| matches!(b, FBlock::Para(_))).collect(); + if self.containers.is_empty() { + self.containers.push(Vec::new()); + } + self.tables.truncate(d.tables); + self.cells.truncate(d.cells); + self.lists_saved.truncate(d.lists_saved); + self.fmt.truncate(d.fmt); + self.lists = d.lists; + self.item_used = d.item_used; + self.frame = None; + self.in_desc = false; + self.para_cont = false; + if let Some((_, r)) = &mut self.region { + r.blocks = blocks; + } + } + + /// Put a deletion's paragraphs back at a `text:change` point, as deleted text. Inside a + /// paragraph, each deleted paragraph mark splits it; between paragraphs, each deleted + /// paragraph stands alone with its mark deleted. + fn place_deletion(&mut self, blocks: Vec, mark: RevMark) { + let mut paras: Vec = Vec::new(); + for b in blocks { + let FBlock::Para(mut p) = b else { continue }; + // Pictures and other objects carry no revision mark: leave them out. + p.inlines.retain(|i| matches!(i, Inline::Text(..) | Inline::Anchor(_))); + for i in &mut p.inlines { + if let Inline::Text(_, f) = i { + f.del = Some(mark.clone()); + } + } + paras.push(p); + } + let append = |dst: &mut Para, src: Vec| { + for i in src { + match i { + Inline::Text(t, f) => dst.push_text(&t, &f), + other => dst.inlines.push(other), + } + } + }; + let Some(mut cur) = self.para.take() else { + self.para_cont = false; + for mut p in paras { + p.mark_del = Some(Box::new(mark.clone())); + self.finish_para(p); + } + return; + }; + let n = paras.len(); + let mut it = paras.into_iter(); + let mut first_mark_ins = None; + if let Some(first) = it.next() { + first_mark_ins = first.mark_ins; + append(&mut cur, first.inlines); + } + if n < 2 { + self.para = Some(cur); + return; + } + let mut rest = Para { kind: cur.kind, list: cur.list, align: cur.align, ..Default::default() }; + // The host paragraph now ends at the first deleted paragraph's mark. + cur.mark_del = Some(Box::new(mark.clone())); + cur.mark_ins = first_mark_ins; + self.finish_para(cur); + for (k, mut p) in it.enumerate() { + if k + 2 == n { + // The last deleted paragraph runs on into the rest of the body paragraph. + rest.kind = p.kind; + rest.align = p.align.or(rest.align); + append(&mut rest, p.inlines); + } else { + p.mark_del = Some(Box::new(mark.clone())); + self.finish_para(p); + } + } + self.para = Some(rest); + } + fn end(&mut self, q: &str) { self.depth = self.depth.saturating_sub(1); if q == "office:text" { + if self.del.is_some() { + // Unclosed deleted content (hostile): drop it. + self.close_deletion(); + } + self.in_tc = false; + self.region = None; + self.info_at = None; self.end_para(); self.in_text = false; return; @@ -1369,6 +1911,40 @@ impl Body<'_> { if !self.in_text { return; } + if let Some(d) = self.info_at { + if self.depth < d { + self.info_at = None; + } + self.info_field = None; + return; + } + if self.in_tc { + match local(q) { + "deletion" if self.del.is_some() => { + self.close_deletion(); + return; + } + "changed-region" => { + // A deletion left open (hostile) ends with its region. + self.close_deletion(); + if let Some((id, r)) = self.region.take() + && r.kind.is_some() + { + // The first region with an id wins. + self.regions.entry(id).or_insert(r); + } + return; + } + "tracked-changes" => { + self.close_deletion(); + self.in_tc = false; + self.region = None; + return; + } + _ if self.del.is_some() => {} + _ => return, + } + } match local(q) { "p" | "h" if !self.in_desc => self.end_para(), "span" | "a" if self.fmt.len() > usize::from(self.para.is_some()) => { @@ -1447,6 +2023,24 @@ impl Body<'_> { } } +/// A `text:change-start` (`start`) or `text:change-end` (attributes `a`) among the insertions +/// `open`: unknown ids, ends without a start and starts past [`MAX_OPEN_INSERTIONS`] are ignored. +fn change_mark(regions: &HashMap, open: &mut Vec<(String, RevMark)>, start: bool, a: &Attrs) { + let Some(id) = get(a, "text:change-id") else { return }; + if !start { + if let Some(k) = open.iter().position(|(o, _)| o == id) { + open.remove(k); + } + return; + } + if let Some(r) = regions.get(id).filter(|r| r.kind == Some(RevisionKind::Insert)) + && open.len() < MAX_OPEN_INSERTIONS + && !open.iter().any(|(o, _)| o == id) + { + open.push((id.to_string(), r.mark())); + } +} + fn read_entry(z: &mut zip::ZipArchive>, name: &str) -> Option> { let f = z.by_name(name).ok()?; if f.size() > MAX_ENTRY { @@ -1490,6 +2084,11 @@ pub fn parse(bytes: &[u8]) -> Result { /// [`parse`], with the last section's properties (the others are on the paragraphs that end them). fn parse_full(bytes: &[u8]) -> Result<(Flow, SectionProps), String> { + parse_tracked(bytes).map(|(f, last, _)| (f, last)) +} + +/// [`parse_full`], plus whether change tracking is on (`None`: the document has no tracked changes). +fn parse_tracked(bytes: &[u8]) -> Result<(Flow, SectionProps, Option), String> { let mut z = zip::ZipArchive::new(Cursor::new(bytes)).map_err(|e| format!("ODT: not a valid package ({e})"))?; if let Some(mt) = read_entry(&mut z, "mimetype") { let mt = String::from_utf8_lossy(&mt); @@ -1536,6 +2135,15 @@ fn parse_full(bytes: &[u8]) -> Result<(Flow, SectionProps), String> { sec_from: 0, sections: 0, cols_at: None, + in_tc: false, + track: None, + regions: HashMap::new(), + region_count: 0, + region: None, + info_at: None, + info_field: None, + del: None, + active_ins: Vec::new(), }; events(&content, |ev| match ev { Ev::Start(q, a) => { @@ -1577,14 +2185,18 @@ fn parse_full(bytes: &[u8]) -> Result<(Flow, SectionProps), String> { if blocks.is_empty() { blocks.push(FBlock::Para(Para::default())); } - Ok((Flow { blocks, meta }, last)) + let track = body.track; + Ok((Flow { blocks, meta }, last, track)) } /// Parse an `.odt` package into a document. pub fn import(bytes: &[u8]) -> Result { - let (flow, last) = parse_full(bytes)?; + let (flow, last, track) = parse_tracked(bytes)?; let mut doc = model::to_doc(&flow); doc.last_section = last; + if let Some(t) = track { + doc.settings.track_changes = t; + } Ok(doc) } @@ -1952,4 +2564,210 @@ mod tests { // A one-column page layout keeps the default column gap. assert_eq!(secs[0].1.columns, wordcraft_doc::section::Columns::default()); } + + // --- Tracked changes (ODF 1.3 Part 3 §5.5) ------------------------------------------------- + + const NS_TC: &str = r#"xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" xmlns:dc="http://purl.org/dc/elements/1.1/""#; + + /// An `.odt` whose body holds `regions` in `text:tracked-changes` and then `body`. + fn tracked(regions: &str, body: &str) -> Vec { + let content = format!( + r#"{regions}{body}"# + ); + package(&content, None) + } + + fn region(id: &str, kind: &str, author: &str, date: &str, content: &str) -> String { + format!( + r#"{author}{date}a remark on the change{content}"# + ) + } + + /// Each body paragraph as (text, revision kind) runs, plus its mark's revision kind. + type RevRuns = Vec<(Vec<(String, Option)>, Option)>; + + fn rev_runs(doc: &Document) -> RevRuns { + let kind = |i: Option| i.and_then(|i| doc.revisions.get(i as usize)).map(|r| r.kind); + doc.body + .iter() + .filter_map(|b| if let wordcraft_doc::Block::Para(p) = &**b { Some(p) } else { None }) + .map(|p| { + let runs = p.run_ranges().map(|(r, cp)| (p.text[r].to_string(), kind(cp.del).or(kind(cp.ins)))).collect(); + (runs, kind(p.mark.del).or(kind(p.mark.ins))) + }) + .collect() + } + + fn insertion_and_deletion() -> Vec { + let regions = region("ct1", "insertion", "Alice", "2024-03-01T10:00:00", "") + + ®ion("ct2", "deletion", "Bob", "2024-03-02T11:30:00", "old"); + tracked( + ®ions, + r#"Keep new words."#, + ) + } + + #[test] + fn tracked_insertion_and_deletion_import_with_authors_and_dates() { + use wordcraft_doc::RevisionKind::{Delete, Insert}; + let doc = import(&insertion_and_deletion()).unwrap(); + let s = |v: &str| v.to_string(); + assert_eq!(rev_runs(&doc), vec![(vec![(s("Keep "), None), (s("new "), Some(Insert)), (s("old "), Some(Delete)), (s("words."), None)], None)]); + let revs: Vec<_> = doc.revisions.iter().map(|r| (r.kind, r.author.as_str(), r.date.as_str())).collect(); + assert_eq!(revs, vec![(Insert, "Alice", "2024-03-01T10:00:00"), (Delete, "Bob", "2024-03-02T11:30:00")]); + assert!(doc.settings.track_changes, "text:track-changes=\"true\" turns tracking on"); + } + + #[test] + fn tracked_deletion_across_paragraphs_keeps_the_deleted_paragraph_mark() { + use wordcraft_doc::RevisionKind::Delete; + // "First tail¶head rest." with "tail¶head " deleted: the body holds the merged paragraph. + let regions = region("d1", "deletion", "Bob", "2024-03-02T11:30:00", "tailhead"); + let doc = import(&tracked(®ions, r#"First rest."#)).unwrap(); + let s = |v: &str| v.to_string(); + assert_eq!( + rev_runs(&doc), + vec![(vec![(s("First "), None), (s("tail"), Some(Delete))], Some(Delete)), (vec![(s("head "), Some(Delete)), (s("rest."), None)], None),] + ); + } + + #[test] + fn tracked_changes_round_trip_through_export() { + let regions = region("ct1", "insertion", "Alice", "2024-03-01T10:00:00", "") + + ®ion("d1", "deletion", "Bob", "2024-03-02T11:30:00", "tailhead"); + let body = r#"First rest addedwhole newEnd."#; + let doc = import(&tracked(®ions, body)).unwrap(); + let back = import(&export(&doc).unwrap()).unwrap(); + assert_eq!(rev_runs(&back), rev_runs(&doc)); + let revs = |d: &Document| d.revisions.iter().map(|r| (r.kind, r.author.clone(), r.date.clone())).collect::>(); + assert_eq!(revs(&back), revs(&doc)); + assert_eq!(back.settings.track_changes, doc.settings.track_changes); + // The inserted paragraph mark survives: "added¶" is part of the insertion. + assert_eq!(rev_runs(&doc)[2].1, Some(wordcraft_doc::RevisionKind::Insert)); + } + + #[test] + fn hostile_tracked_changes_are_bounded() { + // Unknown and duplicate ids, a change-end without a start, nested regions, a deletion + // inside a deletion, a change mark inside the deleted text and tracked changes inside a paragraph. + let regions = region("d1", "deletion", "B", "x", r#"gone"#) + + r#"inner"# + + ®ion("dd", "deletion", "B", "x", "deeperok") + + ®ion("d1", "insertion", "C", "y", ""); + let body = r#"abc"#; + let doc = import(&tracked(®ions, body)).unwrap(); + let text: String = rev_runs(&doc).iter().flat_map(|(r, _)| r.iter().map(|(t, _)| t.clone())).collect(); + assert_eq!(text.matches("gone").count(), 1, "a deletion is placed once: {text}"); + assert!(!text.contains("deeper"), "{text}"); + assert!(text.starts_with('a') && text.ends_with('c'), "{text}"); + + // A huge number of regions is capped. + let many: String = (0..20_000).map(|i| format!(r#""#)).collect(); + let starts: String = (0..20_000).map(|i| format!(r#"x"#)).collect(); + let doc = import(&tracked(&many, &format!("{starts}"))).unwrap(); + assert!(doc.revisions.len() <= 10_000, "{}", doc.revisions.len()); + } + + /// Every paragraph (body and table cells, in order) as (text, inserted, deleted) runs plus + /// its mark's (inserted, deleted). + type Marks = Vec<(Vec<(String, bool, bool)>, (bool, bool))>; + + fn marks(doc: &Document) -> Marks { + fn walk(bl: &wordcraft_doc::Blocks, out: &mut Marks) { + for b in bl { + match &**b { + wordcraft_doc::Block::Para(p) => out.push(( + p.run_ranges().map(|(r, cp)| (p.text[r].to_string(), cp.ins.is_some(), cp.del.is_some())).collect(), + (p.mark.ins.is_some(), p.mark.del.is_some()), + )), + wordcraft_doc::Block::Table(t) => { + for r in &t.rows { + for c in &r.cells { + walk(&c.blocks, out); + } + } + } + } + } + } + let mut out = Vec::new(); + walk(&doc.body, &mut out); + out + } + + #[test] + fn inserted_then_deleted_text_keeps_both_marks() { + // "new" was inserted by Alice, then Bob deleted "old new": the deletion's content holds + // the insertion as a change range. + let regions = region("i1", "insertion", "Alice", "2024-03-01T10:00:00", "") + + ®ion( + "d1", + "deletion", + "Bob", + "2024-03-02T11:30:00", + r#"old new"#, + ); + let doc = import(&tracked(®ions, r#"Keep end."#)).unwrap(); + let s = |v: &str| v.to_string(); + let want = + vec![(vec![(s("Keep "), false, false), (s("old "), false, true), (s("new"), true, true), (s("end."), false, false)], (false, false))]; + assert_eq!(marks(&doc), want); + let back = import(&export(&doc).unwrap()).unwrap(); + assert_eq!(marks(&back), want, "both marks survive a save"); + assert_eq!(back.revisions.len(), doc.revisions.len(), "one deletion stays one region and one revision: {:?}", back.revisions); + } + + #[test] + fn a_deleted_mark_inside_an_insertion_is_not_inserted() { + // "A" inserted by Alice, then a paragraph break Bob deleted, then "C" inserted by Alice: + // the insertion range runs across the deletion point, but the deleted mark wasn't inserted. + let regions = region("i1", "insertion", "Alice", "2024-03-01T10:00:00", "") + + ®ion("d1", "deletion", "Bob", "2024-03-02T11:30:00", ""); + let body = + r#"AC"#; + let doc = import(&tracked(®ions, body)).unwrap(); + let s = |v: &str| v.to_string(); + let want = vec![(vec![(s("A"), true, false)], (false, true)), (vec![(s("C"), true, false)], (false, false))]; + assert_eq!(marks(&doc), want); + let back = import(&export(&doc).unwrap()).unwrap(); + assert_eq!(marks(&back), want, "after a save"); + } + + #[test] + fn insertions_end_before_a_table_and_inside_a_cell() { + let regions = region("i1", "insertion", "Alice", "2024-03-01T10:00:00", "") + ®ion("i2", "insertion", "Bob", "2024-03-02T10:00:00", ""); + let table = r#"Cell added"#; + let body = format!( + r#"StartNew paragraph{table}End."# + ); + let content = format!( + r#"{regions}{body}"# + ); + let doc = import(&package(&content, None)).unwrap(); + let s = |v: &str| v.to_string(); + let want = vec![ + (vec![(s("Start"), false, false)], (false, false)), + (vec![(s("New paragraph"), true, false)], (true, false)), + (vec![(s("Cell "), false, false), (s("added"), true, false)], (true, false)), + (vec![(s("End."), false, false)], (false, false)), + ]; + assert_eq!(marks(&doc), want); + let back = import(&export(&doc).unwrap()).unwrap(); + assert_eq!(marks(&back), want); + let authors = |d: &Document| d.revisions.iter().map(|r| r.author.clone()).collect::>(); + assert_eq!(authors(&back), authors(&doc)); + } + + #[test] + fn exported_changes_always_have_a_date_and_keep_paragraph_separators() { + // A change without a date gets one on save (`dc:date` is required); a U+2029 in deleted + // text is text, not a paragraph break. + let regions = r#"Boba
b"#; + let doc = import(&tracked(regions, r#"xy"#)).unwrap(); + assert_eq!(doc.revisions.first().map(|r| r.date.as_str()), Some("")); + let back = import(&export(&doc).unwrap()).unwrap(); + assert!(back.revisions.first().is_some_and(|r| r.date.len() >= 19 && r.date.starts_with("20")), "{:?}", back.revisions); + assert_eq!(marks(&back), marks(&doc)); + assert_eq!(marks(&back).len(), 1); + } } diff --git a/crates/formats/src/rtf.rs b/crates/formats/src/rtf.rs index d5ac7db4f..53dc89e04 100644 --- a/crates/formats/src/rtf.rs +++ b/crates/formats/src/rtf.rs @@ -1066,7 +1066,7 @@ impl Reader { /// the table is "Unknown". fn rev_mark(&self, i: i32, dttm: u32) -> model::RevMark { let author = usize::try_from(i).ok().and_then(|i| self.rev_authors.get(i)).cloned().unwrap_or_else(|| "Unknown".to_string()); - model::RevMark { author, date: dttm_to_iso(dttm).unwrap_or_default() } + model::RevMark { author, date: dttm_to_iso(dttm).unwrap_or_default(), id: None } } fn commit_font(&mut self) { diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 5272480c1..ba5d774b4 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -1,6 +1,6 @@ # File-format parity with Microsoft Word -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** trivial (All Caps and Small Caps text extracts from PDFs as the capitals shown, #1473; previously trivial: RTF tracked insertions and deletions read and written, gap 19; previously trivial: ODT sections on import, #1379; previously trivial: PDF export: Hebrew and Arabic text extracts as whole words, #1383; previously trivial: HTML header cells bold, cell `text-align` kept, and HTML/Markdown/text open on the region's paper, #1381; previously trivial: `.doc` table cell shading and the extra column, #1380; previously trivial: `.doc` tables from Word 2000+: cell shading, no extra column, merged rows, #1380; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: equations as MathML in HTML export, copy and paste, #816; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** trivial (All Caps and Small Caps text extracts from PDFs as the capitals shown, #1473; previously trivial: ODT tracked changes read and written, gap 19; previously trivial: RTF tracked insertions and deletions read and written, gap 19; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats Every format Word opens or saves, what WordCraft does with it, and how it's tested. Fidelity is the share of a typical real-world file's content and formatting that survives (read: shows correctly; @@ -26,7 +26,7 @@ Weights are the share of a Word user's file traffic (estimate). Read/write: ✅ | `.doc` Word 97-2003 (and `.dot`) | 8% | R/W | 🟡 | ❌ | ~60% | 25–40 | `crates/docbin/src/tests.rs`, `tests/drawing.rs` (56) | Spec-based reader ([MS-DOC]): text, formatting, styles, sections, headers/footers, tables, lists, notes, fields, bookmarks; inline and floating pictures (PNG, JPEG, DIB, TIFF, EMF, WMF; Macintosh PICT as a sized placeholder) with crop, alt text, position and wrap; basic shapes, lines, groups and text boxes ([MS-ODRAW]). Table cell shading from every SHD record (`sprmTDefTableShd`, `sprmTDefTableShdRaw` and their 2nd/3rd ranges, with the Word 97 `Shd80` as fallback; a clear pattern shows its background colour); the row-end mark is not a cell, and each table has its own grid and properties (#1380). Horizontally merged rows stored as fewer, wider cells come with #1123. Freeform shapes, z-order and Word 6/95 or encrypted files are dropped or refused. No writer | | PDF open (PDF Reflow, Word for Windows) | 1% | R | ❌ | — | 0% | 30–50 | — | Converting a PDF into an editable document. A sibling app (PdfCraft) may supply the parser | | `.rtf` Rich Text Format | 3% | R/W | 🟡 | 🟡 | ~50% | 10–15 | `crates/formats/src/tests.rs` | ~1,500 lines: text, character/paragraph formatting, styles, lists, tables, pictures (`\shppict`, #154), fields, bookmarks. 8-bit text in the document's code page (`\ansicpg`) or the current font's (`\fcharset`, `\cpg`): Cyrillic, Central European, Greek, Turkish, Hebrew, Arabic, Baltic, Vietnamese, Thai (was Windows-1252 only). Pasted from the system clipboard where the platform offers it (Windows, Wayland) when there is no HTML (#687). Tracked insertions and deletions read and written (`\revtbl`, `\revised`/`\deleted`, authors, `\revdttm` dates), so Accept/Reject All work after opening or a round trip; paragraph-mark and formatting revisions not yet. Sections on import (#1378): each `\sect` ends a section with its own page size and orientation, margins, header/footer distance, columns, break type, page-number restart, start and format, title page and headers/footers (those it leaves out link to the previous); export still writes one section. Thin: comments, notes, section export, RTL | -| `.odt` OpenDocument Text | 2% | R/W | 🟡 | 🟡 | ~50% | 8–12 | `crates/formats/src/tests.rs`, `crates/formats/src/odt.rs` unit tests, `crates/engine/tests/odt_sections.rs` | Styles, lists (#87), tables, pictures, headers/footers, page breaks (#86), page size, orientation and margins (#810), sections: one per master-page change on a paragraph or table (next page, with its own page size, orientation, margins, columns and page-numbering restart) and per columned `text:section` (continuous, with its columns and separator line) (#1379). No footnotes, comments, tracked changes, fields or RTL | +| `.odt` OpenDocument Text | 2% | R/W | 🟡 | 🟡 | ~50% | 8–12 | `crates/formats/src/tests.rs`, `crates/formats/src/odt.rs` unit tests, `crates/engine/tests/odt_sections.rs` | Styles, lists (#87), tables, pictures, headers/footers, page breaks (#86), page size, orientation and margins (#810), sections: one per master-page change on a paragraph or table (next page, with its own page size, orientation, margins, columns and page-numbering restart) and per columned `text:section` (continuous, with its columns and separator line) (#1379); tracked insertions and deletions with author and date, inserted and deleted paragraph marks and text inserted then deleted included (ODF change tracking; format changes read as none; pictures and tables inside a tracked deletion are dropped). No footnotes, comments, fields or RTL | | `.txt` Plain text | 2% | R/W | ✅ | ✅ | ~95% | 1–2 | unit tests | Encoding detection. Files with other extensions (`.m3u`/`.m3u8` playlists, logs, config files, no extension) open as text when they are text, Word and RTF files by their signature; binary files are refused (#569). A Word or RTF extension that contradicts the content also opens by signature, as in Word: RTF saved as `.doc` (hh.ru résumés, accounting exports), a `.docx` renamed `.doc`, a Word 97-2003 file renamed `.docx`. Word's encoding choice dialog and line-ending options missing; such files save only through Save As | | `.htm/.html` Web Page and Web Page, Filtered | 1% | R/W | 🟡 | 🟡 | ~60% | 6–10 | `crates/formats/src/tests.rs` | Tables (#102), relative pictures embedded (#113), missing `` tolerated (#154). Header cells (``) bold unless their CSS says otherwise; `text-align` on a cell, or inherited from its row, row group or table, aligns the cell's paragraphs (#1381). HTML gives no page size, so it opens on the region's paper (A4 or Letter), as Markdown and plain text do; `@page` isn't read (#1381). No Word "Web Page" (with `_files` folder) round-trip. The same reader takes HTML from the system clipboard: Paste keeps another program's formatting (a Word document, a web page; Windows' "HTML Format" header understood), Merge Formatting keeps its emphasis (#687). Equations are written as MathML (with the linear format as a UnicodeMath annotation) and read back from MathML: our annotation, else the presentation elements, else a TeX annotation (#816) | | `.mht/.mhtml` Single File Web Page | <1% | R/W | ❌ | ❌ | 0% | 3–5 | — | | @@ -170,6 +170,7 @@ raise readiness estimates. No Word-derived files are fixtures. | 2026-10-12 | trivial | Direct run-level `w:contentPart` InkML keeps its relationship, content type and bytes through DOCX edits and repeated saves; nested group/canvas ink and native rendering remain missing | | 2026-10-13 | trivial | HTML export writes equations as MathML instead of linear text, and HTML import (open and paste) reads `` back into equations; a copy holding an equation puts that HTML on the system clipboard: MathML in HTML on the clipboard (Word/LibreOffice paste not yet verified; OMML is #1297) (#816) | | 2026-10-14 | trivial | PDF export: All Caps and Small Caps text extracts in capitals, as displayed, and no longer turns later capitals in uncapitalised text lowercase (#1473) | +| 2026-10-14 | trivial | ODT tracked changes: `text:tracked-changes` insertions and deletions open as revisions (author, date, inserted and deleted paragraph marks, insertions nested in deletions) and are written back, a missing date as the save time; `text:track-changes` maps to Track Changes on/off. Pictures, other objects and tables inside a tracked deletion are dropped | | 2026-10-14 | trivial | RTF reads and writes tracked insertions and deletions with their authors and dates (gap 19); deleted text no longer opens as normal text | | 2026-10-14 | trivial | ODT import reads sections: a paragraph naming another master page starts a new-page section with that page's size, orientation, margins and columns, and its `style:page-number` restart; a columned `text:section` is a continuous section with its columns (#1379) | | 2026-10-14 | trivial | PDF export writes right-to-left runs' glyphs in visual order, so Chrome, Edge (PDFium) and poppler extract Hebrew and Arabic as whole words (vowelled words keep their letters in order but may extract with spaces inside), and Ctrl+F finds them (#1383). `/ReversedChars` (for strict file-order readers) covers plain runs and vowelled runs whose marks split into their own glyphs, not runs with ligatures or mixed directions | diff --git a/docs/gaps.md b/docs/gaps.md index fe2ed20f6..be055dfa3 100644 --- a/docs/gaps.md +++ b/docs/gaps.md @@ -1,6 +1,6 @@ # Where WordCraft falls short of Word -> **Last reviewed:** 2026-10-14 · **Last updated:** 2026-10-14 · **Change:** trivial (RTF tracked insertions and deletions, gap 19; previously trivial: Ink to Shape recognises arrows; previously trivial: sample-text formulas on Enter, #1067; previously trivial: #29: copied equations as MathML, OMML open as #1297, #816; previously trivial: the Font list no longer keeps every previewed font in memory, #848; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: beta distance restated: ~15 points, ~150–230 h; previously: first version; every known shortfall from the 2026-10-10 re-measure, ranked) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** trivial (ODT tracked insertions and deletions, gap 19; previously trivial: RTF tracked insertions and deletions, gap 19; previously trivial: the Font list no longer keeps every previewed font in memory, #848; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: beta distance restated: ~15 points, ~150–230 h; previously: first version; every known shortfall from the 2026-10-10 re-measure, ranked) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 > **Printing update:** 2026-10-12 · direct desktop printing through craft-print; details below. @@ -33,7 +33,7 @@ Native save reliability: synthetic cases on macOS reproduced loss through the pr | 16 | **Right-to-left completeness**: kashida, RTL in HTML/ODT/RTF; eight older RTL bug reports to re-verify after #207 (#215, #211, #199, #66, #63, #48, #19). RTL tables and sections landed (#362); PDF export's Hebrew and Arabic text copies and searches as whole words (#1383) | | [layout-parity.md](layout-parity.md) | Arabic, Persian, Hebrew writers | 7–11 | [layout-parity.md](layout-parity.md) | | 17 | **East Asian typography**: Asian Typography options (Phonetic Guide landed, #288; Enclose Characters landed, #297; document grid landed, #391; vertical text landed, #491) | | Catalog misses `para.asianTypography` | Chinese, Japanese and Korean documents | 11–19 | [layout-parity.md](layout-parity.md) | | 18 | **Screen-reader access** to the document canvas: exposed, not yet tried with real screen readers | | The canvas describes paragraphs, headings, lists, tables, links, pictures and the caret to AccessKit (#488), tested headlessly; no NVDA, VoiceOver or Orca run yet | Blind and low-vision users; public-sector procurement | 3–6 | [ui-parity.md](ui-parity.md) | -| 19 | **RTF and ODT depth**: notes, comments, revisions (RTF insertions and deletions done), sections (RTF import done, #1378; RTF export and ODT left), RTL. ODT sections landed on import (#1379): page-style changes and columned `text:section`s | | ~1,500 lines each in `crates/formats` | Users exchanging with LibreOffice or older tools | 18–27 | [file-format-parity.md](file-format-parity.md) | +| 19 | **RTF and ODT depth**: notes, comments, revisions (RTF and ODT insertions and deletions done), sections (RTF import done, #1378; RTF export and ODT left), RTL. ODT sections landed on import (#1379): page-style changes and columned `text:section`s | | ~1,500 lines each in `crates/formats` | Users exchanging with LibreOffice or older tools | 18–27 | [file-format-parity.md](file-format-parity.md) | | 20 | **Performance on large real-world documents** unmeasured (500+ pages, many pictures) | | Only the 188-page sample benchmark; scrolling a 1,096-page one-table document went from ~600 ms to ~2–5 ms a frame (#811), but opening it still lays out for seconds | Thesis and book authors | 10–20 | [layout-parity.md](layout-parity.md) | | 21 | **Mail merge data sources**: no Outlook/Contacts, Access or database sources, no email merge; CSV, typed lists (#247) and Excel/OpenDocument spreadsheets (#334) only | | Select Recipients reads .csv/.tsv, .xlsx/.xlsm and .ods | Office administrators | 4–8 | [target-app-parity.md](target-app-parity.md) | | 22 | **Ecosystem**: no VBA execution (macros preserved only), no Office add-ins, no cloud storage, no real-time co-authoring | | `.docm` keeps macros (#172) | Enterprise users with macro templates; teams | 40–80 + owner decisions | [target-app-parity.md](target-app-parity.md) | @@ -84,6 +84,7 @@ The same gaps grouped the way the parity documents are, for agents working in on | 2026-10-12 | trivial | Sample-text formulas on Enter (#1067) implemented; numerical limits not compared in Word | | 2026-10-12 | minor | Italian proofing landed (it-IT spelling from the installed Hunspell dictionary, accent-first suggestions, conservative grammar); gap #8 narrowed to the remaining languages | | 2026-10-12 | trivial | #2 narrowed: direct run-level `w:contentPart` InkML survives DOCX edits and repeated saves; nested group/canvas ink and native rendering remain | +| 2026-10-14 | trivial | Gap 19: ODT tracked insertions and deletions (with paragraph marks, and text inserted then deleted) read and written | | 2026-10-14 | trivial | Gap 19: RTF tracked insertions and deletions read and written; RTF paragraph-mark and formatting revisions, and ODT revisions, remain | | 2026-10-14 | trivial | #19 narrowed: ODT import keeps sections (master-page changes, columned `text:section`s, page-numbering restarts) (#1379) | | 2026-10-14 | trivial | #16: PDF export writes right-to-left glyphs in visual order, so Hebrew and Arabic extract, copy and search as whole words (#1383) |