diff --git a/Cargo.lock b/Cargo.lock index 7f9a0d3e5..cd0ff683a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -5193,6 +5193,7 @@ version = "0.9.0" name = "wordcraft-doc" version = "0.9.0" dependencies = [ + "base64", "proptest", "serde", "serde_json", diff --git a/ROADMAP.md b/ROADMAP.md index 9cb35f9e9..bb7789964 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~15 points (60% → 75% ready for real work) and ~150–230 h away -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-21 · **Change:** trivial (personal templates in File › New and Save as Word Template, #2852; previously trivial: progress log: legacy menu keys, #1697; previously trivial: progress log: shape gradients and transparency, #1500; previously trivial: progress log: Focus fills the screen, and Full Screen View shows the pages alone, #1678; previously trivial: progress log: text above a float's anchor wraps around it, #2048; previously trivial: progress log: File › Recover follows the app theme, #3243; previously trivial: progress log: caret inside ligatures, #1974; previously trivial: text typed into a content control takes the control's formatting; localized placeholder styles recognised; previously trivial: forms protection keeps its content controls: no inserting, Properties or Remove Content Control; previously trivial: long navigation headings wrap onto further lines within the pane, #3015; previously trivial: clearing a no-match font query keeps the recent heading visible, #2068; previously trivial: a word after a slash wraps whole, like Word, #2963; previously trivial: progress log: a header-cell picture watermark sits behind the text, #1775; previously trivial: symbols keep their font when the text font changes, #2036; previously trivial: opening a DOCX no longer copies its pictures; previously trivial: Add Font imports every selected font file, #1789); previously trivial (RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: Read Aloud speaks each sentence in its own language, #1843; previously trivial: hint-reliant CJK outlines; previously trivial: Ctrl+wheel zoom no longer scrolls the document, #1743; previously trivial: progress log: Turkish case mapping in AutoCorrect, #1923; previously trivial: a mouse drag on a table leaves the view where it is, #1738; previously trivial: progress log: Hebrew field results and smart quotes, #1670; previously trivial: Backspace removes a selected table, rows or columns, #1844; previously trivial: progress log: mixed Hebrew/Latin text keeps its order in Word, #1543; previously trivial: progress log: web typing in non-Latin text no longer repeats, #1480; previously trivial: progress log: font family caches bounded and math font lookups follow font changes, #1354; previously trivial: `-->` AutoCorrect replacement, #2223; previously trivial: numbered headings, #1793; previously trivial: Navigation Find exposes Results, #2096; previously trivial: progress log: Columns on selected text, #2024; previously trivial: progress log: legacy outline, shadow, emboss and imprint text effects drawn, #2534; previously trivial: progress log: Indonesian proofing, #498; previously trivial: progress log: DOCX glossary documents (building blocks); previously trivial: progress log: Read Mode keeps the keyboard and shows whole, unobscured columns, #1772); previously trivial: progress log: font list entries of tall-ascent fonts no longer cut off, #2570); previously trivial: progress log: Split Cells and Convert Text to Table dialogs follow the Word check, #2592; previously trivial: a new source's Tag is made like Word's, three letters of the author and two digits of the year with a number added when taken, and an edit keeps the Tag, #2094; previously trivial: Web Layout fills the window width, #1876; previously trivial: progress log: Different First Page headers reachable from page 1, #2826; previously trivial: Language dialog keeps inherited proofing language and no-proof choices (#2717); previously trivial: progress log: web preferences kept between visits, #2132; previously trivial: progress log: undrawable private-use list bullets show the level's default bullet, #1683; previously trivial: progress log: `w:eastAsianLayout` kept on save; previously trivial: table style editor formats every part, #714); previously trivial: VML freeforms read and kept on save; previously trivial: progress log: consistent preferred cell widths; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: page-break-only list paragraphs hide their labels; previously trivial: progress log: grouped `flowChartOr` shapes draw their circle and crossing spokes; previously trivial: progress log: copies keep their formatting in other programs, #1523; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-21 · **Change:** trivial (Bullets and Numbering toggle off style-inherited lists at their effective level, including basedOn chains and numbered ListParagraph styles; other paragraphs and list definitions stay unchanged (related to #3459); previously trivial: Russian sorting keeps ё with е and recognises decomposed ё/й; previously trivial: progress log: a text box that keeps the keyboard no longer resets the input method every frame, #2986; previously trivial: progress log: recovery copies and .json files store pictures as base64, #586 follow-up; previously trivial: personal templates in File › New and Save as Word Template, #2852; previously trivial: progress log: legacy menu keys, #1697; previously trivial: progress log: shape gradients and transparency, #1500; previously trivial: progress log: Focus fills the screen, and Full Screen View shows the pages alone, #1678; previously trivial: progress log: text above a float's anchor wraps around it, #2048; previously trivial: progress log: File › Recover follows the app theme, #3243; previously trivial: progress log: caret inside ligatures, #1974; previously trivial: text typed into a content control takes the control's formatting; localized placeholder styles recognised; previously trivial: forms protection keeps its content controls: no inserting, Properties or Remove Content Control; previously trivial: long navigation headings wrap onto further lines within the pane, #3015; previously trivial: clearing a no-match font query keeps the recent heading visible, #2068; previously trivial: a word after a slash wraps whole, like Word, #2963; previously trivial: progress log: a header-cell picture watermark sits behind the text, #1775; previously trivial: symbols keep their font when the text font changes, #2036; previously trivial: opening a DOCX no longer copies its pictures; previously trivial: Add Font imports every selected font file, #1789); previously trivial (RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: Read Aloud speaks each sentence in its own language, #1843; previously trivial: hint-reliant CJK outlines; previously trivial: Ctrl+wheel zoom no longer scrolls the document, #1743; previously trivial: progress log: Turkish case mapping in AutoCorrect, #1923; previously trivial: a mouse drag on a table leaves the view where it is, #1738; previously trivial: progress log: Hebrew field results and smart quotes, #1670; previously trivial: Backspace removes a selected table, rows or columns, #1844; previously trivial: progress log: mixed Hebrew/Latin text keeps its order in Word, #1543; previously trivial: progress log: web typing in non-Latin text no longer repeats, #1480; previously trivial: progress log: font family caches bounded and math font lookups follow font changes, #1354; previously trivial: `-->` AutoCorrect replacement, #2223; previously trivial: numbered headings, #1793; previously trivial: Navigation Find exposes Results, #2096; previously trivial: progress log: Columns on selected text, #2024; previously trivial: progress log: legacy outline, shadow, emboss and imprint text effects drawn, #2534; previously trivial: progress log: Indonesian proofing, #498; previously trivial: progress log: DOCX glossary documents (building blocks); previously trivial: progress log: Read Mode keeps the keyboard and shows whole, unobscured columns, #1772); previously trivial: progress log: font list entries of tall-ascent fonts no longer cut off, #2570); previously trivial: progress log: Split Cells and Convert Text to Table dialogs follow the Word check, #2592; previously trivial: a new source's Tag is made like Word's, three letters of the author and two digits of the year with a number added when taken, and an edit keeps the Tag, #2094; previously trivial: Web Layout fills the window width, #1876; previously trivial: progress log: Different First Page headers reachable from page 1, #2826; previously trivial: Language dialog keeps inherited proofing language and no-proof choices (#2717); previously trivial: progress log: web preferences kept between visits, #2132; previously trivial: progress log: undrawable private-use list bullets show the level's default bullet, #1683; previously trivial: progress log: `w:eastAsianLayout` kept on save; previously trivial: table style editor formats every part, #714); previously trivial: VML freeforms read and kept on save; previously trivial: progress log: consistent preferred cell widths; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: page-break-only list paragraphs hide their labels; previously trivial: progress log: grouped `flowChartOr` shapes draw their circle and crossing spokes; previously trivial: progress log: copies keep their formatting in other programs, #1523; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 > **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-18 · **Change:** trivial (English dialects in proofing, #1281; previously trivial: triple borders draw three lines, #1740; previously trivial: progress log: double-clicking a later page's footer no longer scrolls up, #2103; previously trivial: long navigation headings wrap onto further lines within the pane, #3015; previously trivial: clearing a no-match font query keeps the recent heading visible, #2068; previously trivial: a word after a slash wraps whole, like Word, #2963; previously trivial: RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: progress log: pasted blockquotes are indented paragraphs, #1270; previously trivial: progress log: print-area strings translated in every catalog; previously trivial: progress log: comments readable in the dark theme, #2045; previously trivial: progress log: ODT sections, #1379; previously trivial: progress log: table deletes, Change Case and Unlink Fields respect Track Changes and locked content controls, #1329; previously trivial: progress log: rounded rectangle corners, #1476; previously trivial: progress log: Save As type list, #1399; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 > **Printing update:** 2026-10-12 · direct desktop printing through craft-print; details below. @@ -134,6 +134,8 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm ## Progress log +- 2026-10-21: Bullets and Numbering toggle off style-inherited lists at their effective level, including basedOn chains and numbered ListParagraph styles; other paragraphs and list definitions stay unchanged (related to #3459). + Imported Word 97–2003 `.doc`/`.dot` files now ask for a supported DOCX target on interactive Save (#2445). Save before Close waits for that write; cancel or failure keeps the draft and original file. This does not add binary Word output. Late Save As answers are rejected before writing if automation has replaced the document. - 2026-10-21: File › Recover follows the app theme (#3243, part of #2348): no black page or forced white text on the light theme @@ -144,7 +146,10 @@ Imported Word 97–2003 `.doc`/`.dot` files now ask for a supported DOCX target | Date | What landed | | 2026-10-19 | Personal templates (#2852): File › New lists the `.dotx` and `.dotm` files in your templates folder under Personal (a subfolder per category; the search box covers them), and one opens as a new untitled document attached to it, as in Word. Save As › Save as Word Template starts in that folder, made on first use in the settings folder; File › Options › Advanced › File locations › User templates points it elsewhere. Agents list them with `file.listTemplates`. | | 2026-10-15 | A picture anchored further down a page but placed higher up it (fixed on the page, or moved up from its paragraph) has the paragraphs above its anchor wrap around it too, instead of running behind it, also when that wrap pushes the anchor onto the next page (#2048) | +| 2026-10-19 | Typing on Linux no longer stops after clicking Search (#2986): Search, Go To's page box, the Icons search and a new caption label asked for the keyboard on every frame, and each request reset the input method; they now ask only while they lack it. | +| 2026-10-19 | An equation from LibreOffice in an `.odt` file no longer shows blank (#3099): it's embedded as a `draw:object` formula sub-document holding MathML, with a `draw:image` replacement preview in LibreOffice's own metafile format WordCraft can't decode as a picture; the formula's own MathML is read instead, through the same reader as HTML equations. | | 2026-10-19 | Save As, Export and Open no longer do nothing on Linux desktops whose file chooser portal fails to show its dialog (#2999): a "cancel" quicker than anyone could make one asks again without the parent window, then opens the built-in file browser. | +| 2026-10-20 | Russian sorting keeps ё with е and recognises decomposed ё/й: case-insensitive words and index headings share the corrected collation, with regressions and no new dependencies. | | 2026-10-18 | Long navigation headings wrap onto further lines inside the resizable pane instead of ending in an ellipsis: the row grows with the text and the pane keeps its width (#759, #3015). | | 2026-10-18 | Clearing a font query with no matches keeps Recently Used Fonts visible at the top of a scrollable list. The initial scroll position no longer skips its heading; measured keyboard scrolling is unchanged (#2068). | | 2026-10-17 | Edit Recipient List finds records (#1576): a Find in list box hides rows with no cell containing the text (any case) and counts the matches, and "Show only records that will be merged" hides the rows the filter leaves out; both only change the view. Rows are striped, and edits and removes go to the right record however the grid is narrowed | @@ -242,6 +247,7 @@ Imported Word 97–2003 `.doc`/`.dot` files now ask for a supported DOCX target | 2026-10-16 | Text with the legacy Outline, Shadow, Emboss or Engrave font effects (`w:outline`, `w:shadow`, `w:emboss`, `w:imprint`) is drawn with them on screen and in PDFs, no longer as plain text: hollow outlined letters, or a grey copy behind the letters; the PDF keeps the text (#2534) | | 2026-10-15 | Table Layout › Cell Margins opens an editable dialog initialized from the table defaults; confirming an unchanged dialog preserves imported margins, and changing one side retains the others (#1960) | | 2026-10-15 | Word's native ink inside groups and drawing canvases survives DOCX saves (gap 2): the ink member keeps its InkML part, relationship, content type and picture fallback, stays in its place among the members, moves with the group and is drawn from its fallback picture; deleting the group drops its part | +| 2026-10-15 | Crash recovery copies and native .json files store pictures and kept parts as base64 text instead of arrays of numbers (#586 follow-up, #518): with a 27 MiB picture a recovery copy is 36 MiB instead of 96 MiB and a .json file 36 MiB instead of 286 MiB, written and read about five times faster. Copies and files written before still open | | 2026-10-14 | Consistent preferred cell widths override a stale table grid when both totals agree, for fixed and AutoFit tables. Conflicting widths, spans and percentages remain unchanged. | | 2026-10-14 | Legacy VML freeforms (`v:shape` with its own `path`), such as the rule lines in many older headers, are read as freeform shapes: they show on screen and in PDFs and are saved back as DrawingML custom geometry instead of vanishing | | 2026-10-14 | Field instructions spanning paragraphs no longer add blank lines during pagination; empty result paragraphs still take a line. | @@ -518,6 +524,9 @@ Footer page-table height and body wrapping are corrected. Other pagination gaps | Date | Change | Summary | |---|---|---| | 2026-10-21 | trivial | Progress log: File › Recover follows the app theme (#3243) | +| 2026-10-19 | trivial | Progress log: typing no longer stops after clicking Search (#2986) | +| 2026-10-20 | trivial | Russian sorting keeps ё with е and recognises decomposed ё/й; synthetic ordering, case and index-heading regressions; broader parity estimates unchanged | +| 2026-10-21 | trivial | Bullets and Numbering toggle off style-inherited lists at their effective level, including basedOn chains and numbered ListParagraph styles; other paragraphs and list definitions stay unchanged (related to #3459) | | 2026-10-19 | trivial | Progress log: forms protection keeps its content controls: no inserting, Properties or Remove Content Control | | 2026-10-19 | trivial | Progress log: text typed into a content control takes the control's formatting; localized placeholder styles recognised | | 2026-10-17 | trivial | Progress log: #2048 entry covers anchors pushed onto the next page | @@ -600,6 +609,7 @@ Footer page-table height and body wrapping are corrected. Other pagination gaps | 2026-10-17 | minor | Progress log: Scale to paper size on File › Print, #2108 | | 2026-10-17 | trivial | Non-overlapping body floating tables clear header tables; overlapping tables keep their requested position | | 2026-10-15 | trivial | Progress log: caret inside ligatures (#1974) | +| 2026-10-15 | trivial | Progress log: recovery copies and .json files store pictures as base64 (#586 follow-up) | | 2026-10-14 | trivial | Progress log: row table border exceptions | | 2026-10-12 | minor | Czech proofing review: short-word splits rank after deletion, NOSUGGEST excludes suggestions while accepting typed forms, affix comments apply only to headers, ratios keep spaces and repeated-word messages use Czech quotes; synthetic engine end-to-end and latency checks | | 2026-10-12 | minor | Czech spelling from the installed Hunspell dictionary, runtime-table suggestions and generic checks; Czech interface writes cs-CZ documents; no Czech hyphenation patterns bundled; estimates unchanged | diff --git a/crates/doc/Cargo.toml b/crates/doc/Cargo.toml index f0111eb18..9ab62d34d 100644 --- a/crates/doc/Cargo.toml +++ b/crates/doc/Cargo.toml @@ -10,6 +10,7 @@ workspace = true [dependencies] wordcraft-geom = { workspace = true } +base64 = { workspace = true } serde = { workspace = true } serde_json = { workspace = true } thiserror = { workspace = true } diff --git a/crates/doc/src/byte_store.rs b/crates/doc/src/byte_store.rs new file mode 100644 index 000000000..75fae6453 --- /dev/null +++ b/crates/doc/src/byte_store.rs @@ -0,0 +1,130 @@ +//! How [`crate::Document::media`] and [`crate::Document::passthrough`] are written in the native +//! JSON form (`.json` files and crash-recovery copies): each part's bytes as one base64 string. +//! serde's default for bytes, an array of numbers, took about 3.6 times their size compact and +//! over six times pretty-printed (a 27 MB picture made a 99 MB recovery copy, #586). +//! +//! Reading takes either form, so `.json` files and recovery copies written before still open. +use std::collections::BTreeMap; +use std::fmt; +use std::sync::Arc; + +use base64::Engine as _; +use base64::display::Base64Display; +use base64::engine::general_purpose::STANDARD; +use serde::de::{self, Deserializer, MapAccess, SeqAccess, Visitor}; +use serde::ser::{Serialize, Serializer}; + +type Store = BTreeMap>>; + +/// One part's bytes, encoded as it is written out: no second copy of a large picture is made. +struct Encoded<'a>(&'a [u8]); + +impl Serialize for Encoded<'_> { + fn serialize(&self, s: S) -> Result { + s.collect_str(&Base64Display::new(self.0, &STANDARD)) + } +} + +pub(crate) fn serialize(store: &Store, s: S) -> Result { + s.collect_map(store.iter().map(|(name, bytes)| (name, Encoded(bytes)))) +} + +pub(crate) fn deserialize<'de, D: Deserializer<'de>>(d: D) -> Result { + d.deserialize_map(StoreVisitor) +} + +struct StoreVisitor; + +impl<'de> Visitor<'de> for StoreVisitor { + type Value = Store; + + fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result { + f.write_str("a map of part names to base64 text or byte arrays") + } + + fn visit_map>(self, mut map: A) -> Result { + let mut store = Store::new(); + while let Some((name, Bytes(bytes))) = map.next_entry::()? { + store.insert(name, Arc::new(bytes)); + } + Ok(store) + } +} + +/// A part's bytes as either form. +struct Bytes(Vec); + +impl<'de> de::Deserialize<'de> for Bytes { + fn deserialize>(d: D) -> Result { + d.deserialize_any(BytesVisitor).map(Bytes) + } +} + +struct BytesVisitor; + +impl<'de> Visitor<'de> for BytesVisitor { + type Value = Vec; + + fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result { + f.write_str("base64 text or an array of bytes") + } + + fn visit_str(self, text: &str) -> Result, E> { + STANDARD.decode(text).map_err(|e| E::custom(format!("bytes are not valid base64: {e}"))) + } + + fn visit_bytes(self, bytes: &[u8]) -> Result, E> { + Ok(bytes.to_vec()) + } + + /// The form written before: one number per byte. + fn visit_seq>(self, mut seq: A) -> Result, A::Error> { + // The length hint comes from the file: trust it only so far. + let mut out = Vec::with_capacity(seq.size_hint().unwrap_or(0).min(1 << 20)); + while let Some(b) = seq.next_element::()? { + out.push(b); + } + Ok(out) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[derive(serde::Serialize, serde::Deserialize, PartialEq, Debug, Default)] + struct Parts { + #[serde(default, with = "super")] + media: Store, + } + + fn parts(entries: &[(&str, &[u8])]) -> Parts { + Parts { media: entries.iter().map(|(n, b)| ((*n).to_string(), Arc::new(b.to_vec()))).collect() } + } + + #[test] + fn bytes_are_written_as_base64_and_read_back() { + let all: Vec = (0..=255).collect(); + for n in [0, 1, 2, 3, 4, 255, 256] { + let p = parts(&[("a.png", &all[..n.min(all.len())])]); + let text = serde_json::to_string(&p).unwrap(); + assert_eq!(serde_json::from_str::(&text).unwrap(), p, "{n}"); + } + assert_eq!(serde_json::to_string(&parts(&[("m", b"Man"), ("ma", b"Ma")])).unwrap(), r#"{"media":{"m":"TWFu","ma":"TWE="}}"#); + } + + #[test] + fn number_arrays_written_before_still_read() { + let old = r#"{"media":{"a.png":[0,1,127,128,254,255],"empty":[]}}"#; + assert_eq!(serde_json::from_str::(old).unwrap(), parts(&[("a.png", &[0, 1, 127, 128, 254, 255]), ("empty", &[])])); + } + + #[test] + fn hostile_values_are_errors_not_panics() { + for bad in [r#""T""#, r#""TWFu!""#, r#""-_""#, r#""TQ=x""#, "[256]", "[-1]", "[1.5]", "true", "{}", r#"["a"]"#] { + let text = format!(r#"{{"media":{{"a.png":{bad}}}}}"#); + assert!(serde_json::from_str::(&text).is_err(), "{bad}"); + } + assert!(serde_json::from_str::(r#"{"media":"x"}"#).is_err()); + } +} diff --git a/crates/doc/src/collate.rs b/crates/doc/src/collate.rs index db034e8ac..e27b6dada 100644 --- a/crates/doc/src/collate.rs +++ b/crates/doc/src/collate.rs @@ -150,6 +150,8 @@ const HU: &[Tailor] = &[ const SV: &[Tailor] = &[t("å", 'z', 1), t("ä", 'z', 2), ts("æ", 'z', 2, 16), t("ö", 'z', 3), ts("ø", 'z', 3, 11)]; const DA: &[Tailor] = &[t("æ", 'z', 1), ts("ä", 'z', 1, 4), t("ø", 'z', 2), ts("ö", 'z', 2, 4), t("å", 'z', 3)]; const ES: &[Tailor] = &[t("ñ", 'n', 1)]; +// Russian ё shares е's primary weight; the diaeresis breaks ties (CLDR Russian standard). +const RU: &[Tailor] = &[ts("ё", 'е', 0, 4)]; const TR: &[Tailor] = &[t("ç", 'c', 1), t("ğ", 'g', 1), t("ı", 'h', 1), t("ö", 'o', 1), t("ş", 's', 1), t("ü", 'u', 1)]; const HR: &[Tailor] = &[t("č", 'c', 1), t("ć", 'c', 2), t("dž", 'd', 1), t("đ", 'd', 2), t("lj", 'l', 1), t("nj", 'n', 1), t("š", 's', 1), t("ž", 'z', 1)]; @@ -184,6 +186,7 @@ fn tailoring(lang: Option<&str>) -> &'static [Tailor] { "sv" | "fi" => SV, "da" | "nb" | "nn" | "no" => DA, "es" | "gl" => ES, + "ru" => RU, "tr" | "az" => TR, "hr" | "bs" | "sr" => HR, "ro" => RO, @@ -196,7 +199,7 @@ fn tailoring(lang: Option<&str>) -> &'static [Tailor] { } } -/// A lowercase Latin letter followed by a combining mark, and the single letter Unicode +/// A lowercase Latin or Russian letter followed by a combining mark, and the single letter Unicode /// composes them to (NFC; generated from the Unicode Character Database). Text pasted from /// decomposed sources (macOS file names: `z` + U+0307) collates like the composed letter. const COMPOSE: &[(char, char, char)] = &[ @@ -329,6 +332,8 @@ const COMPOSE: &[(char, char, char)] = &[ ('z', '\u{302}', 'ẑ'), ('z', '\u{307}', 'ż'), ('z', '\u{30c}', 'ž'), + ('е', '\u{308}', 'ё'), + ('и', '\u{306}', 'й'), ]; /// `s` lowercased, with each letter + combining mark that has a single-letter form composed. @@ -476,6 +481,25 @@ mod tests { assert_eq!(sorted(&["Straße", "Strasse", "Strand"], "de"), ["Strand", "Strasse", "Straße"]); } + #[test] + fn russian_yo_sorts_with_e_instead_of_after_ya() { + for lang in ["ru", "ru-RU", "RU_ru"] { + assert_eq!(sorted(&["яблоко", "ёлка", "жук", "ель", "ёж", "еж"], lang), ["еж", "ёж", "ёлка", "ель", "жук", "яблоко"]); + assert_eq!(collate("ЁЛКА", "ёлка", Some(lang)), Ordering::Equal); + assert_eq!(heading_letter("Ёлка", Some(lang)).as_deref(), Some("Ё")); + } + } + + #[test] + fn russian_decomposed_letters_match_their_composed_forms() { + for (composed, decomposed) in [("ёлка", "е\u{308}лка"), ("Ёлка", "Е\u{308}лка"), ("йод", "и\u{306}од"), ("Йод", "И\u{306}од")] + { + assert_eq!(collate(composed, decomposed, Some("ru-RU")), Ordering::Equal); + assert_eq!(heading_letter(composed, Some("ru")), heading_letter(decomposed, Some("ru"))); + } + assert_eq!(sorted(&["к", "и\u{306}", "и"], "ru"), ["и", "и\u{306}", "к"]); + } + #[test] fn spanish_n_tilde_after_n() { assert_eq!(sorted(&["oso", "ñu", "nube"], "es-ES"), ["nube", "ñu", "oso"]); diff --git a/crates/doc/src/lib.rs b/crates/doc/src/lib.rs index 04421a5d2..eb0cea8a0 100644 --- a/crates/doc/src/lib.rs +++ b/crates/doc/src/lib.rs @@ -11,6 +11,7 @@ #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] pub mod bidi; +mod byte_store; pub mod chart; mod cjk_numbers; pub mod cluster; @@ -723,11 +724,11 @@ pub struct Document { /// The document's own endnote separators and continuation notice. #[serde(skip_serializing_if = "NoteSeparators::is_empty")] pub endnote_separators: NoteSeparators, - /// Embedded media (images) by key. - #[serde(skip_serializing_if = "BTreeMap::is_empty")] + /// Embedded media (images) by key. Native JSON writes each as base64 (`byte_store.rs`). + #[serde(skip_serializing_if = "BTreeMap::is_empty", with = "byte_store")] pub media: BTreeMap>>, /// Source-format parts we keep verbatim for round-trip (e.g. custom XML), by part name. - #[serde(skip_serializing_if = "BTreeMap::is_empty")] + #[serde(skip_serializing_if = "BTreeMap::is_empty", with = "byte_store")] pub passthrough: BTreeMap>>, /// The document's own building blocks (its glossary document), in file order. #[serde(skip_serializing_if = "Vec::is_empty")] diff --git a/crates/doc/src/numbering.rs b/crates/doc/src/numbering.rs index 07d20db04..d4323cddd 100644 --- a/crates/doc/src/numbering.rs +++ b/crates/doc/src/numbering.rs @@ -288,8 +288,19 @@ impl Numbering { pub struct Counters { /// Per abstract list: current value per level. state: HashMap, - /// Which nums have started (to apply start overrides once). - seen: HashMap, + /// Which levels of which nums have started: a num's start override for a level applies + /// once, when that level of that num is first used (ECMA-376 Part 1, 17.9.27 startOverride). + seen: HashMap<(u32, u8), bool>, + /// Per abstract list: whether each level has counted yet (a value of 0 can be a real count when a + /// level starts at 0). + flags: HashMap, +} + +/// Whether a level has counted yet, and the value its next item takes if a start override says so. +#[derive(Default, Debug, Clone, Copy)] +struct LevelFlags { + started: bool, + first: Option, } impl Counters { @@ -312,29 +323,36 @@ impl Counters { return None; } let st = self.state.entry(abs.id).or_insert([0; 9]); - if let std::collections::hash_map::Entry::Vacant(e) = self.seen.entry(num) { + let fl = self.flags.entry(abs.id).or_default(); + // Only this level's override, not the num's others: the counters are shared by every + // num of the abstract list, so restarting its other levels here would renumber + // paragraphs of the other nums. A heading style can use one num for levels 1-4 and + // another, with start overrides on every level, for 5-9; its first level-5 heading + // must not restart the chapter count. + if let std::collections::hash_map::Entry::Vacant(e) = self.seen.entry((num, lv as u8)) { e.insert(true); - for (l, start) in &n.start_overrides { - if let Some(s) = st.get_mut(*l as usize) { - *s = start.saturating_sub(1); - } + if let Some((_, start)) = n.start_overrides.iter().find(|(l, _)| *l as usize == lv) + && let (Some(s), Some(f)) = (st.get_mut(lv), fl.get_mut(lv)) + { + *s = 0; + *f = LevelFlags { started: false, first: Some(*start) }; } } - if let Some(v) = st.get_mut(lv) { - *v = if *v == 0 { def.start.max(if def.start == 0 { 0 } else { 1 }) } else { v.saturating_add(1) }; - if *v == 0 { - *v = def.start; - } + if let (Some(v), Some(f)) = (st.get_mut(lv), fl.get_mut(lv)) { + *v = if f.started { v.saturating_add(1) } else { f.first.unwrap_or(def.start) }; + *f = LevelFlags { started: true, first: None }; } // Deeper levels restart, unless they never restart or only restart after a level above // this one (Word's lvlRestart). - for (d, deeper) in st.iter_mut().enumerate().skip(lv + 1) { + for (d, (deeper, f)) in st.iter_mut().zip(fl.iter_mut()).enumerate().skip(lv + 1) { let restarts = level_def(d).is_none_or(|l| l.restart && l.restart_after.is_none_or(|k| lv < k as usize)); if restarts { *deeper = 0; + *f = LevelFlags::default(); } } let vals = *st; + let started = *fl; if def.format == NumFormat::Bullet { return Some((def.text.clone(), def)); } @@ -347,7 +365,8 @@ impl Counters { chars.next(); let k = (d as usize).saturating_sub(1).min(8); let val = vals.get(k).copied().unwrap_or(0); - let val = if val == 0 { level_def(k).map(|l| l.start).unwrap_or(1) } else { val }; + let counted = started.get(k).is_some_and(|f| f.started); + let val = if counted { val } else { level_def(k).map(|l| l.start).unwrap_or(1) }; // Legal numbering (w:isLgl) shows every level in Arabic digits, whatever its own // format (二 of a Chinese outline becomes 2 in "2.1"). let fmt = match level_def(k).map(|l| l.format).unwrap_or(NumFormat::Decimal) { @@ -389,6 +408,29 @@ mod tests { assert_eq!(n.level(id, 2).map(|l| l.format), Some(NumFormat::DecimalZero)); } + #[test] + fn a_start_override_restarts_only_its_own_level_of_a_shared_list() { + // One abstract list numbers the headings through two nums: one for levels 1-4, and one + // restarting every level (startOverride 1 on 0..8) for levels 5-9, first used deep in + // chapter 2. Chapter 3 is still chapter 3, and the deep heading counts from 1. + let mut n = Numbering::default(); + let chapters = n.add_list(ListKind::Legal); + let abs = n.abstract_of(chapters).map(|a| a.id).unwrap(); + let deep = chapters + 1; + n.nums.push(Num { id: deep, abstract_id: abs, start_overrides: (0..9).map(|l| (l, 1)).collect(), ..Num::default() }); + let mut c = Counters::default(); + let mut l = |num, lv| c.next_label(&n, num, lv).unwrap().0; + assert_eq!(l(chapters, 0), "1."); + assert_eq!(l(chapters, 0), "2."); + assert_eq!(l(chapters, 1), "2.1."); + assert_eq!(l(chapters, 2), "2.1.1."); + assert_eq!(l(chapters, 3), "2.1.1.1."); + assert_eq!(l(deep, 4), "2.1.1.1.1.", "the deep num restarts only level 5"); + assert_eq!(l(deep, 4), "2.1.1.1.2."); + assert_eq!(l(chapters, 0), "3.", "and not the chapter count"); + assert_eq!(l(chapters, 1), "3.1."); + } + #[test] fn levels_restart_only_after_their_restart_level() { let mut n = Numbering::default(); @@ -411,6 +453,40 @@ mod tests { assert_eq!(l(2), "(1)", "level 1 restarts level 3"); } + #[test] + fn a_level_starting_at_zero_counts_up() { + let mut n = Numbering::default(); + let id = n.add_list(ListKind::Numbered); + if let Some(a) = n.abstracts.first_mut() { + a.levels[0].start = 0; + } + let mut c = Counters::default(); + let labels: Vec = (0..4).map(|_| c.next_label(&n, id, 0).unwrap().0).collect(); + assert_eq!(labels, ["0.", "1.", "2.", "3."]); + } + + #[test] + fn a_start_override_of_one_or_zero_is_honoured() { + let mut n = Numbering::default(); + let id = n.add_list(ListKind::Numbered); + if let Some(a) = n.abstracts.first_mut() { + a.levels[0].start = 5; + } + if let Some(num) = n.nums.iter_mut().find(|x| x.id == id) { + num.start_overrides = vec![(0, 1)]; + } + let mut c = Counters::default(); + let labels: Vec = (0..2).map(|_| c.next_label(&n, id, 0).unwrap().0).collect(); + assert_eq!(labels, ["1.", "2."]); + + if let Some(num) = n.nums.iter_mut().find(|x| x.id == id) { + num.start_overrides = vec![(0, 0)]; + } + let mut c = Counters::default(); + let labels: Vec = (0..3).map(|_| c.next_label(&n, id, 0).unwrap().0).collect(); + assert_eq!(labels, ["0.", "1.", "2."]); + } + #[test] fn numbered_labels() { let mut n = Numbering::default(); diff --git a/crates/doc/src/props.rs b/crates/doc/src/props.rs index 644e4b31e..af71117c6 100644 --- a/crates/doc/src/props.rs +++ b/crates/doc/src/props.rs @@ -47,8 +47,9 @@ impl Rgb { } /// Parse `RRGGBB` or `#RRGGBB`. pub fn parse(s: &str) -> Option { - let s = s.trim().trim_start_matches('#'); - if s.len() != 6 || !s.is_ascii() { + let s = s.trim(); + let s = s.strip_prefix('#').unwrap_or(s); + if s.len() != 6 || !s.bytes().all(|b| b.is_ascii_hexdigit()) { return None; } let p = |i: usize| s.get(i..i + 2).and_then(|h| u8::from_str_radix(h, 16).ok()); @@ -1388,6 +1389,16 @@ mod tests { assert_eq!(Rgb::parse("é1234"), None); } + #[test] + fn rgb_parse_rejects_non_hex() { + assert_eq!(Rgb::parse("+1+2+3"), None); + assert_eq!(Rgb::parse("-1-2-3"), None); + assert_eq!(Rgb::parse("##ff0000"), None); + assert_eq!(Rgb::parse("ff00 0"), None); + assert_eq!(Rgb::parse("#ff0000"), Some(Rgb(255, 0, 0))); + assert_eq!(Rgb::parse("ff0000"), Some(Rgb(255, 0, 0))); + } + #[test] fn overlay_keeps_unset() { let base = CharProps { bold: Some(true), size: Some(12.0), ..Default::default() }; diff --git a/crates/engine/src/cmd/para.rs b/crates/engine/src/cmd/para.rs index 333d6331f..3ce6d739b 100644 --- a/crates/engine/src/cmd/para.rs +++ b/crates/engine/src/cmd/para.rs @@ -510,17 +510,20 @@ fn apply_style(s: &mut Session, v: &Value) -> CmdResult { sel_result(s) } +/// Toggle lists using effective paragraph numbering; an explicit off request removes numbering. fn list(s: &mut Session, v: &Value, default: ListKind) -> CmdResult { let kind = p::str(v, "kind").and_then(ListKind::parse).unwrap_or(default); let off = p::bool(v, "off").unwrap_or(false); let a = s.sel.ordered().0; // Toggle: if every selected paragraph already has this kind of list, remove it. - let current: Vec> = - super::selected_paths(s).iter().map(|p| s.doc.para(a.story, p).and_then(|x| x.props.numbering).filter(|n| n.num != 0)).collect(); + let current: Vec> = super::selected_paths(s) + .iter() + .map(|p| s.doc.para(a.story, p).and_then(|x| s.doc.styles.resolve_para(&x.props).numbering).filter(|n| n.num != 0)) + .collect(); let same_kind = |n: &NumRef| { - s.doc.numbering.level(n.num, 0).is_some_and(|l| { + s.doc.numbering.level(n.num, n.level).is_some_and(|l| { let want = wordcraft_doc::numbering::levels_for(kind); - want.first().is_some_and(|w| { + want.get(usize::from(n.level)).is_some_and(|w| { (w.format == wordcraft_doc::section::NumFormat::Bullet) == (l.format == wordcraft_doc::section::NumFormat::Bullet) && (p::str(v, "kind").is_none() || w.text == l.text) }) @@ -528,13 +531,14 @@ fn list(s: &mut Session, v: &Value, default: ListKind) -> CmdResult { }; let remove = off || (!current.is_empty() && current.iter().all(|n| n.as_ref().is_some_and(same_kind))); if remove { + let keep_list_style = s.doc.styles.resolve_para(&ParaProps { style: Some("ListParagraph".into()), ..Default::default() }).numbering.is_some(); return fmt(s, &|p| { p.numbering = Some(NumRef { num: 0, level: 0 }); p.indent_left = None; p.indent_first = None; p.indent_left_chars = None; p.indent_first_chars = None; - if p.style.as_deref() == Some("ListParagraph") { + if p.style.as_deref() == Some("ListParagraph") && !keep_list_style { p.style = None; } }); @@ -544,7 +548,7 @@ fn list(s: &mut Session, v: &Value, default: ListKind) -> CmdResult { .doc .prev_para(a.story, &a.path) .and_then(|q| s.doc.para(a.story, &q).and_then(|x| x.props.numbering)) - .filter(|n| n.num != 0 && same_kind(n)); + .filter(|n| n.num != 0 && same_kind(&NumRef { level: 0, ..*n })); let num = match prev_num { Some(n) => n.num, None => match kind { diff --git a/crates/engine/src/recovery/desktop.rs b/crates/engine/src/recovery/desktop.rs index 676a6ed0c..ad38ed0da 100644 --- a/crates/engine/src/recovery/desktop.rs +++ b/crates/engine/src/recovery/desktop.rs @@ -706,6 +706,26 @@ mod tests { assert_eq!(Store::files(&t.0.join(id)).unwrap().len(), 1); } + /// Copies are written with picture bytes in base64 (#586 follow-up); those written before kept + /// them as arrays of numbers and are still listed and restored, bytes intact. + #[test] + fn copies_with_byte_arrays_written_before_still_restore() { + let t = Temp::new(); + let id = new_id().unwrap(); + let mut s = snapshot(&id, "Synthetic draft with a picture"); + s.generation = 1; + s.document.media.insert("media/image1.png".into(), Arc::new(vec![0, 1, 127, 128, 254, 255])); + let mut value = serde_json::to_value(&s).unwrap(); + assert_eq!(value["document"]["media"]["media/image1.png"], "AAF/gP7/"); + value["document"]["media"]["media/image1.png"] = json!([0, 1, 127, 128, 254, 255]); + private_dir(&t.0.join(&id)).unwrap(); + fs::write(t.0.join(&id).join("snapshot-00000000000000000001.json"), serde_json::to_vec(&value).unwrap()).unwrap(); + let mut store = Store::new(t.0.clone()).unwrap(); + store.scan().unwrap(); + assert_eq!(store.inventory().len(), 1); + assert_eq!(store.load(&id).unwrap().document, s.document); + } + #[test] fn failed_pruning_cannot_push_a_record_beyond_the_discovery_limit() { let t = Temp::new(); diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index afc7eaad6..986088c2b 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -5886,6 +5886,9 @@ fn list_buttons_follow_the_paragraphs_own_level() { run(&mut s, "para.listLevel", json!({"level": 1})); assert_eq!(state(&s), ListButtons { bullets: false, numbering: true, multilevel: false }); run(&mut s, "para.numbering", json!({})); + assert_eq!(state(&s), ListButtons::default(), "Numbering toggles off the paragraph's effective numbered level"); + run(&mut s, "para.numbering", json!({})); + run(&mut s, "para.listLevel", json!({"level": 1})); assert_eq!(state(&s), ListButtons { bullets: false, numbering: true, multilevel: false }, "1. a. i. is not multilevel"); // Legal numbering (1. 1.1. 1.1.1.) numbers levels together, even on its first level. (Its // first level reads like "1." numbering, which the button would toggle off, so start plain.) diff --git a/crates/engine/src/tests_typing.rs b/crates/engine/src/tests_typing.rs index c610fd34c..a6b73f9c6 100644 --- a/crates/engine/src/tests_typing.rs +++ b/crates/engine/src/tests_typing.rs @@ -2160,3 +2160,197 @@ fn right_arrow_moves_through_a_ligature_one_letter_at_a_time() { assert_eq!(offs, [0, 1, 2, 3]); assert!(xs.windows(2).all(|w| w[0].1 < w[1].1), "caret x after each Right press: {xs:?}"); } + +/// Two original paragraphs whose list comes from a style, optionally through a basedOn chain. +fn style_inherited_list(kind: wordcraft_doc::numbering::ListKind, chain: bool) -> Session { + use wordcraft_doc::props::NumRef; + use wordcraft_doc::styles::Style; + let mut doc = wordcraft_doc::Document::from_text("Alpha\nBeta"); + let num = doc.numbering.try_add_list(kind).unwrap(); + doc.styles.styles.push(Style { + id: "ListBase".into(), + name: "Original List".into(), + para: wordcraft_doc::ParaProps { numbering: Some(NumRef { num, level: 0 }), ..Default::default() }, + ..Default::default() + }); + if chain { + doc.styles.styles.push(Style { + id: "ListChild".into(), + name: "Original Derived".into(), + based_on: Some("ListBase".into()), + ..Default::default() + }); + } + for i in 0..2 { + doc.para_mut(StoryRef::Body, &wordcraft_doc::Path::top(i)).unwrap().props.style = Some(if chain { "ListChild" } else { "ListBase" }.into()); + } + Session::new(doc) +} + +#[test] +fn toggle_style_inherited_list_off_preserves_other_paragraphs_and_round_trips() { + use wordcraft_doc::numbering::ListKind; + for (kind, command) in [(ListKind::Bullet, "para.bullets"), (ListKind::Numbered, "para.numbering")] { + for chain in [false, true] { + let mut s = style_inherited_list(kind, chain); + let before = s.doc.clone(); + s.sel = crate::Selection::caret(Pos::body(0, 2)); + run(&mut s, command, json!({})); + let p = s.doc.para_at(&Pos::body(0, 0)).unwrap(); + assert_eq!(p.props.numbering.unwrap().num, 0, "{command}, chain={chain}"); + assert_eq!(p.props.style, before.para_at(&Pos::body(0, 0)).unwrap().props.style); + assert_eq!(s.doc.para_at(&Pos::body(1, 0)), before.para_at(&Pos::body(1, 0))); + assert_eq!(s.doc.styles, before.styles); + assert_eq!(s.doc.numbering, before.numbering); + let after = s.doc.clone(); + run(&mut s, "edit.undo", json!({})); + assert_eq!(s.doc, before); + run(&mut s, "edit.redo", json!({})); + assert_eq!(s.doc, after); + let bytes = crate::io::save_bytes("list.docx", &s.doc).unwrap(); + let saved = crate::io::open_bytes("list.docx", &bytes).unwrap(); + let a = saved.para_at(&Pos::body(0, 0)).unwrap(); + let b = saved.para_at(&Pos::body(1, 0)).unwrap(); + assert_eq!(a.props.numbering.unwrap().num, 0); + assert!(saved.styles.resolve_para(&a.props).numbering.is_none()); + assert!(saved.styles.resolve_para(&b.props).numbering.is_some_and(|n| n.num != 0)); + assert_eq!(a.text, "Alpha"); + assert_eq!(b.text, "Beta"); + } + } +} + +#[test] +fn explicit_list_off_override_is_not_toggled_off_again() { + use wordcraft_doc::numbering::ListKind; + use wordcraft_doc::props::NumRef; + for (kind, command) in [(ListKind::Bullet, "para.bullets"), (ListKind::Numbered, "para.numbering")] { + let mut s = style_inherited_list(kind, true); + s.doc.para_mut(StoryRef::Body, &wordcraft_doc::Path::top(0)).unwrap().props.numbering = Some(NumRef { num: 0, level: 0 }); + s.sel = crate::Selection::caret(Pos::body(0, 2)); + run(&mut s, command, json!({})); + assert!(s.doc.para_at(&Pos::body(0, 0)).unwrap().props.numbering.is_some_and(|n| n.num != 0)); + // A mixed selection must turn the list on, rather than remove the inherited item. + s.doc.para_mut(StoryRef::Body, &wordcraft_doc::Path::top(0)).unwrap().props.numbering = Some(NumRef { num: 0, level: 0 }); + s.sel = crate::Selection { anchor: Pos::body(0, 0), focus: Pos::body(1, 4) }; + run(&mut s, command, json!({})); + for i in 0..2 { + let p = s.doc.para_at(&Pos::body(i, 0)).unwrap(); + assert!(s.doc.styles.resolve_para(&p.props).numbering.is_some_and(|n| n.num != 0)); + } + } +} + +#[test] +fn list_toggle_uses_effective_level_and_override() { + use wordcraft_doc::numbering::{ListKind, levels_for}; + use wordcraft_doc::props::NumRef; + for (kind, command, wrong) in [(ListKind::Bullet, "para.bullets", "para.numbering"), (ListKind::Numbered, "para.numbering", "para.bullets")] { + for override_level in [false, true] { + for explicit_kind in [false, true] { + let mut s = style_inherited_list(kind, true); + let opposite = if kind == ListKind::Bullet { ListKind::Numbered } else { ListKind::Bullet }; + let num = s.doc.styles.styles.iter_mut().find(|x| x.id == "ListBase").unwrap().para.numbering.as_mut().unwrap(); + num.level = 1; + let num_id = num.num; + s.doc.numbering.abstracts[0].levels[0] = levels_for(opposite)[0].clone(); + if override_level { + s.doc.numbering.abstracts[0].levels[1] = levels_for(opposite)[1].clone(); + s.doc.numbering.nums[0].level_overrides.push((1, levels_for(kind)[1].clone())); + } + let before = s.doc.clone(); + let params = if explicit_kind { json!({"kind": if kind == ListKind::Bullet { "bullet" } else { "numbered" }}) } else { json!({}) }; + run(&mut s, command, params); + assert_eq!( + s.doc.para_at(&Pos::body(0, 0)).unwrap().props.numbering, + Some(NumRef { num: 0, level: 0 }), + "{command}, override={override_level}, explicit={explicit_kind}" + ); + assert_eq!(s.doc.styles, before.styles); + assert_eq!(s.doc.numbering, before.numbering); + assert_eq!(s.doc.para_at(&Pos::body(1, 0)), before.para_at(&Pos::body(1, 0))); + let mut other = Session::new(before); + run(&mut other, wrong, json!({})); + assert!(other.doc.para_at(&Pos::body(0, 0)).unwrap().props.numbering.is_some_and(|n| n.num != 0)); + assert!(other.doc.numbering.level(num_id, 1).is_some()); + } + } + } +} + +#[test] +fn list_paragraph_style_survives_inherited_toggle_off() { + use wordcraft_doc::numbering::ListKind; + use wordcraft_doc::props::{Align, ParaProps}; + use wordcraft_doc::styles::Style; + for kind in [ListKind::Bullet, ListKind::Numbered] { + let mut s = style_inherited_list(kind, false); + *s.doc.styles.styles.iter_mut().find(|x| x.id == "ListParagraph").unwrap() = Style { + id: "ListParagraph".into(), + name: "Original formatted list".into(), + based_on: Some("ListBase".into()), + para: ParaProps { align: Some(Align::Right), space_after: Some(17.0), ..Default::default() }, + ..Default::default() + }; + s.doc.para_mut(StoryRef::Body, &wordcraft_doc::Path::top(0)).unwrap().props.style = Some("ListParagraph".into()); + let before = s.doc.clone(); + run(&mut s, if kind == ListKind::Bullet { "para.bullets" } else { "para.numbering" }, json!({})); + let p = s.doc.para_at(&Pos::body(0, 0)).unwrap(); + assert_eq!(p.props.style.as_deref(), Some("ListParagraph")); + let effective = s.doc.styles.resolve_para(&p.props); + assert!(effective.numbering.is_none()); + assert_eq!(effective.align, Align::Right); + assert_eq!(effective.space_after, 17.0); + assert_eq!(s.doc.styles, before.styles); + assert_eq!(s.doc.numbering, before.numbering); + assert_eq!(s.doc.para_at(&Pos::body(1, 0)), before.para_at(&Pos::body(1, 0))); + let after = s.doc.clone(); + run(&mut s, "edit.undo", json!({})); + assert_eq!(s.doc, before); + run(&mut s, "edit.redo", json!({})); + assert_eq!(s.doc, after); + let bytes = crate::io::save_bytes("list.docx", &s.doc).unwrap(); + let saved = crate::io::open_bytes("list.docx", &bytes).unwrap(); + let p = saved.para_at(&Pos::body(0, 0)).unwrap(); + assert_eq!(p.props.style.as_deref(), Some("ListParagraph")); + let effective = saved.styles.resolve_para(&p.props); + assert!(effective.numbering.is_none()); + assert_eq!(effective.align, Align::Right); + assert_eq!(effective.space_after, 17.0); + } +} + +#[test] +fn direct_list_toggle_still_clears_the_automatic_list_paragraph_style() { + for command in ["para.bullets", "para.numbering"] { + let mut s = Session::new(wordcraft_doc::Document::from_text("Direct list")); + run(&mut s, command, json!({})); + assert_eq!(s.doc.para_at(&Pos::body(0, 0)).unwrap().props.style.as_deref(), Some("ListParagraph")); + run(&mut s, command, json!({})); + let p = s.doc.para_at(&Pos::body(0, 0)).unwrap(); + assert!(p.props.style.is_none()); + assert!(s.doc.styles.resolve_para(&p.props).numbering.is_none()); + } +} + +#[test] +fn a_new_list_uses_the_requested_kind_after_a_mixed_format_list() { + use wordcraft_doc::numbering::{ListKind, levels_for}; + use wordcraft_doc::props::{NumRef, ParaProps}; + for (kind, command) in [(ListKind::Bullet, "para.bullets"), (ListKind::Numbered, "para.numbering")] { + let mut s = style_inherited_list(kind, false); + let opposite = if kind == ListKind::Bullet { ListKind::Numbered } else { ListKind::Bullet }; + s.doc.numbering.abstracts[0].levels[0] = levels_for(opposite)[0].clone(); + let num = s.doc.numbering.nums[0].id; + s.doc.para_mut(StoryRef::Body, &wordcraft_doc::Path::top(0)).unwrap().props.numbering = Some(NumRef { num, level: 1 }); + s.doc.para_mut(StoryRef::Body, &wordcraft_doc::Path::top(1)).unwrap().props = ParaProps::default(); + let before = s.doc.para_at(&Pos::body(0, 0)).unwrap().clone(); + s.sel = crate::Selection::caret(Pos::body(1, 0)); + run(&mut s, command, json!({})); + let n = s.doc.para_at(&Pos::body(1, 0)).unwrap().props.numbering.unwrap(); + assert_eq!(n.level, 0); + let level = s.doc.numbering.level(n.num, n.level).unwrap(); + assert_eq!(level.format == wordcraft_doc::section::NumFormat::Bullet, kind == ListKind::Bullet); + assert_eq!(s.doc.para_at(&Pos::body(0, 0)).unwrap(), &before); + } +} diff --git a/crates/engine/tests/recovery_format.rs b/crates/engine/tests/recovery_format.rs index 38f372ac7..f889d6a83 100644 --- a/crates/engine/tests/recovery_format.rs +++ b/crates/engine/tests/recovery_format.rs @@ -67,3 +67,88 @@ fn existing_native_json_without_byte_stores_still_opens() { let doc = wordcraft_engine::io::open_bytes("old.json", &serde_json::to_vec(&value).unwrap()).unwrap(); assert_eq!(doc.plain_text(wordcraft_doc::StoryRef::Body), "Old native document"); } + +/// Picture and part bytes are one base64 string each in native JSON (`.json` files and recovery +/// copies, #586). An array of numbers took about 3.6 times their size, and over ten times in a +/// pretty-printed `.json` file. +#[test] +fn native_json_stores_bytes_as_base64() { + let mut doc = Document::from_text("Synthetic"); + let compact = |d: &Document| serde_json::to_vec(d).unwrap().len(); + let pretty = |d: &Document| wordcraft_engine::io::save_bytes("doc.json", d).unwrap(); + let (without, without_pretty) = (compact(&doc), pretty(&doc).len()); + let bytes: Vec = (0..300_000u32).map(|i| (i.wrapping_mul(2_654_435_761) >> 13) as u8).collect(); + doc.media.insert("big.png".into(), Arc::new(bytes.clone())); + doc.passthrough.insert("customXml/item1.xml".into(), Arc::new(b"".to_vec())); + let saved = pretty(&doc); + // Base64 is 4/3 of the bytes; allow a little for names and punctuation. + for (with, before) in [(compact(&doc), without), (saved.len(), without_pretty)] { + assert!(with - before < bytes.len() * 14 / 10, "{} bytes for {}", with - before, bytes.len()); + } + let value: serde_json::Value = serde_json::from_slice(&saved).unwrap(); + assert_eq!(value["passthrough"]["customXml/item1.xml"], "PHN5bnRoZXRpYy8+"); + let reopened = wordcraft_engine::io::open_bytes("doc.json", &saved).unwrap(); + assert_eq!((reopened.media, reopened.passthrough), (doc.media, doc.passthrough)); +} + +/// `.json` files and recovery copies written before kept bytes as arrays of numbers: they still +/// open, with the same bytes. +#[test] +fn native_json_with_byte_arrays_still_opens() { + let doc = complete_document(); + let mut value = serde_json::to_value(&doc).unwrap(); + for (store, parts) in [("media", &doc.media), ("passthrough", &doc.passthrough)] { + for (name, bytes) in parts { + value[store][name] = serde_json::json!(bytes.as_slice()); + } + } + assert!(value["media"]["synthetic.png"].is_array()); + let reopened = wordcraft_engine::io::open_bytes("old.json", &serde_json::to_vec_pretty(&value).unwrap()).unwrap(); + assert_eq!(reopened, doc); +} + +/// Bytes that are neither base64 nor an array of bytes make the file fail to open, never panic. +#[test] +fn native_json_with_bad_bytes_is_an_error() { + let saved = wordcraft_engine::io::save_bytes("doc.json", &complete_document()).unwrap(); + for bad in [serde_json::json!("not base64!"), serde_json::json!([1, 2, 300]), serde_json::json!({"a": 1}), serde_json::json!(7)] { + let mut value: serde_json::Value = serde_json::from_slice(&saved).unwrap(); + value["media"]["synthetic.png"] = bad.clone(); + assert!(wordcraft_engine::io::open_bytes("doc.json", &serde_json::to_vec(&value).unwrap()).is_err(), "{bad}"); + } +} + +/// Measures native JSON with a large picture: the compact form a recovery copy wraps, and a +/// pretty-printed `.json` file. Run on demand: +/// `cargo test --release -p wordcraft-engine --test recovery_format -- --ignored --nocapture`. +#[test] +#[ignore = "measurement"] +fn native_json_size_with_a_large_picture() { + let mut doc = Document::from_text("Synthetic document with one large picture."); + // Incompressible bytes, as a photograph's are. + let mut x = 0x2545_f491_4f6c_dd1du64; + let noise: Vec = (0..27u32 << 20) + .map(|_| { + x ^= x << 13; + x ^= x >> 7; + x ^= x << 17; + x as u8 + }) + .collect(); + doc.media.insert("media/image1.png".into(), Arc::new(noise)); + let mib = |n: usize| n as f64 / f64::from(1u32 << 20); + let t0 = std::time::Instant::now(); + let compact = serde_json::to_vec(&doc).unwrap(); + let written = t0.elapsed(); + let t0 = std::time::Instant::now(); + let back: Document = serde_json::from_slice(&compact).unwrap(); + println!("27 MiB picture, compact (recovery copy): {:.1} MiB, written in {written:?}, read in {:?}", mib(compact.len()), t0.elapsed()); + assert_eq!(back.media, doc.media); + let t0 = std::time::Instant::now(); + let pretty = wordcraft_engine::io::save_bytes("doc.json", &doc).unwrap(); + let written = t0.elapsed(); + let t0 = std::time::Instant::now(); + let back = wordcraft_engine::io::open_bytes("doc.json", &pretty).unwrap(); + println!("27 MiB picture, .json file (pretty): {:.1} MiB, written in {written:?}, read in {:?}", mib(pretty.len()), t0.elapsed()); + assert_eq!(back.media, doc.media); +} diff --git a/crates/formats/src/html.rs b/crates/formats/src/html.rs index 4f2e4384f..3ba14405f 100644 --- a/crates/formats/src/html.rs +++ b/crates/formats/src/html.rs @@ -28,7 +28,14 @@ const MAX_STACK: usize = 256; /// Decode an entity body (`amp`, `#38`, `#x26`) to its text. pub fn decode_entity(name: &str) -> Option { if let Some(num) = name.strip_prefix('#') { - let v = if let Some(h) = num.strip_prefix(['x', 'X']) { u32::from_str_radix(h, 16).ok()? } else { num.parse::().ok()? }; + let (digits, radix) = match num.strip_prefix(['x', 'X']) { + Some(h) => (h, 16), + None => (num, 10), + }; + if !digits.chars().all(|c| c.is_digit(radix)) { + return None; + } + let v = u32::from_str_radix(digits, radix).ok()?; let c = match v { 0 => '\u{FFFD}', 0x80..=0x9F => crate::txt::cp1252(v as u8), @@ -1782,6 +1789,13 @@ pub fn export_flow(flow: &Flow, lang: &str) -> String { mod tests { use super::*; + #[test] + fn numeric_entity_rejects_sign() { + assert_eq!(unescape("a&#+65;b"), "a&#+65;b"); + assert_eq!(unescape("a&#x+41;b"), "a&#x+41;b"); + assert_eq!(unescape("aAAb"), "aAAb"); + } + fn texts(f: &Flow) -> Vec { f.blocks .iter() diff --git a/crates/formats/src/markdown.rs b/crates/formats/src/markdown.rs index 93ef23efe..f62e937aa 100644 --- a/crates/formats/src/markdown.rs +++ b/crates/formats/src/markdown.rs @@ -32,11 +32,130 @@ pub fn import_on(text: &str, page: &wordcraft_doc::SectionProps) -> Document { model::to_doc_in(&parse(text), page.clone()) } +/// Link reference definitions: the normalised label to its destination. +pub type Refs = HashMap; + +/// A label as CommonMark compares them: case-folded, and any run of white space one space +/// (§4.7, "matches"). `[FOO bar]` and `[foo BAR]` are the same label. +fn ref_label(raw: &str) -> String { + raw.split_whitespace().collect::>().join(" ").to_lowercase() +} + +/// A link reference definition on its own line: `[label]: destination "optional title"`. The +/// destination may be wrapped in `<...>`. Returns the label and the destination. +/// +/// CommonMark allows the title on the following line and the label to span lines; a definition +/// written across lines is left as text here, which is what it was before these were read at all. +fn link_definition(line: &str) -> Option<(String, String)> { + let t = line.trim_start(); + // Up to three spaces of indentation; four makes it indented code. + if line.len() - t.len() > 3 || !t.starts_with('[') { + return None; + } + let bytes = t.as_bytes(); + // The label ends at the first unescaped `]`, and cannot be empty or hold an unescaped `[`. + let mut i = 1; + let mut close = None; + while i < bytes.len() { + match bytes[i] { + b'\\' => i += 1, + b'[' => return None, + b']' => { + close = Some(i); + break; + } + _ => {} + } + i += 1; + } + let close = close?; + let label = ref_label(t.get(1..close)?); + // `[^label]: ...` is a footnote definition, which this importer leaves as text. + if label.starts_with('^') { + return None; + } + if label.is_empty() || t.as_bytes().get(close + 1) != Some(&b':') { + return None; + } + let rest = t.get(close + 2..)?.trim_start(); + if rest.is_empty() { + return None; + } + // The destination: `<...>`, else up to the next space. What follows may only be a title. + let (dest, after) = match rest.strip_prefix('<') { + Some(inner) => { + let end = inner.find('>')?; + (inner.get(..end)?.to_string(), inner.get(end + 1..)?) + } + None => { + let end = rest.find(char::is_whitespace).unwrap_or(rest.len()); + (rest.get(..end)?.to_string(), rest.get(end..)?) + } + }; + if dest.is_empty() { + return None; + } + let title = after.trim(); + let titled = + title.is_empty() || (title.len() >= 2 && matches!((title.chars().next()?, title.chars().last()?), ('"', '"') | ('\'', '\'') | ('(', ')'))); + titled.then_some((label, dest)) +} + +/// Take the link reference definitions out of `lines`, leaving the rest. Lines inside a fenced +/// code block are left alone: a definition there is code, not a definition. Fences open and +/// close by the same rules [`parse_blocks`] applies, or a line it reads as code would be read +/// here as a definition and deleted. +/// +/// A definition has to start a block: CommonMark §4.7 forbids one from interrupting a paragraph, +/// and a line like `[Warning]: unstable` in the middle of one is the author's text. This pass +/// sees lines without the block structure around them, so it takes a definition only after a +/// blank line, at the start of the document, after a heading or thematic break, or after another +/// definition. A definition anywhere else stays in the document and reads as text, which is what +/// it did before definitions were understood at all; deleting a line that was part of a +/// paragraph would lose it silently. +fn take_definitions(lines: &mut Vec) -> Refs { + let mut refs = Refs::new(); + let mut open: Option<(char, usize)> = None; + // The start of the document begins a block. + let mut starts_block = true; + lines.retain(|line| { + let t = line.trim_start_matches([' ', '\t']); + // Four spaces of indentation is code, not a fence (`parse_blocks`), on both sides. + let marker = (indent(line) < 4).then(|| fence(t)).flatten(); + match open { + Some((c, n)) => { + // A closing fence holds nothing but its own run of fence characters. + if marker.is_some_and(|(fc, fl)| fc == c && fl >= n) && t.trim_start_matches(c).trim().is_empty() { + open = None; + starts_block = true; + } + return true; + } + None => { + if let Some(f) = marker { + open = Some(f); + return true; + } + } + } + if starts_block && let Some((label, dest)) = link_definition(line) { + // The first definition of a label wins, as CommonMark says, and definitions may + // follow one another. + refs.entry(label).or_insert(dest); + return false; + } + starts_block = is_blank(line) || atx_heading(t).is_some() || thematic_break(t); + true + }); + refs +} + /// Parse Markdown text into the flow model. pub fn parse(text: &str) -> model::Flow { let text = text.strip_prefix('\u{FEFF}').unwrap_or(text); - let lines: Vec = text.split('\n').map(|l| expand_tabs(l.trim_end_matches('\r'))).collect(); - let mut blocks = parse_blocks(&lines, 0, 0); + let mut lines: Vec = text.split('\n').map(|l| expand_tabs(l.trim_end_matches('\r'))).collect(); + let refs = take_definitions(&mut lines); + let mut blocks = parse_blocks(&lines, 0, 0, &refs); // A leading "Title" from front matter isn't supported; a first-level heading stays a heading. if blocks.is_empty() { blocks.push(FBlock::Para(Para::default())); @@ -234,13 +353,13 @@ fn interrupts(l: &str) -> bool { || list_marker(t).is_some_and(|m| (!m.ordered || m.start == 1) && !is_blank(t.get(m.len..).unwrap_or(""))) } -fn flush_para(lines: &mut Vec, out: &mut Vec, depth: usize) { +fn flush_para(lines: &mut Vec, out: &mut Vec, depth: usize, refs: &Refs) { if lines.is_empty() { return; } let text = join_lines(lines); lines.clear(); - let p = Para { inlines: parse_inlines(&text, &Fmt::default(), depth), ..Default::default() }; + let p = Para { inlines: parse_inlines(&text, &Fmt::default(), depth, refs), ..Default::default() }; out.push(FBlock::Para(p)); } @@ -273,13 +392,13 @@ fn code_para(line: &str) -> FBlock { } /// Parse block structure. `level` is the list nesting level for lists found here. -fn parse_blocks(lines: &[String], depth: usize, level: u8) -> Vec { +fn parse_blocks(lines: &[String], depth: usize, level: u8, refs: &Refs) -> Vec { let mut out = Vec::new(); let mut para: Vec = Vec::new(); let mut i = 0; while let Some(line) = lines.get(i) { if is_blank(line) { - flush_para(&mut para, &mut out, depth); + flush_para(&mut para, &mut out, depth, refs); i += 1; continue; } @@ -317,28 +436,28 @@ fn parse_blocks(lines: &[String], depth: usize, level: u8) -> Vec { let text = join_lines(¶); para.clear(); let mut p = Para::new(Kind::Heading(lvl)); - p.inlines = parse_inlines(&text, &Fmt::default(), depth); + p.inlines = parse_inlines(&text, &Fmt::default(), depth, refs); out.push(FBlock::Para(p)); i += 1; continue; } } if let Some((n, body)) = atx_heading(t) { - flush_para(&mut para, &mut out, depth); + flush_para(&mut para, &mut out, depth, refs); let mut p = Para::new(Kind::Heading(n)); - p.inlines = parse_inlines(body, &Fmt::default(), depth); + p.inlines = parse_inlines(body, &Fmt::default(), depth, refs); out.push(FBlock::Para(p)); i += 1; continue; } if thematic_break(t) { - flush_para(&mut para, &mut out, depth); + flush_para(&mut para, &mut out, depth, refs); out.push(FBlock::Para(Para::new(Kind::Rule))); i += 1; continue; } if let Some((fc, fl)) = fence(t) { - flush_para(&mut para, &mut out, depth); + flush_para(&mut para, &mut out, depth, refs); i += 1; while let Some(l) = lines.get(i) { i += 1; @@ -351,7 +470,7 @@ fn parse_blocks(lines: &[String], depth: usize, level: u8) -> Vec { continue; } if t.starts_with('>') { - flush_para(&mut para, &mut out, depth); + flush_para(&mut para, &mut out, depth, refs); let mut inner: Vec = Vec::new(); while let Some(l) = lines.get(i) { let lt = l.trim_start_matches([' ', '\t']); @@ -370,7 +489,7 @@ fn parse_blocks(lines: &[String], depth: usize, level: u8) -> Vec { p.push_text(&inner.join(" "), &Fmt::default()); out.push(FBlock::Para(p)); } else { - for mut b in parse_blocks(&inner, depth + 1, level) { + for mut b in parse_blocks(&inner, depth + 1, level, refs) { if let FBlock::Para(p) = &mut b && p.kind == Kind::Normal { @@ -389,7 +508,7 @@ fn parse_blocks(lines: &[String], depth: usize, level: u8) -> Vec { if head.len() == aligns.len() { let mut table = FTable::default(); let cell = |s: &str, header: bool, a: Option| { - let p = Para { align: a, inlines: parse_inlines(s, &Fmt::default(), depth), ..Default::default() }; + let p = Para { align: a, inlines: parse_inlines(s, &Fmt::default(), depth, refs), ..Default::default() }; Cell { blocks: vec![FBlock::Para(p)], header, ..Default::default() } }; table.rows.push(head.iter().zip(&aligns).map(|(s, a)| cell(s, true, *a)).collect()); @@ -410,19 +529,19 @@ fn parse_blocks(lines: &[String], depth: usize, level: u8) -> Vec { if let Some(m) = list_marker(t) && (para.is_empty() || ((!m.ordered || m.start == 1) && !is_blank(t.get(m.len..).unwrap_or("")))) { - flush_para(&mut para, &mut out, depth); - i = parse_list(lines, i, depth, level, &mut out); + flush_para(&mut para, &mut out, depth, refs); + i = parse_list(lines, i, depth, level, &mut out, refs); continue; } para.push(line.clone()); i += 1; } - flush_para(&mut para, &mut out, depth); + flush_para(&mut para, &mut out, depth, refs); out } /// Parse a list starting at `lines[i]`; returns the index after it. -fn parse_list(lines: &[String], mut i: usize, depth: usize, level: u8, out: &mut Vec) -> usize { +fn parse_list(lines: &[String], mut i: usize, depth: usize, level: u8, out: &mut Vec, refs: &Refs) -> usize { let Some(first) = lines.get(i).and_then(|l| list_marker(l.trim_start_matches([' ', '\t']))) else { return i + 1 }; while let Some(line) = lines.get(i) { let ind = indent(line); @@ -466,7 +585,7 @@ fn parse_list(lines: &[String], mut i: usize, depth: usize, level: u8, out: &mut p.push_text(&item.join(" "), &Fmt::default()); vec![FBlock::Para(p)] } else { - parse_blocks(&item, depth + 1, level.saturating_add(1).min(8)) + parse_blocks(&item, depth + 1, level.saturating_add(1).min(8), refs) }; let mut placed = false; if inner.is_empty() { @@ -539,8 +658,8 @@ fn is_punct(c: char) -> bool { } /// Parse inline Markdown into formatted spans (on top of `base`). -pub fn parse_inlines(s: &str, base: &Fmt, depth: usize) -> Vec { - let nodes = tokenize(s, base, depth); +pub fn parse_inlines(s: &str, base: &Fmt, depth: usize, refs: &Refs) -> Vec { + let nodes = tokenize(s, base, depth, refs); emit(nodes) } @@ -721,7 +840,7 @@ fn tag_attr(tag: &str, name: &str) -> Option { None } -fn tokenize(s: &str, base: &Fmt, depth: usize) -> Vec { +fn tokenize(s: &str, base: &Fmt, depth: usize, refs: &Refs) -> Vec { let mut nodes: Vec = Vec::new(); // Bytes the link and image scans below may look ahead in total. Each unmatched `[` or `![(` // would otherwise rescan the rest of the text, which is quadratic on hostile input; once the @@ -811,12 +930,29 @@ fn tokenize(s: &str, base: &Fmt, depth: usize) -> Vec { } None => None, }; - found.filter(|close| b.get(close + 1) == Some(&b'(') && scan_budget > 0).and_then(|close| { - // A destination that does not parse may have been read to the end of the text. - let dest = link_dest(s, close + 1); - scan_budget = scan_budget.saturating_sub(dest.as_ref().map_or(s.len(), |d| d.1).saturating_sub(close)); - let (dest, end) = dest?; - Some((s.get(open + 1..close)?.to_string(), dest, end)) + found.filter(|_| scan_budget > 0).and_then(|close| { + let text = s.get(open + 1..close)?; + if b.get(close + 1) == Some(&b'(') { + // A destination that does not parse may have been read to the end of the text. + let dest = link_dest(s, close + 1); + scan_budget = scan_budget.saturating_sub(dest.as_ref().map_or(s.len(), |d| d.1).saturating_sub(close)); + let (dest, end) = dest?; + return Some((text.to_string(), dest, end)); + } + // A reference link: `[text][label]`, `[text][]` and `[text]` all take + // their destination from a definition collected before parsing (#3238). + // The shortcut form is only a link when the label is defined, so an + // ordinary bracketed phrase is still text. + let (label, end) = match s.get(close + 1..).and_then(|rest| rest.strip_prefix('[')) { + Some(rest) => { + let shut = rest.find(']')?; + let inner = rest.get(..shut)?; + let label = if inner.trim().is_empty() { text } else { inner }; + (label, close + 2 + shut + 1) + } + None => (text, close + 1), + }; + Some((text.to_string(), refs.get(&ref_label(label))?.clone(), end)) }) } else { None @@ -824,7 +960,7 @@ fn tokenize(s: &str, base: &Fmt, depth: usize) -> Vec { match parsed { Some((inner, dest, end)) if img => { flush!(); - let alt: String = parse_inlines(&inner, &Fmt::default(), depth + 1) + let alt: String = parse_inlines(&inner, &Fmt::default(), depth + 1, refs) .iter() .filter_map(|x| if let Inline::Text(t, _) = x { Some(t.as_str()) } else { None }) .collect(); @@ -839,7 +975,7 @@ fn tokenize(s: &str, base: &Fmt, depth: usize) -> Vec { flush!(); let mut f = base.clone(); f.link = Some(dest); - for x in parse_inlines(&inner, &f, depth + 1) { + for x in parse_inlines(&inner, &f, depth + 1, refs) { nodes.push(Node::Inl(x)); } i = end; @@ -1414,7 +1550,7 @@ mod tests { use super::*; fn inl(s: &str) -> Vec { - parse_inlines(s, &Fmt::default(), 0) + parse_inlines(s, &Fmt::default(), 0, &Refs::new()) } fn spans(s: &str) -> Vec<(String, bool, bool)> { @@ -1449,6 +1585,84 @@ mod tests { assert!(text.contains("*lit* & "), "{text}"); } + /// Reference links: `[text][label]`, `[text][]` and `[text]` take their destination from a + /// definition line, and the definition itself leaves the document. Before, none of the three + /// was a link and the definition stayed as a paragraph of its own (#3238). + #[test] + fn reference_links_resolve_and_their_definitions_leave_the_body() { + let links = |md: &str| -> Vec<(String, String)> { + match &*parse(md).blocks { + [FBlock::Para(p), ..] => p + .inlines + .iter() + .filter_map(|x| match x { + Inline::Text(t, f) => f.link.clone().map(|l| (t.clone(), l)), + _ => None, + }) + .collect(), + _ => vec![], + } + }; + let def = "\n\n[ref1]: https://example.com/ref\n"; + let to_ref = |text: &str| vec![(text.to_string(), "https://example.com/ref".to_string())]; + + assert_eq!(links(&format!("[click here][ref1]{def}")), to_ref("click here"), "full"); + assert_eq!(links(&format!("[ref1][]{def}")), to_ref("ref1"), "collapsed"); + assert_eq!(links(&format!("[ref1]{def}")), to_ref("ref1"), "shortcut"); + // Labels match case-folded with runs of white space collapsed (CommonMark §4.7). + assert_eq!(links(&format!("[click here][ REF1 ]{def}")), to_ref("click here"), "folded label"); + // An undefined label is not a link: an ordinary bracketed phrase stays text. + assert_eq!(links(&format!("[nope]{def}")), vec![], "undefined"); + // Inline links are unaffected. + assert_eq!(links("[x](https://inline.example)"), vec![("x".into(), "https://inline.example".into())], "inline"); + + // The definition is not a paragraph, and its text is nowhere in the body. + let doc = import(&format!("Reference link: [click here][ref1]{def}")); + let text = doc.plain_text(wordcraft_doc::StoryRef::Body); + assert_eq!(text, "Reference link: click here", "{text:?}"); + + // A definition inside a fence is code, not a definition. + let fenced = "```\n[ref1]: https://example.com/ref\n```\n\n[ref1]\n"; + let body = import(fenced).plain_text(wordcraft_doc::StoryRef::Body); + assert!(body.contains("[ref1]: https://example.com/ref"), "the fenced definition is still code: {body:?}"); + assert_eq!(links(fenced), vec![], "a fenced definition defines nothing"); + } + + /// A definition has to start a block (CommonMark §4.7), and a fence opens and closes by the + /// same rules the block parser uses. Reading a line without that context deleted text: a + /// definition-shaped line inside a paragraph, and every such line inside a code block whose + /// fence this pass thought had closed. + #[test] + fn a_definition_shaped_line_that_is_not_a_definition_stays_in_the_document() { + let body = |md: &str| import(md).plain_text(wordcraft_doc::StoryRef::Body); + + // A definition cannot interrupt a paragraph: this is the author's text, lines and all. + let para = "Status update\n[Warning]: unstable\nmore text\n"; + assert_eq!(body(para), "Status update [Warning]: unstable more text", "{:?}", body(para)); + + // `` ```rust `` holds an info string, so it does not close a fence: the definition-shaped + // line after it is still code. + let code = "```\n```rust\n[foo]: /bar\n```\n\nSee [foo].\n"; + assert!(body(code).contains("[foo]: /bar"), "{:?}", body(code)); + assert!(body(code).ends_with("See [foo]."), "nothing defines foo: {:?}", body(code)); + + // Four spaces of indentation is code, not a fence, so the definition that follows is + // taken and the reference resolves. + let indented = " ```\n\n[x]: /y\n\nSee [x].\n"; + let doc = import(indented); + let text = doc.plain_text(wordcraft_doc::StoryRef::Body); + assert!(text.contains("```") && text.ends_with("See x."), "{text:?}"); + + // A definition still gives up its line after a blank line, at the start of the document, + // after a heading, and after another definition. + let starts = "[a]: /a\n[b]: /b\n\n# Head\n[c]: /c\n\nSee [a], [b], [c].\n"; + assert_eq!(body(starts), "Head\nSee a, b, c.", "{:?}", body(starts)); + + // A footnote definition is not a link definition: it stays in the body. + let footnote = "x[^1]\n\n[^1]: Note\n"; + assert!(body(footnote).contains("[^1]: Note"), "{:?}", body(footnote)); + } + #[test] fn blocks() { let f = parse( diff --git a/crates/formats/src/model.rs b/crates/formats/src/model.rs index 449ba1990..b9f231687 100644 --- a/crates/formats/src/model.rs +++ b/crates/formats/src/model.rs @@ -1214,6 +1214,7 @@ pub fn percent_decode(s: &str) -> Vec { while let Some(&c) = b.get(i) { if c == b'%' && let Some(h) = b.get(i + 1..i + 3) + && h.iter().all(u8::is_ascii_hexdigit) && let Some(hex) = std::str::from_utf8(h).ok() && let Ok(v) = u8::from_str_radix(hex, 16) { @@ -1344,6 +1345,14 @@ pub fn parse_color(s: &str) -> Option { mod tests { use super::*; + #[test] + fn percent_decode_requires_hex_digits() { + assert_eq!(percent_decode("%+A"), b"%+A".to_vec()); + assert_eq!(percent_decode("%-A"), b"%-A".to_vec()); + assert_eq!(percent_decode("%G1"), b"%G1".to_vec()); + assert_eq!(percent_decode("%41"), b"A".to_vec()); + } + #[test] fn insert_covered_is_bounded_on_hostile_rowspan() { // Many rows each carrying the maximum rowspan would be O(rows × rowspan) covered cells; diff --git a/crates/formats/src/odt.rs b/crates/formats/src/odt.rs index 597a1020e..0f94db8ac 100644 --- a/crates/formats/src/odt.rs +++ b/crates/formats/src/odt.rs @@ -40,6 +40,8 @@ const MAX_OPEN_INSERTIONS: usize = 64; /// Longest change author or date kept, in characters. const MAX_INFO_CHARS: usize = 256; +mod page_parts; + // --------------------------------------------------------------------------------------------- // Export @@ -1171,6 +1173,62 @@ fn events(xml: &[u8], mut f: impl FnMut(Ev)) { } } +/// An embedded ODF formula sub-document's `content.xml`, read for its `math:math` element +/// (ODF 1.3 §10.4.3.2 wraps it in `office:document-content`/`office:body`/`office:math`, but +/// some writers emit it bare): the first element named `math` in document order, with its +/// subtree, passed to [`crate::mathml::read`]. `None` when there's no such element, it holds +/// nothing, or the XML is malformed. Nesting and element count are bounded like other XML here. +fn read_formula(xml: &[u8]) -> Option<(String, bool)> { + use crate::mathml::{El, Item, MAX_NODES, local}; + let mut stack: Vec = Vec::new(); + let mut math: Option = None; + let mut over = 0usize; + let mut count = 0usize; + events(xml, |ev| { + if math.is_some() { + return; + } + count += 1; + if count > MAX_NODES * 4 { + math = stack.drain(..).next(); + return; + } + match ev { + Ev::Start(q, a) => { + if stack.is_empty() && local(&q) != "math" { + return; + } + if stack.len() < wordcraft_doc::math::MAX_DEPTH { + stack.push(El { name: q, attrs: a, children: Vec::new() }); + } else { + over += 1; + } + } + Ev::End(_) => { + if over > 0 { + over -= 1; + } else if stack.len() > 1 { + if let Some(done) = stack.pop() + && let Some(parent) = stack.last_mut() + { + parent.children.push(Item::El(done)); + } + } else if stack.len() == 1 { + math = stack.pop(); + } + } + Ev::Text(t) => { + if over == 0 + && let Some(top) = stack.last_mut() + { + top.children.push(Item::Text(t)); + } + } + } + }); + crate::mathml::read(&math?) +} + #[derive(Default)] struct Styles { map: HashMap, @@ -1487,7 +1545,8 @@ struct Body<'a> { in_text: bool, skip: usize, depth: usize, - frame: Option<(Option, Option, Option>, String)>, + /// `(width, height, picture bytes, alt text, equation read from an embedded `draw:object`)`. + frame: Option<(Option, Option, Option>, String, Option<(String, bool)>)>, in_desc: bool, /// The master pages' page setups. pages: &'a Pages, @@ -1518,6 +1577,82 @@ struct Body<'a> { active_ins: Vec<(String, RevMark)>, } +impl<'a> Body<'a> { + fn new(styles: &'a Styles, pages: &'a Pages, zip: &'a mut dyn FnMut(&str) -> Option>) -> Self { + let page = pages.first.clone().unwrap_or_default(); + Body { + styles, + zip, + containers: vec![Vec::new()], + fmt: Vec::new(), + para: None, + xml_space_end: false, + para_cont: false, + lists: Vec::new(), + lists_saved: Vec::new(), + item_used: Vec::new(), + tables: Vec::new(), + cells: Vec::new(), + in_text: false, + skip: 0, + depth: 0, + frame: None, + in_desc: false, + pages, + sec: page.clone(), + page, + sec_from: 0, + sections: 0, + cols_at: None, + in_tc: false, + track: None, + regions: HashMap::new(), + region_count: 0, + region: None, + info_at: None, + info_field: None, + del: None, + active_ins: Vec::new(), + } + } + + fn feed(&mut self, ev: Ev) { + match ev { + Ev::Start(q, a) => { + if self.depth < MAX_XML_DEPTH { + self.start(&q, &a); + } else { + self.depth += 1; + } + } + Ev::End(q) => { + if self.depth <= MAX_XML_DEPTH { + self.end(&q); + } else { + self.depth -= 1; + } + } + Ev::Text(t) => self.text(&t), + } + } + + /// The blocks read, with anything still open closed. + fn take_blocks(&mut self) -> Vec { + self.end_para(); + while self.containers.len() > 1 { + let c = self.containers.pop().unwrap_or_default(); + self.container().extend(c); + } + let mut blocks = self.containers.pop().unwrap_or_default(); + for t in self.tables.drain(..) { + if !t.rows.is_empty() { + blocks.push(FBlock::Table(FTable { rows: t.rows, widths: Vec::new(), borderless: false })); + } + } + blocks + } +} + impl Body<'_> { fn container(&mut self) -> &mut Vec { if self.containers.is_empty() { @@ -1929,17 +2064,32 @@ impl Body<'_> { "frame" => { let w = get(a, "svg:width").and_then(length); let h = get(a, "svg:height").and_then(length); - self.frame = Some((w, h, None, String::new())); + self.frame = Some((w, h, None, String::new(), None)); } "image" => { if let Some(href) = get(a, "xlink:href") && let Some(fr) = &mut self.frame && fr.2.is_none() + && fr.4.is_none() { let path = href.trim_start_matches("./"); fr.2 = (self.zip)(path); } } + // An embedded object (ODF 1.3 §10.4.3): a formula sub-document's `content.xml` holds + // a `math:math` tree (#3099). Other object kinds have none, so the frame still falls + // back to its `draw:image` replacement preview, read at `end` below. + "object" => { + if let Some(href) = get(a, "xlink:href") + && let Some(fr) = &mut self.frame + && fr.4.is_none() + { + let path = format!("{}/content.xml", href.trim_start_matches("./").trim_end_matches('/')); + if let Some(xml) = (self.zip)(&path) { + fr.4 = read_formula(&xml); + } + } + } "desc" | "title" if self.frame.is_some() => self.in_desc = true, // A columned section (ODF 1.3 §5.4) is a continuous section with its columns. "section" if q.starts_with("text:") => { @@ -2202,14 +2352,18 @@ impl Body<'_> { self.break_section(next); } "frame" => { - if let Some((w, h, Some(data), alt)) = self.frame.take() - && let Some(img) = make_img(data, w, h, alt.trim()) - { - if self.para.is_none() { - self.start_para(None, None); - } - if let Some(p) = &mut self.para { - p.inlines.push(Inline::Image(img)); + if let Some((w, h, img, alt, eq)) = self.frame.take() { + let inline = match eq { + Some((linear, display)) => Some(Inline::Equation { linear, display }), + None => img.and_then(|data| make_img(data, w, h, alt.trim())).map(Inline::Image), + }; + if let Some(inline) = inline { + if self.para.is_none() { + self.start_para(None, None); + } + if let Some(p) = &mut self.para { + p.inlines.push(inline); + } } } self.frame = None; @@ -2325,11 +2479,11 @@ pub fn parse(bytes: &[u8]) -> Result { /// [`parse`], with the last section's properties (the others are on the paragraphs that end them). fn parse_full(bytes: &[u8]) -> Result<(Flow, SectionProps), String> { - parse_tracked(bytes).map(|(f, last, _, _)| (f, last)) + parse_tracked(bytes).map(|(f, last, ..)| (f, last)) } /// [`parse_full`], plus whether change tracking is on (`None`: the document has no tracked changes). -fn parse_tracked(bytes: &[u8]) -> Result<(Flow, SectionProps, Option, Option), String> { +fn parse_tracked(bytes: &[u8]) -> Result<(Flow, SectionProps, Option, Option, page_parts::Parts), String> { let mut z = zip::ZipArchive::new(Cursor::new(bytes)).map_err(|e| format!("ODT: not a valid package ({e})"))?; if let Some(mt) = read_entry(&mut z, "mimetype") { let mt = String::from_utf8_lossy(&mt); @@ -2340,11 +2494,11 @@ fn parse_tracked(bytes: &[u8]) -> Result<(Flow, SectionProps, Option, Opti let content = read_entry(&mut z, "content.xml").ok_or("ODT: content.xml is missing")?; let mut styles = Styles::default(); let mut pages = Pages::default(); - if let Some(s) = read_entry(&mut z, "styles.xml") { - styles.read(&s, false); - pages = read_pages(&s); + let styles_xml = read_entry(&mut z, "styles.xml"); + if let Some(s) = &styles_xml { + styles.read(s, false); + pages = read_pages(s); } - let page = pages.first.clone().unwrap_or_default(); styles.read(&content, true); let meta = read_entry(&mut z, "meta.xml").map(|m| read_meta(&m)).unwrap_or_default(); let mut pictures = Pictures::new(MAX_ENTRY); @@ -2352,68 +2506,10 @@ fn parse_tracked(bytes: &[u8]) -> Result<(Flow, SectionProps, Option, Opti // A frame may point at any entry, as often as it likes: `Pictures` keeps that bounded. pictures.get(&mut z, path) }; - let mut body = Body { - styles: &styles, - zip: &mut fetch, - containers: vec![Vec::new()], - fmt: Vec::new(), - para: None, - xml_space_end: false, - para_cont: false, - lists: Vec::new(), - lists_saved: Vec::new(), - item_used: Vec::new(), - tables: Vec::new(), - cells: Vec::new(), - in_text: false, - skip: 0, - depth: 0, - frame: None, - in_desc: false, - pages: &pages, - sec: page.clone(), - page, - sec_from: 0, - sections: 0, - cols_at: None, - in_tc: false, - track: None, - regions: HashMap::new(), - region_count: 0, - region: None, - info_at: None, - info_field: None, - del: None, - active_ins: Vec::new(), - }; - events(&content, |ev| match ev { - Ev::Start(q, a) => { - if body.depth < MAX_XML_DEPTH { - body.start(&q, &a); - } else { - body.depth += 1; - } - } - Ev::End(q) => { - if body.depth <= MAX_XML_DEPTH { - body.end(&q); - } else { - body.depth -= 1; - } - } - Ev::Text(t) => body.text(&t), - }); - body.end_para(); - while body.containers.len() > 1 { - let c = body.containers.pop().unwrap_or_default(); - body.container().extend(c); - } - let mut blocks = body.containers.pop().unwrap_or_default(); - for t in body.tables.drain(..) { - if !t.rows.is_empty() { - blocks.push(FBlock::Table(FTable { rows: t.rows, widths: Vec::new(), borderless: false })); - } - } + let parts = styles_xml.as_deref().map(|x| page_parts::read(x, &styles, &mut pages, &mut fetch)).unwrap_or_default(); + let mut body = Body::new(&styles, &pages, &mut fetch); + events(&content, |ev| body.feed(ev)); + let mut blocks = body.take_blocks(); // An open section with nothing in it (after a column section at the end) isn't one: the // section before it ends the document. let mut last = body.sec; @@ -2427,17 +2523,18 @@ fn parse_tracked(bytes: &[u8]) -> Result<(Flow, SectionProps, Option, Opti blocks.push(FBlock::Para(Para::default())); } let track = body.track; - Ok((Flow { blocks, meta }, last, track, styles.doc_font())) + Ok((Flow { blocks, meta }, last, track, styles.doc_font(), parts)) } /// Parse an `.odt` package into a document. pub fn import(bytes: &[u8]) -> Result { - let (flow, last, track, font) = parse_tracked(bytes)?; + let (flow, last, track, font, parts) = parse_tracked(bytes)?; let mut doc = model::to_doc(&flow); if font.is_some() { doc.styles.default_chr.font = font; } doc.last_section = last; + page_parts::add(&mut doc, parts); if let Some(t) = track { doc.settings.track_changes = t; } @@ -2450,6 +2547,7 @@ pub fn import(bytes: &[u8]) -> Result { struct Pages { masters: HashMap, first: Option, + first_name: Option, } fn read_pages(xml: &[u8]) -> Pages { @@ -2501,6 +2599,7 @@ fn read_pages(xml: &[u8]) -> Pages { } if pages.first.is_none() { pages.first = Some(sec.clone()); + pages.first_name = Some(name.clone()); } pages.masters.entry(name).or_insert(sec); } @@ -2657,6 +2756,68 @@ mod tests { assert_eq!(import(&pkg).unwrap().media.len(), 1); } + #[test] + fn embedded_formula_object_reads_its_mathml() { + // #3099: LibreOffice saves an equation as a `draw:object` (a formula sub-document whose + // own `content.xml` is MathML) plus a `draw:image` replacement preview in LibreOffice's + // own metafile format, which WordCraft can't decode as a picture. The object's MathML + // must be read instead of silently dropping the frame (blank equation). + let math = r#"E=mc2E=mc^2"#; + let frame = r#""#; + let content = format!( + r#"{frame}"# + ); + let pkg = zip_of(&[ + ("content.xml", content.as_bytes()), + ("Object 1/content.xml", math.as_bytes()), + // Not a format `sniff_image` recognizes: proves the equation wins over this fallback. + ("ObjectReplacements/Object 1", &[0u8; 50]), + ]); + let flow = parse(&pkg).unwrap(); + let eqs: Vec<(String, bool)> = paras(&flow) + .into_iter() + .flat_map(|p| &p.inlines) + .filter_map(|i| if let Inline::Equation { linear, display } = i { Some((linear.clone(), *display)) } else { None }) + .collect(); + assert_eq!(eqs, vec![("E=mc^2".to_string(), true)]); + } + + #[test] + fn a_trailing_slash_on_the_object_href_still_resolves() { + // Some ODF producers write the object folder with a trailing slash + // (`xlink:href="./Object 1/"`); that must not double the slash before `content.xml`. + let math = r#"x"#; + let frame = r#""#; + let content = format!( + r#"{frame}"# + ); + let pkg = zip_of(&[("content.xml", content.as_bytes()), ("Object 1/content.xml", math.as_bytes())]); + let flow = parse(&pkg).unwrap(); + let eqs: Vec<(String, bool)> = paras(&flow) + .into_iter() + .flat_map(|p| &p.inlines) + .filter_map(|i| if let Inline::Equation { linear, display } = i { Some((linear.clone(), *display)) } else { None }) + .collect(); + assert_eq!(eqs, vec![("x".to_string(), false)]); + } + + #[test] + fn object_without_mathml_falls_back_to_its_replacement_image() { + // A non-formula embedded object (or one whose content.xml has no `math:math`) has no + // equation to read: the frame still shows its `draw:image` replacement, as before. + let png = [b"\x89PNG\r\n\x1a\n".as_slice(), &[7u8; 300]].concat(); + let frame = r#""#; + let content = format!( + r#"{frame}"# + ); + let pkg = zip_of(&[("content.xml", content.as_bytes()), ("Object 1/content.xml", b""), ("Pictures/a.png", &png)]); + let flow = parse(&pkg).unwrap(); + let imgs: Vec<&model::Img> = + paras(&flow).into_iter().flat_map(|p| &p.inlines).filter_map(|i| if let Inline::Image(img) = i { Some(img) } else { None }).collect(); + assert_eq!(imgs.len(), 1); + assert!(imgs[0].data.as_slice() == png.as_slice()); + } + const NS_T: &str = r#"xmlns:office="urn:oasis:names:tc:opendocument:xmlns:office:1.0" xmlns:text="urn:oasis:names:tc:opendocument:xmlns:text:1.0" xmlns:style="urn:oasis:names:tc:opendocument:xmlns:style:1.0" xmlns:table="urn:oasis:names:tc:opendocument:xmlns:table:1.0" xmlns:fo="urn:oasis:names:tc:opendocument:xmlns:xsl-fo-compatible:1.0""#; /// A package with `content.xml` and, when given, `styles.xml`. @@ -2819,6 +2980,87 @@ mod tests { package(&content, Some(&styles)) } + fn part_text(d: &Document, id: Option) -> String { + d.plain_text(wordcraft_doc::StoryRef::Part(id.expect("part"))) + } + + #[test] + fn headers_and_footers_follow_their_master_page() { + // #863: headers and footers live on the master pages in styles.xml. + let styles = format!( + r#" + + + + + + ACME Letterhead + Left page + First page + Page 1 + Hidden + + Wide header + "# + ); + let content = format!( + r#" + + + Upright bodyWide body + "# + ); + let d = import(&package(&content, Some(&styles))).unwrap(); + let secs = d.sections(); + assert_eq!(secs.len(), 2); + let (up, wide) = (&secs[0].1, &secs[1].1); + assert_eq!(part_text(&d, up.headers.default), "ACME Letterhead"); + assert_eq!(part_text(&d, up.headers.even), "Left page"); + assert_eq!(part_text(&d, up.headers.first), "First page"); + assert_eq!(part_text(&d, up.footers.default), "Page 1"); + assert!(up.title_page && d.settings.even_odd_headers); + // The hidden left footer: left pages show the right pages' footer. + assert_eq!((up.footers.even, up.footers.first), (up.footers.default, None)); + assert_eq!(part_text(&d, wide.headers.default), "Wide header"); + assert_eq!((wide.headers.even, wide.title_page), (wide.headers.default, false)); + // Its master page has no footer: an empty one, not the previous section's. + assert_eq!(part_text(&d, wide.footers.default), ""); + assert_eq!(d.parts.len(), 6); + let id = up.headers.default.unwrap(); + let Some(wordcraft_doc::Block::Para(p)) = d.parts[&id].blocks.first().map(|b| &**b) else { panic!("no header paragraph") }; + assert!(p.runs.iter().all(|r| r.props.bold == Some(true)), "{:?}", p.runs); + } + + #[test] + fn documents_without_headers_get_none() { + let content = format!( + r#"Body"# + ); + let styles = format!( + r#""# + ); + for d in [import(&package(&content, Some(&styles))).unwrap(), import(&package(&content, None)).unwrap()] { + assert!(d.parts.is_empty()); + assert_eq!(d.last_section.headers, Default::default()); + assert!(!d.last_section.title_page && !d.settings.even_odd_headers); + } + } + + #[test] + fn deeply_nested_header_content_does_not_panic() { + let n = 4 * MAX_XML_DEPTH; + let (open, close) = ("".repeat(n), "".repeat(n)); + let styles = format!( + r#"{open}Deep{close}After"# + ); + let content = format!( + r#"Body"# + ); + let d = import(&package(&content, Some(&styles))).unwrap(); + assert!(d.last_section.headers.default.is_some()); + assert_eq!(part_text(&d, d.last_section.footers.default), "After"); + } + #[test] fn master_pages_and_column_sections_become_sections() { use wordcraft_doc::section::SectionStart; diff --git a/crates/formats/src/odt/page_parts.rs b/crates/formats/src/odt/page_parts.rs new file mode 100644 index 000000000..0a142590e --- /dev/null +++ b/crates/formats/src/odt/page_parts.rs @@ -0,0 +1,156 @@ +//! Headers and footers of the master pages in `styles.xml` (ODF 1.3 §16.10), read with the body +//! reader (#863). +use std::collections::HashSet; + +use wordcraft_doc::{Part, PartKind}; + +use super::*; + +/// Most XML events kept for all headers and footers together; a slot past it is left out. +const MAX_EVENTS: usize = 1 << 20; + +#[derive(Clone, Copy, PartialEq, Eq)] +enum Slot { + Header, + HeaderLeft, + HeaderFirst, + Footer, + FooterLeft, + FooterFirst, +} + +impl Slot { + fn of(q: &str) -> Option { + Some(match q { + "style:header" => Self::Header, + "style:header-left" => Self::HeaderLeft, + "style:header-first" | "loext:header-first" => Self::HeaderFirst, + "style:footer" => Self::Footer, + "style:footer-left" => Self::FooterLeft, + "style:footer-first" | "loext:footer-first" => Self::FooterFirst, + _ => return None, + }) + } + + fn footer(self) -> bool { + matches!(self, Self::Footer | Self::FooterLeft | Self::FooterFirst) + } + + fn set(self, sec: &mut SectionProps, id: u32) { + let set = if self.footer() { &mut sec.footers } else { &mut sec.headers }; + match self { + Self::Header | Self::Footer => set.default = Some(id), + Self::HeaderLeft | Self::FooterLeft => set.even = Some(id), + Self::HeaderFirst | Self::FooterFirst => { + set.first = Some(id); + sec.title_page = true; + } + } + } +} + +/// The headers and footers read, in part-id order from 1, and whether one is for left pages. +#[derive(Default)] +pub(super) struct Parts { + parts: Vec<(PartKind, Vec)>, + even_odd: bool, +} + +/// Read each visible header and footer of the master pages in `pages`, and name it on its master +/// page's section as part id 1, 2, … in order, so the sections the body starts carry them. +/// [`add`] puts those parts into the document. +pub(super) fn read(xml: &[u8], styles: &Styles, pages: &mut Pages, fetch: &mut dyn FnMut(&str) -> Option>) -> Parts { + let mut found: Vec<(String, Slot, Vec)> = Vec::new(); + let mut master: Option = None; + // The slot being read, its nesting depth and whether it is kept. + let mut slot: Option<(usize, bool)> = None; + let mut seen = HashSet::new(); + let mut room = MAX_EVENTS; + events(xml, |ev| { + if let Some((depth, keep)) = &mut slot { + match &ev { + Ev::End(_) if *depth == 0 => { + slot = None; + return; + } + Ev::Start(..) => *depth += 1, + Ev::End(_) => *depth -= 1, + Ev::Text(_) => {} + } + if *keep && room == 0 { + *keep = false; + found.pop(); + } + if *keep && let Some((.., evs)) = found.last_mut() { + room -= 1; + evs.push(ev); + } + return; + } + match ev { + Ev::Start(q, a) if q == "style:master-page" => { + master = get(&a, "style:name").filter(|n| pages.masters.contains_key(*n)).map(str::to_string); + } + Ev::End(q) if q == "style:master-page" => master = None, + Ev::Start(q, a) => { + if let (Some(m), Some(kind)) = (&master, Slot::of(&q)) { + let keep = get(&a, "style:display") != Some("false") && seen.insert((m.clone(), q.clone())); + if keep { + found.push((m.clone(), kind, Vec::new())); + } + slot = Some((0, keep)); + } + } + _ => {} + } + }); + let none = Pages::default(); + let mut out = Parts::default(); + for (name, kind, evs) in found { + let mut reader = Body::new(styles, &none, fetch); + reader.in_text = true; + for ev in evs { + reader.feed(ev); + } + out.parts.push((if kind.footer() { PartKind::Footer } else { PartKind::Header }, reader.take_blocks())); + if let Some(sec) = pages.masters.get_mut(&name) { + kind.set(sec, out.parts.len() as u32); + } + out.even_odd |= matches!(kind, Slot::HeaderLeft | Slot::FooterLeft); + } + // Each master page stands alone, but a section without its own header or footer continues + // the previous one's: give those an empty one, and left pages the right pages' one. + for footer in [false, true] { + let pick = |s: &SectionProps| if footer { s.footers } else { s.headers }; + if !pages.masters.values().any(|s| pick(s).default.is_some() || pick(s).even.is_some()) { + continue; + } + let mut empty = None; + for sec in pages.masters.values_mut() { + let set = if footer { &mut sec.footers } else { &mut sec.headers }; + if set.default.is_none() { + set.default = Some(*empty.get_or_insert_with(|| { + out.parts.push((if footer { PartKind::Footer } else { PartKind::Header }, Vec::new())); + out.parts.len() as u32 + })); + } + if out.even_odd && set.even.is_none() { + set.even = set.default; + } + } + } + if let Some(first) = pages.first_name.as_ref().and_then(|n| pages.masters.get(n)) { + pages.first = Some(first.clone()); + } + out +} + +/// Put the parts [`read`] found into `doc` under the ids its sections name. `doc` comes from +/// [`model::to_doc`], which adds no parts of its own. +pub(super) fn add(doc: &mut Document, parts: Parts) { + for (i, (kind, blocks)) in parts.parts.into_iter().enumerate() { + let blocks = model::part_blocks(doc, &blocks); + doc.parts.insert(i as u32 + 1, Part { kind, blocks, body: Default::default() }); + } + doc.settings.even_odd_headers |= parts.even_odd; +} diff --git a/crates/formats/src/tests.rs b/crates/formats/src/tests.rs index caad39dce..bf6f59264 100644 --- a/crates/formats/src/tests.rs +++ b/crates/formats/src/tests.rs @@ -303,6 +303,21 @@ fn txt_export_writes_symbol_font_text_as_unicode() { assert_eq!(String::from_utf8(out).unwrap(), "Done \u{263A}\u{2713}"); } +/// #1080: `txt` export (and so `wordcraft-cli text`) keeps list labels, so a list item reads +/// apart from an ordinary paragraph. +#[test] +fn txt_export_keeps_list_labels() { + let mut d = Document::new(); + let bul = d.numbering.add_list(wordcraft_doc::ListKind::Bullet); + let mut p = Paragraph::with_text("alpha", CharProps::default()); + p.props.numbering = Some(wordcraft_doc::props::NumRef { num: bul, level: 0 }); + d.body = vec![para_block(p), para_block(Paragraph::with_text("plain", CharProps::default()))]; + let out = String::from_utf8(export("txt", &d).unwrap().unwrap()).unwrap(); + let mut lines = out.split("\r\n"); + assert!(lines.next().is_some_and(|l| l.contains("alpha") && !l.starts_with("alpha")), "{out:?}"); + assert_eq!(lines.next(), Some("plain")); +} + /// #569: a file with an extension we don't know opens as text when it is text (a playlist), and /// a binary file is told apart. #[test] diff --git a/crates/layout/src/para.rs b/crates/layout/src/para.rs index 6fdb28ac2..67ea269b9 100644 --- a/crates/layout/src/para.rs +++ b/crates/layout/src/para.rs @@ -1807,13 +1807,15 @@ pub fn layout_para(p: &Paragraph, env: &ParaEnv) -> ParaLayout { // A Symbol/Wingdings bullet (U+F0B7…) is drawn as its Unicode equivalent when the font // isn't installed (shaping maps it); the label's text reads as that character too. let shown = |t: &str| -> String { wordcraft_doc::symbol_fonts::map_text(&rc.font, t) }; + // A right-to-left paragraph's label reads right to left too ("1." shows as ".1"), and its + // brackets are mirrored (UAX #9 L4), which the shaper does for the runs the levels report + // as right to left ("(" drawn as ")"), so "(1)" reads "(1)" and not ")1(". + let lv = if rp.bidi { bidi_levels(text, true) } else { Vec::new() }; let before_c = b.clusters.len(); let before_g = b.glyphs.len(); - b.shape_with(text, 0, &rc, None, Dir::Guess); + b.shape_with(text, 0, &rc, None, if lv.is_empty() { Dir::Guess } else { Dir::Own(&lv) }); let mut cl: Vec = b.clusters.drain(before_c..).collect(); - // A right-to-left paragraph's label reads right to left too ("1." shows as ".1"). - if rp.bidi { - let lv = bidi_levels(text, true); + if !lv.is_empty() { let cl_lv: Vec = cl.iter().map(|c| lv.get(c.start).copied().unwrap_or(1)).collect(); let order = visual_order(&cl_lv); cl = order.iter().filter_map(|&i| cl.get(i).cloned()).collect(); diff --git a/crates/layout/src/tests_bidi.rs b/crates/layout/src/tests_bidi.rs index 5d20141f6..ab26eca16 100644 --- a/crates/layout/src/tests_bidi.rs +++ b/crates/layout/src/tests_bidi.rs @@ -561,3 +561,24 @@ fn list_label_moves_with_its_text_when_the_paragraph_is_aligned() { } } } + +/// A numbered label's brackets are mirrored in a right-to-left paragraph (UAX #9 L4), so a +/// `(%1)` label reads `(1)` and not `)1(`. +#[test] +fn rtl_list_label_mirrors_its_brackets() { + let mut d = doc_of(&[("סעיף", true)]); + let lvl = wordcraft_doc::numbering::Level { text: "(%1)".into(), ..Default::default() }; + let num = d.numbering.try_add_levels(None, vec![lvl]).unwrap(); + d.format_paragraphs(&Pos::body(0, 0), &Pos::body(0, 0), &|p: &mut ParaProps| p.numbering = Some(wordcraft_doc::props::NumRef { num, level: 0 })) + .unwrap(); + let l = lay(&d); + let (pl, _) = first_line(&l, 0); + let lab = pl.label.as_ref().expect("numbered"); + assert_eq!(lab.text, "(1)"); + let st = &pl.styles[lab.style as usize]; + let (open, digit, close) = (st.face.glyph_for('('), st.face.glyph_for('1'), st.face.glyph_for(')')); + let mut drawn: Vec<(f32, u32)> = lab.glyphs.iter().map(|g| (g.dx, g.gid)).collect(); + drawn.sort_by(|a, b| a.0.partial_cmp(&b.0).unwrap()); + let gids: Vec = drawn.iter().map(|(_, gid)| *gid).collect(); + assert_eq!(gids, vec![open, digit, close], "left to right, the label reads (1)"); +} diff --git a/crates/ui-egui/src/backstage.rs b/crates/ui-egui/src/backstage.rs index 1bcda1651..48edbea20 100644 --- a/crates/ui-egui/src/backstage.rs +++ b/crates/ui-egui/src/backstage.rs @@ -57,14 +57,13 @@ pub fn show(app: &mut WordApp, ui: &mut Ui) { icons::paint(ui.painter(), Rect::from_center_size(c, vec2(18.0, 18.0)), "chevronLeft", egui::Color32::WHITE, egui::Color32::WHITE); resp } else { - let (r, resp) = ui.allocate_exact_size(vec2(200.0, 40.0), Sense::click()); - icons::paint( - ui.painter(), - Rect::from_center_size(pos2(r.min.x + 26.0, r.center().y), vec2(18.0, 18.0)), - "chevronLeft", - egui::Color32::WHITE, - egui::Color32::WHITE, - ); + // The hit area is scoped to the chevron, not the whole 200px-wide row (#3041): + // the row reads as plain sidebar background next to it, so clicking there must + // not also close the File page, same as the macOS chevron fix (#2972) below. + let (r, _) = ui.allocate_exact_size(vec2(200.0, 40.0), Sense::hover()); + let c = pos2(r.min.x + 26.0, r.center().y); + let resp = ui.interact(Rect::from_center_size(c, vec2(22.0, r.height())), ui.id().with("backstage_back"), Sense::click()); + icons::paint(ui.painter(), Rect::from_center_size(c, vec2(18.0, 18.0)), "chevronLeft", egui::Color32::WHITE, egui::Color32::WHITE); resp }; // Escape in the built-in file browser closes only the browser. @@ -1123,7 +1122,7 @@ mod tests { h.state_mut().run("ui.backstage", json!({"value": true})).unwrap(); h.run_steps(4); // The arrow heads the File page's side bar; it has no label to find it by. - let arrow = egui::pos2(100.0, 32.0); + let arrow = egui::pos2(26.0, 32.0); h.event(egui::Event::PointerMoved(arrow)); for pressed in [true, false] { h.event(egui::Event::PointerButton { pos: arrow, button: egui::PointerButton::Primary, pressed, modifiers: egui::Modifiers::NONE }); @@ -1133,6 +1132,19 @@ mod tests { assert!(h.state().canvas.focused, "and the page has the keyboard"); } + /// #3041: clicking the plain sidebar background next to the back arrow must not also close + /// the File page — only the chevron itself does, same scoping as the macOS chevron (#2972). + #[test] + fn clicking_beside_the_back_arrow_does_not_close_the_file_page() { + let a = WordApp::new(wordcraft_engine::Session::new(wordcraft_doc::Document::new()), Services::default()); + let mut h = a11y_harness(a); + h.state_mut().run("ui.backstage", json!({"value": true})).unwrap(); + h.run_steps(4); + // Well clear of the chevron (centred at x=26) but still inside the old 200px-wide row. + click_at(&mut h, egui::pos2(100.0, 32.0)); + assert!(h.state().ui.backstage, "clicking beside the arrow leaves the File page open"); + } + /// A harness over the whole app; kittest always runs AccessKit, so a frame that names as /// focused a widget it didn't draw panics with "Focused ID … is not in the node list". fn a11y_harness(a: WordApp) -> egui_kittest::Harness<'static, WordApp> { @@ -1229,7 +1241,7 @@ mod tests { a.ui.start_screen = true; assert!(a.start_screen()); } - let back = if mac { mac_chevron(&a) } else { egui::pos2(100.0, 32.0) }; + let back = if mac { mac_chevron(&a) } else { egui::pos2(26.0, 32.0) }; let mut h = a11y_harness(a); if !start { h.state_mut().run("ui.backstage", json!({"value": true})).unwrap(); diff --git a/crates/ui-egui/src/dialogs.rs b/crates/ui-egui/src/dialogs.rs index ae635970d..d7d695600 100644 --- a/crates/ui-egui/src/dialogs.rs +++ b/crates/ui-egui/src/dialogs.rs @@ -2597,7 +2597,11 @@ fn body(app: &mut WordApp, ui: &mut Ui, d: &mut Dialog) -> bool { ui.ctx().input_mut(|i| i.events.retain(|e| !swallow_shortcut_text(swallow, e))); } let r = ui.add(egui::TextEdit::singleline(query).hint_text(tl!("Type a command, e.g. \"insert table\"")).desired_width(380.0)); - r.request_focus(); + // Only when it lacks focus: `request_focus` always interrupts IME, and on X11 that + // recreates the XIM context, so asking every frame left fcitx taking no keys (#2986). + if !r.has_focus() { + r.request_focus(); + } let reg = app.session.registry.clone(); // An empty box suggests a few common commands, as Word's Search does. let mut hits: Vec<&wordcraft_engine::CommandSpec> = if query.trim().is_empty() { @@ -3822,7 +3826,10 @@ fn goto_body(app: &mut WordApp, ui: &mut Ui, what: &mut GoToWhat, target: &mut S ui.vertical(|ui| match what { GoToWhat::Page => { ui.label(tl!("Enter page number:")); - ui.add(egui::TextEdit::singleline(target).desired_width(220.0)).request_focus(); + let r = ui.add(egui::TextEdit::singleline(target).desired_width(220.0)); + if !r.has_focus() { + r.request_focus(); + } ui.label(egui::RichText::new(tl!("Type +2 or -1 to move from the page you are on.")).weak()); } GoToWhat::Bookmark => { @@ -3899,7 +3906,10 @@ fn icons_body(app: &mut WordApp, ui: &mut Ui, query: &mut String, category: &mut let t = Tokens::get(ui.ctx()); ui.set_width(560.0); // Typing goes to the search box (never to the document behind the dialog). - ui.add(egui::TextEdit::singleline(query).hint_text(tl!("Search icons")).desired_width(560.0)).request_focus(); + let r = ui.add(egui::TextEdit::singleline(query).hint_text(tl!("Search icons")).desired_width(560.0)); + if !r.has_focus() { + r.request_focus(); + } ui.add_space(4.0); ui.horizontal_wrapped(|ui| { ui.spacing_mut().item_spacing = vec2(4.0, 4.0); @@ -4231,6 +4241,77 @@ mod tests { crate::i18n::set_current(crate::i18n::Lang::EN); } + /// #2986: a box that keeps the keyboard asks for focus only while it lacks it. `request_focus` + /// always interrupts IME; on X11 that recreates the XIM context, and Search asking every frame + /// left fcitx taking no keys at all. Once the box has focus, no frame interrupts, and typing lands; + /// after Tab takes the keyboard away, the box takes it back with one reset, not one per frame. + #[test] + fn boxes_that_keep_focus_do_not_interrupt_ime_every_frame() { + fn typed(d: Option<&Dialog>) -> Option<&str> { + match d? { + Dialog::Commands { query, .. } | Dialog::Icons { query, .. } => Some(query.as_str()), + Dialog::Find { goto_target, .. } => Some(goto_target.as_str()), + Dialog::Refs(r) => match r.as_ref() { + crate::dialogs_refs::RefsDialog::NewLabel(f) => Some(f.new_label.as_str()), + _ => None, + }, + _ => None, + } + } + let key = |key| egui::Event::Key { key, physical_key: None, pressed: true, repeat: false, modifiers: egui::Modifiers::NONE }; + for name in ["commands", "icons", "goto", "newLabel"] { + let mut app = app_with("Unchanged document"); + let dialog = match name { + "newLabel" => Dialog::Refs(Box::new(crate::dialogs_refs::RefsDialog::NewLabel(crate::dialogs_refs::CaptionForm::read(&app)))), + _ => Dialog::open(name, &mut app).unwrap(), + }; + app.dialog = Some(dialog); + let mut h = app_harness(app); + // The box takes the keyboard from the page in the first frames. + h.run_steps(4); + let interrupted = + |h: &egui_kittest::Harness<'_, WordApp>| h.output().platform_output.ime.as_ref().map(|i| i.should_interrupt_composition); + for frame in 0..6 { + h.step(); + assert_eq!(interrupted(&h), Some(false), "{name}: frame {frame} resets the input method"); + } + for (k, t) in [(egui::Key::A, "a"), (egui::Key::B, "b")] { + h.event(key(k)); + h.event(egui::Event::Text(t.into())); + h.step(); + assert_eq!(interrupted(&h), Some(false), "{name}: typing {t} resets the input method"); + } + assert_eq!(typed(h.state().dialog.as_ref()), Some("ab"), "{name}: the keys reach the box"); + + // Tab moves the keyboard to the dialog's next control and the box takes it back. As + // egui-winit does, count a reset when IME comes back on or is interrupted while on (each + // recreates the XIM context): one or two for the round trip, then none, and the next + // key reaches the box. + let reset = |h: &egui_kittest::Harness<'_, WordApp>, on: &mut bool| { + let ime = h.output().platform_output.ime.as_ref(); + let reset = ime.is_some_and(|i| !*on || i.should_interrupt_composition); + *on = ime.is_some(); + reset + }; + let mut on = true; + h.event(key(egui::Key::Tab)); + let mut back = 0; + for _ in 0..4 { + h.step(); + back += usize::from(reset(&h, &mut on)); + } + assert!((1..=2).contains(&back), "{name}: Tab and back reset the input method {back} times"); + for frame in 0..4 { + h.step(); + assert!(!reset(&h, &mut on), "{name}: frame {frame} after Tab resets the input method"); + } + h.event(key(egui::Key::C)); + h.event(egui::Event::Text("c".into())); + h.step(); + assert_eq!(typed(h.state().dialog.as_ref()), Some("abc"), "{name}: the box has the keyboard back after Tab"); + } + } + /// Open the Paragraph dialog and press OK without changing anything. fn ok_unchanged(app: &mut WordApp) { ok_edit(app, |_| {}); diff --git a/crates/ui-egui/src/dialogs_para.rs b/crates/ui-egui/src/dialogs_para.rs index fb34b1f2e..03575bcf6 100644 --- a/crates/ui-egui/src/dialogs_para.rs +++ b/crates/ui-egui/src/dialogs_para.rs @@ -323,7 +323,7 @@ pub fn tabs(app: &mut WordApp, ui: &mut Ui, f: &mut TabsForm) -> bool { { // `1,5 cm; 3 cm` where the comma is the decimal separator. let sep = if crate::options::decimal() == ',' { "; " } else { ", " }; - f.cleared.iter().map(|p| u.format_with(*p, crate::options::decimal())).collect::>().join(sep) + f.cleared.iter().map(|p| crate::options::length_in(u, *p)).collect::>().join(sep) } }; ui.label(egui::RichText::new(list).weak()); diff --git a/crates/ui-egui/src/dialogs_refs.rs b/crates/ui-egui/src/dialogs_refs.rs index 6b511f54f..580f30437 100644 --- a/crates/ui-egui/src/dialogs_refs.rs +++ b/crates/ui-egui/src/dialogs_refs.rs @@ -484,7 +484,9 @@ fn numbering_ui(ui: &mut Ui, f: &mut CaptionForm, was: &CaptionForm) -> Next { fn new_label_ui(app: &mut WordApp, ui: &mut Ui, f: &mut CaptionForm) -> Next { ui.label(tl!("Label:")); let r = ui.add(egui::TextEdit::singleline(&mut f.new_label).desired_width(260.0)); - r.request_focus(); + if !r.has_focus() { + r.request_focus(); + } let (ok, cancel) = buttons(ui, tl!("OK")); let name: String = f.new_label.chars().filter(|c| !matches!(c, '"' | '\\') && !c.is_control()).take(40).collect::().trim().to_string(); if ok && !name.is_empty() { diff --git a/crates/ui-egui/src/i18n/bg.tsv b/crates/ui-egui/src/i18n/bg.tsv index 032467cf6..ed91388f2 100644 --- a/crates/ui-egui/src/i18n/bg.tsv +++ b/crates/ui-egui/src/i18n/bg.tsv @@ -1317,6 +1317,9 @@ {cols}×{rows} Table Таблица {cols}×{rows} AaBbYyZz AaBbYyZz pt пт + cm см + mm мм + pi pi Document1 Документ1 Username Потребителско име Display name Показвано име diff --git a/crates/ui-egui/src/i18n/cs.tsv b/crates/ui-egui/src/i18n/cs.tsv index 3a210cc85..01c5a67c3 100644 --- a/crates/ui-egui/src/i18n/cs.tsv +++ b/crates/ui-egui/src/i18n/cs.tsv @@ -1393,6 +1393,9 @@ {cols}×{rows} Table Tabulka {cols}×{rows} AaBbYyZz AaBbYyZz ČčŘřŽž pt b. + cm cm + mm mm + pi pi Document1 Dokument1 Username Uživatelské jméno Display name Zobrazované jméno diff --git a/crates/ui-egui/src/i18n/de.tsv b/crates/ui-egui/src/i18n/de.tsv index 3fe150cca..9a6246b9e 100644 --- a/crates/ui-egui/src/i18n/de.tsv +++ b/crates/ui-egui/src/i18n/de.tsv @@ -1298,6 +1298,9 @@ {cols}×{rows} Table {cols}×{rows}-Tabelle AaBbYyZz AaBbYyZz pt pt + cm cm + mm mm + pi pi Document1 Document1 Username Benutzername Display name Anzeigename diff --git a/crates/ui-egui/src/i18n/es.tsv b/crates/ui-egui/src/i18n/es.tsv index 3431633a9..9bae18304 100644 --- a/crates/ui-egui/src/i18n/es.tsv +++ b/crates/ui-egui/src/i18n/es.tsv @@ -1416,6 +1416,9 @@ {cols}×{rows} Table Tabla de {cols}×{rows} AaBbYyZz AaBbYyZz ÑñÁá pt pt + cm cm + mm mm + pi pi Document1 Documento1 Username Nombre de usuario Display name Nombre visible diff --git a/crates/ui-egui/src/i18n/et.tsv b/crates/ui-egui/src/i18n/et.tsv index 263f5dace..d0782ac1c 100644 --- a/crates/ui-egui/src/i18n/et.tsv +++ b/crates/ui-egui/src/i18n/et.tsv @@ -1241,6 +1241,9 @@ {cols}×{rows} Table Tabel {cols}×{rows} AaBbYyZz AaBbYyZz ÕõÄäÖöÜü pt pt + cm cm + mm mm + pi pi Document1 Dokument1 Username Kasutajanimi Display name Kuvatav nimi diff --git a/crates/ui-egui/src/i18n/fr.tsv b/crates/ui-egui/src/i18n/fr.tsv index 6ebff61ee..f2c34d0b0 100644 --- a/crates/ui-egui/src/i18n/fr.tsv +++ b/crates/ui-egui/src/i18n/fr.tsv @@ -1312,6 +1312,9 @@ {cols}×{rows} Table Tableau {cols}×{rows} AaBbYyZz AaBbYyZz pt pt + cm cm + mm mm + pi pi Document1 Document1 Username Nom d’utilisateur Display name Nom d’affichage diff --git a/crates/ui-egui/src/i18n/hr.tsv b/crates/ui-egui/src/i18n/hr.tsv index 404dcae7b..a0faaba73 100644 --- a/crates/ui-egui/src/i18n/hr.tsv +++ b/crates/ui-egui/src/i18n/hr.tsv @@ -2491,6 +2491,9 @@ preview Title Naslov n×n Identity Matrix Jedinična matrica n×n n×n Matrix Matrica n×n pt pt + cm cm + mm mm + pi pi select two or more cells to merge odaberite dvije ili više ćelija za spajanje tOGGLE cASE zAMIJENI vELIČINU sLOVA there is no ink to replay nema rukopisa za ponovnu reprodukciju diff --git a/crates/ui-egui/src/i18n/hu.tsv b/crates/ui-egui/src/i18n/hu.tsv index 0e2b13a52..bc056b5a8 100644 --- a/crates/ui-egui/src/i18n/hu.tsv +++ b/crates/ui-egui/src/i18n/hu.tsv @@ -1125,6 +1125,9 @@ Version Verzió % of all commits Az összes commit %-a pt pt + cm cm + mm mm + pi pi Save the document to turn on AutoSave Az automatikus mentéshez mentsd a dokumentumot Contributors Közreműködők Models Modellek diff --git a/crates/ui-egui/src/i18n/it.tsv b/crates/ui-egui/src/i18n/it.tsv index e2c47a0a4..3b52b94f6 100644 --- a/crates/ui-egui/src/i18n/it.tsv +++ b/crates/ui-egui/src/i18n/it.tsv @@ -1304,6 +1304,9 @@ {cols}×{rows} Table Tabella {cols}×{rows} AaBbYyZz AaBbYyZz ÀàÈé pt pt + cm cm + mm mm + pi pi Document1 Documento1 Username Nome utente Display name Nome visualizzato diff --git a/crates/ui-egui/src/i18n/ja.tsv b/crates/ui-egui/src/i18n/ja.tsv index 50a6a038c..71371d0f2 100644 --- a/crates/ui-egui/src/i18n/ja.tsv +++ b/crates/ui-egui/src/i18n/ja.tsv @@ -1377,6 +1377,9 @@ {cols}×{rows} Table {cols}×{rows} の表 AaBbYyZz AaBbYyZz あア亜 pt pt + cm cm + mm mm + pi pi Document1 文書 1 Username ユーザー名 Display name 表示名 diff --git a/crates/ui-egui/src/i18n/ko.tsv b/crates/ui-egui/src/i18n/ko.tsv index 0ef015a5e..b8b294045 100644 --- a/crates/ui-egui/src/i18n/ko.tsv +++ b/crates/ui-egui/src/i18n/ko.tsv @@ -1367,6 +1367,9 @@ {cols}×{rows} Table {cols}×{rows} 표 AaBbYyZz AaBbYyZz pt pt + cm cm + mm mm + pi pi Document1 문서1 Username 사용자 이름 Display name 표시 이름 diff --git a/crates/ui-egui/src/i18n/mod.rs b/crates/ui-egui/src/i18n/mod.rs index 092e37684..69ad981ae 100644 --- a/crates/ui-egui/src/i18n/mod.rs +++ b/crates/ui-egui/src/i18n/mod.rs @@ -256,6 +256,16 @@ impl Lang { LANGUAGES.iter().map(Lang) } + /// Every registered language in the order a picker lists them: English first (the source + /// language and the fallback), then the rest by their own name with accents folded, so + /// Čeština sits under C rather than after Z. Names in other scripts follow the Latin ones in + /// code point order: Greek, Cyrillic, Arabic, then the CJK languages. + pub fn for_picker() -> Vec { + let mut rest: Vec = Lang::all().filter(|l| l.code() != Lang::EN.code()).collect(); + rest.sort_by_cached_key(|l| name_sort_key(l.name())); + std::iter::once(Lang::EN).chain(rest).collect() + } + fn catalog(self) -> &'static Catalog { self.0.catalog() } @@ -474,3 +484,40 @@ mod audit; pub(crate) mod pseudo; #[cfg(test)] mod tests; + +/// `name` lowercased with the Latin accents that occur in language names folded away, so a +/// byte-wise sort reads alphabetically. Other scripts are left as they are. +fn name_sort_key(name: &str) -> String { + name.chars() + .flat_map(char::to_lowercase) + .map(|c| match c { + 'à'..='å' | 'ā' | 'ă' | 'ą' => 'a', + 'ç' | 'ć' | 'č' => 'c', + 'è'..='ë' | 'ē' | 'ě' | 'ę' | 'ế' | 'ệ' => 'e', + 'ì'..='ï' | 'ī' => 'i', + 'ñ' | 'ń' | 'ň' => 'n', + 'ò'..='ö' | 'ø' | 'ő' => 'o', + 'ś' | 'š' => 's', + 'ù'..='ü' | 'ū' | 'ů' | 'ű' => 'u', + 'ý' | 'ÿ' => 'y', + 'ź' | 'ż' | 'ž' => 'z', + c => c, + }) + .collect() +} + +#[cfg(test)] +mod picker_order_tests { + use super::*; + + #[test] + fn picker_lists_english_first_then_names_alphabetically() { + let order = Lang::for_picker(); + assert_eq!(order.first().map(|l| l.code()), Some(Lang::EN.code())); + assert_eq!(order.len(), Lang::all().count()); + let keys: Vec = order[1..].iter().map(|l| name_sort_key(l.name())).collect(); + assert!(keys.windows(2).all(|w| w[0] <= w[1]), "{keys:?}"); + assert_eq!(name_sort_key("Čeština"), "cestina"); + assert_eq!(name_sort_key("Türkçe"), "turkce"); + } +} diff --git a/crates/ui-egui/src/i18n/pl.tsv b/crates/ui-egui/src/i18n/pl.tsv index 9d29c2a55..2563a89f4 100644 --- a/crates/ui-egui/src/i18n/pl.tsv +++ b/crates/ui-egui/src/i18n/pl.tsv @@ -1365,6 +1365,9 @@ Width and spacing Szerokość i odstępy from Unicode (hex) z Unicode (szesnastkowo) pt pt + cm cm + mm mm + pi pi {cols}×{rows} Table Tabela {cols}×{rows} {contributors} contributors · {commits} commits Współtwórcy: {contributors} · commity: {commits} Copyright Sign Znak praw autorskich diff --git a/crates/ui-egui/src/i18n/pt-br.tsv b/crates/ui-egui/src/i18n/pt-br.tsv index 2f099becf..d5b9ccdfe 100644 --- a/crates/ui-egui/src/i18n/pt-br.tsv +++ b/crates/ui-egui/src/i18n/pt-br.tsv @@ -1389,6 +1389,9 @@ {cols}×{rows} Table Tabela {cols}×{rows} AaBbYyZz AaBbYyZz ÇçÃã pt pt + cm cm + mm mm + pi pi Document1 Documento1 Username Nome de usuário Display name Nome de exibição diff --git a/crates/ui-egui/src/i18n/ru.tsv b/crates/ui-egui/src/i18n/ru.tsv index b9c89ce61..a910213c4 100644 --- a/crates/ui-egui/src/i18n/ru.tsv +++ b/crates/ui-egui/src/i18n/ru.tsv @@ -1138,6 +1138,9 @@ preview Title Пример Version Версия % of all commits % всех коммитов pt пт + cm см + mm мм + pi пик Save the document to turn on AutoSave Сохраните документ, чтобы включить автосохранение Contributors Участники Models Модели diff --git a/crates/ui-egui/src/i18n/sr-latn.tsv b/crates/ui-egui/src/i18n/sr-latn.tsv index b492b07d3..60b724569 100644 --- a/crates/ui-egui/src/i18n/sr-latn.tsv +++ b/crates/ui-egui/src/i18n/sr-latn.tsv @@ -1310,6 +1310,9 @@ {cols}×{rows} Table Tabela {cols}×{rows} AaBbYyZz AaBbYyZz ČčĐđ pt pt + cm cm + mm mm + pi pi Document1 Dokument1 Username Korisničko ime Display name Ime za prikaz diff --git a/crates/ui-egui/src/i18n/sr.tsv b/crates/ui-egui/src/i18n/sr.tsv index 8a0cff04e..176fa1ebb 100644 --- a/crates/ui-egui/src/i18n/sr.tsv +++ b/crates/ui-egui/src/i18n/sr.tsv @@ -1314,6 +1314,9 @@ {cols}×{rows} Table Табела {cols}×{rows} AaBbYyZz AaBbYyZz ЂђЋћ pt пт + cm цм + mm мм + pi pi Document1 Документ1 Username Корисничко име Display name Име за приказ diff --git a/crates/ui-egui/src/i18n/sv.tsv b/crates/ui-egui/src/i18n/sv.tsv index ea569c189..0c46ba8c8 100644 --- a/crates/ui-egui/src/i18n/sv.tsv +++ b/crates/ui-egui/src/i18n/sv.tsv @@ -1406,6 +1406,9 @@ {cols}×{rows} Table Tabell {cols}×{rows} AaBbYyZz AaBbÅåÖö pt pt + cm cm + mm mm + pi pi Document1 Dokument1 Username Användarnamn Display name Visningsnamn diff --git a/crates/ui-egui/src/i18n/tr.tsv b/crates/ui-egui/src/i18n/tr.tsv index 17859beae..4080f08da 100644 --- a/crates/ui-egui/src/i18n/tr.tsv +++ b/crates/ui-egui/src/i18n/tr.tsv @@ -1138,6 +1138,9 @@ preview Title Örnek Version Sürüm % of all commits Commit payı (%) pt pt + cm cm + mm mm + pi pi Save the document to turn on AutoSave Otomatik Kaydet'i açmak için belgeyi kaydedin Models Modeller Good evening İyi akşamlar diff --git a/crates/ui-egui/src/i18n/uk.tsv b/crates/ui-egui/src/i18n/uk.tsv index 859b28e75..bb6ca7efd 100644 --- a/crates/ui-egui/src/i18n/uk.tsv +++ b/crates/ui-egui/src/i18n/uk.tsv @@ -1389,6 +1389,9 @@ {cols}×{rows} Table Таблиця {cols}×{rows} AaBbYyZz AaBbYyZz ЄєЇї pt пт + cm см + mm мм + pi pi Document1 Документ1 Username Ім'я користувача Display name Відображуване ім'я diff --git a/crates/ui-egui/src/i18n/vi.tsv b/crates/ui-egui/src/i18n/vi.tsv index a1af28ce8..ced8dcc2f 100644 --- a/crates/ui-egui/src/i18n/vi.tsv +++ b/crates/ui-egui/src/i18n/vi.tsv @@ -1309,6 +1309,9 @@ {cols}×{rows} Table Bảng {cols}×{rows} AaBbYyZz AaBbYyZz pt pt + cm cm + mm mm + pi pi Document1 Document1 Username Tên đăng nhập Display name Tên hiển thị diff --git a/crates/ui-egui/src/i18n/zh-hans.tsv b/crates/ui-egui/src/i18n/zh-hans.tsv index c081e34bb..1304b7c6c 100644 --- a/crates/ui-egui/src/i18n/zh-hans.tsv +++ b/crates/ui-egui/src/i18n/zh-hans.tsv @@ -1389,6 +1389,9 @@ {cols}×{rows} Table {cols}×{rows} 表格 AaBbYyZz AaBbYyZz 中文 pt 磅 + cm 厘米 + mm 毫米 + pi 派卡 Document1 文档1 Username 用户名 Display name 显示名称 diff --git a/crates/ui-egui/src/i18n/zh-hant.tsv b/crates/ui-egui/src/i18n/zh-hant.tsv index 0780f108f..655c579ca 100644 --- a/crates/ui-egui/src/i18n/zh-hant.tsv +++ b/crates/ui-egui/src/i18n/zh-hant.tsv @@ -1378,6 +1378,9 @@ {cols}×{rows} Table {cols}×{rows} 表格 AaBbYyZz AaBbYyZz 中文 pt 點 + cm 公分 + mm 公釐 + pi 派卡 Document1 文件1 Username 使用者名稱 Display name 顯示名稱 diff --git a/crates/ui-egui/src/options.rs b/crates/ui-egui/src/options.rs index 2962187d6..5b7542bc7 100644 --- a/crates/ui-egui/src/options.rs +++ b/crates/ui-egui/src/options.rs @@ -35,9 +35,44 @@ pub fn decimal() -> char { DECIMAL.get() } -/// `pt` in [`unit`], written the region's way (`2,54 cm`). +/// `pt` in [`unit`], written the region's way (`2,54 cm`), with the unit in the interface +/// language (`2,54 см` in Russian). pub fn length(pt: f32) -> String { - unit().format_with(pt, decimal()) + length_in(unit(), pt) +} + +/// [`length`] in a given unit. +pub fn length_in(unit: Unit, pt: f32) -> String { + format!("{}{}", unit.number(pt, decimal()), unit_suffix(unit)) +} + +/// The unit's suffix as the interface shows it: ` cm`, ` mm`, ` pt` and ` pi` go through the +/// catalog (` см` in Russian, #1734); inches keep `"`. +pub fn unit_suffix(unit: Unit) -> String { + match unit { + Unit::Inches => unit.suffix().to_string(), + Unit::Centimeters => format!(" {}", tl!("cm")), + Unit::Millimeters => format!(" {}", tl!("mm")), + Unit::Points => format!(" {}", tl!("pt")), + Unit::Picas => format!(" {}", tl!("pi")), + } +} + +/// A length typed into a field in `unit`, as a number of that unit: either decimal separator, +/// another unit (`1 in` in a centimetre field), and the units' names in the interface language +/// as well as in English (`3 см`). +pub fn parse_length(text: &str, unit: Unit) -> Option { + let mut text: String = text.chars().filter(|c| !c.is_whitespace()).collect::().to_lowercase(); + for (english, translated) in [("cm", tl!("cm")), ("mm", tl!("mm")), ("pt", tl!("pt")), ("pi", tl!("pi"))] { + let translated = translated.to_lowercase(); + if translated != english + && let Some(number) = text.strip_suffix(translated.as_str()) + { + text = format!("{number}{english}"); + break; + } + } + wordcraft_geom::parse_measure(&text, unit).map(|pt| f64::from(pt / unit.pt_per_unit())) } /// Copy the session's settings the UI reads without the app. @@ -59,7 +94,7 @@ pub trait MeasureExt { impl MeasureExt for egui::DragValue<'_> { fn measure(self, unit: Unit) -> Self { let sep = decimal(); - self.suffix(unit.suffix()) + self.suffix(unit_suffix(unit)) .custom_formatter(move |n, decimals| { let s = egui::emath::format_with_decimals_in_range(n, decimals); if sep == '.' { s } else { s.replace('.', &sep.to_string()) } @@ -71,12 +106,6 @@ impl MeasureExt for egui::DragValue<'_> { } } -/// A length typed in a field, in `unit`s. -fn parse_length(text: &str, unit: Unit) -> Option { - let text: String = text.chars().filter(|c| !c.is_whitespace()).collect(); - wordcraft_geom::parse_measure(&text, unit).map(|pt| f64::from(pt / unit.pt_per_unit())) -} - /// Regions offered in File › Options › Advanced › Region (any locale tag works through /// `file.options`): Letter and A4 countries, decimal point and decimal comma. const REGIONS: [&str; 16] = @@ -672,7 +701,7 @@ fn language_picker(app: &mut WordApp, ui: &mut Ui) { if ui.selectable_label(app.ui.language == AUTO, auto_label.as_str()).clicked() { let _ = app.run("ui.language", json!({"value": AUTO})); } - for lang in Lang::all() { + for lang in Lang::for_picker() { if ui.selectable_label(app.ui.language == lang.code(), lang.name()).clicked() { let _ = app.run("ui.language", json!({"value": lang.code()})); } @@ -771,7 +800,7 @@ mod tests { use serde_json::json; use wordcraft_geom::Unit; - use super::{decimal, length, unit}; + use super::{decimal, length, parse_length, unit}; use crate::theme::Appearance; use crate::{Services, UiState, WordApp}; @@ -1160,4 +1189,40 @@ mod tests { h.run_steps(4); assert_eq!(length(72.0), "1\""); } + + /// #1734: a Russian interface shows centimetres as `см`, as Word does, and a length field + /// takes the unit typed either way. + #[test] + fn length_units_follow_the_interface_language() { + let mut a = app(); + a.run("ui.language", json!({"value": "ru"})).unwrap(); + a.run("file.options", json!({"region": "ru-RU"})).unwrap(); + a.run("file.new", json!({})).unwrap(); + let mut h = egui_kittest::Harness::builder().with_size(egui::vec2(1440.0, 900.0)).build_ui_state( + |ui, app: &mut WordApp| { + let ctx = ui.ctx().clone(); + app.logic(&ctx); + app.ui(ui); + }, + a, + ); + h.run_steps(2); + h.state_mut().run("ui.dialog", json!({"name": "pageSetup"})).unwrap(); + h.run_steps(4); + assert_eq!((unit(), decimal()), (Unit::Centimeters, ',')); + assert_eq!(length(72.0), "2,54 см"); + assert_eq!(h.query_all_by_value("2,54 см").count(), 4, "four margins in centimetres, named in Russian"); + assert_eq!(h.query_all_by_value("2,54 cm").count(), 0); + assert_eq!(parse_length("3 см", Unit::Centimeters), Some(3.0)); + assert_eq!(parse_length("3cm", Unit::Centimeters), Some(3.0)); + assert_eq!(parse_length("10 мм", Unit::Centimeters).map(|v| (v * 100.0).round() / 100.0), Some(1.0)); + assert_eq!(parse_length("72 пт", Unit::Centimeters).map(|v| (v * 100.0).round() / 100.0), Some(2.54)); + + h.state_mut().run("file.options", json!({"units": "millimeters"})).unwrap(); + h.run_steps(4); + assert_eq!(length(72.0), "25,4 мм"); + h.state_mut().run("ui.language", json!({"value": "en"})).unwrap(); + h.run_steps(4); + assert_eq!(length(72.0), "25,4 mm", "English keeps the English names"); + } } diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index b5224965e..afeb37acc 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -1,6 +1,6 @@ # File-format parity with Microsoft Word -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-19 · **Change:** trivial (shape gradients and transparency, #1500; previously trivial: RTF import compares heading and code formatting with the file's stylesheet; previously trivial: text typed into a content control takes the control's formatting; localized placeholder styles recognised; previously trivial: forms protection keeps its content controls: no inserting, Properties or Remove Content Control; previously trivial: Word XML and web pages saved as `.doc` open by what they hold, #2853; previously trivial: a VML shape in a header cell is always placed on the page, user-checked in Word for Mac, #1775; previously trivial: `w:sym` characters keep their symbol font when the text's font changes and are written back as `w:sym`, #2036; previously trivial: RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: two lines in one and horizontal in vertical (`w:eastAsianLayout`) kept on save; previously trivial: PDF outline targets follow leading page breaks; previously trivial: positional tabs `w:ptab` read and kept on save, #2367; previously trivial: DOCX: mixed right-to-left runs split and marked `w:rtl` with their language, #1543; previously trivial: `w:doNotTrackMoves` round-trips and turns move tracking off; previously trivial: legacy outline, shadow, emboss and imprint text effects drawn, #2534; previously trivial: DOCX glossary documents (building blocks) read, kept and written; previously trivial: cached field results keep their visible run formatting; previously trivial: cached REF results retain hyperlink destinations; previously trivial: CLI conversions can explicitly refresh fields; previously trivial: a copied equation also goes in the clipboard's MathML formats, #1297; previously trivial: a pasted HTML `
` is an indented paragraph, not the Quote style, #1270); previously trivial: every table style part round-trips, #714; previously trivial: VML freeforms read and kept on save; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: `flowChartOr` draws its circle and cross spokes, as in Word; previously trivial: RTF: Mac OS Roman and DOS text, #1653; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: PDF export draws horizontal and vertical lines inside groups, #1474; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-19 · **Change:** trivial (native .json stores pictures and kept parts as base64, #586 follow-up; previously trivial: ODT headers and footers read from each master page, #863; previously trivial: shape gradients and transparency, #1500; previously trivial: RTF import compares heading and code formatting with the file's stylesheet; previously trivial: text typed into a content control takes the control's formatting; localized placeholder styles recognised; previously trivial: forms protection keeps its content controls: no inserting, Properties or Remove Content Control; previously trivial: Word XML and web pages saved as `.doc` open by what they hold, #2853; previously trivial: a VML shape in a header cell is always placed on the page, user-checked in Word for Mac, #1775; previously trivial: `w:sym` characters keep their symbol font when the text's font changes and are written back as `w:sym`, #2036; previously trivial: RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: two lines in one and horizontal in vertical (`w:eastAsianLayout`) kept on save; previously trivial: PDF outline targets follow leading page breaks; previously trivial: positional tabs `w:ptab` read and kept on save, #2367; previously trivial: DOCX: mixed right-to-left runs split and marked `w:rtl` with their language, #1543; previously trivial: `w:doNotTrackMoves` round-trips and turns move tracking off; previously trivial: legacy outline, shadow, emboss and imprint text effects drawn, #2534; previously trivial: DOCX glossary documents (building blocks) read, kept and written; previously trivial: cached field results keep their visible run formatting; previously trivial: cached REF results retain hyperlink destinations; previously trivial: CLI conversions can explicitly refresh fields; previously trivial: a copied equation also goes in the clipboard's MathML formats, #1297; previously trivial: a pasted HTML `
` is an indented paragraph, not the Quote style, #1270); previously trivial: every table style part round-trips, #714; previously trivial: VML freeforms read and kept on save; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: `flowChartOr` draws its circle and cross spokes, as in Word; previously trivial: RTF: Mac OS Roman and DOS text, #1653; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: PDF export draws horizontal and vertical lines inside groups, #1474; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats Every format Word opens or saves, what WordCraft does with it, and how it's tested. Fidelity is the share of a typical real-world file's content and formatting that survives (read: shows correctly; @@ -42,7 +42,7 @@ Weights are the share of a Word user's file traffic (estimate). Read/write: ✅ | PDF export | 8% | W | — | ✅ | ~80% | 6–10 | `crates/pdf/src/tests.rs` | krilla: embedded subsets, selectable text incl. field results (#133); Hebrew and Arabic copy and search as whole words (right-to-left glyphs written in visual order, #1383); All Caps and Small Caps text extracts as the capitals shown, as from Word's PDFs, and never changes the case of other text (#1473), links, outline, tagging. Outline destinations follow leading page breaks to the heading text. Fonts it can't subset become outlines (#132). Print Markup (#589): comments in balloons beside each page (scaled to its paper) and tracked changes; HTML/Markdown/text copies end with a comment list. Horizontal and vertical lines inside groups are drawn (#1474). No PDF/A option, no "best for printing/online" choice | | PDF open (PDF Reflow, Word for Windows) | 1% | R | ❌ | — | 0% | 30–50 | — | Converting a PDF into an editable document. A sibling app (PdfCraft) may supply the parser | | `.rtf` Rich Text Format | 3% | R/W | 🟡 | 🟡 | ~50% | 10–15 | `crates/formats/src/tests.rs` | ~1,500 lines: text, character/paragraph formatting, styles, lists, tables, pictures (`\shppict`, #154), fields, bookmarks. 8-bit text in the document's code page (`\ansicpg`) or the current font's (`\fcharset`, `\cpg`): Cyrillic, Central European, Greek, Turkish, Hebrew, Arabic, Baltic, Vietnamese, Thai (was Windows-1252 only), Mac OS Roman (`\mac`, `\fcharset77`) and DOS code pages (`\pc` 437, `\pca` 850, `\cpg` 437/737/850/852/857/866, `\fcharset254`) (#1653). Double-byte East Asian code pages (Japanese `\fcharset128`, Chinese, Korean) not decoded yet. Pasted from the system clipboard where the platform offers it (Windows, Wayland) when there is no HTML (#687). Tracked insertions and deletions read and written (`\revtbl`, `\revised`/`\deleted`, authors, `\revdttm` dates), so Accept/Reject All work after opening or a round trip; tracked paragraph marks (`\par` inside `\revised`/`\deleted` groups, including a `\sect` inside a deletion) and character and paragraph formatting revisions (`\crauth`/`\crdate` with `\oldcprops`, `\prauth`/`\prdate` with `\oldpprops`) read and written; section and table formatting revisions (`\oldsprops`, `\oldtprops`) not yet. Sections on import (#1378): each `\sect` ends a section with its own page size and orientation, margins, header/footer distance, columns, break type, page-number restart, start and format, title page and headers/footers (those it leaves out link to the previous); export still writes one section. Thin: comments, notes, section export, RTL | -| `.odt` OpenDocument Text | 2% | R/W | 🟡 | 🟡 | ~50% | 8–12 | `crates/formats/src/tests.rs`, `crates/formats/src/odt.rs` unit tests, `crates/engine/tests/odt_sections.rs` | Styles, lists (#87), tables, pictures, headers/footers, page breaks (#86), keep with next (#2813), page size, orientation and margins (#810), sections: one per master-page change on a paragraph or table (next page, with its own page size, orientation, margins, columns and page-numbering restart) and per columned `text:section` (continuous, with its columns and separator line) (#1379); export writes them back: a master page and page layout per new-page section (size, orientation, margins, columns, page-numbering restart on the first paragraph or table), a `text:section` with its columns per continuous section whose columns differ from the page's (one column included), a list cut by a section break goes on numbering. Limits: a continuous section's page-numbering restart isn't written (ODF applies it only with a master page); a continuous section with the page's columns merges into the section before it on reopening; even and odd page starts are written as plain new pages; a tracked deletion of a section break is lost; an empty last section is dropped; tracked insertions and deletions with author and date, inserted and deleted paragraph marks and text inserted then deleted included (ODF change tracking; format changes read as none; pictures and tables inside a tracked deletion are dropped). No footnotes, comments, fields or RTL | +| `.odt` OpenDocument Text | 2% | R/W | 🟡 | 🟡 | ~50% | 8–12 | `crates/formats/src/tests.rs`, `crates/formats/src/odt.rs` unit tests, `crates/engine/tests/odt_sections.rs` | Styles, lists (#87), tables, pictures, page breaks (#86), headers and footers of each master page (every, left and first page) read on open, not yet written (#863), keep with next (#2813), an embedded formula object (`draw:object`) is read from its own `content.xml`'s MathML, same as HTML (#3099; export still drops equations, as before), page size, orientation and margins (#810), sections: one per master-page change on a paragraph or table (next page, with its own page size, orientation, margins, columns and page-numbering restart) and per columned `text:section` (continuous, with its columns and separator line) (#1379); export writes them back: a master page and page layout per new-page section (size, orientation, margins, columns, page-numbering restart on the first paragraph or table), a `text:section` with its columns per continuous section whose columns differ from the page's (one column included), a list cut by a section break goes on numbering. Limits: a continuous section's page-numbering restart isn't written (ODF applies it only with a master page); a continuous section with the page's columns merges into the section before it on reopening; even and odd page starts are written as plain new pages; a tracked deletion of a section break is lost; an empty last section is dropped; tracked insertions and deletions with author and date, inserted and deleted paragraph marks and text inserted then deleted included (ODF change tracking; format changes read as none; pictures and tables inside a tracked deletion are dropped). No footnotes, comments, fields or RTL | | `.rtf` Rich Text Format | 3% | R/W | 🟡 | 🟡 | ~50% | 10–15 | `crates/formats/src/tests.rs` | ~1,500 lines: text, character/paragraph formatting, styles, lists, tables, pictures (`\shppict`, #154), fields, bookmarks. 8-bit text in the document's code page (`\ansicpg`) or the current font's (`\fcharset`, `\cpg`): Cyrillic, Central European, Greek, Turkish, Hebrew, Arabic, Baltic, Vietnamese, Thai (was Windows-1252 only), Mac OS Roman (`\mac`, `\fcharset77`) and DOS code pages (`\pc` 437, `\pca` 850, `\cpg` 437/737/850/852/857/866, `\fcharset254`) (#1653). Double-byte East Asian code pages (Japanese `\fcharset128`, Chinese, Korean) not decoded yet. Pasted from the system clipboard where the platform offers it (Windows, Wayland) when there is no HTML (#687). Tracked insertions and deletions read and written (`\revtbl`, `\revised`/`\deleted`, authors, `\revdttm` dates), so Accept/Reject All work after opening or a round trip; paragraph-mark and formatting revisions not yet. Sections on import (#1378): each `\sect` ends a section with its own page size and orientation, margins, header/footer distance, columns, break type, page-number restart, start and format, title page and headers/footers (those it leaves out link to the previous); export writes every section too (break type, page size and orientation, margins, header/footer distance, columns, page numbering, title page; not headers/footers). Thin: comments, notes, RTL | | `.txt` Plain text | 2% | R/W | ✅ | ✅ | ~95% | 1–2 | unit tests | Encoding detection. Files with other extensions (`.m3u`/`.m3u8` playlists, logs, config files, no extension) open as text when they are text, Word and RTF files by their signature; binary files are refused (#569). A Word or RTF extension that contradicts the content also opens by signature, as in Word: RTF saved as `.doc` (hh.ru résumés, accounting exports), a `.docx` renamed `.doc`, a Word 97-2003 file renamed `.docx`; a Word XML Document (Word 2003 XML or Flat OPC) or a web page (``, ``) under a `.doc`, `.docx` or `.rtf` name opens as what it is (#2853, `tests::word_xml_and_web_pages_saved_as_doc_open_by_content`). Word's encoding choice dialog and line-ending options missing; such files save only through Save As | | `.htm/.html` Web Page and Web Page, Filtered | 1% | R/W | 🟡 | 🟡 | ~60% | 6–10 | `crates/formats/src/tests.rs` | Tables (#102), relative pictures embedded (#113), missing `` tolerated (#154). Header cells (``) bold unless their CSS says otherwise; `text-align` on a cell, or inherited from its row, row group or table, aligns the cell's paragraphs (#1381). HTML gives no page size, so it opens on the region's paper (A4 or Letter), as Markdown and plain text do; `@page` isn't read (#1381). No Word "Web Page" (with `_files` folder) round-trip. The same reader takes HTML from the system clipboard: Paste keeps another program's formatting (a Word document, a web page; Windows' "HTML Format" header understood), Merge Formatting keeps its emphasis (#687). A pasted `
` (LibreOffice's paragraph indented on both sides) becomes a Normal paragraph indented by the page's `blockquote` margins, else 0.5 in each side; opening an .html file keeps it as the Quote style (#1270). Equations are written as MathML (with the linear format as a UnicodeMath annotation) and read back from MathML: our annotation, else the presentation elements, else a TeX annotation (#816). A copy of one equation alone also goes in the clipboard's MathML formats (Windows "MathML" and "MathML Presentation", macOS `public.mathml.presentation` and `public.mathml`; W3C MathML 3 §6.2.4), which Microsoft documents Word reading first when it pastes math (#1297; the paste into Word is not yet verified) | @@ -233,6 +233,7 @@ Imported text fields use the first nonempty cached result run's formatting on th | Date | Change | Summary | |---|---|---| +| 2026-10-19 | trivial | An equation from LibreOffice in an `.odt` file no longer shows blank: it's embedded as a `draw:object` formula sub-document (MathML), with a `draw:image` replacement preview in LibreOffice's own metafile format WordCraft can't decode as a picture; the formula's own `content.xml` is read instead, through the same MathML reader as HTML (#3099) | | 2026-10-19 | trivial | Restrict Editing: with Filling in forms enforced, inserting content controls, their Properties and Remove Content Control are refused, as in Word (it greys them out), so a form's fields stay as they are and only their content is filled in | | 2026-10-19 | trivial | Content controls: text typed over a placeholder takes the control's own run properties (`w:sdtPr/w:rPr`, ECMA-376 §17.5.2.27), as in Word, and the placeholder style is recognised by its built-in name "Placeholder Text" whatever its localized id (German Word: `Platzhaltertext`), so entries in forms from localized Word no longer stay grey | | 2026-10-18 | trivial | A Word 2003 XML or Flat OPC document, or a web page, saved under a `.doc`, `.docx` or `.rtf` name opens by what it holds, as in Word, instead of going to the Word 97-2003 reader and showing a blank page (#2853) | @@ -267,9 +268,11 @@ Imported text fields use the first nonempty cached result run's formatting on th | 2026-10-16 | trivial | DOCX: `wp:anchor/@layoutInCell="0"` and VML `o:allowincell="f"` are read and written back (`layoutInCell="0"`), so a header-cell picture watermark is placed on the page (#1775) | | 2026-10-16 | trivial | Equations from Word files no longer lose content: body-level `m:oMathPara`, `mc:AlternateContent Requires="m"`, hyperlinks/simple fields/`mc:AlternateContent` inside an equation, `w:sym` and Symbol-font text in math (#1273) | | 2026-10-13 | trivial | Recipient lists: cell text capped at 64 MiB a sheet (shared-string amplification), an XML spreadsheet named .xls explained (#1287) | +| 2026-10-15 | trivial | Native .json writes pictures and kept parts as base64 text instead of arrays of numbers (a 27 MiB picture: 36 MiB instead of 286 MiB); files with the old arrays still open (#586 follow-up) | | 2026-10-13 | trivial | A cell's explicit top vertical alignment (`w:vAlign w:val="top"`) is kept on read and write and beats a table style that centres or bottom-aligns (#1288) | | 2026-10-14 | trivial | Row table border exceptions survive DOCX saves | | 2026-10-12 | trivial | Fields keep `w:dirty` through DOCX round trips — on buffered fields and range-field begin markers alike — and computing a result (Update Field, Update Table) clears it like Word's F9, so a generator's placeholder TOC still refreshes when Word opens the file until WordCraft replaces the placeholder (#877) | +| 2026-10-15 | trivial | ODT headers and footers (`style:header`/`-left`/`-first`, `style:footer`/…) read from each master page, so each section shows its own (#863) | | 2026-10-14 | trivial | VML freeforms (`v:shape` with a `path`, such as rule lines in headers) read as freeforms and kept on save instead of being dropped | | 2026-10-14 | trivial | DOCX paragraphs inside complex-field instructions carry a hidden paragraph mark; cached result paragraphs remain visible. | | 2026-10-14 | trivial | DrawingML `flowChartOr` preserves its circle and crossing spokes, including when grouped or rotated | diff --git a/docs/gaps.md b/docs/gaps.md index 768a94e9c..57556b6b6 100644 --- a/docs/gaps.md +++ b/docs/gaps.md @@ -1,6 +1,6 @@ # Where WordCraft falls short of Word -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-19 · **Change:** trivial (personal templates in File › New and Save as Word Template, #2852; previously trivial: #8: English dialects — en-GB, en-CA, en-AU… flag each other's spellings, #1281; previously trivial: #1: shape gradients and transparency, #1500; previously trivial: text typed into a content control takes the control's formatting; localized placeholder styles recognised; previously trivial: forms protection keeps its content controls: no inserting, Properties or Remove Content Control; previously trivial: clearing a no-match font query keeps the recent heading visible, #2068; previously trivial: line breaking: a slash opening a word travels whole to the next line, #2963; previously trivial: recipient list search and merged-only view, #1576; previously trivial: #2 impact restated: only ink rendering and OLE editing remain; previously trivial: #2 narrowed: native ink nested in groups and drawing canvases kept on save; previously trivial: opening a DOCX no longer copies its pictures; previously trivial: RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: automatic Asian/Latin and Asian/number spacing laid out, #2358; previously trivial: hint-reliant CJK outlines; previously trivial: numbered headings, #1793; previously trivial: legacy outline, shadow, emboss and imprint text effects drawn, #2534; previously trivial: #8 narrowed: Indonesian proofing from the installed Hunspell dictionary, #498; previously trivial: glossary/building blocks closed; previously trivial: Split Cells and Convert Text to Table follow the Word check, #2592; previously trivial: Language dialog keeps inherited proofing language and no-proof choices (#2717); previously trivial: #29: a copied equation also goes in the MathML clipboard formats, #1297; previously trivial: #24: the Spike, Ctrl+F3 and Ctrl+Shift+F3; previously trivial: VML freeforms read and kept on save; previously trivial: consistent preferred cell widths; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: page-break-only list paragraphs hide their labels; previously trivial: floating-table overlap ignores empty line-break lines; previously trivial: copies keep their formatting on the clipboard as HTML, #1523; previously trivial: sample-text formulas on Enter, #1067; previously trivial: #29: copied equations as MathML, OMML open as #1297, #816; previously trivial: the Font list no longer keeps every previewed font in memory, #848; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: beta distance restated: ~15 points, ~150–230 h; previously: first version; every known shortfall from the 2026-10-10 re-measure, ranked) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-21 · **Change:** trivial (Bullets and Numbering toggle off style-inherited lists at their effective level, including basedOn chains and numbered ListParagraph styles; other paragraphs and list definitions stay unchanged (related to #3459); previously trivial: Russian sorting keeps ё with е and recognises decomposed ё/й; previously trivial: personal templates in File › New and Save as Word Template, #2852; previously trivial: #8: English dialects — en-GB, en-CA, en-AU… flag each other's spellings, #1281; previously trivial: #1: shape gradients and transparency, #1500; previously trivial: text typed into a content control takes the control's formatting; localized placeholder styles recognised; previously trivial: forms protection keeps its content controls: no inserting, Properties or Remove Content Control; previously trivial: clearing a no-match font query keeps the recent heading visible, #2068; previously trivial: line breaking: a slash opening a word travels whole to the next line, #2963; previously trivial: recipient list search and merged-only view, #1576; previously trivial: #2 impact restated: only ink rendering and OLE editing remain; previously trivial: #2 narrowed: native ink nested in groups and drawing canvases kept on save; previously trivial: opening a DOCX no longer copies its pictures; previously trivial: RTF export writes sections (gap 19); previously trivial: ODT export writes sections (gap 19); previously trivial: automatic Asian/Latin and Asian/number spacing laid out, #2358; previously trivial: hint-reliant CJK outlines; previously trivial: numbered headings, #1793; previously trivial: legacy outline, shadow, emboss and imprint text effects drawn, #2534; previously trivial: #8 narrowed: Indonesian proofing from the installed Hunspell dictionary, #498; previously trivial: glossary/building blocks closed; previously trivial: Split Cells and Convert Text to Table follow the Word check, #2592; previously trivial: Language dialog keeps inherited proofing language and no-proof choices (#2717); previously trivial: #29: a copied equation also goes in the MathML clipboard formats, #1297; previously trivial: #24: the Spike, Ctrl+F3 and Ctrl+Shift+F3; previously trivial: VML freeforms read and kept on save; previously trivial: consistent preferred cell widths; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: page-break-only list paragraphs hide their labels; previously trivial: floating-table overlap ignores empty line-break lines; previously trivial: copies keep their formatting on the clipboard as HTML, #1523; previously trivial: sample-text formulas on Enter, #1067; previously trivial: #29: copied equations as MathML, OMML open as #1297, #816; previously trivial: the Font list no longer keeps every previewed font in memory, #848; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: beta distance restated: ~15 points, ~150–230 h; previously: first version; every known shortfall from the 2026-10-10 re-measure, ranked) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 > **Printing update:** 2026-10-12 · direct desktop printing through craft-print; details below. @@ -31,6 +31,8 @@ Legacy Word imports (#2445): interactive Save for `.doc`/`.dot` uses the existin Active footnote and endnote text stays readable while editing (#2545). This fixes a display-context defect; the broader fidelity, accessibility and readiness gates below are unchanged. +Russian language-aware sorting now keeps ё with е instead of after я, with the diaeresis as a secondary difference. Composed and decomposed ё and й compare equally. Fixed synthetic collation tests cover order, case and index headings; this does not establish full Cyrillic or CLDR collation parity. + ## Ranked | # | Gap | B | Evidence | Impact | Hours | Doc | @@ -74,6 +76,8 @@ full number from ~60% to ~75%. Everything else is depth on the way to full parit ## By kind +List toggles now recognise numbering inherited from paragraph styles (related to #3459). This closes that command-level case; the broader issue and readiness estimates are unchanged. See [typing-parity.md](typing-parity.md) for the original-fixture and round-trip coverage. + Sample-text formulas on Enter (#1067) are implemented: `=rand()` and `=lorem()`, optional paragraph/sentence counts and one-step undo. See [typing-parity.md](typing-parity.md) for the accepted parameter range and validation limits; readiness numbers are unchanged. @@ -106,6 +110,8 @@ Footer page-table height and body wrapping are corrected. Other pagination gaps | Date | Change | Summary | |---|---|---| +| 2026-10-20 | trivial | Russian sorting keeps ё with е and recognises decomposed ё/й; synthetic ordering, case and index-heading regressions; broader parity estimates unchanged | +| 2026-10-21 | trivial | Bullets and Numbering toggle off style-inherited lists at their effective level, including basedOn chains and numbered ListParagraph styles; other paragraphs and list definitions stay unchanged (related to #3459) | | 2026-10-19 | trivial | Restrict Editing: with Filling in forms enforced, inserting content controls, their Properties and Remove Content Control are refused, as in Word (it greys them out), so a form's fields stay as they are and only their content is filled in | | 2026-10-19 | trivial | Content controls: text typed over a placeholder takes the control's own run properties (`w:sdtPr/w:rPr`, ECMA-376 §17.5.2.27), as in Word, and the placeholder style is recognised by its built-in name "Placeholder Text" whatever its localized id (German Word: `Platzhaltertext`), so entries in forms from localized Word no longer stay grey | | 2026-10-18 | trivial | Font list starts at its first row after clearing a no-match query; recent heading remains visible (#2068) | diff --git a/docs/localization-parity.md b/docs/localization-parity.md index c2c849ac3..329031d8b 100644 --- a/docs/localization-parity.md +++ b/docs/localization-parity.md @@ -1,6 +1,6 @@ # Localization parity with Microsoft Word -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-16 · **Change:** minor (English dialects: en-GB, en-CA, en-AU and the other British-spelling Englishes proof their own spellings and flag the other dialect's, #1281, #1045; previously minor: how to install the Hunspell dictionaries on Windows, macOS and Linux: `docs/spelling-dictionaries.md`; previously minor: Read Aloud speaks each sentence in its own language, #1843; previously minor: Indonesian proofing: id-ID spelling from the installed Hunspell dictionary, dictionary-driven suggestions, generic checks with Indonesian messages, #498; previously trivial: print-area strings translated in every catalog; previously trivial: Italian: the interface audit checks the Italian catalog, `Privacy Settings` wording; previously trivial: Italian grammar: `po'`, `qual è`, `È`, `un`/`un'` rules; more elisions and truncations in spelling; previously trivial: Czech interface catalog completed, 2,447 entries; previously trivial: interface audit runs in CI again: size labels translated, 20 strings in every catalog, Read Aloud error race fixed, #1128; previously trivial: Review › Language reaches every language again — Set Proofing Language… and Set As Default, #860; previously minor: Italian proofing: it-IT spelling from the installed Hunspell dictionary, accent-first suggestions, conservative grammar with Italian messages; previously minor: Portuguese proofing; previously trivial: Turkish interface catalog completed, 2,325 entries, #774; previously minor: Hungarian interface added; previously trivial: sorting and table numbers follow the text's language, #836, #837; previously minor: Polish interface; Polish proofing — spelling, grammar, hyphenation — and a proofing language per run; previously minor: Turkish interface added; previously major: first version; catalogs measured from `crates/ui-egui/src/i18n`, Word's languages from its bundle listing) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-20 · **Change:** trivial (Russian sorting keeps ё with е and recognises decomposed ё/й; previously minor: English dialects: en-GB, en-CA, en-AU and the other British-spelling Englishes proof their own spellings and flag the other dialect's, #1281, #1045; previously minor: how to install the Hunspell dictionaries on Windows, macOS and Linux: `docs/spelling-dictionaries.md`; previously minor: Read Aloud speaks each sentence in its own language, #1843; previously minor: Indonesian proofing: id-ID spelling from the installed Hunspell dictionary, dictionary-driven suggestions, generic checks with Indonesian messages, #498; previously trivial: print-area strings translated in every catalog; previously trivial: Italian: the interface audit checks the Italian catalog, `Privacy Settings` wording; previously trivial: Italian grammar: `po'`, `qual è`, `È`, `un`/`un'` rules; more elisions and truncations in spelling; previously trivial: Czech interface catalog completed, 2,447 entries; previously trivial: interface audit runs in CI again: size labels translated, 20 strings in every catalog, Read Aloud error race fixed, #1128; previously trivial: Review › Language reaches every language again — Set Proofing Language… and Set As Default, #860; previously minor: Italian proofing: it-IT spelling from the installed Hunspell dictionary, accent-first suggestions, conservative grammar with Italian messages; previously minor: Portuguese proofing; previously trivial: Turkish interface catalog completed, 2,325 entries, #774; previously minor: Hungarian interface added; previously trivial: sorting and table numbers follow the text's language, #836, #837; previously minor: Polish interface; Polish proofing — spelling, grammar, hyphenation — and a proofing language per run; previously minor: Turkish interface added; previously major: first version; catalogs measured from `crates/ui-egui/src/i18n`, Word's languages from its bundle listing) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 Interface languages, script support and proofing per language. How translation works (catalog format, adding a language, clean-room rule): `crates/ui-egui/src/i18n/mod.rs`. @@ -151,6 +151,8 @@ Polish, `1.250,50` in German, `1,250.50` in English) and write formula results w separator (`crates/doc/src/locale_number.rs`). Interface lists (styles, fonts) still sort by code point. +Russian (`ru`) sorting keeps ё with е (a secondary diaeresis difference), before ж. Composed and decomposed ё and й have the same keys and index headings; the stored text is unchanged. Tests cover Russian locale aliases, case, representative words and these two canonical pairs. This is a narrow Cyrillic correction, not full Unicode normalization or CLDR collation. Ordering reference: [Unicode CLDR Russian collation chart](https://unicode.org/cldr/charts/42/collation/ru.html). + ## Region conventions (#838) As in Word, a new blank document takes its paper from the system **region**, not the interface @@ -198,6 +200,7 @@ WordCraft keeps 2,54 cm). | Date | Change | Summary | |---|---|---| +| 2026-10-20 | trivial | Russian sorting keeps ё with е and recognises decomposed ё/й; synthetic ordering, case and index-heading regressions; broader parity estimates unchanged | | 2026-10-14 | trivial | The sentence-capital rule (English, Polish, Croatian, Italian, Turkish, Portuguese) skips the first word of a list item (#1741) | | 2026-10-16 | minor | Indonesian proofing (#498): id-ID spelling from the installed Hunspell `id_ID` dictionary (not bundled), Czech's dictionary-driven suggestions, repeated-word (hyphenated reduplication offered) and spacing checks with Indonesian messages; Review › Language, the status bar, Set Proofing Language and File › Options › Proofing list Indonesian; estimates unchanged | | 2026-10-16 | trivial | Read Aloud picks a voice for the sentence's language (run tag, else script) when the default voice doesn't speak it, reads Windows text as UTF-8 | diff --git a/docs/recovery.md b/docs/recovery.md index 00ce84506..45a05479f 100644 --- a/docs/recovery.md +++ b/docs/recovery.md @@ -32,7 +32,9 @@ be lost if a crash happens before the next snapshot or while its write is still Version 1 snapshots wrap the native JSON document with an id, document name, timestamp and monotonic generation. Native JSON now includes media bytes, opaque preserved parts, and -chart/SmartArt/OLE source markup and relationships. Older JSON without these fields still opens. +chart/SmartArt/OLE source markup and relationships. Media and preserved parts are written as one +base64 string each. Older JSON without these fields, or with their bytes as arrays of numbers (as +written before base64), still opens. Paragraph cache revisions are transient and regenerated on load; undo history and selection are not part of the document snapshot. Recovery preserves the current in-memory model. It cannot recover content already lost during import or make later export more faithful than the exporter. @@ -50,8 +52,9 @@ Windows is therefore unverified. Limits are 256 MiB of encoded JSON per snapshot, 2 GiB of snapshot files per profile, 128 records and 16 generations per record during discovery. Ordinarily only two valid generations remain. Copies are retained until saved or explicitly discarded; there is no automatic age-based deletion. -Over-limit writes report an error and retain the previous valid copy. JSON stores binary parts -as byte arrays, so it can be substantially larger than DOCX. This first version has a fixed +Over-limit writes report an error and retain the previous valid copy. Base64 takes 4/3 of a +part's size (a 27 MiB picture makes a 36 MiB copy), and DOCX compresses, so a copy can still be +larger than the DOCX it came from. This first version has a fixed interval, English fallback for new labels, and no cloud, version-history or browser recovery. ## Commands and verification diff --git a/docs/typing-parity.md b/docs/typing-parity.md index 289ab6dea..8019ea195 100644 --- a/docs/typing-parity.md +++ b/docs/typing-parity.md @@ -1,6 +1,6 @@ # Typing parity with Word -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-18 · **Change:** trivial (a slash opening a word travels whole to the next line, #2963; previously trivial: typing over a selection takes the first selected character's formatting, also when the selection was made right to left, #2879; previously trivial: Turkish and Azerbaijani case mapping in AutoCorrect, #1923; previously trivial: smart quotes in Hebrew pinned: logical order, ” in acronyms as in Word, #1670; previously trivial: Backspace on a selected table, rows or columns deletes them, Delete clears them, #1844; previously trivial: typed text takes its direction from its own letters, #1543; previously trivial: an input method's request to erase text around the caret is honoured, so web typing in Cyrillic and other non-Latin text no longer repeats, #1480; previously trivial: `-->` AutoCorrect replacement, #2223; previously trivial: a heading-list preset keeps Heading styles and makes other paragraphs Heading 1, as in Word, #1793; previously trivial: numbered headings: a heading-linked Multilevel List numbers new headings by their style; numbering keeps a heading a heading, #1793; previously trivial: reversing an automatic capital adds an exception, #1857; previously trivial: smart quotes open after guillemets and low quotes and nest; previously trivial: Enter on an empty list item numbered by its paragraph style moves it up a level, #2143; previously trivial: Backspace after a section break and Delete before it remove the break, also on the last paragraph, #2127; previously trivial: a table inserted in right-to-left text is right to left, so Tab moves left, #2634; previously trivial: Shift+Delete cuts the selection on every platform, #1669; previously trivial: Select All then Delete removes a table the document starts with, #1198; previously trivial: Alt+numpad character codes, #961; previously trivial: Down and Up from a line start beside a table go into it, #1007; previously trivial: sample-text formulas on Enter, #1067; previously trivial: Track Changes: Accept All after a tracked delete from a paragraph end keeps the next paragraph; Reject All after typing into an empty paragraph keeps it (#864); previously minor: Correct TWo INitial CApitals, cAPS LOCK and table-cell capitals, #544; previously trivial: complex-script caret and Backspace, #604; Move Paragraph Up/Down, Promote/Demote, Go Back and Alt+X keys, #343) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-21 · **Change:** trivial (Bullets and Numbering toggle off style-inherited lists at their effective level, including basedOn chains and numbered ListParagraph styles; other paragraphs and list definitions stay unchanged (related to #3459); previously trivial: a text box that keeps the keyboard no longer resets the input method every frame, #2986; previously trivial: a slash opening a word travels whole to the next line, #2963; previously trivial: typing over a selection takes the first selected character's formatting, also when the selection was made right to left, #2879; previously trivial: Turkish and Azerbaijani case mapping in AutoCorrect, #1923; previously trivial: smart quotes in Hebrew pinned: logical order, ” in acronyms as in Word, #1670; previously trivial: Backspace on a selected table, rows or columns deletes them, Delete clears them, #1844; previously trivial: typed text takes its direction from its own letters, #1543; previously trivial: an input method's request to erase text around the caret is honoured, so web typing in Cyrillic and other non-Latin text no longer repeats, #1480; previously trivial: `-->` AutoCorrect replacement, #2223; previously trivial: a heading-list preset keeps Heading styles and makes other paragraphs Heading 1, as in Word, #1793; previously trivial: numbered headings: a heading-linked Multilevel List numbers new headings by their style; numbering keeps a heading a heading, #1793; previously trivial: reversing an automatic capital adds an exception, #1857; previously trivial: smart quotes open after guillemets and low quotes and nest; previously trivial: Enter on an empty list item numbered by its paragraph style moves it up a level, #2143; previously trivial: Backspace after a section break and Delete before it remove the break, also on the last paragraph, #2127; previously trivial: a table inserted in right-to-left text is right to left, so Tab moves left, #2634; previously trivial: Shift+Delete cuts the selection on every platform, #1669; previously trivial: Select All then Delete removes a table the document starts with, #1198; previously trivial: Alt+numpad character codes, #961; previously trivial: Down and Up from a line start beside a table go into it, #1007; previously trivial: sample-text formulas on Enter, #1067; previously trivial: Track Changes: Accept All after a tracked delete from a paragraph end keeps the next paragraph; Reject All after typing into an empty paragraph keeps it (#864); previously minor: Correct TWo INitial CApitals, cAPS LOCK and table-cell capitals, #544; previously trivial: complex-script caret and Backspace, #604; Move Paragraph Up/Down, Promote/Demote, Go Back and Alt+X keys, #343) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 What Word does when you type, observed black-box (same keystrokes typed into Word and WordCraft, results read back paragraph by paragraph) and pinned by tests in @@ -26,6 +26,8 @@ formatting use the existing editing primitives. Tests: `sample_text_formulas_exp ## Lists +Bullets and Numbering decide whether to toggle off from the selected paragraphs' effective numbering, including numbering inherited from a paragraph style or its `basedOn` chain. An explicit `numId=0` override remains unlisted until a list is applied. The effective list level and any level override determine the list kind, including an explicitly requested gallery kind. Toggling preserves custom styles, including a ListParagraph style that supplies numbering, list definitions and paragraphs outside the selection. A directly applied list still clears the automatic ListParagraph style. Original command and DOCX round-trip tests cover this narrow case related to #3459; it is not a new Word UI measurement. Tests: `toggle_style_inherited_list_off_preserves_other_paragraphs_and_round_trips`, `explicit_list_off_override_is_not_toggled_off_again`, `list_toggle_uses_effective_level_and_override`, `list_paragraph_style_survives_inherited_toggle_off`, `direct_list_toggle_still_clears_the_automatic_list_paragraph_style`. + | Keys | Word's result | Test | |---|---|---| | `* item` Enter Enter `text` | the second Enter ends the list; `text` is a **Normal** paragraph (no list indent left behind) | `enter_twice_after_a_list_item_ends_the_list`, `the_paragraph_after_a_list_is_normal_text` | @@ -296,6 +298,12 @@ again. (`automatic_change_asks_the_input_method_to_start_afresh`). The reset itself happens in eframe and hasn't been checked in a browser. +- On the desktop, a text box that keeps the keyboard while its dialog is open (Search, Go To's + page box, the Icons search, a new caption label) asks for focus only while it lacks it. egui + resets the input method on every focus request, and on X11 a reset recreates the XIM input + context, so asking every frame reset it every frame and could leave typing dead until a + restart (#2986, `dialogs::tests::boxes_that_keep_focus_do_not_interrupt_ime_every_frame`). + ## Known gaps - Web (#1480): Chrome has been seen moving the caret of eframe's hidden text field to its @@ -325,6 +333,8 @@ again. | Date | Change | Summary | |---|---|---| +| 2026-10-19 | trivial | Input methods: boxes that keep the keyboard ask for focus only while they lack it, so the input method isn't reset every frame (#2986) | +| 2026-10-21 | trivial | Bullets and Numbering toggle off style-inherited lists at their effective level, including basedOn chains and numbered ListParagraph styles; other paragraphs and list definitions stay unchanged (related to #3459) | | 2026-10-18 | trivial | Typing over a selection takes the first selected character's formatting, also when the selection was made right to left (#2879) | | 2026-10-17 | trivial | A table inserted in a right-to-left paragraph is a right-to-left table, so Tab moves to the cell on the left (#2634) | | 2026-10-15 | trivial | Backspace after a section break and Delete before it remove the break, also one on the last paragraph's mark; the text takes the following section's setup and keeps the headers it showed (#2127) |