From 5795259c45780061c13f5573217bdc0bc764c7b0 Mon Sep 17 00:00:00 2001 From: asystemoffields <193909980+asystemoffields@users.noreply.github.com> Date: Thu, 15 Oct 2026 10:10:23 -0400 Subject: [PATCH] File: recovery copies and .json files store pictures as base64 (#586 follow-up) Native JSON wrote Document::media and Document::passthrough with serde's default for bytes, an array of numbers. With a synthetic 27 MiB picture a recovery copy (compact) was 96.4 MiB and a .json file (pretty-printed, one number per line) 285.5 MiB. Each part is now one standard base64 string (doc::byte_store, using the workspace base64 crate), encoded straight into the writer. Reading accepts base64 or the old array of numbers, so recovery copies and .json files written before still open; anything else is an error, never a panic. Measured (release, same synthetic picture): compact 96.4 -> 36.0 MiB, pretty 285.5 -> 36.1 MiB, write 252/614 -> 52/57 ms, read 404/533 -> 46/37 ms. A recovery snapshot of a 6,000-paragraph document with the picture: 99 -> 39 MB, worker time 2.3 -> 0.26 s, start-up scan 1.03 -> 0.14 s. Tests: byte_store unit tests (round trip, old arrays, hostile values); recovery_format.rs native_json_stores_bytes_as_base64, native_json_with_byte_arrays_still_opens, native_json_with_bad_bytes_is_an_error, ignored measurement native_json_size_with_a_large_picture; desktop.rs copies_with_byte_arrays_written_before_still_restore. Co-Authored-By: Claude Opus 5.5 --- Cargo.lock | 1 + ROADMAP.md | 4 +- crates/doc/Cargo.toml | 1 + crates/doc/src/byte_store.rs | 130 +++++++++++++++++++++++++ crates/doc/src/lib.rs | 7 +- crates/engine/src/recovery/desktop.rs | 20 ++++ crates/engine/tests/recovery_format.rs | 85 ++++++++++++++++ docs/file-format-parity.md | 3 +- docs/recovery.md | 9 +- 9 files changed, 252 insertions(+), 8 deletions(-) create mode 100644 crates/doc/src/byte_store.rs diff --git a/Cargo.lock b/Cargo.lock index 6f7c8bf02..896674a25 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -5186,6 +5186,7 @@ version = "0.7.0" name = "wordcraft-doc" version = "0.7.0" dependencies = [ + "base64", "proptest", "serde", "serde_json", diff --git a/ROADMAP.md b/ROADMAP.md index 6ef6b3370..7f25d57eb 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~15 points (60% → 75% ready for real work) and ~150–230 h away -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** trivial (table style editor formats every part, #714); previously trivial: VML freeforms read and kept on save; previously trivial: progress log: consistent preferred cell widths; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: page-break-only list paragraphs hide their labels; previously trivial: progress log: grouped `flowChartOr` shapes draw their circle and crossing spokes; previously trivial: progress log: copies keep their formatting in other programs, #1523; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-15 · **Change:** trivial (progress log: recovery copies and .json files store pictures as base64, #586 follow-up; previously trivial: table style editor formats every part, #714); previously trivial: VML freeforms read and kept on save; previously trivial: progress log: consistent preferred cell widths; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: page-break-only list paragraphs hide their labels; previously trivial: progress log: grouped `flowChartOr` shapes draw their circle and crossing spokes; previously trivial: progress log: copies keep their formatting in other programs, #1523; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 > **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** trivial (progress log: print-area strings translated in every catalog; previously trivial: progress log: ODT sections, #1379; previously trivial: progress log: table deletes, Change Case and Unlink Fields respect Track Changes and locked content controls, #1329; previously trivial: progress log: rounded rectangle corners, #1476; previously trivial: progress log: Save As type list, #1399; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 > **Printing update:** 2026-10-12 · direct desktop printing through craft-print; details below. @@ -139,6 +139,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm | 2026-10-14 | Tab stops edited on the ruler: the corner selector picks Left/Center/Right/Decimal/Bar, a click sets a stop (Alt skips the snap), dragging moves it with a guide line and position tooltip, dragging off the ruler clears it; markers draw per kind and mirror for RTL; one `para.tabs` call and one undo step per gesture | | 2026-10-12 | Fields keep `w:dirty` through DOCX saves (#877): a begin marker flagged by a generator that couldn't compute a result (pandoc-style TOC placeholders) keeps the flag on open/save — as a buffered field or a range field — so Word still refreshes it on open; computing a result (Update Field, Update Table) clears it, like Word's F9 | |---|---| +| 2026-10-15 | Crash recovery copies and native .json files store pictures and kept parts as base64 text instead of arrays of numbers (#586 follow-up, #518): with a 27 MiB picture a recovery copy is 36 MiB instead of 96 MiB and a .json file 36 MiB instead of 286 MiB, written and read about five times faster. Copies and files written before still open | | 2026-10-14 | Consistent preferred cell widths override a stale table grid when both totals agree, for fixed and AutoFit tables. Conflicting widths, spans and percentages remain unchanged. | | 2026-10-14 | Legacy VML freeforms (`v:shape` with its own `path`), such as the rule lines in many older headers, are read as freeform shapes: they show on screen and in PDFs and are saved back as DrawingML custom geometry instead of vanishing | | 2026-10-14 | Field instructions spanning paragraphs no longer add blank lines during pagination; empty result paragraphs still take a line. | @@ -402,6 +403,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm | Date | Change | Summary | |---|---|---| +| 2026-10-15 | trivial | Progress log: recovery copies and .json files store pictures as base64 (#586 follow-up) | | 2026-10-14 | trivial | Progress log: row table border exceptions | | 2026-10-12 | minor | Czech proofing review: short-word splits rank after deletion, NOSUGGEST excludes suggestions while accepting typed forms, affix comments apply only to headers, ratios keep spaces and repeated-word messages use Czech quotes; synthetic engine end-to-end and latency checks | | 2026-10-12 | minor | Czech spelling from the installed Hunspell dictionary, runtime-table suggestions and generic checks; Czech interface writes cs-CZ documents; no Czech hyphenation patterns bundled; estimates unchanged | diff --git a/crates/doc/Cargo.toml b/crates/doc/Cargo.toml index f0111eb18..9ab62d34d 100644 --- a/crates/doc/Cargo.toml +++ b/crates/doc/Cargo.toml @@ -10,6 +10,7 @@ workspace = true [dependencies] wordcraft-geom = { workspace = true } +base64 = { workspace = true } serde = { workspace = true } serde_json = { workspace = true } thiserror = { workspace = true } diff --git a/crates/doc/src/byte_store.rs b/crates/doc/src/byte_store.rs new file mode 100644 index 000000000..75fae6453 --- /dev/null +++ b/crates/doc/src/byte_store.rs @@ -0,0 +1,130 @@ +//! How [`crate::Document::media`] and [`crate::Document::passthrough`] are written in the native +//! JSON form (`.json` files and crash-recovery copies): each part's bytes as one base64 string. +//! serde's default for bytes, an array of numbers, took about 3.6 times their size compact and +//! over six times pretty-printed (a 27 MB picture made a 99 MB recovery copy, #586). +//! +//! Reading takes either form, so `.json` files and recovery copies written before still open. +use std::collections::BTreeMap; +use std::fmt; +use std::sync::Arc; + +use base64::Engine as _; +use base64::display::Base64Display; +use base64::engine::general_purpose::STANDARD; +use serde::de::{self, Deserializer, MapAccess, SeqAccess, Visitor}; +use serde::ser::{Serialize, Serializer}; + +type Store = BTreeMap>>; + +/// One part's bytes, encoded as it is written out: no second copy of a large picture is made. +struct Encoded<'a>(&'a [u8]); + +impl Serialize for Encoded<'_> { + fn serialize(&self, s: S) -> Result { + s.collect_str(&Base64Display::new(self.0, &STANDARD)) + } +} + +pub(crate) fn serialize(store: &Store, s: S) -> Result { + s.collect_map(store.iter().map(|(name, bytes)| (name, Encoded(bytes)))) +} + +pub(crate) fn deserialize<'de, D: Deserializer<'de>>(d: D) -> Result { + d.deserialize_map(StoreVisitor) +} + +struct StoreVisitor; + +impl<'de> Visitor<'de> for StoreVisitor { + type Value = Store; + + fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result { + f.write_str("a map of part names to base64 text or byte arrays") + } + + fn visit_map>(self, mut map: A) -> Result { + let mut store = Store::new(); + while let Some((name, Bytes(bytes))) = map.next_entry::()? { + store.insert(name, Arc::new(bytes)); + } + Ok(store) + } +} + +/// A part's bytes as either form. +struct Bytes(Vec); + +impl<'de> de::Deserialize<'de> for Bytes { + fn deserialize>(d: D) -> Result { + d.deserialize_any(BytesVisitor).map(Bytes) + } +} + +struct BytesVisitor; + +impl<'de> Visitor<'de> for BytesVisitor { + type Value = Vec; + + fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result { + f.write_str("base64 text or an array of bytes") + } + + fn visit_str(self, text: &str) -> Result, E> { + STANDARD.decode(text).map_err(|e| E::custom(format!("bytes are not valid base64: {e}"))) + } + + fn visit_bytes(self, bytes: &[u8]) -> Result, E> { + Ok(bytes.to_vec()) + } + + /// The form written before: one number per byte. + fn visit_seq>(self, mut seq: A) -> Result, A::Error> { + // The length hint comes from the file: trust it only so far. + let mut out = Vec::with_capacity(seq.size_hint().unwrap_or(0).min(1 << 20)); + while let Some(b) = seq.next_element::()? { + out.push(b); + } + Ok(out) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[derive(serde::Serialize, serde::Deserialize, PartialEq, Debug, Default)] + struct Parts { + #[serde(default, with = "super")] + media: Store, + } + + fn parts(entries: &[(&str, &[u8])]) -> Parts { + Parts { media: entries.iter().map(|(n, b)| ((*n).to_string(), Arc::new(b.to_vec()))).collect() } + } + + #[test] + fn bytes_are_written_as_base64_and_read_back() { + let all: Vec = (0..=255).collect(); + for n in [0, 1, 2, 3, 4, 255, 256] { + let p = parts(&[("a.png", &all[..n.min(all.len())])]); + let text = serde_json::to_string(&p).unwrap(); + assert_eq!(serde_json::from_str::(&text).unwrap(), p, "{n}"); + } + assert_eq!(serde_json::to_string(&parts(&[("m", b"Man"), ("ma", b"Ma")])).unwrap(), r#"{"media":{"m":"TWFu","ma":"TWE="}}"#); + } + + #[test] + fn number_arrays_written_before_still_read() { + let old = r#"{"media":{"a.png":[0,1,127,128,254,255],"empty":[]}}"#; + assert_eq!(serde_json::from_str::(old).unwrap(), parts(&[("a.png", &[0, 1, 127, 128, 254, 255]), ("empty", &[])])); + } + + #[test] + fn hostile_values_are_errors_not_panics() { + for bad in [r#""T""#, r#""TWFu!""#, r#""-_""#, r#""TQ=x""#, "[256]", "[-1]", "[1.5]", "true", "{}", r#"["a"]"#] { + let text = format!(r#"{{"media":{{"a.png":{bad}}}}}"#); + assert!(serde_json::from_str::(&text).is_err(), "{bad}"); + } + assert!(serde_json::from_str::(r#"{"media":"x"}"#).is_err()); + } +} diff --git a/crates/doc/src/lib.rs b/crates/doc/src/lib.rs index 797d6c3e4..18449c949 100644 --- a/crates/doc/src/lib.rs +++ b/crates/doc/src/lib.rs @@ -11,6 +11,7 @@ #![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] pub mod bidi; +mod byte_store; pub mod chart; mod cjk_numbers; pub mod cluster; @@ -676,11 +677,11 @@ pub struct Document { /// The document's own endnote separators and continuation notice. #[serde(skip_serializing_if = "NoteSeparators::is_empty")] pub endnote_separators: NoteSeparators, - /// Embedded media (images) by key. - #[serde(skip_serializing_if = "BTreeMap::is_empty")] + /// Embedded media (images) by key. Native JSON writes each as base64 (`byte_store.rs`). + #[serde(skip_serializing_if = "BTreeMap::is_empty", with = "byte_store")] pub media: BTreeMap>>, /// Source-format parts we keep verbatim for round-trip (e.g. custom XML), by part name. - #[serde(skip_serializing_if = "BTreeMap::is_empty")] + #[serde(skip_serializing_if = "BTreeMap::is_empty", with = "byte_store")] pub passthrough: BTreeMap>>, } diff --git a/crates/engine/src/recovery/desktop.rs b/crates/engine/src/recovery/desktop.rs index 676a6ed0c..ad38ed0da 100644 --- a/crates/engine/src/recovery/desktop.rs +++ b/crates/engine/src/recovery/desktop.rs @@ -706,6 +706,26 @@ mod tests { assert_eq!(Store::files(&t.0.join(id)).unwrap().len(), 1); } + /// Copies are written with picture bytes in base64 (#586 follow-up); those written before kept + /// them as arrays of numbers and are still listed and restored, bytes intact. + #[test] + fn copies_with_byte_arrays_written_before_still_restore() { + let t = Temp::new(); + let id = new_id().unwrap(); + let mut s = snapshot(&id, "Synthetic draft with a picture"); + s.generation = 1; + s.document.media.insert("media/image1.png".into(), Arc::new(vec![0, 1, 127, 128, 254, 255])); + let mut value = serde_json::to_value(&s).unwrap(); + assert_eq!(value["document"]["media"]["media/image1.png"], "AAF/gP7/"); + value["document"]["media"]["media/image1.png"] = json!([0, 1, 127, 128, 254, 255]); + private_dir(&t.0.join(&id)).unwrap(); + fs::write(t.0.join(&id).join("snapshot-00000000000000000001.json"), serde_json::to_vec(&value).unwrap()).unwrap(); + let mut store = Store::new(t.0.clone()).unwrap(); + store.scan().unwrap(); + assert_eq!(store.inventory().len(), 1); + assert_eq!(store.load(&id).unwrap().document, s.document); + } + #[test] fn failed_pruning_cannot_push_a_record_beyond_the_discovery_limit() { let t = Temp::new(); diff --git a/crates/engine/tests/recovery_format.rs b/crates/engine/tests/recovery_format.rs index 38f372ac7..f889d6a83 100644 --- a/crates/engine/tests/recovery_format.rs +++ b/crates/engine/tests/recovery_format.rs @@ -67,3 +67,88 @@ fn existing_native_json_without_byte_stores_still_opens() { let doc = wordcraft_engine::io::open_bytes("old.json", &serde_json::to_vec(&value).unwrap()).unwrap(); assert_eq!(doc.plain_text(wordcraft_doc::StoryRef::Body), "Old native document"); } + +/// Picture and part bytes are one base64 string each in native JSON (`.json` files and recovery +/// copies, #586). An array of numbers took about 3.6 times their size, and over ten times in a +/// pretty-printed `.json` file. +#[test] +fn native_json_stores_bytes_as_base64() { + let mut doc = Document::from_text("Synthetic"); + let compact = |d: &Document| serde_json::to_vec(d).unwrap().len(); + let pretty = |d: &Document| wordcraft_engine::io::save_bytes("doc.json", d).unwrap(); + let (without, without_pretty) = (compact(&doc), pretty(&doc).len()); + let bytes: Vec = (0..300_000u32).map(|i| (i.wrapping_mul(2_654_435_761) >> 13) as u8).collect(); + doc.media.insert("big.png".into(), Arc::new(bytes.clone())); + doc.passthrough.insert("customXml/item1.xml".into(), Arc::new(b"".to_vec())); + let saved = pretty(&doc); + // Base64 is 4/3 of the bytes; allow a little for names and punctuation. + for (with, before) in [(compact(&doc), without), (saved.len(), without_pretty)] { + assert!(with - before < bytes.len() * 14 / 10, "{} bytes for {}", with - before, bytes.len()); + } + let value: serde_json::Value = serde_json::from_slice(&saved).unwrap(); + assert_eq!(value["passthrough"]["customXml/item1.xml"], "PHN5bnRoZXRpYy8+"); + let reopened = wordcraft_engine::io::open_bytes("doc.json", &saved).unwrap(); + assert_eq!((reopened.media, reopened.passthrough), (doc.media, doc.passthrough)); +} + +/// `.json` files and recovery copies written before kept bytes as arrays of numbers: they still +/// open, with the same bytes. +#[test] +fn native_json_with_byte_arrays_still_opens() { + let doc = complete_document(); + let mut value = serde_json::to_value(&doc).unwrap(); + for (store, parts) in [("media", &doc.media), ("passthrough", &doc.passthrough)] { + for (name, bytes) in parts { + value[store][name] = serde_json::json!(bytes.as_slice()); + } + } + assert!(value["media"]["synthetic.png"].is_array()); + let reopened = wordcraft_engine::io::open_bytes("old.json", &serde_json::to_vec_pretty(&value).unwrap()).unwrap(); + assert_eq!(reopened, doc); +} + +/// Bytes that are neither base64 nor an array of bytes make the file fail to open, never panic. +#[test] +fn native_json_with_bad_bytes_is_an_error() { + let saved = wordcraft_engine::io::save_bytes("doc.json", &complete_document()).unwrap(); + for bad in [serde_json::json!("not base64!"), serde_json::json!([1, 2, 300]), serde_json::json!({"a": 1}), serde_json::json!(7)] { + let mut value: serde_json::Value = serde_json::from_slice(&saved).unwrap(); + value["media"]["synthetic.png"] = bad.clone(); + assert!(wordcraft_engine::io::open_bytes("doc.json", &serde_json::to_vec(&value).unwrap()).is_err(), "{bad}"); + } +} + +/// Measures native JSON with a large picture: the compact form a recovery copy wraps, and a +/// pretty-printed `.json` file. Run on demand: +/// `cargo test --release -p wordcraft-engine --test recovery_format -- --ignored --nocapture`. +#[test] +#[ignore = "measurement"] +fn native_json_size_with_a_large_picture() { + let mut doc = Document::from_text("Synthetic document with one large picture."); + // Incompressible bytes, as a photograph's are. + let mut x = 0x2545_f491_4f6c_dd1du64; + let noise: Vec = (0..27u32 << 20) + .map(|_| { + x ^= x << 13; + x ^= x >> 7; + x ^= x << 17; + x as u8 + }) + .collect(); + doc.media.insert("media/image1.png".into(), Arc::new(noise)); + let mib = |n: usize| n as f64 / f64::from(1u32 << 20); + let t0 = std::time::Instant::now(); + let compact = serde_json::to_vec(&doc).unwrap(); + let written = t0.elapsed(); + let t0 = std::time::Instant::now(); + let back: Document = serde_json::from_slice(&compact).unwrap(); + println!("27 MiB picture, compact (recovery copy): {:.1} MiB, written in {written:?}, read in {:?}", mib(compact.len()), t0.elapsed()); + assert_eq!(back.media, doc.media); + let t0 = std::time::Instant::now(); + let pretty = wordcraft_engine::io::save_bytes("doc.json", &doc).unwrap(); + let written = t0.elapsed(); + let t0 = std::time::Instant::now(); + let back = wordcraft_engine::io::open_bytes("doc.json", &pretty).unwrap(); + println!("27 MiB picture, .json file (pretty): {:.1} MiB, written in {written:?}, read in {:?}", mib(pretty.len()), t0.elapsed()); + assert_eq!(back.media, doc.media); +} diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 38a1638f1..a309a9d1f 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -1,6 +1,6 @@ # File-format parity with Microsoft Word -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** trivial (every table style part round-trips, #714); previously trivial: VML freeforms read and kept on save; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: `flowChartOr` draws its circle and cross spokes, as in Word; previously trivial: RTF: Mac OS Roman and DOS text, #1653; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: PDF export draws horizontal and vertical lines inside groups, #1474; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-15 · **Change:** trivial (native .json stores pictures and kept parts as base64, #586 follow-up; previously trivial: every table style part round-trips, #714); previously trivial: VML freeforms read and kept on save; previously trivial: multi-paragraph field instructions no longer add blank lines; previously trivial: `flowChartOr` draws its circle and cross spokes, as in Word; previously trivial: RTF: Mac OS Roman and DOS text, #1653; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: PDF export draws horizontal and vertical lines inside groups, #1474; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats Every format Word opens or saves, what WordCraft does with it, and how it's tested. Fidelity is the share of a typical real-world file's content and formatting that survives (read: shows correctly; @@ -210,6 +210,7 @@ attributes inside stories and style or list definitions; a regenerated theme. Te | Date | Change | Summary | |---|---|---| +| 2026-10-15 | trivial | Native .json writes pictures and kept parts as base64 text instead of arrays of numbers (a 27 MiB picture: 36 MiB instead of 286 MiB); files with the old arrays still open (#586 follow-up) | | 2026-10-13 | trivial | A cell's explicit top vertical alignment (`w:vAlign w:val="top"`) is kept on read and write and beats a table style that centres or bottom-aligns (#1288) | | 2026-10-14 | trivial | Row table border exceptions survive DOCX saves | | 2026-10-12 | trivial | Fields keep `w:dirty` through DOCX round trips — on buffered fields and range-field begin markers alike — and computing a result (Update Field, Update Table) clears it like Word's F9, so a generator's placeholder TOC still refreshes when Word opens the file until WordCraft replaces the placeholder (#877) | diff --git a/docs/recovery.md b/docs/recovery.md index 00ce84506..45a05479f 100644 --- a/docs/recovery.md +++ b/docs/recovery.md @@ -32,7 +32,9 @@ be lost if a crash happens before the next snapshot or while its write is still Version 1 snapshots wrap the native JSON document with an id, document name, timestamp and monotonic generation. Native JSON now includes media bytes, opaque preserved parts, and -chart/SmartArt/OLE source markup and relationships. Older JSON without these fields still opens. +chart/SmartArt/OLE source markup and relationships. Media and preserved parts are written as one +base64 string each. Older JSON without these fields, or with their bytes as arrays of numbers (as +written before base64), still opens. Paragraph cache revisions are transient and regenerated on load; undo history and selection are not part of the document snapshot. Recovery preserves the current in-memory model. It cannot recover content already lost during import or make later export more faithful than the exporter. @@ -50,8 +52,9 @@ Windows is therefore unverified. Limits are 256 MiB of encoded JSON per snapshot, 2 GiB of snapshot files per profile, 128 records and 16 generations per record during discovery. Ordinarily only two valid generations remain. Copies are retained until saved or explicitly discarded; there is no automatic age-based deletion. -Over-limit writes report an error and retain the previous valid copy. JSON stores binary parts -as byte arrays, so it can be substantially larger than DOCX. This first version has a fixed +Over-limit writes report an error and retain the previous valid copy. Base64 takes 4/3 of a +part's size (a 27 MiB picture makes a 36 MiB copy), and DOCX compresses, so a copy can still be +larger than the DOCX it came from. This first version has a fixed interval, English fallback for new labels, and no cloud, version-history or browser recovery. ## Commands and verification