diff --git a/ROADMAP.md b/ROADMAP.md index 43c76188d..7047598df 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** minor (rows auto-fit oversized fonts on open and whole-row/column formatting) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (XLSX dynamic-array formulas keep their `cm` metadata on save) · **Target:** Microsoft Excel (Microsoft 365) GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web. @@ -134,6 +134,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in |---|---| | 2026-10-15 | Function help follows the interface language: argument hints, autocomplete, Insert Function (search and readable categories) and the Formulas-tab menus read per-language tables of descriptions and argument names; Spanish keeps its 10 functions in the new format | | 2026-10-14 | Protect Sheet on a protected sheet: Home › Format, its keytips (HOP, RPS) and the tab menu now read Unprotect Sheet… and unprotect (straight away without a password); a protected sheet that forbids selecting locked (or unlocked) cells is enforced for clicks, arrows and Go To, and the Protect Sheet dialog has the select options (#1346) | +| 2026-10-14 | XLSX: formulas opened with dynamic-array metadata (`cm`) keep it on save, with `xl/metadata.xml`. Before, a formula returning one value lost it (11 of 40 corpus files that have the part lost it entirely), and Excel recalculated some of them with implicit intersection (`SUMPRODUCT(IFERROR(...))` changed value) | | 2026-10-12 | Rows auto-fit oversized fonts: opening a file and whole-row/column formatting grow non-custom rows to their tallest font (#833) | | 2026-10-12 | Number formats: parsing a format code is linear in its length (long runs of `[`, `,` or `/` no longer take quadratic time), and an unclosed `[` is a literal when splitting sections as well as when tokenizing (#810) | | 2026-10-12 | Number formats: a fraction denominator with 65,536 zeros no longer crashes; format codes are capped at 255 characters like Excel (longer codes display as General, `TEXT` gives `#VALUE!`) (#807) | @@ -195,6 +196,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | XLSX dynamic-array `cm` metadata kept on save; progress log row | | 2026-10-12 | minor | Rows auto-fit oversized fonts on open and whole-row/column formatting (#833); progress log row | | 2026-10-12 | minor | Progress log: multi-area row/column delete (#758) | | 2026-10-12 | trivial | Progress log: XLSX chart sheets kept on save (#727) | diff --git a/crates/engine/src/cmd/edit.rs b/crates/engine/src/cmd/edit.rs index e07665399..3f95fa9b8 100644 --- a/crates/engine/src/cmd/edit.rs +++ b/crates/engine/src/cmd/edit.rs @@ -1043,11 +1043,12 @@ fn paste_special(s: &mut Session, p: &Json) -> Result { // The array range moves with its formula. let (dr, dc) = (dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64); moved.array = f.array.and_then(|a| Some(RangeRef::new(a.start.offset(dr, dc)?, a.end.offset(dr, dc)?))); + moved.dynamic = f.dynamic; return Some(std::sync::Arc::new(moved)); } let moved = gridcraft_formula::adjust::shift_relative(e, dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64); - Some(std::sync::Arc::new(Formula::from_expr(moved))) + Some(std::sync::Arc::new(Formula { dynamic: f.dynamic, ..Formula::from_expr(moved) })) }; if link { let sheet_prefix = if dest_sheet == clip.sheet && same_doc { @@ -1249,6 +1250,7 @@ pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: & { let mut nf = Formula::from_expr(new); nf.array = array; + nf.dynamic = f.dynamic; cell.formula = Some(std::sync::Arc::new(nf)); } } diff --git a/crates/engine/src/cmd/formulas.rs b/crates/engine/src/cmd/formulas.rs index 454c545d0..b62a50c6c 100644 --- a/crates/engine/src/cmd/formulas.rs +++ b/crates/engine/src/cmd/formulas.rs @@ -348,6 +348,7 @@ fn rename_name_in_formulas(wb: &mut gridcraft_model::Workbook, old: &str, old_sc let Some(e) = f.expr().and_then(rename) else { continue }; let mut nf = Formula::from_expr(e); nf.array = f.array; + nf.dynamic = f.dynamic; if let Some(cell) = sh.cells.get_mut(c) { cell.formula = Some(std::sync::Arc::new(nf)); } diff --git a/crates/engine/tests/dynamicmetadata.rs b/crates/engine/tests/dynamicmetadata.rs new file mode 100644 index 000000000..e1edfc958 --- /dev/null +++ b/crates/engine/tests/dynamicmetadata.rs @@ -0,0 +1,87 @@ +//! A formula opened with dynamic-array metadata keeps it through edits that only move or adjust +//! it, and loses it when it is retyped (the writer then decides from the formula's shape). + +use std::sync::Arc; + +use gridcraft_engine::core::{CellRef, Value}; +use gridcraft_engine::model::{Cell, Formula, Workbook}; +use gridcraft_engine::{DocState, Session, io}; +use serde_json::json; + +fn at(s: &str) -> CellRef { + CellRef::parse(s).unwrap() +} + +/// A workbook saved by GridCraft whose D1 formula came from a file with `cm`. +fn opened() -> Session { + let mut wb = Workbook::new(); + let sh = wb.sheet_mut(0).unwrap(); + for r in 0..3 { + sh.set_value(CellRef::new(r, 1), Value::Number(f64::from(r + 1))); + sh.set_value(CellRef::new(r, 2), Value::Number(f64::from(r + 2))); + } + let mut f = Formula::new("SUMPRODUCT(B1:B3*C1:C3)"); + f.dynamic = true; + sh.cells.set(at("D1"), Cell::formula(f)); + let bytes = io::save_bytes(&wb, "dynamic.xlsx").unwrap(); + let (back, _) = io::open_bytes("dynamic.xlsx", &bytes).unwrap(); + let mut s = Session::new(); + s.add_document(DocState::new(back, None, "dynamic".into())); + s +} + +fn formula(s: &Session, cell: &str) -> Arc { + s.doc().unwrap().wb.sheet(0).unwrap().cell(at(cell)).unwrap().formula.clone().unwrap() +} + +/// Whether `cell` of the saved file carries dynamic-array metadata. +fn saved_dynamic(s: &Session, cell: &str) -> bool { + let bytes = io::save_bytes(&s.doc().unwrap().wb, "dynamic.xlsx").unwrap(); + let (back, _) = io::open_bytes("dynamic.xlsx", &bytes).unwrap(); + back.sheet(0).unwrap().cell(at(cell)).unwrap().formula.as_ref().unwrap().dynamic +} + +#[test] +fn opened_dynamic_formula_keeps_its_metadata_on_save() { + let s = opened(); + assert!(formula(&s, "D1").dynamic); + assert_eq!(s.doc().unwrap().wb.sheet(0).unwrap().value(at("D1")), Value::Number(20.0)); + assert!(saved_dynamic(&s, "D1")); +} + +#[test] +fn inserting_rows_and_moving_keep_the_mark() { + let mut s = opened(); + s.execute("home.insertRows", json!({"rows": "1:1"})).unwrap(); + let f = formula(&s, "D2"); + assert!(f.dynamic); + assert_eq!(f.text, "SUMPRODUCT(B2:B4*C2:C4)"); + assert!(saved_dynamic(&s, "D2")); + s.execute("selection.set", json!({"range": "D2"})).unwrap(); + s.execute("edit.cut", json!({})).unwrap(); + s.execute("selection.set", json!({"range": "F2"})).unwrap(); + s.execute("edit.paste", json!({})).unwrap(); + assert!(formula(&s, "F2").dynamic); + assert!(saved_dynamic(&s, "F2")); +} + +#[test] +fn retyped_formula_drops_the_mark() { + let mut s = opened(); + s.execute("cell.set", json!({"cell": "D1", "input": "=SUMPRODUCT(B1:B3*C1:C3)"})).unwrap(); + assert!(!formula(&s, "D1").dynamic); + // SUMPRODUCT evaluates its arguments as arrays anyway: saved as a legacy formula. + assert!(!saved_dynamic(&s, "D1")); +} + +#[test] +fn dynamic_mark_is_serialized_only_when_set() { + let s = opened(); + let dynamic = serde_json::to_value(&*formula(&s, "D1")).unwrap(); + assert_eq!(dynamic["dynamic"], json!(true)); + let plain = serde_json::to_value(Formula::new("SUM(A1:A3)")).unwrap(); + assert!(plain.get("dynamic").is_none(), "{plain}"); + // Documents written before the field existed still load. + let old: Formula = serde_json::from_value(json!({"text": "SUM(A1:A3)", "array": null})).unwrap(); + assert!(!old.dynamic); +} diff --git a/crates/model/src/cell.rs b/crates/model/src/cell.rs index 8700f9a28..d4466365f 100644 --- a/crates/model/src/cell.rs +++ b/crates/model/src/cell.rs @@ -17,6 +17,12 @@ pub struct Formula { /// Legacy array formula (Ctrl+Shift+Enter) covering this range; the formula lives in the /// top-left cell. pub array: Option, + /// Read from a file that marks this formula as a dynamic-array formula (cell metadata `cm`). + /// The writer keeps the mark, so a formula Excel evaluates as an array is never saved as a + /// legacy formula (which Excel evaluates with implicit intersection). Formulas entered or + /// rewritten in GridCraft start without it, and the writer decides from their shape. + #[serde(default, skip_serializing_if = "std::ops::Not::not")] + pub dynamic: bool, } impl Formula { @@ -25,12 +31,12 @@ impl Formula { pub fn new(text: &str) -> Formula { let body = text.strip_prefix('=').unwrap_or(text); match gridcraft_formula::parse(body) { - Ok(e) => Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None }, - Err(_) => Formula { text: body.to_string(), expr: None, array: None }, + Ok(e) => Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None, dynamic: false }, + Err(_) => Formula { text: body.to_string(), expr: None, array: None, dynamic: false }, } } pub fn from_expr(e: Expr) -> Formula { - Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None } + Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None, dynamic: false } } /// The parsed expression, re-parsing after deserialization. pub fn expr(&self) -> Option { diff --git a/crates/model/src/workbook.rs b/crates/model/src/workbook.rs index 828ca692e..05b4646f7 100644 --- a/crates/model/src/workbook.rs +++ b/crates/model/src/workbook.rs @@ -219,7 +219,7 @@ impl Workbook { let f = cell.formula.as_ref()?; let e = f.expr()?; let ne = spell_calls(e.clone(), &lambdas); - (ne != e).then(|| (c, Formula { array: f.array, ..Formula::from_expr(ne) })) + (ne != e).then(|| (c, Formula { array: f.array, dynamic: f.dynamic, ..Formula::from_expr(ne) })) }) .collect(); if fixed.is_empty() { diff --git a/crates/xlsx/src/sheet_read.rs b/crates/xlsx/src/sheet_read.rs index 0194d2268..5a904fcd2 100644 --- a/crates/xlsx/src/sheet_read.rs +++ b/crates/xlsx/src/sheet_read.rs @@ -456,6 +456,11 @@ fn read_cell(cx: &mut Ctx<'_>, sheet: &mut Sheet, st: &mut RowState, c: &El, pos } fn read_formula(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynamic: bool) -> Option { + // Keep the file's dynamic-array mark, so saving never turns the formula into a legacy one. + read_formula_text(cx, st, f, pos, dynamic).map(|fm| Formula { dynamic, ..fm }) +} + +fn read_formula_text(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynamic: bool) -> Option { let model = from_file(f.text.trim()); let text = model.as_str(); match f.attr("t").unwrap_or("normal") { diff --git a/crates/xlsx/src/sheet_write.rs b/crates/xlsx/src/sheet_write.rs index 78e1afaae..6d9a4731b 100644 --- a/crates/xlsx/src/sheet_write.rs +++ b/crates/xlsx/src/sheet_write.rs @@ -667,6 +667,10 @@ fn write_cell( Value::Error(CellError::Spill | CellError::Calc) => Value::Error(CellError::Value), v => v, }; + // A formula read with dynamic-array metadata keeps it, even where the shape analysis would + // not need it: Excel decided the formula is evaluated as an array. + let own = formula.filter(|f| f.dynamic && f.array.is_none()).map(|_| RangeRef::cell(c)); + let dynamic = dynamic.or(own.as_ref()); // A non-spilling formula may still need array evaluation to avoid implicit intersection. if dynamic.is_some() && formula.is_some_and(|f| f.array.is_none()) { s.push_str(" cm=\"1\""); diff --git a/crates/xlsx/src/tests/cellmetadata.rs b/crates/xlsx/src/tests/cellmetadata.rs new file mode 100644 index 000000000..3e8791630 --- /dev/null +++ b/crates/xlsx/src/tests/cellmetadata.rs @@ -0,0 +1,97 @@ +//! Dynamic-array cell metadata (`cm`) read from a file is written back, so Excel keeps +//! evaluating those formulas as arrays rather than with implicit intersection. + +use gridcraft_core::CellRef; + +use crate::package::Package; +use crate::xml::El; +use crate::{read_xlsx, write_xlsx}; + +const METADATA: &str = r#""#; +const METADATA_REL: &str = + r#""#; + +/// Formulas Excel saves with `cm` although they return one value, and which GridCraft's own +/// shape analysis would save without it. +const DYNAMIC: [(&str, &str); 5] = [ + ("D1", "SUMPRODUCT(B1:B3*C1:C3)"), + ("D2", "SUMPRODUCT(IFERROR(B1:B3/C1:C3,0))"), + ("D3", "IF(AND(B1:B3>0),\"all\",\"some\")"), + ("D4", "MID(CELL(\"filename\",A1),FIND(\"]\",CELL(\"filename\",A1))+1,31)"), + ("D5", "SUMPRODUCT(--(YEAR(A1:A3)=2024))"), +]; + +fn source() -> Vec { + let mut rows = String::new(); + for r in 1..=3 { + rows.push_str(&format!(r#"{}{r}{}"#, 45290 + r, r + 1)); + let (address, text) = DYNAMIC[r as usize - 1]; + rows.push_str(&format!(r#"{}0"#, crate::xml::esc(text))); + } + for (address, text) in &DYNAMIC[3..] { + let r = &address[1..]; + rows.push_str(&format!( + r#"{}0"#, + crate::xml::esc(text) + )); + } + // The same SUMPRODUCT saved by legacy Excel: no `cm`. + rows.push_str(r#"SUMPRODUCT(B1:B3*C1:C3)20"#); + super::minimal(&format!("{rows}"), &[("xl/metadata.xml", METADATA)], METADATA_REL, "") +} + +fn cell<'a>(sheet: &'a El, address: &str) -> &'a El { + sheet.child("sheetData").unwrap().kids("row").flat_map(|r| r.kids("c")).find(|c| c.attr("r") == Some(address)).unwrap() +} + +fn at(s: &str) -> CellRef { + CellRef::parse(s).unwrap() +} + +fn assert_written_dynamic(bytes: &[u8]) { + let mut pkg = Package::open(bytes).unwrap(); + let sheet = pkg.read_xml("xl/worksheets/sheet1.xml").unwrap().unwrap(); + for (address, text) in DYNAMIC { + let c = cell(&sheet, address); + assert_eq!(c.attr("cm"), Some("1"), "{address} lost its dynamic-array metadata"); + let f = c.child("f").unwrap(); + assert_eq!((f.attr("t"), f.attr("ref")), (Some("array"), Some(address))); + assert_eq!(f.text, text); + } + let legacy = cell(&sheet, "D6"); + assert_eq!(legacy.attr("cm"), None, "a legacy formula must not become dynamic"); + assert_eq!(legacy.child("f").unwrap().attr("t"), None); + + let metadata = pkg.read_xml("xl/metadata.xml").unwrap().expect("metadata part written"); + assert_eq!(metadata.path(&["metadataTypes", "metadataType"]).unwrap().attr("name"), Some("XLDAPR")); + let record = metadata.path(&["cellMetadata", "bk", "rc"]).unwrap(); + assert_eq!((record.attr("t"), record.attr("v")), (Some("1"), Some("0"))); + assert!(pkg.rels("xl/workbook.xml").unwrap().iter().any(|r| r.kind == "sheetMetadata" && r.target == "xl/metadata.xml")); + let types = pkg.read_xml("[Content_Types].xml").unwrap().unwrap(); + assert!(types.kids("Override").any(|e| e.attr("PartName") == Some("/xl/metadata.xml") + && e.attr("ContentType") == Some("application/vnd.openxmlformats-officedocument.spreadsheetml.sheetMetadata+xml"))); +} + +#[test] +fn dynamic_array_metadata_is_read_onto_formulas() { + let (wb, report) = read_xlsx(&source()).unwrap(); + assert!(!report.warnings.iter().any(|w| w.contains("metadata")), "{:?}", report.warnings); + let sh = wb.sheet(0).unwrap(); + for (address, _) in DYNAMIC { + let f = sh.cell(at(address)).unwrap().formula.as_ref().unwrap(); + assert!(f.dynamic, "{address}"); + assert_eq!(f.array, None, "{address} is not a legacy CSE array"); + } + assert!(!sh.cell(at("D6")).unwrap().formula.as_ref().unwrap().dynamic); +} + +#[test] +fn dynamic_array_metadata_survives_saving_scalar_results() { + let (wb, _) = read_xlsx(&source()).unwrap(); + let bytes = write_xlsx(&wb).unwrap(); + assert_written_dynamic(&bytes); + // And again: the mark read from GridCraft's own file is kept too. + let (back, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.is_empty(), "{:?}", report.warnings); + assert_written_dynamic(&write_xlsx(&back).unwrap()); +} diff --git a/crates/xlsx/src/tests/mod.rs b/crates/xlsx/src/tests/mod.rs index bdbd66e05..d5e18041c 100644 --- a/crates/xlsx/src/tests/mod.rs +++ b/crates/xlsx/src/tests/mod.rs @@ -1,6 +1,7 @@ //! Integration-style tests: round trips, hand-written fixtures, hostile input and CSV. mod array_export; +mod cellmetadata; mod chart_bool_default; mod csv_tests; mod encrypted_tests; diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 56cef2cc3..9185dd4dd 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -20,6 +20,16 @@ not interpreted as ink. This does not implement Office InkML or fix inserted ico Command and package tests cover repeated saves and unrelated cell edits; external Excel rendering and repair-free opening remain unverified. +Dynamic-array formulas keep their mark on save. A formula read with dynamic-array cell metadata +(`cm`, the `XLDAPR` record in `xl/metadata.xml`) is written back with it, together with the +metadata part, even when it returns one value and GridCraft's own shape analysis would not need +array evaluation. Dropping the mark turned such formulas into legacy ones, which Excel evaluates +with implicit intersection: `SUMPRODUCT(IFERROR(B1:B3/C1:C3,0))` recalculated to a different +value in Excel. Moving the formula, inserting or deleting rows and columns, and renaming sheets or +names keep the mark; retyping the formula drops it, and the writer then decides from the +formula's shape. Rich-value metadata other than cell pictures (data types, `#SPILL!` error +values) is still dropped. + | Format | Excel | GridCraft read | GridCraft write | Fidelity / notes | Tests | |---|---|---|---|---|---| | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings. **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | @@ -93,6 +103,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here. | 2026-10-19 | minor | HTML (Save as Web Page and clipboard) marks right-to-left sheets and right-to-left cell text with `dir` | | 2026-10-15 | minor | XLSX: formulas and PivotTable sources quote sheet names like Excel (a leading `.`, R1C1-like and cell-like names), so `'.x'!A1` no longer saves as `.x!A1`, which Excel could not open (#1458) | | 2026-10-14 | trivial | Formula parser: text longer than Excel's 8,192 characters, and postfix chains (`1%%%…`, `f(1)(1)…`) past the 256-level nesting limit, are parse errors (the cell keeps its formula text and cached value) instead of trees deep enough to overflow the stack when a hostile file is opened, saved or dropped | +| 2026-10-14 | minor | XLSX formulas read with dynamic-array metadata (`cm`) keep it on save, so Excel does not recalculate them with implicit intersection | | 2026-10-12 | trivial | XLSX chart sheets are kept on save (#727) | | 2026-10-12 | trivial | CSV and TSV export is capped at 512 MiB and refuses a larger used range with an error (#528) | | 2026-10-12 | minor | `.xls` import reads the calculation settings (CalcMode, CalcIter, CalcCount, CalcDelta, CalcPrecision), so circular models calculate instead of showing `#CIRC!` (#724) |