Merge PR #1570: XLSX: keep dynamic-array cell metadata (cm) on save

ROADMAP.md: kept both progress-log rows

ROADMAP.md

@@ -2,7 +2,7 @@
 
 **Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away
 
-> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** minor (rows auto-fit oversized fonts on open and whole-row/column formatting) · **Target:** Microsoft Excel (Microsoft 365)
+> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (XLSX dynamic-array formulas keep their `cm` metadata on save) · **Target:** Microsoft Excel (Microsoft 365)
 
 GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX
 is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web.
@@ -134,6 +134,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
 |---|---|
 | 2026-10-15 | Function help follows the interface language: argument hints, autocomplete, Insert Function (search and readable categories) and the Formulas-tab menus read per-language tables of descriptions and argument names; Spanish keeps its 10 functions in the new format |
 | 2026-10-14 | Protect Sheet on a protected sheet: Home › Format, its keytips (HOP, RPS) and the tab menu now read Unprotect Sheet… and unprotect (straight away without a password); a protected sheet that forbids selecting locked (or unlocked) cells is enforced for clicks, arrows and Go To, and the Protect Sheet dialog has the select options (#1346) |
+| 2026-10-14 | XLSX: formulas opened with dynamic-array metadata (`cm`) keep it on save, with `xl/metadata.xml`. Before, a formula returning one value lost it (11 of 40 corpus files that have the part lost it entirely), and Excel recalculated some of them with implicit intersection (`SUMPRODUCT(IFERROR(...))` changed value) |
 | 2026-10-12 | Rows auto-fit oversized fonts: opening a file and whole-row/column formatting grow non-custom rows to their tallest font (#833) |
 | 2026-10-12 | Number formats: parsing a format code is linear in its length (long runs of `[`, `,` or `/` no longer take quadratic time), and an unclosed `[` is a literal when splitting sections as well as when tokenizing (#810) |
 | 2026-10-12 | Number formats: a fraction denominator with 65,536 zeros no longer crashes; format codes are capped at 255 characters like Excel (longer codes display as General, `TEXT` gives `#VALUE!`) (#807) |
@@ -195,6 +196,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
 
 | Date | Change | Summary |
 |---|---|---|
+| 2026-10-14 | minor | XLSX dynamic-array `cm` metadata kept on save; progress log row |
 | 2026-10-12 | minor | Rows auto-fit oversized fonts on open and whole-row/column formatting (#833); progress log row |
 | 2026-10-12 | minor | Progress log: multi-area row/column delete (#758) |
 | 2026-10-12 | trivial | Progress log: XLSX chart sheets kept on save (#727) |

crates/engine/src/cmd/edit.rs

@@ -1043,11 +1043,12 @@ fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
                                 // The array range moves with its formula.
                                 let (dr, dc) = (dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64);
                                 moved.array = f.array.and_then(|a| Some(RangeRef::new(a.start.offset(dr, dc)?, a.end.offset(dr, dc)?)));
+                                moved.dynamic = f.dynamic;
                                 return Some(std::sync::Arc::new(moved));
                             }
                             let moved =
                                 gridcraft_formula::adjust::shift_relative(e, dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64);
-                            Some(std::sync::Arc::new(Formula::from_expr(moved)))
+                            Some(std::sync::Arc::new(Formula { dynamic: f.dynamic, ..Formula::from_expr(moved) }))
                         };
                         if link {
                             let sheet_prefix = if dest_sheet == clip.sheet && same_doc {
@@ -1249,6 +1250,7 @@ pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: &
             {
                 let mut nf = Formula::from_expr(new);
                 nf.array = array;
+                nf.dynamic = f.dynamic;
                 cell.formula = Some(std::sync::Arc::new(nf));
             }
         }

crates/engine/src/cmd/formulas.rs

@@ -348,6 +348,7 @@ fn rename_name_in_formulas(wb: &mut gridcraft_model::Workbook, old: &str, old_sc
             let Some(e) = f.expr().and_then(rename) else { continue };
             let mut nf = Formula::from_expr(e);
             nf.array = f.array;
+            nf.dynamic = f.dynamic;
             if let Some(cell) = sh.cells.get_mut(c) {
                 cell.formula = Some(std::sync::Arc::new(nf));
             }

crates/engine/tests/dynamicmetadata.rs

@@ -0,0 +1,87 @@
+//! A formula opened with dynamic-array metadata keeps it through edits that only move or adjust
+//! it, and loses it when it is retyped (the writer then decides from the formula's shape).
+
+use std::sync::Arc;
+
+use gridcraft_engine::core::{CellRef, Value};
+use gridcraft_engine::model::{Cell, Formula, Workbook};
+use gridcraft_engine::{DocState, Session, io};
+use serde_json::json;
+
+fn at(s: &str) -> CellRef {
+    CellRef::parse(s).unwrap()
+}
+
+/// A workbook saved by GridCraft whose D1 formula came from a file with `cm`.
+fn opened() -> Session {
+    let mut wb = Workbook::new();
+    let sh = wb.sheet_mut(0).unwrap();
+    for r in 0..3 {
+        sh.set_value(CellRef::new(r, 1), Value::Number(f64::from(r + 1)));
+        sh.set_value(CellRef::new(r, 2), Value::Number(f64::from(r + 2)));
+    }
+    let mut f = Formula::new("SUMPRODUCT(B1:B3*C1:C3)");
+    f.dynamic = true;
+    sh.cells.set(at("D1"), Cell::formula(f));
+    let bytes = io::save_bytes(&wb, "dynamic.xlsx").unwrap();
+    let (back, _) = io::open_bytes("dynamic.xlsx", &bytes).unwrap();
+    let mut s = Session::new();
+    s.add_document(DocState::new(back, None, "dynamic".into()));
+    s
+}
+
+fn formula(s: &Session, cell: &str) -> Arc<Formula> {
+    s.doc().unwrap().wb.sheet(0).unwrap().cell(at(cell)).unwrap().formula.clone().unwrap()
+}
+
+/// Whether `cell` of the saved file carries dynamic-array metadata.
+fn saved_dynamic(s: &Session, cell: &str) -> bool {
+    let bytes = io::save_bytes(&s.doc().unwrap().wb, "dynamic.xlsx").unwrap();
+    let (back, _) = io::open_bytes("dynamic.xlsx", &bytes).unwrap();
+    back.sheet(0).unwrap().cell(at(cell)).unwrap().formula.as_ref().unwrap().dynamic
+}
+
+#[test]
+fn opened_dynamic_formula_keeps_its_metadata_on_save() {
+    let s = opened();
+    assert!(formula(&s, "D1").dynamic);
+    assert_eq!(s.doc().unwrap().wb.sheet(0).unwrap().value(at("D1")), Value::Number(20.0));
+    assert!(saved_dynamic(&s, "D1"));
+}
+
+#[test]
+fn inserting_rows_and_moving_keep_the_mark() {
+    let mut s = opened();
+    s.execute("home.insertRows", json!({"rows": "1:1"})).unwrap();
+    let f = formula(&s, "D2");
+    assert!(f.dynamic);
+    assert_eq!(f.text, "SUMPRODUCT(B2:B4*C2:C4)");
+    assert!(saved_dynamic(&s, "D2"));
+    s.execute("selection.set", json!({"range": "D2"})).unwrap();
+    s.execute("edit.cut", json!({})).unwrap();
+    s.execute("selection.set", json!({"range": "F2"})).unwrap();
+    s.execute("edit.paste", json!({})).unwrap();
+    assert!(formula(&s, "F2").dynamic);
+    assert!(saved_dynamic(&s, "F2"));
+}
+
+#[test]
+fn retyped_formula_drops_the_mark() {
+    let mut s = opened();
+    s.execute("cell.set", json!({"cell": "D1", "input": "=SUMPRODUCT(B1:B3*C1:C3)"})).unwrap();
+    assert!(!formula(&s, "D1").dynamic);
+    // SUMPRODUCT evaluates its arguments as arrays anyway: saved as a legacy formula.
+    assert!(!saved_dynamic(&s, "D1"));
+}
+
+#[test]
+fn dynamic_mark_is_serialized_only_when_set() {
+    let s = opened();
+    let dynamic = serde_json::to_value(&*formula(&s, "D1")).unwrap();
+    assert_eq!(dynamic["dynamic"], json!(true));
+    let plain = serde_json::to_value(Formula::new("SUM(A1:A3)")).unwrap();
+    assert!(plain.get("dynamic").is_none(), "{plain}");
+    // Documents written before the field existed still load.
+    let old: Formula = serde_json::from_value(json!({"text": "SUM(A1:A3)", "array": null})).unwrap();
+    assert!(!old.dynamic);
+}

crates/model/src/cell.rs

@@ -17,6 +17,12 @@ pub struct Formula {
     /// Legacy array formula (Ctrl+Shift+Enter) covering this range; the formula lives in the
     /// top-left cell.
     pub array: Option<RangeRef>,
+    /// Read from a file that marks this formula as a dynamic-array formula (cell metadata `cm`).
+    /// The writer keeps the mark, so a formula Excel evaluates as an array is never saved as a
+    /// legacy formula (which Excel evaluates with implicit intersection). Formulas entered or
+    /// rewritten in GridCraft start without it, and the writer decides from their shape.
+    #[serde(default, skip_serializing_if = "std::ops::Not::not")]
+    pub dynamic: bool,
 }
 
 impl Formula {
@@ -25,12 +31,12 @@ impl Formula {
     pub fn new(text: &str) -> Formula {
         let body = text.strip_prefix('=').unwrap_or(text);
         match gridcraft_formula::parse(body) {
-            Ok(e) => Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None },
-            Err(_) => Formula { text: body.to_string(), expr: None, array: None },
+            Ok(e) => Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None, dynamic: false },
+            Err(_) => Formula { text: body.to_string(), expr: None, array: None, dynamic: false },
         }
     }
     pub fn from_expr(e: Expr) -> Formula {
-        Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None }
+        Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None, dynamic: false }
     }
     /// The parsed expression, re-parsing after deserialization.
     pub fn expr(&self) -> Option<Expr> {

crates/model/src/workbook.rs

@@ -219,7 +219,7 @@ impl Workbook {
                     let f = cell.formula.as_ref()?;
                     let e = f.expr()?;
                     let ne = spell_calls(e.clone(), &lambdas);
-                    (ne != e).then(|| (c, Formula { array: f.array, ..Formula::from_expr(ne) }))
+                    (ne != e).then(|| (c, Formula { array: f.array, dynamic: f.dynamic, ..Formula::from_expr(ne) }))
                 })
                 .collect();
             if fixed.is_empty() {

crates/xlsx/src/sheet_read.rs

@@ -456,6 +456,11 @@ fn read_cell(cx: &mut Ctx<'_>, sheet: &mut Sheet, st: &mut RowState, c: &El, pos
 }
 
 fn read_formula(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynamic: bool) -> Option<Formula> {
+    // Keep the file's dynamic-array mark, so saving never turns the formula into a legacy one.
+    read_formula_text(cx, st, f, pos, dynamic).map(|fm| Formula { dynamic, ..fm })
+}
+
+fn read_formula_text(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynamic: bool) -> Option<Formula> {
     let model = from_file(f.text.trim());
     let text = model.as_str();
     match f.attr("t").unwrap_or("normal") {

crates/xlsx/src/sheet_write.rs

@@ -667,6 +667,10 @@ fn write_cell(
         Value::Error(CellError::Spill | CellError::Calc) => Value::Error(CellError::Value),
         v => v,
     };
+    // A formula read with dynamic-array metadata keeps it, even where the shape analysis would
+    // not need it: Excel decided the formula is evaluated as an array.
+    let own = formula.filter(|f| f.dynamic && f.array.is_none()).map(|_| RangeRef::cell(c));
+    let dynamic = dynamic.or(own.as_ref());
     // A non-spilling formula may still need array evaluation to avoid implicit intersection.
     if dynamic.is_some() && formula.is_some_and(|f| f.array.is_none()) {
         s.push_str(" cm=\"1\"");

crates/xlsx/src/tests/cellmetadata.rs

@@ -0,0 +1,97 @@
+//! Dynamic-array cell metadata (`cm`) read from a file is written back, so Excel keeps
+//! evaluating those formulas as arrays rather than with implicit intersection.
+
+use gridcraft_core::CellRef;
+
+use crate::package::Package;
+use crate::xml::El;
+use crate::{read_xlsx, write_xlsx};
+
+const METADATA: &str = r#"<?xml version="1.0"?><metadata xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" xmlns:xda="http://schemas.microsoft.com/office/spreadsheetml/2017/dynamicarray"><metadataTypes count="1"><metadataType name="XLDAPR" minSupportedVersion="120000" cellMeta="1"/></metadataTypes><futureMetadata name="XLDAPR" count="1"><bk><extLst><ext uri="{bdbb8cdc-fa1e-496e-a857-3c3f30c029c3}"><xda:dynamicArrayProperties fDynamic="1" fCollapsed="0"/></ext></extLst></bk></futureMetadata><cellMetadata count="1"><bk><rc t="1" v="0"/></bk></cellMetadata></metadata>"#;
+const METADATA_REL: &str =
+    r#"<Relationship Id="rId9" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/sheetMetadata" Target="metadata.xml"/>"#;
+
+/// Formulas Excel saves with `cm` although they return one value, and which GridCraft's own
+/// shape analysis would save without it.
+const DYNAMIC: [(&str, &str); 5] = [
+    ("D1", "SUMPRODUCT(B1:B3*C1:C3)"),
+    ("D2", "SUMPRODUCT(IFERROR(B1:B3/C1:C3,0))"),
+    ("D3", "IF(AND(B1:B3>0),\"all\",\"some\")"),
+    ("D4", "MID(CELL(\"filename\",A1),FIND(\"]\",CELL(\"filename\",A1))+1,31)"),
+    ("D5", "SUMPRODUCT(--(YEAR(A1:A3)=2024))"),
+];
+
+fn source() -> Vec<u8> {
+    let mut rows = String::new();
+    for r in 1..=3 {
+        rows.push_str(&format!(r#"<row r="{r}"><c r="A{r}"><v>{}</v></c><c r="B{r}"><v>{r}</v></c><c r="C{r}"><v>{}</v></c>"#, 45290 + r, r + 1));
+        let (address, text) = DYNAMIC[r as usize - 1];
+        rows.push_str(&format!(r#"<c r="{address}" cm="1"><f t="array" ref="{address}">{}</f><v>0</v></c></row>"#, crate::xml::esc(text)));
+    }
+    for (address, text) in &DYNAMIC[3..] {
+        let r = &address[1..];
+        rows.push_str(&format!(
+            r#"<row r="{r}"><c r="{address}" cm="1"><f t="array" ref="{address}">{}</f><v>0</v></c></row>"#,
+            crate::xml::esc(text)
+        ));
+    }
+    // The same SUMPRODUCT saved by legacy Excel: no `cm`.
+    rows.push_str(r#"<row r="6"><c r="D6"><f>SUMPRODUCT(B1:B3*C1:C3)</f><v>20</v></c></row>"#);
+    super::minimal(&format!("<sheetData>{rows}</sheetData>"), &[("xl/metadata.xml", METADATA)], METADATA_REL, "")
+}
+
+fn cell<'a>(sheet: &'a El, address: &str) -> &'a El {
+    sheet.child("sheetData").unwrap().kids("row").flat_map(|r| r.kids("c")).find(|c| c.attr("r") == Some(address)).unwrap()
+}
+
+fn at(s: &str) -> CellRef {
+    CellRef::parse(s).unwrap()
+}
+
+fn assert_written_dynamic(bytes: &[u8]) {
+    let mut pkg = Package::open(bytes).unwrap();
+    let sheet = pkg.read_xml("xl/worksheets/sheet1.xml").unwrap().unwrap();
+    for (address, text) in DYNAMIC {
+        let c = cell(&sheet, address);
+        assert_eq!(c.attr("cm"), Some("1"), "{address} lost its dynamic-array metadata");
+        let f = c.child("f").unwrap();
+        assert_eq!((f.attr("t"), f.attr("ref")), (Some("array"), Some(address)));
+        assert_eq!(f.text, text);
+    }
+    let legacy = cell(&sheet, "D6");
+    assert_eq!(legacy.attr("cm"), None, "a legacy formula must not become dynamic");
+    assert_eq!(legacy.child("f").unwrap().attr("t"), None);
+
+    let metadata = pkg.read_xml("xl/metadata.xml").unwrap().expect("metadata part written");
+    assert_eq!(metadata.path(&["metadataTypes", "metadataType"]).unwrap().attr("name"), Some("XLDAPR"));
+    let record = metadata.path(&["cellMetadata", "bk", "rc"]).unwrap();
+    assert_eq!((record.attr("t"), record.attr("v")), (Some("1"), Some("0")));
+    assert!(pkg.rels("xl/workbook.xml").unwrap().iter().any(|r| r.kind == "sheetMetadata" && r.target == "xl/metadata.xml"));
+    let types = pkg.read_xml("[Content_Types].xml").unwrap().unwrap();
+    assert!(types.kids("Override").any(|e| e.attr("PartName") == Some("/xl/metadata.xml")
+        && e.attr("ContentType") == Some("application/vnd.openxmlformats-officedocument.spreadsheetml.sheetMetadata+xml")));
+}
+
+#[test]
+fn dynamic_array_metadata_is_read_onto_formulas() {
+    let (wb, report) = read_xlsx(&source()).unwrap();
+    assert!(!report.warnings.iter().any(|w| w.contains("metadata")), "{:?}", report.warnings);
+    let sh = wb.sheet(0).unwrap();
+    for (address, _) in DYNAMIC {
+        let f = sh.cell(at(address)).unwrap().formula.as_ref().unwrap();
+        assert!(f.dynamic, "{address}");
+        assert_eq!(f.array, None, "{address} is not a legacy CSE array");
+    }
+    assert!(!sh.cell(at("D6")).unwrap().formula.as_ref().unwrap().dynamic);
+}
+
+#[test]
+fn dynamic_array_metadata_survives_saving_scalar_results() {
+    let (wb, _) = read_xlsx(&source()).unwrap();
+    let bytes = write_xlsx(&wb).unwrap();
+    assert_written_dynamic(&bytes);
+    // And again: the mark read from GridCraft's own file is kept too.
+    let (back, report) = read_xlsx(&bytes).unwrap();
+    assert!(report.warnings.is_empty(), "{:?}", report.warnings);
+    assert_written_dynamic(&write_xlsx(&back).unwrap());
+}

crates/xlsx/src/tests/mod.rs

@@ -1,6 +1,7 @@
 //! Integration-style tests: round trips, hand-written fixtures, hostile input and CSV.
 
 mod array_export;
+mod cellmetadata;
 mod chart_bool_default;
 mod csv_tests;
 mod encrypted_tests;

docs/file-format-parity.md

@@ -20,6 +20,16 @@ not interpreted as ink. This does not implement Office InkML or fix inserted ico
 Command and package tests cover repeated saves and unrelated cell edits; external Excel rendering
 and repair-free opening remain unverified.
 
+Dynamic-array formulas keep their mark on save. A formula read with dynamic-array cell metadata
+(`cm`, the `XLDAPR` record in `xl/metadata.xml`) is written back with it, together with the
+metadata part, even when it returns one value and GridCraft's own shape analysis would not need
+array evaluation. Dropping the mark turned such formulas into legacy ones, which Excel evaluates
+with implicit intersection: `SUMPRODUCT(IFERROR(B1:B3/C1:C3,0))` recalculated to a different
+value in Excel. Moving the formula, inserting or deleting rows and columns, and renaming sheets or
+names keep the mark; retyping the formula drops it, and the writer then decides from the
+formula's shape. Rich-value metadata other than cell pictures (data types, `#SPILL!` error
+values) is still dropped.
+
 | Format | Excel | GridCraft read | GridCraft write | Fidelity / notes | Tests |
 |---|---|---|---|---|---|
 | `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings. **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` |
@@ -93,6 +103,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here.
 | 2026-10-19 | minor | HTML (Save as Web Page and clipboard) marks right-to-left sheets and right-to-left cell text with `dir` |
 | 2026-10-15 | minor | XLSX: formulas and PivotTable sources quote sheet names like Excel (a leading `.`, R1C1-like and cell-like names), so `'.x'!A1` no longer saves as `.x!A1`, which Excel could not open (#1458) |
 | 2026-10-14 | trivial | Formula parser: text longer than Excel's 8,192 characters, and postfix chains (`1%%%…`, `f(1)(1)…`) past the 256-level nesting limit, are parse errors (the cell keeps its formula text and cached value) instead of trees deep enough to overflow the stack when a hostile file is opened, saved or dropped |
+| 2026-10-14 | minor | XLSX formulas read with dynamic-array metadata (`cm`) keep it on save, so Excel does not recalculate them with implicit intersection |
 | 2026-10-12 | trivial | XLSX chart sheets are kept on save (#727) |
 | 2026-10-12 | trivial | CSV and TSV export is capped at 512 MiB and refuses a larger used range with an error (#528) |
 | 2026-10-12 | minor | `.xls` import reads the calculation settings (CalcMode, CalcIter, CalcCount, CalcDelta, CalcPrecision), so circular models calculate instead of showing `#CIRC!` (#724) |