diff --git a/ROADMAP.md b/ROADMAP.md index 6b9b0fce7..3d1a1d0d8 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,7 @@ **Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-25 · **Change:** minor (AutoSum over a selected row puts its total to the right, and fills a selection's empty last row and column) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-25 · **Change:** minor (AutoSum over a selected row puts its total to the right, and fills a selection's empty last row and column; earlier: macro engine and `macro.*` commands; macro interface on the Automate tab) · **Target:** Microsoft Excel (Microsoft 365) GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web. @@ -22,7 +22,7 @@ This page is the summary. The detail is in: | Number | Value | Kind | |---|---|---| | 2026-10-14 | Editing a cell with text that would read back as a number, boolean, date or formula (entered with a leading apostrophe) shows the apostrophe in the editor and formula bar, so Enter unchanged keeps it text (#1362) | -| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 274 / 290 (94.5%), worksheet functions 513 / 523 (98.1%; 504 working, 96.4%) | +| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 276 / 292 (94.5%), worksheet functions 513 / 523 (98.1%; 504 working, 96.4%) | | **Ready for real work** | **~50%** | estimated from written weights (51.6); [target-app-parity.md](docs/target-app-parity.md) | | **Mainstream practitioner** | **~50%** | estimated: typical analyst/finance user, weekly features, × 0.90 interaction × 0.92 stability × 0.88 file exchange; [method](docs/target-app-parity.md#mainstream-practitioner) | | **Essentials user** | **~63%** | estimated: core features only, × 0.92 launch/stability × 0.95 clarity × 0.90 opening sent files; [method](docs/target-app-parity.md#essentials-user) | @@ -86,7 +86,7 @@ collaboration, mobile, AI). | Get & Transform, external data, data model | ~15% | ~10% | 60–100 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | | Insert objects (pictures, shapes, ink, SmartArt, equations) | ~55% | ~45% | 25–40 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | | Review, comments, protection, collaboration | ~55% | ~40% | 35–55 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | -| Automation (recorded scripts; no VBA) | ~30% | ~20% | 12–20 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | +| Automation (recorded scripts; typed macros with a recorder, a Macros dialog and a step editor; no VBA) | ~30% | ~20% | 12–20 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | ## Languages @@ -136,6 +136,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | What landed | |---|---| | 2026-10-25 | AutoSum over a selection does what Excel does: a selected row gets one total to its right (in its last cell if that is empty), not a one-cell total under each cell; a block whose last column is empty gets each row's total there, one whose last row is empty each column's, and with both the corner gets the total of the block; otherwise totals go under each column as before | +| 2026-10-24 | Macro interface ([macros.md](docs/macros.md#in-the-window)): an Automate › Macros group (Record Macro, which turns into Stop Recording, Use Relative References, Run Macro with recent macros); "Recording · N steps" with Stop in the status bar; a Record Macro dialog (checked name, free Ctrl+Shift shortcut, description); the Macros dialog lists macros beside Automate scripts and a VBA project, with Run, Step Into, Edit, Create, Delete and Options; Ctrl+Shift+letter macro shortcuts; questions that offer Run, Overwrite, Shift Down or Shift Right; a step editor with a breakpoint gutter, drag to reorder, a command picker, reference boxes that switch between relative tokens and absolute references, Inputs and Output tabs, a debug bar (Check, Run, Step, Continue, Run to Here, Stop with keep or undo) and grid overlays for inputs, writes, conflicts and the current step | | 2026-10-24 | Go To lists the named ranges of every sheet (`Sheet2!Name` for a name scoped to another sheet; hidden names, constants and formulas left out) and the references entered before, a double-click goes to an entry, and the Reference box takes several names or references separated by commas (`Price, Type`, `B14:C22,F19:G30`) | | 2026-10-24 | Status bar: a right-click opens a menu that sets which selection statistics show (Average, Count and Sum by default; Numerical Count, Minimum and Maximum are new), kept with the preferences (`view.statusBar`) | | 2026-10-24 | Ctrl+Shift+U (Control+Shift+U on a Mac) expands and collapses the formula bar, on the grid and while editing (`view.expandFormulaBar`); on Windows and Linux it used to underline | @@ -179,6 +180,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-13 | XLSX: `[Content_Types].xml`, `_rels/.rels`, the workbook rels and `xl/workbook.xml` are the first zip entries (document properties last), so libmagic and PHP `finfo` detect the file type even with large properties (#1090) | | 2026-10-13 | Conditional formatting: a rule gathers range statistics only when it reads them (top/bottom, above average, duplicates, scales, bars, icons) and borrows them instead of copying them per cell, so the per-frame lookup of an expression or cell-value rule no longer visits its whole range (#1098, part 1) | | 2026-10-12 | Next/previous sheet navigation stops at the first/last visible worksheet instead of activating a hidden boundary sheet; the current selection is retained | +| 2026-10-21 | Macro engine ([macros.md](docs/macros.md)): typed, declarative macros saved in the workbook (a JSON field; hidden names in xlsx); declared inputs checked with messages that name the cells; tokens (`Input.col(2)`, `Row`, `Output`, `Name:Total`, `{{…}}` in formulas); one undo step with full rollback; a footprint dry run with ask, fail, overwrite, shift down or shift right; a policy that keeps macros away from files and the app, with approval for macros in opened files; Repeat, If and Stop steps; a debugger; a relative recorder; `macro.*` commands beside the unchanged Automate scripts | | 2026-10-12 | Security: the `--control` server requires an auth token (random, kept in a 0600 file or chosen with `GRIDCRAFT_CONTROL_TOKEN`) on each connection's first line and closes bad connections (#777) | | 2026-10-12 | Rows auto-fit oversized fonts: opening a file and whole-row/column formatting grow non-custom rows to their tallest font (#833) | | 2026-10-12 | Number formats: parsing a format code is linear in its length (long runs of `[`, `,` or `/` no longer take quadratic time), and an unclosed `[` is a literal when splitting sections as well as when tokenizing (#810) | @@ -243,6 +245,8 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | Change | Summary | |---|---|---| | 2026-10-25 | minor | Progress log: AutoSum over a selected row, and a selection's empty last row and column | +| 2026-10-24 | minor | Macro interface on the Automate tab; progress log row | +| 2026-10-24 | minor | Macro engine and `macro.*` commands; catalog 275 / 291 → 276 / 292 (View › Macros › Record Macro); progress log row | | 2026-10-23 | minor | Progress log: accounting negatives in number text | | 2026-10-23 | minor | Progress log: INDEX with one index on a 2-D reference is #REF! | | 2026-10-22 | trivial | Progress log: formula hardening (#2163, #2162) | diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index 007cbf4bc..45455e00b 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -269,6 +269,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("view.unhideWindow", "View", "Window", "Unhide"), ("view.switchWindows", "View", "Window", "Switch Windows"), ("view.macros", "View", "Macros", "Macros"), + ("macro.record", "View", "Macros", "Record Macro"), // Automate ("automate.newScript", "Automate", "Scripting Tools", "New Script"), ("automate.allScripts", "Automate", "Scripting Tools", "All Scripts"), diff --git a/crates/engine/src/cmd.rs b/crates/engine/src/cmd.rs index 90c8a5227..be1a28f4d 100644 --- a/crates/engine/src/cmd.rs +++ b/crates/engine/src/cmd.rs @@ -12,6 +12,7 @@ pub mod format; pub mod formulas; pub mod insert; pub mod inspect; +pub mod macros; pub mod pivot; pub mod print; pub mod review; @@ -143,6 +144,7 @@ pub fn command_specs() -> &'static [CommandSpec] { v.extend(view::specs()); v.extend(inspect::specs()); v.extend(extra::specs()); + v.extend(macros::specs()); v.extend(draw::specs()); v.extend(spelling::specs()); v.extend(solver::specs()); @@ -337,6 +339,9 @@ pub struct Ctx<'a> { /// The command applied sheet protection itself (row/column inserts and deletes the /// protection allows), so cells it shifts aren't refused as edits of locked cells. pub protection_checked: bool, + /// The structural edits the command made, in order (the sheet's name, the edit), as + /// references were rewritten for them. + pub edits: Vec<(String, gridcraft_formula::adjust::Edit)>, } pub(crate) fn edit(s: &mut Session, f: impl FnOnce(&mut Ctx) -> Result) -> Result { @@ -358,15 +363,33 @@ pub(crate) fn commit(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result) let t0 = if cfg!(target_arch = "wasm32") { None } else { std::env::var_os("GRIDCRAFT_PROFILE").map(|_| wall_clock()) }; let since = |t0: std::time::Duration| wall_clock().saturating_sub(t0); let mut sel = d.selection.clone(); - let mut ctx = Ctx { wb: (*d.wb).clone(), changed: Vec::new(), structural: false, sel: &mut sel, fit_rows: Vec::new(), protection_checked: false }; + let mut ctx = Ctx { + wb: (*d.wb).clone(), + changed: Vec::new(), + structural: false, + sel: &mut sel, + fit_rows: Vec::new(), + protection_checked: false, + edits: Vec::new(), + }; let r = f(&mut ctx)?; - let Ctx { mut wb, changed, structural, mut fit_rows, protection_checked, .. } = ctx; + let Ctx { mut wb, changed, structural, mut fit_rows, protection_checked, edits, .. } = ctx; if !protection_checked { check_protection(&d.wb, &wb, &d.unlocked_ranges)?; } if !structural { check_arrays(&d.wb, &wb)?; } + if let Some(log) = d.touched.as_mut() { + for (si, c) in changed.iter().take(TOUCH_LOG_LIMIT.saturating_sub(log.len())) { + if let Some(sh) = wb.sheet(*si) { + log.push((sh.name.clone(), *c)); + } + } + } + if let Some(log) = d.edits.as_mut() { + log.extend(edits); + } if changed.len() <= 200_000 { fit_rows.extend(changed.iter().map(|(s, c)| (*s, c.row))); } @@ -394,6 +417,10 @@ pub(crate) fn commit(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result) Ok(r) } +/// The most cells a dry run's touch log keeps; a footprint whose log is full may be missing writes +/// that kept a cell's value, so a shift is refused then. +pub(crate) const TOUCH_LOG_LIMIT: usize = 1_000_000; + /// Excel's message for an edit of a locked cell on a protected sheet. pub(crate) const PROTECTED: &str = "The cell or chart you're trying to change is on a protected sheet."; diff --git a/crates/engine/src/cmd/extra/scripts.rs b/crates/engine/src/cmd/extra/scripts.rs index f1118410f..b14fbdb23 100644 --- a/crates/engine/src/cmd/extra/scripts.rs +++ b/crates/engine/src/cmd/extra/scripts.rs @@ -1,10 +1,15 @@ -use super::{Json, Result, SCRIPTS_NAME, Session, Workbook, bad, bool_param, edit, json, stash_get, stash_map, stash_set, str_param}; +use super::{EngineError, Json, Result, SCRIPTS_NAME, Session, Workbook, bad, bool_param, edit, json, stash_get, stash_map, stash_set, str_param}; pub(super) fn record(s: &mut Session, p: &Json) -> Result { if bool_param(p, "stop").unwrap_or(false) || s.recording.is_some() { let Some(start) = s.recording.take() else { return Ok(json!({"recording": false})) }; - let cmds: Vec = - s.journal.iter().skip(start).filter(|(id, _)| !id.starts_with("automate.")).map(|(id, p)| json!({"command": id, "params": p})).collect(); + let cmds: Vec = s + .journal + .iter() + .skip(start) + .filter(|(id, _)| !id.starts_with("automate.") && !crate::record::NOT_RECORDED.contains(&id.as_str())) + .map(|(id, p)| json!({"command": id, "params": p})) + .collect(); let name = str_param(p, "name").map(str::to_string).unwrap_or_else(|| format!("Script {}", start)); s.execute("automate.newScript", json!({"name": name, "commands": cmds}))?; return Ok(json!({"recording": false, "saved": name, "steps": cmds.len()})); @@ -104,12 +109,16 @@ pub(super) fn run_script(s: &mut Session, p: &Json) -> Result { } let _sandbox = crate::io::ScriptSandbox::enter(); let mut n = 0; + // What ran, for a recorder in progress: the recording keeps these steps with the run, so + // editing or deleting the script before the recording stops does not change what it saved. + let mut ran_steps: Vec<(String, Json)> = Vec::new(); for st in &steps { let id = st["command"].as_str().unwrap_or(""); if id.starts_with("automate.") { continue; } let before = s.ui_requests.len(); + let wb_before = s.doc().ok().map(|d| (d.uid, d.wb.clone())); let ran = s.execute(id, st["params"].clone()); // A step that asks the host to open a URL leaves the app: drop the request and stop. if s.ui_requests.get(before..).is_some_and(|new| new.iter().any(|r| matches!(r, crate::UiRequest::OpenUrl(_)))) { @@ -120,17 +129,43 @@ pub(super) fn run_script(s: &mut Session, p: &Json) -> Result { }); return Err(bad("automate.runScript", format!("scripts can't open links (`{id}`)"))); } - ran?; + let result = ran?; n += 1; + if s.recorder.is_some() { + // The recorder's rule for each step, as for a command run on its own: a menu-style + // step that only opened a dialog is left out (playing it back would stop at the + // dialog), and a toggle keeps the state it set. + let opened_dialog = s.ui_requests.get(before..).is_some_and(|new| new.iter().any(|r| matches!(r, crate::UiRequest::Dialog(..)))); + let changed = wb_before.as_ref().is_some_and(|(uid, wb)| s.docs.iter().any(|d| d.uid == *uid && !std::sync::Arc::ptr_eq(&d.wb, wb))); + if !opened_dialog || changed { + ran_steps.push((id.to_string(), crate::record::with_toggle_state(st["params"].clone(), &result))); + } + } + } + if s.recorder.is_some() { + s.script_ran = Some(ran_steps); } Ok(json!({"ran": n})) } -/// Automate › Stop Action: rolls back the undo groups open on the active workbook, so Undo and -/// Redo work there again. +/// Redo work there again. A macro being debugged on that workbook is stopped with them. pub(super) fn stop_action(s: &mut Session, _: &Json) -> Result { + // Inside another command the open groups are that command's own (an action in progress): + // a step of it that stopped them would pull the workbook from under the rest of its run. + if s.depth() > 1 { + return Err(EngineError::Disabled("automate.stopAction".into(), "Stop Action runs on its own, not as a step of a script or macro.".into())); + } let uid = s.doc()?.uid; - Ok(json!({"stopped": s.abort_groups(uid)})) + let mut out = json!({"stopped": s.abort_groups(uid)}); + // A macro being debugged here had its group among them: its run is over. + if let Some((name, wrote_files)) = crate::macros::debug::end_stopped(s, uid) { + out["macro"] = json!(name); + out["message"] = json!(format!( + "Macro '{name}', which was being debugged, was stopped and rolled back. {}", + crate::macros::run::rolled_back(&wrote_files) + )); + } + Ok(out) } pub(super) fn delete_script(s: &mut Session, p: &Json) -> Result { diff --git a/crates/engine/src/cmd/insert.rs b/crates/engine/src/cmd/insert.rs index 7bba4086d..d55af2b6f 100644 --- a/crates/engine/src/cmd/insert.rs +++ b/crates/engine/src/cmd/insert.rs @@ -17,19 +17,6 @@ use super::{ }; use crate::selection::current_region; -mod chart; -mod cond_format; -mod links; -mod objects; -mod table; - -pub use chart::{build_chart, chart_kind, recommend_charts, scatter_x_from_first, series_from_range}; -use chart::{chart_delete, chart_set, chart_switch, insert_chart}; -use cond_format::{add_cf, clear_cf, insert_icon, manage_cf}; -use links::{comment, delete_comment, insert_link, open_link, reply_comment, resolve_comment, show_note}; -use objects::{delete_object, insert_picture, insert_shape, insert_sparkline, move_object, set_anchor_mode, shape_set_text}; -use table::{format_as_table, insert_table, table_convert, table_flag, table_rename, table_style, table_total_fn}; - pub fn specs() -> Vec { vec![ cmd!( @@ -246,6 +233,1171 @@ fn block(s: &Session, p: &Json) -> Result { Ok(current_region(sh, d.selection.active)) } +fn chart_areas(s: &Session, p: &Json) -> Result> { + if p.get("range").is_some() { + return target_ranges(s, p); + } + let d = s.doc()?; + if d.selection.ranges.len() > 1 { + return Ok(d.selection.ranges.clone()); + } + Ok(vec![block(s, p)?]) +} + +// ---------------------------------------------------------------- tables + +fn insert_table(s: &mut Session, p: &Json) -> Result { + let r = block(s, p)?; + let sheet = target_sheet(s, p)?; + let d = s.doc()?; + if d.wb.sheet(sheet).is_some_and(|sh| sh.tables.iter().any(|t| t.range.intersects(&r))) { + return Err(EngineError::Other("A table can't overlap another table.".into())); + } + let arrays = |sh: &gridcraft_model::Sheet| { + sh.cells.iter().any(|(_, x)| x.formula.as_ref().and_then(|f| f.array).is_some_and(|a| !a.is_single() && a.intersects(&r))) + }; + if d.wb.sheet(sheet).is_some_and(arrays) { + return Err(EngineError::Other("Multi-cell array formulas aren't allowed in tables.".into())); + } + if r.count() > 50_000_000 { + return Err(bad("insert.table", "range too large")); + } + let header = bool_param(p, "header").unwrap_or_else(|| { + let sh = d.wb.sheet(sheet); + sh.is_some_and(|sh| (r.start.col..=r.end.col).all(|c| sh.value(CellRef::new(r.start.row, c)).is_text()) && r.height() > 1) + }); + let name = str_param(p, "name") + .map(str::to_string) + .unwrap_or_else(|| (1..).map(|n| format!("Table{n}")).find(|n| d.wb.table(n).is_none()).unwrap_or_else(|| "Table".into())); + if !valid_table_name(&name) { + return Err(EngineError::Other("The name that you entered is not valid.".into())); + } + if d.wb.table(&name).is_some() || d.wb.names.iter().any(|n| n.name.eq_ignore_ascii_case(&name)) { + return Err(EngineError::Other("The name entered already exists. Enter a unique name.".into())); + } + // Without a header row, the data moves down a row to make room for one: never a table with it. + let moves_a_table = |sh: &gridcraft_model::Sheet| { + sh.tables.iter().any(|t| t.range.end.row >= r.start.row && t.range.start.col <= r.end.col && t.range.end.col >= r.start.col) + }; + if !header && d.wb.sheet(sheet).is_some_and(moves_a_table) { + return Err(EngineError::Other("This operation isn't allowed: it would shift cells in a table on your worksheet.".into())); + } + let style = str_param(p, "style").unwrap_or("TableStyleMedium2").to_string(); + let id = d.wb.next_object_id(); + edit(s, |cx| { + // Without a header row, insert one with generic names. + let r = if header { + r + } else { + let sh = cx.sheet_mut(sheet)?; + // The header row pushes the data down one row: never off the end of the sheet. + let last = RangeRef::new(CellRef::new(MAX_ROWS - 1, r.start.col), CellRef::new(MAX_ROWS - 1, r.end.col)); + if sh.cells.iter_range(last).any(|(_, x)| x.formula.is_some() || !x.value.is_empty()) { + return Err(EngineError::Other("To prevent possible loss of data, cells can't be pushed off the end of the worksheet.".into())); + } + sh.cells.shift_rows_in_cols(r.start.col, r.end.col, r.start.row, 1); + // Like Insert Cells > Shift down: move merges, conditional formats, notes, links and pictures, and re-point every formula + // reference (on any sheet) at the cells that moved. + let e = Edit::InsertCells { axis: Axis::Rows, range: RangeRef::new(r.start, CellRef::new(r.start.row, r.end.col)) }; + shift_cell_features(sh, &e); + let name = sh.name.clone(); + rewrite(cx, &name, &e); + let sh = cx.sheet_mut(sheet)?; + for (i, c) in (r.start.col..=r.end.col).enumerate() { + sh.set_value(CellRef::new(r.start.row, c), gridcraft_core::Value::text(format!("Column{}", i + 1))); + } + cx.structural = true; + RangeRef::new(r.start, CellRef::new((r.end.row + 1).min(MAX_ROWS - 1), r.end.col)) + }; + let sh = cx.sheet_mut(sheet)?; + let mut names: Vec = Vec::new(); + // Lower-case names taken, and the next suffix for each header, so that thousands of equal + // headers don't take cubic time. + let mut taken = std::collections::HashSet::new(); + let mut next_suffix = std::collections::HashMap::new(); + for c in r.start.col..=r.end.col { + let mut n = sh.value(CellRef::new(r.start.row, c)).display(); + if n.is_empty() { + n = format!("Column{}", c - r.start.col + 1); + } + let base = n.clone(); + let key = base.to_ascii_lowercase(); + let mut k = next_suffix.get(&key).copied().unwrap_or(2); + while taken.contains(&n.to_ascii_lowercase()) { + n = format!("{base}{k}"); + k += 1; + } + next_suffix.insert(key, k); + taken.insert(n.to_ascii_lowercase()); + // Headers are text. + sh.set_value(CellRef::new(r.start.row, c), gridcraft_core::Value::text(n.as_str())); + names.push(n); + } + sh.tables.push(Table { + id, + name: name.clone(), + range: r, + header_row: true, + totals_row: false, + columns: names + .into_iter() + .map(|n| TableColumn { name: n, totals: TotalsFn::None, totals_label: None, formula: None, unique_name: None, xml: None }) + .collect(), + style, + banded_rows: true, + banded_cols: false, + first_col: false, + last_col: false, + filter_button: true, + filter: vec![], + table_type: None, + connection_id: None, + }); + cx.structural = true; + Ok(json!({"table": name, "range": r.a1()})) + }) +} + +/// Format as Table inside a table restyles that table, as in Excel; elsewhere it makes a new one. +fn format_as_table(s: &mut Session, p: &Json) -> Result { + // Without a range, the active area decides (as in insert_table), not the first one. + let r = if p.get("range").is_some() { target_range(s, p)? } else { s.doc()?.selection.current() }; + let sheet = target_sheet(s, p)?; + let inside = s.doc()?.wb.sheet(sheet).and_then(|sh| sh.tables.iter().find(|t| t.range.contains_range(&r))).map(|t| (t.name.clone(), t.range)); + match inside { + Some((table, range)) => { + table_style(s, &with(p, "table", json!(table)))?; + Ok(json!({"table": table, "range": range.a1()})) + } + None => insert_table(s, p), + } +} + +fn find_table(s: &Session, p: &Json) -> Result<(usize, usize)> { + let d = s.doc()?; + if let Some(n) = str_param(p, "table") { + return d.wb.table(n).ok_or_else(|| bad("table", format!("no table named `{n}`"))); + } + let sheet = d.wb.active_sheet(); + let sh = d.wb.sheet(sheet).ok_or(EngineError::NoDocument)?; + let ti = sh + .tables + .iter() + .position(|t| t.range.contains(d.selection.active)) + .ok_or_else(|| EngineError::Disabled("table".into(), "select a cell in a table".into()))?; + Ok((sheet, ti)) +} + +fn table_flag(s: &mut Session, p: &Json, which: &str) -> Result { + let (si, ti) = find_table(s, p)?; + let on = bool_param(p, "on"); + edit(s, |cx| { + let sh = cx.sheet_mut(si)?; + let Some(t) = sh.tables.get_mut(ti) else { return Ok(Json::Null) }; + match which { + "totals" => { + let v = on.unwrap_or(!t.totals_row); + if v != t.totals_row { + if v && t.range.end.row.saturating_add(1) >= MAX_ROWS { + return Err(EngineError::Other("There's no room below the table for the total row.".into())); + } + t.totals_row = v; + if v { + t.range.end.row += 1; + if let Some(first) = t.columns.first_mut() { + first.totals_label = Some("Total".into()); + } + if t.columns.len() > 1 + && let Some(last) = t.columns.last_mut() + && last.totals == TotalsFn::None + { + last.totals = TotalsFn::Sum; + } + } else { + let row = t.range.end.row; + t.range.end.row = t.range.end.row.saturating_sub(1); + let (c0, c1) = (t.range.start.col, t.range.end.col); + for c in c0..=c1 { + sh.remove_cell(CellRef::new(row, c)); + } + } + write_totals(sh, ti); + cx.structural = true; + } + } + "bandedRows" => t.banded_rows = on.unwrap_or(!t.banded_rows), + "bandedCols" => t.banded_cols = on.unwrap_or(!t.banded_cols), + "firstCol" => t.first_col = on.unwrap_or(!t.first_col), + "lastCol" => t.last_col = on.unwrap_or(!t.last_col), + "header" => { + t.filter_button = on.unwrap_or(!t.filter_button); + if !t.filter_button { + t.filter.clear(); + } + } + _ => {} + } + Ok(Json::Null) + }) +} + +/// Writes the totals row cells (labels and SUBTOTAL formulas). +fn write_totals(sh: &mut Sheet, ti: usize) { + let Some(t) = sh.tables.get(ti).cloned() else { return }; + if !t.totals_row { + return; + } + let row = t.range.end.row; + for (i, col) in t.columns.iter().enumerate() { + let c = CellRef::new(row, t.range.start.col + i as u32); + if let Some(code) = col.totals.subtotal_code() { + let f = format!("=SUBTOTAL({code},{}[{}])", t.name, col.name); + sh.set_cell(c, Cell { formula: Some(std::sync::Arc::new(Formula::new(&f))), value: gridcraft_core::Value::Empty, style: sh.style_id(c) }); + } else if let Some(l) = &col.totals_label { + sh.set_value(c, gridcraft_core::Value::text(l.as_str())); + } else { + sh.remove_cell(c); + } + } +} + +fn table_total_fn(s: &mut Session, p: &Json) -> Result { + let (si, ti) = find_table(s, p)?; + let col = str_param(p, "column").ok_or_else(|| bad("table.totalFunction", "missing `column`"))?.to_string(); + let f = match str_param(p, "function").unwrap_or("sum") { + "average" => TotalsFn::Average, + "count" => TotalsFn::Count, + "countNums" => TotalsFn::CountNums, + "max" => TotalsFn::Max, + "min" => TotalsFn::Min, + "stdDev" => TotalsFn::StdDev, + "var" => TotalsFn::Var, + "none" => TotalsFn::None, + _ => TotalsFn::Sum, + }; + edit(s, |cx| { + let sh = cx.sheet_mut(si)?; + if let Some(t) = sh.tables.get_mut(ti) + && let Some(i) = t.column_index(&col) + && let Some(c) = t.columns.get_mut(i) + { + c.totals = f; + } + write_totals(sh, ti); + cx.structural = true; + Ok(Json::Null) + }) +} + +fn table_style(s: &mut Session, p: &Json) -> Result { + let (si, ti) = find_table(s, p)?; + let style = str_param(p, "style").unwrap_or("TableStyleMedium2").to_string(); + edit(s, |cx| { + if let Some(t) = cx.sheet_mut(si)?.tables.get_mut(ti) { + t.style = style.clone(); + } + Ok(Json::Null) + }) +} + +/// Table names follow the Microsoft name rules, as the xlsx writer does: letters, digits, `_` and `.`, +/// starting with a letter, `_` or `\`, and at most 255 characters. Anything else (a hyphen, say) would +/// read as an operator inside a structured reference, and the writer would rename the table on save. +fn valid_table_name(name: &str) -> bool { + name.chars().count() <= 255 + && name.starts_with(|c: char| c.is_alphabetic() || c == '_' || c == '\\') + && CellRef::parse(name).is_none() + && !super::formulas::is_r1c1(name) + && name.chars().enumerate().all(|(i, c)| c.is_alphanumeric() || c == '_' || c == '.' || (i == 0 && c == '\\')) +} + +fn table_rename(s: &mut Session, p: &Json) -> Result { + let (si, ti) = find_table(s, p)?; + let name = str_param(p, "name").ok_or_else(|| bad("table.rename", "missing `name`"))?.to_string(); + if !valid_table_name(&name) { + return Err(EngineError::Other("The name that you entered is not valid.".into())); + } + let wb = &s.doc()?.wb; + if wb.table(&name).is_some_and(|x| x != (si, ti)) || wb.names.iter().any(|n| n.name.eq_ignore_ascii_case(&name)) { + return Err(EngineError::Other("The name entered already exists. Enter a unique name.".into())); + } + edit(s, |cx| { + let old = cx.wb.sheet(si).and_then(|sh| sh.tables.get(ti)).map(|t| t.name.clone()).unwrap_or_default(); + if let Some(t) = cx.sheet_mut(si)?.tables.get_mut(ti) { + t.name = name.clone(); + } + // Structured references follow the rename. + for i in 0..cx.wb.sheets.len() { + let keys: Vec<(CellRef, std::sync::Arc)> = + cx.wb.sheets.get(i).map(|s| s.cells.iter().filter_map(|(c, x)| x.formula.clone().map(|f| (c, f))).collect()).unwrap_or_default(); + // A bare name that is also a defined name resolves to the defined name, so it is not the table. + let shadowed = cx.wb.name(&old, i).is_some(); + let Some(sh) = cx.wb.sheet_mut(i) else { continue }; + for (c, f) in keys { + let Some(e) = f.expr() else { continue }; + let ne = e.map(&mut |x| match x { + gridcraft_formula::Expr::Struct(mut st) if st.table.eq_ignore_ascii_case(&old) => { + st.table = name.clone(); + gridcraft_formula::Expr::Struct(st) + } + gridcraft_formula::Expr::Name(n) if !shadowed && n.eq_ignore_ascii_case(&old) => gridcraft_formula::Expr::Name(name.clone()), + o => o, + }); + if let Some(cell) = sh.cells.get_mut(c) { + cell.formula = Some(std::sync::Arc::new(Formula::from_expr(ne))); + } + } + } + cx.structural = true; + Ok(Json::Null) + }) +} + +fn table_convert(s: &mut Session, p: &Json) -> Result { + let (si, ti) = find_table(s, p)?; + edit(s, |cx| { + // Apply the table look as plain formatting, then drop the table. + let wb = &mut cx.wb; + let Some(t) = wb.sheet(si).and_then(|sh| sh.tables.get(ti)).cloned() else { return Ok(Json::Null) }; + let looks: Vec<(CellRef, Style)> = t.range.iter().map(|c| (c, crate::tables::table_cell_style(wb, si, &t, c))).collect(); + for (c, st) in looks { + let id = wb.styles.intern(st); + if let Some(sh) = wb.sheet_mut(si) { + sh.set_style(c, id); + } + } + if let Some(sh) = wb.sheet_mut(si) { + sh.tables.remove(ti); + } + cx.structural = true; + Ok(Json::Null) + }) +} + +// ---------------------------------------------------------------- charts + +pub fn chart_kind(t: &str, sub: &str) -> ChartKind { + match (t, sub) { + ("column", "stacked") => ChartKind::ColumnStacked, + ("column", "stacked100") => ChartKind::ColumnStacked100, + ("bar", "stacked") => ChartKind::BarStacked, + ("bar", "stacked100") => ChartKind::BarStacked100, + ("bar", _) => ChartKind::BarClustered, + ("line", "markers") => ChartKind::LineMarkers, + ("line", "stacked") => ChartKind::LineStacked, + ("line", _) => ChartKind::Line, + ("pie", _) => ChartKind::Pie, + ("doughnut", _) => ChartKind::Doughnut, + ("area", "stacked") => ChartKind::AreaStacked, + ("area", "stacked100") => ChartKind::AreaStacked100, + ("area", _) => ChartKind::Area, + ("scatter", "lines") => ChartKind::ScatterLines, + ("scatter", _) => ChartKind::Scatter, + ("bubble", _) => ChartKind::Bubble, + ("radar", _) => ChartKind::Radar, + ("histogram", _) => ChartKind::Histogram, + ("waterfall", _) => ChartKind::Waterfall, + ("funnel", _) => ChartKind::Funnel, + ("treemap", _) => ChartKind::Treemap, + ("sunburst", _) => ChartKind::Sunburst, + ("boxWhisker", _) => ChartKind::BoxWhisker, + ("stock", _) => ChartKind::Stock, + ("combo", _) => ChartKind::Combo, + _ => ChartKind::ColumnClustered, + } +} + +/// Builds series from a block: first row/column as names/categories when they're text. +pub fn series_from_range(sh: &Sheet, r: RangeRef, by_rows: bool) -> Vec { + let q = gridcraft_formula::quote_sheet(&sh.name); + let abs = |r: RangeRef| { + format!( + "{q}!${}${}:${}${}", + gridcraft_core::col_to_letters(r.start.col), + r.start.row + 1, + gridcraft_core::col_to_letters(r.end.col), + r.end.row + 1 + ) + }; + let abs1 = |c: CellRef| format!("{q}!${}${}", gridcraft_core::col_to_letters(c.col), c.row + 1); + let is_text = |c: CellRef| sh.value(c).is_text() || sh.value(c).is_empty(); + let header_row = r.height() > 1 + && (r.start.col..=r.end.col).any(|c| sh.value(CellRef::new(r.start.row, c)).is_text()) + && (r.start.col..=r.end.col).filter(|c| *c != r.start.col).all(|c| is_text(CellRef::new(r.start.row, c))); + // A blank top-left cell above a header row marks the first column as the category labels even + // when it holds numbers (years, X values), the usual spreadsheet convention, so it isn't + // plotted as a series of its own (#287). + let blank_corner = header_row && sh.value(r.start).is_empty(); + let label_col = + r.width() > 1 && (blank_corner || (r.start.row + header_row as u32..=r.end.row).all(|row| is_text(CellRef::new(row, r.start.col)))); + let data = RangeRef::new(CellRef::new(r.start.row + header_row as u32, r.start.col + label_col as u32), r.end); + let mut out = Vec::new(); + if !by_rows { + for c in data.start.col..=data.end.col { + out.push(Series { + name: header_row.then(|| abs1(CellRef::new(r.start.row, c))), + categories: label_col.then(|| abs(RangeRef::new(CellRef::new(data.start.row, r.start.col), CellRef::new(data.end.row, r.start.col)))), + values: abs(RangeRef::new(CellRef::new(data.start.row, c), CellRef::new(data.end.row, c))), + bubble_sizes: None, + color: None, + secondary: false, + kind: None, + quartiles: Default::default(), + smooth: false, + trendlines: Vec::new(), + }); + } + } else { + for row in data.start.row..=data.end.row { + out.push(Series { + name: label_col.then(|| abs1(CellRef::new(row, r.start.col))), + categories: header_row + .then(|| abs(RangeRef::new(CellRef::new(r.start.row, data.start.col), CellRef::new(r.start.row, data.end.col)))), + values: abs(RangeRef::new(CellRef::new(row, data.start.col), CellRef::new(row, data.end.col))), + bubble_sizes: None, + color: None, + secondary: false, + kind: None, + quartiles: Default::default(), + smooth: false, + trendlines: Vec::new(), + }); + } + } + out +} + +fn series_from_areas(sh: &Sheet, areas: &[RangeRef]) -> Result<(RangeRef, bool, Vec)> { + let Some(first) = areas.first().copied() else { return Err(EngineError::Other("Select data for the chart first.".into())) }; + let bounds = areas.iter().skip(1).fold(first, |r, area| r.union(area)); + if areas.len() == 1 { + let by_rows = first.width() > first.height() + 1; + return Ok((bounds, by_rows, series_from_range(sh, first, by_rows))); + } + let valid_shape = first.width() == 1 + && first.height() > 1 + && areas.iter().skip(1).all(|area| area.width() == 1 && area.height() == first.height()) + && first.iter().all(|cell| sh.value(cell).is_text() || sh.value(cell).is_empty()); + if !valid_shape { + return Err(bad("insert.chart", "multi-area charts require a text category column followed by equal-height series columns")); + } + let category = series_from_range(sh, first, false).into_iter().next().map(|series| series.values); + let mut series = Vec::with_capacity(areas.len() - 1); + for area in &areas[1..] { + let mut area_series = series_from_range(sh, *area, false); + let Some(mut item) = area_series.pop() else { + return Err(EngineError::Other("Select data for the chart first.".into())); + }; + if item.name.is_none() { + return Err(bad("insert.chart", "multi-area series columns require headers")); + } + item.categories = category.clone(); + series.push(item); + } + Ok((bounds, false, series)) +} + +/// A scatter plot of a block without category labels takes its first column (or row) as the X +/// values of the others, as Excel does: two plain columns make one series of points, not two (#428). +pub fn scatter_x_from_first(kind: ChartKind, series: &mut Vec) { + if matches!(kind, ChartKind::Scatter | ChartKind::ScatterLines) && series.len() > 1 && series[0].categories.is_none() { + let x = series.remove(0).values; + for se in series.iter_mut() { + se.categories = Some(x.clone()); + } + } +} + +/// Most cells a chart's source block may span (bounds the series it builds). +const MAX_CHART_CELLS: u64 = 1_000_000; + +/// Chart types that suit the shape of the data, best first, as `(type, subtype, label)` for +/// `insert.chart`. Our own heuristic from the shape of the block (series count, points per +/// series, text categories, negative values), not a copy of Excel's ranking: two numeric columns +/// without labels → scatter; one short series → column, pie, bar; long series → line; several +/// series → clustered and stacked columns and bars, line. +pub fn recommend_charts(s: &Session, p: &Json) -> Result> { + let areas = chart_areas(s, p)?; + if areas.iter().fold(0_u64, |count, area| count.saturating_add(area.count())) > MAX_CHART_CELLS { + return Err(bad("insert.chart", "range too large for a chart")); + } + let sheet = target_sheet(s, p)?; + let d = s.doc()?; + let sh = d.wb.sheet(sheet).ok_or(EngineError::NoDocument)?; + let (_, by_rows, series) = series_from_areas(sh, &areas)?; + let ranges: Vec = series + .iter() + .filter_map(|x| match gridcraft_formula::parse(&x.values) { + Ok(gridcraft_formula::Expr::Ref(rf)) => Some(rf.range()), + _ => None, + }) + .collect(); + let points = ranges.first().map_or(0, RangeRef::count); + let (mut any_number, mut non_negative) = (false, true); + for n in ranges.iter().flat_map(|rr| rr.iter()).filter_map(|c| sh.value(c).as_f64()) { + any_number = true; + non_negative &= n >= 0.0; + } + if series.is_empty() || !any_number { + return Err(EngineError::Other("Select data for the chart first.".into())); + } + let labelled = series.first().is_some_and(|x| x.categories.is_some()); + const COLUMN: (&str, &str, &str) = ("column", "", "Clustered Column"); + const BAR: (&str, &str, &str) = ("bar", "", "Clustered Bar"); + const LINE: (&str, &str, &str) = ("line", "", "Line"); + const LINE_MARKERS: (&str, &str, &str) = ("line", "markers", "Line with Markers"); + const PIE: (&str, &str, &str) = ("pie", "", "Pie"); + const DOUGHNUT: (&str, &str, &str) = ("doughnut", "", "Doughnut"); + const AREA: (&str, &str, &str) = ("area", "", "Area"); + const SCATTER: (&str, &str, &str) = ("scatter", "", "Scatter"); + const STACKED_COLUMN: (&str, &str, &str) = ("column", "stacked", "Stacked Column"); + const STACKED_BAR: (&str, &str, &str) = ("bar", "stacked", "Stacked Bar"); + let mut out = Vec::new(); + if series.len() == 2 && !labelled && !by_rows { + out.extend([SCATTER, LINE_MARKERS, COLUMN]); + } else if series.len() == 1 { + if points > 12 { + out.extend([LINE, COLUMN, AREA]); + } else { + out.extend([COLUMN, BAR]); + if non_negative && points >= 2 { + out.extend([PIE, DOUGHNUT]); + } + out.push(LINE_MARKERS); + } + } else if points > 12 { + out.extend([LINE, COLUMN, AREA, STACKED_COLUMN]); + } else { + out.extend([COLUMN, BAR, STACKED_COLUMN, LINE_MARKERS, STACKED_BAR]); + } + Ok(out) +} + +/// The chart `insert.chart` would insert for these params, without inserting it: the sheet it +/// goes on and the chart (previews in the Recommended Charts dialog). +pub fn build_chart(s: &Session, p: &Json) -> Result<(usize, Chart)> { + let areas = chart_areas(s, p)?; + if areas.iter().fold(0_u64, |count, area| count.saturating_add(area.count())) > MAX_CHART_CELLS { + return Err(bad("insert.chart", "range too large for a chart")); + } + let mut t = str_param(p, "type").unwrap_or("column").to_string(); + let mut sub = str_param(p, "subtype").unwrap_or("").to_string(); + if t == "auto" { + let first = recommend_charts(s, p)?.into_iter().next().unwrap_or(("column", "", "")); + t = first.0.into(); + sub = first.1.into(); + } + let sheet = target_sheet(s, p)?; + let d = s.doc()?; + let sh = d.wb.sheet(sheet).ok_or(EngineError::NoDocument)?; + let kind = chart_kind(&t, &sub); + let (bounds, by_rows, mut series) = series_from_areas(sh, &areas)?; + if series.is_empty() { + return Err(EngineError::Other("Select data for the chart first.".into())); + } + scatter_x_from_first(kind, &mut series); + let id = d.wb.next_object_id(); + let at = cell_param_on(s, p, "at", sheet)?.unwrap_or_else(|| CellRef::new(bounds.start.row, bounds.end.col + 2)); + let title = str_param(p, "title").map(str::to_string).or_else(|| { + if series.len() == 1 { + series[0].name.as_ref().map(|n| gridcraft_calc::evaluate(&d.wb, sheet, at, n).display()) + } else { + Some("Chart Title".into()) + } + }); + let chart = Chart { + id, + kind, + anchor: Anchor { + cell: at, + dx: 0.0, + dy: 0.0, + width: f64_param(p, "width").unwrap_or(480.0) as f32, + height: f64_param(p, "height").unwrap_or(288.0) as f32, + mode: anchor_mode_param(p).unwrap_or_default(), + }, + title, + series, + legend: if matches!(kind, ChartKind::Pie | ChartKind::Doughnut) { LegendPos::Right } else { LegendPos::Bottom }, + data_labels: false, + gridlines: !matches!(kind, ChartKind::Pie | ChartKind::Doughnut | ChartKind::Treemap | ChartKind::Sunburst | ChartKind::Funnel), + style: 1, + x_title: None, + y_title: None, + source: (areas.len() == 1).then(|| format!("{}!{}", gridcraft_formula::quote_sheet(&sh.name), bounds.a1())), + by_rows, + pivot: None, + }; + Ok((sheet, chart)) +} + +fn insert_chart(s: &mut Session, p: &Json) -> Result { + let (sheet, chart) = build_chart(s, p)?; + let id = chart.id; + edit(s, |cx| { + cx.sheet_mut(sheet)?.charts.push(chart); + Ok(json!({"chart": id})) + }) +} + +fn find_chart(s: &Session, p: &Json) -> Result<(usize, usize)> { + let d = s.doc()?; + let sheet = d.wb.active_sheet(); + let sh = d.wb.sheet(sheet).ok_or(EngineError::NoDocument)?; + let idx = match u32_param(p, "chart") { + Some(id) => sh.charts.iter().position(|c| c.id == id), + None => sh.charts.len().checked_sub(1), + }; + idx.map(|i| (sheet, i)).ok_or_else(|| bad("chart", "no such chart")) +} + +fn chart_set(s: &mut Session, p: &Json) -> Result { + let (si, ci) = find_chart(s, p)?; + let at = cell_param_on(s, p, "at", si)?; + let quartiles = p + .get("quartiles") + .map(|v| match v.as_str() { + Some("exclusive") => Ok(gridcraft_model::QuartileMethod::Exclusive), + Some("inclusive") => Ok(gridcraft_model::QuartileMethod::Inclusive), + _ => Err(bad("quartiles", "expected exclusive or inclusive")), + }) + .transpose()?; + edit(s, |cx| { + let sh = cx.sheet_mut(si)?; + let Some(c) = sh.charts.get_mut(ci) else { return Ok(Json::Null) }; + if let Some(t) = str_param(p, "type") { + c.kind = chart_kind(t, str_param(p, "subtype").unwrap_or("")); + } + if let Some(method) = quartiles { + for series in &mut c.series { + series.quartiles = method; + } + } + if let Some(t) = p.get("title") { + c.title = t.as_str().map(str::to_string); + } + if let Some(l) = str_param(p, "legend") { + c.legend = match l { + "none" => LegendPos::None, + "top" => LegendPos::Top, + "left" => LegendPos::Left, + "right" => LegendPos::Right, + _ => LegendPos::Bottom, + }; + } + if let Some(v) = bool_param(p, "dataLabels") { + c.data_labels = v; + } + if let Some(v) = bool_param(p, "gridlines") { + c.gridlines = v; + } + if let Some(v) = u32_param(p, "style") { + c.style = v.clamp(1, 16); + } + if let Some(t) = p.get("xTitle") { + c.x_title = t.as_str().map(str::to_string); + } + if let Some(t) = p.get("yTitle") { + c.y_title = t.as_str().map(str::to_string); + } + if let Some(at) = at { + c.anchor.cell = at; + } + if let Some(w) = f64_param(p, "width") { + c.anchor.width = w.clamp(20.0, 10000.0) as f32; + } + if let Some(h) = f64_param(p, "height") { + c.anchor.height = h.clamp(20.0, 10000.0) as f32; + } + if let Some(v) = f64_param(p, "dx") { + c.anchor.dx = v as f32; + } + if let Some(v) = f64_param(p, "dy") { + c.anchor.dy = v as f32; + } + Ok(Json::Null) + }) +} + +fn chart_switch(s: &mut Session, p: &Json) -> Result { + let (si, ci) = find_chart(s, p)?; + edit(s, |cx| { + let Some(sh) = cx.wb.sheet(si) else { return Ok(Json::Null) }; + let Some(c) = sh.charts.get(ci) else { return Ok(Json::Null) }; + // Charts read from a file have no source range: take the block their series span. + let Some((src, by_rows)) = c.source.clone().map(|s| (s, c.by_rows)).or_else(|| series_block(&c.series)) else { return Ok(Json::Null) }; + let (sheet_name, body) = split_sheet(&src); + let src_sheet = sheet_name.and_then(|n| cx.wb.sheet_index(&n)).unwrap_or(si); + let Some(mut r) = RangeRef::parse(body) else { return Ok(Json::Null) }; + let Some(ssh) = cx.wb.sheet(src_sheet) else { return Ok(Json::Null) }; + // Whole-column/row references (common in files) span a million cells: clip the block's + // far edges to the cells in use, then refuse anything still too large to chart. + if let Some(used) = ssh.used_range() { + r.end.row = r.end.row.min(used.end.row).max(r.start.row); + r.end.col = r.end.col.min(used.end.col).max(r.start.col); + } + if r.count() > MAX_CHART_CELLS { + return Err(bad("chart.switchRowColumn", "range too large for a chart")); + } + let by_rows = !by_rows; + let series = series_from_range(ssh, r, by_rows); + if let Some(c) = cx.sheet_mut(si)?.charts.get_mut(ci) { + c.source.get_or_insert(src); + c.by_rows = by_rows; + c.series = series; + scatter_x_from_first(c.kind, &mut c.series); + } + Ok(Json::Null) + }) +} + +/// The block a chart's series come from, and whether each series is a row of it: the bounding +/// range of the series names, categories and values, when they're all references to one sheet. +fn series_block(series: &[Series]) -> Option<(String, bool)> { + let mut sheet: Option = None; + let mut bounds: Option = None; + let mut rows = 0; + for s in series { + for (i, f) in [Some(&s.values), s.categories.as_ref(), s.name.as_ref()].into_iter().flatten().enumerate() { + let Ok(gridcraft_formula::Expr::Ref(r)) = gridcraft_formula::parse(f) else { return None }; + let gridcraft_formula::SheetSel::Named(n) = &r.sheet else { return None }; + if !sheet.get_or_insert_with(|| n.clone()).eq_ignore_ascii_case(n) { + return None; + } + let range = r.range(); + if i == 0 && range.height() == 1 && range.width() > 1 { + rows += 1; + } + bounds = Some(bounds.map_or(range, |b| b.union(&range))); + } + } + Some((format!("{}!{}", gridcraft_formula::quote_sheet(&sheet?), bounds?.a1()), rows > 0 && rows == series.len())) +} + +fn chart_delete(s: &mut Session, p: &Json) -> Result { + let (si, ci) = find_chart(s, p)?; + edit(s, |cx| { + cx.sheet_mut(si)?.charts.remove(ci); + Ok(Json::Null) + }) +} + +fn insert_sparkline(s: &mut Session, p: &Json) -> Result { + let src = str_param(p, "range").ok_or_else(|| bad("insert.sparkline", "missing `range`"))?.to_string(); + let loc = cell_param(s, p, "location")?.ok_or_else(|| bad("insert.sparkline", "missing `location`"))?; + let kind = match str_param(p, "type") { + Some("column") => SparklineKind::Column, + Some("winLoss") => SparklineKind::WinLoss, + _ => SparklineKind::Line, + }; + let markers = bool_param(p, "markers").unwrap_or(false); + let sheet = s.doc()?.wb.active_sheet(); + edit(s, |cx| { + let sh = cx.sheet_mut(sheet)?; + sh.sparklines.retain(|x| x.cell != loc); + sh.sparklines.push(Sparkline { cell: loc, source: src.clone(), kind, color: Color::Theme(4, 0), markers }); + Ok(Json::Null) + }) +} + +fn insert_picture(s: &mut Session, p: &Json) -> Result { + let placement = str_param(p, "placement").unwrap_or("overCells"); + if !matches!(placement, "cell" | "overCells") { + return Err(bad("insert.picture", "placement must be cell or overCells")); + } + let data = if let Some(b) = str_param(p, "base64") { + crate::io::base64_decode(b).ok_or_else(|| bad("insert.picture", "invalid base64"))? + } else if let Some(path) = str_param(p, "path") { + crate::io::read_file(path)? + } else { + s.ui_requests.push(crate::UiRequest::Dialog("insertPicture".into(), json!({"placement": placement}))); + return ok(); + }; + if placement == "cell" { + let mime = validate_cell_picture(&data)?; + let d = s.doc()?; + let sheet = d.wb.active_sheet(); + let sh = d.wb.active().ok_or(EngineError::NoDocument)?; + let at = match str_param(p, "at") { + Some(a) => CellRef::parse(a).ok_or_else(|| bad("insert.picture", "invalid cell address"))?, + None => d.selection.active, + }; + let at = sh.merge_at(at).map(|r| r.start).unwrap_or(at); + let style = sh.style_id(at); + if sh.is_protected() && d.wb.styles.get(style).protection.locked && locked_cell_error(sh, at, &d.unlocked_ranges).is_some() { + return Err(EngineError::Other("The cell you're trying to change is on a protected sheet.".into())); + } + let picture = CellPicture { data, mime: mime.into(), alt: str_param(p, "alt").unwrap_or("").into() }; + return edit(s, |cx| { + cx.sheet_mut(sheet)?.set_picture(at, std::sync::Arc::new(picture)); + cx.touch(sheet, at); + Ok(json!({"cell": at.a1(), "placement": "cell"})) + }); + } + if data.len() > 64 * 1024 * 1024 { + return Err(bad("insert.picture", "image larger than 64 MB")); + } + let mime = str_param(p, "mime") + .map(str::to_string) + .unwrap_or_else(|| if data.starts_with(&[0x89, b'P', b'N', b'G']) { "image/png".into() } else { "image/jpeg".into() }); + let (w, h) = crate::io::image_size(&data).unwrap_or((320, 240)); + let d = s.doc()?; + let id = d.wb.next_object_id(); + let at = cell_param(s, p, "at")?.unwrap_or(d.selection.active); + let scale = (640.0 / w.max(1) as f32).min(1.0); + let img = Image { + id, + anchor: Anchor { + cell: at, + dx: 0.0, + dy: 0.0, + width: f64_param(p, "width").map(|v| v as f32).unwrap_or(w as f32 * scale), + height: f64_param(p, "height").map(|v| v as f32).unwrap_or(h as f32 * scale), + // Excel inserts pictures as "Move but don't size with cells" (`editAs="oneCell"`). + mode: anchor_mode_param(p).unwrap_or(gridcraft_model::AnchorMode::MoveOnly), + }, + data: data.into(), + mime, + alt: str_param(p, "alt").unwrap_or("").into(), + }; + let sheet = d.wb.active_sheet(); + edit(s, |cx| { + cx.sheet_mut(sheet)?.images.push(img); + Ok(json!({"image": id})) + }) +} + +/// Validate before mutating the workbook; header dimensions alone do not prove the pixels decode. +fn validate_cell_picture(data: &[u8]) -> Result<&'static str> { + if data.len() > 16 * 1024 * 1024 { + return Err(bad("insert.picture", "cell pictures must be at most 16 MiB")); + } + let format = image::guess_format(data).map_err(|_| bad("insert.picture", "choose a valid PNG or JPEG picture"))?; + let mime = match format { + image::ImageFormat::Png => "image/png", + image::ImageFormat::Jpeg => "image/jpeg", + _ => return Err(bad("insert.picture", "cell pictures support PNG and JPEG")), + }; + let (width, height) = image::ImageReader::with_format(std::io::Cursor::new(data), format) + .into_dimensions() + .map_err(|_| bad("insert.picture", "invalid image dimensions"))?; + if width == 0 || height == 0 || width > 8192 || height > 8192 || u64::from(width) * u64::from(height) > 16_000_000 { + return Err(bad("insert.picture", "cell pictures are limited to 8192 pixels per side and 16 million pixels")); + } + let mut reader = image::ImageReader::with_format(std::io::Cursor::new(data), format); + let mut limits = image::Limits::default(); + limits.max_image_width = Some(8192); + limits.max_image_height = Some(8192); + limits.max_alloc = Some(64 * 1024 * 1024); + reader.limits(limits); + reader.decode().map_err(|_| bad("insert.picture", "picture cannot be decoded within the 64 MiB limit"))?; + Ok(mime) +} + +fn insert_shape(s: &mut Session, p: &Json) -> Result { + let kind = match str_param(p, "kind").unwrap_or("rectangle") { + "roundedRectangle" => ShapeKind::RoundedRectangle, + "ellipse" | "oval" => ShapeKind::Ellipse, + "triangle" => ShapeKind::Triangle, + "line" => ShapeKind::Line, + "arrow" => ShapeKind::Arrow, + "textBox" => ShapeKind::TextBox, + _ => ShapeKind::Rectangle, + }; + let d = s.doc()?; + let id = d.wb.next_object_id(); + let at = cell_param(s, p, "at")?.unwrap_or(d.selection.active); + let fill = + super::format::color_param(p.get("fill")).unwrap_or(if kind == ShapeKind::TextBox { Color::rgb(255, 255, 255) } else { Color::Theme(4, 0) }); + let line = super::format::color_param(p.get("line")).unwrap_or(if kind == ShapeKind::TextBox { + Color::rgb(0x80, 0x80, 0x80) + } else { + Color::Theme(4, -250) + }); + let shape = Shape { + id, + kind, + anchor: Anchor { + cell: at, + dx: 0.0, + dy: 0.0, + width: f64_param(p, "width").unwrap_or(144.0) as f32, + height: f64_param(p, "height").unwrap_or(if kind == ShapeKind::Line { 0.0 } else { 96.0 }) as f32, + mode: anchor_mode_param(p).unwrap_or_default(), + }, + fill, + line, + text: str_param(p, "text").unwrap_or("").into(), + format: Default::default(), + }; + let sheet = d.wb.active_sheet(); + edit(s, |cx| { + cx.sheet_mut(sheet)?.shapes.push(shape); + Ok(json!({"shape": id})) + }) +} + +/// A text box holds at most as many characters as a cell. +const SHAPE_TEXT_MAX_CHARS: usize = 32_767; + +fn shape_set_text(s: &mut Session, p: &Json) -> Result { + let id = + p.get("id").and_then(Json::as_u64).and_then(|id| u32::try_from(id).ok()).ok_or_else(|| bad("shape.setText", "`id` must be a text box id"))?; + let text = str_param(p, "text").ok_or_else(|| bad("shape.setText", "`text` must be a string"))?; + if text.chars().nth(SHAPE_TEXT_MAX_CHARS).is_some() { + return Err(bad("shape.setText", "`text` is longer than 32,767 characters")); + } + let d = s.doc()?; + let sheet = d.wb.active_sheet(); + let sh = d.wb.active().ok_or_else(|| bad("shape.setText", "no such text box"))?; + let shape = sh.shapes.iter().find(|shape| shape.id == id).ok_or_else(|| bad("shape.setText", "no such text box"))?; + if shape.kind != ShapeKind::TextBox { + return Err(bad("shape.setText", "the shape is not a text box")); + } + // Sheet protection locks drawing objects (there is no "edit objects" allowance yet). + if sh.is_protected() { + return Err(EngineError::Other(PROTECTED.into())); + } + if shape.text == text { + return ok(); + } + edit(s, |cx| { + let shape = cx.sheet_mut(sheet)?.shapes.iter_mut().find(|shape| shape.id == id).ok_or_else(|| bad("shape.setText", "no such text box"))?; + shape.text = text.into(); + Ok(Json::Null) + }) +} + +fn delete_object(s: &mut Session, p: &Json) -> Result { + let id = u32_param(p, "id").ok_or_else(|| bad("object.delete", "missing `id`"))?; + let kind = str_param(p, "kind").unwrap_or("").to_string(); + let sheet = s.doc()?.wb.active_sheet(); + edit(s, |cx| { + let sh = cx.sheet_mut(sheet)?; + match kind.as_str() { + "chart" => sh.charts.retain(|c| c.id != id), + "image" => sh.images.retain(|c| c.id != id), + "slicer" => sh.slicers.retain(|c| c.id != id), + _ => sh.shapes.retain(|c| c.id != id), + } + Ok(Json::Null) + }) +} + +fn move_object(s: &mut Session, p: &Json) -> Result { + let id = u32_param(p, "id").ok_or_else(|| bad("object.move", "missing `id`"))?; + let kind = str_param(p, "kind").unwrap_or("").to_string(); + let sheet = s.doc()?.wb.active_sheet(); + let at = cell_param(s, p, "at")?; + edit(s, |cx| { + let sh = cx.sheet_mut(sheet)?; + let anchor = match kind.as_str() { + "chart" => sh.charts.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), + "image" => sh.images.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), + "slicer" => sh.slicers.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), + "shape" => sh.shapes.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), + _ => None, + }; + let Some(a) = anchor else { return Err(bad("object.move", "no such object")) }; + if let Some(at) = at { + a.cell = at; + } + if let Some(v) = f64_param(p, "dx") { + a.dx = v as f32; + } + if let Some(v) = f64_param(p, "dy") { + a.dy = v as f32; + } + if let Some(v) = f64_param(p, "width") { + a.width = v.clamp(1.0, 10000.0) as f32; + } + if let Some(v) = f64_param(p, "height") { + a.height = v.clamp(0.0, 10000.0) as f32; + } + Ok(Json::Null) + }) +} + +/// Sets an object's anchor mode (Excel's "Move and size with cells" / "Move but don't size with +/// cells" / "Don't move or size with cells"), so it follows — or ignores — its rows and columns +/// when the sheet is sorted or edited structurally. +fn set_anchor_mode(s: &mut Session, p: &Json) -> Result { + let id = u32_param(p, "id").ok_or_else(|| bad("object.setAnchorMode", "missing `id`"))?; + let kind = str_param(p, "kind").unwrap_or("").to_string(); + if !matches!(kind.as_str(), "chart" | "image" | "shape") { + return Err(bad("object.setAnchorMode", "kind must be chart, image or shape")); + } + let mode = anchor_mode_param(p).ok_or_else(|| bad("object.setAnchorMode", "mode must be moveAndSize, moveOnly or absolute"))?; + let sheet = s.doc()?.wb.active_sheet(); + edit(s, |cx| { + let sh = cx.sheet_mut(sheet)?; + let anchor = match kind.as_str() { + "chart" => sh.charts.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), + "image" => sh.images.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), + _ => sh.shapes.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), + }; + let Some(a) = anchor else { return Err(bad("object.setAnchorMode", "no such object")) }; + a.mode = mode; + Ok(json!({"mode": anchor_mode_name(mode)})) + }) +} + +// ---------------------------------------------------------------- links & comments + +fn insert_link(s: &mut Session, p: &Json) -> Result { + let sheet = target_sheet(s, p)?; + let at = cell_param_on(s, p, "cell", sheet)?.unwrap_or(s.doc()?.selection.active); + if bool_param(p, "remove").unwrap_or(false) { + return edit(s, |cx| { + cx.sheet_mut(sheet)?.hyperlinks.remove(&at); + Ok(Json::Null) + }); + } + let Some(target) = str_param(p, "target").map(str::to_string) else { + s.ui_requests.push(crate::UiRequest::Dialog("insertLink".into(), json!({}))); + return ok(); + }; + let text = str_param(p, "text").map(str::to_string); + let tooltip = str_param(p, "tooltip").map(str::to_string); + let external = links_outside(&s.doc()?.wb, sheet, &target); + edit(s, |cx| { + let link_style = cx.wb.styles.derive(cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default(), |st| { + st.font.color = Color::Theme(10, 0); + st.font.underline = Underline::Single; + }); + let sh = cx.sheet_mut(sheet)?; + if let Some(t) = text.clone().or_else(|| sh.value(at).is_empty().then(|| target.clone())) { + sh.set_cell(at, Cell { value: gridcraft_core::Value::text(t.as_str()), formula: None, style: link_style }); + } else { + sh.set_style(at, link_style); + } + sh.hyperlinks.insert(at, Hyperlink { target: target.clone(), tooltip: tooltip.clone(), external }); + cx.touch(sheet, at); + Ok(Json::Null) + }) +} + +/// Follows a link: a reference selects it, an `http`/`https`/`mailto` address is handed to the +/// host (`UiRequest::OpenUrl`); anything else is refused ([`crate::links::classify`]). Opens no +/// dialog; the interactive click asks for confirmation first, in the UI. +fn open_link(s: &mut Session, p: &Json) -> Result { + let target = match str_param(p, "url").or_else(|| str_param(p, "target")) { + Some(t) => t.to_string(), + None => { + let sheet = target_sheet(s, p)?; + let at = cell_param_on(s, p, "cell", sheet)?.unwrap_or(s.doc()?.selection.active); + let Some(h) = s.doc()?.wb.sheet(sheet).and_then(|sh| sh.hyperlinks.get(&at)) else { + return Err(EngineError::Other("There's no hyperlink in this cell.".into())); + }; + h.target.clone() + } + }; + match crate::links::classify(&target) { + crate::links::LinkKind::Internal(r) => super::edit::go_to(s, &json!({"reference": r.trim_start_matches('#')})), + crate::links::LinkKind::External(u) => { + s.ui_requests.push(crate::UiRequest::OpenUrl(u.clone())); + Ok(json!({"opened": u})) + } + crate::links::LinkKind::Refused(why) => Err(EngineError::Other(why)), + } +} + +/// Sorts a link target as the two pages of Excel's Insert Hyperlink dialog do: "Place in This +/// Document" is a sheet reference, a defined name or `#…`; anything else ("Existing File or Web +/// Page") is outside the workbook and saved as a relationship. +fn links_outside(wb: &Workbook, sheet: usize, target: &str) -> bool { + let t = target.trim(); + if t.starts_with('#') || wb.name(t, sheet).is_some() || wb.table(t).is_some() { + return false; + } + let (on, body) = split_sheet(t); + // Sheet names can't hold `: \ / ? * [ ]`, so `https://…!A1` or `C:\…!A1` is no sheet reference, + // and `Book.xlsx#Sheet1!A1` is a place in another file unless a sheet has that name. + let on_sheet = on.is_none_or(|name| { + wb.sheets.iter().any(|s| s.name.eq_ignore_ascii_case(&name)) + || (!name.is_empty() && !name.contains([':', '\\', '/', '?', '*', '[', ']', '#'])) + }); + !(on_sheet && RangeRef::parse(body).is_some()) +} + +fn comment(s: &mut Session, p: &Json, threaded: bool) -> Result { + let sheet = target_sheet(s, p)?; + let at = cell_param_on(s, p, "cell", sheet)?.unwrap_or(s.doc()?.selection.active); + let text = str_param(p, "text").unwrap_or("").to_string(); + let named = str_param(p, "author").map(str::to_string); + let author = named.clone().unwrap_or_else(|| s.prefs.user_name.clone()); + edit(s, |cx| { + let comments = &mut cx.sheet_mut(sheet)?.comments; + // Writing a note on a cell that has one edits it, as Excel's Edit Note does: the box, the + // replies, the dates and who wrote it stay. A note replaces a comment and vice versa. + if let Some(c) = comments.get_mut(&at).filter(|c| c.threaded == threaded) { + c.text = text.clone(); + if let Some(author) = named.clone() { + c.author = author; + } + return Ok(json!({"cell": at.a1()})); + } + comments.insert( + at, + Comment { + author: author.clone(), + text: text.clone(), + replies: vec![], + threaded, + resolved: false, + visible: false, + dates: vec![], + note_box: None, + }, + ); + Ok(json!({"cell": at.a1()})) + }) +} + +fn reply_comment(s: &mut Session, p: &Json) -> Result { + let sheet = target_sheet(s, p)?; + let at = cell_param_on(s, p, "cell", sheet)?.unwrap_or(s.doc()?.selection.active); + let text = str_param(p, "text").unwrap_or("").to_string(); + let author = str_param(p, "author").map(str::to_string).unwrap_or_else(|| s.prefs.user_name.clone()); + edit(s, |cx| { + let c = cx.sheet_mut(sheet)?.comments.get_mut(&at).ok_or_else(|| EngineError::Other("There's no comment in this cell.".into()))?; + c.replies.push((author.clone(), text.clone())); + Ok(Json::Null) + }) +} + +fn resolve_comment(s: &mut Session, p: &Json) -> Result { + let sheet = target_sheet(s, p)?; + let at = cell_param_on(s, p, "cell", sheet)?.unwrap_or(s.doc()?.selection.active); + let v = bool_param(p, "resolved").unwrap_or(true); + edit(s, |cx| { + if let Some(c) = cx.sheet_mut(sheet)?.comments.get_mut(&at) { + c.resolved = v; + } + Ok(Json::Null) + }) +} + +fn delete_comment(s: &mut Session, p: &Json) -> Result { + let sheet = target_sheet(s, p)?; + let ranges = target_ranges(s, p)?; + edit(s, |cx| { + cx.sheet_mut(sheet)?.comments.retain(|c, _| !ranges.iter().any(|r| r.contains(*c))); + Ok(Json::Null) + }) +} + +fn show_note(s: &mut Session, p: &Json) -> Result { + let sheet = target_sheet(s, p)?; + let at = cell_param_on(s, p, "cell", sheet)?.unwrap_or(s.doc()?.selection.active); + edit(s, |cx| { + if let Some(c) = cx.sheet_mut(sheet)?.comments.get_mut(&at) { + c.visible = !c.visible; + } + Ok(Json::Null) + }) +} + fn insert_symbol(s: &mut Session, p: &Json) -> Result { let ch = str_param(p, "char").unwrap_or("").to_string(); let d = s.doc()?; @@ -254,6 +1406,344 @@ fn insert_symbol(s: &mut Session, p: &Json) -> Result { s.execute("cell.set", json!({"cell": at.a1(), "input": format!("{cur}{ch}")})) } +// ---------------------------------------------------------------- conditional formatting + +fn preset_style(name: &str) -> Style { + let mut s = Style::default(); + match name { + "redText" => s.font.color = Color::rgb(0x9C, 0x00, 0x06), + "yellowFill" | "yellowFillDarkYellowText" => { + s.fill = Fill::solid(Color::rgb(0xFF, 0xEB, 0x9C)); + s.font.color = Color::rgb(0x9C, 0x57, 0x00); + } + "greenFill" | "greenFillDarkGreenText" => { + s.fill = Fill::solid(Color::rgb(0xC6, 0xEF, 0xCE)); + s.font.color = Color::rgb(0x00, 0x61, 0x00); + } + "lightRedFill" => s.fill = Fill::solid(Color::rgb(0xFF, 0xC7, 0xCE)), + "redBorder" => { + let l = BorderLine { style: BorderStyle::Thin, color: Color::rgb(0xC0, 0x00, 0x00) }; + s.borders = Borders { left: l, right: l, top: l, bottom: l, ..Default::default() }; + } + _ => { + s.fill = Fill::solid(Color::rgb(0xFF, 0xC7, 0xCE)); + s.font.color = Color::rgb(0x9C, 0x00, 0x06); + } + } + s +} + +/// Whether `v` is one formula string literal: `"…"` with every inner quote doubled. +fn is_string_literal(v: &str) -> bool { + v.len() >= 2 && v.starts_with('"') && v.ends_with('"') && v.get(1..v.len() - 1).is_some_and(|inner| !inner.replace("\"\"", "").contains('"')) +} + +/// A conditional format's `style` as the model's `Style` JSON (`font`, `fill`, `borders`, `align`, +/// `num_fmt`, `protection`, merged over the default by the caller), where a colour may also be +/// written as `format_range` takes it (`"#RRGGBB"`, `{theme, tint}`, `"auto"`) and `fill` may be a +/// colour alone (a solid fill) or `"none"` (no fill). An unknown key is an error rather than +/// dropped: dropped, it left a rule with an empty format that matches and shows nothing. +fn cf_style_json(st: &Json) -> Result { + const KEYS: [&str; 6] = ["font", "fill", "borders", "align", "num_fmt", "protection"]; + let err = |m: String| bad("home.conditionalFormat", m); + let Some(obj) = st.as_object() else { return Err(err("`style` must be an object, e.g. {\"fill\": \"#FFC7CE\"}".into())) }; + let mut st = obj.clone(); + if let Some(k) = st.keys().find(|k| !KEYS.contains(&k.as_str())) { + let hint = if k == "preset" { "; a preset goes in `rule.preset`" } else { "" }; + return Err(err(format!("unknown key `{k}` in `style` (known: {}){hint}", KEYS.join(", ")))); + } + // A colour written the way the other formatting commands take it, as the model stores it. + let color = |v: &mut Json, what: &str| -> Result<()> { + if v.is_string() || v.get("theme").is_some() { + let c = super::format::color_param(Some(v)).ok_or_else(|| err(format!("invalid colour {v} for `{what}`")))?; + *v = serde_json::to_value(c).map_err(|e| EngineError::Other(e.to_string()))?; + } + Ok(()) + }; + if let Some(f) = st.get_mut("font").and_then(Json::as_object_mut) + && let Some(c) = f.get_mut("color") + { + color(c, "font.color")?; + } + if let Some(fill) = st.get_mut("fill") { + // `"none"` (or `"auto"`) is no fill, as `home.fillColor` reads it, not a solid automatic one. + if fill.is_string() && super::format::color_param(Some(fill)) == Some(Color::Auto) { + *fill = serde_json::to_value(Fill::default()).map_err(|e| EngineError::Other(e.to_string()))?; + } else if fill.is_string() || fill.get("theme").is_some() { + *fill = json!({"pattern": "Solid", "fg": fill.clone()}); + } + if let Some(f) = fill.as_object_mut() { + for k in ["fg", "bg"] { + if let Some(c) = f.get_mut(k) { + color(c, &format!("fill.{k}"))?; + } + } + // A colour with no pattern is a solid fill, not an invisible one. + if f.contains_key("fg") && !f.contains_key("pattern") { + f.insert("pattern".into(), json!("Solid")); + } + } + } + if let Some(b) = st.get_mut("borders").and_then(Json::as_object_mut) { + for (side, line) in b.iter_mut() { + if let Some(c) = line.get_mut("color") { + color(c, &format!("borders.{side}.color"))?; + } + } + } + Ok(Json::Object(st)) +} + +/// #1999: a formula that doesn't parse must never reach the workbook — Excel refuses to open +/// files carrying it, even in repair mode. Validated here so the API and the New Rule dialog +/// (which dispatches this command) share one gate, the way Excel's object model refuses. +fn check_cf_formula(cmd: &str, what: &str, formula: &str) -> Result<()> { + // Strip exactly one leading '=', the way the XLSX writer does: a value like '===A1' + // only parses after several strips and would still be saved unparseable. (#1999 review) + if gridcraft_formula::parse(formula.strip_prefix('=').unwrap_or(formula)).is_err() { + return Err(bad(cmd, format!("invalid {what} formula `{formula}` — it does not parse; Excel would reject the saved file"))); + } + Ok(()) +} + +fn check_cf_value(what: &str, v: &gridcraft_model::features::CfValueKind) -> Result<()> { + if let gridcraft_model::features::CfValueKind::Formula(f) = v { + check_cf_formula("home.conditionalFormat", what, f)?; + } + Ok(()) +} + +fn check_cf_rule(rule: &CfRule) -> Result<()> { + const CMD: &str = "home.conditionalFormat"; + match rule { + CfRule::CellIs { a, b, .. } => { + check_cf_formula(CMD, "comparison value", a)?; + if let Some(b) = b { + check_cf_formula(CMD, "second comparison value", b)?; + } + } + CfRule::Expression { formula, .. } => check_cf_formula(CMD, "rule", formula)?, + CfRule::DataBar { min, max, .. } => { + check_cf_value("data bar minimum", min)?; + check_cf_value("data bar maximum", max)?; + } + CfRule::ColorScale { stops } => { + for (kind, _) in stops { + check_cf_value("color scale threshold", kind)?; + } + } + _ => {} + } + Ok(()) +} + +fn add_cf(s: &mut Session, p: &Json) -> Result { + let ranges = target_ranges(s, p)?; + let sheet = target_sheet(s, p)?; + let Some(rule) = p.get("rule") else { + s.ui_requests.push(crate::UiRequest::Dialog("newFormattingRule".into(), json!({}))); + return ok(); + }; + let g = |k: &str| rule.get(k).and_then(Json::as_str).map(str::to_string); + let style = Box::new(match rule.get("style") { + Some(st) if !st.is_null() => { + let st = cf_style_json(st)?; + let base = serde_json::to_value(Style::default()).map_err(|e| EngineError::Other(e.to_string()))?; + serde_json::from_value(super::format::merge_json(base, &st)).map_err(|e| bad("home.conditionalFormat", e.to_string()))? + } + _ => preset_style(g("preset").as_deref().unwrap_or("lightRedFillDarkRedText")), + }); + let kind = g("type").unwrap_or_else(|| "cellIs".into()); + let value_kind = |v: Option<&Json>, default: CfValueKind| -> CfValueKind { + match v { + Some(Json::Object(o)) => { + let t = o.get("type").and_then(Json::as_str).unwrap_or("num"); + let val = o.get("value").and_then(Json::as_f64).unwrap_or(0.0); + match t { + "min" => CfValueKind::Min, + "max" => CfValueKind::Max, + "percent" => CfValueKind::Percent(val), + "percentile" => CfValueKind::Percentile(val), + "formula" => CfValueKind::Formula(o.get("value").and_then(Json::as_str).unwrap_or("").into()), + _ => CfValueKind::Number(val), + } + } + _ => default, + } + }; + let rule = match kind.as_str() { + "cellIs" => { + // Excel's file and VBA names (`lessThan`, …) are accepted too. + let op = match g("operator").as_deref().unwrap_or("greater") { + "between" => CfOperator::Between, + "notBetween" => CfOperator::NotBetween, + "equal" => CfOperator::Equal, + "notEqual" => CfOperator::NotEqual, + "less" | "lessThan" => CfOperator::Less, + "greaterOrEqual" | "greaterThanOrEqual" => CfOperator::GreaterOrEqual, + "lessOrEqual" | "lessThanOrEqual" => CfOperator::LessOrEqual, + "greater" | "greaterThan" => CfOperator::Greater, + other => return Err(bad("home.conditionalFormat", format!("unknown operator `{other}`"))), + }; + let a = rule.get("value").map(super::edit::json_to_input).unwrap_or_default(); + let b = rule.get("value2").map(super::edit::json_to_input); + let quote = |v: String| { + // A number, a formula, or text already written as a string literal (`"Closed"`, as + // in the file and in openpyxl's `CellIsRule`) is kept; other text is quoted. + if gridcraft_core::parse::parse_number_text(&v).is_some() || v.starts_with('=') { + v.trim_start_matches('=').to_string() + } else if is_string_literal(&v) { + v + } else { + format!("\"{}\"", v.replace('"', "\"\"")) + } + }; + CfRule::CellIs { op, a: quote(a), b: b.map(quote), style } + } + "expression" => CfRule::Expression { formula: g("formula").unwrap_or_default().trim_start_matches('=').to_string(), style }, + "containsText" => CfRule::ContainsText { text: g("text").unwrap_or_default(), style }, + "notContainsText" => CfRule::NotContainsText { text: g("text").unwrap_or_default(), style }, + "beginsWith" => CfRule::BeginsWith { text: g("text").unwrap_or_default(), style }, + "endsWith" => CfRule::EndsWith { text: g("text").unwrap_or_default(), style }, + "blanks" => CfRule::Blanks { style }, + "noBlanks" => CfRule::NoBlanks { style }, + "errors" => CfRule::Errors { style }, + "noErrors" => CfRule::NoErrors { style }, + "duplicate" => CfRule::Duplicate { style }, + "unique" => CfRule::Unique { style }, + "top10" => CfRule::Top10 { + bottom: rule.get("bottom").and_then(Json::as_bool).unwrap_or(false), + percent: rule.get("percent").and_then(Json::as_bool).unwrap_or(false), + rank: rule.get("rank").and_then(Json::as_u64).unwrap_or(10) as u32, + style, + }, + "aboveAverage" => CfRule::AboveAverage { below: rule.get("below").and_then(Json::as_bool).unwrap_or(false), equal: false, std_dev: 0, style }, + "timePeriod" => CfRule::TimePeriod { period: g("period").unwrap_or_else(|| "today".into()), style }, + "colorScale" => { + let cols: Vec = rule + .get("colors") + .and_then(Json::as_array) + .map(|a| a.iter().filter_map(|c| super::format::color_param(Some(c))).collect()) + .unwrap_or_else(|| vec![Color::rgb(0xF8, 0x69, 0x6B), Color::rgb(0xFF, 0xEB, 0x84), Color::rgb(0x63, 0xBE, 0x7B)]); + let stops = match cols.len() { + 2 => vec![(CfValueKind::Min, cols[0]), (CfValueKind::Max, cols[1])], + _ => vec![ + (CfValueKind::Min, cols.first().copied().unwrap_or_default()), + (CfValueKind::Percentile(50.0), cols.get(1).copied().unwrap_or_default()), + (CfValueKind::Max, cols.get(2).copied().unwrap_or_default()), + ], + }; + CfRule::ColorScale { stops } + } + "dataBar" => CfRule::DataBar { + min: value_kind(rule.get("min"), CfValueKind::Min), + max: value_kind(rule.get("max"), CfValueKind::Max), + color: super::format::color_param(rule.get("color")).unwrap_or(Color::rgb(0x63, 0x8E, 0xC6)), + gradient: rule.get("gradient").and_then(Json::as_bool).unwrap_or(true), + show_value: true, + }, + "iconSet" => CfRule::IconSet { + set: g("set").unwrap_or_else(|| "3TrafficLights1".into()), + thresholds: vec![], + reverse: rule.get("reverse").and_then(Json::as_bool).unwrap_or(false), + show_value: true, + }, + other => return Err(bad("home.conditionalFormat", format!("unknown rule type `{other}`"))), + }; + check_cf_rule(&rule)?; + edit(s, |cx| { + let sh = cx.sheet_mut(sheet)?; + let priority = sh.cond_formats.iter().map(|c| c.priority).max().unwrap_or(0) + 1; + // New rules take precedence over older ones. + for cf in sh.cond_formats.iter_mut() { + cf.priority += 1; + } + sh.cond_formats.insert(0, CondFormat { ranges: ranges.clone(), rule, priority: 1, stop_if_true: false }); + let _ = priority; + Ok(json!({"rules": sh.cond_formats.len()})) + }) +} + +fn clear_cf(s: &mut Session, p: &Json) -> Result { + let sheet = target_sheet(s, p)?; + let whole = bool_param(p, "sheet").unwrap_or(false); + let ranges = target_ranges(s, p)?; + edit(s, |cx| { + let sh = cx.sheet_mut(sheet)?; + if whole { + sh.cond_formats.clear(); + } else { + for cf in sh.cond_formats.iter_mut() { + cf.ranges.retain(|r| !ranges.iter().any(|x| x.intersects(r))); + } + sh.cond_formats.retain(|c| !c.ranges.is_empty()); + } + Ok(Json::Null) + }) +} + +fn manage_cf(s: &mut Session, p: &Json) -> Result { + let sheet = target_sheet(s, p)?; + if p.get("delete").is_none() && p.get("moveUp").is_none() && p.get("moveDown").is_none() { + let list: Vec = s + .doc()? + .wb + .sheet(sheet) + .map(|sh| { + sh.cond_formats.iter().map(|c| json!({"ranges": c.ranges.iter().map(|r| r.a1()).collect::>(), "rule": c.rule})).collect() + }) + .unwrap_or_default(); + return Ok(json!({"rules": list})); + } + edit(s, |cx| { + let sh = cx.sheet_mut(sheet)?; + // The count is taken again after each step: a delete shortens the list. + if let Some(i) = u32_param(p, "delete").map(|v| v as usize) + && i < sh.cond_formats.len() + { + sh.cond_formats.remove(i); + } + if let Some(i) = u32_param(p, "moveUp").map(|v| v as usize) + && i > 0 + && i < sh.cond_formats.len() + { + sh.cond_formats.swap(i, i - 1); + } + if let Some(i) = u32_param(p, "moveDown").map(|v| v as usize) + && i + 1 < sh.cond_formats.len() + { + sh.cond_formats.swap(i, i + 1); + } + for (k, cf) in sh.cond_formats.iter_mut().enumerate() { + cf.priority = k as u32 + 1; + } + Ok(Json::Null) + }) +} + +fn insert_icon(s: &mut Session, p: &Json) -> Result { + let name = str_param(p, "name").unwrap_or("check").to_string(); + let size = f64_param(p, "size").unwrap_or(48.0).clamp(8.0, 1000.0) as f32; + let d = s.doc()?; + let id = d.wb.next_object_id(); + let at = cell_param(s, p, "at")?.unwrap_or(d.selection.active); + let color = super::format::color_param(p.get("color")).unwrap_or(Color::Theme(1, 0)); + let shape = Shape { + id, + kind: ShapeKind::Icon, + anchor: Anchor { cell: at, dx: 4.0, dy: 4.0, width: size, height: size, mode: anchor_mode_param(p).unwrap_or_default() }, + fill: color, + line: Color::Auto, + text: name, + format: Default::default(), + }; + let sheet = d.wb.active_sheet(); + edit(s, |cx| { + cx.sheet_mut(sheet)?.shapes.push(shape); + Ok(json!({"shape": id})) + }) +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/engine/src/cmd/insert/table.rs b/crates/engine/src/cmd/insert/table.rs index 45f5a2162..381a9e64b 100644 --- a/crates/engine/src/cmd/insert/table.rs +++ b/crates/engine/src/cmd/insert/table.rs @@ -60,7 +60,7 @@ pub(super) fn insert_table(s: &mut Session, p: &Json) -> Result { let e = Edit::InsertCells { axis: Axis::Rows, range: RangeRef::new(r.start, CellRef::new(r.start.row, r.end.col)) }; shift_cell_features(sh, &e); let name = sh.name.clone(); - rewrite(&mut cx.wb, &name, &e); + rewrite(cx, &name, &e); let sh = cx.sheet_mut(sheet)?; for (i, c) in (r.start.col..=r.end.col).enumerate() { sh.set_value(CellRef::new(r.start.row, c), gridcraft_core::Value::text(format!("Column{}", i + 1))); diff --git a/crates/engine/src/cmd/macros.rs b/crates/engine/src/cmd/macros.rs new file mode 100644 index 000000000..396e0e28f --- /dev/null +++ b/crates/engine/src/cmd/macros.rs @@ -0,0 +1,372 @@ +//! Macros: typed command lists that run as one undo step. See [`crate::macros`] for the model, +//! tokens, runner, debugger, recorder and policy. Automate scripts (`automate.*`) are a separate, +//! older feature with their own storage; these commands neither read nor change it. A recording +//! keeps the steps a script run ran while it recorded, as the run reported them. + +use serde_json::{Value as Json, json}; + +use super::{CommandSpec, EngineError, Result, Session, bad, bool_param, cmd, edit, has_doc, str_param}; +use crate::macros::recorder::RecordingStart; + +pub fn specs() -> Vec { + vec![ + cmd!(query "macro.list", "List Macros", [], None, "{} → [{name, description, shortcut?, steps, inputs: [names], onConflict, permissions}]", has_doc, list), + cmd!(query "macro.get", "Get Macro", [], None, "{name} → the macro: {v, name, description, shortcut?, inputs, steps, output, permissions}", has_doc, get), + cmd!( + "macro.save", + "Save Macro", + [], + None, + "{macro: {name, description?, shortcut?, inputs?: [{name, kind: cell|range, rows: [min, max], cols: [min, max], header, colTypes: [number|text|date|bool|any], nonEmpty, defaultRef?}], steps: [{type: command, id, params, enabled?, breakpoint?, note?} | {type: repeat, over, of: rows|columns, max, body} | {type: if, cell, op: eq|ne|gt|ge|lt|le|empty|notEmpty|contains, value, then, else} | {type: stop, message, rollback?}], output?: {anchor?, onConflict: ask|fail|overwrite|shiftDown|shiftRight}, permissions?: [command ids]} (or those fields at the top level), replace?: bool (default false: saving over a macro with the name is refused)} → {name, steps}", + has_doc, + save + ), + cmd!("macro.delete", "Delete Macro", [], None, "{name}", has_doc, delete), + cmd!("macro.rename", "Rename Macro", [], None, "{name, to}", has_doc, rename), + cmd!( + "macro.run", + "Run Macro", + [], + None, + "{name, inputs?: {Input: \"B2:D9\"} | \"B2:D9\", onConflict?: ask|fail|overwrite|shiftDown|shiftRight, overwrite?: bool, approve?: bool, timeLimit?: seconds} runs as one undo step; any error rolls it all back → {macro, ran, inputs, stopped?, shifted?}", + has_doc, + run + ), + cmd!( + query "macro.check", + "Check Macro", + [], + None, + "{name, inputs?, onConflict?, overwrite?, timeLimit?: seconds} validates the inputs and dry-runs the macro on the workbook (then puts everything back) → {ok (macro.run with these parameters would go ahead without a question), errors?, needsApproval?, onConflict, inputs, output?, conflicts, footprint}", + has_doc, + check + ), + cmd!( + noundo "macro.record", + "Record Macro", + ["View", "Macros"], + None, + "{stop?: bool, cancel?: bool (ends a recording without saving it), relative?: bool (default true; when starting), name?, shortcut?: \"Ctrl+Shift+M\" | \"M\", description?} toggles recording; when starting, the name (new and valid) and the shortcut (free) are checked and kept for the save, and without a name the macro gets the next MacroN; when stopping, a name or description given then wins (a taken name gets a suffix); with relative references the selection when it starts is the example input → {recording, name?, relative?, input?} or {recording: false, saved, steps, dropped, skipped, inputs, shortcutDropped? (the shortcut another macro took while recording)}", + has_doc, + record + ), + cmd!( + "macro.debug", + "Step Into", + [], + None, + "{action: start|step|continue|runTo|stop|state, name, inputs?, onConflict?, overwrite?, approve?, timeLimit?: seconds per action (start, as macro.run), step: path (runTo), keep?: bool (stop), breakpoints?: [\"3\", \"2.1\"] (steps of the macro), breakpointsOnly?: bool (the list replaces the steps' own breakpoints)} debugs a macro one step at a time; stop keeps the changes as one undo step or rolls them back. Runs on its own, never as a step of a script or macro. Until it finishes or stops, its workbook runs only macro.debug, queries, selection and sheet changes, Go To, Find, Copy, zoom, Switch Window, Close and Stop Action (other workbooks stay editable) → {state: paused|finished|stopped|failed, current: {step, kind, command?, params?, iteration?}, lastParams, executed, changed (at most 500), changedCells, atBreakpoint?, shifted?, wroteFiles?, message?, error?}", + has_doc, + |s, p| crate::macros::debug::debug(s, p) + ), + ] +} + +fn name_param<'a>(cmd: &str, p: &'a Json) -> Result<&'a str> { + str_param(p, "name").filter(|n| !n.trim().is_empty()).ok_or_else(|| bad(cmd, "missing `name`")) +} + +/// The index of macro `name` in the active workbook, or an error that says there is none. +fn index_of(s: &Session, name: &str) -> Result { + crate::macros::find(&s.doc()?.wb, name).ok_or_else(|| EngineError::Other(format!("There is no macro called '{name}'."))) +} + +fn list(s: &mut Session, _: &Json) -> Result { + let wb = &s.doc()?.wb; + Ok(Json::Array( + wb.macros + .iter() + .map(|m| { + let mut o = json!({ + "name": m.name, + "description": m.description, + "steps": m.step_count(), + "inputs": m.inputs.iter().map(|i| i.name.clone()).collect::>(), + "onConflict": m.output.on_conflict.name(), + "permissions": m.permissions, + }); + if let Some(k) = &m.shortcut { + o["shortcut"] = json!(k); + } + o + }) + .collect(), + )) +} + +fn get(s: &mut Session, p: &Json) -> Result { + let i = index_of(s, name_param("macro.get", p)?)?; + let wb = &s.doc()?.wb; + Ok(wb.macros.get(i).map(crate::macros::macro_json).unwrap_or(Json::Null)) +} + +fn save(s: &mut Session, p: &Json) -> Result { + let v = p.get("macro").filter(|m| m.is_object()).unwrap_or(p); + let m = crate::macros::parse_macro(v).map_err(|e| bad("macro.save", e))?; + // Replacing is asked for, never assumed: a save with a name in use is refused. + let replace = match p.get("replace") { + None | Some(Json::Null) => false, + Some(Json::Bool(b)) => *b, + Some(_) => return Err(bad("macro.save", "`replace` must be true or false")), + }; + let (name, steps) = (m.name.clone(), m.step_count()); + edit(s, |cx| { + if let Some(kept) = crate::macros::kept_name(&cx.wb, &m.name) { + return Err(EngineError::Other(taken_by_kept(&kept))); + } + match crate::macros::find(&cx.wb, &m.name) { + Some(i) if !replace => { + let existing = cx.wb.macros.get(i).map_or(m.name.as_str(), |o| o.name.as_str()); + return Err(EngineError::Other(format!("A macro called '{existing}' already exists. Pass `replace: true` to replace it."))); + } + Some(i) => { + if let Some(old) = cx.wb.macros.get_mut(i) { + *old = m; + } + } + None => { + crate::macros::check_room(&cx.wb).map_err(EngineError::Other)?; + cx.wb.macros.push(m); + } + } + Ok(()) + })?; + Ok(json!({"name": name, "steps": steps})) +} + +/// The refusal of a name that a macro kept in a newer format has. +fn taken_by_kept(name: &str) -> String { + format!( + "A macro called '{name}' made by a newer version of GridCraft is kept in this workbook; this version can't change it. Choose another name." + ) +} + +fn delete(s: &mut Session, p: &Json) -> Result { + let i = index_of(s, name_param("macro.delete", p)?)?; + edit(s, |cx| { + if i < cx.wb.macros.len() { + cx.wb.macros.remove(i); + } + Ok(Json::Null) + }) +} + +fn rename(s: &mut Session, p: &Json) -> Result { + let i = index_of(s, name_param("macro.rename", p)?)?; + let to = str_param(p, "to").map(str::trim).filter(|t| !t.is_empty()).ok_or_else(|| bad("macro.rename", "missing `to`"))?.to_string(); + if to.chars().any(char::is_control) { + return Err(bad("macro.rename", "a macro name can't hold line breaks or other control characters")); + } + let wb = &s.doc()?.wb; + if crate::macros::find(wb, &to).is_some_and(|j| j != i) { + return Err(EngineError::Other(format!("A macro called '{to}' already exists."))); + } + if let Some(kept) = crate::macros::kept_name(wb, &to) { + return Err(EngineError::Other(taken_by_kept(&kept))); + } + edit(s, |cx| { + let Some(m) = cx.wb.macros.get_mut(i) else { return Ok(()) }; + let mut renamed = m.clone(); + renamed.name = to.clone(); + // The whole check, size included: a longer name can take a macro past what a file holds. + crate::macros::check(&renamed).map_err(EngineError::Other)?; + *m = renamed; + Ok(()) + })?; + Ok(json!({"name": to})) +} + +fn run(s: &mut Session, p: &Json) -> Result { + let name = name_param("macro.run", p)?; + // While recording: the steps a recording keeps for this run, taken from the macro as it runs, + // so saving or deleting the macro before the recording stops doesn't change what it saved. + let inline = if s.is_recording() { + let wb = &s.doc()?.wb; + crate::macros::find(wb, name).and_then(|i| wb.macros.get(i)).and_then(crate::macros::recorder::inline_steps) + } else { + None + }; + let out = crate::macros::run::run(s, name, p)?; + if s.is_recording() { + // `None` when the macro's steps don't stand on their own: the recording leaves the run out. + s.script_ran = inline; + } + Ok(out) +} + +fn check(s: &mut Session, p: &Json) -> Result { + let name = name_param("macro.check", p)?; + crate::macros::run::check(s, name, p) +} + +fn record(s: &mut Session, p: &Json) -> Result { + check_record_params(s.is_recording(), p)?; + // `cancel` ends a recording without saving it: the way out of a recording whose macro + // cannot be saved (a name the check refuses, a full workbook). + if bool_param(p, "cancel").unwrap_or(false) { + let was = s.stop_recording().is_some(); + s.macro_recording = None; + return Ok(json!({"recording": false, "cancelled": was})); + } + if bool_param(p, "stop").unwrap_or(false) || s.is_recording() { + return stop_recording(s, p); + } + start_recording(s, p, bool_param(p, "relative").unwrap_or(true)) +} + +/// Refuses parameters `macro.record` would ignore (`null` counts as not given): `relative` and +/// `shortcut` when stopping (they are chosen when recording starts), `stop: false` while +/// recording (the call stops it), anything beside `cancel: true`, and parameters of the wrong +/// kind. `name` and `description` count when starting and when stopping (stopping wins). +fn check_record_params(recording: bool, p: &Json) -> Result<()> { + let given = |k: &str| p.get(k).filter(|v| !v.is_null()); + for k in ["stop", "cancel", "relative"] { + if given(k).is_some_and(|v| !v.is_boolean()) { + return Err(bad("macro.record", format!("`{k}` must be true or false"))); + } + } + for k in ["name", "description", "shortcut"] { + if given(k).is_some_and(|v| !v.is_string()) { + return Err(bad("macro.record", format!("`{k}` must be text"))); + } + } + if given("cancel").and_then(Json::as_bool) == Some(true) { + if let Some(k) = ["stop", "relative", "name", "description", "shortcut"].into_iter().find(|k| given(k).is_some()) { + return Err(bad("macro.record", format!("`cancel: true` ends the recording without saving it, so `{k}` would be ignored"))); + } + return Ok(()); + } + let stop = given("stop").and_then(Json::as_bool); + if recording && stop == Some(false) { + return Err(bad( + "macro.record", + "a recording is in progress, and this call stops it: pass `stop: true`, or `cancel: true` to end it unsaved", + )); + } + if (recording || stop == Some(true)) + && let Some(k) = ["relative", "shortcut"].into_iter().find(|k| given(k).is_some()) + { + return Err(bad("macro.record", format!("`{k}` is chosen when recording starts"))); + } + Ok(()) +} + +/// Starts recording. `name`, `shortcut` and `description` (from the Record Macro dialog) are +/// checked now, so a bad name or a taken shortcut is refused before anything is recorded. +fn start_recording(s: &mut Session, p: &Json, relative: bool) -> Result { + let d = s.doc()?; + let name = match str_param(p, "name").map(str::trim).filter(|n| !n.is_empty()) { + Some(n) => Some(crate::macros::check_new_name(&d.wb, n).map_err(|e| bad("macro.record", e))?), + None => None, + }; + let shortcut = match str_param(p, "shortcut").map(str::trim).filter(|k| !k.is_empty()) { + Some(k) => { + let letter = crate::macros::parse_shortcut(k) + .ok_or_else(|| bad("macro.record", "`shortcut` must be one letter from A to Z (used as Ctrl+Shift+letter)"))?; + if let Some(why) = crate::macros::shortcut_conflict(&d.wb, name.as_deref().unwrap_or(""), letter) { + return Err(bad("macro.record", why)); + } + Some(crate::macros::shortcut_text(letter)) + } + None => None, + }; + let description = match str_param(p, "description") { + Some(t) if t.chars().count() > gridcraft_model::MAX_MACRO_TEXT => { + return Err(bad("macro.record", format!("`description` can be at most {} characters long", gridcraft_model::MAX_MACRO_TEXT))); + } + Some(t) => Some(t.to_string()), + None => None, + }; + let mut start = RecordingStart::new(d, relative); + (start.name, start.shortcut, start.description) = (name.clone(), shortcut, description); + let input = start.selection.current().a1(); + s.start_recording(); + s.macro_recording = Some(start); + let mut out = if relative { json!({"recording": true, "relative": true, "input": input}) } else { json!({"recording": true, "relative": false}) }; + if let Some(n) = name { + out["name"] = json!(n); + } + Ok(out) +} + +/// Stops recording and saves what was recorded as a macro, in the workbook where recording +/// started. When saving fails (a bad name, a full workbook, a macro being debugged there) the +/// recording goes on, so stopping again keeps what was recorded; when that workbook was closed, +/// the recording ends unsaved. +fn stop_recording(s: &mut Session, p: &Json) -> Result { + let Some(rec) = s.stop_recording() else { return Ok(json!({"recording": false})) }; + let start = s.macro_recording.take(); + let at = match &start { + Some(st) => s + .docs + .iter() + .position(|d| d.uid == st.doc) + .ok_or_else(|| EngineError::Other("The workbook where recording started was closed, so the recording was not saved.".into()))?, + None => s.active_index(), + }; + let saved = save_recording(s, p, &rec, start.clone(), at); + if saved.is_err() { + s.recorder = Some(rec); + s.macro_recording = start; + } + saved +} + +/// Saves the recording into the workbook at index `at`. +fn save_recording(s: &mut Session, p: &Json, rec: &crate::record::Recorder, start: Option, at: usize) -> Result { + let d = s.docs.get(at).ok_or(EngineError::NoDocument)?; + // A macro being debugged there could still roll the saved macro back with its run. + if s.macro_debug.as_ref().is_some_and(|m| m.locks(d.uid)) { + return Err(EngineError::Disabled("macro.record".into(), crate::macros::debug::BUSY.into())); + } + // A recording started without `macro.record` (`Session::start_recording`) takes its example + // input from the workbook as it is now. + let start = start.unwrap_or_else(|| RecordingStart::new(d, true)); + let (doc, wb) = (d.uid, d.wb.clone()); + // A name given when stopping wins over the one given when starting; either one, if another + // macro has it, gets a suffix, so the save never replaces a macro. + let name = match str_param(p, "name").map(str::trim).filter(|n| !n.is_empty()).or(start.name.as_deref()) { + Some(n) => crate::macros::unique_name(&wb, n), + None => crate::macros::next_name(&wb), + }; + let mut converted = crate::macros::recorder::convert(&name, &start, &rec.steps, &rec.ran); + // Steps past the macro limit or its size, and ones recorded in another workbook, are left + // out and counted with the ones the recorder dropped. + let mut cut = converted.cut.saturating_add(converted.mac.steps.len().saturating_sub(gridcraft_model::MAX_MACRO_STEPS)); + converted.mac.steps.truncate(gridcraft_model::MAX_MACRO_STEPS); + if let Some(desc) = str_param(p, "description").or(start.description.as_deref()) { + converted.mac.description = desc.chars().take(gridcraft_model::MAX_MACRO_TEXT).collect(); + } + cut = cut.saturating_add(crate::macros::fit_size(&mut converted.mac)); + // The shortcut, unless another macro took it while recording; then it is left off and the + // result says so. + let mut shortcut_dropped = None; + if let Some(k) = &start.shortcut { + if crate::macros::parse_shortcut(k).is_some_and(|c| crate::macros::shortcut_conflict(&wb, &name, c).is_none()) { + converted.mac.shortcut = Some(k.clone()); + } else { + shortcut_dropped = Some(k.clone()); + } + } + let steps = converted.mac.steps.len(); + let inputs: Vec = converted.mac.inputs.iter().map(|i| i.name.clone()).collect(); + // Saved into that workbook even when another one is active now. + let active = s.active_index(); + s.set_active(at); + let saved = s.execute("macro.save", json!({"macro": crate::macros::macro_json(&converted.mac), "replace": false})); + s.set_active(active); + saved?; + // The person recorded it, so it runs without the approval a macro that came with a file + // needs, until it changes. + if let Some(d) = s.docs.iter().find(|d| d.uid == doc) + && let Some(m) = crate::macros::find(&d.wb, &name).and_then(|i| d.wb.macros.get(i)) + { + let key = (doc, crate::macros::fingerprint(m)); + s.macro_approvals.insert(key); + } + let mut out = json!({"recording": false, "saved": name, "steps": steps, "dropped": rec.dropped.saturating_add(cut), "skipped": converted.skipped, "inputs": inputs}); + if let Some(k) = shortcut_dropped { + out["shortcutDropped"] = json!(k); + } + Ok(out) +} diff --git a/crates/engine/src/cmd/sheet.rs b/crates/engine/src/cmd/sheet.rs index 276d14f13..473a877bb 100644 --- a/crates/engine/src/cmd/sheet.rs +++ b/crates/engine/src/cmd/sheet.rs @@ -232,7 +232,9 @@ fn shift_sheet_features(sh: &mut Sheet, axis: Axis, at: u32, count: u32, insert: } /// Rewrites every formula reference (and chart series) for a structural edit on `target`. -pub(super) fn rewrite(wb: &mut gridcraft_model::Workbook, target: &str, e: &Edit) { +pub(super) fn rewrite(cx: &mut Ctx, target: &str, e: &Edit) { + cx.edits.push((target.to_string(), e.clone())); + let wb = &mut cx.wb; super::edit::rewrite_all_formulas(wb, target, e); for si in 0..wb.sheets.len() { let host = wb.sheets.get(si).map(|s| s.name.clone()).unwrap_or_default(); @@ -325,7 +327,7 @@ fn insert_lines(s: &mut Session, p: &Json, axis: Axis) -> Result { // New lines take the format of the line before (Excel's "Format Same As Above"). inherit_line_format(sh, axis, at, count); } - rewrite(&mut cx.wb, &name, &Edit::Insert { axis, at, count }); + rewrite(cx, &name, &Edit::Insert { axis, at, count }); cx.structural = true; Ok(Json::Null) })?; @@ -390,7 +392,7 @@ fn delete_lines(s: &mut Session, p: &Json, axis: Axis) -> Result { } shift_sheet_features(sh, axis, at, count, false); } - rewrite(&mut cx.wb, &name, &Edit::Delete { axis, at, count }); + rewrite(cx, &name, &Edit::Delete { axis, at, count }); } cx.structural = true; Ok(Json::Null) @@ -442,7 +444,7 @@ fn move_columns(s: &mut Session, p: &Json) -> Result { sh.cells.shift_cols(before, count as i64); shift_sheet_features(sh, Axis::Cols, before, count, true); } - rewrite(&mut cx.wb, &name, &Edit::Insert { axis: Axis::Cols, at: before, count }); + rewrite(cx, &name, &Edit::Insert { axis: Axis::Cols, at: before, count }); let mut touched = Vec::new(); { @@ -480,14 +482,14 @@ fn move_columns(s: &mut Session, p: &Json) -> Result { for cell in touched { cx.touch(sheet, cell); } - rewrite(&mut cx.wb, &name, &Edit::Move { from: source, to_row: 0, to_col: destination }); + rewrite(cx, &name, &Edit::Move { from: source, to_row: 0, to_col: destination }); { let sh = cx.sheet_mut(sheet)?; sh.cells.shift_cols(delete_at + count, -(count as i64)); shift_sheet_features(sh, Axis::Cols, delete_at, count, false); } - rewrite(&mut cx.wb, &name, &Edit::Delete { axis: Axis::Cols, at: delete_at, count }); + rewrite(cx, &name, &Edit::Delete { axis: Axis::Cols, at: delete_at, count }); *cx.sel = crate::Selection { active: CellRef::new(active_row, final_start), anchor: CellRef::new(active_row, final_start), @@ -532,7 +534,7 @@ fn insert_cells(s: &mut Session, p: &Json) -> Result { } shift_cell_features(sh, &e); let name = cx.wb.sheet(sheet).map(|s| s.name.clone()).unwrap_or_default(); - rewrite(&mut cx.wb, &name, &e); + rewrite(cx, &name, &e); cx.structural = true; Ok(Json::Null) })?; @@ -567,7 +569,7 @@ fn delete_cells(s: &mut Session, p: &Json) -> Result { shift_cell_features(sh, &e); } // References into the deleted cells become #REF!; the rest move. - rewrite(&mut cx.wb, &name, &e); + rewrite(cx, &name, &e); cx.structural = true; Ok(Json::Null) })?; diff --git a/crates/engine/src/cmd/solver/lp.rs b/crates/engine/src/cmd/solver/lp.rs index 3c1462d71..dc67d7746 100644 --- a/crates/engine/src/cmd/solver/lp.rs +++ b/crates/engine/src/cmd/solver/lp.rs @@ -199,7 +199,7 @@ fn phase_one(tableau: &mut Tableau, normalized: &[(Vec, Sense, f64)], budge return Err(Outcome::Infeasible); } for i in 0..tableau.rows.len() { - if !tableau.basis.get(i).is_some_and(|basic| *basic >= tableau.decision_cols + tableau.slack_cols) { + if tableau.basis.get(i).is_none_or(|basic| *basic < tableau.decision_cols + tableau.slack_cols) { continue; } let pivot_col = tableau diff --git a/crates/engine/src/cmd/stash.rs b/crates/engine/src/cmd/stash.rs index c0e8302ca..9c4554056 100644 --- a/crates/engine/src/cmd/stash.rs +++ b/crates/engine/src/cmd/stash.rs @@ -15,7 +15,7 @@ //! noncharacter, which XML carries but real text does not use, so the escape costs nothing on //! ordinary text. Legacy `_sc_` items are read as they are. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, HashSet}; use std::fmt::Write as _; use gridcraft_model::{DefinedName, Workbook}; @@ -102,15 +102,29 @@ pub fn get(wb: &Workbook, key: &str) -> Option { assemble(&parts) } +/// What a listing found: the first [`MAX_ITEMS`] items or keys, and how many more there were. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct Listing { + pub found: T, + /// Items past the cap, not listed. A caller says so rather than letting them vanish. + pub more: usize, +} + /// The items whose key starts with `prefix` (`"macro_"`), by lower-cased key, read in one pass -/// over the names: the first [`MAX_ITEMS`] of [`keys`], without those over the size limit. -pub fn items(wb: &Workbook, prefix: &str) -> BTreeMap { - group(wb, prefix) - .into_iter() - .filter(|(_, parts)| parts.is_item()) - .take(MAX_ITEMS) - .filter_map(|(k, parts)| assemble(&parts).map(|text| (k, text))) - .collect() +/// over the names: the first [`MAX_ITEMS`] items that can be read, and the count of the readable +/// ones past them. An item over the size limit is neither listed nor counted, so it takes no +/// place another item could have. ([`keys`] lists keys without reading the items.) +pub fn items(wb: &Workbook, prefix: &str) -> Listing> { + let mut listing: Listing> = Listing::default(); + for (k, parts) in group(wb, prefix) { + let Some(text) = parts.is_item().then(|| assemble(&parts)).flatten() else { continue }; + if listing.found.len() < MAX_ITEMS { + listing.found.insert(k, text); + } else { + listing.more = listing.more.saturating_add(1); + } + } + listing } /// The text of the item held by `parts`, as [`get`] reads it. @@ -169,28 +183,44 @@ fn unescape(s: &str) -> String { /// Stores `text` under `key`, replacing what was there (in either format). Empty text removes it. pub fn set(wb: &mut Workbook, key: &str, text: &str) -> Result<()> { - check_key(key)?; - if text.chars().count() > MAX_ITEM_CHARS { - return Err(EngineError::Other(format!("The data is too large to keep in the workbook ({} MB at most).", MAX_ITEM_CHARS / (1024 * 1024)))); + set_many(wb, &[(key.to_string(), text.to_string())]) +} + +/// Stores every `(key, text)` of `items` as [`set`] would, with the checks done before anything +/// changes and one pass over the names for all of them, so writing a thousand items into a +/// workbook with many thousands of names stays near-linear. +pub fn set_many(wb: &mut Workbook, items: &[(String, String)]) -> Result<()> { + let mut planned = Vec::new(); + let mut keys: HashSet = HashSet::new(); + for (key, text) in items { + check_key(key)?; + if text.chars().count() > MAX_ITEM_CHARS { + return Err(EngineError::Other(format!( + "The data is too large to keep in the workbook ({} MB at most).", + MAX_ITEM_CHARS / (1024 * 1024) + ))); + } + // Two items under one key (names compare without case) would leave two hidden names of + // one name, of which an xlsx file keeps one: a caller's mistake, refused. + if !keys.insert(key.to_ascii_lowercase()) { + return Err(EngineError::Other(format!("`{key}` is given twice."))); + } + for (i, formula) in chunk_formulas(text).into_iter().enumerate() { + planned.push(DefinedName { name: format!("{PREFIX}{key}_{i}"), scope: None, formula, comment: "GridCraft data".into(), hidden: true }); + } } - let formulas = chunk_formulas(text); - // A visible name of the item's shape is someone's own (see `item_of`); a hidden name beside - // it with the same name would be the one an xlsx file drops. Refuse before changing anything, - // in one pass over the names (a file can carry many thousands). - let chunks = formulas.len() as u64; - let taken = |n: &&DefinedName| { - n.scope.is_none() && !n.hidden && parse_name(&n.name).is_some_and(|(k, i)| k.eq_ignore_ascii_case(key) && i.is_some_and(|i| i < chunks)) - }; - if let Some(n) = wb.names.iter().find(taken) { + // A visible name of an item's shape is someone's own (see `item_of`); a hidden name beside + // it with the same name would be the one an xlsx file drops. Refuse before changing anything. + let names: HashSet = planned.iter().map(|n| n.name.to_ascii_lowercase()).collect(); + if let Some(n) = wb.names.iter().find(|n| n.scope.is_none() && !n.hidden && names.contains(&n.name.to_ascii_lowercase())) { return Err(EngineError::Other(format!( "The workbook has a defined name '{}', which GridCraft needs for its own data. Rename it first.", n.name ))); } - remove(wb, key); - for (i, formula) in formulas.into_iter().enumerate() { - wb.names.push(DefinedName { name: format!("{PREFIX}{key}_{i}"), scope: None, formula, comment: "GridCraft data".into(), hidden: true }); - } + // What was there under these keys goes, in either format, then the new chunks come. + wb.names.retain(|n| !item_of(n).is_some_and(|(k, _)| keys.contains(&k.to_ascii_lowercase()))); + wb.names.extend(planned); Ok(()) } @@ -211,8 +241,11 @@ pub fn set_json(wb: &mut Workbook, key: &str, v: &Json) -> Result<()> { /// The keys of the items stored in the workbook that start with `prefix` (`"macro_"`), sorted /// and lower-cased: the first [`MAX_ITEMS`] of them. To read them all, [`items`] takes one pass /// where a [`get`] per key would take one each. -pub fn keys(wb: &Workbook, prefix: &str) -> Vec { - group(wb, prefix).into_iter().filter(|(_, parts)| parts.is_item()).map(|(k, _)| k).take(MAX_ITEMS).collect() +pub fn keys(wb: &Workbook, prefix: &str) -> Listing> { + let mut found: Vec = group(wb, prefix).into_iter().filter(|(_, parts)| parts.is_item()).map(|(k, _)| k).collect(); + let more = found.len().saturating_sub(MAX_ITEMS); + found.truncate(MAX_ITEMS); + Listing { found, more } } /// Whether the workbook holds an item under `name` (compared without case): a hidden workbook-scope @@ -417,10 +450,10 @@ mod tests { // Only a later chunk: no item, not listed, and the legacy name is still read. wb.names.push(name("_gc_orphan_2", "\"c\"")); assert_eq!(get(&wb, "orphan"), None); - assert_eq!(keys(&wb, "orphan"), Vec::::new()); + assert_eq!(keys(&wb, "orphan").found, Vec::::new()); wb.names.push(name("_sc_orphan", "\"old\"")); assert_eq!(get(&wb, "orphan").as_deref(), Some("old")); - assert_eq!(keys(&wb, "orphan"), ["orphan"]); + assert_eq!(keys(&wb, "orphan").found, ["orphan"]); // Chunks out of order, a duplicate, junk around the literals, an unterminated literal. wb.names.push(name("_gc_mix_1", "\"b\"&A1&")); wb.names.push(name("_gc_mix_0", "SUM(\"a\")")); @@ -468,7 +501,7 @@ mod tests { let own = DefinedName { hidden: false, ..name("_gc_watch_0", "\"mine\"") }; wb.names.push(own.clone()); assert_eq!(get(&wb, "watch"), None); - assert!(keys(&wb, "").is_empty()); + assert!(keys(&wb, "").found.is_empty()); // Writing the item would put a hidden name of the same name beside it, and an xlsx file // keeps only one of them: refused, with the names as they were. let err = set(&mut wb, "watch", "theirs").unwrap_err().to_string(); @@ -482,6 +515,18 @@ mod tests { assert_eq!(wb.names, vec![own]); } + #[test] + fn set_many_refuses_one_key_twice() { + // Two items under one key, in any case, would leave two hidden names of one name. + let mut wb = Workbook::new(); + wb.names.push(name("_gc_other_0", "\"stays\"")); + let err = set_many(&mut wb, &[("k".into(), "a".into()), ("K".into(), "b".into())]).unwrap_err().to_string(); + assert!(err.contains("given twice"), "{err}"); + assert_eq!(wb.names.len(), 1, "nothing changed"); + set_many(&mut wb, &[("k".into(), "a".into()), ("k2".into(), "b".into())]).unwrap(); + assert_eq!((get(&wb, "k").as_deref(), get(&wb, "K2").as_deref(), get(&wb, "other").as_deref()), (Some("a"), Some("b"), Some("stays"))); + } + #[test] fn a_chunk_number_with_leading_zeros_is_another_name() { // `_gc_watch_00` is not chunk 0 under another spelling: a hidden one is not read, and a @@ -489,7 +534,7 @@ mod tests { let mut wb = Workbook::new(); wb.names.push(name("_gc_watch_00", "\"theirs\"")); assert_eq!(get(&wb, "watch"), None); - assert!(keys(&wb, "").is_empty()); + assert!(keys(&wb, "").found.is_empty()); let own = DefinedName { hidden: false, ..name("_gc_watch_00", "\"mine\"") }; wb.names.push(own.clone()); set(&mut wb, "watch", "ours").unwrap(); @@ -505,7 +550,7 @@ mod tests { set(&mut wb, "macro_2", "b").unwrap(); set(&mut wb, "macro_1", &"a".repeat(9_000)).unwrap(); set(&mut wb, "scripts", "{}").unwrap(); - assert_eq!(keys(&wb, "macro_"), ["macro_1", "macro_2"]); + assert_eq!(keys(&wb, "macro_").found, ["macro_1", "macro_2"]); assert_eq!(item_key("_gc_macro_1_3").as_deref(), Some("macro_1")); assert_eq!(item_key("_SC_Scripts").as_deref(), Some("scripts")); assert_eq!(item_key("Total"), None); @@ -525,7 +570,8 @@ mod tests { wb.names.push(name("_gc_macro_orphan_1", "\"no chunk 0\"")); wb.names.push(name("_gc_other_0", "\"not a macro\"")); NAME_VISITS.with(|v| v.set(0)); - let all = items(&wb, "MACRO_"); + let listing = items(&wb, "MACRO_"); + let all = &listing.found; assert_eq!(NAME_VISITS.with(|v| v.get()), wb.names.len(), "one pass over the names"); assert_eq!(all.len(), MAX_ITEMS); let first: Vec<&String> = all.keys().take(2).collect(); @@ -535,19 +581,41 @@ mod tests { NAME_VISITS.with(|v| v.set(0)); let listed = keys(&wb, "macro_"); assert_eq!(NAME_VISITS.with(|v| v.get()), wb.names.len()); - assert_eq!(listed, all.keys().cloned().collect::>(), "keys and items agree"); + assert_eq!(listed.found, all.keys().cloned().collect::>(), "keys and items agree"); + assert_eq!((listing.more, listed.more), (10_002, 10_002), "the rest are counted, not lost"); // Under the cap every item is there, read as `get` reads it. wb.names.retain(|n| !n.name.starts_with("_gc_macro_0") && !n.name.starts_with("_gc_macro_1")); - let few = items(&wb, "macro_"); + let few = items(&wb, "macro_").found; let want: BTreeMap = [("macro_big", "b".repeat(9_000)), ("macro_legacy", "old".into())].into_iter().map(|(k, t)| (k.to_string(), t)).collect(); assert_eq!(few, want); for (k, text) in &few { assert_eq!(get(&wb, k).as_ref(), Some(text)); } - assert_eq!(keys(&wb, "macro_"), ["macro_big", "macro_legacy"]); - assert_eq!(items(&wb, "").len(), 3, "the other item too"); + assert_eq!(keys(&wb, "macro_").found, ["macro_big", "macro_legacy"]); + assert_eq!((items(&wb, "").found.len(), items(&wb, "").more), (3, 0), "the other item too, none past the cap"); + } + + #[test] + fn an_item_over_the_size_limit_takes_no_place_in_a_listing() { + // One item too large to read sorts first and one sorts last, around MAX_ITEMS readable + // ones: every readable item is listed, and neither oversized one is counted as more. + let mut wb = Workbook::new(); + let oversized = |wb: &mut Workbook, key: &str| { + for (i, formula) in chunk_formulas(&"x".repeat(MAX_ITEM_CHARS + 1)).into_iter().enumerate() { + wb.names.push(name(&format!("_gc_{key}_{i}"), &formula)); + } + }; + oversized(&mut wb, "macro_0"); + for i in 0..MAX_ITEMS { + wb.names.push(name(&format!("_gc_macro_{i:05}_0"), &format!("\"m{i}\""))); + } + oversized(&mut wb, "macro_z"); + assert_eq!(get(&wb, "macro_0"), None, "too large to read"); + let listing = items(&wb, "macro_"); + assert_eq!((listing.found.len(), listing.more), (MAX_ITEMS, 0)); + assert!(listing.found.contains_key("macro_09999")); } #[test] diff --git a/crates/engine/src/io.rs b/crates/engine/src/io.rs index 99548d4d3..60534a6ed 100644 --- a/crates/engine/src/io.rs +++ b/crates/engine/src/io.rs @@ -118,8 +118,14 @@ fn decrypt_message(name: &str, e: gridcraft_xlsx::DecryptError) -> String { } } -/// Parses bytes that aren't encrypted. +/// Parses bytes that aren't encrypted, then takes the workbook's macros out of their hidden names. fn open_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, Vec)> { + let (mut wb, mut warnings) = read_plain(name, bytes, csv_as_text)?; + warnings.extend(crate::macros::load(&mut wb)); + Ok((wb, warnings)) +} + +fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, Vec)> { let kind = FileKind::from_path(name); let sniffed = gridcraft_xlsx::sniff(bytes); if sniffed == gridcraft_xlsx::Format::Xls { @@ -160,7 +166,10 @@ fn open_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, let mut wb: Workbook = serde_json::from_slice(bytes).map_err(|e| EngineError::Other(format!("not a GridCraft JSON workbook: {e}")))?; wb.styles.rebuild_index(); if wb.sheets.is_empty() { + // The file's macros are read after this (`macros::load`), so they come along. + let macros = std::mem::take(&mut wb.macros_read); wb = Workbook::new(); + wb.macros_read = macros; } // The file's band placement and cell count are derived state: rebuild them from the // cells themselves, dropping what is outside the sheet grid (#1008). @@ -238,9 +247,12 @@ pub fn save_bytes_as(wb: &Workbook, path: &str, kind: FileKind) -> Result