mirror of
https://github.com/storytold/gridcraft.git
synced 2026-10-30 13:46:47 -06:00
prxnvt
7d1fee8c78
GridCraft keeps workbook data in hidden defined names so it travels inside the xlsx file. The existing items write one name holding one string literal, which past Excel's 255-character literal and 8,192-character formula limits gives a file Excel does not accept. cmd/stash.rs stores an item `key` as hidden workbook-scope names _gc_<key>_0, _gc_<key>_1, ..., each a formula of string literals of at most 255 characters (UTF-16 units, as Excel counts them) joined with &, at most 8,192 characters per name. Reading concatenates the chunks in order and stops at a gap; it also accepts the older single _sc_<key> name. Names are matched case-insensitively and only at workbook scope. Keys are limited to letters, digits, _ and ., items to 4,194,304 characters, and junk around or inside the literals, duplicate or out-of-order chunks, non-JSON and over-deep JSON all read safely. keys(prefix) lists the stored items that start with a prefix. Custom views, scripts, the watch list and scenarios keep their current storage; nothing reads or writes the new names yet. docs/architecture.md describes the engine building blocks added in this series: undo groups, the recording buffer, document trust and this storage.
94 lines
6.7 KiB
Markdown
94 lines
6.7 KiB
Markdown
# Architecture
|
||
|
||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** minor (engine building blocks for multi-command actions) · **Target:** Microsoft Excel (Microsoft 365)
|
||
|
||
How GridCraft is built today. Rules for contributors are in [`AGENTS.md`](../AGENTS.md).
|
||
|
||
## Workspace and layering
|
||
|
||
An engine-first Cargo workspace. Layering is enforced by `cargo xtask layers`; nothing below L6
|
||
depends on egui, eframe, winit or rfd, so the UI crate is swappable. Line counts are Rust lines
|
||
including tests, 2026-10-10.
|
||
|
||
| Layer | Crate | Lines | Job |
|
||
|---|---|---|---|
|
||
| L0 | `core` | ~1.3k | Cell and range addresses (A1, R1C1, sheet-qualified; 1,048,576 × 16,384), `Value`, `CellError`, Excel coercion, 1900/1904 date serials, typed-input parsing |
|
||
| L0 | `numfmt` | ~2.1k | Excel number-format codes (sections, conditions, colours, dates, elapsed time, fractions, scientific, fill), General, `TEXT()`; en-US only |
|
||
| L1 | `formula` | ~1.7k | Tokenizer, parser, AST, printer (A1 and R1C1), reference adjustment for copy, fill and structural edits |
|
||
| L1 | `functions` | ~13.4k | The function library: `FnSpec` registry (name, arity, category, scalar args, signature, description) and pure implementations; automatic array lifting |
|
||
| L2 | `model` | ~2.2k | Workbook, sheets, copy-on-write cell store (64-row bands behind `Arc`), interned styles, names, tables, CF, validation, comments, charts, shapes, images, sparklines, print settings, protection |
|
||
| L2 | `calc` | ~2.9k | Evaluator (references, names, tables, LET/LAMBDA, special forms like OFFSET/INDIRECT/SUBTOTAL) and dependency-graph recalculation with dynamic arrays, spills and cycle detection; single-threaded |
|
||
| L2 | `xlsx` | ~10.7k | XLSX read/write (transitional and strict read), CSV/TSV, XLSB and ODS data import, PivotTable caches, DrawingML charts and drawings; byte slices only (WASM-safe); lenient reader that reports warnings |
|
||
| L2 | `pdf` | ~1k | Small PDF 1.7 writer (14 standard fonts, vectors, images) for print and export |
|
||
| L3 | `chart` | ~2.9k | Toolkit-free chart layout to drawing primitives, a rasterizer, sparklines |
|
||
| L5 | `engine` | ~17.7k | `Session`: open workbooks, selection, history (COW snapshots), clipboard, fill, sort, filter, PivotTables, what-if, print, file I/O (`io.rs`), and the command registry (`cmd/*`, 313 commands) with the Excel ribbon catalog (`catalog.rs`) |
|
||
| L6 | `ui-egui` | ~11.6k | Excel-style UI (title bar, QAT, ribbon, formula bar, virtualised grid, editor, tabs, status bar, dialogs, task panes, chart view) and the control channel |
|
||
| L6 | `mcp` | ~1.4k | MCP server over stdio (hand-written JSON-RPC), headless or bridged to a running app |
|
||
| L7 | `apps/gridcraft`, `apps/gridcraft-cli`, `apps/gridcraft-web` | ~2.2k | Desktop (eframe/wgpu), CLI (`info`, `convert`, `eval`, `cat`, `run`, `commands`, `functions`, `mcp`, `send`), web (WASM via trunk) |
|
||
| — | `xtask` | — | `ci`, `parity`, `assets`, `layers`, `wasm`, `version`, `stats` |
|
||
|
||
## Data flow
|
||
|
||
1. **Every action is a command.** The UI, CLI, control channel and MCP call
|
||
`Session::execute(id, params)`. Commands are `CommandSpec { id, label, menu, shortcut, params,
|
||
enabled, run, journal, undoable }`; ids follow Excel's ribbon (`home.bold`, `insert.chart`,
|
||
`data.sortAscending`) plus primitives (`cell.set`, `selection.set`, `range.setValues`).
|
||
`Session::execute` catches escaped panics and restores the workbook (never-crash guard).
|
||
2. **Edits** change the model through copy-on-write bands; undo keeps the previous snapshot.
|
||
3. **Recalc** marks dependents of changed cells dirty through the dependency graph (with range
|
||
nodes for range dependents), evaluates in topological order, spills dynamic arrays and reports
|
||
`#SPILL!`/cycles. Volatile functions are always dirty.
|
||
4. **Rendering**: the egui grid reads display values (number formats applied in `engine/display.rs`)
|
||
for the visible window only; charts render through `crates/chart` primitives.
|
||
5. **Files**: `engine/io.rs` sniffs content (XLSX, XLSB, ODS) before trusting the extension and
|
||
dispatches to `crates/xlsx`; imports that lose data return warnings shown to the user.
|
||
|
||
## Engine building blocks for multi-command actions
|
||
|
||
Actions made of many commands (macros, a debugger that runs one step per call) use these parts of
|
||
`Session` and the engine:
|
||
|
||
- **Undo groups.** `Session::grouped(label, f)` runs `f` as one undo step and rolls every change
|
||
back if `f` fails; `group_begin`, `group_end` and `group_rollback` do the same across calls.
|
||
While a group is open on a document, its commands add no undo steps of their own and Undo and
|
||
Redo are disabled there. Groups nest; the outermost one makes the step.
|
||
- **Recording buffer** (`record.rs`). `Session::start_recording` collects each successful
|
||
top-level command with the document, sheet and selection it ran in, without secret parameters.
|
||
Commands run by other commands are not recorded, and the buffer is independent of the journal
|
||
(Automate › Record Actions still records through the journal).
|
||
- **Document trust.** `DocState::trust` is `Local` for workbooks made in the session and
|
||
`FromFile` for workbooks opened from a file, and for workbooks that received formulas, formats
|
||
or validation pasted from one. Active content in a `FromFile` workbook asks before it runs.
|
||
- **Workbook data in hidden names** (`cmd/stash.rs`). An item is split over hidden
|
||
workbook-scope names `_gc_<key>_0`, `_gc_<key>_1`, … of string literals within Excel's 255- and
|
||
8,192-character limits, so it survives an xlsx round trip. The older single-name `_sc_<key>`
|
||
items are read as well.
|
||
|
||
## Agent control
|
||
|
||
- Desktop control channel: `gridcraft --control 7979`, JSON lines on localhost
|
||
(`engine.execute`, `ui.screenshot`, pointer and keyboard input…), see
|
||
[`control-protocol.md`](control-protocol.md).
|
||
- MCP: `gridcraft-cli mcp`, headless or `--connect` to a running app, see [`mcp.md`](mcp.md).
|
||
- CLI: [`cli.md`](cli.md).
|
||
|
||
## Platforms and builds
|
||
|
||
macOS universal, Windows x64/x86/arm64, Linux x86_64/aarch64/riscv64 (AppImage, Flatpak, deb,
|
||
rpm, tarball), FreeBSD, and web (WASM; WebGPU or WebGL2). Release workflows in
|
||
`.github/workflows/release.yml`; see [`releasing.md`](releasing.md).
|
||
|
||
## Known architectural gaps
|
||
|
||
- No threading in calc (see [hardware-parity.md](hardware-parity.md)).
|
||
- No string catalog or text shaping (see [localization-parity.md](localization-parity.md)).
|
||
- XLSX reader builds the whole workbook in memory; no streaming (#175).
|
||
- Unmodelled XLSX parts are not carried through a save.
|
||
|
||
## Revision history
|
||
|
||
| Date | Change | Summary |
|
||
|---|---|---|
|
||
| 2026-10-11 | minor | Engine building blocks for multi-command actions: undo groups, recording buffer, document trust, chunked hidden-name storage |
|
||
| 2026-10-10 | major | Created from the code on main: crates, layers, data flow, agent control, known gaps |
|