When a file is opened, a Python cell falls back to keeping only its value in
four cases: its code is missing from the Python part or the part is damaged
(the xlsx reader), its code is too long for a cell, its kept result is
malformed, or its result is past the adoption budget. The first kept the
values the cell spilled as plain values; the others dropped them, so values
the file showed were lost. All of them now follow one rule, through
Sheet::settle_spill: the anchor and every spilled cell keep their cached
values as plain values, with no live spill range; a cell's own content and
format are never overwritten; and the work is bounded by the spilled values
stored, not by the range. The values move from the spill into the cells, so
nothing is copied. The cut-by-budget test is renamed for what it now checks,
the dropped-result test now expects the values kept, and the adoption warning
and the file-format docs say so.
When a file's cached Python spills were larger than the adoption budget, the
cells past it kept only their own value, but their spill ranges and spilled
values stayed on the sheet, which went on showing values no result held. The
spill of such a cell is now cleared, as for a dropped result; the warning is
unchanged.
A formula calling a Google-only function (GOOGLETRANSLATE, QUERY,
ARRAYFORMULA...) opened as #NAME? where the wrapper had shown the cached
value, as Excel does. The XLSX reader now unwraps a cell formula only when
every function inside is known: read_xlsx_with takes the predicate (the
xlsx crate can't depend on the function library) and the engine passes
gridcraft_calc::is_known_function. Unwrapping stays in the reader so shared
formulas are shifted from the unwrapped master.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Python cell is written to xlsx as [MS-XLSX] 2.3.10 specifies: its
formula is _xlfn._xlws.PY(i,0|1), where i is the zero-based index of the
cell's script in the workbook's Python part and 0|1 the return type, 0
for an Excel value and 1 for a Python object. The part (2.1.23) is
xl/python.xml, content type application/vnd.ms-excel.python+xml,
related from the workbook part with
http://schemas.microsoft.com/office/2023/09/relationships/Python; its
root python holds pythonScripts, one pythonScript per Python cell in the
order the cells are written, with the code in code. The code keeps its
xl() references as text, so the formula has no further arguments. The
cell keeps its cached value. A spilling result is written like a dynamic
array (cm="1", t="array" ref=...) with the spilled values cached, so
other applications show the values (and #NAME? if they recalculate).
The reader turns a PY( formula with or without the _xlfn. and _xlws.
prefixes into a Python cell. An index names a script of the Python part,
and each %Px% in the script becomes the text of the formula's argument x
(2.3.10.4). The string-literal form, PY("chunk"&"chunk",0|1), still
reads, from the formula's own text so line breaks in the code survive. A
formula naming a script the part doesn't hold keeps only its cached
value, with a warning: no calculation can evaluate it. The reader keeps
the cached spilled values as the cell's result, so a workbook from Excel
shows its arrays without running anything. Python errors are cached as
#VALUE!, as #SPILL! and #CALC! already are. Native JSON keeps Cell.py
whole: code, return type, the last result and the exact error.
Calc lays out a Python cell's kept result: Python cells are nodes of the
dependency graph that read nothing, and a pass takes a dirty Python
cell's result instead of evaluating it, so its array spills, is blocked
(#SPILL!) and unblocked, and feeds its dependents exactly like a
formula's. An array that is malformed or has more than 1,000,000 cells
(only a crafted file holds one) shows #SPILL!. A dirty Python cell is
pending in the pass, as a dirty formula is, so a formula, a range read
or a cell-by-cell range walk that reaches it before its turn lays it out
first rather than reading its old value.
Opening a file can't make the app loop over a claimed size: the reader
keeps a Python cell's spill range only when it has at most 1,000,000
cells (a range claiming the whole sheet is dropped with its values, as a
formula's spilled values are), and io::adopt_python rebuilds results for
at most 4,000,000 cells over the whole workbook, with a warning for the
cells it skips. The Python part is read up to 64 MB and 1,000,000
scripts, a script is kept up to one character past what a cell may
hold, and the cells take at most 64 MB of code from the part in all,
since several formulas can name one script. io::enforce_python_limits
drops the code of a cell whose code doesn't fit in 8,192 characters as
PY(...) (the cell keeps its cached value, with a warning) and a kept
result that isn't a well-formed array within the cap, each with its
spill range and spilled values, so the sheet and a save no longer show
the old spill. Formats that keep PY() as an ordinary formula (xls, xlsb,
ods) are adopted the same way. Opening a file runs nothing.
The command-sequence fuzzer sets Python cells (code with xl()
references, quotes, line breaks, and code at and over the size limit),
and its xlsx round-trip check allows the Python errors becoming
#VALUE!, which is what the file format stores.
Tests: the written formula naming its script, the Python part with its
content type and relationship, return types, a spilling table saved as a
dynamic array, Python errors as #VALUE!, the round trip of values, code,
long code with quotes and line breaks and the error, and no Python part
without Python cells; a workbook laid out as the format specifies (an
index, a parameter named as %P2%) read as Python cells with their
spilled values and saved back with its code in a Python part; the
string-literal form; a formula naming a missing script, with and without
a Python part; 20,000 cells naming one long script against the code
budget; the part's reader and writer (escaping, odd %P tokens, a long
script, a repeated parameter); a blocker in the spill area giving #SPILL!
and clearing it spilling again; the native JSON round trip; a spill
range claiming the whole sheet; the workbook-wide budget; hostile JSON
(code over the limit, a result claiming far more cells than it holds); a
malformed result that never spills; PY() formulas at 8,079 and 8,080
characters of code. Calc: formulas, MAX, SUM, COUNTA and SUMPRODUCT read
a dirty Python cell whose array is blocked before its turn, in a full
and an incremental recalculation, and see the array once it is
unblocked; enforce_python_limits removes the spill of dropped code and
of a dropped result, and keeps a sound one.
Cell gains `py: Option<Arc<PyCode>>`, read with a serde default and not
written for ordinary cells, so older native files load and ordinary
cells serialize as before. PyCode holds the code, the return type (an
Excel value or a Python object), the last result (value, type name,
preview, whether it converted to cells) and a stale flag for manual
calculation. The cell's `value` stays the cached result, so formulas,
display and saving read a Python cell like any other. is_blank,
CellStore::has and input_text (which returns the code) take the code
into account; Cell::has_content and clear_content cover value, formula
and code together.
The model reads and writes the PY function's syntax as Microsoft
documents it, PY("chunk"&"chunk",0|1), with string literals of at most
255 characters and the return type 0 for an Excel value, 1 for a Python
object. The reader takes the `_xlfn._xlws.` prefix (and `_xlfn.` or
`_xlws.` alone, in any letter case); to_file_formula writes it. (An xlsx
file keeps the code in its Python part instead; the xlsx commit adds
it.) The size limit is Excel's 8,192 characters for that formula, not a
count of code characters: doubled quotes and the "&" between literals
count, so code with many quotes fits in fewer characters (8,079
characters of plain code fit exactly). The model also finds the
xl("...") literals in code, skipping comments and other strings; a plain
literal may escape a quote or a backslash. A rewritten reference that
holds the literal's quote (a sheet name that needs quotes, 'Q4
2026'!B2) takes the other quote as the literal's delimiter, or an
escaped quote when it holds both, so it never ends the string early.
Edits: cell.set over a Python cell, Clear Contents, Clear All and the
cells a merge empties remove the code; Clear Formats keeps it. Copy and
paste (All, Formulas), Fill, sort, cut-move, sheet copy and undo carry
it; a cut to another sheet first qualifies its xl() references with the
source sheet, as a moved formula's references are, so they keep
reading the same cells. Paste Values and Paste Link do not carry it,
and paste operations do not combine numbers with a Python cell. Fill
copies Python cells instead of extending them as a series. cell.set
with =PY(...) is refused with a message that points to python.set, and
Text to Columns keeps a piece that reads as =PY(...) as text instead of
clearing it. Goal Seek refuses a Python cell as its changing cell, and
Solver as a variable cell, as both refuse formulas: the number written
there would replace the code. The sheet-protection check compares the
code, so a locked Python cell cannot be changed. cell.get reports
{python: code, output, typeName, preview, stale}.
The xl() literals follow reference-moving edits: rewrite_all_formulas
(row and column insert and delete, cell insert and delete, cut-move),
sheet rename and sheet delete adjust each literal like a formula
reference, and replace it only when it changed.
Errors: CellError gains Python, Busy, Timeout and Blocked, and ALL has
14 entries, so the lexer reads them as literals. ERROR.TYPE gives 11 for
#BLOCKED!, the number Excel returns. Microsoft documents no ERROR.TYPE
number for #PYTHON!, #BUSY! or #TIMEOUT!; they take 19, 20 and 21, after
the numbers Excel is known to use. The xlsx writer caches the four as
#VALUE!, as it already does #SPILL! and #CALC!, because they are not
error codes of the file format.
Tests cover the =PY refusal, removal by entry and clearing, undo, copy,
paste values, fill and sort, xl() rewrites through row insert, sheet
rename (also to names that need quotes, with one quote and with both),
cut-move (also to another sheet) and column delete, Goal Seek and
Solver refusing Python cells, Text to Columns keeping a =PY piece,
native JSON round trips, the new errors, the prefixes, escaped quotes in
literals, and the encoded size limit at 8,192 characters exactly and
with doubled quotes.
* Sonar: explicit imports, fn paths for closures, extract helpers for cognitive complexity
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Fix imports
* Fix serde imports
* Fix engine imports
* Fix clippy gate and a type error from the explicit-import change
Remove the unused imports the review flagged (cf.rs, pivot.rs, tables.rs, cmd/sheet.rs) so clippy -D warnings passes, and point the backstage parent-folder lookup at Path::to_str, which the fn-path rewrite had broken.
Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Apply rustfmt to the explicit-import cleanup so the fmt gate passes
* Share the per-directory font path expansion in theme.rs
The candidate font lists repeated the same loop over the Linux font
directories in every role. A single helper expands them in the same order,
so the lookup order is unchanged and the duplicated blocks go away.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Read the Windows fonts folder in one place in theme.rs
Every script-font list repeated the same WINDIR lookup. A single helper
returns the same path, so the duplicated lines go away without changing
which font files are tried.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Build the light and dark UI palettes from one table in theme.rs
Tokens::light() and Tokens::dark() listed the same 40 colour fields in the
same order, so every change had to be made twice. A single palette
constructor takes the mode and picks each colour's value, so each field is
written once and both modes keep their exact colours.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Share the Noto CJK font list between the CJK script lists in theme.rs
The Japanese, Simplified and Korean lists each repeated the same loop over
the Linux CJK directories with a different locale file name. One helper
builds that loop, and each list keeps its own face index, so the lookup
order is unchanged.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Share Linux Noto directory list across Thai and Hebrew font lookups
Removes the duplicated directory arrays flagged by SonarCloud new-code duplication.
* Reuse in_dirs for Thai and Hebrew Noto font candidates
Removes the remaining duplicated per-directory loops flagged by SonarCloud.
* Share one font-list builder for the Thai and Hebrew faces
The two per-script functions repeated the same Windows, macOS, Linux and
variable-font setup. A single script_fonts helper now builds both lists,
keeping the candidate order unchanged, which clears the Sonar duplication gate.
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
- A run that rolls back after writing a file no longer says "Nothing was changed.": the message
says the workbook was put back and names the step whose file stays (`StepError::wrote_files`,
also after a Stop with rollback).
- `macro.run` refuses an input the macro doesn't have (a misspelt input bound the selection),
an input that isn't reference text, and `onConflict`, `overwrite`, `approve` or `timeLimit` of
the wrong kind, which it used to ignore; `null` still counts as not given.
- A step no longer copies itself, and Repeat and If no longer copy their bodies, on every pass:
a body is copied once per run. A run passes its commands at most 256 MB of parameters, all
steps together, which bounds a run's work on the web build, where there is no clock.
- Every step except an approved file step runs inside the scripts' file sandbox, so a command
that touches a file under a parameter other than `path` can't do so from a macro.
- Docs: moving cells that hold data counts as writing over them.
- Tests: `run_parameters_that_would_be_ignored_are_refused`,
`a_run_passes_a_bounded_amount_of_parameters`, and the rollback message in
`each_step_is_checked_for_writes_over_data`.
A macro whose format version is above this build's was dropped on open,
so the next save removed it from the file, and the warning said only
that it couldn't be opened. It is now kept as the file held it
(Workbook::macros_unread, shared between snapshots), never read, listed
or run, and written back unchanged by JSON and xlsx saves, numbered
after the readable macros and around visible names of the storage
shape. Kept macros count toward the 1,000-macro limit after the
readable ones; one too large for an xlsx item is left out. The warning
says it is kept and saved back; damaged and over-limit macros keep
their own warning.
The JSON reader replaces a workbook with an empty sheet list by a new
one before macros::load reads the file's macro entries, so they were
lost with no warning. The entries are carried across the reset. The
reset itself, which also starts the names and styles afresh, is
unchanged.
save_fails_upfront_when_the_temp_name_is_taken,
save_keeps_the_destination_access_restrictions and
open_refuses_directories_and_unknowable_lengths in crates/engine/src/io.rs
all worked in temp_dir()/gridcraft-fs-safety and removed it at the start
and the end. cargo test runs them on parallel threads, so one test could
delete the folder while another was writing to it, and the other test
failed. Each test now uses its own folder, named with a short label and
the process id, as the gridcraft-write and gridcraft-sandbox tests in the
same file do.
- read_file: verify the opened file is regular and cap the bytes actually
taken at the 2 GB limit — a workbook-named symlink to an endless device
reported length 0 but streamed forever (finding 22).
- write_file: create the temporary file exclusively, so a planted file or
symlink at the predictable `book.xlsx.sctmp` path makes the save fail
instead of truncating the link's target (finding 25).
- write_file: inherit the destination's permissions, so saving an
owner-only workbook doesn't become world-readable through the rename
(finding 15); new files note restrictive defaults.
Tests: exclusive temp creation vs planted symlink, permissions preserved,
directory refused on open. Engine: 460 tests pass; clippy clean.
- clip_html: cap wildcard/glob lookback to a 12-byte window instead of scanning
outgoing text arbitrarily far for a semicolon (finding 18).
- print: wrap_lines binary-searches its break instead of rescanning the line
from the start for every band (finding 21).
- autofit: per-row merge index instead of scanning every merge per row during
autofit (finding 27) — behaviour covered by the existing autofit merge test.
- Save as Web Page: the used rectangle bounds the HTML (finding 28, #2387) —
a tall/wide or sparse far-right sheet now errors instead of growing
gigabytes of <td></td>.
- print: decode each embedded picture once per document instead of once per
band that shows it (finding 30).
Engine: 459 tests pass; clippy clean.
- The workbook's active sheet is an atomic set in place (Workbook::active_sheet /
set_active_sheet), so a window swap or sheet switch never clones the
Arc<Workbook> or marks it dirty. Selections already live in DocState and the
window views. Save records the sheet of the most recently active window.
- A dialog taken out of its window while it runs goes back to that window when
a command run from it switched windows (dialogs::show, ui.dialog.confirm).
- view.arrangeAll {arrange?: tiled|horizontal|vertical|cascade, activeWorkbook?}
returns the visible windows, active first; the desktop moves and sizes the
native windows over the main window's monitor. New Window and Arrange All
buttons on the View ribbon; Arrange Windows dialog.
- Docs: ui-parity, hardware-parity, gaps #20, target-app-parity, ROADMAP.
- automate.trustScripts: only a missing `trusted` defaults to true; a
non-boolean value is an error and leaves the workbook untrusted.
- The refusal message points at Trust Scripts in View > Macros (Alt+F8)
and at automate.trustScripts.
- Catalog row for automate.trustScripts; parity checklist regenerated.
- Script steps run inside an io::ScriptSandbox (thread-local) so
read_file/write_file refuse whatever the parameter is called, and a
step that raises OpenUrl is dropped and stops the script. The id/param
deny list stays as the up-front refusal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A macro is plain data in the model (model/src/macros.rs): a name, a
description, an optional shortcut, declared inputs (shape, header,
column types, non-empty, default reference), steps, an output rule
(anchor and what to do on a conflict) and the commands it may run that
read or write files. A step runs a command (id and params) or is one of
three bounded control steps: Repeat over the rows or columns of a range,
If on one cell's value, and Stop with a message. Every step carries
enabled, breakpoint and note flags. These are GridCraft macros, not VBA;
a VBA project in a file stays preserved and is never run.
Workbook::macros holds them. The JSON format keeps the field. In xlsx,
io::save_bytes writes each macro as one chunked hidden-name item,
_gc_macro_<n>, and io::open_bytes takes the items back out of the names,
so the names never linger in the open document.
Limits are checked whenever a macro is read from a file or a command:
at most 1,000 macros, 10,000 steps each, Repeat and If nested 4 deep,
names and notes of bounded length, and at most 4,194,304 characters of
JSON, so that saving a workbook never fails because of a macro; nesting
is checked before the steps are deserialized. What is over or damaged
is dropped with one warning, and clashing names get a suffix.
Automate scripts keep their own storage (_sc_scripts) and are neither
read nor changed by this.
Save as Web Page wrote only the cell table, so pictures and charts on a sheet
were dropped. A sheet that has either now keeps its own geometry and draws them:
- The table gets the sheet's column widths and row heights (`table-layout:fixed`,
a `<colgroup>`, a height per row); hidden columns and rows take no room and
merged-cell spans count only visible lines.
- A picture is an `<img>` with a `data:` URI, a chart is inline SVG built from
the chart crate's drawing primitives (titles, axes, legends and data labels
included), each absolutely positioned at its anchor cell plus offset.
- Sheets without pictures or charts write byte-identical HTML.
Adds contributors/people.toml entry for bilalmughal.
It returned an empty string for every workbook, saved or not, so the classic
sheet-name formula `MID(CELL("filename",A1),FIND("]",…)+1,31)` gave #VALUE!:
with no `]` in the string there is nothing for FIND to find. Excel returns
`<folder>\[<file>]<sheet>`, and empty only before the first save.
The model had no file to report. `source_dir` could not serve: it names the
folder relative links to other workbooks are relative to, and must keep naming
it after a Save As into another folder, because the links held in memory are
still written for the old one — only the saved copy is rebased. `source_path`
is therefore its own field, with that distinction written down.
It is set when a workbook is opened, from the path made absolute so the folder
is the one the user sees, and when a full-fidelity save makes a file the
document's own — before the `saved` snapshot, or the workbook would read as
dirty the moment it was saved, and only when it changes, so an ordinary Save
does not clone the workbook for nothing. A workbook that arrived as bytes knows
its name but never a folder and reports `[name]Sheet`, which still splits on the
`]`; an import reports the file it came from, as Excel does. The sheet is the one
the reference is on, not the active one.
CELL is volatile and already in the volatile sets, so a cell holding it answers
from the next recalculation; after a Save As that means F9 rather than the save
itself, which the engine test asserts rather than papers over. Making a save
recalculate is a larger decision than this issue.
`check` still skips CELL("filename") as volatile, as docs/cli.md says: the
cached value in a file is the folder of the machine that saved it.
file.open takes {csvAsText: true}: a CSV, TSV or TXT file opens with every field as the text
it holds, with no number, date or boolean conversion, so identifiers such as GTINs, SKUs and
ZIP codes keep their digits and leading zeros and save back unchanged. The default is
unchanged. The CsvOptions.parse_values switch already existed in the reader.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
prxnvt