Commit Graph
1975 Commits
Author SHA1 Message Date
prxnvt fcbbfa5142 XLSX: Python cells as _xlfn._xlws.PY() formulas, code in the Python part, cached values; opening limits
A Python cell is written to xlsx as [MS-XLSX] 2.3.10 specifies: its
formula is _xlfn._xlws.PY(i,0|1), where i is the zero-based index of the
cell's script in the workbook's Python part and 0|1 the return type, 0
for an Excel value and 1 for a Python object. The part (2.1.23) is
xl/python.xml, content type application/vnd.ms-excel.python+xml,
related from the workbook part with
http://schemas.microsoft.com/office/2023/09/relationships/Python; its
root python holds pythonScripts, one pythonScript per Python cell in the
order the cells are written, with the code in code. The code keeps its
xl() references as text, so the formula has no further arguments. The
cell keeps its cached value. A spilling result is written like a dynamic
array (cm="1", t="array" ref=...) with the spilled values cached, so
other applications show the values (and #NAME? if they recalculate).

The reader turns a PY( formula with or without the _xlfn. and _xlws.
prefixes into a Python cell. An index names a script of the Python part,
and each %Px% in the script becomes the text of the formula's argument x
(2.3.10.4). The string-literal form, PY("chunk"&"chunk",0|1), still
reads, from the formula's own text so line breaks in the code survive. A
formula naming a script the part doesn't hold keeps only its cached
value, with a warning: no calculation can evaluate it. The reader keeps
the cached spilled values as the cell's result, so a workbook from Excel
shows its arrays without running anything. Python errors are cached as
#VALUE!, as #SPILL! and #CALC! already are. Native JSON keeps Cell.py
whole: code, return type, the last result and the exact error.

Calc lays out a Python cell's kept result: Python cells are nodes of the
dependency graph that read nothing, and a pass takes a dirty Python
cell's result instead of evaluating it, so its array spills, is blocked
(#SPILL!) and unblocked, and feeds its dependents exactly like a
formula's. An array that is malformed or has more than 1,000,000 cells
(only a crafted file holds one) shows #SPILL!. A dirty Python cell is
pending in the pass, as a dirty formula is, so a formula, a range read
or a cell-by-cell range walk that reaches it before its turn lays it out
first rather than reading its old value.

Opening a file can't make the app loop over a claimed size: the reader
keeps a Python cell's spill range only when it has at most 1,000,000
cells (a range claiming the whole sheet is dropped with its values, as a
formula's spilled values are), and io::adopt_python rebuilds results for
at most 4,000,000 cells over the whole workbook, with a warning for the
cells it skips. The Python part is read up to 64 MB and 1,000,000
scripts, a script is kept up to one character past what a cell may
hold, and the cells take at most 64 MB of code from the part in all,
since several formulas can name one script. io::enforce_python_limits
drops the code of a cell whose code doesn't fit in 8,192 characters as
PY(...) (the cell keeps its cached value, with a warning) and a kept
result that isn't a well-formed array within the cap, each with its
spill range and spilled values, so the sheet and a save no longer show
the old spill. Formats that keep PY() as an ordinary formula (xls, xlsb,
ods) are adopted the same way. Opening a file runs nothing.

The command-sequence fuzzer sets Python cells (code with xl()
references, quotes, line breaks, and code at and over the size limit),
and its xlsx round-trip check allows the Python errors becoming
#VALUE!, which is what the file format stores.

Tests: the written formula naming its script, the Python part with its
content type and relationship, return types, a spilling table saved as a
dynamic array, Python errors as #VALUE!, the round trip of values, code,
long code with quotes and line breaks and the error, and no Python part
without Python cells; a workbook laid out as the format specifies (an
index, a parameter named as %P2%) read as Python cells with their
spilled values and saved back with its code in a Python part; the
string-literal form; a formula naming a missing script, with and without
a Python part; 20,000 cells naming one long script against the code
budget; the part's reader and writer (escaping, odd %P tokens, a long
script, a repeated parameter); a blocker in the spill area giving #SPILL!
and clearing it spilling again; the native JSON round trip; a spill
range claiming the whole sheet; the workbook-wide budget; hostile JSON
(code over the limit, a result claiming far more cells than it holds); a
malformed result that never spills; PY() formulas at 8,079 and 8,080
characters of code. Calc: formulas, MAX, SUM, COUNTA and SUMPRODUCT read
a dirty Python cell whose array is blocked before its turn, in a full
and an incremental recalculation, and see the array once it is
unblocked; enforce_python_limits removes the spill of dropped code and
of a dropped result, and keeps a sound one.
2026-10-26 10:18:38 -04:00
prxnvt c6c54c57e6 Engine: python.set makes a Python cell; this build stores code and
doesn't run it

python.set {cell?, code, output?: object|value} makes the cell (the
active cell by default) a Python cell, keeping its style and taking a
formula and its spill away. `output` is the return type Microsoft
documents for PY(): an Excel value (the default) or a Python object;
without it a Python cell keeps the one it had. The code must fit in
Excel's 8,192-character formula once written as PY(...), quotes doubled
and literals joined, so the command checks the encoded formula rather
than a character count. A table's header row refuses Python (it holds
the column's name, as Excel keeps formulas out of headers), and a locked
cell on a protected sheet refuses it through the usual protection check.

Nothing runs Python here. Unchanged code keeps its cached result and
value; new or changed code shows #BLOCKED!. One undo step takes the cell
back.

Sheet::clear_spill removes the spilled values inside a range with work
bounded by the values stored, not by the range's size.

In the interface, F2, a double click or the formula bar on a Python cell
edits its code, and a changed text is committed through python.set,
which keeps the return type; through cell.set it would have replaced the
Python cell with an ordinary cell holding the code as text. Typing over
a Python cell replaces it, as typing over a formula does, and committing
the code unchanged writes nothing. Deleting all the code clears the
cell through cell.set, as an empty entry clears any other cell; code
of spaces is still code.

Tests: the code and return type in cell.get, #BLOCKED! for new code,
cached results kept for unchanged code, the active-cell default, undo,
refused params (missing or non-text code, a bad cell, a bad return
type), the encoded limit at 8,079 and 8,080 characters and with doubled
quotes, a table header, a protected sheet, and a formula's spill taken
away. A UI test edits a Python cell's code with F2 and Enter, commits it
unchanged, and types over it; another clears a Python cell by deleting
all its code, keeps code of spaces, and undoes the clearing.
2026-10-26 10:18:38 -04:00
prxnvt 44c663aea2 Model: Python cells, and the #PYTHON!, #BUSY!, #TIMEOUT! and #BLOCKED! errors
Cell gains `py: Option<Arc<PyCode>>`, read with a serde default and not
written for ordinary cells, so older native files load and ordinary
cells serialize as before. PyCode holds the code, the return type (an
Excel value or a Python object), the last result (value, type name,
preview, whether it converted to cells) and a stale flag for manual
calculation. The cell's `value` stays the cached result, so formulas,
display and saving read a Python cell like any other. is_blank,
CellStore::has and input_text (which returns the code) take the code
into account; Cell::has_content and clear_content cover value, formula
and code together.

The model reads and writes the PY function's syntax as Microsoft
documents it, PY("chunk"&"chunk",0|1), with string literals of at most
255 characters and the return type 0 for an Excel value, 1 for a Python
object. The reader takes the `_xlfn._xlws.` prefix (and `_xlfn.` or
`_xlws.` alone, in any letter case); to_file_formula writes it. (An xlsx
file keeps the code in its Python part instead; the xlsx commit adds
it.) The size limit is Excel's 8,192 characters for that formula, not a
count of code characters: doubled quotes and the "&" between literals
count, so code with many quotes fits in fewer characters (8,079
characters of plain code fit exactly). The model also finds the
xl("...") literals in code, skipping comments and other strings; a plain
literal may escape a quote or a backslash. A rewritten reference that
holds the literal's quote (a sheet name that needs quotes, 'Q4
2026'!B2) takes the other quote as the literal's delimiter, or an
escaped quote when it holds both, so it never ends the string early.

Edits: cell.set over a Python cell, Clear Contents, Clear All and the
cells a merge empties remove the code; Clear Formats keeps it. Copy and
paste (All, Formulas), Fill, sort, cut-move, sheet copy and undo carry
it; a cut to another sheet first qualifies its xl() references with the
source sheet, as a moved formula's references are, so they keep
reading the same cells. Paste Values and Paste Link do not carry it,
and paste operations do not combine numbers with a Python cell. Fill
copies Python cells instead of extending them as a series. cell.set
with =PY(...) is refused with a message that points to python.set, and
Text to Columns keeps a piece that reads as =PY(...) as text instead of
clearing it. Goal Seek refuses a Python cell as its changing cell, and
Solver as a variable cell, as both refuse formulas: the number written
there would replace the code. The sheet-protection check compares the
code, so a locked Python cell cannot be changed. cell.get reports
{python: code, output, typeName, preview, stale}.

The xl() literals follow reference-moving edits: rewrite_all_formulas
(row and column insert and delete, cell insert and delete, cut-move),
sheet rename and sheet delete adjust each literal like a formula
reference, and replace it only when it changed.

Errors: CellError gains Python, Busy, Timeout and Blocked, and ALL has
14 entries, so the lexer reads them as literals. ERROR.TYPE gives 11 for
#BLOCKED!, the number Excel returns. Microsoft documents no ERROR.TYPE
number for #PYTHON!, #BUSY! or #TIMEOUT!; they take 19, 20 and 21, after
the numbers Excel is known to use. The xlsx writer caches the four as
#VALUE!, as it already does #SPILL! and #CALC!, because they are not
error codes of the file format.

Tests cover the =PY refusal, removal by entry and clearing, undo, copy,
paste values, fill and sort, xl() rewrites through row insert, sheet
rename (also to names that need quotes, with one quote and with both),
cut-move (also to another sheet) and column delete, Goal Seek and
Solver refusing Python cells, Text to Columns keeping a =PY piece,
native JSON round trips, the new errors, the prefixes, escaped quotes in
literals, and the encoded size limit at 8,192 characters exactly and
with doubled quotes.
2026-10-26 10:18:38 -04:00
aiongg f6b0cec67e Merge pull request #2619 from aiongg/g4
Window per workbook (4/4): two windows on one workbook without copying it; Arrange All
2026-10-26 00:00:24 -04:00
aiongg 862dfc1920 Merge pull request #2618 from aiongg/g3
Window per workbook (3/4): a native window per workbook window
2026-10-25 23:59:45 -04:00
aiongg 1b63c3f335 Merge pull request #2617 from aiongg/g2
Window per workbook (2/4): windows in the engine; New Window shares the workbook
2026-10-25 23:58:35 -04:00
aiongg ecb0e546d8 Merge pull request #2616 from aiongg/g1
Window per workbook (1/4): a cell draft stays in its workbook; Switch Windows
2026-10-25 23:57:56 -04:00
aiongg 1f2da4048d Merge g3 into g4 2026-10-26 03:36:29 +00:00
aiongg dbec5c5800 Merge g2 into g3 2026-10-26 03:36:18 +00:00
aiongg 57da3d4641 Merge g1 into g2 2026-10-26 03:35:47 +00:00
aiongg 380be36ac3 G1: the W, W key tip opens Switch Windows through its control 2026-10-26 03:33:34 +00:00
aiongg 5c250a7fea Merge g3 into g4 2026-10-26 03:29:08 +00:00
aiongg c7911c836a G3: craft-libs at storytold main b926b5c0 2026-10-26 03:28:46 +00:00
aiongg 0f5fc81a6c Merge g2 into g3 2026-10-26 03:27:27 +00:00
aiongg 0316765204 Merge g1 into g2 2026-10-26 03:16:42 +00:00
aiongg 320bcfcf54 G1: regenerate the key tip tests 2026-10-26 03:16:42 +00:00
aiongg 7fdceaddc6 Merge g3 into g4 2026-10-26 02:57:21 +00:00
aiongg ec56e80f7f Merge g2 into g3 2026-10-26 02:57:21 +00:00
aiongg b29f05a2c7 Merge g1 into g2 2026-10-26 02:57:21 +00:00
aiongg 5900247075 G1: Switch Windows draws its check with the chrome fallback when no face has ✓ 2026-10-26 02:56:59 +00:00
aiongg f102fe29a1 Merge g3 into g4
# Conflicts:
#	ROADMAP.md
#	crates/engine/src/cmd/edit.rs
#	crates/engine/src/cmd/formulas.rs
#	crates/engine/src/cmd/inspect.rs
#	crates/engine/src/cmd/pivot.rs
#	crates/ui-egui/src/lib.rs
#	crates/ui-egui/src/tabs.rs
#	docs/gaps.md
#	docs/ui-parity.md
2026-10-26 02:45:41 +00:00
aiongg 7085aa1c47 G3: build on current main
Screenshots are admitted in logic for the root viewport and issued from its pass, so nothing
reads the focused native window any more; the field goes.
2026-10-26 02:45:24 +00:00
aiongg 94fcbf8b4c G3: craft-libs at e8b4dde5 (craft-libs#340)
craft-update, craft-window, craft-ui and craft-print all come from one craft-libs revision, the
head of storytold/craft-libs#340 on the aiongg fork, until #340 merges.
2026-10-26 02:37:42 +00:00
aiongg 015b9abe5b Merge g2 into g3
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	ROADMAP.md
#	apps/gridcraft/src/main.rs
#	crates/ui-egui/src/lib.rs
#	docs/ui-parity.md
2026-10-26 02:37:36 +00:00
aiongg 929495e274 Merge g1 into g2
# Conflicts:
#	ROADMAP.md
#	crates/engine/src/cmd/extra.rs
#	crates/engine/src/lib.rs
#	crates/engine/src/tests.rs
#	docs/control-protocol.md
#	docs/ui-parity.md
2026-10-26 02:37:33 +00:00
aiongg 22aec7e667 G1: build on current main
Scripts can't run the View tab's window commands (Switch Windows, New Window, Hide, Unhide):
they change the active workbook, as window.* steps do. The Switch Windows button names its
accessibility role with egui::Role (egui 0.37).
2026-10-26 02:37:27 +00:00
aiongg 903bece86f Merge upstream main into g1
# Conflicts:
#	ROADMAP.md
#	crates/engine/src/tests.rs
#	crates/ui-egui/src/lib.rs
#	docs/ui-parity.md
2026-10-26 02:23:11 +00:00
Dylan M. Taylor 42bbf6baa2 refactor: split complex functions for Sonar S3776 (#2612)
Extract group_bracket from outline layout and move_marks from data
outline_settings. No behaviour change.
2026-10-25 21:39:17 -04:00
Dylan M. Taylor 52cf825395 refactor: split complex functions for Sonar S3776 (#2609)
Extract helpers from auto_sum (selection and single-cell guess), update_cursor
(drag, hover, freeze, and header-edge cursors) and read_styles (dxf reader).
No behaviour change.
2026-10-25 21:37:09 -04:00
Dylan M. Taylor 6f0e43c21c Linux: default to XWayland for file drag-and-drop on Wayland desktops (#165)
* Linux: default to XWayland for file drag-and-drop on Wayland desktops

* Ignore an empty DISPLAY before forcing X11
2026-10-25 19:37:08 -04:00
Dylan M. Taylor 0870b1ad1c Merge pull request #2595 from storytold/chore/deps-safe-refresh
chore(deps): safe dependency refresh
2026-10-25 19:00:54 -04:00
skylar 14dd23db6c chore(deps): refresh serde_json, arboard, muda, winresource, sha2 2026-10-25 16:36:12 -06:00
Dylan M. Taylor ad10881cc3 build: pin craft-update to craft-libs main (495e597) (#2591)
Picks up the startup prompt without the glyphs a font may lack (storytold/craft-libs#343, #344): the close mark is painted and the version pill is ASCII, so apps that panic on a missing glyph no longer take the prompt down.

Part of tracking issue storytold/craft-libs#319
2026-10-25 17:50:44 -04:00
Dylan M. TaylorandCodebuff ee7cbf32ab build: pin craft-update to craft-libs main (495e597) (#2586)
Picks up the update prompt fix that draws its close mark and version pill
without glyphs a font may lack, so the prompt no longer takes the app down
when a bundled face has no ✕ or →.

Part of tracking issue storytold/craft-libs#319

🤖 Generated with Codebuff

Co-authored-by: Codebuff <noreply@codebuff.com>
2026-10-25 17:37:15 -04:00
Dylan M. Taylor 2fbbed33bc Merge pull request #783 from vangel76/fix/control-auth-777
Security: the --control server requires an auth token and closes bad connections (#777)
2026-10-25 16:45:46 -04:00
Dylan M. Taylor 1a36d9db07 feat: startup update prompt modal and direct self-updating (#2566)
* feat: startup update prompt modal and direct self-updating

The update flow (check at start or on request, the startup prompt, the self-update worker) is craft-update's Updater; the app only supplies the suite, its close path and its status message. An install holds back while the workbook is unsaved.

Ref: storytold/craft-libs#319

* build: refresh the lock

* build: pin craft-update to craft-libs main (db19c37)

* fix(updates): address review feedback

Wake the UI when a manual check is queued from a menu callback; the update prompt owns the keyboard while it is open (LightCraft); Download opens through the desktop URL hook (FilmCraft); the Help menu dispatches through menus::invoke (FigCraft); print jobs and comment drafts hold an install back (WordCraft); Check for Updates is a registered command (GridCraft); an install holds back while documents are unsaved.

* fix(updates): add Check for Updates… to the native macOS Help menu
2026-10-25 15:37:52 -04:00
skylar taylor-barrick 41769f1d16 Migrate GridCraft to egui 0.37 (#2543)
* E37: migrate GridCraft to egui 0.37

Pin the reviewed fork and shared dependencies, adapt input and rendering to the
new frame lifecycle, and preserve bounded capture handling and font notices.
Add representative accessibility checks and meaningful control labels.

Validation: full repository CI passed all ten stages, with 2,249 tests passed,
zero failed and one existing timing test ignored. Native workflows and paired
performance qualification remain documented separately.

* egui037: record final published pins and local qualification

* M0.37: fix RTL clipping and preserve screenshot budgets

* test: share migrated GridCraft UI harness setup

* egui037: document migrated UI and regression contracts

* egui-037: align popup test fonts and avoid redundant tooltip probes
2026-10-25 14:55:00 -04:00
3b65b9af58 Sonar maintainability: explicit imports, method paths, smaller functions (#2412)
* Sonar: explicit imports, fn paths for closures, extract helpers for cognitive complexity

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* Fix imports

* Fix serde imports

* Fix engine imports

* Fix clippy gate and a type error from the explicit-import change

Remove the unused imports the review flagged (cf.rs, pivot.rs, tables.rs, cmd/sheet.rs) so clippy -D warnings passes, and point the backstage parent-folder lookup at Path::to_str, which the fn-path rewrite had broken.

Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Apply rustfmt to the explicit-import cleanup so the fmt gate passes

* Share the per-directory font path expansion in theme.rs

The candidate font lists repeated the same loop over the Linux font
directories in every role. A single helper expands them in the same order,
so the lookup order is unchanged and the duplicated blocks go away.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Read the Windows fonts folder in one place in theme.rs

Every script-font list repeated the same WINDIR lookup. A single helper
returns the same path, so the duplicated lines go away without changing
which font files are tried.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Build the light and dark UI palettes from one table in theme.rs

Tokens::light() and Tokens::dark() listed the same 40 colour fields in the
same order, so every change had to be made twice. A single palette
constructor takes the mode and picks each colour's value, so each field is
written once and both modes keep their exact colours.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Share the Noto CJK font list between the CJK script lists in theme.rs

The Japanese, Simplified and Korean lists each repeated the same loop over
the Linux CJK directories with a different locale file name. One helper
builds that loop, and each list keeps its own face index, so the lookup
order is unchanged.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>

* Share Linux Noto directory list across Thai and Hebrew font lookups

Removes the duplicated directory arrays flagged by SonarCloud new-code duplication.

* Reuse in_dirs for Thai and Hebrew Noto font candidates

Removes the remaining duplicated per-directory loops flagged by SonarCloud.

* Share one font-list builder for the Thai and Hebrew faces

The two per-script functions repeated the same Windows, macOS, Linux and
variable-font setup. A single script_fonts helper now builds both lists,
keeping the candidate order unchanged, which clears the Sonar duplication gate.

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
2026-10-25 12:32:02 -04:00
Brandon ThomasandClaude Opus 5.5 4eb2fb5c8e Review fix for #2473: a repeated sheet name matches in any case
Sheet1!A1:SHEET1!B2 is a range whose second end repeats the sheet, as
Sheet1!A1:Sheet1!B2 is: the check uses same_name, Excel's case-insensitive
name comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 09:56:18 -04:00
Brandon Thomas 0665476b35 Merge pull request #2493 from Harrison-Booth:feat/options-iteration-fields
# Conflicts:
#	docs/function-parity.md
#	docs/ui-parity.md
2026-10-25 09:36:59 -04:00
Brandon Thomas 189c3cdb0f Merge pull request #2497 from marcoarnulfo:fix/full-column-extend-scroll 2026-10-25 09:36:38 -04:00
Brandon Thomas 674e1c05a3 Merge pull request #2481 from marcoarnulfo:fix/trace-arrows-stay-on-traced-cell 2026-10-25 09:36:37 -04:00
Brandon Thomas 0469b51c93 Merge pull request #2444 from OdooPudakMachinery:web-chrome-flag 2026-10-25 09:36:37 -04:00
Brandon Thomas 4a44dec500 Merge pull request #2515 from Fubeo:fub/edit-note-keeps-thread 2026-10-25 09:36:36 -04:00
Brandon Thomas 0f5227ddde Merge pull request #2513 from Fubeo:fub/keep-sheet-id 2026-10-25 09:36:35 -04:00
Brandon Thomas 97ec7653aa Merge pull request #2512 from Fubeo:fub/open-saved-without-recalc 2026-10-25 09:36:34 -04:00
Brandon Thomas 27e96a9a58 Merge pull request #2561 from arcanii:autosum-across-a-row 2026-10-25 09:36:34 -04:00
Brandon Thomas 2c81adf94f Merge pull request #2557 from Silvertree2010:fix/2517-a-new-font-derived-from-the-them 2026-10-25 09:36:33 -04:00
Brandon Thomas 9413c57e2c Merge pull request #2555 from Silvertree2010:fix/2407-xlsx-only-one-warning-when-sever 2026-10-25 09:36:32 -04:00
Brandon Thomas bd7b867a1f Merge pull request #2552 from Silvertree2010:fix/h2457-cli-eval-cat-print-huge-or-tiny 2026-10-25 09:36:31 -04:00