A script's code was cut at the character limit before its ST_Xstring escapes
were decoded, so a script whose escapes (`_x0009_`, `_x000D_`) made the file's
text longer than the limit lost its end without a warning, although its code
fits. The code is now decoded first and then cut one character past the limit,
so a script that is too long once decoded still fails the check and its cell
keeps only its value, as before. Decoding is linear and reads at most a bounded
number of encoded characters, enough for any code that fits.
A PY formula whose script the file doesn't hold (the Python part missing or
damaged, or the script over the code budget) keeps only its cached value, but
its array range stayed recorded, so the cached values over the rest of its
spill were dropped as a formula's spill would be, and nothing filled them
again. The range of such a cell is now forgotten when it is read: the anchor
and the cells of its spill keep their cached values as plain values, as the
file shows them, with no live spill. An ordinary formula's dynamic-array spill
is handled as before.
A Python part whose zip entry failed to read (a bad checksum or compressed
stream) made the whole workbook fail to open: the error was passed up, and the
failed read had also ended the package's read budget, so even a warning would
have left the sheets unread. The part is now read with
Package::read_optional_limited, which charges a failed read only that part's
allowance, and every error other than TooLarge is a warning. The Python cells
then keep only their cached values, as when the part is missing. Malformed XML
in the part was already a warning; the test covers it too.
Since #2425 checks Agile encryption's dataIntegrity, every workbook Excel
encrypts fails with "the decrypted data failed its integrity check". The
reader keyed the HMAC with the decrypted encryptedHmacKey cut to the salt
size (16 bytes), as [MS-OFFCRYPTO] 2.3.4.14 describes it, but Excel's key
is as long as the hash: 64 bytes for SHA-512. The check now accepts a key
of either length, so spec-literal writers still open and a modified
package still fails.
The test encryptor writes the integrity key as Excel does, so every
encrypted fixture now has Excel's layout; a new test covers both key
lengths, a key of neither length, and tampering with each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saving a large workbook spent about half its time building worksheet XML,
mostly in small per-cell allocations. The writer now appends to the part
directly:
- Cell references, row numbers, style ids, shared string indexes and
numbers are appended without temporary strings (push_a1, push_u64,
push_num), and text is escaped in place (push_esc, with a fast path for
plain ASCII).
- sheetData walks the cell store in its own row-major order instead of
copying every address into a BTreeSet and a per-row BTreeMap and looking
each cell up again. Cells outside the store (pictures, spill values,
table headers) still go through the sorted set.
- Formulas with nothing the file spells differently (no LET/LAMBDA, no
_xlfn function, no spill, @, trim operator or structured reference) are
printed from the shared expression instead of being cloned and rebuilt
twice by the file rewrite.
- The A1 printer in gridcraft-formula appends references to the output
instead of formatting each anchor into its own string.
The output is byte-for-byte unchanged.
write_perf, 200,000 rows (2.0M cells, 14.2 MB package), release, Ryzen
16 threads / Linux, 36 interleaved runs each:
before: median 2083.9 ms, min 1615.7 ms
after: median 1501.3 ms, min 1192.3 ms (-28.0 % median, -26.2 % min)
What remains is mostly deflate (zlib-rs, level 6), about 70 % of the save.
Tests pin the exact sheetData of a workbook covering row settings between
cell rows, spill values, every value kind, number spellings, escaping and
formulas with and without a file spelling (generated with the previous
writer), and check each appending helper and the formula fast path against
the code they replace.
The formula fast path is guarded by a test that runs both paths over every
function in the gridcraft-functions registry (a new dev-dependency, allowed
by the layer rules), in 15 surrounding shapes covering every operator and
rewrite, so a function or rewrite added later without updating the trigger
list fails it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Python cell is written to xlsx as [MS-XLSX] 2.3.10 specifies: its
formula is _xlfn._xlws.PY(i,0|1), where i is the zero-based index of the
cell's script in the workbook's Python part and 0|1 the return type, 0
for an Excel value and 1 for a Python object. The part (2.1.23) is
xl/python.xml, content type application/vnd.ms-excel.python+xml,
related from the workbook part with
http://schemas.microsoft.com/office/2023/09/relationships/Python; its
root python holds pythonScripts, one pythonScript per Python cell in the
order the cells are written, with the code in code. The code keeps its
xl() references as text, so the formula has no further arguments. The
cell keeps its cached value. A spilling result is written like a dynamic
array (cm="1", t="array" ref=...) with the spilled values cached, so
other applications show the values (and #NAME? if they recalculate).
The reader turns a PY( formula with or without the _xlfn. and _xlws.
prefixes into a Python cell. An index names a script of the Python part,
and each %Px% in the script becomes the text of the formula's argument x
(2.3.10.4). The string-literal form, PY("chunk"&"chunk",0|1), still
reads, from the formula's own text so line breaks in the code survive. A
formula naming a script the part doesn't hold keeps only its cached
value, with a warning: no calculation can evaluate it. The reader keeps
the cached spilled values as the cell's result, so a workbook from Excel
shows its arrays without running anything. Python errors are cached as
#VALUE!, as #SPILL! and #CALC! already are. Native JSON keeps Cell.py
whole: code, return type, the last result and the exact error.
Calc lays out a Python cell's kept result: Python cells are nodes of the
dependency graph that read nothing, and a pass takes a dirty Python
cell's result instead of evaluating it, so its array spills, is blocked
(#SPILL!) and unblocked, and feeds its dependents exactly like a
formula's. An array that is malformed or has more than 1,000,000 cells
(only a crafted file holds one) shows #SPILL!. A dirty Python cell is
pending in the pass, as a dirty formula is, so a formula, a range read
or a cell-by-cell range walk that reaches it before its turn lays it out
first rather than reading its old value.
Opening a file can't make the app loop over a claimed size: the reader
keeps a Python cell's spill range only when it has at most 1,000,000
cells (a range claiming the whole sheet is dropped with its values, as a
formula's spilled values are), and io::adopt_python rebuilds results for
at most 4,000,000 cells over the whole workbook, with a warning for the
cells it skips. The Python part is read up to 64 MB and 1,000,000
scripts, a script is kept up to one character past what a cell may
hold, and the cells take at most 64 MB of code from the part in all,
since several formulas can name one script. io::enforce_python_limits
drops the code of a cell whose code doesn't fit in 8,192 characters as
PY(...) (the cell keeps its cached value, with a warning) and a kept
result that isn't a well-formed array within the cap, each with its
spill range and spilled values, so the sheet and a save no longer show
the old spill. Formats that keep PY() as an ordinary formula (xls, xlsb,
ods) are adopted the same way. Opening a file runs nothing.
The command-sequence fuzzer sets Python cells (code with xl()
references, quotes, line breaks, and code at and over the size limit),
and its xlsx round-trip check allows the Python errors becoming
#VALUE!, which is what the file format stores.
Tests: the written formula naming its script, the Python part with its
content type and relationship, return types, a spilling table saved as a
dynamic array, Python errors as #VALUE!, the round trip of values, code,
long code with quotes and line breaks and the error, and no Python part
without Python cells; a workbook laid out as the format specifies (an
index, a parameter named as %P2%) read as Python cells with their
spilled values and saved back with its code in a Python part; the
string-literal form; a formula naming a missing script, with and without
a Python part; 20,000 cells naming one long script against the code
budget; the part's reader and writer (escaping, odd %P tokens, a long
script, a repeated parameter); a blocker in the spill area giving #SPILL!
and clearing it spilling again; the native JSON round trip; a spill
range claiming the whole sheet; the workbook-wide budget; hostile JSON
(code over the limit, a result claiming far more cells than it holds); a
malformed result that never spills; PY() formulas at 8,079 and 8,080
characters of code. Calc: formulas, MAX, SUM, COUNTA and SUMPRODUCT read
a dirty Python cell whose array is blocked before its turn, in a full
and an incremental recalculation, and see the array once it is
unblocked; enforce_python_limits removes the spill of dropped code and
of a dropped result, and keeps a sound one.
Cell gains `py: Option<Arc<PyCode>>`, read with a serde default and not
written for ordinary cells, so older native files load and ordinary
cells serialize as before. PyCode holds the code, the return type (an
Excel value or a Python object), the last result (value, type name,
preview, whether it converted to cells) and a stale flag for manual
calculation. The cell's `value` stays the cached result, so formulas,
display and saving read a Python cell like any other. is_blank,
CellStore::has and input_text (which returns the code) take the code
into account; Cell::has_content and clear_content cover value, formula
and code together.
The model reads and writes the PY function's syntax as Microsoft
documents it, PY("chunk"&"chunk",0|1), with string literals of at most
255 characters and the return type 0 for an Excel value, 1 for a Python
object. The reader takes the `_xlfn._xlws.` prefix (and `_xlfn.` or
`_xlws.` alone, in any letter case); to_file_formula writes it. (An xlsx
file keeps the code in its Python part instead; the xlsx commit adds
it.) The size limit is Excel's 8,192 characters for that formula, not a
count of code characters: doubled quotes and the "&" between literals
count, so code with many quotes fits in fewer characters (8,079
characters of plain code fit exactly). The model also finds the
xl("...") literals in code, skipping comments and other strings; a plain
literal may escape a quote or a backslash. A rewritten reference that
holds the literal's quote (a sheet name that needs quotes, 'Q4
2026'!B2) takes the other quote as the literal's delimiter, or an
escaped quote when it holds both, so it never ends the string early.
Edits: cell.set over a Python cell, Clear Contents, Clear All and the
cells a merge empties remove the code; Clear Formats keeps it. Copy and
paste (All, Formulas), Fill, sort, cut-move, sheet copy and undo carry
it; a cut to another sheet first qualifies its xl() references with the
source sheet, as a moved formula's references are, so they keep
reading the same cells. Paste Values and Paste Link do not carry it,
and paste operations do not combine numbers with a Python cell. Fill
copies Python cells instead of extending them as a series. cell.set
with =PY(...) is refused with a message that points to python.set, and
Text to Columns keeps a piece that reads as =PY(...) as text instead of
clearing it. Goal Seek refuses a Python cell as its changing cell, and
Solver as a variable cell, as both refuse formulas: the number written
there would replace the code. The sheet-protection check compares the
code, so a locked Python cell cannot be changed. cell.get reports
{python: code, output, typeName, preview, stale}.
The xl() literals follow reference-moving edits: rewrite_all_formulas
(row and column insert and delete, cell insert and delete, cut-move),
sheet rename and sheet delete adjust each literal like a formula
reference, and replace it only when it changed.
Errors: CellError gains Python, Busy, Timeout and Blocked, and ALL has
14 entries, so the lexer reads them as literals. ERROR.TYPE gives 11 for
#BLOCKED!, the number Excel returns. Microsoft documents no ERROR.TYPE
number for #PYTHON!, #BUSY! or #TIMEOUT!; they take 19, 20 and 21, after
the numbers Excel is known to use. The xlsx writer caches the four as
#VALUE!, as it already does #SPILL! and #CALC!, because they are not
error codes of the file format.
Tests cover the =PY refusal, removal by entry and clearing, undo, copy,
paste values, fill and sort, xl() rewrites through row insert, sheet
rename (also to names that need quotes, with one quote and with both),
cut-move (also to another sheet) and column delete, Goal Seek and
Solver refusing Python cells, Text to Columns keeping a =PY piece,
native JSON round trips, the new errors, the prefixes, escaped quotes in
literals, and the encoded size limit at 8,192 characters exactly and
with doubled quotes.
Extract helpers from auto_sum (selection and single-cell guess), update_cursor
(drag, hover, freeze, and header-edge cursors) and read_styles (dxf reader).
No behaviour change.
* Sonar: explicit imports, fn paths for closures, extract helpers for cognitive complexity
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Fix imports
* Fix serde imports
* Fix engine imports
* Fix clippy gate and a type error from the explicit-import change
Remove the unused imports the review flagged (cf.rs, pivot.rs, tables.rs, cmd/sheet.rs) so clippy -D warnings passes, and point the backstage parent-folder lookup at Path::to_str, which the fn-path rewrite had broken.
Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Apply rustfmt to the explicit-import cleanup so the fmt gate passes
* Share the per-directory font path expansion in theme.rs
The candidate font lists repeated the same loop over the Linux font
directories in every role. A single helper expands them in the same order,
so the lookup order is unchanged and the duplicated blocks go away.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Read the Windows fonts folder in one place in theme.rs
Every script-font list repeated the same WINDIR lookup. A single helper
returns the same path, so the duplicated lines go away without changing
which font files are tried.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Build the light and dark UI palettes from one table in theme.rs
Tokens::light() and Tokens::dark() listed the same 40 colour fields in the
same order, so every change had to be made twice. A single palette
constructor takes the mode and picks each colour's value, so each field is
written once and both modes keep their exact colours.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Share the Noto CJK font list between the CJK script lists in theme.rs
The Japanese, Simplified and Korean lists each repeated the same loop over
the Linux CJK directories with a different locale file name. One helper
builds that loop, and each list keeps its own face index, so the lookup
order is unchanged.
🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
* Share Linux Noto directory list across Thai and Hebrew font lookups
Removes the duplicated directory arrays flagged by SonarCloud new-code duplication.
* Reuse in_dirs for Thai and Hebrew Noto font candidates
Removes the remaining duplicated per-directory loops flagged by SonarCloud.
* Share one font-list builder for the Thai and Hebrew faces
The two per-script functions repeated the same Windows, macOS, Linux and
variable-font setup. A single script_fonts helper now builds both lists,
keeping the candidate order unchanged, which clears the Sonar duplication gate.
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
Insert > Line or Area and the All Charts list offer 100% Stacked Area,
and insert.chart and chart.set take it as type area with subtype
stacked100. Each series' area stacks on the ones before it as its
share of the category's total. When every series on the value axis is
100% stacked and no value is negative, the axis runs from 0% to 100%.
A blank cell counts as zero, and a category whose total is zero stays
on the baseline.
An XLSX area chart with percentStacked grouping now opens as 100%
stacked and is saved back as percentStacked. It used to open as a
stacked area and was saved as stacked.
The writer numbers sheetIds from 1 in sheet order, so a rewritten workbook
changes the id of any sheet that is not part of a gapless series from 1.
Anything that recognises a sheet by its id (parts carried through as they
are, external tools) loses it. The model keeps the sheetId it read
(Sheet::file_id, PreservedSheet::sheet_id, not serialised to JSON) and the
writer writes it back; a new sheet, or a copy with a taken id, gets the first
free one.
Data > Outline > Settings now has Summary rows below detail and Summary
columns to right of detail. With one cleared, each group's - / + button
sits on the line before the group, its bracket runs down to the last
detail line, and Hide Detail, Show Detail, the level buttons and
Ungroup mark or clear the collapsed flag on that line. Changing a
direction moves the flags of collapsed groups to their new summary
lines.
- model: Sheet::summary_below and summary_right (on by default, also
for older JSON).
- xlsx: read and written as sheetPr/outlinePr summaryBelow and
summaryRight; files that set them no longer lose them on save.
- engine: data.outlineSettings {sheet?, summaryBelow?, summaryRight?},
undoable; outline::groups_with places the summary line.
- ui: a Settings button in the Data tab's Outline group opens the
dialog; the outline band follows the setting.
A PivotTable has an Enable show details option, set with the new
pivot.options command ({pivot?, enableShowDetails?: bool}). While it is
off, pivot.showDetails and double-clicking a value cell list no records:
the double-click shows a message and does not open the cell for editing.
Labels and headers still edit. Changing the option is one undo step and
does not refresh the report. pivot.fields reports the setting.
XLSX files keep it: an off option is written as enableDrill="0" on the
pivotTableDefinition, and enableDrill is read on open (on when absent).
Agile decryption checked only the password verifier, which authenticates
the key — not the package. An attacker who could replace EncryptedPackage
ciphertext could change workbook content undetected ([MS-OFFCRYPTO]
2.3.4.14, the descriptor's dataIntegrity element).
The reader now requires the dataIntegrity pair, decrypts it with the
recovered package key, and checks HMAC(hash, hmacKey) over the whole
EncryptedPackage stream (StreamSize field included) against the stored
value before any plaintext is returned. Excel always writes the element
("SHOULD" in the spec); descriptors without it are refused as damaged
rather than accepted unauthenticated.
- crypto: RFC 2104/4231 HMAC test vectors, RFC-test cases for a key
longer than the block size
- fixtures write the pair like Excel; new tests: missing/empty/ragged
integrity fields, tampered ciphertext and descriptor, a changed
StreamSize field
Security-sweep findings from the scan at revision ac8da7d4 (storytold-
gridcraft-agent-findings.json), no longer exploitable on main at 72c1d57d.
Each change caps work or storage per sheet/part so a hostile file cannot buy
unbounded work with ordinary bytes; each has a regression test:
- finding 2 (package.rs): a part rejected by the MAX_PART size gate is
charged its declared compressed size up front, so a loop of oversized
parts still ends the allowance instead of re-entering the package.
- finding 4 (xml.rs): elements past the kept-node cap are consumed and
dropped instead of kept — parse cost stays linear, memory stops growing
with the same input (a small compressed workbook previously asked for
gigabytes of declared-sized metadata tree).
- finding 10 (records.rs): .remaining() caches the suffix of a shared
string's Continue parts, so crossing N Continue records is O(N) instead
of O(N^2) in records scanned per character.
- finding 9 (xls shared formulas): the SHRFMLA master template is parsed
once and re-emitted per dependent cell instead of decoded and reprinted
for every referencing cell.
- findings 3, 6, 8 (sheet_read.rs): hyperlink-cell expansion, conditional-
format kept range cells and dynamic-array (deleted-spill) checks are
capped with one import warning each instead of the compact-workbook
multi-gigabyte allocation or the scanned-per-cell cost.
- finding 31 (core/parse.rs): datetime recognition no longer reparses a
prefix at every space with a time-shaped tail; an all-digit field length
is gated before the derivation machinery runs, staying linear in the
text.
🤖 Generated with Codebuff
Co-authored-by: Codebuff <noreply@codebuff.com>
builtin_id already wrote the currency formats 5-8 as custom formats
because readers resolve those ids by locale. The accounting formats
with a currency, 42 and 44, are resolved the same way but were written
as bare ids, so LibreOffice showed a 44 cell holding -0.14 as "- 0.14"
where US Excel shows "$ (0.14)". Treat them like 5-8.
- The workbook's active sheet is an atomic set in place (Workbook::active_sheet /
set_active_sheet), so a window swap or sheet switch never clones the
Arc<Workbook> or marks it dirty. Selections already live in DocState and the
window views. Save records the sheet of the most recently active window.
- A dialog taken out of its window while it runs goes back to that window when
a command run from it switched windows (dialogs::show, ui.dialog.confirm).
- view.arrangeAll {arrange?: tiled|horizontal|vertical|cascade, activeWorkbook?}
returns the visible windows, active first; the desktop moves and sizes the
native windows over the main window's monitor. New Window and Arrange All
buttons on the View ribbon; Arrange Windows dialog.
- Docs: ui-parity, hardware-parity, gaps #20, target-app-parity, ROADMAP.
A sheet can hide its outline's level buttons and brackets while keeping
the outline (Excel's Ctrl+8, saved as showOutlineSymbols="0"). It wasn't
modelled, so such sheets showed a wide outline band Excel doesn't.
- model: Sheet::show_outline_symbols (default on, also for older JSON).
- xlsx: read and written on the sheet view.
- engine: view.showOutlineSymbols {on?} (Ctrl+8), undoable.
- ui: no outline bands while it's off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found while testing crates/xlsx/src/fmla.rs:101 (with crates/formula/src/printer.rs:181)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
prxnvt