Every path that adds a macro runs macros::check, and an xlsx save of a
macro that skipped it fails (the stash refuses the oversized item). A
JSON save wrote such a macro instead, and the next open dropped it with
a warning. json_bytes now checks each macro first, so the save fails in
either format with the check's message and the macro stays in the open
workbook. Tests that build hostile JSON files write them directly.
stash::items capped the items at MAX_ITEMS before reading them, so one
over the size limit took a place (a readable item past the cap was then
left out with fewer than MAX_ITEMS listed) and one past the cap was
counted in `more` though it could never be listed. Items are now read
first: the first MAX_ITEMS readable ones are listed and only readable
ones past them are counted.
The stash reads a legacy _sc_<key> name as data whether or not it is
hidden, so writing key macro_0 removed a person's visible _sc_macro_0
name with no word. The index allocator steps around that key as well;
this test fails without it.
One file with a visible name of the storage shape, a macro in a newer
format and one whose input declaration breaks the rules, saved and
reopened xlsx then JSON and JSON then xlsx: the visible name stays, the
newer macro comes back whole and numbered around the name, the broken
one is left out once, and the readable one survives.
A macro whose format version is above this build's was dropped on open,
so the next save removed it from the file, and the warning said only
that it couldn't be opened. It is now kept as the file held it
(Workbook::macros_unread, shared between snapshots), never read, listed
or run, and written back unchanged by JSON and xlsx saves, numbered
after the readable macros and around visible names of the storage
shape. Kept macros count toward the 1,000-macro limit after the
readable ones; one too large for an xlsx item is left out. The warning
says it is kept and saved back; damaged and over-limit macros keep
their own warning.
check_limits capped the name, description, notes, step count, depth and
input names, and left the shortcut, permissions, output anchor, default
range, column types, command ids, Repeat ranges, If cells and values and
Stop messages bounded only by the item size; the input declaration was
stored unchecked (rows [5, 2], a cell input with cols [2, 5]).
Now each of them has a limit (docs/macros.md lists them), one-line
fields refuse control characters, and a command id is letters, digits,
. and _ (every registered id passes, tested). An If value may be null
(Empty and NotEmpty ignore it); Repeat.max stays the runner's to bound.
An input's least rows and columns can't pass its most, a cell input is
one row and one column with no header and at most one column type, and
no input lists more column types than columns. check_limits takes every
struct apart field by field, so a new field doesn't compile there until
its limit is decided. A macro from a file that breaks a rule is left
out with the usual warning, never changed to fit.
The JSON reader replaces a workbook with an empty sheet list by a new
one before macros::load reads the file's macro entries, so they were
lost with no warning. The entries are carried across the reset. The
reset itself, which also starts the names and styles afresh, is
unchanged.
with_names numbered macros 0..N, and stash::set_many refuses to write a
hidden name beside a visible one of the same name, so a visible
_gc_macro_0_0 (from a file, or defined after the file opened, when the
stored names are no longer in the workbook) made every xlsx save of a
workbook with macros fail. The index is GridCraft's choice: macros now
take the free indices from 0 up, skipping every key a visible
workbook-scope name of the storage shape uses (compared as the stash
reads names), in their order. No arithmetic depends on a number from
the file.
The recorder left out every call that opened a dialog, so a script with an
edit before a menu-style step lost that edit from the recording. A call is
left out only when it opened a dialog and changed nothing (the dialog's
confirming call is the step); one that changed the workbook is kept.
Run from inside another command, automate.stopAction rolled back that
command's own group, pulling the workbook from under the rest of its run; and
the recording buffer kept it, so a recording could replay the stop inside the
run it was part of. Nested, the command is refused and the group stays; the
recorder leaves it out (record::NOT_RECORDED).
stash::items and stash::keys listed the first 10,000 items and let the rest
vanish without a word. They return a Listing { found, more } now, so a caller
can say what it did not list.
The guard asked for a hidden name, but the stash reads a legacy _sc_<key>
name whether it is hidden or not, so a visible one could be removed or
replaced without force. stash::holds_item is the one rule (item_of), shared
by the guard; the test covers a visible legacy name.
The guard refused any name of the stash's shape, so a visible name a person
had defined, such as _gc_rate_0, could not be edited or deleted from Name
Manager, which cannot pass force. It now refuses a name only when the
workbook holds a hidden workbook-scope item under it; a visible name of the
shape is the person's own, as the stash itself treats it. The test covers
such a name. The query comment says what the copy skip relies on: a query
runs nothing that changes the history.
`Trust` defaulted to `Local`, and `DocState::new` set it, so a
document was trusted unless its maker remembered to mark it. Every
current path that reads a file goes through `file.open`, which marks
it `FromFile`, but a later way of building a document from bytes
(recovery, templates, a new import) would have come out trusted by
omission.
`DocState::new` now takes the `Trust`, and `Trust` has no default.
New workbooks and samples pass `Local`, `file.open` passes
`FromFile`, a new window passes the trust of its workbook, and the
tests that build documents from bytes pass `FromFile`.
`execute` copied the document's undo and redo history on every call,
so a panic could put it back: up to 100 entries per stack, each with
a label and a selection. Queries paid for it too, although no query
runs a command or changes the history, and front ends and agents run
queries all the time.
The copy is now skipped for queries (commands that neither journal
nor make undo steps). A test runs every query, with empty and with
typical parameters, and checks the history is unchanged, so a query
that ever changes it fails there. Commands that can change the
history, nested ones included, still take the copy: a nested command
that panics is restored at its own level, and the outer command can
carry on, so relying on the outermost copy would weaken that restore.
`formulas.defineName {"name": "_gc_macro_1_0"}` replaced the hidden
chunk of a stored item with a visible name: the item vanished from
`stash::get` and `stash::keys`, and its other chunks stayed behind as
orphan hidden names. `formulas.deleteName` could remove chunks the
same way, and `formulas.createFromSelection` with `replace` could
replace one from a cell label. `stash::set` refused a clash with a
visible name, but nothing guarded the other direction.
Defining, renaming or deleting a name that `stash::item_key`
recognises (`_gc_<key>_<n>` and legacy `_sc_<key>`) is now refused
with an error that names `force: true`, which lets the call go ahead.
Create from Selection lists such labels in `skipped` unless `force`
is true. The params docs of the three commands say so.
Each `stash::get` looks at every defined name, so listing the items
with `keys` and then reading each with `get` took time quadratic in
the number of names. A file with 20,000 hidden `_gc_macro_<i>_0`
names took about ten seconds in a debug build, and the macro list
reads its items this way from data that comes from the file.
`stash::items(wb, prefix)` now groups the names by item in one pass
and returns each item's text by key, and `keys` lists from the same
pass. Both return at most `MAX_ITEMS` (10,000) items, the first keys
in sorted order, so a file with any number of hidden names can't make
a listing unbounded. A test counts the names each listing looks at.
The xlsx writer drops characters XML 1.0 can't carry: controls other
than tab, line feed and carriage return, and U+FFFE and U+FFFF. A
stash item holding one lost it on save without a warning, and so did
a JSON item, since serde_json leaves U+FFFE and U+FFFF unescaped: a
stored macro parameter changed silently.
`chunk_formulas` now writes each such character, and the marker
itself, as the noncharacter U+FDD0 followed by four hex digits, and
`get` turns them back for `_gc_` items. A marker that isn't followed
by four hex digits naming a character is read as it is. An escape
counts as five units against the 255-unit literal limit and is never
split across literals, and the guard on the stored size allows the
longest item even when every character is escaped. Legacy `_sc_`
items are read unchanged.
A menu-style call with no parameters, such as `file.open`,
`data.fromTextCsv` or `home.formatCells` with `{}`, only queues its
dialog and succeeds. The recorder kept it as a step with empty
parameters, and then kept the dialog's confirming call as well, so
playing the recording back ran the command twice, or failed on the
empty step when played back headless. Only `execute_headless` took
the step back, and only for the open and save-as dialogs.
`execute` now compares the queued UI requests before and after a
top-level command and does not record one that queued a dialog. The
special case in `execute_headless` goes (it keeps turning a missing
file into an error), and with it `Recorder::truncate`, which had no
other caller.
A command that opened an undo group and then returned an error (a `?`
between `group_begin` and `group_end`) left the group open for good:
Undo and Redo refused with "finish or stop the running action first",
later edits made no undo steps, and nothing could stop the group.
`execute` now rolls back every group a failed command opened, as
`group_rollback` does, restoring the workbook, selection, undo and
redo history to where each group began. Only a command that succeeds
keeps a group it opened, which is what a debugger's start needs.
`Session::abort_groups(doc)` and the menu-less command
`automate.stopAction` roll back the groups left open on a document,
for a debugger the person abandons or a caller that dropped its
token. The command is not undoable and works while a group is open,
and the Undo refusal now names it.
The touch log keeps at most a million writes; past that, a write that kept a
cell's value is missing from the footprint, so a shift derived from the
block of written cells could split a row the macro writes. The footprint
now says when the log filled up (logFull in macro.check), and a shift is
refused then, as it is when the macro deletes sheets or objects. Regression
test with a million-cell fill in a dry run.
Every nested command a dry run executes is journaled, so a long macro pushes
the journal past its limit and the journal drops its oldest entries. The
restore cut the journal back to its old length, which then kept entries from
the dry run and pointed Record Actions at the wrong place. The session now
counts the entries its journal has dropped (journal_dropped) and truncates
with its size bookkeeping (truncate_journal); the restore keeps the old
length less what was dropped meanwhile and moves the recording's start back
by the same count. Regression test with 12,000 nested steps.
Four fixes from a review of the runner, each with a regression test that
fails without it:
- A shift inserts cells over the block of every written cell, which moves
that block's columns (or rows) from there on. When a bound input lay in
the way, its data moved and sheared while the run kept the old coordinates
and reported success. The shift is now refused, naming the input and its
place. The output is not counted: it is where the macro writes, and the
second footprint catches data a shift moved into it.
- A dry run or a refused run left a clean workbook dirty: view.zoom,
sheet.activate and the view modes move the saved mark to the current
workbook on a clean document, and the rollback put the workbook back but
not the mark. Saved keeps and restores it.
- overwrite: true answers the question however ask was given; with an
explicit onConflict: ask the answer was ignored and the question came back
forever.
- The dry runs and the run share one step budget as they share one clock;
each phase counted its own 100,000 steps before. ran reports the run's
own steps.
Input.col("date").data resolved to the whole column, header included, because
every method cleared the header state. col and cols keep it; the other
methods still clear it, and only data keeps the header row for col("…").
The token test asks for D3:D11, cols(2..3).data and col(2).data.col(1).
Two items under one key, in any case, would leave two hidden names of one
name, of which an xlsx file keeps one, and get would mix their chunks. A
caller's mistake, refused before anything changes. Regression test.
Once a step has been dropped, or the step count is full, push serialized
each later step only to count it; a file.open step holds a whole file. The
checks that need no size come first.
Six fixes from a review of the resolver, each with a regression test that
fails without it:
- A bare Row or Column under a general reference key (to, from, at, area,
source, target, destination, location) stays text: Change Spelling's
replacement word and a pivot field's area were being rewritten into the
current row's A1 range. Row.col(2), {{Row}} and {"$ref": "Row"} still
resolve there, and the bare word still resolves under cell and range keys.
- {{{{ in a string is a literal {{, so text that needs the braces can be
written; an unknown token is still an error.
- Input.data on a range that is only a header row, and Name: of a table
with no data rows, are refused instead of reading the header.
- below(0), below(-3) and right(0) are refused like resize(0, 1) is, instead
of becoming 1.
- A defined name on a sheet whose name holds a dollar sign resolves: the
dollar anchors are stripped from the reference part only.
- quote_sheet is the engine's one rule (gridcraft_core::quote_sheet), which
quotes a sheet named like a cell reference, an R1C1 token or a boolean and
leaves a period unquoted.
docs/macros.md says which keys take a bare token and how to write a literal
brace.
A macro could declare an input called Q1, AB12 or Sum1, and a step written
with plain A1 text, as a recorded one is, would then resolve to the input's
range instead of the cell. check_limits refuses such names; the docs say so.
Regression test.
Session::grouped ran its action without a panic guard, so a caller outside
a command that caught the panic found the group still open: Undo and Redo
disabled, the partial edits kept, and the next grouped action nested in the
abandoned group. The group is now rolled back and the panic resumed.
Regression test.
- Workbook::macros is a shared list (Macros, a copy-on-write newtype): a
workbook copy made for a cell edit or an undo snapshot holds the same list,
and the undo budget charges a list once a snapshot owns its own copy.
Before, every mutating command copied every step of every macro, and a
hundred undo entries kept a hundred copies the budget never saw.
- Saving to xlsx drops the last save's items and writes the macros in one
pass each (stash::set_many), instead of a scan of every name per macro.
- A hidden _gc_macro_<x>_<k> name with a key this build did not write is
kept on open and on save, as a visible one is, instead of being dropped
without a warning.
- macros::check is the one check for every path that adds a macro (JSON, a
file, built in code): the model's limits and the xlsx size limit.
Regression tests for each.
The depth check refused any step list at the fifth level, so a fourth Repeat
or If could not even hold an empty body: three levels nested, not the four
MAX_MACRO_DEPTH and the docs state. The check now counts the Repeat or If
steps themselves, in the model's limits and in the parser's pass over the
JSON. The test asks for four and refuses a fifth.
A group open before a command and ended or rolled back inside it lost its
undo step when the command then panicked: the restored history held no entry
for the group's changes, so Undo skipped the state before the group. The
panic path now pushes the outermost such group's step from its start to the
restored workbook. The group stays ended, because its token went with the
command. Regression test for a group ended and one rolled back.
Truncating the undo history to its length at the command's start cannot
bring back an entry that a nested edit.undo popped. The guard now keeps the
undo and redo history as they were and puts both back. The regression test
gains the undo-then-panic case.
The source now carries a note and a column width, and the test checks that
the destination received them while its trust stays Local and no cell is
written.
stash::keys reported a key that get read as missing, because only a later
chunk of it existed. keys now counts chunk 0 and legacy names only, as get
does. Regression assertions in the hostile-names test.
file.open with no file succeeds by queuing a dialog, which execute_headless
turns into an error for callers with no UI; the recording kept the step.
Recorder::truncate forgets it and keeps the byte count right. Regression test.
The panic guard restored the workbook and selection but not the history, so
a nested edit made before the panic left a no-op undo step and the redo
history it had cleared stayed lost. The guard now truncates the steps made
during the command and puts the redo history back. Regression test, and the
architecture note says what the guard restores.
`_gc_watch_00` parsed as chunk 0, so a hidden one was read as the item's first
chunk and a visible one stopped `_gc_watch_0` from being written. Only the
chunk number as `set` writes it counts now. Regression test.
Paste Link writes a formula into every cell whatever `what` says, so a linked
paste of formats from a file now makes the destination FromFile as a paste of
formulas does. Column widths and notes write no cells, linked or not, and
still carry nothing. Regression test for the three kinds.
Session::grouped ended its group after the action, and when the action had closed the group's document (file.close, or file.open taking the place of an untouched new workbook) the group had gone with it, so the action was reported as failed although it had done what it was asked. A group that is gone because its document is gone now counts as ended; a group ended by someone else while the document is open is still an error.
The history limit is applied once the outermost command has finished, so the lengths its bookkeeping compares stay true (round 3). That left the steps nested commands make unbounded until then: a script of up to 100,000 steps, or a subtotal with a step per group, kept every one of them alive, each with the cells it retains, and before this branch every push trimmed the history at once.
Each nested command now bounds the steps made since the latest mark on its document to the undo limit and byte budget, dropping the oldest of them. The marks are the starts of the running commands, the starts of the open groups and the end of the last group: the positions a command or group collapses its steps by when it finishes, so nothing before them moves. What the bound drops is only what the final trim would drop.
prxnvt