ui.pythonMode {on: true} (or a switch with Python mode off) answered {"on": false}
when Python mode could not start, so a control or MCP client saw success. It now
returns an error naming why: a locked cell on a protected sheet, a cell or draft
inside another cell's spill, or code hidden on a protected sheet.
Test: a_refused_ui_python_mode_start_is_an_error_with_the_reason (fails without
the fix); python_mode_starts_no_edit_on_hidden_code and
python_mode_does_not_start_on_a_draft_typed_inside_a_spill now expect the error.
Python mode refuses a cell inside another cell's spill, but a draft typed on a spill child could
still switch to it with Ctrl+Alt+Shift+P, the PY badge, ui.pythonMode or =PY then Tab, and
Ctrl+Enter then wrote code into the spill. toggle_python_mode and the formula bar's switch now
check the draft's own cell (draft_in_spill) and keep such a draft an ordinary entry.
Tests: python_mode_does_not_start_on_a_draft_typed_inside_a_spill (control) and
a_draft_typed_inside_a_spill_does_not_enter_python_mode (keys); both fail without the fix.
An ordinary draft typed over an unlocked, hidden Python cell on a protected sheet holds only what
was typed. follow_open_draft dropped it after any command, and ui.inspect hid its text, because
both checked whether the cell was a Python cell. They now check whether the draft holds code: a
draft in Python mode, or one that began on a Python cell's code (py_cell).
Test: a_draft_typed_over_a_hidden_python_cell_stays_open_through_commands (fails without the fix:
the draft was dropped).
Deleting part of a range that xl() reads (row 2 of A1:A3) shortens the
reference instead of making it #REF!, and inserting rows inside the range
lengthens it; either way the data the code reads changed, but the cell kept
its result. rewrite_python_refs now treats a reference whose rows or columns
changed in number like a deleted one: the cell is new code (#BLOCKED!) with no
kept result. A reference that only moves keeps the result.
Test: deleting_the_data_python_code_reads_drops_its_kept_result now covers a
partial deletion and an insertion inside the range (fails without the fix).
A structural edit that deleted the data an xl() reference names rewrote the
reference to #REF! but kept the cell's cached result, so the cell went on
showing a result for data that is gone. rewrite_python_refs now makes such a
cell new code (#BLOCKED!, no kept result), and the recalculation removes its
spill. A rename or a move, after which the reference names the same data,
still keeps the result.
When the reader's range-scan budget ran out part way through a Python spill,
the rest of its cells were read as plain cells, and the result rebuilt on open
was then blocked by them (#SPILL!). On such a sheet every Python spill now
keeps its values as plain values, with no live spill.
Tests: deleting_the_data_python_code_reads_drops_its_kept_result and
python_spills_read_past_the_scan_budget_keep_their_values_as_plain_values;
both fail without the fix.
A Python cell's spill range was recorded only when the file held a record for
one of its other cells, so a result whose other cells are blank, which a file
may leave out, opened with no spill and no rebuilt result. The range is now
recorded when the anchor is read, if a result can fill it.
A Python range larger than a result may be (more than 1,000,000 cells, but
within what the reader tracks) emptied the cells it covered, although it is
no spill and file-format-parity.md says such cells keep the values the file
shows. They are now kept as plain values; a formula's spilled values are still
dropped, since recalculation fills them again.
Tests: a_python_spill_whose_blank_cells_have_no_records_is_still_a_spill and
a_python_spill_range_larger_than_a_result_keeps_its_values_as_plain_values;
both fail without the fix.
* UI: an unsaved macro-editor draft counts as unsaved work when its workbook closes
Closing a workbook (File > Close, Cmd+W, the Backstage Close, a window's
close button) while the macro editor holds unsaved changes now asks about
the draft first, with the editor's own Save changes to macro question,
even if the workbook itself is saved. Before, the workbook closed and the
draft was discarded without a question. Save or Don't Save goes on to
close the workbook, which then asks about its own unsaved changes (a
saved macro is one); Cancel keeps both open. A draft that cannot be
saved keeps its workbook open with the reason shown. A new macro that
has not been edited does not ask, and a close that leaves the workbook
open (one of several windows) or a forced close goes ahead as before.
* UI: a new macro's name change alone counts as an unsaved macro-editor draft
The check compared the draft with an untouched new macro that took the
draft's current name, so a new macro whose only change was its name
compared equal, and closing its workbook discarded the name without
asking. It now compares with the name the editor opened with.
* UI: Quit asks about an unsaved macro-editor draft before quitting
Quit (Cmd+Q, the macOS Quit menu, a quit from the Dock) went straight to
the unsaved workbooks and never consulted the macro step editor, so with
every workbook saved the app quit and discarded the editor's unsaved
steps. It now lets the editor ask first, as the window's close button
already does: Save or Don't Save goes on with the quit, Cancel stops it.
With one native window per workbook, the editor goes on with the app's
quit directly once it has closed: a close sent to the window it is drawn
in would close only that workbook's window.
* UI: a refused macro save while quitting stops the quit
When the macro editor's Save changes question was answered Save during
a quit (the window's close button or Quit) and the save was refused (a
new macro with a name already in use, a limit), the editor stayed open
with the reason shown but kept its quit intent. Closing the editor later
on its own then quit the app. A refused save now stops the quit, as
Cancel does.
* UI: a quit takes over a workbook close waiting on the macro editor
When File > Close was waiting on the macro editor's question and Quit
came in (or the other way round), closing the editor went on with the
quit and then replayed the close. With one native window per workbook,
the quit had already moved to the next unsaved workbook's window, so the
replayed close acted on that workbook and replaced the quit's Save
Changes question with an ordinary close question. The quit now takes
over: it asks about every unsaved workbook itself, and the waiting close
is dropped.
* UI: an untouched new macro closes without asking, however the editor closes
Closing a workbook already left an untouched new macro alone, but the
editor's own close checks still read its dirty flag, which a new macro
starts with: its close button, a quit, the control channel's cancel and
the close after a stopped debug run asked Save changes to macro (or
refused) for a macro nobody had edited. Every close check now asks
whether the draft has unsaved changes, so all of them agree.
* UI: closing the window that shows the macro editor asks about its draft
The editor's unsaved draft was consulted only when its workbook would
close. Closing the window that shows the editor while the workbook
stayed open in another window (window.close, or File > Close in that
window) went ahead: the window's dialog went with it, and the editor's
state was left behind with no window to show it, so the next macro
opened in the editor replaced the draft without asking. That close now
asks first, like a workbook close.
An editor left with no window to show it (a saved one, or a forced
close) is now dropped, as when its workbook closes. A window that has
closed no longer counts as showing a dialog while its parked state is
still kept.
* UI: a close waiting on the macro editor runs in the window it was asked in
A workbook or window close that waited on the editor's Save changes
question was replayed with the command and its parameters alone, so
file.close acted on whichever window was the session's active one when
the editor closed. The editor is drawn with its own window active, but
nothing tied the replay to it. The pending close now records the
editor's window and enters it before running again; if that window has
closed meanwhile, nothing else is closed and a message says so.
* UI: a close asked while the macro editor is asking waits for the answer
A second file.close or window.close for the editor's workbook or window
while its Save changes question stood replaced the pending close with
the new one: the answer then closed whatever the last request asked for,
not what the question had been opened for. A further request for the
same close now waits on the open question (no second question, and the
close runs once); a request for a different close (the workbook while
only the editor's window was closing, or the reverse) is refused with
the reason until the question is answered. A quit keeps taking over as
before.
* UI: test the quit with a macro draft in one window and unsaved cells in another
The quit asks about the editor's draft first, in its window, then about
each unsaved workbook in its window; Cancel at either question stops it
with every window and workbook open, and a draft discarded with Don't
Save before the quit was cancelled stays discarded. Covered in the
single-window harness, which presses the real buttons, and in the
native-windows harness, which now takes a key press in the root window.
---------
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
* Data: Analysis ToolPak two-factor Anova (with and without replication) and z-Test
Builds on the ToolPak of #320. data.analysisToolPak gains three tools:
Anova: Two-Factor Without Replication (rows and columns as factors, the
residual as the error term), Anova: Two-Factor With Replication (samples of
rowsPerSample rows, with the interaction and within-cell terms) and z-Test:
Two Sample for Means (known variances, normal tails and critical values from
NORM.S.DIST and NORM.S.INV). Each has a form in the Data Analysis dialog.
Tests check the reports against Montgomery's hardness-testing and
battery-life examples.
* Data: Analysis ToolPak Fourier Analysis
data.analysisToolPak gains Fourier Analysis: the discrete Fourier transform
or its inverse of one row or column of 2^k values (at most 4096), by an
iterative radix-2 FFT. Input cells are read as IMREAL and IMAGINARY read
them, so complex text such as 3+4i works, and the results are written as
COMPLEX text. Tests check known transforms (1, 2, 3, 4; a sampled cosine),
the inverse round trip and the input limits.
* Data: Analysis ToolPak Random Number Generation and Sampling
data.analysisToolPak gains the last two of Excel's 19 tools. Random Number
Generation draws uniform, normal, Bernoulli, binomial, Poisson and discrete
numbers, or writes a patterned sequence, in one or more columns; binomial and
Poisson draws invert tables built from BINOM.DIST and POISSON.DIST. Sampling
takes every period-th value or a random sample with replacement. Both use a
seeded xoshiro256** generator: the same seed repeats the numbers, and a run
without one reports the seed it chose. Output is capped at 1,000,000 numbers.
* Data: Analysis ToolPak two-factor Anova without replication: Labels caption names the first column
With Labels ticked, Anova: Two-Factor Without Replication takes the column
names from the first row and the row names from the first cell of every data
row. The form's caption read "Labels in first row", so a user with column
headers but no row names who ticked it would lose the first data column. The
caption now reads "Labels in first row and first column", and the command's
params doc says the same. A dialog test checks the caption.
* Data: Analysis ToolPak docs: timestamp lines and revision history for the 19 tools
docs/gaps.md (§13) and docs/target-app-parity.md (what-if and analysis) gain
their Last updated line and a revision-history row for the Analysis ToolPak
reaching all 19 of Excel's tools.
* Data: Analysis ToolPak new tools refuse options that are not numbers
Random Number Generation (from, to, mean, standardDeviation, p, lambda, step,
and the whole-number options) and z-Test (hypothesizedDifference) read their
options with a default for anything that did not parse, so a form value typed
as "0,3" or a typo quietly ran with the default. A blank or absent option
still takes the default; any other value that is not a finite number is now
an error naming the option.
* Data: Analysis ToolPak refuses an Alpha that is not a number
alpha() took any value that did not parse as the default 0.05, so a typed
"0,01" ran z-Test, both two-factor Anovas (and the existing t-tests, F-Test
and single-factor Anova, which share it) at the wrong significance level
without a word. A blank or absent Alpha still means 0.05; any other value that
is not a finite number is now an error.
* Data: Analysis ToolPak Random Number Generation form hides the count for a patterned sequence
A patterned sequence takes its length from From, To, step and the two repeat
counts and ignores Number of Random Numbers, so a count typed there did
nothing. The form shows the count only for the random distributions, and the
patterned options carry a note that the repeats set how many values are
written.
* Data: Analysis ToolPak Random Seed caption says a blank seed draws a new sequence
Random Number Generation and random Sampling labelled the seed "blank for
none", but a blank seed makes the engine choose a new one, so each run draws
different numbers. The caption now reads "blank for a new sequence".
* Data: Analysis ToolPak form scrolls its options in a short window
The Data Analysis window cannot be resized, and the patterned Random Number
Generation form (seven distribution choices and five pattern fields, each
label above its box) is taller than a 600-point window, so OK ended below the
window's edge. The tool's options now sit in a vertical scroll area capped to
the window's height less the title, output options and buttons, which always
stay in view.
* Data: Analysis ToolPak docs: Forecast Sheet is no longer listed as missing
docs/target-app-parity.md's what-if and analysis row still listed Forecast
Sheet as missing, while data.forecastSheet is in the catalog and docs/gaps.md
§13 lists it as present. The row now lists it with the other analysis tools,
and today's revision-history row says so.
* Data: Analysis ToolPak seed applies only where numbers are drawn
A patterned sequence and periodic sampling draw nothing, but the form showed
the Random Seed field for both, and Random Number Generation read the seed for
every distribution: a seed left in the form that was not a whole number
refused a patterned run, and a valid one was reported as the seed used. The
seed field now shows only for the random distributions and random sampling,
and a patterned run neither reads nor reports a seed.
* Data: Analysis ToolPak Fourier Analysis keeps small parts beside large ones
The cleanup after the transform zeroed every part at or below 1e-13 of the
largest output, so a one-value input such as 1e14 + 1i lost its imaginary
part although a transform of one value performs no arithmetic, and a small
harmonic beside a large DC term was lost the same way. The cutoff is now the
rounding bound of the transform itself, 8 · ε · log2(N) · Σ(|re_n| + |im_n|),
divided by N for the inverse (whose outputs are divided by N): it depends on
the input and the number of butterfly stages, never on the largest output,
and is zero for N = 1. An input sum that overflows disables the cleanup
instead of zeroing everything. Two regression tests cover the one-value case
and an eight-point round trip of 1e6 + 1e-7 cos(2πn/8).
* Data: Analysis ToolPak binomial and Poisson tables grow with the distribution, not the trials
Random Number Generation tabulated a binomial or Poisson distribution by
evaluating BINOM.DIST or POISSON.DIST through worksheet-function dispatch
for every outcome from 0 upward, so a million trials with p = 1 meant
1,000,001 calls and a table of as many entries. The table is now built from
one value at the mode and the recurrence pmf(k + 1) = pmf(k) · (n − k)/(k + 1)
· p/(1 − p) (λ/(k + 1) for Poisson), outward in both directions: the left
tail ends where its terms vanish, the right tail once the cumulative
probability reaches 1 within f64 resolution, so the table is as wide as the
outcomes that matter (about 23,000 entries for Binomial(1,000,000, 0.5)).
p = 0 and p = 1 are exact cases with one outcome. The trials limit and the
draws are unchanged. Tests check the recurrence against BINOM.DIST and
POISSON.DIST to 1e-12 relative, the table width at a million trials, and
that the largest inputs complete in milliseconds with the right means.
---------
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
ui-parity.md gains a Python mode row and a shortcut row for
Ctrl+Alt+Shift+P with its Mac form. function-parity.md's Python in
cells row names Python mode. architecture.md describes Python mode on
the shared editor, code_field and its stop keys, and drafts that follow
their sheet by its identity. control-protocol.md lists ui.pythonMode,
the editor's python flag in ui.inspect, and when ui.inspect leaves a
draft out.
ROADMAP.md records the progress.
The name of the PY badge beside fx, which turns Python mode on and off,
in all 18 interface catalogs. cargo xtask i18n-report doesn't list it as
missing.
Python cells get an editing mode of their own in the formula bar, in the
code editor (monospace, Python colours). It starts with Ctrl+Alt+Shift+P
(Cmd+Option+Shift+P on macOS), with =PY then Tab as in Excel, or by
editing a Python cell (F2, a double-click, a click on the formula bar);
typing over a Python cell still replaces it. The shortcut again leaves
Python mode and keeps the text as an ordinary entry.
In Python mode Enter starts a new line with the indentation, Tab and
Shift+Tab indent, Ctrl+Enter (Cmd+Enter on macOS) commits the code
through python.set and Esc cancels. Every commit path of a Python edit
goes through python.set, the formula bar's Enter button and a click on
the sheet included. When the command refuses the code (longer than a
cell holds), its message shows and the edit stays open with the code as
typed. A locked cell on a protected sheet is refused before a draft
exists, and a draft whose cell protection locks while it is open is
dropped with the warning, as an ordinary edit is (#2945). A Python cell's code committed unchanged writes
nothing (no undo step), as F2 then Enter on any other cell does. With
all the code deleted, the commit clears a Python cell through cell.set,
as an empty entry clears any other cell (code of spaces is still code);
on another cell it writes nothing. The formula bar shows while a Python
edit is open, also when View > Formula Bar hides it, since Python mode
edits only there. A Python draft records the window it was typed in,
as an ordinary draft does, so it is committed in that window when
another comes forward.
Python mode replaces the route that sent a changed edit of a Python
cell's code through python.set (EditState.code_edit and
commit_code_edit): every such edit now starts in Python mode, so that
route is removed, and leaving Python mode commits the text as an
ordinary entry through cell.set. The UI tests of that route commit with
Ctrl+Enter, since Enter starts a new line in Python mode; the guard that
keeps a column completion out of a code edit now reads Python mode.
The keys are applied in input order with the typing, on main's
apply_in_order: the code editor takes a set of stop keys from its caller
(apply_in_order_until), so Ctrl+Enter, Esc and the shortcut act on the
text as it was when they were pressed. After a key the text field acts
on (select all), a stop key waits for the next pass with the editor's
keys before it, so select all, Tab and Ctrl+Enter in one frame commits
the indented code. Keyboard events after a stop key in the same pass are
dropped. code_field is the editor without its frame, for the bar.
The PY badge beside fx shows in Python mode and on a selected Python
cell; clicking it switches Python mode as the shortcut does.
ui.pythonMode {on?} does the same from the control channel, without a
dialog, and engine.commands lists it. ui.inspect reports editor.python,
and leaves a Python draft's code out (text null, hidden true) while its
cell's code is hidden on a protected sheet, checked when it is called:
a protection command earlier in the same batch runs before the next
frame. Theme tokens badge_py,
on_badge, sheet_py_tag and on_sheet_py_tag join the contrast test.
A draft, Python or not, records its sheet's identity (Sheet::uid) when
it starts and follows that sheet, since commands that run while it is
open (sheet.move, home.insertSheet, home.deleteSheet, their undo, a
macro, a tool that adds a report sheet) change which sheet sits at its
index. A commit goes to the draft's own sheet; before, a draft committed
after a sheet was inserted before its own went into the inserted sheet,
an ordinary draft as well. A draft whose sheet is gone is dropped with a
message, as one whose cell protection locks is. ui.inspect checks for
hidden code on the draft's own sheet, and once that sheet is gone it
leaves out a Python draft, or one that began on a Python cell.
A Python cell's return type had no command of its own: python.set takes
it, but only together with the code. python.output {cell?, mode?} sets
it on its own; its command entry lists Excel's shortcut,
Ctrl+Alt+Shift+M. Without a mode it switches to the other one, so the
shortcut needs no parameters; a cell that isn't a Python cell is an
error. A cached result belongs to the return type it was made for, so
a switch treats it as python.set treats a new return type: the result
goes, the cell shows #BLOCKED! until it runs, and its old spill goes
with what read it recalculated. Both commands write the cell through
one path, so python.output refuses a table header and a locked cell on
a protected sheet as python.set does. Each switch is one undo step, and
setting the mode a cell already has adds none.
python.set now refuses an output that isn't text instead of ignoring it.
The command-sequence fuzzer runs python.output too. A macro recording
keeps the return type a switch without a mode set, as its mode (as Bold
records on), also for a switch inside a recorded script run, so playing
it back sets that type rather than switching again.
Tests: switching without a mode drops the cached result, its spill and
the value its reader saw, the same as python.set with the other return
type; switching back brings no result back; the code is kept; one undo
step per switch, undo brings the result back, and none for the mode a
cell already has; a cell without a result stays #BLOCKED!; the
refusals (no Python cell, a bad or non-text mode, a bad cell, a table
header, a locked cell, a non-text output for python.set), an unlocked
Hidden cell on a protected sheet switches, and a recorded switch played
back on a cell that already has the new type keeps it.
Each sheet gets an identity (gridcraft_model::SheetUid, Sheet::uid) for
what must follow one sheet while sheets are inserted, deleted, moved or
renamed, which a sheet index or a name can't do. It lives only while the
app runs: it isn't saved to JSON or a file, and a new, inserted, copied
or opened sheet gets a fresh one. A rename, a move and the copies undo
keeps don't change it, so undo of a deletion brings back the same sheet.
Sheets compare by content, whatever their identity: any two identities
compare equal, and SheetUid::get compares them.
sheet.move with copy gives the copy a fresh identity. Tests cover new
sheets, clones, JSON, renames, moves, deletion and undo, copies and a
workbook read from a file.
New query command formulas.traceError and a Trace Error item in the
Error Checking menu, between Error Checking… and Circular References.
From the cell it follows the first reference that shows the same error
(in a range, its first cell showing it) to the cell the error starts in,
as measured in Excel, and returns that source and the arrows: links
between the error cells, and arrows from the source's own references.
The walk visits each cell once and stops after 10,000 links.
The grid's trace now holds arrows with the sheets of both ends, built
once from the command's result. Error links are drawn in a new red sheet
token, the other arrows in a token for the blue they had. "Trace Error"
is translated in all 18 catalogs (German: Spur zum Fehler).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
A sheet more than one page wide and tall always printed down, then
over. PrintSettings gains the page order (serde default Down, then
over), pageLayout.pageSetup takes pageOrder, and the pagination numbers
the pages in it, so printing, PDF export, the print preview, Page Break
Preview, Page Layout view and the &P/&N, first-page and odd/even
headers follow the printed sequence. XLSX reads <pageSetup pageOrder>
and writes pageOrder="overThenDown" only, as Excel does.
The Page Setup dialog gets a Page order choice with a small picture of
the order and runs pageLayout.pageSetup, so the order and the margins
are one undo step; the labels are translated where the dialog is, and
the dialog is now in German too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
A zoom changes the workbook's revision, which the trace arrows are tied
to, so every zoom removed the arrows of Trace Precedents and Trace
Dependents. Zooming moves no cell: Trace::keep_through_zoom carries
arrows that were current over a zoom from the View tab, the Zoom dialog,
the status bar's buttons and slider, Ctrl+wheel and the control channel.
Undoing a zoom is still an edit and removes them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
Structural edits, sheet renames and deletes, and a cut to another sheet
rewrite the xl() references in Python code, and could make a cell's code
longer than the 8,192-character limit that python.set enforces. Opening the
saved file then dropped that code to its value. Such an edit is now refused
with a message naming the cell, and nothing changes; keeping the old text
would have left a reference the edit had moved.
Add command and UI controls for chart data tables, shared chart and PDF rendering, and native XLSX persistence. Cover live values, blanks, formatting, alignment, undo/redo, bounds and table flags with regression tests.
Co-Authored-By: Codex <noreply@openai.com>
python.set kept the stored result whenever the code was unchanged, also when
the call switched the return type between an Excel value and a Python object.
The cell then showed a value and preview of the other type, and a save wrote
the new type with that old cached value. The result is now kept only when both
the code and the return type match; otherwise the cell is new code, which
shows #BLOCKED! until it runs, and its old spill goes with its readers
recalculated.
python.set, and Replace in a Python cell's code, removed the cell's spill
registration and its spilled values before the recalculation ran, so the
recalculation never saw the freed range and formulas reading those cells kept
their old values until a full recalculation. The commands now leave the spill
to the recalculation. When a Python cell's content changes and it has no array
result to lay out, cells_changed removes the spill and queues the freed range,
as it does for a formula replaced by a constant, so its readers are
recalculated; a cell that keeps its array result (the same code again) keeps
its spill and lays it out again. Clearing, pasting over and blocking a Python
spill already recalculated its readers, and undo and redo restore the values
along with the cells; a test covers them.
Save Model (new on main) asks before it replaces data in the column it writes
to, but it looked for a value or a formula only, so a Python cell whose cached
value is empty was taken for an empty cell and its code was replaced without
the question. Python code now counts as content there, as it does elsewhere.
* Data: Solver Load/Save Model and Reset All
data.solverSaveModel writes a model (the sheet's saved model, or one
given as data.solver params) down a column of its sheet as formulas:
the objective and goal (=MAX(...), =MIN(...) or =cell=value), the
variable cells (=COUNT(...)), one =AND(relation) per constraint, which
shows TRUE while the constraint holds, and the method and its options
as an array constant. It refuses a range that overlaps the model's
cells or runs off the sheet.
data.solverLoadModel reads such a range (or reads down from its first
cell), checks the model and makes it the active sheet's model.
data.solverReset (Reset All) removes the active sheet's model. Each is
one undo step.
The Solver Parameters dialog has Reset All and Load/Save buttons; the
dialog can also be opened with fields that go over the saved model.
* Data: Solver Reset All also clears a model read from workbook-level names
A sheet's Solver model can come from workbook-level names, which every
sheet without a model of its own reads. Reset All removed only the
sheet-level names, so such a model stayed and the dialog showed it
again.
Removing the workbook-level names would change every other sheet that
reads them. Reset All instead leaves them in place and writes an empty
sheet-level solver_opt over them on the active sheet only; the result
says masked: true. saved_model now reads a sheet's own names when the
sheet has a solver_opt, and the workbook-level names only otherwise,
never a mix of both, and treats an empty solver_opt as no model. The
mask survives XLSX and is replaced when a model is saved on the sheet.
* Data: Solver Save Model clears the rest of a longer model saved at the same cell
Saving a model over a longer one saved at the same cell left the old
model's last rows on the sheet, so Load Model from that cell read the new
options row followed by stale constraint rows and refused the range.
Save Model now clears what is left of a model saved there before (only
when those cells hold a complete model written by Save Model, and never
the new model's own cells), keeping the cells' formatting, in the same
undo step. Load Model reading down from one cell also stops at the
options row, so formulas just below a saved model no longer break it.
* Data: Solver Load Model keeps the Evolutionary convergence and generations
Save Model writes Convergence and the generations without improvement in
its options row and Load Model reads them, but the sheet's saved model
(hidden names) did not hold them, so after Load Model, or any solve,
they fell back to their defaults. The saved model now keeps them for the
Evolutionary method: Convergence as solver_cvg, which means the same in
Excel, and the generations as gridcraft_solver_stall, because Excel's
solver_mni is a time in seconds. Saving a model again and Reset All
remove the GridCraft name with the solver_* names.
* Data: Solver Load Model refuses a range with a gap
Load Model from a range of several cells kept only the cells that hold a
formula, so the rows below a cell without one moved up. A constraint cell
overwritten by a constant gave a model without that constraint, and an
empty cell before the options row gave the default options, both without
an error. Load Model now refuses a range in which a cell before the last
formula holds no formula, and names that cell. Empty cells after the
model are still allowed.
* Data: Solver removes only its own names when it saves or resets a model
Saving a model (by a solve or Load Model) and Reset All removed every
name of the sheet that starts with solver_, so a user's own name such as
solver_budget was deleted with the model. They now remove only the
Solver's names: the ones Excel's Solver writes (solver_opt, solver_adj,
solver_lhs1, solver_rel1, solver_rhs1 and so on) and GridCraft's own.
The names are matched by name rather than by the hidden flag, so a model
whose names lost that flag is still replaced in full.
* Data: Solver Save Model refuses a place where the options cannot spill
The options row that Save Model writes is an array constant of eight
values, which spills into the seven cells to the right of its first
cell. Content in any of those cells blocked the spill and the options
cell showed #SPILL!. Save Model now checks the whole spill range before
it writes anything and refuses, naming the cell, when a cell there holds
content or another formula's spilled values or is a cell of the model,
when merged cells or a table meet the row (a dynamic array cannot spill
into a table), or when the range runs past the last column. It also
refuses a place where the model's column would cover another formula's
spilled values, which that formula would lose to #SPILL!. The options of
a model saved at the same cell before may be replaced, as before.
* Data: Solver Reset All and Load Model drop the old model's options
Reset All cleared the model fields of the Solver Parameters form but
kept the Evolutionary options (Convergence, Mutation Rate, Population
Size, Random Seed, generations without improvement and Require Bounds),
and Load Model kept any of them that the loaded model omits, such as all
of them for a GRG Nonlinear or Simplex LP model. A new Evolutionary model
then solved with the old options, for example with Require Bounds off.
Both now remove the model's options from the form first (one list of
their keys), so an option the model doesn't set takes its default.
* Data: Solver Save Model asks before it replaces data
Save Model wrote its formulas over whatever the model range held.
It is now refused, with the sheet unchanged, when a cell of the
range holds a value or a formula (a model saved there before
included), unless `overwrite: true` is passed. The message names
the first such cell and carries the same "already holds data"
phrase as Data Analysis, so the Solver Parameters dialog asks
"Replace it with the model?" with Replace and Don't Replace. The
question belongs to the range it asked about: editing the range,
Load or Reset All drops it. The check runs after the spill checks,
so the question is asked only when replacing would succeed. A save
remains one undo step.
Tests: the engine test refuses a value, a formula and an earlier
model, accepts only a true boolean, and undoes and redoes the
replacing save in one step; the dialog test covers Don't Replace,
an edited range and Replace. Both fail without the change.
* Data: Solver Load Model and Reset All tests for limits and undo
Tests for behaviour already in place: reading down from one cell of
a column of 5,000 formulas stops after the most rows a model takes,
a range larger than a model is refused before any cell is read, and
malformed or hostile cells and ranges (unbalanced brackets, an
overflowing number, an unknown sheet, 10,000 nested brackets, a
non-text `from`) each give an error and leave no model. Undoing
Reset All brings the model back with its options (the population
size).
* Data: Solver Load Model refuses a model without its options row
Load Model from one cell read down to the first cell without a
formula, and a range dropped its trailing cells without formulas.
A constraint cleared in the middle of a saved model, or its options
row cleared or replaced by a constant, therefore left a prefix of
the model, which was loaded as the whole model with default options
and replaced the sheet's model. Every load now requires the model's
last row to be the options row ({...}) that Save Model writes last;
otherwise it is refused, naming the cell without a formula when
reading down from one cell, and the sheet's model stays as it was.
The test clears a middle constraint, clears the options row and
replaces it by a constant, loads from the first cell and as two
ranges, and checks after each refusal that the old model (three
constraints, Simplex LP) is unchanged. It fails without the change.
* Data: Solver Options
The Solver Parameters dialog's Options button shows the Solver Options
in the same window: Max Evaluations and Integer Optimality for all
methods, and Convergence, Mutation Rate, Population Size, Random Seed,
Maximum Generations without Improvement and Require Bounds on Variables
for Evolutionary. OK keeps the changes for the next solve; Cancel
restores them.
The evaluation limit and Integer Optimality are now part of the model:
they are saved in the hidden names (solver_itr, which counts evaluations
in place of iterations, and solver_tol) and in the options row of Save
Model. Both accept numbers or text, as the other options do.
* Data: Solver keeps its own limits out of Excel's Solver names
The evaluation limit was saved as solver_itr. Excel uses that name for
its iteration limit (100 by default), so a workbook saved by Excel would
give GridCraft 100 evaluations and stop its search early. GridCraft now
saves the evaluation limit as gridcraft_solver_evals, which Excel
ignores, and no longer reads solver_itr. Saving a model again and Reset
All remove it with the other model names. Integer Optimality
(solver_tol) keeps Excel's name, as its meaning is the same.
* Data: Solver Reset All also resets the Solver Options
Reset All removes the model's options from the Solver Parameters form,
the Solver Options fields (the evaluation limit, Integer Optimality and
the Evolutionary options) and Require Bounds, so the next Solve uses the
defaults. It now also removes the copy of the options that Options keeps
for Cancel. A test checks that the options are gone after Reset All and
that Options then shows the defaults.
* Data: Solver reads Integer Optimality with a percent sign as percent
Integer Optimality is given in percent, but text such as "5%" was read
by the number parser as 0.05 and then divided by 100 again, so the
solver used 0.05 % instead of 5 %. A trailing percent sign is now
dropped before the number is read, so "5", "5%" and 5 all mean 5 %.
* Data: Solver Options closes with Escape back to the Solver Parameters
Escape in the Solver Options view went to the shared dialog handler,
which closed the whole Solver dialog: the options were not restored and
parameter edits made before opening Options were lost. Escape (and
closing the window) in Solver Options now does what Cancel does: it
restores the options and returns to the Solver Parameters. Escape in the
parameters still closes the dialog.
* Data: Solver keeps the Evolutionary options when another method solves
Solver Options sets the Evolutionary options whatever the solving
method, but the saved model and the model in data.solver's result held
them only for the Evolutionary method. A GRG Nonlinear or Simplex LP
solve (which saves the model by default) dropped them, and switching
back to Evolutionary brought back the defaults. They are now saved and
returned for every method. Two tests that expected them to be absent
for other methods now check their values.
* Data: Solver Options keeps its OK and Cancel row at button height
The Solver Options' OK and Cancel row was a right-to-left layout placed
straight in the window, which takes all the height it is offered, so
the buttons sat well below the last option and the window was taller
than its content. The row is now wrapped in a horizontal row as tall as
the buttons, as the other form dialogs do (ok_cancel).
The window's height now changes when it switches between the parameters
and the options, and the window re-centres over the next frames, so the
test that switches back and forth lets a few frames pass after each
switch before it clicks.
* Data: Solver Parameters keeps its Solve and Close row at button height
The Solve and Close row was a right-to-left layout placed straight in
the window. The window keeps its size when its content gets shorter, as
it does when the Load/Save area or a note is hidden again, and that
layout then filled the space left over, so Solve moved further below
the last row. The row is now a horizontal row as tall as the buttons,
as ok_cancel and the Solver Options do.
* Data: Solver dialog opened in its options sets them up as the Options button does
Opening the Solver dialog straight in the Solver Options (optionsView:
true) only changed the title. It skipped what the Options button does:
filling each option the form lacks with its default and keeping the
current values for Cancel. With no saved model the fields were blank,
and Cancel or Escape found nothing to restore, so the edits stayed and
the next Solve used them. The button and the dialog now share one
function for this.
* Data: Solver reads Integer Optimality back without floating-point noise
Integer Optimality is stored as a fraction, and the product with 100 is not exact for many percents (0.07 * 100 is 7.000000000000001). The percent is now rounded to 1E-10 % wherever it is read back: the saved model, the solve's model params and Save Model's options row.
* Data: Solver reads whole-number options back as integers
The model's numeric options were returned as floating-point JSON, so after a solve the Solver Options showed 20000.0, 100.0, 0.0 and 50.0. Values read from the hidden names, and Integer Optimality in the solve's result, are now JSON integers where they are whole, and the dialog shows 20000, 100, 0 and 50.
* Data: Solver test checks the model saved by the second solve
The test of Excel's iteration and time limits ignored the result of
its second `data.solver` call and checked only names the first call
had already created. It now reads the saved model afterwards and
checks the values the second call changed (Simplex LP and an
evaluation limit of 700). The call itself still fails, as Simplex LP
refuses the nonlinear model, after the model is saved.
* Data: Solver Load Model refuses an options row cut short
The options row of a saved model was read leniently: a missing value
or one that isn't a number took its default, and values after the
sixteenth were not read at all. A row cut short or edited by hand,
such as ={1}, therefore loaded the model with default options in
place of the saved ones (here: GRG Nonlinear in place of Simplex
LP). The row must now hold at least the eight numbers Save Model
writes (method, non-negative, population size, mutation rate, random
seed, require bounds, convergence, generations without improvement),
and every value in it, however many, must be a number; otherwise
Load Model is refused and the sheet's model stays as it was. Only
the first sixteen values are kept. More numbers than eight are still
read, so a row with options a later version adds loads too.
The test loads rows of one value, seven values, a text value among
the eight, after them and after the sixteenth, and a TRUE, from the
first cell and as a range, checking the old model after each
refusal, and a row of ten numbers, which loads. It fails without the
change.
---------
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
* Data: Solver Load/Save Model and Reset All
data.solverSaveModel writes a model (the sheet's saved model, or one
given as data.solver params) down a column of its sheet as formulas:
the objective and goal (=MAX(...), =MIN(...) or =cell=value), the
variable cells (=COUNT(...)), one =AND(relation) per constraint, which
shows TRUE while the constraint holds, and the method and its options
as an array constant. It refuses a range that overlaps the model's
cells or runs off the sheet.
data.solverLoadModel reads such a range (or reads down from its first
cell), checks the model and makes it the active sheet's model.
data.solverReset (Reset All) removes the active sheet's model. Each is
one undo step.
The Solver Parameters dialog has Reset All and Load/Save buttons; the
dialog can also be opened with fields that go over the saved model.
* Data: Solver Reset All also clears a model read from workbook-level names
A sheet's Solver model can come from workbook-level names, which every
sheet without a model of its own reads. Reset All removed only the
sheet-level names, so such a model stayed and the dialog showed it
again.
Removing the workbook-level names would change every other sheet that
reads them. Reset All instead leaves them in place and writes an empty
sheet-level solver_opt over them on the active sheet only; the result
says masked: true. saved_model now reads a sheet's own names when the
sheet has a solver_opt, and the workbook-level names only otherwise,
never a mix of both, and treats an empty solver_opt as no model. The
mask survives XLSX and is replaced when a model is saved on the sheet.
* Data: Solver Save Model clears the rest of a longer model saved at the same cell
Saving a model over a longer one saved at the same cell left the old
model's last rows on the sheet, so Load Model from that cell read the new
options row followed by stale constraint rows and refused the range.
Save Model now clears what is left of a model saved there before (only
when those cells hold a complete model written by Save Model, and never
the new model's own cells), keeping the cells' formatting, in the same
undo step. Load Model reading down from one cell also stops at the
options row, so formulas just below a saved model no longer break it.
* Data: Solver Load Model keeps the Evolutionary convergence and generations
Save Model writes Convergence and the generations without improvement in
its options row and Load Model reads them, but the sheet's saved model
(hidden names) did not hold them, so after Load Model, or any solve,
they fell back to their defaults. The saved model now keeps them for the
Evolutionary method: Convergence as solver_cvg, which means the same in
Excel, and the generations as gridcraft_solver_stall, because Excel's
solver_mni is a time in seconds. Saving a model again and Reset All
remove the GridCraft name with the solver_* names.
* Data: Solver Load Model refuses a range with a gap
Load Model from a range of several cells kept only the cells that hold a
formula, so the rows below a cell without one moved up. A constraint cell
overwritten by a constant gave a model without that constraint, and an
empty cell before the options row gave the default options, both without
an error. Load Model now refuses a range in which a cell before the last
formula holds no formula, and names that cell. Empty cells after the
model are still allowed.
* Data: Solver removes only its own names when it saves or resets a model
Saving a model (by a solve or Load Model) and Reset All removed every
name of the sheet that starts with solver_, so a user's own name such as
solver_budget was deleted with the model. They now remove only the
Solver's names: the ones Excel's Solver writes (solver_opt, solver_adj,
solver_lhs1, solver_rel1, solver_rhs1 and so on) and GridCraft's own.
The names are matched by name rather than by the hidden flag, so a model
whose names lost that flag is still replaced in full.
* Data: Solver Save Model refuses a place where the options cannot spill
The options row that Save Model writes is an array constant of eight
values, which spills into the seven cells to the right of its first
cell. Content in any of those cells blocked the spill and the options
cell showed #SPILL!. Save Model now checks the whole spill range before
it writes anything and refuses, naming the cell, when a cell there holds
content or another formula's spilled values or is a cell of the model,
when merged cells or a table meet the row (a dynamic array cannot spill
into a table), or when the range runs past the last column. It also
refuses a place where the model's column would cover another formula's
spilled values, which that formula would lose to #SPILL!. The options of
a model saved at the same cell before may be replaced, as before.
* Data: Solver Reset All and Load Model drop the old model's options
Reset All cleared the model fields of the Solver Parameters form but
kept the Evolutionary options (Convergence, Mutation Rate, Population
Size, Random Seed, generations without improvement and Require Bounds),
and Load Model kept any of them that the loaded model omits, such as all
of them for a GRG Nonlinear or Simplex LP model. A new Evolutionary model
then solved with the old options, for example with Require Bounds off.
Both now remove the model's options from the form first (one list of
their keys), so an option the model doesn't set takes its default.
* Data: Solver Save Model asks before it replaces data
Save Model wrote its formulas over whatever the model range held.
It is now refused, with the sheet unchanged, when a cell of the
range holds a value or a formula (a model saved there before
included), unless `overwrite: true` is passed. The message names
the first such cell and carries the same "already holds data"
phrase as Data Analysis, so the Solver Parameters dialog asks
"Replace it with the model?" with Replace and Don't Replace. The
question belongs to the range it asked about: editing the range,
Load or Reset All drops it. The check runs after the spill checks,
so the question is asked only when replacing would succeed. A save
remains one undo step.
Tests: the engine test refuses a value, a formula and an earlier
model, accepts only a true boolean, and undoes and redoes the
replacing save in one step; the dialog test covers Don't Replace,
an edited range and Replace. Both fail without the change.
* Data: Solver Load Model and Reset All tests for limits and undo
Tests for behaviour already in place: reading down from one cell of
a column of 5,000 formulas stops after the most rows a model takes,
a range larger than a model is refused before any cell is read, and
malformed or hostile cells and ranges (unbalanced brackets, an
overflowing number, an unknown sheet, 10,000 nested brackets, a
non-text `from`) each give an error and leave no model. Undoing
Reset All brings the model back with its options (the population
size).
* Data: Solver Load Model refuses a model without its options row
Load Model from one cell read down to the first cell without a
formula, and a range dropped its trailing cells without formulas.
A constraint cleared in the middle of a saved model, or its options
row cleared or replaced by a constant, therefore left a prefix of
the model, which was loaded as the whole model with default options
and replaced the sheet's model. Every load now requires the model's
last row to be the options row ({...}) that Save Model writes last;
otherwise it is refused, naming the cell without a formula when
reading down from one cell, and the sheet's model stays as it was.
The test clears a middle constraint, clears the options row and
replaces it by a constant, loads from the first cell and as two
ranges, and checks after each refusal that the old model (three
constraints, Simplex LP) is unchanged. It fails without the change.
* Data: Solver Load Model refuses an options row cut short
The options row of a saved model was read leniently: a missing value
or one that isn't a number took its default, and values after the
sixteenth were not read at all. A row cut short or edited by hand,
such as ={1}, therefore loaded the model with default options in
place of the saved ones (here: GRG Nonlinear in place of Simplex
LP). The row must now hold at least the eight numbers Save Model
writes (method, non-negative, population size, mutation rate, random
seed, require bounds, convergence, generations without improvement),
and every value in it, however many, must be a number; otherwise
Load Model is refused and the sheet's model stays as it was. Only
the first sixteen values are kept. More numbers than eight are still
read, so a row with options a later version adds loads too.
The test loads rows of one value, seven values, a text value among
the eight, after them and after the sixteenth, and a TRUE, from the
first cell and as a range, checking the old model after each
refusal, and a row of ten numbers, which loads. It fails without the
change.
---------
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
Merge Across emptied the value and formula of the other cells in each row but
left their Python code, so a hidden cell stayed a Python cell with an empty
value and was saved as a PY formula. It now clears their whole content, as
Merge Cells and Merge & Center already did. In the same command, a merge
change evaluates again the blocked arrays that could reach the area, but only
formulas were picked, so a Python result blocked by a merged cell stayed
#SPILL! after the cells were unmerged; Python cells are picked too.
While a Python cell's code was edited (F2 or the formula bar), typing at the
end of the code could offer AutoComplete from a text value in the same column,
and Enter then committed that text as the code. The editor no longer offers a
completion during a code edit, and committing a code edit ignores one.
A range read during recalculation walked every array laid out in the pass on
the sheet, whatever the range, and a block read (visit_range) gave up its fast
path as soon as any array was laid out there. A Python cell's cached result
can be up to 1,000,000 cells, so 1,000 formulas reading small ranges beside it
could visit about a billion values in one full recalculation; a large formula
array costs the same. Both reads now look only at the arrays whose area meets
the range, and only at the part inside it. The spilled values the sheet stores
from the last calculation are found row by row when the range has fewer rows
than there are spilled values, so a read beside them doesn't scan them either.
When a file is opened, a Python cell falls back to keeping only its value in
four cases: its code is missing from the Python part or the part is damaged
(the xlsx reader), its code is too long for a cell, its kept result is
malformed, or its result is past the adoption budget. The first kept the
values the cell spilled as plain values; the others dropped them, so values
the file showed were lost. All of them now follow one rule, through
Sheet::settle_spill: the anchor and every spilled cell keep their cached
values as plain values, with no live spill range; a cell's own content and
format are never overwritten; and the work is bounded by the spilled values
stored, not by the range. The values move from the spill into the cells, so
nothing is copied. The cut-by-budget test is renamed for what it now checks,
the dropped-result test now expects the values kept, and the adoption warning
and the file-format docs say so.
A script's code was cut at the character limit before its ST_Xstring escapes
were decoded, so a script whose escapes (`_x0009_`, `_x000D_`) made the file's
text longer than the limit lost its end without a warning, although its code
fits. The code is now decoded first and then cut one character past the limit,
so a script that is too long once decoded still fails the check and its cell
keeps only its value, as before. Decoding is linear and reads at most a bounded
number of encoded characters, enough for any code that fits.
A PY formula whose script the file doesn't hold (the Python part missing or
damaged, or the script over the code budget) keeps only its cached value, but
its array range stayed recorded, so the cached values over the rest of its
spill were dropped as a formula's spill would be, and nothing filled them
again. The range of such a cell is now forgotten when it is read: the anchor
and the cells of its spill keep their cached values as plain values, as the
file shows them, with no live spill. An ordinary formula's dynamic-array spill
is handled as before.
A Python part whose zip entry failed to read (a bad checksum or compressed
stream) made the whole workbook fail to open: the error was passed up, and the
failed read had also ended the package's read budget, so even a warning would
have left the sheets unread. The part is now read with
Package::read_optional_limited, which charges a failed read only that part's
allowance, and every error other than TooLarge is a warning. The Python cells
then keep only their cached values, as when the part is missing. Malformed XML
in the part was already a warning; the test covers it too.
When a file's cached Python spills were larger than the adoption budget, the
cells past it kept only their own value, but their spill ranges and spilled
values stayed on the sheet, which went on showing values no result held. The
spill of such a cell is now cleared, as for a dropped result; the warning is
unchanged.
prxnvt