mirror of
https://github.com/storytold/gridcraft.git
synced 2026-11-08 03:19:42 -06:00
7c851d7906573d4df66d7b94de50d5123a27deda
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 prxnvt
|
7c5f417a5d |
Merge origin/main (3379f12a) into the Python cells branch
Main gained #470 (floating objects) and #501 (Solver Sensitivity report). Conflicts: - cmd/edit.rs, edit/paste_special.rs, sheet/tabs.rs: #470 adds rewrite_cell_links for chart references beside this branch's rewrite_python_refs; every structural edit, sheet rename and sheet delete runs both, the Python one last where it can refuse the edit (the edit then changes nothing). - cmd/sheet.rs: rewrite() keeps this branch's Result with main's doc comment. - model/src/sheet.rs and engine tests.rs: both sides' additions kept. - ROADMAP.md: header combined, both sides' rows kept. Main's new code writes no cells, so no Python handling was needed beyond these. |
||
|
PranavandFredApps
|
3379f12a56 |
Data: Solver Sensitivity report (#501)
* Data: Solver Sensitivity report Builds on the Answer and Limits reports. reports: [sensitivity] (on data.solver and data.solverReport) writes a Sensitivity report for models solved by Simplex LP or GRG Nonlinear without integer constraints and with a Max or Min objective. Simplex LP: the model is linearised at the solution and solved again, and the report is read from the final tableau: each variable's reduced cost and the range of its objective coefficient over which the basis stays optimal, and each constraint's shadow price and the range of its right side over which that price holds (1E+30 for no limit). GRG Nonlinear: difference gradients at the solution give each binding constraint's Lagrange multiplier (a least-squares fit on the variables inside their bounds, dropping constraints whose multiplier has the wrong sign) and each variable's reduced gradient. The Solver Results dialog offers the report. * Data: Solver Results offers the Sensitivity report only where it applies Integer and binary models, the Evolutionary method and Value Of goals have no Sensitivity report, and choosing it with other reports made the whole report request fail. Solver Results now disables the Sensitivity choice for such models (with the reason on hover) and leaves it out of the request, so the other chosen reports are still written. sensitivity_applies (in cmd::solver) reads the model from data.solver params, as the dialog holds them; a test checks that it agrees with the report for each kind of model. * Data: Solver bounds the nonlinear Sensitivity report's work The Lagrange multipliers of a GRG Nonlinear model are found by least squares over the binding constraints; a constraint whose multiplier has the wrong sign is dropped and the rest are solved again. Each pass costs about 2·k²·f operations for k binding constraints and f free variables, and each pass drops at most one constraint, so a model at the 1,000-constraint limit could take minutes and hold the window. The passes now share a limit of about 10⁹ operations (about a second). Past it the report is refused with a message, before anything changes: the workbook and the Undo history stay as they were, as for the other report refusals. The limit is lowered in tests so the refusal can be reached with a small model. * Data: Solver refuses a Simplex Sensitivity report for a point that is not optimal The Simplex LP Sensitivity report solves the linearised model again to read its final tableau, but shows the final values it was given. data.solverReport accepts any values, so a report could pair values that are not optimal (or not feasible) with the shadow prices, reduced costs and ranges of the true optimum. The report now checks that the given point satisfies every constraint and bound and reaches the optimal objective (both within the constraint precision; another optimal point with the same objective is accepted) and is refused otherwise, before anything changes. * Data: Solver compares the Sensitivity report's objective in both directions The check that a point is optimal compared its objective with the optimum in one direction only, as for an inequality. A point just outside a bound (within the constraint precision) can still have an objective far below the minimum when its costs are large, and that point was accepted. The objective is now compared by the size of the difference, so such a point is refused. * Data: Solver refuses a nonlinear Sensitivity report for a point that is not optimal data.solverReport accepts any finite variable values. The Simplex LP report already refuses values that are not optimal, but the GRG Nonlinear report described whatever point it was given: for example, a minimisation of A1^2 given A1 = 1 found no binding rows and reported a reduced gradient of 2 instead of requiring the optimum at A1 = 0. The nonlinear report now checks, from the gradients it already computes, that the values are a Karush-Kuhn-Tucker point before anything changes: every constraint holds within the tolerance the nonlinear method accepts a solution within, every bound holds within the constraint precision, each variable inside its bounds has a reduced gradient near zero, and a variable at a bound has a reduced gradient that points only out of that bound for the objective's direction. A binding row whose multiplier has the wrong sign is still dropped, and the gradient it would have explained now causes the refusal. The stationarity tolerance is 1E-3 of the largest objective-gradient component (at least 1), well above the reduced gradients left at optima the method solves. * Data: Solver writes the other reports when the Sensitivity report does not apply to the solution Since the nonlinear Sensitivity report refuses values that are not optimal, data.solver with reports that include sensitivity failed as a whole when the run stopped before an optimum (an evaluation limit) or the values were not one, so the solution and the Answer and Limits reports were lost. Solver Results also offered Sensitivity after any solve, and ticking it with Answer after such a run wrote no report. data.solver now writes the Sensitivity report only when the run reached an optimal or converged solution that the report accepts. Otherwise the solve still applies, the other reports are written, and the result lists the report in skippedReports with the reason. Models that have no Sensitivity report (integer, Evolutionary, Value Of) are refused as before. data.solverReport still refuses values that are not optimal; its new skipNotOptimal parameter leaves the report out instead. Solver Results offers Sensitivity only after an optimal or converged status, with the reason on hover, passes skipNotOptimal, and shows the reason when the report is left out. * Data: Solver skips a Sensitivity report in a protected workbook without refusing the call Structure protection is now checked only when a report sheet will be written. A data.solverReport request with skipNotOptimal whose only report is skipped changes nothing, so it succeeds in a workbook whose structure is protected and returns skippedReports; a request that would write a sheet is still refused. * Data: Solver refuses report sheets in a protected workbook before building them A request that must write a report sheet (any Answer or Limits report, or a Sensitivity report that is not allowed to be skipped) is refused before any report is built, so a protected workbook does not run the Limits report's optimisations first. The check after the reports are built remains for a Sensitivity report that may be skipped. * Data: Solver reports Simplex sensitivity only at the vertex it describes The Simplex LP Sensitivity report solves the linearised model again and takes its prices and ranges from that final tableau, but it discarded the vertex the solve returned and showed the values it was given. When the optimum is not unique, another optimal point passed the optimality check, so the report paired those values with the ranges of a different vertex. The report now also requires the given values to match the re-solved vertex within the constraint precision, and refuses another optimum with a message that says so. A Sensitivity report that data.solver or Solver Results may skip is left out instead, as for other points the report does not support. Values from a Simplex LP solve still match, so the usual Solver Results flow is unchanged. * Data: Solver gives a redundant constraint's right side no allowable change The Simplex LP Sensitivity report ranges each right side over the basic values of the real columns only. A row that the other rows make redundant (max x with x = 2 and 2x = 4) keeps its artificial column basic at zero after phase one, and changing either right side alone moves that artificial, so no change has a solution. The report showed 1E+30 and 2 for the first row instead. When a basic artificial has a nonzero entry in the column of the right side being changed, both allowable changes are now 0. The objective-coefficient ranges are unaffected: such a row has no cost and no entries in the real columns, and a cost change does not affect feasibility. * Data: Solver builds the Simplex Sensitivity report's program at the solve's start values The Simplex LP solve linearises the model at the start values, but the Sensitivity report linearised it again at the solution. The difference coefficients then differ in their last bits, and when the optimum is not unique that can break a tie between pivots the other way: the report's solve returned the other vertex and refused the solve's own values. With max 0.1x + 0.1y over a tied edge, more than half of the sampled starting points were refused. The solve and the report now build the program with one function, simplex_program, and the report builds it from the start values it is given (data.solver's start values, which Solver Results passes back as original), so its solve repeats the solve's pivots and returns the same vertex. Start values at which the model cannot be evaluated fall back to the solution, and values that are not the solve's are still refused by the vertex check. * Data: Solver's Sensitivity report sets its noise levels from the coefficients, not the objective's value The figures a Sensitivity report prints as 0 were those below 1E-9 of the objective's value (1E-6 for the nonlinear figures), so a constant in the objective hid real prices: max 1E9 + x with x <= 10 wrote the row's shadow price, exactly 1, as 0, although the constant changes nothing about the optimisation. A shadow price and a reduced cost are combinations of the objective coefficients, so Simplex LP now takes the noise level from the largest coefficient; a reduced gradient is a part of the objective's gradient and a Lagrange multiplier objective per unit of its row's gradient, so GRG Nonlinear takes it from the gradient's size, and the multipliers' sign test compares lambda times the row's gradient size with it. Allowable changes keep the tableau's pivot tolerance as before. * Data: Solver Sensitivity report tests for variables at a bound and for both goals Hand-derived cases pin the report's conventions. Simplex LP: a maximum's variable at its upper bound (reduced cost 1, Allowable Increase 1E+30, Allowable Decrease 1), a minimum with a >= row and a variable at its upper bound (reduced cost -1, Allowable Increase 1, Allowable Decrease 1E+30), and Winston's Dakota Furniture problem, whose variable at zero has reduced cost -5 with Allowable Increase 5 and Allowable Decrease 1E+30, with every shadow price and allowable range worked out from the final basis. GRG Nonlinear: the Lagrange multiplier is the derivative of the optimal objective with respect to the right side and the reduced gradient that with respect to the variable's bound for a minimum (-1) and the negated maximum (+1), and the mirror image of the existing minimum with a variable at its bound (multiplier +2, reduced gradient -4). * Docs: ROADMAP.md records the Solver Sensitivity report The detail documents (docs/gaps.md, docs/target-app-parity.md) record the report, but the roadmap summary did not: its header change note, progress log and revision history now carry it. * Data: Solver's Sensitivity report bounds each figure's rounding noise by its own terms, not the model's largest coefficient or gradient The noise level a figure had to exceed to print was 1E-9 of the largest objective coefficient (Simplex LP) or 1E-6 of the largest gradient component (GRG Nonlinear) for every figure, so an unrelated large coefficient hid real values: max 1E9*x + y with x <= 1 and y <= 1 as rows wrote the second row's shadow price, exactly 1, as 0, and the same model's Lagrange multiplier 1 and a third variable's reduced cost or gradient -1 went the same way. The rounding error of a sum scales with the sum of its terms' sizes, so each figure now has its own bound from the terms it is made of. Simplex LP: a shadow price y_i = sum over the basis rows of c_B(r) * Binv(r, i) is bounded by 1E-9 of those terms' sizes (the row's unit column of the final tableau holds Binv's column i); a reduced cost cf_j - sum of a_ij * y_i by 1E-9 of |cf_j| plus |a_ij| times each price's term sizes, so a price that is a small difference of large terms carries their error into the reduced costs it is used in. GRG Nonlinear: a multiplier solved from its row of the normal equations is bounded by 1E-6 of its own terms (the row's gradient times the objective's, and its coupling terms with the other active rows times their multipliers) over the row's squared norm; the multipliers' sign test uses the same bound, so a row leaves the active set on the criterion the report rounds with; a reduced gradient is bounded by 1E-6 of its own component and the multipliers' terms plus the multipliers' bounds times |g_ij|. Nearly parallel active gradients amplify every multiplier's error beyond a bound local to a row; such multipliers print as computed. A constant in the objective still changes no bound, since none depends on the objective's value. The nonlinear method stops on the 1E9 model from (0, 0) with "unbounded", so its two regression tests ask for the report at the optimum through data.solverReport. * Data: Solver's Sensitivity report tests a nonlinear point's optimality by each variable's and row's own scale, not the model's largest The Karush-Kuhn-Tucker check that refuses a Sensitivity report for a point that is not optimal accepted a reduced gradient within 1E-3 of the largest gradient component, and a row violation within ten times the constraint precision of the largest constant right side. Both scales were the model's, not the figure's: on max 1E9*x + y with x <= 1 and y <= 1 as rows the tolerance was 1E6, so (1, 0.5), where raising y gains 1 per unit, passed as optimal and the report was written for it; on max y - (x - 2)^2 with x <= 1E9 and y <= 1, (2, 1.0001) passed although the second row is over by 1E-4 against its own precision of 2E-5. Each reduced gradient is now tested against 1E-3 of the terms it is made of (the objective's component and the multipliers' parts, at least 1, the same floor as before), and each row against ten times the constraint precision of its own right side. The nonlinear method itself accepts a solution relative to the largest constant right side; a row over by more than its own precision is not satisfied whatever the other rows' sizes, so the report refuses it. The term sizes are shared with the reduced gradients' rounding bound. Two regression tests, each proving the workbook is left as it was and that the optimum is still reported. * Data: Solver refuses a Sensitivity report the model has none of before any report is built or the solve runs The refusals that apply to the model rather than the solution (integer or binary constraints, the Evolutionary method, a Value Of goal) were made inside the report loop, when it reached the Sensitivity report, so a request listing the Limits or Answer report first built those reports for an error: a Value Of model asking for limits and sensitivity ran the Limits report's two optimisations per variable and received only the refusal, and data.solver ran its whole solve before it. prepare_reports now checks the model's refusal before the loop, in both modes (a refusal of the model was already an error in both; only the solution's not-optimal refusal is skipped under skipNotOptimal), and data.solver checks it before the solve. The regression test asks for limits and sensitivity on an integer model, which now gets the Sensitivity refusal instead of the Limits report's, and on a Value Of model, which gets the Sensitivity refusal with no sheet written. --------- Co-authored-by: FredApps <FredApps@users.noreply.github.com> |
||
|
prxnvt
|
745e937b0b | Merge main into Python cells as data | ||
|
PranavandFredApps
|
f5030432fc |
Data: Solver Options (#3087)
* Data: Solver Load/Save Model and Reset All
data.solverSaveModel writes a model (the sheet's saved model, or one
given as data.solver params) down a column of its sheet as formulas:
the objective and goal (=MAX(...), =MIN(...) or =cell=value), the
variable cells (=COUNT(...)), one =AND(relation) per constraint, which
shows TRUE while the constraint holds, and the method and its options
as an array constant. It refuses a range that overlaps the model's
cells or runs off the sheet.
data.solverLoadModel reads such a range (or reads down from its first
cell), checks the model and makes it the active sheet's model.
data.solverReset (Reset All) removes the active sheet's model. Each is
one undo step.
The Solver Parameters dialog has Reset All and Load/Save buttons; the
dialog can also be opened with fields that go over the saved model.
* Data: Solver Reset All also clears a model read from workbook-level names
A sheet's Solver model can come from workbook-level names, which every
sheet without a model of its own reads. Reset All removed only the
sheet-level names, so such a model stayed and the dialog showed it
again.
Removing the workbook-level names would change every other sheet that
reads them. Reset All instead leaves them in place and writes an empty
sheet-level solver_opt over them on the active sheet only; the result
says masked: true. saved_model now reads a sheet's own names when the
sheet has a solver_opt, and the workbook-level names only otherwise,
never a mix of both, and treats an empty solver_opt as no model. The
mask survives XLSX and is replaced when a model is saved on the sheet.
* Data: Solver Save Model clears the rest of a longer model saved at the same cell
Saving a model over a longer one saved at the same cell left the old
model's last rows on the sheet, so Load Model from that cell read the new
options row followed by stale constraint rows and refused the range.
Save Model now clears what is left of a model saved there before (only
when those cells hold a complete model written by Save Model, and never
the new model's own cells), keeping the cells' formatting, in the same
undo step. Load Model reading down from one cell also stops at the
options row, so formulas just below a saved model no longer break it.
* Data: Solver Load Model keeps the Evolutionary convergence and generations
Save Model writes Convergence and the generations without improvement in
its options row and Load Model reads them, but the sheet's saved model
(hidden names) did not hold them, so after Load Model, or any solve,
they fell back to their defaults. The saved model now keeps them for the
Evolutionary method: Convergence as solver_cvg, which means the same in
Excel, and the generations as gridcraft_solver_stall, because Excel's
solver_mni is a time in seconds. Saving a model again and Reset All
remove the GridCraft name with the solver_* names.
* Data: Solver Load Model refuses a range with a gap
Load Model from a range of several cells kept only the cells that hold a
formula, so the rows below a cell without one moved up. A constraint cell
overwritten by a constant gave a model without that constraint, and an
empty cell before the options row gave the default options, both without
an error. Load Model now refuses a range in which a cell before the last
formula holds no formula, and names that cell. Empty cells after the
model are still allowed.
* Data: Solver removes only its own names when it saves or resets a model
Saving a model (by a solve or Load Model) and Reset All removed every
name of the sheet that starts with solver_, so a user's own name such as
solver_budget was deleted with the model. They now remove only the
Solver's names: the ones Excel's Solver writes (solver_opt, solver_adj,
solver_lhs1, solver_rel1, solver_rhs1 and so on) and GridCraft's own.
The names are matched by name rather than by the hidden flag, so a model
whose names lost that flag is still replaced in full.
* Data: Solver Save Model refuses a place where the options cannot spill
The options row that Save Model writes is an array constant of eight
values, which spills into the seven cells to the right of its first
cell. Content in any of those cells blocked the spill and the options
cell showed #SPILL!. Save Model now checks the whole spill range before
it writes anything and refuses, naming the cell, when a cell there holds
content or another formula's spilled values or is a cell of the model,
when merged cells or a table meet the row (a dynamic array cannot spill
into a table), or when the range runs past the last column. It also
refuses a place where the model's column would cover another formula's
spilled values, which that formula would lose to #SPILL!. The options of
a model saved at the same cell before may be replaced, as before.
* Data: Solver Reset All and Load Model drop the old model's options
Reset All cleared the model fields of the Solver Parameters form but
kept the Evolutionary options (Convergence, Mutation Rate, Population
Size, Random Seed, generations without improvement and Require Bounds),
and Load Model kept any of them that the loaded model omits, such as all
of them for a GRG Nonlinear or Simplex LP model. A new Evolutionary model
then solved with the old options, for example with Require Bounds off.
Both now remove the model's options from the form first (one list of
their keys), so an option the model doesn't set takes its default.
* Data: Solver Save Model asks before it replaces data
Save Model wrote its formulas over whatever the model range held.
It is now refused, with the sheet unchanged, when a cell of the
range holds a value or a formula (a model saved there before
included), unless `overwrite: true` is passed. The message names
the first such cell and carries the same "already holds data"
phrase as Data Analysis, so the Solver Parameters dialog asks
"Replace it with the model?" with Replace and Don't Replace. The
question belongs to the range it asked about: editing the range,
Load or Reset All drops it. The check runs after the spill checks,
so the question is asked only when replacing would succeed. A save
remains one undo step.
Tests: the engine test refuses a value, a formula and an earlier
model, accepts only a true boolean, and undoes and redoes the
replacing save in one step; the dialog test covers Don't Replace,
an edited range and Replace. Both fail without the change.
* Data: Solver Load Model and Reset All tests for limits and undo
Tests for behaviour already in place: reading down from one cell of
a column of 5,000 formulas stops after the most rows a model takes,
a range larger than a model is refused before any cell is read, and
malformed or hostile cells and ranges (unbalanced brackets, an
overflowing number, an unknown sheet, 10,000 nested brackets, a
non-text `from`) each give an error and leave no model. Undoing
Reset All brings the model back with its options (the population
size).
* Data: Solver Load Model refuses a model without its options row
Load Model from one cell read down to the first cell without a
formula, and a range dropped its trailing cells without formulas.
A constraint cleared in the middle of a saved model, or its options
row cleared or replaced by a constant, therefore left a prefix of
the model, which was loaded as the whole model with default options
and replaced the sheet's model. Every load now requires the model's
last row to be the options row ({...}) that Save Model writes last;
otherwise it is refused, naming the cell without a formula when
reading down from one cell, and the sheet's model stays as it was.
The test clears a middle constraint, clears the options row and
replaces it by a constant, loads from the first cell and as two
ranges, and checks after each refusal that the old model (three
constraints, Simplex LP) is unchanged. It fails without the change.
* Data: Solver Options
The Solver Parameters dialog's Options button shows the Solver Options
in the same window: Max Evaluations and Integer Optimality for all
methods, and Convergence, Mutation Rate, Population Size, Random Seed,
Maximum Generations without Improvement and Require Bounds on Variables
for Evolutionary. OK keeps the changes for the next solve; Cancel
restores them.
The evaluation limit and Integer Optimality are now part of the model:
they are saved in the hidden names (solver_itr, which counts evaluations
in place of iterations, and solver_tol) and in the options row of Save
Model. Both accept numbers or text, as the other options do.
* Data: Solver keeps its own limits out of Excel's Solver names
The evaluation limit was saved as solver_itr. Excel uses that name for
its iteration limit (100 by default), so a workbook saved by Excel would
give GridCraft 100 evaluations and stop its search early. GridCraft now
saves the evaluation limit as gridcraft_solver_evals, which Excel
ignores, and no longer reads solver_itr. Saving a model again and Reset
All remove it with the other model names. Integer Optimality
(solver_tol) keeps Excel's name, as its meaning is the same.
* Data: Solver Reset All also resets the Solver Options
Reset All removes the model's options from the Solver Parameters form,
the Solver Options fields (the evaluation limit, Integer Optimality and
the Evolutionary options) and Require Bounds, so the next Solve uses the
defaults. It now also removes the copy of the options that Options keeps
for Cancel. A test checks that the options are gone after Reset All and
that Options then shows the defaults.
* Data: Solver reads Integer Optimality with a percent sign as percent
Integer Optimality is given in percent, but text such as "5%" was read
by the number parser as 0.05 and then divided by 100 again, so the
solver used 0.05 % instead of 5 %. A trailing percent sign is now
dropped before the number is read, so "5", "5%" and 5 all mean 5 %.
* Data: Solver Options closes with Escape back to the Solver Parameters
Escape in the Solver Options view went to the shared dialog handler,
which closed the whole Solver dialog: the options were not restored and
parameter edits made before opening Options were lost. Escape (and
closing the window) in Solver Options now does what Cancel does: it
restores the options and returns to the Solver Parameters. Escape in the
parameters still closes the dialog.
* Data: Solver keeps the Evolutionary options when another method solves
Solver Options sets the Evolutionary options whatever the solving
method, but the saved model and the model in data.solver's result held
them only for the Evolutionary method. A GRG Nonlinear or Simplex LP
solve (which saves the model by default) dropped them, and switching
back to Evolutionary brought back the defaults. They are now saved and
returned for every method. Two tests that expected them to be absent
for other methods now check their values.
* Data: Solver Options keeps its OK and Cancel row at button height
The Solver Options' OK and Cancel row was a right-to-left layout placed
straight in the window, which takes all the height it is offered, so
the buttons sat well below the last option and the window was taller
than its content. The row is now wrapped in a horizontal row as tall as
the buttons, as the other form dialogs do (ok_cancel).
The window's height now changes when it switches between the parameters
and the options, and the window re-centres over the next frames, so the
test that switches back and forth lets a few frames pass after each
switch before it clicks.
* Data: Solver Parameters keeps its Solve and Close row at button height
The Solve and Close row was a right-to-left layout placed straight in
the window. The window keeps its size when its content gets shorter, as
it does when the Load/Save area or a note is hidden again, and that
layout then filled the space left over, so Solve moved further below
the last row. The row is now a horizontal row as tall as the buttons,
as ok_cancel and the Solver Options do.
* Data: Solver dialog opened in its options sets them up as the Options button does
Opening the Solver dialog straight in the Solver Options (optionsView:
true) only changed the title. It skipped what the Options button does:
filling each option the form lacks with its default and keeping the
current values for Cancel. With no saved model the fields were blank,
and Cancel or Escape found nothing to restore, so the edits stayed and
the next Solve used them. The button and the dialog now share one
function for this.
* Data: Solver reads Integer Optimality back without floating-point noise
Integer Optimality is stored as a fraction, and the product with 100 is not exact for many percents (0.07 * 100 is 7.000000000000001). The percent is now rounded to 1E-10 % wherever it is read back: the saved model, the solve's model params and Save Model's options row.
* Data: Solver reads whole-number options back as integers
The model's numeric options were returned as floating-point JSON, so after a solve the Solver Options showed 20000.0, 100.0, 0.0 and 50.0. Values read from the hidden names, and Integer Optimality in the solve's result, are now JSON integers where they are whole, and the dialog shows 20000, 100, 0 and 50.
* Data: Solver test checks the model saved by the second solve
The test of Excel's iteration and time limits ignored the result of
its second `data.solver` call and checked only names the first call
had already created. It now reads the saved model afterwards and
checks the values the second call changed (Simplex LP and an
evaluation limit of 700). The call itself still fails, as Simplex LP
refuses the nonlinear model, after the model is saved.
* Data: Solver Load Model refuses an options row cut short
The options row of a saved model was read leniently: a missing value
or one that isn't a number took its default, and values after the
sixteenth were not read at all. A row cut short or edited by hand,
such as ={1}, therefore loaded the model with default options in
place of the saved ones (here: GRG Nonlinear in place of Simplex
LP). The row must now hold at least the eight numbers Save Model
writes (method, non-negative, population size, mutation rate, random
seed, require bounds, convergence, generations without improvement),
and every value in it, however many, must be a number; otherwise
Load Model is refused and the sheet's model stays as it was. Only
the first sixteen values are kept. More numbers than eight are still
read, so a row with options a later version adds loads too.
The test loads rows of one value, seven values, a text value among
the eight, after them and after the sixteenth, and a TRUE, from the
first cell and as a range, checking the old model after each
refusal, and a row of ten numbers, which loads. It fails without the
change.
---------
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
|
||
|
PranavandFredApps
|
71b130be2d |
Data: Solver Load/Save Model and Reset All (#3086)
* Data: Solver Load/Save Model and Reset All
data.solverSaveModel writes a model (the sheet's saved model, or one
given as data.solver params) down a column of its sheet as formulas:
the objective and goal (=MAX(...), =MIN(...) or =cell=value), the
variable cells (=COUNT(...)), one =AND(relation) per constraint, which
shows TRUE while the constraint holds, and the method and its options
as an array constant. It refuses a range that overlaps the model's
cells or runs off the sheet.
data.solverLoadModel reads such a range (or reads down from its first
cell), checks the model and makes it the active sheet's model.
data.solverReset (Reset All) removes the active sheet's model. Each is
one undo step.
The Solver Parameters dialog has Reset All and Load/Save buttons; the
dialog can also be opened with fields that go over the saved model.
* Data: Solver Reset All also clears a model read from workbook-level names
A sheet's Solver model can come from workbook-level names, which every
sheet without a model of its own reads. Reset All removed only the
sheet-level names, so such a model stayed and the dialog showed it
again.
Removing the workbook-level names would change every other sheet that
reads them. Reset All instead leaves them in place and writes an empty
sheet-level solver_opt over them on the active sheet only; the result
says masked: true. saved_model now reads a sheet's own names when the
sheet has a solver_opt, and the workbook-level names only otherwise,
never a mix of both, and treats an empty solver_opt as no model. The
mask survives XLSX and is replaced when a model is saved on the sheet.
* Data: Solver Save Model clears the rest of a longer model saved at the same cell
Saving a model over a longer one saved at the same cell left the old
model's last rows on the sheet, so Load Model from that cell read the new
options row followed by stale constraint rows and refused the range.
Save Model now clears what is left of a model saved there before (only
when those cells hold a complete model written by Save Model, and never
the new model's own cells), keeping the cells' formatting, in the same
undo step. Load Model reading down from one cell also stops at the
options row, so formulas just below a saved model no longer break it.
* Data: Solver Load Model keeps the Evolutionary convergence and generations
Save Model writes Convergence and the generations without improvement in
its options row and Load Model reads them, but the sheet's saved model
(hidden names) did not hold them, so after Load Model, or any solve,
they fell back to their defaults. The saved model now keeps them for the
Evolutionary method: Convergence as solver_cvg, which means the same in
Excel, and the generations as gridcraft_solver_stall, because Excel's
solver_mni is a time in seconds. Saving a model again and Reset All
remove the GridCraft name with the solver_* names.
* Data: Solver Load Model refuses a range with a gap
Load Model from a range of several cells kept only the cells that hold a
formula, so the rows below a cell without one moved up. A constraint cell
overwritten by a constant gave a model without that constraint, and an
empty cell before the options row gave the default options, both without
an error. Load Model now refuses a range in which a cell before the last
formula holds no formula, and names that cell. Empty cells after the
model are still allowed.
* Data: Solver removes only its own names when it saves or resets a model
Saving a model (by a solve or Load Model) and Reset All removed every
name of the sheet that starts with solver_, so a user's own name such as
solver_budget was deleted with the model. They now remove only the
Solver's names: the ones Excel's Solver writes (solver_opt, solver_adj,
solver_lhs1, solver_rel1, solver_rhs1 and so on) and GridCraft's own.
The names are matched by name rather than by the hidden flag, so a model
whose names lost that flag is still replaced in full.
* Data: Solver Save Model refuses a place where the options cannot spill
The options row that Save Model writes is an array constant of eight
values, which spills into the seven cells to the right of its first
cell. Content in any of those cells blocked the spill and the options
cell showed #SPILL!. Save Model now checks the whole spill range before
it writes anything and refuses, naming the cell, when a cell there holds
content or another formula's spilled values or is a cell of the model,
when merged cells or a table meet the row (a dynamic array cannot spill
into a table), or when the range runs past the last column. It also
refuses a place where the model's column would cover another formula's
spilled values, which that formula would lose to #SPILL!. The options of
a model saved at the same cell before may be replaced, as before.
* Data: Solver Reset All and Load Model drop the old model's options
Reset All cleared the model fields of the Solver Parameters form but
kept the Evolutionary options (Convergence, Mutation Rate, Population
Size, Random Seed, generations without improvement and Require Bounds),
and Load Model kept any of them that the loaded model omits, such as all
of them for a GRG Nonlinear or Simplex LP model. A new Evolutionary model
then solved with the old options, for example with Require Bounds off.
Both now remove the model's options from the form first (one list of
their keys), so an option the model doesn't set takes its default.
* Data: Solver Save Model asks before it replaces data
Save Model wrote its formulas over whatever the model range held.
It is now refused, with the sheet unchanged, when a cell of the
range holds a value or a formula (a model saved there before
included), unless `overwrite: true` is passed. The message names
the first such cell and carries the same "already holds data"
phrase as Data Analysis, so the Solver Parameters dialog asks
"Replace it with the model?" with Replace and Don't Replace. The
question belongs to the range it asked about: editing the range,
Load or Reset All drops it. The check runs after the spill checks,
so the question is asked only when replacing would succeed. A save
remains one undo step.
Tests: the engine test refuses a value, a formula and an earlier
model, accepts only a true boolean, and undoes and redoes the
replacing save in one step; the dialog test covers Don't Replace,
an edited range and Replace. Both fail without the change.
* Data: Solver Load Model and Reset All tests for limits and undo
Tests for behaviour already in place: reading down from one cell of
a column of 5,000 formulas stops after the most rows a model takes,
a range larger than a model is refused before any cell is read, and
malformed or hostile cells and ranges (unbalanced brackets, an
overflowing number, an unknown sheet, 10,000 nested brackets, a
non-text `from`) each give an error and leave no model. Undoing
Reset All brings the model back with its options (the population
size).
* Data: Solver Load Model refuses a model without its options row
Load Model from one cell read down to the first cell without a
formula, and a range dropped its trailing cells without formulas.
A constraint cleared in the middle of a saved model, or its options
row cleared or replaced by a constant, therefore left a prefix of
the model, which was loaded as the whole model with default options
and replaced the sheet's model. Every load now requires the model's
last row to be the options row ({...}) that Save Model writes last;
otherwise it is refused, naming the cell without a formula when
reading down from one cell, and the sheet's model stays as it was.
The test clears a middle constraint, clears the options row and
replaces it by a constant, loads from the first cell and as two
ranges, and checks after each refusal that the old model (three
constraints, Simplex LP) is unchanged. It fails without the change.
* Data: Solver Load Model refuses an options row cut short
The options row of a saved model was read leniently: a missing value
or one that isn't a number took its default, and values after the
sixteenth were not read at all. A row cut short or edited by hand,
such as ={1}, therefore loaded the model with default options in
place of the saved ones (here: GRG Nonlinear in place of Simplex
LP). The row must now hold at least the eight numbers Save Model
writes (method, non-negative, population size, mutation rate, random
seed, require bounds, convergence, generations without improvement),
and every value in it, however many, must be a number; otherwise
Load Model is refused and the sheet's model stays as it was. Only
the first sixteen values are kept. More numbers than eight are still
read, so a row with options a later version adds loads too.
The test loads rows of one value, seven values, a text value among
the eight, after them and after the sixteenth, and a TRUE, from the
first cell and as a range, checking the old model after each
refusal, and a row of ten numbers, which loads. It fails without the
change.
---------
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
|
||
|
prxnvt
|
c03c61fd05 | Merge main into Python cells as data | ||
|
Dylan M. Taylor
|
9717ccbac1 | Refactor engine for strict Clippy checks | ||
|
aiongg
|
58465af95a | refactor(engine): move cmd/solver.rs items to cmd/solver/ | ||
|
aiongg
|
6fee0e9134 | refactor(engine): move cmd/solver tests to src/cmd/solver/tests.rs | ||
|
prxnvt
|
44c663aea2 |
Model: Python cells, and the #PYTHON!, #BUSY!, #TIMEOUT! and #BLOCKED! errors
Cell gains `py: Option<Arc<PyCode>>`, read with a serde default and not
written for ordinary cells, so older native files load and ordinary
cells serialize as before. PyCode holds the code, the return type (an
Excel value or a Python object), the last result (value, type name,
preview, whether it converted to cells) and a stale flag for manual
calculation. The cell's `value` stays the cached result, so formulas,
display and saving read a Python cell like any other. is_blank,
CellStore::has and input_text (which returns the code) take the code
into account; Cell::has_content and clear_content cover value, formula
and code together.
The model reads and writes the PY function's syntax as Microsoft
documents it, PY("chunk"&"chunk",0|1), with string literals of at most
255 characters and the return type 0 for an Excel value, 1 for a Python
object. The reader takes the `_xlfn._xlws.` prefix (and `_xlfn.` or
`_xlws.` alone, in any letter case); to_file_formula writes it. (An xlsx
file keeps the code in its Python part instead; the xlsx commit adds
it.) The size limit is Excel's 8,192 characters for that formula, not a
count of code characters: doubled quotes and the "&" between literals
count, so code with many quotes fits in fewer characters (8,079
characters of plain code fit exactly). The model also finds the
xl("...") literals in code, skipping comments and other strings; a plain
literal may escape a quote or a backslash. A rewritten reference that
holds the literal's quote (a sheet name that needs quotes, 'Q4
2026'!B2) takes the other quote as the literal's delimiter, or an
escaped quote when it holds both, so it never ends the string early.
Edits: cell.set over a Python cell, Clear Contents, Clear All and the
cells a merge empties remove the code; Clear Formats keeps it. Copy and
paste (All, Formulas), Fill, sort, cut-move, sheet copy and undo carry
it; a cut to another sheet first qualifies its xl() references with the
source sheet, as a moved formula's references are, so they keep
reading the same cells. Paste Values and Paste Link do not carry it,
and paste operations do not combine numbers with a Python cell. Fill
copies Python cells instead of extending them as a series. cell.set
with =PY(...) is refused with a message that points to python.set, and
Text to Columns keeps a piece that reads as =PY(...) as text instead of
clearing it. Goal Seek refuses a Python cell as its changing cell, and
Solver as a variable cell, as both refuse formulas: the number written
there would replace the code. The sheet-protection check compares the
code, so a locked Python cell cannot be changed. cell.get reports
{python: code, output, typeName, preview, stale}.
The xl() literals follow reference-moving edits: rewrite_all_formulas
(row and column insert and delete, cell insert and delete, cut-move),
sheet rename and sheet delete adjust each literal like a formula
reference, and replace it only when it changed.
Errors: CellError gains Python, Busy, Timeout and Blocked, and ALL has
14 entries, so the lexer reads them as literals. ERROR.TYPE gives 11 for
#BLOCKED!, the number Excel returns. Microsoft documents no ERROR.TYPE
number for #PYTHON!, #BUSY! or #TIMEOUT!; they take 19, 20 and 21, after
the numbers Excel is known to use. The xlsx writer caches the four as
#VALUE!, as it already does #SPILL! and #CALC!, because they are not
error codes of the file format.
Tests cover the =PY refusal, removal by entry and clearing, undo, copy,
paste values, fill and sort, xl() rewrites through row insert, sheet
rename (also to names that need quotes, with one quote and with both),
cut-move (also to another sheet) and column delete, Goal Seek and
Solver refusing Python cells, Text to Columns keeping a =PY piece,
native JSON round trips, the new errors, the prefixes, escaped quotes in
literals, and the encoded size limit at 8,192 characters exactly and
with doubled quotes.
|
||
|
aiongg
|
f102fe29a1 |
Merge g3 into g4
# Conflicts: # ROADMAP.md # crates/engine/src/cmd/edit.rs # crates/engine/src/cmd/formulas.rs # crates/engine/src/cmd/inspect.rs # crates/engine/src/cmd/pivot.rs # crates/ui-egui/src/lib.rs # crates/ui-egui/src/tabs.rs # docs/gaps.md # docs/ui-parity.md |
||
|
|
3b65b9af58 |
Sonar maintainability: explicit imports, method paths, smaller functions (#2412)
* Sonar: explicit imports, fn paths for closures, extract helpers for cognitive complexity Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Fix imports * Fix serde imports * Fix engine imports * Fix clippy gate and a type error from the explicit-import change Remove the unused imports the review flagged (cf.rs, pivot.rs, tables.rs, cmd/sheet.rs) so clippy -D warnings passes, and point the backstage parent-folder lookup at Path::to_str, which the fn-path rewrite had broken. Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Apply rustfmt to the explicit-import cleanup so the fmt gate passes * Share the per-directory font path expansion in theme.rs The candidate font lists repeated the same loop over the Linux font directories in every role. A single helper expands them in the same order, so the lookup order is unchanged and the duplicated blocks go away. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Read the Windows fonts folder in one place in theme.rs Every script-font list repeated the same WINDIR lookup. A single helper returns the same path, so the duplicated lines go away without changing which font files are tried. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Build the light and dark UI palettes from one table in theme.rs Tokens::light() and Tokens::dark() listed the same 40 colour fields in the same order, so every change had to be made twice. A single palette constructor takes the mode and picks each colour's value, so each field is written once and both modes keep their exact colours. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Share the Noto CJK font list between the CJK script lists in theme.rs The Japanese, Simplified and Korean lists each repeated the same loop over the Linux CJK directories with a different locale file name. One helper builds that loop, and each list keeps its own face index, so the lookup order is unchanged. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com> * Share Linux Noto directory list across Thai and Hebrew font lookups Removes the duplicated directory arrays flagged by SonarCloud new-code duplication. * Reuse in_dirs for Thai and Hebrew Noto font candidates Removes the remaining duplicated per-directory loops flagged by SonarCloud. * Share one font-list builder for the Thai and Hebrew faces The two per-script functions repeated the same Windows, macOS, Linux and variable-font setup. A single script_fonts helper now builds both lists, keeping the candidate order unchanged, which clears the Sonar duplication gate. --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Codebuff <noreply@codebuff.com> |
||
|
aiongg
|
2ed3df2bd6 |
G4: two windows on one workbook without copying it; Arrange All
- The workbook's active sheet is an atomic set in place (Workbook::active_sheet /
set_active_sheet), so a window swap or sheet switch never clones the
Arc<Workbook> or marks it dirty. Selections already live in DocState and the
window views. Save records the sheet of the most recently active window.
- A dialog taken out of its window while it runs goes back to that window when
a command run from it switched windows (dialogs::show, ui.dialog.confirm).
- view.arrangeAll {arrange?: tiled|horizontal|vertical|cascade, activeWorkbook?}
returns the visible windows, active first; the desktop moves and sizes the
native windows over the main window's monitor. New Window and Arrange All
buttons on the View ribbon; Arrange Windows dialog.
- Docs: ui-parity, hardware-parity, gaps #20, target-app-parity, ROADMAP.
|
||
|
euisuhandOpenAI Codex
|
370b220834 |
fix(solver): commit solutions and reports atomically
Prepare report data before editing so refused reports cannot leave solution cells or the saved model changed. Commit accepted reports with the optional solution and model, retaining array/protection checks and row autofit after sheet insertion. Closes #1050 Co-Authored-By: OpenAI Codex <noreply@openai.com> |
||
|
prxnvt
|
019d5de98e |
Data: Solver Answer and Limits reports
Builds on the Solver of #327. data.solver takes reports: [answer, limits] and writes them on new sheets before the model's sheet once a solution is found. The Answer report lists the objective and variable cells with their original and final values and each constraint's value, formula, status (Binding or Not Binding) and slack; cells are named by the nearest labels to their left and above. The Limits report minimises and maximises each variable with the others held at the solution, under the model's constraints, and gives the objective at both ends (Simplex LP and GRG Nonlinear, without integer constraints). data.solverReport writes the reports for a given solution and model, so data.solver's result (which now carries the original values and the model) can be passed to it as is. The Solver Results dialog offers the two reports; they are written after Keep Solver Solution or Restore Original Values. |
||
|
prxnvt
|
fa2b9566fb |
Data: Solver Evolutionary method
Builds on the Solver of #327. method: evolutionary runs a genetic search for models that aren't smooth: tournament selection, blend crossover and Gaussian mutation inside the variable bounds, feasibility-first ranking and integer rounding, then a compass search around the best member. It stops when the population converges, when the best member stops improving for a number of generations, or at the evaluation budget. A seeded generator makes runs repeatable. Population size, mutation rate, random seed, convergence and Require Bounds are options; the method and its options are saved in the hidden solver_* names. The Solver Parameters dialog offers the method. |
||
|
vangel76andClaude Opus 5.5
|
dc75b2cf5e |
Data: Solver with Simplex LP, integer branch and bound and a nonlinear method (#326)
data.solver maximises, minimises or targets a value in an objective cell by changing variable cells under <=, >=, =, int and bin constraints. Simplex LP linearises the model through recalculation, checks linearity and solves it with a two-phase simplex; integers use branch and bound. The GRG Nonlinear choice runs an augmented-Lagrangian projected-gradient method. The model is saved as hidden solver_* names; Solver Parameters and Solver Results dialogs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |