mirror of
https://github.com/storytold/gridcraft.git
synced 2026-11-03 00:41:56 -06:00
Data: Solver Load/Save Model and Reset All (#3086)
* Data: Solver Load/Save Model and Reset All
data.solverSaveModel writes a model (the sheet's saved model, or one
given as data.solver params) down a column of its sheet as formulas:
the objective and goal (=MAX(...), =MIN(...) or =cell=value), the
variable cells (=COUNT(...)), one =AND(relation) per constraint, which
shows TRUE while the constraint holds, and the method and its options
as an array constant. It refuses a range that overlaps the model's
cells or runs off the sheet.
data.solverLoadModel reads such a range (or reads down from its first
cell), checks the model and makes it the active sheet's model.
data.solverReset (Reset All) removes the active sheet's model. Each is
one undo step.
The Solver Parameters dialog has Reset All and Load/Save buttons; the
dialog can also be opened with fields that go over the saved model.
* Data: Solver Reset All also clears a model read from workbook-level names
A sheet's Solver model can come from workbook-level names, which every
sheet without a model of its own reads. Reset All removed only the
sheet-level names, so such a model stayed and the dialog showed it
again.
Removing the workbook-level names would change every other sheet that
reads them. Reset All instead leaves them in place and writes an empty
sheet-level solver_opt over them on the active sheet only; the result
says masked: true. saved_model now reads a sheet's own names when the
sheet has a solver_opt, and the workbook-level names only otherwise,
never a mix of both, and treats an empty solver_opt as no model. The
mask survives XLSX and is replaced when a model is saved on the sheet.
* Data: Solver Save Model clears the rest of a longer model saved at the same cell
Saving a model over a longer one saved at the same cell left the old
model's last rows on the sheet, so Load Model from that cell read the new
options row followed by stale constraint rows and refused the range.
Save Model now clears what is left of a model saved there before (only
when those cells hold a complete model written by Save Model, and never
the new model's own cells), keeping the cells' formatting, in the same
undo step. Load Model reading down from one cell also stops at the
options row, so formulas just below a saved model no longer break it.
* Data: Solver Load Model keeps the Evolutionary convergence and generations
Save Model writes Convergence and the generations without improvement in
its options row and Load Model reads them, but the sheet's saved model
(hidden names) did not hold them, so after Load Model, or any solve,
they fell back to their defaults. The saved model now keeps them for the
Evolutionary method: Convergence as solver_cvg, which means the same in
Excel, and the generations as gridcraft_solver_stall, because Excel's
solver_mni is a time in seconds. Saving a model again and Reset All
remove the GridCraft name with the solver_* names.
* Data: Solver Load Model refuses a range with a gap
Load Model from a range of several cells kept only the cells that hold a
formula, so the rows below a cell without one moved up. A constraint cell
overwritten by a constant gave a model without that constraint, and an
empty cell before the options row gave the default options, both without
an error. Load Model now refuses a range in which a cell before the last
formula holds no formula, and names that cell. Empty cells after the
model are still allowed.
* Data: Solver removes only its own names when it saves or resets a model
Saving a model (by a solve or Load Model) and Reset All removed every
name of the sheet that starts with solver_, so a user's own name such as
solver_budget was deleted with the model. They now remove only the
Solver's names: the ones Excel's Solver writes (solver_opt, solver_adj,
solver_lhs1, solver_rel1, solver_rhs1 and so on) and GridCraft's own.
The names are matched by name rather than by the hidden flag, so a model
whose names lost that flag is still replaced in full.
* Data: Solver Save Model refuses a place where the options cannot spill
The options row that Save Model writes is an array constant of eight
values, which spills into the seven cells to the right of its first
cell. Content in any of those cells blocked the spill and the options
cell showed #SPILL!. Save Model now checks the whole spill range before
it writes anything and refuses, naming the cell, when a cell there holds
content or another formula's spilled values or is a cell of the model,
when merged cells or a table meet the row (a dynamic array cannot spill
into a table), or when the range runs past the last column. It also
refuses a place where the model's column would cover another formula's
spilled values, which that formula would lose to #SPILL!. The options of
a model saved at the same cell before may be replaced, as before.
* Data: Solver Reset All and Load Model drop the old model's options
Reset All cleared the model fields of the Solver Parameters form but
kept the Evolutionary options (Convergence, Mutation Rate, Population
Size, Random Seed, generations without improvement and Require Bounds),
and Load Model kept any of them that the loaded model omits, such as all
of them for a GRG Nonlinear or Simplex LP model. A new Evolutionary model
then solved with the old options, for example with Require Bounds off.
Both now remove the model's options from the form first (one list of
their keys), so an option the model doesn't set takes its default.
* Data: Solver Save Model asks before it replaces data
Save Model wrote its formulas over whatever the model range held.
It is now refused, with the sheet unchanged, when a cell of the
range holds a value or a formula (a model saved there before
included), unless `overwrite: true` is passed. The message names
the first such cell and carries the same "already holds data"
phrase as Data Analysis, so the Solver Parameters dialog asks
"Replace it with the model?" with Replace and Don't Replace. The
question belongs to the range it asked about: editing the range,
Load or Reset All drops it. The check runs after the spill checks,
so the question is asked only when replacing would succeed. A save
remains one undo step.
Tests: the engine test refuses a value, a formula and an earlier
model, accepts only a true boolean, and undoes and redoes the
replacing save in one step; the dialog test covers Don't Replace,
an edited range and Replace. Both fail without the change.
* Data: Solver Load Model and Reset All tests for limits and undo
Tests for behaviour already in place: reading down from one cell of
a column of 5,000 formulas stops after the most rows a model takes,
a range larger than a model is refused before any cell is read, and
malformed or hostile cells and ranges (unbalanced brackets, an
overflowing number, an unknown sheet, 10,000 nested brackets, a
non-text `from`) each give an error and leave no model. Undoing
Reset All brings the model back with its options (the population
size).
* Data: Solver Load Model refuses a model without its options row
Load Model from one cell read down to the first cell without a
formula, and a range dropped its trailing cells without formulas.
A constraint cleared in the middle of a saved model, or its options
row cleared or replaced by a constant, therefore left a prefix of
the model, which was loaded as the whole model with default options
and replaced the sheet's model. Every load now requires the model's
last row to be the options row ({...}) that Save Model writes last;
otherwise it is refused, naming the cell without a formula when
reading down from one cell, and the sheet's model stays as it was.
The test clears a middle constraint, clears the options row and
replaces it by a constant, loads from the first cell and as two
ranges, and checks after each refusal that the old model (three
constraints, Simplex LP) is unchanged. It fails without the change.
* Data: Solver Load Model refuses an options row cut short
The options row of a saved model was read leniently: a missing value
or one that isn't a number took its default, and values after the
sixteenth were not read at all. A row cut short or edited by hand,
such as ={1}, therefore loaded the model with default options in
place of the saved ones (here: GRG Nonlinear in place of Simplex
LP). The row must now hold at least the eight numbers Save Model
writes (method, non-negative, population size, mutation rate, random
seed, require bounds, convergence, generations without improvement),
and every value in it, however many, must be a number; otherwise
Load Model is refused and the sheet's model stays as it was. Only
the first sixteen values are kept. More numbers than eight are still
read, so a row with options a later version adds loads too.
The test loads rows of one value, seven values, a text value among
the eight, after them and after the sixteenth, and a TRUE, from the
first cell and as a range, checking the old model after each
refusal, and a row of ten numbers, which loads. It fails without the
change.
---------
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
This commit is contained in:
Pranav
@@ -293,8 +293,9 @@ pub(super) fn parse_model(s: &Session, p: &Json) -> Result<Model> {
|
||||
|
||||
// ---------------------------------------------------------------- saved model
|
||||
|
||||
fn ref_text(sheet_name: &str, cells: &str) -> String {
|
||||
let q = crate::pivot::quote_sheet(sheet_name);
|
||||
/// `"B2:B4,D1"` as absolute references, each qualified by `sheet_name` when given.
|
||||
fn abs_text(sheet_name: Option<&str>, cells: &str) -> String {
|
||||
let q = sheet_name.map(|n| format!("{}!", crate::pivot::quote_sheet(n))).unwrap_or_default();
|
||||
cells
|
||||
.split(',')
|
||||
.map(str::trim)
|
||||
@@ -302,16 +303,46 @@ fn ref_text(sheet_name: &str, cells: &str) -> String {
|
||||
.filter_map(|part| {
|
||||
let r = RangeRef::parse(split_sheet(part).1.trim())?;
|
||||
let abs = |c: CellRef| format!("${}${}", gridcraft_core::col_to_letters(c.col), c.row as u64 + 1);
|
||||
Some(if r.is_single() { format!("{q}!{}", abs(r.start)) } else { format!("{q}!{}:{}", abs(r.start), abs(r.end)) })
|
||||
Some(if r.is_single() { format!("{q}{}", abs(r.start)) } else { format!("{q}{}:{}", abs(r.start), abs(r.end)) })
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
.join(",")
|
||||
}
|
||||
|
||||
fn ref_text(sheet_name: &str, cells: &str) -> String {
|
||||
abs_text(Some(sheet_name), cells)
|
||||
}
|
||||
|
||||
/// The generations without improvement have no Excel equivalent: Excel's `solver_mni` is a time
|
||||
/// in seconds. They are saved under a GridCraft name, which Excel ignores.
|
||||
const STALL_NAME: &str = "gridcraft_solver_stall";
|
||||
|
||||
/// The names of a saved model after `solver_`, as Excel's Solver writes them (GridCraft writes
|
||||
/// some of them). Saving a model and Reset All remove these and no other `solver_` names.
|
||||
const SOLVER_KEYS: [&str; 28] = [
|
||||
"adj", "cvg", "drv", "eng", "est", "itr", "lin", "mip", "mni", "mrt", "msl", "neg", "nod", "num", "nwt", "opt", "pre", "rbv", "rlx", "rsd",
|
||||
"scl", "sho", "ssz", "tim", "tol", "typ", "val", "ver",
|
||||
];
|
||||
|
||||
/// Whether `name` is one of the names of a saved model: Excel's Solver names (`solver_opt`,
|
||||
/// `solver_lhs1` and so on) and GridCraft's own. A user's name such as `solver_budget` is not.
|
||||
pub(super) fn is_model_name(name: &str) -> bool {
|
||||
let name = name.to_ascii_lowercase();
|
||||
if name == STALL_NAME {
|
||||
return true;
|
||||
}
|
||||
let Some(key) = name.strip_prefix("solver_") else { return false };
|
||||
if SOLVER_KEYS.contains(&key) {
|
||||
return true;
|
||||
}
|
||||
// One name per constraint: solver_lhs1, solver_rel1, solver_rhs1, …
|
||||
["lhs", "rel", "rhs"].iter().filter_map(|p| key.strip_prefix(p)).any(|n| !n.is_empty() && n.bytes().all(|b| b.is_ascii_digit()))
|
||||
}
|
||||
|
||||
/// The model as hidden names scoped to its sheet (replacing any saved before).
|
||||
pub(super) fn save_model(wb: &mut Workbook, m: &Model) {
|
||||
let Some(sheet_name) = wb.sheet(m.sheet).map(|s| s.name.clone()) else { return };
|
||||
wb.names.retain(|n| !(n.scope == Some(m.sheet) && n.name.to_ascii_lowercase().starts_with("solver_")));
|
||||
wb.names.retain(|n| !(n.scope == Some(m.sheet) && is_model_name(&n.name)));
|
||||
let mut put = |name: String, formula: String| {
|
||||
wb.names.push(DefinedName { name, scope: Some(m.sheet), formula, comment: String::new(), hidden: true });
|
||||
};
|
||||
@@ -354,21 +385,24 @@ pub(super) fn save_model(wb: &mut Workbook, m: &Model) {
|
||||
put("solver_mrt".into(), m.evo.mutation.to_string());
|
||||
put("solver_rsd".into(), m.evo.seed.to_string());
|
||||
put("solver_rbv".into(), if m.evo.require_bounds { "1" } else { "2" }.into());
|
||||
put("solver_cvg".into(), m.evo.convergence.to_string());
|
||||
put(STALL_NAME.into(), m.evo.stall.to_string());
|
||||
}
|
||||
put("solver_ver".into(), "3".into());
|
||||
}
|
||||
|
||||
/// The saved model of a sheet as `data.solver` params, if any.
|
||||
///
|
||||
/// A sheet's own (sheet-level) model wins over one saved as workbook-level names, and its names
|
||||
/// are read alone, so the two never mix. A sheet-level `solver_opt` of `""` (written by Reset
|
||||
/// All over a workbook-level model) means the sheet has no model.
|
||||
pub fn saved_model(wb: &Workbook, sheet: usize) -> Option<Json> {
|
||||
let get = |n: &str| {
|
||||
wb.names
|
||||
.iter()
|
||||
.find(|d| d.name.eq_ignore_ascii_case(n) && (d.scope == Some(sheet) || d.scope.is_none()))
|
||||
.map(|d| d.formula.trim().trim_start_matches('=').to_string())
|
||||
};
|
||||
let named = |n: &str, scope: Option<usize>| wb.names.iter().find(|d| d.name.eq_ignore_ascii_case(n) && d.scope == scope);
|
||||
let scope = if named("solver_opt", Some(sheet)).is_some() { Some(sheet) } else { None };
|
||||
let get = |n: &str| named(n, scope).map(|d| d.formula.trim().trim_start_matches('=').to_string());
|
||||
let local = |t: String| -> String { t.split(',').map(|p| split_sheet(p.trim()).1.replace('$', "")).collect::<Vec<_>>().join(",") };
|
||||
let objective = local(get("solver_opt")?);
|
||||
if objective.is_empty() {
|
||||
if objective.trim_matches('"').trim().is_empty() {
|
||||
return None;
|
||||
}
|
||||
let goal = match get("solver_typ").as_deref().map(str::trim) {
|
||||
@@ -407,7 +441,13 @@ pub fn saved_model(wb: &Workbook, sheet: usize) -> Option<Json> {
|
||||
"nonNegative": get("solver_neg").as_deref().map(str::trim) != Some("2"),
|
||||
"method": method,
|
||||
});
|
||||
for (name, key) in [("solver_ssz", "populationSize"), ("solver_mrt", "mutationRate"), ("solver_rsd", "randomSeed")] {
|
||||
for (name, key) in [
|
||||
("solver_ssz", "populationSize"),
|
||||
("solver_mrt", "mutationRate"),
|
||||
("solver_rsd", "randomSeed"),
|
||||
("solver_cvg", "convergence"),
|
||||
(STALL_NAME, "generationsWithoutImprovement"),
|
||||
] {
|
||||
if let Some(v) = get(name).and_then(|v| num_text(&v)) {
|
||||
model[key] = json!(v);
|
||||
}
|
||||
@@ -460,3 +500,163 @@ pub(super) fn model_params(wb: &Workbook, m: &Model) -> Json {
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- model on cells
|
||||
|
||||
/// The cells Save Model writes, top to bottom, as formula text.
|
||||
pub(super) fn model_formulas(m: &Model) -> Result<Vec<String>> {
|
||||
let one_area = |t: &str| !t.contains(',');
|
||||
let mut rows = Vec::with_capacity(m.cons.len() + 3);
|
||||
let obj = abs_text(None, &m.objective.a1());
|
||||
rows.push(match m.goal {
|
||||
Goal::Max => format!("=MAX({obj})"),
|
||||
Goal::Min => format!("=MIN({obj})"),
|
||||
Goal::Value(v) => format!("={obj}={}", gridcraft_core::number_to_text(v)),
|
||||
});
|
||||
rows.push(format!("=COUNT({})", abs_text(None, &m.vars_text)));
|
||||
for c in &m.cons {
|
||||
if !one_area(&c.lhs_text) || !one_area(&c.rhs_text) {
|
||||
return Err(EngineError::Other("Save Model needs each side of a constraint to be one range.".into()));
|
||||
}
|
||||
let lhs = abs_text(None, &c.lhs_text);
|
||||
rows.push(match (c.rel, &c.rhs) {
|
||||
(Rel::Int, _) => format!("=AND({lhs}=INT({lhs}))"),
|
||||
(Rel::Bin, _) => format!("=AND(({lhs}=0)+({lhs}=1))"),
|
||||
(rel, Rhs::Cells(_)) => format!("=AND({lhs}{}{})", rel.text(), abs_text(None, &c.rhs_text)),
|
||||
(rel, Rhs::Const(v)) => format!("=AND({lhs}{}{})", rel.text(), gridcraft_core::number_to_text(*v)),
|
||||
(rel, Rhs::None) => format!("=AND({lhs}{}0)", rel.text()),
|
||||
});
|
||||
}
|
||||
let e = &m.evo;
|
||||
let n = gridcraft_core::number_to_text;
|
||||
let eng = match m.method {
|
||||
Method::Grg => 1,
|
||||
Method::Simplex => 2,
|
||||
Method::Evolutionary => 3,
|
||||
};
|
||||
let yes = |b: bool| if b { 1 } else { 2 };
|
||||
rows.push(format!(
|
||||
"={{{eng},{},{},{},{},{},{},{}}}",
|
||||
yes(m.non_negative),
|
||||
e.population,
|
||||
n(e.mutation),
|
||||
e.seed,
|
||||
yes(e.require_bounds),
|
||||
n(e.convergence),
|
||||
e.stall
|
||||
));
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
/// `NAME(inner)` → inner, ignoring case.
|
||||
fn call_args(t: &str, name: &str) -> Option<String> {
|
||||
let head = t.get(..name.len() + 1)?;
|
||||
(head.eq_ignore_ascii_case(&format!("{name}(")) && t.ends_with(')')).then(|| t.get(name.len() + 1..t.len() - 1).map(str::to_string))?
|
||||
}
|
||||
|
||||
/// References without sheet names or `$`.
|
||||
fn local_refs(t: &str) -> String {
|
||||
t.split(',').map(|p| split_sheet(p.trim()).1.replace('$', "")).collect::<Vec<_>>().join(",")
|
||||
}
|
||||
|
||||
fn not_a_model() -> EngineError {
|
||||
EngineError::Other("The range doesn't hold a Solver model saved with Save Model.".into())
|
||||
}
|
||||
|
||||
/// How many values the options row of Save Model holds at least: method, non-negative,
|
||||
/// population size, mutation rate, random seed, require bounds, convergence and generations
|
||||
/// without improvement.
|
||||
const OPTIONS_FIELDS: usize = 8;
|
||||
|
||||
/// The options row of Save Model, `{method, non-negative, …}`, into `model`. Refused when it
|
||||
/// holds fewer than [`OPTIONS_FIELDS`] values or a value that isn't a number: a row cut short
|
||||
/// or edited by hand would otherwise load with default options in place of the saved ones.
|
||||
fn read_options_row(model: &mut Json, list: &str) -> Result<()> {
|
||||
let incomplete = || {
|
||||
EngineError::Other(format!(
|
||||
"The model's options row doesn't hold the {OPTIONS_FIELDS} or more numbers Save Model writes, so the range isn't a complete model saved with Save Model."
|
||||
))
|
||||
};
|
||||
// Every value is checked; only the first ones are kept (the row is one formula, so its
|
||||
// length is bounded by the formula's).
|
||||
let mut v: Vec<f64> = Vec::with_capacity(16);
|
||||
let mut count = 0usize;
|
||||
for t in list.split([',', ';']) {
|
||||
let x = num_text(t).ok_or_else(incomplete)?;
|
||||
count = count.saturating_add(1);
|
||||
if v.len() < 16 {
|
||||
v.push(x);
|
||||
}
|
||||
}
|
||||
if count < OPTIONS_FIELDS {
|
||||
return Err(incomplete());
|
||||
}
|
||||
let get = |k: usize| v.get(k).copied();
|
||||
model["method"] = json!(match get(0).map(|x| x as i64) {
|
||||
Some(2) => "simplex",
|
||||
Some(3) => "evolutionary",
|
||||
_ => "grg",
|
||||
});
|
||||
model["nonNegative"] = json!(get(1) != Some(2.0));
|
||||
for (k, key) in [(2, "populationSize"), (3, "mutationRate"), (4, "randomSeed"), (6, "convergence"), (7, "generationsWithoutImprovement")] {
|
||||
if let Some(x) = get(k) {
|
||||
model[key] = json!(x);
|
||||
}
|
||||
}
|
||||
model["requireBounds"] = json!(get(5) != Some(2.0));
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// One constraint row of Save Model (`AND(relation)`) as a `data.solver` constraint.
|
||||
fn read_constraint_row(t: String) -> Result<String> {
|
||||
let t = call_args(&t, "AND").unwrap_or(t);
|
||||
if let Some((x, y)) = t.split_once("=INT(").or_else(|| t.split_once("=int(")) {
|
||||
if y.strip_suffix(')') != Some(x) {
|
||||
return Err(not_a_model());
|
||||
}
|
||||
return Ok(format!("{} int", local_refs(x)));
|
||||
}
|
||||
if let Some(x) = t.strip_prefix('(').and_then(|x| x.strip_suffix("=1)")).and_then(|x| x.split_once("=0)+(")).filter(|(a, b)| a == b) {
|
||||
return Ok(format!("{} bin", local_refs(x.0)));
|
||||
}
|
||||
let (l, rel, r) = split_constraint(&t).ok_or_else(not_a_model)?;
|
||||
let r = r.trim_matches('"');
|
||||
Ok(match r.to_ascii_lowercase().as_str() {
|
||||
"integer" => format!("{} int", local_refs(&l)),
|
||||
"binary" => format!("{} bin", local_refs(&l)),
|
||||
_ if num_text(r).is_some() => format!("{} {} {r}", local_refs(&l), rel.text()),
|
||||
_ => format!("{} {} {}", local_refs(&l), rel.text(), local_refs(r)),
|
||||
})
|
||||
}
|
||||
|
||||
/// Reads formula text written by Save Model back into `data.solver` params.
|
||||
pub(super) fn parse_model_formulas(rows: &[String]) -> Result<Json> {
|
||||
let body = |t: &str| t.trim().trim_start_matches('=').replace(' ', "");
|
||||
let first = body(rows.first().ok_or_else(not_a_model)?);
|
||||
let (goal, objective, value) = if let Some(o) = call_args(&first, "MAX") {
|
||||
("max", o, 0.0)
|
||||
} else if let Some(o) = call_args(&first, "MIN") {
|
||||
("min", o, 0.0)
|
||||
} else {
|
||||
let (o, v) = first.split_once('=').ok_or_else(not_a_model)?;
|
||||
("value", o.to_string(), num_text(v).ok_or_else(not_a_model)?)
|
||||
};
|
||||
let variables = call_args(&body(rows.get(1).ok_or_else(not_a_model)?), "COUNT").ok_or_else(not_a_model)?;
|
||||
let mut model = json!({"objective": local_refs(&objective), "goal": goal, "value": value, "variables": local_refs(&variables)});
|
||||
let mut constraints = Vec::new();
|
||||
let rest = rows.get(2..).unwrap_or(&[]);
|
||||
for (i, row) in rest.iter().enumerate() {
|
||||
let t = body(row);
|
||||
if let Some(list) = t.strip_prefix('{').and_then(|x| x.strip_suffix('}')) {
|
||||
// The options: the last row.
|
||||
if i + 1 != rest.len() {
|
||||
return Err(not_a_model());
|
||||
}
|
||||
read_options_row(&mut model, list)?;
|
||||
break;
|
||||
}
|
||||
constraints.push(json!(read_constraint_row(t)?));
|
||||
}
|
||||
model["constraints"] = json!(constraints);
|
||||
Ok(model)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user