mirror of
https://github.com/storytold/gridcraft.git
synced 2026-11-03 00:41:56 -06:00
Merge pull request #2637 from prxnvt/pr/python-cells
Python cells as data: stored, edited and saved as Excel's PY formula (no code runs)
This commit is contained in:
+3
-1
@@ -2,7 +2,7 @@
|
||||
|
||||
**Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away
|
||||
|
||||
> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365)
|
||||
> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365)
|
||||
|
||||
GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX
|
||||
is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web.
|
||||
@@ -135,6 +135,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
|
||||
|
||||
| Date | What landed |
|
||||
|---|---|
|
||||
| 2026-11-02 | Python cells as data: `python.set` stores code in a cell, Python cells copy, move, fill and undo like formulas, and `.xlsx` reads and writes Python cells as [MS-XLSX] specifies (`_xlfn._xlws.PY(index,0 or 1)`, the code in the workbook's Python part) with cached values and spills, so such workbooks open with their results; nothing runs Python (new code shows `#BLOCKED!`), and opening a file limits code size, result size and spill ranges |
|
||||
| 2026-11-02 | Equations: Insert › Equation (a split button with a gallery of examples) and an editor with quick-insert buttons, the error underlined at its position and a live preview; `insert.equation` and `equation.edit`; LaTeX laid out by RaTeX inside the new `gridcraft-math` crate, capped and panic-guarded, and cached on the shape so that painting never runs the layout engine; xlsx writes Office Math with a plain-text fallback and the source in `descr`, and reads Office Math back into LaTeX; PDF prints equations with the standard Symbol font and `[?]` for anything else |
|
||||
| 2026-11-02 | Solver writes a Sensitivity report: Simplex LP shadow prices, reduced costs and allowable ranges from the final tableau; GRG Nonlinear reduced gradients and Lagrange multipliers from difference gradients, at a Karush-Kuhn-Tucker point only; Solver Results offers it where it applies (no score change) |
|
||||
| 2026-11-02 | Floating objects: charts, pictures, shapes and slicers share one selection with a frame of eight resize handles (each moves only its own edges, also on right-to-left sheets); Delete or Backspace deletes the selected object and Escape drops the selection; slicers follow row and column edits and sorts like the other objects; chart series follow cut-moves and sheet renames and deletes; a copied sheet gives its objects fresh ids; `ui.inspect` reports `selectedObject` and `ui.object.select` selects any object (#190, #213) |
|
||||
@@ -254,6 +255,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-11-02 | minor | Progress log: Python cells as data |
|
||||
| 2026-11-02 | minor | Progress log: equations (Insert › Equation, the editor, gridcraft-math) |
|
||||
| 2026-11-02 | minor | What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear (no score change) |
|
||||
| 2026-11-02 | minor | Progress log: one selection and eight resize handles for every floating object |
|
||||
|
||||
@@ -135,6 +135,10 @@ impl Calc {
|
||||
if cell.value == Value::Error(CellError::Circ) {
|
||||
circ.push((si, c));
|
||||
}
|
||||
} else if cell.py.is_some() {
|
||||
// A Python cell lays out its kept result, so it is calculated like a formula
|
||||
// that reads nothing: its spill is blocked and unblocked like a formula's.
|
||||
self.graph.insert((si, c), Node { areas: Vec::new(), dynamic: false });
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -206,11 +210,29 @@ impl Calc {
|
||||
let mut cleared: Vec<(usize, RangeRef)> = Vec::new();
|
||||
for &k in changed {
|
||||
let formula = wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.formula.clone());
|
||||
let py = wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.py.as_ref());
|
||||
let python = py.is_some();
|
||||
// A Python cell whose result (if any) is not an array spills nothing.
|
||||
let spills = py.and_then(|p| p.result_value()).is_some_and(|v| matches!(v, Value::Array(_)));
|
||||
match formula.and_then(|f| f.expr()) {
|
||||
Some(e) => {
|
||||
let (areas, dynamic) = precedents(wb, k.0, &e);
|
||||
self.graph.insert(k, Node { areas, dynamic });
|
||||
}
|
||||
// See `rebuild`: its spill is laid out again from its result when it is calculated.
|
||||
None if python => {
|
||||
self.graph.insert(k, Node { areas: Vec::new(), dynamic: false });
|
||||
// New code has no result yet: the spill the cell had (its old result's, or a
|
||||
// formula's) goes, and what read it is recalculated, as for a formula
|
||||
// replaced by a constant.
|
||||
if !spills
|
||||
&& let Some(sh) = wb.sheet_mut(k.0)
|
||||
&& let Some(r) = sh.spill_ranges.remove(&k.1)
|
||||
{
|
||||
sh.clear_spill(r);
|
||||
cleared.push((k.0, r));
|
||||
}
|
||||
}
|
||||
None => {
|
||||
self.graph.remove(k);
|
||||
// A formula replaced by a constant (or cleared) takes its spilled values with it.
|
||||
@@ -338,11 +360,15 @@ impl Calc {
|
||||
exprs.insert(*k, e);
|
||||
}
|
||||
}
|
||||
let python: HashMap<Key, Value> = dirty
|
||||
.iter()
|
||||
.filter_map(|k| wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.py.as_ref()).and_then(|p| p.result_value()).map(|v| (*k, v)))
|
||||
.collect();
|
||||
let now = self.now();
|
||||
let mut rng = self.rng;
|
||||
let tp = prof_now();
|
||||
let (results, spills, cycles) = {
|
||||
let mut host = PassHost::new(wb, &exprs, &mut rng, now);
|
||||
let mut host = PassHost::new(wb, &exprs, &mut rng, now).with_python(python);
|
||||
host.iterate = wb.calc.iterative;
|
||||
for k in &dirty {
|
||||
host.calculate(*k);
|
||||
@@ -547,6 +573,48 @@ pub(crate) mod tests {
|
||||
v
|
||||
}
|
||||
|
||||
/// A 1,000 x 1,000 Python result laid out from A1 in a pass, with the same values already
|
||||
/// spilled on the sheet from the last calculation, as a full recalculation finds them.
|
||||
fn big_python_spill() -> (Workbook, HashMap<Key, Value>) {
|
||||
let n = 1_000usize;
|
||||
let data: Vec<Value> = (0..n * n).map(|i| Value::Number(i as f64)).collect();
|
||||
let array = gridcraft_core::Array::new(n, n, data.clone()).unwrap();
|
||||
let mut wb = Workbook::new();
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
let anchor = CellRef::new(0, 0);
|
||||
let py = gridcraft_model::PyCode::new("big");
|
||||
sh.set_cell(anchor, Cell { value: Value::Number(0.0), py: Some(std::sync::Arc::new(py)), ..Cell::default() });
|
||||
let area = RangeRef::new(anchor, CellRef::new(n as u32 - 1, n as u32 - 1));
|
||||
sh.spill_ranges.insert(anchor, area);
|
||||
for (i, c) in area.iter().enumerate().skip(1) {
|
||||
sh.spill.insert(c, data[i].clone());
|
||||
}
|
||||
let mut python = HashMap::default();
|
||||
python.insert((0, anchor), Value::Array(std::sync::Arc::new(array)));
|
||||
(wb, python)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_read_beside_a_large_spill_walks_none_of_it() {
|
||||
let (wb, python) = big_python_spill();
|
||||
let exprs = HashMap::default();
|
||||
let mut rng = 1u64;
|
||||
let mut host = PassHost::new(&wb, &exprs, &mut rng, 0.0).with_python(python);
|
||||
assert_eq!(host.cell_value(0, CellRef::new(0, 0)), Value::Number(0.0));
|
||||
// Beside the spill (column ALM, the 1,001st), over the rows it covers.
|
||||
let beside = RangeRef::parse("ALM1:ALM1000").unwrap();
|
||||
host.spill_visits = 0;
|
||||
assert!(host.range_values(0, beside).iter().all(Value::is_empty));
|
||||
assert!(host.visit_range(0, beside, &mut |_| true), "a disjoint read takes the fast path");
|
||||
assert_eq!(host.spill_visits, 0, "nothing of the spill is walked");
|
||||
// A read inside it still sees the spilled values, walking only the cells it reads.
|
||||
let inside = RangeRef::parse("B2:C3").unwrap();
|
||||
host.spill_visits = 0;
|
||||
assert_eq!(host.range_values(0, inside), [1_001.0, 1_002.0, 2_001.0, 2_002.0].map(Value::Number).to_vec());
|
||||
assert!(host.spill_visits <= 8, "{}", host.spill_visits);
|
||||
assert!(!host.visit_range(0, inside, &mut |_| true), "an overlapping read leaves the block to range_values");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn intervals_match_a_linear_scan() {
|
||||
let mut iv: Intervals<u32> = Intervals::default();
|
||||
|
||||
@@ -45,6 +45,10 @@ pub(super) struct PassHost<'a> {
|
||||
/// Suspensions so far (tests check that a wide read at the limit needs one, not one per cell).
|
||||
#[cfg(test)]
|
||||
pub(super) suspensions: usize,
|
||||
/// Spilled values looked at by range reads (tests check that a read outside a large spill
|
||||
/// looks at none of it).
|
||||
#[cfg(test)]
|
||||
pub(super) spill_visits: usize,
|
||||
/// The formulas in `cycles`.
|
||||
cycle_seen: HashSet<Key>,
|
||||
/// Iterative calculation: a circular reference reads the cell's previous value instead of
|
||||
@@ -64,6 +68,10 @@ pub(super) struct PassHost<'a> {
|
||||
stack_top: usize,
|
||||
/// Whether an evaluation ran past the stack budget since the innermost formula started.
|
||||
out_of_stack: bool,
|
||||
/// The kept results of the Python cells in the pass: a Python cell isn't evaluated, its
|
||||
/// result is laid out (an array spills from the cell, as a formula's does). Each one is in
|
||||
/// `pending` until it is laid out, so a formula that reads it first gets its new value.
|
||||
python: HashMap<Key, Value>,
|
||||
}
|
||||
|
||||
impl<'a> PassHost<'a> {
|
||||
@@ -87,6 +95,8 @@ impl<'a> PassHost<'a> {
|
||||
also: Vec::new(),
|
||||
#[cfg(test)]
|
||||
suspensions: 0,
|
||||
#[cfg(test)]
|
||||
spill_visits: 0,
|
||||
cycle_seen: HashSet::default(),
|
||||
iterate: false,
|
||||
prev: HashMap::default(),
|
||||
@@ -96,8 +106,17 @@ impl<'a> PassHost<'a> {
|
||||
taint: false,
|
||||
stack_top: 0,
|
||||
out_of_stack: false,
|
||||
python: HashMap::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Adds the dirty Python cells of the pass and their kept results: like the formulas, they
|
||||
/// are pending until laid out, so every read path calculates them before reading them.
|
||||
pub(super) fn with_python(mut self, python: HashMap<Key, Value>) -> Self {
|
||||
self.pending.extend(python.keys().copied());
|
||||
self.python = python;
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether an array result at `k` can't spill: it is in a table, or its area runs off the sheet,
|
||||
@@ -111,7 +130,10 @@ fn spill_blocked(host: &PassHost<'_>, (sheet, anchor): Key, a: &Array) -> bool {
|
||||
let own = sh.spill_ranges.get(&anchor);
|
||||
let spilled_over = |c: CellRef| {
|
||||
!own.is_some_and(|o| o.contains(c))
|
||||
&& sh.spill_ranges.iter().any(|(o, r)| *o != anchor && r.contains(c) && !host.exprs.contains_key(&(sheet, *o)))
|
||||
&& sh.spill_ranges.iter().any(|(o, r)| {
|
||||
// A formula or Python cell recalculated in this pass lays its array out again.
|
||||
*o != anchor && r.contains(c) && !host.exprs.contains_key(&(sheet, *o)) && !host.python.contains_key(&(sheet, *o))
|
||||
})
|
||||
};
|
||||
end.row >= gridcraft_core::MAX_ROWS
|
||||
|| end.col >= gridcraft_core::MAX_COLS
|
||||
@@ -135,6 +157,42 @@ pub(super) fn spills_as_before(sheet: &gridcraft_model::Sheet, anchor: CellRef,
|
||||
.all(|(i, c)| c == anchor || (!sheet.cells.has(c) && sheet.spill.get(&c) == Some(arr.data.get(i).unwrap_or(&Value::Empty))))
|
||||
}
|
||||
|
||||
/// The area an array anchored at `anchor` covers.
|
||||
fn array_area(anchor: CellRef, a: &Array) -> RangeRef {
|
||||
let end =
|
||||
CellRef::new(anchor.row.saturating_add((a.rows as u32).saturating_sub(1)), anchor.col.saturating_add((a.cols as u32).saturating_sub(1)));
|
||||
RangeRef::new(anchor, end)
|
||||
}
|
||||
|
||||
/// The spilled values `sheet` stores inside `range`, passed to `f` until it returns false. Values
|
||||
/// outside the range are not walked: with fewer rows in the range than spilled values, each row
|
||||
/// is looked up on its own; otherwise one ordered scan visits each spilled value at most once.
|
||||
/// Returns how many values were looked at.
|
||||
fn stored_spill_in<'s>(sheet: &'s gridcraft_model::Sheet, range: RangeRef, f: &mut dyn FnMut(CellRef, &'s Value) -> bool) -> usize {
|
||||
let mut seen = 0usize;
|
||||
if sheet.spill.is_empty() {
|
||||
return 0;
|
||||
}
|
||||
if (range.height() as usize) < sheet.spill.len() {
|
||||
for row in range.start.row..=range.end.row {
|
||||
for (c, v) in sheet.spill.range(CellRef::new(row, range.start.col)..=CellRef::new(row, range.end.col)) {
|
||||
seen += 1;
|
||||
if !f(*c, v) {
|
||||
return seen;
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
for (c, v) in sheet.spill.range(range.start..=range.end) {
|
||||
seen += 1;
|
||||
if range.contains(*c) && !f(*c, v) {
|
||||
return seen;
|
||||
}
|
||||
}
|
||||
}
|
||||
seen
|
||||
}
|
||||
|
||||
impl PassHost<'_> {
|
||||
/// Calculates `k` (and the formulas it reads) with bounded stack: when [`Self::compute`] would
|
||||
/// go deeper than [`MAX_NESTED`], it suspends what is in progress, the formula it needed is
|
||||
@@ -188,6 +246,9 @@ impl PassHost<'_> {
|
||||
}
|
||||
return v;
|
||||
}
|
||||
if let Some(result) = self.python.get(&k).cloned() {
|
||||
return self.python_result(k, result);
|
||||
}
|
||||
let Some(expr) = self.exprs.get(&k) else {
|
||||
return self.wb.sheet(k.0).map(|s| s.value(k.1)).unwrap_or_default();
|
||||
};
|
||||
@@ -305,6 +366,36 @@ impl PassHost<'_> {
|
||||
v
|
||||
}
|
||||
|
||||
/// A Python cell's kept result as its value: an array spills from the cell unless it is
|
||||
/// blocked, malformed or larger than [`gridcraft_model::python::MAX_RESULT_CELLS`] (only a
|
||||
/// crafted file holds one), which shows `#SPILL!`.
|
||||
fn python_result(&mut self, k: Key, result: Value) -> Value {
|
||||
self.pending.remove(&k);
|
||||
let v = match result {
|
||||
Value::Array(a) => {
|
||||
let sane = a.rows >= 1
|
||||
&& a.cols >= 1
|
||||
&& a.rows.checked_mul(a.cols).is_some_and(|n| n == a.data.len() && n <= gridcraft_model::python::MAX_RESULT_CELLS);
|
||||
if sane {
|
||||
self.spills.insert(k, a.clone());
|
||||
a.data.first().cloned().unwrap_or_default()
|
||||
} else {
|
||||
Value::Error(CellError::Spill)
|
||||
}
|
||||
}
|
||||
v => v,
|
||||
};
|
||||
let v = match self.spills.get(&k) {
|
||||
Some(a) if spill_blocked(self, k, a) => {
|
||||
self.spills.remove(&k);
|
||||
Value::Error(CellError::Spill)
|
||||
}
|
||||
_ => v,
|
||||
};
|
||||
self.results.insert(k, v.clone());
|
||||
v
|
||||
}
|
||||
|
||||
/// A formula's previous value for iterative calculation: from the last iteration, else the
|
||||
/// value it shows. Blank, and the `#CIRC!` of a loop calculated with iteration off, are 0.
|
||||
fn previous(&self, k: Key) -> Value {
|
||||
@@ -445,26 +536,29 @@ impl Host for PassHost<'_> {
|
||||
let (h, w) = (range.height() as usize, range.width() as usize);
|
||||
let mut out = vec![Value::Empty; h * w];
|
||||
let Some(sh) = self.wb.sheet(sheet) else { return out };
|
||||
// Sparse walk: only stored cells; formula cells that are (or were) dirty go through
|
||||
// `compute`, constants are copied straight from the store.
|
||||
// Sparse walk: only stored cells; formula and Python cells that are (or were) dirty go
|
||||
// through `compute`, constants are copied straight from the store.
|
||||
let mut formulas: Vec<(usize, CellRef)> = Vec::new();
|
||||
for (c, cell) in sh.cells.iter_range(range) {
|
||||
let i = (c.row - range.start.row) as usize * w + (c.col - range.start.col) as usize;
|
||||
if cell.formula.is_some() {
|
||||
if cell.formula.is_some() || cell.py.is_some() {
|
||||
formulas.push((i, c));
|
||||
} else if let Some(slot) = out.get_mut(i) {
|
||||
*slot = cell.value.clone();
|
||||
}
|
||||
}
|
||||
for (c, v) in sh.spill.range(range.start..=range.end) {
|
||||
if range.contains(*c) {
|
||||
let i = (c.row - range.start.row) as usize * w + (c.col - range.start.col) as usize;
|
||||
if let Some(slot) = out.get_mut(i)
|
||||
&& slot.is_empty()
|
||||
{
|
||||
*slot = v.clone();
|
||||
}
|
||||
let _seen = stored_spill_in(sh, range, &mut |c, v| {
|
||||
let i = (c.row - range.start.row) as usize * w + (c.col - range.start.col) as usize;
|
||||
if let Some(slot) = out.get_mut(i)
|
||||
&& slot.is_empty()
|
||||
{
|
||||
*slot = v.clone();
|
||||
}
|
||||
true
|
||||
});
|
||||
#[cfg(test)]
|
||||
{
|
||||
self.spill_visits += _seen;
|
||||
}
|
||||
for (i, c) in formulas {
|
||||
let v = self.cell_value(sheet, c);
|
||||
@@ -473,26 +567,31 @@ impl Host for PassHost<'_> {
|
||||
}
|
||||
}
|
||||
if !self.spills.is_empty() {
|
||||
let new: Vec<(usize, Value)> = self
|
||||
.spills
|
||||
.iter()
|
||||
.filter(|(a, _)| a.0 == sheet)
|
||||
.flat_map(|(a, arr)| {
|
||||
let mut v = Vec::new();
|
||||
for r in 0..arr.rows {
|
||||
for cc in 0..arr.cols {
|
||||
let c = CellRef::new(a.1.row.saturating_add(r as u32), a.1.col.saturating_add(cc as u32));
|
||||
if c != a.1 && range.contains(c) {
|
||||
v.push((
|
||||
(c.row - range.start.row) as usize * w + (c.col - range.start.col) as usize,
|
||||
arr.get(r, cc).cloned().unwrap_or_default(),
|
||||
));
|
||||
}
|
||||
// Only the part of each array inside the range is walked: a large array beside a small
|
||||
// read costs nothing.
|
||||
let mut new: Vec<(usize, Value)> = Vec::new();
|
||||
for (a, arr) in self.spills.iter().filter(|(a, _)| a.0 == sheet) {
|
||||
let area = array_area(a.1, arr);
|
||||
if !area.intersects(&range) {
|
||||
continue;
|
||||
}
|
||||
let (r0, r1) = (area.start.row.max(range.start.row), area.end.row.min(range.end.row));
|
||||
let (c0, c1) = (area.start.col.max(range.start.col), area.end.col.min(range.end.col));
|
||||
for row in r0..=r1 {
|
||||
for col in c0..=c1 {
|
||||
let c = CellRef::new(row, col);
|
||||
if c == a.1 {
|
||||
continue;
|
||||
}
|
||||
let v = arr.get((row - a.1.row) as usize, (col - a.1.col) as usize).cloned().unwrap_or_default();
|
||||
new.push(((row - range.start.row) as usize * w + (col - range.start.col) as usize, v));
|
||||
}
|
||||
v
|
||||
})
|
||||
.collect();
|
||||
}
|
||||
}
|
||||
#[cfg(test)]
|
||||
{
|
||||
self.spill_visits += new.len();
|
||||
}
|
||||
// Iterating: an array spilled by a formula that read a previous value.
|
||||
if self.iterate
|
||||
&& self.spills.iter().any(|(a, arr)| {
|
||||
@@ -516,21 +615,31 @@ impl Host for PassHost<'_> {
|
||||
fn visit_range(&mut self, sheet: usize, range: RangeRef, f: &mut dyn FnMut(&Value) -> bool) -> bool {
|
||||
let wb = self.wb;
|
||||
let Some(sh) = wb.sheet(sheet) else { return false };
|
||||
// Formula cells first, in row-major order like `range_values`: reading one calculates it
|
||||
// and the pass remembers the value, so the walk below gets the same value again without
|
||||
// this method holding any.
|
||||
// Formula and Python cells first, in row-major order like `range_values`: reading one
|
||||
// calculates it (a Python cell lays out its result) and the pass remembers the value, so
|
||||
// the walk below gets the same value again without this method holding any.
|
||||
for (c, cell) in sh.cells.iter_range(range) {
|
||||
if cell.formula.is_some() {
|
||||
if cell.formula.is_some() || cell.py.is_some() {
|
||||
let _ = self.cell_value(sheet, c);
|
||||
}
|
||||
}
|
||||
// Spilled values sit between the stored cells (and the formulas just calculated may have
|
||||
// produced some): leave such blocks to `range_values`.
|
||||
if self.spills.keys().any(|a| a.0 == sheet) || sh.spill.range(range.start..=range.end).any(|(c, _)| range.contains(*c)) {
|
||||
let arrays = self.spills.iter().any(|(a, arr)| a.0 == sheet && array_area(a.1, arr).intersects(&range));
|
||||
let mut stored = false;
|
||||
let _seen = stored_spill_in(sh, range, &mut |_, _| {
|
||||
stored = true;
|
||||
false
|
||||
});
|
||||
#[cfg(test)]
|
||||
{
|
||||
self.spill_visits += _seen;
|
||||
}
|
||||
if arrays || stored {
|
||||
return false;
|
||||
}
|
||||
for (c, cell) in sh.cells.iter_range(range) {
|
||||
let go = if cell.formula.is_some() { f(&self.cell_value(sheet, c)) } else { f(&cell.value) };
|
||||
let go = if cell.formula.is_some() || cell.py.is_some() { f(&self.cell_value(sheet, c)) } else { f(&cell.value) };
|
||||
if !go {
|
||||
break;
|
||||
}
|
||||
|
||||
@@ -2312,3 +2312,63 @@ fn loan_schedule_residue_keeps_excels_sign() {
|
||||
}
|
||||
assert_eq!(t.get("P7"), Value::Text("".into()));
|
||||
}
|
||||
|
||||
/// Makes `at` a Python cell showing `cached` (its value before this calculation), with `result`
|
||||
/// as its kept result, and returns the cell's key for `Calc::cells_changed`.
|
||||
fn plant_python(t: &mut T, at: &str, cached: Value, result: Value) -> (usize, CellRef) {
|
||||
use gridcraft_model::python::{PyCode, PyOutput};
|
||||
let output = PyOutput { value: result, type_name: "list".into(), preview: String::new(), convertible: true };
|
||||
let py = PyCode { output: Some(output), ..PyCode::new("[5, 6]") };
|
||||
let cell = Cell { value: cached, formula: None, style: gridcraft_model::StyleId::default(), py: Some(std::sync::Arc::new(py)) };
|
||||
t.wb.sheet_mut(0).unwrap().cells.set(c(at), cell);
|
||||
(0, c(at))
|
||||
}
|
||||
|
||||
fn column(values: &[f64]) -> Value {
|
||||
let data = values.iter().map(|v| Value::Number(*v)).collect();
|
||||
Value::Array(std::sync::Arc::new(gridcraft_core::Array::new(values.len(), 1, data).unwrap()))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn formulas_read_a_dirty_python_cell_after_it_is_laid_out() {
|
||||
// B1's result can't spill (B2 holds a value), so it is #SPILL!. A1, A2 and A3 come before
|
||||
// it in the pass; each must lay it out first rather than read its old value, 1.
|
||||
let build = |t: &mut T| {
|
||||
t.set("B2", "10");
|
||||
t.set("B3", "100");
|
||||
t.set("A1", "=B1");
|
||||
t.set("A2", "=MAX(B1:B3)");
|
||||
t.set("A3", "=SUM(B1:B3)");
|
||||
t.set("A4", "=COUNTA(B1:B3)");
|
||||
t.set("A5", "=SUMPRODUCT(--ISERROR(B1:B3))");
|
||||
};
|
||||
let check = |t: &T, how: &str| {
|
||||
assert_eq!(t.get("B1"), Value::Error(CellError::Spill), "{how}");
|
||||
assert_eq!(t.get("A1"), Value::Error(CellError::Spill), "{how}: a reference");
|
||||
assert_eq!(t.get("A2"), Value::Error(CellError::Spill), "{how}: a range read");
|
||||
assert_eq!(t.get("A3"), Value::Error(CellError::Spill), "{how}: a range walked cell by cell");
|
||||
assert_eq!(t.num("A4"), 3.0, "{how}");
|
||||
assert_eq!(t.num("A5"), 1.0, "{how}: the error is seen in the range");
|
||||
};
|
||||
|
||||
// A full recalculation.
|
||||
let mut t = T::new();
|
||||
build(&mut t);
|
||||
plant_python(&mut t, "B1", Value::Number(1.0), column(&[5.0, 6.0]));
|
||||
t.calc.recalc_all(&mut t.wb);
|
||||
check(&t, "full");
|
||||
|
||||
// An incremental one: B1 becomes a Python cell after its readers.
|
||||
let mut t = T::new();
|
||||
t.set("B1", "1");
|
||||
build(&mut t);
|
||||
assert_eq!((t.num("A1"), t.num("A3")), (1.0, 111.0));
|
||||
let k = plant_python(&mut t, "B1", Value::Number(1.0), column(&[5.0, 6.0]));
|
||||
t.calc.cells_changed(&mut t.wb, &[k]);
|
||||
check(&t, "incremental");
|
||||
|
||||
// Unblocked, the readers see the new array.
|
||||
t.set("B2", "");
|
||||
assert_eq!((t.num("B1"), t.num("B2")), (5.0, 6.0));
|
||||
assert_eq!((t.num("A1"), t.num("A2"), t.num("A3")), (5.0, 100.0, 111.0));
|
||||
}
|
||||
|
||||
@@ -21,10 +21,18 @@ pub enum CellError {
|
||||
Calc,
|
||||
/// Circular reference that could not be resolved (shown as 0 by Excel; we keep it explicit).
|
||||
Circ,
|
||||
/// A Python cell's code raised an error, or Python stopped.
|
||||
Python,
|
||||
/// A Python cell is waiting to run.
|
||||
Busy,
|
||||
/// A Python cell ran longer than the time limit.
|
||||
Timeout,
|
||||
/// Python can't run here: the workbook's content isn't enabled, or this build has no Python.
|
||||
Blocked,
|
||||
}
|
||||
|
||||
impl CellError {
|
||||
pub const ALL: [CellError; 10] = [
|
||||
pub const ALL: [CellError; 14] = [
|
||||
CellError::Null,
|
||||
CellError::Div0,
|
||||
CellError::Value,
|
||||
@@ -35,6 +43,10 @@ impl CellError {
|
||||
CellError::GettingData,
|
||||
CellError::Spill,
|
||||
CellError::Calc,
|
||||
CellError::Python,
|
||||
CellError::Busy,
|
||||
CellError::Timeout,
|
||||
CellError::Blocked,
|
||||
];
|
||||
pub fn as_str(&self) -> &'static str {
|
||||
match self {
|
||||
@@ -49,9 +61,16 @@ impl CellError {
|
||||
CellError::Spill => "#SPILL!",
|
||||
CellError::Calc => "#CALC!",
|
||||
CellError::Circ => "#CIRC!",
|
||||
CellError::Python => "#PYTHON!",
|
||||
CellError::Busy => "#BUSY!",
|
||||
CellError::Timeout => "#TIMEOUT!",
|
||||
CellError::Blocked => "#BLOCKED!",
|
||||
}
|
||||
}
|
||||
/// `ERROR.TYPE` number.
|
||||
/// `ERROR.TYPE` number. Microsoft documents 1 to 8; 9 (#SPILL!), 11 (#BLOCKED!) and 14 (#CALC!)
|
||||
/// are the numbers Excel returns. Microsoft documents no number for #PYTHON!, #BUSY! or
|
||||
/// #TIMEOUT!: 19, 20 and 21 are GridCraft's own choice, after the numbers Excel is known to use,
|
||||
/// and may change if Excel's numbers become known.
|
||||
pub fn code(&self) -> u32 {
|
||||
match self {
|
||||
CellError::Null => 1,
|
||||
@@ -65,6 +84,10 @@ impl CellError {
|
||||
CellError::Spill => 9,
|
||||
CellError::Calc => 14,
|
||||
CellError::Circ => 15,
|
||||
CellError::Blocked => 11,
|
||||
CellError::Python => 19,
|
||||
CellError::Busy => 20,
|
||||
CellError::Timeout => 21,
|
||||
}
|
||||
}
|
||||
pub fn parse(s: &str) -> Option<CellError> {
|
||||
|
||||
@@ -16,6 +16,7 @@ pub mod inspect;
|
||||
pub mod macros;
|
||||
pub mod pivot;
|
||||
pub mod print;
|
||||
pub mod python;
|
||||
pub mod review;
|
||||
pub mod sheet;
|
||||
pub mod slicer;
|
||||
@@ -176,6 +177,7 @@ pub fn command_specs() -> &'static [CommandSpec] {
|
||||
v.extend(spelling::specs());
|
||||
v.extend(solver::specs());
|
||||
v.extend(pivot::specs());
|
||||
v.extend(python::specs());
|
||||
v.extend(slicer::specs());
|
||||
v.extend(whatif::specs());
|
||||
v.extend(forecast::specs());
|
||||
@@ -522,7 +524,7 @@ fn check_arrays(old: &Workbook, new: &Workbook) -> Result<()> {
|
||||
continue;
|
||||
}
|
||||
for c in sh.cells.diff(&after.cells) {
|
||||
let filled = after.cell(c).is_some_and(|x| x.formula.is_some() || !x.value.is_empty());
|
||||
let filled = after.cell(c).is_some_and(gridcraft_model::Cell::has_content);
|
||||
if !filled || !sh.spill.contains_key(&c) || same_content(sh.cell(c), after.cell(c)) {
|
||||
continue;
|
||||
}
|
||||
@@ -538,13 +540,18 @@ fn check_arrays(old: &Workbook, new: &Workbook) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Two cells hold the same value or formula (formats aside).
|
||||
/// Two cells hold the same value, formula or Python code (formats aside; a Python cell's result
|
||||
/// is not something the person edits).
|
||||
fn same_content(a: Option<&gridcraft_model::Cell>, b: Option<&gridcraft_model::Cell>) -> bool {
|
||||
let empty = |c: Option<&gridcraft_model::Cell>| c.is_none_or(|c| c.formula.is_none() && c.value.is_empty());
|
||||
let empty = |c: Option<&gridcraft_model::Cell>| c.is_none_or(|c| !c.has_content());
|
||||
match (a, b) {
|
||||
(Some(a), Some(b)) => match (&a.formula, &b.formula) {
|
||||
(Some(x), Some(y)) => x.text == y.text && x.array == y.array,
|
||||
(None, None) => a.value == b.value,
|
||||
(None, None) => match (&a.py, &b.py) {
|
||||
(Some(x), Some(y)) => x.code == y.code && x.object == y.object,
|
||||
(None, None) => a.value == b.value,
|
||||
_ => false,
|
||||
},
|
||||
_ => false,
|
||||
},
|
||||
_ => empty(a) && empty(b),
|
||||
|
||||
@@ -1788,7 +1788,7 @@ fn write_report(s: &mut Session, p: &Json, tool: &str, g: Grid) -> Result<Json>
|
||||
{
|
||||
for row in rect.start.row..=rect.end.row {
|
||||
for (c, cell) in sh.cells.row(row, rect.start.col, rect.end.col) {
|
||||
if !cell.value.is_empty() || cell.formula.is_some() {
|
||||
if cell.has_content() {
|
||||
occupied.push(CellRef::new(row, c));
|
||||
}
|
||||
}
|
||||
@@ -1816,13 +1816,13 @@ fn write_report(s: &mut Session, p: &Json, tool: &str, g: Grid) -> Result<Json>
|
||||
let sh = cx.sheet_mut(si)?;
|
||||
for c in &occupied {
|
||||
let style = sh.style_id(*c);
|
||||
sh.set_cell(*c, Cell { value: Value::Empty, formula: None, style });
|
||||
sh.set_cell(*c, Cell { value: Value::Empty, formula: None, style, py: None });
|
||||
}
|
||||
let mut touched = occupied.clone();
|
||||
for ((r, c), v) in &g.cells {
|
||||
let cell = CellRef::new(at.row + r, at.col + c);
|
||||
let style = pct_styles.get(&cell).copied().unwrap_or_else(|| sh.style_id(cell));
|
||||
sh.set_cell(cell, Cell { value: v.clone(), formula: None, style });
|
||||
sh.set_cell(cell, Cell { value: v.clone(), formula: None, style, py: None });
|
||||
touched.push(cell);
|
||||
}
|
||||
for c in touched {
|
||||
|
||||
@@ -146,7 +146,16 @@ pub(super) fn text_to_columns(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
}
|
||||
}
|
||||
for (c, t) in out {
|
||||
let cell = crate::cmd::edit::input_to_cell(&t, sheet, c, &mut cx.wb).unwrap_or(None);
|
||||
let cell = match crate::cmd::edit::input_to_cell(&t, sheet, c, &mut cx.wb) {
|
||||
Ok(cell) => cell,
|
||||
// `=PY(…)` doesn't make a Python cell (that takes python.set): the piece is kept
|
||||
// as text rather than lost.
|
||||
Err(_) if gridcraft_model::python::is_py_formula(&t) => {
|
||||
let style = cx.wb.sheet(sheet).map(|sh| sh.style_id(c)).unwrap_or_default();
|
||||
Some(Cell { value: gridcraft_core::Value::text(t.as_str()), formula: None, style, py: None })
|
||||
}
|
||||
Err(_) => None,
|
||||
};
|
||||
let shm = cx.sheet_mut(sheet)?;
|
||||
match cell {
|
||||
Some(x) => shm.set_cell(c, x),
|
||||
|
||||
@@ -35,7 +35,7 @@ use go_to::go_to_special;
|
||||
#[cfg(test)]
|
||||
use go_to::{GO_TO_MAX_AREAS, GO_TO_RECENT, reference_parts};
|
||||
use paste_special::paste_special;
|
||||
pub(crate) use paste_special::{rewrite_all_formulas, rewrite_cell_links};
|
||||
pub(crate) use paste_special::{rewrite_all_formulas, rewrite_cell_links, rewrite_python_refs};
|
||||
use selection::{select_all, select_col, select_region, select_row, selection_move, selection_next, selection_set};
|
||||
|
||||
pub fn specs() -> Vec<CommandSpec> {
|
||||
@@ -132,7 +132,7 @@ pub fn specs() -> Vec<CommandSpec> {
|
||||
"Replace…",
|
||||
["Home", "Editing", "Find & Select"],
|
||||
Some("Ctrl+H"),
|
||||
"{what, with, matchCase?, wholeCell?, all?: true, within?: sheet|workbook}",
|
||||
"{what, with, matchCase?, wholeCell?, all?: true, within?: sheet|workbook} → {replaced, skipped?} (in a Python cell the code is edited; skipped counts Python cells whose code would no longer fit)",
|
||||
has_doc,
|
||||
replace
|
||||
),
|
||||
@@ -267,6 +267,7 @@ fn clear(s: &mut Session, p: &Json, what: &str) -> Result<Json> {
|
||||
for (_, cell) in cells.iter_mut() {
|
||||
let checkbox = cx.wb.styles.get(cell.style).num_fmt.as_str() == "checkbox";
|
||||
cell.formula = None;
|
||||
cell.py = None;
|
||||
if checkbox && uncheck_only {
|
||||
cell.value = Value::Bool(false);
|
||||
} else {
|
||||
|
||||
@@ -5,6 +5,9 @@ use super::{
|
||||
|
||||
// ---------------------------------------------------------------- entry
|
||||
|
||||
/// Typing `=PY(` doesn't make a Python cell; this says what does.
|
||||
pub(crate) const PY_FORMULA: &str = "Python cells are made with the python.set command (the code as `code`), not by typing =PY(…).";
|
||||
|
||||
/// Whether typing `input` into a cell enters a formula: `=…`, or `+…`/`-…` followed by a name or
|
||||
/// a parenthesis (not a number), as Excel takes `+SUM(1)`.
|
||||
pub(crate) fn is_formula_input(input: &str) -> bool {
|
||||
@@ -21,14 +24,18 @@ pub(crate) fn is_formula_input(input: &str) -> bool {
|
||||
pub(crate) fn input_to_cell(input: &str, sheet: usize, at: CellRef, wb: &mut gridcraft_model::Workbook) -> Result<Option<Cell>> {
|
||||
let style = wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default();
|
||||
if input.is_empty() {
|
||||
let c = Cell { value: Value::Empty, formula: None, style };
|
||||
let c = Cell { value: Value::Empty, formula: None, style, py: None };
|
||||
return Ok(if c.is_blank() { None } else { Some(c) });
|
||||
}
|
||||
let fmt_is_text = wb.styles.get(style).num_fmt.as_str() == "@";
|
||||
if fmt_is_text {
|
||||
return Ok(Some(Cell { value: Value::text(input), formula: None, style }));
|
||||
return Ok(Some(Cell { value: Value::text(input), formula: None, style, py: None }));
|
||||
}
|
||||
if is_formula_input(input) {
|
||||
let is_formula = is_formula_input(input);
|
||||
if is_formula && gridcraft_model::python::is_py_formula(input) {
|
||||
return Err(EngineError::Other(PY_FORMULA.into()));
|
||||
}
|
||||
if is_formula {
|
||||
let body = input.strip_prefix('=').unwrap_or(input);
|
||||
let expr = wb.name_call_case(parse_typed_formula(body)?);
|
||||
check_arg_counts(&expr)?;
|
||||
@@ -79,7 +86,7 @@ pub(crate) fn input_to_cell(input: &str, sheet: usize, at: CellRef, wb: &mut gri
|
||||
if input.contains('\n') && !wb.styles.get(style).align.wrap {
|
||||
style = wb.styles.derive(style, |s| s.align.wrap = true);
|
||||
}
|
||||
Ok(Some(Cell { value: parsed.value, formula: None, style }))
|
||||
Ok(Some(Cell { value: parsed.value, formula: None, style, py: None }))
|
||||
}
|
||||
|
||||
/// Parses a formula as it is typed (without the `=`). Excel closes missing parentheses for you,
|
||||
@@ -147,7 +154,7 @@ pub(super) fn cell_set(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
// The array takes the whole range: other cells in it lose their contents.
|
||||
for c in range.iter().skip(1).take(1_000_000) {
|
||||
if let Some(old) = sh.cells.get(c).filter(|x| !x.value.is_empty() || x.formula.is_some()) {
|
||||
if let Some(old) = sh.cells.get(c).filter(|x| x.has_content()) {
|
||||
let style = old.style;
|
||||
sh.set_cell(c, Cell { style, ..Cell::default() });
|
||||
}
|
||||
@@ -239,10 +246,14 @@ pub(super) fn range_set_values(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
value: Value::number(n.as_f64().unwrap_or(0.0)),
|
||||
formula: None,
|
||||
style: cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default(),
|
||||
py: None,
|
||||
}),
|
||||
Json::Bool(b) => Some(Cell {
|
||||
value: Value::Bool(*b),
|
||||
formula: None,
|
||||
style: cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default(),
|
||||
py: None,
|
||||
}),
|
||||
Json::Bool(b) => {
|
||||
Some(Cell { value: Value::Bool(*b), formula: None, style: cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default() })
|
||||
}
|
||||
// Text as given: no date or number reading, and a leading `'` is part of the
|
||||
// text rather than the marker that makes it text. A formula is still a
|
||||
// formula, which is what the ticket asks for and what `=` means in a cell.
|
||||
@@ -258,7 +269,7 @@ pub(super) fn range_set_values(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
if text.contains('\n') && !cx.wb.styles.get(style).align.wrap {
|
||||
style = cx.wb.styles.derive(style, |s| s.align.wrap = true);
|
||||
}
|
||||
Some(Cell { value: Value::Text(text.as_str().into()), formula: None, style })
|
||||
Some(Cell { value: Value::Text(text.as_str().into()), formula: None, style, py: None })
|
||||
}
|
||||
}
|
||||
_ => input_to_cell(&input, sheet, at, &mut cx.wb)?,
|
||||
|
||||
@@ -92,7 +92,7 @@ fn slide_together(wb: &gridcraft_model::Workbook, sheet: usize, areas: &[RangeRe
|
||||
for c in cells {
|
||||
let Some(to) = c.offset(dr, dc) else { continue };
|
||||
let style = src.cell(c).map(|x| x.style).unwrap_or_default();
|
||||
dst.set_cell(to, Cell { value: src.value(c), formula: None, style });
|
||||
dst.set_cell(to, Cell { value: src.value(c), formula: None, style, py: None });
|
||||
if let Some(picture) = src.cell_pictures.get(&c) {
|
||||
dst.cell_pictures.insert(to, picture.clone());
|
||||
}
|
||||
|
||||
@@ -115,6 +115,7 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let all = bool_param(p, "all").unwrap_or(true);
|
||||
let workbook = str_param(p, "within") == Some("workbook");
|
||||
let active = s.doc()?.selection.active;
|
||||
let mut skipped = 0usize;
|
||||
let n = edit(s, |cx| {
|
||||
let sheets: Vec<usize> = if workbook { (0..cx.wb.sheets.len()).collect() } else { vec![cx.wb.active_sheet()] };
|
||||
let mut n = 0;
|
||||
@@ -135,6 +136,30 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
} else {
|
||||
replace_ci(&text, &what, &with)
|
||||
};
|
||||
// A Python cell's text is its code: the replacement edits the code and the cell
|
||||
// stays a Python cell. Code that would no longer fit in a PY formula is left as it was.
|
||||
let old_py = cx.wb.sheet(si).and_then(|sh| sh.cell(c)).and_then(|x| x.py.clone());
|
||||
if let Some(old) = old_py {
|
||||
let py = gridcraft_model::PyCode { object: old.object, ..gridcraft_model::PyCode::new(&new) };
|
||||
if py.code == old.code {
|
||||
continue;
|
||||
}
|
||||
if !py.fits() {
|
||||
skipped += 1;
|
||||
continue;
|
||||
}
|
||||
// Its old spill is left to the recalculation, which removes it and
|
||||
// recalculates what read it.
|
||||
let sh = cx.sheet_mut(si)?;
|
||||
let style = sh.style_id(c);
|
||||
sh.set_cell(
|
||||
c,
|
||||
Cell { value: Value::Error(gridcraft_core::CellError::Blocked), formula: None, style, py: Some(std::sync::Arc::new(py)) },
|
||||
);
|
||||
cx.touch(si, c);
|
||||
n += 1;
|
||||
continue;
|
||||
}
|
||||
let cell = input_to_cell(&new, si, c, &mut cx.wb).unwrap_or_else(|_| Some(Cell::value(Value::text(new.as_str()))));
|
||||
let sh = cx.sheet_mut(si)?;
|
||||
match cell {
|
||||
@@ -149,6 +174,9 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
}
|
||||
Ok(n)
|
||||
})?;
|
||||
if skipped > 0 {
|
||||
return Ok(json!({"replaced": n, "skipped": skipped}));
|
||||
}
|
||||
Ok(json!({"replaced": n}))
|
||||
}
|
||||
|
||||
|
||||
@@ -174,8 +174,8 @@ pub(super) fn go_to_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let v = sheet.value(c);
|
||||
let ok = match kind {
|
||||
"blanks" => v.is_empty(),
|
||||
"constants" => cell.is_some_and(|x| x.formula.is_none() && !x.value.is_empty()),
|
||||
"formulas" => cell.is_some_and(|x| x.formula.is_some()),
|
||||
"constants" => cell.is_some_and(|x| x.formula.is_none() && x.py.is_none() && !x.value.is_empty()),
|
||||
"formulas" => cell.is_some_and(|x| x.formula.is_some() || x.py.is_some()),
|
||||
"comments" | "notes" => sheet.comments.contains_key(&c),
|
||||
"errors" => v.is_error(),
|
||||
"numbers" => v.is_number(),
|
||||
|
||||
@@ -145,6 +145,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
}
|
||||
moved_formulas = taken;
|
||||
}
|
||||
let dest_name = cx.wb.sheet(dest_sheet).map(|sh| sh.name.clone()).unwrap_or_default();
|
||||
for (tr, tc, r, c) in tiled_cells(src.height(), src.width(), tiles_r, tiles_c) {
|
||||
let sc = CellRef::new(src.start.row + r, src.start.col + c);
|
||||
let (dr, dc) = if transpose { (c, r) } else { (r, c) };
|
||||
@@ -152,7 +153,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let src_cell = if cut_move { moved_formulas.iter().find(|(k, _)| *k == sc).map(|(_, v)| v.clone()) } else { src_sheet.cell(sc).cloned() };
|
||||
let src_val = src_sheet.value(sc);
|
||||
let src_picture = if cut_move { moved_pictures.get(&sc) } else { src_sheet.cell_pictures.get(&sc) }.cloned();
|
||||
if skip_blanks && src_picture.is_none() && src_val.is_empty() && src_cell.as_ref().is_none_or(|x| x.formula.is_none()) {
|
||||
if skip_blanks && src_picture.is_none() && src_val.is_empty() && src_cell.as_ref().is_none_or(|x| x.formula.is_none() && x.py.is_none()) {
|
||||
continue;
|
||||
}
|
||||
let old = cx.wb.sheet(dest_sheet).and_then(|sh| sh.cell(dest)).cloned().unwrap_or_default();
|
||||
@@ -186,6 +187,33 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let moved = gridcraft_formula::adjust::shift_relative(e, dest.row as i64 - sc.row as i64, dest.col as i64 - sc.col as i64);
|
||||
Some(std::sync::Arc::new(Formula { dynamic: f.dynamic, ..Formula::from_expr(moved) }))
|
||||
};
|
||||
// A Python cell's code travels with it. Cut to another sheet, it keeps reading the
|
||||
// cells its `xl()` references named on its original sheet, as a moved formula does:
|
||||
// they are qualified with that sheet before the workbook-wide move adjustment below.
|
||||
let moved_py = |cell: &Cell| -> Result<Option<std::sync::Arc<gridcraft_model::PyCode>>> {
|
||||
let Some(py) = cell.py.clone() else { return Ok(None) };
|
||||
if !cut_move || dest_sheet == clip.sheet {
|
||||
return Ok(Some(py));
|
||||
}
|
||||
let qualified = gridcraft_model::python::rewrite_xl(&py.code, |reference| {
|
||||
let e = gridcraft_formula::parse(reference).ok()?.map(&mut |x| match x {
|
||||
gridcraft_formula::Expr::Ref(mut r) if r.sheet == gridcraft_formula::SheetSel::Current => {
|
||||
r.sheet = gridcraft_formula::SheetSel::Named(src_sheet_name.clone());
|
||||
gridcraft_formula::Expr::Ref(r)
|
||||
}
|
||||
other => other,
|
||||
});
|
||||
Some(gridcraft_formula::print(&e))
|
||||
});
|
||||
Ok(Some(match qualified {
|
||||
Some(code) => {
|
||||
let moved = gridcraft_model::PyCode { code, ..(*py).clone() };
|
||||
check_python_fits(&moved, &dest_name, dest)?;
|
||||
std::sync::Arc::new(moved)
|
||||
}
|
||||
None => py,
|
||||
}))
|
||||
};
|
||||
if link {
|
||||
let sheet_prefix = if dest_sheet == clip.sheet && same_doc {
|
||||
String::new()
|
||||
@@ -194,6 +222,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
};
|
||||
new.formula = Some(std::sync::Arc::new(Formula::new(&format!("={sheet_prefix}{}", sc.a1()))));
|
||||
new.value = Value::Empty;
|
||||
new.py = None;
|
||||
new_picture = None;
|
||||
} else {
|
||||
let sc_cell = src_cell.clone().unwrap_or_default();
|
||||
@@ -202,6 +231,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
new_picture = src_picture.clone();
|
||||
new.formula = shift(&sc_cell);
|
||||
new.value = if new.formula.is_some() { Value::Empty } else { sc_cell.value.clone() };
|
||||
new.py = moved_py(&sc_cell)?;
|
||||
let st = map_style(sc_cell.style, &mut cx.wb);
|
||||
new.style = if what == What::AllExceptBorders {
|
||||
let borders = cx.wb.styles.get(old.style).borders;
|
||||
@@ -214,6 +244,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
new_picture = src_picture.clone();
|
||||
new.formula = shift(&sc_cell);
|
||||
new.value = if new.formula.is_some() { Value::Empty } else { sc_cell.value.clone() };
|
||||
new.py = moved_py(&sc_cell)?;
|
||||
if what == What::FormulasAndNumberFormats {
|
||||
let nf = src_wb.styles.get(sc_cell.style).num_fmt.clone();
|
||||
new.style = cx.wb.styles.derive(old.style, |s| s.num_fmt = nf);
|
||||
@@ -222,6 +253,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
What::Values | What::ValuesAndNumberFormats => {
|
||||
new_picture = src_picture.clone();
|
||||
new.formula = None;
|
||||
new.py = None;
|
||||
new.value = src_val.clone();
|
||||
if what == What::ValuesAndNumberFormats {
|
||||
let nf = src_wb.styles.get(sc_cell.style).num_fmt.clone();
|
||||
@@ -232,7 +264,11 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
What::Comments | What::Validation | What::ColumnWidths => {}
|
||||
}
|
||||
// Operations combine numbers with what is already there.
|
||||
if op != "none" && matches!(what, What::All | What::Values | What::ValuesAndNumberFormats | What::Formulas) && new.formula.is_none() {
|
||||
if op != "none"
|
||||
&& matches!(what, What::All | What::Values | What::ValuesAndNumberFormats | What::Formulas)
|
||||
&& new.formula.is_none()
|
||||
&& new.py.is_none()
|
||||
{
|
||||
if old_picture.is_some() || new_picture.is_some() {
|
||||
new_picture = None;
|
||||
new.value = Value::Error(gridcraft_core::CellError::Value);
|
||||
@@ -292,7 +328,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
} else {
|
||||
gridcraft_formula::adjust::Edit::MoveToSheet { from: src, to_sheet: dest_name, to_row: at.row, to_col: at.col }
|
||||
};
|
||||
rewrite_all_formulas(&mut cx.wb, &src_sheet_name, &e);
|
||||
rewrite_all_formulas(&mut cx.wb, &src_sheet_name, &e)?;
|
||||
cx.structural = true;
|
||||
}
|
||||
let end = at.offset_clamped((tiles_r * h) as i64 - 1, (tiles_c * w) as i64 - 1);
|
||||
@@ -316,7 +352,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
}
|
||||
|
||||
/// Applies a reference adjustment to every cell formula and defined name in the workbook.
|
||||
pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: &str, e: &gridcraft_formula::adjust::Edit) {
|
||||
pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: &str, e: &gridcraft_formula::adjust::Edit) -> Result<()> {
|
||||
use gridcraft_formula::adjust::Edit;
|
||||
// Rows and columns inserted or deleted on the target sheet move its legacy array ranges too
|
||||
// (a cut moves its arrays itself).
|
||||
@@ -356,6 +392,7 @@ pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: &
|
||||
}
|
||||
}
|
||||
rewrite_cell_links(wb, |host, expr| gridcraft_formula::adjust::adjust(expr, host, target, e));
|
||||
rewrite_python_refs(wb, &|host, expr| gridcraft_formula::adjust::adjust(expr, host, target, e))
|
||||
}
|
||||
|
||||
/// Runs `f` over every reference a floating object holds (a chart's series values, categories,
|
||||
@@ -385,3 +422,78 @@ pub(crate) fn rewrite_cell_links(wb: &mut gridcraft_model::Workbook, f: impl Fn(
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Rewrites the `xl("…")` literals in every Python cell with `f`, which maps a reference held on
|
||||
/// sheet `host` to its new form. Every reference-moving edit (insert and delete of lines or cells,
|
||||
/// cut-move, sheet rename and delete) runs it, as it adjusts formulas. A literal is only replaced
|
||||
/// when the reference it names changed, and text that doesn't parse is kept. A cell's cached
|
||||
/// result is kept while its references still name the same data (a rename, a move); a reference
|
||||
/// the edit deleted (`#REF!`), or one that grew or shrank (rows or columns inserted or deleted
|
||||
/// inside it), leaves the result stale, so the cell is new code (`#BLOCKED!`) and the
|
||||
/// recalculation removes its spill. A rewrite that would
|
||||
/// make a cell's code longer than one cell holds (a longer sheet name, `Z1` becoming `AA1`) refuses
|
||||
/// the whole edit, which then changes nothing: see [`check_python_fits`].
|
||||
pub(crate) fn rewrite_python_refs(
|
||||
wb: &mut gridcraft_model::Workbook,
|
||||
f: &impl Fn(&str, gridcraft_formula::Expr) -> gridcraft_formula::Expr,
|
||||
) -> Result<()> {
|
||||
for si in 0..wb.sheets.len() {
|
||||
let Some(sh) = wb.sheets.get(si) else { continue };
|
||||
let host = sh.name.clone();
|
||||
let mut new_code: Vec<(CellRef, String, bool)> = Vec::new();
|
||||
for (c, cell) in sh.cells.iter() {
|
||||
let Some(py) = &cell.py else { continue };
|
||||
let mut lost = false;
|
||||
let changed = gridcraft_model::python::rewrite_xl(&py.code, |reference| {
|
||||
let expr = gridcraft_formula::parse(reference).ok()?;
|
||||
let before = gridcraft_formula::print(&expr);
|
||||
let after = gridcraft_formula::print(&f(&host, expr));
|
||||
lost |= (after.contains("#REF!") && !before.contains("#REF!")) || area_shape(&before) != area_shape(&after);
|
||||
(after != before).then_some(after)
|
||||
});
|
||||
if let Some(code) = changed {
|
||||
check_python_fits(&gridcraft_model::PyCode { code: code.clone(), object: py.object, ..Default::default() }, &host, c)?;
|
||||
new_code.push((c, code, lost));
|
||||
}
|
||||
}
|
||||
if new_code.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let Some(sh) = wb.sheet_mut(si) else { continue };
|
||||
for (c, code, lost) in new_code {
|
||||
if let Some(cell) = sh.cells.get_mut(c)
|
||||
&& let Some(py) = cell.py.as_mut()
|
||||
{
|
||||
let py = std::sync::Arc::make_mut(py);
|
||||
py.code = code;
|
||||
if lost {
|
||||
py.output = None;
|
||||
cell.value = Value::Error(gridcraft_core::CellError::Blocked);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The rows and columns a printed reference covers (`A1:B3`, `Sheet2!$A$1`, `C:E`), or `None`
|
||||
/// for any other form (a name, an error).
|
||||
fn area_shape(reference: &str) -> Option<(u32, u32)> {
|
||||
let r = RangeRef::parse(reference.rsplit('!').next()?)?;
|
||||
Some((r.height(), r.width()))
|
||||
}
|
||||
|
||||
/// Refuses Python code an edit has made longer than one cell holds: the limit `python.set`
|
||||
/// enforces and opening a file applies (code over it is dropped to its value on open). The edit
|
||||
/// is refused rather than made, so no code is lost and none keeps a reference the edit moved.
|
||||
pub(crate) fn check_python_fits(py: &gridcraft_model::PyCode, sheet: &str, at: CellRef) -> Result<()> {
|
||||
if py.fits() {
|
||||
return Ok(());
|
||||
}
|
||||
Err(EngineError::Other(format!(
|
||||
"The edit would make the Python code in {}!{} longer than Excel's {}-character formula limit, so nothing was changed. Shorten the code first.",
|
||||
gridcraft_formula::quote_sheet(sheet),
|
||||
at.a1(),
|
||||
gridcraft_model::python::MAX_FORMULA_CHARS
|
||||
)))
|
||||
}
|
||||
|
||||
@@ -51,6 +51,7 @@ pub(super) fn toggle_checkbox(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let sh = cx.sheet_mut(si)?;
|
||||
if let Some(cell) = sh.cells.get_mut(c)
|
||||
&& cell.formula.is_none()
|
||||
&& cell.py.is_none()
|
||||
{
|
||||
// An empty checkbox cell counts as FALSE.
|
||||
let next = match cell.value {
|
||||
|
||||
@@ -315,7 +315,7 @@ fn forecast_sheet(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let vstyle = cx.wb.styles.derive(StyleId::DEFAULT, |st| st.num_fmt = vfmt.clone());
|
||||
let mut sheet = Sheet::new(sheet_name.clone());
|
||||
let put = |sheet: &mut Sheet, row: u32, col: u32, v: Value, style: StyleId| {
|
||||
sheet.set_cell(CellRef::new(row, col), Cell { value: v, formula: None, style });
|
||||
sheet.set_cell(CellRef::new(row, col), Cell { value: v, formula: None, style, py: None });
|
||||
};
|
||||
let formula = |sheet: &mut Sheet, row: u32, col: u32, text: String, style: StyleId| {
|
||||
let mut cell = Cell::formula(Formula::new(&text));
|
||||
|
||||
@@ -656,8 +656,7 @@ fn merge(s: &mut Session, p: &Json, how: &str) -> Result<Json> {
|
||||
let others: Vec<CellRef> = sh.cells.iter_range(m).map(|(c, _)| c).filter(|c| *c != m.start).collect();
|
||||
for c in others {
|
||||
if let Some(mut cell) = sh.cells.get(c).cloned() {
|
||||
cell.value = gridcraft_core::Value::Empty;
|
||||
cell.formula = None;
|
||||
cell.clear_content();
|
||||
sh.set_cell(c, cell);
|
||||
}
|
||||
touched.push((sheet, c));
|
||||
@@ -674,8 +673,7 @@ fn merge(s: &mut Session, p: &Json, how: &str) -> Result<Json> {
|
||||
let others: Vec<CellRef> = sh.cells.iter_range(*r).map(|(c, _)| c).filter(|c| *c != r.start).collect();
|
||||
for c in others {
|
||||
if let Some(mut cell) = sh.cells.get(c).cloned() {
|
||||
cell.value = gridcraft_core::Value::Empty;
|
||||
cell.formula = None;
|
||||
cell.clear_content();
|
||||
sh.set_cell(c, cell);
|
||||
}
|
||||
touched.push((sheet, c));
|
||||
@@ -692,7 +690,10 @@ fn merge(s: &mut Session, p: &Json, how: &str) -> Result<Json> {
|
||||
let blocked = sh
|
||||
.cells
|
||||
.iter_range(reach)
|
||||
.filter(|(_, x)| x.formula.is_some() && x.value == gridcraft_core::Value::Error(gridcraft_core::CellError::Spill));
|
||||
// A Python cell's result is blocked and unblocked like a formula's.
|
||||
.filter(|(_, x)| {
|
||||
(x.formula.is_some() || x.py.is_some()) && x.value == gridcraft_core::Value::Error(gridcraft_core::CellError::Spill)
|
||||
});
|
||||
touched.extend(blocked.map(|(c, _)| (sheet, c)));
|
||||
}
|
||||
cx.changed.extend(touched);
|
||||
|
||||
@@ -635,7 +635,7 @@ fn paste_name(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
let style = sh.style_id(c);
|
||||
// The second column holds the refers-to text as a text value, with no formula.
|
||||
sh.set_cell(c, Cell { value: Value::text(text.as_str()), formula: None, style });
|
||||
sh.set_cell(c, Cell { value: Value::text(text.as_str()), formula: None, style, py: None });
|
||||
cx.touch(sheet, c);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -316,7 +316,7 @@ fn insert_table(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
// The header row pushes the data down one row: never off the end of the sheet.
|
||||
let last = RangeRef::new(CellRef::new(MAX_ROWS - 1, r.start.col), CellRef::new(MAX_ROWS - 1, r.end.col));
|
||||
if sh.cells.iter_range(last).any(|(_, x)| x.formula.is_some() || !x.value.is_empty()) {
|
||||
if sh.cells.iter_range(last).any(|(_, x)| x.has_content()) {
|
||||
return Err(EngineError::Other("To prevent possible loss of data, cells can't be pushed off the end of the worksheet.".into()));
|
||||
}
|
||||
sh.cells.shift_rows_in_cols(r.start.col, r.end.col, r.start.row, 1);
|
||||
@@ -325,7 +325,7 @@ fn insert_table(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let e = Edit::InsertCells { axis: Axis::Rows, range: RangeRef::new(r.start, CellRef::new(r.start.row, r.end.col)) };
|
||||
shift_cell_features(sh, &e);
|
||||
let name = sh.name.clone();
|
||||
rewrite(cx, &name, &e);
|
||||
rewrite(cx, &name, &e)?;
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
for (i, c) in (r.start.col..=r.end.col).enumerate() {
|
||||
sh.set_value(CellRef::new(r.start.row, c), gridcraft_core::Value::text(format!("Column{}", i + 1)));
|
||||
@@ -476,7 +476,10 @@ fn write_totals(sh: &mut Sheet, ti: usize) {
|
||||
let c = CellRef::new(row, t.range.start.col + i as u32);
|
||||
if let Some(code) = col.totals.subtotal_code() {
|
||||
let f = format!("=SUBTOTAL({code},{}[{}])", t.name, col.name);
|
||||
sh.set_cell(c, Cell { formula: Some(std::sync::Arc::new(Formula::new(&f))), value: gridcraft_core::Value::Empty, style: sh.style_id(c) });
|
||||
sh.set_cell(
|
||||
c,
|
||||
Cell { formula: Some(std::sync::Arc::new(Formula::new(&f))), value: gridcraft_core::Value::Empty, style: sh.style_id(c), py: None },
|
||||
);
|
||||
} else if let Some(l) = &col.totals_label {
|
||||
sh.set_value(c, gridcraft_core::Value::text(l.as_str()));
|
||||
} else {
|
||||
@@ -1323,7 +1326,7 @@ fn insert_link(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
});
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
if let Some(t) = text.clone().or_else(|| sh.value(at).is_empty().then(|| target.clone())) {
|
||||
sh.set_cell(at, Cell { value: gridcraft_core::Value::text(t.as_str()), formula: None, style: link_style });
|
||||
sh.set_cell(at, Cell { value: gridcraft_core::Value::text(t.as_str()), formula: None, style: link_style, py: None });
|
||||
} else {
|
||||
sh.set_style(at, link_style);
|
||||
}
|
||||
|
||||
@@ -60,7 +60,7 @@ pub(super) fn insert_table(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let e = Edit::InsertCells { axis: Axis::Rows, range: RangeRef::new(r.start, CellRef::new(r.start.row, r.end.col)) };
|
||||
shift_cell_features(sh, &e);
|
||||
let name = sh.name.clone();
|
||||
rewrite(cx, &name, &e);
|
||||
rewrite(cx, &name, &e)?;
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
for (i, c) in (r.start.col..=r.end.col).enumerate() {
|
||||
sh.set_value(CellRef::new(r.start.row, c), gridcraft_core::Value::text(format!("Column{}", i + 1)));
|
||||
|
||||
@@ -9,7 +9,7 @@ pub fn specs() -> Vec<CommandSpec> {
|
||||
vec![
|
||||
cmd!(query "document.inspect", "Inspect Workbook", [], None, "{} → sheets, selection, names, tables, charts, dirty, trust (local|fromFile), history", has_doc, inspect),
|
||||
cmd!(query "sheet.read", "Read Range", [], None, "{range?: \"A1:D10\" (default: used range), sheet?, formulas?: bool, formatted?: bool} → rows of values", has_doc, read),
|
||||
cmd!(query "cell.get", "Get Cell", [], None, "{cell?, sheet?} → value, formula, display text, style, comment, link", has_doc, cell_get),
|
||||
cmd!(query "cell.get", "Get Cell", [], None, "{cell?, sheet?} → value, formula, display text, style, comment, link, python {code, output, typeName, preview, stale}", has_doc, cell_get),
|
||||
cmd!(query "selection.stats", "Selection Statistics", [], None, "{} → count, sum, average, min, max", has_doc, sel_stats),
|
||||
cmd!(query "history.list", "Undo History", [], None, "{} → undo/redo labels, grouping (an undo group is open)", has_doc, history),
|
||||
cmd!(query "app.commands", "List Commands", [], None, "{search?} → ids, labels, params", always, list_commands),
|
||||
@@ -137,6 +137,13 @@ fn cell_get(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
"merge": sh.merge_at(c).map(|m| m.a1()),
|
||||
"comment": sh.comments.get(&c),
|
||||
"hyperlink": sh.hyperlinks.get(&c),
|
||||
"python": cell.and_then(|x| x.py.as_deref()).map(|py| json!({
|
||||
"code": py.code,
|
||||
"output": if py.object { "object" } else { "value" },
|
||||
"typeName": py.output.as_ref().map(|o| o.type_name.as_str()),
|
||||
"preview": py.output.as_ref().map(|o| o.preview.as_str()),
|
||||
"stale": py.stale,
|
||||
})),
|
||||
}))
|
||||
}
|
||||
|
||||
|
||||
@@ -255,7 +255,10 @@ pub(super) fn show_details(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
}
|
||||
}
|
||||
for (c, h) in src.headers.iter().enumerate() {
|
||||
sh.set_cell(CellRef::new(0, c as u32), Cell { value: gridcraft_core::Value::text(h.as_str()), formula: None, style: StyleId::default() });
|
||||
sh.set_cell(
|
||||
CellRef::new(0, c as u32),
|
||||
Cell { value: gridcraft_core::Value::text(h.as_str()), formula: None, style: StyleId::default(), py: None },
|
||||
);
|
||||
}
|
||||
for (r, i) in records.iter().enumerate() {
|
||||
let Some(rec) = src.rows.get(*i) else { continue };
|
||||
@@ -264,7 +267,7 @@ pub(super) fn show_details(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
if v.is_empty() && style == StyleId::default() {
|
||||
continue;
|
||||
}
|
||||
sh.set_cell(CellRef::new(r as u32 + 1, c as u32), Cell { value: v.clone(), formula: None, style });
|
||||
sh.set_cell(CellRef::new(r as u32 + 1, c as u32), Cell { value: v.clone(), formula: None, style, py: None });
|
||||
}
|
||||
}
|
||||
let name = sh.name.clone();
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
//! Python in cells: making and editing Python cells. This build stores the code and the last
|
||||
//! result a file cached; it doesn't run Python, so a cell with new or changed code shows
|
||||
//! `#BLOCKED!`.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use gridcraft_core::{CellRef, Value};
|
||||
use gridcraft_model::{Cell, PyCode, python::MAX_FORMULA_CHARS};
|
||||
use serde_json::{Value as Json, json};
|
||||
|
||||
use super::{CommandSpec, bad, cell_param_on, cmd, edit, has_doc, str_param, target_sheet};
|
||||
use crate::{EngineError, Result, Session};
|
||||
|
||||
pub fn specs() -> Vec<CommandSpec> {
|
||||
vec![cmd!(
|
||||
"python.set",
|
||||
"Set Python Code",
|
||||
[],
|
||||
None,
|
||||
"{cell?, code, output?: object|value} makes the cell (default: the active cell) a Python cell with this code; the code must fit in Excel's 8,192-character formula once written as PY(…). `output` is the return type: `value` (an Excel value, the default) or `object` (a Python object)",
|
||||
has_doc,
|
||||
set
|
||||
)]
|
||||
}
|
||||
|
||||
fn target_cell(s: &Session, p: &Json) -> Result<(usize, CellRef)> {
|
||||
let sheet = target_sheet(s, p)?;
|
||||
let at = match cell_param_on(s, p, "cell", sheet)? {
|
||||
Some(c) => c,
|
||||
None => s.doc()?.selection.active,
|
||||
};
|
||||
Ok((sheet, at))
|
||||
}
|
||||
|
||||
fn existing(s: &Session, sheet: usize, at: CellRef) -> Result<Option<PyCode>> {
|
||||
Ok(s.doc()?.wb.sheet(sheet).and_then(|sh| sh.cell(at)).and_then(|c| c.py.as_deref()).cloned())
|
||||
}
|
||||
|
||||
fn output_mode(p: &Json, key: &str) -> Result<Option<bool>> {
|
||||
match str_param(p, key) {
|
||||
None => Ok(None),
|
||||
Some(m) if m.eq_ignore_ascii_case("object") => Ok(Some(true)),
|
||||
Some(m) if m.eq_ignore_ascii_case("value") => Ok(Some(false)),
|
||||
Some(m) => Err(bad("python.set", format!("`{key}` is `object` or `value`, not `{m}`"))),
|
||||
}
|
||||
}
|
||||
|
||||
fn set(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let (sheet, at) = target_cell(s, p)?;
|
||||
let code = match p.get("code") {
|
||||
Some(Json::String(c)) => c.as_str(),
|
||||
Some(_) => return Err(bad("python.set", "`code` must be text")),
|
||||
None => return Err(bad("python.set", "missing `code`")),
|
||||
};
|
||||
let mode = output_mode(p, "output")?;
|
||||
let old = existing(s, sheet, at)?;
|
||||
let object = mode.or(old.as_ref().map(|o| o.object)).unwrap_or(false);
|
||||
let mut py = PyCode { object, ..PyCode::new(code) };
|
||||
if !py.fits() {
|
||||
return Err(bad(
|
||||
"python.set",
|
||||
format!("the code doesn't fit in one cell: written as PY(…) it would be longer than Excel's {MAX_FORMULA_CHARS}-character formula limit"),
|
||||
));
|
||||
}
|
||||
// A table header is the column's name, never a formula or code (as in Excel).
|
||||
let header =
|
||||
s.doc()?.wb.sheet(sheet).is_some_and(|sh| sh.tables.iter().any(|t| t.header_row && t.range.start.row == at.row && t.range.contains(at)));
|
||||
if header {
|
||||
return Err(EngineError::Other(format!("{} is a table header; Python cells can't go in a table's header row", at.a1())));
|
||||
}
|
||||
// Unchanged code with the same return type keeps its result (and its value); new code, or a
|
||||
// new return type, has none until it runs: a result of the other type would show a value and
|
||||
// preview that don't belong to the cell.
|
||||
let keep = old.filter(|o| o.code == py.code && o.object == py.object);
|
||||
let value = match &keep {
|
||||
Some(o) => {
|
||||
py.output = o.output.clone();
|
||||
py.stale = o.stale;
|
||||
None
|
||||
}
|
||||
None => Some(gridcraft_model::python::BLOCKED),
|
||||
};
|
||||
edit(s, |cx| {
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
let style = sh.style_id(at);
|
||||
let value = value.unwrap_or_else(|| sh.cell(at).map(|c| c.value.clone()).unwrap_or(Value::Empty));
|
||||
// A spill the cell had (a formula's, or the old result's) is left to the recalculation,
|
||||
// which removes it and recalculates what read it.
|
||||
sh.set_cell(at, Cell { value, formula: None, style, py: Some(Arc::new(py)) });
|
||||
cx.touch(sheet, at);
|
||||
Ok(())
|
||||
})?;
|
||||
let d = s.doc()?;
|
||||
let shown = d.wb.sheet(sheet).map(|sh| sh.value(at)).unwrap_or_default();
|
||||
Ok(json!({"cell": at.a1(), "value": crate::cmd::inspect::value_json(&shown)}))
|
||||
}
|
||||
@@ -217,10 +217,10 @@ fn shift_sheet_features(sh: &mut Sheet, axis: Axis, at: u32, count: u32, insert:
|
||||
|
||||
/// Rewrites every formula reference (chart series included), sparkline source and PivotTable
|
||||
/// source for a structural edit on `target`.
|
||||
pub(super) fn rewrite(cx: &mut Ctx, target: &str, e: &Edit) {
|
||||
pub(super) fn rewrite(cx: &mut Ctx, target: &str, e: &Edit) -> Result<()> {
|
||||
cx.edits.push((target.to_string(), e.clone()));
|
||||
let wb = &mut cx.wb;
|
||||
super::edit::rewrite_all_formulas(wb, target, e);
|
||||
super::edit::rewrite_all_formulas(wb, target, e)?;
|
||||
for si in 0..wb.sheets.len() {
|
||||
let host = wb.sheets.get(si).map(|s| s.name.clone()).unwrap_or_default();
|
||||
let Some(sh) = wb.sheet_mut(si) else { continue };
|
||||
@@ -246,6 +246,7 @@ pub(super) fn rewrite(cx: &mut Ctx, target: &str, e: &Edit) {
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Inserting or deleting cells moves the copied block away from the stored source range, so, like
|
||||
@@ -303,7 +304,7 @@ fn insert_lines(s: &mut Session, p: &Json, axis: Axis) -> Result<Json> {
|
||||
// New lines take the format of the line before (Excel's "Format Same As Above").
|
||||
inherit_line_format(sh, axis, at, count);
|
||||
}
|
||||
rewrite(cx, &name, &Edit::Insert { axis, at, count });
|
||||
rewrite(cx, &name, &Edit::Insert { axis, at, count })?;
|
||||
cx.structural = true;
|
||||
Ok(Json::Null)
|
||||
})?;
|
||||
@@ -368,7 +369,7 @@ fn delete_lines(s: &mut Session, p: &Json, axis: Axis) -> Result<Json> {
|
||||
}
|
||||
shift_sheet_features(sh, axis, at, count, false);
|
||||
}
|
||||
rewrite(cx, &name, &Edit::Delete { axis, at, count });
|
||||
rewrite(cx, &name, &Edit::Delete { axis, at, count })?;
|
||||
}
|
||||
cx.structural = true;
|
||||
Ok(Json::Null)
|
||||
@@ -420,7 +421,7 @@ fn move_columns(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
sh.cells.shift_cols(before, count as i64);
|
||||
shift_sheet_features(sh, Axis::Cols, before, count, true);
|
||||
}
|
||||
rewrite(cx, &name, &Edit::Insert { axis: Axis::Cols, at: before, count });
|
||||
rewrite(cx, &name, &Edit::Insert { axis: Axis::Cols, at: before, count })?;
|
||||
|
||||
let mut touched = Vec::new();
|
||||
{
|
||||
@@ -458,14 +459,14 @@ fn move_columns(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
for cell in touched {
|
||||
cx.touch(sheet, cell);
|
||||
}
|
||||
rewrite(cx, &name, &Edit::Move { from: source, to_row: 0, to_col: destination });
|
||||
rewrite(cx, &name, &Edit::Move { from: source, to_row: 0, to_col: destination })?;
|
||||
|
||||
{
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
sh.cells.shift_cols(delete_at + count, -(count as i64));
|
||||
shift_sheet_features(sh, Axis::Cols, delete_at, count, false);
|
||||
}
|
||||
rewrite(cx, &name, &Edit::Delete { axis: Axis::Cols, at: delete_at, count });
|
||||
rewrite(cx, &name, &Edit::Delete { axis: Axis::Cols, at: delete_at, count })?;
|
||||
*cx.sel = crate::Selection {
|
||||
active: CellRef::new(active_row, final_start),
|
||||
anchor: CellRef::new(active_row, final_start),
|
||||
@@ -496,7 +497,7 @@ fn insert_cells(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
} else {
|
||||
RangeRef::new(CellRef::new(r.start.row.max(MAX_ROWS.saturating_sub(r.height())), r.start.col), CellRef::new(MAX_ROWS - 1, r.end.col))
|
||||
};
|
||||
if sh.cells.iter_range(doomed).any(|(_, x)| x.formula.is_some() || !x.value.is_empty()) {
|
||||
if sh.cells.iter_range(doomed).any(|(_, x)| x.has_content()) {
|
||||
return Err(EngineError::Other("To prevent possible loss of data, GridCraft cannot shift nonblank cells off of the worksheet.".into()));
|
||||
}
|
||||
}
|
||||
@@ -510,7 +511,7 @@ fn insert_cells(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
}
|
||||
shift_cell_features(sh, &e);
|
||||
let name = cx.wb.sheet(sheet).map(|s| s.name.clone()).unwrap_or_default();
|
||||
rewrite(cx, &name, &e);
|
||||
rewrite(cx, &name, &e)?;
|
||||
cx.structural = true;
|
||||
Ok(Json::Null)
|
||||
})?;
|
||||
@@ -545,7 +546,7 @@ fn delete_cells(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
shift_cell_features(sh, &e);
|
||||
}
|
||||
// References into the deleted cells become #REF!; the rest move.
|
||||
rewrite(cx, &name, &e);
|
||||
rewrite(cx, &name, &e)?;
|
||||
cx.structural = true;
|
||||
Ok(Json::Null)
|
||||
})?;
|
||||
|
||||
@@ -63,6 +63,7 @@ pub(super) fn delete_sheet(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
}
|
||||
}
|
||||
}
|
||||
crate::cmd::edit::rewrite_python_refs(&mut cx.wb, &|_, e| gridcraft_formula::adjust::delete_sheet(e, &name, &order))?;
|
||||
// Chart series into the deleted sheet become #REF!, like formulas.
|
||||
crate::cmd::edit::rewrite_cell_links(&mut cx.wb, |_, e| gridcraft_formula::adjust::delete_sheet(e, &name, &order));
|
||||
cx.wb.names.retain(|n| n.scope != Some(i));
|
||||
@@ -133,6 +134,7 @@ pub(super) fn rename_sheet(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
n.formula = gridcraft_formula::print(&gridcraft_formula::adjust::rename_sheet(e, &old, &name));
|
||||
}
|
||||
}
|
||||
crate::cmd::edit::rewrite_python_refs(&mut cx.wb, &|_, e| gridcraft_formula::adjust::rename_sheet(e, &old, &name))?;
|
||||
cx.structural = true;
|
||||
Ok(Json::Null)
|
||||
})
|
||||
|
||||
@@ -303,7 +303,7 @@ fn run(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
for (c, v) in model.vars.iter().zip(x) {
|
||||
let style = sh.style_id(*c);
|
||||
sh.set_cell(*c, Cell { value: Value::number(*v), formula: None, style });
|
||||
sh.set_cell(*c, Cell { value: Value::number(*v), formula: None, style, py: None });
|
||||
}
|
||||
for c in &model.vars {
|
||||
cx.touch(sheet + count, *c);
|
||||
|
||||
@@ -263,7 +263,8 @@ pub(super) fn parse_model(s: &Session, p: &Json) -> Result<Model> {
|
||||
}
|
||||
let mut seen = std::collections::HashSet::new();
|
||||
vars.retain(|c| seen.insert(*c));
|
||||
if vars.iter().any(|c| sh.cell(*c).is_some_and(|c| c.formula.is_some())) {
|
||||
// Python cells count as formulas: a solution written over one would lose its code.
|
||||
if vars.iter().any(|c| sh.cell(*c).is_some_and(|c| c.formula.is_some() || c.py.is_some())) {
|
||||
return Err(EngineError::Other("Variable cells must hold constants, not formulas.".into()));
|
||||
}
|
||||
let mut cons = Vec::new();
|
||||
|
||||
@@ -107,12 +107,12 @@ fn check_column_spills(sheet: &Sheet, column: RangeRef, ours: &[CellRef]) -> Res
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Refused, unless `overwrite`, when a cell of the model's column holds a value or a formula
|
||||
/// (a model saved there before included): Save Model then asks before it replaces data, as
|
||||
/// Excel does. The message carries [`OVERWRITE_ERROR`], which the dialog turns into its
|
||||
/// Refused, unless `overwrite`, when a cell of the model's column holds a value, a formula or
|
||||
/// Python code (a model saved there before included): Save Model then asks before it replaces
|
||||
/// data, as Excel does. The message carries [`OVERWRITE_ERROR`], which the dialog turns into its
|
||||
/// question.
|
||||
fn check_column_empty(sheet: &Sheet, column: RangeRef, overwrite: bool) -> Result<()> {
|
||||
let taken = column.iter().find(|c| sheet.cell(*c).is_some_and(|x| !x.value.is_empty() || x.formula.is_some()));
|
||||
let taken = column.iter().find(|c| sheet.cell(*c).is_some_and(Cell::has_content));
|
||||
match taken {
|
||||
Some(c) if !overwrite => Err(EngineError::Other(format!(
|
||||
"The range {} {OVERWRITE_ERROR} ({} isn't empty); pass overwrite: true to replace it.",
|
||||
@@ -176,7 +176,7 @@ pub(super) fn save_model_cells(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
}
|
||||
for c in &stale {
|
||||
let style = sh.style_id(*c);
|
||||
sh.set_cell(*c, Cell { value: Value::Empty, formula: None, style });
|
||||
sh.set_cell(*c, Cell { value: Value::Empty, formula: None, style, py: None });
|
||||
}
|
||||
for c in cells.into_iter().chain(stale) {
|
||||
cx.touch(sheet, c);
|
||||
|
||||
@@ -352,7 +352,7 @@ pub(super) fn write_reports(cx: &mut Ctx, at: usize, built: Vec<(String, Lines)>
|
||||
let mut sheet = Sheet::new(name);
|
||||
for (r, c, v, b) in lines {
|
||||
let style = if b { bold } else { gridcraft_model::StyleId::DEFAULT };
|
||||
sheet.set_cell(CellRef::new(r, c), Cell { value: v, formula: None, style });
|
||||
sheet.set_cell(CellRef::new(r, c), Cell { value: v, formula: None, style, py: None });
|
||||
}
|
||||
for (c, w) in [(0u32, 14.0f32), (1, 110.0), (2, 140.0), (3, 120.0), (4, 120.0), (5, 155.0), (6, 140.0), (7, 140.0)] {
|
||||
sheet.cols.insert(c, gridcraft_model::LineInfo { size: Some(w), ..Default::default() });
|
||||
|
||||
@@ -213,7 +213,7 @@ impl Trial {
|
||||
for (si, c, v) in sets {
|
||||
if let Some(sh) = wb.sheet_mut(*si) {
|
||||
let style = sh.style_id(*c);
|
||||
sh.set_cell(*c, Cell { value: v.clone(), formula: None, style });
|
||||
sh.set_cell(*c, Cell { value: v.clone(), formula: None, style, py: None });
|
||||
keys.push((*si, *c));
|
||||
}
|
||||
}
|
||||
@@ -329,7 +329,8 @@ fn goal_seek(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
if !has_formula(ssi, set) {
|
||||
return Err(EngineError::Other("Cell must contain a formula.".into()));
|
||||
}
|
||||
if has_formula(csi, chg) {
|
||||
// A Python cell is a formula to Excel (`=PY(…)`): writing a number over it would lose its code.
|
||||
if has_formula(csi, chg) || d.wb.sheet(csi).and_then(|sh| sh.cell(chg)).is_some_and(|c| c.py.is_some()) {
|
||||
return Err(EngineError::Other("Cell must contain a value.".into()));
|
||||
}
|
||||
let x0 = d.wb.sheet(csi).map(|sh| sh.value(chg)).and_then(|v| v.as_f64()).unwrap_or(0.0);
|
||||
@@ -342,7 +343,7 @@ fn goal_seek(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
edit(s, |cx| {
|
||||
let sh = cx.sheet_mut(csi)?;
|
||||
let style = sh.style_id(chg);
|
||||
sh.set_cell(chg, Cell { value: Value::number(r.x), formula: None, style });
|
||||
sh.set_cell(chg, Cell { value: Value::number(r.x), formula: None, style, py: None });
|
||||
cx.touch(csi, chg);
|
||||
Ok(())
|
||||
})?;
|
||||
@@ -422,7 +423,7 @@ fn data_table(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
for (c, v) in out {
|
||||
let sh = cx.sheet_mut(si)?;
|
||||
let style = sh.style_id(c);
|
||||
sh.set_cell(c, Cell { value: v, formula: None, style });
|
||||
sh.set_cell(c, Cell { value: v, formula: None, style, py: None });
|
||||
cx.touch(si, c);
|
||||
}
|
||||
Ok(())
|
||||
|
||||
@@ -186,7 +186,7 @@ pub(super) fn advanced_filter(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
// The full output footprint was checked above.
|
||||
let to = CellRef::new(dest.row + i as u32, dest.col + c - list.start.col);
|
||||
let style = sh.style_id(src);
|
||||
(to, Cell { value: sh.value(src), formula: None, style })
|
||||
(to, Cell { value: sh.value(src), formula: None, style, py: None })
|
||||
})
|
||||
})
|
||||
.collect();
|
||||
|
||||
@@ -185,7 +185,7 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
if v.is_empty() && style == StyleId::DEFAULT {
|
||||
sh.remove_cell(c);
|
||||
} else {
|
||||
sh.set_cell(c, Cell { value: v, formula: None, style });
|
||||
sh.set_cell(c, Cell { value: v, formula: None, style, py: None });
|
||||
}
|
||||
cx.touch(dsi, c);
|
||||
}
|
||||
|
||||
@@ -160,7 +160,7 @@ pub(super) fn scenario_show(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let cell = match v.as_f64() {
|
||||
Some(n) => {
|
||||
let style = cx.wb.sheet(si).map(|sh| sh.style_id(*c)).unwrap_or_default();
|
||||
Some(Cell { value: Value::Number(n), formula: None, style })
|
||||
Some(Cell { value: Value::Number(n), formula: None, style, py: None })
|
||||
}
|
||||
None => crate::cmd::edit::input_to_cell(&crate::cmd::edit::json_to_input(v), si, *c, &mut cx.wb)?,
|
||||
};
|
||||
@@ -245,7 +245,7 @@ pub(super) fn scenario_summary(s: &mut Session, p: &Json) -> Result<Json> {
|
||||
let bold = cx.wb.styles.derive(StyleId::DEFAULT, |st| st.font.bold = true);
|
||||
let mut sheet = Sheet::new(name.clone());
|
||||
let put = |sheet: &mut Sheet, row: u32, col: u32, v: Value, style: StyleId| {
|
||||
sheet.set_cell(CellRef::new(row, col), Cell { value: v, formula: None, style });
|
||||
sheet.set_cell(CellRef::new(row, col), Cell { value: v, formula: None, style, py: None });
|
||||
};
|
||||
put(&mut sheet, 1, 1, Value::text("Scenario Summary"), bold);
|
||||
put(&mut sheet, 2, 2, Value::text("Current Values:"), bold);
|
||||
|
||||
@@ -90,14 +90,14 @@ pub fn edit_text(wb: &Workbook, sheet: &Sheet, c: CellRef) -> String {
|
||||
if reads_back(&iso) { iso } else { raw }
|
||||
}
|
||||
|
||||
/// Whether `c` has a formula that must not be shown: its cell is formatted Hidden on a
|
||||
/// protected sheet. A spilled cell follows its anchor.
|
||||
/// Whether `c` has a formula or Python code that must not be shown: its cell is formatted
|
||||
/// Hidden on a protected sheet. A spilled cell follows its anchor.
|
||||
pub fn formula_hidden(wb: &Workbook, sheet: &Sheet, c: CellRef) -> bool {
|
||||
if !sheet.is_protected() {
|
||||
return false;
|
||||
}
|
||||
let at = match sheet.cell(c) {
|
||||
Some(x) if x.formula.is_some() => c,
|
||||
Some(x) if x.formula.is_some() || x.py.is_some() => c,
|
||||
_ => match sheet.spill_ranges.iter().find(|(_, r)| r.contains(c)) {
|
||||
Some((anchor, _)) => *anchor,
|
||||
None => return false,
|
||||
|
||||
@@ -171,10 +171,12 @@ pub fn fill(cx: &mut Ctx, sheet: usize, src: RangeRef, target: RangeRef, mode: F
|
||||
cell.formula = Some(Arc::new(Formula::from_expr(shifted)));
|
||||
cell.value = Value::Empty;
|
||||
}
|
||||
} else if let Some(v) = match per_cell.get(src_i) {
|
||||
Some(p) => p.value(k as i64 / src_cells.len() as i64 + 1, forward, 1),
|
||||
None => pattern.value(step, forward, src_cells.len()),
|
||||
} {
|
||||
} else if cell.py.is_none()
|
||||
&& let Some(v) = match per_cell.get(src_i) {
|
||||
Some(p) => p.value(k as i64 / src_cells.len() as i64 + 1, forward, 1),
|
||||
None => pattern.value(step, forward, src_cells.len()),
|
||||
}
|
||||
{
|
||||
cell.value = v;
|
||||
}
|
||||
cell.style = style;
|
||||
@@ -296,7 +298,8 @@ impl Pattern {
|
||||
|
||||
fn detect(cells: &[(CellRef, Option<Cell>)], custom: &[Vec<String>], sys: gridcraft_core::DateSystem, fmt: &str) -> Pattern {
|
||||
let vals: Vec<Value> = cells.iter().map(|(_, c)| c.as_ref().map(|c| c.value.clone()).unwrap_or_default()).collect();
|
||||
if cells.iter().any(|(_, c)| c.as_ref().is_some_and(|c| c.formula.is_some())) {
|
||||
// Formulas and Python cells copy.
|
||||
if cells.iter().any(|(_, c)| c.as_ref().is_some_and(|c| c.formula.is_some() || c.py.is_some())) {
|
||||
return Pattern::Copy;
|
||||
}
|
||||
// Numbers.
|
||||
@@ -598,7 +601,7 @@ pub fn series(cx: &mut Ctx, sheet: usize, options: SeriesOptions<'_>) -> Result<
|
||||
break;
|
||||
}
|
||||
let c = if rows { CellRef::new(lane, r.start.col + i) } else { CellRef::new(r.start.row + i, lane) };
|
||||
writes.push((c, Cell { value: Value::number(round15(v)), formula: None, style }));
|
||||
writes.push((c, Cell { value: Value::number(round15(v)), formula: None, style, py: None }));
|
||||
}
|
||||
}
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
@@ -657,7 +660,7 @@ pub fn flash_fill(cx: &mut Ctx, sheet: usize, at: CellRef) -> Result<usize> {
|
||||
}
|
||||
let sh = cx.sheet_mut(sheet)?;
|
||||
for (c, out) in &writes {
|
||||
sh.set_cell(*c, Cell { value: Value::text(out.as_str()), formula: None, style });
|
||||
sh.set_cell(*c, Cell { value: Value::text(out.as_str()), formula: None, style, py: None });
|
||||
}
|
||||
for (c, _) in writes {
|
||||
cx.touch(sheet, c);
|
||||
|
||||
+130
-1
@@ -118,13 +118,142 @@ fn decrypt_message(name: &str, e: gridcraft_xlsx::DecryptError) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
/// Parses bytes that aren't encrypted, then takes the workbook's macros out of their hidden names.
|
||||
/// Parses bytes that aren't encrypted, then takes the workbook's macros out of their hidden names
|
||||
/// and adopts the Python cells they hold (see [`adopt_python`]) within the limits of
|
||||
/// [`enforce_python_limits`].
|
||||
fn open_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, Vec<String>)> {
|
||||
let (mut wb, mut warnings) = read_plain(name, bytes, csv_as_text)?;
|
||||
warnings.extend(crate::macros::load(&mut wb));
|
||||
warnings.extend(adopt_python(&mut wb));
|
||||
warnings.extend(enforce_python_limits(&mut wb));
|
||||
Ok((wb, warnings))
|
||||
}
|
||||
|
||||
/// The most result cells [`adopt_python`] rebuilds from a file's cached spills, over all its
|
||||
/// Python cells (each one is also limited to [`gridcraft_model::python::MAX_RESULT_CELLS`]).
|
||||
pub const MAX_ADOPTED_RESULT_CELLS: u64 = 4_000_000;
|
||||
|
||||
/// Python cells from a file: a cell whose formula is `PY("…",0|1)` (a format that kept it as a
|
||||
/// formula) becomes a Python cell, and a Python cell whose spilled values the file cached gets
|
||||
/// them back as its result, so it shows them until it runs again. Results are rebuilt for at most
|
||||
/// [`MAX_ADOPTED_RESULT_CELLS`] cells in all; the rest keep only their own cached value, and the
|
||||
/// values they spilled stay as plain values. Returns the warnings.
|
||||
pub fn adopt_python(wb: &mut Workbook) -> Vec<String> {
|
||||
let mut budget = MAX_ADOPTED_RESULT_CELLS;
|
||||
let mut skipped = 0usize;
|
||||
for si in 0..wb.sheets.len() {
|
||||
let Some(sh) = wb.sheets.get(si) else { continue };
|
||||
let formulas: Vec<(CellRef, gridcraft_model::PyCode)> = sh
|
||||
.cells
|
||||
.iter()
|
||||
.filter_map(|(c, cell)| cell.formula.as_ref().and_then(|f| gridcraft_model::PyCode::from_formula(&f.text)).map(|p| (c, p)))
|
||||
.collect();
|
||||
let max = gridcraft_model::python::MAX_RESULT_CELLS as u64;
|
||||
let spills: Vec<(CellRef, RangeRef)> = sh
|
||||
.spill_ranges
|
||||
.iter()
|
||||
.filter(|(a, r)| r.count() <= max && sh.cell(**a).and_then(|c| c.py.as_ref()).is_some_and(|p| p.output.is_none()))
|
||||
.map(|(a, r)| (*a, *r))
|
||||
.collect();
|
||||
if formulas.is_empty() && spills.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let Some(sh) = wb.sheet_mut(si) else { continue };
|
||||
for (c, py) in formulas {
|
||||
if let Some(cell) = sh.cells.get_mut(c) {
|
||||
cell.formula = None;
|
||||
cell.py = Some(std::sync::Arc::new(py));
|
||||
}
|
||||
}
|
||||
for (anchor, range) in spills {
|
||||
let Some(left) = budget.checked_sub(range.count()) else {
|
||||
skipped += 1;
|
||||
// The cell keeps only its own value; the values it spilled stay as plain values, so
|
||||
// nothing shows as a spill that no result holds, and nothing the file showed is lost.
|
||||
sh.settle_spill(anchor);
|
||||
continue;
|
||||
};
|
||||
budget = left;
|
||||
let data: Vec<gridcraft_core::Value> =
|
||||
range.iter().map(|c| if c == anchor { sh.cell(c).map(|x| x.value.clone()).unwrap_or_default() } else { sh.value(c) }).collect();
|
||||
let Some(array) = gridcraft_core::Array::new(range.height() as usize, range.width() as usize, data) else { continue };
|
||||
if let Some(py) = sh.cells.get_mut(anchor).and_then(|x| x.py.as_mut()) {
|
||||
std::sync::Arc::make_mut(py).output =
|
||||
Some(gridcraft_model::PyOutput { value: array.into(), type_name: "list".into(), preview: String::new(), convertible: true });
|
||||
}
|
||||
}
|
||||
}
|
||||
if skipped == 0 {
|
||||
return Vec::new();
|
||||
}
|
||||
vec![format!(
|
||||
"The file's Python results are larger than {MAX_ADOPTED_RESULT_CELLS} cells in all, so {skipped} Python cell(s) have no result until they run again; the values they spilled were kept as plain values."
|
||||
)]
|
||||
}
|
||||
|
||||
/// A file is somebody else's work: a Python cell whose code doesn't fit in Excel's
|
||||
/// 8,192-character formula keeps only its cached value, and a kept result that is not a
|
||||
/// well-formed array of at most [`gridcraft_model::python::MAX_RESULT_CELLS`] cells is dropped
|
||||
/// (the cell keeps its value). Either way the values the cell spilled stay as plain values
|
||||
/// ([`gridcraft_model::Sheet::settle_spill`]). Returns the warnings.
|
||||
pub fn enforce_python_limits(wb: &mut Workbook) -> Vec<String> {
|
||||
let mut long = Vec::new();
|
||||
for si in 0..wb.sheets.len() {
|
||||
let Some(sh) = wb.sheets.get(si) else { continue };
|
||||
let mut over = Vec::new();
|
||||
let mut bad_output = Vec::new();
|
||||
for (c, cell) in sh.cells.iter() {
|
||||
let Some(py) = &cell.py else { continue };
|
||||
if !py.fits() {
|
||||
long.push(format!("{}!{}", sh.name, c.a1()));
|
||||
over.push(c);
|
||||
} else if py.output.as_ref().is_some_and(|o| !sane_python_output(&o.value)) {
|
||||
bad_output.push(c);
|
||||
}
|
||||
}
|
||||
if over.is_empty() && bad_output.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let dropped: Vec<CellRef> = over.iter().chain(&bad_output).copied().collect();
|
||||
let Some(sh) = wb.sheet_mut(si) else { continue };
|
||||
for c in over {
|
||||
if let Some(cell) = sh.cells.get_mut(c) {
|
||||
cell.py = None;
|
||||
}
|
||||
}
|
||||
for c in bad_output {
|
||||
if let Some(py) = sh.cells.get_mut(c).and_then(|x| x.py.as_mut()) {
|
||||
std::sync::Arc::make_mut(py).output = None;
|
||||
}
|
||||
}
|
||||
// A dropped result (or code dropped to its value) leaves the values it spilled as plain
|
||||
// values, as the file showed them, with no live spill.
|
||||
for c in dropped {
|
||||
sh.settle_spill(c);
|
||||
}
|
||||
}
|
||||
let Some(first) = long.first() else { return vec![] };
|
||||
let more = if long.len() > 1 { format!(" (and {} more)", long.len() - 1) } else { String::new() };
|
||||
vec![format!(
|
||||
"A Python cell's code is longer than Excel's {}-character formula limit, so {first}{more} kept only its last calculated value.",
|
||||
gridcraft_model::python::MAX_FORMULA_CHARS
|
||||
)]
|
||||
}
|
||||
|
||||
/// A Python result a file may hold: a scalar, or an array whose size matches its data and has at
|
||||
/// most [`gridcraft_model::python::MAX_RESULT_CELLS`] cells.
|
||||
fn sane_python_output(v: &gridcraft_core::Value) -> bool {
|
||||
match v {
|
||||
gridcraft_core::Value::Array(a) => {
|
||||
a.rows >= 1
|
||||
&& a.cols >= 1
|
||||
&& a.rows.checked_mul(a.cols).is_some_and(|n| n == a.data.len() && n <= gridcraft_model::python::MAX_RESULT_CELLS)
|
||||
}
|
||||
_ => true,
|
||||
}
|
||||
}
|
||||
|
||||
/// Parses bytes that aren't encrypted, as they are.
|
||||
fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook, Vec<String>)> {
|
||||
let kind = FileKind::from_path(name);
|
||||
let sniffed = gridcraft_xlsx::sniff(bytes);
|
||||
|
||||
@@ -461,17 +461,17 @@ fn csv_writes_displayed_text() {
|
||||
(styled("yyyy-mm-dd"), styled("$#,##0.00"), styled("0.0%"), styled("# ?/?"), styled("@"), styled("0.00"), styled("#,##0"));
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
let cells = [
|
||||
("A1", Cell { value: Value::Number(46023.0), formula: None, style: date }),
|
||||
("B1", Cell { value: Value::Number(1234.5), formula: None, style: money }),
|
||||
("C1", Cell { value: Value::Number(0.125), formula: None, style: pct }),
|
||||
("D1", Cell { value: Value::Number(1.5), formula: None, style: frac }),
|
||||
("E1", Cell { value: Value::text("007"), formula: None, style: text }),
|
||||
("F1", Cell { value: Value::Error(CellError::NA), formula: None, style: money }),
|
||||
("G1", Cell { value: Value::Bool(true), formula: None, style: StyleId::DEFAULT }),
|
||||
("A1", Cell { value: Value::Number(46023.0), formula: None, style: date, py: None }),
|
||||
("B1", Cell { value: Value::Number(1234.5), formula: None, style: money, py: None }),
|
||||
("C1", Cell { value: Value::Number(0.125), formula: None, style: pct, py: None }),
|
||||
("D1", Cell { value: Value::Number(1.5), formula: None, style: frac, py: None }),
|
||||
("E1", Cell { value: Value::text("007"), formula: None, style: text, py: None }),
|
||||
("F1", Cell { value: Value::Error(CellError::NA), formula: None, style: money, py: None }),
|
||||
("G1", Cell { value: Value::Bool(true), formula: None, style: StyleId::DEFAULT, py: None }),
|
||||
("A2", Cell { value: Value::Number(46024.0), style: date, ..Cell::formula(Formula::new("A1+1")) }),
|
||||
("B2", Cell { value: Value::Number(12345678.9), formula: None, style: fixed }),
|
||||
("C2", Cell { value: Value::Number(-0.25), formula: None, style: StyleId::DEFAULT }),
|
||||
("D2", Cell { value: Value::Number(1234567.0), formula: None, style: thousands }),
|
||||
("B2", Cell { value: Value::Number(12345678.9), formula: None, style: fixed, py: None }),
|
||||
("C2", Cell { value: Value::Number(-0.25), formula: None, style: StyleId::DEFAULT, py: None }),
|
||||
("D2", Cell { value: Value::Number(1234567.0), formula: None, style: thousands, py: None }),
|
||||
];
|
||||
for (a, c) in cells {
|
||||
sh.set_cell(CellRef::parse(a).unwrap(), c);
|
||||
|
||||
@@ -614,7 +614,10 @@ mod tests {
|
||||
let mut wb = wb();
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
let formula = Some(std::sync::Arc::new(gridcraft_model::Formula::new("\"Amo\"&\"unt\"")));
|
||||
sh.set_cell(CellRef::new(1, 2), gridcraft_model::Cell { value: gridcraft_core::Value::text("Amount"), formula, style: Default::default() });
|
||||
sh.set_cell(
|
||||
CellRef::new(1, 2),
|
||||
gridcraft_model::Cell { value: gridcraft_core::Value::text("Amount"), formula, style: Default::default(), py: None },
|
||||
);
|
||||
sh.show_formulas = true;
|
||||
let inputs = vec![("Input".to_string(), place("B2:D11"), true)];
|
||||
let env = Env { wb: &wb, active_sheet: 0, inputs: &inputs, output: None, item: None };
|
||||
|
||||
@@ -1237,26 +1237,39 @@ pub(crate) fn pair_sheets(before: &Workbook, after: &Workbook) -> (Vec<(usize, O
|
||||
(pairs, spare.collect())
|
||||
}
|
||||
|
||||
/// Only a formula's cached value changed (a recalculation, not a write).
|
||||
/// Only a formula's (or a Python cell's) cached value changed (a recalculation, not a write): a
|
||||
/// Python cell whose spill a step blocked shows #SPILL! with its code untouched.
|
||||
fn recalculated_only(b: Option<&Cell>, a: Option<&Cell>) -> bool {
|
||||
match (b, a) {
|
||||
(Some(b), Some(a)) => match (&b.formula, &a.formula) {
|
||||
(Some(x), Some(y)) => x.text == y.text && x.array == y.array && b.style == a.style,
|
||||
(None, None) => match (&b.py, &a.py) {
|
||||
(Some(x), Some(y)) => x.code == y.code && x.object == y.object && b.style == a.style,
|
||||
_ => false,
|
||||
},
|
||||
_ => false,
|
||||
},
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Python code is content, as a formula is, whatever the cell shows.
|
||||
fn has_content(c: &Cell) -> bool {
|
||||
c.formula.is_some() || !(c.value.is_empty() || c.value.as_text() == Some(""))
|
||||
c.formula.is_some() || c.py.is_some() || !(c.value.is_empty() || c.value.as_text() == Some(""))
|
||||
}
|
||||
|
||||
/// Whether a write changed what the person entered: the formula, the Python code and its return
|
||||
/// type, or the value. A Python cell replaced by its own cached value, or by other code that
|
||||
/// shows the same `#BLOCKED!`, has changed.
|
||||
fn content_differs(b: Option<&Cell>, a: Option<&Cell>) -> bool {
|
||||
match (b, a) {
|
||||
(Some(b), Some(a)) => match (&b.formula, &a.formula) {
|
||||
(Some(x), Some(y)) => x.text != y.text,
|
||||
(None, None) => b.value != a.value,
|
||||
(None, None) => match (&b.py, &a.py) {
|
||||
(Some(x), Some(y)) => x.code != y.code || x.object != y.object,
|
||||
(None, None) => b.value != a.value,
|
||||
_ => true,
|
||||
},
|
||||
_ => true,
|
||||
},
|
||||
(Some(b), None) => has_content(b),
|
||||
|
||||
@@ -228,7 +228,7 @@ pub fn refresh(wb: &mut Workbook, sheet: usize, idx: usize, replace: bool) -> Re
|
||||
}
|
||||
if !replace {
|
||||
for (c, cell) in sh.cells.iter_range(area) {
|
||||
if (!cell.value.is_empty() || cell.formula.is_some()) && !pt.last_range.is_some_and(|r| r.contains(c)) {
|
||||
if cell.has_content() && !pt.last_range.is_some_and(|r| r.contains(c)) {
|
||||
return Err(format!(
|
||||
"There's already data in {}!{}. Do you want to replace it? (Run again with replace: true to overwrite.)",
|
||||
quote_sheet(&sh.name),
|
||||
@@ -278,7 +278,10 @@ pub fn refresh(wb: &mut Workbook, sheet: usize, idx: usize, replace: bool) -> Re
|
||||
if oc.value.is_empty() && id == StyleId::default() {
|
||||
continue;
|
||||
}
|
||||
cells.push((CellRef::new(origin.row + ri as u32, origin.col + ci as u32), Cell { value: oc.value.clone(), formula: None, style: id }));
|
||||
cells.push((
|
||||
CellRef::new(origin.row + ri as u32, origin.col + ci as u32),
|
||||
Cell { value: oc.value.clone(), formula: None, style: id, py: None },
|
||||
));
|
||||
}
|
||||
}
|
||||
// Column widths that fit the labels.
|
||||
|
||||
@@ -190,7 +190,7 @@ pub fn jump(sheet: &Sheet, from: CellRef, dr: i64, dc: i64) -> CellRef {
|
||||
|
||||
/// The contiguous data region around `c` (Ctrl+A / Ctrl+Shift+8, AutoSum, sort, tables).
|
||||
pub fn current_region(sheet: &Sheet, c: CellRef) -> RangeRef {
|
||||
let filled = |c: CellRef| sheet.cell(c).is_some_and(|x| !x.value.is_empty() || x.formula.is_some()) || sheet.spill.contains_key(&c);
|
||||
let filled = |c: CellRef| sheet.cell(c).is_some_and(gridcraft_model::Cell::has_content) || sheet.spill.contains_key(&c);
|
||||
// Whether a row (or column) has a filled cell between two columns (rows), inclusive.
|
||||
let row_has = |row: u32, a: u32, b: u32| (a..=b).any(|col| filled(CellRef::new(row, col)));
|
||||
let col_has = |col: u32, a: u32, b: u32| (a..=b).any(|row| filled(CellRef::new(row, col)));
|
||||
|
||||
@@ -5246,6 +5246,9 @@ fn a_script_cannot_switch_windows() {
|
||||
mod equation;
|
||||
mod macros;
|
||||
mod objects;
|
||||
mod python_cells;
|
||||
mod python_commands;
|
||||
mod python_files;
|
||||
mod seams;
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -0,0 +1,532 @@
|
||||
//! Python cells under edits: what keeps, moves, rewrites or removes their code.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use gridcraft_core::{CellRef, Value};
|
||||
use gridcraft_model::{Cell, PyCode};
|
||||
use serde_json::json;
|
||||
|
||||
use super::s;
|
||||
use crate::Session;
|
||||
|
||||
/// Makes `cell` on sheet `sheet` a Python cell with `code` and a cached value.
|
||||
fn plant(s: &mut Session, sheet: usize, cell: &str, code: &str) {
|
||||
let d = s.doc_mut().unwrap();
|
||||
let mut wb = (*d.wb).clone();
|
||||
let sh = wb.sheet_mut(sheet).unwrap();
|
||||
let at = CellRef::parse(cell).unwrap();
|
||||
let style = sh.style_id(at);
|
||||
sh.set_cell(at, Cell { value: Value::Number(1.0), formula: None, style, py: Some(Arc::new(PyCode::new(code))) });
|
||||
d.wb = Arc::new(wb);
|
||||
}
|
||||
|
||||
fn code(s: &Session, sheet: usize, cell: &str) -> Option<String> {
|
||||
let d = s.doc().unwrap();
|
||||
d.wb.sheet(sheet).unwrap().cell(CellRef::parse(cell).unwrap()).and_then(|c| c.py.as_ref()).map(|p| p.code.clone())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn typing_py_formulas_points_to_python_set() {
|
||||
let mut s = s();
|
||||
let e = s.execute("cell.set", json!({"cell": "A1", "input": "=PY(\"1+1\",0)"})).unwrap_err();
|
||||
assert!(e.to_string().contains("python.set"), "{e}");
|
||||
assert!(s.execute("cell.set", json!({"cell": "A1", "input": "=py"})).is_err());
|
||||
// Other functions starting with PY are not affected.
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "=PI()"})).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn entering_or_clearing_a_python_cell_removes_its_code() {
|
||||
let mut s = s();
|
||||
plant(&mut s, 0, "A1", "1 + 1");
|
||||
plant(&mut s, 0, "A2", "2 + 2");
|
||||
plant(&mut s, 0, "A3", "3 + 3");
|
||||
let got = s.execute("cell.get", json!({"cell": "A1"})).unwrap();
|
||||
assert_eq!(got["input"], "1 + 1");
|
||||
assert_eq!(got["python"]["code"], "1 + 1");
|
||||
assert_eq!(got["python"]["output"], "value");
|
||||
s.execute("cell.set", json!({"cell": "A1", "input": "hello"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A1"), None);
|
||||
s.execute("edit.clearFormats", json!({"range": "A2"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A2").as_deref(), Some("2 + 2"), "clearing formats keeps the code");
|
||||
s.execute("edit.clearContents", json!({"range": "A2"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A2"), None);
|
||||
s.execute("edit.clearAll", json!({"range": "A3"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A3"), None);
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A3").as_deref(), Some("3 + 3"), "undo brings the code back");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn copy_fill_and_sort_carry_the_code_and_paste_values_drops_it() {
|
||||
let mut s = s();
|
||||
plant(&mut s, 0, "A1", "xl('B1') * 2");
|
||||
s.execute("edit.copy", json!({"range": "A1"})).unwrap();
|
||||
s.execute("edit.paste", json!({"at": "C1"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "C1").as_deref(), Some("xl('B1') * 2"), "the code is copied as it is");
|
||||
s.execute("edit.pasteSpecial", json!({"at": "D1", "what": "values"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "D1"), None);
|
||||
assert_eq!(super::v(&s, "D1"), Value::Number(1.0), "the cached value is pasted");
|
||||
s.execute("edit.fillDown", json!({"range": "A1:A3"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A3").as_deref(), Some("xl('B1') * 2"));
|
||||
// Sorting moves Python cells with their rows.
|
||||
s.execute("range.setValues", json!({"range": "E1", "values": [[3], [1], [2]]})).unwrap();
|
||||
plant(&mut s, 0, "F1", "'three'");
|
||||
plant(&mut s, 0, "F2", "'one'");
|
||||
s.execute("data.sort", json!({"range": "E1:F3", "keys": [{"column": "E"}]})).unwrap();
|
||||
assert_eq!(code(&s, 0, "F1").as_deref(), Some("'one'"));
|
||||
assert_eq!(code(&s, 0, "F3").as_deref(), Some("'three'"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn xl_references_follow_inserts_moves_and_renames() {
|
||||
let mut s = s();
|
||||
s.execute("home.insertSheet", json!({})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": 0})).unwrap();
|
||||
let other = s.doc().unwrap().wb.sheet(1).unwrap().name.clone();
|
||||
plant(&mut s, 0, "C5", &format!("a = xl(\"A1:A3\")\nb = xl('{other}!B2', headers=True)\nc = xl(dynamic)\n# xl(\"A1\") stays in comments"));
|
||||
s.execute("home.insertRows", json!({"rows": "2:2"})).unwrap();
|
||||
let c = code(&s, 0, "C6").unwrap();
|
||||
assert!(c.starts_with("a = xl(\"A1:A4\")\n"), "{c}");
|
||||
assert!(c.contains(&format!("xl('{other}!B2', headers=True)")), "{c}");
|
||||
assert!(c.contains("# xl(\"A1\") stays"), "{c}");
|
||||
s.execute("sheet.rename", json!({"sheet": 1, "name": "Data"})).unwrap();
|
||||
assert!(code(&s, 0, "C6").unwrap().contains("xl('Data!B2', headers=True)"));
|
||||
// Cutting the cells a reference names moves the reference with them.
|
||||
s.execute("edit.cut", json!({"range": "A1:A4"})).unwrap();
|
||||
s.execute("edit.paste", json!({"at": "H1"})).unwrap();
|
||||
assert!(code(&s, 0, "C6").unwrap().starts_with("a = xl(\"H1:H4\")"), "{:?}", code(&s, 0, "C6"));
|
||||
// Deleting the referenced cells leaves #REF!, which fails when the code runs.
|
||||
// The selection is elsewhere, so only the named column goes.
|
||||
s.execute("selection.set", json!({"range": "A1"})).unwrap();
|
||||
s.execute("home.deleteColumns", json!({"cols": "H:H"})).unwrap();
|
||||
assert!(code(&s, 0, "C6").unwrap().starts_with("a = xl(\"#REF!\")"), "{:?}", code(&s, 0, "C6"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_rename_to_a_name_that_needs_quotes_rewrites_single_quoted_references() {
|
||||
let mut s = s();
|
||||
s.execute("home.insertSheet", json!({})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": 0})).unwrap();
|
||||
let other = s.doc().unwrap().wb.sheet(1).unwrap().name.clone();
|
||||
plant(&mut s, 0, "A1", &format!("a = xl('{other}!B2')\nb = xl(\"{other}!C3\")"));
|
||||
s.execute("sheet.rename", json!({"sheet": 1, "name": "Q4 2026"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A1").unwrap(), "a = xl(\"'Q4 2026'!B2\")\nb = xl(\"'Q4 2026'!C3\")");
|
||||
// A name with both quotes in it: the literal's own quote is escaped.
|
||||
s.execute("sheet.rename", json!({"sheet": 1, "name": "Bob's \"Q4\""})).unwrap();
|
||||
let c = code(&s, 0, "A1").unwrap();
|
||||
assert_eq!(c, "a = xl(\"'Bob''s \\\"Q4\\\"'!B2\")\nb = xl(\"'Bob''s \\\"Q4\\\"'!C3\")");
|
||||
let refs: Vec<Option<String>> = gridcraft_model::python::xl_calls(&c).into_iter().map(|x| x.reference).collect();
|
||||
assert_eq!(refs, [Some("'Bob''s \"Q4\"'!B2".to_string()), Some("'Bob''s \"Q4\"'!C3".to_string())]);
|
||||
// And back to a plain name.
|
||||
s.execute("sheet.rename", json!({"sheet": 1, "name": "Data"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A1").unwrap(), "a = xl(\"Data!B2\")\nb = xl(\"Data!C3\")");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_cut_to_another_sheet_keeps_references_on_the_source_sheet() {
|
||||
let mut s = s();
|
||||
s.execute("home.insertSheet", json!({})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": 0})).unwrap();
|
||||
let (src, dest) = {
|
||||
let wb = &s.doc().unwrap().wb;
|
||||
(wb.sheet(0).unwrap().name.clone(), wb.sheet(1).unwrap().name.clone())
|
||||
};
|
||||
plant(&mut s, 0, "A1", "a = xl(\"B5\")\nb = xl('A2')");
|
||||
plant(&mut s, 0, "A2", "1");
|
||||
s.execute("edit.cut", json!({"range": "A1:A2"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": 1})).unwrap();
|
||||
s.execute("edit.paste", json!({"at": "C1"})).unwrap();
|
||||
// B5 stays on the source sheet; A2 moved with the cut, so it follows to the destination.
|
||||
assert_eq!(code(&s, 1, "C1").unwrap(), format!("a = xl(\"{src}!B5\")\nb = xl('{dest}!C2')"));
|
||||
assert_eq!(code(&s, 0, "A1"), None);
|
||||
// Within one sheet nothing is qualified.
|
||||
s.execute("edit.cut", json!({"range": "C1"})).unwrap();
|
||||
s.execute("edit.paste", json!({"at": "E1"})).unwrap();
|
||||
assert_eq!(code(&s, 1, "E1").unwrap(), format!("a = xl(\"{src}!B5\")\nb = xl('{dest}!C2')"));
|
||||
// A copy to another sheet still carries the code as it is.
|
||||
plant(&mut s, 0, "A9", "xl(\"B5\")");
|
||||
s.execute("sheet.activate", json!({"sheet": 0})).unwrap();
|
||||
s.execute("edit.copy", json!({"range": "A9"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": 1})).unwrap();
|
||||
s.execute("edit.paste", json!({"at": "A9"})).unwrap();
|
||||
assert_eq!(code(&s, 1, "A9").as_deref(), Some("xl(\"B5\")"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn goal_seek_and_solver_refuse_to_change_a_python_cell() {
|
||||
let mut s = s();
|
||||
plant(&mut s, 0, "A1", "2");
|
||||
s.execute("cell.set", json!({"cell": "B1", "input": "=A1*2"})).unwrap();
|
||||
let e = s.execute("data.goalSeek", json!({"set": "B1", "to": 10, "changing": "A1"})).unwrap_err();
|
||||
assert!(e.to_string().contains("must contain a value"), "{e}");
|
||||
assert_eq!(code(&s, 0, "A1").as_deref(), Some("2"), "the code is kept");
|
||||
// The set cell must be a formula: a Python cell isn't one this build can recalculate.
|
||||
s.execute("cell.set", json!({"cell": "C1", "input": "3"})).unwrap();
|
||||
assert!(s.execute("data.goalSeek", json!({"set": "A1", "to": 10, "changing": "C1"})).is_err());
|
||||
let e = s.execute("data.solver", json!({"objective": "B1", "goal": "max", "variables": "A1"})).unwrap_err();
|
||||
assert!(e.to_string().contains("constants"), "{e}");
|
||||
assert_eq!(code(&s, 0, "A1").as_deref(), Some("2"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn text_to_columns_keeps_a_py_piece_as_text() {
|
||||
let mut s = s();
|
||||
s.execute("cell.set", json!({"cell": "A1", "input": "'=PY(\"x\");7"})).unwrap();
|
||||
assert_eq!(super::v(&s, "A1"), Value::text("=PY(\"x\");7"));
|
||||
s.execute("data.textToColumns", json!({"range": "A1", "delimiters": [";"]})).unwrap();
|
||||
// The default text qualifier takes the quotes away, as in Excel.
|
||||
assert_eq!(super::v(&s, "A1"), Value::text("=PY(x)"), "the piece is kept, not cleared");
|
||||
assert_eq!(code(&s, 0, "A1"), None, "and it is not a Python cell");
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(7.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn python_cells_survive_native_json() {
|
||||
let mut s = s();
|
||||
plant(&mut s, 0, "B2", "x = 1\nx");
|
||||
let wb = s.doc().unwrap().wb.clone();
|
||||
let json = serde_json::to_string(&*wb).unwrap();
|
||||
let back: gridcraft_model::Workbook = serde_json::from_str(&json).unwrap();
|
||||
let cell = back.sheet(0).unwrap().cell(CellRef::parse("B2").unwrap()).unwrap();
|
||||
assert_eq!(cell.py.as_ref().unwrap().code, "x = 1\nx");
|
||||
// Cells written before Python existed have no `py` field.
|
||||
let plain: Cell = serde_json::from_value(json!({"value": {"t": "Number", "v": 2.0}, "formula": null, "style": 0})).unwrap();
|
||||
assert!(plain.py.is_none());
|
||||
assert!(!serde_json::to_string(&plain).unwrap().contains("py"), "no field is written for ordinary cells");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn new_errors_parse_and_have_error_type_numbers() {
|
||||
use gridcraft_core::CellError;
|
||||
for (text, e) in
|
||||
[("#PYTHON!", CellError::Python), ("#BUSY!", CellError::Busy), ("#TIMEOUT!", CellError::Timeout), ("#BLOCKED!", CellError::Blocked)]
|
||||
{
|
||||
assert_eq!(CellError::parse(text), Some(e));
|
||||
assert_eq!(e.as_str(), text);
|
||||
}
|
||||
let mut s = s();
|
||||
s.execute("cell.set", json!({"cell": "A1", "input": "=ERROR.TYPE(#BLOCKED!)"})).unwrap();
|
||||
assert_eq!(super::v(&s, "A1"), Value::Number(11.0));
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "=ISERROR(#PYTHON!)"})).unwrap();
|
||||
assert_eq!(super::v(&s, "A2"), Value::Bool(true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn go_to_special_counts_a_python_cell_as_a_formula_not_a_constant() {
|
||||
let mut s = s();
|
||||
plant(&mut s, 0, "A1", "1 + 1");
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "=1+1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A3", "input": "7"})).unwrap();
|
||||
s.execute("selection.set", json!({"range": "A1"})).unwrap();
|
||||
let r = s.execute("edit.goToSpecial", json!({"kind": "formulas"})).unwrap();
|
||||
assert_eq!(r["count"], 2, "{r}");
|
||||
assert_eq!(r["selection"], "A1:A2", "{r}");
|
||||
s.execute("selection.set", json!({"range": "A1"})).unwrap();
|
||||
let r = s.execute("edit.goToSpecial", json!({"kind": "constants"})).unwrap();
|
||||
assert_eq!(r["count"], 1, "{r}");
|
||||
assert_eq!(r["selection"], "A3", "{r}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_macro_counts_python_code_as_data_and_a_blocked_spill_as_no_write() {
|
||||
let mut s = s();
|
||||
// F2 shows 1, its cached value: a step that enters 1 there still replaces the code.
|
||||
plant(&mut s, 0, "F2", "1");
|
||||
// H2's code shows #BLOCKED!, as any other new code there would.
|
||||
s.execute("python.set", json!({"cell": "H2", "code": "'a'"})).unwrap();
|
||||
// J2's result spills over J2:J3. A step that writes J3 blocks it, which doesn't write J2.
|
||||
s.execute("python.set", json!({"cell": "J2", "code": "[5, 6]"})).unwrap();
|
||||
{
|
||||
let d = s.doc_mut().unwrap();
|
||||
let wb = Arc::make_mut(&mut d.wb);
|
||||
let cell = wb.sheet_mut(0).unwrap().cells.get_mut(CellRef::parse("J2").unwrap()).unwrap();
|
||||
let data = vec![Value::Number(5.0), Value::Number(6.0)];
|
||||
let value = Value::Array(Arc::new(gridcraft_core::Array::new(2, 1, data).unwrap()));
|
||||
Arc::make_mut(cell.py.as_mut().unwrap()).output =
|
||||
Some(gridcraft_model::PyOutput { value, type_name: "list".into(), preview: String::new(), convertible: true });
|
||||
}
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(super::v(&s, "J3"), Value::Number(6.0));
|
||||
super::macros::put(
|
||||
&mut s,
|
||||
json!({"name": "Py", "steps": [
|
||||
{"command": "cell.set", "params": {"cell": "F2", "input": "1"}},
|
||||
{"command": "python.set", "params": {"cell": "H2", "code": "'b'"}},
|
||||
{"command": "cell.set", "params": {"cell": "J3", "input": "x"}}
|
||||
]}),
|
||||
);
|
||||
let c = s.execute("macro.check", json!({"name": "Py"})).unwrap();
|
||||
assert_eq!(c["footprint"]["sheets"][0]["cells"], json!(["F2", "H2", "J3"]), "{c}");
|
||||
assert_eq!(c["footprint"]["sheets"][0]["conflicts"], json!(["F2", "H2"]), "{c}");
|
||||
let e = s.execute("macro.run", json!({"name": "Py", "onConflict": "fail"})).unwrap_err().to_string();
|
||||
assert!(e.contains("F2") && e.contains("H2") && !e.contains("J2"), "{e}");
|
||||
assert_eq!(code(&s, 0, "F2").as_deref(), Some("1"), "nothing was changed");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replace_edits_a_python_cells_code_and_keeps_it_a_python_cell() {
|
||||
let mut s = s();
|
||||
plant(&mut s, 0, "A1", "x = 1\nx");
|
||||
{
|
||||
let d = s.doc_mut().unwrap();
|
||||
let mut wb = (*d.wb).clone();
|
||||
let cell = wb.sheet_mut(0).unwrap().cells.get_mut(CellRef::parse("A1").unwrap()).unwrap();
|
||||
Arc::make_mut(cell.py.as_mut().unwrap()).object = true;
|
||||
d.wb = Arc::new(wb);
|
||||
}
|
||||
let r = s.execute("edit.replace", json!({"what": "x", "with": "y"})).unwrap();
|
||||
assert_eq!(r["replaced"], 1);
|
||||
assert_eq!(code(&s, 0, "A1").as_deref(), Some("y = 1\ny"));
|
||||
let d = s.doc().unwrap();
|
||||
let cell = d.wb.sheet(0).unwrap().cell(CellRef::parse("A1").unwrap()).unwrap();
|
||||
assert!(cell.py.as_ref().unwrap().object, "the return type is kept");
|
||||
assert_eq!(cell.value, Value::Error(gridcraft_core::CellError::Blocked));
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A1").as_deref(), Some("x = 1\nx"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replace_leaves_python_code_that_would_no_longer_fit() {
|
||||
let mut s = s();
|
||||
let long = "a".repeat(gridcraft_model::python::MAX_FORMULA_CHARS / 2);
|
||||
plant(&mut s, 0, "A1", &long);
|
||||
let r = s.execute("edit.replace", json!({"what": "a", "with": "bb"})).unwrap();
|
||||
assert_eq!(r["replaced"], 0);
|
||||
assert_eq!(r["skipped"], 1);
|
||||
assert_eq!(code(&s, 0, "A1").as_deref(), Some(long.as_str()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_hidden_python_cell_on_a_protected_sheet_hides_its_code() {
|
||||
let mut s = s();
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "secret = 1\nsecret"})).unwrap();
|
||||
s.execute("python.set", json!({"cell": "B1", "code": "'shown'"})).unwrap();
|
||||
s.execute("home.formatCells", json!({"range": "A1", "style": {"protection": {"locked": true, "hidden": true}}})).unwrap();
|
||||
let hidden = |s: &Session, a: &str| {
|
||||
let d = s.doc().unwrap();
|
||||
let sh = d.wb.active().unwrap();
|
||||
crate::display::formula_hidden(&d.wb, sh, CellRef::parse(a).unwrap())
|
||||
};
|
||||
assert!(!hidden(&s, "A1"), "an unprotected sheet hides nothing");
|
||||
s.execute("review.protectSheet", json!({})).unwrap();
|
||||
assert!(hidden(&s, "A1"), "Hidden on a protected sheet hides the code, as it hides a formula");
|
||||
assert!(!hidden(&s, "B1"));
|
||||
// Editing it is refused where a formula's would be: the cell is locked.
|
||||
assert!(s.execute("python.set", json!({"cell": "A1", "code": "2"})).is_err());
|
||||
assert_eq!(code(&s, 0, "A1").as_deref(), Some("secret = 1\nsecret"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_merge_keeps_only_the_upper_left_cells_python_code() {
|
||||
let mut s = s();
|
||||
for (cmd, row) in [("home.mergeAcross", 1), ("home.mergeCells", 2), ("home.mergeCenter", 3)] {
|
||||
plant(&mut s, 0, &format!("A{row}"), "'kept'");
|
||||
plant(&mut s, 0, &format!("B{row}"), "'merged away'");
|
||||
s.execute(cmd, json!({"range": format!("A{row}:B{row}")})).unwrap();
|
||||
assert_eq!(code(&s, 0, &format!("A{row}")).as_deref(), Some("'kept'"), "{cmd}");
|
||||
assert_eq!(code(&s, 0, &format!("B{row}")), None, "{cmd}: the hidden cell is no Python cell");
|
||||
assert_eq!(super::v(&s, &format!("B{row}")), Value::Empty, "{cmd}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unmerging_lets_a_blocked_python_result_spill_again() {
|
||||
let mut s = s();
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "[5, 6]"})).unwrap();
|
||||
{
|
||||
let d = s.doc_mut().unwrap();
|
||||
let wb = Arc::make_mut(&mut d.wb);
|
||||
let cell = wb.sheet_mut(0).unwrap().cells.get_mut(CellRef::parse("A1").unwrap()).unwrap();
|
||||
let value = Value::Array(Arc::new(gridcraft_core::Array::new(2, 1, vec![Value::Number(5.0), Value::Number(6.0)]).unwrap()));
|
||||
Arc::make_mut(cell.py.as_mut().unwrap()).output =
|
||||
Some(gridcraft_model::PyOutput { value, type_name: "list".into(), preview: String::new(), convertible: true });
|
||||
}
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(super::v(&s, "A2"), Value::Number(6.0));
|
||||
s.execute("home.mergeCells", json!({"range": "A2:B2"})).unwrap();
|
||||
assert_eq!(super::v(&s, "A1"), Value::Error(gridcraft_core::CellError::Spill), "a merged cell blocks the spill");
|
||||
s.execute("home.unmergeCells", json!({"range": "A2:B2"})).unwrap();
|
||||
assert_eq!((super::v(&s, "A1"), super::v(&s, "A2")), (Value::Number(5.0), Value::Number(6.0)), "it spills again");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn save_model_asks_before_it_writes_over_a_python_cell_that_shows_nothing() {
|
||||
let mut s = s();
|
||||
for (cell, input) in [("B1", "=5*A1+4*A2"), ("B2", "=6*A1+4*A2")] {
|
||||
s.execute("cell.set", json!({"cell": cell, "input": input})).unwrap();
|
||||
}
|
||||
// H2 is a Python cell whose cached value is empty: its code is still content.
|
||||
{
|
||||
let d = s.doc_mut().unwrap();
|
||||
let wb = Arc::make_mut(&mut d.wb);
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
sh.set_cell(
|
||||
CellRef::parse("H2").unwrap(),
|
||||
Cell { value: Value::Empty, formula: None, style: Default::default(), py: Some(Arc::new(PyCode::new("None"))) },
|
||||
);
|
||||
}
|
||||
let model = json!({"objective": "B1", "variables": "A1:A2", "constraints": ["B2 <= 24"]});
|
||||
let e = s.execute("data.solverSaveModel", json!({"to": "H1", "model": model})).unwrap_err().to_string();
|
||||
assert!(e.contains(crate::cmd::analysis::OVERWRITE_ERROR) && e.contains("H2"), "{e}");
|
||||
assert_eq!(code(&s, 0, "H2").as_deref(), Some("None"), "nothing was written");
|
||||
}
|
||||
|
||||
/// A1 is a Python cell whose kept result `[5, 6]` spills over A1:A2, and B1 reads A2.
|
||||
pub(super) fn spill_with_a_reader() -> crate::Session {
|
||||
let mut s = s();
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "[5, 6]"})).unwrap();
|
||||
{
|
||||
let d = s.doc_mut().unwrap();
|
||||
let wb = Arc::make_mut(&mut d.wb);
|
||||
let cell = wb.sheet_mut(0).unwrap().cells.get_mut(CellRef::parse("A1").unwrap()).unwrap();
|
||||
let value = Value::Array(Arc::new(gridcraft_core::Array::new(2, 1, vec![Value::Number(5.0), Value::Number(6.0)]).unwrap()));
|
||||
Arc::make_mut(cell.py.as_mut().unwrap()).output =
|
||||
Some(gridcraft_model::PyOutput { value, type_name: "list".into(), preview: String::new(), convertible: true });
|
||||
}
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "B1", "input": "=SUM(A2)"})).unwrap();
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(6.0));
|
||||
s
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn new_python_code_recalculates_the_readers_of_its_old_spill() {
|
||||
let mut s = spill_with_a_reader();
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "[7, 8]"})).unwrap();
|
||||
assert_eq!(super::v(&s, "A2"), Value::Empty, "new code has no result yet");
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(0.0), "B1 read the old spill");
|
||||
// The same code again keeps its result, and its spill.
|
||||
let mut s = spill_with_a_reader();
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "[5, 6]"})).unwrap();
|
||||
assert_eq!((super::v(&s, "A2"), super::v(&s, "B1")), (Value::Number(6.0), Value::Number(6.0)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replace_in_python_code_recalculates_the_readers_of_its_old_spill() {
|
||||
let mut s = spill_with_a_reader();
|
||||
let r = s.execute("edit.replace", json!({"what": "[5,", "with": "[7,"})).unwrap();
|
||||
assert_eq!(r["replaced"], 1);
|
||||
assert_eq!(super::v(&s, "A2"), Value::Empty);
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(0.0), "B1 read the old spill");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn clearing_pasting_over_or_blocking_a_python_spill_recalculates_its_readers() {
|
||||
let mut s = spill_with_a_reader();
|
||||
s.execute("edit.clearContents", json!({"range": "A1"})).unwrap();
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(0.0), "clear");
|
||||
let mut s = spill_with_a_reader();
|
||||
s.execute("cell.set", json!({"cell": "D1", "input": "9"})).unwrap();
|
||||
s.execute("edit.copy", json!({"range": "D1"})).unwrap();
|
||||
s.execute("edit.paste", json!({"at": "A1"})).unwrap();
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(0.0), "paste");
|
||||
let mut s = spill_with_a_reader();
|
||||
s.execute("home.mergeCells", json!({"range": "A2:A3"})).unwrap();
|
||||
assert_eq!(super::v(&s, "A1"), Value::Error(gridcraft_core::CellError::Spill));
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(0.0), "merge");
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(6.0), "undo");
|
||||
s.execute("edit.redo", json!({})).unwrap();
|
||||
assert_eq!(super::v(&s, "B1"), Value::Number(0.0), "redo");
|
||||
}
|
||||
|
||||
/// `prefix` and a comment padded to the longest code that still fits in one cell.
|
||||
fn at_the_limit(prefix: &str) -> String {
|
||||
let with = |n: usize| format!("{prefix}\n#{}", "x".repeat(n));
|
||||
let (mut lo, mut hi) = (0, gridcraft_model::python::MAX_FORMULA_CHARS);
|
||||
while lo < hi {
|
||||
let mid = (lo + hi).div_ceil(2);
|
||||
if PyCode::new(&with(mid)).fits() { lo = mid } else { hi = mid - 1 }
|
||||
}
|
||||
with(lo)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_edit_that_would_make_python_code_too_long_is_refused() {
|
||||
let mut s = s();
|
||||
s.execute("home.insertSheet", json!({})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": 0})).unwrap();
|
||||
let other = s.doc().unwrap().wb.sheet(1).unwrap().name.clone();
|
||||
let refused = |e: crate::EngineError, cell: &str| {
|
||||
let e = e.to_string();
|
||||
assert!(e.contains(&format!("!{cell} longer than Excel's 8192-character")) && e.contains("nothing was changed"), "{e}");
|
||||
};
|
||||
// A sheet rename that lengthens a reference.
|
||||
let a1 = at_the_limit(&format!("a = xl(\"{other}!B2\")"));
|
||||
plant(&mut s, 0, "A1", &a1);
|
||||
refused(s.execute("sheet.rename", json!({"sheet": 1, "name": "A much longer sheet name"})).unwrap_err(), "A1");
|
||||
assert_eq!(code(&s, 0, "A1").unwrap(), a1);
|
||||
assert_eq!(s.doc().unwrap().wb.sheet(1).unwrap().name, other);
|
||||
// One of the same length is made.
|
||||
let same = "Z".repeat(other.chars().count());
|
||||
s.execute("sheet.rename", json!({"sheet": 1, "name": same})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A1").unwrap(), a1.replace(&other, &same));
|
||||
// Code short of the limit still follows each edit.
|
||||
plant(&mut s, 0, "A7", "d = xl(\"Z1\")");
|
||||
s.execute("selection.set", json!({"range": "B10"})).unwrap();
|
||||
s.execute("home.insertColumns", json!({"cols": "Y:Y"})).unwrap();
|
||||
assert_eq!(code(&s, 0, "A7").unwrap(), "d = xl(\"AA1\")");
|
||||
// An inserted column that turns Z1 into AA1.
|
||||
let a3 = at_the_limit("b = xl(\"Z1\")");
|
||||
plant(&mut s, 0, "A3", &a3);
|
||||
s.execute("cell.set", json!({"cell": "D1", "input": "kept"})).unwrap();
|
||||
s.execute("selection.set", json!({"range": "C1"})).unwrap();
|
||||
refused(s.execute("home.insertColumns", json!({"cols": "C:C"})).unwrap_err(), "A3");
|
||||
assert_eq!(code(&s, 0, "A3").unwrap(), a3);
|
||||
assert_eq!(s.doc().unwrap().wb.sheet(0).unwrap().value(CellRef::parse("D1").unwrap()), Value::text("kept"));
|
||||
// A cut to another sheet, which qualifies B9 with the source sheet.
|
||||
let a5 = at_the_limit("c = xl(\"B9\")");
|
||||
plant(&mut s, 0, "A5", &a5);
|
||||
s.execute("edit.cut", json!({"range": "A5"})).unwrap();
|
||||
s.execute("sheet.activate", json!({"sheet": 1})).unwrap();
|
||||
refused(s.execute("edit.paste", json!({"at": "C1"})).unwrap_err(), "C1");
|
||||
assert_eq!(code(&s, 0, "A5").unwrap(), a5);
|
||||
assert_eq!(code(&s, 1, "C1"), None);
|
||||
}
|
||||
|
||||
/// Makes `cell` a Python cell whose cached value is empty, as a file's `PY` formula with no cached
|
||||
/// value reads.
|
||||
fn plant_empty(s: &mut Session, cell: &str) {
|
||||
plant(s, 0, cell, "x = 1");
|
||||
let d = s.doc_mut().unwrap();
|
||||
let mut wb = (*d.wb).clone();
|
||||
if let Some(c) = wb.sheet_mut(0).unwrap().cells.get_mut(CellRef::parse(cell).unwrap()) {
|
||||
c.value = Value::Empty;
|
||||
}
|
||||
d.wb = Arc::new(wb);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn python_code_with_an_empty_value_counts_as_content() {
|
||||
// A report or a PivotTable asks before replacing it.
|
||||
let mut s = s();
|
||||
s.execute("range.setValues", json!({"range": "A1", "values": [["Region", "Sales"], ["East", 1], ["West", 2], ["East", 3]]})).unwrap();
|
||||
plant_empty(&mut s, "D2");
|
||||
let report = json!({"tool": "Descriptive Statistics", "range": "B1:B4", "labels": true, "output": "D1"});
|
||||
assert!(s.execute("data.analysisToolPak", report).is_err(), "the report asks for overwrite");
|
||||
plant_empty(&mut s, "H4");
|
||||
let pivot = s.execute("insert.pivotTable", json!({"destination": "Sheet1!H3", "rows": ["Region"], "values": ["Sales"]}));
|
||||
assert!(pivot.is_err_and(|e| e.to_string().contains("already data")));
|
||||
assert_eq!((code(&s, 0, "D2"), code(&s, 0, "H4")), (Some("x = 1".into()), Some("x = 1".into())), "the code is kept");
|
||||
// Cells aren't pushed off the sheet over it.
|
||||
plant_empty(&mut s, "K1048576");
|
||||
assert!(s.execute("home.insertCells", json!({"range": "K1", "shift": "down"})).is_err());
|
||||
assert_eq!(code(&s, 0, "K1048576"), Some("x = 1".into()));
|
||||
// A table without a header row doesn't push it off the sheet either.
|
||||
s.execute("range.setValues", json!({"range": "X1", "values": [[1], [2]]})).unwrap();
|
||||
plant_empty(&mut s, "X1048576");
|
||||
assert!(s.execute("insert.table", json!({"range": "X1:X2", "header": false})).is_err());
|
||||
assert_eq!(code(&s, 0, "X1048576"), Some("x = 1".into()));
|
||||
// It is part of the data region.
|
||||
s.execute("range.setValues", json!({"range": "P1", "values": [[1], [""], [3]]})).unwrap();
|
||||
plant_empty(&mut s, "P2");
|
||||
let sh = s.doc().unwrap().wb.sheet(0).unwrap().clone();
|
||||
assert_eq!(crate::selection::current_region(&sh, CellRef::parse("P1").unwrap()).a1(), "P1:P3");
|
||||
// Pasted into part of a legacy array, it is refused like any content.
|
||||
s.execute("selection.set", json!({"range": "R1:R3"})).unwrap();
|
||||
s.execute("cell.set", json!({"input": "=5", "array": true})).unwrap();
|
||||
plant_empty(&mut s, "T1");
|
||||
s.execute("edit.copy", json!({"range": "T1"})).unwrap();
|
||||
assert!(s.execute("edit.paste", json!({"at": "R2"})).is_err_and(|e| e.to_string().contains("part of an array")));
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
//! The python.* commands.
|
||||
|
||||
use gridcraft_core::{CellError, Value};
|
||||
use serde_json::json;
|
||||
|
||||
use super::{s, v};
|
||||
|
||||
fn set(s: &mut crate::Session, cell: &str, code: &str) -> serde_json::Value {
|
||||
s.execute("python.set", json!({"cell": cell, "code": code})).unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_stores_the_code_and_blocks_the_cell_until_it_can_run() {
|
||||
let mut s = s();
|
||||
let r = set(&mut s, "B1", "xl('A1') + 1");
|
||||
assert_eq!(r["cell"], "B1");
|
||||
assert_eq!(v(&s, "B1"), Value::Error(CellError::Blocked));
|
||||
let got = s.execute("cell.get", json!({"cell": "B1"})).unwrap();
|
||||
assert_eq!(got["python"]["code"], "xl('A1') + 1");
|
||||
assert_eq!(got["python"]["output"], "value");
|
||||
assert_eq!(got["input"], "xl('A1') + 1");
|
||||
// The active cell by default.
|
||||
s.execute("selection.set", json!({"cell": "D4"})).unwrap();
|
||||
s.execute("python.set", json!({"code": "2"})).unwrap();
|
||||
assert_eq!(s.execute("cell.get", json!({"cell": "D4"})).unwrap()["python"]["code"], "2");
|
||||
// One undo step takes the cell back.
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
assert!(s.execute("cell.get", json!({"cell": "D4"})).unwrap()["python"].is_null());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unchanged_code_keeps_its_cached_result() {
|
||||
let mut s = s();
|
||||
set(&mut s, "A1", "21 * 2");
|
||||
// As a file would have cached it.
|
||||
{
|
||||
let d = s.doc_mut().unwrap();
|
||||
let wb = std::sync::Arc::make_mut(&mut d.wb);
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
let cell = sh.cells.get_mut(gridcraft_core::CellRef::new(0, 0)).unwrap();
|
||||
let py = std::sync::Arc::make_mut(cell.py.as_mut().unwrap());
|
||||
py.output = Some(gridcraft_model::PyOutput { value: Value::Number(42.0), type_name: "int".into(), preview: "42".into(), convertible: true });
|
||||
cell.value = Value::Number(42.0);
|
||||
}
|
||||
set(&mut s, "A1", "21 * 2");
|
||||
assert_eq!(v(&s, "A1"), Value::Number(42.0), "same code, same result");
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "21 * 2", "output": "object"})).unwrap();
|
||||
assert_eq!(s.execute("cell.get", json!({"cell": "A1"})).unwrap()["python"]["output"], "object");
|
||||
set(&mut s, "A1", "21 * 3");
|
||||
assert_eq!(v(&s, "A1"), Value::Error(CellError::Blocked), "new code has no result yet");
|
||||
assert_eq!(s.execute("cell.get", json!({"cell": "A1"})).unwrap()["python"]["output"], "object", "the return type stays");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_refuses_bad_params_long_code_headers_and_protected_cells() {
|
||||
let mut s = s();
|
||||
for p in [
|
||||
json!({"cell": "A1"}),
|
||||
json!({"cell": "A1", "code": 5}),
|
||||
json!({"cell": "not a cell", "code": "1"}),
|
||||
json!({"cell": "A1", "code": "1", "output": "sideways"}),
|
||||
] {
|
||||
assert!(s.execute("python.set", p.clone()).is_err(), "{p}");
|
||||
}
|
||||
// The limit is the stored PY(…) formula's 8,192 characters.
|
||||
assert!(s.execute("python.set", json!({"cell": "A1", "code": "y".repeat(8_080)})).is_err());
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "y".repeat(8_079)})).unwrap();
|
||||
let quoted = format!("{}{}", "\"".repeat(1_200), "y".repeat(5_800));
|
||||
let e = s.execute("python.set", json!({"cell": "A2", "code": quoted})).unwrap_err();
|
||||
assert!(e.to_string().contains("8192") || e.to_string().contains("8,192"), "{e}");
|
||||
// A table header is the column's name.
|
||||
s.execute("range.setValues", json!({"range": "C1", "values": [["Name", "Score"], ["a", 1]]})).unwrap();
|
||||
s.execute("insert.table", json!({"range": "C1:D2", "headers": true})).unwrap();
|
||||
assert!(s.execute("python.set", json!({"cell": "D1", "code": "1"})).is_err());
|
||||
assert_eq!(v(&s, "D1"), Value::text("Score"));
|
||||
set(&mut s, "D2", "1");
|
||||
// A locked cell on a protected sheet refuses it.
|
||||
s.execute("review.protectSheet", json!({})).unwrap();
|
||||
let e = s.execute("python.set", json!({"cell": "F1", "code": "1"})).unwrap_err();
|
||||
assert!(e.to_string().contains("protected"), "{e}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn set_over_a_spilling_formula_takes_the_spill_away() {
|
||||
let mut s = s();
|
||||
s.execute("cell.set", json!({"cell": "A1", "input": "=SEQUENCE(3)"})).unwrap();
|
||||
assert_eq!(v(&s, "A3"), Value::Number(3.0));
|
||||
set(&mut s, "A1", "[1, 2, 3]");
|
||||
assert_eq!(v(&s, "A3"), Value::Empty);
|
||||
assert!(s.doc().unwrap().wb.sheet(0).unwrap().spill_ranges.is_empty());
|
||||
}
|
||||
@@ -0,0 +1,744 @@
|
||||
//! Python cells in files: native JSON, and xlsx as `_xlfn._xlws.PY(index,0|1)` with the code in
|
||||
//! the Python part ([MS-XLSX] 2.1.23, 2.3.10) and cached values. Nothing here runs Python: results
|
||||
//! are cached ones, as a file would hold them.
|
||||
|
||||
use std::io::{Read, Write};
|
||||
use std::sync::Arc;
|
||||
|
||||
use gridcraft_core::{Array, CellError, CellRef, RangeRef, Value};
|
||||
use gridcraft_model::{Cell, PyCode, PyOutput, Workbook};
|
||||
use serde_json::json;
|
||||
|
||||
use super::{s, v};
|
||||
use crate::{Session, Trust};
|
||||
|
||||
fn py(s: &mut Session, cell: &str, code: &str) {
|
||||
s.execute("python.set", json!({"cell": cell, "code": code})).unwrap();
|
||||
}
|
||||
|
||||
/// Gives a Python cell a result, as running it (or a file's cache) would.
|
||||
fn cache(s: &mut Session, cell: &str, value: Value, type_name: &str) {
|
||||
let d = s.doc_mut().unwrap();
|
||||
let wb = Arc::make_mut(&mut d.wb);
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
let c = sh.cells.get_mut(CellRef::parse(cell).unwrap()).unwrap();
|
||||
let p = Arc::make_mut(c.py.as_mut().unwrap());
|
||||
p.output = Some(PyOutput { value, type_name: type_name.into(), preview: String::new(), convertible: true });
|
||||
}
|
||||
|
||||
fn array(rows: usize, cols: usize, data: Vec<Value>) -> Value {
|
||||
Value::Array(Arc::new(Array::new(rows, cols, data).unwrap()))
|
||||
}
|
||||
|
||||
fn save(s: &mut Session, format: &str) -> Vec<u8> {
|
||||
let r = s.execute("file.saveBytes", json!({"format": format})).unwrap();
|
||||
crate::io::base64_decode(r["base64"].as_str().unwrap()).unwrap()
|
||||
}
|
||||
|
||||
fn open(s: &mut Session, name: &str, bytes: &[u8]) {
|
||||
s.execute("file.open", json!({"name": name, "base64": crate::io::base64_encode(bytes)})).unwrap();
|
||||
}
|
||||
|
||||
fn part(bytes: &[u8], name: &str) -> String {
|
||||
let mut z = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
|
||||
let mut xml = String::new();
|
||||
z.by_name(name).unwrap().read_to_string(&mut xml).unwrap();
|
||||
xml
|
||||
}
|
||||
|
||||
fn sheet_xml(bytes: &[u8]) -> String {
|
||||
part(bytes, "xl/worksheets/sheet1.xml")
|
||||
}
|
||||
|
||||
/// The package with `edit` applied to each XML part (by name), and `add` added.
|
||||
fn with_parts(bytes: &[u8], edit: impl Fn(&str, &str) -> String, add: &[(&str, &str)]) -> Vec<u8> {
|
||||
let mut z = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
|
||||
let mut out = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
|
||||
for i in 0..z.len() {
|
||||
let mut f = z.by_index(i).unwrap();
|
||||
let name = f.name().to_string();
|
||||
let mut data = Vec::new();
|
||||
f.read_to_end(&mut data).unwrap();
|
||||
if let Ok(text) = std::str::from_utf8(&data) {
|
||||
data = edit(&name, text).into_bytes();
|
||||
}
|
||||
out.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
|
||||
out.write_all(&data).unwrap();
|
||||
}
|
||||
for (name, text) in add {
|
||||
out.start_file(*name, zip::write::SimpleFileOptions::default()).unwrap();
|
||||
out.write_all(text.as_bytes()).unwrap();
|
||||
}
|
||||
out.finish().unwrap().into_inner()
|
||||
}
|
||||
|
||||
/// The package with `edit` applied to the first sheet's XML.
|
||||
fn with_sheet_xml(bytes: &[u8], edit: impl Fn(&str) -> String) -> Vec<u8> {
|
||||
with_parts(bytes, |name, text| if name == "xl/worksheets/sheet1.xml" { edit(text) } else { text.to_string() }, &[])
|
||||
}
|
||||
|
||||
/// `xml` with its `<sheetData>` replaced by `data`.
|
||||
fn replace_sheet_data(xml: &str, data: &str) -> String {
|
||||
match (xml.find("<sheetData"), xml.find("</sheetData>"), xml.find("<sheetData/>")) {
|
||||
(Some(a), Some(b), _) => format!("{}{data}{}", &xml[..a], &xml[b + "</sheetData>".len()..]),
|
||||
(_, _, Some(a)) => format!("{}{data}{}", &xml[..a], &xml[a + "<sheetData/>".len()..]),
|
||||
_ => panic!("no sheetData in {xml}"),
|
||||
}
|
||||
}
|
||||
|
||||
/// An empty workbook's package with `data` as its first sheet's cells and, when `scripts` is
|
||||
/// given, a Python part holding them (the relationship and the content type included), as
|
||||
/// [MS-XLSX] 2.1.23 describes it.
|
||||
fn excel_shaped(data: &str, scripts: Option<&[&str]>) -> Vec<u8> {
|
||||
let mut s = s();
|
||||
let plain = save(&mut s, "xlsx");
|
||||
let Some(scripts) = scripts else { return with_sheet_xml(&plain, |xml| replace_sheet_data(xml, data)) };
|
||||
let codes: String = scripts.iter().map(|c| format!("<pythonScript><code>{c}</code></pythonScript>")).collect();
|
||||
let python = format!(
|
||||
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<python xmlns=\"http://schemas.microsoft.com/office/spreadsheetml/2023/python\"><environmentDefinition id=\"{{00000000-0000-0000-0000-000000000000}}\"/><pythonScripts>{codes}</pythonScripts></python>"
|
||||
);
|
||||
const REL: &str =
|
||||
"<Relationship Id=\"rIdPy\" Type=\"http://schemas.microsoft.com/office/2023/09/relationships/Python\" Target=\"python/python1.xml\"/>";
|
||||
const TYPE: &str = "<Override PartName=\"/xl/python/python1.xml\" ContentType=\"application/vnd.ms-excel.python+xml\"/>";
|
||||
let edit = |name: &str, text: &str| match name {
|
||||
"xl/worksheets/sheet1.xml" => replace_sheet_data(text, data),
|
||||
"xl/_rels/workbook.xml.rels" => text.replace("</Relationships>", &format!("{REL}</Relationships>")),
|
||||
"[Content_Types].xml" => text.replace("</Types>", &format!("{TYPE}</Types>")),
|
||||
_ => text.to_string(),
|
||||
};
|
||||
with_parts(&plain, edit, &[("xl/python/python1.xml", &python)])
|
||||
}
|
||||
|
||||
fn code(s: &Session, cell: &str) -> Option<PyCode> {
|
||||
s.doc().unwrap().wb.active().unwrap().cell(CellRef::parse(cell).unwrap()).and_then(|c| c.py.as_deref()).cloned()
|
||||
}
|
||||
|
||||
/// A workbook with a value, a formula over a Python cell, a spilling table, an object, long code
|
||||
/// with quotes and line breaks, and an error.
|
||||
fn build() -> (Session, String) {
|
||||
let mut s = s();
|
||||
s.execute("cell.set", json!({"cell": "A1", "input": "21"})).unwrap();
|
||||
py(&mut s, "B1", "xl('A1') * 2");
|
||||
cache(&mut s, "B1", Value::Number(42.0), "int");
|
||||
s.execute("cell.set", json!({"cell": "A5", "input": "=B1+1"})).unwrap();
|
||||
py(&mut s, "C1", "[[1, 'two'], [3, \"four\"]]");
|
||||
cache(&mut s, "C1", array(2, 2, vec![1.0.into(), Value::text("two"), 3.0.into(), Value::text("four")]), "list");
|
||||
s.execute("python.set", json!({"cell": "E1", "code": "[1, 2]", "output": "object"})).unwrap();
|
||||
cache(&mut s, "E1", array(2, 1, vec![1.0.into(), 2.0.into()]), "list");
|
||||
let long = format!("# {}\ntext = \"say \\\"hi\\\"\"\nlen(text)", "x".repeat(7_900));
|
||||
assert!(PyCode::new(&long).fits());
|
||||
py(&mut s, "F1", &long);
|
||||
cache(&mut s, "F1", Value::Number(8.0), "int");
|
||||
py(&mut s, "G1", "1/0");
|
||||
{
|
||||
let d = s.doc_mut().unwrap();
|
||||
let sh = Arc::make_mut(&mut d.wb).sheet_mut(0).unwrap();
|
||||
sh.cells.get_mut(CellRef::parse("G1").unwrap()).unwrap().value = Value::Error(CellError::Python);
|
||||
}
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(v(&s, "B1"), Value::Number(42.0));
|
||||
assert_eq!(v(&s, "A5"), Value::Number(43.0));
|
||||
assert_eq!(v(&s, "D2"), Value::text("four"));
|
||||
assert_eq!(v(&s, "E1"), Value::text("list"), "a Python object shows its type");
|
||||
assert_eq!(v(&s, "E2"), Value::Empty, "and doesn't spill");
|
||||
(s, long)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn xlsx_stores_the_code_in_the_python_part_with_cached_values() {
|
||||
let (mut s, long) = build();
|
||||
let bytes = save(&mut s, "xlsx");
|
||||
let xml = sheet_xml(&bytes);
|
||||
// The formula names the cell's script by index, in the order the cells are written: B1, C1,
|
||||
// E1, F1, G1.
|
||||
assert!(xml.contains("<f>_xlfn._xlws.PY(0,0)</f><v>42</v>"), "{xml}");
|
||||
assert!(xml.contains(">_xlfn._xlws.PY(2,1)</f>"), "the Python object cell has return type 1: {xml}");
|
||||
assert!(!xml.contains("xl('A1')"), "the code is not in the sheet");
|
||||
let python = part(&bytes, "xl/python.xml");
|
||||
assert!(
|
||||
python.contains("<python xmlns=\"http://schemas.microsoft.com/office/spreadsheetml/2023/python\"><pythonScripts><pythonScript><code>xl('A1') * 2</code></pythonScript>"),
|
||||
"{python}"
|
||||
);
|
||||
assert_eq!(python.matches("<pythonScript>").count(), 5);
|
||||
assert!(
|
||||
part(&bytes, "[Content_Types].xml").contains("<Override PartName=\"/xl/python.xml\" ContentType=\"application/vnd.ms-excel.python+xml\"/>")
|
||||
);
|
||||
assert!(
|
||||
part(&bytes, "xl/_rels/workbook.xml.rels")
|
||||
.contains("Type=\"http://schemas.microsoft.com/office/2023/09/relationships/Python\" Target=\"python.xml\"")
|
||||
);
|
||||
assert!(xml.contains("t=\"array\" ref=\"C1:D2\""), "the spill is saved like a dynamic array's: {xml}");
|
||||
assert!(xml.contains("#VALUE!") && !xml.contains("#PYTHON!"), "Python errors are cached as #VALUE!");
|
||||
|
||||
open(&mut s, "book.xlsx", &bytes);
|
||||
assert_eq!(s.doc().unwrap().trust, Trust::FromFile);
|
||||
// Cached values show, spills included, and the code is back.
|
||||
assert_eq!(v(&s, "B1"), Value::Number(42.0));
|
||||
assert_eq!(v(&s, "A5"), Value::Number(43.0));
|
||||
assert_eq!(
|
||||
(v(&s, "C1"), v(&s, "D1"), v(&s, "C2"), v(&s, "D2")),
|
||||
(Value::Number(1.0), Value::text("two"), Value::Number(3.0), Value::text("four"))
|
||||
);
|
||||
assert_eq!(code(&s, "B1").unwrap().code, "xl('A1') * 2");
|
||||
assert!(code(&s, "E1").unwrap().object);
|
||||
assert_eq!(code(&s, "F1").unwrap().code, long, "long code with quotes and line breaks survives");
|
||||
assert_eq!(v(&s, "G1"), Value::Error(CellError::Value));
|
||||
// A blocker typed into the spill area gives #SPILL!; clearing it spills again, without
|
||||
// running anything.
|
||||
s.execute("cell.set", json!({"cell": "D2", "input": "x"})).unwrap();
|
||||
assert_eq!(v(&s, "C1"), Value::Error(CellError::Spill));
|
||||
s.execute("edit.clearContents", json!({"range": "D2"})).unwrap();
|
||||
assert_eq!(v(&s, "D2"), Value::text("four"));
|
||||
// Saved again, the file holds the same formulas and scripts.
|
||||
let again = save(&mut s, "xlsx");
|
||||
assert!(sheet_xml(&again).contains("<f>_xlfn._xlws.PY(0,0)</f>"));
|
||||
assert_eq!(part(&again, "xl/python.xml"), python);
|
||||
// A workbook without Python cells has no Python part.
|
||||
let mut plain = super::s();
|
||||
assert!(!save(&mut plain, "xlsx").windows(10).any(|w| w == b"python.xml"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn excel_authored_python_cells_are_read_as_python() {
|
||||
// A workbook as [MS-XLSX] 2.3.10 describes it: `_xlfn._xlws.PY(index,0|1,params…)` naming a
|
||||
// script of the Python part, whose code names the parameters as `%Px%`, with a spilled result
|
||||
// cached.
|
||||
let data = "<sheetData><row r=\"1\"><c r=\"A1\" cm=\"1\"><f t=\"array\" ref=\"A1:A2\">_xlfn._xlws.PY(0,0)</f><v>1</v></c>\
|
||||
<c r=\"B1\" t=\"str\"><f>_xlfn._xlws.PY(1,1,C1:C3)</f><v>DataFrame</v></c></row>\
|
||||
<row r=\"2\"><c r=\"A2\"><v>2</v></c></row></sheetData>";
|
||||
let bytes = excel_shaped(data, Some(&["[1, 2]", "df = xl(\"%P2%\", headers=True)\ndf.describe()"]));
|
||||
let mut s = s();
|
||||
open(&mut s, "excel.xlsx", &bytes);
|
||||
let a1 = code(&s, "A1").expect("an Excel Python cell is a Python cell");
|
||||
assert_eq!(a1.code, "[1, 2]");
|
||||
assert!(!a1.object);
|
||||
assert_eq!((v(&s, "A1"), v(&s, "A2")), (Value::Number(1.0), Value::Number(2.0)), "its spilled values are kept");
|
||||
let cell = s.doc().unwrap().wb.active().unwrap().cell(CellRef::new(0, 0)).unwrap().clone();
|
||||
assert!(cell.formula.is_none());
|
||||
let b1 = code(&s, "B1").unwrap();
|
||||
assert_eq!(b1.code, "df = xl(\"C1:C3\", headers=True)\ndf.describe()", "a parameter is put back as its reference");
|
||||
assert!(b1.object);
|
||||
assert_eq!(v(&s, "B1"), Value::text("DataFrame"));
|
||||
// Saved by GridCraft, the code goes back into a Python part, the reference as text.
|
||||
let again = save(&mut s, "xlsx");
|
||||
assert!(part(&again, "xl/python.xml").contains("<code>df = xl(\"C1:C3\", headers=True)\ndf.describe()</code>"));
|
||||
// The string-literal form (other applications, files GridCraft wrote before) still reads.
|
||||
let literal = "<sheetData><row r=\"1\"><c r=\"A1\"><f>_xlfn._xlws.PY(\"1 + 1\",1)</f><v>2</v></c></row></sheetData>";
|
||||
open(&mut s, "literal.xlsx", &excel_shaped(literal, None));
|
||||
assert_eq!(code(&s, "A1").map(|p| (p.code, p.object)), Some(("1 + 1".to_string(), true)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_py_formula_naming_a_missing_script_keeps_only_its_value() {
|
||||
let data = "<sheetData><row r=\"1\"><c r=\"A1\"><f>_xlfn._xlws.PY(0,0)</f><v>5</v></c><c r=\"B1\"><f>_xlfn._xlws.PY(3,0)</f><v>6</v></c></row></sheetData>";
|
||||
for (name, bytes) in [("one.xlsx", excel_shaped(data, Some(&["5"]))), ("none.xlsx", excel_shaped(data, None))] {
|
||||
let (wb, warnings) = crate::io::open_bytes(name, &bytes).unwrap();
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let b1 = sh.cell(CellRef::parse("B1").unwrap()).unwrap();
|
||||
assert!(b1.py.is_none() && b1.formula.is_none(), "{name}: no code to keep, and no formula #NAME? would replace");
|
||||
assert_eq!(b1.value, Value::Number(6.0), "{name}");
|
||||
assert!(warnings.iter().any(|w| w.contains("Python")), "{name}: {warnings:?}");
|
||||
}
|
||||
let (wb, _) = crate::io::open_bytes("one.xlsx", &excel_shaped(data, Some(&["5"]))).unwrap();
|
||||
assert_eq!(wb.sheet(0).unwrap().cell(CellRef::parse("A1").unwrap()).unwrap().py.as_ref().map(|p| p.code.as_str()), Some("5"));
|
||||
}
|
||||
|
||||
/// `bytes` with the part `name` stored uncompressed and one byte of its data changed, so the
|
||||
/// archive is intact but reading the part fails its checksum.
|
||||
fn with_damaged_part(bytes: &[u8], name: &str, marker: &str) -> Vec<u8> {
|
||||
let mut z = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
|
||||
let mut out = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
|
||||
for i in 0..z.len() {
|
||||
let mut f = z.by_index(i).unwrap();
|
||||
let entry = f.name().to_string();
|
||||
let mut data = Vec::new();
|
||||
f.read_to_end(&mut data).unwrap();
|
||||
let options = zip::write::SimpleFileOptions::default();
|
||||
let options = if entry == name { options.compression_method(zip::CompressionMethod::Stored) } else { options };
|
||||
out.start_file(entry, options).unwrap();
|
||||
out.write_all(&data).unwrap();
|
||||
}
|
||||
let mut zipped = out.finish().unwrap().into_inner();
|
||||
let at = zipped.windows(marker.len()).position(|w| w == marker.as_bytes()).expect("the marker is stored as it is");
|
||||
zipped[at + 1] ^= 0x20;
|
||||
zipped
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_damaged_python_part_is_a_warning_and_the_workbook_opens_with_its_values() {
|
||||
let data = "<sheetData><row r=\"1\"><c r=\"A1\"><f>_xlfn._xlws.PY(0,0)</f><v>5</v></c><c r=\"B1\"><v>9</v></c></row></sheetData>";
|
||||
let good = excel_shaped(data, Some(&["5"]));
|
||||
// The part's data fails its checksum (a damaged archive entry).
|
||||
let damaged = with_damaged_part(&good, "xl/python/python1.xml", "<pythonScripts>");
|
||||
// The part is not well-formed XML, before its first script.
|
||||
let broken = |at: &'static str, with: &'static str| {
|
||||
with_parts(&good, move |name, text| if name == "xl/python/python1.xml" { text.replace(at, with) } else { text.to_string() }, &[])
|
||||
};
|
||||
let malformed = broken("<pythonScripts>", "<pythonScripts><!-- never closed");
|
||||
for (name, bytes) in [("damaged.xlsx", damaged), ("malformed.xlsx", malformed)] {
|
||||
let (wb, warnings) = crate::io::open_bytes(name, &bytes).unwrap_or_else(|e| panic!("{name}: {e}"));
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let a1 = sh.cell(CellRef::parse("A1").unwrap()).unwrap();
|
||||
assert_eq!(a1.value, Value::Number(5.0), "{name}: the cached value is kept");
|
||||
assert!(a1.py.is_none() && a1.formula.is_none(), "{name}: as for a missing script, only the value");
|
||||
assert_eq!(sh.value(CellRef::parse("B1").unwrap()), Value::Number(9.0), "{name}: the rest of the sheet is read");
|
||||
let part = warnings.iter().filter(|w| w.contains("Python part is damaged")).count();
|
||||
assert_eq!(part, 1, "{name}: {warnings:?}");
|
||||
}
|
||||
// Damaged after its first script, the part keeps what it read before the damage.
|
||||
let (wb, warnings) = crate::io::open_bytes("late.xlsx", &broken("</pythonScripts>", "<!-- never closed")).unwrap();
|
||||
let a1 = wb.sheet(0).unwrap().cell(CellRef::parse("A1").unwrap()).unwrap().clone();
|
||||
assert_eq!(a1.py.as_ref().map(|p| p.code.as_str()), Some("5"));
|
||||
assert_eq!(warnings.iter().filter(|w| w.contains("Python part is damaged")).count(), 1, "{warnings:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_python_cell_without_its_code_keeps_its_whole_cached_result_as_values() {
|
||||
// A1's cached result spills over A1:A3; C1 is an ordinary dynamic array with a stale cache.
|
||||
let data = "<sheetData><row r=\"1\"><c r=\"A1\" cm=\"1\"><f t=\"array\" ref=\"A1:A3\">_xlfn._xlws.PY(0,0)</f><v>1</v></c>\
|
||||
<c r=\"C1\" cm=\"1\"><f t=\"array\" ref=\"C1:C3\">SEQUENCE(3)</f><v>1</v></c></row>\
|
||||
<row r=\"2\"><c r=\"A2\"><v>2</v></c><c r=\"C2\"><v>9</v></c></row>\
|
||||
<row r=\"3\"><c r=\"A3\"><v>3</v></c><c r=\"C3\"><v>9</v></c></row></sheetData>";
|
||||
let missing = excel_shaped(data, None);
|
||||
let damaged = with_damaged_part(&excel_shaped(data, Some(&["[1, 2, 3]"])), "xl/python/python1.xml", "<pythonScripts>");
|
||||
for (name, bytes) in [("missing.xlsx", missing), ("damaged.xlsx", damaged)] {
|
||||
let mut s = s();
|
||||
open(&mut s, name, &bytes);
|
||||
assert_eq!((v(&s, "A1"), v(&s, "A2"), v(&s, "A3")), (Value::Number(1.0), Value::Number(2.0), Value::Number(3.0)), "{name}");
|
||||
let sh = s.doc().unwrap().wb.active().unwrap();
|
||||
for a in ["A1", "A2", "A3"] {
|
||||
let cell = sh.cell(CellRef::parse(a).unwrap()).unwrap_or_else(|| panic!("{name}: {a} is stored"));
|
||||
assert!(cell.py.is_none() && cell.formula.is_none(), "{name}: {a} is a plain value");
|
||||
}
|
||||
assert!(!sh.spill_ranges.contains_key(&CellRef::new(0, 0)), "{name}: no live spill");
|
||||
// The ordinary formula fills its spill again, over the stale cached values.
|
||||
assert_eq!((v(&s, "C2"), v(&s, "C3")), (Value::Number(2.0), Value::Number(3.0)), "{name}");
|
||||
assert!(sh.cell(CellRef::parse("C2").unwrap()).is_none(), "{name}: C2 is the formula's spill, not a cell");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_python_spill_whose_blank_cells_have_no_records_is_still_a_spill() {
|
||||
// A1's result spills over A1:A3, and the file holds no records for the blank A2 and A3.
|
||||
let data = "<sheetData><row r=\"1\"><c r=\"A1\" cm=\"1\"><f t=\"array\" ref=\"A1:A3\">_xlfn._xlws.PY(0,0)</f><v>1</v></c></row></sheetData>";
|
||||
let mut s = s();
|
||||
open(&mut s, "sparse.xlsx", &excel_shaped(data, Some(&["[1, None, None]"])));
|
||||
let sh = s.doc().unwrap().wb.active().unwrap();
|
||||
let a1 = CellRef::new(0, 0);
|
||||
assert_eq!(sh.spill_ranges.get(&a1), Some(&RangeRef::parse("A1:A3").unwrap()), "the range is recorded at its anchor");
|
||||
assert_eq!(sh.spill_anchor(CellRef::parse("A3").unwrap()), Some(a1));
|
||||
let kept = sh.cell(a1).and_then(|c| c.py.as_ref()).and_then(|p| p.output.as_ref()).map(|o| o.value.clone());
|
||||
assert_eq!(kept, Some(array(3, 1, vec![1.0.into(), Value::Empty, Value::Empty])), "the result is rebuilt over the whole range");
|
||||
// A formula entered in the spill takes the area, as with any spill.
|
||||
s.execute("cell.set", json!({"cell": "A3", "input": "7"})).unwrap();
|
||||
assert_eq!(v(&s, "A1"), Value::Error(CellError::Spill));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_python_spill_range_larger_than_a_result_keeps_its_values_as_plain_values() {
|
||||
// C1:D600000 is 1,200,000 cells, more than a result may have but within what the reader
|
||||
// tracks; its cached values are kept as the file shows them, and nothing spills.
|
||||
let (mut s, _) = build();
|
||||
let bytes = save(&mut s, "xlsx");
|
||||
let crafted = with_sheet_xml(&bytes, |xml| xml.replace("ref=\"C1:D2\"", "ref=\"C1:D600000\""));
|
||||
assert_ne!(crafted, bytes);
|
||||
open(&mut s, "crafted.xlsx", &crafted);
|
||||
let sh = s.doc().unwrap().wb.active().unwrap();
|
||||
assert!(!sh.spill_ranges.contains_key(&CellRef::new(0, 2)), "{:?}", sh.spill_ranges);
|
||||
assert_eq!(
|
||||
(v(&s, "C1"), v(&s, "D1"), v(&s, "C2"), v(&s, "D2")),
|
||||
(Value::Number(1.0), Value::text("two"), Value::Number(3.0), Value::text("four"))
|
||||
);
|
||||
let d2 = sh.cell(CellRef::parse("D2").unwrap()).expect("D2 is stored");
|
||||
assert!(d2.py.is_none() && d2.formula.is_none(), "a plain value");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn python_spills_read_past_the_scan_budget_keep_their_values_as_plain_values() {
|
||||
// A1's result spills over A1:A5000. Four thousand ordinary dynamic arrays on row 1 make each
|
||||
// later plain cell's range lookup cost 4,001 checks, so the reader's scan budget runs out
|
||||
// part way down column A, and the rest of the spill is read as plain cells.
|
||||
let arrays: String = (1..=4_000u32)
|
||||
.map(|c| {
|
||||
let at = CellRef::new(0, c).a1();
|
||||
let below = CellRef::new(1, c).a1();
|
||||
format!("<c r=\"{at}\" cm=\"1\"><f t=\"array\" ref=\"{at}:{below}\">SEQUENCE(2)</f><v>1</v></c>")
|
||||
})
|
||||
.collect();
|
||||
let rows: String = (2..=5_000u32).map(|r| format!("<row r=\"{r}\"><c r=\"A{r}\"><v>{r}</v></c></row>")).collect();
|
||||
let data = format!(
|
||||
"<sheetData><row r=\"1\"><c r=\"A1\" cm=\"1\"><f t=\"array\" ref=\"A1:A5000\">_xlfn._xlws.PY(0,0)</f><v>1</v></c>{arrays}</row>{rows}</sheetData>"
|
||||
);
|
||||
let mut s = s();
|
||||
open(&mut s, "budget.xlsx", &excel_shaped(&data, Some(&["list(range(1, 5001))"])));
|
||||
let sh = s.doc().unwrap().wb.active().unwrap();
|
||||
assert!(!sh.spill_ranges.contains_key(&CellRef::new(0, 0)), "no live spill");
|
||||
assert!(code(&s, "A1").is_some_and(|p| p.output.is_none()), "no result is rebuilt");
|
||||
assert_eq!((v(&s, "A2"), v(&s, "A5000")), (Value::Number(2.0), Value::Number(5000.0)), "the values stay");
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(v(&s, "A1"), Value::Number(1.0), "the cell keeps its value, not #SPILL!");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deleting_the_data_python_code_reads_drops_its_kept_result() {
|
||||
let mut s = s();
|
||||
s.execute("cell.set", json!({"cell": "A1", "input": "1"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "2"})).unwrap();
|
||||
s.execute("cell.set", json!({"cell": "A3", "input": "3"})).unwrap();
|
||||
py(&mut s, "C5", "xl(\"A1:A2\") * 2");
|
||||
cache(&mut s, "C5", array(2, 1, vec![2.0.into(), 4.0.into()]), "list");
|
||||
py(&mut s, "E5", "xl(\"A1:A3\")");
|
||||
cache(&mut s, "E5", Value::Number(3.0), "int");
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(v(&s, "C6"), Value::Number(4.0));
|
||||
// A move keeps the result: the reference still names the same data.
|
||||
s.execute("home.insertRows", json!({"rows": "1"})).unwrap();
|
||||
assert!(code(&s, "C6").is_some_and(|p| p.output.is_some() && !p.code.contains("A1:A2")), "{:?}", code(&s, "C6"));
|
||||
assert_eq!(v(&s, "C7"), Value::Number(4.0));
|
||||
assert!(code(&s, "E6").is_some_and(|p| p.output.is_some()));
|
||||
// Deleting the rows C6 reads leaves `#REF!`, and deleting part of E6's range shrinks it: in
|
||||
// both, the kept result no longer belongs to the code.
|
||||
s.execute("home.deleteRows", json!({"rows": "2:3"})).unwrap();
|
||||
let c4 = code(&s, "C4").expect("the Python cell moved up to C4");
|
||||
assert!(c4.code.contains("#REF!") && c4.output.is_none(), "{c4:?}");
|
||||
assert_eq!(v(&s, "C4"), Value::Error(CellError::Blocked));
|
||||
assert_eq!(v(&s, "C5"), Value::Empty, "its spill is gone");
|
||||
let e4 = code(&s, "E4").expect("the Python cell moved up to E4");
|
||||
assert!(!e4.code.contains("#REF!") && e4.output.is_none(), "{e4:?}");
|
||||
assert_eq!(v(&s, "E4"), Value::Error(CellError::Blocked));
|
||||
// Rows inserted inside a range grow it: also new data.
|
||||
py(&mut s, "G1", "xl(\"A1:A2\")");
|
||||
cache(&mut s, "G1", Value::Number(2.0), "int");
|
||||
s.execute("home.insertRows", json!({"rows": "2"})).unwrap();
|
||||
assert!(code(&s, "G1").is_some_and(|p| p.output.is_none()), "{:?}", code(&s, "G1"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_script_is_measured_after_its_escapes_are_decoded() {
|
||||
let max = gridcraft_model::python::MAX_FORMULA_CHARS;
|
||||
// 2,000 tabs written as `_x0009_`: 14,000 characters in the file, 2,004 as code, which fits.
|
||||
let escaped = format!("x = 1{}end", "_x0009_".repeat(2_000));
|
||||
assert!(escaped.chars().count() > max);
|
||||
// Code longer than a cell may hold once decoded is still refused.
|
||||
let long = format!("{}{}", "_x0009_".repeat(10), "y".repeat(max));
|
||||
let data = "<sheetData><row r=\"1\"><c r=\"A1\"><f>_xlfn._xlws.PY(0,0)</f><v>1</v></c><c r=\"B1\"><f>_xlfn._xlws.PY(1,0)</f><v>2</v></c></row></sheetData>";
|
||||
let (wb, warnings) = crate::io::open_bytes("escapes.xlsx", &excel_shaped(data, Some(&[&escaped, &long]))).unwrap();
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let a1 = sh.cell(CellRef::parse("A1").unwrap()).unwrap().py.as_ref().map(|p| p.code.clone());
|
||||
assert_eq!(a1, Some(format!("x = 1{}end", "\t".repeat(2_000))), "read in full");
|
||||
let b1 = sh.cell(CellRef::parse("B1").unwrap()).unwrap();
|
||||
assert!(b1.py.is_none(), "too long once decoded: only the value");
|
||||
assert_eq!(b1.value, Value::Number(2.0));
|
||||
assert!(warnings.iter().any(|w| w.contains("B1")), "{warnings:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn many_cells_naming_one_long_script_take_from_one_budget() {
|
||||
// 20,000 cells name one script of 8,000 characters: 160 MB of code if each took a copy.
|
||||
let long = "x".repeat(8_000);
|
||||
let mut data = String::from("<sheetData>");
|
||||
for r in 1..=20_000 {
|
||||
data.push_str(&format!("<row r=\"{r}\"><c r=\"A{r}\"><f>_xlfn._xlws.PY(0,0)</f><v>1</v></c></row>"));
|
||||
}
|
||||
data.push_str("</sheetData>");
|
||||
let (wb, warnings) = crate::io::open_bytes("many.xlsx", &excel_shaped(&data, Some(&[&long]))).unwrap();
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let with_code = sh.cells.iter().filter(|(_, c)| c.py.is_some()).count();
|
||||
assert_eq!(with_code, 64 * 1024 * 1024 / 8_000);
|
||||
assert!(warnings.iter().any(|w| w.contains("Python")), "{warnings:?}");
|
||||
assert_eq!(sh.value(CellRef::parse("A20000").unwrap()), Value::Number(1.0), "the others keep their values");
|
||||
}
|
||||
|
||||
/// Plants a cached spill over `range` (anchored at its start) with `value` at `at`, as a file
|
||||
/// read leaves it.
|
||||
fn cached_spill(sh: &mut gridcraft_model::Sheet, range: RangeRef, spilled: &[(CellRef, Value)]) {
|
||||
sh.spill_ranges.insert(range.start, range);
|
||||
for (c, v) in spilled {
|
||||
sh.spill.insert(*c, v.clone());
|
||||
}
|
||||
}
|
||||
|
||||
/// The value stored in the cell itself (not a spilled one), with whether it holds code or a
|
||||
/// formula.
|
||||
fn stored(sh: &gridcraft_model::Sheet, c: CellRef) -> (Value, bool) {
|
||||
sh.cell(c).map(|x| (x.value.clone(), x.py.is_some() || x.formula.is_some())).unwrap_or((Value::Empty, false))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_result_cut_by_the_budget_keeps_its_cached_spill_as_values() {
|
||||
// Five Python cells each with a cached 1,000 x 1,000 spill: the budget rebuilds four results;
|
||||
// the fifth cell keeps only its own value, and the values it spilled stay as plain values.
|
||||
let mut wb = Workbook::new();
|
||||
let bold = wb.styles.derive(gridcraft_model::StyleId::DEFAULT, |st| st.font.bold = true);
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
for i in 0..5u32 {
|
||||
let col = i * 1_000;
|
||||
let anchor = CellRef::new(0, col);
|
||||
sh.set_cell(anchor, Cell { py: Some(Arc::new(PyCode::new("x"))), ..Cell::value(Value::Number(1.0)) });
|
||||
// A cell with its own content, and a formatted empty one, inside the spill.
|
||||
sh.set_cell(CellRef::new(2, col), Cell::value(Value::text("own")));
|
||||
sh.set_cell(CellRef::new(3, col), Cell::styled(Value::Empty, bold));
|
||||
let spilled = [
|
||||
(CellRef::new(1, col), Value::Number(7.0)),
|
||||
(CellRef::new(2, col), Value::Number(9.0)),
|
||||
(CellRef::new(3, col), Value::Number(6.0)),
|
||||
(CellRef::new(999, col + 999), Value::Number(8.0)),
|
||||
];
|
||||
cached_spill(sh, RangeRef::new(anchor, CellRef::new(999, col + 999)), &spilled);
|
||||
}
|
||||
let warnings = crate::io::adopt_python(&mut wb);
|
||||
assert!(warnings.iter().any(|w| w.contains("Python")), "{warnings:?}");
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let mut cut = 0;
|
||||
for i in 0..5u32 {
|
||||
let col = i * 1_000;
|
||||
let anchor = CellRef::new(0, col);
|
||||
let py = sh.cell(anchor).and_then(|c| c.py.clone()).expect("the code is kept");
|
||||
let shown =
|
||||
[sh.value(CellRef::new(1, col)), sh.value(CellRef::new(2, col)), sh.value(CellRef::new(3, col)), sh.value(CellRef::new(999, col + 999))];
|
||||
assert_eq!(shown, [Value::Number(7.0), Value::text("own"), Value::Number(6.0), Value::Number(8.0)], "every result shows the same values");
|
||||
if py.output.is_some() {
|
||||
assert!(sh.spill_ranges.contains_key(&anchor), "a result within the budget keeps its spill");
|
||||
continue;
|
||||
}
|
||||
cut += 1;
|
||||
assert!(!sh.spill_ranges.contains_key(&anchor), "no live spill");
|
||||
assert!(sh.spill.range(anchor..=CellRef::new(999, col + 999)).all(|(c, _)| c.col < col || c.col > col + 999), "no spilled values left");
|
||||
assert_eq!(stored(sh, anchor), (Value::Number(1.0), true), "the anchor keeps its value and its code");
|
||||
assert_eq!(stored(sh, CellRef::new(1, col)), (Value::Number(7.0), false), "a spilled value is a plain value now");
|
||||
assert_eq!(stored(sh, CellRef::new(999, col + 999)), (Value::Number(8.0), false));
|
||||
assert_eq!(stored(sh, CellRef::new(2, col)), (Value::text("own"), false), "content is never overwritten");
|
||||
assert_eq!(sh.style_id(CellRef::new(3, col)), bold, "nor a format");
|
||||
}
|
||||
assert_eq!(cut, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn code_too_long_for_a_cell_keeps_its_whole_cached_result_as_values() {
|
||||
let long = PyCode::new(&"x".repeat(gridcraft_model::python::MAX_FORMULA_CHARS));
|
||||
assert!(!long.fits());
|
||||
let mut wb = Workbook::new();
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
// A1 spills over A1:B2; D1's own value is empty.
|
||||
sh.set_cell(CellRef::new(0, 0), Cell { py: Some(Arc::new(long.clone())), ..Cell::value(Value::Number(1.0)) });
|
||||
sh.set_cell(CellRef::new(1, 1), Cell::value(Value::text("own")));
|
||||
let spilled = [(CellRef::new(0, 1), Value::Number(2.0)), (CellRef::new(1, 0), Value::Number(3.0)), (CellRef::new(1, 1), Value::Number(4.0))];
|
||||
cached_spill(sh, RangeRef::new(CellRef::new(0, 0), CellRef::new(1, 1)), &spilled);
|
||||
sh.set_cell(CellRef::new(0, 3), Cell { py: Some(Arc::new(long)), ..Cell::default() });
|
||||
cached_spill(sh, RangeRef::new(CellRef::new(0, 3), CellRef::new(1, 3)), &[(CellRef::new(1, 3), Value::Number(5.0))]);
|
||||
let warnings = crate::io::enforce_python_limits(&mut wb);
|
||||
assert!(warnings.iter().any(|w| w.contains("A1")), "{warnings:?}");
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
assert!(sh.spill_ranges.is_empty() && sh.spill.is_empty(), "no live spill: {:?}", sh.spill_ranges);
|
||||
assert_eq!(stored(sh, CellRef::new(0, 0)), (Value::Number(1.0), false), "the anchor keeps only its value");
|
||||
assert_eq!(stored(sh, CellRef::new(0, 1)), (Value::Number(2.0), false));
|
||||
assert_eq!(stored(sh, CellRef::new(1, 0)), (Value::Number(3.0), false));
|
||||
assert_eq!(stored(sh, CellRef::new(1, 1)), (Value::text("own"), false), "content is never overwritten");
|
||||
assert_eq!(stored(sh, CellRef::new(0, 3)), (Value::Empty, false), "an empty anchor stays empty, without code");
|
||||
assert_eq!(stored(sh, CellRef::new(1, 3)), (Value::Number(5.0), false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn native_json_keeps_python_cells_and_their_results() {
|
||||
let (mut s, long) = build();
|
||||
let bytes = save(&mut s, "json");
|
||||
open(&mut s, "book.json", &bytes);
|
||||
assert_eq!(s.doc().unwrap().trust, Trust::FromFile);
|
||||
assert_eq!(v(&s, "B1"), Value::Number(42.0));
|
||||
assert_eq!(v(&s, "D2"), Value::text("four"));
|
||||
assert_eq!(v(&s, "G1"), Value::Error(CellError::Python), "the native format keeps the exact error");
|
||||
assert_eq!(code(&s, "F1").unwrap().code, long);
|
||||
let e1 = code(&s, "E1").unwrap();
|
||||
assert!(e1.object);
|
||||
assert_eq!(e1.output.unwrap().type_name, "list");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_spill_range_claiming_the_whole_sheet_is_dropped_on_open() {
|
||||
let (mut s, _) = build();
|
||||
let bytes = save(&mut s, "xlsx");
|
||||
let evil = with_sheet_xml(&bytes, |xml| xml.replace("ref=\"C1:D2\"", "ref=\"C1:XFD1048576\""));
|
||||
assert_ne!(evil, bytes);
|
||||
// Walking the claimed range would take hours; the test finishing is the check.
|
||||
open(&mut s, "evil.xlsx", &evil);
|
||||
let sh = s.doc().unwrap().wb.active().unwrap();
|
||||
assert!(sh.spill_ranges.values().all(|r| r.count() <= gridcraft_model::python::MAX_RESULT_CELLS as u64), "{:?}", sh.spill_ranges);
|
||||
// Edits afterwards don't walk the claimed range either.
|
||||
s.execute("cell.set", json!({"cell": "H9", "input": "1"})).unwrap();
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(v(&s, "C1"), Value::Number(1.0), "the anchor keeps its cached value");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rebuilt_results_have_a_budget_across_the_workbook() {
|
||||
// Five Python cells each claiming a 1,000 x 1,000 spill: more than the budget in all.
|
||||
let mut wb = Workbook::new();
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
for i in 0..5u32 {
|
||||
let anchor = CellRef::new(0, i * 1_000);
|
||||
sh.set_cell(anchor, Cell { py: Some(Arc::new(PyCode::new("x"))), ..Cell::value(Value::Number(1.0)) });
|
||||
sh.spill_ranges.insert(anchor, RangeRef::new(anchor, CellRef::new(999, i * 1_000 + 999)));
|
||||
}
|
||||
let warnings = crate::io::adopt_python(&mut wb);
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let rebuilt = sh.cells.iter().filter(|(_, c)| c.py.as_ref().is_some_and(|p| p.output.is_some())).count();
|
||||
assert_eq!(rebuilt as u64, crate::io::MAX_ADOPTED_RESULT_CELLS / 1_000_000);
|
||||
assert!(warnings.iter().any(|w| w.contains("Python")), "{warnings:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hostile_python_content_in_files_is_capped() {
|
||||
let (mut s, _) = build();
|
||||
let bytes = save(&mut s, "json");
|
||||
// Make the Python cells hostile: code over the limit, and a result whose shape claims far
|
||||
// more cells than it holds (the array fields are public, so a crafted file can say anything).
|
||||
let mut wb: Workbook = serde_json::from_slice(&bytes).unwrap();
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
let b1 = sh.cells.get_mut(CellRef::parse("B1").unwrap()).unwrap();
|
||||
Arc::make_mut(b1.py.as_mut().unwrap()).code = "y".repeat(9_000);
|
||||
let c1 = sh.cells.get_mut(CellRef::parse("C1").unwrap()).unwrap();
|
||||
let out = Arc::make_mut(c1.py.as_mut().unwrap()).output.as_mut().unwrap();
|
||||
out.value = Value::Array(Arc::new(Array { rows: 100_000, cols: 100_000, data: vec![Value::Number(1.0)] }));
|
||||
let bytes = serde_json::to_vec(&wb).unwrap();
|
||||
let (wb, warnings) = crate::io::open_bytes("evil.json", &bytes).unwrap();
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let b1 = sh.cell(CellRef::parse("B1").unwrap()).unwrap();
|
||||
assert!(b1.py.is_none(), "code over the formula limit is dropped");
|
||||
assert_eq!(b1.value, Value::Number(42.0), "the cached value stays");
|
||||
assert!(warnings.iter().any(|w| w.contains("Python cell")), "{warnings:?}");
|
||||
let c1 = sh.cell(CellRef::parse("C1").unwrap()).unwrap();
|
||||
assert!(c1.py.as_ref().unwrap().output.is_none(), "a malformed result is dropped");
|
||||
open(&mut s, "evil.json", &bytes);
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(v(&s, "C1"), Value::Number(1.0));
|
||||
|
||||
// PY formulas from other formats become Python cells; non-literal PY calls stay formulas.
|
||||
let mut wb = Workbook::new();
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
sh.set_cell(CellRef::new(0, 0), Cell::formula(gridcraft_model::Formula::new("=PY(\"1+\"&\"1\",1)")));
|
||||
sh.set_cell(CellRef::new(1, 0), Cell::formula(gridcraft_model::Formula::new("=PY(A1,0)")));
|
||||
sh.set_cell(CellRef::new(2, 0), Cell::formula(gridcraft_model::Formula::new("=_xlfn._xlws.PY(\"2\",0)")));
|
||||
crate::io::adopt_python(&mut wb);
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let a1 = sh.cell(CellRef::new(0, 0)).unwrap();
|
||||
assert_eq!(a1.py.as_ref().map(|p| (p.code.as_str(), p.object)), Some(("1+1", true)));
|
||||
assert!(a1.formula.is_none());
|
||||
assert!(sh.cell(CellRef::new(1, 0)).unwrap().py.is_none());
|
||||
assert_eq!(sh.cell(CellRef::new(2, 0)).unwrap().py.as_ref().map(|p| p.code.as_str()), Some("2"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_malformed_result_never_spills() {
|
||||
let mut s = s();
|
||||
py(&mut s, "A1", "x");
|
||||
let bad = Array { rows: 4_000_000_000, cols: 4_000_000_000, data: vec![] };
|
||||
cache(&mut s, "A1", Value::Array(Arc::new(bad)), "list");
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(v(&s, "A1"), Value::Error(CellError::Spill));
|
||||
assert!(s.doc().unwrap().wb.active().unwrap().spill.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn py_formulas_over_the_size_limit_keep_only_their_value() {
|
||||
let lits = |n: usize| {
|
||||
let mut v = vec![format!("\"{}\"", "x".repeat(255)); n / 255];
|
||||
if !n.is_multiple_of(255) {
|
||||
v.push(format!("\"{}\"", "x".repeat(n % 255)));
|
||||
}
|
||||
v.join("&")
|
||||
};
|
||||
let mut wb = Workbook::new();
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
let cell = |text: String| Cell { value: Value::text("cached"), ..Cell::formula(gridcraft_model::Formula::new(&text)) };
|
||||
sh.set_cell(CellRef::new(0, 0), cell(format!("_xlfn._xlws.PY({},0)", lits(8_080))));
|
||||
sh.set_cell(CellRef::new(1, 0), cell(format!("_xlfn._xlws.PY({},0)", lits(8_079))));
|
||||
let xlsx = gridcraft_xlsx::write_xlsx(&wb).unwrap();
|
||||
let (wb, warnings) = crate::io::open_bytes("long.xlsx", &xlsx).unwrap();
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
let over = sh.cell(CellRef::new(0, 0)).unwrap();
|
||||
assert!(over.py.is_none() && over.formula.is_none(), "8,080 characters of code is over the limit");
|
||||
assert_eq!(over.value, Value::text("cached"));
|
||||
let within = sh.cell(CellRef::new(1, 0)).unwrap();
|
||||
assert_eq!(within.py.as_ref().map(|p| p.code.chars().count()), Some(8_079));
|
||||
assert!(warnings.iter().any(|w| w.contains("A1")), "{warnings:?}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_dropped_python_result_keeps_its_spill_as_values() {
|
||||
// A1's code is over the limit and B1's result is malformed; each had spilled down to row 3,
|
||||
// and C1's well-formed result keeps its spill.
|
||||
let mut wb = Workbook::new();
|
||||
let sh = wb.sheet_mut(0).unwrap();
|
||||
let out = |value: Value| Some(PyOutput { value, type_name: "list".into(), preview: String::new(), convertible: true });
|
||||
let col = array(3, 1, vec![Value::Number(7.0), Value::Number(8.0), Value::Number(9.0)]);
|
||||
let bad = Value::Array(Arc::new(Array { rows: 3, cols: 1, data: vec![Value::Number(7.0)] }));
|
||||
for (at, code, value) in [("A1", "y".repeat(9_000), col.clone()), ("B1", "b".into(), bad), ("C1", "c".into(), col)] {
|
||||
let at = CellRef::parse(at).unwrap();
|
||||
let py = PyCode { output: out(value), ..PyCode::new(&code) };
|
||||
sh.set_cell(at, Cell { value: Value::Number(7.0), formula: None, style: Default::default(), py: Some(Arc::new(py)) });
|
||||
let r = RangeRef::new(at, CellRef::new(2, at.col));
|
||||
sh.spill_ranges.insert(at, r);
|
||||
for c in r.iter().skip(1) {
|
||||
sh.spill.insert(c, Value::Number(8.0));
|
||||
}
|
||||
}
|
||||
let warnings = crate::io::enforce_python_limits(&mut wb);
|
||||
assert!(warnings.iter().any(|w| w.contains("A1")), "{warnings:?}");
|
||||
let sh = wb.sheet(0).unwrap();
|
||||
for (anchor, below) in [("A1", "A2"), ("B1", "B3")] {
|
||||
assert!(!sh.spill_ranges.contains_key(&CellRef::parse(anchor).unwrap()), "{anchor}'s spill range stays");
|
||||
let cell = sh.cell(CellRef::parse(below).unwrap()).unwrap_or_else(|| panic!("{below} is a plain value"));
|
||||
assert_eq!((&cell.value, cell.py.is_none() && cell.formula.is_none()), (&Value::Number(8.0), true), "{below} keeps {anchor}'s value");
|
||||
assert_eq!(sh.value(CellRef::parse(anchor).unwrap()), Value::Number(7.0), "{anchor} keeps its value");
|
||||
}
|
||||
assert_eq!(sh.value(CellRef::parse("C2").unwrap()), Value::Number(8.0), "a sound result keeps its spill");
|
||||
// A save writes the kept values as plain values, and the sound spill as a spill.
|
||||
let xml = sheet_xml(&gridcraft_xlsx::write_xlsx(&wb).unwrap());
|
||||
assert!(xml.contains("<c r=\"A2\"><v>8</v></c>") && xml.contains("<c r=\"B3\"><v>8</v></c>"), "{xml}");
|
||||
assert!(xml.contains("r=\"C2\""), "{xml}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_formula_entered_in_a_python_spill_takes_the_area_as_a_formula_would() {
|
||||
// A1's cached array spills over A1:A3; =SEQUENCE(2) entered in A2 is not blocked by that old
|
||||
// spill (the Python cell is laid out again in the same pass), so it spills and A1 is blocked.
|
||||
let mut s = s();
|
||||
py(&mut s, "A1", "[1, 2, 3]");
|
||||
cache(&mut s, "A1", array(3, 1, vec![10.0.into(), 20.0.into(), 30.0.into()]), "list");
|
||||
s.execute("formulas.calculateNow", json!({})).unwrap();
|
||||
assert_eq!(v(&s, "A3"), Value::Number(30.0));
|
||||
s.execute("cell.set", json!({"cell": "A2", "input": "=SEQUENCE(2)"})).unwrap();
|
||||
assert_eq!((v(&s, "A2"), v(&s, "A3")), (Value::Number(1.0), Value::Number(2.0)));
|
||||
assert_eq!(v(&s, "A1"), Value::Error(CellError::Spill));
|
||||
let wb = &s.doc().unwrap().wb;
|
||||
let mut fresh = (**wb).clone();
|
||||
gridcraft_calc::Calc::new().recalc_all(&mut fresh);
|
||||
for c in ["A1", "A2", "A3"] {
|
||||
let c = CellRef::parse(c).unwrap();
|
||||
assert_eq!(wb.active().unwrap().value(c), fresh.active().unwrap().value(c), "{c}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_new_return_type_drops_the_kept_result() {
|
||||
// Same code, the other return type: the result belongs to the old type, so it goes.
|
||||
let mut s = super::python_cells::spill_with_a_reader();
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "[5, 6]", "output": "object"})).unwrap();
|
||||
let a1 = code(&s, "A1").unwrap();
|
||||
assert!(a1.object && a1.output.is_none(), "no result of the other type is kept");
|
||||
assert_eq!(v(&s, "A1"), Value::Error(CellError::Blocked));
|
||||
assert_eq!(v(&s, "A2"), Value::Empty, "no spill");
|
||||
assert!(s.doc().unwrap().wb.active().unwrap().spill_ranges.is_empty());
|
||||
assert_eq!(v(&s, "B1"), Value::Number(0.0), "the reader of the old spill is recalculated");
|
||||
let xml = sheet_xml(&save(&mut s, "xlsx"));
|
||||
assert!(xml.contains("<f>_xlfn._xlws.PY(0,1)</f><v>#VALUE!</v>") && !xml.contains("r=\"A2\""), "no old cached value is written: {xml}");
|
||||
// Undo brings the result back.
|
||||
s.execute("edit.undo", json!({})).unwrap();
|
||||
let a1 = code(&s, "A1").unwrap();
|
||||
assert!(!a1.object && a1.output.is_some());
|
||||
assert_eq!((v(&s, "A1"), v(&s, "A2"), v(&s, "B1")), (Value::Number(5.0), Value::Number(6.0), Value::Number(6.0)));
|
||||
// Same code, same return type: the result is kept as before.
|
||||
s.execute("python.set", json!({"cell": "A1", "code": "[5, 6]", "output": "value"})).unwrap();
|
||||
assert!(code(&s, "A1").unwrap().output.is_some());
|
||||
assert_eq!((v(&s, "A1"), v(&s, "A2"), v(&s, "B1")), (Value::Number(5.0), Value::Number(6.0), Value::Number(6.0)));
|
||||
}
|
||||
@@ -32,7 +32,7 @@
|
||||
use std::collections::BTreeSet;
|
||||
use std::sync::Arc;
|
||||
|
||||
use gridcraft_core::{CellRef, MAX_COLS, MAX_ROWS, RangeRef, Value, col_to_letters};
|
||||
use gridcraft_core::{CellError, CellRef, MAX_COLS, MAX_ROWS, RangeRef, Value, col_to_letters};
|
||||
use gridcraft_engine::calc::Calc;
|
||||
use gridcraft_engine::model::{CalcMode, Formula, Sheet, SheetViewMode, Workbook};
|
||||
use gridcraft_engine::{EngineError, Session, command_specs, find_command, io};
|
||||
@@ -433,7 +433,12 @@ fn value_diff(a: &Workbook, b: &Workbook, what: &str) -> Option<String> {
|
||||
}
|
||||
for c in keys {
|
||||
let (va, vb) = (x.value(c), y.value(c));
|
||||
if !same_value(&va, &vb) {
|
||||
// A file caches the Python errors as #VALUE! (they aren't error codes of the file
|
||||
// format), in a Python cell and in the formulas that read one; reading it back runs
|
||||
// nothing.
|
||||
let python_error = matches!(va, Value::Error(CellError::Python | CellError::Busy | CellError::Timeout | CellError::Blocked))
|
||||
&& vb == Value::Error(CellError::Value);
|
||||
if !same_value(&va, &vb) && !python_error {
|
||||
let f = x.cell(c).and_then(|c| c.formula.as_ref()).map(|f| format!(" (formula ={})", f.text)).unwrap_or_default();
|
||||
return Some(format!("{}!{c}{f}: {va:?} in memory, {vb:?} after {what}", x.name));
|
||||
}
|
||||
@@ -787,6 +792,7 @@ const CURATED: &[(&str, usize)] = &[
|
||||
("formulas.defineName", 2),
|
||||
("formulas.deleteName", 1),
|
||||
("edit.replace", 1),
|
||||
("python.set", 3),
|
||||
];
|
||||
|
||||
fn generate(rng: &mut Rng, s: &Session) -> Step {
|
||||
@@ -820,6 +826,25 @@ fn curated_params(rng: &mut Rng, wb: &Workbook, id: &str) -> Json {
|
||||
p
|
||||
}
|
||||
"range.setValues" => json!({"range": near_cell(rng).a1(), "values": values_2d(rng, wb)}),
|
||||
"python.set" => {
|
||||
// Code with xl() references (which follow structural edits), quotes and line breaks;
|
||||
// sometimes long enough to meet the formula limit.
|
||||
let mut code = match rng.below(5) {
|
||||
0 => format!("xl(\"{}\") + 1", near_range(rng).a1()),
|
||||
1 => format!("a = xl('{}')\nb = \"q\\\"uote\"\na", near_cell(rng).a1()),
|
||||
2 => "[[1, 2], [3, 4]]".to_string(),
|
||||
3 => String::new(),
|
||||
_ => "\"".repeat(rng.int(1, 5000) as usize),
|
||||
};
|
||||
if rng.chance(5) {
|
||||
code.push_str(&"x".repeat(8_100));
|
||||
}
|
||||
let mut p = json!({"cell": near_cell(rng).a1(), "code": code});
|
||||
if rng.chance(30) {
|
||||
p["output"] = json!(if rng.chance(50) { "object" } else { "value" });
|
||||
}
|
||||
p
|
||||
}
|
||||
"range.fill" => json!({"range": near_range(rng).a1(), "input": input(rng, wb)}),
|
||||
"selection.set" => {
|
||||
let mut p = json!({"range": range(rng)});
|
||||
|
||||
@@ -6,6 +6,7 @@ use gridcraft_core::{RangeRef, Value};
|
||||
use gridcraft_formula::Expr;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::python::PyCode;
|
||||
use crate::style::StyleId;
|
||||
|
||||
/// A parsed formula. `text` is the canonical formula text without the leading `=`.
|
||||
@@ -60,25 +61,45 @@ pub struct CellPicture {
|
||||
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Cell {
|
||||
/// Constant value, or the cached result of the formula.
|
||||
/// Constant value, or the cached result of the formula or Python code.
|
||||
pub value: Value,
|
||||
pub formula: Option<Arc<Formula>>,
|
||||
pub style: StyleId,
|
||||
/// Python code, for a Python cell (which has no formula).
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub py: Option<Arc<PyCode>>,
|
||||
}
|
||||
|
||||
impl Cell {
|
||||
pub fn value(v: Value) -> Cell {
|
||||
Cell { value: v, formula: None, style: StyleId::DEFAULT }
|
||||
Cell { value: v, formula: None, style: StyleId::DEFAULT, py: None }
|
||||
}
|
||||
pub fn formula(f: Formula) -> Cell {
|
||||
Cell { value: Value::Empty, formula: Some(Arc::new(f)), style: StyleId::DEFAULT }
|
||||
Cell { value: Value::Empty, formula: Some(Arc::new(f)), style: StyleId::DEFAULT, py: None }
|
||||
}
|
||||
/// No value, no formula and the default style: storing it is pointless.
|
||||
/// A value cell with this style.
|
||||
pub fn styled(v: Value, style: StyleId) -> Cell {
|
||||
Cell { value: v, formula: None, style, py: None }
|
||||
}
|
||||
/// No value, no formula, no Python code and the default style: storing it is pointless.
|
||||
pub fn is_blank(&self) -> bool {
|
||||
self.value.is_empty() && self.formula.is_none() && self.style == StyleId::DEFAULT
|
||||
self.value.is_empty() && self.formula.is_none() && self.style == StyleId::DEFAULT && self.py.is_none()
|
||||
}
|
||||
/// What the formula bar shows.
|
||||
/// Whether the cell holds content (a value, a formula or Python code), formats aside.
|
||||
pub fn has_content(&self) -> bool {
|
||||
!self.value.is_empty() || self.formula.is_some() || self.py.is_some()
|
||||
}
|
||||
/// Removes the content (value, formula and Python code), keeping the style.
|
||||
pub fn clear_content(&mut self) {
|
||||
self.value = Value::Empty;
|
||||
self.formula = None;
|
||||
self.py = None;
|
||||
}
|
||||
/// What the formula bar shows: a formula, a Python cell's code, or the value.
|
||||
pub fn input_text(&self) -> String {
|
||||
if let Some(py) = &self.py {
|
||||
return py.code.clone();
|
||||
}
|
||||
if let Some(f) = &self.formula {
|
||||
return format!("={}", f.text);
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ pub mod features;
|
||||
pub mod macros;
|
||||
pub mod password;
|
||||
pub mod preserved;
|
||||
pub mod python;
|
||||
pub mod sheet;
|
||||
pub mod store;
|
||||
pub mod style;
|
||||
@@ -21,6 +22,7 @@ pub use features::*;
|
||||
pub use macros::*;
|
||||
pub use password::PasswordHash;
|
||||
pub use preserved::{Preserved, PreservedPart, PreservedRel, PreservedSheet, PreservedXml};
|
||||
pub use python::{PyCode, PyOutput};
|
||||
pub use sheet::{DEFAULT_COL_WIDTH, DEFAULT_ROW_HEIGHT, LineIndex, Sheet, SheetViewMode, Visibility, points_to_chars};
|
||||
pub use store::CellStore;
|
||||
pub use style::{
|
||||
|
||||
@@ -0,0 +1,443 @@
|
||||
//! Python cells: the code a cell holds, its last result, and the text forms around it (the
|
||||
//! `PY("…",0|1)` formula of the function's syntax, and the `xl("…")` references inside it). An
|
||||
//! xlsx file keeps the code in its Python part instead (see the xlsx crate).
|
||||
|
||||
use gridcraft_core::{CellError, Value};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Excel's limit on the characters of one formula. A Python cell's code must fit in it once it is
|
||||
/// written as the PY formula of the function's syntax ([`PyCode::to_file_formula`]): quotes are
|
||||
/// doubled and every 255 characters add `"&"`, so the limit on the code itself depends on what the
|
||||
/// code holds.
|
||||
pub const MAX_FORMULA_CHARS: usize = 8_192;
|
||||
|
||||
/// The most cells a Python result may have (a crafted file can claim any size).
|
||||
pub const MAX_RESULT_CELLS: usize = 1_000_000;
|
||||
|
||||
/// The prefix files put before `PY(`, as Excel writes it ([MS-XLSX]: `_xlfn._xlws.PY`).
|
||||
pub const FILE_PREFIX: &str = "_xlfn._xlws.";
|
||||
|
||||
/// Characters in one string literal of a `PY(…)` formula (Excel's limit for a literal).
|
||||
const CHUNK_CHARS: usize = 255;
|
||||
|
||||
/// A Python cell.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(default, rename_all = "camelCase")]
|
||||
pub struct PyCode {
|
||||
pub code: String,
|
||||
/// Show the result as a Python object (true) or convert it to cell values (false).
|
||||
pub object: bool,
|
||||
/// The last result, when the code ran without an error.
|
||||
pub output: Option<PyOutput>,
|
||||
/// The cell's inputs changed while calculation was manual: its value is out of date until
|
||||
/// the workbook is calculated.
|
||||
pub stale: bool,
|
||||
}
|
||||
|
||||
/// A Python result kept on the cell.
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
#[serde(default, rename_all = "camelCase")]
|
||||
pub struct PyOutput {
|
||||
/// The result as cells (an array spills), or its preview as text when it has no cell form.
|
||||
pub value: Value,
|
||||
/// Python's name for the result's type.
|
||||
pub type_name: String,
|
||||
/// The result's `repr`, shortened.
|
||||
pub preview: String,
|
||||
/// Whether `value` is the result converted to cells.
|
||||
pub convertible: bool,
|
||||
}
|
||||
|
||||
impl PyCode {
|
||||
pub fn new(code: &str) -> PyCode {
|
||||
PyCode { code: normalize_code(code), ..PyCode::default() }
|
||||
}
|
||||
|
||||
/// What the cell shows for its result: cell values (an array spills from the cell), or, for a
|
||||
/// Python object, its type name. `None` before the code has run.
|
||||
pub fn result_value(&self) -> Option<Value> {
|
||||
let out = self.output.as_ref()?;
|
||||
let scalar = !matches!(out.value, Value::Array(_)) && out.convertible;
|
||||
Some(if self.object && !scalar { Value::text(out.type_name.as_str()) } else { out.value.clone() })
|
||||
}
|
||||
|
||||
/// The cell as a formula (without `=`): `PY("…"&"…",0)`, with the code in string literals of
|
||||
/// at most 255 characters and the return type Excel documents: 0 for an Excel value, 1 for a
|
||||
/// Python object.
|
||||
pub fn to_formula(&self) -> String {
|
||||
let chunks = chunk(&self.code, CHUNK_CHARS);
|
||||
let lits: Vec<String> = chunks.iter().map(|c| format!("\"{}\"", c.replace('"', "\"\""))).collect();
|
||||
let lits = if lits.is_empty() { "\"\"".to_string() } else { lits.join("&") };
|
||||
format!("PY({lits},{})", u8::from(self.object))
|
||||
}
|
||||
|
||||
/// [`PyCode::to_formula`] with Excel's `_xlfn._xlws.` prefix: the string-literal form
|
||||
/// [`PyCode::from_formula`] reads from files that hold it, and what the size limit measures.
|
||||
/// An xlsx file names the code by index instead ([MS-XLSX] 2.3.10.2).
|
||||
pub fn to_file_formula(&self) -> String {
|
||||
format!("{FILE_PREFIX}{}", self.to_formula())
|
||||
}
|
||||
|
||||
/// Whether the code fits in a PY formula of at most [`MAX_FORMULA_CHARS`] characters.
|
||||
pub fn fits(&self) -> bool {
|
||||
// Code longer than the limit can't fit; this also bounds the work for hostile input.
|
||||
self.code.chars().nth(MAX_FORMULA_CHARS).is_none() && self.to_file_formula().chars().nth(MAX_FORMULA_CHARS).is_none()
|
||||
}
|
||||
|
||||
/// Reads a `PY(…)` formula (with or without `=`, and with or without the `_xlfn.` and
|
||||
/// `_xlws.` prefixes): string literals joined by `&`, optionally followed by `,0` or `,1`.
|
||||
/// `None` for anything else.
|
||||
pub fn from_formula(text: &str) -> Option<PyCode> {
|
||||
let body = py_body(text);
|
||||
let rest = body.get(..3).filter(|h| h.eq_ignore_ascii_case("PY(")).and_then(|_| body.get(3..))?;
|
||||
let mut chars = rest.chars().peekable();
|
||||
let mut code = String::new();
|
||||
loop {
|
||||
skip_space(&mut chars);
|
||||
if chars.next()? != '"' {
|
||||
return None;
|
||||
}
|
||||
loop {
|
||||
match chars.next()? {
|
||||
'"' if chars.peek() == Some(&'"') => {
|
||||
chars.next();
|
||||
code.push('"');
|
||||
}
|
||||
'"' => break,
|
||||
c => code.push(c),
|
||||
}
|
||||
}
|
||||
skip_space(&mut chars);
|
||||
match chars.next()? {
|
||||
'&' => continue,
|
||||
',' => {
|
||||
skip_space(&mut chars);
|
||||
let object = match chars.next()? {
|
||||
'0' => false,
|
||||
'1' => true,
|
||||
_ => return None,
|
||||
};
|
||||
skip_space(&mut chars);
|
||||
return (chars.next()? == ')' && chars.all(char::is_whitespace)).then(|| PyCode { object, ..PyCode::new(&code) });
|
||||
}
|
||||
')' => return chars.all(char::is_whitespace).then(|| PyCode::new(&code)),
|
||||
_ => return None,
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `text` without a leading `=`, spaces and the `_xlfn.` and `_xlws.` prefixes files put before
|
||||
/// newer functions (in any letter case).
|
||||
fn py_body(text: &str) -> &str {
|
||||
let mut body = text.trim().trim_start_matches('=').trim_start();
|
||||
for prefix in ["_xlfn.", "_xlws."] {
|
||||
if body.get(..prefix.len()).is_some_and(|h| h.eq_ignore_ascii_case(prefix)) {
|
||||
body = body.get(prefix.len()..).unwrap_or_default();
|
||||
}
|
||||
}
|
||||
body
|
||||
}
|
||||
|
||||
fn skip_space(chars: &mut std::iter::Peekable<std::str::Chars<'_>>) {
|
||||
while chars.peek().is_some_and(|c| c.is_whitespace()) {
|
||||
chars.next();
|
||||
}
|
||||
}
|
||||
|
||||
/// Line breaks as `\n`.
|
||||
pub fn normalize_code(code: &str) -> String {
|
||||
code.replace("\r\n", "\n").replace('\r', "\n")
|
||||
}
|
||||
|
||||
/// `s` in pieces of at most `n` characters.
|
||||
fn chunk(s: &str, n: usize) -> Vec<String> {
|
||||
let chars: Vec<char> = s.chars().collect();
|
||||
chars.chunks(n.max(1)).map(|c| c.iter().collect()).collect()
|
||||
}
|
||||
|
||||
/// Whether `text` (a formula, with or without `=`) is a `PY(` call: such formulas are Python cells,
|
||||
/// set with `python.set`.
|
||||
pub fn is_py_formula(text: &str) -> bool {
|
||||
let body = py_body(text);
|
||||
body.get(..3).is_some_and(|h| h.eq_ignore_ascii_case("PY(")) || body.eq_ignore_ascii_case("PY")
|
||||
}
|
||||
|
||||
/// An `xl()` call in Python code.
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct XlCall {
|
||||
/// The reference text, when the first argument is a plain string literal.
|
||||
pub reference: Option<String>,
|
||||
/// Byte range of the literal's contents (between the quotes) in the code.
|
||||
pub span: Option<std::ops::Range<usize>>,
|
||||
}
|
||||
|
||||
/// The `xl(…)` calls in `code`, skipping comments and other string literals. A call whose first
|
||||
/// argument isn't a plain string literal has no reference: the cells it reads are only known when
|
||||
/// it runs.
|
||||
pub fn xl_calls(code: &str) -> Vec<XlCall> {
|
||||
let bytes = code.as_bytes();
|
||||
let mut out = Vec::new();
|
||||
let mut i = 0usize;
|
||||
while let Some(&b) = bytes.get(i) {
|
||||
match b {
|
||||
b'#' => {
|
||||
// A comment runs to the end of the line.
|
||||
while bytes.get(i).is_some_and(|c| *c != b'\n') {
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
b'"' | b'\'' => i = skip_string(bytes, i),
|
||||
b'x' if bytes.get(i + 1) == Some(&b'l') && (i == 0 || bytes.get(i - 1).is_some_and(|p| !is_ident(*p) && *p != b'.')) => {
|
||||
let mut j = i + 2;
|
||||
if bytes.get(j).is_some_and(|c| is_ident(*c)) {
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
while bytes.get(j).is_some_and(|c| *c == b' ' || *c == b'\t') {
|
||||
j += 1;
|
||||
}
|
||||
if bytes.get(j) != Some(&b'(') {
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
j += 1;
|
||||
while bytes.get(j).is_some_and(|c| c.is_ascii_whitespace()) {
|
||||
j += 1;
|
||||
}
|
||||
let call = match bytes.get(j) {
|
||||
Some(&q @ (b'"' | b'\'')) => {
|
||||
let start = j + 1;
|
||||
let end = skip_string(bytes, j);
|
||||
// A plain literal: closed on its own line, not part of an expression
|
||||
// (`xl("A" + n)`), not a triple-quoted string, and escaping nothing but a
|
||||
// quote or a backslash (`\'`, `\"`, `\\`), which a sheet name may need.
|
||||
let close = end.saturating_sub(1);
|
||||
let mut k = end;
|
||||
while bytes.get(k).is_some_and(|c| *c == b' ' || *c == b'\t') {
|
||||
k += 1;
|
||||
}
|
||||
let plain = close > start
|
||||
&& bytes.get(close) == Some(&q)
|
||||
&& bytes.get(start) != Some(&q)
|
||||
&& matches!(bytes.get(k), Some(b')' | b','));
|
||||
i = end;
|
||||
match code.get(start..close).filter(|_| plain).and_then(unescape) {
|
||||
Some(t) => XlCall { reference: Some(t), span: Some(start..close) },
|
||||
None => XlCall { reference: None, span: None },
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
i = j;
|
||||
XlCall { reference: None, span: None }
|
||||
}
|
||||
};
|
||||
out.push(call);
|
||||
}
|
||||
_ => i += 1,
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn is_ident(b: u8) -> bool {
|
||||
b.is_ascii_alphanumeric() || b == b'_' || b >= 0x80
|
||||
}
|
||||
|
||||
/// The index just past the string literal starting at `at` (single, double or triple quoted).
|
||||
fn skip_string(bytes: &[u8], at: usize) -> usize {
|
||||
let Some(&q) = bytes.get(at) else { return at.saturating_add(1) };
|
||||
let triple = bytes.get(at + 1) == Some(&q) && bytes.get(at + 2) == Some(&q);
|
||||
let mut i = if triple { at + 3 } else { at + 1 };
|
||||
while let Some(&c) = bytes.get(i) {
|
||||
if c == b'\\' {
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
if c == q {
|
||||
if !triple {
|
||||
return i + 1;
|
||||
}
|
||||
if bytes.get(i + 1) == Some(&q) && bytes.get(i + 2) == Some(&q) {
|
||||
return i + 3;
|
||||
}
|
||||
} else if c == b'\n' && !triple {
|
||||
return i;
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
i
|
||||
}
|
||||
|
||||
/// The text of a plain literal's contents: `\'`, `\"` and `\\` stand for the character after the
|
||||
/// backslash. `None` for any other escape or a line break, which make the literal not plain.
|
||||
fn unescape(raw: &str) -> Option<String> {
|
||||
let mut out = String::with_capacity(raw.len());
|
||||
let mut chars = raw.chars();
|
||||
while let Some(c) = chars.next() {
|
||||
match c {
|
||||
'\n' => return None,
|
||||
'\\' => match chars.next()? {
|
||||
e @ ('\\' | '\'' | '"') => out.push(e),
|
||||
_ => return None,
|
||||
},
|
||||
c => out.push(c),
|
||||
}
|
||||
}
|
||||
Some(out)
|
||||
}
|
||||
|
||||
/// `text` as a plain literal that was delimited by `quote`: its delimiter (`quote`, or the other
|
||||
/// quote when `text` holds `quote` but not the other one, so `xl('Sheet2!B2')` becomes
|
||||
/// `xl("'Q4 2026'!B2")` on a rename) and its contents, with backslashes escaped, and the
|
||||
/// delimiter too when `text` holds both quotes. `None` for a line break, which no literal on
|
||||
/// one line holds.
|
||||
fn literal(text: &str, quote: char) -> Option<(char, String)> {
|
||||
if text.contains('\n') {
|
||||
return None;
|
||||
}
|
||||
let other = if quote == '\'' { '"' } else { '\'' };
|
||||
let delim = if text.contains(quote) && !text.contains(other) { other } else { quote };
|
||||
let mut body = String::with_capacity(text.len());
|
||||
for c in text.chars() {
|
||||
if c == '\\' || c == delim {
|
||||
body.push('\\');
|
||||
}
|
||||
body.push(c);
|
||||
}
|
||||
Some((delim, body))
|
||||
}
|
||||
|
||||
/// Rewrites the reference of every `xl("…")` literal in `code` with `f`; `None` from `f` leaves
|
||||
/// it. Returns the new code when anything changed. A new reference holding the literal's quote
|
||||
/// (a sheet name that needs quotes, `'Q4 2026'!B2`) gets the other quote as its delimiter, or an
|
||||
/// escaped quote when it holds both, so it never ends the string early.
|
||||
pub fn rewrite_xl(code: &str, mut f: impl FnMut(&str) -> Option<String>) -> Option<String> {
|
||||
let mut out = String::with_capacity(code.len());
|
||||
let mut last = 0usize;
|
||||
let mut changed = false;
|
||||
for call in xl_calls(code) {
|
||||
let (Some(reference), Some(span)) = (call.reference, call.span) else { continue };
|
||||
let Some(new) = f(&reference).filter(|n| *n != reference) else { continue };
|
||||
let open = span.start.saturating_sub(1);
|
||||
let Some(quote) = code.get(open..span.start).and_then(|q| q.chars().next()) else { continue };
|
||||
let Some((delim, body)) = literal(&new, quote) else { continue };
|
||||
// The literal with its quotes: the delimiter may change.
|
||||
out.push_str(code.get(last..open).unwrap_or_default());
|
||||
out.push(delim);
|
||||
out.push_str(&body);
|
||||
out.push(delim);
|
||||
last = span.end.saturating_add(1);
|
||||
changed = true;
|
||||
}
|
||||
if !changed {
|
||||
return None;
|
||||
}
|
||||
out.push_str(code.get(last..).unwrap_or_default());
|
||||
Some(out)
|
||||
}
|
||||
|
||||
/// The value a Python cell shows when it can't run: `#BLOCKED!`.
|
||||
pub const BLOCKED: Value = Value::Error(CellError::Blocked);
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn formula_round_trip_with_chunks_and_quotes() {
|
||||
let code = format!("s = \"quoted\"\n{}\nlen(s)", "x".repeat(600));
|
||||
let py = PyCode { object: true, ..PyCode::new(&code) };
|
||||
let f = py.to_formula();
|
||||
assert!(f.starts_with("PY(\"s = \"\"quoted\"\"\n"));
|
||||
assert!(f.ends_with(",1)"));
|
||||
assert_eq!(f.matches("\"&\"").count(), 2, "{f}");
|
||||
let back = PyCode::from_formula(&format!("={f}")).unwrap();
|
||||
assert_eq!(back.code, code);
|
||||
assert!(back.object);
|
||||
assert_eq!(PyCode::from_formula("_xlfn.PY(\"1+1\")").unwrap().code, "1+1");
|
||||
// Excel writes `_xlfn._xlws.PY`; the reader takes it in any letter case.
|
||||
assert_eq!(PyCode::from_formula("=_xlfn._xlws.PY(\"1+1\",1)").unwrap().code, "1+1");
|
||||
assert_eq!(PyCode::from_formula("_XLFN._XLWS.py(\"2\",0)").unwrap().code, "2");
|
||||
assert_eq!(PyCode::from_formula("_xlws.PY(\"3\",0)").unwrap().code, "3");
|
||||
assert!(is_py_formula("=_xlfn._xlws.PY(\"1\",0)"));
|
||||
assert_eq!(py.to_file_formula(), format!("_xlfn._xlws.{f}"));
|
||||
assert_eq!(PyCode::from_formula(&py.to_file_formula()).unwrap(), PyCode::from_formula(&f).unwrap());
|
||||
assert_eq!(PyCode::from_formula(" py( \"a\" & \"b\" , 0 ) ").unwrap().code, "ab");
|
||||
assert_eq!(PyCode::new("a\r\nb").code, "a\nb");
|
||||
for bad in ["PY(A1)", "PY(\"a\",2)", "PY(\"a\"", "PY(\"a\")+1", "SUM(1)", "PY(\"a\"&B1)", "PY(", ""] {
|
||||
assert!(PyCode::from_formula(bad).is_none(), "{bad}");
|
||||
}
|
||||
assert!(is_py_formula("=PY(\"1\")") && is_py_formula("=py") && !is_py_formula("=PYTHON(1)"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_size_limit_counts_the_encoded_formula() {
|
||||
// Plain code: the limit is the formula's, so it is a little under 8,192 characters of code.
|
||||
// `_xlfn._xlws.PY(` + 31 literals of 255 + 30 `&` + `&"…"` + `,0)`: 8,079 characters of
|
||||
// code give a formula of exactly 8,192.
|
||||
let at = PyCode::new(&"y".repeat(8_079));
|
||||
assert_eq!(at.to_file_formula().chars().count(), MAX_FORMULA_CHARS);
|
||||
assert!(at.fits());
|
||||
assert!(!PyCode::new(&"y".repeat(8_080)).fits());
|
||||
assert_eq!(PyCode::from_formula(&at.to_file_formula()).unwrap().code, at.code);
|
||||
// Quotes are doubled in the file, so 7,000 characters with 1,200 of them don't fit.
|
||||
let quoted = format!("{}{}", "\"".repeat(1_200), "y".repeat(5_800));
|
||||
assert!(PyCode::new("ed).to_file_formula().chars().count() > MAX_FORMULA_CHARS);
|
||||
assert!(!PyCode::new("ed).fits());
|
||||
// Hostile sizes are refused without encoding them.
|
||||
assert!(!PyCode::new(&"y".repeat(4_000_000)).fits());
|
||||
assert!(PyCode::new("").fits());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn result_values() {
|
||||
let arr = gridcraft_core::Array::new(2, 1, vec![1.0.into(), 2.0.into()]).map(Value::from).unwrap();
|
||||
let out = PyOutput { value: arr.clone(), type_name: "list".into(), preview: "[1, 2]".into(), convertible: true };
|
||||
let mut py = PyCode { output: Some(out), ..PyCode::new("[1, 2]") };
|
||||
assert_eq!(py.result_value(), Some(arr));
|
||||
py.object = true;
|
||||
assert_eq!(py.result_value(), Some(Value::text("list")));
|
||||
py.output = Some(PyOutput { value: Value::Number(3.0), type_name: "int".into(), preview: "3".into(), convertible: true });
|
||||
assert_eq!(py.result_value(), Some(Value::Number(3.0)), "scalars show as values in object mode too");
|
||||
py.output = None;
|
||||
assert_eq!(py.result_value(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn finds_xl_literals() {
|
||||
let code = "a = xl(\"A1:B2\", headers=True)\n# xl('C1') in a comment\ns = \"xl('D1')\"\nb = xl( 'Sheet2!E5' )\nc = xl(name)\nd = df.xl('F1')\nxlx('G1')\ne = xl(\"A\" + \"1\")";
|
||||
let calls = xl_calls(code);
|
||||
let refs: Vec<Option<&str>> = calls.iter().map(|c| c.reference.as_deref()).collect();
|
||||
assert_eq!(refs, vec![Some("A1:B2"), Some("Sheet2!E5"), None, None]);
|
||||
let new = rewrite_xl(code, |r| (r == "A1:B2").then(|| "A2:B3".to_string())).unwrap();
|
||||
assert!(new.starts_with("a = xl(\"A2:B3\", headers=True)"));
|
||||
assert_eq!(new.len(), code.len());
|
||||
assert_eq!(rewrite_xl(code, |_| None), None);
|
||||
// A replacement can't break out of its string: a quote in it changes the delimiter, or is
|
||||
// escaped when it holds both quotes, and the result reads back as the same reference.
|
||||
assert_eq!(rewrite_xl("xl(\"A1\")", |_| Some("'Q'!A1".into())), Some("xl(\"'Q'!A1\")".into()));
|
||||
assert_eq!(rewrite_xl("xl('Sheet2!B2', headers=True)", |_| Some("'Q4 2026'!B2".into())), Some("xl(\"'Q4 2026'!B2\", headers=True)".into()));
|
||||
assert_eq!(rewrite_xl("xl(\"A1\")", |_| Some("'Q\"4'!A1".into())), Some("xl(\"'Q\\\"4'!A1\")".into()));
|
||||
assert_eq!(rewrite_xl("xl('A1')", |_| Some("'Q\"4'!A1".into())), Some("xl('\\'Q\"4\\'!A1')".into()));
|
||||
for (delim, new) in [("'", "'Q4 2026'!B2"), ("'", "'Q\"4'!A1"), ("\"", "'Q\"4'!A1"), ("\"", "'a\\b'!A1"), ("'", "B2")] {
|
||||
let code = format!("x = xl({delim}A1{delim}) + 1");
|
||||
let out = rewrite_xl(&code, |_| Some(new.to_string())).unwrap();
|
||||
assert_eq!(xl_calls(&out)[0].reference.as_deref(), Some(new), "{out}");
|
||||
assert!(out.ends_with(") + 1"), "{out}");
|
||||
}
|
||||
assert_eq!(rewrite_xl("xl('A1')", |_| Some("A\n1".into())), None, "a line break can't go in a literal");
|
||||
// Escaped quotes and backslashes read as the characters; any other escape is not plain.
|
||||
assert_eq!(xl_calls("xl('Bob\\'s!A1')")[0].reference.as_deref(), Some("Bob's!A1"));
|
||||
assert_eq!(xl_calls("xl(\"A\\\\1\")")[0].reference.as_deref(), Some("A\\1"));
|
||||
assert_eq!(xl_calls("xl(\"A\\n1\")")[0].reference, None);
|
||||
assert_eq!(xl_calls("xl(\"A1\\\")")[0].reference, None, "an escaped closing quote leaves the literal open");
|
||||
// Unclosed and odd input doesn't loop or panic.
|
||||
for odd in ["xl(", "xl(\"", "xl('''A1''')", "x", "xl", "\"unterminated", "xl ( \"A1\" )", "é xl('A1')"] {
|
||||
let _ = xl_calls(odd);
|
||||
let _ = rewrite_xl(odd, |_| Some("B2".into()));
|
||||
}
|
||||
assert_eq!(xl_calls("xl ( \"A1\" )")[0].reference.as_deref(), Some("A1"));
|
||||
assert_eq!(xl_calls("xl('''A1''')")[0].reference, None);
|
||||
}
|
||||
}
|
||||
@@ -202,6 +202,42 @@ impl Sheet {
|
||||
self.spill_ranges.iter().find(|(anchor, range)| **anchor != c && range.contains(c)).map(|(anchor, _)| *anchor)
|
||||
}
|
||||
|
||||
/// Removes the spilled values inside `r`. The work is bounded by the values stored, not by
|
||||
/// the size of `r`, so a range from a file that claims the whole sheet costs no more than the
|
||||
/// values it holds.
|
||||
pub fn clear_spill(&mut self, r: RangeRef) {
|
||||
if r.count() <= self.spill.len() as u64 {
|
||||
for c in r.iter() {
|
||||
self.spill.remove(&c);
|
||||
}
|
||||
} else {
|
||||
self.spill.retain(|c, _| !r.contains(*c));
|
||||
}
|
||||
}
|
||||
|
||||
/// Turns the spill anchored at `anchor` into plain values, for a Python cell that keeps only
|
||||
/// its value: the range is forgotten, and each spilled value in it becomes its cell's value,
|
||||
/// unless the cell holds content of its own (its format is kept either way). The work is
|
||||
/// bounded by the spilled values stored, not by the size of the range.
|
||||
pub fn settle_spill(&mut self, anchor: CellRef) {
|
||||
let Some(r) = self.spill_ranges.remove(&anchor) else { return };
|
||||
let inside: Vec<CellRef> = self.spill.range(r.start..=r.end).map(|(c, _)| *c).filter(|c| r.contains(*c)).collect();
|
||||
for c in inside {
|
||||
let Some(v) = self.spill.remove(&c) else { continue };
|
||||
if v.is_empty() || c == anchor {
|
||||
continue;
|
||||
}
|
||||
match self.cells.get_mut(c) {
|
||||
Some(cell) if cell.has_content() => {}
|
||||
Some(cell) => cell.value = v,
|
||||
None => {
|
||||
let style = self.style_id(c);
|
||||
self.cells.set(c, Cell::styled(v, style));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The floating object with workbook-unique id `id` on this sheet.
|
||||
pub fn object_ref(&self, id: u32) -> Option<ObjRef> {
|
||||
let kind = if self.charts.iter().any(|o| o.id == id) {
|
||||
@@ -338,7 +374,7 @@ impl Sheet {
|
||||
}
|
||||
pub fn set_value(&mut self, c: CellRef, v: Value) {
|
||||
let style = self.style_id(c);
|
||||
self.set_cell(c, Cell { value: v, formula: None, style });
|
||||
self.set_cell(c, Cell { value: v, formula: None, style, py: None });
|
||||
}
|
||||
/// Replaces content, removing any picture at this address. Style-only edits use set_style.
|
||||
pub fn set_cell(&mut self, c: CellRef, cell: Cell) {
|
||||
@@ -357,7 +393,7 @@ impl Sheet {
|
||||
/// Pictures retain a scalar fallback for calculation and unsupported consumers.
|
||||
pub fn set_picture(&mut self, c: CellRef, picture: Arc<CellPicture>) {
|
||||
let style = self.style_id(c);
|
||||
self.cells.set(c, Cell { value: Value::Error(gridcraft_core::CellError::Value), formula: None, style });
|
||||
self.cells.set(c, Cell { value: Value::Error(gridcraft_core::CellError::Value), formula: None, style, py: None });
|
||||
self.cell_pictures.insert(c, picture);
|
||||
}
|
||||
pub fn input_text(&self, c: CellRef) -> String {
|
||||
|
||||
@@ -14,7 +14,7 @@ use crate::cell::Cell;
|
||||
const BAND: u32 = 64;
|
||||
|
||||
fn has_value(cell: &Cell) -> bool {
|
||||
!cell.value.is_empty() || cell.formula.is_some()
|
||||
cell.has_content()
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
use super::{SheetApp, clean_error, editor, formula_bar, formula_locale, json, now_ms};
|
||||
use super::{Json, SheetApp, clean_error, editor, formula_bar, formula_locale, json, now_ms};
|
||||
|
||||
impl SheetApp {
|
||||
/// Starts editing the active cell. `text` replaces the content (typing) or pre-fills it.
|
||||
@@ -42,6 +42,7 @@ impl SheetApp {
|
||||
None => (current.clone(), false),
|
||||
};
|
||||
let mut ed = editor::EditState::new(d.uid, d.wb.active_sheet(), at, text, replace, from_formula_bar);
|
||||
ed.code_edit = !replace && sh.cell(at).is_some_and(|c| c.py.is_some());
|
||||
ed.window = self.session.active_window();
|
||||
ed.locale = locale;
|
||||
if replace && self.ui.autocomplete && !ed.is_formula() {
|
||||
@@ -99,11 +100,12 @@ impl SheetApp {
|
||||
let Some(ed) = self.editor.take() else { return true };
|
||||
self.session.mode = gridcraft_engine::Mode::Ready;
|
||||
let text = match &ed.completion {
|
||||
Some(full) if full.to_lowercase().starts_with(&ed.text.to_lowercase()) => full.clone(),
|
||||
// A code edit commits the code as typed, whatever a completion holds.
|
||||
Some(full) if !ed.code_edit && full.to_lowercase().starts_with(&ed.text.to_lowercase()) => full.clone(),
|
||||
_ => ed.text.clone(),
|
||||
};
|
||||
// F2 then Enter (or a file command) on an untouched draft: nothing to write.
|
||||
if !fill_selection && !array && ed.original.as_ref() == Some(&text) {
|
||||
if (ed.code_edit || (!fill_selection && !array)) && ed.original.as_ref() == Some(&text) {
|
||||
if dr != 0 || dc != 0 {
|
||||
self.move_after_enter(dr, dc);
|
||||
}
|
||||
@@ -116,6 +118,11 @@ impl SheetApp {
|
||||
self.message = Some(("GridCraft".into(), m));
|
||||
return true;
|
||||
}
|
||||
// All the code deleted: the cell is cleared through cell.set, as an empty entry clears any
|
||||
// other cell. Code of spaces is still code.
|
||||
if ed.code_edit && !text.is_empty() {
|
||||
return self.commit_code_edit(ed, text, dr, dc);
|
||||
}
|
||||
let text = match self.session.active().map(|d| formula_locale::canonical(&text, ed.locale, &d.wb, ed.sheet, ed.cell)) {
|
||||
Some(Ok(text)) => text,
|
||||
Some(Err(error)) => {
|
||||
@@ -153,11 +160,9 @@ impl SheetApp {
|
||||
let unchanged = !fill_selection
|
||||
&& !array
|
||||
&& text.is_empty()
|
||||
&& self
|
||||
.session
|
||||
.active()
|
||||
.and_then(|d| d.wb.sheet(ed.sheet))
|
||||
.is_some_and(|sh| sh.cell(ed.cell).is_none_or(|c| c.formula.is_none() && c.value == gridcraft_engine::core::Value::Empty));
|
||||
&& self.session.active().and_then(|d| d.wb.sheet(ed.sheet)).is_some_and(|sh| {
|
||||
sh.cell(ed.cell).is_none_or(|c| c.formula.is_none() && c.py.is_none() && c.value == gridcraft_engine::core::Value::Empty)
|
||||
});
|
||||
let r = if unchanged {
|
||||
Ok(json!({}))
|
||||
} else if fill_selection {
|
||||
@@ -182,6 +187,34 @@ impl SheetApp {
|
||||
}
|
||||
}
|
||||
|
||||
/// Commits a changed edit of a Python cell's code through `python.set`, which keeps the cell's
|
||||
/// return type; `cell.set` would replace the Python cell with an ordinary one holding the code
|
||||
/// as text. When the command refuses the code, its message shows and the edit stays open.
|
||||
fn commit_code_edit(&mut self, ed: editor::EditState, code: String, dr: i64, dc: i64) -> bool {
|
||||
// Point mode can leave another sheet active; the edit belongs to the sheet it started on.
|
||||
let r = if self.session.active().is_some_and(|d| d.wb.active_sheet() != ed.sheet) {
|
||||
self.session.run("sheet.activate", json!({"sheet": ed.sheet}))
|
||||
} else {
|
||||
Ok(Json::Null)
|
||||
};
|
||||
let r = r.and_then(|_| self.session.run("python.set", json!({"sheet": ed.sheet, "cell": ed.cell.a1(), "code": code})));
|
||||
match r {
|
||||
Ok(_) => {
|
||||
if dr != 0 || dc != 0 {
|
||||
self.move_after_enter(dr, dc);
|
||||
}
|
||||
self.after_engine();
|
||||
true
|
||||
}
|
||||
Err(e) => {
|
||||
self.message = Some(("GridCraft".into(), clean_error(&e)));
|
||||
self.editor = Some(ed);
|
||||
self.session.mode = gridcraft_engine::Mode::Edit;
|
||||
false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Commits the edit with an arrow key. The arrow is an ordinary move, so it ends a run of Tabs
|
||||
/// instead of continuing it (Right) or going back to its column like Enter (Down), as in Excel.
|
||||
pub fn commit_edit_arrow(&mut self, dr: i64, dc: i64) -> bool {
|
||||
|
||||
@@ -47,6 +47,10 @@ pub struct EditState {
|
||||
/// The cell's text when an F2-style edit began. Committing it unchanged leaves the cell alone
|
||||
/// (no undo step, the workbook stays clean). `None` for an edit started by typing.
|
||||
pub original: Option<String>,
|
||||
/// The edit began on a Python cell's existing content (F2, a double click or the formula
|
||||
/// bar), so the text is the cell's code: a change goes in as new code through `python.set`.
|
||||
/// `false` for an edit started by typing, which replaces the cell as it does a formula.
|
||||
pub code_edit: bool,
|
||||
}
|
||||
|
||||
impl EditState {
|
||||
@@ -75,6 +79,7 @@ impl EditState {
|
||||
caret_to_end: true,
|
||||
completion: None,
|
||||
original,
|
||||
code_edit: false,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -173,7 +173,8 @@ fn active_input(app: &SheetApp) -> String {
|
||||
.unwrap_or_default();
|
||||
}
|
||||
if let Some(c) = sh.cell(a) {
|
||||
if c.formula.is_some() && d.wb.styles.get(c.style).protection.hidden && sh.is_protected() {
|
||||
// A Hidden cell on a protected sheet shows neither its formula nor its Python code.
|
||||
if (c.formula.is_some() || c.py.is_some()) && d.wb.styles.get(c.style).protection.hidden && sh.is_protected() {
|
||||
return String::new();
|
||||
}
|
||||
return if let Some(f) = &c.formula {
|
||||
@@ -389,8 +390,10 @@ pub fn editor_widget(app: &mut SheetApp, ui: &mut egui::Ui, id: egui::Id, font:
|
||||
}
|
||||
if resp.changed() {
|
||||
ed.completion = None;
|
||||
// Python code is never completed from the column's text values.
|
||||
if app.ui.autocomplete
|
||||
&& !ed.is_formula()
|
||||
&& !ed.code_edit
|
||||
&& !entry_shrank(chars_before, &ed.text)
|
||||
&& ed.caret == ed.text.chars().count()
|
||||
&& let Some(sh) = app.session.active().and_then(|d| d.wb.sheet(ed.sheet))
|
||||
@@ -753,6 +756,23 @@ mod spill_tests {
|
||||
app.run("sheet.activate", json!({"sheet": "Sheet1"})).unwrap();
|
||||
check(&mut app, "A2", "=FILTER(D1:D3,D1:D3>0)", true);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_hidden_python_cell_on_a_protected_sheet_shows_no_code() {
|
||||
let mut app = app();
|
||||
app.run("python.set", json!({"cell": "A1", "code": "secret = 1\nsecret"})).unwrap();
|
||||
app.run("python.set", json!({"cell": "B1", "code": "'shown'"})).unwrap();
|
||||
set(&mut app, "C1", "=1+1");
|
||||
app.run("home.formatCells", json!({"range": "A1:C1", "style": {"protection": {"locked": true, "hidden": true}}})).unwrap();
|
||||
app.run("home.formatCells", json!({"range": "B1", "style": {"protection": {"locked": true, "hidden": false}}})).unwrap();
|
||||
select(&mut app, "A1");
|
||||
assert_eq!(active_input(&app), "secret = 1\nsecret", "an unprotected sheet hides nothing");
|
||||
app.run("review.protectSheet", json!({})).unwrap();
|
||||
for (cell, text) in [("A1", ""), ("B1", "'shown'"), ("C1", "")] {
|
||||
select(&mut app, cell);
|
||||
assert_eq!(active_input(&app), text, "{cell}");
|
||||
}
|
||||
}
|
||||
#[test]
|
||||
/// A disabled spill formula paints disabled and does not edit; clicking an active formula starts formula-bar editing in both light and dark themes, with accessibility checks.
|
||||
fn formula_bar_click_and_theme_presentation() {
|
||||
|
||||
@@ -86,6 +86,7 @@ mod pivot_show_details_menu;
|
||||
mod popup_dismissal;
|
||||
mod protect_password;
|
||||
mod protected_cell;
|
||||
mod python_cell_edit;
|
||||
mod ribbon_layout;
|
||||
mod right_to_left;
|
||||
mod scroll_position;
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
//! Editing a Python cell's content (F2, the formula bar) edits its code; typing over it replaces
|
||||
//! it, as typing over a formula does.
|
||||
|
||||
use egui::{Event, Key, Modifiers};
|
||||
use gridcraft_engine::Session;
|
||||
use gridcraft_engine::core::{CellRef, Value};
|
||||
use gridcraft_ui_egui::SheetApp;
|
||||
use serde_json::json;
|
||||
|
||||
type Harness = egui_kittest::Harness<'static, SheetApp>;
|
||||
|
||||
fn harness() -> Harness {
|
||||
let mut session = Session::new();
|
||||
session.new_workbook();
|
||||
session.execute("python.set", json!({"cell": "A1", "code": "1 + 1", "output": "object"})).unwrap();
|
||||
session.execute("selection.set", json!({"range": "A1"})).unwrap();
|
||||
crate::common::harness_with(crate::common::builder().with_size(egui::vec2(1200.0, 800.0)), crate::common::app(session), 4)
|
||||
}
|
||||
|
||||
fn text(h: &mut Harness, text: &str) {
|
||||
h.input_mut().events.push(Event::Text(text.into()));
|
||||
h.run_steps(2);
|
||||
}
|
||||
|
||||
fn key(h: &mut Harness, key: Key) {
|
||||
for pressed in [true, false] {
|
||||
h.input_mut().events.push(Event::Key { key, physical_key: None, pressed, repeat: false, modifiers: Modifiers::NONE });
|
||||
}
|
||||
h.run_steps(2);
|
||||
}
|
||||
|
||||
/// A1's code and return type (true: a Python object), or `None` when it isn't a Python cell.
|
||||
fn python(h: &Harness) -> Option<(String, bool)> {
|
||||
let sh = h.state().session.active().unwrap().wb.active().unwrap();
|
||||
sh.cell(CellRef::parse("A1").unwrap()).and_then(|c| c.py.as_deref()).map(|p| (p.code.clone(), p.object))
|
||||
}
|
||||
|
||||
fn select_a1(h: &mut Harness) {
|
||||
h.state_mut().session.execute("selection.set", json!({"range": "A1"})).unwrap();
|
||||
h.run_steps(2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn f2_edits_a_python_cells_code_and_typing_replaces_it() {
|
||||
let mut h = harness();
|
||||
let undo_depth = |h: &Harness| h.state().session.active().unwrap().undo.len();
|
||||
let before = undo_depth(&h);
|
||||
|
||||
// F2 and Enter without a change writes nothing.
|
||||
key(&mut h, Key::F2);
|
||||
assert_eq!(h.state().editor.as_ref().map(|e| e.text.as_str()), Some("1 + 1"));
|
||||
key(&mut h, Key::Enter);
|
||||
assert!(h.state().editor.is_none());
|
||||
assert_eq!(undo_depth(&h), before, "an unchanged edit is no undo step");
|
||||
assert_eq!(python(&h), Some(("1 + 1".into(), true)));
|
||||
|
||||
// A change is new code for the same Python cell, which keeps its return type.
|
||||
select_a1(&mut h);
|
||||
key(&mut h, Key::F2);
|
||||
text(&mut h, " + 2");
|
||||
key(&mut h, Key::Enter);
|
||||
assert!(h.state().editor.is_none(), "{:?}", h.state().message);
|
||||
assert_eq!(python(&h), Some(("1 + 1 + 2".into(), true)), "the cell stays a Python cell with the new code");
|
||||
|
||||
// Typing over the cell replaces it.
|
||||
select_a1(&mut h);
|
||||
text(&mut h, "5");
|
||||
key(&mut h, Key::Enter);
|
||||
assert_eq!(python(&h), None);
|
||||
assert_eq!(h.state().session.active().unwrap().wb.active().unwrap().value(CellRef::parse("A1").unwrap()), Value::Number(5.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deleting_all_the_code_clears_the_cell() {
|
||||
let mut h = harness();
|
||||
let a1 = |h: &Harness| h.state().session.active().unwrap().wb.active().unwrap().value(CellRef::parse("A1").unwrap());
|
||||
// Code of spaces is still code, for the same Python cell.
|
||||
key(&mut h, Key::F2);
|
||||
h.state_mut().editor.as_mut().unwrap().text = " ".into();
|
||||
assert!(h.state_mut().commit_edit(0, 0, false, false));
|
||||
assert_eq!(python(&h), Some((" ".into(), true)));
|
||||
// No code at all clears the cell, as an empty entry clears any other cell.
|
||||
select_a1(&mut h);
|
||||
key(&mut h, Key::F2);
|
||||
h.state_mut().editor.as_mut().unwrap().text.clear();
|
||||
assert!(h.state_mut().commit_edit(0, 0, false, false));
|
||||
assert!(h.state().editor.is_none());
|
||||
assert_eq!(python(&h), None, "the cell is no longer a Python cell");
|
||||
assert_eq!(a1(&h), Value::Empty);
|
||||
h.state_mut().run("edit.undo", json!({})).unwrap();
|
||||
assert_eq!(python(&h), Some((" ".into(), true)), "one undo step brings the code back");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deleting_all_the_code_clears_a_python_cell_with_no_cached_value() {
|
||||
let mut h = harness();
|
||||
// A Python cell from a file that cached no value (the value is empty, the code is not).
|
||||
{
|
||||
let d = h.state_mut().session.doc_mut().unwrap();
|
||||
let mut wb = (*d.wb).clone();
|
||||
let i = wb.active_sheet();
|
||||
let sh = wb.sheet_mut(i).unwrap();
|
||||
let at = CellRef::parse("A1").unwrap();
|
||||
let mut cell = sh.cell(at).cloned().unwrap();
|
||||
cell.value = Value::Empty;
|
||||
sh.set_cell(at, cell);
|
||||
d.wb = std::sync::Arc::new(wb);
|
||||
}
|
||||
h.run_steps(2);
|
||||
key(&mut h, Key::F2);
|
||||
h.state_mut().editor.as_mut().unwrap().text.clear();
|
||||
assert!(h.state_mut().commit_edit(0, 0, false, false));
|
||||
assert!(h.state().editor.is_none());
|
||||
assert_eq!(python(&h), None, "the code is removed, not kept as an unchanged empty cell");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_code_edit_is_never_completed_from_the_column() {
|
||||
let mut h = harness();
|
||||
// A2 holds text that starts with A1's code: typing at the end of the code would offer it.
|
||||
h.state_mut().session.execute("cell.set", json!({"cell": "A2", "input": "1 + 1 and more"})).unwrap();
|
||||
select_a1(&mut h);
|
||||
key(&mut h, Key::F2);
|
||||
text(&mut h, " ");
|
||||
assert_eq!(h.state().editor.as_ref().and_then(|e| e.completion.clone()), None, "no completion while code is edited");
|
||||
key(&mut h, Key::Enter);
|
||||
assert_eq!(python(&h), Some(("1 + 1 ".into(), true)), "the typed code is kept");
|
||||
}
|
||||
@@ -21,7 +21,7 @@ fn build(rows: u32) -> Workbook {
|
||||
let products = ["Widget", "Gadget", "Gizmo", "Doohickey", "Sprocket", "Thingamajig", "Whatsit"];
|
||||
let mut s = Sheet::new("Sales");
|
||||
let set = |s: &mut Sheet, row: u32, col: u32, value: Value, style: StyleId| {
|
||||
s.cells.set(CellRef::new(row, col), Cell { value, formula: None, style });
|
||||
s.cells.set(CellRef::new(row, col), Cell { value, formula: None, style, py: None });
|
||||
};
|
||||
for (col, title) in ["Order", "Date", "Region", "Product", "Units", "Price", "Revenue", "Share", "Paid", "Note"].iter().enumerate() {
|
||||
set(&mut s, 0, col as u32, Value::text(*title), bold);
|
||||
@@ -53,10 +53,10 @@ fn build(rows: u32) -> Workbook {
|
||||
let mut total = Cell::formula(Formula::new(&format!("=SUM(Sales!G2:G{})", rows + 1)));
|
||||
total.value = Value::Number(1.0e7);
|
||||
summary.cells.set(CellRef::new(0, 1), total);
|
||||
summary.cells.set(CellRef::new(0, 0), Cell { value: Value::text("Total"), formula: None, style: bold });
|
||||
summary.cells.set(CellRef::new(0, 0), Cell { value: Value::text("Total"), formula: None, style: bold, py: None });
|
||||
for (k, region) in regions.iter().enumerate() {
|
||||
let row = k as u32 + 2;
|
||||
summary.cells.set(CellRef::new(row, 0), Cell { value: Value::text(*region), formula: None, style: StyleId::DEFAULT });
|
||||
summary.cells.set(CellRef::new(row, 0), Cell { value: Value::text(*region), formula: None, style: StyleId::DEFAULT, py: None });
|
||||
let mut f = Cell::formula(Formula::new(&format!("=SUMIFS(Sales!G:G,Sales!C:C,A{})", row + 1)));
|
||||
f.value = Value::Number(2.0e6);
|
||||
f.style = money;
|
||||
|
||||
@@ -523,7 +523,7 @@ pub fn read_csv_with_report(bytes: &[u8], opts: &CsvOptions) -> Result<(Workbook
|
||||
},
|
||||
None => StyleId::DEFAULT,
|
||||
};
|
||||
Cell { value: p.value, formula: None, style }
|
||||
Cell { value: p.value, formula: None, style, py: None }
|
||||
} else {
|
||||
Cell::value(Value::Text(Arc::from(f)))
|
||||
};
|
||||
|
||||
@@ -25,6 +25,7 @@ mod omml;
|
||||
mod package;
|
||||
mod passthrough;
|
||||
mod pivot;
|
||||
mod python_part;
|
||||
mod read;
|
||||
mod richdata;
|
||||
mod sheet_read;
|
||||
|
||||
@@ -61,6 +61,19 @@ impl<'a> Package<'a> {
|
||||
self.read_limited(part, MAX_PART)
|
||||
}
|
||||
|
||||
/// Reads a part the reader can do without (the caller warns and goes on when it fails). A read
|
||||
/// that fails partway is charged this part's whole allowance (`maximum`, and the byte that
|
||||
/// detects an overrun) instead of ending the package's: the decompression work stays bounded,
|
||||
/// and the parts read after it still open.
|
||||
pub fn read_optional_limited(&mut self, part: &str, maximum: u64) -> Result<Option<Vec<u8>>, IoError> {
|
||||
let before = self.total;
|
||||
let read = self.read_limited(part, maximum);
|
||||
if let Err(IoError::Zip(_)) = &read {
|
||||
self.total = before.saturating_add(maximum.min(MAX_PART).saturating_add(1)).min(MAX_TOTAL);
|
||||
}
|
||||
read
|
||||
}
|
||||
|
||||
/// Reads a small metadata part without allowing it to expand to the general part limit.
|
||||
pub fn read_limited(&mut self, part: &str, maximum: u64) -> Result<Option<Vec<u8>>, IoError> {
|
||||
let maximum = maximum.min(MAX_PART);
|
||||
@@ -227,4 +240,27 @@ mod tests {
|
||||
// Oversized repeated attempts stay cheap errors now: no allowance is left to pay for.
|
||||
assert!(matches!(pkg.read("b.xml"), Err(IoError::TooLarge(_))));
|
||||
}
|
||||
|
||||
/// A part the reader can do without, damaged inside an intact archive, costs its own
|
||||
/// allowance: the parts after it are still read.
|
||||
#[test]
|
||||
fn a_damaged_optional_part_costs_only_its_allowance() {
|
||||
use std::io::Write as _;
|
||||
let mut w = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
|
||||
let stored = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored);
|
||||
w.start_file("a.xml", stored).unwrap();
|
||||
w.write_all(b"<a>damaged</a>").unwrap();
|
||||
w.start_file("b.xml", zip::write::SimpleFileOptions::default()).unwrap();
|
||||
w.write_all(b"hello").unwrap();
|
||||
let mut z = w.finish().unwrap().into_inner();
|
||||
let at = z.windows(7).position(|x| x == b"damaged").unwrap();
|
||||
z[at] ^= 0x20;
|
||||
let mut pkg = Package::open(&z).unwrap();
|
||||
assert!(matches!(pkg.read_optional_limited("a.xml", 1024), Err(IoError::Zip(_))));
|
||||
assert_eq!(pkg.total, 1025, "the failed read is charged its allowance");
|
||||
assert_eq!(pkg.read("b.xml").unwrap().as_deref(), Some(&b"hello"[..]));
|
||||
// An ordinary read of it still ends the package's budget.
|
||||
assert!(pkg.read("a.xml").is_err());
|
||||
assert_eq!(pkg.total, MAX_TOTAL);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,336 @@
|
||||
//! The Python part ([MS-XLSX] 2.1.23, 2.3.10): where a workbook keeps the code of its Python cells.
|
||||
//!
|
||||
//! A Python cell's formula is `_xlfn._xlws.PY(index, return_type, …)` ([MS-XLSX] 2.3.10.2): the
|
||||
//! first argument is the zero-based index of a `pythonScript` element in this part, whose `code`
|
||||
//! is the cell's Python code, and the second is 0 (an Excel value) or 1 (a Python object). Further
|
||||
//! arguments are references the code names as `%Px%`, `x` being the argument's zero-based index
|
||||
//! ([MS-XLSX] 2.3.10.4). The part's root is `python` ([MS-XLSX] 2.4.102, CT_Python 2.6.249), with
|
||||
//! `pythonScripts` (2.6.253) holding the `pythonScript` elements (2.6.252).
|
||||
//!
|
||||
//! GridCraft writes one script per Python cell, in the order the cells are written, with the code
|
||||
//! as it is (its `xl()` references stay text in the code, so no further arguments). Reading
|
||||
//! resolves the index and puts each `%Px%` back as the text of argument `x`. A formula that names
|
||||
//! no script the part has keeps only its cached value. Hostile input is bounded: the part's size,
|
||||
//! the number of scripts, each script's length, and the code all cells take from the part
|
||||
//! together.
|
||||
|
||||
use gridcraft_formula::Expr;
|
||||
use gridcraft_model::PyCode;
|
||||
use gridcraft_model::python::{FILE_PREFIX, MAX_FORMULA_CHARS};
|
||||
|
||||
use crate::IoError;
|
||||
use crate::read::Ctx;
|
||||
use crate::xml::{self, El};
|
||||
|
||||
/// Namespace of the part.
|
||||
pub const NS: &str = "http://schemas.microsoft.com/office/spreadsheetml/2023/python";
|
||||
/// Relationship type from the workbook part.
|
||||
pub const REL: &str = "http://schemas.microsoft.com/office/2023/09/relationships/Python";
|
||||
/// Last segment of [`REL`], as [`crate::package::Rel::kind`] holds it.
|
||||
const REL_KIND: &str = "Python";
|
||||
/// Content type of the part.
|
||||
pub const CONTENT_TYPE: &str = "application/vnd.ms-excel.python+xml";
|
||||
/// Part name we write (the format names none; the relationship finds it).
|
||||
pub const PART: &str = "xl/python.xml";
|
||||
|
||||
/// Largest Python part read.
|
||||
const MAX_PART_BYTES: u64 = 64 * 1024 * 1024;
|
||||
/// Most scripts read from the part.
|
||||
const MAX_SCRIPTS: usize = 1_000_000;
|
||||
/// Most characters kept of one script: one more than a cell's code may have, so a longer script
|
||||
/// still fails [`PyCode::fits`] and its cell keeps only its value, with the usual warning.
|
||||
const MAX_SCRIPT_CHARS: usize = MAX_FORMULA_CHARS + 1;
|
||||
/// Most bytes of code all the cells of a workbook take from the part together. Several formulas
|
||||
/// may name one script, so without it a small part could fill memory.
|
||||
pub const MAX_CODE_BYTES: usize = 64 * 1024 * 1024;
|
||||
|
||||
/// The scripts read from a workbook's Python part, and what the cells took from them.
|
||||
#[derive(Default)]
|
||||
pub struct ScriptsIn {
|
||||
scripts: Vec<String>,
|
||||
/// Bytes of code the cells may still take ([`MAX_CODE_BYTES`] in all).
|
||||
budget: usize,
|
||||
/// Python cells that kept only their value: no such script, or over the budget.
|
||||
missing: usize,
|
||||
}
|
||||
|
||||
impl ScriptsIn {
|
||||
/// The warning for the Python cells that kept only their value, if any.
|
||||
pub fn warning(&self) -> Option<String> {
|
||||
(self.missing > 0).then(|| {
|
||||
format!(
|
||||
"{} Python cell(s) name code the file's Python part doesn't hold (or holds too much of); they kept only their values",
|
||||
self.missing
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Reads the workbook's Python part, if it has one. A damaged part is a warning: its cells keep
|
||||
/// their values.
|
||||
pub fn read(cx: &mut Ctx<'_>, rels: &[crate::package::Rel]) -> Result<ScriptsIn, IoError> {
|
||||
let mut out = ScriptsIn { budget: MAX_CODE_BYTES, ..ScriptsIn::default() };
|
||||
let Some(rel) = rels.iter().find(|r| !r.external && r.kind.eq_ignore_ascii_case(REL_KIND)) else { return Ok(out) };
|
||||
let bytes = match cx.pkg.read_optional_limited(&rel.target, MAX_PART_BYTES) {
|
||||
Ok(Some(b)) => b,
|
||||
Ok(None) => {
|
||||
cx.warn(format!("missing part {}", rel.target));
|
||||
return Ok(out);
|
||||
}
|
||||
Err(IoError::TooLarge(m)) => {
|
||||
cx.warn(format!("the Python part is too large ({m}); Python cells kept only their values"));
|
||||
return Ok(out);
|
||||
}
|
||||
// A damaged entry (a bad checksum or compressed stream) is a damaged part: its cells keep
|
||||
// their values, as when the part is missing.
|
||||
Err(e) => {
|
||||
cx.warn(format!("the Python part is damaged ({e}); Python cells kept only their values"));
|
||||
return Ok(out);
|
||||
}
|
||||
};
|
||||
let mut over = 0usize;
|
||||
let parsed = xml::parse_streaming(&bytes, &["pythonScript"], &mut |el: El| {
|
||||
if out.scripts.len() < MAX_SCRIPTS {
|
||||
out.scripts.push(script_code(&el));
|
||||
} else {
|
||||
over += 1;
|
||||
}
|
||||
Ok(())
|
||||
});
|
||||
match parsed {
|
||||
Ok(root) if root.name != "python" => {
|
||||
cx.warn(format!("the Python part has the root <{}>; Python cells kept only their values", root.name));
|
||||
out.scripts.clear();
|
||||
}
|
||||
Ok(_) => {}
|
||||
Err(e) => cx.warn(format!("the Python part is damaged ({e}); kept {} scripts", out.scripts.len())),
|
||||
}
|
||||
if over > 0 {
|
||||
cx.warn(format!("the Python part holds more than {MAX_SCRIPTS} scripts; {over} were not read"));
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Encoded characters of a script that are decoded: enough for its first [`MAX_SCRIPT_CHARS`]
|
||||
/// characters of code, whatever its escapes. One character takes at most 14 encoded characters
|
||||
/// (a UTF-16 pair of `_xHHHH_` escapes) and a `\r\n` line break becomes one character, so
|
||||
/// `2 × (MAX_SCRIPT_CHARS + 1)` decoded characters cover the code kept, and two more escapes'
|
||||
/// worth keep the last of them, and the line break after it, from being cut.
|
||||
const MAX_ENCODED_CHARS: usize = 14 * (2 * (MAX_SCRIPT_CHARS + 1) + 2);
|
||||
|
||||
/// A `pythonScript` element's code (ST_Xstring), with `\n` line breaks, cut after
|
||||
/// [`MAX_SCRIPT_CHARS`] characters. The limit is measured on the decoded code, so escapes in the
|
||||
/// file (`_x000D_`, `_x0009_`) don't cut a script that fits; decoding is linear and reads at most
|
||||
/// [`MAX_ENCODED_CHARS`] characters.
|
||||
fn script_code(el: &El) -> String {
|
||||
let text = el.child("code").map(|c| c.text.as_str()).unwrap_or_default();
|
||||
let encoded = match text.char_indices().nth(MAX_ENCODED_CHARS) {
|
||||
Some((at, _)) => text.get(..at).unwrap_or_default(),
|
||||
None => text,
|
||||
};
|
||||
let code = gridcraft_model::python::normalize_code(&xml::decode_xstring(encoded));
|
||||
match code.char_indices().nth(MAX_SCRIPT_CHARS) {
|
||||
Some((at, _)) => code.get(..at).unwrap_or_default().to_string(),
|
||||
None => code,
|
||||
}
|
||||
}
|
||||
|
||||
/// `PY(index, 0|1, params…)`, as a file's formula holds it: the index, whether the return type is
|
||||
/// a Python object, and the parameters. `None` for anything else (the code in a string literal
|
||||
/// is [`PyCode::from_formula`]'s form).
|
||||
fn py_call(expr: &Expr) -> Option<(usize, bool, &[Expr])> {
|
||||
let Expr::Call(name, args) = expr else { return None };
|
||||
let base = name.trim_start_matches("_XLFN.").trim_start_matches("_XLWS.");
|
||||
if !base.eq_ignore_ascii_case("PY") {
|
||||
return None;
|
||||
}
|
||||
let index = match args.first()? {
|
||||
Expr::Number(n) if *n >= 0.0 && n.fract() == 0.0 && *n < MAX_SCRIPTS as f64 => *n as usize,
|
||||
_ => return None,
|
||||
};
|
||||
let object = match args.get(1)? {
|
||||
Expr::Number(n) if *n == 0.0 => false,
|
||||
Expr::Number(n) if *n == 1.0 => true,
|
||||
_ => return None,
|
||||
};
|
||||
Some((index, object, args.get(2..).unwrap_or_default()))
|
||||
}
|
||||
|
||||
/// What a file formula is to Python.
|
||||
#[derive(Debug, PartialEq)]
|
||||
pub enum PyFormula {
|
||||
/// Not a `PY(…)` call: an ordinary formula.
|
||||
Not,
|
||||
/// A Python cell with this code.
|
||||
Code(PyCode),
|
||||
/// A `PY(index, …)` call naming a script the part doesn't hold (or over the budget): the
|
||||
/// cell keeps only its cached value, since no calculation can evaluate the formula.
|
||||
Missing,
|
||||
}
|
||||
|
||||
/// What a file formula makes: a Python cell from the Python part when it names a script, or from
|
||||
/// string literals ([`PyCode::from_formula`]). `formula` is the parsed formula, `raw` the
|
||||
/// formula's own text, which keeps line breaks in literal code.
|
||||
pub fn cell_code(scripts: &mut ScriptsIn, formula: &gridcraft_model::Formula, raw: &str) -> PyFormula {
|
||||
// The cheap test first: most formulas are not `PY(` calls.
|
||||
if !gridcraft_model::python::is_py_formula(&formula.text) {
|
||||
return PyFormula::Not;
|
||||
}
|
||||
let expr = formula.expr_arc();
|
||||
let Some((index, object, params)) = expr.as_deref().and_then(py_call) else {
|
||||
return PyCode::from_formula(raw).map_or(PyFormula::Not, PyFormula::Code);
|
||||
};
|
||||
let code = scripts.scripts.get(index).map(|code| with_params(code, params));
|
||||
match code.filter(|c| c.len() <= scripts.budget) {
|
||||
Some(code) => {
|
||||
scripts.budget -= code.len();
|
||||
PyFormula::Code(PyCode { object, ..PyCode::new(&code) })
|
||||
}
|
||||
None => {
|
||||
scripts.missing += 1;
|
||||
PyFormula::Missing
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// `code` with every `%Px%` (`x` at least 2, naming one of `params`) replaced by the parameter's
|
||||
/// text. Stops growing past [`MAX_SCRIPT_CHARS`] characters: such code can't be a cell's.
|
||||
fn with_params(code: &str, params: &[Expr]) -> String {
|
||||
if params.is_empty() || !code.contains("%P") {
|
||||
return code.to_string();
|
||||
}
|
||||
let texts: Vec<String> = params.iter().map(gridcraft_formula::print).collect();
|
||||
let mut out = String::with_capacity(code.len());
|
||||
let mut chars = 0usize;
|
||||
let mut push = |out: &mut String, piece: &str| {
|
||||
out.push_str(piece);
|
||||
chars = chars.saturating_add(piece.chars().count());
|
||||
chars > MAX_SCRIPT_CHARS
|
||||
};
|
||||
let mut rest = code;
|
||||
while let Some(at) = rest.find("%P") {
|
||||
if push(&mut out, rest.get(..at).unwrap_or_default()) {
|
||||
return out;
|
||||
}
|
||||
let after = rest.get(at + 2..).unwrap_or_default();
|
||||
let digits = after.bytes().take_while(u8::is_ascii_digit).count();
|
||||
let param = (1..=6)
|
||||
.contains(&digits)
|
||||
.then(|| after.get(..digits).and_then(|d| d.parse::<usize>().ok()))
|
||||
.flatten()
|
||||
.filter(|_| after.as_bytes().get(digits) == Some(&b'%'))
|
||||
.and_then(|x| x.checked_sub(2))
|
||||
.and_then(|x| texts.get(x));
|
||||
let full = match param {
|
||||
Some(text) => {
|
||||
rest = after.get(digits + 1..).unwrap_or_default();
|
||||
push(&mut out, text)
|
||||
}
|
||||
None => {
|
||||
rest = after;
|
||||
push(&mut out, "%P")
|
||||
}
|
||||
};
|
||||
if full {
|
||||
return out;
|
||||
}
|
||||
}
|
||||
out.push_str(rest);
|
||||
out
|
||||
}
|
||||
|
||||
/// The formula a Python cell is saved as: `_xlfn._xlws.PY(index,0|1)`, its code being script
|
||||
/// `index` of the part ([`write`]).
|
||||
pub fn file_formula(index: usize, object: bool) -> String {
|
||||
format!("{FILE_PREFIX}PY({index},{})", u8::from(object))
|
||||
}
|
||||
|
||||
/// The Python part holding `scripts`, in order. Only written when there is at least one: the
|
||||
/// format requires one `pythonScript` in `pythonScripts`.
|
||||
pub fn write(scripts: &[String]) -> String {
|
||||
let mut xml = format!("{}<python xmlns=\"{NS}\"><pythonScripts>", crate::write::XML_DECL);
|
||||
for code in scripts {
|
||||
xml.push_str("<pythonScript><code>");
|
||||
xml.push_str(&xml::esc(&xml::encode_xstring(code)));
|
||||
xml.push_str("</code></pythonScript>");
|
||||
}
|
||||
xml.push_str("</pythonScripts></python>");
|
||||
xml
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn formula(text: &str) -> gridcraft_model::Formula {
|
||||
gridcraft_model::Formula::new(&crate::fmla::from_file(text))
|
||||
}
|
||||
|
||||
fn scripts(list: &[&str]) -> ScriptsIn {
|
||||
ScriptsIn { scripts: list.iter().map(|s| s.to_string()).collect(), budget: MAX_CODE_BYTES, missing: 0 }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reads_the_index_form_and_parameters() {
|
||||
let mut s = scripts(&["1 + 1", "df = xl(\"%P2%\", headers=True)\nxl('%P3%') + %P9% + %P1%"]);
|
||||
let code = |s: &mut ScriptsIn, text: &str| match cell_code(s, &formula(text), text) {
|
||||
PyFormula::Code(py) => Some(py),
|
||||
_ => None,
|
||||
};
|
||||
let a = code(&mut s, "_xlfn._xlws.PY(0,1)").unwrap();
|
||||
assert_eq!((a.code.as_str(), a.object), ("1 + 1", true));
|
||||
let b = code(&mut s, "_xlfn._xlws.PY(1,0,A1:B3,Sheet2!$C$4)").unwrap();
|
||||
assert_eq!(b.code, "df = xl(\"A1:B3\", headers=True)\nxl('Sheet2!$C$4') + %P9% + %P1%");
|
||||
assert!(!b.object);
|
||||
// The literal form still reads (other applications, older files).
|
||||
assert_eq!(code(&mut s, "_xlfn._xlws.PY(\"x = 1\",0)").unwrap().code, "x = 1");
|
||||
// A script the part doesn't hold: the cell keeps its value, and is counted for the warning.
|
||||
let gone = "_xlfn._xlws.PY(7,0)";
|
||||
assert_eq!(cell_code(&mut s, &formula(gone), gone), PyFormula::Missing);
|
||||
assert!(s.warning().is_some());
|
||||
for not in ["_xlfn._xlws.PY(-1,0)", "_xlfn._xlws.PY(0.5,0)", "_xlfn._xlws.PY(0,2)", "PY(A1,0)", "SUM(0,1)", "PYTHON_TYPE(A1)"] {
|
||||
assert_eq!(cell_code(&mut s, &formula(not), not), PyFormula::Not, "{not}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hostile_scripts_are_bounded() {
|
||||
// Many formulas naming one script take from one budget.
|
||||
let code = "x".repeat(1_000);
|
||||
let mut s = ScriptsIn { scripts: vec![code], budget: 2_500, missing: 0 };
|
||||
let f = formula("_xlfn._xlws.PY(0,0)");
|
||||
let taken = (0..5).filter(|_| matches!(cell_code(&mut s, &f, ""), PyFormula::Code(_))).count();
|
||||
assert_eq!(taken, 2);
|
||||
assert_eq!(s.missing, 3);
|
||||
// A parameter repeated many times stops growing the code just past a cell's limit.
|
||||
let many = "%P2%".repeat(5_000);
|
||||
let out = with_params(&many, &[Expr::Name("A_VERY_LONG_NAME_FOR_A_RANGE".into())]);
|
||||
assert!((MAX_SCRIPT_CHARS..MAX_SCRIPT_CHARS + 40).contains(&out.chars().count()), "{}", out.len());
|
||||
assert!(!PyCode::new(&out).fits());
|
||||
// Odd tokens stay as they are.
|
||||
assert_eq!(with_params("%P%P2%%P99999999%%P", &[Expr::Number(1.0)]), "%P1%P99999999%%P");
|
||||
// A long script is cut just past the limit, so it fails the cell's check.
|
||||
let el = El {
|
||||
name: "pythonScript".into(),
|
||||
children: vec![El { name: "code".into(), text: "y".repeat(20_000), ..El::default() }],
|
||||
..El::default()
|
||||
};
|
||||
assert_eq!(script_code(&el).chars().count(), MAX_SCRIPT_CHARS);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn written_scripts_read_back() {
|
||||
let codes = vec!["a = 1\nb = \"two\" & 'three' <tag>".to_string(), "_x0041_ \u{1}".to_string(), String::new()];
|
||||
let xml = write(&codes);
|
||||
assert!(xml.contains(NS) && xml.contains("<pythonScripts><pythonScript><code>a = 1\nb = \"two\" & 'three' <tag></code>"), "{xml}");
|
||||
let mut read = Vec::new();
|
||||
let root = xml::parse_streaming(xml.as_bytes(), &["pythonScript"], &mut |el| {
|
||||
read.push(script_code(&el));
|
||||
Ok(())
|
||||
})
|
||||
.unwrap_or_default();
|
||||
assert_eq!(root.name, "python");
|
||||
assert_eq!(read, codes);
|
||||
assert_eq!(file_formula(3, true), "_xlfn._xlws.PY(3,1)");
|
||||
}
|
||||
}
|
||||
@@ -37,6 +37,8 @@ pub struct Ctx<'a> {
|
||||
/// Whether GridCraft calculates a function, when the caller says: cell formulas wrapped by
|
||||
/// Google Sheets' export are then read as the formula inside (see [`crate::read_xlsx_with`]).
|
||||
pub known_function: Option<&'a dyn Fn(&str) -> bool>,
|
||||
/// The code of the workbook's Python cells, from its Python part.
|
||||
pub python: crate::python_part::ScriptsIn,
|
||||
}
|
||||
|
||||
impl Ctx<'_> {
|
||||
@@ -251,6 +253,7 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) ->
|
||||
cell_pictures: vec![],
|
||||
cf_ext_ids: vec![],
|
||||
known_function,
|
||||
python: Default::default(),
|
||||
};
|
||||
let root_rels = cx.pkg.rels("")?;
|
||||
let wb_part = find_rel(&root_rels, "officeDocument").map(|r| r.target.clone()).unwrap_or_else(|| "xl/workbook.xml".into());
|
||||
@@ -264,6 +267,8 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) ->
|
||||
vec![]
|
||||
}
|
||||
};
|
||||
// The code of Python cells, which their formulas name by index.
|
||||
cx.python = crate::python_part::read(&mut cx, &wb_rels)?;
|
||||
let mut wb = Workbook::new();
|
||||
wb.sheets.clear();
|
||||
|
||||
@@ -440,6 +445,9 @@ pub fn read_xlsx<'a>(bytes: &'a [u8], known_function: Option<&'a dyn Fn(&str) ->
|
||||
}
|
||||
}
|
||||
wb.preserved = kept.workbook(&mut cx, &wb_part, &wb_xml, &file_to_model);
|
||||
if let Some(w) = cx.python.warning() {
|
||||
cx.warn(w);
|
||||
}
|
||||
// Calls of LAMBDA names are spelled like the names (the parser upper-cases them).
|
||||
wb.apply_name_call_case();
|
||||
wb.styles = cx.styles;
|
||||
|
||||
@@ -48,16 +48,17 @@ struct RowState {
|
||||
rows: u64,
|
||||
}
|
||||
|
||||
/// Whether `pos` lies in a known dynamic range that isn't its anchor: the cell's cached value
|
||||
/// gives way to the formula that will fill it. Charges every scanned range against one work
|
||||
/// budget, then gives up consulting the ranges at all: the worst kind of file may then import
|
||||
/// a few cached spill values, but it never stalls the whole import.
|
||||
fn in_dynamic(st: &mut RowState, pos: CellRef) -> bool {
|
||||
/// The known dynamic range `pos` lies in, when it isn't the range's anchor: the cell's cached
|
||||
/// value gives way to the formula that will fill it (or, for a Python cell, becomes its
|
||||
/// result). Charges every scanned range against one work budget, then gives up consulting the
|
||||
/// ranges at all: the worst kind of file may then import a few cached spill values, but it
|
||||
/// never stalls the whole import.
|
||||
fn dynamic_range(st: &mut RowState, pos: CellRef) -> Option<RangeRef> {
|
||||
if st.dynamic_checks > MAX_DYNAMIC_CHECKS || st.dynamic.is_empty() {
|
||||
return false;
|
||||
return None;
|
||||
}
|
||||
let ranges = st.dynamic.len();
|
||||
let hit = st.dynamic.iter().any(|r| r.contains(pos) && r.start != pos);
|
||||
let hit = st.dynamic.iter().find(|r| r.contains(pos) && r.start != pos).copied();
|
||||
st.dynamic_checks = st.dynamic_checks.saturating_add(ranges as u64);
|
||||
hit
|
||||
}
|
||||
@@ -78,6 +79,47 @@ fn push_dynamic(st: &mut RowState, range: RangeRef) {
|
||||
st.dynamic.push(range);
|
||||
}
|
||||
|
||||
/// Forgets the range anchored at `pos` (a Python cell whose code the file doesn't hold, which
|
||||
/// keeps only its cached value): the cached values over the rest of its spill are kept as plain
|
||||
/// values too, as the file shows them, rather than left for a result that won't come. Charged
|
||||
/// like [`dynamic_range`]'s scans.
|
||||
fn forget_dynamic(st: &mut RowState, pos: CellRef) {
|
||||
if st.dynamic_checks > MAX_DYNAMIC_CHECKS {
|
||||
return;
|
||||
}
|
||||
st.dynamic_checks = st.dynamic_checks.saturating_add(st.dynamic.len() as u64);
|
||||
if let Some(i) = st.dynamic.iter().rposition(|r| r.start == pos) {
|
||||
st.dynamic.remove(i);
|
||||
}
|
||||
}
|
||||
|
||||
/// Records the spill range of the Python cell read at `pos`, its anchor, when the file declares
|
||||
/// one that a result can fill: the spill then exists even when the file holds no records for its
|
||||
/// blank cells. Charged like [`dynamic_range`]'s scans.
|
||||
fn record_python_spill(st: &mut RowState, sheet: &mut Sheet, pos: CellRef) {
|
||||
if st.dynamic_checks > MAX_DYNAMIC_CHECKS {
|
||||
return;
|
||||
}
|
||||
st.dynamic_checks = st.dynamic_checks.saturating_add(st.dynamic.len() as u64);
|
||||
let Some(r) = st.dynamic.iter().rev().find(|r| r.start == pos).copied() else { return };
|
||||
if r.count() > 1 && r.count() <= gridcraft_model::python::MAX_RESULT_CELLS as u64 {
|
||||
sheet.spill_ranges.insert(pos, r);
|
||||
}
|
||||
}
|
||||
|
||||
/// Past the scan budget, the cells of a Python spill read afterwards were kept as plain cells, not
|
||||
/// as its spilled values, and a result rebuilt over them would block itself. Every Python spill of
|
||||
/// such a sheet keeps its values as plain values instead, with no live spill.
|
||||
fn settle_python_spills_past_budget(st: &RowState, sheet: &mut Sheet) {
|
||||
if st.dynamic_checks <= MAX_DYNAMIC_CHECKS {
|
||||
return;
|
||||
}
|
||||
let anchors: Vec<CellRef> = sheet.spill_ranges.keys().copied().filter(|a| sheet.cells.get(*a).is_some_and(|c| c.py.is_some())).collect();
|
||||
for anchor in anchors {
|
||||
sheet.settle_spill(anchor);
|
||||
}
|
||||
}
|
||||
|
||||
const MAX_HYPERLINK_CELLS: u64 = 10_000;
|
||||
/// Target and tooltip bytes kept for hyperlinks of one sheet, over all cells: the same URL is
|
||||
/// copied for every cell of its range, so length × cells must stay bounded whatever the file.
|
||||
@@ -134,6 +176,7 @@ pub fn read_sheet(cx: &mut Ctx<'_>, part: &str, name: &str) -> Result<Sheet, IoE
|
||||
if st.dropped > 0 {
|
||||
cx.warn(format!("sheet \"{name}\": {} cells or rows outside the 1,048,576 × 16,384 grid were dropped", st.dropped));
|
||||
}
|
||||
settle_python_spills_past_budget(&st, &mut sheet);
|
||||
drop(bytes);
|
||||
|
||||
let mut fit_to_page = false;
|
||||
@@ -526,8 +569,45 @@ fn read_cell(cx: &mut Ctx<'_>, sheet: &mut Sheet, st: &mut RowState, c: &El, pos
|
||||
}
|
||||
let dynamic = c.attr("cm").is_some();
|
||||
let formula = c.child("f").and_then(|f| read_formula(cx, st, f, pos, dynamic));
|
||||
let value = if formula.is_none() && in_dynamic(st, pos) { Value::Empty } else { value };
|
||||
sheet.cells.set(pos, Cell { value, formula: formula.map(Arc::new), style });
|
||||
// A `PY(index,0|1)` formula is a Python cell whose code is a script of the Python part. The
|
||||
// older `PY("…",0|1)` form is read from the formula's own text: the formula parser would fold
|
||||
// line breaks in the code. A formula naming a script the part lacks keeps only its cached
|
||||
// values, its own and its spill's, as plain values.
|
||||
let py = match (&formula, c.child("f")) {
|
||||
(Some(fm), Some(f)) => crate::python_part::cell_code(&mut cx.python, fm, &f.text),
|
||||
_ => crate::python_part::PyFormula::Not,
|
||||
};
|
||||
let (formula, py) = match py {
|
||||
crate::python_part::PyFormula::Not => (formula, None),
|
||||
crate::python_part::PyFormula::Code(py) => (None, Some(Arc::new(py))),
|
||||
crate::python_part::PyFormula::Missing => {
|
||||
forget_dynamic(st, pos);
|
||||
(None, None)
|
||||
}
|
||||
};
|
||||
if py.is_some() {
|
||||
record_python_spill(st, sheet, pos);
|
||||
}
|
||||
let spilled_from = if formula.is_none() && py.is_none() { dynamic_range(st, pos) } else { None };
|
||||
let python_anchor = spilled_from.is_some_and(|r| sheet.cells.get(r.start).is_some_and(|a| a.py.is_some()));
|
||||
let value = match spilled_from {
|
||||
// A Python cell's spilled values are kept: they are its result until it runs again.
|
||||
Some(r) if python_anchor && r.count() <= gridcraft_model::python::MAX_RESULT_CELLS as u64 => {
|
||||
if !value.is_empty() {
|
||||
sheet.spill.insert(pos, value);
|
||||
}
|
||||
sheet.spill_ranges.insert(r.start, r);
|
||||
Value::Empty
|
||||
}
|
||||
// A Python cell's range larger than a result may be is a crafted file's: it is no spill,
|
||||
// and its cells keep the values the file shows as plain values, as when the cell keeps
|
||||
// only its own value.
|
||||
Some(_) if python_anchor => value,
|
||||
// A formula's spilled values are dropped: recalculation fills them again.
|
||||
Some(_) => Value::Empty,
|
||||
None => value,
|
||||
};
|
||||
sheet.cells.set(pos, Cell { value, formula: formula.map(Arc::new), style, py });
|
||||
}
|
||||
|
||||
fn read_formula(cx: &mut Ctx<'_>, st: &mut RowState, f: &El, pos: CellRef, dynamic: bool) -> Option<Formula> {
|
||||
|
||||
@@ -78,7 +78,9 @@ pub fn write_sheet(wb: &Workbook, si: usize, selected: bool, out: &mut Out) -> (
|
||||
let dynamic: BTreeMap<CellRef, RangeRef> = sheet
|
||||
.spill_ranges
|
||||
.iter()
|
||||
.filter(|(a, r)| !r.is_single() && sheet.cells.get(**a).is_some_and(|c| c.formula.as_ref().is_some_and(|f| f.array.is_none())))
|
||||
.filter(|(a, r)| {
|
||||
!r.is_single() && sheet.cells.get(**a).is_some_and(|c| c.py.is_some() || c.formula.as_ref().is_some_and(|f| f.array.is_none()))
|
||||
})
|
||||
.map(|(a, r)| (*a, *r))
|
||||
.collect();
|
||||
let mut extra: BTreeMap<CellRef, Value> = BTreeMap::new();
|
||||
@@ -721,6 +723,17 @@ fn write_cell(s: &mut String, input: CellWriteInput<'_>, sid: &dyn Fn(gridcraft_
|
||||
return;
|
||||
}
|
||||
let formula = if ov.is_some() { None } else { cell.and_then(|x| x.formula.as_ref()) };
|
||||
// A Python cell is saved as [MS-XLSX] 2.3.10 specifies, `_xlfn._xlws.PY(index,0|1)` with its
|
||||
// code as script `index` of the Python part, and with its cached value; an application without
|
||||
// Python shows the value (and #NAME? if it recalculates).
|
||||
let py_text = if ov.is_some() || formula.is_some() {
|
||||
None
|
||||
} else {
|
||||
cell.and_then(|x| x.py.as_ref()).map(|p| {
|
||||
out.python_scripts.push(p.code.clone());
|
||||
crate::python_part::file_formula(out.python_scripts.len().saturating_sub(1), p.object)
|
||||
})
|
||||
};
|
||||
let value: Value = match (ov, extra) {
|
||||
(Some(t), _) => Value::text(t.as_str()),
|
||||
(None, Some(v)) => v.clone(),
|
||||
@@ -733,8 +746,10 @@ fn write_cell(s: &mut String, input: CellWriteInput<'_>, sid: &dyn Fn(gridcraft_
|
||||
let value = match value {
|
||||
Value::Error(CellError::Circ) => Value::Number(0.0),
|
||||
// #SPILL! and #CALC! are not error codes of the file format (Excel refuses a file that
|
||||
// caches them); Excel itself caches them as #VALUE!.
|
||||
Value::Error(CellError::Spill | CellError::Calc) => Value::Error(CellError::Value),
|
||||
// caches them); Excel itself caches them as #VALUE!. The Python errors are the same kind.
|
||||
Value::Error(CellError::Spill | CellError::Calc | CellError::Python | CellError::Busy | CellError::Timeout | CellError::Blocked) => {
|
||||
Value::Error(CellError::Value)
|
||||
}
|
||||
v => v,
|
||||
};
|
||||
// A formula read with dynamic-array metadata keeps it, even where the shape analysis would
|
||||
@@ -742,11 +757,12 @@ fn write_cell(s: &mut String, input: CellWriteInput<'_>, sid: &dyn Fn(gridcraft_
|
||||
let own = formula.filter(|f| f.dynamic && f.array.is_none()).map(|_| RangeRef::cell(c));
|
||||
let dynamic = dynamic.or(own.as_ref());
|
||||
// A non-spilling formula may still need array evaluation to avoid implicit intersection.
|
||||
if dynamic.is_some() && formula.is_some_and(|f| f.array.is_none()) {
|
||||
if dynamic.is_some() && (formula.is_some_and(|f| f.array.is_none()) || py_text.is_some()) {
|
||||
s.push_str(" cm=\"1\"");
|
||||
out.dynamic_arrays = true;
|
||||
}
|
||||
if let Some(f) = formula {
|
||||
let text = formula.map(|f| esc(&crate::fmla::to_file(f))).or_else(|| py_text.map(|t| esc(&t)));
|
||||
if let Some(text) = text {
|
||||
let t = match &value {
|
||||
Value::Text(_) => " t=\"str\"",
|
||||
Value::Bool(_) => " t=\"b\"",
|
||||
@@ -755,8 +771,7 @@ fn write_cell(s: &mut String, input: CellWriteInput<'_>, sid: &dyn Fn(gridcraft_
|
||||
};
|
||||
s.push_str(t);
|
||||
s.push('>');
|
||||
let text = esc(&crate::fmla::to_file(f));
|
||||
match (f.array, dynamic) {
|
||||
match (formula.and_then(|f| f.array), dynamic) {
|
||||
(Some(r), _) => {
|
||||
let _ = write!(s, "<f t=\"array\" ref=\"{}\">{text}</f>", r.a1());
|
||||
}
|
||||
|
||||
@@ -25,7 +25,7 @@ fn round_trip(code: &str) -> (u32, String, String) {
|
||||
let mut wb = Workbook::new();
|
||||
let style = wb.styles.derive(StyleId::DEFAULT, |s| s.num_fmt = NumFmt::new(code));
|
||||
let mut sheet = Sheet::new("Sheet1");
|
||||
sheet.cells.set(CellRef::new(0, 0), Cell { value: Value::Number(-0.14), formula: None, style });
|
||||
sheet.cells.set(CellRef::new(0, 0), Cell { value: Value::Number(-0.14), formula: None, style, py: None });
|
||||
wb.sheets = vec![Arc::new(sheet)];
|
||||
let bytes = write_xlsx(&wb).unwrap();
|
||||
let xml = styles_xml(&bytes);
|
||||
|
||||
@@ -33,7 +33,7 @@ fn part(bytes: &[u8], name: &str) -> String {
|
||||
}
|
||||
|
||||
fn text_cell(sheet: &mut gridcraft_model::Sheet, a: &str, text: &str) {
|
||||
sheet.cells.set(at(a), Cell { value: Value::text(text), formula: None, style: StyleId::DEFAULT });
|
||||
sheet.cells.set(at(a), Cell { value: Value::text(text), formula: None, style: StyleId::DEFAULT, py: None });
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -52,7 +52,7 @@ fn workbook() -> Workbook {
|
||||
let r = r as u32 + 1;
|
||||
data.cells.set(CellRef::new(r, 0), Cell::value(Value::text(*a)));
|
||||
data.cells.set(CellRef::new(r, 1), Cell::value(Value::text(*b)));
|
||||
data.cells.set(CellRef::new(r, 2), Cell { value: Value::number(*d), formula: None, style: date_style });
|
||||
data.cells.set(CellRef::new(r, 2), Cell { value: Value::number(*d), formula: None, style: date_style, py: None });
|
||||
data.cells.set(CellRef::new(r, 3), Cell::value(Value::number(*u)));
|
||||
data.cells.set(CellRef::new(r, 4), Cell::value(Value::number(*s)));
|
||||
data.cells.set(CellRef::new(r, 5), Cell::value(Value::text(*ch)));
|
||||
|
||||
@@ -64,7 +64,7 @@ fn sample() -> Workbook {
|
||||
};
|
||||
|
||||
let mut s = Sheet::new("Data & Stuff");
|
||||
let set = |s: &mut Sheet, a: &str, v: Value, style: StyleId| s.cells.set(at(a), Cell { value: v, formula: None, style });
|
||||
let set = |s: &mut Sheet, a: &str, v: Value, style: StyleId| s.cells.set(at(a), Cell { value: v, formula: None, style, py: None });
|
||||
set(&mut s, "A1", Value::text("Region"), bold);
|
||||
set(&mut s, "B1", Value::text("Amount"), bold);
|
||||
set(&mut s, "C1", Value::text(" spaced\nline _x0041_ \u{1} "), StyleId::DEFAULT);
|
||||
@@ -859,7 +859,7 @@ fn writing_repairs_invalid_models() {
|
||||
table_type: None,
|
||||
connection_id: None,
|
||||
});
|
||||
s.cells.set(at("B2"), Cell { value: Value::Number(1.0), formula: None, style: StyleId(9999) });
|
||||
s.cells.set(at("B2"), Cell { value: Value::Number(1.0), formula: None, style: StyleId(9999), py: None });
|
||||
let (back, _) = read_xlsx(&write_xlsx(&wb).unwrap()).unwrap();
|
||||
let s = back.sheet(0).unwrap();
|
||||
assert_eq!(s.visibility, Visibility::Visible, "at least one sheet stays visible");
|
||||
|
||||
@@ -17,7 +17,7 @@ fn at(a: &str) -> CellRef {
|
||||
}
|
||||
|
||||
fn formula(text: &str, value: Value, style: StyleId) -> Cell {
|
||||
Cell { value, formula: Some(Arc::new(Formula::new(text))), style }
|
||||
Cell { value, formula: Some(Arc::new(Formula::new(text))), style, py: None }
|
||||
}
|
||||
|
||||
fn sheet_data(wb: &Workbook) -> String {
|
||||
@@ -37,7 +37,8 @@ fn sheet_data_is_written_byte_for_byte_as_pinned() {
|
||||
sh.rows.insert(1, LineInfo { hidden: true, ..LineInfo::default() });
|
||||
sh.rows.insert(4, LineInfo { size: Some(30.0), custom: true, ..LineInfo::default() });
|
||||
sh.rows.insert(7, LineInfo { outline: 2, style: Some(money), ..LineInfo::default() });
|
||||
let set = |sh: &mut gridcraft_model::Sheet, a: &str, value: Value, style: StyleId| sh.cells.set(at(a), Cell { value, formula: None, style });
|
||||
let set =
|
||||
|sh: &mut gridcraft_model::Sheet, a: &str, value: Value, style: StyleId| sh.cells.set(at(a), Cell { value, formula: None, style, py: None });
|
||||
set(sh, "A3", Value::text("a & <b> \"q\""), StyleId::DEFAULT);
|
||||
set(sh, "B3", Value::Number(-5.0), StyleId::DEFAULT);
|
||||
set(sh, "C3", Value::Number(0.1), money);
|
||||
|
||||
@@ -128,6 +128,8 @@ pub struct Out {
|
||||
pub cell_pictures: crate::richdata::PicturesOut,
|
||||
/// Kept (pass-through) parts written so far.
|
||||
pub kept: crate::passthrough::KeptOut,
|
||||
/// The code of the Python cells written so far, in order: script `i` of the Python part.
|
||||
pub python_scripts: Vec<String>,
|
||||
}
|
||||
|
||||
impl Out {
|
||||
@@ -259,6 +261,7 @@ pub fn write_xlsx_named(wb: &Workbook, kind: XlsxKind, book_name: Option<&str>)
|
||||
needs_calc: false,
|
||||
cell_pictures: Default::default(),
|
||||
kept: Default::default(),
|
||||
python_scripts: vec![],
|
||||
};
|
||||
out.kept.reserve_vml_blocks(wb);
|
||||
out.default_type("rels", "application/vnd.openxmlformats-package.relationships+xml");
|
||||
@@ -339,6 +342,12 @@ pub fn write_xlsx_named(wb: &Workbook, kind: XlsxKind, book_name: Option<&str>)
|
||||
pivots.finish(wb, &mut out, &|code: &str| crate::tables::builtin_id(code).or_else(|| fmt_ids.get(code).copied()));
|
||||
out.part("xl/styles.xml", Some("application/vnd.openxmlformats-officedocument.spreadsheetml.styles+xml"), styles.into_bytes());
|
||||
wb_rels.add("styles", "styles.xml");
|
||||
if !out.python_scripts.is_empty() {
|
||||
use crate::python_part as py;
|
||||
let part = py::write(&out.python_scripts);
|
||||
out.part(py::PART, Some(py::CONTENT_TYPE), part.into_bytes());
|
||||
wb_rels.add_uri(py::REL, "python.xml");
|
||||
}
|
||||
if wb.styles.iter().any(|(_, st)| st.num_fmt.as_str() == "checkbox") {
|
||||
use crate::feature_bags as fpb;
|
||||
out.part(fpb::PART, Some(fpb::CONTENT_TYPE), fpb::checkbox_part().into_bytes());
|
||||
|
||||
@@ -449,7 +449,7 @@ impl SheetState<'_> {
|
||||
return Ok(Some(pos));
|
||||
}
|
||||
budget.cell()?;
|
||||
self.sheet.cells.set(pos, Cell { value, formula: None, style });
|
||||
self.sheet.cells.set(pos, Cell { value, formula: None, style, py: None });
|
||||
Ok(Some(pos))
|
||||
}
|
||||
|
||||
@@ -467,7 +467,7 @@ impl SheetState<'_> {
|
||||
if let Some(cell) = self.sheet.cells.get_mut(pos) {
|
||||
cell.formula = Some(Arc::new(f));
|
||||
} else {
|
||||
self.sheet.cells.set(pos, Cell { value: Value::Empty, formula: Some(Arc::new(f)), style: StyleId::DEFAULT });
|
||||
self.sheet.cells.set(pos, Cell { value: Value::Empty, formula: Some(Arc::new(f)), style: StyleId::DEFAULT, py: None });
|
||||
}
|
||||
}
|
||||
Err(Unsupported(why)) => self.unsupported(why),
|
||||
@@ -526,7 +526,7 @@ fn blank_array_followers(st: &mut SheetState<'_>, pending: &[Pending]) {
|
||||
let filled = st.sheet.cell(p.anchor).and_then(|c| c.formula.as_ref()).and_then(|f| f.array).is_some_and(|r| r.contains(p.pos));
|
||||
if p.pos != p.anchor && filled {
|
||||
let style = st.sheet.style_id(p.pos);
|
||||
st.sheet.cells.set(p.pos, Cell { value: Value::Empty, formula: None, style });
|
||||
st.sheet.cells.set(p.pos, Cell { value: Value::Empty, formula: None, style, py: None });
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -672,7 +672,7 @@ fn sheet_record(rec: &Record<'_>, st: &mut SheetState<'_>, budget: &mut Budget,
|
||||
if let Some(cell) = st.sheet.cells.get_mut(pos) {
|
||||
cell.value = Value::from(s);
|
||||
} else {
|
||||
st.sheet.cells.set(pos, Cell { value: Value::from(s), formula: None, style });
|
||||
st.sheet.cells.set(pos, Cell { value: Value::from(s), formula: None, style, py: None });
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -827,7 +827,7 @@ fn formula_cell(c: &mut Cursor<'_, '_>, st: &mut SheetState<'_>, budget: &mut Bu
|
||||
let Some(pos) = st.pos(r, col) else { return Ok(()) };
|
||||
budget.cell()?;
|
||||
let style = st.style(x);
|
||||
st.sheet.cells.set(pos, Cell { value, formula: None, style });
|
||||
st.sheet.cells.set(pos, Cell { value, formula: None, style, py: None });
|
||||
st.last_formula = Some(pos);
|
||||
if string_result {
|
||||
st.pending_string = Some((pos, style));
|
||||
|
||||
+31
-1
@@ -1,6 +1,6 @@
|
||||
# Architecture
|
||||
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-01 · **Change:** minor (equations: the `math` crate and the layout cache; earlier: recently used functions kept in the preferences; Function Library module; accent icons; earlier: the hidden names that hold macros) · **Target:** Microsoft Excel (Microsoft 365)
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-11-02 · **Change:** minor (Python cells as data; earlier: equations: the `math` crate and the layout cache; earlier: recently used functions kept in the preferences; Function Library module; accent icons; earlier: the hidden names that hold macros) · **Target:** Microsoft Excel (Microsoft 365)
|
||||
|
||||
How GridCraft is built today. Rules for contributors are in [`AGENTS.md`](../AGENTS.md).
|
||||
|
||||
@@ -152,6 +152,35 @@ The macro engine (`engine/src/macros/`, commands in `cmd/macros.rs`, described i
|
||||
recording buffer, the policy asks before a macro in a `FromFile` workbook runs, and xlsx keeps
|
||||
macros as chunked hidden names (`_gc_macro_<n>_<k>`: macro `n`, chunk `k`).
|
||||
|
||||
## Python cells
|
||||
|
||||
A Python cell is a `Cell` whose `py` holds a `model::PyCode`: the code, the return type (an Excel
|
||||
value or a Python object, the `0` or `1` of Excel's `PY(code, return_type)`), the last result and a
|
||||
stale flag. Its `value` is the cached result, so formulas, display and saving read it like any
|
||||
other cell. `model::python` has the text forms: `PY("…"&"…",n)` with literals of at most 255
|
||||
characters (the function's syntax, which some files hold), the `_xlfn._xlws.` prefix, the
|
||||
8,192-character limit measured on that formula (`PyCode::fits`), and the `xl("…")` literals in
|
||||
code, which structural edits rewrite like formula references.
|
||||
|
||||
Only those edits rewrite them: inserting or deleting cells, rows and columns, a cut and paste, and
|
||||
renaming or deleting a sheet. An edit that would make a cell's code longer than the limit (a
|
||||
longer sheet name, `Z1` becoming `AA1`, a cut to another sheet that qualifies the references) is
|
||||
refused and changes nothing, since code over the limit would be dropped to its value on the next
|
||||
open. Copying, filling and sorting carry the code unchanged, so an `xl()`
|
||||
reference acts as an absolute reference. A sort that moves a Python cell to another row leaves its
|
||||
code reading the cells it named before (a formula moved by the same sort has its relative references
|
||||
shifted), and references to the sorted cells from other cells, formulas and Python code alike, keep
|
||||
naming the same positions.
|
||||
|
||||
Calc treats a Python cell as a graph node that reads nothing: a pass takes its kept result instead
|
||||
of evaluating it, so an array result spills, is blocked and unblocked, and reaches its dependents
|
||||
through the same code as a formula's. The xlsx reader and writer map Python cells to and from
|
||||
`_xlfn._xlws.PY(index,n)` formulas with cached values and the workbook's Python part, which holds
|
||||
the code (`xlsx::python_part`, [MS-XLSX] 2.3.10); `io::adopt_python` and `io::enforce_python_limits`
|
||||
bound what a file can make the app hold (result size, spill ranges, total rebuilt cells, code
|
||||
size). This build runs no Python: `python.set` stores code, and new or changed code shows
|
||||
`#BLOCKED!`.
|
||||
|
||||
## Equations
|
||||
|
||||
An equation is a shape (`ShapeKind::Equation`) whose `text` is its LaTeX source; it selects,
|
||||
@@ -191,6 +220,7 @@ rpm, tarball), FreeBSD, and web (WASM; WebGPU or WebGL2). Release workflows in
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-11-02 | minor | Python cells as data: `model::PyCode`, the file forms, calc laying out kept results, the limits on open, and what copying, filling and sorting do to `xl()` references; edits that would push code past the limit are refused |
|
||||
| 2026-11-01 | trivial | Equations: the layout cache answers for a source over the cap by its length alone, without hashing it |
|
||||
| 2026-11-01 | minor | Equations: the `math` crate (L1), the layout cache and code editor marks |
|
||||
| 2026-10-31 | minor | Recently used functions (`recent.rs`, `Prefs::recent_functions`) behind `formulas.recentlyUsed` |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# File-format parity
|
||||
|
||||
> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (equations in xlsx as Office Math with a marker in `descr` so an edit made in Excel reads back, read back only up to the equation source cap, and a brace with a label and a script on its other side keeps both; Symbol font in PDF; earlier: XLSX reads and writes Page Setup's page order, `pageSetup pageOrder`, and PDF export follows it; earlier: chart data tables: native c:dTable import/export; earlier: XLSX workbook-view selection, normalized tab ratios and sheet-edit regressions; fidelity estimates unchanged; earlier: formulas from Google Sheets' XLSX export open as the original formula instead of `IFERROR(__xludf.DUMMYFUNCTION(…),…)` when GridCraft calculates every function in it, and save as it; `file.exportCsv {values: true}` writes raw values for automation; .xls dialog sheets are skipped instead of read as worksheets; `.xls` import reads each sheet's display options from WINDOW2: right-to-left, gridlines, headings, zeros, formulas; XLSX text boxes and shapes keep their paragraphs' right-to-left direction on save; CSV/TSV/TXT can open with every field kept as text via `file.open {csvAsText}`; formulas over Excel's 8,192 characters, and `%` or call chains past the nesting limit, are parse errors instead of stack overflows when a file is opened; XLSX PivotTables read and write `rowGrandTotals`/`colGrandTotals` the way Excel does, so a table saved with only one grand total opens with the right one; HTML export keeps right-to-left; CSV and text import detects likely Windows-1256 Arabic when invalid UTF-8 has Arabic byte runs; other invalid UTF-8 remains Windows-1252; HTML export places pictures and charts, as inline SVG, at their anchors; earlier: `file.exportCsv {values: true}` writes raw values for automation; .xls dialog sheets are skipped instead of read as worksheets; `.xls` import reads each sheet's display options from WINDOW2: right-to-left, gridlines, headings, zeros, formulas; XLSX text boxes and shapes keep their paragraphs' right-to-left direction on save; CSV/TSV/TXT can open with every field kept as text via `file.open {csvAsText}`; formulas over Excel's 8,192 characters, and `%` or call chains past the nesting limit, are parse errors instead of stack overflows when a file is opened; XLSX PivotTables read and write `rowGrandTotals`/`colGrandTotals` the way Excel does, so a table saved with only one grand total opens with the right one; HTML export keeps right-to-left; CSV and text import detects likely Windows-1256 Arabic when invalid UTF-8 has Arabic byte runs; other invalid UTF-8 remains Windows-1252; HTML export places pictures and charts, as inline SVG, at their anchors; CSV and text open detects each column's date order from values that read only one way, warns when every date is ambiguous, and keeps a column with both kinds as text (#2765)) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4
|
||||
> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (Python cells in `.xlsx` as [MS-XLSX] specifies, `_xlfn._xlws.PY(…)` naming code in the Python part, with cached values, and in GridCraft JSON, with limits on open; earlier: equations in xlsx as Office Math with a marker in `descr` so an edit made in Excel reads back, read back only up to the equation source cap, and a brace with a label and a script on its other side keeps both; Symbol font in PDF; earlier: XLSX reads and writes Page Setup's page order, `pageSetup pageOrder`, and PDF export follows it; earlier: chart data tables: native c:dTable import/export; earlier: XLSX workbook-view selection, normalized tab ratios and sheet-edit regressions; fidelity estimates unchanged; earlier: formulas from Google Sheets' XLSX export open as the original formula instead of `IFERROR(__xludf.DUMMYFUNCTION(…),…)` when GridCraft calculates every function in it, and save as it; `file.exportCsv {values: true}` writes raw values for automation; .xls dialog sheets are skipped instead of read as worksheets; `.xls` import reads each sheet's display options from WINDOW2: right-to-left, gridlines, headings, zeros, formulas; XLSX text boxes and shapes keep their paragraphs' right-to-left direction on save; CSV/TSV/TXT can open with every field kept as text via `file.open {csvAsText}`; formulas over Excel's 8,192 characters, and `%` or call chains past the nesting limit, are parse errors instead of stack overflows when a file is opened; XLSX PivotTables read and write `rowGrandTotals`/`colGrandTotals` the way Excel does, so a table saved with only one grand total opens with the right one; HTML export keeps right-to-left; CSV and text import detects likely Windows-1256 Arabic when invalid UTF-8 has Arabic byte runs; other invalid UTF-8 remains Windows-1252; HTML export places pictures and charts, as inline SVG, at their anchors; earlier: `file.exportCsv {values: true}` writes raw values for automation; .xls dialog sheets are skipped instead of read as worksheets; `.xls` import reads each sheet's display options from WINDOW2: right-to-left, gridlines, headings, zeros, formulas; XLSX text boxes and shapes keep their paragraphs' right-to-left direction on save; CSV/TSV/TXT can open with every field kept as text via `file.open {csvAsText}`; formulas over Excel's 8,192 characters, and `%` or call chains past the nesting limit, are parse errors instead of stack overflows when a file is opened; XLSX PivotTables read and write `rowGrandTotals`/`colGrandTotals` the way Excel does, so a table saved with only one grand total opens with the right one; HTML export keeps right-to-left; CSV and text import detects likely Windows-1256 Arabic when invalid UTF-8 has Arabic byte runs; other invalid UTF-8 remains Windows-1252; HTML export places pictures and charts, as inline SVG, at their anchors; CSV and text open detects each column's date order from values that read only one way, warns when every date is ambiguous, and keeps a column with both kinds as text (#2765)) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4
|
||||
|
||||
Every format Excel opens or saves, with GridCraft's read and write support, fidelity and tests.
|
||||
Part of [target-app-parity.md](target-app-parity.md). **File formats: ~50% ready, 80–120 h**
|
||||
@@ -128,6 +128,16 @@ Server, Access, ODBC, …), web pages, SharePoint and online services, through P
|
||||
GridCraft imports CSV/TSV, JSON and HTML tables only, with no query editor, no connections and no
|
||||
refresh. Counted under features (Get & Transform, ~10% ready), not here.
|
||||
|
||||
## Python cells
|
||||
|
||||
| Where | GridCraft | Notes |
|
||||
|---|---|---|
|
||||
| `.xlsx` | read and write | Written as [MS-XLSX] 2.3.10 specifies: the cell's formula is `_xlfn._xlws.PY(i,n)`, where `i` is the index of the cell's script in the workbook's Python part (`xl/python.xml`, `application/vnd.ms-excel.python+xml`) and `n` the return type, 0 for an Excel value or 1 for a Python object, with the cached value. The code keeps its `xl()` references as text, so the formula has no further arguments. A spilling result is written as a dynamic array with its spilled values cached. Read back with or without the `_xlfn.` and `_xlws.` prefixes: an index names a script of the Python part, whose `%Px%` references become the text of the formula's argument `x`, and the older form with the code in string literals, `PY("…"&"…",n)`, still reads. So workbooks laid out as the format specifies open with Python cells, their cached values and spills. Python errors are cached as `#VALUE!`, as Excel caches `#SPILL!` and `#CALC!` |
|
||||
| `.xls`, `.xlsb`, `.ods` | read | A `PY("…",0)` or `PY("…",1)` formula read from any format becomes a Python cell; the `.xlsb` and `.ods` imports keep values only, so Python cells there come in as their cached values |
|
||||
| GridCraft JSON | read and write | The whole Python cell: code, return type, the last result and the exact error |
|
||||
| Limits on open | — | Code that doesn't fit Excel's 8,192-character formula as `PY("…",n)` keeps only its cached value, with a warning, and so does a formula naming a script the Python part doesn't hold. The Python part is read up to 64 MB and 1,000,000 scripts, and the cells take at most 64 MB of code from it in all (several formulas can name one script). A kept result that is not a well-formed array of at most 1,000,000 cells is dropped, a spill range larger than that is ignored, and results are rebuilt for at most 4,000,000 cells over the workbook. A Python cell that keeps only its value (code too long or not in the part, a damaged part, a dropped result, a result past that budget) keeps the values it spilled as plain values, without overwriting a cell's own content or format. Opening runs nothing |
|
||||
| Not kept | — | Python object results stored as rich values (Excel's `richData` parts) come in as their cached values. The Python part's environment definition (its id and initialization code) is not read, and a save writes none |
|
||||
|
||||
## Work (76–114 h)
|
||||
|
||||
| Work | Hours |
|
||||
@@ -145,6 +155,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here.
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-11-02 | minor | Python cells: `.xlsx` reads and writes `_xlfn._xlws.PY(i,0 or 1)` with the code in the Python part (and reads the string-literal form) with cached values and spills, other formats adopt PY formulas, GridCraft JSON keeps the whole cell, and opening limits code size, result size and spill ranges |
|
||||
| 2026-11-02 | trivial | An equation without Office Math takes its stored source only while its text, paragraphs included, is what the writer wrote for it; a paragraph split that changes no character reads back as a text box with the split text |
|
||||
| 2026-11-02 | trivial | Each equation box carries the marker of its own body in its description, the Office Math box and the plain-text fallback alike, and the reader takes the stored source only while the body it reads matches the marker: the text of an equation without Office Math that another application edited reads back as a text box with the edited text instead of the stale source |
|
||||
| 2026-11-02 | trivial | A brace with a label and a script on its other side (`\underbrace{x}_{n}^{2}`) writes the label as the brace's limit and the other script around the whole (`m:sSup` or `m:sSub`), so an application reading the Office Math sees both; without a label the script goes around the bare brace, with no empty limit. A limit over a brace alone reads back as the brace with the label as its script (`\underbrace{x}_{n}`), so both scripts return from Office Math |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Function parity: worksheet functions and formulas
|
||||
|
||||
> **Last reviewed:** 2026-10-31 · **Last updated:** 2026-10-31 · **Change:** minor (YEARFRAC basis 0 February-end rule, MOD quotient limit, PERCENTRANK significance; earlier: edge cases matched to Excel: base conversions, BITLSHIFT/BITRSHIFT, UNICHAR noncharacters, UPPER/LOWER/PROPER casing, FIXED/DOLLAR output, F/T/GAMMA/WEIBULL/BETA distributions, BINOM trial cap, HARMEAN; earlier: ECMA.CEILING listed with USDOLLAR as not on Microsoft's list; earlier: IMSQRT reproduces the stray real part Excel gives for negative reals) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4
|
||||
> **Last reviewed:** 2026-10-31 · **Last updated:** 2026-11-01 · **Change:** minor (the Python errors `#PYTHON!`, `#BUSY!`, `#TIMEOUT!` and `#BLOCKED!`, and a Python in cells row; earlier: YEARFRAC basis 0 February-end rule, MOD quotient limit, PERCENTRANK significance; earlier: edge cases matched to Excel: base conversions, BITLSHIFT/BITRSHIFT, UNICHAR noncharacters, UPPER/LOWER/PROPER casing, FIXED/DOLLAR output, F/T/GAMMA/WEIBULL/BETA distributions, BINOM trial cap, HARMEAN; earlier: ECMA.CEILING listed with USDOLLAR as not on Microsoft's list; earlier: IMSQRT reproduces the stray real part Excel gives for negative reals) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4
|
||||
|
||||
Excel's worksheet functions, category by category, against GridCraft's function library. Part of
|
||||
[target-app-parity.md](target-app-parity.md); gaps feed [gaps.md](gaps.md).
|
||||
@@ -97,7 +97,8 @@ This improves series preparation; it does not change the model-fitting or season
|
||||
| External workbook references (`[Book.xlsx]Sheet1!A1`) | parsed; links dropped on open, no update |
|
||||
| Localized function names and `;` separators | **missing** (#45); en-US only |
|
||||
| Long flat operator chains (`=A1+A2+…`, `-`, `*`, `/`, `^`, `&`, comparisons) | done: no operand limit below Excel's 8,192-character formula limit (Excel, observed: `=1+1+…` with 4,096 operands is exactly 8,192 characters with the `=` and calculates; one more character is refused). Evaluated (#484), printed (#628), stored, copied, adjusted, saved and reopened without recursing once per operand, on a 1 MiB stack (#916); written to XLSX as plain formulas, not dynamic arrays; Spanish entry accepts them |
|
||||
| Error values | the seven classic errors plus `#SPILL!`, `#CALC!`, `#GETTING_DATA` done; `#FIELD!`, `#BLOCKED!`, `#CONNECT!`, `#BUSY!`, `#UNKNOWN!`, `#PYTHON!` missing (minor: they come from data types, cloud and Python features) |
|
||||
| Error values | the seven classic errors plus `#SPILL!`, `#CALC!`, `#GETTING_DATA`, and the Python errors `#PYTHON!`, `#BUSY!`, `#TIMEOUT!`, `#BLOCKED!` done; `#FIELD!`, `#CONNECT!`, `#UNKNOWN!` missing (minor: they come from data types and cloud features). `ERROR.TYPE` gives 11 for `#BLOCKED!` as Excel does; Microsoft documents no number for `#PYTHON!`, `#BUSY!` or `#TIMEOUT!`, and GridCraft gives them 19, 20 and 21 |
|
||||
| Python in cells | partial: Python cells are stored, edited (`python.set`), copied, moved and saved, and Excel's `_xlfn._xlws.PY` cells open with their cached results; the code doesn't run, so new or changed code shows `#BLOCKED!`. Typing `=PY(` is refused with a pointer to `python.set`, so `PY` stays counted as missing above |
|
||||
|
||||
## Depth (estimated)
|
||||
|
||||
@@ -133,6 +134,7 @@ workbooks; results checked in as data, not Excel files) and fixing what it finds
|
||||
|
||||
| Date | Change | Summary |
|
||||
|---|---|---|
|
||||
| 2026-11-01 | minor | Python in cells (stored, not run): `#PYTHON!`, `#BUSY!`, `#TIMEOUT!` and `#BLOCKED!` done, with their `ERROR.TYPE` numbers; `PY` still counted as missing |
|
||||
| 2026-10-31 | minor | BIN2*, HEX2* and OCT2* reject surrounding spaces and tabs in source strings with `#NUM!`, matching Excel; plain unpadded values remain valid |
|
||||
| 2026-10-31 | minor | DEC2BIN, DEC2HEX and DEC2OCT validate `places` before returning negative two's-complement output, and Boolean places return `#VALUE!`, matching Excel |
|
||||
| 2026-10-31 | minor | BITLSHIFT and BITRSHIFT return `0` for zero operands shifted beyond 53 bits in either direction; nonzero operands retain the `#NUM!` limit, matching Excel |
|
||||
|
||||
Reference in New Issue
Block a user