aleleba/gridcraft
1
0
Fork 0
mirror of https://github.com/storytold/gridcraft.git synced 2026-11-01 22:56:54 -06:00
Code Issues Packages Projects Releases Wiki Activity
Files
71b130be2dbe154feb6da0e7fbe3518e100eea7e
gridcraft/docs/gaps.md
T
PranavandFredApps 71b130be2d Data: Solver Load/Save Model and Reset All (#3086) 
* Data: Solver Load/Save Model and Reset All

data.solverSaveModel writes a model (the sheet's saved model, or one
given as data.solver params) down a column of its sheet as formulas:
the objective and goal (=MAX(...), =MIN(...) or =cell=value), the
variable cells (=COUNT(...)), one =AND(relation) per constraint, which
shows TRUE while the constraint holds, and the method and its options
as an array constant. It refuses a range that overlaps the model's
cells or runs off the sheet.

data.solverLoadModel reads such a range (or reads down from its first
cell), checks the model and makes it the active sheet's model.
data.solverReset (Reset All) removes the active sheet's model. Each is
one undo step.

The Solver Parameters dialog has Reset All and Load/Save buttons; the
dialog can also be opened with fields that go over the saved model.

* Data: Solver Reset All also clears a model read from workbook-level names

A sheet's Solver model can come from workbook-level names, which every
sheet without a model of its own reads. Reset All removed only the
sheet-level names, so such a model stayed and the dialog showed it
again.

Removing the workbook-level names would change every other sheet that
reads them. Reset All instead leaves them in place and writes an empty
sheet-level solver_opt over them on the active sheet only; the result
says masked: true. saved_model now reads a sheet's own names when the
sheet has a solver_opt, and the workbook-level names only otherwise,
never a mix of both, and treats an empty solver_opt as no model. The
mask survives XLSX and is replaced when a model is saved on the sheet.

* Data: Solver Save Model clears the rest of a longer model saved at the same cell

Saving a model over a longer one saved at the same cell left the old
model's last rows on the sheet, so Load Model from that cell read the new
options row followed by stale constraint rows and refused the range.
Save Model now clears what is left of a model saved there before (only
when those cells hold a complete model written by Save Model, and never
the new model's own cells), keeping the cells' formatting, in the same
undo step. Load Model reading down from one cell also stops at the
options row, so formulas just below a saved model no longer break it.

* Data: Solver Load Model keeps the Evolutionary convergence and generations

Save Model writes Convergence and the generations without improvement in
its options row and Load Model reads them, but the sheet's saved model
(hidden names) did not hold them, so after Load Model, or any solve,
they fell back to their defaults. The saved model now keeps them for the
Evolutionary method: Convergence as solver_cvg, which means the same in
Excel, and the generations as gridcraft_solver_stall, because Excel's
solver_mni is a time in seconds. Saving a model again and Reset All
remove the GridCraft name with the solver_* names.

* Data: Solver Load Model refuses a range with a gap

Load Model from a range of several cells kept only the cells that hold a
formula, so the rows below a cell without one moved up. A constraint cell
overwritten by a constant gave a model without that constraint, and an
empty cell before the options row gave the default options, both without
an error. Load Model now refuses a range in which a cell before the last
formula holds no formula, and names that cell. Empty cells after the
model are still allowed.

* Data: Solver removes only its own names when it saves or resets a model

Saving a model (by a solve or Load Model) and Reset All removed every
name of the sheet that starts with solver_, so a user's own name such as
solver_budget was deleted with the model. They now remove only the
Solver's names: the ones Excel's Solver writes (solver_opt, solver_adj,
solver_lhs1, solver_rel1, solver_rhs1 and so on) and GridCraft's own.
The names are matched by name rather than by the hidden flag, so a model
whose names lost that flag is still replaced in full.

* Data: Solver Save Model refuses a place where the options cannot spill

The options row that Save Model writes is an array constant of eight
values, which spills into the seven cells to the right of its first
cell. Content in any of those cells blocked the spill and the options
cell showed #SPILL!. Save Model now checks the whole spill range before
it writes anything and refuses, naming the cell, when a cell there holds
content or another formula's spilled values or is a cell of the model,
when merged cells or a table meet the row (a dynamic array cannot spill
into a table), or when the range runs past the last column. It also
refuses a place where the model's column would cover another formula's
spilled values, which that formula would lose to #SPILL!. The options of
a model saved at the same cell before may be replaced, as before.

* Data: Solver Reset All and Load Model drop the old model's options

Reset All cleared the model fields of the Solver Parameters form but
kept the Evolutionary options (Convergence, Mutation Rate, Population
Size, Random Seed, generations without improvement and Require Bounds),
and Load Model kept any of them that the loaded model omits, such as all
of them for a GRG Nonlinear or Simplex LP model. A new Evolutionary model
then solved with the old options, for example with Require Bounds off.
Both now remove the model's options from the form first (one list of
their keys), so an option the model doesn't set takes its default.

* Data: Solver Save Model asks before it replaces data

Save Model wrote its formulas over whatever the model range held.
It is now refused, with the sheet unchanged, when a cell of the
range holds a value or a formula (a model saved there before
included), unless `overwrite: true` is passed. The message names
the first such cell and carries the same "already holds data"
phrase as Data Analysis, so the Solver Parameters dialog asks
"Replace it with the model?" with Replace and Don't Replace. The
question belongs to the range it asked about: editing the range,
Load or Reset All drops it. The check runs after the spill checks,
so the question is asked only when replacing would succeed. A save
remains one undo step.

Tests: the engine test refuses a value, a formula and an earlier
model, accepts only a true boolean, and undoes and redoes the
replacing save in one step; the dialog test covers Don't Replace,
an edited range and Replace. Both fail without the change.

* Data: Solver Load Model and Reset All tests for limits and undo

Tests for behaviour already in place: reading down from one cell of
a column of 5,000 formulas stops after the most rows a model takes,
a range larger than a model is refused before any cell is read, and
malformed or hostile cells and ranges (unbalanced brackets, an
overflowing number, an unknown sheet, 10,000 nested brackets, a
non-text `from`) each give an error and leave no model. Undoing
Reset All brings the model back with its options (the population
size).

* Data: Solver Load Model refuses a model without its options row

Load Model from one cell read down to the first cell without a
formula, and a range dropped its trailing cells without formulas.
A constraint cleared in the middle of a saved model, or its options
row cleared or replaced by a constant, therefore left a prefix of
the model, which was loaded as the whole model with default options
and replaced the sheet's model. Every load now requires the model's
last row to be the options row ({...}) that Save Model writes last;
otherwise it is refused, naming the cell without a formula when
reading down from one cell, and the sheet's model stays as it was.

The test clears a middle constraint, clears the options row and
replaces it by a constant, loads from the first cell and as two
ranges, and checks after each refusal that the old model (three
constraints, Simplex LP) is unchanged. It fails without the change.

* Data: Solver Load Model refuses an options row cut short

The options row of a saved model was read leniently: a missing value
or one that isn't a number took its default, and values after the
sixteenth were not read at all. A row cut short or edited by hand,
such as ={1}, therefore loaded the model with default options in
place of the saved ones (here: GRG Nonlinear in place of Simplex
LP). The row must now hold at least the eight numbers Save Model
writes (method, non-negative, population size, mutation rate, random
seed, require bounds, convergence, generations without improvement),
and every value in it, however many, must be a number; otherwise
Load Model is refused and the sheet's model stays as it was. Only
the first sixteen values are kept. More numbers than eight are still
read, so a row with options a later version adds loads too.

The test loads rows of one value, seven values, a text value among
the eight, after them and after the sixteenth, and a TRUE, from the
first cell and as a range, checking the old model after each
refusal, and a row of ten numbers, which loads. It fails without the
change.

---------

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-01 20:13:00 +01:00

222 lines
16 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Gaps
> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-31 · **Change:** minor (§13: Solver Load/Save Model and Reset All; earlier: §8: PivotTable Enable show details) · **Target:** Microsoft Excel (Microsoft 365)
Every known shortfall against Excel, one entry each, ranked by user impact (what blocks someone
using GridCraft for real work first). This is the work list: pick from the top, check the open
PRs first (50 open on 2026-10-10; several already address entries here), and update the entry
when you land work. Estimates are Opus 5.5 agent-hours. Area detail lives in the linked parity
docs; the summary is in [target-app-parity.md](target-app-parity.md).
Kinds: **F** feature · **U** UI/UX · **FF** file format · **H** hardware · **L** localization ·
**P** performance · **S** stability.
## Beta blockers
No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha-gate).
### 1. XLSX output not proven in Excel or on real files (FF)
- **Missing:** a run over a real-world XLSX corpus, and confirmation that Excel opens what we save
with no repair prompt (PivotTables, charts, CF, validation, dynamic arrays, tables).
- **Evidence:** round-trip tests cover our own files only (`crates/xlsx/src/tests/`); PivotTable
round-trip noted as "unverified in real Excel" since 10-07; #64 (dynamic-array flag) shows the
class of bug.
- **Impact:** the main format; a repair prompt or silent loss destroys trust.
- **Estimate:** 15–25 h, plus owner time in Excel. [file-format-parity.md](file-format-parity.md)
### 2. Large workbooks don't open; recalc is single-threaded (P, H)
- **Missing:** streaming read for ~100 MB XLSX; multi-threaded recalc; 100k-dependent edit under
100 ms (was ~0.4 s); whole-sheet operations bounded.
- **Evidence:** #175; no threading in `crates/calc`; PR #152 (whole-sheet Fill/Remove Duplicates
out of memory), PR #151 (current region past 10,000 rows).
- **Impact:** finance and data users routinely have big workbooks.
- **Estimate:** 25–40 h. [target-app-parity.md](target-app-parity.md#performance), [hardware-parity.md](hardware-parity.md)
### 3. Some unmodelled XLSX content is dropped on save (FF)
- **Missing:** slicers and their caches, timelines, the data model, query tables attached to
tables, unmodelled `extLst` entries, chart details. Pass-through (#330) keeps custom XML,
external links, connections, sheet query tables, printer settings, form and ActiveX controls,
OLE objects and sheet backgrounds; never checked in Excel (repair prompts unknown).
- **Evidence:** `crates/xlsx/src/passthrough.rs` (what is kept, and why the rest isn't).
- **Impact:** open-edit-save still simplifies workbooks with slicers, timelines or Power Pivot.
- **Estimate:** 6–10 h. [file-format-parity.md](file-format-parity.md)
### 4. Charts lose options on round trip; shallow chart model (F, FF)
- **Missing:** axis scaling, label options, error bars, per-point formatting.
- **Evidence:** `Chart` holds only title, legend position, two flags and axis titles
(`crates/model/src/features.rs`); PRs #158, #161 open; charts can't be resized (#190). Smooth lines (#62), Switch Row/Column (#116) and trendlines (read, written, drawn) landed. Multi-level category axes (year beside month) are read, drawn and written back.
(`crates/model/src/features.rs`); all 2-D kinds survive XLSX round trip (#158, #161); charts
can't be resized (#190). Smooth lines (#62), Switch Row/Column (#116) and trendlines (read,
written, drawn) landed.
- **Impact:** charts are in most business workbooks.
- **Estimate:** 25–35 h for beta (40–60 h full). [chart-parity.md](chart-parity.md)
### 5. Text in non-Latin scripts doesn't render or export (L)
- **Missing:** CJK font fallback, complex shaping (Arabic, Devanagari, Thai), bidi, non-Latin PDF.
- **Evidence:** system-font fallback for CJK, Thai and Hebrew landed in #38 (closed #32); #142
(Chinese text) awaits confirmation; #54 (symbols as boxes on Fedora); no shaping for Arabic or
Devanagari; PDF writer uses 14 standard fonts only.
- **Impact:** blocks most of the world's users even with an English UI.
- **Estimate:** 12–20 h. [localization-parity.md](localization-parity.md)
### 6. Calculation correctness bugs (F)
- **Missing:** nothing known. The four this entry tracked have landed: 15-significant-digit
equality (#57, so `=0.1+0.2=0.3` is TRUE and MATCH and COUNTIF agree), spill updates after
structural edits and cleared blockers (#133, #139; PRs #148, #157), 3-D span deletion (#140),
and qualified names after a sheet rename (#134). Iterative calculation for circular references
landed too, with its settings in File › Options › Formulas.
- **Impact:** wrong numbers are worse than missing features, so this entry is kept until the
oracle suite of §15 can say the category is clear rather than only these five reports being so.
- **Estimate:** — [function-parity.md](function-parity.md#formula-language)
### 7. Legacy `.xls` only partly read; encrypted workbooks can't be saved with a password (FF)
- **Evidence:** #121: Excel 97-2003 (BIFF8) `.xls` files now open (values, formulas, number
formats, merges, sizes, names, and cell formatting: fonts, fills, borders, alignment); exact theme colours (XFExt), charts, pictures, comments, conditional
formats, validation and BIFF5 files are not read, and `.xls` is never written. Password-encrypted
`.xlsx` workbooks (Agile and Standard encryption) open with their password since #11, but are
saved without it: GridCraft can't write an encrypted file yet. Encrypted `.xls` (FILEPASS) and
RC4-encrypted files are still refused with a message.
- **Impact:** common in finance, government and anything older than 2007; encrypted files are
routine in business.
- **Estimate:** 14–21 h (`.xls` objects 10–15, encryption 4–6). [file-format-parity.md](file-format-parity.md)
### 8. PivotTables: no slicers, calculated items, PivotCharts; unverified in Excel (F)
- **Evidence:** `parity-checklist.md` missing `insert.slicer`, `insert.timeline`,
`insert.pivotChart`, `table.insertSlicer`; calculated items and GETPIVOTDATA missing (calculated fields
and values on rows done). Show Details (double-click a value, subtotal or grand total cell) lists
its source records on a new sheet. The "Enable show details" option (`pivot.options`
`enableShowDetails`, saved as `enableDrill`) turns it off; it has no PivotTable Options dialog
yet (command only).
- **Impact:** pivots are the analyst's main tool.
- **Estimate:** 20–30 h for beta (25–40 h full).
### 9. Keyboard-first users: key tips, shortcuts, context menus (U)
- Next/previous sheet navigation now skips Hidden and VeryHidden worksheets without activating a
hidden sheet at the workbook boundary. Other keyboard gaps below remain.
- **Evidence:** #33, #49 (key tips: initial Home-tab support landed in #43 on 10-10, the rest of the ribbon not yet), #110, #28 (selection),
59 command shortcuts vs Excel's 200+.
- **Estimate:** 12–20 h. [ui-parity.md](ui-parity.md)
### 10. Crashes and freezes reported from the field (S)
- **Evidence:** #120 (Intel UHD crash on Windows), #117 (error on close, macOS), #166 (freeze opening
from Finder; PRs #164, #167), #76 (flashing on Windows 11), #130 (caught panic in table styles).
- **Estimate:** 8–12 h plus an autosave/recovery soak.
## After beta
### 11. No localization at all (L)
- Catalog: ribbon-only in five languages (#31, #104, #82; ~12% of strings each), and 943 strings
(~43%) in Japanese (10-17) and Brazilian Portuguese, including the dialog labels, messages, File
view and menus. The remaining dialogs, menus and messages are English; en-US function names and
separators (#45); German in PR #163. **Estimate:**
100–165 h including #5. [localization-parity.md](localization-parity.md)
### 12. Get & Transform, external data, data model (F)
- No Power Query editor, connections, refresh, From Web/database/folder sources, relationships or
data model (`data.fromWeb`, `data.relationships` missing). **Estimate:** 60–100 h.
### 13. Analysis tools: Solver, Analysis ToolPak, Forecast Sheet (F)
- `data.solver`, `data.analysisToolPak` missing. Forecast Sheet (`data.forecastSheet`) and the
FORECAST.ETS family are in, including Forecast Start (hindcast, #785). **Estimate:** 15–25 h.
- Solver (`data.solver`, #327) has Simplex LP with integer/binary branch and bound, a local
nonlinear method (augmented Lagrangian, not GRG), the Evolutionary method, and Answer and
Limits reports. Report refusal leaves the solution, saved model and history unchanged; a
successful solve and its reports share one Undo step (#1050). Load/Save Model writes the
model on cells of its sheet and reads it back; Reset All removes it. AllDifferent constraints,
the Solver Options dialog and the Sensitivity and Population reports remain.
The Analysis ToolPak
(`data.analysisToolPak`) has 13 of 19 tools. Forecast Sheet (`data.forecastSheet`) and the
FORECAST.ETS family are in; the sheet has no Forecast Start (hindcast) option yet. **Estimate:** 12–20 h.
### 14. Page Layout view and print polish (U, F)
- Page Layout view landed (#329): paper view with rulers and in-place header/footer editing.
Margins are dragged on the rulers (#792). Remaining: pictures in headers/footers, Different First Page / Odd & Even pages. Desktop now submits directly through craft-print with printer choice, copies, duplex and grayscale; browser printing stays available. Print ranges and driver tray/options UI remain. **Estimate:** 5–10 h.
### 15. Remaining functions and an oracle suite (F)
- 18 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 16–26 h.
- 20 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 17–29 h.
- 21 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 18–30 h.
- 18 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 18–30 h.
[function-parity.md](function-parity.md)
### 16. Collaboration and review (F)
- No co-authoring, Show Changes, comment navigation, notes ↔ comments
conversion; cloud storage requested (#86). **Estimate:** 35–55 h (co-authoring scope needs the owner).
### 17. Insert objects (F)
- Ink strokes now survive XLSX save/reopen as editable polylines. Inserted icons still lose their
identity/artwork on save (#551); native Office InkML import is not implemented.
- SmartArt, WordArt, equations, signature line, objects, 3-D models, screenshot; 8 shape kinds vs
~170; pictures in cells (#61, PR #105). **Estimate:** 25–40 h.
### 18. Accessibility (U)
- AccessKit on, but no accessible grid/ribbon tree, no high contrast, no checker. **Estimate:** 12–18 h.
[ui-parity.md](ui-parity.md#accessibility)
### 19. XLSB/ODS beyond values; minor formats (FF)
- XLSB and ODS import values only, no write; no `.prn`, `.dif`, `.slk`, XML Spreadsheet 2003; CSV
writes raw instead of displayed values; templates saved with the workbook content type.
**Estimate:** 15–23 h.
### 20. Multiple windows; pen pressure (H)
- Desktop windows done: a native window per workbook window, New Window on the same workbook,
Switch Windows, Arrange All. Left: Arrange All across monitors, one window on the web, no
pressure or tilt in ink. **Estimate:** 5–8 h.
[hardware-parity.md](hardware-parity.md)
### 21. Automation and add-ins (F)
- VBA out of scope (dropped on open, #6); no Office Scripts compatibility; no plugin or custom
function API. **Estimate:** 25–40 h plus an owner decision.
### 22. AI features (F)
- No in-app assistant, natural-language formulas or Analyze Data; agent control via MCP is
already strong. **Estimate:** 30–50 h; model choice needs the owner.
### 23. Mobile (platform)
- No iOS, iPadOS or Android layout. **Estimate:** 30–60 h.
### 24. Fonts and look (U)
- Font weights beyond regular/bold (#168), system font list (#4), Calibri metrics (#29), macOS
chrome (#169, #170), Linux title bar (#54), icon consistency across apps (#173). **Estimate:** 7–12 h.
## Revision history
| Date | Change | Summary |
|---|---|---|
| 2026-10-31 | minor | §13: Solver Load/Save Model (`data.solverSaveModel`, `data.solverLoadModel`) and Reset All (`data.solverReset`), with buttons in the Solver Parameters dialog |
| 2026-10-24 | minor | §8: PivotTable Enable show details: `pivot.options` `enableShowDetails` turns Show Details off for a PivotTable, read and written in XLSX as `enableDrill` |
| 2026-10-23 | minor | §8: PivotTable Show Details: double-clicking a value cell lists its source records on a new sheet (#2364) |
| 2026-10-22 | minor | §20: a native window per workbook window, New Window on the same workbook and Arrange All on the desktop; left: Arrange All across monitors, the web's one window, pen pressure (estimate 10–14 h → 5–8 h) |
| 2026-10-20 | minor | §11: the Japanese catalog covers the dialogs, messages and menus — 943 keys (~43%), matching the pt-BR coverage; a test keeps the two catalogs in step |
| 2026-10-17 | minor | §6: the five reports it tracked are closed and their PRs merged (#57, #133, #139, #140, #134; PRs #148, #155, #156, #157); the entry stays open for §15's oracle suite |
| 2026-10-15 | minor | §6: Maximum Iterations and Maximum Change in File › Options › Formulas; dropped the stale "no Options dialog yet" |
| 2026-10-15 | minor | §13: Solver prepares reports before committing the solution, model and report sheets together; report refusal preserves the workbook and Undo/Redo (#1050) |
| 2026-10-13 | trivial | §4: multi-level category axes landed |
| 2026-10-12 | minor | §9: next/previous sheet commands retain the current visible sheet and selection when there is no visible destination |
| 2026-10-12 | minor | rows auto-fit oversized fonts on open and whole-row/column formatting (#833) |
| 2026-10-12 | minor | Page Layout view: dragging margins on the rulers (#792) removed from the §14 remainder |
| 2026-10-12 | minor | §13: Forecast Sheet Forecast Start (hindcast) landed (#785) |
| 2026-10-12 | minor | §17: ink geometry survives XLSX saves; the icon portion of #551 remains |
| 2026-10-12 | minor | §8: values on rows (Σ Values in Rows, `dataOnRows`) laid out, moved and round-tripped |
| 2026-10-12 | minor | Desktop File > Print now discovers printers and submits PDF jobs through pinned craft-print with printer choice, copies, duplex and grayscale; browser printing and PDF export remain available. Remaining print parity includes page ranges and driver-specific tray/options UI; physical printer fidelity needs hardware validation. |
| 2026-10-11 | minor | §6: iterative calculation for circular references landed (Maximum Iterations, Maximum Change) |
| 2026-10-11 | minor | §15: trim-reference operators (`A1:.E10`, `A1.:E10`, `A1.:.E10`) parse and evaluate |
| 2026-10-11 | minor | §4: trendlines (six types, forecast, equation and R²) read, written and drawn |
| 2026-10-11 | minor | §7: password-encrypted workbooks open with their password; saving with one remains (#11) |
| 2026-10-11 | minor | §16: comment navigation, Show Comments, Show All Notes and Convert to Comments landed (#281) |
| 2026-10-11 | minor | §7: `.xls` (BIFF8) import of values, formulas and number formats (#121) |
| 2026-10-11 | minor | §3: XLSX pass-through keeps custom XML, external links, connections, query tables, printer settings, controls and OLE objects (#330, #324) |
| 2026-10-11 | minor | §13: Forecast Sheet landed (#322) |
| 2026-10-11 | minor | §15: ODDFPRICE, ODDFYIELD, ODDLPRICE, ODDLYIELD added (18 functions missing) |
| 2026-10-11 | minor | §15: PERCENTOF and EUROCONVERT added (20 functions missing) |
| 2026-10-11 | minor | §15: BAHTTEXT added (21 functions missing) |
| 2026-10-11 | minor | §15: TRIMRANGE added (21 functions missing; trim-reference operators still missing) |
| 2026-10-11 | minor | §16: Allow Edit Ranges landed (#307) |
| 2026-10-11 | minor | §8: calculated fields added |
| 2026-10-11 | minor | §8: PivotCharts added |
| 2026-10-11 | minor | §13, §15: FORECAST.ETS family landed (#314) |
| 2026-10-10 | minor | Updated for the 10-10 merges (#17, #31, #38, #62, #82, #104, #116) and the new chart resize bug (#190) |
| 2026-10-10 | minor | Noted that no gap blocks the alpha gate |
| 2026-10-10 | major | Created: 24 ranked gaps with evidence, from the full re-measure and the 60 open issues |
Reference in New Issue View Git Blame Copy Permalink