Compare commits

..
Author SHA1 Message Date
Dylan M. Taylor 4484157cbe merge main into fix/sec-html-export-merge-index 2026-11-11 01:13:05 +00:00
Dylan M. Taylor 6a08240eb6 io: refuse a FIFO before opening it; keep the replaced file's group on save (#3313)
* io: refuse a FIFO before opening it; keep the replaced file's group on save

* address review: create the replacement owner-only, then restore group and mode; never widen it when the group cannot be restored

* address review: keep the old owner and other bits when the group cannot be restored
2026-11-10 18:05:44 -05:00
Dylan M. Taylor a3a98cf745 functions: REGEXREPLACE keeps only the requested occurrence's captures (#3288)
* REGEXREPLACE keeps only the requested occurrence's captures

* address review: do not truncate REGEXREPLACE occurrence on 32-bit targets
2026-11-10 18:05:25 -05:00
Dylan M. Taylor fe39331ac4 engine: Consolidate finds keys through hash maps and refuses an oversized table (#3285)
* engine: Consolidate finds keys through hash maps and refuses an oversized table

data.consolidate searched the keys collected so far for every row and
column key, so 100,000 positional rows cost about five billion
comparisons. It also built every row × column combination without a
size check. Keys are now indexed, and a table over 5 million cells is
refused before it is built.

* address review: count label cells in the consolidate output limit
2026-11-10 18:05:08 -05:00
Dylan M. Taylor a84259d58c ui-egui: embed craft-fonts files only from inside the checkout (#3281)
* ui-egui: embed craft-fonts files only from inside the checkout

The build hook canonicalised each manifest row's file and embedded it
with include_bytes! wherever it led, so an absolute path, a .. path or
a symlink in a craft-fonts checkout could put any readable file of the
build machine into the binary. Manifests and font files must now
resolve inside the canonical checkout directory.

* address review: register the manifest path before validation and keep the Chinese manifest fallback optional
2026-11-10 18:04:49 -05:00
Dylan M. Taylor 85d32c20be engine: show overlong format codes as General without hashing them per cell (#3273)
* engine: show overlong format codes as General without hashing them per cell

The display cache hashed the full format code on every lookup, so one
huge code shared by many cells cost its length for every cell drawn.
A code with more than 4 × 255 bytes can't be within Excel's 255
characters and is now formatted as General straight away.

* address review: keep the original code in the public number_format for overlong formats
2026-11-10 18:04:31 -05:00
Dylan M. Taylor 1deb108a7c xtask: probe keytip commands without file access, and refuse text that could escape generated code (#3269)
* xtask: probe keytip commands without file access, and refuse text that could escape generated code

cargo xtask keytips ran every command keytips.json names, with its
params, under the user's file permissions (file.exportPdf could replace
any file, also during --check). It also wrote target, excel and note
strings into generated Rust without escaping. Probes now run inside the
engine's ScriptSandbox, and those fields may not hold quotes,
backslashes, braces or control characters.

* address review: make ScriptSandbox !Send so it drops on the thread that entered it
2026-11-10 18:04:13 -05:00
Dylan M. Taylor 2c9e2d1eec Go To: index defined names when resolving and listing them (#3206)
Go To resolved every visible name, and each alias inside a name was
looked up with Workbook::name, a linear scan of all names. A workbook
of many short aliases made the Go To list quadratic (or worse along
alias chains) in the number of names.

A Resolver now indexes the names by scope and folded spelling once
per command and remembers, per name, sheet and depth, whether a name
resolves and to how many areas, so listing names resolves each alias
once. Lookups keep the old precedence and matching rules.
2026-11-10 18:03:53 -05:00
Dylan M. Taylor 9c8b4a37ea PDF export: index merged areas when drawing a block (#3203)
draw_block checked every cell against every merge meeting the block,
and the fill, border and text passes searched the merge list again
for each merge start, so an imported sheet with many merge ranges in
a print area cost cells x merges (and merges x merges) comparisons.

The merges meeting a block are now gathered once and swept down the
rows: whether a cell is covered is a BTreeMap lookup among the merges
active on its row, and merge starts are found through a HashMap. A
merge overlapping one kept before it (only possible in a damaged file)
is left out, as Excel does when it repairs the file. Hyperlink
rectangles use the same index instead of scanning all sheet merges.
2026-11-10 18:03:35 -05:00
FredAppsandFredApps a36e4987c1 i18n(uk): translate Chart as Діаграма (#3703)
Use the shared craft-i18n Ukrainian term, distinct from Diagram (Схема).
Also translate the remaining Chart-labelled strings that were still English.

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 17:07:10 +01:00
FredAppsandFredApps 457bc7e374 Naming: use Diagram and Decorative Text in the feature catalog (#3622)
* Naming: use Diagram and Decorative Text in the feature catalog

* Naming: align the Chinese decorative text term

* test(naming): guard neutral display labels

* Naming: Diagram no longer repeats the chart word in de, uk and ro

German Schema and Ukrainian Схема match craft-i18n#211; Romanian Schemă
avoids GridCraft's own chart word Diagramă. A catalog test checks every
shipped language translates both labels and that Diagram differs from
Chart/Charts. The display-name guard now handles escaped char literals
and any #[cfg(test)] item, asserts it scanned catalog.rs and a minimum
of files and literals, and tests word boundaries. The catalog test
asserts each ID appears once. Docs carry timestamps and revision history.

---------

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 16:13:13 +01:00
Harrison Booth 50a9e9ede4 XLS: formulas open as legacy formulas with Excel's implicit @ (#410) (#1367)
.xls formulas are read as legacy formulas with Excel's implicit @ (apply_legacy via read_xls_with), one-cell Ctrl+Shift+Enter arrays keep their range, @ of a 2-D range meets the formula's row and column, and defined-name analyses are shared across the formulas of a pass and replayed exactly.

Closes #410
2026-11-10 15:05:02 +01:00
74ab5cc969 M12: regional format data (Region, 20 regions) (#3048)
* M12: regional format data (Region, 15 regions, Workbook::region)

Adds the data model for Excel-style regional formats, with no behaviour
change: nothing reads it yet.

- core: region.rs with `Region` (separators, grouping, date order, short and
  long date and time codes, month and day names, AM/PM words, currency) and
  the 15 regions en-US, en-GB, en-IN, es-ES, de-DE, fr-FR, it-IT, pt-BR,
  pt-PT, ru-RU, cs-CZ, ja-JP, ko-KR, zh-CN, zh-TW. Values follow Unicode CLDR;
  the deliberate departures (four-digit short-date year, Windows grouping
  spaces) are listed in the module doc. `all`, `find` (strict tag),
  `lookup` (system or browser tag, best effort) and `default_region` (en-US)
  select one; `EN_US` reproduces today's behaviour exactly. Unit tests check
  the tables and the lookups.
- model: `Workbook::region`, a `#[serde(skip)]` runtime field that is never
  saved and defaults to en-US. It is here so that the following changes can
  be reviewed independently of each other.

The region is independent of the interface language and of the formula
language. Builds on this, in order: typed input (core `parse_input_in`),
display (numfmt and engine display), the engine session and `app.region`
command, then the Options UI and the app defaults.

* M12: region data: review fixes (tag extensions, zh territory, month periods)

- lookup stops at an extension or private-use singleton:
  en-GB-u-ca-gregory is en-GB (ca was read as Canada) and
  zh-Hant-u-nu-latn is zh-TW.
- For Chinese an explicit territory wins over the script, so
  zh-Hant-CN is China's conventions, consistent with zh-Hans-TW.
- In regions whose month abbreviations end in a period (ru-RU, fr-FR),
  month_from_name accepts any abbreviation typed with one (мар., июн.);
  elsewhere a period stays outside the name, so en-US is unchanged.

* Region: document where dates and month names follow Windows, not CLDR

* M12: Swiss regions (en-CH, de-CH, fr-CH, it-CH) and nl-NL

The Swiss regions group digits with an apostrophe (CLDR 48) and accept the
typographic one glibc uses; lookup maps CH and LI tags to them. Dutch
(Netherlands) follows CLDR 48.

* Region: table-driven lookup test, under main's cognitive-complexity limit

* Region: French and Italian Liechtenstein tags resolve to the Swiss regions

* Region: credit the Unicode CLDR data under the Unicode License v3 (NOTICE, ATTRIBUTION.md)

* Region: month_from_name takes one word; the currency format has the currency's own decimals

---------

Co-authored-by: quantmind-br <dsrodrigues20@gmail.com>
Co-authored-by: quantforge-br <218010578+quantforge-br@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-10 13:54:53 +01:00
RockerHXandFredApps ec12eba83e UI.1265: align CJK fallback text with the Latin baseline (#3081)
* UI.1265: align CJK fallback text with the Latin baseline

Use epaint's existing baseline alignment for system CJK fallback faces
and embedded faces tagged with CJK scripts. Mixed CJK, Latin and digits
then share a baseline in cells, editors and window titles.

Add regression coverage for zoom, font styles, wrapping, fixed-height
clipping, vertical alignment, editor height and title truncation.

Refs #1265, #2621.

* UI.1265: assert title baselines and document visual validation

* UI.1265: add baseline comparison and review validation

* UI.1265: adapt tooltip regression after upstream merge

The upstream shortcut update uses the runtime OS to choose tooltip text.
Use the harness OS for the existing split-button test expectation too.

Validation: cargo xtask ci; all 10 steps pass; 3,002 tests passed, 1 ignored, no exclusions.

* UI.1265: propagate CJK fallback to installed fonts

* UI.1265: align fallback baselines across cells

* UI.1265: align wrapped and unwrapped cell text

* UI.1265: preserve blank lines in wrapped cell alignment

* UI.1265: preserve centered fallback placement

* UI.1265: find CJK cell text by its visible bounds

Use the overlap of text ink, clip rect and cell rect to locate painted text
instead of testing a fixed offset from the galley origin. Baseline-aligned
fallbacks can move that origin above the cell on Ubuntu.

Keep the existing baseline and wrapping assertions unchanged.

Validation: cargo xtask ci on macOS; all 3 cjk_baseline tests on Ubuntu 24.04 x86_64.

---------

Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-10 12:44:19 +01:00
ca39b5ab9f XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel (#417)
* XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel; implied @ saved back without cm

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* XLSX: long operator chains analysed in full and unfinished legacy analysis leaves the formula unchanged (never saved as SINGLE); @ of a union keeps its parentheses; IF's condition intersected inside SUMPRODUCT

* XLSX: a legacy union already in parentheses saves back without a second pair (never as SINGLE); a name's definition analysed as an array throughout, inserting no @

* XLSX: drop the unreachable keep-unions retry; an @ of a bare union saves as a dynamic formula

* XLSX: an @ keeps parentheses the reader would not put back (redundant ones, a union's)

* XLSX: a redundant @ keeps a legacy formula legacy; IFERROR and IFNA intersected inside SUMPRODUCT; file parentheses and unfinished analyses saved as read

An edit could leave an implied @ on a single cell: deleting rows 2:3 under the legacy SQRT(A1:A3), read as SQRT(@A1:A3), gives SQRT(@A1:A1), and a name redefined to one cell does the same to SQRT(@Rng). The writer kept an @ only when re-analysing the formula without it put the same @ back, which it no longer does for one cell, so the formula was saved as SQRT(_xlfn.SINGLE(A1:A1)) with cm, readable only by dynamic-array Excel. The writer now analyses the formula with its own @ in place: an @ is accepted when the analysis implies it at that position or when its operand is a single value there (such an @ changes nothing), and any @ inserted elsewhere or any other @ still makes the formula dynamic. SQRT(@A1:A1) saves as SQRT(A1:A1), SQRT(@A1) as SQRT(A1) and @SUM(A1:A3) as SUM(A1:A3), all without cm; SUMPRODUCT(@SQRT(A1:A3)) stays dynamic because SQRT(A1:A3) is an array there.

Legacy Excel intersects IFERROR's and IFNA's arguments inside SUMPRODUCT unless the formula is entered with Ctrl+Shift+Enter, like IF's condition: =SUMPRODUCT(IFERROR(VALUE(C19:C21),0)) returns one row's value or #VALUE!, and the accepted fix is CSE or N(+range) (https://stackoverflow.com/questions/59228707). They now get the same single-value parameter class as IF's condition (Param::Single), so the legacy SUMPRODUCT(IFERROR(A1:A3/A1:A3,0)) reads as SUMPRODUCT(IFERROR(@A1:A3/@A1:A3,0)), and a typed SUMPRODUCT(IFERROR(VALUE(A1:A3),0)), which dynamic-array Excel evaluates as an array, is saved with cm. IF's result arguments are left as arrays: in https://stackoverflow.com/questions/29682328, a non-CSE =SUMPRODUCT(--(B4=IF(NOT($C$4:$C$11),$B$4:$B$11))) in row 4 returns 2 (both "apple" rows of B4:B11 counted, so the result branch was not intersected), 0 in a row where C is TRUE and #VALUE! outside rows 4:11 (the condition was intersected).

The reader's @ in front of a whole-formula array result took no extra parentheses when the file already had them, so ({1,2}+1) and {1,2}+1 both read as @({1,2}+1), and the writer dropped the file's parentheses. An operand already in parentheses now gets a second pair, so ({1,2}+1) saves as written.

A legacy formula the analysis could not finish (node budget, nesting) was read unchanged, evaluated as a dynamic-array formula and saved with cm, so Excel spilled it. Names were analysed again at every use, so a 399-term name used six times exhausted the 4096-node budget; their kinds are now remembered per formula, and the budget is 16384 nodes. The nesting limit is 128, so Excel's 64 levels of nested functions are analysed. A formula that still exceeds the limits is saved as it was read, without cm.

* XLSX: an implied @ is dropped and any other @ of a single value kept as _xlfn.SINGLE in a legacy formula (as Excel writes it); a legacy formula the analysis cannot finish or whose semantics it does not know opens with a whole-formula @ and never spills; analysis depth follows the parser's limit

* XLSX: a legacy union given to a function's single-value parameter keeps its implied @, so it calculates to #VALUE! instead of spilling and saves as read; checked on a calculated workbook

* XLSX: only a one-cell range an @ applies to counts as one an edit shrank; a reading past the parser's nesting limit leaves the formula as read, and the writer saves the text a formula it cannot analyse was read from

* XLSX: a formula nested past the parser's limit for its reading is left as read (documented: it may spill); test covers it

* XLSX: a name's definition counts its one-cell ranges as collapsed only when it is itself a reference, so CHOOSE's index or IF's condition in it keeps the name legacy

* XLSX: an @ on a name or one-cell range counts as implied when ranges an edit may have shrunk, anywhere the name returns them (CHOOSE's choices, parentheses), make it so; deleting rows keeps such formulas legacy

* XLSX: the collapsed-range retry of an @ runs only when the first pass reached a one-cell range written as a range (tracked through cached names)

* docs: describe unfinished formula export fallback

* docs: the unfinished-analysis fallback differs for formulas with an explicit @

* Preserve main formula analyzer fixes

* Order file-format revision history

---------

Co-authored-by: neilcawse <8085959+neilcawse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Local contributor <contributor@localhost>
2026-11-10 12:43:07 +01:00
e9addf52fe Arabic bidi layout and cell reading direction in the grid and editors (#947)
* M12.AR4: preserve Arabic logical text in bidi display and editing

* M12.AR4: drop the stale "no bidi" note from the RTL sheet status

* M12.AR4: open the Arabic editors and place the caret through key and pointer events in tests

* M12.AR4: skip the conditional-format lookup in the formula bar unless a rule sets reading order

* M12.AR4: test that the caret after typed Arabic is drawn at the text's left end (#1070)

* M12.AR4: pass the theme body font in the reading-order style test

* M12.AR4: adapt the Arabic UI tests to egui 0.37 (row direction, editor ids, Ctrl+End to the logical end)

* M12.AR4: read the active sheet through Workbook::active_sheet() in the formula bar

main made `Workbook::active_sheet` an `ActiveSheet` (shared between windows on one
workbook); the formula bar's reading-order lookup takes the index from the accessor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR4: keep explicit Left on wrapped right-to-left cells and trailing spaces in the editor

Wrapped cell text lays out every row from the left when the painted alignment is Left, instead of aligning right-to-left rows to their start. The cell and formula bar editor keep trailing whitespace in their layout so a right-to-left entry ending in a space keeps the caret inside the text.

* M12.AR4: keep Right and General on right-to-left sheets aligned right in wrapped cells

* M12.AR4: keep right-aligned Arabic with a marked first letter inside its cell

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-09 15:55:43 +01:00
cda767b49b Arabic font fallback in cells, the formula bar and the web (#931)
* M12.AR2: Arabic font fallback in cells and the interface (#229)

* M12.AR2: reserve the preferred Arabic file only once it has loaded

load_fallback treated fontconfig's Arabic file as already loaded even
when load_preferred_arabic could not load it, so a system Arabic face
from the same file was skipped and the chains could end up with no
Arabic system face. The file is now reserved only when the preferred
face is in the font set; the candidate loop moves to load_faces so a
test can drive it with a fixture file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR2: say where fontconfig's Arabic choice applies

fontconfig's Arabic face leads the Arabic fallbacks, so it draws Arabic
where the UI or cell font has none. A UI font with Arabic of its own,
such as DejaVu Sans in the Flatpak, still draws interface Arabic (#229),
which is left for a follow-up. The docs and the test's doc comment no
longer say the choice comes first in every chain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-09 15:54:55 +01:00
c35234e5f5 Number formats: honour the B1/B2 calendar prefixes (Hijri dates for B2) (#1918)
* M12.AR6: honour the B1/B2 calendar prefixes in number formats (Hijri for B2)

* M12.AR6: match Excel's B1/B2 behaviour measured in review

Changes from a reviewer's Excel run (TEXT and Range.NumberFormat):
- A [$-...] locale tag before B2 keeps the date Gregorian (prefix still
  hidden); B2[$-401] stays Hijri.
- Any other unquoted B and digit (B3dd, dd/mm/yyyyB2, dd B2 mm), and B2
  before a number format (B20.00), make the code invalid: TEXT gives
  #VALUE!, a cell shows General, as for a code over 255 characters.
- Under B2, ddd and dddd show the Arabic weekday name (full name for both).
- Serial 60 has no Hijri date: TEXT gives #VALUE!, a cell shows ####.
- Serial 0 shows as serial 1 (29/08/1317).

Also fixes a break from merging main: days_from_civil now returns 0 for
day 0 and 29 February 1900, which sent serials 0 and 60 to 1970
(23/10/1389) under B2. The Hijri day count no longer passes those dates.

Docs: localization-parity states the behaviour, including the one
measured case not matched (mmmm next to dddd prints the weekday in Excel).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR6: Hijri leap year in cycle year 15, not 16, as Excel shows

A reviewer compared Excel's B2yyyy-mm-dd on 43,512 serials with this
formula: it disagreed by one day on 1,372, all in cycle years 15 and 16,
and moving the leap year from 16 to 15 left no disagreement. The leap
years are now 2, 5, 7, 10, 13, 15, 18, 21, 24, 26 and 29 (still 10,631
days a cycle).

The count of leap years before a year becomes (4 + 11y) div 30; it
differs from the old (3 + 11y) div 30 only when y = 16 (mod 30), which is
exactly the move from 16 to 15. The inverse's year estimate can now be one
year off at a boundary and is corrected by one step.

Tests: Excel values for 2 March 2005 (1426-01-21) and 29 January 2006
(1426-12-29) plus the 1425/1426 year boundaries they imply, the reviewer's
other anchors, and exhaustive checks: year, month and 30-year cycle
lengths for Hijri years 1 to 10,000; day -> Hijri -> day round trip and
day-by-day succession for every day from 1 Muharram 1 AH to one cycle past
31 December 9999; every 1900-system serial 1 to 2,958,465 (except 60).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR6: match B1/B2 details checked in Excel 16.0 build 20527

Compared GridCraft with Excel 16.0 (build 20527, en-US) through COM: TEXT
with B2dd/mm/yyyy on every serial from 1 to 2,958,465 (no difference), and
about 1,750 format/value pairs through both a cell's NumberFormat and TEXT.
Fixed what differed and is in scope:

- Two-word Hijri month names use a no-break space (U+00A0), as Excel does.
- B1 before a number format is rejected like B2 (B10.00, B1#,##0).
- Prefix runs: a repeated prefix is accepted (B2B2, B2b2, [Red]B2B2, B1B1);
  a mixed one (B1B2, B2B1), more digits (B12, B21, B02), a prefix after a
  tag or space (B2[Red]B2, B2 B2) and an escaped \B2 are rejected.
- Serial 60 under B2 is #VALUE! for any date or clock part (hh:mm, ss,
  AM/PM), not only date parts; [h] and literal-only sections still show.

Regression tests use the Excel values (all 12 month and 7 weekday names,
the accepted and rejected codes, serial 60). Not changed, listed in the
docs: Excel's weekday/month-name mix-ups when ddd(d) and mmm(m) share a B2
section, weekday-only B2 sections (real weekday for serials 0-59, #VALUE!
for the last days of 9999), and Buddhist-year b/bb/bbbb tokens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-09 15:54:05 +01:00
004523c286 AutoFit: estimate Arabic text width from measured glyph widths (#1933)
* M12.AR12: estimate Arabic text width from measured glyph widths in AutoFit

* M12.AR12: treat U+0897–089F as marks and U+08E2 as visible; size the honorific ligatures (review)

* M12.AR12: size the remaining honorific ligatures like their siblings (review)

U+FD4C-FD4F (Unicode 14) and U+FDC8-FDCF (FDCF Unicode 14, FDC8-FDCE
Unicode 17) are honorific word ligatures, not letters, but fell into
the 0.48 em letter arm. Give them the 1.6 em of the measured
U+FD40-FD4B honorifics and assert each range's endpoints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR12: size Arabic-Indic digits like ASCII digits and the remaining honorific ligatures (review)

Arabic-Indic (U+0660-0669) and Extended Arabic-Indic (U+06F0-06F9)
digits measure 0.53-0.57 em in Noto Sans Arabic, Arial, Tahoma and
Times New Roman, the same as each font's ASCII digits, so they get
the ASCII digit estimate (0.56) instead of the letter estimate.
Arabic punctuation measures 0.17-0.50 em, so it keeps 0.48 rather
than falling back to 1 em.

U+FBC3-FBD2 and U+FD90-FD91 are honorific word ligatures like
U+FD40-FD4F and get the same 1.6 em.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-09 15:52:55 +01:00
517a4b9256 Sort: order Arabic text by its letters, not its vowel marks or digit script (#1940)
* M12.AR7: sort Arabic text by its letters, not its vowel marks or digit script

* M12.AR7: order marked Arabic text the way Excel does

Checked compare_text against Excel 16.0 build 20527 (en-US) via COM:
Range.Sort, =SORT() and the < / = operators over ~300 strings. Fixes the
mismatches inside this change's scope:

- The tie-break compared raw code points, so an RLM-prefixed word sorted
  after every vowelled form of it, "‏12" after "١٢", and a tatweel
  form after a shadda form. Ignorable characters are now skipped at the
  tie-break too.
- Marks are compared letter by letter (an unmarked letter first), so a
  mark on an earlier letter outranks the digit or letter after it, as in
  Excel ("ب١" < "بَ1", "بپ" < "بَپ").
- Marks rank in Excel's measured order (sukun before shadda), and the
  Quranic marks Excel skips (U+0610–061A, U+06D8–06ED except 06DD/06DE/
  06DF/06E0/06E5/06E6/06E9) are skipped by letter order.
- Excel also ignores bidi embeddings/overrides/isolates, BOM, soft hyphen,
  CGJ and word joiner; they now are ignored too (ZWSP is not, as in Excel).

Equality is unchanged. Regression tests use the orders Excel produced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR7: compare the mark level of Arabic text without allocating

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-09 15:49:26 +01:00
Valentin KleinandClaude Opus 5.5 7102429a9a perf(grid): skip hidden row runs and keep sheet geometry and CF statistics between frames (#3580) (#3629)
A sheet whose filter hides a million rows walked every hidden row
several times per frame (cells, gridlines, headers), rebuilt the row
index, the outline and the used range every frame, and a conditional
format over a whole column rescanned it every frame for its statistics.

- LineIndex::next_visible / visible step over a run of hidden lines in
  one O(log n) search; the grid's row and column loops use them.
- The grid keeps the active sheet's line index, outlines, used range and
  CfCache between frames, tied to the sheet by a Weak: any change goes
  through Arc::make_mut, which moves or copies a sheet that has a weak
  reference, so the cache can't outlive a change.
- Scroll offsets for the scrollbars and the scroll limit come from the
  line index instead of walking every custom row.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 08:46:53 +01:00
Dylan M. Taylor 76416d4869 Merge pull request #3641 from dotCooCoo/chore/craft-update-1b34eba41
Updates: take craft-update from craft-libs main at 1b34eba41
2026-11-08 22:56:18 -05:00
dotCooCoo 3004a05e46 Updates: take craft-update from craft-libs main at 1b34eba41 2026-11-08 19:45:34 -08:00
Dylan M. Taylor 01033f2c8f Merge pull request #3621 from dotCooCoo/chore/craft-update-updater-fixes
Updates: the updater reports installing, ready, installed and failed, and installs at exit
2026-11-08 22:12:47 -05:00
dotCooCoo 99bbd237fc Updates: the updater reports installing, ready, installed and failed, and installs at exit
craft-update, craft-window, craft-print and craft-ui move from craft-libs
c5a1cbfb to c7c84230, a branch with storytold/craft-libs#687 and #696
merged into craft-libs main.

- The check result window shows whenever the updater's status is not
  idle, so the first start after an update reports that it installed or
  why it failed. Restart to Update asks the app to quit once nothing is
  unsaved.
- An update's close request quits the app (Quit asks about each unsaved
  workbook), so every workbook window closes.
- on_exit starts the install helper of a ready update before the macOS
  process exit.
2026-11-08 11:53:25 -08:00
FredApps 2c58c76914 Merge pull request #2696 from vangel76/perf/calc-spill-unblock-2684
perf(calc): find formulas a cleared cell may unblock from a set of #SPILL! cells
2026-11-08 13:01:25 +01:00
FredApps c000e1b3ff Merge pull request #2694 from vangel76/perf/calc-spill-index-2683
perf(calc): index spill areas so cell and range reads don't scan every spill
2026-11-08 12:58:44 +01:00
FredApps 21920d3d61 Merge pull request #3297: complete equation input caps with #3328
Equations: refuse macro definitions and cap matrix cells
2026-11-08 12:47:35 +01:00
FredApps a2509bdf8b Merge newline matrix-cap fix into prerequisite (#3297, #3328)
Keep the complete reviewed matrix bound together so main never receives the known newline-row bypass. This merge tree is identical to the validated #3328 head 83670dfe3a.
2026-11-08 12:33:22 +01:00
vangel76andClaude Opus 5.5 0f5a860d74 Merge perf/calc-spill-index-2683 (with upstream/main) into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:23:30 +01:00
vangel76andClaude Opus 5.5 17131fb0e5 Merge upstream/main into perf/calc-spill-index-2683
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:18:31 +01:00
FredApps 94a8fb084a Merge pull request #1152 from vangel76/perf/chart-measure-allocs
Charts: label fitting and legend measuring avoid hundreds of text layouts per render (#1149)
2026-11-08 12:08:59 +01:00
FredApps fca22f6e93 Merge pull request #1155 from vangel76/fix/chart-work-budgets
Charts: work and memory are bounded by series x points; ranges are clipped before evaluation (#1147)
2026-11-08 12:08:35 +01:00
FredApps 300b074894 Merge pull request #3325 from vangel76/fix/binom-poisson-accuracy
BINOM.DIST, POISSON.DIST: Loader mass and recurrence sums, error 1e-11 -> 1e-14 (#3324)
2026-11-08 12:08:16 +01:00
FredApps 9dd3b51bc6 Merge pull request #3383 from vangel76/fix/3350-ctrl-tab-arrows
Tabs: Ctrl-click on the sheet arrows jumps to the first or last sheet
2026-11-08 12:07:28 +01:00
FredApps 4a93722ea6 Merge pull request #2691 from vangel76/perf/calc-range-memo-2681
perf(calc): SUM/COUNT/AVERAGE over big ranges reuse constant-only results
2026-11-08 12:06:13 +01:00
FredApps 6338cef450 Merge pull request #3578 from vangel76/fix/3577-json-formula-trees
Cache formula trees when a workbook is read from JSON
2026-11-08 12:05:30 +01:00
FredApps 24119e49fe Merge pull request #3581 from vangel76/perf/3575-store-shifts
Move whole rows when inserting or deleting rows and columns
2026-11-08 12:05:00 +01:00
FredApps cbd0879c44 Merge pull request #3591 from vangel76/perf/engine-adjust-if-changed
Structural edits rewrite only the formulas they change
2026-11-08 12:03:48 +01:00
FredApps 0057fc00c9 Merge pull request #3579 from vangel76/perf/adjust-unchanged-check
Formulas: adjust_if_changed reports when a structural edit leaves a formula alone
2026-11-08 12:02:12 +01:00
vangel76andClaude Opus 5.5 9ab297ced1 perf(engine): rewrite only the formulas a structural edit changes
rewrite_all_formulas borrows each tree with Formula::with_expr and calls
adjust_if_changed, so a formula the edit leaves alone is neither copied
nor printed, and a changed one is printed once. A legacy array whose
range moves while its formula stays the same still gets the new range.

Sheet rename and delete compare the adjusted tree with the old one
instead of printed text, and replace only formulas that changed. Delete
no longer rebuilds every formula in the workbook, so unrelated formulas
keep their array range and dynamic-array mark.

A stored non-canonical formula text is no longer re-printed by an edit
that does not change the formula; a test pins this.

Part of #3576

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 04:06:21 +01:00
vangel76andClaude Opus 5.5 2ee261c0e8 perf(model): move whole rows when inserting or deleting rows and columns (#3575)
CellStore::shift_rows, shift_cols, shift_cols_in_rows and take_range removed
and re-set every affected cell one by one. They now take whole row maps (or
the split-off part of one) out of the bands and put them back at their new
position, so a row shift costs a few map operations per row instead of a
band lookup, a remove and a set per cell. take_range does this when the
block reaches the last column.

Behaviour is unchanged, including blank cells left behind by get_mut, cells off the grid (they
are still dropped, and still clear the cell they land on), cells shifted off
the sheet, and copy-on-write: bands without moved cells stay shared with undo
snapshots. A differential test runs 12,000 random operations against the old
per-cell code and compares contents, length, iteration order and band sharing.

shift_rows_in_cols, and take_range on a block with columns to its right, keep
moving cell by cell: the row-map version measured no faster there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 03:51:57 +01:00
vangel76andClaude Opus 5.5 7cb0088361 perf(formula): adjust_if_changed skips formulas a structural edit leaves alone
adjust had no way to report that an edit does not affect a formula, so
callers copied, rebuilt and printed every formula in the workbook.
adjust_if_changed borrows the formula and returns None when nothing
moves. Its check reuses adjust_ref on each reference into the edited
sheet, on a copy without the sheet name, so it cannot disagree with
adjust. adjust_ref and shift_ref take the reference by value and no
longer clone sheet names.

An integration test keeps the previous implementation as an oracle
over every edit kind and a randomized property test.

Fixes #3576

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 03:04:24 +01:00
vangel76andClaude Opus 5.5 4bba98fd50 perf(model): cache formula trees when reading a workbook from JSON
A Formula read from JSON had no cached tree (`expr` is not saved), so
every graph rebuild and every recalculation parsed each formula again.
Reading one now parses its text once and keeps the tree, as
Formula::new does. The saved text is kept as written.

Fixes #3577

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 02:57:51 +01:00
640fbf22f6 fix(edit): paste, Ctrl+Enter, Fill and Copy skip rows a filter hides (#1101) (#1175)
* Paste, Ctrl+Enter and Fill skip rows hidden by a filter (#1101)

Repeating one cell or row down a filtered selection (paste of a single
cell or row, Ctrl+Enter, Fill Down/Up/Right/Left) now leaves the rows an
applied filter hides alone, as Excel does. A taller pasted block still
goes straight down, and rows hidden by hand are still written, also as
in Excel. Each case was measured in Excel for Microsoft 365.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2bJ3LxrZ6YnmppgrfwHwr

* Copying a filtered range takes only the rows showing (#1101)

A copied range with rows hidden by an applied filter is split into its
runs of showing rows and packed like a multi-area copy (formulas as
values), so pasting it inside GridCraft no longer brings the hidden rows
back. One run is copied as a plain range, keeping formulas. Cut still
moves every row. All as measured in Excel for Microsoft 365.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2bJ3LxrZ6YnmppgrfwHwr

* Find the rows a filter hides without rescanning every table per row

filtered_rows asks the sheet for its filter row spans once instead of
checking every table for each hidden row.

* Paste over filtered rows leaves their validation rules alone

When one row is tiled over filtered rows, paste_validations no longer
clears or copies rules on the rows the filter hides.

* Filtered copy packs links and validation with the rows showing

slide_together now moves hyperlinks and validation rules with their rows, and a
filtered whole-row copy records the packed rows as its selection so Insert
Copied Cells inserts only the rows copied.

* Group validation paste options and format the new tests

* Pack a filtered copy in one pass over notes, merges, links and spills

slide_together finds each entry's area by search instead of scanning every map
once per area, so a filter leaving thousands of separate runs no longer costs
runs times map size on every copy.

---------

Co-authored-by: Richard <richard1912@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: skylar <hi@skylar.cc>
2026-11-07 19:30:06 -06:00
vangel76andClaude Sonnet 5.5 865a7f6a06 Merge upstream/main into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 19:09:51 +01:00
vangel76andClaude Sonnet 5.5 1458f45214 Merge upstream/main into perf/calc-spill-index-2683
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 19:09:31 +01:00
vangel76andClaude Sonnet 5.5 05a5845234 Merge upstream/main into perf/calc-range-memo-2681
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:38:18 +01:00
vangel76andClaude Sonnet 5.5 762be28e29 Merge upstream/main into fix/math-macro-defs-and-matrix-cells
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:38:03 +01:00
vangel76andClaude Sonnet 5.5 8e5bcf1575 Merge upstream/main into fix/binom-poisson-accuracy
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:37:33 +01:00
vangel76andClaude Sonnet 5.5 83670dfe3a Merge upstream/main into fix/math-cap-lookahead
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:37:18 +01:00
vangel76andClaude Sonnet 5.5 93dab0c420 Merge upstream/main into fix/3350-ctrl-tab-arrows
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:36:49 +01:00
FredApps dea064b857 Merge pull request #3547 from FredApps/fix/chart-table-series-index-1150
fix(chart): pass series index to data-table keys
2026-11-07 17:47:58 +01:00
vangel76andClaude Sonnet 5.5 9354fdb0c4 Chart: fit labels with a line break by a linear scan
The width of a prefix plus an ellipsis is not monotone when the text has a
line break, so the binary search could drop a label that fits. Also pass the
series index to plot_kind in the data table (upstream/main does not compile
without it).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:37:20 +01:00
vangel76 5f84fadaf5 Merge remote-tracking branch 'upstream/main' into c1152 2026-11-07 17:32:43 +01:00
vangel76 9ba79c3e38 fmt 2026-11-07 17:31:41 +01:00
vangel76andClaude Sonnet 5.5 6a7cf9cf66 Chart: bound the category axis by the point cap, not the raw cell count
Read only the cells the category levels use and cut the category dimension
at the per-series cap, so a multi-level axis keeps all its levels.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:31:27 +01:00
vangel76andClaude Sonnet 5.5 c35878948a Merge upstream/main into fix/chart-work-budgets
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:31:27 +01:00
vangel76andClaude Sonnet 5.5 42370639e0 Merge upstream/main into perf/calc-range-memo-2681
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:29:04 +01:00
FredApps 6b19e445f4 fix(chart): pass series index to data-table keys (#1150) 2026-11-07 17:27:25 +01:00
vangel76andClaude Sonnet 5.5 05cffdceb1 Tab strip: test the Previous arrow reveal for the already active first tab
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:27:07 +01:00
vangel76andClaude Sonnet 5.5 87cb0f11d2 Tab strip: reveal the edge tab after a repeated Ctrl-click on an arrow
When the edge sheet was already active nothing changed, so the strip was not
scrolled back to it. A Ctrl-click on the arrow now asks for the reveal itself.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:22:54 +01:00
vangel76andClaude Sonnet 5.5 12f69f99bf BINOM.DIST: keep small-k masses whose complement power underflows
The small-k product also needs (1 - p)^(n - k) to stay a normal number;
otherwise it returned 0 for BINOM.DIST(15,1100,0.5,FALSE) (exact
2.1367646606291324e-298) and the cumulative form built on it. Those cases
now use the logarithmic formula. Reference grid gets k = 10..25 at n = 1100.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:20:53 +01:00
vangel76andClaude Sonnet 5.5 225496e544 Math: refuse computed environment names in the alignat pair cap
A name such as align\@char{97}t reads as alignat to the parser but split at
the first } looked like another environment, so the pair limit was skipped.
Names that contain a backslash or brace, or do not close in the window, are
now refused like an unreadable count.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:19:20 +01:00
vangel76 c125ce4780 Merge branch 'fa-3297' into fa-3328 2026-11-07 17:19:20 +01:00
vangel76andClaude Sonnet 5.5 76002fe0bd Reuse remembered range sums for every COUNT block
COUNT never reads the running sum, so the zero-sum gate that keeps SUM
and AVERAGE additions in order does not apply to it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 16:58:19 +01:00
FredApps 505cae34c8 Merge pull request #1150 from vangel76/fix/chart-combo-logic-cleanup
Charts: one combo plot-kind function, no dead scale helpers (#1146)
2026-11-07 16:52:46 +01:00
FredApps ee1b5df3db Merge pull request #2693 from vangel76/perf/calc-expr-arc-2685
calc: rebuild the dependency graph from shared syntax trees
2026-11-07 16:51:36 +01:00
FredApps cbd765332c Merge pull request #2686 from vangel76/perf/stat-select-2679
perf: select instead of sort in MEDIAN, PERCENTILE, QUARTILE, LARGE, SMALL
2026-11-07 16:51:02 +01:00
FredApps b9e5e0ff89 Merge pull request #3309 from vangel76/fix-autosum-gap-3291
fix(autosum): stop the guessed range at a blank cell
2026-11-07 16:50:28 +01:00
FredApps 66bb6ae239 Merge pull request #3331 from vangel76/contributors-vangel76-names
contributors: set names for vangel76
2026-11-07 16:49:24 +01:00
vangel76andClaude Sonnet 5.5 bc1625248e Chart: cap category levels in the clean-data check, recompute overflowing moving-average windows, keep disjoint stacked hairline spans apart
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 16:25:23 +01:00
FredApps 4fc2c9c3b3 Merge pull request #3524 from helenkwok/web/page-api
gridcraft-web: window.gridcraft, the page command API from craft-web-api
2026-11-07 15:38:37 +01:00
FredApps 546a546943 Merge pull request #3508 from ZIDAN44/fix/3484-ctrl-shift-arrow-active-line
Selection: Ctrl+Shift+Arrow searches the active cell's line (#3484)
2026-11-07 15:38:30 +01:00
FredApps 8774fa45fb Merge pull request #3507 from ZIDAN44/fix/3485-scroll-per-window
View: a sheet keeps its own scroll position when sheets move (#3485)
2026-11-07 15:38:25 +01:00
FredApps 8325dbe50e Merge pull request #3506 from ZIDAN44/fix/3446-sort-dialog-column-label
Sort: the closed Sort by box names the column, as its list does (#3446)
2026-11-07 15:38:20 +01:00
Helen Kwok fb17d0d1d5 gridcraft-web: say a call the app already took still finishes after a timeout 2026-11-07 10:28:03 +08:00
Helen Kwok 89ec06f041 gridcraft-web: window.gridcraft, the page command API from craft-web-api
The web shell queues the page's calls as control requests (the channel the desktop control server uses) and answers them on the next frame. A call the page gave up on is released, so the app skips it instead of running it late. The API is installed in every mode; opening, saving, the file name and the unsaved flag stay with the host messages.
2026-11-07 08:03:07 +08:00
Zinadin ZidanandOlivier 20473a4fae Selection: Ctrl+Shift+Arrow searches the active cell's line (#3484)
Ctrl+Shift+Arrow searched from the selection's moving corner, so after
Ctrl+Shift+Right from A2 the next Ctrl+Shift+Down searched the far column
and ran to the last row. As in Excel, it now searches the active cell's
column (Up/Down) or row (Left/Right) from the edge being moved, and moves
only that edge. A diagonal move from command params keeps the old jump.

Co-Authored-By: Olivier <5681030+tutis@users.noreply.github.com>
2026-11-07 00:27:08 +06:00
Zinadin ZidanandOlivier 1688b43ae2 View: a sheet keeps its own scroll position when sheets move (#3485)
Scroll positions were stored by (window, sheet index), and nothing re-keyed
them when sheets were inserted, deleted or moved. A new PivotTable sheet,
which goes in front of its source, opened at the source's position, and the
source took over whatever was stored at its new index. Positions are now
keyed by (window, sheet uid), so they follow the sheet through inserts,
deletes, moves and undo. A closed window's positions are dropped. Saving the
active cell still happens when no window is active.

Co-Authored-By: Olivier <5681030+tutis@users.noreply.github.com>
2026-11-07 00:27:07 +06:00
Zinadin Zidan 7d3ede854f Sort: the closed Sort by box names the column, as its list does (#3446)
The Sort dialog's closed "Sort by" box showed only the stored column
letter ("B") while its open list showed "B: Revenue". It now looks the
letter up in the dialog's column labels and shows the same label, falling
back to the bare letter for a column outside the data block.
2026-11-07 00:27:07 +06:00
vangel76andClaude Sonnet 5.5 558312cb53 Merge upstream/main into c1155
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 05:20:58 +01:00
vangel76andClaude Sonnet 5.5 23c909e477 chart perf tests: import LegendPos (legend module now imports it itself)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 01:53:24 +01:00
vangel76andClaude Sonnet 5.5 0955378d8e Merge upstream/main into perf/chart-measure-allocs
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 01:50:34 +01:00
vangel76andClaude Sonnet 5.5 756d5981b1 Merge upstream/main into fix/binom-poisson-accuracy
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 01:41:42 +01:00
vangel76andClaude Sonnet 5.5 7e828a982c Tabs: Ctrl-click on the sheet arrows jumps to the first or last visible sheet (#3350)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 01:02:21 +01:00
Dylan M. Taylor 7e967f6489 address review: check the web page size after each chart and before returning it 2026-11-04 01:20:55 +00:00
vangel76andClaude Sonnet 5.5 b3a709f314 contributors: set names for vangel76
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-04 02:12:09 +01:00
vangel76andClaude Sonnet 5.5 86c5c8dd81 Math: count \newline as a row break in the matrix cell cap (#3327)
RaTeX reads \newline as \\, but the source scan only counted \\ and \cr, so
a matrix built with \newline skipped the 2,500 cell cap.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-04 01:31:04 +01:00
vangel76andClaude Sonnet 5.5 f077b9ef36 Merge upstream/main
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-04 00:05:58 +01:00
vangel76andClaude Sonnet 5.5 40e75c7912 binom/poisson reference test: fail on errors, NaN and non-zero results for zero references (review point)
CodeRabbit pointed out that an error or NaN result passed the accuracy test and
that a zero reference allowed any finite value (the bound was infinite). Errors
now count as infinite error and a zero reference allows 1e-300.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-04 00:05:58 +01:00
vangel76andClaude Sonnet 5.5 d8ba397bb7 math: refuse an alignedat count hidden by a comment or spaces (review point: cap bypass)
CodeRabbit found that \begin{alignedat}%<newline>{99999999} and {9 9} made the
count parser return None, which skipped the 50-pair cap. The lookahead now drops
comments and spaces as RaTeX's lexer does, and a matched alignat name with a
count that is not a plain digit string is refused.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-04 00:04:32 +01:00
vangel76andClaude Sonnet 5.5 7e8b94fc39 BINOM.DIST: keep tiny masses and huge n out of the product form
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-03 23:54:53 +01:00
vangel76andClaude Sonnet 5.5 22c06b7a4a BINOM.DIST, POISSON.DIST: Loader mass, recurrence sums, exact prefactors (#3324)
The mass came from ln_gamma differences (relative error 1.1e-13 at n = 100,
1.5e-11 at n = 10,000) and the cumulative forms summed up to 10,000 of them.
The mass now comes from Loader's saddle-point formula, the cumulative sum
walks away from the mode by the ratio of neighbouring terms and stops when
terms stop mattering, and the incomplete beta / gamma branches take the same
mass as their prefactor.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-03 23:49:52 +01:00
vangel76andClaude Sonnet 5.5 1ff10df557 fix(autosum): stop the guessed range at a blank cell (#3291)
The single-cell AutoSum guess stepped over one blank cell when a number sat
above it, so a second block of numbers below a gap was summed together with
the first block. It now stops at the first non-number, as Excel does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-03 20:56:00 +01:00
vangel76andClaude Sonnet 5.5 472132b158 math: refuse macro definitions and cap matrix cells
RaTeX counts macro expansions (1,000) but not how large they grow, so a
definition whose argument doubles on every call keeps the expander busy
without bound. User macros have no use in an equation, so check_caps now
refuses the definers (def, gdef, edef, xdef, let, futurelet, global, long,
newcommand, renewcommand, providecommand) before RaTeX sees the source. The
scan reads control words, verbatim text and comments the way RaTeX's lexer
does.

A matrix is laid out and written as rows times the widest row, so one long row
in a tall matrix multiplied the output. check_caps now counts the padded cells
of every environment from the source and refuses more than 2,500 in total;
to_omml checks the parsed grid as well.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-03 19:50:27 +01:00
Dylan M. Taylor 52e64c9626 engine: index merges and bound the page in HTML export and clipboard HTML
Save as Web Page and the clipboard's HTML scanned every merge of the
sheet for every cell, counted a merge's span over all the rows the file
gave it, placed each picture by summing every custom row before it, and
grew the page without a byte limit. Merges are now indexed once over
the exported rectangle (overlapping merges past a few visits per cell
are refused), spans use prefix counts clipped to the table, anchors use
LineIndex, and the page stops at 256 MiB with an error.
2026-11-03 12:24:25 +00:00
vangel76andClaude Sonnet 5.5 bcad4a75cf Merge calc: register a Python result's spill in the spill index (#2694) into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 20:32:11 +01:00
vangel76andClaude Sonnet 5.5 e85b24df57 calc: register a Python result's spill in the spill index
python_result wrote into spills but not into spill_index, so indexed readers
(cell_value, range_values, visit_range) missed a Python array laid out in the
same pass. Insert on spill, remove when blocked, as the formula path does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 20:31:52 +01:00
vangel76andClaude Sonnet 5.5 4fe0816ca0 Merge upstream/main into fix/chart-work-budgets
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:49:07 +01:00
vangel76andClaude Sonnet 5.5 00f565bc11 Factor out types clippy flags as too complex
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:44:16 +01:00
vangel76andClaude Sonnet 5.5 4b98b11996 Merge upstream/main into perf/calc-range-memo-2681
A Python cell is not a constant for the range_sum block memo.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:41:44 +01:00
vangel76andClaude Sonnet 5.5 21504859c8 Merge upstream/main into perf/calc-expr-arc-2685
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:36:41 +01:00
vangel76andClaude Sonnet 5.5 a9db1da90d Merge upstream/main into perf/calc-spill-index-2683
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:34:16 +01:00
vangel76andClaude Sonnet 5.5 96672e8ad5 Drop helpers the merged range reads no longer use
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:33:53 +01:00
vangel76andClaude Sonnet 5.5 628da5117a Merge upstream/main into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:28:52 +01:00
vangel76 33cadfc0a8 Merge remote-tracking branch 'upstream/main' into HEAD 2026-10-30 00:54:56 +01:00
vangel76andClaude Sonnet 5.5 494a7dbb5c Keep clustered hairline bars of each series in their own strip of the pixel column
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-30 00:54:56 +01:00
vangel76 95ba076778 Merge remote-tracking branch 'origin/perf/calc-spill-index-2683' into HEAD 2026-10-30 00:04:34 +01:00
vangel76andClaude Sonnet 5.5 bdbd29e623 perf(calc): size the spill-index decision by the sheet's own changed cells
A batch spread over several sheets used its total length for every sheet, so a
one-cell sheet with many spills built the block index as if it had the whole
batch's lookups.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-30 00:04:25 +01:00
vangel76 edfde80f95 Merge remote-tracking branch 'upstream/main' into HEAD 2026-10-30 00:02:22 +01:00
vangel76 54ba74cfc0 Merge remote-tracking branch 'upstream/main' into HEAD 2026-10-29 23:50:34 +01:00
vangel76andClaude Sonnet 5.5 c1e1c7bdc9 Merge origin/perf/calc-spill-index-2683 into perf/calc-spill-unblock-2684
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 23:49:01 +01:00
vangel76andClaude Sonnet 5.5 6c912d2eb1 perf(calc): keep tall and wide spill areas out of the block index
A full-height column spill took 65,536 block registrations, on every pass
that produced it. Areas over 1024 blocks are now tested directly by each
lookup instead. Adds a randomized comparison against a scan of all areas.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 23:48:11 +01:00
vangel76 1801f4a7cf Merge upstream/main into perf/calc-spill-unblock-2684 2026-10-29 23:09:25 +01:00
vangel76 c3865b5dbd Merge upstream/main into perf/calc-spill-index-2683 2026-10-29 23:07:21 +01:00
vangel76 95c3bb9b89 Fix clippy complexity after merging upstream/main 2026-10-29 23:05:54 +01:00
vangel76 72e959f117 Merge upstream/main into fix/chart-work-budgets 2026-10-29 23:04:21 +01:00
vangel76 bc7e37fa7d Merge upstream/main into fix/chart-combo-logic-cleanup 2026-10-29 23:03:18 +01:00
vangel76andClaude Sonnet 5.5 8b330ac9e2 Tighten selection comment and test index arithmetic
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 00:52:59 +01:00
vangel76andClaude Sonnet 5.5 b6bbdcd5cf calc: build precedents from a borrowed syntax tree, no Arc for text-only formulas
Formula::with_expr runs a closure on the cached tree, or on a freshly parsed
one that is dropped afterwards, so rebuild and cells_changed need neither a
tree copy nor an Arc allocation / reference count.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 00:51:45 +01:00
vangel76andClaude Sonnet 5.5 4523b817ce Read already-sorted input directly instead of selecting
A full sort recognises ascending and descending runs in O(n), which a
selection cannot beat, so sorted, reverse-sorted and tie-block input was
up to 1.8x slower than before. Check for a monotone run first (one pass
that stops at the first out-of-order pair) and index into it. Extend the
bit-for-bit oracle test with ascending, descending and nearly ascending
data.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 00:51:12 +01:00
vangel76 d71cf10e70 Merge remote-tracking branch 'upstream/main' into perf/calc-expr-arc-2685 2026-10-29 00:48:45 +01:00
vangel76 b7c5a69dbd Merge remote-tracking branch 'upstream/main' into perf/stat-select-2679 2026-10-29 00:47:31 +01:00
vangel76andClaude Sonnet 5.5 f404a08357 Merge upstream/main into perf/calc-range-memo-2681
eval.rs and recalc.rs were split into submodules; re-applied the range_sum hunks in eval.rs, eval/call.rs and recalc/pass_host.rs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 00:54:49 +01:00
vangel76andClaude Sonnet 5.5 a434efabf0 Merge upstream/main into perf/stat-select-2679
stat.rs was split into stat/; re-applied the selection changes in stat/descriptive.rs and moved the oracle test to stat/tests.rs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-28 00:53:37 +01:00
vangel76andClaude Sonnet 5.5 2fe8a33c5d Merge upstream/main into fix/chart-combo-logic-cleanup
render.rs was split into render/ modules on main; the plot-kind and fixed-scale cleanups are re-applied in the new modules.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 19:52:57 +01:00
vangel76andClaude Sonnet 5.5 2f36fe1b34 Merge upstream/main into perf/chart-measure-allocs
render.rs was split into render/ modules on main; the label-fitting, legend and treemap changes are re-applied in the new modules.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 19:51:31 +01:00
vangel76andClaude Sonnet 5.5 5144ebe1de Merge upstream/main into fix/chart-work-budgets
render.rs was split into render/ modules on main; the budget and hairline-bar changes are re-applied in render.rs and render/cartesian.rs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 19:47:54 +01:00
vangel76andClaude Sonnet 5.5 5f3fcc8140 perf(calc): look up #SPILL! cells by row and column range, not by row band
The spill-error set was ranged by rows only and filtered by column afterwards, so an edit in
column A walked every #SPILL! cell to its right in the same rows. Each row now keeps its own
column set and a window lookup takes one column range per row that has an entry. Results are
unchanged. Also covers an array spilling right over the cleared cell.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 19:45:28 +01:00
vangel76andClaude Sonnet 5.5 356ccdcf1a perf(calc): index spills for a paste only when registering them costs less than testing them
A full-height column spill takes 65,536 blocks to register, which a 17-cell paste elsewhere
never needed. A sheet's spill areas are now indexed only when their block count is below what
testing each area against each changed cell would cost; otherwise the plain containment check
runs, as before. Results are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 19:43:47 +01:00
vangel76 0b7f674925 Merge upstream/main 2026-10-27 19:40:32 +01:00
vangel76andClaude Sonnet 5.5 72236e4aab perf(calc): find the formulas a cleared cell may unblock from a set of #SPILL! cells (#2684)
Clearing a cell walked every formula in the graph, and each changed cell
scanned a 64 x 64 window of stored cells, to find formulas showing
#SPILL!. Calc now keeps the set of cells whose value is #SPILL! (filled
in rebuild, kept up to date by cells_changed and the write-back of a
run) and takes those candidates from it. The same set replaces the walk
over the graph in run for areas a formula stopped spilling over. Each
entry is checked against the workbook before it is used.

Fixes #2684

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 22:26:15 +01:00
vangel76andClaude Sonnet 5.5 6c37a63704 perf(calc): index the pass's spill areas so reads don't scan every spill (#2683)
A cell read and a range read looked at every spill computed so far in
the pass, and cells_changed looked at every spill range of the sheet for
each changed cell. A spatial index (16 x 16 cell blocks) now finds the
areas covering a cell or meeting a range. Where areas overlap, the one
that counted before (first/last in the pass's spill map order) still
counts.

Fixes #2683

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 22:22:22 +01:00
vangel76andClaude Sonnet 5.5 c567511cf1 calc: rebuild the dependency graph from shared syntax trees
Calc::rebuild and Calc::cells_changed called Formula::expr(), which deep-copies each formula's
syntax tree only to read its references. Use expr_arc() (already used when formulas are
calculated), which hands out the cached Arc without copying.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 22:21:15 +01:00
vangel76andClaude Sonnet 5.5 96bfba93f8 perf(calc): remember SUM/COUNT/AVERAGE of constant-only blocks per pass (#2681)
A block that holds only constants can't change during a pass, so its sum
and count are worked out once and reused by every formula that reads the
same block. A block that is the same rows from the top but longer (a
running total) continues the shorter block's sum, which is the same
additions in the same order as one walk from the start.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 22:17:29 +01:00
vangel76andClaude Sonnet 5.5 890870cd57 perf: select instead of sort in MEDIAN, PERCENTILE, QUARTILE, LARGE, SMALL
Read the needed order statistics with select_nth_unstable_by(total_cmp)
instead of a full sort. Results are bit-identical; a test compares them
against the previous full-sort code on randomised data.

Fixes #2679

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 22:11:08 +01:00
vangel76andClaude Sonnet 5.5 ee13457b2f Merge upstream/main into fix/chart-combo-logic-cleanup
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 23:55:05 +01:00
vangel76andClaude Sonnet 5.5 8ae680547a Merge upstream/main into fix/chart-work-budgets
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 23:54:05 +01:00
vangel76andClaude Sonnet 5.5 8f34c42e88 Merge upstream/main into fix/chart-combo-logic-cleanup
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 10:22:03 +01:00
vangel76andClaude Sonnet 5.5 6c32845740 Merge upstream/main into fix/chart-work-budgets
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 10:19:09 +01:00
vangel76andClaude Opus 5.5 a34e109451 Merge upstream/main into perf/chart-measure-allocs
Also removes conflict markers committed to ROADMAP.md by the previous merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:53:52 +02:00
vangel76 c0d0d909fc Merge remote-tracking branch 'upstream/main' into fix/chart-work-budgets
# Conflicts:
#	ROADMAP.md
#	crates/chart/src/resolve.rs
2026-10-22 19:43:16 +02:00
vangel76 562004f198 Merge remote-tracking branch 'upstream/main' into perf/chart-measure-allocs
# Conflicts:
#	ROADMAP.md
2026-10-22 19:35:18 +02:00
vangel76 7f8ac83dc3 Merge remote-tracking branch 'upstream/main' into fix/chart-combo-logic-cleanup
# Conflicts:
#	ROADMAP.md
#	crates/chart/src/tests.rs
2026-10-22 19:34:40 +02:00
vangel76andClaude Sonnet 5.5 4a1e6de303 Charts: work and memory are bounded by series x points; ranges are clipped before evaluation (#1147)
A chart keeps at most 2,000,000 series x points (every series and the
categories keep their first points), 16 trendlines per chart; columns and
bars narrower than 0.25 px are merged per pixel column; series references
are cut after the rows that can matter before calc evaluates them; the
moving average is linear; fit_trendline allocates no Vec per point;
render borrows clean data instead of cloning it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-13 21:21:24 +02:00
vangel76andClaude Sonnet 5.5 79cfd9b151 Charts: label fitting and legend measuring avoid hundreds of text layouts per render (#1149)
Binary search in Ctx::fit, early exit of the side-legend column measure,
no per-point Vecs in value_range/squarify, treemap carries the value.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-13 21:14:02 +02:00
vangel76andClaude Sonnet 5.5 5cf0e24f43 Charts: one combo plot-kind rule, no fixed_scale helper, ticks off the step grid (#1146)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-13 21:12:43 +02:00
171 changed files with 12082 additions and 927 deletions
+1
View File
@@ -12,6 +12,7 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra
| `epaint_default_fonts::HACK_REGULAR` (dependency font input) | Source Foundry Authors (Hack; DejaVu and Bitstream Vera foundations) | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Hack](https://github.com/source-foundry/Hack) | MIT + Bitstream Vera License (full upstream texts in `NOTICE`) | Hack-Regular.ttf is egui's bundled monospace face and GridCraft's last-resort symbol fallback. Dependency asset unmodified here, no font file copied into this repository. |
| `epaint_default_fonts::NOTO_EMOJI_REGULAR` (dependency font input) | The Noto Project Authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Noto Emoji](https://fonts.google.com/noto/specimen/Noto+Emoji) | SIL Open Font License 1.1 (full upstream text in `NOTICE`) | NotoEmoji-Regular.ttf is enabled by egui's `monochrome_emoji_fonts` feature and used as the monochrome emoji fallback. Dependency asset unmodified here, no font file copied into this repository. |
| `epaint_default_fonts::UBUNTU_LIGHT` (dependency font input) | Canonical Ltd. and the Ubuntu Font Family authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Ubuntu font](https://design.ubuntu.com/font) | Ubuntu Font Licence 1.0 (full upstream text in `NOTICE`) | Ubuntu-Light.ttf is egui's default proportional face. Dependency asset unmodified here, no font file copied into this repository. |
| Regional format data in `crates/core/src/region.rs` (data, no file of its own) | Unicode, Inc. (Unicode CLDR) | [cldr-json](https://github.com/unicode-org/cldr-json) (`cldr-numbers-full`, `cldr-dates-full`) and ICU's CLDR 48 data | Unicode License v3 (`Unicode-3.0`; full text in `NOTICE`) | Separators, month and weekday names, AM/PM words, currency symbols and date order of the built-in regions. No CLDR file is copied into this repository. |
| `docs/brand/artcraft-logo.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms (`docs/brand/LICENSE-brand.txt`) | Trademark, not open source |
| `docs/brand/artcraft-logo.png` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark |
| `docs/brand/artcraft-logo-white.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark |
Generated
+39 -27
View File
@@ -910,12 +910,12 @@ dependencies = [
[[package]]
name = "craft-fonts-build"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
[[package]]
name = "craft-i18n"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
dependencies = [
"fixed_decimal",
"icu_decimal",
@@ -938,7 +938,7 @@ dependencies = [
[[package]]
name = "craft-print"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
dependencies = [
"craft-winprint-gdi",
"log",
@@ -952,7 +952,7 @@ dependencies = [
[[package]]
name = "craft-ui"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
dependencies = [
"craft-fonts-build",
"craft-i18n",
@@ -972,7 +972,7 @@ dependencies = [
[[package]]
name = "craft-update"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
dependencies = [
"chrono",
"dunce",
@@ -995,10 +995,21 @@ dependencies = [
"zip 9.1.0",
]
[[package]]
name = "craft-web-api"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=fd5e2feff6ea25bed371a99cc732b3968d621505#fd5e2feff6ea25bed371a99cc732b3968d621505"
dependencies = [
"js-sys",
"serde_json",
"wasm-bindgen",
"wasm-bindgen-futures",
]
[[package]]
name = "craft-web-host"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=2e2312ddaf21bc3d85cd3395d0955c7d239306f6#2e2312ddaf21bc3d85cd3395d0955c7d239306f6"
source = "git+https://github.com/storytold/craft-libs?rev=fd5e2feff6ea25bed371a99cc732b3968d621505#fd5e2feff6ea25bed371a99cc732b3968d621505"
dependencies = [
"wasm-bindgen",
"web-sys",
@@ -1007,7 +1018,7 @@ dependencies = [
[[package]]
name = "craft-window"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=c5a1cbfb5f9049f9270955caffb5bee5997aebf7#c5a1cbfb5f9049f9270955caffb5bee5997aebf7"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
dependencies = [
"egui",
"objc2 0.6.5",
@@ -1018,7 +1029,7 @@ dependencies = [
[[package]]
name = "craft-winprint-gdi"
version = "0.1.0"
source = "git+https://github.com/storytold/winprint.rs?rev=417ac3c4d14d3da5485a99c7d5ce8f05664ec312#417ac3c4d14d3da5485a99c7d5ce8f05664ec312"
source = "git+https://github.com/storytold/winprint.rs?rev=64e7afd4d1483512385012c340ecaaaf0f3beb7f#64e7afd4d1483512385012c340ecaaaf0f3beb7f"
dependencies = [
"windows-sys 0.61.2",
]
@@ -2011,7 +2022,7 @@ dependencies = [
[[package]]
name = "gridcraft"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"arboard",
"chrono",
@@ -2038,7 +2049,7 @@ dependencies = [
[[package]]
name = "gridcraft-calc"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-core",
"gridcraft-formula",
@@ -2050,7 +2061,7 @@ dependencies = [
[[package]]
name = "gridcraft-chart"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-calc",
"gridcraft-core",
@@ -2061,7 +2072,7 @@ dependencies = [
[[package]]
name = "gridcraft-cli"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-engine",
"gridcraft-mcp",
@@ -2073,7 +2084,7 @@ dependencies = [
[[package]]
name = "gridcraft-core"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"chrono",
"js-sys",
@@ -2084,7 +2095,7 @@ dependencies = [
[[package]]
name = "gridcraft-engine"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-calc",
"gridcraft-chart",
@@ -2107,7 +2118,7 @@ dependencies = [
[[package]]
name = "gridcraft-formula"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-core",
"serde",
@@ -2116,7 +2127,7 @@ dependencies = [
[[package]]
name = "gridcraft-functions"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-core",
"regex",
@@ -2124,7 +2135,7 @@ dependencies = [
[[package]]
name = "gridcraft-math"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"ratex-font",
"ratex-layout",
@@ -2134,7 +2145,7 @@ dependencies = [
[[package]]
name = "gridcraft-mcp"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-engine",
"log",
@@ -2143,7 +2154,7 @@ dependencies = [
[[package]]
name = "gridcraft-model"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-core",
"gridcraft-formula",
@@ -2156,18 +2167,18 @@ dependencies = [
[[package]]
name = "gridcraft-numfmt"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-core",
]
[[package]]
name = "gridcraft-pdf"
version = "0.17.0"
version = "0.16.0"
[[package]]
name = "gridcraft-ui-egui"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"craft-ui",
"craft-update",
@@ -2186,6 +2197,7 @@ dependencies = [
"skrifa",
"sys-locale",
"toml",
"unicode-bidi",
"web-sys",
"web-time",
"zbus",
@@ -2193,8 +2205,9 @@ dependencies = [
[[package]]
name = "gridcraft-web"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"craft-web-api",
"craft-web-host",
"eframe",
"egui",
@@ -2211,7 +2224,7 @@ dependencies = [
[[package]]
name = "gridcraft-xlsx"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"aes",
"gridcraft-core",
@@ -6473,7 +6486,7 @@ dependencies = [
[[package]]
name = "winprint"
version = "0.2.1"
source = "git+https://github.com/storytold/winprint.rs?rev=5f80b9baf6b58937c401656634e42d33f00ddf49#5f80b9baf6b58937c401656634e42d33f00ddf49"
source = "git+https://github.com/storytold/winprint.rs?rev=51934db382f225454cf152a8657eb03789be63c4#51934db382f225454cf152a8657eb03789be63c4"
dependencies = [
"flate2",
"fmt-derive",
@@ -6662,7 +6675,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.17.0"
version = "0.16.0"
dependencies = [
"gridcraft-engine",
"serde_json",
@@ -6923,7 +6936,6 @@ dependencies = [
"nix",
"typed-path",
"windows-sys 0.61.2",
"zopfli",
]
[[package]]
+4 -4
View File
@@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
default-members = ["crates/*", "apps/gridcraft", "apps/gridcraft-cli", "xtask"]
[workspace.package]
version = "0.17.0"
version = "0.16.0"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.95"
@@ -26,9 +26,9 @@ gridcraft-mcp = { path = "crates/mcp" }
gridcraft-pdf = { path = "crates/pdf" }
gridcraft-math = { path = "crates/math" }
# Every craft-libs crate (craft-update and craft-window here, craft-ui in ui-egui, craft-print in
# the app) comes from one storytold/craft-libs revision.
# the app) comes from one craft-libs revision.
# craft-update: app-specific pretend versions apply to startup and manual checks.
craft-update = { git = "https://github.com/storytold/craft-libs", rev = "c5a1cbfb5f9049f9270955caffb5bee5997aebf7", version = "=0.1.0", features = ["egui"] }
craft-update = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d", version = "=0.1.0", features = ["egui"] }
# foundation
thiserror = "2"
@@ -71,7 +71,7 @@ skrifa = { version = "0.44", default-features = false, features = ["std"] }
epaint_default_fonts = "0.37"
# One native window per workbook: the window list, Window menu titles and the immediate-viewport
# driver. Same craft-libs revision as craft-update above.
craft-window = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "c5a1cbfb5f9049f9270955caffb5bee5997aebf7", default-features = false, features = ["egui"] }
craft-window = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d", default-features = false, features = ["egui"] }
[workspace.lints.rust]
unsafe_code = "forbid"
+49
View File
@@ -275,6 +275,55 @@ DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER
DEALINGS IN THE FONT SOFTWARE.
The regional format data in crates/core/src/region.rs (separators, month and weekday names, AM/PM
words, currency symbols and date order) follows the Unicode Common Locale Data Repository (CLDR),
read from unicode-org/cldr-json and from ICU's CLDR 48 data, under the Unicode License v3.
Source: https://github.com/unicode-org/cldr-json
The following text is reproduced verbatim from cldr-json's LICENSE:
UNICODE LICENSE V3
COPYRIGHT AND PERMISSION NOTICE
Copyright © 2015-2024 Unicode, Inc.
NOTICE TO USER: Carefully read the following legal agreement. BY
DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR
SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT
DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE.
Permission is hereby granted, free of charge, to any person obtaining a
copy of data files and any associated documentation (the "Data Files") or
software and any associated documentation (the "Software") to deal in the
Data Files or Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, and/or sell
copies of the Data Files or Software, and to permit persons to whom the
Data Files or Software are furnished to do so, provided that either (a)
this copyright and permission notice appear with all copies of the Data
Files or Software, or (b) this copyright and permission notice appear in
associated Documentation.
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
THIRD PARTY RIGHTS.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE
BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES,
OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA
FILES OR SOFTWARE.
Except as contained in this notice, the name of a copyright holder shall
not be used in advertising or otherwise to promote the sale, use or other
dealings in these Data Files or Software without prior written
authorization of the copyright holder.
SPDX-License-Identifier: Unicode-3.0
Linux Wayland window decorations
-------------------------------
sctk-adwaita 0.10.1 (https://github.com/PolyMeilex/sctk-adwaita,
+15 -4
View File
@@ -2,7 +2,7 @@
**Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away
> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (form controls: buttons, sliders and spin buttons; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis take ranges by selection on the grid; earlier: Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365)
> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-05 · **Change:** minor (CJK fallback text shares the Latin baseline; earlier: form controls: buttons, sliders and spin buttons; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis take ranges by selection on the grid; earlier: Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365)
GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX
is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web.
@@ -43,8 +43,7 @@ long `.xls` track, so about half parallelizes (≈30–50 wall-clock hours with
Why alpha: all six core workflows in the [alpha gate](docs/roadmap.md#alpha-gate) pass end to end on macOS with save and reopen, and the core workflow works end to end (open an XLSX, enter formulas and data, format, sort
and filter, chart, pivot, print to PDF, save XLSX), but users still hit blocking gaps on real
files: 100 MB workbooks don't open (#175), there's no `.xls` (#121), Arabic and Devanagari text
isn't shaped (Chinese text still reported in #142), charts lose their type and options on round trip, and
files: 100 MB workbooks don't open (#175), there's no `.xls` (#121), Devanagari text needs shaping verification (Chinese text still reported in #142), charts lose their type and options on round trip, and
nobody has yet confirmed that Excel opens our output without a repair prompt across a real
corpus. Beta needs those closed (see [beta gates](docs/roadmap.md#beta-gates)).
@@ -105,7 +104,7 @@ Localized German formulas and numbers are in PR #163. Detail: [localization-pari
| Simplified Chinese | zh-Hans | menus only | ~12% |
| Spanish | es | none | 0% |
| Hindi | hi | none | 0% |
| Arabic | ar | none (no RTL layout, no shaping) | 0% |
| Arabic | ar | catalog pending; shared shaping/bidi works | 0% |
| French | fr | none | 0% |
| Portuguese | pt-BR | menus, File view, dialogs | ~43% |
| Indonesian | id | none | 0% |
@@ -135,6 +134,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| Date | What landed |
|---|---|
| 2026-11-06 | A sheet keeps its own scroll position when sheets are inserted, deleted or moved before it: a new PivotTable or inserted sheet opens at the top instead of where its neighbour was scrolled; a closed window's positions are let go (#3485) |
| 2026-11-02 | Form controls: Insert › Controls places buttons, sliders (scroll bars) and spin buttons on the sheet. A slider or spin button holds a whole number from 0 to 30000 in a linked cell (a drag or a held arrow is one undo step, and editing the cell moves the control); a button runs a macro or one command through the macro policy, asking first in a workbook opened from a file; Assign Action and Format Control edit them, and XLSX files read and write them (control properties, `<controls>` and VML), alongside the form controls GridCraft keeps unchanged |
| 2026-11-02 | Dialogs take ranges by selection on the grid: reference boxes (RefEdit) with a collapse button that shrinks the dialog while the person selects, in Goal Seek, Solver (including the Model Range of Load/Save) and Data Analysis; while collapsed, the arrow keys move the selection and Shift+arrow extends it, and F4 cycles a reference's anchors |
| 2026-11-02 | Python mode in the formula bar: Ctrl+Alt+Shift+P, `=PY` then Tab, or editing a Python cell edits its code in the formula bar, with Enter for a new line, Tab to indent and Ctrl+Enter to commit through `python.set`. The code still doesn't run |
@@ -148,6 +148,9 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-31 | CSV and text open detects each column's date order: a value that reads only day first (`31/05/2026`) makes the column day first, a value that reads only month first (`12/31/2026`) makes it month first, an all-ambiguous column keeps month first with a warning, and a column with both kinds stays text with a warning (#2765) |
| 2026-10-31 | Recently Used lists the functions most recently entered in a formula from any source (ribbon, control channel, MCP, scripts), kept with the preferences and returned by `formulas.recentlyUsed`; the Insert Function dialog takes Excel's title and a working category list (Most Recently Used, All, each category) |
| 2026-10-31 | Formulas › Function Library as a big Insert Function button, the AutoSum split button and three columns of labelled small buttons, each category book in its own colour; category menus end with Insert Function… on that category; More Functions cascades into the remaining categories; Home › Editing shares the AutoSum menu; the MR key tip opens Recently Used |
| 2026-10-31 | System and embedded CJK fallback text shares the Latin baseline in cells, editors and window titles (#1265, #2621). Symbol and emoji alignment, manual row heights and clipping are unchanged |
| 2026-10-26 | Arabic bidi display/editing preserve logical source text; per-cell reading direction is editable and survives undo, XLSX and conditional-format priority |
| 2026-10-26 | Arabic text shows in cells and the interface through a system, fontconfig or craft-fonts Arabic face; workbook fonts keep their script fallbacks |
| 2026-10-25 | AutoSum over a selection does what Excel does: a selected row gets one total to its right (in its last cell if that is empty), not a one-cell total under each cell; a block whose last column is empty gets each row's total there, one whose last row is empty each column's, and with both the corner gets the total of the block; otherwise totals go under each column as before |
| 2026-10-24 | Macro interface ([macros.md](docs/macros.md#in-the-window)): an Automate › Macros group (Record Macro, which turns into Stop Recording, Use Relative References, Run Macro with recent macros); "Recording · N steps" with Stop in the status bar; a Record Macro dialog (checked name, free Ctrl+Shift shortcut, description); the Macros dialog lists macros beside Automate scripts and a VBA project, with Run, Step Into, Edit, Create, Delete and Options; Ctrl+Shift+letter macro shortcuts; questions that offer Run, Overwrite, Shift Down or Shift Right; a step editor with a breakpoint gutter, drag to reorder, a command picker, reference boxes that switch between relative tokens and absolute references, Inputs and Output tabs, a debug bar (Check, Run, Step, Continue, Run to Here, Stop with keep or undo) and grid overlays for inputs, writes, conflicts and the current step |
| 2026-10-24 | Go To lists the named ranges of every sheet (`Sheet2!Name` for a name scoped to another sheet; hidden names, constants and formulas left out) and the references entered before, a double-click goes to an entry, and the Reference box takes several names or references separated by commas (`Price, Type`, `B14:C22,F19:G30`) |
@@ -186,6 +189,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-14 | Recalculation: a dynamic array's anchor given a new formula that reads its old spill area (`A1` was `=SEQUENCE(3)`, becomes `=SUM(A2:A3)+1` or `=SEQUENCE(1,A2+1)`) sees that area empty instead of the old values |
| 2026-10-14 | Recalculation: when two arrays of one recalculation want the same empty cells, the formulas that read the one laid out second (at its anchor or over the cells it would have spilled into) read `#SPILL!` and empty cells instead of its values |
| 2026-10-14 | Border keytips: each Alt, H, B letter applies the preset Excel's Borders menu gives it (H,B,O bottom, H,B,B bottom double, H,B,N no border, H,B,A all borders and the rest); they all drew a bottom border before. H,B,M opens Format Cells on the Border tab |
| 2026-10-13 | Charts: label shortening needs about nine text measurements instead of up to 400, side legends stop measuring once the column is at its cap and treemap layout avoids per-item allocation (#1149) |
| 2026-10-13 | Formula-entry Command/Ctrl-click inserts another reference with the locale's argument separator; Shift-click extends from a stable, sheet-aware anchor. Pointer tests cover repeated clicks, reversed ranges, ordinary replacement, typed separators, Spanish syntax, sheet switches and keyboard continuation. Readiness estimates unchanged. |
| 2026-10-13 | `formulas.evaluate` (`gridcraft-cli eval`, MCP `evaluate_formula`) refuses what entering the formula in a cell refuses, with the reason (`=SUM(1;2)`: "expected `,` or `)`"), instead of `#NAME?` for any formula that doesn't parse |
| 2026-10-13 | Colour menus: Fill Color, Font Color and the Format Cells pickers end with Recent Colors (the last ten custom colours) and More Colors… (hue/saturation area, RGB, hex box, preview, OK/Cancel) (#928) |
@@ -196,6 +200,8 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-13 | Conditional formatting: a rule gathers range statistics only when it reads them (top/bottom, above average, duplicates, scales, bars, icons) and borrows them instead of copying them per cell, so the per-frame lookup of an expression or cell-value rule no longer visits its whole range (#1098, part 1) |
| 2026-10-12 | Next/previous sheet navigation stops at the first/last visible worksheet instead of activating a hidden boundary sheet; the current selection is retained |
| 2026-10-21 | Macro engine ([macros.md](docs/macros.md)): typed, declarative macros saved in the workbook (a JSON field; hidden names in xlsx); declared inputs checked with messages that name the cells; tokens (`Input.col(2)`, `Row`, `Output`, `Name:Total`, `{{…}}` in formulas); one undo step with full rollback; a footprint dry run with ask, fail, overwrite, shift down or shift right; a policy that keeps macros away from files and the app, with approval for macros in opened files; Repeat, If and Stop steps; a debugger; a relative recorder; `macro.*` commands beside the unchanged Automate scripts |
| 2026-10-13 | Charts: work and memory are bounded by series x points (2,000,000 per chart: every series keeps its first points; 16 trendlines per chart), hairline columns and bars are merged per pixel column, series ranges are clipped before they are evaluated, the moving average is linear and rendering clean data no longer copies it (#1147) |
| 2026-10-13 | Charts: one combo plot-kind rule for resolving and drawing; ticks of a scale off the step grid keep their own values (#1146) |
| 2026-10-12 | Security: the `--control` server requires an auth token (random, kept in a 0600 file or chosen with `GRIDCRAFT_CONTROL_TOKEN`) on each connection's first line and closes bad connections (#777) |
| 2026-10-12 | Rows auto-fit oversized fonts: opening a file and whole-row/column formatting grow non-custom rows to their tallest font (#833) |
| 2026-10-12 | Number formats: parsing a format code is linear in its length (long runs of `[`, `,` or `/` no longer take quadratic time), and an unclosed `[` is a literal when splitting sections as well as when tokenizing (#810) |
@@ -259,6 +265,8 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| Date | Change | Summary |
|---|---|---|
| 2026-11-06 | trivial | Progress log: scroll positions follow their sheet through sheet inserts, deletes and moves (#3485) |
| 2026-11-05 | minor | Reconcile upstream form-control progress with the CJK baseline update |
| 2026-11-02 | minor | Form controls; catalog 277 / 292 → 281 / 296 (Developer › Controls); progress log row |
| 2026-11-02 | minor | Progress log: RefEdit reference boxes in dialogs |
| 2026-11-02 | minor | Progress log: Python mode in the formula bar |
@@ -271,6 +279,9 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-31 | minor | File-page text fields keep the view open when Esc cancels an edit (#3035) |
| 2026-10-31 | minor | Progress log: Recently Used and Insert Function categories |
| 2026-10-31 | minor | Progress log: Function Library |
| 2026-10-31 | minor | Progress log: shared CJK/Latin baselines in cells, editors and window titles |
| 2026-10-26 | minor | Progress log: Arabic bidi display and editing |
| 2026-10-26 | minor | Progress log: Arabic font fallback |
| 2026-10-25 | minor | Progress log: AutoSum over a selected row, and a selection's empty last row and column |
| 2026-10-24 | minor | Macro interface on the Automate tab; progress log row |
| 2026-10-24 | minor | Macro engine and `macro.*` commands; catalog 275 / 291 → 276 / 292 (View › Macros › Record Macro); progress log row |
+3 -1
View File
@@ -17,7 +17,9 @@ path = "src/main.rs"
gridcraft-engine = { workspace = true }
gridcraft-ui-egui = { workspace = true }
# The host messages (storytold/craft-libs): one contract and one bridge for the web builds.
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "2e2312ddaf21bc3d85cd3395d0955c7d239306f6", version = "0.1.0" }
# The page command API (storytold/craft-libs#413): window.gridcraft over the control channel.
craft-web-api = { git = "https://github.com/storytold/craft-libs", rev = "fd5e2feff6ea25bed371a99cc732b3968d621505", version = "0.1.0" }
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "fd5e2feff6ea25bed371a99cc732b3968d621505", version = "0.1.0" }
egui = { workspace = true }
log = { workspace = true }
serde_json = { workspace = true }
+151
View File
@@ -0,0 +1,151 @@
//! `window.gridcraft`: the page command API (`craft-web-api`, storytold/craft-libs#413) over the
//! app's own control channel, the one the desktop control server drives (`docs/control-protocol.md`).
//!
//! The crate owns the JavaScript side (`request`, `execute`, `commands`, `docInspect`, `uiInspect`,
//! `screenshot`, as promises with a timeout). This module is GridCraft's part: a [`Request`] that
//! queues a [`ControlRequest`] for the app, which answers it on its next frame, and waits for the
//! reply. A call the page gave up on is released ([`Pending`]), so the app skips it if it has not started it, instead of
//! applying a command the page was told had failed.
use std::sync::Arc;
use std::sync::atomic::AtomicU8;
use std::sync::mpsc::{Receiver, TryRecvError};
use gridcraft_ui_egui::ControlRequest;
use serde_json::{Value, json};
/// How often a waiting call looks for the app's reply, in milliseconds (about a frame).
#[cfg(target_arch = "wasm32")]
const POLL_MS: i32 = 16;
/// A queued control request whose reply has not arrived. Dropping it before the reply (the page's
/// timeout fired, or the call was abandoned) releases the request: the app skips it if it has not started it (one it has taken still finishes).
struct Pending {
reply: Receiver<Value>,
state: Arc<AtomicU8>,
answered: bool,
}
impl Pending {
/// `Some` once the app has answered or gone away; `None` while the request is still waiting.
fn poll(&mut self) -> Option<Value> {
match self.reply.try_recv() {
Ok(reply) => {
self.answered = true;
Some(reply)
}
Err(TryRecvError::Empty) => None,
Err(TryRecvError::Disconnected) => {
self.answered = true;
Some(json!({"ok": false, "error": "the app dropped the request"}))
}
}
}
}
impl Drop for Pending {
fn drop(&mut self) {
if !self.answered {
ControlRequest::abandon(&self.state);
}
}
}
#[cfg(target_arch = "wasm32")]
pub use web::install;
#[cfg(target_arch = "wasm32")]
mod web {
use std::rc::Rc;
use std::sync::mpsc::channel;
use craft_web_api::methods::Methods;
use craft_web_api::page::{PageApi, Request};
use gridcraft_ui_egui::{ControlRequest, SheetApp};
use serde_json::json;
use wasm_bindgen_futures::JsFuture;
use super::{POLL_MS, Pending};
/// The app id in `window.<app>`.
const APP: &str = "gridcraft";
/// Give the app its control queue and put `window.gridcraft` on the page. An existing
/// `window.gridcraft` keeps the functions it already has.
pub fn install(app: &mut SheetApp, ctx: egui::Context) {
let (queue, requests) = channel::<ControlRequest>();
app.control_rx = Some(requests);
let request: Request = Rc::new(move |method, params| {
let (control, reply) = ControlRequest::new(method, params);
let mut pending = Pending { reply, state: control.state.clone(), answered: false };
if queue.send(control).is_err() {
pending.answered = true;
return Box::pin(async { json!({"ok": false, "error": "the app is not running"}) });
}
ctx.request_repaint();
Box::pin(async move {
loop {
if let Some(reply) = pending.poll() {
return reply;
}
sleep(POLL_MS).await;
}
})
});
if let Err(e) = Rc::new(PageApi::new(APP, Methods::default(), request)).install() {
log::warn!("couldn't install window.{APP}: {e:?}");
}
}
async fn sleep(ms: i32) {
let promise = js_sys::Promise::new(&mut |resolve, _| {
if let Some(window) = web_sys::window() {
let _ = window.set_timeout_with_callback_and_timeout_and_arguments_0(&resolve, ms);
}
});
let _ = JsFuture::from(promise).await;
}
}
#[cfg(test)]
mod tests {
use std::sync::atomic::Ordering;
use super::*;
/// `ControlRequest`'s `state` values (`gridcraft_ui_egui::control`): the app has started it, or its
/// sender gave up.
const TAKEN: u8 = 1;
const ABANDONED: u8 = 2;
fn pending() -> (Pending, ControlRequest) {
let (control, reply) = ControlRequest::new("document.inspect", json!({}));
(Pending { reply, state: control.state.clone(), answered: false }, control)
}
#[test]
fn a_call_the_page_gave_up_on_is_released_so_the_app_skips_it() {
let (waiting, control) = pending();
drop(waiting);
assert_eq!(control.state.load(Ordering::SeqCst), ABANDONED);
assert!(!control.take(), "the app must not run a request the page was told had failed");
}
#[test]
fn an_answered_call_is_not_released() {
let (mut waiting, control) = pending();
assert!(control.take());
control.reply.send(json!({"ok": true, "result": 1})).unwrap_or_default();
assert_eq!(waiting.poll(), Some(json!({"ok": true, "result": 1})));
drop(waiting);
assert_eq!(control.state.load(Ordering::SeqCst), TAKEN);
}
#[test]
fn a_request_the_app_never_runs_waits_until_it_is_dropped() {
let (mut waiting, control) = pending();
assert_eq!(waiting.poll(), None);
drop(control);
assert_eq!(waiting.poll().and_then(|r| r["ok"].as_bool()), Some(false), "the app went away");
}
}
+5
View File
@@ -13,11 +13,16 @@
//! workbook to open and stores what GridCraft saves (`host`). A page can host GridCraft in its
//! own document (no iframe) with `data-host="page"` on the canvas, talking over DOM `CustomEvent`s
//! on the canvas. `&author=` names the user for notes and comments in either host mode.
//! `window.gridcraft` (`craft-web-api`) is the page command API in every mode: `request`,
//! `execute`, `commands`, `docInspect`, `uiInspect` and `screenshot` over the control methods
//! (`api`).
//! `?chrome=none|notitlebar|nofiletab` leaves the title bar and/or the File tab undrawn for a host
//! that embeds the sheet in its own window (see `Chrome`). Preferences (theme, interface size,
//! editing options…) are kept in `localStorage` (`prefs`).
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
#[cfg(any(target_arch = "wasm32", test))]
mod api;
#[cfg(target_arch = "wasm32")]
mod host;
#[cfg(any(target_arch = "wasm32", test))]
+1
View File
@@ -99,6 +99,7 @@ pub fn start() {
}
bridge.post_ready();
}
crate::api::install(&mut app, cc.egui_ctx.clone());
Ok(Box::new(WebShell { app, inbox, bridge: app_bridge, prefs }))
}),
)
+1 -1
View File
@@ -34,7 +34,7 @@ winit = { version = "0.30", default-features = false, features = ["x11", "waylan
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
# Same craft-libs revision as the workspace's craft-update and craft-window (see the root
# Cargo.toml).
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "c5a1cbfb5f9049f9270955caffb5bee5997aebf7" }
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d" }
craft-update = { workspace = true }
rfd = { workspace = true }
arboard = { workspace = true, features = ["image-data"] }
+3
View File
@@ -64,6 +64,9 @@ impl eframe::App for App {
fn on_exit(&mut self) {
print::finish_pending_jobs();
save_prefs(&self.0);
// A ready update installs once the process has exited. The macOS exit below skips
// dropping the app, so the helper is started here.
self.0.start_pending_update();
// macOS (#117): end the process here, before eframe tears the window down. AccessKit's
// macOS adapter subclasses the window's content view and restores its class on teardown;
// AppKit's Touch Bar support observes that view (KVO), so in the next display cycle
+5 -1
View File
@@ -33,7 +33,8 @@ sync_github_name = false
[people.vangel76]
real_name = "Valentin Klein"
sync_github_name = true
display_name = "vangel76"
sync_github_name = false
[people.MacJedi42]
sync_github_name = true
@@ -109,3 +110,6 @@ sync_github_name = true
[people.maxm11]
sync_github_name = true
[people.RockerHX]
sync_github_name = true
+11
View File
@@ -115,6 +115,12 @@ pub trait Host {
fn visit_range(&mut self, _sheet: usize, _range: RangeRef, _f: &mut dyn FnMut(&Value) -> bool) -> bool {
false
}
/// `(sum, count)` of the numbers in a block that holds nothing but constants (no formula,
/// error or spilled value), added row-major from `0.0` exactly as `visit_range` would hand
/// them over. `None`: the block isn't like that, or this host can't tell.
fn range_sum(&mut self, _sheet: usize, _range: RangeRef) -> Option<(f64, u64)> {
None
}
/// While a formula is calculated on demand inside another one: `stack_position()` where the
/// outermost one started, which the stack budget is counted from.
fn stack_top(&self) -> Option<usize> {
@@ -537,6 +543,11 @@ impl<'h> Evaluator<'h> {
if r.height() == 1 && r.start.col <= self.at.col && self.at.col <= r.end.col {
return Some(CellRef::new(r.start.row, self.at.col));
}
// A 2-D range intersects on both the formula's row and column (observed in Excel, on
// another sheet too); one that includes the formula's own cell is a circular reference.
if r.start.row <= self.at.row && self.at.row <= r.end.row && r.start.col <= self.at.col && self.at.col <= r.end.col {
return Some(self.at);
}
None
}
+9
View File
@@ -383,6 +383,15 @@ impl<'h> Evaluator<'h> {
/// Adds one area to `acc`; `Err` is the result the function ends with.
fn aggregate_area(&mut self, kind: Agg, a: &Area, acc: &mut Acc) -> Result<(), Ev> {
if !a.range.is_single() {
// Adding the block's remembered sum to anything but +0.0 would add in another order.
// COUNT never reads the sum, so it may reuse every block.
if (kind == Agg::Count || acc.sum.to_bits() == 0)
&& let Some((sum, count)) = self.host.range_sum(a.sheet, a.range)
{
acc.sum = sum;
acc.count = acc.count.saturating_add(count);
return Ok(());
}
let mut error = None;
let walked = self.host.visit_range(a.sheet, a.range, &mut |v| match acc.add_cell(kind, v) {
Ok(()) => true,
+1
View File
@@ -6,6 +6,7 @@
pub mod eval;
mod precision;
pub mod recalc;
mod spill_index;
mod trimrange;
pub use eval::{Area, SPECIAL_FUNCTIONS, is_known_function};
+181 -33
View File
@@ -10,13 +10,24 @@
//! and the formulas that read one are evaluated again until the loop settles
//! (`PassHost::iterate_loops`).
use std::collections::VecDeque;
use std::collections::{BTreeMap, BTreeSet, VecDeque};
use gridcraft_core::{Array, CellError, CellRef, RangeRef, Value};
use gridcraft_formula::Expr;
use gridcraft_model::{CalcMode, Workbook};
use crate::eval::{Evaluator, Host, precedents, stack_position};
use crate::spill_index::{SpillIndex, indexing_pays};
/// Changed cells (in one `cells_changed`) from which a sheet's spill areas may be indexed.
const INDEX_SPILLS_FROM: usize = 16;
/// Whether a sheet's spill areas are indexed for a batch of `batch` changed cells of which
/// `on_sheet` are on that sheet: the batch must be a paste, and registering the areas must cost
/// less than walking them once for each of the sheet's own changed cells.
fn index_spills<'a>(batch: usize, on_sheet: usize, areas: impl ExactSizeIterator<Item = &'a RangeRef>) -> bool {
batch > INDEX_SPILLS_FROM && areas.len() > 0 && indexing_pays(areas, on_sheet)
}
mod graph;
mod pass_host;
@@ -37,11 +48,61 @@ pub struct Calc {
/// Cells whose formulas have errors from cycles.
pub circular: Vec<Key>,
pub last_recalc_cells: usize,
/// The cells whose value is `#SPILL!`, kept so that finding the formulas an emptied cell or a
/// freed spill area may unblock doesn't walk every formula or window of cells. Filled in
/// `rebuild`, kept up to date by `cells_changed` and by the write-back of a run; an entry is
/// always checked against the workbook before it is used.
spill_errors: SpillErrors,
}
/// A set of cells by row, each row's columns in their own set: a window of rows and columns is
/// found by one column range per row that has an entry, so the cost follows the cells in the
/// window, not those to the side of it in the same rows.
#[derive(Clone, Debug, Default)]
struct SpillErrors {
rows: BTreeMap<(usize, u32), BTreeSet<u32>>,
}
impl SpillErrors {
fn is_empty(&self) -> bool {
self.rows.is_empty()
}
fn clear(&mut self) {
self.rows.clear();
}
fn insert(&mut self, (sheet, c): Key) {
self.rows.entry((sheet, c.row)).or_default().insert(c.col);
}
fn remove(&mut self, (sheet, c): &Key) {
if let Some(cols) = self.rows.get_mut(&(*sheet, c.row)) {
cols.remove(&c.col);
if cols.is_empty() {
self.rows.remove(&(*sheet, c.row));
}
}
}
/// The cells of `sheet` in `rows` x `cols` (both inclusive), in row-major order.
fn range(&self, sheet: usize, rows: (u32, u32), cols: (u32, u32)) -> impl Iterator<Item = CellRef> + '_ {
self.rows
.range((sheet, rows.0)..=(sheet, rows.1))
.flat_map(move |(&(_, row), set)| set.range(cols.0..=cols.1).map(move |&col| CellRef::new(row, col)))
}
}
impl Default for Calc {
fn default() -> Self {
Calc { graph: Graph::default(), rng: 0x9E37_79B9_7F4A_7C15, fixed_now: None, circular: vec![], last_recalc_cells: 0 }
Calc {
graph: Graph::default(),
rng: 0x9E37_79B9_7F4A_7C15,
fixed_now: None,
circular: vec![],
last_recalc_cells: 0,
spill_errors: SpillErrors::default(),
}
}
}
@@ -125,12 +186,15 @@ impl Calc {
pub fn rebuild(&mut self, wb: &Workbook) {
self.graph = Graph::default();
let mut circ = Vec::new();
self.spill_errors.clear();
for (si, sheet) in wb.sheets.iter().enumerate() {
for (c, cell) in sheet.cells.iter() {
if cell.value == Value::Error(CellError::Spill) {
self.spill_errors.insert((si, c));
}
if let Some(f) = &cell.formula
&& let Some(e) = f.expr()
&& let Some((areas, dynamic)) = f.with_expr(|e| precedents(wb, si, e))
{
let (areas, dynamic) = precedents(wb, si, &e);
self.graph.insert((si, c), Node { areas, dynamic });
if cell.value == Value::Error(CellError::Circ) {
circ.push((si, c));
@@ -188,6 +252,20 @@ impl Calc {
out
}
/// The cells of `sheet` in `rows` x `cols` (inclusive, `cols` a lower and an upper column) that
/// show `#SPILL!`, in row-major order.
fn spill_errors_in(&self, wb: &Workbook, sheet: usize, rows: (u32, u32), cols: (u32, u32)) -> Vec<CellRef> {
let Some(sh) = wb.sheet(sheet) else { return Vec::new() };
self.spill_errors.range(sheet, rows, cols).filter(|c| sh.cells.get(*c).is_some_and(|x| x.value == Value::Error(CellError::Spill))).collect()
}
/// The formulas showing `#SPILL!` on `sheet` at or above and to the left of `end`: what a
/// cell emptied or a spill area freed there may unblock.
fn unblock_candidates(&self, wb: &Workbook, sheet: usize, end: CellRef) -> Vec<Key> {
let cells = self.spill_errors_in(wb, sheet, (0, end.row), (0, end.col));
cells.into_iter().filter(|c| self.graph.nodes.contains_key(&(sheet, *c))).map(|c| (sheet, c)).collect()
}
/// Recalculates every formula (F9 / Ctrl+Alt+F9, and after loading).
pub fn recalc_all(&mut self, wb: &mut Workbook) {
self.rebuild(wb);
@@ -209,14 +287,19 @@ impl Calc {
pub fn cells_changed(&mut self, wb: &mut Workbook, changed: &[Key]) {
let mut cleared: Vec<(usize, RangeRef)> = Vec::new();
for &k in changed {
let formula = wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.formula.clone());
if wb.sheet(k.0).and_then(|s| s.cell(k.1)).is_some_and(|c| c.value == Value::Error(CellError::Spill)) {
self.spill_errors.insert(k);
} else {
self.spill_errors.remove(&k);
}
let precedent =
wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.formula.as_ref()).and_then(|f| f.with_expr(|e| precedents(wb, k.0, e)));
let py = wb.sheet(k.0).and_then(|s| s.cell(k.1)).and_then(|c| c.py.as_ref());
let python = py.is_some();
// A Python cell whose result (if any) is not an array spills nothing.
let spills = py.and_then(|p| p.result_value()).is_some_and(|v| matches!(v, Value::Array(_)));
match formula.and_then(|f| f.expr()) {
Some(e) => {
let (areas, dynamic) = precedents(wb, k.0, &e);
match precedent {
Some((areas, dynamic)) => {
self.graph.insert(k, Node { areas, dynamic });
}
// See `rebuild`: its spill is laid out again from its result when it is calculated.
@@ -249,6 +332,7 @@ impl Calc {
&& cell.formula.is_some()
{
cell.value = Value::Error(CellError::Name);
self.spill_errors.remove(&k);
}
}
}
@@ -264,17 +348,40 @@ impl Calc {
let mut emptied: Vec<Key> = Vec::new();
// Typing into (or clearing) a cell a dynamic array spills over (or would spill over)
// re-evaluates the anchor, which may now be blocked or unblocked.
// With many changed cells (a paste) a sheet's spill areas are indexed once instead of being
// walked for each one, when registering them costs less than walking them would.
let mut areas = SpillIndex::default();
let mut strategy: HashMap<usize, bool> = HashMap::default();
let mut per_sheet: HashMap<usize, usize> = HashMap::default();
for k in changed {
*per_sheet.entry(k.0).or_insert(0) += 1;
}
let mut at = Vec::new();
for &k in changed {
if let Some(sh) = wb.sheet(k.0) {
for (anchor, r) in &sh.spill_ranges {
if *anchor != k.1 && r.contains(k.1) {
seeds.push((k.0, *anchor));
let indexed = *strategy.entry(k.0).or_insert_with(|| {
let indexed = index_spills(changed.len(), per_sheet[&k.0], sh.spill_ranges.values());
if indexed {
for (anchor, r) in &sh.spill_ranges {
areas.insert_rect((k.0, *anchor), *r, false);
}
}
indexed
});
if indexed {
// Anchor order, as walking `spill_ranges` gives.
areas.meeting(k.0, RangeRef::new(k.1, k.1), &mut at);
seeds.extend(at.iter().filter(|a| **a != k.1).map(|a| (k.0, *a)));
} else {
for (anchor, r) in &sh.spill_ranges {
if *anchor != k.1 && r.contains(k.1) {
seeds.push((k.0, *anchor));
}
}
}
for (c, cell) in sh.cells.iter_range(RangeRef::new(CellRef::new(k.1.row.saturating_sub(64), k.1.col.saturating_sub(64)), k.1)) {
if c != k.1 && cell.value == Value::Error(CellError::Spill) {
seeds.push((k.0, c));
}
if !self.spill_errors.is_empty() {
let (r0, c0) = (k.1.row.saturating_sub(64), k.1.col.saturating_sub(64));
seeds.extend(self.spill_errors_in(wb, k.0, (r0, k.1.row), (c0, k.1.col)).into_iter().filter(|c| *c != k.1).map(|c| (k.0, c)));
}
if !sh.cells.has(k.1) {
match emptied.iter_mut().find(|x| x.0 == k.0) {
@@ -286,11 +393,7 @@ impl Calc {
}
// An emptied cell can unblock an anchor any distance above or to the left of it.
for (si, end) in emptied {
for &(s, c) in self.graph.nodes.keys() {
if s == si && c.row <= end.row && c.col <= end.col && wb.sheet(s).is_some_and(|sh| sh.value(c) == Value::Error(CellError::Spill)) {
seeds.push((s, c));
}
}
seeds.extend(self.unblock_candidates(wb, si, end));
}
// Spill areas of changed anchors also change. They stay rectangles: one seed per spilled
// cell listed the formulas reading the area once per cell.
@@ -454,10 +557,15 @@ impl Calc {
let mut freed = Vec::new();
for (key, value) in results {
let Some(sheet) = wb.sheet_mut(key.0) else { continue };
if let Some(cell) = sheet.cells.get_mut(key.1)
&& cell.value != value
{
cell.value = value;
if let Some(cell) = sheet.cells.get_mut(key.1) {
if value == Value::Error(CellError::Spill) {
self.spill_errors.insert(key);
} else {
self.spill_errors.remove(&key);
}
if cell.value != value {
cell.value = value;
}
}
}
for key in dirty {
@@ -487,6 +595,7 @@ impl Calc {
if blocked {
if let Some(cell) = sheet.cells.get_mut(key.1) {
cell.value = Value::Error(CellError::Spill);
self.spill_errors.insert(*key);
}
let mut blocked_dependents = Vec::new();
self.graph.dependents_of_range(key.0, range, &mut blocked_dependents);
@@ -513,14 +622,7 @@ impl Calc {
}
}
for (sheet_index, range) in freed {
for &(sheet, cell) in self.graph.nodes.keys() {
let was_blocked_cell = cell.row <= range.end.row
&& cell.col <= range.end.col
&& wb.sheet(sheet).is_some_and(|sheet| sheet.value(cell) == Value::Error(CellError::Spill));
if sheet == sheet_index && was_blocked_cell {
spill_changes.push((sheet, cell));
}
}
spill_changes.extend(self.unblock_candidates(wb, sheet_index, range.end));
}
spill_changes
}
@@ -577,7 +679,7 @@ pub(crate) mod tests {
use gridcraft_core::{CellRef, RangeRef, Value};
use gridcraft_model::{Cell, Formula, Workbook};
use super::{Calc, HashMap, Host, Intervals, Key, PassHost};
use super::{Calc, HashMap, Host, Intervals, Key, PassHost, index_spills};
/// xorshift64 for repeatable pseudo-random cases.
fn rng(state: &mut u64) -> u64 {
@@ -594,6 +696,19 @@ pub(crate) mod tests {
v
}
/// One changed cell on a sheet with many small spills does not index them just because the
/// batch (mostly on another sheet) is large; the sheet's own share of the batch decides.
#[test]
fn the_spill_index_is_chosen_from_the_sheets_own_changed_cells() {
let small: Vec<RangeRef> = (0..1_000).map(|i| RangeRef::new(CellRef::new(i * 3, 0), CellRef::new(i * 3 + 2, 0))).collect();
// Same areas, same batch of 5000: indexed when 5000 of them are on the sheet, walked when 1 is.
assert!(index_spills(5_000, 5_000, small.iter()));
assert!(!index_spills(5_000, 1, small.iter()));
// Not a paste, or nothing to index: never.
assert!(!index_spills(16, 16, small.iter()));
assert!(!index_spills(5_000, 5_000, [].iter()));
}
/// A 1,000 x 1,000 Python result laid out from A1 in a pass, with the same values already
/// spilled on the sheet from the last calculation, as a full recalculation finds them.
fn big_python_spill() -> (Workbook, HashMap<Key, Value>) {
@@ -636,6 +751,39 @@ pub(crate) mod tests {
assert!(!host.visit_range(0, inside, &mut |_| true), "an overlapping read leaves the block to range_values");
}
/// A Python array laid out in this pass is seen by the indexed readers, whether the sheet
/// has no spill from before or an older, smaller one.
#[test]
fn a_python_spill_found_in_the_pass_is_read_through_the_index() {
for stale in [false, true] {
let mut wb = Workbook::new();
let sh = wb.sheet_mut(0).unwrap();
let anchor = CellRef::new(0, 0);
let py = gridcraft_model::PyCode::new("grid");
sh.set_cell(anchor, Cell { value: Value::Number(0.0), py: Some(std::sync::Arc::new(py)), ..Cell::default() });
if stale {
// The last calculation spilled a 2 x 2 block of 100s.
sh.spill_ranges.insert(anchor, RangeRef::new(anchor, CellRef::new(1, 1)));
for c in RangeRef::new(anchor, CellRef::new(1, 1)).iter().skip(1) {
sh.spill.insert(c, Value::Number(100.0));
}
}
let data: Vec<Value> = (1..=9).map(|i| Value::Number(f64::from(i))).collect();
let array = gridcraft_core::Array::new(3, 3, data).unwrap();
let mut python = HashMap::default();
python.insert((0, anchor), Value::Array(std::sync::Arc::new(array)));
let exprs = HashMap::default();
let mut rng = 1u64;
let mut host = PassHost::new(&wb, &exprs, &mut rng, 0.0).with_python(python);
assert_eq!(host.cell_value(0, anchor), Value::Number(1.0));
let all = RangeRef::parse("A1:C3").unwrap();
let sum: f64 = host.range_values(0, all).iter().map(|v| if let Value::Number(n) = v { *n } else { 0.0 }).sum();
assert_eq!(sum, 45.0, "stale={stale}: a SUM over the new 3 x 3 spill reads all of it");
assert_eq!(host.cell_value(0, CellRef::new(2, 2)), Value::Number(9.0), "stale={stale}");
assert!(!host.visit_range(0, all, &mut |_| true), "stale={stale}: the new spill leaves the block to range_values");
}
}
#[test]
fn intervals_match_a_linear_scan() {
let mut iv: Intervals<u32> = Intervals::default();
+139 -57
View File
@@ -1,5 +1,6 @@
use super::graph::{HashMap, HashSet, Key};
use super::{Evaluator, Host, Workbook, stack_position};
use crate::spill_index::{Cover, SpillIndex};
use gridcraft_core::{Array, CellError, CellRef, RangeRef, Value};
use gridcraft_formula::Expr;
@@ -13,6 +14,9 @@ const MAX_NESTED: usize = 64;
/// evaluated again), however high Maximum Iterations is.
const MAX_ITERATION_WORK: usize = 10_000_000;
/// A block's results by last row: its sum and count, or `None` when it can't be summed from constants.
type BlockSums = std::collections::BTreeMap<u32, Option<(f64, u64)>>;
pub(super) struct PassHost<'a> {
wb: &'a Workbook,
exprs: &'a HashMap<Key, std::sync::Arc<Expr>>,
@@ -21,6 +25,8 @@ pub(super) struct PassHost<'a> {
in_progress: HashSet<Key>,
/// New spills found during the pass: anchor → array.
pub(super) spills: HashMap<Key, std::sync::Arc<Array>>,
/// Where the areas in `spills` are, so a read finds the ones it meets without scanning them all.
spill_index: SpillIndex,
/// Used range per sheet: the workbook doesn't change during a pass, and working it out walks
/// every row, so once per sheet rather than once per range read.
used: HashMap<usize, Option<RangeRef>>,
@@ -71,6 +77,10 @@ pub(super) struct PassHost<'a> {
stack_top: usize,
/// Whether an evaluation ran past the stack budget since the innermost formula started.
out_of_stack: bool,
/// Sum and count of blocks that hold only constants, for `range_sum`. Keyed by sheet, first
/// cell and last column; each value maps a last row to the block's result (`None`: the block
/// has a formula, a Python cell, an error value or a spilled value, so every longer one has too).
sums: HashMap<(usize, CellRef, u32), BlockSums>,
/// The kept results of the Python cells in the pass: a Python cell isn't evaluated, its
/// result is laid out (an array spills from the cell, as a formula's does). Each one is in
/// `pending` until it is laid out, so a formula that reads it first gets its new value.
@@ -86,6 +96,7 @@ impl<'a> PassHost<'a> {
pending: exprs.keys().copied().collect(),
in_progress: HashSet::default(),
spills: HashMap::default(),
spill_index: SpillIndex::default(),
used: HashMap::default(),
rng,
now,
@@ -110,6 +121,7 @@ impl<'a> PassHost<'a> {
precision: wb.calc.precision_as_displayed.then(crate::DisplayedPrecision::new),
stack_top: 0,
out_of_stack: false,
sums: HashMap::default(),
python: HashMap::default(),
}
}
@@ -123,6 +135,30 @@ impl<'a> PassHost<'a> {
}
}
/// Most spill areas a range read meets before they are checked for overlapping one another by
/// walking all of the pass's spills instead of comparing pairs.
const MAX_UNORDERED_HITS: usize = 16;
/// Whether an array at `anchor` covers `c`.
fn spill_covers(anchor: CellRef, arr: &Array, c: CellRef) -> bool {
c.row >= anchor.row && c.col >= anchor.col && ((c.row - anchor.row) as usize) < arr.rows && ((c.col - anchor.col) as usize) < arr.cols
}
/// Whether two of the spill areas anchored at `hits` share a cell inside `range`.
fn spill_hits_overlap(index: &SpillIndex, sheet: usize, hits: &[CellRef], range: RangeRef) -> bool {
let rects: Vec<RangeRef> = hits.iter().filter_map(|a| index.area(sheet, *a)).filter(|(_, empty)| !empty).map(|(r, _)| r).collect();
rects.iter().enumerate().any(|(i, a)| {
rects.iter().skip(i + 1).any(|b| {
a.intersects(b)
&& RangeRef::new(
CellRef::new(a.start.row.max(b.start.row), a.start.col.max(b.start.col)),
CellRef::new(a.end.row.min(b.end.row), a.end.col.min(b.end.col)),
)
.intersects(&range)
})
})
}
/// Whether an array result at `k` can't spill: it is in a table, or its area runs off the sheet,
/// overlaps merged cells, holds other cells, or another formula's spill (unless that formula is
/// recalculated in this pass, and so may spill elsewhere now).
@@ -161,13 +197,6 @@ pub(super) fn spills_as_before(sheet: &gridcraft_model::Sheet, anchor: CellRef,
.all(|(i, c)| c == anchor || (!sheet.cells.has(c) && sheet.spill.get(&c) == Some(arr.data.get(i).unwrap_or(&Value::Empty))))
}
/// The area an array anchored at `anchor` covers.
fn array_area(anchor: CellRef, a: &Array) -> RangeRef {
let end =
CellRef::new(anchor.row.saturating_add((a.rows as u32).saturating_sub(1)), anchor.col.saturating_add((a.cols as u32).saturating_sub(1)));
RangeRef::new(anchor, end)
}
/// The spilled values `sheet` stores inside `range`, passed to `f` until it returns false. Values
/// outside the range are not walked: with fewer rows in the range than spilled values, each row
/// is looked up on its own; otherwise one ordered scan visits each spilled value at most once.
@@ -369,6 +398,7 @@ impl PassHost<'_> {
// value -- a FILTER matching one row, a SORT or UNIQUE over one item -- so `C1#`
// is the 1 x 1 range and `ROWS(C1#)` is 1, as in Excel (#1491). A formula that
// returns a scalar never reaches this arm, so it does not become an anchor.
self.spill_index.insert(k, a.rows, a.cols);
self.spills.insert(k, a.clone());
a.data.first().cloned().unwrap_or_default()
}
@@ -383,6 +413,7 @@ impl PassHost<'_> {
let v = match self.spills.get(&k) {
Some(a) if spill_blocked(self, k, a) => {
self.spills.remove(&k);
self.spill_index.remove(k);
Value::Error(CellError::Spill)
}
_ => v,
@@ -402,6 +433,7 @@ impl PassHost<'_> {
&& a.cols >= 1
&& a.rows.checked_mul(a.cols).is_some_and(|n| n == a.data.len() && n <= gridcraft_model::python::MAX_RESULT_CELLS);
if sane {
self.spill_index.insert(k, a.rows, a.cols);
self.spills.insert(k, a.clone());
a.data.first().cloned().unwrap_or_default()
} else {
@@ -413,6 +445,7 @@ impl PassHost<'_> {
let v = match self.spills.get(&k) {
Some(a) if spill_blocked(self, k, a) => {
self.spills.remove(&k);
self.spill_index.remove(k);
Value::Error(CellError::Spill)
}
_ => v,
@@ -459,7 +492,11 @@ impl PassHost<'_> {
while passes < max_iterations.min(max_passes) && !self.converged(max_change) {
self.prev = again.iter().filter_map(|k| self.results.remove(k).map(|v| (*k, v))).collect();
self.pending = set.clone();
self.spills.retain(|a, _| !set.contains(a));
let gone: Vec<Key> = self.spills.keys().filter(|a| set.contains(a)).copied().collect();
for a in gone {
self.spills.remove(&a);
self.spill_index.remove(a);
}
for k in &again {
self.calculate(*k);
}
@@ -543,17 +580,19 @@ impl Host for PassHost<'_> {
if self.pending.contains(&k) || self.results.contains_key(&k) {
return self.compute(k);
}
// A cell covered by a spill computed in this pass.
for (anchor, arr) in &self.spills {
if anchor.0 == sheet && c.row >= anchor.1.row && c.col >= anchor.1.col {
let (dr, dc) = ((c.row - anchor.1.row) as usize, (c.col - anchor.1.col) as usize);
if dr < arr.rows && dc < arr.cols {
if self.iterate && self.tainted.contains(anchor) {
self.taint = true;
}
return arr.get(dr, dc).cloned().unwrap_or_default();
}
// A cell covered by a spill computed in this pass. Where several overlap, the first in
// `spills`' iteration order counts, as it always has.
let covering = match self.spill_index.covering(sheet, c) {
Cover::None => None,
Cover::One(a) => self.spills.get_key_value(&(sheet, a)),
Cover::Many => self.spills.iter().find(|(a, arr)| a.0 == sheet && spill_covers(a.1, arr, c)),
};
if let Some((anchor, arr)) = covering {
let (dr, dc) = ((c.row - anchor.1.row) as usize, (c.col - anchor.1.col) as usize);
if self.iterate && self.tainted.contains(anchor) {
self.taint = true;
}
return arr.get(dr, dc).cloned().unwrap_or_default();
}
self.wb.sheet(sheet).map(|s| s.value(c)).unwrap_or_default()
}
@@ -592,46 +631,44 @@ impl Host for PassHost<'_> {
}
}
if !self.spills.is_empty() {
// Only the part of each array inside the range is walked: a large array beside a small
// read costs nothing.
let mut new: Vec<(usize, Value)> = Vec::new();
for (a, arr) in self.spills.iter().filter(|(a, _)| a.0 == sheet) {
let area = array_area(a.1, arr);
if !area.intersects(&range) {
continue;
}
let (r0, r1) = (area.start.row.max(range.start.row), area.end.row.min(range.end.row));
let (c0, c1) = (area.start.col.max(range.start.col), area.end.col.min(range.end.col));
for row in r0..=r1 {
for col in c0..=c1 {
let c = CellRef::new(row, col);
if c == a.1 {
continue;
}
let v = arr.get((row - a.1.row) as usize, (col - a.1.col) as usize).cloned().unwrap_or_default();
new.push(((row - range.start.row) as usize * w + (col - range.start.col) as usize, v));
}
}
}
#[cfg(test)]
{
self.spill_visits += new.len();
}
let mut hits = Vec::new();
self.spill_index.meeting(sheet, range, &mut hits);
// Iterating: an array spilled by a formula that read a previous value.
if self.iterate
&& self.spills.iter().any(|(a, arr)| {
let end = CellRef::new(
a.1.row.saturating_add((arr.rows as u32).saturating_sub(1)),
a.1.col.saturating_add((arr.cols as u32).saturating_sub(1)),
);
a.0 == sheet && self.tainted.contains(a) && RangeRef::new(a.1, end).intersects(&range)
})
{
if self.iterate && hits.iter().any(|a| self.tainted.contains(&(sheet, *a))) {
self.taint = true;
}
for (i, v) in new {
if let Some(slot) = out.get_mut(i) {
*slot = v;
// Where areas overlap inside the range, the last in `spills`' iteration order wins,
// as it always has; otherwise the order doesn't matter.
let ordered = hits.len() > MAX_UNORDERED_HITS || spill_hits_overlap(&self.spill_index, sheet, &hits, range);
let mut put = |anchor: CellRef, arr: &Array| {
if arr.rows == 0 || arr.cols == 0 {
return;
}
let (r0, c0) = (range.start.row.max(anchor.row), range.start.col.max(anchor.col));
let r1 = range.end.row.min(anchor.row.saturating_add((arr.rows as u32).saturating_sub(1)));
let c1 = range.end.col.min(anchor.col.saturating_add((arr.cols as u32).saturating_sub(1)));
for r in r0..=r1 {
for cc in c0..=c1 {
if (r, cc) != (anchor.row, anchor.col)
&& let Some(slot) = out.get_mut((r - range.start.row) as usize * w + (cc - range.start.col) as usize)
{
*slot = arr.get((r - anchor.row) as usize, (cc - anchor.col) as usize).cloned().unwrap_or_default();
}
}
}
};
if ordered {
let wanted: HashSet<Key> = hits.iter().map(|a| (sheet, *a)).collect();
for (a, arr) in &self.spills {
if wanted.contains(a) {
put(a.1, arr);
}
}
} else {
for a in &hits {
if let Some(arr) = self.spills.get(&(sheet, *a)) {
put(*a, arr);
}
}
}
}
@@ -650,7 +687,8 @@ impl Host for PassHost<'_> {
}
// Spilled values sit between the stored cells (and the formulas just calculated may have
// produced some): leave such blocks to `range_values`.
let arrays = self.spills.iter().any(|(a, arr)| a.0 == sheet && array_area(a.1, arr).intersects(&range));
let mut arrays = Vec::new();
self.spill_index.meeting(sheet, range, &mut arrays);
let mut stored = false;
let _seen = stored_spill_in(sh, range, &mut |_, _| {
stored = true;
@@ -660,7 +698,7 @@ impl Host for PassHost<'_> {
{
self.spill_visits += _seen;
}
if arrays || stored {
if !arrays.is_empty() || stored {
return false;
}
for (c, cell) in sh.cells.iter_range(range) {
@@ -671,6 +709,50 @@ impl Host for PassHost<'_> {
}
true
}
fn range_sum(&mut self, sheet: usize, range: RangeRef) -> Option<(f64, u64)> {
// Constants can't change during a pass, so a block's result stays valid for it. Spills
// found in this pass (and iteration, which tracks what a read depends on) go the long way.
if self.iterate || self.spills.keys().any(|a| a.0 == sheet) {
return None;
}
let sh = self.wb.sheet(sheet)?;
let (key, last) = ((sheet, range.start, range.end.col), range.end.row);
let known = self.sums.entry(key).or_default();
if let Some(r) = known.get(&last) {
return *r;
}
// Start from the longest shorter block of the same rows from the top: summing row-major
// from there on is the same additions in the same order as one walk from the start.
let (from, mut sum, mut count) = match known.range(..last).next_back() {
Some((_, None)) => {
known.insert(last, None);
return None;
}
Some((&row, &Some((sum, count)))) => (row + 1, sum, count),
None => (range.start.row, 0.0, 0),
};
let rest = RangeRef::new(CellRef::new(from, range.start.col), range.end);
let mut plain = sh.spill.range(rest.start..=rest.end).all(|(c, _)| !rest.contains(*c));
if plain {
for (_, cell) in sh.cells.iter_range(rest) {
match &cell.value {
_ if cell.formula.is_some() || cell.py.is_some() => plain = false,
Value::Error(_) => plain = false,
Value::Number(n) => {
sum += n;
count = u64::saturating_add(count, 1);
}
_ => {}
}
if !plain {
break;
}
}
}
let result = plain.then_some((sum, count));
known.insert(last, result);
result
}
fn used_range(&mut self, sheet: usize) -> Option<RangeRef> {
let wb = self.wb;
*self.used.entry(sheet).or_insert_with(|| wb.sheet(sheet).and_then(|s| s.used_range()))
+296
View File
@@ -0,0 +1,296 @@
//! A spatial index over spill areas: which areas cover a cell, and which meet a range, without
//! looking at every area. Areas are registered in 16 x 16 cell blocks, so a lookup touches one
//! block (a cell) or the blocks a range covers (a range) rather than every spill on the sheet.
//! An area that would take more than `BIG_AREA_BLOCKS` blocks (a full-height column spill takes
//! 65,536) is not registered in blocks but kept in a short list that every lookup tests directly,
//! so registering costs what the area's size warrants only for small areas.
use crate::recalc::{FxHasher, Key};
use gridcraft_core::{CellRef, RangeRef};
use std::hash::BuildHasherDefault;
type HashMap<K, V> = std::collections::HashMap<K, V, BuildHasherDefault<FxHasher>>;
/// Block edge in cells (rows and columns).
const BLOCK: u32 = 16;
/// Areas spanning more blocks than this are tested directly instead of being registered.
const BIG_AREA_BLOCKS: u64 = 1024;
/// One indexed area: its rectangle, and whether it covers no cells (an array with no rows or no
/// columns still meets the ranges its anchor is in, as a 1 x 1 area does for taint tracking).
#[derive(Clone, Copy)]
struct Entry {
rect: RangeRef,
empty: bool,
}
#[derive(Default)]
struct SheetAreas {
entries: HashMap<CellRef, Entry>,
blocks: HashMap<(u32, u32), Vec<CellRef>>,
/// Anchors of the areas too big to register in blocks.
big: Vec<CellRef>,
}
/// What covers a cell.
pub(crate) enum Cover {
None,
One(CellRef),
/// Several areas overlap here: the caller decides which one counts.
Many,
}
#[derive(Default)]
pub(crate) struct SpillIndex {
sheets: HashMap<usize, SheetAreas>,
}
fn block_span(rect: &RangeRef) -> (u32, u32, u32, u32) {
(rect.start.row / BLOCK, rect.end.row / BLOCK, rect.start.col / BLOCK, rect.end.col / BLOCK)
}
fn block_count(rect: &RangeRef) -> u64 {
let (r0, r1, c0, c1) = block_span(rect);
u64::from(r1 - r0 + 1) * u64::from(c1 - c0 + 1)
}
/// Whether registering `areas` in blocks costs less than checking each of them against each of
/// `lookups` cells would: a few block entries cost about one containment check each. A tall or
/// wide area is not registered at all (see `BIG_AREA_BLOCKS`) and so costs nothing here.
pub(crate) fn indexing_pays<'a>(areas: impl ExactSizeIterator<Item = &'a RangeRef>, lookups: usize) -> bool {
const BLOCK_COST: u64 = 16;
let budget = (lookups as u64).saturating_mul(areas.len() as u64) / BLOCK_COST;
let mut blocks = 0u64;
for r in areas {
let n = block_count(r);
if n > BIG_AREA_BLOCKS {
continue;
}
blocks = blocks.saturating_add(n);
if blocks > budget {
return false;
}
}
true
}
impl SpillIndex {
/// Whether any area is indexed on `sheet`.
#[cfg(test)]
pub(crate) fn has_sheet(&self, sheet: usize) -> bool {
self.sheets.get(&sheet).is_some_and(|s| !s.entries.is_empty())
}
/// Indexes the area an array of `rows` x `cols` anchored at `anchor` covers (replacing the
/// one already indexed for that anchor).
pub(crate) fn insert(&mut self, (sheet, anchor): Key, rows: usize, cols: usize) {
let end =
CellRef::new(anchor.row.saturating_add((rows as u32).saturating_sub(1)), anchor.col.saturating_add((cols as u32).saturating_sub(1)));
self.insert_rect((sheet, anchor), RangeRef::new(anchor, end), rows == 0 || cols == 0);
}
/// Indexes an area given as a range.
pub(crate) fn insert_rect(&mut self, (sheet, anchor): Key, rect: RangeRef, empty: bool) {
self.remove((sheet, anchor));
let areas = self.sheets.entry(sheet).or_default();
if block_count(&rect) > BIG_AREA_BLOCKS {
areas.big.push(anchor);
} else {
let (r0, r1, c0, c1) = block_span(&rect);
for br in r0..=r1 {
for bc in c0..=c1 {
areas.blocks.entry((br, bc)).or_default().push(anchor);
}
}
}
areas.entries.insert(anchor, Entry { rect, empty });
}
pub(crate) fn remove(&mut self, (sheet, anchor): Key) {
let Some(areas) = self.sheets.get_mut(&sheet) else { return };
let Some(old) = areas.entries.remove(&anchor) else { return };
if block_count(&old.rect) > BIG_AREA_BLOCKS {
areas.big.retain(|a| *a != anchor);
return;
}
let (r0, r1, c0, c1) = block_span(&old.rect);
for br in r0..=r1 {
for bc in c0..=c1 {
if let Some(list) = areas.blocks.get_mut(&(br, bc)) {
list.retain(|a| *a != anchor);
if list.is_empty() {
areas.blocks.remove(&(br, bc));
}
}
}
}
}
/// The areas that cover `c`.
pub(crate) fn covering(&self, sheet: usize, c: CellRef) -> Cover {
let Some(areas) = self.sheets.get(&sheet) else { return Cover::None };
let in_block = areas.blocks.get(&(c.row / BLOCK, c.col / BLOCK)).map_or(&[][..], |l| l.as_slice());
let mut found = Cover::None;
for a in in_block.iter().chain(&areas.big) {
if let Some(e) = areas.entries.get(a)
&& !e.empty
&& e.rect.contains(c)
{
found = match found {
Cover::None => Cover::One(*a),
_ => return Cover::Many,
};
}
}
found
}
/// The anchors of the areas whose rectangle meets `range` (an empty area counts through its
/// anchor cell), sorted by anchor.
pub(crate) fn meeting(&self, sheet: usize, range: RangeRef, out: &mut Vec<CellRef>) {
out.clear();
let Some(areas) = self.sheets.get(&sheet) else { return };
let (r0, r1, c0, c1) = block_span(&range);
let touched = u64::from(r1 - r0 + 1) * u64::from(c1 - c0 + 1);
if touched > 4 * areas.entries.len() as u64 + 16 {
// A range wider than the areas are many: look at each area once.
out.extend(areas.entries.iter().filter(|(_, e)| e.rect.intersects(&range)).map(|(a, _)| *a));
} else {
for br in r0..=r1 {
for bc in c0..=c1 {
if let Some(list) = areas.blocks.get(&(br, bc)) {
out.extend(list.iter().copied().filter(|a| areas.entries.get(a).is_some_and(|e| e.rect.intersects(&range))));
}
}
}
out.extend(areas.big.iter().copied().filter(|a| areas.entries.get(a).is_some_and(|e| e.rect.intersects(&range))));
}
out.sort_unstable();
out.dedup();
}
/// The rectangle indexed for `anchor` and whether it covers no cells.
pub(crate) fn area(&self, sheet: usize, anchor: CellRef) -> Option<(RangeRef, bool)> {
self.sheets.get(&sheet)?.entries.get(&anchor).map(|e| (e.rect, e.empty))
}
}
#[cfg(test)]
mod tests {
use super::*;
fn at(r: u32, c: u32) -> CellRef {
CellRef::new(r, c)
}
#[test]
fn finds_the_areas_covering_a_cell_and_meeting_a_range() {
let mut ix = SpillIndex::default();
ix.insert((0, at(0, 0)), 40, 3);
ix.insert((0, at(30, 1)), 1, 40);
ix.insert((1, at(0, 0)), 5, 5);
assert!(matches!(ix.covering(0, at(39, 2)), Cover::One(a) if a == at(0, 0)));
assert!(matches!(ix.covering(0, at(30, 2)), Cover::Many));
assert!(matches!(ix.covering(0, at(40, 0)), Cover::None));
assert!(matches!(ix.covering(2, at(0, 0)), Cover::None));
let mut hits = Vec::new();
ix.meeting(0, RangeRef::new(at(30, 30), at(50, 60)), &mut hits);
assert_eq!(hits, vec![at(30, 1)]);
ix.meeting(0, RangeRef::new(at(0, 0), at(1_000_000, 16_000)), &mut hits);
assert_eq!(hits, vec![at(0, 0), at(30, 1)]);
ix.remove((0, at(30, 1)));
assert!(matches!(ix.covering(0, at(30, 2)), Cover::One(_)));
assert!(ix.has_sheet(1) && !ix.has_sheet(2));
ix.insert((0, at(0, 0)), 1, 1);
assert!(matches!(ix.covering(0, at(39, 2)), Cover::None));
}
#[test]
fn tall_or_wide_areas_are_not_registered_in_blocks() {
let column = RangeRef::new(at(0, 0), at(1_048_575, 0));
assert!(block_count(&column) > BIG_AREA_BLOCKS);
let mut ix = SpillIndex::default();
ix.insert_rect((0, at(0, 0)), column, false);
assert!(ix.sheets[&0].blocks.is_empty());
assert!(matches!(ix.covering(0, at(900_000, 0)), Cover::One(a) if a == at(0, 0)));
assert!(matches!(ix.covering(0, at(900_000, 1)), Cover::None));
let mut hits = Vec::new();
ix.meeting(0, RangeRef::new(at(5, 0), at(6, 3)), &mut hits);
assert_eq!(hits, vec![at(0, 0)]);
ix.remove((0, at(0, 0)));
assert!(matches!(ix.covering(0, at(900_000, 0)), Cover::None) && ix.sheets[&0].big.is_empty());
// Registering a big area costs nothing (lookups test it directly), so a small paste still indexes next to many small areas.
assert!(indexing_pays([column].iter(), 17));
// Many small areas pay back for a paste of a hundred cells, but not for a handful.
let small: Vec<RangeRef> = (0..1_000).map(|i| RangeRef::new(at(i * 3, 0), at(i * 3 + 2, 0))).collect();
assert!(indexing_pays(small.iter(), 100));
assert!(!indexing_pays(small.iter(), 17));
}
/// The index against the old way: test every area for every lookup. Mixed shapes: small,
/// thin and tall, wide, big (kept out of the blocks), empty, on two sheets, replaced and removed.
#[test]
fn matches_a_scan_of_every_area_on_mixed_shapes() {
let mut seed = 0x2545_F491_4F6C_DD1Du64;
let mut next = |n: u32| {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
(seed >> 11) as u32 % n
};
let mut ix = SpillIndex::default();
let mut model: std::collections::BTreeMap<(usize, CellRef), (RangeRef, bool)> = Default::default();
for step in 0..3_000 {
let sheet = next(2) as usize;
let anchor = at(next(400) * 5, next(40) * 3);
if next(4) == 0 {
ix.remove((sheet, anchor));
model.remove(&(sheet, anchor));
} else {
let (rows, cols) = match next(8) {
0 => (0, 1 + next(5) as usize),
1 => (1 + next(5) as usize, 0),
2 => (5_000 + next(100_000) as usize, 1),
3 => (1, 1 + next(40_000) as usize),
4 => (200 + next(300) as usize, 40 + next(40) as usize),
_ => (1 + next(60) as usize, 1 + next(40) as usize),
};
ix.insert((sheet, anchor), rows, cols);
let end = at(anchor.row + (rows as u32).saturating_sub(1), anchor.col + (cols as u32).saturating_sub(1));
model.insert((sheet, anchor), (RangeRef::new(anchor, end), rows == 0 || cols == 0));
}
if step % 7 != 0 {
continue;
}
let sheet = next(3) as usize;
for _ in 0..20 {
let cell = at(next(2_100), next(130));
let want: Vec<CellRef> =
model.iter().filter(|((s, _), (r, e))| *s == sheet && !e && r.contains(cell)).map(|((_, a), _)| *a).collect();
match ix.covering(sheet, cell) {
Cover::None => assert!(want.is_empty(), "{cell:?}"),
Cover::One(a) => assert_eq!(want, vec![a], "{cell:?}"),
Cover::Many => assert!(want.len() > 1, "{cell:?}"),
}
let (r0, c0) = (next(2_100), next(130));
let (tall, wide) = (if next(5) == 0 { 3_000 } else { 40 }, if next(5) == 0 { 130 } else { 20 });
let range = RangeRef::new(at(r0, c0), at(r0 + next(tall), c0 + next(wide)));
let want: Vec<CellRef> = model.iter().filter(|((s, _), (r, _))| *s == sheet && r.intersects(&range)).map(|((_, a), _)| *a).collect();
let mut got = Vec::new();
ix.meeting(sheet, range, &mut got);
assert_eq!(got, want, "{range:?}");
}
}
}
#[test]
fn an_area_without_cells_covers_none_but_is_met_at_its_anchor() {
let mut ix = SpillIndex::default();
ix.insert((0, at(3, 3)), 0, 4);
assert!(matches!(ix.covering(0, at(3, 3)), Cover::None));
let mut hits = Vec::new();
ix.meeting(0, RangeRef::new(at(3, 3), at(3, 3)), &mut hits);
assert_eq!(hits, vec![at(3, 3)]);
}
}
+396 -2
View File
@@ -1922,6 +1922,54 @@ fn offset_extreme_arguments_return_ref() {
}
}
/// `@` intersects a reference with the formula's row (a column range) or column (a row range);
/// a 2D range both (circular when it includes the formula's cell); a range the formula does not
/// line up with gives #VALUE!. An array gives its top-left value. Legacy formulas read from files use it where Excel intersects implicitly.
#[test]
fn implicit_intersection_operator() {
let mut t = T::new();
for (at, v) in [("A1", "1"), ("A2", "2"), ("A3", "3"), ("A5", "4"), ("B5", "5"), ("C5", "6")] {
t.set(at, v);
}
t.set("B2", "=TANH(@A1:A3)");
t.set("B7", "=TANH(@A5:C5)");
t.set("B3", "=@A1:C3");
t.set("B9", "=@A1:A3");
t.set("E9", "=@A5:C5");
t.set("D1", "=@{7,8;9,10}");
t.set("D2", "=@A2");
t.set("D3", "=SUM(@A1:A3)");
assert_eq!(t.num("B2"), 2f64.tanh());
assert_eq!(t.num("B7"), 5f64.tanh());
for at in ["B9", "E9"] {
assert_eq!(t.get(at), Value::Error(CellError::Value), "{at}");
}
// A 2D range that includes the formula's own cell is a circular reference (Excel shows 0).
assert_eq!(t.get("B3"), Value::Error(CellError::Circ));
// From another sheet a 2D range intersects on both row and column; outside it is #VALUE!.
t.wb.sheets.push(std::sync::Arc::new(gridcraft_model::Sheet::new("Other")));
for (at, input) in [("B2", "=@Sheet1!A1:C3"), ("C1", "=@Sheet1!A1:C3"), ("B9", "=@Sheet1!A1:C3"), ("E2", "=@Sheet1!A1:C3")] {
t.wb.sheet_mut(1).unwrap().cells.set(c(at), Cell::formula(Formula::new(input)));
}
t.calc.recalc_all(&mut t.wb);
let other = |at: &str| t.wb.sheet(1).unwrap().value(c(at));
assert_eq!(other("B2"), Value::Number(2f64.tanh())); // Sheet1!B2
assert_eq!(other("C1"), Value::Number(0.0));
assert_eq!(other("B9"), Value::Error(CellError::Value));
assert_eq!(other("E2"), Value::Error(CellError::Value));
assert_eq!(t.num("D1"), 7.0);
assert_eq!(t.num("D2"), 2.0);
assert_eq!(t.num("D3"), 3.0);
// An array result shows its top-left value (T of a reference reads its top-left cell).
t.set("A11", "x");
t.set("C11", "y");
t.set("E1", "=@T(A11:C11)");
assert_eq!(t.get("E1"), Value::text("x"));
// Without `@` (a dynamic-array formula) the same formula spills.
t.set("F1", "=TANH(A1:A3)");
assert_eq!((t.num("F1"), t.num("F2"), t.num("F3")), (1f64.tanh(), 2f64.tanh(), 3f64.tanh()));
}
/// "Set precision as displayed": results are rounded to their cell's format before they are
/// stored and before other formulas read them. Values observed in Excel (COM,
/// `Workbook.PrecisionAsDisplayed = True`) with the same synthetic formulas.
@@ -1987,7 +2035,6 @@ fn precision_as_displayed_rounds_results_by_cell_format() {
t.calc.recalc_all(&mut t.wb);
assert_eq!(t.num("A2"), (-130.0 + 657.41) * 10.0);
}
#[test]
fn analysis_toolpak_value_params_reject_multi_cell_references() {
// Excel 365 (observed): EOMONTH, WORKDAY, COUPDAYS, BIN2DEC... give #VALUE! for a reference to more
@@ -2072,7 +2119,6 @@ fn analysis_toolpak_value_params_reject_multi_cell_references() {
t.set("H5", "");
}
}
/// A formula reading one further down (or on a later sheet) is calculated on demand, inside the
/// formula that reads it. Chains of thousands of such formulas, common in financial models, used
/// to overflow the stack (or give `#CIRC!` past 2,000 levels); they now calculate in full.
@@ -2379,6 +2425,354 @@ fn loan_schedule_residue_keeps_excels_sign() {
assert_eq!(t.get("P7"), Value::Text("".into()));
}
#[test]
fn clearing_a_cell_unblocks_the_arrays_above_and_left_of_it() {
let mut t = T::new();
for r in 1..=50 {
t.set(&format!("D{r}"), &format!("={r}*2"));
}
t.set("B2", "7");
t.set("B1", "=SEQUENCE(3)");
t.set("C1", "=SEQUENCE(1,3)");
t.set("A20", "=B1+1");
assert_eq!(t.get("B1"), Value::Error(CellError::Spill));
assert_eq!(t.get("A20"), Value::Error(CellError::Spill));
// Emptied: the array above it spills.
t.wb.sheet_mut(0).unwrap().cells.remove(c("B2"));
t.calc.cells_changed(&mut t.wb, &[(0, c("B2"))]);
assert_eq!(t.num("B3"), 3.0);
assert_eq!(t.num("A20"), 2.0);
// Typing over the area blocks it again; far from it, nothing is disturbed.
t.set("B3", "9");
assert_eq!(t.get("B1"), Value::Error(CellError::Spill));
t.set("D50", "1");
assert_eq!(t.get("B1"), Value::Error(CellError::Spill));
assert_eq!(t.num("D49"), 98.0);
// Clearing it again frees the array.
t.set("B3", "");
assert_eq!(t.num("B3"), 3.0);
}
/// A sheet with `#SPILL!` cells in rows 0..64, columns 100..16384 (as blocked arrays leave), none
/// near column A.
fn spill_error_field() -> T {
let mut t = T::new();
let sh = t.wb.sheet_mut(0).unwrap();
for r in 0..64 {
for col in 100..16_384 {
sh.cells.set(CellRef::new(r, col), Cell::value(Value::Error(CellError::Spill)));
}
}
t.calc.recalc_all(&mut t.wb);
t
}
#[test]
fn an_edit_far_left_of_many_spill_errors_does_not_disturb_them() {
let mut t = spill_error_field();
t.set("A65", "5");
assert_eq!(t.num("A65"), 5.0);
assert_eq!(t.get("CW1"), Value::Error(CellError::Spill));
assert_eq!(t.get("XFD64"), Value::Error(CellError::Spill));
// A blocked formula inside the window is still found when a cell to its right is cleared.
t.set("DA66", "7");
t.set("CZ66", "=SEQUENCE(1,3)");
assert_eq!(t.get("CZ66"), Value::Error(CellError::Spill));
t.wb.sheet_mut(0).unwrap().cells.remove(c("DA66"));
t.calc.cells_changed(&mut t.wb, &[(0, c("DA66"))]);
assert_eq!(t.num("DA66"), 2.0);
}
/// Probe: single-cell edits in column A below a block of `#SPILL!` cells to its right.
/// `cargo test -p gridcraft-calc --release -- --ignored --nocapture spill_error_field_probe`
#[test]
#[ignore]
#[allow(clippy::disallowed_methods)]
fn spill_error_field_probe() {
let mut t = spill_error_field();
let mut best = std::time::Duration::MAX;
for i in 0..9 {
let start = std::time::Instant::now();
for _ in 0..100 {
t.set("A65", &format!("{i}"));
}
best = best.min(start.elapsed());
}
eprintln!("100 edits of A65 beside 1,040,000 spill errors: best of 9 = {best:?}");
}
/// A full-height column spill (inside the 10,000,000-cell cap) and a paste of 17 cells elsewhere:
/// the paste must not register the spill's 65,536 blocks, and a paste over the spill must still
/// block it.
fn tall_spill_book() -> T {
let mut t = T::new();
t.set("A1", "=SEQUENCE(1048576)");
t.calc.recalc_all(&mut t.wb);
assert_eq!(t.num("A1048576"), 1_048_576.0);
t
}
fn paste_into(t: &mut T, cells: &[(u32, u32)]) {
let changed: Vec<(usize, CellRef)> = cells.iter().map(|&(r, c)| (0, CellRef::new(r, c))).collect();
for k in &changed {
t.wb.sheet_mut(0).unwrap().cells.set(k.1, Cell::value(Value::Number(1.0)));
}
t.calc.cells_changed(&mut t.wb, &changed);
}
#[test]
fn a_paste_beside_a_tall_spill_leaves_it_alone_and_one_over_it_blocks_it() {
let mut t = tall_spill_book();
let beside: Vec<(u32, u32)> = (0..17).map(|i| (i, 5)).collect();
paste_into(&mut t, &beside);
assert_eq!(t.num("A1048576"), 1_048_576.0);
let over: Vec<(u32, u32)> = (0..17).map(|i| (500_000 + i, if i == 9 { 0 } else { 5 })).collect();
paste_into(&mut t, &over);
assert_eq!(t.get("A1"), Value::Error(CellError::Spill));
}
/// Probe: `cells_changed` for a 17-cell paste in column F beside the tall spill.
/// `cargo test -p gridcraft-calc --release -- --ignored --nocapture tall_spill_paste_probe`
#[test]
#[ignore]
#[allow(clippy::disallowed_methods)]
fn tall_spill_paste_probe() {
let mut t = tall_spill_book();
let mut best = std::time::Duration::MAX;
for round in 0..9u32 {
let beside: Vec<(u32, u32)> = (0..17).map(|i| (round * 20 + i, 5)).collect();
let start = std::time::Instant::now();
paste_into(&mut t, &beside);
best = best.min(start.elapsed());
}
eprintln!("tall spill, 17-cell paste beside it: best of 9 = {best:?}");
}
#[test]
fn many_spills_are_read_and_blocked_by_a_paste() {
let mut t = T::new();
// 40 arrays of 3 rows side by side in rows 1-3, and a reader of a range over two of them.
for i in 0..40u32 {
let anchor = CellRef::new(0, 2 * i);
t.wb.sheet_mut(0).unwrap().cells.set(anchor, Cell::formula(Formula::new(&format!("=SEQUENCE(3,1,{})", 10 * i))));
}
t.set("A10", "=SUM(C1:F3)");
t.set("A12", "=COUNT(C1:F3)");
t.set("A11", "=C2*1");
t.calc.recalc_all(&mut t.wb);
assert_eq!(t.num("A10"), (10 + 11 + 12 + 20 + 21 + 22) as f64);
assert_eq!(t.num("A11"), 11.0);
assert_eq!(t.num("A12"), 6.0);
// Pasting 20 cells (more than one spill's width worth, so the changed set is indexed) over
// the second row of every other array blocks those arrays; the rest are unchanged.
let changed: Vec<(usize, CellRef)> = (0..20u32).map(|i| (0, CellRef::new(1, 4 * i))).collect();
for k in &changed {
t.wb.sheet_mut(0).unwrap().cells.set(k.1, Cell::value(Value::Number(1.0)));
}
t.calc.cells_changed(&mut t.wb, &changed);
assert_eq!(t.get("E1"), Value::Error(CellError::Spill));
assert_eq!(t.get("C1"), Value::Number(10.0));
assert_eq!(t.get("G2"), Value::Number(31.0));
assert_eq!(t.get("A10"), Value::Error(CellError::Spill));
assert_eq!(t.num("A12"), 4.0);
}
#[test]
fn a_paste_over_small_and_tall_spills_blocks_those_it_touches() {
let mut t = T::new();
for i in 0..40u32 {
t.wb.sheet_mut(0).unwrap().cells.set(CellRef::new(0, 2 * i), Cell::formula(Formula::new("=SEQUENCE(3)")));
}
t.wb.sheet_mut(0).unwrap().cells.set(CellRef::new(0, 200), Cell::formula(Formula::new("=SEQUENCE(1048576)")));
t.calc.recalc_all(&mut t.wb);
// 20 cells: the middle row of the small arrays in columns 0, 8, 16 ... 72 (columns 80 and up hit
// nothing), and one deep inside the tall one.
let mut cells: Vec<(u32, u32)> = (0..19u32).map(|i| (1, 8 * i)).collect();
cells.push((700_000, 200));
paste_into(&mut t, &cells);
assert_eq!(t.get("A1"), Value::Error(CellError::Spill));
assert_eq!(t.get("GS1"), Value::Error(CellError::Spill));
assert_eq!(t.num("C1"), 1.0);
assert_eq!(t.num("C3"), 3.0);
assert_eq!(t.get("I1"), Value::Error(CellError::Spill));
assert_eq!(t.num("E3"), 3.0);
}
#[test]
fn a_big_paste_on_one_sheet_and_one_cell_over_many_spills_on_another() {
let mut t = T::new();
t.wb.sheets.push(std::sync::Arc::new(gridcraft_model::Sheet::new("Data")));
for i in 0..1_500u32 {
t.wb.sheet_mut(1).unwrap().cells.set(CellRef::new(4 * i, 0), Cell::formula(Formula::new(&format!("=SEQUENCE(3,1,{})", 10 * i))));
}
t.calc.recalc_all(&mut t.wb);
assert_eq!(t.wb.sheet(1).unwrap().value(CellRef::new(4 * 700 + 2, 0)), Value::Number(7002.0));
// 5000 cells pasted on the first sheet, one over the middle of the 701st array on the second.
let mut changed: Vec<(usize, CellRef)> = (0..5_000u32).map(|i| (0, CellRef::new(i / 50, i % 50))).collect();
changed.push((1, CellRef::new(4 * 700 + 1, 0)));
for k in &changed {
t.wb.sheet_mut(k.0).unwrap().cells.set(k.1, Cell::value(Value::Number(1.0)));
}
t.calc.cells_changed(&mut t.wb, &changed);
let data = t.wb.sheet(1).unwrap();
assert_eq!(data.value(CellRef::new(4 * 700, 0)), Value::Error(CellError::Spill));
assert_eq!(data.value(CellRef::new(4 * 699 + 2, 0)), Value::Number(6992.0));
assert_eq!(data.value(CellRef::new(4 * 701 + 2, 0)), Value::Number(7012.0));
}
/// `SUM`, `COUNT` and `AVERAGE` over blocks of constants share one pass-wide answer per block, and a
/// running total extends the shorter block's. Every result must equal the left-to-right sum of the
/// cells, bit for bit, with text, blanks, errors, formulas and a nonzero start in the mix.
#[test]
fn aggregates_over_ranges_match_a_plain_fold_bit_for_bit() {
struct Rng(u64);
impl Rng {
fn next(&mut self, n: u64) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0 % n
}
}
let nums = [0.1, 0.2, 0.3, 1e16, -1e16, 1.0, 3.7, -0.0, 12345.678, 1e-9, 7.0];
let rows = 40u32;
let mut rng = Rng(0x9E37_79B9_7F4A_7C15);
let mut wb = Workbook::new();
let fresh = |rng: &mut Rng, plain: bool| match rng.next(if plain { 8 } else { 14 }) {
0 => Cell::value(Value::text("t")),
1 => Cell::default(),
2 => Cell::value(Value::Bool(true)),
3 if !plain => Cell::value(Value::Error(CellError::NA)),
4 if !plain => Cell::formula(Formula::new("=A1*2")),
5 if !plain => Cell::formula(Formula::new("=\"x\"&B2")),
_ => Cell::value(Value::Number(nums[rng.next(nums.len() as u64) as usize])),
};
// Columns A:C mostly numbers; E holds a formula column and some errors only in a few rows.
for r in 0..rows {
for col in 0..3 {
let plain = !(r % 13 == 7 && col == 1);
wb.sheet_mut(0).unwrap().cells.set(CellRef::new(r, col), fresh(&mut rng, plain));
}
}
// Formulas in column H and up: (function, optional leading number, blocks).
type Oracle<'a> = (CellRef, &'a str, Option<f64>, Vec<(u32, u32, u32, u32)>);
let mut oracle: Vec<Oracle> = vec![];
let funcs = ["SUM", "COUNT", "AVERAGE"];
let mut at = 0;
let mut add = |wb: &mut Workbook, f: &'static str, lead: Option<f64>, blocks: Vec<(u32, u32, u32, u32)>| {
let args: Vec<String> = blocks
.iter()
.map(|&(r0, c0, r1, c1)| format!("{}{}:{}{}", (b'A' + c0 as u8) as char, r0 + 1, (b'A' + c1 as u8) as char, r1 + 1))
.collect();
let lead_s = lead.map(|l| format!("{l:e},")).unwrap_or_default();
let cell = CellRef::new(at % rows, 7 + at / rows);
wb.sheet_mut(0).unwrap().cells.set(cell, Cell::formula(Formula::new(&format!("={f}({lead_s}{})", args.join(",")))));
oracle.push((cell, f, lead, blocks));
at += 1;
};
// Running totals over a clean column and over a column with an error further down.
for r in 0..rows {
add(&mut wb, "SUM", None, vec![(0, 0, r, 0)]);
add(&mut wb, "SUM", None, vec![(0, 1, r, 1)]);
add(&mut wb, "COUNT", None, vec![(0, 0, r, 2)]);
}
for _ in 0..120 {
let (r0, r1) = (rng.next(rows as u64) as u32, rng.next(rows as u64) as u32);
let (c0, c1) = (rng.next(3) as u32, rng.next(3) as u32);
let n = 1 + rng.next(2) as usize;
let blocks = (0..n).map(|_| (r0.min(r1), c0.min(c1), r0.max(r1), c0.max(c1))).collect();
let lead = [None, Some(1e16), Some(0.5), Some(2.0)][rng.next(4) as usize];
let f = funcs[rng.next(3) as usize];
add(&mut wb, f, lead, blocks);
}
let check = |wb: &Workbook, when: &str| {
let sh = wb.sheet(0).unwrap();
for (cell, f, lead, blocks) in &oracle {
let (mut sum, mut count, mut error) = (lead.unwrap_or(0.0), u64::from(lead.is_some()), None);
for &(r0, c0, r1, c1) in blocks {
for r in r0..=r1 {
for col in c0..=c1 {
match sh.value(CellRef::new(r, col)) {
Value::Number(n) => {
sum += n;
count += 1;
}
Value::Error(e) if *f != "COUNT" => error = error.or(Some(e)),
_ => {}
}
}
}
}
let want = match (error, *f) {
(Some(e), _) => Value::Error(e),
(_, "SUM") => Value::number(sum),
(_, "COUNT") => Value::number(count as f64),
(_, _) if count == 0 => Value::Error(CellError::Div0),
_ => Value::number(sum / count as f64),
};
let got = sh.value(*cell);
let same = match (&got, &want) {
(Value::Number(a), Value::Number(b)) => a.to_bits() == b.to_bits(),
(a, b) => a == b,
};
assert!(same, "{when}: {:?} = {:?}, plain fold {:?}", cell, got, want);
}
};
let mut calc = Calc::new();
calc.rebuild(&wb);
calc.recalc_all(&mut wb);
check(&wb, "full recalc");
for round in 0..30 {
let mut changed = vec![];
for _ in 0..1 + rng.next(4) {
let at = CellRef::new(rng.next(rows as u64) as u32, rng.next(3) as u32);
wb.sheet_mut(0).unwrap().cells.set(at, fresh(&mut rng, round % 2 == 0));
changed.push((0, at));
}
calc.cells_changed(&mut wb, &changed);
check(&wb, &format!("edit round {round}"));
}
}
/// `rebuild` and `cells_changed` read formulas through `Formula::with_expr`, which borrows the
/// cached tree or parses on the fly. The dependency graph must come out the same for formulas that
/// carry their tree and for formulas that only carry their text (as after loading a file).
#[test]
fn graph_built_from_text_only_formulas_matches_cached() {
fn run(cached: bool) -> Vec<Value> {
let mut t = T::new();
t.set("A1", "2");
for (at, f) in [("B1", "=A1*10"), ("C1", "=B1+A1"), ("D1", "=SUM(A1:C1)"), ("E1", "=NOPE(")] {
// E1 does not parse, so it has no tree either way and only covers the `None` branch.
let mut formula = Formula::new(f);
if !cached {
formula.expr = None;
}
t.wb.sheet_mut(0).unwrap().cells.set(c(at), Cell::formula(formula));
}
t.calc.rebuild(&t.wb);
t.calc.recalc_all(&mut t.wb);
let mut out: Vec<Value> = ["B1", "C1", "D1", "E1"].iter().map(|a| t.get(a)).collect();
// Editing the input must reach the dependents through the rebuilt graph.
t.set("A1", "5");
out.extend(["B1", "C1", "D1"].iter().map(|a| t.get(a)));
// A text-only formula that is rewritten through `cells_changed` joins the graph too.
let mut formula = Formula::new("=A1+100");
if !cached {
formula.expr = None;
}
t.wb.sheet_mut(0).unwrap().cells.set(c("F1"), Cell::formula(formula));
t.calc.cells_changed(&mut t.wb, &[(0, c("F1"))]);
t.set("A1", "7");
out.push(t.get("F1"));
out
}
let cached = run(true);
assert_eq!(cached, run(false));
assert_eq!(cached[4], Value::Number(50.0));
assert_eq!(cached[7], Value::Number(107.0));
}
/// Makes `at` a Python cell showing `cached` (its value before this calculation), with `result`
/// as its kept result, and returns the cell's key for `Calc::cells_changed`.
fn plant_python(t: &mut T, at: &str, cached: Value, result: Value) -> (usize, CellRef) {
+4 -7
View File
@@ -25,9 +25,11 @@ impl Scale {
(0..=n)
.map(|i| {
let v = self.min + i as f64 * self.step;
// Snap away accumulated float error.
// Snap away accumulated float error, but only onto a multiple of the step that is
// really that close: a `min` off the step grid must keep its own ticks.
let snapped = (v / self.step).round() * self.step;
if snapped.abs() < self.step * 1e-9 { 0.0 } else { snapped }
let v = if (snapped - v).abs() < self.step * 1e-9 { snapped } else { v };
if v.abs() < self.step * 1e-9 { 0.0 } else { v }
})
.collect()
}
@@ -107,11 +109,6 @@ pub fn nice_scale(lo: f64, hi: f64, force_zero: bool) -> Scale {
Scale { min, max, step }
}
/// Exact fixed scale (100% stacked axes).
pub(crate) fn fixed_scale(min: f64, max: f64, step: f64) -> Scale {
Scale { min, max, step }
}
/// A number format prepared once and reused for many labels.
pub(crate) struct Fmt {
general: bool,
+93 -12
View File
@@ -1,11 +1,12 @@
//! Chart layout: title, legend, axes and every chart kind, drawn into [`Prim`]s.
use std::borrow::Cow;
use std::f32::consts::{PI, TAU};
use gridcraft_model::{Chart, ChartKind, LineDash, QuartileMethod, Theme, TrendlineKind};
use crate::axis::{Fmt, Scale, clean, fixed_scale, nice_scale};
use crate::resolve::{MAX_POINTS, MAX_SERIES, MAX_TRENDLINES, series_color};
use crate::axis::{Fmt, Scale, clean, nice_scale};
use crate::resolve::{MAX_POINTS, MAX_SERIES, MAX_TRENDLINES, points_per_series, series_color};
use crate::trend::{auto_name, fit_trendline, moving_average};
use crate::{ChartData, HAlign, Measure, Prim, Rgba, SeriesData, VAlign};
@@ -44,6 +45,8 @@ const MAX_LINE_PTS: usize = 2000;
const MAX_MARKERS: usize = 5000;
/// Data labels are skipped when a series has more points than this.
const MAX_LABELS: usize = 200;
/// Columns and bars narrower than this (in pixels) are merged per pixel column into one rectangle.
const MIN_BAR_PX: f32 = 0.25;
/// Most wedges a doughnut draws for the whole chart. The per-series and per-category limits do not
/// bound their product: 255 series of 20,000 categories would be millions of wedges. Every drawable
/// ring shares this budget, so the innermost ring keeps its largest slices and the outer rings are
@@ -55,25 +58,53 @@ struct Ctx<'a> {
m: &'a dyn Measure,
}
#[cfg(test)]
#[path = "render_perf_tests.rs"]
mod perf_tests;
impl Ctx<'_> {
fn tw(&self, s: &str, size: f32) -> f32 {
let w = self.m.text_width(s, size, false);
if w.is_finite() { w.max(0.0) } else { 0.0 }
}
/// `s` shortened with an ellipsis to fit `maxw`; empty when not even one character fits.
///
/// Binary search over the number of kept characters (at most 400 are tried), about nine
/// `Measure` calls instead of one per character. It assumes the width of `prefix + "…"` does not
/// shrink as the prefix grows, which holds for per-character advance sums (`ApproxMeasure`, the
/// PDF metrics) and for the UI's text layout up to kerning of a few hundredths of a point; with
/// such a non-monotone measure the result may differ from the old linear scan by a character.
/// A label with a line break is not monotone (the ellipsis moves to a new line and the widest
/// line can get shorter), so it takes the linear scan, longest prefix first.
fn fit(&self, s: &str, size: f32, maxw: f32) -> String {
if self.tw(s, size) <= maxw {
return s.to_string();
}
let chars: Vec<char> = s.chars().collect();
let mut n = chars.len().min(400);
while n > 0 {
n -= 1;
let t: String = chars.iter().take(n).collect::<String>() + "…";
if self.tw(&t, size) <= maxw {
return if n == 0 { String::new() } else { t };
// Byte offset where each of the first 400 characters starts; a prefix of `k` characters
// (k < number of characters, so the offset exists) ends at `starts[k]`.
let starts: Vec<usize> = s.char_indices().map(|(i, _)| i).take(400).collect();
let top = starts.len().saturating_sub(1);
let fits = |k: usize| -> bool {
let end = starts.get(k).copied().unwrap_or(s.len());
let mut t = String::with_capacity(end + 3);
t.push_str(s.get(..end).unwrap_or(""));
t.push('…');
self.tw(&t, size) <= maxw
};
if s.contains('\n') {
let k = (1..=top).rev().find(|&k| fits(k));
return k.map_or_else(String::new, |k| format!("{}…", s.get(..starts.get(k).copied().unwrap_or(s.len())).unwrap_or("")));
}
let (mut lo, mut hi) = (0usize, top);
while lo < hi {
let mid = lo + (hi - lo).div_ceil(2);
if fits(mid) {
lo = mid;
} else {
hi = mid - 1;
}
}
String::new()
if lo == 0 { String::new() } else { format!("{}…", s.get(..starts.get(lo).copied().unwrap_or(s.len())).unwrap_or("")) }
}
fn text(&mut self, x: f32, y: f32, s: &str, size: f32, color: Rgba, (align, valign): (HAlign, VAlign)) {
if s.is_empty() {
@@ -101,7 +132,7 @@ fn sanitize_vals(v: &[Option<f64>]) -> Vec<Option<f64>> {
v.iter().take(MAX_POINTS).map(|x| x.filter(|n| n.is_finite()).map(clean)).collect()
}
fn sanitize(d: &ChartData) -> ChartData {
pub(crate) fn sanitize(d: &ChartData) -> ChartData {
ChartData {
palette: d.palette.iter().take(MAX_POINTS).copied().collect(),
categories: d.categories.iter().take(MAX_POINTS).cloned().collect(),
@@ -126,12 +157,62 @@ fn sanitize(d: &ChartData) -> ChartData {
}
}
/// Whether `v` survives `sanitize_vals` unchanged.
fn is_clean(v: &Option<f64>) -> bool {
v.is_none_or(|n| n.is_finite() && clean(n) == n)
}
/// Whether [`sanitize`] and [`apply_budget`] would change `d`.
fn needs_sanitizing(d: &ChartData) -> bool {
let cap = points_per_series(d.series.len());
let long = |n: usize| n > cap;
let mut trendlines = 0usize;
d.palette.len() > MAX_POINTS
|| long(d.categories.len())
|| d.category_levels.len() > 3
|| d.category_levels.iter().any(|l| l.len() > MAX_POINTS)
|| d.series.len() > MAX_SERIES
|| d.series.iter().any(|s| {
trendlines += s.trendlines.len();
long(s.values.len())
|| !s.values.iter().all(is_clean)
|| s.x.as_ref().is_some_and(|x| long(x.len()) || !x.iter().all(is_clean))
|| s.sizes.as_ref().is_some_and(|x| long(x.len()) || !x.iter().all(is_clean))
|| s.trendlines.len() > MAX_TRENDLINES
})
|| trendlines > MAX_TRENDLINES
}
/// Enforces the chart's budget of points (`MAX_TOTAL_POINTS` = series x points: every series and
/// the category list keep their first `MAX_TOTAL_POINTS / series` points) and of trendlines
/// (`MAX_TRENDLINES` per chart, in series order).
fn apply_budget(mut d: ChartData) -> ChartData {
let cap = points_per_series(d.series.len());
d.categories.truncate(cap);
let mut trendlines = MAX_TRENDLINES;
for s in &mut d.series {
s.values.truncate(cap);
for v in [&mut s.x, &mut s.sizes].into_iter().flatten() {
v.truncate(cap);
}
s.trendlines.truncate(trendlines);
trendlines = trendlines.saturating_sub(s.trendlines.len());
}
d
}
/// The data `render` draws: `d` itself when it is already clean and within budget (the usual
/// case: no copy), else a cleaned, truncated copy.
pub(crate) fn sanitized(d: &ChartData) -> Cow<'_, ChartData> {
if needs_sanitizing(d) { Cow::Owned(apply_budget(sanitize(d))) } else { Cow::Borrowed(d) }
}
/// Lays out and draws a chart into a `w`×`h` box at the origin. Empty for a degenerate box.
pub fn render(chart: &Chart, data: &ChartData, w: f32, h: f32, m: &dyn Measure) -> Vec<Prim> {
if !(w.is_finite() && h.is_finite()) || w < 1.0 || h < 1.0 || w > 1e7 || h > 1e7 {
return vec![];
}
let data = sanitize(data);
let data = sanitized(data);
let mut ctx = Ctx { out: Vec::new(), m };
ctx.out.push(Prim::Rect { x: 0.0, y: 0.0, w, h, fill: Some(WHITE), stroke: Some((GRID, 1.0)), radius: 0.0 });
let pad = (w.min(h) * 0.04).clamp(2.0, 10.0);
+115 -12
View File
@@ -3,8 +3,8 @@ use super::legend::{cat_label, plot_kind};
use super::lines::{decimate_area, draw_markers, draw_runs};
use super::trend::{draw_trend_labels, draw_trendlines, max_backward, max_forward, max_forward_slots, supports_trendline};
use super::{
Chart, ChartData, ChartKind, Ctx, Fmt, HAlign, LABEL_SIZE, MAX_CATS, MAX_LABELS, MAX_LINE_PTS, MAX_MARKERS, Prim, Scale, SeriesData, TEXT,
VAlign, WHITE, clean, fixed_scale, nice_scale,
Chart, ChartData, ChartKind, Ctx, Fmt, HAlign, LABEL_SIZE, MAX_CATS, MAX_LABELS, MAX_LINE_PTS, MAX_MARKERS, MIN_BAR_PX, Prim, Rgba, Scale,
SeriesData, TEXT, VAlign, WHITE, clean, nice_scale,
};
// ---------------------------------------------------------------- cartesian (column/bar/line/area/combo)
@@ -51,6 +51,110 @@ fn is_line(f: Fam) -> bool {
matches!(f, Fam::Line | Fam::LineStack)
}
/// Bars that fall into one pixel column, merged: extent along the category axis (`a0..=a1`, the
/// bars' start positions) and along the value axis (`lo..hi`).
struct BarRun {
bucket: f32,
a0: f32,
a1: f32,
lo: f32,
hi: f32,
/// More than one bar was merged.
merged: bool,
}
impl BarRun {
fn new(pos: f32, lo: f32, hi: f32) -> Self {
BarRun { bucket: pos.floor(), a0: pos, a1: pos, lo, hi, merged: false }
}
/// Whether `lo..hi` touches the run's value span; a disjoint bar would be joined by a filled gap.
fn touches(&self, lo: f32, hi: f32) -> bool {
lo <= self.hi && hi >= self.lo
}
fn add(&mut self, pos: f32, lo: f32, hi: f32) {
self.merged = true;
self.a0 = self.a0.min(pos);
self.a1 = self.a1.max(pos);
self.lo = self.lo.min(lo);
self.hi = self.hi.max(hi);
}
/// `band` is the part of the pixel column reserved for this series (`start`, `len`): in a
/// clustered chart a merged run stays inside it, so the other series keep a visible strip.
fn draw(&self, ctx: &mut Ctx, bw: f32, horizontal: bool, color: Rgba, band: Option<(f32, f32)>) {
let (a0, along) = match band {
Some((start, len)) if self.merged => (self.bucket + start, len),
_ => (self.a0, self.a1 - self.a0 + bw),
};
if horizontal {
ctx.rect(self.lo, a0, self.hi - self.lo, along, color, None);
} else {
ctx.rect(a0, self.lo, along, self.hi - self.lo, color, None);
}
}
}
/// Whether bars of width `bw` are merged per pixel column (never when they carry data labels).
fn merge_hairlines(bw: f32, labels_on: bool) -> bool {
bw < MIN_BAR_PX && !labels_on
}
/// Draws the bars of one series: a rectangle each, or, for hairline bars, one merged [`BarRun`]
/// per pixel column.
struct BarSink {
run: Option<BarRun>,
bw: f32,
horizontal: bool,
color: Rgba,
nslots: f32,
k: f32,
}
impl BarSink {
fn new(bw: f32, horizontal: bool, color: Rgba, nslots: f32, k: f32) -> Self {
BarSink { run: None, bw, horizontal, color, nslots, k }
}
/// Start of this series' slot along the category axis, for the category centered at `c`.
fn pos(&self, c: f32) -> f32 {
if self.horizontal { c + self.nslots * self.bw / 2.0 - (self.k + 1.0) * self.bw } else { c - self.nslots * self.bw / 2.0 + self.k * self.bw }
}
/// This series' strip of a pixel column (offset, length) when the columns hold several
/// clustered series, in the direction the slots run.
fn band(&self) -> Option<(f32, f32)> {
(self.nslots > 1.0).then(|| {
let idx = if self.horizontal { self.nslots - 1.0 - self.k } else { self.k };
(idx / self.nslots, 1.0 / self.nslots)
})
}
/// Draws the bar at category center `c` spanning `lo..lo + len` on the value axis, or, when
/// `merge` is set, adds it to the current run.
fn bar(&mut self, ctx: &mut Ctx, merge: bool, c: f32, lo: f32, len: f32) {
let pos = self.pos(c);
let band = self.band();
if !merge {
if self.horizontal {
ctx.rect(lo, pos, len, self.bw, self.color, None);
} else {
ctx.rect(pos, lo, self.bw, len, self.color, None);
}
return;
}
match &mut self.run {
Some(m) if m.bucket == pos.floor() && m.touches(lo, lo + len) => m.add(pos, lo, lo + len),
run => {
if let Some(m) = run.replace(BarRun::new(pos, lo, lo + len)) {
m.draw(ctx, self.bw, self.horizontal, self.color, band);
}
}
}
}
fn finish(self, ctx: &mut Ctx) {
let band = self.band();
if let Some(m) = self.run {
m.draw(ctx, self.bw, self.horizontal, self.color, band);
}
}
}
fn series_spans(data: &ChartData, fams: &[Fam], sec: &[bool], ncat: usize) -> Vec<Vec<Option<(f64, f64)>>> {
// Totals for 100% stacking, per axis and family (100% columns and 100% areas stack apart).
let mut totals: Vec<((bool, Fam), Vec<f64>)> = Vec::new();
@@ -141,11 +245,12 @@ fn axis_scales(data: &ChartData, fams: &[Fam], sec: &[bool], spans: &[Vec<Option
let all100 = members.iter().all(|i| fams.get(*i).is_some_and(|f| is_100(*f)));
let range = value_range(members.iter().flat_map(|i| {
let line = fams.get(*i).is_some_and(|f| is_line(*f));
spans.get(*i).into_iter().flatten().flatten().flat_map(move |(b, t)| if line { vec![*t] } else { vec![*b, *t] })
// Lines contribute only their top value, bars both ends (no per-point `Vec`).
spans.get(*i).into_iter().flatten().flatten().flat_map(move |(b, t)| [*b, *t].into_iter().skip(usize::from(line)))
}));
let (lo, hi) = range.unwrap_or((0.0, 0.0));
let (scale, fmt) = if all100 {
(fixed_scale(if lo < -1e-9 { -1.0 } else { 0.0 }, if hi > 1e-9 || lo >= -1e-9 { 1.0 } else { 0.0 }, 0.2), Fmt::new("0%"))
(Scale { min: if lo < -1e-9 { -1.0 } else { 0.0 }, max: if hi > 1e-9 || lo >= -1e-9 { 1.0 } else { 0.0 }, step: 0.2 }, Fmt::new("0%"))
} else {
let code = members.first().and_then(|i| data.series.get(*i)).map(|s| s.number_format.as_str()).unwrap_or("General");
(nice_scale(lo, hi, force_zero), Fmt::new(code))
@@ -187,7 +292,7 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx
if ncat == 0 {
return;
}
let kinds: Vec<ChartKind> = data.series.iter().map(|s| if horizontal { chart.kind } else { plot_kind(chart, s) }).collect();
let kinds: Vec<ChartKind> = data.series.iter().enumerate().map(|(i, s)| if horizontal { chart.kind } else { plot_kind(chart, s, i) }).collect();
let fams: Vec<Fam> = kinds.iter().map(|k| family(*k)).collect();
let any_primary = data.series.iter().any(|s| !s.secondary);
let sec: Vec<bool> = data.series.iter().map(|s| s.secondary && any_primary && !horizontal).collect();
@@ -278,19 +383,16 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx
}
1 => {
let k = slot_of.get(i).copied().unwrap_or(0) as f32;
// Hairline bars are merged per pixel column (their envelope is what shows).
let cull = merge_hairlines(bw, labels_on);
let mut sink = BarSink::new(bw, horizontal, s.color, nslots, k);
for (j, sp) in row.iter().enumerate() {
let Some((b, t)) = sp else { continue };
let (pb, pt) = (vpos(*b, &sc), vpos(*t, &sc));
let c = center(j);
let lo = pb.min(pt);
let len = (pb - pt).abs();
if horizontal {
let y = c + nslots * bw / 2.0 - (k + 1.0) * bw;
ctx.rect(lo, y, len, bw, s.color, None);
} else {
let x = c - nslots * bw / 2.0 + k * bw;
ctx.rect(x, lo, bw, len, s.color, None);
}
sink.bar(ctx, cull, c, lo, len);
if labels_on && let Some(v) = s.values.get(j).copied().flatten() {
let t = fmt.format(v, None);
let stacked = fam != Fam::Col;
@@ -316,6 +418,7 @@ pub(super) fn cartesian(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx
}
}
}
sink.finish(ctx);
}
_ => {
let pts: Vec<Option<[f32; 2]>> = row.iter().enumerate().map(|(j, sp)| sp.map(|(_, t)| [center(j), vpos(t, &sc)])).collect();
+17 -16
View File
@@ -340,7 +340,7 @@ pub(super) fn doughnut(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx)
// ---------------------------------------------------------------- treemap
/// Worst aspect ratio of a row with the given sum, smallest and largest area.
/// Worst aspect ratio of a row with the given area sum, smallest and largest item, laid along `side`.
fn worst(s: f64, mn: f64, mx: f64, side: f64) -> f64 {
if s <= 0.0 || side <= 0.0 {
return f64::INFINITY;
@@ -350,8 +350,9 @@ fn worst(s: f64, mn: f64, mx: f64, side: f64) -> f64 {
(w2 * mx / s2).max(s2 / (w2 * mn))
}
/// Squarified treemap layout of (index, area) items, sorted descending, into `r`.
fn squarify(items: &[(usize, f64)], r: Bx) -> Vec<(usize, Bx)> {
/// Squarified treemap layout of (index, value) items, sorted descending, into `r`; each block
/// comes back with its index and value.
pub(super) fn squarify(items: &[(usize, f64)], r: Bx) -> Vec<(usize, f64, Bx)> {
let total: f64 = items.iter().map(|i| i.1).sum();
let area = (r.w as f64) * (r.h as f64);
if !(total > 0.0 && area > 0.0) {
@@ -360,11 +361,10 @@ fn squarify(items: &[(usize, f64)], r: Bx) -> Vec<(usize, Bx)> {
let scale = area / total;
let mut out = Vec::with_capacity(items.len());
let mut rem = r;
let mut row: Vec<(usize, f64)> = Vec::new();
// The row's sum, smallest and largest area, kept as items join so each step is O(1).
let mut row: Vec<(usize, f64, f64)> = Vec::new();
let (mut sum, mut mn, mut mx) = (0.0f64, f64::INFINITY, 0.0f64);
let mut i = 0;
let layout = |row: &[(usize, f64)], rem: &mut Bx, out: &mut Vec<(usize, Bx)>| {
let layout = |row: &[(usize, f64, f64)], rem: &mut Bx, out: &mut Vec<(usize, f64, Bx)>| {
let s: f64 = row.iter().map(|x| x.1).sum();
if s <= 0.0 {
return;
@@ -372,9 +372,9 @@ fn squarify(items: &[(usize, f64)], r: Bx) -> Vec<(usize, Bx)> {
if rem.w >= rem.h {
let cw = (s / rem.h.max(1e-6) as f64) as f32;
let mut y = rem.y;
for (j, a) in row {
for (j, a, v) in row {
let h = (*a / cw.max(1e-6) as f64) as f32;
out.push((*j, Bx { x: rem.x, y, w: cw, h }));
out.push((*j, *v, Bx { x: rem.x, y, w: cw, h }));
y += h;
}
rem.x += cw;
@@ -382,9 +382,9 @@ fn squarify(items: &[(usize, f64)], r: Bx) -> Vec<(usize, Bx)> {
} else {
let rh = (s / rem.w.max(1e-6) as f64) as f32;
let mut x = rem.x;
for (j, a) in row {
for (j, a, v) in row {
let w = (*a / rh.max(1e-6) as f64) as f32;
out.push((*j, Bx { x, y: rem.y, w, h: rh }));
out.push((*j, *v, Bx { x, y: rem.y, w, h: rh }));
x += w;
}
rem.y += rh;
@@ -394,9 +394,12 @@ fn squarify(items: &[(usize, f64)], r: Bx) -> Vec<(usize, Bx)> {
while let Some(&(j, v)) = items.get(i) {
let a = v * scale;
let side = rem.w.min(rem.h) as f64;
if row.is_empty() || worst(sum + a, mn.min(a), mx.max(a), side) <= worst(sum, mn, mx, side) {
row.push((j, a));
(sum, mn, mx) = (sum + a, mn.min(a), mx.max(a));
// Sum, smallest and largest item of the open row, kept incrementally (same fold order as
// summing the row afresh, so the numbers are bit-identical).
let with = (sum + a, mn.min(a), mx.max(a));
if row.is_empty() || worst(with.0, with.1, with.2, side) <= worst(sum, mn, mx, side) {
row.push((j, a, v));
(sum, mn, mx) = with;
i += 1;
} else {
layout(&row, &mut rem, &mut out);
@@ -416,15 +419,13 @@ pub(super) fn treemap(ctx: &mut Ctx, chart: &Chart, data: &ChartData, area: Bx)
s.values.iter().take(MAX_MARKERS).enumerate().filter_map(|(j, v)| v.filter(|v| *v > 0.0).map(|v| (j, v))).collect();
items.sort_by(|a, b| b.1.total_cmp(&a.1));
let f = Fmt::new(&s.number_format);
let value_of: std::collections::HashMap<usize, f64> = items.iter().copied().collect();
for (j, b) in squarify(&items, area) {
for (j, v, b) in squarify(&items, area) {
let fill = point_color(data, j);
ctx.rect(b.x, b.y, b.w, b.h, fill, Some((WHITE, 1.5)));
if b.w > 24.0 && b.h > LABEL_SIZE * 1.6 {
let name = ctx.fit(&cat_label(data, j), LABEL_SIZE, b.w - 8.0);
ctx.text(b.x + 4.0, b.y + 4.0, &name, LABEL_SIZE, on(fill), (HAlign::Left, VAlign::Top));
if chart.data_labels && b.h > LABEL_SIZE * 3.2 {
let v = value_of.get(&j).copied().unwrap_or(0.0);
let t = ctx.fit(&f.format(v, None), LABEL_SIZE, b.w - 8.0);
ctx.text(b.x + 4.0, b.y + 4.0 + LABEL_SIZE * 1.4, &t, LABEL_SIZE, on(fill), (HAlign::Left, VAlign::Top));
}
+19 -8
View File
@@ -15,9 +15,10 @@ pub enum Swatch {
Marker,
}
/// The kind a series is drawn as (per-series kinds only apply to combo charts).
pub(super) fn plot_kind(chart: &Chart, s: &SeriesData) -> ChartKind {
if chart.kind == ChartKind::Combo { if s.kind == ChartKind::Combo { ChartKind::Line } else { s.kind } } else { chart.kind }
/// The kind series `i` is drawn as (per-series kinds only apply to combo charts); the same rule
/// as `series_plot_kind`, applied to the already resolved series kind.
pub(super) fn plot_kind(chart: &Chart, s: &SeriesData, i: usize) -> ChartKind {
crate::resolve::plot_kind(chart.kind, Some(s.kind), i)
}
/// Colour of point `j` for charts that colour by point (pie, doughnut, treemap, sunburst).
@@ -60,8 +61,9 @@ pub fn legend_entries(chart: &Chart, data: &ChartData) -> Vec<(String, Rgba, Swa
let series = data.series.iter().take(MAX_SERIES);
let mut out: Vec<(String, Rgba, Swatch)> = series
.clone()
.map(|s| {
let sw = match plot_kind(chart, s) {
.enumerate()
.map(|(i, s)| {
let sw = match plot_kind(chart, s, i) {
ChartKind::Line | ChartKind::LineStacked | ChartKind::Radar | ChartKind::ScatterLines => Swatch::Line,
ChartKind::LineMarkers => Swatch::LineMarker,
ChartKind::Scatter | ChartKind::Bubble => Swatch::Marker,
@@ -71,7 +73,7 @@ pub fn legend_entries(chart: &Chart, data: &ChartData) -> Vec<(String, Rgba, Swa
})
.collect();
// Trendlines follow the series, named "Linear (Sales)" unless given a name.
for s in series.filter(|s| supports_trendline(plot_kind(chart, s))) {
for (_, s) in series.enumerate().filter(|(i, s)| supports_trendline(plot_kind(chart, s, *i))) {
for t in s.trendlines.iter().take(MAX_TRENDLINES) {
out.push((t.spec.name.clone().unwrap_or_else(|| auto_name(t.spec.kind, &s.name)), t.color, Swatch::Line));
}
@@ -83,7 +85,7 @@ pub fn legend_entries(chart: &Chart, data: &ChartData) -> Vec<(String, Rgba, Swa
// ---------------------------------------------------------------- legend
fn swatch_w(sw: Swatch, s: f32) -> f32 {
pub(super) fn swatch_w(sw: Swatch, s: f32) -> f32 {
if sw == Swatch::Box { s } else { s * 2.0 }
}
@@ -147,7 +149,16 @@ pub(super) fn draw_legend(ctx: &mut Ctx, entries: &[(String, Rgba, Swatch)], pos
if pos == LegendPos::Bottom { Bx { h: area.h - used, ..area } } else { Bx { y: area.y + used, h: area.h - used, ..area } }
}
LegendPos::Left | LegendPos::Right => {
let colw = entries.iter().map(|e| item_w(ctx, e)).fold(0.0f32, f32::max).min(area.w * 0.35);
// Widest entry, capped at 35% of the area: once the cap is reached the rest cannot change it.
let cap = area.w * 0.35;
let mut colw = 0.0f32;
for e in entries {
if colw >= cap {
break;
}
colw = colw.max(item_w(ctx, e));
}
let colw = colw.min(cap);
let max_rows = ((area.h / row_h).floor() as usize).max(1);
let n = entries.len().min(max_rows);
let lh = n as f32 * row_h;
+1 -1
View File
@@ -70,7 +70,7 @@ pub(super) fn draw(ctx: &mut Ctx, chart: &Chart, data: &ChartData, plot: Bx, lay
let y = top + (layout.headers as f32 + i as f32 + 0.5) * layout.row_height;
let name_x = if settings.show_keys { x0 + 25.0 } else { x0 + 4.0 };
if settings.show_keys {
let swatch = match plot_kind(chart, series) {
let swatch = match plot_kind(chart, series, i) {
ChartKind::Line | ChartKind::LineStacked => Swatch::Line,
ChartKind::LineMarkers => Swatch::LineMarker,
_ => Swatch::Box,
+298
View File
@@ -0,0 +1,298 @@
//! Equality and counter tests for the label-fitting, legend-measuring and treemap rewrites.
//! The old implementations live here as references.
use std::cell::Cell;
use super::kinds::squarify;
use super::legend::swatch_w;
use super::*;
use crate::{ApproxMeasure, Measure};
use gridcraft_model::LegendPos;
/// The pre-rewrite `Ctx::fit`: one `Measure` call per candidate length, longest first.
fn fit_reference(ctx: &Ctx, s: &str, size: f32, maxw: f32) -> String {
if ctx.tw(s, size) <= maxw {
return s.to_string();
}
let chars: Vec<char> = s.chars().collect();
let mut n = chars.len().min(400);
while n > 0 {
n -= 1;
let t: String = chars.iter().take(n).collect::<String>() + "…";
if ctx.tw(&t, size) <= maxw {
return if n == 0 { String::new() } else { t };
}
}
String::new()
}
/// Counts calls; advances differ per character (wide CJK/emoji, zero-width combining marks) and
/// "AV"-style pairs are kerned tighter, but a character never makes the text narrower.
struct Varied {
calls: Cell<usize>,
}
impl Measure for Varied {
fn text_width(&self, text: &str, size: f32, bold: bool) -> f32 {
self.calls.set(self.calls.get() + 1);
let mut w = 0.0f32;
let mut prev = ' ';
for c in text.chars() {
let adv = match c as u32 {
0x300..=0x36f => 0.0,
0x1f300..=0x1faff | 0x4e00..=0x9fff => 1.0,
u => 0.3 + (u % 7) as f32 * 0.07,
};
let kern = if prev == 'A' && c == 'V' { 0.1 } else { 0.0 };
w += (adv - kern) * size;
prev = c;
}
if bold { w * 1.1 } else { w }
}
}
struct Counting<'a>(&'a dyn Measure, Cell<usize>);
impl Measure for Counting<'_> {
fn text_width(&self, text: &str, size: f32, bold: bool) -> f32 {
self.1.set(self.1.get() + 1);
self.0.text_width(text, size, bold)
}
}
/// Like a text layout that breaks at `\n`: the width is that of the widest line. Adding a
/// newline can make the text narrower, so the width of `prefix + "…"` is not monotone.
struct WidestLine;
impl Measure for WidestLine {
fn text_width(&self, text: &str, size: f32, _bold: bool) -> f32 {
text.split('\n')
.map(|line| {
line.chars()
.map(|c| {
size * if c == 'i' {
0.25
} else if c == 'M' {
0.9
} else {
0.5
}
})
.sum::<f32>()
})
.fold(0.0, f32::max)
}
}
fn corpus() -> Vec<String> {
let mut v: Vec<String> =
["", "a", "ab", "Hello, world", "AVAVAV tight", "日本語のラベルです", "👨‍👩‍👧 family 🎉 party", "e\u{301}e\u{301}e\u{301} combining", "…", "……"]
.iter()
.map(|s| s.to_string())
.collect();
v.push("x".repeat(399));
v.push("x".repeat(400));
v.push("x".repeat(401));
v.push("é🎉".repeat(500));
v.push("long label ".repeat(100));
for s in ["iiii\nMM", "iiii\n", "\niiii", "ab\ncd\nef", "iiiiiiii\niiiiMMMM\nMM", "line one\nline two is longer\nx"] {
v.push(s.to_string());
}
v
}
fn run_fit(m: &dyn Measure) {
let ctx = Ctx { out: vec![], m };
for s in corpus() {
for size in [9.0f32, 11.0, 40.0] {
for maxw in [-5.0, 0.0, 0.4, 6.0, 25.0, 61.5, 120.0, 800.0, 1e6, f32::INFINITY, f32::NAN] {
assert_eq!(ctx.fit(&s, size, maxw), fit_reference(&ctx, &s, size, maxw), "{s:?} size {size} maxw {maxw}");
}
}
}
}
#[test]
fn fit_matches_linear_reference() {
run_fit(&ApproxMeasure);
run_fit(&Varied { calls: Cell::new(0) });
run_fit(&WidestLine);
}
#[test]
fn a_multiline_label_keeps_its_fitting_prefix() {
// The whole label is 18 wide and "iii…" is 12.5, both too wide for 10.5, but "iiii\n…" is
// 10 wide and fits. A binary search that stops at the failing midpoint would return nothing.
let ctx = Ctx { out: vec![], m: &WidestLine };
let s = "iiii\nMM";
let size = 10.0;
let maxw = 10.5;
assert!(ctx.tw(s, size) > maxw);
assert!(ctx.tw("iii…", size) > maxw && ctx.tw("iiii\n…", size) <= maxw);
assert_eq!(ctx.fit(s, size, maxw), "iiii\n…");
}
#[test]
fn fit_measures_logarithmically() {
let long = "x".repeat(1000);
let c = Counting(&ApproxMeasure, Cell::new(0));
let ctx = Ctx { out: vec![], m: &c };
let r = ctx.fit(&long, 11.0, 60.0);
assert!(r.ends_with('…') && r.chars().count() > 3);
// 1 whole-string check + ceil(log2(399)) + 1 probes; the old scan needed ~390 here.
assert!(c.1.get() <= 12, "{} Measure calls", c.1.get());
let c2 = Counting(&ApproxMeasure, Cell::new(0));
let ctx2 = Ctx { out: vec![], m: &c2 };
assert_eq!(ctx2.fit("short", 11.0, 500.0), "short");
assert_eq!(c2.1.get(), 1);
}
fn legend_entries(n: usize, label: &dyn Fn(usize) -> String) -> Vec<(String, Rgba, Swatch)> {
(0..n).map(|i| (label(i), [10, 20, 30, 255], if i % 3 == 0 { Swatch::Line } else { Swatch::Box })).collect()
}
/// Colour column width as the old code computed it, from the returned remaining area.
fn old_colw(m: &dyn Measure, entries: &[(String, Rgba, Swatch)], area: Bx) -> f32 {
let ctx = Ctx { out: vec![], m };
let size = LABEL_SIZE;
let s = size * 0.8;
entries.iter().map(|e| swatch_w(e.2, s) + 4.0 + ctx.tw(&e.0, size)).fold(0.0f32, f32::max).min(area.w * 0.35)
}
#[test]
fn side_legend_matches_old_column_width_and_stops_measuring_at_the_cap() {
let area = Bx { x: 0.0, y: 0.0, w: 400.0, h: 300.0 };
let labels: [&dyn Fn(usize) -> String; 3] =
[&|i| format!("entry {i}"), &|i| if i == 7 { "w".repeat(80) } else { format!("e{i}") }, &|i| "very long legend label ".repeat(1 + i % 3)];
for pos in [LegendPos::Left, LegendPos::Right] {
for (k, label) in labels.iter().enumerate() {
for n in [0usize, 1, 5, 40] {
let entries = legend_entries(n, label);
let mut ctx = Ctx { out: vec![], m: &ApproxMeasure };
let rest = draw_legend(&mut ctx, &entries, pos, area);
let colw = old_colw(&ApproxMeasure, &entries, area);
assert!((rest.w - (area.w - (colw + 8.0))).abs() < 1e-4, "{pos:?} labels {k} n {n}");
}
}
}
// 20,000 entries whose first one already fills the 35% cap: the old code measured all of them
// plus the drawn rows, now only the first entry and the drawn rows are measured.
let entries = legend_entries(20_000, &|i| if i == 0 { "w".repeat(200) } else { format!("e{i}") });
let c = Counting(&ApproxMeasure, Cell::new(0));
let mut ctx = Ctx { out: vec![], m: &c };
let _ = draw_legend(&mut ctx, &entries, LegendPos::Right, area);
let drawn = ctx.out.iter().filter(|p| matches!(p, Prim::Text { .. })).count();
assert!(c.1.get() <= drawn * 10 + 5, "{} Measure calls for {drawn} drawn entries", c.1.get());
assert!(c.1.get() < 500);
}
/// The pre-rewrite `worst`/`squarify` (a `Vec` per item, row recomputed from scratch).
fn worst_reference(row: &[f64], side: f64) -> f64 {
let s: f64 = row.iter().sum();
if s <= 0.0 || side <= 0.0 {
return f64::INFINITY;
}
let (mn, mx) = row.iter().fold((f64::INFINITY, 0.0f64), |(a, b), v| (a.min(*v), b.max(*v)));
let s2 = s * s;
let w2 = side * side;
(w2 * mx / s2).max(s2 / (w2 * mn))
}
fn squarify_reference(items: &[(usize, f64)], r: Bx) -> Vec<(usize, Bx)> {
let total: f64 = items.iter().map(|i| i.1).sum();
let area = (r.w as f64) * (r.h as f64);
if !(total > 0.0 && area > 0.0) {
return vec![];
}
let scale = area / total;
let mut out = Vec::with_capacity(items.len());
let mut rem = r;
let mut row: Vec<(usize, f64)> = Vec::new();
let mut i = 0;
let layout = |row: &[(usize, f64)], rem: &mut Bx, out: &mut Vec<(usize, Bx)>| {
let s: f64 = row.iter().map(|x| x.1).sum();
if s <= 0.0 {
return;
}
if rem.w >= rem.h {
let cw = (s / rem.h.max(1e-6) as f64) as f32;
let mut y = rem.y;
for (j, a) in row {
let h = (*a / cw.max(1e-6) as f64) as f32;
out.push((*j, Bx { x: rem.x, y, w: cw, h }));
y += h;
}
rem.x += cw;
rem.w = (rem.w - cw).max(0.0);
} else {
let rh = (s / rem.w.max(1e-6) as f64) as f32;
let mut x = rem.x;
for (j, a) in row {
let w = (*a / rh.max(1e-6) as f64) as f32;
out.push((*j, Bx { x, y: rem.y, w, h: rh }));
x += w;
}
rem.y += rh;
rem.h = (rem.h - rh).max(0.0);
}
};
while let Some(&(j, v)) = items.get(i) {
let a = v * scale;
let side = rem.w.min(rem.h) as f64;
let cur: Vec<f64> = row.iter().map(|x| x.1).collect();
let mut with = cur.clone();
with.push(a);
if row.is_empty() || worst_reference(&with, side) <= worst_reference(&cur, side) {
row.push((j, a));
i += 1;
} else {
layout(&row, &mut rem, &mut out);
row.clear();
}
}
if !row.is_empty() {
layout(&row, &mut rem, &mut out);
}
out
}
#[test]
fn squarify_matches_reference() {
let mut seed = 0x2545_f491_4f6c_dd1du64;
let mut rnd = move || {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
(seed >> 11) as f64 / (1u64 << 53) as f64
};
let mut cases: Vec<Vec<f64>> = vec![
vec![],
vec![5.0],
vec![6.0, 6.0, 4.0, 3.0, 2.0, 2.0, 1.0],
vec![3.0, 0.0, 0.0, 1.0],
vec![0.0, 0.0],
vec![4.0, -1.0, 2.0],
vec![4.0, f64::NAN, 2.0, 1.0],
vec![f64::INFINITY, 1.0],
vec![1e-300, 1e-310, 0.0],
];
for n in [2usize, 9, 40, 300] {
let mut v: Vec<f64> = (0..n).map(|_| rnd() * 100.0).collect();
v.sort_by(|a, b| b.total_cmp(a));
cases.push(v);
}
for r in [Bx { x: 0.0, y: 0.0, w: 300.0, h: 200.0 }, Bx { x: 5.0, y: 7.0, w: 90.0, h: 400.0 }, Bx { x: 0.0, y: 0.0, w: 0.0, h: 50.0 }] {
for vals in &cases {
let items: Vec<(usize, f64)> = vals.iter().copied().enumerate().collect();
let new: Vec<(usize, Bx)> = squarify(&items, r)
.into_iter()
.map(|(j, v, b)| {
assert!(vals.get(j).is_some_and(|x| x.to_bits() == v.to_bits()), "value carried through");
(j, b)
})
.collect();
let old = squarify_reference(&items, r);
assert_eq!(format!("{new:?}"), format!("{old:?}"), "{vals:?} in {r:?}");
}
}
}
+180 -23
View File
@@ -1,7 +1,7 @@
//! Pulls chart data (names, categories, values, formats, colours) out of a workbook.
use gridcraft_calc::evaluate;
use gridcraft_core::{CellRef, RangeRef, Value, category_levels, number_to_text};
use gridcraft_core::{CellRef, MAX_COLS, MAX_ROWS, RangeRef, Value, category_levels, number_to_text};
use gridcraft_model::{Chart, ChartKind, Theme, Workbook, style::apply_tint};
use gridcraft_numfmt::{NumberFormat, format_value};
@@ -11,8 +11,17 @@ use crate::{ChartData, Rgba, SeriesData, TrendlineData};
pub(crate) const MAX_POINTS: usize = 100_000;
/// Cap on series per chart.
pub(crate) const MAX_SERIES: usize = 255;
/// Cap on trendlines drawn per series.
/// Cap on trendlines per chart (not per series: each one is a fit over a series' points).
pub(crate) const MAX_TRENDLINES: usize = 16;
/// Budget for series x points in one chart. When a chart would hold more, every series (and the
/// category list) keeps its first `MAX_TOTAL_POINTS / series` points; no series is dropped.
pub(crate) const MAX_TOTAL_POINTS: usize = 2_000_000;
/// Points kept per series (and categories) in a chart of `series` series: `MAX_POINTS`, or less
/// when `series x MAX_POINTS` would pass `MAX_TOTAL_POINTS`.
pub(crate) fn points_per_series(series: usize) -> usize {
(MAX_TOTAL_POINTS / series.clamp(1, MAX_SERIES)).clamp(1, MAX_POINTS)
}
/// Tints used for series beyond the six theme accents: lighter, darker, lighter still…
const CYCLE_TINTS: [f64; 6] = [0.0, 0.4, -0.25, 0.6, -0.5, 0.2];
@@ -77,10 +86,51 @@ fn union_parts(f: &str) -> Option<Vec<&str>> {
/// Evaluates a series formula to a flat list of values (row-major), capped. A union of
/// references (`(A1:A3,A5)`, as PivotCharts use) is read part by part.
fn eval_list(wb: &Workbook, sheet: usize, formula: &str) -> Vec<Value> {
eval_list_in(wb, sheet, formula, 0)
eval_list_in(wb, sheet, formula, MAX_POINTS, 0)
}
fn eval_list_in(wb: &Workbook, sheet: usize, formula: &str, depth: u32) -> Vec<Value> {
/// calc reads a reference of up to this many cells at its full size and trims a larger one to the
/// sheet's used range (`FULL_SIZE_CELLS` in `gridcraft_calc::eval`). A clipped reference must stay
/// above it, or the clip would turn a trimmed read into a full-size one with trailing blanks.
const CALC_FULL_SIZE_CELLS: u64 = 16_384;
/// Bounded references up to this many cells are read at their full size too
/// (`FULL_SIZE_BOUNDED_CELLS` in `gridcraft_calc::eval`).
const CALC_FULL_SIZE_BOUNDED_CELLS: u64 = 1_048_576;
/// `formula` as a plain reference (`Sheet!A1:A1000000`) cut after the rows that can still hold
/// one of the first `limit` points, so that calc never reads the rest; `None` when `formula` is
/// no reference or already small enough. Values are read row by row, and calc trims only the
/// bottom and right of a reference, so keeping `limit` rows keeps the first `limit` values.
fn clipped_reference(wb: &Workbook, sheet: usize, formula: &str, limit: usize) -> Option<String> {
let (sh, mut r) = parse_ref(wb, sheet, formula)?;
let cells = u64::from(r.height()).saturating_mul(u64::from(r.width()));
// calc trims a block that runs to the sheet's edge, and any block over a column's worth, to
// the used range. Do it here, or the clipped block below would be read at its full size.
let to_edge = r.end.row == MAX_ROWS - 1 || r.end.col == MAX_COLS - 1;
let trimmed = cells > CALC_FULL_SIZE_CELLS && (to_edge || cells > CALC_FULL_SIZE_BOUNDED_CELLS);
let mut changed = false;
if trimmed {
let end = match wb.sheet(sh).and_then(|s| s.used_range()) {
Some(u) => CellRef::new(r.end.row.min(u.end.row.max(r.start.row)), r.end.col.min(u.end.col.max(r.start.col))),
None => r.start,
};
changed = end != r.end;
r = RangeRef::new(r.start, end);
}
let prefix = formula.rfind('!').and_then(|i| formula.get(..=i)).unwrap_or("");
let (h, w) = (u64::from(r.height()), u64::from(r.width()));
if h.saturating_mul(w) > CALC_FULL_SIZE_CELLS {
let keep = (limit as u64).max(CALC_FULL_SIZE_CELLS / w + 1);
if h > keep {
let end = CellRef::new(r.start.row.saturating_add(u32::try_from(keep).ok()?).saturating_sub(1), r.end.col);
r = RangeRef::new(r.start, end);
changed = true;
}
}
changed.then(|| format!("{prefix}{r}"))
}
fn eval_list_in(wb: &Workbook, sheet: usize, formula: &str, limit: usize, depth: u32) -> Vec<Value> {
let f = formula.trim();
if f.is_empty() {
return vec![];
@@ -90,16 +140,17 @@ fn eval_list_in(wb: &Workbook, sheet: usize, formula: &str, depth: u32) -> Vec<V
{
let mut out = Vec::new();
for p in parts {
if out.len() >= MAX_POINTS {
if out.len() >= limit {
break;
}
let rest = MAX_POINTS - out.len();
out.extend(eval_list_in(wb, sheet, p, depth + 1).into_iter().take(rest));
let rest = limit - out.len();
out.extend(eval_list_in(wb, sheet, p, rest, depth + 1).into_iter().take(rest));
}
return out;
}
match evaluate(wb, sheet, CellRef::default(), f) {
Value::Array(a) => a.iter().take(MAX_POINTS).cloned().collect(),
let clipped = clipped_reference(wb, sheet, f, limit);
match evaluate(wb, sheet, CellRef::default(), clipped.as_deref().unwrap_or(f)) {
Value::Array(a) => a.iter().take(limit).cloned().collect(),
v => vec![v],
}
}
@@ -111,8 +162,8 @@ fn to_num(v: &Value) -> Option<f64> {
}
}
fn nums(wb: &Workbook, sheet: usize, formula: &str) -> Vec<Option<f64>> {
eval_list(wb, sheet, formula).iter().map(to_num).collect()
fn nums(wb: &Workbook, sheet: usize, formula: &str, limit: usize) -> Vec<Option<f64>> {
eval_list_in(wb, sheet, formula, limit, 0).iter().map(to_num).collect()
}
/// Number format code of the first cell of a reference ("General" when not a reference).
@@ -137,9 +188,38 @@ fn value_text(v: &Value) -> String {
/// the outer levels of a multi-level axis: a category range wider than a column of values (or
/// taller than a row of values) groups its innermost labels, year beside month. Each outer level
/// has one entry per category, non-empty where a group starts, as the cells are.
fn category_table(wb: &Workbook, sheet: usize, formula: &str, values: &str) -> (Vec<String>, Vec<Vec<String>>) {
let vals = eval_list(wb, sheet, formula);
fn category_table(wb: &Workbook, sheet: usize, formula: &str, values: &str, limit: usize) -> (Vec<String>, Vec<Vec<String>>) {
let cells: Option<(usize, RangeRef)> = parse_ref(wb, sheet, formula);
let value_range = parse_ref(wb, sheet, values).map(|(_, r)| r);
// Bound the category dimension by `limit` and read only the cells the levels use (the
// innermost label line and up to three outer ones), so that every kept category keeps its
// cells in every level. Counting raw cells would cut a two-level axis in the middle.
let levels = cells.and_then(|(_, r)| category_levels(&r, value_range.as_ref()));
let (cells, read) = match (cells, levels) {
(Some((sh, r)), Some(lv)) => {
let cut = u32::try_from(limit.saturating_sub(1)).unwrap_or(u32::MAX);
let outer = lv.outer;
let end = if lv.by_columns {
CellRef::new(r.end.row.min(r.start.row.saturating_add(cut)), r.end.col)
} else {
CellRef::new(r.end.row, r.end.col.min(r.start.col.saturating_add(cut)))
};
let start = if lv.by_columns {
CellRef::new(r.start.row, r.start.col.max(end.col.saturating_sub(outer)))
} else {
CellRef::new(r.start.row.max(end.row.saturating_sub(outer)), r.start.col)
};
let r = RangeRef::new(start, end);
let prefix = formula.rfind('!').and_then(|i| formula.get(..=i)).unwrap_or("");
let cap = (r.width() as usize).saturating_mul(r.height() as usize);
(Some((sh, r)), Some((format!("{prefix}{r}"), cap)))
}
(c, _) => (c, None),
};
let vals = match &read {
Some((f, cap)) => eval_list_in(wb, sheet, f, *cap, 0),
None => eval_list_in(wb, sheet, formula, limit, 0),
};
let src = cells.and_then(|(sh, r)| wb.sheet(sh).map(|s| (s, r)));
let texts: Vec<String> = vals
.iter()
@@ -159,8 +239,7 @@ fn category_table(wb: &Workbook, sheet: usize, formula: &str, values: &str) -> (
})
.collect();
let Some((_, r)) = src else { return (texts, vec![]) };
let value_range = parse_ref(wb, sheet, values).map(|(_, r)| r);
let Some(levels) = category_levels(&r, value_range.as_ref()) else { return (texts, vec![]) };
let Some(levels) = levels else { return (texts, vec![]) };
let (w, h) = (r.width().max(1) as usize, r.height().max(1) as usize);
if texts.len() != w.saturating_mul(h) {
return (texts, vec![]); // truncated or not a plain block: a flat list
@@ -201,7 +280,7 @@ fn series_name(wb: &Workbook, sheet: usize, name: &str) -> String {
t.to_string()
}
fn effective_kind(chart_kind: ChartKind, series_kind: Option<ChartKind>, i: usize) -> ChartKind {
pub(crate) fn effective_kind(chart_kind: ChartKind, series_kind: Option<ChartKind>, i: usize) -> ChartKind {
match (chart_kind, series_kind) {
(ChartKind::Combo, Some(k)) if k != ChartKind::Combo => k,
(ChartKind::Combo, _) => {
@@ -216,9 +295,15 @@ fn effective_kind(chart_kind: ChartKind, series_kind: Option<ChartKind>, i: usiz
}
}
/// The kind series `i` is drawn as: its own kind in a combo chart (first series a column, the
/// rest lines when it has none), else the chart's. The one rule shared by resolving and drawing.
pub(crate) fn plot_kind(chart_kind: ChartKind, series_kind: Option<ChartKind>, i: usize) -> ChartKind {
if chart_kind == ChartKind::Combo { effective_kind(chart_kind, series_kind, i) } else { chart_kind }
}
/// The kind series `i` of `chart` is drawn as: its own kind in a combo chart, else the chart's.
pub fn series_plot_kind(chart: &Chart, i: usize) -> ChartKind {
if chart.kind == ChartKind::Combo { effective_kind(chart.kind, chart.series.get(i).and_then(|s| s.kind), i) } else { chart.kind }
plot_kind(chart.kind, chart.series.get(i).and_then(|s| s.kind), i)
}
/// Evaluates a chart's series formulas: names, values, categories (default "1", "2", …), X
@@ -228,37 +313,43 @@ pub fn resolve(wb: &Workbook, sheet: usize, chart: &Chart) -> ChartData {
let numeric_x = matches!(chart.kind, ChartKind::Scatter | ChartKind::ScatterLines | ChartKind::Bubble);
let mut series = Vec::new();
let mut categories: Option<Vec<String>> = None;
let limit = points_per_series(chart.series.len());
let mut trend_left = MAX_TRENDLINES;
let mut category_levels = Vec::new();
for (i, s) in chart.series.iter().take(MAX_SERIES).enumerate() {
let values = nums(wb, sheet, &s.values);
let values = nums(wb, sheet, &s.values, limit);
let name = s.name.as_deref().map(|n| series_name(wb, sheet, n)).filter(|n| !n.is_empty()).unwrap_or_else(|| format!("Series{}", i + 1));
let color =
s.color.as_ref().and_then(|c| c.resolve(&wb.theme)).map(|[r, g, b]| [r, g, b, 0xFF]).unwrap_or_else(|| series_color(&wb.theme, i));
let kind = effective_kind(chart.kind, s.kind, i);
let x = match (&s.categories, numeric_x || matches!(kind, ChartKind::Scatter | ChartKind::ScatterLines | ChartKind::Bubble)) {
// X values with text in them (labels) are ignored: the points are numbered 1..n.
(Some(f), true) => Some(eval_list(wb, sheet, f)).filter(|xs| !xs.iter().any(Value::is_text)).map(|xs| xs.iter().map(to_num).collect()),
(Some(f), true) => {
Some(eval_list_in(wb, sheet, f, limit, 0)).filter(|xs| !xs.iter().any(Value::is_text)).map(|xs| xs.iter().map(to_num).collect())
}
_ => None,
};
if categories.is_none()
&& let Some(f) = &s.categories
{
let (c, outer) = category_table(wb, sheet, f, &s.values);
let (mut c, outer) = category_table(wb, sheet, f, &s.values, limit);
c.truncate(limit);
if !c.is_empty() {
categories = Some(c);
category_levels = outer;
}
}
let sizes = s.bubble_sizes.as_deref().map(|f| nums(wb, sheet, f));
let sizes = s.bubble_sizes.as_deref().map(|f| nums(wb, sheet, f, limit));
let trendlines = s
.trendlines
.iter()
.take(MAX_TRENDLINES)
.take(trend_left)
.map(|t| TrendlineData {
spec: t.clone(),
color: t.color.as_ref().and_then(|c| c.resolve(&wb.theme)).map(|[r, g, b]| [r, g, b, 0xFF]).unwrap_or(color),
})
.collect();
.collect::<Vec<_>>();
trend_left = trend_left.saturating_sub(trendlines.len());
series.push(SeriesData {
name,
values,
@@ -277,3 +368,69 @@ pub fn resolve(wb: &Workbook, sheet: usize, chart: &Chart) -> ChartData {
let palette = default_palette(&wb.theme, n.min(PALETTE_CYCLE));
ChartData { categories, category_levels, series, palette }
}
#[cfg(test)]
mod tests {
use gridcraft_model::Workbook;
use super::*;
fn book(rows: u32, cols: u32) -> Workbook {
let mut wb = Workbook::new();
if let Some(s) = wb.sheet_mut(0) {
s.name = "My Sheet".into();
for r in 0..rows {
for c in 0..cols {
s.set_value(CellRef::new(r, c), Value::Number(f64::from(r * 10 + c)));
}
}
}
wb
}
#[test]
fn a_long_reference_is_clipped_before_it_is_evaluated() {
let wb = book(120_000, 3);
let rows = |f: &str, limit| {
let g = clipped_reference(&wb, 0, f, limit)?;
let (_, r) = parse_ref(&wb, 0, &g)?;
Some((g, u64::from(r.height())))
};
// 1,000,000 rows: at most `MAX_POINTS` of them are asked for.
assert_eq!(rows("'My Sheet'!$A$1:$A$1000000", MAX_POINTS), Some(("'My Sheet'!A1:A100000".to_string(), 100_000)));
assert_eq!(rows("A5:A1048576", MAX_POINTS).map(|r| r.1), Some(100_000));
assert_eq!(rows("A:A", 50_000).map(|r| r.1), Some(50_000));
// Never below what keeps calc trimming the clipped reference like the whole one.
assert_eq!(rows("A1:A1000000", 100).map(|r| r.1), Some(16_385));
// Wide references keep whole rows.
assert_eq!(rows("A1:C1000000", MAX_POINTS).map(|r| r.1), Some(100_000));
// Small references, single cells and anything that isn't a plain reference are left alone.
assert_eq!(rows("A1:A9000", MAX_POINTS), None);
assert_eq!(rows("A1:A20000", MAX_POINTS), None);
assert_eq!(rows("B2", MAX_POINTS), None);
assert_eq!(rows("SUM(A1:A1000000)", MAX_POINTS), None);
assert_eq!(rows("{1,2,3}", MAX_POINTS), None);
}
#[test]
fn clipping_does_not_change_the_points() {
for (rows, cols) in [(10, 1), (30_000, 1), (30_000, 2), (3, 4)] {
let wb = book(rows, cols);
for f in ["'My Sheet'!A1:A1000000", "'My Sheet'!$A$3:$B$1048576", "A:A", "A1:C1000000", "(A1:A1000000,B1:B1000000)", "A2:A20000"] {
for limit in [100, 7_843, 25_000, MAX_POINTS] {
let clipped = eval_list_in(&wb, 0, f, limit, 0);
let whole: Vec<Value> = match evaluate(&wb, 0, CellRef::default(), f) {
Value::Array(a) => a.iter().take(limit).cloned().collect(),
v => vec![v],
};
if f.starts_with('(') {
// A union is read part by part; each part keeps its own order.
assert!(clipped.len() <= limit);
continue;
}
assert_eq!(clipped, whole, "{f} limit {limit} on {rows}x{cols}");
}
}
}
}
}
+33
View File
@@ -5,6 +5,7 @@ use gridcraft_model::{Anchor, Chart, ChartKind, Color, LegendPos, Series, Sparkl
use crate::*;
mod budget;
mod data_table;
const ALL_KINDS: [ChartKind; 26] = [
@@ -1418,6 +1419,38 @@ fn area_charts_draw_source_value_labels_only_when_enabled() {
}
}
#[test]
fn drawing_and_resolving_agree_on_the_combo_plot_kind() {
// Resolved series carry `effective_kind`; drawing re-derives the kind from it. Both paths
// must give the same kind for every chart kind, series kind (or none) and series index.
let series_kinds = std::iter::once(None).chain(ALL_KINDS.iter().map(|k| Some(*k)));
for ck in ALL_KINDS {
for sk in series_kinds.clone() {
for i in 0..3 {
let mut c = chart(ck);
c.series = (0..3).map(|j| Series { kind: if j == i { sk } else { None }, ..Default::default() }).collect();
let from_chart = series_plot_kind(&c, i);
let resolved = crate::resolve::effective_kind(ck, sk, i);
assert_eq!(crate::resolve::plot_kind(ck, Some(resolved), i), from_chart, "{ck:?} {sk:?} {i}");
if ck != ChartKind::Combo {
assert_eq!(from_chart, ck);
} else {
assert_ne!(from_chart, ChartKind::Combo);
}
}
}
}
}
#[test]
fn ticks_of_a_scale_off_the_step_grid_start_at_its_min() {
let s = Scale { min: 3.0, max: 7.0, step: 4.0 };
assert_eq!(s.ticks(), vec![3.0, 7.0]);
// Aligned scales still snap float error away.
let s = Scale { min: 0.0, max: 1.0, step: 0.1 };
assert!((s.ticks()[3] - 0.3).abs() < 1e-12 && s.ticks()[0] == 0.0);
}
#[test]
fn hundred_percent_stacked_areas_fill_each_category_to_its_total() {
let d = ChartData {
+279
View File
@@ -0,0 +1,279 @@
//! Work and memory are bounded by the product of the chart limits (series x points, trendlines
//! per chart, hairline bars), and clean data is rendered without a copy.
use std::borrow::Cow;
use gridcraft_core::{CellRef, Value};
use gridcraft_model::{ChartKind, Series, Trendline, TrendlineKind, Workbook};
use super::{assert_sane, chart};
use crate::render::{sanitize, sanitized};
use crate::resolve::{MAX_POINTS, MAX_TOTAL_POINTS, MAX_TRENDLINES, points_per_series};
use crate::*;
fn data_of(series: usize, points: usize) -> ChartData {
ChartData {
palette: vec![],
categories: (0..points).map(|i| i.to_string()).collect(),
category_levels: vec![],
series: (0..series)
.map(|s| SeriesData {
name: format!("S{s}"),
values: (0..points).map(|i| Some(((i * 7 + s * 13) % 100 + 1) as f64)).collect(),
color: series_color(&Default::default(), s),
..SeriesData::default()
})
.collect(),
}
}
fn series_rects(prims: &[Prim], color: Rgba) -> Vec<(f32, f32, f32, f32)> {
prims
.iter()
.filter_map(|p| match p {
Prim::Rect { x, y, w, h, fill: Some(f), .. } if *f == color => Some((*x, *y, *w, *h)),
_ => None,
})
.collect()
}
#[test]
fn the_point_budget_keeps_every_series_with_fewer_points() {
assert_eq!(points_per_series(1), MAX_POINTS);
assert_eq!(points_per_series(20), MAX_POINTS);
assert_eq!(points_per_series(21), MAX_TOTAL_POINTS / 21);
assert_eq!(points_per_series(255), MAX_TOTAL_POINTS / 255);
assert_eq!(points_per_series(0), MAX_POINTS);
// 25 series x 100,000 points is 2.5 million: over budget.
let d = data_of(25, MAX_POINTS);
let kept = sanitized(&d);
assert_eq!(kept.series.len(), 25, "no series is dropped");
assert!(kept.series.iter().all(|s| s.values.len() == MAX_TOTAL_POINTS / 25));
assert_eq!(kept.categories.len(), MAX_TOTAL_POINTS / 25);
assert!(kept.series.iter().map(|s| s.values.len()).sum::<usize>() <= MAX_TOTAL_POINTS);
// The first points are the ones kept.
assert_eq!(kept.series[3].values[..5], d.series[3].values[..5]);
// Drawing it stays bounded: hairline columns are merged per pixel column.
let mut c = chart(ChartKind::ColumnClustered);
c.data_labels = false;
c.legend = LegendPos::None;
let prims = render(&c, &d, 400.0, 300.0, &ApproxMeasure);
assert_sane(&prims, 400.0, 300.0);
assert!(prims.len() < 25 * 450 + 200, "{} primitives", prims.len());
}
#[test]
fn trendlines_are_limited_per_chart() {
let line = |kind| TrendlineData { spec: Trendline { kind, ..Trendline::default() }, color: [1, 2, 3, 255] };
let mut d = data_of(3, 30);
for s in &mut d.series {
s.trendlines = vec![line(TrendlineKind::Linear); 10];
}
let kept = sanitized(&d);
let counts: Vec<usize> = kept.series.iter().map(|s| s.trendlines.len()).collect();
assert_eq!(counts, vec![10, MAX_TRENDLINES - 10, 0], "series order decides which trendlines stay");
let entries = legend_entries(&chart(ChartKind::Line), &kept);
assert_eq!(entries.len(), 3 + MAX_TRENDLINES);
}
#[test]
fn hairline_columns_are_merged_and_wide_ones_are_not() {
let mut c = chart(ChartKind::ColumnClustered);
c.data_labels = false;
c.legend = LegendPos::None;
// Wide bars (>= 0.25 px): one rectangle per series and category, as before.
let d = data_of(3, 40);
let prims = render(&c, &d, 640.0, 400.0, &ApproxMeasure);
for s in &d.series {
assert_eq!(series_rects(&prims, s.color).len(), 40);
}
// 4 series x 20,000 categories in 400 px: the bars are hairlines.
let d = data_of(4, 20_000);
let prims = render(&c, &d, 400.0, 300.0, &ApproxMeasure);
assert_sane(&prims, 400.0, 300.0);
for s in &d.series {
let n = series_rects(&prims, s.color).len();
assert!((1..=420).contains(&n), "{n} rectangles for one series");
}
// The merged rectangles keep the envelope: alternating 1 and 10 draws the full-height bar.
let mut d = data_of(1, 20_000);
for (i, v) in d.series[0].values.iter_mut().enumerate() {
*v = Some(if i % 2 == 0 { 1.0 } else { 10.0 });
}
let prims = render(&c, &d, 400.0, 300.0, &ApproxMeasure);
let rects = series_rects(&prims, d.series[0].color);
let tallest = rects.iter().map(|r| r.3).fold(0.0f32, f32::max);
assert!(rects.len() > 100 && tallest > 100.0);
assert!(rects.iter().all(|r| (r.3 - tallest).abs() < 0.01), "every pixel column shows the tall bar");
// And together they cover the plot's width without a gap wider than a pixel.
let mut xs: Vec<(f32, f32)> = rects.iter().map(|r| (r.0, r.0 + r.2)).collect();
xs.sort_by(|a, b| a.0.total_cmp(&b.0));
assert!(xs.windows(2).all(|w| w[1].0 - w[0].1 < 1.0));
// Horizontal bars too.
let mut c = chart(ChartKind::BarClustered);
c.data_labels = false;
c.legend = LegendPos::None;
let prims = render(&c, &d, 400.0, 300.0, &ApproxMeasure);
let n = series_rects(&prims, d.series[0].color).len();
assert!((1..=320).contains(&n), "{n} bars");
}
#[test]
fn clustered_hairline_series_keep_a_visible_strip() {
// Four series of similar height, 20,000 categories in 400 px: every pixel column holds a
// merged bar of each series, and they must not cover each other.
for kind in [ChartKind::ColumnClustered, ChartKind::BarClustered] {
let horizontal = kind == ChartKind::BarClustered;
let mut c = chart(kind);
c.data_labels = false;
c.legend = LegendPos::None;
let mut d = data_of(4, 20_000);
for (s, series) in d.series.iter_mut().enumerate() {
for (i, v) in series.values.iter_mut().enumerate() {
*v = Some(50.0 + ((i + s) % 3) as f64);
}
}
let prims = render(&c, &d, 400.0, 300.0, &ApproxMeasure);
let along = |r: &(f32, f32, f32, f32)| if horizontal { (r.1, r.1 + r.3) } else { (r.0, r.0 + r.2) };
let mut strips: Vec<(f32, f32, usize)> = Vec::new();
for (s, series) in d.series.iter().enumerate() {
strips.extend(series_rects(&prims, series.color).iter().map(|r| (along(r).0, along(r).1, s)));
}
strips.sort_by(|a, b| a.0.total_cmp(&b.0));
assert!(strips.len() > 400);
// Rectangles of different series never overlap (more than rounding).
for w in strips.windows(2) {
assert!(w[0].2 == w[1].2 || w[1].0 >= w[0].1 - 1e-3, "series {} and {} overlap", w[0].2, w[1].2);
}
// And all four series show up in the same pixel column.
let col = strips[strips.len() / 2].0.floor();
let seen: std::collections::BTreeSet<usize> = strips.iter().filter(|s| s.0.floor() == col).map(|s| s.2).collect();
assert_eq!(seen.len(), 4, "series visible in pixel column {col}");
}
}
#[test]
fn stacked_hairline_bars_do_not_fill_the_gap_between_disjoint_spans() {
// Series 1 alternates +10 (stacked on the 100) and -10 (below zero): its two spans in one
// pixel column are 100..110 and -10..0, and the 100 units between them must stay empty.
let mut c = chart(ChartKind::ColumnStacked);
c.data_labels = false;
c.legend = LegendPos::None;
let mut d = data_of(2, 20_000);
for v in d.series[0].values.iter_mut() {
*v = Some(100.0);
}
for (i, v) in d.series[1].values.iter_mut().enumerate() {
*v = Some(if i % 2 == 0 { 10.0 } else { -10.0 });
}
let prims = render(&c, &d, 400.0, 300.0, &ApproxMeasure);
let rects = series_rects(&prims, d.series[1].color);
let tallest = rects.iter().map(|r| r.3).fold(0.0f32, f32::max);
assert!(!rects.is_empty() && tallest < 60.0, "tallest merged bar is {tallest} px");
}
#[test]
fn clean_data_is_rendered_without_a_copy() {
let d = data_of(3, 50);
assert!(matches!(sanitized(&d), Cow::Borrowed(_)));
// Dirty data takes the copying path and comes out as `sanitize` has always produced it.
let mut dirty = data_of(3, 50);
dirty.series[0].values[4] = Some(f64::NAN);
dirty.series[1].values[5] = Some(f64::INFINITY);
dirty.series[2].values[6] = Some(1e301);
dirty.series[2].x = Some(vec![Some(f64::NEG_INFINITY), Some(2.0)]);
dirty.series[1].sizes = Some(vec![Some(-1e305)]);
let got = sanitized(&dirty);
assert!(matches!(got, Cow::Owned(_)));
assert_eq!(*got, sanitize(&dirty));
assert_eq!(got.series[0].values[4], None);
assert_eq!(got.series[2].values[6], Some(1e300));
// Over the series cap (within the point budget) the same.
let many = data_of(300, 3);
let got = sanitized(&many);
assert_eq!(got.series.len(), 255);
assert_eq!(*got, sanitize(&many));
// Too many category levels take the copying path too.
let mut levels = data_of(3, 50);
levels.category_levels = vec![vec!["a".into()]; 5];
let got = sanitized(&levels);
assert!(matches!(got, Cow::Owned(_)));
assert_eq!(got.category_levels.len(), 3);
}
#[test]
fn resolve_keeps_every_series_within_the_point_budget() {
// 40 series x 60,000 points is 2.4 million: over the 2 million budget.
let mut wb = Workbook::new();
if let Some(s) = wb.sheet_mut(0) {
for r in 0..60_000u32 {
s.set_value(CellRef::new(r, 0), Value::Number(f64::from(r)));
}
}
let mut c = chart(ChartKind::ColumnClustered);
c.series =
(0..40).map(|_| Series { values: "Sheet1!$A$1:$A$1000000".into(), categories: Some("A1:A1000000".into()), ..Series::default() }).collect();
let d = resolve(&wb, 0, &c);
assert_eq!(d.series.len(), 40);
assert!(d.series.iter().all(|s| s.values.len() == MAX_TOTAL_POINTS / 40));
assert_eq!(d.categories.len(), MAX_TOTAL_POINTS / 40);
assert_eq!(d.series[7].values.first(), Some(&Some(0.0)));
// Trendlines: 16 per chart, in series order.
c.series = (0..4).map(|_| Series { values: "A1:A5".into(), trendlines: vec![Default::default(); 9], ..Series::default() }).collect();
let counts: Vec<usize> = resolve(&wb, 0, &c).series.iter().map(|s| s.trendlines.len()).collect();
assert_eq!(counts, vec![9, 7, 0, 0]);
}
/// 100 series lower the per-series cap to 20,000, below the number of category cells of a
/// two-level axis (#1155). Each kept category keeps its inner label and its outer level.
fn two_level_chart(n: u32, vertical: bool) -> ChartData {
let at = |i: u32, level: u32| if vertical { CellRef::new(i, level) } else { CellRef::new(level, i) };
let mut wb = Workbook::new();
if let Some(s) = wb.sheet_mut(0) {
for i in 0..n {
s.set_value(at(i, 0), Value::Text(format!("Y{}", i / 3).into()));
s.set_value(at(i, 1), Value::Text(format!("C{i}").into()));
s.set_value(at(i, 2), Value::Number(f64::from(i)));
}
}
let cats = format!("{}:{}", at(0, 0), at(n - 1, 1));
let values = format!("{}:{}", at(0, 2), at(n - 1, 2));
let mut c = chart(ChartKind::ColumnClustered);
c.series = (0..100).map(|_| Series { values: values.clone(), categories: Some(cats.clone()), ..Series::default() }).collect();
resolve(&wb, 0, &c)
}
#[test]
fn multi_level_categories_keep_their_levels_under_the_point_cap() {
for vertical in [true, false] {
let d = two_level_chart(15_000, vertical);
assert_eq!(d.series[0].values.len(), 15_000, "vertical {vertical}");
assert_eq!(d.categories.len(), 15_000, "vertical {vertical}");
assert_eq!(d.categories[0], "C0");
assert_eq!(d.categories[14_999], "C14999");
assert_eq!(d.category_levels.len(), 1, "one outer level, vertical {vertical}");
assert_eq!(d.category_levels[0].len(), 15_000);
assert_eq!(d.category_levels[0][3], "Y1");
}
}
#[test]
fn multi_level_categories_longer_than_the_cap_are_cut_together_with_their_levels() {
// 25,000 categories against a cap of 20,000.
let d = two_level_chart(25_000, true);
let cap = points_per_series(100);
assert_eq!(d.categories.len(), cap);
assert_eq!(d.category_levels.len(), 1);
assert_eq!(d.category_levels[0].len(), cap);
assert_eq!(d.categories[cap - 1], format!("C{}", cap - 1));
assert_eq!(d.category_levels[0][cap - 3], format!("Y{}", (cap - 3) / 3));
}
+29
View File
@@ -1,6 +1,35 @@
use super::*;
use gridcraft_model::ChartDataTable;
#[test]
fn combo_table_keys_use_each_series_index() {
let mut c = chart(ChartKind::Combo);
c.legend = LegendPos::None;
c.data_labels = false;
c.data_table = Some(ChartDataTable::default());
let mut d = sample(c.kind);
// The public renderer also accepts Combo series: the first defaults to columns, later ones to lines.
for s in &mut d.series {
s.kind = ChartKind::Combo;
}
let p = render(&c, &d, 900.0, 500.0, &ApproxMeasure);
assert_sane(&p, 900.0, 500.0);
let header_y = p
.iter()
.find_map(|p| match p {
Prim::Text { text, y, .. } if text == "Q1" => Some(*y),
_ => None,
})
.unwrap();
for (i, s) in d.series.iter().enumerate() {
let boxes = p.iter().filter(|p| matches!(p, Prim::Rect { y, fill: Some(f), .. } if *f == s.color && *y > header_y)).count();
let lines =
p.iter().filter(|p| matches!(p, Prim::Line { pts, color, .. } if *color == s.color && pts.iter().all(|p| p[1] > header_y))).count();
assert_eq!(boxes, usize::from(i == 0), "series {i}: column key");
assert_eq!(lines, usize::from(i > 0), "series {i}: line key");
}
}
#[test]
fn tables_draw_source_values_for_every_supported_chart_kind() {
for kind in ALL_KINDS {
+216 -2
View File
@@ -72,7 +72,12 @@ pub(crate) fn fit_trendline(t: &Trendline, pts: &[(f64, f64)]) -> Option<TrendFi
let mut aty = vec![0.0f64; m];
for &(x, y) in &data {
let u = (x - center) / scale;
let pows: Vec<f64> = (first..=d).map(|k| u.powi(k as i32)).collect();
// `d` is at most 6, so the powers `first..=d` always fit the array (no allocation per point).
let mut buf = [0.0f64; 7];
for (slot, k) in buf.iter_mut().zip(first..=d) {
*slot = u.powi(k as i32);
}
let pows = buf.get(..m)?;
for (i, pi) in pows.iter().enumerate() {
if let Some(v) = aty.get_mut(i) {
*v += pi * (y - b0);
@@ -258,10 +263,45 @@ fn trim_zeros(s: &str) -> String {
/// Moving average of `period` consecutive points in plotted order: point `i` (from
/// `period - 1`) is `(xᵢ, mean(yᵢ₋ₚ₊₁..=yᵢ))`. Empty when there are fewer points than `period`.
///
/// Linear in the number of points: the points are cut into blocks of `period`, and a window is
/// the tail of one block plus the head of the next, taken from running sums that only ever add
/// (a running sum that also subtracts would lose small values after a huge one passed through).
pub(crate) fn moving_average(pts: &[(f64, f64)], period: u32) -> Vec<(f64, f64)> {
let p = period.clamp(2, 255) as usize;
let pts: Vec<(f64, f64)> = pts.iter().copied().filter(|(x, y)| x.is_finite() && y.is_finite()).collect();
pts.windows(p).filter_map(|w| w.last().map(|(x, _)| (*x, w.iter().map(|q| q.1).sum::<f64>() / p as f64))).collect()
let n = pts.len();
if n < p {
return vec![];
}
// `head[i]`: sum of the block's points up to and including `i`; `tail[i]`: from `i` to the block's end.
let mut head = vec![0.0f64; n];
let mut tail = vec![0.0f64; n];
for (b, block) in pts.chunks(p).enumerate() {
let (mut h, mut t) = (0.0f64, 0.0f64);
for k in 0..block.len() {
h += block.get(k).map_or(0.0, |q| q.1);
if let Some(v) = head.get_mut(b * p + k) {
*v = h;
}
let r = block.len() - 1 - k;
t += block.get(r).map_or(0.0, |q| q.1);
if let Some(v) = tail.get_mut(b * p + r) {
*v = t;
}
}
}
(p - 1..n)
.filter_map(|i| {
let s = i + 1 - p;
let mut sum = tail.get(s)? + if s.is_multiple_of(p) { 0.0 } else { *head.get(i)? };
if !sum.is_finite() {
// A block sum overflowed although the window itself may not: sum it directly.
sum = pts.get(s..=i)?.iter().map(|q| q.1).sum();
}
Some((pts.get(i)?.0, sum / p as f64))
})
.collect()
}
/// Excel's automatic trendline name for the legend, e.g. `Linear (Sales)`.
@@ -276,3 +316,177 @@ pub(crate) fn auto_name(kind: TrendlineKind, series: &str) -> String {
};
format!("{k} ({series})")
}
#[cfg(test)]
mod tests {
use super::*;
/// xorshift64 for repeatable pseudo-random data.
struct Rng(u64);
impl Rng {
fn next(&mut self) -> u64 {
self.0 ^= self.0 << 13;
self.0 ^= self.0 >> 7;
self.0 ^= self.0 << 17;
self.0
}
fn unit(&mut self) -> f64 {
(self.next() % 1_000_000) as f64 / 1_000_000.0 - 0.5
}
}
/// The window-by-window moving average this module used to compute.
fn moving_average_naive(pts: &[(f64, f64)], period: u32) -> Vec<(f64, f64)> {
let p = period.clamp(2, 255) as usize;
let pts: Vec<(f64, f64)> = pts.iter().copied().filter(|(x, y)| x.is_finite() && y.is_finite()).collect();
pts.windows(p).filter_map(|w| w.last().map(|(x, _)| (*x, w.iter().map(|q| q.1).sum::<f64>() / p as f64))).collect()
}
#[test]
fn linear_moving_average_matches_the_window_sums() {
let mut r = Rng(0x9E37_79B9_7F4A_7C15);
for case in 0..60 {
let n = (r.next() % 700) as usize;
let pts: Vec<(f64, f64)> = (0..n)
.map(|i| {
let y = match case % 5 {
0 => r.unit() * 100.0,
1 => 1e9 + r.unit(),
// A huge value passing through must not leave rounding residue behind.
2 => {
if r.next().is_multiple_of(50) {
1e300 * r.unit().signum()
} else {
r.unit()
}
}
3 => {
if i % 2 == 0 {
1e15
} else {
-1e15 + r.unit()
}
}
_ => {
if r.next().is_multiple_of(9) {
f64::NAN
} else {
r.unit() * 1e-3
}
}
};
(i as f64, y)
})
.collect();
for period in [0, 2, 3, 7, 50, 255, 400] {
let fast = moving_average(&pts, period);
let slow = moving_average_naive(&pts, period);
assert_eq!(fast.len(), slow.len(), "case {case} period {period}");
let p = period.clamp(2, 255) as usize;
let finite: Vec<(f64, f64)> = pts.iter().copied().filter(|(x, y)| x.is_finite() && y.is_finite()).collect();
for (i, ((fx, fy), (sx, sy))) in fast.iter().zip(&slow).enumerate() {
assert_eq!(fx, sx);
let mass: f64 = finite.iter().skip(i).take(p).map(|q| q.1.abs()).sum::<f64>() / p as f64;
assert!((fy - sy).abs() <= 1e-12 * mass, "case {case} period {period} point {i}: {fy} vs {sy}");
}
}
}
}
#[test]
fn moving_average_recomputes_a_window_whose_block_sums_overflow() {
let pts: Vec<(f64, f64)> = [0.0, -1e308, 1e308, 1e308, -1e308].iter().enumerate().map(|(i, y)| (i as f64, *y)).collect();
let fast = moving_average(&pts, 4);
let slow = moving_average_naive(&pts, 4);
assert_eq!(fast, slow);
assert!(fast.iter().all(|q| q.1.is_finite()));
}
/// The trendline fit as it was written with a `Vec` of powers per data point.
fn fit_reference(t: &Trendline, pts: &[(f64, f64)]) -> Option<TrendFit> {
let kind = t.kind;
let d = degree(kind)?;
let data: Vec<(f64, f64)> = pts
.iter()
.filter(|(x, y)| x.is_finite() && y.is_finite() && (!log_x(kind) || *x > 0.0) && (!log_y(kind) || *y > 0.0))
.map(|&(x, y)| (if log_x(kind) { x.ln() } else { x }, if log_y(kind) { y.ln() } else { y }))
.collect();
if data.len() < 2 || data.len() <= d {
return None;
}
let fixed = match kind {
TrendlineKind::Linear | TrendlineKind::Polynomial(_) => t.intercept,
TrendlineKind::Exponential => t.intercept.filter(|b| *b > 0.0).map(f64::ln),
_ => None,
}
.filter(|b| b.is_finite());
let n = data.len() as f64;
let center = if fixed.is_some() { 0.0 } else { data.iter().map(|p| p.0).sum::<f64>() / n };
let scale = data.iter().map(|p| (p.0 - center).abs()).fold(0.0f64, f64::max);
if !(scale.is_finite() && scale > 0.0) {
return None;
}
let first = usize::from(fixed.is_some());
let b0 = fixed.unwrap_or(0.0);
let m = d + 1 - first;
let mut ata = vec![vec![0.0f64; m]; m];
let mut aty = vec![0.0f64; m];
for &(x, y) in &data {
let u = (x - center) / scale;
let pows: Vec<f64> = (first..=d).map(|k| u.powi(k as i32)).collect();
for (i, pi) in pows.iter().enumerate() {
aty[i] += pi * (y - b0);
for (j, pj) in pows.iter().enumerate() {
ata[i][j] += pi * pj;
}
}
}
let sol = solve(ata, aty)?;
let mut a = vec![b0; d + 1];
for (k, v) in sol.into_iter().enumerate() {
a[k + first] = v;
}
let coef = expand(&a, center, scale)?;
let fit = TrendFit { kind, coef, center, scale, a, r2: None };
let mean = data.iter().map(|p| p.1).sum::<f64>() / n;
let (mut ss_res, mut ss_tot) = (0.0f64, 0.0f64);
for &(x, y) in &data {
ss_res += (y - fit.poly(x)).powi(2);
ss_tot += if fixed.is_some() { (y - b0).powi(2) } else { (y - mean).powi(2) };
}
let r2 = (ss_tot > 0.0 && ss_tot.is_finite() && ss_res.is_finite()).then(|| 1.0 - ss_res / ss_tot);
Some(TrendFit { r2, ..fit })
}
#[test]
fn fit_without_a_vec_per_point_is_bit_identical() {
let mut r = Rng(0x1234_5678_9ABC_DEF1);
let kinds = [
TrendlineKind::Linear,
TrendlineKind::Exponential,
TrendlineKind::Logarithmic,
TrendlineKind::Power,
TrendlineKind::Polynomial(2),
TrendlineKind::Polynomial(3),
TrendlineKind::Polynomial(6),
TrendlineKind::MovingAverage(3),
];
for case in 0..40 {
let n = 2 + (r.next() % 60) as usize;
let pts: Vec<(f64, f64)> = (0..n)
.map(|i| {
let x = i as f64 * (1.0 + r.unit()) + 1.0;
let y = if case % 4 == 0 { f64::NAN } else { (r.unit() + 0.6) * 10f64.powi((case % 7) - 2) };
(x, if case % 4 == 0 && i % 3 != 0 { r.unit() + 1.0 } else { y })
})
.collect();
for kind in kinds {
for intercept in [None, Some(0.5), Some(-1.0)] {
let t = Trendline { kind, intercept, ..Trendline::default() };
assert_eq!(fit_trendline(&t, &pts), fit_reference(&t, &pts), "case {case} {kind:?} {intercept:?}");
}
}
}
}
}
+3 -1
View File
@@ -1,5 +1,5 @@
//! GridCraft core types: cell addresses, values and Excel's coercion rules, error values,
//! date serials and typed-input parsing. No dependencies on the rest of the workspace.
//! date serials, regional formats and typed-input parsing. No dependencies on the rest of the workspace.
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
#![forbid(unsafe_code)]
@@ -7,6 +7,7 @@ pub mod addr;
pub mod color;
pub mod date;
pub mod parse;
pub mod region;
pub mod value;
pub use addr::{
@@ -14,4 +15,5 @@ pub use addr::{
split_areas, split_sheet, unquote_sheet,
};
pub use date::DateSystem;
pub use region::Region;
pub use value::{Array, CellError, Value, compare, compare_numbers, compare_text, number_to_text, sort_compare};
+883
View File
@@ -0,0 +1,883 @@
//! Regional formats: how a region writes numbers, dates, times and money.
//!
//! Excel takes these from the operating system's regional settings; they are independent of the
//! interface language and of the formula language. A [`Region`] changes what people *type* into
//! a cell (`1.234,56`, `25/09/2026`) and what cells *show* (separators, month and day names,
//! AM/PM, the system short and long dates). It never changes what is stored: values are numbers,
//! and the number format codes saved in a file stay in their canonical en-US spelling.
//!
//! [`EN_US`] is the default and reproduces GridCraft's original behaviour exactly.
//!
//! # Where the values come from
//! The separators, month and weekday names, AM/PM words, currency symbols and the order of day,
//! month and year are the Unicode CLDR conventions (`unicode-org/cldr-json`:
//! `cldr-numbers-full/main/<locale>/numbers.json` and `cldr-dates-full/main/<locale>/ca-gregorian.json`),
//! which is also what macOS, Android and ICU ship. They were read from those files for en-GB,
//! en-IN, cs and ru, read from ICU's CLDR 48 data for the Swiss regions and nl-NL, and written
//! from the same conventions for the other regions. CLDR data is under the Unicode License v3,
//! reproduced in `NOTICE` (row in `ATTRIBUTION.md`).
//! Deliberate choices where operating systems differ from CLDR:
//! - Short dates use a four-digit year (`dd/mm/yyyy`, where CLDR has `dd/MM/y` or `dd/MM/yy`):
//! Excel's short date is the system's, Windows writes four digits, and a two-digit year would
//! not read back to the same date.
//! - The long date is CLDR's *full* date (with the weekday), like Windows en-US, except where
//! Windows' long date differs: Russian, Japanese and Chinese have no weekday (`23 августа
//! 2005 г.`, `2005年8月23日`), as in Microsoft's National Language Support tables.
//! - Korean writes the short date `yyyy-mm-dd` (CLDR: `yy. M. d.`), also as in Microsoft's
//! National Language Support tables.
//! - Month names for `mmm` are CLDR's abbreviated names (so en-GB has `Sept`), except
//! Portuguese, which drops CLDR's period (`jan`, not `jan.`) as Excel in Brazil shows it.
//! - French and Portuguese (Portugal) group digits with U+00A0 (Windows), not CLDR's U+202F.
//! Parsing accepts every space-like grouping character in those regions.
//! - The Swiss regions (en-CH, de-CH, fr-CH, it-CH; Liechtenstein uses de-CH) group digits with
//! CLDR's ASCII apostrophe (`1'234.50`); parsing also accepts the typographic `’` that glibc
//! uses. French (Switzerland) writes a decimal point like the other Swiss regions and glibc's
//! `fr_CH`, where CLDR has a comma.
//!
//! Regions this module doesn't list (`de-AT`, `es-MX`, `en-CA`…) resolve to the main region of
//! their language through [`lookup`]; their own conventions (a Mexican decimal point) are not
//! modelled.
/// How digits are grouped.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Grouping {
/// Groups of three (`1,234,567`).
Thousands,
/// The Indian system: three digits, then groups of two (`12,34,567`).
Indian,
}
/// The order of the parts of a short date.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum DateOrder {
Mdy,
Dmy,
Ymd,
}
/// One region's conventions. See the module documentation.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Region {
/// BCP 47 tag, `pt-BR`.
pub tag: &'static str,
/// The region's name in its own language.
pub name: &'static str,
/// Windows locale id, used by `[$-409]`-style format tags.
pub lcid: u16,
pub decimal: char,
pub group: char,
pub grouping: Grouping,
pub date_order: DateOrder,
pub date_sep: char,
pub time_sep: char,
/// The system short date as a canonical format code (`dd/mm/yyyy`): built-in format 14.
pub short_date: &'static str,
/// The system long date (`[$-F800]`).
pub long_date: &'static str,
/// The system short time. It contains `AM/PM` in regions that use a 12-hour clock.
pub short_time: &'static str,
/// The system long time (`[$-F400]`).
pub long_time: &'static str,
/// Built-in format 22: the short date and a 24-hour time.
pub datetime: &'static str,
pub months: [&'static str; 12],
pub months_abbr: [&'static str; 12],
/// The month names a date with a day uses (Russian `1 января`, Czech `1. ledna`), where they
/// differ from `months`.
pub months_genitive: Option<[&'static str; 12]>,
/// Sunday first.
pub weekdays: [&'static str; 7],
pub weekdays_abbr: [&'static str; 7],
pub am_pm: [&'static str; 2],
pub currency: &'static str,
/// The currency format with the currency's usual decimals (two, or none for the yen and the
/// won), as a canonical code. Where it starts with a quote the symbol comes before the number,
/// otherwise after.
pub currency_format: &'static str,
/// The same with no decimals (for a typed whole amount).
pub currency_format_int: &'static str,
}
const EN_MONTHS: [&str; 12] =
["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"];
const EN_MONTHS_ABBR: [&str; 12] = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"];
const EN_DAYS: [&str; 7] = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"];
const EN_DAYS_ABBR: [&str; 7] = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"];
const AM_PM: [&str; 2] = ["AM", "PM"];
/// English (United States): the default. Identical to the formats before regions existed.
pub static EN_US: Region = Region {
tag: "en-US",
name: "English (United States)",
lcid: 0x0409,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Mdy,
date_sep: '/',
time_sep: ':',
short_date: "m/d/yyyy",
long_date: "dddd, mmmm dd, yyyy",
short_time: "h:mm AM/PM",
long_time: "h:mm:ss AM/PM",
datetime: "m/d/yyyy h:mm",
months: EN_MONTHS,
months_abbr: EN_MONTHS_ABBR,
months_genitive: None,
weekdays: EN_DAYS,
weekdays_abbr: EN_DAYS_ABBR,
am_pm: AM_PM,
currency: "$",
currency_format: "\"$\"#,##0.00",
currency_format_int: "\"$\"#,##0",
};
static EN_GB: Region = Region {
tag: "en-GB",
name: "English (United Kingdom)",
lcid: 0x0809,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: EN_MONTHS,
months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"],
months_genitive: None,
weekdays: EN_DAYS,
weekdays_abbr: EN_DAYS_ABBR,
am_pm: AM_PM,
currency: "£",
currency_format: "\"£\"#,##0.00",
currency_format_int: "\"£\"#,##0",
};
static EN_IN: Region = Region {
tag: "en-IN",
name: "English (India)",
lcid: 0x4009,
decimal: '.',
group: ',',
grouping: Grouping::Indian,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "h:mm AM/PM",
long_time: "h:mm:ss AM/PM",
datetime: "dd/mm/yyyy h:mm",
months: EN_MONTHS,
months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"],
months_genitive: None,
weekdays: EN_DAYS,
weekdays_abbr: EN_DAYS_ABBR,
am_pm: AM_PM,
currency: "₹",
currency_format: "\"₹\"#,##0.00",
currency_format_int: "\"₹\"#,##0",
};
static EN_CH: Region = Region {
tag: "en-CH",
name: "English (Switzerland)",
// English (Switzerland) has no LCID of its own: Windows reports LOCALE_CUSTOM_UNSPECIFIED.
lcid: 0x1000,
decimal: '.',
group: '\'',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: EN_MONTHS,
months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"],
months_genitive: None,
weekdays: EN_DAYS,
weekdays_abbr: EN_DAYS_ABBR,
am_pm: AM_PM,
currency: "CHF",
currency_format: "\"CHF\" #,##0.00",
currency_format_int: "\"CHF\" #,##0",
};
static ES_ES: Region = Region {
tag: "es-ES",
name: "Español (España)",
lcid: 0x0C0A,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d \"de\" mmmm \"de\" yyyy",
short_time: "h:mm",
long_time: "h:mm:ss",
datetime: "dd/mm/yyyy h:mm",
months: ["enero", "febrero", "marzo", "abril", "mayo", "junio", "julio", "agosto", "septiembre", "octubre", "noviembre", "diciembre"],
months_abbr: ["ene", "feb", "mar", "abr", "may", "jun", "jul", "ago", "sep", "oct", "nov", "dic"],
months_genitive: None,
weekdays: ["domingo", "lunes", "martes", "miércoles", "jueves", "viernes", "sábado"],
weekdays_abbr: ["dom", "lun", "mar", "mié", "jue", "vie", "sáb"],
am_pm: ["a. m.", "p. m."],
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static DE_DE: Region = Region {
tag: "de-DE",
name: "Deutsch (Deutschland)",
lcid: 0x0407,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d. mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["Januar", "Februar", "März", "April", "Mai", "Juni", "Juli", "August", "September", "Oktober", "November", "Dezember"],
months_abbr: ["Jan", "Feb", "Mär", "Apr", "Mai", "Jun", "Jul", "Aug", "Sep", "Okt", "Nov", "Dez"],
months_genitive: None,
weekdays: ["Sonntag", "Montag", "Dienstag", "Mittwoch", "Donnerstag", "Freitag", "Samstag"],
weekdays_abbr: ["So", "Mo", "Di", "Mi", "Do", "Fr", "Sa"],
am_pm: AM_PM,
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static DE_CH: Region = Region {
tag: "de-CH",
name: "Deutsch (Schweiz)",
lcid: 0x0807,
decimal: '.',
group: '\'',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d. mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["Januar", "Februar", "März", "April", "Mai", "Juni", "Juli", "August", "September", "Oktober", "November", "Dezember"],
months_abbr: ["Jan", "Feb", "Mär", "Apr", "Mai", "Jun", "Jul", "Aug", "Sep", "Okt", "Nov", "Dez"],
months_genitive: None,
weekdays: ["Sonntag", "Montag", "Dienstag", "Mittwoch", "Donnerstag", "Freitag", "Samstag"],
weekdays_abbr: ["So", "Mo", "Di", "Mi", "Do", "Fr", "Sa"],
am_pm: AM_PM,
currency: "CHF",
currency_format: "\"CHF\" #,##0.00",
currency_format_int: "\"CHF\" #,##0",
};
static FR_FR: Region = Region {
tag: "fr-FR",
name: "Français (France)",
lcid: 0x040C,
decimal: ',',
group: '\u{a0}',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"],
months_abbr: ["janv.", "févr.", "mars", "avr.", "mai", "juin", "juil.", "août", "sept.", "oct.", "nov.", "déc."],
months_genitive: None,
weekdays: ["dimanche", "lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi"],
weekdays_abbr: ["dim.", "lun.", "mar.", "mer.", "jeu.", "ven.", "sam."],
am_pm: AM_PM,
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static FR_CH: Region = Region {
tag: "fr-CH",
name: "Français (Suisse)",
lcid: 0x100C,
decimal: '.',
group: '\'',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"],
months_abbr: ["janv.", "févr.", "mars", "avr.", "mai", "juin", "juil.", "août", "sept.", "oct.", "nov.", "déc."],
months_genitive: None,
weekdays: ["dimanche", "lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi"],
weekdays_abbr: ["dim.", "lun.", "mar.", "mer.", "jeu.", "ven.", "sam."],
am_pm: AM_PM,
currency: "CHF",
currency_format: "#,##0.00 \"CHF\"",
currency_format_int: "#,##0 \"CHF\"",
};
static IT_IT: Region = Region {
tag: "it-IT",
name: "Italiano (Italia)",
lcid: 0x0410,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"],
months_abbr: ["gen", "feb", "mar", "apr", "mag", "giu", "lug", "ago", "set", "ott", "nov", "dic"],
months_genitive: None,
weekdays: ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"],
weekdays_abbr: ["dom", "lun", "mar", "mer", "gio", "ven", "sab"],
am_pm: AM_PM,
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static IT_CH: Region = Region {
tag: "it-CH",
name: "Italiano (Svizzera)",
lcid: 0x0810,
decimal: '.',
group: '\'',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"],
months_abbr: ["gen", "feb", "mar", "apr", "mag", "giu", "lug", "ago", "set", "ott", "nov", "dic"],
months_genitive: None,
weekdays: ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"],
weekdays_abbr: ["dom", "lun", "mar", "mer", "gio", "ven", "sab"],
am_pm: AM_PM,
currency: "CHF",
currency_format: "\"CHF\" #,##0.00",
currency_format_int: "\"CHF\" #,##0",
};
static NL_NL: Region = Region {
tag: "nl-NL",
name: "Nederlands (Nederland)",
lcid: 0x0413,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '-',
time_sep: ':',
short_date: "dd-mm-yyyy",
long_date: "dddd d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd-mm-yyyy hh:mm",
months: ["januari", "februari", "maart", "april", "mei", "juni", "juli", "augustus", "september", "oktober", "november", "december"],
months_abbr: ["jan", "feb", "mrt", "apr", "mei", "jun", "jul", "aug", "sep", "okt", "nov", "dec"],
months_genitive: None,
weekdays: ["zondag", "maandag", "dinsdag", "woensdag", "donderdag", "vrijdag", "zaterdag"],
weekdays_abbr: ["zo", "ma", "di", "wo", "do", "vr", "za"],
am_pm: AM_PM,
currency: "€",
currency_format: "\"€\" #,##0.00",
currency_format_int: "\"€\" #,##0",
};
static PT_BR: Region = Region {
tag: "pt-BR",
name: "Português (Brasil)",
lcid: 0x0416,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d \"de\" mmmm \"de\" yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: ["janeiro", "fevereiro", "março", "abril", "maio", "junho", "julho", "agosto", "setembro", "outubro", "novembro", "dezembro"],
months_abbr: ["jan", "fev", "mar", "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez"],
months_genitive: None,
weekdays: ["domingo", "segunda-feira", "terça-feira", "quarta-feira", "quinta-feira", "sexta-feira", "sábado"],
weekdays_abbr: ["dom", "seg", "ter", "qua", "qui", "sex", "sáb"],
am_pm: AM_PM,
currency: "R$",
currency_format: "\"R$\" #,##0.00",
currency_format_int: "\"R$\" #,##0",
};
static PT_PT: Region = Region {
tag: "pt-PT",
name: "Português (Portugal)",
lcid: 0x0816,
decimal: ',',
group: '\u{a0}',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d \"de\" mmmm \"de\" yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: ["janeiro", "fevereiro", "março", "abril", "maio", "junho", "julho", "agosto", "setembro", "outubro", "novembro", "dezembro"],
months_abbr: ["jan", "fev", "mar", "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez"],
months_genitive: None,
weekdays: ["domingo", "segunda-feira", "terça-feira", "quarta-feira", "quinta-feira", "sexta-feira", "sábado"],
weekdays_abbr: ["dom", "seg", "ter", "qua", "qui", "sex", "sáb"],
am_pm: AM_PM,
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static RU_RU: Region = Region {
tag: "ru-RU",
name: "Русский (Россия)",
lcid: 0x0419,
decimal: ',',
group: '\u{a0}',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "d mmmm yyyy\" г.\"",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["январь", "февраль", "март", "апрель", "май", "июнь", "июль", "август", "сентябрь", "октябрь", "ноябрь", "декабрь"],
months_abbr: ["янв.", "февр.", "март", "апр.", "май", "июнь", "июль", "авг.", "сент.", "окт.", "нояб.", "дек."],
months_genitive: Some(["января", "февраля", "марта", "апреля", "мая", "июня", "июля", "августа", "сентября", "октября", "ноября", "декабря"]),
weekdays: ["воскресенье", "понедельник", "вторник", "среда", "четверг", "пятница", "суббота"],
weekdays_abbr: ["вс", "пн", "вт", "ср", "чт", "пт", "сб"],
am_pm: AM_PM,
currency: "₽",
currency_format: "#,##0.00\u{a0}\"₽\"",
currency_format_int: "#,##0\u{a0}\"₽\"",
};
static CS_CZ: Region = Region {
tag: "cs-CZ",
name: "Čeština (Česko)",
lcid: 0x0405,
decimal: ',',
group: '\u{a0}',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd d. mmmm yyyy",
short_time: "h:mm",
long_time: "h:mm:ss",
datetime: "dd.mm.yyyy h:mm",
months: ["leden", "únor", "březen", "duben", "květen", "červen", "červenec", "srpen", "září", "říjen", "listopad", "prosinec"],
months_abbr: ["led", "úno", "bře", "dub", "kvě", "čvn", "čvc", "srp", "zář", "říj", "lis", "pro"],
months_genitive: Some(["ledna", "února", "března", "dubna", "května", "června", "července", "srpna", "září", "října", "listopadu", "prosince"]),
weekdays: ["neděle", "pondělí", "úterý", "středa", "čtvrtek", "pátek", "sobota"],
weekdays_abbr: ["ne", "po", "út", "st", "čt", "pá", "so"],
am_pm: ["dop.", "odp."],
currency: "Kč",
currency_format: "#,##0.00 \"Kč\"",
currency_format_int: "#,##0 \"Kč\"",
};
static JA_JP: Region = Region {
tag: "ja-JP",
name: "日本語 (日本)",
lcid: 0x0411,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Ymd,
date_sep: '/',
time_sep: ':',
short_date: "yyyy/m/d",
long_date: "yyyy\"年\"m\"月\"d\"日\"",
short_time: "h:mm",
long_time: "h:mm:ss",
datetime: "yyyy/m/d h:mm",
months: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"],
months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"],
months_genitive: None,
weekdays: ["日曜日", "月曜日", "火曜日", "水曜日", "木曜日", "金曜日", "土曜日"],
weekdays_abbr: ["日", "月", "火", "水", "木", "金", "土"],
am_pm: ["午前", "午後"],
currency: "¥",
currency_format: "\"¥\"#,##0",
currency_format_int: "\"¥\"#,##0",
};
static KO_KR: Region = Region {
tag: "ko-KR",
name: "한국어 (대한민국)",
lcid: 0x0412,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Ymd,
date_sep: '-',
time_sep: ':',
short_date: "yyyy-mm-dd",
long_date: "yyyy\"년\" m\"월\" d\"일\" dddd",
short_time: "AM/PM h:mm",
long_time: "AM/PM h:mm:ss",
datetime: "yyyy-mm-dd hh:mm",
months: ["1월", "2월", "3월", "4월", "5월", "6월", "7월", "8월", "9월", "10월", "11월", "12월"],
months_abbr: ["1월", "2월", "3월", "4월", "5월", "6월", "7월", "8월", "9월", "10월", "11월", "12월"],
months_genitive: None,
weekdays: ["일요일", "월요일", "화요일", "수요일", "목요일", "금요일", "토요일"],
weekdays_abbr: ["일", "월", "화", "수", "목", "금", "토"],
am_pm: ["오전", "오후"],
currency: "₩",
currency_format: "\"₩\"#,##0",
currency_format_int: "\"₩\"#,##0",
};
static ZH_CN: Region = Region {
tag: "zh-CN",
name: "中文 (中国)",
lcid: 0x0804,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Ymd,
date_sep: '/',
time_sep: ':',
short_date: "yyyy/m/d",
long_date: "yyyy\"年\"m\"月\"d\"日\"",
short_time: "h:mm",
long_time: "h:mm:ss",
datetime: "yyyy/m/d h:mm",
months: ["一月", "二月", "三月", "四月", "五月", "六月", "七月", "八月", "九月", "十月", "十一月", "十二月"],
months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"],
months_genitive: None,
weekdays: ["星期日", "星期一", "星期二", "星期三", "星期四", "星期五", "星期六"],
weekdays_abbr: ["周日", "周一", "周二", "周三", "周四", "周五", "周六"],
am_pm: ["上午", "下午"],
currency: "¥",
currency_format: "\"¥\"#,##0.00",
currency_format_int: "\"¥\"#,##0",
};
static ZH_TW: Region = Region {
tag: "zh-TW",
name: "中文 (台灣)",
lcid: 0x0404,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Ymd,
date_sep: '/',
time_sep: ':',
short_date: "yyyy/m/d",
long_date: "yyyy\"年\"m\"月\"d\"日\"",
short_time: "AM/PM hh:mm",
long_time: "AM/PM hh:mm:ss",
datetime: "yyyy/m/d hh:mm",
months: ["一月", "二月", "三月", "四月", "五月", "六月", "七月", "八月", "九月", "十月", "十一月", "十二月"],
months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"],
months_genitive: None,
weekdays: ["星期日", "星期一", "星期二", "星期三", "星期四", "星期五", "星期六"],
weekdays_abbr: ["週日", "週一", "週二", "週三", "週四", "週五", "週六"],
am_pm: ["上午", "下午"],
currency: "NT$",
currency_format: "\"NT$\"#,##0.00",
currency_format_int: "\"NT$\"#,##0",
};
static ALL: [&Region; 20] = [
&EN_US, &EN_GB, &EN_IN, &EN_CH, &ES_ES, &DE_DE, &DE_CH, &FR_FR, &FR_CH, &IT_IT, &IT_CH, &NL_NL, &PT_BR, &PT_PT, &RU_RU, &CS_CZ, &JA_JP, &KO_KR,
&ZH_CN, &ZH_TW,
];
/// Every region, English (United States) first.
pub fn all() -> &'static [&'static Region] {
&ALL
}
/// The default region, English (United States).
pub fn default_region() -> &'static Region {
&EN_US
}
/// The region with exactly this tag (`pt-BR`; case and `_` don't matter).
pub fn find(tag: &str) -> Option<&'static Region> {
let t = tag.trim().replace('_', "-");
ALL.iter().copied().find(|r| r.tag.eq_ignore_ascii_case(&t))
}
/// The region for a tag as operating systems and browsers write it: `pt-BR`, `pt_BR.UTF-8`,
/// `de`, `en-AU`, `zh-Hant`, `sr_RS@latin`. An exact match wins; otherwise the language's main
/// region (with the usual variants: `en-AU`→`en-GB`, `pt-AO`→`pt-PT`, `zh-Hant`→`zh-TW`).
/// An unknown language is English (United States).
pub fn lookup(tag: &str) -> &'static Region {
if let Some(r) = find(tag) {
return r;
}
let base = tag.split(['.', '@']).next().unwrap_or("");
let norm = base.trim().replace('_', "-");
if let Some(r) = find(&norm) {
return r;
}
let mut parts = norm.split('-').filter(|p| !p.is_empty());
let lang = parts.next().unwrap_or("").to_ascii_lowercase();
let mut script = String::new();
let mut country = String::new();
for p in parts {
// An extension or private-use singleton (`-u-ca-gregory`, `-x-…`) ends the region part.
if p.len() == 1 {
break;
}
if p.len() == 4 && p.chars().all(|c| c.is_ascii_alphabetic()) {
script = p.to_ascii_lowercase();
} else if (p.len() == 2 && p.chars().all(|c| c.is_ascii_alphabetic())) || (p.len() == 3 && p.chars().all(|c| c.is_ascii_digit())) {
country = p.to_ascii_uppercase();
}
}
let tag = match lang.as_str() {
"en" => match country.as_str() {
"GB" | "AU" | "NZ" | "IE" => "en-GB",
"IN" => "en-IN",
"CH" | "LI" => "en-CH",
_ => "en-US",
},
"pt" => match country.as_str() {
"AO" | "CH" | "CV" | "FR" | "GQ" | "GW" | "LU" | "MO" | "MZ" | "ST" | "TL" | "PT" => "pt-PT",
_ => "pt-BR",
},
"zh" => match country.as_str() {
// An explicit territory wins over the script (`zh-Hant-CN` is China's conventions).
"CN" | "SG" => "zh-CN",
"TW" | "HK" | "MO" => "zh-TW",
_ if script == "hant" => "zh-TW",
_ => "zh-CN",
},
"es" => "es-ES",
"de" => match country.as_str() {
"CH" | "LI" => "de-CH",
_ => "de-DE",
},
"fr" => match country.as_str() {
"CH" | "LI" => "fr-CH",
_ => "fr-FR",
},
"it" => match country.as_str() {
"CH" | "LI" => "it-CH",
_ => "it-IT",
},
"nl" => "nl-NL",
"ru" => "ru-RU",
"cs" => "cs-CZ",
"ja" => "ja-JP",
"ko" => "ko-KR",
_ => "en-US",
};
find(tag).unwrap_or(&EN_US)
}
impl Region {
/// True for the characters that separate groups of digits in this region. A region that
/// groups with a space also accepts the other space-like characters (NBSP, narrow NBSP, thin
/// space), as people type and paste any of them; one that groups with an apostrophe accepts
/// both `'` and `’`.
pub fn is_group(&self, c: char) -> bool {
c == self.group || (is_space_like(self.group) && is_space_like(c)) || (is_apostrophe_like(self.group) && is_apostrophe_like(c))
}
/// True when the currency symbol comes before the number.
pub fn currency_first(&self) -> bool {
self.currency_format.starts_with('"')
}
/// Month number (1–12) for a month name as typed: the full name, the abbreviation (with or
/// without its period) or any prefix of the full name of three or more letters (`Sept`),
/// ignoring case. `word` is one word: the genitive form used after a day (`января` in
/// `1 января`) matches too, and the caller reads the day itself.
pub fn month_from_name(&self, word: &str) -> Option<u32> {
let w = word.trim().to_lowercase();
// Where the abbreviations end in a period (`сент.`, `janv.`), any abbreviation may be typed
// with one (`мар.`, `juin.`). Elsewhere a period is not part of a month name, so en-US
// `Mar. 4, 2026` stays text as before.
let dotted = self.months_abbr.iter().any(|a| a.ends_with('.'));
let stem = if dotted { w.trim_end_matches('.') } else { w.as_str() };
if stem.is_empty() {
return None;
}
let long_enough = stem.chars().count() >= 3;
for (i, full) in self.months.iter().enumerate() {
let full_l = full.to_lowercase();
let abbr_l = self.months_abbr.get(i).map(|a| a.to_lowercase()).unwrap_or_default();
let gen_l = self.months_genitive.as_ref().and_then(|g| g.get(i)).map(|g| g.to_lowercase());
let hit = w == full_l
|| w == abbr_l
|| stem == abbr_l.trim_end_matches('.')
|| gen_l.as_deref() == Some(w.as_str())
|| (long_enough && full_l.starts_with(stem))
|| (long_enough && gen_l.as_deref().is_some_and(|g| g.starts_with(stem)));
if hit {
return Some(i as u32 + 1);
}
}
None
}
/// The name of month `month` (1–12) for `mmmm`; `with_day` selects the genitive form where
/// the region has one.
pub fn month_name(&self, month: u32, with_day: bool) -> &'static str {
let i = (month as usize).wrapping_sub(1);
if with_day && let Some(g) = &self.months_genitive {
return g.get(i).copied().unwrap_or("");
}
self.months.get(i).copied().unwrap_or("")
}
/// True when the short time uses a 12-hour clock.
pub fn is_12_hour(&self) -> bool {
self.short_time.contains("AM/PM")
}
}
fn is_space_like(c: char) -> bool {
matches!(c, ' ' | '\u{a0}' | '\u{202f}' | '\u{2009}')
}
fn is_apostrophe_like(c: char) -> bool {
matches!(c, '\'' | '\u{2019}')
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn lookup_accepts_system_and_browser_tags() {
let cases = [
("pt-BR", "pt-BR"),
("pt_BR.UTF-8", "pt-BR"),
("pt", "pt-BR"),
("pt-PT", "pt-PT"),
("pt-AO", "pt-PT"),
("de", "de-DE"),
("de-AT", "de-DE"),
("de_CH.UTF-8", "de-CH"),
("de-LI", "de-CH"),
("fr-CH", "fr-CH"),
("fr-LI", "fr-CH"),
("fr-BE", "fr-FR"),
("it_CH.UTF-8", "it-CH"),
("it-LI", "it-CH"),
("en-CH", "en-CH"),
("nl_NL.UTF-8", "nl-NL"),
("nl", "nl-NL"),
("en-AU", "en-GB"),
("en_GB.UTF-8", "en-GB"),
("en-IN", "en-IN"),
("en-CA", "en-US"),
("zh-Hant", "zh-TW"),
("zh-HK", "zh-TW"),
("zh", "zh-CN"),
// Extensions don't replace the region or script; an explicit territory beats the script.
("en-GB-u-ca-gregory", "en-GB"),
("zh-Hant-u-nu-latn", "zh-TW"),
("zh-Hant-CN", "zh-CN"),
("zh-Hans-TW", "zh-TW"),
("cs_CZ.UTF-8", "cs-CZ"),
("C", "en-US"),
("", "en-US"),
("xx-YY", "en-US"),
];
for (tag, expected) in cases {
assert_eq!(lookup(tag).tag, expected, "{tag}");
}
assert_eq!(find("DE-de").map(|r| r.tag), Some("de-DE"));
assert!(find("xx").is_none());
}
#[test]
fn every_region_is_consistent() {
let mut tags = std::collections::HashSet::new();
for r in all() {
assert!(tags.insert(r.tag), "duplicate {}", r.tag);
assert_ne!(r.decimal, r.group, "{}", r.tag);
assert!(r.currency_format.contains(r.currency), "{}", r.tag);
assert!(r.months.iter().all(|m| !m.is_empty()) && r.months_abbr.iter().all(|m| !m.is_empty()), "{}", r.tag);
assert!(r.weekdays.iter().all(|m| !m.is_empty()) && r.weekdays_abbr.iter().all(|m| !m.is_empty()), "{}", r.tag);
assert_eq!(find(r.tag).map(|x| x.tag), Some(r.tag));
}
}
#[test]
fn month_names_match_loosely() {
let de = find("de-DE").unwrap();
assert_eq!(de.month_from_name("März"), Some(3));
assert_eq!(de.month_from_name("mär"), Some(3));
assert_eq!(de.month_from_name("DEZ"), Some(12));
assert_eq!(de.month_from_name("de"), None);
let gb = find("en-GB").unwrap();
assert_eq!(gb.month_from_name("Sept"), Some(9));
assert_eq!(gb.month_from_name("sep"), Some(9));
let ru = find("ru-RU").unwrap();
assert_eq!(ru.month_from_name("сентября"), Some(9));
assert_eq!(ru.month_from_name("сент."), Some(9));
// Abbreviations typed with a period the table doesn't carry.
assert_eq!(ru.month_from_name("мар."), Some(3));
assert_eq!(ru.month_from_name("июн."), Some(6));
assert_eq!(find("fr-FR").unwrap().month_from_name("juin."), Some(6));
assert_eq!(EN_US.month_from_name("Mar."), None);
assert_eq!(de.month_from_name("."), None);
assert_eq!(EN_US.month_from_name("Ma"), None);
assert_eq!(EN_US.month_from_name("May"), Some(5));
}
#[test]
fn space_like_and_apostrophe_groups_are_interchangeable() {
let fr = find("fr-FR").unwrap();
assert!(fr.is_group('\u{a0}') && fr.is_group(' ') && fr.is_group('\u{202f}'));
assert!(!find("de-DE").unwrap().is_group(' '));
let ch = find("de-CH").unwrap();
assert!(ch.is_group('\'') && ch.is_group('\u{2019}'));
assert!(!ch.is_group(' ') && !find("de-DE").unwrap().is_group('\''));
}
}
+258 -10
View File
@@ -346,19 +346,91 @@ pub fn compare_numbers(a: f64, b: f64) -> Ordering {
round15(a).partial_cmp(&round15(b)).unwrap_or(Ordering::Equal)
}
/// Case-insensitive text order (Unicode lowercase).
/// Case-insensitive text order in three levels, as Excel's collation sorts: first by letters and
/// digits alone (Arabic marks, tatweel and invisible format characters skipped, Arabic-Indic digits
/// read as digits); then, letter by letter, by the marks on each letter (none first, then in
/// Excel's mark order) and the digit script (ASCII, Arabic-Indic, Extended Arabic-Indic); then, to
/// break a remaining tie, by every character (Unicode lowercase). Only text that is equal at the
/// last level compares equal.
pub fn compare_text(a: &str, b: &str) -> Ordering {
let mut ai = a.chars().flat_map(char::to_lowercase);
let mut bi = b.chars().flat_map(char::to_lowercase);
loop {
match (ai.next(), bi.next()) {
(None, None) => return Ordering::Equal,
(None, Some(_)) => return Ordering::Less,
(Some(_), None) => return Ordering::Greater,
(Some(x), Some(y)) if x != y => return x.cmp(&y),
_ => {}
if !(a.is_ascii() && b.is_ascii()) {
let primary = primary_chars(a).cmp(primary_chars(b));
if primary != Ordering::Equal {
return primary;
}
let secondary = secondary_weights(a).cmp(secondary_weights(b));
if secondary != Ordering::Equal {
return secondary;
}
}
lowercase_chars(a).cmp(lowercase_chars(b))
}
fn lowercase_chars(s: &str) -> impl Iterator<Item = char> + '_ {
s.chars().flat_map(char::to_lowercase)
}
/// Characters Excel's text order skips entirely: tatweel and invisible format characters (soft
/// hyphen, combining grapheme joiner, zero-width (non-)joiner, bidi marks, embeddings, overrides
/// and isolates, word joiner, byte order mark). The zero-width space is not one.
fn ignorable(c: char) -> bool {
matches!(
c,
'\u{00AD}'
| '\u{034F}'
| '\u{061C}'
| '\u{0640}'
| '\u{200C}'..='\u{200F}'
| '\u{202A}'..='\u{202E}'
| '\u{2060}'
| '\u{2066}'..='\u{2069}'
| '\u{FEFF}'
)
}
/// Arabic combining marks in the order Excel sorts a letter carrying them; an unmarked letter
/// sorts before all of them. Measured in Excel 16.0 (build 20527, en-US) with one mark on a letter.
/// U+06D6 and U+06D7 sort as letters in Excel, so they are not listed.
const ARABIC_MARK_ORDER: [char; 48] = [
'\u{06EC}', '\u{0611}', '\u{0612}', '\u{0613}', '\u{0614}', '\u{06DB}', '\u{0656}', '\u{0657}', '\u{0610}', '\u{06EB}', '\u{0615}', '\u{06D9}',
'\u{06E4}', '\u{06E1}', '\u{06DA}', '\u{06DC}', '\u{06EA}', '\u{06E3}', '\u{06D8}', '\u{0655}', '\u{065F}', '\u{0654}', '\u{06E7}', '\u{0653}',
'\u{06E8}', '\u{06ED}', '\u{06E2}', '\u{0658}', '\u{064B}', '\u{064C}', '\u{064D}', '\u{064E}', '\u{064F}', '\u{0650}', '\u{0652}', '\u{0651}',
'\u{0670}', '\u{0659}', '\u{065A}', '\u{065B}', '\u{065C}', '\u{065D}', '\u{065E}', '\u{0616}', '\u{0617}', '\u{0618}', '\u{0619}', '\u{061A}',
];
/// Rank of an Arabic mark that letter order skips (its index in `ARABIC_MARK_ORDER`).
fn arabic_mark_rank(c: char) -> Option<u16> {
ARABIC_MARK_ORDER.iter().position(|&m| m == c).and_then(|i| u16::try_from(i).ok())
}
/// Digit script of `c` (0 ASCII, 1 Arabic-Indic, 2 Extended Arabic-Indic) and its ASCII digit.
fn digit(c: char) -> Option<(u16, char)> {
let (script, zero) = match c {
'0'..='9' => return Some((0, c)),
'\u{0660}'..='\u{0669}' => (1, 0x0660),
'\u{06F0}'..='\u{06F9}' => (2, 0x06F0),
_ => return None,
};
char::from_u32(u32::from(c) - zero + u32::from('0')).map(|d| (script, d))
}
/// The characters that decide text order before any tie-break.
fn primary_chars(s: &str) -> impl Iterator<Item = char> + '_ {
lowercase_chars(s).filter_map(|c| if ignorable(c) || arabic_mark_rank(c).is_some() { None } else { Some(digit(c).map_or(c, |(_, d)| d)) })
}
/// The second level, for text whose primary characters are equal: per letter, its digit script and
/// then its marks, the letters' lists closed by 0 so that an unmarked letter sorts first. Marks
/// before the first letter form a list of their own.
fn secondary_weights(s: &str) -> impl Iterator<Item = u16> + '_ {
lowercase_chars(s)
.filter(|&c| !ignorable(c))
.flat_map(|c| match arabic_mark_rank(c) {
Some(rank) => [Some(rank.saturating_add(1)), None],
None => [Some(0), Some(digit(c).map_or(0, |(script, _)| script))],
})
.flatten()
.chain(std::iter::once(0))
}
/// Sorting order used by Sort (ascending): numbers, text, booleans, errors, then blanks last.
@@ -457,6 +529,182 @@ mod tests {
assert_eq!(sort_compare(&Value::Empty, &Value::Number(1.0)), Ordering::Greater);
}
#[test]
fn arabic_diacritics_sort_with_their_base_letter() {
assert_eq!(compare_text("بَاب", "بيت"), Ordering::Less);
assert_eq!(compare_text("بيت", "بَاب"), Ordering::Greater);
assert_eq!(compare_text("كِتاب", "كتب"), Ordering::Less);
assert_eq!(compare_text("رحمٰن", "رحمي"), Ordering::Less);
assert_eq!(compare(&Value::from("بَاب"), &Value::from("بيت")), Ordering::Less);
}
#[test]
fn arabic_indic_digits_sort_with_digits() {
assert_eq!(compare_text("١٢", "ب"), Ordering::Less);
assert_eq!(compare_text("۱۲", "ب"), Ordering::Less);
assert_eq!(compare_text("١", "a"), Ordering::Less);
assert_eq!(compare_text("١٢", "13"), Ordering::Less);
assert_eq!(compare_text("۱۴", "13"), Ordering::Greater);
}
#[test]
fn tatweel_and_bidi_marks_do_not_move_sort_order() {
assert_eq!(compare_text("بـاب", "بت"), Ordering::Less);
assert_eq!(compare_text("\u{200F}apple", "banana"), Ordering::Less);
assert_eq!(compare_text("\u{200E}zebra", "apple"), Ordering::Greater);
assert_eq!(compare_text("\u{061C}a", "b"), Ordering::Less);
assert_eq!(compare_text("a\u{200C}b", "ac"), Ordering::Less);
assert_eq!(compare_text("a\u{200D}b", "ac"), Ordering::Less);
}
#[test]
fn primary_ties_keep_equality_unchanged() {
assert_eq!(compare_text("ABC", "abc"), Ordering::Equal);
assert_eq!(compare_text("باب", "بَاب"), Ordering::Less);
assert_eq!(compare_text("12", "١٢"), Ordering::Less);
assert_eq!(compare_text("a", "\u{200F}a"), Ordering::Less);
let samples = [
"",
"abc",
"ABC",
"abd",
"ab",
"باب",
"بَاب",
"بِاب",
"بـاب",
"بيت",
"١٢",
"۱۲",
"12",
"\u{200F}12",
"\u{200F}a",
"a",
"A\u{200C}",
"a\u{200F}",
"b",
"Straße",
"STRASSE",
"ΣΑΣ",
"σας",
"İ",
"i̇",
];
let lowered = |s: &str| s.chars().flat_map(char::to_lowercase).collect::<String>();
for a in samples {
for b in samples {
let order = compare_text(a, b);
assert_eq!(order.is_eq(), lowered(a) == lowered(b), "{a:?} vs {b:?}");
assert_eq!(order, compare_text(b, a).reverse(), "{a:?} vs {b:?}");
for c in samples {
if order.is_le() && compare_text(b, c).is_le() {
assert!(compare_text(a, c).is_le(), "{a:?} <= {b:?} <= {c:?}");
}
}
}
}
}
/// Sorts `expected` from reversed and rotated starts and checks each comes back as `expected`.
#[track_caller]
fn assert_sorts_to(expected: &[&str]) {
let mut reversed: Vec<&str> = expected.iter().rev().copied().collect();
reversed.sort_by(|a, b| compare_text(a, b));
assert_eq!(reversed, expected);
let mut rotated: Vec<&str> = expected.to_vec();
rotated.rotate_left(expected.len() / 2);
rotated.sort_by(|a, b| compare_text(a, b));
assert_eq!(rotated, expected);
}
// The expected orders below are Excel's, observed with Range.Sort, =SORT() and the `<`
// operator in Excel 16.0 (build 20527, en-US). Strings Excel ranks equal are left out.
#[test]
fn excel_orders_a_letter_by_its_mark() {
assert_sorts_to(&[
"باب",
"ب\u{0611}اب",
"ب\u{0610}اب",
"ب\u{0654}اب",
"ب\u{064B}اب",
"ب\u{064C}اب",
"ب\u{064D}اب",
"ب\u{064E}اب",
"ب\u{064F}اب",
"ب\u{0650}اب",
"ب\u{0652}اب",
"ب\u{0651}اب",
"ب\u{0670}اب",
"ب\u{065E}اب",
"ب\u{061A}اب",
]);
// Quranic marks are skipped by letter order like harakat (hamza sorts before alef).
assert_eq!(compare_text("ب\u{0610}ء", "باب"), Ordering::Less);
assert_eq!(compare_text("ب\u{06E1}ء", "باب"), Ordering::Less);
}
#[test]
fn excel_compares_marks_letter_by_letter() {
assert_sorts_to(&[
"بت",
"بت\u{064E}",
"بت\u{064F}",
"بت\u{0650}",
"ب\u{064E}ت",
"ب\u{064E}ت\u{064E}",
"ب\u{064E}ت\u{064F}",
"ب\u{064F}ت",
"ب\u{064F}ت\u{064E}",
"ب\u{0650}ت\u{0650}",
]);
assert_sorts_to(&["مـحـمـد", "محم\u{0651}د", "م\u{064E}حمد", "م\u{064F}ح\u{064E}م\u{064E}\u{0651}د", "محمدي"]);
}
#[test]
fn excel_weighs_marks_and_digit_scripts_by_position() {
// A mark on an earlier letter outranks the digit script or letter after it, whatever the
// code point of that next character.
assert_sorts_to(&[
"ب",
"ب\u{064E}",
"ب1",
"ب١",
"ب۱",
"ب\u{064E}1",
"ب\u{064E}١",
"ب\u{064E}۱",
"بa",
"ب\u{064E}a",
"با",
"ب\u{064E}ا",
"ب\u{067E}",
"ب\u{064E}\u{067E}",
"ب\u{06CC}",
"ب\u{064E}\u{06CC}",
]);
// And an earlier digit script outranks a later mark.
assert_sorts_to(&["12", "1٢", "١2", "١٢", "1ب", "1ب\u{064E}", "1ب\u{0650}", "١ب", "١ب\u{064E}", "١ب\u{0650}", "۱ب", "۱ب\u{064E}"]);
assert_sorts_to(&["ب1ب", "ب1ب\u{064E}", "ب١ب", "ب\u{064E}ب", "بب1", "بب١", "ب\u{064E}ب1"]);
}
#[test]
fn excel_skips_invisible_format_characters() {
for c in ['\u{00AD}', '\u{034F}', '\u{061C}', '\u{200C}', '\u{200D}', '\u{200E}', '\u{200F}', '\u{202A}', '\u{202B}']
.into_iter()
.chain(['\u{202C}', '\u{202D}', '\u{202E}', '\u{2060}', '\u{2066}', '\u{2067}', '\u{2068}', '\u{2069}', '\u{FEFF}'])
{
let s = format!("{c}b");
assert_eq!(compare_text(&s, "a"), Ordering::Greater, "{s:?}");
assert_eq!(compare_text(&s, "ba"), Ordering::Less, "{s:?}");
assert_eq!(compare_text(&format!("ب{c}\u{064E}اب"), "باب\u{064B}"), Ordering::Greater, "{c:?}");
}
// Not even as a tie-break: a leading mark doesn't push text past marked or other-script forms.
assert_sorts_to(&["\u{200F}باب", "ب\u{064B}اب", "ب\u{064E}اب", "ب\u{0670}اب"]);
assert_sorts_to(&["\u{200F}12", "١٢", "۱۲"]);
assert_sorts_to(&["ب\u{200F}ب\u{064E}", "\u{200F}ب\u{064E}ب"]);
}
#[test]
fn errors() {
for e in CellError::ALL {
+14 -2
View File
@@ -118,7 +118,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[
("insert.picture", "Insert", "Illustrations", "Pictures"),
("insert.shape", "Insert", "Illustrations", "Shapes"),
("insert.icons", "Insert", "Illustrations", "Icons"),
("insert.smartArt", "Insert", "Illustrations", "SmartArt"),
("insert.smartArt", "Insert", "Illustrations", "Diagram"),
("insert.screenshot", "Insert", "Illustrations", "Screenshot"),
("insert.recommendedCharts", "Insert", "Charts", "Recommended Charts"),
("insert.chart", "Insert", "Charts", "Charts"),
@@ -131,7 +131,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[
("review.newComment", "Insert", "Comments", "Comment"),
("insert.textBox", "Insert", "Text", "Text Box"),
("pageLayout.headerFooter", "Insert", "Text", "Header & Footer"),
("insert.wordArt", "Insert", "Text", "WordArt"),
("insert.wordArt", "Insert", "Text", "Decorative Text"),
("insert.signatureLine", "Insert", "Text", "Signature Line"),
("insert.object", "Insert", "Text", "Object"),
("insert.equation", "Insert", "Symbols", "Equation"),
@@ -340,3 +340,15 @@ pub fn parity() -> (usize, usize) {
let done = ids.iter().filter(|id| crate::find_command(id).is_some()).count();
(done, ids.len())
}
#[cfg(test)]
mod feature_name_tests {
#[test]
fn own_feature_labels_keep_command_ids() {
for (id, expected) in [("insert.smartArt", "Diagram"), ("insert.wordArt", "Decorative Text")] {
let entries: Vec<_> = super::CATALOG.iter().filter(|entry| entry.0 == id).collect();
assert_eq!(entries.len(), 1, "{id} appears once");
assert_eq!(entries[0].3, expected);
}
}
}
+25
View File
@@ -368,6 +368,9 @@ impl CfCache {
/// Adds the parts `lower` sets that `style` leaves unset (differential styles: default parts
/// are unset).
fn layer_under(style: &mut Style, lower: &Style) {
if style.align.reading_order.is_none() {
style.align.reading_order = lower.align.reading_order;
}
if style.fill.pattern == PatternType::None {
style.fill = lower.fill;
}
@@ -417,6 +420,28 @@ mod tests {
use super::*;
#[test]
fn reading_order_respects_conditional_rule_priority_and_explicit_context() {
for (higher, expected) in [
(json!({"font": {"bold": true}}), "RightToLeft"),
(json!({"align": {"reading_order": "Context"}}), "Context"),
(json!({"align": {"reading_order": "LeftToRight"}}), "LeftToRight"),
] {
let mut s = crate::Session::new();
s.new_workbook();
s.execute("cell.set", json!({"cell": "A1", "input": "مرحبا 123 ABC"})).unwrap();
for style in [json!({"align": {"reading_order": "RightToLeft"}}), higher] {
s.execute("home.conditionalFormat", json!({"range": "A1", "rule": {"type": "noBlanks", "style": style}})).unwrap();
}
let wb = &s.doc().unwrap().wb;
let (reopened, _) = gridcraft_xlsx::read_xlsx(&gridcraft_xlsx::write_xlsx(wb).unwrap()).unwrap();
for wb in [wb, &reopened] {
let style = CfCache::new().look(wb, 0, CellRef::new(0, 0)).unwrap().style.unwrap();
assert_eq!(serde_json::to_value(style.align).unwrap()["reading_order"], expected);
}
}
}
#[test]
fn every_matching_rule_applies() {
let mut s = crate::Session::new();
+5 -1
View File
@@ -317,5 +317,9 @@ pub fn filter_values(wb: &Workbook, sheet: usize, col: u32) -> Vec<(String, bool
let key = if blank { sort_key(&Value::Empty) } else { sort_key(&v) };
seen.entry((key, t)).or_insert(on);
}
seen.into_iter().map(|((_, t), on)| (t, on)).collect()
let mut values: Vec<_> = seen.into_iter().collect();
values.sort_by(|((a, ta), _), ((b, tb), _)| {
a.0.cmp(&b.0).then(a.1.cmp(&b.1)).then_with(|| gridcraft_core::compare_text(&a.2, &b.2)).then_with(|| ta.cmp(tb))
});
values.into_iter().map(|((_, t), on)| (t, on)).collect()
}
+11
View File
@@ -53,6 +53,17 @@ fn subtotal_puts_summary_rows_below_their_detail() {
assert_eq!(sh.value(CellRef::new(3, 0)).display(), "West");
}
#[test]
fn filter_list_orders_arabic_like_sort() {
let mut s = Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [["H"], ["بج"], ["بُت"], ["\u{200F}بب"]]})).unwrap();
s.execute("data.filter", json!({"range": "A1:A4"})).unwrap();
let doc = s.doc().unwrap();
let list: Vec<String> = filter_values(&doc.wb, 0, 0).into_iter().map(|(t, _)| t).collect();
assert_eq!(list, ["\u{200F}بب", "بُت", "بج"]);
}
#[test]
fn filter_by_huge_numeric_column_is_rejected_not_overflowed() {
let mut s = Session::new();
+6 -1
View File
@@ -239,7 +239,12 @@ fn redo(s: &mut Session, p: &Json) -> Result<Json> {
/// The rows of `range` an applied filter hides (rows hidden by hand are not among them).
fn filtered_rows(sh: &gridcraft_model::Sheet, range: RangeRef) -> std::collections::BTreeSet<u32> {
sh.rows.range(range.start.row..=range.end.row).filter(|(r, info)| info.hidden && sh.is_row_filtered(**r)).map(|(r, _)| *r).collect()
let spans = sh.filter_row_spans();
if spans.is_empty() {
return Default::default();
}
let filtered = |row: u32| spans.iter().any(|&(first, last)| row >= first && row <= last);
sh.rows.range(range.start.row..=range.end.row).filter(|(r, info)| info.hidden && filtered(**r)).map(|(r, _)| *r).collect()
}
/// The Clear commands (Delete is Clear Contents). As in Excel, the rows an applied filter hides
+4
View File
@@ -328,7 +328,11 @@ pub(super) fn range_fill(s: &mut Session, p: &Json) -> Result<Json> {
if r.count() > 2_000_000 {
return Err(bad("range.fill", "range too large"));
}
let skip = cx.wb.sheet(sheet).map(|sh| super::filtered_rows(sh, *r)).unwrap_or_default();
for at in r.iter() {
if skip.contains(&at.row) {
continue;
}
let style = cx.wb.sheet(sheet).map(|sh| sh.style_id(at)).unwrap_or_default();
let base = match by_style.get(&style) {
Some(c) => c.clone(),
+126 -16
View File
@@ -1,8 +1,8 @@
use super::cell::input_to_cell;
use super::paste_special::paste_special;
use super::{
Cell, Clipboard, EngineError, Json, RangeRef, Result, Session, Value, bad, bool_param, cell_param_on, edit, json, ok, str_param, target_ranges,
target_sheet,
Cell, CellRef, Clipboard, EngineError, Json, RangeRef, Result, Session, Value, bad, bool_param, cell_param_on, edit, json, ok, str_param,
target_ranges, target_sheet,
};
// ---------------------------------------------------------------- clipboard
@@ -15,7 +15,7 @@ fn copy_impl(s: &mut Session, p: &Json, cut: bool) -> Result<Json> {
return Err(multiple());
}
// The selection as made (untrimmed), for Insert Copied Cells.
let selected = ranges.first().copied().unwrap_or_default();
let mut selected = ranges.first().copied().unwrap_or_default();
let d = s.doc()?;
let Some(sh) = d.wb.sheet(sheet) else { return Err(EngineError::NoDocument) };
let trim = |range: RangeRef| {
@@ -31,7 +31,27 @@ fn copy_impl(s: &mut Session, p: &Json, cut: bool) -> Result<Json> {
let (wb, block) = slide_together(&d.wb, sheet, &areas).ok_or_else(multiple)?;
(std::sync::Arc::new(wb), block, areas)
} else {
(d.wb.clone(), trim(ranges.first().copied().unwrap_or_default()), Vec::new())
let range = trim(ranges.first().copied().unwrap_or_default());
// A copied range with rows a filter hides takes only the rows showing: they are copied
// like a multi-area selection stacked top to bottom (formulas as values), as in Excel.
// A cut still takes every row.
let hidden = if cut { Default::default() } else { super::filtered_rows(sh, range) };
let visible = if hidden.is_empty() { Vec::new() } else { visible_row_runs(range, &hidden) };
let any_visible = !visible.is_empty();
let packed = match visible.as_slice() {
[] => (d.wb.clone(), range, Vec::new()),
[only] => (d.wb.clone(), *only, Vec::new()),
_ => {
let (wb, block) = slide_together(&d.wb, sheet, &visible).ok_or_else(multiple)?;
(std::sync::Arc::new(wb), block, visible)
}
};
// Whole rows copied with some hidden by a filter come as the block of rows showing, so
// Insert Copied Cells makes room for those rows, not for the hidden ones too.
if !hidden.is_empty() && any_visible && selected.is_full_rows() {
selected = RangeRef::rows(packed.1.start.row, packed.1.end.row);
}
packed
};
let text = crate::display::range_text(&wb, sheet, range);
// The HTML rendering is for the host clipboard (UI); it can reach 4 MB, so agents and
@@ -46,6 +66,56 @@ fn copy_impl(s: &mut Session, p: &Json, cut: bool) -> Result<Json> {
Ok(result)
}
/// The runs of consecutive rows of `range` not in `hidden`, as ranges of `range`'s columns.
fn visible_row_runs(range: RangeRef, hidden: &std::collections::BTreeSet<u32>) -> Vec<RangeRef> {
let mut runs = Vec::new();
let mut start = None;
for row in range.start.row..=range.end.row {
match (hidden.contains(&row), start) {
(false, None) => start = Some(row),
(true, Some(first)) => {
runs.push(RangeRef::new(CellRef::new(first, range.start.col), CellRef::new(row - 1, range.end.col)));
start = None;
}
_ => {}
}
}
if let Some(first) = start {
runs.push(RangeRef::new(CellRef::new(first, range.start.col), CellRef::new(range.end.row, range.end.col)));
}
runs
}
/// The validation rules of `src` cut into the areas of a packed block: one rule per area a rule
/// touches, over that area's part moved by the area's shift. `sorted` areas run in order.
fn pack_validations(src: &gridcraft_model::Sheet, sorted: &[RangeRef], shifts: &[(i64, i64)], across: bool) -> Vec<gridcraft_model::Validation> {
let key = |c: CellRef| if across { c.col } else { c.row };
let mut packed_rules = Vec::new();
for dv in &src.validations {
let Some(from) = dv.ranges.first().map(|r| r.start) else { continue };
// The areas each of the rule's ranges touches lie side by side in the sorted list.
let end_key = |a: &RangeRef| if across { a.end.col } else { a.end.row };
let mut by_area: std::collections::BTreeMap<usize, Vec<RangeRef>> = Default::default();
for r in &dv.ranges {
let first = sorted.partition_point(|a| end_key(a) < key(r.start));
for (i, a) in sorted.iter().enumerate().skip(first).take_while(|(_, a)| key(a.start) <= end_key(r)) {
let (dr, dc) = shifts[i];
let moved = r.intersection(a).and_then(|x| Some(RangeRef::new(x.start.offset(dr, dc)?, x.end.offset(dr, dc)?)));
by_area.entry(i).or_default().extend(moved);
}
}
for ranges in by_area.into_values() {
let Some(to) = ranges.first().map(|r| r.start) else { continue };
let mut rule = dv.clone();
rule.f1 = super::super::data::move_rule_formula(&dv.f1, from, to);
rule.f2 = dv.f2.as_ref().map(|f| super::super::data::move_rule_formula(f, from, to));
rule.ranges = ranges;
packed_rules.push(rule);
}
}
packed_rules
}
/// Excel copies a multi-area selection only when the areas line up: all cover the same rows (they
/// slide together side by side, left to right) or all cover the same columns (stacked, top to
/// bottom), without overlapping. Returns a copy of the workbook whose sheet holds the areas packed
@@ -80,17 +150,35 @@ fn slide_together(wb: &gridcraft_model::Workbook, sheet: usize, areas: &[RangeRe
dst.spill.retain(|c, _| !block.contains(*c));
dst.spill_ranges.retain(|c, _| !block.contains(*c));
dst.comments.retain(|c, _| !block.contains(*c));
dst.hyperlinks.retain(|c, _| !block.contains(*c));
super::super::data::remove_validation_cells(&mut dst.validations, &[block]);
dst.merges.retain(|m| !m.intersects(&block));
if across {
dst.cols.retain(|c, _| !(block.start.col..=block.end.col).contains(c));
} else {
dst.rows.retain(|r, _| !(block.start.row..=block.end.row).contains(r));
}
// Where each area lands: the shift of its cells, and its first row or column in the block.
let key = |c: CellRef| if across { c.col } else { c.row };
let mut next = if across { anchor.col } else { anchor.row };
let mut shifts: Vec<(i64, i64)> = Vec::with_capacity(sorted.len());
for a in &sorted {
let (dr, dc) = if across { (0, i64::from(next) - i64::from(a.start.col)) } else { (i64::from(next) - i64::from(a.start.row), 0) };
let cells = src.cells.iter_range(*a).map(|(c, _)| c).chain(src.spill.keys().copied().filter(|c| a.contains(*c)));
for c in cells {
let to = i64::from(next);
shifts.push(if across { (0, to - i64::from(a.start.col)) } else { (to - i64::from(a.start.row), 0) });
next = next.saturating_add(if across { a.width() } else { a.height() });
}
// The areas never overlap and run in order, so the area holding a cell is found by search:
// each map below is read once, whatever the number of areas.
let area_of = |c: CellRef| -> Option<usize> {
let i = sorted.partition_point(|a| key(a.start) <= key(c)).checked_sub(1)?;
sorted.get(i)?.contains(c).then_some(i)
};
let moved = |c: CellRef| -> Option<CellRef> {
let (dr, dc) = shifts[area_of(c)?];
c.offset(dr, dc)
};
for (a, &(dr, dc)) in sorted.iter().zip(&shifts) {
for (c, _) in src.cells.iter_range(*a) {
let Some(to) = c.offset(dr, dc) else { continue };
let style = src.cell(c).map(|x| x.style).unwrap_or_default();
dst.set_cell(to, Cell { value: src.value(c), formula: None, style, py: None });
@@ -98,17 +186,38 @@ fn slide_together(wb: &gridcraft_model::Workbook, sheet: usize, areas: &[RangeRe
dst.cell_pictures.insert(to, picture.clone());
}
}
for (c, comment) in src.comments.iter().filter(|(c, _)| a.contains(**c)) {
if let Some(to) = c.offset(dr, dc) {
dst.comments.insert(to, comment.clone());
}
}
for c in src.spill.keys().copied() {
let Some(to) = moved(c) else { continue };
let style = src.cell(c).map(|x| x.style).unwrap_or_default();
dst.set_cell(to, Cell { value: src.value(c), formula: None, style, py: None });
if let Some(picture) = src.cell_pictures.get(&c) {
dst.cell_pictures.insert(to, picture.clone());
}
for m in src.merges.iter().filter(|m| a.contains_range(m)) {
if let (Some(s), Some(e)) = (m.start.offset(dr, dc), m.end.offset(dr, dc)) {
dst.merges.push(RangeRef::new(s, e));
}
}
// Links and validation rules follow their cells too, so a row the filter hides leaves its own
// behind.
for (c, link) in &src.hyperlinks {
if let Some(to) = moved(*c) {
dst.hyperlinks.insert(to, link.clone());
}
// Column widths (side by side) or row heights and hidden rows (stacked) follow their cells.
}
let packed_rules = pack_validations(src, &sorted, &shifts, across);
for (c, comment) in &src.comments {
if let Some(to) = moved(*c) {
dst.comments.insert(to, comment.clone());
}
}
for m in &src.merges {
let Some(i) = area_of(m.start).filter(|&i| sorted[i].contains_range(m)) else { continue };
let (dr, dc) = shifts[i];
if let (Some(s), Some(e)) = (m.start.offset(dr, dc), m.end.offset(dr, dc)) {
dst.merges.push(RangeRef::new(s, e));
}
}
// Column widths (side by side) or row heights and hidden rows (stacked) follow their cells.
let mut next = if across { anchor.col } else { anchor.row };
for a in &sorted {
if across {
for (c, info) in src.cols.range(a.start.col..=a.end.col) {
dst.cols.insert(next.saturating_add(c.saturating_sub(a.start.col)), *info);
@@ -120,6 +229,7 @@ fn slide_together(wb: &gridcraft_model::Workbook, sheet: usize, areas: &[RangeRe
}
next = next.saturating_add(if across { a.width() } else { a.height() });
}
dst.validations.extend(packed_rules);
Some((out, block))
}
+2 -1
View File
@@ -31,8 +31,9 @@ pub(super) fn fill_dir(s: &mut Session, p: &Json, dr: i64, dc: i64) -> Result<Js
} else {
RangeRef::new(CellRef::new(r.start.row, r.end.col), r.end)
};
let skip = s.doc()?.wb.sheet(sheet).map(|sh| super::filtered_rows(sh, r)).unwrap_or_default();
edit(s, |cx| {
crate::fill::fill(cx, sheet, src, r, crate::fill::FillMode::Copy)?;
crate::fill::fill_except(cx, sheet, src, r, crate::fill::FillMode::Copy, &skip)?;
Ok(Json::Null)
})
}
+111 -50
View File
@@ -17,11 +17,13 @@ pub(in super::super) fn go_to(s: &mut Session, p: &Json) -> Result<Json> {
}
let Some(reference) = str_param(p, "reference").or_else(|| str_param(p, "range")) else { return Err(bad("edit.goTo", "missing `reference`")) };
let d = s.doc()?;
let mut names = Resolver::new(&d.wb);
let mut target: Option<usize> = None;
let mut areas: Vec<RangeRef> = Vec::new();
let mut named = false;
for part in reference_parts(reference) {
let Some((sheet, found, by_name)) = go_to_target(&d.wb, d.wb.active_sheet(), part, 0) else {
let mut found = Vec::new();
let Some((sheet, _, by_name)) = names.target(d.wb.active_sheet(), part, 0, Some(&mut found)) else {
return Err(EngineError::Other("Reference isn't valid.".into()));
};
// One selection holds the areas of one sheet.
@@ -77,57 +79,114 @@ pub(super) fn reference_parts(text: &str) -> impl Iterator<Item = &str> {
})
}
/// Where one part of a Go To reference points, seen from sheet `sheet`: the sheet, its areas,
/// and whether a defined name or a table gave them. `Sheet2!Name` is the name scoped to Sheet2.
/// A name counts when what it refers to is a reference; a constant or a formula is not one.
fn go_to_target(wb: &gridcraft_model::Workbook, sheet: usize, part: &str, depth: usize) -> Option<(usize, Vec<RangeRef>, bool)> {
if part.is_empty() || depth > GO_TO_NAME_DEPTH {
return None;
}
let (prefix, body) = split_sheet(part);
let on = match &prefix {
Some(n) => wb.sheet_index(n)?,
None => sheet,
};
if let Some(r) = RangeRef::parse(&body.replace('$', "")) {
return Some((on, vec![r], false));
}
let name = match prefix {
Some(_) => wb.names.iter().find(|n| n.scope == Some(on) && gridcraft_formula::same_name(&n.name, body)),
None => wb.name(body, sheet),
};
if let Some(n) = name {
return name_areas(wb, n, sheet, depth).map(|(t, areas)| (t, areas, true));
}
if prefix.is_none()
&& let Some((si, ti)) = wb.table(body)
{
// As in Excel, a table's name stands for its data, without the header and totals rows.
let t = wb.sheet(si)?.tables.get(ti)?;
return Some((si, vec![t.data_range().unwrap_or(t.range)], true));
}
None
/// Resolves Go To references against one workbook. Defined names are found through an index
/// built once, and whether a name refers to a reference is remembered: listing the names of a
/// workbook full of names that alias other names would otherwise scan every name for every alias
/// of every name.
struct Resolver<'a> {
wb: &'a gridcraft_model::Workbook,
/// Indexes into `wb.names` by scope and folded name, in workbook order.
by_name: std::collections::HashMap<(Option<usize>, String), Vec<usize>>,
/// [`Resolver::name_areas`] without areas, by name index, sheet and depth: the target sheet
/// and the number of areas.
memo: std::collections::HashMap<(usize, usize, usize), Option<(usize, usize)>>,
}
/// The sheet and areas defined name `n` refers to, seen from sheet `sheet`, when that is a
/// reference.
fn name_areas(wb: &gridcraft_model::Workbook, n: &gridcraft_model::DefinedName, sheet: usize, depth: usize) -> Option<(usize, Vec<RangeRef>)> {
let from = n.scope.unwrap_or(sheet);
let formula = n.formula.trim().trim_start_matches('=');
let mut target: Option<usize> = None;
let mut areas = Vec::new();
for p in reference_parts(formula) {
let (s, found, _) = go_to_target(wb, from, p, depth + 1)?;
if target.is_some_and(|t| t != s) {
return None;
}
target = Some(s);
areas.extend(found);
if areas.len() > GO_TO_MAX_AREAS {
return None;
impl<'a> Resolver<'a> {
fn new(wb: &'a gridcraft_model::Workbook) -> Self {
let mut by_name: std::collections::HashMap<(Option<usize>, String), Vec<usize>> = std::collections::HashMap::new();
for (i, n) in wb.names.iter().enumerate() {
by_name.entry((n.scope, gridcraft_formula::fold_name(&n.name))).or_default().push(i);
}
Self { wb, by_name, memo: std::collections::HashMap::new() }
}
/// The first name of scope `scope` that `same` accepts, as a scan of `wb.names` finds it.
/// Names `same` accepts fold alike, so only that bucket is checked.
fn find(&self, scope: Option<usize>, body: &str, same: impl Fn(&str, &str) -> bool) -> Option<usize> {
self.by_name
.get(&(scope, gridcraft_formula::fold_name(body)))?
.iter()
.copied()
.find(|&i| self.wb.names.get(i).is_some_and(|n| same(&n.name, body)))
}
/// Where one part of a Go To reference points, seen from sheet `sheet`: the sheet, how many
/// areas (appended to `areas` when given), and whether a defined name or a table gave them.
/// `Sheet2!Name` is the name scoped to Sheet2. A name counts when what it refers to is a
/// reference; a constant or a formula is not one.
fn target(&mut self, sheet: usize, part: &str, depth: usize, areas: Option<&mut Vec<RangeRef>>) -> Option<(usize, usize, bool)> {
let wb = self.wb;
if part.is_empty() || depth > GO_TO_NAME_DEPTH {
return None;
}
let (prefix, body) = split_sheet(part);
let on = match &prefix {
Some(n) => wb.sheet_index(n)?,
None => sheet,
};
if let Some(r) = RangeRef::parse(&body.replace('$', "")) {
if let Some(a) = areas {
a.push(r);
}
return Some((on, 1, false));
}
// As `Workbook::name`: the sheet's own name, else the workbook's.
let name = match prefix {
Some(_) => self.find(Some(on), body, gridcraft_formula::same_name),
None => self.find(Some(sheet), body, str::eq_ignore_ascii_case).or_else(|| self.find(None, body, str::eq_ignore_ascii_case)),
};
if let Some(i) = name {
return self.name_areas(i, sheet, depth, areas).map(|(t, n)| (t, n, true));
}
if prefix.is_none()
&& let Some((si, ti)) = wb.table(body)
{
// As in Excel, a table's name stands for its data, without the header and totals rows.
let t = wb.sheet(si)?.tables.get(ti)?;
if let Some(a) = areas {
a.push(t.data_range().unwrap_or(t.range));
}
return Some((si, 1, true));
}
None
}
/// The sheet and number of areas defined name `wb.names[i]` refers to, seen from sheet
/// `sheet`, when that is a reference; the areas are appended to `areas` when given.
fn name_areas(&mut self, i: usize, sheet: usize, depth: usize, mut areas: Option<&mut Vec<RangeRef>>) -> Option<(usize, usize)> {
let key = (i, sheet, depth);
if areas.is_none()
&& let Some(known) = self.memo.get(&key)
{
return *known;
}
let out = self.name_areas_uncached(i, sheet, depth, areas.as_deref_mut());
if areas.is_none() {
self.memo.insert(key, out);
}
out
}
fn name_areas_uncached(&mut self, i: usize, sheet: usize, depth: usize, mut areas: Option<&mut Vec<RangeRef>>) -> Option<(usize, usize)> {
let n = self.wb.names.get(i)?;
let from = n.scope.unwrap_or(sheet);
let formula = n.formula.trim().trim_start_matches('=');
let mut target: Option<usize> = None;
let mut count = 0usize;
for p in reference_parts(formula) {
let (s, found, _) = self.target(from, p, depth + 1, areas.as_deref_mut())?;
if target.is_some_and(|t| t != s) {
return None;
}
target = Some(s);
count += found;
if count > GO_TO_MAX_AREAS {
return None;
}
}
target.map(|t| (t, count))
}
target.map(|t| (t, areas))
}
/// The names the Go To dialog lists, seen from sheet `sheet`: the named ranges of the workbook and
@@ -137,11 +196,13 @@ fn name_areas(wb: &gridcraft_model::Workbook, n: &gridcraft_model::DefinedName,
pub fn go_to_names(wb: &gridcraft_model::Workbook, sheet: usize) -> Vec<String> {
let own: std::collections::HashSet<String> =
wb.names.iter().filter(|n| n.scope == Some(sheet)).map(|n| gridcraft_formula::fold_name(&n.name)).collect();
let mut names = Resolver::new(wb);
let mut out: Vec<String> = wb
.names
.iter()
.filter(|n| !n.hidden && name_areas(wb, n, sheet, 0).is_some())
.filter_map(|n| match n.scope {
.enumerate()
.filter(|(i, n)| !n.hidden && names.name_areas(*i, sheet, 0, None).is_some())
.filter_map(|(_, n)| match n.scope {
None if own.contains(&gridcraft_formula::fold_name(&n.name)) => None,
None => Some(n.name.clone()),
Some(s) if s == sheet => Some(n.name.clone()),
+52 -14
View File
@@ -45,6 +45,12 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
&& !sel.is_full_rows()
&& sel.count() <= 4_000_000;
let (tiles_r, tiles_c) = if tile { (sel.height() / h, sel.width() / w) } else { (1, 1) };
// A single cell or row repeated down a filtered selection skips the rows the filter hides; a
// taller block is pasted straight down, hidden rows included, as in Excel.
let skip_rows = match s.doc()?.wb.sheet(dest_sheet) {
Some(sh) if tile && h == 1 => super::filtered_rows(sh, sel),
_ => Default::default(),
};
// Excel refuses a paste that would run off the sheet (whole columns pasted below row 1, whole
// rows right of column A, a whole column transposed) instead of pasting part of it.
if at.offset(i64::from(tiles_r * h) - 1, i64::from(tiles_c * w) - 1).is_none() {
@@ -134,6 +140,9 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
let sc = CellRef::new(src.start.row + r, src.start.col + c);
let (dr, dc) = if transpose { (c, r) } else { (r, c) };
let Some(dest) = at.offset((tr * h + dr) as i64, (tc * w + dc) as i64) else { continue };
if skip_rows.contains(&dest.row) {
continue;
}
let src_cell = if cut_move { moved_formulas.get(&sc).cloned() } else { src_sheet.cell(sc).cloned() };
let src_val = src_sheet.value(sc);
let src_picture = if cut_move { moved_pictures.get(&sc) } else { src_sheet.cell_pictures.get(&sc) }.cloned();
@@ -326,7 +335,7 @@ pub(super) fn paste_special(s: &mut Session, p: &Json) -> Result<Json> {
}
let pasted = RangeRef::new(at, at.offset_clamped(i64::from(tiles_r * h) - 1, i64::from(tiles_c * w) - 1));
if !link && matches!(what, What::All | What::AllExceptBorders | What::Validation) {
paste_validations(cx, src_sheet, src, dest_sheet, pasted, transpose, skip_blanks)?;
paste_validations(cx, src_sheet, src, dest_sheet, pasted, PasteMode { transpose, skip_blanks, skip_rows: &skip_rows })?;
}
if what == What::ColumnWidths {
// Each pasted column takes the width of the column copied to it (transposed, the
@@ -394,6 +403,31 @@ const MAX_PASTE_LINK_CELLS: u64 = 1_048_576;
/// selection makes one range per tile).
const MAX_PASTE_VALIDATION_RANGES: usize = 100_000;
/// How a validation paste lays the rules out: transposed or not, keeping the destination's rule under
/// blank source cells, and leaving the rows a filter hides as they are.
#[derive(Clone, Copy)]
struct PasteMode<'a> {
transpose: bool,
skip_blanks: bool,
skip_rows: &'a std::collections::BTreeSet<u32>,
}
/// `range` less the rows in `skip`: the runs of rows between them, as ranges of its columns.
fn without_rows(range: RangeRef, skip: &std::collections::BTreeSet<u32>) -> Vec<RangeRef> {
let mut out = Vec::new();
let mut next = range.start.row;
for &row in skip.range(range.start.row..=range.end.row) {
if row > next {
out.push(RangeRef::new(CellRef::new(next, range.start.col), CellRef::new(row - 1, range.end.col)));
}
next = row + 1;
}
if next <= range.end.row {
out.push(RangeRef::new(CellRef::new(next, range.start.col), CellRef::new(range.end.row, range.end.col)));
}
out
}
/// Pasting validation works on ranges, as Excel stores it: the pasted rectangle is cut out of the
/// destination's rules (cells whose source had no rule end up with none), then each source rule
/// that covers part of the copied block `src` comes along as one rule over the translated parts.
@@ -404,9 +438,9 @@ fn paste_validations(
src: RangeRef,
dest_sheet: usize,
pasted: RangeRef,
transpose: bool,
skip_blanks: bool,
mode: PasteMode<'_>,
) -> Result<()> {
let PasteMode { transpose, skip_blanks, skip_rows } = mode;
let (h, w) = if transpose { (src.width(), src.height()) } else { (src.height(), src.width()) };
let (tiles_r, tiles_c) = (pasted.height() / h.max(1), pasted.width() / w.max(1));
let mut rules = Vec::new();
@@ -426,7 +460,9 @@ fn paste_validations(
let (r, c) = if transpose { (c, r) } else { (r, c) };
for tr in 0..tiles_r {
for tc in 0..tiles_c {
if let Some(d) = pasted.start.offset(i64::from(tr * h + r), i64::from(tc * w + c)) {
if let Some(d) = pasted.start.offset(i64::from(tr * h + r), i64::from(tc * w + c))
&& !skip_rows.contains(&d.row)
{
cells.push((d.row, d.col));
}
}
@@ -490,6 +526,8 @@ fn paste_validations(
}
}
}
// The rows a filter hides keep what they have.
let ranges: Vec<RangeRef> = ranges.into_iter().flat_map(|r| without_rows(r, skip_rows)).collect();
let Some(anchor) = ranges.first().map(|r| r.start) else { continue };
let mut rule = dv.clone();
// Relative references move with the rule, as pasted formulas do: they are written for the
@@ -507,7 +545,7 @@ fn paste_validations(
cleared.extend(rules.iter().flat_map(|r| r.ranges.iter().copied()));
super::super::data::remove_validation_cells(&mut sh.validations, &cleared);
} else {
super::super::data::remove_validation_cells(&mut sh.validations, &[pasted]);
super::super::data::remove_validation_cells(&mut sh.validations, &without_rows(pasted, skip_rows));
}
sh.validations.extend(rules);
Ok(())
@@ -529,19 +567,19 @@ pub(crate) fn rewrite_all_formulas(wb: &mut gridcraft_model::Workbook, target: &
let on_target = moves_arrays && host.eq_ignore_ascii_case(target);
let Some(sheet) = wb.sheet_mut(si) else { continue };
for (c, f) in keys {
let Some(expr) = f.expr() else { continue };
let new = gridcraft_formula::adjust::adjust(expr, &host, target, e);
let text = gridcraft_formula::print(&new);
// Borrows the tree and copies and prints only the formulas the edit changes. A formula
// the edit leaves alone keeps its stored text, even if that text is not canonical.
let Some(new) = f.with_expr(|x| gridcraft_formula::adjust::adjust_if_changed(x, &host, target, e)) else { continue };
let array = match f.array {
Some(a) if on_target => crate::cmd::sheet::map_range(a, e),
a => a,
};
if (text != f.text || array != f.array)
&& let Some(cell) = sheet.cells.get_mut(c)
{
let mut nf = Formula::from_expr(new);
nf.array = array;
nf.dynamic = f.dynamic;
let nf = match new {
Some(new) => Formula { array, dynamic: f.dynamic, ..Formula::from_expr(new) },
None if array != f.array => Formula { array, ..(*f).clone() },
None => continue,
};
if let Some(cell) = sheet.cells.get_mut(c) {
cell.formula = Some(std::sync::Arc::new(nf));
}
}
+8 -1
View File
@@ -66,7 +66,14 @@ pub(super) fn selection_move(s: &mut Session, p: &Json) -> Result<Json> {
// Extension moves the far corner, not the active cell.
let from = if extend { sel.moving_end() } else { sel.active };
// Skip hidden rows/columns and step over merged areas.
let mut to = if jmp {
let mut to = if jmp && extend && (dr == 0) != (dc == 0) {
// Excel: Ctrl+Shift+arrow searches the active cell's column (or row) from the moving
// edge, so a header row selected with Ctrl+Shift+Right extends down to the end of the
// active column's data.
let probe = if dr != 0 { CellRef::new(from.row, sel.active.col) } else { CellRef::new(sel.active.row, from.col) };
let edge = jump(sheet, probe, dr.signum(), dc.signum());
if dr != 0 { CellRef::new(edge.row, from.col) } else { CellRef::new(from.row, edge.col) }
} else if jmp {
jump(sheet, from, dr.signum(), dc.signum())
} else {
let mut t = from.offset_clamped(dr, dc);
+22
View File
@@ -42,6 +42,28 @@ fn go_to_lists_the_named_ranges_of_every_sheet() {
assert_eq!(list["names"], json!(["Areas", "Local", "Price", "Sheet1!Own", "type"]), "{list}");
}
/// Thousands of names that alias other names are listed with one lookup per alias, not a scan of
/// every name for each; aliases resolve through the chain like a single name does.
#[test]
fn go_to_lists_many_aliases_and_follows_them() {
let mut s = go_to_workbook();
let wb = std::sync::Arc::make_mut(&mut s.doc_mut().unwrap().wb);
let alias = |name: String, formula: String| gridcraft_model::DefinedName { name, scope: None, formula, comment: String::new(), hidden: false };
wb.names.push(alias("Chain0".into(), "Areas".into()));
for i in 1..20_000 {
wb.names.push(alias(format!("Chain{i}"), format!("Chain{},price", i - 1)));
}
let list = s.execute("edit.goTo", json!({"list": true})).unwrap();
let names: Vec<&str> = list["names"].as_array().unwrap().iter().filter_map(|n| n.as_str()).collect();
// Chain0..Chain6 are within the name depth Go To follows; longer chains are not ranges.
for i in 0..7 {
assert!(names.contains(&format!("Chain{i}").as_str()), "Chain{i} in {names:?}");
}
assert!(!names.contains(&"Chain7"), "{names:?}");
let out = s.execute("edit.goTo", json!({"reference": "chain1"})).unwrap();
assert_eq!(out["selection"], "A1,C3:C4,B2:B5", "{out}");
}
#[test]
fn go_to_a_name_scoped_to_another_sheet_goes_to_that_sheet() {
let mut s = go_to_workbook();
+47 -1
View File
@@ -85,7 +85,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Format Cells…",
["Home", "Cells", "Format"],
Some("Cmd+1"),
"{range?, style: {font?, fill?, borders?, align?, numFmt?, protection?}} (partial Style JSON merged in)",
"{range?, style: {font?, fill?, borders?, align?: {reading_order?: Context|LeftToRight|RightToLeft|null, ...}, numFmt?, protection?}} (partial Style JSON merged in; null clears reading_order)",
has_doc,
format_cells
),
@@ -870,6 +870,52 @@ mod tests {
assert_eq!(s.execute("cell.get", json!({"cell": "A1"})).unwrap()["style"]["num_fmt"], "0.00");
}
#[test]
fn arabic_vowel_and_direction_marks_add_no_width() {
let plain = approx_text_width("محمد يدرس في المدرسة", 11.0, false);
assert_eq!(approx_text_width("مُحَمَّد يَدْرُسُ فِي الْمَدْرَسَةِ", 11.0, false), plain);
assert_eq!(approx_text_width("\u{200F}محمد يدرس في المدرسة\u{200C}", 11.0, false), plain);
assert_eq!(approx_text_width("ب\u{0897}\u{0898}\u{089F}", 11.0, false), approx_text_width("ب", 11.0, false));
assert!(approx_text_width("\u{08E2}", 11.0, false) > 0.0, "the disputed end of ayah is a visible sign");
}
#[test]
fn arabic_text_fits_a_column_about_as_wide_as_latin_text_of_the_same_length() {
let mut s = crate::Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [["مُحَمَّد يَدْرُسُ فِي الْمَدْرَسَةِ", "Hello there in school"]]})).unwrap();
s.execute("home.autofitColumnWidth", json!({"cols": "A:B"})).unwrap();
let doc = s.doc().unwrap();
let width = |c: u32| doc.wb.active().and_then(|sh| sh.cols.get(&c)).and_then(|e| e.size).unwrap();
let ratio = width(0) / width(1);
assert!((0.75..=1.15).contains(&ratio), "Arabic {} vs Latin {}", width(0), width(1));
}
#[test]
fn arabic_word_ligatures_are_wider_than_letters() {
let em = |t: &str| approx_text_width(t, 72.0, false) / 96.0;
assert!(em("\u{FDFD}") > 8.0);
assert!(em("\u{FDFA}") > 1.0);
let honorifics = ['\u{FD40}', '\u{FD4B}', '\u{FD4C}', '\u{FD4F}', '\u{FDC8}', '\u{FDCF}', '\u{FDFE}', '\u{FDFF}'];
for honorific in honorifics.into_iter().chain(['\u{FBC3}', '\u{FBD2}', '\u{FD90}', '\u{FD91}']) {
assert!(em(&honorific.to_string()) > 1.0, "U+{:04X}", u32::from(honorific));
}
assert!(em("ب") < 0.6);
}
#[test]
fn arabic_digits_are_as_wide_as_ascii_digits_and_punctuation_as_a_letter() {
let width = |t: &str| approx_text_width(t, 11.0, false);
assert_eq!(width("١٢٣"), width("123"));
assert_eq!(width("۴۵۶"), width("456"));
assert_eq!(width("،؟"), width("بب"));
}
#[test]
fn cjk_text_keeps_a_full_em_per_character() {
assert_eq!(approx_text_width("中文", 12.0, false), 2.0 * 12.0 * 96.0 / 72.0);
}
#[test]
fn width_units() {
assert_eq!(chars_to_points(8.43), 64.0);
+44 -1
View File
@@ -136,12 +136,55 @@ pub fn approx_text_width(text: &str, size: f32, bold: bool) -> f32 {
c if c.is_ascii_uppercase() => 0.65,
c if c.is_ascii_digit() => 0.56,
c if c.is_ascii() => 0.52,
_ => 1.0,
c if is_zero_width(c) => 0.0,
c => arabic_em(c).unwrap_or(1.0),
};
}
w * size * if bold { 1.07 } else { 1.0 } * 96.0 / 72.0
}
/// Arabic vowel and Quranic marks, which sit on the previous letter, and invisible format
/// characters such as direction marks and zero-width joiners.
fn is_zero_width(c: char) -> bool {
matches!(
c,
'\u{0610}'..='\u{061A}'
| '\u{061C}'
| '\u{064B}'..='\u{065F}'
| '\u{0670}'
| '\u{06D6}'..='\u{06DC}'
| '\u{06DF}'..='\u{06E4}'
| '\u{06E7}'..='\u{06E8}'
| '\u{06EA}'..='\u{06ED}'
| '\u{0897}'..='\u{089F}'
| '\u{08CA}'..='\u{08E1}'
| '\u{08E3}'..='\u{08FF}'
| '\u{200B}'..='\u{200F}'
| '\u{202A}'..='\u{202E}'
| '\u{2060}'..='\u{2069}'
| '\u{FE20}'..='\u{FE2F}'
| '\u{FEFF}'
)
}
/// Width in ems of an Arabic-script character, measured from shaped Noto Sans Arabic, Noto Naskh
/// Arabic and DejaVu Sans: joined letters average about half an em, word ligatures such as ﷺ and the
/// rial sign about 1.2 em, the honorific ligatures (U+FD40–FD4B) about 1.6 em, and the bismillah
/// ligature about 11 em. The later honorific ligatures (U+FBC3–FBD2, U+FD4C–FD4F, U+FD90–FD91,
/// U+FDC8–FDCF) are not measured and get the 1.6 em of their siblings rather than the letter width.
/// Arabic-Indic digits are as wide as ASCII digits (0.53–0.57 em in Noto Sans Arabic, Arial, Tahoma
/// and Times New Roman); Arabic punctuation (0.2–0.5 em in the same fonts) keeps the letter width.
fn arabic_em(c: char) -> Option<f32> {
match c {
'\u{FDFD}' => Some(11.0),
'\u{FBC3}'..='\u{FBD2}' | '\u{FD40}'..='\u{FD4F}' | '\u{FD90}'..='\u{FD91}' | '\u{FDC8}'..='\u{FDCF}' => Some(1.6),
'\u{FDF0}'..='\u{FDFC}' | '\u{FDFE}'..='\u{FDFF}' => Some(1.2),
'\u{0660}'..='\u{0669}' | '\u{06F0}'..='\u{06F9}' => Some(0.56),
'\u{0600}'..='\u{06FF}' | '\u{0750}'..='\u{077F}' | '\u{0870}'..='\u{08FF}' | '\u{FB50}'..='\u{FDFF}' | '\u{FE70}'..='\u{FEFC}' => Some(0.48),
_ => None,
}
}
/// Distinct texts AutoFit measures exactly; past this many, `Session::text_width` is asked for a
/// quick measure so a huge column doesn't lay out every value.
const EXACT_TEXT_WIDTHS: usize = 2_000;
+1 -4
View File
@@ -179,10 +179,7 @@ fn guessed_range(sh: &gridcraft_model::sheet::Sheet, at: CellRef) -> Option<Rang
let mut range = None;
if at.row > 0 && (numeric(CellRef::new(at.row - 1, at.col)) || at.col == 0 || !numeric(CellRef::new(at.row, at.col - 1))) {
let mut top = at.row - 1;
while top > 0
&& (numeric(CellRef::new(top - 1, at.col))
|| sh.value(CellRef::new(top - 1, at.col)).is_empty() && top > 1 && numeric(CellRef::new(top - 2, at.col)))
{
while top > 0 && numeric(CellRef::new(top - 1, at.col)) {
top -= 1;
}
if numeric(CellRef::new(at.row - 1, at.col)) {
+128 -9
View File
@@ -228,6 +228,82 @@ fn draw_border(pg: &mut Page, b: &BorderLine, wb: &Workbook, start: Point, end:
pg.stroke_line_dashed(Point::new(x0, y0), Point::new(x1, y1), lw, color, &border_dash(b.style, k));
}
/// Merged areas active at one row of a downward sweep, kept disjoint: by first column, their last
/// column and last row. Adding, dropping and looking up a merge costs log(active merges).
#[derive(Default)]
struct MergeSweep {
cols: std::collections::BTreeMap<u32, (u32, u32)>,
/// (last row, first column) of the active merges, soonest ending first.
ends: std::collections::BinaryHeap<std::cmp::Reverse<(u32, u32)>>,
}
impl MergeSweep {
/// Moves down to `row` (rows never decrease), dropping the merges that end above it.
fn advance(&mut self, row: u32) {
while let Some(std::cmp::Reverse((end, col))) = self.ends.peek().copied() {
if end >= row {
break;
}
self.ends.pop();
self.cols.remove(&col);
}
}
/// Does an active merge meet columns `c0..=c1`? The intervals are disjoint, so only the last
/// one starting at or before `c1` can.
fn meets(&self, c0: u32, c1: u32) -> bool {
self.cols.range(..=c1).next_back().is_some_and(|(_, (end, _))| *end >= c0)
}
/// Adds merge `m`, at row `row`, unless it overlaps an active one.
fn add(&mut self, m: RangeRef, row: u32) -> bool {
self.advance(row);
if self.meets(m.start.col, m.end.col) {
return false;
}
self.cols.insert(m.start.col, (m.end.col, m.end.row));
self.ends.push(std::cmp::Reverse((m.end.row, m.start.col)));
true
}
}
/// The merged areas that meet `block`, ordered by the row where they enter it, then by column,
/// leaving out any that overlaps one kept before it (only a damaged file has overlapping merges;
/// Excel drops them when it repairs one). One sweep down the rows instead of a scan of every
/// merge per cell.
fn block_merges(sh: &Sheet, block: RangeRef) -> Vec<RangeRef> {
let top = |m: &RangeRef| m.start.row.max(block.start.row);
let mut merges: Vec<RangeRef> = sh.merges.iter().filter(|m| m.intersects(&block)).copied().collect();
merges.sort_by_key(|m| (top(m), m.start.col));
let mut sweep = MergeSweep::default();
merges.retain(|m| sweep.add(*m, top(m)));
merges
}
/// Answers "is this cell inside a merge?" for cells asked in row-major order, from the merges of
/// [`block_merges`].
struct Covered<'a> {
merges: &'a [RangeRef],
top: u32,
next: usize,
sweep: MergeSweep,
}
impl<'a> Covered<'a> {
fn new(merges: &'a [RangeRef], block: RangeRef) -> Self {
Self { merges, top: block.start.row, next: 0, sweep: MergeSweep::default() }
}
fn covers(&mut self, c: CellRef) -> bool {
while let Some(m) = self.merges.get(self.next).filter(|m| m.start.row.max(self.top) <= c.row) {
self.sweep.add(*m, m.start.row.max(self.top));
self.next += 1;
}
self.sweep.advance(c.row);
self.sweep.meets(c.col, c.col)
}
}
/// Draws the cells of a block: fills, gridlines, data bars, borders and text.
pub(super) fn draw_block(pg: &mut Page, b: &Block, gridlines: bool, cf: &mut Option<crate::cf::CfCache>) {
let (bx, by) = (b.xs.first().copied().unwrap_or(0.0), b.ys.first().copied().unwrap_or(0.0));
@@ -237,17 +313,18 @@ pub(super) fn draw_block(pg: &mut Page, b: &Block, gridlines: bool, cf: &mut Opt
}
let block = b.range();
let sh = b.sh;
let merges: Vec<RangeRef> = sh.merges.iter().filter(|m| m.intersects(&block)).copied().collect();
let covered = |c: CellRef| merges.iter().any(|m| m.contains(c));
// Cells worth visiting.
let merges = block_merges(sh, block);
let by_start: std::collections::HashMap<CellRef, RangeRef> = merges.iter().map(|m| (m.start, *m)).collect();
// Cells worth visiting, in row-major order (which `Covered` needs).
let mut covered = Covered::new(&merges, block);
let cells: Vec<CellRef> = if block.count() <= DENSE_BLOCK {
block.iter().filter(|c| !covered(*c)).collect()
block.iter().filter(|c| !covered.covers(*c)).collect()
} else {
let mut set: Vec<CellRef> = sh.cells.iter_range(block).map(|(c, _)| c).collect();
set.extend(sh.spill.range(block.start..=block.end).map(|(c, _)| *c).filter(|c| block.contains(*c)));
set.sort_unstable();
set.dedup();
set.retain(|c| !covered(*c));
set.retain(|c| !covered.covers(*c));
set
};
pg.save();
@@ -277,14 +354,14 @@ pub(super) fn draw_block(pg: &mut Page, b: &Block, gridlines: bool, cf: &mut Opt
.filter(|(c, l)| {
l.fill.is_some()
|| l.cf.is_some()
|| merges.iter().any(|m| m.start == *c)
|| by_start.contains_key(c)
|| !sh.value(*c).is_empty()
|| l.style.borders != Borders::default()
|| l.style.num_fmt.as_str() == "checkbox"
})
.collect();
let rect_of = |c: CellRef| -> (f32, f32, f32, f32, bool) {
match merges.iter().find(|m| m.start == c) {
match by_start.get(&c) {
Some(m) => {
let (x, y, w, h) = b.rect(*m);
(x, y, w, h, true)
@@ -322,7 +399,7 @@ pub(super) fn draw_block(pg: &mut Page, b: &Block, gridlines: bool, cf: &mut Opt
continue;
}
let bd = &l.style.borders;
let (right, bottom) = match merges.iter().find(|m| m.start == *c).filter(|_| merged) {
let (right, bottom) = match by_start.get(c).filter(|_| merged) {
Some(m) => (
b.wb.styles.get(sh.style_id(CellRef::new(m.start.row, m.end.col))).borders.right,
b.wb.styles.get(sh.style_id(CellRef::new(m.end.row, m.start.col))).borders.bottom,
@@ -407,11 +484,12 @@ pub(super) struct PrintedBlock {
pub(super) fn draw_block_links(pg: &mut Page, b: &Block, page: usize, pending: &mut Vec<PendingPdfLink>) {
let range = b.range();
let (Some(&bx), Some(&by), Some(&right), Some(&bottom)) = (b.xs.first(), b.ys.first(), b.xs.last(), b.ys.last()) else { return };
let by_start: std::collections::HashMap<CellRef, RangeRef> = block_merges(b.sh, range).into_iter().map(|m| (m.start, m)).collect();
for (cell, link) in b.sh.hyperlinks.range(range.start..=range.end) {
if !range.contains(*cell) {
continue;
}
let cell_range = b.sh.merges.iter().find(|m| m.start == *cell).copied().unwrap_or(RangeRef::cell(*cell));
let cell_range = by_start.get(cell).copied().unwrap_or(RangeRef::cell(*cell));
let (x, y, w, h) = b.rect(cell_range);
let (x0, y0, x1, y1) = (x.max(bx), y.max(by), (x + w).min(right), (y + h).min(bottom));
if x1 <= x0 || y1 <= y0 {
@@ -503,3 +581,44 @@ pub(super) fn make_block<'a>(
}
Block { wb, si, sh, rows, cols, xs, ys, k, scale: lay.scale }
}
#[cfg(test)]
mod merge_tests {
use super::{CellRef, Covered, RangeRef, Sheet, block_merges};
fn r(a: &str) -> RangeRef {
RangeRef::parse(a).unwrap()
}
#[test]
fn block_merges_drop_overlaps_and_cover_like_a_scan() {
let mut sh = Sheet::new("S");
sh.merges = vec![r("B2:C3"), r("E1:F8"), r("B2:B5"), r("A10:D10"), r("H3:H4"), r("Z1:Z2"), r("C3:D4"), r("A1:A1048576")];
let block = r("B1:H12");
let merges = block_merges(&sh, block);
// B2:B5 and C3:D4 overlap B2:C3, which comes first; Z1:Z2 is outside the block.
assert_eq!(merges, vec![r("E1:F8"), r("B2:C3"), r("H3:H4"), r("A10:D10")]);
let mut covered = Covered::new(&merges, block);
for c in block.iter() {
assert_eq!(covered.covers(c), merges.iter().any(|m| m.contains(c)), "{}", c.a1());
}
// Cells asked sparsely, still in order.
let mut covered = Covered::new(&merges, block);
for (a, want) in [("C1", false), ("F5", true), ("G5", false), ("D10", true), ("H12", false)] {
assert_eq!(covered.covers(CellRef::parse(a).unwrap()), want, "{a}");
}
}
#[test]
fn many_stacked_merges_are_swept_once() {
// 50,000 merges over the same cells: one is kept, and the block is checked against it
// without comparing every cell with every merge.
let mut sh = Sheet::new("S");
sh.merges = (0..50_000u32).map(|i| RangeRef::new(CellRef::new(0, 0), CellRef::new(199, 1 + i % 50))).collect();
let block = r("A1:AZ200");
let merges = block_merges(&sh, block);
assert_eq!(merges, vec![r("A1:B200")]);
let mut covered = Covered::new(&merges, block);
assert_eq!(block.iter().filter(|c| covered.covers(*c)).count(), 400);
}
}
+12 -13
View File
@@ -55,11 +55,12 @@ pub(super) fn delete_sheet(s: &mut Session, p: &Json) -> Result<Json> {
cx.wb.sheets.get(si).map(|s| s.cells.iter().filter_map(|(c, x)| x.formula.clone().map(|f| (c, f))).collect()).unwrap_or_default();
let Some(sh) = cx.wb.sheet_mut(si) else { continue };
for (c, f) in keys {
if let Some(e) = f.expr() {
let ne = gridcraft_formula::adjust::delete_sheet(e, &name, &order);
if let Some(cell) = sh.cells.get_mut(c) {
cell.formula = Some(Arc::new(gridcraft_model::Formula::from_expr(ne)));
}
// Only formulas whose tree changes are rewritten; the others keep their text.
let changed = f.with_expr(|e| Some(gridcraft_formula::adjust::delete_sheet(e.clone(), &name, &order)).filter(|ne| ne != e));
if let Some(Some(ne)) = changed
&& let Some(cell) = sh.cells.get_mut(c)
{
cell.formula = Some(Arc::new(gridcraft_model::Formula::from_expr(ne)));
}
}
}
@@ -109,14 +110,12 @@ pub(super) fn rename_sheet(s: &mut Session, p: &Json) -> Result<Json> {
cx.wb.sheets.get(si).map(|s| s.cells.iter().filter_map(|(c, x)| x.formula.clone().map(|f| (c, f))).collect()).unwrap_or_default();
let Some(sh) = cx.wb.sheet_mut(si) else { continue };
for (c, f) in keys {
if let Some(e) = f.expr() {
let ne = gridcraft_formula::adjust::rename_sheet(e, &old, &name);
let text = gridcraft_formula::print(&ne);
if text != f.text
&& let Some(cell) = sh.cells.get_mut(c)
{
cell.formula = Some(Arc::new(gridcraft_model::Formula::from_expr(ne)));
}
// Compares trees, so unchanged formulas are not printed; they keep their text.
let changed = f.with_expr(|e| Some(gridcraft_formula::adjust::rename_sheet(e.clone(), &old, &name)).filter(|ne| ne != e));
if let Some(Some(ne)) = changed
&& let Some(cell) = sh.cells.get_mut(c)
{
cell.formula = Some(Arc::new(gridcraft_model::Formula::from_expr(ne)));
}
}
for ch in sh.charts.iter_mut() {
+31 -10
View File
@@ -61,6 +61,24 @@ enum Key {
const MAX_CONSOLIDATE_CELLS: u64 = 5_000_000;
/// The cells a consolidated table writes: every row key meets every column key, plus a label for
/// each key on the sides that have labels.
fn output_cells(rows: usize, cols: usize, top: bool, left: bool) -> u64 {
let (r, c) = (rows as u64, cols as u64);
r.saturating_mul(c).saturating_add(if top { c } else { 0 }).saturating_add(if left { r } else { 0 })
}
#[cfg(test)]
#[test]
fn the_output_limit_counts_label_cells() {
// A million row keys and four column keys are 4 million cells, 5 million with the row labels.
assert_eq!(output_cells(1_000_000, 4, false, false), 4_000_000);
assert_eq!(output_cells(1_000_000, 4, false, true), 5_000_000);
assert!(output_cells(1_000_000, 5, false, true) > MAX_CONSOLIDATE_CELLS);
assert_eq!(output_cells(3, 2, true, true), 6 + 2 + 3);
assert_eq!(output_cells(usize::MAX, usize::MAX, true, true), u64::MAX);
}
pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
const CMD: &str = "data.consolidate";
let func = str_param(p, "function").unwrap_or("sum").to_string();
@@ -89,20 +107,20 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
}
None => (d.wb.active_sheet(), d.selection.active),
};
// Keys in first-seen order, each with its index: a lookup is one probe, not a scan.
let mut row_keys: Vec<Key> = Vec::new();
let mut col_keys: Vec<Key> = Vec::new();
let mut row_index: HashMap<Key, usize> = HashMap::new();
let mut col_index: HashMap<Key, usize> = HashMap::new();
let mut row_label: HashMap<Key, String> = HashMap::new();
let mut col_label: HashMap<Key, String> = HashMap::new();
let mut aggs: HashMap<(usize, usize), Agg> = HashMap::new();
let mut total: u64 = 0;
let index_of = |keys: &mut Vec<Key>, k: Key| -> usize {
match keys.iter().position(|x| *x == k) {
Some(i) => i,
None => {
keys.push(k);
keys.len() - 1
}
}
let index_of = |keys: &mut Vec<Key>, index: &mut HashMap<Key, usize>, k: Key| -> usize {
*index.entry(k).or_insert_with_key(|k| {
keys.push(k.clone());
keys.len() - 1
})
};
for t in &srcs {
let (si, r) = parse_range(&d.wb, t, CMD)?;
@@ -131,7 +149,7 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
} else {
Key::Pos(col - c0)
};
cols.push((col, index_of(&mut col_keys, k)));
cols.push((col, index_of(&mut col_keys, &mut col_index, k)));
}
for row in r0..=r.end.row {
let k = if left {
@@ -145,7 +163,7 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
} else {
Key::Pos(row - r0)
};
let ri = index_of(&mut row_keys, k);
let ri = index_of(&mut row_keys, &mut row_index, k);
for (col, ci) in &cols {
aggs.entry((ri, *ci)).or_default().add(&sh.value(CellRef::new(row, *col)));
}
@@ -157,6 +175,9 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
{
return Err(EngineError::Other("The consolidated data doesn't fit below and right of the destination.".into()));
}
if output_cells(row_keys.len(), col_keys.len(), top, left) > MAX_CONSOLIDATE_CELLS {
return Err(bad(CMD, "the consolidated table would be too large"));
}
let mut cells: Vec<(CellRef, Value)> = Vec::new();
if top {
for (ci, k) in col_keys.iter().enumerate() {
+13
View File
@@ -297,3 +297,16 @@ fn consolidate_by_position_and_label() {
assert!(s.execute("data.consolidate", json!({"sources": []})).is_err());
assert!(s.execute("data.consolidate", json!({"sources": ["A1"], "function": "median"})).is_err());
}
/// A hundred thousand positional rows take one key lookup each; a tall source and a wide one
/// whose keys would meet in a table of 160 million cells are refused before it is built.
#[test]
fn consolidate_keys_are_indexed_and_the_output_is_bounded() {
let mut s = s();
s.execute("range.setValues", json!({"range": "A1", "values": [[1], [2]]})).unwrap();
let r = s.execute("data.consolidate", json!({"sources": ["A1:A100000"], "destination": "C1"})).unwrap();
assert_eq!(r["rows"], 100_000);
assert_eq!(n(&s, "Sheet1!C2"), 2.0);
let r = s.execute("data.consolidate", json!({"sources": ["A1:A10000", "A1:XFC1"], "destination": "A1"}));
assert!(r.unwrap_err().to_string().contains("too large"));
}
+39 -1
View File
@@ -8,8 +8,13 @@ use gridcraft_model::{Sheet, Workbook};
use gridcraft_numfmt::{Formatted, NumberFormat, format_value};
/// Parsed number formats are cached by code.
fn parsed(code: &str) -> NumberFormat {
pub(crate) fn parsed(code: &str) -> NumberFormat {
static CACHE: std::sync::LazyLock<Mutex<HashMap<String, NumberFormat>>> = std::sync::LazyLock::new(|| Mutex::new(HashMap::new()));
// A code of more bytes than 4 × the character limit can't be within it: it shows as General
// without being hashed or scanned again for every cell that shares it.
if code.len() > 4 * gridcraft_numfmt::MAX_FORMAT_CODE_CHARS {
return NumberFormat::parse("General");
}
let lock = || CACHE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(f) = lock().get(code) {
return f.clone();
@@ -24,7 +29,12 @@ fn parsed(code: &str) -> NumberFormat {
f
}
/// The parsed format of `code`; one longer than the character limit is a General format that keeps
/// the code it was made from. The engine itself formats through `parsed`, which doesn't copy it.
pub fn number_format(code: &str) -> NumberFormat {
if code.len() > 4 * gridcraft_numfmt::MAX_FORMAT_CODE_CHARS {
return NumberFormat::parse(code);
}
parsed(code)
}
@@ -228,3 +238,31 @@ pub fn stats(sheet: &Sheet, ranges: &[RangeRef]) -> Stats {
}
s
}
#[cfg(test)]
mod tests {
use super::*;
/// A format code far past Excel's 255 characters shows its values as General, for every cell
/// that shares it, without being hashed into the cache each time.
#[test]
fn an_overlong_code_formats_as_general() {
let wb = Workbook::new();
let code = format!("0.00{}", "\"x\"".repeat(400_000));
for _ in 0..1_000 {
assert_eq!(format(&Value::Number(1.5), &code, &wb).text, "1.5");
}
assert_eq!(format(&Value::Number(1.5), "0.00", &wb).text, "1.50");
}
/// The public `number_format` keeps the code of an overlong format; it is General, but not the
/// same format as a real `General`.
#[test]
fn number_format_keeps_an_overlong_code() {
let code = format!("0.00{}", "\"x\"".repeat(2_000));
let f = number_format(&code);
assert!(f.is_general());
assert_eq!(f.code(), code);
assert_ne!(f, number_format("General"));
}
}
+14 -1
View File
@@ -87,6 +87,18 @@ pub fn holds_source_in_corner(src: RangeRef, target: RangeRef) -> bool {
/// Fills `target` (which contains `src` at one edge) from the pattern in `src`.
pub fn fill(cx: &mut Ctx, sheet: usize, src: RangeRef, target: RangeRef, mode: FillMode) -> Result<()> {
fill_except(cx, sheet, src, target, mode, &std::collections::BTreeSet::new())
}
/// [`fill`], leaving the rows in `skip_rows` untouched (Fill Down over a filtered range).
pub fn fill_except(
cx: &mut Ctx,
sheet: usize,
src: RangeRef,
target: RangeRef,
mode: FillMode,
skip_rows: &std::collections::BTreeSet<u32>,
) -> Result<()> {
let custom = cx.wb.custom_lists.clone();
let Some(sh) = cx.wb.sheet(sheet) else { return Ok(()) };
// Direction: down/up if target extends rows, else right/left.
@@ -119,7 +131,7 @@ pub fn fill(cx: &mut Ctx, sheet: usize, src: RangeRef, target: RangeRef, mode: F
(c, sh.cell(c).cloned())
})
.collect();
let targets: Vec<CellRef> = if vertical {
let mut targets: Vec<CellRef> = if vertical {
let rows: Vec<u32> =
if forward { (src.end.row + 1..=target.end.row).collect() } else { (target.start.row..src.start.row).rev().collect() };
rows.into_iter().map(|r| CellRef::new(r, lane)).collect()
@@ -128,6 +140,7 @@ pub fn fill(cx: &mut Ctx, sheet: usize, src: RangeRef, target: RangeRef, mode: F
if forward { (src.end.col + 1..=target.end.col).collect() } else { (target.start.col..src.start.col).rev().collect() };
cols.into_iter().map(|c| CellRef::new(lane, c)).collect()
};
targets.retain(|t| !skip_rows.contains(&t.row));
let fmt = |c: &(CellRef, Option<Cell>)| cx.wb.styles.get(c.1.as_ref().map(|c| c.style).unwrap_or_default()).num_fmt.as_str();
let pattern = if mode.extends_series() || mode == FillMode::Alternate {
let first_fmt = src_cells.first().map(fmt).unwrap_or_default();
+126 -28
View File
@@ -3,12 +3,19 @@
use std::fmt::Write as _;
use gridcraft_core::{CellRef, RangeRef};
use gridcraft_model::{Sheet, Workbook};
use gridcraft_model::{LineIndex, Sheet, Workbook};
use crate::{EngineError, Result};
mod sylk;
/// The function library's answers to what the XLSX and XLS readers and the XLSX writer need to know about functions
/// (which names are built in, which parameters take a single value).
pub const XLSX_FUNCTIONS: gridcraft_xlsx::FunctionInfo = gridcraft_xlsx::FunctionInfo {
is_builtin: |name| gridcraft_functions::lookup(name).is_some(),
is_scalar_arg: |name, i| gridcraft_functions::lookup(name).is_none_or(|s| s.is_scalar_arg(i)),
};
/// File formats GridCraft reads or writes.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum FileKind {
@@ -264,7 +271,8 @@ fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook,
let kind = FileKind::from_path(name);
let sniffed = gridcraft_xlsx::sniff(bytes);
if sniffed == gridcraft_xlsx::Format::Xls {
let (wb, report) = gridcraft_xlsx::read_xls(bytes).map_err(|e| EngineError::Other(format!("We can't open '{name}': {e}")))?;
let (wb, report) =
gridcraft_xlsx::read_xls_with(bytes, XLSX_FUNCTIONS).map_err(|e| EngineError::Other(format!("We can't open '{name}': {e}")))?;
return Ok((wb, report.warnings));
}
if sniffed == gridcraft_xlsx::Format::LegacyBinary {
@@ -289,12 +297,13 @@ fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook,
// Text named `.xlsx` is read as text.
if sniffed == gridcraft_xlsx::Format::Xlsx || (kind == Some(FileKind::Xlsx) && sniffed != gridcraft_xlsx::Format::Csv) {
// Google Sheets' export wrapper is read as the formula inside when GridCraft calculates it (#2871).
let (wb, report) = gridcraft_xlsx::read_xlsx_with(bytes, &gridcraft_calc::is_known_function).map_err(|e| {
EngineError::Other(match (&e, gridcraft_xlsx::oversized_part(bytes)) {
(gridcraft_xlsx::IoError::TooLarge(_), Some((part, size))) => part_too_large_message(name, &part, size),
_ => format!("We can't open '{name}': {e}"),
})
})?;
let (wb, report) = gridcraft_xlsx::read_xlsx_with(bytes, (XLSX_FUNCTIONS, &gridcraft_calc::is_known_function as &dyn Fn(&str) -> bool))
.map_err(|e| {
EngineError::Other(match (&e, gridcraft_xlsx::oversized_part(bytes)) {
(gridcraft_xlsx::IoError::TooLarge(_), Some((part, size))) => part_too_large_message(name, &part, size),
_ => format!("We can't open '{name}': {e}"),
})
})?;
return Ok((wb, report.warnings));
}
if let Some(read) = read_html_table(name, kind, bytes, csv_as_text) {
@@ -502,7 +511,7 @@ pub fn save_bytes_as(wb: &Workbook, path: &str, kind: FileKind) -> Result<Vec<u8
None => named,
};
let book = std::path::Path::new(path).file_name().and_then(std::ffi::OsStr::to_str).unwrap_or_default();
gridcraft_xlsx::write_xlsx_named(named.as_ref().unwrap_or(wb), gridcraft_xlsx::XlsxKind::from_path(path), book)
gridcraft_xlsx::write_xlsx_named_with(named.as_ref().unwrap_or(wb), gridcraft_xlsx::XlsxKind::from_path(path), book, XLSX_FUNCTIONS)
.map_err(|e| EngineError::Other(e.to_string()))
}
FileKind::Xls => {
@@ -567,7 +576,7 @@ pub fn csv_values_bytes(wb: &Workbook) -> Result<Vec<u8>> {
let Some(sh) = wb.sheet(wb.active_sheet()) else { return Ok(Vec::new()) };
gridcraft_xlsx::write_csv(sh, wb.csv_delimiter.unwrap_or(b','), |c| match sh.value(c) {
gridcraft_core::Value::Number(n) if n.is_finite() => {
let shape = crate::display::number_format(wb.styles.get(sh.style_id(c)).num_fmt.as_str()).date_shape(n);
let shape = crate::display::parsed(wb.styles.get(sh.style_id(c)).num_fmt.as_str()).date_shape(n);
match shape.zip(gridcraft_core::date::datetime_from_serial(wb.date_system, n)) {
Some(((date, time), d)) => iso(&d, date, time),
None => n.to_string(),
@@ -672,6 +681,23 @@ fn table_dir(sh: &Sheet) -> &'static str {
const CLIPBOARD_HTML_MAX_BYTES: usize = 4 * 1024 * 1024;
const CLIPBOARD_HTML_MAX_CELLS: u64 = 10_000;
/// The merge covering each cell of `rect`, as `Sheet::merge_at` would find it (the first in the
/// sheet's list), built once so a table lookup is one probe instead of a scan of every merge.
/// `None` when overlapping merges would cost more than a few visits per cell of `rect`: a
/// hostile file's merge list can't multiply the work of a bounded export.
fn merge_index(sh: &Sheet, rect: RangeRef) -> Option<std::collections::HashMap<CellRef, RangeRef>> {
let mut map = std::collections::HashMap::new();
let mut left = rect.count().saturating_mul(4);
for m in &sh.merges {
let Some(part) = m.intersection(&rect) else { continue };
left = left.checked_sub(part.count())?;
for c in part.iter() {
map.entry(c).or_insert(*m);
}
}
Some(map)
}
/// An HTML fragment for the same range as `plain_text`, previously produced by
/// `display::range_text`. Large copies keep plain text only; never return a partial table.
pub(crate) fn range_html(wb: &Workbook, sheet: usize, r: RangeRef, plain_text: &str) -> Option<String> {
@@ -680,6 +706,7 @@ pub(crate) fn range_html(wb: &Workbook, sheet: usize, r: RangeRef, plain_text: &
return None;
}
let sh = wb.sheet(sheet)?;
let merges = merge_index(sh, r)?;
let mut out = ClipboardHtml(String::new());
write!(out, "<table{} style=\"border-collapse:collapse;font-family:Calibri,Carlito,Arial,sans-serif;font-size:11pt\">", table_dir(sh)).ok()?;
for row in r.start.row..=r.end.row {
@@ -689,7 +716,7 @@ pub(crate) fn range_html(wb: &Workbook, sheet: usize, r: RangeRef, plain_text: &
out.write_str("<tr>").ok()?;
for col in r.start.col..=r.end.col {
let c = CellRef::new(row, col);
let span = if let Some(merged) = sh.merges.iter().find(|m| m.contains(c)).and_then(|m| m.intersection(&r)) {
let span = if let Some(merged) = merges.get(&c).and_then(|m| m.intersection(&r)) {
let first_row = (merged.start.row..=merged.end.row).find(|&row| !sh.is_row_hidden(row))?;
if c != CellRef::new(first_row, merged.start.col) {
continue;
@@ -752,6 +779,14 @@ const HTML_EXPORT_MAX_ROWS: u32 = 100_001;
/// tall, wide one) would otherwise be gigabytes of `<td></td>` before finishing (#2387).
const HTML_EXPORT_MAX_CELLS: u64 = 4_000_000;
/// Most bytes of one web page. Every cell may hold long text (shared across many cells in the
/// file), so the cell count alone doesn't bound the page; past this the export fails instead.
const HTML_EXPORT_MAX_BYTES: usize = 256 * 1024 * 1024;
fn html_too_big() -> EngineError {
EngineError::Other(format!("the web page would be over {} MiB. Save as .xlsx or CSV instead.", HTML_EXPORT_MAX_BYTES / (1024 * 1024)))
}
/// A sheet as a standalone HTML table (Save as Web Page). A sheet with pictures gets the sheet's
/// own column widths and row heights so each picture and chart (inline SVG) can sit at its anchor.
pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
@@ -794,6 +829,23 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
r.width()
)));
}
let merges = merge_index(sh, r).ok_or_else(|| EngineError::Other("the sheet's merged cells overlap too much for a web page.".into()))?;
// Visible rows and columns of the table before each line, so a merge's span is counted
// in constant time whatever the file says its size is.
let visible_before = |hidden: &dyn Fn(u32) -> bool, from: u32, to: u32| {
let mut v = Vec::with_capacity((to - from) as usize + 2);
v.push(0u32);
for line in from..=to {
v.push(v.last().copied().unwrap_or(0) + u32::from(!hidden(line)));
}
v
};
let rows_before = visible_before(&|y| sh.is_row_hidden(y), r.start.row, r.end.row);
let cols_before = visible_before(&|x| sh.is_col_hidden(x), r.start.col, r.end.col);
let visible = |before: &[u32], from: u32, a: u32, b: u32| {
let at = |line: u32| before.get((line - from) as usize).copied().unwrap_or(0);
at(b + 1) - at(a)
};
if fixed {
out.push_str("<colgroup>");
for col in (r.start.col..=r.end.col).filter(|&c| !sh.is_col_hidden(c)) {
@@ -812,20 +864,19 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
}
for col in r.start.col..=r.end.col {
let c = CellRef::new(row, col);
if sh.merges.iter().any(|m| m.contains(c) && m.start != c) || (fixed && sh.is_col_hidden(col)) {
let merge = merges.get(&c);
if merge.is_some_and(|m| m.start != c) || (fixed && sh.is_col_hidden(col)) {
continue;
}
let text = crate::display::cell_text(wb, sh, c);
let dir = CellDir::of(sh, c, &text);
let css = html_cell_css(wb, sh, c, dir);
let span = sh
.merges
.iter()
.find(|m| m.start == c)
let span = merge
.map(|m| {
if fixed {
let rows = (m.start.row..=m.end.row).filter(|&y| !sh.is_row_hidden(y)).count();
let cols = (m.start.col..=m.end.col).filter(|&x| !sh.is_col_hidden(x)).count();
// Spans count the table's visible lines: the merge is clipped to the table.
let rows = visible(&rows_before, r.start.row, m.start.row, m.end.row.min(r.end.row));
let cols = visible(&cols_before, r.start.col, m.start.col, m.end.col.min(r.end.col));
format!(" rowspan=\"{rows}\" colspan=\"{cols}\"")
} else {
format!(" rowspan=\"{}\" colspan=\"{}\"", m.height(), m.width())
@@ -833,6 +884,9 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
})
.unwrap_or_default();
let _ = write!(out, "<td style=\"{css}\"{}{span}>{}</td>", dir.attr(), esc(&text));
if out.len() > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
}
out.push_str("</tr>");
}
@@ -840,12 +894,15 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
out.push_str("</table>");
if fixed {
// Sheet units are CSS pixels; the table starts at the used range, not at A1.
let (left, top) = used.map_or((0.0, 0.0), |r| (sh.col_left(r.start.col), sh.row_top(r.start.row)));
// Indexed positions: each anchor would otherwise sum every custom row and column before it.
let ri = LineIndex::new(&sh.rows, sh.default_row_height, gridcraft_core::MAX_ROWS);
let ci = LineIndex::new(&sh.cols, sh.default_col_width, gridcraft_core::MAX_COLS);
let (left, top) = used.map_or((0.0, 0.0), |r| (ci.start(r.start.col), ri.start(r.start.row)));
let place = |a: &gridcraft_model::Anchor| {
format!(
"position:absolute;left:{}px;top:{}px;width:{}px;height:{}px",
sh.col_left(a.cell.col) - left + f64::from(a.dx),
sh.row_top(a.cell.row) - top + f64::from(a.dy),
ci.start(a.cell.col) - left + f64::from(a.dx),
ri.start(a.cell.row) - top + f64::from(a.dy),
a.width.max(0.0),
a.height.max(0.0)
)
@@ -859,15 +916,24 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
esc(&im.alt),
place(&im.anchor)
);
if out.len() > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
}
for ch in &sh.charts {
if let Some(svg) = crate::chart_svg::chart_svg(wb, sheet, ch, &place(&ch.anchor)) {
out.push_str(&svg);
if out.len() > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
}
}
out.push_str("</div>");
}
out.push_str("</body></html>");
if out.len() > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
Ok(out)
}
@@ -879,12 +945,12 @@ thread_local! {
/// While one is alive, [`read_file`] and [`write_file`] on this thread refuse every path: a
/// stored script's steps run inside one (#2178), so no command reaches the file system from a
/// script whatever its parameters are called. Dropping it (also on unwind) lifts the refusal.
pub(crate) struct ScriptSandbox(());
pub struct ScriptSandbox(std::marker::PhantomData<*const ()>);
impl ScriptSandbox {
pub(crate) fn enter() -> Self {
pub fn enter() -> Self {
let _ = SCRIPT_SANDBOX.try_with(|c| c.set(c.get().saturating_add(1)));
ScriptSandbox(())
ScriptSandbox(std::marker::PhantomData)
}
}
@@ -912,6 +978,12 @@ const MAX_READ_BYTES: u64 = 2 * 1024 * 1024 * 1024;
pub fn read_file(path: &str) -> Result<Vec<u8>> {
use std::io::Read;
script_sandbox_check(path)?;
// Checked before opening: opening a FIFO nobody writes to blocks the calling (GUI) thread.
// shortcut: a FIFO swapped in between this check and the open still blocks; close that with
// O_NONBLOCK when the crate takes a libc dependency.
if !std::fs::metadata(path).map_err(|e| EngineError::Other(format!("{path}: {e}")))?.is_file() {
return Err(EngineError::Other(format!("{path}: only regular files can be opened")));
}
let f = std::fs::File::open(path).map_err(|e| EngineError::Other(format!("{path}: {e}")))?;
// A symlink to an endless device reports length 0 but streams forever, so metadata
// alone never bounds the read: require a real file and cap the bytes actually taken.
@@ -943,18 +1015,37 @@ pub fn write_file(path: &str, bytes: &[u8]) -> Result<()> {
// link's target; and it inherits the destination's access restrictions, so saving a
// private workbook doesn't widen who can read it.
let tmp = format!("{path}.sctmp");
let prior = std::fs::metadata(path).ok().map(|m| m.permissions());
let prior_meta = std::fs::metadata(path).ok();
let prior = prior_meta.as_ref().map(std::fs::Metadata::permissions);
let mut opts = std::fs::OpenOptions::new();
opts.write(true).create_new(true);
// Over an existing file the replacement starts owner-only, whatever the umask: its group is
// still the saving user's primary one until it is changed below, and a member of that group
// must not be able to open it (and keep the descriptor) before then.
#[cfg(unix)]
if let Some(p) = &prior {
use std::os::unix::fs::{OpenOptionsExt, PermissionsExt};
opts.mode(p.mode());
if prior.is_some() {
use std::os::unix::fs::OpenOptionsExt;
opts.mode(0o600);
}
let mut f = opts.open(&tmp).map_err(|e| {
let _ = std::fs::remove_file(&tmp);
EngineError::Other(format!("{path}: {e}"))
})?;
// The replacement belongs to the saving user's primary group: restore the old file's group and
// then its mode. When the group can't be restored its access is dropped rather than handed to a
// different group; failing to read that back is a failed save.
#[cfg(unix)]
if let Some(m) = &prior_meta {
use std::os::unix::fs::{MetadataExt, PermissionsExt};
let _ = std::os::unix::fs::fchown(&f, None, Some(m.gid()));
let group_kept = f.metadata().map(|now| now.gid() == m.gid()).map_err(|e| {
let _ = std::fs::remove_file(&tmp);
EngineError::Other(format!("{path}: {e}"))
})?;
// Best effort: a file system that can't change modes keeps the owner-only one. Without the
// group, the old owner and other bits are kept and the group, setuid and setgid bits dropped.
let _ = f.set_permissions(std::fs::Permissions::from_mode(restored_mode(m.mode(), group_kept)));
}
#[cfg(not(unix))]
if let Some(p) = &prior {
let _ = f.set_permissions(p.clone());
@@ -971,6 +1062,13 @@ pub fn write_file(path: &str, bytes: &[u8]) -> Result<()> {
})
}
/// The mode a saved file gets from the file it replaces (`old`): all of it when its group was
/// restored; without the group, the owner and other bits only.
#[cfg(unix)]
fn restored_mode(old: u32, group_kept: bool) -> u32 {
if group_kept { old & 0o7777 } else { old & 0o707 }
}
#[cfg(target_arch = "wasm32")]
pub fn write_file(path: &str, _bytes: &[u8]) -> Result<()> {
script_sandbox_check(path)?;
+103
View File
@@ -51,6 +51,14 @@ fn save_keeps_the_destination_access_restrictions() {
write_file(dest.to_str().unwrap(), b"v2").unwrap();
let mode = std::fs::metadata(&dest).unwrap().permissions().mode() & 0o777;
assert_eq!(mode, 0o600);
// The mode is set outright from the old file's, not through the umask: group and other access
// it had stay as they were.
for old in [0o640, 0o664] {
std::fs::set_permissions(&dest, std::fs::Permissions::from_mode(old)).unwrap();
write_file(dest.to_str().unwrap(), b"v3").unwrap();
assert_eq!(std::fs::metadata(&dest).unwrap().permissions().mode() & 0o777, old);
}
assert!(!dir.join("private.xlsx.sctmp").exists());
// And a brand-new file is created with the restrictive, not permissive, default.
let fresh = dir.join("new.xlsx");
write_file(fresh.to_str().unwrap(), b"v").unwrap();
@@ -59,6 +67,17 @@ fn save_keeps_the_destination_access_restrictions() {
let _ = std::fs::remove_dir_all(&dir);
}
/// Without the old group, a saved file keeps the old owner and other bits and nothing for a group.
#[cfg(not(target_arch = "wasm32"))]
#[cfg(unix)]
#[test]
fn restored_mode_drops_the_group_and_special_bits_when_the_group_is_lost() {
assert_eq!(super::restored_mode(0o100_664, true), 0o664);
assert_eq!(super::restored_mode(0o100_664, false), 0o604);
assert_eq!(super::restored_mode(0o102_444, false), 0o404, "an original read-only file stays read-only, setgid is dropped");
assert_eq!(super::restored_mode(0o104_755, false), 0o705, "setuid is dropped");
}
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn open_refuses_directories_and_unknowable_lengths() {
@@ -351,6 +370,19 @@ fn script_sandbox_refuses_file_access_until_dropped() {
let _ = std::fs::remove_dir_all(&dir);
}
/// The guard's drop lowers the counter of the thread it runs on, so it must not move between threads:
/// naming the trait's method is ambiguous (a compile error) if `ScriptSandbox` is `Send`.
#[test]
fn script_sandbox_is_not_send() {
trait AmbiguousIfSend<A> {
fn check() {}
}
impl<T: ?Sized> AmbiguousIfSend<()> for T {}
struct IsSend;
impl<T: ?Sized + Send> AmbiguousIfSend<IsSend> for T {}
<ScriptSandbox as AmbiguousIfSend<_>>::check();
}
/// A minimal compound (CFB/OLE2) file — FAT in sector 0, directory in sector 1 — holding a
/// root entry and the given stream names.
fn cfb(streams: &[&str]) -> Vec<u8> {
@@ -761,3 +793,74 @@ fn sylk_survives_huge_relative_offsets_and_width_floods() {
let (wb, _) = open_bytes("widths.slk", &flood).unwrap();
assert_eq!(wb.sheet(0).unwrap().value(CellRef::parse("A1").unwrap()), gridcraft_core::Value::Number(1.0));
}
/// A FIFO with no writer would block an open forever: it is refused before it is opened.
#[cfg(unix)]
#[test]
fn reading_a_fifo_is_refused_without_blocking() {
let dir = std::env::temp_dir().join(format!("gridcraft-fifo-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let fifo = dir.join("pipe.xlsx");
if !std::process::Command::new("mkfifo").arg(&fifo).status().is_ok_and(|s| s.success()) {
let _ = std::fs::remove_dir_all(&dir);
return; // no mkfifo here
}
let err = read_file(fifo.to_str().unwrap()).unwrap_err().to_string();
assert!(err.contains("only regular files"), "{err}");
let _ = std::fs::remove_dir_all(&dir);
}
/// Merges far from what is copied or exported, in a hostile number, are indexed once instead of
/// being scanned for every cell; a merge running past the table is clipped to it.
#[test]
fn html_merge_lookups_and_spans_stay_bounded() {
let mut wb = book_with_picture("A1", (0, 0));
let sh = wb.sheet_mut(0).unwrap();
for i in 0..200_000u32 {
sh.merges.push(RangeRef::new(CellRef::new(500_000 + i * 2, 5), CellRef::new(500_000 + i * 2, 6)));
}
sh.merges.push(RangeRef::new(CellRef::new(0, 0), CellRef::new(gridcraft_core::MAX_ROWS - 1, 1)));
sh.set_value(CellRef::new(2, 1), gridcraft_core::Value::Number(2.0));
let html = copied(&wb, "A1:C50");
assert!(html.contains("rowspan=\"50\" colspan=\"2\""), "{html}");
let page = to_html(&wb, 0).unwrap();
// The table is A1:B3 (the used range): the whole-height merge spans its three rows only.
assert!(page.contains("rowspan=\"3\" colspan=\"2\""), "{page}");
// Merges stacked over one another past a few per cell are refused, not walked.
let sh = wb.sheet_mut(0).unwrap();
sh.merges = vec![RangeRef::new(CellRef::new(0, 0), CellRef::new(2, 1)); 100];
assert!(to_html(&wb, 0).is_err());
assert!(range_html(&wb, 0, RangeRef::parse("A1:B3").unwrap(), "").is_none());
}
/// Thousands of pictures over thousands of custom rows: each is placed through a row index.
#[test]
fn html_export_places_many_pictures_over_many_custom_rows() {
let mut wb = book_with_picture("A1", (0, 0));
let sh = wb.sheet_mut(0).unwrap();
for row in 0..50_000 {
sh.rows.insert(row, gridcraft_model::LineInfo { size: Some(30.0), ..Default::default() });
}
let image = sh.images[0].clone();
for i in 0..5_000u32 {
let mut im = image.clone();
im.anchor.cell = CellRef::new(40_000 + i, 0);
sh.images.push(im);
}
let page = to_html(&wb, 0).unwrap();
assert!(page.contains(&format!("top:{}px", 30 * 44_999)), "last picture placed by the index");
}
/// One long text shown by many cells: the page fails at its byte limit instead of growing on.
#[test]
fn html_export_refuses_a_page_past_its_byte_limit() {
let mut wb = Workbook::new();
let sh = wb.sheet_mut(0).unwrap();
let text = gridcraft_core::Value::text("x".repeat(32_000));
for row in 0..10_000 {
sh.set_value(CellRef::new(row, 0), text.clone());
}
let err = to_html(&wb, 0).unwrap_err();
assert!(err.to_string().contains("MiB"), "{err}");
}
+1 -1
View File
@@ -440,7 +440,7 @@ pub fn infer_input(wb: &Workbook, sheet: usize, r: RangeRef) -> NamedInput {
Some(match v {
Value::Empty => return None,
Value::Text(t) if t.is_empty() => return None,
Value::Number(_) if crate::display::number_format(wb.styles.get(sh.style_id(c)).num_fmt.as_str()).is_date() => ColType::Date,
Value::Number(_) if crate::display::parsed(wb.styles.get(sh.style_id(c)).num_fmt.as_str()).is_date() => ColType::Date,
Value::Number(_) => ColType::Number,
Value::Text(_) => ColType::Text,
Value::Bool(_) => ColType::Bool,
+1 -1
View File
@@ -335,7 +335,7 @@ fn check_input(wb: &Workbook, input: &gridcraft_model::NamedInput, place: Place)
gridcraft_model::ColType::Number => v.is_number(),
gridcraft_model::ColType::Text => v.is_text(),
gridcraft_model::ColType::Bool => matches!(v, Value::Bool(_)),
gridcraft_model::ColType::Date => v.is_number() && crate::display::number_format(&date()).is_date(),
gridcraft_model::ColType::Date => v.is_number() && crate::display::parsed(&date()).is_date(),
};
if !ok {
wrong.push((c, holds(&v, ty)));
+1 -1
View File
@@ -182,7 +182,7 @@ pub fn read_source(wb: &Workbook, text: &str, default_sheet: usize) -> Result<So
for (n, a) in numeric.iter_mut().zip(any.iter()) {
*n = *n && *a;
}
let dates = numeric.iter().zip(formats.iter()).map(|(n, f)| *n && crate::display::number_format(f).is_date()).collect();
let dates = numeric.iter().zip(formats.iter()).map(|(n, f)| *n && crate::display::parsed(f).is_date()).collect();
Ok(Source { sheet: si, range: r, headers, formats, numeric, dates, rows })
}
+1 -1
View File
@@ -21,7 +21,7 @@ pub(super) struct FieldItems {
}
fn cmp_items(a: &Item, b: &Item) -> Ordering {
a.class.cmp(&b.class).then(a.num.total_cmp(&b.num)).then_with(|| a.text.cmp(&b.text))
a.class.cmp(&b.class).then(a.num.total_cmp(&b.num)).then_with(|| gridcraft_core::compare_text(&a.text, &b.text))
}
/// (identity key, item) for a source value under a field's date grouping.
+412
View File
@@ -18,6 +18,44 @@ fn v(s: &Session, a: &str) -> Value {
s.doc().unwrap().wb.active().unwrap().value(CellRef::parse(a).unwrap())
}
#[test]
fn format_cells_reading_order_survives_undo_and_xlsx() {
let mut s = s();
s.execute("cell.set", json!({"cell": "A1", "input": "مرحبا 123 ABC"})).unwrap();
s.execute("home.bold", json!({"range": "A1"})).unwrap();
s.execute("home.formatCells", json!({"range": "A1", "style": {"align": {"reading_order": "RightToLeft"}}})).unwrap();
let alignment = |s: &Session| {
let d = s.doc().unwrap();
let style = d.wb.styles.get(d.wb.active().unwrap().style_id(CellRef::new(0, 0)));
assert!(style.font.bold);
serde_json::to_value(style.align).unwrap()
};
assert_eq!(alignment(&s)["reading_order"], "RightToLeft");
s.execute("edit.undo", json!({})).unwrap();
assert!(alignment(&s)["reading_order"].is_null());
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(alignment(&s)["reading_order"], "RightToLeft");
for order in [json!("invalid"), json!(2)] {
assert!(s.execute("home.formatCells", json!({"range": "A1", "style": {"align": {"reading_order": order}}})).is_err());
assert_eq!(alignment(&s)["reading_order"], "RightToLeft");
}
let wb = &s.doc().unwrap().wb;
let (reopened, _) = gridcraft_xlsx::read_xlsx(&gridcraft_xlsx::write_xlsx(wb).unwrap()).unwrap();
let cell = CellRef::new(0, 0);
assert_eq!(reopened.styles.get(reopened.active().unwrap().style_id(cell)), wb.styles.get(wb.active().unwrap().style_id(cell)));
assert_eq!(reopened.active().unwrap().value(cell), Value::text("مرحبا 123 ABC"));
for order in ["LeftToRight", "Context"] {
s.execute("home.formatCells", json!({"range": "A1", "style": {"align": {"reading_order": order}}})).unwrap();
assert_eq!(alignment(&s)["reading_order"], order);
}
s.execute("home.formatCells", json!({"range": "A1", "style": {"align": {"reading_order": null}}})).unwrap();
assert!(alignment(&s)["reading_order"].is_null());
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(alignment(&s)["reading_order"], "Context");
s.execute("edit.redo", json!({})).unwrap();
assert!(alignment(&s)["reading_order"].is_null());
}
#[test]
fn entering_empty_cells_preserves_whole_sheet_formatting() {
let mut s = s();
@@ -1459,6 +1497,28 @@ fn case_sensitive_sort_puts_lowercase_first() {
assert_eq!(col, ["a", "A", "aB", "Ab", "b", "B"].map(Value::from));
}
#[test]
fn data_sort_orders_vowelled_arabic_by_letters() {
let mut s = s();
s.execute("range.setValues", json!({"range": "A1", "values": [["بيت"], ["بَاب"], ["تمر"], ["\u{200F}بحر"]]})).unwrap();
s.execute("data.sort", json!({"range": "A1:A4", "header": false, "keys": [{"column": "A"}]})).unwrap();
let col: Vec<Value> = (1..=4).map(|r| v(&s, &format!("A{r}"))).collect();
assert_eq!(col, ["بَاب", "\u{200F}بحر", "بيت", "تمر"].map(Value::from));
}
#[test]
fn data_sort_orders_marks_and_digit_scripts_like_excel() {
// Excel 16.0 (build 20527, en-US) sorts this column, as text, into exactly this order.
let want = ["١٢", "۱۲", "ب\u{064E}1", "ب\u{064E}١", "بa", "\u{200F}باب", "ب\u{064B}اب", "ب\u{064E}اب", "ب\u{0652}اب", "ب\u{0651}اب"];
let mut s = s();
s.execute("home.numberFormat", json!({"range": "A1:A10", "code": "@"})).unwrap();
let input: Vec<Vec<&str>> = want.iter().rev().map(|t| vec![*t]).collect();
s.execute("range.setValues", json!({"range": "A1", "values": input})).unwrap();
s.execute("data.sort", json!({"range": "A1:A10", "header": false, "keys": [{"column": "A"}]})).unwrap();
let col: Vec<Value> = (1..=10).map(|r| v(&s, &format!("A{r}"))).collect();
assert_eq!(col, want.map(Value::from));
}
#[test]
fn sort_with_a_sheet_prefix_sorts_the_named_sheet() {
let col = |s: &Session, sheet: &str| -> Vec<Value> {
@@ -2067,6 +2127,19 @@ fn formula_at(s: &Session, a: &str) -> Option<String> {
s.doc().unwrap().wb.active().unwrap().cell(CellRef::parse(a).unwrap()).and_then(|c| c.formula.as_ref().map(|f| f.text.clone()))
}
#[test]
fn autosum_of_a_single_cell_stops_at_a_blank_cell() {
// #3291: a blank row between two blocks of numbers ended the guess one block too late, so the
// second total summed the first block (and its total) as well.
let mut s = s();
s.execute("range.setValues", json!({"range": "B4", "values": [[1], [2], [3]]})).unwrap();
s.execute("cell.set", json!({"cell": "B7", "input": "=SUM(B4:B6)"})).unwrap();
s.execute("range.setValues", json!({"range": "B9", "values": [[10], [20]]})).unwrap();
s.execute("selection.set", json!({"cell": "B11"})).unwrap();
s.execute("formulas.autoSum", json!({})).unwrap();
assert_eq!(v(&s, "B11"), Value::Number(30.0));
}
#[test]
fn autosum_over_a_row_totals_it_to_the_right() {
// As in Excel: a row selected and AutoSum puts one total to its right, not a one-cell total
@@ -3591,6 +3664,79 @@ fn xls_import_keeps_formulas_and_never_saves_back_to_xls() {
assert!(s.execute("file.saveBytes", json!({"format": "xlsx"})).is_ok());
}
#[test]
fn xls_formulas_calculate_with_legacy_implicit_intersection() {
// Data!A1 = 11, B1:G1 = 1, A2 = 5, A3 = 7. Report!B1:G1 are filled, so a formula in Report!A1
// that spilled would show #SPILL!. Every cached result is 0, so the values below are GridCraft's.
fn rec(id: u16, payload: &[u8]) -> Vec<u8> {
[&id.to_le_bytes()[..], &(payload.len() as u16).to_le_bytes(), payload].concat()
}
let u16s = |v: &[u16]| v.iter().flat_map(|x| x.to_le_bytes()).collect::<Vec<u8>>();
let bof = |dt: u16| rec(0x0809, &[&0x0600u16.to_le_bytes()[..], &dt.to_le_bytes(), &[0; 12]].concat());
let number = |r: u16, c: u16, n: f64| rec(0x0203, &[&u16s(&[r, c, 0])[..], &n.to_le_bytes()].concat());
let formula = |r: u16, c: u16, rgce: &[u8]| {
rec(0x0006, &[&u16s(&[r, c, 0])[..], &0f64.to_le_bytes(), &[0; 6], &(rgce.len() as u16).to_le_bytes(), rgce].concat())
};
let area3d = |r0: u16, r1: u16, c0: u16, c1: u16| [&[0x3Bu8, 0, 0][..], &u16s(&[r0, r1, c0, c1])].concat();
let a2a3 = area3d(1, 2, 0, 0);
let mut data = bof(0x0010);
data.extend([number(0, 0, 11.0), number(1, 0, 5.0), number(2, 0, 7.0)].concat());
(1..7).for_each(|c| data.extend(number(0, c, 1.0)));
data.extend(rec(0x000A, &[]));
let mut report = bof(0x0010);
(1..7).for_each(|c| report.extend(number(0, c, 0.0)));
// A1 = Data!A1:G1; B2 = Data!$A$2:$A$3*10; C2 = SQRT(Data!$A$2:$A$3) (row 2: SQRT(5));
// H1 = {SUM(Data!$A$2:$A$3*2)}, a one-cell Ctrl+Shift+Enter array.
let cse = [&a2a3[..], &[0x1E, 2, 0, 0x05, 0x22, 1, 4, 0]].concat();
report.extend(formula(0, 0, &area3d(0, 0, 0xC000, 6 | 0xC000)));
report.extend(formula(1, 1, &[&a2a3[..], &[0x1E, 10, 0, 0x05]].concat()));
report.extend(formula(1, 2, &[&a2a3[..], &[0x21, 20, 0]].concat()));
report.extend(formula(0, 7, &[1, 0, 0, 7, 0]));
report.extend(rec(0x0221, &[&u16s(&[0, 0])[..], &[7, 7], &[0; 6], &(cse.len() as u16).to_le_bytes(), &cse].concat()));
report.extend(rec(0x000A, &[]));
let boundsheet = |pos: u32, name: &str| rec(0x0085, &[&pos.to_le_bytes()[..], &[0, 0, name.len() as u8, 0], name.as_bytes()].concat());
let globals = |p1: u32, p2: u32| {
[
bof(0x0005),
boundsheet(p1, "Data"),
boundsheet(p2, "Report"),
rec(0x01AE, &u16s(&[2, 0x0401])),
rec(0x0017, &u16s(&[1, 0, 0, 0])),
rec(0x000A, &[]),
]
.concat()
};
let p1 = globals(0, 0).len() as u32;
let stream = [globals(p1, p1 + data.len() as u32), data, report].concat();
let bytes = gridcraft_xlsx::testing::compound_file(&[("Workbook", &stream)]);
let mut s = s();
s.execute("file.open", json!({"name": "legacy.xls", "base64": crate::io::base64_encode(&bytes)})).unwrap();
s.execute("formulas.calculateNow", json!({})).unwrap();
let report = |s: &Session, a1: &str| s.doc().unwrap().wb.sheets[1].value(CellRef::parse(a1).unwrap());
assert_eq!(report(&s, "A1"), Value::Number(11.0));
assert_eq!(report(&s, "B2"), Value::Number(50.0));
assert_eq!(report(&s, "C2"), Value::Number(5f64.sqrt()));
assert_eq!(report(&s, "H1"), Value::Number(24.0));
// Nothing spilled out of A1 or B2.
assert_eq!(report(&s, "B1"), Value::Number(0.0));
assert_eq!(report(&s, "B3"), Value::Empty);
// Saved as .xlsx, the formulas carry no dynamic-array metadata.
let r = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let saved = crate::io::base64_decode(r["base64"].as_str().unwrap()).unwrap();
let (wb, _) = gridcraft_xlsx::read_xlsx(&saved).unwrap();
let f = |a1: &str| wb.sheets[1].cell(CellRef::parse(a1).unwrap()).and_then(|c| c.formula.clone()).unwrap();
assert_eq!(f("A1").text, "@Data!A1:G1");
assert_eq!((f("H1").text.as_str(), f("H1").array), ("SUM(Data!$A$2:$A$3*2)", gridcraft_core::RangeRef::parse("H1")));
let mut sheet = String::new();
std::io::Read::read_to_string(
&mut zip::ZipArchive::new(std::io::Cursor::new(&saved)).unwrap().by_name("xl/worksheets/sheet2.xml").unwrap(),
&mut sheet,
)
.unwrap();
assert!(!sheet.contains("cm=") && !sheet.contains("SINGLE"), "{sheet}");
}
#[test]
fn malformed_xlsb_keeps_the_current_workbook() {
let mut s = s();
@@ -4984,6 +5130,246 @@ fn precision_as_displayed_rounds_numbers_written_to_a_new_sheet() {
assert!(rounded > 0, "the histogram's formatted cells should be rounded");
}
/// #1101: repeating one cell or row down a filtered selection (paste, Ctrl+Enter, Fill Down) skips
/// the rows the filter hides, as in Excel. A taller pasted block goes straight down, and rows
/// hidden by hand are written, also as in Excel (each case measured in Excel for Microsoft 365).
#[test]
fn fills_and_single_row_pastes_skip_filtered_rows() {
let fresh = s;
let filtered = || {
let mut s = fresh();
let mut rows = vec![json!(["Val", "Tag"])];
for i in 2..=11 {
rows.push(json!([format!("v{i}"), if [3, 5, 7, 9].contains(&i) { "hide" } else { "keep" }]));
}
s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap();
s.execute("range.setValues", json!({"range": "E1", "values": [["X", "W"], ["Y", ""]]})).unwrap();
s.execute("selection.set", json!({"range": "A1:B1"})).unwrap();
s.execute("data.filter", json!({})).unwrap();
assert_eq!(s.execute("data.filterBy", json!({"column": "B", "values": ["keep"]})).unwrap()["hiddenRows"], 4);
s
};
let col = |s: &Session, c: &str| -> Vec<Value> { (2..=11).map(|r| v(s, &format!("{c}{r}"))).collect() };
let skipped =
|x: &str| -> Vec<Value> { (2..=11).map(|r| Value::from(if [3, 5, 7, 9].contains(&r) { format!("v{r}") } else { x.to_string() })).collect() };
// One cell pasted over the filtered rows.
let mut s = filtered();
s.execute("edit.copy", json!({"range": "E1"})).unwrap();
s.execute("selection.set", json!({"range": "A2:A11"})).unwrap();
s.execute("edit.paste", json!({})).unwrap();
assert_eq!(col(&s, "A"), skipped("X"));
// One row (two cells) pasted over the filtered rows.
let mut s = filtered();
s.execute("edit.copy", json!({"range": "E1:F1"})).unwrap();
s.execute("selection.set", json!({"range": "A2:B11"})).unwrap();
s.execute("edit.paste", json!({})).unwrap();
assert_eq!(col(&s, "A"), skipped("X"));
assert_eq!(v(&s, "B3"), Value::from("hide"));
assert_eq!(v(&s, "B4"), Value::from("W"));
// A two-row block is pasted straight down, hidden rows included.
let mut s = filtered();
s.execute("edit.copy", json!({"range": "E1:E2"})).unwrap();
s.execute("selection.set", json!({"range": "A2:A11"})).unwrap();
s.execute("edit.paste", json!({})).unwrap();
assert_eq!(col(&s, "A"), ["X", "Y", "X", "Y", "X", "Y", "X", "Y", "X", "Y"].map(Value::from));
// Ctrl+Enter.
let mut s = filtered();
s.execute("selection.set", json!({"range": "A2:A11"})).unwrap();
s.execute("range.fill", json!({"input": "Z"})).unwrap();
assert_eq!(col(&s, "A"), skipped("Z"));
// Fill Down, and a formula filled down still adjusts to the row it lands in.
let mut s = filtered();
s.execute("edit.fillDown", json!({"range": "A2:A11"})).unwrap();
assert_eq!(col(&s, "A"), skipped("v2"));
s.execute("cell.set", json!({"cell": "C2", "input": "=ROW()"})).unwrap();
s.execute("edit.fillDown", json!({"range": "C2:C11"})).unwrap();
assert_eq!(v(&s, "C4"), Value::Number(4.0));
assert!(v(&s, "C3").is_empty());
// Fill Up and Fill Right skip them too.
let mut s = filtered();
s.execute("edit.fillUp", json!({"range": "A2:A11"})).unwrap();
assert_eq!(col(&s, "A"), skipped("v11"));
let mut s = filtered();
s.execute("range.setValues", json!({"range": "C2", "values": [["c2"], ["c3"], ["c4"]]})).unwrap();
s.execute("edit.fillRight", json!({"range": "A2:C4"})).unwrap();
assert_eq!([v(&s, "C2"), v(&s, "C3"), v(&s, "C4")], [Value::from("v2"), Value::from("c3"), Value::from("v4")]);
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(v(&s, "C4"), Value::from("c4"));
// Copying the filtered range takes only the rows showing, formulas as their values; a cut
// still moves every row.
let mut s = filtered();
s.execute("cell.set", json!({"cell": "C2", "input": "=ROW()"})).unwrap();
s.execute("edit.fillDown", json!({"range": "C2:C11"})).unwrap();
let r = s.execute("edit.copy", json!({"range": "A1:C11"})).unwrap();
assert_eq!(
r["text"],
"Val Tag
v2 keep 2
v4 keep 4
v6 keep 6
v8 keep 8
v10 keep 10
v11 keep 11
"
);
s.execute("edit.paste", json!({"at": "H20"})).unwrap();
let pasted: Vec<Value> = (20..=27).map(|r| v(&s, &format!("H{r}"))).collect();
assert_eq!(pasted, ["Val", "v2", "v4", "v6", "v8", "v10", "v11"].map(Value::from).into_iter().chain([Value::Empty]).collect::<Vec<_>>());
assert_eq!(v(&s, "J22"), Value::Number(4.0));
assert!(s.doc().unwrap().wb.active().unwrap().cell(CellRef::parse("J22").unwrap()).unwrap().formula.is_none());
let mut s = filtered();
s.execute("edit.cut", json!({"range": "A1:B11"})).unwrap();
s.execute("edit.paste", json!({"at": "H20"})).unwrap();
assert_eq!(v(&s, "H22"), Value::from("v3"));
// Rows hidden by hand (no filter) are written, as in Excel.
let mut s = fresh();
s.execute("range.setValues", json!({"range": "A1", "values": [["a"], ["b"], ["c"]]})).unwrap();
s.execute("home.hideRows", json!({"rows": "2:2"})).unwrap();
s.execute("selection.set", json!({"range": "A1:A3"})).unwrap();
s.execute("range.fill", json!({"input": "Z"})).unwrap();
assert_eq!(v(&s, "A2"), Value::from("Z"));
}
/// A table of ten rows (A1:B11) with a link on every value cell, four of its rows hidden by a
/// filter (3, 5, 7, 9), and a spare corner at E1 to copy from.
fn filtered_rows_with_links() -> Session {
let mut s = s();
let mut rows = vec![json!(["Val", "Tag"])];
for i in 2..=11 {
rows.push(json!([format!("v{i}"), if [3, 5, 7, 9].contains(&i) { "hide" } else { "keep" }]));
}
s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap();
s.execute("range.setValues", json!({"range": "E1", "values": [["X"]]})).unwrap();
let sh = std::sync::Arc::make_mut(&mut s.doc_mut().unwrap().wb).sheet_mut(0).unwrap();
for r in 2..=11u32 {
sh.hyperlinks
.insert(CellRef::new(r - 1, 0), gridcraft_model::Hyperlink { target: format!("https://example.com/{r}"), tooltip: None, external: true });
}
s.execute("selection.set", json!({"range": "A1:B1"})).unwrap();
s.execute("data.filter", json!({})).unwrap();
s.execute("data.filterBy", json!({"column": "B", "values": ["keep"]})).unwrap();
s
}
fn cell_link_target(s: &Session, cell: &str) -> Option<String> {
s.doc().unwrap().wb.active().unwrap().hyperlinks.get(&CellRef::parse(cell).unwrap()).map(|h| h.target.clone())
}
fn cell_validation(s: &Session, cell: &str) -> Option<String> {
let c = CellRef::parse(cell).unwrap();
s.doc().unwrap().wb.active().unwrap().validations.iter().find(|d| d.ranges.iter().any(|r| r.contains(c))).map(|d| d.f1.clone())
}
/// Copying a filtered range packs the rows showing into one block; their links and validation
/// rules travel with them, and those of the hidden rows are left behind.
#[test]
fn filtered_copy_pastes_links_and_validation_of_the_rows_showing_only() {
let mut s = filtered_rows_with_links();
for (cell, source) in [("A3", "\"hidden\""), ("A4", "\"showing\""), ("A9", "\"hidden too\"")] {
s.execute("data.validation", json!({"range": cell, "type": "list", "formula1": source})).unwrap();
}
s.execute("edit.copy", json!({"range": "A1:B11"})).unwrap();
s.execute("edit.paste", json!({"at": "H20"})).unwrap();
let links: Vec<Option<String>> = (20..=26).map(|r| cell_link_target(&s, &format!("H{r}"))).collect();
let expected: Vec<Option<String>> =
[None, Some(2u32), Some(4), Some(6), Some(8), Some(10), Some(11)].iter().map(|n| n.map(|n| format!("https://example.com/{n}"))).collect();
assert_eq!(links, expected);
let rules: Vec<Option<String>> = (20..=26).map(|r| cell_validation(&s, &format!("H{r}"))).collect();
assert_eq!(rules, [None, None, Some("\"showing\"".to_string()), None, None, None, None]);
// The source keeps its own links and rules where they were.
assert_eq!(cell_link_target(&s, "A3").as_deref(), Some("https://example.com/3"));
assert_eq!(cell_validation(&s, "A3").as_deref(), Some("\"hidden\""));
}
/// One cell pasted over a filtered column leaves the rules of the rows the filter hides alone, and
/// does not give them the copied rule, with or without Skip Blanks.
#[test]
fn single_row_paste_over_filtered_rows_leaves_their_validation_alone() {
for skip_blanks in [false, true] {
let mut s = filtered_rows_with_links();
s.execute("data.validation", json!({"range": "E1", "type": "list", "formula1": "\"copied\""})).unwrap();
s.execute("data.validation", json!({"range": "A2:A5", "type": "list", "formula1": "\"old\""})).unwrap();
s.execute("edit.copy", json!({"range": "E1"})).unwrap();
s.execute("selection.set", json!({"range": "A2:A11"})).unwrap();
s.execute("edit.pasteSpecial", json!({"what": "all", "skipBlanks": skip_blanks})).unwrap();
let rules: Vec<Option<String>> = (2..=11).map(|r| cell_validation(&s, &format!("A{r}"))).collect();
let showing = Some("\"copied\"".to_string());
let old = Some("\"old\"".to_string());
assert_eq!(
rules,
[showing.clone(), old.clone(), showing.clone(), old, showing.clone(), None, showing.clone(), None, showing.clone(), showing],
"skip blanks: {skip_blanks}"
);
}
}
/// Insert Copied Cells after a filtered copy of whole rows makes room for the rows copied, not for
/// the rows that were selected.
#[test]
fn insert_copied_cells_after_a_filtered_row_copy_inserts_only_the_rows_showing() {
let mut s = filtered_rows_with_links();
s.execute("cell.set", json!({"cell": "A30", "input": "below"})).unwrap();
s.execute("edit.copy", json!({"range": "1:11"})).unwrap();
s.execute("selection.set", json!({"range": "A20"})).unwrap();
s.execute("home.insertCopiedCells", json!({})).unwrap();
// Seven rows were copied (the header and six rows showing).
assert_eq!(v(&s, "A37"), Value::from("below"));
assert_eq!(v(&s, "A20"), Value::from("Val"));
assert_eq!(v(&s, "A26"), Value::from("v11"));
assert_eq!(v(&s, "A27"), Value::Empty);
}
/// Copying a filtered range with many separate runs of rows showing moves each row's note, link and
/// merge with it, and leaves the hidden rows' behind.
#[test]
fn filtered_copy_with_many_runs_moves_notes_and_merges_with_their_rows() {
let mut s = s();
let mut rows = vec![json!(["Val", "Tag"])];
for i in 2..=2001 {
rows.push(json!([format!("v{i}"), if i % 2 == 1 { "hide" } else { "keep" }]));
}
s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap();
let sh = std::sync::Arc::make_mut(&mut s.doc_mut().unwrap().wb).sheet_mut(0).unwrap();
for r in 2..=2001u32 {
let note = gridcraft_model::Comment {
author: "a".into(),
text: format!("note {r}"),
replies: vec![],
threaded: false,
resolved: false,
visible: false,
dates: vec![],
note_box: None,
};
sh.comments.insert(CellRef::new(r - 1, 0), note);
}
sh.merges.push(gridcraft_core::RangeRef::new(CellRef::new(3, 1), CellRef::new(3, 2)));
sh.merges.push(gridcraft_core::RangeRef::new(CellRef::new(2, 1), CellRef::new(2, 2)));
s.execute("selection.set", json!({"range": "A1:B1"})).unwrap();
s.execute("data.filter", json!({})).unwrap();
s.execute("data.filterBy", json!({"column": "B", "values": ["keep"]})).unwrap();
s.execute("edit.copy", json!({"range": "A1:C2001"})).unwrap();
s.execute("edit.paste", json!({"at": "H1"})).unwrap();
let sh = s.doc().unwrap().wb.active().unwrap();
let note = |row: u32| sh.comments.get(&CellRef::new(row - 1, 7)).map(|c| c.text.clone());
assert_eq!(note(1), None);
assert_eq!(note(2).as_deref(), Some("note 2"));
assert_eq!(note(3).as_deref(), Some("note 4"));
assert_eq!(note(1001).as_deref(), Some("note 2000"));
assert_eq!(sh.comments.keys().filter(|c| c.col == 7).count(), 1000);
// Only the merge on a row showing (row 4) came along, packed to the row it landed in (row 3).
assert!(sh.merges.contains(&gridcraft_core::RangeRef::new(CellRef::new(2, 8), CellRef::new(2, 9))));
assert_eq!(sh.merges.iter().filter(|m| m.start.col >= 7).count(), 1);
}
#[test]
fn editing_quote_prefixed_text_keeps_it_text() {
let mut s = s();
@@ -5244,6 +5630,32 @@ fn paste_validation_skip_blanks_large_populated_paste() {
assert!(validation_areas(&s).is_empty());
}
/// A whole-column copy of a filtered range takes only the rows showing, also when they form one
/// block (the hidden rows are all at the end) and the selection covers the whole column.
#[test]
fn whole_column_copy_of_a_filtered_range_leaves_the_hidden_rows_out() {
for hidden_tail in [false, true] {
let mut s = s();
let mut rows = vec![json!(["Val", "Tag"])];
for i in 2..=11 {
let hide = if hidden_tail { i >= 8 } else { [3, 5, 7, 9].contains(&i) };
rows.push(json!([format!("v{i}"), if hide { "hide" } else { "keep" }]));
}
s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap();
s.execute("selection.set", json!({"range": "A1:B1"})).unwrap();
s.execute("data.filter", json!({})).unwrap();
s.execute("data.filterBy", json!({"column": "B", "values": ["keep"]})).unwrap();
s.execute("cell.set", json!({"cell": "H30", "input": "old"})).unwrap();
s.execute("edit.copy", json!({"range": "A:B"})).unwrap();
s.execute("edit.paste", json!({"at": "H1"})).unwrap();
let col: Vec<Value> = (1..=11).map(|r| v(&s, &format!("H{r}"))).collect();
let want: Vec<&str> =
if hidden_tail { vec!["Val", "v2", "v3", "v4", "v5", "v6", "v7"] } else { vec!["Val", "v2", "v4", "v6", "v8", "v10", "v11"] };
let want: Vec<Value> = want.into_iter().map(Value::from).chain(std::iter::repeat(Value::Empty)).take(11).collect();
assert_eq!(col, want, "hidden rows stay out of a whole-column copy (hidden tail: {hidden_tail})");
}
}
/// Open a workbook by path, and `CELL("filename")` names the file it came from, as Excel does;
/// Save As makes the new file the one it names. The sheet-name formula of #1489 depends on it.
#[test]
@@ -0,0 +1,56 @@
//! Ctrl/Cmd+Shift+arrow searches the active cell's column (or row) from the moving edge (#3484).
use gridcraft_engine::Session;
use serde_json::json;
fn table() -> Session {
// Headers in A2:K2, a key column in A3:A13, nothing else filled.
let mut s = Session::new();
s.new_workbook();
let header: Vec<i64> = (1..=11).collect();
s.execute("range.setValues", json!({"range": "A2:K2", "values": [header]})).unwrap();
let keys: Vec<Vec<i64>> = (1..=11).map(|i| vec![i]).collect();
s.execute("range.setValues", json!({"range": "A3:A13", "values": keys})).unwrap();
s
}
fn ctrl_shift(s: &mut Session, dr: i64, dc: i64) -> String {
s.execute("selection.move", json!({"dr": dr, "dc": dc, "extend": true, "jump": true})).unwrap();
s.doc().unwrap().selection.a1()
}
#[test]
fn ctrl_shift_down_after_ctrl_shift_right_follows_the_active_column() {
let mut s = table();
s.execute("selection.set", json!({"cell": "A2"})).unwrap();
assert_eq!(ctrl_shift(&mut s, 0, 1), "A2:K2");
assert_eq!(ctrl_shift(&mut s, 1, 0), "A2:K13", "column A's data ends on row 13; column K's ends on row 2");
// From the end of column A's data the next press runs to the sheet's last row.
assert_eq!(ctrl_shift(&mut s, 1, 0), "A2:K1048576");
}
#[test]
fn ctrl_shift_right_after_ctrl_shift_down_follows_the_active_row() {
let mut s = table();
s.execute("selection.set", json!({"cell": "A2"})).unwrap();
assert_eq!(ctrl_shift(&mut s, 1, 0), "A2:A13");
assert_eq!(ctrl_shift(&mut s, 0, 1), "A2:K13", "row 2's data ends in column K; row 13's ends in column A");
}
#[test]
fn ctrl_shift_down_after_ctrl_shift_left_follows_the_active_column() {
// The active cell is on the right: column K, empty below the header, decides, not column A.
let mut s = table();
s.execute("selection.set", json!({"cell": "K2"})).unwrap();
assert_eq!(ctrl_shift(&mut s, 0, -1), "A2:K2");
assert_eq!(ctrl_shift(&mut s, 1, 0), "A2:K1048576");
}
#[test]
fn ctrl_shift_right_after_ctrl_shift_up_follows_the_active_row() {
// The active cell is at the bottom: row 13, empty right of the key, decides, not row 2.
let mut s = table();
s.execute("selection.set", json!({"cell": "A13"})).unwrap();
assert_eq!(ctrl_shift(&mut s, -1, 0), "A2:A13");
assert_eq!(ctrl_shift(&mut s, 0, 1), "2:13", "row 13's data ends in column A, so the rows extend to the last column");
}
+2
View File
@@ -7,6 +7,7 @@ mod chart_workbook_names;
mod command_sequence_fuzz;
mod cross_sheet_cut;
mod csv_export_budget;
mod ctrl_shift_arrow_active_line;
mod custom_filter_blanks;
mod custom_view_freeze;
mod dynamicmetadata;
@@ -28,6 +29,7 @@ mod sheet_copy_spill;
mod sheet_name_quoting;
mod sparse_clear_invalidation;
mod stored_list_shape;
mod structural_edit_rewrite;
mod table_resize_overlap;
mod table_total_row_last_row;
mod trace_error;
@@ -0,0 +1,84 @@
//! Structural edits (insert and delete of lines, sheet rename and delete) rewrite only the formulas
//! whose tree they change. A formula they leave alone keeps its stored text and its flags.
use std::sync::Arc;
use gridcraft_core::{CellRef, RangeRef};
use gridcraft_engine::Session;
use gridcraft_model::{Cell, Formula};
use serde_json::{Value as Json, json};
fn run(s: &mut Session, command: &str, params: Json) {
s.execute(command, params).unwrap();
}
fn workbook() -> Session {
let mut s = Session::new();
s.new_workbook();
run(&mut s, "home.insertSheet", json!({"name": "Other"}));
run(&mut s, "home.insertSheet", json!({"name": "Spare"}));
run(&mut s, "sheet.activate", json!({"sheet": "Sheet1"}));
s
}
fn formula(s: &Session, sheet: &str, cell: &str) -> Arc<Formula> {
let wb = &s.doc().unwrap().wb;
wb.sheet(wb.sheet_index(sheet).unwrap()).unwrap().cell(CellRef::parse(cell).unwrap()).unwrap().formula.clone().unwrap()
}
/// Stores a formula with this exact text and no cached tree, as a JSON workbook loads it.
fn put_raw(s: &mut Session, cell: &str, text: &str) {
let wb = Arc::make_mut(&mut s.doc_mut().unwrap().wb);
let i = wb.sheet_index("Sheet1").unwrap();
let f = Formula { text: text.to_string(), expr: None, array: None, dynamic: false };
wb.sheet_mut(i).unwrap().cells.set(CellRef::parse(cell).unwrap(), Cell::formula(f));
}
#[test]
fn an_edit_that_leaves_a_formula_alone_keeps_its_stored_text() {
// The rewrite compares trees, not printed text, so a stored non-canonical text is not
// re-printed by an edit that does not change the formula. An edit that changes it prints it.
let mut s = workbook();
put_raw(&mut s, "C1", "sum(a1, Other!b2)");
run(&mut s, "home.insertRows", json!({"rows": "5:5"}));
run(&mut s, "home.insertColumns", json!({"cols": "F:F"}));
run(&mut s, "sheet.rename", json!({"sheet": "Spare", "name": "Spare2"}));
run(&mut s, "home.deleteSheet", json!({"sheet": "Spare2"}));
assert_eq!(formula(&s, "Sheet1", "C1").text, "sum(a1, Other!b2)");
run(&mut s, "home.insertRows", json!({"rows": "1:1"}));
assert_eq!(formula(&s, "Sheet1", "C2").text, "SUM(A2,Other!B2)");
}
#[test]
fn renaming_a_referenced_sheet_rewrites_the_formula() {
let mut s = workbook();
put_raw(&mut s, "C1", "sum(a1, Other!b2)");
run(&mut s, "sheet.rename", json!({"sheet": "Other", "name": "Data"}));
assert_eq!(formula(&s, "Sheet1", "C1").text, "SUM(A1,Data!B2)");
run(&mut s, "home.deleteSheet", json!({"sheet": "Data"}));
assert_eq!(formula(&s, "Sheet1", "C1").text, "SUM(A1,#REF!)");
}
#[test]
fn inserting_rows_above_a_legacy_array_moves_its_range_when_the_formula_is_unchanged() {
let mut s = workbook();
run(&mut s, "range.setValues", json!({"range": "B1", "values": [[1], [2]]}));
run(&mut s, "selection.set", json!({"range": "C5:C6"}));
run(&mut s, "cell.set", json!({"cell": "C5", "input": "=B1:B2*2", "array": true}));
assert_eq!(formula(&s, "Sheet1", "C5").array, Some(RangeRef::parse("C5:C6").unwrap()));
run(&mut s, "home.insertRows", json!({"rows": "3:3"}));
let f = formula(&s, "Sheet1", "C6");
assert_eq!(f.text, "B1:B2*2");
assert_eq!(f.array, Some(RangeRef::parse("C6:C7").unwrap()));
}
#[test]
fn deleting_an_unrelated_sheet_keeps_legacy_arrays() {
let mut s = workbook();
run(&mut s, "range.setValues", json!({"range": "B1", "values": [[1], [2]]}));
run(&mut s, "selection.set", json!({"range": "C1:C2"}));
run(&mut s, "cell.set", json!({"cell": "C1", "input": "=B1:B2*2", "array": true}));
run(&mut s, "home.deleteSheet", json!({"sheet": "Spare"}));
assert_eq!(formula(&s, "Sheet1", "C1").array, Some(RangeRef::parse("C1:C2").unwrap()));
}
@@ -0,0 +1,366 @@
//! Legacy formulas (no dynamic-array metadata) keep one value through GridCraft (issue #410), on
//! a calculated workbook:
//! - every function that returns several values: a legacy cell formula calling it opens with
//! Excel's implicit `@` and does not spill, and a typed `=@F(…)` saves and reopens with the same
//! single value. The library's functions are found by calling each with sample arguments, so
//! one added later is covered without editing this test.
//! - a legacy formula the analysis cannot finish does not spill and saves as read, without `cm`.
use std::io::Write;
use gridcraft_engine::Session;
use gridcraft_engine::core::{CellRef, DateSystem, Value};
use gridcraft_functions::{Arg, Ctx, FnSpec};
use serde_json::json;
struct Probe;
impl Ctx for Probe {
fn date_system(&self) -> DateSystem {
DateSystem::D1900
}
fn now_serial(&self) -> f64 {
45_000.0
}
fn random(&mut self) -> f64 {
0.5
}
}
/// The 2×2 block of numbers at `A1:B2` in the probe sheets.
fn block() -> Value {
Value::Array(std::sync::Arc::new(
gridcraft_engine::core::Array::new(2, 2, vec![Value::Number(1.0), Value::Number(2.0), Value::Number(3.0), Value::Number(4.0)]).unwrap(),
))
}
/// Sample arguments (formula text, value). Single values only at a position the function lifts
/// over, so an array result is the function's own, not lifting.
fn candidates(scalar: bool) -> Vec<(&'static str, Arg)> {
let mut out = vec![
("3", Arg::val(Value::Number(3.0))),
("1", Arg::val(Value::Number(1.0))),
("\"a1,b22\"", Arg::val(Value::text("a1,b22"))),
("\",\"", Arg::val(Value::text(","))),
("\"[0-9]+\"", Arg::val(Value::text("[0-9]+"))),
("TRUE", Arg::val(Value::Bool(true))),
];
if !scalar {
out.insert(0, ("A1:B2", Arg::reference(block())));
out.insert(1, ("{1,2;3,4}", Arg::val(block())));
}
out
}
/// The first sample call of `spec` that returns more than one value, as formula text.
fn array_call(spec: &FnSpec) -> Option<String> {
let max = spec.max_args.min(spec.min_args.max(1) + 2).min(4).max(spec.min_args);
for n in spec.min_args..=max {
let per: Vec<_> = (0..n).map(|i| candidates(spec.is_scalar_arg(i))).collect();
let per: Vec<_> = if n > 4 { per.into_iter().map(|c| c.into_iter().take(3).collect::<Vec<_>>()).collect() } else { per };
let mut idx = vec![0usize; n];
let mut tries = 0;
loop {
let args: Vec<Arg> = idx.iter().zip(&per).map(|(&i, c)| c[i].1.clone()).collect();
if let Value::Array(a) = gridcraft_functions::call(spec, &args, &mut Probe)
&& a.data.len() > 1
{
let text: Vec<&str> = idx.iter().zip(&per).map(|(&i, c)| c[i].0).collect();
return Some(format!("{}({})", spec.name, text.join(",")));
}
tries += 1;
// Next combination, last position fastest.
let mut k = n;
loop {
if k == 0 || tries > 3000 {
break;
}
k -= 1;
idx[k] += 1;
if idx[k] < per[k].len() {
break;
}
idx[k] = 0;
if k == 0 {
k = usize::MAX;
break;
}
}
if n == 0 || k == usize::MAX || tries > 3000 {
break;
}
}
}
None
}
/// Every library function with an array result, as a sample call, and the functions the
/// evaluator implements itself that return several values or a range.
fn array_calls() -> Vec<String> {
let mut calls: Vec<String> = gridcraft_functions::all().iter().filter_map(array_call).collect();
for found in ["MUNIT", "MODE.MULT", "TEXTSPLIT", "REGEXEXTRACT", "MMULT", "SEQUENCE"] {
assert!(calls.iter().any(|c| c.starts_with(&format!("{found}("))), "{found} not found: {calls:?}");
}
calls.extend(
["TRIMRANGE(A1:B2)", "OFFSET(A1:B2,0,0)", "INDIRECT(\"A1:B2\")", "INDEX(A1:B2,0,1)", "FILTERXML(\"<a><b>1</b><b>2</b></a>\",\"//b\")"]
.map(String::from),
);
calls
}
/// Where the `i`th sample call goes: column E, five rows apart, away from the data in `A1:B2`.
fn slot(i: usize) -> CellRef {
CellRef::new(9 + 5 * i as u32, 4)
}
fn value(s: &Session, c: CellRef) -> Value {
s.doc().unwrap().wb.active().unwrap().value(c)
}
/// The cells right of and below `c` are empty and `c` is not #SPILL!: the formula did not spill.
fn one_value(s: &Session, c: CellRef, what: &str) -> Result<Value, String> {
let v = value(s, c);
let right = value(s, CellRef::new(c.row, c.col + 1));
let below = value(s, CellRef::new(c.row + 1, c.col));
if right != Value::Empty || below != Value::Empty || v.display().contains("SPILL") {
return Err(format!("{what}: {} spilled ({right:?} right, {below:?} below)", v.display()));
}
Ok(v)
}
#[test]
fn legacy_formulas_with_array_functions_show_one_value() {
let calls = array_calls();
let formulas: Vec<_> = calls.iter().enumerate().map(|(i, f)| (slot(i).a1(), f.clone())).collect();
let s = open(&legacy_xlsx(&formulas, ""));
let wrong: Vec<_> = calls.iter().enumerate().filter_map(|(i, f)| one_value(&s, slot(i), f).err()).collect();
assert!(wrong.is_empty(), "{wrong:#?}");
}
#[test]
fn typed_at_of_array_functions_saves_one_value() {
let calls = array_calls();
let mut s = session();
for (a, v) in [("A1", 1), ("B1", 2), ("A2", 3), ("B2", 4)] {
s.execute("cell.set", json!({"cell": a, "input": v.to_string()})).unwrap();
}
for (i, f) in calls.iter().enumerate() {
s.execute("cell.set", json!({"cell": slot(i).a1(), "input": format!("=@{f}")})).unwrap();
}
let before: Vec<_> = (0..calls.len()).map(|i| value(&s, slot(i))).collect();
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let mut back = Session::new();
back.execute("file.open", json!({"name": "typed.xlsx", "base64": saved["base64"]})).unwrap();
let wrong: Vec<_> = calls
.iter()
.enumerate()
.filter_map(|(i, f)| match one_value(&back, slot(i), f) {
Ok(v) if v == before[i] => None,
// A volatile function recalculates on open: one number again.
Ok(Value::Number(_)) if volatile(f) => None,
Ok(v) => Some(format!("@{f}: {} before saving, {} after", before[i].display(), v.display())),
Err(e) => Some(e),
})
.collect();
assert!(wrong.is_empty(), "{wrong:#?}");
}
fn volatile(call: &str) -> bool {
call.split('(').next().and_then(gridcraft_functions::lookup).is_some_and(|spec| spec.volatile)
}
/// Formulas the analysis covers at Excel's nesting limits, and one it cannot finish (names too
/// large to analyse, whose ranges would spill three rows): calculated, none spills, and saving
/// writes each as read, as a plain `<f>`.
#[test]
fn deep_and_unfinished_legacy_formulas_calculate_and_save_as_read() {
let nested = format!("{}A1:A3{}", "ABS(".repeat(200), ")".repeat(200));
let parens = format!("{}A1:A3{}", "ABS((".repeat(64), "))".repeat(64));
let parts = (1..=30).map(|i| format!("Part_{i}")).collect::<Vec<_>>().join("+");
let part = format!("Sheet1!$A$1:$A$3+{}", vec!["Sheet1!$B$1"; 599].join("+"));
let names: String = (1..=30).map(|i| format!(r#"<definedName name="Part_{i}">{part}</definedName>"#)).collect();
let formulas = [("E10".to_string(), nested), ("E15".into(), parens), ("E20".into(), parts)];
let mut s = open(&legacy_xlsx(&formulas, &names));
for (a, _) in &formulas {
one_value(&s, CellRef::parse(a).unwrap(), a).unwrap();
}
// Legacy Excel intersects A1:A3 with row 10 and 15: #VALUE!. The unfinished formula shows the
// top-left value of its sum, 1 + 599 × 2 per name.
assert_eq!(value(&s, CellRef::parse("E10").unwrap()).display(), "#VALUE!");
assert_eq!(value(&s, CellRef::parse("E15").unwrap()).display(), "#VALUE!");
assert_eq!(value(&s, CellRef::parse("E20").unwrap()), Value::Number(30.0 * (1.0 + 599.0 * 2.0)));
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let mut xml = String::new();
std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap();
for (a, f) in &formulas {
assert!(xml.contains(&format!("<f>{f}</f>")), "{a} not saved as read: {xml}");
}
assert!(!xml.contains("cm=") && !xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}");
}
/// Legacy unions where one value is expected, a name whose definition is IF of a range, and
/// multi-valued results in the file's own parentheses: calculated, none spills (legacy Excel
/// intersects a union to #VALUE!), and saving writes each as read, as a plain `<f>`.
#[test]
fn legacy_unions_names_and_parentheses_calculate_and_save_as_read() {
let formulas: Vec<(String, String)> = ["SQRT((A1:A2,B1:B2))", "ABS(((A1:A2,B1:B2)))", "(A1:A2,B1:B2)+1", "SQRT(Flag)", "({1,2}+1)", "{1,2}+1"]
.iter()
.enumerate()
.map(|(i, f)| (slot(i).a1(), f.to_string()))
.collect();
let names = r#"<definedName name="Flag">IF(Sheet1!$A$1:$A$3&gt;1,1,0)</definedName>"#;
let mut s = open(&legacy_xlsx(&formulas, names));
for (a, _) in &formulas {
one_value(&s, CellRef::parse(a).unwrap(), a).unwrap();
}
let shown: Vec<String> = formulas.iter().map(|(a, _)| value(&s, CellRef::parse(a).unwrap()).display()).collect();
assert_eq!(shown, ["#VALUE!", "#VALUE!", "#VALUE!", "0", "2", "2"]);
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let mut xml = String::new();
std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap();
for (a, f) in &formulas {
assert!(xml.contains(&format!("<f>{f}</f>")), "{a} not saved as read: {xml}");
}
assert!(!xml.contains("cm=") && !xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}");
}
/// A whole column in a scalar argument of a legacy formula meets the formula's row (issue #1636,
/// from PR #1660): `ISBLANK(B:B)` in row 4 tests `B4`, so the formula shows `C4` instead of
/// spilling into `G5`, and in row 6, where `B6` is blank, shows 0.
#[test]
fn whole_column_in_a_legacy_scalar_argument_meets_the_formulas_row() {
let formulas: Vec<(String, String)> = [
("B4", r#""Thierry""#),
("C4", "7"),
("G4", r#"IF(ISBLANK(B:B),0,IF(B4="Thierry",C4,0))"#),
("G5", "1"),
("G6", r#"IF(ISBLANK(B:B),0,IF(B6="Thierry",C6,0))"#),
]
.iter()
.map(|(a, f)| (a.to_string(), f.to_string()))
.collect();
let s = open(&legacy_xlsx(&formulas, ""));
let at = |a: &str| CellRef::parse(a).unwrap();
assert_eq!(value(&s, at("G4")), Value::Number(7.0), "B:B meets row 4 instead of spilling");
assert_eq!(value(&s, at("G5")), Value::Number(1.0), "nothing spills below G4");
assert_eq!(one_value(&s, at("G6"), "G6").unwrap(), Value::Number(0.0), "B6 is blank");
}
/// One-cell ranges written as ranges (`B1:B1`) outside any `@`, and formulas whose reading
/// reaches the parser's nesting limit (254 nested functions, far past Excel's 64): saved as read,
/// without `@`. Only a one-cell range under an `@` counts as one an edit may have shrunk. All but
/// the last are calculated without spilling and saved as a plain `<f>`.
#[test]
fn one_cell_ranges_and_formulas_at_the_nesting_limit_save_as_read() {
let deep = |inner: &str| format!("{}{inner}{}", "ABS(".repeat(254), ")".repeat(254));
let formulas: Vec<(String, String)> = [
"SQRT(A1:A3)+B1:B1".to_string(),
"A1:A3*B2:B2".into(),
"SQRT(A1:A3)+$B$2:$B$2".into(),
// Names with a one-cell range inside CHOOSE's index or IF's condition.
"SQRT(NC)".into(),
"SQRT(NI)".into(),
"NC+1".into(),
"NI*2".into(),
deep("A1:A3%"),
deep("-A1:A3"),
]
.into_iter()
.enumerate()
.map(|(i, f)| (slot(i).a1(), f))
.collect();
let names = r#"<definedName name="NC">CHOOSE(Sheet1!$B$1:$B$1,Sheet1!$A$1:$A$3,Sheet1!$A$1:$A$3)</definedName><definedName name="NI">IF(Sheet1!$B$1:$B$1&gt;0,Sheet1!$A$1:$A$3)</definedName>"#;
let (last, rest) = formulas.split_last().unwrap();
let mut s = open(&legacy_xlsx(&formulas, names));
for (a, _) in rest {
one_value(&s, CellRef::parse(a).unwrap(), a).unwrap();
}
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let mut xml = String::new();
std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap();
for (a, f) in rest {
assert!(xml.contains(&format!("<f>{f}</f>")), "{a} not saved as read: {xml}");
}
// A reading the parser cannot nest leaves the formula as read: it calculates as a
// dynamic-array formula would, but the file never gets an `@`.
let (a, f) = last;
assert!(xml.contains(&format!("{f}</f>")), "{a} not saved as read: {xml}");
assert!(!xml.contains('@') && !xml.contains("SINGLE"), "{xml}");
assert_eq!(xml.matches("cm=").count(), 1, "{xml}");
}
/// Deleting rows shrinks the ranges names return (directly, through another name, CHOOSE's and
/// IF's results, parentheses) to one cell: the `@` the reader put on each name is still the one
/// legacy Excel implies, so the formulas save as legacy ones, without `_xlfn.SINGLE`.
#[test]
fn names_shrunk_by_deleting_rows_save_as_legacy() {
let formulas: Vec<(String, String)> =
["SQRT(MM)", "SQRT(CHM)", "SQRT(NCH)", "SQRT(NIF)", "SQRT(PM)"].into_iter().enumerate().map(|(i, f)| (slot(i).a1(), f.to_string())).collect();
let names = r#"<definedName name="MM">Sheet1!$A$1:$A$3</definedName><definedName name="CHM">MM</definedName><definedName name="NCH">CHOOSE(1,Sheet1!$A$1:$A$3,Sheet1!$B$1:$B$3)</definedName><definedName name="NIF">IF(1,Sheet1!$A$1:$A$3,Sheet1!$B$1:$B$3)</definedName><definedName name="PM">(Sheet1!$A$1:$A$3)</definedName>"#;
let mut s = open(&legacy_xlsx(&formulas, names));
s.execute("home.deleteRows", json!({"rows": "2:3"})).unwrap();
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let mut xml = String::new();
std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap();
for (a, f) in &formulas {
assert!(xml.contains(&format!("<f>{f}</f>")), "{a} not saved as legacy {f}: {xml}");
}
assert!(!xml.contains("SINGLE") && !xml.contains("cm="), "{xml}");
}
const CT: &str = r#"<?xml version="1.0"?><Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="xml" ContentType="application/xml"/><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Override PartName="/xl/workbook.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml"/><Override PartName="/xl/worksheets/sheet1.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml"/></Types>"#;
/// A one-sheet package: numbers 1–4 in `A1:B2` and `formulas` (address, file text) saved as
/// legacy formulas, without dynamic-array metadata; `names` is `<definedName>` elements.
fn legacy_xlsx(formulas: &[(String, String)], names: &str) -> Vec<u8> {
let esc = |s: &str| s.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;").replace('"', "&quot;");
let mut rows =
String::from(r#"<row r="1"><c r="A1"><v>1</v></c><c r="B1"><v>2</v></c></row><row r="2"><c r="A2"><v>3</v></c><c r="B2"><v>4</v></c></row>"#);
// Cells in one row share its `<row>`, in the order given.
let mut by_row: Vec<(String, String)> = Vec::new();
for (a, f) in formulas {
let r: String = a.chars().filter(char::is_ascii_digit).collect();
let c = format!(r#"<c r="{a}"><f>{}</f></c>"#, esc(f));
match by_row.iter_mut().find(|(row, _)| *row == r) {
Some((_, cells)) => cells.push_str(&c),
None => by_row.push((r, c)),
}
}
for (r, cells) in by_row {
rows.push_str(&format!(r#"<row r="{r}">{cells}</row>"#));
}
let names = if names.is_empty() { String::new() } else { format!("<definedNames>{names}</definedNames>") };
let parts: [(&str, String); 5] = [
("[Content_Types].xml", CT.into()),
("_rels/.rels", r#"<?xml version="1.0"?><Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="xl/workbook.xml"/></Relationships>"#.into()),
("xl/workbook.xml", format!(r#"<?xml version="1.0"?><workbook xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"><sheets><sheet name="Sheet1" sheetId="1" r:id="rId1"/></sheets>{names}</workbook>"#)),
("xl/_rels/workbook.xml.rels", r#"<?xml version="1.0"?><Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/worksheet" Target="worksheets/sheet1.xml"/></Relationships>"#.into()),
("xl/worksheets/sheet1.xml", format!(r#"<?xml version="1.0"?><worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"><sheetData>{rows}</sheetData></worksheet>"#)),
];
let mut z = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
for (name, data) in parts {
z.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
z.write_all(data.as_bytes()).unwrap();
}
z.finish().unwrap().into_inner()
}
fn session() -> Session {
let mut s = Session::new();
s.new_workbook();
s
}
fn open(bytes: &[u8]) -> Session {
let mut s = Session::new();
s.execute("file.open", json!({"name": "legacy.xlsx", "base64": gridcraft_engine::io::base64_encode(bytes)})).unwrap();
s
}
+50 -23
View File
@@ -8,7 +8,7 @@ use crate::ast::{Anchor, Expr, RefKind, Reference, SheetSel, clamp_anchor};
/// Copy/fill: relative parts move by `(dr, dc)`. References pushed off the sheet become `#REF!`.
pub fn shift_relative(e: Expr, dr: i64, dc: i64) -> Expr {
e.map(&mut |x| match x {
Expr::Ref(r) => match shift_ref(&r, dr, dc) {
Expr::Ref(r) => match shift_ref(r, dr, dc) {
Some(r) => Expr::Ref(r),
None => Expr::Error(CellError::Ref),
},
@@ -60,14 +60,14 @@ fn shift_line(v: u32, abs: bool, d: i64, max: u32) -> Option<u32> {
if (0..max as i64).contains(&n) { Some(n as u32) } else { None }
}
fn shift_ref(r: &Reference, dr: i64, dc: i64) -> Option<Reference> {
fn shift_ref(r: Reference, dr: i64, dc: i64) -> Option<Reference> {
let kind = match &r.kind {
RefKind::Cell(a) => RefKind::Cell(shift_anchor(a, dr, dc)?),
RefKind::Range(a, b) => RefKind::Range(shift_anchor(a, dr, dc)?, shift_anchor(b, dr, dc)?),
RefKind::Rows(r0, a0, r1, a1) => RefKind::Rows(shift_line(*r0, *a0, dr, MAX_ROWS)?, *a0, shift_line(*r1, *a1, dr, MAX_ROWS)?, *a1),
RefKind::Cols(c0, a0, c1, a1) => RefKind::Cols(shift_line(*c0, *a0, dc, MAX_COLS)?, *a0, shift_line(*c1, *a1, dc, MAX_COLS)?, *a1),
};
Some(Reference { sheet: r.sheet.clone(), kind })
Some(Reference { sheet: r.sheet, kind })
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
@@ -95,27 +95,54 @@ pub enum Edit {
MoveToSheet { from: RangeRef, to_sheet: String, to_row: u32, to_col: u32 },
}
/// Whether `r`, in a formula living on sheet `host`, points into sheet `target`.
fn on_target(r: &Reference, host: &str, target: &str) -> bool {
match &r.sheet {
SheetSel::Current => host.eq_ignore_ascii_case(target),
SheetSel::Named(n) => n.eq_ignore_ascii_case(target),
SheetSel::Span(..) => false,
}
}
/// Whether [`adjust`] would change `r`. It runs the same [`adjust_ref`] on a copy without the
/// sheet name, so no name is copied: the adjustment reads only the position, and the sheet a cut
/// to another sheet assigns is compared afterwards.
fn ref_changes(r: &Reference, edit: &Edit) -> bool {
let probe = Reference { sheet: SheetSel::Current, kind: r.kind.clone() };
match adjust_ref(probe, edit) {
None => true,
Some(p) => p.kind != r.kind || (p.sheet != SheetSel::Current && p.sheet != r.sheet),
}
}
/// Whether [`adjust`] would change the formula `e` at all, without copying or rebuilding it.
fn changes(e: &Expr, host: &str, target: &str, edit: &Edit) -> bool {
let mut changed = false;
e.walk(&mut |x| {
if !changed && let Expr::Ref(r) = x {
changed = on_target(r, host, target) && ref_changes(r, edit);
}
});
changed
}
/// Adjusts a formula living on sheet `host` after `edit` on sheet `target`.
pub fn adjust(e: Expr, host: &str, target: &str, edit: &Edit) -> Expr {
e.map(&mut |x| match x {
Expr::Ref(r) => {
let on_target = match &r.sheet {
SheetSel::Current => host.eq_ignore_ascii_case(target),
SheetSel::Named(n) => n.eq_ignore_ascii_case(target),
SheetSel::Span(..) => false,
};
if !on_target {
return Expr::Ref(r);
}
match adjust_ref(&r, edit) {
Some(r) => Expr::Ref(r),
None => Expr::Error(CellError::Ref),
}
}
Expr::Ref(r) if on_target(&r, host, target) => match adjust_ref(r, edit) {
Some(r) => Expr::Ref(r),
None => Expr::Error(CellError::Ref),
},
other => other,
})
}
/// [`adjust`] for a borrowed formula: `None` when the edit leaves it exactly as it is, so a
/// caller can skip unaffected formulas without copying, rebuilding or printing them.
pub fn adjust_if_changed(e: &Expr, host: &str, target: &str, edit: &Edit) -> Option<Expr> {
changes(e, host, target, edit).then(|| adjust(e.clone(), host, target, edit))
}
/// New position of an index after an insert (`None` never) or delete (`None` = deleted).
fn map_index(v: u32, at: u32, count: u32, insert: bool, max: u32) -> Option<u32> {
if insert {
@@ -168,7 +195,7 @@ fn map_span(lo: u32, hi: u32, at: u32, count: u32, insert: bool, max: u32) -> Op
}
}
fn adjust_ref(r: &Reference, edit: &Edit) -> Option<Reference> {
fn adjust_ref(r: Reference, edit: &Edit) -> Option<Reference> {
match edit {
Edit::Insert { axis, at, count } | Edit::Delete { axis, at, count } => {
let insert = matches!(edit, Edit::Insert { .. });
@@ -193,7 +220,7 @@ fn adjust_ref(r: &Reference, edit: &Edit) -> Option<Reference> {
}
(k, _) => k.clone(),
};
Some(Reference { sheet: r.sheet.clone(), kind })
Some(Reference { sheet: r.sheet, kind })
}
Edit::InsertCells { axis, range } | Edit::DeleteCells { axis, range } => {
// Like inserting/deleting lines, for references that lie within the shifted
@@ -204,7 +231,7 @@ fn adjust_ref(r: &Reference, edit: &Edit) -> Option<Reference> {
Axis::Cols => (range.start.row <= span.start.row && span.end.row <= range.end.row, range.start.col, range.width()),
};
if !inside {
return Some(r.clone());
return Some(r);
}
let axis = *axis;
let lines = if matches!(edit, Edit::InsertCells { .. }) { Edit::Insert { axis, at, count } } else { Edit::Delete { axis, at, count } };
@@ -215,7 +242,7 @@ fn adjust_ref(r: &Reference, edit: &Edit) -> Option<Reference> {
let dc = *to_col as i64 - from.start.col as i64;
let range = r.range();
if !from.contains_range(&range) {
return Some(r.clone());
return Some(r);
}
let mv = |a: &Anchor| -> Option<Anchor> {
let (row, col) = clamp_anchor(a.row as i64 + dr, a.col as i64 + dc)?;
@@ -233,11 +260,11 @@ fn adjust_ref(r: &Reference, edit: &Edit) -> Option<Reference> {
RefKind::Cols(shift_line(*a, false, dc, MAX_COLS)?, *aa, shift_line(*b, false, dc, MAX_COLS)?, *ba)
}
};
Some(Reference { sheet: r.sheet.clone(), kind })
Some(Reference { sheet: r.sheet, kind })
}
Edit::MoveToSheet { from, to_sheet, to_row, to_col } => {
if !from.contains_range(&r.range()) {
return Some(r.clone());
return Some(r);
}
let mut moved = adjust_ref(r, &Edit::Move { from: *from, to_row: *to_row, to_col: *to_col })?;
moved.sheet = SheetSel::Named(to_sheet.clone());
+5 -1
View File
@@ -302,11 +302,15 @@ impl Parser {
{
return Ok(Expr::Unary(UnOp::Spill, Box::new(r)));
}
// Files write `@x` as `_xlfn.SINGLE(x)`.
// Files write `@x` as `_xlfn.SINGLE(x)`; `SINGLE(A1:A3*2)` is `@(A1:A3*2)`.
if name == "SINGLE"
&& args.len() == 1
&& let Some(x) = args.pop()
{
let x = match x {
Expr::Binary(..) | Expr::Unary(UnOp::Neg | UnOp::Plus | UnOp::Percent, _) => Expr::Paren(Box::new(x)),
x => x,
};
return Ok(Expr::Unary(UnOp::At, Box::new(x)));
}
// Files may spell a trim reference as a function call (`_xlfn._TRO_ALL(A1:E10)`).
+9
View File
@@ -106,6 +106,15 @@ fn spill_operator() -> Result<(), Box<dyn std::error::Error>> {
Ok(())
}
/// Files spell `@x` as `_xlfn.SINGLE(x)`; an operation inside keeps its parentheses.
#[test]
fn single_is_the_implicit_intersection_operator() {
assert_eq!(rt("_xlfn.SINGLE(A1:A3)*2"), "@A1:A3*2");
assert_eq!(rt("_xlfn.SINGLE(A1:A3*2)"), "@(A1:A3*2)");
assert_eq!(rt("_xlfn.SINGLE(-A1:A3)"), "@(-A1:A3)");
assert_eq!(rt("SQRT(_xlfn.SINGLE(Data[Qty]))"), "SQRT(@Data[Qty])");
}
#[test]
fn delete_sheet_shrinks_3d_span() -> Result<(), Box<dyn std::error::Error>> {
let order: Vec<String> = ["Sheet1", "Sheet2", "Sheet3", "Results"].iter().map(|s| s.to_string()).collect();
+494
View File
@@ -0,0 +1,494 @@
//! `adjust`, `adjust_if_changed` and `shift_relative` against a verbatim copy of the previous
//! implementation, which cloned and rebuilt every formula. The rewrite must give the same tree
//! (and so the same text) for every formula and edit, and `adjust_if_changed` must say
//! "unchanged" exactly when the old code returned the formula as it was.
use gridcraft_core::{CellRef, MAX_COLS, MAX_ROWS, RangeRef};
use gridcraft_formula::adjust::{Axis, Edit, adjust, adjust_if_changed, shift_relative};
use gridcraft_formula::{Expr, parse, print};
/// The implementation before the borrowed check, kept as the test oracle.
mod old {
use gridcraft_core::{CellError, MAX_COLS, MAX_ROWS};
use gridcraft_formula::adjust::{Axis, Edit};
use gridcraft_formula::{Anchor, Expr, RefKind, Reference, SheetSel};
fn clamp_anchor(row: i64, col: i64) -> Option<(u32, u32)> {
if (0..MAX_ROWS as i64).contains(&row) && (0..MAX_COLS as i64).contains(&col) { Some((row as u32, col as u32)) } else { None }
}
pub fn shift_relative(e: Expr, dr: i64, dc: i64) -> Expr {
e.map(&mut |x| match x {
Expr::Ref(r) => match shift_ref(&r, dr, dc) {
Some(r) => Expr::Ref(r),
None => Expr::Error(CellError::Ref),
},
other => other,
})
}
fn shift_anchor(a: &Anchor, dr: i64, dc: i64) -> Option<Anchor> {
let row = if a.row_abs { a.row as i64 } else { a.row as i64 + dr };
let col = if a.col_abs { a.col as i64 } else { a.col as i64 + dc };
let (row, col) = clamp_anchor(row, col)?;
Some(Anchor { row, col, ..*a })
}
fn shift_line(v: u32, abs: bool, d: i64, max: u32) -> Option<u32> {
if abs {
return Some(v);
}
let n = v as i64 + d;
if (0..max as i64).contains(&n) { Some(n as u32) } else { None }
}
fn shift_ref(r: &Reference, dr: i64, dc: i64) -> Option<Reference> {
let kind = match &r.kind {
RefKind::Cell(a) => RefKind::Cell(shift_anchor(a, dr, dc)?),
RefKind::Range(a, b) => RefKind::Range(shift_anchor(a, dr, dc)?, shift_anchor(b, dr, dc)?),
RefKind::Rows(r0, a0, r1, a1) => RefKind::Rows(shift_line(*r0, *a0, dr, MAX_ROWS)?, *a0, shift_line(*r1, *a1, dr, MAX_ROWS)?, *a1),
RefKind::Cols(c0, a0, c1, a1) => RefKind::Cols(shift_line(*c0, *a0, dc, MAX_COLS)?, *a0, shift_line(*c1, *a1, dc, MAX_COLS)?, *a1),
};
Some(Reference { sheet: r.sheet.clone(), kind })
}
pub fn adjust(e: Expr, host: &str, target: &str, edit: &Edit) -> Expr {
e.map(&mut |x| match x {
Expr::Ref(r) => {
let on_target = match &r.sheet {
SheetSel::Current => host.eq_ignore_ascii_case(target),
SheetSel::Named(n) => n.eq_ignore_ascii_case(target),
SheetSel::Span(..) => false,
};
if !on_target {
return Expr::Ref(r);
}
match adjust_ref(&r, edit) {
Some(r) => Expr::Ref(r),
None => Expr::Error(CellError::Ref),
}
}
other => other,
})
}
fn map_index(v: u32, at: u32, count: u32, insert: bool, max: u32) -> Option<u32> {
if insert {
if v >= at {
let n = v as u64 + count as u64;
if n >= max as u64 { None } else { Some(n as u32) }
} else {
Some(v)
}
} else if v < at {
Some(v)
} else if v >= at.saturating_add(count) {
Some(v - count)
} else {
None
}
}
fn map_span(lo: u32, hi: u32, at: u32, count: u32, insert: bool, max: u32) -> Option<(u32, u32)> {
if insert {
let lo2 = if lo >= at { lo as u64 + count as u64 } else { lo as u64 };
let hi2 = if hi >= at { hi as u64 + count as u64 } else { hi as u64 };
let hi2 = hi2.min(max as u64 - 1);
if lo2 > hi2 {
return None;
}
Some((lo2 as u32, hi2 as u32))
} else {
let end = at.saturating_add(count);
if lo >= at && hi < end {
return None;
}
let lo2 = if lo < at {
lo
} else if lo >= end {
lo - count
} else {
at
};
let hi2 = if hi < at {
hi
} else if hi >= end {
hi - count
} else {
at.saturating_sub(1)
};
Some((lo2, hi2.max(lo2)))
}
}
fn adjust_ref(r: &Reference, edit: &Edit) -> Option<Reference> {
match edit {
Edit::Insert { axis, at, count } | Edit::Delete { axis, at, count } => {
let insert = matches!(edit, Edit::Insert { .. });
let kind = match (&r.kind, axis) {
(RefKind::Cell(a), Axis::Rows) => RefKind::Cell(Anchor { row: map_index(a.row, *at, *count, insert, MAX_ROWS)?, ..*a }),
(RefKind::Cell(a), Axis::Cols) => RefKind::Cell(Anchor { col: map_index(a.col, *at, *count, insert, MAX_COLS)?, ..*a }),
(RefKind::Range(a, b), Axis::Rows) => {
let (lo, hi) = map_span(a.row.min(b.row), a.row.max(b.row), *at, *count, insert, MAX_ROWS)?;
RefKind::Range(Anchor { row: lo, ..*a }, Anchor { row: hi, ..*b })
}
(RefKind::Range(a, b), Axis::Cols) => {
let (lo, hi) = map_span(a.col.min(b.col), a.col.max(b.col), *at, *count, insert, MAX_COLS)?;
RefKind::Range(Anchor { col: lo, ..*a }, Anchor { col: hi, ..*b })
}
(RefKind::Rows(r0, a0, r1, a1), Axis::Rows) => {
let (lo, hi) = map_span(*r0.min(r1), *r0.max(r1), *at, *count, insert, MAX_ROWS)?;
RefKind::Rows(lo, *a0, hi, *a1)
}
(RefKind::Cols(c0, a0, c1, a1), Axis::Cols) => {
let (lo, hi) = map_span(*c0.min(c1), *c0.max(c1), *at, *count, insert, MAX_COLS)?;
RefKind::Cols(lo, *a0, hi, *a1)
}
(k, _) => k.clone(),
};
Some(Reference { sheet: r.sheet.clone(), kind })
}
Edit::InsertCells { axis, range } | Edit::DeleteCells { axis, range } => {
let span = r.range();
let (inside, at, count) = match axis {
Axis::Rows => (range.start.col <= span.start.col && span.end.col <= range.end.col, range.start.row, range.height()),
Axis::Cols => (range.start.row <= span.start.row && span.end.row <= range.end.row, range.start.col, range.width()),
};
if !inside {
return Some(r.clone());
}
let axis = *axis;
let lines =
if matches!(edit, Edit::InsertCells { .. }) { Edit::Insert { axis, at, count } } else { Edit::Delete { axis, at, count } };
adjust_ref(r, &lines)
}
Edit::Move { from, to_row, to_col } => {
let dr = *to_row as i64 - from.start.row as i64;
let dc = *to_col as i64 - from.start.col as i64;
let range = r.range();
if !from.contains_range(&range) {
return Some(r.clone());
}
let mv = |a: &Anchor| -> Option<Anchor> {
let (row, col) = clamp_anchor(a.row as i64 + dr, a.col as i64 + dc)?;
Some(Anchor { row, col, ..*a })
};
let kind = match &r.kind {
RefKind::Cell(a) => RefKind::Cell(mv(a)?),
RefKind::Range(a, b) => RefKind::Range(mv(a)?, mv(b)?),
RefKind::Rows(a, aa, b, ba) => {
let dr = *to_row as i64 - from.start.row as i64;
RefKind::Rows(shift_line(*a, false, dr, MAX_ROWS)?, *aa, shift_line(*b, false, dr, MAX_ROWS)?, *ba)
}
RefKind::Cols(a, aa, b, ba) => {
let dc = *to_col as i64 - from.start.col as i64;
RefKind::Cols(shift_line(*a, false, dc, MAX_COLS)?, *aa, shift_line(*b, false, dc, MAX_COLS)?, *ba)
}
};
Some(Reference { sheet: r.sheet.clone(), kind })
}
Edit::MoveToSheet { from, to_sheet, to_row, to_col } => {
if !from.contains_range(&r.range()) {
return Some(r.clone());
}
let mut moved = adjust_ref(r, &Edit::Move { from: *from, to_row: *to_row, to_col: *to_col })?;
moved.sheet = SheetSel::Named(to_sheet.clone());
Some(moved)
}
}
}
}
/// Formulas with every reference shape the adjustment distinguishes.
const FORMULAS: &[&str] = &[
"A1",
"B2+C3*D4",
"$B$2+B$2+$B2",
"SUM(A1:A10)+SUM(B2:C3)+SUM($B$2:$C$3)",
"SUM(C3:B2)",
"SUM(A:A)+SUM($B:$C)+SUM(2:2)+SUM($3:$5)",
"SUM(A1:.A7)+SUM(B:.B)+ROWS(2.:.6)+SUM(A1.:B1)",
"S!B2+s!C3+'S'!A1:B9",
"Other!B2+Other!A:A+Other!2:3",
"'My Sheet'!B2+'my sheet'!C3:D4",
"Dest!B2+S!B2",
"SUM(S:Other!B2)+SUM(Other:S!A1:C3)",
"XFD1048576+XFD1+A1048576+SUM(XFC1048575:XFD1048576)",
"SUM(1048576:1048576)+SUM(XFD:XFD)",
"VLOOKUP($B2,Lookup!$A$1:$D$500,3,FALSE)",
"IF(AND(C2>0,D2<100),C2*1.07,IF(E2=\"x\",\"yes\",\"no\"))",
"Table1[@Qty]*Table1[@Price]+SUM(Table1[[#Data],[Qty]])",
"SUMPRODUCT(A2:C2,{1,2,3})",
"LET(x,A2*2,y,B2+x,x*y)",
"LAMBDA(x,x+B2)(C3)",
"IFERROR(INDEX(Lookup!$A:$A,MATCH(F2,Lookup!$B:$B,0)),\"\")",
"-B2%+(C3)^2&\"a\"",
"A1#+S!B2#",
"S!MyName+MyName+#REF!+S!#REF!",
"COUNTIFS(Data!$A:$A,A2,Data!$C:$C,\">=\"&$G$2)",
"1+2",
"\"text\"",
"B2:C3 B3:D4",
"(B2:C3,E5)",
"INDIRECT(\"B2\")+OFFSET(B2,1,1)",
// From the parser edge cases (prefixes and LET parameter names).
"LET(_xlpm.TRUE,1,_xlpm.TRUE+1)",
"LET(_xlpm.A1,1,_xlpm.A1)",
"LAMBDA(_xlpm.x,_xlpm.x+1)(2)",
"_xlfn.LET(_xlpm.R1C1,1,_xlpm.R1C1*2)",
"LET(_xlpm.FALSE,2,_xlpm.FALSE)",
"_xlpm.SUM(1)",
"LET(_xlpm.x_1,1,_xlpm.x_1)",
"_xlfn.ANCHORARRAY(A1)+_xlfn.SINGLE(B1:B3)",
"SUM(_xlfn._TRO_ALL(A1:E10))",
];
fn col(c: u32) -> String {
gridcraft_core::addr::col_to_letters(c)
}
/// The realistic mix the structural-edit benchmark uses, row-dependent.
fn mix(i: usize, r: u32) -> String {
let r1 = r + 1;
match i % 12 {
0 => format!("SUM(A{r1}:A{})", r1 + 9),
1 => format!("VLOOKUP($B{r1},Lookup!$A$1:$D$500,3,FALSE)"),
2 => format!("IF(AND(C{r1}>0,D{r1}<100),C{r1}*1.07,IF(E{r1}=\"x\",\"yes\",\"no\"))"),
3 => format!("'My Sheet'!B{r1}+Data!C{r1}*2-1"),
4 => "Table1[@Qty]*Table1[@Price]".to_string(),
5 => format!("SUMPRODUCT(A{r1}:C{r1},{{1,2,3}})+SUM(Table1[[#Data],[Qty]])"),
6 => format!("\"Lorem ipsum\"&A{r1}&\" units\""),
7 => format!("IFERROR(INDEX(Lookup!$A:$A,MATCH(F{r1},Lookup!$B:$B,0)),\"\")"),
8 => (0..20).map(|k| format!("{}{r1}", col(k))).collect::<Vec<_>>().join("+"),
9 => format!("LET(x,A{r1}*2,y,B{r1}+x,x*y)"),
10 => format!("ROUND(AVERAGE(B{r1}:F{r1})*(1+$H$1)^2,2)"),
_ => format!("COUNTIFS(Data!$A:$A,A{r1},Data!$C:$C,\">=\"&$G$2)"),
}
}
fn range(r0: u32, c0: u32, r1: u32, c1: u32) -> RangeRef {
RangeRef::new(CellRef::new(r0, c0), CellRef::new(r1, c1))
}
/// Every edit kind, at the sheet's start, middle and edges, with counts that run past the end.
fn edits() -> Vec<Edit> {
let mut v = Vec::new();
for axis in [Axis::Rows, Axis::Cols] {
let max = if axis == Axis::Rows { MAX_ROWS } else { MAX_COLS };
for at in [0, 1, 2, 3, 9, max - 2, max - 1] {
for count in [0, 1, 2, 5, max - 1, max, u32::MAX] {
v.push(Edit::Insert { axis, at, count });
v.push(Edit::Delete { axis, at, count });
}
}
for r in [
range(1, 1, 2, 2),
range(0, 0, 0, 0),
range(1, 0, 1, MAX_COLS - 1),
range(0, 1, MAX_ROWS - 1, 1),
range(MAX_ROWS - 2, MAX_COLS - 2, MAX_ROWS - 1, MAX_COLS - 1),
] {
v.push(Edit::InsertCells { axis, range: r });
v.push(Edit::DeleteCells { axis, range: r });
}
}
let blocks = [
range(1, 1, 2, 2),
range(0, 0, 0, 0),
range(0, 0, MAX_ROWS - 1, MAX_COLS - 1),
range(0, 0, MAX_ROWS - 1, 2),
range(1, 0, 4, MAX_COLS - 1),
range(0, 0, 20, 10),
];
let targets = [(9, 3), (0, 0), (1, 1), (MAX_ROWS - 1, MAX_COLS - 1), (MAX_ROWS - 2, 0)];
for from in blocks {
for (to_row, to_col) in targets {
v.push(Edit::Move { from, to_row, to_col });
for to_sheet in ["Dest", "S", "s", "Other"] {
v.push(Edit::MoveToSheet { from, to_sheet: to_sheet.to_string(), to_row, to_col });
}
}
}
v
}
/// Compares the new and the old adjustment of `src` for every host and edit; returns the
/// number of (formula, host, edit) cases checked, or 0 when `src` does not parse.
fn check(src: &str, edits: &[Edit]) -> usize {
let Ok(e) = parse(src) else { return 0 };
let mut n = 0;
for host in ["S", "s", "Other", "My Sheet"] {
for edit in edits {
let want = old::adjust(e.clone(), host, "S", edit);
let got = adjust(e.clone(), host, "S", edit);
assert!(got == want, "{src} on {host} after {edit:?}: {} vs {}", print(&got), print(&want));
assert_eq!(print(&got), print(&want), "{src} on {host} after {edit:?}");
match adjust_if_changed(&e, host, "S", edit) {
None => assert!(want == e, "{src} on {host} after {edit:?}: reported unchanged, old gives {}", print(&want)),
Some(x) => {
assert!(want != e, "{src} on {host} after {edit:?}: reported changed but the old code keeps it");
assert!(x == want, "{src} on {host} after {edit:?}: {} vs {}", print(&x), print(&want));
}
}
n += 1;
}
}
for (dr, dc) in [(0, 0), (1, 0), (0, 1), (-1, -1), (5, 3), (-3, 2), (MAX_ROWS as i64, 0), (0, -(MAX_COLS as i64)), (1_048_575, 16_383)] {
let want = old::shift_relative(e.clone(), dr, dc);
let got = shift_relative(e.clone(), dr, dc);
assert!(got == want, "{src} shifted by ({dr}, {dc}): {} vs {}", print(&got), print(&want));
}
n
}
#[test]
fn adjust_matches_the_previous_implementation_for_every_edit_kind() {
let edits = edits();
let mut cases = 0;
for src in FORMULAS {
let n = check(src, &edits);
assert!(n > 0, "{src} does not parse");
cases += n;
}
for i in 0..48 {
// Rows 1-2 and 1,048,565-1,048,566 of the mix (its longest range reaches 9 rows further
// down, to the last row): near both edges of the sheet.
let row = if i < 24 { (i / 12) as u32 } else { MAX_ROWS - 12 + (i / 12 - 2) as u32 };
let src = mix(i, row);
let n = check(&src, &edits);
assert!(n > 0, "{src} does not parse");
cases += n;
}
assert!(cases > 100_000, "{cases}");
}
/// A small deterministic generator: a failing case is reproduced by rerunning the test, and
/// its message names the case's index in the stream.
struct Rng(u64);
impl Rng {
fn next(&mut self) -> u64 {
self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
self.0 >> 33
}
fn below(&mut self, n: u64) -> u64 {
self.next() % n
}
fn line(&mut self, max: u32) -> u32 {
match self.below(3) {
0 => self.below(12) as u32,
1 => max - 1 - self.below(12) as u32,
_ => self.below(max as u64) as u32,
}
}
}
fn random_ref(rng: &mut Rng) -> String {
let sheet = ["", "", "S!", "s!", "Other!", "'My Sheet'!", "S:Other!"][rng.below(7) as usize];
let dollar = |rng: &mut Rng| if rng.below(3) == 0 { "$" } else { "" };
let cell = |rng: &mut Rng| format!("{}{}{}{}", dollar(rng), col(rng.line(MAX_COLS)), dollar(rng), rng.line(MAX_ROWS) + 1);
let body = match rng.below(4) {
0 => cell(rng),
1 => format!("{}:{}", cell(rng), cell(rng)),
2 => {
let (a, b) = (rng.line(MAX_ROWS) + 1, rng.line(MAX_ROWS) + 1);
format!("{}{a}:{}{b}", dollar(rng), dollar(rng))
}
_ => {
let (a, b) = (rng.line(MAX_COLS), rng.line(MAX_COLS));
format!("{}{}:{}{}", dollar(rng), col(a), dollar(rng), col(b))
}
};
format!("{sheet}{body}")
}
fn random_edit(rng: &mut Rng) -> Edit {
let axis = if rng.below(2) == 0 { Axis::Rows } else { Axis::Cols };
let max = if axis == Axis::Rows { MAX_ROWS } else { MAX_COLS };
let block = |rng: &mut Rng| {
let (r0, r1) = (rng.line(MAX_ROWS), rng.line(MAX_ROWS));
let (c0, c1) = (rng.line(MAX_COLS), rng.line(MAX_COLS));
range(r0.min(r1), c0.min(c1), r0.max(r1), c0.max(c1))
};
let count = if rng.below(8) == 0 { rng.below(u32::MAX as u64) as u32 } else { rng.below(20) as u32 };
match rng.below(7) {
0 => Edit::Insert { axis, at: rng.line(max), count },
1 => Edit::Delete { axis, at: rng.line(max), count },
2 => Edit::InsertCells { axis, range: block(rng) },
3 => Edit::DeleteCells { axis, range: block(rng) },
4 => Edit::Move { from: block(rng), to_row: rng.line(MAX_ROWS), to_col: rng.line(MAX_COLS) },
_ => Edit::MoveToSheet {
from: block(rng),
to_sheet: ["Dest", "S", "other"][rng.below(3) as usize].to_string(),
to_row: rng.line(MAX_ROWS),
to_col: rng.line(MAX_COLS),
},
}
}
/// Random references near the sheet's edges and random edits: "unchanged" is reported exactly
/// when the previous implementation leaves the formula as it was, and a reported change is the
/// old result.
#[test]
fn adjust_if_changed_reports_unchanged_exactly_when_nothing_moves() {
let mut rng = Rng(0x5eed);
let (mut unchanged, mut changed) = (0, 0);
for case in 0..20_000 {
let refs: Vec<String> = (0..1 + rng.below(4)).map(|_| random_ref(&mut rng)).collect();
let src = format!("SUM({})", refs.join(","));
let Ok(e) = parse(&src) else { panic!("case {case}: {src} does not parse") };
let edit = random_edit(&mut rng);
let host = ["S", "Other"][rng.below(2) as usize];
let want = old::adjust(e.clone(), host, "S", &edit);
assert!(adjust(e.clone(), host, "S", &edit) == want, "case {case}: {src} on {host} after {edit:?}");
match adjust_if_changed(&e, host, "S", &edit) {
None => {
assert!(want == e, "case {case}: {src} on {host} after {edit:?}: reported unchanged, old gives {}", print(&want));
unchanged += 1;
}
Some(x) => {
assert!(want != e && x == want, "case {case}: {src} on {host} after {edit:?}");
changed += 1;
}
}
}
// Both outcomes are exercised in quantity.
assert!(unchanged > 2_000 && changed > 2_000, "unchanged {unchanged}, changed {changed}");
}
/// The whole formula corpus (one formula per line, with or without `=`) when
/// `GRIDCRAFT_ADJUST_CORPUS` names a file: `cargo test -p gridcraft-formula --test it -- --ignored adjust_corpus`.
#[test]
#[ignore = "needs GRIDCRAFT_ADJUST_CORPUS"]
fn adjust_corpus_matches_the_previous_implementation() {
let Ok(path) = std::env::var("GRIDCRAFT_ADJUST_CORPUS") else { return };
let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}"));
let edits = edits();
let (mut parsed, mut skipped, mut cases) = (0, 0, 0);
for line in text.lines().filter(|l| !l.trim().is_empty()) {
match check(line, &edits) {
0 => skipped += 1,
n => {
parsed += 1;
cases += n;
}
}
}
eprintln!("corpus: {parsed} formulas parsed, {skipped} skipped (do not parse), {cases} cases");
}
#[test]
fn unaffected_formula_is_returned_as_is() {
let e: Expr = parse("Other!B2+SUM(A1:A3)").unwrap();
let ins = Edit::Insert { axis: Axis::Rows, at: 5, count: 1 };
// Row 6 inserted: A1:A3 lies above it and Other!B2 is not on S, so nothing moves, whichever
// sheet the formula lives on (on Other, A1:A3 is on Other too).
assert!(adjust_if_changed(&e, "S", "S", &ins).is_none());
assert!(adjust_if_changed(&e, "Other", "S", &ins).is_none());
// Row 1 inserted: A1:A3 on S moves, Other!B2 does not.
let top = Edit::Insert { axis: Axis::Rows, at: 0, count: 1 };
assert_eq!(adjust_if_changed(&e, "S", "S", &top).map(|x| print(&x)).as_deref(), Some("Other!B2+SUM(A2:A4)"));
// A cut that moves a block onto the same place still qualifies its references with the sheet.
let same_place = Edit::MoveToSheet { from: range(0, 0, 2, 0), to_sheet: "S".into(), to_row: 0, to_col: 0 };
assert_eq!(adjust_if_changed(&e, "S", "S", &same_place).map(|x| print(&x)).as_deref(), Some("Other!B2+SUM(S!A1:A3)"));
}
+1
View File
@@ -1,4 +1,5 @@
//! The crate's integration tests, one module per file, built as one test binary.
mod adjust_oracle;
mod printer_roundtrip;
mod spanish_input;
+6
View File
@@ -308,6 +308,12 @@ mod tests {
assert_eq!(ev("REGEXREPLACE", vec![t("2026-10-22"), t("(\\d+)-(\\d+)-(\\d+)"), t("$3/$2/$1")]), Value::text("22/10/2026"));
assert_eq!(ev("REGEXREPLACE", vec![t("abc"), t("x*"), t("-")]), Value::text("-a-b-c-"));
assert_eq!(ev("REGEXREPLACE", vec![t("a1b2"), t("[0-9]"), t("#"), n(2.0)]), Value::text("a1b#"));
assert_eq!(ev("REGEXREPLACE", vec![t("a1b2"), t("[0-9]"), t("#"), n(-2.0)]), Value::text("a#b2"));
assert_eq!(ev("REGEXREPLACE", vec![t("a1b2"), t("[0-9]"), t("#"), n(-3.0)]), Value::text("a1b2"));
// 2^32 is no occurrence the text has (and on 32-bit targets it must not wrap to 0).
for occurrence in [4_294_967_296.0, -4_294_967_296.0] {
assert_eq!(ev("REGEXREPLACE", vec![t("a1b2"), t("[0-9]"), t("#"), n(occurrence)]), Value::text("a1b2"));
}
// One match whose replacement repeats it 10,000 times: refused before it is built.
assert_eq!(ev("REGEXREPLACE", vec![t(&many), t("^.*$"), t(&"$0".repeat(10_000))]), Value::Error(CellError::Value));
assert_eq!(ev("REGEXREPLACE", vec![t(&many), t("^.*$"), t(&"$0".repeat(10_000)), n(1.0)]), Value::Error(CellError::Value));
+28 -8
View File
@@ -88,11 +88,17 @@ pub(crate) fn reg_gamma_p(a: f64, x: f64) -> f64 {
if x.is_infinite() {
return 1.0;
}
if x < a + 1.0 { gamma_series(a, x) } else { 1.0 - gamma_cf(a, x) }
if x < a + 1.0 { gamma_series(a, x, front_of(a, x)) } else { 1.0 - gamma_cf(a, x, front_of(a, x)) }
}
/// Regularised upper incomplete gamma Q(a, x) = 1 − P(a, x).
pub(crate) fn reg_gamma_q(a: f64, x: f64) -> f64 {
let front = if a > 0.0 && x > 0.0 && x.is_finite() { (-x + a * x.ln() - ln_gamma(a)).exp() } else { 0.0 };
reg_gamma_q_front(a, x, front)
}
/// Q(a, x) given the prefactor `e^-x x^a / Γ(a)` (the Poisson mass at a - 1, times x).
pub(crate) fn reg_gamma_q_front(a: f64, x: f64, front: f64) -> f64 {
if a.is_nan() || a <= 0.0 || x.is_nan() || x < 0.0 || !a.is_finite() {
return f64::NAN;
}
@@ -102,10 +108,15 @@ pub(crate) fn reg_gamma_q(a: f64, x: f64) -> f64 {
if x.is_infinite() {
return 0.0;
}
if x < a + 1.0 { 1.0 - gamma_series(a, x) } else { gamma_cf(a, x) }
if x < a + 1.0 { 1.0 - gamma_series(a, x, front) } else { gamma_cf(a, x, front) }
}
fn gamma_series(a: f64, x: f64) -> f64 {
/// `e^-x x^a / Γ(a)`.
fn front_of(a: f64, x: f64) -> f64 {
(-x + a * x.ln() - ln_gamma(a)).exp()
}
fn gamma_series(a: f64, x: f64, front: f64) -> f64 {
let mut ap = a;
let mut sum = 1.0 / a;
let mut del = sum;
@@ -117,10 +128,10 @@ fn gamma_series(a: f64, x: f64) -> f64 {
break;
}
}
sum * (-x + a * x.ln() - ln_gamma(a)).exp()
sum * front
}
fn gamma_cf(a: f64, x: f64) -> f64 {
fn gamma_cf(a: f64, x: f64, front: f64) -> f64 {
let tiny = 1e-300;
let mut b = x + 1.0 - a;
let mut c = 1.0 / tiny;
@@ -144,7 +155,7 @@ fn gamma_cf(a: f64, x: f64) -> f64 {
break;
}
}
(-x + a * x.ln() - ln_gamma(a)).exp() * h
front * h
}
/// Error function.
@@ -187,6 +198,17 @@ pub(crate) fn erfc(x: f64) -> f64 {
/// Regularised incomplete beta I_x(a, b).
pub(crate) fn reg_inc_beta(x: f64, a: f64, b: f64) -> f64 {
if x > 0.0 && x < 1.0 && a > 0.0 && b > 0.0 {
let ln_bt = -ln_beta(a, b) + a * x.ln() + b * (1.0 - x).ln();
return reg_inc_beta_front(x, a, b, ln_bt.exp());
}
reg_inc_beta_front(x, a, b, 0.0)
}
/// I_x(a, b) given the prefactor `bt = x^a (1-x)^b / B(a, b)` (symmetric in the two sides). A
/// caller that knows `bt` more accurately than the log-gamma route gives (the binomial mass, for
/// instance) passes it here.
pub(crate) fn reg_inc_beta_front(x: f64, a: f64, b: f64, bt: f64) -> f64 {
if a.is_nan() || a <= 0.0 || b.is_nan() || b <= 0.0 || !(0.0..=1.0).contains(&x) {
return f64::NAN;
}
@@ -196,8 +218,6 @@ pub(crate) fn reg_inc_beta(x: f64, a: f64, b: f64) -> f64 {
if x == 1.0 {
return 1.0;
}
let ln_bt = -ln_beta(a, b) + a * x.ln() + b * (1.0 - x).ln();
let bt = ln_bt.exp();
if x < (a + 1.0) / (a + b + 2.0) { bt * beta_cf(a, b, x) / a } else { 1.0 - bt * beta_cf(b, a, 1.0 - x) / b }
}
+4 -1
View File
@@ -3,7 +3,10 @@
use gridcraft_core::{CellError, Value};
use crate::special::{erf, gamma, invert, ln_beta, ln_gamma, norm_cdf, norm_inv, norm_pdf, reg_gamma_p, reg_gamma_q, reg_inc_beta};
use crate::special::{
erf, gamma, invert, ln_beta, ln_gamma, norm_cdf, norm_inv, norm_pdf, reg_gamma_p, reg_gamma_q, reg_gamma_q_front, reg_inc_beta,
reg_inc_beta_front,
};
use crate::util::{A, R, S, arg, array_numbers, array_val, as_array, boolean, has, num, num_val, numbers, numbers_a, opt_num};
use crate::{Arg, Ctx, FnSpec, VAR};
+58 -18
View File
@@ -33,19 +33,40 @@ pub(super) fn var_p(x: &[f64]) -> R<f64> {
Ok(devsq(x) / x.len() as f64)
}
fn sorted(mut v: Vec<f64>) -> Vec<f64> {
pub(super) fn sorted(mut v: Vec<f64>) -> Vec<f64> {
v.sort_by(f64::total_cmp);
v
}
/// Sorts `v` if it is already monotone in `total_cmp` order (a descending slice is reversed) and
/// says whether it did. At most two passes, each stopping at the first out-of-order pair, so
/// unordered data costs next to nothing. Needed because the std sort recognises ascending and
/// descending runs in O(n) with a small constant; selection is O(n) too but measured 1.2-1.8x
/// slower on such input.
fn sorted_if_monotone(v: &mut [f64]) -> bool {
if v.is_sorted_by(|a, b| a.total_cmp(b).is_le()) {
return true;
}
if v.is_sorted_by(|a, b| a.total_cmp(b).is_ge()) {
v.reverse();
return true;
}
false
}
pub(super) fn median(v: Vec<f64>) -> R<f64> {
if v.is_empty() {
return Err(CellError::Num);
}
let s = sorted(v);
let n = s.len();
let hi = s.get(n / 2).copied().unwrap_or(0.0);
if n % 2 == 1 { Ok(hi) } else { Ok(s.get(n / 2 - 1).copied().unwrap_or(0.0).midpoint(hi)) }
let mut v = v;
let n = v.len();
if sorted_if_monotone(&mut v) {
let hi = v.get(n / 2).copied().unwrap_or(0.0);
return if n % 2 == 1 { Ok(hi) } else { Ok(v.get(n / 2 - 1).copied().unwrap_or(0.0).midpoint(hi)) };
}
let (lower, hi, _) = v.select_nth_unstable_by(n / 2, f64::total_cmp);
let hi = *hi;
if n % 2 == 1 { Ok(hi) } else { Ok(lower.iter().copied().max_by(f64::total_cmp).unwrap_or(0.0).midpoint(hi)) }
}
/// Values occurring at least twice, ordered by first occurrence, with their (equal, maximal) count.
@@ -126,34 +147,54 @@ pub(super) fn max_of(v: &[f64]) -> f64 {
}
/// k-th largest (k is rounded up like Excel).
fn kth(v: Vec<f64>, k: f64, largest: bool) -> R<f64> {
pub(super) fn kth(v: Vec<f64>, k: f64, largest: bool) -> R<f64> {
let k = k.ceil();
if k.is_nan() || k < 1.0 || k > v.len() as f64 {
return Err(CellError::Num);
}
let s = sorted(v);
let mut v = v;
let k = k as usize;
let idx = if largest { s.len() - k } else { k - 1 };
s.get(idx).copied().ok_or(CellError::Num)
let idx = if largest { v.len() - k } else { k - 1 };
// Selection instead of a full sort (monotone input is read directly); same total order, so the
// same bit pattern as `sorted(v)[idx]`.
if sorted_if_monotone(&mut v) {
return v.get(idx).copied().ok_or(CellError::Num);
}
Ok(*v.select_nth_unstable_by(idx, f64::total_cmp).1)
}
fn interp(s: &[f64], pos: f64) -> R<f64> {
/// Value at fractional rank `pos` of the sorted data, linearly interpolated, found by selection.
fn interp(mut v: Vec<f64>, pos: f64) -> R<f64> {
let i = pos.floor();
let frac = pos - i;
let i = i as usize;
let lo = *s.get(i).ok_or(CellError::Num)?;
match s.get(i + 1) {
Some(hi) if frac > 0.0 => Ok(lo + frac * (hi - lo)),
_ => Ok(lo),
if i >= v.len() {
return Err(CellError::Num);
}
if sorted_if_monotone(&mut v) {
let lo = v.get(i).copied().ok_or(CellError::Num)?;
return match v.get(i + 1) {
Some(hi) if frac > 0.0 => Ok(lo + frac * (hi - lo)),
_ => Ok(lo),
};
}
let (_, lo, upper) = v.select_nth_unstable_by(i, f64::total_cmp);
let lo = *lo;
// The next order statistic is the minimum of the upper partition.
if frac > 0.0
&& let Some(hi) = upper.iter().copied().min_by(f64::total_cmp)
{
return Ok(lo + frac * (hi - lo));
}
Ok(lo)
}
pub(super) fn percentile_inc(v: Vec<f64>, k: f64) -> R<f64> {
if v.is_empty() || !(0.0..=1.0).contains(&k) {
return Err(CellError::Num);
}
let s = sorted(v);
interp(&s, k * (s.len() - 1) as f64)
let pos = k * (v.len() - 1) as f64;
interp(v, pos)
}
pub(super) fn percentile_exc(v: Vec<f64>, k: f64) -> R<f64> {
@@ -165,8 +206,7 @@ pub(super) fn percentile_exc(v: Vec<f64>, k: f64) -> R<f64> {
if pos < 1.0 || pos > n {
return Err(CellError::Num);
}
let s = sorted(v);
interp(&s, pos - 1.0)
interp(v, pos - 1.0)
}
pub(super) fn quartile_inc(v: Vec<f64>, q: f64) -> R<f64> {
+183 -17
View File
@@ -1,7 +1,7 @@
use super::descriptive::{arg_nums, arg_pairs, mean, var_s};
use super::{
Arg, C, CellError, R, Value, arg, as_array, boolean, erf, gamma, has, invert, ln_beta, ln_gamma, norm_cdf, norm_inv, norm_pdf, num, num_val,
reg_gamma_p, reg_gamma_q, reg_inc_beta,
reg_gamma_p, reg_gamma_q, reg_gamma_q_front, reg_inc_beta, reg_inc_beta_front,
};
// ---------------------------------------------------------------------------------------------
@@ -93,6 +93,97 @@ fn f_pdf(x: f64, d1: f64, d2: f64) -> f64 {
const MAX_BINOM_TRIALS: f64 = 2_147_483_646.0;
/// Stirling-series error `ln n! - ((n + 1/2) ln n - n + ln(2π)/2)` for integer `n >= 1`: exact
/// table (computed with 60-digit Decimal arithmetic) up to 30, then the series, whose first
/// omitted term is below 1e-19 there.
fn stirlerr(n: f64) -> f64 {
const TABLE: [f64; 31] = [
0.0,
0.08106146679532726,
0.0413406959554093,
0.02767792568499834,
0.020790672103765093,
0.016644691189821193,
0.013876128823070748,
0.01189670994589177,
0.010411265261972096,
0.009255462182712733,
0.00833056343336287,
0.007573675487951841,
0.00694284010720953,
0.006408994188004207,
0.0059513701127588475,
0.005554733551962801,
0.0052076559196096404,
0.004901395948434738,
0.004629153749334028,
0.004385560249232324,
0.004166319691996922,
0.00396795421864086,
0.0037876180684444346,
0.0036229602246830948,
0.003472021382978767,
0.003333155636728093,
0.003204970228055038,
0.0030862786826087773,
0.002976063983550409,
0.0028734493623524663,
0.0027776749297526936,
];
if n <= 30.0 {
return TABLE.get(n as usize).copied().unwrap_or(0.0);
}
let r = 1.0 / n;
let r2 = r * r;
r * (1.0 / 12.0 - r2 * (1.0 / 360.0 - r2 * (1.0 / 1260.0 - r2 * (1.0 / 1680.0 - r2 * (1.0 / 1188.0)))))
}
/// `x ln(x/np) + np - x` without the cancellation of the direct formula when `x` is near `np`
/// (Loader, 2000).
fn bd0(x: f64, np: f64) -> f64 {
if (x - np).abs() < 0.1 * (x + np) {
let v = (x - np) / (x + np);
let mut s = (x - np) * v;
if s.abs() < f64::MIN_POSITIVE {
return s;
}
let mut ej = 2.0 * x * v;
let v2 = v * v;
for j in 1..1000 {
ej *= v2;
let s1 = s + ej / f64::from(2 * j + 1);
if s1 == s {
return s1;
}
s = s1;
}
}
x * (x / np).ln() + np - x
}
/// `(1 - p)^e` with the rounding error of `1 - p` (it is not exact for `p < 1/2`) folded back in,
/// which `e` would otherwise multiply.
fn pow_one_minus(p: f64, e: f64) -> f64 {
let hi = 1.0 - p;
let lo = (1.0 - hi) - p;
let r = hi.powf(e) * (e * (lo / hi).ln_1p()).exp();
// An exponent so large that the two factors under- and overflow: the plain form is enough.
if r.is_nan() { (e * (-p).ln_1p()).exp() } else { r }
}
/// `C(n, k)` for a small `k` as a product of `k` ratios.
fn choose_small(n: f64, k: f64) -> f64 {
let mut c = 1.0;
let mut j = 0.0;
while j < k {
c *= (n - j) / (j + 1.0);
j += 1.0;
}
c
}
/// Binomial mass at integer `k`, `0 <= k <= n`, by Loader's saddle-point formula: no log-gamma
/// of large numbers, so the relative error stays near 1e-15 for any `n`.
fn binom_pmf(k: f64, n: f64, p: f64) -> f64 {
if k < 0.0 || k > n {
return 0.0;
@@ -103,7 +194,67 @@ fn binom_pmf(k: f64, n: f64, p: f64) -> f64 {
if p == 1.0 {
return if k == n { 1.0 } else { 0.0 };
}
(ln_comb(n, k) + k * p.ln() + (n - k) * (1.0 - p).ln()).exp()
if k == 0.0 {
return if n == 0.0 { 1.0 } else { pow_one_minus(p, n) };
}
if k == n {
return p.powf(n);
}
// A few successes or a few failures: the product form has no large cancelling terms.
// (Not for astronomically large n, or when either power underflows: the
// saddle-point form below handles those through logs.)
if n < 1e30 && p.powf(k) > 1e-280 {
if k < 16.0 {
let rest = pow_one_minus(p, n - k);
if rest >= f64::MIN_POSITIVE {
return choose_small(n, k) * p.powi(k as i32) * rest;
}
}
if n - k < 16.0 {
return choose_small(n, n - k) * p.powf(k) * pow_one_minus(p, n - k);
}
}
let q = 1.0 - p;
let lc = stirlerr(n) - stirlerr(k) - stirlerr(n - k) - bd0(k, n * p) - bd0(n - k, n * q);
let lf = std::f64::consts::TAU.ln() + k.ln() + (-k / n).ln_1p();
(lc - 0.5 * lf).exp()
}
/// Sum of the binomial masses 0..=k for `n <= 10_000`: starts at the term nearest the mode
/// (accurate by the formula above) and walks away from it by the ratio of neighbouring masses,
/// one multiply per term, stopping once the terms no longer matter. Above the mode it sums the
/// upper tail instead and subtracts it from 1.
fn binom_cdf_sum(k: f64, n: f64, p: f64) -> f64 {
let q = 1.0 - p;
if p == 0.0 {
return 1.0;
}
if p == 1.0 {
return 0.0;
}
if k <= n * p {
// mass(i - 1) = mass(i) * i / (n - i + 1) * q / p
let mut t = binom_pmf(k, n, p);
let mut s = t;
let mut i = k;
while i >= 1.0 && t > s * 1e-18 {
t *= i / (n - i + 1.0) * (q / p);
s += t;
i -= 1.0;
}
s.min(1.0)
} else {
// mass(i + 1) = mass(i) * (n - i) / (i + 1) * p / q
let mut t = binom_pmf(k + 1.0, n, p);
let mut s = t;
let mut i = k + 1.0;
while i < n && t > s * 1e-18 {
t *= (n - i) / (i + 1.0) * (p / q);
s += t;
i += 1.0;
}
(1.0 - s).max(0.0)
}
}
fn binom_cdf(k: f64, n: f64, p: f64) -> f64 {
@@ -114,13 +265,7 @@ fn binom_cdf(k: f64, n: f64, p: f64) -> f64 {
return 1.0;
}
if n <= 10_000.0 {
let mut s = 0.0;
let mut i = 0.0;
while i <= k {
s += binom_pmf(i, n, p);
i += 1.0;
}
return s.min(1.0);
return binom_cdf_sum(k, n, p);
}
if p == 0.0 {
return 1.0;
@@ -128,14 +273,19 @@ fn binom_cdf(k: f64, n: f64, p: f64) -> f64 {
if p == 1.0 {
return 0.0;
}
reg_inc_beta(1.0 - p, n - k, k + 1.0)
// I_(1-p)(n-k, k+1) has prefactor (n-k) p mass(k).
reg_inc_beta_front(1.0 - p, n - k, k + 1.0, (n - k) * p * binom_pmf(k, n, p))
}
/// Poisson mass at integer `x >= 0` by Loader's formula.
fn poisson_pmf(x: f64, m: f64) -> f64 {
if m == 0.0 {
return if x == 0.0 { 1.0 } else { 0.0 };
}
(-m + x * m.ln() - ln_gamma(x + 1.0)).exp()
if x == 0.0 {
return (-m).exp();
}
(-stirlerr(x) - bd0(x, m)).exp() / (std::f64::consts::TAU * x).sqrt()
}
fn poisson_cdf(x: f64, m: f64) -> f64 {
@@ -143,15 +293,31 @@ fn poisson_cdf(x: f64, m: f64) -> f64 {
return 1.0;
}
if x <= 1000.0 {
let mut s = 0.0;
let mut i = 0.0;
while i <= x {
s += poisson_pmf(i, m);
if x <= m {
// mass(i - 1) = mass(i) * i / m, downwards from x while the terms matter
let mut t = poisson_pmf(x, m);
let mut s = t;
let mut i = x;
while i >= 1.0 && t > s * 1e-18 {
t *= i / m;
s += t;
i -= 1.0;
}
return s.min(1.0);
}
// Above the mean: 1 minus the upper tail, mass(i + 1) = mass(i) * m / (i + 1).
let mut t = poisson_pmf(x + 1.0, m);
let mut s = t;
let mut i = x + 1.0;
while t > s * 1e-18 {
t *= m / (i + 1.0);
s += t;
i += 1.0;
}
return s.min(1.0);
return (1.0 - s).max(0.0);
}
reg_gamma_q(x + 1.0, m)
// Q(x+1, m) has prefactor m mass(x).
reg_gamma_q_front(x + 1.0, m, m * poisson_pmf(x, m))
}
pub(super) fn norm_dist(a: &[Arg], _: C) -> R<Value> {
+364
View File
@@ -1,4 +1,6 @@
use super::descriptive::{kth, median, percentile_exc, percentile_inc, sorted};
use super::{MAX_REGRESSORS, aggregate_values, aggregate_values_k, check_regression_size};
use crate::util::R;
use crate::util::testutil::*;
use gridcraft_core::{CellError, Value};
@@ -200,6 +202,126 @@ fn min_max_order() {
close(ev("RANK.AVG", vec![n(3.5), r2, n(1.0)]), 3.5);
}
// Reference implementations: the previous full-sort versions, as the oracle for the selection ones.
fn old_interp(s: &[f64], pos: f64) -> R<f64> {
let i = pos.floor();
let frac = pos - i;
let i = i as usize;
let lo = *s.get(i).ok_or(CellError::Num)?;
match s.get(i + 1) {
Some(hi) if frac > 0.0 => Ok(lo + frac * (hi - lo)),
_ => Ok(lo),
}
}
fn old_median(v: Vec<f64>) -> R<f64> {
if v.is_empty() {
return Err(CellError::Num);
}
let s = sorted(v);
let n = s.len();
let hi = s.get(n / 2).copied().unwrap_or(0.0);
if n % 2 == 1 { Ok(hi) } else { Ok(s.get(n / 2 - 1).copied().unwrap_or(0.0).midpoint(hi)) }
}
fn old_kth(v: Vec<f64>, k: f64, largest: bool) -> R<f64> {
let k = k.ceil();
if k.is_nan() || k < 1.0 || k > v.len() as f64 {
return Err(CellError::Num);
}
let s = sorted(v);
let k = k as usize;
let idx = if largest { s.len() - k } else { k - 1 };
s.get(idx).copied().ok_or(CellError::Num)
}
fn old_percentile_inc(v: Vec<f64>, k: f64) -> R<f64> {
if v.is_empty() || !(0.0..=1.0).contains(&k) {
return Err(CellError::Num);
}
let s = sorted(v);
old_interp(&s, k * (s.len() - 1) as f64)
}
fn old_percentile_exc(v: Vec<f64>, k: f64) -> R<f64> {
let n = v.len() as f64;
if v.is_empty() || !(k > 0.0 && k < 1.0) {
return Err(CellError::Num);
}
let pos = k * (n + 1.0);
if pos < 1.0 || pos > n {
return Err(CellError::Num);
}
let s = sorted(v);
old_interp(&s, pos - 1.0)
}
/// Same Ok bit pattern or same error.
fn same(a: R<f64>, b: R<f64>) -> bool {
match (a, b) {
(Ok(x), Ok(y)) => x.to_bits() == y.to_bits(),
(Err(x), Err(y)) => x == y,
_ => false,
}
}
#[test]
fn selection_matches_full_sort_bit_for_bit() {
let mut state = 0x9E37_79B9_7F4A_7C15_u64;
let mut next = move || {
state ^= state << 13;
state ^= state >> 7;
state ^= state << 17;
state
};
let specials = [0.0, -0.0, f64::INFINITY, f64::NEG_INFINITY, f64::NAN, -f64::NAN, f64::MAX, f64::MIN_POSITIVE, 1.0, -1.0];
for len in (0..=40).chain([63, 64, 65, 100, 257, 1000, 4096]) {
for round in 0..24 {
let mut v: Vec<f64> = (0..len)
.map(|_| match (round % 4, next() % 8) {
// Few distinct values: many ties.
(0, r) => (r % 3) as f64,
// Specials, including both zeros and NaNs.
(1, r) => specials[(next() as usize).wrapping_add(r as usize) % specials.len()],
// Wide random range.
(2, _) => (next() as i64 as f64) / 1e3,
// Integers with some specials mixed in.
(_, 0) => specials[next() as usize % specials.len()],
_ => (next() % 50) as f64,
})
.collect();
// Rounds 12..24 repeat the data as already ascending, already descending and nearly
// ascending (ends swapped): the monotone shortcut and its fallback to selection.
match round / 4 {
3 => v.sort_by(f64::total_cmp),
4 => {
v.sort_by(f64::total_cmp);
v.reverse();
}
5 => {
v.sort_by(f64::total_cmp);
if let Some(last) = v.len().checked_sub(1) {
v.swap(0, last);
}
}
_ => {}
}
assert!(same(median(v.clone()), old_median(v.clone())), "median len={len} round={round}");
let ks = [-1.0, 0.0, 0.2, 1.0, 1.5, 2.0, len as f64 / 2.0, len as f64 - 0.5, len as f64, len as f64 + 1.0, f64::NAN];
for &k in &ks {
for largest in [true, false] {
assert!(same(kth(v.clone(), k, largest), old_kth(v.clone(), k, largest)), "kth len={len} k={k}");
}
}
let ps = [-0.1, 0.0, 0.01, 0.1, 0.25, 0.3, 0.5, 0.75, 0.9, 0.99, 1.0, 1.1, f64::NAN, (next() % 1001) as f64 / 1000.0];
for &p in &ps {
assert!(same(percentile_inc(v.clone(), p), old_percentile_inc(v.clone(), p)), "pinc len={len} p={p}");
assert!(same(percentile_exc(v.clone(), p), old_percentile_exc(v.clone(), p)), "pexc len={len} p={p}");
}
}
}
}
#[test]
fn percentiles() {
close(ev("PERCENTILE.INC", vec![c(&[1.0, 3.0, 2.0, 4.0]), n(0.3)]), 1.9);
@@ -618,6 +740,248 @@ fn aggregate_helper() {
is_err(aggregate_values_k(14, &vals, &nv(10.0)), CellError::Num);
}
// B1: BINOM.DIST and POISSON.DIST went through ln_gamma differences (relative error 1.1e-13 at
// n = 100, 1.5e-11 at n = 10,000). Reference values: 70-digit Decimal arithmetic on the exact
// double `p` (mass by the neighbour-ratio recurrence from q^n, cdf as the running sum), rounded
// once to f64. Columns: k, n, p, mass, cdf.
#[rustfmt::skip]
const BINOM_REF: &[(f64, f64, f64, f64, f64)] = &[
(0.0, 10.0, 0.5, 0.0009765625, 0.0009765625),
(1.0, 10.0, 0.5, 0.009765625, 0.0107421875),
(3.0, 10.0, 0.5, 0.1171875, 0.171875),
(5.0, 10.0, 0.5, 0.24609375, 0.623046875),
(7.0, 10.0, 0.5, 0.1171875, 0.9453125),
(9.0, 10.0, 0.5, 0.009765625, 0.9990234375),
(0.0, 10.0, 0.3, 0.028247524900000005, 0.028247524900000005),
(1.0, 10.0, 0.3, 0.12106082100000001, 0.14930834590000003),
(2.0, 10.0, 0.3, 0.23347444050000002, 0.3827827864),
(3.0, 10.0, 0.3, 0.266827932, 0.6496107184000001),
(4.0, 10.0, 0.3, 0.200120949, 0.8497316674000001),
(5.0, 10.0, 0.3, 0.10291934519999998, 0.9526510126),
(7.0, 10.0, 0.3, 0.009001691999999999, 0.9984096136),
(9.0, 10.0, 0.3, 0.00013778099999999996, 0.9999940951),
(0.0, 10.0, 0.999, 1.0000000000000088e-30, 1.0000000000000088e-30),
(1.0, 10.0, 0.999, 9.99000000000008e-27, 9.99100000000008e-27),
(5.0, 10.0, 0.999, 2.507425174812609e-13, 2.50951798425701e-13),
(9.0, 10.0, 0.999, 0.00991035916125875, 0.009955119790251798),
(0.0, 100.0, 0.5, 7.888609052210118e-31, 7.888609052210118e-31),
(1.0, 100.0, 0.5, 7.888609052210118e-29, 7.967495142732219e-29),
(35.0, 100.0, 0.5, 0.0008638556657416528, 0.0017588208614850792),
(45.0, 100.0, 0.5, 0.048474296626430755, 0.18410080866334813),
(50.0, 100.0, 0.5, 0.07958923738717877, 0.5397946186935894),
(55.0, 100.0, 0.5, 0.048474296626430755, 0.8643734879630827),
(65.0, 100.0, 0.5, 0.0008638556657416528, 0.9991050348042566),
(99.0, 100.0, 0.5, 7.888609052210118e-29, 1.0),
(0.0, 100.0, 0.01, 0.3660323412732295, 0.3660323412732295),
(1.0, 100.0, 0.01, 0.3697296376497268, 0.7357619789229563),
(2.0, 100.0, 0.01, 0.1848648188248634, 0.9206267977478196),
(4.0, 100.0, 0.01, 0.014941714856895142, 0.9965676784122455),
(50.0, 100.0, 0.01, 6.103987557173001e-72, 1.0),
(99.0, 100.0, 0.01, 9.900000000000021e-197, 1.0),
(0.0, 100.0, 0.7, 5.15377520732019e-53, 5.15377520732019e-53),
(1.0, 100.0, 0.7, 1.2025475483747105e-50, 1.207701323582031e-50),
(50.0, 100.0, 0.7, 1.302622713144541e-05, 2.2060913327166086e-05),
(56.0, 100.0, 0.7, 0.001028871216633018, 0.0021146172813184595),
(65.0, 100.0, 0.7, 0.04677968235272982, 0.16285828837178737),
(70.0, 100.0, 0.7, 0.08678386475342809, 0.5376602639846381),
(75.0, 100.0, 0.7, 0.04955992276217012, 0.8864298182958186),
(84.0, 100.0, 0.7, 0.0005638655029474704, 0.9995950004580563),
(99.0, 100.0, 0.7, 1.3862042184106021e-14, 0.9999999999999997),
(0.0, 1000.0, 0.5, 9.332636185032189e-302, 9.332636185032189e-302),
(1.0, 1000.0, 0.5, 9.332636185032189e-299, 9.341968821217221e-299),
(453.0, 1000.0, 0.5, 0.00030353498700958767, 0.00162629990658169),
(484.0, 1000.0, 0.5, 0.015123665829158384, 0.1634683412157977),
(500.0, 1000.0, 0.5, 0.0252250181783608, 0.5126125090891804),
(516.0, 1000.0, 0.5, 0.015123665829158384, 0.8516553246133607),
(547.0, 1000.0, 0.5, 0.00030353498700958767, 0.9986772350804279),
(999.0, 1000.0, 0.5, 9.332636185032189e-299, 1.0),
(0.0, 1000.0, 0.3, 1.2532566399657381e-155, 1.2532566399657381e-155),
(1.0, 1000.0, 0.3, 5.371099885567449e-153, 5.383632451967106e-153),
(257.0, 1000.0, 0.3, 0.00030959977077928017, 0.0014835731103312242),
(286.0, 1000.0, 0.3, 0.0174263984921581, 0.1759288949498673),
(300.0, 1000.0, 0.3, 0.027521003821268385, 0.5155935198141203),
(314.0, 1000.0, 0.3, 0.017109129819617666, 0.8414952343056957),
(343.0, 1000.0, 0.3, 0.00036310112605077133, 0.9985016810663147),
(500.0, 1000.0, 0.3, 3.4791609391626546e-40, 1.0),
(999.0, 1000.0, 0.3, 0.0, 1.0),
(0.0, 1000.0, 0.001, 0.36769542477096406, 0.36769542477096406),
(1.0, 1000.0, 0.001, 0.36806348825922325, 0.7357589130301873),
(2.0, 1000.0, 0.001, 0.18403174412961162, 0.9197906571597989),
(4.0, 1000.0, 0.001, 0.01528995542083862, 0.9963631219704782),
(500.0, 1000.0, 0.001, 0.0, 1.0),
(999.0, 1000.0, 0.001, 0.0, 1.0),
(0.0, 10000.0, 0.5, 0.0, 0.0),
(1.0, 10000.0, 0.5, 0.0, 0.0),
(4850.0, 10000.0, 0.5, 8.861480551702583e-05, 0.0013939837588563594),
(4950.0, 10000.0, 0.5, 0.004839495142316416, 0.161087099897656),
(5000.0, 10000.0, 0.5, 0.007978646139382154, 0.5039893230696911),
(5050.0, 10000.0, 0.5, 0.004839495142316416, 0.8437523952446604),
(5150.0, 10000.0, 0.5, 8.861480551702583e-05, 0.9986946310466607),
(9999.0, 10000.0, 0.5, 0.0, 1.0),
(0.0, 10000.0, 0.3, 0.0, 0.0),
(1.0, 10000.0, 0.3, 0.0, 0.0),
(2863.0, 10000.0, 0.3, 9.71726742023389e-05, 0.0013927715709303387),
(2954.0, 10000.0, 0.3, 0.0052756073908582085, 0.16038574844431697),
(3000.0, 10000.0, 0.3, 0.008705361365067056, 0.504932983819298),
(3046.0, 10000.0, 0.3, 0.005244996814931252, 0.8448693061622872),
(3137.0, 10000.0, 0.3, 0.00010232641238302388, 0.9986013755801396),
(5000.0, 10000.0, 0.3, 0.0, 1.0),
(9999.0, 10000.0, 0.3, 0.0, 1.0),
(0.0, 10000.0, 0.0001, 0.3678610464329299, 0.3678610464329299),
(1.0, 10000.0, 0.0001, 0.3678978362165516, 0.7357588826494815),
(2.0, 10000.0, 0.0001, 0.1839489181082758, 0.9197078007577573),
(4.0, 10000.0, 0.0001, 0.015324477632794846, 0.996342452182816),
(5000.0, 10000.0, 0.0001, 0.0, 1.0),
(9999.0, 10000.0, 0.0001, 0.0, 1.0),
(0.0, 10000.0, 0.999, 0.0, 0.0),
(1.0, 10000.0, 0.999, 0.0, 0.0),
(5000.0, 10000.0, 0.999, 0.0, 0.0),
(9981.0, 10000.0, 0.999, 0.0037205945263045544, 0.007158161342006954),
(9987.0, 10000.0, 0.999, 0.07292252318084555, 0.20834867055478343),
(9990.0, 10000.0, 0.999, 0.12517263665023892, 0.5421328760209826),
(9993.0, 10000.0, 0.999, 0.09007019224850273, 0.8699847268232276),
(9999.0, 10000.0, 0.999, 0.00045218564541589875, 0.9999548266540229),
(0.0, 100000.0, 0.5, 0.0, 0.0),
(1.0, 100000.0, 0.5, 0.0, 0.0),
(49526.0, 100000.0, 0.5, 2.821098952769392e-05, 0.00137358872189177),
(49842.0, 100000.0, 0.5, 0.0015314620321580088, 0.15959653953914368),
(50000.0, 100000.0, 0.5, 0.00252312621419674, 0.5012615631070984),
(50158.0, 100000.0, 0.5, 0.0015314620321580088, 0.8419349224930143),
(50474.0, 100000.0, 0.5, 2.821098952769392e-05, 0.9986546222676359),
(99999.0, 100000.0, 0.5, 0.0, 1.0),
(0.0, 100000.0, 0.3, 0.0, 0.0),
(1.0, 100000.0, 0.3, 0.0, 0.0),
(29565.0, 100000.0, 0.3, 3.016598258904529e-05, 0.0013408650955754554),
(29855.0, 100000.0, 0.3, 0.0016703029966719045, 0.15934769308464344),
(30000.0, 100000.0, 0.3, 0.002752954648397428, 0.5015600059172672),
(30145.0, 100000.0, 0.3, 0.0016672321665529399, 0.8423208126454166),
(30435.0, 100000.0, 0.3, 3.067097885268188e-05, 0.9986570625478413),
(50000.0, 100000.0, 0.3, 0.0, 1.0),
(99999.0, 100000.0, 0.3, 0.0, 1.0),
(0.0, 100000.0, 0.99, 0.0, 0.0),
(1.0, 100000.0, 0.99, 0.0, 0.0),
(50000.0, 100000.0, 0.99, 0.0, 0.0),
(98906.0, 100000.0, 0.99, 0.00015962501729101418, 0.001680685850467731),
(98969.0, 100000.0, 0.99, 0.007723834924565287, 0.1660975153530963),
(99000.0, 100000.0, 0.99, 0.012678161323544589, 0.5042686879721229),
(99031.0, 100000.0, 0.99, 0.007885853372194001, 0.8416321140066325),
(99094.0, 100000.0, 0.99, 0.00013276363518518057, 0.9988439452168577),
(99999.0, 100000.0, 0.99, 0.0, 1.0),
// Small k where (1 - p)^(n - k) underflows although the mass is a normal number.
(10.0, 1100.0, 0.5, 5.050689010687844e-308, 5.097367491037505e-308),
(14.0, 1100.0, 0.5, 2.9513324041838846e-300, 2.989803284040754e-300),
(15.0, 1100.0, 0.5, 2.1367646606291324e-298, 2.16666269346954e-298),
(16.0, 1100.0, 0.5, 1.4489935354891303e-296, 1.4706601624238258e-296),
(20.0, 1100.0, 0.5, 1.711087104904223e-289, 1.7433098849775326e-289),
(25.0, 1100.0, 0.5, 3.906996229178926e-281, 3.999839082741416e-281),
];
// Columns: x, mean, mass, cdf. The last block has x > 1000, which takes the incomplete-gamma branch.
#[rustfmt::skip]
const POISSON_REF: &[(f64, f64, f64, f64)] = &[
(0.0, 0.5, 0.6065306597126334, 0.6065306597126334),
(1.0, 0.5, 0.3032653298563167, 0.9097959895689501),
(3.0, 0.5, 0.012636055410679864, 0.9982483774437092),
(0.0, 5.0, 0.006737946999085467, 0.006737946999085467),
(2.0, 5.0, 0.08422433748856833, 0.12465201948308115),
(5.0, 5.0, 0.1754673697678507, 0.6159606548330632),
(10.0, 5.0, 0.018132788707821874, 0.986304731401617),
(25.0, 5.0, 1.294589320478486e-10, 0.9999999999695003),
(70.0, 100.0, 0.00031056089812001654, 0.0009714440282773882),
(100.0, 100.0, 0.039860996809147134, 0.5265621985299984),
(130.0, 100.0, 0.000575252683815463, 0.9982931596294985),
(900.0, 1000.0, 7.516954352125952e-05, 0.0006977673277963068),
(1000.0, 1000.0, 0.012614611348721499, 0.508409367168506),
(1100.0, 1000.0, 9.498944242299507e-05, 0.9991323590365564),
(1050.0, 1000.0, 0.003599490404803648, 0.9439711616363267),
(1200.0, 1000.0, 7.992642848843571e-11, 0.9999999996115061),
(1500.0, 2000.0, 1.878037542328086e-32, 7.468034373687017e-32),
(2000.0, 2000.0, 0.00892024889598624, 0.5059467004517837),
(2100.0, 2000.0, 0.0007442234118916615, 0.9872092799080792),
(2300.0, 2000.0, 4.011826753210622e-12, 0.999999999973895),
(4800.0, 5000.0, 9.987733294132511e-05, 0.002269403878343364),
(5000.0, 5000.0, 0.005641801804664022, 0.5037611677708467),
];
fn rel_err(got: Value, want: f64) -> f64 {
// An error or a non-finite result is the worst possible miss, not a skipped one.
let Some(g) = got.as_f64().filter(|g| g.is_finite()) else { return f64::INFINITY };
if want == 0.0 { g.abs() } else { ((g - want) / want).abs() }
}
/// Error bound the grid below must meet: 1e-13, plus 2e-16 per unit of |ln value| for far-tail
/// results (an exp of a sum of large logs cannot do better in double precision). Measured: at
/// most 9e-14 on the grid (the n = 100,000 cumulative cases), 2e-14 for n <= 10,000 away from the
/// far tails, 2e-15 on the Excel-sized central cases. The log-gamma code was off by 1.5e-11.
fn allowed(want: f64) -> f64 {
// A reference that underflows to 0 must come back as (almost) 0, not as any finite value.
if want == 0.0 {
return 1e-300;
}
1e-13 + 2e-16 * want.abs().ln().abs()
}
#[test]
fn binom_and_poisson_match_exact_decimal_references() {
// Worst error relative to the allowance (above 1 is a failure) and plain worst error.
let (mut worst, mut worst_plain) = (0.0f64, 0.0f64);
let mut check = |what: &str, got: Value, want: f64| {
let e = rel_err(got, want);
let ratio = e / allowed(want);
if ratio > 1.0 {
println!("{what}: error {e:e}, allowed {:e}", allowed(want));
}
worst = worst.max(ratio);
if want > 1e-3 {
worst_plain = worst_plain.max(e);
}
};
for &(k, nn, p, pmf, cdf) in BINOM_REF {
check(&format!("BINOM.DIST({k},{nn},{p},FALSE)"), ev("BINOM.DIST", vec![n(k), n(nn), n(p), b(false)]), pmf);
check(&format!("BINOM.DIST({k},{nn},{p},TRUE)"), ev("BINOM.DIST", vec![n(k), n(nn), n(p), b(true)]), cdf);
}
for &(x, m, pmf, cdf) in POISSON_REF {
check(&format!("POISSON.DIST({x},{m},FALSE)"), ev("POISSON.DIST", vec![n(x), n(m), b(false)]), pmf);
check(&format!("POISSON.DIST({x},{m},TRUE)"), ev("POISSON.DIST", vec![n(x), n(m), b(true)]), cdf);
}
println!("worst error / allowance {worst:.3}; worst relative error where the result is above 1e-3: {worst_plain:e}");
assert!(worst <= 1.0);
assert!(worst_plain < 1e-13);
}
#[test]
fn binom_and_poisson_edge_cases() {
for cum in [false, true] {
let pm = |k: f64, nn: f64, p: f64| ev("BINOM.DIST", vec![n(k), n(nn), n(p), b(cum)]);
// p = 0 and p = 1
close(pm(0.0, 50.0, 0.0), 1.0);
close(pm(3.0, 50.0, 1.0), 0.0);
close(pm(50.0, 50.0, 1.0), 1.0);
// n = 0
close(pm(0.0, 0.0, 0.3), 1.0);
// k = n is certain when cumulative and p^n otherwise
close(pm(20.0, 20.0, 0.5), if cum { 1.0 } else { 0.5f64.powi(20) });
close_tol(pm(10_000.0, 10_000.0, 0.999), if cum { 1.0 } else { 0.999f64.powi(10_000) }, 1e-12);
// k = 0 is q^n
close_tol(pm(0.0, 1000.0, 0.001), 0.999f64.powi(1000), 1e-12);
close_tol(pm(0.0, 100_000.0, 0.00001), (1.0f64 - 1e-5).powi(100_000), 1e-9);
}
close(ev("POISSON.DIST", vec![n(0.0), n(0.0), b(false)]), 1.0);
close(ev("POISSON.DIST", vec![n(4.0), n(0.0), b(false)]), 0.0);
close(ev("POISSON.DIST", vec![n(4.0), n(0.0), b(true)]), 1.0);
close_tol(ev("POISSON.DIST", vec![n(0.0), n(3.0), b(true)]), (-3.0f64).exp(), 1e-15);
// Fractions truncate as before.
close(ev("BINOM.DIST", vec![n(6.9), n(10.9), n(0.5), b(false)]), 0.205078125);
// The 10,000 trial switch of the method no longer shows: neighbours agree to 1e-14.
let lo = ev("BINOM.DIST", vec![n(5000.0), n(10_000.0), n(0.5), b(true)]).as_f64().unwrap();
let hi = ev("BINOM.DIST", vec![n(5000.0), n(10_001.0), n(0.5), b(true)]).as_f64().unwrap();
let pmf = ev("BINOM.DIST", vec![n(5000.0), n(10_000.0), n(0.5), b(false)]).as_f64().unwrap();
// cdf(5000; 10,001) = cdf(5000; 10,000) - p * mass(5000; 10,000) by the recurrence on n
assert!(((hi - (lo - 0.5 * pmf)) / hi).abs() < 1e-13, "{lo} {hi}");
// Inverse stays consistent: the smallest k with cdf >= alpha.
close(ev("BINOM.INV", vec![n(10_000.0), n(0.5), n(0.5)]), 5000.0);
}
/// Security: FREQUENCY finds each value's bin by binary search, and MODE counts distinct numbers
/// through an index, so neither is quadratic in its inputs. Results are as before: unsorted and
/// duplicate bins keep their positions (a value goes to the first of equal bins), and tied modes
+17 -5
View File
@@ -914,13 +914,25 @@ fn regexreplace(a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
out.push_str(s.get(last..).unwrap_or_default());
return check_len(out);
}
let caps: Vec<regex::Captures> = re.captures_iter(&s).collect();
let k = occ.unsigned_abs() as usize;
let idx = if occ > 0 { k.checked_sub(1) } else { caps.len().checked_sub(k) };
let Some(cap) = idx.and_then(|i| caps.get(i)) else { return text_val(s) };
// An occurrence past what `usize` holds (32-bit targets) names no match: the text is returned as it was.
let Ok(k) = usize::try_from(occ.unsigned_abs()) else { return text_val(s) };
// Keeps only the wanted match's captures: the nth from the front, or a window of the last `k`.
let cap = if occ > 0 {
re.captures_iter(&s).nth(k - 1)
} else {
let mut last = std::collections::VecDeque::new();
for cap in re.captures_iter(&s) {
if last.len() == k {
last.pop_front();
}
last.push_back(cap);
}
if last.len() == k { last.pop_front() } else { None }
};
let Some(cap) = cap else { return text_val(s) };
let Some(m) = cap.get(0) else { return text_val(s) };
out.push_str(s.get(..m.start()).unwrap_or_default());
expand_capped(cap, &rep, &mut out, limit).ok_or(CellError::Value)?;
expand_capped(&cap, &rep, &mut out, limit).ok_or(CellError::Value)?;
out.push_str(s.get(m.end()..).unwrap_or_default());
check_len(out)
}
+295 -7
View File
@@ -3,7 +3,7 @@
//! without calling RaTeX again.
//!
//! RaTeX is young (0.1.x), so it only ever runs inside this crate's entry points ([`layout`],
//! [`to_omml`], [`check`]). Each one caps its input (4 KB, nesting 32) and runs under
//! [`to_omml`], [`check`]). Each one caps its input (4 KB, nesting 32, no macro definitions, at most [`MAX_CELLS`] matrix cells) and runs under
//! `catch_unwind`, so a stray panic becomes a [`MathError`]. See the RaTeX gate in
//! `docs/decisions.md` and `tests/gate.rs`.
//!
@@ -33,6 +33,9 @@ pub const MAX_DEPTH: usize = 32;
pub const MAX_EM: f32 = 10_000.0;
/// Most items one equation may produce.
pub const MAX_ITEMS: usize = 50_000;
/// Most matrix cells one equation may have, counting the empty cells that pad a short row to the
/// width of the widest one. A ragged matrix costs rows times the widest row, not what is typed.
pub const MAX_CELLS: usize = 2_500;
/// The error for a source over [`MAX_SOURCE_BYTES`]. It is one shared value so that a cache can
/// answer for such a source by its length alone, without reading it.
@@ -223,20 +226,61 @@ fn parse(latex: &str) -> Result<Vec<ratex_parser::ParseNode>, MathError> {
})
}
/// The size and nesting caps, checked before RaTeX sees the source.
/// Control words that define or rebind macros. User macros have no use in an equation, and
/// RaTeX caps only how many expansions run, not how large they grow, so a self-referencing
/// definition whose argument doubles on every call keeps the expander busy without bound. They are
/// refused before RaTeX sees the source.
const MACRO_DEFINERS: &[&str] =
&["def", "gdef", "edef", "xdef", "let", "futurelet", "global", "long", "newcommand", "renewcommand", "providecommand"];
/// One open `\begin…\end` environment, counted from the source text.
struct Grid {
/// Rows ended so far (by `\\`, `\cr` or `\newline`).
rows: usize,
/// Cells in the current row.
cells: usize,
/// Cells in the widest row so far.
widest: usize,
}
impl Grid {
fn new() -> Grid {
Grid { rows: 0, cells: 1, widest: 0 }
}
fn end_row(&mut self) {
self.widest = self.widest.max(self.cells);
self.rows += 1;
self.cells = 1;
}
/// The padded grid RaTeX lays out and `to_omml` writes: every row as wide as the widest.
fn padded_cells(&self) -> usize {
(self.rows + 1).saturating_mul(self.widest.max(self.cells))
}
}
/// The size, nesting, macro and matrix caps, checked before RaTeX sees the source.
fn check_caps(latex: &str) -> Result<(), MathError> {
if latex.len() > MAX_SOURCE_BYTES {
return Err(SOURCE_TOO_LONG.clone());
}
// `%` starts a comment, except inside the address of `\href` and `\url`, where RaTeX reads it
// as a character. Which one it is decides what the rest of the line means, so the two are not
// allowed together.
if latex.contains('%') && (latex.contains("\\href") || latex.contains("\\url")) {
return Err(MathError::new("A % can't be combined with a link in an equation.", latex.find('%').map(|i| i..i + 1)));
}
let mut depth = 0usize;
let mut grids: Vec<Grid> = Vec::new();
let mut cells_total = 0usize;
let mut chars = latex.char_indices().peekable();
while let Some((i, c)) = chars.next() {
let open = match c {
'\\' => {
// A control word, or an escaped character (`\{` is not a group).
// A control word (letters and `@`, as RaTeX's lexer reads it), or an escaped
// character (`\{` is not a group).
let mut word = String::new();
while let Some(&(_, n)) = chars.peek() {
if n.is_ascii_alphabetic() {
if n.is_ascii_alphabetic() || n == '@' {
word.push(n);
chars.next();
} else {
@@ -244,15 +288,70 @@ fn check_caps(latex: &str) -> Result<(), MathError> {
}
}
if word.is_empty() {
chars.next();
if let Some((_, n)) = chars.next()
&& n == '\\'
&& let Some(g) = grids.last_mut()
{
g.end_row();
}
continue;
}
if MACRO_DEFINERS.contains(&word.as_str()) {
return Err(MathError::new("Macro definitions are not supported in equations.", Some(i..i + 1 + word.len())));
}
match word.as_str() {
"left" | "begin" => Some(true),
"right" | "end" => Some(false),
"verb" => {
// Verbatim text runs to the next copy of its delimiter, whatever it
// holds: no group, comment, row or definition starts inside it.
if chars.peek().is_some_and(|&(_, n)| n == '*') {
chars.next();
}
if let Some((_, delim)) = chars.next() {
for (_, n) in chars.by_ref() {
if n == delim {
break;
}
}
}
None
}
// RaTeX reads `\newline` as `\\`, so it ends a row like `\cr` does.
"cr" | "newline" => {
if let Some(g) = grids.last_mut() {
g.end_row();
}
None
}
"begin" => {
if let Some(n) = alignat_columns(latex.get(i + 6..).unwrap_or_default())
&& n > MAX_ALIGNAT_PAIRS
{
return Err(too_many_cells(i..i + 6));
}
grids.push(Grid::new());
Some(true)
}
"end" => {
if let Some(g) = grids.pop() {
cells_total = cells_total.saturating_add(g.padded_cells());
if cells_total > MAX_CELLS {
return Err(too_many_cells(i..i + 1 + word.len()));
}
}
Some(false)
}
"left" => Some(true),
"right" => Some(false),
_ => None,
}
}
'&' | '@' => {
// `@` makes an arrow cell and an empty cell beside it in a `CD` diagram.
if let Some(g) = grids.last_mut() {
g.cells += if c == '@' { 2 } else { 1 };
}
None
}
'%' => {
// A comment runs to the end of the line.
for (_, n) in chars.by_ref() {
@@ -277,9 +376,72 @@ fn check_caps(latex: &str) -> Result<(), MathError> {
None => {}
}
}
// An environment left open is a syntax error for the parser to report, but it still counts.
for g in grids {
cells_total = cells_total.saturating_add(g.padded_cells());
}
if cells_total > MAX_CELLS {
return Err(too_many_cells(latex.len()..latex.len()));
}
Ok(())
}
/// Most column pairs `\begin{alignedat}{N}` may ask for. RaTeX allocates 2N column specs before
/// it lays anything out.
const MAX_ALIGNAT_PAIRS: usize = 50;
/// N of `{alignat}{N}`, `{alignat*}{N}` or `{alignedat}{N}` at the start of `rest` (just after
/// `\begin`). `usize::MAX` when the count is present but is not a plain digit string RaTeX reads
/// the same way (too long, or hidden by a comment, inner spaces or a macro): refused, not skipped.
/// An environment name that is computed (contains `\` or `{`) is refused the same way.
///
/// RaTeX reads the environment name and the count through its lexer, which drops comments and
/// spaces, so they are dropped here too before the two groups are compared.
fn alignat_columns(rest: &str) -> Option<usize> {
// Enough for the longest name and any count that fits `usize`; a longer count is refused.
const WINDOW: usize = 64;
let mut kept = String::new();
let mut chars = rest.chars();
while kept.len() < WINDOW {
match chars.next() {
None => break,
Some('%') => {
for n in chars.by_ref() {
if n == '\n' {
break;
}
}
}
Some(c) if c.is_whitespace() => {}
Some(c) => kept.push(c),
}
}
let group = kept.strip_prefix('{')?;
// A name that does not close inside the window cannot be read, so it is refused.
let Some((name, after)) = group.split_once('}') else {
return Some(usize::MAX);
};
// A computed name (`align\@char{97}t` reads as `alignat`) cannot be resolved here: refused
// too. Every real environment name is letters, optionally with a trailing `*`.
if name.contains(['\\', '{']) {
return Some(usize::MAX);
}
if !matches!(name, "alignat" | "alignat*" | "alignedat") {
return None;
}
let Some(digits) = after.strip_prefix('{').and_then(|a| a.split_once('}')).map(|(d, _)| d) else {
return Some(usize::MAX);
};
if digits.is_empty() || !digits.bytes().all(|b| b.is_ascii_digit()) {
return Some(usize::MAX);
}
Some(digits.parse().unwrap_or(usize::MAX))
}
fn too_many_cells(span: Range<usize>) -> MathError {
MathError::new(format!("An equation can have at most {MAX_CELLS} matrix cells (rows times the widest row)."), Some(span))
}
fn color(c: &Color) -> Option<[u8; 3]> {
let q = |v: f32| (v.clamp(0.0, 1.0) * 255.0).round() as u8;
let rgb = [q(c.r), q(c.g), q(c.b)];
@@ -509,6 +671,116 @@ mod tests {
}
}
/// Runs `f` on its own thread and fails if it has not returned within `secs` seconds.
fn within<T: Send + 'static>(secs: u64, f: impl FnOnce() -> T + Send + 'static) -> T {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::Builder::new()
.stack_size(2 << 20)
.spawn(move || {
let _ = tx.send(f());
})
.unwrap();
match rx.recv_timeout(std::time::Duration::from_secs(secs)) {
Ok(v) => v,
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => panic!("did not return within {secs} s"),
Err(e) => panic!("the worker thread died: {e}"),
}
}
#[test]
fn macro_definitions_are_refused_before_the_expander_runs() {
let defs = [
"\\def\\a#1{\\a{#1#1}}\\a{x}",
"\\gdef\\a#1{\\a{#1#1#1#1}}\\a{x}",
"\\edef\\a#1{\\a{#1#1}}\\a{x}",
"\\xdef\\a#1{{#1}{#1}}\\a{x}",
"\\newcommand{\\a}[1]{\\a{#1#1}}\\a{x}",
"\\renewcommand{\\a}[1]{\\a{#1#1}}\\a{x}",
"\\providecommand{\\a}[1]{\\a{#1#1}}\\a{x}",
"\\let\\a\\frac \\a{1}{2}",
"\\futurelet\\a\\frac \\a{1}{2}",
"\\global\\def\\a#1{\\a{#1#1}}\\a{x}",
"\\long\\def\\a#1{\\a{#1#1}}\\a{x}",
"{\\def\\a#1{\\a{#1#1}}}\\a{x}",
];
for src in defs {
// `check_caps` alone, so this cannot hang even if the refusal regresses.
let e = check_caps(src).unwrap_err();
assert!(e.message.contains("Macro definitions"), "{src}: {e}");
assert_eq!(e.span.as_ref().map(|r| &src[r.clone()]).map(|w| w.starts_with('\\')), Some(true));
let s = src.to_string();
let (c, l, o) = within(10, move || (check(&s).is_err(), layout(&s).is_err(), to_omml(&s, 11.0, None).is_err()));
assert!(c && l && o, "{src}");
}
// Names that merely start with a definer, escaped backslashes and comments are not macros.
for ok in ["\\default", "\\letter", "\\\\def", "x % \\def\\a\n+1", "\\longrightarrow"] {
assert!(check_caps(ok).is_ok(), "{ok}");
}
assert!(layout("\\longrightarrow x").is_ok());
// Text the lexer reads differently from a plain scan must not hide a definition: a
// verbatim `%` is no comment, a `%` in a link address is no comment, `@` joins a word.
for src in [
"\\verb|%|\\def\\a#1{\\a{#1#1}}\\a{x}",
"\\verb\\x\\\\def\\a#1{\\a{#1#1}}\\a{x}",
"%\\verb|\n\\def\\a#1{\\a{#1#1}}\\a{x}",
"\\href{a%b}{x}\\def\\a#1{\\a{#1#1}}\\a{x}",
"\\url{a%}\\def\\a#1{\\a{#1#1}}\\a{x}",
] {
assert!(check_caps(src).is_err(), "{src}");
}
assert!(check_caps("\\verb|\\def| x").is_ok(), "verbatim text is not a definition");
assert!(check_caps("\\def@x").is_ok(), "`\\def@x` is one unknown word to the lexer");
}
#[test]
fn ragged_matrices_are_capped_by_their_padded_size() {
let ragged = format!("\\begin{{matrix}}{}{}\\end{{matrix}}", "a\\\\".repeat(650), "a&".repeat(999) + "a");
assert!(ragged.len() < MAX_SOURCE_BYTES);
let e = check_caps(&ragged).unwrap_err();
assert!(e.message.contains("matrix cells"), "{e}");
let r = ragged.clone();
let (l, o) = within(10, move || (layout(&r).unwrap_err(), to_omml(&r, 11.0, None).unwrap_err()));
assert!(l.message.contains("matrix cells") && o.message.contains("matrix cells"));
// Without the environment words there is no grid to count; the parser rejects it.
// Ordinary and wide-but-regular matrices still work, and so does a trailing `\\`.
let wide = format!("\\begin{{pmatrix}}{}\\end{{pmatrix}}", vec!["a&".repeat(39) + "a"; 40].join("\\\\"));
assert!(wide.len() < MAX_SOURCE_BYTES);
assert!(layout(&wide).is_ok());
assert!(to_omml(&wide, 11.0, None).is_ok());
assert!(layout("\\begin{pmatrix} a & b \\\\ c & d \\\\ \\end{pmatrix}").is_ok());
// Diagram arrows and `alignedat` column counts size a grid without any `&`.
let cd = format!("\\begin{{CD}}{}{}\\end{{CD}}", "A\\\\".repeat(900), "@>>>".repeat(300));
assert!(cd.len() < MAX_SOURCE_BYTES);
assert!(check_caps(&cd).is_err_and(|e| e.message.contains("matrix cells")));
let r = "\\begin{alignedat}{99999999}a\\end{alignedat}";
assert!(check_caps(r).is_err_and(|e| e.message.contains("matrix cells")));
assert!(check_caps("\\begin{alignedat}{2}a&b&c&d\\end{alignedat}").is_ok());
// A comment, inner spaces or a macro in the count do not hide it from the cap.
for hidden in [
"\\begin{alignedat}%\n{99999999}a\\end{alignedat}",
"\\begin %c\n {alignedat}{99999999}a\\end{alignedat}",
"\\begin{alignedat}{9 9}a\\end{alignedat}",
"\\begin{alignedat}{9%x\n9}a\\end{alignedat}",
"\\begin{align%x\nedat}{99}a\\end{alignedat}",
"\\begin{alignedat}{9\\relax9}a\\end{alignedat}",
] {
assert!(check_caps(hidden).is_err_and(|e| e.message.contains("matrix cells")), "{hidden}");
}
// A computed environment name reads as `alignat` to the parser, so it is capped too.
for name in ["align\\@char{97}t", "alignedat\\relax", "al\\@char{105}gnat*"] {
let over = format!("\\begin{{{name}}}{{51}}a\\end{{alignat}}");
assert!(check_caps(&over).is_err_and(|e| e.message.contains("matrix cells")), "{over}");
}
let long = format!("\\begin{{{}}}{{51}}a\\end{{alignat}}", "a".repeat(80));
assert!(check_caps(&long).is_err_and(|e| e.message.contains("matrix cells")));
assert!(check_caps("\\begin{alignat}{50}a\\end{alignat}").is_ok());
assert!(check_caps("\\begin{alignedat} {2} % two pairs\na&b&c&d\\end{alignedat}").is_ok());
assert!(layout("\\begin{CD}A @>f>> B \\\\ @VVV @VVV \\\\ C @>>> D\\end{CD}").is_ok());
assert!(check_caps("\\operatorname{curl} 50\\%").is_ok());
let two = format!("{0}{0}", "\\begin{matrix}a&b\\\\c&d\\end{matrix}");
assert!(check_caps(&two).is_ok());
}
#[test]
fn an_injected_panic_is_an_error_not_a_crash() {
INJECT_PANIC.with(|p| p.set(true));
@@ -530,4 +802,20 @@ mod tests {
let l = layout("").unwrap();
assert!(l.items.is_empty());
}
#[test]
fn newline_rows_count_toward_the_matrix_cap() {
// RaTeX reads `\newline` as `\\`, so it ends a row like `\cr` does.
for brk in ["\\newline ", "\\cr ", "\\\\"] {
let src = format!("\\begin{{matrix}}{}{}\\end{{matrix}}", format!("a{brk}").repeat(250), "a&".repeat(700) + "a");
assert!(src.len() < MAX_SOURCE_BYTES);
let e = check_caps(&src).unwrap_err();
assert!(e.message.contains("matrix cells"), "{brk}: {e}");
let s = src.clone();
let (l, o) = within(10, move || (layout(&s).unwrap_err(), to_omml(&s, 11.0, None).unwrap_err()));
assert!(l.message.contains("matrix cells") && o.message.contains("matrix cells"), "{brk}");
}
assert!(check_caps("\\begin{matrix}a&b\\newline c&d\\end{matrix}").is_ok());
assert!(layout("\\begin{matrix}a&b\\newline c&d\\end{matrix}").is_ok());
}
}
+3
View File
@@ -406,6 +406,9 @@ impl Writer {
}
ParseNode::Array { body, is_cd: None | Some(false), .. } => {
let cols = body.iter().map(Vec::len).max().unwrap_or(0).max(1);
if body.len().saturating_mul(cols) > crate::MAX_CELLS {
return Err(MathError::new(format!("An equation can have at most {} matrix cells.", crate::MAX_CELLS), None));
}
self.out.push_str(&format!(
"<m:m><m:mPr><m:mcs><m:mc><m:mcPr><m:count m:val=\"{cols}\"/><m:mcJc m:val=\"center\"/></m:mcPr></m:mc></m:mcs></m:mPr>"
));
+88 -2
View File
@@ -11,6 +11,7 @@ use crate::style::StyleId;
/// A parsed formula. `text` is the canonical formula text without the leading `=`.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(from = "SavedFormula")]
pub struct Formula {
pub text: String,
#[serde(skip)]
@@ -39,14 +40,43 @@ impl Formula {
pub fn from_expr(e: Expr) -> Formula {
Formula { text: gridcraft_formula::print(&e), expr: Some(Arc::new(e)), array: None, dynamic: false }
}
/// The parsed expression, re-parsing after deserialization.
/// The parsed expression, re-parsing when no tree is cached.
pub fn expr(&self) -> Option<Expr> {
self.expr.as_deref().cloned().or_else(|| gridcraft_formula::parse(&self.text).ok())
}
/// The parsed expression without copying it (re-parses after deserialization).
/// The parsed expression without copying it (re-parses when no tree is cached).
pub fn expr_arc(&self) -> Option<Arc<Expr>> {
self.expr.clone().or_else(|| gridcraft_formula::parse(&self.text).ok().map(Arc::new))
}
/// Runs `f` on the parsed expression, borrowing the cached tree or, when none is cached,
/// a freshly parsed one that is dropped afterwards. Unlike [`expr`](Self::expr) it never copies
/// the tree, and unlike [`expr_arc`](Self::expr_arc) it needs no reference count or `Arc`
/// allocation. `None` when the text does not parse.
pub fn with_expr<R>(&self, f: impl FnOnce(&Expr) -> R) -> Option<R> {
match &self.expr {
Some(e) => Some(f(e)),
None => gridcraft_formula::parse(&self.text).ok().map(|e| f(&e)),
}
}
}
/// A formula as a saved workbook holds it: the tree is not saved. Reading one parses its text
/// once and caches the tree, so a workbook opened from JSON does not parse every formula again on
/// each recalculation. The text is kept as saved, and text that doesn't parse caches nothing, as
/// [`Formula::new`] does.
#[derive(Deserialize)]
struct SavedFormula {
text: String,
array: Option<RangeRef>,
#[serde(default)]
dynamic: bool,
}
impl From<SavedFormula> for Formula {
fn from(f: SavedFormula) -> Formula {
let expr = gridcraft_formula::parse(&f.text).ok().map(Arc::new);
Formula { text: f.text, expr, array: f.array, dynamic: f.dynamic }
}
}
/// A static picture stored as cell content, independent of floating drawing anchors.
@@ -121,3 +151,59 @@ impl Cell {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use gridcraft_core::CellRef;
#[test]
fn formulas_read_from_json_keep_their_text_and_cache_the_tree() -> Result<(), serde_json::Error> {
let mut array = Formula::new("=A1:B2*2");
array.array = Some(RangeRef::new(CellRef::new(0, 2), CellRef::new(1, 3)));
array.dynamic = true;
for f in [Formula::new("=SUM(A1:A3)+'My Sheet'!B2"), Formula::new("=LET(x,A1*2,x+1)"), array] {
let back: Formula = serde_json::from_str(&serde_json::to_string(&f)?)?;
assert!(back.expr.is_some(), "{}: the tree is cached after reading", f.text);
assert_eq!(back, f);
}
// Saved text is kept as written, even when the canonical print differs, and its tree is
// the one the text parses to.
let back: Formula = serde_json::from_str(r#"{"text": "sum( a1 , 2 )", "array": null}"#)?;
assert_eq!(back.text, "sum( a1 , 2 )");
assert!(!back.dynamic);
assert_eq!(back.expr.as_deref(), gridcraft_formula::parse("sum( a1 , 2 )").ok().as_ref());
assert!(back.expr.is_some());
// Text that doesn't parse caches nothing, as `Formula::new` does.
let bad = Formula::new("=SUM(");
let back: Formula = serde_json::from_str(&serde_json::to_string(&bad)?)?;
assert_eq!(back, bad);
assert!(back.expr.is_none());
Ok(())
}
#[test]
fn a_workbook_read_from_json_has_every_formula_tree() -> Result<(), Box<dyn std::error::Error>> {
let mut wb = crate::Workbook::new();
let mut array = Formula::new("=A1:A3*2");
array.array = Some(RangeRef::new(CellRef::new(0, 3), CellRef::new(2, 3)));
let sheet = wb.sheet_mut(0).ok_or("no sheet")?;
sheet.cells.set(CellRef::new(0, 0), Cell::value(Value::Number(1.0)));
sheet.cells.set(CellRef::new(0, 1), Cell::formula(Formula::new("=SUM(A1:A3)")));
sheet.cells.set(CellRef::new(1, 1), Cell::formula(Formula::new("=IF(A1>0,\"yes\",\"no\")")));
sheet.cells.set(CellRef::new(2, 1), Cell::formula(Formula::new("=SUM(")));
sheet.cells.set(CellRef::new(0, 3), Cell::formula(array));
let json = serde_json::to_string(&wb)?;
let back: crate::Workbook = serde_json::from_str(&json)?;
let (before, after) = (wb.sheet(0).ok_or("no sheet")?, back.sheet(0).ok_or("no sheet")?);
let mut formulas = 0;
for (c, cell) in before.cells.iter() {
let (Some(f), Some(read)) = (&cell.formula, after.cell(c).and_then(|cell| cell.formula.as_ref())) else { continue };
assert_eq!(read, f, "{c:?}: text, tree and flags as saved");
assert_eq!(read.expr.is_some(), gridcraft_formula::parse(&f.text).is_ok(), "{c:?}: every formula that parses has its tree");
formulas += 1;
}
assert_eq!(formulas, 4);
Ok(())
}
}
+24
View File
@@ -230,6 +230,30 @@ pub enum CfRule {
},
}
impl CfRule {
/// The differential style a matching rule applies; colour scales, data bars and icon sets have none.
pub fn style(&self) -> Option<&Style> {
match self {
CfRule::CellIs { style, .. }
| CfRule::Expression { style, .. }
| CfRule::ContainsText { style, .. }
| CfRule::NotContainsText { style, .. }
| CfRule::BeginsWith { style, .. }
| CfRule::EndsWith { style, .. }
| CfRule::Blanks { style }
| CfRule::NoBlanks { style }
| CfRule::Errors { style }
| CfRule::NoErrors { style }
| CfRule::Duplicate { style }
| CfRule::Unique { style }
| CfRule::Top10 { style, .. }
| CfRule::AboveAverage { style, .. }
| CfRule::TimePeriod { style, .. } => Some(style),
CfRule::ColorScale { .. } | CfRule::DataBar { .. } | CfRule::IconSet { .. } => None,
}
}
}
/// Number of icons in an icon set (`3Arrows` → 3, `5Quarters` → 5), always 3 to 5.
pub fn icon_count(set: &str) -> usize {
set.chars().next().and_then(|c| c.to_digit(10)).unwrap_or(3).clamp(3, 5) as usize
+2 -2
View File
@@ -26,8 +26,8 @@ pub use python::{PyCode, PyOutput};
pub use sheet::{DEFAULT_COL_WIDTH, DEFAULT_ROW_HEIGHT, LineIndex, Sheet, SheetUid, SheetViewMode, Visibility, points_to_chars};
pub use store::CellStore;
pub use style::{
Alignment, BorderLine, BorderStyle, Borders, Color, Fill, Font, HAlign, NumFmt, PatternType, Style, StyleId, StyleTable, Theme, Underline,
VAlign, VertAlign,
Alignment, BorderLine, BorderStyle, Borders, Color, Fill, Font, HAlign, NumFmt, PatternType, ReadingOrder, Style, StyleId, StyleTable, Theme,
Underline, VAlign, VertAlign,
};
pub use workbook::{
CalcMode, CalcSettings, DefinedName, DocProps, EQUATION_BUDGET_MESSAGE, MAX_EQUATION_LAYOUT_BYTES, PhoneticProps, PhoneticRun, PhoneticString,
+106 -7
View File
@@ -491,12 +491,14 @@ impl Sheet {
/// sheet AutoFilter while it has criteria, or a hidden data row of a filtered table.
/// Other hidden rows were hidden by hand.
pub fn is_row_filtered(&self, row: u32) -> bool {
if !self.is_row_hidden(row) {
return false;
}
let in_sheet_filter =
self.autofilter.as_ref().is_some_and(|af| !af.criteria.is_empty() && row > af.range.start.row && row <= af.range.end.row);
in_sheet_filter || self.tables.iter().any(|t| !t.filter.is_empty() && t.data_range().is_some_and(|r| row >= r.start.row && row <= r.end.row))
self.is_row_hidden(row) && self.filter_row_spans().iter().any(|&(first, last)| row >= first && row <= last)
}
/// The row spans (first, last) an applied filter can hide: the data rows of the sheet
/// AutoFilter while it has criteria, and of each filtered table.
pub fn filter_row_spans(&self) -> Vec<(u32, u32)> {
let sheet = self.autofilter.as_ref().filter(|af| !af.criteria.is_empty()).map(|af| (af.range.start.row.saturating_add(1), af.range.end.row));
let tables = self.tables.iter().filter(|t| !t.filter.is_empty()).filter_map(|t| t.data_range()).map(|r| (r.start.row, r.end.row));
sheet.into_iter().chain(tables).collect()
}
pub fn is_col_hidden(&self, col: u32) -> bool {
self.cols.get(&col).is_some_and(|r| r.hidden)
@@ -564,7 +566,9 @@ impl LineIndex {
let mut prev = 0u32;
for (&line, info) in custom {
pos += (line - prev) as f64 * default;
let size = if info.hidden { 0.0 } else { info.size.map_or(default, |s| s as f64) };
// A negative size (only a malformed file has one) counts as zero, so positions only
// grow and the searches below stay valid.
let size = if info.hidden { 0.0 } else { info.size.map_or(default, |s| s as f64).max(0.0) };
lines.push((line, pos, size));
pos += size;
prev = line + 1;
@@ -582,6 +586,44 @@ impl LineIndex {
},
}
}
/// The first line at or after `line` (and before the axis end) with a size above zero, or
/// `None` when every line from there on is hidden or zero-sized. O(log n): a run of hidden
/// lines all start where the next shown line starts, so one search steps over the whole run.
pub fn next_visible(&self, line: u32) -> Option<u32> {
if line >= self.max {
return None;
}
let i = self.lines.partition_point(|e| e.0 < line);
let custom = self.lines.get(i).is_some_and(|e| e.0 == line);
if !custom && self.default > 0.0 {
return Some(line);
}
// Every line from `line` up to the answer is zero-sized, so they all start at `p`; the
// last custom line starting at `p` either has a size (the answer) or is the end of the run.
let p = self.start(line);
let k = self.lines.partition_point(|e| e.1 <= p);
let &(last, _, size) = self.lines.get(k.checked_sub(1)?)?;
if last < line {
return None;
}
if size > 0.0 {
return Some(last).filter(|l| *l < self.max);
}
// The line after the run has no custom line (a custom one would start at `p` too and be
// part of the run): it is shown when the default size is.
let next = last.checked_add(1).filter(|n| *n < self.max)?;
(self.default > 0.0).then_some(next)
}
/// The lines from `first` to `last` (inclusive) with a size above zero, in order. Runs of
/// hidden lines cost one O(log n) step each, not one per line.
pub fn visible(&self, first: u32, last: u32) -> impl Iterator<Item = u32> + '_ {
let mut next = self.next_visible(first);
std::iter::from_fn(move || {
let line = next.filter(|l| *l <= last)?;
next = line.checked_add(1).and_then(|n| self.next_visible(n));
Some(line)
})
}
/// The line containing `pos`.
pub fn at(&self, pos: f64) -> u32 {
if pos <= 0.0 {
@@ -642,6 +684,63 @@ mod tests {
use super::*;
use crate::cell::Formula;
/// `LineIndex::visible` lists exactly the lines a line-by-line size check shows, for hidden
/// runs at the start, in the middle and up to the end of the axis, custom sizes (zero and
/// negative included), lines with only a style, and a zero default size.
#[test]
fn visible_lines_match_a_line_by_line_check() {
let max = 400u32;
let mut seed = 0x2545_f491_4f6c_dd1du64;
let mut rnd = |n: u64| {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
seed % n
};
for case in 0..300 {
let mut custom = BTreeMap::new();
let runs = rnd(6);
for _ in 0..runs {
let start = rnd(u64::from(max)) as u32;
let len = rnd(if case % 3 == 0 { 400 } else { 30 }) as u32;
for l in start..(start + len).min(max) {
custom.insert(l, LineInfo { hidden: true, ..Default::default() });
}
}
// Runs that start at the first line or end at the last one.
if case % 5 == 0 {
for l in 0..rnd(20) as u32 {
custom.insert(l, LineInfo { hidden: true, ..Default::default() });
}
}
if case % 5 == 1 {
for l in max - rnd(20) as u32 - 1..max {
custom.insert(l, LineInfo { hidden: true, ..Default::default() });
}
}
for _ in 0..rnd(8) {
let size = [Some(0.0), Some(3.0), Some(20.0), Some(-5.0), None][rnd(5) as usize];
custom.insert(rnd(u64::from(max)) as u32, LineInfo { size, ..Default::default() });
}
let default = if case % 7 == 0 { 0.0 } else { 15.0 };
let size = |l: u32| match custom.get(&l) {
Some(i) if i.hidden => 0.0,
Some(LineInfo { size: Some(s), .. }) => s.max(0.0),
_ => default,
};
let index = LineIndex::new(&custom, default, max);
for (first, last) in [(0, max - 1), (rnd(u64::from(max)) as u32, max - 1), (5, 5), (max - 1, max - 1)] {
let want: Vec<u32> = (first..=last).filter(|&l| size(l) > 0.0).collect();
assert_eq!(index.visible(first, last).collect::<Vec<_>>(), want, "case {case}, {first}..={last}");
}
assert_eq!(index.next_visible(max), None);
}
// The row after a million hidden rows comes straight after the last shown one.
let custom: BTreeMap<u32, LineInfo> = (47..1_048_422).map(|r| (r, LineInfo { hidden: true, ..Default::default() })).collect();
let index = LineIndex::new(&custom, 15.0, 1_048_576);
assert_eq!(index.visible(40, 1_048_424).collect::<Vec<_>>(), [40, 41, 42, 43, 44, 45, 46, 1_048_422, 1_048_423, 1_048_424]);
}
#[test]
fn a_sheet_keeps_its_identity_through_clones_and_a_new_sheet_gets_its_own() {
let a = Sheet::new("A");
+367 -35
View File
@@ -4,6 +4,8 @@
//! copies only the band pointers; editing a cell copies just its band.
use std::collections::BTreeMap;
use std::collections::btree_map::Entry;
use std::ops::RangeInclusive;
use std::sync::Arc;
use gridcraft_core::{CellRef, RangeRef};
@@ -266,64 +268,187 @@ impl CellStore {
}
bytes
}
/// Removes every cell in `r` and returns them.
/// Removes every cell in `r` and returns them, row-major.
pub fn take_range(&mut self, r: RangeRef) -> Vec<(CellRef, Cell)> {
let keys: Vec<CellRef> = self.iter_range(r).map(|(c, _)| c).collect();
keys.into_iter().filter_map(|c| self.remove(c).map(|cell| (c, cell))).collect()
if r.end.col < gridcraft_core::MAX_COLS - 1 {
// A block with columns to its right: cutting it out of each row map measured slower
// than removing its cells one by one (#3575).
let keys: Vec<CellRef> = self.iter_range(r).map(|(c, _)| c).collect();
return keys.into_iter().filter_map(|c| self.remove(c).map(|cell| (c, cell))).collect();
}
let rows = self.take_rows(r.start.row..=r.end.row, r.start.col..=r.end.col);
rows.into_iter().flat_map(|(row, cols)| cols.into_iter().map(move |(col, cell)| (CellRef::new(row, col), cell))).collect()
}
/// Rows strictly greater than or equal to `from` moved by `delta` (positive = down). Rows that
/// would land before `from`... callers delete the vacated block first.
pub fn shift_rows(&mut self, from: u32, delta: i64) {
let moved: Vec<(CellRef, Cell)> = {
let keys: Vec<CellRef> = self.iter().filter(|(c, _)| c.row >= from).map(|(c, _)| c).collect();
keys.into_iter().filter_map(|c| self.remove(c).map(|cell| (c, cell))).collect()
};
for (c, cell) in moved {
let nr = c.row as i64 + delta;
if (0..gridcraft_core::MAX_ROWS as i64).contains(&nr) {
self.set(CellRef::new(nr as u32, c.col), cell);
}
}
let moved = self.take_rows(from..=u32::MAX, 0..=u32::MAX);
self.put_rows(moved.into_iter().filter_map(|(row, cols)| Some((shifted(row, delta, gridcraft_core::MAX_ROWS)?, cols))));
}
pub fn shift_cols(&mut self, from: u32, delta: i64) {
let keys: Vec<CellRef> = self.iter().filter(|(c, _)| c.col >= from).map(|(c, _)| c).collect();
let moved: Vec<(CellRef, Cell)> = keys.into_iter().filter_map(|c| self.remove(c).map(|cell| (c, cell))).collect();
for (c, cell) in moved {
let nc = c.col as i64 + delta;
if (0..gridcraft_core::MAX_COLS as i64).contains(&nc) {
self.set(CellRef::new(c.row, nc as u32), cell);
}
}
let moved = self.take_rows(0..=u32::MAX, from..=u32::MAX);
self.put_rows(moved.into_iter().map(|(row, cols)| (row, shift_keys(cols, delta))));
}
/// Shifts cells within rows `r0..=r1` and columns `>= from` horizontally (Insert/Delete Cells
/// with shift right/left).
pub fn shift_cols_in_rows(&mut self, r0: u32, r1: u32, from: u32, delta: i64) {
let range = RangeRef::new(CellRef::new(r0, from), CellRef::new(r1, gridcraft_core::MAX_COLS - 1));
let moved = self.take_range(range);
for (c, cell) in moved {
let nc = c.col as i64 + delta;
if (0..gridcraft_core::MAX_COLS as i64).contains(&nc) {
self.set(CellRef::new(c.row, nc as u32), cell);
}
}
let r = RangeRef::new(CellRef::new(r0, from), CellRef::new(r1, gridcraft_core::MAX_COLS - 1));
let moved = self.take_rows(r.start.row..=r.end.row, r.start.col..=r.end.col);
self.put_rows(moved.into_iter().map(|(row, cols)| (row, shift_keys(cols, delta))));
}
/// Shifts cells within columns `c0..=c1` and rows `>= from` vertically.
pub fn shift_rows_in_cols(&mut self, c0: u32, c1: u32, from: u32, delta: i64) {
// Cell by cell: the moved cells sit between columns that stay, so moving row maps
// measured no faster here (#3575).
let range = RangeRef::new(CellRef::new(from, c0), CellRef::new(gridcraft_core::MAX_ROWS - 1, c1));
let moved = self.take_range(range);
for (c, cell) in moved {
let nr = c.row as i64 + delta;
if (0..gridcraft_core::MAX_ROWS as i64).contains(&nr) {
self.set(CellRef::new(nr as u32, c.col), cell);
for (c, cell) in self.take_range(range) {
if let Some(nr) = shifted(c.row, delta, gridcraft_core::MAX_ROWS) {
self.set(CellRef::new(nr, c.col), cell);
}
}
}
/// Removes the cells in `rows` x `cols` and returns them grouped by row, in row order. Moves
/// whole column maps (or the split-off part of one) instead of removing cell by cell, and only
/// copies bands that hold a cell in the block, so snapshots keep sharing the others.
fn take_rows(&mut self, rows: RangeInclusive<u32>, cols: RangeInclusive<u32>) -> Vec<(u32, BTreeMap<u32, Cell>)> {
let (r0, r1, c0, c1) = (*rows.start(), *rows.end(), *cols.start(), *cols.end());
let mut out = Vec::new();
if r0 > r1 || c0 > c1 {
return out;
}
let whole_rows = c0 == 0 && c1 == u32::MAX;
let keys: Vec<u32> = self
.bands
.range(r0 / BAND..=r1 / BAND)
.filter(|(_, band)| band.rows.range(r0..=r1).any(|(_, row)| row.range(c0..=c1).next().is_some()))
.map(|(k, _)| *k)
.collect();
for key in keys {
let Some(band) = self.bands.get_mut(&key) else { continue };
let inside = |row: Option<&u32>| row.is_some_and(|r| (r0..=r1).contains(r));
if whole_rows && inside(band.rows.keys().next()) && inside(band.rows.keys().next_back()) {
// The whole band moves: no splitting.
let Some(band) = self.bands.remove(&key) else { continue };
for (row, taken) in Arc::unwrap_or_clone(band).rows {
self.len -= taken.len();
out.push((row, taken));
}
continue;
}
let band = Arc::make_mut(band);
let mut block = band.rows.split_off(&r0);
let mut tail = match r1.checked_add(1) {
Some(next) => block.split_off(&next),
None => BTreeMap::new(),
};
for (row, mut kept) in block {
let taken = if whole_rows {
std::mem::take(&mut kept)
} else if c1.checked_add(1).is_none_or(|next| kept.range(next..).next().is_none()) {
// Nothing right of the block: one split.
kept.split_off(&c0)
} else {
let mut taken = BTreeMap::new();
for (col, cell) in kept.extract_if(c0..=c1, |_, _| true) {
taken.insert(col, cell);
}
taken
};
if !kept.is_empty() {
band.rows.insert(row, kept);
}
if !taken.is_empty() {
self.len -= taken.len();
out.push((row, taken));
}
}
band.rows.append(&mut tail);
if band.rows.is_empty() {
self.bands.remove(&key);
}
}
out
}
/// Stores rows of cells (each row at most once) the way `set` stores each cell: a cell
/// replaces what is there, and a blank cell clears its destination instead of being stored.
/// A row with nothing stored yet takes the whole map. Rows in order share one band lookup.
fn put_rows(&mut self, rows: impl Iterator<Item = (u32, BTreeMap<u32, Cell>)>) {
let mut rows = rows.peekable();
let mut group = Vec::new();
while let Some((first, cols)) = rows.next() {
let key = first / BAND;
group.clear();
group.push((first, cols));
while let Some(next) = rows.next_if(|(row, _)| row / BAND == key) {
group.push(next);
}
// Leave the band shared when only blank cells land on empty positions.
let existing = self.bands.get(&key);
let writes = group.iter().any(|(row, cols)| {
let dest = existing.and_then(|band| band.rows.get(row));
cols.iter().any(|(col, cell)| !cell.is_blank() || dest.is_some_and(|d| d.contains_key(col)))
});
if !writes {
continue;
}
let band = Arc::make_mut(self.bands.entry(key).or_default());
for (row, mut cols) in group.drain(..) {
match band.rows.entry(row) {
Entry::Vacant(slot) => {
if cols.values().any(Cell::is_blank) {
cols.retain(|_, cell| !cell.is_blank());
}
if !cols.is_empty() {
self.len += cols.len();
slot.insert(cols);
}
}
Entry::Occupied(mut slot) => {
let dest = slot.get_mut();
for (col, cell) in cols {
if cell.is_blank() {
if dest.remove(&col).is_some() {
self.len -= 1;
}
} else if dest.insert(col, cell).is_none() {
self.len += 1;
}
}
if dest.is_empty() {
slot.remove();
}
}
}
}
if band.rows.is_empty() {
self.bands.remove(&key);
}
}
}
}
/// `at + delta` when it lies in `0..limit`.
fn shifted(at: u32, delta: i64, limit: u32) -> Option<u32> {
let to = (at as i64).checked_add(delta)?;
if (0..limit as i64).contains(&to) { Some(to as u32) } else { None }
}
/// The cells of one row with their columns moved by `delta`; cells that would leave the sheet are
/// dropped.
fn shift_keys(cols: BTreeMap<u32, Cell>, delta: i64) -> BTreeMap<u32, Cell> {
let mut out = BTreeMap::new();
for (col, cell) in cols {
if let Some(c) = shifted(col, delta, gridcraft_core::MAX_COLS) {
out.insert(c, cell);
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use gridcraft_core::Value;
use gridcraft_core::{MAX_COLS, MAX_ROWS, Value};
fn c(r: u32, col: u32) -> CellRef {
CellRef::new(r, col)
@@ -396,4 +521,211 @@ mod tests {
s.shift_cols(0, 1);
assert!(s.get(c(7, 1)).is_some());
}
/// The per-cell algorithms the shifts used before they moved whole rows (#3575), kept as the
/// reference the structural versions must match cell for cell.
mod per_cell {
use super::*;
fn reset(s: &mut CellStore, moved: Vec<(CellRef, Cell)>, to: impl Fn(CellRef) -> Option<CellRef>) {
for (at, cell) in moved {
if let Some(at) = to(at) {
s.set(at, cell);
}
}
}
fn row_by(delta: i64) -> impl Fn(CellRef) -> Option<CellRef> {
move |at| {
let r = at.row as i64 + delta;
(0..MAX_ROWS as i64).contains(&r).then(|| CellRef::new(r as u32, at.col))
}
}
fn col_by(delta: i64) -> impl Fn(CellRef) -> Option<CellRef> {
move |at| {
let c = at.col as i64 + delta;
(0..MAX_COLS as i64).contains(&c).then(|| CellRef::new(at.row, c as u32))
}
}
pub fn take_range(s: &mut CellStore, r: RangeRef) -> Vec<(CellRef, Cell)> {
let keys: Vec<CellRef> = s.iter_range(r).map(|(c, _)| c).collect();
keys.into_iter().filter_map(|c| s.remove(c).map(|cell| (c, cell))).collect()
}
pub fn shift_rows(s: &mut CellStore, from: u32, delta: i64) {
let keys: Vec<CellRef> = s.iter().filter(|(c, _)| c.row >= from).map(|(c, _)| c).collect();
let moved = keys.into_iter().filter_map(|c| s.remove(c).map(|cell| (c, cell))).collect();
reset(s, moved, row_by(delta));
}
pub fn shift_cols(s: &mut CellStore, from: u32, delta: i64) {
let keys: Vec<CellRef> = s.iter().filter(|(c, _)| c.col >= from).map(|(c, _)| c).collect();
let moved = keys.into_iter().filter_map(|c| s.remove(c).map(|cell| (c, cell))).collect();
reset(s, moved, col_by(delta));
}
pub fn shift_cols_in_rows(s: &mut CellStore, r0: u32, r1: u32, from: u32, delta: i64) {
let moved = take_range(s, RangeRef::new(CellRef::new(r0, from), CellRef::new(r1, MAX_COLS - 1)));
reset(s, moved, col_by(delta));
}
pub fn shift_rows_in_cols(s: &mut CellStore, c0: u32, c1: u32, from: u32, delta: i64) {
let moved = take_range(s, RangeRef::new(CellRef::new(from, c0), CellRef::new(MAX_ROWS - 1, c1)));
reset(s, moved, row_by(delta));
}
}
struct Lcg(u64);
impl Lcg {
fn below(&mut self, n: u64) -> u64 {
self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
(self.0 >> 33) % n.max(1)
}
fn pick<T: Copy>(&mut self, xs: &[T]) -> T {
xs[self.below(xs.len() as u64) as usize]
}
}
/// A position near a band edge, the sheet edge, or anywhere below `max`.
fn coord(rng: &mut Lcg, max: u32, edges: &[u32]) -> u32 {
match rng.below(3) {
0 => rng.pick(edges).min(max - 1),
1 => max - 1 - rng.below(70.min(max as u64)) as u32,
_ => rng.below(max as u64) as u32,
}
}
/// Random store with values, styled-only cells, a few cells off the grid, and blank cells left
/// behind by `get_mut` (which `set` never stores, but the old shifts dropped and let clear
/// their destination).
fn random_store(rng: &mut Lcg, max_r: u32, max_c: u32) -> CellStore {
let row_edges = [0, 1, 62, 63, 64, 65, 127, 128, 129, 191, 192, MAX_ROWS - 64, MAX_ROWS - 1];
let col_edges = [0, 1, 2, 63, 64, 65, MAX_COLS - 2, MAX_COLS - 1];
let mut s = CellStore::new();
for _ in 0..rng.below(300) {
let mut at = CellRef::new(coord(rng, max_r, &row_edges), coord(rng, max_c, &col_edges));
// Now and then a cell outside the grid (`set` does not check): the shifts must treat
// it as the old code did.
match rng.below(60) {
0 => at.row = MAX_ROWS + rng.below(3) as u32,
1 => at.col = MAX_COLS + rng.below(3) as u32,
_ => {}
}
let cell = match rng.below(5) {
0 => Cell { style: crate::style::StyleId(1 + rng.below(3) as u32), ..Cell::default() },
1 => Cell::value(Value::from("t")),
_ => Cell::value(Value::Number(rng.below(1000) as f64)),
};
s.set(at, cell);
if rng.below(8) == 0
&& let Some(cell) = s.get_mut(at)
{
*cell = Cell::default();
}
}
s
}
fn assert_same(new: &CellStore, old: &CellStore, base: &CellStore, base_copy: &CellStore, what: &str) {
assert_eq!(new, old, "{what}");
assert_eq!(new.len(), new.iter().count(), "{what}: len");
assert!(new.iter().map(|(c, x)| (c, x.clone())).eq(old.iter().map(|(c, x)| (c, x.clone()))), "{what}: iteration");
assert!(new.bands.values().all(|b| !b.rows.is_empty() && b.rows.values().all(|r| !r.is_empty())), "{what}: empty band or row");
// Copy-on-write: every band the old code left shared with the snapshot stays shared.
for (k, b) in &base.bands {
if old.bands.get(k).is_some_and(|o| Arc::ptr_eq(o, b)) {
assert!(new.bands.get(k).is_some_and(|n| Arc::ptr_eq(n, b)), "{what}: band {k} copied");
}
}
assert_eq!(base, base_copy, "{what}: snapshot changed");
}
#[test]
fn structural_shifts_match_per_cell_shifts() {
let mut rng = Lcg(3575);
let deltas = [0i64, 1, -1, 2, -2, 63, -63, 64, -64, 65, -65, 128, -128, 1000, -1000, MAX_ROWS as i64, -(MAX_ROWS as i64)];
let row_edges = [0, 1, 63, 64, 65, 127, 128, MAX_ROWS - 1, MAX_ROWS];
let col_edges = [0, 1, 63, 64, 65, MAX_COLS - 1, MAX_COLS];
for trial in 0..3000 {
let max_r = [200, 5000, MAX_ROWS][trial % 3];
let max_c = [30, MAX_COLS][trial / 3 % 2];
let mut base = random_store(&mut rng, max_r, max_c);
for step in 0..4 {
let snapshot = base.clone();
let (mut new, mut old) = (base.clone(), base.clone());
let delta = if rng.below(2) == 0 { rng.pick(&deltas) } else { rng.below(301) as i64 - 150 };
let row = if rng.below(2) == 0 { rng.pick(&row_edges) } else { rng.below(max_r as u64 + 1) as u32 };
let col = if rng.below(2) == 0 { rng.pick(&col_edges) } else { rng.below(max_c as u64 + 1) as u32 };
let (row2, col2) = (rng.below(max_r as u64 + 1) as u32, rng.below(max_c as u64 + 1) as u32);
let op = rng.below(5);
match op {
0 => {
new.shift_rows(row, delta);
per_cell::shift_rows(&mut old, row, delta);
}
1 => {
new.shift_cols(col, delta);
per_cell::shift_cols(&mut old, col, delta);
}
2 => {
new.shift_cols_in_rows(row, row2, col, delta);
per_cell::shift_cols_in_rows(&mut old, row, row2, col, delta);
}
3 => {
new.shift_rows_in_cols(col, col2, row, delta);
per_cell::shift_rows_in_cols(&mut old, col, col2, row, delta);
}
_ => {
let r = RangeRef::new(CellRef::new(row, col), CellRef::new(row2, col2));
assert_eq!(new.take_range(r), per_cell::take_range(&mut old, r), "trial {trial} step {step}: taken cells");
}
}
let what = format!("trial {trial} step {step} op {op} row {row}..{row2} col {col}..{col2} delta {delta}");
assert_same(&new, &old, &base, &snapshot, &what);
base = new;
}
}
}
#[test]
fn off_grid_cells_right_of_a_block_stay_put() {
let mut s = CellStore::new();
for col in [0, 5, MAX_COLS - 1, MAX_COLS + 1] {
s.set(c(3, col), Cell::value((col as f64).into()));
}
let mut old = s.clone();
s.shift_cols_in_rows(3, 3, 0, 1);
per_cell::shift_cols_in_rows(&mut old, 3, 3, 0, 1);
assert_eq!(s, old);
assert_eq!(s.get(c(3, MAX_COLS + 1)).map(|x| x.value.clone()), Some(Value::Number((MAX_COLS + 1) as f64)));
let r = RangeRef::rows(3, 3);
assert_eq!(s.take_range(r), per_cell::take_range(&mut old, r));
assert_eq!(s, old);
}
#[test]
fn moved_blank_cell_clears_its_destination() {
let mut s = CellStore::new();
s.set(c(3, 0), Cell::value(1.0.into()));
s.set(c(5, 0), Cell::value(2.0.into()));
s.set(c(5, 1), Cell::value(3.0.into()));
if let Some(cell) = s.get_mut(c(5, 0)) {
*cell = Cell::default();
}
s.shift_rows(5, -2);
assert!(s.get(c(3, 0)).is_none());
assert_eq!(s.get(c(3, 1)).map(|x| x.value.clone()), Some(Value::Number(3.0)));
assert_eq!(s.len(), 1);
}
#[test]
fn shifting_keeps_untouched_bands_shared() {
let mut s = CellStore::new();
for r in 0..256 {
s.set(c(r, 0), Cell::value((r as f64).into()));
}
s.set(c(200, 5), Cell::value("x".into()));
let snap = s.clone();
s.shift_rows(130, 1);
assert!(Arc::ptr_eq(&s.bands[&0], &snap.bands[&0]));
assert!(Arc::ptr_eq(&s.bands[&1], &snap.bands[&1]));
s.shift_cols(5, 1);
assert!(Arc::ptr_eq(&s.bands[&0], &snap.bands[&0]));
assert_eq!(s.get(c(201, 6)).map(|x| x.value.clone()), Some(Value::from("x")));
}
}
+30
View File
@@ -350,6 +350,14 @@ pub enum VAlign {
Distributed,
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum ReadingOrder {
#[default]
Context,
LeftToRight,
RightToLeft,
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct Alignment {
pub h: HAlign,
@@ -359,6 +367,9 @@ pub struct Alignment {
pub indent: u8,
/// Degrees -90..=90, or 255 for vertical stacked text.
pub rotation: i16,
/// Unspecified inherits in differential styles and uses context for ordinary cells.
#[serde(default)]
pub reading_order: Option<ReadingOrder>,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
@@ -488,6 +499,25 @@ static DEFAULT_STYLE: std::sync::LazyLock<Style> = std::sync::LazyLock::new(Styl
mod tests {
use super::*;
#[test]
fn reading_order_defaults_for_older_json_and_distinguishes_styles() {
let align: Alignment = serde_json::from_str(r#"{"h":"General","v":"Bottom","wrap":false,"shrink":false,"indent":0,"rotation":0}"#).unwrap();
assert_eq!(align.reading_order, None);
let mut styles = StyleTable::default();
let context = styles.derive(StyleId::DEFAULT, |s| s.align.reading_order = Some(ReadingOrder::Context));
let ltr = styles.derive(StyleId::DEFAULT, |s| s.align.reading_order = Some(ReadingOrder::LeftToRight));
let rtl = styles.derive(StyleId::DEFAULT, |s| s.align.reading_order = Some(ReadingOrder::RightToLeft));
assert_ne!(ltr, rtl);
assert_ne!(context, StyleId::DEFAULT);
assert_ne!(context, ltr);
assert_ne!(context, rtl);
assert_ne!(ltr, StyleId::DEFAULT);
assert_ne!(rtl, StyleId::DEFAULT);
assert_eq!(styles.derive(ltr, |s| s.align.reading_order = Some(ReadingOrder::RightToLeft)), rtl);
assert_eq!(styles.derive(rtl, |s| s.align.reading_order = Some(ReadingOrder::Context)), context);
assert_eq!(styles.derive(context, |s| s.align.reading_order = None), StyleId::DEFAULT);
}
#[test]
fn interning() {
let mut t = StyleTable::default();
+6
View File
@@ -268,6 +268,11 @@ pub struct Workbook {
/// `preserved` are relative to. Saving into another folder rewrites them from there. Not in JSON.
#[serde(skip)]
pub source_dir: Option<std::path::PathBuf>,
/// The region whose formats this workbook is typed and shown in (Options › Region). A runtime
/// setting of the application, not of the file: it is not saved, and the engine session sets
/// it on every document it holds. en-US until then.
#[serde(skip, default = "gridcraft_core::region::default_region")]
pub region: &'static gridcraft_core::Region,
/// The file this workbook reports as its own, which is what `CELL("filename")` returns.
/// `None` = no file yet, which Excel reports as an empty string until the workbook has been
/// saved once. Deliberately separate from `source_dir`: that one names the folder relative
@@ -365,6 +370,7 @@ impl Workbook {
macros_unread: Arc::default(),
preserved: None,
source_dir: None,
region: gridcraft_core::region::default_region(),
source_path: None,
}
}
+206
View File
@@ -0,0 +1,206 @@
//! Calendars a format section can show dates in: the `B1` (Gregorian) and `B2` (Hijri) prefixes.
use gridcraft_core::date::{DateTime, MONTHS, WEEKDAYS, days_from_civil, days_in_month};
/// The calendar a section's year, month and day parts count in.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub(crate) enum Calendar {
#[default]
Gregorian,
/// The tabular (arithmetic) Islamic calendar.
Hijri,
}
/// Arabic names of the Hijri months, Muharram first. Two-word names are joined by a no-break
/// space (U+00A0), as Excel writes them.
const HIJRI_MONTHS: [&str; 12] = [
"محرم",
"صفر",
"ربيع\u{a0}الأول",
"ربيع\u{a0}الثاني",
"جمادى\u{a0}الأولى",
"جمادى\u{a0}الثانية",
"رجب",
"شعبان",
"رمضان",
"شوال",
"ذو\u{a0}القعدة",
"ذو\u{a0}الحجة",
];
/// Arabic weekday names, Sunday first, shown for `ddd` and `dddd` under `B2`.
const ARABIC_WEEKDAYS: [&str; 7] = ["الأحد", "الإثنين", "الثلاثاء", "الأربعاء", "الخميس", "الجمعة", "السبت"];
/// Year, month and day of a date in a section's calendar, with the month and weekday names.
pub(crate) struct CalendarDate {
pub year: i64,
pub month: u32,
pub day: u32,
/// `mmmm`.
pub month_name: &'static str,
/// `mmm`.
pub month_short: &'static str,
/// `dddd`.
pub day_name: &'static str,
/// `ddd`.
pub day_short: &'static str,
}
impl Calendar {
/// The date of a day in this calendar, `None` when it has none: under `B2`, the 1900 system's
/// fictitious 29 February 1900 (serial 60, which Excel shows as `#VALUE!`) and days before
/// 1 Muharram 1 AH.
pub(crate) fn date(self, dt: &DateTime) -> Option<CalendarDate> {
match self {
Calendar::Hijri => hijri_date(dt),
Calendar::Gregorian => Some(gregorian_date(dt)),
}
}
}
fn gregorian_date(dt: &DateTime) -> CalendarDate {
let month_name = MONTHS.get((dt.month as usize).wrapping_sub(1)).copied().unwrap_or("");
let day_name = WEEKDAYS.get(dt.weekday as usize).copied().unwrap_or("");
CalendarDate {
year: dt.year.into(),
month: dt.month,
day: dt.day,
month_name,
month_short: month_name.get(..3).unwrap_or(month_name),
day_name,
day_short: day_name.get(..3).unwrap_or(day_name),
}
}
/// Days from 1 Muharram 1 AH to the first of a Hijri month. Leap years (30 days in the last
/// month) are years 2, 5, 7, 10, 13, 15, 18, 21, 24, 26 and 29 of each 30-year cycle (10,631
/// days), which is what Excel shows; odd months have 30 days and even months 29.
fn days_before_month(year: i64, month: i64) -> i64 {
// `(4 + 11 * year) / 30` is the number of leap years before `year`.
(year - 1) * 354 + (4 + 11 * year).div_euclid(30) + 29 * (month - 1) + month / 2
}
/// Hijri year, month and day of the day `days` after 1 Muharram 1 AH (`days >= 0`).
fn hijri_from_days(days: i64) -> Option<(i64, i64, i64)> {
if days < 0 {
return None;
}
// The estimate is the mean year length rounded; it can be one year off at a year boundary.
// Estimate and `days_before_month` both move by 30 years per 10,631 days, so one step is
// enough everywhere if it is within one cycle (`every_day_round_trips` checks many).
let mut year = (30 * days + 10646) / 10631;
if days < days_before_month(year, 1) {
year -= 1;
} else if days >= days_before_month(year + 1, 1) {
year += 1;
}
let month = ((11 * (days - days_before_month(year, 1)) + 330) / 325).clamp(1, 12);
let day = days - days_before_month(year, month) + 1;
Some((year, month, day))
}
/// 1 Muharram 1 AH is 18 July 622 (proleptic Gregorian), so that 1 January 2001 is 6 Shawwal 1421.
fn hijri_date(dt: &DateTime) -> Option<CalendarDate> {
// 29 February 1900 (serial 60) is no real day and has no Hijri date. The 1900 system's day 0
// (`1900-01-00`, serial 0) shows as 1 January 1900, as Excel shows it.
if dt.day > days_in_month(dt.year, dt.month) {
return None;
}
let (year, month, day) = hijri_from_days(days_from_civil(dt.year, dt.month, dt.day.max(1)) - days_from_civil(622, 7, 18))?;
let month_name = HIJRI_MONTHS.get(usize::try_from(month - 1).ok()?).copied()?;
// Excel shows the full Arabic name for `ddd` as well as `dddd`, as it does for `mmm`.
let day_name = ARABIC_WEEKDAYS.get(dt.weekday as usize).copied()?;
Some(CalendarDate {
year,
month: u32::try_from(month).ok()?,
day: u32::try_from(day).ok()?,
month_name,
month_short: month_name,
day_name,
day_short: day_name,
})
}
#[cfg(test)]
mod tests {
use super::*;
use gridcraft_core::DateSystem;
use gridcraft_core::date::{datetime_from_serial, max_serial};
/// Leap years of the 30-year cycle (Excel's set, per the reviewer's 43,512-serial comparison).
const LEAP: [i64; 11] = [2, 5, 7, 10, 13, 15, 18, 21, 24, 26, 29];
fn is_leap(year: i64) -> bool {
LEAP.contains(&((year - 1).rem_euclid(30) + 1))
}
fn month_len(year: i64, month: i64) -> i64 {
if month == 12 { 29 + i64::from(is_leap(year)) } else { 30 - (month + 1) % 2 }
}
/// Years 1 to 10,000 (every cycle up to and past 31 December 9999, which is 9666 AH): year,
/// month and cycle lengths follow the leap-year set.
#[test]
fn year_and_month_lengths_follow_the_leap_set() {
assert_eq!(days_before_month(1, 1), 0);
for year in 1..=10_000 {
let start = days_before_month(year, 1);
let len = days_before_month(year + 1, 1) - start;
assert_eq!(len, if is_leap(year) { 355 } else { 354 }, "year {year}");
for month in 1..=12 {
let next = if month == 12 { days_before_month(year + 1, 1) } else { days_before_month(year, month + 1) };
assert_eq!(next - days_before_month(year, month), month_len(year, month), "{year}-{month}");
}
assert_eq!(days_before_month(year + 30, 1) - start, 10_631, "cycle from {year}");
}
// One cycle: exactly the eleven leap years, 10,631 days.
let cycle: Vec<i64> = (1..=30).filter(|&y| days_before_month(y + 1, 1) - days_before_month(y, 1) == 355).collect();
assert_eq!(cycle, LEAP);
}
/// Every day from 1 Muharram 1 AH to one cycle past 31 December 9999: day -> Hijri -> day
/// round-trips, the day is within its month, and consecutive days follow each other.
#[test]
fn every_day_round_trips() {
let last = days_from_civil(9999, 12, 31) - days_from_civil(622, 7, 18) + 10_631;
let mut prev: Option<(i64, i64, i64)> = None;
for days in 0..=last {
let (y, m, d) = hijri_from_days(days).unwrap();
assert!((1..=12).contains(&m) && (1..=month_len(y, m)).contains(&d), "{days}: {y}-{m}-{d}");
assert_eq!(days_before_month(y, m) + d - 1, days, "{y}-{m}-{d}");
if let Some((py, pm, pd)) = prev {
let expected = if pd < month_len(py, pm) {
(py, pm, pd + 1)
} else if pm < 12 {
(py, pm + 1, 1)
} else {
(py + 1, 1, 1)
};
assert_eq!((y, m, d), expected, "after {py}-{pm}-{pd}");
}
prev = Some((y, m, d));
}
assert_eq!(hijri_from_days(-1), None);
}
/// Every serial of the 1900 system: the Hijri date names the serial's day, one day apart.
#[test]
fn every_serial_has_the_right_hijri_day() {
let epoch = days_from_civil(622, 7, 18);
let mut prev = None;
for serial in 1..=max_serial(DateSystem::D1900) {
if serial == 60 {
continue;
}
let dt = datetime_from_serial(DateSystem::D1900, serial as f64).unwrap();
let h = Calendar::Hijri.date(&dt).unwrap();
assert!((1..=month_len(h.year, i64::from(h.month))).contains(&i64::from(h.day)), "serial {serial}");
let days = days_before_month(h.year, i64::from(h.month)) + i64::from(h.day) - 1;
assert_eq!(days, days_from_civil(dt.year, dt.month, dt.day) - epoch, "serial {serial}");
if let Some(p) = prev {
assert_eq!(days, p + 1, "serial {serial}");
}
prev = Some(days);
}
}
}
+17 -5
View File
@@ -4,6 +4,7 @@
#![forbid(unsafe_code)]
mod builtin;
mod calendar;
mod decimal;
mod parse;
mod render;
@@ -96,6 +97,9 @@ struct Inner {
num_sections: usize,
/// Index of the section that applies to text, if any.
text_section: Option<usize>,
/// A code Excel rejects (a misplaced `B1`/`B2` calendar prefix): shown as General, and
/// `TEXT` returns `#VALUE!`, as for a code that is too long.
invalid: bool,
}
/// A parsed number format code. Cheap to clone.
@@ -130,7 +134,8 @@ fn too_long(code: &str) -> bool {
impl NumberFormat {
/// Parses a format code. Never fails: unknown or malformed parts degrade to literal text, and
/// a code longer than [`MAX_FORMAT_CODE_CHARS`] characters degrades to General.
/// a code longer than [`MAX_FORMAT_CODE_CHARS`] characters, or one Excel rejects for a
/// misplaced `B1`/`B2` calendar prefix, degrades to General.
pub fn parse(code: &str) -> NumberFormat {
Self::parse_inner(code, false)
}
@@ -147,12 +152,16 @@ impl NumberFormat {
let trimmed = code.trim();
// `checkbox` marks a checkbox cell (Insert › Checkbox): TRUE/FALSE draw as a box, any other
// value displays as General.
let general = |invalid: bool| NumberFormat {
inner: Arc::new(Inner { code: code.to_string(), general: true, sections: Vec::new(), num_sections: 0, text_section: None, invalid }),
};
if trimmed.is_empty() || trimmed.eq_ignore_ascii_case("general") || trimmed == "checkbox" || (!uncapped && too_long(code)) {
return NumberFormat {
inner: Arc::new(Inner { code: code.to_string(), general: true, sections: Vec::new(), num_sections: 0, text_section: None }),
};
return general(false);
}
let sections = parse::parse_sections(code);
if sections.iter().any(|s| s.invalid) {
return general(true);
}
let len = sections.len();
let last_is_text = sections.last().is_some_and(|s| s.kind == SecKind::Text);
let text_section = if len >= 4 {
@@ -169,7 +178,7 @@ impl NumberFormat {
} else {
len
};
NumberFormat { inner: Arc::new(Inner { code: code.to_string(), general: false, sections, num_sections, text_section }) }
NumberFormat { inner: Arc::new(Inner { code: code.to_string(), general: false, sections, num_sections, text_section, invalid: false }) }
}
pub fn general() -> NumberFormat {
@@ -487,6 +496,9 @@ pub fn text_function(v: &Value, code: &str, sys: DateSystem) -> Result<String, C
return Err(CellError::Value);
}
let fmt = NumberFormat::parse_cached(code);
if fmt.inner.invalid {
return Err(CellError::Value);
}
let n = match v.scalar() {
Value::Error(e) => return Err(e),
Value::Empty => 0.0,
+67 -2
View File
@@ -1,6 +1,7 @@
//! Format code parsing: sections, bracket tags and tokens.
use crate::FormatColor;
use crate::calendar::Calendar;
/// At most this many sections are honoured (`pos;neg;zero;text`); extra ones are ignored.
const MAX_SECTIONS: usize = 4;
@@ -24,6 +25,11 @@ pub(crate) enum DatePart {
}
impl DatePart {
/// `[h]`, `[m]` or `[s]`.
pub(crate) fn is_elapsed(self) -> bool {
matches!(self, DatePart::ElapsedH(_) | DatePart::ElapsedM(_) | DatePart::ElapsedS(_))
}
pub(crate) fn is_calendar(self) -> bool {
matches!(self, DatePart::Year(_) | DatePart::Month(_) | DatePart::Day(_))
}
@@ -122,6 +128,10 @@ pub(crate) struct Section {
pub fraction: bool,
pub currency: bool,
pub fill: bool,
pub calendar: Calendar,
/// Excel rejects the whole code because of this section: a `B` and digit that is not a
/// calendar prefix, or a prefix before a number format (`B20.00`).
pub invalid: bool,
}
/// Splits a code into sections at unquoted, unescaped, unbracketed semicolons.
@@ -251,17 +261,60 @@ struct Raw {
color: Option<FormatColor>,
cond: Option<Cond>,
currency: bool,
calendar: Calendar,
/// The section's calendar prefix as written, `'1'` or `'2'` (whether or not a locale tag
/// cancels it).
prefix: Option<char>,
/// An unquoted `B` and digit that is not a calendar prefix: Excel rejects the code.
invalid: bool,
}
/// Where a section's body starts: after its leading `[...]` tags, read as `tokenize` reads them.
/// Also whether one of those tags is a `[$...]` locale tag.
fn after_leading_brackets(cs: &[char]) -> (usize, bool) {
let mut i = 0usize;
let mut locale = false;
while cs.get(i) == Some(&'[') {
match cs.get(i + 1..).and_then(|r| r.iter().position(|&d| d == ']')) {
Some(p) => {
locale |= cs.get(i + 1) == Some(&'$');
i += p + 2;
}
None => break,
}
}
(i, locale)
}
fn tokenize(src: &str) -> Raw {
let cs: Vec<char> = src.chars().collect();
let mut raw = Raw { toks: Vec::new(), color: None, cond: None, currency: false };
let mut raw = Raw { toks: Vec::new(), color: None, cond: None, currency: false, calendar: Calendar::Gregorian, prefix: None, invalid: false };
let toks = &mut raw.toks;
// With no `]` at or after a `[`, that `[` is a literal; checking this once keeps the scan linear.
let last_close = cs.iter().rposition(|&c| c == ']');
let (body_start, locale_first) = after_leading_brackets(&cs);
// Where the next calendar prefix may stand: the body's start, or right after another prefix.
let mut prefix_at = body_start;
let mut i = 0usize;
while let Some(&c) = cs.get(i) {
match c {
// A calendar prefix, never shown: `B1` or `B2`, any case, which Excel accepts repeated
// (`B2B2`) but not mixed (`B2B1`) or with more digits (`B12`, `B02`).
'B' | 'b' if i == prefix_at && cs.get(i + 1).is_some_and(char::is_ascii_digit) => {
let d = cs.get(i + 1).copied();
let ok = matches!(d, Some('1' | '2')) && !cs.get(i + 2).is_some_and(char::is_ascii_digit) && raw.prefix.is_none_or(|p| Some(p) == d);
raw.invalid |= !ok;
raw.prefix = d;
i += 2;
prefix_at = i;
continue;
}
// Any other `B` and digit (`B3dd`, `dd/mm/yyyyB2`, `dd B2 mm`, `B2[Red]B2`) makes Excel
// reject the code.
'B' | 'b' if cs.get(i + 1).is_some_and(char::is_ascii_digit) => {
raw.invalid = true;
push_lit(toks, c.encode_utf8(&mut [0; 4]));
}
'"' => {
let mut s = String::new();
i += 1;
@@ -283,6 +336,8 @@ fn tokenize(src: &str) -> Raw {
if is_currency_char(d) {
raw.currency = true;
}
// Excel rejects an escaped `\B2` as well.
raw.invalid |= matches!(d, 'B' | 'b') && cs.get(i + 2).is_some_and(char::is_ascii_digit);
push_lit(toks, d.encode_utf8(&mut [0; 4]));
}
i += 2;
@@ -383,6 +438,11 @@ fn tokenize(src: &str) -> Raw {
}
i += 1;
}
// Excel keeps the date Gregorian when a `[$-401]`-style locale or currency tag comes before
// `B2` (`B2[$-401]` is still Hijri).
if raw.prefix == Some('2') && !locale_first {
raw.calendar = Calendar::Hijri;
}
raw
}
@@ -438,7 +498,7 @@ fn date_part(t: &Tok) -> Option<DatePart> {
}
fn analyze(raw: Raw) -> Section {
let Raw { toks, color, cond, currency } = raw;
let Raw { toks, color, cond, currency, calendar, prefix, invalid } = raw;
let is_date = toks.iter().any(|t| matches!(t, Tok::Date(_)));
let has_general = toks.iter().any(|t| matches!(t, Tok::General));
let mut sec = Section {
@@ -455,6 +515,8 @@ fn analyze(raw: Raw) -> Section {
fraction: false,
currency,
fill: toks.iter().any(|t| matches!(t, Tok::Fill(_))),
calendar,
invalid,
};
if is_date {
sec.kind = SecKind::Date;
@@ -469,6 +531,9 @@ fn analyze(raw: Raw) -> Section {
sec.kind = SecKind::Text;
return analyze_text(toks, sec);
}
// Number section. Excel rejects a calendar prefix before digits (`B20.00`, `B1#,##0`);
// `B2General`, `B2@` and `B2"x"` show as without the prefix.
sec.invalid |= prefix.is_some() && sec.has_digits;
analyze_number(toks, sec)
}
+22 -12
View File
@@ -1,8 +1,9 @@
//! Rendering a value with a parsed section.
use gridcraft_core::DateSystem;
use gridcraft_core::date::{MONTHS, WEEKDAYS, datetime_from_serial, max_serial};
use gridcraft_core::date::{datetime_from_serial, max_serial};
use crate::calendar::Calendar;
use crate::decimal::{decimal_digits, fixed, fixed_from_digits, format_general_fit};
use crate::parse::{DatePart, Section, Tok};
@@ -72,6 +73,15 @@ pub(crate) fn render_date(sec: &Section, v: f64, sys: DateSystem) -> Option<Out>
day += 1;
}
let total_ms = day.saturating_mul(86_400_000).saturating_add(ms);
// Under `B2`, Excel shows the 1900 system's fictitious 29 February (serial 60) as `#VALUE!`
// whenever a clock or calendar part is shown, even for `hh:mm` (`[h]` alone still works).
if sec.calendar == Calendar::Hijri
&& sys == DateSystem::D1900
&& day == 60
&& sec.toks.iter().any(|t| matches!(t, Tok::Date(d) if !d.is_elapsed()))
{
return None;
}
let needs_date = sec.toks.iter().any(|t| matches!(t, Tok::Date(d) if d.is_calendar()));
let dt = if needs_date { datetime_from_serial(sys, day as f64)? } else { datetime_from_serial(sys, 0.0).unwrap_or_default() };
let ampm = sec.toks.iter().any(|t| matches!(t, Tok::Date(DatePart::AmPm | DatePart::AP(..))));
@@ -80,8 +90,8 @@ pub(crate) fn render_date(sec: &Section, v: f64, sys: DateSystem) -> Option<Out>
let second = (ms / 1000) % 60;
let milli = ms % 1000;
let pad = |n: i64, w: u8| if w >= 2 { format!("{:0w$}", n, w = w.min(20) as usize) } else { n.to_string() };
let month_name = MONTHS.get((dt.month as usize).wrapping_sub(1)).copied().unwrap_or("");
let day_name = WEEKDAYS.get(dt.weekday as usize).copied().unwrap_or("");
// No date in the section's calendar (serial 60 under `B2`): shown like an out-of-range date.
let date = sec.calendar.date(&dt)?;
let mut out = Out::default();
for t in &sec.toks {
match t {
@@ -89,21 +99,21 @@ pub(crate) fn render_date(sec: &Section, v: f64, sys: DateSystem) -> Option<Out>
let s = match *d {
DatePart::Year(n) => {
if n <= 2 {
format!("{:02}", dt.year.rem_euclid(100))
format!("{:02}", date.year.rem_euclid(100))
} else {
format!("{:04}", dt.year)
format!("{:04}", date.year)
}
}
DatePart::Month(n) => match n {
1 | 2 => pad(dt.month as i64, n),
3 => month_name.chars().take(3).collect(),
5 => month_name.chars().take(1).collect(),
_ => month_name.to_string(),
1 | 2 => pad(date.month as i64, n),
3 => date.month_short.to_string(),
5 => date.month_name.chars().take(1).collect(),
_ => date.month_name.to_string(),
},
DatePart::Day(n) => match n {
1 | 2 => pad(dt.day as i64, n),
3 => day_name.chars().take(3).collect(),
_ => day_name.to_string(),
1 | 2 => pad(date.day as i64, n),
3 => date.day_short.to_string(),
_ => date.day_name.to_string(),
},
DatePart::Hour(n) => {
let h = if ampm {
+234
View File
@@ -902,6 +902,240 @@ fn closed_brackets_still_hold_their_semicolons() {
assert_eq!(f("\"[\"0;0", 5.0), "[5");
}
const JAN_1_2001: f64 = 36892.0;
#[test]
fn b1_prefix_is_hidden_and_gregorian() {
assert_eq!(f("B1dd/mm/yyyy", 46314.0), "19/10/2026");
assert_eq!(f("b1dddd, mmmm d, yyyy", 46314.0), "Monday, October 19, 2026");
assert_eq!(f("B1yyyy-mm-dd hh:mm", 46314.5), "2026-10-19 12:00");
}
#[test]
fn b2_prefix_renders_hijri_date() {
assert_eq!(f("B2dd/mm/yyyy", JAN_1_2001), "06/10/1421");
assert_eq!(f("b2d/m/yy", JAN_1_2001), "6/10/21");
assert_eq!(f("B2dddd dd/mm/yyyy hh:mm", JAN_1_2001 + 0.75), "الإثنين 06/10/1421 18:00");
assert_eq!(text_function(&Value::Number(46314.0), "B2dd/mm/yyyy", S).as_deref(), Ok("08/05/1448"));
assert_eq!(f("B2e", 46314.0), "1448");
assert_eq!(f("B2ee", 46314.0), "1448");
assert_eq!(f("B2d/m/yyyy;@", 46314.0), "8/5/1448");
}
/// Under `B2`, Excel shows Arabic weekday names, the full name for `ddd` as well as `dddd`.
#[test]
fn b2_prefix_names_weekdays_in_arabic() {
// Monday 19 and Tuesday 20 October 2026.
assert_eq!(f("B2dddd", 46314.0), "الإثنين");
assert_eq!(f("B2ddd", 46314.0), "الإثنين");
assert_eq!(f("B2dddd", 46315.0), "الثلاثاء");
assert_eq!(f("B1dddd", 46314.0), "Monday");
assert_eq!(f("B1ddd", 46315.0), "Tue");
// Every weekday has a name.
for v in 46314..46321 {
assert!(!f("B2dddd", v as f64).is_empty());
}
}
/// `B2` on a General or text section changes nothing: Excel shows the plain serial.
#[test]
fn b2_prefix_on_general_and_text_sections() {
assert_eq!(f("B2General", 46314.0), "46314");
assert_eq!(f("B2@", 46314.0), "46314");
}
#[test]
fn b2_prefix_names_hijri_months() {
assert_eq!(f("B2d mmmm yyyy", JAN_1_2001), "6 شوال 1421");
assert_eq!(f("B2mmm", 37034.0), "ربيع\u{a0}الأول");
assert_eq!(f("B2mmmm", JAN_1_2001 + 23.0), "شوال");
assert_eq!(f("B2mmmm", JAN_1_2001 + 24.0), "ذو\u{a0}القعدة");
assert_eq!(f("B2mmmmm", JAN_1_2001 + 24.0), "ذ");
}
/// All twelve month names and seven weekday names as Excel 16.0 (build 20527, en-US) shows them,
/// read with TEXT and a cell format: two-word month names are joined by a no-break space.
#[test]
fn b2_names_match_excel() {
let months = [
(46219.0, "صفر"),
(46248.0, "ربيع\u{a0}الأول"),
(46278.0, "ربيع\u{a0}الثاني"),
(46307.0, "جمادى\u{a0}الأولى"),
(46337.0, "جمادى\u{a0}الثانية"),
(46366.0, "رجب"),
(46396.0, "شعبان"),
(46425.0, "رمضان"),
(46455.0, "شوال"),
(46484.0, "ذو\u{a0}القعدة"),
(46514.0, "ذو\u{a0}الحجة"),
(46543.0, "محرم"),
];
for (serial, name) in months {
assert_eq!(f("B2dd", serial), "01", "{serial}");
assert_eq!(f("B2mmmm", serial), name, "{serial}");
assert_eq!(f("B2mmm", serial), name, "{serial}");
assert_eq!(f("B2mmmmmm", serial), name, "{serial}");
}
// Sunday 18 to Saturday 24 October 2026.
let days = ["الأحد", "الإثنين", "الثلاثاء", "الأربعاء", "الخميس", "الجمعة", "السبت"];
for (k, name) in days.iter().enumerate() {
let serial = 46313.0 + k as f64;
assert_eq!(f("B2dddd", serial), *name, "{serial}");
assert_eq!(f("B2ddd", serial), *name, "{serial}");
}
assert_eq!(f("B2yy", 46314.0), "48");
assert_eq!(f("B2y", 46314.0), "48");
assert_eq!(f("B2yyy", 46314.0), "1448");
assert_eq!(f("B2yyyyy", 46314.0), "1448");
}
#[test]
fn b2_month_and_year_rollover() {
assert_eq!(f("B2dd/mm/yyyy", 36915.0), "29/10/1421");
assert_eq!(f("B2dd/mm/yyyy", 36916.0), "01/11/1421");
assert_eq!(f("B2dd/mm/yyyy", 36974.0), "29/12/1421");
assert_eq!(f("B2dd/mm/yyyy", 36975.0), "01/01/1422");
assert_eq!(f("B2dd/mm/yyyy", 37683.0), "30/12/1423");
assert_eq!(f("B2dd/mm/yyyy", 37684.0), "01/01/1424");
}
/// Cycle years 15 (1425, leap) and 16 (1426, not leap): Excel's leap year is 15, not 16. The
/// 2 March 2005 and 29 January 2006 values were read from Excel by a reviewer; the year
/// boundaries follow from them and the 354/355-day years. The same comparison covered every
/// serial from 36,000 to 49,000 with no disagreement.
#[test]
fn b2_leap_year_is_cycle_year_15() {
assert_eq!(f("B2yyyy-mm-dd", 38413.0), "1426-01-21"); // 2005-03-02, from Excel
assert_eq!(f("B2yyyy-mm-dd", 38746.0), "1426-12-29"); // 2006-01-29, from Excel
assert_eq!(f("B2dd/mm/yyyy", 38037.0), "29/12/1424"); // 2004-02-20
assert_eq!(f("B2dd/mm/yyyy", 38038.0), "01/01/1425"); // 2004-02-21
assert_eq!(f("B2dd/mm/yyyy", 38392.0), "30/12/1425"); // 2005-02-09
assert_eq!(f("B2dd/mm/yyyy", 38393.0), "01/01/1426"); // 2005-02-10
assert_eq!(f("B2dd/mm/yyyy", 38747.0), "01/01/1427"); // 2006-01-30
// Anchors the reviewer read from Excel.
assert_eq!(f("B2dd/mm/yyyy", 46314.0), "08/05/1448"); // 2026-10-19
assert_eq!(f("B2dd/mm/yyyy", 46313.0), "07/05/1448"); // 2026-10-18
assert_eq!(f("B2dd/mm/yyyy", JAN_1_2001), "06/10/1421");
assert_eq!(f("B2dd/mm/yyyy", 61.0), "29/10/1317"); // 1900-03-01
assert_eq!(f("B2dd/mm/yyyy", 64970.0), "01/01/1501"); // 2077-11-16
assert_eq!(f("B2dd/mm/yyyy", 64971.0), "02/01/1501"); // 2077-11-17
assert_eq!(f("B2dd/mm/yyyy", 0.0), "29/08/1317");
assert_eq!(f("B2dd/mm/yyyy", 2958465.0), "03/04/9666"); // 9999-12-31
}
/// Excel honours `B2` after a colour tag, but a `[$-...]` locale tag before it keeps the date
/// Gregorian (the prefix is still hidden); `B2[$-401]` is Hijri.
#[test]
fn calendar_prefix_and_bracket_tags() {
assert_eq!(f("[Red]B2dd/mm/yyyy", 46314.0), "08/05/1448");
assert_eq!(f("[<0]0;B2dd/mm/yyyy", JAN_1_2001), "06/10/1421");
assert_eq!(f("B2[$-401]dd/mm/yyyy", 46314.0), "08/05/1448");
for code in ["[$-401]B2dd/mm/yyyy", "[$-409]B2dd/mm/yyyy", "[$-1010401]B2dd/mm/yyyy", "[$-ar-SA]B2dd/mm/yyyy"] {
assert_eq!(f(code, 46314.0), "19/10/2026", "{code}");
assert_eq!(text_function(&Value::Number(46314.0), code, S).as_deref(), Ok("19/10/2026"), "{code}");
}
assert_eq!(f("[Red][$-401]B1dd/mm/yyyy", JAN_1_2001), "01/01/2001");
// Checked in Excel 16.0 (build 20527, en-US): a currency tag cancels `B2` like a locale tag,
// a tag after the prefix does not, and a repeated prefix is accepted.
assert_eq!(f("[$€-407]B2dd/mm/yyyy", 46314.0), "€19/10/2026");
assert_eq!(f("[$$-409]B2dd/mm/yyyy", 46314.0), "$19/10/2026");
assert_eq!(f("[$-401][Red]B2dd/mm/yyyy", 46314.0), "19/10/2026");
assert_eq!(f("B2[Red]dd/mm/yyyy", 46314.0), "08/05/1448");
for code in ["B2B2dd/mm/yyyy", "B2B2B2dd/mm/yyyy", "B2b2dd/mm/yyyy", "[Red]B2B2dd/mm/yyyy"] {
assert_eq!(f(code, 46314.0), "08/05/1448", "{code}");
}
assert_eq!(f("B1B1dd/mm/yyyy", 46314.0), "19/10/2026");
assert_eq!(f("B2 dd/mm/yyyy", 46314.0), " 08/05/1448");
assert_eq!(f("B2\"x\"dd", 46314.0), "x08");
for code in ["B1\"x\"", "B2\"x\""] {
assert_eq!(f(code, 46314.0), "x", "{code}");
}
for code in ["B1General", "B1@", "0;0;0;B2@", "0;B2dd/mm/yyyy"] {
assert_eq!(f(code, 46314.0), "46314", "{code}");
}
}
/// A quoted `B2` is literal text; any other `B` and digit makes Excel reject the code (`TEXT` is
/// `#VALUE!`), as does a prefix before a number format. A rejected code shows as General. All
/// checked in Excel 16.0 (build 20527, en-US), with TEXT and with a cell format.
#[test]
fn misplaced_calendar_prefix_is_rejected() {
assert_eq!(f("\"B2\"dd/mm/yyyy", 46314.0), "B219/10/2026");
assert_eq!(f("dd/\"B2\"/yyyy", 46314.0), "19/B2/2026");
for code in [
"B0dd/mm/yyyy",
"B3dd/mm/yyyy",
"B4dd/mm/yyyy",
"B9dd/mm/yyyy",
"B02dd",
"B12dd",
"B21dd",
"B1B2dd/mm/yyyy",
"B2B1dd/mm/yyyy",
"B2[Red]B2dd/mm/yyyy",
"B2 B2dd",
"dd/mm/yyyyB2",
"dd/mm/yyyyB1",
"dd B2 mm",
"dd b2 mm",
"dd \\B2 yyyy",
"B2\\B2dd",
"B20.00",
"B2#,##0",
"B10.00",
"B1#,##0",
"0;B3dd",
] {
assert_eq!(text_function(&Value::Number(46314.0), code, S), Err(CellError::Value), "{code}");
assert_eq!(f(code, 46314.0), "46314", "{code}");
assert!(NumberFormat::parse(code).is_general(), "{code}");
}
}
#[test]
fn b2_edges_of_the_date_range() {
// Serial 60 is the 1900 system's fictitious 29 February: Excel has no Hijri date for it.
assert_eq!(text_function(&Value::Number(60.0), "B2dd/mm/yyyy", S), Err(CellError::Value));
assert_eq!(f("B2dd/mm/yyyy", 60.0), "########");
assert_eq!(f("B1dd/mm/yyyy", 60.0), "29/02/1900");
assert_eq!(f("B2dd/mm/yyyy", 59.0), "28/10/1317");
assert_eq!(f("B2dd/mm/yyyy", 61.0), "29/10/1317");
// So does any clock part (Excel 16.0, build 20527, en-US), but not an elapsed `[h]`, a literal,
// or a locale tag that cancels `B2`.
for code in ["B2hh:mm", "B2hh", "B2ss", "B2am/pm", "B2hh:mm AM/PM", "B2[$-401]hh:mm", "B2dd/mm/yyyy;\"x\""] {
assert_eq!(text_function(&Value::Number(60.0), code, S), Err(CellError::Value), "{code}");
assert_eq!(text_function(&Value::Number(60.5), code, S), Err(CellError::Value), "{code}");
}
assert_eq!(f("B2[h]", 60.0), "1440");
assert_eq!(f("B2\"x\"", 60.0), "x");
assert_eq!(f("[$-401]B2hh:mm", 60.0), "00:00");
assert_eq!(f("B1hh:mm", 60.0), "00:00");
assert_eq!(f("B2hh:mm", 61.0), "00:00");
// Serial 0 shows as serial 1.
assert_eq!(f("B2dd/mm/yyyy", 0.0), "29/08/1317");
assert_eq!(f("B2dd/mm/yyyy", 1.0), "29/08/1317");
// No upper limit short of 31 December 9999: 16 and 17 November 2077, 31 December 9999.
assert_eq!(f("B2dd/mm/yyyy", 64970.0), "01/01/1501");
assert_eq!(f("B2dd/mm/yyyy", 64971.0), "02/01/1501");
assert_eq!(f("B2dd/mm/yyyy", 2958465.0), "03/04/9666");
// In the 1904 system, serial 60 is a real day.
assert_ne!(format_value(&Value::Number(60.0), &NumberFormat::parse("B2dd/mm/yyyy"), DateSystem::D1904).text, "########");
}
#[test]
fn b2_out_of_range_does_not_panic() {
for v in [-1.0, 0.0, 60.0, 60.5, 2958465.0, 2958466.0, 1e300, f64::NAN] {
let _ = f("B2dd/mm/yyyy mmmm dddd", v);
let _ = text_function(&Value::Number(v), "B2dd/mm/yyyy mmmm dddd", S);
}
assert_eq!(f("B2dd/mm/yyyy", -1.0), f("dd/mm/yyyy", -1.0));
assert_eq!(f("B2dd/mm/yyyy", 2958466.0), f("dd/mm/yyyy", 2958466.0));
// Before 1 Muharram 1 AH there is no Hijri date (no serial reaches it).
let before_epoch = gridcraft_core::date::DateTime { year: 600, month: 3, day: 9, ..Default::default() };
assert!(calendar::Calendar::Hijri.date(&before_epoch).is_none());
}
/// "Set precision as displayed": the value each format stores. Expected values observed in Excel
/// (COM, `Workbook.PrecisionAsDisplayed = True`) with synthetic values.
#[test]
+2 -1
View File
@@ -11,7 +11,7 @@ workspace = true
[dependencies]
# The shared capture slot (`craft_ui::capture`) comes from the one craft-libs revision every
# craft-libs dependency uses (see the root Cargo.toml).
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "c5a1cbfb5f9049f9270955caffb5bee5997aebf7" }
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d" }
gridcraft-engine = { workspace = true }
gridcraft-chart = { workspace = true }
# `monochrome_emoji_fonts` restores the Noto Emoji and emoji-icon faces that 0.37 moved behind
@@ -24,6 +24,7 @@ serde_json = { workspace = true }
log = { workspace = true }
image = { workspace = true }
skrifa = { workspace = true }
unicode-bidi = "=0.3.18"
web-time = { workspace = true }
# Linux only, where winit reports no system theme: the desktop portal's light/dark answer (see
+21 -3
View File
@@ -76,7 +76,21 @@ fn emit_craft_fonts() {
/// One `CraftFont { .. }` initialiser per manifest row GridCraft may embed.
fn craft_fonts(dir: &std::path::Path, wasm: bool, required: bool) -> Result<String, String> {
let manifest = dir.join("fonts/manifest.txt");
// Every file embedded comes from inside the checkout: a manifest row (or a symlink) naming a
// path outside it would put that file of the build machine into the binary.
let root = dir.canonicalize().map_err(|e| format!("{}: {e}", dir.display()))?;
let inside = |path: std::path::PathBuf| -> Result<std::path::PathBuf, String> {
let real = path.canonicalize().map_err(|e| format!("{}: {e}", path.display()))?;
if real.starts_with(&root) { Ok(real) } else { Err(format!("{} is outside the craft-fonts checkout {}", path.display(), root.display())) }
};
// Registered before validation, by the path as given (and its folder, to see the manifest appear
// or a symlink replaced): a build that skipped the fonts must rebuild once they are there.
let manifest_path = dir.join("fonts/manifest.txt");
println!("cargo::rerun-if-changed={}", manifest_path.display());
if let Some(parent) = manifest_path.parent() {
println!("cargo::rerun-if-changed={}", parent.display());
}
let manifest = inside(manifest_path)?;
println!("cargo::rerun-if-changed={}", manifest.display());
let text = std::fs::read_to_string(&manifest).map_err(|e| format!("{}: {e}", manifest.display()))?;
let rows = craft_fonts_policy::parse_manifest(&text)?;
@@ -84,7 +98,7 @@ fn craft_fonts(dir: &std::path::Path, wasm: bool, required: bool) -> Result<Stri
if required {
craft_fonts_policy::check_required(&rows)?;
}
let chinese = if wasm { Vec::new() } else { chinese_font_rows(dir, required)? };
let chinese = if wasm { Vec::new() } else { chinese_font_rows(&root, required)? };
for row in rows.iter().chain(&chinese) {
if craft_fonts_policy::barred(&row.family) {
println!("cargo::warning=craft-fonts: not embedding {} {} (repository asset policy)", row.family, row.style);
@@ -92,7 +106,7 @@ fn craft_fonts(dir: &std::path::Path, wasm: bool, required: bool) -> Result<Stri
}
let mut out = String::new();
for row in craft_fonts_policy::embeddable(&rows, wasm).into_iter().chain(craft_fonts_policy::embeddable(&chinese, wasm)) {
let path = dir.join(&row.file).canonicalize().map_err(|e| format!("{}: {e}", row.file))?;
let path = inside(root.join(&row.file)).map_err(|e| format!("{}: {e}", row.file))?;
println!("cargo::rerun-if-changed={}", path.display());
let scripts: Vec<String> = row.scripts.iter().map(|s| format!("{s:?}")).collect();
let family = &row.family;
@@ -120,6 +134,10 @@ fn chinese_font_rows(dir: &std::path::Path, required: bool) -> Result<Vec<craft_
Ok(Vec::new())
}
};
// `dir` is the canonical checkout: the manifest itself must not lead out of it either.
if manifest.canonicalize().is_ok_and(|real| !real.starts_with(dir)) {
return optional(format!("{} is outside the craft-fonts checkout", manifest.display()));
}
match std::fs::read_to_string(&manifest) {
Ok(text) => match craft_fonts_policy::parse_manifest(&text).and_then(|rows| {
craft_fonts_policy::check_declared_scripts(&rows)?;
+1
View File
@@ -302,6 +302,7 @@ impl SheetApp {
// The requests belong to the window the command left active.
self.follow_active_window();
self.prune_selected_object();
self.prune_views();
for req in self.session.take_ui_requests() {
match req {
UiRequest::Dialog(name, params) => self.open_dialog(&name, params),
+8
View File
@@ -64,4 +64,12 @@ impl SheetApp {
self.message = Some(("GridCraft".into(), "The web version is always the latest release.".into()));
}
}
/// Starts the install helper of an update that is ready, so it installs once the process has
/// exited (a no-op on the web, and when no update is ready). Call it in `on_exit` when the
/// process ends there, since the app is then never dropped.
pub fn start_pending_update(&mut self) {
#[cfg(not(target_arch = "wasm32"))]
updates_ui::start_pending_update(self);
}
}
+12 -4
View File
@@ -7,8 +7,8 @@ impl SheetApp {
}
/// The active window and sheet: each window scrolls on its own.
fn scroll_key(&self) -> Option<(u64, usize)> {
Some((self.session.active_window()?, self.session.active()?.wb.active_sheet()))
fn scroll_key(&self) -> Option<(u64, u64)> {
Some((self.session.active_window()?, self.session.active()?.wb.active()?.uid.get()))
}
pub fn view(&self) -> SheetView {
@@ -41,8 +41,10 @@ impl SheetApp {
let window = self.session.active_window();
let Some(d) = self.session.active_mut() else { return };
let mut tops = Vec::new();
for (&(w, si), v) in &self.views {
let Some(sh) = d.wb.sheets.get(si).filter(|sh| Some(w) == window && sh.view == gridcraft_engine::model::SheetViewMode::Normal) else {
for (si, sh) in d.wb.sheets.iter().enumerate() {
let Some(v) =
window.and_then(|w| self.views.get(&(w, sh.uid.get()))).filter(|_| sh.view == gridcraft_engine::model::SheetViewMode::Normal)
else {
continue;
};
let (fr, fc) = sh.freeze.unwrap_or((0, 0));
@@ -53,4 +55,10 @@ impl SheetApp {
}
d.store_view(&tops);
}
/// Drops the scroll positions of windows that have closed: window ids are never reused.
pub(super) fn prune_views(&mut self) {
let session = &self.session;
self.views.retain(|&(w, _), _| session.window(w).is_some());
}
}
+27
View File
@@ -581,3 +581,30 @@ pub fn hit(app: &SheetApp, geo: &Geo, sh: &Sheet, p: Pos2) -> Option<(ObjRef, Re
pub(crate) fn ink_color(line: Color32, width_pt: f32) -> Color32 {
if width_pt > 8.0 { line.gamma_multiply(0.45) } else { line }
}
#[cfg(test)]
mod tests {
use gridcraft_chart::Measure;
use super::*;
/// The production measure is not monotone for labels with a line break: egui lays the text out
/// line by line, so ending a prefix with `\n…` can be narrower than the shorter `iii…`. The
/// chart's label fitting must not assume otherwise (#1152).
#[test]
fn egui_measure_shrinks_when_a_prefix_ends_in_a_line_break() {
let ctx = egui::Context::default();
ctx.set_fonts(theme::pinned_font_definitions());
let mut checked = false;
// Fonts are ready from the second pass on.
for _ in 0..2 {
let mut out = ctx.run_ui(Default::default(), |ui| {
let m = EguiMeasure { painter: ui.painter(), z: 1.0 };
let (short, long) = (m.text_width("iii…", 11.0, false), m.text_width("iiii\n…", 11.0, false));
checked = long < short;
});
out.textures_delta.clear();
}
assert!(checked);
}
}
+3
View File
@@ -24,6 +24,7 @@ pub const REQUIRED_FACES: &[(&str, &str, &[&str])] = &[
("NanumGothic", "Regular", &["Hang", "Kore"]),
("Noto Sans Symbols", "Regular", &["Zsym"]),
("Noto Sans Symbols 2", "Regular", &["Zsym"]),
("Noto Sans Arabic", "Regular", &["Arab"]),
];
/// The native-only Simplified Chinese face from craft-fonts' separate
@@ -41,6 +42,7 @@ pub const WEB_ALLOWED: &[(&str, &str)] = &[
("NanumGothic", "Regular"),
("Noto Sans Symbols", "Regular"),
("Noto Sans Symbols 2", "Regular"),
("Noto Sans Arabic", "Regular"),
];
/// One validated `fonts/manifest.txt` row. Only the fields GridCraft consumes are kept.
@@ -172,6 +174,7 @@ mod tests {
("NanumGothic", "Regular"),
("Noto Sans Symbols", "Regular"),
("Noto Sans Symbols 2", "Regular"),
("Noto Sans Arabic", "Regular"),
]
);
for (family, _) in WEB_ALLOWED {
+4 -1
View File
@@ -218,8 +218,11 @@ pub(super) fn sort(app: &mut SheetApp, ui: &mut egui::Ui, d: &mut Dialog, confir
if columns.is_empty() {
ui.add(egui::TextEdit::singleline(&mut col).desired_width(50.0)).labelled_by(label.id);
} else {
// The closed box shows the same "B: Revenue" label as the list, not the bare letter (#3446).
let shown =
columns.iter().find(|(letter, _)| letter.eq_ignore_ascii_case(&col)).map_or_else(|| col.clone(), |(_, label)| label.clone());
craft_ui::ComboBox::from_id_salt(("sort_col", i))
.selected_text(col.clone())
.selected_text(shown)
.show_ui(ui, |ui| {
for (letter, label) in &columns {
ui.selectable_value(&mut col, letter.clone(), label.as_str());

Some files were not shown because too many files have changed in this diff Show More