Compare commits

...
Author SHA1 Message Date
Dylan M. Taylor e73c4e147f chore: bump version to 0.19.1 for the Flatpak update fix 2026-11-11 14:40:23 -05:00
Dylan M. Taylor 0c5b1e85e6 sonar: stop counting the locale table as duplicated code (#3805) 2026-11-11 14:33:16 -05:00
Dylan M. Taylor 7f6a79fd98 Fix Flatpak permissions for update checks and self-updates (#3803)
* fix(flatpak): allow release checks and self-update helpers

* test: keep Flatpak permission parsing within strict Clippy rules
2026-11-11 14:24:13 -05:00
Dylan M. Taylor 6dbfbc4058 Merge pull request #3795 from storytold/chore/craft-update-6e03f65
Updates: take craft-update from craft-libs main at 6e03f65bf
2026-11-11 12:44:31 -05:00
Dylan M. Taylor e258db5f19 Updates: take craft-update from craft-libs main at 6e03f65bf 2026-11-11 12:44:25 -05:00
Dylan M. Taylor 763816d27c ci: run Harden-Runner on every job (#3786)
Adds step-security/harden-runner (audit, pinned to v2.22.4 like
craft-common-actions) as the first step of each job that did not have it,
so every workflow in the org's public repos reports its egress to
StepSecurity. Audit, not block: block-mode allow lists come from each
job's observed traffic.
2026-11-11 12:11:59 -05:00
FredApps 90180f4d87 Sandbox: build on riscv64, where libc names no kexec_file_load (#3777)
The seccomp list named libc::SYS_kexec_file_load, which libc (0.2.190) defines for x86_64 and
aarch64 but not riscv64, so the release's Linux riscv64 build stopped with E0425. riscv64 takes
the number from the kernel's generic table, 294, as aarch64 does; the test checks libc's aarch64
value against it.
2026-11-11 17:35:44 +01:00
FredApps d69a285efb Release: GridCraft v0.19.0 (#3770) 2026-11-11 16:38:52 +01:00
Dylan M. Taylor 94fc52efae xlsx: share inline dxf number formats within a styles part (#3260)
* xlsx: share each custom number format across cell formats, cap font names at 31 characters

Every cell format record copied its font (name included) and built its
own copy of its number-format code, so many short records naming one
long font or code multiplied it in memory. Custom codes are now built
once and shared, and font names are cut to Excel's 31 characters.

* xlsx: drop stray worktree entries added by mistake

* address review: compare truncated theme font for the scheme link; intern inline dxf number formats
2026-11-11 08:50:20 -05:00
Dylan M. Taylor 7b825d00be Save as Web Page: a byte budget for the charts' SVG (an error past it) (#3209)
* Charts: one point budget per chart and a byte budget for web-page SVG

A chart resolved every series reference into its own vector of up to
100,000 points, so 255 short whole-column series retained and drew
25.5 million points, and Save as Web Page wrote one SVG element per
mark without any output limit.

- resolve() shares MAX_CHART_POINTS (1,000,000) across the values,
  X values and bubble sizes of all series; a series past the budget
  gets what is left.
- chart_svg() stops writing once a chart passes the bytes left of a
  64 MiB per-page budget and leaves that chart out.

* address review: charge X values to the chart point budget only when they are kept

* address review: charge scatter Y and X values to the chart point budget as a pair
2026-11-11 08:39:25 -05:00
Dylan M. Taylor 7726e9c6ef xls: charge expanded shared-formula text to the import's text allowance (#3256)
* xls: charge expanded shared-formula text to the import's text allowance

Shared-formula followers each kept their own printed formula text, but
none of it counted against the 256 MB text allowance, so compact PtgExp
records could expand into gigabytes of retained formulas. Each attach
now charges its text; past the allowance the formula is dropped and the
cell keeps its cached value.

* address review: keep resolving array formulas after the shared-formula text allowance is spent

* address review: keep the shared-formula text cut-off across worksheets

* address review: expand shared formulas only after every sheet's mandatory text is charged

* address review: refuse duplicate sheet names at import; adapt tests to read_xls_budgeted
2026-11-11 08:28:28 -05:00
Dylan M. Taylor 9a025205bd pivot: Rank and Running Total computed once per sibling set; accumulator, caption and chart-series bounds (#3306)
* pivot: cached Rank and Running Total, accumulator-width limit, capped subtotal captions and PivotChart series

* address review: store running-total entries only for items with a number

* address review: running totals visit only the top-level items that hold the path
2026-11-11 08:16:57 -05:00
Dylan M. Taylor 01abd95420 xlsx: resolve cell pictures through a per-structure layout and a relationship index (#3255)
* xlsx: resolve cell pictures through a per-structure layout and a relationship index

Every rich value rebuilt its structure's whole key list before checking
its value count, and scanned the image relationship table for a
duplicate id. Structures are now laid out once and relationships are
indexed by id (a repeated id stays invalid), so reading cell-picture
metadata is linear in its records.

* address review: wide-structure fixture uses the right count and loads a valid picture
2026-11-11 08:02:00 -05:00
Dylan M. Taylor e5ac9474df engine: Scenario Summary size limit, bounded solver report names, undo history counts comment and link text (#3308)
* engine: bound Scenario Summary size and solver report name searches; charge comment and link text to undo history

* address review: count result rows in the scenario summary limit and reply/date vector storage in the history estimate
2026-11-11 07:40:55 -05:00
Dylan M. Taylor b21a8541e7 Row auto-fit: sweep merges once per sheet instead of per row (#3195)
* Row auto-fit: sweep merges once per sheet instead of per row

auto_row_height() picked the merges touching a row by scanning every
merge on the sheet, and opening a workbook (fit_imported_rows) or a
commit fitting many rows called it per row, so rows x merges
comparisons ran synchronously on open.

Rows are fitted in ascending order, so a MergeSweep sorts the sheet's
merges by first row once and keeps only those active on the current
row. All comparisons count against MERGE_FIT_WORK (50,000,000) per
pass; past it the remaining rows keep the heights they have.
auto_row_height keeps its signature for single rows.

* address review: share the merge-comparison budget across the whole auto-fit pass

* address review: build the merge sweep lazily, after the row fast paths

* address review: charge the merge sort to the fit budget, keep row heights once it is spent

* address review: a pass that fits one row scans the merges instead of sorting them
2026-11-11 07:26:18 -05:00
Dylan M. Taylor a6c331b63f calc: one cumulative work budget per formula, and INDEX decides multi-cell picks from their shape (#3188)
* calc: INDEX decides a multi-cell position from its shape before reading it

* calc: charge each formula's evaluation against one cumulative work budget

* address review: a database function whose computed criterion ran out of work is #CALC!

* address review: give the test's criteria cell its stored value

* address review: stop matching database records once a computed criterion exhausts the work budget

* address review: skip the criteria-row scan once work is exhausted

* address review: stop between conditions once the work budget is spent
2026-11-11 07:17:04 -05:00
Dylan M. Taylor 12d26d53cc xlsx: analyse each kept chart sheet's graph once per save (#3261)
* xlsx: analyse each kept chart sheet's graph once per save

chart_sheet_usable walked a chart sheet's relationship graph and parsed
its charts for every workbook entry, so many entries naming one shared
graph multiplied the work. The referenced sheet names are now cached by
relationship id.

* address review: cache the chart-sheet usability verdict per relationship id instead of every reference

* address review: cache chart-sheet analysis by the graph root, not the relationship id
2026-11-11 07:06:18 -05:00
Dylan M. Taylor 9dc7e3826a numfmt: TEXT returns #VALUE! when its result passes a cell's 32,767 characters (#3264)
* numfmt: TEXT returns #VALUE! when its result passes a cell's 32,767 characters

A text section repeats its input once per @ placeholder, and TEXT handed
back whatever that built, so nested TEXT calls could multiply a long
text until memory ran out. Results past the cell text limit are now
#VALUE!.

* address review: stop TEXT rendering past the cell limit and count UTF-16 units
2026-11-11 06:44:34 -05:00
Dylan M. Taylor f79a425ca9 xlsx: find a reused picture through a content hash instead of comparing every media part (#3214)
* xlsx: find a reused picture through a content hash instead of comparing every media part

Out::image_part compared each new picture byte by byte with every
media part written so far, so a workbook of many distinct pictures
sharing a long prefix was exported in quadratic time. Media parts are
now indexed by extension and SHA-256.

* address review: index preserved media parts when they are added

* address review: do not hash an image twice when it is first added
2026-11-11 06:32:44 -05:00
Dylan M. Taylor 76e53085e3 xlsx: keep the cached value of a follower whose shared-formula copy is refused (#3212)
* xlsx: one hyperlink allowance per sheet, and a copy limit for unparsable shared masters

Every <hyperlinks> element of a worksheet started a fresh cell and byte
allowance although all of them fill the same map. The sheet now keeps
one allowance across its elements.

Followers of a shared master the parser refuses each copied its full
text; a sheet full of followers of a long master multiplied that text
without bound. The copies are now charged against a 64 MiB per-sheet
limit, past which the followers keep their cached values.

* address review: keep the cached value of a follower dropped for the shared-formula copy limit inside array ranges
2026-11-11 06:15:08 -05:00
Dylan M. Taylor bc2dabc7ea FORMULATEXT and the in-cell editor keep Hidden formulas hidden (#3298)
* FORMULATEXT and the cell editor keep a hidden formula's source hidden

* address review: recalculate formulas when sheet protection changes

* address review: skip the full recalculation for a no-op unprotect

* address review: recalculate on protect/unprotect without marking the edit structural
2026-11-11 05:48:31 -05:00
Dylan M. Taylor e14f783503 Analysis ToolPak: stop building a report once it outgrows the output limit (#3207)
* Analysis ToolPak: stop building a report once it outgrows the output limit

Report builders filled their whole output map before write_report
checked its size, so Descriptive Statistics grouped by rows over a
tall, nearly empty column allocated about 16 entries for each of up
to a million variables before refusing the report.

Grid now tracks its extent as values are stored and keeps nothing
once the report's rows x columns pass MAX_OUTPUT_CELLS; write_report
refuses such a report. extent() no longer rescans every entry.

* address review: stop the descriptive builder once the grid is too large

* address review: descriptive statistics reads variables one at a time
2026-11-11 05:40:31 -05:00
Dylan M. Taylor 542aa176ce Wildcards: match * and ? patterns in one pass over the text (#3205)
* Wildcards: match * and ? patterns in one pass over the text

After a mismatch the matcher restarted the pattern after the last
star at the next text position, so a long repetitive cell against a
near-matching pattern cost text x pattern comparisons. AutoFilter
custom criteria, Find and Replace all call it once per cell.

wildcard() now runs the pattern as a bit-parallel automaton: one pass
over the text, one step per 64 pattern positions for each character.
Results are unchanged; a test compares it with the old matcher on
every short text and pattern over a small alphabet.

* address review: build wildcard literal masks lazily with a bounded cache

* address review: keep masks of the most frequent wildcard literals past the cache cap
2026-11-11 05:19:24 -05:00
Dylan M. Taylor 67a0b7f325 Security: chart rasterising, spline points and category labels stay bounded (#3204)
* chart: bound scan conversion, spline points and category label bytes

- The rasteriser tested every edge of a path at every sub-sample of every
  covered row. It now keeps an active-edge list (edges sorted by top, joining
  and leaving as the scan passes them), and one rasterize call has a total
  scan-work allowance after which remaining fills are skipped.
- Smoothed lines budgeted interpolated points per run, so gap-separated short
  runs escaped both decimation and the cap. The budget is now shared by all
  runs of a line; once spent, runs are drawn through their data points.
- Category labels copied every text value in full. Labels are clipped to 255
  characters before copying, within an 8 MiB budget per series.

* address review: chart raster and label budget fixes

* address review: bound polyline point count before stroke_paths
2026-11-11 05:07:04 -05:00
Dylan M. Taylor bbe0825404 Security: TEXTJOIN builds its result piece by piece within the text cap (#3202)
* functions: TEXTJOIN builds its result piece by piece within the text cap

TEXTJOIN converted every input and delimiter into an owned String before
joining, so many references to one long shared text copied gigabytes before
the joined length was ever checked. Errors are now found first without
building text (so they still win in argument order), then each piece and
delimiter is converted only as it is appended, stopping at the cap.

* address review: check the cap before copying each text piece
2026-11-11 04:55:21 -05:00
Dylan M. Taylor aa4951fdf4 Security: WORKDAY, PIVOTBY, SORT and the D-functions stop rescanning their input (#3200)
* functions: stop WORKDAY, PIVOTBY, SORT and the D-functions rescanning their input

- WORKDAY/WORKDAY.INTL counted the holidays each step passed by scanning every
  holiday, once per step; a run of consecutive holidays made that quadratic.
  The working-day holidays are kept sorted and counted by binary search.
- PIVOTBY totals rescanned the whole source (rebuilding keys) once per total
  row and column. The grouping pass now also indexes rows by row key and by
  column key, and totals read from those indexes in the same order.
- SORT copied one key per sort_index entry for every row, so a long
  sort_index with repeated columns multiplied the source. Repeats can't break
  a tie their first use left, so only first uses are kept.
- DSUM and the other database functions tested every record against every
  criteria row. A selection whose records x conditions exceed 5e8 is #NUM!,
  and each criteria label's database column is found once, not per cell.

* address review: bounds-check SORT indices before narrowing; empty criteria rows are not charged
2026-11-11 04:36:48 -05:00
Dylan M. Taylor 79fe7b8eb6 Forecast Sheet: count hindcast rows against the work limit (#3199)
* Forecast Sheet: count hindcast rows against the work limit

MAX_WORK bounded history rows x forecast rows, but a Forecast Start
inside the history adds a hindcast row for every history point after
it, each holding the same ETS formulas that refit the model over the
history before the start. Those rows were not counted, so a start in
the middle of 10,000 rows of history built millions of uncharged
model fits per recalculation.

The forecast row cap is now computed from the rows the model is fit
to, after subtracting the hindcast rows; a start so early that the
hindcast alone passes the limit is refused, and the default number of
periods stays within the cap.

* address review: base the in-history forecast cap on the slots the fit range spans

* address review: document the hindcast-aware forecast work limit

* address review: document the fitted-row minimum in the forecast work limit

* address review: say the forecast start row counts toward the hindcast work limit
2026-11-11 04:22:54 -05:00
Dylan M. Taylor ff9f4b8ace Find and Replace: bound the text a replacement builds (#3194)
* Find and Replace: bound the text a replacement builds

Each occurrence copied the whole replacement string, so a cell full of
short matches and a long replacement expanded tens of kilobytes of
input into about a gigabyte, in both the case-sensitive and the
case-insensitive paths.

Replacement now builds the new text only while it stays within
Excel's 32,767 characters per cell (counted as UTF-8 bytes, 4 each at
most) and within 256 MiB for the whole command, checking before each
copy. Past either limit the command fails and changes nothing.

* address review: enforce the 32,767-character limit on replacements and borrow the replacement text
2026-11-11 04:02:22 -05:00
Dylan M. Taylor 5ce63e5fc2 Get Data: refuse an import whose dense rows would be too large (#3193)
* Get Data: refuse an import whose dense rows would be too large

data.getData and data.fromTextCsv built every row densely before
writing it: JSON records were padded to the union of all keys (found
with a quadratic Vec::contains scan), and CSV cells were expanded over
the whole used rectangle. A few sparse records or one far-away CSV
cell therefore allocated billions of slots.

The importer now collects JSON keys with a HashSet, refuses more keys
than a sheet has columns, and checks the projected rows x columns
against MAX_IMPORT_CELLS (10,000,000) before building any row.

* address review: stop collecting JSON keys at the column limit and reject oversized row counts before building rows

* address review: enforce the CSV import cell limit while parsing
2026-11-11 03:48:32 -05:00
Dylan M. Taylor b7fd198e06 Conditional formatting: one scan budget per rule, no thresholds for unsupported colour scales (#3201)
* Conditional formatting: one scan budget per rule, and no thresholds for unsupported colour scales

numbers_in() scanned each of a rule's ranges separately with its own
2,000,000-cell limit, so a rule listing the same (or overlapping)
large range many times multiplied the scan while painting a single
cell. Colour scales also evaluated every stop's threshold, formulas
included, before discarding scales that are not 2- or 3-colour.

- A rule's statistics now share STATS_SCAN_LIMIT across all its ranges,
  and an exact repeat of a range is scanned once.
- A colour scale with a stop count other than 2 or 3 paints nothing
  and evaluates none of its thresholds.

* address review: count stored and spilled cells of sparse ranges, skip statistics for unsupported colour scales

* address review: skip unrelated spills a row at a time in the statistics scan
2026-11-11 03:18:38 -05:00
Valentin KleinandClaude Sonnet 5.5 55f3004cb2 Numfmt: one built-in format table, fewer allocations rendering numbers and General (#817) (#820)
* Numfmt: one built-in format table, no dead render arms, fewer allocations (#817)

S1: the built-in format table lives only in gridcraft-xlsx (file formats do not
depend on numfmt); numfmt's unused copy and its exports are removed, xlsx gets
tests pinning its lookup rules.
S3: drop the render_number arm that printed what the fallback prints; literal
text is appended to the output instead of built as a String per call.
O3: render_number keeps pieces, text-before and literal replacements in one
buffer (Parts), fill_int writes digits left to right, Section stores the date
flags (sub-second digits, calendar, AM/PM) computed at parse time, render_date
writes into one buffer.
O4: significant_digits formats into a stack buffer; format_general_fit re-rounds
the 15 digits per width into reused strings.

An equality test compares format_value, TEXT() and format_general_fit with
digests recorded from the previous implementation.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* xlsx test: builtin ids 42 and 44 are written as custom formats like 5-8

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* numfmt: push_str for short month/day names (clippy)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 08:57:06 +01:00
FredApps 022074344b Merge pull request #2196 from Higgs32584/fix/2170-shape-font-attr-escape
xlsx: escape shape font typeface attribute (#2170)
2026-11-11 08:12:56 +01:00
FredApps ac4fb45c3f Merge pull request #2326 from nihilist-narwhal/fix/2164-mcp-stdio-line-bound
MCP: the stdio server keeps at most 16 MB of a message (#2164)
2026-11-11 08:12:47 +01:00
FredApps 3d52a20dbd Merge pull request #379 from vangel76/fix/lookup-blank-result
Functions: lookups keep an empty target cell blank (#374)
2026-11-11 08:12:38 +01:00
FredApps 4cc2e21d7c Merge pull request #3382 from vangel76/fix/3321-manage-rules
Manage Rules: scroll the rule list instead of laying out every rule
2026-11-11 08:11:07 +01:00
FredApps 088b43be28 Merge branch 'main' into fix/2170-shape-font-attr-escape 2026-11-11 07:58:02 +01:00
FredApps 942f639fbf Merge branch 'main' into fix/2164-mcp-stdio-line-bound 2026-11-11 07:57:52 +01:00
FredApps a01ff3e7b3 Merge branch 'main' into fix/lookup-blank-result 2026-11-11 07:57:42 +01:00
FredApps b79e992576 Merge branch 'main' into fix/3321-manage-rules 2026-11-11 07:57:35 +01:00
Pranav 0fed26a14f Sandbox: a crate that runs a helper program confined (Landlock and seccomp on Linux, sandbox-exec on macOS)
## Stack

The Python cells work is split by layer. It merges in this order, each with a merge commit rather than a squash. Nobody can switch Python on before #3481.

1. #3475 depends on `main`
2. #3476 depends on `main` **(this pull request)**
3. #3477 depends on [#3475](https://github.com/storytold/gridcraft/pull/3475) and [#3476](https://github.com/storytold/gridcraft/pull/3476)
4. #3478 depends on [#3477](https://github.com/storytold/gridcraft/pull/3477)
5. #3479 depends on [#3478](https://github.com/storytold/gridcraft/pull/3478)
6. #3480 depends on [#3479](https://github.com/storytold/gridcraft/pull/3479)
7. #3481 depends on [#3480](https://github.com/storytold/gridcraft/pull/3480)

## Summary

`crates/sandbox` runs a program that can read only the paths its policy lists, write nothing except `/dev/null`, start no process, and use no network. An application re-executes itself with a launcher flag, closes every inherited descriptor above standard error, applies the confinement, checks that it holds, and only then runs the program. If any of that fails, the program does not run.

## Changes

- **`crates/sandbox`:** the policy, the Linux backend (Landlock, and seccomp that denies all sockets, with an x32 filter), the macOS backend (`/usr/bin/sandbox-exec` with a deny-default profile), the probes and a canary.
- **`apps/gridcraft`, `apps/gridcraft-cli`:** the launcher flag comes first in `main`, before any window or log.
- **`gridcraft-cli sandbox-check`:** says whether this system confines programs, and can run a confined self-test.
- **`docs/sandbox.md`:** what is refused and how.
- **`nix`:** a new dependency on Linux and macOS, for closing descriptors without `unsafe`.

## Compatibility

Windows and FreeBSD have no backend here. There the crate reports that it can't confine, and nothing that needs it runs. On Linux a launch needs Landlock ABI 3, which is Linux 6.2 and later, so an older kernel is treated the same way: this excludes, for example, Debian 12's kernel (6.1) and Ubuntu 22.04's original kernel (5.15). Nothing changes for a process started without the launcher flag.

## Review fixes

- **File metadata on Linux.** Landlock has no right for a file's permissions, owner, extended attributes, times or flags, and the seccomp filter allows what it doesn't name, so a confined program could change them on any file its user owns, outside the policy too. The filter now refuses the `chmod`, `chown`, `setxattr`, `removexattr` and timestamp families with their `*at` variants, by path and by descriptor, and any `ioctl` whose number says it hands the kernel data, as `FS_IOC_SETFLAGS` and `FS_IOC_FSSETXATTR` do on a descriptor opened only to read. A filter allows what it doesn't name, so a call a later kernel adds is allowed until it is named; the newest named is `file_setattr` (Linux 6.17).
- **Truncation on Linux.** Landlock covers truncation only from ABI 3, so on ABI 1 or 2 `os.truncate(path, 0)` emptied any file its user can write, and `open` with `O_RDONLY | O_TRUNC` emptied one it may read. A launch now requires that right. A filter can't take its place: `openat2` hides its flags from seccomp, and refusing `O_TRUNC` would refuse opening `/dev/null` to write, as `open(os.devnull, "w")` does.
- **Standard streams.** They stay open with the rights they were opened with, so a stream that is a file can be written and truncated, and one that is a socket can still send. The page and `launch_command` say so, and that a parent gives the program pipes.
- **The enforcement check, again.** It now also tries to change the canary's permissions and to truncate it, asking for what it already has, so nothing changes if one works.
- **Other processes on Linux.** On a kernel without Landlock's signal scope, a confined program could signal any process of the same user, and `kill(-1, 9)` would reach them all. The seccomp filter now refuses signals to any process but the program itself, whose ID `exec` keeps; changes to the limits, priority or scheduling of any other process; `pidfd_open`, `pidfd_send_signal`, `kcmp`, `get_robust_list`, `migrate_pages` and `move_pages`; every System V shared memory, message queue and semaphore call and every POSIX message queue call, which no path rule covers; and `syslog`.
- **The terminal on Linux.** A program started from a shell could type into its terminal. `TIOCSTI` and `TIOCLINUX` are refused, compared in 32 bits as the kernel reads them, so high bits set on the argument don't get past the rule.
- **The enforcement check.** It now also tries signal 0 on the launcher's parent, which delivers nothing, so `sandbox-check` reports a gap rather than assuming none. The launcher tries the same signal before it confines itself and refuses if that fails, and the check refuses if its parent has changed since, since a parent that refuses anyone would prove nothing. A launcher's arguments must include the current executable, which that check starts; otherwise the file rules refused the start and the check proved nothing about new processes.
- **A refused launch.** The launcher read its arguments only after closing every inherited descriptor, so a refused launch had already closed them. It reads them first.
- **A kernel without Landlock.** The refusal says so in that case too, which the tests that skip on such a machine look for. Only the first ABI's rights being refused, or `ENOSYS` or `EOPNOTSUPP` from creating the ruleset, says so. Any other failure, such as `EINVAL` or too many layers, is reported as what it is and doesn't skip the tests.
- **Folders in the exec list on macOS.** A folder got a rule that matches only the folder, so a launcher under it failed to start where the same policy works on Linux. It now covers what is in it, in both rules.
- **The sandbox page** names only what this layer has: only `gridcraft-cli sandbox-check` starts the launcher here.

## Testing

All 10 steps of `cargo xtask ci` pass on macOS (arm64), and `cargo clippy` passes for this crate for both Linux architectures. The gate on ubuntu-24.04 is the hosted one on this pull request, and it executes the Linux backend through the real binary, including `a_confined_program_leaves_other_processes_alone` and `a_confined_program_changes_no_file`, in which a confined Python tries each refused call on a file inside the policy, one outside it and a descriptor, and the test then compares contents, permissions and modification time.

A separate run on the fork, on Linux 7.0 with Landlock ABI 8, covered the same commit's code with parts of the fix taken out. Without the metadata rules, the launch check refuses with `change the canary's permissions: it worked`. Without those rules or the new checks, `a_confined_program_changes_no_file` fails with `permissions of ...: it worked`. With the Landlock rights limited to ABI 2, as on an older kernel, it fails with `truncate ...: it worked`. A kernel that is actually older than ABI 3, or that has no Landlock at all, has not been run. On macOS, terminal input was checked by hand on a pseudo-terminal: `TIOCSTI` is refused when confined and works when not.

The commits group the change by area for reading. Only the last commit of the branch is expected to build.


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
  * Added sandbox support for running commands with policy-limited file access and restrictions on network and process operations. The sandbox is available on supported Linux and macOS systems; launches are refused if confinement cannot be established or verified.
  * Added the `sandbox-check` command to check sandbox support and enforcement, or run a specified program in the sandbox. It reports when sandboxing is unsupported.
* **Documentation**
  * Added guidance on sandbox behavior, platform support, access restrictions, launch checks, and `sandbox-check` usage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-11 07:31:42 +01:00
Dylan M. Taylor 6a08240eb6 io: refuse a FIFO before opening it; keep the replaced file's group on save (#3313)
* io: refuse a FIFO before opening it; keep the replaced file's group on save

* address review: create the replacement owner-only, then restore group and mode; never widen it when the group cannot be restored

* address review: keep the old owner and other bits when the group cannot be restored
2026-11-10 18:05:44 -05:00
Dylan M. Taylor a3a98cf745 functions: REGEXREPLACE keeps only the requested occurrence's captures (#3288)
* REGEXREPLACE keeps only the requested occurrence's captures

* address review: do not truncate REGEXREPLACE occurrence on 32-bit targets
2026-11-10 18:05:25 -05:00
Dylan M. Taylor fe39331ac4 engine: Consolidate finds keys through hash maps and refuses an oversized table (#3285)
* engine: Consolidate finds keys through hash maps and refuses an oversized table

data.consolidate searched the keys collected so far for every row and
column key, so 100,000 positional rows cost about five billion
comparisons. It also built every row × column combination without a
size check. Keys are now indexed, and a table over 5 million cells is
refused before it is built.

* address review: count label cells in the consolidate output limit
2026-11-10 18:05:08 -05:00
Dylan M. Taylor a84259d58c ui-egui: embed craft-fonts files only from inside the checkout (#3281)
* ui-egui: embed craft-fonts files only from inside the checkout

The build hook canonicalised each manifest row's file and embedded it
with include_bytes! wherever it led, so an absolute path, a .. path or
a symlink in a craft-fonts checkout could put any readable file of the
build machine into the binary. Manifests and font files must now
resolve inside the canonical checkout directory.

* address review: register the manifest path before validation and keep the Chinese manifest fallback optional
2026-11-10 18:04:49 -05:00
Dylan M. Taylor 85d32c20be engine: show overlong format codes as General without hashing them per cell (#3273)
* engine: show overlong format codes as General without hashing them per cell

The display cache hashed the full format code on every lookup, so one
huge code shared by many cells cost its length for every cell drawn.
A code with more than 4 × 255 bytes can't be within Excel's 255
characters and is now formatted as General straight away.

* address review: keep the original code in the public number_format for overlong formats
2026-11-10 18:04:31 -05:00
Dylan M. Taylor 1deb108a7c xtask: probe keytip commands without file access, and refuse text that could escape generated code (#3269)
* xtask: probe keytip commands without file access, and refuse text that could escape generated code

cargo xtask keytips ran every command keytips.json names, with its
params, under the user's file permissions (file.exportPdf could replace
any file, also during --check). It also wrote target, excel and note
strings into generated Rust without escaping. Probes now run inside the
engine's ScriptSandbox, and those fields may not hold quotes,
backslashes, braces or control characters.

* address review: make ScriptSandbox !Send so it drops on the thread that entered it
2026-11-10 18:04:13 -05:00
Dylan M. Taylor 2c9e2d1eec Go To: index defined names when resolving and listing them (#3206)
Go To resolved every visible name, and each alias inside a name was
looked up with Workbook::name, a linear scan of all names. A workbook
of many short aliases made the Go To list quadratic (or worse along
alias chains) in the number of names.

A Resolver now indexes the names by scope and folded spelling once
per command and remembers, per name, sheet and depth, whether a name
resolves and to how many areas, so listing names resolves each alias
once. Lookups keep the old precedence and matching rules.
2026-11-10 18:03:53 -05:00
Dylan M. Taylor 9c8b4a37ea PDF export: index merged areas when drawing a block (#3203)
draw_block checked every cell against every merge meeting the block,
and the fill, border and text passes searched the merge list again
for each merge start, so an imported sheet with many merge ranges in
a print area cost cells x merges (and merges x merges) comparisons.

The merges meeting a block are now gathered once and swept down the
rows: whether a cell is covered is a BTreeMap lookup among the merges
active on its row, and merge starts are found through a HashMap. A
merge overlapping one kept before it (only possible in a damaged file)
is left out, as Excel does when it repairs the file. Hyperlink
rectangles use the same index instead of scanning all sheet merges.
2026-11-10 18:03:35 -05:00
FredAppsandFredApps a36e4987c1 i18n(uk): translate Chart as Діаграма (#3703)
Use the shared craft-i18n Ukrainian term, distinct from Diagram (Схема).
Also translate the remaining Chart-labelled strings that were still English.

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 17:07:10 +01:00
FredAppsandFredApps 457bc7e374 Naming: use Diagram and Decorative Text in the feature catalog (#3622)
* Naming: use Diagram and Decorative Text in the feature catalog

* Naming: align the Chinese decorative text term

* test(naming): guard neutral display labels

* Naming: Diagram no longer repeats the chart word in de, uk and ro

German Schema and Ukrainian Схема match craft-i18n#211; Romanian Schemă
avoids GridCraft's own chart word Diagramă. A catalog test checks every
shipped language translates both labels and that Diagram differs from
Chart/Charts. The display-name guard now handles escaped char literals
and any #[cfg(test)] item, asserts it scanned catalog.rs and a minimum
of files and literals, and tests word boundaries. The catalog test
asserts each ID appears once. Docs carry timestamps and revision history.

---------

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 16:13:13 +01:00
Harrison Booth 50a9e9ede4 XLS: formulas open as legacy formulas with Excel's implicit @ (#410) (#1367)
.xls formulas are read as legacy formulas with Excel's implicit @ (apply_legacy via read_xls_with), one-cell Ctrl+Shift+Enter arrays keep their range, @ of a 2-D range meets the formula's row and column, and defined-name analyses are shared across the formulas of a pass and replayed exactly.

Closes #410
2026-11-10 15:05:02 +01:00
74ab5cc969 M12: regional format data (Region, 20 regions) (#3048)
* M12: regional format data (Region, 15 regions, Workbook::region)

Adds the data model for Excel-style regional formats, with no behaviour
change: nothing reads it yet.

- core: region.rs with `Region` (separators, grouping, date order, short and
  long date and time codes, month and day names, AM/PM words, currency) and
  the 15 regions en-US, en-GB, en-IN, es-ES, de-DE, fr-FR, it-IT, pt-BR,
  pt-PT, ru-RU, cs-CZ, ja-JP, ko-KR, zh-CN, zh-TW. Values follow Unicode CLDR;
  the deliberate departures (four-digit short-date year, Windows grouping
  spaces) are listed in the module doc. `all`, `find` (strict tag),
  `lookup` (system or browser tag, best effort) and `default_region` (en-US)
  select one; `EN_US` reproduces today's behaviour exactly. Unit tests check
  the tables and the lookups.
- model: `Workbook::region`, a `#[serde(skip)]` runtime field that is never
  saved and defaults to en-US. It is here so that the following changes can
  be reviewed independently of each other.

The region is independent of the interface language and of the formula
language. Builds on this, in order: typed input (core `parse_input_in`),
display (numfmt and engine display), the engine session and `app.region`
command, then the Options UI and the app defaults.

* M12: region data: review fixes (tag extensions, zh territory, month periods)

- lookup stops at an extension or private-use singleton:
  en-GB-u-ca-gregory is en-GB (ca was read as Canada) and
  zh-Hant-u-nu-latn is zh-TW.
- For Chinese an explicit territory wins over the script, so
  zh-Hant-CN is China's conventions, consistent with zh-Hans-TW.
- In regions whose month abbreviations end in a period (ru-RU, fr-FR),
  month_from_name accepts any abbreviation typed with one (мар., июн.);
  elsewhere a period stays outside the name, so en-US is unchanged.

* Region: document where dates and month names follow Windows, not CLDR

* M12: Swiss regions (en-CH, de-CH, fr-CH, it-CH) and nl-NL

The Swiss regions group digits with an apostrophe (CLDR 48) and accept the
typographic one glibc uses; lookup maps CH and LI tags to them. Dutch
(Netherlands) follows CLDR 48.

* Region: table-driven lookup test, under main's cognitive-complexity limit

* Region: French and Italian Liechtenstein tags resolve to the Swiss regions

* Region: credit the Unicode CLDR data under the Unicode License v3 (NOTICE, ATTRIBUTION.md)

* Region: month_from_name takes one word; the currency format has the currency's own decimals

---------

Co-authored-by: quantmind-br <dsrodrigues20@gmail.com>
Co-authored-by: quantforge-br <218010578+quantforge-br@users.noreply.github.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-10 13:54:53 +01:00
RockerHXandFredApps ec12eba83e UI.1265: align CJK fallback text with the Latin baseline (#3081)
* UI.1265: align CJK fallback text with the Latin baseline

Use epaint's existing baseline alignment for system CJK fallback faces
and embedded faces tagged with CJK scripts. Mixed CJK, Latin and digits
then share a baseline in cells, editors and window titles.

Add regression coverage for zoom, font styles, wrapping, fixed-height
clipping, vertical alignment, editor height and title truncation.

Refs #1265, #2621.

* UI.1265: assert title baselines and document visual validation

* UI.1265: add baseline comparison and review validation

* UI.1265: adapt tooltip regression after upstream merge

The upstream shortcut update uses the runtime OS to choose tooltip text.
Use the harness OS for the existing split-button test expectation too.

Validation: cargo xtask ci; all 10 steps pass; 3,002 tests passed, 1 ignored, no exclusions.

* UI.1265: propagate CJK fallback to installed fonts

* UI.1265: align fallback baselines across cells

* UI.1265: align wrapped and unwrapped cell text

* UI.1265: preserve blank lines in wrapped cell alignment

* UI.1265: preserve centered fallback placement

* UI.1265: find CJK cell text by its visible bounds

Use the overlap of text ink, clip rect and cell rect to locate painted text
instead of testing a fixed offset from the galley origin. Baseline-aligned
fallbacks can move that origin above the cell on Ubuntu.

Keep the existing baseline and wrapping assertions unchanged.

Validation: cargo xtask ci on macOS; all 3 cjk_baseline tests on Ubuntu 24.04 x86_64.

---------

Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-10 12:44:19 +01:00
ca39b5ab9f XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel (#417)
* XLSX: legacy formulas (no dynamic-array cm) open with Excel's implicit @ and calculate like Excel; implied @ saved back without cm

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* XLSX: long operator chains analysed in full and unfinished legacy analysis leaves the formula unchanged (never saved as SINGLE); @ of a union keeps its parentheses; IF's condition intersected inside SUMPRODUCT

* XLSX: a legacy union already in parentheses saves back without a second pair (never as SINGLE); a name's definition analysed as an array throughout, inserting no @

* XLSX: drop the unreachable keep-unions retry; an @ of a bare union saves as a dynamic formula

* XLSX: an @ keeps parentheses the reader would not put back (redundant ones, a union's)

* XLSX: a redundant @ keeps a legacy formula legacy; IFERROR and IFNA intersected inside SUMPRODUCT; file parentheses and unfinished analyses saved as read

An edit could leave an implied @ on a single cell: deleting rows 2:3 under the legacy SQRT(A1:A3), read as SQRT(@A1:A3), gives SQRT(@A1:A1), and a name redefined to one cell does the same to SQRT(@Rng). The writer kept an @ only when re-analysing the formula without it put the same @ back, which it no longer does for one cell, so the formula was saved as SQRT(_xlfn.SINGLE(A1:A1)) with cm, readable only by dynamic-array Excel. The writer now analyses the formula with its own @ in place: an @ is accepted when the analysis implies it at that position or when its operand is a single value there (such an @ changes nothing), and any @ inserted elsewhere or any other @ still makes the formula dynamic. SQRT(@A1:A1) saves as SQRT(A1:A1), SQRT(@A1) as SQRT(A1) and @SUM(A1:A3) as SUM(A1:A3), all without cm; SUMPRODUCT(@SQRT(A1:A3)) stays dynamic because SQRT(A1:A3) is an array there.

Legacy Excel intersects IFERROR's and IFNA's arguments inside SUMPRODUCT unless the formula is entered with Ctrl+Shift+Enter, like IF's condition: =SUMPRODUCT(IFERROR(VALUE(C19:C21),0)) returns one row's value or #VALUE!, and the accepted fix is CSE or N(+range) (https://stackoverflow.com/questions/59228707). They now get the same single-value parameter class as IF's condition (Param::Single), so the legacy SUMPRODUCT(IFERROR(A1:A3/A1:A3,0)) reads as SUMPRODUCT(IFERROR(@A1:A3/@A1:A3,0)), and a typed SUMPRODUCT(IFERROR(VALUE(A1:A3),0)), which dynamic-array Excel evaluates as an array, is saved with cm. IF's result arguments are left as arrays: in https://stackoverflow.com/questions/29682328, a non-CSE =SUMPRODUCT(--(B4=IF(NOT($C$4:$C$11),$B$4:$B$11))) in row 4 returns 2 (both "apple" rows of B4:B11 counted, so the result branch was not intersected), 0 in a row where C is TRUE and #VALUE! outside rows 4:11 (the condition was intersected).

The reader's @ in front of a whole-formula array result took no extra parentheses when the file already had them, so ({1,2}+1) and {1,2}+1 both read as @({1,2}+1), and the writer dropped the file's parentheses. An operand already in parentheses now gets a second pair, so ({1,2}+1) saves as written.

A legacy formula the analysis could not finish (node budget, nesting) was read unchanged, evaluated as a dynamic-array formula and saved with cm, so Excel spilled it. Names were analysed again at every use, so a 399-term name used six times exhausted the 4096-node budget; their kinds are now remembered per formula, and the budget is 16384 nodes. The nesting limit is 128, so Excel's 64 levels of nested functions are analysed. A formula that still exceeds the limits is saved as it was read, without cm.

* XLSX: an implied @ is dropped and any other @ of a single value kept as _xlfn.SINGLE in a legacy formula (as Excel writes it); a legacy formula the analysis cannot finish or whose semantics it does not know opens with a whole-formula @ and never spills; analysis depth follows the parser's limit

* XLSX: a legacy union given to a function's single-value parameter keeps its implied @, so it calculates to #VALUE! instead of spilling and saves as read; checked on a calculated workbook

* XLSX: only a one-cell range an @ applies to counts as one an edit shrank; a reading past the parser's nesting limit leaves the formula as read, and the writer saves the text a formula it cannot analyse was read from

* XLSX: a formula nested past the parser's limit for its reading is left as read (documented: it may spill); test covers it

* XLSX: a name's definition counts its one-cell ranges as collapsed only when it is itself a reference, so CHOOSE's index or IF's condition in it keeps the name legacy

* XLSX: an @ on a name or one-cell range counts as implied when ranges an edit may have shrunk, anywhere the name returns them (CHOOSE's choices, parentheses), make it so; deleting rows keeps such formulas legacy

* XLSX: the collapsed-range retry of an @ runs only when the first pass reached a one-cell range written as a range (tracked through cached names)

* docs: describe unfinished formula export fallback

* docs: the unfinished-analysis fallback differs for formulas with an explicit @

* Preserve main formula analyzer fixes

* Order file-format revision history

---------

Co-authored-by: neilcawse <8085959+neilcawse@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Local contributor <contributor@localhost>
2026-11-10 12:43:07 +01:00
e9addf52fe Arabic bidi layout and cell reading direction in the grid and editors (#947)
* M12.AR4: preserve Arabic logical text in bidi display and editing

* M12.AR4: drop the stale "no bidi" note from the RTL sheet status

* M12.AR4: open the Arabic editors and place the caret through key and pointer events in tests

* M12.AR4: skip the conditional-format lookup in the formula bar unless a rule sets reading order

* M12.AR4: test that the caret after typed Arabic is drawn at the text's left end (#1070)

* M12.AR4: pass the theme body font in the reading-order style test

* M12.AR4: adapt the Arabic UI tests to egui 0.37 (row direction, editor ids, Ctrl+End to the logical end)

* M12.AR4: read the active sheet through Workbook::active_sheet() in the formula bar

main made `Workbook::active_sheet` an `ActiveSheet` (shared between windows on one
workbook); the formula bar's reading-order lookup takes the index from the accessor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR4: keep explicit Left on wrapped right-to-left cells and trailing spaces in the editor

Wrapped cell text lays out every row from the left when the painted alignment is Left, instead of aligning right-to-left rows to their start. The cell and formula bar editor keep trailing whitespace in their layout so a right-to-left entry ending in a space keeps the caret inside the text.

* M12.AR4: keep Right and General on right-to-left sheets aligned right in wrapped cells

* M12.AR4: keep right-aligned Arabic with a marked first letter inside its cell

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-09 15:55:43 +01:00
cda767b49b Arabic font fallback in cells, the formula bar and the web (#931)
* M12.AR2: Arabic font fallback in cells and the interface (#229)

* M12.AR2: reserve the preferred Arabic file only once it has loaded

load_fallback treated fontconfig's Arabic file as already loaded even
when load_preferred_arabic could not load it, so a system Arabic face
from the same file was skipped and the chains could end up with no
Arabic system face. The file is now reserved only when the preferred
face is in the font set; the candidate loop moves to load_faces so a
test can drive it with a fixture file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR2: say where fontconfig's Arabic choice applies

fontconfig's Arabic face leads the Arabic fallbacks, so it draws Arabic
where the UI or cell font has none. A UI font with Arabic of its own,
such as DejaVu Sans in the Flatpak, still draws interface Arabic (#229),
which is left for a follow-up. The docs and the test's doc comment no
longer say the choice comes first in every chain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-09 15:54:55 +01:00
c35234e5f5 Number formats: honour the B1/B2 calendar prefixes (Hijri dates for B2) (#1918)
* M12.AR6: honour the B1/B2 calendar prefixes in number formats (Hijri for B2)

* M12.AR6: match Excel's B1/B2 behaviour measured in review

Changes from a reviewer's Excel run (TEXT and Range.NumberFormat):
- A [$-...] locale tag before B2 keeps the date Gregorian (prefix still
  hidden); B2[$-401] stays Hijri.
- Any other unquoted B and digit (B3dd, dd/mm/yyyyB2, dd B2 mm), and B2
  before a number format (B20.00), make the code invalid: TEXT gives
  #VALUE!, a cell shows General, as for a code over 255 characters.
- Under B2, ddd and dddd show the Arabic weekday name (full name for both).
- Serial 60 has no Hijri date: TEXT gives #VALUE!, a cell shows ####.
- Serial 0 shows as serial 1 (29/08/1317).

Also fixes a break from merging main: days_from_civil now returns 0 for
day 0 and 29 February 1900, which sent serials 0 and 60 to 1970
(23/10/1389) under B2. The Hijri day count no longer passes those dates.

Docs: localization-parity states the behaviour, including the one
measured case not matched (mmmm next to dddd prints the weekday in Excel).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR6: Hijri leap year in cycle year 15, not 16, as Excel shows

A reviewer compared Excel's B2yyyy-mm-dd on 43,512 serials with this
formula: it disagreed by one day on 1,372, all in cycle years 15 and 16,
and moving the leap year from 16 to 15 left no disagreement. The leap
years are now 2, 5, 7, 10, 13, 15, 18, 21, 24, 26 and 29 (still 10,631
days a cycle).

The count of leap years before a year becomes (4 + 11y) div 30; it
differs from the old (3 + 11y) div 30 only when y = 16 (mod 30), which is
exactly the move from 16 to 15. The inverse's year estimate can now be one
year off at a boundary and is corrected by one step.

Tests: Excel values for 2 March 2005 (1426-01-21) and 29 January 2006
(1426-12-29) plus the 1425/1426 year boundaries they imply, the reviewer's
other anchors, and exhaustive checks: year, month and 30-year cycle
lengths for Hijri years 1 to 10,000; day -> Hijri -> day round trip and
day-by-day succession for every day from 1 Muharram 1 AH to one cycle past
31 December 9999; every 1900-system serial 1 to 2,958,465 (except 60).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR6: match B1/B2 details checked in Excel 16.0 build 20527

Compared GridCraft with Excel 16.0 (build 20527, en-US) through COM: TEXT
with B2dd/mm/yyyy on every serial from 1 to 2,958,465 (no difference), and
about 1,750 format/value pairs through both a cell's NumberFormat and TEXT.
Fixed what differed and is in scope:

- Two-word Hijri month names use a no-break space (U+00A0), as Excel does.
- B1 before a number format is rejected like B2 (B10.00, B1#,##0).
- Prefix runs: a repeated prefix is accepted (B2B2, B2b2, [Red]B2B2, B1B1);
  a mixed one (B1B2, B2B1), more digits (B12, B21, B02), a prefix after a
  tag or space (B2[Red]B2, B2 B2) and an escaped \B2 are rejected.
- Serial 60 under B2 is #VALUE! for any date or clock part (hh:mm, ss,
  AM/PM), not only date parts; [h] and literal-only sections still show.

Regression tests use the Excel values (all 12 month and 7 weekday names,
the accepted and rejected codes, serial 60). Not changed, listed in the
docs: Excel's weekday/month-name mix-ups when ddd(d) and mmm(m) share a B2
section, weekday-only B2 sections (real weekday for serials 0-59, #VALUE!
for the last days of 9999), and Buddhist-year b/bb/bbbb tokens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-09 15:54:05 +01:00
004523c286 AutoFit: estimate Arabic text width from measured glyph widths (#1933)
* M12.AR12: estimate Arabic text width from measured glyph widths in AutoFit

* M12.AR12: treat U+0897–089F as marks and U+08E2 as visible; size the honorific ligatures (review)

* M12.AR12: size the remaining honorific ligatures like their siblings (review)

U+FD4C-FD4F (Unicode 14) and U+FDC8-FDCF (FDCF Unicode 14, FDC8-FDCE
Unicode 17) are honorific word ligatures, not letters, but fell into
the 0.48 em letter arm. Give them the 1.6 em of the measured
U+FD40-FD4B honorifics and assert each range's endpoints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR12: size Arabic-Indic digits like ASCII digits and the remaining honorific ligatures (review)

Arabic-Indic (U+0660-0669) and Extended Arabic-Indic (U+06F0-06F9)
digits measure 0.53-0.57 em in Noto Sans Arabic, Arial, Tahoma and
Times New Roman, the same as each font's ASCII digits, so they get
the ASCII digit estimate (0.56) instead of the letter estimate.
Arabic punctuation measures 0.17-0.50 em, so it keeps 0.48 rather
than falling back to 1 em.

U+FBC3-FBD2 and U+FD90-FD91 are honorific word ligatures like
U+FD40-FD4F and get the same 1.6 em.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-09 15:52:55 +01:00
517a4b9256 Sort: order Arabic text by its letters, not its vowel marks or digit script (#1940)
* M12.AR7: sort Arabic text by its letters, not its vowel marks or digit script

* M12.AR7: order marked Arabic text the way Excel does

Checked compare_text against Excel 16.0 build 20527 (en-US) via COM:
Range.Sort, =SORT() and the < / = operators over ~300 strings. Fixes the
mismatches inside this change's scope:

- The tie-break compared raw code points, so an RLM-prefixed word sorted
  after every vowelled form of it, "‏12" after "١٢", and a tatweel
  form after a shadda form. Ignorable characters are now skipped at the
  tie-break too.
- Marks are compared letter by letter (an unmarked letter first), so a
  mark on an earlier letter outranks the digit or letter after it, as in
  Excel ("ب١" < "بَ1", "بپ" < "بَپ").
- Marks rank in Excel's measured order (sukun before shadda), and the
  Quranic marks Excel skips (U+0610–061A, U+06D8–06ED except 06DD/06DE/
  06DF/06E0/06E5/06E6/06E9) are skipped by letter order.
- Excel also ignores bidi embeddings/overrides/isolates, BOM, soft hyphen,
  CGJ and word joiner; they now are ignored too (ZWSP is not, as in Excel).

Equality is unchanged. Regression tests use the orders Excel produced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR7: compare the mark level of Arabic text without allocating

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>
2026-11-09 15:49:26 +01:00
vangel76andClaude Opus 5.5 642fa97fa5 fix(functions): a blank element of a lifted argument is given, not omitted
CEILING.MATH(6.7,VLOOKUP({"k","k"},A1:B1,2,FALSE)) with B1 empty now gives 0 per
element, as the scalar call does, instead of treating the significance as left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 14:19:05 +01:00
vangel76andClaude Opus 5.5 63150810b3 fix(functions): a blank lookup given as an argument is not an omitted one
FILTER(1,FALSE,VLOOKUP(...)) with a blank lookup result returned #CALC!
because util::has read the blank value as a left-out argument. Arguments
written in the formula that compute a blank are now marked as given.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 13:27:08 +01:00
vangel76andClaude Opus 5.5 dae5b52d78 Merge upstream/main into fix/lookup-blank-result
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:55:54 +01:00
vangel76andClaude Opus 5.5 684f051f0d Manage Rules: test reaching rules by scrolling down and sideways
Scroll the 3,000-rule list to a row that was off screen and move and
delete that rule through its own buttons; give one rule a label of 200
ranges whose buttons start outside the dialog, scroll sideways, then move
and delete it. Each test checks the intended rule changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:51:27 +01:00
vangel76andClaude Opus 5.5 91bd22635e Merge upstream/main into fix/lookup-blank-result
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:23:30 +01:00
vangel76andClaude Sonnet 5.5 019033e095 Merge upstream/main into fix/3321-manage-rules
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:37:04 +01:00
vangel76andClaude Sonnet 5.5 d7c0158f0a Manage Rules: let the rule list scroll sideways
A long range list pushed the move and Delete buttons out of the vertical-only
scroll area, where they could not be clicked.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:08:44 +01:00
vangel76andClaude Sonnet 5.5 51e242e1fa Manage Rules test: check the dialog rect and which rule was deleted
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 00:52:15 +01:00
vangel76andClaude Sonnet 5.5 8c1c7cfd27 Manage Rules: scroll the rule list and lay out only the visible rules (#3321)
The dialog laid out one row per rule with no height limit and ran
home.manageRules (copying and serialising every rule, and journaling a
step) on every frame. A sheet with thousands of rules made the window
as tall as the whole list and the application window go blank.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 00:47:47 +01:00
Michael Higgins 94eed2bd83 Merge branch 'storytold:main' into fix/2170-shape-font-attr-escape 2026-11-03 10:10:26 -05:00
nihilist-narwhalandClaude Opus 5.5 e7cda6337b MCP: the stdio server keeps at most 16 MB of a message (#2164)
`serve` read a whole line before checking it against `MAX_LINE`, so a client sending a
newline-free payload grew the buffer without limit. The line is now read through `take`, as the
control server does; the rest of an over-long line is read and dropped, the client gets the
"message too large" error, and the next message is served (stdio has one client, so the stream
stays open rather than closing as the control server's connections do).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 07:49:34 -06:00
Michael Higgins c31e4af95c Merge branch 'storytold:main' into fix/2170-shape-font-attr-escape 2026-10-27 11:17:04 -04:00
Michael Higgins 793420b849 Merge branch 'storytold:main' into fix/2170-shape-font-attr-escape 2026-10-26 18:00:13 -04:00
Higgs32584andClaude Opus 5.5 4ab3edaf3b xlsx: escape shape font typeface attribute (#2170)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:06:33 -04:00
9977d850a4 test(calc): Excel-verified blank results for VLOOKUP, HLOOKUP and LOOKUP (#374)
Excel 365 for Windows (2609, build 16.0.20430.20032) keeps a lookup of an
empty cell blank inside ISBLANK and concatenation for VLOOKUP, HLOOKUP and
LOOKUP. The cases were checked in Excel by scottcampbelldata in
vangel76/gridcraft#1.

Co-Authored-By: scottcampbelldata <251750033+scottcampbelldata@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 15:23:42 +02:00
vangel76andClaude Sonnet 5.5 d446fde718 Merge upstream/main into fix/lookup-blank-result
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-13 19:05:24 +02:00
vangel76andClaude Sonnet 5.5 423810750d Functions: don't assert ISBLANK(VLOOKUP/HLOOKUP/LOOKUP), unverified against Excel (#374)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 19:18:01 +02:00
vangel76andClaude Opus 5.5 d2b9affbc5 Functions: test that arithmetic reads a blank lookup result as 0 (#374)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:17:48 +02:00
vangel76andClaude Opus 5.5 691883f99c Functions: VLOOKUP/HLOOKUP/LOOKUP/XLOOKUP keep an empty target blank (#374)
A looked-up empty cell was turned into the number 0 inside the function,
so ISBLANK saw FALSE and & / CONCATENATE inserted "0". The value now stays
blank, like INDEX; a bare lookup still displays 0 as a formula's final result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:17:48 +02:00
179 changed files with 12768 additions and 1083 deletions
+4
View File
@@ -27,6 +27,10 @@ jobs:
name: Windows installation policy (no app build)
runs-on: windows-latest
steps:
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
+1
View File
@@ -12,6 +12,7 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra
| `epaint_default_fonts::HACK_REGULAR` (dependency font input) | Source Foundry Authors (Hack; DejaVu and Bitstream Vera foundations) | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Hack](https://github.com/source-foundry/Hack) | MIT + Bitstream Vera License (full upstream texts in `NOTICE`) | Hack-Regular.ttf is egui's bundled monospace face and GridCraft's last-resort symbol fallback. Dependency asset unmodified here, no font file copied into this repository. |
| `epaint_default_fonts::NOTO_EMOJI_REGULAR` (dependency font input) | The Noto Project Authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Noto Emoji](https://fonts.google.com/noto/specimen/Noto+Emoji) | SIL Open Font License 1.1 (full upstream text in `NOTICE`) | NotoEmoji-Regular.ttf is enabled by egui's `monochrome_emoji_fonts` feature and used as the monochrome emoji fallback. Dependency asset unmodified here, no font file copied into this repository. |
| `epaint_default_fonts::UBUNTU_LIGHT` (dependency font input) | Canonical Ltd. and the Ubuntu Font Family authors | [egui 6d518fc](https://github.com/storytold/egui/tree/6d518fc799681571963842f95b55a6b68431b15a/crates/epaint_default_fonts/fonts), [Ubuntu font](https://design.ubuntu.com/font) | Ubuntu Font Licence 1.0 (full upstream text in `NOTICE`) | Ubuntu-Light.ttf is egui's default proportional face. Dependency asset unmodified here, no font file copied into this repository. |
| Regional format data in `crates/core/src/region.rs` (data, no file of its own) | Unicode, Inc. (Unicode CLDR) | [cldr-json](https://github.com/unicode-org/cldr-json) (`cldr-numbers-full`, `cldr-dates-full`) and ICU's CLDR 48 data | Unicode License v3 (`Unicode-3.0`; full text in `NOTICE`) | Separators, month and weekday names, AM/PM words, currency symbols and date order of the built-in regions. No CLDR file is copied into this repository. |
| `docs/brand/artcraft-logo.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms (`docs/brand/LICENSE-brand.txt`) | Trademark, not open source |
| `docs/brand/artcraft-logo.png` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark |
| `docs/brand/artcraft-logo-white.svg` | ArtCraft Team | ArtCraft brand kit | ArtCraft brand terms | Trademark |
Generated
+60 -24
View File
@@ -910,12 +910,12 @@ dependencies = [
[[package]]
name = "craft-fonts-build"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
[[package]]
name = "craft-i18n"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"fixed_decimal",
"icu_decimal",
@@ -938,7 +938,7 @@ dependencies = [
[[package]]
name = "craft-print"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"craft-winprint-gdi",
"log",
@@ -952,7 +952,7 @@ dependencies = [
[[package]]
name = "craft-ui"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"craft-fonts-build",
"craft-i18n",
@@ -972,7 +972,7 @@ dependencies = [
[[package]]
name = "craft-update"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"chrono",
"dunce",
@@ -982,6 +982,7 @@ dependencies = [
"image",
"log",
"plist",
"rustix 1.1.5",
"semver",
"serde",
"serde_json",
@@ -1018,7 +1019,7 @@ dependencies = [
[[package]]
name = "craft-window"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"egui",
"objc2 0.6.5",
@@ -2022,7 +2023,7 @@ dependencies = [
[[package]]
name = "gridcraft"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"arboard",
"chrono",
@@ -2034,6 +2035,7 @@ dependencies = [
"getrandom 0.4.3",
"gridcraft-engine",
"gridcraft-mcp",
"gridcraft-sandbox",
"gridcraft-ui-egui",
"image",
"log",
@@ -2049,7 +2051,7 @@ dependencies = [
[[package]]
name = "gridcraft-calc"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-core",
"gridcraft-formula",
@@ -2061,7 +2063,7 @@ dependencies = [
[[package]]
name = "gridcraft-chart"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-calc",
"gridcraft-core",
@@ -2072,10 +2074,11 @@ dependencies = [
[[package]]
name = "gridcraft-cli"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-engine",
"gridcraft-mcp",
"gridcraft-sandbox",
"log",
"quick-xml 0.41.0",
"serde_json",
@@ -2084,7 +2087,7 @@ dependencies = [
[[package]]
name = "gridcraft-core"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"chrono",
"js-sys",
@@ -2095,7 +2098,7 @@ dependencies = [
[[package]]
name = "gridcraft-engine"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-calc",
"gridcraft-chart",
@@ -2118,7 +2121,7 @@ dependencies = [
[[package]]
name = "gridcraft-formula"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-core",
"serde",
@@ -2127,7 +2130,7 @@ dependencies = [
[[package]]
name = "gridcraft-functions"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-core",
"regex",
@@ -2135,7 +2138,7 @@ dependencies = [
[[package]]
name = "gridcraft-math"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"ratex-font",
"ratex-layout",
@@ -2145,7 +2148,7 @@ dependencies = [
[[package]]
name = "gridcraft-mcp"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-engine",
"log",
@@ -2154,7 +2157,7 @@ dependencies = [
[[package]]
name = "gridcraft-model"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-core",
"gridcraft-formula",
@@ -2167,18 +2170,29 @@ dependencies = [
[[package]]
name = "gridcraft-numfmt"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-core",
]
[[package]]
name = "gridcraft-pdf"
version = "0.16.0"
version = "0.19.1"
[[package]]
name = "gridcraft-sandbox"
version = "0.19.1"
dependencies = [
"landlock",
"libc",
"nix",
"seccompiler",
"thiserror 2.0.21",
]
[[package]]
name = "gridcraft-ui-egui"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"craft-ui",
"craft-update",
@@ -2197,6 +2211,7 @@ dependencies = [
"skrifa",
"sys-locale",
"toml",
"unicode-bidi",
"web-sys",
"web-time",
"zbus",
@@ -2204,7 +2219,7 @@ dependencies = [
[[package]]
name = "gridcraft-web"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"craft-web-api",
"craft-web-host",
@@ -2223,7 +2238,7 @@ dependencies = [
[[package]]
name = "gridcraft-xlsx"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"aes",
"gridcraft-core",
@@ -2872,6 +2887,17 @@ dependencies = [
"smallvec",
]
[[package]]
name = "landlock"
version = "0.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cca98e95f35b29d469dade6724c6f96cec9236640f745a0e99b0334ec320ab1"
dependencies = [
"enumflags2",
"libc",
"thiserror 2.0.21",
]
[[package]]
name = "lazy_static"
version = "1.5.1"
@@ -4543,6 +4569,15 @@ dependencies = [
"tiny-skia",
]
[[package]]
name = "seccompiler"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4ae55de56877481d112a559bbc12667635fdaf5e005712fd4e2b2fa50ffc884"
dependencies = [
"libc",
]
[[package]]
name = "security-framework"
version = "3.7.0"
@@ -6485,7 +6520,7 @@ dependencies = [
[[package]]
name = "winprint"
version = "0.2.1"
source = "git+https://github.com/storytold/winprint.rs?rev=51934db382f225454cf152a8657eb03789be63c4#51934db382f225454cf152a8657eb03789be63c4"
source = "git+https://github.com/storytold/winprint.rs?rev=bc06a7c48b768eb27a828c7986f6d2868b245ee0#bc06a7c48b768eb27a828c7986f6d2868b245ee0"
dependencies = [
"flate2",
"fmt-derive",
@@ -6500,6 +6535,7 @@ dependencies = [
"windows",
"windows-core",
"windows-implement",
"windows-numerics",
"xml",
]
@@ -6674,7 +6710,7 @@ checksum = "e450f9b2ed1dff33c94c12589a87338689467b9c4f5d8a5710bd09a847d2c8a7"
[[package]]
name = "xtask"
version = "0.16.0"
version = "0.19.1"
dependencies = [
"gridcraft-engine",
"serde_json",
+4 -3
View File
@@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
default-members = ["crates/*", "apps/gridcraft", "apps/gridcraft-cli", "xtask"]
[workspace.package]
version = "0.16.0"
version = "0.19.1"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.95"
@@ -14,6 +14,7 @@ repository = "https://github.com/storytold/gridcraft"
# internal
gridcraft-core = { path = "crates/core" }
gridcraft-numfmt = { path = "crates/numfmt" }
gridcraft-sandbox = { path = "crates/sandbox" }
gridcraft-formula = { path = "crates/formula" }
gridcraft-functions = { path = "crates/functions" }
gridcraft-model = { path = "crates/model" }
@@ -28,7 +29,7 @@ gridcraft-math = { path = "crates/math" }
# Every craft-libs crate (craft-update and craft-window here, craft-ui in ui-egui, craft-print in
# the app) comes from one craft-libs revision.
# craft-update: app-specific pretend versions apply to startup and manual checks.
craft-update = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d", version = "=0.1.0", features = ["egui"] }
craft-update = { git = "https://github.com/storytold/craft-libs", rev = "6e03f65bf45181db77ba6229716e6624db9eb25b", version = "=0.1.0", features = ["egui"] }
# foundation
thiserror = "2"
@@ -71,7 +72,7 @@ skrifa = { version = "0.44", default-features = false, features = ["std"] }
epaint_default_fonts = "0.37"
# One native window per workbook: the window list, Window menu titles and the immediate-viewport
# driver. Same craft-libs revision as craft-update above.
craft-window = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d", default-features = false, features = ["egui"] }
craft-window = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "6e03f65bf45181db77ba6229716e6624db9eb25b", default-features = false, features = ["egui"] }
[workspace.lints.rust]
unsafe_code = "forbid"
+49
View File
@@ -275,6 +275,55 @@ DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER
DEALINGS IN THE FONT SOFTWARE.
The regional format data in crates/core/src/region.rs (separators, month and weekday names, AM/PM
words, currency symbols and date order) follows the Unicode Common Locale Data Repository (CLDR),
read from unicode-org/cldr-json and from ICU's CLDR 48 data, under the Unicode License v3.
Source: https://github.com/unicode-org/cldr-json
The following text is reproduced verbatim from cldr-json's LICENSE:
UNICODE LICENSE V3
COPYRIGHT AND PERMISSION NOTICE
Copyright © 2015-2024 Unicode, Inc.
NOTICE TO USER: Carefully read the following legal agreement. BY
DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR
SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT
DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE.
Permission is hereby granted, free of charge, to any person obtaining a
copy of data files and any associated documentation (the "Data Files") or
software and any associated documentation (the "Software") to deal in the
Data Files or Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, and/or sell
copies of the Data Files or Software, and to permit persons to whom the
Data Files or Software are furnished to do so, provided that either (a)
this copyright and permission notice appear with all copies of the Data
Files or Software, or (b) this copyright and permission notice appear in
associated Documentation.
THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
THIRD PARTY RIGHTS.
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE
BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES,
OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA
FILES OR SOFTWARE.
Except as contained in this notice, the name of a copyright holder shall
not be used in advertising or otherwise to promote the sale, use or other
dealings in these Data Files or Software without prior written
authorization of the copyright holder.
SPDX-License-Identifier: Unicode-3.0
Linux Wayland window decorations
-------------------------------
sctk-adwaita 0.10.1 (https://github.com/PolyMeilex/sctk-adwaita,
+12 -4
View File
@@ -2,7 +2,7 @@
**Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away
> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-02 · **Change:** minor (form controls: buttons, sliders and spin buttons; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis take ranges by selection on the grid; earlier: Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365)
> **Last reviewed:** 2026-11-01 · **Last updated:** 2026-11-05 · **Change:** minor (CJK fallback text shares the Latin baseline; earlier: form controls: buttons, sliders and spin buttons; earlier: RefEdit reference boxes in dialogs: Goal Seek, Solver and Data Analysis take ranges by selection on the grid; earlier: Python mode in the formula bar; earlier: Python cells stored and kept in files; earlier: equations: Insert › Equation, editor, gridcraft-math; earlier: What-if and analysis: Solver Sensitivity report for Simplex LP and GRG Nonlinear; earlier: floating objects: one selection, eight resize handles, keyboard delete; earlier: Chart data tables: live source values beneath column, bar, line, area and compatible combo charts; legend keys and borders; XLSX round-trip and print/PDF; earlier: Formulas › Function Library as labelled buttons with category menus; Recently Used and Insert Function categories; XLSX workbook-view selection, bounded tab ratios and sheet-edit regressions; readiness estimates unchanged; earlier: CSV and text open detects each column's date order, so day-first dates are not silently swapped) · **Target:** Microsoft Excel (Microsoft 365)
GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX
is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web.
@@ -43,8 +43,7 @@ long `.xls` track, so about half parallelizes (≈30–50 wall-clock hours with
Why alpha: all six core workflows in the [alpha gate](docs/roadmap.md#alpha-gate) pass end to end on macOS with save and reopen, and the core workflow works end to end (open an XLSX, enter formulas and data, format, sort
and filter, chart, pivot, print to PDF, save XLSX), but users still hit blocking gaps on real
files: 100 MB workbooks don't open (#175), there's no `.xls` (#121), Arabic and Devanagari text
isn't shaped (Chinese text still reported in #142), charts lose their type and options on round trip, and
files: 100 MB workbooks don't open (#175), there's no `.xls` (#121), Devanagari text needs shaping verification (Chinese text still reported in #142), charts lose their type and options on round trip, and
nobody has yet confirmed that Excel opens our output without a repair prompt across a real
corpus. Beta needs those closed (see [beta gates](docs/roadmap.md#beta-gates)).
@@ -105,7 +104,7 @@ Localized German formulas and numbers are in PR #163. Detail: [localization-pari
| Simplified Chinese | zh-Hans | menus only | ~12% |
| Spanish | es | none | 0% |
| Hindi | hi | none | 0% |
| Arabic | ar | none (no RTL layout, no shaping) | 0% |
| Arabic | ar | catalog pending; shared shaping/bidi works | 0% |
| French | fr | none | 0% |
| Portuguese | pt-BR | menus, File view, dialogs | ~43% |
| Indonesian | id | none | 0% |
@@ -149,6 +148,9 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-31 | CSV and text open detects each column's date order: a value that reads only day first (`31/05/2026`) makes the column day first, a value that reads only month first (`12/31/2026`) makes it month first, an all-ambiguous column keeps month first with a warning, and a column with both kinds stays text with a warning (#2765) |
| 2026-10-31 | Recently Used lists the functions most recently entered in a formula from any source (ribbon, control channel, MCP, scripts), kept with the preferences and returned by `formulas.recentlyUsed`; the Insert Function dialog takes Excel's title and a working category list (Most Recently Used, All, each category) |
| 2026-10-31 | Formulas › Function Library as a big Insert Function button, the AutoSum split button and three columns of labelled small buttons, each category book in its own colour; category menus end with Insert Function… on that category; More Functions cascades into the remaining categories; Home › Editing shares the AutoSum menu; the MR key tip opens Recently Used |
| 2026-10-31 | System and embedded CJK fallback text shares the Latin baseline in cells, editors and window titles (#1265, #2621). Symbol and emoji alignment, manual row heights and clipping are unchanged |
| 2026-10-26 | Arabic bidi display/editing preserve logical source text; per-cell reading direction is editable and survives undo, XLSX and conditional-format priority |
| 2026-10-26 | Arabic text shows in cells and the interface through a system, fontconfig or craft-fonts Arabic face; workbook fonts keep their script fallbacks |
| 2026-10-25 | AutoSum over a selection does what Excel does: a selected row gets one total to its right (in its last cell if that is empty), not a one-cell total under each cell; a block whose last column is empty gets each row's total there, one whose last row is empty each column's, and with both the corner gets the total of the block; otherwise totals go under each column as before |
| 2026-10-24 | Macro interface ([macros.md](docs/macros.md#in-the-window)): an Automate › Macros group (Record Macro, which turns into Stop Recording, Use Relative References, Run Macro with recent macros); "Recording · N steps" with Stop in the status bar; a Record Macro dialog (checked name, free Ctrl+Shift shortcut, description); the Macros dialog lists macros beside Automate scripts and a VBA project, with Run, Step Into, Edit, Create, Delete and Options; Ctrl+Shift+letter macro shortcuts; questions that offer Run, Overwrite, Shift Down or Shift Right; a step editor with a breakpoint gutter, drag to reorder, a command picker, reference boxes that switch between relative tokens and absolute references, Inputs and Output tabs, a debug bar (Check, Run, Step, Continue, Run to Here, Stop with keep or undo) and grid overlays for inputs, writes, conflicts and the current step |
| 2026-10-24 | Go To lists the named ranges of every sheet (`Sheet2!Name` for a name scoped to another sheet; hidden names, constants and formulas left out) and the references entered before, a double-click goes to an entry, and the Reference box takes several names or references separated by commas (`Price, Type`, `B14:C22,F19:G30`) |
@@ -197,6 +199,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-13 | XLSX: `[Content_Types].xml`, `_rels/.rels`, the workbook rels and `xl/workbook.xml` are the first zip entries (document properties last), so libmagic and PHP `finfo` detect the file type even with large properties (#1090) |
| 2026-10-13 | Conditional formatting: a rule gathers range statistics only when it reads them (top/bottom, above average, duplicates, scales, bars, icons) and borrows them instead of copying them per cell, so the per-frame lookup of an expression or cell-value rule no longer visits its whole range (#1098, part 1) |
| 2026-10-12 | Next/previous sheet navigation stops at the first/last visible worksheet instead of activating a hidden boundary sheet; the current selection is retained |
| 2026-10-12 | Number formatting: one table of built-in formats (xlsx); numbers, dates and General render with one scratch buffer instead of per-token strings; output unchanged, checked against digests recorded from the old code (#817) |
| 2026-10-21 | Macro engine ([macros.md](docs/macros.md)): typed, declarative macros saved in the workbook (a JSON field; hidden names in xlsx); declared inputs checked with messages that name the cells; tokens (`Input.col(2)`, `Row`, `Output`, `Name:Total`, `{{…}}` in formulas); one undo step with full rollback; a footprint dry run with ask, fail, overwrite, shift down or shift right; a policy that keeps macros away from files and the app, with approval for macros in opened files; Repeat, If and Stop steps; a debugger; a relative recorder; `macro.*` commands beside the unchanged Automate scripts |
| 2026-10-13 | Charts: work and memory are bounded by series x points (2,000,000 per chart: every series keeps its first points; 16 trendlines per chart), hairline columns and bars are merged per pixel column, series ranges are clipped before they are evaluated, the moving average is linear and rendering clean data no longer copies it (#1147) |
| 2026-10-13 | Charts: one combo plot-kind rule for resolving and drawing; ticks of a scale off the step grid keep their own values (#1146) |
@@ -264,6 +267,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| Date | Change | Summary |
|---|---|---|
| 2026-11-06 | trivial | Progress log: scroll positions follow their sheet through sheet inserts, deletes and moves (#3485) |
| 2026-11-05 | minor | Reconcile upstream form-control progress with the CJK baseline update |
| 2026-11-02 | minor | Form controls; catalog 277 / 292 → 281 / 296 (Developer › Controls); progress log row |
| 2026-11-02 | minor | Progress log: RefEdit reference boxes in dialogs |
| 2026-11-02 | minor | Progress log: Python mode in the formula bar |
@@ -276,6 +280,9 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-31 | minor | File-page text fields keep the view open when Esc cancels an edit (#3035) |
| 2026-10-31 | minor | Progress log: Recently Used and Insert Function categories |
| 2026-10-31 | minor | Progress log: Function Library |
| 2026-10-31 | minor | Progress log: shared CJK/Latin baselines in cells, editors and window titles |
| 2026-10-26 | minor | Progress log: Arabic bidi display and editing |
| 2026-10-26 | minor | Progress log: Arabic font fallback |
| 2026-10-25 | minor | Progress log: AutoSum over a selected row, and a selection's empty last row and column |
| 2026-10-24 | minor | Macro interface on the Automate tab; progress log row |
| 2026-10-24 | minor | Macro engine and `macro.*` commands; catalog 275 / 291 → 276 / 292 (View › Macros › Record Macro); progress log row |
@@ -306,6 +313,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-13 | trivial | Progress log: shared pictures written once on XLSX export (#985) |
| 2026-10-13 | minor | Progress log: conditional-format lookups no longer scan the range for rules without statistics (#1098) |
| 2026-10-12 | minor | Progress log: hidden-sheet navigation at workbook boundaries |
| 2026-10-12 | minor | Numfmt cleanup: one built-in format table (gridcraft-xlsx), fewer allocations rendering numbers, dates and General, equality test against the previous output (#817); progress log row |
| 2026-10-12 | minor | The control server requires an auth token and closes bad connections (#777); progress log row |
| 2026-10-12 | minor | Rows auto-fit oversized fonts on open and whole-row/column formatting (#833); progress log row |
| 2026-10-12 | minor | Progress log: multi-area row/column delete (#758) |
+1
View File
@@ -14,6 +14,7 @@ name = "gridcraft-cli"
path = "src/main.rs"
[dependencies]
gridcraft-sandbox = { workspace = true }
gridcraft-engine = { workspace = true }
gridcraft-mcp = { workspace = true }
serde_json = { workspace = true }
+10
View File
@@ -11,6 +11,7 @@
//! gridcraft-cli functions [--json] [--search X] [--category C]
//! gridcraft-cli mcp [--connect PORT] [--in FILE | --sample NAME]
//! gridcraft-cli send <port> <method> [json]
//! gridcraft-cli sandbox-check [--read PATH]... [--exec PATH]... [-- PROGRAM ARGS...]
//! gridcraft-cli version | --version
//! ```
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
@@ -30,6 +31,7 @@ macro_rules! outln {
}
mod check;
mod sandbox_check;
use gridcraft_engine::Session;
use serde_json::{Value, json};
@@ -49,6 +51,8 @@ const USAGE: &str = "usage:
gridcraft-cli functions [--json] [--search X] [--category C]
gridcraft-cli mcp [--connect PORT] [--in FILE | --sample NAME]
gridcraft-cli send <port> <method> [json]
gridcraft-cli sandbox-check [--read PATH]... [--exec PATH]... [-- PROGRAM ARGS...]
whether this system confines programs (no program: a confined self-test)
gridcraft-cli version";
/// stdout went away. A reader that stopped early (a closed pipe) ends the program quietly; any
@@ -62,6 +66,11 @@ fn stdout_failed(e: std::io::Error) -> ! {
}
fn main() -> ExitCode {
// A confined launcher run (and its probes) comes first, before anything else starts.
let raw: Vec<std::ffi::OsString> = std::env::args_os().skip(1).collect();
if let Some(code) = gridcraft_sandbox::handle_args(&raw) {
return ExitCode::from(u8::try_from(code).unwrap_or(1));
}
let args: Vec<String> = std::env::args().skip(1).collect();
let rest = args.get(1..).unwrap_or_default();
let r = match args.first().map(String::as_str) {
@@ -79,6 +88,7 @@ fn main() -> ExitCode {
Some("cat") => cat(rest),
Some("run") => run(rest),
Some("check") => return check::main(rest),
Some("sandbox-check") => return sandbox_check::main(rest),
Some("commands") => commands(rest),
Some("functions") => functions(rest),
Some("mcp") => mcp(rest),
+98
View File
@@ -0,0 +1,98 @@
//! `gridcraft-cli sandbox-check [--read PATH]... [--exec PATH]... [-- PROGRAM ARGS...]`: whether
//! this system confines programs, for a person or a reviewer checking a machine.
//!
//! Without a program it starts a confined copy of itself that tries what the sandbox must deny
//! (reading outside the policy, writing, the network, starting a process) and what it must still
//! allow (a thread, reading an allowed file), and prints one line per attempt. With a program it
//! runs that program confined, with the given read and executable paths and the system's shared
//! libraries, and passes its output and exit status through; its own lines then go to standard
//! error. Every path is absolute. Exit status: 0 when every attempt is
//! as required (or the program succeeded), 1 otherwise (a refused launch included), 2 when GridCraft
//! has no sandbox for this kind of system.
use std::ffi::OsString;
use std::path::PathBuf;
use std::process::ExitCode;
use gridcraft_sandbox::{Canary, Policy, REFUSED_EXIT_CODE, REPORT_FLAG, Support};
pub fn main(args: &[String]) -> ExitCode {
match run(args) {
Ok(code) => code,
Err(e) => {
eprintln!("gridcraft-cli: sandbox-check: {e}");
ExitCode::FAILURE
}
}
}
fn run(args: &[String]) -> Result<ExitCode, String> {
let mut policy = Policy::new().with_system_libraries();
let mut it = args.iter();
let mut program: Option<(PathBuf, Vec<OsString>)> = None;
while let Some(a) = it.next() {
match a.as_str() {
"--read" => policy = policy.read(it.next().ok_or("--read needs a path")?),
"--exec" => policy = policy.exec(it.next().ok_or("--exec needs a path")?),
"--" => {
let p = it.next().ok_or("a program follows `--`")?;
program = Some((PathBuf::from(p), it.by_ref().map(OsString::from).collect()));
}
other => return Err(format!("unknown option `{other}`")),
}
}
// With a program, its standard output is passed through untouched, so this command's own
// lines go to standard error; the self-test's report is the output, on standard output.
let report = program.is_none();
let say = |line: String| {
if report {
outln!("{line}");
} else {
eprintln!("{line}");
}
};
match gridcraft_sandbox::support() {
Support::Available(how) => say(format!("sandbox: available ({how})")),
Support::Unsupported(why) => {
say(format!("sandbox: unsupported ({why})"));
return Ok(ExitCode::from(2));
}
}
let canary = Canary::create().map_err(|e| e.to_string())?;
let exe = std::env::current_exe().map_err(|e| e.to_string())?;
let (program, program_args) = match program {
Some((p, a)) => (p, a),
// A confined copy of this program, reporting each attempt; it reads its own executable as
// the allowed file.
None => {
let home = std::env::var_os("HOME").unwrap_or_default();
(exe.clone(), vec![REPORT_FLAG.into(), canary.file().into(), exe.clone().into(), home])
}
};
let policy = policy.exec(program.clone());
let mut cmd = gridcraft_sandbox::launch_command(&policy, &canary, &program, &program_args).map_err(|e| e.to_string())?;
let status = cmd.status().map_err(|e| format!("couldn't start the launcher: {e}"))?;
drop(canary);
match status.code() {
Some(0) => {
say(if report { "result: every attempt is as the sandbox requires" } else { "result: the program ran confined and succeeded" }.into());
Ok(ExitCode::SUCCESS)
}
Some(REFUSED_EXIT_CODE) => {
// The launcher's refusal and a program's own exit share the code; the launcher's reason,
// when it refused, is the `gridcraft-sandbox: refused:` line above.
say(format!(
"result: exit code {REFUSED_EXIT_CODE}: the launcher refused to run the program (its reason is above), or the program itself exited with {REFUSED_EXIT_CODE}"
));
Ok(ExitCode::FAILURE)
}
other => {
say(format!(
"result: {} ({})",
if report { "some attempts are not as the sandbox requires" } else { "the program failed" },
other.map_or("no exit code".into(), |c| format!("exit code {c}"))
));
Ok(ExitCode::FAILURE)
}
}
}
+1
View File
@@ -1,3 +1,4 @@
//! The crate's integration tests, one module per file, built as one test binary.
mod cli;
mod sandbox;
mod text_to_columns_quotes;
+384
View File
@@ -0,0 +1,384 @@
//! The confined launcher, through the real binary: what a confined program can't do, what it
//! still can, and that a launch whose check fails runs nothing. Linux and macOS confine; other
//! systems report that they can't.
use std::path::{Path, PathBuf};
use std::process::{Command, Output};
/// Whether this machine confines programs: a supported system whose self-test passes. On a Linux
/// or macOS machine that reports no sandbox (another architecture, no `sandbox-exec`) or whose
/// kernel doesn't enforce Landlock, the confinement tests have nothing to check and pass with a
/// note. On CI (`CI` set) the machine must confine, so a failure there is never skipped.
fn confines() -> bool {
static CONFINES: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
*CONFINES.get_or_init(|| {
if !cfg!(any(target_os = "linux", target_os = "macos")) {
return false;
}
let o = cli(&["sandbox-check"]);
let out = text(&o);
let missing = out.contains("sandbox: unsupported") || out.contains("Landlock is not enforced");
if missing && std::env::var_os("CI").is_none() {
eprintln!("this machine doesn't confine programs; skipping:\n{out}");
return false;
}
true
})
}
fn cli(args: &[&str]) -> Output {
Command::new(env!("CARGO_BIN_EXE_gridcraft-cli")).args(args).output().unwrap()
}
fn text(o: &Output) -> String {
format!("{}{}", String::from_utf8_lossy(&o.stdout), String::from_utf8_lossy(&o.stderr))
}
/// A directory of the test's own in the temporary directory, with one file.
fn scratch(name: &str) -> PathBuf {
let dir = std::env::temp_dir().canonicalize().unwrap().join(format!("gridcraft-sandbox-it-{name}-{}", std::process::id()));
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(dir.join("allowed.txt"), "allowed text\n").unwrap();
dir
}
/// `cat` as this system names it.
fn cat() -> &'static str {
if Path::new("/bin/cat").exists() { "/bin/cat" } else { "/usr/bin/cat" }
}
#[test]
fn the_self_test_finds_every_denial() {
// A home directory of the test's own, so the home-directory attempt always runs.
let home = scratch("home");
let o = Command::new(env!("CARGO_BIN_EXE_gridcraft-cli")).arg("sandbox-check").env("HOME", &home).output().unwrap();
let out = text(&o);
if !cfg!(any(target_os = "linux", target_os = "macos")) {
assert_eq!(o.status.code(), Some(2), "{out}");
assert!(out.contains("sandbox: unsupported"), "{out}");
return;
}
if !confines() {
return;
}
assert!(o.status.success(), "{out}");
for line in [
"denied: read the canary",
"denied: change the canary's permissions",
"denied: truncate the canary",
"denied: write next to the canary",
"denied: write in the temporary directory",
"denied: TCP connection",
"denied: start a process",
"denied: signal another process",
"denied: UDP socket",
"denied: list the home directory",
"works: start a thread",
"works: read an allowed file",
] {
assert!(out.contains(line), "missing `{line}` in:\n{out}");
}
std::fs::remove_dir_all(home).unwrap();
}
#[test]
fn a_confined_program_reads_only_what_it_is_given() {
if !confines() {
return;
}
let dir = scratch("read");
let outside = scratch("outside");
let d = dir.to_string_lossy().into_owned();
let allowed = dir.join("allowed.txt").to_string_lossy().into_owned();
let o = cli(&["sandbox-check", "--read", &d, "--", cat(), &allowed]);
assert!(o.status.success(), "{}", text(&o));
// The program's output is passed through untouched; this command's own lines go to stderr.
assert_eq!(String::from_utf8_lossy(&o.stdout), "allowed text\n", "{}", text(&o));
assert!(String::from_utf8_lossy(&o.stderr).contains("result: the program ran confined"), "{}", text(&o));
// The same program, a file outside the policy: the read is refused and nothing is printed.
let other = outside.join("allowed.txt").to_string_lossy().into_owned();
let o = cli(&["sandbox-check", "--read", &d, "--", cat(), &other]);
assert!(!o.status.success(), "{}", text(&o));
assert!(!String::from_utf8_lossy(&o.stdout).contains("allowed text"), "{}", text(&o));
std::fs::remove_dir_all(dir).unwrap();
std::fs::remove_dir_all(outside).unwrap();
}
#[test]
fn a_launch_whose_check_fails_runs_nothing() {
// A run that says it is already confined and isn't finds the canary readable and must refuse.
// On macOS that is the second stage started without `sandbox-exec`; on Linux the launcher never
// uses that flag, and the same refusal must hold.
let canary = scratch("canary");
let echo = if Path::new("/bin/echo").exists() { "/bin/echo" } else { "/usr/bin/echo" };
let probe = canary.join("allowed.txt").to_string_lossy().into_owned();
// A dynamically linked `echo` needs the system's shared libraries and its loader.
// The second stage takes the process the first looked at: here, the parent of what `cli` starts.
let mut launch = vec![
"--sandbox-confined".to_string(),
std::process::id().to_string(),
"--exec".into(),
echo.into(),
"--exec".into(),
env!("CARGO_BIN_EXE_gridcraft-cli").into(),
];
for lib in ["/lib", "/lib64", "/usr/lib", "/usr/lib64"].into_iter().filter(|d| Path::new(d).exists()) {
launch.extend(["--exec".to_string(), lib.into()]);
}
launch.extend(["--probe".to_string(), probe.clone(), "--".into(), echo.into(), "the program ran".into()]);
let o = cli(&launch.iter().map(String::as_str).collect::<Vec<_>>());
let out = text(&o);
assert_eq!(o.status.code(), Some(97), "{out}");
assert!(!out.contains("the program ran"), "{out}");
assert!(out.contains("gridcraft-sandbox: refused"), "{out}");
assert!(out.contains("read the canary"), "the check refused, not the arguments: {out}");
// Without that process, the second stage has nothing to try a signal on.
let without: Vec<&str> = std::iter::once("--sandbox-confined").chain(launch[2..].iter().map(String::as_str)).collect();
let o = cli(&without);
assert_eq!(o.status.code(), Some(97), "{}", text(&o));
assert!(text(&o).contains("needs the process"), "{}", text(&o));
// The normal path: the launcher confines itself first, and the program runs.
if confines() {
launch.splice(0..2, ["--sandbox-launch".to_string()]);
// The launcher may be under a folder in the list, on either system.
let mut by_folder = launch.clone();
by_folder[4] = Path::new(env!("CARGO_BIN_EXE_gridcraft-cli")).parent().unwrap().to_string_lossy().into_owned();
let o = cli(&by_folder.iter().map(String::as_str).collect::<Vec<_>>());
assert!(o.status.success(), "{}", text(&o));
assert!(text(&o).contains("the program ran"), "{}", text(&o));
let o = cli(&launch.iter().map(String::as_str).collect::<Vec<_>>());
let out = text(&o);
assert!(o.status.success() && out.contains("the program ran"), "{out}");
}
// A canary inside an allowed path is refused before anything runs, everywhere.
let dir = canary.to_string_lossy().into_owned();
let o = cli(&["--sandbox-launch", "--read", &dir, "--exec", echo, "--probe", &probe, "--", echo, "the program ran"]);
assert_eq!(o.status.code(), Some(97), "{}", text(&o));
assert!(!text(&o).contains("the program ran"));
std::fs::remove_dir_all(canary).unwrap();
}
/// A Python found on this system: the interpreter to run confined, and its installation, the path it
/// needs to read. `None` when there is no `python3`.
fn system_python() -> Option<(String, String)> {
// On macOS `/usr/bin/python3` is a stub that starts Xcode's Python, outside any read path given
// here; an installed Python comes first.
let candidates = if cfg!(target_os = "macos") {
["/opt/homebrew/bin/python3", "/usr/local/bin/python3"]
} else {
["/usr/bin/python3", "/usr/local/bin/python3"]
};
let python = candidates.into_iter().find(|p| Path::new(p).exists())?;
let prefix = Command::new(python).args(["-c", "import sys; print(sys.base_prefix)"]).output().unwrap();
let prefix = String::from_utf8_lossy(&prefix.stdout).trim().to_string();
// A framework build's `bin/python3` only starts `Python.app`'s interpreter as a new process,
// which the sandbox refuses; the interpreter itself is run instead.
let framework = Path::new(&prefix).join("Resources/Python.app/Contents/MacOS/Python");
let real = if framework.exists() { framework } else { std::fs::canonicalize(python).unwrap() };
Some((real.to_string_lossy().into_owned(), prefix))
}
/// Confined, a program can't signal another process (this test, signal 0 delivering nothing), but can
/// still signal itself and its own threads. On Linux, where the filter names them, it also can't
/// change another process's limits or priority, or type into a terminal: no call here reaches a real
/// terminal, since its standard input isn't one.
#[test]
fn a_confined_program_leaves_other_processes_alone() {
if !confines() {
return;
}
let Some((python, prefix)) = system_python() else {
return;
};
let code = r#"
import os, signal, sys, threading
other = int(sys.argv[1])
def refused(call, *args):
try:
call(*args)
except PermissionError:
return True
except OSError as e:
print("other error", call, args, e)
return False
assert refused(os.kill, other, 0), "a signal to another process"
os.kill(os.getpid(), 0)
seen = []
signal.signal(signal.SIGUSR1, lambda *_: seen.append(1))
signal.raise_signal(signal.SIGUSR1)
worker = threading.Thread(target=lambda: signal.pthread_kill(threading.main_thread().ident, signal.SIGUSR1))
worker.start()
worker.join()
assert len(seen) >= 1, "a signal to itself"
if sys.platform == "linux":
import ctypes, errno, fcntl, resource, termios
libc = ctypes.CDLL(None, use_errno=True)
assert libc.shmget(0, 4096, 0o600) == -1 and ctypes.get_errno() == errno.EPERM, "System V shared memory"
assert libc.msgget(0, 0o600) == -1 and ctypes.get_errno() == errno.EPERM, "a System V message queue"
assert libc.semget(0, 1, 0o600) == -1 and ctypes.get_errno() == errno.EPERM, "a System V semaphore"
if hasattr(libc, "mq_open"):
assert libc.mq_open(b"/gridcraft-check", os.O_RDONLY) == -1 and ctypes.get_errno() == errno.EPERM, "a POSIX message queue"
# macOS answers these for the processes the profile lets it signal, which are only itself.
assert refused(os.kill, -1, 0), "a signal to every process"
assert refused(os.kill, 0, 0), "a signal to the process group"
assert refused(resource.prlimit, other, resource.RLIMIT_NOFILE), "another process's limits"
assert refused(os.setpriority, os.PRIO_PROCESS, other, 0), "another process's priority"
assert refused(os.setpriority, os.PRIO_USER, 0, 0), "every process's priority"
assert refused(os.sched_setaffinity, other, {0}), "another process's processors"
assert refused(os.pidfd_open, other), "a handle on another process"
resource.prlimit(0, resource.RLIMIT_NOFILE)
resource.prlimit(os.getpid(), resource.RLIMIT_NOFILE)
os.setpriority(os.PRIO_PROCESS, 0, os.getpriority(os.PRIO_PROCESS, 0))
assert refused(fcntl.ioctl, 0, termios.TIOCSTI, b"x"), "typing into a terminal"
assert refused(fcntl.ioctl, 0, termios.TIOCSTI | (1 << 32), b"x"), "the same with high bits set"
assert refused(fcntl.ioctl, 0, 0x541C, b"x"), "TIOCLINUX"
print("left alone")
"#;
let me = std::process::id().to_string();
let o = cli(&["sandbox-check", "--read", &prefix, "--", &python, "-I", "-c", code, &me]);
assert!(o.status.success(), "{}", text(&o));
assert!(String::from_utf8_lossy(&o.stdout).contains("left alone"), "{}", text(&o));
}
/// Confined, a program changes no file, whether the policy lets it read that file or not, by path or
/// by a descriptor: not its contents, permissions, owner, extended attributes, times or flags.
/// Skipped when there is no `python3`.
#[test]
fn a_confined_program_changes_no_file() {
if !confines() {
return;
}
let Some((python, prefix)) = system_python() else {
return;
};
let code = r#"
import errno, fcntl, os, struct, sys
readable, outside = sys.argv[1:3]
def refused(what, call, *args):
try:
call(*args)
except PermissionError:
return
except OSError as e:
sys.exit(f"{what}: another error, not a refusal: {e}")
sys.exit(f"{what}: it worked")
fd = os.open(readable, os.O_RDONLY)
for target in (readable, outside, fd):
refused(f"permissions of {target}", os.chmod, target, 0)
refused(f"owner of {target}", os.chown, target, os.getuid(), os.getgid())
refused(f"times of {target}", os.utime, target, (0, 0))
if sys.platform == "linux":
refused(f"setting an attribute of {target}", os.setxattr, target, "user.gridcraft", b"1")
refused(f"removing an attribute of {target}", os.removexattr, target, "user.gridcraft")
for path in (readable, outside):
refused(f"truncate {path}", os.truncate, path, 0)
refused(f"open {path} with O_TRUNC", os.open, path, os.O_RDONLY | os.O_TRUNC)
refused(f"open {path} to write", os.open, path, os.O_WRONLY)
# A descriptor opened to read can't be truncated either way; the file's contents are checked after.
try:
os.ftruncate(fd, 0)
except OSError:
pass
if sys.platform == "linux":
import ctypes
libc = ctypes.CDLL(None, use_errno=True)
# Newer calls, which the kernel running this may not have: the filter answers before it would.
for name, number in (("fchmodat2", 452), ("setxattrat", 463), ("removexattrat", 466), ("file_setattr", 469)):
ctypes.set_errno(0)
result = libc.syscall(number, -100, readable.encode(), 0, 0)
assert result == -1 and ctypes.get_errno() == errno.EPERM, f"{name}: {result}, {ctypes.get_errno()}"
# On a descriptor opened only to read, these change the file's flags and attributes.
refused("FS_IOC_SETFLAGS", fcntl.ioctl, fd, 0x40086602, struct.pack("l", 0))
refused("FS_IOC_FSSETXATTR", fcntl.ioctl, fd, 0x401C5820, bytes(28))
refused("FS_IOC_SET_ENCRYPTION_POLICY", fcntl.ioctl, fd, 0x800C6613, bytes(12))
# What a program still needs: its own descriptors' flags, and a file of its own in memory.
os.set_inheritable(fd, True)
os.set_inheritable(fd, False)
os.set_blocking(fd, False)
own = os.memfd_create("gridcraft-check")
os.ftruncate(own, 16)
assert os.fstat(own).st_size == 16
print("unchanged")
"#;
let inside = scratch("unchanged-inside");
let elsewhere = scratch("unchanged-outside");
let (readable, outside) = (inside.join("allowed.txt"), elsewhere.join("allowed.txt"));
let state = |file: &Path| {
let m = std::fs::metadata(file).unwrap();
(std::fs::read(file).unwrap(), m.permissions(), m.modified().unwrap())
};
let before = (state(&readable), state(&outside));
let d = inside.to_string_lossy().into_owned();
let (r, x) = (readable.to_string_lossy().into_owned(), outside.to_string_lossy().into_owned());
let o = cli(&["sandbox-check", "--read", &prefix, "--read", &d, "--", &python, "-I", "-c", code, &r, &x]);
assert!(o.status.success(), "{}", text(&o));
assert!(String::from_utf8_lossy(&o.stdout).contains("unchanged"), "{}", text(&o));
assert_eq!((state(&readable), state(&outside)), before);
std::fs::remove_dir_all(inside).unwrap();
std::fs::remove_dir_all(elsewhere).unwrap();
}
/// A Python found on this system starts and imports its standard library while confined, with
/// read access to its installation only. Skipped when there is no `python3`.
#[test]
fn python_starts_confined() {
if !confines() {
return;
}
let Some((real, prefix)) = system_python() else {
return;
};
let o =
cli(&["sandbox-check", "--read", &prefix, "--", &real, "-I", "-c", "import json, math, datetime, decimal; print(json.dumps([math.pi > 3]))"]);
assert!(o.status.success(), "{}", text(&o));
assert!(String::from_utf8_lossy(&o.stdout).contains("[true]"), "{}", text(&o));
// Confined Python can't open the network or a file outside its installation.
let o = cli(&["sandbox-check", "--read", &prefix, "--", &real, "-I", "-c", "import socket; socket.create_connection(('127.0.0.1', 9), 2)"]);
assert!(!o.status.success(), "{}", text(&o));
assert!(text(&o).contains("PermissionError") || text(&o).contains("Operation not permitted"), "{}", text(&o));
}
#[test]
fn a_confined_program_inherits_no_secrets() {
if !confines() {
return;
}
let env = if Path::new("/usr/bin/env").exists() { "/usr/bin/env" } else { "/bin/env" };
let o = Command::new(env!("CARGO_BIN_EXE_gridcraft-cli"))
.args(["sandbox-check", "--", env])
.env("GRIDCRAFT_CONTROL_TOKEN", "secret-token")
.env("AWS_SECRET_ACCESS_KEY", "secret-key")
.env("LANG", "en_US.UTF-8")
.output()
.unwrap();
let out = text(&o);
assert!(o.status.success(), "{out}");
assert!(!out.contains("secret-"), "the environment reached the program:\n{out}");
assert!(out.contains("LANG=en_US.UTF-8"), "the locale is kept:\n{out}");
assert!(out.contains("PATH=/usr/bin:/bin"), "{out}");
}
/// A descriptor the parent leaves open (here fd 9, opened by a shell on a file outside any policy)
/// would survive `exec` with access the sandbox doesn't revoke, so the launcher closes it: the
/// program runs, and can't read through it. (A desktop app's system libraries leave such files open,
/// so refusing to run would block Python there.)
#[test]
fn an_inherited_descriptor_never_reaches_the_program() {
if !confines() {
return;
}
let outside = scratch("leak");
let leaked = outside.join("allowed.txt").to_string_lossy().into_owned();
let cli = env!("CARGO_BIN_EXE_gridcraft-cli");
// The shell opens fd 9 (not close-on-exec) and then becomes the command, which inherits it. The
// confined shell then tries to read through it, with a built-in: starting `cat` is refused. Not
// fd 3: on macOS the system's libraries hold a socket of their own there under the sandbox.
let bash = if cfg!(target_os = "macos") { "--exec /bin/bash" } else { "" };
let program = "read line <&9 && echo \"leaked: $line\"; echo finished";
let script = format!("exec 9<'{leaked}'; exec '{cli}' sandbox-check {bash} -- /bin/sh -c '{program}'");
let o = Command::new("/bin/sh").args(["-c", &script]).output().unwrap();
let out = text(&o);
assert!(out.contains("finished"), "the program ran:\n{out}");
assert!(!out.contains("leaked:"), "the program read through the descriptor:\n{out}");
std::fs::remove_dir_all(outside).unwrap();
}
+2 -1
View File
@@ -13,6 +13,7 @@ workspace = true
[dependencies]
gridcraft-engine = { workspace = true }
gridcraft-ui-egui = { workspace = true }
gridcraft-sandbox = { workspace = true }
eframe = { workspace = true, features = ["default_fonts", "wgpu", "accesskit", "wayland", "x11", "clipboard"] }
egui = { workspace = true }
serde_json = { workspace = true }
@@ -34,7 +35,7 @@ winit = { version = "0.30", default-features = false, features = ["x11", "waylan
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
# Same craft-libs revision as the workspace's craft-update and craft-window (see the root
# Cargo.toml).
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d" }
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "6e03f65bf45181db77ba6229716e6624db9eb25b" }
craft-update = { workspace = true }
rfd = { workspace = true }
arboard = { workspace = true, features = ["image-data"] }
+6
View File
@@ -241,6 +241,12 @@ fn icon() -> Option<egui::IconData> {
}
fn main() -> eframe::Result<()> {
// A confined launcher run (`gridcraft_sandbox::launch_command`) comes before everything else:
// it opens no window, writes no log, and becomes the program it confines.
let raw: Vec<std::ffi::OsString> = std::env::args_os().skip(1).collect();
if let Some(code) = gridcraft_sandbox::handle_args(&raw) {
std::process::exit(code);
}
// First, so the panic hook and every start-up warning are recorded (`logging`).
let logger = logging::install();
// Commands catch their panics (`Session::execute`), but the default hook only prints to
+3
View File
@@ -110,3 +110,6 @@ sync_github_name = true
[people.maxm11]
sync_github_name = true
[people.RockerHX]
sync_github_name = true
+215 -8
View File
@@ -39,6 +39,15 @@ const MAX_DEPTH: usize = 400;
/// every legitimate call; hostile work that grows past one dimension (a LAMBDA that recurses
/// or nest `MAP` into `MAP`) then stops at the same bound instead of squaring.
const LAMBDA_FUEL: u64 = MAX_CELLS;
/// Work one cell's formula may do: one unit per expression evaluated, per cell read from a
/// reference, per array value handed to a function, per pair of areas intersected and per
/// character of a defined name's formula parsed where the name is used. Each limit
/// above bounds one kind of step, but a compact formula could still repeat bounded steps
/// exponentially often (defined names that each use the one before twice, a computed criterion
/// whose database function reruns it, an aggregate over a large array inside every LAMBDA call).
/// Past this the formula is `#CALC!`. It is far above what a formula that reads every cell of a
/// full-size array a few times needs.
const EVAL_WORK: u64 = 16 * MAX_CELLS;
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)]
pub struct Area {
@@ -145,6 +154,16 @@ pub struct Evaluator<'h> {
/// runs out of work after the same number of calls that a legitimately wide one could
/// make, instead of an amount of work that squares with array nesting (#2382).
fuel: u64,
/// Work this cell still has left (see [`EVAL_WORK`]), shared by every branch of the formula
/// and by the formulas it computes on the way (names, computed criteria).
work: u64,
/// A spend found too little work left. A spend that leaves exactly nothing is fine, so this
/// can't be told from `work` alone; a database function checks it after its computed
/// criteria ran, whose failures it can't otherwise see.
exhausted: bool,
/// Computed criteria being evaluated, outermost first: one that is reached again from inside
/// itself counts as not met instead of repeating its work at every level.
active_criteria: Vec<(usize, CellRef)>,
}
struct FnCtx<'a, 'h> {
@@ -174,6 +193,9 @@ impl Ctx for FnCtx<'_, '_> {
fn computed_criterion(&mut self, row: usize, col: usize, db: &Array, record: usize) -> Option<bool> {
let area = self.criteria?;
let at = area.range.start.offset(row as i64, col as i64).filter(|c| area.range.contains(*c))?;
if self.ev.active_criteria.contains(&(area.sheet, at)) {
return Some(false);
}
let expr = self.ev.host.workbook().sheet(area.sheet)?.cell(at)?.formula.as_ref()?.expr_arc()?;
let shifted = gridcraft_formula::adjust::shift_relative((*expr).clone(), record as i64 - 1, 0);
let env = (0..db.cols)
@@ -193,12 +215,24 @@ impl Ctx for FnCtx<'_, '_> {
calls: self.ev.calls,
stack_top: self.ev.stack_top,
fuel: self.ev.fuel,
work: self.ev.work,
exhausted: false,
active_criteria: std::mem::take(&mut self.ev.active_criteria),
};
ev.active_criteria.push((area.sheet, at));
let v = matches!(ev.value(&shifted).scalar().to_bool(), Ok(true));
// The child's spend counts against this cell's budget.
self.ev.fuel = ev.fuel;
self.ev.work = ev.work;
self.ev.exhausted |= ev.exhausted;
let mut active = std::mem::take(&mut ev.active_criteria);
active.pop();
self.ev.active_criteria = active;
Some(v)
}
fn exhausted(&self) -> bool {
self.ev.exhausted
}
fn has_lambda(&self) -> bool {
self.lambda.is_some()
}
@@ -216,7 +250,42 @@ impl Ctx for FnCtx<'_, '_> {
impl<'h> Evaluator<'h> {
pub fn new(host: &'h mut dyn Host, sheet: usize, at: CellRef) -> Self {
Evaluator { host, sheet, at, env: Vec::new(), depth: 0, calls: 0, stack_top: 0, fuel: LAMBDA_FUEL }
Evaluator {
host,
sheet,
at,
env: Vec::new(),
depth: 0,
calls: 0,
stack_top: 0,
fuel: LAMBDA_FUEL,
work: EVAL_WORK,
exhausted: false,
active_criteria: Vec::new(),
}
}
/// Spends `units` of this cell's work; `false` (and nothing left) when there isn't enough.
pub(crate) fn spend(&mut self, units: u64) -> bool {
match self.work.checked_sub(units) {
Some(w) => {
self.work = w;
true
}
None => {
self.work = 0;
self.exhausted = true;
false
}
}
}
/// Charges the values of an array (one unit per value; one for anything else).
pub(crate) fn spend_on(&mut self, v: &Value) -> bool {
match v {
Value::Array(a) => self.spend(a.data.len() as u64),
_ => self.spend(1),
}
}
/// Evaluates to a value (references are dereferenced; multi-cell → array).
@@ -266,18 +335,27 @@ impl<'h> Evaluator<'h> {
}
// A reference keeps its size, so the array has its shape and its blanks count
// (`=A1:A10`, `SUMPRODUCT(--(A1:A10=""))`).
let range = self.read_range(a);
let (h, w) = (range.height() as usize, range.width() as usize);
if (h as u64) * (w as u64) > MAX_CELLS {
return Value::Error(CellError::Num);
}
if !self.spend((h as u64) * (w as u64)) {
return Value::Error(CellError::Calc);
}
let data = self.host.range_values(a.sheet, range);
Array::new(h, w, data).map(Value::from).unwrap_or(Value::Error(CellError::Value))
}
/// The block that reading `a` gives: `a` itself, or when it is large (or runs to the sheet's
/// edge) the part inside the sheet's used range.
pub(crate) fn read_range(&mut self, a: &Area) -> RangeRef {
let cells = u64::from(a.range.height()) * u64::from(a.range.width());
// `A:A`, `1:1` and `A2:A1048576` run to the edge of the sheet: they stand for "the rest of
// the column", not for a block of a chosen size.
let to_edge = a.range.end.row == MAX_ROWS - 1 || a.range.end.col == MAX_COLS - 1;
let full_size = cells <= FULL_SIZE_CELLS || (!to_edge && cells <= FULL_SIZE_BOUNDED_CELLS);
let range = if full_size { a.range } else { self.trim(a) };
let (h, w) = (range.height() as usize, range.width() as usize);
if (h as u64) * (w as u64) > MAX_CELLS {
return Value::Error(CellError::Num);
}
let data = self.host.range_values(a.sheet, range);
Array::new(h, w, data).map(Value::from).unwrap_or(Value::Error(CellError::Value))
if full_size { a.range } else { self.trim(a) }
}
/// Trims the bottom/right of an area to the sheet's used range (keeps the top-left so
@@ -310,6 +388,9 @@ impl<'h> Evaluator<'h> {
self.host.stack_exhausted();
return err(CellError::Num);
}
if !self.spend(1) {
return err(CellError::Calc);
}
self.depth += 1;
if self.depth > MAX_DEPTH {
self.depth -= 1;
@@ -382,6 +463,11 @@ impl<'h> Evaluator<'h> {
if let Some(def) = wb.name(bare, scope_sheet) {
let text = def.formula.clone();
let scope = def.scope.unwrap_or(self.sheet);
// Each use parses and evaluates the definition again: names that each use the one
// before twice double the work per name, so it is charged.
if !self.spend(text.len() as u64) {
return err(CellError::Calc);
}
return match gridcraft_formula::parse_name(&text) {
Ok(expr) => {
// Relative references in a name are relative to A1: used here, they move by
@@ -543,6 +629,11 @@ impl<'h> Evaluator<'h> {
if r.height() == 1 && r.start.col <= self.at.col && self.at.col <= r.end.col {
return Some(CellRef::new(r.start.row, self.at.col));
}
// A 2-D range intersects on both the formula's row and column (observed in Excel, on
// another sheet too); one that includes the formula's own cell is a circular reference.
if r.start.row <= self.at.row && self.at.row <= r.end.row && r.start.col <= self.at.col && self.at.col <= r.end.col {
return Some(self.at);
}
None
}
@@ -574,6 +665,10 @@ impl<'h> Evaluator<'h> {
let (x, y) = (self.eval(a), self.eval(b));
match (x, y) {
(Ev::R(mut xa), Ev::R(ya)) => {
// A union reused through LET doubles at each step: its areas count.
if !self.spend((xa.len() as u64).saturating_add(ya.len() as u64)) {
return err(CellError::Calc);
}
xa.extend(ya);
Ev::R(xa)
}
@@ -585,6 +680,11 @@ impl<'h> Evaluator<'h> {
let (x, y) = (self.eval(a), self.eval(b));
match (x, y) {
(Ev::R(xa), Ev::R(ya)) => {
// Every pair is looked at, and each may be kept: charged before the loop,
// so unions squared through LET can't build an enormous list.
if !self.spend((xa.len() as u64).saturating_mul(ya.len() as u64)) {
return err(CellError::Calc);
}
let mut out = Vec::new();
for p in &xa {
for q in &ya {
@@ -678,6 +778,113 @@ mod fuel_tests {
ev.value(&gridcraft_formula::parser::parse(src).unwrap_or_else(|e| panic!("{e}")))
}
/// Evaluates `src` in `wb` with `work` units of work, and the work left.
fn eval_work(wb: Workbook, work: u64, src: &str) -> (Value, u64) {
let mut host = Empty { wb };
let mut ev = Evaluator::new(&mut host, 0, CellRef::new(0, 0));
ev.work = work;
let v = ev.value(&gridcraft_formula::parser::parse(src).unwrap_or_else(|e| panic!("{e}")));
(v, ev.work)
}
fn is_calc(v: &Value) -> bool {
matches!(v, Value::Error(CellError::Calc))
}
/// Serves the workbook's stored values, for database functions to read.
struct Stored {
wb: Workbook,
}
impl Host for Stored {
fn workbook(&self) -> &Workbook {
&self.wb
}
fn cell_value(&mut self, sheet: usize, c: CellRef) -> Value {
self.wb.sheet(sheet).and_then(|s| s.cell(c)).map_or(Value::Empty, |cell| cell.value.clone())
}
fn now_serial(&self) -> f64 {
0.0
}
fn random(&mut self) -> f64 {
0.0
}
}
/// A computed criterion that runs out of work counts as not met for its record, which the
/// database function can't see: it must answer #CALC!, not a count that leaves records out.
#[test]
fn a_computed_criterion_out_of_work_makes_the_database_function_calc() {
use gridcraft_model::{Cell, Formula};
let mut wb = Workbook::new();
let name = |name: String, formula: String| gridcraft_model::DefinedName { name, scope: None, formula, comment: String::new(), hidden: false };
wb.names.push(name("Step0".into(), "1".into()));
for i in 1..=60 {
wb.names.push(name(format!("Step{i}"), format!("Step{}+Step{}", i - 1, i - 1)));
}
let at = |s: &str| CellRef::parse(s).unwrap();
let sheet = wb.sheet_mut(0).unwrap();
for (r, (n, age)) in [("name", Value::text("age")), ("a", Value::number(1.0)), ("b", Value::number(2.0))].into_iter().enumerate() {
sheet.cells.set(at(&format!("A{}", r + 1)), Cell::value(Value::text(n)));
sheet.cells.set(at(&format!("B{}", r + 1)), Cell::value(age));
}
sheet.cells.set(at("D1"), Cell::value(Value::text("Formula")));
let host_wb = |criterion: &str| {
let mut wb = wb.clone();
// A criteria cell is read by its value first (a blank one matches everything): give it
// the value a recalculation would have stored.
let mut cell = Cell::formula(Formula::new(criterion));
cell.value = Value::Bool(true);
wb.sheet_mut(0).unwrap().cells.set(at("D2"), cell);
wb
};
let run = |criterion: &str| {
let mut host = Stored { wb: host_wb(criterion) };
let mut ev = Evaluator::new(&mut host, 0, CellRef::new(9, 9));
ev.work = 100_000;
ev.value(&gridcraft_formula::parser::parse("=DCOUNT(A1:B3,\"age\",D1:D2)").unwrap())
};
assert_eq!(run("=B2>0"), Value::number(2.0), "a cheap criterion counts every record");
assert!(is_calc(&run("=Step60>0")), "{:?}", run("=Step60>0"));
}
#[test]
fn names_that_double_at_each_step_run_out_of_work() {
// Step1 = Step0+Step0, Step2 = Step1+Step1, ...: Step60 would evaluate Step0 2^60 times.
let mut wb = Workbook::default();
let name = |name: String, formula: String| gridcraft_model::DefinedName { name, scope: None, formula, comment: String::new(), hidden: false };
wb.names.push(name("Step0".into(), "1".into()));
for i in 1..=60 {
wb.names.push(name(format!("Step{i}"), format!("Step{}+Step{}", i - 1, i - 1)));
}
let (v, _) = eval_work(wb.clone(), EVAL_WORK, "=Step5");
assert_eq!(v, Value::number(32.0));
let (v, left) = eval_work(wb, 100_000, "=Step60");
assert!(is_calc(&v), "{v:?}");
assert_eq!(left, 0);
}
#[test]
fn intersections_of_unions_are_charged_per_pair() {
let wb = Workbook::default();
// 4 x 4 pairs, 4 of which meet.
let src = "=AREAS((A1,B1,C1,D1) (A1,B1,C1,D1))";
let (v, _) = eval_work(wb.clone(), EVAL_WORK, src);
assert_eq!(v, Value::number(4.0));
let (v, _) = eval_work(wb, 20, src);
assert!(is_calc(&v), "{v:?}");
}
#[test]
fn an_aggregate_inside_every_lambda_call_spends_work() {
let wb = Workbook::default();
// 100 calls, each summing the 10,000 values of a captured array: about a million units.
let src = "=LET(big,SEQUENCE(10000),SUM(MAP(SEQUENCE(100),LAMBDA(x,SUM(big)))))";
let (v, _) = eval_work(wb.clone(), 2_000_000, src);
assert_eq!(v, Value::number(100.0 * 50_005_000.0));
let (v, _) = eval_work(wb, 200_000, src);
assert!(is_calc(&v), "{v:?}");
}
#[test]
fn lambda_calls_spend_fuel_and_run_out_quietly() {
// Three applications, one charge each: 3 inside the budget. (A nested MAP wouldn't do
+35 -8
View File
@@ -52,13 +52,16 @@ impl<'h> Evaluator<'h> {
"ISFORMULA" | "FORMULATEXT" => {
let Some(Ev::R(a)) = args.first().map(|a| self.eval(a)) else { return err(CellError::NA) };
let Some(a) = a.first() else { return err(CellError::NA) };
let f = self.host.workbook().sheet(a.sheet).and_then(|s| s.cell(a.range.start)).and_then(|c| c.formula.clone());
let wb = self.host.workbook();
let f = wb.sheet(a.sheet).and_then(|s| s.cell(a.range.start)).and_then(|c| c.formula.clone());
// A formula formatted Hidden on a protected sheet stays hidden from FORMULATEXT too.
let hidden = wb.sheet(a.sheet).is_some_and(|s| s.is_protected() && wb.styles.get(s.style_id(a.range.start)).protection.hidden);
if name == "ISFORMULA" {
Ev::V(Value::Bool(f.is_some()))
} else {
match f {
Some(f) => Ev::V(Value::text(format!("={}", f.text))),
None => err(CellError::NA),
Some(f) if !hidden => Ev::V(Value::text(format!("={}", f.text))),
_ => err(CellError::NA),
}
}
}
@@ -203,9 +206,9 @@ impl<'h> Evaluator<'h> {
let index = self.value(&args[0]);
if let Value::Array(_) = index {
let Some(spec) = gridcraft_functions::lookup("CHOOSE") else { return err(CellError::Name) };
let mut evaluated = vec![Arg { value: index, from_ref: false }];
let mut evaluated = vec![Arg::val(index)];
for argument in &args[1..] {
evaluated.push(Arg { value: self.value(argument), from_ref: false });
evaluated.push(Arg::val(self.value(argument)));
}
let mut context = FnCtx { ev: self, criteria: None, lambda: None };
return gridcraft_functions::call(spec, &evaluated, &mut context).pipe(Ev::V);
@@ -351,10 +354,25 @@ impl<'h> Evaluator<'h> {
}
ev => self.deref(ev),
};
evaluated.push(Arg { value, from_ref });
// Only an argument left out (`f(a,,b)`) is omitted; one that computes a blank is given.
let blank_value = !from_ref && value == Value::Empty && lambda_at != Some(i) && !matches!(a, Expr::Missing);
evaluated.push(Arg { value, from_ref, blank_value });
}
// The function may go through every value it is given (a SUM inside each call of a
// LAMBDA over a large array): the values count against the cell's work.
let values = evaluated.iter().fold(0u64, |n, a| n.saturating_add(if let Value::Array(x) = &a.value { x.data.len() as u64 } else { 1 }));
if !self.spend(values) {
return err(CellError::Calc);
}
let database = criteria.is_some();
let mut ctx = FnCtx { ev: self, criteria, lambda };
gridcraft_functions::call(spec, &evaluated, &mut ctx).pipe(Ev::V)
let v = gridcraft_functions::call(spec, &evaluated, &mut ctx);
// A computed criterion that ran out of work counted as not met: the result leaves
// records out.
if database && self.exhausted {
return err(CellError::Calc);
}
Ev::V(v)
}
/// Performs aggregation operations without copying its ranges: blocks are folded over the stored
@@ -392,6 +410,12 @@ impl<'h> Evaluator<'h> {
acc.count = acc.count.saturating_add(count);
return Ok(());
}
// Folded without building an array, but still walked: the part inside the used
// range counts against the cell's work.
let r = self.read_range(a);
if !self.spend(u64::from(r.height()) * u64::from(r.width())) {
return Err(err(CellError::Calc));
}
let mut error = None;
let walked = self.host.visit_range(a.sheet, a.range, &mut |v| match acc.add_cell(kind, v) {
Ok(()) => true,
@@ -411,10 +435,13 @@ impl<'h> Evaluator<'h> {
/// Adds one argument's value by calling `SUM` and `COUNT` on it, so coercion and error rules
/// stay in one place (`AVERAGE` of an argument is its `SUM` over its `COUNT`).
fn aggregate_value(&mut self, kind: Agg, value: Value, from_ref: bool, acc: &mut Acc) -> Result<(), Ev> {
if !from_ref && !self.spend_on(&value) {
return Err(err(CellError::Calc));
}
let (Some(sum), Some(count)) = (gridcraft_functions::lookup("SUM"), gridcraft_functions::lookup("COUNT")) else {
return Err(err(CellError::Name));
};
let args = [Arg { value, from_ref }];
let args = [Arg { value, from_ref, blank_value: false }];
if kind != Agg::Count {
match gridcraft_functions::call(sum, &args, &mut FnCtx { ev: self, criteria: None, lambda: None }) {
Value::Number(n) => acc.sum += n,
+103 -6
View File
@@ -118,11 +118,29 @@ impl<'h> Evaluator<'h> {
for c in 0..w {
let pos = (num(&Some(ra.get_broadcast(r, c))), num(&Some(ca.get_broadcast(r, c))));
let v = match pos {
// A position that picks several cells is #CALC!. That is decided from the
// shape: reading the block first, for every position, let a short formula
// read one large range over and over only to discard it.
(Ok(row), Ok(col)) if index_picks_several_values(&src, row, col, two) => Value::Error(CellError::Calc),
(Ok(row), Ok(col)) => {
let one = self.index_one(src.clone(), row, col, area_n, two);
match self.deref(one) {
Value::Array(a) if a.rows * a.cols > 1 => Value::Error(CellError::Calc),
v => v.scalar(),
let cells = match &one {
Ev::R(areas) => match areas.as_slice() {
[a] => self.area_cells(a),
_ => 0,
},
_ => 0,
};
if cells > MAX_CELLS {
// As reading it would answer.
Value::Error(CellError::Num)
} else if cells > 1 {
Value::Error(CellError::Calc)
} else {
match self.deref(one) {
Value::Array(a) if a.rows * a.cols > 1 => Value::Error(CellError::Calc),
v => v.scalar(),
}
}
}
(Err(e), _) | (_, Err(e)) => Value::Error(e),
@@ -139,6 +157,16 @@ impl<'h> Evaluator<'h> {
self.index_one(src, row, col, area_n, two)
}
/// Cells that reading `a` gives: its own size, or for a large block the part inside the
/// sheet's used range (what [`Evaluator::area_value`] reads).
fn area_cells(&mut self, a: &Area) -> u64 {
if a.range.is_single() {
return 1;
}
let r = self.read_range(a);
u64::from(r.height()) * u64::from(r.width())
}
/// INDEX for one row and column position (0 = the whole column or row).
fn index_one(&mut self, src: Ev, row: i64, col: i64, area_n: i64, two_args: bool) -> Ev {
if row < 0 || col < 0 {
@@ -203,7 +231,7 @@ impl<'h> Evaluator<'h> {
let horizontal = l.range.height() == 1 && l.range.width() > 1;
let same_len = if vertical { r.range.height() == l.range.height() } else { r.range.width() == l.range.width() };
if !matches!(key, Value::Array(_)) && (vertical || horizontal) && same_len {
let mut xargs = vec![Arg { value: key, from_ref: false }, Arg { value: self.area_value(&l), from_ref: true }];
let mut xargs = vec![Arg::val(key), Arg::reference(self.area_value(&l))];
// An empty match_mode before a given search_mode is the default (0), not the end.
let given = |i: usize| args.get(i).is_some_and(|e| !matches!(e, Expr::Missing));
let last = if given(5) {
@@ -215,10 +243,11 @@ impl<'h> Evaluator<'h> {
};
for i in 4..=last {
match args.get(i) {
Some(Expr::Missing) | None => xargs.push(Arg { value: Value::Number(0.0), from_ref: false }),
Some(Expr::Missing) | None => xargs.push(Arg::val(Value::Number(0.0))),
Some(e) => {
let value = self.value(e);
xargs.push(Arg { value, from_ref: false });
let blank_value = value == Value::Empty;
xargs.push(Arg { value, from_ref: false, blank_value });
}
}
}
@@ -407,3 +436,71 @@ impl<'h> Evaluator<'h> {
Ev::V(gridcraft_functions::aggregate_values(func, &values))
}
}
/// Whether INDEX on an array picks a whole row or column of several values at position
/// (`row`, `col`), which the array-of-positions form answers with #CALC!.
fn index_picks_several_values(src: &Ev, row: i64, col: i64, two_args: bool) -> bool {
let Ev::V(Value::Array(arr)) = src else { return false };
let (row, col) = if two_args && arr.rows == 1 { (1, row) } else { (row, col) };
if row < 0 || col < 0 || row as u64 > arr.rows as u64 || col as u64 > arr.cols as u64 {
return false;
}
(row == 0 && arr.rows > 1) || (col == 0 && arr.cols > 1)
}
#[cfg(test)]
mod tests {
use super::super::Host;
use super::*;
use gridcraft_model::Workbook;
/// A sheet of blanks that counts the blocks it is asked for.
struct Counting {
wb: Workbook,
cells_read: usize,
}
impl Host for Counting {
fn workbook(&self) -> &Workbook {
&self.wb
}
fn cell_value(&mut self, _sheet: usize, _c: CellRef) -> Value {
Value::Empty
}
fn now_serial(&self) -> f64 {
0.0
}
fn random(&mut self) -> f64 {
0.0
}
fn range_values(&mut self, _sheet: usize, range: RangeRef) -> Vec<Value> {
self.cells_read += range.count() as usize;
vec![Value::Empty; range.count() as usize]
}
}
fn eval(src: &str) -> (Value, usize) {
let mut host = Counting { wb: Workbook::default(), cells_read: 0 };
let mut ev = Evaluator::new(&mut host, 0, CellRef::new(0, 0));
let v = ev.value(&gridcraft_formula::parser::parse(src).unwrap_or_else(|e| panic!("{e}")));
(v, host.cells_read)
}
fn all_calc(v: &Value, n: usize) -> bool {
matches!(v, Value::Array(a) if a.data.len() == n && a.data.iter().all(|x| *x == Value::Error(CellError::Calc)))
}
#[test]
fn index_positions_that_pick_several_cells_are_calc_without_reading_them() {
// Each zero row position picks the whole of A1:A50000: #CALC!, decided from the shape. It
// used to read the 50,000 cells once per position before discarding them.
let (v, read) = eval("=INDEX(A1:A50000,{0,0,0,0},1)");
assert!(all_calc(&v, 4), "{v:?}");
assert_eq!(read, 0);
// The same for an array source: a whole column of a 3x3 array.
let (v, _) = eval("=INDEX({1,2,3;4,5,6;7,8,9},{0,0},2)");
assert!(all_calc(&v, 2), "{v:?}");
// Single cells are still read.
let (v, _) = eval("=INDEX({1,2,3;4,5,6;7,8,9},{1,3},2)");
assert!(matches!(&v, Value::Array(a) if a.data == vec![Value::number(2.0), Value::number(8.0)]), "{v:?}");
}
}
+112 -2
View File
@@ -1826,6 +1826,54 @@ fn trim_references() {
assert_eq!(t.get("K20"), Value::Error(CellError::Calc));
}
#[test]
fn lookup_of_an_empty_cell_stays_blank() {
// #374: B1 (the value for key "k") is empty.
let mut t = T::new();
t.set("A1", "k");
t.set("A2", "j");
t.set("B2", "v");
for (at, f, want) in [
("D2", "=\"x\"&VLOOKUP(\"k\",A1:B2,2,0)&\"y\"", Value::from("xy")),
("D3", "=CONCATENATE(\"x\",VLOOKUP(\"k\",A1:B2,2,0),\"y\")", Value::from("xy")),
("D4", "=ISBLANK(XLOOKUP(\"k\",A1:A2,B1:B2))", Value::Bool(true)),
("D5", "=\"x\"&XLOOKUP(\"k\",A1:A2,B1:B2)&\"y\"", Value::from("xy")),
// A bare lookup still shows 0, like a reference to an empty cell.
("D8", "=VLOOKUP(\"k\",A1:B2,2,0)", Value::Number(0.0)),
// Arithmetic still reads the blank as 0.
("D9", "=VLOOKUP(\"k\",A1:B2,2,0)+1", Value::Number(1.0)),
// A blank lookup given as an argument is still given, not omitted.
("D10", "=FILTER(1,FALSE,VLOOKUP(\"k\",A1:B2,2,FALSE))", Value::Number(0.0)),
("D11", "=XLOOKUP(\"z\",A1:A2,B1:B2,VLOOKUP(\"k\",A1:B2,2,0))", Value::Number(0.0)),
// The same holds per element when the call is lifted over an array of blank lookups.
("D12", "=CEILING.MATH(6.7,VLOOKUP(\"k\",A1:B2,2,0))", Value::Number(0.0)),
("D13", "=SUM(CEILING.MATH(6.7,VLOOKUP({\"k\",\"k\"},A1:B2,2,0)))", Value::Number(0.0)),
("D14", "=FLOOR.MATH(6.7,VLOOKUP(\"k\",A1:B2,2,0))", Value::Number(0.0)),
("D15", "=SUM(FLOOR.MATH(6.7,VLOOKUP({\"k\",\"k\"},A1:B2,2,0)))", Value::Number(0.0)),
] {
t.set(at, f);
assert_eq!(t.get(at), want, "{f}");
}
}
#[test]
fn blank_lookup_values_match_excel_windows_observation() {
// Excel 365 Windows 2609, build 16.0.20430.20032, black-box check 2026-10-13:
// A1="k", B1 and A2 genuinely empty.
// All three value lookups preserve blankness inside ISBLANK/concatenation.
let mut t = T::new();
t.set("A1", "k");
for (at, f, want) in [
("D1", "=ISBLANK(VLOOKUP(\"k\",A1:B1,2,FALSE))", Value::Bool(true)),
("D2", "=ISBLANK(HLOOKUP(\"k\",A1:A2,2,FALSE))", Value::Bool(true)),
("D3", "=ISBLANK(LOOKUP(\"k\",A1:A1,B1:B1))", Value::Bool(true)),
("D4", "=\"x\"&HLOOKUP(\"k\",A1:A2,2,FALSE)&\"y\"", Value::from("xy")),
] {
t.set(at, f);
assert_eq!(t.get(at), want, "{f}");
}
}
#[test]
fn cell_width_is_the_column_width_dialog_value_rounded() {
// #735: Excel 365 values. Widths in px as `home.columnWidth {"chars"}` sets them
@@ -1922,6 +1970,54 @@ fn offset_extreme_arguments_return_ref() {
}
}
/// `@` intersects a reference with the formula's row (a column range) or column (a row range);
/// a 2D range both (circular when it includes the formula's cell); a range the formula does not
/// line up with gives #VALUE!. An array gives its top-left value. Legacy formulas read from files use it where Excel intersects implicitly.
#[test]
fn implicit_intersection_operator() {
let mut t = T::new();
for (at, v) in [("A1", "1"), ("A2", "2"), ("A3", "3"), ("A5", "4"), ("B5", "5"), ("C5", "6")] {
t.set(at, v);
}
t.set("B2", "=TANH(@A1:A3)");
t.set("B7", "=TANH(@A5:C5)");
t.set("B3", "=@A1:C3");
t.set("B9", "=@A1:A3");
t.set("E9", "=@A5:C5");
t.set("D1", "=@{7,8;9,10}");
t.set("D2", "=@A2");
t.set("D3", "=SUM(@A1:A3)");
assert_eq!(t.num("B2"), 2f64.tanh());
assert_eq!(t.num("B7"), 5f64.tanh());
for at in ["B9", "E9"] {
assert_eq!(t.get(at), Value::Error(CellError::Value), "{at}");
}
// A 2D range that includes the formula's own cell is a circular reference (Excel shows 0).
assert_eq!(t.get("B3"), Value::Error(CellError::Circ));
// From another sheet a 2D range intersects on both row and column; outside it is #VALUE!.
t.wb.sheets.push(std::sync::Arc::new(gridcraft_model::Sheet::new("Other")));
for (at, input) in [("B2", "=@Sheet1!A1:C3"), ("C1", "=@Sheet1!A1:C3"), ("B9", "=@Sheet1!A1:C3"), ("E2", "=@Sheet1!A1:C3")] {
t.wb.sheet_mut(1).unwrap().cells.set(c(at), Cell::formula(Formula::new(input)));
}
t.calc.recalc_all(&mut t.wb);
let other = |at: &str| t.wb.sheet(1).unwrap().value(c(at));
assert_eq!(other("B2"), Value::Number(2f64.tanh())); // Sheet1!B2
assert_eq!(other("C1"), Value::Number(0.0));
assert_eq!(other("B9"), Value::Error(CellError::Value));
assert_eq!(other("E2"), Value::Error(CellError::Value));
assert_eq!(t.num("D1"), 7.0);
assert_eq!(t.num("D2"), 2.0);
assert_eq!(t.num("D3"), 3.0);
// An array result shows its top-left value (T of a reference reads its top-left cell).
t.set("A11", "x");
t.set("C11", "y");
t.set("E1", "=@T(A11:C11)");
assert_eq!(t.get("E1"), Value::text("x"));
// Without `@` (a dynamic-array formula) the same formula spills.
t.set("F1", "=TANH(A1:A3)");
assert_eq!((t.num("F1"), t.num("F2"), t.num("F3")), (1f64.tanh(), 2f64.tanh(), 3f64.tanh()));
}
/// "Set precision as displayed": results are rounded to their cell's format before they are
/// stored and before other formulas read them. Values observed in Excel (COM,
/// `Workbook.PrecisionAsDisplayed = True`) with the same synthetic formulas.
@@ -1987,7 +2083,6 @@ fn precision_as_displayed_rounds_results_by_cell_format() {
t.calc.recalc_all(&mut t.wb);
assert_eq!(t.num("A2"), (-130.0 + 657.41) * 10.0);
}
#[test]
fn analysis_toolpak_value_params_reject_multi_cell_references() {
// Excel 365 (observed): EOMONTH, WORKDAY, COUPDAYS, BIN2DEC... give #VALUE! for a reference to more
@@ -2072,7 +2167,6 @@ fn analysis_toolpak_value_params_reject_multi_cell_references() {
t.set("H5", "");
}
}
/// A formula reading one further down (or on a later sheet) is calculated on demand, inside the
/// formula that reads it. Chains of thousands of such formulas, common in financial models, used
/// to overflow the stack (or give `#CIRC!` past 2,000 levels); they now calculate in full.
@@ -2786,3 +2880,19 @@ fn formulas_read_a_dirty_python_cell_after_it_is_laid_out() {
assert_eq!((t.num("B1"), t.num("B2")), (5.0, 6.0));
assert_eq!((t.num("A1"), t.num("A2"), t.num("A3")), (5.0, 100.0, 111.0));
}
/// A computed criterion whose formula runs a database function over the same criteria counts as
/// not met inside itself: before, each record's check ran the function again for every record, at
/// every level, so the work grew exponentially with the nesting allowed.
#[test]
fn computed_criterion_that_reenters_itself_is_not_met() {
let mut t = T::new();
for (r, (name, age)) in [("name", "age"), ("a", "1"), ("b", "2"), ("c", "3"), ("d", "4"), ("e", "5")].iter().enumerate() {
t.set(&format!("F{}", r + 2), name);
t.set(&format!("G{}", r + 2), age);
}
t.set("N46", "Formula");
t.set("N47", "=DCOUNT($F$2:$G$7,\"age\",$N$46:$N$47)>0");
t.set("B60", "=DCOUNT($F$2:$G$7,\"age\",N46:N47)");
assert_eq!(t.num("B60"), 0.0);
}
+84 -7
View File
@@ -7,6 +7,20 @@ use crate::{Prim, Rgba};
/// Largest image we allocate (pixels).
const MAX_PIXELS: u64 = 64 * 1024 * 1024;
const SUB: usize = 4;
/// Scan-conversion work one [`rasterize`] call may do: edge tests (an active edge at one
/// sub-sample) plus covered pixels blended. Edges are only tested on the rows they cross, but a
/// caller can still pass enough geometry to keep the rasteriser busy for minutes on a small
/// image; once the allowance is spent the remaining fills are skipped. 2^28 is about a second,
/// far beyond any chart.
const MAX_SCAN_WORK: u64 = 1 << 28;
/// Most points one filled path may have: more is skipped before any edge is built. Chart
/// renderers cap and decimate their paths far below this; it only bounds direct callers.
const MAX_FILL_POINTS: usize = 1 << 22;
/// Most points one stroked polyline may have: more is skipped before any segment path is built
/// (each segment becomes its own quad). Charts hold at most 100,000 points per series and draw far
/// fewer; it only bounds direct callers.
const MAX_STROKE_POINTS: usize = 1 << 17;
type Path = Vec<Vec<[f32; 2]>>;
@@ -15,6 +29,8 @@ struct Canvas {
h: usize,
buf: Vec<u8>,
acc: Vec<f32>,
/// Scan-conversion work left (see [`MAX_SCAN_WORK`]).
work: u64,
}
impl Canvas {
@@ -41,7 +57,12 @@ impl Canvas {
/// Fills a multi-contour path with the even-odd rule.
fn fill(&mut self, path: &Path, color: Rgba) {
let mut edges: Vec<[f32; 4]> = Vec::new();
// Out of work, or more geometry than the allowance could ever scan: no edge list is built.
if self.work == 0 || path.iter().map(Vec::len).fold(0usize, usize::saturating_add) > MAX_FILL_POINTS {
return;
}
// Each edge as [top y, bottom y, x0, y0, x1, y1].
let mut edges: Vec<[f32; 6]> = Vec::new();
let (mut minx, mut miny, mut maxx, mut maxy) = (f32::INFINITY, f32::INFINITY, f32::NEG_INFINITY, f32::NEG_INFINITY);
for c in path {
if c.len() < 3 || !c.iter().all(|p| p[0].is_finite() && p[1].is_finite()) {
@@ -54,7 +75,7 @@ impl Canvas {
miny = miny.min(p[1]);
maxy = maxy.max(p[1]);
if p[1] != q[1] {
edges.push([p[0], p[1], q[0], q[1]]);
edges.push([p[1].min(q[1]), p[1].max(q[1]), p[0], p[1], q[0], q[1]]);
}
}
}
@@ -72,8 +93,28 @@ impl Canvas {
if self.acc.len() < bw + 1 {
self.acc.resize(bw + 1, 0.0);
}
// Active-edge list: edges join when the scan reaches their top and leave past their
// bottom, so each row tests only the edges crossing it instead of every edge.
edges.sort_by(|a, b| a[0].total_cmp(&b[0]));
let mut pending = edges.iter().peekable();
let mut active: Vec<[f32; 6]> = Vec::new();
let mut xs: Vec<f32> = Vec::new();
for py in y0..y1 {
while let Some(e) = pending.next_if(|e| e[0] < py as f32 + 1.0) {
active.push(*e);
}
active.retain(|e| e[1] > py as f32);
if active.is_empty() {
continue;
}
// The row's edge tests and its `bw` accumulator clears and blends, charged before any of
// them, so rows with no coverage still count.
let cost = (active.len() * SUB) as u64 + bw as u64;
if cost > self.work {
self.work = 0;
return;
}
self.work -= cost;
for v in self.acc.iter_mut().take(bw) {
*v = 0.0;
}
@@ -81,10 +122,9 @@ impl Canvas {
for s in 0..SUB {
let sy = py as f32 + (s as f32 + 0.5) / SUB as f32;
xs.clear();
for e in &edges {
let (ya, yb) = if e[1] < e[3] { (e[1], e[3]) } else { (e[3], e[1]) };
if sy >= ya && sy < yb {
xs.push(e[0] + (sy - e[1]) * (e[2] - e[0]) / (e[3] - e[1]));
for e in &active {
if sy >= e[0] && sy < e[1] {
xs.push(e[2] + (sy - e[3]) * (e[4] - e[2]) / (e[5] - e[3]));
}
}
xs.sort_by(f32::total_cmp);
@@ -154,6 +194,9 @@ fn rounded_rect(x: f32, y: f32, w: f32, h: f32, r: f32) -> Vec<[f32; 2]> {
/// Thick polyline → one quad per segment (each a separate path so overlaps don't cancel).
fn stroke_paths(pts: &[[f32; 2]], width: f32, dash: bool, closed: bool) -> Vec<Path> {
if pts.len() > MAX_STROKE_POINTS {
return Vec::new();
}
let width = width.clamp(1.0, 50.0);
let mut segs: Vec<([f32; 2], [f32; 2])> = pts.windows(2).map(|w| (w[0], w[1])).collect();
if closed && let (Some(a), Some(b)) = (pts.last(), pts.first()) {
@@ -212,7 +255,7 @@ pub fn rasterize(prims: &[Prim], w: u32, h: u32, background: Rgba) -> Vec<u8> {
for _ in 0..w * h {
buf.extend_from_slice(&background);
}
let mut cv = Canvas { w, h, buf, acc: Vec::new() };
let mut cv = Canvas { w, h, buf, acc: Vec::new(), work: MAX_SCAN_WORK };
for p in prims {
match p {
Prim::Rect { x, y, w, h, fill, stroke, radius } => {
@@ -274,6 +317,40 @@ pub fn rasterize(prims: &[Prim], w: u32, h: u32, background: Rgba) -> Vec<u8> {
mod tests {
use super::*;
/// Security: each row tests only the edges crossing it. A polygon of 100,000 tiny edges over
/// 4,000 rows tested every edge at every sub-sample of every row (1.6 billion tests).
#[test]
fn many_short_edges_scan_only_their_rows() {
let mut pts: Vec<[f32; 2]> = (0..100_000).map(|i| [if i % 2 == 0 { 0.0 } else { 0.5 }, i as f32 * 0.04]).collect();
pts.push([2.0, 4_000.0]);
pts.push([2.0, 0.0]);
let img = rasterize(&[Prim::Polygon { pts, fill: [255, 0, 0, 255], stroke: None }], 2, 4_000, [0, 0, 0, 0]);
// The right-hand pixel of every row is inside the polygon.
assert_eq!(img.get(4..8), Some(&[255, 0, 0, 255][..]));
assert_eq!(img.get(4 * (2 * 3_999 + 1)..4 * (2 * 3_999 + 2)), Some(&[255, 0, 0, 255][..]));
}
/// Security: scan conversion stops once the call's work allowance is spent.
#[test]
fn fills_stop_when_the_work_allowance_is_spent() {
let square = vec![vec![[0.0, 0.0], [4.0, 0.0], [4.0, 4.0], [0.0, 4.0]]];
let mut cv = Canvas { w: 4, h: 4, buf: vec![0; 64], acc: Vec::new(), work: 2 * SUB as u64 * 4 + 16 };
cv.fill(&square, [255; 4]);
assert!(cv.buf.iter().all(|&b| b == 255));
cv.buf = vec![0; 64];
cv.fill(&square, [255; 4]);
assert!(cv.buf.iter().all(|&b| b == 0));
}
/// Security: a polyline over the point cap builds no segment paths at all, solid or dashed.
#[test]
fn oversized_polylines_build_no_segment_paths() {
let line = |n: usize| -> Vec<[f32; 2]> { (0..n).map(|i| [i as f32, (i % 2) as f32]).collect() };
assert!(stroke_paths(&line(MAX_STROKE_POINTS + 1), 1.0, false, false).is_empty());
assert!(stroke_paths(&line(MAX_STROKE_POINTS + 1), 1.0, true, false).is_empty());
assert_eq!(stroke_paths(&line(MAX_STROKE_POINTS), 1.0, false, false).len(), MAX_STROKE_POINTS - 1);
}
#[test]
fn dashed_geometry_stays_within_the_dash_budget() {
// Forty very long dashed lines, each allowed 10_000 dashes per segment: before the
+34 -6
View File
@@ -136,7 +136,13 @@ pub(super) fn decimate_area(top: Vec<[f32; 2]>, base: Vec<[f32; 2]>, max: usize)
/// Draws a polyline split at gaps (`None`), decimated when long. With `smooth` (the plot box),
/// each run is interpolated with a Catmull-Rom spline so the line curves through the data points.
///
/// The points of the smoothed runs (data points included) are budgeted across all the runs of the line, not per run:
/// short runs between gaps escape decimation, and 100 sub-segments each would turn a
/// 100,000-point series of three-point runs into millions of points. Past the budget the
/// remaining runs are drawn through their data points only.
pub(super) fn draw_runs(ctx: &mut Ctx, pts: &[Option<[f32; 2]>], color: Rgba, width: f32, smooth: Option<Bx>) {
let mut budget = MAX_LINE_PTS * 2;
let mut run: Vec<[f32; 2]> = Vec::new();
for p in pts.iter().chain(std::iter::once(&None)) {
match p {
@@ -145,7 +151,11 @@ pub(super) fn draw_runs(ctx: &mut Ctx, pts: &[Option<[f32; 2]>], color: Rgba, wi
if run.len() >= 2 {
let r = decimate(std::mem::take(&mut run), MAX_LINE_PTS);
let r = match smooth {
Some(plot) => smooth_run(r, plot),
Some(plot) => {
let r = smooth_run(r, plot, budget);
budget = budget.saturating_sub(r.len());
r
}
None => r,
};
ctx.line(r, color, width);
@@ -158,15 +168,15 @@ pub(super) fn draw_runs(ctx: &mut Ctx, pts: &[Option<[f32; 2]>], color: Rgba, wi
}
/// Catmull-Rom spline through `pts`: every segment gets the same number of sub-segments (up to
/// 100), chosen so the whole curve stays within `MAX_LINE_PTS * 2` points. Endpoints are
/// 100), chosen so the whole curve stays within `cap` points (one per data point when `cap` is
/// smaller than that). Endpoints are
/// duplicated so the curve passes through the first and last points, and the curve is clamped
/// to the `plot` box (a spline overshoots its data near sharp turns).
fn smooth_run(pts: Vec<[f32; 2]>, plot: Bx) -> Vec<[f32; 2]> {
fn smooth_run(pts: Vec<[f32; 2]>, plot: Bx, cap: usize) -> Vec<[f32; 2]> {
let n = pts.len();
if n < 3 {
return pts;
}
let cap = MAX_LINE_PTS * 2;
let steps = (cap / n).clamp(1, 100);
let mut out: Vec<[f32; 2]> = Vec::with_capacity(n.saturating_mul(steps).min(cap) + 1);
let at = |i: isize| -> [f32; 2] {
@@ -219,7 +229,7 @@ mod smooth_tests {
fn long_runs_are_smoothed_uniformly_within_the_cap() {
let plot = Bx { x: 0.0, y: 0.0, w: 3000.0, h: 100.0 };
let pts: Vec<[f32; 2]> = (0..1500).map(|i| [i as f32 * 2.0, if i % 2 == 0 { 20.0 } else { 80.0 }]).collect();
let out = smooth_run(pts, plot);
let out = smooth_run(pts, plot, MAX_LINE_PTS * 2);
assert!(out.len() <= MAX_LINE_PTS * 2 + 1, "{}", out.len());
// Two sub-segments per segment all the way to the end, not 100 each and then a straight tail.
assert_eq!(out.len(), 1499 * 2 + 1);
@@ -231,11 +241,29 @@ mod smooth_tests {
let plot = Bx { x: 0.0, y: 10.0, w: 300.0, h: 100.0 };
// A step right at the plot edges: Catmull-Rom overshoots past both.
let pts = vec![[0.0, 110.0], [100.0, 110.0], [110.0, 10.0], [200.0, 10.0], [300.0, 10.0]];
let out = smooth_run(pts, plot);
let out = smooth_run(pts, plot, MAX_LINE_PTS * 2);
assert!(out.len() > 5);
assert!(out.iter().all(|p| (0.0..=300.0).contains(&p[0]) && (10.0..=110.0).contains(&p[1])));
}
/// Security: smoothing budgets its added points across every run of a line. A series of
/// three-point runs between gaps used to get 100 sub-segments per run, 67 times its data.
#[test]
fn gap_separated_runs_share_one_smoothing_budget() {
let plot = Bx { x: 0.0, y: 0.0, w: 1e6, h: 100.0 };
let pts: Vec<Option<[f32; 2]>> =
(0..100_000).map(|i| if i % 4 == 3 { None } else { Some([i as f32, if i % 2 == 0 { 20.0 } else { 80.0 }]) }).collect();
let m = crate::ApproxMeasure;
let mut ctx = Ctx { out: Vec::new(), m: &m };
draw_runs(&mut ctx, &pts, WHITE, 1.0, Some(plot));
let total: usize = ctx.out.iter().map(|p| if let Prim::Line { pts, .. } = p { pts.len() } else { 0 }).sum();
// 75,000 data points, plus at most the budget added by interpolation.
assert!(total <= 75_000 + MAX_LINE_PTS * 2, "{total}");
assert_eq!(ctx.out.len(), 25_000);
// The first runs are still smoothed.
assert!(matches!(ctx.out.first(), Some(Prim::Line { pts, .. }) if pts.len() > 3));
}
#[test]
fn decimate_area_preserves_paired_vertical_strips() {
let top: Vec<[f32; 2]> = (0..1000).map(|i| [i as f32 * 0.25, ((i * 37) % 100) as f32]).collect();
+103 -16
View File
@@ -184,6 +184,53 @@ fn value_text(v: &Value) -> String {
}
}
/// Characters kept of one category label: an axis label never shows more (labels are fitted to
/// the axis), and copying more only costs memory.
const MAX_LABEL_CHARS: usize = 255;
/// Bytes of category label text kept per series. One long text broadcast by a short formula
/// (`IF(SEQUENCE(100000), long_text)`) is a single shared value, but turning every element into
/// its own label copied it 100,000 times (gigabytes). Labels past the budget are empty.
const MAX_CATEGORY_BYTES: usize = 8 << 20;
/// At most [`MAX_LABEL_CHARS`] characters and `room` bytes of `s`, cut at a character boundary.
fn clip_label(s: &str, room: usize) -> String {
let mut end = 0;
for (n, (i, c)) in s.char_indices().enumerate() {
if n >= MAX_LABEL_CHARS || i + c.len_utf8() > room {
break;
}
end = i + c.len_utf8();
}
s.get(..end).unwrap_or_default().to_string()
}
/// The label of category `i` (value `v`) in at most `room` bytes: formatted with its source
/// cell's number format when it is a number.
fn category_label(wb: &Workbook, src: Option<(&gridcraft_model::Sheet, RangeRef)>, i: usize, v: &Value, room: usize) -> String {
if room == 0 {
return String::new(); // nothing fits: skip the number formatting
}
if let (Some((s, r)), Value::Number(_)) = (src, v) {
let w = r.width().max(1) as usize;
let at = r.start.offset((i / w) as i64, (i % w) as i64);
if let Some(c) = at {
let code = &wb.styles.get(s.style_id(c)).num_fmt.0;
if !code.eq_ignore_ascii_case("general") {
return clip_label(&format_value(v, &NumberFormat::parse_cached(code), wb.date_system).text, room);
}
}
}
// Text is clipped before it is copied: the source may be one long shared string.
let mut v = v;
while let Value::Array(a) = v {
v = a.get(0, 0).unwrap_or(&Value::Empty);
}
match v {
Value::Text(t) => clip_label(t, room),
_ => clip_label(&value_text(v), room),
}
}
/// Category labels, formatted with each source cell's number format (dates, currency…), and
/// the outer levels of a multi-level axis: a category range wider than a column of values (or
/// taller than a row of values) groups its innermost labels, year beside month. Each outer level
@@ -221,29 +268,29 @@ fn category_table(wb: &Workbook, sheet: usize, formula: &str, values: &str, limi
None => eval_list_in(wb, sheet, formula, limit, 0),
};
let src = cells.and_then(|(sh, r)| wb.sheet(sh).map(|s| (s, r)));
// The retained levels are known before any label is charged: a cell on a discarded level
// gets no label and spends none of the budget. `None`: a flat list (every cell is a label).
let layout = src.and_then(|(_, r)| {
let levels = levels?;
let (w, h) = (r.width().max(1) as usize, r.height().max(1) as usize);
(vals.len() == w.saturating_mul(h)).then_some((levels, w, h)) // else truncated or not a plain block
});
let kept =
|i: usize| layout.is_none_or(|(l, w, h)| if l.by_columns { i % w + 1 + l.outer as usize >= w } else { i / w + 1 + l.outer as usize >= h });
let mut room = MAX_CATEGORY_BYTES;
let texts: Vec<String> = vals
.iter()
.enumerate()
.map(|(i, v)| {
if let (Some((s, r)), Value::Number(_)) = (src, v) {
let w = r.width().max(1) as usize;
let at = r.start.offset((i / w) as i64, (i % w) as i64);
if let Some(c) = at {
let code = &wb.styles.get(s.style_id(c)).num_fmt.0;
if !code.eq_ignore_ascii_case("general") {
return format_value(v, &NumberFormat::parse_cached(code), wb.date_system).text;
}
}
if !kept(i) {
return String::new();
}
value_text(v)
let label = category_label(wb, src, i, v, room);
room = room.saturating_sub(label.len());
label
})
.collect();
let Some((_, r)) = src else { return (texts, vec![]) };
let Some(levels) = levels else { return (texts, vec![]) };
let (w, h) = (r.width().max(1) as usize, r.height().max(1) as usize);
if texts.len() != w.saturating_mul(h) {
return (texts, vec![]); // truncated or not a plain block: a flat list
}
let Some((levels, w, h)) = layout else { return (texts, vec![]) };
let at = |row: usize, col: usize| texts.get(row * w + col).cloned().unwrap_or_default();
let column = |c: usize| (0..h).map(|row| at(row, c)).collect::<Vec<String>>();
let row = |rr: usize| (0..w).map(|c| at(rr, c)).collect::<Vec<String>>();
@@ -369,6 +416,46 @@ pub fn resolve(wb: &Workbook, sheet: usize, chart: &Chart) -> ChartData {
ChartData { categories, category_levels, series, palette }
}
#[cfg(test)]
mod label_budget_tests {
use gridcraft_model::Workbook;
/// Security: category labels are clipped before they are copied, with one byte budget per
/// series. A formula broadcasting one long text to 100,000 categories used to copy it
/// 100,000 times (3.3 GB).
#[test]
fn category_labels_share_one_byte_budget() {
let wb = Workbook::new();
let (texts, _) = super::category_table(&wb, 0, r#"=IF(SEQUENCE(100000),REPT("x",32767))"#, "", super::MAX_POINTS);
assert_eq!(texts.len(), 100_000);
assert_eq!(texts.first().map(String::len), Some(super::MAX_LABEL_CHARS));
assert!(texts.iter().map(String::len).sum::<usize>() <= super::MAX_CATEGORY_BYTES);
// Labels are cut at character boundaries.
assert_eq!(super::clip_label("aé", 2), "a");
assert_eq!(super::clip_label("aé", 3), "aé");
}
/// Cells on discarded outer levels (only the innermost labels and three outer levels are
/// kept) spend none of the byte budget.
#[test]
fn discarded_levels_do_not_spend_the_label_budget() {
let mut wb = Workbook::new();
let rows = 8000u32;
if let Some(sh) = wb.sheet_mut(0) {
for r in 0..rows {
for c in 0..5 {
sh.set_value(gridcraft_core::CellRef::new(r, c), gridcraft_core::Value::text("x".repeat(255)));
}
}
}
// Five columns: column A is a fourth outer level and is discarded.
let (inner, outer) = super::category_table(&wb, 0, "=A1:E8000", "", super::MAX_POINTS);
assert_eq!(outer.len(), 3);
assert_eq!(inner.last().map(String::len), Some(super::MAX_LABEL_CHARS));
assert_eq!(outer.first().and_then(|l| l.last()).map(String::len), Some(super::MAX_LABEL_CHARS));
}
}
#[cfg(test)]
mod tests {
use gridcraft_model::Workbook;
+3 -1
View File
@@ -1,5 +1,5 @@
//! GridCraft core types: cell addresses, values and Excel's coercion rules, error values,
//! date serials and typed-input parsing. No dependencies on the rest of the workspace.
//! date serials, regional formats and typed-input parsing. No dependencies on the rest of the workspace.
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
#![forbid(unsafe_code)]
@@ -7,6 +7,7 @@ pub mod addr;
pub mod color;
pub mod date;
pub mod parse;
pub mod region;
pub mod value;
pub use addr::{
@@ -14,4 +15,5 @@ pub use addr::{
split_areas, split_sheet, unquote_sheet,
};
pub use date::DateSystem;
pub use region::Region;
pub use value::{Array, CellError, Value, compare, compare_numbers, compare_text, number_to_text, sort_compare};
+883
View File
@@ -0,0 +1,883 @@
//! Regional formats: how a region writes numbers, dates, times and money.
//!
//! Excel takes these from the operating system's regional settings; they are independent of the
//! interface language and of the formula language. A [`Region`] changes what people *type* into
//! a cell (`1.234,56`, `25/09/2026`) and what cells *show* (separators, month and day names,
//! AM/PM, the system short and long dates). It never changes what is stored: values are numbers,
//! and the number format codes saved in a file stay in their canonical en-US spelling.
//!
//! [`EN_US`] is the default and reproduces GridCraft's original behaviour exactly.
//!
//! # Where the values come from
//! The separators, month and weekday names, AM/PM words, currency symbols and the order of day,
//! month and year are the Unicode CLDR conventions (`unicode-org/cldr-json`:
//! `cldr-numbers-full/main/<locale>/numbers.json` and `cldr-dates-full/main/<locale>/ca-gregorian.json`),
//! which is also what macOS, Android and ICU ship. They were read from those files for en-GB,
//! en-IN, cs and ru, read from ICU's CLDR 48 data for the Swiss regions and nl-NL, and written
//! from the same conventions for the other regions. CLDR data is under the Unicode License v3,
//! reproduced in `NOTICE` (row in `ATTRIBUTION.md`).
//! Deliberate choices where operating systems differ from CLDR:
//! - Short dates use a four-digit year (`dd/mm/yyyy`, where CLDR has `dd/MM/y` or `dd/MM/yy`):
//! Excel's short date is the system's, Windows writes four digits, and a two-digit year would
//! not read back to the same date.
//! - The long date is CLDR's *full* date (with the weekday), like Windows en-US, except where
//! Windows' long date differs: Russian, Japanese and Chinese have no weekday (`23 августа
//! 2005 г.`, `2005年8月23日`), as in Microsoft's National Language Support tables.
//! - Korean writes the short date `yyyy-mm-dd` (CLDR: `yy. M. d.`), also as in Microsoft's
//! National Language Support tables.
//! - Month names for `mmm` are CLDR's abbreviated names (so en-GB has `Sept`), except
//! Portuguese, which drops CLDR's period (`jan`, not `jan.`) as Excel in Brazil shows it.
//! - French and Portuguese (Portugal) group digits with U+00A0 (Windows), not CLDR's U+202F.
//! Parsing accepts every space-like grouping character in those regions.
//! - The Swiss regions (en-CH, de-CH, fr-CH, it-CH; Liechtenstein uses de-CH) group digits with
//! CLDR's ASCII apostrophe (`1'234.50`); parsing also accepts the typographic `’` that glibc
//! uses. French (Switzerland) writes a decimal point like the other Swiss regions and glibc's
//! `fr_CH`, where CLDR has a comma.
//!
//! Regions this module doesn't list (`de-AT`, `es-MX`, `en-CA`…) resolve to the main region of
//! their language through [`lookup`]; their own conventions (a Mexican decimal point) are not
//! modelled.
/// How digits are grouped.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Grouping {
/// Groups of three (`1,234,567`).
Thousands,
/// The Indian system: three digits, then groups of two (`12,34,567`).
Indian,
}
/// The order of the parts of a short date.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum DateOrder {
Mdy,
Dmy,
Ymd,
}
/// One region's conventions. See the module documentation.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct Region {
/// BCP 47 tag, `pt-BR`.
pub tag: &'static str,
/// The region's name in its own language.
pub name: &'static str,
/// Windows locale id, used by `[$-409]`-style format tags.
pub lcid: u16,
pub decimal: char,
pub group: char,
pub grouping: Grouping,
pub date_order: DateOrder,
pub date_sep: char,
pub time_sep: char,
/// The system short date as a canonical format code (`dd/mm/yyyy`): built-in format 14.
pub short_date: &'static str,
/// The system long date (`[$-F800]`).
pub long_date: &'static str,
/// The system short time. It contains `AM/PM` in regions that use a 12-hour clock.
pub short_time: &'static str,
/// The system long time (`[$-F400]`).
pub long_time: &'static str,
/// Built-in format 22: the short date and a 24-hour time.
pub datetime: &'static str,
pub months: [&'static str; 12],
pub months_abbr: [&'static str; 12],
/// The month names a date with a day uses (Russian `1 января`, Czech `1. ledna`), where they
/// differ from `months`.
pub months_genitive: Option<[&'static str; 12]>,
/// Sunday first.
pub weekdays: [&'static str; 7],
pub weekdays_abbr: [&'static str; 7],
pub am_pm: [&'static str; 2],
pub currency: &'static str,
/// The currency format with the currency's usual decimals (two, or none for the yen and the
/// won), as a canonical code. Where it starts with a quote the symbol comes before the number,
/// otherwise after.
pub currency_format: &'static str,
/// The same with no decimals (for a typed whole amount).
pub currency_format_int: &'static str,
}
const EN_MONTHS: [&str; 12] =
["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"];
const EN_MONTHS_ABBR: [&str; 12] = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"];
const EN_DAYS: [&str; 7] = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"];
const EN_DAYS_ABBR: [&str; 7] = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"];
const AM_PM: [&str; 2] = ["AM", "PM"];
/// English (United States): the default. Identical to the formats before regions existed.
pub static EN_US: Region = Region {
tag: "en-US",
name: "English (United States)",
lcid: 0x0409,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Mdy,
date_sep: '/',
time_sep: ':',
short_date: "m/d/yyyy",
long_date: "dddd, mmmm dd, yyyy",
short_time: "h:mm AM/PM",
long_time: "h:mm:ss AM/PM",
datetime: "m/d/yyyy h:mm",
months: EN_MONTHS,
months_abbr: EN_MONTHS_ABBR,
months_genitive: None,
weekdays: EN_DAYS,
weekdays_abbr: EN_DAYS_ABBR,
am_pm: AM_PM,
currency: "$",
currency_format: "\"$\"#,##0.00",
currency_format_int: "\"$\"#,##0",
};
static EN_GB: Region = Region {
tag: "en-GB",
name: "English (United Kingdom)",
lcid: 0x0809,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: EN_MONTHS,
months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"],
months_genitive: None,
weekdays: EN_DAYS,
weekdays_abbr: EN_DAYS_ABBR,
am_pm: AM_PM,
currency: "£",
currency_format: "\"£\"#,##0.00",
currency_format_int: "\"£\"#,##0",
};
static EN_IN: Region = Region {
tag: "en-IN",
name: "English (India)",
lcid: 0x4009,
decimal: '.',
group: ',',
grouping: Grouping::Indian,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "h:mm AM/PM",
long_time: "h:mm:ss AM/PM",
datetime: "dd/mm/yyyy h:mm",
months: EN_MONTHS,
months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"],
months_genitive: None,
weekdays: EN_DAYS,
weekdays_abbr: EN_DAYS_ABBR,
am_pm: AM_PM,
currency: "₹",
currency_format: "\"₹\"#,##0.00",
currency_format_int: "\"₹\"#,##0",
};
static EN_CH: Region = Region {
tag: "en-CH",
name: "English (Switzerland)",
// English (Switzerland) has no LCID of its own: Windows reports LOCALE_CUSTOM_UNSPECIFIED.
lcid: 0x1000,
decimal: '.',
group: '\'',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: EN_MONTHS,
months_abbr: ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sept", "Oct", "Nov", "Dec"],
months_genitive: None,
weekdays: EN_DAYS,
weekdays_abbr: EN_DAYS_ABBR,
am_pm: AM_PM,
currency: "CHF",
currency_format: "\"CHF\" #,##0.00",
currency_format_int: "\"CHF\" #,##0",
};
static ES_ES: Region = Region {
tag: "es-ES",
name: "Español (España)",
lcid: 0x0C0A,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d \"de\" mmmm \"de\" yyyy",
short_time: "h:mm",
long_time: "h:mm:ss",
datetime: "dd/mm/yyyy h:mm",
months: ["enero", "febrero", "marzo", "abril", "mayo", "junio", "julio", "agosto", "septiembre", "octubre", "noviembre", "diciembre"],
months_abbr: ["ene", "feb", "mar", "abr", "may", "jun", "jul", "ago", "sep", "oct", "nov", "dic"],
months_genitive: None,
weekdays: ["domingo", "lunes", "martes", "miércoles", "jueves", "viernes", "sábado"],
weekdays_abbr: ["dom", "lun", "mar", "mié", "jue", "vie", "sáb"],
am_pm: ["a. m.", "p. m."],
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static DE_DE: Region = Region {
tag: "de-DE",
name: "Deutsch (Deutschland)",
lcid: 0x0407,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d. mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["Januar", "Februar", "März", "April", "Mai", "Juni", "Juli", "August", "September", "Oktober", "November", "Dezember"],
months_abbr: ["Jan", "Feb", "Mär", "Apr", "Mai", "Jun", "Jul", "Aug", "Sep", "Okt", "Nov", "Dez"],
months_genitive: None,
weekdays: ["Sonntag", "Montag", "Dienstag", "Mittwoch", "Donnerstag", "Freitag", "Samstag"],
weekdays_abbr: ["So", "Mo", "Di", "Mi", "Do", "Fr", "Sa"],
am_pm: AM_PM,
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static DE_CH: Region = Region {
tag: "de-CH",
name: "Deutsch (Schweiz)",
lcid: 0x0807,
decimal: '.',
group: '\'',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d. mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["Januar", "Februar", "März", "April", "Mai", "Juni", "Juli", "August", "September", "Oktober", "November", "Dezember"],
months_abbr: ["Jan", "Feb", "Mär", "Apr", "Mai", "Jun", "Jul", "Aug", "Sep", "Okt", "Nov", "Dez"],
months_genitive: None,
weekdays: ["Sonntag", "Montag", "Dienstag", "Mittwoch", "Donnerstag", "Freitag", "Samstag"],
weekdays_abbr: ["So", "Mo", "Di", "Mi", "Do", "Fr", "Sa"],
am_pm: AM_PM,
currency: "CHF",
currency_format: "\"CHF\" #,##0.00",
currency_format_int: "\"CHF\" #,##0",
};
static FR_FR: Region = Region {
tag: "fr-FR",
name: "Français (France)",
lcid: 0x040C,
decimal: ',',
group: '\u{a0}',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"],
months_abbr: ["janv.", "févr.", "mars", "avr.", "mai", "juin", "juil.", "août", "sept.", "oct.", "nov.", "déc."],
months_genitive: None,
weekdays: ["dimanche", "lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi"],
weekdays_abbr: ["dim.", "lun.", "mar.", "mer.", "jeu.", "ven.", "sam."],
am_pm: AM_PM,
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static FR_CH: Region = Region {
tag: "fr-CH",
name: "Français (Suisse)",
lcid: 0x100C,
decimal: '.',
group: '\'',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["janvier", "février", "mars", "avril", "mai", "juin", "juillet", "août", "septembre", "octobre", "novembre", "décembre"],
months_abbr: ["janv.", "févr.", "mars", "avr.", "mai", "juin", "juil.", "août", "sept.", "oct.", "nov.", "déc."],
months_genitive: None,
weekdays: ["dimanche", "lundi", "mardi", "mercredi", "jeudi", "vendredi", "samedi"],
weekdays_abbr: ["dim.", "lun.", "mar.", "mer.", "jeu.", "ven.", "sam."],
am_pm: AM_PM,
currency: "CHF",
currency_format: "#,##0.00 \"CHF\"",
currency_format_int: "#,##0 \"CHF\"",
};
static IT_IT: Region = Region {
tag: "it-IT",
name: "Italiano (Italia)",
lcid: 0x0410,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"],
months_abbr: ["gen", "feb", "mar", "apr", "mag", "giu", "lug", "ago", "set", "ott", "nov", "dic"],
months_genitive: None,
weekdays: ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"],
weekdays_abbr: ["dom", "lun", "mar", "mer", "gio", "ven", "sab"],
am_pm: AM_PM,
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static IT_CH: Region = Region {
tag: "it-CH",
name: "Italiano (Svizzera)",
lcid: 0x0810,
decimal: '.',
group: '\'',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd, d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"],
months_abbr: ["gen", "feb", "mar", "apr", "mag", "giu", "lug", "ago", "set", "ott", "nov", "dic"],
months_genitive: None,
weekdays: ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"],
weekdays_abbr: ["dom", "lun", "mar", "mer", "gio", "ven", "sab"],
am_pm: AM_PM,
currency: "CHF",
currency_format: "\"CHF\" #,##0.00",
currency_format_int: "\"CHF\" #,##0",
};
static NL_NL: Region = Region {
tag: "nl-NL",
name: "Nederlands (Nederland)",
lcid: 0x0413,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '-',
time_sep: ':',
short_date: "dd-mm-yyyy",
long_date: "dddd d mmmm yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd-mm-yyyy hh:mm",
months: ["januari", "februari", "maart", "april", "mei", "juni", "juli", "augustus", "september", "oktober", "november", "december"],
months_abbr: ["jan", "feb", "mrt", "apr", "mei", "jun", "jul", "aug", "sep", "okt", "nov", "dec"],
months_genitive: None,
weekdays: ["zondag", "maandag", "dinsdag", "woensdag", "donderdag", "vrijdag", "zaterdag"],
weekdays_abbr: ["zo", "ma", "di", "wo", "do", "vr", "za"],
am_pm: AM_PM,
currency: "€",
currency_format: "\"€\" #,##0.00",
currency_format_int: "\"€\" #,##0",
};
static PT_BR: Region = Region {
tag: "pt-BR",
name: "Português (Brasil)",
lcid: 0x0416,
decimal: ',',
group: '.',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d \"de\" mmmm \"de\" yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: ["janeiro", "fevereiro", "março", "abril", "maio", "junho", "julho", "agosto", "setembro", "outubro", "novembro", "dezembro"],
months_abbr: ["jan", "fev", "mar", "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez"],
months_genitive: None,
weekdays: ["domingo", "segunda-feira", "terça-feira", "quarta-feira", "quinta-feira", "sexta-feira", "sábado"],
weekdays_abbr: ["dom", "seg", "ter", "qua", "qui", "sex", "sáb"],
am_pm: AM_PM,
currency: "R$",
currency_format: "\"R$\" #,##0.00",
currency_format_int: "\"R$\" #,##0",
};
static PT_PT: Region = Region {
tag: "pt-PT",
name: "Português (Portugal)",
lcid: 0x0816,
decimal: ',',
group: '\u{a0}',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '/',
time_sep: ':',
short_date: "dd/mm/yyyy",
long_date: "dddd, d \"de\" mmmm \"de\" yyyy",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd/mm/yyyy hh:mm",
months: ["janeiro", "fevereiro", "março", "abril", "maio", "junho", "julho", "agosto", "setembro", "outubro", "novembro", "dezembro"],
months_abbr: ["jan", "fev", "mar", "abr", "mai", "jun", "jul", "ago", "set", "out", "nov", "dez"],
months_genitive: None,
weekdays: ["domingo", "segunda-feira", "terça-feira", "quarta-feira", "quinta-feira", "sexta-feira", "sábado"],
weekdays_abbr: ["dom", "seg", "ter", "qua", "qui", "sex", "sáb"],
am_pm: AM_PM,
currency: "€",
currency_format: "#,##0.00 \"€\"",
currency_format_int: "#,##0 \"€\"",
};
static RU_RU: Region = Region {
tag: "ru-RU",
name: "Русский (Россия)",
lcid: 0x0419,
decimal: ',',
group: '\u{a0}',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "d mmmm yyyy\" г.\"",
short_time: "hh:mm",
long_time: "hh:mm:ss",
datetime: "dd.mm.yyyy hh:mm",
months: ["январь", "февраль", "март", "апрель", "май", "июнь", "июль", "август", "сентябрь", "октябрь", "ноябрь", "декабрь"],
months_abbr: ["янв.", "февр.", "март", "апр.", "май", "июнь", "июль", "авг.", "сент.", "окт.", "нояб.", "дек."],
months_genitive: Some(["января", "февраля", "марта", "апреля", "мая", "июня", "июля", "августа", "сентября", "октября", "ноября", "декабря"]),
weekdays: ["воскресенье", "понедельник", "вторник", "среда", "четверг", "пятница", "суббота"],
weekdays_abbr: ["вс", "пн", "вт", "ср", "чт", "пт", "сб"],
am_pm: AM_PM,
currency: "₽",
currency_format: "#,##0.00\u{a0}\"₽\"",
currency_format_int: "#,##0\u{a0}\"₽\"",
};
static CS_CZ: Region = Region {
tag: "cs-CZ",
name: "Čeština (Česko)",
lcid: 0x0405,
decimal: ',',
group: '\u{a0}',
grouping: Grouping::Thousands,
date_order: DateOrder::Dmy,
date_sep: '.',
time_sep: ':',
short_date: "dd.mm.yyyy",
long_date: "dddd d. mmmm yyyy",
short_time: "h:mm",
long_time: "h:mm:ss",
datetime: "dd.mm.yyyy h:mm",
months: ["leden", "únor", "březen", "duben", "květen", "červen", "červenec", "srpen", "září", "říjen", "listopad", "prosinec"],
months_abbr: ["led", "úno", "bře", "dub", "kvě", "čvn", "čvc", "srp", "zář", "říj", "lis", "pro"],
months_genitive: Some(["ledna", "února", "března", "dubna", "května", "června", "července", "srpna", "září", "října", "listopadu", "prosince"]),
weekdays: ["neděle", "pondělí", "úterý", "středa", "čtvrtek", "pátek", "sobota"],
weekdays_abbr: ["ne", "po", "út", "st", "čt", "pá", "so"],
am_pm: ["dop.", "odp."],
currency: "Kč",
currency_format: "#,##0.00 \"Kč\"",
currency_format_int: "#,##0 \"Kč\"",
};
static JA_JP: Region = Region {
tag: "ja-JP",
name: "日本語 (日本)",
lcid: 0x0411,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Ymd,
date_sep: '/',
time_sep: ':',
short_date: "yyyy/m/d",
long_date: "yyyy\"年\"m\"月\"d\"日\"",
short_time: "h:mm",
long_time: "h:mm:ss",
datetime: "yyyy/m/d h:mm",
months: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"],
months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"],
months_genitive: None,
weekdays: ["日曜日", "月曜日", "火曜日", "水曜日", "木曜日", "金曜日", "土曜日"],
weekdays_abbr: ["日", "月", "火", "水", "木", "金", "土"],
am_pm: ["午前", "午後"],
currency: "¥",
currency_format: "\"¥\"#,##0",
currency_format_int: "\"¥\"#,##0",
};
static KO_KR: Region = Region {
tag: "ko-KR",
name: "한국어 (대한민국)",
lcid: 0x0412,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Ymd,
date_sep: '-',
time_sep: ':',
short_date: "yyyy-mm-dd",
long_date: "yyyy\"년\" m\"월\" d\"일\" dddd",
short_time: "AM/PM h:mm",
long_time: "AM/PM h:mm:ss",
datetime: "yyyy-mm-dd hh:mm",
months: ["1월", "2월", "3월", "4월", "5월", "6월", "7월", "8월", "9월", "10월", "11월", "12월"],
months_abbr: ["1월", "2월", "3월", "4월", "5월", "6월", "7월", "8월", "9월", "10월", "11월", "12월"],
months_genitive: None,
weekdays: ["일요일", "월요일", "화요일", "수요일", "목요일", "금요일", "토요일"],
weekdays_abbr: ["일", "월", "화", "수", "목", "금", "토"],
am_pm: ["오전", "오후"],
currency: "₩",
currency_format: "\"₩\"#,##0",
currency_format_int: "\"₩\"#,##0",
};
static ZH_CN: Region = Region {
tag: "zh-CN",
name: "中文 (中国)",
lcid: 0x0804,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Ymd,
date_sep: '/',
time_sep: ':',
short_date: "yyyy/m/d",
long_date: "yyyy\"年\"m\"月\"d\"日\"",
short_time: "h:mm",
long_time: "h:mm:ss",
datetime: "yyyy/m/d h:mm",
months: ["一月", "二月", "三月", "四月", "五月", "六月", "七月", "八月", "九月", "十月", "十一月", "十二月"],
months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"],
months_genitive: None,
weekdays: ["星期日", "星期一", "星期二", "星期三", "星期四", "星期五", "星期六"],
weekdays_abbr: ["周日", "周一", "周二", "周三", "周四", "周五", "周六"],
am_pm: ["上午", "下午"],
currency: "¥",
currency_format: "\"¥\"#,##0.00",
currency_format_int: "\"¥\"#,##0",
};
static ZH_TW: Region = Region {
tag: "zh-TW",
name: "中文 (台灣)",
lcid: 0x0404,
decimal: '.',
group: ',',
grouping: Grouping::Thousands,
date_order: DateOrder::Ymd,
date_sep: '/',
time_sep: ':',
short_date: "yyyy/m/d",
long_date: "yyyy\"年\"m\"月\"d\"日\"",
short_time: "AM/PM hh:mm",
long_time: "AM/PM hh:mm:ss",
datetime: "yyyy/m/d hh:mm",
months: ["一月", "二月", "三月", "四月", "五月", "六月", "七月", "八月", "九月", "十月", "十一月", "十二月"],
months_abbr: ["1月", "2月", "3月", "4月", "5月", "6月", "7月", "8月", "9月", "10月", "11月", "12月"],
months_genitive: None,
weekdays: ["星期日", "星期一", "星期二", "星期三", "星期四", "星期五", "星期六"],
weekdays_abbr: ["週日", "週一", "週二", "週三", "週四", "週五", "週六"],
am_pm: ["上午", "下午"],
currency: "NT$",
currency_format: "\"NT$\"#,##0.00",
currency_format_int: "\"NT$\"#,##0",
};
static ALL: [&Region; 20] = [
&EN_US, &EN_GB, &EN_IN, &EN_CH, &ES_ES, &DE_DE, &DE_CH, &FR_FR, &FR_CH, &IT_IT, &IT_CH, &NL_NL, &PT_BR, &PT_PT, &RU_RU, &CS_CZ, &JA_JP, &KO_KR,
&ZH_CN, &ZH_TW,
];
/// Every region, English (United States) first.
pub fn all() -> &'static [&'static Region] {
&ALL
}
/// The default region, English (United States).
pub fn default_region() -> &'static Region {
&EN_US
}
/// The region with exactly this tag (`pt-BR`; case and `_` don't matter).
pub fn find(tag: &str) -> Option<&'static Region> {
let t = tag.trim().replace('_', "-");
ALL.iter().copied().find(|r| r.tag.eq_ignore_ascii_case(&t))
}
/// The region for a tag as operating systems and browsers write it: `pt-BR`, `pt_BR.UTF-8`,
/// `de`, `en-AU`, `zh-Hant`, `sr_RS@latin`. An exact match wins; otherwise the language's main
/// region (with the usual variants: `en-AU`→`en-GB`, `pt-AO`→`pt-PT`, `zh-Hant`→`zh-TW`).
/// An unknown language is English (United States).
pub fn lookup(tag: &str) -> &'static Region {
if let Some(r) = find(tag) {
return r;
}
let base = tag.split(['.', '@']).next().unwrap_or("");
let norm = base.trim().replace('_', "-");
if let Some(r) = find(&norm) {
return r;
}
let mut parts = norm.split('-').filter(|p| !p.is_empty());
let lang = parts.next().unwrap_or("").to_ascii_lowercase();
let mut script = String::new();
let mut country = String::new();
for p in parts {
// An extension or private-use singleton (`-u-ca-gregory`, `-x-…`) ends the region part.
if p.len() == 1 {
break;
}
if p.len() == 4 && p.chars().all(|c| c.is_ascii_alphabetic()) {
script = p.to_ascii_lowercase();
} else if (p.len() == 2 && p.chars().all(|c| c.is_ascii_alphabetic())) || (p.len() == 3 && p.chars().all(|c| c.is_ascii_digit())) {
country = p.to_ascii_uppercase();
}
}
let tag = match lang.as_str() {
"en" => match country.as_str() {
"GB" | "AU" | "NZ" | "IE" => "en-GB",
"IN" => "en-IN",
"CH" | "LI" => "en-CH",
_ => "en-US",
},
"pt" => match country.as_str() {
"AO" | "CH" | "CV" | "FR" | "GQ" | "GW" | "LU" | "MO" | "MZ" | "ST" | "TL" | "PT" => "pt-PT",
_ => "pt-BR",
},
"zh" => match country.as_str() {
// An explicit territory wins over the script (`zh-Hant-CN` is China's conventions).
"CN" | "SG" => "zh-CN",
"TW" | "HK" | "MO" => "zh-TW",
_ if script == "hant" => "zh-TW",
_ => "zh-CN",
},
"es" => "es-ES",
"de" => match country.as_str() {
"CH" | "LI" => "de-CH",
_ => "de-DE",
},
"fr" => match country.as_str() {
"CH" | "LI" => "fr-CH",
_ => "fr-FR",
},
"it" => match country.as_str() {
"CH" | "LI" => "it-CH",
_ => "it-IT",
},
"nl" => "nl-NL",
"ru" => "ru-RU",
"cs" => "cs-CZ",
"ja" => "ja-JP",
"ko" => "ko-KR",
_ => "en-US",
};
find(tag).unwrap_or(&EN_US)
}
impl Region {
/// True for the characters that separate groups of digits in this region. A region that
/// groups with a space also accepts the other space-like characters (NBSP, narrow NBSP, thin
/// space), as people type and paste any of them; one that groups with an apostrophe accepts
/// both `'` and `’`.
pub fn is_group(&self, c: char) -> bool {
c == self.group || (is_space_like(self.group) && is_space_like(c)) || (is_apostrophe_like(self.group) && is_apostrophe_like(c))
}
/// True when the currency symbol comes before the number.
pub fn currency_first(&self) -> bool {
self.currency_format.starts_with('"')
}
/// Month number (1–12) for a month name as typed: the full name, the abbreviation (with or
/// without its period) or any prefix of the full name of three or more letters (`Sept`),
/// ignoring case. `word` is one word: the genitive form used after a day (`января` in
/// `1 января`) matches too, and the caller reads the day itself.
pub fn month_from_name(&self, word: &str) -> Option<u32> {
let w = word.trim().to_lowercase();
// Where the abbreviations end in a period (`сент.`, `janv.`), any abbreviation may be typed
// with one (`мар.`, `juin.`). Elsewhere a period is not part of a month name, so en-US
// `Mar. 4, 2026` stays text as before.
let dotted = self.months_abbr.iter().any(|a| a.ends_with('.'));
let stem = if dotted { w.trim_end_matches('.') } else { w.as_str() };
if stem.is_empty() {
return None;
}
let long_enough = stem.chars().count() >= 3;
for (i, full) in self.months.iter().enumerate() {
let full_l = full.to_lowercase();
let abbr_l = self.months_abbr.get(i).map(|a| a.to_lowercase()).unwrap_or_default();
let gen_l = self.months_genitive.as_ref().and_then(|g| g.get(i)).map(|g| g.to_lowercase());
let hit = w == full_l
|| w == abbr_l
|| stem == abbr_l.trim_end_matches('.')
|| gen_l.as_deref() == Some(w.as_str())
|| (long_enough && full_l.starts_with(stem))
|| (long_enough && gen_l.as_deref().is_some_and(|g| g.starts_with(stem)));
if hit {
return Some(i as u32 + 1);
}
}
None
}
/// The name of month `month` (1–12) for `mmmm`; `with_day` selects the genitive form where
/// the region has one.
pub fn month_name(&self, month: u32, with_day: bool) -> &'static str {
let i = (month as usize).wrapping_sub(1);
if with_day && let Some(g) = &self.months_genitive {
return g.get(i).copied().unwrap_or("");
}
self.months.get(i).copied().unwrap_or("")
}
/// True when the short time uses a 12-hour clock.
pub fn is_12_hour(&self) -> bool {
self.short_time.contains("AM/PM")
}
}
fn is_space_like(c: char) -> bool {
matches!(c, ' ' | '\u{a0}' | '\u{202f}' | '\u{2009}')
}
fn is_apostrophe_like(c: char) -> bool {
matches!(c, '\'' | '\u{2019}')
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn lookup_accepts_system_and_browser_tags() {
let cases = [
("pt-BR", "pt-BR"),
("pt_BR.UTF-8", "pt-BR"),
("pt", "pt-BR"),
("pt-PT", "pt-PT"),
("pt-AO", "pt-PT"),
("de", "de-DE"),
("de-AT", "de-DE"),
("de_CH.UTF-8", "de-CH"),
("de-LI", "de-CH"),
("fr-CH", "fr-CH"),
("fr-LI", "fr-CH"),
("fr-BE", "fr-FR"),
("it_CH.UTF-8", "it-CH"),
("it-LI", "it-CH"),
("en-CH", "en-CH"),
("nl_NL.UTF-8", "nl-NL"),
("nl", "nl-NL"),
("en-AU", "en-GB"),
("en_GB.UTF-8", "en-GB"),
("en-IN", "en-IN"),
("en-CA", "en-US"),
("zh-Hant", "zh-TW"),
("zh-HK", "zh-TW"),
("zh", "zh-CN"),
// Extensions don't replace the region or script; an explicit territory beats the script.
("en-GB-u-ca-gregory", "en-GB"),
("zh-Hant-u-nu-latn", "zh-TW"),
("zh-Hant-CN", "zh-CN"),
("zh-Hans-TW", "zh-TW"),
("cs_CZ.UTF-8", "cs-CZ"),
("C", "en-US"),
("", "en-US"),
("xx-YY", "en-US"),
];
for (tag, expected) in cases {
assert_eq!(lookup(tag).tag, expected, "{tag}");
}
assert_eq!(find("DE-de").map(|r| r.tag), Some("de-DE"));
assert!(find("xx").is_none());
}
#[test]
fn every_region_is_consistent() {
let mut tags = std::collections::HashSet::new();
for r in all() {
assert!(tags.insert(r.tag), "duplicate {}", r.tag);
assert_ne!(r.decimal, r.group, "{}", r.tag);
assert!(r.currency_format.contains(r.currency), "{}", r.tag);
assert!(r.months.iter().all(|m| !m.is_empty()) && r.months_abbr.iter().all(|m| !m.is_empty()), "{}", r.tag);
assert!(r.weekdays.iter().all(|m| !m.is_empty()) && r.weekdays_abbr.iter().all(|m| !m.is_empty()), "{}", r.tag);
assert_eq!(find(r.tag).map(|x| x.tag), Some(r.tag));
}
}
#[test]
fn month_names_match_loosely() {
let de = find("de-DE").unwrap();
assert_eq!(de.month_from_name("März"), Some(3));
assert_eq!(de.month_from_name("mär"), Some(3));
assert_eq!(de.month_from_name("DEZ"), Some(12));
assert_eq!(de.month_from_name("de"), None);
let gb = find("en-GB").unwrap();
assert_eq!(gb.month_from_name("Sept"), Some(9));
assert_eq!(gb.month_from_name("sep"), Some(9));
let ru = find("ru-RU").unwrap();
assert_eq!(ru.month_from_name("сентября"), Some(9));
assert_eq!(ru.month_from_name("сент."), Some(9));
// Abbreviations typed with a period the table doesn't carry.
assert_eq!(ru.month_from_name("мар."), Some(3));
assert_eq!(ru.month_from_name("июн."), Some(6));
assert_eq!(find("fr-FR").unwrap().month_from_name("juin."), Some(6));
assert_eq!(EN_US.month_from_name("Mar."), None);
assert_eq!(de.month_from_name("."), None);
assert_eq!(EN_US.month_from_name("Ma"), None);
assert_eq!(EN_US.month_from_name("May"), Some(5));
}
#[test]
fn space_like_and_apostrophe_groups_are_interchangeable() {
let fr = find("fr-FR").unwrap();
assert!(fr.is_group('\u{a0}') && fr.is_group(' ') && fr.is_group('\u{202f}'));
assert!(!find("de-DE").unwrap().is_group(' '));
let ch = find("de-CH").unwrap();
assert!(ch.is_group('\'') && ch.is_group('\u{2019}'));
assert!(!ch.is_group(' ') && !find("de-DE").unwrap().is_group('\''));
}
}
+258 -10
View File
@@ -346,19 +346,91 @@ pub fn compare_numbers(a: f64, b: f64) -> Ordering {
round15(a).partial_cmp(&round15(b)).unwrap_or(Ordering::Equal)
}
/// Case-insensitive text order (Unicode lowercase).
/// Case-insensitive text order in three levels, as Excel's collation sorts: first by letters and
/// digits alone (Arabic marks, tatweel and invisible format characters skipped, Arabic-Indic digits
/// read as digits); then, letter by letter, by the marks on each letter (none first, then in
/// Excel's mark order) and the digit script (ASCII, Arabic-Indic, Extended Arabic-Indic); then, to
/// break a remaining tie, by every character (Unicode lowercase). Only text that is equal at the
/// last level compares equal.
pub fn compare_text(a: &str, b: &str) -> Ordering {
let mut ai = a.chars().flat_map(char::to_lowercase);
let mut bi = b.chars().flat_map(char::to_lowercase);
loop {
match (ai.next(), bi.next()) {
(None, None) => return Ordering::Equal,
(None, Some(_)) => return Ordering::Less,
(Some(_), None) => return Ordering::Greater,
(Some(x), Some(y)) if x != y => return x.cmp(&y),
_ => {}
if !(a.is_ascii() && b.is_ascii()) {
let primary = primary_chars(a).cmp(primary_chars(b));
if primary != Ordering::Equal {
return primary;
}
let secondary = secondary_weights(a).cmp(secondary_weights(b));
if secondary != Ordering::Equal {
return secondary;
}
}
lowercase_chars(a).cmp(lowercase_chars(b))
}
fn lowercase_chars(s: &str) -> impl Iterator<Item = char> + '_ {
s.chars().flat_map(char::to_lowercase)
}
/// Characters Excel's text order skips entirely: tatweel and invisible format characters (soft
/// hyphen, combining grapheme joiner, zero-width (non-)joiner, bidi marks, embeddings, overrides
/// and isolates, word joiner, byte order mark). The zero-width space is not one.
fn ignorable(c: char) -> bool {
matches!(
c,
'\u{00AD}'
| '\u{034F}'
| '\u{061C}'
| '\u{0640}'
| '\u{200C}'..='\u{200F}'
| '\u{202A}'..='\u{202E}'
| '\u{2060}'
| '\u{2066}'..='\u{2069}'
| '\u{FEFF}'
)
}
/// Arabic combining marks in the order Excel sorts a letter carrying them; an unmarked letter
/// sorts before all of them. Measured in Excel 16.0 (build 20527, en-US) with one mark on a letter.
/// U+06D6 and U+06D7 sort as letters in Excel, so they are not listed.
const ARABIC_MARK_ORDER: [char; 48] = [
'\u{06EC}', '\u{0611}', '\u{0612}', '\u{0613}', '\u{0614}', '\u{06DB}', '\u{0656}', '\u{0657}', '\u{0610}', '\u{06EB}', '\u{0615}', '\u{06D9}',
'\u{06E4}', '\u{06E1}', '\u{06DA}', '\u{06DC}', '\u{06EA}', '\u{06E3}', '\u{06D8}', '\u{0655}', '\u{065F}', '\u{0654}', '\u{06E7}', '\u{0653}',
'\u{06E8}', '\u{06ED}', '\u{06E2}', '\u{0658}', '\u{064B}', '\u{064C}', '\u{064D}', '\u{064E}', '\u{064F}', '\u{0650}', '\u{0652}', '\u{0651}',
'\u{0670}', '\u{0659}', '\u{065A}', '\u{065B}', '\u{065C}', '\u{065D}', '\u{065E}', '\u{0616}', '\u{0617}', '\u{0618}', '\u{0619}', '\u{061A}',
];
/// Rank of an Arabic mark that letter order skips (its index in `ARABIC_MARK_ORDER`).
fn arabic_mark_rank(c: char) -> Option<u16> {
ARABIC_MARK_ORDER.iter().position(|&m| m == c).and_then(|i| u16::try_from(i).ok())
}
/// Digit script of `c` (0 ASCII, 1 Arabic-Indic, 2 Extended Arabic-Indic) and its ASCII digit.
fn digit(c: char) -> Option<(u16, char)> {
let (script, zero) = match c {
'0'..='9' => return Some((0, c)),
'\u{0660}'..='\u{0669}' => (1, 0x0660),
'\u{06F0}'..='\u{06F9}' => (2, 0x06F0),
_ => return None,
};
char::from_u32(u32::from(c) - zero + u32::from('0')).map(|d| (script, d))
}
/// The characters that decide text order before any tie-break.
fn primary_chars(s: &str) -> impl Iterator<Item = char> + '_ {
lowercase_chars(s).filter_map(|c| if ignorable(c) || arabic_mark_rank(c).is_some() { None } else { Some(digit(c).map_or(c, |(_, d)| d)) })
}
/// The second level, for text whose primary characters are equal: per letter, its digit script and
/// then its marks, the letters' lists closed by 0 so that an unmarked letter sorts first. Marks
/// before the first letter form a list of their own.
fn secondary_weights(s: &str) -> impl Iterator<Item = u16> + '_ {
lowercase_chars(s)
.filter(|&c| !ignorable(c))
.flat_map(|c| match arabic_mark_rank(c) {
Some(rank) => [Some(rank.saturating_add(1)), None],
None => [Some(0), Some(digit(c).map_or(0, |(script, _)| script))],
})
.flatten()
.chain(std::iter::once(0))
}
/// Sorting order used by Sort (ascending): numbers, text, booleans, errors, then blanks last.
@@ -457,6 +529,182 @@ mod tests {
assert_eq!(sort_compare(&Value::Empty, &Value::Number(1.0)), Ordering::Greater);
}
#[test]
fn arabic_diacritics_sort_with_their_base_letter() {
assert_eq!(compare_text("بَاب", "بيت"), Ordering::Less);
assert_eq!(compare_text("بيت", "بَاب"), Ordering::Greater);
assert_eq!(compare_text("كِتاب", "كتب"), Ordering::Less);
assert_eq!(compare_text("رحمٰن", "رحمي"), Ordering::Less);
assert_eq!(compare(&Value::from("بَاب"), &Value::from("بيت")), Ordering::Less);
}
#[test]
fn arabic_indic_digits_sort_with_digits() {
assert_eq!(compare_text("١٢", "ب"), Ordering::Less);
assert_eq!(compare_text("۱۲", "ب"), Ordering::Less);
assert_eq!(compare_text("١", "a"), Ordering::Less);
assert_eq!(compare_text("١٢", "13"), Ordering::Less);
assert_eq!(compare_text("۱۴", "13"), Ordering::Greater);
}
#[test]
fn tatweel_and_bidi_marks_do_not_move_sort_order() {
assert_eq!(compare_text("بـاب", "بت"), Ordering::Less);
assert_eq!(compare_text("\u{200F}apple", "banana"), Ordering::Less);
assert_eq!(compare_text("\u{200E}zebra", "apple"), Ordering::Greater);
assert_eq!(compare_text("\u{061C}a", "b"), Ordering::Less);
assert_eq!(compare_text("a\u{200C}b", "ac"), Ordering::Less);
assert_eq!(compare_text("a\u{200D}b", "ac"), Ordering::Less);
}
#[test]
fn primary_ties_keep_equality_unchanged() {
assert_eq!(compare_text("ABC", "abc"), Ordering::Equal);
assert_eq!(compare_text("باب", "بَاب"), Ordering::Less);
assert_eq!(compare_text("12", "١٢"), Ordering::Less);
assert_eq!(compare_text("a", "\u{200F}a"), Ordering::Less);
let samples = [
"",
"abc",
"ABC",
"abd",
"ab",
"باب",
"بَاب",
"بِاب",
"بـاب",
"بيت",
"١٢",
"۱۲",
"12",
"\u{200F}12",
"\u{200F}a",
"a",
"A\u{200C}",
"a\u{200F}",
"b",
"Straße",
"STRASSE",
"ΣΑΣ",
"σας",
"İ",
"i̇",
];
let lowered = |s: &str| s.chars().flat_map(char::to_lowercase).collect::<String>();
for a in samples {
for b in samples {
let order = compare_text(a, b);
assert_eq!(order.is_eq(), lowered(a) == lowered(b), "{a:?} vs {b:?}");
assert_eq!(order, compare_text(b, a).reverse(), "{a:?} vs {b:?}");
for c in samples {
if order.is_le() && compare_text(b, c).is_le() {
assert!(compare_text(a, c).is_le(), "{a:?} <= {b:?} <= {c:?}");
}
}
}
}
}
/// Sorts `expected` from reversed and rotated starts and checks each comes back as `expected`.
#[track_caller]
fn assert_sorts_to(expected: &[&str]) {
let mut reversed: Vec<&str> = expected.iter().rev().copied().collect();
reversed.sort_by(|a, b| compare_text(a, b));
assert_eq!(reversed, expected);
let mut rotated: Vec<&str> = expected.to_vec();
rotated.rotate_left(expected.len() / 2);
rotated.sort_by(|a, b| compare_text(a, b));
assert_eq!(rotated, expected);
}
// The expected orders below are Excel's, observed with Range.Sort, =SORT() and the `<`
// operator in Excel 16.0 (build 20527, en-US). Strings Excel ranks equal are left out.
#[test]
fn excel_orders_a_letter_by_its_mark() {
assert_sorts_to(&[
"باب",
"ب\u{0611}اب",
"ب\u{0610}اب",
"ب\u{0654}اب",
"ب\u{064B}اب",
"ب\u{064C}اب",
"ب\u{064D}اب",
"ب\u{064E}اب",
"ب\u{064F}اب",
"ب\u{0650}اب",
"ب\u{0652}اب",
"ب\u{0651}اب",
"ب\u{0670}اب",
"ب\u{065E}اب",
"ب\u{061A}اب",
]);
// Quranic marks are skipped by letter order like harakat (hamza sorts before alef).
assert_eq!(compare_text("ب\u{0610}ء", "باب"), Ordering::Less);
assert_eq!(compare_text("ب\u{06E1}ء", "باب"), Ordering::Less);
}
#[test]
fn excel_compares_marks_letter_by_letter() {
assert_sorts_to(&[
"بت",
"بت\u{064E}",
"بت\u{064F}",
"بت\u{0650}",
"ب\u{064E}ت",
"ب\u{064E}ت\u{064E}",
"ب\u{064E}ت\u{064F}",
"ب\u{064F}ت",
"ب\u{064F}ت\u{064E}",
"ب\u{0650}ت\u{0650}",
]);
assert_sorts_to(&["مـحـمـد", "محم\u{0651}د", "م\u{064E}حمد", "م\u{064F}ح\u{064E}م\u{064E}\u{0651}د", "محمدي"]);
}
#[test]
fn excel_weighs_marks_and_digit_scripts_by_position() {
// A mark on an earlier letter outranks the digit script or letter after it, whatever the
// code point of that next character.
assert_sorts_to(&[
"ب",
"ب\u{064E}",
"ب1",
"ب١",
"ب۱",
"ب\u{064E}1",
"ب\u{064E}١",
"ب\u{064E}۱",
"بa",
"ب\u{064E}a",
"با",
"ب\u{064E}ا",
"ب\u{067E}",
"ب\u{064E}\u{067E}",
"ب\u{06CC}",
"ب\u{064E}\u{06CC}",
]);
// And an earlier digit script outranks a later mark.
assert_sorts_to(&["12", "1٢", "١2", "١٢", "1ب", "1ب\u{064E}", "1ب\u{0650}", "١ب", "١ب\u{064E}", "١ب\u{0650}", "۱ب", "۱ب\u{064E}"]);
assert_sorts_to(&["ب1ب", "ب1ب\u{064E}", "ب١ب", "ب\u{064E}ب", "بب1", "بب١", "ب\u{064E}ب1"]);
}
#[test]
fn excel_skips_invisible_format_characters() {
for c in ['\u{00AD}', '\u{034F}', '\u{061C}', '\u{200C}', '\u{200D}', '\u{200E}', '\u{200F}', '\u{202A}', '\u{202B}']
.into_iter()
.chain(['\u{202C}', '\u{202D}', '\u{202E}', '\u{2060}', '\u{2066}', '\u{2067}', '\u{2068}', '\u{2069}', '\u{FEFF}'])
{
let s = format!("{c}b");
assert_eq!(compare_text(&s, "a"), Ordering::Greater, "{s:?}");
assert_eq!(compare_text(&s, "ba"), Ordering::Less, "{s:?}");
assert_eq!(compare_text(&format!("ب{c}\u{064E}اب"), "باب\u{064B}"), Ordering::Greater, "{c:?}");
}
// Not even as a tie-break: a leading mark doesn't push text past marked or other-script forms.
assert_sorts_to(&["\u{200F}باب", "ب\u{064B}اب", "ب\u{064E}اب", "ب\u{0670}اب"]);
assert_sorts_to(&["\u{200F}12", "١٢", "۱۲"]);
assert_sorts_to(&["ب\u{200F}ب\u{064E}", "\u{200F}ب\u{064E}ب"]);
}
#[test]
fn errors() {
for e in CellError::ALL {
+14 -2
View File
@@ -118,7 +118,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[
("insert.picture", "Insert", "Illustrations", "Pictures"),
("insert.shape", "Insert", "Illustrations", "Shapes"),
("insert.icons", "Insert", "Illustrations", "Icons"),
("insert.smartArt", "Insert", "Illustrations", "SmartArt"),
("insert.smartArt", "Insert", "Illustrations", "Diagram"),
("insert.screenshot", "Insert", "Illustrations", "Screenshot"),
("insert.recommendedCharts", "Insert", "Charts", "Recommended Charts"),
("insert.chart", "Insert", "Charts", "Charts"),
@@ -131,7 +131,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[
("review.newComment", "Insert", "Comments", "Comment"),
("insert.textBox", "Insert", "Text", "Text Box"),
("pageLayout.headerFooter", "Insert", "Text", "Header & Footer"),
("insert.wordArt", "Insert", "Text", "WordArt"),
("insert.wordArt", "Insert", "Text", "Decorative Text"),
("insert.signatureLine", "Insert", "Text", "Signature Line"),
("insert.object", "Insert", "Text", "Object"),
("insert.equation", "Insert", "Symbols", "Equation"),
@@ -340,3 +340,15 @@ pub fn parity() -> (usize, usize) {
let done = ids.iter().filter(|id| crate::find_command(id).is_some()).count();
(done, ids.len())
}
#[cfg(test)]
mod feature_name_tests {
#[test]
fn own_feature_labels_keep_command_ids() {
for (id, expected) in [("insert.smartArt", "Diagram"), ("insert.wordArt", "Decorative Text")] {
let entries: Vec<_> = super::CATALOG.iter().filter(|entry| entry.0 == id).collect();
assert_eq!(entries.len(), 1, "{id} appears once");
assert_eq!(entries[0].3, expected);
}
}
}
+158 -4
View File
@@ -1,7 +1,7 @@
//! Conditional formatting evaluation: what extra style, data bar, colour scale or icon a cell
//! gets.
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use gridcraft_core::{CellRef, RangeRef, Value, compare};
use gridcraft_model::{CfOperator, CfRule, CfValueKind, Color, CondFormat, PatternType, Sheet, Style, Underline, Workbook, default_icon_thresholds};
@@ -47,6 +47,8 @@ enum StatsNeed {
fn stats_need(rule: &CfRule) -> Option<StatsNeed> {
match rule {
CfRule::Duplicate { .. } | CfRule::Unique { .. } => Some(StatsNeed::Counts),
// only two- and three-colour scales paint: no statistics for the others
CfRule::ColorScale { stops } if !matches!(stops.len(), 2 | 3) => None,
CfRule::Top10 { .. } | CfRule::AboveAverage { .. } | CfRule::ColorScale { .. } | CfRule::DataBar { .. } | CfRule::IconSet { .. } => {
Some(StatsNeed::Numbers)
}
@@ -70,15 +72,58 @@ thread_local! {
static SCANNED: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };
}
/// Most cells one rule's statistics visit, across all of its ranges together: a rule repeating
/// the same range many times must not multiply the scan.
const STATS_SCAN_LIMIT: usize = 2_000_000;
/// Steps through the spilled cells inside `r`, row-major: `Some(cell)` for each, and `None` for a
/// jump past the spills left or right of the columns on a row (one step per row, however many
/// there are), so unrelated spills between a narrow range's corners cost little.
fn spill_steps(sh: &Sheet, r: RangeRef) -> impl Iterator<Item = Option<CellRef>> + '_ {
let mut cur = Some(r.start);
std::iter::from_fn(move || {
let at = cur.filter(|at| *at <= r.end)?;
let (&c, _) = sh.spill.range(at..=r.end).next()?;
if c.col < r.start.col {
cur = Some(CellRef::new(c.row, r.start.col));
Some(None)
} else if c.col > r.end.col {
cur = c.row.checked_add(1).map(|row| CellRef::new(row, r.start.col));
Some(None)
} else {
cur = c.col.checked_add(1).map(|col| CellRef::new(c.row, col));
Some(Some(c))
}
})
}
fn numbers_in(sh: &Sheet, ranges: &[RangeRef], need: StatsNeed) -> (Vec<f64>, HashMap<String, usize>) {
let mut nums = Vec::new();
let mut counts = HashMap::new();
let mut budget = STATS_SCAN_LIMIT;
let mut seen: HashSet<RangeRef> = HashSet::new();
let used = sh.used_range();
for r in ranges {
let r = match sh.used_range().and_then(|u| r.intersection(&u)) {
if budget == 0 {
break;
}
let r = match used.and_then(|u| r.intersection(&u)) {
Some(x) => x,
None => continue,
};
for c in r.iter().take(2_000_000) {
// The same range listed twice counts its cells once.
if !seen.insert(r) {
continue;
}
// Only stored and spilled cells hold values: blank cells of a sparse range cost nothing,
// so a value far down a huge range is still counted. Every cell looked at is charged.
let stored = sh.cells.iter_range(r).map(|(c, _)| (Some(c), true));
let spilled = spill_steps(sh, r).map(|c| (c, false));
for (c, is_stored) in stored.chain(spilled).take(budget) {
budget -= 1;
// a skip over spills outside the columns (charged, not read), or a spilled value
// under a stored cell, which was read with it
let Some(c) = c.filter(|c| is_stored || sh.cells.get(*c).is_none()) else { continue };
#[cfg(test)]
SCANNED.with(|n| n.set(n.get() + 1));
let v = sh.value(c);
@@ -278,7 +323,8 @@ impl CfCache {
_ => None,
},
CfRule::ColorScale { stops } => {
if let Some(n) = v.as_f64() {
// Only two- and three-colour scales paint: evaluate no thresholds for others.
if let Some(n) = v.as_f64().filter(|_| matches!(stops.len(), 2 | 3)) {
let pts: Vec<(f64, [u8; 3])> = stops
.iter()
.map(|(k, col)| (threshold(wb, sheet, c, st, k), col.resolve(&wb.theme).unwrap_or([255, 255, 255])))
@@ -368,6 +414,9 @@ impl CfCache {
/// Adds the parts `lower` sets that `style` leaves unset (differential styles: default parts
/// are unset).
fn layer_under(style: &mut Style, lower: &Style) {
if style.align.reading_order.is_none() {
style.align.reading_order = lower.align.reading_order;
}
if style.fill.pattern == PatternType::None {
style.fill = lower.fill;
}
@@ -417,6 +466,28 @@ mod tests {
use super::*;
#[test]
fn reading_order_respects_conditional_rule_priority_and_explicit_context() {
for (higher, expected) in [
(json!({"font": {"bold": true}}), "RightToLeft"),
(json!({"align": {"reading_order": "Context"}}), "Context"),
(json!({"align": {"reading_order": "LeftToRight"}}), "LeftToRight"),
] {
let mut s = crate::Session::new();
s.new_workbook();
s.execute("cell.set", json!({"cell": "A1", "input": "مرحبا 123 ABC"})).unwrap();
for style in [json!({"align": {"reading_order": "RightToLeft"}}), higher] {
s.execute("home.conditionalFormat", json!({"range": "A1", "rule": {"type": "noBlanks", "style": style}})).unwrap();
}
let wb = &s.doc().unwrap().wb;
let (reopened, _) = gridcraft_xlsx::read_xlsx(&gridcraft_xlsx::write_xlsx(wb).unwrap()).unwrap();
for wb in [wb, &reopened] {
let style = CfCache::new().look(wb, 0, CellRef::new(0, 0)).unwrap().style.unwrap();
assert_eq!(serde_json::to_value(style.align).unwrap()["reading_order"], expected);
}
}
}
#[test]
fn every_matching_rule_applies() {
let mut s = crate::Session::new();
@@ -560,6 +631,89 @@ mod tests {
assert_eq!(SCANNED.with(|n| n.get()), 10_000);
}
/// A rule listing the same range many times scans it once, and never more than the shared
/// limit however many ranges it holds.
#[test]
fn repeated_ranges_share_one_scan_budget() {
let mut s = crate::Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [[1], [2], [3]]})).unwrap();
s.execute("range.setValues", json!({"range": "XFD1048576", "values": [[4]]})).unwrap();
let d = s.doc().unwrap();
let sh = d.wb.sheet(0).unwrap();
let dense = RangeRef::parse("A1:XFD1048576").unwrap();
let small = RangeRef::parse("A1:A3").unwrap();
SCANNED.with(|n| n.set(0));
let st = RuleStats::build(sh, &vec![small; 1000], StatsNeed::Numbers);
assert_eq!(st.sorted, vec![1.0, 2.0, 3.0]);
assert_eq!(SCANNED.with(|n| n.get()), 3);
let shifted: Vec<RangeRef> = (0..50).map(|i| RangeRef::new(CellRef::new(i, 0), dense.end)).collect();
SCANNED.with(|n| n.set(0));
RuleStats::build(sh, &shifted, StatsNeed::Numbers);
assert!(SCANNED.with(|n| n.get()) <= STATS_SCAN_LIMIT);
}
/// Blank cells of a sparse range are not charged to the scan budget: a value at the end of two
/// million-row columns still counts, so duplicates are found.
#[test]
fn sparse_ranges_keep_their_values_under_the_scan_budget() {
let mut s = crate::Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [[7]]})).unwrap();
s.execute("range.setValues", json!({"range": "B1048576", "values": [[7]]})).unwrap();
let d = s.doc().unwrap();
let sh = d.wb.sheet(0).unwrap();
let ranges = [RangeRef::parse("A1:A1048576").unwrap(), RangeRef::parse("B1:B1048576").unwrap()];
SCANNED.with(|n| n.set(0));
let st = RuleStats::build(sh, &ranges, StatsNeed::Counts);
assert_eq!(st.counts.get("7"), Some(&2));
assert_eq!(SCANNED.with(|n| n.get()), 2);
}
/// Spills left and right of a narrow range, between its corners, are skipped a row at a time:
/// they do not use the budget that the in-range spills need.
#[test]
fn spills_outside_a_narrow_range_do_not_use_its_scan_budget() {
let mut s = crate::Session::new();
s.new_workbook();
let d = s.doc().unwrap();
let mut sh = d.wb.sheet(0).unwrap().clone();
for row in 0..1000 {
for col in [0u32, 5, 6, 7] {
sh.spill.insert(CellRef::new(row, col), Value::Number(1.0));
}
}
let r = RangeRef::parse("F1:G1000").unwrap();
let steps: Vec<_> = spill_steps(&sh, r).collect();
assert_eq!(steps.iter().flatten().count(), 2000);
// column A is skipped one step a row, never one per cell
assert!(steps.len() <= 2000 + 1000, "{}", steps.len());
let narrow = RangeRef::parse("C1:C1000").unwrap();
assert_eq!(spill_steps(&sh, narrow).flatten().count(), 0);
}
/// A colour scale with an unsupported number of stops paints nothing and evaluates none of them.
#[test]
fn colour_scales_with_other_stop_counts_paint_nothing() {
let mut s = crate::Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [[1], [5]]})).unwrap();
let mut wb: Workbook = (*s.doc().unwrap().wb).clone();
let stop = (CfValueKind::Formula("=1".into()), Color::default());
let rule = |n: usize| CondFormat {
ranges: vec![RangeRef::parse("A1:A2").unwrap()],
rule: CfRule::ColorScale { stops: vec![stop.clone(); n] },
priority: 1,
stop_if_true: false,
};
wb.sheet_mut(0).unwrap().cond_formats.push(rule(4));
let mut cache = CfCache::new();
assert_eq!(cache.look(&wb, 0, CellRef::parse("A1").unwrap()).and_then(|l| l.scale_fill), None);
wb.sheet_mut(0).unwrap().cond_formats = vec![rule(2)];
let mut cache = CfCache::new();
assert!(cache.look(&wb, 0, CellRef::parse("A1").unwrap()).and_then(|l| l.scale_fill).is_some());
}
#[test]
fn cell_value_operator_names() {
let mut s = crate::Session::new();
+36 -5
View File
@@ -10,6 +10,10 @@ use crate::io::esc;
/// Widest or tallest chart written; larger anchors are clamped.
const MAX_SIDE: f32 = 16_384.0;
/// Most bytes of SVG written for the charts of one web page together. A chart past what is
/// left makes the export fail rather than grow without bound or be dropped without a word.
pub(crate) const MAX_PAGE_SVG_BYTES: usize = 64 * 1024 * 1024;
struct HelveticaMeasure;
impl gridcraft_chart::Measure for HelveticaMeasure {
@@ -18,24 +22,40 @@ impl gridcraft_chart::Measure for HelveticaMeasure {
}
}
/// `chart` of sheet `sheet` as an `<svg>` element placed by `style`, or `None` for a degenerate size.
pub(crate) fn chart_svg(wb: &Workbook, sheet: usize, chart: &Chart, style: &str) -> Option<String> {
/// `chart` of sheet `sheet` as an `<svg>` element placed by `style`, `None` for a degenerate size,
/// or an error for an element longer than `budget` bytes.
pub(crate) fn chart_svg(wb: &Workbook, sheet: usize, chart: &Chart, style: &str, budget: usize) -> crate::Result<Option<String>> {
let (w, h) = (chart.anchor.width.min(MAX_SIDE), chart.anchor.height.min(MAX_SIDE));
if !(w >= 1.0 && h >= 1.0) {
return None;
return Ok(None);
}
let data = gridcraft_chart::resolve(wb, sheet, chart);
let prims = gridcraft_chart::render(chart, &data, w, h, &HelveticaMeasure);
Some(prims_svg(&prims, w, h, style))
match prims_svg_within(&prims, w, h, style, budget) {
Some(svg) => Ok(Some(svg)),
None => Err(crate::EngineError::Other(format!(
"the charts would make the web page over {} MiB. Save as .xlsx or PDF instead.",
MAX_PAGE_SVG_BYTES / (1024 * 1024)
))),
}
}
#[cfg(test)]
fn prims_svg(prims: &[Prim], w: f32, h: f32, style: &str) -> String {
prims_svg_within(prims, w, h, style, usize::MAX).unwrap_or_default()
}
/// The `<svg>` element, or `None` as soon as it grows past `budget` bytes.
fn prims_svg_within(prims: &[Prim], w: f32, h: f32, style: &str, budget: usize) -> Option<String> {
let mut out = String::new();
let _ = write!(
out,
"<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"{w}\" height=\"{h}\" viewBox=\"0 0 {w} {h}\" font-family=\"Helvetica,Arial,sans-serif\" style=\"{style}\">"
);
for p in prims {
if out.len() > budget {
return None;
}
match p {
Prim::Rect { x, y, w, h, fill, stroke, radius } => {
let _ = write!(out, "<rect x=\"{}\" y=\"{}\" width=\"{}\" height=\"{}\"", num(*x), num(*y), num(*w), num(*h));
@@ -92,7 +112,7 @@ fn prims_svg(prims: &[Prim], w: f32, h: f32, style: &str) -> String {
}
}
out.push_str("</svg>");
out
(out.len() <= budget).then_some(out)
}
/// Fill and stroke attributes of an open shape element; the caller closes it.
@@ -149,6 +169,17 @@ fn num(v: f32) -> String {
mod tests {
use super::*;
#[test]
fn an_svg_past_its_byte_budget_is_left_out() {
let rect = Prim::Rect { x: 1.0, y: 2.0, w: 3.0, h: 4.0, fill: Some([255, 0, 0, 255]), stroke: None, radius: 0.0 };
let prims = vec![rect; 10_000];
let full = prims_svg(&prims, 100.0, 50.0, "");
assert!(full.len() > 100_000);
assert_eq!(prims_svg_within(&prims, 100.0, 50.0, "", full.len()), Some(full.clone()));
assert_eq!(prims_svg_within(&prims, 100.0, 50.0, "", full.len() - 1), None);
assert_eq!(prims_svg_within(&prims, 100.0, 50.0, "", 1_000), None);
}
#[test]
fn every_primitive_becomes_an_svg_element() {
let red = [255, 0, 0, 255];
+150 -5
View File
@@ -366,6 +366,10 @@ pub struct Ctx<'a> {
pub wb: Workbook,
pub changed: Vec<Key>,
pub structural: bool,
/// Recalculate every formula without the other effects of `structural` (table headers are not
/// resynchronized, array formulas are still checked): for edits that change only what a
/// function reads, like sheet protection.
pub recalc_all: bool,
pub sel: &'a mut Selection,
/// Rows whose automatic height should be recomputed (sheet, row).
pub fit_rows: Vec<(usize, u32)>,
@@ -400,13 +404,14 @@ pub(crate) fn commit<R>(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result<R>)
wb: (*d.wb).clone(),
changed: Vec::new(),
structural: false,
recalc_all: false,
sel: &mut sel,
fit_rows: Vec::new(),
protection_checked: false,
edits: Vec::new(),
};
let r = f(&mut ctx)?;
let Ctx { mut wb, mut changed, structural, mut fit_rows, protection_checked, edits, .. } = ctx;
let Ctx { mut wb, mut changed, structural, recalc_all, mut fit_rows, protection_checked, edits, .. } = ctx;
// Equations whose source changed (or that arrived without a layout, as from a paste) are
// laid out here, so painting only ever reads the cache.
wb.refresh_equations();
@@ -435,7 +440,7 @@ pub(crate) fn commit<R>(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result<R>)
if let Some(t0) = t0 {
eprintln!("commit: edit {:?}, {} changed", since(t0), changed.len());
}
if structural {
if structural || recalc_all {
d.calc.recalc_all(&mut wb);
} else if !changed.is_empty() {
d.calc.cells_changed(&mut wb, &changed);
@@ -446,8 +451,18 @@ pub(crate) fn commit<R>(d: &mut DocState, f: impl FnOnce(&mut Ctx) -> Result<R>)
}
fit_rows.sort_unstable();
fit_rows.dedup();
let mut sweep: Option<(usize, MergeSweep)> = None;
let mut left = MERGE_FIT_WORK;
for (si, row) in fit_rows {
auto_row_height(&mut wb, si, row);
if sweep.as_ref().is_none_or(|(at, _)| *at != si) {
if let Some((_, old)) = &sweep {
left = old.left();
}
sweep = Some((si, MergeSweep::new(left)));
}
if let Some((_, sw)) = sweep.as_mut() {
fit_row_height(&mut wb, si, row, sw);
}
}
if let Some(t0) = t0 {
eprintln!("commit: + rows {:?}", since(t0));
@@ -721,6 +736,94 @@ fn wrapped_line_count(line: &str, width: f32, size: f32, bold: bool) -> f32 {
/// Line breaks in a cell's text count only when the cell wraps, as in Excel: with Wrap Text off
/// a multi-line value shows on one line and AutoFit gives a one-line row.
pub fn auto_row_height(wb: &mut Workbook, si: usize, row: u32) {
let mut sweep = MergeSweep::new(MERGE_FIT_WORK);
fit_row_height(wb, si, row, &mut sweep);
}
/// Most merge comparisons one auto-fit pass (a commit, or opening a workbook) makes. Past it the
/// remaining rows keep the heights they have rather than stall on a sheet whose merges overlap
/// the fitted rows many times over.
pub(crate) const MERGE_FIT_WORK: usize = 50_000_000;
/// The merged areas of one sheet that touch a row, for rows visited in ascending order: the
/// merges are sorted by first row once and swept, so fitting many rows never rescans every merge
/// per row. All comparisons count against [`MERGE_FIT_WORK`].
pub(crate) struct MergeSweep {
/// Built on the first row that needs merge checks, so rows that return early never pay for it.
by_start: Option<Vec<gridcraft_core::RangeRef>>,
next: usize,
active: Vec<gridcraft_core::RangeRef>,
budget: usize,
/// The first row was answered by a plain scan: a pass that fits one row never pays for a sort.
scanned: bool,
}
impl MergeSweep {
/// A sweep for one sheet's merges with `budget` comparisons left for the pass (carry
/// [`MergeSweep::left`] of the previous sheet's sweep into the next). Every call to
/// [`MergeSweep::at`] must pass that same sheet.
pub(crate) fn new(budget: usize) -> Self {
Self { by_start: None, next: 0, active: Vec::new(), budget, scanned: false }
}
/// Comparisons left for the rest of the pass.
pub(crate) fn left(&self) -> usize {
self.budget
}
/// Takes `n` comparisons from the budget; `false` once it is spent.
fn charge(&mut self, n: usize) -> bool {
match self.budget.checked_sub(n) {
Some(rest) => {
self.budget = rest;
true
}
None => {
self.budget = 0;
false
}
}
}
/// The merges covering `row`, or `None` when the budget is spent. Rows must not decrease
/// between calls (skipping rows is fine).
fn at(&mut self, sh: &gridcraft_model::Sheet, row: u32) -> Option<Vec<gridcraft_core::RangeRef>> {
if self.by_start.is_none() && !self.scanned {
self.scanned = true;
if !self.charge(sh.merges.len()) {
return None;
}
return Some(sh.merges.iter().copied().filter(|m| !m.is_single() && m.start.row <= row && row <= m.end.row).collect());
}
if self.by_start.is_none() {
// Sorting is about n log2 n comparisons, charged before it starts.
let n = sh.merges.len();
if !self.charge(n.saturating_mul((usize::BITS - n.leading_zeros()) as usize)) {
return None;
}
let mut v: Vec<gridcraft_core::RangeRef> = sh.merges.iter().copied().filter(|m| !m.is_single()).collect();
v.sort_unstable_by_key(|m| m.start.row);
self.by_start = Some(v);
}
let by_start = self.by_start.as_ref()?;
while let Some(m) = by_start.get(self.next).filter(|m| m.start.row <= row) {
self.active.push(*m);
self.next += 1;
}
if !self.charge(self.active.len()) {
return None;
}
self.active.retain(|m| m.end.row >= row);
Some(self.active.clone())
}
}
/// [`auto_row_height`] with the sheet's merges taken from `sweep`.
pub(crate) fn fit_row_height(wb: &mut Workbook, si: usize, row: u32, sweep: &mut MergeSweep) {
// Out of budget: the row keeps the height it has.
if sweep.left() == 0 {
return;
}
let Some(sh) = wb.sheet(si) else { return };
if sh.rows.get(&row).is_some_and(|i| i.custom) {
return;
@@ -750,9 +853,14 @@ pub fn auto_row_height(wb: &mut Workbook, si: usize, row: u32) {
// picked out once, and per cell the candidates are re-checked (normally none or a couple;
// a sheet with many merges and many saved columns would otherwise scan all of them per
// cell, rows × merges × columns).
let row_merges: Vec<gridcraft_core::RangeRef> =
sh.merges.iter().copied().filter(|m| !m.is_single() && m.start.row <= row && row <= m.end.row).collect();
// Rows are fitted in ascending order, so the merges touching this one come from a sweep over
// the sheet's merges sorted once, not from a scan of all of them. Out of budget: the row keeps
// the height it has.
let Some(row_merges) = sweep.at(sh, row) else { return };
for (c, cell) in sh.cells.row(row, 0, gridcraft_core::MAX_COLS - 1) {
if !sweep.charge(row_merges.len()) {
return;
}
// Merged cells never auto-fit their rows, like Excel (column autofit skips them too).
if row_merges.iter().any(|m| m.contains(CellRef::new(row, c))) {
continue;
@@ -796,3 +904,40 @@ impl Ctx<'_> {
self.changed.push((sheet, c));
}
}
#[cfg(test)]
mod merge_sweep_tests {
use super::*;
#[test]
fn the_budget_carries_from_one_sheets_sweep_to_the_next() {
let mut sheet = gridcraft_model::Sheet::new("S");
for r in ["A1:B2", "C1:D2", "E1:F2"] {
sheet.merges.push(gridcraft_core::RangeRef::parse(r).unwrap());
}
let mut first = MergeSweep::new(5);
assert!(first.at(&sheet, 0).is_some(), "the first row is a plain scan: three comparisons");
assert!(first.at(&sheet, 1).is_none(), "the second row needs the sort: six comparisons, two are left");
assert_eq!(first.left(), 0);
// The next sheet starts with what is left: nothing, so it fits nothing.
let mut second = MergeSweep::new(first.left());
assert!(second.at(&sheet, 0).is_none());
assert!(MergeSweep::new(MERGE_FIT_WORK).at(&sheet, 0).is_some());
}
#[test]
fn sorting_the_merges_is_charged() {
let mut sheet = gridcraft_model::Sheet::new("S");
for r in ["A1:B2", "C1:D2", "E1:F2", "G1:H2"] {
sheet.merges.push(gridcraft_core::RangeRef::parse(r).unwrap());
}
// The first row costs a plain scan (4); from the second on the merges are sorted, about
// 4 * 3 comparisons: 7 left is not enough, 12 plus the row's 4 is.
let mut sweep = MergeSweep::new(11);
assert!(sweep.at(&sheet, 0).is_some());
assert!(sweep.at(&sheet, 1).is_none());
assert_eq!(sweep.left(), 0);
let mut sweep = MergeSweep::new(20);
assert!(sweep.at(&sheet, 0).is_some() && sweep.at(&sheet, 1).is_some());
}
}
+67 -20
View File
@@ -152,26 +152,32 @@ fn input_range(s: &Session, p: &Json, keys: &[&str]) -> Result<(usize, RangeRef)
/// Splits a range into variables (columns, or rows when `by_rows`); with `labels` the first
/// cell of each names it, else `default(i)` does.
fn variables(s: &Session, si: usize, r: RangeRef, by_rows: bool, labels: bool, default: &dyn Fn(usize) -> String) -> Result<Vec<Var>> {
let d = s.doc()?;
let sh = d.wb.sheet(si).ok_or(EngineError::NoDocument)?;
let (outer, inner) = if by_rows { (r.height(), r.width()) } else { (r.width(), r.height()) };
let outer = if by_rows { r.height() } else { r.width() };
let mut out = Vec::with_capacity(outer as usize);
for i in 0..outer {
let cell = |j: u32| if by_rows { CellRef::new(r.start.row + i, r.start.col + j) } else { CellRef::new(r.start.row + j, r.start.col + i) };
let mut vals: Vec<Value> = (0..inner).map(|j| sh.value(cell(j))).collect();
let mut name = default(i as usize);
if labels && !vals.is_empty() {
let first = vals.remove(0);
let t = first.display();
if !t.trim().is_empty() {
name = t;
}
}
out.push(Var { name, vals });
out.push(variable(s, si, r, by_rows, labels, default, i)?);
}
Ok(out)
}
/// Variable `i` of [`variables`], read alone so a caller that may stop early needn't build them all.
fn variable(s: &Session, si: usize, r: RangeRef, by_rows: bool, labels: bool, default: &dyn Fn(usize) -> String, i: u32) -> Result<Var> {
let d = s.doc()?;
let sh = d.wb.sheet(si).ok_or(EngineError::NoDocument)?;
let inner = if by_rows { r.width() } else { r.height() };
let cell = |j: u32| if by_rows { CellRef::new(r.start.row + i, r.start.col + j) } else { CellRef::new(r.start.row + j, r.start.col + i) };
let mut vals: Vec<Value> = (0..inner).map(|j| sh.value(cell(j))).collect();
let mut name = default(i as usize);
if labels && !vals.is_empty() {
let first = vals.remove(0);
let t = first.display();
if !t.trim().is_empty() {
name = t;
}
}
Ok(Var { name, vals })
}
/// The numbers of a variable; blanks are skipped when `skip_blank`, anything else that isn't a
/// number is an error.
fn numbers(v: &[Value], skip_blank: bool) -> Result<Vec<f64>> {
@@ -272,11 +278,25 @@ struct Grid {
cells: BTreeMap<(u32, u32), Value>,
/// Cells shown as percentages.
pct: Vec<(u32, u32)>,
/// Rows and columns spanned by `cells`.
rows: u32,
cols: u32,
/// A cell was dropped because the report outgrew [`MAX_OUTPUT_CELLS`]: the report is refused.
too_large: bool,
}
impl Grid {
/// Stores a value. Past [`MAX_OUTPUT_CELLS`] (the most a report writes) nothing more is kept,
/// so a builder that would produce millions of entries stops allocating them, and
/// [`write_report`] refuses the report.
fn put(&mut self, r: u32, c: u32, v: Value) {
if !v.is_empty() {
let (rows, cols) = (self.rows.max(r.saturating_add(1)), self.cols.max(c.saturating_add(1)));
if self.too_large || u64::from(rows) * u64::from(cols) > MAX_OUTPUT_CELLS {
self.too_large = true;
return;
}
(self.rows, self.cols) = (rows, cols);
self.cells.insert((r, c), v);
}
}
@@ -296,9 +316,7 @@ impl Grid {
}
/// (rows, cols) spanned.
fn extent(&self) -> (u32, u32) {
let rows = self.cells.keys().map(|(r, _)| r + 1).max().unwrap_or(0);
let cols = self.cells.keys().map(|(_, c)| c + 1).max().unwrap_or(0);
(rows, cols)
(self.rows, self.cols)
}
}
@@ -310,7 +328,7 @@ fn idx(i: usize) -> u32 {
fn descriptive(s: &Session, p: &Json) -> Result<Grid> {
let (si, r) = input_range(s, p, &["range", "input"])?;
let vars = variables(s, si, r, by_rows(p), flag(p, "labels"), &|i| format!("Column{}", i + 1))?;
let (by_rows, labels) = (by_rows(p), flag(p, "labels"));
let summary = p.get("summary").map(|_| flag(p, "summary")).unwrap_or(true);
let conf = confidence(p, "confidence")?;
let kl = num(p, "kthLargest").filter(|k| *k >= 1.0).map(|k| k as usize);
@@ -319,7 +337,14 @@ fn descriptive(s: &Session, p: &Json) -> Result<Grid> {
return Err(bad(CMD, "choose at least one of summary, confidence, kthLargest or kthSmallest"));
}
let mut g = Grid::default();
for (i, v) in vars.iter().enumerate() {
// Variables are read one at a time: a million-row range grouped by rows is a million variables.
for i in 0..(if by_rows { r.height() } else { r.width() }) {
// The report is refused once the grid is too large: the rest would only build discarded labels.
if g.too_large {
break;
}
let v = variable(s, si, r, by_rows, labels, &|i| format!("Column{}", i + 1), i)?;
let i = i as usize;
let x = numbers(&v.vals, true)?;
let c = idx(i).saturating_mul(2);
g.txt(0, c, &v.name);
@@ -1749,7 +1774,7 @@ fn sampling(s: &Session, p: &Json, seed: Option<u64>) -> Result<Grid> {
fn write_report(s: &mut Session, p: &Json, tool: &str, g: Grid) -> Result<Json> {
let (rows, cols) = g.extent();
let (rows, cols) = (rows.max(1), cols.max(1));
if u64::from(rows) * u64::from(cols) > MAX_OUTPUT_CELLS {
if g.too_large || u64::from(rows) * u64::from(cols) > MAX_OUTPUT_CELLS {
return Err(bad(CMD, "the report is too large"));
}
let d = s.doc()?;
@@ -1914,6 +1939,28 @@ mod tests {
x
}
#[test]
fn a_report_stops_growing_past_the_output_limit() {
// Entries past the limit are not kept, and the report is refused.
let mut g = super::Grid::default();
g.num(1999, 1999, 1.0);
assert!(!g.too_large);
g.num(0, 2000, 1.0);
assert!(g.too_large);
assert_eq!(g.cells.len(), 1);
assert_eq!(g.extent(), (2000, 2000));
// Row-grouped statistics of a tall, almost empty column: one variable per row would make
// millions of report entries; the report is refused instead of built.
let mut s = s();
s.execute("range.setValues", json!({"range": "A1", "values": [[1]]})).unwrap();
s.execute("range.setValues", json!({"range": "A1000000", "values": [[2]]})).unwrap();
let err = s
.execute("data.analysisToolPak", json!({"tool": "descriptive", "range": "A1:A1000000", "groupedBy": "rows", "output": "C1"}))
.expect_err("report too large");
assert!(err.to_string().contains("too large"), "{err}");
assert_eq!(v(&s, "C1"), Value::Empty);
}
#[test]
fn forecast_standard_errors_match_the_full_window_rescan() {
// #1043: the running sum must give what re-adding every window gave, gaps included.
+5 -1
View File
@@ -317,5 +317,9 @@ pub fn filter_values(wb: &Workbook, sheet: usize, col: u32) -> Vec<(String, bool
let key = if blank { sort_key(&Value::Empty) } else { sort_key(&v) };
seen.entry((key, t)).or_insert(on);
}
seen.into_iter().map(|((_, t), on)| (t, on)).collect()
let mut values: Vec<_> = seen.into_iter().collect();
values.sort_by(|((a, ta), _), ((b, tb), _)| {
a.0.cmp(&b.0).then(a.1.cmp(&b.1)).then_with(|| gridcraft_core::compare_text(&a.2, &b.2)).then_with(|| ta.cmp(tb))
});
values.into_iter().map(|((_, t), on)| (t, on)).collect()
}
+11
View File
@@ -53,6 +53,17 @@ fn subtotal_puts_summary_rows_below_their_detail() {
assert_eq!(sh.value(CellRef::new(3, 0)).display(), "West");
}
#[test]
fn filter_list_orders_arabic_like_sort() {
let mut s = Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [["H"], ["بج"], ["بُت"], ["\u{200F}بب"]]})).unwrap();
s.execute("data.filter", json!({"range": "A1:A4"})).unwrap();
let doc = s.doc().unwrap();
let list: Vec<String> = filter_values(&doc.wb, 0, 0).into_iter().map(|(t, _)| t).collect();
assert_eq!(list, ["\u{200F}بب", "بُت", "بج"]);
}
#[test]
fn filter_by_huge_numeric_column_is_rejected_not_overflowed() {
let mut s = Session::new();
+142 -28
View File
@@ -14,8 +14,13 @@ fn matches(hay: &str, what: &str, case: bool, whole: bool) -> bool {
}
/// `*`/`?` wildcard match with `~` escapes.
///
/// The pattern runs as a bit-parallel automaton over the text: one pass, each character costing
/// one step per 64 pattern positions. Backtracking to the last `*` after a mismatch rescans the
/// text and costs text × pattern comparisons on a long, repetitive cell, for every cell a
/// filter or Find visits.
pub(crate) fn wildcard(text: &str, pat: &str) -> bool {
let t: Vec<char> = text.chars().collect();
// The pattern as (character, escaped) items, runs of `*` folded into one.
let mut p: Vec<(char, bool)> = Vec::new();
let mut it = pat.chars();
while let Some(c) = it.next() {
@@ -23,32 +28,85 @@ pub(crate) fn wildcard(text: &str, pat: &str) -> bool {
if let Some(n) = it.next() {
p.push((n, true));
}
} else {
} else if c != '*' || p.last() != Some(&('*', false)) {
p.push((c, false));
}
}
let (mut ti, mut pi) = (0usize, 0usize);
let (mut star, mut mark) = (None, 0usize);
while ti < t.len() {
if pi < p.len() && (p[pi] == ('?', false) || (p[pi].0 == t[ti] && p[pi] != ('*', false))) {
ti += 1;
pi += 1;
} else if pi < p.len() && p[pi] == ('*', false) {
star = Some(pi);
mark = ti;
pi += 1;
} else if let Some(sp) = star {
pi = sp + 1;
mark += 1;
ti = mark;
} else {
return false;
let words = (p.len() + 1).div_ceil(64);
let bit = |set: &mut [u64], i: usize| set[i / 64] |= 1 << (i % 64);
// Positions holding a `*` and a `?`, and where each literal character sits in the pattern.
let mut stars = vec![0u64; words];
let mut any = vec![0u64; words];
let mut at: std::collections::HashMap<char, Vec<usize>> = std::collections::HashMap::new();
for (i, item) in p.iter().enumerate() {
match item {
('*', false) => bit(&mut stars, i),
('?', false) => bit(&mut any, i),
(c, _) => at.entry(*c).or_default().push(i),
}
}
while pi < p.len() && p[pi] == ('*', false) {
pi += 1;
// A literal's bit mask is built when the text first reads it, and kept for at most
// `MAX_CACHED_MASKS` literals: a long pattern of mostly distinct characters would otherwise
// allocate a mask of the pattern's width for each one up front. When there are more literals
// than that, the ones kept are those at the most pattern positions (the dearest to rebuild);
// each of the rest has at most pattern length / `MAX_CACHED_MASKS` positions, so rebuilding
// its mask in `scratch` costs less than one step of the scan.
const MAX_CACHED_MASKS: usize = 256;
let cacheable: Option<std::collections::HashSet<char>> = (at.len() > MAX_CACHED_MASKS).then(|| {
let mut by_count: Vec<(&char, usize)> = at.iter().map(|(c, v)| (c, v.len())).collect();
by_count.sort_unstable_by_key(|&(_, n)| std::cmp::Reverse(n));
by_count.into_iter().take(MAX_CACHED_MASKS).map(|(c, _)| *c).collect()
});
let mut masks: std::collections::HashMap<char, Vec<u64>> = std::collections::HashMap::new();
let mut scratch = vec![0u64; words];
// A set bit i: the text read so far matches the first i pattern items. A `*` matches nothing
// as well, so a state at a `*` is also at the item after it.
let close = |set: &mut Vec<u64>| {
let mut carry = 0u64;
for w in 0..words {
let s = set[w] & stars[w];
let next = (s << 1) | carry;
carry = s >> 63;
set[w] |= next;
}
};
let mut state = vec![0u64; words];
bit(&mut state, 0);
close(&mut state);
let mut next = vec![0u64; words];
for ch in text.chars() {
let lit: Option<&[u64]> = match at.get(&ch) {
None => None,
Some(positions) if cacheable.as_ref().is_none_or(|k| k.contains(&ch)) => {
let mask = masks.entry(ch).or_insert_with(|| {
let mut m = vec![0u64; words];
positions.iter().for_each(|&i| bit(&mut m, i));
m
});
Some(mask)
}
Some(positions) => {
scratch.fill(0);
positions.iter().for_each(|&i| bit(&mut scratch, i));
Some(&scratch)
}
};
let mut carry = 0u64;
let mut alive = 0u64;
for w in 0..words {
// Items that read `ch` advance; a `*` reads it and stays.
let m = state[w] & (any[w] | lit.map_or(0, |l| l[w]));
next[w] = (m << 1) | carry | (state[w] & stars[w]);
carry = m >> 63;
alive |= next[w];
}
if alive == 0 {
return false;
}
std::mem::swap(&mut state, &mut next);
close(&mut state);
}
pi == p.len()
state[p.len() / 64] & (1 << (p.len() % 64)) != 0
}
pub(super) fn find(s: &mut Session, p: &Json) -> Result<Json> {
@@ -112,9 +170,19 @@ pub(super) fn find(s: &mut Session, p: &Json) -> Result<Json> {
Ok(json!({"cell": c.a1(), "sheet": si, "value": t, "count": hits.len()}))
}
/// Most characters a cell holds (Excel's limit).
const REPLACE_CELL_MAX_CHARS: usize = 32_767;
/// Longest text one replacement may give a cell, in bytes: the character limit, each up to four
/// bytes. Every occurrence copies the replacement text, so a short cell full of matches and a
/// long replacement would otherwise expand to gigabytes. The character limit is checked on what
/// fits in this.
const REPLACE_CELL_MAX_BYTES: usize = REPLACE_CELL_MAX_CHARS * 4;
/// Most bytes of replaced text one Replace All builds across all its cells.
const REPLACE_TOTAL_MAX_BYTES: usize = 256 * 1024 * 1024;
pub(super) fn replace(s: &mut Session, p: &Json) -> Result<Json> {
let what = str_param(p, "what").unwrap_or("").to_string();
let with = str_param(p, "with").unwrap_or("").to_string();
let with = str_param(p, "with").unwrap_or("");
if what.is_empty() {
return Err(bad("edit.replace", "missing `what`"));
}
@@ -128,6 +196,7 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result<Json> {
let n = edit(s, |cx| {
let sheets: Vec<usize> = if workbook { (0..cx.wb.sheets.len()).collect() } else { vec![cx.wb.active_sheet()] };
let mut n = 0;
let mut total = 0usize;
for si in sheets {
let cells: Vec<(CellRef, String)> =
cx.wb.sheet(si).map(|sh| sh.cells.iter().map(|(c, _)| (c, sh.input_text(c))).collect()).unwrap_or_default();
@@ -139,12 +208,20 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result<Json> {
continue;
}
let new = if whole {
with.clone()
(with.len() <= REPLACE_CELL_MAX_BYTES).then(|| with.to_string())
} else if case {
text.replace(&what, &with)
replace_cs_within(&text, &what, with, REPLACE_CELL_MAX_BYTES)
} else {
replace_ci(&text, &what, &with)
replace_ci_within(&text, &what, with, REPLACE_CELL_MAX_BYTES)
};
let new = new.filter(|n| n.chars().count() <= REPLACE_CELL_MAX_CHARS);
let Some(new) = new else {
return Err(EngineError::Other(format!("The replacement makes the text of {} too long.", c.a1())));
};
total = total.saturating_add(new.len());
if total > REPLACE_TOTAL_MAX_BYTES {
return Err(EngineError::Other("The replacements make too much text. Replace in a smaller range.".into()));
}
// A Python cell's text is its code: the replacement edits the code and the cell
// stays a Python cell. Code that would no longer fit in a PY formula is left as it was.
let old_py = cx.wb.sheet(si).and_then(|sh| sh.cell(c)).and_then(|x| x.py.clone());
@@ -199,7 +276,37 @@ fn folds_in_place(s: &str) -> bool {
})
}
#[cfg(test)]
pub(super) fn replace_ci(text: &str, what: &str, with: &str) -> String {
replace_ci_within(text, what, with, usize::MAX).unwrap_or_default()
}
/// `text` with every `what` replaced by `with`, or `None` once the result passes `limit` bytes
/// (checked before each copy, so nothing larger is allocated).
pub(super) fn replace_cs_within(text: &str, what: &str, with: &str, limit: usize) -> Option<String> {
if what.is_empty() {
return (text.len() <= limit).then(|| text.to_string());
}
let mut out = String::new();
let mut i = 0;
for (pos, m) in text.match_indices(what) {
if out.len() + (pos - i) + with.len() > limit {
return None;
}
out.push_str(text.get(i..pos).unwrap_or(""));
out.push_str(with);
i = pos + m.len();
}
let rest = text.get(i..).unwrap_or("");
if out.len() + rest.len() > limit {
return None;
}
out.push_str(rest);
Some(out)
}
/// Case-insensitive [`replace_cs_within`].
pub(super) fn replace_ci_within(text: &str, what: &str, with: &str, limit: usize) -> Option<String> {
let lower = text.to_lowercase();
let w = what.to_lowercase();
// Slicing `text` at positions found in `lower` is only sound when folding kept every byte
@@ -207,15 +314,22 @@ pub(super) fn replace_ci(text: &str, what: &str, with: &str) -> String {
// boundaries (U+212A KELVIN SIGN shrinks 3→1 bytes, U+0130 grows 2→3), where the tail used
// to be dropped. Those fall back to the case-sensitive replace, as length changes already do.
if lower.len() != text.len() || w.is_empty() || !folds_in_place(text) || !folds_in_place(what) {
return text.replace(what, with);
return replace_cs_within(text, what, with, limit);
}
let mut out = String::new();
let mut i = 0;
while let Some(pos) = lower.get(i..).and_then(|s| s.find(&w)) {
if out.len() + pos + with.len() > limit {
return None;
}
out.push_str(text.get(i..i + pos).unwrap_or(""));
out.push_str(with);
i += pos + w.len();
}
out.push_str(text.get(i..).unwrap_or(""));
out
let rest = text.get(i..).unwrap_or("");
if out.len() + rest.len() > limit {
return None;
}
out.push_str(rest);
Some(out)
}
+111 -50
View File
@@ -17,11 +17,13 @@ pub(in super::super) fn go_to(s: &mut Session, p: &Json) -> Result<Json> {
}
let Some(reference) = str_param(p, "reference").or_else(|| str_param(p, "range")) else { return Err(bad("edit.goTo", "missing `reference`")) };
let d = s.doc()?;
let mut names = Resolver::new(&d.wb);
let mut target: Option<usize> = None;
let mut areas: Vec<RangeRef> = Vec::new();
let mut named = false;
for part in reference_parts(reference) {
let Some((sheet, found, by_name)) = go_to_target(&d.wb, d.wb.active_sheet(), part, 0) else {
let mut found = Vec::new();
let Some((sheet, _, by_name)) = names.target(d.wb.active_sheet(), part, 0, Some(&mut found)) else {
return Err(EngineError::Other("Reference isn't valid.".into()));
};
// One selection holds the areas of one sheet.
@@ -77,57 +79,114 @@ pub(super) fn reference_parts(text: &str) -> impl Iterator<Item = &str> {
})
}
/// Where one part of a Go To reference points, seen from sheet `sheet`: the sheet, its areas,
/// and whether a defined name or a table gave them. `Sheet2!Name` is the name scoped to Sheet2.
/// A name counts when what it refers to is a reference; a constant or a formula is not one.
fn go_to_target(wb: &gridcraft_model::Workbook, sheet: usize, part: &str, depth: usize) -> Option<(usize, Vec<RangeRef>, bool)> {
if part.is_empty() || depth > GO_TO_NAME_DEPTH {
return None;
}
let (prefix, body) = split_sheet(part);
let on = match &prefix {
Some(n) => wb.sheet_index(n)?,
None => sheet,
};
if let Some(r) = RangeRef::parse(&body.replace('$', "")) {
return Some((on, vec![r], false));
}
let name = match prefix {
Some(_) => wb.names.iter().find(|n| n.scope == Some(on) && gridcraft_formula::same_name(&n.name, body)),
None => wb.name(body, sheet),
};
if let Some(n) = name {
return name_areas(wb, n, sheet, depth).map(|(t, areas)| (t, areas, true));
}
if prefix.is_none()
&& let Some((si, ti)) = wb.table(body)
{
// As in Excel, a table's name stands for its data, without the header and totals rows.
let t = wb.sheet(si)?.tables.get(ti)?;
return Some((si, vec![t.data_range().unwrap_or(t.range)], true));
}
None
/// Resolves Go To references against one workbook. Defined names are found through an index
/// built once, and whether a name refers to a reference is remembered: listing the names of a
/// workbook full of names that alias other names would otherwise scan every name for every alias
/// of every name.
struct Resolver<'a> {
wb: &'a gridcraft_model::Workbook,
/// Indexes into `wb.names` by scope and folded name, in workbook order.
by_name: std::collections::HashMap<(Option<usize>, String), Vec<usize>>,
/// [`Resolver::name_areas`] without areas, by name index, sheet and depth: the target sheet
/// and the number of areas.
memo: std::collections::HashMap<(usize, usize, usize), Option<(usize, usize)>>,
}
/// The sheet and areas defined name `n` refers to, seen from sheet `sheet`, when that is a
/// reference.
fn name_areas(wb: &gridcraft_model::Workbook, n: &gridcraft_model::DefinedName, sheet: usize, depth: usize) -> Option<(usize, Vec<RangeRef>)> {
let from = n.scope.unwrap_or(sheet);
let formula = n.formula.trim().trim_start_matches('=');
let mut target: Option<usize> = None;
let mut areas = Vec::new();
for p in reference_parts(formula) {
let (s, found, _) = go_to_target(wb, from, p, depth + 1)?;
if target.is_some_and(|t| t != s) {
return None;
}
target = Some(s);
areas.extend(found);
if areas.len() > GO_TO_MAX_AREAS {
return None;
impl<'a> Resolver<'a> {
fn new(wb: &'a gridcraft_model::Workbook) -> Self {
let mut by_name: std::collections::HashMap<(Option<usize>, String), Vec<usize>> = std::collections::HashMap::new();
for (i, n) in wb.names.iter().enumerate() {
by_name.entry((n.scope, gridcraft_formula::fold_name(&n.name))).or_default().push(i);
}
Self { wb, by_name, memo: std::collections::HashMap::new() }
}
/// The first name of scope `scope` that `same` accepts, as a scan of `wb.names` finds it.
/// Names `same` accepts fold alike, so only that bucket is checked.
fn find(&self, scope: Option<usize>, body: &str, same: impl Fn(&str, &str) -> bool) -> Option<usize> {
self.by_name
.get(&(scope, gridcraft_formula::fold_name(body)))?
.iter()
.copied()
.find(|&i| self.wb.names.get(i).is_some_and(|n| same(&n.name, body)))
}
/// Where one part of a Go To reference points, seen from sheet `sheet`: the sheet, how many
/// areas (appended to `areas` when given), and whether a defined name or a table gave them.
/// `Sheet2!Name` is the name scoped to Sheet2. A name counts when what it refers to is a
/// reference; a constant or a formula is not one.
fn target(&mut self, sheet: usize, part: &str, depth: usize, areas: Option<&mut Vec<RangeRef>>) -> Option<(usize, usize, bool)> {
let wb = self.wb;
if part.is_empty() || depth > GO_TO_NAME_DEPTH {
return None;
}
let (prefix, body) = split_sheet(part);
let on = match &prefix {
Some(n) => wb.sheet_index(n)?,
None => sheet,
};
if let Some(r) = RangeRef::parse(&body.replace('$', "")) {
if let Some(a) = areas {
a.push(r);
}
return Some((on, 1, false));
}
// As `Workbook::name`: the sheet's own name, else the workbook's.
let name = match prefix {
Some(_) => self.find(Some(on), body, gridcraft_formula::same_name),
None => self.find(Some(sheet), body, str::eq_ignore_ascii_case).or_else(|| self.find(None, body, str::eq_ignore_ascii_case)),
};
if let Some(i) = name {
return self.name_areas(i, sheet, depth, areas).map(|(t, n)| (t, n, true));
}
if prefix.is_none()
&& let Some((si, ti)) = wb.table(body)
{
// As in Excel, a table's name stands for its data, without the header and totals rows.
let t = wb.sheet(si)?.tables.get(ti)?;
if let Some(a) = areas {
a.push(t.data_range().unwrap_or(t.range));
}
return Some((si, 1, true));
}
None
}
/// The sheet and number of areas defined name `wb.names[i]` refers to, seen from sheet
/// `sheet`, when that is a reference; the areas are appended to `areas` when given.
fn name_areas(&mut self, i: usize, sheet: usize, depth: usize, mut areas: Option<&mut Vec<RangeRef>>) -> Option<(usize, usize)> {
let key = (i, sheet, depth);
if areas.is_none()
&& let Some(known) = self.memo.get(&key)
{
return *known;
}
let out = self.name_areas_uncached(i, sheet, depth, areas.as_deref_mut());
if areas.is_none() {
self.memo.insert(key, out);
}
out
}
fn name_areas_uncached(&mut self, i: usize, sheet: usize, depth: usize, mut areas: Option<&mut Vec<RangeRef>>) -> Option<(usize, usize)> {
let n = self.wb.names.get(i)?;
let from = n.scope.unwrap_or(sheet);
let formula = n.formula.trim().trim_start_matches('=');
let mut target: Option<usize> = None;
let mut count = 0usize;
for p in reference_parts(formula) {
let (s, found, _) = self.target(from, p, depth + 1, areas.as_deref_mut())?;
if target.is_some_and(|t| t != s) {
return None;
}
target = Some(s);
count += found;
if count > GO_TO_MAX_AREAS {
return None;
}
}
target.map(|t| (t, count))
}
target.map(|t| (t, areas))
}
/// The names the Go To dialog lists, seen from sheet `sheet`: the named ranges of the workbook and
@@ -137,11 +196,13 @@ fn name_areas(wb: &gridcraft_model::Workbook, n: &gridcraft_model::DefinedName,
pub fn go_to_names(wb: &gridcraft_model::Workbook, sheet: usize) -> Vec<String> {
let own: std::collections::HashSet<String> =
wb.names.iter().filter(|n| n.scope == Some(sheet)).map(|n| gridcraft_formula::fold_name(&n.name)).collect();
let mut names = Resolver::new(wb);
let mut out: Vec<String> = wb
.names
.iter()
.filter(|n| !n.hidden && name_areas(wb, n, sheet, 0).is_some())
.filter_map(|n| match n.scope {
.enumerate()
.filter(|(i, n)| !n.hidden && names.name_areas(*i, sheet, 0, None).is_some())
.filter_map(|(_, n)| match n.scope {
None if own.contains(&gridcraft_formula::fold_name(&n.name)) => None,
None => Some(n.name.clone()),
Some(s) if s == sheet => Some(n.name.clone()),
+163
View File
@@ -42,6 +42,28 @@ fn go_to_lists_the_named_ranges_of_every_sheet() {
assert_eq!(list["names"], json!(["Areas", "Local", "Price", "Sheet1!Own", "type"]), "{list}");
}
/// Thousands of names that alias other names are listed with one lookup per alias, not a scan of
/// every name for each; aliases resolve through the chain like a single name does.
#[test]
fn go_to_lists_many_aliases_and_follows_them() {
let mut s = go_to_workbook();
let wb = std::sync::Arc::make_mut(&mut s.doc_mut().unwrap().wb);
let alias = |name: String, formula: String| gridcraft_model::DefinedName { name, scope: None, formula, comment: String::new(), hidden: false };
wb.names.push(alias("Chain0".into(), "Areas".into()));
for i in 1..20_000 {
wb.names.push(alias(format!("Chain{i}"), format!("Chain{},price", i - 1)));
}
let list = s.execute("edit.goTo", json!({"list": true})).unwrap();
let names: Vec<&str> = list["names"].as_array().unwrap().iter().filter_map(|n| n.as_str()).collect();
// Chain0..Chain6 are within the name depth Go To follows; longer chains are not ranges.
for i in 0..7 {
assert!(names.contains(&format!("Chain{i}").as_str()), "Chain{i} in {names:?}");
}
assert!(!names.contains(&"Chain7"), "{names:?}");
let out = s.execute("edit.goTo", json!({"reference": "chain1"})).unwrap();
assert_eq!(out["selection"], "A1,C3:C4,B2:B5", "{out}");
}
#[test]
fn go_to_a_name_scoped_to_another_sheet_goes_to_that_sheet() {
let mut s = go_to_workbook();
@@ -302,6 +324,103 @@ fn wildcards() {
assert!(wildcard("", "*"));
}
/// The backtracking matcher `wildcard` replaced, as the reference for its results.
fn wildcard_backtracking(text: &str, pat: &str) -> bool {
let t: Vec<char> = text.chars().collect();
let mut p: Vec<(char, bool)> = Vec::new();
let mut it = pat.chars();
while let Some(c) = it.next() {
if c == '~' {
if let Some(n) = it.next() {
p.push((n, true));
}
} else {
p.push((c, false));
}
}
let (mut ti, mut pi) = (0usize, 0usize);
let (mut star, mut mark) = (None, 0usize);
while ti < t.len() {
if pi < p.len() && (p[pi] == ('?', false) || (p[pi].0 == t[ti] && p[pi] != ('*', false))) {
ti += 1;
pi += 1;
} else if pi < p.len() && p[pi] == ('*', false) {
star = Some(pi);
mark = ti;
pi += 1;
} else if let Some(sp) = star {
pi = sp + 1;
mark += 1;
ti = mark;
} else {
return false;
}
}
while pi < p.len() && p[pi] == ('*', false) {
pi += 1;
}
pi == p.len()
}
#[test]
fn wildcard_agrees_with_backtracking() {
// Every text up to 5 characters over a small alphabet against every pattern up to 4 items.
let alphabet = ['a', 'b', '*', '?', '~'];
let all = |n: usize, chars: &[char]| -> Vec<String> {
let mut out = vec![String::new()];
let mut level = vec![String::new()];
for _ in 0..n {
level = level.iter().flat_map(|s| chars.iter().map(move |c| format!("{s}{c}"))).collect();
out.extend(level.iter().cloned());
}
out
};
let texts = all(5, &['a', 'b', '*']);
for pat in all(4, &alphabet) {
for text in &texts {
assert_eq!(wildcard(text, &pat), wildcard_backtracking(text, &pat), "{text:?} ~ {pat:?}");
}
}
// Patterns longer than one 64-bit word.
let long = format!("*{}*b?", "a".repeat(70));
for text in [format!("x{}yb1", "a".repeat(70)), format!("{}b", "a".repeat(71)), "a".repeat(200)] {
assert_eq!(wildcard(&text, &long), wildcard_backtracking(&text, &long), "{text}");
}
}
#[test]
fn wildcard_with_many_distinct_literals_agrees_with_backtracking() {
// 600 distinct characters in the pattern: more than the masks kept, so the rest are rebuilt.
let chars: Vec<char> = (0x4E00u32..0x4E00 + 600).filter_map(char::from_u32).collect();
let word: String = chars.iter().collect();
let pat = format!("*{word}?{}", chars[3]);
for text in [format!("x{word}yq{}", chars[3]), word.clone(), format!("{word}{word}"), chars.iter().rev().collect::<String>()] {
assert_eq!(wildcard(&text, &pat), wildcard_backtracking(&text, &pat), "{} chars", text.chars().count());
}
assert!(wildcard(&format!("{word}zz{}", chars[3]).replace("zz", "z"), &pat));
}
#[test]
fn wildcard_keeps_masks_of_frequent_literals_past_the_cache_limit() {
// 300 distinct characters come before the repeated `a`: past the cached masks, `a` (5,000 positions)
// still has to be read from a kept mask, or it is rebuilt for each of the 20,000 `a` in the text.
let rare: String = (0x4E00u32..0x4E00 + 300).filter_map(char::from_u32).collect();
let pat = format!("*{rare}{}b", "a".repeat(5_000));
let text = "a".repeat(20_000);
assert!(!wildcard(&text, &pat));
assert!(wildcard(&format!("{rare}{}b", "a".repeat(5_000)), &pat));
assert!(!wildcard(&format!("{rare}{}", "a".repeat(5_000)), &pat));
}
#[test]
fn wildcard_on_long_repetitive_text_is_linear() {
// The backtracking matcher compared about text × pattern characters here (10^8).
let text = "a".repeat(20_000);
let pat = format!("*{}b", "a".repeat(5_000));
assert!(!wildcard(&text, &pat));
assert!(wildcard(&format!("{text}b"), &pat));
}
#[test]
fn tsv() {
assert_eq!(parse_tsv("a\tb\n1\t2\n"), vec![vec!["a", "b"], vec!["1", "2"]]);
@@ -424,6 +543,50 @@ fn drag_and_drop_on_another_sheet_stays_on_that_sheet() {
assert_eq!(wb.sheet(two).unwrap().value(CellRef::new(0, 1)), Value::Empty);
}
#[test]
fn replace_refuses_text_that_would_grow_past_a_cell() {
// 10,000 matches × a 1,000-character replacement would make a 10 MB cell (and a few such
// cells gigabytes); the replacement is refused before that text is built.
let mut s = Session::new();
s.new_workbook();
let long = "a".repeat(10_000);
s.execute("range.setValues", json!({"range": "A1", "values": [[long.clone()], ["abc"]]})).unwrap();
for case in [false, true] {
let err = s.execute("edit.replace", json!({"what": "a", "with": "x".repeat(1_000), "matchCase": case})).expect_err("too long");
assert!(err.to_string().contains("too long"), "{err}");
assert_eq!(s.execute("cell.get", json!({"cell": "A1"})).unwrap()["value"], json!(long));
assert_eq!(s.execute("cell.get", json!({"cell": "A2"})).unwrap()["value"], json!("abc"));
}
// A replacement that stays within a cell's limit still happens.
s.execute("edit.replace", json!({"what": "b", "with": "x".repeat(100)})).unwrap();
assert_eq!(s.execute("cell.get", json!({"cell": "A2"})).unwrap()["value"], json!(format!("a{}c", "x".repeat(100))));
}
#[test]
fn replace_refuses_a_result_past_the_character_limit_though_it_fits_the_byte_limit() {
let mut s = Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [["z"]]})).unwrap();
for whole in [false, true] {
let err = s.execute("edit.replace", json!({"what": "z", "with": "x".repeat(32_768), "wholeCell": whole})).expect_err("32,768 characters");
assert!(err.to_string().contains("too long"), "{err}");
assert_eq!(s.execute("cell.get", json!({"cell": "A1"})).unwrap()["value"], json!("z"));
}
s.execute("edit.replace", json!({"what": "z", "with": "x".repeat(32_767)})).unwrap();
assert_eq!(s.execute("cell.get", json!({"cell": "A1"})).unwrap()["value"], json!("x".repeat(32_767)));
}
#[test]
fn bounded_replace_stops_at_its_limit() {
use super::find::{replace_ci_within, replace_cs_within};
assert_eq!(replace_cs_within("abab", "a", "xy", 6).as_deref(), Some("xybxyb"));
assert_eq!(replace_cs_within("abab", "a", "xy", 5), None);
assert_eq!(replace_ci_within("AbAb", "a", "xy", 6).as_deref(), Some("xybxyb"));
assert_eq!(replace_ci_within("AbAb", "a", "xy", 5), None);
assert_eq!(replace_cs_within("abc", "z", "", 3).as_deref(), Some("abc"));
assert_eq!(replace_cs_within("abc", "z", "", 2), None);
}
#[test]
fn replace_case_insensitive() {
assert_eq!(replace_ci("Apple apple APPLE", "apple", "pear"), "pear pear pear");
+63 -3
View File
@@ -42,6 +42,15 @@ pub(super) fn table_resize(s: &mut Session, p: &Json) -> Result<Json> {
})
}
/// Most cells (rows × columns, blanks included) Get Data imports. The rows are built densely
/// before they are written, so a few sparse records or one far-away CSV cell must not expand
/// into a rectangle of billions of slots.
const MAX_IMPORT_CELLS: u64 = 10_000_000;
fn too_many_cells() -> EngineError {
EngineError::Other(format!("The data is too large to import ({MAX_IMPORT_CELLS} cells at most)."))
}
pub(super) fn from_csv(s: &mut Session, p: &Json) -> Result<Json> {
let bytes = if let Some(t) = str_param(p, "text") {
t.as_bytes().to_vec()
@@ -57,15 +66,27 @@ pub(super) fn from_csv(s: &mut Session, p: &Json) -> Result<Json> {
if text.trim_start().starts_with('[') {
let arr: Vec<Json> = serde_json::from_str(&text).map_err(|e| bad("data.getData", e.to_string()))?;
let mut keys: Vec<String> = Vec::new();
let mut seen: std::collections::HashSet<&str> = std::collections::HashSet::new();
for o in &arr {
if let Some(m) = o.as_object() {
for k in m.keys() {
if !keys.contains(k) {
if !seen.contains(k.as_str()) {
// Stop collecting at the column limit.
if keys.len() >= gridcraft_core::MAX_COLS as usize {
return Err(EngineError::Other("The data has more fields than a sheet has columns.".into()));
}
seen.insert(k.as_str());
keys.push(k.clone());
}
}
}
}
if arr.len() as u64 + 1 > u64::from(gridcraft_core::MAX_ROWS) {
return Err(EngineError::Other("The data has more rows than a sheet has.".into()));
}
if (arr.len() as u64 + 1).saturating_mul(keys.len().max(1) as u64) > MAX_IMPORT_CELLS {
return Err(too_many_cells());
}
let mut out = vec![keys.iter().map(|k| json!(k)).collect::<Vec<_>>()];
for o in &arr {
out.push(keys.iter().map(|k| o.get(k).cloned().unwrap_or(Json::Null)).collect());
@@ -75,10 +96,17 @@ pub(super) fn from_csv(s: &mut Session, p: &Json) -> Result<Json> {
let delim = str_param(p, "delimiter")
.and_then(|d| d.chars().next())
.unwrap_or(if text.lines().next().is_some_and(|l| l.contains('\t')) { '\t' } else { ',' });
let opts = gridcraft_xlsx::CsvOptions { delimiter: delim as u8, ..Default::default() };
let wb = gridcraft_xlsx::read_csv(&bytes, &opts).map_err(|e| EngineError::Other(e.to_string()))?;
let opts = gridcraft_xlsx::CsvOptions { delimiter: delim as u8, max_used_cells: Some(MAX_IMPORT_CELLS), ..Default::default() };
// The cell limit is enforced while parsing, before an oversized file is stored.
let wb = gridcraft_xlsx::read_csv(&bytes, &opts).map_err(|e| match e {
gridcraft_xlsx::IoError::TooLarge(_) => too_many_cells(),
e => EngineError::Other(e.to_string()),
})?;
let sh = wb.sheet(0).ok_or(EngineError::NoDocument)?;
let Some(u) = sh.used_range() else { return Err(EngineError::Other("The file is empty.".into())) };
if u.count() > MAX_IMPORT_CELLS {
return Err(too_many_cells());
}
(u.start.row..=u.end.row)
.map(|r| {
(u.start.col..=u.end.col)
@@ -129,3 +157,35 @@ pub(super) fn queries(s: &mut Session, _: &Json) -> Result<Json> {
.collect(),
))
}
#[cfg(test)]
mod tests {
use serde_json::json;
use crate::Session;
fn s() -> Session {
let mut s = Session::new();
s.new_workbook();
s
}
#[test]
fn sparse_imports_are_refused_before_their_rows_are_built() {
let mut s = s();
let sheets = s.doc().unwrap().wb.sheets.len();
// 5,000 records with one field each: 5,001 × 5,000 slots once every record gets every field.
let records: Vec<_> = (0..5_000).map(|i| json!({ format!("k{i}"): i })).collect();
let text = serde_json::to_string(&records).unwrap();
let err = s.execute("data.getData", json!({"text": text})).expect_err("too many cells");
assert!(err.to_string().contains("too large"), "{err}");
// A CSV whose last line reaches 16,001 columns, 1,000 lines down.
let text = format!("1{}{}x", "\n".repeat(999), ",".repeat(16_000));
let err = s.execute("data.fromTextCsv", json!({"text": text})).expect_err("too many cells");
assert!(err.to_string().contains("too large"), "{err}");
assert_eq!(s.doc().unwrap().wb.sheets.len(), sheets);
// Small imports still work.
let out = s.execute("data.getData", json!({"text": r#"[{"a": 1}, {"b": 2}]"#})).unwrap();
assert_eq!((out["rows"].as_u64(), out["cols"].as_u64()), (Some(3), Some(2)), "{out}");
}
}
+73 -3
View File
@@ -36,13 +36,28 @@ fn max_periods(history: usize) -> u32 {
u32::try_from(MAX_WORK / h).unwrap_or(u32::MAX).clamp(1, MAX_PERIODS)
}
/// Most forecast rows when the model is fit to the first `fit` of `history` rows, which span
/// `slots` timeline slots once their gaps are filled. With a Forecast Start inside the history,
/// every row after it is forecast too (a hindcast row holds the same ETS formulas, each refitting
/// the model over the `slots` slots), so those rows (counted from the raw rows) count against
/// [`MAX_WORK`] before any forecast row does. `None` when the hindcast alone exceeds it.
fn max_periods_from(history: usize, fit: usize, slots: usize) -> Option<u32> {
if fit >= history {
return Some(max_periods(history));
}
let f = u64::try_from(slots.max(fit)).unwrap_or(u64::MAX).max(1);
let hindcast = u64::try_from(history - fit).unwrap_or(u64::MAX);
let rest = (MAX_WORK / f).checked_sub(hindcast).filter(|r| *r >= 1)?;
Some(u32::try_from(rest).unwrap_or(u32::MAX).min(MAX_PERIODS))
}
pub fn specs() -> Vec<CommandSpec> {
vec![cmd!(
"data.forecastSheet",
"Forecast Sheet",
["Data", "Forecast"],
None,
"{timeline?: \"A2:A25\", values?: \"B2:B25\" (default: the first two columns of `range`, the selection or the current region; a text first cell is the header), end?: timeline value or date text | periods?: n (default: a third of the history, at least 1; ≤10,000 and history × periods ≤ 500,000; history ≤ 10,000 rows), forecastStart?: a timeline value or date text on the history, before its last point (hindcast: the model is fit to the points before it, the forecast columns start there and run over the actuals; default and the last point: forecast after the history), confidenceInterval?: bool (default true), confidence?: 0.95 or 95, seasonality?: \"auto\" | 0 (none) | 2–8,760, fillMissing?: interpolate|zeros, aggregate?: average|count|counta|max|median|min|sum, statistics?: bool, chart?: line|column} → {sheet, table, chart, history, periods}. Inserts a sheet before the active one: timeline, values, Forecast (FORECAST.ETS), Lower/Upper Confidence Bound (FORECAST.ETS.CONFINT), optional FORECAST.ETS.STAT table, and a chart.",
"{timeline?: \"A2:A25\", values?: \"B2:B25\" (default: the first two columns of `range`, the selection or the current region; a text first cell is the header), end?: timeline value or date text | periods?: n (default: a third of the history, at least 1; ≤10,000 and history × periods ≤ 500,000, or with a forecastStart inside the history (hindcast rows are forecast too) (the greater of the fitted rows and fitted slots × (periods + rows from forecastStart on, that row included)) ≤ 500,000; history ≤ 10,000 rows), forecastStart?: a timeline value or date text on the history, before its last point (hindcast: the model is fit to the points before it, the forecast columns start there and run over the actuals; default and the last point: forecast after the history), confidenceInterval?: bool (default true), confidence?: 0.95 or 95, seasonality?: \"auto\" | 0 (none) | 2–8,760, fillMissing?: interpolate|zeros, aggregate?: average|count|counta|max|median|min|sum, statistics?: bool, chart?: line|column} → {sheet, table, chart, history, periods}. Inserts a sheet before the active one: timeline, values, Forecast (FORECAST.ETS), Lower/Upper Confidence Bound (FORECAST.ETS.CONFINT), optional FORECAST.ETS.STAT table, and a chart.",
has_doc,
forecast_sheet
)]
@@ -257,13 +272,23 @@ fn forecast_sheet(s: &mut Session, p: &Json) -> Result<Json> {
}
}
};
let cap = max_periods(rows.len());
// The slots the model is fit over: the fit rows' span of the timeline, gaps included.
let slots = match (fit < rows.len(), times.get(fit.wrapping_sub(1)), times.first()) {
(true, Some(last_fit), Some(first)) => match (timeline.steps_after_last(*last_fit), timeline.steps_after_last(*first)) {
(Some(b), Some(a)) => ((b - a).round() + 1.0).clamp(0.0, 1e9) as usize,
_ => fit,
},
_ => fit,
};
let cap = max_periods_from(rows.len(), fit, slots).ok_or_else(|| {
EngineError::Other("The forecast start is too early for this much history: choose a later forecast start or less history.".into())
})?;
let periods = match end {
Some(end) => steps_to(&timeline, end, cap)?,
None => match number(p, "periods")? {
Some(n) if n >= 1.0 && n.ceil() <= f64::from(cap) => n.ceil() as u32,
Some(_) => return Err(bad(CMD, format!("`periods` is from 1 to {cap} for {} rows of history", rows.len()))),
None => default_periods(rows.len()),
None => default_periods(rows.len()).min(cap),
},
};
let future: Vec<f64> =
@@ -532,6 +557,51 @@ mod tests {
assert_eq!(s.doc().unwrap().wb.sheets.len(), sheets - 1);
}
#[test]
fn hindcast_rows_count_against_the_work_limit() {
use super::{MAX_WORK, max_periods, max_periods_from};
assert_eq!(max_periods_from(1_300, 1_300, 1_300), Some(max_periods(1_300)));
// 2,000 rows fit to the first 1,899: 101 hindcast rows, then 162 forecast rows.
assert_eq!(max_periods_from(2_000, 1_899, 1_899), Some(162));
// Fit to the first 1,000: the 1,000 hindcast rows alone are twice the limit.
assert_eq!(max_periods_from(2_000, 1_000, 1_000), None);
// 800 rows fit to the first 400, which span 571 slots (gaps filled): each formula refits
// 571 slots, so 475 forecast rows after the 400 hindcast rows, not 850.
assert_eq!(max_periods_from(800, 400, 571), Some(475));
assert!((475 + 400) * 571 <= MAX_WORK as usize);
for (h, f) in [(2_000usize, 1_899usize), (10_000, 9_990), (100, 2), (60, 50)] {
if let Some(p) = max_periods_from(h, f, f) {
assert!((u64::from(p) + (h - f) as u64) * f as u64 <= MAX_WORK, "{h} {f}");
}
}
let mut s = Session::new();
s.new_workbook();
let mut rows = vec![json!(["t", "y"])];
rows.extend((1..=2_000).map(|t| json!([t, (t % 7) as f64 + t as f64 * 0.5])));
s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap();
let sheets = s.doc().unwrap().wb.sheets.len();
let err = s.execute("data.forecastSheet", json!({"range": "A1:B2001", "forecastStart": 1_001})).expect_err("hindcast too long");
assert!(err.to_string().contains("forecast start is too early"), "{err}");
let err = s.execute("data.forecastSheet", json!({"range": "A1:B2001", "forecastStart": 1_900, "periods": 163})).expect_err("past the cap");
assert!(err.to_string().contains("from 1 to 162"), "{err}");
assert_eq!(s.doc().unwrap().wb.sheets.len(), sheets);
}
#[test]
fn the_hindcast_cap_follows_the_slots_the_fit_spans_not_its_rows() {
// 800 rows, 20% of the slots missing: the first 400 rows span 500 slots, so each ETS
// formula refits 500 slots and the cap is 1,000 - 400 = 600 (it was 850 by rows).
let mut s = Session::new();
s.new_workbook();
let times: Vec<u32> = (1u32..).filter(|t| !matches!(t % 10, 3 | 6)).take(800).collect();
let mut rows = vec![json!(["t", "y"])];
rows.extend(times.iter().map(|t| json!([t, (t % 7) as f64 + f64::from(*t) * 0.5])));
s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap();
let start = times[400];
let err = s.execute("data.forecastSheet", json!({"range": "A1:B801", "forecastStart": start, "periods": 601})).expect_err("past the cap");
assert!(err.to_string().contains("from 1 to 600"), "{err}");
}
#[test]
fn history_and_periods_are_capped() {
use super::{MAX_HISTORY, MAX_PERIODS, MAX_WORK, default_periods, max_periods};
+47 -1
View File
@@ -85,7 +85,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Format Cells…",
["Home", "Cells", "Format"],
Some("Cmd+1"),
"{range?, style: {font?, fill?, borders?, align?, numFmt?, protection?}} (partial Style JSON merged in)",
"{range?, style: {font?, fill?, borders?, align?: {reading_order?: Context|LeftToRight|RightToLeft|null, ...}, numFmt?, protection?}} (partial Style JSON merged in; null clears reading_order)",
has_doc,
format_cells
),
@@ -870,6 +870,52 @@ mod tests {
assert_eq!(s.execute("cell.get", json!({"cell": "A1"})).unwrap()["style"]["num_fmt"], "0.00");
}
#[test]
fn arabic_vowel_and_direction_marks_add_no_width() {
let plain = approx_text_width("محمد يدرس في المدرسة", 11.0, false);
assert_eq!(approx_text_width("مُحَمَّد يَدْرُسُ فِي الْمَدْرَسَةِ", 11.0, false), plain);
assert_eq!(approx_text_width("\u{200F}محمد يدرس في المدرسة\u{200C}", 11.0, false), plain);
assert_eq!(approx_text_width("ب\u{0897}\u{0898}\u{089F}", 11.0, false), approx_text_width("ب", 11.0, false));
assert!(approx_text_width("\u{08E2}", 11.0, false) > 0.0, "the disputed end of ayah is a visible sign");
}
#[test]
fn arabic_text_fits_a_column_about_as_wide_as_latin_text_of_the_same_length() {
let mut s = crate::Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [["مُحَمَّد يَدْرُسُ فِي الْمَدْرَسَةِ", "Hello there in school"]]})).unwrap();
s.execute("home.autofitColumnWidth", json!({"cols": "A:B"})).unwrap();
let doc = s.doc().unwrap();
let width = |c: u32| doc.wb.active().and_then(|sh| sh.cols.get(&c)).and_then(|e| e.size).unwrap();
let ratio = width(0) / width(1);
assert!((0.75..=1.15).contains(&ratio), "Arabic {} vs Latin {}", width(0), width(1));
}
#[test]
fn arabic_word_ligatures_are_wider_than_letters() {
let em = |t: &str| approx_text_width(t, 72.0, false) / 96.0;
assert!(em("\u{FDFD}") > 8.0);
assert!(em("\u{FDFA}") > 1.0);
let honorifics = ['\u{FD40}', '\u{FD4B}', '\u{FD4C}', '\u{FD4F}', '\u{FDC8}', '\u{FDCF}', '\u{FDFE}', '\u{FDFF}'];
for honorific in honorifics.into_iter().chain(['\u{FBC3}', '\u{FBD2}', '\u{FD90}', '\u{FD91}']) {
assert!(em(&honorific.to_string()) > 1.0, "U+{:04X}", u32::from(honorific));
}
assert!(em("ب") < 0.6);
}
#[test]
fn arabic_digits_are_as_wide_as_ascii_digits_and_punctuation_as_a_letter() {
let width = |t: &str| approx_text_width(t, 11.0, false);
assert_eq!(width("١٢٣"), width("123"));
assert_eq!(width("۴۵۶"), width("456"));
assert_eq!(width("،؟"), width("بب"));
}
#[test]
fn cjk_text_keeps_a_full_em_per_character() {
assert_eq!(approx_text_width("中文", 12.0, false), 2.0 * 12.0 * 96.0 / 72.0);
}
#[test]
fn width_units() {
assert_eq!(chars_to_points(8.43), 64.0);
+44 -1
View File
@@ -136,12 +136,55 @@ pub fn approx_text_width(text: &str, size: f32, bold: bool) -> f32 {
c if c.is_ascii_uppercase() => 0.65,
c if c.is_ascii_digit() => 0.56,
c if c.is_ascii() => 0.52,
_ => 1.0,
c if is_zero_width(c) => 0.0,
c => arabic_em(c).unwrap_or(1.0),
};
}
w * size * if bold { 1.07 } else { 1.0 } * 96.0 / 72.0
}
/// Arabic vowel and Quranic marks, which sit on the previous letter, and invisible format
/// characters such as direction marks and zero-width joiners.
fn is_zero_width(c: char) -> bool {
matches!(
c,
'\u{0610}'..='\u{061A}'
| '\u{061C}'
| '\u{064B}'..='\u{065F}'
| '\u{0670}'
| '\u{06D6}'..='\u{06DC}'
| '\u{06DF}'..='\u{06E4}'
| '\u{06E7}'..='\u{06E8}'
| '\u{06EA}'..='\u{06ED}'
| '\u{0897}'..='\u{089F}'
| '\u{08CA}'..='\u{08E1}'
| '\u{08E3}'..='\u{08FF}'
| '\u{200B}'..='\u{200F}'
| '\u{202A}'..='\u{202E}'
| '\u{2060}'..='\u{2069}'
| '\u{FE20}'..='\u{FE2F}'
| '\u{FEFF}'
)
}
/// Width in ems of an Arabic-script character, measured from shaped Noto Sans Arabic, Noto Naskh
/// Arabic and DejaVu Sans: joined letters average about half an em, word ligatures such as ﷺ and the
/// rial sign about 1.2 em, the honorific ligatures (U+FD40–FD4B) about 1.6 em, and the bismillah
/// ligature about 11 em. The later honorific ligatures (U+FBC3–FBD2, U+FD4C–FD4F, U+FD90–FD91,
/// U+FDC8–FDCF) are not measured and get the 1.6 em of their siblings rather than the letter width.
/// Arabic-Indic digits are as wide as ASCII digits (0.53–0.57 em in Noto Sans Arabic, Arial, Tahoma
/// and Times New Roman); Arabic punctuation (0.2–0.5 em in the same fonts) keeps the letter width.
fn arabic_em(c: char) -> Option<f32> {
match c {
'\u{FDFD}' => Some(11.0),
'\u{FBC3}'..='\u{FBD2}' | '\u{FD40}'..='\u{FD4F}' | '\u{FD90}'..='\u{FD91}' | '\u{FDC8}'..='\u{FDCF}' => Some(1.6),
'\u{FDF0}'..='\u{FDFC}' | '\u{FDFE}'..='\u{FDFF}' => Some(1.2),
'\u{0660}'..='\u{0669}' | '\u{06F0}'..='\u{06F9}' => Some(0.56),
'\u{0600}'..='\u{06FF}' | '\u{0750}'..='\u{077F}' | '\u{0870}'..='\u{08FF}' | '\u{FB50}'..='\u{FDFF}' | '\u{FE70}'..='\u{FEFC}' => Some(0.48),
_ => None,
}
}
/// Distinct texts AutoFit measures exactly; past this many, `Session::text_width` is asked for a
/// quick measure so a huge column doesn't lay out every value.
const EXACT_TEXT_WIDTHS: usize = 2_000;
+35
View File
@@ -209,6 +209,41 @@ fn show_values_as() {
assert_eq!(text(&s, "B3"), "Biggest sale");
}
/// Running Total over nested rows adds up each child's value across the top-level items, in the
/// order they appear: the same value as summing the plain values of the earlier items' rows.
#[test]
fn running_total_over_nested_rows_adds_up_across_the_top_level_items() {
let mut s = sales();
s.execute("insert.pivotTable", json!({"rows": ["Region", "Product"], "values": ["Sales"]})).unwrap();
let r = pt(&s).last_range.unwrap();
let rows: Vec<u32> = (r.start.row + 1..r.end.row).collect();
let read = |s: &Session| -> Vec<(String, f64)> {
rows.iter().map(|row| (text(s, &CellRef::new(*row, 0).a1()), val(s, &CellRef::new(*row, 1).a1()).as_f64().unwrap_or(0.0))).collect()
};
let before = read(&s);
s.execute("pivot.valueSettings", json!({"index": 0, "showAs": "runningTotal"})).unwrap();
let after = read(&s);
// Region rows carry the report's top-level items (not indented); the others are products.
let indent = |s: &Session, row: u32| {
let d = s.doc().unwrap();
let sh = d.wb.active().unwrap();
d.wb.styles.get(sh.style_id(CellRef::new(row, 0))).align.indent
};
let (mut regions, mut products) = (0.0, std::collections::HashMap::<String, f64>::new());
for (i, row) in rows.iter().enumerate() {
let name = &before[i].0;
let total = if indent(&s, *row) == 0 {
regions += before[i].1;
regions
} else {
let t = products.entry(name.clone()).or_insert(0.0);
*t += before[i].1;
*t
};
assert!((after[i].1 - total).abs() < 1e-9, "row {name}: {} vs {total}", after[i].1);
}
}
#[test]
fn compact_vs_tabular_shapes() {
let mut s = sales();
+128 -9
View File
@@ -228,6 +228,82 @@ fn draw_border(pg: &mut Page, b: &BorderLine, wb: &Workbook, start: Point, end:
pg.stroke_line_dashed(Point::new(x0, y0), Point::new(x1, y1), lw, color, &border_dash(b.style, k));
}
/// Merged areas active at one row of a downward sweep, kept disjoint: by first column, their last
/// column and last row. Adding, dropping and looking up a merge costs log(active merges).
#[derive(Default)]
struct MergeSweep {
cols: std::collections::BTreeMap<u32, (u32, u32)>,
/// (last row, first column) of the active merges, soonest ending first.
ends: std::collections::BinaryHeap<std::cmp::Reverse<(u32, u32)>>,
}
impl MergeSweep {
/// Moves down to `row` (rows never decrease), dropping the merges that end above it.
fn advance(&mut self, row: u32) {
while let Some(std::cmp::Reverse((end, col))) = self.ends.peek().copied() {
if end >= row {
break;
}
self.ends.pop();
self.cols.remove(&col);
}
}
/// Does an active merge meet columns `c0..=c1`? The intervals are disjoint, so only the last
/// one starting at or before `c1` can.
fn meets(&self, c0: u32, c1: u32) -> bool {
self.cols.range(..=c1).next_back().is_some_and(|(_, (end, _))| *end >= c0)
}
/// Adds merge `m`, at row `row`, unless it overlaps an active one.
fn add(&mut self, m: RangeRef, row: u32) -> bool {
self.advance(row);
if self.meets(m.start.col, m.end.col) {
return false;
}
self.cols.insert(m.start.col, (m.end.col, m.end.row));
self.ends.push(std::cmp::Reverse((m.end.row, m.start.col)));
true
}
}
/// The merged areas that meet `block`, ordered by the row where they enter it, then by column,
/// leaving out any that overlaps one kept before it (only a damaged file has overlapping merges;
/// Excel drops them when it repairs one). One sweep down the rows instead of a scan of every
/// merge per cell.
fn block_merges(sh: &Sheet, block: RangeRef) -> Vec<RangeRef> {
let top = |m: &RangeRef| m.start.row.max(block.start.row);
let mut merges: Vec<RangeRef> = sh.merges.iter().filter(|m| m.intersects(&block)).copied().collect();
merges.sort_by_key(|m| (top(m), m.start.col));
let mut sweep = MergeSweep::default();
merges.retain(|m| sweep.add(*m, top(m)));
merges
}
/// Answers "is this cell inside a merge?" for cells asked in row-major order, from the merges of
/// [`block_merges`].
struct Covered<'a> {
merges: &'a [RangeRef],
top: u32,
next: usize,
sweep: MergeSweep,
}
impl<'a> Covered<'a> {
fn new(merges: &'a [RangeRef], block: RangeRef) -> Self {
Self { merges, top: block.start.row, next: 0, sweep: MergeSweep::default() }
}
fn covers(&mut self, c: CellRef) -> bool {
while let Some(m) = self.merges.get(self.next).filter(|m| m.start.row.max(self.top) <= c.row) {
self.sweep.add(*m, m.start.row.max(self.top));
self.next += 1;
}
self.sweep.advance(c.row);
self.sweep.meets(c.col, c.col)
}
}
/// Draws the cells of a block: fills, gridlines, data bars, borders and text.
pub(super) fn draw_block(pg: &mut Page, b: &Block, gridlines: bool, cf: &mut Option<crate::cf::CfCache>) {
let (bx, by) = (b.xs.first().copied().unwrap_or(0.0), b.ys.first().copied().unwrap_or(0.0));
@@ -237,17 +313,18 @@ pub(super) fn draw_block(pg: &mut Page, b: &Block, gridlines: bool, cf: &mut Opt
}
let block = b.range();
let sh = b.sh;
let merges: Vec<RangeRef> = sh.merges.iter().filter(|m| m.intersects(&block)).copied().collect();
let covered = |c: CellRef| merges.iter().any(|m| m.contains(c));
// Cells worth visiting.
let merges = block_merges(sh, block);
let by_start: std::collections::HashMap<CellRef, RangeRef> = merges.iter().map(|m| (m.start, *m)).collect();
// Cells worth visiting, in row-major order (which `Covered` needs).
let mut covered = Covered::new(&merges, block);
let cells: Vec<CellRef> = if block.count() <= DENSE_BLOCK {
block.iter().filter(|c| !covered(*c)).collect()
block.iter().filter(|c| !covered.covers(*c)).collect()
} else {
let mut set: Vec<CellRef> = sh.cells.iter_range(block).map(|(c, _)| c).collect();
set.extend(sh.spill.range(block.start..=block.end).map(|(c, _)| *c).filter(|c| block.contains(*c)));
set.sort_unstable();
set.dedup();
set.retain(|c| !covered(*c));
set.retain(|c| !covered.covers(*c));
set
};
pg.save();
@@ -277,14 +354,14 @@ pub(super) fn draw_block(pg: &mut Page, b: &Block, gridlines: bool, cf: &mut Opt
.filter(|(c, l)| {
l.fill.is_some()
|| l.cf.is_some()
|| merges.iter().any(|m| m.start == *c)
|| by_start.contains_key(c)
|| !sh.value(*c).is_empty()
|| l.style.borders != Borders::default()
|| l.style.num_fmt.as_str() == "checkbox"
})
.collect();
let rect_of = |c: CellRef| -> (f32, f32, f32, f32, bool) {
match merges.iter().find(|m| m.start == c) {
match by_start.get(&c) {
Some(m) => {
let (x, y, w, h) = b.rect(*m);
(x, y, w, h, true)
@@ -322,7 +399,7 @@ pub(super) fn draw_block(pg: &mut Page, b: &Block, gridlines: bool, cf: &mut Opt
continue;
}
let bd = &l.style.borders;
let (right, bottom) = match merges.iter().find(|m| m.start == *c).filter(|_| merged) {
let (right, bottom) = match by_start.get(c).filter(|_| merged) {
Some(m) => (
b.wb.styles.get(sh.style_id(CellRef::new(m.start.row, m.end.col))).borders.right,
b.wb.styles.get(sh.style_id(CellRef::new(m.end.row, m.start.col))).borders.bottom,
@@ -407,11 +484,12 @@ pub(super) struct PrintedBlock {
pub(super) fn draw_block_links(pg: &mut Page, b: &Block, page: usize, pending: &mut Vec<PendingPdfLink>) {
let range = b.range();
let (Some(&bx), Some(&by), Some(&right), Some(&bottom)) = (b.xs.first(), b.ys.first(), b.xs.last(), b.ys.last()) else { return };
let by_start: std::collections::HashMap<CellRef, RangeRef> = block_merges(b.sh, range).into_iter().map(|m| (m.start, m)).collect();
for (cell, link) in b.sh.hyperlinks.range(range.start..=range.end) {
if !range.contains(*cell) {
continue;
}
let cell_range = b.sh.merges.iter().find(|m| m.start == *cell).copied().unwrap_or(RangeRef::cell(*cell));
let cell_range = by_start.get(cell).copied().unwrap_or(RangeRef::cell(*cell));
let (x, y, w, h) = b.rect(cell_range);
let (x0, y0, x1, y1) = (x.max(bx), y.max(by), (x + w).min(right), (y + h).min(bottom));
if x1 <= x0 || y1 <= y0 {
@@ -503,3 +581,44 @@ pub(super) fn make_block<'a>(
}
Block { wb, si, sh, rows, cols, xs, ys, k, scale: lay.scale }
}
#[cfg(test)]
mod merge_tests {
use super::{CellRef, Covered, RangeRef, Sheet, block_merges};
fn r(a: &str) -> RangeRef {
RangeRef::parse(a).unwrap()
}
#[test]
fn block_merges_drop_overlaps_and_cover_like_a_scan() {
let mut sh = Sheet::new("S");
sh.merges = vec![r("B2:C3"), r("E1:F8"), r("B2:B5"), r("A10:D10"), r("H3:H4"), r("Z1:Z2"), r("C3:D4"), r("A1:A1048576")];
let block = r("B1:H12");
let merges = block_merges(&sh, block);
// B2:B5 and C3:D4 overlap B2:C3, which comes first; Z1:Z2 is outside the block.
assert_eq!(merges, vec![r("E1:F8"), r("B2:C3"), r("H3:H4"), r("A10:D10")]);
let mut covered = Covered::new(&merges, block);
for c in block.iter() {
assert_eq!(covered.covers(c), merges.iter().any(|m| m.contains(c)), "{}", c.a1());
}
// Cells asked sparsely, still in order.
let mut covered = Covered::new(&merges, block);
for (a, want) in [("C1", false), ("F5", true), ("G5", false), ("D10", true), ("H12", false)] {
assert_eq!(covered.covers(CellRef::parse(a).unwrap()), want, "{a}");
}
}
#[test]
fn many_stacked_merges_are_swept_once() {
// 50,000 merges over the same cells: one is kept, and the block is checked against it
// without comparing every cell with every merge.
let mut sh = Sheet::new("S");
sh.merges = (0..50_000u32).map(|i| RangeRef::new(CellRef::new(0, 0), CellRef::new(199, 1 + i % 50))).collect();
let block = r("A1:AZ200");
let merges = block_merges(&sh, block);
assert_eq!(merges, vec![r("A1:B200")]);
let mut covered = Covered::new(&merges, block);
assert_eq!(block.iter().filter(|c| covered.covers(*c)).count(), 400);
}
}
+21 -1
View File
@@ -163,6 +163,8 @@ fn protect_sheet(s: &mut Session, p: &Json) -> Result<Json> {
let what = d.wb.sheet(sheet).map(|sh| sh.name.clone()).unwrap_or_default();
let prot = SheetProtection { password: str_param(p, "password").filter(|x| !x.is_empty()).map(|pw| new_password_hash(d, &what, pw)), ..prot };
edit(s, |cx| {
// FORMULATEXT reads a sheet's protection: its results are recalculated.
cx.recalc_all = true;
cx.sheet_mut(sheet)?.protection = Some(prot.clone());
Ok(Json::Null)
})
@@ -182,7 +184,10 @@ fn unprotect_sheet(s: &mut Session, p: &Json) -> Result<Json> {
}
}
edit(s, |cx| {
cx.sheet_mut(sheet)?.protection = None;
// Only removing protection changes what FORMULATEXT reads; a no-op unprotect skips the full recalculation.
if cx.sheet_mut(sheet)?.protection.take().is_some() {
cx.recalc_all = true;
}
Ok(Json::Null)
})
}
@@ -404,6 +409,21 @@ mod tests {
assert_eq!(at(&s), (0, "A1".to_string()));
}
#[test]
fn protecting_and_unprotecting_leave_table_headers_alone() {
let mut s = Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [["a", "b"], [1, 2]]})).unwrap();
s.execute("insert.table", json!({"range": "A1:B2", "header": true})).unwrap();
// An imported header can hold a number: only a structural edit would turn it back into text.
let d = s.doc_mut().unwrap();
std::sync::Arc::make_mut(&mut d.wb).sheet_mut(0).unwrap().set_value(gridcraft_core::CellRef::new(0, 0), gridcraft_core::Value::Number(7.0));
s.execute("review.protectSheet", json!({})).unwrap();
s.execute("review.unprotectSheet", json!({})).unwrap();
let v = s.doc().unwrap().wb.sheet(0).unwrap().value(gridcraft_core::CellRef::new(0, 0));
assert_eq!(v, gridcraft_core::Value::Number(7.0));
}
#[test]
fn protection_that_forbids_selecting_locked_cells_is_enforced_and_kept() {
let mut s = book();
+21 -2
View File
@@ -52,14 +52,19 @@ pub(super) fn abs_ref(c: CellRef) -> String {
format!("${}${}", gridcraft_core::col_to_letters(c.col), u64::from(c.row) + 1)
}
/// How many cells to the left and above a cell its name is searched for.
const NAME_REACH: u32 = 1_000;
/// A cell's name in the reports: the nearest text to its left and the nearest text above it.
pub(super) fn cell_name(sh: &Sheet, c: CellRef) -> String {
let text = |c: CellRef| match sh.value(c) {
Value::Text(t) if !t.trim().is_empty() => Some(t.to_string()),
_ => None,
};
let left = (0..c.col).rev().find_map(|col| text(CellRef::new(c.row, col)));
let top = (0..c.row).rev().find_map(|row| text(CellRef::new(row, c.col)));
// Labels are next to their cells: looking further than this on a sparse sheet only costs time
// (a thousand cells near the last row would each probe a million empty coordinates).
let left = (c.col.saturating_sub(NAME_REACH)..c.col).rev().find_map(|col| text(CellRef::new(c.row, col)));
let top = (c.row.saturating_sub(NAME_REACH)..c.row).rev().find_map(|row| text(CellRef::new(row, c.col)));
[left, top].into_iter().flatten().collect::<Vec<_>>().join(" ")
}
@@ -438,3 +443,17 @@ pub(super) fn report(s: &mut Session, p: &Json) -> Result<Json> {
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_name_is_searched_only_a_bounded_distance() {
let mut sh = Sheet::new("S".to_string());
sh.set_cell(CellRef::new(0, 0), gridcraft_model::Cell::value(Value::text("Far")));
sh.set_cell(CellRef::new(5, 4), gridcraft_model::Cell::value(Value::text("Near")));
assert_eq!(cell_name(&sh, CellRef::new(5, 5)), "Near");
assert_eq!(cell_name(&sh, CellRef::new(0, NAME_REACH + 1)), "");
}
}
+31 -10
View File
@@ -61,6 +61,24 @@ enum Key {
const MAX_CONSOLIDATE_CELLS: u64 = 5_000_000;
/// The cells a consolidated table writes: every row key meets every column key, plus a label for
/// each key on the sides that have labels.
fn output_cells(rows: usize, cols: usize, top: bool, left: bool) -> u64 {
let (r, c) = (rows as u64, cols as u64);
r.saturating_mul(c).saturating_add(if top { c } else { 0 }).saturating_add(if left { r } else { 0 })
}
#[cfg(test)]
#[test]
fn the_output_limit_counts_label_cells() {
// A million row keys and four column keys are 4 million cells, 5 million with the row labels.
assert_eq!(output_cells(1_000_000, 4, false, false), 4_000_000);
assert_eq!(output_cells(1_000_000, 4, false, true), 5_000_000);
assert!(output_cells(1_000_000, 5, false, true) > MAX_CONSOLIDATE_CELLS);
assert_eq!(output_cells(3, 2, true, true), 6 + 2 + 3);
assert_eq!(output_cells(usize::MAX, usize::MAX, true, true), u64::MAX);
}
pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
const CMD: &str = "data.consolidate";
let func = str_param(p, "function").unwrap_or("sum").to_string();
@@ -89,20 +107,20 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
}
None => (d.wb.active_sheet(), d.selection.active),
};
// Keys in first-seen order, each with its index: a lookup is one probe, not a scan.
let mut row_keys: Vec<Key> = Vec::new();
let mut col_keys: Vec<Key> = Vec::new();
let mut row_index: HashMap<Key, usize> = HashMap::new();
let mut col_index: HashMap<Key, usize> = HashMap::new();
let mut row_label: HashMap<Key, String> = HashMap::new();
let mut col_label: HashMap<Key, String> = HashMap::new();
let mut aggs: HashMap<(usize, usize), Agg> = HashMap::new();
let mut total: u64 = 0;
let index_of = |keys: &mut Vec<Key>, k: Key| -> usize {
match keys.iter().position(|x| *x == k) {
Some(i) => i,
None => {
keys.push(k);
keys.len() - 1
}
}
let index_of = |keys: &mut Vec<Key>, index: &mut HashMap<Key, usize>, k: Key| -> usize {
*index.entry(k).or_insert_with_key(|k| {
keys.push(k.clone());
keys.len() - 1
})
};
for t in &srcs {
let (si, r) = parse_range(&d.wb, t, CMD)?;
@@ -131,7 +149,7 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
} else {
Key::Pos(col - c0)
};
cols.push((col, index_of(&mut col_keys, k)));
cols.push((col, index_of(&mut col_keys, &mut col_index, k)));
}
for row in r0..=r.end.row {
let k = if left {
@@ -145,7 +163,7 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
} else {
Key::Pos(row - r0)
};
let ri = index_of(&mut row_keys, k);
let ri = index_of(&mut row_keys, &mut row_index, k);
for (col, ci) in &cols {
aggs.entry((ri, *ci)).or_default().add(&sh.value(CellRef::new(row, *col)));
}
@@ -157,6 +175,9 @@ pub(super) fn consolidate(s: &mut Session, p: &Json) -> Result<Json> {
{
return Err(EngineError::Other("The consolidated data doesn't fit below and right of the destination.".into()));
}
if output_cells(row_keys.len(), col_keys.len(), top, left) > MAX_CONSOLIDATE_CELLS {
return Err(bad(CMD, "the consolidated table would be too large"));
}
let mut cells: Vec<(CellRef, Value)> = Vec::new();
if top {
for (ci, k) in col_keys.iter().enumerate() {
+11 -1
View File
@@ -189,6 +189,9 @@ pub(super) fn scenario_delete(s: &mut Session, p: &Json) -> Result<Json> {
Ok(json!({"deleted": removed.name}))
}
/// Most changing-cell-by-scenario cells one Scenario Summary writes.
const MAX_SUMMARY_CELLS: usize = 1_000_000;
pub(super) fn scenario_summary(s: &mut Session, p: &Json) -> Result<Json> {
const CMD: &str = "data.scenarioSummary";
let d = s.doc()?;
@@ -211,17 +214,24 @@ pub(super) fn scenario_summary(s: &mut Session, p: &Json) -> Result<Json> {
// Changing cells of every scenario, first-seen order; per scenario its values.
let mut changing: Vec<CellRef> = Vec::new();
let mut per: Vec<HashMap<CellRef, Value>> = Vec::new();
let mut seen: std::collections::HashSet<CellRef> = Default::default();
for sc in &scenarios {
let (_, cells) = cell_list(&d.wb, &sc.changing, CMD, MAX_SCENARIO_CELLS)?;
let mut m = HashMap::new();
for (c, v) in cells.iter().zip(sc.values.iter()) {
if !changing.contains(c) {
if seen.insert(*c) {
changing.push(*c);
}
m.insert(*c, json_value(v, &d.wb));
}
per.push(m);
}
// The summary has a cell for every changing cell in every scenario, filled from the current
// value where a scenario doesn't set it, so small disjoint scenarios multiply.
// Each result cell is a row of the summary too, with a cell per scenario.
if changing.len().saturating_add(results.len()).saturating_mul(per.len()) > MAX_SUMMARY_CELLS {
return Err(bad(CMD, "the scenarios together change too many cells for a summary"));
}
let current: Vec<Value> = changing.iter().map(|c| sh.value(*c)).collect();
let current_results: Vec<Value> = results.iter().map(|c| sh.value(*c)).collect();
let mut trial = Trial::new(d);
+35
View File
@@ -297,3 +297,38 @@ fn consolidate_by_position_and_label() {
assert!(s.execute("data.consolidate", json!({"sources": []})).is_err());
assert!(s.execute("data.consolidate", json!({"sources": ["A1"], "function": "median"})).is_err());
}
/// A hundred thousand positional rows take one key lookup each; a tall source and a wide one
/// whose keys would meet in a table of 160 million cells are refused before it is built.
#[test]
fn consolidate_keys_are_indexed_and_the_output_is_bounded() {
let mut s = s();
s.execute("range.setValues", json!({"range": "A1", "values": [[1], [2]]})).unwrap();
let r = s.execute("data.consolidate", json!({"sources": ["A1:A100000"], "destination": "C1"})).unwrap();
assert_eq!(r["rows"], 100_000);
assert_eq!(n(&s, "Sheet1!C2"), 2.0);
let r = s.execute("data.consolidate", json!({"sources": ["A1:A10000", "A1:XFC1"], "destination": "A1"}));
assert!(r.unwrap_err().to_string().contains("too large"));
}
#[test]
fn scenario_summary_refuses_a_matrix_over_its_cell_limit() {
let mut s = s();
// 200 scenarios of 32 disjoint cells: 6,400 changing cells x 200 scenarios = 1.28 million cells.
for i in 0..200 {
let (a, b) = (i * 32 + 1, i * 32 + 32);
s.execute("data.scenarioAdd", json!({"name": format!("S{i}"), "changing": format!("A{a}:A{b}")})).unwrap();
}
assert!(s.execute("data.scenarioSummary", json!({})).is_err());
}
#[test]
fn scenario_summary_counts_result_rows_in_its_cell_limit() {
let mut s = s();
// 900 scenarios of one distinct cell each are 810,000 cells; 255 result rows take it past the limit.
for i in 0..900 {
s.execute("data.scenarioAdd", json!({"name": format!("S{i}"), "changing": format!("A{}", i + 1)})).unwrap();
}
let err = s.execute("data.scenarioSummary", json!({"resultCells": "C1:C255"})).unwrap_err().to_string();
assert!(err.contains("too many cells"), "{err}");
}
+39 -1
View File
@@ -8,8 +8,13 @@ use gridcraft_model::{Sheet, Workbook};
use gridcraft_numfmt::{Formatted, NumberFormat, format_value};
/// Parsed number formats are cached by code.
fn parsed(code: &str) -> NumberFormat {
pub(crate) fn parsed(code: &str) -> NumberFormat {
static CACHE: std::sync::LazyLock<Mutex<HashMap<String, NumberFormat>>> = std::sync::LazyLock::new(|| Mutex::new(HashMap::new()));
// A code of more bytes than 4 × the character limit can't be within it: it shows as General
// without being hashed or scanned again for every cell that shares it.
if code.len() > 4 * gridcraft_numfmt::MAX_FORMAT_CODE_CHARS {
return NumberFormat::parse("General");
}
let lock = || CACHE.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(f) = lock().get(code) {
return f.clone();
@@ -24,7 +29,12 @@ fn parsed(code: &str) -> NumberFormat {
f
}
/// The parsed format of `code`; one longer than the character limit is a General format that keeps
/// the code it was made from. The engine itself formats through `parsed`, which doesn't copy it.
pub fn number_format(code: &str) -> NumberFormat {
if code.len() > 4 * gridcraft_numfmt::MAX_FORMAT_CODE_CHARS {
return NumberFormat::parse(code);
}
parsed(code)
}
@@ -228,3 +238,31 @@ pub fn stats(sheet: &Sheet, ranges: &[RangeRef]) -> Stats {
}
s
}
#[cfg(test)]
mod tests {
use super::*;
/// A format code far past Excel's 255 characters shows its values as General, for every cell
/// that shares it, without being hashed into the cache each time.
#[test]
fn an_overlong_code_formats_as_general() {
let wb = Workbook::new();
let code = format!("0.00{}", "\"x\"".repeat(400_000));
for _ in 0..1_000 {
assert_eq!(format(&Value::Number(1.5), &code, &wb).text, "1.5");
}
assert_eq!(format(&Value::Number(1.5), "0.00", &wb).text, "1.50");
}
/// The public `number_format` keeps the code of an overlong format; it is General, but not the
/// same format as a real `General`.
#[test]
fn number_format_keeps_an_overlong_code() {
let code = format!("0.00{}", "\"x\"".repeat(2_000));
let f = number_format(&code);
assert!(f.is_general());
assert_eq!(f.code(), code);
assert_ne!(f, number_format("General"));
}
}
+66 -18
View File
@@ -9,6 +9,13 @@ use crate::{EngineError, Result};
mod sylk;
/// The function library's answers to what the XLSX and XLS readers and the XLSX writer need to know about functions
/// (which names are built in, which parameters take a single value).
pub const XLSX_FUNCTIONS: gridcraft_xlsx::FunctionInfo = gridcraft_xlsx::FunctionInfo {
is_builtin: |name| gridcraft_functions::lookup(name).is_some(),
is_scalar_arg: |name, i| gridcraft_functions::lookup(name).is_none_or(|s| s.is_scalar_arg(i)),
};
/// File formats GridCraft reads or writes.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum FileKind {
@@ -93,14 +100,18 @@ pub fn open_bytes_with(name: &str, bytes: &[u8], password: Option<&str>, csv_as_
/// line-height estimate can differ from Excel's, and shrinking files that already look
/// right would be a visible regression. Custom-height rows are left alone entirely.
fn fit_imported_rows(wb: &mut Workbook) {
// The whole pass shares one budget of merge comparisons: later sheets keep their heights once it is spent.
let mut left = crate::cmd::MERGE_FIT_WORK;
for si in 0..wb.sheets.len() {
let Some(sh) = wb.sheet(si) else { continue };
let mut rows: Vec<u32> = sh.cells.iter().map(|(c, _)| c.row).collect();
rows.sort_unstable();
rows.dedup();
// One sweep over the sheet's merges for all of its rows, in ascending order.
let mut sweep = crate::cmd::MergeSweep::new(left);
for row in rows {
let before = wb.sheet(si).and_then(|sh| sh.rows.get(&row)).and_then(|i| i.size);
crate::cmd::auto_row_height(wb, si, row);
crate::cmd::fit_row_height(wb, si, row, &mut sweep);
let Some(before) = before else { continue };
let after = wb.sheet(si).and_then(|sh| sh.rows.get(&row)).and_then(|i| i.size);
if after.is_none_or(|a| before > a + 0.5)
@@ -109,6 +120,7 @@ fn fit_imported_rows(wb: &mut Workbook) {
sh.rows.entry(row).or_default().size = Some(before);
}
}
left = sweep.left();
}
}
@@ -264,7 +276,8 @@ fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook,
let kind = FileKind::from_path(name);
let sniffed = gridcraft_xlsx::sniff(bytes);
if sniffed == gridcraft_xlsx::Format::Xls {
let (wb, report) = gridcraft_xlsx::read_xls(bytes).map_err(|e| EngineError::Other(format!("We can't open '{name}': {e}")))?;
let (wb, report) =
gridcraft_xlsx::read_xls_with(bytes, XLSX_FUNCTIONS).map_err(|e| EngineError::Other(format!("We can't open '{name}': {e}")))?;
return Ok((wb, report.warnings));
}
if sniffed == gridcraft_xlsx::Format::LegacyBinary {
@@ -289,12 +302,13 @@ fn read_plain(name: &str, bytes: &[u8], csv_as_text: bool) -> Result<(Workbook,
// Text named `.xlsx` is read as text.
if sniffed == gridcraft_xlsx::Format::Xlsx || (kind == Some(FileKind::Xlsx) && sniffed != gridcraft_xlsx::Format::Csv) {
// Google Sheets' export wrapper is read as the formula inside when GridCraft calculates it (#2871).
let (wb, report) = gridcraft_xlsx::read_xlsx_with(bytes, &gridcraft_calc::is_known_function).map_err(|e| {
EngineError::Other(match (&e, gridcraft_xlsx::oversized_part(bytes)) {
(gridcraft_xlsx::IoError::TooLarge(_), Some((part, size))) => part_too_large_message(name, &part, size),
_ => format!("We can't open '{name}': {e}"),
})
})?;
let (wb, report) = gridcraft_xlsx::read_xlsx_with(bytes, (XLSX_FUNCTIONS, &gridcraft_calc::is_known_function as &dyn Fn(&str) -> bool))
.map_err(|e| {
EngineError::Other(match (&e, gridcraft_xlsx::oversized_part(bytes)) {
(gridcraft_xlsx::IoError::TooLarge(_), Some((part, size))) => part_too_large_message(name, &part, size),
_ => format!("We can't open '{name}': {e}"),
})
})?;
return Ok((wb, report.warnings));
}
if let Some(read) = read_html_table(name, kind, bytes, csv_as_text) {
@@ -502,7 +516,7 @@ pub fn save_bytes_as(wb: &Workbook, path: &str, kind: FileKind) -> Result<Vec<u8
None => named,
};
let book = std::path::Path::new(path).file_name().and_then(std::ffi::OsStr::to_str).unwrap_or_default();
gridcraft_xlsx::write_xlsx_named(named.as_ref().unwrap_or(wb), gridcraft_xlsx::XlsxKind::from_path(path), book)
gridcraft_xlsx::write_xlsx_named_with(named.as_ref().unwrap_or(wb), gridcraft_xlsx::XlsxKind::from_path(path), book, XLSX_FUNCTIONS)
.map_err(|e| EngineError::Other(e.to_string()))
}
FileKind::Xls => {
@@ -567,7 +581,7 @@ pub fn csv_values_bytes(wb: &Workbook) -> Result<Vec<u8>> {
let Some(sh) = wb.sheet(wb.active_sheet()) else { return Ok(Vec::new()) };
gridcraft_xlsx::write_csv(sh, wb.csv_delimiter.unwrap_or(b','), |c| match sh.value(c) {
gridcraft_core::Value::Number(n) if n.is_finite() => {
let shape = crate::display::number_format(wb.styles.get(sh.style_id(c)).num_fmt.as_str()).date_shape(n);
let shape = crate::display::parsed(wb.styles.get(sh.style_id(c)).num_fmt.as_str()).date_shape(n);
match shape.zip(gridcraft_core::date::datetime_from_serial(wb.date_system, n)) {
Some(((date, time), d)) => iso(&d, date, time),
None => n.to_string(),
@@ -860,8 +874,10 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
place(&im.anchor)
);
}
let mut svg_budget = crate::chart_svg::MAX_PAGE_SVG_BYTES;
for ch in &sh.charts {
if let Some(svg) = crate::chart_svg::chart_svg(wb, sheet, ch, &place(&ch.anchor)) {
if let Some(svg) = crate::chart_svg::chart_svg(wb, sheet, ch, &place(&ch.anchor), svg_budget)? {
svg_budget -= svg.len();
out.push_str(&svg);
}
}
@@ -879,12 +895,12 @@ thread_local! {
/// While one is alive, [`read_file`] and [`write_file`] on this thread refuse every path: a
/// stored script's steps run inside one (#2178), so no command reaches the file system from a
/// script whatever its parameters are called. Dropping it (also on unwind) lifts the refusal.
pub(crate) struct ScriptSandbox(());
pub struct ScriptSandbox(std::marker::PhantomData<*const ()>);
impl ScriptSandbox {
pub(crate) fn enter() -> Self {
pub fn enter() -> Self {
let _ = SCRIPT_SANDBOX.try_with(|c| c.set(c.get().saturating_add(1)));
ScriptSandbox(())
ScriptSandbox(std::marker::PhantomData)
}
}
@@ -912,6 +928,12 @@ const MAX_READ_BYTES: u64 = 2 * 1024 * 1024 * 1024;
pub fn read_file(path: &str) -> Result<Vec<u8>> {
use std::io::Read;
script_sandbox_check(path)?;
// Checked before opening: opening a FIFO nobody writes to blocks the calling (GUI) thread.
// shortcut: a FIFO swapped in between this check and the open still blocks; close that with
// O_NONBLOCK when the crate takes a libc dependency.
if !std::fs::metadata(path).map_err(|e| EngineError::Other(format!("{path}: {e}")))?.is_file() {
return Err(EngineError::Other(format!("{path}: only regular files can be opened")));
}
let f = std::fs::File::open(path).map_err(|e| EngineError::Other(format!("{path}: {e}")))?;
// A symlink to an endless device reports length 0 but streams forever, so metadata
// alone never bounds the read: require a real file and cap the bytes actually taken.
@@ -943,18 +965,37 @@ pub fn write_file(path: &str, bytes: &[u8]) -> Result<()> {
// link's target; and it inherits the destination's access restrictions, so saving a
// private workbook doesn't widen who can read it.
let tmp = format!("{path}.sctmp");
let prior = std::fs::metadata(path).ok().map(|m| m.permissions());
let prior_meta = std::fs::metadata(path).ok();
let prior = prior_meta.as_ref().map(std::fs::Metadata::permissions);
let mut opts = std::fs::OpenOptions::new();
opts.write(true).create_new(true);
// Over an existing file the replacement starts owner-only, whatever the umask: its group is
// still the saving user's primary one until it is changed below, and a member of that group
// must not be able to open it (and keep the descriptor) before then.
#[cfg(unix)]
if let Some(p) = &prior {
use std::os::unix::fs::{OpenOptionsExt, PermissionsExt};
opts.mode(p.mode());
if prior.is_some() {
use std::os::unix::fs::OpenOptionsExt;
opts.mode(0o600);
}
let mut f = opts.open(&tmp).map_err(|e| {
let _ = std::fs::remove_file(&tmp);
EngineError::Other(format!("{path}: {e}"))
})?;
// The replacement belongs to the saving user's primary group: restore the old file's group and
// then its mode. When the group can't be restored its access is dropped rather than handed to a
// different group; failing to read that back is a failed save.
#[cfg(unix)]
if let Some(m) = &prior_meta {
use std::os::unix::fs::{MetadataExt, PermissionsExt};
let _ = std::os::unix::fs::fchown(&f, None, Some(m.gid()));
let group_kept = f.metadata().map(|now| now.gid() == m.gid()).map_err(|e| {
let _ = std::fs::remove_file(&tmp);
EngineError::Other(format!("{path}: {e}"))
})?;
// Best effort: a file system that can't change modes keeps the owner-only one. Without the
// group, the old owner and other bits are kept and the group, setuid and setgid bits dropped.
let _ = f.set_permissions(std::fs::Permissions::from_mode(restored_mode(m.mode(), group_kept)));
}
#[cfg(not(unix))]
if let Some(p) = &prior {
let _ = f.set_permissions(p.clone());
@@ -971,6 +1012,13 @@ pub fn write_file(path: &str, bytes: &[u8]) -> Result<()> {
})
}
/// The mode a saved file gets from the file it replaces (`old`): all of it when its group was
/// restored; without the group, the owner and other bits only.
#[cfg(unix)]
fn restored_mode(old: u32, group_kept: bool) -> u32 {
if group_kept { old & 0o7777 } else { old & 0o707 }
}
#[cfg(target_arch = "wasm32")]
pub fn write_file(path: &str, _bytes: &[u8]) -> Result<()> {
script_sandbox_check(path)?;
+49
View File
@@ -51,6 +51,14 @@ fn save_keeps_the_destination_access_restrictions() {
write_file(dest.to_str().unwrap(), b"v2").unwrap();
let mode = std::fs::metadata(&dest).unwrap().permissions().mode() & 0o777;
assert_eq!(mode, 0o600);
// The mode is set outright from the old file's, not through the umask: group and other access
// it had stay as they were.
for old in [0o640, 0o664] {
std::fs::set_permissions(&dest, std::fs::Permissions::from_mode(old)).unwrap();
write_file(dest.to_str().unwrap(), b"v3").unwrap();
assert_eq!(std::fs::metadata(&dest).unwrap().permissions().mode() & 0o777, old);
}
assert!(!dir.join("private.xlsx.sctmp").exists());
// And a brand-new file is created with the restrictive, not permissive, default.
let fresh = dir.join("new.xlsx");
write_file(fresh.to_str().unwrap(), b"v").unwrap();
@@ -59,6 +67,17 @@ fn save_keeps_the_destination_access_restrictions() {
let _ = std::fs::remove_dir_all(&dir);
}
/// Without the old group, a saved file keeps the old owner and other bits and nothing for a group.
#[cfg(not(target_arch = "wasm32"))]
#[cfg(unix)]
#[test]
fn restored_mode_drops_the_group_and_special_bits_when_the_group_is_lost() {
assert_eq!(super::restored_mode(0o100_664, true), 0o664);
assert_eq!(super::restored_mode(0o100_664, false), 0o604);
assert_eq!(super::restored_mode(0o102_444, false), 0o404, "an original read-only file stays read-only, setgid is dropped");
assert_eq!(super::restored_mode(0o104_755, false), 0o705, "setuid is dropped");
}
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn open_refuses_directories_and_unknowable_lengths() {
@@ -351,6 +370,19 @@ fn script_sandbox_refuses_file_access_until_dropped() {
let _ = std::fs::remove_dir_all(&dir);
}
/// The guard's drop lowers the counter of the thread it runs on, so it must not move between threads:
/// naming the trait's method is ambiguous (a compile error) if `ScriptSandbox` is `Send`.
#[test]
fn script_sandbox_is_not_send() {
trait AmbiguousIfSend<A> {
fn check() {}
}
impl<T: ?Sized> AmbiguousIfSend<()> for T {}
struct IsSend;
impl<T: ?Sized + Send> AmbiguousIfSend<IsSend> for T {}
<ScriptSandbox as AmbiguousIfSend<_>>::check();
}
/// A minimal compound (CFB/OLE2) file — FAT in sector 0, directory in sector 1 — holding a
/// root entry and the given stream names.
fn cfb(streams: &[&str]) -> Vec<u8> {
@@ -761,3 +793,20 @@ fn sylk_survives_huge_relative_offsets_and_width_floods() {
let (wb, _) = open_bytes("widths.slk", &flood).unwrap();
assert_eq!(wb.sheet(0).unwrap().value(CellRef::parse("A1").unwrap()), gridcraft_core::Value::Number(1.0));
}
/// A FIFO with no writer would block an open forever: it is refused before it is opened.
#[cfg(unix)]
#[test]
fn reading_a_fifo_is_refused_without_blocking() {
let dir = std::env::temp_dir().join(format!("gridcraft-fifo-{}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
let fifo = dir.join("pipe.xlsx");
if !std::process::Command::new("mkfifo").arg(&fifo).status().is_ok_and(|s| s.success()) {
let _ = std::fs::remove_dir_all(&dir);
return; // no mkfifo here
}
let err = read_file(fifo.to_str().unwrap()).unwrap_err().to_string();
assert!(err.contains("only regular files"), "{err}");
let _ = std::fs::remove_dir_all(&dir);
}
+17
View File
@@ -237,6 +237,22 @@ const JOURNAL_MAX_ENTRIES: usize = 10_000;
pub(crate) const JOURNAL_MAX_BYTES: usize = 64 * 1024 * 1024;
const UNDO_BYTES_LIMIT: usize = 128 * 1024 * 1024;
/// Comment and hyperlink text of a sheet: a changed sheet's snapshot keeps its own copy of these
/// strings (an edit deep-copies them), so they count against the history budget with the cells.
fn sheet_text_bytes(sheet: &gridcraft_model::Sheet) -> usize {
let comments = sheet.comments.values().fold(0usize, |sum, c| {
// The vectors' own storage counts too: an empty reply still takes an element.
let replies = c.replies.iter().fold(0usize, |n, (a, t)| n.saturating_add(a.len()).saturating_add(t.len()));
let replies = replies.saturating_add(c.replies.capacity().saturating_mul(std::mem::size_of::<(String, String)>()));
let dates = c.dates.iter().fold(0usize, |n, d| n.saturating_add(d.len()));
let dates = dates.saturating_add(c.dates.capacity().saturating_mul(std::mem::size_of::<String>()));
sum.saturating_add(std::mem::size_of::<gridcraft_model::Comment>() + c.author.len() + c.text.len() + replies + dates)
});
sheet.hyperlinks.values().fold(comments, |sum, h| {
sum.saturating_add(std::mem::size_of::<gridcraft_model::Hyperlink>() + h.target.len() + h.tooltip.as_ref().map_or(0, String::len))
})
}
fn history_retained_bytes(older: &Workbook, newer: &Workbook) -> usize {
let mut bytes = std::mem::size_of::<Workbook>();
for (index, sheet) in older.sheets.iter().enumerate() {
@@ -248,6 +264,7 @@ fn history_retained_bytes(older: &Workbook, newer: &Workbook) -> usize {
let empty = gridcraft_model::CellStore::new();
let next = newer.sheets.get(index).map_or(&empty, |next| &next.cells);
bytes = bytes.saturating_add(sheet.cells.retained_bytes_against(next));
bytes = bytes.saturating_add(sheet_text_bytes(sheet));
}
// Snapshots share the macro list until a change to it; a snapshot with its own list keeps it.
if !older.macros.shares(&newer.macros) {
+1 -1
View File
@@ -440,7 +440,7 @@ pub fn infer_input(wb: &Workbook, sheet: usize, r: RangeRef) -> NamedInput {
Some(match v {
Value::Empty => return None,
Value::Text(t) if t.is_empty() => return None,
Value::Number(_) if crate::display::number_format(wb.styles.get(sh.style_id(c)).num_fmt.as_str()).is_date() => ColType::Date,
Value::Number(_) if crate::display::parsed(wb.styles.get(sh.style_id(c)).num_fmt.as_str()).is_date() => ColType::Date,
Value::Number(_) => ColType::Number,
Value::Text(_) => ColType::Text,
Value::Bool(_) => ColType::Bool,
+1 -1
View File
@@ -335,7 +335,7 @@ fn check_input(wb: &Workbook, input: &gridcraft_model::NamedInput, place: Place)
gridcraft_model::ColType::Number => v.is_number(),
gridcraft_model::ColType::Text => v.is_text(),
gridcraft_model::ColType::Bool => matches!(v, Value::Bool(_)),
gridcraft_model::ColType::Date => v.is_number() && crate::display::number_format(&date()).is_date(),
gridcraft_model::ColType::Date => v.is_number() && crate::display::parsed(&date()).is_date(),
};
if !ok {
wrong.push((c, holds(&v, ty)));
+1 -1
View File
@@ -182,7 +182,7 @@ pub fn read_source(wb: &Workbook, text: &str, default_sheet: usize) -> Result<So
for (n, a) in numeric.iter_mut().zip(any.iter()) {
*n = *n && *a;
}
let dates = numeric.iter().zip(formats.iter()).map(|(n, f)| *n && crate::display::number_format(f).is_date()).collect();
let dates = numeric.iter().zip(formats.iter()).map(|(n, f)| *n && crate::display::parsed(f).is_date()).collect();
Ok(Source { sheet: si, range: r, headers, formats, numeric, dates, rows })
}
+24
View File
@@ -57,6 +57,10 @@ fn rows_ref(sheet: &str, origin: CellRef, col: u32, rows: &[u32]) -> Option<Stri
/// Most category rows a PivotChart plots.
const MAX_CHART_ROWS: usize = 10_000;
/// Most series a PivotChart gets: the chart renderer reads at most 255 series, and each one owns
/// reference text as long as its category rows, copied into every linked chart.
const MAX_CHART_SERIES: usize = 255;
/// A PivotChart's series for a report written at `origin`: categories are the innermost row
/// items, one series per item column (column items × value fields); totals are left out.
pub fn chart_series(sheet: &str, origin: CellRef, out: &Output) -> Vec<Series> {
@@ -64,6 +68,7 @@ pub fn chart_series(sheet: &str, origin: CellRef, out: &Output) -> Vec<Series> {
let categories = out.label_col.and_then(|c| rows_ref(sheet, origin, c, &rows));
out.chart_cols
.iter()
.take(MAX_CHART_SERIES)
.filter_map(|(c, name)| {
Some(Series {
name: Some(format!("\"{}\"", name.replace('"', "\"\""))),
@@ -106,3 +111,22 @@ pub fn clear_output(wb: &mut Workbook, sheet: usize, idx: usize) {
p.last_range = None;
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_pivot_chart_gets_at_most_the_series_the_renderer_reads() {
let out = Output {
grid: vec![],
row_kinds: vec![],
body_offset: 0,
width: 1000,
chart_rows: vec![1, 2, 3],
chart_cols: (1..1000).map(|c| (c, format!("S{c}"))).collect(),
label_col: Some(0),
};
assert_eq!(chart_series("Sheet1", CellRef::new(0, 0), &out).len(), MAX_CHART_SERIES);
}
}
+85 -28
View File
@@ -1,5 +1,7 @@
use super::items::{Acc, FieldItems, build_tree, collect_siblings, is_collapsed, item_label};
use super::output::{AggMap, CellKind, ColKind, ColSpec, Ctx, Grid, Line, LineKind, Output, RowKind, ValSrc, data_cols_exceed_sheet, flatten_cols};
use super::output::{
AggMap, CellKind, ColKind, ColSpec, Ctx, Grid, Line, LineKind, Output, RowKind, ValSrc, caption, data_cols_exceed_sheet, flatten_cols,
};
use super::{
CalcSet, CellError, CellRef, HashMap, HashSet, MAX_AXIS_FIELDS, MAX_COLS, MAX_VALUE_FIELDS, PivotDateGroup, PivotField, PivotFilter, PivotFunc,
PivotLayout, PivotShowAs, PivotTable, PivotValue, Source, Value, Workbook, read_source,
@@ -94,6 +96,10 @@ struct AggregateData {
col_leaves: HashSet<Vec<u32>>,
}
/// Most accumulators (groups times value columns) one report may hold: twice what the 5 million
/// group limit allowed at one value field.
const MAX_ACCUMULATORS: usize = 16_000_000;
fn aggregate_record(
input: &AggregateInput<'_>,
data: &mut AggregateData,
@@ -152,7 +158,9 @@ fn aggregate_record(
if !input.cols.is_empty() && data_cols_exceed_sheet(data.col_leaves.len(), input.pt.grand_totals_rows, input.vals.len()) {
return Err("The PivotTable report will not fit on the sheet.".into());
}
if data.aggs.len() > 5_000_000 {
// Each entry holds one accumulator per value field and calculated-field source column, so the
// width counts too: a handful of groups over hundreds of fields is large as well.
if data.aggs.len() > 5_000_000 || data.aggs.len().saturating_mul(input.vals.len() + input.extra.len()) > MAX_ACCUMULATORS {
return Err("The PivotTable report is too large.".into());
}
Ok(())
@@ -259,8 +267,8 @@ fn write_headers(g: &mut Grid, input: HeaderInput<'_>) -> Result<(), String> {
let lv = spec.path.len().saturating_sub(1);
let label = spec.path.last().and_then(|it| col_items.get(lv).map(|fi| fi.label(*it))).unwrap_or_default();
let text = match vi {
Some(vi) if k > 1 => format!("{label} {}", val_name(*vi)),
_ => format!("{label} Total"),
Some(vi) if k > 1 => caption(&label, &val_name(*vi)),
_ => caption(&label, "Total"),
};
g.set(first_level + lv, c, Value::text(text), CellKind::ColLabel, 0, None);
}
@@ -318,6 +326,7 @@ struct ShowContext<'a> {
value_index: usize,
level0: &'a [u32],
siblings: &'a HashMap<Vec<u32>, Vec<u32>>,
cache: &'a ShowCache,
}
fn number(value: &Option<Value>) -> Option<f64> {
@@ -334,42 +343,89 @@ fn ratio(a: Value, b: Option<Value>) -> Value {
}
}
/// What Running Total and Rank need from other cells, computed once per sibling set instead of
/// once per displayed value (which rescanned the whole set every time).
type ShowKey = (Vec<u32>, Vec<u32>, usize);
/// Cumulative values by top-level item, and the total of all of them.
type RunningTotals = (HashMap<u32, f64>, f64);
/// Top-level items by (rest of a row path, column path).
type ItemsByRest = HashMap<(Vec<u32>, Vec<u32>), Vec<u32>>;
#[derive(Default)]
struct ShowCache {
/// Ranked values' sibling numbers, sorted ascending without repeats, by (parent, column, value).
ranks: std::cell::RefCell<HashMap<ShowKey, Vec<f64>>>,
/// Running totals by top-level item, with the total of all of them, by (rest of path, column, value).
running: std::cell::RefCell<HashMap<ShowKey, RunningTotals>>,
/// The top-level items that have a row path ending in `rest` (with column path `col`), in
/// top-level order, by (`rest`, `col`): built once from the report's cells, so a running total
/// visits the items that can hold a number, not every top-level item per distinct `rest`.
items_by_rest: std::cell::RefCell<Option<ItemsByRest>>,
}
fn running_total(raw: Value, lookup: &ValueLookup<'_>, context: &ShowContext<'_>) -> Value {
let Some((first, _)) = context.row_path.split_first() else { return raw };
let Some((first, rest)) = context.row_path.split_first() else { return raw };
if !matches!(raw, Value::Number(_)) {
return raw;
}
let mut total = 0.0;
for item in context.level0 {
let mut path = context.row_path.to_vec();
if let Some(first_item) = path.first_mut() {
*first_item = *item;
let mut index = context.cache.items_by_rest.borrow_mut();
let items_by_rest = index.get_or_insert_with(|| {
let position: HashMap<u32, usize> = context.level0.iter().enumerate().rev().map(|(at, item)| (*item, at)).collect();
let mut map = ItemsByRest::new();
for (row, col) in lookup.aggs.keys() {
if let Some((item, rest)) = row.split_first()
&& position.contains_key(item)
{
map.entry((rest.to_vec(), col.clone())).or_default().push(*item);
}
}
total += number(&lookup.get(&path, context.col_path, context.value_index)).unwrap_or(0.0);
if item == first {
break;
for items in map.values_mut() {
items.sort_by_key(|item| position.get(item).copied().unwrap_or(usize::MAX));
}
}
Value::number(total)
map
});
let mut cache = context.cache.running.borrow_mut();
let (by_item, all) = cache.entry((rest.to_vec(), context.col_path.to_vec(), context.value_index)).or_insert_with(|| {
let mut total = 0.0;
let mut by_item = HashMap::new();
for item in items_by_rest.get(&(rest.to_vec(), context.col_path.to_vec())).into_iter().flatten() {
let mut path = Vec::with_capacity(rest.len() + 1);
path.push(*item);
path.extend_from_slice(rest);
// An item with no number here adds nothing and is not stored (the shown cell always has one).
let Some(value) = number(&lookup.get(&path, context.col_path, context.value_index)) else { continue };
total += value;
by_item.entry(*item).or_insert(total);
}
(by_item, total)
});
Value::number(by_item.get(first).copied().unwrap_or(*all))
}
fn ranked_value(raw: Value, lookup: &ValueLookup<'_>, context: &ShowContext<'_>) -> Value {
let Some((_, parent)) = context.row_path.split_last() else { return Value::Empty };
let Some(me) = raw.as_f64() else { return raw };
let siblings = context.siblings.get(parent).cloned().unwrap_or_default();
let mut cache = context.cache.ranks.borrow_mut();
let sorted = cache.entry((parent.to_vec(), context.col_path.to_vec(), context.value_index)).or_insert_with(|| {
let mut all: Vec<f64> = context
.siblings
.get(parent)
.into_iter()
.flatten()
.filter_map(|sibling| {
let mut path = parent.to_vec();
path.push(*sibling);
number(&lookup.get(&path, context.col_path, context.value_index))
})
.collect();
all.sort_by(f64::total_cmp);
all.dedup();
all
});
// Excel's ranks are dense: ties share a rank and the next value takes the next one
// (5, 4, 3, 3, 4 rank 1, 2, 3, 3, 2), so count the distinct larger values.
let mut larger: Vec<f64> = siblings
.into_iter()
.filter_map(|sibling| {
let mut path = parent.to_vec();
path.push(sibling);
number(&lookup.get(&path, context.col_path, context.value_index)).filter(|value| *value > me)
})
.collect();
larger.sort_by(f64::total_cmp);
larger.dedup();
Value::number(larger.len() as f64 + 1.0)
let larger = sorted.len() - sorted.partition_point(|value| *value <= me);
Value::number(larger as f64 + 1.0)
}
fn show_value(definition: &PivotValue, raw: Value, lookup: &ValueLookup<'_>, context: &ShowContext<'_>) -> Value {
@@ -642,6 +698,7 @@ pub(super) fn compute_from(wb: &Workbook, pt: &PivotTable, src: &Source, mut pro
})
.collect();
let lookup = ValueLookup { wb, source: src, calc: &calc, vals: &vals, extra: &extra, aggs: &aggs, value_count: k };
let show_cache = ShowCache::default();
let mut chart_rows: Vec<u32> = Vec::new();
let chart_cols: Vec<(u32, String)> = data_cols
.iter()
@@ -699,7 +756,7 @@ pub(super) fn compute_from(wb: &Workbook, pt: &PivotTable, src: &Source, mut pro
let Some((_, vdef)) = vals.get(*vi) else { continue };
let cp = spec.path.as_slice();
let Some(raw) = lookup.get(rp, cp, *vi) else { continue };
let context = ShowContext { row_path: rp, col_path: cp, value_index: *vi, level0: &level0, siblings: &siblings };
let context = ShowContext { row_path: rp, col_path: cp, value_index: *vi, level0: &level0, siblings: &siblings, cache: &show_cache };
let shown = show_value(vdef, raw, &lookup, &context);
if let Some(pr) = probe.as_deref_mut() {
if pr.at == Some((r, l + j)) {
+1 -1
View File
@@ -21,7 +21,7 @@ pub(super) struct FieldItems {
}
fn cmp_items(a: &Item, b: &Item) -> Ordering {
a.class.cmp(&b.class).then(a.num.total_cmp(&b.num)).then_with(|| a.text.cmp(&b.text))
a.class.cmp(&b.class).then(a.num.total_cmp(&b.num)).then_with(|| gridcraft_core::compare_text(&a.text, &b.text))
}
/// (identity key, item) for a source value under a field's date grouping.
+29 -3
View File
@@ -121,6 +121,18 @@ pub(super) enum ColKind {
Grand,
}
/// Longest item caption Excel keeps; a subtotal caption repeats its parent's label for every value
/// field at every level, so a long source label is cut rather than multiplied.
const MAX_CAPTION_LABEL_CHARS: usize = 255;
/// `label` (cut to [`MAX_CAPTION_LABEL_CHARS`]) and `suffix`, as in "East Total".
pub(super) fn caption(label: &str, suffix: &str) -> String {
let mut out: String = label.chars().take(MAX_CAPTION_LABEL_CHARS).collect();
out.push(' ');
out.push_str(suffix);
out
}
pub(super) struct ColSpec {
pub(super) kind: ColKind,
pub(super) path: Vec<u32>,
@@ -162,7 +174,7 @@ impl Ctx<'_> {
leaf: false,
path: prefix.to_vec(),
values: true,
labels: vec![(col, Value::text(format!("{label} {name}")), indent, None)],
labels: vec![(col, Value::text(caption(label, name)), indent, None)],
vi: Some(vi),
});
}
@@ -249,7 +261,7 @@ impl Ctx<'_> {
leaf: false,
path: prefix.clone(),
values: true,
labels: vec![(col, Value::text(format!("{label} Total")), indent, None)],
labels: vec![(col, Value::text(caption(&label, "Total")), indent, None)],
vi,
});
}
@@ -316,7 +328,7 @@ impl Ctx<'_> {
leaf: false,
path: prefix.clone(),
values: true,
labels: vec![(col, Value::text(format!("{label} Total")), 0, None)],
labels: vec![(col, Value::text(caption(&label, "Total")), 0, None)],
vi,
});
}
@@ -438,3 +450,17 @@ pub(super) fn cell_style(lk: &Look, rk: RowKind, cell: &OutCell, last_header: bo
}
s
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn captions_cut_a_long_parent_label() {
assert_eq!(caption("East", "Total"), "East Total");
let long = "é".repeat(100_000);
let c = caption(&long, "Sum of Sales");
assert_eq!(c.chars().count(), MAX_CAPTION_LABEL_CHARS + " Sum of Sales".len());
assert!(c.ends_with(" Sum of Sales"));
}
}
+185
View File
@@ -18,6 +18,44 @@ fn v(s: &Session, a: &str) -> Value {
s.doc().unwrap().wb.active().unwrap().value(CellRef::parse(a).unwrap())
}
#[test]
fn format_cells_reading_order_survives_undo_and_xlsx() {
let mut s = s();
s.execute("cell.set", json!({"cell": "A1", "input": "مرحبا 123 ABC"})).unwrap();
s.execute("home.bold", json!({"range": "A1"})).unwrap();
s.execute("home.formatCells", json!({"range": "A1", "style": {"align": {"reading_order": "RightToLeft"}}})).unwrap();
let alignment = |s: &Session| {
let d = s.doc().unwrap();
let style = d.wb.styles.get(d.wb.active().unwrap().style_id(CellRef::new(0, 0)));
assert!(style.font.bold);
serde_json::to_value(style.align).unwrap()
};
assert_eq!(alignment(&s)["reading_order"], "RightToLeft");
s.execute("edit.undo", json!({})).unwrap();
assert!(alignment(&s)["reading_order"].is_null());
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(alignment(&s)["reading_order"], "RightToLeft");
for order in [json!("invalid"), json!(2)] {
assert!(s.execute("home.formatCells", json!({"range": "A1", "style": {"align": {"reading_order": order}}})).is_err());
assert_eq!(alignment(&s)["reading_order"], "RightToLeft");
}
let wb = &s.doc().unwrap().wb;
let (reopened, _) = gridcraft_xlsx::read_xlsx(&gridcraft_xlsx::write_xlsx(wb).unwrap()).unwrap();
let cell = CellRef::new(0, 0);
assert_eq!(reopened.styles.get(reopened.active().unwrap().style_id(cell)), wb.styles.get(wb.active().unwrap().style_id(cell)));
assert_eq!(reopened.active().unwrap().value(cell), Value::text("مرحبا 123 ABC"));
for order in ["LeftToRight", "Context"] {
s.execute("home.formatCells", json!({"range": "A1", "style": {"align": {"reading_order": order}}})).unwrap();
assert_eq!(alignment(&s)["reading_order"], order);
}
s.execute("home.formatCells", json!({"range": "A1", "style": {"align": {"reading_order": null}}})).unwrap();
assert!(alignment(&s)["reading_order"].is_null());
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(alignment(&s)["reading_order"], "Context");
s.execute("edit.redo", json!({})).unwrap();
assert!(alignment(&s)["reading_order"].is_null());
}
#[test]
fn entering_empty_cells_preserves_whole_sheet_formatting() {
let mut s = s();
@@ -1312,6 +1350,18 @@ fn scripts_leave_file_access_working_after_they_run() {
let _ = std::fs::remove_dir_all(&dir);
}
#[test]
fn get_data_refuses_json_with_too_many_fields_or_rows_before_building_rows() {
let mut s = s();
let wide = format!("[{}]", (0..=gridcraft_core::MAX_COLS).map(|i| format!("{{\"k{i}\":1}}")).collect::<Vec<_>>().join(","));
let err = s.execute("data.fromTextCsv", json!({"text": wide})).unwrap_err().to_string();
assert!(err.contains("more fields"), "{err}");
// MAX_ROWS objects plus the header row are one too many.
let tall = format!("[{}]", vec!["{}"; gridcraft_core::MAX_ROWS as usize].join(","));
let err = s.execute("data.fromTextCsv", json!({"text": tall})).unwrap_err().to_string();
assert!(err.contains("more rows"), "{err}");
}
#[test]
fn scripts_cannot_reach_outside_the_workbook_even_when_trusted() {
let dir = std::env::temp_dir().join(format!("gridcraft-script-deny-{}", std::process::id()));
@@ -1459,6 +1509,28 @@ fn case_sensitive_sort_puts_lowercase_first() {
assert_eq!(col, ["a", "A", "aB", "Ab", "b", "B"].map(Value::from));
}
#[test]
fn data_sort_orders_vowelled_arabic_by_letters() {
let mut s = s();
s.execute("range.setValues", json!({"range": "A1", "values": [["بيت"], ["بَاب"], ["تمر"], ["\u{200F}بحر"]]})).unwrap();
s.execute("data.sort", json!({"range": "A1:A4", "header": false, "keys": [{"column": "A"}]})).unwrap();
let col: Vec<Value> = (1..=4).map(|r| v(&s, &format!("A{r}"))).collect();
assert_eq!(col, ["بَاب", "\u{200F}بحر", "بيت", "تمر"].map(Value::from));
}
#[test]
fn data_sort_orders_marks_and_digit_scripts_like_excel() {
// Excel 16.0 (build 20527, en-US) sorts this column, as text, into exactly this order.
let want = ["١٢", "۱۲", "ب\u{064E}1", "ب\u{064E}١", "بa", "\u{200F}باب", "ب\u{064B}اب", "ب\u{064E}اب", "ب\u{0652}اب", "ب\u{0651}اب"];
let mut s = s();
s.execute("home.numberFormat", json!({"range": "A1:A10", "code": "@"})).unwrap();
let input: Vec<Vec<&str>> = want.iter().rev().map(|t| vec![*t]).collect();
s.execute("range.setValues", json!({"range": "A1", "values": input})).unwrap();
s.execute("data.sort", json!({"range": "A1:A10", "header": false, "keys": [{"column": "A"}]})).unwrap();
let col: Vec<Value> = (1..=10).map(|r| v(&s, &format!("A{r}"))).collect();
assert_eq!(col, want.map(Value::from));
}
#[test]
fn sort_with_a_sheet_prefix_sorts_the_named_sheet() {
let col = |s: &Session, sheet: &str| -> Vec<Value> {
@@ -3604,6 +3676,79 @@ fn xls_import_keeps_formulas_and_never_saves_back_to_xls() {
assert!(s.execute("file.saveBytes", json!({"format": "xlsx"})).is_ok());
}
#[test]
fn xls_formulas_calculate_with_legacy_implicit_intersection() {
// Data!A1 = 11, B1:G1 = 1, A2 = 5, A3 = 7. Report!B1:G1 are filled, so a formula in Report!A1
// that spilled would show #SPILL!. Every cached result is 0, so the values below are GridCraft's.
fn rec(id: u16, payload: &[u8]) -> Vec<u8> {
[&id.to_le_bytes()[..], &(payload.len() as u16).to_le_bytes(), payload].concat()
}
let u16s = |v: &[u16]| v.iter().flat_map(|x| x.to_le_bytes()).collect::<Vec<u8>>();
let bof = |dt: u16| rec(0x0809, &[&0x0600u16.to_le_bytes()[..], &dt.to_le_bytes(), &[0; 12]].concat());
let number = |r: u16, c: u16, n: f64| rec(0x0203, &[&u16s(&[r, c, 0])[..], &n.to_le_bytes()].concat());
let formula = |r: u16, c: u16, rgce: &[u8]| {
rec(0x0006, &[&u16s(&[r, c, 0])[..], &0f64.to_le_bytes(), &[0; 6], &(rgce.len() as u16).to_le_bytes(), rgce].concat())
};
let area3d = |r0: u16, r1: u16, c0: u16, c1: u16| [&[0x3Bu8, 0, 0][..], &u16s(&[r0, r1, c0, c1])].concat();
let a2a3 = area3d(1, 2, 0, 0);
let mut data = bof(0x0010);
data.extend([number(0, 0, 11.0), number(1, 0, 5.0), number(2, 0, 7.0)].concat());
(1..7).for_each(|c| data.extend(number(0, c, 1.0)));
data.extend(rec(0x000A, &[]));
let mut report = bof(0x0010);
(1..7).for_each(|c| report.extend(number(0, c, 0.0)));
// A1 = Data!A1:G1; B2 = Data!$A$2:$A$3*10; C2 = SQRT(Data!$A$2:$A$3) (row 2: SQRT(5));
// H1 = {SUM(Data!$A$2:$A$3*2)}, a one-cell Ctrl+Shift+Enter array.
let cse = [&a2a3[..], &[0x1E, 2, 0, 0x05, 0x22, 1, 4, 0]].concat();
report.extend(formula(0, 0, &area3d(0, 0, 0xC000, 6 | 0xC000)));
report.extend(formula(1, 1, &[&a2a3[..], &[0x1E, 10, 0, 0x05]].concat()));
report.extend(formula(1, 2, &[&a2a3[..], &[0x21, 20, 0]].concat()));
report.extend(formula(0, 7, &[1, 0, 0, 7, 0]));
report.extend(rec(0x0221, &[&u16s(&[0, 0])[..], &[7, 7], &[0; 6], &(cse.len() as u16).to_le_bytes(), &cse].concat()));
report.extend(rec(0x000A, &[]));
let boundsheet = |pos: u32, name: &str| rec(0x0085, &[&pos.to_le_bytes()[..], &[0, 0, name.len() as u8, 0], name.as_bytes()].concat());
let globals = |p1: u32, p2: u32| {
[
bof(0x0005),
boundsheet(p1, "Data"),
boundsheet(p2, "Report"),
rec(0x01AE, &u16s(&[2, 0x0401])),
rec(0x0017, &u16s(&[1, 0, 0, 0])),
rec(0x000A, &[]),
]
.concat()
};
let p1 = globals(0, 0).len() as u32;
let stream = [globals(p1, p1 + data.len() as u32), data, report].concat();
let bytes = gridcraft_xlsx::testing::compound_file(&[("Workbook", &stream)]);
let mut s = s();
s.execute("file.open", json!({"name": "legacy.xls", "base64": crate::io::base64_encode(&bytes)})).unwrap();
s.execute("formulas.calculateNow", json!({})).unwrap();
let report = |s: &Session, a1: &str| s.doc().unwrap().wb.sheets[1].value(CellRef::parse(a1).unwrap());
assert_eq!(report(&s, "A1"), Value::Number(11.0));
assert_eq!(report(&s, "B2"), Value::Number(50.0));
assert_eq!(report(&s, "C2"), Value::Number(5f64.sqrt()));
assert_eq!(report(&s, "H1"), Value::Number(24.0));
// Nothing spilled out of A1 or B2.
assert_eq!(report(&s, "B1"), Value::Number(0.0));
assert_eq!(report(&s, "B3"), Value::Empty);
// Saved as .xlsx, the formulas carry no dynamic-array metadata.
let r = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let saved = crate::io::base64_decode(r["base64"].as_str().unwrap()).unwrap();
let (wb, _) = gridcraft_xlsx::read_xlsx(&saved).unwrap();
let f = |a1: &str| wb.sheets[1].cell(CellRef::parse(a1).unwrap()).and_then(|c| c.formula.clone()).unwrap();
assert_eq!(f("A1").text, "@Data!A1:G1");
assert_eq!((f("H1").text.as_str(), f("H1").array), ("SUM(Data!$A$2:$A$3*2)", gridcraft_core::RangeRef::parse("H1")));
let mut sheet = String::new();
std::io::Read::read_to_string(
&mut zip::ZipArchive::new(std::io::Cursor::new(&saved)).unwrap().by_name("xl/worksheets/sheet2.xml").unwrap(),
&mut sheet,
)
.unwrap();
assert!(!sheet.contains("cm=") && !sheet.contains("SINGLE"), "{sheet}");
}
#[test]
fn malformed_xlsb_keeps_the_current_workbook() {
let mut s = s();
@@ -6335,6 +6480,23 @@ fn clearing_links_and_notes_keeps_allow_edit_ranges() {
assert!(sh.hyperlinks.is_empty() && sh.comments.is_empty());
}
#[test]
fn formulatext_does_not_reveal_a_hidden_formula() {
let mut s = s();
s.execute("cell.set", json!({"cell": "A1", "input": "=1+1"})).unwrap();
s.execute("home.formatCells", json!({"range": "A1", "style": {"protection": {"locked": false, "hidden": true}}})).unwrap();
s.execute("home.formatCells", json!({"range": "B1", "style": {"protection": {"locked": false}}})).unwrap();
s.execute("cell.set", json!({"cell": "B1", "input": "=FORMULATEXT(A1)"})).unwrap();
assert_eq!(v(&s, "B1"), Value::from("=1+1"), "not protected yet");
s.execute("review.protectSheet", json!({})).unwrap();
// Without entering the formula again: protecting recalculates what reads the protection.
assert_eq!(v(&s, "B1"), Value::Error(gridcraft_core::CellError::NA));
s.execute("cell.set", json!({"cell": "B1", "input": "=FORMULATEXT(A1)"})).unwrap();
assert_eq!(v(&s, "B1"), Value::Error(gridcraft_core::CellError::NA));
s.execute("review.unprotectSheet", json!({})).unwrap();
assert_eq!(v(&s, "B1"), Value::from("=1+1"), "unprotecting shows it again without re-entering");
}
#[test]
fn protected_sheet_refuses_layout_commands_its_permissions_do_not_allow() {
let mut s = s();
@@ -6393,3 +6555,26 @@ fn pivot_new_sheet_destination_respects_workbook_structure_protection() {
}
assert!(s.execute("insert.pivotTable", json!({"source": "A1:B3", "location": "New"})).is_err());
}
#[test]
fn history_charges_a_changed_sheets_comment_text() {
let mut s = s();
s.execute("review.newNote", json!({"cell": "A1", "text": "n".repeat(30_000), "author": "Example"})).unwrap();
let older = (*s.doc().unwrap().wb).clone();
s.execute("cell.set", json!({"cell": "B2", "input": "x"})).unwrap();
let newer = (*s.doc().unwrap().wb).clone();
// The older snapshot owns the comment text now that the sheet changed under it.
assert!(crate::history_retained_bytes(&older, &newer) >= 30_000);
}
#[test]
fn history_charges_the_storage_of_empty_comment_replies() {
let mut s = s();
s.execute("review.newComment", json!({"cell": "A1", "text": "c", "author": "Ann"})).unwrap();
for _ in 0..500 {
s.execute("review.replyComment", json!({"cell": "A1", "text": "", "author": ""})).unwrap();
}
let sheet = s.doc().unwrap().wb.sheet(0).unwrap().clone();
// 500 replies of no text are still 500 elements.
assert!(crate::sheet_text_bytes(&sheet) >= 500 * std::mem::size_of::<(String, String)>());
}
+28
View File
@@ -59,3 +59,31 @@ fn whole_row_format_keeps_hidden_rows_hidden() {
assert!(info.size.unwrap_or(before) > before + 10.0, "hidden row still grows to its font: before={before}, size={:?}", info.size);
assert!(info.hidden, "formatting a hidden row must not unhide it");
}
#[test]
fn many_merges_fit_rows_the_same_and_skip_merged_cells() {
// Rows are fitted with one sweep over the sheet's merges; a merged cell still never grows
// its row, and rows next to thousands of merges still fit.
let mut s = gridcraft_engine::Session::new();
s.new_workbook();
let values: Vec<_> = (0..2_000).map(|i| json!([format!("v{i}")])).collect();
s.execute("range.setValues", json!({"range": "A1", "values": values})).unwrap();
{
let sh = sheet_mut(&mut s);
sh.merges.push(gridcraft_engine::core::RangeRef::parse("A3:B4").unwrap());
for i in 0..20_000u32 {
let row = 4 + i % 1_990;
sh.merges.push(gridcraft_engine::core::RangeRef::new(
gridcraft_engine::core::CellRef::new(row, 3 + (i % 7) * 2),
gridcraft_engine::core::CellRef::new(row + 3, 4 + (i % 7) * 2),
));
}
}
s.execute("home.fontSize", json!({"range": "A:A", "size": 20})).unwrap();
let sh = s.doc().unwrap().wb.active().unwrap();
let default = sh.default_row_height;
assert!(sh.row_height(0) > 30.0 && sh.row_height(1_999) > 30.0, "rows with big text grow");
assert_eq!(sh.row_height(2), default, "a merged cell does not grow its row");
assert_eq!(sh.row_height(3), default, "nor any row of the merge");
assert!(sh.row_height(4) > 30.0 && sh.row_height(1_000) > 30.0, "rows beside other merges grow");
}
@@ -0,0 +1,366 @@
//! Legacy formulas (no dynamic-array metadata) keep one value through GridCraft (issue #410), on
//! a calculated workbook:
//! - every function that returns several values: a legacy cell formula calling it opens with
//! Excel's implicit `@` and does not spill, and a typed `=@F(…)` saves and reopens with the same
//! single value. The library's functions are found by calling each with sample arguments, so
//! one added later is covered without editing this test.
//! - a legacy formula the analysis cannot finish does not spill and saves as read, without `cm`.
use std::io::Write;
use gridcraft_engine::Session;
use gridcraft_engine::core::{CellRef, DateSystem, Value};
use gridcraft_functions::{Arg, Ctx, FnSpec};
use serde_json::json;
struct Probe;
impl Ctx for Probe {
fn date_system(&self) -> DateSystem {
DateSystem::D1900
}
fn now_serial(&self) -> f64 {
45_000.0
}
fn random(&mut self) -> f64 {
0.5
}
}
/// The 2×2 block of numbers at `A1:B2` in the probe sheets.
fn block() -> Value {
Value::Array(std::sync::Arc::new(
gridcraft_engine::core::Array::new(2, 2, vec![Value::Number(1.0), Value::Number(2.0), Value::Number(3.0), Value::Number(4.0)]).unwrap(),
))
}
/// Sample arguments (formula text, value). Single values only at a position the function lifts
/// over, so an array result is the function's own, not lifting.
fn candidates(scalar: bool) -> Vec<(&'static str, Arg)> {
let mut out = vec![
("3", Arg::val(Value::Number(3.0))),
("1", Arg::val(Value::Number(1.0))),
("\"a1,b22\"", Arg::val(Value::text("a1,b22"))),
("\",\"", Arg::val(Value::text(","))),
("\"[0-9]+\"", Arg::val(Value::text("[0-9]+"))),
("TRUE", Arg::val(Value::Bool(true))),
];
if !scalar {
out.insert(0, ("A1:B2", Arg::reference(block())));
out.insert(1, ("{1,2;3,4}", Arg::val(block())));
}
out
}
/// The first sample call of `spec` that returns more than one value, as formula text.
fn array_call(spec: &FnSpec) -> Option<String> {
let max = spec.max_args.min(spec.min_args.max(1) + 2).min(4).max(spec.min_args);
for n in spec.min_args..=max {
let per: Vec<_> = (0..n).map(|i| candidates(spec.is_scalar_arg(i))).collect();
let per: Vec<_> = if n > 4 { per.into_iter().map(|c| c.into_iter().take(3).collect::<Vec<_>>()).collect() } else { per };
let mut idx = vec![0usize; n];
let mut tries = 0;
loop {
let args: Vec<Arg> = idx.iter().zip(&per).map(|(&i, c)| c[i].1.clone()).collect();
if let Value::Array(a) = gridcraft_functions::call(spec, &args, &mut Probe)
&& a.data.len() > 1
{
let text: Vec<&str> = idx.iter().zip(&per).map(|(&i, c)| c[i].0).collect();
return Some(format!("{}({})", spec.name, text.join(",")));
}
tries += 1;
// Next combination, last position fastest.
let mut k = n;
loop {
if k == 0 || tries > 3000 {
break;
}
k -= 1;
idx[k] += 1;
if idx[k] < per[k].len() {
break;
}
idx[k] = 0;
if k == 0 {
k = usize::MAX;
break;
}
}
if n == 0 || k == usize::MAX || tries > 3000 {
break;
}
}
}
None
}
/// Every library function with an array result, as a sample call, and the functions the
/// evaluator implements itself that return several values or a range.
fn array_calls() -> Vec<String> {
let mut calls: Vec<String> = gridcraft_functions::all().iter().filter_map(array_call).collect();
for found in ["MUNIT", "MODE.MULT", "TEXTSPLIT", "REGEXEXTRACT", "MMULT", "SEQUENCE"] {
assert!(calls.iter().any(|c| c.starts_with(&format!("{found}("))), "{found} not found: {calls:?}");
}
calls.extend(
["TRIMRANGE(A1:B2)", "OFFSET(A1:B2,0,0)", "INDIRECT(\"A1:B2\")", "INDEX(A1:B2,0,1)", "FILTERXML(\"<a><b>1</b><b>2</b></a>\",\"//b\")"]
.map(String::from),
);
calls
}
/// Where the `i`th sample call goes: column E, five rows apart, away from the data in `A1:B2`.
fn slot(i: usize) -> CellRef {
CellRef::new(9 + 5 * i as u32, 4)
}
fn value(s: &Session, c: CellRef) -> Value {
s.doc().unwrap().wb.active().unwrap().value(c)
}
/// The cells right of and below `c` are empty and `c` is not #SPILL!: the formula did not spill.
fn one_value(s: &Session, c: CellRef, what: &str) -> Result<Value, String> {
let v = value(s, c);
let right = value(s, CellRef::new(c.row, c.col + 1));
let below = value(s, CellRef::new(c.row + 1, c.col));
if right != Value::Empty || below != Value::Empty || v.display().contains("SPILL") {
return Err(format!("{what}: {} spilled ({right:?} right, {below:?} below)", v.display()));
}
Ok(v)
}
#[test]
fn legacy_formulas_with_array_functions_show_one_value() {
let calls = array_calls();
let formulas: Vec<_> = calls.iter().enumerate().map(|(i, f)| (slot(i).a1(), f.clone())).collect();
let s = open(&legacy_xlsx(&formulas, ""));
let wrong: Vec<_> = calls.iter().enumerate().filter_map(|(i, f)| one_value(&s, slot(i), f).err()).collect();
assert!(wrong.is_empty(), "{wrong:#?}");
}
#[test]
fn typed_at_of_array_functions_saves_one_value() {
let calls = array_calls();
let mut s = session();
for (a, v) in [("A1", 1), ("B1", 2), ("A2", 3), ("B2", 4)] {
s.execute("cell.set", json!({"cell": a, "input": v.to_string()})).unwrap();
}
for (i, f) in calls.iter().enumerate() {
s.execute("cell.set", json!({"cell": slot(i).a1(), "input": format!("=@{f}")})).unwrap();
}
let before: Vec<_> = (0..calls.len()).map(|i| value(&s, slot(i))).collect();
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let mut back = Session::new();
back.execute("file.open", json!({"name": "typed.xlsx", "base64": saved["base64"]})).unwrap();
let wrong: Vec<_> = calls
.iter()
.enumerate()
.filter_map(|(i, f)| match one_value(&back, slot(i), f) {
Ok(v) if v == before[i] => None,
// A volatile function recalculates on open: one number again.
Ok(Value::Number(_)) if volatile(f) => None,
Ok(v) => Some(format!("@{f}: {} before saving, {} after", before[i].display(), v.display())),
Err(e) => Some(e),
})
.collect();
assert!(wrong.is_empty(), "{wrong:#?}");
}
fn volatile(call: &str) -> bool {
call.split('(').next().and_then(gridcraft_functions::lookup).is_some_and(|spec| spec.volatile)
}
/// Formulas the analysis covers at Excel's nesting limits, and one it cannot finish (names too
/// large to analyse, whose ranges would spill three rows): calculated, none spills, and saving
/// writes each as read, as a plain `<f>`.
#[test]
fn deep_and_unfinished_legacy_formulas_calculate_and_save_as_read() {
let nested = format!("{}A1:A3{}", "ABS(".repeat(200), ")".repeat(200));
let parens = format!("{}A1:A3{}", "ABS((".repeat(64), "))".repeat(64));
let parts = (1..=30).map(|i| format!("Part_{i}")).collect::<Vec<_>>().join("+");
let part = format!("Sheet1!$A$1:$A$3+{}", vec!["Sheet1!$B$1"; 599].join("+"));
let names: String = (1..=30).map(|i| format!(r#"<definedName name="Part_{i}">{part}</definedName>"#)).collect();
let formulas = [("E10".to_string(), nested), ("E15".into(), parens), ("E20".into(), parts)];
let mut s = open(&legacy_xlsx(&formulas, &names));
for (a, _) in &formulas {
one_value(&s, CellRef::parse(a).unwrap(), a).unwrap();
}
// Legacy Excel intersects A1:A3 with row 10 and 15: #VALUE!. The unfinished formula shows the
// top-left value of its sum, 1 + 599 × 2 per name.
assert_eq!(value(&s, CellRef::parse("E10").unwrap()).display(), "#VALUE!");
assert_eq!(value(&s, CellRef::parse("E15").unwrap()).display(), "#VALUE!");
assert_eq!(value(&s, CellRef::parse("E20").unwrap()), Value::Number(30.0 * (1.0 + 599.0 * 2.0)));
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let mut xml = String::new();
std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap();
for (a, f) in &formulas {
assert!(xml.contains(&format!("<f>{f}</f>")), "{a} not saved as read: {xml}");
}
assert!(!xml.contains("cm=") && !xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}");
}
/// Legacy unions where one value is expected, a name whose definition is IF of a range, and
/// multi-valued results in the file's own parentheses: calculated, none spills (legacy Excel
/// intersects a union to #VALUE!), and saving writes each as read, as a plain `<f>`.
#[test]
fn legacy_unions_names_and_parentheses_calculate_and_save_as_read() {
let formulas: Vec<(String, String)> = ["SQRT((A1:A2,B1:B2))", "ABS(((A1:A2,B1:B2)))", "(A1:A2,B1:B2)+1", "SQRT(Flag)", "({1,2}+1)", "{1,2}+1"]
.iter()
.enumerate()
.map(|(i, f)| (slot(i).a1(), f.to_string()))
.collect();
let names = r#"<definedName name="Flag">IF(Sheet1!$A$1:$A$3&gt;1,1,0)</definedName>"#;
let mut s = open(&legacy_xlsx(&formulas, names));
for (a, _) in &formulas {
one_value(&s, CellRef::parse(a).unwrap(), a).unwrap();
}
let shown: Vec<String> = formulas.iter().map(|(a, _)| value(&s, CellRef::parse(a).unwrap()).display()).collect();
assert_eq!(shown, ["#VALUE!", "#VALUE!", "#VALUE!", "0", "2", "2"]);
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let mut xml = String::new();
std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap();
for (a, f) in &formulas {
assert!(xml.contains(&format!("<f>{f}</f>")), "{a} not saved as read: {xml}");
}
assert!(!xml.contains("cm=") && !xml.contains("SINGLE") && !xml.contains("t=\"array\""), "{xml}");
}
/// A whole column in a scalar argument of a legacy formula meets the formula's row (issue #1636,
/// from PR #1660): `ISBLANK(B:B)` in row 4 tests `B4`, so the formula shows `C4` instead of
/// spilling into `G5`, and in row 6, where `B6` is blank, shows 0.
#[test]
fn whole_column_in_a_legacy_scalar_argument_meets_the_formulas_row() {
let formulas: Vec<(String, String)> = [
("B4", r#""Thierry""#),
("C4", "7"),
("G4", r#"IF(ISBLANK(B:B),0,IF(B4="Thierry",C4,0))"#),
("G5", "1"),
("G6", r#"IF(ISBLANK(B:B),0,IF(B6="Thierry",C6,0))"#),
]
.iter()
.map(|(a, f)| (a.to_string(), f.to_string()))
.collect();
let s = open(&legacy_xlsx(&formulas, ""));
let at = |a: &str| CellRef::parse(a).unwrap();
assert_eq!(value(&s, at("G4")), Value::Number(7.0), "B:B meets row 4 instead of spilling");
assert_eq!(value(&s, at("G5")), Value::Number(1.0), "nothing spills below G4");
assert_eq!(one_value(&s, at("G6"), "G6").unwrap(), Value::Number(0.0), "B6 is blank");
}
/// One-cell ranges written as ranges (`B1:B1`) outside any `@`, and formulas whose reading
/// reaches the parser's nesting limit (254 nested functions, far past Excel's 64): saved as read,
/// without `@`. Only a one-cell range under an `@` counts as one an edit may have shrunk. All but
/// the last are calculated without spilling and saved as a plain `<f>`.
#[test]
fn one_cell_ranges_and_formulas_at_the_nesting_limit_save_as_read() {
let deep = |inner: &str| format!("{}{inner}{}", "ABS(".repeat(254), ")".repeat(254));
let formulas: Vec<(String, String)> = [
"SQRT(A1:A3)+B1:B1".to_string(),
"A1:A3*B2:B2".into(),
"SQRT(A1:A3)+$B$2:$B$2".into(),
// Names with a one-cell range inside CHOOSE's index or IF's condition.
"SQRT(NC)".into(),
"SQRT(NI)".into(),
"NC+1".into(),
"NI*2".into(),
deep("A1:A3%"),
deep("-A1:A3"),
]
.into_iter()
.enumerate()
.map(|(i, f)| (slot(i).a1(), f))
.collect();
let names = r#"<definedName name="NC">CHOOSE(Sheet1!$B$1:$B$1,Sheet1!$A$1:$A$3,Sheet1!$A$1:$A$3)</definedName><definedName name="NI">IF(Sheet1!$B$1:$B$1&gt;0,Sheet1!$A$1:$A$3)</definedName>"#;
let (last, rest) = formulas.split_last().unwrap();
let mut s = open(&legacy_xlsx(&formulas, names));
for (a, _) in rest {
one_value(&s, CellRef::parse(a).unwrap(), a).unwrap();
}
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let mut xml = String::new();
std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap();
for (a, f) in rest {
assert!(xml.contains(&format!("<f>{f}</f>")), "{a} not saved as read: {xml}");
}
// A reading the parser cannot nest leaves the formula as read: it calculates as a
// dynamic-array formula would, but the file never gets an `@`.
let (a, f) = last;
assert!(xml.contains(&format!("{f}</f>")), "{a} not saved as read: {xml}");
assert!(!xml.contains('@') && !xml.contains("SINGLE"), "{xml}");
assert_eq!(xml.matches("cm=").count(), 1, "{xml}");
}
/// Deleting rows shrinks the ranges names return (directly, through another name, CHOOSE's and
/// IF's results, parentheses) to one cell: the `@` the reader put on each name is still the one
/// legacy Excel implies, so the formulas save as legacy ones, without `_xlfn.SINGLE`.
#[test]
fn names_shrunk_by_deleting_rows_save_as_legacy() {
let formulas: Vec<(String, String)> =
["SQRT(MM)", "SQRT(CHM)", "SQRT(NCH)", "SQRT(NIF)", "SQRT(PM)"].into_iter().enumerate().map(|(i, f)| (slot(i).a1(), f.to_string())).collect();
let names = r#"<definedName name="MM">Sheet1!$A$1:$A$3</definedName><definedName name="CHM">MM</definedName><definedName name="NCH">CHOOSE(1,Sheet1!$A$1:$A$3,Sheet1!$B$1:$B$3)</definedName><definedName name="NIF">IF(1,Sheet1!$A$1:$A$3,Sheet1!$B$1:$B$3)</definedName><definedName name="PM">(Sheet1!$A$1:$A$3)</definedName>"#;
let mut s = open(&legacy_xlsx(&formulas, names));
s.execute("home.deleteRows", json!({"rows": "2:3"})).unwrap();
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
let bytes = gridcraft_engine::io::base64_decode(saved["base64"].as_str().unwrap()).unwrap();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let mut xml = String::new();
std::io::Read::read_to_string(&mut zip.by_name("xl/worksheets/sheet1.xml").unwrap(), &mut xml).unwrap();
for (a, f) in &formulas {
assert!(xml.contains(&format!("<f>{f}</f>")), "{a} not saved as legacy {f}: {xml}");
}
assert!(!xml.contains("SINGLE") && !xml.contains("cm="), "{xml}");
}
const CT: &str = r#"<?xml version="1.0"?><Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="xml" ContentType="application/xml"/><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Override PartName="/xl/workbook.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml"/><Override PartName="/xl/worksheets/sheet1.xml" ContentType="application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml"/></Types>"#;
/// A one-sheet package: numbers 1–4 in `A1:B2` and `formulas` (address, file text) saved as
/// legacy formulas, without dynamic-array metadata; `names` is `<definedName>` elements.
fn legacy_xlsx(formulas: &[(String, String)], names: &str) -> Vec<u8> {
let esc = |s: &str| s.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;").replace('"', "&quot;");
let mut rows =
String::from(r#"<row r="1"><c r="A1"><v>1</v></c><c r="B1"><v>2</v></c></row><row r="2"><c r="A2"><v>3</v></c><c r="B2"><v>4</v></c></row>"#);
// Cells in one row share its `<row>`, in the order given.
let mut by_row: Vec<(String, String)> = Vec::new();
for (a, f) in formulas {
let r: String = a.chars().filter(char::is_ascii_digit).collect();
let c = format!(r#"<c r="{a}"><f>{}</f></c>"#, esc(f));
match by_row.iter_mut().find(|(row, _)| *row == r) {
Some((_, cells)) => cells.push_str(&c),
None => by_row.push((r, c)),
}
}
for (r, cells) in by_row {
rows.push_str(&format!(r#"<row r="{r}">{cells}</row>"#));
}
let names = if names.is_empty() { String::new() } else { format!("<definedNames>{names}</definedNames>") };
let parts: [(&str, String); 5] = [
("[Content_Types].xml", CT.into()),
("_rels/.rels", r#"<?xml version="1.0"?><Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="xl/workbook.xml"/></Relationships>"#.into()),
("xl/workbook.xml", format!(r#"<?xml version="1.0"?><workbook xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"><sheets><sheet name="Sheet1" sheetId="1" r:id="rId1"/></sheets>{names}</workbook>"#)),
("xl/_rels/workbook.xml.rels", r#"<?xml version="1.0"?><Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/worksheet" Target="worksheets/sheet1.xml"/></Relationships>"#.into()),
("xl/worksheets/sheet1.xml", format!(r#"<?xml version="1.0"?><worksheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main"><sheetData>{rows}</sheetData></worksheet>"#)),
];
let mut z = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
for (name, data) in parts {
z.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
z.write_all(data.as_bytes()).unwrap();
}
z.finish().unwrap().into_inner()
}
fn session() -> Session {
let mut s = Session::new();
s.new_workbook();
s
}
fn open(bytes: &[u8]) -> Session {
let mut s = Session::new();
s.execute("file.open", json!({"name": "legacy.xlsx", "base64": gridcraft_engine::io::base64_encode(bytes)})).unwrap();
s
}
+5 -1
View File
@@ -302,11 +302,15 @@ impl Parser {
{
return Ok(Expr::Unary(UnOp::Spill, Box::new(r)));
}
// Files write `@x` as `_xlfn.SINGLE(x)`.
// Files write `@x` as `_xlfn.SINGLE(x)`; `SINGLE(A1:A3*2)` is `@(A1:A3*2)`.
if name == "SINGLE"
&& args.len() == 1
&& let Some(x) = args.pop()
{
let x = match x {
Expr::Binary(..) | Expr::Unary(UnOp::Neg | UnOp::Plus | UnOp::Percent, _) => Expr::Paren(Box::new(x)),
x => x,
};
return Ok(Expr::Unary(UnOp::At, Box::new(x)));
}
// Files may spell a trim reference as a function call (`_xlfn._TRO_ALL(A1:E10)`).
+9
View File
@@ -106,6 +106,15 @@ fn spill_operator() -> Result<(), Box<dyn std::error::Error>> {
Ok(())
}
/// Files spell `@x` as `_xlfn.SINGLE(x)`; an operation inside keeps its parentheses.
#[test]
fn single_is_the_implicit_intersection_operator() {
assert_eq!(rt("_xlfn.SINGLE(A1:A3)*2"), "@A1:A3*2");
assert_eq!(rt("_xlfn.SINGLE(A1:A3*2)"), "@(A1:A3*2)");
assert_eq!(rt("_xlfn.SINGLE(-A1:A3)"), "@(-A1:A3)");
assert_eq!(rt("SQRT(_xlfn.SINGLE(Data[Qty]))"), "SQRT(@Data[Qty])");
}
#[test]
fn delete_sheet_shrinks_3d_span() -> Result<(), Box<dyn std::error::Error>> {
let order: Vec<String> = ["Sheet1", "Sheet2", "Sheet3", "Results"].iter().map(|s| s.to_string()).collect();
+96 -19
View File
@@ -70,39 +70,59 @@ fn cell_criterion(v: &Value) -> Option<Criterion> {
/// when the label is unknown, criterion).
type CriteriaRow = (usize, Vec<(usize, Option<usize>, Criterion)>);
/// Most condition tests one selection may make (records times conditions): every record can be
/// tested against every criteria row when none matches, so a modest database and criteria table
/// multiply into billions of tests. Past it the function is `#NUM!`, as lifted lookups are; 5e8
/// is a few seconds of matching.
const SELECTION_WORK_BUDGET: usize = 500_000_000;
/// Rows (1-based within the database, excluding the header) of matching records.
fn matching_records(db: &Array, crit: &Array, ctx: &mut dyn Ctx) -> Vec<usize> {
fn matching_records(db: &Array, crit: &Array, ctx: &mut dyn Ctx) -> R<Vec<usize>> {
// The database column under each criteria label, found once per label rather than per cell.
let columns: Vec<Option<usize>> = (0..crit.cols)
.map(|c| match crit.get(0, c) {
Some(Value::Text(t)) => find_column(db, t),
Some(Value::Number(n)) => find_column(db, &gridcraft_core::number_to_text(*n)),
_ => None,
})
.collect();
// Per criteria row: its index and a list of (criteria column, database column or None when
// the label is unknown, criterion).
let mut rows: Vec<CriteriaRow> = Vec::new();
let mut tests = 0usize;
for r in 1..crit.rows {
let mut conds = Vec::new();
for c in 0..crit.cols {
let Some(cell) = crit.get(r, c) else { continue };
let Some(cr) = cell_criterion(cell) else { continue };
let col = match crit.get(0, c) {
Some(Value::Text(t)) => find_column(db, t),
Some(Value::Number(n)) => find_column(db, &gridcraft_core::number_to_text(*n)),
_ => None,
};
conds.push((c, col, cr));
conds.push((c, columns.get(c).copied().flatten(), cr));
}
tests = tests.saturating_add(conds.len().max(1));
rows.push((r, conds));
}
let all_match = rows.is_empty();
(1..db.rows)
// An empty criteria row matches every record without testing a condition, so nothing is charged.
let all_match = rows.is_empty() || rows.iter().any(|(_, conds)| conds.is_empty());
// Computed criteria (a formula evaluated per record) count like any other test.
if !all_match && db.rows.saturating_sub(1).saturating_mul(tests) > SELECTION_WORK_BUDGET {
return Err(CellError::Num);
}
Ok((1..db.rows)
.filter(|&rec| {
all_match
|| rows.iter().any(|(r, conds)| {
conds.iter().all(|(c, col, cr)| match col {
Some(f) => cr.matches(db.get(rec, *f).unwrap_or(&Value::Empty)),
// Under a label that names no field, only a formula (a computed
// criterion) can hold.
None => ctx.computed_criterion(*r, *c, db, rec).unwrap_or(false),
})
})
// Out of work: a computed criterion would fail at once, so records are left out
// without scanning the criteria rows; the per-row check still stops mid-scan.
|| (!ctx.exhausted()
&& rows.iter().any(|(r, conds)| {
!ctx.exhausted()
&& conds.iter().all(|(c, col, cr)| !ctx.exhausted() && match col {
Some(f) => cr.matches(db.get(rec, *f).unwrap_or(&Value::Empty)),
// Under a label that names no field, only a formula (a computed
// criterion) can hold.
None => ctx.computed_criterion(*r, *c, db, rec).unwrap_or(false),
})
}))
})
.collect()
.collect())
}
struct Selection<'a> {
@@ -123,7 +143,7 @@ fn select<'a>(a: &'a [Arg], ctx: &mut dyn Ctx) -> R<Selection<'a>> {
let db = as_array(dbv);
let crit = as_array(critv);
let field = field_index(&db, a)?;
let records = matching_records(&db, &crit, ctx);
let records = matching_records(&db, &crit, ctx)?;
Ok(Selection { db, field, records })
}
@@ -352,6 +372,63 @@ mod tests {
is_err(d("DSUM", n(9.0)), CellError::Value);
}
/// Computed criteria are no longer asked for once the host is out of work.
#[test]
fn matching_stops_asking_computed_criteria_when_exhausted() {
struct Spent {
asked: usize,
}
impl crate::Ctx for Spent {
fn date_system(&self) -> gridcraft_core::DateSystem {
gridcraft_core::DateSystem::D1900
}
fn now_serial(&self) -> f64 {
0.0
}
fn random(&mut self) -> f64 {
0.0
}
fn computed_criterion(&mut self, _: usize, _: usize, _: &gridcraft_core::Array, _: usize) -> Option<bool> {
self.asked += 1;
Some(true)
}
fn exhausted(&self) -> bool {
self.asked > 0
}
}
let criteria = arr(vec![vec![tv("Formula")], vec![nv(1.0)]]);
let mut ctx = Spent { asked: 0 };
let spec = crate::lookup("DCOUNT").unwrap();
crate::call(spec, &[rf(db()), t("Age"), rf(criteria)], &mut ctx);
assert_eq!(ctx.asked, 1);
// Within one criteria row, the conditions after the one that spent the work are skipped too.
let criteria = arr(vec![vec![tv("Formula"), tv("Other")], vec![nv(1.0), nv(1.0)]]);
let mut ctx = Spent { asked: 0 };
crate::call(spec, &[rf(db()), t("Age"), rf(criteria)], &mut ctx);
assert_eq!(ctx.asked, 1);
}
/// Security: records times criteria conditions past the selection budget is `#NUM!` before
/// any record is tested; a nonmatching criteria table used to test every record against
/// every criteria row.
#[test]
fn selection_work_is_bounded() {
use gridcraft_core::Array;
let column = |label: &str, len: usize, v: Value| {
let mut data = vec![tv(label)];
data.extend(std::iter::repeat_n(v, len));
Value::from(Array::column(data))
};
let db = column("Tree", 100_000, tv("Apple"));
let wide = column("Tree", 5_001, tv("=Plum"));
is_err(ev("DCOUNTA", vec![rf(db.clone()), t("Tree"), rf(wide)]), CellError::Num);
let narrow = column("Tree", 2, tv("=Apple"));
close(ev("DCOUNTA", vec![rf(db.clone()), t("Tree"), rf(narrow)]), 100_000.0);
// Empty criteria rows test no condition: the first one matches every record, so they cost nothing.
let empty_rows = column("Tree", 5_001, Value::Empty);
close(ev("DCOUNTA", vec![rf(db), t("Tree"), rf(empty_rows)]), 100_000.0);
}
#[test]
fn simple_criteria() {
let c = |cells: Vec<Value>| arr(vec![vec![tv("Tree"), tv("Height")], cells]);
+22 -1
View File
@@ -416,13 +416,17 @@ fn workday_core(a: &[Arg], c: &mut dyn Ctx, mask: [bool; 7], hol: Option<&Arg>)
if days == 0 {
return num_val(start as f64);
}
// The holidays on working days, still sorted (`holidays` sorts them), so each step counts
// the ones it passed with two binary searches rather than a scan of every holiday: a run of
// consecutive holidays takes one step per holiday.
let hs: Vec<i64> = hs.into_iter().filter(|&h| !mask[weekday0(sys, h)]).collect();
let dir = days.signum();
let mut cur = start;
let mut rem = days.abs();
while rem > 0 {
let next = advance(sys, cur, rem, dir, &mask, per_week);
let (lo, hi) = if dir > 0 { (cur + 1, next) } else { (next, cur - 1) };
rem = hs.iter().filter(|&&h| h >= lo && h <= hi && !mask[weekday0(sys, h)]).count() as i64;
rem = hs.partition_point(|&h| h <= hi).saturating_sub(hs.partition_point(|&h| h < lo)) as i64;
cur = next;
if !(0..=MAX_SERIAL as i64).contains(&cur) {
return Err(CellError::Num);
@@ -755,6 +759,23 @@ mod tests {
is_err(ev("TIMEVALUE", vec![t("x")]), CellError::Value);
}
/// Security: WORKDAY counts the holidays each step passes by binary search, so a long run of
/// consecutive holidays is linear work, not one scan of every holiday per holiday.
#[test]
fn workday_over_many_consecutive_holidays() {
let start = d(2000.0, 1.0, 3.0);
let hol: Vec<f64> = (1..=200_000).map(|i| start + f64::from(i)).collect();
let out = ev("WORKDAY.INTL", vec![n(start), n(1.0), n(11.0), av(col(&hol))]);
// Sundays are the only weekend (11): 200,000 days of holidays are skipped, landing on the
// first working day after them.
let after = start + 200_001.0;
let sunday = ev("WEEKDAY", vec![n(after), n(2.0)]) == nv(7.0);
close(out, if sunday { after + 1.0 } else { after });
// Holidays on a weekend don't push the result out.
let fri = d(2026.0, 10.0, 9.0);
close(ev("WORKDAY", vec![n(fri), n(1.0), av(col(&[fri + 1.0, fri + 3.0]))]), fri + 4.0);
}
#[test]
fn day_counts() {
close(ev("DAYS", vec![n(d(2021.0, 3.0, 15.0)), n(d(2021.0, 2.0, 1.0))]), 42.0);
+45 -14
View File
@@ -171,13 +171,17 @@ fn sort(args: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
let a = if by_col { a.transpose() } else { a };
let idx = if has(args, 1) { int_list(&arg(args, 1)?.value)? } else { vec![1] };
let ord = if has(args, 2) { int_list(&arg(args, 2)?.value)? } else { vec![1] };
if idx.is_empty() || idx.iter().any(|&i| i < 1 || i as usize > a.cols) || ord.iter().any(|&o| o != 1 && o != -1) {
if idx.is_empty() || idx.iter().any(|&i| i < 1 || i > a.cols as i64) || ord.iter().any(|&o| o != 1 && o != -1) {
return Err(CellError::Value);
}
if ord.len() > 1 && ord.len() != idx.len() {
return Err(CellError::Value);
}
let orders: Vec<i64> = (0..idx.len()).map(|i| *ord.get(i).or(ord.first()).unwrap_or(&1)).collect();
// A column repeated in sort_index can't break a tie its first use left, so only first uses
// stay: the keys copied per row are then at most the row itself, however long sort_index is.
let mut seen = std::collections::HashSet::new();
let (idx, orders): (Vec<i64>, Vec<i64>) = idx.into_iter().zip(orders).filter(|(i, _)| seen.insert(*i)).unzip();
let mut lines: Vec<(Vec<Value>, Vec<Value>)> =
rows_of(&a).into_iter().map(|r| (idx.iter().map(|&i| r.get(i as usize - 1).cloned().unwrap_or(Value::Empty)).collect(), r)).collect();
lines.sort_by(|x, y| cmp_keys(&x.0, &y.0, &orders));
@@ -928,9 +932,16 @@ fn pivotby(args: &[Arg], ctx: &mut dyn Ctx) -> R<Value> {
let ckeys: Vec<Vec<Value>> = p.rows.iter().map(|&r| key_row(&cf, r)).collect();
let rk = distinct_keys(&rkeys, if row_sort.unsigned_abs() as usize <= rf.cols { row_sort } else { 0 });
let ck = distinct_keys(&ckeys, if col_sort.unsigned_abs() as usize <= cf.cols { col_sort } else { 0 });
// Source rows of every row key, column key and pair, built in one pass, so a total reads
// its rows instead of rescanning the whole source once per total.
let mut cell: HashMap<(String, String), Vec<usize>> = HashMap::new();
let mut by_row: HashMap<String, Vec<usize>> = HashMap::new();
let mut by_col: HashMap<String, Vec<usize>> = HashMap::new();
for ((r, c), &i) in rkeys.iter().zip(ckeys.iter()).zip(p.rows.iter()) {
cell.entry((row_key(r), row_key(c))).or_default().push(i);
let (r, c) = (row_key(r), row_key(c));
by_row.entry(r.clone()).or_default().push(i);
by_col.entry(c.clone()).or_default().push(i);
cell.entry((r, c)).or_default().push(i);
}
let mut agg = |rows: &[usize], c: usize| -> Value {
if rows.is_empty() {
@@ -940,14 +951,12 @@ fn pivotby(args: &[Arg], ctx: &mut dyn Ctx) -> R<Value> {
aggregate_with(f, &vs, ctx)
};
let rows_for = |r: Option<&Vec<Value>>, c: Option<&Vec<Value>>| -> Vec<usize> {
let rkey = r.map(|k| row_key(k));
let ckey = c.map(|k| row_key(k));
p.rows
.iter()
.zip(rkeys.iter().zip(ckeys.iter()))
.filter(|(_, (a, b))| rkey.as_ref().is_none_or(|k| *k == row_key(a)) && ckey.as_ref().is_none_or(|k| *k == row_key(b)))
.map(|(&i, _)| i)
.collect()
match (r, c) {
(Some(r), Some(c)) => cell.get(&(row_key(r), row_key(c))).cloned().unwrap_or_default(),
(Some(r), None) => by_row.get(&row_key(r)).cloned().unwrap_or_default(),
(None, Some(c)) => by_col.get(&row_key(c)).cloned().unwrap_or_default(),
(None, None) => p.rows.clone(),
}
};
let mut cols: Vec<Option<&Vec<Value>>> = ck.iter().map(Some).collect();
if col_total > 0 {
@@ -997,10 +1006,7 @@ fn pivotby(args: &[Arg], ctx: &mut dyn Ctx) -> R<Value> {
}
};
for c in &cols {
let rows = match (r, c) {
(Some(rk), Some(ck)) => cell.get(&(row_key(rk), row_key(ck))).cloned().unwrap_or_default(),
_ => rows_for(r, *c),
};
let rows = rows_for(r, *c);
for vc in 0..vals.cols {
line.push(agg(&rows, vc));
}
@@ -1204,6 +1210,21 @@ mod tests {
assert_eq!(rows_of(&r), vec![vec![nv(0.0), nv(5.0)], vec![nv(1.0), nv(2.0)], vec![nv(1.0), nv(1.0)]]);
}
/// Security: a column repeated in sort_index keys the sort once. The keys copied per row stay
/// within the row, however long sort_index is, and a repeat (even with the other order) can't
/// change the result, since its first use already ordered the rows.
#[test]
fn sort_drops_repeated_sort_indices() {
let m = || av(arr(vec![vec![nv(1.0), nv(2.0)], vec![nv(1.0), nv(1.0)], vec![nv(0.0), nv(5.0)]]));
let want = vec![vec![nv(0.0), nv(5.0)], vec![nv(1.0), nv(2.0)], vec![nv(1.0), nv(1.0)]];
let r = ev("SORT", vec![m(), av(row(&[1.0, 1.0, 2.0, 1.0])), av(row(&[1.0, -1.0, -1.0, 1.0]))]);
assert_eq!(rows_of(&r), want);
let idx = vec![1.0; 1_000_000];
let tall = av(gridcraft_core::Array::filled(2_000, 1, nv(1.0)).into());
let r = ev("SORT", vec![tall, av(row(&idx))]);
assert_eq!(rows_of(&r).len(), 2_000);
}
#[test]
fn sortby() {
let r = ev("SORTBY", vec![av(col(&[1.0, 2.0, 3.0])), av(col(&[30.0, 10.0, 20.0]))]);
@@ -1447,6 +1468,16 @@ mod tests {
);
}
/// Security: PIVOTBY totals read their source rows from indexes built in the grouping pass,
/// rather than rescanning the source once per total, so many distinct keys stay linear.
#[test]
fn pivotby_totals_with_many_groups() {
let keys: Vec<f64> = (0..20_000).map(f64::from).collect();
let r = rows_of(&ev("PIVOTBY", vec![av(col(&keys)), av(col(&vec![1.0; 20_000])), av(col(&vec![2.0; 20_000])), t("SUM")]));
assert_eq!(r.get(1), Some(&vec![nv(0.0), nv(2.0), nv(2.0)]));
assert_eq!(r.last(), Some(&vec![tv("Total"), nv(40_000.0), nv(40_000.0)]));
}
#[test]
fn pivotby_size_check_counts_column_header_rows() {
// One row, n column fields and n value columns: n header rows + value-name row + body row + total row,
+1 -1
View File
@@ -91,7 +91,7 @@ fn na(_a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
fn n_fn(a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
// N(range) reads the range's first cell, but a computed array (N(A1:A3>0), N({...})) is
// converted element by element, so N can stand in for `--` inside SUMPRODUCT.
if let Some(Arg { value: Value::Array(arr), from_ref: false }) = a.first() {
if let Some(Arg { value: Value::Array(arr), from_ref: false, .. }) = a.first() {
let data = arr.data.iter().map(|v| n_of(v.clone()).unwrap_or_else(Value::Error)).collect();
return Array::new(arr.rows, arr.cols, data).map(Value::from).ok_or(CellError::Calc);
}
+16 -2
View File
@@ -87,16 +87,19 @@ pub use stat::{aggregate_values, aggregate_values_k};
pub struct Arg {
pub value: Value,
pub from_ref: bool,
/// A written argument whose value came out blank (a lookup of an empty cell): given, not
/// omitted, although its value is `Value::Empty`.
pub blank_value: bool,
}
impl Arg {
/// A literal (non-reference) argument.
pub fn val(value: impl Into<Value>) -> Arg {
Arg { value: value.into(), from_ref: false }
Arg { value: value.into(), from_ref: false, blank_value: false }
}
/// An argument that came from a cell or range reference.
pub fn reference(value: impl Into<Value>) -> Arg {
Arg { value: value.into(), from_ref: true }
Arg { value: value.into(), from_ref: true, blank_value: false }
}
}
@@ -113,6 +116,11 @@ pub trait Ctx {
fn computed_criterion(&mut self, _row: usize, _col: usize, _db: &Array, _record: usize) -> Option<bool> {
None
}
/// Whether the cell's work budget has run out: the database functions stop asking for computed
/// criteria then, as every one would fail at once.
fn exhausted(&self) -> bool {
false
}
/// For GROUPBY/PIVOTBY: whether their function argument is a LAMBDA (its value is then not used).
fn has_lambda(&self) -> bool {
false
@@ -308,6 +316,12 @@ mod tests {
assert_eq!(ev("REGEXREPLACE", vec![t("2026-10-22"), t("(\\d+)-(\\d+)-(\\d+)"), t("$3/$2/$1")]), Value::text("22/10/2026"));
assert_eq!(ev("REGEXREPLACE", vec![t("abc"), t("x*"), t("-")]), Value::text("-a-b-c-"));
assert_eq!(ev("REGEXREPLACE", vec![t("a1b2"), t("[0-9]"), t("#"), n(2.0)]), Value::text("a1b#"));
assert_eq!(ev("REGEXREPLACE", vec![t("a1b2"), t("[0-9]"), t("#"), n(-2.0)]), Value::text("a#b2"));
assert_eq!(ev("REGEXREPLACE", vec![t("a1b2"), t("[0-9]"), t("#"), n(-3.0)]), Value::text("a1b2"));
// 2^32 is no occurrence the text has (and on 32-bit targets it must not wrap to 0).
for occurrence in [4_294_967_296.0, -4_294_967_296.0] {
assert_eq!(ev("REGEXREPLACE", vec![t("a1b2"), t("[0-9]"), t("#"), n(occurrence)]), Value::text("a1b2"));
}
// One match whose replacement repeats it 10,000 times: refused before it is built.
assert_eq!(ev("REGEXREPLACE", vec![t(&many), t("^.*$"), t(&"$0".repeat(10_000))]), Value::Error(CellError::Value));
assert_eq!(ev("REGEXREPLACE", vec![t(&many), t("^.*$"), t(&"$0".repeat(10_000)), n(1.0)]), Value::Error(CellError::Value));
+3 -6
View File
@@ -50,12 +50,7 @@ fn reject_multi_cell_refs(spec: &FnSpec, args: &[Arg]) -> Option<Vec<Arg>> {
if !args.iter().enumerate().any(|(i, a)| multi(i, a)) {
return None;
}
Some(
args.iter()
.enumerate()
.map(|(i, a)| if multi(i, a) { Arg { value: Value::Error(CellError::Value), from_ref: false } } else { a.clone() })
.collect(),
)
Some(args.iter().enumerate().map(|(i, a)| if multi(i, a) { Arg::val(Value::Error(CellError::Value)) } else { a.clone() }).collect())
}
/// Lookups whose every call searches a lookup vector from its start, so a call lifted over `n`
@@ -120,6 +115,8 @@ pub(crate) fn call_lifted(spec: &FnSpec, args: &[Arg], ctx: &mut dyn Ctx) -> Val
{
let v = arr.get_broadcast(r, c);
slot.value = if let Value::Array(inner) = &v { inner.data.first().cloned().unwrap_or(Value::Empty) } else { v };
// A blank element of a written array argument is given, as a scalar blank is.
slot.blank_value = !slot.from_ref && matches!(slot.value, Value::Empty);
}
}
let out = (spec.imp)(&cell_args, ctx);
+10 -12
View File
@@ -132,11 +132,6 @@ fn row(a: &Array, r: usize) -> Vec<Value> {
(0..a.cols).map(|c| a.get(r, c).cloned().unwrap_or(Value::Empty)).collect()
}
/// A looked-up cell value: blanks read as 0 like a reference to an empty cell.
fn cell_result(v: Value) -> Value {
if matches!(v, Value::Empty) { Value::Number(0.0) } else { v }
}
// ---------------------------------------------------------------------------------------------
// VLOOKUP / HLOOKUP / LOOKUP / MATCH
@@ -160,7 +155,9 @@ fn vh_lookup(args: &[Arg], vertical: bool) -> R<Value> {
let pos = if approx { bsearch_last(&x, &keys, false) } else { find_exact(&x, &keys, true, false) };
let pos = pos.ok_or(CellError::NA)?;
let v = if vertical { table.get(pos, idx) } else { table.get(idx, pos) };
Ok(cell_result(v.cloned().unwrap_or(Value::Empty)))
// An empty target stays blank, as in Excel: ISBLANK sees it as blank and `&` /
// CONCATENATE treat it as ""; only a formula's final result displays 0 (#374).
Ok(v.cloned().unwrap_or(Value::Empty))
}
fn vlookup(a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
@@ -193,18 +190,18 @@ fn lookup(args: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
} else {
None
};
return v.map(cell_result).ok_or(CellError::NA);
return v.ok_or(CellError::NA);
}
// Array form: search the first row of a wide array, else the first column; return the
// matching entry from the last row / column.
if la.cols > la.rows {
let keys = row(&la, 0);
let pos = bsearch_last(&x, &keys, false).ok_or(CellError::NA)?;
Ok(cell_result(la.get(la.rows - 1, pos).cloned().unwrap_or(Value::Empty)))
Ok(la.get(la.rows - 1, pos).cloned().unwrap_or(Value::Empty))
} else {
let keys = column(&la, 0);
let pos = bsearch_last(&x, &keys, false).ok_or(CellError::NA)?;
Ok(cell_result(la.get(pos, la.cols - 1).cloned().unwrap_or(Value::Empty)))
Ok(la.get(pos, la.cols - 1).cloned().unwrap_or(Value::Empty))
}
}
@@ -325,13 +322,13 @@ fn xlookup(args: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
if vertical {
let r = row(&ra, p);
if r.len() == 1 {
return Ok(cell_result(r.into_iter().next().unwrap_or(Value::Empty)));
return Ok(r.into_iter().next().unwrap_or(Value::Empty));
}
array_val(1, r.len(), r)
} else {
let c = column(&ra, p);
if c.len() == 1 {
return Ok(cell_result(c.into_iter().next().unwrap_or(Value::Empty)));
return Ok(c.into_iter().next().unwrap_or(Value::Empty));
}
array_val(c.len(), 1, c)
}
@@ -595,7 +592,8 @@ mod tests {
is_err(ev("VLOOKUP", vec![n(4.0), rf(table()), n(2.0), b(false)]), CellError::NA);
is_err(ev("VLOOKUP", vec![n(4.0), rf(table()), n(2.0), empty()]), CellError::NA);
close(ev("VLOOKUP", vec![n(3.0), rf(table()), n(3.0), b(false)]), 30.0);
close(ev("VLOOKUP", vec![n(5.0), rf(table()), n(3.0), b(false)]), 0.0);
// An empty target stays blank (the cell displays 0 only as a final result) (#374).
assert_eq!(ev("VLOOKUP", vec![n(5.0), rf(table()), n(3.0), b(false)]), Value::Empty);
is_err(ev("VLOOKUP", vec![n(3.0), rf(table()), n(4.0)]), CellError::Ref);
is_err(ev("VLOOKUP", vec![n(3.0), rf(table()), n(0.0)]), CellError::Value);
close(ev("VLOOKUP", vec![t("b*"), rf(names()), n(2.0), b(false)]), 2.0);
+79 -23
View File
@@ -30,17 +30,27 @@ fn check_len(s: String) -> R<Value> {
/// Pushes `piece` onto `out`, failing once the result exceeds the cell limit.
fn push_capped(out: &mut String, piece: &str) -> R<()> {
out.push_str(piece);
if out.len() > MAX_TEXT * 4 {
// Checked before copying, so one oversized piece is refused without being appended.
if out.len().saturating_add(piece.len()) > MAX_TEXT * 4 {
return Err(CellError::Value);
}
out.push_str(piece);
Ok(())
}
/// `v.to_text()` without copying text values.
fn piece(v: &Value) -> R<std::borrow::Cow<'_, str>> {
match v {
Value::Text(t) => Ok(std::borrow::Cow::Borrowed(t)),
Value::Array(a) => a.data.first().map_or(Ok(std::borrow::Cow::Borrowed("")), piece),
_ => v.to_text().map(std::borrow::Cow::Owned),
}
}
fn concatenate(a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
let mut out = String::new();
for x in a {
push_capped(&mut out, &scalar(x).to_text()?)?;
push_capped(&mut out, &piece(&scalar(x))?)?;
}
check_len(out)
}
@@ -49,31 +59,49 @@ fn concat(a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
let mut out = String::new();
for x in a {
for v in as_array(&x.value).iter() {
push_capped(&mut out, &v.to_text()?)?;
push_capped(&mut out, &piece(v)?)?;
}
}
check_len(out)
}
/// The error `v.to_text()` would give, found without building any text.
fn text_error(v: &Value) -> Option<CellError> {
match v {
Value::Error(e) => Some(*e),
Value::Array(a) => a.data.first().and_then(text_error),
_ => None,
}
}
fn textjoin(a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
let delims: Vec<String> = as_array(&arg(a, 0)?.value).iter().map(|v| v.to_text()).collect::<R<_>>()?;
// Errors are found first, in argument order, so they win as before; text is then built one
// piece at a time into the capped output, so many references to one long text (or a long
// delimiter list) stop at the cap instead of first copying every piece.
let delims = as_array(&arg(a, 0)?.value);
if let Some(e) = delims.iter().find_map(text_error) {
return Err(e);
}
let ignore = boolean(a, 1)?;
let mut items = Vec::new();
for x in a.iter().skip(2) {
for v in as_array(&x.value).iter() {
let s = v.to_text()?;
if ignore && s.is_empty() {
continue;
}
items.push(s);
}
let items: Vec<_> = a.iter().skip(2).map(|x| as_array(&x.value)).collect();
if let Some(e) = items.iter().flat_map(|x| x.iter()).find_map(text_error) {
return Err(e);
}
let mut out = String::new();
for (i, s) in items.iter().enumerate() {
if i > 0 && !delims.is_empty() {
push_capped(&mut out, &delims[(i - 1) % delims.len()])?;
let mut joined = 0usize;
for v in items.iter().flat_map(|x| x.iter()) {
let s = piece(v)?;
if ignore && s.is_empty() {
continue;
}
push_capped(&mut out, s)?;
if joined > 0
&& !delims.data.is_empty()
&& let Some(d) = delims.data.get((joined - 1) % delims.data.len())
{
push_capped(&mut out, &piece(d)?)?;
}
push_capped(&mut out, &s)?;
joined += 1;
}
check_len(out)
}
@@ -914,13 +942,25 @@ fn regexreplace(a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
out.push_str(s.get(last..).unwrap_or_default());
return check_len(out);
}
let caps: Vec<regex::Captures> = re.captures_iter(&s).collect();
let k = occ.unsigned_abs() as usize;
let idx = if occ > 0 { k.checked_sub(1) } else { caps.len().checked_sub(k) };
let Some(cap) = idx.and_then(|i| caps.get(i)) else { return text_val(s) };
// An occurrence past what `usize` holds (32-bit targets) names no match: the text is returned as it was.
let Ok(k) = usize::try_from(occ.unsigned_abs()) else { return text_val(s) };
// Keeps only the wanted match's captures: the nth from the front, or a window of the last `k`.
let cap = if occ > 0 {
re.captures_iter(&s).nth(k - 1)
} else {
let mut last = std::collections::VecDeque::new();
for cap in re.captures_iter(&s) {
if last.len() == k {
last.pop_front();
}
last.push_back(cap);
}
if last.len() == k { last.pop_front() } else { None }
};
let Some(cap) = cap else { return text_val(s) };
let Some(m) = cap.get(0) else { return text_val(s) };
out.push_str(s.get(..m.start()).unwrap_or_default());
expand_capped(cap, &rep, &mut out, limit).ok_or(CellError::Value)?;
expand_capped(&cap, &rep, &mut out, limit).ok_or(CellError::Value)?;
out.push_str(s.get(m.end()..).unwrap_or_default());
check_len(out)
}
@@ -1171,6 +1211,22 @@ mod tests {
assert_eq!(rows_of(&ev("CONCATENATE", vec![av(row_t(&["a", "b"])), t("x")])), vec![vec![tv("ax"), tv("bx")]]);
}
/// Security: TEXTJOIN builds its result piece by piece and stops at the text cap, rather
/// than first copying every input (100,000 references to one 32,767-character text were
/// 3 GB of copies). Errors among the inputs still win, in argument order.
#[test]
fn textjoin_stops_at_the_text_cap() {
use gridcraft_core::Array;
let long = tv(&"x".repeat(32_767));
let many = || av(Array::filled(100_000, 1, long.clone()).into());
is_err(ev("TEXTJOIN", vec![t(","), b(true), many()]), CellError::Value);
is_err(ev("TEXTJOIN", vec![av(Array::filled(1, 100_000, long.clone()).into()), b(false), t("a"), t("b")]), CellError::Value);
is_err(ev("TEXTJOIN", vec![t(","), b(true), many(), e(CellError::Div0)]), CellError::Div0);
is_err(ev("TEXTJOIN", vec![av(arr(vec![vec![tv(","), Value::Error(CellError::NA)]])), b(true), t("a")]), CellError::NA);
// Delimiters cycle and are converted only where used.
is_text(ev("TEXTJOIN", vec![av(arr(vec![vec![nv(1.0), tv(";")]])), b(false), t("a"), t("b"), t("c"), t("d")]), "a1b;c1d");
}
fn row_t(xs: &[&str]) -> Value {
arr(vec![xs.iter().map(|x| tv(x)).collect()])
}
+1 -1
View File
@@ -51,7 +51,7 @@ pub(crate) fn arg(args: &[Arg], i: usize) -> R<&Arg> {
/// Whether argument `i` was supplied (present and not an empty literal like `F(1,)`).
pub(crate) fn has(args: &[Arg], i: usize) -> bool {
matches!(args.get(i), Some(a) if !matches!(a.value, Value::Empty) || a.from_ref)
matches!(args.get(i), Some(a) if !matches!(a.value, Value::Empty) || a.from_ref || a.blank_value)
}
/// Number coercion for a scalar parameter: empty = 0, TRUE = 1, numeric text parses.
+47 -9
View File
@@ -1,6 +1,6 @@
//! JSON-RPC 2.0 framing and the MCP lifecycle / tools / resources / prompts methods.
use std::io::{BufRead, Write};
use std::io::{BufRead, Read, Write};
use serde_json::{Value, json};
@@ -45,6 +45,45 @@ fn response(id: Value, result: Value) -> Value {
json!({"jsonrpc": "2.0", "id": id, "result": result})
}
/// Reads the next newline-delimited message into `buf`, keeping at most `MAX_LINE` bytes of it:
/// a client sending no newline can't make the server allocate more (#2164). `None` at the end of
/// `input`; `Some(false)` for a line over the limit, whose rest is read and dropped so the next
/// message is read whole.
pub(crate) fn read_message(input: &mut impl BufRead, buf: &mut Vec<u8>) -> std::io::Result<Option<bool>> {
buf.clear();
// One byte past the limit tells an over-long line from one of exactly `MAX_LINE` bytes.
let limit = u64::try_from(MAX_LINE).unwrap_or(u64::MAX).saturating_add(1);
if input.take(limit).read_until(b'\n', buf)? == 0 {
return Ok(None);
}
if buf.len() > MAX_LINE && buf.last() != Some(&b'\n') {
skip_line(input)?;
buf.clear();
return Ok(Some(false));
}
Ok(Some(true))
}
/// Reads up to and including the next newline (or to the end of `input`) without keeping it.
fn skip_line(input: &mut impl BufRead) -> std::io::Result<()> {
loop {
let chunk = input.fill_buf()?;
if chunk.is_empty() {
return Ok(());
}
match chunk.iter().position(|&b| b == b'\n') {
Some(i) => {
input.consume(i + 1);
return Ok(());
}
None => {
let n = chunk.len();
input.consume(n);
}
}
}
}
fn error(id: Value, code: i64, message: impl Into<String>) -> Value {
json!({"jsonrpc": "2.0", "id": id, "error": {"code": code, "message": message.into()}})
}
@@ -68,14 +107,13 @@ impl Server {
pub fn serve(&mut self, mut input: impl BufRead, mut output: impl Write) -> std::io::Result<()> {
let mut buf = Vec::new();
loop {
buf.clear();
let n = input.read_until(b'\n', &mut buf)?;
if n == 0 {
return Ok(());
}
let reply = match std::str::from_utf8(&buf) {
Ok(line) => self.handle_line(line),
Err(_) => Some(error(Value::Null, PARSE_ERROR, "message is not valid UTF-8").to_string()),
let reply = match read_message(&mut input, &mut buf)? {
None => return Ok(()),
Some(false) => Some(error(Value::Null, INVALID_REQUEST, "message too large (16 MB max)").to_string()),
Some(true) => match std::str::from_utf8(&buf) {
Ok(line) => self.handle_line(line),
Err(_) => Some(error(Value::Null, PARSE_ERROR, "message is not valid UTF-8").to_string()),
},
};
if let Some(reply) = reply {
output.write_all(reply.as_bytes())?;
+65
View File
@@ -359,6 +359,71 @@ fn serve_loop_over_buffers() {
assert!(lines[2]["result"]["content"][0]["text"].as_str().unwrap().contains("42"));
}
/// Streams `len` bytes of `a` from one reused buffer, then `tail`: a huge line costs no memory to
/// produce, so only the server can make it expensive.
struct HugeLine {
left: u64,
tail: &'static [u8],
chunk: Vec<u8>,
}
impl std::io::Read for HugeLine {
fn read(&mut self, out: &mut [u8]) -> std::io::Result<usize> {
let buf = std::io::BufRead::fill_buf(self)?;
let n = buf.len().min(out.len());
out[..n].copy_from_slice(&buf[..n]);
std::io::BufRead::consume(self, n);
Ok(n)
}
}
impl std::io::BufRead for HugeLine {
fn fill_buf(&mut self) -> std::io::Result<&[u8]> {
if self.left > 0 {
let n = self.chunk.len().min(usize::try_from(self.left).unwrap_or(usize::MAX));
return Ok(&self.chunk[..n]);
}
Ok(self.tail)
}
fn consume(&mut self, n: usize) {
if self.left > 0 {
self.left -= n as u64;
} else {
self.tail = &self.tail[n..];
}
}
}
#[test]
fn a_line_without_newline_keeps_at_most_the_limit_in_memory() {
// #2164: the whole line was buffered before its size was checked (4 GiB here; any size a
// client cares to send).
let mut input =
HugeLine { left: 4 << 30, tail: concat!("\n", r#"{"jsonrpc":"2.0","id":7,"method":"ping"}"#, "\n").as_bytes(), chunk: vec![b'a'; 1 << 20] };
let mut buf = Vec::new();
assert_eq!(crate::server::read_message(&mut input, &mut buf).unwrap(), Some(false));
assert!(buf.capacity() <= 2 * (16 << 20) + 1, "kept {} bytes", buf.capacity());
assert_eq!(crate::server::read_message(&mut input, &mut buf).unwrap(), Some(true));
assert!(buf.starts_with(br#"{"jsonrpc""#));
assert_eq!(crate::server::read_message(&mut input, &mut buf).unwrap(), None);
}
#[test]
fn serve_loop_bounds_a_line_without_newline() {
// #2164: a 4 GiB line was read whole before its size was checked. Now the server keeps at
// most 16 MB of it, answers that it is too large, and reads the next message normally.
let input =
HugeLine { left: 4 << 30, tail: concat!("\n", r#"{"jsonrpc":"2.0","id":7,"method":"ping"}"#, "\n").as_bytes(), chunk: vec![b'a'; 1 << 20] };
let mut out = Vec::new();
server().serve(input, &mut out).unwrap();
let lines: Vec<Value> = String::from_utf8(out).unwrap().lines().map(|l| serde_json::from_str(l).unwrap()).collect();
assert_eq!(lines.len(), 2);
assert_eq!(lines[0]["error"]["code"], -32600);
assert!(lines[0]["error"]["message"].as_str().unwrap().contains("too large"));
assert_eq!(lines[1]["id"], 7);
assert!(lines[1].get("result").is_some(), "{}", lines[1]);
}
#[test]
fn cell_set_without_input_is_error() {
let mut s = server();
+24
View File
@@ -230,6 +230,30 @@ pub enum CfRule {
},
}
impl CfRule {
/// The differential style a matching rule applies; colour scales, data bars and icon sets have none.
pub fn style(&self) -> Option<&Style> {
match self {
CfRule::CellIs { style, .. }
| CfRule::Expression { style, .. }
| CfRule::ContainsText { style, .. }
| CfRule::NotContainsText { style, .. }
| CfRule::BeginsWith { style, .. }
| CfRule::EndsWith { style, .. }
| CfRule::Blanks { style }
| CfRule::NoBlanks { style }
| CfRule::Errors { style }
| CfRule::NoErrors { style }
| CfRule::Duplicate { style }
| CfRule::Unique { style }
| CfRule::Top10 { style, .. }
| CfRule::AboveAverage { style, .. }
| CfRule::TimePeriod { style, .. } => Some(style),
CfRule::ColorScale { .. } | CfRule::DataBar { .. } | CfRule::IconSet { .. } => None,
}
}
}
/// Number of icons in an icon set (`3Arrows` → 3, `5Quarters` → 5), always 3 to 5.
pub fn icon_count(set: &str) -> usize {
set.chars().next().and_then(|c| c.to_digit(10)).unwrap_or(3).clamp(3, 5) as usize
+2 -2
View File
@@ -26,8 +26,8 @@ pub use python::{PyCode, PyOutput};
pub use sheet::{DEFAULT_COL_WIDTH, DEFAULT_ROW_HEIGHT, LineIndex, Sheet, SheetUid, SheetViewMode, Visibility, points_to_chars};
pub use store::CellStore;
pub use style::{
Alignment, BorderLine, BorderStyle, Borders, Color, Fill, Font, HAlign, NumFmt, PatternType, Style, StyleId, StyleTable, Theme, Underline,
VAlign, VertAlign,
Alignment, BorderLine, BorderStyle, Borders, Color, Fill, Font, HAlign, NumFmt, PatternType, ReadingOrder, Style, StyleId, StyleTable, Theme,
Underline, VAlign, VertAlign,
};
pub use workbook::{
CalcMode, CalcSettings, DefinedName, DocProps, EQUATION_BUDGET_MESSAGE, MAX_EQUATION_LAYOUT_BYTES, PhoneticProps, PhoneticRun, PhoneticString,
+30
View File
@@ -350,6 +350,14 @@ pub enum VAlign {
Distributed,
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum ReadingOrder {
#[default]
Context,
LeftToRight,
RightToLeft,
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub struct Alignment {
pub h: HAlign,
@@ -359,6 +367,9 @@ pub struct Alignment {
pub indent: u8,
/// Degrees -90..=90, or 255 for vertical stacked text.
pub rotation: i16,
/// Unspecified inherits in differential styles and uses context for ordinary cells.
#[serde(default)]
pub reading_order: Option<ReadingOrder>,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
@@ -488,6 +499,25 @@ static DEFAULT_STYLE: std::sync::LazyLock<Style> = std::sync::LazyLock::new(Styl
mod tests {
use super::*;
#[test]
fn reading_order_defaults_for_older_json_and_distinguishes_styles() {
let align: Alignment = serde_json::from_str(r#"{"h":"General","v":"Bottom","wrap":false,"shrink":false,"indent":0,"rotation":0}"#).unwrap();
assert_eq!(align.reading_order, None);
let mut styles = StyleTable::default();
let context = styles.derive(StyleId::DEFAULT, |s| s.align.reading_order = Some(ReadingOrder::Context));
let ltr = styles.derive(StyleId::DEFAULT, |s| s.align.reading_order = Some(ReadingOrder::LeftToRight));
let rtl = styles.derive(StyleId::DEFAULT, |s| s.align.reading_order = Some(ReadingOrder::RightToLeft));
assert_ne!(ltr, rtl);
assert_ne!(context, StyleId::DEFAULT);
assert_ne!(context, ltr);
assert_ne!(context, rtl);
assert_ne!(ltr, StyleId::DEFAULT);
assert_ne!(rtl, StyleId::DEFAULT);
assert_eq!(styles.derive(ltr, |s| s.align.reading_order = Some(ReadingOrder::RightToLeft)), rtl);
assert_eq!(styles.derive(rtl, |s| s.align.reading_order = Some(ReadingOrder::Context)), context);
assert_eq!(styles.derive(context, |s| s.align.reading_order = None), StyleId::DEFAULT);
}
#[test]
fn interning() {
let mut t = StyleTable::default();
+6
View File
@@ -268,6 +268,11 @@ pub struct Workbook {
/// `preserved` are relative to. Saving into another folder rewrites them from there. Not in JSON.
#[serde(skip)]
pub source_dir: Option<std::path::PathBuf>,
/// The region whose formats this workbook is typed and shown in (Options › Region). A runtime
/// setting of the application, not of the file: it is not saved, and the engine session sets
/// it on every document it holds. en-US until then.
#[serde(skip, default = "gridcraft_core::region::default_region")]
pub region: &'static gridcraft_core::Region,
/// The file this workbook reports as its own, which is what `CELL("filename")` returns.
/// `None` = no file yet, which Excel reports as an empty string until the workbook has been
/// saved once. Deliberately separate from `source_dir`: that one names the folder relative
@@ -365,6 +370,7 @@ impl Workbook {
macros_unread: Arc::default(),
preserved: None,
source_dir: None,
region: gridcraft_core::region::default_region(),
source_path: None,
}
}
-56
View File
@@ -1,56 +0,0 @@
//! Built-in number formats (ECMA-376 §18.8.30, en-US).
const BUILTINS: [(u32, &str); 36] = [
(0, "General"),
(1, "0"),
(2, "0.00"),
(3, "#,##0"),
(4, "#,##0.00"),
(5, "\"$\"#,##0_);\\(\"$\"#,##0\\)"),
(6, "\"$\"#,##0_);[Red]\\(\"$\"#,##0\\)"),
(7, "\"$\"#,##0.00_);\\(\"$\"#,##0.00\\)"),
(8, "\"$\"#,##0.00_);[Red]\\(\"$\"#,##0.00\\)"),
(9, "0%"),
(10, "0.00%"),
(11, "0.00E+00"),
(12, "# ?/?"),
(13, "# ??/??"),
(14, "m/d/yyyy"),
(15, "d-mmm-yy"),
(16, "d-mmm"),
(17, "mmm-yy"),
(18, "h:mm AM/PM"),
(19, "h:mm:ss AM/PM"),
(20, "h:mm"),
(21, "h:mm:ss"),
(22, "m/d/yyyy h:mm"),
(37, "#,##0 ;(#,##0)"),
(38, "#,##0 ;[Red](#,##0)"),
(39, "#,##0.00;(#,##0.00)"),
(40, "#,##0.00;[Red](#,##0.00)"),
(41, "_(* #,##0_);_(* \\(#,##0\\);_(* \"-\"_);_(@_)"),
(42, "_(\"$\"* #,##0_);_(\"$\"* \\(#,##0\\);_(\"$\"* \"-\"_);_(@_)"),
(43, "_(* #,##0.00_);_(* \\(#,##0.00\\);_(* \"-\"??_);_(@_)"),
(44, "_(\"$\"* #,##0.00_);_(\"$\"* \\(#,##0.00\\);_(\"$\"* \"-\"??_);_(@_)"),
(45, "mm:ss"),
(46, "[h]:mm:ss"),
(47, "mmss.0"),
(48, "##0.0E+0"),
(49, "@"),
];
/// Built-in format code for an id (0..=49, en-US), e.g. 14 => "m/d/yyyy". Locale-specific ids
/// (23–36) and unknown ids return `None`.
pub fn builtin_format(id: u32) -> Option<&'static str> {
BUILTINS.iter().find(|(i, _)| *i == id).map(|(_, c)| *c)
}
/// Id of a built-in format code. Matching ignores case and backslash escapes.
pub fn builtin_id(code: &str) -> Option<u32> {
if let Some((id, _)) = BUILTINS.iter().find(|(_, c)| *c == code) {
return Some(*id);
}
let norm = |s: &str| s.chars().filter(|&c| c != '\\').flat_map(char::to_lowercase).collect::<String>();
let n = norm(code);
BUILTINS.iter().find(|(_, c)| norm(c) == n).map(|(id, _)| *id)
}
+206
View File
@@ -0,0 +1,206 @@
//! Calendars a format section can show dates in: the `B1` (Gregorian) and `B2` (Hijri) prefixes.
use gridcraft_core::date::{DateTime, MONTHS, WEEKDAYS, days_from_civil, days_in_month};
/// The calendar a section's year, month and day parts count in.
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub(crate) enum Calendar {
#[default]
Gregorian,
/// The tabular (arithmetic) Islamic calendar.
Hijri,
}
/// Arabic names of the Hijri months, Muharram first. Two-word names are joined by a no-break
/// space (U+00A0), as Excel writes them.
const HIJRI_MONTHS: [&str; 12] = [
"محرم",
"صفر",
"ربيع\u{a0}الأول",
"ربيع\u{a0}الثاني",
"جمادى\u{a0}الأولى",
"جمادى\u{a0}الثانية",
"رجب",
"شعبان",
"رمضان",
"شوال",
"ذو\u{a0}القعدة",
"ذو\u{a0}الحجة",
];
/// Arabic weekday names, Sunday first, shown for `ddd` and `dddd` under `B2`.
const ARABIC_WEEKDAYS: [&str; 7] = ["الأحد", "الإثنين", "الثلاثاء", "الأربعاء", "الخميس", "الجمعة", "السبت"];
/// Year, month and day of a date in a section's calendar, with the month and weekday names.
pub(crate) struct CalendarDate {
pub year: i64,
pub month: u32,
pub day: u32,
/// `mmmm`.
pub month_name: &'static str,
/// `mmm`.
pub month_short: &'static str,
/// `dddd`.
pub day_name: &'static str,
/// `ddd`.
pub day_short: &'static str,
}
impl Calendar {
/// The date of a day in this calendar, `None` when it has none: under `B2`, the 1900 system's
/// fictitious 29 February 1900 (serial 60, which Excel shows as `#VALUE!`) and days before
/// 1 Muharram 1 AH.
pub(crate) fn date(self, dt: &DateTime) -> Option<CalendarDate> {
match self {
Calendar::Hijri => hijri_date(dt),
Calendar::Gregorian => Some(gregorian_date(dt)),
}
}
}
fn gregorian_date(dt: &DateTime) -> CalendarDate {
let month_name = MONTHS.get((dt.month as usize).wrapping_sub(1)).copied().unwrap_or("");
let day_name = WEEKDAYS.get(dt.weekday as usize).copied().unwrap_or("");
CalendarDate {
year: dt.year.into(),
month: dt.month,
day: dt.day,
month_name,
month_short: month_name.get(..3).unwrap_or(month_name),
day_name,
day_short: day_name.get(..3).unwrap_or(day_name),
}
}
/// Days from 1 Muharram 1 AH to the first of a Hijri month. Leap years (30 days in the last
/// month) are years 2, 5, 7, 10, 13, 15, 18, 21, 24, 26 and 29 of each 30-year cycle (10,631
/// days), which is what Excel shows; odd months have 30 days and even months 29.
fn days_before_month(year: i64, month: i64) -> i64 {
// `(4 + 11 * year) / 30` is the number of leap years before `year`.
(year - 1) * 354 + (4 + 11 * year).div_euclid(30) + 29 * (month - 1) + month / 2
}
/// Hijri year, month and day of the day `days` after 1 Muharram 1 AH (`days >= 0`).
fn hijri_from_days(days: i64) -> Option<(i64, i64, i64)> {
if days < 0 {
return None;
}
// The estimate is the mean year length rounded; it can be one year off at a year boundary.
// Estimate and `days_before_month` both move by 30 years per 10,631 days, so one step is
// enough everywhere if it is within one cycle (`every_day_round_trips` checks many).
let mut year = (30 * days + 10646) / 10631;
if days < days_before_month(year, 1) {
year -= 1;
} else if days >= days_before_month(year + 1, 1) {
year += 1;
}
let month = ((11 * (days - days_before_month(year, 1)) + 330) / 325).clamp(1, 12);
let day = days - days_before_month(year, month) + 1;
Some((year, month, day))
}
/// 1 Muharram 1 AH is 18 July 622 (proleptic Gregorian), so that 1 January 2001 is 6 Shawwal 1421.
fn hijri_date(dt: &DateTime) -> Option<CalendarDate> {
// 29 February 1900 (serial 60) is no real day and has no Hijri date. The 1900 system's day 0
// (`1900-01-00`, serial 0) shows as 1 January 1900, as Excel shows it.
if dt.day > days_in_month(dt.year, dt.month) {
return None;
}
let (year, month, day) = hijri_from_days(days_from_civil(dt.year, dt.month, dt.day.max(1)) - days_from_civil(622, 7, 18))?;
let month_name = HIJRI_MONTHS.get(usize::try_from(month - 1).ok()?).copied()?;
// Excel shows the full Arabic name for `ddd` as well as `dddd`, as it does for `mmm`.
let day_name = ARABIC_WEEKDAYS.get(dt.weekday as usize).copied()?;
Some(CalendarDate {
year,
month: u32::try_from(month).ok()?,
day: u32::try_from(day).ok()?,
month_name,
month_short: month_name,
day_name,
day_short: day_name,
})
}
#[cfg(test)]
mod tests {
use super::*;
use gridcraft_core::DateSystem;
use gridcraft_core::date::{datetime_from_serial, max_serial};
/// Leap years of the 30-year cycle (Excel's set, per the reviewer's 43,512-serial comparison).
const LEAP: [i64; 11] = [2, 5, 7, 10, 13, 15, 18, 21, 24, 26, 29];
fn is_leap(year: i64) -> bool {
LEAP.contains(&((year - 1).rem_euclid(30) + 1))
}
fn month_len(year: i64, month: i64) -> i64 {
if month == 12 { 29 + i64::from(is_leap(year)) } else { 30 - (month + 1) % 2 }
}
/// Years 1 to 10,000 (every cycle up to and past 31 December 9999, which is 9666 AH): year,
/// month and cycle lengths follow the leap-year set.
#[test]
fn year_and_month_lengths_follow_the_leap_set() {
assert_eq!(days_before_month(1, 1), 0);
for year in 1..=10_000 {
let start = days_before_month(year, 1);
let len = days_before_month(year + 1, 1) - start;
assert_eq!(len, if is_leap(year) { 355 } else { 354 }, "year {year}");
for month in 1..=12 {
let next = if month == 12 { days_before_month(year + 1, 1) } else { days_before_month(year, month + 1) };
assert_eq!(next - days_before_month(year, month), month_len(year, month), "{year}-{month}");
}
assert_eq!(days_before_month(year + 30, 1) - start, 10_631, "cycle from {year}");
}
// One cycle: exactly the eleven leap years, 10,631 days.
let cycle: Vec<i64> = (1..=30).filter(|&y| days_before_month(y + 1, 1) - days_before_month(y, 1) == 355).collect();
assert_eq!(cycle, LEAP);
}
/// Every day from 1 Muharram 1 AH to one cycle past 31 December 9999: day -> Hijri -> day
/// round-trips, the day is within its month, and consecutive days follow each other.
#[test]
fn every_day_round_trips() {
let last = days_from_civil(9999, 12, 31) - days_from_civil(622, 7, 18) + 10_631;
let mut prev: Option<(i64, i64, i64)> = None;
for days in 0..=last {
let (y, m, d) = hijri_from_days(days).unwrap();
assert!((1..=12).contains(&m) && (1..=month_len(y, m)).contains(&d), "{days}: {y}-{m}-{d}");
assert_eq!(days_before_month(y, m) + d - 1, days, "{y}-{m}-{d}");
if let Some((py, pm, pd)) = prev {
let expected = if pd < month_len(py, pm) {
(py, pm, pd + 1)
} else if pm < 12 {
(py, pm + 1, 1)
} else {
(py + 1, 1, 1)
};
assert_eq!((y, m, d), expected, "after {py}-{pm}-{pd}");
}
prev = Some((y, m, d));
}
assert_eq!(hijri_from_days(-1), None);
}
/// Every serial of the 1900 system: the Hijri date names the serial's day, one day apart.
#[test]
fn every_serial_has_the_right_hijri_day() {
let epoch = days_from_civil(622, 7, 18);
let mut prev = None;
for serial in 1..=max_serial(DateSystem::D1900) {
if serial == 60 {
continue;
}
let dt = datetime_from_serial(DateSystem::D1900, serial as f64).unwrap();
let h = Calendar::Hijri.date(&dt).unwrap();
assert!((1..=month_len(h.year, i64::from(h.month))).contains(&i64::from(h.day)), "serial {serial}");
let days = days_before_month(h.year, i64::from(h.month)) + i64::from(h.day) - 1;
assert_eq!(days, days_from_civil(dt.year, dt.month, dt.day) - epoch, "serial {serial}");
if let Some(p) = prev {
assert_eq!(days, p + 1, "serial {serial}");
}
prev = Some(days);
}
}
}
+146 -53
View File
@@ -4,6 +4,8 @@
//! digits and then rounded half away from zero on that decimal string. That makes 2.675 with
//! "0.00" show 2.68 and 1.005 show 1.01, as in Excel, even though neither is exact in binary.
use std::fmt::Write as _;
/// Most decimals we ever compute; placeholders beyond this are padded with zeros.
pub(crate) const MAX_DECIMALS: usize = 400;
@@ -21,7 +23,8 @@ pub(crate) fn decimal_digits(a: f64) -> ([u8; 15], i32) {
}
// 16 digits, rounded. Unless the 16th is a 5 it decides the 15th exactly (4 or less rounds
// down, 6 or more up), so one format serves almost every number.
let (d16, exp) = significant_digits(a, 16);
let mut d16 = [0u8; 16];
let exp = significant_digits(a, &mut d16);
for (slot, d) in digits.iter_mut().zip(&d16) {
*slot = *d;
}
@@ -30,11 +33,8 @@ pub(crate) fn decimal_digits(a: f64) -> ([u8; 15], i32) {
5 if is_exact_tie(a) => (digits, exp),
5 => {
// Not a tie: round the exact value (the 16-digit rounding above may be a double rounding).
let (d15, exp15) = significant_digits(a, 15);
let mut out = [0u8; 15];
for (slot, d) in out.iter_mut().zip(&d15) {
*slot = *d;
}
let exp15 = significant_digits(a, &mut out);
(out, exp15)
}
_ => {
@@ -59,12 +59,54 @@ pub(crate) fn decimal_digits(a: f64) -> ([u8; 15], i32) {
}
}
/// `n` significant digits of `a` (rounded half to even on the exact binary value) and the power of
/// ten of the first.
fn significant_digits(a: f64, n: usize) -> (Vec<u8>, i32) {
let s = format!("{:.*e}", n.saturating_sub(1), a);
let (mant, exp) = s.split_once('e').unwrap_or((s.as_str(), "0"));
(mant.bytes().filter(u8::is_ascii_digit).map(|b| b - b'0').collect(), exp.parse().unwrap_or(0))
/// Writes `out.len()` significant digits of `a` (rounded half to even on the exact binary value)
/// into `out` and returns the power of ten of the first.
///
/// Up to 32 digits are formatted in a stack buffer; more (the 800 of the tie check) use a string.
fn significant_digits(a: f64, out: &mut [u8]) -> i32 {
fn parse(s: &str, out: &mut [u8]) -> i32 {
let (mant, exp) = s.split_once('e').unwrap_or((s, "0"));
let digits = mant.bytes().filter(u8::is_ascii_digit).map(|b| b - b'0');
for (slot, d) in out.iter_mut().zip(digits) {
*slot = d;
}
exp.parse().unwrap_or(0)
}
let prec = out.len().saturating_sub(1);
if out.len() <= 32 {
let mut buf = StackStr::default();
if write!(buf, "{:.*e}", prec, a).is_ok() {
return parse(buf.as_str(), out);
}
}
parse(&format!("{:.*e}", prec, a), out)
}
/// A fixed-size string buffer for short number formatting; a write that does not fit fails.
struct StackStr {
buf: [u8; 64],
len: usize,
}
impl Default for StackStr {
fn default() -> Self {
StackStr { buf: [0; 64], len: 0 }
}
}
impl StackStr {
fn as_str(&self) -> &str {
self.buf.get(..self.len).and_then(|b| std::str::from_utf8(b).ok()).unwrap_or("")
}
}
impl std::fmt::Write for StackStr {
fn write_str(&mut self, s: &str) -> std::fmt::Result {
let end = self.len.checked_add(s.len()).ok_or(std::fmt::Error)?;
self.buf.get_mut(self.len..end).ok_or(std::fmt::Error)?.copy_from_slice(s.as_bytes());
self.len = end;
Ok(())
}
}
/// True when `a`'s exact decimal expansion has a 5 as its 16th significant digit and only zeros after.
@@ -72,27 +114,59 @@ fn is_exact_tie(a: f64) -> bool {
let is_tie = |ds: &[u8]| ds.get(15) == Some(&5) && ds.get(16..).is_some_and(|t| t.iter().all(|&d| d == 0));
// A cheap screen first (30 digits), then the exact expansion: a double has at most 767
// significant digits and Rust prints every one of them.
is_tie(&significant_digits(a, 30).0) && is_tie(&significant_digits(a, 800).0)
let mut screen = [0u8; 30];
significant_digits(a, &mut screen);
if !is_tie(&screen) {
return false;
}
let mut exact = vec![0u8; 800];
significant_digits(a, &mut exact);
is_tie(&exact)
}
/// Rounds the number `digits × 10^exp` (see [`decimal_digits`]) to `decimals` places, half away
/// from zero. Returns the integer digits ("" when the integer part is zero) and exactly
/// `decimals` fraction digits.
pub(crate) fn fixed_from_digits(digits: &[u8; 15], exp: i32, decimals: usize) -> (String, String) {
let (mut int, mut frac) = (String::new(), String::new());
fixed_into(digits, exp, decimals, &mut int, &mut frac);
(int, frac)
}
/// [`fixed_from_digits`] into caller-owned strings (cleared first), so a loop can reuse them.
///
/// The rounded digits are `lead` (at most 16, a carry can add a leading 1) followed by `zeros`
/// zero digits (placeholders beyond the 15 known digits), which is all a `Vec` of them held.
fn fixed_into(digits: &[u8; 15], exp: i32, decimals: usize, int: &mut String, frac: &mut String) {
int.clear();
frac.clear();
let decimals_calc = decimals.min(MAX_DECIMALS);
let keep = exp as i64 + decimals_calc as i64 + 1;
let mut m: Vec<u8> = if keep <= 0 {
if keep == 0 && digits.first().copied().unwrap_or(0) >= 5 { vec![1] } else { Vec::new() }
let mut head = [0u8; 16];
let mut head_len = 0usize;
let mut zeros = 0usize;
if keep <= 0 {
if keep == 0 && digits.first().copied().unwrap_or(0) >= 5 {
if let Some(h) = head.first_mut() {
*h = 1;
}
head_len = 1;
}
} else if keep as usize >= digits.len() {
let mut v = digits.to_vec();
v.resize(keep as usize, 0);
v
for (h, d) in head.iter_mut().zip(digits) {
*h = *d;
}
head_len = digits.len();
zeros = keep as usize - digits.len();
} else {
let keep = keep as usize;
let mut v = digits.get(..keep).map(<[u8]>::to_vec).unwrap_or_default();
for (h, d) in head.iter_mut().zip(digits.iter().take(keep)) {
*h = *d;
}
head_len = keep;
if digits.get(keep).copied().unwrap_or(0) >= 5 {
let mut carry = true;
for d in v.iter_mut().rev() {
for d in head.iter_mut().take(keep).rev() {
if *d == 9 {
*d = 0;
} else {
@@ -102,26 +176,36 @@ pub(crate) fn fixed_from_digits(digits: &[u8; 15], exp: i32, decimals: usize) ->
}
}
if carry {
v.insert(0, 1);
head.copy_within(0..keep, 1);
if let Some(h) = head.first_mut() {
*h = 1;
}
head_len += 1;
}
}
v
};
let lead = m.iter().take_while(|&&d| d == 0).count();
m.drain(..lead);
let to_s = |ds: &[u8]| ds.iter().map(|d| char::from(b'0' + d)).collect::<String>();
let (int, mut frac) = if m.len() <= decimals_calc {
let mut f = "0".repeat(decimals_calc - m.len());
f.push_str(&to_s(&m));
(String::new(), f)
} else {
let split = m.len() - decimals_calc;
(to_s(m.get(..split).unwrap_or(&[])), to_s(m.get(split..).unwrap_or(&[])))
};
if decimals > decimals_calc {
frac.push_str(&"0".repeat(decimals - decimals_calc));
}
(int, frac)
let total = head_len + zeros;
let mut lead = head.iter().take(head_len).take_while(|&&d| d == 0).count();
if lead == head_len {
lead = total; // all zeros
}
let len = total - lead;
// Digit `k` of the rounded number after its leading zeros.
let digit_at = |k: usize| {
let pos = lead + k;
if pos < head_len { head.get(pos).map_or('0', |&d| char::from(b'0' + d)) } else { '0' }
};
if len <= decimals_calc {
frac.extend(std::iter::repeat_n('0', decimals_calc - len));
frac.extend((0..len).map(digit_at));
} else {
let split = len - decimals_calc;
int.extend((0..split).map(digit_at));
frac.extend((split..len).map(digit_at));
}
if decimals > decimals_calc {
frac.extend(std::iter::repeat_n('0', decimals - decimals_calc));
}
}
/// Rounds a non-negative number to `decimals` places (see [`fixed_from_digits`]).
@@ -148,21 +232,25 @@ pub fn format_general_fit(n: f64, max_chars: usize) -> Option<String> {
let sign = if n < 0.0 { "-" } else { "" };
let (digits, x) = decimal_digits(n.abs());
let int_len = sign.len() + (x.max(0) as usize) + 1;
// Fixed notation for magnitudes from 1E-4 up to what fits.
// Fixed notation for magnitudes from 1E-4 up to what fits. The 15 digits are computed once;
// each width only re-rounds them, into strings that are reused.
let (mut i, mut f, mut s) = (String::new(), String::new(), String::new());
if x >= -4 && int_len <= max_chars {
let avail = max_chars - int_len;
let mut decimals = if avail >= 2 { avail - 1 } else { 0 };
// Zeros past the 400th decimal are trimmed below anyway, so a huge width needs no more.
let mut decimals = (if avail >= 2 { avail - 1 } else { 0 }).min(MAX_DECIMALS);
loop {
let (i, f) = fixed_from_digits(&digits, x, decimals);
let f = f.trim_end_matches('0');
if i.is_empty() && f.is_empty() {
fixed_into(&digits, x, decimals, &mut i, &mut f);
let ft = f.trim_end_matches('0');
if i.is_empty() && ft.is_empty() {
break; // rounded away to zero: use scientific
}
let mut s = String::from(sign);
s.clear();
s.push_str(sign);
s.push_str(if i.is_empty() { "0" } else { &i });
if !f.is_empty() {
if !ft.is_empty() {
s.push('.');
s.push_str(f);
s.push_str(ft);
}
if s.chars().count() <= max_chars {
return Some(s);
@@ -175,19 +263,20 @@ pub fn format_general_fit(n: f64, max_chars: usize) -> Option<String> {
}
for md in (0..=max_chars.min(14)).rev() {
let mut e = x;
let (mut mi, mut mf) = fixed_from_digits(&digits, 0, md);
if mi.len() > 1 {
fixed_into(&digits, 0, md, &mut i, &mut f);
if i.len() > 1 {
e += 1;
(mi, mf) = fixed_from_digits(&digits, -1, md);
fixed_into(&digits, -1, md, &mut i, &mut f);
}
let mf = mf.trim_end_matches('0');
let mut s = String::from(sign);
s.push_str(if mi.is_empty() { "0" } else { &mi });
if !mf.is_empty() {
let ft = f.trim_end_matches('0');
s.clear();
s.push_str(sign);
s.push_str(if i.is_empty() { "0" } else { &i });
if !ft.is_empty() {
s.push('.');
s.push_str(mf);
s.push_str(ft);
}
s.push_str(&format!("E{}{:02}", if e < 0 { '-' } else { '+' }, e.unsigned_abs()));
let _ = write!(s, "E{}{:02}", if e < 0 { '-' } else { '+' }, e.unsigned_abs());
if s.chars().count() <= max_chars {
return Some(s);
}
@@ -211,6 +300,10 @@ mod tests {
assert_eq!(fixed(0.0005, 3), ("".into(), "001".into()));
assert_eq!(fixed(1e20, 0).0, "100000000000000000000");
assert_eq!(fixed(0.0, 2), ("".into(), "00".into()));
// Past 400 decimals only zeros are added; huge exponents keep all their digits.
assert_eq!(fixed(1.5, 450), ("1".into(), format!("5{}", "0".repeat(449))));
assert_eq!(fixed(1e300, 1), (format!("1{}", "0".repeat(300)), "0".into()));
assert_eq!(fixed(123456789012345678.0, 0).0, "123456789012346000");
}
#[test]
+32 -8
View File
@@ -3,7 +3,7 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
#![forbid(unsafe_code)]
mod builtin;
mod calendar;
mod decimal;
mod parse;
mod render;
@@ -12,7 +12,6 @@ use std::sync::Arc;
use gridcraft_core::{CellError, DateSystem, Value};
pub use builtin::{builtin_format, builtin_id};
pub use decimal::format_general_fit;
use parse::{DatePart, SecKind, Section, Tok};
@@ -96,6 +95,9 @@ struct Inner {
num_sections: usize,
/// Index of the section that applies to text, if any.
text_section: Option<usize>,
/// A code Excel rejects (a misplaced `B1`/`B2` calendar prefix): shown as General, and
/// `TEXT` returns `#VALUE!`, as for a code that is too long.
invalid: bool,
}
/// A parsed number format code. Cheap to clone.
@@ -130,7 +132,8 @@ fn too_long(code: &str) -> bool {
impl NumberFormat {
/// Parses a format code. Never fails: unknown or malformed parts degrade to literal text, and
/// a code longer than [`MAX_FORMAT_CODE_CHARS`] characters degrades to General.
/// a code longer than [`MAX_FORMAT_CODE_CHARS`] characters, or one Excel rejects for a
/// misplaced `B1`/`B2` calendar prefix, degrades to General.
pub fn parse(code: &str) -> NumberFormat {
Self::parse_inner(code, false)
}
@@ -147,12 +150,16 @@ impl NumberFormat {
let trimmed = code.trim();
// `checkbox` marks a checkbox cell (Insert › Checkbox): TRUE/FALSE draw as a box, any other
// value displays as General.
let general = |invalid: bool| NumberFormat {
inner: Arc::new(Inner { code: code.to_string(), general: true, sections: Vec::new(), num_sections: 0, text_section: None, invalid }),
};
if trimmed.is_empty() || trimmed.eq_ignore_ascii_case("general") || trimmed == "checkbox" || (!uncapped && too_long(code)) {
return NumberFormat {
inner: Arc::new(Inner { code: code.to_string(), general: true, sections: Vec::new(), num_sections: 0, text_section: None }),
};
return general(false);
}
let sections = parse::parse_sections(code);
if sections.iter().any(|s| s.invalid) {
return general(true);
}
let len = sections.len();
let last_is_text = sections.last().is_some_and(|s| s.kind == SecKind::Text);
let text_section = if len >= 4 {
@@ -169,7 +176,7 @@ impl NumberFormat {
} else {
len
};
NumberFormat { inner: Arc::new(Inner { code: code.to_string(), general: false, sections, num_sections, text_section }) }
NumberFormat { inner: Arc::new(Inner { code: code.to_string(), general: false, sections, num_sections, text_section, invalid: false }) }
}
pub fn general() -> NumberFormat {
@@ -386,6 +393,14 @@ impl NumberFormat {
None => Formatted { text: t.to_string(), color: None, fill: None, numeric: false },
}
}
/// [`Self::format_text`]'s text, or `None` once it passes `max_units` UTF-16 code units.
fn format_text_within(&self, t: &str, max_units: usize) -> Option<String> {
match self.inner.text_section.and_then(|i| self.inner.sections.get(i)) {
Some(sec) => render::render_text_within(sec, t, max_units).map(|out| out.text),
None => (t.encode_utf16().count() <= max_units).then(|| t.to_string()),
}
}
}
/// Formats a cell value for display.
@@ -479,6 +494,10 @@ impl NumberFormat {
/// The `TEXT()` worksheet function: formats a value with a format code. Numeric text is
/// converted to a number first; other text and booleans (as `TRUE`/`FALSE`) go through the text
/// section, or are returned as is.
/// Longest text a cell holds, in UTF-16 code units (Excel's 32,767 characters, which XLSB also
/// stores that way), and so the longest `TEXT` result.
const MAX_CELL_TEXT_CHARS: usize = 32_767;
pub fn text_function(v: &Value, code: &str, sys: DateSystem) -> Result<String, CellError> {
if code.is_empty() {
return Ok(String::new());
@@ -487,6 +506,9 @@ pub fn text_function(v: &Value, code: &str, sys: DateSystem) -> Result<String, C
return Err(CellError::Value);
}
let fmt = NumberFormat::parse_cached(code);
if fmt.inner.invalid {
return Err(CellError::Value);
}
let n = match v.scalar() {
Value::Error(e) => return Err(e),
Value::Empty => 0.0,
@@ -494,7 +516,9 @@ pub fn text_function(v: &Value, code: &str, sys: DateSystem) -> Result<String, C
Value::Bool(b) => return Ok(fmt.format_text(if b { "TRUE" } else { "FALSE" }).text),
Value::Text(t) => match gridcraft_core::parse::parse_number_text(&t) {
Some(n) => n,
None => return Ok(fmt.format_text(&t).text),
// Every `@` repeats the text: a result past a cell's limit is `#VALUE!`, so nested
// TEXT calls can't multiply it further.
None => return fmt.format_text_within(&t, MAX_CELL_TEXT_CHARS).ok_or(CellError::Value),
},
Value::Array(_) => return Err(CellError::Value),
};
+100 -12
View File
@@ -1,6 +1,7 @@
//! Format code parsing: sections, bracket tags and tokens.
use crate::FormatColor;
use crate::calendar::Calendar;
/// At most this many sections are honoured (`pos;neg;zero;text`); extra ones are ignored.
const MAX_SECTIONS: usize = 4;
@@ -24,6 +25,11 @@ pub(crate) enum DatePart {
}
impl DatePart {
/// `[h]`, `[m]` or `[s]`.
pub(crate) fn is_elapsed(self) -> bool {
matches!(self, DatePart::ElapsedH(_) | DatePart::ElapsedM(_) | DatePart::ElapsedS(_))
}
pub(crate) fn is_calendar(self) -> bool {
matches!(self, DatePart::Year(_) | DatePart::Month(_) | DatePart::Day(_))
}
@@ -54,18 +60,28 @@ pub(crate) enum Tok {
}
impl Tok {
/// Appends the text this token shows when it has no special meaning in its section.
pub(crate) fn push_literal(&self, out: &mut String) {
match self {
Tok::Lit(s) | Tok::FixedDenom(s) => out.push_str(s),
Tok::Digit(c) => out.push(*c),
Tok::Point => out.push('.'),
Tok::Comma => out.push(','),
Tok::Percent => out.push('%'),
Tok::Exp { upper, plus } => {
out.push(if *upper { 'E' } else { 'e' });
out.push(if *plus { '+' } else { '-' });
}
Tok::Slash => out.push('/'),
Tok::Fill(_) | Tok::At | Tok::General | Tok::Date(_) => {}
}
}
/// The text this token shows when it has no special meaning in its section.
pub(crate) fn literal_text(&self) -> String {
match self {
Tok::Lit(s) | Tok::FixedDenom(s) => s.clone(),
Tok::Digit(c) => c.to_string(),
Tok::Point => ".".into(),
Tok::Comma => ",".into(),
Tok::Percent => "%".into(),
Tok::Exp { upper, plus } => format!("{}{}", if *upper { 'E' } else { 'e' }, if *plus { '+' } else { '-' }),
Tok::Slash => "/".into(),
Tok::Fill(_) | Tok::At | Tok::General | Tok::Date(_) => String::new(),
}
let mut s = String::new();
self.push_literal(&mut s);
s
}
}
@@ -122,6 +138,17 @@ pub(crate) struct Section {
pub fraction: bool,
pub currency: bool,
pub fill: bool,
/// Date sections: the most digits of any `.0`/`.00`/`.000` part (at most 3), which decides how
/// the time of day is rounded.
pub sub_digits: u8,
/// Date sections: has a year, month or day part, so the serial's date is needed.
pub needs_date: bool,
/// Date sections: has `AM/PM` or `A/P`, so hours show on the 12-hour clock.
pub ampm: bool,
pub calendar: Calendar,
/// Excel rejects the whole code because of this section: a `B` and digit that is not a
/// calendar prefix, or a prefix before a number format (`B20.00`).
pub invalid: bool,
}
/// Splits a code into sections at unquoted, unescaped, unbracketed semicolons.
@@ -251,17 +278,60 @@ struct Raw {
color: Option<FormatColor>,
cond: Option<Cond>,
currency: bool,
calendar: Calendar,
/// The section's calendar prefix as written, `'1'` or `'2'` (whether or not a locale tag
/// cancels it).
prefix: Option<char>,
/// An unquoted `B` and digit that is not a calendar prefix: Excel rejects the code.
invalid: bool,
}
/// Where a section's body starts: after its leading `[...]` tags, read as `tokenize` reads them.
/// Also whether one of those tags is a `[$...]` locale tag.
fn after_leading_brackets(cs: &[char]) -> (usize, bool) {
let mut i = 0usize;
let mut locale = false;
while cs.get(i) == Some(&'[') {
match cs.get(i + 1..).and_then(|r| r.iter().position(|&d| d == ']')) {
Some(p) => {
locale |= cs.get(i + 1) == Some(&'$');
i += p + 2;
}
None => break,
}
}
(i, locale)
}
fn tokenize(src: &str) -> Raw {
let cs: Vec<char> = src.chars().collect();
let mut raw = Raw { toks: Vec::new(), color: None, cond: None, currency: false };
let mut raw = Raw { toks: Vec::new(), color: None, cond: None, currency: false, calendar: Calendar::Gregorian, prefix: None, invalid: false };
let toks = &mut raw.toks;
// With no `]` at or after a `[`, that `[` is a literal; checking this once keeps the scan linear.
let last_close = cs.iter().rposition(|&c| c == ']');
let (body_start, locale_first) = after_leading_brackets(&cs);
// Where the next calendar prefix may stand: the body's start, or right after another prefix.
let mut prefix_at = body_start;
let mut i = 0usize;
while let Some(&c) = cs.get(i) {
match c {
// A calendar prefix, never shown: `B1` or `B2`, any case, which Excel accepts repeated
// (`B2B2`) but not mixed (`B2B1`) or with more digits (`B12`, `B02`).
'B' | 'b' if i == prefix_at && cs.get(i + 1).is_some_and(char::is_ascii_digit) => {
let d = cs.get(i + 1).copied();
let ok = matches!(d, Some('1' | '2')) && !cs.get(i + 2).is_some_and(char::is_ascii_digit) && raw.prefix.is_none_or(|p| Some(p) == d);
raw.invalid |= !ok;
raw.prefix = d;
i += 2;
prefix_at = i;
continue;
}
// Any other `B` and digit (`B3dd`, `dd/mm/yyyyB2`, `dd B2 mm`, `B2[Red]B2`) makes Excel
// reject the code.
'B' | 'b' if cs.get(i + 1).is_some_and(char::is_ascii_digit) => {
raw.invalid = true;
push_lit(toks, c.encode_utf8(&mut [0; 4]));
}
'"' => {
let mut s = String::new();
i += 1;
@@ -283,6 +353,8 @@ fn tokenize(src: &str) -> Raw {
if is_currency_char(d) {
raw.currency = true;
}
// Excel rejects an escaped `\B2` as well.
raw.invalid |= matches!(d, 'B' | 'b') && cs.get(i + 2).is_some_and(char::is_ascii_digit);
push_lit(toks, d.encode_utf8(&mut [0; 4]));
}
i += 2;
@@ -383,6 +455,11 @@ fn tokenize(src: &str) -> Raw {
}
i += 1;
}
// Excel keeps the date Gregorian when a `[$-401]`-style locale or currency tag comes before
// `B2` (`B2[$-401]` is still Hijri).
if raw.prefix == Some('2') && !locale_first {
raw.calendar = Calendar::Hijri;
}
raw
}
@@ -438,7 +515,7 @@ fn date_part(t: &Tok) -> Option<DatePart> {
}
fn analyze(raw: Raw) -> Section {
let Raw { toks, color, cond, currency } = raw;
let Raw { toks, color, cond, currency, calendar, prefix, invalid } = raw;
let is_date = toks.iter().any(|t| matches!(t, Tok::Date(_)));
let has_general = toks.iter().any(|t| matches!(t, Tok::General));
let mut sec = Section {
@@ -455,6 +532,11 @@ fn analyze(raw: Raw) -> Section {
fraction: false,
currency,
fill: toks.iter().any(|t| matches!(t, Tok::Fill(_))),
sub_digits: 0,
needs_date: false,
ampm: false,
calendar,
invalid,
};
if is_date {
sec.kind = SecKind::Date;
@@ -469,6 +551,9 @@ fn analyze(raw: Raw) -> Section {
sec.kind = SecKind::Text;
return analyze_text(toks, sec);
}
// Number section. Excel rejects a calendar prefix before digits (`B20.00`, `B1#,##0`);
// `B2General`, `B2@` and `B2"x"` show as without the prefix.
sec.invalid |= prefix.is_some() && sec.has_digits;
analyze_number(toks, sec)
}
@@ -508,6 +593,9 @@ fn analyze_date(mut toks: Vec<Tok>, mut sec: Section) -> Section {
}
i += 1;
}
sec.sub_digits = out.iter().filter_map(|t| if let Tok::Date(DatePart::SubSec(n)) = t { Some(*n) } else { None }).max().unwrap_or(0).min(3);
sec.needs_date = out.iter().any(|t| matches!(t, Tok::Date(d) if d.is_calendar()));
sec.ampm = out.iter().any(|t| matches!(t, Tok::Date(DatePart::AmPm | DatePart::AP(..))));
sec.toks = out;
sec
}
+243 -134
View File
@@ -1,8 +1,11 @@
//! Rendering a value with a parsed section.
use gridcraft_core::DateSystem;
use gridcraft_core::date::{MONTHS, WEEKDAYS, datetime_from_serial, max_serial};
use std::fmt::Write as _;
use gridcraft_core::DateSystem;
use gridcraft_core::date::{datetime_from_serial, max_serial};
use crate::calendar::Calendar;
use crate::decimal::{decimal_digits, fixed, fixed_from_digits, format_general_fit};
use crate::parse::{DatePart, Section, Tok};
@@ -24,6 +27,10 @@ impl Out {
self.fill = Some((c, self.text.len()));
}
}
/// Appends what `t` shows when it has no special meaning (no temporary string).
fn push_tok(&mut self, t: &Tok) {
t.push_literal(&mut self.text);
}
pub(crate) fn prepend(&mut self, s: &str) {
self.text.insert_str(0, s);
if let Some((_, p)) = &mut self.fill {
@@ -39,12 +46,41 @@ pub(crate) fn render_text(sec: &Section, text: &str) -> Out {
match t {
Tok::At => out.push(text),
Tok::Fill(c) => out.fill(*c),
other => out.push(&other.literal_text()),
other => out.push_tok(other),
}
}
out
}
/// [`render_text`], but `None` once the output passes `max_units` UTF-16 code units (a cell's
/// limit), checked before each piece is appended, so an oversized result is never built.
pub(crate) fn render_text_within(sec: &Section, text: &str, max_units: usize) -> Option<Out> {
let text_units = text.encode_utf16().count();
let mut out = Out::default();
let mut units = 0usize;
for t in &sec.toks {
match t {
Tok::At => {
units = units.saturating_add(text_units);
if units > max_units {
return None;
}
out.push(text);
}
Tok::Fill(c) => out.fill(*c),
other => {
let literal = other.literal_text();
units = units.saturating_add(literal.encode_utf16().count());
if units > max_units {
return None;
}
out.push(&literal);
}
}
}
Some(out)
}
/// Renders a number through a General section (literals around the General keyword).
pub(crate) fn render_general(sec: &Section, v: f64, width: usize) -> Out {
let mut out = Out::default();
@@ -52,7 +88,7 @@ pub(crate) fn render_general(sec: &Section, v: f64, width: usize) -> Out {
match t {
Tok::General => out.push(&format_general_fit(v, width).unwrap_or_else(|| "#".repeat(width.clamp(1, 11)))),
Tok::Fill(c) => out.fill(*c),
other => out.push(&other.literal_text()),
other => out.push_tok(other),
}
}
out
@@ -63,8 +99,7 @@ pub(crate) fn render_date(sec: &Section, v: f64, sys: DateSystem) -> Option<Out>
if !v.is_finite() || !(0.0..max_serial(sys) as f64 + 1.0).contains(&v) {
return None;
}
let sub_digits = sec.toks.iter().filter_map(|t| if let Tok::Date(DatePart::SubSec(n)) = t { Some(*n) } else { None }).max().unwrap_or(0).min(3);
let unit: i64 = 10i64.pow(3 - sub_digits as u32);
let unit: i64 = 10i64.pow(3 - u32::from(sec.sub_digits.min(3)));
let mut day = v.floor() as i64;
let mut ms = (((v - v.floor()) * 86_400_000.0) / unit as f64).round() as i64 * unit;
if ms >= 86_400_000 {
@@ -72,38 +107,42 @@ pub(crate) fn render_date(sec: &Section, v: f64, sys: DateSystem) -> Option<Out>
day += 1;
}
let total_ms = day.saturating_mul(86_400_000).saturating_add(ms);
let needs_date = sec.toks.iter().any(|t| matches!(t, Tok::Date(d) if d.is_calendar()));
let dt = if needs_date { datetime_from_serial(sys, day as f64)? } else { datetime_from_serial(sys, 0.0).unwrap_or_default() };
let ampm = sec.toks.iter().any(|t| matches!(t, Tok::Date(DatePart::AmPm | DatePart::AP(..))));
// Under `B2`, Excel shows the 1900 system's fictitious 29 February (serial 60) as `#VALUE!`
// whenever a clock or calendar part is shown, even for `hh:mm` (`[h]` alone still works).
if sec.calendar == Calendar::Hijri
&& sys == DateSystem::D1900
&& day == 60
&& sec.toks.iter().any(|t| matches!(t, Tok::Date(d) if !d.is_elapsed()))
{
return None;
}
let dt = if sec.needs_date { datetime_from_serial(sys, day as f64)? } else { datetime_from_serial(sys, 0.0).unwrap_or_default() };
let ampm = sec.ampm;
let hour = ms / 3_600_000;
let minute = (ms / 60_000) % 60;
let second = (ms / 1000) % 60;
let milli = ms % 1000;
let pad = |n: i64, w: u8| if w >= 2 { format!("{:0w$}", n, w = w.min(20) as usize) } else { n.to_string() };
let month_name = MONTHS.get((dt.month as usize).wrapping_sub(1)).copied().unwrap_or("");
let day_name = WEEKDAYS.get(dt.weekday as usize).copied().unwrap_or("");
// No date in the section's calendar (serial 60 under `B2`): shown like an out-of-range date.
let date = sec.calendar.date(&dt)?;
let mut out = Out::default();
for t in &sec.toks {
match t {
Tok::Date(d) => {
let s = match *d {
let text = &mut out.text;
match *d {
DatePart::Year(n) => {
if n <= 2 {
format!("{:02}", dt.year.rem_euclid(100))
} else {
format!("{:04}", dt.year)
}
let _ = if n <= 2 { write!(text, "{:02}", date.year.rem_euclid(100)) } else { write!(text, "{:04}", date.year) };
}
DatePart::Month(n) => match n {
1 | 2 => pad(dt.month as i64, n),
3 => month_name.chars().take(3).collect(),
5 => month_name.chars().take(1).collect(),
_ => month_name.to_string(),
1 | 2 => push_padded(text, date.month as i64, n),
3 => text.push_str(date.month_short),
5 => text.extend(date.month_name.chars().take(1)),
_ => text.push_str(date.month_name),
},
DatePart::Day(n) => match n {
1 | 2 => pad(dt.day as i64, n),
3 => day_name.chars().take(3).collect(),
_ => day_name.to_string(),
1 | 2 => push_padded(text, date.day as i64, n),
3 => text.push_str(date.day_short),
_ => text.push_str(date.day_name),
},
DatePart::Hour(n) => {
let h = if ampm {
@@ -112,78 +151,168 @@ pub(crate) fn render_date(sec: &Section, v: f64, sys: DateSystem) -> Option<Out>
} else {
hour
};
pad(h, n)
push_padded(text, h, n);
}
DatePart::Minute(n) => pad(minute, n),
DatePart::Second(n) => pad(second, n),
DatePart::Minute(n) => push_padded(text, minute, n),
DatePart::Second(n) => push_padded(text, second, n),
DatePart::SubSec(n) => {
let mut s = format!(".{:03}", milli);
s.truncate(1 + (n as usize).min(3));
if n > 3 {
s.push_str(&"0".repeat(n as usize - 3));
// `.` and the first `n` (at most 3) of the three millisecond digits, then
// zeros for the digits beyond milliseconds.
text.push('.');
for (k, div) in [100, 10, 1].into_iter().enumerate() {
if k < n as usize {
text.push(char::from(b'0' + (milli / div % 10) as u8));
}
}
for _ in 3..n {
text.push('0');
}
s
}
DatePart::ElapsedH(n) => pad(total_ms / 3_600_000, n),
DatePart::ElapsedM(n) => pad(total_ms / 60_000, n),
DatePart::ElapsedS(n) => pad(total_ms / 1000, n),
DatePart::AmPm => (if hour >= 12 { "PM" } else { "AM" }).to_string(),
DatePart::AP(a, p) => (if hour >= 12 { p } else { a }).to_string(),
};
out.push(&s);
DatePart::ElapsedH(n) => push_padded(text, total_ms / 3_600_000, n),
DatePart::ElapsedM(n) => push_padded(text, total_ms / 60_000, n),
DatePart::ElapsedS(n) => push_padded(text, total_ms / 1000, n),
DatePart::AmPm => text.push_str(if hour >= 12 { "PM" } else { "AM" }),
DatePart::AP(a, p) => text.push(if hour >= 12 { p } else { a }),
}
}
Tok::Fill(c) => out.fill(*c),
other => out.push(&other.literal_text()),
other => out.push_tok(other),
}
}
Some(out)
}
/// Appends `n`, zero padded to `w` digits (at most 20) when `w` is 2 or more.
fn push_padded(out: &mut String, n: i64, w: u8) {
let _ = if w >= 2 { write!(out, "{:0w$}", n, w = w.min(20) as usize) } else { write!(out, "{n}") };
}
/// Scratch text of one number rendering: every token's own output (`piece`, replaces the token's
/// text), the text inserted ahead of a token (`before`) and the replacement of a literal
/// (`lit_spaces`: fractions blank the literals of a hidden part) live in ONE buffer, addressed
/// by byte ranges, instead of one string per token. Writing a piece again appends the new text
/// and points the slot at it.
struct Parts {
buf: String,
slots: Vec<Slot>,
}
#[derive(Clone, Copy, Default)]
struct Slot {
piece: (usize, usize),
before: (usize, usize),
/// `Some(k)`: show `k` spaces instead of the literal.
lit_spaces: Option<usize>,
}
impl Parts {
fn new(n: usize) -> Parts {
Parts { buf: String::new(), slots: vec![Slot::default(); n + 1] }
}
/// Start of a piece written straight into the buffer; finish it with [`Parts::end_piece`].
fn mark(&self) -> usize {
self.buf.len()
}
fn end_piece(&mut self, ti: usize, start: usize) {
let end = self.buf.len();
if let Some(sl) = self.slots.get_mut(ti) {
sl.piece = (start, end);
}
}
fn set_piece(&mut self, ti: usize, text: &str) {
let start = self.mark();
self.buf.push_str(text);
self.end_piece(ti, start);
}
fn set_spaces(&mut self, ti: usize, k: usize) {
let start = self.mark();
self.buf.extend(std::iter::repeat_n(' ', k));
self.end_piece(ti, start);
}
fn set_before(&mut self, ti: usize, text: &str) {
let start = self.mark();
self.buf.push_str(text);
let end = self.buf.len();
if let Some(sl) = self.slots.get_mut(ti) {
sl.before = (start, end);
}
}
fn set_lit_spaces(&mut self, ti: usize, k: usize) {
if let Some(sl) = self.slots.get_mut(ti) {
sl.lit_spaces = Some(k);
}
}
fn text(&self, (a, b): (usize, usize)) -> &str {
self.buf.get(a..b).unwrap_or("")
}
fn piece(&self, ti: usize) -> &str {
self.slots.get(ti).map_or("", |sl| self.text(sl.piece))
}
fn before(&self, ti: usize) -> &str {
self.slots.get(ti).map_or("", |sl| self.text(sl.before))
}
}
/// Assigns integer digits right-to-left to the placeholders at `idxs`; the leftmost placeholder
/// takes all remaining digits. `force_zero` shows `0` in the units placeholder when there are no
/// digits. A blank `?` pads with a space, and so does the thousands separator it would carry.
fn fill_int(toks: &[Tok], idxs: &[usize], digits: &str, thousands: bool, force_zero: bool, pieces: &mut [String]) {
let ds: Vec<char> = digits.chars().collect();
///
/// `digits` is ASCII (digit strings made by this crate), so bytes are characters. Each
/// placeholder's text is written left to right: a digit, then the separator that follows it when
/// its position counted from the right is a multiple of 3.
fn fill_int(toks: &[Tok], idxs: &[usize], digits: &str, thousands: bool, force_zero: bool, parts: &mut Parts) {
let ds = digits.as_bytes();
let mut rem = ds.len();
let mut pos = 0usize;
let count = idxs.len();
let sep = |p: usize| thousands && p > 0 && p.is_multiple_of(3);
for (k, &ti) in idxs.iter().rev().enumerate() {
let leftmost = k + 1 == count;
let ph = match toks.get(ti) {
Some(Tok::Digit(c)) => *c,
_ => '0',
};
let mut rev = String::new();
let push_digit = |c: char, rev: &mut String, pos: &mut usize| {
if thousands && *pos > 0 && pos.is_multiple_of(3) {
rev.push(',');
}
rev.push(c);
*pos += 1;
};
let start = parts.mark();
let take = if leftmost { rem } else { rem.min(1) };
if take == 0 {
match ph {
'0' => push_digit('0', &mut rev, &mut pos),
_ if force_zero && k == 0 => push_digit('0', &mut rev, &mut pos),
'0' => {
parts.buf.push('0');
if sep(pos) {
parts.buf.push(',');
}
pos += 1;
}
_ if force_zero && k == 0 => {
parts.buf.push('0');
if sep(pos) {
parts.buf.push(',');
}
pos += 1;
}
'?' => {
if thousands && pos > 0 && pos.is_multiple_of(3) {
rev.push(' ');
parts.buf.push(' ');
if sep(pos) {
parts.buf.push(' ');
}
rev.push(' ');
pos += 1;
}
_ => {}
}
} else {
for _ in 0..take {
rem -= 1;
push_digit(ds.get(rem).copied().unwrap_or('0'), &mut rev, &mut pos);
// The `take` digits just left of `rem`, most significant first; the digit `s` places
// from the right end of the run sits at position `pos + s`.
for s in (0..take).rev() {
let d = ds.get(rem.saturating_sub(1 + s)).copied().unwrap_or(b'0');
parts.buf.push(char::from(d));
if sep(pos + s) {
parts.buf.push(',');
}
}
rem -= take;
pos += take;
}
if let Some(p) = pieces.get_mut(ti) {
*p = rev.chars().rev().collect();
}
parts.end_piece(ti, start);
}
}
@@ -203,26 +332,26 @@ fn with_commas(digits: &str, thousands: bool) -> String {
}
/// Fraction-digit placeholders: trailing zeros show as `0`, nothing (`#`) or a space (`?`).
fn fill_frac(toks: &[Tok], idxs: &[usize], frac: &str, pieces: &mut [String]) {
let fd: Vec<char> = frac.chars().collect();
let last_sig = fd.iter().rposition(|&c| c != '0');
/// `frac` is ASCII digits.
fn fill_frac(toks: &[Tok], idxs: &[usize], frac: &str, parts: &mut Parts) {
let fd = frac.as_bytes();
let last_sig = fd.iter().rposition(|&c| c != b'0');
for (i, &ti) in idxs.iter().enumerate() {
let ph = match toks.get(ti) {
Some(Tok::Digit(c)) => *c,
_ => '0',
};
let s = if last_sig.is_some_and(|l| i <= l) {
fd.get(i).copied().unwrap_or('0').to_string()
let start = parts.mark();
if last_sig.is_some_and(|l| i <= l) {
parts.buf.push(char::from(fd.get(i).copied().unwrap_or(b'0')));
} else {
match ph {
'0' => "0".into(),
'?' => " ".into(),
_ => String::new(),
'0' => parts.buf.push('0'),
'?' => parts.buf.push(' '),
_ => {}
}
};
if let Some(p) = pieces.get_mut(ti) {
*p = s;
}
parts.end_piece(ti, start);
}
}
@@ -337,15 +466,12 @@ pub(crate) fn render_number(sec: &Section, v: f64) -> Out {
}
let toks = &sec.toks;
let n = toks.len();
// Pieces replace the token's own output; `before` is inserted ahead of a token; `lits`
// replaces a literal's text (fractions blank the literals of a hidden part).
let mut pieces: Vec<String> = vec![String::new(); n];
let mut before: Vec<String> = vec![String::new(); n + 1];
let mut lits: Vec<Option<String>> = Vec::new();
// Pieces replace the token's own output; `before` is inserted ahead of a token; literal
// replacements blank the literals of a hidden fraction part (see [`Parts`]).
let mut parts = Parts::new(n);
let mut zero = false;
if sec.fraction {
lits = vec![None; n];
render_fraction(sec, x, &mut pieces, &mut lits);
render_fraction(sec, x, &mut parts);
} else {
let NumberLayout { point, exp, int_idx, frac_idx } = number_layout(toks);
let (int_s, frac_s) = if let Some(e_at) = exp {
@@ -359,7 +485,6 @@ pub(crate) fn render_number(sec: &Section, v: f64) -> Out {
if int_s.is_empty() && frac_s.chars().all(|c| c == '0') {
e = 0;
}
let head = if upper { "E" } else { "e" };
let sign = if e < 0 {
"-"
} else if plus {
@@ -372,18 +497,16 @@ pub(crate) fn render_number(sec: &Section, v: f64) -> Out {
// `E` (observed: `|0|e+|0|` shows `|1|e|+5|`); `?` pads the digits with spaces.
match exp_idx.first() {
Some(&first) => {
if let Some(p) = pieces.get_mut(e_at) {
*p = head.to_string();
}
if let Some(b) = before.get_mut(first) {
*b = sign.to_string();
}
fill_int(toks, &exp_idx, &digits, false, false, &mut pieces);
parts.set_piece(e_at, if upper { "E" } else { "e" });
parts.set_before(first, sign);
fill_int(toks, &exp_idx, &digits, false, false, &mut parts);
}
None => {
if let Some(p) = pieces.get_mut(e_at) {
*p = format!("{head}{sign}{digits}");
}
let start = parts.mark();
parts.buf.push(if upper { 'E' } else { 'e' });
parts.buf.push_str(sign);
parts.buf.push_str(&digits);
parts.end_piece(e_at, start);
}
}
(int_s, frac_s)
@@ -391,32 +514,32 @@ pub(crate) fn render_number(sec: &Section, v: f64) -> Out {
fixed(x, frac_idx.len())
};
zero = int_s.chars().chain(frac_s.chars()).all(|c| c == '0');
fill_int(toks, &int_idx, &int_s, sec.thousands, false, &mut pieces);
fill_int(toks, &int_idx, &int_s, sec.thousands, false, &mut parts);
if int_idx.is_empty()
&& !int_s.is_empty()
&& let Some(p) = point.or(exp)
&& let Some(b) = before.get_mut(p)
{
*b = with_commas(&int_s, sec.thousands);
parts.set_before(p, &with_commas(&int_s, sec.thousands));
}
fill_frac(toks, &frac_idx, &frac_s, &mut pieces);
if let Some(p) = point
&& let Some(s) = pieces.get_mut(p)
{
*s = ".".into();
fill_frac(toks, &frac_idx, &frac_s, &mut parts);
if let Some(p) = point {
parts.set_piece(p, ".");
}
}
let mut out = Out::default();
for (i, t) in toks.iter().enumerate() {
if let Some(b) = before.get(i) {
out.push(b);
}
out.push(parts.before(i));
match t {
Tok::Lit(s) => out.push(lits.get(i).and_then(Option::as_deref).unwrap_or(s)),
Tok::Lit(s) => match parts.slots.get(i).and_then(|sl| sl.lit_spaces) {
Some(k) => out.text.extend(std::iter::repeat_n(' ', k)),
None => out.push(s),
},
Tok::Percent => out.push("%"),
Tok::Fill(c) => out.fill(*c),
Tok::At | Tok::General | Tok::Date(_) | Tok::Comma => {}
_ => out.push(pieces.get(i).map(String::as_str).unwrap_or("")),
// Every other token shows its piece. `At` (`0@`) is reachable here, `Comma`, `General`
// and `Date` are not (analysis turns commas into literals and routes sections with
// dates or General elsewhere); none of them is ever given a piece, so they print "".
_ => out.push(parts.piece(i)),
}
}
out.zero = zero;
@@ -472,22 +595,14 @@ fn shown_width(t: &Tok) -> usize {
}
/// Replaces the literals in `range` with spaces (`spaces`) or nothing; placeholders and the bar
/// in it get the same treatment through `pieces`.
fn blank_range(toks: &[Tok], range: std::ops::Range<usize>, spaces: bool, pieces: &mut [String], lits: &mut [Option<String>]) {
/// in it get the same treatment as pieces.
fn blank_range(toks: &[Tok], range: std::ops::Range<usize>, spaces: bool, parts: &mut Parts) {
for i in range {
let Some(t) = toks.get(i) else { break };
let s = if spaces { " ".repeat(shown_width(t)) } else { String::new() };
let width = if spaces { shown_width(t) } else { 0 };
match t {
Tok::Lit(_) => {
if let Some(l) = lits.get_mut(i) {
*l = Some(s);
}
}
Tok::Digit(_) | Tok::Slash | Tok::FixedDenom(_) => {
if let Some(p) = pieces.get_mut(i) {
*p = s;
}
}
Tok::Lit(_) => parts.set_lit_spaces(i, width),
Tok::Digit(_) | Tok::Slash | Tok::FixedDenom(_) => parts.set_spaces(i, width),
_ => {}
}
}
@@ -571,7 +686,7 @@ fn fraction_parts(lay: &FractionLayout, x: f64) -> (f64, f64, u64) {
/// becomes spaces when the whole part or the numerator has a `?`); when the fraction is zero and
/// the numerator has no `0` placeholder, the separator, numerator, bar and denominator disappear
/// (or become spaces when any part has a `?`) and the whole part shows at least `0`.
fn render_fraction(sec: &Section, x: f64, pieces: &mut [String], lits: &mut [Option<String>]) {
fn render_fraction(sec: &Section, x: f64, parts: &mut Parts) {
let toks = &sec.toks;
let Some(lay) = fraction_layout(toks) else { return };
let (whole, num, den) = fraction_parts(&lay, x);
@@ -581,32 +696,28 @@ fn render_fraction(sec: &Section, x: f64, pieces: &mut [String], lits: &mut [Opt
let whole_s = if whole >= 1.0 { fixed(whole, 0).0 } else { String::new() };
let num_zero_ph = num_idx.iter().any(|&i| toks.get(i) == Some(&Tok::Digit('0')));
if has_whole && num == 0.0 && !num_zero_ph {
fill_int(toks, &whole_idx, &whole_s, sec.thousands, true, pieces);
fill_int(toks, &whole_idx, &whole_s, sec.thousands, true, parts);
let spaces = has_q(&whole_idx) || has_q(&num_idx) || has_q(&den_idx);
blank_range(toks, sep_start..den_end, spaces, pieces, lits);
blank_range(toks, sep_start..den_end, spaces, parts);
return;
}
if has_whole {
// A zero value shows `0` in a `?` or `0` units place, nothing in a `#` one.
let units_hash = whole_idx.last().is_some_and(|&i| toks.get(i) == Some(&Tok::Digit('#')));
let force = whole_s.is_empty() && num == 0.0 && !units_hash;
fill_int(toks, &whole_idx, &whole_s, sec.thousands, force, pieces);
let whole_shown = whole_idx.iter().any(|&i| pieces.get(i).is_some_and(|p| p.chars().any(|c| c.is_ascii_digit())));
fill_int(toks, &whole_idx, &whole_s, sec.thousands, force, parts);
let whole_shown = whole_idx.iter().any(|&i| parts.piece(i).chars().any(|c| c.is_ascii_digit()));
if !whole_shown {
let spaces = has_q(&whole_idx) || has_q(&num_idx);
blank_range(toks, sep_start..num_start, spaces, pieces, lits);
blank_range(toks, sep_start..num_start, spaces, parts);
}
}
let num_s = if num < 1e15 { (num as u64).to_string() } else { fixed(num, 0).0 };
fill_int(toks, &num_idx, &num_s, false, true, pieces);
if let Some(p) = pieces.get_mut(slash) {
*p = "/".into();
}
fill_int(toks, &num_idx, &num_s, false, true, parts);
parts.set_piece(slash, "/");
let den_s = den.to_string();
if fixed_den.is_some() {
if let Some(p) = pieces.get_mut(den_start) {
*p = den_s;
}
parts.set_piece(den_start, &den_s);
} else {
// Denominator digits are zero padded to the `0` placeholders, then left aligned; `?`
// pads with spaces after them.
@@ -631,9 +742,7 @@ fn render_fraction(sec: &Section, x: f64, pieces: &mut [String], lits: &mut [Opt
} else {
s
};
if let Some(p) = pieces.get_mut(ti) {
*p = s;
}
parts.set_piece(ti, &s);
}
}
}
+251 -22
View File
@@ -15,6 +15,9 @@ fn format_colors_resolve_named_and_indexed_rgb_safely() {
const S: DateSystem = DateSystem::D1900;
/// Built-in format 44 (accounting with a currency symbol, two decimals).
const ACCOUNTING_2DP: &str = "_(\"$\"* #,##0.00_);_(\"$\"* \\(#,##0.00\\);_(\"$\"* \"-\"??_);_(@_)";
fn f(code: &str, n: f64) -> String {
format_value(&Value::Number(n), &NumberFormat::parse(code), S).text
}
@@ -256,7 +259,7 @@ fn literals_and_fill() -> Result<(), Box<dyn std::error::Error>> {
assert_eq!(f("(0)", 5.0), "(5)");
let r = full("*-0", Value::Number(5.0));
assert_eq!((r.text.as_str(), r.fill), ("5", Some(('-', 0))));
let acct = builtin_format(44).ok_or("none")?;
let acct = ACCOUNTING_2DP;
let r = full(acct, Value::Number(0.0));
assert_eq!((r.text.as_str(), r.fill), (" $- ", Some((' ', 2))));
let r = full(acct, Value::Number(1234.5));
@@ -356,7 +359,7 @@ fn kinds() -> Result<(), Box<dyn std::error::Error>> {
assert_eq!(NumberFormat::parse("0.00").kind(), FormatKind::Number);
assert_eq!(NumberFormat::parse("#,##0.00;[Red](#,##0.00)").kind(), FormatKind::Number);
assert_eq!(NumberFormat::parse("\"$\"#,##0.00").kind(), FormatKind::Currency);
assert_eq!(NumberFormat::parse(builtin_format(44).ok_or("none")?).kind(), FormatKind::Accounting);
assert_eq!(NumberFormat::parse(ACCOUNTING_2DP).kind(), FormatKind::Accounting);
assert_eq!(NumberFormat::parse("m/d/yyyy").kind(), FormatKind::Date);
assert_eq!(NumberFormat::parse("h:mm AM/PM").kind(), FormatKind::Time);
assert_eq!(NumberFormat::parse("0%").kind(), FormatKind::Percentage);
@@ -369,26 +372,11 @@ fn kinds() -> Result<(), Box<dyn std::error::Error>> {
}
#[test]
fn builtins() -> Result<(), Box<dyn std::error::Error>> {
assert_eq!(builtin_format(0), Some("General"));
assert_eq!(builtin_format(14), Some("m/d/yyyy"));
assert_eq!(builtin_format(49), Some("@"));
assert_eq!(builtin_format(30), None);
assert_eq!(builtin_format(50), None);
assert_eq!(builtin_id("m/d/yyyy"), Some(14));
assert_eq!(builtin_id("0.00%"), Some(10));
assert_eq!(builtin_id("GENERAL"), Some(0));
assert_eq!(builtin_id("[h]:mm:ss"), Some(46));
assert_eq!(builtin_id("#,##0.00_);(#,##0.00)"), None);
for id in 0..=60 {
if let Some(code) = builtin_format(id) {
assert_eq!(builtin_id(code), Some(id));
}
}
assert_eq!(f(builtin_format(5).ok_or("none")?, -1234.0), "($1,234)");
assert_eq!(f(builtin_format(12).ok_or("none")?, 0.5), " 1/2");
assert_eq!(f(builtin_format(47).ok_or("none")?, 61.2 / 86400.0), "0101.2");
Ok(())
fn builtin_codes_render() {
// Built-in ids 5, 12 and 47 (the id tables live with the file formats, in gridcraft-xlsx).
assert_eq!(f("\"$\"#,##0_);\\(\"$\"#,##0\\)", -1234.0), "($1,234)");
assert_eq!(f("# ?/?", 0.5), " 1/2");
assert_eq!(f("mmss.0", 61.2 / 86400.0), "0101.2");
}
#[test]
@@ -396,6 +384,13 @@ fn text_fn() {
assert_eq!(text_function(&Value::Number(1234.5), "$#,##0.00", S), Ok("$1,234.50".into()));
assert_eq!(text_function(&Value::from("5"), "0.00", S), Ok("5.00".into()));
assert_eq!(text_function(&Value::from("abc"), "0.00", S), Ok("abc".into()));
// Each `@` repeats the text: up to a cell's 32,767 characters, then #VALUE!.
let at = "@".repeat(255);
assert_eq!(text_function(&Value::from("x".repeat(128).as_str()), &at, S).map(|s| s.len()), Ok(128 * 255));
assert_eq!(text_function(&Value::from("x".repeat(32_000).as_str()), &at, S), Err(CellError::Value));
// The limit counts UTF-16 units: 16,384 emoji are 32,768 of them, one too many.
assert_eq!(text_function(&Value::from("\u{1F600}".repeat(16_383).as_str()), "@", S).map(|s| s.chars().count()), Ok(16_383));
assert_eq!(text_function(&Value::from("\u{1F600}".repeat(16_384).as_str()), "@", S), Err(CellError::Value));
assert_eq!(text_function(&Value::Error(CellError::NA), "0", S), Err(CellError::NA));
assert_eq!(text_function(&Value::Number(46302.0), "yyyy", S), Ok("2026".into()));
assert_eq!(text_function(&Value::Number(0.5), "General", S), Ok("0.5".into()));
@@ -902,6 +897,240 @@ fn closed_brackets_still_hold_their_semicolons() {
assert_eq!(f("\"[\"0;0", 5.0), "[5");
}
const JAN_1_2001: f64 = 36892.0;
#[test]
fn b1_prefix_is_hidden_and_gregorian() {
assert_eq!(f("B1dd/mm/yyyy", 46314.0), "19/10/2026");
assert_eq!(f("b1dddd, mmmm d, yyyy", 46314.0), "Monday, October 19, 2026");
assert_eq!(f("B1yyyy-mm-dd hh:mm", 46314.5), "2026-10-19 12:00");
}
#[test]
fn b2_prefix_renders_hijri_date() {
assert_eq!(f("B2dd/mm/yyyy", JAN_1_2001), "06/10/1421");
assert_eq!(f("b2d/m/yy", JAN_1_2001), "6/10/21");
assert_eq!(f("B2dddd dd/mm/yyyy hh:mm", JAN_1_2001 + 0.75), "الإثنين 06/10/1421 18:00");
assert_eq!(text_function(&Value::Number(46314.0), "B2dd/mm/yyyy", S).as_deref(), Ok("08/05/1448"));
assert_eq!(f("B2e", 46314.0), "1448");
assert_eq!(f("B2ee", 46314.0), "1448");
assert_eq!(f("B2d/m/yyyy;@", 46314.0), "8/5/1448");
}
/// Under `B2`, Excel shows Arabic weekday names, the full name for `ddd` as well as `dddd`.
#[test]
fn b2_prefix_names_weekdays_in_arabic() {
// Monday 19 and Tuesday 20 October 2026.
assert_eq!(f("B2dddd", 46314.0), "الإثنين");
assert_eq!(f("B2ddd", 46314.0), "الإثنين");
assert_eq!(f("B2dddd", 46315.0), "الثلاثاء");
assert_eq!(f("B1dddd", 46314.0), "Monday");
assert_eq!(f("B1ddd", 46315.0), "Tue");
// Every weekday has a name.
for v in 46314..46321 {
assert!(!f("B2dddd", v as f64).is_empty());
}
}
/// `B2` on a General or text section changes nothing: Excel shows the plain serial.
#[test]
fn b2_prefix_on_general_and_text_sections() {
assert_eq!(f("B2General", 46314.0), "46314");
assert_eq!(f("B2@", 46314.0), "46314");
}
#[test]
fn b2_prefix_names_hijri_months() {
assert_eq!(f("B2d mmmm yyyy", JAN_1_2001), "6 شوال 1421");
assert_eq!(f("B2mmm", 37034.0), "ربيع\u{a0}الأول");
assert_eq!(f("B2mmmm", JAN_1_2001 + 23.0), "شوال");
assert_eq!(f("B2mmmm", JAN_1_2001 + 24.0), "ذو\u{a0}القعدة");
assert_eq!(f("B2mmmmm", JAN_1_2001 + 24.0), "ذ");
}
/// All twelve month names and seven weekday names as Excel 16.0 (build 20527, en-US) shows them,
/// read with TEXT and a cell format: two-word month names are joined by a no-break space.
#[test]
fn b2_names_match_excel() {
let months = [
(46219.0, "صفر"),
(46248.0, "ربيع\u{a0}الأول"),
(46278.0, "ربيع\u{a0}الثاني"),
(46307.0, "جمادى\u{a0}الأولى"),
(46337.0, "جمادى\u{a0}الثانية"),
(46366.0, "رجب"),
(46396.0, "شعبان"),
(46425.0, "رمضان"),
(46455.0, "شوال"),
(46484.0, "ذو\u{a0}القعدة"),
(46514.0, "ذو\u{a0}الحجة"),
(46543.0, "محرم"),
];
for (serial, name) in months {
assert_eq!(f("B2dd", serial), "01", "{serial}");
assert_eq!(f("B2mmmm", serial), name, "{serial}");
assert_eq!(f("B2mmm", serial), name, "{serial}");
assert_eq!(f("B2mmmmmm", serial), name, "{serial}");
}
// Sunday 18 to Saturday 24 October 2026.
let days = ["الأحد", "الإثنين", "الثلاثاء", "الأربعاء", "الخميس", "الجمعة", "السبت"];
for (k, name) in days.iter().enumerate() {
let serial = 46313.0 + k as f64;
assert_eq!(f("B2dddd", serial), *name, "{serial}");
assert_eq!(f("B2ddd", serial), *name, "{serial}");
}
assert_eq!(f("B2yy", 46314.0), "48");
assert_eq!(f("B2y", 46314.0), "48");
assert_eq!(f("B2yyy", 46314.0), "1448");
assert_eq!(f("B2yyyyy", 46314.0), "1448");
}
#[test]
fn b2_month_and_year_rollover() {
assert_eq!(f("B2dd/mm/yyyy", 36915.0), "29/10/1421");
assert_eq!(f("B2dd/mm/yyyy", 36916.0), "01/11/1421");
assert_eq!(f("B2dd/mm/yyyy", 36974.0), "29/12/1421");
assert_eq!(f("B2dd/mm/yyyy", 36975.0), "01/01/1422");
assert_eq!(f("B2dd/mm/yyyy", 37683.0), "30/12/1423");
assert_eq!(f("B2dd/mm/yyyy", 37684.0), "01/01/1424");
}
/// Cycle years 15 (1425, leap) and 16 (1426, not leap): Excel's leap year is 15, not 16. The
/// 2 March 2005 and 29 January 2006 values were read from Excel by a reviewer; the year
/// boundaries follow from them and the 354/355-day years. The same comparison covered every
/// serial from 36,000 to 49,000 with no disagreement.
#[test]
fn b2_leap_year_is_cycle_year_15() {
assert_eq!(f("B2yyyy-mm-dd", 38413.0), "1426-01-21"); // 2005-03-02, from Excel
assert_eq!(f("B2yyyy-mm-dd", 38746.0), "1426-12-29"); // 2006-01-29, from Excel
assert_eq!(f("B2dd/mm/yyyy", 38037.0), "29/12/1424"); // 2004-02-20
assert_eq!(f("B2dd/mm/yyyy", 38038.0), "01/01/1425"); // 2004-02-21
assert_eq!(f("B2dd/mm/yyyy", 38392.0), "30/12/1425"); // 2005-02-09
assert_eq!(f("B2dd/mm/yyyy", 38393.0), "01/01/1426"); // 2005-02-10
assert_eq!(f("B2dd/mm/yyyy", 38747.0), "01/01/1427"); // 2006-01-30
// Anchors the reviewer read from Excel.
assert_eq!(f("B2dd/mm/yyyy", 46314.0), "08/05/1448"); // 2026-10-19
assert_eq!(f("B2dd/mm/yyyy", 46313.0), "07/05/1448"); // 2026-10-18
assert_eq!(f("B2dd/mm/yyyy", JAN_1_2001), "06/10/1421");
assert_eq!(f("B2dd/mm/yyyy", 61.0), "29/10/1317"); // 1900-03-01
assert_eq!(f("B2dd/mm/yyyy", 64970.0), "01/01/1501"); // 2077-11-16
assert_eq!(f("B2dd/mm/yyyy", 64971.0), "02/01/1501"); // 2077-11-17
assert_eq!(f("B2dd/mm/yyyy", 0.0), "29/08/1317");
assert_eq!(f("B2dd/mm/yyyy", 2958465.0), "03/04/9666"); // 9999-12-31
}
/// Excel honours `B2` after a colour tag, but a `[$-...]` locale tag before it keeps the date
/// Gregorian (the prefix is still hidden); `B2[$-401]` is Hijri.
#[test]
fn calendar_prefix_and_bracket_tags() {
assert_eq!(f("[Red]B2dd/mm/yyyy", 46314.0), "08/05/1448");
assert_eq!(f("[<0]0;B2dd/mm/yyyy", JAN_1_2001), "06/10/1421");
assert_eq!(f("B2[$-401]dd/mm/yyyy", 46314.0), "08/05/1448");
for code in ["[$-401]B2dd/mm/yyyy", "[$-409]B2dd/mm/yyyy", "[$-1010401]B2dd/mm/yyyy", "[$-ar-SA]B2dd/mm/yyyy"] {
assert_eq!(f(code, 46314.0), "19/10/2026", "{code}");
assert_eq!(text_function(&Value::Number(46314.0), code, S).as_deref(), Ok("19/10/2026"), "{code}");
}
assert_eq!(f("[Red][$-401]B1dd/mm/yyyy", JAN_1_2001), "01/01/2001");
// Checked in Excel 16.0 (build 20527, en-US): a currency tag cancels `B2` like a locale tag,
// a tag after the prefix does not, and a repeated prefix is accepted.
assert_eq!(f("[$€-407]B2dd/mm/yyyy", 46314.0), "€19/10/2026");
assert_eq!(f("[$$-409]B2dd/mm/yyyy", 46314.0), "$19/10/2026");
assert_eq!(f("[$-401][Red]B2dd/mm/yyyy", 46314.0), "19/10/2026");
assert_eq!(f("B2[Red]dd/mm/yyyy", 46314.0), "08/05/1448");
for code in ["B2B2dd/mm/yyyy", "B2B2B2dd/mm/yyyy", "B2b2dd/mm/yyyy", "[Red]B2B2dd/mm/yyyy"] {
assert_eq!(f(code, 46314.0), "08/05/1448", "{code}");
}
assert_eq!(f("B1B1dd/mm/yyyy", 46314.0), "19/10/2026");
assert_eq!(f("B2 dd/mm/yyyy", 46314.0), " 08/05/1448");
assert_eq!(f("B2\"x\"dd", 46314.0), "x08");
for code in ["B1\"x\"", "B2\"x\""] {
assert_eq!(f(code, 46314.0), "x", "{code}");
}
for code in ["B1General", "B1@", "0;0;0;B2@", "0;B2dd/mm/yyyy"] {
assert_eq!(f(code, 46314.0), "46314", "{code}");
}
}
/// A quoted `B2` is literal text; any other `B` and digit makes Excel reject the code (`TEXT` is
/// `#VALUE!`), as does a prefix before a number format. A rejected code shows as General. All
/// checked in Excel 16.0 (build 20527, en-US), with TEXT and with a cell format.
#[test]
fn misplaced_calendar_prefix_is_rejected() {
assert_eq!(f("\"B2\"dd/mm/yyyy", 46314.0), "B219/10/2026");
assert_eq!(f("dd/\"B2\"/yyyy", 46314.0), "19/B2/2026");
for code in [
"B0dd/mm/yyyy",
"B3dd/mm/yyyy",
"B4dd/mm/yyyy",
"B9dd/mm/yyyy",
"B02dd",
"B12dd",
"B21dd",
"B1B2dd/mm/yyyy",
"B2B1dd/mm/yyyy",
"B2[Red]B2dd/mm/yyyy",
"B2 B2dd",
"dd/mm/yyyyB2",
"dd/mm/yyyyB1",
"dd B2 mm",
"dd b2 mm",
"dd \\B2 yyyy",
"B2\\B2dd",
"B20.00",
"B2#,##0",
"B10.00",
"B1#,##0",
"0;B3dd",
] {
assert_eq!(text_function(&Value::Number(46314.0), code, S), Err(CellError::Value), "{code}");
assert_eq!(f(code, 46314.0), "46314", "{code}");
assert!(NumberFormat::parse(code).is_general(), "{code}");
}
}
#[test]
fn b2_edges_of_the_date_range() {
// Serial 60 is the 1900 system's fictitious 29 February: Excel has no Hijri date for it.
assert_eq!(text_function(&Value::Number(60.0), "B2dd/mm/yyyy", S), Err(CellError::Value));
assert_eq!(f("B2dd/mm/yyyy", 60.0), "########");
assert_eq!(f("B1dd/mm/yyyy", 60.0), "29/02/1900");
assert_eq!(f("B2dd/mm/yyyy", 59.0), "28/10/1317");
assert_eq!(f("B2dd/mm/yyyy", 61.0), "29/10/1317");
// So does any clock part (Excel 16.0, build 20527, en-US), but not an elapsed `[h]`, a literal,
// or a locale tag that cancels `B2`.
for code in ["B2hh:mm", "B2hh", "B2ss", "B2am/pm", "B2hh:mm AM/PM", "B2[$-401]hh:mm", "B2dd/mm/yyyy;\"x\""] {
assert_eq!(text_function(&Value::Number(60.0), code, S), Err(CellError::Value), "{code}");
assert_eq!(text_function(&Value::Number(60.5), code, S), Err(CellError::Value), "{code}");
}
assert_eq!(f("B2[h]", 60.0), "1440");
assert_eq!(f("B2\"x\"", 60.0), "x");
assert_eq!(f("[$-401]B2hh:mm", 60.0), "00:00");
assert_eq!(f("B1hh:mm", 60.0), "00:00");
assert_eq!(f("B2hh:mm", 61.0), "00:00");
// Serial 0 shows as serial 1.
assert_eq!(f("B2dd/mm/yyyy", 0.0), "29/08/1317");
assert_eq!(f("B2dd/mm/yyyy", 1.0), "29/08/1317");
// No upper limit short of 31 December 9999: 16 and 17 November 2077, 31 December 9999.
assert_eq!(f("B2dd/mm/yyyy", 64970.0), "01/01/1501");
assert_eq!(f("B2dd/mm/yyyy", 64971.0), "02/01/1501");
assert_eq!(f("B2dd/mm/yyyy", 2958465.0), "03/04/9666");
// In the 1904 system, serial 60 is a real day.
assert_ne!(format_value(&Value::Number(60.0), &NumberFormat::parse("B2dd/mm/yyyy"), DateSystem::D1904).text, "########");
}
#[test]
fn b2_out_of_range_does_not_panic() {
for v in [-1.0, 0.0, 60.0, 60.5, 2958465.0, 2958466.0, 1e300, f64::NAN] {
let _ = f("B2dd/mm/yyyy mmmm dddd", v);
let _ = text_function(&Value::Number(v), "B2dd/mm/yyyy mmmm dddd", S);
}
assert_eq!(f("B2dd/mm/yyyy", -1.0), f("dd/mm/yyyy", -1.0));
assert_eq!(f("B2dd/mm/yyyy", 2958466.0), f("dd/mm/yyyy", 2958466.0));
// Before 1 Muharram 1 AH there is no Hijri date (no serial reaches it).
let before_epoch = gridcraft_core::date::DateTime { year: 600, month: 3, day: 9, ..Default::default() };
assert!(calendar::Calendar::Hijri.date(&before_epoch).is_none());
}
/// "Set precision as displayed": the value each format stores. Expected values observed in Excel
/// (COM, `Workbook.PrecisionAsDisplayed = True`) with synthetic values.
#[test]
+152
View File
@@ -0,0 +1,152 @@
1427c0507a69d2eb 79 "0.0000000000000000000000000"
c37039701ea91f71 79 "#,###,###,###,###,###,###,###,##0.00"
c886ef7cc0381aca 79 "0.0000000000000000000000000E+00"
16878498a86d6270 79 "# ?????/?????"
25810fe3aa348757 79 "General"
637698d5ec4840e7 79 "0"
f480db9c3f30f4f9 79 "0.00"
233cef56f68009ce 79 "#,##0"
d2fae9f7b5756366 79 "#,##0.00"
fcf13b3494133da6 79 "#.##"
93b3291c9e9847f7 79 "0.0##"
20ca0c941ad5c11f 79 "00000"
db86a5c476fc29cd 79 "000.000"
7cc10690ed9769ab 79 "#,###"
ea3312bb2d5c1342 79 "#,##0,"
d14a7d6e48db33f0 79 "#,##0,,"
e44a9c0ed1f49c8a 79 "0.0,,\"M\""
3c70c4c8ff8af422 79 "?.??"
93f48e4f2d83fd29 79 "##0.0"
a06b56173759b328 79 "# "
5a35d2ec11553fa4 79 "#"
f3495305635eab2d 79 ".00"
1f2ae975e66d556b 79 ",0"
253e097c9bdaf643 79 "0,0"
ae00b79ca5b8d288 79 "0.00,"
1b989416a1ec6e7c 79 "\"$\"#,##0.00"
f82256c8ac29c8f3 79 "\"$\"#,##0_);\\(\"$\"#,##0\\)"
7fc54d574b4ee62c 79 "#,##0.00;[Red](#,##0.00)"
7a81a86d7a4d3c69 79 "#,##0 ;(#,##0)"
57a11793e92c9715 79 "0;-0;;@"
a3d6c52367c35259 79 "0.00;;"
b45d3fdec1e8ff9a 79 ";;;"
11a021e5408029bb 79 "0_);(0)"
99b2e56004ca9b4a 79 "0\" units\""
c9aaa3ef8150f3a6 79 "\\A0"
641306b16f0036fe 79 "(0)"
19af7c406782b02c 79 "00-00"
c298712a7d3815bd 79 "000-00-0000"
28e5bc8fc3d88c8e 79 "[<=9999999]###-####;(###) ###-####"
f494c7ff5ff0fd7a 79 "[$€-407]#,##0.00"
4e11e82e14ea8f05 79 "_(* #,##0_);_(* \\(#,##0\\);_(* \"-\"_);_(@_)"
fc2d989610e136f3 79 "_(\"$\"* #,##0.00_);_(\"$\"* \\(#,##0.00\\);_(\"$\"* \"-\"??_);_(@_)"
091f29e23e924f36 79 "*-0"
93217b43f4b7af91 79 "0*x"
6a7561a5ce77014d 79 "*0 0.0"
9a6620cdfa67c60b 79 "@*_"
0b8f6e85e6ff12f0 79 "0%"
d2f558040c30c4cc 79 "0.00%"
371df01b15432009 79 "#,##0%"
e5016314db39e07b 79 "0.0%%"
f603836d162018f0 79 "%0"
7d08aed42e8d71e0 79 "0%;[Red]-0%"
d7d44df9f81bff2e 79 "0.00E+00"
275678f83b4d4113 79 "##0.0E+0"
497033fffb9c0967 79 "0E+0"
9679af8d3aa45c26 79 "0.0e-0"
eafc341422b759ce 79 "0.00E+000"
67cf5d053575d3df 79 "E+0"
f9d5b40fa2875a1e 79 "0.00E+"
e633fc616f9a3ff3 79 "#.##E+00"
8ecbeef181fb381e 79 "0.0E+00 \"x\""
7b74fb5636bda27e 79 "# ?/?"
aa283fbec589cdfe 79 "# ??/??"
9c9625146b582644 79 "# ???/???"
7d7dd7cefba77e8e 79 "?/?"
3babcf6e56373ec4 79 "# ?/8"
c2e84ddf50f1e21d 79 "# ??/16"
d5bbf7173f227e7a 79 "0/0"
6fe467468cd9a182 79 "# #/#"
a73c12b90cfc3656 79 "#/#"
d034fba50699f7ea 79 "?/100"
02ee892d135e01dc 79 "# ?/?;-# ?/?"
4bd7e5c1d9b3503f 79 "0 0/0"
2162452594679802 154 "m/d/yyyy"
d04b4c59e67bea74 154 "d-mmm-yy"
5dec91c67babe5f2 154 "d-mmm"
c384b64cedef18eb 154 "mmm-yy"
d7aa777afafbc668 154 "h:mm AM/PM"
b175c2057a22f6de 154 "h:mm:ss AM/PM"
aded84fe78136262 154 "h:mm"
5a79757ea5163c90 154 "h:mm:ss"
cd019c5ea6050d45 154 "m/d/yyyy h:mm"
54dbe04a04108479 154 "mm:ss"
e0718e196529a8f2 154 "[h]:mm:ss"
a38b480cf5c5a364 154 "[h]:mm"
2a913161d1313180 154 "[m]:ss"
76fdaf50d35fb2a2 154 "[s]"
ff9403ec80b0bc4f 154 "mmss.0"
425d4aa682fdc092 154 "yyyy-mm-dd"
058ce7638d05bbb9 154 "dddd, mmmm d, yyyy"
fe4bc250009a8329 154 "ddd mmm dd yy"
ccb6419ee78b2955 154 "mmmmm"
6c0289cc8797f513 154 "yy"
47cc3767e6229ec4 154 "h:mm:ss.000"
e6112ef1c8665ef8 154 "h:mm:ss.0000"
0bf1d0573ca9d704 154 "hh:mm:ss A/P"
e2a3d66078e1cf02 154 "[hh]:mm:ss"
8707505efd8a6284 154 "d"
7fa481a6092fa43c 154 "dd"
7d4b6020f781a3ba 154 "dddd"
949ed74d9b020535 154 "m"
0aa17a2f309cbecd 154 "mm"
c70a36183af563af 154 "s.00"
a8c8c009a7b0012f 154 "yyyy-mm-dd hh:mm:ss.00"
a70fc60b044d8b5e 154 "h AM/PM"
cf3e0c4fccbedb80 154 "[$-409]d-mmm-yy"
80b17497f3b9d5dd 154 "yyyy\"年\"m\"月\""
e00bb34ae8360b4c 154 "[h]\" hrs\""
8b75446f8310605f 154 "mmmm yyyy;@"
01123dabdab536a7 79 "@"
59fa7b60ca99b9b7 79 "\"x\"@"
b1eee7ca88961e38 79 "0;\"neg\"0;\"zero\";\"txt: \"@"
4d3f8a891532765e 79 "General;General;General;@"
59dc8b6bd14d719a 79 "[Red]0.0;[Blue]-0.0"
6a4d8666a78e97b3 79 "[Green]0;[Magenta]0;[Cyan]0;[Yellow]@"
15b794f754adc79d 79 "[Color3]0.00"
158526f0a5b736c5 79 "[>100]\"big\"0;[<-100]\"small\"0;0.0"
7a94ed36a6bb6573 79 "[=0]\"zero\";0.0"
30d048d3f229b1ba 79 "[>=1000]#,##0,\"K\";0"
2b2b1f13faeb468d 79 "[<1]0.00%;0.0"
b78758e73c416a1e 79 "[Red][>5]0;0.00"
8b2379e7de300595 79 "General\" kg\""
73f3485fe971d2bd 79 "\"v\"General"
2d2ce0ee57d199a2 79 "[Red]General"
ea2f102aad8c3c1b 79 "0.0;[Red]General"
ef5fc7ae5e0cf36c 79 "["
d80872ea795758e3 79 "0.0.0"
145e66eaa5b6a665 79 "0\""
ddb909bb72f98e33 79 "\\"
cc44573b688554a4 154 "[h"
c78707c90ee7a1b6 79 "[$"
aaf7da8233ee4f96 154 "yyyy-mm-dd;"
b1bbc169427ccdf8 79 ";"
2cf377cde4d0586d 79 "0;0;0;0;0"
9293c3fbfd2434b1 79 "\"abc"
3a0ef807a097eb69 79 "[>x]0"
1ef168bf45be1582 79 "ggg"
b39c47241ef60000 79 "#?#0"
92199ef29ea54843 154 "0E"
fe0eba9c258b10e7 154 "E0"
c5adbc0bf74eded8 79 "/"
d77317d9cc005a8c 79 "0/"
86aefad6fd9ab034 79 "/0"
d08e2f5c1fb6d629 79 "*"
5257c26622f2cfe0 79 "_"
ef14d84a8bf4f1b4 79 "_0"
c1a24c12c4381ef2 154 "[Red"
1653b1ac403ed200 79 "0.00.00%"
aac2038aa4ae7651 154 "mm:ss.0.0"
00083314e9f89199 79 "[ABC]0"
a456ed739adcc87c 79 "[=1]0;[=2]0;0"
b0b3aa7b9f08690c 79 "0 ??/??;0"
+555
View File
@@ -0,0 +1,555 @@
5fda7bf42a47c479 1 0e0
e400479573b2baf2 1 -0e0
57f99cd6daf9cc72 1 1e0
0ddafbadc6bf1a04 1 -1e0
882d8b786644b3c6 1 5e-1
846783d06acc0a60 1 -5e-1
eb4b115b5778e14e 1 2.5e-1
d320cfc393d5b774 1 1.005e0
e7497067a4de5fb0 1 2.675e0
68aed91bbaac1534 1 9.999995e2
96a3c4703fc3b01a 1 9.995e-1
df98061474126a71 1 9.995e0
5916478863984e9b 1 9.95e1
13813099e41458c9 1 5e-2
e6381aba4f09a980 1 5e-3
6c09e99da5f3e27f 1 5e-4
1e9840c7c9074f0a 1 1e-5
b8e3e7ad44ea4900 1 1e-10
a9d70b3df9c809d1 1 1e-300
4c1d46df8304d5b7 1 5e-324
c2a28fae0bcd7949 1 1.23456789e8
c38f4a506cc3267c 1 1.2345678e3
bde85249e1dcc206 1 1.2345678901e10
f61e4f9f86db3eea 1 1e11
72057cacab3997fa 1 1e15
4183c6065332dc5d 1 1.23456789012345e14
e1cb4037184e1fe4 1 1.234567890123455e15
503e585193837caa 1 1e20
a59688774ce6d2a1 1 1e100
20a1bd96371a3de0 1 1.7976931348623157e308
3b202f375efc1640 1 -1.2345e3
816da9ca801294d6 1 -4e-3
f8adad661b001d1f 1 -1e-10
a8aaed4940499212 1 3.333333333333333e-1
b146e4ee19146f95 1 6.666666666666666e-1
7433fb99403da062 1 3.0000000000000004e-1
1c0a0bcb4c0422b8 1 1e2
4d9afc58f74b7ddb 1 1e3
0d48b7819a8a4b8d 1 5.9e1
01452d3bffb28025 1 6e1
4921ebb52b8c73c1 1 6.1e1
09ea6a1671f8c563 1 3.6526e4
b63b4cad6f332f99 1 3.652675e4
2bc41104f2e19271 1 4.38315e4
4de1bb6ce4b9358b 1 4.5000123456e4
ed7fda487b9bcf16 1 2.958465e6
9059129e9a79299b 1 2.958465999999e6
a32ca3ab1dbe969f 1 2.958466e6
90b8f9a1e42bd0a4 1 -5e0
7e564b2cabd8de16 1 9.9999999e-1
8781fec461d58bfd 1 9.99e-4
7c708dbcf3460659 1 4.199999999999e1
20b2def65bc7174d 1 NaN
957f300224ed1cab 1 inf
4b2f7ba9aa3ce108 1 -inf
9ae51ec9ae4d5114 1 4.1666666666666664e-2
2689dad0ffde687c 1 9.999999999e-1
1599f472f24a81fa 1 1.0000000001e0
5f1f3c268b6168ac 1 1.23456789e1
c513d218d1599535 1 3.14159265358979e0
8140c7ee3b5a5345 1 6.5535e4
bd982ddec1e00aaa 1 1e5
97614145ed7f215c 1 1.23456e5
26cde2690b2bcbcc 1 4.5e0
1a8add9df90d93c9 1 3.75e-1
eef82337e19fa9f3 1 1e300
884afff127bc9c8a 1 1.5e-8
723565414e3a8eb5 1 9.99999999999999e14
410dc1f39f0470e8 1 9.999999999999998e13
abe0337311e85bc2 1 1.234e-5
e05b983d81a775a8 1 1.2345678901234568e17
f1b04c74292211ec 1 1e22
8bd34401b876c52c 1 1e23
8162c16a7c44dc93 1 1.234567890123456e-1
5df098e551929df8 1 7.083333333333334e-4
403536010a422332 1 -2.4456788999965284e-22
5fda7bf42a47c479 1 0e0
6f8d525947f4496b 1 5.078754219653861e-20
5fda7bf42a47c479 1 0e0
db9fc2cd4be272f9 1 4.762524884899572e-19
5fda7bf42a47c479 1 0e0
4e42d845bdfe1e05 1 -9.481878134872896e-18
5fda7bf42a47c479 1 0e0
e2b246d2b6df4843 1 4.677068358276083e-17
5fda7bf42a47c479 1 0e0
db3ef5385bc881e0 1 4.2817210912547903e-16
5fda7bf42a47c479 1 0e0
bbf8726d918fa10a 1 -1.2049587102547775e-16
5fda7bf42a47c479 1 0e0
c0efc68af4f6f086 1 1.3789513024013423e-14
5fda7bf42a47c479 1 0e0
790b21d90f49a509 1 6.059016384138212e-13
5fda7bf42a47c479 1 0e0
ce56d3a5c4486cd1 1 -2.0839028757975573e-12
5fda7bf42a47c479 1 0e0
975db8510513c040 1 8.895735060164848e-11
5fda7bf42a47c479 1 0e0
472bbe20b67f109c 1 2.924228743531876e-11
5fda7bf42a47c479 1 0e0
a9c44b1d793ee953 1 -2.790173620128004e-9
5fda7bf42a47c479 1 0e0
b65a4a2339333f83 1 3.047667157619239e-8
5fda7bf42a47c479 1 0e0
0c9b50a66593ab52 1 9.768724036381636e-7
5fda7bf42a47c479 1 0e0
deb83d6ed094b3b2 1 -6.624321624908681e-7
5fda7bf42a47c479 1 0e0
cfb05c2a70b35a2d 1 1.6894338933954122e-5
5fda7bf42a47c479 1 0e0
dfa7bf97c3a3f630 1 5.610159605226053e-4
aa35c0cb97f7addc 1 1e-3
6f2e8f1578d1b6f7 1 -4.5736943003751186e-3
e6381aba4f09a980 1 5e-3
618e160ad7d75e3f 1 1.2177853639931391e-2
7b3b780b251205f7 1 1.2e-2
bc71798a9a9fd283 1 5.376575791159676e-1
552d6fe712db0701 1 5.38e-1
30f24eae79fdbea7 1 -1.6906316761960716e0
9a85be1470daf3e4 1 1.691e0
428a08f2bbd5c129 1 6.804349057483373e1
36f22379ff98da63 1 6.8043e1
42c9ce8792ca41ca 1 5.180578200923046e2
b753a84983c4b8fb 1 5.18058e2
d48ddf21b575ae2e 1 -6.281034210276412e3
335061600a21e480 1 6.281034e3
93205c65ad863f72 1 3.435675535581831e4
1abaaae29d8209f0 1 3.4356755e4
d09ef10e029d4d19 1 6.613076865085881e5
b1c40c1667e951b0 1 6.61307687e5
8e6cdd6bc70fbd22 1 -4.48220238062414e6
c083e5d00d97cef4 1 4.482202381e6
e127f684d7698427 1 1.0000341372267152e8
d4d94bdd00026ac9 1 1.00003413723e8
0cab466bd5e1d8fa 1 1.5849160403504446e8
d4c67910bde50f32 1 1.58491604035e8
0e7f466ac9853aae 1 -1.4058601193934329e9
a643a7677b4c5ac7 1 1.405860119393e9
2776bab16cf335d4 1 2.8871069252525196e10
fc7419911bfbaca8 1 2.8871069252525e10
a76f1c3dce467478 1 9.690172366761895e11
be689212e28414e9 1 9.6901723667619e11
c4c5bd5055043ad5 1 -2.346417551608747e12
93b654146869cfc6 1 2.346417551608747e12
645dfd0b8d651920 1 6.228668439208695e12
645dfd0b8d651920 1 6.228668439208695e12
f147744d6ebe1c8c 1 2.2428112791069544e14
f147744d6ebe1c8c 1 2.2428112791069544e14
ba8da370ef0b6f07 1 -7.327413159633475e15
76ca14ec7527c55d 1 7.327413159633475e15
813a82002c70fe12 1 6.675488044574204e16
813a82002c70fe12 1 6.675488044574204e16
8a50e37fec07f962 1 6.706148330480582e17
8a50e37fec07f962 1 6.706148330480582e17
b2bcb6262b7ad869 1 -7.27445337914015e18
a958885d3e964a90 1 7.27445337914015e18
f8c71a4f984138a2 1 3.874586892728994e19
f8c71a4f984138a2 1 3.874586892728994e19
143942bc00828b3f 1 4.207699940806956e-21
5fda7bf42a47c479 1 0e0
cca09550d6d0af64 1 -6.321408874535118e-20
5fda7bf42a47c479 1 0e0
804ebfa29a23426e 1 3.689215801512328e-19
5fda7bf42a47c479 1 0e0
34d878721f2d74d3 1 9.392721997790782e-18
5fda7bf42a47c479 1 0e0
bf13de1377b6a335 1 -5.788399687409319e-17
5fda7bf42a47c479 1 0e0
9164e2fa13905b35 1 2.279056237090581e-16
5fda7bf42a47c479 1 0e0
4fc8a97fbf0232f5 1 3.463636269309186e-15
5fda7bf42a47c479 1 0e0
eb0a80785bd2f283 1 -4.489928391258454e-14
5fda7bf42a47c479 1 0e0
9f70d66705199764 1 7.703173974678871e-13
5fda7bf42a47c479 1 0e0
fb73018ae299a802 1 1.3090206448341722e-12
5fda7bf42a47c479 1 0e0
c064d37e034a6497 1 -6.338150789669849e-11
5fda7bf42a47c479 1 0e0
53be38791b9b3a71 1 3.809051908390393e-11
5fda7bf42a47c479 1 0e0
87e6cfab84040f20 1 1.0070431565221817e-9
5fda7bf42a47c479 1 0e0
5db583e96d56b61f 1 -5.266594942615038e-8
5fda7bf42a47c479 1 0e0
87730d5ab9b90358 1 2.039725677192573e-7
5fda7bf42a47c479 1 0e0
2c827aff0066cbce 1 4.851068050995432e-6
5fda7bf42a47c479 1 0e0
31b9628affbc1814 1 -7.75941473244553e-5
5fda7bf42a47c479 1 0e0
8be16fa85aa991e0 1 7.381056138192671e-4
aa35c0cb97f7addc 1 1e-3
6c3609330a25ae41 1 1.575376680923546e-4
5fda7bf42a47c479 1 0e0
02745e58c6fb95fe 1 -8.309622978077537e-2
3ae24003b2396fd4 1 8.3e-2
35cb86c8421094f5 1 5.938677033303984e-1
cd17bcf30868b4d7 1 5.94e-1
5fc071051c10dcf2 1 2.1055700940714903e0
3301c223c3ee7087 1 2.106e0
374453ca4e69570f 1 -4.979516792760796e0
30ac305e01179f6c 1 4.98e0
dc9118090ba75534 1 7.702392128251413e2
5668bd8822fd37eb 1 7.70239e2
bc0fea3db4020ccf 1 5.295197566581555e2
6a1f8e82ef33f9d0 1 5.2952e2
8f67ffd819348bfb 1 -9.175765278181084e4
7431aae5707ad8ce 1 9.1757653e4
e1050c33ba3c5491 1 9.482550236540656e5
a129426c85bdbae0 1 9.48255024e5
e6d83dc1bd22212c 1 5.347153721033019e6
e026e88cae226949 1 5.347153721e6
03a3405b7ce1bb74 1 -2.3568277599192787e7
61da64c36fd8c5eb 1 2.3568277599e7
b3b4116a1d7c332a 1 1.0018445295210177e8
ad683901e4974657 1 1.00184452952e8
0be5c818158d8aba 1 8.356382245461647e7
9656312094a4767c 1 8.3563822455e7
cf7cd1e0127e076f 1 -4.557698877252628e10
d82db809145f4cb7 1 4.5576988772526e10
e21e145b37008e47 1 5.270348510870472e11
3eeab7eb66861c31 1 5.27034851087047e11
d51733208c288c7c 1 5.782033227859766e12
d51733208c288c7c 1 5.782033227859766e12
f0e51f3bf67fc7a3 1 -5.931696066185405e13
e15f77a9ff4fb907 1 5.931696066185405e13
9a7a6b9b153d807e 1 5.1455458768478e14
3cce9402217011dc 1 5.1455458768478006e14
c9f98f1fa9b23bc5 1 1.3538071072562253e15
c9f98f1fa9b23bc5 1 1.3538071072562253e15
bbff8f21267563e7 1 -9.90253824456554e15
1a63f8bd04651493 1 9.90253824456554e15
64140d1f4b064459 1 8.527630873184316e17
64140d1f4b064459 1 8.527630873184316e17
c42d45691a199adf 1 3.2925484547629455e18
c42d45691a199adf 1 3.2925484547629455e18
6d966e6760bed43d 1 -3.91537728865695e19
efdecd348456b99a 1 3.91537728865695e19
51ff7d6fc5809c27 1 4.8757246809796105e-21
5fda7bf42a47c479 1 0e0
84bceb480eb2dcb6 1 2.3420748266718282e-20
5fda7bf42a47c479 1 0e0
de6b157cd8b3068b 1 -6.293170081812841e-19
5fda7bf42a47c479 1 0e0
fca52fc30bf77496 1 5.53513945740202e-18
5fda7bf42a47c479 1 0e0
fef0b8399d29f15b 1 9.77860691981301e-17
5fda7bf42a47c479 1 0e0
8956eb136a63fc19 1 -5.1566694357030435e-17
5fda7bf42a47c479 1 0e0
c8d0796283c1a680 1 5.364510629593392e-15
5fda7bf42a47c479 1 0e0
c10f295101314665 1 9.661665594611814e-14
5fda7bf42a47c479 1 0e0
71f69f42ed1abd5f 1 -3.925165717977516e-13
5fda7bf42a47c479 1 0e0
4b3b18313d353d45 1 4.634788713583143e-12
5fda7bf42a47c479 1 0e0
1714fa5b40a6cf7b 1 2.1289153323939538e-11
5fda7bf42a47c479 1 0e0
fbe658ebfc667eeb 1 -3.7731542867717375e-10
5fda7bf42a47c479 1 0e0
ed1059691b29437f 1 3.0793918014219435e-9
5fda7bf42a47c479 1 0e0
d68a671d72cbb7d4 1 1.398193907677646e-8
5fda7bf42a47c479 1 0e0
bf3150d59315bcd7 1 -5.243893844613249e-7
5fda7bf42a47c479 1 0e0
93b6b3ec8b06d2ff 1 9.40673096940851e-6
5fda7bf42a47c479 1 0e0
71583d853e2cc61b 1 9.755344390282753e-5
5fda7bf42a47c479 1 0e0
a2726686baf8eaa6 1 -7.322925001904205e-4
aa35c0cb97f7addc 1 1e-3
7b84f8a587116b61 1 2.20079478849153e-3
97b75dab0686cb37 1 2e-3
69efff5278fdc06b 1 5.4369720160556294e-2
22dfdcd990abd9ff 1 5.4e-2
bdd2e88716b365e0 1 -1.90566789746209e-1
a419c77256cf893c 1 1.91e-1
cfb6f41a39039f7f 1 1.1954616597037193e0
8818e7a995d67595 1 1.195e0
d936acae76796026 1 3.1198426146081694e1
f7c4c11aa02f61b9 1 3.1198e1
4d61587438c8539f 1 -1.7276546543447557e2
31d3905e625db28b 1 1.72765e2
592a0f53b05918db 1 3.716258271721481e3
c5e0c130f048add8 1 3.716258e3
9964fe1fd34a9e66 1 6.354064796316502e4
a3273ac06e88377e 1 6.3540648e4
9d1cc1cc40d1e375 1 -6.247169817552513e5
6e32184b6303fecd 1 6.24716982e5
a59dec2d400de042 1 9.672875989854455e6
c6184456a5f3d6dc 1 9.67287599e6
952a4c5d6d9f3cc7 1 5.101929774891923e7
00398960ce567662 1 5.1019297749e7
2985fb1731569c12 1 -7.868553457247695e8
cbe14af9c8d6397f 1 7.86855345725e8
6ddbb1af610f79cf 1 6.192364765951433e9
e146c8288f6cd3d2 1 6.192364765951e9
674dfcf3b4a569b6 1 5.0322395435787346e10
4eaeab0011c8ca30 1 5.0322395435787e10
2a2da0e805cf5409 1 -8.387392837998628e11
6e45a8daf47f9904 1 8.38739283799863e11
adea7a9a9dfe1129 1 3.475227374631468e12
adea7a9a9dfe1129 1 3.475227374631468e12
928198bdae5603d2 1 6.046260397573301e13
928198bdae5603d2 1 6.046260397573301e13
3b2d1a775477a31c 1 -9.366184270581025e14
856529b379f79ef7 1 9.366184270581025e14
ac839bfff6b8812f 1 9.330285659281354e15
ac839bfff6b8812f 1 9.330285659281354e15
73160e61af42cec9 1 3.539370193508612e16
73160e61af42cec9 1 3.539370193508612e16
dedb455241069fa2 1 -9.996650818086104e17
ec679a20500b031d 1 9.996650818086104e17
26215fcc4901f42b 1 7.905433521846371e18
26215fcc4901f42b 1 7.905433521846371e18
ec9afc963bc85c7b 1 2.961789170831489e19
ec9afc963bc85c7b 1 2.961789170831489e19
f2482a70bd7552a8 1 -4.148371208994727e-21
5fda7bf42a47c479 1 0e0
a55409a5a800406d 1 7.170663257742498e-20
5fda7bf42a47c479 1 0e0
17f938f9cc679cd3 1 3.9583585927085557e-20
5fda7bf42a47c479 1 0e0
9a113924c396bb44 1 -5.892919583042963e-18
5fda7bf42a47c479 1 0e0
f0c40ba08b4e8d1c 1 6.485392183833719e-17
5fda7bf42a47c479 1 0e0
da9719b7ced0671f 1 7.180575172217469e-16
5fda7bf42a47c479 1 0e0
90ac83c7b7ea92c2 1 -2.7801459081525537e-16
5fda7bf42a47c479 1 0e0
069d079990d92448 1 8.42511493280958e-14
5fda7bf42a47c479 1 0e0
1c765f9111408737 1 1.0841559011512436e-13
5fda7bf42a47c479 1 0e0
7530679d8e6abd87 1 -2.1480013377898877e-12
5fda7bf42a47c479 1 0e0
873ab178136be7f4 1 2.8019016665180154e-11
5fda7bf42a47c479 1 0e0
4f3329b0b1b9010d 1 4.740529408461166e-10
5fda7bf42a47c479 1 0e0
84f8fb1f92637dc1 1 -7.552388125782508e-9
5fda7bf42a47c479 1 0e0
3d7d836e4bdb867e 1 9.755292951606497e-9
5fda7bf42a47c479 1 0e0
3fa8b742b75bff54 1 3.8472751651678525e-7
5fda7bf42a47c479 1 0e0
d349c8520a8cc6de 1 -6.977907562349183e-6
5fda7bf42a47c479 1 0e0
e1df5d730caf3475 1 8.218407552134532e-5
5fda7bf42a47c479 1 0e0
3df6765577ebd93a 1 4.996897368127074e-4
5fda7bf42a47c479 1 0e0
81b26f24e3ef8f0d 1 -3.151243912693312e-3
77147cf949266682 1 3e-3
bd1c5f937c96ac6a 1 3.0347488452323525e-2
fc87ec37c8733143 1 3e-2
a73ab956230ebaef 1 6.889166586655288e-1
5fa353279229150c 1 6.89e-1
f568fd4cfe3142c6 1 -9.743654785321732e0
1bc6257bb3848aa9 1 9.744e0
0f5ba1866b7d2845 1 5.668388855766816e1
ce72a327bacd2b00 1 5.6684e1
b4efc6800c4b914d 1 2.628257089515055e2
21784204a4be65c0 1 2.62826e2
e67a2d6c6630643f 1 -1.6168861365096154e3
47815c5434eded45 1 1.616886e3
4e3b51240c986695 1 3.280045990086601e4
6890cb1d3a7f9b75 1 3.280046e4
cabedf8f491b936e 1 1.4504200063410145e5
e3836c93597707d8 1 1.45042001e5
4271771f51534d50 1 -7.313314211991392e6
1fb3dc6157f32057 1 7.313314212e6
46c2ec9b7d7dd831 1 2.618837245417135e7
3b956a81d84ff17d 1 2.6188372454e7
5a8bbd3fdc03e83f 1 2.2253015708835256e8
394a2739b0947d1f 1 2.22530157088e8
40691261266728fd 1 -9.870234060874973e9
eb3d9a317a8874aa 1 9.870234060875e9
6a990a269a5c74f0 1 1.867926582347318e10
f2ca9284e3a4f179 1 1.8679265823473e10
280ece39f3f50572 1 3.567370490275957e11
c14e0f480d1a7640 1 3.56737049027596e11
3789dd57fc5c815c 1 -7.912946695129458e12
b5ae37bd691873c7 1 7.912946695129458e12
9f1553246fdb40cb 1 4.524229340690999e12
9f1553246fdb40cb 1 4.524229340690999e12
b24a577eb86c3e84 1 2.3102331568926594e14
b24a577eb86c3e84 1 2.3102331568926594e14
cdce2c77867a3106 1 -4.547025311981324e15
e947b96f57d17bec 1 4.547025311981324e15
cf45a38c70fcf953 1 4.6288039353802424e16
cf45a38c70fcf953 1 4.6288039353802424e16
014ea8cd6a1290e2 1 6.162091464101659e17
014ea8cd6a1290e2 1 6.162091464101659e17
ce7e9262127eff44 1 -4.849757599353334e18
e50f3fe044d209e7 1 4.849757599353334e18
d85b0205e7941c69 1 6.452894445273737e19
d85b0205e7941c69 1 6.452894445273737e19
f2c90f259094e92e 1 4.7803298279157025e-21
5fda7bf42a47c479 1 0e0
9a93ee439db2dc35 1 -7.233406321955394e-20
5fda7bf42a47c479 1 0e0
d1e6a63ae1d3a9bb 1 2.950670879528398e-19
5fda7bf42a47c479 1 0e0
9230461e2e26d575 1 3.063434779812392e-18
5fda7bf42a47c479 1 0e0
314136843a1e646c 1 -5.4890433629545986e-17
5fda7bf42a47c479 1 0e0
6f3c47632e4d68b3 1 3.820800755734363e-16
5fda7bf42a47c479 1 0e0
6f9edd60cd32e0d5 1 5.829213571595438e-16
5fda7bf42a47c479 1 0e0
78eb51a008a64997 1 -2.4171609780651217e-14
5fda7bf42a47c479 1 0e0
44e17d26d950269c 1 1.1540630209261826e-13
5fda7bf42a47c479 1 0e0
8c1d1f942eaccbbf 1 1.84539975340409e-12
5fda7bf42a47c479 1 0e0
04d9cdb28aac62c3 1 -1.1081220338777577e-11
5fda7bf42a47c479 1 0e0
ed11aa9ba5e5353f 1 6.04223566612428e-10
5fda7bf42a47c479 1 0e0
f99d682b436ba383 1 5.35352701590592e-9
5fda7bf42a47c479 1 0e0
98b4918f9567bfd3 1 -8.711002748201745e-8
5fda7bf42a47c479 1 0e0
1ad36850fde48c38 1 6.114307847139126e-7
5fda7bf42a47c479 1 0e0
08f08db80439f666 1 7.209491487707519e-6
5fda7bf42a47c479 1 0e0
6a2bea77bad32684 1 -9.107581134395797e-5
5fda7bf42a47c479 1 0e0
a6e64a2db8e7c580 1 1.502958882607809e-4
5fda7bf42a47c479 1 0e0
bb4f7310114464cd 1 9.694573849985441e-3
e40ccf007d65d503 1 1e-2
99d400856d68de78 1 -6.734696325484274e-2
6f6224f1a2f8d8f1 1 6.7e-2
e1bf6652cb8fa67a 1 5.736804397820615e-1
3503377f10431893 1 5.74e-1
abde9a0944f0075c 1 8.385525171527297e-1
bba484bf8ca102d3 1 8.39e-1
f0cadc58f065b9d4 1 -7.281427491271538e1
3c3889813ef35db4 1 7.2814e1
977a2a510c565470 1 8.277766877236081e1
1342b4ca96de7c68 1 8.2778e1
7d0118272533a618 1 6.537925550437317e3
eb730fb885709956 1 6.537926e3
5cbc224dc75590af 1 -9.556341676004886e4
651cd96eb2cd481d 1 9.5563417e4
7454f12c9c69f37a 1 8.917218866870608e5
66dff276a09c18ca 1 8.91721887e5
6cd5d2549504ffe4 1 1.7117781599952288e6
b1b8b8d63f96b6e3 1 1.71177816e6
4a42c81491eef17c 1 -7.887393405242969e7
84299b08c59543e1 1 7.8873934052e7
e59890374c3b92fd 1 4.975792917240606e8
744d7f2f33d43087 1 4.97579291724e8
62bc9e9bf68188bf 1 2.1101063788838792e9
d651f84aed764523 1 2.110106378884e9
03d61121d089d8a5 1 -5.020627637227572e10
e7dec7d4092686f3 1 5.0206276372276e10
f17fe9bd3b512a80 1 6.810322205888846e11
734531128a13110d 1 6.81032220588885e11
9c1dec292ef7b37f 1 1.1358071611803188e12
dd424abd6b3af684 1 1.135807161180319e12
694f57c35b260310 1 -3.644307586777936e13
13611f86ee78a0ed 1 3.644307586777936e13
d2a2a06dea91c0a6 1 4.619231907602407e14
d2a2a06dea91c0a6 1 4.619231907602407e14
77901e2ac387e970 1 6.618779769599664e15
77901e2ac387e970 1 6.618779769599664e15
89f326d5337edbf9 1 -1.6799818086520645e15
4522b28cd51ce9fc 1 1.6799818086520645e15
f9e0155a978000b9 1 8.738754212610816e17
f9e0155a978000b9 1 8.738754212610816e17
4bd5f489ae1421a8 1 3.782517862984159e18
4bd5f489ae1421a8 1 3.782517862984159e18
40b6664a43480fc1 1 -2.4148772852069077e19
9441359dbc20f6b6 1 2.4148772852069077e19
76f2826b72e1fda6 1 8.877221241287098e-21
5fda7bf42a47c479 1 0e0
bb4a3ae1f5949c11 1 1.8341454532358146e-20
5fda7bf42a47c479 1 0e0
766ae042bb11d920 1 -9.887584766107361e-19
5fda7bf42a47c479 1 0e0
c959d5910bc866f9 1 9.166579699039185e-18
5fda7bf42a47c479 1 0e0
b2699aa224c131ce 1 3.2452783206646016e-18
5fda7bf42a47c479 1 0e0
e784da29c1b96c28 1 -4.798816460260819e-16
5fda7bf42a47c479 1 0e0
c3bc806511396a74 1 5.664578770036096e-15
5fda7bf42a47c479 1 0e0
6c133c6895703b73 1 8.829601615987267e-14
5fda7bf42a47c479 1 0e0
b7721097837aef20 1 -4.3573790293046475e-13
5fda7bf42a47c479 1 0e0
af58a97244a9c6c3 1 5.551460106733779e-12
5fda7bf42a47c479 1 0e0
4646a79cd8815ccf 1 6.050294303578548e-11
5fda7bf42a47c479 1 0e0
5a4ffdfaaac64359 1 -3.3879885578121597e-10
5fda7bf42a47c479 1 0e0
d30c3dba6f8b8d72 1 9.069160028722885e-9
5fda7bf42a47c479 1 0e0
35719d8fb5e8e209 1 3.1971165267170906e-8
5fda7bf42a47c479 1 0e0
8f3a36e216599613 1 -8.76974502693964e-7
5fda7bf42a47c479 1 0e0
4a7c13e5c2e949a0 1 4.680359748821511e-6
5fda7bf42a47c479 1 0e0
c58c1c5093cfd666 1 7.563237465046436e-5
5fda7bf42a47c479 1 0e0
71c0d2f679b366ab 1 -1.79232054129694e-4
5fda7bf42a47c479 1 0e0
cc39dc0b3f31ef1a 1 8.188147563049265e-3
1e7c9f5936317e01 1 8e-3
7810fc63819aa54f 1 4.8276882501172715e-2
316f14fbacc0f6dc 1 4.8e-2
91fcbed9b6c1e1eb 1 -4.4497532569195164e-1
1ba135db6a0ed229 1 4.45e-1
6a646bd946016bba 1 7.6485650387578605e0
49f07016ba5d174a 1 7.649e0
35aacb91d397e63c 1 6.863911531586618e1
ad8b40222f9c0b67 1 6.8639e1
b62458bc71ec92fd 1 -4.564364881857964e2
d8bdf521d8767965 1 4.56436e2
f0d014cf4e9edf7f 1 1.9163551023901937e3
15c1eb9d0605d6b0 1 1.916355e3
b051547190f1dead 1 1.850604973023119e4
a8b5ae0e34ac6db2 1 1.850605e4
57da0f4c75eb541f 1 -6.297725994049538e5
41c4a242cd154e87 1 6.29772599e5
3053c5c8720a172e 1 7.738395738921099e4
aaeee63373ab5c58 1 7.7383957e4
122c1b2c08853c0c 1 8.396373407043353e7
7b7f558a837722ff 1 8.396373407e7
ba34066cdc427569 1 -2.4288547122230375e8
420f589afd6b0565 1 2.42885471222e8
041703a36b497da8 1 2.812715976368404e9
c889fbd6f49dfffc 1 2.812715976368e9
72f6aaac4485ab61 1 5.522396329298781e10
60ac6434d6875dde 1 5.5223963292988e10
36803089910ba7e2 1 -7.073708200884093e11
5afe826153ea2a19 1 7.07370820088409e11
8a4f45dfac6a7733 1 3.73164417144063e12
8a4f45dfac6a7733 1 3.73164417144063e12
69bca16e7f65b43d 1 1.69380681630822e13
69bca16e7f65b43d 1 1.69380681630822e13
+225
View File
@@ -0,0 +1,225 @@
//! Equality tests for the render/General cleanup: output over a large grid of format codes x
//! values (format_value and text_function in both date systems, then General at widths 1..=30 and a few larger ones)
//! is compared with golden digests recorded from the implementation BEFORE the cleanup
//! (`tests/golden/*.txt`, one FNV-1a digest per format code / per value, so the files stay
//! small). A failure names the code or value that differs; set `NUMFMT_DUMP_DIR` to write the
//! full text of the current output for diffing against a dump from the old code. To re-record
//! after an intended behaviour change run with `NUMFMT_BLESS=1`.
#![allow(clippy::approx_constant, clippy::excessive_precision)] // values are test data, not constants
use std::fmt::Write as _;
use gridcraft_core::{DateSystem, Value};
use gridcraft_numfmt::{NumberFormat, format_general_fit, format_value, text_function};
// Placeholders far beyond the 15 significant digits.
const LONG_DECIMALS: &str = "0.0000000000000000000000000";
#[rustfmt::skip]
const CODES: &[&str] = &[
LONG_DECIMALS, "#,###,###,###,###,###,###,###,##0.00", "0.0000000000000000000000000E+00", "# ?????/?????",
// numbers
"General", "0", "0.00", "#,##0", "#,##0.00", "#.##", "0.0##", "00000", "000.000", "#,###", "#,##0,", "#,##0,,", "0.0,,\"M\"", "?.??", "##0.0", "# ", "#", ".00",
",0", "0,0", "0.00,", "\"$\"#,##0.00", "\"$\"#,##0_);\\(\"$\"#,##0\\)", "#,##0.00;[Red](#,##0.00)", "#,##0 ;(#,##0)", "0;-0;;@", "0.00;;", ";;;",
"0_);(0)", "0\" units\"", "\\A0", "(0)", "00-00", "000-00-0000", "[<=9999999]###-####;(###) ###-####", "[$€-407]#,##0.00",
// currency/accounting with fill
"_(* #,##0_);_(* \\(#,##0\\);_(* \"-\"_);_(@_)", "_(\"$\"* #,##0.00_);_(\"$\"* \\(#,##0.00\\);_(\"$\"* \"-\"??_);_(@_)", "*-0", "0*x", "*0 0.0", "@*_",
// percent
"0%", "0.00%", "#,##0%", "0.0%%", "%0", "0%;[Red]-0%",
// scientific
"0.00E+00", "##0.0E+0", "0E+0", "0.0e-0", "0.00E+000", "E+0", "0.00E+", "#.##E+00", "0.0E+00 \"x\"",
// fractions
"# ?/?", "# ??/??", "# ???/???", "?/?", "# ?/8", "# ??/16", "0/0", "# #/#", "#/#", "?/100", "# ?/?;-# ?/?", "0 0/0",
// dates/times
"m/d/yyyy", "d-mmm-yy", "d-mmm", "mmm-yy", "h:mm AM/PM", "h:mm:ss AM/PM", "h:mm", "h:mm:ss", "m/d/yyyy h:mm", "mm:ss", "[h]:mm:ss", "[h]:mm", "[m]:ss", "[s]", "mmss.0",
"yyyy-mm-dd", "dddd, mmmm d, yyyy", "ddd mmm dd yy", "mmmmm", "yy", "h:mm:ss.000", "h:mm:ss.0000", "hh:mm:ss A/P", "[hh]:mm:ss", "d", "dd", "dddd", "m", "mm", "s.00",
"yyyy-mm-dd hh:mm:ss.00", "h AM/PM", "[$-409]d-mmm-yy", "yyyy\"年\"m\"月\"", "[h]\" hrs\"", "mmmm yyyy;@",
// text sections, conditions, colours
"@", "\"x\"@", "0;\"neg\"0;\"zero\";\"txt: \"@", "General;General;General;@", "[Red]0.0;[Blue]-0.0", "[Green]0;[Magenta]0;[Cyan]0;[Yellow]@", "[Color3]0.00",
"[>100]\"big\"0;[<-100]\"small\"0;0.0", "[=0]\"zero\";0.0", "[>=1000]#,##0,\"K\";0", "[<1]0.00%;0.0", "[Red][>5]0;0.00", "General\" kg\"", "\"v\"General",
"[Red]General", "0.0;[Red]General",
// invalid / odd
"[", "0.0.0", "0\"", "\\", "[h", "[$", "yyyy-mm-dd;", ";", "0;0;0;0;0", "\"abc", "[>x]0", "ggg", "#?#0", "0E", "E0", "/", "0/", "/0", "*", "_", "_0", "[Red", "0.00.00%", "mm:ss.0.0",
"[ABC]0", "[=1]0;[=2]0;0", "0 ??/??;0",
];
fn values() -> Vec<f64> {
let mut v = vec![
0.0,
-0.0,
1.0,
-1.0,
0.5,
-0.5,
0.25,
1.005,
2.675,
999.9995,
0.9995,
9.995,
99.5,
0.05,
0.005,
0.0005,
1e-5,
1e-10,
1e-300,
5e-324,
123456789.0,
1234.5678,
12345678901.0,
1e11,
1e15,
123456789012345.0,
1234567890123455.0,
1e20,
1e100,
1.7976931348623157e308,
-1234.5,
-0.004,
-1e-10,
1.0 / 3.0,
2.0 / 3.0,
0.1 + 0.2,
100.0,
1000.0,
59.0,
60.0,
61.0,
36526.0,
36526.75,
43831.5,
45000.123456,
2958465.0,
2958465.999999,
2958466.0,
-5.0,
0.99999999,
0.000999,
41.99999999999,
f64::NAN,
f64::INFINITY,
f64::NEG_INFINITY,
0.041666666666666664,
0.9999999999,
1.0000000001,
12.3456789,
3.14159265358979,
65535.0,
1e5,
123456.0,
4.5,
0.375,
1e300,
1.5e-8,
999999999999999.0,
99999999999999.99,
0.00001234,
123456789012345678.0,
1e22,
9.999999999999999e22,
0.1234567890123456,
];
v.push(61.2 / 86400.0);
v
}
fn fmt_formatted(f: &gridcraft_numfmt::Formatted) -> String {
format!("{:?}|{:?}|{:?}|{}", f.text, f.color, f.fill, f.numeric)
}
fn dump_codes() -> Vec<(String, String)> {
assert!(CODES.len() >= 80 && values().len() >= 40);
let mut rows = Vec::new();
let others = [Value::Text("abc".into()), Value::Text("12.5".into()), Value::Bool(true), Value::Empty];
for code in CODES {
let mut out = String::new();
let nf = NumberFormat::parse(code);
let systems: &[(DateSystem, &str)] =
if nf.is_date() { &[(DateSystem::D1900, "1900"), (DateSystem::D1904, "1904")] } else { &[(DateSystem::D1900, "1900")] };
for (sys, name) in systems {
for n in values() {
let v = Value::Number(n);
let _ =
writeln!(out, "{code:?} {name} {n:e} => {} || {:?}", fmt_formatted(&format_value(&v, &nf, *sys)), text_function(&v, code, *sys));
}
}
for v in &others {
let _ = writeln!(
out,
"{code:?} {v:?} => {} || {:?}",
fmt_formatted(&format_value(v, &nf, DateSystem::D1900)),
text_function(v, code, DateSystem::D1900)
);
}
rows.push((format!("{code:?}"), out));
}
rows
}
fn dump_general() -> Vec<(String, String)> {
let mut vals = values();
let mut x = 0.7f64;
// Deterministic spread over many magnitudes and digit patterns (no RNG dependency).
for i in 0..240 {
x = (x * 997.0 + 0.123_456_789).fract() + 1e-3;
let e = (i % 41) - 20;
let m = x * 10f64.powi(e);
vals.push(if i % 3 == 0 { -m } else { m });
vals.push((m * 1000.0).round() / 1000.0);
}
assert!(vals.len() >= 300);
let mut rows = Vec::new();
for n in vals {
let mut out = String::new();
let _ = write!(out, "{n:e} =>");
for w in (1..=30usize).chain([31, 40, 100, 399, 400, 401, 500, 1000]) {
let _ = write!(out, " {:?}", format_general_fit(n, w));
}
out.push('\n');
rows.push((format!("{n:e}"), out));
}
rows
}
fn fnv1a(s: &str) -> u64 {
s.bytes().fold(0xcbf2_9ce4_8422_2325, |h, b| (h ^ u64::from(b)).wrapping_mul(0x100_0000_01b3))
}
fn check(name: &str, rows: &[(String, String)], golden: &str, path: &str) {
if let Some(dir) = std::env::var_os("NUMFMT_DUMP_DIR") {
let all: String = rows.iter().map(|(_, b)| b.as_str()).collect();
std::fs::write(std::path::Path::new(&dir).join(format!("{name}.txt")), all).expect("write dump");
}
let actual: String = rows.iter().map(|(k, b)| format!("{:016x} {} {k}\n", fnv1a(b), b.lines().count())).collect();
if std::env::var_os("NUMFMT_BLESS").is_some() {
std::fs::write(path, &actual).expect("write golden");
return;
}
let bad: Vec<String> = rows
.iter()
.zip(actual.lines().zip(golden.lines()))
.filter(|(_, (a, g))| a != g)
.take(3)
.map(|((_, body), (a, g))| {
format!(" new: {a}\n old: {g}\n new output of that row (first 12 lines):\n{}", body.lines().take(12).collect::<Vec<_>>().join("\n"))
})
.collect();
assert!(
actual.lines().count() == golden.lines().count() && bad.is_empty(),
"{name}: output differs from the recorded golden ({} vs {} rows). If the change is intended, re-record with\n NUMFMT_BLESS=1 cargo test -p gridcraft-numfmt --test render_equality\n{}",
actual.lines().count(),
golden.lines().count(),
bad.join("\n")
);
}
#[test]
fn format_value_and_text_function_match_golden() {
check("codes", &dump_codes(), include_str!("golden/codes.txt"), concat!(env!("CARGO_MANIFEST_DIR"), "/tests/golden/codes.txt"));
}
#[test]
fn general_fit_matches_golden() {
check("general", &dump_general(), include_str!("golden/general.txt"), concat!(env!("CARGO_MANIFEST_DIR"), "/tests/golden/general.txt"));
}
+22
View File
@@ -0,0 +1,22 @@
[package]
name = "gridcraft-sandbox"
version.workspace = true
edition.workspace = true
license.workspace = true
description = "GridCraft's confined launcher for helper processes: Landlock and seccomp on Linux, sandbox-exec on macOS, refused elsewhere"
[lints]
workspace = true
[dependencies]
thiserror = { workspace = true }
# The confinement layers' system calls sit behind these crates' safe APIs; GridCraft's own code
# has no `unsafe`. Exact pins: they are the security boundary.
[target.'cfg(target_os = "linux")'.dependencies]
landlock = "=0.4.7"
seccompiler = "=0.5.0"
libc = "0.2"
[target.'cfg(any(target_os = "linux", target_os = "macos"))'.dependencies]
nix = { version = "=0.31.3", default-features = false, features = ["fs", "signal"] }
+50
View File
@@ -0,0 +1,50 @@
//! The canary: a file outside every allowed path, which a confined launcher must fail to read.
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
use crate::Result;
static NEXT: AtomicU64 = AtomicU64::new(0);
/// A fresh directory in the system's temporary directory holding one file. The launcher's
/// enforcement check must be denied reading the file and writing next to it. Both are removed
/// when the canary is dropped.
#[derive(Debug)]
pub struct Canary {
dir: PathBuf,
file: PathBuf,
}
impl Canary {
pub fn create() -> Result<Canary> {
let base = std::env::temp_dir().canonicalize()?;
// `create_dir` (not `create_dir_all`) fails if the name exists, so the directory found is
// new; a name another process left behind is skipped.
let mut tries = 0;
let dir = loop {
let dir = base.join(format!("gridcraft-sandbox-{}-{}", std::process::id(), NEXT.fetch_add(1, Ordering::Relaxed)));
match std::fs::create_dir(&dir) {
Ok(()) => break dir,
Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists && tries < 100 => tries += 1,
Err(e) => return Err(e.into()),
}
};
let file = dir.join("canary");
if let Err(e) = std::fs::write(&file, b"gridcraft sandbox canary\n") {
let _ = std::fs::remove_dir_all(&dir);
return Err(e.into());
}
Ok(Canary { dir, file })
}
pub fn file(&self) -> &Path {
&self.file
}
}
impl Drop for Canary {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.dir);
}
}
+263
View File
@@ -0,0 +1,263 @@
//! A confined launcher for helper processes that run code GridCraft doesn't trust, such as a
//! workbook's Python cells.
//!
//! The parent builds a [`Policy`] (the paths the program may read, and the programs it may
//! execute), creates a [`Canary`], and spawns the command [`launch_command`] returns. That command
//! runs the application itself again with [`LAUNCH_FLAG`]; the application hands its arguments to
//! [`handle_args`] before anything else, and the launcher:
//!
//! 1. closes every descriptor besides standard input, output and error, and refuses if one stays
//! open: the confinement doesn't revoke what is already open, and an app's system libraries leave
//! files open that a child inherits;
//! 2. confines itself: on Linux with Landlock (read-only access to the policy's paths, execute
//! access only to its programs, no writes) and seccomp (no new processes, no sockets, none of
//! the calls that escape or inspect a sandbox or that change a file where Landlock has no right
//! for it); on macOS it runs itself again under
//! `sandbox-exec` with a deny-by-default profile built from the same policy ([`CONFINED_FLAG`]);
//! 3. checks that the confinement holds: reading the canary, changing its permissions, truncating
//! it, writing next to it, a TCP connection and starting a process must each be denied;
//! 4. only then replaces itself with the program.
//!
//! When any step fails, the program never runs: the launcher writes one line,
//! `gridcraft-sandbox: refused: <reason>`, to its standard error and exits with
//! [`REFUSED_EXIT_CODE`]. Other platforms (Windows, FreeBSD, the web) have no confinement here:
//! [`support`] says so and [`launch_command`] refuses, so callers show `#BLOCKED!`.
//!
//! The crate has no `unsafe` code: the system calls sit behind the `landlock`, `seccompiler` and
//! `nix` crates, and behind `/usr/bin/sandbox-exec` on macOS.
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
#![forbid(unsafe_code)]
mod canary;
#[cfg(target_os = "linux")]
mod linux;
#[cfg(target_os = "macos")]
mod macos;
mod policy;
mod probe;
use std::ffi::{OsStr, OsString};
use std::path::Path;
use std::process::Command;
pub use canary::Canary;
pub use policy::Policy;
pub use probe::{Check, Outcome};
/// The first argument of a launcher run: the application passes its arguments to [`handle_args`].
pub const LAUNCH_FLAG: &str = "--sandbox-launch";
/// The first argument of a launcher already running confined (macOS, under `sandbox-exec`): it
/// goes straight to the enforcement check, which refuses when it isn't confined after all.
pub const CONFINED_FLAG: &str = "--sandbox-confined";
/// The first argument of the child the enforcement check tries to start; it exits at once.
pub const PROBE_CHILD_FLAG: &str = "--sandbox-probe-child";
/// The first argument of a confined run that reports what it may and may not do
/// (`gridcraft-cli sandbox-check`).
pub const REPORT_FLAG: &str = "--sandbox-report";
/// The exit code of a launcher that refused to run its program.
pub const REFUSED_EXIT_CODE: i32 = 97;
/// Why a program can't be run confined.
#[derive(Debug, thiserror::Error)]
pub enum SandboxError {
#[error("this system has no supported sandbox: {0}")]
Unsupported(String),
#[error("bad sandbox policy: {0}")]
BadPolicy(String),
#[error("the sandbox is not enforced: {0}")]
NotEnforced(String),
#[error("couldn't start the sandbox: {0}")]
Io(#[from] std::io::Error),
}
pub type Result<T> = std::result::Result<T, SandboxError>;
/// Whether this system can confine a program.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Support {
/// The sandbox is available; the text names what provides it.
Available(String),
/// It is not, and why.
Unsupported(String),
}
/// Whether this system can confine a program. A [`launch_command`] can still be refused when the
/// running kernel turns out not to enforce a restriction: the launcher checks again.
pub fn support() -> Support {
platform::support()
}
/// The command that runs `program` with `args`, confined by `policy`, checked against `canary`.
/// The parent spawns it like any other command and keeps `canary` alive until the program has
/// started (its standard streams are the program's). Those stay open with the rights they were
/// opened with, which the confinement doesn't revoke: a stream that is a file can be written and
/// truncated, and one that is a socket can still send. So give the program pipes. The command runs the current executable
/// again, which must pass its arguments to [`handle_args`] first.
pub fn launch_command(policy: &Policy, canary: &Canary, program: &Path, args: &[OsString]) -> Result<Command> {
launch_command_with(&std::env::current_exe()?, policy, canary, program, args)
}
/// [`launch_command`] with another executable as the launcher: one that also passes its arguments
/// to [`handle_args`] first.
pub fn launch_command_with(exe: &Path, policy: &Policy, canary: &Canary, program: &Path, args: &[OsString]) -> Result<Command> {
let checked = policy.checked(exe, program, canary)?;
let mut launch = vec![OsString::from(LAUNCH_FLAG)];
launch.extend(checked.to_args(canary.file()));
launch.push(OsString::from("--"));
launch.push(checked.program().as_os_str().to_owned());
launch.extend(args.iter().cloned());
platform::command(&checked, exe, &launch)
}
/// Handles the application's own sandbox runs; call it with the process's arguments (without the
/// program name) before anything else. `None`: the arguments are not one of those runs, carry on.
/// `Some(code)`: exit with `code`. A launcher run whose checks pass doesn't return at all: it
/// becomes the program.
pub fn handle_args(args: &[OsString]) -> Option<i32> {
let (first, rest) = args.split_first()?;
if first == OsStr::new(LAUNCH_FLAG) || first == OsStr::new(CONFINED_FLAG) {
let Err(e) = if first == OsStr::new(LAUNCH_FLAG) { launch(rest) } else { confined_again(rest) };
eprintln!("gridcraft-sandbox: refused: {e}");
return Some(REFUSED_EXIT_CODE);
}
if first == OsStr::new(PROBE_CHILD_FLAG) {
return Some(0);
}
if first == OsStr::new(REPORT_FLAG) {
return Some(probe::report(rest));
}
None
}
/// The launcher: check what it inherited, confine, check the confinement, become the program.
/// Returns only on failure.
fn launch(args: &[OsString]) -> Result<std::convert::Infallible> {
// Arguments that can't be a launch are refused with nothing done. Reading them leaves nothing open.
let (checked, _) = policy::Checked::from_args(args)?;
// Before the launcher opens anything itself. On macOS this is also before the sandbox: under a
// deny-by-default profile the system's libraries hold a socket of their own, which closes at
// `exec` and which must not be taken for an inherited one.
close_inherited_descriptors()?;
// What the check tries on a process has to be allowed now, or its refusal then proves nothing.
let target = probe::signal_target().map_err(SandboxError::NotEnforced)?;
platform::confine(&checked, target, args)?;
confined(target, args)
}
/// The launcher run again under the confinement ([`CONFINED_FLAG`]): the process the first run looked
/// at comes before its arguments.
fn confined_again(args: &[OsString]) -> Result<std::convert::Infallible> {
let parsed = args.split_first().and_then(|(target, rest)| Some((target.to_str()?.parse().ok()?, rest)));
let (target, rest) = parsed.ok_or_else(|| SandboxError::NotEnforced(format!("{CONFINED_FLAG} needs the process the launcher looked at")))?;
confined(target, rest)
}
/// The launcher once it is confined: check that it holds, then become the program.
fn confined(target: u32, args: &[OsString]) -> Result<std::convert::Infallible> {
let (checked, canary) = policy::Checked::from_args(args)?;
let exe = std::env::current_exe()?;
for check in probe::enforcement(&canary, &exe, target) {
if let Outcome::Allowed(detail) = check.outcome {
return Err(SandboxError::NotEnforced(format!("{}: {detail}", check.name)));
}
}
platform::exec(checked.program(), checked.args())
}
#[cfg(target_os = "linux")]
use linux as platform;
#[cfg(target_os = "macos")]
use macos as platform;
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
use unsupported as platform;
/// Platforms without confinement: every launch is refused.
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
mod unsupported {
use super::{Command, OsString, Path, Result, SandboxError, Support, policy::Checked};
const WHY: &str = "GridCraft confines programs on Linux and macOS only";
pub fn support() -> Support {
Support::Unsupported(WHY.into())
}
pub fn command(_: &Checked, _: &Path, _: &[OsString]) -> Result<Command> {
Err(SandboxError::Unsupported(WHY.into()))
}
pub fn confine(_: &Checked, _: u32, _: &[OsString]) -> Result<()> {
Err(SandboxError::Unsupported(WHY.into()))
}
pub fn exec(_: &Path, _: &[OsString]) -> Result<std::convert::Infallible> {
Err(SandboxError::Unsupported(WHY.into()))
}
}
/// Closes every descriptor besides standard input, output and error, and refuses the launch if one
/// stays open or the list can't be read. One left open (a file outside the policy, a connected
/// socket) would survive `exec`, and the confinement doesn't revoke what is already open. Refusing
/// alone wouldn't do: in a desktop app the system's own libraries (Metal's shader cache on macOS)
/// leave files open without close-on-exec, and they reach every child.
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn close_inherited_descriptors() -> Result<()> {
for fd in descriptors_above_stdio()? {
// The listing's own descriptor is on the list and is closed already; that error is fine.
let _ = nix::unistd::close(fd);
}
let left = descriptors_above_stdio()?;
// The second listing holds its own directory open, so one above stdio is expected.
if left.len() > 1 {
return Err(SandboxError::NotEnforced(format!("descriptors other than standard input, output and error stayed open ({left:?})")));
}
Ok(())
}
/// The open descriptors above standard error, the listing's own included.
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn descriptors_above_stdio() -> Result<Vec<i32>> {
let dir = if cfg!(target_os = "linux") { "/proc/self/fd" } else { "/dev/fd" };
let listing = std::fs::read_dir(dir).map_err(|e| SandboxError::NotEnforced(format!("can't list open descriptors ({dir}): {e}")))?;
let mut above_stdio = Vec::new();
for entry in listing {
let entry = entry.map_err(|e| SandboxError::NotEnforced(format!("can't list open descriptors ({dir}): {e}")))?;
match entry.file_name().to_str().and_then(|n| n.parse::<i32>().ok()) {
Some(fd) if fd > 2 => above_stdio.push(fd),
Some(_) => {}
None => return Err(SandboxError::NotEnforced(format!("unexpected entry in {dir}"))),
}
}
above_stdio.sort_unstable();
Ok(above_stdio)
}
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
fn close_inherited_descriptors() -> Result<()> {
Ok(())
}
/// Replaces this process with `program` (the confining platforms).
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn exec_unix(program: &Path, args: &[OsString]) -> Result<std::convert::Infallible> {
use std::os::unix::process::CommandExt as _;
let mut cmd = Command::new(program);
cmd.args(args).env_clear().env("PATH", CONFINED_PATH);
for name in KEPT_ENV {
if let Some(v) = std::env::var_os(name) {
cmd.env(name, v);
}
}
// `exec` returns only when it failed.
Err(SandboxError::Io(cmd.exec()))
}
/// The only variables a confined program inherits, besides [`CONFINED_PATH`]: the locale and the
/// time zone. Everything else (tokens, credentials, the user's paths) stays with the parent.
pub const KEPT_ENV: [&str; 4] = ["LANG", "LC_ALL", "LC_CTYPE", "TZ"];
/// The `PATH` a confined program sees.
pub const CONFINED_PATH: &str = "/usr/bin:/bin";
#[cfg(test)]
mod tests;
+506
View File
@@ -0,0 +1,506 @@
//! Linux: the launcher confines itself with Landlock, then seccomp, then `exec`s the program,
//! which inherits both.
use std::collections::BTreeMap;
use std::ffi::OsString;
use std::path::Path;
use std::process::Command;
use landlock::{
ABI, Access, AccessFs, AccessNet, CompatLevel, Compatible, CreateRulesetError, Ruleset, RulesetAttr, RulesetCreatedAttr, RulesetError,
RulesetStatus, Scope, path_beneath_rules,
};
use seccompiler::{BpfProgram, SeccompAction, SeccompCmpArgLen, SeccompCmpOp, SeccompCondition, SeccompFilter, SeccompRule, TargetArch};
use crate::policy::Checked;
use crate::{Result, SandboxError, Support};
/// The oldest Landlock ABI a launch accepts, whose filesystem rights are required: the first that
/// covers truncating a file (Linux 6.2). Before it, `truncate` and `open` with `O_TRUNC` empty a file
/// whatever the rules say. A filter can't take its place: `openat2` hides its flags from seccomp, and
/// refusing `O_TRUNC` would refuse opening `/dev/null` to write, as `open(os.devnull, "w")` does.
const OLDEST: ABI = ABI::V3;
/// The newest Landlock ABI this crate knows. Rights it adds past [`OLDEST`] are requested on a
/// best-effort basis.
const NEWEST: ABI = ABI::V9;
// System calls the `libc` crate doesn't name on both architectures. Numbers from 424 up are the same
// on x86_64 and aarch64.
const SYS_FCHMODAT2: libc::c_long = 452;
const SYS_SETXATTRAT: libc::c_long = 463;
const SYS_REMOVEXATTRAT: libc::c_long = 466;
const SYS_OPEN_TREE_ATTR: libc::c_long = 467;
const SYS_FILE_SETATTR: libc::c_long = 469;
// `libc` names no kexec_file_load on riscv64 (0.2.190). It takes 294 from the kernel's generic
// table there, as on aarch64, where `libc` names it.
#[cfg(target_arch = "riscv64")]
const SYS_KEXEC_FILE_LOAD: libc::c_long = 294;
#[cfg(not(target_arch = "riscv64"))]
const SYS_KEXEC_FILE_LOAD: libc::c_long = libc::SYS_kexec_file_load;
pub fn support() -> Support {
match arch() {
Some(_) => Support::Available("Landlock and seccomp (checked again at each launch)".into()),
None => Support::Unsupported(format!("seccomp filters are built for x86_64 and aarch64, not {}", std::env::consts::ARCH)),
}
}
/// Linux runs the launcher directly: it confines itself.
pub fn command(_: &Checked, exe: &Path, launch: &[OsString]) -> Result<Command> {
arch().ok_or_else(|| SandboxError::Unsupported(format!("no seccomp filter for {}", std::env::consts::ARCH)))?;
let mut c = Command::new(exe);
c.args(launch);
Ok(c)
}
pub fn confine(policy: &Checked, _: u32, _: &[OsString]) -> Result<()> {
landlock(policy)?;
seccomp()
}
pub fn exec(program: &Path, args: &[OsString]) -> Result<std::convert::Infallible> {
crate::exec_unix(program, args)
}
fn not_enforced(e: impl std::fmt::Display) -> SandboxError {
SandboxError::NotEnforced(e.to_string())
}
/// A kernel without Landlock, or with it off, in the Landlock crate's own text. Only that case says
/// this, since the tests that skip on such a machine look for it: any other failure, such as too many
/// layers or a refused ruleset, comes from a machine that has Landlock, which must not be taken for
/// one that doesn't.
fn landlock_not_enforced(e: impl std::fmt::Display) -> SandboxError {
SandboxError::NotEnforced(format!("Landlock is not enforced: {e}"))
}
/// What a failed `landlock_create_ruleset` says about the kernel: `ENOSYS` is one built without
/// Landlock and `EOPNOTSUPP` one with it off at boot. Anything else, such as `EINVAL` or too many open
/// files, comes from a kernel that has it.
fn says_no_landlock(errno: Option<i32>) -> bool {
matches!(errno, Some(libc::ENOSYS | libc::EOPNOTSUPP))
}
/// A kernel without the rights a launch requires ([`OLDEST`]).
fn too_old_for_truncation(e: impl std::fmt::Display) -> SandboxError {
landlock_not_enforced(format!("this kernel has no Landlock, or one older than ABI 3 (Linux 6.2), which can't refuse truncating a file: {e}"))
}
fn ruleset_error(e: RulesetError) -> SandboxError {
let absent =
matches!(&e, RulesetError::CreateRuleset(CreateRulesetError::CreateRulesetCall { source, .. }) if says_no_landlock(source.raw_os_error()));
if absent { landlock_not_enforced(e) } else { not_enforced(e) }
}
/// Filesystem: every right Landlock handles is denied except reading the policy's read paths,
/// executing (and reading) its exec paths, and `/dev/null`. On kernels with network rights (ABI
/// 4), TCP binding and connecting are denied too; with scopes (ABI 6), signals to and abstract
/// Unix sockets of processes outside the sandbox.
fn landlock(policy: &Checked) -> Result<()> {
let ruleset = Ruleset::default()
.set_compatibility(CompatLevel::HardRequirement)
.handle_access(AccessFs::from_all(OLDEST))
.map_err(too_old_for_truncation)?
.set_compatibility(CompatLevel::BestEffort)
.handle_access(AccessFs::from_all(NEWEST))
.and_then(|r| r.handle_access(AccessNet::from_all(NEWEST)))
.and_then(|r| r.scope(Scope::from_all(NEWEST)))
.map_err(not_enforced)?
.create()
.map_err(ruleset_error)?;
let read = AccessFs::ReadFile | AccessFs::ReadDir;
let ruleset = ruleset
.set_compatibility(CompatLevel::HardRequirement)
.add_rules(path_beneath_rules(&policy.read, read))
.and_then(|r| r.add_rules(path_beneath_rules(&policy.exec, read | AccessFs::Execute)))
.and_then(|r| r.add_rules(path_beneath_rules(["/dev/null"], AccessFs::ReadFile | AccessFs::WriteFile)))
.map_err(not_enforced)?;
let status = ruleset.restrict_self().map_err(not_enforced)?;
if !status.no_new_privs {
return Err(SandboxError::NotEnforced("no_new_privs is not set".into()));
}
// The rights of `OLDEST` are a hard requirement, so "partially" means only newer rights are
// missing on this kernel; "not enforced" means nothing is.
if status.ruleset == RulesetStatus::NotEnforced {
return Err(SandboxError::NotEnforced(format!("Landlock is not enforced ({:?})", status.landlock)));
}
Ok(())
}
fn arch() -> Option<TargetArch> {
match std::env::consts::ARCH {
"x86_64" => Some(TargetArch::x86_64),
"aarch64" => Some(TargetArch::aarch64),
_ => None,
}
}
/// System calls that fail with `EPERM` in the sandbox, unconditionally.
fn denied_calls() -> Vec<libc::c_long> {
let calls = vec![
// Network: no socket of any family (a Unix datagram socket can send without
// `connect`), and no connection or listener on one the program might hold.
libc::SYS_socket,
libc::SYS_socketpair,
libc::SYS_connect,
libc::SYS_bind,
libc::SYS_listen,
libc::SYS_accept,
libc::SYS_accept4,
// io_uring runs operations, connections included, outside the seccomp filter.
libc::SYS_io_uring_setup,
libc::SYS_io_uring_enter,
libc::SYS_io_uring_register,
// Inspecting or changing other processes.
libc::SYS_ptrace,
libc::SYS_process_vm_readv,
libc::SYS_process_vm_writev,
libc::SYS_pidfd_getfd,
libc::SYS_pidfd_open,
libc::SYS_pidfd_send_signal,
libc::SYS_kcmp,
libc::SYS_get_robust_list,
libc::SYS_migrate_pages,
libc::SYS_move_pages,
// System V shared memory, message queues and semaphores: a segment or queue this user can
// open holds what another process put there, and no path rule covers them.
libc::SYS_shmget,
libc::SYS_shmat,
libc::SYS_shmctl,
libc::SYS_shmdt,
libc::SYS_msgget,
libc::SYS_msgsnd,
libc::SYS_msgrcv,
libc::SYS_msgctl,
libc::SYS_semget,
libc::SYS_semop,
libc::SYS_semctl,
libc::SYS_semtimedop,
// What changes a file without Landlock having a right for it: permissions, owner, extended
// attributes, times and flags, by path or by descriptor, on any file this user owns.
libc::SYS_fchmod,
libc::SYS_fchmodat,
SYS_FCHMODAT2,
libc::SYS_fchown,
libc::SYS_fchownat,
libc::SYS_setxattr,
libc::SYS_lsetxattr,
libc::SYS_fsetxattr,
SYS_SETXATTRAT,
libc::SYS_removexattr,
libc::SYS_lremovexattr,
libc::SYS_fremovexattr,
SYS_REMOVEXATTRAT,
libc::SYS_utimensat,
SYS_FILE_SETATTR,
// POSIX message queues, which no path rule covers either: another process of the same user can
// open one, by name.
libc::SYS_mq_open,
libc::SYS_mq_unlink,
libc::SYS_mq_timedsend,
libc::SYS_mq_timedreceive,
libc::SYS_mq_notify,
libc::SYS_mq_getsetattr,
// Mounts, namespaces and file handles (ways around path-based rules).
libc::SYS_mount,
libc::SYS_umount2,
libc::SYS_pivot_root,
libc::SYS_chroot,
libc::SYS_unshare,
libc::SYS_setns,
libc::SYS_open_by_handle_at,
libc::SYS_name_to_handle_at,
libc::SYS_open_tree,
SYS_OPEN_TREE_ATTR,
libc::SYS_move_mount,
libc::SYS_fsopen,
libc::SYS_fsconfig,
libc::SYS_fsmount,
libc::SYS_fspick,
libc::SYS_mount_setattr,
// The kernel's own extension points and system state.
libc::SYS_bpf,
libc::SYS_perf_event_open,
libc::SYS_userfaultfd,
libc::SYS_keyctl,
libc::SYS_add_key,
libc::SYS_request_key,
libc::SYS_init_module,
libc::SYS_finit_module,
libc::SYS_delete_module,
libc::SYS_kexec_load,
SYS_KEXEC_FILE_LOAD,
libc::SYS_swapon,
libc::SYS_swapoff,
libc::SYS_reboot,
libc::SYS_syslog,
];
// x86_64 keeps the older process and file calls and port I/O; aarch64 has none of them.
#[cfg(target_arch = "x86_64")]
let calls = [
calls,
vec![
libc::SYS_fork,
libc::SYS_vfork,
libc::SYS_iopl,
libc::SYS_ioperm,
libc::SYS_chmod,
libc::SYS_chown,
libc::SYS_lchown,
libc::SYS_utime,
libc::SYS_utimes,
libc::SYS_futimesat,
],
]
.concat();
calls
}
/// `who` and `which` of `setpriority` and `ioprio_set` for one process.
const PRIO_PROCESS: u64 = 0;
const IOPRIO_WHO_PROCESS: u64 = 1;
// `libc::Ioctl` differs by C library.
#[allow(clippy::unnecessary_cast)]
const TIOCSTI: u64 = libc::TIOCSTI as u64;
#[allow(clippy::unnecessary_cast)]
const TIOCLINUX: u64 = libc::TIOCLINUX as u64;
/// The bit of an `ioctl` request, on both architectures, that says it hands the kernel data.
const IOC_WRITE: u64 = 0x4000_0000;
/// Changes a directory, though its number says it only reads, as the kernel's headers have it.
const FS_IOC_SET_ENCRYPTION_POLICY: u64 = 0x800C_6613;
/// What the main filter refuses with `EPERM`: the calls above, and these when their arguments match.
/// - `clone` without `CLONE_THREAD`: a new process.
/// - A signal to any process but `me`, whose ID `exec` keeps. Landlock's signal scope needs a newer
/// kernel than the rest (ABI 6), and `kill(-1, 9)` must not get through on an older one.
/// - A change to the limits, priority or scheduling of any process but `me`.
/// - `TIOCSTI` and `TIOCLINUX`, which type into a terminal the program inherited, as when
/// `gridcraft-cli sandbox-check` runs a program from a shell.
/// - An `ioctl` that hands the kernel data ([`IOC_WRITE`]), as those that change a file's flags or
/// attributes do (`FS_IOC_SETFLAGS`, `FS_IOC_FSSETXATTR`) on a descriptor opened only to read.
/// Landlock's rights don't cover them for files. Those that ask, as `isatty` does, and those that
/// set a descriptor's own flags, as `FIONBIO` does, carry no such bit and still work.
///
/// Process IDs and `ioctl` requests are compared as the kernel reads them, in 32 bits, so high bits
/// set on the 64-bit argument don't get past a rule.
fn main_rules(me: u32) -> Result<BTreeMap<i64, Vec<SeccompRule>>> {
let me = u64::from(me);
let word = |arg: u8, op: SeccompCmpOp, value: u64| SeccompCondition::new(arg, SeccompCmpArgLen::Dword, op, value).map_err(not_enforced);
let rule = |conditions: Vec<SeccompCondition>| SeccompRule::new(conditions).map_err(not_enforced);
let mut rules: BTreeMap<i64, Vec<SeccompRule>> = denied_calls().into_iter().map(|nr| (nr, Vec::new())).collect();
let clone_thread = u64::try_from(libc::CLONE_THREAD).map_err(not_enforced)?;
let new_process = SeccompCondition::new(0, SeccompCmpArgLen::Qword, SeccompCmpOp::MaskedEq(clone_thread), 0).map_err(not_enforced)?;
rules.insert(libc::SYS_clone, vec![rule(vec![new_process])?]);
for nr in [libc::SYS_kill, libc::SYS_tkill, libc::SYS_tgkill, libc::SYS_rt_sigqueueinfo, libc::SYS_rt_tgsigqueueinfo] {
rules.insert(nr, vec![rule(vec![word(0, SeccompCmpOp::Ne, me)?])?]);
}
// These take 0 for the caller too.
for nr in [libc::SYS_prlimit64, libc::SYS_sched_setaffinity, libc::SYS_sched_setscheduler, libc::SYS_sched_setparam, libc::SYS_sched_setattr] {
rules.insert(nr, vec![rule(vec![word(0, SeccompCmpOp::Ne, me)?, word(0, SeccompCmpOp::Ne, 0)?])?]);
}
for (nr, one_process) in [(libc::SYS_setpriority, PRIO_PROCESS), (libc::SYS_ioprio_set, IOPRIO_WHO_PROCESS)] {
let another = rule(vec![word(1, SeccompCmpOp::Ne, me)?, word(1, SeccompCmpOp::Ne, 0)?])?;
rules.insert(nr, vec![rule(vec![word(0, SeccompCmpOp::Ne, one_process)?])?, another]);
}
rules.insert(
libc::SYS_ioctl,
vec![
rule(vec![word(1, SeccompCmpOp::Eq, TIOCSTI)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, TIOCLINUX)?])?,
rule(vec![word(1, SeccompCmpOp::MaskedEq(IOC_WRITE), IOC_WRITE)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, FS_IOC_SET_ENCRYPTION_POLICY)?])?,
],
);
Ok(rules)
}
/// The filters: the first answers `clone3` with `ENOSYS`, so the C library falls back to `clone`,
/// whose flags a filter can read (a thread passes `CLONE_THREAD`; a new process doesn't); the
/// second is [`main_rules`]; on x86_64 a third refuses every x32 call, which carries the same
/// architecture value but other numbers.
fn seccomp() -> Result<()> {
let arch = arch().ok_or_else(|| SandboxError::Unsupported(format!("no seccomp filter for {}", std::env::consts::ARCH)))?;
let program = |rules: BTreeMap<i64, Vec<SeccompRule>>, errno: i32| -> Result<BpfProgram> {
let errno = u32::try_from(errno).map_err(not_enforced)?;
SeccompFilter::new(rules, SeccompAction::Allow, SeccompAction::Errno(errno), arch).and_then(BpfProgram::try_from).map_err(not_enforced)
};
let clone3 = program(BTreeMap::from([(libc::SYS_clone3, Vec::new())]), libc::ENOSYS)?;
let main = program(main_rules(std::process::id())?, libc::EPERM)?;
// Landlock already set no_new_privs, which installing a filter without privileges needs.
if std::env::consts::ARCH == "x86_64" {
seccompiler::apply_filter(&x32_filter()).map_err(not_enforced)?;
}
seccompiler::apply_filter(&clone3).map_err(not_enforced)?;
seccompiler::apply_filter(&main).map_err(not_enforced)?;
Ok(())
}
/// The x32 bit of a system call number on x86_64.
const X32_SYSCALL_BIT: u32 = 0x4000_0000;
/// A filter that refuses with `EPERM` every call whose number has the x32 bit. x32 calls report
/// the x86_64 architecture value, so the other filters' architecture check lets them through and
/// their numbers match none of the rules. A call of another architecture is left to the other
/// filters, which end the process.
fn x32_filter() -> BpfProgram {
// `struct seccomp_data`: the call number at offset 0, the architecture at offset 4.
const LD_W_ABS: u16 = 0x20;
const JEQ_K: u16 = 0x15;
const JGE_K: u16 = 0x35;
const RET_K: u16 = 0x06;
const AUDIT_ARCH_X86_64: u32 = 62 | 0x8000_0000 | 0x4000_0000;
let op = |code: u16, jt: u8, jf: u8, k: u32| seccompiler::sock_filter { code, jt, jf, k };
let errno = u32::try_from(libc::EPERM).unwrap_or(1);
vec![
op(LD_W_ABS, 0, 0, 4),
op(JEQ_K, 1, 0, AUDIT_ARCH_X86_64),
op(RET_K, 0, 0, libc::SECCOMP_RET_ALLOW),
op(LD_W_ABS, 0, 0, 0),
op(JGE_K, 0, 1, X32_SYSCALL_BIT),
op(RET_K, 0, 0, libc::SECCOMP_RET_ERRNO | errno),
op(RET_K, 0, 0, libc::SECCOMP_RET_ALLOW),
]
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn the_filters_build_for_both_architectures() {
// Building only: installing one would confine the test process.
for arch in [TargetArch::x86_64, TargetArch::aarch64] {
let f = SeccompFilter::new(main_rules(4242).unwrap(), SeccompAction::Allow, SeccompAction::Errno(1), arch);
assert!(f.and_then(BpfProgram::try_from).is_ok(), "{arch:?}");
}
}
#[test]
fn the_calls_that_reach_another_process_have_rules() {
let rules = main_rules(4242).unwrap();
for nr in [
libc::SYS_kill,
libc::SYS_tkill,
libc::SYS_tgkill,
libc::SYS_rt_sigqueueinfo,
libc::SYS_rt_tgsigqueueinfo,
libc::SYS_prlimit64,
libc::SYS_setpriority,
libc::SYS_ioprio_set,
libc::SYS_sched_setaffinity,
libc::SYS_sched_setscheduler,
libc::SYS_sched_setparam,
libc::SYS_sched_setattr,
] {
assert_eq!(rules.get(&nr).map(Vec::len), Some(if nr == libc::SYS_setpriority || nr == libc::SYS_ioprio_set { 2 } else { 1 }), "{nr}");
}
for nr in [
libc::SYS_pidfd_open,
libc::SYS_pidfd_send_signal,
libc::SYS_kcmp,
libc::SYS_get_robust_list,
libc::SYS_migrate_pages,
libc::SYS_move_pages,
libc::SYS_shmget,
libc::SYS_shmat,
libc::SYS_msgget,
libc::SYS_semget,
libc::SYS_mq_open,
libc::SYS_mq_unlink,
libc::SYS_mq_timedsend,
libc::SYS_mq_timedreceive,
libc::SYS_mq_notify,
libc::SYS_mq_getsetattr,
libc::SYS_syslog,
] {
assert_eq!(rules.get(&nr).map(Vec::len), Some(0), "{nr} is refused outright");
}
assert_eq!(rules.get(&libc::SYS_ioctl).map(Vec::len), Some(4));
}
#[test]
fn what_landlock_has_no_right_for_is_refused_outright() {
let rules = main_rules(4242).unwrap();
let every_architecture = [
(libc::SYS_fchmod, "fchmod"),
(libc::SYS_fchmodat, "fchmodat"),
(SYS_FCHMODAT2, "fchmodat2"),
(libc::SYS_fchown, "fchown"),
(libc::SYS_fchownat, "fchownat"),
(libc::SYS_setxattr, "setxattr"),
(libc::SYS_lsetxattr, "lsetxattr"),
(libc::SYS_fsetxattr, "fsetxattr"),
(SYS_SETXATTRAT, "setxattrat"),
(libc::SYS_removexattr, "removexattr"),
(libc::SYS_lremovexattr, "lremovexattr"),
(libc::SYS_fremovexattr, "fremovexattr"),
(SYS_REMOVEXATTRAT, "removexattrat"),
(libc::SYS_utimensat, "utimensat"),
(SYS_FILE_SETATTR, "file_setattr"),
(SYS_OPEN_TREE_ATTR, "open_tree_attr"),
];
for (nr, name) in every_architecture {
assert_eq!(rules.get(&nr).map(Vec::len), Some(0), "{name}");
}
#[cfg(target_arch = "x86_64")]
for (nr, name) in [
(libc::SYS_chmod, "chmod"),
(libc::SYS_chown, "chown"),
(libc::SYS_lchown, "lchown"),
(libc::SYS_utime, "utime"),
(libc::SYS_utimes, "utimes"),
(libc::SYS_futimesat, "futimesat"),
] {
assert_eq!(rules.get(&nr).map(Vec::len), Some(0), "{name}");
}
}
#[test]
fn the_numbers_written_here_are_the_kernels() {
// `libc` names this one on x86_64, so a slip in the shared table shows there.
#[cfg(target_arch = "x86_64")]
assert_eq!(SYS_FCHMODAT2, libc::SYS_fchmodat2);
assert_eq!((SYS_SETXATTRAT, SYS_REMOVEXATTRAT, SYS_OPEN_TREE_ATTR, SYS_FILE_SETATTR), (463, 466, 467, 469));
// The generic table's number, which riscv64 writes here, is the one `libc` gives aarch64.
#[cfg(target_arch = "aarch64")]
assert_eq!(libc::SYS_kexec_file_load, 294);
}
#[test]
fn a_kernel_that_cant_refuse_truncating_a_file_is_not_accepted() {
assert!(AccessFs::from_all(OLDEST).contains(AccessFs::Truncate));
assert!(!AccessFs::from_all(ABI::V2).contains(AccessFs::Truncate), "the oldest ABI with the right");
// It reads as a machine that can't confine, which the tests that skip on one look for.
let e = too_old_for_truncation("the crate's text").to_string();
assert!(e.contains("Landlock is not enforced:") && e.contains("Linux 6.2") && e.contains("the crate's text"), "{e}");
}
#[test]
fn a_kernel_without_landlock_says_so() {
// What the tests and a person read to tell this from a sandbox that failed its check.
assert!(landlock_not_enforced("the crate's text").to_string().contains("Landlock is not enforced: the crate's text"));
}
#[test]
fn only_a_missing_or_disabled_landlock_says_it_is_not_enforced() {
assert!(says_no_landlock(Some(libc::ENOSYS)));
assert!(says_no_landlock(Some(libc::EOPNOTSUPP)));
for errno in [libc::EINVAL, libc::E2BIG, libc::EFAULT, libc::EMFILE, libc::ENOMEM, libc::EPERM] {
assert!(!says_no_landlock(Some(errno)), "{errno}");
}
assert!(!says_no_landlock(None));
// A ruleset failure of another kind, from the crate itself, on any kernel.
let e = Ruleset::default().create().err().map(ruleset_error).map(|e| e.to_string());
assert!(e.is_some_and(|e| !e.contains("Landlock is not enforced")));
}
#[test]
fn the_x32_filter_refuses_numbers_with_the_x32_bit() {
let f = x32_filter();
assert_eq!(f.len(), 7);
// Load the number, then jump past the refusal for numbers below the x32 bit.
assert_eq!((f[3].code, f[3].k), (0x20, 0));
assert_eq!((f[4].code, f[4].jt, f[4].jf, f[4].k), (0x35, 0, 1, X32_SYSCALL_BIT));
assert_eq!(f[5].k, libc::SECCOMP_RET_ERRNO | 1);
}
}
+92
View File
@@ -0,0 +1,92 @@
//! macOS: the launcher runs itself again under `sandbox-exec` with a deny-by-default profile built
//! from the policy, checks that the profile holds, and `exec`s the program, which stays under it.
use std::ffi::OsString;
use std::os::unix::process::CommandExt as _;
use std::path::Path;
use std::process::Command;
use crate::policy::Checked;
use crate::{CONFINED_FLAG, Result, SandboxError, Support};
const SANDBOX_EXEC: &str = "/usr/bin/sandbox-exec";
pub fn support() -> Support {
if Path::new(SANDBOX_EXEC).exists() {
Support::Available("sandbox-exec with a deny-by-default profile (checked again at each launch)".into())
} else {
Support::Unsupported(format!("{SANDBOX_EXEC} is missing"))
}
}
/// The launcher starts unconfined, so it can look at what it inherited first ([`confine`]). A
/// policy the profile can't express is refused here, before anything starts.
pub fn command(policy: &Checked, exe: &Path, launch: &[OsString]) -> Result<Command> {
if !Path::new(SANDBOX_EXEC).exists() {
return Err(SandboxError::Unsupported(format!("{SANDBOX_EXEC} is missing")));
}
profile(policy)?;
let mut c = Command::new(exe);
c.args(launch);
Ok(c)
}
/// Runs the launcher again under `sandbox-exec`, with the profile applied before it starts; the
/// enforcement check of that run proves it. Returns only on failure.
pub fn confine(policy: &Checked, target: u32, args: &[OsString]) -> Result<()> {
let exe = std::env::current_exe()?;
let mut c = Command::new(SANDBOX_EXEC);
c.arg("-p").arg(profile(policy)?).arg(exe).arg(CONFINED_FLAG).arg(target.to_string()).args(args);
Err(SandboxError::Io(c.exec()))
}
pub fn exec(program: &Path, args: &[OsString]) -> Result<std::convert::Infallible> {
crate::exec_unix(program, args)
}
/// The profile: everything is denied except what the base system profile allows a process to
/// start (`system.sb`: the system libraries, the dynamic loader's cache and a few system
/// services), reading the policy's paths, executing its programs, reading file metadata (path
/// resolution), reading system settings (`sysctl`) and writing `/dev/null`. There is no rule for
/// starting processes, writing files or the network, so they stay denied.
pub fn profile(policy: &Checked) -> Result<String> {
let mut p = String::from("(version 1)\n(deny default)\n(import \"system.sb\")\n");
p.push_str("(allow process-exec");
for path in &policy.exec {
p.push_str(&path_rule(path)?);
}
p.push_str(")\n(allow file-read*");
for path in policy.exec.iter().chain(&policy.read) {
p.push_str(&path_rule(path)?);
}
p.push_str(")\n(allow file-read-metadata)\n(allow sysctl-read)\n(allow file-write-data (literal \"/dev/null\"))\n");
Ok(p)
}
/// A profile rule for `path`. A literal matches only that path, so a directory, whose contents the
/// policy means, as Landlock takes it on Linux, is a subpath.
fn path_rule(path: &Path) -> Result<String> {
let rule = if path.is_dir() { "subpath" } else { "literal" };
Ok(format!(" ({rule} {})", quoted(path)?))
}
/// `path` as a profile string literal: in double quotes, with `"` and `\` escaped. Paths with
/// other control characters (the policy already refuses NUL and line breaks) or that aren't UTF-8
/// are refused rather than quoted.
pub fn quoted(path: &Path) -> Result<String> {
let s = path.to_str().ok_or_else(|| SandboxError::BadPolicy(format!("{} is not UTF-8", path.display())))?;
let mut out = String::with_capacity(s.len() + 2);
out.push('"');
for c in s.chars() {
match c {
'"' | '\\' => {
out.push('\\');
out.push(c);
}
c if c.is_control() => return Err(SandboxError::BadPolicy(format!("{} holds a control character", path.display()))),
c => out.push(c),
}
}
out.push('"');
Ok(out)
}
+214
View File
@@ -0,0 +1,214 @@
//! What a confined program may touch, and how the launcher receives it on its command line.
use std::ffi::{OsStr, OsString};
use std::path::{Path, PathBuf};
use crate::{Canary, Result, SandboxError};
/// At most this many paths in each list.
pub const MAX_PATHS: usize = 64;
/// At most this many bytes in one path.
pub const MAX_PATH_BYTES: usize = 4096;
/// The paths a confined program may read and the programs it may execute. Everything else is
/// denied: writing anywhere, network access and starting other processes have no switch.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Policy {
read: Vec<PathBuf>,
exec: Vec<PathBuf>,
}
impl Policy {
pub fn new() -> Policy {
Policy::default()
}
/// Lets the program read `path`: a file, or a directory and everything under it.
pub fn read(mut self, path: impl Into<PathBuf>) -> Policy {
self.read.push(path.into());
self
}
/// Lets the program execute (and read) the file `path`.
pub fn exec(mut self, path: impl Into<PathBuf>) -> Policy {
self.exec.push(path.into());
self
}
/// Adds what a dynamically linked program needs to start on this system: the shared-library
/// directories and the dynamic loader's cache on Linux (macOS's base profile covers its own).
/// Paths that don't exist here are skipped.
pub fn with_system_libraries(mut self) -> Policy {
if cfg!(target_os = "linux") {
for dir in ["/lib", "/lib64", "/usr/lib", "/usr/lib64"] {
// The dynamic loader is executed, so its directories get execute access.
if Path::new(dir).exists() {
self.exec.push(dir.into());
}
}
// What the C library and Python's `zoneinfo` read for the time zone.
for path in ["/etc/ld.so.cache", "/etc/localtime", "/usr/share/zoneinfo"] {
if Path::new(path).exists() {
self.read.push(path.into());
}
}
}
// Fonts, which a plot's text is drawn with. Only the system's: fonts in the home folder stay
// out of reach. macOS's own `/System/Library/Fonts` comes with its base profile.
let fonts: &[&str] = if cfg!(target_os = "linux") {
&["/usr/share/fonts", "/usr/local/share/fonts"]
} else if cfg!(target_os = "macos") {
&["/Library/Fonts"]
} else {
&[]
};
for dir in fonts {
if Path::new(dir).exists() {
self.read.push((*dir).into());
}
}
self
}
pub fn read_paths(&self) -> &[PathBuf] {
&self.read
}
pub fn exec_paths(&self) -> &[PathBuf] {
&self.exec
}
/// The policy as the launcher will apply it: every path absolute, existing and canonical, the
/// lists within their caps, the current executable `exe` executable (the enforcement check
/// starts it), `program` executable, and the canary outside every allowed path.
pub(crate) fn checked(&self, exe: &Path, program: &Path, canary: &Canary) -> Result<Checked> {
let read = canonical_all(&self.read)?;
let mut exec = canonical_all(&self.exec)?;
let program = canonical(program)?;
for extra in [canonical(exe)?, program.clone()] {
if !exec.contains(&extra) {
exec.push(extra);
}
}
if exec.len() > MAX_PATHS {
return Err(SandboxError::BadPolicy(format!("more than {MAX_PATHS} executable paths")));
}
let probe = canonical(canary.file())?;
if read.iter().chain(&exec).any(|p| probe.starts_with(p)) {
return Err(SandboxError::BadPolicy(format!("the canary {} is inside an allowed path", probe.display())));
}
Ok(Checked { read, exec, program, args: Vec::new() })
}
}
fn canonical_all(paths: &[PathBuf]) -> Result<Vec<PathBuf>> {
if paths.len() > MAX_PATHS {
return Err(SandboxError::BadPolicy(format!("more than {MAX_PATHS} paths")));
}
let mut out: Vec<PathBuf> = Vec::with_capacity(paths.len());
for p in paths {
let c = canonical(p)?;
if !out.contains(&c) {
out.push(c);
}
}
Ok(out)
}
/// `path` made canonical, after refusing what can't be one: an empty or relative path, one over
/// [`MAX_PATH_BYTES`], one with a NUL or a line break (which no sandbox profile can quote), and
/// one that doesn't exist.
fn canonical(path: &Path) -> Result<PathBuf> {
let bytes = path.as_os_str().as_encoded_bytes();
if bytes.is_empty() || !path.is_absolute() {
return Err(SandboxError::BadPolicy(format!("{} is not an absolute path", path.display())));
}
if bytes.len() > MAX_PATH_BYTES {
return Err(SandboxError::BadPolicy(format!("a path is longer than {MAX_PATH_BYTES} bytes")));
}
if bytes.iter().any(|b| matches!(b, 0 | b'\n' | b'\r')) {
return Err(SandboxError::BadPolicy("a path holds a NUL or a line break".into()));
}
let c = path.canonicalize().map_err(|e| SandboxError::BadPolicy(format!("{}: {e}", path.display())))?;
if c.as_os_str().as_encoded_bytes().len() > MAX_PATH_BYTES {
return Err(SandboxError::BadPolicy(format!("a path is longer than {MAX_PATH_BYTES} bytes")));
}
Ok(c)
}
/// A policy checked by [`Policy::checked`], with the program and its arguments: what the launcher
/// applies.
#[derive(Clone, Debug, PartialEq, Eq)]
pub(crate) struct Checked {
pub read: Vec<PathBuf>,
pub exec: Vec<PathBuf>,
program: PathBuf,
args: Vec<OsString>,
}
impl Checked {
pub fn program(&self) -> &Path {
&self.program
}
pub fn args(&self) -> &[OsString] {
&self.args
}
/// The launcher's options for this policy (`--read`, `--exec`, `--probe`), without the
/// program, which follows a `--`.
pub fn to_args(&self, canary: &Path) -> Vec<OsString> {
let mut out = Vec::new();
for (flag, paths) in [("--read", &self.read), ("--exec", &self.exec)] {
for p in paths {
out.push(OsString::from(flag));
out.push(p.as_os_str().to_owned());
}
}
out.push(OsString::from("--probe"));
out.push(canary.as_os_str().to_owned());
out
}
/// Reads the launcher's arguments back: options, `--`, the program and its arguments. The
/// paths are checked again (the launcher trusts nothing), and the canary is required.
pub fn from_args(args: &[OsString]) -> Result<(Checked, PathBuf)> {
let mut read = Vec::new();
let mut exec = Vec::new();
let mut probe: Option<PathBuf> = None;
let mut it = args.iter();
loop {
let Some(flag) = it.next() else { return Err(SandboxError::BadPolicy("no program after `--`".into())) };
if flag == OsStr::new("--") {
break;
}
let value = it.next().ok_or_else(|| SandboxError::BadPolicy(format!("{} needs a path", flag.to_string_lossy())))?;
let path = canonical(Path::new(value))?;
match flag.to_str() {
Some("--read") => read.push(path),
Some("--exec") => exec.push(path),
Some("--probe") if probe.is_none() => probe = Some(path),
_ => return Err(SandboxError::BadPolicy(format!("unknown launcher option {}", flag.to_string_lossy()))),
}
if read.len() > MAX_PATHS || exec.len() > MAX_PATHS {
return Err(SandboxError::BadPolicy(format!("more than {MAX_PATHS} paths")));
}
}
let program = it.next().ok_or_else(|| SandboxError::BadPolicy("no program after `--`".into()))?;
let program = canonical(Path::new(program))?;
if !exec.contains(&program) {
return Err(SandboxError::BadPolicy(format!("{} is not an executable path of the policy", program.display())));
}
let probe = probe.ok_or_else(|| SandboxError::BadPolicy("no canary (`--probe`)".into()))?;
if read.iter().chain(&exec).any(|p| probe.starts_with(p)) {
return Err(SandboxError::BadPolicy(format!("the canary {} is inside an allowed path", probe.display())));
}
// The enforcement check starts the current executable. Outside the executable paths, the file
// rules would refuse that start, and the check would prove nothing about new processes.
let exe = canonical(&std::env::current_exe()?)?;
if !exec.iter().any(|p| exe.starts_with(p)) {
return Err(SandboxError::BadPolicy(format!("the current executable {} is not an executable path of the policy", exe.display())));
}
Ok((Checked { read, exec, program, args: it.cloned().collect() }, probe))
}
}
+248
View File
@@ -0,0 +1,248 @@
//! The enforcement check a confined launcher runs before it starts its program, and the fuller
//! report `gridcraft-cli sandbox-check` prints.
use std::ffi::OsString;
use std::io::ErrorKind;
use std::net::{SocketAddr, TcpStream, UdpSocket};
use std::path::Path;
use std::process::{Command, Stdio};
use std::time::Duration;
use crate::PROBE_CHILD_FLAG;
/// What one attempt found.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Outcome {
/// The attempt was refused by the system, as the sandbox requires.
Denied,
/// The attempt was not refused (it worked, or failed for another reason, which doesn't prove
/// the sandbox); the text says what happened.
Allowed(String),
}
/// One attempt and its outcome.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Check {
pub name: &'static str,
pub outcome: Outcome,
}
/// A loopback address with nothing to talk to: unconfined, connecting fails with "refused", which
/// is not a denial; confined, creating the socket or connecting is denied.
const TCP_TARGET: ([u8; 4], u16) = ([127, 0, 0, 1], 9);
/// Every attempt that must be denied, against `canary` (a file outside the policy), `exe` (the
/// current executable, which the process check tries to start) and `target` (from
/// [`signal_target`]). Each must report [`Outcome::Denied`] before the launcher runs its program.
pub fn enforcement(canary: &Path, exe: &Path, target: u32) -> Vec<Check> {
vec![
Check { name: "read the canary", outcome: denied(std::fs::read(canary).map(drop)) },
Check { name: "change the canary's permissions", outcome: denied(chmod_probe(canary)) },
Check { name: "truncate the canary", outcome: denied(truncate_probe(canary)) },
Check { name: "write next to the canary", outcome: write_probe(canary.parent().unwrap_or(canary)) },
Check { name: "write in the temporary directory", outcome: write_probe(&std::env::temp_dir()) },
Check { name: "TCP connection", outcome: denied(tcp_probe()) },
Check { name: "start a process", outcome: spawn_probe(exe) },
Check { name: "signal another process", outcome: signal_probe(target) },
]
}
/// Asks for the permissions the file has, so nothing changes even where this is allowed.
fn chmod_probe(file: &Path) -> std::io::Result<()> {
std::fs::set_permissions(file, std::fs::metadata(file)?.permissions())
}
/// Asks for the length the file has, so nothing changes even where this is allowed.
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn truncate_probe(file: &Path) -> std::io::Result<()> {
let length = nix::libc::off_t::try_from(std::fs::metadata(file)?.len()).map_err(|e| std::io::Error::new(ErrorKind::InvalidInput, e))?;
nix::unistd::truncate(file, length).map_err(std::io::Error::from)
}
/// Other systems have no confinement, so no launch gets this far.
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
fn truncate_probe(_: &Path) -> std::io::Result<()> {
Err(std::io::Error::new(ErrorKind::Unsupported, "this system has no sandbox"))
}
/// Signal 0 to `process`, which delivers nothing: it only asks whether a signal would be allowed.
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn signal_zero(process: u32) -> std::io::Result<()> {
let pid = i32::try_from(process).map_err(|e| std::io::Error::new(ErrorKind::InvalidInput, e))?;
nix::sys::signal::kill(nix::unistd::Pid::from_raw(pid), None).map_err(std::io::Error::from)
}
/// The process the check tries a signal on: this launcher's parent, if a signal to it is allowed
/// now, before any confinement. One that refuses already (PID 1, another user's process) would be
/// refused afterwards too, which would prove nothing, so the launch is refused instead.
#[cfg(any(target_os = "linux", target_os = "macos"))]
pub fn signal_target() -> std::result::Result<u32, String> {
signalable(std::os::unix::process::parent_id())
}
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn signalable(process: u32) -> std::result::Result<u32, String> {
signal_zero(process)
.map(|()| process)
.map_err(|e| format!("a signal to process {process} fails before any confinement, so its refusal would prove nothing: {e}"))
}
/// Whether a signal to `target` is refused now. A parent that changed since [`signal_target`] looked
/// is another process, which may refuse for its own reasons (a launcher whose parent exited has PID 1
/// for a parent).
#[cfg(any(target_os = "linux", target_os = "macos"))]
fn signal_probe(target: u32) -> Outcome {
if std::os::unix::process::parent_id() != target {
return Outcome::Allowed(format!("this launcher's parent is no longer process {target}"));
}
denied(signal_zero(target))
}
/// Other systems have no confinement, so no launch gets this far.
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
pub fn signal_target() -> std::result::Result<u32, String> {
Err("this system has no sandbox".into())
}
/// Other systems have no confinement, so no launch gets this far.
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
fn signal_probe(_: u32) -> Outcome {
Outcome::Allowed("this system has no sandbox".into())
}
/// `Denied` when `r` failed with "permission denied" (`EACCES` or `EPERM`), else what happened.
fn denied(r: std::io::Result<()>) -> Outcome {
match r {
Ok(()) => Outcome::Allowed("it worked".into()),
Err(e) if e.kind() == ErrorKind::PermissionDenied => Outcome::Denied,
Err(e) => Outcome::Allowed(format!("it failed for another reason: {e}")),
}
}
fn write_probe(dir: &Path) -> Outcome {
let file = dir.join(format!("gridcraft-sandbox-write-probe-{}", std::process::id()));
let r = std::fs::OpenOptions::new().write(true).create_new(true).open(&file);
if r.is_ok() {
let _ = std::fs::remove_file(&file);
}
denied(r.map(drop))
}
fn tcp_probe() -> std::io::Result<()> {
TcpStream::connect_timeout(&SocketAddr::from(TCP_TARGET), Duration::from_secs(2)).map(drop)
}
fn spawn_probe(exe: &Path) -> Outcome {
let child = Command::new(exe).arg(PROBE_CHILD_FLAG).stdin(Stdio::null()).stdout(Stdio::null()).stderr(Stdio::null()).spawn();
match child {
Ok(mut c) => {
let _ = c.wait();
Outcome::Allowed("a process started".into())
}
// A start refused by the sandbox: no new process (seccomp, or the macOS profile).
Err(e) if e.kind() == ErrorKind::PermissionDenied => Outcome::Denied,
Err(e) => Outcome::Allowed(format!("it failed for another reason: {e}")),
}
}
/// `--sandbox-report <canary> [<readable file> [<home directory>]]`, run confined: prints one line per attempt,
/// `denied: <name>` or `allowed: <name> (<detail>)`, then the attempts that must work, and exits
/// with 0 when every line is as the sandbox requires, 1 otherwise. Used by
/// `gridcraft-cli sandbox-check` and the tests.
pub fn report(args: &[OsString]) -> i32 {
let (Some(canary), readable) = (args.first(), args.get(1)) else {
eprintln!("gridcraft-sandbox: --sandbox-report needs the canary's path");
return 2;
};
let exe = std::env::current_exe().unwrap_or_default();
let mut ok = true;
// The launcher looked at this process's parent before it confined itself, and `exec` keeps it.
#[cfg(any(target_os = "linux", target_os = "macos"))]
let target = std::os::unix::process::parent_id();
#[cfg(not(any(target_os = "linux", target_os = "macos")))]
let target = 0;
let mut checks = enforcement(Path::new(canary), &exe, target);
checks.push(Check { name: "UDP socket", outcome: denied(UdpSocket::bind(SocketAddr::from(([127, 0, 0, 1], 0))).map(drop)) });
// The person's own files: not part of the launch check, since a policy may allow a directory
// under the home directory (a Python installed there), but no policy here allows the home
// directory itself.
// The confined process inherits no `HOME` (its environment is cleared), so the directory is
// an argument.
if let Some(home) = args.get(2).filter(|h| !h.is_empty()) {
checks.push(Check { name: "list the home directory", outcome: denied(std::fs::read_dir(home).map(drop)) });
}
for c in &checks {
match &c.outcome {
Outcome::Denied => println!("denied: {}", c.name),
Outcome::Allowed(detail) => {
ok = false;
println!("allowed: {} ({detail})", c.name);
}
}
}
// What a confined program still needs.
let thread = std::thread::Builder::new().spawn(|| 2 + 2).map(|h| h.join().is_ok());
ok &= worked("start a thread", thread.as_ref().map(|j| *j).unwrap_or(false), thread.err().map(|e| e.to_string()));
if let Some(file) = readable {
let r = std::fs::read(file);
ok &= worked("read an allowed file", r.is_ok(), r.err().map(|e| e.to_string()));
}
i32::from(!ok)
}
fn worked(name: &str, ok: bool, err: Option<String>) -> bool {
if ok {
println!("works: {name}");
} else {
println!("broken: {name} ({})", err.unwrap_or_default());
}
ok
}
#[cfg(all(test, any(target_os = "linux", target_os = "macos")))]
mod tests {
use super::*;
#[test]
fn a_signal_that_fails_before_any_confinement_proves_nothing_after_it() {
// This test isn't confined, and its parent has the same user.
let parent = std::os::unix::process::parent_id();
assert_eq!(signal_target(), Ok(parent));
assert!(matches!(signal_probe(parent), Outcome::Allowed(_)), "allowed unconfined, so no denial");
// No such process: there is nothing to compare a refusal with, so the launch is refused.
for process in [u32::MAX, i32::MAX as u32] {
let e = signalable(process).unwrap_err();
assert!(e.contains("before any confinement"), "{e}");
}
}
#[test]
fn the_checks_on_the_canary_change_nothing_where_they_are_allowed() {
let canary = crate::Canary::create().unwrap();
let file = canary.file();
let state = || {
let m = std::fs::metadata(file).unwrap();
(std::fs::read(file).unwrap(), m.permissions(), m.len())
};
let before = state();
// This test isn't confined, so each works, which isn't a denial.
assert_eq!(denied(chmod_probe(file)), Outcome::Allowed("it worked".into()));
assert_eq!(denied(truncate_probe(file)), Outcome::Allowed("it worked".into()));
assert_eq!(state(), before);
}
#[test]
fn a_missing_canary_proves_nothing() {
let canary = crate::Canary::create().unwrap();
let gone = canary.file().with_file_name("not there");
for outcome in [denied(chmod_probe(&gone)), denied(truncate_probe(&gone))] {
assert!(matches!(&outcome, Outcome::Allowed(detail) if detail.contains("another reason")), "{outcome:?}");
}
}
#[test]
fn a_parent_that_changed_is_not_taken_for_the_one_looked_at() {
let other = std::os::unix::process::parent_id().wrapping_add(1);
assert!(matches!(signal_probe(other), Outcome::Allowed(detail) if detail.contains("no longer")));
}
}
+160
View File
@@ -0,0 +1,160 @@
use std::ffi::OsString;
use std::path::{Path, PathBuf};
use crate::policy::{Checked, MAX_PATH_BYTES, MAX_PATHS};
use crate::{Canary, PROBE_CHILD_FLAG, Policy, SandboxError, handle_args};
fn os(args: &[&str]) -> Vec<OsString> {
args.iter().map(OsString::from).collect()
}
fn bad_policy<T: std::fmt::Debug>(r: crate::Result<T>) -> String {
match r {
Err(SandboxError::BadPolicy(why)) => why,
other => panic!("expected a bad-policy error, got {other:?}"),
}
}
/// A directory of its own with a file in it, outside the system's temporary directory's canary.
fn scratch() -> (Canary, PathBuf) {
let canary = Canary::create().unwrap();
let dir =
std::env::temp_dir().canonicalize().unwrap().join(format!("gridcraft-sandbox-test-{}-{:?}", std::process::id(), std::thread::current().id()));
std::fs::create_dir_all(&dir).unwrap();
std::fs::write(dir.join("data.txt"), "data").unwrap();
(canary, dir)
}
#[test]
fn a_checked_policy_is_canonical_and_executes_the_launcher_and_the_program() {
let (canary, dir) = scratch();
let exe = std::env::current_exe().unwrap().canonicalize().unwrap();
let file = dir.join("data.txt");
let checked = Policy::new().read(&dir).read(&dir).exec(&file).checked(&exe, &file, &canary).unwrap();
assert_eq!(checked.read, vec![dir.clone()], "duplicates collapse");
assert_eq!(checked.exec, vec![file.clone(), exe.clone()], "the launcher joins the program");
assert_eq!(checked.program(), file);
std::fs::remove_dir_all(dir).unwrap();
}
#[test]
fn what_a_policy_refuses() {
let (canary, dir) = scratch();
let exe = std::env::current_exe().unwrap();
let ok = dir.join("data.txt");
let check = |p: Policy| bad_policy(p.checked(&exe, &ok, &canary));
assert!(check(Policy::new().read("relative/path")).contains("absolute"));
assert!(check(Policy::new().read("")).contains("absolute"));
assert!(check(Policy::new().read(dir.join("missing"))).contains("missing"));
assert!(check(Policy::new().read(PathBuf::from(format!("/{}", "a".repeat(MAX_PATH_BYTES))))).contains("longer"));
let mut many = Policy::new();
for _ in 0..=MAX_PATHS {
many = many.read(&dir);
}
assert!(check(many).contains("more than"));
// The canary must stay outside every allowed path, or the check would read it.
let around = canary.file().parent().unwrap().to_path_buf();
assert!(check(Policy::new().read(around)).contains("canary"));
#[cfg(unix)]
{
use std::os::unix::ffi::OsStrExt as _;
let line_break = PathBuf::from(std::ffi::OsStr::from_bytes(b"/tmp/a\nb"));
assert!(check(Policy::new().read(line_break)).contains("line break"));
}
std::fs::remove_dir_all(dir).unwrap();
}
#[test]
fn the_launcher_reads_its_arguments_back() {
let (canary, dir) = scratch();
let exe = std::env::current_exe().unwrap();
let program = dir.join("data.txt");
let checked = Policy::new().read(&dir).checked(&exe, &program, &canary).unwrap();
let mut args = checked.to_args(canary.file());
args.push("--".into());
args.push(program.clone().into());
args.extend(os(&["-c", "print(1)"]));
let (back, probe) = Checked::from_args(&args).unwrap();
assert_eq!(probe, canary.file().canonicalize().unwrap());
assert_eq!((back.read.clone(), back.exec.clone()), (checked.read.clone(), checked.exec.clone()));
assert_eq!(back.args(), os(&["-c", "print(1)"]).as_slice());
let p = program.to_string_lossy().into_owned();
let c = canary.file().to_string_lossy().into_owned();
// No canary, a program the policy doesn't execute, an unknown option, a missing program.
assert!(bad_policy(Checked::from_args(&os(&["--exec", &p, "--", &p]))).contains("canary"));
assert!(bad_policy(Checked::from_args(&os(&["--probe", &c, "--", &p]))).contains("not an executable path"));
assert!(bad_policy(Checked::from_args(&os(&["--write", &p, "--probe", &c, "--", &p]))).contains("unknown"));
assert!(bad_policy(Checked::from_args(&os(&["--exec", &p, "--probe", &c, "--"]))).contains("no program"));
assert!(bad_policy(Checked::from_args(&os(&["--exec", &p, "--probe", &c]))).contains("no program"));
assert!(bad_policy(Checked::from_args(&os(&["--exec"]))).contains("needs a path"));
// A second canary isn't taken.
assert!(bad_policy(Checked::from_args(&os(&["--exec", &p, "--probe", &c, "--probe", &c, "--", &p]))).contains("unknown"));
// The enforcement check starts the current executable, so a hand-built list must let it.
assert!(bad_policy(Checked::from_args(&os(&["--exec", &p, "--probe", &c, "--", &p]))).contains("current executable"));
std::fs::remove_dir_all(dir).unwrap();
}
#[test]
fn only_the_sandbox_flags_are_handled() {
assert_eq!(handle_args(&[]), None);
assert_eq!(handle_args(&os(&["--control", "7979"])), None);
assert_eq!(handle_args(&os(&[PROBE_CHILD_FLAG])), Some(0));
}
/// A launch whose arguments are refused runs nothing and leaves the process's descriptors alone:
/// closing them here would pull files out from under the tests running beside this one.
#[test]
fn a_launch_refused_for_its_arguments_closes_nothing() {
use std::io::Read as _;
let (_canary, dir) = scratch();
let mut held = std::fs::File::open(dir.join("data.txt")).unwrap();
assert_eq!(handle_args(&os(&[crate::LAUNCH_FLAG, "--", "/bin/sh"])), Some(crate::REFUSED_EXIT_CODE));
let mut text = String::new();
held.read_to_string(&mut text).unwrap();
assert_eq!(text, "data");
std::fs::remove_dir_all(dir).unwrap();
}
#[test]
fn the_canary_is_a_fresh_file_removed_on_drop() {
let a = Canary::create().unwrap();
let b = Canary::create().unwrap();
assert_ne!(a.file(), b.file());
assert!(a.file().is_file());
let dir = a.file().parent().unwrap().to_path_buf();
drop(a);
assert!(!dir.exists());
assert!(Path::new(b.file()).exists());
}
#[cfg(target_os = "macos")]
#[test]
fn a_directory_of_programs_is_a_subpath_in_both_of_its_rules() {
let (canary, dir) = scratch();
let exe = std::env::current_exe().unwrap();
let checked = Policy::new().exec(&dir).checked(&exe, &dir.join("data.txt"), &canary).unwrap();
let profile = crate::macos::profile(&checked).unwrap();
let rule = format!("(subpath \"{}\")", dir.display());
assert_eq!(profile.matches(&rule).count(), 2, "{profile}");
assert!(!profile.contains(&format!("(literal \"{}\")", dir.display())), "{profile}");
std::fs::remove_dir_all(dir).unwrap();
}
#[cfg(target_os = "macos")]
#[test]
fn profile_strings_are_escaped() {
assert_eq!(crate::macos::quoted(Path::new("/a/b c")).unwrap(), "\"/a/b c\"");
assert_eq!(crate::macos::quoted(Path::new("/a\"b\\c")).unwrap(), "\"/a\\\"b\\\\c\"");
assert!(crate::macos::quoted(Path::new("/a\tb")).is_err(), "a control character is refused");
let (canary, dir) = scratch();
let exe = std::env::current_exe().unwrap();
let checked = Policy::new().read(&dir).checked(&exe, &dir.join("data.txt"), &canary).unwrap();
let profile = crate::macos::profile(&checked).unwrap();
assert!(profile.starts_with("(version 1)\n(deny default)\n"));
assert!(profile.contains(&format!("(subpath \"{}\")", dir.display())));
for absent in ["process-fork", "network", "file-write*"] {
assert!(!profile.contains(absent), "no rule allows {absent}");
}
std::fs::remove_dir_all(dir).unwrap();
}
+2 -1
View File
@@ -11,7 +11,7 @@ workspace = true
[dependencies]
# The shared capture slot (`craft_ui::capture`) comes from the one craft-libs revision every
# craft-libs dependency uses (see the root Cargo.toml).
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d" }
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "6e03f65bf45181db77ba6229716e6624db9eb25b" }
gridcraft-engine = { workspace = true }
gridcraft-chart = { workspace = true }
# `monochrome_emoji_fonts` restores the Noto Emoji and emoji-icon faces that 0.37 moved behind
@@ -24,6 +24,7 @@ serde_json = { workspace = true }
log = { workspace = true }
image = { workspace = true }
skrifa = { workspace = true }
unicode-bidi = "=0.3.18"
web-time = { workspace = true }
# Linux only, where winit reports no system theme: the desktop portal's light/dark answer (see
+21 -3
View File
@@ -76,7 +76,21 @@ fn emit_craft_fonts() {
/// One `CraftFont { .. }` initialiser per manifest row GridCraft may embed.
fn craft_fonts(dir: &std::path::Path, wasm: bool, required: bool) -> Result<String, String> {
let manifest = dir.join("fonts/manifest.txt");
// Every file embedded comes from inside the checkout: a manifest row (or a symlink) naming a
// path outside it would put that file of the build machine into the binary.
let root = dir.canonicalize().map_err(|e| format!("{}: {e}", dir.display()))?;
let inside = |path: std::path::PathBuf| -> Result<std::path::PathBuf, String> {
let real = path.canonicalize().map_err(|e| format!("{}: {e}", path.display()))?;
if real.starts_with(&root) { Ok(real) } else { Err(format!("{} is outside the craft-fonts checkout {}", path.display(), root.display())) }
};
// Registered before validation, by the path as given (and its folder, to see the manifest appear
// or a symlink replaced): a build that skipped the fonts must rebuild once they are there.
let manifest_path = dir.join("fonts/manifest.txt");
println!("cargo::rerun-if-changed={}", manifest_path.display());
if let Some(parent) = manifest_path.parent() {
println!("cargo::rerun-if-changed={}", parent.display());
}
let manifest = inside(manifest_path)?;
println!("cargo::rerun-if-changed={}", manifest.display());
let text = std::fs::read_to_string(&manifest).map_err(|e| format!("{}: {e}", manifest.display()))?;
let rows = craft_fonts_policy::parse_manifest(&text)?;
@@ -84,7 +98,7 @@ fn craft_fonts(dir: &std::path::Path, wasm: bool, required: bool) -> Result<Stri
if required {
craft_fonts_policy::check_required(&rows)?;
}
let chinese = if wasm { Vec::new() } else { chinese_font_rows(dir, required)? };
let chinese = if wasm { Vec::new() } else { chinese_font_rows(&root, required)? };
for row in rows.iter().chain(&chinese) {
if craft_fonts_policy::barred(&row.family) {
println!("cargo::warning=craft-fonts: not embedding {} {} (repository asset policy)", row.family, row.style);
@@ -92,7 +106,7 @@ fn craft_fonts(dir: &std::path::Path, wasm: bool, required: bool) -> Result<Stri
}
let mut out = String::new();
for row in craft_fonts_policy::embeddable(&rows, wasm).into_iter().chain(craft_fonts_policy::embeddable(&chinese, wasm)) {
let path = dir.join(&row.file).canonicalize().map_err(|e| format!("{}: {e}", row.file))?;
let path = inside(root.join(&row.file)).map_err(|e| format!("{}: {e}", row.file))?;
println!("cargo::rerun-if-changed={}", path.display());
let scripts: Vec<String> = row.scripts.iter().map(|s| format!("{s:?}")).collect();
let family = &row.family;
@@ -120,6 +134,10 @@ fn chinese_font_rows(dir: &std::path::Path, required: bool) -> Result<Vec<craft_
Ok(Vec::new())
}
};
// `dir` is the canonical checkout: the manifest itself must not lead out of it either.
if manifest.canonicalize().is_ok_and(|real| !real.starts_with(dir)) {
return optional(format!("{} is outside the craft-fonts checkout", manifest.display()));
}
match std::fs::read_to_string(&manifest) {
Ok(text) => match craft_fonts_policy::parse_manifest(&text).and_then(|rows| {
craft_fonts_policy::check_declared_scripts(&rows)?;
+5
View File
@@ -50,6 +50,11 @@ impl SheetApp {
self.session.mode = gridcraft_engine::Mode::Edit;
return;
}
// A hidden formula's source isn't loaded into an edit (a blank edit would also write over
// it on commit): no edit starts. Typing still replaces the cell.
if text.is_none() && gridcraft_engine::display::formula_hidden(&d.wb, sh, at) {
return;
}
let locale = self.ui.language.formula_locale();
let current = if sh.cell_pictures.contains_key(&at) {
String::new()
+3
View File
@@ -24,6 +24,7 @@ pub const REQUIRED_FACES: &[(&str, &str, &[&str])] = &[
("NanumGothic", "Regular", &["Hang", "Kore"]),
("Noto Sans Symbols", "Regular", &["Zsym"]),
("Noto Sans Symbols 2", "Regular", &["Zsym"]),
("Noto Sans Arabic", "Regular", &["Arab"]),
];
/// The native-only Simplified Chinese face from craft-fonts' separate
@@ -41,6 +42,7 @@ pub const WEB_ALLOWED: &[(&str, &str)] = &[
("NanumGothic", "Regular"),
("Noto Sans Symbols", "Regular"),
("Noto Sans Symbols 2", "Regular"),
("Noto Sans Arabic", "Regular"),
];
/// One validated `fonts/manifest.txt` row. Only the fields GridCraft consumes are kept.
@@ -172,6 +174,7 @@ mod tests {
("NanumGothic", "Regular"),
("Noto Sans Symbols", "Regular"),
("Noto Sans Symbols 2", "Regular"),
("Noto Sans Arabic", "Regular"),
]
);
for (family, _) in WEB_ALLOWED {
+33 -21
View File
@@ -1,28 +1,40 @@
use super::{Json, SheetApp, json};
/// The list scrolls inside the dialog: a sheet read from a file can carry thousands of rules, and
/// a list laid out in full made the window as tall as all of them (issue #3321).
const LIST_HEIGHT: f32 = 360.0;
pub(super) fn manage_rules(app: &mut SheetApp, ui: &mut egui::Ui) {
let list = app.session.run("home.manageRules", json!({})).unwrap_or(Json::Null);
let rules = list["rules"].as_array().cloned().unwrap_or_default();
if rules.is_empty() {
// Read the rules in place: running `home.manageRules` for them each frame copied and
// serialised every rule and journaled a step 60 times a second.
let count = app.session.active().and_then(|d| d.wb.active()).map_or(0, |sh| sh.cond_formats.len());
if count == 0 {
ui.label("There are no conditional formatting rules on this sheet.");
return;
}
for (i, r) in rules.iter().enumerate() {
ui.horizontal(|ui| {
let kind = r["rule"].as_object().and_then(|o| o.keys().next().cloned()).unwrap_or_default();
ui.label(format!(
"{}. {kind} applies to {}",
i + 1,
r["ranges"].as_array().map(|a| a.iter().filter_map(|x| x.as_str()).collect::<Vec<_>>().join(",")).unwrap_or_default()
));
if ui.small_button("▲").clicked() {
app.run_or_alert("home.manageRules", json!({"moveUp": i}));
}
if ui.small_button("▼").clicked() {
app.run_or_alert("home.manageRules", json!({"moveDown": i}));
}
if ui.small_button("Delete").clicked() {
app.run_or_alert("home.manageRules", json!({"delete": i}));
}
});
let mut action: Option<Json> = None;
let row_height = ui.spacing().interact_size.y;
egui::ScrollArea::both().max_height(LIST_HEIGHT).auto_shrink([false, true]).show_rows(ui, row_height, count, |ui, rows| {
let Some(sh) = app.session.active().and_then(|d| d.wb.active()) else { return };
for i in rows {
let Some(cf) = sh.cond_formats.get(i) else { break };
let kind = serde_json::to_value(&cf.rule).ok().and_then(|r| r.as_object().and_then(|o| o.keys().next().cloned())).unwrap_or_default();
let ranges = cf.ranges.iter().map(|r| r.a1()).collect::<Vec<_>>().join(",");
ui.horizontal(|ui| {
ui.label(format!("{}. {kind} applies to {ranges}", i + 1));
if ui.small_button("▲").clicked() {
action = Some(json!({"moveUp": i}));
}
if ui.small_button("▼").clicked() {
action = Some(json!({"moveDown": i}));
}
if ui.small_button("Delete").clicked() {
action = Some(json!({"delete": i}));
}
});
}
});
if let Some(params) = action {
app.run_or_alert("home.manageRules", params);
}
}
+12 -1
View File
@@ -69,7 +69,7 @@ pub(super) fn format_cells(app: &mut SheetApp, ui: &mut egui::Ui, d: &mut Dialog
st.num_fmt = gridcraft_engine::model::NumFmt::new(&code);
}
"Alignment" => {
use gridcraft_engine::model::{HAlign, VAlign};
use gridcraft_engine::model::{HAlign, ReadingOrder, VAlign};
ui.label(egui::RichText::new(lang.tr("Horizontal")).strong());
ui.horizontal_wrapped(|ui| {
for (h, n) in [
@@ -97,6 +97,17 @@ pub(super) fn format_cells(app: &mut SheetApp, ui: &mut egui::Ui, d: &mut Dialog
ui.radio_value(&mut st.align.v, v, lang.tr(n));
}
});
ui.label(egui::RichText::new(lang.tr("Reading direction")).strong());
ui.horizontal_wrapped(|ui| {
for (order, name) in
[(ReadingOrder::Context, "Context"), (ReadingOrder::LeftToRight, "Left to right"), (ReadingOrder::RightToLeft, "Right to left")]
{
let selected = st.align.reading_order.unwrap_or_default() == order;
if ui.radio(selected, lang.tr(name)).clicked() {
st.align.reading_order = Some(order);
}
}
});
let mut indent = st.align.indent as f32;
ui.add(egui::Slider::new(&mut indent, 0.0..=15.0).text(lang.tr("Indent")));
st.align.indent = indent as u8;

Some files were not shown because too many files have changed in this diff Show More