Compare commits

..
Author SHA1 Message Date
Dylan M. Taylor d593a8f99d Use existing craft-xml escaping for XLSX output 2026-11-15 10:19:11 -05:00
skylar taylor-barrick 8bde636f27 Sandbox: test the socket owner ioctls with a real pointer (#4245)
The merged test asserts FIOSETOWN and FIOASYNC with the ioctl argument passed
by value, where the kernel reads the number as an address and fails with
EFAULT whether or not the filter refuses it, so the refusal alone is not the
reason the call came back. Passing a pointer, as a C caller does, the call
succeeds unconfined and is EPERM confined.

This covers FIOSETOWN and SIOCSPGRP that way on a socket the parent made and
the program inherited as a standard stream, after reading its socket name so a
pipe cannot pass for one. SIOCSPGRP is in the rules and in the filter's count
assertion but in no test until now.

Deleting either rule line fails this test at that ioctl's assertion; restored,
it passes. The test above is untouched.
2026-11-14 22:04:39 -06:00
skylar taylor-barrick 2f52ae5c7e Sandbox: refuse fcntl's asynchronous notification on Linux (#3846)
* Sandbox: refuse fcntl's asynchronous notification on Linux

* Sandbox: tests for fcntl's notification rules and the new probe

* Sandbox: the socket ioctls that notify another process, too
2026-11-14 21:12:49 -06:00
Dylan M. Taylor a62a532753 Merge pull request #4242 from storytold/ci/sonar-pinned-fonts
ci: provision pinned fonts for coverage tests
2026-11-14 20:53:30 -05:00
Dylan M. Taylor 69219c0d5c ci: provision pinned fonts for coverage tests 2026-11-14 20:38:23 -05:00
FredApps f2044337c7 Merge pull request #4190 from storytold/fix/oversized-pivot-cache-warning-text
Tests: the oversized pivot cache warning names the part limit (main is red)
2026-11-14 18:29:41 +01:00
FredApps 34ab393c72 Tests: the oversized pivot cache warning names the part limit
The package reader reports a part over the limit as "expands to more
than <limit> bytes"; the test added in #2491 expected the declared size
and failed on main.
2026-11-14 18:20:30 +01:00
c2aba7a3f4 perf: win back formula-entry and XLSX-save time lost since 23829d8c (#2654)
Measured with crates/engine/examples/bench.rs (perf/xtask-bench), 100k rows, M2 Pro:
enter.formulas 465 -> 420 ms (23829d8c: 375), xlsx.save 540 -> 488 ms (466).

- number_to_text: the round-up-to-1E+15 check (f29ac7d9e) formatted every number with
  exact {:.14e} and parsed it back; formula printing calls it once per literal. Run it
  only for magnitudes that can round up (>= 9.9999999999999e14). Same output; boundary
  tests added.
- Expr clone/map, xlsx prefix_params: a binary node whose left operand is not a chain
  no longer allocates a spine Vec (2aeb21d4d); map rewrites the operands in their boxes.
  Expr::walk reserves room for a typical formula up front.
- grow_autofilter checks the sheet has an AutoFilter before looking up the cell.
- retained_bytes_against looks up the newer band's row once per row, not per cell.
- The journal measures an entry's JSON size with a counting writer instead of building
  the text (a big paste's params are MBs).

The rest of the enter.formulas gap and all of edit.fanout's (~+12%) is Undo's byte
budget (72db972de, #1227): sizing each snapshot and freeing evicted ones, an intended
cost of bounded memory.

Co-authored-by: nihilist-narwhal <240798405+nihilist-narwhal@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 16:57:27 +01:00
Harrison BoothandClaude Opus 5.5 a8a51e3cf7 XLSX: warn when a pivot cache can't be read or is too large (#1039) (#2491)
read_cache turned every pivot cache error into absence with
optional_xml(part).ok().flatten(), so a cache over the part size limit
dropped its PivotTable with no warning, and a malformed or missing cache
only left a generic "skipped part" line. Each case now adds a warning that
the PivotTable was kept as plain cells, and the workbook keeps opening
(the oversized part is refused before it inflates).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 16:56:12 +01:00
NetterSilverandClaude Haiku 5.5 45ee4eceed Keep mid-field quotes when splitting with Text to Columns (#2438)
Found while testing crates/engine/src/cmd/data.rs:992 (split_delimited, used by text_to_columns)

Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
2026-11-14 16:54:24 +01:00
Michael HigginsandClaude Opus 5.5 fdba683246 calc: linear dependency extraction for range-operator chains (#2183) (#2198)
* calc: linear dependency extraction for range-operator chains (#2183)

precedents() handled a range operator with non-reference operands by recursing into both operands and then letting Expr::walk descend into them again, so a left-associative chain of k operands cost 2^k (a 60-operand INDEX(...):INDEX(...) chain would hang on open). It is now a hand-written collector that visits every node once, growing the per-sheet bounding boxes of the enclosing range operator as it goes. Past a depth (4096) or node (1,000,000) bound it marks the formula dynamic (always recalculated) instead of recursing further.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* calc: drop wall-clock timing from precedents tests (clippy)

std::time::Instant::now is a disallowed method (clippy.toml: traps on wasm).
The regression test only needs to finish; the old 2^k walk would never return.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* calc: bound precedents to 100 ms and box plain-reference range chains (#2183)

- range_operator_chain_precedents_are_linear times the direct precedents
  call against the 100 ms bound in the issue.
- A Range node with two plain Ref operands (A1:B2:C3, which the parser
  can't fold) now collects its bounding box like any other range node, so
  an edit to C1 recalculates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 16:53:03 +01:00
Mex 10849d5003 fix(functions): reject negative fractions in GCD and LCM (#2072)
* M0: reject negative fractional GCD and LCM arguments

Validate the original input before truncation. Add scalar and range regressions, including signed zero, positive fractions and zero-first LCM validation.

* M0: document negative-fraction GCD and LCM regression

Record tested edge-case behavior and update the parity revision history. Function counts and readiness estimates remain unchanged.

* M0: preserve GCD and LCM regression after math test extraction

Move the original negative-fraction regression into the existing extracted math tests. Scalar and range cases, zero-first validation, signed zero and positive truncation coverage are unchanged.
2026-11-14 16:51:29 +01:00
gyurix 494b830240 Linux: the title bar carries the window controls, as in Excel (#192) 2026-11-14 16:48:20 +01:00
Zinadin Zidan 40783ab184 functions: ACCRINTM accrues zero when issue equals settlement (#4096) (#4102)
ACCRINTM returned #NUM! whenever issue >= settlement. Excel rejects only
an issue date after settlement: when the two dates are equal after
truncation to whole days (including distinct times on the same day) it
returns 0 in every basis and both date systems. Relax the guard to
issue > settle; zero rate, zero par and an invalid basis stay #NUM!.
2026-11-14 15:22:59 +01:00
FredApps c0ee120ac2 Merge pull request #1140 from vangel76/perf/chart-frame-cache
Charts: resolved data and rendering are cached between frames (#1139)
2026-11-12 19:55:40 +01:00
Codex bf76e6e63b test: keep ref-edit three-sheet check clippy-clean 2026-11-12 19:41:19 +01:00
Codex 467d36a9c8 Merge main after PDF cleanup; retain both roadmap entries 2026-11-12 19:30:40 +01:00
FredApps efc11acede Merge pull request #818 from vangel76/perf/pdf-writer-cleanup
PDF: writer cleanup, q nesting limit, image dedupe, style only on change (#816)
2026-11-12 18:59:02 +01:00
Codex 0c5436de76 Merge main into chart frame cache; preserve both roadmap entries 2026-11-12 18:13:44 +01:00
Codex ce0beb6900 Merge main into PDF writer cleanup; preserve both roadmap entries 2026-11-12 18:13:44 +01:00
FredApps a60db4f1c7 Merge pull request #3323 from vangel76/fix/linest-qr
LINEST, LOGEST, TREND, GROWTH: solve by Householder QR (#3322)
2026-11-12 17:59:31 +01:00
FredApps a3366cff65 Merge pull request #1135 from vangel76/fix/group-resize-on-release-1082
Grid: resizing several selected columns applies the width on release (#1082)
2026-11-12 17:58:46 +01:00
FredApps 591b8354f2 Merge pull request #3624 from vangel76/fix/3619-merged-shift-enter
Shift+arrows, Enter and Tab no longer get stuck on a merged cell
2026-11-12 17:57:39 +01:00
FredApps 0b31cebe5d Merge pull request #789 from vangel76/feat/wordart-784
Insert: WordArt (insert.wordArt) (#784)
2026-11-12 17:56:55 +01:00
Dylan M. Taylor 37941f0624 Merge pull request #3844 from storytold/switch-craft-libs-xls-dv
xlsx: .xls data validation from craft-xls
2026-11-12 01:43:22 -05:00
Dylan M. Taylor b1bd8d8b20 share legacy XLS test helpers 2026-11-12 06:29:50 +00:00
Dylan M. Taylor 794b65ecac fix XLS validation warnings 2026-11-12 06:13:59 +00:00
Dylan M. Taylor 6308fc9edb Merge pull request #3843 from storytold/switch-craft-libs-opc
xlsx: OPC packages through craft-opc; craft-libs at aa84415b
2026-11-12 01:10:14 -05:00
Dylan M. Taylor 48659c1668 Merge pull request #3842 from storytold/switch-craft-libs-offcrypto
xlsx: encrypted workbooks through craft-offcrypto and craft-cfb
2026-11-12 01:09:49 -05:00
Dylan M. Taylor e8845f648a Merge pull request #3841 from storytold/switch-craft-libs-csv
xlsx, engine: CSV and SYLK through craft-csv
2026-11-12 01:09:16 -05:00
Dylan M. Taylor 3858482465 Merge pull request #3840 from storytold/switch-craft-libs-xls
xlsx: read .xls through craft-xls
2026-11-12 01:00:41 -05:00
Dylan M. Taylor 73a18c4f4c Merge remote-tracking branch 'origin/switch-craft-libs-opc' into codex/reviewfix-3844 2026-11-12 05:41:20 +00:00
Dylan M. Taylor 0b4b36b865 Merge remote-tracking branch 'origin/switch-craft-libs-offcrypto' into codex/reviewfix-3843 2026-11-12 05:41:13 +00:00
Dylan M. Taylor 4251dad939 Merge remote-tracking branch 'origin/switch-craft-libs-csv' into codex/reviewfix-3842 2026-11-12 05:41:06 +00:00
Dylan M. Taylor 1b7e4f00dc Merge remote-tracking branch 'origin/switch-craft-libs-xls' into codex/reviewfix-3841 2026-11-12 05:40:59 +00:00
Dylan M. Taylor 8d4fb0b12e Merge remote-tracking branch 'origin/switch-craft-libs-opc' into codex/propagate-3844 2026-11-12 05:37:34 +00:00
Dylan M. Taylor 203872b4db Merge remote-tracking branch 'origin/switch-craft-libs-offcrypto' into codex/propagate-3843 2026-11-12 05:37:27 +00:00
Dylan M. Taylor d3954beacf Merge remote-tracking branch 'origin/switch-craft-libs-csv' into codex/propagate-3842 2026-11-12 05:37:21 +00:00
Dylan M. Taylor 26e7112e0f Merge remote-tracking branch 'origin/switch-craft-libs-xls' into codex/propagate-3840 2026-11-12 05:37:12 +00:00
Dylan M. Taylor 3ba4945e72 GridCraft: use shared XLS validation operator type 2026-11-12 05:24:28 +00:00
Dylan M. Taylor 0752f3181b Merge current OPC switch into XLS validation switch 2026-11-12 05:20:25 +00:00
Dylan M. Taylor b24ec5df6d Merge current encrypted workbook switch into OPC switch 2026-11-12 05:19:49 +00:00
Dylan M. Taylor d0c567992e Merge current CSV switch into encrypted workbook switch 2026-11-12 05:18:19 +00:00
Dylan M. Taylor 1c84e32f58 GridCraft: update XLS validation switch to current shared crates 2026-11-12 05:16:38 +00:00
Dylan M. Taylor cf3b11c858 Merge remote-tracking branch 'origin/switch-craft-libs-opc' into codex/gridcraft-3844 2026-11-12 05:16:08 +00:00
Dylan M. Taylor 1e6c8b8b72 Merge remote-tracking branch 'origin/switch-craft-libs-opc' into codex/gridcraft-3843 2026-11-12 05:15:57 +00:00
Dylan M. Taylor 3e0f073e9a GridCraft: update OPC switch to current shared crates 2026-11-12 05:15:56 +00:00
Dylan M. Taylor eec5db66f0 Merge remote-tracking branch 'origin/switch-craft-libs-offcrypto' into codex/gridcraft-3843 2026-11-12 05:15:18 +00:00
Dylan M. Taylor a440cfac28 Merge remote-tracking branch 'origin/switch-craft-libs-offcrypto' into codex/gridcraft-3842 2026-11-12 05:14:54 +00:00
Dylan M. Taylor 92a79db048 GridCraft: update encrypted XLS switch to current shared crates 2026-11-12 05:14:47 +00:00
Dylan M. Taylor 652c08a89b Merge remote-tracking branch 'origin/switch-craft-libs-csv' into codex/gridcraft-3842 2026-11-12 05:14:07 +00:00
Dylan M. Taylor aeb3fb19ce Merge latest CSV branch updates 2026-11-12 05:13:54 +00:00
Dylan M. Taylor e4ec0d7ed1 GridCraft: update CSV switch to current shared crates 2026-11-12 05:13:10 +00:00
Dylan M. Taylor 23487b0cb5 Merge current XLS switch into CSV switch 2026-11-12 05:12:51 +00:00
Dylan M. Taylor b45698d109 tests: give the xls fixture its compound-file byte order mark
craft-cfb refuses a header without the 0xFFFE mark at offset 28
([MS-CFB] 2.2), so craft-xls's sniff called the fixture LegacyBinary.
The old hand-written parser never read that field, which is why the test
only broke with the switch. The fixture is a real workbook otherwise:
sniff now says Xls and read_xls parses it.
2026-11-12 05:08:22 +00:00
Dylan M. Taylor c913bf6615 tests: give the xls fixture its compound-file byte order mark
craft-cfb refuses a header without the 0xFFFE mark at offset 28
([MS-CFB] 2.2), so craft-xls's sniff called the fixture LegacyBinary.
The old hand-written parser never read that field, which is why the test
only broke with the switch. The fixture is a real workbook otherwise:
sniff now says Xls and read_xls parses it.
2026-11-12 05:08:20 +00:00
Dylan M. Taylor 0431741359 tests: give the xls fixture its compound-file byte order mark
craft-cfb refuses a header without the 0xFFFE mark at offset 28
([MS-CFB] 2.2), so craft-xls's sniff called the fixture LegacyBinary.
The old hand-written parser never read that field, which is why the test
only broke with the switch. The fixture is a real workbook otherwise:
sniff now says Xls and read_xls parses it.
2026-11-12 04:37:07 +00:00
Dylan M. Taylorandkiwi4boy ae7fbd81b7 xlsx: .xls data validation from craft-xls
craft-xls reads Dv/DVal records (storytold/craft-libs#999, a port of #3636).
xls.rs maps each rule onto the model as the .xlsx reader maps a dataValidation:
a non-literal list source becomes =ref. The tests are #3636's, run through
read_xls; the cap test now expects craft-xls's skipped-rules warning.

Co-authored-by: kiwi4boy <kiwi4boy@gmail.com>
2026-11-12 02:47:52 +00:00
Dylan M. Taylor f800472b0a xlsx: OPC package access through craft-opc; craft-libs at aa84415b
package.rs keeps GridCraft's Package, Rel and ZipOut, now on craft_opc::LazyPackage
and craft_opc::ZipOut (on craft-zip). Limits stay GridCraft's (200,000 entries,
512 MB a part, 1 GB a package) and so do the budget rules, which craft-opc took
from GridCraft. The budget tests moved with them. normalize, resolve and
split_dir are craft-opc's (the same code).

Every craft-libs crate moves to aa84415b. craft-xls now reads data validation;
until GridCraft maps it, a sheet with rules still warns that .xls data validation
is not imported. ods.rs matches entity names through a slice, since the new
dependencies make the old as_ref() call ambiguous.
2026-11-12 02:37:30 +00:00
Dylan M. Taylor 9c87c22662 xlsx: encrypted workbooks and compound files through craft-offcrypto and craft-cfb
Decrypting password-protected workbooks ([MS-OFFCRYPTO], Agile and Standard) moved
to storytold/craft-libs as craft-offcrypto (#931), on craft-cfb. GridCraft keeps
its DecryptError and the engine's messages; crypto.rs maps the crate's errors onto
it. The private compound-file reader and its tests are gone: sniffing looks for
the encryption streams with craft_cfb::has_entry_named (with the same fallback for
a damaged directory), and the test builders encrypt with craft-offcrypto and write
compound files with craft-cfb.

One difference, as craft-offcrypto has it: the dataIntegrity HMAC key is used at
the length it decrypts to (up to the hash size); GridCraft accepted only a
salt-size or hash-size key. Both verify what Office writes.
2026-11-12 01:34:24 +00:00
Dylan M. Taylor c1adf9b239 xlsx, engine: CSV decoding, parsing and quoting, and SYLK, through craft-csv
Text decoding (byte order marks, UTF-8, Windows-1256/1252), delimiter sniffing,
the record parser and field quoting moved to storytold/craft-libs as craft-csv
(#889), with the SYLK reader as craft_csv::sylk. GridCraft keeps typed values,
date-order inference, the used-range limit and the sheet writer in csv.rs, and in
io/sylk.rs the workbook a SYLK file becomes: formulas parsed (an unparsable one
is skipped and counted, as before), error values, number-format styles and column
widths.
2026-11-12 01:17:21 +00:00
vangel76andClaude Sonnet 5.5 2ff6001d3d Merge upstream/main
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 21:54:23 +01:00
vangel76andClaude Sonnet 5.5 20917cfd01 Merge upstream/main
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 21:54:20 +01:00
vangel76andClaude Sonnet 5.5 4dd1c6cf6c Merge upstream/main into perf/chart-frame-cache
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-11 21:54:05 +01:00
vangel76andClaude Sonnet 5.5 60063b85cd Keep the Decorative Text ribbon button two lines wide
The one-line label widened the Insert tab enough to scroll the Equation button
out of view at 1400 px, which broke the equation editor tests in CI.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 23:20:06 +01:00
vangel76andClaude Sonnet 5.5 40c1afbe84 Use the Decorative Text label for the WordArt ribbon button
Upstream retires WordArt as a display name (display_names guard); the
button and its tooltip use the already translated Decorative Text key,
and the parity checklist is regenerated.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 22:17:09 +01:00
vangel76andClaude Sonnet 5.5 75c07a1a83 Merge upstream/main into feat/wordart-784
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-10 22:01:55 +01:00
vangel76 353908fca5 Merge remote-tracking branch 'upstream/main' into fix3624-wt 2026-11-09 23:30:34 +01:00
vangel76andClaude Opus 5.5 10545dc4e1 fix(selection): keep the old walk when the hidden axis has no direction
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 23:30:29 +01:00
vangel76andClaude Opus 5.5 f8070bbbd0 fix(selection): keep the Tab run's column when Enter skips hidden rows after a merged cell
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-09 23:21:03 +01:00
vangel76andClaude Opus 5.5 5e5b5d082d fix(selection): Enter from a merged cell keeps the Tab run's column
Leaving a merged cell downwards kept the start column and dropped the
column offset Enter passes to return to the Tab run's start column, so
Tab A2 -> B2:C3, Enter landed on B4 instead of A4. Keep the computed
column (and the computed row when leaving to the right).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 23:22:22 +01:00
vangel76andClaude Opus 5.5 2bc50341a0 fix(selection): Shift+arrows and Enter/Tab treat a merged cell as one cell (#3619)
Shift+Left/Up from a selected merged cell now widen the selection past
its edges, Shift+arrows shrink an area back over a merged cell, and
Enter/Tab leave a selected merged cell instead of walking the cells it
covers. Inside a larger selection Enter visits a merged cell once.

Fixes #3619

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 22:23:32 +01:00
vangel76andClaude Opus 5.5 7cd0a6cc72 Resize drag: spend Esc on the cancelled drag
Esc during a header-boundary resize was only peeked at, so the grid's
own Esc handling saw it as well and dropped a marked copy and Format
Painter. The cancel now removes the Esc presses from the frame's input,
with any modifiers, as before, cancelling the drag. The regression test
covers a pending copy and sticky Format Painter for rows and columns,
with the release in a later frame or the same frame, and Shift+Esc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 13:01:31 +01:00
vangel76andClaude Opus 5.5 14aae7cee1 WordArt: keep ordinary shapes with textNoShape; print the text outline
textNoShape is the standard no-warp value, so it alone no longer makes a
shape WordArt: only a rectangle that is a text box (as the writer saves
WordArt) or has no fill and no outline reads as WordArt. A filled ellipse
with an explicit textNoShape now reads and saves as the same ellipse.

The PDF/print path drew WordArt text without its outline, so Mint
Outline (white text, green outline) printed as a blank page. The outline
is now drawn like on screen: the text in the outline colour, offset in
eight directions, under the filled text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:54:30 +01:00
vangel76andClaude Opus 5.5 8605fb02f0 Charts: follow defined names when deciding a chart is volatile
A series like =FreshClock, where the workbook name FreshClock is NOW(),
was cached because only the literal series text was searched for
volatile calls. The check now also follows defined names the formulas
use (workbook or sheet scope, bare or sheet-qualified, LAMBDA names),
the way the evaluator looks them up. Nesting deeper than 8, cycles and
more than 64 lookups per chart count as volatile, so a chart whose
stability isn't established is not cached.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:52:00 +01:00
vangel76 032b280b04 Merge upstream/main into feat/wordart-784 2026-11-08 12:39:59 +01:00
vangel76andClaude Opus 5.5 40c3ada226 Merge upstream/main into perf/chart-frame-cache
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:28:15 +01:00
vangel76andClaude Opus 5.5 01fe67010d PDF: look up page images by hash instead of scanning every earlier image
Page::image_rgba compared each new picture with all earlier pictures on
the page, so N distinct pictures cost N(N-1)/2 comparisons. Keep a
page-local hash-to-index map (pixels still compared with ImageData::same)
and add a regression test with 3,000 distinct pictures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:19:46 +01:00
vangel76andClaude Opus 5.5 86ed3d93b4 LINEST: treat only identical raw values as a constant predictor
The 64-ulp spread threshold still classified distinct representable
predictors such as [1e15, 1e15 + 1, 1e15 + 2] as constant and dropped
them. A column is now constant only when every raw value is the same
double; the 1e15 case is added to the offset regression test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-08 12:19:34 +01:00
vangel76 ef320b988d Merge remote-tracking branch 'upstream/main' into HEAD 2026-11-08 12:18:59 +01:00
vangel76andClaude Sonnet 5.5 0144ffa51b Merge upstream/main into feat/wordart-784
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:39:57 +01:00
vangel76andClaude Sonnet 5.5 522120c34c Merge upstream/main into perf/pdf-writer-cleanup
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:39:27 +01:00
vangel76andClaude Sonnet 5.5 3e64ddbd80 Merge upstream/main into fix/group-resize-on-release-1082
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:39:12 +01:00
vangel76andClaude Sonnet 5.5 2cae4aa316 Merge upstream/main into perf/chart-frame-cache
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:38:57 +01:00
vangel76andClaude Sonnet 5.5 ed15db4101 Merge upstream/main into fix/linest-qr
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 18:37:48 +01:00
vangel76andClaude Sonnet 5.5 3d50413aff Resize drag: keep the last accepted preview and the redo history on Esc
A refused size no longer replaces the size the release applies to the
selection, and Esc puts back the redo stack the drag began with.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:34:38 +01:00
vangel76andClaude Sonnet 5.5 224d0e96d7 WordArt: refuse insertion on a protected sheet
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:32:14 +01:00
vangel76andClaude Sonnet 5.5 1baa723ae7 Merge upstream/main into perf/chart-frame-cache
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:28:55 +01:00
vangel76andClaude Sonnet 5.5 cca09cc8f4 PDF: track the first text run with a flag
text_rotated counted the Tf operators written so far on every run, which is
quadratic for text that alternates between WinAnsi and CID characters.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:19:10 +01:00
vangel76andClaude Sonnet 5.5 583c8a4819 LINEST: keep predictors that vary by little next to a large offset
A centred column was dropped when its length fell under 1e-12 of the raw
column, which also removed real variation such as [1e13, 1e13+1, 1e13+2].
Constant columns are now recognised on the raw values, within a few ulps.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 17:01:20 +01:00
vangel76andClaude Sonnet 5.5 8e3fba8a25 Chart cache: drop closed documents once per frame, skip volatile series
Move retain_docs out of the per-chart paint path into SheetApp::logic so closed
documents are forgotten even when no chart is painted. Charts whose series call
NOW, TODAY, RAND, RANDBETWEEN or RANDARRAY bypass the cache because their
resolved values change between evaluations.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-07 16:54:59 +01:00
vangel76andClaude Sonnet 5.5 dd5cd48b8f Regenerate parity checklist after merging main
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 04:29:48 +01:00
vangel76andClaude Sonnet 5.5 2a9a21d41c WordArt xlsx: only the textNoShape marker reads as WordArt, an empty WordArt keeps its body
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 04:25:50 +01:00
vangel76andClaude Sonnet 5.5 6a22a3b357 Merge upstream/main into feat/wordart-784
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 04:24:34 +01:00
vangel76andClaude Sonnet 5.5 38f48ad974 Merge upstream/main into fix/group-resize-on-release-1082
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 02:00:21 +01:00
vangel76andClaude Sonnet 5.5 ac0588dbaf Merge upstream/main into perf/chart-frame-cache
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-05 01:54:42 +01:00
vangel76andClaude Sonnet 5.5 a5c74edfec linest test: use f64::from directly (clippy)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-04 00:18:00 +01:00
vangel76andClaude Sonnet 5.5 d0b40c3b30 linest: drop a constant column that centres to rounding noise (review point)
CodeRabbit pointed out that the drop test used the centred column length, so a
constant column such as [0.1; 3] (centred to about -1.4e-17) was scaled to a unit
vector and kept, giving a meaningless coefficient and wrong df. A centred column
shorter than 1e-12 of its uncentred length is now treated as collinear with the
intercept. The suggested 1e-6 threshold was not used: it would also drop columns
that sit far from zero, such as 1e9 + i.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-04 00:05:38 +01:00
vangel76andClaude Sonnet 5.5 f74d95542f LINEST: solve by Householder QR instead of the normal equations (#3322)
The normal equations square the condition number, so a degree-5 polynomial on
x = 1..12 came back with errors up to 2.6e-6 and x, x^2 near 1e6 gave slope
2.99976 and intercept 251 for y = 3x + 7. LINEST, LOGEST, TREND and GROWTH now
factor the centred, column-scaled data with Householder QR. Collinear columns
are still dropped under the same relative tolerance.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-03 23:38:55 +01:00
vangel76andClaude Haiku 5.5 f3e24e2b68 Regenerate parity checklist after merging main
The parity CI step failed because docs/parity-checklist.md was stale
after merging upstream main. Regenerated with cargo xtask parity.
Counts now 278 / 292 (Insert: WordArt moved to implemented).

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
2026-11-02 20:58:00 +01:00
vangel76andClaude Sonnet 5.5 5b3b3bdc0e Keytip pending-sequence test: use NSC so it does not race the NJ go-live test
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 20:19:10 +01:00
vangel76andClaude Sonnet 5.5 f8a764f20c Fix merge: Shape::math for WordArt, fmt
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 20:07:54 +01:00
vangel76andClaude Sonnet 5.5 0d1296e532 Merge upstream/main into feat/wordart-784
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 20:03:35 +01:00
vangel76andClaude Sonnet 5.5 b396e6c0bd Merge upstream/main into perf/pdf-writer-cleanup
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:52:51 +01:00
vangel76andClaude Sonnet 5.5 9bd6fa1d75 Merge upstream/main into perf/chart-frame-cache
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-02 19:50:48 +01:00
vangel76 e624f4d178 Merge remote-tracking branch 'upstream/main' into HEAD 2026-10-30 00:58:28 +01:00
vangel76andClaude Sonnet 5.5 96aece4e57 Keep Esc on a resize drag from undoing an earlier size edit
A resize drag shares its undo label with an earlier width/height edit, so
coalescing removed the drag's own entry and Esc undid the earlier edit too.
Track the workbook the drag started from and merge/cancel only entries from
there. Esc is also handled when egui has already ended the drag.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-30 00:58:28 +01:00
vangel76andClaude Sonnet 5.5 218aa884be Merge upstream/main into feat/wordart-784
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 23:17:05 +01:00
vangel76 4f372ff470 Name the test case type (clippy type_complexity) 2026-10-29 23:09:29 +01:00
vangel76andClaude Sonnet 5.5 5ca5e6b61e Merge upstream/main into perf/pdf-writer-cleanup
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 23:09:10 +01:00
vangel76andClaude Sonnet 5.5 5d440c82ac Merge upstream/main into fix/group-resize-on-release-1082
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-29 23:07:01 +01:00
vangel76 57e0eb5b0f Merge upstream/main into perf/chart-frame-cache 2026-10-29 23:06:07 +01:00
vangel76andClaude Sonnet 5.5 71469495bc Merge upstream/main into feat/wordart-784
Main moved shape text editing, PDF shape drawing, grid interaction and the Insert ribbon into submodules; the WordArt changes are re-applied there.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 20:16:35 +01:00
vangel76andClaude Sonnet 5.5 527abcfc7d Merge upstream/main into fix/group-resize-on-release-1082
grid.rs was split into grid/ modules on main; the resize-on-release changes are re-applied in grid.rs and grid/interact.rs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 20:01:56 +01:00
vangel76andClaude Sonnet 5.5 37cf274a03 Merge upstream/main into perf/chart-frame-cache
SheetApp setup and font handling moved to app.rs and app/fonts.rs on main; the cache field and its clears are re-applied there. The cache tests use the ActiveSheet accessors.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-27 19:57:18 +01:00
vangel76andClaude Sonnet 5.5 f9bd1e7eca Fix WordArt for ActiveSheet API change after merge
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 20:52:27 +01:00
vangel76andClaude Sonnet 5.5 5a93bc3311 Merge upstream/main into feat/wordart-784
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 20:47:05 +01:00
vangel76andClaude Sonnet 5.5 f1d8d2e1e8 Merge upstream/main into feat/wordart-784
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 00:07:13 +01:00
vangel76andClaude Sonnet 5.5 3afaf39584 Merge upstream/main into perf/pdf-writer-cleanup
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 00:02:43 +01:00
vangel76andClaude Sonnet 5.5 4777381dec Merge upstream/main into fix/group-resize-on-release-1082
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-26 00:00:00 +01:00
vangel76andClaude Sonnet 5.5 97e43b84fa Chart cache test: set the MouseWheel source field added upstream
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 23:59:30 +01:00
vangel76andClaude Sonnet 5.5 6e2ec96c85 Merge upstream/main into perf/chart-frame-cache
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 23:55:51 +01:00
vangel76andClaude Sonnet 5.5 e6b6d2d65f Merge upstream/main into perf/chart-frame-cache
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-25 10:22:23 +01:00
vangel76andClaude Opus 5.5 d59c51ba1f Merge upstream/main into feat/wordart-784
print.rs: main moved shape drawing into draw_shape; the WordArt shadow
pass now sits in its text loop. ja.rs: main's catalog and tests, plus
the two WordArt CORE labels.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:12:18 +02:00
vangel76andClaude Opus 5.5 bce01e7789 Merge upstream/main into fix/group-resize-on-release-1082
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:54:27 +02:00
vangel76andClaude Sonnet 5.5 4b75f3ea00 docs: regenerate parity checklist after merging main
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-22 20:26:34 +02:00
vangel76 a9069cb8f9 Merge remote-tracking branch 'upstream/main' into feat/wordart-784 2026-10-22 20:23:53 +02:00
vangel76 fb71f723fc Merge upstream/main into perf/pdf-writer-cleanup 2026-10-22 19:49:10 +02:00
vangel76 caf897daad Add WordArt labels to the catalogs added on main 2026-10-22 19:40:36 +02:00
vangel76 a9cc381332 Merge upstream/main into feat/wordart-784 2026-10-22 19:35:37 +02:00
vangel76 451debe5e8 Merge upstream/main into fix/group-resize-on-release-1082 2026-10-22 19:32:49 +02:00
vangel76 5f53afd72e Merge remote-tracking branch 'upstream/main' into perf/chart-frame-cache
# Conflicts:
#	ROADMAP.md
#	crates/ui-egui/src/chartview.rs
2026-10-22 19:30:01 +02:00
vangel76andClaude Sonnet 5.5 c7e4d6a19c shape.setText: apply the WordArt length limit
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-20 01:35:46 +02:00
vangel76 30dfc3fbd8 PR789: merge main and preserve WordArt with RTL paragraph export 2026-10-20 00:57:39 +02:00
vangel76andClaude Sonnet 5.5 8e30c095db Charts: resolved data and rendering are cached between frames (#1139)
Part of #1098. Charts were resolved and rendered on every frame. The
primitives are now cached per document/sheet/chart and checked against a
Weak<Workbook> (any write to the workbook moves or clones it), the zoom and
pixels-per-point; capped, LRU, closed documents dropped.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-13 20:26:01 +02:00
vangel76andClaude Sonnet 5.5 6090366562 Grid: resizing several selected columns applies the width on release (#1082)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-13 20:09:53 +02:00
vangel76andClaude Sonnet 5.5 c59575b766 Merge main into #789
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-13 19:05:22 +02:00
vangel76andClaude Sonnet 5.5 30ef61982b Merge upstream/main into perf/pdf-writer-cleanup
Combine the writer cleanup with the PDF hardening (#812) and UTF-16BE info strings (#813): sanitised widths/dashes and finite/extent guards now feed the style tracking; tests adapted to emit-on-change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-13 19:02:32 +02:00
vangel76andClaude Sonnet 5.5 e04e559da3 PDF: writer cleanup, q nesting limit, image dedupe, style only on change (#816)
Direct object writing, image dedupe, per-page style tracking, 28-level q limit,
explicit joins, /ProcSet dropped. See the PR description.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 22:28:19 +02:00
vangel76andClaude Sonnet 5.5 3109f2a9e0 Keytips: NW runs insert.wordArt now that the command exists; pending-sequence test uses NJ (insert.object)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 21:47:39 +02:00
vangel76andClaude Sonnet 5.5 a39c6b8977 Calc: SUBTOTAL(10x) recalculates when Row Height, AutoFit Row Height or Ungroup reveal rows
home.rowHeight, home.autofitRowHeight and data.ungroup change row hidden
state without marking the edit structural, so SUBTOTAL(101-111) kept a stale
value. Found by the command-sequence fuzz (seed 3).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 21:09:41 +02:00
vangel76andClaude Sonnet 5.5 470968fcdc Insert: WordArt (insert.wordArt) (#784)
Floating decorative text built on the text-box shape path: ShapeKind::WordArt, ten original
styles, outline/shadow, wordArt.set, gallery in the ribbon, XLSX and print.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 20:38:11 +02:00
114 changed files with 5262 additions and 2726 deletions
+3
View File
@@ -18,4 +18,7 @@ jobs:
sonarcloud:
# Tracked at @main so org-wide CI improvements propagate without PR churn.
uses: storytold/craft-common-actions/.github/workflows/sonarcloud.yml@main
with:
# Match the main CI gate so coverage exercises every glyph-sensitive UI case.
fonts-ref: b076417b5880bd4154bd22f00695b1d28c6bea83
secrets: inherit
Generated
+107 -14
View File
@@ -739,6 +739,12 @@ dependencies = [
"cc",
]
[[package]]
name = "cmov"
version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a"
[[package]]
name = "cmpv2"
version = "0.2.0"
@@ -757,7 +763,7 @@ version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b77c319abfd5219629c45c34c89ba945ed3c5e49fcde9d16b6c3885f118a730"
dependencies = [
"const-oid",
"const-oid 0.9.6",
"der",
"spki",
"x509-cert",
@@ -827,6 +833,12 @@ version = "0.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8"
[[package]]
name = "const-oid"
version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "be95d4a884358a21a199689323c29e9256b4a1b315e22f183d155422041d3a9a"
[[package]]
name = "core-foundation"
version = "0.9.4"
@@ -912,15 +924,23 @@ name = "craft-cfb"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=bb57d422d0bc842cbde430955825553ab7b976f4#bb57d422d0bc842cbde430955825553ab7b976f4"
[[package]]
name = "craft-csv"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=bb57d422d0bc842cbde430955825553ab7b976f4#bb57d422d0bc842cbde430955825553ab7b976f4"
dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "craft-fonts-build"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=119415f46f601e20b9bc6b1b4d6979ceda86d88e#119415f46f601e20b9bc6b1b4d6979ceda86d88e"
source = "git+https://github.com/storytold/craft-libs?rev=aa84415bae3b765fef6b7ef42404f66ad1cff9d9#aa84415bae3b765fef6b7ef42404f66ad1cff9d9"
[[package]]
name = "craft-i18n"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=119415f46f601e20b9bc6b1b4d6979ceda86d88e#119415f46f601e20b9bc6b1b4d6979ceda86d88e"
source = "git+https://github.com/storytold/craft-libs?rev=aa84415bae3b765fef6b7ef42404f66ad1cff9d9#aa84415bae3b765fef6b7ef42404f66ad1cff9d9"
dependencies = [
"fixed_decimal",
"icu_decimal",
@@ -940,10 +960,38 @@ dependencies = [
"writeable",
]
[[package]]
name = "craft-offcrypto"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=bb57d422d0bc842cbde430955825553ab7b976f4#bb57d422d0bc842cbde430955825553ab7b976f4"
dependencies = [
"aes",
"base64 0.23.1",
"craft-cfb",
"craft-xml",
"getrandom 0.3.4",
"hmac",
"sha1 0.11.0",
"sha2 0.11.1",
"thiserror 2.0.21",
"zeroize",
]
[[package]]
name = "craft-opc"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=bb57d422d0bc842cbde430955825553ab7b976f4#bb57d422d0bc842cbde430955825553ab7b976f4"
dependencies = [
"craft-xml",
"craft-zip",
"log",
"thiserror 2.0.21",
]
[[package]]
name = "craft-print"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=119415f46f601e20b9bc6b1b4d6979ceda86d88e#119415f46f601e20b9bc6b1b4d6979ceda86d88e"
source = "git+https://github.com/storytold/craft-libs?rev=aa84415bae3b765fef6b7ef42404f66ad1cff9d9#aa84415bae3b765fef6b7ef42404f66ad1cff9d9"
dependencies = [
"craft-winprint-gdi",
"log",
@@ -962,7 +1010,7 @@ source = "git+https://github.com/storytold/craft-libs?rev=bb57d422d0bc842cbde430
[[package]]
name = "craft-ui"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=119415f46f601e20b9bc6b1b4d6979ceda86d88e#119415f46f601e20b9bc6b1b4d6979ceda86d88e"
source = "git+https://github.com/storytold/craft-libs?rev=aa84415bae3b765fef6b7ef42404f66ad1cff9d9#aa84415bae3b765fef6b7ef42404f66ad1cff9d9"
dependencies = [
"craft-fonts-build",
"craft-i18n",
@@ -1009,7 +1057,7 @@ dependencies = [
[[package]]
name = "craft-web-api"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=119415f46f601e20b9bc6b1b4d6979ceda86d88e#119415f46f601e20b9bc6b1b4d6979ceda86d88e"
source = "git+https://github.com/storytold/craft-libs?rev=aa84415bae3b765fef6b7ef42404f66ad1cff9d9#aa84415bae3b765fef6b7ef42404f66ad1cff9d9"
dependencies = [
"js-sys",
"serde_json",
@@ -1020,7 +1068,7 @@ dependencies = [
[[package]]
name = "craft-web-host"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=119415f46f601e20b9bc6b1b4d6979ceda86d88e#119415f46f601e20b9bc6b1b4d6979ceda86d88e"
source = "git+https://github.com/storytold/craft-libs?rev=aa84415bae3b765fef6b7ef42404f66ad1cff9d9#aa84415bae3b765fef6b7ef42404f66ad1cff9d9"
dependencies = [
"wasm-bindgen",
"web-sys",
@@ -1058,6 +1106,25 @@ dependencies = [
"thiserror 2.0.21",
]
[[package]]
name = "craft-xml"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=bb57d422d0bc842cbde430955825553ab7b976f4#bb57d422d0bc842cbde430955825553ab7b976f4"
dependencies = [
"quick-xml 0.41.0",
"thiserror 2.0.21",
]
[[package]]
name = "craft-zip"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=bb57d422d0bc842cbde430955825553ab7b976f4#bb57d422d0bc842cbde430955825553ab7b976f4"
dependencies = [
"flate2",
"thiserror 2.0.21",
"zip 9.1.0",
]
[[package]]
name = "crc32fast"
version = "1.5.2"
@@ -1138,6 +1205,15 @@ dependencies = [
"hybrid-array",
]
[[package]]
name = "ctutils"
version = "0.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "261af4dc7a06ea8a527a8717d8cbba3417530d6ea14e1edb3de6b67806be474f"
dependencies = [
"cmov",
]
[[package]]
name = "cursor-icon"
version = "1.2.0"
@@ -1150,7 +1226,7 @@ version = "0.7.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb"
dependencies = [
"const-oid",
"const-oid 0.9.6",
"der_derive",
"flagset",
"pem-rfc7468",
@@ -1215,7 +1291,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
dependencies = [
"block-buffer 0.12.1",
"const-oid 0.10.3",
"crypto-common 0.2.2",
"ctutils",
]
[[package]]
@@ -1906,9 +1984,11 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
dependencies = [
"cfg-if",
"js-sys",
"libc",
"r-efi 5.3.0",
"wasip2",
"wasm-bindgen",
]
[[package]]
@@ -2123,6 +2203,7 @@ dependencies = [
name = "gridcraft-engine"
version = "0.19.1"
dependencies = [
"craft-csv",
"gridcraft-calc",
"gridcraft-chart",
"gridcraft-core",
@@ -2263,15 +2344,18 @@ dependencies = [
name = "gridcraft-xlsx"
version = "0.19.1"
dependencies = [
"aes",
"craft-cfb",
"craft-csv",
"craft-offcrypto",
"craft-opc",
"craft-xls",
"craft-xml",
"gridcraft-core",
"gridcraft-formula",
"gridcraft-functions",
"gridcraft-math",
"gridcraft-model",
"quick-xml 0.41.0",
"sha1 0.11.0",
"sha2 0.11.1",
"thiserror 2.0.21",
"zip 3.0.0",
@@ -2376,6 +2460,15 @@ version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
[[package]]
name = "hmac"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6303bc9732ae41b04cb554b844a762b4115a61bfaa81e3e83050991eeb56863f"
dependencies = [
"digest 0.11.3",
]
[[package]]
name = "http"
version = "1.5.0"
@@ -4838,7 +4931,7 @@ checksum = "da90c3d9af638898a5fdc347479b18f950bb0dde11ecf2244f94cd28135da53b"
dependencies = [
"aws-lc-rs",
"base64 0.22.1",
"const-oid",
"const-oid 0.9.6",
"der",
"digest 0.10.7",
"pem",
@@ -4912,7 +5005,7 @@ dependencies = [
"chrono",
"cmpv2",
"cms",
"const-oid",
"const-oid 0.9.6",
"der",
"hex",
"rand 0.10.3",
@@ -4951,7 +5044,7 @@ dependencies = [
"base64 0.22.1",
"chrono",
"cms",
"const-oid",
"const-oid 0.9.6",
"hex",
"pem",
"rustls-pki-types",
@@ -6676,7 +6769,7 @@ version = "0.2.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1301e935010a701ae5f8655edc0ad17c44bad3ac5ce8c39185f75453b720ae94"
dependencies = [
"const-oid",
"const-oid 0.9.6",
"der",
"sha1 0.10.7",
"signature",
+9 -3
View File
@@ -31,7 +31,16 @@ gridcraft-math = { path = "crates/math" }
# craft-libs revision.
# craft-update: app-specific pretend versions apply to startup and manual checks.
craft-update = { git = "https://github.com/storytold/craft-libs", rev = "bb57d422d0bc842cbde430955825553ab7b976f4", version = "=0.1.0", features = ["egui"] }
# Compound files ([MS-CFB]) and password-encrypted OOXML packages ([MS-OFFCRYPTO]).
craft-cfb = { git = "https://github.com/storytold/craft-libs", rev = "bb57d422d0bc842cbde430955825553ab7b976f4", version = "0.1.0" }
craft-offcrypto = { git = "https://github.com/storytold/craft-libs", rev = "bb57d422d0bc842cbde430955825553ab7b976f4", version = "0.1.0" }
# OPC zip packages (parts, relationships, the zip writer) under `crates/xlsx/src/package.rs`.
craft-opc = { git = "https://github.com/storytold/craft-libs", rev = "bb57d422d0bc842cbde430955825553ab7b976f4", version = "0.1.0" }
# Delimited text (CSV, TSV) and SYLK; `crates/xlsx/src/csv.rs` and `crates/engine/src/io/sylk.rs`
# turn them into workbooks.
craft-csv = { git = "https://github.com/storytold/craft-libs", rev = "bb57d422d0bc842cbde430955825553ab7b976f4", version = "0.1.0" }
# Excel 97-2003 (.xls, BIFF8) workbooks; `crates/xlsx/src/xls.rs` maps them onto the model.
craft-xml = { git = "https://github.com/storytold/craft-libs", rev = "bb57d422d0bc842cbde430955825553ab7b976f4", version = "0.1.0" }
craft-xls = { git = "https://github.com/storytold/craft-libs", rev = "bb57d422d0bc842cbde430955825553ab7b976f4", version = "0.1.0" }
# foundation
@@ -47,9 +56,6 @@ regex = { version = "1", default-features = false, features = ["std", "unicode",
quick-xml = "0.41"
zip = { version = "3", default-features = false, features = ["deflate"] }
sha2 = { version = "0.11", default-features = false }
# Decrypting password-protected workbooks ([MS-OFFCRYPTO]): AES and SHA-1, pure Rust (wasm-safe).
aes = { version = "0.9", default-features = false }
sha1 = { version = "0.11", default-features = false }
image = { version = "0.25", default-features = false, features = ["png", "jpeg"] }
# equations: RaTeX (MIT) lays LaTeX out as positioned glyphs, rules and paths. Exact pins: it is
+9
View File
@@ -180,6 +180,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-15 | File › Options › Formulas: Maximum Iterations and Maximum Change fields for iterative calculation, disabled while it is off, with Excel's limits (1 to 32767, 0 or more) |
| 2026-10-15 | Function help follows the interface language: argument hints, autocomplete, Insert Function (search and readable categories) and the Formulas-tab menus read per-language tables of descriptions and argument names; Spanish keeps its 10 functions in the new format |
| 2026-10-15 | Row auto-fit counts a cell's line breaks only when the cell wraps, as Excel does: a file whose unwrapped cells hold multi-line text no longer opens with tall rows (typing a break still turns wrap on and grows the row) |
| 2026-10-14 | XLSX import warns when a pivot cache is missing, malformed or too large, naming the cache and saying its PivotTable was kept as plain cells, instead of dropping the PivotTable silently (#1039) |
| 2026-10-14 | Set precision as displayed is applied: with the option on, formula results, spills and numbers typed or reformatted are stored as their number format shows them and formulas calculate from those values; `formulas.calculationOptions` gains `precisionAsDisplayed` (turning it on rounds existing constants; locked constants on a protected sheet keep their value; numbers written to a new sheet are rounded too) (#1343) |
| 2026-10-14 | Lookups (MATCH, XMATCH, VLOOKUP, HLOOKUP, XLOOKUP, LOOKUP) compare numbers exactly like Excel, so a computed key such as `=0.1+0.2` no longer finds 0.3; `=` and COUNTIF/SUMIF keep their 15-digit comparison |
| 2026-10-14 | Functions: EOMONTH, EDATE, WORKDAY, NETWORKDAYS, YEARFRAC, WEEKNUM, the bond and coupon functions, DISC, BIN2DEC, IM* and the other Analysis ToolPak functions give `#VALUE!` for a reference to more than one cell in a value argument, as Excel 365 does, instead of spilling; array constants and computed arrays still lift |
@@ -198,9 +199,13 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-13 | XLSX export writes a picture used on several sheets once, as one `xl/media` part every drawing points to (it wrote a copy per drawing: a 16-sheet workbook with one 112 KB logo grew from 0.4 to 1.9 MB) (#985) |
| 2026-10-13 | XLSX: `[Content_Types].xml`, `_rels/.rels`, the workbook rels and `xl/workbook.xml` are the first zip entries (document properties last), so libmagic and PHP `finfo` detect the file type even with large properties (#1090) |
| 2026-10-13 | Conditional formatting: a rule gathers range statistics only when it reads them (top/bottom, above average, duplicates, scales, bars, icons) and borrows them instead of copying them per cell, so the per-frame lookup of an expression or cell-value rule no longer visits its whole range (#1098, part 1) |
| 2026-10-13 | Grid: resizing several selected columns or rows previews on the dragged one and applies the size to all on release; Esc cancels (#1082) |
| 2026-10-12 | Next/previous sheet navigation stops at the first/last visible worksheet instead of activating a hidden boundary sheet; the current selection is retained |
| 2026-10-12 | PDF writer: objects are written straight to the output, identical pictures are stored once, style operators are emitted only when they change, `q` nesting is limited to 28 and polyline joins no longer leak into later strokes (#816) |
| 2026-10-12 | Insert › Text › WordArt (`insert.wordArt`, `wordArt.set`): floating decorative text with 10 original styles, outline and shadow, in-place editing, XLSX and print (#784) |
| 2026-10-12 | Number formatting: one table of built-in formats (xlsx); numbers, dates and General render with one scratch buffer instead of per-token strings; output unchanged, checked against digests recorded from the old code (#817) |
| 2026-10-21 | Macro engine ([macros.md](docs/macros.md)): typed, declarative macros saved in the workbook (a JSON field; hidden names in xlsx); declared inputs checked with messages that name the cells; tokens (`Input.col(2)`, `Row`, `Output`, `Name:Total`, `{{…}}` in formulas); one undo step with full rollback; a footprint dry run with ask, fail, overwrite, shift down or shift right; a policy that keeps macros away from files and the app, with approval for macros in opened files; Repeat, If and Stop steps; a debugger; a relative recorder; `macro.*` commands beside the unchanged Automate scripts |
| 2026-10-13 | Performance: a chart is resolved and rendered once per change of its workbook, zoom or fonts instead of on every frame; idle frames, hovering and scrolling reuse it (#1139, part of #1098) |
| 2026-10-13 | Charts: work and memory are bounded by series x points (2,000,000 per chart: every series keeps its first points; 16 trendlines per chart), hairline columns and bars are merged per pixel column, series ranges are clipped before they are evaluated, the moving average is linear and rendering clean data no longer copies it (#1147) |
| 2026-10-13 | Charts: one combo plot-kind rule for resolving and drawing; ticks of a scale off the step grid keep their own values (#1146) |
| 2026-10-12 | Security: the `--control` server requires an auth token (random, kept in a 0600 file or chosen with `GRIDCRAFT_CONTROL_TOKEN`) on each connection's first line and closes bad connections (#777) |
@@ -301,6 +306,7 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-16 | minor | Progress log: floating pictures' textures bounded and kept per workbook |
| 2026-10-16 | trivial | Progress log: web preferences kept between visits (#1562) |
| 2026-10-16 | minor | Progress log: interface fonts held once instead of twice |
| 2026-10-14 | trivial | Progress log: pivot cache import warnings (#1039) |
| 2026-10-14 | minor | Progress log: Set precision as displayed applied in calculation (#1343) |
| 2026-10-14 | minor | Progress log: `.xls` import reads cell formatting |
| 2026-10-14 | minor | Progress log: lookups compare numbers exactly |
@@ -313,6 +319,9 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in
| 2026-10-13 | trivial | Progress log: shared pictures written once on XLSX export (#985) |
| 2026-10-13 | minor | Progress log: conditional-format lookups no longer scan the range for rules without statistics (#1098) |
| 2026-10-12 | minor | Progress log: hidden-sheet navigation at workbook boundaries |
| 2026-10-13 | trivial | Progress log: charts cached between frames (#1139) |
| 2026-10-12 | minor | Progress log: PDF writer cleanup (#816) |
| 2026-10-12 | minor | Catalog 276 → 277 / 292: Insert › WordArt (#784) |
| 2026-10-12 | minor | Numfmt cleanup: one built-in format table (gridcraft-xlsx), fewer allocations rendering numbers, dates and General, equality test against the previous output (#817); progress log row |
| 2026-10-12 | minor | The control server requires an auth token and closes bad connections (#777); progress log row |
| 2026-10-12 | minor | Rows auto-fit oversized fonts on open and whole-row/column formatting (#833); progress log row |
+107 -1
View File
@@ -78,6 +78,8 @@ fn the_self_test_finds_every_denial() {
] {
assert!(out.contains(line), "missing `{line}` in:\n{out}");
}
#[cfg(target_os = "linux")]
assert!(out.contains("denied: switch a descriptor's notifications on"), "missing the notification switch in:\n{out}");
std::fs::remove_dir_all(home).unwrap();
}
@@ -181,7 +183,9 @@ fn system_python() -> Option<(String, String)> {
/// Confined, a program can't signal another process (this test, signal 0 delivering nothing), but can
/// still signal itself and its own threads. On Linux, where the filter names them, it also can't
/// change another process's limits or priority, or type into a terminal: no call here reaches a real
/// terminal, since its standard input isn't one.
/// terminal, since its standard input isn't one. Linux also refuses the `fcntl` commands that would
/// have the kernel signal a process for it: naming the owner, choosing the signal, or switching the
/// notification on.
#[test]
fn a_confined_program_leaves_other_processes_alone() {
if !confines() {
@@ -226,6 +230,13 @@ if sys.platform == "linux":
assert refused(os.setpriority, os.PRIO_USER, 0, 0), "every process's priority"
assert refused(os.sched_setaffinity, other, {0}), "another process's processors"
assert refused(os.pidfd_open, other), "a handle on another process"
read, write = os.pipe()
assert refused(fcntl.fcntl, write, fcntl.F_SETOWN, other), "giving a descriptor's notifications to another process"
# 15 is F_SETOWN_EX and 10 is F_SETSIG, which Python's fcntl doesn't name; the filter refuses
# them whatever owner or signal they name, so the argument doesn't matter.
assert refused(fcntl.fcntl, write, 15), "naming a descriptor's owner the extended way"
assert refused(fcntl.fcntl, write, 10), "choosing a descriptor's notification signal"
assert refused(fcntl.fcntl, write, fcntl.F_SETFL, fcntl.fcntl(write, fcntl.F_GETFL) | os.O_ASYNC), "switching a descriptor's notifications on"
resource.prlimit(0, resource.RLIMIT_NOFILE)
resource.prlimit(os.getpid(), resource.RLIMIT_NOFILE)
os.setpriority(os.PRIO_PROCESS, 0, os.getpriority(os.PRIO_PROCESS, 0))
@@ -240,6 +251,101 @@ print("left alone")
assert!(String::from_utf8_lossy(&o.stdout).contains("left alone"), "{}", text(&o));
}
/// A socket the parent made, which the program inherited as a standard stream, can't be pointed at
/// another process through the ioctls that do what `fcntl`'s notification commands do: `FIOSETOWN`
/// (0x8901) and `FIOASYNC` (0x5452), which Python's `fcntl` passes straight through. Linux only:
/// macOS's profile has no rule for them, and its standard streams here are pipes.
#[cfg(target_os = "linux")]
#[test]
fn a_confined_program_cant_point_an_inherited_socket_at_another_process() {
if !confines() {
return;
}
let Some((python, prefix)) = system_python() else {
return;
};
let code = r#"
import fcntl, sys
other = int(sys.argv[1])
def refused(call, *args):
try:
call(*args)
except PermissionError:
return True
except OSError as e:
print("other error", e)
return False
assert refused(fcntl.ioctl, 0, 0x8901, other), "giving a socket's notifications to another process"
assert refused(fcntl.ioctl, 0, 0x5452, 1), "switching a socket's notifications on"
print("socket quiet")
"#;
let (mine, _) = std::os::unix::net::UnixStream::pair().unwrap();
let socket: std::os::fd::OwnedFd = mine.into();
let me = std::process::id().to_string();
let o = Command::new(env!("CARGO_BIN_EXE_gridcraft-cli"))
.args(["sandbox-check", "--read", &prefix, "--", &python, "-I", "-c", code, &me])
.stdin(std::process::Stdio::from(socket))
.output()
.unwrap();
let out = text(&o);
assert!(o.status.success(), "{out}");
assert!(String::from_utf8_lossy(&o.stdout).contains("socket quiet"), "{out}");
}
/// The same inherited socket, by both ioctls that name the process to notify, `FIOSETOWN` (0x8901)
/// and `SIOCSPGRP` (0x8902), with the argument a real pointer. Named by value the kernel reads the
/// number as an address and fails with `EFAULT` whether or not the filter refuses it, so that form
/// shows only that a rule matched a request number; by pointer each call succeeds unconfined and is
/// `EPERM` confined, which is the transition the page claims. Kept apart from the test above so a
/// rule dropped for one request number fails for that request, not behind another. Linux only, as
/// those rules are: macOS's profile has none, and its standard streams here are pipes.
#[cfg(target_os = "linux")]
#[test]
fn a_confined_program_cant_point_an_inherited_socket_at_another_process_by_pointer() {
if !confines() {
return;
}
let Some((python, prefix)) = system_python() else {
return;
};
let code = r#"
import fcntl, socket, struct, sys
other = int(sys.argv[1])
def refused(call, *args):
try:
call(*args)
except PermissionError:
return True
except OSError as e:
print("other error", e)
return False
# Read first, so a standard stream that is not a socket can't pass for one: a pipe has no socket name.
# Detached, not dropped: a socket object closes the descriptor it wrapped when it is collected, and then
# the refusals below would be read off an invalid descriptor.
wrapped = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM, fileno=0)
name = wrapped.getsockname()
wrapped.detach()
assert name == ""
# The process group is a pointer, as a C caller passes it; naming it by value has the kernel read the
# number as an address and fail with EFAULT, which would leave the refusal unexplained by the filter.
group = bytearray(struct.pack("i", other))
assert refused(fcntl.ioctl, 0, 0x8901, group), "giving a socket's notifications to another process"
assert refused(fcntl.ioctl, 0, 0x8902, group), "giving a socket's notifications to another process's group"
print("socket owner quiet")
"#;
let (mine, _) = std::os::unix::net::UnixStream::pair().unwrap();
let socket: std::os::fd::OwnedFd = mine.into();
let me = std::process::id().to_string();
let o = Command::new(env!("CARGO_BIN_EXE_gridcraft-cli"))
.args(["sandbox-check", "--read", &prefix, "--", &python, "-I", "-c", code, &me])
.stdin(std::process::Stdio::from(socket))
.output()
.unwrap();
let out = text(&o);
assert!(o.status.success(), "{out}");
assert!(String::from_utf8_lossy(&o.stdout).contains("socket owner quiet"), "{out}");
}
/// Confined, a program changes no file, whether the policy lets it read that file or not, by path or
/// by a descriptor: not its contents, permissions, owner, extended attributes, times or flags.
/// Skipped when there is no `python3`.
+2 -2
View File
@@ -18,8 +18,8 @@ gridcraft-engine = { workspace = true }
gridcraft-ui-egui = { workspace = true }
# The host messages (storytold/craft-libs): one contract and one bridge for the web builds.
# The page command API (storytold/craft-libs#413): window.gridcraft over the control channel.
craft-web-api = { git = "https://github.com/storytold/craft-libs", rev = "119415f46f601e20b9bc6b1b4d6979ceda86d88e", version = "0.1.0" }
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "119415f46f601e20b9bc6b1b4d6979ceda86d88e", version = "0.1.0" }
craft-web-api = { git = "https://github.com/storytold/craft-libs", rev = "aa84415bae3b765fef6b7ef42404f66ad1cff9d9", version = "0.1.0" }
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "aa84415bae3b765fef6b7ef42404f66ad1cff9d9", version = "0.1.0" }
egui = { workspace = true }
log = { workspace = true }
serde_json = { workspace = true }
+1 -1
View File
@@ -35,7 +35,7 @@ winit = { version = "0.30", default-features = false, features = ["x11", "waylan
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
# Same craft-libs revision as the workspace's craft-update and craft-window (see the root
# Cargo.toml).
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "119415f46f601e20b9bc6b1b4d6979ceda86d88e" }
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "aa84415bae3b765fef6b7ef42404f66ad1cff9d9" }
craft-update = { workspace = true }
rfd = { workspace = true }
arboard = { workspace = true, features = ["image-data"] }
+5 -1
View File
@@ -340,12 +340,16 @@ fn main() -> eframe::Result<()> {
for f in &files {
app.open_path(f);
}
// Linux/BSD: Excel's layout — no window-manager title bar above ours; our title bar carries the
// window controls. `GRIDCRAFT_SYSTEM_TITLEBAR=1` keeps the system frame.
app.window_controls = cfg!(not(any(target_os = "macos", target_os = "windows"))) && std::env::var_os("GRIDCRAFT_SYSTEM_TITLEBAR").is_none();
let mut viewport = egui::ViewportBuilder::default()
.with_title("GridCraft")
.with_inner_size([1440.0, 900.0])
.with_min_inner_size([640.0, 420.0])
.with_app_id("ai.storyteller.gridcraft")
.with_drag_and_drop(true);
.with_drag_and_drop(true)
.with_decorations(!app.window_controls);
if cfg!(target_os = "macos") {
viewport = viewport.with_fullsize_content_view(true).with_titlebar_shown(false).with_title_shown(false);
}
+137 -46
View File
@@ -27,62 +27,153 @@ pub fn resized_sum_range(sum: RangeRef, height: u32, width: u32) -> RangeRef {
RangeRef::new(sum.start, CellRef::new(row, col))
}
/// Most expression nodes [`precedents`] visits before giving up and marking the formula dynamic.
const MAX_PRECEDENT_NODES: usize = 1_000_000;
/// Collects the static references of an expression with their sheets resolved; `dynamic` is
/// set when the formula uses INDIRECT/OFFSET/names/tables whose targets can change.
///
/// Every node is visited once (#2183: range-operator chains used to re-walk shared operands,
/// costing 2^k for k operands), with an explicit stack, so a flat chain as long as Excel allows
/// (`1+1+…`, about 4,000 levels deep) never recurses. Past [`MAX_PRECEDENT_NODES`] the result
/// is conservative: `dynamic` is set so the formula recalculates every time.
pub fn precedents(wb: &Workbook, sheet: usize, e: &Expr) -> (Vec<Area>, bool) {
let mut out = Vec::new();
let mut dynamic = false;
let push = |out: &mut Vec<Area>, r: &Reference, range: RangeRef| match &r.sheet {
SheetSel::Current => out.push(Area { sheet, range }),
SheetSel::Named(n) => {
if let Some(i) = wb.sheet_index(n) {
out.push(Area { sheet: i, range });
}
let mut c = Precedents { wb, sheet, out: Vec::new(), dynamic: false, boxes: Vec::new() };
c.collect(e);
(c.out, c.dynamic)
}
/// Grows the per-sheet bounding box in `boxes` to cover `area`.
fn union_box(boxes: &mut Vec<Area>, area: Area) {
match boxes.iter_mut().find(|x| x.sheet == area.sheet) {
Some(x) => x.range = x.range.union(&area.range),
None => boxes.push(area),
}
}
/// One entry of [`Precedents::collect`]'s explicit stack. `boxes` indexes
/// [`Precedents::boxes`]: the bounding boxes of the innermost enclosing range operator, if any.
enum PrecedentStep<'e> {
/// Visit an expression and queue its children.
Visit(&'e Expr, Option<usize>),
/// Both operands of a range operator have been visited: record the bounding boxes they
/// collected in `inner`, growing the `outer` operator's boxes too.
Close { inner: usize, outer: Option<usize> },
}
struct Precedents<'w> {
wb: &'w Workbook,
sheet: usize,
out: Vec<Area>,
dynamic: bool,
/// Per-sheet bounding boxes, one list per range operator (`A1:INDEX(…)`) met so far.
boxes: Vec<Vec<Area>>,
}
impl Precedents<'_> {
/// Records `area`, also growing the bounding boxes of the enclosing range operator, if any.
fn add(&mut self, area: Area, boxes: Option<usize>) {
if let Some(b) = boxes.and_then(|i| self.boxes.get_mut(i)) {
union_box(b, area);
}
SheetSel::Span(a, b) => {
if let (Some(i), Some(j)) = (wb.sheet_index(a), wb.sheet_index(b)) {
for s in i.min(j)..=i.max(j) {
out.push(Area { sheet: s, range });
self.out.push(area);
}
fn push(&mut self, r: &Reference, range: RangeRef, boxes: Option<usize>) {
match &r.sheet {
SheetSel::Current => self.add(Area { sheet: self.sheet, range }, boxes),
SheetSel::Named(n) => {
if let Some(i) = self.wb.sheet_index(n) {
self.add(Area { sheet: i, range }, boxes);
}
}
SheetSel::Span(a, b) => {
if let (Some(i), Some(j)) = (self.wb.sheet_index(a), self.wb.sheet_index(b)) {
for s in i.min(j)..=i.max(j) {
self.add(Area { sheet: s, range }, boxes);
}
}
}
}
};
e.walk(&mut |x| match x {
Expr::Ref(r) => push(&mut out, r, r.range()),
// The cells SUMIF/AVERAGEIF really read from the sum range (see `resized_sum_range`); when
// the criteria range's size isn't known statically, recalculate the formula every time.
Expr::Call(n, args) if resizes_sum_range(n, args.len()) => match (args.first(), args.get(2)) {
(Some(Expr::Ref(c)), Some(Expr::Ref(s))) => {
let (crit, sum) = (c.range(), s.range());
push(&mut out, s, resized_sum_range(sum, crit.height(), crit.width()));
}
(_, Some(Expr::Number(_) | Expr::Text(_) | Expr::Bool(_) | Expr::Error(_) | Expr::Array(_) | Expr::Missing)) => {}
_ => dynamic = true,
},
Expr::Call(n, _)
if matches!(n.as_str(), "INDIRECT" | "OFFSET" | "NOW" | "TODAY" | "RAND" | "RANDBETWEEN" | "RANDARRAY" | "CELL" | "INFO") =>
{
dynamic = true
}
Expr::Name(_) | Expr::Struct(_) => dynamic = true,
Expr::Call(n, _) if gridcraft_functions::lookup(n).is_none() && !is_special(n) => dynamic = true,
// A range ending in a function's result (`A1:INDEX(B1:B9,n)`) covers the cells between
// the references inside it, not only those references.
Expr::Binary(BinOp::Range, a, b) if !matches!((&**a, &**b), (Expr::Ref(_), Expr::Ref(_))) => {
let mut inner = precedents(wb, sheet, a).0;
inner.extend(precedents(wb, sheet, b).0);
let mut boxes: Vec<Area> = Vec::new();
for area in inner {
match boxes.iter_mut().find(|x| x.sheet == area.sheet) {
Some(x) => x.range = x.range.union(&area.range),
None => boxes.push(area),
}
/// Visits every node of `root` once, parents before children, left to right. Areas found
/// inside a range operator (`A1:INDEX(…)`) also grow that operator's per-sheet bounding boxes.
fn collect(&mut self, root: &Expr) {
let mut pending = vec![PrecedentStep::Visit(root, None)];
let mut nodes = 0usize;
while let Some(step) = pending.pop() {
let (x, boxes) = match step {
PrecedentStep::Visit(x, boxes) => (x, boxes),
PrecedentStep::Close { inner, outer } => {
let areas = self.boxes.get_mut(inner).map(std::mem::take).unwrap_or_default();
for area in areas {
self.add(area, outer);
}
continue;
}
};
nodes = nodes.saturating_add(1);
if nodes > MAX_PRECEDENT_NODES {
self.dynamic = true;
return;
}
out.extend(boxes);
self.visit(x, boxes, &mut pending);
}
_ => {}
});
(out, dynamic)
}
/// Handles one node for [`Precedents::collect`], queueing the children still to visit.
fn visit<'e>(&mut self, x: &'e Expr, boxes: Option<usize>, pending: &mut Vec<PrecedentStep<'e>>) {
match x {
Expr::Ref(r) => self.push(r, r.range(), boxes),
// The cells SUMIF/AVERAGEIF really read from the sum range (see `resized_sum_range`); when
// the criteria range's size isn't known statically, recalculate the formula every time.
Expr::Call(n, args) if resizes_sum_range(n, args.len()) => match (args.first(), args.get(2)) {
(Some(Expr::Ref(c)), Some(Expr::Ref(s))) => {
let (crit, sum) = (c.range(), s.range());
self.push(s, resized_sum_range(sum, crit.height(), crit.width()), boxes);
}
(_, Some(Expr::Number(_) | Expr::Text(_) | Expr::Bool(_) | Expr::Error(_) | Expr::Array(_) | Expr::Missing)) => {}
_ => self.dynamic = true,
},
Expr::Call(n, _)
if matches!(n.as_str(), "INDIRECT" | "OFFSET" | "NOW" | "TODAY" | "RAND" | "RANDBETWEEN" | "RANDARRAY" | "CELL" | "INFO") =>
{
self.dynamic = true
}
Expr::Name(_) | Expr::Struct(_) => self.dynamic = true,
Expr::Call(n, _) if gridcraft_functions::lookup(n).is_none() && !is_special(n) => self.dynamic = true,
// A range ending in a function's result (`A1:INDEX(B1:B9,n)`), or a chain of plain
// references the parser can't fold into one (`A1:B2:C3`), covers the cells between the
// references inside it, not only those references. Both operands are visited once,
// collecting their bounding boxes on the way, which `Close` then records.
Expr::Binary(BinOp::Range, a, b) => {
let inner = self.boxes.len();
self.boxes.push(Vec::new());
pending.push(PrecedentStep::Close { inner, outer: boxes });
pending.push(PrecedentStep::Visit(b, Some(inner)));
pending.push(PrecedentStep::Visit(a, Some(inner)));
return;
}
_ => {}
}
// Children are pushed in reverse so they pop left to right.
let visit = |e: &'e Expr| PrecedentStep::Visit(e, boxes);
match x {
Expr::Array(rows) => pending.extend(rows.iter().flatten().rev().map(visit)),
Expr::Unary(_, e) | Expr::Paren(e) => pending.push(visit(e)),
Expr::Binary(_, a, b) => {
pending.push(visit(b));
pending.push(visit(a));
}
Expr::Call(_, args) => pending.extend(args.iter().rev().map(visit)),
Expr::Invoke(c, args) => {
pending.extend(args.iter().rev().map(visit));
pending.push(visit(c));
}
_ => {}
}
}
}
fn is_special(n: &str) -> bool {
+79
View File
@@ -2331,6 +2331,85 @@ fn median_large_values_recalculate_dependents() {
}
}
/// #2183: `INDEX(…):INDEX(…):…` used to re-walk each operand of the left-associative range chain,
/// costing 2^k for k operands (28 took 40 s; 60 would take millennia), on open and on every edit.
#[test]
fn range_operator_chain_precedents_are_linear() {
let operands: Vec<String> = (1..=60).map(|i| format!("INDEX(A{i},1)")).collect();
let src = format!("=SUM({})", operands.join(":"));
let e = gridcraft_formula::parse(&src).unwrap();
let wb = Workbook::new();
// With the old 2^k walk this call would never return; #2183 asks for under 100 ms.
#[allow(clippy::disallowed_methods)] // a native test: the clock is there
let t0 = std::time::Instant::now();
let (areas, dynamic) = crate::eval::precedents(&wb, 0, &e);
let took = t0.elapsed();
assert!(took < std::time::Duration::from_millis(100), "precedents took {took:?}");
assert!(!dynamic);
// The chain covers A1:A60, not only the 60 cells named in it.
let whole = gridcraft_core::RangeRef::new(c("A1"), c("A60"));
assert!(areas.iter().any(|a| a.sheet == 0 && a.range == whole), "{areas:?}");
// Each node is visited once: 60 references plus one box per range operator.
assert!(areas.len() < 200, "{} areas", areas.len());
// Through the recalculation engine: setting it finishes and editing a cell in the box recalculates.
let mut t = T::new();
for i in 1..=60 {
t.set(&format!("A{i}"), "1");
}
t.set("B1", &src);
assert_eq!(t.num("B1"), 60.0);
t.set("A30", "5");
assert_eq!(t.num("B1"), 64.0);
}
/// A chained range of plain references (`A1:B2:C3`, which the parser can't fold into one
/// reference) covers their bounding box, as the evaluator expands it: an edit inside the box but
/// outside each operand still recalculates.
#[test]
fn chained_plain_range_precedents_cover_the_bounding_box() {
let e = gridcraft_formula::parse("=SUM(A1:B2:C3)").unwrap();
let (areas, dynamic) = crate::eval::precedents(&Workbook::new(), 0, &e);
assert!(!dynamic);
let whole = gridcraft_core::RangeRef::new(c("A1"), c("C3"));
assert!(areas.iter().any(|a| a.sheet == 0 && a.range == whole), "{areas:?}");
let mut t = T::new();
t.set("A1", "1");
t.set("E1", "=SUM(A1:B2:C3)");
assert_eq!(t.num("E1"), 1.0);
t.set("C1", "4");
assert_eq!(t.num("E1"), 5.0);
}
/// `precedents` walks with an explicit stack: an expression nested deeper than any formula
/// within the length limit can build still finds its references, on a small stack.
#[test]
fn precedents_of_deep_nesting_do_not_recurse() {
let (areas, dynamic) = std::thread::Builder::new()
.stack_size(256 * 1024)
.spawn(|| {
let mut e = gridcraft_formula::parse("=A1").unwrap();
for _ in 0..10_000 {
e = gridcraft_formula::Expr::Paren(Box::new(e));
}
let found = crate::eval::precedents(&Workbook::new(), 0, &e);
// Dropping the deep tree recurses; leak it so only `precedents` is under test.
std::mem::forget(e);
found
})
.unwrap()
.join()
.unwrap();
assert!(!dynamic);
assert_eq!(areas.len(), 1);
// Shallow nesting stays static.
let e = gridcraft_formula::parse("=((((A1))))").unwrap();
let (areas, dynamic) = crate::eval::precedents(&Workbook::new(), 0, &e);
assert!(!dynamic);
assert_eq!(areas.len(), 1);
}
#[test]
fn cell_filename_names_the_workbook_file() {
// Excel returns `<folder>/[<file>]<sheet>`, and the classic sheet-name formula splits it on
+1 -1
View File
@@ -26,7 +26,7 @@ use serde::{Deserialize, Serialize};
pub use axis::{Scale, format_number, nice_scale};
pub use raster::rasterize;
pub use render::{Swatch, legend_entries, render, supports_trendline};
pub use resolve::{default_palette, resolve, series_color, series_plot_kind};
pub use resolve::{default_palette, has_volatile_series, resolve, series_color, series_plot_kind};
pub use sparkline::render_sparkline;
/// Straight (non-premultiplied) RGBA colour.
+126 -5
View File
@@ -155,6 +155,131 @@ fn eval_list_in(wb: &Workbook, sheet: usize, formula: &str, limit: usize, depth:
}
}
/// Whether a series formula of `ch` (chart of `sheet`) calls a function whose result changes
/// between evaluations (`NOW`, `TODAY`, `RAND`, `RANDBETWEEN`, `RANDARRAY`), so a result resolved
/// earlier is stale. Defined names the formulas use (`=FreshClock`, `Sheet1!Rates`, a LAMBDA name)
/// are followed into their own formulas, the way the evaluator looks them up; each name once per
/// chart, so a cycle ends. Names nested deeper than [`MAX_NAME_DEPTH`], or more than
/// [`MAX_NAMES_FOLLOWED`] distinct names, count as volatile: a chart whose stability isn't
/// established is not cached.
pub fn has_volatile_series(wb: &Workbook, sheet: usize, ch: &Chart) -> bool {
let mut seen = Seen::new();
ch.series.iter().any(|s| {
[s.name.as_deref(), s.categories.as_deref(), Some(s.values.as_str()), s.bubble_sizes.as_deref()]
.into_iter()
.flatten()
.any(|f| formula_volatile(wb, sheet, f, 0, &mut seen))
})
}
/// Names followed inside names before a formula counts as volatile.
const MAX_NAME_DEPTH: u32 = 8;
/// Distinct defined names followed per chart before the chart counts as volatile.
const MAX_NAMES_FOLLOWED: usize = 64;
/// Defined names already followed for one chart, each with the sheet its formula is read on.
type Seen<'a> = Vec<(&'a gridcraft_model::DefinedName, usize)>;
fn formula_volatile<'a>(wb: &'a Workbook, sheet: usize, f: &str, depth: u32, seen: &mut Seen<'a>) -> bool {
if calls_volatile(f) {
return true;
}
name_tokens(f).into_iter().any(|t| {
let Some((n, at)) = defined_name(wb, sheet, t) else { return false };
// Followed already, or being followed (a cycle): a volatile call in it is found there.
if seen.iter().any(|&(m, s)| std::ptr::eq(m, n) && s == at) {
return false;
}
if depth >= MAX_NAME_DEPTH || seen.len() >= MAX_NAMES_FOLLOWED {
return true;
}
seen.push((n, at));
formula_volatile(wb, at, &n.formula, depth + 1, seen)
})
}
/// Whether formula text `f` calls one of the volatile functions by name.
fn calls_volatile(f: &str) -> bool {
const VOLATILE: [&str; 5] = ["NOW", "TODAY", "RAND", "RANDBETWEEN", "RANDARRAY"];
// Formulas read from a file may carry the `_xlfn.` prefix of newer functions.
let up = f.to_ascii_uppercase().replace("_XLFN.", "");
VOLATILE.iter().any(|name| {
up.match_indices(name).any(|(i, _)| {
let before = up[..i].chars().next_back();
let after = up[i + name.len()..].trim_start();
before.is_none_or(|c| !(c.is_alphanumeric() || c == '_' || c == '.')) && after.starts_with('(')
})
})
}
fn is_ident_char(c: char) -> bool {
c.is_alphanumeric() || c == '_' || c == '.' || c == '\\'
}
/// End of the identifier starting at byte `from` of `f`.
fn ident_end(f: &str, from: usize) -> usize {
f[from..].char_indices().find(|&(_, c)| !is_ident_char(c)).map_or(f.len(), |(j, _)| from + j)
}
/// End (past the closing quote) of the `q`-quoted text opening at byte `from`; a doubled quote
/// is an escaped one. `None` when it is not closed.
fn quoted_end(f: &str, from: usize, q: char) -> Option<usize> {
let mut i = from + q.len_utf8();
loop {
let j = i + f[i..].find(q)?;
if f[j + q.len_utf8()..].starts_with(q) {
i = j + 2 * q.len_utf8();
} else {
return Some(j + q.len_utf8());
}
}
}
/// The words of formula text `f` that may be defined names: `Name`, `Sheet1!Name`,
/// `'Bob''s'!Name`. String literals and numbers are skipped; function names, column letters and
/// the like come along too and simply aren't found as names.
fn name_tokens(f: &str) -> Vec<&str> {
let mut out = Vec::new();
let mut i = 0;
while let Some(c) = f[i..].chars().next() {
let start = i;
let end = match c {
'"' => {
i = quoted_end(f, i, '"').unwrap_or(f.len());
continue;
}
'\'' => quoted_end(f, i, '\'').unwrap_or(f.len()),
c if is_ident_char(c) => ident_end(f, i),
c => {
i += c.len_utf8();
continue;
}
};
i = end;
if f[end..].starts_with('!') {
let e = ident_end(f, end + 1);
if e > end + 1 {
out.push(&f[start..e]);
}
i = e;
} else if c != '\'' && !c.is_ascii_digit() {
out.push(&f[start..end]);
}
}
out
}
/// The defined name `t` (`Name` or `Sheet1!Name`) seen from `sheet`, and the sheet its own
/// formula is read on.
fn defined_name<'a>(wb: &'a Workbook, sheet: usize, t: &str) -> Option<(&'a gridcraft_model::DefinedName, usize)> {
// `'Bob''s'!Name`: the sheet is unquoted as everywhere else (#2012).
let (at, bare) = match gridcraft_core::split_sheet(t) {
(Some(s), bare) => (wb.sheet_index(&s)?, bare),
(None, bare) => (sheet, bare),
};
wb.name(bare, at).map(|n| (n, n.scope.unwrap_or(at)))
}
fn to_num(v: &Value) -> Option<f64> {
match v {
Value::Number(n) if n.is_finite() => Some(*n),
@@ -305,11 +430,7 @@ fn category_table(wb: &Workbook, sheet: usize, formula: &str, values: &str, limi
/// Whether `t` names a defined name seen from `sheet` (`Name`, or `Sheet1!Name` looked up on
/// that sheet), the way the evaluator looks names up.
fn is_defined_name(wb: &Workbook, sheet: usize, t: &str) -> bool {
// `'Bob''s'!Name`: the sheet is unquoted as everywhere else (#2012).
match gridcraft_core::split_sheet(t) {
(Some(s), bare) => wb.sheet_index(&s).is_some_and(|i| wb.name(bare, i).is_some()),
(None, _) => wb.name(t, sheet).is_some(),
}
defined_name(wb, sheet, t).is_some()
}
/// A series name: a reference/formula or a defined name (`[0]!Title` in a file) is evaluated,
+93
View File
@@ -1680,3 +1680,96 @@ fn a_series_named_by_a_name_on_a_sheet_with_a_bang_resolves() {
let d = resolve(&wb, 0, &c);
assert_eq!(d.series.first().map(|s| s.name.as_str()), Some("Takings"));
}
#[test]
fn volatile_series_formulas_are_detected() {
let wb = Workbook::new();
let with = |values: &str, name: Option<&str>| {
let mut c = chart(ChartKind::Line);
c.series.push(Series {
name: name.map(Into::into),
categories: None,
values: values.into(),
bubble_sizes: None,
color: None,
secondary: false,
kind: None,
smooth: false,
quartiles: Default::default(),
trendlines: vec![],
});
c
};
assert!(has_volatile_series(&wb, 0, &with("=NOW()", None)));
assert!(has_volatile_series(&wb, 0, &with("=SUM(A1:A3)*rand ()", None)));
assert!(has_volatile_series(&wb, 0, &with("Sheet1!$B$2:$B$9", Some("=TODAY()"))));
assert!(has_volatile_series(&wb, 0, &with("=RANDARRAY(5)", None)));
assert!(has_volatile_series(&wb, 0, &with("=_xlfn.RANDARRAY(5)", None)));
assert!(!has_volatile_series(&wb, 0, &with("Sheet1!$B$2:$B$9", Some("Sheet1!$B$1"))));
assert!(!has_volatile_series(&wb, 0, &with("=MYRAND(A1)", None)));
assert!(!has_volatile_series(&wb, 0, &with("=Sheet1!$NOW", None)));
assert!(!has_volatile_series(&wb, 0, &chart(ChartKind::Line)));
}
#[test]
fn volatile_defined_names_are_followed() {
let with = |values: &str| {
let mut c = chart(ChartKind::Line);
c.series.push(Series {
name: None,
categories: None,
values: values.into(),
bubble_sizes: None,
color: None,
secondary: false,
kind: None,
smooth: false,
quartiles: Default::default(),
trendlines: vec![],
});
c
};
let name = |name: &str, scope: Option<usize>, formula: &str| gridcraft_model::DefinedName {
name: name.into(),
scope,
formula: formula.into(),
comment: String::new(),
hidden: false,
};
let mut wb = Workbook::new();
wb.sheet_mut(0).unwrap().name = "Bob's".into();
wb.names.push(name("FreshClock", None, "NOW()"));
wb.names.push(name("Later", None, "FreshClock+1"));
wb.names.push(name("Dice", Some(0), "RANDBETWEEN(1,6)"));
wb.names.push(name("Rates", None, "'Bob''s'!$A$1:$A$3"));
wb.names.push(name("Loop", None, "Loop+1"));
wb.names.push(name("Spin", None, "Spin+NOW()"));
wb.names.push(name("Roll", None, "LAMBDA(n,n*RAND())"));
// A workbook-scoped name, a name used inside another, and a sheet-scoped one, bare or qualified.
assert!(has_volatile_series(&wb, 0, &with("=FreshClock")));
assert!(has_volatile_series(&wb, 0, &with("=SUM(Later, 1)")));
assert!(has_volatile_series(&wb, 0, &with("=Dice*2")));
assert!(has_volatile_series(&wb, 0, &with("='Bob''s'!Dice")));
assert!(has_volatile_series(&wb, 0, &with("=Roll(3)")));
// A self-referencing name ends (it evaluates to an error, not a changing value); one that
// also calls NOW is volatile.
assert!(!has_volatile_series(&wb, 0, &with("=Loop")));
assert!(has_volatile_series(&wb, 0, &with("=Spin")));
// Many series over the same stable name stay cacheable: each name is followed once.
let mut many = with("=Rates");
let first = many.series[0].clone();
many.series.extend(std::iter::repeat_n(first, 100));
assert!(!has_volatile_series(&wb, 0, &many));
// Names nested deeper than the limit are not known to be stable.
for i in 0..10 {
wb.names.push(name(&format!("Deep{i}"), None, &format!("Deep{}", i + 1)));
}
assert!(has_volatile_series(&wb, 0, &with("=Deep0")));
// A name that refers to cells, a name inside a string and a name of another scope are stable.
assert!(!has_volatile_series(&wb, 0, &with("=Rates")));
assert!(!has_volatile_series(&wb, 0, &with("=\"FreshClock\"")));
// Dice belongs to sheet 0: seen from another sheet it isn't defined.
wb.sheets.push(Arc::new(gridcraft_model::Sheet::new("Other")));
assert!(!has_volatile_series(&wb, 1, &with("=Dice")));
assert!(has_volatile_series(&wb, 1, &with("=FreshClock")));
}
+6 -2
View File
@@ -454,8 +454,9 @@ pub fn number_to_text(n: f64) -> String {
if !(1e-9..1e15).contains(&a) && a.round() != a || a >= 1e15 {
return sci_text(n, 15);
}
// A value just below 1e15 can round up to 1e15 at 15 significant digits.
if format!("{:.14e}", a).parse::<f64>().is_ok_and(|r| r >= 1e15) {
// A value just below 1e15 can round up to 1e15 at 15 significant digits. Only that narrow
// band pays for the exact formatting: it runs for every number literal a formula prints.
if a >= 9.999_999_999_999_9e14 && format!("{:.14e}", a).parse::<f64>().is_ok_and(|r| r >= 1e15) {
return sci_text(n, 15);
}
// 15 significant digits, trailing zeros trimmed.
@@ -506,6 +507,9 @@ mod tests {
assert_eq!(number_to_text(999999999999999.9), "1E+15");
assert_eq!(number_to_text(-999999999999999.9), "-1E+15");
assert_eq!(number_to_text(999999999999999.0), "999999999999999");
assert_eq!(number_to_text(999999999999999.5), "1E+15");
assert_eq!(number_to_text(999999999999999.4), "999999999999999");
assert_eq!(number_to_text(999999999999989.9), "999999999999990");
}
#[test]
+1
View File
@@ -21,6 +21,7 @@ web-time = { workspace = true }
thiserror = { workspace = true }
log = { workspace = true }
gridcraft-xlsx = { workspace = true }
craft-csv = { workspace = true }
gridcraft-chart = { workspace = true }
gridcraft-pdf = { workspace = true }
gridcraft-math = { workspace = true }
+2
View File
@@ -26,6 +26,7 @@ pub mod spelling;
pub mod stash;
pub mod view;
pub mod whatif;
pub mod wordart;
use gridcraft_calc::Key;
use gridcraft_core::{CellRef, RangeRef};
@@ -176,6 +177,7 @@ pub fn command_specs() -> &'static [CommandSpec] {
v.extend(extra::specs());
v.extend(macros::specs());
v.extend(draw::specs());
v.extend(wordart::specs());
v.extend(spelling::specs());
v.extend(solver::specs());
v.extend(pivot::specs());
+4 -3
View File
@@ -166,14 +166,15 @@ fn filter_blank(v: &Value) -> bool {
/// A table's own filter is left to the table, and a row typed after a blank row does not count.
/// A formula counts before it is calculated: this runs inside the edit, ahead of the recalc.
pub(crate) fn grow_autofilter(sh: &mut Sheet, at: CellRef) {
if !sh.cells.get(at).is_some_and(|c| !filter_blank(&c.value) || c.formula.is_some()) {
return;
}
// The AutoFilter checks come first: they are cheap, and most sheets have none.
let Some(af) = sh.autofilter.as_ref() else { return };
let r = af.range;
if r.end.row.checked_add(1) != Some(at.row) || at.col < r.start.col || at.col > r.end.col || sh.tables.iter().any(|t| t.range == r) {
return;
}
if !sh.cells.get(at).is_some_and(|c| !filter_blank(&c.value) || c.formula.is_some()) {
return;
}
let grown = RangeRef::new(r.start, CellRef::new(at.row, r.end.col));
if let Some(af) = sh.autofilter.as_mut() {
af.range = grown;
+6 -1
View File
@@ -183,8 +183,11 @@ pub(super) fn split_delimited(text: &str, delims: &[char], consecutive: bool, qu
let mut in_q = false;
let mut last_delim = false;
let mut chars = text.chars().peekable();
let mut at_start = true;
while let Some(c) = chars.next() {
if c == quote {
if c == quote && (in_q || at_start) {
at_start = false;
last_delim = false;
if in_q && chars.peek() == Some(&quote) {
cur.push(quote);
chars.next();
@@ -199,9 +202,11 @@ pub(super) fn split_delimited(text: &str, delims: &[char], consecutive: bool, qu
}
out.push(std::mem::take(&mut cur));
last_delim = true;
at_start = true;
continue;
}
last_delim = false;
at_start = false;
cur.push(c);
}
out.push(cur);
+46 -10
View File
@@ -63,6 +63,11 @@ pub(super) fn selection_move(s: &mut Session, p: &Json) -> Result<Json> {
let d = s.doc_mut()?;
let Some(sheet) = d.wb.active() else { return Err(EngineError::NoDocument) };
let sel = &mut d.selection;
if extend && !jmp {
// Shift+arrows move the edge the anchor doesn't hold, keeping merged cells whole.
sel.extend_by(sheet, dr, dc);
return Ok(json!({"selection": sel.a1(), "active": sel.active.a1()}));
}
// Extension moves the far corner, not the active cell.
let from = if extend { sel.moving_end() } else { sel.active };
// Skip hidden rows/columns and step over merged areas.
@@ -77,23 +82,32 @@ pub(super) fn selection_move(s: &mut Session, p: &Json) -> Result<Json> {
jump(sheet, from, dr.signum(), dc.signum())
} else {
let mut t = from.offset_clamped(dr, dc);
if !extend && let Some(m) = sheet.merge_at(from) {
// Leaving a merged cell moves from its edge.
if dr > 0 {
t = CellRef::new(m.end.offset_clamped(dr, 0).row, t.col);
} else if dc > 0 {
t = CellRef::new(t.row, m.end.offset_clamped(0, dc).col);
}
}
let mut guard = 0;
while (sheet.is_row_hidden(t.row) || sheet.is_col_hidden(t.col)) && guard < 100_000 {
let n = t.offset_clamped(dr.signum(), dc.signum());
// Step each hidden axis along its own direction, so a diagonal exit from a merge
// (Tab-run return column) keeps its column while skipping hidden rows.
let mut n = CellRef::new(
if sheet.is_row_hidden(t.row) { t.offset_clamped(dr.signum(), 0).row } else { t.row },
if sheet.is_col_hidden(t.col) { t.offset_clamped(0, dc.signum()).col } else { t.col },
);
if n == t {
// The hidden axis has no direction (a move along the other axis): walk as before.
n = t.offset_clamped(dr.signum(), dc.signum());
}
if n == t {
break;
}
t = n;
guard += 1;
}
if !extend && let Some(m) = sheet.merge_at(from) {
// Leaving a merged cell moves from its edge.
if dr > 0 {
t = CellRef::new(m.end.offset_clamped(dr, 0).row, from.col);
} else if dc > 0 {
t = CellRef::new(from.row, m.end.offset_clamped(0, dc).col);
}
}
t
};
if let Some(m) = sheet.merge_at(to)
@@ -140,7 +154,29 @@ pub(super) fn selection_next(s: &mut Session, p: &Json) -> Result<Json> {
let fwd = bool_param(p, "forward").unwrap_or(true);
let by_row = bool_param(p, "byRow").unwrap_or(false);
let d = s.doc_mut()?;
d.selection.cycle(fwd, by_row);
let Some(sheet) = d.wb.active() else { return Err(EngineError::NoDocument) };
let sel = &d.selection;
if sel.is_one_merged_cell(sheet) {
// A selection that is just one merged cell is one cell, as in Excel: Enter and Tab leave
// it for the next cell instead of walking the cells it covers.
let step = if fwd { 1 } else { -1 };
let (dr, dc) = if by_row { (0, step) } else { (step, 0) };
selection_move(s, &json!({"dr": dr, "dc": dc}))?;
return Ok(json!({"active": s.doc()?.selection.active.a1()}));
}
// Inside a larger selection the cells a merged cell covers are skipped: it is visited once,
// at its top-left cell.
let area = sel.current();
let cells =
(u64::from(area.end.row.saturating_sub(area.start.row)) + 1).saturating_mul(u64::from(area.end.col.saturating_sub(area.start.col)) + 1);
let mut next = sel.clone();
for _ in 0..cells.min(1 << 20) {
next.cycle(fwd, by_row);
if sheet.merge_at(next.active).is_none_or(|m| m.start == next.active) {
break;
}
}
d.selection = next;
Ok(json!({"active": d.selection.active.a1()}))
}
+7 -1
View File
@@ -1222,9 +1222,15 @@ fn shape_set_text(s: &mut Session, p: &Json) -> Result<Json> {
let sheet = d.wb.active_sheet();
let sh = d.wb.active().ok_or_else(|| bad("shape.setText", "no such text box"))?;
let shape = sh.shapes.iter().find(|shape| shape.id == id).ok_or_else(|| bad("shape.setText", "no such text box"))?;
if shape.kind != ShapeKind::TextBox {
if !matches!(shape.kind, ShapeKind::TextBox | ShapeKind::WordArt) {
return Err(bad("shape.setText", "the shape is not a text box"));
}
if shape.kind == ShapeKind::WordArt && text.is_empty() {
return Err(bad("shape.setText", "WordArt text must not be empty"));
}
if shape.kind == ShapeKind::WordArt && text.chars().nth(super::wordart::MAX_CHARS).is_some() {
return Err(bad("shape.setText", format!("`text` is longer than {} characters", super::wordart::MAX_CHARS)));
}
// Sheet protection locks drawing objects (there is no "edit objects" allowance yet).
if sh.is_protected() {
return Err(EngineError::Other(PROTECTED.into()));
+15
View File
@@ -213,6 +213,21 @@ fn draw_shape(pg: &mut Page, wb: &Workbook, shp: &gridcraft_model::Shape, b: &Bl
ShapeTextAlign::Center => x + (w - lw) / 2.0,
ShapeTextAlign::Right => x + w - 2.0 * b.k - lw,
};
// WordArt's drop shadow: the line again, offset, in the shadow colour (under the text).
if let Some(sc) = fm.text_shadow.and_then(|c| rgb_of(c, wb)) {
let off = (size * 0.06).max(0.5);
pg.text(lx + off, yy + off, size, font, sc, l);
}
// WordArt's text outline, as on screen: the line in the outline colour, offset in eight
// directions by half the outline width, under the filled text.
if let Some((c, ow)) = fm.text_outline
&& let Some(oc) = rgb_of(c, wb)
{
let d = (ow * b.scale / 2.0).clamp(0.25, 20.0);
for (dx, dy) in [(-1.0, -1.0), (0.0, -1.0), (1.0, -1.0), (-1.0, 0.0), (1.0, 0.0), (-1.0, 1.0), (0.0, 1.0), (1.0, 1.0)] {
pg.text(lx + dx * d, yy + dy * d, size, font, oc, l);
}
}
pg.text(lx, yy, size, font, color, l);
yy += size * 1.2;
}
+241
View File
@@ -0,0 +1,241 @@
//! Insert › Text › WordArt: decorative text as a floating shape (`ShapeKind::WordArt`) built on
//! the text-box machinery (anchor, move/resize/delete, undo, XLSX shape text body, printing).
//! The style palette below is original to GridCraft.
use gridcraft_model::{Anchor, Color, Shape, ShapeFormat, ShapeKind, ShapeTextAlign, ShapeTextAnchor};
use serde_json::{Value as Json, json};
use super::{
CommandSpec, EngineError, PROTECTED, Result, Session, anchor_mode_param, bad, bool_param, cell_param, cmd, edit, f64_param, has_doc, ok,
u32_param,
};
/// One entry of the style gallery.
pub struct WordArtStyle {
pub name: &'static str,
pub fill: Color,
pub outline: Option<(Color, f32)>,
pub shadow: Option<Color>,
pub bold: bool,
pub italic: bool,
/// Font size in points.
pub size: f32,
}
const fn rgb(v: u32) -> Color {
Color::Rgb(v)
}
/// GridCraft's own WordArt presets. Theme colours where it makes sense (they follow the theme),
/// fixed colours for the rest.
pub const STYLES: [WordArtStyle; 10] = [
WordArtStyle { name: "Classic", fill: Color::Theme(4, 0), outline: None, shadow: None, bold: true, italic: false, size: 36.0 },
WordArtStyle { name: "Midnight", fill: rgb(0x1B2A49), outline: None, shadow: Some(rgb(0x8A94A6)), bold: true, italic: false, size: 40.0 },
WordArtStyle { name: "Sunrise", fill: rgb(0xF2A33A), outline: Some((rgb(0xB5451B), 1.5)), shadow: None, bold: true, italic: false, size: 40.0 },
WordArtStyle {
name: "Lagoon",
fill: rgb(0x1F9E9A),
outline: Some((rgb(0x0B4F5C), 1.0)),
shadow: Some(rgb(0x9CC9C7)),
bold: true,
italic: true,
size: 36.0,
},
WordArtStyle { name: "Ember", fill: rgb(0xD64527), outline: None, shadow: Some(rgb(0x5A1A0E)), bold: true, italic: false, size: 44.0 },
WordArtStyle {
name: "Mint Outline",
fill: rgb(0xFFFFFF),
outline: Some((rgb(0x2E9B6A), 2.0)),
shadow: None,
bold: true,
italic: false,
size: 40.0,
},
WordArtStyle { name: "Ink Stamp", fill: rgb(0x2F3A8F), outline: Some((rgb(0x2F3A8F), 0.5)), shadow: None, bold: true, italic: false, size: 32.0 },
WordArtStyle { name: "Orchid", fill: rgb(0x8E4FB5), outline: None, shadow: Some(rgb(0xD9C2E8)), bold: false, italic: true, size: 40.0 },
WordArtStyle {
name: "Graphite",
fill: rgb(0x4A4F57),
outline: Some((rgb(0xFFFFFF), 1.0)),
shadow: Some(rgb(0x9AA0A8)),
bold: true,
italic: false,
size: 36.0,
},
WordArtStyle {
name: "Lime Pop",
fill: rgb(0xA6D61C),
outline: Some((rgb(0x3F5A00), 1.5)),
shadow: Some(rgb(0x6E8B2C)),
bold: true,
italic: true,
size: 40.0,
},
];
/// The text of a WordArt inserted without `text`.
pub const DEFAULT_TEXT: &str = "Your text here";
/// Longest WordArt text (characters).
pub const MAX_CHARS: usize = 1000;
const MAX_POINTS: f64 = 400.0;
const MAX_SIDE: f64 = 4000.0;
pub fn specs() -> Vec<CommandSpec> {
vec![
cmd!(
"insert.wordArt",
"WordArt",
["Insert", "Text"],
None,
"{style?: preset name or 0-based index (Classic, Midnight, Sunrise, Lagoon, Ember, Mint Outline, Ink Stamp, Orchid, Graphite, Lime Pop), text?, at?, width?, height?, fontSize?, bold?, italic?, fill?: hex, outline?: hex|false, outlineWidth?, shadow?: hex|bool}",
has_doc,
insert_word_art
),
cmd!(
"wordArt.set",
"Format WordArt",
[],
None,
"{id: WordArt id, style?, text?, fontSize?, bold?, italic?, fill?: hex, outline?: hex|false, outlineWidth?, shadow?: hex|bool}",
has_doc,
set_word_art
),
]
}
fn style_param(command: &str, p: &Json) -> Result<Option<&'static WordArtStyle>> {
match p.get("style") {
None | Some(Json::Null) => Ok(None),
Some(Json::Number(n)) => n
.as_u64()
.and_then(|i| STYLES.get(usize::try_from(i).ok()?))
.map(Some)
.ok_or_else(|| bad(command, format!("`style` index must be 0..{}", STYLES.len() - 1))),
Some(Json::String(name)) => STYLES
.iter()
.find(|s| s.name.eq_ignore_ascii_case(name.trim()))
.map(Some)
.ok_or_else(|| bad(command, format!("unknown WordArt style `{name}`"))),
Some(_) => Err(bad(command, "`style` must be a preset name or index")),
}
}
fn text_param(command: &str, p: &Json) -> Result<Option<String>> {
match p.get("text") {
None | Some(Json::Null) => Ok(None),
Some(Json::String(t)) if t.is_empty() => Err(bad(command, "`text` must not be empty")),
Some(Json::String(t)) if t.chars().nth(MAX_CHARS).is_some() => Err(bad(command, format!("`text` is longer than {MAX_CHARS} characters"))),
Some(Json::String(t)) => Ok(Some(t.clone())),
Some(_) => Err(bad(command, "`text` must be a string")),
}
}
/// Applies a preset, then explicit overrides, to a shape's format and fill.
fn apply(command: &str, shape: &mut Shape, style: Option<&WordArtStyle>, p: &Json) -> Result<()> {
if let Some(st) = style {
shape.format.text_color = Some(st.fill);
shape.format.text_outline = st.outline;
shape.format.text_shadow = st.shadow;
shape.format.bold = st.bold;
shape.format.italic = st.italic;
shape.format.font_size = Some(st.size);
}
let f = &mut shape.format;
if let Some(c) = p.get("fill").filter(|v| !v.is_null()) {
f.text_color = Some(super::format::color_param(Some(c)).ok_or_else(|| bad(command, "`fill` must be a hex colour"))?);
}
match p.get("outline") {
None | Some(Json::Null) => {}
Some(Json::Bool(false)) => f.text_outline = None,
Some(v) => {
let c = super::format::color_param(Some(v)).ok_or_else(|| bad(command, "`outline` must be a hex colour or false"))?;
let w = f64_param(p, "outlineWidth").map_or(f.text_outline.map_or(1.5, |o| o.1), |w| w.clamp(0.25, 20.0) as f32);
f.text_outline = Some((c, w));
}
}
if let (Some(w), Some((c, _))) = (f64_param(p, "outlineWidth"), f.text_outline) {
f.text_outline = Some((c, w.clamp(0.25, 20.0) as f32));
}
match p.get("shadow") {
None | Some(Json::Null) => {}
Some(Json::Bool(false)) => f.text_shadow = None,
Some(Json::Bool(true)) => f.text_shadow = Some(f.text_shadow.unwrap_or(rgb(0x808080))),
Some(v) => {
f.text_shadow = Some(super::format::color_param(Some(v)).ok_or_else(|| bad(command, "`shadow` must be a hex colour or a boolean"))?)
}
}
if let Some(b) = bool_param(p, "bold") {
f.bold = b;
}
if let Some(b) = bool_param(p, "italic") {
f.italic = b;
}
if let Some(sz) = f64_param(p, "fontSize") {
f.font_size = Some(sz.clamp(1.0, MAX_POINTS) as f32);
}
Ok(())
}
fn insert_word_art(s: &mut Session, p: &Json) -> Result<Json> {
let style = style_param("insert.wordArt", p)?.unwrap_or(&STYLES[0]);
let text = text_param("insert.wordArt", p)?.unwrap_or_else(|| DEFAULT_TEXT.to_string());
let d = s.doc()?;
if d.wb.active().is_some_and(|sh| sh.is_protected()) {
return Err(EngineError::Other(PROTECTED.into()));
}
let id = d.wb.next_object_id();
let at = cell_param(s, p, "at")?.unwrap_or(d.selection.active);
let mut shape = Shape {
id,
kind: ShapeKind::WordArt,
anchor: Anchor { cell: at, dx: 0.0, dy: 0.0, width: 0.0, height: 0.0, mode: anchor_mode_param(p).unwrap_or_default() },
fill: Color::Auto,
line: Color::Auto,
text: text.clone(),
format: ShapeFormat { no_line: true, align: Some(ShapeTextAlign::Center), anchor: Some(ShapeTextAnchor::Middle), ..Default::default() },
math: Default::default(),
};
apply("insert.wordArt", &mut shape, Some(style), p)?;
// Default box: the longest line at the font size (about 0.62 em per character) plus padding.
let pt = f64::from(shape.format.font_size.unwrap_or(36.0));
let longest = text.split('\n').map(|l| l.chars().count()).max().unwrap_or(1) as f64;
let lines = text.split('\n').count() as f64;
let width = f64_param(p, "width").unwrap_or(longest * pt * 0.62 * 96.0 / 72.0 + 24.0).clamp(8.0, MAX_SIDE);
let height = f64_param(p, "height").unwrap_or(lines * pt * 1.35 * 96.0 / 72.0 + 12.0).clamp(8.0, MAX_SIDE);
shape.anchor.width = width as f32;
shape.anchor.height = height as f32;
let sheet = d.wb.active_sheet();
edit(s, |cx| {
cx.sheet_mut(sheet)?.shapes.push(shape);
Ok(json!({"shape": id}))
})
}
fn set_word_art(s: &mut Session, p: &Json) -> Result<Json> {
let id = u32_param(p, "id").ok_or_else(|| bad("wordArt.set", "`id` must be a WordArt id"))?;
let style = style_param("wordArt.set", p)?;
let text = text_param("wordArt.set", p)?;
let d = s.doc()?;
let sheet = d.wb.active_sheet();
let sh = d.wb.active().ok_or_else(|| bad("wordArt.set", "no such WordArt"))?;
let current = sh.shapes.iter().find(|x| x.id == id).ok_or_else(|| bad("wordArt.set", "no such WordArt"))?;
if current.kind != ShapeKind::WordArt {
return Err(bad("wordArt.set", "the shape is not WordArt"));
}
if sh.is_protected() {
return Err(EngineError::Other(PROTECTED.into()));
}
let mut updated = current.clone();
apply("wordArt.set", &mut updated, style, p)?;
if let Some(t) = text {
updated.text = t;
}
if &updated == current {
return ok();
}
edit(s, |cx| {
let shape = cx.sheet_mut(sheet)?.shapes.iter_mut().find(|x| x.id == id).ok_or_else(|| bad("wordArt.set", "no such WordArt"))?;
*shape = updated;
Ok(Json::Null)
})
}
+37 -301
View File
@@ -1,156 +1,57 @@
//! SYLK (`.slk`) import: the symbolic link text format older spreadsheets and some legacy systems
//! export. One record per line, fields separated by `;` (`;;` is a literal semicolon):
//!
//! - `ID` starts the file; `E` ends it.
//! - `C;Y<row>;X<col>;K<value>;E<formula>` is a cell. `Y` and `X` may be left out to stay on the
//! current row or column. `K` is a number, `"text"`, `TRUE`/`FALSE` or an error; `E` is a
//! formula in R1C1 notation; `S;R<row>;C<col>` shares the formula of another cell.
//! - `P;P<code>` adds a number format to the file's list; `F;P<n>;Y<row>;X<col>` gives a cell the
//! n-th format (0-based), and `F;W<first> <last> <width>` sets column widths in characters.
//!
//! Other records (fonts, options, window settings) are skipped. Formulas are converted to A1 with
//! the same relative and absolute references, as Excel does when it opens the file.
//! SYLK (`.slk`) import. `craft_csv::sylk` reads the file (values, formulas rewritten from R1C1 to
//! A1, number formats, column widths); this module makes it a one-sheet workbook: formulas parsed
//! (a formula GridCraft can't parse is skipped and counted), error values recognised, formats
//! interned as styles and widths converted from characters.
use std::collections::HashMap;
use std::sync::Arc;
use gridcraft_core::{CellError, CellRef, MAX_COLS, MAX_ROWS, Value, col_to_letters};
use craft_csv::sylk;
use gridcraft_core::{CellError, CellRef, Value};
use gridcraft_model::{Cell, Formula, NumFmt, Sheet, Style, Workbook};
use crate::{EngineError, Result};
/// The largest SYLK file read, as for other text imports: past it the file is refused.
const MAX_BYTES: usize = 512 << 20;
/// Column-width records kept from one file; each can cover every column.
const MAX_WIDTH_RECORDS: usize = 1024;
/// Reads a SYLK file into a one-sheet workbook. Returns the workbook and its warnings.
pub fn read(name: &str, bytes: &[u8]) -> Result<(Workbook, Vec<String>)> {
if bytes.len() > MAX_BYTES {
return Err(EngineError::Other(format!("We can't open '{name}': the SYLK file is larger than {} MB.", MAX_BYTES >> 20)));
}
let text = match std::str::from_utf8(bytes) {
Ok(s) => s.to_string(),
// Older exports are in a single-byte code page: keep each byte as its Latin-1 character.
Err(_) => bytes.iter().map(|&b| char::from(b)).collect(),
};
let text = text.strip_prefix('\u{feff}').unwrap_or(&text);
if !text.starts_with("ID") {
return Err(EngineError::Other(format!("We can't open '{name}': it isn't a SYLK file (it doesn't start with an ID record).")));
}
let file = sylk::read(bytes).map_err(|e| match e {
sylk::Error::TooLarge => EngineError::Other(format!("We can't open '{name}': the SYLK file is larger than {} MB.", sylk::MAX_BYTES >> 20)),
_ => EngineError::Other(format!("We can't open '{name}': it isn't a SYLK file (it doesn't start with an ID record).")),
})?;
let mut wb = Workbook::new();
let mut sheet = Sheet::new("Sheet1");
let mut formats: Vec<String> = Vec::new();
// Cells' number formats by index into `formats`, applied once every format is known.
let mut cell_formats: Vec<(CellRef, usize)> = Vec::new();
let mut formulas: HashMap<CellRef, String> = HashMap::new();
let mut widths: Vec<(u32, u32, f64)> = Vec::new();
let (mut row, mut col) = (0u32, 0u32);
let mut skipped = 0usize;
for line in text.lines() {
let fields = split_fields(line);
let Some((kind, rest)) = fields.split_first() else { continue };
match kind.as_str() {
"C" => {
let (mut value, mut formula, mut shared) = (None, None, None::<(Option<u32>, Option<u32>)>);
let mut bad = false;
for f in rest {
let (tag, arg) = f.split_at(f.chars().next().map_or(0, char::len_utf8));
match tag {
"Y" => match position(arg, MAX_ROWS) {
Some(r) => row = r,
None => bad = true,
},
"X" => match position(arg, MAX_COLS) {
Some(c) => col = c,
None => bad = true,
},
"K" => value = Some(parse_value(arg)),
"E" => formula = Some(arg.to_string()),
"S" => shared = Some((None, None)),
// In a shared-formula cell, R and C name the cell whose formula it shares.
"R" if shared.is_some() => shared = shared.map(|(_, c)| (position(arg, MAX_ROWS), c)),
"C" if shared.is_some() => shared = shared.map(|(r, _)| (r, position(arg, MAX_COLS))),
_ => {}
}
}
if bad {
skipped += 1;
continue;
}
let at = CellRef::new(row, col);
if let Some((Some(r), Some(c))) = shared {
formula = formulas.get(&CellRef::new(r, c)).cloned();
}
let mut cell = sheet.cell(at).cloned().unwrap_or_else(|| Cell::value(Value::Empty));
if let Some(v) = value {
cell.value = v;
}
if let Some(r1c1) = formula {
let a1 = r1c1_formula_to_a1(&r1c1, at);
let f = Formula::new(&a1);
if f.expr.is_some() {
cell.formula = Some(Arc::new(f));
formulas.insert(at, r1c1);
} else {
skipped += 1;
}
}
sheet.set_cell(at, cell);
let mut skipped = file.skipped;
let mut styles = HashMap::new();
for (&(row, col), c) in &file.cells {
let value = match &c.value {
sylk::Value::Number(n) => Value::Number(*n),
sylk::Value::Text(s) => Value::Text(Arc::from(s.as_str())),
sylk::Value::Bool(b) => Value::Bool(*b),
sylk::Value::Error(s) => CellError::parse(s).map_or_else(|| Value::Text(Arc::from(s.as_str())), Value::Error),
_ => Value::Empty,
};
let mut cell = Cell::value(value);
if let Some(a1) = &c.formula {
let f = Formula::new(a1);
if f.expr.is_some() {
cell.formula = Some(Arc::new(f));
} else {
skipped += 1;
}
"F" => {
let mut format = None;
for f in rest {
let (tag, arg) = f.split_at(f.chars().next().map_or(0, char::len_utf8));
match tag {
"Y" => row = position(arg, MAX_ROWS).unwrap_or(row),
"X" => col = position(arg, MAX_COLS).unwrap_or(col),
"P" => format = arg.parse::<usize>().ok(),
"W" => {
let mut parts = arg.split_whitespace();
if let (Some(first), Some(last), Some(width)) = (parts.next(), parts.next(), parts.next())
&& let (Some(first), Some(last), Ok(width)) =
(position(first, MAX_COLS), position(last, MAX_COLS), width.parse::<f64>())
&& first <= last
&& (0.0..=255.0).contains(&width)
// A file of width records can't make the import loop over every column
// again and again.
&& widths.len() < MAX_WIDTH_RECORDS
{
widths.push((first, last, width));
}
}
_ => {}
}
}
if let Some(format) = format {
cell_formats.push((CellRef::new(row, col), format));
}
}
// Number formats only: `P;F…` and `P;E…` are fonts.
"P" => {
if let Some(code) = rest.first().and_then(|f| f.strip_prefix('P')) {
formats.push(code.to_string());
}
}
"E" => break,
_ => {}
}
if let Some(code) = &c.format {
cell.style = *styles
.entry(code.as_str())
.or_insert_with(|| wb.styles.intern(Style { num_fmt: NumFmt(Arc::from(code.as_str())), ..Style::default() }));
}
sheet.set_cell(CellRef::new(row, col), cell);
}
for (first, last, width) in widths {
let size = crate::cmd::format::chars_to_points(width) as f32;
for c in first..=last {
for w in &file.column_widths {
let size = crate::cmd::format::chars_to_points(w.width) as f32;
for c in w.first..=w.last {
sheet.cols.entry(c).or_default().size = Some(size);
}
}
let mut wb = Workbook::new();
let mut styles = HashMap::new();
for (at, index) in cell_formats {
let Some(code) = formats.get(index).filter(|code| !code.is_empty() && *code != "General") else { continue };
let style = *styles.entry(index).or_insert_with(|| wb.styles.intern(Style { num_fmt: NumFmt(Arc::from(code.as_str())), ..Style::default() }));
let mut cell = sheet.cell(at).cloned().unwrap_or_else(|| Cell::value(Value::Empty));
cell.style = style;
sheet.set_cell(at, cell);
}
if let Some(first) = wb.sheet_mut(0) {
sheet.name = std::mem::take(&mut first.name);
*first = sheet;
@@ -161,168 +62,3 @@ pub fn read(name: &str, bytes: &[u8]) -> Result<(Workbook, Vec<String>)> {
}
Ok((wb, warnings))
}
/// A record's fields: `;` separates them and `;;` is a literal semicolon.
fn split_fields(line: &str) -> Vec<String> {
let mut fields = vec![String::new()];
let mut chars = line.trim_end_matches('\r').chars().peekable();
while let Some(c) = chars.next() {
if c == ';' {
if chars.peek() == Some(&';') {
chars.next();
if let Some(f) = fields.last_mut() {
f.push(';');
}
} else {
fields.push(String::new());
}
} else if let Some(f) = fields.last_mut() {
f.push(c);
}
}
fields
}
/// A 1-based row or column number → 0-based, within `max`.
fn position(arg: &str, max: u32) -> Option<u32> {
arg.trim().parse::<u32>().ok().filter(|n| (1..=max).contains(n)).map(|n| n - 1)
}
/// A `K` value: `"text"` (everything between the first and the last quote, as written), a
/// boolean, an error or a number.
fn parse_value(arg: &str) -> Value {
if let Some(text) = arg.strip_prefix('"') {
let text = text.strip_suffix('"').unwrap_or(text);
return Value::Text(Arc::from(text));
}
if arg.eq_ignore_ascii_case("TRUE") || arg.eq_ignore_ascii_case("FALSE") {
return Value::Bool(arg.eq_ignore_ascii_case("TRUE"));
}
if let Some(e) = CellError::parse(arg) {
return Value::Error(e);
}
match arg.trim().parse::<f64>() {
Ok(n) if n.is_finite() => Value::Number(n),
_ => Value::Text(Arc::from(arg)),
}
}
/// One part of an R1C1 reference: `R`/`C` followed by an absolute number, a bracketed offset or
/// nothing (the cell's own row or column).
#[derive(Clone, Copy, Debug, PartialEq)]
enum Part {
Absolute(u32),
Relative(i64),
}
impl Part {
/// The 0-based index and whether it is absolute, or `None` when it falls off the sheet.
fn resolve(self, base: u32, max: u32) -> Option<(u32, bool)> {
match self {
Part::Absolute(n) => (1..=max).contains(&n).then(|| (n - 1, true)),
Part::Relative(k) => i64::from(base).checked_add(k).and_then(|v| u32::try_from(v).ok()).filter(|v| *v < max).map(|v| (v, false)),
}
}
}
/// Parses the number after `R` or `C` at `i`: digits, `[offset]` or nothing. Returns the part and
/// the index after it, or `None` when the text isn't a reference part.
fn parse_part(chars: &[char], i: usize) -> Option<(Part, usize)> {
match chars.get(i) {
Some('[') => {
let close = chars[i..].iter().position(|&c| c == ']')? + i;
let inner: String = chars[i + 1..close].iter().collect();
Some((Part::Relative(inner.parse().ok()?), close + 1))
}
Some(c) if c.is_ascii_digit() => {
let end = chars[i..].iter().position(|c| !c.is_ascii_digit()).map_or(chars.len(), |p| p + i);
let n: String = chars[i..end].iter().collect();
Some((Part::Absolute(n.parse().unwrap_or(u32::MAX)), end))
}
_ => Some((Part::Relative(0), i)),
}
}
fn is_word_char(c: char) -> bool {
c.is_alphanumeric() || matches!(c, '_' | '.' | '\\' | '?' | '$')
}
/// Rewrites the R1C1 references in a SYLK formula as A1 references for the cell at `at`:
/// `RC[-1]` → `B2`, `R1C1` → `$A$1`, `R[-2]C:R[-1]C` → `D2:D3`, a whole column `C4` → `$D:$D` and
/// a whole row `R3` → `$3:$3`. Text in quotes and quoted sheet names are left alone; a reference
/// off the sheet becomes `#REF!`.
pub(crate) fn r1c1_formula_to_a1(formula: &str, at: CellRef) -> String {
let chars: Vec<char> = formula.chars().collect();
let mut out = String::with_capacity(formula.len());
let mut i = 0;
while i < chars.len() {
let c = chars[i];
if c == '"' || c == '\'' {
// Copy the quoted run as it is; a doubled quote stays inside it.
let mut j = i + 1;
while j < chars.len() && !(chars[j] == c && chars.get(j + 1) != Some(&c)) {
j += if chars[j] == c { 2 } else { 1 };
}
let stop = (j + 1).min(chars.len());
out.extend(&chars[i..stop]);
i = stop;
continue;
}
let starts_word = i == 0 || !is_word_char(chars[i - 1]);
if starts_word
&& matches!(c, 'R' | 'r' | 'C' | 'c')
&& let Some((text, next)) = reference_at(&chars, i, at)
{
out.push_str(&text);
i = next;
continue;
}
// Copy the rest of a word as it is, so `ROUND` or `Sheet2` isn't read as a reference.
if is_word_char(c) {
let end = (i..chars.len()).find(|&j| !is_word_char(chars[j])).unwrap_or(chars.len());
out.extend(&chars[i..end]);
i = end;
} else {
out.push(c);
i += 1;
}
}
out
}
/// The A1 text of the R1C1 reference starting at `i`, and the index after it, or `None` when the
/// word there isn't a reference (a function or a name).
fn reference_at(chars: &[char], i: usize, at: CellRef) -> Option<(String, usize)> {
let mut j = i;
let mut row = None;
let mut col = None;
if matches!(chars.get(j), Some('R' | 'r')) {
let (part, next) = parse_part(chars, j + 1)?;
row = Some(part);
j = next;
}
if matches!(chars.get(j), Some('C' | 'c')) {
let (part, next) = parse_part(chars, j + 1)?;
col = Some(part);
j = next;
}
if row.is_none() && col.is_none() {
return None;
}
// A reference ends the word: `ROUND(`, `RATE` or `R2D2` are names.
if chars.get(j).is_some_and(|&c| is_word_char(c) || c == '(' || c == '[' || c == '!') {
return None;
}
let row_text = |p: Part| p.resolve(at.row, MAX_ROWS).map(|(r, abs)| format!("{}{}", if abs { "$" } else { "" }, r + 1));
let col_text = |p: Part| p.resolve(at.col, MAX_COLS).map(|(c, abs)| format!("{}{}", if abs { "$" } else { "" }, col_to_letters(c)));
// One side of a range (`R2:R4`, `C1:C3`) is written alone; a lone row or column is the whole of it.
let prev = chars[..i].iter().rev().find(|c| !c.is_whitespace());
let in_range = prev == Some(&':') || chars[j..].iter().find(|c| !c.is_whitespace()) == Some(&':');
let text = match (row, col) {
(Some(r), Some(c)) => row_text(r).zip(col_text(c)).map(|(r, c)| format!("{c}{r}")),
(Some(r), None) => row_text(r).map(|r| if in_range { r } else { format!("{r}:{r}") }),
(None, Some(c)) => col_text(c).map(|c| if in_range { c } else { format!("{c}:{c}") }),
(None, None) => None,
};
Some((text.unwrap_or_else(|| "#REF!".into()), j))
}
+1 -1
View File
@@ -763,7 +763,7 @@ fn sylk_r1c1_formulas_become_a1() {
("R[-5]C", "#REF!"),
("\"RC\"&R1C1", "\"RC\"&$A$1"),
] {
assert_eq!(sylk::r1c1_formula_to_a1(r1c1, b2), a1, "{r1c1}");
assert_eq!(craft_csv::sylk::r1c1_formula_to_a1(r1c1, (b2.row, b2.col)), a1, "{r1c1}");
}
}
+18 -1
View File
@@ -305,6 +305,19 @@ fn trim_to_budget(history: &mut Vec<HistoryEntry>, byte_budget: usize) {
}
}
/// A writer that only counts the bytes written to it.
struct ByteCounter(usize);
impl std::io::Write for ByteCounter {
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
self.0 = self.0.saturating_add(buf.len());
Ok(buf.len())
}
fn flush(&mut self) -> std::io::Result<()> {
Ok(())
}
}
impl DocState {
/// A document for `wb`, which came from where `trust` says: [`Trust::FromFile`] for anything
/// read from bytes (a file, a download, a recovery copy), [`Trust::Local`] only for a workbook
@@ -638,7 +651,11 @@ impl Session {
/// Journals a command, then trims the oldest entries past either cap. A recording in progress
/// keeps pointing at the same entry (the steps the trim dropped are lost).
fn push_journal(&mut self, id: &str, params: Json) {
let size = |e: &(String, Json)| serde_json::to_vec(e).map_or(0, |v| v.len());
// Counted as it is written, without building the text: a paste's params can be MBs.
let size = |e: &(String, Json)| {
let mut counter = ByteCounter(0);
serde_json::to_writer(&mut counter, e).map_or(0, |()| counter.0)
};
// `journal` is public: if it was changed from outside, measure it again.
if self.journal_sizes.len() != self.journal.len() {
self.journal_sizes = self.journal.iter().map(size).collect();
+141 -17
View File
@@ -43,6 +43,10 @@ impl Selection {
pub fn is_single_cell(&self) -> bool {
self.ranges.len() == 1 && self.current().is_single()
}
/// Whether the selection is exactly one merged cell, which Enter and Tab treat as one cell.
pub fn is_one_merged_cell(&self, sheet: &Sheet) -> bool {
self.ranges.len() == 1 && sheet.merge_at(self.active).is_some_and(|m| m == self.current())
}
/// The corner of the current area opposite the anchor: the end that Shift+arrows move.
pub fn moving_end(&self) -> CellRef {
let r = self.current();
@@ -67,23 +71,28 @@ impl Selection {
/// they are: clicking column B's header selects just B even when B1:C1 is merged, as in Excel.
pub fn expand_merges(&mut self, sheet: &Sheet) {
for r in self.ranges.iter_mut() {
if r.is_full_cols() || r.is_full_rows() {
continue;
}
let mut cur = *r;
loop {
let mut grown = cur;
for m in &sheet.merges {
if m.intersects(&grown) {
grown = grown.union(m);
}
}
if grown == cur {
break;
}
cur = grown;
}
*r = cur;
*r = with_merges(sheet, *r);
}
}
/// Shift+arrows: moves the edge of the current area that the anchor doesn't hold by `dr`
/// rows and `dc` columns, keeping merged cells whole, as Excel does. The anchor is the whole
/// merged cell it is in, so from a selected merged cell Shift+Left and Shift+Up widen the
/// selection past its left and top edges. A step that would shrink the area onto the inside
/// of a merged cell (where keeping it whole would undo the step) goes on to the next line
/// that does shrink it, and when the area can't shrink any more it grows in that direction.
pub fn extend_by(&mut self, sheet: &Sheet, dr: i64, dc: i64) {
let anchor = sheet.merge_at(self.anchor).unwrap_or(RangeRef::cell(self.anchor));
let mut r = self.current();
if dc != 0 {
r = step_edge(sheet, r, anchor, dc, true);
}
if dr != 0 {
r = step_edge(sheet, r, anchor, dr, false);
}
if let Some(last) = self.ranges.last_mut() {
*last = r;
} else {
self.ranges.push(r);
}
}
/// Moves the active cell within the selection (Enter/Tab inside a multi-cell selection).
@@ -128,6 +137,91 @@ impl Selection {
}
}
/// `r` grown until every merged cell it touches is inside it. Whole columns and whole rows stay
/// as they are.
fn with_merges(sheet: &Sheet, r: RangeRef) -> RangeRef {
if r.is_full_cols() || r.is_full_rows() {
return r;
}
let mut cur = r;
loop {
let mut grown = cur;
for m in &sheet.merges {
if m.intersects(&grown) {
grown = grown.union(m);
}
}
if grown == cur {
return cur;
}
cur = grown;
}
}
/// One Shift+arrow step of [`Selection::extend_by`] along one axis (`cols`: columns, else rows).
fn step_edge(sheet: &Sheet, r: RangeRef, anchor: RangeRef, d: i64, cols: bool) -> RangeRef {
let max = i64::from(if cols { MAX_COLS } else { MAX_ROWS }) - 1;
let hidden = |i: i64| if cols { sheet.is_col_hidden(i as u32) } else { sheet.is_row_hidden(i as u32) };
// `from` moved by `by`, then on past hidden lines in the same direction (not off the sheet).
let land = |from: i64, by: i64| {
let mut t = from.saturating_add(by).clamp(0, max);
// Bounded like the arrow keys' walk over hidden lines.
for _ in 0..100_000 {
if !hidden(t) {
break;
}
let n = (t + by.signum()).clamp(0, max);
if n == t {
break;
}
t = n;
}
t
};
let span = |x: &RangeRef| {
if cols { (i64::from(x.start.col), i64::from(x.end.col)) } else { (i64::from(x.start.row), i64::from(x.end.row)) }
};
let with = |lo: i64, hi: i64| {
let (lo, hi) = (lo.clamp(0, max) as u32, hi.clamp(0, max) as u32);
let (s, e) = if cols {
(CellRef::new(r.start.row, lo), CellRef::new(r.end.row, hi))
} else {
(CellRef::new(lo, r.start.col), CellRef::new(hi, r.end.col))
};
with_merges(sheet, RangeRef::new(s, e))
};
let (lo, hi) = span(&r);
let (alo, ahi) = span(&anchor);
// The anchor holds the low edge unless the area reaches past it on that side.
let moving_hi = lo >= alo;
let shrinking = if moving_hi { d < 0 && hi > ahi } else { d > 0 && lo < alo };
if shrinking {
let mut t = if moving_hi { hi } else { lo };
let mut by = d;
loop {
t = if moving_hi { land(t, by).max(ahi) } else { land(t, by).min(alo) };
let cand = if moving_hi { with(lo, t) } else { with(t, hi) };
if cand != r {
return cand;
}
if t == if moving_hi { ahi } else { alo } {
break;
}
// The step was undone because a merged cell crosses the new edge, and so is every
// edge up to where that merged cell starts (ends, for the low edge): go on from there
// so a tall or wide merged cell costs one step, not one per line it covers.
let across = sheet.merges.iter().filter(|m| m.intersects(&r)).filter_map(|m| {
let (ms, me) = span(m);
if moving_hi { (ms <= t && me > t).then_some(ms) } else { (ms < t && me >= t).then_some(me) }
});
t = if moving_hi { across.min().map_or(t, |ms| ms.min(t)) } else { across.max().map_or(t, |me| me.max(t)) };
by = d.signum();
}
}
// Grow the area on the side the arrow points to.
if d > 0 { with(lo, land(hi, d)) } else { with(land(lo, d), hi) }
}
impl Selection {
/// Ctrl+. (Next Corner of Selection): the active cell moves clockwise to the next corner of
/// the current area (top-left, top-right, bottom-right, bottom-left) and becomes the anchor,
@@ -287,6 +381,36 @@ mod tests {
assert_eq!(current_region(&s, CellRef::new(5, 0)).a1(), "A1:B20000");
}
#[test]
fn shift_arrows_from_a_merged_cell() {
let mut s = Sheet::new("S");
s.merges.push(RangeRef::parse("B2:C3").unwrap());
let at = |s: &Sheet, a1: &str| {
let mut sel = Selection::at(CellRef::parse(a1).unwrap());
sel.expand_merges(s);
sel
};
for ((dr, dc), want) in [((0, -1), "A2:C3"), ((-1, 0), "B1:C3"), ((0, 1), "B2:D3"), ((1, 0), "B2:C4")] {
let mut sel = at(&s, "B2");
sel.extend_by(&s, dr, dc);
assert_eq!(sel.a1(), want);
}
// Shrinking onto the inside of the merged cell goes on past it.
let mut sel = at(&s, "A1");
sel.extend_by(&s, 1, 1);
assert_eq!(sel.a1(), "A1:C3");
sel.extend_by(&s, 0, -1);
assert_eq!(sel.a1(), "A1:A3");
// A hidden column is stepped over, and the sheet edge stops the area.
s.cols.entry(3).or_default().hidden = true;
let mut sel = at(&s, "B2");
sel.extend_by(&s, 0, 1);
assert_eq!(sel.a1(), "B2:E3");
let mut sel = at(&s, "A1");
sel.extend_by(&s, -1, -1);
assert_eq!(sel.a1(), "A1");
}
#[test]
fn cycling() {
let mut sel = Selection::range(RangeRef::parse("A1:B2").unwrap());
+122
View File
@@ -532,6 +532,118 @@ fn shape_text_edit_survives_xlsx_roundtrip() {
assert_eq!(shape.text, text);
}
#[test]
fn word_art_insert_respects_sheet_protection() {
let mut s = s();
s.execute("review.protectSheet", json!({})).unwrap();
let before = s.doc().unwrap().wb.clone();
let undo_len = s.doc().unwrap().undo.len();
assert!(s.execute("insert.wordArt", json!({})).is_err());
assert!(std::sync::Arc::ptr_eq(&s.doc().unwrap().wb, &before));
assert_eq!(s.doc().unwrap().undo.len(), undo_len);
s.execute("review.unprotectSheet", json!({})).unwrap();
s.execute("insert.wordArt", json!({})).unwrap();
assert_eq!(s.doc().unwrap().wb.active().unwrap().shapes.len(), 1);
}
#[test]
fn word_art_inserts_with_defaults_and_undoes() {
use gridcraft_model::ShapeKind;
let mut s = s();
// Empty params: default style and text, no dialog.
let r = s.execute("insert.wordArt", json!({})).unwrap();
assert!(s.take_ui_requests().is_empty());
let shape = s.doc().unwrap().wb.active().unwrap().shapes[0].clone();
assert_eq!(shape.id, r["shape"].as_u64().unwrap() as u32);
assert_eq!(shape.kind, ShapeKind::WordArt);
assert_eq!(shape.text, crate::cmd::wordart::DEFAULT_TEXT);
assert!(shape.anchor.width > 8.0 && shape.format.text_color.is_some());
// Named style plus overrides; restyle and edit the text through wordArt.set.
s.execute("insert.wordArt", json!({"style": "ember", "text": "Q3", "fontSize": 60, "shadow": false, "outline": "#112233", "outlineWidth": 3}))
.unwrap();
let ember = s.doc().unwrap().wb.active().unwrap().shapes[1].clone();
assert_eq!(ember.format.font_size, Some(60.0));
assert_eq!((ember.format.text_shadow, ember.format.text_outline.map(|o| o.1)), (None, Some(3.0)));
s.execute("wordArt.set", json!({"id": ember.id, "style": 3, "text": "Q4"})).unwrap();
let styled = s.doc().unwrap().wb.active().unwrap().shapes[1].clone();
assert_eq!((styled.text.as_str(), styled.format.italic), ("Q4", true));
s.execute("shape.setText", json!({"id": ember.id, "text": "Q5"})).unwrap();
s.execute("edit.undo", json!({})).unwrap();
s.execute("edit.undo", json!({})).unwrap();
assert_eq!(s.doc().unwrap().wb.active().unwrap().shapes[1], ember);
s.execute("edit.redo", json!({})).unwrap();
assert_eq!(s.doc().unwrap().wb.active().unwrap().shapes[1].text, "Q4");
// Moves and deletes like any floating shape.
s.execute("object.move", json!({"kind": "shape", "id": ember.id, "at": "D4"})).unwrap();
s.execute("object.delete", json!({"kind": "shape", "id": ember.id})).unwrap();
assert_eq!(s.doc().unwrap().wb.active().unwrap().shapes.len(), 1);
}
#[test]
fn word_art_rejects_hostile_params() {
let mut s = s();
let id = s.execute("insert.wordArt", json!({})).unwrap()["shape"].clone();
let rectangle = s.execute("insert.shape", json!({"kind": "rectangle"})).unwrap()["shape"].clone();
let before = s.doc().unwrap().wb.clone();
for (cmd, params) in [
("insert.wordArt", json!({"style": "No Such Style"})),
("insert.wordArt", json!({"style": 99})),
("insert.wordArt", json!({"style": -1})),
("insert.wordArt", json!({"text": 5})),
("insert.wordArt", json!({"text": ""})),
("insert.wordArt", json!({"text": "x".repeat(crate::cmd::wordart::MAX_CHARS + 1)})),
("insert.wordArt", json!({"fill": "not a colour"})),
("wordArt.set", json!({})),
("wordArt.set", json!({"id": 999})),
("wordArt.set", json!({"id": rectangle})),
("shape.setText", json!({"id": id, "text": ""})),
] {
assert!(matches!(s.execute(cmd, params), Err(crate::EngineError::BadParams { .. })), "{cmd}");
assert!(std::sync::Arc::ptr_eq(&s.doc().unwrap().wb, &before));
}
// Absurd sizes are clamped, not trusted.
s.execute("insert.wordArt", json!({"fontSize": 1e12, "width": 1e300, "height": -5})).unwrap();
let big = &s.doc().unwrap().wb.active().unwrap().shapes[2];
assert!(big.anchor.width <= 4000.0 && big.anchor.height >= 8.0 && big.format.font_size == Some(400.0));
}
#[test]
fn word_art_survives_xlsx_roundtrip() {
use gridcraft_model::ShapeKind;
let mut s = s();
s.execute("insert.wordArt", json!({"style": "Lagoon", "text": "Sales <2026>\nOutlook & more"})).unwrap();
let before = s.doc().unwrap().wb.active().unwrap().shapes[0].clone();
let saved = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap();
s.execute("file.open", json!({"name": "wordart.xlsx", "base64": saved["base64"]})).unwrap();
let after = &s.doc().unwrap().wb.active().unwrap().shapes[0];
assert_eq!(after.kind, ShapeKind::WordArt);
assert_eq!(after.text, before.text);
assert_eq!(after.format.text_color, before.format.text_color);
assert_eq!(after.format.text_outline, before.format.text_outline);
assert_eq!(after.format.text_shadow, before.format.text_shadow);
assert_eq!((after.format.bold, after.format.italic, after.format.font_size), (true, true, before.format.font_size));
assert!(after.format.no_line);
}
/// White text with a green outline (Mint Outline) prints its outline, so it is not invisible on a
/// white page.
#[test]
fn word_art_outline_prints() {
let mut s = s();
s.execute("insert.wordArt", json!({"style": "Mint Outline", "text": "VISIBLE", "at": "A1"})).unwrap();
let pdf = s.execute("file.exportPdf", json!({"range": "A1:G10"})).unwrap();
let data = crate::io::base64_decode(pdf["base64"].as_str().unwrap()).unwrap();
let text = String::from_utf8_lossy(&data);
let outline = text.matches("0.18 0.608 0.416 rg").count();
assert!(outline >= 1, "the outline colour (2E9B6A) is drawn");
assert_eq!(text.matches("(VISIBLE) Tj").count(), 9, "eight outline copies under the white text");
assert!(text.contains("1 1 1 rg"), "the white fill is drawn on top");
// The last copy drawn is the filled text, in white; every outline copy comes before it.
let before_fill = &text[..text.rfind("(VISIBLE) Tj").unwrap()];
assert!(before_fill.rfind("1 1 1 rg") > before_fill.rfind("0.18 0.608 0.416 rg"), "the white fill is drawn last");
assert_eq!(before_fill.matches("(VISIBLE) Tj").count(), 8);
}
#[test]
fn no_border_clears_only_shared_edges_and_undo_restores_them() {
use gridcraft_core::RangeRef;
@@ -6266,6 +6378,16 @@ mod objects;
mod python_cells;
mod python_commands;
mod python_files;
#[test]
fn text_to_columns_keeps_a_mid_field_quote() {
let mut s = s();
s.execute("cell.set", json!({"cell": "A1", "input": "'5\" pipe,abc"})).unwrap();
s.execute("data.textToColumns", json!({"range": "A1", "delimiters": [","]})).unwrap();
assert_eq!(v(&s, "A1"), Value::text("5\" pipe"));
assert_eq!(v(&s, "B1"), Value::text("abc"));
}
mod seams;
#[test]
+2 -2
View File
@@ -175,8 +175,8 @@ fn text_to_columns_keeps_a_py_piece_as_text() {
s.execute("cell.set", json!({"cell": "A1", "input": "'=PY(\"x\");7"})).unwrap();
assert_eq!(super::v(&s, "A1"), Value::text("=PY(\"x\");7"));
s.execute("data.textToColumns", json!({"range": "A1", "delimiters": [";"]})).unwrap();
// The default text qualifier takes the quotes away, as in Excel.
assert_eq!(super::v(&s, "A1"), Value::text("=PY(x)"), "the piece is kept, not cleared");
// A quote in the middle of a field is kept, as in Excel.
assert_eq!(super::v(&s, "A1"), Value::text("=PY(\"x\")"), "the piece is kept, not cleared");
assert_eq!(code(&s, 0, "A1"), None, "and it is not a Python cell");
assert_eq!(super::v(&s, "B1"), Value::Number(7.0));
}
+1
View File
@@ -18,6 +18,7 @@ mod hidden_sheet_navigation;
mod ink_roundtrip;
mod line_count_overflow;
mod long_operator_chains;
mod merged_selection_keys;
mod offset_overflow;
mod percent_entry;
mod pivot_unknown_func;
@@ -0,0 +1,162 @@
//! Shift+arrows and Enter/Tab on merged cells, as Excel does them.
use gridcraft_engine::Session;
use serde_json::json;
/// A new workbook with B2:C3 merged and nothing selected but A1.
fn merged_book() -> Session {
let mut s = Session::new();
s.new_workbook();
s.execute("selection.set", json!({"range": "B2:C3"})).unwrap();
s.execute("home.mergeCells", json!({})).unwrap();
s.execute("selection.set", json!({"cell": "A1"})).unwrap();
s
}
fn shift(s: &mut Session, dr: i64, dc: i64) -> String {
s.execute("selection.move", json!({"dr": dr, "dc": dc, "extend": true})).unwrap();
s.doc().unwrap().selection.a1()
}
fn enter(s: &mut Session, forward: bool, by_row: bool) -> (String, String) {
s.execute("selection.next", json!({"forward": forward, "byRow": by_row})).unwrap();
let sel = &s.doc().unwrap().selection;
(sel.active.a1(), sel.a1())
}
#[test]
fn shift_arrows_from_a_merged_cell_widen_past_each_edge() {
let mut s = merged_book();
for ((dr, dc), want) in [((0, -1), "A2:C3"), ((-1, 0), "B1:C3"), ((0, 1), "B2:D3"), ((1, 0), "B2:C4")] {
s.execute("selection.set", json!({"cell": "B2"})).unwrap();
assert_eq!(s.doc().unwrap().selection.a1(), "B2:C3");
assert_eq!(shift(&mut s, dr, dc), want, "Shift+({dr},{dc}) from the merged cell B2:C3");
assert_eq!(s.doc().unwrap().selection.active.a1(), "B2", "the active cell stays");
}
}
#[test]
fn shift_arrows_keep_going_and_come_back_to_the_merged_cell() {
let mut s = merged_book();
s.execute("selection.set", json!({"cell": "B2"})).unwrap();
assert_eq!(shift(&mut s, 1, 0), "B2:C4");
assert_eq!(shift(&mut s, 1, 0), "B2:C5");
assert_eq!(shift(&mut s, 1, 0), "B2:C6");
assert_eq!(shift(&mut s, -1, 0), "B2:C5");
s.execute("selection.set", json!({"cell": "B2"})).unwrap();
assert_eq!(shift(&mut s, 0, -1), "A2:C3");
assert_eq!(shift(&mut s, 0, 1), "B2:C3", "back to just the merged cell");
assert_eq!(shift(&mut s, 0, 1), "B2:D3");
assert_eq!(shift(&mut s, -1, 0), "B1:D3");
assert_eq!(shift(&mut s, 1, 0), "B2:D3");
}
#[test]
fn shift_arrows_shrink_past_a_merged_cell_they_pulled_in() {
let mut s = merged_book();
assert_eq!(shift(&mut s, 0, 1), "A1:B1");
assert_eq!(shift(&mut s, 1, 0), "A1:C3", "B2 pulls in the whole merged cell");
assert_eq!(shift(&mut s, 0, -1), "A1:A3", "one step back leaves the merged cell out");
assert_eq!(shift(&mut s, 0, 1), "A1:C3");
assert_eq!(shift(&mut s, -1, 0), "A1:C1");
assert_eq!(shift(&mut s, 1, 0), "A1:C3");
assert_eq!(shift(&mut s, 1, 0), "A1:C4");
}
#[test]
fn enter_and_tab_leave_a_selected_merged_cell() {
let mut s = merged_book();
for (forward, by_row, want) in [(true, false, "B4"), (false, false, "B1"), (true, true, "D2"), (false, true, "A2")] {
s.execute("selection.set", json!({"cell": "B2"})).unwrap();
assert_eq!(enter(&mut s, forward, by_row), (want.to_string(), want.to_string()), "forward {forward} byRow {by_row}");
}
}
#[test]
fn enter_and_tab_out_of_a_merged_cell_step_over_hidden_lines() {
let mut s = merged_book();
s.execute("selection.set", json!({"range": "A4"})).unwrap();
s.execute("home.hideRows", json!({})).unwrap();
s.execute("selection.set", json!({"range": "D1"})).unwrap();
s.execute("home.hideColumns", json!({})).unwrap();
s.execute("selection.set", json!({"cell": "B2"})).unwrap();
assert_eq!(enter(&mut s, true, false).0, "B5", "row 4 is hidden");
s.execute("selection.set", json!({"cell": "B2"})).unwrap();
assert_eq!(enter(&mut s, true, true).0, "E2", "column D is hidden");
s.execute("selection.set", json!({"cell": "B2"})).unwrap();
s.execute("selection.move", json!({"dr": 1})).unwrap();
assert_eq!(s.doc().unwrap().selection.a1(), "B5", "the Down arrow too");
}
#[test]
fn enter_inside_a_selection_visits_a_merged_cell_once() {
let mut s = merged_book();
s.execute("selection.set", json!({"range": "A1:C3", "active": "A1"})).unwrap();
let seen: Vec<String> = (0..6).map(|_| enter(&mut s, true, false).0).collect();
assert_eq!(seen, ["A2", "A3", "B1", "B2", "C1", "A1"]);
assert_eq!(s.doc().unwrap().selection.a1(), "A1:C3", "the selection stays");
let back: Vec<String> = (0..2).map(|_| enter(&mut s, false, false).0).collect();
assert_eq!(back, ["C1", "B2"]);
}
#[test]
fn shift_arrow_that_cannot_shrink_the_area_grows_it() {
// A2:B3 merged, anchor B1: Shift+Down pulls the area out to column A, past the anchor.
// Shift+Right can't shrink it from the left (the merged cell would come back), so it grows.
let mut s = Session::new();
s.new_workbook();
s.execute("selection.set", json!({"range": "A2:B3"})).unwrap();
s.execute("home.mergeCells", json!({})).unwrap();
s.execute("selection.set", json!({"cell": "B1"})).unwrap();
assert_eq!(shift(&mut s, 1, 0), "A1:B3");
assert_eq!(shift(&mut s, 0, 1), "A1:C3");
assert_eq!(s.doc().unwrap().selection.active.a1(), "B1");
}
#[test]
fn shift_left_shrinks_past_a_chain_of_merged_cells_in_one_step() {
// B2:C3 and D3:E4 are both inside A1:E4. Shift+Left from E can't stop in column D (D3:E4
// would come back) and stops in C, where B2:C3 is whole.
let mut s = merged_book();
s.execute("selection.set", json!({"range": "D3:E4"})).unwrap();
s.execute("home.mergeCells", json!({})).unwrap();
s.execute("selection.set", json!({"cell": "A1"})).unwrap();
assert_eq!(shift(&mut s, 3, 4), "A1:E4");
assert_eq!(shift(&mut s, 0, -1), "A1:C4");
assert_eq!(shift(&mut s, -1, 0), "A1:C3", "row 4 holds nothing merged once column D is out");
assert_eq!(shift(&mut s, 0, -1), "A1:A3", "C can't stop inside B2:C3");
}
#[test]
fn shift_up_over_a_tall_merged_cell_takes_one_step() {
let mut s = Session::new();
s.new_workbook();
s.execute("selection.set", json!({"range": "B2:C500000"})).unwrap();
s.execute("home.mergeCells", json!({})).unwrap();
s.execute("selection.set", json!({"cell": "A1"})).unwrap();
assert_eq!(shift(&mut s, 0, 1), "A1:B1");
assert_eq!(shift(&mut s, 1, 0), "A1:C500000");
assert_eq!(shift(&mut s, -1, 0), "A1:C1", "row 1 is the last row above the merged cell");
assert_eq!(shift(&mut s, 1, 0), "A1:C500000");
assert_eq!(shift(&mut s, 0, -1), "A1:A500000");
}
#[test]
fn shift_arrows_change_only_the_last_area() {
let mut s = merged_book();
s.execute("selection.set", json!({"range": "E5:F6,B2"})).unwrap();
assert_eq!(s.doc().unwrap().selection.a1(), "E5:F6,B2:C3");
assert_eq!(shift(&mut s, 0, -1), "E5:F6,A2:C3");
}
#[test]
fn enter_skips_a_merged_cell_whose_start_is_outside_the_selection() {
// A whole column isn't widened to the merged cells it touches. B1 is covered by A1:B1, whose
// top-left cell is outside column B, so Enter skips it.
let mut s = Session::new();
s.new_workbook();
s.execute("selection.set", json!({"range": "A1:B1"})).unwrap();
s.execute("home.mergeCells", json!({})).unwrap();
s.execute("selection.set", json!({"range": "B:B", "active": "B2"})).unwrap();
assert_eq!(enter(&mut s, false, false).0, "B1048576", "B1 is skipped, going up wraps to the bottom");
}
+22 -1
View File
@@ -215,7 +215,9 @@ impl Expr {
/// Visits every node depth-first, parents before children, left to right.
pub fn walk<'a>(&'a self, f: &mut dyn FnMut(&'a Expr)) {
// An explicit stack: a long operator chain must not recurse once per operand.
let mut pending = vec![self];
// Room for a typical formula up front, so walking one costs one allocation, not several.
let mut pending = Vec::with_capacity(8);
pending.push(self);
while let Some(e) = pending.pop() {
f(e);
match e {
@@ -236,6 +238,19 @@ impl Expr {
}
/// Rewrites every node bottom-up, children left to right before their parent.
pub fn map(self, f: &mut dyn FnMut(Expr) -> Expr) -> Expr {
// One operator on a non-chain operand (`A1*2`), the common case, needs no spine: rewrite
// the operands in their boxes.
if let Expr::Binary(op, mut a, mut b) = self {
if !matches!(*a, Expr::Binary(..)) {
*a = std::mem::replace(&mut *a, Expr::Missing).map(f);
*b = std::mem::replace(&mut *b, Expr::Missing).map(f);
return f(Expr::Binary(op, a, b));
}
return Expr::Binary(op, a, b).map_chain(f);
}
self.map_chain(f)
}
fn map_chain(self, f: &mut dyn FnMut(Expr) -> Expr) -> Expr {
// Unwind the left spine of a long operator chain into a list, then rebuild it from the
// innermost operand outwards, in the order a recursive rewrite would visit the nodes.
let mut spine = Vec::new();
@@ -283,6 +298,12 @@ impl Expr {
impl Clone for Expr {
fn clone(&self) -> Self {
// One operator on a non-chain operand needs no spine.
if let Expr::Binary(op, a, b) = self
&& !matches!(**a, Expr::Binary(..))
{
return Expr::Binary(*op, a.clone(), b.clone());
}
// Copy the left spine of an operator chain in a loop (see [`Expr`]).
let mut spine = Vec::new();
let mut node = self;
+2 -1
View File
@@ -71,7 +71,8 @@ pub(super) fn accrintm(a: &[Arg], c: &mut dyn Ctx) -> R<Value> {
let (issue, settle, rate) = (date(a, 0)?, date(a, 1)?, num(a, 2)?);
let par = if has(a, 3) { num(a, 3)? } else { 1000.0 };
let basis = basis_arg(a, 4)?;
if issue >= settle || rate <= 0.0 || par <= 0.0 {
// Excel accepts equal (whole-day) dates and accrues nothing; only issue after settlement is #NUM!.
if issue > settle || rate <= 0.0 || par <= 0.0 {
return Err(CellError::Num);
}
num_val(par * rate * year_frac(c.date_system(), issue, settle, basis)?)
+25
View File
@@ -317,3 +317,28 @@ fn securities_read_dates_in_the_1904_system() {
let odd = vec![d4(2008, 11, 11), d4(2021, 3, 1), d4(2008, 10, 15), d4(2009, 3, 1), n(0.0785), n(0.0625), n(100.0), n(2.0), n(1.0)];
close_tol(ev4("ODDFPRICE", odd), 113.597717474079, 1e-9);
}
/// Excel truncates ACCRINTM's dates to whole days and accrues nothing when issue equals
/// settlement, in every basis and both date systems; only issue after settlement is #NUM! (#4096).
#[test]
fn accrintm_equal_dates_accrue_zero() {
let day = serial_from_ymd(DateSystem::D1900, 2025, 1, 15).unwrap();
for basis in 0..=4 {
close(ev("ACCRINTM", vec![n(day), n(day), n(0.05), n(1000.0), n(f64::from(basis))]), 0.0);
}
// Distinct times on the same day truncate to equal dates.
close(ev("ACCRINTM", vec![n(day + 0.1), n(day + 0.9), n(0.05), n(1000.0), n(2.0)]), 0.0);
close_tol(ev("ACCRINTM", vec![n(day), n(day + 1.0), n(0.05), n(1000.0), n(2.0)]), 0.1388888888888889, 1e-15);
is_err(ev("ACCRINTM", vec![n(day + 1.0), n(day), n(0.05), n(1000.0), n(2.0)]), CellError::Num);
// Equal dates do not excuse a zero rate or par, or an invalid basis.
is_err(ev("ACCRINTM", vec![n(day), n(day), n(0.0), n(1000.0), n(2.0)]), CellError::Num);
is_err(ev("ACCRINTM", vec![n(day), n(day), n(0.05), n(0.0), n(2.0)]), CellError::Num);
is_err(ev("ACCRINTM", vec![n(day), n(day), n(0.05), n(1000.0), n(5.0)]), CellError::Num);
let day4 = serial_from_ymd(DateSystem::D1904, 2025, 1, 15).unwrap();
for basis in 0..=4 {
let mut ctx = TestCtx { sys: DateSystem::D1904, ..TestCtx::default() };
let args = vec![n(day4), n(day4), n(0.05), n(1000.0), n(f64::from(basis))];
close(crate::call(crate::lookup("ACCRINTM").unwrap(), &args, &mut ctx), 0.0);
}
}
+1 -1
View File
@@ -520,7 +520,7 @@ fn integer_args(a: &[Arg]) -> R<Vec<u64>> {
v.iter()
.map(|x| {
let t = x.trunc();
if !(0.0..9.007_199_254_740_992e15).contains(&t) { Err(CellError::Num) } else { Ok(t as u64) }
if !(0.0..9.007_199_254_740_992e15).contains(x) { Err(CellError::Num) } else { Ok(t as u64) }
})
.collect()
}
+14
View File
@@ -229,6 +229,20 @@ fn combinatorics() {
is_err(ev("MULTINOMIAL", vec![n(-1.0), n(2.0)]), Num);
}
#[test]
fn gcd_and_lcm_reject_negative_fractions_before_truncation() {
for name in ["GCD", "LCM"] {
for x in [-0.1, -0.5, -0.999] {
is_err(ev2(name, x, 2.0), Num);
is_err(ev(name, vec![rf(row(&[2.0, x]))]), Num);
// A zero elsewhere must not hide an invalid negative argument.
is_err(ev2(name, 0.0, x), Num);
}
close(ev2(name, -0.0, 2.0), if name == "GCD" { 2.0 } else { 0.0 });
close(ev2(name, 12.9, 18.9), if name == "GCD" { 6.0 } else { 36.0 });
}
}
#[test]
fn trig() {
close(ev1("SIN", PI / 2.0), 1.0);
+145 -92
View File
@@ -55,12 +55,12 @@ fn strict_num(v: &Value) -> R<f64> {
}
}
/// Most predictor columns LINEST, LOGEST, TREND and GROWTH accept. The normal equations are a
/// k×k matrix inverted in O(k³), so an unbounded k (one `SEQUENCE(1,16384)` argument) allocated
/// Most predictor columns LINEST, LOGEST, TREND and GROWTH accept. The fit works on a k×k
/// triangle in O(k³), so an unbounded k (one `SEQUENCE(1,16384)` argument) allocated
/// gigabytes and could abort the process.
pub(super) const MAX_REGRESSORS: usize = 255;
/// Most multiply-adds (rows × k²) a regression may spend building its normal equations.
/// Most multiply-adds (rows × k²) a regression may spend on its factorisation.
const MAX_REGRESSION_WORK: usize = 1_000_000_000;
/// Rejects regressions too large to fit: more than [`MAX_REGRESSORS`] predictors is `#VALUE!`,
@@ -121,59 +121,136 @@ struct Fit {
ssresid: f64,
}
/// Inverts a symmetric positive semi-definite matrix by Gauss–Jordan on the diagonal, dropping
/// (zeroing) collinear variables. Returns the inverse and the dropped flags.
fn sym_inverse(m: &[Vec<f64>]) -> (Vec<Vec<f64>>, Vec<bool>) {
let p = m.len();
let mut a: Vec<Vec<f64>> = m.to_vec();
let mut inv: Vec<Vec<f64>> = (0..p).map(|i| (0..p).map(|j| if i == j { 1.0 } else { 0.0 }).collect()).collect();
let mut dropped = vec![false; p];
let diag: Vec<f64> = (0..p).map(|i| m.get(i).and_then(|r| r.get(i)).copied().unwrap_or(0.0).abs()).collect();
for j in 0..p {
let piv = a.get(j).and_then(|r| r.get(j)).copied().unwrap_or(0.0);
let scale = diag.get(j).copied().unwrap_or(0.0);
if piv.abs() <= 1e-12 * scale || piv.abs() < 1e-300 {
if let Some(d) = dropped.get_mut(j) {
*d = true;
}
for row in a.iter_mut() {
if let Some(v) = row.get_mut(j) {
*v = 0.0;
}
}
if let Some(r) = a.get_mut(j) {
r.iter_mut().for_each(|v| *v = 0.0);
}
if let Some(r) = inv.get_mut(j) {
r.iter_mut().for_each(|v| *v = 0.0);
}
/// Euclidean norm without intermediate overflow or underflow.
fn norm2(v: &[f64]) -> f64 {
let m = v.iter().fold(0.0f64, |m, x| m.max(x.abs()));
if m == 0.0 || !m.is_finite() {
return m;
}
m * v.iter().map(|x| (x / m) * (x / m)).sum::<f64>().sqrt()
}
/// Result of [`qr_solve`]: the least-squares solution on the kept columns, in the caller's
/// column scale, and what the standard errors need.
struct QrFit {
coef: Vec<f64>,
dropped: Vec<bool>,
/// Residual sum of squares of the fit.
ssresid: f64,
/// Diagonal of `(X'X)^-1` over the kept columns (0 for dropped ones).
diag: Vec<f64>,
/// `xm' (X'X)^-1 xm` over the kept columns.
quad: f64,
}
/// Least squares by Householder QR on unit-norm columns, instead of the normal equations, whose
/// condition number is the square of the data's. Columns are taken left to right; one whose part
/// outside the span of the kept columns is under 1e-6 of its own length (the 1e-12 relative
/// pivot of the earlier normal-equation solver, on squares) is collinear: it is dropped, with
/// coefficient and standard error 0, as Excel does.
///
/// `constant[j]` says column `j` was constant before centring. Centring it leaves rounding
/// residue, not 0 (`[0.1; 3]` centres to about -1.4e-17): it is collinear with the intercept, so
/// it is dropped instead of being scaled up to unit length. Whether a column is constant is
/// decided on its raw values, not by comparing the centred length with the offset: a predictor
/// like `[1e13, 1e13 + 1, 1e13 + 2]` is far smaller than its offset and still varies.
fn qr_solve(mut cols: Vec<Vec<f64>>, mut y: Vec<f64>, xm: &[f64], constant: &[bool]) -> QrFit {
let n = y.len();
let k = cols.len();
let mut scale = vec![1.0; k];
let mut dropped = vec![false; k];
// Householder vectors (unit norm, acting on rows `at..n`) and the columns of R.
let mut refl: Vec<(usize, Vec<f64>)> = Vec::new();
let mut r_cols: Vec<Vec<f64>> = Vec::new();
let mut kept: Vec<usize> = Vec::new();
for (j, col) in cols.iter_mut().enumerate() {
let len = norm2(col);
if constant.get(j).copied().unwrap_or(false) || !(len > 1e-300 && len.is_finite()) {
dropped[j] = true;
continue;
}
let arow: Vec<f64> = a.get(j).map(|r| r.iter().map(|v| v / piv).collect()).unwrap_or_default();
let irow: Vec<f64> = inv.get(j).map(|r| r.iter().map(|v| v / piv).collect()).unwrap_or_default();
for i in 0..p {
if i == j {
continue;
}
let f = a.get(i).and_then(|r| r.get(j)).copied().unwrap_or(0.0);
if f == 0.0 {
continue;
}
if let Some(r) = a.get_mut(i) {
r.iter_mut().zip(&arow).for_each(|(v, w)| *v -= f * w);
}
if let Some(r) = inv.get_mut(i) {
r.iter_mut().zip(&irow).for_each(|(v, w)| *v -= f * w);
}
col.iter_mut().for_each(|v| *v /= len);
scale[j] = len;
for (at, v) in &refl {
apply_reflector(*at, v, col);
}
if let Some(r) = a.get_mut(j) {
*r = arow;
let at = refl.len();
let tail = col.get(at..).unwrap_or(&[]);
let alpha = norm2(tail);
if alpha * alpha <= 1e-12 {
dropped[j] = true;
continue;
}
if let Some(r) = inv.get_mut(j) {
*r = irow;
let sign = if tail.first().is_some_and(|t| *t < 0.0) { -1.0 } else { 1.0 };
let mut v: Vec<f64> = tail.to_vec();
if let Some(h) = v.first_mut() {
*h += sign * alpha;
}
let vn = norm2(&v);
v.iter_mut().for_each(|x| *x /= vn);
let mut rc: Vec<f64> = col.get(..at).unwrap_or(&[]).to_vec();
rc.push(-sign * alpha);
r_cols.push(rc);
refl.push((at, v));
kept.push(j);
}
for (at, v) in &refl {
apply_reflector(*at, v, &mut y);
}
let r = kept.len();
// Back substitution R c = (Q'y)[..r], then undo the column scaling.
let mut c = vec![0.0; r];
for i in (0..r).rev() {
let mut t = y.get(i).copied().unwrap_or(0.0);
for jj in i + 1..r {
t -= r_cols.get(jj).and_then(|rc| rc.get(i)).copied().unwrap_or(0.0) * c.get(jj).copied().unwrap_or(0.0);
}
if let Some(ci) = c.get_mut(i) {
*ci = t / r_cols.get(i).and_then(|rc| rc.get(i)).copied().unwrap_or(f64::NAN);
}
}
(inv, dropped)
let ssresid = y.get(r..n).unwrap_or(&[]).iter().map(|v| v * v).sum::<f64>();
// R^-1 (upper triangular), row i of the inverse at rinv[i].
let rij = |i: usize, j: usize| r_cols.get(j).and_then(|rc| rc.get(i)).copied().unwrap_or(0.0);
let mut rinv = vec![vec![0.0; r]; r];
for t in 0..r {
for i in (0..=t).rev() {
let mut v = if i == t { 1.0 } else { 0.0 };
for jj in i + 1..=t {
v -= rij(i, jj) * rinv.get(jj).and_then(|row| row.get(t)).copied().unwrap_or(0.0);
}
if let Some(cell) = rinv.get_mut(i).and_then(|row| row.get_mut(t)) {
*cell = v / rij(i, i);
}
}
}
let mut coef = vec![0.0; k];
let mut diag = vec![0.0; k];
for (i, &j) in kept.iter().enumerate() {
let s = scale.get(j).copied().unwrap_or(1.0);
if let Some(v) = coef.get_mut(j) {
*v = c.get(i).copied().unwrap_or(0.0) / s;
}
let row = rinv.get(i);
if let Some(v) = diag.get_mut(j) {
*v = row.map_or(0.0, |rw| rw.iter().map(|x| x * x).sum::<f64>()) / (s * s);
}
}
// xm' (S R'R S)^-1 xm = |R^-T S^-1 xm|².
let w: Vec<f64> = kept.iter().map(|&j| xm.get(j).copied().unwrap_or(0.0) / scale.get(j).copied().unwrap_or(1.0)).collect();
let quad = (0..r)
.map(|t| {
(0..=t).map(|i| rinv.get(i).and_then(|row| row.get(t)).copied().unwrap_or(0.0) * w.get(i).copied().unwrap_or(0.0)).sum::<f64>().powi(2)
})
.sum::<f64>();
QrFit { coef, dropped, ssresid, diag, quad }
}
/// Applies the Householder reflection `I - 2vv'` on rows `at..` of `x`.
fn apply_reflector(at: usize, v: &[f64], x: &mut [f64]) {
let Some(tail) = x.get_mut(at..) else { return };
let dot: f64 = v.iter().zip(tail.iter()).map(|(a, b)| a * b).sum();
tail.iter_mut().zip(v).for_each(|(t, a)| *t -= 2.0 * dot * a);
}
fn regress(y: &[f64], x: &[Vec<f64>], k: usize, constant: bool) -> R<Fit> {
@@ -189,35 +266,22 @@ fn regress(y: &[f64], x: &[Vec<f64>], k: usize, constant: bool) -> R<Fit> {
} else {
(vec![0.0; k], 0.0)
};
// Normal equations on (centred) data.
let mut xtx = vec![vec![0.0; k]; k];
let mut xty = vec![0.0; k];
for (row, yv) in x.iter().zip(y) {
let d: Vec<f64> = (0..k).map(|j| row.get(j).copied().unwrap_or(0.0) - xm.get(j).copied().unwrap_or(0.0)).collect();
let yd = yv - ym;
for (i, di) in d.iter().enumerate() {
if let Some(t) = xty.get_mut(i) {
*t += di * yd;
}
if let Some(r) = xtx.get_mut(i) {
for (j, dj) in d.iter().enumerate() {
if let Some(v) = r.get_mut(j) {
*v += di * dj;
}
}
}
}
}
let (inv, dropped) = sym_inverse(&xtx);
let coef: Vec<f64> = (0..k).map(|i| inv.get(i).map(|r| r.iter().zip(&xty).map(|(a, b)| a * b).sum()).unwrap_or(0.0)).collect();
// (Centred) predictor columns and response.
let cols: Vec<Vec<f64>> =
(0..k).map(|j| x.iter().map(|r| r.get(j).copied().unwrap_or(0.0) - xm.get(j).copied().unwrap_or(0.0)).collect()).collect();
let yc: Vec<f64> = y.iter().map(|v| v - ym).collect();
let sstot: f64 = if constant { yc.iter().map(|v| v * v).sum() } else { y.iter().map(|v| v * v).sum() };
// Only centring leaves rounding residue behind; without an intercept a constant column is data.
// Constant means every raw value is the same double: distinct values, however close, vary.
let constant_cols: Vec<bool> = (0..k)
.map(|j| {
let mut col = x.iter().map(|r| r.get(j).copied().unwrap_or(0.0));
let first = col.next();
constant && first.is_some_and(|f| col.all(|v| v == f))
})
.collect();
let QrFit { coef, dropped, ssresid, diag, quad } = qr_solve(cols, yc, &xm, &constant_cols);
let b = if constant { ym - coef.iter().zip(&xm).map(|(c, m)| c * m).sum::<f64>() } else { 0.0 };
let mut ssresid = 0.0;
let mut sstot = 0.0;
for (row, yv) in x.iter().zip(y) {
let pred = b + coef.iter().zip(row).map(|(c, v)| c * v).sum::<f64>();
ssresid += (yv - pred).powi(2);
sstot += if constant { (yv - ym).powi(2) } else { yv * yv };
}
let used = dropped.iter().filter(|d| !**d).count() + usize::from(constant);
let df = nf - used as f64;
let ssreg = sstot - ssresid;
@@ -230,20 +294,9 @@ fn regress(y: &[f64], x: &[Vec<f64>], k: usize, constant: bool) -> R<Fit> {
_ => Err(CellError::Num),
};
let s2 = sey2.unwrap_or(f64::NAN);
let se: Vec<f64> = (0..k)
.map(|i| if dropped.get(i).copied().unwrap_or(true) { 0.0 } else { (inv.get(i).and_then(|r| r.get(i)).copied().unwrap_or(0.0) * s2).sqrt() })
.collect();
let se_b = if constant {
let mut q = 0.0;
for (i, mi) in xm.iter().enumerate() {
for (j, mj) in xm.iter().enumerate() {
q += mi * mj * inv.get(i).and_then(|r| r.get(j)).copied().unwrap_or(0.0);
}
}
Some((s2 * (1.0 / nf + q)).sqrt())
} else {
None
};
let se: Vec<f64> =
(0..k).map(|i| if dropped.get(i).copied().unwrap_or(true) { 0.0 } else { (diag.get(i).copied().unwrap_or(0.0) * s2).sqrt() }).collect();
let se_b = if constant { Some((s2 * (1.0 / nf + quad)).sqrt()) } else { None };
Ok(Fit { coef, b, se, se_b, r2, sey, f, df, ssreg, ssresid })
}
+179
View File
@@ -433,6 +433,185 @@ fn linest_family() {
is_err(ev("LINEST", vec![c(&[1.0, 2.0]), c(&[1.0, 2.0, 3.0])]), CellError::Ref);
}
/// LINEST on columns of predictors, `stats` on, as rows of numbers (`None` for an error cell).
fn linest_rows(y: &[f64], xs: &[Vec<f64>], constant: bool) -> Vec<Vec<Option<f64>>> {
let x = arr((0..y.len()).map(|i| xs.iter().map(|col| nv(col[i])).collect()).collect());
let r = rows_of(&ev("LINEST", vec![c(y), av(x), b(constant), b(true)]));
r.iter().map(|row| row.iter().map(Value::as_f64).collect()).collect()
}
fn rel_close(got: Option<f64>, want: f64, tol: f64, what: &str) {
let got = got.unwrap_or(f64::NAN);
assert!((got - want).abs() <= tol * want.abs().max(1e-300), "{what}: got {got:e}, want {want:e}, rel tol {tol:e}");
}
// B2: LINEST solved the normal equations, whose condition number is the square of the data's.
// Expected values below come from exact rational arithmetic: the normal equations solved with
// Python `fractions.Fraction` (no rounding at all), rounded once to f64 (script in the PR text).
#[test]
fn linest_ill_conditioned_polynomial_is_exact() {
// y = 1 + x + x² + x³ + x⁴ + x⁵ on x = 1..12: every coefficient is exactly 1. The normal
// equations were off by up to 2.6e-6.
let xs: Vec<f64> = (1..=12).map(f64::from).collect();
let ys: Vec<f64> = xs.iter().map(|x| (0..=5).map(|j| x.powi(j)).sum()).collect();
let cols: Vec<Vec<f64>> = (1..=5).map(|j| xs.iter().map(|x| x.powi(j)).collect()).collect();
let r = linest_rows(&ys, &cols, true);
for (i, v) in r[0].iter().enumerate() {
rel_close(*v, 1.0, 1e-9, &format!("coefficient {i}"));
}
}
#[test]
fn linest_large_offset_regressors_recover_the_line() {
// y = 3x + 7 exactly with x = 1e6 + i and regressors [x, x²]: the true fit has x² coefficient
// 0, x coefficient 3, intercept 7. The normal equations returned 2.99976 and 251.14.
let xs: Vec<f64> = (0..12).map(|i| 1_000_000.0 + f64::from(i)).collect();
let ys: Vec<f64> = xs.iter().map(|x| 3.0 * x + 7.0).collect();
let r = linest_rows(&ys, &[xs.clone(), xs.iter().map(|x| x * x).collect()], true);
assert!(r[0][0].unwrap().abs() < 1e-12, "x² coefficient {:?}", r[0][0]);
rel_close(r[0][1], 3.0, 1e-9, "x coefficient");
// The intercept subtracts 3e6 from 3e6 + 7 and carries the x² coefficient times 1e12, so
// it is good to about 1e-4 in absolute terms, not better.
assert!((r[0][2].unwrap() - 7.0).abs() < 1e-3, "intercept {:?}", r[0][2]);
// One regressor: exact.
let one = linest_rows(&ys, std::slice::from_ref(&xs), true);
rel_close(one[0][0], 3.0, 1e-12, "slope");
rel_close(one[0][1], 7.0, 1e-9, "intercept");
}
#[test]
fn linest_noisy_fits_match_exact_rational_least_squares() {
// Two predictors with noise, constant fitted. Order of each row: x2, x1, intercept.
let x1 = [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0];
let x2 = [2.0, 1.0, 4.0, 3.0, 7.0, 5.0, 9.0, 6.0];
let y = [5.0, 6.0, 11.0, 12.0, 20.0, 19.0, 30.0, 28.0];
let r = linest_rows(&y, &[x1.to_vec(), x2.to_vec()], true);
let (coef, se) = ([1.4270152505446623, 2.452069716775599, -1.2592592592592593], [0.3376442444771937, 0.3679392851360571, 1.0847489958915801]);
for i in 0..3 {
rel_close(r[0][i], coef[i], 1e-12, "coef");
rel_close(r[1][i], se[i], 1e-12, "se");
}
rel_close(r[2][0], 0.985070139277778, 1e-12, "r2");
rel_close(r[2][1], 1.367057173604725, 1e-12, "sey");
rel_close(r[3][0], 164.94965318256004, 1e-11, "F");
assert_eq!(r[3][1], Some(5.0));
rel_close(r[4][0], 616.5307734204792, 1e-12, "ssreg");
rel_close(r[4][1], 9.344226579520697, 1e-11, "ssresid");
assert_eq!(r[2][2], None);
// const = FALSE: through the origin, R² and F from the uncentred sums.
let r = linest_rows(&y, &[x1.to_vec(), x2.to_vec()], false);
rel_close(r[0][0], 1.3529411764705883, 1e-12, "coef");
rel_close(r[0][1], 2.303921568627451, 1e-12, "coef");
assert_eq!(r[0][2], Some(0.0));
rel_close(r[1][0], 0.34102994374364876, 1e-12, "se");
rel_close(r[1][1], 0.35495521934550084, 1e-12, "se");
assert_eq!(r[1][2], None);
rel_close(r[2][0], 0.995718966041848, 1e-12, "r2");
rel_close(r[2][1], 1.4061024795535124, 1e-12, "sey");
rel_close(r[3][0], 697.7652892561983, 1e-11, "F");
assert_eq!(r[3][1], Some(6.0));
rel_close(r[4][0], 2759.1372549019607, 1e-12, "ssreg");
rel_close(r[4][1], 11.862745098039216, 1e-11, "ssresid");
// Cubic and quintic with noise on x = 1..12; order x^k .. x, intercept.
let xs: Vec<f64> = (1..=12).map(f64::from).collect();
let noise = [3.0, -2.0, 4.0, -1.0, 2.0, -3.0, 1.0, -4.0, 2.0, 3.0, -2.0, 1.0];
let cub: Vec<f64> = xs.iter().zip(noise).map(|(x, e)| 1.0 + x + x * x + x.powi(3) + e).collect();
let r = linest_rows(&cub, &(1..=3).map(|j| xs.iter().map(|x| x.powi(j)).collect()).collect::<Vec<Vec<f64>>>(), true);
let coef = [0.9955969955969955, 1.1452991452991452, -0.33514633514633513, 4.373737373737374];
let se = [0.02755463331301788, 0.5434124199147191, 3.102549978874372, 4.851885058051787];
for i in 0..4 {
rel_close(r[0][i], coef[i], 1e-11, "cubic coef");
rel_close(r[1][i], se[i], 1e-10, "cubic se");
}
rel_close(r[2][0], 0.9999837292113533, 1e-12, "cubic r2");
rel_close(r[4][1], 70.35586635586635, 1e-9, "cubic ssresid");
let quint: Vec<f64> = xs.iter().zip(noise).map(|(x, e)| (0..=5).map(|j| x.powi(j)).sum::<f64>() + 50.0 * e).collect();
let r = linest_rows(&quint, &(1..=5).map(|j| xs.iter().map(|x| x.powi(j)).collect()).collect::<Vec<Vec<f64>>>(), true);
let coef = [0.9330693815987934, 3.022273070067188, -20.97398190045249, 108.20489681886741, -250.24940010969422, 271.45454545454544];
let se = [0.20046539191103213, 6.538253746916523, 78.39635356777087, 421.2608646012634, 976.3504976093745, 736.5866411764556];
for i in 0..6 {
rel_close(r[0][i], coef[i], 1e-8, "quintic coef");
rel_close(r[1][i], se[i], 1e-8, "quintic se");
}
rel_close(r[2][1], 168.58174051770763, 1e-9, "quintic sey");
rel_close(r[4][1], 170518.81941587824, 1e-7, "quintic ssresid");
}
#[test]
fn linest_drops_collinear_columns() {
// x2 = 2·x1: the later column is dropped (coefficient and standard error 0), the fit is the
// one on x1 alone, and the degrees of freedom count only the kept column.
let x1 = [1.0, 2.0, 3.0, 4.0, 5.0];
let y = [2.0, 4.0, 5.0, 4.0, 5.0];
let x2: Vec<f64> = x1.iter().map(|v| 2.0 * v).collect();
let r = linest_rows(&y, &[x1.to_vec(), x2], true);
// Exact: slope 0.6, intercept 2.2, ssresid 2.4, ssreg 3.6, df 3 (Fraction arithmetic).
assert_eq!(r[0][0], Some(0.0));
rel_close(r[0][1], 0.6, 1e-12, "slope");
rel_close(r[0][2], 2.2, 1e-12, "intercept");
assert_eq!(r[1][0], Some(0.0));
rel_close(r[1][1], (0.8f64 / 10.0).sqrt(), 1e-12, "se slope");
assert_eq!(r[3][1], Some(3.0));
rel_close(r[4][0], 3.6, 1e-12, "ssreg");
rel_close(r[4][1], 2.4, 1e-12, "ssresid");
// A constant column is collinear with the intercept and dropped as well.
let r = linest_rows(&y, &[vec![7.0; 5], x1.to_vec()], true);
assert_eq!(r[0][1], Some(0.0));
assert_eq!(r[1][1], Some(0.0));
rel_close(r[0][0], 0.6, 1e-12, "slope");
rel_close(r[0][2], 2.2, 1e-12, "intercept");
// A constant column whose mean does not centre to exactly 0 (the mean of three 0.1 is
// 0.10000000000000002) is just as collinear with the intercept: dropped, not scaled up to a
// unit vector. Some length hits a rounding mean for each constant.
for c in [0.1, 1.1, 1e6 + 0.3] {
for n in 3..=12 {
let xs: Vec<f64> = (1..=n).map(f64::from).collect();
let ys: Vec<f64> = (1..=n).map(|i| f64::from(i * i % 7) + 0.5 * f64::from(i)).collect();
let r = linest_rows(&ys, &[vec![c; n as usize], xs], true);
assert_eq!(r[0][1], Some(0.0), "coefficient of the constant {c}, n={n}");
assert_eq!(r[1][1], Some(0.0), "standard error of the constant {c}, n={n}");
assert_eq!(r[3][1], Some(f64::from(n) - 2.0), "df with the constant {c}, n={n}");
}
}
// A column that merely sits far from zero is not noise and is kept.
let off: Vec<f64> = x1.iter().map(|v| 1e9 + v).collect();
let r = linest_rows(&y, &[off], true);
rel_close(r[0][0], 0.6, 1e-6, "slope on an offset column");
assert_eq!(r[3][1], Some(3.0));
// Distinct values far from zero, whose spread is below 1e-12 of the offset, still vary.
let r = linest_rows(&[0.0, 1.0, 2.0], &[vec![1e13, 1e13 + 1.0, 1e13 + 2.0]], true);
rel_close(r[0][0], 1.0, 1e-9, "slope on a predictor with a 1e13 offset");
assert_eq!(r[3][1], Some(1.0), "df: the predictor is kept");
// Same at 1e15, where the spread is only a few ulps of the offset: still distinct doubles.
let x15 = [1e15, 1e15 + 1.0, 1e15 + 2.0];
let r = linest_rows(&[0.0, 1.0, 2.0], &[x15.to_vec()], true);
rel_close(r[0][0], 1.0, 1e-9, "slope on a predictor with a 1e15 offset");
rel_close(r[0][1], -1e15, 1e-9, "intercept on a predictor with a 1e15 offset");
assert_eq!(r[3][1], Some(1.0), "df: the 1e15 predictor is kept");
assert!(r[4][1].is_some_and(|v| v.abs() < 1e-9), "ssresid: the 1e15 line is exact, got {:?}", r[4][1]);
let fitted = rows_of(&ev("TREND", vec![c(&[0.0, 1.0, 2.0]), c(&x15)]));
close(fitted[2][0].clone(), 2.0);
// More predictors than observations: the surplus columns drop, no panic.
let r = linest_rows(&[1.0, 2.0], &[vec![1.0, 2.0], vec![2.0, 1.0], vec![0.0, 5.0]], true);
assert_eq!(r[0].len(), 4);
}
#[test]
fn logest_is_linest_of_ln_y() {
// y = 5·2^x: m = 2, b = 5.
let x = [0.0, 1.0, 2.0, 3.0, 4.0];
let y: Vec<f64> = x.iter().map(|v| 5.0 * 2f64.powf(*v)).collect();
let l = rows_of(&ev("LOGEST", vec![c(&y), c(&x)]));
close_tol(l[0][0].clone(), 2.0, 1e-12);
close_tol(l[0][1].clone(), 5.0, 1e-12);
// The statistics are those of the fit on ln y, untransformed.
let ln: Vec<f64> = y.iter().map(|v| v.ln()).collect();
let ls = rows_of(&ev("LOGEST", vec![c(&y), c(&x), b(true), b(true)]));
let ns = rows_of(&ev("LINEST", vec![c(&ln), c(&x), b(true), b(true)]));
assert_eq!(ls[1..], ns[1..]);
close_tol(ls[0][0].clone(), ns[0][0].as_f64().unwrap().exp(), 1e-14);
}
// #2177: one predictor row of 16,384 columns built three 16,384² matrices (6 GB) and an O(k³)
// inversion; a wider TOROW aborted the process. It is now refused before allocating.
#[test]
+5
View File
@@ -668,6 +668,9 @@ pub enum ShapeKind {
/// A freehand ink stroke; `Shape::text` holds the points as `x,y;x,y;…` relative to the
/// anchor (sheet points), `Shape::line` the ink colour and `fill` unused.
Ink,
/// Decorative text (`insert.wordArt`): a transparent shape whose `Shape::text` is drawn with
/// the large, filled/outlined/shadowed text properties of `Shape::format`.
WordArt,
/// An equation: `Shape::text` holds its LaTeX source, `Shape::line` its ink colour
/// (`Color::Auto` for the sheet's text colour) and `Shape::math` its cached layout.
Equation,
@@ -1026,6 +1029,8 @@ pub struct ShapeFormat {
pub text_color: Option<Color>,
/// Text outline (`a:rPr/a:ln`): colour and width in points.
pub text_outline: Option<(Color, f32)>,
/// Text drop shadow colour (`a:rPr/a:effectLst/a:outerShdw`); `None` is no shadow.
pub text_shadow: Option<Color>,
/// `None`: the shape's default (left for text boxes, centred for other shapes).
pub align: Option<ShapeTextAlign>,
/// `None`: the shape's default (top for text boxes, middle for other shapes).
+2 -1
View File
@@ -235,11 +235,12 @@ impl CellStore {
bytes = bytes.saturating_add(size_of::<Band>());
for (row, cols) in &band.rows {
bytes = bytes.saturating_add(size_of::<u32>() + 3 * size_of::<usize>());
let next_row = next.and_then(|b| b.rows.get(row));
for (col, cell) in cols {
// Cells and BTreeMap nodes are duplicated with a cloned band even when
// their formula/value Arcs are still shared with the next snapshot.
bytes = bytes.saturating_add(size_of::<u32>() + size_of::<Cell>() + 3 * size_of::<usize>());
let next_cell = next.and_then(|b| b.rows.get(row)).and_then(|r| r.get(col));
let next_cell = next_row.and_then(|r| r.get(col));
if let Some(formula) = &cell.formula
&& !next_cell.and_then(|c| c.formula.as_ref()).is_some_and(|f| Arc::ptr_eq(f, formula))
{
+353 -134
View File
@@ -19,7 +19,10 @@ mod symbol;
pub use symbol::{symbol_code, symbol_width};
use std::collections::HashMap;
use std::collections::hash_map::DefaultHasher;
use std::fmt::Write as _;
use std::hash::{Hash, Hasher};
/// RGB colour, 0–255 per channel.
pub type Rgb = [u8; 3];
@@ -322,6 +325,31 @@ fn n(v: f32) -> String {
}
}
/// Appends `v / 255` with at most three decimals, exactly what `n(v as f32 / 255.0)` gives, but
/// without a float round trip or an allocation per channel.
fn push_unit(s: &mut String, v: u8) {
let m = (u32::from(v) * 1000 + 127) / 255;
match m {
0 => s.push('0'),
1000 => s.push('1'),
_ => {
let _ = write!(s, "0.{m:03}");
while s.ends_with('0') {
s.pop();
}
}
}
}
/// Appends a colour operator line without the trailing newline: `r g b rg` (fill) or `r g b RG`.
fn push_rgb(s: &mut String, c: Rgb, stroke: bool) {
for v in c {
push_unit(s, v);
s.push(' ');
}
s.push_str(if stroke { "RG" } else { "rg" });
}
/// True when every value is finite (NaN and infinities are rejected).
fn finite(v: &[f32]) -> bool {
v.iter().all(|x| x.is_finite())
@@ -348,16 +376,44 @@ fn dash_values(dash: &[f32]) -> String {
d.iter().map(|v| n(*v)).collect::<Vec<_>>().join(" ")
}
fn rgb_op(c: Rgb, stroke: bool) -> String {
let f = |v: u8| n(v as f32 / 255.0);
format!("{} {} {} {}", f(c[0]), f(c[1]), f(c[2]), if stroke { "RG" } else { "rg" })
}
struct ImageData {
width: u32,
height: u32,
rgb: Vec<u8>,
/// `None` when every pixel is fully opaque (no soft mask is written).
alpha: Option<Vec<u8>>,
/// Hash of all of the above, to find duplicates quickly; equality still compares the bytes.
hash: u64,
}
impl ImageData {
fn new(width: u32, height: u32, rgb: Vec<u8>, alpha: Option<Vec<u8>>) -> ImageData {
let mut h = DefaultHasher::new();
width.hash(&mut h);
height.hash(&mut h);
rgb.hash(&mut h);
alpha.hash(&mut h);
ImageData { width, height, rgb, alpha, hash: h.finish() }
}
/// Same pixels (a hash match alone is never trusted).
fn same(&self, o: &ImageData) -> bool {
self.hash == o.hash && self.width == o.width && self.height == o.height && self.rgb == o.rgb && self.alpha == o.alpha
}
}
/// The part of the graphics state the writer tracks so that a style operator is only emitted
/// when it changes. `None` = not set yet on this page (or since the last restore that undid it).
#[derive(Clone, Debug, Default, PartialEq)]
struct Style {
fill: Option<Rgb>,
stroke: Option<Rgb>,
/// Formatted line width.
width: Option<String>,
/// Formatted dash array contents.
dash: Option<String>,
/// Line join; PDF's initial value is 0 (miter).
join: u8,
}
enum LinkTarget {
@@ -378,8 +434,18 @@ pub struct Page {
content: String,
fonts: [bool; FONT_SLOTS],
images: Vec<ImageData>,
/// Indexes into `images`, by content hash; a hash match is confirmed with [`ImageData::same`].
image_index: HashMap<u64, Vec<usize>>,
/// Pixel comparisons made while looking up duplicate pictures (test instrumentation).
#[cfg(test)]
image_comparisons: usize,
/// Style operators already in effect at the end of `content`.
style: Style,
/// One entry per accepted [`Page::save`], restored by [`Page::restore`].
saved: Vec<Style>,
/// `save` calls refused at the nesting limit; the same number of `restore` calls are ignored.
refused: u32,
links: Vec<LinkAnnotation>,
depth: u32,
}
/// Most bytes of content per page before further drawing is dropped (keeps hostile inputs from
@@ -387,6 +453,11 @@ pub struct Page {
const MAX_PAGE_CONTENT: usize = 64 * 1024 * 1024;
/// Largest image (pixels) embedded.
const MAX_IMAGE_PIXELS: u64 = 40_000_000;
/// Deepest `q` nesting a PDF consumer must support (the PDF specification's implementation limits).
const MAX_SAVE_DEPTH: usize = 28;
/// Line join styles, as the `j` operator takes them.
const JOIN_MITER: u8 = 0;
const JOIN_ROUND: u8 = 1;
const MAX_PAGE_LINKS: usize = 4096;
const MAX_LINK_URI_BYTES: usize = 4096;
@@ -410,6 +481,38 @@ impl Page {
}
}
/// Appends a fill colour operator to `s` unless that colour is already in effect.
fn set_fill(&mut self, s: &mut String, c: Rgb) {
if self.style.fill != Some(c) {
push_rgb(s, c, false);
s.push('\n');
self.style.fill = Some(c);
}
}
/// Appends the stroke colour, width, dash and join operators to `s` that differ from the
/// style already in effect.
fn set_stroke(&mut self, s: &mut String, c: Rgb, width: f32, dash: &str, join: u8) {
if self.style.stroke != Some(c) {
push_rgb(s, c, true);
s.push('\n');
self.style.stroke = Some(c);
}
let w = n(stroke_width(width));
if self.style.width.as_deref() != Some(w.as_str()) {
let _ = writeln!(s, "{w} w");
self.style.width = Some(w);
}
if self.style.dash.as_deref() != Some(dash) {
let _ = writeln!(s, "[{dash}] 0 d");
self.style.dash = Some(dash.to_string());
}
if self.style.join != join {
let _ = writeln!(s, "{join} j");
self.style.join = join;
}
}
fn link_rect(&self, rect: Rect) -> Option<[f32; 4]> {
let Rect { x, y, width, height } = rect;
if !finite(&[x, y, width, height]) || !extent(width) || !extent(height) || self.links.len() >= MAX_PAGE_LINKS {
@@ -447,15 +550,23 @@ impl Page {
}
}
/// Saves the graphics state (pair with [`Page::restore`]).
/// Saves the graphics state (pair with [`Page::restore`]). Nesting is limited to 28 levels, the
/// depth every PDF consumer must support; a `save` beyond that is ignored, and so is the
/// `restore` that would have matched it.
pub fn save(&mut self) {
self.depth = self.depth.saturating_add(1);
if self.saved.len() >= MAX_SAVE_DEPTH {
self.refused = self.refused.saturating_add(1);
return;
}
self.saved.push(self.style.clone());
self.content.push_str("q\n");
}
/// Restores the graphics state saved by [`Page::save`]; extra calls are ignored.
pub fn restore(&mut self) {
if self.depth > 0 {
self.depth -= 1;
if self.refused > 0 {
self.refused -= 1;
} else if let Some(style) = self.saved.pop() {
self.style = style;
self.content.push_str("Q\n");
}
}
@@ -466,18 +577,22 @@ impl Page {
}
pub fn fill_rect(&mut self, x: f32, y: f32, w: f32, h: f32, color: Rgb) {
if !(extent(w) && extent(h) && finite(&[x, y])) {
if !(extent(w) && extent(h) && finite(&[x, y])) || self.full() {
return;
}
let s = format!("{}\n{} {} {} {} re f", rgb_op(color, false), n(x), n(self.y(y + h)), n(w), n(h));
let mut s = String::new();
self.set_fill(&mut s, color);
let _ = write!(s, "{} {} {} {} re f", n(x), n(self.y(y + h)), n(w), n(h));
self.op(&s);
}
pub fn rect_stroke(&mut self, x: f32, y: f32, w: f32, h: f32, width: f32, color: Rgb) {
if !finite(&[x, y, w, h]) {
if !finite(&[x, y, w, h]) || self.full() {
return;
}
let s = format!("{}\n{} w [] 0 d\n{} {} {} {} re S", rgb_op(color, true), n(stroke_width(width)), n(x), n(self.y(y + h)), n(w), n(h));
let mut s = String::new();
self.set_stroke(&mut s, color, width, "", JOIN_MITER);
let _ = write!(s, "{} {} {} {} re S", n(x), n(self.y(y + h)), n(w), n(h));
self.op(&s);
}
@@ -487,28 +602,22 @@ impl Page {
/// A line with a dash pattern (`[on, off, …]` in points; empty = solid).
pub fn stroke_line_dashed(&mut self, start: Point, end: Point, width: f32, color: Rgb, dash: &[f32]) {
if !finite(&[start.x, start.y, end.x, end.y]) {
if !finite(&[start.x, start.y, end.x, end.y]) || self.full() {
return;
}
let s = format!(
"{}\n{} w [{}] 0 d\n{} {} m {} {} l S",
rgb_op(color, true),
n(stroke_width(width)),
dash_values(dash),
n(start.x),
n(self.y(start.y)),
n(end.x),
n(self.y(end.y))
);
let mut s = String::new();
self.set_stroke(&mut s, color, width, &dash_values(dash), JOIN_MITER);
let _ = write!(s, "{} {} m {} {} l S", n(start.x), n(self.y(start.y)), n(end.x), n(self.y(end.y)));
self.op(&s);
}
/// An open polyline.
/// An open polyline (round joins).
pub fn polyline(&mut self, pts: &[[f32; 2]], width: f32, color: Rgb, dash: &[f32]) {
if pts.len() < 2 || !pts.iter().all(|p| finite(p)) {
if pts.len() < 2 || !pts.iter().all(|p| finite(p)) || self.full() {
return;
}
let mut s = format!("{}\n{} w [{}] 0 d 1 j\n", rgb_op(color, true), n(stroke_width(width)), dash_values(dash));
let mut s = String::new();
self.set_stroke(&mut s, color, width, &dash_values(dash), JOIN_ROUND);
self.path(&mut s, pts, false);
s.push_str(" S");
self.op(&s);
@@ -523,18 +632,17 @@ impl Page {
}
}
/// A closed polygon, filled and/or stroked.
/// A closed polygon, filled and/or stroked (round joins).
pub fn polygon(&mut self, pts: &[[f32; 2]], fill: Option<Rgb>, stroke: Option<(Rgb, f32)>) {
if pts.len() < 2 || (fill.is_none() && stroke.is_none()) || !pts.iter().all(|p| finite(p)) {
if pts.len() < 2 || (fill.is_none() && stroke.is_none()) || !pts.iter().all(|p| finite(p)) || self.full() {
return;
}
let mut s = String::new();
if let Some(f) = fill {
s.push_str(&rgb_op(f, false));
s.push('\n');
self.set_fill(&mut s, f);
}
if let Some((c, w)) = stroke {
let _ = writeln!(s, "{}\n{} w [] 0 d 1 j", rgb_op(c, true), n(stroke_width(w)));
self.set_stroke(&mut s, c, w, "", JOIN_ROUND);
}
self.path(&mut s, pts, true);
s.push_str(match (fill.is_some(), stroke.is_some()) {
@@ -547,7 +655,7 @@ impl Page {
/// A circle approximated by four cubic Béziers.
pub fn circle(&mut self, cx: f32, cy: f32, r: f32, fill: Option<Rgb>, stroke: Option<(Rgb, f32)>) {
if !extent(r) || !finite(&[cx, cy]) || (fill.is_none() && stroke.is_none()) {
if !extent(r) || !finite(&[cx, cy]) || (fill.is_none() && stroke.is_none()) || self.full() {
return;
}
const K: f32 = 0.552_284_8;
@@ -555,11 +663,10 @@ impl Page {
let y = self.y(cy);
let mut s = String::new();
if let Some(f) = fill {
s.push_str(&rgb_op(f, false));
s.push('\n');
self.set_fill(&mut s, f);
}
if let Some((c, w)) = stroke {
let _ = writeln!(s, "{}\n{} w [] 0 d", rgb_op(c, true), n(stroke_width(w)));
self.set_stroke(&mut s, c, w, "", JOIN_MITER);
}
let _ = write!(
s,
@@ -614,19 +721,24 @@ impl Page {
let size = n(size.min(10_000.0));
let (sn, cs) = if angle.is_finite() { angle.sin_cos() } else { (0.0, 1.0) };
// Clockwise in y-down = counter-clockwise negated in PDF's y-up space.
let mut s = format!("BT\n{}\n{} {} {} {} {} {} Tm\n", rgb_op(color, false), n(cs), n(-sn), n(sn), n(cs), n(origin.x), n(self.y(origin.y)));
let tm = format!("{} {} {} {} {} {} Tm\n", n(cs), n(-sn), n(sn), n(cs), n(origin.x), n(self.y(origin.y)));
// Symbol text is one run in Symbol's own encoding (the CID font has no maths symbols).
if font == Font::Symbol {
if let Some(f) = self.fonts.get_mut(fi) {
*f = true;
}
let _ = writeln!(s, "/F{} {} Tf\n{} Tj", fi + 1, size, pdf_string_in(text, true));
s.push_str("ET");
if self.full() {
return;
}
let mut s = String::new();
self.set_fill(&mut s, color);
let _ = write!(s, "BT\n{tm}/F{} {} Tf\n{} Tj\nET", fi + 1, size, pdf_string_in(text, true));
self.op(&s);
return;
}
// Runs of WinAnsi text use `font`; runs it cannot encode use the CID font. Each `Tj`
// advances the text position, so the runs follow each other on the baseline.
let mut runs = String::new();
let mut rest = text;
while let Some(first) = rest.chars().next() {
let cid = needs_cid(first);
@@ -636,10 +748,21 @@ impl Page {
if let Some(f) = self.fonts.get_mut(slot) {
*f = true;
}
let _ = writeln!(s, "/F{} {} Tf\n{} Tj", slot + 1, size, string);
let first_run = runs.is_empty();
let _ = writeln!(runs, "/F{} {} Tf", slot + 1, size);
if first_run {
runs.push_str(&tm);
}
let _ = writeln!(runs, "{string} Tj");
rest = tail;
}
s.push_str("ET");
if self.full() {
return;
}
// The colour goes in front of the text object, so it is tracked like any other style.
let mut colour = String::new();
self.set_fill(&mut colour, color);
let s = format!("{colour}BT\n{runs}ET");
self.op(&s);
}
@@ -652,16 +775,47 @@ impl Page {
if pw == 0 || ph == 0 || px > MAX_IMAGE_PIXELS || (rgba.len() as u64) < px * 4 || !(extent(w) && extent(h) && finite(&[x, y])) {
return;
}
// The image is drawn inside its own q/Q, which counts towards the nesting limit.
if self.saved.len() >= MAX_SAVE_DEPTH {
return;
}
let px = px as usize;
let mut rgb = Vec::with_capacity(px * 3);
let mut alpha = Vec::with_capacity(px);
for p in rgba.as_chunks::<4>().0.iter().take(px) {
// The alpha plane is only allocated once a non-opaque pixel shows up.
let mut alpha: Option<Vec<u8>> = None;
for (i, p) in rgba.as_chunks::<4>().0.iter().take(px).enumerate() {
rgb.extend_from_slice(&[p[0], p[1], p[2]]);
alpha.push(p[3]);
match &mut alpha {
Some(a) => a.push(p[3]),
None if p[3] != 255 => {
let mut a = Vec::with_capacity(px);
a.resize(i, 255);
a.push(p[3]);
alpha = Some(a);
}
None => {}
}
}
let alpha = if alpha.iter().all(|a| *a == 255) { None } else { Some(alpha) };
let idx = self.images.len();
self.images.push(ImageData { width: pw, height: ph, rgb, alpha });
let img = ImageData::new(pw, ph, rgb, alpha);
// The same picture drawn again on this page shares one XObject.
let candidates = self.image_index.entry(img.hash).or_default();
#[cfg(test)]
let comparisons = &mut self.image_comparisons;
let known = candidates.iter().copied().find(|&i| {
#[cfg(test)]
{
*comparisons += 1;
}
self.images.get(i).is_some_and(|o| o.same(&img))
});
let idx = match known {
Some(i) => i,
None => {
candidates.push(self.images.len());
self.images.push(img);
self.images.len() - 1
}
};
let s = format!("q {} 0 0 {} {} {} cm /Im{} Do Q", n(w), n(h), n(x), n(self.y(y + h)), idx + 1);
self.op(&s);
}
@@ -738,6 +892,49 @@ fn pdf_text_string(text: &str) -> String {
s
}
/// Starts object `id` at the current end of `out` and records its offset for the xref table.
fn begin_obj(out: &mut Vec<u8>, offsets: &mut Vec<usize>, id: u32) {
let i = id as usize;
if offsets.len() <= i {
offsets.resize(i + 1, 0);
}
if let Some(slot) = offsets.get_mut(i) {
*slot = out.len();
}
let _ = writeln!(Utf8(out), "{id} 0 obj");
}
/// Writes `id 0 obj <body> endobj`.
fn dict_obj(out: &mut Vec<u8>, offsets: &mut Vec<usize>, id: u32, body: &str) {
begin_obj(out, offsets, id);
out.extend_from_slice(body.as_bytes());
out.extend_from_slice(b"\nendobj\n");
}
/// Writes a stream object: `dict` (the dictionary entries without `<<`, `>>` and `/Length`, may be
/// empty) and `data` as the stream body.
fn stream_obj(out: &mut Vec<u8>, offsets: &mut Vec<usize>, id: u32, dict: &str, data: &[u8]) {
begin_obj(out, offsets, id);
out.extend_from_slice(b"<<");
if !dict.is_empty() {
out.push(b' ');
out.extend_from_slice(dict.as_bytes());
}
let _ = write!(Utf8(out), " /Length {} >>\nstream\n", data.len());
out.extend_from_slice(data);
out.extend_from_slice(b"\nendstream\nendobj\n");
}
/// Lets `write!` append text to a byte buffer.
struct Utf8<'a>(&'a mut Vec<u8>);
impl std::fmt::Write for Utf8<'_> {
fn write_str(&mut self, s: &str) -> std::fmt::Result {
self.0.extend_from_slice(s.as_bytes());
Ok(())
}
}
impl PdfDoc {
pub fn new() -> PdfDoc {
PdfDoc::default()
@@ -747,7 +944,20 @@ impl PdfDoc {
pub fn add_page(&mut self, width_pt: f32, height_pt: f32) -> &mut Page {
let width = if width_pt.is_finite() { width_pt.clamp(3.0, 14_400.0) } else { 612.0 };
let height = if height_pt.is_finite() { height_pt.clamp(3.0, 14_400.0) } else { 792.0 };
self.pages.push(Page { width, height, content: String::new(), fonts: [false; FONT_SLOTS], images: Vec::new(), links: Vec::new(), depth: 0 });
self.pages.push(Page {
width,
height,
content: String::new(),
fonts: [false; FONT_SLOTS],
images: Vec::new(),
image_index: HashMap::new(),
#[cfg(test)]
image_comparisons: 0,
links: Vec::new(),
style: Style::default(),
saved: Vec::new(),
refused: 0,
});
let i = self.pages.len() - 1;
&mut self.pages[i]
}
@@ -762,10 +972,22 @@ impl PdfDoc {
}
/// Serializes the document. A document without pages gets one blank Letter page.
///
/// Every object is written straight into the output buffer (nothing is staged in an
/// intermediate list of bodies); only the xref table needs the offset of each object id, and
/// the ids are contiguous from 1. Objects are therefore in the file in the order they are
/// numbered during writing (fonts, then per page its new images, content stream and page
/// dictionary), with the catalog, page tree and info dictionary last.
pub fn finish(mut self) -> Vec<u8> {
if self.pages.is_empty() {
self.add_page(612.0, 792.0);
}
for page in &mut self.pages {
page.refused = 0;
while !page.saved.is_empty() {
page.restore();
}
}
// Object numbering: 1 catalog, 2 page tree, 3 info, then fonts, then per page.
let mut used = [false; FONT_SLOTS];
for p in &self.pages {
@@ -782,7 +1004,13 @@ impl PdfDoc {
next += if i == CID_FONT { 3 } else { 1 };
}
}
let mut objects: Vec<(u32, Vec<u8>)> = Vec::new();
// Page contents are moved out so each one is freed as soon as it is in the output.
let contents: Vec<String> = self.pages.iter_mut().map(|p| std::mem::take(&mut p.content)).collect();
let page_links: Vec<Vec<LinkAnnotation>> = self.pages.iter_mut().map(|p| std::mem::take(&mut p.links)).collect();
let reserve = contents.iter().fold(4096usize, |acc, c| acc.saturating_add(c.len()).saturating_add(1024));
let mut out: Vec<u8> = Vec::with_capacity(reserve);
out.extend_from_slice(b"%PDF-1.7\n%\xE2\xE3\xCF\xD3\n");
let mut offsets: Vec<usize> = vec![0; 4];
let mut kids = Vec::new();
let mut annotations = Vec::new();
let page_count = self.pages.len();
@@ -792,86 +1020,87 @@ impl PdfDoc {
let _ = write!(font_dict, "/F{} {} 0 R ", i + 1, id);
if i == CID_FONT {
let (cid_id, desc_id) = (id + 1, id + 2);
objects.push((
dict_obj(
&mut out,
&mut offsets,
*id,
format!(
&format!(
"<< /Type /Font /Subtype /Type0 /BaseFont /{CID_BASE_FONT} /Encoding /UniGB-UTF16-H /DescendantFonts [{cid_id} 0 R] >>"
)
.into_bytes(),
));
objects.push((
),
);
dict_obj(
&mut out,
&mut offsets,
cid_id,
format!(
&format!(
"<< /Type /Font /Subtype /CIDFontType0 /BaseFont /{CID_BASE_FONT} /CIDSystemInfo << /Registry (Adobe) /Ordering (GB1) /Supplement 4 >> /FontDescriptor {desc_id} 0 R /DW {CID_WIDTH} >>"
)
.into_bytes(),
));
objects.push((
),
);
dict_obj(
&mut out,
&mut offsets,
desc_id,
format!(
&format!(
"<< /Type /FontDescriptor /FontName /{CID_BASE_FONT} /Flags 6 /FontBBox [-25 -254 1000 880] /ItalicAngle 0 /Ascent 880 /Descent -120 /CapHeight 880 /StemV 93 >>"
)
.into_bytes(),
));
),
);
continue;
}
let name = Font::ALL.get(i).copied().unwrap_or_default().base_name();
let enc = if name.starts_with("Symbol") { "" } else { " /Encoding /WinAnsiEncoding" };
objects.push((*id, format!("<< /Type /Font /Subtype /Type1 /BaseFont /{name}{enc} >>").into_bytes()));
dict_obj(&mut out, &mut offsets, *id, &format!("<< /Type /Font /Subtype /Type1 /BaseFont /{name}{enc} >>"));
}
}
for page in &mut self.pages {
while page.depth > 0 {
page.restore();
}
// Images already written, by content hash: (page, image, object id). A hash match is
// confirmed by comparing the pixels before an object is shared.
let mut written: HashMap<u64, Vec<(usize, usize, u32)>> = HashMap::new();
for (pi, ((page, content), links)) in self.pages.iter().zip(contents).zip(page_links).enumerate() {
let mut xobjects = String::new();
for (k, img) in page.images.iter().enumerate() {
let img_id = next;
next += 1;
let smask = match &img.alpha {
Some(a) => {
let id = next;
let known = written.get(&img.hash).and_then(|c| {
c.iter().find_map(|(p, i, id)| {
let other = self.pages.get(*p)?.images.get(*i)?;
img.same(other).then_some(*id)
})
});
let img_id = match known {
Some(id) => id,
None => {
let img_id = next;
next += 1;
let data = run_length(a);
let mut o = format!(
"<< /Type /XObject /Subtype /Image /Width {} /Height {} /ColorSpace /DeviceGray /BitsPerComponent 8 /Filter /RunLengthDecode /Length {} >>\nstream\n",
img.width,
img.height,
data.len()
)
.into_bytes();
o.extend_from_slice(&data);
o.extend_from_slice(b"\nendstream");
objects.push((id, o));
format!(" /SMask {id} 0 R")
let mut dict = format!("/Type /XObject /Subtype /Image /Width {} /Height {} /BitsPerComponent 8", img.width, img.height);
let smask_id = img.alpha.as_ref().map(|_| {
let id = next;
next += 1;
id
});
dict.push_str(" /ColorSpace /DeviceRGB");
if let Some(id) = smask_id {
let _ = write!(dict, " /SMask {id} 0 R");
}
dict.push_str(" /Filter /RunLengthDecode");
stream_obj(&mut out, &mut offsets, img_id, &dict, &run_length(&img.rgb));
if let (Some(id), Some(a)) = (smask_id, &img.alpha) {
let dict = format!(
"/Type /XObject /Subtype /Image /Width {} /Height {} /ColorSpace /DeviceGray /BitsPerComponent 8 /Filter /RunLengthDecode",
img.width, img.height
);
stream_obj(&mut out, &mut offsets, id, &dict, &run_length(a));
}
written.entry(img.hash).or_default().push((pi, k, img_id));
img_id
}
None => String::new(),
};
let data = run_length(&img.rgb);
let mut o = format!(
"<< /Type /XObject /Subtype /Image /Width {} /Height {} /ColorSpace /DeviceRGB /BitsPerComponent 8{} /Filter /RunLengthDecode /Length {} >>\nstream\n",
img.width,
img.height,
smask,
data.len()
)
.into_bytes();
o.extend_from_slice(&data);
o.extend_from_slice(b"\nendstream");
objects.push((img_id, o));
let _ = write!(xobjects, "/Im{} {} 0 R ", k + 1, img_id);
}
let content_id = next;
let page_id = next + 1;
next += 2;
let body = std::mem::take(&mut page.content);
let mut o = format!("<< /Length {} >>\nstream\n", body.len()).into_bytes();
o.extend_from_slice(body.as_bytes());
o.extend_from_slice(b"endstream");
objects.push((content_id, o));
stream_obj(&mut out, &mut offsets, content_id, "", content.as_bytes());
drop(content);
let xo = if xobjects.is_empty() { String::new() } else { format!(" /XObject << {xobjects}>>") };
let mut link_refs = String::new();
for link in std::mem::take(&mut page.links) {
for link in links {
if matches!(&link.target, LinkTarget::Page { index, .. } if *index >= page_count) {
continue;
}
@@ -881,19 +1110,20 @@ impl PdfDoc {
annotations.push((id, link));
}
let annots = if link_refs.is_empty() { String::new() } else { format!(" /Annots [{link_refs}]") };
objects.push((
dict_obj(
&mut out,
&mut offsets,
page_id,
format!(
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 {} {}] /Resources << /ProcSet [/PDF /Text /ImageB /ImageC] /Font << {}>>{} >> /Contents {} 0 R{} >>",
&format!(
"<< /Type /Page /Parent 2 0 R /MediaBox [0 0 {} {}] /Resources << /Font << {}>>{} >> /Contents {} 0 R{} >>",
n(page.width),
n(page.height),
font_dict,
xo,
content_id,
annots
)
.into_bytes(),
));
),
);
kids.push(page_id);
}
for (id, link) in annotations {
@@ -906,14 +1136,16 @@ impl PdfDoc {
format!("/Dest [{page_id} 0 R /XYZ {} {} null]", n(x), n(height - y))
}
};
objects.push((
dict_obj(
&mut out,
&mut offsets,
id,
format!("<< /Type /Annot /Subtype /Link /Rect [{} {} {} {}] /Border [0 0 0] {action} >>", n(x0), n(y0), n(x1), n(y1)).into_bytes(),
));
&format!("<< /Type /Annot /Subtype /Link /Rect [{} {} {} {}] /Border [0 0 0] {action} >>", n(x0), n(y0), n(x1), n(y1)),
);
}
objects.push((1, b"<< /Type /Catalog /Pages 2 0 R >>".to_vec()));
dict_obj(&mut out, &mut offsets, 1, "<< /Type /Catalog /Pages 2 0 R >>");
let kids_s: Vec<String> = kids.iter().map(|k| format!("{k} 0 R")).collect();
objects.push((2, format!("<< /Type /Pages /Kids [{}] /Count {} >>", kids_s.join(" "), kids.len()).into_bytes()));
dict_obj(&mut out, &mut offsets, 2, &format!("<< /Type /Pages /Kids [{}] /Count {} >>", kids_s.join(" "), kids.len()));
let creator = if self.info.creator.is_empty() { "GridCraft" } else { &self.info.creator };
let mut info = format!("<< /Producer {} /Creator {}", pdf_text_string("GridCraft PDF"), pdf_text_string(creator));
if !self.info.title.is_empty() {
@@ -923,27 +1155,14 @@ impl PdfDoc {
let _ = write!(info, " /Author {}", pdf_text_string(&self.info.author));
}
info.push_str(" >>");
objects.push((3, info.into_bytes()));
objects.sort_by_key(|(id, _)| *id);
dict_obj(&mut out, &mut offsets, 3, &info);
let mut out: Vec<u8> = Vec::new();
out.extend_from_slice(b"%PDF-1.7\n%\xE2\xE3\xCF\xD3\n");
let mut offsets = vec![0usize; next as usize];
for (id, body) in &objects {
if let Some(slot) = offsets.get_mut(*id as usize) {
*slot = out.len();
}
out.extend_from_slice(format!("{id} 0 obj\n").as_bytes());
out.extend_from_slice(body);
out.extend_from_slice(b"\nendobj\n");
}
let xref = out.len();
let mut x = format!("xref\n0 {}\n0000000000 65535 f \n", next);
let _ = write!(Utf8(&mut out), "xref\n0 {}\n0000000000 65535 f \n", next);
for off in offsets.iter().skip(1) {
let _ = writeln!(x, "{off:010} 00000 n ");
let _ = writeln!(Utf8(&mut out), "{off:010} 00000 n ");
}
let _ = write!(x, "trailer\n<< /Size {} /Root 1 0 R /Info 3 0 R >>\nstartxref\n{}\n%%EOF\n", next, xref);
out.extend_from_slice(x.as_bytes());
let _ = write!(Utf8(&mut out), "trailer\n<< /Size {} /Root 1 0 R /Info 3 0 R >>\nstartxref\n{}\n%%EOF\n", next, xref);
out
}
}
+574 -2
View File
@@ -2,6 +2,9 @@ use super::*;
use std::error::Error;
use std::io;
mod reference;
use reference::{OldPage, interpret};
fn find(hay: &[u8], needle: &[u8]) -> Option<usize> {
hay.windows(needle.len()).position(|w| w == needle)
}
@@ -189,6 +192,558 @@ fn hostile_inputs_never_panic() -> Result<(), Box<dyn Error>> {
Ok(())
}
// ---------------------------------------------------------------------------------------------
// Writer cleanup: equivalence with the previous content writer, structure, and one test per
// behaviour change.
// ---------------------------------------------------------------------------------------------
/// Offsets of the in-use objects, by id, from the xref table.
fn xref_offsets(pdf: &[u8]) -> Vec<usize> {
let sx = rfind(pdf, b"startxref\n").unwrap();
let rest = std::str::from_utf8(&pdf[sx + 10..]).unwrap();
let off: usize = rest.lines().next().unwrap().trim().parse().unwrap();
let table = std::str::from_utf8(&pdf[off..sx]).unwrap();
table.lines().skip(2).take_while(|l| l.len() == 18 || l.len() == 19).map(|e| e[..10].parse().unwrap()).collect()
}
struct Obj {
dict: String,
stream: Option<Vec<u8>>,
}
/// Parses every object of a document written by this crate, checking that each stream's
/// `/Length` matches its data and is followed by `endstream`/`endobj`.
fn parse_objects(pdf: &[u8]) -> Vec<Obj> {
let offsets = xref_offsets(pdf);
let mut objs = Vec::new();
for (id, off) in offsets.iter().enumerate() {
if id == 0 {
objs.push(Obj { dict: String::new(), stream: None });
continue;
}
let tail = &pdf[*off..];
let head = format!("{id} 0 obj\n");
assert!(tail.starts_with(head.as_bytes()), "object {id}");
let body = &tail[head.len()..];
let end = find(body, b"\nendobj\n").unwrap();
match find(body, b"\nstream\n").filter(|s| *s < end) {
Some(s) => {
let dict = String::from_utf8(body[..s].to_vec()).unwrap();
let l = dict.find("/Length ").unwrap() + 8;
let len: usize = dict[l..].split(' ').next().unwrap().parse().unwrap();
let data = &body[s + 8..];
assert!(data[len..].starts_with(b"\nendstream\nendobj\n"), "object {id}: /Length {len} is wrong");
objs.push(Obj { dict, stream: Some(data[..len].to_vec()) });
}
None => objs.push(Obj { dict: String::from_utf8(body[..end].to_vec()).unwrap(), stream: None }),
}
}
objs
}
fn ref_of(dict: &str, key: &str) -> Option<usize> {
let i = dict.find(key)? + key.len();
dict[i..].trim_start().split(' ').next()?.parse().ok()
}
struct ParsedPage {
media: String,
content: String,
/// `(Im name, object id)`.
xobjects: Vec<(String, usize)>,
}
fn parse_pages(pdf: &[u8]) -> (Vec<Obj>, Vec<ParsedPage>) {
let objs = parse_objects(pdf);
let mut pages = Vec::new();
for o in &objs {
if !o.dict.contains("/Type /Page ") {
continue;
}
let m = o.dict.find("/MediaBox [").unwrap() + 10;
let media = o.dict[m..m + o.dict[m..].find(']').unwrap() + 1].to_string();
let content_id = ref_of(&o.dict, "/Contents").unwrap();
let content = String::from_utf8(objs[content_id].stream.clone().unwrap()).unwrap();
let mut xobjects = Vec::new();
if let Some(x) = o.dict.find("/XObject << ") {
let rest = &o.dict[x + 11..];
let rest = &rest[..rest.find(">>").unwrap()];
let toks: Vec<&str> = rest.split_whitespace().collect();
for c in toks.chunks(4) {
xobjects.push((c[0].to_string(), c[1].parse().unwrap()));
}
}
pages.push(ParsedPage { media, content, xobjects });
}
(objs, pages)
}
fn rle_decode(d: &[u8]) -> Vec<u8> {
let mut out = Vec::new();
let mut i = 0;
while i < d.len() {
let l = d[i];
i += 1;
if l == 128 {
break;
} else if l < 128 {
out.extend_from_slice(&d[i..i + l as usize + 1]);
i += l as usize + 1;
} else {
out.extend(std::iter::repeat_n(d[i], 257 - l as usize));
i += 1;
}
}
out
}
/// Decodes an image XObject: `(width, height, rgb, alpha)`.
fn decode_image(objs: &[Obj], id: usize) -> (u32, u32, Vec<u8>, Option<Vec<u8>>) {
let o = &objs[id];
let w = ref_of(&o.dict, "/Width").unwrap() as u32;
let h = ref_of(&o.dict, "/Height").unwrap() as u32;
let rgb = rle_decode(o.stream.as_ref().unwrap());
let alpha = ref_of(&o.dict, "/SMask").map(|m| rle_decode(objs[m].stream.as_ref().unwrap()));
(w, h, rgb, alpha)
}
/// Deterministic pseudo-random numbers for the corpus.
struct Lcg(u64);
impl Lcg {
fn next(&mut self) -> u64 {
self.0 = self.0.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
self.0 >> 33
}
fn below(&mut self, n: u64) -> u64 {
self.next() % n
}
fn f(&mut self) -> f32 {
self.below(600) as f32 + self.below(4) as f32 * 0.25
}
}
const PALETTE: [Rgb; 4] = [[0, 0, 0], [255, 0, 0], [12, 200, 99], [255, 255, 255]];
const WIDTHS: [f32; 3] = [0.5, 1.0, 2.5];
const DASHES: [&[f32]; 3] = [&[], &[3.0, 2.0], &[1.0]];
/// A picture used by the corpus: `(rgba, w, h)`.
fn pictures() -> Vec<(Vec<u8>, u32, u32)> {
let mut with_alpha = Vec::new();
for i in 0..16u8 {
with_alpha.extend_from_slice(&[i * 10, 0, 7, if i < 8 { 255 } else { 128 }]);
}
let opaque: Vec<u8> = (0..4 * 9).map(|i| if i % 4 == 3 { 255 } else { (i * 7) as u8 }).collect();
let mut last_px = vec![200u8; 4 * 6];
for px in last_px.chunks_mut(4) {
px[3] = 255;
}
last_px[4 * 5 + 3] = 3; // only the very last pixel is translucent
vec![(with_alpha, 4, 4), (opaque, 3, 3), (last_px, 3, 2)]
}
/// Applies a random operation to both writers.
fn random_op(rng: &mut Lcg, new: &mut Page, old: &mut OldPage, pics: &[(Vec<u8>, u32, u32)], depth: &mut usize) {
let color = PALETTE[rng.below(4) as usize];
let width = WIDTHS[rng.below(3) as usize];
let dash = DASHES[rng.below(3) as usize];
let (x, y, w, h) = (rng.f(), rng.f(), rng.f() / 4.0 + 1.0, rng.f() / 4.0 + 1.0);
let pts = [[x, y], [x + w, y], [x + w, y + h]];
let stroke = if rng.below(2) == 0 { Some((PALETTE[rng.below(4) as usize], WIDTHS[rng.below(3) as usize])) } else { None };
let fill = if rng.below(2) == 0 { Some(color) } else { None };
match rng.below(14) {
0 | 1 => {
new.fill_rect(x, y, w, h, color);
old.fill_rect(x, y, w, h, color);
}
2 => {
new.rect_stroke(x, y, w, h, width, color);
old.rect_stroke(x, y, w, h, width, color);
}
3 => {
new.stroke_line_dashed(Point::new(x, y), Point::new(x + w, y + h), width, color, dash);
old.stroke_line_dashed(Point::new(x, y), Point::new(x + w, y + h), width, color, dash);
}
4 => {
new.polyline(&pts, width, color, dash);
old.polyline(&pts, width, color, dash);
}
5 => {
new.polygon(&pts, fill, stroke);
old.polygon(&pts, fill, stroke);
}
6 => {
new.circle(x, y, w, fill, stroke);
old.circle(x, y, w, fill, stroke);
}
7 | 8 => {
let font = Font::ALL[rng.below(12) as usize];
let angle = if rng.below(3) == 0 { 0.5 } else { 0.0 };
new.text_rotated(Point::new(x, y), 10.0, font, color, "Hi (there)", angle);
old.text_rotated(Point::new(x, y), 10.0, font, color, "Hi (there)", angle);
}
9 => {
new.clip_rect(x, y, w, h);
old.clip_rect(x, y, w, h);
}
10 if *depth < 20 => {
new.save();
old.save();
*depth += 1;
}
11 if *depth > 0 => {
new.restore();
old.restore();
*depth -= 1;
}
12 => {
let (px, pw, ph) = &pics[rng.below(3) as usize];
new.image_rgba(Rect::new(x, y, w, h), px, [*pw, *ph]);
old.image(x, y, w, h);
}
_ => {
new.text(x, y, 9.0, Font::Helvetica, color, "Plain");
old.text_rotated(Point::new(x, y), 9.0, Font::Helvetica, color, "Plain", 0.0);
}
}
}
/// Builds the same random multi-page document with the new writer and the old content writer.
fn corpus_doc(seed: u64) -> (Vec<u8>, Vec<String>) {
let mut rng = Lcg(seed);
let pics = pictures();
let mut doc = PdfDoc::new();
let mut old_pages = Vec::new();
for _ in 0..(1 + rng.below(4)) {
let mut old = OldPage::new(792.0);
let mut depth = 0usize;
let page = doc.add_page(612.0, 792.0);
for _ in 0..120 {
random_op(&mut rng, page, &mut old, &pics, &mut depth);
}
while depth > 0 {
page.restore();
old.restore();
depth -= 1;
}
old_pages.push(old.content);
}
(doc.finish(), old_pages)
}
#[test]
fn new_writer_is_equivalent_to_the_old_one() -> Result<(), Box<dyn Error>> {
let mut style_new = 0;
let mut style_old = 0;
for seed in 1..=40u64 {
let (pdf, old_pages) = corpus_doc(seed);
check_xref(&pdf)?;
let (_, pages) = parse_pages(&pdf);
assert_eq!(pages.len(), old_pages.len());
for (p, old) in pages.iter().zip(&old_pages) {
assert_eq!(p.media, "[0 0 612 792]");
let new = interpret(&p.content);
let old = interpret(old);
assert!(new.balanced && old.balanced, "q/Q balanced (seed {seed})");
assert_eq!(new.paints.len(), old.paints.len(), "seed {seed}");
for (i, (a, b)) in new.paints.iter().zip(&old.paints).enumerate() {
// Round joins leaking from polylines into later rectangles is the old bug (B7),
// compared separately in `join_does_not_leak_from_polylines`.
let mut a = a.clone();
let mut b = b.clone();
a.join.clear();
b.join.clear();
assert_eq!(a, b, "seed {seed}, painting op {i}");
}
style_new += new.style_ops;
style_old += old.style_ops;
}
}
// Sanity: the corpus really contains style changes, and the new writer emits fewer.
assert!(style_old > 1000, "corpus too small: {style_old}");
assert!(style_new < style_old, "{style_new} vs {style_old}");
Ok(())
}
#[test]
fn structure_is_valid_for_all_documents() -> Result<(), Box<dyn Error>> {
for seed in 1..=40u64 {
let (pdf, _) = corpus_doc(seed);
check_xref(&pdf)?;
parse_objects(&pdf); // every /Length matches its data
}
let empty = PdfDoc::new().finish();
check_xref(&empty)?;
parse_objects(&empty);
Ok(())
}
#[test]
fn images_decode_to_their_pixels() {
let pics = pictures();
let mut d = PdfDoc::new();
for _ in 0..2 {
let p = d.add_page(300.0, 300.0);
for (px, w, h) in &pics {
p.image_rgba(Rect::new(1.0, 1.0, 20.0, 20.0), px, [*w, *h]);
}
}
let pdf = d.finish();
let (objs, pages) = parse_pages(&pdf);
assert_eq!(pages.len(), 2);
for page in &pages {
assert_eq!(page.xobjects.len(), 3);
for ((px, w, h), (_, id)) in pics.iter().zip(&page.xobjects) {
let (dw, dh, rgb, alpha) = decode_image(&objs, *id);
assert_eq!((dw, dh), (*w, *h));
let want_rgb: Vec<u8> = px.chunks(4).flat_map(|c| [c[0], c[1], c[2]]).collect();
assert_eq!(rgb, want_rgb);
let want_alpha: Vec<u8> = px.chunks(4).map(|c| c[3]).collect();
match alpha {
Some(a) => assert_eq!(a, want_alpha),
None => assert!(want_alpha.iter().all(|a| *a == 255)),
}
}
}
}
/// O6: no alpha plane (and no SMask object) for an opaque image; a single translucent pixel
/// anywhere (first, middle, last) produces the full plane.
#[test]
fn opaque_images_have_no_soft_mask() {
let mut d = PdfDoc::new();
let p = d.add_page(100.0, 100.0);
p.image_rgba(Rect::new(0.0, 0.0, 10.0, 10.0), &[255; 4 * 12], [4, 3]);
assert!(p.images[0].alpha.is_none());
let pdf = d.finish();
assert!(find(&pdf, b"/SMask").is_none());
assert_eq!(pdf.windows(15).filter(|w| *w == b"/Subtype /Image").count(), 1);
for translucent in [0usize, 5, 11] {
let mut px = vec![255u8; 4 * 12];
px[translucent * 4 + 3] = 7;
let mut d = PdfDoc::new();
d.add_page(100.0, 100.0).image_rgba(Rect::new(0.0, 0.0, 10.0, 10.0), &px, [4, 3]);
let (objs, pages) = parse_pages(&d.finish());
let (_, _, _, alpha) = decode_image(&objs, pages[0].xobjects[0].1);
let mut want = vec![255u8; 12];
want[translucent] = 7;
assert_eq!(alpha, Some(want));
}
}
/// O4: the same picture on five pages is stored once (image + mask), by every page.
#[test]
fn identical_images_are_stored_once() -> Result<(), Box<dyn Error>> {
let pics = pictures();
let (px, w, h) = &pics[0]; // has alpha: image object + mask object
let mut d = PdfDoc::new();
for _ in 0..5 {
let p = d.add_page(200.0, 200.0);
p.image_rgba(Rect::new(1.0, 1.0, 20.0, 20.0), px, [*w, *h]);
p.image_rgba(Rect::new(30.0, 1.0, 20.0, 20.0), px, [*w, *h]); // twice on one page
}
let pdf = d.finish();
check_xref(&pdf)?;
assert_eq!(pdf.windows(15).filter(|w| *w == b"/Subtype /Image").count(), 2, "one image + one soft mask");
let (_, pages) = parse_pages(&pdf);
let ids: Vec<usize> = pages.iter().flat_map(|p| p.xobjects.iter().map(|x| x.1)).collect();
assert_eq!(ids.len(), 5, "one XObject entry per page");
assert!(ids.iter().all(|i| *i == ids[0]));
for p in &pages {
assert_eq!(p.content.matches(" Do").count(), 2);
}
Ok(())
}
/// O4: a picture that differs in one byte, or only in size, is not merged.
#[test]
fn different_images_are_not_merged() {
let a = vec![50u8; 4 * 4];
let mut b = a.clone();
b[5] = 51;
let mut d = PdfDoc::new();
let p = d.add_page(100.0, 100.0);
p.image_rgba(Rect::new(0.0, 0.0, 5.0, 5.0), &a, [4, 1]);
p.image_rgba(Rect::new(0.0, 0.0, 5.0, 5.0), &b, [4, 1]);
p.image_rgba(Rect::new(0.0, 0.0, 5.0, 5.0), &a, [2, 2]);
d.add_page(100.0, 100.0).image_rgba(Rect::new(0.0, 0.0, 5.0, 5.0), &a, [4, 1]);
let pdf = d.finish();
assert_eq!(pdf.windows(15).filter(|w| *w == b"/Subtype /Image").count(), 6, "three distinct pictures, each with a soft mask (alpha 50)");
}
/// O4: many distinct pictures on one page are looked up by hash, not by comparing each new
/// picture with every earlier one (which made dense picture exports quadratic).
#[test]
fn many_distinct_images_on_one_page_are_not_compared_pairwise() {
const N: u32 = 3000;
let mut d = PdfDoc::new();
let p = d.add_page(612.0, 792.0);
for i in 0..N {
let [a, b, c, _] = i.to_le_bytes();
p.image_rgba(Rect::new(0.0, 0.0, 1.0, 1.0), &[a, b, c, 255], [1, 1]);
}
// A repeat of the first picture still shares its XObject.
p.image_rgba(Rect::new(5.0, 5.0, 1.0, 1.0), &[0, 0, 0, 255], [1, 1]);
assert_eq!(p.images.len(), N as usize);
assert!(p.content.ends_with("/Im1 Do Q\n"), "the repeated picture reuses /Im1");
assert!(p.image_comparisons <= 1, "{} pixel comparisons for {N} distinct pictures", p.image_comparisons);
let pdf = d.finish();
assert_eq!(pdf.windows(15).filter(|w| *w == b"/Subtype /Image").count(), N as usize);
}
/// O4: equal hashes alone never merge images; the pixels are compared.
#[test]
fn hash_collisions_do_not_merge_images() {
let mut d = PdfDoc::new();
for v in [10u8, 20] {
d.add_page(100.0, 100.0).image_rgba(Rect::new(0.0, 0.0, 5.0, 5.0), &[v; 16], [2, 2]);
}
for p in 0..2 {
d.page_mut(p).unwrap().images[0].hash = 42; // forced collision
}
let (objs, pages) = parse_pages(&d.finish());
let ids: Vec<usize> = pages.iter().map(|p| p.xobjects[0].1).collect();
assert_ne!(ids[0], ids[1]);
assert_eq!(decode_image(&objs, ids[0]).2, vec![10; 12]);
assert_eq!(decode_image(&objs, ids[1]).2, vec![20; 12]);
}
/// S5: the colour component table equals the old float formatting for every byte.
#[test]
fn colour_components_match_the_float_formatting() {
for v in 0..=255u8 {
let mut s = String::new();
push_unit(&mut s, v);
assert_eq!(s, n(v as f32 / 255.0), "byte {v}");
}
let mut s = String::new();
push_rgb(&mut s, [255, 128, 0], true);
assert_eq!(s, "1 0.502 0 RG");
}
/// S3: the obsolete /ProcSet entry is gone.
#[test]
fn no_procset() {
let (pdf, _) = corpus_doc(3);
assert!(find(&pdf, b"/ProcSet").is_none());
}
/// O3: a style that is already in effect is not emitted again.
#[test]
fn unchanged_style_is_emitted_once() {
let mut d = PdfDoc::new();
let p = d.add_page(200.0, 200.0);
for i in 0..50 {
p.rect_stroke(i as f32, 0.0, 5.0, 5.0, 1.5, [10, 20, 30]);
}
for i in 0..50 {
p.stroke_line_dashed(Point::new(0.0, i as f32), Point::new(9.0, 9.0), 1.5, [10, 20, 30], &[2.0, 1.0]);
}
for _ in 0..50 {
p.fill_rect(0.0, 0.0, 4.0, 4.0, [1, 2, 3]);
p.text(5.0, 5.0, 9.0, Font::Helvetica, [1, 2, 3], "x");
}
let s = interpret(&p.content);
assert_eq!(p.content.matches(" RG\n").count(), 1);
assert_eq!(p.content.matches(" rg\n").count(), 1);
assert_eq!(p.content.matches(" w\n").count(), 1);
assert_eq!(p.content.matches(" d\n").count(), 2, "solid, then [2 1]");
assert_eq!(s.style_ops, 5);
assert_eq!(s.paints.len(), 50 * 4);
}
/// O3: the tracked style follows q/Q, and a new page starts from scratch.
#[test]
fn style_tracking_follows_save_restore_and_pages() {
let red = [255, 0, 0];
let blue = [0, 0, 255];
let mut d = PdfDoc::new();
let p = d.add_page(100.0, 100.0);
p.fill_rect(0.0, 0.0, 5.0, 5.0, red);
p.save();
p.fill_rect(0.0, 0.0, 5.0, 5.0, red); // still red inside: nothing to emit
p.fill_rect(0.0, 0.0, 5.0, 5.0, blue);
p.restore();
p.fill_rect(0.0, 0.0, 5.0, 5.0, red); // red again after Q without a new operator
p.fill_rect(0.0, 0.0, 5.0, 5.0, blue); // but blue must be re-emitted
let fills: Vec<String> = interpret(&p.content).paints.iter().map(|p| p.fill.clone()).collect();
assert_eq!(fills, ["1 0 0", "1 0 0", "0 0 1", "1 0 0", "0 0 1"]);
assert_eq!(p.content.matches(" rg\n").count(), 3, "red once, blue twice");
let p2 = d.add_page(100.0, 100.0);
p2.fill_rect(0.0, 0.0, 5.0, 5.0, red);
assert_eq!(p2.content.matches(" rg\n").count(), 1, "a new page re-emits the colour");
}
/// B6: only 28 nested saves are written, the matching extra restores are no-ops.
#[test]
fn save_nesting_is_limited_to_28() -> Result<(), Box<dyn Error>> {
let mut d = PdfDoc::new();
let p = d.add_page(100.0, 100.0);
for _ in 0..40 {
p.save();
}
p.fill_rect(0.0, 0.0, 5.0, 5.0, [1, 1, 1]);
assert_eq!(p.content.matches("q\n").count(), 28);
// The 12 restores that match refused saves do nothing...
for _ in 0..12 {
p.restore();
}
assert_eq!(p.content.matches("Q\n").count(), 0);
assert_eq!(interpret(&p.content).max_depth, 28);
// ...the next 28 undo the accepted ones, and further restores are ignored.
for _ in 0..40 {
p.restore();
}
assert_eq!(p.content.matches("Q\n").count(), 28);
let i = interpret(&p.content);
assert!(i.balanced);
// An image needs its own q: dropped at the limit, drawn otherwise.
for _ in 0..28 {
p.save();
}
p.image_rgba(Rect::new(0.0, 0.0, 5.0, 5.0), &[9; 16], [2, 2]);
assert!(!p.content.contains(" Do"));
p.restore();
p.image_rgba(Rect::new(0.0, 0.0, 5.0, 5.0), &[9; 16], [2, 2]);
assert!(p.content.contains(" Do"));
let pdf = d.finish();
check_xref(&pdf)?;
Ok(())
}
/// B6: `finish()` closes what is still open, without exceeding the limit.
#[test]
fn finish_closes_open_saves() -> Result<(), Box<dyn Error>> {
let mut d = PdfDoc::new();
let p = d.add_page(100.0, 100.0);
for _ in 0..33 {
p.save();
}
let (_, pages) = parse_pages(&d.finish());
let i = interpret(&pages[0].content);
assert!(i.balanced);
assert_eq!(i.qs, 28);
Ok(())
}
/// B7: round joins set for a polyline do not leak into later rectangles, lines and circles.
#[test]
fn join_does_not_leak_from_polylines() {
let mut d = PdfDoc::new();
let p = d.add_page(200.0, 200.0);
let pts = [[0.0, 0.0], [10.0, 0.0], [10.0, 10.0]];
p.rect_stroke(0.0, 0.0, 5.0, 5.0, 1.0, [0, 0, 0]);
p.polyline(&pts, 1.0, [0, 0, 0], &[]);
p.polyline(&pts, 1.0, [0, 0, 0], &[]);
p.rect_stroke(0.0, 0.0, 5.0, 5.0, 1.0, [0, 0, 0]);
p.stroke_line(0.0, 0.0, 9.0, 9.0, 1.0, [0, 0, 0]);
p.polygon(&pts, None, Some(([0, 0, 0], 1.0)));
p.circle(50.0, 50.0, 5.0, None, Some(([0, 0, 0], 1.0)));
let joins: Vec<String> = interpret(&p.content).paints.iter().map(|p| p.join.clone()).collect();
assert_eq!(joins, ["0", "1", "1", "0", "0", "1", "0"]);
assert_eq!(p.content.matches(" j\n").count(), 4, "1, 0, 1, 0: only on change");
}
fn page_content(d: &PdfDoc) -> &str {
&d.pages[0].content
}
@@ -210,7 +765,10 @@ fn all_zero_dash_is_drawn_solid() {
p.polyline(&[[0.0, 0.0], [5.0, 5.0]], 1.0, [0, 0, 0], &[0.0, 0.0001, f32::NAN]);
p.stroke_line_dashed(Point::new(0.0, 0.0), Point::new(50.0, 0.0), 1.0, [0, 0, 0], &[3.0, 2.0]);
let c = page_content(&d);
assert_eq!(c.matches("[] 0 d").count(), 2);
// The writer emits style operators only on change: the two solid lines share one `[] 0 d`.
let dashes: Vec<String> = interpret(c).paints.iter().map(|p| p.dash.clone()).collect();
assert_eq!(dashes, ["", "", "3 2"]);
assert_eq!(c.matches("[] 0 d").count(), 1);
assert!(!c.contains("[0 0]"));
assert!(c.contains("[3 2] 0 d"));
}
@@ -224,7 +782,9 @@ fn negative_and_nan_stroke_widths_become_zero() {
p.polygon(&[[0.0, 0.0], [5.0, 0.0], [5.0, 5.0]], None, Some(([0, 0, 0], -1.0)));
p.circle(20.0, 20.0, 5.0, None, Some(([0, 0, 0], f32::NEG_INFINITY)));
let c = page_content(&d);
assert_eq!(c.matches("\n0 w ").count(), 4);
// Style operators are emitted only on change, so judge the width in effect for each stroke.
let widths: Vec<String> = interpret(c).paints.iter().map(|p| p.width.clone()).collect();
assert_eq!(widths, ["0", "0", "0", "0"]);
assert!(!c.contains("-1 w") && !c.contains("-3 w"));
}
@@ -333,6 +893,18 @@ fn cjk_text_is_not_replaced_by_question_marks() -> Result<(), Box<dyn Error>> {
Ok(())
}
#[test]
fn many_alternating_runs_share_one_text_matrix() {
let mut doc = PdfDoc::new();
let p = doc.add_page(612.0, 792.0);
p.text(10.0, 20.0, 10.0, Font::Helvetica, [0, 0, 0], &"a\u{4e2d}".repeat(2000));
let pdf = doc.finish();
let count = |needle: &[u8]| pdf.windows(needle.len()).filter(|w| *w == needle).count();
assert_eq!(count(b" Tm\n"), 1, "the position is set once, before the first run");
assert_eq!(count(b" Tf\n"), 4000, "every run selects its font");
assert_eq!(count(b" Tj\n"), 4000);
}
#[test]
fn symbol_font_draws_maths_in_its_own_encoding() {
let mut doc = PdfDoc::new();
+351
View File
@@ -0,0 +1,351 @@
//! The content-stream emission of the writer before the cleanup (a verbatim copy of the old
//! per-operator formatting, which re-emitted the full style for every operator, had no `q` limit
//! and left `1 j` set after polylines), plus an interpreter that reduces a content stream to the
//! effective graphics state of every painting operator.
use super::super::*;
fn old_rgb_op(c: Rgb, stroke: bool) -> String {
let f = |v: u8| n(v as f32 / 255.0);
format!("{} {} {} {}", f(c[0]), f(c[1]), f(c[2]), if stroke { "RG" } else { "rg" })
}
/// The old content writer.
#[derive(Default)]
pub struct OldPage {
pub height: f32,
pub content: String,
depth: u32,
images: usize,
}
impl OldPage {
pub fn new(height: f32) -> OldPage {
OldPage { height, ..Default::default() }
}
fn y(&self, y: f32) -> f32 {
self.height - y
}
fn op(&mut self, s: &str) {
self.content.push_str(s);
self.content.push('\n');
}
pub fn save(&mut self) {
self.depth += 1;
self.content.push_str("q\n");
}
pub fn restore(&mut self) {
if self.depth > 0 {
self.depth -= 1;
self.content.push_str("Q\n");
}
}
pub fn clip_rect(&mut self, x: f32, y: f32, w: f32, h: f32) {
let s = format!("{} {} {} {} re W n", n(x), n(self.y(y + h)), n(w.max(0.0)), n(h.max(0.0)));
self.op(&s);
}
pub fn fill_rect(&mut self, x: f32, y: f32, w: f32, h: f32, color: Rgb) {
if w <= 0.0 || h <= 0.0 {
return;
}
let s = format!("{}\n{} {} {} {} re f", old_rgb_op(color, false), n(x), n(self.y(y + h)), n(w), n(h));
self.op(&s);
}
pub fn rect_stroke(&mut self, x: f32, y: f32, w: f32, h: f32, width: f32, color: Rgb) {
let s = format!("{}\n{} w [] 0 d\n{} {} {} {} re S", old_rgb_op(color, true), n(width), n(x), n(self.y(y + h)), n(w), n(h));
self.op(&s);
}
pub fn stroke_line_dashed(&mut self, start: Point, end: Point, width: f32, color: Rgb, dash: &[f32]) {
let (x0, y0, x1, y1) = (start.x, start.y, end.x, end.y);
let d: Vec<String> = dash.iter().take(8).map(|v| n(v.max(0.0))).collect();
let s = format!(
"{}\n{} w [{}] 0 d\n{} {} m {} {} l S",
old_rgb_op(color, true),
n(width),
d.join(" "),
n(x0),
n(self.y(y0)),
n(x1),
n(self.y(y1))
);
self.op(&s);
}
pub fn polyline(&mut self, pts: &[[f32; 2]], width: f32, color: Rgb, dash: &[f32]) {
if pts.len() < 2 {
return;
}
let d: Vec<String> = dash.iter().take(8).map(|v| n(v.max(0.0))).collect();
let mut s = format!("{}\n{} w [{}] 0 d 1 j\n", old_rgb_op(color, true), n(width), d.join(" "));
self.path(&mut s, pts, false);
s.push_str(" S");
self.op(&s);
}
fn path(&self, s: &mut String, pts: &[[f32; 2]], close: bool) {
for (i, p) in pts.iter().enumerate() {
let _ = write!(s, "{} {} {} ", n(p[0]), n(self.y(p[1])), if i == 0 { "m" } else { "l" });
}
if close {
s.push('h');
}
}
pub fn polygon(&mut self, pts: &[[f32; 2]], fill: Option<Rgb>, stroke: Option<(Rgb, f32)>) {
if pts.len() < 2 || (fill.is_none() && stroke.is_none()) {
return;
}
let mut s = String::new();
if let Some(f) = fill {
s.push_str(&old_rgb_op(f, false));
s.push('\n');
}
if let Some((c, w)) = stroke {
let _ = writeln!(s, "{}\n{} w [] 0 d 1 j", old_rgb_op(c, true), n(w));
}
self.path(&mut s, pts, true);
s.push_str(match (fill.is_some(), stroke.is_some()) {
(true, true) => " B",
(true, false) => " f",
_ => " S",
});
self.op(&s);
}
pub fn circle(&mut self, cx: f32, cy: f32, r: f32, fill: Option<Rgb>, stroke: Option<(Rgb, f32)>) {
if r <= 0.0 || (fill.is_none() && stroke.is_none()) {
return;
}
let y = self.y(cy);
let mut s = String::new();
if let Some(f) = fill {
s.push_str(&old_rgb_op(f, false));
s.push('\n');
}
if let Some((c, w)) = stroke {
let _ = writeln!(s, "{}\n{} w [] 0 d", old_rgb_op(c, true), n(w));
}
const K: f32 = 0.552_284_8;
let k = r * K;
let _ = write!(
s,
"{} {} m {} {} {} {} {} {} c {} {} {} {} {} {} c {} {} {} {} {} {} c {} {} {} {} {} {} c h",
n(cx + r),
n(y),
n(cx + r),
n(y + k),
n(cx + k),
n(y + r),
n(cx),
n(y + r),
n(cx - k),
n(y + r),
n(cx - r),
n(y + k),
n(cx - r),
n(y),
n(cx - r),
n(y - k),
n(cx - k),
n(y - r),
n(cx),
n(y - r),
n(cx + k),
n(y - r),
n(cx + r),
n(y - k),
n(cx + r),
n(y)
);
s.push_str(match (fill.is_some(), stroke.is_some()) {
(true, true) => " B",
(true, false) => " f",
_ => " S",
});
self.op(&s);
}
pub fn text_rotated(&mut self, origin: Point, size: f32, font: Font, color: Rgb, text: &str, angle: f32) {
let (x, y) = (origin.x, origin.y);
let fi = font.index();
let (sn, cs) = angle.sin_cos();
let s = format!(
"BT\n{}\n/F{} {} Tf\n{} {} {} {} {} {} Tm\n{} Tj\nET",
old_rgb_op(color, false),
fi + 1,
n(size),
n(cs),
n(-sn),
n(sn),
n(cs),
n(x),
n(self.y(y)),
pdf_string(text)
);
self.op(&s);
}
pub fn image(&mut self, x: f32, y: f32, w: f32, h: f32) {
self.images += 1;
let s = format!("q {} 0 0 {} {} {} cm /Im{} Do Q", n(w), n(h), n(x), n(self.y(y + h)), self.images);
self.op(&s);
}
}
/// The effective state at one painting operator. Fields that cannot influence the operator
/// (the stroke state of a fill, the fill of a stroke) are left empty.
#[derive(Clone, Debug, PartialEq, Default)]
pub struct Paint {
pub op: String,
/// Path construction / text operands since the previous painting operator.
pub geometry: String,
pub fill: String,
pub stroke: String,
pub width: String,
pub dash: String,
pub join: String,
/// Clip paths in effect (innermost last).
pub clip: Vec<String>,
}
#[derive(Default, Debug)]
pub struct Interp {
pub paints: Vec<Paint>,
pub max_depth: usize,
/// Number of `q` operators.
pub qs: usize,
/// Number of colour/width/dash/join operators.
pub style_ops: usize,
pub balanced: bool,
}
fn tokens(content: &str) -> Vec<String> {
let mut out = Vec::new();
let mut cur = String::new();
let mut chars = content.chars();
while let Some(c) = chars.next() {
match c {
'(' => {
if !cur.is_empty() {
out.push(std::mem::take(&mut cur));
}
let mut s = String::from("(");
while let Some(d) = chars.next() {
s.push(d);
if d == '\\' {
if let Some(e) = chars.next() {
s.push(e);
}
} else if d == ')' {
break;
}
}
out.push(s);
}
'[' | ']' => {
if !cur.is_empty() {
out.push(std::mem::take(&mut cur));
}
out.push(c.to_string());
}
c if c.is_whitespace() => {
if !cur.is_empty() {
out.push(std::mem::take(&mut cur));
}
}
c => cur.push(c),
}
}
if !cur.is_empty() {
out.push(cur);
}
out
}
#[derive(Clone)]
struct State {
fill: String,
stroke: String,
width: String,
dash: String,
join: String,
clip: Vec<String>,
}
/// Interprets a content stream: tracks q/Q, colours, width, dash, join and clips.
pub fn interpret(content: &str) -> Interp {
let mut st = State { fill: "0 0 0".into(), stroke: "0 0 0".into(), width: "1".into(), dash: String::new(), join: "0".into(), clip: Vec::new() };
let mut stack: Vec<State> = Vec::new();
let mut r = Interp { balanced: true, ..Default::default() };
let mut operands: Vec<String> = Vec::new();
let mut path: Vec<String> = Vec::new();
let mut pending_clip = false;
for t in tokens(content) {
let is_op = t.chars().next().is_some_and(|c| c.is_ascii_alphabetic() || c == '*') && !t.starts_with('/');
if !is_op {
operands.push(t);
continue;
}
let ops = std::mem::take(&mut operands);
match t.as_str() {
"q" => {
stack.push(st.clone());
r.qs += 1;
r.max_depth = r.max_depth.max(stack.len());
}
"Q" => match stack.pop() {
Some(s) => st = s,
None => r.balanced = false,
},
"rg" => {
st.fill = ops.join(" ");
r.style_ops += 1;
}
"RG" => {
st.stroke = ops.join(" ");
r.style_ops += 1;
}
"w" => {
st.width = ops.join(" ");
r.style_ops += 1;
}
"d" => {
// `[a b] 0` -> `a b`
st.dash = ops.iter().take_while(|o| *o != "]").skip(1).cloned().collect::<Vec<_>>().join(" ");
r.style_ops += 1;
}
"j" => {
st.join = ops.join(" ");
r.style_ops += 1;
}
"re" | "m" | "l" | "c" | "h" | "cm" | "Tf" | "Tm" => {
path.push(format!("{} {}", ops.join(" "), t));
}
"BT" | "ET" => {}
"W" => pending_clip = true,
"n" => {
if pending_clip {
st.clip.push(path.join(" "));
pending_clip = false;
}
path.clear();
}
"Tj" => {
path.push(format!("{} {}", ops.join(" "), t));
r.paints.push(Paint { op: t.clone(), geometry: path.join(" "), fill: st.fill.clone(), clip: st.clip.clone(), ..Default::default() });
path.clear();
}
"f" | "S" | "B" | "Do" => {
let mut p = Paint { op: t.clone(), geometry: path.join(" "), clip: st.clip.clone(), ..Default::default() };
if t == "f" || t == "B" {
p.fill = st.fill.clone();
}
if t == "S" || t == "B" {
p.stroke = st.stroke.clone();
p.width = st.width.clone();
p.dash = st.dash.clone();
p.join = st.join.clone();
}
r.paints.push(p);
path.clear();
}
_ => {}
}
}
r.balanced &= stack.is_empty();
r
}
+44 -1
View File
@@ -268,12 +268,38 @@ const TIOCLINUX: u64 = libc::TIOCLINUX as u64;
const IOC_WRITE: u64 = 0x4000_0000;
/// Changes a directory, though its number says it only reads, as the kernel's headers have it.
const FS_IOC_SET_ENCRYPTION_POLICY: u64 = 0x800C_6613;
#[allow(clippy::unnecessary_cast)]
const F_SETOWN: u64 = libc::F_SETOWN as u64;
#[allow(clippy::unnecessary_cast)]
const F_SETFL: u64 = libc::F_SETFL as u64;
#[allow(clippy::unnecessary_cast)]
const O_ASYNC: u64 = libc::O_ASYNC as u64;
/// The two `fcntl` commands of the same family that the kernel's headers name and `libc` doesn't;
/// the numbers are the same on every architecture (`asm-generic/fcntl.h`).
const F_SETOWN_EX: u64 = 15;
const F_SETSIG: u64 = 10;
/// Socket ioctls that do what those commands do: `FIOSETOWN` and `SIOCSPGRP` name the process to
/// notify, and `FIOASYNC` switches the notification on. Their requests carry no data-direction
/// bit, so the [`IOC_WRITE`] rule doesn't name them. `libc` doesn't name them; the numbers are
/// the kernel's, the same on every architecture (`asm-generic/sockios.h`).
const FIOSETOWN: u64 = 0x8901;
const SIOCSPGRP: u64 = 0x8902;
const FIOASYNC: u64 = 0x5452;
/// What the main filter refuses with `EPERM`: the calls above, and these when their arguments match.
/// - `clone` without `CLONE_THREAD`: a new process.
/// - A signal to any process but `me`, whose ID `exec` keeps. Landlock's signal scope needs a newer
/// kernel than the rest (ABI 6), and `kill(-1, 9)` must not get through on an older one.
/// - A change to the limits, priority or scheduling of any process but `me`.
/// - The `fcntl` commands that arrange asynchronous notification: naming the process to notify
/// (`F_SETOWN`, `F_SETOWN_EX`), choosing its signal (`F_SETSIG`), and `F_SETFL` when it sets
/// `O_ASYNC`, the switch without which nothing is sent. None is a signal call, and the only rule
/// that would cover them is Landlock's signal scope, which needs ABI 6, newer than a launch
/// requires; without these a confined program could have the kernel signal any process of its
/// user. `F_SETFL` itself still works, as `O_NONBLOCK` comes from it. A socket answers the same
/// arrangement through ioctls — `FIOSETOWN` and `SIOCSPGRP` to name the process, `FIOASYNC` to
/// switch the notification on — and those requests carry no data-direction bit, so they are
/// named alongside the others.
/// - `TIOCSTI` and `TIOCLINUX`, which type into a terminal the program inherited, as when
/// `gridcraft-cli sandbox-check` runs a program from a shell.
/// - An `ioctl` that hands the kernel data ([`IOC_WRITE`]), as those that change a file's flags or
@@ -307,10 +333,22 @@ fn main_rules(me: u32) -> Result<BTreeMap<i64, Vec<SeccompRule>>> {
vec![
rule(vec![word(1, SeccompCmpOp::Eq, TIOCSTI)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, TIOCLINUX)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, FIOSETOWN)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, SIOCSPGRP)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, FIOASYNC)?])?,
rule(vec![word(1, SeccompCmpOp::MaskedEq(IOC_WRITE), IOC_WRITE)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, FS_IOC_SET_ENCRYPTION_POLICY)?])?,
],
);
rules.insert(
libc::SYS_fcntl,
vec![
rule(vec![word(1, SeccompCmpOp::Eq, F_SETOWN)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, F_SETOWN_EX)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, F_SETSIG)?])?,
rule(vec![word(1, SeccompCmpOp::Eq, F_SETFL)?, word(2, SeccompCmpOp::MaskedEq(O_ASYNC), O_ASYNC)?])?,
],
);
Ok(rules)
}
@@ -378,6 +416,9 @@ mod tests {
#[test]
fn the_calls_that_reach_another_process_have_rules() {
let rules = main_rules(4242).unwrap();
// `fcntl`'s notification commands: naming the owner to notify, both ways, choosing its
// signal, and the switch itself.
assert_eq!(rules.get(&libc::SYS_fcntl).map(Vec::len), Some(4), "the asynchronous-notification commands");
for nr in [
libc::SYS_kill,
libc::SYS_tkill,
@@ -415,7 +456,9 @@ mod tests {
] {
assert_eq!(rules.get(&nr).map(Vec::len), Some(0), "{nr} is refused outright");
}
assert_eq!(rules.get(&libc::SYS_ioctl).map(Vec::len), Some(4));
// The terminal rules, the socket notification ioctls (`FIOSETOWN`, `SIOCSPGRP`,
// `FIOASYNC`), the data-carrying rule and the encryption rule.
assert_eq!(rules.get(&libc::SYS_ioctl).map(Vec::len), Some(7));
}
#[test]
+25
View File
@@ -44,6 +44,8 @@ pub fn enforcement(canary: &Path, exe: &Path, target: u32) -> Vec<Check> {
Check { name: "TCP connection", outcome: denied(tcp_probe()) },
Check { name: "start a process", outcome: spawn_probe(exe) },
Check { name: "signal another process", outcome: signal_probe(target) },
#[cfg(target_os = "linux")]
Check { name: "switch a descriptor's notifications on", outcome: async_probe() },
]
}
@@ -128,6 +130,20 @@ fn write_probe(dir: &Path) -> Outcome {
denied(r.map(drop))
}
/// `O_ASYNC` on a descriptor of the program's own: the switch that has the kernel signal the
/// descriptor's owner when its state changes. The owner may be a process the program didn't name, a
/// standard stream the parent made, so on Linux the filter refuses setting it, the same way it
/// refuses the commands that name another owner. Naming one can't be asked here without `fcntl`'s
/// variable-argument form, which no safe API wraps; the rules refuse those commands, and the
/// confined-program test covers them.
#[cfg(target_os = "linux")]
fn async_probe() -> Outcome {
let r = nix::unistd::pipe().map_err(std::io::Error::from).and_then(|(_, write)| {
nix::fcntl::fcntl(write, nix::fcntl::FcntlArg::F_SETFL(nix::fcntl::OFlag::O_ASYNC)).map(drop).map_err(std::io::Error::from)
});
denied(r)
}
fn tcp_probe() -> std::io::Result<()> {
TcpStream::connect_timeout(&SocketAddr::from(TCP_TARGET), Duration::from_secs(2)).map(drop)
}
@@ -216,6 +232,15 @@ mod tests {
}
}
/// Unconfined, the notification probe reports what the system allows; confined, the launch
/// check needs it `Denied`. Either way it comes out of the same three calls, so this proves
/// the probe runs and classifies rather than failing outright.
#[cfg(target_os = "linux")]
#[test]
fn the_notification_probe_classifies() {
assert!(matches!(async_probe(), Outcome::Allowed(_)), "unconfined the switch can be set");
}
#[test]
fn the_checks_on_the_canary_change_nothing_where_they_are_allowed() {
let canary = crate::Canary::create().unwrap();
+1 -1
View File
@@ -11,7 +11,7 @@ workspace = true
[dependencies]
# The shared capture slot (`craft_ui::capture`) comes from the one craft-libs revision every
# craft-libs dependency uses (see the root Cargo.toml).
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "119415f46f601e20b9bc6b1b4d6979ceda86d88e" }
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "aa84415bae3b765fef6b7ef42404f66ad1cff9d9" }
gridcraft-engine = { workspace = true }
gridcraft-chart = { workspace = true }
# `monochrome_emoji_fonts` restores the Noto Emoji and emoji-icon faces that 0.37 moved behind
+13
View File
@@ -71,12 +71,15 @@ impl SheetApp {
capture_geometry: None,
capture_geometry_from_raw: false,
selected_object: None,
chart_cache: Default::default(),
clipboard_text_at_copy: None,
cut_text: None,
tab_run: None,
macros: macro_ui::MacroUi::default(),
started: now_ms(),
allow_close: false,
window_controls: false,
viewport_commands: Vec::new(),
bring_forward: false,
open_switch_windows: false,
ui_window: None,
@@ -114,6 +117,7 @@ impl SheetApp {
// Floating pictures' textures go with their workbook, even when no sheet is painted.
let open: Vec<u64> = self.session.documents().iter().map(|d| d.uid).collect();
floating_pictures::forget_closed(ctx, &open);
self.chart_cache.get_mut().unwrap_or_else(std::sync::PoisonError::into_inner).retain_docs(open.iter().copied());
self.apply_font_state(ctx);
if let Some(answer) = self.desktop_theme.take_change() {
self.desktop_dark = Some(answer);
@@ -250,6 +254,9 @@ impl SheetApp {
macro_ui::question_box(self, &ctx);
widgets::message_box(self, &ctx);
widgets::toast(self, &ctx);
// Last, so its resize cursors win over the widgets under the window edges.
ribbon::window_frame(self, &ctx);
self.flush_viewport_commands(&ctx);
// Requests are admitted in `logic`, for the root viewport, so only its pass issues them.
if ctx.viewport_id() == egui::ViewportId::ROOT {
control::issue_screenshots(self, &ctx);
@@ -302,6 +309,12 @@ impl SheetApp {
}
sel.active.a1()
}
pub(crate) fn flush_viewport_commands(&mut self, ctx: &egui::Context) {
for command in self.viewport_commands.drain(..) {
ctx.send_viewport_cmd(command);
}
}
}
#[cfg(test)]
+20
View File
@@ -163,6 +163,26 @@ impl SheetApp {
/// UI-only commands (dialogs, view toggles that aren't saved in the file).
fn run_ui_command(&mut self, id: &str, p: &Json) -> Option<Result<Json, String>> {
let r = match id {
"ui.window.minimize" => {
self.viewport_commands.push(egui::ViewportCommand::Minimized(true));
Ok(Json::Null)
}
"ui.window.maximize" => {
let on = p.get("on").and_then(Json::as_bool).unwrap_or(true);
self.viewport_commands.push(egui::ViewportCommand::Maximized(on));
Ok(Json::Null)
}
"ui.window.close" => {
self.viewport_commands.push(egui::ViewportCommand::Close);
Ok(Json::Null)
}
"ui.window.beginResize" => {
let Some(direction) = p.get("direction").and_then(Json::as_str).and_then(crate::parse_resize_direction) else {
return Some(Err("direction must be north, south, east, west, northeast, northwest, southeast, or southwest".into()));
};
self.viewport_commands.push(egui::ViewportCommand::BeginResize(direction));
Ok(Json::Null)
}
"app.checkForUpdates" | "help.checkForUpdates" => {
self.check_for_updates();
Ok(Json::Null)
+63 -1
View File
@@ -409,7 +409,11 @@ impl SheetApp {
/// Enter/Tab move inside a multi-cell selection, otherwise to the next cell.
pub fn move_after_enter(&mut self, dr: i64, dc: i64) {
let multi = self.session.active().is_some_and(|d| !d.selection.is_single_cell());
// A selection that is just one merged cell moves like one cell (Enter leaves it).
let multi = self
.session
.active()
.is_some_and(|d| !d.selection.is_single_cell() && !d.wb.active().is_some_and(|sh| d.selection.is_one_merged_cell(sh)));
// Any other move (arrows, clicks, another sheet) leaves the selection off the cell the
// last Tab landed on, which ends the run.
let here = self.session.active().map(|d| (d.uid, d.wb.active_sheet(), d.selection.active));
@@ -508,3 +512,61 @@ mod protected_cell_tests {
assert!(app.run("file.saveBytes", json!({"format": "xlsx"})).is_ok());
}
}
#[cfg(test)]
mod merged_cell_enter_tests {
use super::*;
use gridcraft_engine::Session;
#[test]
fn enter_and_tab_leave_a_selected_merged_cell_and_keep_the_tab_run() {
let mut session = Session::new();
session.new_workbook();
let mut app = SheetApp::new(session, Default::default());
app.run("selection.set", json!({"range": "B2:C3"})).unwrap();
app.run("home.mergeCells", json!({})).unwrap();
let sel = |app: &SheetApp| app.session.active().map(|d| d.selection.a1()).unwrap_or_default();
for ((dr, dc), want) in [((1, 0), "B4"), ((-1, 0), "B1"), ((0, 1), "D2"), ((0, -1), "A2")] {
app.run("selection.set", json!({"cell": "B2"})).unwrap();
assert_eq!(sel(&app), "B2:C3");
app.move_after_enter(dr, dc);
assert_eq!(sel(&app), want, "Enter/Tab ({dr},{dc}) from the merged cell B2:C3");
}
// A Tab run through the merged cell goes on past it: Enter then goes back to the column
// the run started in, on the next row.
app.tab_run = None; // the Shift+Tab above ended on A2 and started a run there
app.run("selection.set", json!({"cell": "A2"})).unwrap();
app.move_after_enter(0, 1);
assert_eq!(sel(&app), "B2:C3");
app.move_after_enter(0, 1);
assert_eq!(sel(&app), "D2");
app.move_after_enter(1, 0);
assert_eq!(sel(&app), "A3");
// Enter straight from the merged cell in a Tab run leaves it below its bottom edge and
// still goes back to the run's start column.
app.tab_run = None;
app.run("selection.set", json!({"cell": "A2"})).unwrap();
app.move_after_enter(0, 1);
assert_eq!(sel(&app), "B2:C3");
app.move_after_enter(1, 0);
assert_eq!(sel(&app), "A4");
}
#[test]
fn enter_from_a_merged_cell_in_a_tab_run_skips_a_hidden_row_and_keeps_the_column() {
let mut session = Session::new();
session.new_workbook();
let mut app = SheetApp::new(session, Default::default());
app.run("selection.set", json!({"range": "D2:E3"})).unwrap();
app.run("home.mergeCells", json!({})).unwrap();
app.run("selection.set", json!({"range": "A4"})).unwrap();
app.run("home.hideRows", json!({})).unwrap();
let sel = |app: &SheetApp| app.session.active().map(|d| d.selection.a1()).unwrap_or_default();
app.run("selection.set", json!({"cell": "C2"})).unwrap();
app.move_after_enter(0, 1);
assert_eq!(sel(&app), "D2:E3");
// Row 4 is hidden: Enter lands on C5, the run's column, not B5.
app.move_after_enter(1, 0);
assert_eq!(sel(&app), "C5");
}
}
+3
View File
@@ -65,6 +65,7 @@ impl SheetApp {
} else if self.fonts_rebind {
// A font-set change made last frame is active now.
self.fonts_rebind = false;
self.chart_cache.get_mut().unwrap_or_else(std::sync::PoisonError::into_inner).clear();
} else if !defer_font_change && !self.prepared_fonts && fonts::scan_done() {
self.rebind_workbook_fonts(ctx);
}
@@ -75,6 +76,8 @@ impl SheetApp {
fn start_font_setup(&mut self, ctx: &egui::Context) {
if self.fonts_set {
self.fonts_ready = true;
// The new set is active now: chart text was measured with the old one.
self.chart_cache.get_mut().unwrap_or_else(std::sync::PoisonError::into_inner).clear();
return;
}
if self.prepared_fonts {
+495
View File
@@ -0,0 +1,495 @@
//! Rendered charts kept between frames.
//!
//! Resolving a chart's series and laying it out is O(points). Doing it for every visible chart on
//! every frame kept the UI thread busy while scrolling a workbook with a few big charts (#1139,
//! part of #1098). The primitives a chart renders to depend on the workbook, the chart's place in
//! it, the zoom, the screen's pixels-per-point and the fonts text is measured with, so those are
//! what an entry is checked against.
//!
//! **Why "the workbook is unchanged" can be known without a change counter.** A document's
//! workbook is an `Arc<Workbook>` and its content is only reachable for writing through
//! `Arc::make_mut` (or by replacing the `Arc`), because the model has no interior mutability and
//! the code base forbids `unsafe`. While a `Weak` to the allocation exists, `make_mut` never
//! mutates it in place: it clones (other strong owners, e.g. an undo snapshot) or moves the value
//! to a new allocation (the `Weak` is cut loose). So if the document's current `Arc` still points
//! at the allocation the cache holds a `Weak` to, nothing was written since the entries were
//! built, however the write would have been made (command, undo, redo, recalculation, file load,
//! a direct edit from the UI, a future code path). The `Weak` also keeps the allocation from being
//! freed and reused, so a pointer match cannot be a stranger that landed on the same address.
use std::collections::HashMap;
use std::sync::{Arc, Weak};
use gridcraft_chart::Prim;
use gridcraft_engine::model::Workbook;
/// Most charts kept, over all documents.
const MAX_ENTRIES: usize = 64;
/// Most memory kept (estimated from the primitives), over all documents.
const MAX_BYTES: usize = 48 << 20;
struct Entry {
/// Zoom and pixels-per-point the primitives were measured for (`f32::to_bits`).
z: u32,
ppp: u32,
prims: Arc<[Prim]>,
bytes: usize,
/// Tick of the last use; the smallest is evicted first.
used: u64,
}
struct DocCharts {
/// The workbook the entries were built from.
wb: Weak<Workbook>,
/// (sheet index, index in the sheet's `charts`). The workbook is unchanged while the entries
/// live, so the pair names one chart.
charts: HashMap<(usize, usize), Entry>,
}
#[derive(Default)]
pub(crate) struct ChartCache {
docs: HashMap<u64, DocCharts>,
tick: u64,
bytes: usize,
/// Charts built (resolved and rendered) so far; for tests.
#[cfg(test)]
pub(crate) builds: usize,
}
fn prim_bytes(p: &Prim) -> usize {
std::mem::size_of::<Prim>()
+ match p {
Prim::Line { pts, .. } | Prim::Polygon { pts, .. } => pts.len().saturating_mul(std::mem::size_of::<[f32; 2]>()),
Prim::Text { text, .. } => text.len(),
_ => 0,
}
}
impl ChartCache {
/// The primitives of chart `index` of `sheet` in document `uid` whose workbook is `wb`, for
/// zoom and pixels per point `(z, ppp)`. `build` renders the chart when there is no valid entry.
pub(crate) fn get(
&mut self,
uid: u64,
wb: &Arc<Workbook>,
sheet: usize,
index: usize,
(z, ppp): (f32, f32),
build: impl FnOnce() -> Vec<Prim>,
) -> Arc<[Prim]> {
self.tick = self.tick.wrapping_add(1);
let tick = self.tick;
let doc = self.docs.entry(uid).or_insert_with(|| DocCharts { wb: Arc::downgrade(wb), charts: HashMap::new() });
if !std::ptr::eq(doc.wb.as_ptr(), Arc::as_ptr(wb)) {
// The document changed (or is another one with this uid): nothing built from the old
// workbook can be trusted.
let dropped: usize = doc.charts.values().map(|e| e.bytes).sum();
doc.charts.clear();
doc.wb = Arc::downgrade(wb);
self.bytes = self.bytes.saturating_sub(dropped);
}
let (zb, pb) = (z.to_bits(), ppp.to_bits());
if let Some(e) = doc.charts.get_mut(&(sheet, index))
&& e.z == zb
&& e.ppp == pb
{
e.used = tick;
return e.prims.clone();
}
#[cfg(test)]
{
self.builds += 1;
}
let prims: Arc<[Prim]> = build().into();
let bytes = prims.iter().map(prim_bytes).fold(0usize, usize::saturating_add);
let Some(doc) = self.docs.get_mut(&uid) else { return prims };
if let Some(old) = doc.charts.remove(&(sheet, index)) {
self.bytes = self.bytes.saturating_sub(old.bytes);
}
// A chart that alone would fill a quarter of the budget is not worth keeping.
if bytes <= MAX_BYTES / 4 {
doc.charts.insert((sheet, index), Entry { z: zb, ppp: pb, prims: prims.clone(), bytes, used: tick });
self.bytes = self.bytes.saturating_add(bytes);
self.evict();
}
prims
}
/// Drops least recently used entries until the caps hold.
fn evict(&mut self) {
loop {
let count: usize = self.docs.values().map(|d| d.charts.len()).sum();
if count <= MAX_ENTRIES && self.bytes <= MAX_BYTES {
return;
}
let oldest = self.docs.iter().flat_map(|(uid, d)| d.charts.iter().map(move |(k, e)| (e.used, *uid, *k))).min();
let Some((_, uid, key)) = oldest else { return };
if let Some(e) = self.docs.get_mut(&uid).and_then(|d| d.charts.remove(&key)) {
self.bytes = self.bytes.saturating_sub(e.bytes);
}
}
}
/// Forgets the documents that are not in `live` (closed workbooks).
pub(crate) fn retain_docs(&mut self, live: impl Iterator<Item = u64>) {
let live: Vec<u64> = live.collect();
let bytes = &mut self.bytes;
self.docs.retain(|uid, d| {
let keep = live.contains(uid);
if !keep {
*bytes = bytes.saturating_sub(d.charts.values().map(|e| e.bytes).sum());
}
keep
});
}
/// Forgets everything (the fonts text was measured with changed).
pub(crate) fn clear(&mut self) {
self.docs.clear();
self.bytes = 0;
}
#[cfg(test)]
pub(crate) fn len(&self) -> usize {
self.docs.values().map(|d| d.charts.len()).sum()
}
#[cfg(test)]
pub(crate) fn docs(&self) -> usize {
self.docs.len()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::SheetApp;
use crate::chartview::tests::RENDERS;
use egui::{Event, vec2};
use gridcraft_chart::ApproxMeasure;
use gridcraft_engine::Session;
use serde_json::json;
type Harness = egui_kittest::Harness<'static, SheetApp>;
fn wb() -> Arc<Workbook> {
Arc::new(Workbook::new())
}
fn rect_prim(w: f32) -> Vec<Prim> {
vec![Prim::Rect { x: 0.0, y: 0.0, w, h: 1.0, fill: None, stroke: None, radius: 0.0 }]
}
fn line_prim(points: usize) -> Vec<Prim> {
vec![Prim::Line { pts: vec![[0.0, 0.0]; points], color: [0, 0, 0, 255], width: 1.0, dash: false }]
}
#[test]
fn a_workbook_written_after_caching_is_never_taken_for_the_cached_one() {
let mut cache = ChartCache::default();
let mut w = wb();
let build = || rect_prim(1.0);
cache.get(1, &w, 0, 0, (1.0, 1.0), build);
cache.get(1, &w, 0, 0, (1.0, 1.0), build);
assert_eq!(cache.builds, 1, "same workbook: a hit");
// Sole owner of the Arc: without the Weak this write would happen in place, at the same address.
Arc::make_mut(&mut w).active_sheet.set(0);
cache.get(1, &w, 0, 0, (1.0, 1.0), build);
assert_eq!(cache.builds, 2, "a write through make_mut moved the workbook to a new allocation");
// Shared with an undo snapshot: make_mut clones, the pointer changes as well.
let snapshot = w.clone();
Arc::make_mut(&mut w).active_sheet.set(0);
cache.get(1, &w, 0, 0, (1.0, 1.0), build);
assert_eq!(cache.builds, 3);
// Replacing the Arc with an equal workbook (a reload) is a change too.
w = Arc::new((*snapshot).clone());
cache.get(1, &w, 0, 0, (1.0, 1.0), build);
assert_eq!(cache.builds, 4);
// The old allocation is still held by the cache's Weak while its owner is gone, so a new
// workbook can't be mistaken for it by landing on the same address.
drop(snapshot);
cache.get(1, &w, 0, 0, (1.0, 1.0), build);
assert_eq!(cache.builds, 4);
}
#[test]
fn zoom_scale_chart_and_document_are_part_of_the_key() {
let mut cache = ChartCache::default();
let w = wb();
let build = || rect_prim(1.0);
cache.get(1, &w, 0, 0, (1.0, 1.0), build);
cache.get(1, &w, 0, 0, (1.25, 1.0), build);
cache.get(1, &w, 0, 0, (1.25, 2.0), build);
cache.get(1, &w, 0, 1, (1.25, 2.0), build);
cache.get(1, &w, 1, 0, (1.25, 2.0), build);
cache.get(2, &w, 0, 0, (1.25, 2.0), build);
assert_eq!(cache.builds, 6);
cache.get(1, &w, 0, 1, (1.25, 2.0), build);
cache.get(2, &w, 0, 0, (1.25, 2.0), build);
assert_eq!(cache.builds, 6, "each of those is cached");
}
#[test]
fn the_cache_is_capped_in_entries_and_bytes_and_drops_the_least_recently_used() {
let mut cache = ChartCache::default();
let w = wb();
for i in 0..MAX_ENTRIES + 10 {
cache.get(1, &w, 0, i, (1.0, 1.0), || rect_prim(1.0));
// Chart 0 is used all the time, so it survives.
cache.get(1, &w, 0, 0, (1.0, 1.0), || rect_prim(1.0));
}
assert_eq!(cache.len(), MAX_ENTRIES);
let before = cache.builds;
cache.get(1, &w, 0, 0, (1.0, 1.0), || rect_prim(1.0));
assert_eq!(cache.builds, before, "the busy chart is still there");
cache.get(1, &w, 0, 1, (1.0, 1.0), || rect_prim(1.0));
assert_eq!(cache.builds, before + 1, "the idle one was evicted");
// Bytes: each of these is ~4 MiB of points; only a handful fit.
let mut cache = ChartCache::default();
let points = (4 << 20) / std::mem::size_of::<[f32; 2]>();
for i in 0..40 {
cache.get(1, &w, 0, i, (1.0, 1.0), || line_prim(points));
}
assert!(cache.bytes <= MAX_BYTES && cache.len() < 40, "{} entries, {} bytes", cache.len(), cache.bytes);
// One that would fill a quarter of the budget is not kept, and is still returned.
let mut cache = ChartCache::default();
let big = cache.get(1, &w, 0, 0, (1.0, 1.0), || line_prim(MAX_BYTES / 4 / std::mem::size_of::<[f32; 2]>() + 1000));
assert!(!big.is_empty());
assert_eq!((cache.len(), cache.bytes), (0, 0));
}
#[test]
fn closed_documents_are_forgotten() {
let mut cache = ChartCache::default();
let w = wb();
cache.get(1, &w, 0, 0, (1.0, 1.0), || line_prim(10));
cache.get(2, &w, 0, 0, (1.0, 1.0), || line_prim(10));
cache.retain_docs([2].into_iter());
assert_eq!((cache.docs(), cache.len()), (1, 1));
cache.retain_docs(std::iter::empty());
assert_eq!((cache.docs(), cache.len(), cache.bytes), (0, 0, 0));
}
/// A workbook with charts of several kinds over numbers, dates and categories.
fn corpus() -> Session {
let mut s = Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [["Date", "Sales", "Cost", "Share"]]})).unwrap();
for i in 0..40u32 {
let date = format!("2024-{:02}-{:02}", 1 + i / 28, 1 + i % 28);
s.execute("cell.set", json!({"cell": format!("A{}", i + 2), "input": date})).unwrap();
s.execute("cell.set", json!({"cell": format!("B{}", i + 2), "input": format!("{}", (i * 37) % 101)})).unwrap();
s.execute("cell.set", json!({"cell": format!("C{}", i + 2), "input": format!("=B{}/2+{}", i + 2, i)})).unwrap();
s.execute("cell.set", json!({"cell": format!("D{}", i + 2), "input": format!("{}", (i % 5) + 1)})).unwrap();
}
for (i, (kind, range)) in [
("column", "A1:C41"),
("line", "A1:C41"),
("scatter", "B1:C41"),
("pie", "A1:B6"),
("combo", "A1:C41"),
("area", "A1:B41"),
("bar", "A1:D41"),
("doughnut", "A1:B6"),
]
.into_iter()
.enumerate()
{
s.execute("insert.chart", json!({"range": range, "type": kind, "at": format!("H{}", 2 + i * 16), "width": 320, "height": 220})).unwrap();
}
s
}
#[test]
fn cached_output_equals_a_fresh_render_for_a_corpus_of_charts() {
let s = corpus();
let d = s.doc().unwrap();
let sh = d.wb.active().unwrap();
assert_eq!(sh.charts.len(), 8);
let mut cache = ChartCache::default();
for (i, ch) in sh.charts.iter().enumerate() {
let fresh = || {
let data = gridcraft_chart::resolve(&d.wb, d.wb.active_sheet.get(), ch);
gridcraft_chart::render(ch, &data, ch.anchor.width, ch.anchor.height, &ApproxMeasure)
};
let expect = fresh();
assert!(expect.len() > 3, "{:?} renders something", ch.kind);
let first = cache.get(d.uid, &d.wb, d.wb.active_sheet.get(), i, (1.0, 1.0), fresh);
let second = cache.get(d.uid, &d.wb, d.wb.active_sheet.get(), i, (1.0, 1.0), || panic!("must be a hit"));
assert_eq!(&*first, &expect[..], "{:?}: built entry", ch.kind);
assert_eq!(&*second, &expect[..], "{:?}: cached entry", ch.kind);
}
assert_eq!(cache.builds, 8);
}
// --- frames ---
fn harness(s: Session) -> Harness {
let mut h = egui_kittest::Harness::builder().with_size(vec2(1400.0, 900.0)).build_ui_state(
|ui, app: &mut SheetApp| {
let ctx = ui.ctx().clone();
app.logic(&ctx);
app.ui(ui);
},
SheetApp::new(s, Default::default()),
);
h.run_steps(8);
h
}
/// Two charts side by side, both in view, over A1:B12 (B3 is `=B2*2`).
fn two_charts() -> Session {
let mut s = Session::new();
s.new_workbook();
s.execute("range.setValues", json!({"range": "A1", "values": [["Day", "Sales"]]})).unwrap();
for i in 2..=12u32 {
let input = if i == 3 { "=B2*2".to_string() } else { format!("{}", i * 3 % 17) };
s.execute("cell.set", json!({"cell": format!("A{i}"), "input": format!("2024-03-{i:02}")})).unwrap();
s.execute("cell.set", json!({"cell": format!("B{i}"), "input": input})).unwrap();
}
s.execute("insert.chart", json!({"range": "A1:B12", "type": "column", "at": "D2", "width": 300, "height": 200})).unwrap();
s.execute("insert.chart", json!({"range": "A1:B12", "type": "scatter", "at": "L2", "width": 300, "height": 200})).unwrap();
s
}
fn builds() -> usize {
RENDERS.with(|c| c.get())
}
/// Runs `change`, lets a few frames pass and returns how many charts were rendered meanwhile.
fn rendered_after(h: &mut Harness, change: impl FnOnce(&mut Harness)) -> usize {
let before = builds();
change(h);
h.run_steps(4);
builds() - before
}
fn exec(h: &mut Harness, id: &str, params: serde_json::Value) {
h.state_mut().session.execute(id, params).unwrap();
}
#[test]
fn frames_without_a_change_render_each_chart_once() {
let mut h = harness(two_charts());
assert!(builds() >= 2, "the charts were drawn");
let n = rendered_after(&mut h, |h| h.run_steps(30));
assert_eq!(n, 0, "30 idle frames render nothing again");
// Pointer movement and wheel scrolling (view state only) change no workbook.
let scroll_before = h.state().view().scroll;
let n = rendered_after(&mut h, |h| {
for i in 0..10 {
h.input_mut().events.push(Event::PointerMoved(egui::pos2(300.0 + i as f32 * 7.0, 400.0)));
h.input_mut().events.push(Event::MouseWheel {
unit: egui::MouseWheelUnit::Point,
source: egui::MouseWheelSource::Unknown,
delta: vec2(0.0, -2.0),
phase: egui::TouchPhase::Move,
modifiers: egui::Modifiers::NONE,
});
h.run_steps(1);
}
});
assert_ne!(h.state().view().scroll, scroll_before, "the sheet did scroll");
assert_eq!(n, 0, "hover and scrolling reuse the rendered charts");
assert_eq!(h.state().chart_cache.lock().unwrap().len(), 2);
}
#[test]
fn every_kind_of_change_renders_the_charts_again_exactly_once() {
let mut h = harness(two_charts());
type Case<'a> = (&'a str, Box<dyn Fn(&mut Harness)>);
let cases: Vec<Case> = vec![
("cell edit", Box::new(|h| exec(h, "cell.set", json!({"cell": "B5", "input": "99"})))),
("recalc through a dependency", Box::new(|h| exec(h, "cell.set", json!({"cell": "B2", "input": "7"})))),
("calculate now", Box::new(|h| exec(h, "formulas.calculateNow", json!({})))),
("undo", Box::new(|h| exec(h, "edit.undo", json!({})))),
("redo", Box::new(|h| exec(h, "edit.redo", json!({})))),
("row hide", Box::new(|h| exec(h, "home.hideRows", json!({"rows": "6:7"})))),
("chart resize", Box::new(|h| exec(h, "chart.set", json!({"chart": 1, "width": 360, "height": 240})))),
("format change", Box::new(|h| exec(h, "home.bold", json!({"range": "A1:B1"})))),
("number format", Box::new(|h| exec(h, "home.numberFormat", json!({"range": "B2:B12", "format": "0.00"})))),
(
"document theme",
Box::new(|h| {
exec(
h,
"pageLayout.themeColors",
json!({"colors": ["#000000", "#FFFFFF", "#44546A", "#E7E6E6", "#C00000", "#ED7D31", "#A5A5A5", "#FFC000", "#5B9BD5", "#70AD47", "#0563C1", "#954F72"]}),
)
}),
),
];
for (what, change) in cases {
let n = rendered_after(&mut h, |h| change(h));
assert_eq!(n, 2, "{what}: both visible charts rendered once");
let n = rendered_after(&mut h, |h| h.run_steps(5));
assert_eq!(n, 0, "{what}: and then nothing more");
}
// Zoom is part of the key too.
let n = rendered_after(&mut h, |h| exec(h, "view.zoom", json!({"percent": 150})));
assert_eq!(n, 2, "zoom");
}
#[test]
fn switching_sheets_and_loading_files_render_the_new_charts_once() {
let mut h = harness(two_charts());
exec(&mut h, "home.insertSheet", json!({"before": 1}));
h.run_steps(4);
// The new sheet has no charts; coming back renders this sheet's two once.
let n = rendered_after(&mut h, |h| exec(h, "sheet.activate", json!({"sheet": 0})));
assert_eq!(n, 2, "sheet switch");
let n = rendered_after(&mut h, |h| h.run_steps(5));
assert_eq!(n, 0);
// File load: the same workbook opened from bytes is a new document with its own charts.
let bytes = h.state_mut().session.execute("file.saveBytes", json!({})).unwrap();
let b64 = bytes["base64"].as_str().unwrap().to_string();
let n = rendered_after(&mut h, |h| exec(h, "file.open", json!({"name": "again.xlsx", "base64": b64})));
assert_eq!(n, 2, "file load");
assert_eq!(h.state().session.documents().len(), 2);
let n = rendered_after(&mut h, |h| h.run_steps(5));
assert_eq!(n, 0);
assert_eq!(h.state().chart_cache.lock().unwrap().docs(), 2, "both open documents are cached");
// Back in the first document nothing is rendered: its entries were kept.
let n = rendered_after(&mut h, |h| h.state_mut().session.set_active(0));
assert_eq!(n, 0, "the first document's charts were still cached");
// Closing a document forgets its entries; the other one is shown (and rendered) now.
let n = rendered_after(&mut h, |h| exec(h, "file.close", json!({"force": true})));
assert_eq!(h.state().session.documents().len(), 1);
assert_eq!(h.state().chart_cache.lock().unwrap().docs(), 1);
assert_eq!(n, 0, "the remaining document's charts were cached too");
}
#[test]
fn a_chart_with_a_volatile_series_is_rendered_on_every_frame() {
let mut h = harness(two_charts());
{
let d = h.state_mut().session.active_mut().unwrap();
let sh = Arc::make_mut(&mut d.wb).sheet_mut(0).unwrap();
sh.charts[0].series[0].values = "=RAND()".into();
}
h.run_steps(4);
let n = rendered_after(&mut h, |h| h.run_steps(6));
assert!(n >= 6, "the volatile chart is resolved again on each frame, got {n}");
assert!(n < 12, "the other chart still comes from the cache, got {n}");
}
#[test]
fn closed_documents_are_forgotten_without_painting_a_chart() {
let mut h = harness(two_charts());
assert_eq!(h.state().chart_cache.lock().unwrap().docs(), 1);
exec(&mut h, "file.new", json!({}));
h.run_steps(4);
assert_eq!(h.state().session.documents().len(), 2);
// The charts' document is no longer shown, but still open.
assert_eq!(h.state().chart_cache.lock().unwrap().docs(), 1);
h.state_mut().session.set_active(0);
h.run_steps(2);
exec(&mut h, "file.close", json!({"force": true, "document": true}));
h.run_steps(2);
assert_eq!(h.state().session.documents().len(), 1);
assert_eq!(h.state().chart_cache.lock().unwrap().docs(), 0, "no chart is painted, yet the closed document's entries are gone");
}
}
+47 -7
View File
@@ -1,6 +1,8 @@
//! Floating objects on the sheet: charts (rendered by `gridcraft-chart` into primitives that
//! we paint with egui), pictures, shapes, equations, slicers and form controls (`form_controls.rs`).
use std::sync::Arc;
use egui::epaint::{PathShape, PathStroke};
use egui::{Align2, Color32, FontId, Painter, Pos2, Rect, Stroke, StrokeKind, pos2, vec2};
use gridcraft_chart::{HAlign, Prim, VAlign};
@@ -198,10 +200,11 @@ fn area_mesh(v: &[Pos2], col: Color32) -> egui::Mesh {
fn paint_shape_text(p: &Painter, wb: &Workbook, sp: &gridcraft_engine::model::Shape, r: Rect, z: f32) {
use gridcraft_engine::model::{ShapeTextAlign, ShapeTextAnchor};
let f = &sp.format;
let word_art = sp.kind == ShapeKind::WordArt;
let text_box = sp.kind == ShapeKind::TextBox;
let align = f.align.unwrap_or(if text_box { ShapeTextAlign::Left } else { ShapeTextAlign::Center });
let anchor = f.anchor.unwrap_or(if text_box { ShapeTextAnchor::Top } else { ShapeTextAnchor::Middle });
let default_color = if text_box { Color32::BLACK } else { Color32::WHITE };
let default_color = if text_box || word_art { Color32::BLACK } else { Color32::WHITE };
let color = f.text_color.and_then(|c| c.resolve(&wb.theme)).map(theme::color32).unwrap_or(default_color);
// Default 14 px (as before); a set size is in points.
let size = f.font_size.map_or(14.0, |pt| pt * 96.0 / 72.0).clamp(1.0, 2000.0) * z;
@@ -227,6 +230,11 @@ fn paint_shape_text(p: &Painter, wb: &Workbook, sp: &gridcraft_engine::model::Sh
};
let pos = pos2(x, y);
let clip = p.with_clip_rect(r.intersect(p.clip_rect()));
// WordArt's drop shadow: the text again, offset down and right, under everything else.
if let Some(sc) = f.text_shadow.and_then(|c| c.resolve(&wb.theme)).map(theme::color32) {
let d = (size * 0.05).clamp(1.0, 24.0);
clip.galley_with_override_text_color(pos + vec2(d, d), g.clone(), sc);
}
// egui has no stroked text: the outline is the text drawn in the outline colour, offset in
// eight directions by the outline width, under the filled text.
if let Some((c, w)) = f.text_outline
@@ -240,6 +248,31 @@ fn paint_shape_text(p: &Painter, wb: &Workbook, sp: &gridcraft_engine::model::Sh
clip.galley(pos, g, color);
}
/// Resolves and renders `ch` (chart `ci` of sheet `si`), at zoom `z`.
fn render_chart(p: &Painter, z: f32, wb: &Workbook, si: usize, ch: &gridcraft_engine::model::Chart) -> Vec<Prim> {
#[cfg(test)]
tests::RENDERS.with(|c| c.set(c.get() + 1));
let data = gridcraft_chart::resolve(wb, si, ch);
let m = EguiMeasure { painter: p, z };
gridcraft_chart::render(ch, &data, ch.anchor.width, ch.anchor.height, &m)
}
/// The primitives of chart `ci` of sheet `si`: from the frame cache when the workbook, zoom and
/// screen scale are those it was rendered for, else rendered now (see [`crate::chartcache`]).
fn chart_prims(app: &SheetApp, p: &Painter, z: f32, wb: &Arc<Workbook>, si: usize, ci: usize, ch: &gridcraft_engine::model::Chart) -> Arc<[Prim]> {
// Only the active document's own workbook is cached: anything else cannot be tied to a uid.
let Some(d) = app.session.active().filter(|d| Arc::ptr_eq(&d.wb, wb)) else {
return render_chart(p, z, wb, si, ch).into();
};
// A series that calls NOW or RAND, directly or through a defined name, must be evaluated
// again on every paint.
if gridcraft_chart::has_volatile_series(wb, si, ch) {
return render_chart(p, z, wb, si, ch).into();
}
let mut cache = app.chart_cache.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
cache.get(d.uid, wb, si, ci, (z, p.ctx().pixels_per_point()), || render_chart(p, z, wb, si, ch))
}
pub(crate) fn anchor_rect(geo: &Geo, sh: &Sheet, a: &Anchor) -> Rect {
let x = geo.x(sh, a.cell.col) + a.dx * geo.z;
let y = geo.y(sh, a.cell.row) + a.dy * geo.z;
@@ -259,7 +292,7 @@ pub(crate) fn objects_hidden(wb: &Workbook) -> bool {
wb.show_objects == gridcraft_engine::model::ShowObjects::None
}
pub(crate) fn paint_objects(app: &SheetApp, p: &Painter, geo: &Geo, wb: &Workbook, si: usize, sh: &Sheet) -> Vec<PaintedObject> {
pub(crate) fn paint_objects(app: &SheetApp, p: &Painter, geo: &Geo, wb: &Arc<Workbook>, si: usize, sh: &Sheet) -> Vec<PaintedObject> {
let mut painted = Vec::new();
if objects_hidden(wb) {
return painted;
@@ -295,6 +328,8 @@ pub(crate) fn paint_objects(app: &SheetApp, p: &Painter, geo: &Geo, wb: &Workboo
let line =
if sp.format.no_line { Color32::TRANSPARENT } else { sp.line.resolve(&wb.theme).map(theme::color32).unwrap_or(Color32::DARK_GRAY) };
match sp.kind {
// WordArt is only its text (no fill or outline of its own).
ShapeKind::WordArt => {}
ShapeKind::Rectangle | ShapeKind::TextBox => {
p.rect_filled(r, 0.0, fill);
p.rect_stroke(r, 0.0, Stroke::new(1.0, line), StrokeKind::Inside);
@@ -365,7 +400,7 @@ pub(crate) fn paint_objects(app: &SheetApp, p: &Painter, geo: &Geo, wb: &Workboo
}
done(start, r);
}
for ch in &sh.charts {
for (ci, ch) in sh.charts.iter().enumerate() {
let r = anchor_rect(geo, sh, &ch.anchor);
if !r.intersects(p.clip_rect()) {
continue;
@@ -374,9 +409,7 @@ pub(crate) fn paint_objects(app: &SheetApp, p: &Painter, geo: &Geo, wb: &Workboo
// Soft shadow + white card.
p.rect_filled(r.translate(vec2(0.0, 1.5)), 2.0, Color32::from_black_alpha(18));
p.rect_filled(r, 0.0, Tokens::get(p.ctx()).grid_bg);
let data = gridcraft_chart::resolve(wb, si, ch);
let m = EguiMeasure { painter: p, z: geo.z };
let prims = gridcraft_chart::render(ch, &data, ch.anchor.width, ch.anchor.height, &m);
let prims = chart_prims(app, p, geo.z, wb, si, ci, ch);
paint_prims(&p.with_clip_rect(r.intersect(p.clip_rect())), r.min, &prims, geo.z);
done(start, r);
}
@@ -583,11 +616,18 @@ pub(crate) fn ink_color(line: Color32, width_pt: f32) -> Color32 {
}
#[cfg(test)]
mod tests {
pub(crate) mod tests {
use std::cell::Cell;
use gridcraft_chart::Measure;
use super::*;
thread_local! {
/// Charts resolved and rendered on this thread (a test runs on its own thread).
pub(crate) static RENDERS: Cell<usize> = const { Cell::new(0) };
}
/// The production measure is not monotone for labels with a line break: egui lays the text out
/// line by line, so ending a prefix with `\n…` can be narrower than the shorter `iii…`. The
/// chart's label fitting must not assume otherwise (#1152).
+4
View File
@@ -275,6 +275,10 @@ pub fn handle(app: &mut SheetApp, ctx: &egui::Context, method: &str, p: &Json) -
("options.autoComplete", "Enable AutoComplete for cell values", "{on?: bool} (toggles without it) → {on}"),
("options.highlightActiveRow", "Highlight the active cell's row", "{on?: bool} (toggles without it) → {on}"),
("view.interfaceScale", "Interface Size", "{percent: 75–200 | \"auto\"} (100 = auto; clamped)"),
("ui.window.minimize", "Minimize Window", ""),
("ui.window.maximize", "Maximize Window", "{on: bool}"),
("ui.window.close", "Close Window", ""),
("ui.window.beginResize", "Resize Window", "{direction: north|south|east|west|northeast|northwest|southeast|southwest}"),
("ui.dialog", "Open Dialog", ""),
("ui.backstage", "File (Backstage view)", "{page?: home|new|open|info|saveAs|print|export|options|account|close}"),
(
+10
View File
@@ -112,15 +112,19 @@ pub enum Drag {
source: RangeRef,
before: Option<u32>,
},
/// `to`: the previewed size once the pointer moved. Only the dragged line changes while the
/// button is held; the release applies the size to every selected line (Excel).
ResizeCol {
col: u32,
start: f32,
orig: f32,
to: Option<f32>,
},
ResizeRow {
row: u32,
start: f32,
orig: f32,
to: Option<f32>,
},
/// Dragging the fill handle; `menu` when the right button drags it (the fill menu opens on release).
Fill {
@@ -244,6 +248,12 @@ pub struct GridState {
/// active sheet's rules, kept the same way: they are read from that sheet's cells alone.
pub(crate) cf: Option<(std::sync::Weak<Sheet>, CfCache)>,
pub drag: Drag,
/// A header-boundary resize drag: the workbook as it was when the drag began. The drag's own
/// undo entries are the ones from the first entry that snapshots it; earlier edits with the
/// same label are not the drag's to merge or cancel.
pub resize_base: Option<Arc<Workbook>>,
/// The redo stack when that drag began; Esc puts it back (the preview commands clear it).
pub resize_redo: Vec<gridcraft_engine::HistoryEntry>,
pub ensure_visible: bool,
/// With `ensure_visible`: scroll to the moving end of the selection instead of the active cell
/// (Shift/Ctrl+Shift+arrows keep the active cell on the anchor while the far corner moves).
+124 -27
View File
@@ -325,7 +325,8 @@ fn handle_column_header_press(app: &mut SheetApp, input: &InteractionInput<'_>,
let in_col_header = input.in_column_header;
if in_col_header {
if let Some((c, x)) = col_edge(geo, sh, p.x) {
app.grid.drag = Drag::ResizeCol { col: c, start: p.x, orig: sh.col_width(c).max(0.0) * 0.0 + (x - geo.x(sh, c)) / geo.z };
begin_resize(app);
app.grid.drag = Drag::ResizeCol { col: c, start: p.x, orig: sh.col_width(c).max(0.0) * 0.0 + (x - geo.x(sh, c)) / geo.z, to: None };
return;
}
let c = geo.col_at(sh, p.x);
@@ -362,7 +363,8 @@ fn handle_row_header_press(app: &mut SheetApp, input: &InteractionInput<'_>, p:
let in_row_header = input.in_row_header;
if in_row_header {
if let Some((r, y)) = row_edge(geo, sh, p.y) {
app.grid.drag = Drag::ResizeRow { row: r, start: p.y, orig: (y - geo.y(sh, r)) / geo.z };
begin_resize(app);
app.grid.drag = Drag::ResizeRow { row: r, start: p.y, orig: (y - geo.y(sh, r)) / geo.z, to: None };
return;
}
let r = geo.row_at(sh, p.y);
@@ -631,6 +633,31 @@ fn handle_context_menu_and_keyboard(app: &mut SheetApp, input: &InteractionInput
fn handle_dragging(app: &mut SheetApp, input: &InteractionInput<'_>) {
let (geo, sh, wb, sel, pos) = (input.geo, input.sheet, input.workbook, &input.selection, input.position);
let resp = input.response;
// Esc during a header-boundary drag cancels it: undo the preview, ignore the release. egui
// aborts the drag on Esc, so `resp.dragged()` is already false in this frame. The key is
// spent on the drag, so the grid's Esc doesn't also drop a pending copy or Format Painter.
if matches!(app.grid.drag, Drag::ResizeCol { .. } | Drag::ResizeRow { .. })
&& input.context.input_mut(|i| {
let pressed = i.key_pressed(egui::Key::Escape);
if pressed {
// With any modifiers, as before: the drag is cancelled and every Esc press spent.
i.events.retain(|e| !matches!(e, egui::Event::Key { key: egui::Key::Escape, pressed: true, .. }));
}
pressed
})
{
// Undo back to where the drag began, but never past it into an earlier edit.
let steps = resize_steps(app);
if steps > 0 {
let _ = app.session.run("edit.undo", json!({"steps": steps}));
}
// Undo pushes the cancelled preview onto the redo stack; put back what the drag began with.
if let Some(d) = app.session.active_mut() {
d.redo = std::mem::take(&mut app.grid.resize_redo);
}
app.grid.drag = Drag::None;
app.grid.resize_base = None;
}
if resp.dragged()
&& let Some(p) = pos
{
@@ -682,35 +709,22 @@ fn handle_dragging(app: &mut SheetApp, input: &InteractionInput<'_>) {
app.grid.drag = Drag::MoveCols { source, before };
scroll_by(app, vec2(auto.x, 0.0));
}
Drag::ResizeCol { col, start, orig } => {
let w = (orig + (p.x - start) / geo.z).max(0.0);
Drag::ResizeCol { col, start, orig, to } => {
let w = nan_safe_size(orig + (p.x - start) / geo.z);
app.toast =
Some((format!("Width: {:.2} ({} pixels)", gridcraft_engine::cmd::format::points_to_chars(w as f64), w.round()), crate::now_ms()));
// Inside a selection of whole columns, every selected column takes the new width
// (Ctrl-selected areas included); otherwise only the dragged one.
let cols = if sel.ranges.iter().any(|r| r.is_full_cols() && col >= r.start.col && col <= r.end.col) {
sel.ranges.iter().filter(|r| r.is_full_cols()).map(|r| r.a1()).collect::<Vec<_>>().join(",")
} else {
RangeRef::cols(col, col).a1()
};
let _ = app.session.run("home.columnWidth", json!({"cols": cols, "width": w.round()}));
coalesce_last_undo(app);
// Preview on the dragged column only; the release applies it to the selection. A
// rejected size leaves the last shown preview (and `to`) as it was.
let ok = app.session.run("home.columnWidth", json!({"cols": RangeRef::cols(col, col).a1(), "width": w.round()})).is_ok();
coalesce_resize_undo(app);
app.grid.drag = Drag::ResizeCol { col, start, orig, to: if ok { Some(w) } else { to } };
}
Drag::ResizeRow { row, start, orig } => {
let h = (orig + (p.y - start) / geo.z).max(0.0);
Drag::ResizeRow { row, start, orig, to } => {
let h = nan_safe_size(orig + (p.y - start) / geo.z);
app.toast = Some((format!("Height: {:.2} ({} pixels)", h * 0.75, h.round()), crate::now_ms()));
let rows = if sel.ranges.iter().any(|r| r.is_full_rows() && row >= r.start.row && row <= r.end.row) {
sel.ranges
.iter()
.filter(|r| r.is_full_rows())
.map(|r| format!("{}:{}", r.start.row + 1, r.end.row + 1))
.collect::<Vec<_>>()
.join(",")
} else {
format!("{}:{}", row + 1, row + 1)
};
let _ = app.session.run("home.rowHeight", json!({"rows": rows, "height": h.round()}));
coalesce_last_undo(app);
let ok = app.session.run("home.rowHeight", json!({"rows": format!("{}:{}", row + 1, row + 1), "height": h.round()})).is_ok();
coalesce_resize_undo(app);
app.grid.drag = Drag::ResizeRow { row, start, orig, to: if ok { Some(h) } else { to } };
}
Drag::Freeze { cols, .. } => {
app.grid.drag = Drag::Freeze { cols, to: Some(freeze_boundary(geo, sh, p, cols)) };
@@ -794,6 +808,21 @@ fn handle_release(app: &mut SheetApp, input: &InteractionInput<'_>) {
crate::macro_ui::call(app, "GridCraft", "range.move", json!({"range": src.a1(), "to": to.a1()}));
}
}
Drag::ResizeCol { col, to: Some(w), .. } => {
// The preview already sized the dragged column; the other selected ones follow now.
let cols = resize_targets(&sel.ranges, col, true);
if cols != RangeRef::cols(col, col).a1() {
let _ = app.session.run("home.columnWidth", json!({"cols": cols, "width": w.round()}));
coalesce_resize_undo(app);
}
}
Drag::ResizeRow { row, to: Some(h), .. } => {
let rows = resize_targets(&sel.ranges, row, false);
if rows != format!("{}:{}", row + 1, row + 1) {
let _ = app.session.run("home.rowHeight", json!({"rows": rows, "height": h.round()}));
coalesce_resize_undo(app);
}
}
Drag::MoveCols { source, before: Some(before) } => {
if let Err(e) = app.run("sheet.moveColumns", json!({"cols": source.a1(), "before": before})) {
app.message = Some(("GridCraft".into(), crate::clean_error(&e)));
@@ -936,6 +965,29 @@ fn select_to_pointer(app: &mut SheetApp, geo: &Geo, sh: &Sheet, wb: &Workbook, p
}
}
/// A drag size in points: NaN or negative becomes 0 (a hidden line).
fn nan_safe_size(v: f32) -> f32 {
if v.is_nan() { 0.0 } else { v.max(0.0) }
}
/// The lines a header-boundary drag resizes on release: every selected whole line when the
/// dragged one is part of the selection (Ctrl-selected areas included), otherwise only it.
fn resize_targets(ranges: &[RangeRef], index: u32, cols: bool) -> String {
let whole = |r: &RangeRef| if cols { r.is_full_cols() } else { r.is_full_rows() };
let span = |r: &RangeRef| if cols { (r.start.col, r.end.col) } else { (r.start.row, r.end.row) };
let name = |r: &RangeRef| {
let (a, b) = span(r);
if cols { RangeRef::cols(a, b).a1() } else { format!("{}:{}", a + 1, b + 1) }
};
if ranges.iter().any(|r| whole(r) && (span(r).0..=span(r).1).contains(&index)) {
ranges.iter().filter(|r| whole(r)).map(name).collect::<Vec<_>>().join(",")
} else if cols {
RangeRef::cols(index, index).a1()
} else {
format!("{}:{}", index + 1, index + 1)
}
}
fn coalesce_last_undo(app: &mut SheetApp) {
// Merge consecutive drag steps into one undo entry.
if let Some(d) = app.session.active_mut()
@@ -948,6 +1000,29 @@ fn coalesce_last_undo(app: &mut SheetApp) {
}
}
/// Starts a header resize drag: remembers the workbook and the redo stack it began with.
fn begin_resize(app: &mut SheetApp) {
app.grid.resize_base = app.session.active().map(|d| d.wb.clone());
app.grid.resize_redo = app.session.active().map(|d| d.redo.clone()).unwrap_or_default();
}
/// Undo entries a header resize drag has pushed so far: those from the first one that snapshots
/// the workbook the drag started from (`resize_base`). A failed command pushes none.
fn resize_steps(app: &SheetApp) -> usize {
let (Some(base), Some(d)) = (&app.grid.resize_base, app.session.active()) else { return 0 };
d.undo.iter().position(|e| std::sync::Arc::ptr_eq(&e.wb, base)).map_or(0, |first| d.undo.len() - first)
}
/// Like [`coalesce_last_undo`] for a resize drag: merges only the drag's own entries, so an
/// earlier edit with the same label ("Column width") stays a separate undo step.
fn coalesce_resize_undo(app: &mut SheetApp) {
if resize_steps(app) >= 2
&& let Some(d) = app.session.active_mut()
{
d.undo.pop();
}
}
fn scroll_by(app: &mut SheetApp, d: egui::Vec2) {
if d != egui::Vec2::ZERO
&& let Some(v) = app.view_mut()
@@ -1161,6 +1236,28 @@ pub fn point_move(app: &mut SheetApp, ed: &mut EditState, dr: i64, dc: i64, exte
app.grid.ensure_visible = true;
}
#[cfg(test)]
mod resize_tests {
use super::*;
#[test]
fn resize_targets_follow_the_selection_only_when_the_dragged_line_is_in_it() {
let sel = [RangeRef::cols(0, 0), RangeRef::cols(2, 3)];
assert_eq!(resize_targets(&sel, 3, true), "A:A,C:D");
assert_eq!(resize_targets(&sel, 1, true), "B:B");
assert_eq!(resize_targets(&[RangeRef::rows(1, 2)], 2, false), "2:3");
assert_eq!(resize_targets(&[RangeRef::rows(1, 2)], 5, false), "6:6");
assert_eq!(resize_targets(&[], 0, true), "A:A");
}
#[test]
fn drag_sizes_are_nan_safe() {
assert_eq!(nan_safe_size(f32::NAN), 0.0);
assert_eq!(nan_safe_size(-4.0), 0.0);
assert_eq!(nan_safe_size(12.5), 12.5);
}
}
#[cfg(test)]
mod tests {
use super::*;
+3
View File
@@ -278,6 +278,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Uložit",
"Save As" => "Uložit jako",
"Export" => "Exportovat",
"Minimize" => "Minimalizovat",
"Maximize" => "Maximalizovat",
"Restore Down" => "Obnovit dolů",
"Close" => "Zavřít",
"Options" => "Možnosti",
"Account" => "Účet",
+3
View File
@@ -294,6 +294,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Speichern",
"Save As" => "Speichern unter",
"Export" => "Exportieren",
"Minimize" => "Minimieren",
"Maximize" => "Maximieren",
"Restore Down" => "Nach unten wiederherstellen",
"Close" => "Schließen",
"Options" => "Optionen",
"Account" => "Konto",
+3
View File
@@ -271,6 +271,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Αποθήκευση",
"Save As" => "Αποθήκευση ως",
"Export" => "Εξαγωγή",
"Minimize" => "Ελαχιστοποίηση",
"Maximize" => "Μεγιστοποίηση",
"Restore Down" => "Επαναφορά",
"Close" => "Κλείσιμο",
"Options" => "Επιλογές",
"Blank workbook" => "Κενό βιβλίο εργασίας",
+3
View File
@@ -283,6 +283,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Guardar",
"Save As" => "Guardar como",
"Export" => "Exportar",
"Minimize" => "Minimizar",
"Maximize" => "Maximizar",
"Restore Down" => "Restaurar",
"Close" => "Cerrar",
"Options" => "Opciones",
"Account" => "Cuenta",
+3
View File
@@ -370,6 +370,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Enregistrer",
"Save As" => "Enregistrer sous",
"Export" => "Exporter",
"Minimize" => "Réduire",
"Maximize" => "Agrandir",
"Restore Down" => "Restaurer",
"Close" => "Fermer",
"Options" => "Options",
"Account" => "Compte",
+3
View File
@@ -365,6 +365,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Salva",
"Save As" => "Salva con nome",
"Export" => "Esporta",
"Minimize" => "Riduci a icona",
"Maximize" => "Ingrandisci",
"Restore Down" => "Ripristina",
"Close" => "Chiudi",
"Options" => "Opzioni",
"Blank workbook" => "Cartella di lavoro vuota",
+3
View File
@@ -364,6 +364,9 @@ pub(super) const CORE: &[(&str, &str)] = &[
("Save", "上書き保存"),
("Save As", "名前を付けて保存"),
("Export", "エクスポート"),
("Minimize", "最小化"),
("Maximize", "最大化"),
("Restore Down", "元のサイズに戻す"),
("Close", "閉じる"),
("Options", "オプション"),
("Blank workbook", "空白のブック"),
+3
View File
@@ -370,6 +370,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "저장",
"Save As" => "다른 이름으로 저장",
"Export" => "내보내기",
"Minimize" => "최소화",
"Maximize" => "최대화",
"Restore Down" => "이전 크기로 복원",
"Close" => "닫기",
"Options" => "옵션",
"Account" => "계정",
+3
View File
@@ -364,6 +364,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Opslaan",
"Save As" => "Opslaan als",
"Export" => "Exporteren",
"Minimize" => "Minimaliseren",
"Maximize" => "Maximaliseren",
"Restore Down" => "Vorig formaat",
"Close" => "Sluiten",
"Options" => "Opties",
"Blank workbook" => "Lege werkmap",
+3
View File
@@ -357,6 +357,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Zapisz",
"Save As" => "Zapisz jako",
"Export" => "Eksportuj",
"Minimize" => "Minimalizuj",
"Maximize" => "Maksymalizuj",
"Restore Down" => "Przywróć w dół",
"Close" => "Zamknij",
"Options" => "Opcje",
"Blank workbook" => "Pusty skoroszyt",
+3
View File
@@ -367,6 +367,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Salvar",
"Save As" => "Salvar como",
"Export" => "Exportar",
"Minimize" => "Minimizar",
"Maximize" => "Maximizar",
"Restore Down" => "Restaurar",
"Close" => "Fechar",
"Options" => "Opções",
"Account" => "Conta",
+3
View File
@@ -364,6 +364,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Salvare",
"Save As" => "Salvare ca",
"Export" => "Exportare",
"Minimize" => "Minimizare",
"Maximize" => "Maximizare",
"Restore Down" => "Restaurare",
"Close" => "Închidere",
"Options" => "Opțiuni",
"Blank workbook" => "Registru gol",
+3
View File
@@ -372,6 +372,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Сохранить",
"Save As" => "Сохранить как",
"Export" => "Экспорт",
"Minimize" => "Свернуть",
"Maximize" => "Развернуть",
"Restore Down" => "Восстановить",
"Close" => "Закрыть",
"Options" => "Параметры",
"Account" => "Учетная запись",
+3
View File
@@ -363,6 +363,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Sačuvaj",
"Save As" => "Sačuvaj kao",
"Export" => "Izvezi",
"Minimize" => "Minimizuj",
"Maximize" => "Maksimizuj",
"Restore Down" => "Vrati",
"Close" => "Zatvori",
"Options" => "Opcije",
"Account" => "Nalog",
+3
View File
@@ -277,6 +277,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Kaydet",
"Save As" => "Farklı Kaydet",
"Export" => "Dışarı Aktar",
"Minimize" => "Simge durumuna küçült",
"Maximize" => "Ekranı kapla",
"Restore Down" => "Önceki boyuta getir",
"Close" => "Kapat",
"Options" => "Seçenekler",
"Blank workbook" => "Boş çalışma kitabı",
+3
View File
@@ -361,6 +361,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Зберегти",
"Save As" => "Зберегти як",
"Export" => "Експорт",
"Minimize" => "Згорнути",
"Maximize" => "Розгорнути",
"Restore Down" => "Відновити",
"Close" => "Закрити",
"Options" => "Параметри",
"Blank workbook" => "Порожня книга",
+3
View File
@@ -367,6 +367,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "Lưu",
"Save As" => "Lưu thành",
"Export" => "Xuất",
"Minimize" => "Thu nhỏ",
"Maximize" => "Phóng to",
"Restore Down" => "Khôi phục",
"Close" => "Đóng",
"Options" => "Tùy chọn",
"Blank workbook" => "Sổ làm việc trống",
+3
View File
@@ -368,6 +368,9 @@ pub(super) fn translate(text: &str) -> Option<&'static str> {
"Save" => "保存",
"Save As" => "另存为",
"Export" => "导出",
"Minimize" => "最小化",
"Maximize" => "最大化",
"Restore Down" => "向下还原",
"Close" => "关闭",
"Options" => "选项",
"Account" => "账户",
+5
View File
@@ -103,6 +103,7 @@ pub enum Icon {
Comment,
Note,
TextBox,
WordArt,
Symbol,
/// An equation: a radical sign over an italic x.
Equation,
@@ -631,6 +632,10 @@ fn paint_with(p: &Painter, rect: Rect, icon: Icon, c: Color32, pal: &Palette) {
g.rect(2.0, 3.0, 12.0, 10.0, None, Some(c));
g.text(8.0, 8.0, "A", 8.0, c, (false, false));
}
Icon::WordArt => {
g.text(9.0, 9.0, "A", 13.0, Color32::from_rgb(0x8E, 0x4F, 0xB5), (true, false));
g.text(8.0, 8.0, "A", 13.0, c, (true, false));
}
Icon::Symbol => g.text(8.0, 8.0, "Ω", 12.0, c, (false, false)),
Icon::Equation => {
g.line(&[(1.5, 9.0), (3.5, 8.0), (5.5, 13.5), (8.5, 2.5), (14.5, 2.5)], c, 1.2);
@@ -194,7 +194,7 @@ pub const SEQUENCES: &[(&str, Entry)] = &[
("NU", Entry::Finish(Action::Command("insert.symbol"))), // Excel: Symbols; containment(Symbol)
("NZS", Entry::Finish(Action::Command("insert.symbol"))), // Excel: Excel: Symbol (modern)
("NV", Entry::Finish(Action::Command("insert.pivotTable"))), // Excel: PivotTable
("NW", Entry::Finish(Action::Pending("insert.wordArt"))), // pending; Excel: WordArt; WordArt; reserved until the command lands
("NW", Entry::Finish(Action::Command("insert.wordArt"))), // Excel: WordArt
("NX", Entry::Finish(Action::Command("insert.textBox"))), // Excel: Text Box
("NZK", Entry::Continue), // Excel: Sparklines Menu; group level:
("NZKSL", Entry::Finish(Action::Dialog("sparkline"))), // Excel: Line Sparkline
+1 -3
View File
@@ -1336,9 +1336,7 @@
"seq": "NW",
"kind": "command",
"target": "insert.wordArt",
"pending": true,
"excel": "WordArt",
"note": "WordArt; reserved until the command lands"
"excel": "WordArt"
},
{
"seq": "NX",
+51
View File
@@ -18,6 +18,7 @@ pub const UPDATE_APP: craft_update::app::App = craft_update::app::App {
pub mod backstage;
mod border_preview;
mod cell_pictures;
mod chartcache;
pub mod chartview;
pub mod code_editor;
pub mod control;
@@ -59,6 +60,7 @@ pub mod theme;
mod updates_ui;
pub mod widgets;
mod window_ui;
pub mod wordart;
pub use window_ui::WindowUi;
@@ -81,6 +83,20 @@ pub use services::{Chrome, Inbox, Services};
pub use state::{NoticeKind, OpenWindow, SaveAnswer, SheetView};
pub mod desktop_theme;
fn parse_resize_direction(value: &str) -> Option<egui::ResizeDirection> {
match value {
"north" => Some(egui::ResizeDirection::North),
"south" => Some(egui::ResizeDirection::South),
"east" => Some(egui::ResizeDirection::East),
"west" => Some(egui::ResizeDirection::West),
"northeast" => Some(egui::ResizeDirection::NorthEast),
"northwest" => Some(egui::ResizeDirection::NorthWest),
"southeast" => Some(egui::ResizeDirection::SouthEast),
"southwest" => Some(egui::ResizeDirection::SouthWest),
_ => None,
}
}
/// Persisted UI preferences.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
@@ -287,6 +303,8 @@ pub struct SheetApp {
pub(crate) capture_geometry_from_raw: bool,
/// The floating object selected on the sheet (chart, picture, shape, equation or slicer).
pub selected_object: Option<gridcraft_engine::model::ObjRef>,
/// Rendered charts kept between frames ([`chartcache`]).
pub(crate) chart_cache: std::sync::Mutex<chartcache::ChartCache>,
/// The system clipboard's text when GridCraft last copied without publishing to it (a
/// control-channel or MCP `edit.copy`/`edit.cut`). While the system clipboard still holds this
/// text, GridCraft's copy is the newer one and Paste uses it.
@@ -304,6 +322,10 @@ pub struct SheetApp {
/// Every unsaved workbook has been answered for quitting (or `app.quit` asked): the next
/// window close goes through.
pub allow_close: bool,
/// Linux/BSD: the window has no system frame; the title bar draws minimize, maximize and
/// close, and the window resizes from its edges (`ribbon::window_frame`).
pub window_controls: bool,
viewport_commands: Vec<egui::ViewportCommand>,
/// A quit question is waiting in the active window: [`doc_windows`] brings its native window
/// to the front, also when the app is hidden or the window minimized (a Dock quit).
pub bring_forward: bool,
@@ -337,3 +359,32 @@ pub fn clean_error(e: &str) -> String {
pub fn cell(r: u32, c: u32) -> CellRef {
CellRef::new(r, c)
}
#[cfg(test)]
mod viewport_command_tests {
use super::parse_resize_direction;
#[test]
fn resize_direction_parser_accepts_all_edges_and_rejects_unknown_values() {
for name in ["north", "south", "east", "west", "northeast", "northwest", "southeast", "southwest"] {
assert!(parse_resize_direction(name).is_some(), "{name}");
}
assert!(parse_resize_direction("middle").is_none());
}
#[test]
fn window_commands_queue_viewport_commands_without_a_workbook() {
use egui::ViewportCommand;
let mut a = crate::SheetApp::new(gridcraft_engine::Session::new(), crate::Services::default());
for (id, params, expected) in [
("ui.window.minimize", serde_json::json!({}), ViewportCommand::Minimized(true)),
("ui.window.maximize", serde_json::json!({"on": true}), ViewportCommand::Maximized(true)),
("ui.window.close", serde_json::json!({}), ViewportCommand::Close),
("ui.window.beginResize", serde_json::json!({"direction": "east"}), ViewportCommand::BeginResize(egui::ResizeDirection::East)),
] {
assert!(a.run(id, params).is_ok(), "{id} should route without a workbook");
assert!(a.viewport_commands.contains(&expected), "{id} queues {expected:?}");
}
assert!(a.run("ui.window.beginResize", serde_json::json!({"direction": "diagonal"})).is_err());
}
}
+3 -1
View File
@@ -128,7 +128,9 @@ pub fn open_action(sh: &Sheet, o: ObjRef) -> OpenAction {
match o.kind {
ObjKind::Chart => OpenAction::FormatChart(o.id),
ObjKind::Equation => OpenAction::EditEquation(o.id),
ObjKind::Shape if sh.shapes.iter().any(|s| s.id == o.id && s.kind == ShapeKind::TextBox) => OpenAction::EditText(o.id),
ObjKind::Shape if sh.shapes.iter().any(|s| s.id == o.id && matches!(s.kind, ShapeKind::TextBox | ShapeKind::WordArt)) => {
OpenAction::EditText(o.id)
}
ObjKind::Control => OpenAction::EditControl(o.id),
ObjKind::Shape | ObjKind::Image | ObjKind::Slicer => OpenAction::Select,
}
+1 -1
View File
@@ -38,7 +38,7 @@ use insert::{draw, insert};
use page_layout::page_layout;
use review::review;
pub use shortcuts::shortcut;
pub use title_bar::{MAC_CONTENT_LEFT, MAC_TITLE_BAR, TITLE_LEFT_PAD, title_bar};
pub use title_bar::{MAC_CONTENT_LEFT, MAC_TITLE_BAR, TITLE_LEFT_PAD, title_bar, window_frame};
use title_bar::{TITLE_BAR_H, TITLE_CONTROL_H};
use view::view;
+6
View File
@@ -208,6 +208,12 @@ pub(super) fn insert(app: &mut SheetApp, ui: &mut Ui) {
if __k.clicked() {
act(app, "insert.textBox", json!({}));
}
// Two lines like its neighbours, so the Insert tab stays as wide as before; the translated label is split, the catalog key is one line.
let label = app.ui.language.tr("Decorative Text").replacen(' ', "\n", 1);
let wa = big_button(ui, Icon::WordArt, &label, "Decorative Text", true);
anchor_keytip(app, ui, &wa, &["insert.wordArt"]);
// Menu invocation shows the style gallery; the command itself never opens a dialog.
egui::Popup::menu(&wa).show(|ui| crate::wordart::gallery(app, ui));
let __k = big_button(ui, Icon::PageLayout, "Header &\nFooter", "Header & Footer", false);
control_keytip(app, ui, &__k, "dialog:headerFooter");
if __k.clicked() {
+117
View File
@@ -120,6 +120,9 @@ pub fn title_bar(app: &mut SheetApp, ui: &mut Ui) {
});
let left_end = ui.min_rect().right();
let right = ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if app.window_controls {
ui.add_space(3.0 * CONTROL_W);
}
ui.add_space(8.0);
let __k = icon_button(ui, Icon::Search, t.text_dim, "Search commands", vec2(28.0, ch));
control_keytip(app, ui, &__k, "commandSearch");
@@ -206,6 +209,120 @@ fn paint_title(ui: &Ui, center: egui::Pos2, title: &str, dirty: bool, max_w: f32
}
}
/// Width of one window-control button on the title bar.
pub const CONTROL_W: f32 = 46.0;
const CONTROL_TITLE_H: f32 = 38.0;
const RESIZE_BAND: f32 = 5.0;
/// Draw application controls and edge resizing for frameless windows.
pub fn window_frame(app: &mut SheetApp, ctx: &egui::Context) {
if !app.window_controls {
return;
}
let (maximized, fullscreen) = ctx.input(|i| (i.viewport().maximized.unwrap_or(false), i.viewport().fullscreen.unwrap_or(false)));
if fullscreen {
return;
}
let t = Tokens::get(ctx);
let screen = ctx.content_rect();
let controls = egui::Rect::from_min_size(egui::pos2(screen.max.x - 3.0 * CONTROL_W, screen.min.y), egui::vec2(3.0 * CONTROL_W, CONTROL_TITLE_H));
egui::Area::new(egui::Id::unique("gc_window_controls")).order(egui::Order::Foreground).fixed_pos(controls.min).show(ctx, |ui| {
ui.spacing_mut().item_spacing = egui::vec2(0.0, 0.0);
ui.horizontal(|ui| {
for kind in ["minimize", "maximize", "close"] {
let (r, resp) = ui.allocate_exact_size(egui::vec2(CONTROL_W, CONTROL_TITLE_H), egui::Sense::click());
let label = app.ui.language.tr(match kind {
"minimize" => "Minimize",
"maximize" if maximized => "Restore Down",
"maximize" => "Maximize",
_ => "Close",
});
let hot = resp.hovered() || resp.is_pointer_button_down_on();
if hot {
ui.painter().rect_filled(
r,
0.0,
if kind == "close" {
egui::Color32::from_rgb(0xC4, 0x2B, 0x1C)
} else if resp.is_pointer_button_down_on() {
t.pressed
} else {
t.hover
},
);
}
resp.widget_info(|| egui::WidgetInfo::labeled(egui::Role::Button, true, label));
let m = r.center();
let stroke = egui::Stroke::new(1.0, if kind == "close" && hot { egui::Color32::WHITE } else { t.text });
match kind {
"minimize" => {
ui.painter().line_segment([egui::pos2(m.x - 5.0, m.y), egui::pos2(m.x + 5.0, m.y)], stroke);
}
"maximize" if maximized => {
let front = egui::Rect::from_min_size(egui::pos2(m.x - 5.0, m.y - 3.0), egui::vec2(8.0, 8.0));
ui.painter().rect_stroke(front, 0.0, stroke, egui::StrokeKind::Inside);
ui.painter().line_segment([egui::pos2(m.x - 3.0, m.y - 5.0), egui::pos2(m.x + 5.0, m.y - 5.0)], stroke);
ui.painter().line_segment([egui::pos2(m.x + 5.0, m.y - 5.0), egui::pos2(m.x + 5.0, m.y + 3.0)], stroke);
}
"maximize" => {
ui.painter().rect_stroke(r.shrink(15.0), 0.0, stroke, egui::StrokeKind::Inside);
}
_ => {
ui.painter().line_segment([m - egui::vec2(5.0, 5.0), m + egui::vec2(5.0, 5.0)], stroke);
ui.painter().line_segment([m + egui::vec2(5.0, -5.0), m - egui::vec2(5.0, -5.0)], stroke);
}
}
if resp.on_hover_text(label).clicked() {
match kind {
"minimize" => app.run_or_alert("ui.window.minimize", json!({})),
"maximize" => app.run_or_alert("ui.window.maximize", json!({"on": !maximized})),
"close" => app.run_or_alert("ui.window.close", json!({})),
_ => {}
}
}
}
});
});
if maximized {
return;
}
let Some(pos) = ctx.input(|i| i.pointer.hover_pos()) else {
return;
};
let l = pos.x <= screen.min.x + RESIZE_BAND;
let r = pos.x >= screen.max.x - RESIZE_BAND;
let top = pos.y <= screen.min.y + RESIZE_BAND;
let b = pos.y >= screen.max.y - RESIZE_BAND;
let dir = match (l, r, top, b) {
(true, false, true, false) => Some(egui::viewport::ResizeDirection::NorthWest),
(false, true, true, false) => Some(egui::viewport::ResizeDirection::NorthEast),
(true, false, false, true) => Some(egui::viewport::ResizeDirection::SouthWest),
(false, true, false, true) => Some(egui::viewport::ResizeDirection::SouthEast),
(true, false, _, _) => Some(egui::viewport::ResizeDirection::West),
(false, true, _, _) => Some(egui::viewport::ResizeDirection::East),
(_, _, true, false) => Some(egui::viewport::ResizeDirection::North),
(_, _, false, true) => Some(egui::viewport::ResizeDirection::South),
_ => None,
};
if let Some(dir) = dir
&& ctx.input(|i| i.pointer.primary_pressed())
&& !controls.contains(pos)
{
let direction = match dir {
egui::ResizeDirection::North => "north",
egui::ResizeDirection::South => "south",
egui::ResizeDirection::East => "east",
egui::ResizeDirection::West => "west",
egui::ResizeDirection::NorthEast => "northeast",
egui::ResizeDirection::NorthWest => "northwest",
egui::ResizeDirection::SouthEast => "southeast",
egui::ResizeDirection::SouthWest => "southwest",
};
app.run_or_alert("ui.window.beginResize", json!({"direction": direction}));
}
app.flush_viewport_commands(ctx);
}
#[cfg(test)]
mod title_bar_tests {
use super::*;
+11 -3
View File
@@ -33,7 +33,7 @@ impl SheetApp {
}
let Some(d) = self.session.active() else { return false };
let Some(sh) = d.wb.active() else { return false };
let Some(shape) = sh.shapes.iter().find(|s| s.id == id && s.kind == ShapeKind::TextBox) else {
let Some(shape) = sh.shapes.iter().find(|s| s.id == id && matches!(s.kind, ShapeKind::TextBox | ShapeKind::WordArt)) else {
return false;
};
// shape.setText refuses edits on a protected sheet; don't open an editor that can't commit.
@@ -118,7 +118,7 @@ pub(crate) fn show(app: &mut SheetApp, ui: &mut egui::Ui, geo: &Geo, sh: &gridcr
app.text_box_editor = Some(ed);
return;
}
let Some(shape) = sh.shapes.iter().find(|s| s.id == ed.id && s.kind == ShapeKind::TextBox) else {
let Some(shape) = sh.shapes.iter().find(|s| s.id == ed.id && matches!(s.kind, ShapeKind::TextBox | ShapeKind::WordArt)) else {
app.session.mode = Mode::Ready;
return;
};
@@ -153,7 +153,15 @@ pub(crate) fn show(app: &mut SheetApp, ui: &mut egui::Ui, geo: &Geo, sh: &gridcr
.add(
TextEdit::multiline(&mut ed.text)
.id(id)
.font(FontId::new(14.0 * geo.z, egui::FontFamily::Name(theme::CELL.into())))
.font(FontId::new(
// WordArt is edited at its own size (clamped so the box stays usable).
if shape.kind == ShapeKind::WordArt {
shape.format.font_size.map_or(14.0, |pt| (pt * 96.0 / 72.0).clamp(8.0, 200.0))
} else {
14.0
} * geo.z,
egui::FontFamily::Name(theme::CELL.into()),
))
.text_color(Color32::BLACK)
.frame(egui::Frame::NONE)
.margin(egui::Margin::ZERO)
+50
View File
@@ -0,0 +1,50 @@
//! Insert › Text › WordArt: the style gallery. Previews are drawn in code from the engine's
//! preset palette (`gridcraft_engine::cmd::wordart::STYLES`); picking one runs `insert.wordArt`.
use egui::{Align2, Color32, FontId, Painter, Rect, Sense, Ui, vec2};
use gridcraft_engine::cmd::wordart::{STYLES, WordArtStyle};
use gridcraft_engine::model::Theme;
use serde_json::json;
use crate::{SheetApp, theme};
/// Paints "Abc" in `style` centred in `r` (fill, outline and shadow as the sheet object has them).
pub(crate) fn paint_sample(p: &Painter, r: Rect, style: &WordArtStyle, theme: &Theme) {
let px = 24.0;
let family = theme::cell_family("", style.bold, style.italic);
let font = FontId::new(px, family);
let c = r.center();
let col = |c: gridcraft_engine::model::Color| c.resolve(theme).map(theme::color32).unwrap_or(Color32::BLACK);
if let Some(sh) = style.shadow {
p.text(c + vec2(1.5, 1.5), Align2::CENTER_CENTER, "Abc", font.clone(), col(sh));
}
if let Some((oc, w)) = style.outline {
let d = (w * 96.0 / 72.0 / 2.0).clamp(0.5, 3.0);
for (dx, dy) in [(-1.0, -1.0), (0.0, -1.0), (1.0, -1.0), (-1.0, 0.0), (1.0, 0.0), (-1.0, 1.0), (0.0, 1.0), (1.0, 1.0)] {
p.text(c + vec2(dx * d, dy * d), Align2::CENTER_CENTER, "Abc", font.clone(), col(oc));
}
}
p.text(c, Align2::CENTER_CENTER, "Abc", font, col(style.fill));
}
/// The gallery popup content: two rows of previews; a click inserts that style.
pub(crate) fn gallery(app: &mut SheetApp, ui: &mut Ui) {
let theme = app.session.active().map(|d| d.wb.theme.clone()).unwrap_or_default();
ui.set_width(5.0 * 72.0);
egui::Grid::new("wordart_gallery").spacing(vec2(4.0, 4.0)).show(ui, |ui| {
for (i, style) in STYLES.iter().enumerate() {
let (r, resp) = ui.allocate_exact_size(vec2(68.0, 44.0), Sense::click());
let t = theme::Tokens::get(ui.ctx());
ui.painter().rect_filled(r, 4.0, if resp.hovered() { t.hover } else { Color32::WHITE });
ui.painter().rect_stroke(r, 4.0, egui::Stroke::new(1.0, t.input_border), egui::StrokeKind::Inside);
paint_sample(&ui.painter().with_clip_rect(r), r, style, &theme);
if resp.on_hover_text(style.name).clicked() {
app.run_or_alert("insert.wordArt", json!({"style": style.name}));
ui.close();
}
if (i + 1) % 5 == 0 {
ui.end_row();
}
}
});
}
+224
View File
@@ -153,6 +153,110 @@ fn dragging_a_boundary_inside_a_multi_area_selection_resizes_every_selected_line
}
}
#[test]
fn a_multi_selection_resize_previews_on_the_dragged_line_and_applies_on_release() {
// Issue #1082: only the dragged line moves while the button is held; Esc cancels.
for rows in [false, true] {
let mut h = harness(if rows { "1:3" } else { "A:C" });
let before = sizes(&h, rows);
let (start, zoom) = edge(&h, rows, 1);
let delta = if rows { vec2(0.0, 30.0) } else { vec2(30.0, 0.0) };
button(&mut h, start, true);
h.input_mut().events.push(Event::PointerMoved(start + delta / 2.0));
h.step();
h.input_mut().events.push(Event::PointerMoved(start + delta));
h.step();
let held = sizes(&h, rows);
assert_eq!(held[1], (before[1] + 30.0 / zoom).round(), "the dragged line previews");
for index in [0, 2, 3] {
assert_eq!(held[index], before[index], "line {index} waits for the release");
}
button(&mut h, start + delta, false);
h.run_steps(2);
let after = sizes(&h, rows);
for (index, size) in after.iter().take(3).enumerate() {
assert_eq!(*size, held[1], "line {index} takes the width on release");
}
assert_eq!(after[3], before[3]);
assert_eq!(h.state().session.active().unwrap().undo.len(), 1);
h.state_mut().session.execute("edit.undo", json!({})).unwrap();
assert_eq!(sizes(&h, rows), before);
// Esc during the drag cancels everything.
let mut h = harness(if rows { "1:3" } else { "A:C" });
button(&mut h, start, true);
h.input_mut().events.push(Event::PointerMoved(start + delta));
h.step();
h.input_mut().events.push(Event::Key {
key: egui::Key::Escape,
physical_key: None,
pressed: true,
repeat: false,
modifiers: Modifiers::NONE,
});
h.step();
button(&mut h, start + delta, false);
h.run_steps(2);
assert_eq!(sizes(&h, rows), before, "Esc restores the sizes");
}
}
#[test]
fn escaping_a_resize_drag_keeps_an_earlier_size_edit() {
// A resize drag right after another resize shares its undo label; Esc must cancel only the drag.
for (rows, selection) in [(false, "B:B"), (true, "2:2"), (false, "A:C"), (true, "1:3")] {
let mut h = harness(selection);
let key = if rows { ("home.rowHeight", "rows", "2:2", "height") } else { ("home.columnWidth", "cols", "B:B", "width") };
h.state_mut().session.execute(key.0, json!({key.1: key.2, key.3: 80})).unwrap();
h.run_steps(2);
let earlier = sizes(&h, rows);
let undo_len = h.state().session.active().unwrap().undo.len();
let (start, _) = edge(&h, rows, 1);
let delta = if rows { vec2(0.0, 30.0) } else { vec2(30.0, 0.0) };
button(&mut h, start, true);
h.input_mut().events.push(Event::PointerMoved(start + delta / 2.0));
h.step();
h.input_mut().events.push(Event::PointerMoved(start + delta));
h.step();
assert_ne!(sizes(&h, rows), earlier, "the drag previews");
h.input_mut().events.push(Event::Key {
key: egui::Key::Escape,
physical_key: None,
pressed: true,
repeat: false,
modifiers: Modifiers::NONE,
});
h.step();
button(&mut h, start + delta, false);
h.run_steps(2);
assert_eq!(sizes(&h, rows), earlier, "Esc cancels only the drag (rows: {rows}, selection {selection})");
assert_eq!(h.state().session.active().unwrap().undo.len(), undo_len, "the earlier edit stays on the undo stack");
}
}
#[test]
fn a_resize_drag_after_another_resize_is_its_own_undo_step() {
for (rows, selection) in [(false, "B:B"), (true, "2:2"), (false, "A:C"), (true, "1:3")] {
let mut h = harness(selection);
let key = if rows { ("home.rowHeight", "rows", "2:2", "height") } else { ("home.columnWidth", "cols", "B:B", "width") };
h.state_mut().session.execute(key.0, json!({key.1: key.2, key.3: 80})).unwrap();
h.run_steps(2);
let earlier = sizes(&h, rows);
let (start, _) = edge(&h, rows, 1);
let delta = if rows { vec2(0.0, 30.0) } else { vec2(30.0, 0.0) };
button(&mut h, start, true);
h.input_mut().events.push(Event::PointerMoved(start + delta / 2.0));
h.step();
h.input_mut().events.push(Event::PointerMoved(start + delta));
h.step();
button(&mut h, start + delta, false);
h.run_steps(2);
assert_eq!(h.state().session.active().unwrap().undo.len(), 2, "selection {selection}: two undo steps");
h.state_mut().session.execute("edit.undo", json!({})).unwrap();
assert_eq!(sizes(&h, rows), earlier, "selection {selection}: one undo reverts only the drag");
}
}
#[test]
fn table_filter_autofit_works_through_command_and_header_double_click() {
for zoom in [50, 100, 200] {
@@ -191,3 +295,123 @@ fn table_filter_autofit_works_through_command_and_header_double_click() {
}
}
}
fn press_escape(h: &mut egui_kittest::Harness<'_, SheetApp>) {
h.input_mut().events.push(Event::Key { key: egui::Key::Escape, physical_key: None, pressed: true, repeat: false, modifiers: Modifiers::NONE });
h.step();
}
#[test]
fn a_rejected_final_preview_releases_with_the_size_last_shown() {
// The engine refuses rows over 546 px and columns over 1800 px; the release must apply the last
// accepted preview to the whole selection, not retry the refused size.
for rows in [false, true] {
let mut h = harness(if rows { "1:3" } else { "A:C" });
let before = sizes(&h, rows);
let (start, zoom) = edge(&h, rows, 1);
let unit = |d: f32| if rows { vec2(0.0, d) } else { vec2(d, 0.0) };
button(&mut h, start, true);
h.input_mut().events.push(Event::PointerMoved(start + unit(15.0)));
h.step();
h.input_mut().events.push(Event::PointerMoved(start + unit(30.0)));
h.step();
let shown = sizes(&h, rows)[1];
assert_eq!(shown, (before[1] + 30.0 / zoom).round());
h.input_mut().events.push(Event::PointerMoved(start + unit(4000.0 * zoom)));
h.step();
assert_eq!(sizes(&h, rows)[1], shown, "the refused size is not shown");
button(&mut h, start + unit(4000.0 * zoom), false);
h.run_steps(2);
let after = sizes(&h, rows);
for (index, size) in after.iter().take(3).enumerate() {
assert_eq!(*size, shown, "line {index} takes the size that was shown (rows: {rows})");
}
assert_eq!(after[3], before[3]);
}
}
#[test]
fn escaping_a_resize_drag_leaves_redo_history_as_it_was() {
for rows in [false, true] {
for with_redo in [false, true] {
let mut h = harness(if rows { "1:3" } else { "A:C" });
let cmd = if rows { ("home.rowHeight", "rows", "4:4", "height") } else { ("home.columnWidth", "cols", "D:D", "width") };
let mut redo_len = 0;
let mut d_size = 0.0;
if with_redo {
h.state_mut().session.execute(cmd.0, json!({cmd.1: cmd.2, cmd.3: 80})).unwrap();
h.state_mut().session.execute("edit.undo", json!({})).unwrap();
h.run_steps(2);
redo_len = h.state().session.active().unwrap().redo.len();
assert_eq!(redo_len, 1);
d_size = sizes(&h, rows)[3];
}
let before = sizes(&h, rows);
let (start, _) = edge(&h, rows, 1);
let delta = if rows { vec2(0.0, 30.0) } else { vec2(30.0, 0.0) };
button(&mut h, start, true);
h.input_mut().events.push(Event::PointerMoved(start + delta / 2.0));
h.step();
h.input_mut().events.push(Event::PointerMoved(start + delta));
h.step();
assert_ne!(sizes(&h, rows), before, "the drag previews");
press_escape(&mut h);
button(&mut h, start + delta, false);
h.run_steps(2);
assert_eq!(sizes(&h, rows), before);
assert_eq!(h.state().session.active().unwrap().redo.len(), redo_len, "Esc leaves the redo stack alone (rows: {rows}, redo: {with_redo})");
// Ctrl+Y brings back only what was undone before the drag.
let redone = h.state_mut().session.execute("edit.redo", json!({}));
if with_redo {
redone.unwrap();
let redone = sizes(&h, rows);
assert_eq!(redone[3], 80.0);
assert_ne!(redone[3], d_size);
assert_eq!(redone[..3], before[..3], "the cancelled drag does not come back");
} else {
assert!(redone.is_err(), "nothing to redo after a cancelled drag");
}
}
}
}
#[test]
fn escaping_a_resize_drag_keeps_a_pending_copy_and_format_painter() {
// Esc cancels the drag only: the marked copy and Format Painter stay, whether the release comes
// in a later frame or in the same frame as Esc.
for rows in [false, true] {
for (same_frame, modifiers) in [(false, Modifiers::NONE), (true, Modifiers::NONE), (false, Modifiers::SHIFT)] {
let mut h = harness(if rows { "1:3" } else { "A:C" });
h.state_mut().session.execute("edit.copy", json!({"range": "A1:B2"})).unwrap();
h.state_mut().session.execute("edit.formatPainter", json!({"sticky": true})).unwrap();
h.run_steps(2);
assert!(h.state().session.clipboard.is_some() && h.state().session.format_painter.is_some());
let before = sizes(&h, rows);
let (start, _) = edge(&h, rows, 1);
let delta = if rows { vec2(0.0, 30.0) } else { vec2(30.0, 0.0) };
button(&mut h, start, true);
h.input_mut().events.push(Event::PointerMoved(start + delta / 2.0));
h.step();
h.input_mut().events.push(Event::PointerMoved(start + delta));
h.step();
assert_ne!(sizes(&h, rows), before, "the drag previews");
let pos = start + delta;
let esc = Event::Key { key: egui::Key::Escape, physical_key: None, pressed: true, repeat: false, modifiers };
if same_frame {
h.input_mut().events.extend([esc, Event::PointerButton { pos, button: PointerButton::Primary, pressed: false, modifiers }]);
} else {
h.input_mut().events.push(esc);
h.step();
h.input_mut()
.events
.extend([Event::PointerMoved(pos), Event::PointerButton { pos, button: PointerButton::Primary, pressed: false, modifiers }]);
}
h.step();
h.run_steps(2);
let what = format!("rows: {rows}, same frame: {same_frame}, modifiers: {modifiers:?}");
assert_eq!(sizes(&h, rows), before, "the drag is cancelled ({what})");
assert!(h.state().session.clipboard.is_some(), "the copy is still marked ({what})");
assert!(h.state().session.format_painter.is_some(), "Format Painter is still on ({what})");
}
}
}
+1 -4
View File
@@ -1417,10 +1417,7 @@ fn keytip_nv_() {
fn keytip_nw_() {
let mut h = harness(blank());
sequence(&mut h, "NW", false);
assert!(h.state().dialog.is_none(), "NW must not open a dialog");
assert!(h.state().message.is_none(), "NW must not open a message box");
assert!(h.state().toast.is_none(), "NW must not raise a toast");
assert!(h.state().session.journal.is_empty(), "NW must not journal");
assert!(h.state().session.journal.iter().any(|(id, _)| id == "insert.wordArt"), "NW must journal insert.wordArt");
}
#[test]
+2 -2
View File
@@ -1064,8 +1064,8 @@ fn the_windows_cancel_with_an_unsavable_draft_closes_the_dialog_and_says_why() {
assert_eq!(draft_text(&h).as_deref(), Some("Picked note"), "the draft is kept");
assert_eq!(selection(&h).0, 2, "the selection stays where it is");
let wb = &h.state().session.active().unwrap().wb;
for sheet in 0..3 {
assert!(wb.sheets[sheet].value(CellRef::parse("H1").unwrap()).is_empty(), "nothing ran on sheet {sheet}");
for (index, sheet) in wb.sheets[..3].iter().enumerate() {
assert!(sheet.value(CellRef::parse("H1").unwrap()).is_empty(), "nothing ran on sheet {index}");
}
}
+1 -1
View File
@@ -495,7 +495,7 @@ fn vertical_alignment_keytips_run_their_commands() {
fn pending_sequences_are_silent_no_ops() {
let mut h = harness(blank());
let journal = h.state().session.journal.len();
sequence(&mut h, &[Key::N, Key::W], false); // NW -> insert.wordArt (missing)
sequence(&mut h, &[Key::N, Key::S, Key::C], false); // NSC -> insert.screenshot (missing)
assert!(h.state().dialog.is_none(), "no dialog for a pending sequence");
assert!(h.state().message.is_none(), "no message box for a pending sequence");
assert!(h.state().toast.is_none(), "no toast for a pending sequence");
+5 -2
View File
@@ -18,12 +18,15 @@ gridcraft-core = { workspace = true }
gridcraft-formula = { workspace = true }
gridcraft-math = { workspace = true }
gridcraft-model = { workspace = true }
craft-cfb = { workspace = true }
craft-csv = { workspace = true }
craft-offcrypto = { workspace = true }
craft-opc = { workspace = true }
craft-xls = { workspace = true }
craft-xml = { workspace = true }
thiserror = { workspace = true }
quick-xml = { workspace = true }
zip = { workspace = true }
aes = { workspace = true }
sha1 = { workspace = true }
sha2 = { workspace = true }
[dev-dependencies]
-450
View File
@@ -1,450 +0,0 @@
//! Just enough of the Compound File Binary (CFB/OLE2) container to tell what a non-zip
//! spreadsheet file is: Excel wraps password-protected workbooks in a CFB holding an
//! `EncryptedPackage` stream, while Excel 97-2003 `.xls` (and other legacy Office files) are
//! CFB files without one. Every read is bounds-checked and every loop bounded.
/// The CFB signature at the start of the file.
pub(crate) const SIGNATURE: [u8; 8] = [0xD0, 0xCF, 0x11, 0xE0, 0xA1, 0xB1, 0x1A, 0xE1];
const END_OF_CHAIN: u32 = 0xFFFF_FFFE;
/// Sector ids at or above this are markers (free, end of chain, FAT/DIFAT sector), not sectors.
const MAX_REG_SECT: u32 = 0xFFFF_FFFA;
/// Directory sectors followed at most (4096 entries with 512-byte sectors).
const MAX_DIR_SECTORS: usize = 1024;
/// DIFAT sectors followed at most.
const MAX_DIFAT_SECTORS: usize = 1024;
/// Stream names that mark an encrypted (password-protected) OOXML package.
const ENCRYPTED_STREAMS: [&str; 2] = ["EncryptedPackage", "EncryptionInfo"];
fn u16_at(b: &[u8], at: usize) -> Option<u16> {
let s = b.get(at..at.checked_add(2)?)?;
Some(u16::from_le_bytes([*s.first()?, *s.get(1)?]))
}
fn u32_at(b: &[u8], at: usize) -> Option<u32> {
let s = b.get(at..at.checked_add(4)?)?;
Some(u32::from_le_bytes([*s.first()?, *s.get(1)?, *s.get(2)?, *s.get(3)?]))
}
/// Whether the compound file holds an encryption stream. Walks the directory through the FAT;
/// when the structure can't be followed (truncated or malformed), falls back to scanning the
/// bytes for the UTF-16LE stream names.
pub(crate) fn is_encrypted(bytes: &[u8]) -> bool {
match directory_names(bytes) {
Some(names) => names.iter().any(|n| ENCRYPTED_STREAMS.iter().any(|e| n.eq_ignore_ascii_case(e))),
None => ENCRYPTED_STREAMS.iter().any(|e| contains_utf16(bytes, e)),
}
}
fn contains_utf16(hay: &[u8], needle: &str) -> bool {
let n: Vec<u8> = needle.encode_utf16().flat_map(u16::to_le_bytes).collect();
!n.is_empty() && hay.windows(n.len()).any(|w| w == n.as_slice())
}
/// Names of every directory entry, or `None` when the directory can't be read.
fn directory_names(b: &[u8]) -> Option<Vec<String>> {
if !b.starts_with(&SIGNATURE) {
return None;
}
let shift = u16_at(b, 30)?;
if shift != 9 && shift != 12 {
return None;
}
let size = 1usize << shift;
let per = size / 4;
let sector = |id: u32| -> Option<&[u8]> {
let start = (id as usize).checked_add(1)?.checked_mul(size)?;
b.get(start..start.checked_add(size)?)
};
// The DIFAT: the FAT's sector ids, 109 in the header then chained DIFAT sectors.
let mut difat: Vec<u32> = (0..109).filter_map(|i| u32_at(b, 76 + i * 4)).filter(|&s| s < MAX_REG_SECT).collect();
let mut next = u32_at(b, 68)?;
for _ in 0..MAX_DIFAT_SECTORS {
if next >= MAX_REG_SECT {
break;
}
let s = sector(next)?;
difat.extend((0..per - 1).filter_map(|i| u32_at(s, i * 4)).filter(|&x| x < MAX_REG_SECT));
next = u32_at(s, (per - 1) * 4)?;
}
let fat_next = |id: u32| -> Option<u32> {
let idx = id as usize;
let fat_sector = *difat.get(idx / per)?;
u32_at(sector(fat_sector)?, (idx % per) * 4)
};
let mut names = Vec::new();
let mut id = u32_at(b, 48)?;
for _ in 0..MAX_DIR_SECTORS {
if id == END_OF_CHAIN {
return Some(names);
}
if id >= MAX_REG_SECT {
return None;
}
let s = sector(id)?;
for e in s.as_chunks::<128>().0 {
let len = usize::from(u16_at(e, 64)?);
if !(2..=64).contains(&len) {
continue;
}
let units: Vec<u16> = (0..(len - 2) / 2).filter_map(|i| u16_at(e, i * 2)).collect();
names.push(String::from_utf16_lossy(&units));
}
id = fat_next(id)?;
}
// A directory longer than the bound (or a cyclic chain): report what was read.
Some(names)
}
/// Directory entry type of a stream object ([MS-CFB] 2.6.1).
const STREAM_OBJECT: u8 = 2;
/// "No entry" in the directory tree (NOSTREAM).
const NO_STREAM: u32 = 0xFFFF_FFFF;
/// Streams larger than this are refused (the format allows more, a workbook never needs it).
const MAX_STREAM: u64 = 1 << 30;
/// One directory entry ([MS-CFB] 2.6).
struct DirEntry {
name: String,
kind: u8,
left: u32,
right: u32,
child: u32,
start: u32,
size: u64,
}
/// A parsed compound file: the FAT, the mini FAT and the directory, all bounds-checked.
/// Implemented from Microsoft's public [MS-CFB] specification.
pub(crate) struct Compound<'a> {
bytes: &'a [u8],
sector_size: usize,
mini_sector_size: usize,
mini_cutoff: u64,
fat: Vec<u32>,
mini_fat: Vec<u32>,
entries: Vec<DirEntry>,
}
impl<'a> Compound<'a> {
/// Parses the header, FAT, mini FAT and directory. Errors on anything that can't be followed.
pub(crate) fn open(b: &'a [u8]) -> Result<Compound<'a>, String> {
let bad = |m: &str| format!("damaged compound file: {m}");
if !b.starts_with(&SIGNATURE) {
return Err(bad("missing signature"));
}
let major = u16_at(b, 26).ok_or_else(|| bad("truncated header"))?;
let shift = u16_at(b, 30).ok_or_else(|| bad("truncated header"))?;
if shift != 9 && shift != 12 {
return Err(bad("unsupported sector size"));
}
let mini_shift = u16_at(b, 32).ok_or_else(|| bad("truncated header"))?;
if mini_shift != 6 {
return Err(bad("unsupported mini sector size"));
}
let sector_size = 1usize << shift;
let per = sector_size / 4;
// Sectors that physically exist after the header sector: every chain is bounded by it.
let sectors = (b.len() / sector_size).saturating_sub(1);
let sector = |id: u32| -> Option<&[u8]> {
let start = (id as usize).checked_add(1)?.checked_mul(sector_size)?;
b.get(start..start.checked_add(sector_size)?)
};
// DIFAT: 109 FAT sector ids in the header, then chained DIFAT sectors.
let mut fat_sectors: Vec<u32> = (0..109).filter_map(|i| u32_at(b, 76 + i * 4)).filter(|&s| s < MAX_REG_SECT).collect();
let mut next = u32_at(b, 68).ok_or_else(|| bad("truncated header"))?;
let mut hops = 0usize;
while next < MAX_REG_SECT {
hops += 1;
if hops > sectors.min(MAX_DIFAT_SECTORS * 64) {
return Err(bad("DIFAT chain loops"));
}
let s = sector(next).ok_or_else(|| bad("DIFAT sector out of range"))?;
fat_sectors.extend((0..per - 1).filter_map(|i| u32_at(s, i * 4)).filter(|&x| x < MAX_REG_SECT));
next = u32_at(s, (per - 1) * 4).ok_or_else(|| bad("truncated DIFAT sector"))?;
}
if fat_sectors.len() > sectors {
return Err(bad("more FAT sectors than the file holds"));
}
let mut fat = Vec::with_capacity(fat_sectors.len().saturating_mul(per));
for id in fat_sectors {
let s = sector(id).ok_or_else(|| bad("FAT sector out of range"))?;
fat.extend((0..per).filter_map(|i| u32_at(s, i * 4)));
}
let mut cf = Compound { bytes: b, sector_size, mini_sector_size: 64, mini_cutoff: 4096, fat, mini_fat: vec![], entries: vec![] };
if let Some(c) = u32_at(b, 56) {
cf.mini_cutoff = u64::from(c);
}
// Directory.
let dir_start = u32_at(b, 48).ok_or_else(|| bad("truncated header"))?;
let dir = cf.read_chain(dir_start, u64::MAX)?;
for e in dir.as_chunks::<128>().0 {
let len = usize::from(u16_at(e, 64).unwrap_or(0)).min(64);
let units: Vec<u16> = (0..len.saturating_sub(2) / 2).filter_map(|i| u16_at(e, i * 2)).collect();
let mut size = u32_at(e, 120).map(u64::from).unwrap_or(0);
if major >= 4 {
size |= u64::from(u32_at(e, 124).unwrap_or(0)) << 32;
}
cf.entries.push(DirEntry {
name: String::from_utf16_lossy(&units),
kind: e.get(66).copied().unwrap_or(0),
left: u32_at(e, 68).unwrap_or(NO_STREAM),
right: u32_at(e, 72).unwrap_or(NO_STREAM),
child: u32_at(e, 76).unwrap_or(NO_STREAM),
start: u32_at(e, 116).unwrap_or(END_OF_CHAIN),
size,
});
}
if cf.entries.first().is_none_or(|r| r.kind != 5) {
return Err(bad("missing root entry"));
}
// Mini FAT.
let mini_start = u32_at(b, 60).ok_or_else(|| bad("truncated header"))?;
if mini_start < MAX_REG_SECT {
let raw = cf.read_chain(mini_start, u64::MAX)?;
cf.mini_fat = (0..raw.len() / 4).filter_map(|i| u32_at(&raw, i * 4)).collect();
}
Ok(cf)
}
/// Follows a FAT chain from `start`, reading at most `limit` bytes. A chain longer than the
/// FAT (so a cycle) or pointing outside the file is an error. The file may end inside the
/// chain's last sector (some writers don't pad it, and Excel opens those files): that sector
/// reads short, and the caller checks the stream's declared size against what was read.
fn read_chain(&self, start: u32, limit: u64) -> Result<Vec<u8>, String> {
let limit = usize::try_from(limit.min(MAX_STREAM)).unwrap_or(usize::MAX);
let mut out = Vec::new();
let mut id = start;
let mut seen = vec![false; self.fat.len()];
let mut cut = false;
while id != END_OF_CHAIN && out.len() < limit {
if cut {
return Err("damaged compound file: sector out of range (the file may be truncated)".into());
}
if id >= MAX_REG_SECT {
return Err("damaged compound file: broken sector chain".into());
}
match seen.get_mut(id as usize) {
Some(v) if !*v => *v = true,
Some(_) => return Err("damaged compound file: sector chain loops".into()),
None => return Err("damaged compound file: sector missing from the FAT".into()),
}
let start = (id as usize).checked_add(1).and_then(|i| i.checked_mul(self.sector_size));
let s = start
.and_then(|s| self.bytes.get(s..s.checked_add(self.sector_size)?.min(self.bytes.len())))
.ok_or("damaged compound file: sector out of range (the file may be truncated)")?;
cut = s.len() < self.sector_size;
out.extend_from_slice(s);
id = *self.fat.get(id as usize).ok_or("damaged compound file: sector missing from the FAT")?;
}
Ok(out)
}
/// Follows a mini FAT chain inside the mini stream container. Like `read_chain`, the
/// container may end inside the chain's last mini sector.
fn read_mini_chain(&self, container: &[u8], start: u32, limit: usize) -> Result<Vec<u8>, String> {
let mut out = Vec::new();
let mut id = start;
let mut seen = vec![false; self.mini_fat.len()];
let mut cut = false;
while id != END_OF_CHAIN && out.len() < limit {
if cut {
return Err("damaged compound file: mini sector out of range".into());
}
if id >= MAX_REG_SECT {
return Err("damaged compound file: broken mini sector chain".into());
}
match seen.get_mut(id as usize) {
Some(v) if !*v => *v = true,
Some(_) => return Err("damaged compound file: mini sector chain loops".into()),
None => return Err("damaged compound file: mini sector missing from the mini FAT".into()),
}
let at = (id as usize).checked_mul(self.mini_sector_size);
let s = at
.and_then(|a| container.get(a..a.checked_add(self.mini_sector_size)?.min(container.len())))
.ok_or("damaged compound file: mini sector out of range")?;
cut = s.len() < self.mini_sector_size;
out.extend_from_slice(s);
id = *self.mini_fat.get(id as usize).ok_or("damaged compound file: mini sector missing from the mini FAT")?;
}
Ok(out)
}
/// The contents of the stream called `name` (case-insensitive) directly under the root
/// storage, or `None` when there is no such stream.
/// The directory entry of the stream called `name` (case-insensitive) directly under the
/// root storage.
fn root_entry(&self, name: &str) -> Result<Option<&DirEntry>, String> {
let Some(root) = self.entries.first() else { return Ok(None) };
// Walk the root's children (a red-black tree through left/right siblings), bounded.
let mut stack = vec![root.child];
let mut seen = 0usize;
while let Some(i) = stack.pop() {
if i == NO_STREAM {
continue;
}
seen += 1;
if seen > self.entries.len() {
return Err("damaged compound file: directory tree loops".into());
}
let Some(e) = self.entries.get(i as usize) else { continue };
if e.kind == STREAM_OBJECT && e.name.eq_ignore_ascii_case(name) {
return Ok(Some(e));
}
stack.push(e.left);
stack.push(e.right);
}
Ok(None)
}
/// The contents of the stream called `name` (case-insensitive) directly under the root
/// storage, or `None` when there is no such stream.
pub(crate) fn root_stream(&self, name: &str) -> Result<Option<Vec<u8>>, String> {
let Some(root) = self.entries.first() else { return Ok(None) };
let found = self.root_entry(name)?;
let Some(e) = found else { return Ok(None) };
if e.size > MAX_STREAM {
return Err("the workbook stream is larger than 1 GiB".into());
}
let size = usize::try_from(e.size).map_err(|_| "stream too large".to_string())?;
let mut data = if e.size < self.mini_cutoff {
let container = self.read_chain(root.start, root.size)?;
self.read_mini_chain(&container, e.start, size)?
} else {
self.read_chain(e.start, e.size)?
};
if data.len() < size {
return Err("damaged compound file: a stream is shorter than its declared size (the file may be truncated)".into());
}
data.truncate(size);
Ok(Some(data))
}
}
#[cfg(test)]
pub(crate) mod tests {
use super::*;
/// A minimal valid compound file (512-byte sectors: FAT at sector 0, directory at sector 1)
/// whose directory holds a root entry plus the given stream names.
pub(crate) fn build(streams: &[&str]) -> Vec<u8> {
let mut b = vec![0u8; 512 * 3];
b[..8].copy_from_slice(&SIGNATURE);
b[24..26].copy_from_slice(&0x3Eu16.to_le_bytes()); // minor version
b[26..28].copy_from_slice(&3u16.to_le_bytes()); // major version
b[28..30].copy_from_slice(&0xFFFEu16.to_le_bytes()); // byte order
b[30..32].copy_from_slice(&9u16.to_le_bytes()); // sector shift
b[32..34].copy_from_slice(&6u16.to_le_bytes()); // mini sector shift
b[44..48].copy_from_slice(&1u32.to_le_bytes()); // FAT sectors
b[48..52].copy_from_slice(&1u32.to_le_bytes()); // first directory sector
b[60..64].copy_from_slice(&END_OF_CHAIN.to_le_bytes()); // no mini FAT
b[68..72].copy_from_slice(&END_OF_CHAIN.to_le_bytes()); // no DIFAT sectors
for i in 0..109 {
b[76 + i * 4..80 + i * 4].copy_from_slice(&0xFFFF_FFFFu32.to_le_bytes());
}
b[76..80].copy_from_slice(&0u32.to_le_bytes()); // FAT lives in sector 0
// FAT (sector 0, at byte 512): sector 0 is a FAT sector, sector 1 ends its chain.
for i in 0..128 {
b[512 + i * 4..516 + i * 4].copy_from_slice(&0xFFFF_FFFFu32.to_le_bytes());
}
b[512..516].copy_from_slice(&0xFFFF_FFFDu32.to_le_bytes());
b[516..520].copy_from_slice(&END_OF_CHAIN.to_le_bytes());
// Directory (sector 1, at byte 1024).
for (k, name) in std::iter::once("Root Entry").chain(streams.iter().copied()).take(4).enumerate() {
let e = 1024 + k * 128;
let units: Vec<u16> = name.encode_utf16().take(31).collect();
for (i, u) in units.iter().enumerate() {
b[e + i * 2..e + i * 2 + 2].copy_from_slice(&u.to_le_bytes());
}
b[e + 64..e + 66].copy_from_slice(&((units.len() as u16 + 1) * 2).to_le_bytes());
b[e + 66] = if k == 0 { 5 } else { 2 };
}
b
}
#[test]
fn reads_directory_names() {
let b = build(&["EncryptionInfo", "EncryptedPackage"]);
assert_eq!(directory_names(&b), Some(vec!["Root Entry".into(), "EncryptionInfo".into(), "EncryptedPackage".into()]));
assert!(is_encrypted(&b));
assert!(!is_encrypted(&build(&["Workbook", "\u{5}SummaryInformation"])));
}
#[test]
fn hostile_headers_never_panic() {
let mut b = build(&["Workbook"]);
for at in [30usize, 48, 68, 76, 512, 516, 1024 + 64] {
for v in [0u32, 1, 7, 0xFFFF_FFFA, 0xFFFF_FFFF, 0x7FFF_FFFF] {
let mut c = b.clone();
c[at..at + 4].copy_from_slice(&v.to_le_bytes());
let _ = is_encrypted(&c);
for cut in [0usize, 8, 100, 600, 1100] {
let _ = is_encrypted(&c[..cut.min(c.len())]);
}
}
}
// A directory chain that loops back on itself stops at the bound.
b[516..520].copy_from_slice(&1u32.to_le_bytes());
assert!(!is_encrypted(&b));
}
/// `compound_file` with one regular-sector stream of `len` bytes (FAT in sector 0, directory
/// in sector 1, the stream from sector 2), cut right after the stream's last byte, the way
/// some writers leave the final sector unpadded.
fn unpadded(len: usize) -> (Vec<u8>, Vec<u8>) {
let data: Vec<u8> = (0..len).map(|i| (i % 251) as u8).collect();
let mut file = crate::testing::compound_file(&[("Workbook", &data)]);
file.truncate(512 * 3 + len);
(file, data)
}
#[test]
fn an_unpadded_last_sector_still_reads() {
let (file, data) = unpadded(5000);
assert_ne!(file.len() % 512, 0);
let cf = Compound::open(&file).unwrap();
assert_eq!(cf.root_stream("Workbook").unwrap().unwrap(), data);
// Bytes the stream needs that the file doesn't hold are still damage.
let cf = Compound::open(&file[..file.len() - 1]).unwrap();
assert!(cf.root_stream("Workbook").is_err());
}
#[test]
fn an_unpadded_last_mini_sector_still_reads() {
let data: Vec<u8> = (0..1000u32).map(|i| (i % 251) as u8).collect();
let mut file = crate::testing::compound_file(&[("Workbook", &data)]);
// Header, FAT, directory, mini FAT, then the mini stream holding the stream's bytes.
file.truncate(512 * 4 + data.len());
let cf = Compound::open(&file).unwrap();
assert_eq!(cf.root_stream("Workbook").unwrap().unwrap(), data);
let cf = Compound::open(&file[..file.len() - 1]).unwrap();
assert!(cf.root_stream("Workbook").is_err());
}
#[test]
fn a_cut_sector_inside_a_chain_is_damage() {
let (mut file, _) = unpadded(5000);
// The cut sector (11) points on to the directory sector and the stream claims those
// bytes too: the partial sector would splice 392 bytes into the middle of the stream.
file[512 + 11 * 4..512 + 12 * 4].copy_from_slice(&1u32.to_le_bytes());
file[1024 + 128 + 120..1024 + 128 + 124].copy_from_slice(&5512u32.to_le_bytes());
let cf = Compound::open(&file).unwrap();
assert!(cf.root_stream("Workbook").is_err());
}
#[test]
fn a_cut_mini_sector_inside_a_chain_is_damage() {
let small = [7u8; 100];
let data: Vec<u8> = (0..1000u32).map(|i| (i % 251) as u8).collect();
let mut file = crate::testing::compound_file(&[("Small", &small), ("Workbook", &data)]);
// "Small" holds mini sectors 0-1, "Workbook" 2-17; the mini stream starts at sector 3.
file.truncate(512 * 4 + 128 + data.len());
assert_eq!(Compound::open(&file).unwrap().root_stream("Workbook").unwrap().unwrap(), data);
// The cut mini sector (17) points on to sector 0 and the stream claims those bytes too.
file[1536 + 17 * 4..1536 + 18 * 4].copy_from_slice(&0u32.to_le_bytes());
file[1024 + 2 * 128 + 120..1024 + 2 * 128 + 124].copy_from_slice(&1064u32.to_le_bytes());
let cf = Compound::open(&file).unwrap();
assert!(cf.root_stream("Workbook").is_err());
}
}
+19 -511
View File
@@ -1,42 +1,7 @@
//! Password-encrypted workbooks. Excel wraps the ordinary OOXML package in a compound file with
//! two streams: `EncryptionInfo` (how the key comes from the password, and a verifier that tells
//! a wrong password) and `EncryptedPackage` (the package, encrypted with AES). Agile encryption
//! (Excel 2010 onward) and Standard encryption (Excel 2007) are read; RC4, Extensible and
//! certificate-only encryption are reported as unsupported.
//!
//! Clean-room from Microsoft's public [MS-OFFCRYPTO] specification. Every read is
//! bounds-checked, and the costly inputs (spin count, stated sizes) are capped.
use aes::cipher::{BlockCipherDecrypt, KeyInit};
use sha2::Digest;
use crate::cfb::Compound;
use crate::xml::El;
/// Spin counts above this are refused (the cap sheet-protection hashes use too).
const MAX_SPIN_COUNT: u32 = 10_000_000;
/// Standard encryption always iterates this many times.
const STANDARD_SPIN_COUNT: u32 = 50_000;
/// Agile packages are encrypted in segments of this many bytes, each with its own IV.
pub(crate) const SEGMENT: usize = 4096;
/// AES block size.
const BLOCK: usize = 16;
/// Agile block keys ([MS-OFFCRYPTO] 2.3.4.13): one per value derived from the password.
pub(crate) const BLOCK_VERIFIER_INPUT: [u8; 8] = [0xFE, 0xA7, 0xD2, 0x76, 0x3B, 0x4B, 0x9E, 0x79];
pub(crate) const BLOCK_VERIFIER_HASH: [u8; 8] = [0xD7, 0xAA, 0x0F, 0x6D, 0x30, 0x61, 0x34, 0x4E];
pub(crate) const BLOCK_KEY: [u8; 8] = [0x14, 0x6E, 0x0B, 0xE7, 0xAB, 0xAC, 0xD0, 0xD6];
/// The two dataIntegrity block keys ([MS-OFFCRYPTO] 2.3.4.14), one per HMAC field.
pub(crate) const BLOCK_HMAC_KEY: [u8; 8] = [0x5F, 0xB2, 0xAD, 0x01, 0x0C, 0xB9, 0xE1, 0xF6];
pub(crate) const BLOCK_HMAC_VALUE: [u8; 8] = [0xA0, 0x67, 0x7F, 0x02, 0xB2, 0x2C, 0x84, 0x33];
/// The password key encryptor's URI (the other kind is certificate-based).
pub(crate) const PASSWORD_ENCRYPTOR: &str = "http://schemas.microsoft.com/office/2006/keyEncryptor/password";
/// Standard encryption header flags ([MS-OFFCRYPTO] 2.3.1).
pub(crate) const F_CRYPTOAPI: u32 = 0x04;
const F_EXTERNAL: u32 = 0x10;
pub(crate) const F_AES: u32 = 0x20;
//! two streams, `EncryptionInfo` and `EncryptedPackage`. `craft-offcrypto` reads them (Agile and
//! Standard encryption; RC4, Extensible, certificate-only and rights-managed files are named as
//! unsupported); this module keeps GridCraft's error type.
/// Why an encrypted workbook couldn't be decrypted.
#[derive(Clone, Debug, PartialEq, Eq, thiserror::Error)]
@@ -50,489 +15,32 @@ pub enum DecryptError {
Malformed(String),
}
fn malformed(m: impl Into<String>) -> DecryptError {
DecryptError::Malformed(m.into())
impl From<craft_offcrypto::Error> for DecryptError {
fn from(e: craft_offcrypto::Error) -> Self {
use craft_offcrypto::Error as E;
match e {
E::WrongPassword | E::PasswordRequired => DecryptError::WrongPassword,
E::Unsupported(scheme) => DecryptError::Unsupported(scheme),
E::Malformed(detail) | E::TooLarge(detail) => DecryptError::Malformed(detail),
e => DecryptError::Malformed(e.to_string()),
}
}
}
/// The plain OOXML package (a zip) inside an encrypted workbook.
pub fn decrypt(bytes: &[u8], password: &str) -> Result<Vec<u8>, DecryptError> {
let (info, package) = streams(bytes)?;
decrypt_streams(&info, &package, password)
craft_offcrypto::decrypt(bytes, Some(password)).map_err(DecryptError::from)
}
/// Whether the right password could decrypt this encrypted workbook, checked without one. An
/// error names a scheme no password opens (RC4, rights management…) or damage, so callers can
/// say so instead of asking for a password first.
pub fn check_encryption(bytes: &[u8]) -> Result<(), DecryptError> {
let (info, _) = streams(bytes)?;
match scheme(&info)? {
Scheme::Agile(xml) => agile_descriptor(xml).map(|_| ()),
Scheme::Standard(info) => standard_header(info).map(|_| ()),
}
craft_offcrypto::check(bytes).map_err(DecryptError::from)
}
/// The `EncryptionInfo` and `EncryptedPackage` streams. Rights management (IRM) keeps the
/// package but describes its encryption elsewhere, with no `EncryptionInfo`.
fn streams(bytes: &[u8]) -> Result<(Vec<u8>, Vec<u8>), DecryptError> {
let cf = Compound::open(bytes).map_err(malformed)?;
let info = cf.root_stream("EncryptionInfo").map_err(malformed)?;
let package = cf.root_stream("EncryptedPackage").map_err(malformed)?.ok_or_else(|| malformed("no EncryptedPackage stream"))?;
let info = info.ok_or_else(|| DecryptError::Unsupported("rights-management".into()))?;
Ok((info, package))
/// Whether a compound file holds an encryption stream, at any depth. A damaged directory falls
/// back to a search for the stream names, so the file is still reported as encrypted.
pub(crate) fn is_encrypted(bytes: &[u8]) -> bool {
["EncryptedPackage", "EncryptionInfo"].iter().any(|n| craft_cfb::has_entry_named(bytes, n, &craft_cfb::Limits::default()))
}
/// Decrypts from the two streams' contents.
pub(crate) fn decrypt_streams(info: &[u8], package: &[u8], password: &str) -> Result<Vec<u8>, DecryptError> {
match scheme(info)? {
Scheme::Agile(xml) => agile(xml, package, password),
Scheme::Standard(info) => standard(info, package, password),
}
}
/// The two schemes read, with the part of `EncryptionInfo` each one parses.
enum Scheme<'a> {
/// The XML descriptor after the version and reserved field.
Agile(&'a [u8]),
/// Everything after the version.
Standard(&'a [u8]),
}
fn scheme(info: &[u8]) -> Result<Scheme<'_>, DecryptError> {
let short = || malformed("truncated EncryptionInfo");
let major = u16_at(info, 0).ok_or_else(short)?;
let minor = u16_at(info, 2).ok_or_else(short)?;
match (major, minor) {
(4, 4) => Ok(Scheme::Agile(info.get(8..).ok_or_else(short)?)),
(2..=4, 2) => Ok(Scheme::Standard(info.get(4..).ok_or_else(short)?)),
(3 | 4, 3) => Err(DecryptError::Unsupported("Extensible".into())),
_ => Err(DecryptError::Unsupported(format!("version {major}.{minor}"))),
}
}
fn u16_at(b: &[u8], at: usize) -> Option<u16> {
let s = b.get(at..at.checked_add(2)?)?;
Some(u16::from_le_bytes([*s.first()?, *s.get(1)?]))
}
fn u32_at(b: &[u8], at: usize) -> Option<u32> {
let s = b.get(at..at.checked_add(4)?)?;
Some(u32::from_le_bytes([*s.first()?, *s.get(1)?, *s.get(2)?, *s.get(3)?]))
}
/// The plaintext size an `EncryptedPackage` states, and the ciphertext after it. A size larger
/// than the ciphertext is damage, so it never drives an allocation.
fn package_parts(package: &[u8]) -> Result<(usize, &[u8]), DecryptError> {
let head: [u8; 8] = package.get(..8).and_then(|h| h.try_into().ok()).ok_or_else(|| malformed("truncated EncryptedPackage"))?;
let data = package.get(8..).unwrap_or_default();
let size = usize::try_from(u64::from_le_bytes(head)).unwrap_or(usize::MAX);
if size > data.len() {
return Err(malformed("the package is shorter than its stated size"));
}
Ok((size, data))
}
// ---- hashing -------------------------------------------------------------------------------
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum Hash {
Sha1,
Sha256,
Sha384,
Sha512,
}
impl Hash {
fn from_name(name: &str) -> Result<Hash, DecryptError> {
Ok(match name {
"SHA1" | "SHA-1" => Hash::Sha1,
"SHA256" | "SHA-256" => Hash::Sha256,
"SHA384" | "SHA-384" => Hash::Sha384,
"SHA512" | "SHA-512" => Hash::Sha512,
other => return Err(DecryptError::Unsupported(format!("{other} hash"))),
})
}
/// The digest of the concatenated `parts`.
pub(crate) fn digest(self, parts: &[&[u8]]) -> Vec<u8> {
fn run<D: Digest>(parts: &[&[u8]]) -> Vec<u8> {
let mut d = D::new();
for p in parts {
d.update(p);
}
d.finalize().to_vec()
}
match self {
Hash::Sha1 => run::<sha1::Sha1>(parts),
Hash::Sha256 => run::<sha2::Sha256>(parts),
Hash::Sha384 => run::<sha2::Sha384>(parts),
Hash::Sha512 => run::<sha2::Sha512>(parts),
}
}
/// The digest's length in bytes.
fn digest_len(self) -> usize {
match self {
Hash::Sha1 => 20,
Hash::Sha256 => 32,
Hash::Sha384 => 48,
Hash::Sha512 => 64,
}
}
/// The bytes the algorithm's compression function takes: 128 for SHA-384/512, else 64.
fn block_size(self) -> usize {
match self {
Hash::Sha384 | Hash::Sha512 => 128,
Hash::Sha1 | Hash::Sha256 => 64,
}
}
}
/// H0 = H(salt ‖ UTF-16LE(password)), then Hn = H(LE32(n - 1) ‖ Hn-1) for `spin` rounds.
pub(crate) fn iterated_hash(hash: Hash, password: &str, salt: &[u8], spin: u32) -> Vec<u8> {
let pw: Vec<u8> = password.encode_utf16().flat_map(u16::to_le_bytes).collect();
let mut h = hash.digest(&[salt, &pw]);
for i in 0..spin {
h = hash.digest(&[&i.to_le_bytes(), &h]);
}
h
}
/// H(a ‖ b), truncated or padded with 0x36 to `len`: Agile's derived keys and IVs.
pub(crate) fn derive(hash: Hash, a: &[u8], b: &[u8], len: usize) -> Vec<u8> {
let mut k = hash.digest(&[a, b]);
k.resize(len, 0x36);
k
}
/// HMAC-SHA-`hash` ([RFC2104]), as Agile's dataIntegrity uses.
pub(crate) fn hmac(hash: Hash, key: &[u8], message: &[u8]) -> Vec<u8> {
let bs = hash.block_size();
let mut k = key.to_vec();
if k.len() > bs {
k = hash.digest(&[&k]);
}
k.resize(bs, 0);
let pad = |x: u8| k.iter().map(|b| b ^ x).collect::<Vec<u8>>();
let inner = hash.digest(&[&pad(0x36), message]);
hash.digest(&[&pad(0x5C), &inner])
}
/// The Standard encryption key ([MS-OFFCRYPTO] 2.3.4.7): SHA-1 iterated 50,000 times, the
/// block-0 hash, then the 0x36 / 0x5C expansion, truncated to `key_len`.
pub(crate) fn standard_key(password: &str, salt: &[u8], key_len: usize) -> Vec<u8> {
let h = iterated_hash(Hash::Sha1, password, salt, STANDARD_SPIN_COUNT);
let h = Hash::Sha1.digest(&[&h, &0u32.to_le_bytes()]);
let expand = |pad: u8| {
let mut buf = [pad; 64];
for (b, x) in buf.iter_mut().zip(&h) {
*b ^= x;
}
Hash::Sha1.digest(&[&buf])
};
let mut key = expand(0x36);
key.extend(expand(0x5C));
key.truncate(key_len);
key
}
// ---- AES -----------------------------------------------------------------------------------
enum Aes {
A128(aes::Aes128),
A192(aes::Aes192),
A256(aes::Aes256),
}
impl Aes {
fn new(key: &[u8]) -> Result<Aes, DecryptError> {
let bad = |_| malformed("bad AES key");
Ok(match key.len() {
16 => Aes::A128(aes::Aes128::new_from_slice(key).map_err(bad)?),
24 => Aes::A192(aes::Aes192::new_from_slice(key).map_err(bad)?),
32 => Aes::A256(aes::Aes256::new_from_slice(key).map_err(bad)?),
n => return Err(malformed(format!("a {n}-byte AES key"))),
})
}
fn decrypt_block(&self, block: &mut [u8; BLOCK]) {
let b = aes::Block::cast_from_core_mut(block);
match self {
Aes::A128(c) => c.decrypt_block(b),
Aes::A192(c) => c.decrypt_block(b),
Aes::A256(c) => c.decrypt_block(b),
}
}
/// ECB: each block on its own. `data` must be whole blocks.
fn decrypt_ecb(&self, data: &mut [u8]) -> Result<(), DecryptError> {
let (blocks, rest) = data.as_chunks_mut::<BLOCK>();
if !rest.is_empty() {
return Err(malformed("ciphertext isn't whole AES blocks"));
}
blocks.iter_mut().for_each(|b| self.decrypt_block(b));
Ok(())
}
/// CBC: each block XORed with the previous ciphertext block (the IV for the first).
fn decrypt_cbc(&self, iv: &[u8], data: &mut [u8]) -> Result<(), DecryptError> {
let mut prev: [u8; BLOCK] = iv.try_into().map_err(|_| malformed("an IV that isn't one AES block"))?;
let (blocks, rest) = data.as_chunks_mut::<BLOCK>();
if !rest.is_empty() {
return Err(malformed("ciphertext isn't whole AES blocks"));
}
for b in blocks {
let cipher = *b;
self.decrypt_block(b);
b.iter_mut().zip(prev).for_each(|(x, p)| *x ^= p);
prev = cipher;
}
Ok(())
}
}
// ---- Standard encryption -------------------------------------------------------------------
/// What a Standard `EncryptionInfo` holds: the key size and the password verifier.
struct StandardHeader<'a> {
key_len: usize,
salt: &'a [u8],
verifier: &'a [u8],
verifier_hash: &'a [u8],
}
/// `info` starts after the version: flags, header size, header, verifier.
fn standard_header(info: &[u8]) -> Result<StandardHeader<'_>, DecryptError> {
let short = || malformed("truncated Standard encryption header");
let flags = u32_at(info, 0).ok_or_else(short)?;
if flags & F_EXTERNAL != 0 {
return Err(DecryptError::Unsupported("external".into()));
}
if flags & F_AES == 0 || flags & F_CRYPTOAPI == 0 {
return Err(DecryptError::Unsupported("RC4".into()));
}
let header_size = usize::try_from(u32_at(info, 4).ok_or_else(short)?).map_err(|_| short())?;
let header = info.get(8..8usize.checked_add(header_size).ok_or_else(short)?).ok_or_else(short)?;
let alg_id = u32_at(header, 8).ok_or_else(short)?;
let alg_hash = u32_at(header, 12).ok_or_else(short)?;
let key_bits = u32_at(header, 16).ok_or_else(short)?;
let key_len = match (alg_id, key_bits) {
(0x660E | 0, 128) => 16,
(0x660F, 192) => 24,
(0x6610, 256) => 32,
(0x660E..=0x6610, _) => return Err(malformed(format!("AES with a {key_bits}-bit key"))),
_ => return Err(DecryptError::Unsupported(format!("cipher 0x{alg_id:04X}"))),
};
if alg_hash != 0x8004 && alg_hash != 0 {
return Err(DecryptError::Unsupported(format!("hash 0x{alg_hash:04X}")));
}
// Verifier: salt size, salt, encrypted verifier, verifier hash size, encrypted hash.
let v = info.get(8 + header_size..).ok_or_else(short)?;
if u32_at(v, 0) != Some(16) {
return Err(malformed("a Standard salt that isn't 16 bytes"));
}
let salt = v.get(4..20).ok_or_else(short)?;
let verifier = v.get(20..36).ok_or_else(short)?;
let verifier_hash = v.get(40..72).ok_or_else(short)?;
Ok(StandardHeader { key_len, salt, verifier, verifier_hash })
}
fn standard(info: &[u8], package: &[u8], password: &str) -> Result<Vec<u8>, DecryptError> {
let h = standard_header(info)?;
let (mut verifier, mut verifier_hash) = (h.verifier.to_vec(), h.verifier_hash.to_vec());
let aes = Aes::new(&standard_key(password, h.salt, h.key_len))?;
aes.decrypt_ecb(&mut verifier)?;
aes.decrypt_ecb(&mut verifier_hash)?;
if verifier_hash.get(..20) != Some(Hash::Sha1.digest(&[&verifier]).as_slice()) {
return Err(DecryptError::WrongPassword);
}
let (size, data) = package_parts(package)?;
let mut out = data.get(..size.div_ceil(BLOCK) * BLOCK).ok_or_else(|| malformed("the package isn't whole AES blocks"))?.to_vec();
aes.decrypt_ecb(&mut out)?;
out.truncate(size);
Ok(out)
}
// ---- Agile encryption ----------------------------------------------------------------------
/// The cipher parameters shared by `keyData` and a password `encryptedKey` element.
pub(crate) struct Params {
pub(crate) hash: Hash,
pub(crate) salt: Vec<u8>,
pub(crate) key_len: usize,
}
impl Params {
fn read(e: &El) -> Result<Params, DecryptError> {
let cipher = e.attr("cipherAlgorithm").unwrap_or("AES");
if cipher != "AES" {
return Err(DecryptError::Unsupported(cipher.into()));
}
let chaining = e.attr("cipherChaining").unwrap_or("ChainingModeCBC");
if chaining != "ChainingModeCBC" {
return Err(DecryptError::Unsupported(format!("AES {chaining}")));
}
if e.attr_u32("blockSize") != Some(16) {
return Err(malformed("an AES block size that isn't 16"));
}
let key_len = match e.attr_u32("keyBits") {
Some(128) => 16,
Some(192) => 24,
Some(256) => 32,
other => return Err(malformed(format!("AES with keyBits {}", other.unwrap_or(0)))),
};
let hash = Hash::from_name(e.attr("hashAlgorithm").unwrap_or("SHA1"))?;
let salt = base64(e.attr("saltValue").ok_or_else(|| malformed("no saltValue"))?)?;
if salt.is_empty() || salt.len() > 64 {
return Err(malformed("a salt that isn't 1 to 64 bytes"));
}
Ok(Params { hash, salt, key_len })
}
/// An IV: H(salt ‖ block key) for a block key, else the salt, sized to one AES block.
pub(crate) fn iv(&self, block_key: Option<&[u8]>) -> Vec<u8> {
match block_key {
Some(k) => derive(self.hash, &self.salt, k, BLOCK),
None => {
let mut iv = self.salt.clone();
iv.resize(BLOCK, 0x36);
iv
}
}
}
}
/// What an Agile descriptor holds: the package's cipher, the password key encryptor with its
/// three encrypted values (verifier input, verifier hash, package key), and the package's
/// integrity pair, which Excel always writes and the reader requires.
struct AgileDescriptor {
key_data: Params,
key: Params,
spin: u32,
verifier_input: Vec<u8>,
verifier_hash: Vec<u8>,
key_value: Vec<u8>,
hmac_key: Vec<u8>,
hmac_value: Vec<u8>,
}
/// `xml` is the XML encryption descriptor after the 8-byte version and reserved field.
fn agile_descriptor(xml: &[u8]) -> Result<AgileDescriptor, DecryptError> {
let root = crate::xml::parse(xml).map_err(|e| malformed(e.to_string()))?;
let key_data = Params::read(root.child("keyData").ok_or_else(|| malformed("no keyData"))?)?;
let encryptor = root
.child("keyEncryptors")
.and_then(|k| k.kids("keyEncryptor").find(|e| e.attr("uri") == Some(PASSWORD_ENCRYPTOR)))
.ok_or_else(|| DecryptError::Unsupported("certificate".into()))?
.child("encryptedKey")
.ok_or_else(|| malformed("no encryptedKey"))?;
let key = Params::read(encryptor)?;
let spin = encryptor.attr_u32("spinCount").ok_or_else(|| malformed("no spinCount"))?;
if spin > MAX_SPIN_COUNT {
return Err(malformed(format!("a spin count of {spin}, more than the {MAX_SPIN_COUNT} allowed")));
}
let field = |name: &str| base64(encryptor.attr(name).ok_or_else(|| malformed(format!("no {name}")))?);
// A descriptor without dataIntegrity can't prove its ciphertext is the one the password
// protects: the verifier only authenticates the key, not the package ([MS-OFFCRYPTO] 2.3.4.14).
let integrity = root.child("dataIntegrity").ok_or_else(|| malformed("no dataIntegrity"))?;
let hmac_field = |attr: &str| {
let v = base64(integrity.attr(attr).ok_or_else(|| malformed(format!("no {attr}")))?)?;
if v.is_empty() || v.len() % BLOCK != 0 {
return Err(malformed(format!("a whole-block {attr}")));
}
Ok(v)
};
Ok(AgileDescriptor {
key_data,
key,
spin,
verifier_input: field("encryptedVerifierHashInput")?,
verifier_hash: field("encryptedVerifierHashValue")?,
key_value: field("encryptedKeyValue")?,
hmac_key: hmac_field("encryptedHmacKey")?,
hmac_value: hmac_field("encryptedHmacValue")?,
})
}
/// Decrypts an Agile package ([MS-OFFCRYPTO] 2.3.4.10): checks the password against the
/// verifier, unwraps the package key, verifies the dataIntegrity HMAC, then decrypts the segments.
fn agile(xml: &[u8], package: &[u8], password: &str) -> Result<Vec<u8>, DecryptError> {
let descriptor = agile_descriptor(xml)?;
let AgileDescriptor { key_data, key, spin, verifier_input, verifier_hash, key_value, hmac_key, hmac_value } = descriptor;
let h = iterated_hash(key.hash, password, &key.salt, spin);
let iv = key.iv(None);
let field = |data: &[u8], block_key: &[u8]| -> Result<Vec<u8>, DecryptError> {
let mut data = data.to_vec();
Aes::new(&derive(key.hash, &h, block_key, key.key_len))?.decrypt_cbc(&iv, &mut data)?;
Ok(data)
};
let input = field(&verifier_input, &BLOCK_VERIFIER_INPUT)?;
let expected = field(&verifier_hash, &BLOCK_VERIFIER_HASH)?;
let input = input.get(..key.salt.len()).unwrap_or(&input);
let actual = key.hash.digest(&[input]);
if expected.get(..actual.len()) != Some(actual.as_slice()) {
return Err(DecryptError::WrongPassword);
}
let secret = field(&key_value, &BLOCK_KEY)?;
let aes = Aes::new(secret.get(..key_data.key_len).ok_or_else(|| malformed("a short encryptedKeyValue"))?)?;
// Package integrity, [MS-OFFCRYPTO] 2.3.4.14: both HMAC fields are encrypted with the
// package key (the verifier above only authenticates the password), computed over the whole
// EncryptedPackage stream, StreamSize field included. The specification keys the HMAC with a
// random salt-size array, but Excel's key is as long as the hash (64 bytes for SHA-512), so
// both lengths are tried.
let mut mac_field = hmac_key;
aes.decrypt_cbc(&key_data.iv(Some(&BLOCK_HMAC_KEY)), &mut mac_field)?;
let mut declared = hmac_value;
aes.decrypt_cbc(&key_data.iv(Some(&BLOCK_HMAC_VALUE)), &mut declared)?;
let verified = [key_data.hash.digest_len(), key_data.salt.len()].into_iter().any(|len| {
let Some(key) = mac_field.get(..len) else { return false };
let mac = hmac(key_data.hash, key, package);
declared.get(..mac.len()) == Some(mac.as_slice())
});
if !verified {
return Err(malformed("the decrypted data failed its integrity check"));
}
let (size, data) = package_parts(package)?;
let mut out = Vec::with_capacity(size);
for (i, segment) in data.chunks(SEGMENT).enumerate() {
if out.len() >= size {
break;
}
let index = u32::try_from(i).map_err(|_| malformed("too many segments"))?;
let start = out.len();
out.extend_from_slice(segment);
let block = out.get_mut(start..).unwrap_or_default();
aes.decrypt_cbc(&key_data.iv(Some(&index.to_le_bytes())), block)?;
}
out.truncate(size);
Ok(out)
}
// ---- base64 --------------------------------------------------------------------------------
/// Standard base64 (whitespace ignored), as Agile's XML uses for every binary value.
fn base64(s: &str) -> Result<Vec<u8>, DecryptError> {
let mut out = Vec::with_capacity(s.len() / 4 * 3);
let (mut buf, mut bits) = (0u32, 0u32);
for c in s.bytes().filter(|c| !c.is_ascii_whitespace() && *c != b'=') {
let v = match c {
b'A'..=b'Z' => c - b'A',
b'a'..=b'z' => c - b'a' + 26,
b'0'..=b'9' => c - b'0' + 52,
b'+' => 62,
b'/' => 63,
_ => return Err(malformed("invalid base64")),
};
buf = (buf << 6) | u32::from(v);
bits += 6;
if bits >= 8 {
bits -= 8;
out.push((buf >> bits) as u8);
buf &= (1 << bits) - 1;
}
}
Ok(out)
}
#[cfg(test)]
mod tests;
-250
View File
@@ -1,250 +0,0 @@
//! Decryption tests. The expected keys and IVs below were computed independently with
//! msoffcrypto-tool 6.0.0 (MIT) for the synthetic password and salts used here; the encrypted
//! files are built in code by `crate::testing`. No Excel-produced file is used.
use super::*;
use crate::testing::{agile_info, encrypt_agile, encrypt_agile_with_hmac_key, encrypt_standard, with_agile_header};
const PASSWORD: &str = "Pass1234";
/// 16 zero bytes (whole AES blocks), a stand-in for a real integrity field where the values
/// can't be reached anyway (scheme errors, descriptor-parse damage).
const HMAC_STUB: &str = "AAAAAAAAAAAAAAAAAAAAAA==";
fn hex(s: &str) -> Vec<u8> {
(0..s.len()).step_by(2).map(|i| u8::from_str_radix(&s[i..i + 2], 16).unwrap()).collect()
}
fn salt() -> Vec<u8> {
(0u8..16).collect()
}
#[test]
fn aes_decrypts_the_fips_197_vectors() {
let mut b: [u8; 16] = hex("69c4e0d86a7b0430d8cdb78070b4c55a").try_into().unwrap();
Aes::new(&hex("000102030405060708090a0b0c0d0e0f")).unwrap().decrypt_block(&mut b);
assert_eq!(b.to_vec(), hex("00112233445566778899aabbccddeeff"));
let mut b: [u8; 16] = hex("8ea2b7ca516745bfeafc49904b496089").try_into().unwrap();
Aes::new(&hex("000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f")).unwrap().decrypt_block(&mut b);
assert_eq!(b.to_vec(), hex("00112233445566778899aabbccddeeff"));
}
#[test]
fn agile_keys_match_msoffcrypto() {
let h = iterated_hash(Hash::Sha512, PASSWORD, &salt(), 1000);
assert_eq!(derive(Hash::Sha512, &h, &BLOCK_VERIFIER_INPUT, 32), hex("c57def04938e1e83497271f6ec099e890bfd2bd9cf05ff208f8b2405df361ae6"));
assert_eq!(derive(Hash::Sha512, &h, &BLOCK_KEY, 32), hex("4ab08cf2ed95199d58d04f19331049a8a07cc1ff508431a1dd0785c84e261d03"));
let h = iterated_hash(Hash::Sha1, PASSWORD, &salt(), 1000);
assert_eq!(derive(Hash::Sha1, &h, &BLOCK_VERIFIER_INPUT, 16), hex("88c7567d028eae33a9a55f65729dcd2b"));
let key_data = Params { hash: Hash::Sha512, salt: (16u8..32).collect(), key_len: 32 };
assert_eq!(key_data.iv(Some(&1u32.to_le_bytes())), hex("da22ccba062aff577d207b2dee4e165b"));
}
#[test]
fn agile_pads_short_hashes_with_0x36() {
// SHA-1 gives 20 bytes; an AES-256 key needs 32, so 12 bytes of 0x36 follow.
let k = derive(Hash::Sha1, b"a", b"b", 32);
assert_eq!(k[..20], Hash::Sha1.digest(&[b"ab"])[..]);
assert_eq!(k[20..], [0x36; 12]);
}
#[test]
fn standard_keys_match_msoffcrypto() {
assert_eq!(standard_key(PASSWORD, &salt(), 16), hex("ae0b402aeb23a779471a130843e431d4"));
assert_eq!(standard_key(PASSWORD, &salt(), 32), hex("ae0b402aeb23a779471a130843e431d47b416314b5146b491cb815283663d26f"));
}
#[test]
fn base64_decodes_and_rejects_junk() {
assert_eq!(base64("AAECAwQFBgcICQoLDA0ODw==").unwrap(), salt());
assert_eq!(base64(" AAEC\nAw== ").unwrap(), vec![0, 1, 2, 3]);
assert!(matches!(base64("AA*C"), Err(DecryptError::Malformed(_))));
}
/// A payload that isn't a whole number of segments or blocks.
fn payload(len: usize) -> Vec<u8> {
(0..len).map(|i| (i * 7 + i / 251) as u8).collect()
}
#[test]
fn agile_round_trips() {
let plain = payload(10_000);
let (info, package) = encrypt_agile(&plain, PASSWORD, 1000);
assert_eq!(decrypt_streams(&info, &package, PASSWORD).unwrap(), plain);
}
#[test]
fn standard_round_trips() {
let plain = payload(10_000);
let (info, package) = encrypt_standard(&plain, PASSWORD);
assert_eq!(decrypt_streams(&info, &package, PASSWORD).unwrap(), plain);
}
#[test]
fn non_ascii_passwords_round_trip() {
// UTF-16LE, surrogate pairs included: "Pässwörd€" plus an emoji outside the BMP.
let pw = "P\u{e4}ssw\u{f6}rd\u{20ac}\u{1F511}";
let (info, package) = encrypt_agile(&payload(100), pw, 1000);
assert_eq!(decrypt_streams(&info, &package, pw).unwrap(), payload(100));
assert_eq!(decrypt_streams(&info, &package, "Passwrd"), Err(DecryptError::WrongPassword));
let (info, package) = encrypt_standard(&payload(100), pw);
assert_eq!(decrypt_streams(&info, &package, pw).unwrap(), payload(100));
}
#[test]
fn a_wrong_password_is_reported_as_such() {
let (info, package) = encrypt_agile(&payload(100), PASSWORD, 1000);
assert_eq!(decrypt_streams(&info, &package, "pass1234"), Err(DecryptError::WrongPassword));
assert_eq!(decrypt_streams(&info, &package, ""), Err(DecryptError::WrongPassword));
let (info, package) = encrypt_standard(&payload(100), PASSWORD);
assert_eq!(decrypt_streams(&info, &package, "Pass12345"), Err(DecryptError::WrongPassword));
}
#[test]
fn unsupported_schemes_are_named() {
let unsupported = |info: &[u8]| match decrypt_streams(info, &[0; 8], PASSWORD) {
Err(DecryptError::Unsupported(s)) => s,
other => panic!("expected Unsupported, got {other:?}"),
};
let (mut rc4, _) = encrypt_standard(&payload(10), PASSWORD);
rc4[4..8].copy_from_slice(&F_CRYPTOAPI.to_le_bytes()); // fCryptoAPI without fAES
assert_eq!(unsupported(&rc4), "RC4");
assert_eq!(unsupported(&[3, 0, 3, 0, 0, 0, 0, 0]), "Extensible");
assert_eq!(unsupported(&[1, 0, 1, 0, 0, 0, 0, 0]), "version 1.1");
let des = agile_info(PASSWORD, 1000, HMAC_STUB, HMAC_STUB).replace("cipherAlgorithm=\"AES\"", "cipherAlgorithm=\"3DES\"");
assert_eq!(unsupported(&with_agile_header(&des)), "3DES");
let cert = agile_info(PASSWORD, 1000, HMAC_STUB, HMAC_STUB).replace("keyEncryptor/password", "keyEncryptor/certificate");
assert_eq!(unsupported(&with_agile_header(&cert)), "certificate");
}
#[test]
fn damaged_files_are_malformed_not_panics() {
let is_malformed = |r: Result<Vec<u8>, DecryptError>| matches!(r, Err(DecryptError::Malformed(_)));
let plain = payload(5000);
for (info, package) in [encrypt_agile(&plain, PASSWORD, 1000), encrypt_standard(&plain, PASSWORD)] {
// Every truncation of either stream.
for cut in (0..info.len()).step_by(7) {
let _ = decrypt_streams(&info[..cut], &package, PASSWORD);
}
for cut in (0..package.len()).step_by(97) {
let _ = decrypt_streams(&info, &package[..cut], PASSWORD);
}
// A stated size past the ciphertext never drives an allocation.
let mut huge = package.clone();
huge[..8].copy_from_slice(&u64::MAX.to_le_bytes());
assert!(is_malformed(decrypt_streams(&info, &huge, PASSWORD)));
// Ciphertext that isn't whole AES blocks.
let mut ragged = package.clone();
ragged.truncate(ragged.len() - 3);
let len = (ragged.len() - 8) as u64;
ragged[..8].copy_from_slice(&len.to_le_bytes());
assert!(is_malformed(decrypt_streams(&info, &ragged, PASSWORD)));
}
let xml = agile_info(PASSWORD, 1000, HMAC_STUB, HMAC_STUB);
let bad = |s: String| is_malformed(decrypt_streams(&with_agile_header(&s), &[0; 8], PASSWORD));
assert!(bad(xml.replace("spinCount=\"1000\"", "spinCount=\"4000000000\"")));
assert!(bad(xml.replace("saltValue=\"", "saltValue=\"*")));
assert!(bad(xml.replace("keyBits=\"256\"", "keyBits=\"100\"")));
assert!(bad(xml.replace("blockSize=\"16\"", "blockSize=\"8\"")));
assert!(bad(xml.replace("<keyData", "<other")));
assert!(bad("<encryption".into()));
}
#[test]
fn non_ascii_passwords_hash_as_utf16le_like_msoffcrypto() {
// "Pässwörd€" plus an emoji outside the BMP, a surrogate pair in UTF-16.
let h = iterated_hash(Hash::Sha512, "P\u{e4}ssw\u{f6}rd\u{20ac}\u{1F511}", &salt(), 1000);
assert_eq!(
h,
hex("2bca96ca0a7e8d88d9314a2eb1819a1ece94ff9ce34266ff82f891a88c4240c4c9f39c729b7c5c410c27fc8e2b7ad90d8c005eb68a603377b8a4ab858041c76a")
);
}
#[test]
fn hmac_matches_the_rfc2104_test_vectors() {
use crate::crypto::hmac as mac;
let h = |v: Vec<u8>| v.iter().map(|b| format!("{b:02x}")).collect::<String>();
// RFC 2202 test case 1 (HMAC-SHA-1) and HMAC-SHA-256 vectors.
assert_eq!(h(mac(Hash::Sha1, &[0x0b; 20], b"Hi There")), "b617318655057264e28bc0b6fb378c8ef146be00");
assert_eq!(h(mac(Hash::Sha256, &[0x0b; 20], b"Hi There")), "b0344c61d8db38535ca8afceaf0bf12b881dc200c9833da726e9376c2e32cff7");
// RFC 4231 test case 5 (HMAC-SHA-256): a key longer than the block size is hashed first.
assert_eq!(
h(mac(Hash::Sha256, &[0xaa; 131], b"Test Using Larger Than Block-Size Key - Hash Key First")),
"60e431591ee0b67f0d8a26aacbf5b77f8e0bc6213728c5140546040f0ee37f54"
);
}
/// Strips the `<dataIntegrity …/>` element from an Agile descriptor.
fn without_integrity(info: &[u8]) -> Vec<u8> {
let xml = std::str::from_utf8(&info[8..]).unwrap().to_string();
let start = xml.find("<dataIntegrity").unwrap();
let end = start + xml[start..].find("/>").unwrap() + 2;
let stripped = format!("{}{}", &xml[..start], &xml[end..]);
with_agile_header(&stripped)
}
#[test]
fn integrity_is_required_and_enforced() {
let plain = payload(300);
let (info, package) = encrypt_agile(&plain, PASSWORD, 1000);
// An attacker can't just delete the integrity element to get the package verified.
assert!(matches!(decrypt_streams(&without_integrity(&info), &package, PASSWORD),
Err(DecryptError::Malformed(m)) if m.contains("dataIntegrity")));
// An empty or ragged field is damage, not a pass.
for bad in [("encryptedHmacKey", ""), ("encryptedHmacKey", "AA=="), ("encryptedHmacValue", "")] {
let xml =
agile_info(PASSWORD, 1000, HMAC_STUB, HMAC_STUB).replace(&format!("{}=\"{HMAC_STUB}\"", bad.0), &format!("{}=\"{}\"", bad.0, bad.1));
let r = decrypt_streams(&with_agile_header(&xml), &package, PASSWORD);
assert!(matches!(r, Err(DecryptError::Malformed(_))), "{}=\"{}\"", bad.0, bad.1);
}
}
#[test]
fn modified_ciphertext_fails_the_integrity_check() {
let plain = payload(300);
let (info, package) = encrypt_agile(&plain, PASSWORD, 1000);
// The correct password proves the verifier can't vouch for a modified package.
for at in [8usize, 100, package.len() - 5] {
let mut tampered = package.clone();
tampered[at] ^= 0x80;
assert!(
matches!(decrypt_streams(&info, &tampered, PASSWORD),
Err(DecryptError::Malformed(m)) if m.contains("integrity")),
"byte {at}"
);
}
// And neither can the descriptor: a swapped integrity value is refused.
let xml = String::from_utf8_lossy(&info[8..]).to_string();
let at = xml.find("encryptedHmacValue=\"").unwrap() + "encryptedHmacValue=\"".len();
let mut xml: Vec<u8> = xml.into_bytes();
xml[at] ^= b'A' ^ b'B';
assert!(matches!(decrypt_streams(&with_agile_header(std::str::from_utf8(&xml).unwrap()), &package, PASSWORD), Err(DecryptError::Malformed(_))));
}
#[test]
fn integrity_matches_the_stream_not_the_plaintext() {
// The MAC covers the whole EncryptedPackage stream, StreamSize field included: changing the
// stated size alone must fail, while the actual bytes round-trip.
let plain = payload(4200);
let (info, package) = encrypt_agile(&plain, PASSWORD, 1000);
let mut ours = package.clone();
ours[..8].copy_from_slice(&((plain.len() + 1) as u64).to_le_bytes());
assert!(matches!(decrypt_streams(&info, &ours, PASSWORD), Err(DecryptError::Malformed(m)) if m.contains("integrity")));
assert_eq!(decrypt_streams(&info, &package, PASSWORD).unwrap(), plain);
}
#[test]
fn integrity_keys_as_long_as_the_hash_or_the_salt_verify() {
// Excel keys the HMAC with 64 bytes (#2712); [MS-OFFCRYPTO] 2.3.4.14 describes a salt-size
// key. Both verify, and neither lets a modified package through.
let plain = payload(5000);
for key in [vec![7u8; 64], vec![7u8; 16]] {
let (info, package) = encrypt_agile_with_hmac_key(&plain, PASSWORD, 1000, &key);
assert_eq!(decrypt_streams(&info, &package, PASSWORD).unwrap(), plain, "{}-byte key", key.len());
let mut tampered = package.clone();
tampered[100] ^= 1;
assert!(matches!(decrypt_streams(&info, &tampered, PASSWORD), Err(DecryptError::Malformed(m)) if m.contains("integrity")));
}
// Another length (a key that matches neither) fails.
let (info, package) = encrypt_agile_with_hmac_key(&plain, PASSWORD, 1000, &[7u8; 32]);
assert!(matches!(decrypt_streams(&info, &package, PASSWORD), Err(DecryptError::Malformed(m)) if m.contains("integrity")));
}
+10 -251
View File
@@ -1,4 +1,6 @@
//! CSV import and export (RFC 4180 with the usual real-world leniency).
//! CSV import and export (RFC 4180 with the usual real-world leniency). Decoding, delimiter
//! sniffing, the record parser and field quoting are `craft-csv`'s; this module turns records into
//! typed cells (with each column's date order) and writes a sheet.
use std::sync::Arc;
@@ -12,241 +14,11 @@ use crate::{CsvOptions, IoError, ReadReport};
/// The largest CSV or tab-separated file written (bytes).
pub const MAX_CSV_BYTES: usize = 512 << 20;
/// Windows-1252 code points for bytes 0x80–0x9F (others map to Latin-1).
const CP1252: [u32; 32] = [
0x20AC, 0x81, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0x8D, 0x017D, 0x8F, 0x90, 0x2018, 0x2019,
0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0x9D, 0x017E, 0x0178,
];
pub use craft_csv::{decode_text, sniff_delimiter};
/// Windows-1256 code points for bytes 0x80–0xFF (unicode.org `CP1256.TXT`).
pub(crate) const CP1256: [u32; 128] = [
0x20AC, 0x067E, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0679, 0x2039, 0x0152, 0x0686, 0x0698, 0x0688, 0x06AF, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x06A9, 0x2122, 0x0691, 0x203A, 0x0153, 0x200C, 0x200D, 0x06BA, 0x00A0, 0x060C, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x06BE, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x061B, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x061F, 0x06C1, 0x0621, 0x0622, 0x0623, 0x0624, 0x0625, 0x0626, 0x0627,
0x0628, 0x0629, 0x062A, 0x062B, 0x062C, 0x062D, 0x062E, 0x062F, 0x0630, 0x0631, 0x0632, 0x0633, 0x0634, 0x0635, 0x0636, 0x00D7, 0x0637, 0x0638,
0x0639, 0x063A, 0x0640, 0x0641, 0x0642, 0x0643, 0x00E0, 0x0644, 0x00E2, 0x0645, 0x0646, 0x0647, 0x0648, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0649, 0x064A, 0x00EE, 0x00EF, 0x064B, 0x064C, 0x064D, 0x064E, 0x00F4, 0x064F, 0x0650, 0x00F7, 0x0651, 0x00F9, 0x0652, 0x00FB, 0x00FC, 0x200E,
0x200F, 0x06D2,
];
/// Evidence that bytes which aren't UTF-8 are Windows-1256 Arabic rather than Windows-1252.
#[derive(Default)]
struct Cp1256Evidence {
/// High bytes that decode to different characters in the two code pages, apart from invisible
/// marks and ordinals after a digit.
differing: usize,
/// Of those, Arabic letters and Arabic punctuation (، ؛ ؟) in Windows-1256.
arabic: usize,
/// Arabic bytes next to an ASCII letter: Western accents sit inside Latin words, Arabic doesn't.
arabic_touching_latin: usize,
/// Accented Latin bytes the two code pages share, next to another high byte: Hebrew
/// Windows-1255 letters land there, Arabic and Western text rarely do.
shared_letters_in_runs: usize,
}
impl Cp1256Evidence {
fn of(bytes: &[u8]) -> Self {
let mut e = Self::default();
for (i, &b) in bytes.iter().enumerate() {
if b < 0x80 {
continue;
}
let before = i.checked_sub(1).and_then(|j| bytes.get(j)).copied();
let after = bytes.get(i + 1).copied();
let next_to = |pred: fn(u8) -> bool| before.is_some_and(pred) || after.is_some_and(pred);
if same_in_cp1252_and_cp1256(b) {
if is_shared_latin_letter(b) && next_to(|x| x >= 0x80) {
e.shared_letters_in_runs += 1;
}
continue;
}
if is_invisible_mark(b) || is_ordinal_after_digit(b, before) {
continue;
}
e.differing += 1;
if is_cp1256_arabic(b) {
e.arabic += 1;
if next_to(|x| x.is_ascii_alphabetic()) {
e.arabic_touching_latin += 1;
}
}
}
e
}
/// Thresholds chosen against month, day and language names from 95 glibc locales (Western in
/// Windows-1252, Arabic, Persian and Urdu in 1256, Hebrew and Yiddish in 1255) plus CSV samples.
fn is_arabic(&self) -> bool {
self.arabic >= 2
&& self.arabic * 10 >= self.differing * 9
&& self.arabic_touching_latin * 10 <= self.arabic * 3
&& self.shared_letters_in_runs * 20 <= self.arabic
}
}
fn same_in_cp1252_and_cp1256(b: u8) -> bool {
matches!(
b,
0x80 | 0x82..=0x89 | 0x8B..=0x8C | 0x91..=0x97 | 0x99 | 0x9B..=0x9C | 0xA0 | 0xA2..=0xA9 | 0xAB..=0xB9 | 0xBB..=0xBE | 0xD7 | 0xE0 | 0xE2 | 0xE7..=0xEB
| 0xEE..=0xEF | 0xF4 | 0xF7 | 0xF9 | 0xFB..=0xFC
)
}
/// ZWNJ, ZWJ, LRM and RLM: common inside Persian words and around Arabic cells, but no evidence
/// either way.
fn is_invisible_mark(b: u8) -> bool {
matches!(b, 0x9D | 0x9E | 0xFD | 0xFE)
}
/// `º` and `ª` after a digit (`1º`, `2ª`) are Western ordinals, though Windows-1256 reads the bytes
/// as `؛` and `ھ`; either way they are no evidence.
fn is_ordinal_after_digit(b: u8, before: Option<u8>) -> bool {
matches!(b, 0xAA | 0xBA) && before.is_some_and(|x| x.is_ascii_digit())
}
fn is_shared_latin_letter(b: u8) -> bool {
matches!(b, 0xE0 | 0xE2 | 0xE7..=0xEB | 0xEE..=0xEF | 0xF4 | 0xF9 | 0xFB..=0xFC)
}
fn is_cp1256_arabic(b: u8) -> bool {
matches!(
b,
0x81 | 0x8A | 0x8D..=0x90 | 0x98 | 0x9A | 0x9F | 0xA1 | 0xAA | 0xBA | 0xBF..=0xD6 | 0xD8..=0xDB | 0xDD..=0xDF | 0xE1 | 0xE3..=0xE6 | 0xEC..=0xED | 0xFF
)
}
/// Decodes CSV bytes with BOM-aware UTF-8 and UTF-16 handling. Without a BOM, valid UTF-8 is kept.
/// Other bytes decode as Windows-1256 when they read as Arabic (see [`Cp1256Evidence`]), else as
/// Windows-1252.
pub fn decode_text(bytes: &[u8]) -> String {
if let Some(rest) = bytes.strip_prefix(&[0xEF, 0xBB, 0xBF]) {
return String::from_utf8_lossy(rest).into_owned();
}
let utf16 = |b: &[u8], le: bool| -> String {
let units = b.as_chunks::<2>().0.iter().map(|c| if le { u16::from_le_bytes(*c) } else { u16::from_be_bytes(*c) });
char::decode_utf16(units).map(|r| r.unwrap_or('\u{FFFD}')).collect()
};
if let Some(rest) = bytes.strip_prefix(&[0xFF, 0xFE]) {
return utf16(rest, true);
}
if let Some(rest) = bytes.strip_prefix(&[0xFE, 0xFF]) {
return utf16(rest, false);
}
match std::str::from_utf8(bytes) {
Ok(s) => s.to_string(),
Err(_) => {
let use_windows_1256 = Cp1256Evidence::of(bytes).is_arabic();
bytes
.iter()
.map(|&b| match b {
0x80..=0xFF if use_windows_1256 => {
char::from_u32(CP1256.get((b - 0x80) as usize).copied().unwrap_or(0xFFFD)).unwrap_or('\u{FFFD}')
}
0x80..=0x9F => char::from_u32(CP1252.get((b - 0x80) as usize).copied().unwrap_or(0xFFFD)).unwrap_or('\u{FFFD}'),
_ => b as char,
})
.collect()
}
}
}
/// Picks comma, semicolon or tab: the candidate with the most consistent count per line
/// (outside quotes) over the first non-blank lines. A candidate must appear on every such line, otherwise comma wins.
pub fn sniff_delimiter(text: &str, quote: char) -> u8 {
let mut best = (b',', 0usize, 0usize);
for d in [',', ';', '\t'] {
let mut counts = vec![];
let mut n = 0usize;
let mut in_q = false;
let mut blank = true;
for ch in text.chars().take(64 * 1024) {
if !in_q && ch == '\n' {
// Blank lines carry no delimiter evidence.
if !blank {
counts.push(n);
}
n = 0;
blank = true;
if counts.len() >= 20 {
break;
}
continue;
}
if ch != '\r' {
blank = false;
}
if ch == quote {
in_q = !in_q;
} else if !in_q && ch == d {
n += 1;
}
}
if !blank {
counts.push(n);
}
let min = counts.iter().copied().min().unwrap_or(0);
let total: usize = counts.iter().sum();
if min > 0 && (min, total) > (best.1, best.2) {
best = (d as u8, min, total);
}
}
best.0
}
/// Splits CSV text into records of fields. A record keeps at most [`MAX_COLS`] fields; the rest are
/// dropped as they are read, so a row of endless delimiters cannot allocate a string per field.
pub(crate) fn parse_records(text: &str, delim: char, quote: char, mut on_record: impl FnMut(Vec<String>) -> bool) {
let mut field = String::new();
let mut record: Vec<String> = vec![];
let mut in_q = false;
let mut chars = text.chars().peekable();
let mut field_started_quoted = false;
while let Some(c) = chars.next() {
if in_q {
if c == quote {
if chars.peek() == Some(&quote) {
field.push(quote);
chars.next();
} else {
in_q = false;
}
} else {
field.push(c);
}
continue;
}
if c == quote && field.is_empty() && !field_started_quoted {
in_q = true;
field_started_quoted = true;
} else if c == delim {
let f = std::mem::take(&mut field);
if record.len() < MAX_COLS as usize {
record.push(f);
}
field_started_quoted = false;
} else if c == '\r' || c == '\n' {
if c == '\r' && chars.peek() == Some(&'\n') {
chars.next();
}
let f = std::mem::take(&mut field);
if record.len() < MAX_COLS as usize {
record.push(f);
}
field_started_quoted = false;
if !on_record(std::mem::take(&mut record)) {
return;
}
} else {
field.push(c);
}
}
if !field.is_empty() || !record.is_empty() || field_started_quoted {
if record.len() < MAX_COLS as usize {
record.push(field);
}
on_record(record);
}
/// [`craft_csv::parse_records`] with a record kept to the grid's [`MAX_COLS`] fields.
fn parse_records(text: &str, delim: char, quote: char, on_record: impl FnMut(Vec<String>) -> bool) {
craft_csv::parse_records(text, delim, quote, MAX_COLS as usize, on_record);
}
/// How many rows and bytes of a text file are read to decide each column's date order.
@@ -587,25 +359,12 @@ pub fn write_csv_limited(sheet: &Sheet, delimiter: u8, limit: usize, mut cell_te
out.push(delim);
}
let t = cell_text(CellRef::new(row, col));
let quoted = t.contains(delim) || t.contains('"') || t.contains('\n') || t.contains('\r');
// The exact size of the field as written (the delimiter was pushed above), so a row
// never grows past the cap unseen and no doubled-quote copy of the text is built.
let field_len = if quoted { t.len() + t.matches('"').count() + 2 } else { t.len() };
if out.len().saturating_add(field_len) > limit {
// never grows past the cap unseen.
if out.len().saturating_add(craft_csv::field_len(&t, delim)) > limit {
return Err(too_large(format!("more than {} MiB (stopped at row {})", limit >> 20, row + 1)));
}
if quoted {
out.push('"');
for ch in t.chars() {
if ch == '"' {
out.push('"');
}
out.push(ch);
}
out.push('"');
} else {
out.push_str(&t);
}
craft_csv::push_field(&mut out, &t, delim);
}
out.push_str("\r\n");
if out.len() > limit {
+28 -3
View File
@@ -221,7 +221,16 @@ fn read_shape(obj: &El, anchor: Anchor) -> Option<Shape> {
let tx_box = obj.path(&["nvSpPr", "cNvSpPr"]).is_some_and(|c| c.flag("txBox", false));
let ln = sp.child("ln");
let arrow = ln.is_some_and(|l| l.child("tailEnd").or_else(|| l.child("headEnd")).is_some_and(|t| t.attr("type").is_some_and(|v| v != "none")));
// WordArt: our marker, a text body with the non-warping text warp `textNoShape`, on a rectangle
// that is a text box (as the writer saves it) or has no fill and no outline of its own (as
// WordArt usually comes). `textNoShape` is the standard "no warp" value, so on an ordinary
// shape (an ellipse, a filled rectangle) it, like any other warp, leaves the shape what its
// geometry says.
let word_art = prst == "rect"
&& (tx_box || (sp.child("noFill").is_some() && !ln.is_some_and(|l| l.child("solidFill").is_some())))
&& obj.path(&["txBody", "bodyPr", "prstTxWarp"]).is_some_and(|w| w.attr("prst") == Some("textNoShape"));
let kind = match prst {
_ if word_art => ShapeKind::WordArt,
_ if tx_box => ShapeKind::TextBox,
"roundRect" => ShapeKind::RoundedRectangle,
"ellipse" => ShapeKind::Ellipse,
@@ -294,6 +303,7 @@ fn read_text_format(body: Option<&El>) -> ShapeFormat {
f.italic = r.flag("i", false);
f.font = r.child("latin").and_then(|l| l.attr("typeface")).filter(|t| !t.is_empty() && !t.starts_with('+')).map(str::to_string);
f.text_color = r.child("solidFill").and_then(dml_color);
f.text_shadow = r.child("effectLst").and_then(|e| e.child("outerShdw")).and_then(dml_color);
f.text_outline = r.child("ln").and_then(|l| {
let c = l.child("solidFill").and_then(dml_color)?;
// Width in EMU (12700 per point); Excel's default hairline is 0.75 pt.
@@ -520,7 +530,7 @@ pub fn write_drawing(sheet: &Sheet, theme: &Theme, objs: &[Obj<'_>]) -> String {
ShapeKind::Triangle => ("triangle", false),
ShapeKind::Line | ShapeKind::Arrow => ("line", false),
// Equations are written by `write_equation`; this arm keeps the match whole.
ShapeKind::TextBox | ShapeKind::Equation => ("rect", true),
ShapeKind::TextBox | ShapeKind::WordArt | ShapeKind::Equation => ("rect", true),
ShapeKind::Icon | ShapeKind::Ink => ("rect", false),
};
let _ = write!(
@@ -550,7 +560,8 @@ pub fn write_drawing(sheet: &Sheet, theme: &Theme, objs: &[Obj<'_>]) -> String {
s.push_str("</a:ln>");
}
s.push_str("</xdr:spPr>");
if !sh.text.is_empty() {
// A WordArt with no text still writes its body, which carries the WordArt marker.
if !sh.text.is_empty() || sh.kind == ShapeKind::WordArt {
let f = &sh.format;
let anchor = match f.anchor {
Some(ShapeTextAnchor::Top) => " anchor=\"t\"",
@@ -559,7 +570,15 @@ pub fn write_drawing(sheet: &Sheet, theme: &Theme, objs: &[Obj<'_>]) -> String {
None if sh.kind == ShapeKind::TextBox => " anchor=\"t\"",
None => "",
};
let _ = write!(s, "<xdr:txBody><a:bodyPr wrap=\"square\" rtlCol=\"0\"{anchor}/><a:lstStyle/>");
if sh.kind == ShapeKind::WordArt {
// Our WordArt marker: a (non-warping) text warp, which the reader maps back to WordArt.
let _ = write!(
s,
"<xdr:txBody><a:bodyPr wrap=\"square\" rtlCol=\"0\"{anchor}><a:prstTxWarp prst=\"textNoShape\"><a:avLst/></a:prstTxWarp></a:bodyPr><a:lstStyle/>"
);
} else {
let _ = write!(s, "<xdr:txBody><a:bodyPr wrap=\"square\" rtlCol=\"0\"{anchor}/><a:lstStyle/>");
}
let algn = match f.align {
None => "",
Some(ShapeTextAlign::Left) => " algn=\"l\"",
@@ -614,6 +633,12 @@ fn run_props(f: &ShapeFormat, theme: &Theme) -> (String, String) {
if let Some(c) = f.text_color.as_ref().and_then(|c| srgb(c, theme)) {
let _ = write!(body, "<a:solidFill>{c}</a:solidFill>");
}
if let Some(c) = f.text_shadow.as_ref().and_then(|c| srgb(c, theme)) {
let _ = write!(
body,
"<a:effectLst><a:outerShdw blurRad=\"38100\" dist=\"38100\" dir=\"2700000\" algn=\"tl\" rotWithShape=\"0\">{c}</a:outerShdw></a:effectLst>"
);
}
if let Some(font) = &f.font {
let _ = write!(body, "<a:latin typeface=\"{}\"/>", esc_attr(font));
}
+1
View File
@@ -47,6 +47,7 @@ fn prefix_params(e: Expr, scope: &[String]) -> Expr {
Expr::Name(n) if scope.iter().any(|s| s.eq_ignore_ascii_case(&n)) => Expr::Name(format!("{PARAM_PREFIX}{n}")),
Expr::Unary(op, x) => Expr::Unary(op, sub(x, scope)),
Expr::Paren(x) => Expr::Paren(sub(x, scope)),
Expr::Binary(op, a, b) if !matches!(*a, Expr::Binary(..)) => Expr::Binary(op, sub(a, scope), sub(b, scope)),
Expr::Binary(op, a, b) => {
// A long chain such as A1+A2+… nests on its left: follow it in a loop, not one
// level of recursion per operand.
+2 -3
View File
@@ -12,7 +12,6 @@
#![forbid(unsafe_code)]
mod array_formula;
mod cfb;
mod chart;
mod chartex;
mod controls;
@@ -272,8 +271,8 @@ pub fn sniff(bytes: &[u8]) -> Format {
// CFB/OLE2 compound file: a password-protected workbook when it holds an encryption
// stream, an Excel 97-2003 workbook when it holds a workbook stream, otherwise another
// legacy binary (.doc…).
if bytes.starts_with(&cfb::SIGNATURE) {
return if cfb::is_encrypted(bytes) {
if bytes.starts_with(&craft_cfb::SIGNATURE) {
return if crypto::is_encrypted(bytes) {
Format::Encrypted
} else if xls::is_xls(bytes) {
Format::Xls
+1 -1
View File
@@ -280,7 +280,7 @@ impl Import {
Event::GeneralRef(r) => {
let ch = match r.resolve_char_ref().map_err(xml_error)? {
Some(c) => c,
None => match r.as_ref() {
None => match &*r {
b"amp" => '&',
b"lt" => '<',
b"gt" => '>',
+40 -167
View File
@@ -1,12 +1,12 @@
//! OPC package access: reading zip parts with decompression limits, relationships and part-name
//! resolution; writing a zip package deterministically.
use std::collections::HashMap;
use std::io::{Cursor, Read, Write};
//! OPC package access through `craft-opc`: zip parts read against GridCraft's limits,
//! relationships with GridCraft's `Rel`, part-name resolution, and a deterministic zip writer.
//! Parts are parsed with this crate's XML reader.
use crate::IoError;
use crate::xml::{self, El};
pub use craft_opc::{normalize, resolve, split_dir};
/// Largest decompressed part accepted.
pub const MAX_PART: u64 = 512 * 1024 * 1024;
/// Largest total decompressed size read from one package.
@@ -14,11 +14,10 @@ pub const MAX_TOTAL: u64 = 1024 * 1024 * 1024;
/// Most entries accepted in the zip directory.
pub const MAX_ENTRIES: usize = 200_000;
const LIMITS: craft_opc::Limits = craft_opc::Limits::new(MAX_ENTRIES, MAX_PART, MAX_TOTAL);
pub struct Package<'a> {
zip: zip::ZipArchive<Cursor<&'a [u8]>>,
/// Lower-case normalized part name → zip index.
names: HashMap<String, usize>,
total: u64,
opc: craft_opc::LazyPackage<'a>,
}
/// A relationship.
@@ -33,88 +32,49 @@ pub struct Rel {
pub external: bool,
}
pub fn normalize(name: &str) -> String {
name.replace('\\', "/").trim_start_matches('/').to_ascii_lowercase()
/// GridCraft's wording of a package error.
fn error(e: craft_opc::Error) -> IoError {
use craft_opc::Error as E;
match e {
E::Zip(m) | E::Write(m) => IoError::Zip(m),
E::TooManyEntries { declared, .. } => IoError::TooLarge(format!("{declared} zip entries")),
E::PartTooLarge { part, limit } => IoError::TooLarge(format!("{part} expands to more than {limit} bytes")),
E::PackageTooLarge { .. } => IoError::TooLarge("package expands to more than 1 GB".into()),
E::Xml { part, error } => IoError::Xml(format!("{part}: {error}")),
e if e.is_limit() => IoError::TooLarge(e.to_string()),
e => IoError::Zip(e.to_string()),
}
}
impl<'a> Package<'a> {
pub fn open(bytes: &'a [u8]) -> Result<Package<'a>, IoError> {
let zip = zip::ZipArchive::new(Cursor::new(bytes)).map_err(|e| IoError::Zip(e.to_string()))?;
if zip.len() > MAX_ENTRIES {
return Err(IoError::TooLarge(format!("{} zip entries", zip.len())));
}
let mut names = HashMap::new();
for i in 0..zip.len() {
if let Some(n) = zip.name_for_index(i) {
names.entry(normalize(n)).or_insert(i);
}
}
Ok(Package { zip, names, total: 0 })
Ok(Package { opc: craft_opc::LazyPackage::open(bytes, &LIMITS).map_err(error)? })
}
/// Every part's (normalized) name, sorted.
#[cfg(test)]
pub fn names(&self) -> Vec<String> {
let mut v: Vec<String> = self.names.keys().cloned().collect();
v.sort();
v
self.opc.names().into_iter().map(str::to_string).collect()
}
pub fn has(&self, part: &str) -> bool {
self.names.contains_key(&normalize(part))
self.opc.has(part)
}
/// Reads a part's bytes; `Ok(None)` when it doesn't exist.
pub fn read(&mut self, part: &str) -> Result<Option<Vec<u8>>, IoError> {
self.read_limited(part, MAX_PART)
self.opc.read(part).map_err(error)
}
/// Reads a part the reader can do without (the caller warns and goes on when it fails). A read
/// that fails partway is charged this part's whole allowance (`maximum`, and the byte that
/// detects an overrun) instead of ending the package's: the decompression work stays bounded,
/// and the parts read after it still open.
/// that fails partway is charged this part's whole allowance instead of ending the package's.
pub fn read_optional_limited(&mut self, part: &str, maximum: u64) -> Result<Option<Vec<u8>>, IoError> {
let before = self.total;
let read = self.read_limited(part, maximum);
if let Err(IoError::Zip(_)) = &read {
self.total = before.saturating_add(maximum.min(MAX_PART).saturating_add(1)).min(MAX_TOTAL);
}
read
self.opc.read_optional_limited(part, maximum).map_err(error)
}
/// Reads a small metadata part without allowing it to expand to the general part limit.
pub fn read_limited(&mut self, part: &str, maximum: u64) -> Result<Option<Vec<u8>>, IoError> {
let maximum = maximum.min(MAX_PART);
let Some(&idx) = self.names.get(&normalize(part)) else {
return Ok(None);
};
let mut f = self.zip.by_index(idx).map_err(|e| IoError::Zip(format!("{part}: {e}")))?;
if f.size() > maximum {
return Err(IoError::TooLarge(format!("{part} expands to {} bytes", f.size())));
}
let remaining = MAX_TOTAL.saturating_sub(self.total);
let limit = maximum.min(remaining);
let mut out = Vec::with_capacity(f.size().min(16 * 1024 * 1024) as usize);
let read = (&mut f).take(limit + 1).read_to_end(&mut out).map_err(|e| {
// A read that fails partway has already spent decompression work and buffered data
// the package budget never sees: count the allowance as spent, so a loop over many
// broken parts can't decompress it over and over.
self.total = MAX_TOTAL;
IoError::Zip(format!("{part}: {e}"))
});
if out.len() as u64 > limit {
// The allowance was spent getting here; make the exhaustion terminal instead of
// letting later reads pay the same decompression cost again.
self.total = MAX_TOTAL;
return Err(IoError::TooLarge(if remaining < maximum {
"package expands to more than 1 GB".into()
} else {
format!("{part} expands to more than {maximum} bytes")
}));
}
read?;
self.total += out.len() as u64;
Ok(Some(out))
self.opc.read_limited(part, maximum).map_err(error)
}
pub fn read_xml(&mut self, part: &str) -> Result<Option<El>, IoError> {
@@ -127,25 +87,13 @@ impl<'a> Package<'a> {
/// Relationships of a part (`xl/workbook.xml` → `xl/_rels/workbook.xml.rels`). Missing or
/// broken rels parts give an empty list.
pub fn rels(&mut self, part: &str) -> Result<Vec<Rel>, IoError> {
let (dir, file) = split_dir(part);
let rels_name = if dir.is_empty() { format!("_rels/{file}.rels") } else { format!("{dir}/_rels/{file}.rels") };
let root = match self.read(&rels_name)? {
Some(b) => match xml::parse(&b) {
Ok(r) => r,
Err(_) => return Ok(vec![]),
},
None => return Ok(vec![]),
};
Ok(root
.kids("Relationship")
.filter_map(|r| {
let id = r.attr("Id")?.to_string();
let ty = r.attr("Type").unwrap_or("");
let kind = ty.rsplit('/').next().unwrap_or("").to_string();
let target_raw = r.attr("Target")?;
let external = r.attr("TargetMode").is_some_and(|m| m.eq_ignore_ascii_case("External"));
let target = if external { target_raw.to_string() } else { resolve(dir, target_raw) };
Some(Rel { id, kind, target, external })
let rels = self.opc.rels(part).map_err(error)?;
Ok(rels
.list
.into_iter()
.map(|r| {
let kind = r.kind().to_string();
Rel { id: r.id, kind, target: r.target, external: r.external }
})
.collect())
}
@@ -153,11 +101,7 @@ impl<'a> Package<'a> {
/// The first part whose declared unpacked size is over [`MAX_PART`], with that size.
pub fn oversized_part(bytes: &[u8]) -> Option<(String, u64)> {
let mut zip = zip::ZipArchive::new(Cursor::new(bytes)).ok()?;
(0..zip.len().min(MAX_ENTRIES)).find_map(|i| {
let f = zip.by_index_raw(i).ok()?;
(f.size() > MAX_PART).then(|| (f.name().to_string(), f.size()))
})
craft_opc::declared_oversized(bytes, &LIMITS)
}
fn prefix(part: &str, e: IoError) -> IoError {
@@ -167,54 +111,22 @@ fn prefix(part: &str, e: IoError) -> IoError {
}
}
/// `xl/worksheets/sheet1.xml` → (`xl/worksheets`, `sheet1.xml`).
pub fn split_dir(part: &str) -> (&str, &str) {
let p = part.trim_start_matches('/');
match p.rfind('/') {
Some(i) => (p.get(..i).unwrap_or(""), p.get(i + 1..).unwrap_or("")),
None => ("", p),
}
}
/// Resolves a relationship target against the source part's directory.
pub fn resolve(dir: &str, target: &str) -> String {
let target = target.replace('\\', "/");
let target = target.split('#').next().unwrap_or("").to_string();
let mut segs: Vec<&str> = if target.starts_with('/') { vec![] } else { dir.split('/').filter(|s| !s.is_empty()).collect() };
for s in target.split('/') {
match s {
"" | "." => {}
".." => {
segs.pop();
}
s => segs.push(s),
}
}
segs.join("/")
}
// ---------------------------------------------------------------- writing
/// A zip package being written: every entry deflated, with one fixed timestamp.
pub struct ZipOut {
zip: zip::ZipWriter<Cursor<Vec<u8>>>,
zip: craft_opc::ZipOut,
}
impl ZipOut {
pub fn new() -> ZipOut {
ZipOut { zip: zip::ZipWriter::new(Cursor::new(Vec::new())) }
ZipOut { zip: craft_opc::ZipOut::new() }
}
pub fn add(&mut self, name: &str, data: &[u8]) -> Result<(), IoError> {
let opts = zip::write::SimpleFileOptions::default()
.compression_method(zip::CompressionMethod::Deflated)
.compression_level(Some(6))
.last_modified_time(zip::DateTime::default())
.large_file(data.len() as u64 >= u32::MAX as u64);
self.zip.start_file(name, opts).map_err(|e| IoError::Zip(e.to_string()))?;
self.zip.write_all(data).map_err(|e| IoError::Zip(e.to_string()))?;
Ok(())
self.zip.add(name, data, craft_opc::Compression::Deflate).map_err(error)
}
pub fn finish(self) -> Result<Vec<u8>, IoError> {
self.zip.finish().map(|c| c.into_inner()).map_err(|e| IoError::Zip(e.to_string()))
self.zip.finish().map_err(error)
}
}
@@ -232,43 +144,4 @@ mod tests {
assert_eq!(split_dir("xl/workbook.xml"), ("xl", "workbook.xml"));
assert_eq!(split_dir("x.xml"), ("", "x.xml"));
}
/// An over-budget read must spend the package's allowance once and for all: loops over many
/// broken parts (a sheet's optional VML relationships, for one) otherwise re-decompress the
/// remaining allowance again and again.
#[test]
fn a_failed_part_ends_the_package_budget() {
let z = crate::tests::make_zip(&[("a.xml", b"0123456789"), ("b.xml", b"hello")]);
let mut pkg = Package::open(&z).unwrap();
assert!(pkg.read("a.xml").unwrap().is_some());
// Nothing left to spend within the per-part limit either.
pkg.total = MAX_TOTAL - 1;
assert!(pkg.read("b.xml").is_err());
assert_eq!(pkg.total, MAX_TOTAL, "the failed read is charged");
// Oversized repeated attempts stay cheap errors now: no allowance is left to pay for.
assert!(matches!(pkg.read("b.xml"), Err(IoError::TooLarge(_))));
}
/// A part the reader can do without, damaged inside an intact archive, costs its own
/// allowance: the parts after it are still read.
#[test]
fn a_damaged_optional_part_costs_only_its_allowance() {
use std::io::Write as _;
let mut w = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
let stored = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Stored);
w.start_file("a.xml", stored).unwrap();
w.write_all(b"<a>damaged</a>").unwrap();
w.start_file("b.xml", zip::write::SimpleFileOptions::default()).unwrap();
w.write_all(b"hello").unwrap();
let mut z = w.finish().unwrap().into_inner();
let at = z.windows(7).position(|x| x == b"damaged").unwrap();
z[at] ^= 0x20;
let mut pkg = Package::open(&z).unwrap();
assert!(matches!(pkg.read_optional_limited("a.xml", 1024), Err(IoError::Zip(_))));
assert_eq!(pkg.total, 1025, "the failed read is charged its allowance");
assert_eq!(pkg.read("b.xml").unwrap().as_deref(), Some(&b"hello"[..]));
// An ordinary read of it still ends the package's budget.
assert!(pkg.read("a.xml").is_err());
assert_eq!(pkg.total, MAX_TOTAL);
}
}
+17 -1
View File
@@ -69,7 +69,23 @@ fn item_label_el(e: &El, sys: DateSystem) -> String {
}
fn read_cache(cx: &mut Ctx<'_>, part: &str, sheet_name: &str) -> Option<CacheIn> {
let x = cx.optional_xml(part).ok().flatten()?;
// A cache that can't be read loses its PivotTable: say so instead of dropping it silently.
// An oversized cache is refused before it inflates, so warning and opening the rest is safe.
let x = match cx.optional_xml(part) {
Ok(Some(x)) => x,
Ok(None) => {
cx.warn(format!("a PivotTable's cache {part} is missing or could not be read; kept as plain cells"));
return None;
}
Err(IoError::TooLarge(m)) => {
cx.warn(format!("a PivotTable's cache is too large to read ({m}); kept as plain cells"));
return None;
}
Err(e) => {
cx.warn(format!("a PivotTable's cache could not be read ({e}); kept as plain cells"));
return None;
}
};
if x.name != "pivotCacheDefinition" {
cx.warn(format!("{part} is not a pivot cache definition; its PivotTable was kept as plain cells"));
return None;
+6 -263
View File
@@ -1,282 +1,25 @@
//! Test support: builds password-encrypted workbooks in code, so tests here and in the crates
//! above (engine, UI) never need an encrypted file on disk. Compiled for this crate's tests and,
//! elsewhere, only with the `test-support` feature (a dev-dependency feature, never shipped).
//!
//! Salts and keys are fixed so the output is deterministic. This is not an encryptor for real
//! use: a fixed key protects nothing.
//! The encryption is `craft-offcrypto`'s and the compound file `craft-cfb`'s.
#![allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
use aes::cipher::{BlockCipherEncrypt, KeyInit};
use crate::crypto::{
BLOCK_HMAC_KEY, BLOCK_HMAC_VALUE, BLOCK_KEY, BLOCK_VERIFIER_HASH, BLOCK_VERIFIER_INPUT, F_AES, F_CRYPTOAPI, Hash, PASSWORD_ENCRYPTOR, Params,
SEGMENT, derive, hmac, iterated_hash, standard_key,
};
/// The spin count `encrypt_workbook` uses: Excel's default.
pub const SPIN_COUNT: u32 = 100_000;
const PASSWORD_SALT: [u8; 16] = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15];
const KEY_DATA_SALT: [u8; 16] = [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31];
const SECRET_KEY: [u8; 32] =
[64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95];
const VERIFIER: [u8; 16] = [32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47];
/// The dataIntegrity HMAC key, as long as the hash: Excel writes 64 bytes for SHA-512, not the
/// salt-size key [MS-OFFCRYPTO] 2.3.4.14 describes.
const HMAC_KEY: [u8; 64] = [
112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140,
141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169,
170, 171, 172, 173, 174, 175,
];
/// `package` (normally an `.xlsx` zip) as Excel stores it with a password: Agile encryption
/// (AES-256, SHA-512) inside a compound file.
pub fn encrypt_workbook(package: &[u8], password: &str) -> Vec<u8> {
let (info, encrypted) = encrypt_agile(package, password, SPIN_COUNT);
compound_file(&[("EncryptionInfo", &info), ("EncryptedPackage", &encrypted)])
craft_offcrypto::encrypt_with_spin_count(package, password, SPIN_COUNT).unwrap()
}
/// The same with Standard encryption (AES-128, SHA-1), as Excel 2007 wrote it.
pub fn encrypt_workbook_standard(package: &[u8], password: &str) -> Vec<u8> {
let (info, encrypted) = encrypt_standard(package, password);
compound_file(&[("EncryptionInfo", &info), ("EncryptedPackage", &encrypted)])
craft_offcrypto::encrypt_standard(package, password, 128).unwrap()
}
/// A version 3 compound file (512-byte sectors) holding `streams` under the root, laid out as
/// Excel lays it out: streams under 4096 bytes in the mini stream, larger ones in regular
/// sectors. Up to 109 FAT sectors (about 7 MB of streams).
/// A version 3 compound file (512-byte sectors) holding `streams` under the root: streams under
/// 4096 bytes in the mini stream, larger ones in regular sectors.
pub fn compound_file(streams: &[(&str, &[u8])]) -> Vec<u8> {
const SECTOR: usize = 512;
const MINI: usize = 64;
const CUTOFF: usize = 4096;
const FREE: u32 = 0xFFFF_FFFF;
const END: u32 = 0xFFFF_FFFE;
const FAT_SECTOR: u32 = 0xFFFF_FFFD;
let is_mini = |d: &[u8]| d.len() < CUTOFF;
// The mini stream: small streams in 64-byte mini sectors, chained by the mini FAT.
let mut container = vec![];
let mut mini_fat: Vec<u32> = vec![];
let mut mini_starts = vec![];
for (_, d) in streams {
if !is_mini(d) || d.is_empty() {
mini_starts.push(END);
continue;
}
let first = mini_fat.len();
let n = d.len().div_ceil(MINI);
for k in 0..n {
mini_fat.push(if k + 1 == n { END } else { (first + k + 1) as u32 });
}
mini_starts.push(first as u32);
container.extend_from_slice(d);
container.resize((first + n) * MINI, 0);
}
// Regular-sector blobs, in order: directory, mini FAT, mini stream, then each large stream.
let dir_len = (streams.len() + 1).div_ceil(4) * SECTOR;
let mut blobs: Vec<Vec<u8>> = vec![vec![0; dir_len], mini_fat.iter().flat_map(|v| v.to_le_bytes()).collect(), container.clone()];
blobs.extend(streams.iter().map(|(_, d)| if is_mini(d) { vec![] } else { d.to_vec() }));
let counts: Vec<usize> = blobs.iter().map(|b| b.len().div_ceil(SECTOR)).collect();
let body: usize = counts.iter().sum();
let mut fat_sectors = 1;
while fat_sectors * 128 < fat_sectors + body {
fat_sectors += 1;
}
assert!(fat_sectors <= 109, "the test writer has no DIFAT sectors");
let mut fat = vec![FREE; fat_sectors * 128];
fat[..fat_sectors].fill(FAT_SECTOR);
let mut starts = vec![];
let mut next = fat_sectors;
for &n in &counts {
if n == 0 {
starts.push(END);
continue;
}
starts.push(next as u32);
for k in 0..n {
fat[next + k] = if k + 1 == n { END } else { (next + k + 1) as u32 };
}
next += n;
}
let put16 = |b: &mut [u8], at: usize, v: u16| b[at..at + 2].copy_from_slice(&v.to_le_bytes());
let put32 = |b: &mut [u8], at: usize, v: u32| b[at..at + 4].copy_from_slice(&v.to_le_bytes());
// Directory: the root (owning the mini stream), then the streams chained as right siblings.
let dir = &mut blobs[0];
for k in 0..dir_len / 128 {
for at in [68, 72, 76] {
put32(dir, k * 128 + at, FREE);
}
}
let mut entry = |k: usize, name: &str, kind: u8, start: u32, size: usize| {
let e = k * 128;
let units: Vec<u8> = name.encode_utf16().flat_map(u16::to_le_bytes).collect();
dir[e..e + units.len()].copy_from_slice(&units);
put16(dir, e + 64, units.len() as u16 + 2);
dir[e + 66] = kind;
dir[e + 67] = 1; // black
if kind == 5 && !streams.is_empty() {
put32(dir, e + 76, 1);
}
if kind == 2 && k < streams.len() {
put32(dir, e + 72, k as u32 + 1);
}
put32(dir, e + 116, start);
put32(dir, e + 120, size as u32);
};
entry(0, "Root Entry", 5, starts[2], container.len());
for (k, (name, data)) in streams.iter().enumerate() {
let start = if is_mini(data) { mini_starts[k] } else { starts[3 + k] };
entry(k + 1, name, 2, start, data.len());
}
let mut out = vec![0u8; SECTOR * (1 + fat_sectors + body)];
out[..8].copy_from_slice(&crate::cfb::SIGNATURE);
put16(&mut out, 24, 0x3E); // minor version
put16(&mut out, 26, 3); // major version
put16(&mut out, 28, 0xFFFE); // byte order
put16(&mut out, 30, 9); // 512-byte sectors
put16(&mut out, 32, 6); // 64-byte mini sectors
put32(&mut out, 44, fat_sectors as u32);
put32(&mut out, 48, starts[0]);
put32(&mut out, 56, CUTOFF as u32);
put32(&mut out, 60, starts[1]); // first mini FAT sector (END when there is none)
put32(&mut out, 64, counts[1] as u32);
put32(&mut out, 68, END); // no DIFAT sectors
for i in 0..109 {
put32(&mut out, 76 + i * 4, if i < fat_sectors { i as u32 } else { FREE });
}
for (i, v) in fat.iter().enumerate() {
put32(&mut out, SECTOR + i * 4, *v);
}
for (blob, start) in blobs.iter().zip(&starts) {
if !blob.is_empty() {
let at = SECTOR * (1 + *start as usize);
out[at..at + blob.len()].copy_from_slice(blob);
}
}
out
}
fn encrypt_block(key: &[u8], block: &mut [u8; 16]) {
let b = aes::Block::cast_from_core_mut(block);
match key.len() {
16 => aes::Aes128::new_from_slice(key).unwrap().encrypt_block(b),
32 => aes::Aes256::new_from_slice(key).unwrap().encrypt_block(b),
n => panic!("no {n}-byte AES in the test encryptor"),
}
}
fn pad(data: &[u8]) -> Vec<u8> {
let mut v = data.to_vec();
v.resize(data.len().div_ceil(16) * 16, 0);
v
}
fn encrypt_ecb(key: &[u8], data: &[u8]) -> Vec<u8> {
let mut v = pad(data);
v.as_chunks_mut::<16>().0.iter_mut().for_each(|b| encrypt_block(key, b));
v
}
fn encrypt_cbc(key: &[u8], iv: &[u8], data: &[u8]) -> Vec<u8> {
let mut v = pad(data);
let mut prev: [u8; 16] = iv.try_into().unwrap();
for b in v.as_chunks_mut::<16>().0 {
b.iter_mut().zip(prev).for_each(|(x, p)| *x ^= p);
encrypt_block(key, b);
prev = *b;
}
v
}
fn base64_encode(b: &[u8]) -> String {
const T: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
let mut s = String::new();
for c in b.chunks(3) {
let n = (u32::from(c[0]) << 16) | (u32::from(*c.get(1).unwrap_or(&0)) << 8) | u32::from(*c.get(2).unwrap_or(&0));
for i in 0..4 {
s.push(if i <= c.len() { T[(n >> (18 - 6 * i)) as usize & 63] as char } else { '=' });
}
}
s
}
fn package(size: usize, body: Vec<u8>) -> Vec<u8> {
let mut p = (size as u64).to_le_bytes().to_vec();
p.extend(body);
p
}
/// The Agile XML descriptor: AES-256, SHA-512, a password encryptor salted with 0..16, and the
/// dataIntegrity pair (`hmac_key_b64`/`hmac_value_b64`) a compliant file carries.
pub(crate) fn agile_info(password: &str, spin: u32, hmac_key_b64: &str, hmac_value_b64: &str) -> String {
let key = Params { hash: Hash::Sha512, salt: PASSWORD_SALT.to_vec(), key_len: 32 };
let h = iterated_hash(Hash::Sha512, password, &key.salt, spin);
let iv = key.iv(None);
let field = |block_key: &[u8], data: &[u8]| base64_encode(&encrypt_cbc(&derive(Hash::Sha512, &h, block_key, 32), &iv, data));
let input = field(&BLOCK_VERIFIER_INPUT, &VERIFIER);
let value = field(&BLOCK_VERIFIER_HASH, &Hash::Sha512.digest(&[&VERIFIER]));
let secret = field(&BLOCK_KEY, &SECRET_KEY);
let params = |salt: &[u8]| {
format!(
"saltSize=\"16\" blockSize=\"16\" keyBits=\"256\" hashSize=\"64\" cipherAlgorithm=\"AES\" cipherChaining=\"ChainingModeCBC\" hashAlgorithm=\"SHA512\" saltValue=\"{}\"",
base64_encode(salt)
)
};
// The dataIntegrity pair ([MS-OFFCRYPTO] 2.3.4.14), encrypted with the package key.
format!(
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\r\n<encryption xmlns=\"http://schemas.microsoft.com/office/2006/encryption\" xmlns:p=\"http://schemas.microsoft.com/office/2006/keyEncryptor/password\"><keyData {}/><dataIntegrity encryptedHmacKey=\"{hmac_key_b64}\" encryptedHmacValue=\"{hmac_value_b64}\"/><keyEncryptors><keyEncryptor uri=\"{PASSWORD_ENCRYPTOR}\"><p:encryptedKey spinCount=\"{spin}\" {} encryptedVerifierHashInput=\"{input}\" encryptedVerifierHashValue=\"{value}\" encryptedKeyValue=\"{secret}\"/></keyEncryptor></keyEncryptors></encryption>",
params(KEY_DATA_SALT.as_slice()),
params(&PASSWORD_SALT),
)
}
/// `EncryptionInfo` for an Agile descriptor: version 4.4, reserved 0x40, then the XML.
pub(crate) fn with_agile_header(xml: &str) -> Vec<u8> {
let mut info = vec![4, 0, 4, 0, 0x40, 0, 0, 0];
info.extend_from_slice(xml.as_bytes());
info
}
/// `EncryptionInfo` and `EncryptedPackage` for `plain`, Agile (AES-256, SHA-512), with the
/// dataIntegrity HMAC over the whole EncryptedPackage stream as Excel writes it.
pub(crate) fn encrypt_agile(plain: &[u8], password: &str, spin: u32) -> (Vec<u8>, Vec<u8>) {
encrypt_agile_with_hmac_key(plain, password, spin, &HMAC_KEY)
}
/// [`encrypt_agile`] with the dataIntegrity HMAC keyed by `hmac_key`.
pub(crate) fn encrypt_agile_with_hmac_key(plain: &[u8], password: &str, spin: u32, hmac_key: &[u8]) -> (Vec<u8>, Vec<u8>) {
let key_data = Params { hash: Hash::Sha512, salt: KEY_DATA_SALT.to_vec(), key_len: 32 };
let mut body = vec![];
for (i, segment) in plain.chunks(SEGMENT).enumerate() {
body.extend(encrypt_cbc(&SECRET_KEY, &key_data.iv(Some(&(i as u32).to_le_bytes())), segment));
}
let stream = package(plain.len(), body);
let (hmac_key_b64, hmac_value_b64) = (
base64_encode(&encrypt_cbc(&SECRET_KEY, &key_data.iv(Some(&BLOCK_HMAC_KEY)), hmac_key)),
base64_encode(&encrypt_cbc(&SECRET_KEY, &key_data.iv(Some(&BLOCK_HMAC_VALUE)), &hmac(Hash::Sha512, hmac_key, &stream))),
);
(with_agile_header(&agile_info(password, spin, &hmac_key_b64, &hmac_value_b64)), stream)
}
/// `EncryptionInfo` and `EncryptedPackage` for `plain`, Standard (AES-128, SHA-1).
pub(crate) fn encrypt_standard(plain: &[u8], password: &str) -> (Vec<u8>, Vec<u8>) {
let key = standard_key(password, &PASSWORD_SALT, 16);
let mut header = vec![];
for v in [F_CRYPTOAPI | F_AES, 0, 0x660E, 0x8004, 128, 0x18, 0, 0] {
header.extend_from_slice(&u32::to_le_bytes(v));
}
header.extend("\0".encode_utf16().flat_map(u16::to_le_bytes));
let mut info = vec![4, 0, 2, 0];
info.extend_from_slice(&(F_CRYPTOAPI | F_AES).to_le_bytes());
info.extend_from_slice(&(header.len() as u32).to_le_bytes());
info.extend(header);
info.extend_from_slice(&16u32.to_le_bytes());
info.extend(PASSWORD_SALT);
info.extend(encrypt_ecb(&key, &VERIFIER));
info.extend_from_slice(&20u32.to_le_bytes());
info.extend(encrypt_ecb(&key, &Hash::Sha1.digest(&[&VERIFIER])));
(info, package(plain.len(), encrypt_ecb(&key, plain)))
craft_cfb::write_streams(craft_cfb::Version::V3, streams).unwrap()
}
+1
View File
@@ -32,6 +32,7 @@ mod sheet_data_pin;
mod sheet_names;
mod smooth_roundtrip;
mod trendline_roundtrip;
mod xls_dv_tests;
mod xls_style_tests;
mod xls_tests;
mod xlsb_tests;
+6 -32
View File
@@ -1,8 +1,8 @@
use gridcraft_core::{CellRef, DateSystem, MAX_COLS, Value};
use gridcraft_model::{Sheet, Workbook};
use crate::csv::{decode_text, parse_records, sniff_delimiter};
use crate::{CsvOptions, Format, IoError, read_csv, read_csv_with_report, sniff, write_csv, write_csv_limited};
use craft_csv::{decode_text, parse_records, sniff_delimiter};
fn v(wb: &Workbook, r: u32, c: u32) -> Value {
wb.sheet(0).unwrap().value(CellRef::new(r, c))
@@ -148,33 +148,6 @@ fn encodings() {
assert_eq!(v(&wb, 0, 1), Value::Number(1.0));
}
#[test]
fn windows_1256_table_matches_the_unicode_mapping() {
let at = |b: u8| crate::csv::CP1256.get(usize::from(b - 0x80)).and_then(|&c| char::from_u32(c));
let expected = [
(0x80, '€'),
(0x81, 'پ'),
(0x8D, 'چ'),
(0x98, 'ک'),
(0x9D, '\u{200C}'),
(0xA1, '،'),
(0xBA, '؛'),
(0xBF, '؟'),
(0xC7, 'ا'),
(0xC8, 'ب'),
(0xDC, 'ـ'),
(0xE1, 'ل'),
(0xE3, 'م'),
(0xE9, 'é'),
(0xFD, '\u{200E}'),
(0xFE, '\u{200F}'),
(0xFF, 'ے'),
];
for (b, c) in expected {
assert_eq!(at(b), Some(c), "0x{b:02X}");
}
}
#[test]
fn arabic_csv_in_windows_1256_reads_as_arabic() {
let bytes = [
@@ -302,8 +275,9 @@ fn sniffing_formats() {
assert_eq!(sniff(b"a,b\n1,2"), Format::Csv);
// An Excel password-protected file is a CFB/OLE2 compound file with an EncryptedPackage
// stream; any other compound file is a legacy binary (.xls), not an encrypted workbook.
assert_eq!(sniff(&crate::cfb::tests::build(&["EncryptionInfo", "EncryptedPackage"])), Format::Encrypted);
assert_eq!(sniff(&crate::cfb::tests::build(&["Workbook"])), Format::LegacyBinary);
let compound = crate::testing::compound_file;
assert_eq!(sniff(&compound(&[("EncryptionInfo", b"x"), ("EncryptedPackage", b"x")])), Format::Encrypted);
assert_eq!(sniff(&compound(&[("WordDocument", b"x")])), Format::LegacyBinary);
assert_eq!(sniff(&[0xD0, 0xCF, 0x11, 0xE0, 0xA1, 0xB1, 0x1A, 0xE1, 0, 0]), Format::LegacyBinary);
assert_eq!(sniff(&[0xD0, 0xCF, 0x11, 0xE0, 0, 0, 1, 2]), Format::Unknown);
assert_eq!(sniff(b""), Format::Unknown);
@@ -393,14 +367,14 @@ fn parser_stops_collecting_fields_at_max_cols() {
let mut text = ",".repeat((MAX_COLS + extra) as usize - 1);
text.push_str("end\nx,y\n");
let mut lens = vec![];
parse_records(&text, ',', '"', |rec| {
parse_records(&text, ',', '"', MAX_COLS as usize, |rec| {
lens.push(rec.len());
true
});
assert_eq!(lens, vec![MAX_COLS as usize, 2]);
// An unterminated last row is capped the same way.
let mut lens = vec![];
parse_records(&",".repeat(MAX_COLS as usize + extra as usize), ',', '"', |rec| {
parse_records(&",".repeat(MAX_COLS as usize + extra as usize), ',', '"', MAX_COLS as usize, |rec| {
lens.push(rec.len());
true
});

Some files were not shown because too many files have changed in this diff Show More