Compare commits

..
Author SHA1 Message Date
FredApps bda04cf4c5 CI: nightly release dry run
release.yml also runs nightly on main with craft-common-actions' dry-run: every release build
except macOS, with nothing signed, pushed or drafted. Breakage that only the release build hits
(riscv64, Flatpak, FreeBSD, packaging) then shows up the next morning instead of on release
day, as it did on 11 Nov 2026.
2026-11-11 22:45:43 +01:00
Dylan M. Taylor 572bec289b engine: work limits for Advanced Filter, Remove Duplicates, filters, AutoFit, macro footprints; range sizes saturate (#3295)
* engine: bound Advanced Filter, Remove Duplicates, AutoFilter, Circle Invalid Data; one-pass AutoFit; saturating range dimensions

* address review: AutoFit scans each run of adjacent selected columns separately

* address review: report a truncated circle-invalid scan, no spill scan for plain cells in Replace, one pass for many AutoFit runs

* address review: warn in the app when Circle Invalid Data stopped early; Replace test requires success

* address review: the scan note joins an open-time notice; say cell visits
2026-11-11 16:10:10 -05:00
Dylan M. Taylor 55eb394044 engine: index merges and bound the page in HTML export and clipboard HTML (#3266)
* engine: index merges and bound the page in HTML export and clipboard HTML

Save as Web Page and the clipboard's HTML scanned every merge of the
sheet for every cell, counted a merge's span over all the rows the file
gave it, placed each picture by summing every custom row before it, and
grew the page without a byte limit. Merges are now indexed once over
the exported rectangle (overlapping merges past a few visits per cell
are refused), spans use prefix counts clipped to the table, anchors use
LineIndex, and the page stops at 256 MiB with an error.

* address review: check the web page size after each chart and before returning it

* address review: bound the merge index, escape cell text within the page's room, size images before encoding

* address review: escape image attributes within the page's room
2026-11-11 16:09:43 -05:00
Dylan M. Taylor 9e9f13114f check, xtask stats: visited directories keyed by (dev, inode); stats reads only regular files (#3287)
* Directory walkers key visited directories by (dev, inode); stats skips non-regular or oversized .rs targets

* address review: record a walked directory only within the depth limit; stats skips symlinked sources and bounds the read

* address review: rescan a directory reached from a shallower path, skipping files already collected

* xtask stats: test the directory walk (links, identity, size limit)

* address review: create the test directories atomically and privately

* xtask stats tests: no unreachable! in the directory helper

* gridcraft check: unit-test that each workbook is listed once whatever path reaches it
2026-11-11 16:09:17 -05:00
Dylan M. Taylor 35e9815603 Security: one work budget across every call of a lifted function (#3197)
* functions: one work budget across every call of a lifted function

A function lifted over a computed array runs once per element, and only the
six lookups counted their repeated work against LIFTED_SCAN_BUDGET. Criteria
aggregates rescanned their ranges per criterion, CUMIPMT/CUMPRINC stepped
through up to 1e7 periods per element, and array-producing functions such as
MUNIT built a whole result per element before keeping its first cell.

The budget now covers all of it: COUNTIF/SUMIF/AVERAGEIF(S)/MAXIFS/MINIFS
charge their range cells per call, the period loops of every element are
added up before any runs, and array results charge their cells as they are
built. Past the budget the call is #NUM!, as lifted lookups already are.

* address review: charge numeric-text periods of lifted CUMIPMT/CUMPRINC to the work budget

* address review: charge only calls that reach the cumulative period loop

* address review: charge the numeric text a lifted period call parses

* address review: SUMIF/AVERAGEIF charge only the sum-range cells they read; weight cumulative period work
2026-11-11 16:08:50 -05:00
Dylan M. Taylor 6ea8d11522 print: bound the print area to the grid; render charts once and budget picture and chart output (#3314)
* print: hold the print area to the grid; render each chart once and budget picture and chart output across pages

* address review: budget chart path points and drop charts that do not fit the budget
2026-11-11 16:08:24 -05:00
Dylan M. Taylor 7c3af8a421 fix(formula): bound the wrappers stacked on one path of a parsed formula (#3813) 2026-11-11 16:07:58 -05:00
Dylan M. Taylor ca7ea0131e fix(mcp): send the control token only to a loopback address (#3814) 2026-11-11 16:07:32 -05:00
Dylan M. Taylor aa0347cc2e fix(engine): calculated pivot fields may not build arrays from scalars (#3812) 2026-11-11 16:07:06 -05:00
Dylan M. Taylor 5cf52c5fbb chore: bump version to 0.19.1 for the Flatpak update fix (#3809) 2026-11-11 14:55:53 -05:00
Dylan M. Taylor 0c5b1e85e6 sonar: stop counting the locale table as duplicated code (#3805) 2026-11-11 14:33:16 -05:00
Dylan M. Taylor 7f6a79fd98 Fix Flatpak permissions for update checks and self-updates (#3803)
* fix(flatpak): allow release checks and self-update helpers

* test: keep Flatpak permission parsing within strict Clippy rules
2026-11-11 14:24:13 -05:00
Dylan M. Taylor 6dbfbc4058 Merge pull request #3795 from storytold/chore/craft-update-6e03f65
Updates: take craft-update from craft-libs main at 6e03f65bf
2026-11-11 12:44:31 -05:00
Dylan M. Taylor e258db5f19 Updates: take craft-update from craft-libs main at 6e03f65bf 2026-11-11 12:44:25 -05:00
Dylan M. Taylor 763816d27c ci: run Harden-Runner on every job (#3786)
Adds step-security/harden-runner (audit, pinned to v2.22.4 like
craft-common-actions) as the first step of each job that did not have it,
so every workflow in the org's public repos reports its egress to
StepSecurity. Audit, not block: block-mode allow lists come from each
job's observed traffic.
2026-11-11 12:11:59 -05:00
39 changed files with 958 additions and 116 deletions
+4
View File
@@ -27,6 +27,10 @@ jobs:
name: Windows installation policy (no app build)
runs-on: windows-latest
steps:
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
+5
View File
@@ -1,6 +1,10 @@
name: Release
on:
# A nightly dry run on main: every release build except macOS, nothing signed or released
# (craft-common-actions release.yml `dry-run`). Finds release-only breakage before release day.
schedule:
- cron: "56 1 * * *"
push:
branches: [release]
workflow_dispatch:
@@ -23,6 +27,7 @@ jobs:
# Intentionally tracked at @main (not SHA-pinned) so org-wide CI improvements, security fixes, and build optimizations propagate without PR churn across all craftapp repos.
uses: storytold/craft-common-actions/.github/workflows/release.yml@main
with:
dry-run: ${{ github.event_name == 'schedule' }}
app-name: gridcraft
app-title: GridCraft
craft-fonts-ref: b076417b5880bd4154bd22f00695b1d28c6bea83
Generated
+9 -7
View File
@@ -910,12 +910,12 @@ dependencies = [
[[package]]
name = "craft-fonts-build"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
[[package]]
name = "craft-i18n"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"fixed_decimal",
"icu_decimal",
@@ -938,7 +938,7 @@ dependencies = [
[[package]]
name = "craft-print"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"craft-winprint-gdi",
"log",
@@ -952,7 +952,7 @@ dependencies = [
[[package]]
name = "craft-ui"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"craft-fonts-build",
"craft-i18n",
@@ -972,7 +972,7 @@ dependencies = [
[[package]]
name = "craft-update"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"chrono",
"dunce",
@@ -982,6 +982,7 @@ dependencies = [
"image",
"log",
"plist",
"rustix 1.1.5",
"semver",
"serde",
"serde_json",
@@ -1018,7 +1019,7 @@ dependencies = [
[[package]]
name = "craft-window"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=1b34eba41e62138fc2438d2734523b302840b45d#1b34eba41e62138fc2438d2734523b302840b45d"
source = "git+https://github.com/storytold/craft-libs?rev=6e03f65bf45181db77ba6229716e6624db9eb25b#6e03f65bf45181db77ba6229716e6624db9eb25b"
dependencies = [
"egui",
"objc2 0.6.5",
@@ -6519,7 +6520,7 @@ dependencies = [
[[package]]
name = "winprint"
version = "0.2.1"
source = "git+https://github.com/storytold/winprint.rs?rev=51934db382f225454cf152a8657eb03789be63c4#51934db382f225454cf152a8657eb03789be63c4"
source = "git+https://github.com/storytold/winprint.rs?rev=bc06a7c48b768eb27a828c7986f6d2868b245ee0#bc06a7c48b768eb27a828c7986f6d2868b245ee0"
dependencies = [
"flate2",
"fmt-derive",
@@ -6534,6 +6535,7 @@ dependencies = [
"windows",
"windows-core",
"windows-implement",
"windows-numerics",
"xml",
]
+2 -2
View File
@@ -29,7 +29,7 @@ gridcraft-math = { path = "crates/math" }
# Every craft-libs crate (craft-update and craft-window here, craft-ui in ui-egui, craft-print in
# the app) comes from one craft-libs revision.
# craft-update: app-specific pretend versions apply to startup and manual checks.
craft-update = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d", version = "=0.1.0", features = ["egui"] }
craft-update = { git = "https://github.com/storytold/craft-libs", rev = "6e03f65bf45181db77ba6229716e6624db9eb25b", version = "=0.1.0", features = ["egui"] }
# foundation
thiserror = "2"
@@ -72,7 +72,7 @@ skrifa = { version = "0.44", default-features = false, features = ["std"] }
epaint_default_fonts = "0.37"
# One native window per workbook: the window list, Window menu titles and the immediate-viewport
# driver. Same craft-libs revision as craft-update above.
craft-window = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d", default-features = false, features = ["egui"] }
craft-window = { version = "0.1.0", git = "https://github.com/storytold/craft-libs", rev = "6e03f65bf45181db77ba6229716e6624db9eb25b", default-features = false, features = ["egui"] }
[workspace.lints.rust]
unsafe_code = "forbid"
+38 -18
View File
@@ -214,7 +214,7 @@ pub fn main(args: &[String]) -> ExitCode {
Err(_) => return usage(format!("--max-report needs a whole number, not `{n}`")),
}
}
let Discovery { files, failures: mut reports } = collect_inputs(&a.positional);
let Discovery { files, failures: mut reports, .. } = collect_inputs(&a.positional);
reports.extend(files.iter().map(|p| {
let path = p.to_string_lossy().into_owned();
match gridcraft_engine::io::read_file(&path) {
@@ -233,6 +233,9 @@ pub fn main(args: &[String]) -> ExitCode {
#[derive(Default)]
struct Discovery {
files: Vec<PathBuf>,
/// Identity (dev+inode) of each collected file, so a rescan reached by a shallower path doesn't add it twice.
#[cfg(unix)]
file_ids: std::collections::HashSet<(u64, u64)>,
failures: Vec<FileReport>,
}
@@ -257,11 +260,11 @@ impl Discovery {
/// (`~$…`) are skipped.
fn collect_inputs(inputs: &[String]) -> Discovery {
let mut out = Discovery::default();
let mut seen = std::collections::HashSet::new();
let mut seen = std::collections::HashMap::new();
for i in inputs {
let p = PathBuf::from(i);
if p.is_dir() {
if !walk(&p, 0, &mut out, &mut seen) {
if !walk(&p, 0, false, &mut out, &mut seen) {
break;
}
} else if !out.add_file(p) {
@@ -277,19 +280,29 @@ fn collect_inputs(inputs: &[String]) -> Discovery {
out
}
/// Returns false only when the global file budget stops traversal.
fn walk(dir: &Path, depth: usize, out: &mut Discovery, seen: &mut std::collections::HashSet<u64>) -> bool {
// A directory is visited once by identity (dev+inode on unix): several links to the same
// directory are aliases, not exponential branches of the scan. Other platforms keep the
// depth cap only.
/// Returns false only when the global file budget stops traversal. `seen` holds the shallowest
/// depth each directory was scanned from; `rescan` is set below a directory reached again from a
/// shallower path (its depth-limited subtree may now fit), where already collected files are skipped.
#[cfg_attr(not(unix), allow(unused_mut))]
fn walk(dir: &Path, depth: usize, mut rescan: bool, out: &mut Discovery, seen: &mut std::collections::HashMap<(u64, u64), usize>) -> bool {
// A directory is visited once by identity (dev+inode on unix) unless reached from a shallower
// path: several links to the same directory are aliases, not exponential branches of the scan
// (each directory is scanned at most MAX_DEPTH + 1 times). Other platforms keep the depth cap only.
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt as _;
if let Ok(m) = std::fs::metadata(dir)
&& let Some(id) = m.dev().checked_mul(1 << 60).and_then(|d| d.checked_add(m.ino()))
&& !seen.insert(id)
{
return true;
if let Ok(m) = std::fs::metadata(dir) {
let identity = (m.dev(), m.ino());
// A directory past the depth limit is not scanned, so it is not recorded: asked for
// by name later, it is walked from depth 0.
match seen.get(&identity) {
Some(&shallowest) if shallowest <= depth => return true,
Some(_) => rescan = true,
None => {}
}
if depth <= MAX_DEPTH {
seen.insert(identity, depth);
}
}
}
if depth > MAX_DEPTH {
@@ -324,13 +337,20 @@ fn walk(dir: &Path, depth: usize, out: &mut Discovery, seen: &mut std::collectio
}
};
if metadata.is_dir() {
if !walk(&p, depth + 1, out, seen) {
if !walk(&p, depth + 1, rescan, out, seen) {
return false;
}
} else if p.extension().and_then(std::ffi::OsStr::to_str).is_some_and(|e| e.eq_ignore_ascii_case("xlsx") || e.eq_ignore_ascii_case("xlsm")) {
#[cfg(unix)]
{
use std::os::unix::fs::MetadataExt as _;
if !out.file_ids.insert((metadata.dev(), metadata.ino())) && rescan {
continue;
}
}
if !out.add_file(p) {
return false;
}
} else if p.extension().and_then(std::ffi::OsStr::to_str).is_some_and(|e| e.eq_ignore_ascii_case("xlsx") || e.eq_ignore_ascii_case("xlsm"))
&& !out.add_file(p)
{
return false;
}
}
true
+43 -2
View File
@@ -11,8 +11,8 @@ fn discovery_reports_a_directory_read_error() {
let path = std::env::temp_dir().join(format!("gridcraft-check-not-a-directory-{}", std::process::id()));
std::fs::write(&path, "not a directory").unwrap();
let mut discovery = Discovery::default();
let mut seen = std::collections::HashSet::new();
assert!(walk(&path, 0, &mut discovery, &mut seen));
let mut seen = std::collections::HashMap::new();
assert!(walk(&path, 0, false, &mut discovery, &mut seen));
assert!(discovery.files.is_empty());
assert_eq!(discovery.failures.len(), 1);
assert_eq!(discovery.failures[0].path, path.to_string_lossy());
@@ -31,6 +31,47 @@ fn discovery_reports_omissions_only_after_the_file_limit() {
assert!(discovery.failures[0].error.as_ref().unwrap().contains("file limit"));
}
/// A new directory under the temp dir: `create_dir` fails on a name somebody else made first.
#[cfg(unix)]
fn fresh_dir(tag: &str) -> PathBuf {
let mut n = 0u32;
loop {
let dir = std::env::temp_dir().join(format!("gridcraft-check-{tag}-{}-{n}", std::process::id()));
match std::fs::create_dir(&dir) {
Ok(()) => return dir,
Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => n += 1,
Err(e) => panic!("{e}"),
}
}
}
/// A directory named again by a deeper path, or by a link back to a parent, lists each workbook once.
#[cfg(unix)]
#[test]
fn discovery_lists_each_workbook_once_whatever_path_reaches_it() {
let dir = fresh_dir("once");
let sub = dir.join("sub");
std::fs::create_dir(&sub).unwrap();
std::fs::write(dir.join("a.xlsx"), b"x").unwrap();
std::fs::write(sub.join("b.xlsx"), b"x").unwrap();
std::os::unix::fs::symlink(&dir, sub.join("loop")).unwrap();
let names = |d: &Discovery| {
let mut v: Vec<String> = d.files.iter().map(|p| p.file_name().unwrap().to_string_lossy().into_owned()).collect();
v.sort();
v
};
// The link back to the top is one directory, not another branch of the scan.
let found = collect_inputs(&[dir.to_string_lossy().into_owned()]);
assert_eq!(names(&found), ["a.xlsx", "b.xlsx"]);
// `sub` named after its parent is reached from a shallower depth: scanned again, nothing listed twice.
let found = collect_inputs(&[dir.to_string_lossy().into_owned(), sub.to_string_lossy().into_owned()]);
assert_eq!(names(&found), ["a.xlsx", "b.xlsx"]);
// Named first, the same `sub` is the shallowest path and the parent's scan skips it.
let found = collect_inputs(&[sub.to_string_lossy().into_owned(), dir.to_string_lossy().into_owned()]);
assert_eq!(names(&found), ["a.xlsx", "b.xlsx"]);
let _ = std::fs::remove_dir_all(dir);
}
fn at(a1: &str) -> CellRef {
CellRef::parse(a1).unwrap()
}
+38
View File
@@ -205,6 +205,44 @@ fn check_reports_a_depth_limited_scan_while_checking_readable_files() {
let _ = std::fs::remove_dir_all(dir);
}
#[cfg(unix)]
#[test]
fn a_directory_past_the_depth_limit_is_still_scanned_when_named_as_an_input() {
let dir = tmpdir("check-depth-named");
// The first folder past the limit (33 levels down): the scan from the top reports it unreached.
let mut deep = dir.clone();
for _ in 0..33 {
deep.push("d");
}
std::fs::create_dir_all(&deep).unwrap();
ok(&["run", "--out", deep.join("deep.xlsx").to_str().unwrap(), "--quiet"]);
let out = run(&["check", dir.to_str().unwrap(), deep.to_str().unwrap(), "--json"]);
let report: serde_json::Value = serde_json::from_slice(&out.stdout).unwrap();
// The first scan reports the subtree it could not reach; the named directory is checked.
assert_eq!(report["totals"]["opened"], 1, "{report}");
let _ = std::fs::remove_dir_all(dir);
}
#[cfg(unix)]
#[test]
fn a_directory_first_reached_at_the_depth_limit_is_completed_when_named_as_an_input() {
let dir = tmpdir("check-depth-shallower");
// The 32nd folder is scanned from the top, but its child (and the workbook in it) is past the limit.
let mut limit = dir.clone();
for _ in 0..32 {
limit.push("d");
}
let child = limit.join("c");
std::fs::create_dir_all(&child).unwrap();
ok(&["run", "--out", limit.join("at_limit.xlsx").to_str().unwrap(), "--quiet"]);
ok(&["run", "--out", child.join("below.xlsx").to_str().unwrap(), "--quiet"]);
let out = run(&["check", dir.to_str().unwrap(), limit.to_str().unwrap(), "--json"]);
let report: serde_json::Value = serde_json::from_slice(&out.stdout).unwrap();
// Named from depth 0 the subtree is read: the workbook below the limit is checked, the one at it only once.
assert_eq!(report["totals"]["opened"], 2, "{report}");
let _ = std::fs::remove_dir_all(dir);
}
/// A script line whose parameters are under a key `run` does not read used to run the command
/// with no parameters at all, and the run still exited 0: three mistaken lines made Sheet2,
/// Sheet3 and Sheet4 instead of the named sheets, and nothing said so (#2256).
+1 -1
View File
@@ -35,7 +35,7 @@ winit = { version = "0.30", default-features = false, features = ["x11", "waylan
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
# Same craft-libs revision as the workspace's craft-update and craft-window (see the root
# Cargo.toml).
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d" }
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "6e03f65bf45181db77ba6229716e6624db9eb25b" }
craft-update = { workspace = true }
rfd = { workspace = true }
arboard = { workspace = true, features = ["image-data"] }
+9 -2
View File
@@ -147,10 +147,10 @@ impl RangeRef {
RangeRef::new(CellRef::new(0, 0), CellRef::new(MAX_ROWS - 1, MAX_COLS - 1))
}
pub fn height(&self) -> u32 {
self.end.row - self.start.row + 1
self.end.row.saturating_sub(self.start.row).saturating_add(1)
}
pub fn width(&self) -> u32 {
self.end.col - self.start.col + 1
self.end.col.saturating_sub(self.start.col).saturating_add(1)
}
pub fn count(&self) -> u64 {
self.height() as u64 * self.width() as u64
@@ -321,6 +321,13 @@ pub fn split_areas(reference: &str) -> Vec<&str> {
mod tests {
use super::*;
#[test]
fn hostile_range_dimensions_saturate_instead_of_wrapping() {
let r = RangeRef { start: CellRef::new(0, 0), end: CellRef::new(u32::MAX, u32::MAX) };
assert_eq!((r.height(), r.width()), (u32::MAX, u32::MAX));
assert_eq!(r.count(), u64::from(u32::MAX) * u64::from(u32::MAX));
}
#[test]
fn areas_split_only_at_commas_outside_quoted_sheet_names() {
assert_eq!(split_areas("B2:B4,D1"), ["B2:B4", "D1"]);
+1 -1
View File
@@ -98,7 +98,7 @@ pub fn specs() -> Vec<CommandSpec> {
validation
),
cmd!(query "data.validate", "Check Value Against Validation", [], None, "{cell?, input} → {ok, message?}", has_doc, validate_cmd),
cmd!(noundo "data.circleInvalid", "Circle Invalid Data", ["Data", "Data Tools", "Data Validation"], None, "{} → invalid cells", has_doc, circle_invalid),
cmd!(noundo "data.circleInvalid", "Circle Invalid Data", ["Data", "Data Tools", "Data Validation"], None, "{} → {invalid: [cell], truncated: bool (true when the 100,000-cell scan allowance ended before every range was checked)}", has_doc, circle_invalid),
CommandSpec {
shortcut_mac: Some("Cmd+Shift+K"),
..cmd!(
+2 -1
View File
@@ -184,7 +184,8 @@ pub(crate) fn grow_autofilter(sh: &mut Sheet, at: CellRef) {
pub(crate) fn apply_filter(wb: &mut Workbook, sheet: usize) {
let Some(sh) = wb.sheet(sheet) else { return };
let Some(af) = sh.autofilter.clone() else { return };
let r = af.range;
// An imported range can name rows past the grid; none of those exist to hide.
let r = RangeRef::new(af.range.start, CellRef::new(af.range.end.row.min(gridcraft_core::MAX_ROWS - 1), af.range.end.col));
let mut hide = Vec::new();
// Precompute per-criterion top10 threshold and average once, not per row.
let mut col_stats: std::collections::HashMap<usize, (Option<f64>, f64)> = std::collections::HashMap::new();
+7
View File
@@ -5,6 +5,9 @@ use super::{
// ---------------------------------------------------------------- tools
/// Most cells (rows times compared columns) one Remove Duplicates call reads.
const MAX_COMPARED_CELLS: u64 = 50_000_000;
pub(super) fn remove_duplicates(s: &mut Session, p: &Json) -> Result<Json> {
let (sheet, r) = data_target(s, p, "data.removeDuplicates")?;
let d = s.doc()?;
@@ -24,6 +27,10 @@ pub(super) fn remove_duplicates(s: &mut Session, p: &Json) -> Result<Json> {
if cols.is_empty() || cols.iter().any(|c| !(r.start.col..=r.end.col).contains(c)) {
return Err(bad("data.removeDuplicates", "`columns` must name at least one column of the range"));
}
// Rows are compared one by one, so a sparse sheet's enclosing rectangle is bounded like a list.
if r.count() > MAX_COMPARED_CELLS {
return Err(bad("data.removeDuplicates", "the range is too large"));
}
let start = r.start.row + header as u32;
let mut removed = 0;
edit(s, |cx| {
+8 -2
View File
@@ -337,11 +337,17 @@ pub(super) fn circle_invalid(s: &mut Session, _: &Json) -> Result<Json> {
let sheet = d.wb.active_sheet();
let Some(sh) = d.wb.sheet(sheet) else { return Err(EngineError::NoDocument) };
let mut bad_cells = Vec::new();
// One scan allowance for the whole command: duplicate or overlapping ranges don't multiply it.
let mut budget = 100_000usize;
// Cells were left unchecked because the allowance ran out: said in the result.
let mut truncated = false;
for dv in &sh.validations {
for r in &dv.ranges {
let r = sh.used_range().and_then(|u| r.intersection(&u));
let Some(r) = r else { continue };
for c in r.iter().take(100_000) {
truncated |= r.count() > budget as u64;
for c in r.iter().take(budget) {
budget -= 1;
let text = sh.input_text(c);
if !text.is_empty() && check_validation(&d.wb, sheet, c, &text).is_some() {
bad_cells.push(c.a1());
@@ -349,5 +355,5 @@ pub(super) fn circle_invalid(s: &mut Session, _: &Json) -> Result<Json> {
}
}
}
Ok(json!({"invalid": bad_cells}))
Ok(json!({"invalid": bad_cells, "truncated": truncated}))
}
+5 -1
View File
@@ -199,7 +199,11 @@ pub(super) fn replace(s: &mut Session, p: &Json) -> Result<Json> {
let mut total = 0usize;
for si in sheets {
let cells: Vec<(CellRef, String)> =
cx.wb.sheet(si).map(|sh| sh.cells.iter().map(|(c, _)| (c, sh.input_text(c))).collect()).unwrap_or_default();
// A hidden formula's source is not searched: a count or error would reveal its text.
cx.wb
.sheet(si)
.map(|sh| sh.cells.iter().filter(|(c, cell)| cell.formula.is_none() && cell.py.is_none() || !crate::display::formula_hidden(&cx.wb, sh, *c)).map(|(c, _)| (c, sh.input_text(c))).collect())
.unwrap_or_default();
for (c, text) in cells {
if (!all && c != active) || selection.as_ref().is_some_and(|sel| !sel.contains(c)) {
continue;
+15
View File
@@ -891,6 +891,21 @@ mod tests {
assert!((0.75..=1.15).contains(&ratio), "Arabic {} vs Latin {}", width(0), width(1));
}
#[test]
fn autofit_of_many_separate_columns_fits_only_the_selected_ones() {
let mut s = crate::Session::new();
s.new_workbook();
let long = "a much longer text than the others";
// Columns A, C, E, G, I, K are selected (six runs), the ones between hold long text too.
let row: Vec<_> = (0..11).map(|c| if c % 2 == 0 { json!("x") } else { json!(long) }).collect();
s.execute("range.setValues", json!({"range": "A1", "values": [row]})).unwrap();
s.execute("home.autofitColumnWidth", json!({"cols": "A:A,C:C,E:E,G:G,I:I,K:K"})).unwrap();
let doc = s.doc().unwrap();
let size = |c: u32| doc.wb.active().and_then(|sh| sh.cols.get(&c)).and_then(|e| e.size);
assert!([0, 2, 4, 6, 8, 10].iter().all(|&c| size(c).is_some()));
assert!([1, 3, 5, 7, 9].iter().all(|&c| size(c).is_none()));
}
#[test]
fn arabic_word_ligatures_are_wider_than_letters() {
let em = |t: &str| approx_text_width(t, 72.0, false) / 96.0;
+31 -12
View File
@@ -194,16 +194,35 @@ pub(super) fn autofit_cols(s: &mut Session, p: &Json) -> Result<Json> {
let measure = s.text_width.clone();
edit(s, |cx| {
let Some(sh) = cx.wb.sheet(si) else { return Ok(Json::Null) };
let mut widths = Vec::new();
let mut measured: std::collections::HashMap<StyleId, std::collections::HashMap<String, f32>> = Default::default();
let mut distinct = 0usize;
let used = sh.used_range();
for (a, b) in lines {
for c in a..=b.min(a + 16383) {
let Some(u) = used else { continue };
let col = RangeRef::new(CellRef::new(u.start.row, c), CellRef::new(u.end.row, c));
let mut max: f32 = 0.0;
for (cell, _) in sh.cells.iter_range(col) {
// Selected columns, then one pass over the populated cells (the store is row-oriented, so a
// scan per column would revisit every row for each selected column, empty or not).
let cols: std::collections::BTreeSet<u32> = lines.iter().flat_map(|&(a, b)| a..=b.min(a + 16383)).collect();
let mut maxes: std::collections::BTreeMap<u32, f32> = Default::default();
if let Some(u) = sh.used_range() {
// One pass per run of adjacent selected columns, so columns between distant selections
// (A and XFD) are never visited.
let mut runs: Vec<(u32, u32)> = Vec::new();
for &c in &cols {
match runs.last_mut() {
Some((_, end)) if *end + 1 == c => *end = c,
_ => runs.push((c, c)),
}
}
// Every run is a pass over all the stored rows: many runs (alternating columns) are one
// pass over their whole span instead, the unselected columns skipped.
if runs.len() > 4
&& let (Some(&(first, _)), Some(&(_, last))) = (runs.first(), runs.last())
{
runs = vec![(first, last)];
}
for (first, last) in runs {
let span = RangeRef::new(CellRef::new(u.start.row, first), CellRef::new(u.end.row, last));
for (cell, _) in sh.cells.iter_range(span) {
if !cols.contains(&cell.col) {
continue;
}
if sh.merge_at(cell).is_some_and(|m| !m.is_single()) {
continue;
}
@@ -226,14 +245,14 @@ pub(super) fn autofit_cols(s: &mut Session, p: &Json) -> Result<Json> {
}
None => approx_text_width(line, st.font.size, st.font.bold),
};
max = max.max(w + 6.0 + st.align.indent as f32 * 9.0 + sh.filter_button_space(cell));
let w = w + 6.0 + st.align.indent as f32 * 9.0 + sh.filter_button_space(cell);
let max = maxes.entry(cell.col).or_insert(0.0);
*max = max.max(w);
}
}
if max > 0.0 {
widths.push((c, max.clamp(8.0, 1785.0)));
}
}
}
let widths: Vec<(u32, f32)> = maxes.into_iter().filter(|&(_, m)| m > 0.0).map(|(c, m)| (c, m.clamp(8.0, 1785.0))).collect();
let sh = cx.sheet_mut(si)?;
for (c, w) in widths {
let e = sh.cols.entry(c).or_default();
+64 -3
View File
@@ -32,8 +32,69 @@ use header_footer::draw_header_footer;
pub use header_footer::{
HF_FIELDS, HfContext, HfSection, codes_to_fields, expand_header_footer, fields_to_codes, hf_date_time, join_header_footer, split_header_footer,
};
/// Decoded pictures (bytes, width, height) keyed by (sheet, picture); `None` = undecodable.
type PicCache = std::collections::HashMap<(usize, usize), Option<(Vec<u8>, u32, u32)>>;
type DecodedPicture = (Vec<u8>, u32, u32);
/// What one PDF render shares across the pages an object spans.
#[derive(Default)]
struct PicCache {
/// Decoded pictures (bytes, width, height) keyed by (sheet, picture); `None` = undecodable.
decoded: std::collections::HashMap<(usize, usize), Option<DecodedPicture>>,
/// Laid-out charts (with their [`chart_cost`]) keyed by (sheet, chart), rendered once however
/// many pages they touch. Only charts that can still be drawn are kept.
charts: std::collections::HashMap<(usize, usize), (Vec<gridcraft_chart::Prim>, usize)>,
/// Charts too big for what was left of the budget: never drawn, so not rendered again.
rejected_charts: std::collections::HashSet<(usize, usize)>,
/// Bytes of picture pixels embedded so far, and chart cost units emitted so far. Each page an
/// object touches gets its own copy in the PDF, so a workbook-controlled anchor over thousands
/// of pages would multiply them without bound.
image_bytes: usize,
chart_prims: usize,
}
impl PicCache {
/// The chart's primitives for one more page, or `None` once they no longer fit
/// [`MAX_CHART_PRIMS`]. `render` runs at most once per chart; a chart that doesn't fit is
/// dropped at once and remembered, so its vectors are never retained.
fn chart(&mut self, key: (usize, usize), render: impl FnOnce() -> Vec<gridcraft_chart::Prim>) -> Option<&[gridcraft_chart::Prim]> {
if self.rejected_charts.contains(&key) {
return None;
}
let (prims, cost) = match self.charts.remove(&key) {
Some(cached) => cached,
None => {
let prims = render();
let cost = chart_cost(&prims);
(prims, cost)
}
};
if self.chart_prims.saturating_add(cost) > MAX_CHART_PRIMS {
self.rejected_charts.insert(key);
return None;
}
self.chart_prims += cost;
Some(&self.charts.entry(key).or_insert((prims, cost)).0)
}
}
/// What a chart's drawing costs in the PDF: one unit per primitive plus one per path point (a
/// smoothed line holds thousands of points, each written out) and per point of a wedge's outline.
fn chart_cost(prims: &[gridcraft_chart::Prim]) -> usize {
use gridcraft_chart::Prim;
prims
.iter()
.map(|p| match p {
Prim::Line { pts, .. } | Prim::Polygon { pts, .. } => pts.len().saturating_add(1),
// At most 400 steps along each of the outer and inner arcs (see `draw_prims`).
Prim::Wedge { .. } => 803,
_ => 1,
})
.fold(0, usize::saturating_add)
}
/// Most picture pixel bytes (RGBA, before PDF packing) one render embeds across all pages.
const MAX_EMBEDDED_IMAGE_BYTES: usize = 512 * 1024 * 1024;
/// Most chart cost units (primitives and path points, see [`chart_cost`]) one render draws across all pages.
const MAX_CHART_PRIMS: usize = 4_000_000;
#[cfg(test)]
use layout::bands;
@@ -102,7 +163,7 @@ pub fn render_pdf(wb: &Workbook, layouts: &[SheetLayout], file: &str, path: &str
let mut printed_blocks = Vec::new();
// Decoded pictures are reused across the pages their anchor covers: a floating picture
// spanning 40 pages was decoded (and pushed into the PDF) 40 times (#2389).
let mut pics = PicCache::new();
let mut pics = PicCache::default();
for lay in layouts {
let Some(sh) = wb.sheet(lay.sheet) else { continue };
let mut cf = if sh.cond_formats.is_empty() { None } else { Some(crate::cf::CfCache::new()) };
+22
View File
@@ -224,6 +224,12 @@ pub fn layout_sheet(wb: &Workbook, si: usize, opts: &PrintOptions, range_overrid
Some(r) => trim_full(sh, r),
None => print_range(sh).unwrap_or(RangeRef::cell(CellRef::new(0, 0))),
};
// A print area from a file or JSON can name coordinates past the grid; pagination walks every
// line of the area (zero-height rows never fill a band), so it is held to the grid first.
let area = RangeRef::new(
CellRef::new(area.start.row.min(MAX_ROWS - 1), area.start.col.min(MAX_COLS - 1)),
CellRef::new(area.end.row.min(MAX_ROWS - 1), area.end.col.min(MAX_COLS - 1)),
);
let title_rows = title_span(st.title_rows, MAX_ROWS);
let title_cols = title_span(st.title_cols, MAX_COLS);
let digits = format!("{}", area.end.row + 1).len() as f32;
@@ -379,3 +385,19 @@ pub(super) fn segments(titles: Option<(u32, u32)>, band: (u32, u32)) -> Vec<(u32
pub(super) fn seg_size(sh: &Sheet, seg: (u32, u32), rows: bool) -> f32 {
if rows { (seg.0..=seg.1).map(|r| sh.row_height(r)).sum() } else { (seg.0..=seg.1).map(|c| sh.col_width(c)).sum() }
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_print_area_past_the_grid_is_held_to_the_grid() {
let mut wb = Workbook::new();
let sh = wb.sheet_mut(0).unwrap();
sh.default_row_height = 0.0;
sh.print.print_area = Some(RangeRef { start: CellRef::new(0, 0), end: CellRef::new(u32::MAX, u32::MAX) });
let lay = layout_sheet(&wb, 0, &PrintOptions::default(), None).unwrap();
assert_eq!((lay.area.end.row, lay.area.end.col), (MAX_ROWS - 1, MAX_COLS - 1));
assert!(lay.pages.len() <= MAX_PAGES);
}
}
+16 -6
View File
@@ -88,9 +88,14 @@ pub(super) fn draw_objects(pg: &mut Page, wb: &Workbook, si: usize, sh: &Sheet,
let (x, y, w, h) = place(r);
// About 150 dpi is plenty for print.
let (mw, mh) = (((w / 72.0) * 150.0).ceil().clamp(1.0, 4000.0) as u32, ((h / 72.0) * 150.0).ceil().clamp(1.0, 4000.0) as u32);
let decoded = pics.entry((si, imi)).or_insert_with(|| decode_image(&im.data, mw, mh));
let decoded = pics.decoded.entry((si, imi)).or_insert_with(|| decode_image(&im.data, mw, mh));
if let Some((px, pw, ph)) = decoded.as_ref() {
pg.image_rgba(Rect::new(x, y, w, h), px, [*pw, *ph]);
// Every page the picture touches embeds its own copy: past the budget it is left out.
let bytes = px.len();
if pics.image_bytes.saturating_add(bytes) <= super::MAX_EMBEDDED_IMAGE_BYTES {
pics.image_bytes += bytes;
pg.image_rgba(Rect::new(x, y, w, h), px, [*pw, *ph]);
}
}
}
for shp in &sh.shapes {
@@ -120,15 +125,20 @@ pub(super) fn draw_objects(pg: &mut Page, wb: &Workbook, si: usize, sh: &Sheet,
}
draw_shape(pg, wb, shp, b, (x, y, w, h));
}
for ch in &sh.charts {
for (chi, ch) in sh.charts.iter().enumerate() {
let r = anchor_rect(sh, &ch.anchor);
if !visible(r) {
continue;
}
let (x, y, _, _) = place(r);
let data = gridcraft_chart::resolve(wb, si, ch);
let prims = gridcraft_chart::render(ch, &data, r.2 as f32, r.3 as f32, &PdfMeasure);
draw_prims(pg, &prims, Xf { ox: x, oy: y, k: b.k });
// As for pictures, each page gets its own copy of the drawing commands.
let prims = pics.chart((si, chi), || {
let data = gridcraft_chart::resolve(wb, si, ch);
gridcraft_chart::render(ch, &data, r.2 as f32, r.3 as f32, &PdfMeasure)
});
if let Some(prims) = prims {
draw_prims(pg, prims, Xf { ox: x, oy: y, k: b.k });
}
}
pg.restore();
}
+24
View File
@@ -938,3 +938,27 @@ fn equations_print_with_the_symbol_font() {
// A glyph no standard font has is a [?] box.
assert!(contains(&pdf, "(?) Tj"));
}
#[test]
fn chart_budget_counts_path_points_and_drops_charts_that_do_not_fit() {
use gridcraft_chart::Prim;
let line = |n: usize| Prim::Line { pts: vec![[0.0, 0.0]; n], color: [0, 0, 0, 255], width: 1.0, dash: false };
// A smoothed line is its points, not one unit.
assert_eq!(chart_cost(&[line(4000), Prim::Circle { cx: 0.0, cy: 0.0, r: 1.0, fill: None, stroke: None }]), 4002);
// Room for exactly two copies of a 1,000-point chart.
let mut pics = PicCache { chart_prims: MAX_CHART_PRIMS - 2002, ..Default::default() };
let mut renders = 0;
for expect in [true, true, false, false] {
let got = pics.chart((0, 0), || {
renders += 1;
vec![line(1000)]
});
assert_eq!(got.is_some(), expect);
}
assert_eq!(renders, 1, "a rejected chart is not rendered again");
assert!(pics.charts.is_empty(), "and its primitives are not kept");
// A chart too big for what is left is dropped without being cached at all.
let mut pics = PicCache { chart_prims: MAX_CHART_PRIMS - 10, ..Default::default() };
assert!(pics.chart((0, 1), || vec![line(1000)]).is_none());
assert!(pics.charts.is_empty() && pics.rejected_charts.contains(&(0, 1)));
}
@@ -83,6 +83,9 @@ pub(crate) fn criterion_matches(v: &Value, text: &str, crit: &Value) -> bool {
}
}
/// Most (list row, criteria row) pairs one Advanced Filter tests.
const MAX_FILTER_TESTS: u64 = 50_000_000;
pub(super) fn advanced_filter(s: &mut Session, p: &Json) -> Result<Json> {
const CMD: &str = "data.advancedFilter";
let d = s.doc()?;
@@ -133,6 +136,10 @@ pub(super) fn advanced_filter(s: &mut Session, p: &Json) -> Result<Json> {
rows.push(conds);
}
}
// Each list row is tried against every criteria row, so bound their product too.
if u64::from(list.height()) * rows.len() as u64 > MAX_FILTER_TESTS {
return Err(bad(CMD, "the list and criteria ranges together are too large"));
}
let matches_row = |row: u32| -> bool {
if rows.is_empty() {
return true;
+9
View File
@@ -298,6 +298,15 @@ fn consolidate_by_position_and_label() {
assert!(s.execute("data.consolidate", json!({"sources": ["A1"], "function": "median"})).is_err());
}
#[test]
fn advanced_filter_refuses_a_list_times_criteria_product_over_the_work_limit() {
let mut s = s();
s.execute("range.setValues", json!({"range": "A1", "values": [["Name"], ["A"]]})).unwrap();
s.execute("range.setValues", json!({"range": "H1", "values": [["Name"], ["A"]]})).unwrap();
// A sparse million-row list against 9,999 blank criteria rows would be ~10 billion tests.
assert!(s.execute("data.advancedFilter", json!({"range": "A2:A1048576", "criteria": "H1:H10000"})).is_err());
}
/// A hundred thousand positional rows take one key lookup each; a tall source and a wide one
/// whose keys would meet in a table of 160 million cells are refused before it is built.
#[test]
+106 -21
View File
@@ -3,7 +3,7 @@
use std::fmt::Write as _;
use gridcraft_core::{CellRef, RangeRef};
use gridcraft_model::{Sheet, Workbook};
use gridcraft_model::{LineIndex, Sheet, Workbook};
use crate::{EngineError, Result};
@@ -611,6 +611,24 @@ pub(crate) fn esc(s: &str) -> String {
s.replace('&', "&amp;").replace('<', "&lt;").replace('>', "&gt;").replace('"', "&quot;")
}
/// [`esc`], or `None` once the escaped text would pass `limit` bytes (nothing larger is built).
fn esc_within(s: &str, limit: usize) -> Option<String> {
let mut out = String::with_capacity(s.len().min(limit));
for c in s.chars() {
match c {
'&' => out.push_str("&amp;"),
'<' => out.push_str("&lt;"),
'>' => out.push_str("&gt;"),
'"' => out.push_str("&quot;"),
c => out.push(c),
}
if out.len() > limit {
return None;
}
}
Some(out)
}
fn html_cell_css(wb: &Workbook, sh: &Sheet, c: CellRef, dir: CellDir) -> String {
let st = wb.styles.get(sh.style_id(c));
let mut css = String::new();
@@ -686,6 +704,30 @@ fn table_dir(sh: &Sheet) -> &'static str {
const CLIPBOARD_HTML_MAX_BYTES: usize = 4 * 1024 * 1024;
const CLIPBOARD_HTML_MAX_CELLS: u64 = 10_000;
/// Most cells the merge index holds: one entry per merged cell inside the exported range.
const MERGE_INDEX_MAX_ENTRIES: usize = 1_000_000;
/// The merge covering each cell of `rect`, as `Sheet::merge_at` would find it (the first in the
/// sheet's list), built once so a table lookup is one probe instead of a scan of every merge.
/// `None` when overlapping merges would cost more than a few visits per cell of `rect`, or merged
/// cells fill more than [`MERGE_INDEX_MAX_ENTRIES`]: a hostile file's merge list can't multiply the
/// work or the memory of a bounded export.
fn merge_index(sh: &Sheet, rect: RangeRef) -> Option<std::collections::HashMap<CellRef, RangeRef>> {
let mut map = std::collections::HashMap::new();
let mut left = rect.count().saturating_mul(4);
for m in &sh.merges {
let Some(part) = m.intersection(&rect) else { continue };
left = left.checked_sub(part.count())?;
for c in part.iter() {
map.entry(c).or_insert(*m);
}
if map.len() > MERGE_INDEX_MAX_ENTRIES {
return None;
}
}
Some(map)
}
/// An HTML fragment for the same range as `plain_text`, previously produced by
/// `display::range_text`. Large copies keep plain text only; never return a partial table.
pub(crate) fn range_html(wb: &Workbook, sheet: usize, r: RangeRef, plain_text: &str) -> Option<String> {
@@ -694,6 +736,7 @@ pub(crate) fn range_html(wb: &Workbook, sheet: usize, r: RangeRef, plain_text: &
return None;
}
let sh = wb.sheet(sheet)?;
let merges = merge_index(sh, r)?;
let mut out = ClipboardHtml(String::new());
write!(out, "<table{} style=\"border-collapse:collapse;font-family:Calibri,Carlito,Arial,sans-serif;font-size:11pt\">", table_dir(sh)).ok()?;
for row in r.start.row..=r.end.row {
@@ -703,7 +746,7 @@ pub(crate) fn range_html(wb: &Workbook, sheet: usize, r: RangeRef, plain_text: &
out.write_str("<tr>").ok()?;
for col in r.start.col..=r.end.col {
let c = CellRef::new(row, col);
let span = if let Some(merged) = sh.merges.iter().find(|m| m.contains(c)).and_then(|m| m.intersection(&r)) {
let span = if let Some(merged) = merges.get(&c).and_then(|m| m.intersection(&r)) {
let first_row = (merged.start.row..=merged.end.row).find(|&row| !sh.is_row_hidden(row))?;
if c != CellRef::new(first_row, merged.start.col) {
continue;
@@ -766,6 +809,14 @@ const HTML_EXPORT_MAX_ROWS: u32 = 100_001;
/// tall, wide one) would otherwise be gigabytes of `<td></td>` before finishing (#2387).
const HTML_EXPORT_MAX_CELLS: u64 = 4_000_000;
/// Most bytes of one web page. Every cell may hold long text (shared across many cells in the
/// file), so the cell count alone doesn't bound the page; past this the export fails instead.
const HTML_EXPORT_MAX_BYTES: usize = 256 * 1024 * 1024;
fn html_too_big() -> EngineError {
EngineError::Other(format!("the web page would be over {} MiB. Save as .xlsx or CSV instead.", HTML_EXPORT_MAX_BYTES / (1024 * 1024)))
}
/// A sheet as a standalone HTML table (Save as Web Page). A sheet with pictures gets the sheet's
/// own column widths and row heights so each picture and chart (inline SVG) can sit at its anchor.
pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
@@ -808,6 +859,24 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
r.width()
)));
}
let merges = merge_index(sh, r)
.ok_or_else(|| EngineError::Other("the sheet's merged cells overlap too much, or cover too many cells, for a web page.".into()))?;
// Visible rows and columns of the table before each line, so a merge's span is counted
// in constant time whatever the file says its size is.
let visible_before = |hidden: &dyn Fn(u32) -> bool, from: u32, to: u32| {
let mut v = Vec::with_capacity((to - from) as usize + 2);
v.push(0u32);
for line in from..=to {
v.push(v.last().copied().unwrap_or(0) + u32::from(!hidden(line)));
}
v
};
let rows_before = visible_before(&|y| sh.is_row_hidden(y), r.start.row, r.end.row);
let cols_before = visible_before(&|x| sh.is_col_hidden(x), r.start.col, r.end.col);
let visible = |before: &[u32], from: u32, a: u32, b: u32| {
let at = |line: u32| before.get((line - from) as usize).copied().unwrap_or(0);
at(b + 1) - at(a)
};
if fixed {
out.push_str("<colgroup>");
for col in (r.start.col..=r.end.col).filter(|&c| !sh.is_col_hidden(c)) {
@@ -826,27 +895,31 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
}
for col in r.start.col..=r.end.col {
let c = CellRef::new(row, col);
if sh.merges.iter().any(|m| m.contains(c) && m.start != c) || (fixed && sh.is_col_hidden(col)) {
let merge = merges.get(&c);
if merge.is_some_and(|m| m.start != c) || (fixed && sh.is_col_hidden(col)) {
continue;
}
let text = crate::display::cell_text(wb, sh, c);
let dir = CellDir::of(sh, c, &text);
let css = html_cell_css(wb, sh, c, dir);
let span = sh
.merges
.iter()
.find(|m| m.start == c)
let span = merge
.map(|m| {
if fixed {
let rows = (m.start.row..=m.end.row).filter(|&y| !sh.is_row_hidden(y)).count();
let cols = (m.start.col..=m.end.col).filter(|&x| !sh.is_col_hidden(x)).count();
// Spans count the table's visible lines: the merge is clipped to the table.
let rows = visible(&rows_before, r.start.row, m.start.row, m.end.row.min(r.end.row));
let cols = visible(&cols_before, r.start.col, m.start.col, m.end.col.min(r.end.col));
format!(" rowspan=\"{rows}\" colspan=\"{cols}\"")
} else {
format!(" rowspan=\"{}\" colspan=\"{}\"", m.height(), m.width())
}
})
.unwrap_or_default();
let _ = write!(out, "<td style=\"{css}\"{}{span}>{}</td>", dir.attr(), esc(&text));
// The text is escaped only as far as the page still has room for it.
let Some(escaped) = esc_within(&text, HTML_EXPORT_MAX_BYTES.saturating_sub(out.len())) else { return Err(html_too_big()) };
let _ = write!(out, "<td style=\"{css}\"{}{span}>{escaped}</td>", dir.attr());
if out.len() > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
}
out.push_str("</tr>");
}
@@ -854,36 +927,48 @@ pub fn to_html(wb: &Workbook, sheet: usize) -> Result<String> {
out.push_str("</table>");
if fixed {
// Sheet units are CSS pixels; the table starts at the used range, not at A1.
let (left, top) = used.map_or((0.0, 0.0), |r| (sh.col_left(r.start.col), sh.row_top(r.start.row)));
// Indexed positions: each anchor would otherwise sum every custom row and column before it.
let ri = LineIndex::new(&sh.rows, sh.default_row_height, gridcraft_core::MAX_ROWS);
let ci = LineIndex::new(&sh.cols, sh.default_col_width, gridcraft_core::MAX_COLS);
let (left, top) = used.map_or((0.0, 0.0), |r| (ci.start(r.start.col), ri.start(r.start.row)));
let place = |a: &gridcraft_model::Anchor| {
format!(
"position:absolute;left:{}px;top:{}px;width:{}px;height:{}px",
sh.col_left(a.cell.col) - left + f64::from(a.dx),
sh.row_top(a.cell.row) - top + f64::from(a.dy),
ci.start(a.cell.col) - left + f64::from(a.dx),
ri.start(a.cell.row) - top + f64::from(a.dy),
a.width.max(0.0),
a.height.max(0.0)
)
};
for im in &sh.images {
let _ = write!(
out,
"<img src=\"data:{};base64,{}\" alt=\"{}\" style=\"{}\">",
esc(&im.mime),
base64_encode(&im.data),
esc(&im.alt),
place(&im.anchor)
);
// The base64 text is a third longer than the picture: refused before it is built.
if out.len().saturating_add(im.data.len().div_ceil(3).saturating_mul(4)) > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
// The attributes are escaped only as far as the page has room for them.
let room = HTML_EXPORT_MAX_BYTES.saturating_sub(out.len());
let (Some(mime), Some(alt)) = (esc_within(&im.mime, room), esc_within(&im.alt, room)) else { return Err(html_too_big()) };
let _ = write!(out, "<img src=\"data:{mime};base64,{}\" alt=\"{alt}\" style=\"{}\">", base64_encode(&im.data), place(&im.anchor));
if out.len() > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
}
let mut svg_budget = crate::chart_svg::MAX_PAGE_SVG_BYTES;
for ch in &sh.charts {
if let Some(svg) = crate::chart_svg::chart_svg(wb, sheet, ch, &place(&ch.anchor), svg_budget)? {
svg_budget -= svg.len();
out.push_str(&svg);
if out.len() > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
}
}
out.push_str("</div>");
}
out.push_str("</body></html>");
if out.len() > HTML_EXPORT_MAX_BYTES {
return Err(html_too_big());
}
Ok(out)
}
+71
View File
@@ -810,3 +810,74 @@ fn reading_a_fifo_is_refused_without_blocking() {
assert!(err.contains("only regular files"), "{err}");
let _ = std::fs::remove_dir_all(&dir);
}
/// Merges far from what is copied or exported, in a hostile number, are indexed once instead of
/// being scanned for every cell; a merge running past the table is clipped to it.
#[test]
fn html_merge_lookups_and_spans_stay_bounded() {
let mut wb = book_with_picture("A1", (0, 0));
let sh = wb.sheet_mut(0).unwrap();
for i in 0..200_000u32 {
sh.merges.push(RangeRef::new(CellRef::new(500_000 + i * 2, 5), CellRef::new(500_000 + i * 2, 6)));
}
sh.merges.push(RangeRef::new(CellRef::new(0, 0), CellRef::new(gridcraft_core::MAX_ROWS - 1, 1)));
sh.set_value(CellRef::new(2, 1), gridcraft_core::Value::Number(2.0));
let html = copied(&wb, "A1:C50");
assert!(html.contains("rowspan=\"50\" colspan=\"2\""), "{html}");
let page = to_html(&wb, 0).unwrap();
// The table is A1:B3 (the used range): the whole-height merge spans its three rows only.
assert!(page.contains("rowspan=\"3\" colspan=\"2\""), "{page}");
// Merges stacked over one another past a few per cell are refused, not walked.
let sh = wb.sheet_mut(0).unwrap();
sh.merges = vec![RangeRef::new(CellRef::new(0, 0), CellRef::new(2, 1)); 100];
assert!(to_html(&wb, 0).is_err());
assert!(range_html(&wb, 0, RangeRef::parse("A1:B3").unwrap(), "").is_none());
}
/// Thousands of pictures over thousands of custom rows: each is placed through a row index.
#[test]
fn html_export_places_many_pictures_over_many_custom_rows() {
let mut wb = book_with_picture("A1", (0, 0));
let sh = wb.sheet_mut(0).unwrap();
for row in 0..50_000 {
sh.rows.insert(row, gridcraft_model::LineInfo { size: Some(30.0), ..Default::default() });
}
let image = sh.images[0].clone();
for i in 0..5_000u32 {
let mut im = image.clone();
im.anchor.cell = CellRef::new(40_000 + i, 0);
sh.images.push(im);
}
let page = to_html(&wb, 0).unwrap();
assert!(page.contains(&format!("top:{}px", 30 * 44_999)), "last picture placed by the index");
}
/// One long text shown by many cells: the page fails at its byte limit instead of growing on.
#[test]
fn html_export_refuses_a_page_past_its_byte_limit() {
let mut wb = Workbook::new();
let sh = wb.sheet_mut(0).unwrap();
let text = gridcraft_core::Value::text("x".repeat(32_000));
for row in 0..10_000 {
sh.set_value(CellRef::new(row, 0), text.clone());
}
let err = to_html(&wb, 0).unwrap_err();
assert!(err.to_string().contains("MiB"), "{err}");
}
#[test]
fn escaping_stops_at_the_room_left() {
assert_eq!(esc_within("a&b<c>\"d", 100).as_deref(), Some(esc("a&b<c>\"d").as_str()));
assert_eq!(esc_within("&&&", 14), None, "three `&amp;` are 15 bytes");
assert_eq!(esc_within("&&&", 15).as_deref(), Some("&amp;&amp;&amp;"));
}
#[test]
fn the_merge_index_holds_a_bounded_number_of_cells() {
let mut sh = Sheet::new("S");
let rect = RangeRef::new(CellRef::new(0, 0), CellRef::new(1_100_000, 0));
sh.merges.push(RangeRef::new(CellRef::new(0, 0), CellRef::new(999_999, 0)));
assert_eq!(merge_index(&sh, rect).map(|m| m.len()), Some(1_000_000));
sh.merges = vec![RangeRef::new(CellRef::new(0, 0), CellRef::new(1_000_000, 0))];
assert!(merge_index(&sh, rect).is_none());
}
+12 -3
View File
@@ -1281,9 +1281,18 @@ fn content_differs(b: Option<&Cell>, a: Option<&Cell>) -> bool {
/// Objects added, changed and removed between two versions of a sheet.
fn objects(b: &Sheet, a: &Sheet) -> (usize, usize, usize) {
fn count<T: PartialEq>(b: &[T], a: &[T], id: impl Fn(&T) -> u32) -> (usize, usize, usize) {
let added = a.iter().filter(|x| !b.iter().any(|y| id(y) == id(x))).count();
let removed = b.iter().filter(|x| !a.iter().any(|y| id(y) == id(x))).count();
let changed = b.iter().filter(|x| a.iter().any(|y| id(y) == id(x) && y != *x)).count();
// Objects grouped by id, so each lookup is a map probe instead of a scan of the list.
fn group<'a, T>(v: &'a [T], id: &impl Fn(&T) -> u32) -> std::collections::HashMap<u32, Vec<&'a T>> {
let mut m: std::collections::HashMap<u32, Vec<&T>> = std::collections::HashMap::new();
for x in v {
m.entry(id(x)).or_default().push(x);
}
m
}
let (gb, ga) = (group(b, &id), group(a, &id));
let added = a.iter().filter(|x| !gb.contains_key(&id(x))).count();
let removed = b.iter().filter(|x| !ga.contains_key(&id(x))).count();
let changed = b.iter().filter(|x| ga.get(&id(x)).is_some_and(|ys| ys.iter().any(|y| *y != *x))).count();
(added, changed, removed)
}
let parts = [
+15
View File
@@ -153,6 +153,11 @@ pub fn compile(formula: &str, field: &dyn Fn(&str) -> Option<String>) -> Result<
Expr::Name(n) if n.starts_with(PREFIX) => {}
Expr::Name(n) => problem = Some(format!("There's no field named '{n}'.")),
Expr::Ref(_) | Expr::Struct(_) => problem = Some("A calculated field can't refer to cells or tables; use field names.".into()),
// Evaluated once per group and per report, with only the first value kept: Excel does not
// allow functions that need arrays in a calculated field, and these build one from scalars.
Expr::Call(f, _) if matches!(f.as_str(), "SEQUENCE" | "RANDARRAY" | "MAKEARRAY" | "EXPAND" | "MUNIT") => {
problem = Some(format!("A calculated field can't use {f}: it builds an array."))
}
_ => {}
}
});
@@ -332,6 +337,16 @@ mod tests {
assert!(compile(&format!("={}", "1+".repeat(MAX_FORMULA_LEN)), &fields).is_err());
}
#[test]
fn a_calculated_field_cannot_build_arrays_from_scalars() {
for f in
["=SEQUENCE(5000,1000)", "=SUM(RANDARRAY(1000,1000))", "=MAKEARRAY(9,9,LAMBDA(r,c,r*c))+Sales", "=expand(Sales,1000,1000)", "=MUNIT(500)"]
{
assert!(compile(f, &fields).unwrap_err().contains("builds an array"), "{f}");
}
assert!(compile("=SUM({1,2,3})*Sales", &fields).is_ok(), "an array constant is bounded by the formula's own length");
}
#[test]
fn deep_and_cyclic_chains_are_bounded() {
// 300 calculated fields, each using the previous one, and a two-field cycle.
+26
View File
@@ -6480,6 +6480,19 @@ fn clearing_links_and_notes_keeps_allow_edit_ranges() {
assert!(sh.hyperlinks.is_empty() && sh.comments.is_empty());
}
#[test]
fn replace_does_not_search_hidden_formula_source() {
let mut s = s();
s.execute("cell.set", json!({"cell": "A1", "input": "=SEQUENCE(3)*7"})).unwrap();
s.execute("home.formatCells", json!({"range": "A1", "style": {"protection": {"locked": true, "hidden": true}}})).unwrap();
s.execute("review.protectSheet", json!({})).unwrap();
// Both give the same answer, and it is not an error: a hidden formula's source is not searched.
let hit = s.execute("edit.replace", json!({"what": "SEQUENCE", "with": "X"})).unwrap();
let miss = s.execute("edit.replace", json!({"what": "NOSUCHTEXT", "with": "X"})).unwrap();
assert_eq!(hit, miss);
assert_eq!(hit["replaced"], json!(0), "{hit}");
}
#[test]
fn formulatext_does_not_reveal_a_hidden_formula() {
let mut s = s();
@@ -6556,6 +6569,19 @@ fn pivot_new_sheet_destination_respects_workbook_structure_protection() {
assert!(s.execute("insert.pivotTable", json!({"source": "A1:B3", "location": "New"})).is_err());
}
#[test]
fn circle_invalid_says_when_its_scan_allowance_ended_early() {
let mut s = s();
s.execute("range.setValues", json!({"range": "A1", "values": [[-1]]})).unwrap();
s.execute("range.setValues", json!({"range": "A200000", "values": [[-1]]})).unwrap();
s.execute("data.validation", json!({"range": "A1:A5", "type": "whole", "operator": "greater", "formula1": "0"})).unwrap();
let out = s.execute("data.circleInvalid", json!({})).unwrap();
assert_eq!((out["invalid"].as_array().map(Vec::len), out["truncated"].as_bool()), (Some(1), Some(false)), "{out}");
s.execute("data.validation", json!({"range": "A1:A200000", "type": "whole", "operator": "greater", "formula1": "0"})).unwrap();
let out = s.execute("data.circleInvalid", json!({})).unwrap();
assert_eq!(out["truncated"].as_bool(), Some(true), "{out}");
}
#[test]
fn history_charges_a_changed_sheets_comment_text() {
let mut s = s();
+24
View File
@@ -69,11 +69,35 @@ fn parse_top(src: &str, locale: crate::FormulaLocale, allow_union: bool) -> Resu
let mut p = Parser { toks, i: 0, depth: 0, input: locale == crate::FormulaLocale::Es };
let e = p.expr(0, allow_union)?;
match p.peek() {
Tok::Eof if wrapper_height(&e) > MAX_DEPTH => Err(ParseError { msg: "formula is nested too deeply".into(), pos: 0 }),
Tok::Eof => Ok(e),
t => Err(p.err(format!("unexpected {}", describe(t)))),
}
}
/// The most unary operators, parentheses and invocations on one path of `e`. The parser bounds
/// each level's postfix wrappers by its own depth, so levels inside levels can still stack them
/// far past [`MAX_DEPTH`]; code that maps over the tree recurses once per wrapper.
fn wrapper_height(e: &Expr) -> usize {
let mut most = 0;
let mut pending = vec![(e, 0usize)];
while let Some((e, h)) = pending.pop() {
most = most.max(h);
match e {
Expr::Unary(_, a) | Expr::Paren(a) => pending.push((a, h + 1)),
Expr::Invoke(callee, args) => {
pending.push((callee, h + 1));
pending.extend(args.iter().map(|a| (a, h + 1)));
}
Expr::Binary(_, a, b) => pending.extend([(&**a, h), (&**b, h)]),
Expr::Call(_, args) => pending.extend(args.iter().map(|a| (a, h))),
Expr::Array(rows) => pending.extend(rows.iter().flatten().map(|a| (a, h))),
_ => {}
}
}
most
}
fn describe(t: &Tok) -> String {
match t {
Tok::RParen => "`)`".into(),
+16
View File
@@ -576,3 +576,19 @@ fn column_starting_with_at_round_trips() {
assert_eq!(parse(&text).ok(), Some(Expr::Struct(s)), "printed as {text}");
}
}
#[test]
fn wrappers_stacked_across_levels_are_bounded_as_a_whole() {
// Each level adds 40 percents inside the parser's per-level bound, so no level is too deep alone.
let stacked = |levels: usize| {
let mut f = "1".to_string();
for _ in 0..levels {
f = format!("({f}){}", "%".repeat(40));
}
f
};
assert!(parse(&stacked(5)).is_ok());
assert!(parse(&stacked(7)).is_err(), "7 x 41 wrappers on one path");
assert!(parse(&format!("{}1{}", "(".repeat(200), ")".repeat(200))).is_ok());
assert!(parse(&format!("={}", vec!["A1%"; 100].join("+"))).is_ok(), "siblings are not stacked");
}
+1
View File
@@ -13,6 +13,7 @@ mod day_count;
mod depreciation;
mod securities;
mod tvm;
pub(crate) use tvm::cumulative_args;
use day_count::{coupdaybs, coupdays, coupdaysnc, coupncd, coupnum, couppcd};
use depreciation::{amordegrc, amorlinc, db, ddb, sln, syd, vdb};
+8 -1
View File
@@ -116,7 +116,9 @@ pub(super) fn ppmt(a: &[Arg], _c: &mut dyn Ctx) -> R<Value> {
num_val(pmt_calc(rate, np, pv, fv, typ) - ipmt_calc(rate, per, np, pv, fv, typ))
}
fn cumulative(a: &[Arg], principal: bool) -> R<Value> {
/// The validated (rate, periods, pv, start, end, type) of CUMIPMT/CUMPRINC: every check that
/// refuses a call before its period loop, shared with the lifted-call work preflight.
pub(crate) fn cumulative_args(a: &[Arg]) -> R<(f64, f64, f64, f64, f64, f64)> {
let (rate, np, pv) = (num(a, 0)?, num(a, 1)?, num(a, 2)?);
let (start, end, typ) = (num(a, 3)?.trunc(), num(a, 4)?.trunc(), num(a, 5)?);
let np_i = np.trunc();
@@ -126,6 +128,11 @@ fn cumulative(a: &[Arg], principal: bool) -> R<Value> {
if end - start > 1e7 || np_i > 1e15 {
return Err(CellError::Num);
}
Ok((rate, np_i, pv, start, end, typ))
}
fn cumulative(a: &[Arg], principal: bool) -> R<Value> {
let (rate, np_i, pv, start, end, typ) = cumulative_args(a)?;
let p = pmt_calc(rate, np_i, pv, 0.0, typ);
let mut sum = 0.0;
let mut per = start;
+195 -26
View File
@@ -63,26 +63,86 @@ const SCANS: &[&str] = &["VLOOKUP", "HLOOKUP", "LOOKUP", "MATCH", "XLOOKUP", "XM
/// `XMATCH(SEQUENCE(1048576), -SEQUENCE(1048576))` runs for hours and hangs opening the file.
/// Past the budget the call is `#NUM!`, as MMULT is past 2e9 multiply-adds. 5e8 keeps the worst
/// case to a few seconds and still admits 5,000 needles over a 100,000-row table.
///
/// The same budget covers all the work of a lifted call added up across its elements: criteria
/// aggregates rescanning their ranges, period loops, and array results built per element.
const LIFTED_SCAN_BUDGET: usize = 500_000_000;
/// The length of the longest vector a lifted call of `spec` searches per needle: 0 unless `spec`
/// is one of the [`SCANS`] lookups and a non-lifted argument is an array.
/// Criteria aggregates: every call tests each cell of its ranges against its criteria, so a call
/// lifted over `n` criteria does `n` x (range cells) work, like the [`SCANS`] lookups.
const CRITERIA: &[&str] = &["COUNTIF", "COUNTIFS", "SUMIF", "SUMIFS", "AVERAGEIF", "AVERAGEIFS", "MAXIFS", "MINIFS"];
/// Cumulative TVM functions that step through every period from `start_period` (argument 3) to
/// `end_period` (argument 4) on each call.
const PERIOD_LOOPS: &[&str] = &["CUMIPMT", "CUMPRINC"];
/// Budget units one period of a [`PERIOD_LOOPS`] call costs: each period computes a payment and a
/// balance with powers, many times a scanned cell.
const PERIOD_COST: usize = 16;
/// The work every lifted call of `spec` repeats over its non-lifted arguments: the longest vector
/// a [`SCANS`] lookup searches, or every cell a [`CRITERIA`] aggregate tests. 0 for other
/// functions.
fn scan_len(spec: &FnSpec, args: &[Arg]) -> usize {
if !SCANS.contains(&spec.name) {
let scans = SCANS.contains(&spec.name);
if !scans && !CRITERIA.contains(&spec.name) {
return 0;
}
args.iter()
.enumerate()
.filter(|(i, _)| !spec.is_scalar_arg(*i))
.filter_map(|(_, a)| match &a.value {
Value::Array(arr) => Some(arr.rows.max(arr.cols)),
_ => None,
})
.max()
.unwrap_or(0)
// SUMIF and AVERAGEIF read the optional third range only at the coordinates of the first.
let first = args.first().and_then(|a| if let Value::Array(arr) = &a.value { Some(arr.data.len()) } else { None });
let lens = args.iter().enumerate().filter(|(i, _)| !spec.is_scalar_arg(*i)).filter_map(|(i, a)| match &a.value {
Value::Array(arr) if scans => Some(arr.rows.max(arr.cols)),
Value::Array(arr) if i == 2 && matches!(spec.name, "SUMIF" | "AVERAGEIF") => Some(arr.data.len().min(first.unwrap_or(usize::MAX))),
Value::Array(arr) => Some(arr.data.len()),
_ => None,
});
if scans { lens.max().unwrap_or(0) } else { lens.fold(0, usize::saturating_add) }
}
/// The work of one lifted call that depends on its own (lifted) scalar arguments: the number of
/// periods a [`PERIOD_LOOPS`] function steps through. 0 for other functions.
fn call_cost(spec: &FnSpec, cell_args: &[Arg]) -> usize {
if !PERIOD_LOOPS.contains(&spec.name) {
return 0;
}
// Coercing numeric text reads all of it, even in a call the function then refuses, so each
// character is charged as well: of the arguments it coerces in order, up to the first that fails.
let mut parse = 0usize;
for (i, a) in cell_args.iter().enumerate().take(6) {
if let Value::Text(t) = &a.value {
parse = parse.saturating_add(t.len());
}
if crate::util::num(cell_args, i).is_err() {
break;
}
}
// The function's own validation and coercion (text like "10000000" counts), so a call it
// refuses before its loop costs nothing and no loop goes uncharged.
let periods = crate::financial::cumulative_args(cell_args).map_or(0, |(.., start, end, _)| (end - start + 1.0) as usize);
parse.saturating_add(periods.saturating_mul(PERIOD_COST))
}
pub(crate) fn call_lifted(spec: &FnSpec, args: &[Arg], ctx: &mut dyn Ctx) -> Value {
call_lifted_within(spec, args, ctx, LIFTED_SCAN_BUDGET)
}
/// Sets the lifted arguments of `cell_args` to their elements at (`r`, `c`).
fn element_args(spec: &FnSpec, args: &[Arg], r: usize, c: usize, cell_args: &mut [Arg]) {
for (i, a) in args.iter().enumerate() {
if spec.is_scalar_arg(i)
&& let Value::Array(arr) = &a.value
&& let Some(slot) = cell_args.get_mut(i)
{
let v = arr.get_broadcast(r, c);
slot.value = if let Value::Array(inner) = &v { inner.data.first().cloned().unwrap_or(Value::Empty) } else { v };
// A blank element of a written array argument is given, as a scalar blank is.
slot.blank_value = !slot.from_ref && matches!(slot.value, Value::Empty);
}
}
}
/// [`call_lifted`] with every call's work added up against `budget` (see [`LIFTED_SCAN_BUDGET`]).
fn call_lifted_within(spec: &FnSpec, args: &[Arg], ctx: &mut dyn Ctx, budget: usize) -> Value {
let rejected = if spec.value_args.is_empty() { None } else { reject_multi_cell_refs(spec, args) };
let args = rejected.as_deref().unwrap_or(args);
let mut rows = 0usize;
@@ -101,27 +161,42 @@ pub(crate) fn call_lifted(spec: &FnSpec, args: &[Arg], ctx: &mut dyn Ctx) -> Val
return (spec.imp)(args, ctx);
}
let needles = rows.saturating_mul(cols);
if needles > MAX_CELLS || needles.saturating_mul(scan_len(spec, args)) > LIFTED_SCAN_BUDGET {
// Every call is charged its repeated scan (at least 1), its own period loop, and the cells of
// an array result (built in full, though only its first cell is kept), so lifting a scalar
// argument can't multiply what one call may do into hours of work:
// `COUNTIF(big_range, SEQUENCE(1e4))`, `CUMIPMT(0.1, 1e7, 1, 1, SEQUENCE(1e4,,1e7,0), 0)`,
// `MUNIT(SEQUENCE(1e4,,3000,0))`.
let per_call = scan_len(spec, args).max(1);
let mut cell_args: Vec<Arg> = args.to_vec();
let mut spent = needles.saturating_mul(per_call);
if needles > MAX_CELLS || spent > budget {
return Value::Error(CellError::Num);
}
let mut cell_args: Vec<Arg> = args.to_vec();
// Period loops are known before any call runs, so a call over budget does none of them.
if PERIOD_LOOPS.contains(&spec.name) {
for r in 0..rows {
for c in 0..cols {
element_args(spec, args, r, c, &mut cell_args);
spent = spent.saturating_add(call_cost(spec, &cell_args));
if spent > budget {
return Value::Error(CellError::Num);
}
}
}
}
let mut data = Vec::with_capacity(rows * cols);
for r in 0..rows {
for c in 0..cols {
for (i, a) in args.iter().enumerate() {
if spec.is_scalar_arg(i)
&& let Value::Array(arr) = &a.value
&& let Some(slot) = cell_args.get_mut(i)
{
let v = arr.get_broadcast(r, c);
slot.value = if let Value::Array(inner) = &v { inner.data.first().cloned().unwrap_or(Value::Empty) } else { v };
// A blank element of a written array argument is given, as a scalar blank is.
slot.blank_value = !slot.from_ref && matches!(slot.value, Value::Empty);
}
}
element_args(spec, args, r, c, &mut cell_args);
let out = (spec.imp)(&cell_args, ctx);
data.push(match out {
Value::Array(a) => a.data.first().cloned().unwrap_or(Value::Empty),
Value::Array(a) => {
spent = spent.saturating_add(a.data.len());
if spent > budget {
return Value::Error(CellError::Num);
}
a.data.first().cloned().unwrap_or(Value::Empty)
}
v => v,
});
}
@@ -206,6 +281,100 @@ mod tests {
assert_eq!(rows_of(&out), vec![vec![nv(-1.0)]; 1000]);
}
/// Security: lifting a criteria aggregate rescans its range for every criterion, so criteria
/// x range cells past the budget is `#NUM!` before any scan runs.
#[test]
fn lifted_criteria_aggregates_stop_past_their_work_budget() {
use gridcraft_core::Array;
let tall = |n: usize| av(Array::filled(n, 1, nv(-1.0)).into());
is_err(ev("COUNTIF", vec![tall(1_000_000), tall(501)]), CellError::Num);
is_err(ev("SUMIF", vec![tall(1_000_000), tall(501)]), CellError::Num);
is_err(ev("COUNTIFS", vec![tall(1_000_000), tall(501)]), CellError::Num);
// Two ranges of 250,000 cells each count together.
is_err(ev("SUMIFS", vec![tall(250_000), tall(250_000), tall(1001)]), CellError::Num);
// SUMIF reads its sum range only where the criteria range has cells: a one-cell range over a
// million-cell sum range is one cell of work per criterion.
let one = av(Array::filled(1, 1, nv(-1.0)).into());
assert_eq!(rows_of(&ev("SUMIF", vec![one, tall(501), tall(1_000_000)])).len(), 501);
let counts = ev("COUNTIF", vec![av(col(&[1.0, 2.0, 2.0])), av(col(&[2.0, 1.0, 5.0]))]);
assert_eq!(rows_of(&counts), vec![vec![nv(2.0)], vec![nv(1.0)], vec![nv(0.0)]]);
for name in super::CRITERIA.iter().chain(super::PERIOD_LOOPS) {
assert!(crate::lookup(name).is_some(), "{name} isn't registered");
}
}
/// Security: CUMIPMT and CUMPRINC loop over their periods once per lifted element; the
/// periods of all elements are added up first, so an over-budget call runs none of them.
#[test]
fn lifted_period_loops_stop_past_their_work_budget() {
use gridcraft_core::Array;
let ends = |len: usize, end: f64| av(Array::filled(len, 1, nv(end)).into());
let args = |len: usize, end: f64| vec![n(0.01), n(1e7), n(1000.0), n(1.0), ends(len, end), n(0.0)];
// 51 elements of 1e7 periods each is past 5e8.
is_err(ev("CUMIPMT", args(51, 1e7)), CellError::Num);
is_err(ev("CUMPRINC", args(51, 1e7)), CellError::Num);
// A period costs more than a scanned cell, so fewer elements are enough.
is_err(ev("CUMIPMT", args(49, 1e7)), CellError::Num);
// Numeric text is coerced by the function, so its periods are charged the same.
let text_ends = av(Array::filled(51, 1, Value::text("10000000")).into());
is_err(ev("CUMIPMT", vec![n(0.01), n(1e7), n(1000.0), n(1.0), text_ends, n(0.0)]), CellError::Num);
let out = ev("CUMIPMT", args(2, 12.0));
assert_eq!(rows_of(&out).len(), 2);
assert!(matches!(rows_of(&out).first().and_then(|r| r.first()), Some(Value::Number(x)) if *x < 0.0));
}
/// Security: numeric text is parsed once per lifted element even when the call is refused; the
/// characters read are charged, so a long text repeated down a column can't cost gigabytes of parsing.
#[test]
fn lifted_period_loops_charge_the_text_they_parse() {
use gridcraft_core::Array;
// 20,000 elements x 32,000 characters is past 5e8 though no call reaches its loop (rate is negative).
let long = Value::text(format!("{}5", "0".repeat(32_000)));
let ends = av(Array::filled(20_000, 1, long).into());
is_err(ev("CUMIPMT", vec![n(-0.01), n(10.0), n(1000.0), n(1.0), ends, n(0.0)]), CellError::Num);
// A short text period costs next to nothing.
let ends = av(Array::filled(3, 1, Value::text("12")).into());
assert_eq!(rows_of(&ev("CUMIPMT", vec![n(0.01), n(20.0), n(1000.0), n(1.0), ends, n(0.0)])).len(), 3);
}
/// Text after the first argument that fails to coerce is never read, so it is not charged: the
/// elements keep their own errors instead of one budget `#NUM!`.
#[test]
fn lifted_period_loops_charge_only_the_text_they_reach() {
use gridcraft_core::Array;
let rates = av(Array::filled(20_000, 1, Value::text("x")).into());
let long = Value::text(format!("{}5", "0".repeat(32_000)));
let out = ev("CUMIPMT", vec![rates, n(10.0), n(1000.0), n(1.0), av(long), n(0.0)]);
assert!(rows_of(&out).iter().all(|r| r.first() == Some(&Value::Error(CellError::Value))));
}
/// A call the function refuses before its loop (here: end period past the period count) is
/// not charged, so rejected elements can't push a valid one over the budget.
#[test]
fn lifted_period_loops_do_not_charge_rejected_calls() {
use gridcraft_core::Array;
let mut ends = vec![nv(1e7 + 1.0); 60];
ends.push(nv(12.0));
let ends = av(Array::new(ends.len(), 1, ends).unwrap_or_else(|| panic!("shape")).into());
let out = ev("CUMIPMT", vec![n(0.01), n(1e7), n(1000.0), n(1.0), ends, n(0.0)]);
let rows = rows_of(&out);
assert_eq!(rows.len(), 61);
assert!(matches!(rows.last().and_then(|r| r.first()), Some(Value::Number(x)) if *x < 0.0));
}
/// Security: an array-producing function lifted over a computed array builds each element's
/// whole result before keeping one cell; their cells count against the budget.
#[test]
fn lifted_array_results_count_against_the_budget() {
use gridcraft_core::Array;
let spec = crate::lookup("MUNIT").unwrap_or_else(|| panic!("MUNIT isn't registered"));
let dims = |n: usize| [av(Array::filled(n, 1, nv(10.0)).into())];
// Ten 10x10 identity matrices are 1,000 cells.
let out = super::call_lifted_within(spec, &dims(10), &mut TestCtx::default(), 1_010);
assert_eq!(rows_of(&out), vec![vec![nv(1.0)]; 10]);
is_err(super::call_lifted_within(spec, &dims(11), &mut TestCtx::default(), 1_010), CellError::Num);
}
#[test]
fn value_params_reject_multi_cell_references() {
let s = FnSpec { value_args: &[true, false], ..spec() };
+9
View File
@@ -53,6 +53,15 @@ impl Remote {
self.conn = None;
let mut last = std::io::Error::new(std::io::ErrorKind::NotFound, format!("cannot resolve {}", self.addr));
for sa in self.addr.to_socket_addrs()? {
// The token goes over plain TCP, so only to this machine: a network peer could read it,
// and the app's control port listens on loopback only.
if !sa.ip().is_loopback() {
last = std::io::Error::new(
std::io::ErrorKind::PermissionDenied,
format!("{sa} is not a loopback address: the control token is sent only to 127.0.0.1 or ::1"),
);
continue;
}
match TcpStream::connect_timeout(&sa, Duration::from_secs(2)) {
Ok(s) => {
s.set_nodelay(true).ok();
+3
View File
@@ -482,6 +482,9 @@ fn remote_backend_over_tcp() {
drop(s);
handle.join().unwrap();
assert!(Remote::connect_with_token("127.0.0.1:1", "x".into()).is_err());
// A network address is refused before anything, the token included, is sent.
let err = Remote::connect_with_token("192.0.2.1:7979", "x".into()).err().unwrap();
assert_eq!(err.kind(), std::io::ErrorKind::PermissionDenied, "{err}");
}
#[test]
+1 -1
View File
@@ -11,7 +11,7 @@ workspace = true
[dependencies]
# The shared capture slot (`craft_ui::capture`) comes from the one craft-libs revision every
# craft-libs dependency uses (see the root Cargo.toml).
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "1b34eba41e62138fc2438d2734523b302840b45d" }
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "6e03f65bf45181db77ba6229716e6624db9eb25b" }
gridcraft-engine = { workspace = true }
gridcraft-chart = { workspace = true }
# `monochrome_emoji_fonts` restores the Noto Emoji and emoji-icon faces that 0.37 moved behind
+16 -1
View File
@@ -1,5 +1,5 @@
use super::{
INTERFACE_SCALE_MAX, INTERFACE_SCALE_MIN, Json, NoticeKind, SheetApp, UiRequest, backstage, clamp_interface_scale, clean_error, dialogs,
INTERFACE_SCALE_MAX, INTERFACE_SCALE_MIN, Json, Notice, NoticeKind, SheetApp, UiRequest, backstage, clamp_interface_scale, clean_error, dialogs,
formula_locale, i18n, json, now_ms, printing,
};
@@ -96,6 +96,21 @@ impl SheetApp {
{
self.deliver_pdf(r);
}
// The scan stopped at its cell allowance: say that some cells were not checked.
if id == "data.circleInvalid"
&& let Ok(r) = &r
&& r.get("truncated").and_then(Json::as_bool) == Some(true)
{
let scan = "The scan stopped after 100,000 cell visits; some validated cells were not checked for invalid data.";
// An open-time warning that is still shown keeps its text and action; the scan note joins it.
match self.notice.as_mut() {
Some(n) => {
n.text = format!("{} {scan}", n.text);
n.kind = NoticeKind::Warning;
}
None => self.notice = Some(Notice { text: scan.into(), kind: NoticeKind::Warning, action: None }),
}
}
r
}
+22
View File
@@ -1396,6 +1396,28 @@ fn macro_workbook_bytes() -> Vec<u8> {
gridcraft_engine::io::save_bytes(&wb, "macros.xlsm").unwrap()
}
#[test]
fn circle_invalid_data_warns_when_its_scan_was_cut_short() {
let mut h = harness(blank());
h.state_mut().run("range.setValues", json!({"range": "A1", "values": [[-1]]})).unwrap();
h.state_mut().run("range.setValues", json!({"range": "A200000", "values": [[-1]]})).unwrap();
h.state_mut().run("data.validation", json!({"range": "A1:A200000", "type": "whole", "operator": "greater", "formula1": "0"})).unwrap();
h.state_mut().run("data.circleInvalid", json!({})).unwrap();
let n = h.state().notice.clone().expect("a notice was shown");
assert!(n.text.contains("100,000 cell visits"), "{}", n.text);
// An open-time warning keeps its action when the scan note joins it.
h.state_mut().notice = Some(gridcraft_ui_egui::Notice {
text: "Macros.".into(),
kind: gridcraft_ui_egui::NoticeKind::Info,
action: Some(("More".into(), "macros".into())),
});
h.state_mut().run("data.circleInvalid", json!({})).unwrap();
let n = h.state().notice.clone().unwrap();
assert!(n.text.starts_with("Macros. ") && n.text.contains("cell visits"), "{}", n.text);
assert_eq!(n.action.as_ref().map(|a| a.1.as_str()), Some("macros"));
assert_eq!(n.kind, gridcraft_ui_egui::NoticeKind::Warning);
}
#[test]
fn opening_a_macro_workbook_shows_a_notice() {
// The reader's "macros kept but not executed" warning must reach a dismissible info bar,
+4
View File
@@ -1,2 +1,6 @@
sonar.projectKey=storytold_gridcraft
sonar.organization=storytold
# The region table is one record per locale with the same field names and many shared values
# (month names, formats); that repetition is data, not copy-pasted logic.
sonar.cpd.exclusions=crates/core/src/region.rs
+69 -5
View File
@@ -13,7 +13,10 @@ struct Counts {
proptests: usize,
}
fn walk(dir: &Path, c: &mut Counts, seen: &mut std::collections::HashSet<u64>, budget: &mut usize) {
/// Larger files (or devices behind a symlink) are not source files and are not read.
const MAX_SOURCE_BYTES: u64 = 16 << 20;
fn walk(dir: &Path, c: &mut Counts, seen: &mut std::collections::HashSet<(u64, u64)>, budget: &mut usize) {
if *budget == 0 {
// Hard stop for pathological trees: two directory symlinks pointing at each other
// would otherwise be walked exponentially.
@@ -22,9 +25,9 @@ fn walk(dir: &Path, c: &mut Counts, seen: &mut std::collections::HashSet<u64>, b
// Directory identities (dev+inode on unix) make a symlink to a visited directory a
// no-op, not another exponential branch of the scan.
#[cfg(unix)]
let id = std::fs::metadata(dir).ok().and_then(|m| m.dev().checked_mul(1 << 60).and_then(|d| d.checked_add(m.ino())));
let id = std::fs::metadata(dir).ok().map(|m| (m.dev(), m.ino()));
#[cfg(not(unix))]
let id: Option<u64> = None;
let id: Option<(u64, u64)> = None;
if let Some(id) = id
&& !seen.insert(id)
{
@@ -45,7 +48,9 @@ fn walk(dir: &Path, c: &mut Counts, seen: &mut std::collections::HashSet<u64>, b
walk(&p, c, seen, budget);
}
} else if name.ends_with(".rs")
&& let Ok(s) = std::fs::read_to_string(&p)
// The entry's own metadata does not follow a symlink: only regular files are sources.
&& e.metadata().is_ok_and(|m| m.is_file() && m.len() <= MAX_SOURCE_BYTES)
&& let Some(s) = read_source(&p)
{
c.files += 1;
c.lines += s.lines().count();
@@ -56,6 +61,15 @@ fn walk(dir: &Path, c: &mut Counts, seen: &mut std::collections::HashSet<u64>, b
}
}
/// A source file's text, read up to [`MAX_SOURCE_BYTES`]: one that grew or was replaced since its
/// size was checked is skipped, not read whole.
fn read_source(p: &Path) -> Option<String> {
use std::io::Read as _;
let mut s = String::new();
std::fs::File::open(p).ok()?.take(MAX_SOURCE_BYTES + 1).read_to_string(&mut s).ok()?;
(s.len() as u64 <= MAX_SOURCE_BYTES).then_some(s)
}
/// `#[test]` attributes and `proptest!` blocks (each block may hold many
/// property tests, and macro-generated tests are only counted once — use
/// `--exact` for harness-accurate numbers).
@@ -129,11 +143,61 @@ pub fn run(_root: &Path, exact: bool) -> Result<(), String> {
#[cfg(test)]
mod tests {
use super::count_tests;
use super::{Counts, MAX_SOURCE_BYTES, count_tests, read_source, walk};
/// A new, private directory: `create_dir` fails if the name exists (even as a link), so a
/// directory somebody else made in advance is never used.
fn private_dir(tag: &str) -> std::path::PathBuf {
let mut n = 0u32;
loop {
let dir = std::env::temp_dir().join(format!("xtask-stats-{tag}-{}-{n}", std::process::id()));
let mut b = std::fs::DirBuilder::new();
#[cfg(unix)]
std::os::unix::fs::DirBuilderExt::mode(&mut b, 0o700);
match b.create(&dir) {
Ok(()) => return dir,
Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => {}
Err(e) => panic!("{e}"),
}
n += 1;
}
}
#[test]
fn counts_attributes_and_proptest_blocks() {
let src = "#[test]\nfn a() {}\n// #[test] commented\n #[test]\nfn b() {}\nproptest! {\n}\n";
assert_eq!(count_tests(src), (2, 1));
}
#[test]
fn a_source_past_the_size_limit_is_not_read() {
let dir = private_dir("size");
let (small, big) = (dir.join("a.rs"), dir.join("b.rs"));
std::fs::write(&small, "fn a() {}\n").unwrap();
let file = std::fs::File::create(&big).unwrap();
file.set_len(MAX_SOURCE_BYTES + 1).unwrap(); // sparse: nothing is written
drop(file);
assert_eq!(read_source(&small).as_deref(), Some("fn a() {}\n"));
assert_eq!(read_source(&big), None);
let _ = std::fs::remove_dir_all(dir);
}
#[cfg(unix)]
#[test]
fn walk_counts_regular_sources_once_and_skips_links_and_big_files() {
let dir = private_dir("walk");
let sub = dir.join("sub");
std::fs::create_dir(&sub).unwrap();
std::fs::write(dir.join("a.rs"), "#[test]\nfn a() {}\n").unwrap();
std::fs::write(sub.join("b.rs"), "fn b() {}\n").unwrap();
// A directory link back to the root is one identity, a file link is not a regular file,
// and a source over the size limit is not read.
std::os::unix::fs::symlink(&dir, sub.join("loop")).unwrap();
std::os::unix::fs::symlink(dir.join("a.rs"), sub.join("link.rs")).unwrap();
std::fs::File::create(dir.join("big.rs")).unwrap().set_len(MAX_SOURCE_BYTES + 1).unwrap();
let mut c = Counts::default();
walk(&dir, &mut c, &mut std::collections::HashSet::new(), &mut 100);
assert_eq!((c.files, c.tests), (2, 1));
let _ = std::fs::remove_dir_all(dir);
}
}