diff --git a/docs/scorecard.md b/docs/scorecard.md index 42ae824b4d..ac55e35596 100644 --- a/docs/scorecard.md +++ b/docs/scorecard.md @@ -6,8 +6,8 @@ Generated by `cargo xtask scorecard` from `perf/budgets.toml`, `perf/baseline.js | Area | Done | Partial | Missing | Key numbers | |---|---:|---:|---:|---| -| [Performance](#performance) | 3 | 22 | 25 | 59 scenarios: 3 within budget, 22 over budget or failing, 9 tracked only, 20 not run, 5 not measurable yet (baseline 2026-10-05) | -| [File compatibility](#file-compatibility) | 7 | 14 | 6 | io corpus: oracle ≥ 148/170, round trip ≥ 169/170; psd-tools corpus: oracle ≥ 249/309, round trip ≥ 308/309; photoshop oracles: oracle ≥ 188/256, round trip ≥ 258/256 | +| [Performance](#performance) | 3 | 22 | 26 | 60 scenarios: 3 within budget, 22 over budget or failing, 9 tracked only, 22 not run, 4 not measurable yet (baseline 2026-10-05) | +| [File compatibility](#file-compatibility) | 8 | 13 | 6 | io corpus: oracle ≥ 148/170, round trip ≥ 169/170; psd-tools corpus: oracle ≥ 249/309, round trip ≥ 308/309; photoshop oracles: oracle ≥ 188/256, round trip ≥ 258/256 | | [Tools](#tools) | 33 | 11 | 0 | | | [Workspace and UI](#workspace-and-ui) | 13 | 6 | 4 | settings that do nothing: 26 of 170 (target 0) | | [Type](#type) | 4 | 3 | 2 | | @@ -43,6 +43,8 @@ Baseline: 2026-10-05, commit `f5d2bcb54a`, Apple M4 Pro (48 GB), GPU Apple M4 Pr | P17 | Move a layer on a 32-bit document with 20 layers | ≤ 100 ms p95 | 386 | 681 | 681 | 1.2 GB | 1.6 GB | over budget (target not met yet) | [#209](https://github.com/storytold/photocraft/issues/209) | | P18 | Open a 20 MP layered PSD: first pixels (today: fully opened) | ≤ 1000 ms p95 | 1753 | 2794 | 2794 | 2.2 GB | 1.9 GB | over budget (target not met yet) | [#210](https://github.com/storytold/photocraft/issues/210) | | P19 | Open a 20 MP layered PSD: editable | ≤ 3000 ms p95 | 1753 | 2794 | 2794 | 2.2 GB | 1.9 GB | pass | [#210](https://github.com/storytold/photocraft/issues/210) | +| P20 | Open a 2 GB PSD: editable | opens with progress, editable, never blank | – | – | – | – | – | not run | [#210](https://github.com/storytold/photocraft/issues/210) | +| P63 | Open a 4–5 GB PSB: editable | opens with progress, editable, never blank | – | – | – | – | – | not run | [#210](https://github.com/storytold/photocraft/issues/210) | | P21 | Content-Aware Fill on 10 MP | ≤ 1500 ms p95 | 218 | 256 | 256 | 1.8 GB | 1.8 GB | pass | [#210](https://github.com/storytold/photocraft/issues/210) | | P22 | Spot Healing stroke on 10 MP | ≤ 500 ms p95 | 157 | 247 | 247 | 1.8 GB | 1.9 GB | pass | [#210](https://github.com/storytold/photocraft/issues/210) | | P24 | Import 500 brushes (longest UI-thread step) | ≤ 100 ms max | – | – | – | – | – | not run | [#210](https://github.com/storytold/photocraft/issues/210) | @@ -84,7 +86,6 @@ Baseline: 2026-10-05, commit `f5d2bcb54a`, Apple M4 Pro (48 GB), GPU Apple M4 Pr | Id | Scenario | Target | Why | Issue | |---|---|---|---|---| | P15 | Zoom / pan frame time on a 15000×10000 16-bit document | ≤ 16 ms per frame | no headless canvas frame harness yet (zoom and pan sample the GPU texture in the egui paint callback) | [#209](https://github.com/storytold/photocraft/issues/209) | -| P20 | Open a 2 GB PSD / 4–5 GB PSB | opens with progress, editable, never blank | opens run in the background with progress (#210), but there are no generated large-file fixtures yet (#216) | [#210](https://github.com/storytold/photocraft/issues/210) | | P23 | Warp / Puppet / Perspective preview at 4K | ≤ 16 ms per frame, commit in the background | the previews run inside the UI tools; no headless frame harness, no background commit | [#210](https://github.com/storytold/photocraft/issues/210) | | P34 | Brush preview memory, small dab on an 8K viewport | ≤ 2 MiB | the live brush preview buffers are UI-side; no headless counter yet | [#211](https://github.com/storytold/photocraft/issues/211) | | P35 | Marquee drag on 24 MP with Alt/Shift | ≤ 17 ms p95 | Shift/Alt marquee modifiers don't exist yet (#208) and the drag preview is UI-side | [#211](https://github.com/storytold/photocraft/issues/211) | @@ -123,7 +124,7 @@ Corpus floors (`crates/io/tests/corpus.rs`): the enforced minimums; the corpus t | FILE-215-20 | Artboard tool with handles and + buttons, Duplicate Artboard beside, vector PDF pages | partial | [#215](https://github.com/storytold/photocraft/issues/215) | artboard_tool.rs: Artboard tool draws, selects, moves and resizes boards by their handles (⇧ keeps proportions), its + buttons add a board or (⌥) a copy beside the selected one (#3033); Duplicate Artboard places the copy beside (#1584); raster PDF pages only | | FILE-215-21 | Local libraries of colours, patterns, images and layers, as linked items or copies | missing | [#215](https://github.com/storytold/photocraft/issues/215) | Only preset libraries (preset_store.rs) | | FILE-216-1 | Every corpus fetched and enforced in CI, cached by pin | done | [#216](https://github.com/storytold/photocraft/issues/216) | ci.yml has a corpus job (name: corpus tests) on every PR and on main: the corpora are pinned in xtask/src/corpus_pins.rs, sha256-verified after both a cache restore and a download, and a missing corpus fails the tests. The cache key follows the pins, so it is cached by pin as this row asks (#190) | -| FILE-216-2 | Large-file fixtures: 20 MP, 150-layer, generated 2 GB PSD and 4-5 GB PSB stress tests | partial | [#216](https://github.com/storytold/photocraft/issues/216) | 4-5 GB PSB fixtures: psd example big_psb writes them (raw and RLE, planes past 2 GiB), io example big_psb_check opens and checks them; opt-in io tests/large_psb.rs (2.5 GB merged image). No 2 GB PSD, 150-layer or 20 MP fixture, nothing in CI | +| FILE-216-2 | Large-file fixtures: 20 MP, 150-layer, generated 2 GB PSD and 4-5 GB PSB stress tests | done | [#216](https://github.com/storytold/photocraft/issues/216) | Generated at test time by psd fixtures.rs (feature testgen; never committed). io tests/large_files.rs: layered_20_mp_psd_opens_and_saves (5472x3648, 5 layers in 2 groups, mask, drop shadow) and layout_with_150_layers_opens_and_saves (A4 300 ppi, 150 layers in 15 groups, masks, hidden layers, closed groups, drop shadows) run in every cargo test (~2 s each); opt-in psd_just_under_2_gb_opens_and_saves (17,000 px, 2.02 GB PSD) and psb_of_4_gb_opens_and_saves (25,000 px, 4.38 GB PSB, data past 4 GiB), each opened, pixel-checked, saved and re-opened; plus large_psb.rs (2.5 GB merged image). Open times: ui-egui large_file_bench (P20, P63). The 2 GB and 4 GB files are not run in CI (6.4 GB on disk; ~12 GB of memory for the PSD test, ~25 GB for the PSB test) (#5107) | | FILE-216-3 | TIFF, RAW and EXR corpora read by tests; per-camera RAW coverage reported | partial | [#216](https://github.com/storytold/photocraft/issues/216) | EXR: corpus/exr (the OpenEXR deep test images, pinned) is read by codecs tests/deep_exr.rs in the CI corpus job, against the ID manifests of the upstream sidecars. corpus/tiff and corpus/raw are still copied in by hand and no test reads them; no per-camera RAW report | | FILE-216-4 | Exports re-open in a third-party oracle editor | missing | [#216](https://github.com/storytold/photocraft/issues/216) | The round trip is our export -> our import | | FILE-216-5 | Save-path tests: layered TIFF, PDF, SVG, ACO/ASE, .shc, Smart Filters, atomic-save failure | partial | [#216](https://github.com/storytold/photocraft/issues/216) | Every save path that exists is tested by saving and reading back: layered TIFF io tests/tiff_layers.rs (every_compression_keeps_the_layers, advanced_blending_survives_layered_tiff_in_both_byte_orders), print PDF engine print_cmds/tests.rs (print_dry_run_renders_a_pdf_and_reports_the_platform_backend decodes the image back), ACO/ASE engine swatch_cmds/tests.rs (aco_round_trip_keeps_16_bit_values_and_books, ase_round_trip_keeps_groups_floats_and_kinds), .shc psd shc.rs shc_round_trips_and_rejects_hostile_input and engine layer_style.rs contour_presets_and_custom_curves_apply_in_one_undo_step, Smart Filters engine tests/smart_psd.rs (smart_filters_survive_psd_in_every_colour_model), atomic-save failure format/src/atomic/tests.rs (BUG-203-6). There is no SVG writer (SVG opens only) and no Photoshop PDF save (FILE-215-10), so those have nothing to test yet; the row stays partial until they exist | diff --git a/perf/budgets.toml b/perf/budgets.toml index 17c6ed3625..0b068ede0f 100644 --- a/perf/budgets.toml +++ b/perf/budgets.toml @@ -64,7 +64,7 @@ note = "196 MP: open and full refresh" name = "large_file_bench" package = "photocraft-ui-egui" args = ["--reps", "3"] -note = "generated 2 GB PSD and 4.4 GB PSB (6.4 GB cached in /fixtures): read + decode + open; needs ~16 GB of memory" +note = "generated 2 GB PSD and 4.4 GB PSB (6.4 GB cached in /fixtures): read + decode + open; needs ~30 GB of memory (the PSB)" # Hook: the layout-document bench lands with perf/layout-documents (#125/#128). It runs (and its # rows are kept in results.json) as soon as the example exists; add scenarios for its rows then. diff --git a/scorecard/file_compat.toml b/scorecard/file_compat.toml index b3fe5a23ff..3076c96f40 100644 --- a/scorecard/file_compat.toml +++ b/scorecard/file_compat.toml @@ -164,7 +164,7 @@ id = "FILE-216-2" target = "Large-file fixtures: 20 MP, 150-layer, generated 2 GB PSD and 4-5 GB PSB stress tests" status = "done" issue = 216 -note = "Generated at test time by psd fixtures.rs (feature testgen; never committed). io tests/large_files.rs: layered_20_mp_psd_opens_and_saves (5472x3648, 5 layers in 2 groups, mask, drop shadow) and layout_with_150_layers_opens_and_saves (A4 300 ppi, 150 layers in 15 groups, masks, hidden layers, closed groups, drop shadows) run in every cargo test (~2 s each); opt-in psd_just_under_2_gb_opens_and_saves (17,000 px, 2.02 GB PSD) and psb_of_4_gb_opens_and_saves (25,000 px, 4.38 GB PSB, data past 4 GiB), each opened, pixel-checked, saved and re-opened; plus large_psb.rs (2.5 GB merged image). Open times: ui-egui large_file_bench (P20, P63). The 2 GB and 4 GB files are not run in CI (6.4 GB on disk, ~16 GB of memory) (#5107)" +note = "Generated at test time by psd fixtures.rs (feature testgen; never committed). io tests/large_files.rs: layered_20_mp_psd_opens_and_saves (5472x3648, 5 layers in 2 groups, mask, drop shadow) and layout_with_150_layers_opens_and_saves (A4 300 ppi, 150 layers in 15 groups, masks, hidden layers, closed groups, drop shadows) run in every cargo test (~2 s each); opt-in psd_just_under_2_gb_opens_and_saves (17,000 px, 2.02 GB PSD) and psb_of_4_gb_opens_and_saves (25,000 px, 4.38 GB PSB, data past 4 GiB), each opened, pixel-checked, saved and re-opened; plus large_psb.rs (2.5 GB merged image). Open times: ui-egui large_file_bench (P20, P63). The 2 GB and 4 GB files are not run in CI (6.4 GB on disk; ~12 GB of memory for the PSD test, ~25 GB for the PSB test) (#5107)" [[item]] id = "FILE-216-3"