mirror of
https://github.com/storytold/effectcraft.git
synced 2026-11-10 18:31:30 -06:00
Elias
e5d5ed80e9
- [profile.ci] (release without LTO, 16 codegen units); `xtask ci` runs clippy and tests with --profile ci instead of --release. - Wall-clock asserts in engine memory_tick, gpu OpenGL decline and script socket tests keep their intent with bounds that survive a loaded machine. - docs/testing.md and CONTRIBUTING.md mention the profile. (cherry picked from commit b45c8a379351ebdfda0d7076ad4b740baf6f2087) (cherry picked from commit fa1ea6a107c4c1936cde89b518a05adc73f2b4b0)
98 lines
5.0 KiB
Markdown
98 lines
5.0 KiB
Markdown
# Testing
|
||
|
||
```sh
|
||
cargo test --workspace # everything
|
||
cargo xtask ci # fmt, clippy -D warnings, tests, layers, assets, wasm
|
||
cargo xtask ci --affected # the same gates, only for the crates your change can affect
|
||
```
|
||
|
||
## Affected-only CI
|
||
|
||
The full `cargo xtask ci` builds and tests every crate in release mode and takes over an hour.
|
||
For each change, agents and contributors run the affected-only variant; the full CI runs before a
|
||
release (and on any change it falls back to, below).
|
||
|
||
```sh
|
||
cargo xtask ci --affected # base: the merge-base of HEAD with main
|
||
cargo xtask ci --affected --base HEAD~3 # any git revision
|
||
cargo xtask ci --affected --dry-run # print the plan only
|
||
```
|
||
|
||
It collects every file that differs from the base (committed, staged, unstaged and untracked) and
|
||
maps each one to:
|
||
|
||
- the workspace package whose directory holds it, and
|
||
- every package whose sources point at it with a `../` path, such as
|
||
`include_bytes!("../../../assets/fonts/Inter-Regular.ttf")` or
|
||
`concat!(env!("CARGO_MANIFEST_DIR"), "/../../docs/effects.md")`. So editing
|
||
`docs/preferences.md` reruns the engine tests that check it.
|
||
|
||
The affected set is those packages plus everything that depends on them (transitively through
|
||
normal and build dependencies) and every package that uses one of them as a dev-dependency. For
|
||
that set it runs `cargo fmt --check` (whole workspace, it is fast), `cargo clippy --all-targets
|
||
--release -- -D warnings`, the tests (`cargo nextest run --release` plus `cargo test --release
|
||
--doc` when cargo-nextest is installed, otherwise `cargo test --release`), the layers check, the
|
||
assets check only when an asset, an `.attribution` sidecar or `ATTRIBUTION.md` changed, and the
|
||
wasm check only for affected L0–L4 crates and the web crates. It prints the plan first and a timing
|
||
table at the end:
|
||
|
||
```text
|
||
ci --affected: base bc0570888 (HEAD), 1 changed file(s)
|
||
changed: automation
|
||
affected: automation, cli (2 of 35 crates)
|
||
steps: fmt, clippy (2), nextest (2), doctests (1), layers (assets skipped: no asset changes)
|
||
```
|
||
|
||
Changes to the root `Cargo.toml`, `Cargo.lock`, `.cargo/`, `xtask/`, `rust-toolchain(.toml)` or
|
||
`clippy.toml` can change every crate's build, so they run the full CI instead and say why. Changes
|
||
no crate reads (most of `docs/`, `README.md`, `.github/`, `packaging/`) only get the cheap checks.
|
||
|
||
Limits: a crate that reads a repo file at run time through a path that is not a `../` literal in its
|
||
sources is not detected, and building a subset of packages can unify features differently from the
|
||
whole workspace. Run the full `cargo xtask ci` before a release.
|
||
|
||
`xtask ci` builds clippy and tests with `--profile ci` (release optimisation without LTO, 16 codegen
|
||
units, output in `target/ci/`): fast to build, still optimised enough for the pixel and perf tests.
|
||
|
||
## What the tests cover
|
||
|
||
- **Engine commands** (`crates/engine/src/tests*.rs`): every command family, including undo and
|
||
redo, and that every menu entry resolves to a registered command.
|
||
- **Rendering** (`crates/render`): pixel checks on small compositions, for blend modes, masks,
|
||
mattes, time remapping, 3D projection, lighting and shadows. Fast paths (the layer cache, the
|
||
affine compositor, blurs, motion blur accumulation) are checked against their reference
|
||
implementations.
|
||
- **Effects** (`crates/effects`): each effect has a determinism test and at least one behaviour
|
||
test (identity at neutral settings, known pixel results, or simulations giving the same frame
|
||
whether you seek straight to it or play up to it). A registry-wide test checks that effects
|
||
which read the clock are declared time-dependent. `crates/effects/tests/sim_golden.rs` pins
|
||
exact pixel hashes of the simulation effects the GPU shares plans with (CC Rainfall … Card
|
||
Wipe) at several settings, times, bit depths and resolutions, so any change to their CPU output
|
||
fails (re-pin with `SIM_GOLDEN_PRINT=1` after an intended change; pinned for aarch64 macOS,
|
||
with the hashes another macOS release's libm gives for unchanged code in `LIBM_VARIANTS`).
|
||
- **Export** (`crates/export/tests`): every format is encoded and decoded back, checking frame
|
||
count, size and pixels. When `ffmpeg`/`ffprobe` are installed they are used as an outside
|
||
check; they are never linked or shipped.
|
||
- **Automation** (`crates/automation`, `apps/effectcraft-cli/tests`): MCP protocol round trips and
|
||
the command-line tool's JSON output.
|
||
- **Interface** (`crates/ui-egui`): headless egui_kittest tests for panels and dialogs.
|
||
|
||
## Looking at the interface without a window
|
||
|
||
```sh
|
||
cargo run -p effectcraft-ui-egui --example snapshot -- --out ui.png \
|
||
--step '{"method":"engine.execute","params":{"command":"layer.select","params":{"layers":["#2"]}}}'
|
||
```
|
||
|
||
Each `--step` is a control-channel request; `{"method":"snap","params":{"path":"x.png"}}` writes an
|
||
intermediate image.
|
||
|
||
## Benchmarks
|
||
|
||
```sh
|
||
cargo run --release -p effectcraft-cli -- bench --n 10 --play 30
|
||
```
|
||
|
||
prints per-layer and per-effect timings for one frame, and playback timings with and without the
|
||
layer cache.
|