Files
pdfcraft/.github/workflows/ci.yml
T
Tanish MankuandClaude Opus 5 f988cf87ed feat(oracle): measure rendering against Poppler, and turn the pixel test on
Gap 12 of docs/gaps.md: until rendering is measured, every claim about its quality is a judgement.
Text extraction had an oracle and written output had qpdf; rendering had nothing.

`cargo xtask render-oracle` renders page 1 of every corpus file with our CLI and with Poppler's
pdftoppm and reports how far apart they are. The metric is the point. A raw pixel comparison is
useless here -- two correct renderers disagree on every glyph edge, so it calls a perfect page
badly wrong and says nothing about whether a shading is missing. `pixels::compare` counts a pixel
as differing only when it differs from every pixel in the other image's 3x3 neighbourhood by more
than a tolerance, in both directions, taking the worse. A one-pixel shift is forgiven; absent or
miscoloured content is not.

First reading, 250 files at 72 dpi, tolerance 8: 229 compared, median 0.89%, mean 2.16%, p90
5.35%, 25 within 0.1%. It reports rather than gates -- the corpus is not on CI, and a first
reading of a borrowed renderer is a baseline to improve on, not a bar to clear. The worst results
cluster on JBIG2 `bitmap-*` files and on ZapfDingbats, which aims the renderer work at something
specific for the first time. 15 files could not be compared because the two renderers round the
page box to different pixel sizes; they are reported as unmeasurable rather than scored.

Verified against the knob's bounds end to end: tolerance 255 scores every file 0.00%, tolerance 0
takes the median to 0.95% and p90 to 26.36%.

Also turns on `extract_type3_shared_xobjects_matches_poppler_pixels`, which had never run anywhere.
Poppler is now installed on the Linux runner, and the test is version-robust by construction --
both sides of its comparison go through the same pdftoppm, so it is about our two documents rather
than about Poppler's output. Verified in all three states: runs and passes with pdftoppm present,
skips without it, fails without it when PDFCRAFT_ORACLES_REQUIRED is set.

`run_capture` keeps its exact behaviour, including leaving the buffer empty on non-UTF-8 output;
the byte-oriented capture an image needs is a second entry point on the same timeout logic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-10-14 14:28:19 -07:00

153 lines
5.0 KiB
YAML

name: CI
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
schedule:
- cron: "17 3 * * *" # nightly fuzzing
env:
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true
jobs:
test:
name: test (${{ matrix.os }})
runs-on: ${{ matrix.os }}
env:
CARGO_PROFILE_DEV_DEBUG: line-tables-only
CARGO_PROFILE_TEST_DEBUG: line-tables-only
strategy:
fail-fast: false
matrix:
# No macOS on PRs, main or nightly: the org's 5 macOS slots are kept for release builds,
# and release.yml builds macOS (a failing release is fixed and rebuilt).
os: [ubuntu-latest, windows-latest]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# The optional craft-fonts build input (Japanese fonts), so the Japanese font tests run on
# Linux; Windows tests the build without it. See craftrules standards/fonts.md.
- name: Check out craft-fonts (Linux)
if: runner.os == 'Linux'
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: storytold/craft-fonts
ref: 3d9ab3a132ed0eff457e424226a435a901da30d3
path: craft-fonts
- name: Configure craft-fonts (Linux)
if: runner.os == 'Linux'
run: |
echo "CRAFT_FONTS_DIR=$GITHUB_WORKSPACE/craft-fonts" >> "$GITHUB_ENV"
echo "CRAFT_FONTS_REQUIRED=1" >> "$GITHUB_ENV"
- name: Install Linux system dependencies (egui/winit/rfd) and the oracles
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
libxkbcommon-dev libwayland-dev libx11-dev libxrandr-dev libxi-dev \
libgl1-mesa-dev libgtk-3-dev \
qpdf poppler-utils
# The independent oracles (crates/oracle) are only worth having when they run. With this
# set, a missing one fails the job instead of printing a skip notice that reads as a pass.
# Only the tools the job actually uses are installed above.
- name: Require the oracles (Linux)
if: runner.os == 'Linux'
run: echo "PDFCRAFT_ORACLES_REQUIRED=1" >> "$GITHUB_ENV"
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
with:
toolchain: stable
components: clippy, rustfmt
- name: Require Windows executable resources
if: runner.os == 'Windows'
run: Add-Content -Path $env:GITHUB_ENV -Value 'PDFCRAFT_REQUIRE_WINRES=1'
shell: pwsh
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
- name: Format
if: runner.os == 'Linux'
run: cargo fmt --all -- --check
- name: Clippy
run: cargo clippy --workspace --all-targets -- -D warnings
- name: Test
if: runner.os != 'Linux'
run: cargo test --workspace
- name: Test (with craft-fonts)
if: runner.os == 'Linux'
run: cargo test --workspace
- name: Layering
if: runner.os == 'Linux'
run: cargo xtask layers
- name: Assets (licences and attribution)
if: runner.os == 'Linux'
run: cargo xtask assets
- name: CLI without the MCP server still builds
if: runner.os == 'Linux'
run: cargo check -p pdfcraft-cli --no-default-features
wasm:
name: wasm32 check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
with:
toolchain: stable
targets: wasm32-unknown-unknown
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: wasm32
- name: Check library crates for wasm32
run: cargo xtask wasm
deny:
name: dependency licences and advisories
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: EmbarkStudios/cargo-deny-action@v2
with:
command: check
fuzz:
name: fuzz (mutation, 15 min)
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@686976e191b89faba57d3206551f0f330d8cb249 # stable
with:
toolchain: stable
- uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
key: fuzz
- name: Fetch the pdf.js corpus (seeds)
run: cargo xtask corpus
- name: Fuzz
run: cargo xtask fuzz --time 900 --seed ${{ github.run_number }}
- name: Upload findings
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: fuzz-findings
path: fuzz-out/findings
if-no-files-found: ignore