mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-11-01 14:08:23 -06:00
Grzegorz MartensandClaude Opus 5.5
deb70c92ca
A review found that a dense page came out as a stream PdfCraft's own renderer refuses, so the page drew blank after optimizing: hayro-jbig2 decodes a text region of at most 10,000 symbol instances, and refuses 32,768 symbols or more. The stream was valid (Poppler decoded it), but every mark went into one text region and up to 65,536 symbols were kept. A 300 dpi letter page of the optimizer test's own text crossed it. A page with more than 10,000 marks is now placed by several text regions, each over a band of rows and all referring to the one symbol dictionary, which each region but the last keeps retained (7.2.4). Past 32,767 distinct symbols the image is coded generically. New tests: 25,000 copies placed by three regions, which decode; the symbol caps decoded, not just reached; and in the optimizer, a whole letter page of rendered text that looks the same as JBIG2 symbols (it drew blank before). The READMEs say "still looked at" for the caps, and that the scans figure is the six pages' total (20% on one page). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
80 lines
6.0 KiB
Markdown
80 lines
6.0 KiB
Markdown
# pdfcraft-optimize
|
||
|
||
Layer L4: the PDF Optimizer (execution plan M11.1; Acrobat's Reduce File Size and Optimize PDF ▸
|
||
Advanced optimization).
|
||
|
||
```rust
|
||
let report = optimize(&mut doc, &Settings::default())?; // Reduce File Size's choices
|
||
let report = optimize(&mut doc, &Preset::Mobile.settings())?; // smaller, for screens
|
||
// Or with progress: called before each image and the clean-up; `false` cancels.
|
||
let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going)?;
|
||
```
|
||
|
||
- **Presets:** Standard (Reduce File Size's choices: images above 225 ppi to 150 ppi, JPEG
|
||
medium) and Mobile (above 144 ppi to 96 ppi, JPEG high), as in Acrobat's Settings menu.
|
||
- **Merge image fragments** (`fragments`, on by default): some PDF printers draw a page picture
|
||
as thousands of images one pixel row high. Runs of upright 8-bit grey or RGB images in one
|
||
colour space, sharing their left and right edges and stacked edge to edge (at least four,
|
||
drawn with nothing else between them), become one lossless image at their finest resolution,
|
||
which the image settings then resample; the strips are dropped by Clean Up. A real page
|
||
printed this way went from 4.3 MB to 231 KB (Standard) and 161 KB (Mobile). Not merged yet:
|
||
side-by-side tiles, rotated strips, masks. Content arrays use the shared space boundary
|
||
policy, so trailing comments, split operands and graphics state keep their meaning after merging.
|
||
- **Images:** the effective resolution of each image is measured where pages draw it (the CTM
|
||
through form XObjects, up to 12 deep; the smallest over all uses).
|
||
Page-local `/UserUnit` scaling is included, so large-format pages are measured in physical inches.
|
||
If a use can't be
|
||
measured (forms nested deeper, or drawn too many times), no image is resampled. Colour and grayscale images above a
|
||
threshold are resampled with a bicubic (Catmull-Rom) filter to a target resolution and
|
||
stored as JPEG (Acrobat's quality steps) or ZIP; soft masks are resampled with their image.
|
||
An image is replaced only if the result is smaller. Left alone: CMYK and other colour spaces,
|
||
image masks, decode arrays, more than 8 bits, JPEG 2000, JBIG2 and CCITT. A JPEG is read with
|
||
its colour transform (an Adobe marker's, else `/ColorTransform`, ISO 32000-2 §7.4.8), so one
|
||
stored untransformed keeps its colours; one whose marker and `/ColorTransform` disagree is left
|
||
alone, since readers differ on which wins. Samples are read with `/DecodeParms` given by
|
||
reference too, the numbers inside included (`Document::with_direct_parms`), here and below:
|
||
without them a predictor's tag bytes would shift every row of an image written back.
|
||
- **Monochrome images** (`Settings::mono`, CCITT Group 4 by default, or JBIG2): a 1-bit image
|
||
with one colour component (grey, indexed, a separation) or an image mask, stored with no filter
|
||
or with general-purpose ones, is stored again as CCITT Group 4 (`pdfcraft_filters::encode_g4`)
|
||
or lossless JBIG2 (`encode_jbig2`; a document below PDF 1.4 gets `/Version /1.4` in
|
||
its catalog) when that is smaller as stored. JBIG2 stores each distinct mark once, as a
|
||
symbol, when the image's marks repeat exactly, as rendered text's do: on six pages of
|
||
300 dpi text that came to 28% of a generic region's size and 17% of Group 4's. On scans,
|
||
whose copies of a letter are seldom identical, symbols gain little: on the same pages with 2%
|
||
or more of the letters' edge pixels flipped at random, at most 8% below a generic region in
|
||
total (20% on one page), and the generic region was 27–33% below Group 4. It is lossless: the
|
||
samples decode to the same bits, so the colour space, decode array and masks stay. Group 4
|
||
wins on scans, whose ragged edges defeat Flate; on clean computer-drawn 1-bit art Flate can
|
||
be smaller, and the image then stays. Not yet:
|
||
JBIG2 refinement coding (lossless gains on scans), a symbol dictionary shared between images
|
||
(`/JBIG2Globals`), lossy JBIG2, monochrome downsampling, inline images, soft masks and
|
||
`/Mask` images, and Group 4 for rows wider than 32,767 samples (JBIG2 takes rows of up to
|
||
1,048,576). Redaction doesn't decode JBIG2 or Group 4 yet: a mark over part of such an image
|
||
removes the whole image (nothing leaks). Export reads JBIG2, but nothing that exports an image
|
||
reads Group 4 yet: Export ▸ Images and Save Image As refuse it, Copy Image fails, and Word,
|
||
PowerPoint and HTML export leave the picture out.
|
||
- **Discard objects:** thumbnails, alternate images, document tags, print settings.
|
||
- **Clean up:** Flate for streams with no filter; for streams with an outer ASCII85 or ASCIIHex
|
||
layer, that layer is removed when the result is smaller (the inner filters keep their encoded
|
||
bytes; streams with `/DecodeParms`, images and object streams keep their ASCII layer. LZW-compressed
|
||
streams, content and images alike, are decoded and stored with Flate instead (Acrobat's "In
|
||
streams that use LZW encoding, use Flate instead"): a predictor carries over to Flate without
|
||
`EarlyChange`, filters applied before the LZW one go with it and those after it keep their
|
||
bytes. A stream whose LZW data doesn't decode cleanly is left as it is. Not yet: inline images.
|
||
|
||
The engine runs Remove Hidden Information (`pdfcraft-redact`) for the user-data categories,
|
||
then this, then merges identical objects and writes a full, compressed save
|
||
(`pdfcraft_engine::optimizer::OptimizeJob`, which the UI runs on a worker thread with a
|
||
progress bar). JPEG decoding comes from `zune-jpeg` (the decoder behind `image` and the
|
||
renderer), JPEG encoding and resampling from the `image` crate.
|
||
|
||
## Target PDF version: partial compatibility
|
||
|
||
The target selects the header version, clears the catalog `/Version`, controls object
|
||
streams (disabled for PDF 1.4), and checks encryption requirements. It does **not**
|
||
detect or convert content newer than the target: optional content, JPEG 2000 images,
|
||
16-bit images, OpenType fonts and PDF 2.0 features can remain. A PDF 1.4 header
|
||
therefore does not guarantee compatibility with PDF 1.4 readers or validators.
|
||
`optimize.compatibility-level` remains partial until content compatibility is checked.
|