M11: merge image fragments, and the Standard and Mobile optimizer presets (closes #1783) (#1784)

* M11: merge image fragments, and the Standard and Mobile optimizer presets

Some PDF printers draw a page picture as thousands of images one pixel row
high (one real page: 3,300 strips, 4.3 MB). The optimizer works image by
image, a one-row image can't be resampled, and every JPEG strip carries its
own headers, so Reduce File Size saved 2 bytes on it.

- pdfcraft-optimize `fragments` (Settings::merge_fragments, on by default):
  runs of `q … cm /Im Do Q` drawing upright 8-bit grey or RGB images in one
  colour space, sharing their left and right edges and stacked edge to edge
  (at least four, nothing else drawn between them; or, failing that, the
  rows drawn one under the other up to a gap) become one lossless image at
  their finest resolution. The image settings then resample it, and Clean
  Up drops the strips. A page whose content can't be read back exactly is
  left alone.
- Presets as in Acrobat's Settings menu: Standard (Reduce File Size's
  choices) and Mobile (images above 144 ppi to 96 ppi, JPEG high). The PDF
  Optimizer dialog has a Settings menu (Custom once an option changes) and
  a Merge image fragments box; doc_optimize takes preset and
  merge_fragments and reports merged_images and fragments_merged.
- A progress stage for the merge; four new strings in all 21 catalogs.

On the real page: Standard/Reduce 4.3 MB → 231 KB, Mobile → 161 KB, with
a rendering closer to the original than Acrobat's Mobile result (147 KB).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Refresh catalog hashes after merging main

---------

Co-authored-by: magdale76 <16279778+magdale76@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
This commit is contained in:
magdale76
2026-10-15 02:25:19 -04:00
committed by GitHub
co-authored by Claude Opus 5.5 magdale76 Brandon Thomas
parent 72a46facb8
commit df0bc0ffbc
34 changed files with 803 additions and 36 deletions
+21 -21
View File
@@ -7,7 +7,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Greek UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "a579e60a58e88ea83732609f2fb0921afa3d5d61082bea3be969db7ad6d3a12f"
sha256 = "0ef505ad4ae53d404d4474d2881f54bc3692739fe9200692fad2b3ace9036a03"
# ATTRIBUTION.toml: one entry per asset (AGENTS.md §1). The source of truth for provenance and licences.
# ATTRIBUTION.md is generated from this file: `cargo xtask assets --write`. Checked by `cargo xtask assets` (part of `xtask ci`).
@@ -30,7 +30,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Simplified Chinese UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "eae078763f4d4b6267cce596d405e28102ebb872d9cb8bc375671cd4d264da9e"
sha256 = "73efe98c3a4190715f3f76d714390396887f45d2092926729832b79a194488c0"
[[asset]]
path = "crates/ui-egui/src/i18n/zh-hant.tsv"
@@ -41,7 +41,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Traditional Chinese UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "b94db0634220f21673b649be78e65bf5d9b2f939afd3992b1a5ab567d77677b0"
sha256 = "e0959af2ee8e317372d9076e487e41d025015460b99906482efa1a0e528802c3"
[[asset]]
path = "crates/ui-egui/src/i18n/cs.tsv"
@@ -52,7 +52,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Czech UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "6f2d46364ee4a703f6a05017d5cf6f2da4216bb83b577092b456db742a961d2a"
sha256 = "0a327af10b0bb916ef1b912e6797214bd0b2c18c1e0ef27d9f7a92a3cbeac5b7"
[[asset]]
path = "crates/ui-egui/src/i18n/pt-br.tsv"
@@ -63,7 +63,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Brazilian Portuguese UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "0a44a27428691465074177508dde58475188fc3ad176d5f43a74dd4d518a2495"
sha256 = "c2cf9490a9f9c828382e2f24df3eaea511b4dfab6543ca00ac61e0014029f051"
[[asset]]
path = "crates/ui-egui/src/i18n/es.tsv"
@@ -74,7 +74,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Spanish UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "4320582e8212a8387bce630107e19ef2ffb78586d453d36872ac40a9bf917b1b"
sha256 = "fccce321ccb64bdfcb75e1d6d09f661fb6a738b30cc0ec1133134ea91959e3ad"
[[asset]]
path = "crates/ui-egui/src/i18n/fr.tsv"
@@ -85,7 +85,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "French UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "0060ce0ff2ee45ac5d6ebfcc39eb9aec565006fee2867623e12572a1b4b80b5e"
sha256 = "64722973eaf542884ddd170dd9541649c1b1b365ad8c63d0d2864ffbecd20941"
[[asset]]
path = "crates/ui-egui/src/i18n/de.tsv"
@@ -96,7 +96,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "German UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "abeb8b6758e460d9d1fac894392f280fc2d0dadb30edce784ee0625eb318726c"
sha256 = "0475c03f0b2190998e247d6996d5473d3fd85ea6c55e582b7159b42096c63f9e"
[[asset]]
path = "crates/ui-egui/src/i18n/te.tsv"
@@ -107,7 +107,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Telugu UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "7fc8def1baa4680ac1821928c4b33c27ad15257b6be8ecaa43d6bad28d49d2b3"
sha256 = "6423cc0904fd4e49af25121d9fe4d67d09d684f45acbb8165d4ea7a474a3275c"
[[asset]]
path = "crates/ui-egui/src/i18n/it.tsv"
@@ -118,7 +118,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Italian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "ef898359cd992adff06d1889536aea90246ae63443f5181e71a8b8b12cdfd4d8"
sha256 = "588adcef83d5ec8e9a1e1fac093afa6afee30d94ac68e8b428ebd487cdc23c3e"
[[asset]]
path = "crates/ui-egui/src/i18n/pl.tsv"
@@ -129,7 +129,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Polish UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "2ee8d548ad696820a9710937d4d27d5dfe529244cd20df2664a4192ded639408"
sha256 = "9fda2deb8db4e919199ae032b9a9a220f32896462640578cd155074809e71f0d"
[[asset]]
path = "crates/ui-egui/src/i18n/ru.tsv"
@@ -140,7 +140,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Russian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "e7f2727e4ee17a2f9a37bf8d8bf563c625c8700e1389de0b79c22002b8d3919c"
sha256 = "b60244d234bcec0602a10a077167861d60c86ab3a364c3dfc4b4e0466dd54872"
[[asset]]
path = "crates/ui-egui/src/i18n/hu.tsv"
@@ -151,7 +151,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Hungarian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "b1bb541fa2a453537c383b9346d8c9504b7559004ae94a851f18348de2910694"
sha256 = "18416c5dc7b28bfb999e1f8127337d37915c67a91d8b8b3fc52e3286a11205ba"
[[asset]]
path = "crates/ui-egui/src/i18n/bg.tsv"
@@ -162,7 +162,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Bulgarian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "f90f24f1d30c8e8cdc595e2b229e1aca35125b55af0fce5b26d0befbb31f1fc1"
sha256 = "e50991d9d81a9a45e78c719c790029486e4c03045c30b10b4fee349fcc6702d0"
[[asset]]
path = "crates/ui-egui/src/i18n/kk.tsv"
@@ -173,7 +173,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Kazakh UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "5bc5f87e61cc1feb64d9ae94ec39f218aa7abe6697065647152104197a356d48"
sha256 = "73fb4046cd4fdccd8c1da4abfab90ecd0ca2395fc9a6302dd00dd4f810a14418"
[[asset]]
path = "crates/ui-egui/src/i18n/ja.tsv"
@@ -184,7 +184,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Japanese UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "95d004849c3760587098ea0bfc658aa0d94887ff4f2a22d7f3d1565df5ddaf30"
sha256 = "e090ca970a518bbf667959181167ff8f8058b78e4239c0c62654c58450738a99"
[[asset]]
path = "crates/ui-egui/src/i18n/uk.tsv"
@@ -195,7 +195,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Ukrainian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "ca109e277cbe196394fb50323c632ff65972f9aa60941e69cbb244857930eec0"
sha256 = "a5e4cbf2fd2b1f840d31482a090a68fe58dbbd03e2187deecd1398d091c9fb8f"
[[asset]]
path = "crates/ui-egui/src/i18n/ar.tsv"
@@ -206,7 +206,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Arabic UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "aa1b6741974392a8b246bab04169608211d6d6f19f2adb000cf033a6d1a3ef2e"
sha256 = "036dfc2952a758352933ec9ab7f1668a699ed108c8902340566f149df4bc0b79"
[[asset]]
path = "crates/ui-egui/src/i18n/tr.tsv"
@@ -217,7 +217,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Turkish UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "895f2f8e6b44f7cf7f383e13765e62b5875938ded5135119b3e0621f6375c3d5"
sha256 = "d4ac259fc9dad897c9ca8a55eb830d2027f3387eafd641744dd7b9d43a5d1cb4"
[[asset]]
path = "crates/ui-egui/src/i18n/lt.tsv"
@@ -228,7 +228,7 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Lithuanian UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "e5f36292f84ec43ae42479735f88c2667bef05f719ab11e47daaa517cd070f3e"
sha256 = "b0e3dacf8ce437663e3cb9946ced5b9f3cde6a62a2bf5ff8dd3d45268b264bf5"
[[asset]]
@@ -3363,4 +3363,4 @@ licence = "MIT OR Apache-2.0"
licence_file = "LICENSE-MIT"
kind = "translation"
usage = "Slovene UI strings, compiled in by crates/ui-egui/src/i18n/mod.rs"
sha256 = "25523c021600c5aecc778f53e707a114267a4463ab911fd83658a4c241ece6d3"
sha256 = "eb1f9d2990ab5e177be78d5d5e6d4c3cbdb0961c64df04bbd34e5d9ad5b42a3d"
+8 -2
View File
@@ -995,11 +995,14 @@ impl Automation {
}
fn doc_optimize(&mut self, a: &Args) -> Result<Value> {
use pdfcraft_engine::optimize::{Compression, ImageSettings, Settings};
use pdfcraft_engine::optimize::{Compression, ImageSettings, Preset};
let id = self.doc(a)?.id;
let before = self.doc(a)?.bytes.len();
let path = self.resolve(a.str("path")?, true)?;
let mut settings = Settings::default();
let mut settings = match a.opt_str("preset")? {
None => Preset::Standard.settings(),
Some(p) => Preset::from_name(p).ok_or_else(|| ToolError::InvalidArgs(format!("unknown preset {p:?} (standard, mobile)")))?.settings(),
};
let images = |key: &str, base: ImageSettings| -> Result<ImageSettings> {
let Some(v) = a.get(key) else { return Ok(base) };
let mut s = base;
@@ -1027,6 +1030,7 @@ impl Automation {
settings.color = images("color", settings.color)?;
settings.gray = images("gray", settings.gray)?;
for (key, flag) in [
("merge_fragments", &mut settings.merge_fragments),
("discard_thumbnails", &mut settings.discard_thumbnails),
("discard_alternate_images", &mut settings.discard_alternate_images),
("discard_tags", &mut settings.discard_tags),
@@ -1059,6 +1063,8 @@ impl Automation {
"path": path.to_string_lossy(),
"bytes_before": before,
"bytes_after": bytes.len(),
"merged_images": o.merged_images,
"fragments_merged": o.fragments_merged,
"images": o.images,
"images_resampled": o.images_resampled,
"images_recompressed": o.images_recompressed,
+3 -1
View File
@@ -1237,12 +1237,14 @@ pub fn tools() -> Vec<ToolDef> {
.with(schema(json!({ "doc": doc() }), &["doc"])),
t("doc_open_revision", "Open a revision", "Open saved revision `revision` (1 = the oldest) of a document as a new, unsaved document, to see the file as it was then.")
.with(schema(json!({ "doc": doc(), "revision": { "type": "integer", "minimum": 1 } }), &["doc", "revision"])),
t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
.cmd("optimize.advanced")
.with(schema(
json!({
"doc": doc(),
"path": { "type": "string" },
"preset": { "type": "string", "enum": ["standard", "mobile"] },
"merge_fragments": { "type": "boolean" },
"color": { "type": "object" },
"gray": { "type": "object" },
"discard_thumbnails": { "type": "boolean" },
+119
View File
@@ -4183,6 +4183,125 @@ fn optimizing_through_tools() {
assert!(matches!(a.call("doc_optimize", &json!({ "doc": doc, "path": "x.pdf", "discard": ["everything"] })), Err(ToolError::InvalidArgs(_))));
}
/// A one-page PDF whose page picture is drawn as `rows` grey images one pixel row high, 600 px
/// wide on a 72 pt square (600 ppi), as some PDF printers write it.
fn printed_in_strips(rows: usize) -> Vec<u8> {
let mut objs: Vec<Vec<u8>> = vec![b"<< /Type /Catalog /Pages 2 0 R >>".to_vec()];
let names: String = (0..rows).map(|r| format!("/R{r} {} 0 R ", 5 + r)).collect();
objs.push(b"<< /Type /Pages /Kids [3 0 R] /Count 1 >>".to_vec());
objs.push(format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 72 72] /Resources << /XObject << {names}>> >> /Contents 4 0 R >>").into_bytes());
let k = 72.0 / rows as f64;
let content: String = (0..rows)
.map(|r| {
format!(
"q 72 0 0 {k} 0 {} cm /R{r} Do Q
",
(rows - 1 - r) as f64 * k
)
})
.collect();
objs.push(
format!(
"<< /Length {} >>
stream
{content}
endstream",
content.len() + 1
)
.into_bytes(),
);
for r in 0..rows {
let px: Vec<u8> = (0..600).map(|x| ((x + r * 7) % 256) as u8).collect();
let mut o = format!(
"<< /Type /XObject /Subtype /Image /Width 600 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8 /Length {} >>
stream
",
px.len()
)
.into_bytes();
o.extend_from_slice(&px);
o.extend_from_slice(
b"
endstream",
);
objs.push(o);
}
let mut out = b"%PDF-1.5
"
.to_vec();
let mut offs = Vec::new();
for (i, o) in objs.iter().enumerate() {
offs.push(out.len());
out.extend_from_slice(
format!(
"{} 0 obj
",
i + 1
)
.as_bytes(),
);
out.extend_from_slice(o);
out.extend_from_slice(
b"
endobj
",
);
}
let x = out.len();
out.extend_from_slice(
format!(
"xref
0 {}
0000000000 65535 f
",
objs.len() + 1
)
.as_bytes(),
);
for o in offs {
out.extend_from_slice(
format!(
"{o:010} 00000 n
"
)
.as_bytes(),
);
}
out.extend_from_slice(
format!(
"trailer
<< /Size {} /Root 1 0 R >>
startxref
{x}
%%EOF
",
objs.len() + 1
)
.as_bytes(),
);
out
}
#[test]
fn a_page_printed_in_strips_gets_small_through_tools() {
let dir = workdir("optimize-strips");
std::fs::write(dir.join("strips.pdf"), printed_in_strips(300)).unwrap();
let mut a = auto(&dir);
let doc = ok(&mut a, "doc_open", json!({ "path": "strips.pdf" }))["doc"].as_u64().unwrap();
let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "mobile.pdf", "preset": "mobile" }));
assert_eq!((r["merged_images"].as_u64(), r["fragments_merged"].as_u64(), r["unused_xobjects"].as_u64()), (Some(1), Some(300), Some(300)), "{r}");
assert!(r["bytes_after"].as_u64().unwrap() * 10 < r["bytes_before"].as_u64().unwrap(), "{r}");
// Reduce File Size merges too; with the option off, the strips stay.
let reduced = ok(&mut a, "doc_reduce", json!({ "doc": doc, "path": "reduced.pdf" }));
assert!(reduced["bytes_after"].as_u64().unwrap() * 10 < reduced["bytes_before"].as_u64().unwrap(), "{reduced}");
let off = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "off.pdf", "merge_fragments": false }));
assert_eq!(off["merged_images"].as_u64(), Some(0), "{off}");
// The merged page opens and renders.
let mobile = ok(&mut a, "doc_open", json!({ "path": "mobile.pdf" }))["doc"].as_u64().unwrap();
assert!(a.call("page_render", &json!({ "doc": mobile, "page": 1, "dpi": 72 })).is_ok());
assert!(matches!(a.call("doc_optimize", &json!({ "doc": doc, "path": "x.pdf", "preset": "tiny" })), Err(ToolError::InvalidArgs(_))));
}
#[test]
fn initial_view_through_tools() {
let dir = workdir("initial-view");
+4
View File
@@ -17,6 +17,8 @@ pub enum OptimizeStage {
/// Removing the user data chosen under Discard User Data.
#[default]
Discarding,
/// Merging the fragments of a page picture into one image.
Fragments,
/// About to process image `done` (0-based) of `total`.
Images { done: usize, total: usize },
/// Discarding objects and compressing unencoded streams.
@@ -33,6 +35,7 @@ impl OptimizeStage {
pub fn fraction(self) -> f32 {
match self {
OptimizeStage::Discarding => 0.0,
OptimizeStage::Fragments => 0.03,
OptimizeStage::Images { done, total } => 0.05 + 0.75 * if total == 0 { 1.0 } else { done.min(total) as f32 / total as f32 },
OptimizeStage::CleaningUp => 0.8,
OptimizeStage::Merging => 0.85,
@@ -85,6 +88,7 @@ impl OptimizeJob {
let discarded = if discard.is_empty() { Vec::new() } else { pdfcraft_redact::sanitize::remove_hidden(&mut cos, &discard)? };
let report = optimize::optimize_with_progress(&mut cos, &settings, &mut |s| {
step(match s {
optimize::Stage::Fragments => OptimizeStage::Fragments,
optimize::Stage::Images { done, total } => OptimizeStage::Images { done, total },
optimize::Stage::CleanUp => OptimizeStage::CleaningUp,
})
+10
View File
@@ -5,10 +5,20 @@ Advanced optimization).
```rust
let report = optimize(&mut doc, &Settings::default())?; // Reduce File Size's choices
let report = optimize(&mut doc, &Preset::Mobile.settings())?; // smaller, for screens
// Or with progress: called before each image and the clean-up; `false` cancels.
let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going)?;
```
- **Presets:** Standard (Reduce File Size's choices: images above 225 ppi to 150 ppi, JPEG
medium) and Mobile (above 144 ppi to 96 ppi, JPEG high), as in Acrobat's Settings menu.
- **Merge image fragments** (`fragments`, on by default): some PDF printers draw a page picture
as thousands of images one pixel row high. Runs of upright 8-bit grey or RGB images in one
colour space, sharing their left and right edges and stacked edge to edge (at least four,
drawn with nothing else between them), become one lossless image at their finest resolution,
which the image settings then resample; the strips are dropped by Clean Up. A real page
printed this way went from 4.3 MB to 231 KB (Standard) and 161 KB (Mobile). Not merged yet:
side-by-side tiles, rotated strips, masks.
- **Images:** the effective resolution of each image is measured where pages draw it (the CTM
through form XObjects; the smallest over all uses). Colour and grayscale images above a
threshold are resampled with a bicubic (Catmull-Rom) filter to a target resolution and
+299
View File
@@ -0,0 +1,299 @@
//! Merge image fragments: some PDF printers draw a page picture as hundreds or thousands of
//! images one pixel row high, stacked edge to edge. Each fragment is too small to resample, and
//! every JPEG fragment carries its own headers, so a one-page document can weigh megabytes and
//! no image setting makes it smaller. Here the fragments are put back together into one image,
//! which the image settings then resample and recompress like any other.
//!
//! What counts as a run of fragments, and is merged:
//! - consecutive `q … cm /Name Do Q` groups in a page's content stream, with nothing else
//! between them, each drawing an image upright (no rotation, skew or flip);
//! - the images are 8-bit grey or RGB in the same colour space, without masks or a decode array
//! that changes anything, in a codec PdfCraft decodes;
//! - they share their left and right edges and are stacked edge to edge with no gap or overlap
//! (in any drawing order), at least [`MIN_FRAGMENTS`] of them.
//!
//! The merged image keeps the finest resolution of its fragments both ways, so nothing is lost
//! until the image settings downsample it; it is stored losslessly. A page whose content
//! stream can't be read back exactly is left alone. The fragments are then unused, and Clean Up
//! drops them.
use pdfcraft_content::{Matrix, Op, parse, serialize_ops};
use pdfcraft_cos::{Dict, Document, ObjRef, Object, Stream};
use crate::images::{components, content_bytes, decodable, page_resources, pixels, resample};
use crate::{OptimizeError, Report, Stage};
/// Fewest stacked images treated as fragments of one picture.
pub const MIN_FRAGMENTS: usize = 4;
/// Most pixels a merged image may have (memory).
const MAX_PIXELS: u64 = 80_000_000;
/// One fragment: its image, its box in the coordinates the run is drawn in, and its size.
struct Fragment {
image: ObjRef,
rect: [f64; 4],
width: u32,
height: u32,
}
/// The ops `q`, one or more `cm`, `Do`, `Q` starting at `i`: the index after them, the matrix
/// and the name drawn.
fn group_at(ops: &[Op], i: usize) -> Option<(usize, Matrix, Vec<u8>)> {
if !ops.get(i)?.is("q") {
return None;
}
let mut j = i + 1;
let mut m = Matrix([1.0, 0.0, 0.0, 1.0, 0.0, 0.0]);
while let Some(op) = ops.get(j).filter(|o| o.is("cm")) {
m = Matrix::from_operands(&op.operands)?.then(&m);
j += 1;
}
let draw = ops.get(j).filter(|o| o.is("Do"))?;
let name = draw.name(0)?.to_vec();
ops.get(j + 1).filter(|o| o.is("Q"))?;
(j > i + 1).then_some((j + 2, m, name))
}
/// The image `name` draws, if it can be a fragment: its reference, size, colour space and
/// component count.
fn fragment_image(doc: &Document, xobjects: &Dict, name: &[u8]) -> Option<(ObjRef, u32, u32, Object, usize)> {
let r = xobjects.get(name).and_then(Object::as_ref)?;
let obj = doc.get(r);
let Object::Stream(s) = &*obj else { return None };
let d = &s.dict;
if d.name(b"Subtype") != Some(b"Image") || d.int(b"BitsPerComponent") != Some(8) || d.contains(b"Mask") || d.contains(b"SMask") {
return None;
}
if matches!(d.get(b"ImageMask"), Some(Object::Bool(true))) || !decodable(doc, s)? {
return None;
}
let cs = d.get(b"ColorSpace")?.clone();
let n = components(doc, &cs)?;
let (w, h) = (u32::try_from(d.int(b"Width")?).ok()?, u32::try_from(d.int(b"Height")?).ok()?);
(w > 0 && h > 0).then_some((r, w, h, cs, n))
}
/// Whether `run` tiles one rectangle top to bottom: same left and right edges, edge to edge.
/// Sorts it from the top down.
fn stacked(run: &mut [Fragment]) -> bool {
let Some(first) = run.first() else { return false };
let [x0, _, x1, _] = first.rect;
let width = x1 - x0;
if !(width.is_finite() && width > 0.0) {
return false;
}
let tol = 1e-3 * width.max(1.0);
if run.iter().any(|f| {
(f.rect[0] - x0).abs() > tol || (f.rect[2] - x1).abs() > tol || f.rect[3].partial_cmp(&f.rect[1]) != Some(std::cmp::Ordering::Greater)
}) {
return false;
}
run.sort_by(|a, b| b.rect[3].total_cmp(&a.rect[3]));
run.windows(2).all(|w| match w {
[a, b] => (a.rect[1] - b.rect[3]).abs() <= 0.01 * (a.rect[3] - a.rect[1]).min(b.rect[3] - b.rect[1]),
_ => true,
})
}
/// The fragment groups from `i` on while each is drawn right under (or right over) the one
/// before it, in the same direction: a picture drawn row by row, up to a gap or a jump.
fn drawn_in_sequence(ops: &[Op], i: usize, xobjects: &Dict, doc: &Document) -> Vec<Fragment> {
let mut run: Vec<Fragment> = Vec::new();
let mut down: Option<bool> = None;
let mut j = i;
while let Some((next, m, name)) = group_at(ops, j) {
let Some((image, width, height, ..)) = fragment_image(doc, xobjects, &name) else { break };
let [a, _, _, d, e, f] = m.0;
let f2 = Fragment { image, rect: [e, f, e + a, f + d], width, height };
if let Some(prev) = run.last() {
let tol = 0.01 * (prev.rect[3] - prev.rect[1]).min(f2.rect[3] - f2.rect[1]);
let below = (prev.rect[1] - f2.rect[3]).abs() <= tol;
let above = (f2.rect[1] - prev.rect[3]).abs() <= tol;
let same_x = (prev.rect[0] - f2.rect[0]).abs() <= 1e-3 * a.abs().max(1.0) && (prev.rect[2] - f2.rect[2]).abs() <= 1e-3 * a.abs().max(1.0);
match (down, below, above) {
_ if !same_x => break,
(None | Some(true), true, _) => down = Some(true),
(None | Some(false), _, true) => down = Some(false),
_ => break,
}
}
run.push(f2);
j = next;
}
run
}
/// The op index after the first `count` fragment groups from `i`.
fn run_end(ops: &[Op], i: usize, count: usize) -> usize {
let mut j = i;
for _ in 0..count {
match group_at(ops, j) {
Some((next, ..)) => j = next,
None => break,
}
}
j
}
/// The run's pixels as one image, top row first: `(pixels, width, height)`.
fn merge(doc: &Document, run: &[Fragment], n: usize) -> Option<(Vec<u8>, u32, u32)> {
let (top, bottom) = (run.first()?.rect[3], run.last()?.rect[1]);
let width = run.iter().map(|f| f.width).max()?;
// Pixel rows per unit of height: the finest fragment's.
let density = run.iter().map(|f| f.height as f64 / (f.rect[3] - f.rect[1])).fold(0.0, f64::max);
let height = ((top - bottom) * density).round();
if !(height.is_finite() && height >= 1.0) || (width as u64) * (height as u64) > MAX_PIXELS {
return None;
}
let height = height as u32;
let row = width as usize * n;
let mut out = vec![255u8; row * height as usize];
for f in run {
let r0 = (((top - f.rect[3]) * density).round().max(0.0) as u32).min(height);
let r1 = (((top - f.rect[1]) * density).round().max(0.0) as u32).min(height);
if r1 <= r0 {
continue;
}
let obj = doc.get(f.image);
let Object::Stream(s) = &*obj else { return None };
let px = pixels(s, n)?;
let px = if (f.width, f.height) == (width, r1 - r0) { px } else { resample(&px, n, f.width, f.height, width, r1 - r0)? };
out.get_mut(r0 as usize * row..r1 as usize * row)?.copy_from_slice(px.get(..(r1 - r0) as usize * row)?);
}
Some((out, width, height))
}
/// A name for the merged image that `xobjects` doesn't use yet.
fn fresh_name(xobjects: &Dict, k: &mut usize) -> Vec<u8> {
loop {
*k += 1;
let name = format!("PCMerged{k}").into_bytes();
if !xobjects.contains(&name) {
return name;
}
}
}
/// Add `name` → `image` to the page's image resources, wherever they live.
fn add_xobject(doc: &mut Document, page: ObjRef, name: &[u8], image: ObjRef) -> Result<(), OptimizeError> {
let res = doc.get(page).as_dict().and_then(|d| d.get(b"Resources").cloned());
let put = |d: &mut Dict| {
let mut x = d.get(b"XObject").and_then(Object::as_dict).cloned().unwrap_or_default();
x.set(name.to_vec(), Object::Ref(image));
d.set(b"XObject".to_vec(), Object::Dict(x));
};
match res {
Some(Object::Ref(r)) => match doc.get(r).as_dict().and_then(|d| d.reference(b"XObject")) {
Some(x) => doc.update_dict(x, |d| d.set(name.to_vec(), Object::Ref(image)))?,
None => doc.update_dict(r, put)?,
},
Some(Object::Dict(d)) => match d.reference(b"XObject") {
Some(x) => doc.update_dict(x, |d| d.set(name.to_vec(), Object::Ref(image)))?,
None => doc.update_dict(page, |p| {
let mut d = d.clone();
put(&mut d);
p.set(b"Resources".to_vec(), Object::Dict(d));
})?,
},
// Inherited (or none): the page gets its own copy with the new image.
_ => {
let mut d = page_resources(doc, page).unwrap_or_default();
if let Some(x) = d.reference(b"XObject") {
let x = doc.get(x).as_dict().cloned().unwrap_or_default();
d.set(b"XObject".to_vec(), Object::Dict(x));
}
put(&mut d);
doc.update_dict(page, |p| p.set(b"Resources".to_vec(), Object::Dict(d)))?;
}
}
Ok(())
}
/// Merge the fragment runs of one page; returns how many fragments were merged.
fn merge_page(doc: &mut Document, page: ObjRef, report: &mut Report, progress: &mut dyn FnMut(Stage) -> bool) -> Result<usize, OptimizeError> {
let Some(contents) = doc.get(page).as_dict().and_then(|d| d.get(b"Contents").cloned()) else { return Ok(0) };
let parsed = parse(&content_bytes(doc, &contents));
if parsed.skipped > 0 {
return Ok(0);
}
let resources = page_resources(doc, page).unwrap_or_default();
let mut xobjects = resources.get(b"XObject").map(|x| doc.resolve(x)).and_then(|x| x.as_dict().cloned()).unwrap_or_default();
let ops = parsed.ops;
let mut out: Vec<Op> = Vec::with_capacity(ops.len());
let (mut i, mut merged, mut k) = (0, 0, 0);
while i < ops.len() {
// The longest run of fragment groups from here with one colour space.
let mut run = Vec::new();
let mut kind: Option<(Object, usize)> = None;
let mut j = i;
while let Some((next, m, name)) = group_at(&ops, j) {
let Some((image, width, height, cs, n)) = fragment_image(doc, &xobjects, &name) else { break };
let [a, b, c, d, e, f] = m.0;
if b.abs() > 1e-9 || c.abs() > 1e-9 || !(a > 0.0 && d > 0.0) {
break;
}
match &kind {
Some((k_cs, k_n)) if *k_cs != cs || *k_n != n => break,
None => kind = Some((cs, n)),
_ => {}
}
run.push(Fragment { image, rect: [e, f, e + a, f + d], width, height });
j = next;
}
// Not one picture as a whole: the rows drawn one under the other from the start may be.
if run.len() >= MIN_FRAGMENTS && !stacked(&mut run) {
run = drawn_in_sequence(&ops, i, &xobjects, doc);
j = run_end(&ops, i, run.len());
}
let Some((cs, n)) = kind.filter(|_| run.len() >= MIN_FRAGMENTS && stacked(&mut run)) else {
// Not a run: keep this op and look again from the next one.
if let Some(op) = ops.get(i) {
out.push(op.clone());
}
i += 1;
continue;
};
if !progress(Stage::Fragments) {
return Err(OptimizeError::Cancelled);
}
let Some((px, w, h)) = merge(doc, &run, n) else {
out.extend(ops.get(i..j).unwrap_or_default().iter().cloned());
i = j;
continue;
};
let mut dict = Dict::new();
dict.set(b"Type".to_vec(), Object::name("XObject"));
dict.set(b"Subtype".to_vec(), Object::name("Image"));
dict.set(b"Width".to_vec(), Object::Int(i64::from(w)));
dict.set(b"Height".to_vec(), Object::Int(i64::from(h)));
dict.set(b"ColorSpace".to_vec(), cs);
dict.set(b"BitsPerComponent".to_vec(), Object::Int(8));
let image = doc.add(Object::Stream(Stream::flate(dict, &px)));
let name = fresh_name(&xobjects, &mut k);
xobjects.set(name.clone(), Object::Ref(image));
add_xobject(doc, page, &name, image)?;
let (Some(first), Some(last)) = (run.first(), run.last()) else { break };
let [x0, y0, x1, y1] = [first.rect[0], last.rect[1], first.rect[2], first.rect[3]];
let num = Object::Real;
out.push(Op::new("q", Vec::new()));
out.push(Op::new("cm", vec![num(x1 - x0), Object::Int(0), Object::Int(0), num(y1 - y0), num(x0), num(y0)]));
out.push(Op::new("Do", vec![Object::Name(name)]));
out.push(Op::new("Q", Vec::new()));
merged += run.len();
report.merged_images += 1;
i = j;
}
if merged > 0 {
let stream = doc.add(Object::Stream(Stream::flate(Dict::new(), &serialize_ops(&out))));
doc.update_dict(page, |p| p.set(b"Contents".to_vec(), Object::Ref(stream)))?;
}
Ok(merged)
}
pub(crate) fn run(doc: &mut Document, pages: &[ObjRef], report: &mut Report, progress: &mut dyn FnMut(Stage) -> bool) -> Result<(), OptimizeError> {
for p in pages {
report.fragments_merged += merge_page(doc, *p, report, progress)?;
}
Ok(())
}
+12 -5
View File
@@ -11,7 +11,7 @@ use crate::{Compression, ImageSettings, OptimizeError, Report, Settings, Stage};
const MAX_PIXELS: u64 = 80_000_000;
/// The resources of a page, inherited from the page tree if the page has none.
fn page_resources(doc: &Document, page: ObjRef) -> Option<Dict> {
pub(crate) fn page_resources(doc: &Document, page: ObjRef) -> Option<Dict> {
let mut node = Some(page);
for _ in 0..64 {
let d = doc.get(node?).as_dict()?.clone();
@@ -23,7 +23,7 @@ fn page_resources(doc: &Document, page: ObjRef) -> Option<Dict> {
None
}
fn content_bytes(doc: &Document, contents: &Object) -> Vec<u8> {
pub(crate) fn content_bytes(doc: &Document, contents: &Object) -> Vec<u8> {
match &*doc.resolve(contents) {
Object::Stream(s) => s.decoded().unwrap_or_default(),
Object::Array(a) => a
@@ -111,7 +111,7 @@ pub fn effective_resolutions(doc: &Document, pages: &[ObjRef]) -> HashMap<ObjRef
}
/// Colour components of a colour space PdfCraft resamples: gray (1) or RGB (3).
fn components(doc: &Document, cs: &Object) -> Option<usize> {
pub(crate) fn components(doc: &Document, cs: &Object) -> Option<usize> {
let cs = doc.resolve(cs);
match &*cs {
Object::Name(n) => match n.as_slice() {
@@ -179,7 +179,7 @@ fn is_dct(s: &Stream) -> Option<bool> {
}
/// 8-bit samples of an image (`n` components), decoded.
fn pixels(s: &Stream, n: usize) -> Option<Vec<u8>> {
pub(crate) fn pixels(s: &Stream, n: usize) -> Option<Vec<u8>> {
let (w, h) = (s.dict.int(b"Width")? as usize, s.dict.int(b"Height")? as usize);
if is_dct(s)? {
let img = image::load_from_memory_with_format(&s.raw, image::ImageFormat::Jpeg).ok()?;
@@ -192,7 +192,7 @@ fn pixels(s: &Stream, n: usize) -> Option<Vec<u8>> {
}
}
fn resample(px: &[u8], n: usize, w: u32, h: u32, nw: u32, nh: u32) -> Option<Vec<u8>> {
pub(crate) fn resample(px: &[u8], n: usize, w: u32, h: u32, nw: u32, nh: u32) -> Option<Vec<u8>> {
use image::imageops::{FilterType, resize};
Some(match n {
1 => resize(&image::GrayImage::from_raw(w, h, px.to_vec())?, nw, nh, FilterType::CatmullRom).into_raw(),
@@ -208,6 +208,13 @@ fn jpeg(px: &[u8], n: usize, w: u32, h: u32, quality: u8) -> Option<Vec<u8>> {
Some(out)
}
/// Whether [`pixels`] reads `s` as it is drawn: a codec it decodes and no decode array that
/// changes anything. `None` when the colour space isn't grey or RGB.
pub(crate) fn decodable(doc: &Document, s: &Stream) -> Option<bool> {
let n = components(doc, s.dict.get(b"ColorSpace")?)?;
Some(is_dct(s).is_some() && s.dict.get(b"Decode").is_none_or(|d| is_identity_decode(doc, d, n)))
}
/// Whether `decode` is the default decode array for `n` components ([0 1] each), which changes
/// nothing. Scanners write it on page images (#490).
fn is_identity_decode(doc: &Document, decode: &Object, n: usize) -> bool {
+57 -1
View File
@@ -1,6 +1,9 @@
//! The PDF Optimizer (execution plan M11.1; Acrobat: Reduce File Size and Optimize PDF ▸
//! Advanced optimization).
//!
//! - **Merge image fragments:** a page picture some PDF printers draw as thousands of images one
//! pixel row high is put back together into one image first (see [`fragments`]), so the image
//! settings can work on it.
//! - **Images:** each image's effective resolution is measured where pages draw it (the
//! smallest over its uses, through form XObjects); colour and grayscale images above a
//! threshold are resampled (bicubic) to a target resolution and recompressed as JPEG or Flate.
@@ -20,6 +23,7 @@
#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)]
mod audit;
pub mod fragments;
mod images;
mod links;
mod unused;
@@ -45,6 +49,8 @@ pub enum OptimizeError {
pub enum Stage {
/// About to process image `done` (0-based) of `total`; `done == total` once all are done.
Images { done: usize, total: usize },
/// Merging the fragments of a page picture into one image (only when a page has some).
Fragments,
/// Discarding objects, cleaning up links and compressing unencoded streams.
CleanUp,
}
@@ -73,12 +79,18 @@ pub struct ImageSettings {
}
impl ImageSettings {
/// Reduce File Size: bicubic to 150 ppi above 225 ppi, JPEG medium quality.
/// Reduce File Size and the Standard preset: bicubic to 150 ppi above 225 ppi, JPEG medium
/// quality.
pub const REDUCE: ImageSettings = ImageSettings { downsample: true, target_ppi: 150.0, above_ppi: 225.0, compression: Compression::Jpeg(60) };
/// The Mobile preset: bicubic to 96 ppi above 144 ppi, JPEG high quality. Pages read on a
/// screen; text in scans stays legible.
pub const MOBILE: ImageSettings = ImageSettings { downsample: true, target_ppi: 96.0, above_ppi: 144.0, compression: Compression::Jpeg(80) };
}
#[derive(Clone, Debug, PartialEq)]
pub struct Settings {
/// Put page pictures drawn as many stacked fragments back together first ([`fragments`]).
pub merge_fragments: bool,
pub color: ImageSettings,
pub gray: ImageSettings,
pub discard_thumbnails: bool,
@@ -98,6 +110,7 @@ impl Default for Settings {
/// Acrobat's Reduce File Size choices.
fn default() -> Self {
Self {
merge_fragments: true,
color: ImageSettings::REDUCE,
gray: ImageSettings::REDUCE,
discard_thumbnails: true,
@@ -111,9 +124,49 @@ impl Default for Settings {
}
}
/// The optimizer's ready-made settings, as in Acrobat's Settings menu.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Preset {
/// Reduce File Size's choices ([`Settings::default`]).
Standard,
/// Smaller files for reading on a phone or screen: images to 96 ppi above 144, JPEG high.
Mobile,
}
impl Preset {
pub const ALL: [Preset; 2] = [Preset::Standard, Preset::Mobile];
/// The name tools use.
pub fn name(self) -> &'static str {
match self {
Preset::Standard => "standard",
Preset::Mobile => "mobile",
}
}
pub fn from_name(name: &str) -> Option<Preset> {
Preset::ALL.into_iter().find(|p| p.name() == name)
}
pub fn settings(self) -> Settings {
match self {
Preset::Standard => Settings::default(),
Preset::Mobile => Settings { color: ImageSettings::MOBILE, gray: ImageSettings::MOBILE, ..Settings::default() },
}
}
/// The preset `settings` are, if they are one.
pub fn of(settings: &Settings) -> Option<Preset> {
Preset::ALL.into_iter().find(|p| p.settings() == *settings)
}
}
/// What an optimization did.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Report {
/// Images merged from fragments, and how many fragments went into them.
pub merged_images: usize,
pub fragments_merged: usize,
pub images: usize,
pub images_resampled: usize,
pub images_recompressed: usize,
@@ -144,6 +197,9 @@ pub fn optimize(doc: &mut Document, settings: &Settings) -> Result<Report, Optim
pub fn optimize_with_progress(doc: &mut Document, settings: &Settings, progress: &mut dyn FnMut(Stage) -> bool) -> Result<Report, OptimizeError> {
let mut report = Report::default();
let pages = pdfcraft_annot::page_refs(doc).map_err(|_| OptimizeError::NoPages)?;
if settings.merge_fragments {
fragments::run(doc, &pages, &mut report, progress)?;
}
images::run(doc, &pages, settings, &mut report, progress)?;
if !progress(Stage::CleanUp) {
return Err(OptimizeError::Cancelled);
+134
View File
@@ -687,3 +687,137 @@ fn a_second_run_leaves_the_stripped_stream_alone() {
assert_eq!(r.streams_compressed, 0);
assert_eq!(stream(&doc, c), once);
}
/// A grey image `w` px wide and one row high, every pixel `v`; JPEG or Flate.
fn strip(doc: &mut Document, w: u32, v: u8, jpeg: bool) -> ObjRef {
let mut d = Dict::new();
d.set(b"Type".to_vec(), Object::name("XObject"));
d.set(b"Subtype".to_vec(), Object::name("Image"));
d.set(b"Width".to_vec(), Object::Int(w as i64));
d.set(b"Height".to_vec(), Object::Int(1));
d.set(b"BitsPerComponent".to_vec(), Object::Int(8));
d.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
let px = vec![v; w as usize];
let s = if jpeg {
let mut out = Vec::new();
image::codecs::jpeg::JpegEncoder::new_with_quality(&mut out, 95).encode(&px, w, 1, image::ExtendedColorType::L8).unwrap();
d.set(b"Filter".to_vec(), Object::name("DCTDecode"));
Stream::from_raw(d, out)
} else {
Stream::flate(d, &px)
};
doc.add(Object::Stream(s))
}
/// A page drawn the way some PDF printers write a page picture: `rows` images one pixel row
/// high, 1000 units wide, scaled into a 24 pt square (so 600 ppi across at 200 px). Row `r`
/// from the top is grey `r`; `edit` changes the content of each row.
fn printed_page(rows: usize, jpeg: impl Fn(usize) -> bool, edit: impl Fn(usize, String) -> String) -> (Document, Vec<ObjRef>) {
let mut doc = Document::new_empty();
let mut names = Vec::new();
let mut refs = Vec::new();
let mut content = String::from("q 0.024 0 0 0.024 0 0 cm\n");
for r in 0..rows {
let img = strip(&mut doc, 200, r as u8, jpeg(r));
refs.push(img);
let y = (rows - 1 - r) as f64;
content += &edit(r, format!("q 1000 0 0 1 0 {y} cm\n/R{r} Do\nQ\n"));
names.push(format!("R{r}"));
}
content += "Q\n";
let xo: Vec<(&str, ObjRef)> = names.iter().map(String::as_str).zip(refs.iter().copied()).collect();
let p = page(&mut doc, &xo, &content);
// No thumbnail: it would be the old content stream, which still names the strips.
doc.update_dict(p, |d| {
d.remove(b"Thumb");
})
.unwrap();
(doc, refs)
}
/// The images the first page's content stream draws, by name, with their streams.
fn drawn_images(doc: &Document) -> Vec<Stream> {
let pages = pdfcraft_annot::page_refs(doc).unwrap();
let p = doc.get(pages[0]).as_dict().cloned().unwrap();
let res = doc.resolve(p.get(b"Resources").unwrap()).as_dict().cloned().unwrap();
let xo = doc.resolve(res.get(b"XObject").unwrap()).as_dict().cloned().unwrap();
let c = match &*doc.resolve(p.get(b"Contents").unwrap()) {
Object::Stream(s) => s.decoded().unwrap(),
_ => panic!("contents"),
};
pdfcraft_content::parse(&c)
.ops
.iter()
.filter(|o| o.is("Do"))
.map(|o| match &*doc.resolve(xo.get(o.name(0).unwrap()).unwrap()) {
Object::Stream(s) => s.clone(),
_ => panic!("image"),
})
.collect()
}
#[test]
fn a_page_picture_printed_as_one_pixel_strips_is_merged_into_one_image() {
// 120 strips, every other one JPEG and half as wide (as one PDF printer writes them).
let (mut doc, _) = printed_page(120, |r| r % 2 == 1, |_, s| s);
let keep = Settings {
gray: ImageSettings { downsample: false, target_ppi: 150.0, above_ppi: 225.0, compression: Compression::Retain },
..Settings::default()
};
let r = optimize(&mut doc, &keep).unwrap();
assert_eq!((r.merged_images, r.fragments_merged), (1, 120), "{r:?}");
let drawn = drawn_images(&doc);
assert_eq!(drawn.len(), 1, "one image instead of 120");
let img = &drawn[0];
assert_eq!((img.dict.int(b"Width"), img.dict.int(b"Height")), (Some(200), Some(120)), "the finest resolution both ways");
let px = img.decoded().unwrap();
// Each row is its strip's grey, top row first; JPEG rows within a few levels.
for (row, chunk) in px.chunks(200).enumerate() {
assert!(chunk.iter().all(|&v| (v as i32 - row as i32).abs() <= 3), "row {row}: {:?}", &chunk[..4]);
}
assert_eq!(r.unused_xobjects, 120, "the strips are dropped");
}
#[test]
fn the_mobile_preset_makes_a_printed_page_small() {
use pdfcraft_optimize::Preset;
let (doc, _) = printed_page(200, |_| true, |_, s| s);
let bytes = |d: &Document| write_full(d, &SaveOptions::default()).unwrap().len();
let before = bytes(&doc);
let mut mobile = doc.clone();
let r = optimize(&mut mobile, &Preset::Mobile.settings()).unwrap();
assert_eq!((r.merged_images, r.images_resampled), (1, 1), "{r:?}");
let img = &drawn_images(&mobile)[0];
// 24 pt = 1/3 in at 96 ppi.
assert_eq!((img.dict.int(b"Width"), img.dict.int(b"Height")), (Some(32), Some(32)));
assert_eq!(img.dict.name(b"Filter"), Some(&b"DCTDecode"[..]));
assert!(bytes(&mobile) * 10 < before, "{} → {}", before, bytes(&mobile));
assert_eq!(Preset::of(&Preset::Mobile.settings()), Some(Preset::Mobile));
assert_eq!(Preset::from_name("mobile"), Some(Preset::Mobile));
}
#[test]
fn images_that_do_not_tile_a_picture_are_left_alone() {
let unmerged = |doc: &mut Document, settings: &Settings| {
let r = optimize(doc, settings).unwrap();
assert_eq!(r.merged_images, 0, "{r:?}");
};
let all = Settings::default();
// A gap between two rows.
let (mut gap, _) =
printed_page(40, |_| false, |r, s| if r < 20 { s.replace(&format!(" 0 {} cm", 39 - r), &format!(" 0 {} cm", 41 - r)) } else { s });
let r = optimize(&mut gap, &all).unwrap();
assert_eq!((r.merged_images, r.fragments_merged), (2, 40), "one picture each side of the gap: {r:?}");
// Rotated rows.
let (mut turned, _) = printed_page(40, |_| false, |_, s| s.replace("q 1000 0 0 1", "q 0 1 -1000 0"));
unmerged(&mut turned, &all);
// Something else drawn between the rows.
let (mut between, _) = printed_page(40, |_| false, |r, s| if r == 10 { format!("0 0 10 10 re f\n{s}") } else { s });
let r = optimize(&mut between, &all).unwrap();
assert_eq!((r.merged_images, r.fragments_merged), (2, 40), "two runs, each side of the fill: {r:?}");
// Too few rows, and the option off.
let (mut few, _) = printed_page(3, |_| false, |_, s| s);
unmerged(&mut few, &all);
let (mut off, _) = printed_page(40, |_| false, |_, s| s);
unmerged(&mut off, &Settings { merge_fragments: false, ..Settings::default() });
}
+4
View File
@@ -2394,3 +2394,7 @@ animation effects Off معطّلة
Word counting stopped unexpectedly. توقف عد الكلمات بشكل غير متوقع.
Word count… عدد الكلمات…
@id file.word_count عدد الكلمات…
Standard قياسي
Mobile الهاتف المحمول
Merge image fragments دمج أجزاء الصور
Optimizing… merging image fragments جارٍ التحسين… دمج أجزاء الصور
Can't render this file because it contains an unexpected character in line 6 and column 47.
+4
View File
@@ -2336,3 +2336,7 @@ animation effects Off Изключени
Word counting stopped unexpectedly. Преброяването на думи спря неочаквано.
Word count… Брой думи…
@id file.word_count Брой думи…
Standard Стандартни
Mobile Мобилни устройства
Merge image fragments Обединяване на фрагменти от изображения
Optimizing… merging image fragments Оптимизиране… обединяване на фрагменти от изображения
Can't render this file because it contains an unexpected character in line 8 and column 16.
+4
View File
@@ -2447,3 +2447,7 @@ animation effects Off Vypnuto
Word counting stopped unexpectedly. Počítání slov se neočekávaně zastavilo.
Word count… Počet slov…
@id file.word_count Počet slov…
Standard Standardní
Mobile Mobilní zařízení
Merge image fragments Sloučit fragmenty obrázků
Optimizing… merging image fragments Optimalizace… slučování fragmentů obrázků
Can't render this file because it contains an unexpected character in line 6 and column 47.
+4
View File
@@ -2374,3 +2374,7 @@ animation effects Off Aus
Some content could not be classified safely and was kept in the count. Einige Inhalte konnten nicht sicher zugeordnet werden und wurden mitgezählt.
Counting words… {n} pages read Wörter werden gezählt… {n} Seiten gelesen
Word counting stopped unexpectedly. Die Wortzählung wurde unerwartet beendet.
Standard Standard
Mobile Mobilgerät
Merge image fragments Bildfragmente zusammenführen
Optimizing… merging image fragments Optimieren… Bildfragmente werden zusammengeführt
Can't render this file because it contains an unexpected character in line 6 and column 51.
+4
View File
@@ -2383,3 +2383,7 @@ animation effects Off Ανενεργά
Word counting stopped unexpectedly. Η καταμέτρηση λέξεων σταμάτησε απροσδόκητα.
Word count… Καταμέτρηση λέξεων…
@id file.word_count Καταμέτρηση λέξεων…
Standard Τυπικές
Mobile Κινητό
Merge image fragments Συγχώνευση τμημάτων εικόνας
Optimizing… merging image fragments Βελτιστοποίηση… συγχώνευση τμημάτων εικόνας
Can't render this file because it contains an unexpected character in line 469 and column 9.
+4
View File
@@ -2503,3 +2503,7 @@ animation effects Off Desactivados
Word counting stopped unexpectedly. El recuento de palabras se detuvo inesperadamente.
Word count… Recuento de palabras…
@id file.word_count Recuento de palabras…
Standard Estándar
Mobile Móvil
Merge image fragments Combinar fragmentos de imagen
Optimizing… merging image fragments Optimizando… combinando fragmentos de imagen
Can't render this file because it contains an unexpected character in line 6 and column 60.
+4
View File
@@ -2373,3 +2373,7 @@ animation effects Off Désactivés
Word counting stopped unexpectedly. Le comptage des mots s’est arrêté de manière inattendue.
Word count… Nombre de mots…
@id file.word_count Nombre de mots…
Standard Standard
Mobile Mobile
Merge image fragments Fusionner les fragments d’image
Optimizing… merging image fragments Optimisation… fusion des fragments d’image
Can't render this file because it contains an unexpected character in line 6 and column 39.
+4
View File
@@ -2372,3 +2372,7 @@ animation effects Off Ki
Word counting stopped unexpectedly. A szavak számlálása váratlanul leállt.
Word count… Szavak száma…
@id file.word_count Szavak száma…
Standard Normál
Mobile Mobil
Merge image fragments Képtöredékek egyesítése
Optimizing… merging image fragments Optimalizálás… képtöredékek egyesítése
Can't render this file because it contains an unexpected character in line 7 and column 44.
+4
View File
@@ -2382,3 +2382,7 @@ animation effects Off Disattivati
Word counting stopped unexpectedly. Il conteggio delle parole si è interrotto in modo imprevisto.
Word count… Conteggio parole…
@id file.word_count Conteggio parole…
Standard Standard
Mobile Cellulare
Merge image fragments Unisci frammenti di immagine
Optimizing… merging image fragments Ottimizzazione… unione dei frammenti di immagine
Can't render this file because it contains an unexpected character in line 6 and column 47.
+4
View File
@@ -2373,3 +2373,7 @@ animation effects Off オフ
Word counting stopped unexpectedly. 単語の集計が予期せず停止しました。
Word count… 文字数と単語数…
@id file.word_count 文字数と単語数…
Standard 標準
Mobile モバイル
Merge image fragments 画像の断片を結合
Optimizing… merging image fragments 最適化中… 画像の断片を結合しています
Can't render this file because it contains an unexpected character in line 6 and column 47.
+4
View File
@@ -2371,3 +2371,7 @@ animation effects Off Өшірулі
Word counting stopped unexpectedly. Сөздерді санау күтпеген жерден тоқтады.
Word count… Сөздер саны…
@id file.word_count Сөздер саны…
Standard Стандартты
Mobile Мобильді құрылғы
Merge image fragments Сурет фрагменттерін біріктіру
Optimizing… merging image fragments Оңтайландыру… сурет фрагменттерін біріктіру
Can't render this file because it contains an unexpected character in line 816 and column 9.
+4
View File
@@ -2617,3 +2617,7 @@ key Space Tarpas
Word counting stopped unexpectedly. Žodžių skaičiavimas netikėtai sustojo.
Word count… Žodžių skaičius…
@id file.word_count Žodžių skaičius…
Standard Standartinis
Mobile Mobilusis
Merge image fragments Sujungti vaizdo fragmentus
Optimizing… merging image fragments Optimizuojama… jungiami vaizdo fragmentai
Can't render this file because it contains an unexpected character in line 395 and column 9.
+4
View File
@@ -2353,3 +2353,7 @@ animation effects Off Wyłączone
Word counting stopped unexpectedly. Liczenie słów zostało nieoczekiwanie przerwane.
Word count… Liczba słów…
@id file.word_count Liczba słów…
Standard Standardowe
Mobile Urządzenie mobilne
Merge image fragments Scal fragmenty obrazów
Optimizing… merging image fragments Optymalizacja… scalanie fragmentów obrazów
Can't render this file because it contains an unexpected character in line 6 and column 47.
+4
View File
@@ -2387,3 +2387,7 @@ animation effects Off Desativados
Word counting stopped unexpectedly. A contagem de palavras parou inesperadamente.
Word count… Contagem de palavras…
@id file.word_count Contagem de palavras…
Standard Padrão
Mobile Celular
Merge image fragments Mesclar fragmentos de imagem
Optimizing… merging image fragments Otimizando… mesclando fragmentos de imagem
Can't render this file because it contains an unexpected character in line 9 and column 16.
+4
View File
@@ -2286,3 +2286,7 @@ animation effects Off Выкл
Word counting stopped unexpectedly. Подсчёт слов неожиданно остановился.
Word count… Количество слов…
@id file.word_count Количество слов…
Standard Стандартные
Mobile Мобильные устройства
Merge image fragments Объединять фрагменты изображений
Optimizing… merging image fragments Оптимизация… объединение фрагментов изображений
Can't render this file because it contains an unexpected character in line 9 and column 16.
+4
View File
@@ -2378,3 +2378,7 @@ animation effects Off Izklopljeni
Word counting stopped unexpectedly. Štetje besed se je nepričakovano ustavilo.
Word count… Število besed…
@id file.word_count Število besed…
Standard Standardno
Mobile Mobilna naprava
Merge image fragments Združi delce slik
Optimizing… merging image fragments Optimiziranje… združevanje delcev slik
Can't render this file because it contains an unexpected character in line 9 and column 16.
+4
View File
@@ -2286,3 +2286,7 @@ animation effects Off ఆఫ్
Word counting stopped unexpectedly. పదాల లెక్క అనుకోకుండా ఆగిపోయింది.
Word count… పదాల లెక్క…
@id file.word_count పదాల లెక్క…
Standard ప్రామాణికం
Mobile మొబైల్
Merge image fragments చిత్ర శకలాలను విలీనం చేయి
Optimizing… merging image fragments ఆప్టిమైజ్ చేస్తోంది… చిత్ర శకలాలను విలీనం చేస్తోంది
Can't render this file because it contains an unexpected character in line 13 and column 16.
+4
View File
@@ -2369,3 +2369,7 @@ animation effects Off Kapalı
Word counting stopped unexpectedly. Sözcük sayımı beklenmedik şekilde durdu.
Word count… Sözcük sayısı…
@id file.word_count Sözcük sayısı…
Standard Standart
Mobile Mobil
Merge image fragments Görüntü parçalarını birleştir
Optimizing… merging image fragments İyileştiriliyor… görüntü parçaları birleştiriliyor
Can't render this file because it contains an unexpected character in line 815 and column 9.
+4
View File
@@ -2395,3 +2395,7 @@ animation effects Off Вимкнено
Word counting stopped unexpectedly. Підрахунок слів несподівано зупинився.
Word count… Кількість слів…
@id file.word_count Кількість слів…
Standard Стандартні
Mobile Мобільні пристрої
Merge image fragments Об’єднувати фрагменти зображень
Optimizing… merging image fragments Оптимізація… об’єднання фрагментів зображень
Can't render this file because it contains an unexpected character in line 6 and column 47.
+4
View File
@@ -2337,3 +2337,7 @@ animation effects Off 关
Word counting stopped unexpectedly. 字词统计意外停止。
Word count… 字词统计…
@id file.word_count 字词统计…
Standard 标准
Mobile 移动设备
Merge image fragments 合并图像碎片
Optimizing… merging image fragments 正在优化… 正在合并图像碎片
Can't render this file because it contains an unexpected character in line 9 and column 16.
+4
View File
@@ -2327,3 +2327,7 @@ animation effects Off 關
Word counting stopped unexpectedly. 字詞統計意外停止。
Word count… 字詞統計…
@id file.word_count 字詞統計…
Standard 標準
Mobile 行動裝置
Merge image fragments 合併影像片段
Optimizing… merging image fragments 正在最佳化… 正在合併影像片段
Can't render this file because it contains an unexpected character in line 9 and column 16.
+19 -1
View File
@@ -8,7 +8,7 @@
use std::sync::{Arc, Mutex};
use egui::{Align, Layout};
use pdfcraft_engine::optimize::{Compression, ImageSettings, QUALITIES, Settings};
use pdfcraft_engine::optimize::{Compression, ImageSettings, Preset, QUALITIES, Settings};
use pdfcraft_engine::optimizer::{OptimizeStage, Optimized};
use pdfcraft_engine::{DocId, EditError, Hidden, OptimizeReport};
@@ -78,6 +78,7 @@ pub struct OptimizeRun {
fn stage_label(stage: OptimizeStage) -> String {
match stage {
OptimizeStage::Discarding => tl!("Optimizing… removing user data").to_string(),
OptimizeStage::Fragments => tl!("Optimizing… merging image fragments").to_string(),
OptimizeStage::Images { done, total } => crate::i18n::fmt(
tl!("Optimizing… image {d} of {t}"),
&[("d", &(done + 1).min(total.max(1)).to_string()), ("t", &total.max(1).to_string())],
@@ -157,6 +158,22 @@ fn discard_box(ui: &mut egui::Ui, list: &mut Vec<Hidden>, h: Hidden, label: &str
pub(crate) fn body(ui: &mut egui::Ui, d: &mut OptimizeDraft, t: &Tokens) -> (bool, bool) {
ui.label(egui::RichText::new(tl!("PDF Optimizer")).font(theme::semibold(18.0)));
ui.add_space(6.0);
ui.horizontal(|ui| {
ui.label(tl!("Settings"));
let name = |p: Preset| match p {
Preset::Standard => tl!("Standard"),
Preset::Mobile => tl!("Mobile"),
};
let current = Preset::of(&d.settings);
egui::ComboBox::from_id_salt("optimize-preset").selected_text(current.map_or(tl!("Custom"), name)).show_ui(ui, |ui| {
for p in Preset::ALL {
if ui.selectable_label(current == Some(p), name(p)).clicked() {
d.settings = p.settings();
}
}
});
});
ui.add_space(6.0);
ui.horizontal_wrapped(|ui| {
for (tab, label) in [
(OptimizeTab::Images, tl!("Images")),
@@ -174,6 +191,7 @@ pub(crate) fn body(ui: &mut egui::Ui, d: &mut OptimizeDraft, t: &Tokens) -> (boo
let s = &mut d.settings;
match d.tab {
OptimizeTab::Images => {
ui.add(widgets::checkbox(&mut s.merge_fragments, tl!("Merge image fragments")));
image_row(ui, "color", "Color Images", &mut s.color);
image_row(ui, "gray", "Grayscale Images", &mut s.gray);
ui.label(
+28
View File
@@ -212,6 +212,34 @@ fn the_pdf_optimizer_dialog_saves_an_optimized_copy() {
assert_eq!(h.state().dialog, None);
}
#[test]
fn the_pdf_optimizer_dialog_picks_a_preset() {
use egui_kittest::Harness;
use egui_kittest::kittest::Queryable;
use pdfcraft_engine::optimize::Preset;
let mut h = Harness::builder().with_size(egui::vec2(1200.0, 800.0)).build_eframe(|_cc| {
let mut app = PdfCraftApp::new();
app.set_option("language", "en").unwrap();
app.open_bytes("notes.txt", None, b"lorem ipsum".to_vec()).unwrap();
app
});
h.run_steps(4);
assert!(h.state_mut().execute("optimize.advanced"));
h.run_steps(2);
assert_eq!(Preset::of(&h.state().optimize_draft.settings), Some(Preset::Standard));
h.get_by_label("Merge image fragments");
h.get_by_value("Standard").click();
h.run_steps(4);
h.get_by_label("Mobile").click();
h.run_steps(2);
assert_eq!(h.state().optimize_draft.settings, Preset::Mobile.settings(), "Mobile chosen");
// Changing an option makes the settings custom.
h.get_by_label("Merge image fragments").click();
h.run_steps(2);
assert!(!h.state().optimize_draft.settings.merge_fragments);
h.get_by_value("Custom");
}
fn mixed_files(app: &PdfCraftApp) -> Vec<(String, Vec<u8>)> {
let pdf = app.session.create_from_text("a", "from a pdf").unwrap();
vec![
+5 -5
View File
@@ -7225,7 +7225,7 @@ status = "shipped"
commands = ["optimize.reduce"]
tools = ["doc_reduce"]
evidence = ["crates/ui-egui/tests/create.rs::reduce_file_size_writes_a_compact_copy", "crates/ui-egui/tests/create.rs::reduce_file_size_saves_nothing_when_the_file_is_already_small", "crates/engine/src/tests.rs::create_and_reduce", "crates/automation/tests/automation.rs::creating_and_reducing_through_tools", "crates/automation/tests/automation.rs::optimizing_through_tools", "crates/optimize/tests/optimize.rs::an_identity_decode_array_does_not_stop_reduce"]
notes = "images above 225 ppi to 150 ppi JPEG medium, thumbnails dropped, resources merged, object streams; scanner images with the default decode array are included; a copy that isn't smaller is not saved (\"already as small as it can be made\"); runs in the background with a progress bar and Cancel"
notes = "images above 225 ppi to 150 ppi JPEG medium (after merging page pictures printed as one-pixel strips into one image), thumbnails dropped, resources merged, object streams; scanner images with the default decode array are included; a copy that isn't smaller is not saved (\"already as small as it can be made\"); runs in the background with a progress bar and Cancel"
[[feature]]
id = "optimize.compatibility-level"
@@ -7248,8 +7248,8 @@ milestone = "M11"
status = "partial"
commands = ["optimize.advanced"]
tools = ["doc_optimize"]
evidence = ["crates/ui-egui/tests/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy", "crates/automation/tests/automation.rs::optimizing_through_tools", "crates/ui-egui/src/optimize_ui.rs::the_progress_card_shows_the_stage_and_cancels", "crates/engine/src/tests.rs::an_optimize_job_reports_its_stages_and_can_be_cancelled", "crates/optimize/tests/optimize.rs::images_and_forms_nothing_draws_are_dropped", "crates/automation/tests/automation.rs::a_deleted_image_is_not_kept_by_reduce_or_optimize"]
notes = "Images, Discard Objects, Discard User Data, Clean Up (also drops images and forms the pages list but never draw, such as deleted images); runs in the background with a progress bar (per image) and Cancel; no Fonts or Transparency panels, no space audit"
evidence = ["crates/ui-egui/tests/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy", "crates/automation/tests/automation.rs::optimizing_through_tools", "crates/ui-egui/src/optimize_ui.rs::the_progress_card_shows_the_stage_and_cancels", "crates/engine/src/tests.rs::an_optimize_job_reports_its_stages_and_can_be_cancelled", "crates/optimize/tests/optimize.rs::images_and_forms_nothing_draws_are_dropped", "crates/automation/tests/automation.rs::a_deleted_image_is_not_kept_by_reduce_or_optimize", "crates/ui-egui/tests/create.rs::the_pdf_optimizer_dialog_picks_a_preset", "crates/automation/tests/automation.rs::a_page_printed_in_strips_gets_small_through_tools", "crates/optimize/tests/optimize.rs::the_mobile_preset_makes_a_printed_page_small"]
notes = "Settings menu with the Standard and Mobile presets (Custom once an option changes; doc_optimize preset), Images (with Merge image fragments: a page picture drawn as one-pixel strips, as some PDF printers write it, is put back together so it can be resampled), Discard Objects, Discard User Data, Clean Up (also drops images and forms the pages list but never draw, such as deleted images); runs in the background with a progress bar (per image) and Cancel; no Fonts or Transparency panels, no saved custom presets, no space audit"
[[feature]]
id = "optimize.downsample-images"
@@ -7260,8 +7260,8 @@ milestone = "M11"
status = "partial"
commands = ["optimize.advanced"]
tools = ["doc_optimize"]
evidence = ["crates/optimize/tests/optimize.rs::images_are_measured_where_drawn_and_downsampled", "crates/optimize/tests/optimize.rs::settings_choose_what_happens", "crates/automation/tests/automation.rs::optimizing_through_tools"]
notes = "colour and grayscale, bicubic, effective resolution measured where drawn; monochrome not yet"
evidence = ["crates/optimize/tests/optimize.rs::images_are_measured_where_drawn_and_downsampled", "crates/optimize/tests/optimize.rs::settings_choose_what_happens", "crates/automation/tests/automation.rs::optimizing_through_tools", "crates/optimize/tests/optimize.rs::a_page_picture_printed_as_one_pixel_strips_is_merged_into_one_image", "crates/optimize/tests/optimize.rs::images_that_do_not_tile_a_picture_are_left_alone"]
notes = "colour and grayscale, bicubic, effective resolution measured where drawn; image fragments stacked edge to edge are merged first (side-by-side tiles not yet); monochrome not yet"
[[feature]]
id = "optimize.recompress-images"