mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-11-02 14:38:13 -06:00
M2.6: a JPEG pays for being read before its header is
Reading a JPEG's header was free, and it can be most of the file: a mask of 1 MiB of tables that disagreed with its dictionary, shared by 4,000 images, took 101 s with none refused, and that time grows as the square of the file. A JPEG now pays 4 steps a byte before its tables are parsed, on every path, and a mask's header is read once. That is a bound and not a speed-up: the same file of 4,000 images fits the allowance and still takes 97 s, and 12,000 stop after 10,233 in 250 s. A chain that can't run pays nothing for its pixels, and the docs say what past the allowance isn't counted: LZW streams the image pass didn't read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
7bfdfd293a
commit
2fb88feaf7
@@ -1340,7 +1340,7 @@ pub fn tools() -> Vec<ToolDef> {
|
||||
}),
|
||||
&["from"],
|
||||
)),
|
||||
t("doc_reduce", "Reduce file size", "Write a smaller copy of the document to `path` with Acrobat's Reduce File Size choices: colour and grey images above 225 ppi downsampled to 150 ppi and JPEG-compressed (medium quality), black-and-white images averaged down to 300 ppi above 450 ppi and each stored as CCITT Group 4 (or 1-bit Flate, once resampled or packed from more bits) whenever that is smaller, thumbnails dropped, identical fonts and images merged, unused objects dropped, compressed object streams. If the copy wouldn't be smaller, nothing is written and `written` is false. Images past what one run may decode stay as they are, counted in images_refused. The open document is unchanged.")
|
||||
t("doc_reduce", "Reduce file size", "Write a smaller copy of the document to `path` with Acrobat's Reduce File Size choices: colour and grey images above 225 ppi downsampled to 150 ppi and JPEG-compressed (medium quality), black-and-white images averaged down to 300 ppi above 450 ppi and each stored as CCITT Group 4 (or 1-bit Flate, once resampled or packed from more bits) whenever that is smaller, thumbnails dropped, identical fonts and images merged, unused objects dropped, compressed object streams. If the copy wouldn't be smaller, nothing is written and `written` is false. Images past what one run may decode stay as they are, counted in images_refused; streams it only stores with Flate past that stay LZW, uncounted. The open document is unchanged.")
|
||||
.cmd("optimize.reduce")
|
||||
.with(schema(json!({ "doc": doc(), "path": { "type": "string" } }), &["doc", "path"])),
|
||||
t("doc_initial_view", "Initial view", "Read or change how the document opens (Document Properties ▸ Initial View) and its reading options: navigation (page, bookmarks, pages, attachments, layers), layout (default, single, continuous, two_up, two_up_continuous, two_up_cover, two_up_continuous_cover), magnification (default, actual, fit_page, fit_width, fit_height, fit_visible, or a percentage), page, window options (fit_window, center_window, full_screen, display_title), interface options (hide_menubar, hide_toolbar, hide_window_ui), language and binding (left, right). Only the given ones change; returns the result. Undoable.").with(schema(
|
||||
@@ -1442,7 +1442,7 @@ pub fn tools() -> Vec<ToolDef> {
|
||||
.with(schema(json!({ "doc": doc() }), &["doc"])),
|
||||
t("doc_open_revision", "Open a revision", "Open saved revision `revision` (1 = the oldest) of a document as a new, unsaved document, to see the file as it was then.")
|
||||
.with(schema(json!({ "doc": doc(), "revision": { "type": "integer", "minimum": 1 } }), &["doc", "revision"])),
|
||||
t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). mono: { downsample, ppi, above_ppi, compression: ccitt_g4|jbig2|retain } for black-and-white images and image masks (default: averaged down to 300 ppi above 450 ppi, counted in images_resampled; stored losslessly as CCITT Group 4 or JBIG2 (or 1-bit Flate, once resampled or packed from more bits) when that is smaller, Group 3 fax images, JBIG2 images, black-and-white RGB and indexed images using only palette entries 0 and 1 among them, counted in images_recompressed; retain resamples only, a CCITT image staying CCITT, a JBIG2 image JBIG2 and any other becoming Flate). Images past what one run may decode stay as they are, counted in images_refused. Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
|
||||
t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). mono: { downsample, ppi, above_ppi, compression: ccitt_g4|jbig2|retain } for black-and-white images and image masks (default: averaged down to 300 ppi above 450 ppi, counted in images_resampled; stored losslessly as CCITT Group 4 or JBIG2 (or 1-bit Flate, once resampled or packed from more bits) when that is smaller, Group 3 fax images, JBIG2 images, black-and-white RGB and indexed images using only palette entries 0 and 1 among them, counted in images_recompressed; retain resamples only, a CCITT image staying CCITT, a JBIG2 image JBIG2 and any other becoming Flate). Images past what one run may decode stay as they are, counted in images_refused; streams it only stores with Flate past that stay LZW, uncounted. Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.")
|
||||
.cmd("optimize.advanced")
|
||||
.with(schema(
|
||||
json!({
|
||||
|
||||
+42
-11
@@ -28,6 +28,9 @@ pub const MAX_PAGE_FORMS: usize = 50_000;
|
||||
/// The steps a sample of a JPEG costs: a few bytes can hold a page of one grey, and decoding,
|
||||
/// resampling and storing it took about 20 to 45 ns a sample.
|
||||
const JPEG_WORK: u64 = 2;
|
||||
/// The steps a byte of a JPEG costs to read: its tables took 24 ns a byte before a size was known,
|
||||
/// and images can share one, so reading it again for each has to count.
|
||||
const JPEG_READ_WORK: u64 = 4;
|
||||
|
||||
/// A stream decode within a byte bound, with the bytes its filter stages decoded: the data or
|
||||
/// the error, each with that count.
|
||||
@@ -77,7 +80,9 @@ impl ImageBudget {
|
||||
/// with its `/DecodeParms` given by reference read too ([`Document::with_direct_parms`]), for an
|
||||
/// image that has no fax or JBIG2 decoder to go through.
|
||||
pub fn decode_filters(&mut self, doc: &Document, s: &Stream, max: usize) -> Result<Vec<u8>, CosError> {
|
||||
self.pay_size(s)?;
|
||||
if s.filters().iter().all(|(f, _)| f.is_decodable()) {
|
||||
self.pay_size(s)?;
|
||||
}
|
||||
let decoded = doc.with_direct_parms(s).decoded_spending(max, &mut self.work);
|
||||
self.count_refused(s, &decoded);
|
||||
decoded
|
||||
@@ -109,9 +114,11 @@ impl ImageBudget {
|
||||
paid
|
||||
}
|
||||
|
||||
/// Pays for decoding a JPEG of `samples`, as [`ImageBudget::pay_pixels`] does.
|
||||
pub fn pay_jpeg(&mut self, samples: u64) -> bool {
|
||||
self.pay_pixels(samples.saturating_mul(JPEG_WORK))
|
||||
/// Pays for reading JPEG `data` and decoding its `samples`, as [`ImageBudget::pay_pixels`] does.
|
||||
/// A caller pays before its tables are parsed, which can be most of the file.
|
||||
pub fn pay_jpeg(&mut self, samples: u64, data: &[u8]) -> bool {
|
||||
let reading = (data.len() as u64).saturating_mul(JPEG_READ_WORK);
|
||||
self.pay_pixels(samples.saturating_mul(JPEG_WORK).saturating_add(reading))
|
||||
}
|
||||
|
||||
/// Pays for the pixels of image `s`, unless it is JBIG2, whose decoder pays for its page, or
|
||||
@@ -143,10 +150,15 @@ impl ImageBudget {
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `s` takes only filters and decoders this charges.
|
||||
/// Whether `s` can be decoded here and so charged: general filters, a fax or JBIG2 decoder last at
|
||||
/// most. Those decoders anywhere else can't run.
|
||||
fn charged(s: &Stream) -> bool {
|
||||
use pdfcraft_filters::Filter;
|
||||
s.readable_filters().is_ok_and(|chain| chain.iter().all(|(f, _)| f.is_decodable() || matches!(f, Filter::Jbig2 | Filter::CcittFax)))
|
||||
s.readable_filters().is_ok_and(|chain| {
|
||||
chain.split_last().is_none_or(|((last, _), before)| {
|
||||
(last.is_decodable() || matches!(last, Filter::Jbig2 | Filter::CcittFax)) && before.iter().all(|(f, _)| f.is_decodable())
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/// What one operation may still decode and enter across all the streams it reads.
|
||||
@@ -507,11 +519,11 @@ mod image_tests {
|
||||
let mut budget = ImageBudget::with_work(u64::MAX);
|
||||
budget.decode(&doc, &Stream::from_raw(odd, vec![1, 2]), 1 << 24).unwrap();
|
||||
assert_eq!(u64::MAX - budget.work_left(), 2);
|
||||
// A JPEG pays twice over for each sample, once decoded.
|
||||
let mut budget = ImageBudget::with_work(20);
|
||||
assert!(budget.pay_jpeg(9));
|
||||
assert_eq!((budget.work_left(), budget.refused()), (2, 0));
|
||||
assert!(!budget.pay_jpeg(2));
|
||||
// A JPEG pays twice over for each sample, and four times for each byte it is read from.
|
||||
let mut budget = ImageBudget::with_work(30);
|
||||
assert!(budget.pay_jpeg(9, b"ab"));
|
||||
assert_eq!((budget.work_left(), budget.refused()), (4, 0));
|
||||
assert!(!budget.pay_jpeg(0, b"ab"));
|
||||
assert_eq!((budget.work_left(), budget.refused()), (0, 1));
|
||||
}
|
||||
|
||||
@@ -567,6 +579,25 @@ mod image_tests {
|
||||
let mut budget = ImageBudget::with_work(u64::MAX);
|
||||
assert!(budget.decode_filters(&doc, &jbig2(None), 1 << 20).is_err());
|
||||
assert_eq!(budget.work_left(), u64::MAX, "no decoder here, so nothing was read");
|
||||
// Nor for a fax image, nor for a decoder that isn't last, however large they say they are.
|
||||
for chain in [vec!["CCITTFaxDecode"], vec!["CCITTFaxDecode", "FlateDecode"], vec!["JBIG2Decode", "FlateDecode"]] {
|
||||
let mut dict = Dict::new();
|
||||
dict.set(b"Width".to_vec(), Object::Int(1000));
|
||||
dict.set(b"Height".to_vec(), Object::Int(1000));
|
||||
dict.set(b"Filter".to_vec(), Object::Array(chain.iter().map(|f| Object::name(f)).collect()));
|
||||
let image = Stream::from_raw(dict, b"data".to_vec());
|
||||
let mut budget = ImageBudget::with_work(u64::MAX);
|
||||
assert!(budget.decode_filters(&doc, &image, 1 << 20).is_err());
|
||||
assert_eq!(budget.work_left(), u64::MAX, "{chain:?}");
|
||||
}
|
||||
// Through `decode` a decoder that isn't last pays nothing either; one that is does.
|
||||
let mut dict = Dict::new();
|
||||
dict.set(b"Width".to_vec(), Object::Int(1000));
|
||||
dict.set(b"Height".to_vec(), Object::Int(1000));
|
||||
dict.set(b"Filter".to_vec(), Object::Array(vec![Object::name("JBIG2Decode"), Object::name("FlateDecode")]));
|
||||
let mut budget = ImageBudget::with_work(u64::MAX);
|
||||
assert!(budget.decode(&doc, &Stream::from_raw(dict, b"data".to_vec()), 1 << 20).is_err());
|
||||
assert_eq!(budget.work_left(), u64::MAX);
|
||||
}
|
||||
|
||||
/// The images of a file share its globals, so an image refused doesn't read them: this one's
|
||||
|
||||
@@ -33,7 +33,7 @@ whose JBIG2 page isn't its size (PdfCraft's renderer draws the whole page into t
|
||||
or whose globals can't be read or decode to more than 32 MiB (they are decoded before the page's size is known, so they have a
|
||||
limit of their own), is skipped, and such a mask is left out and the image exported opaque, as JPX and CCITT masks are.
|
||||
Past what one export may decode, which its images share, filters, decoders and pixels alike (`ImageBudget`; a JPEG when it has to be
|
||||
decoded, as a mask or to apply one, and not when it is written as it is), an image is skipped, and so is one whose mask is past it, which would otherwise come out showing what the mask hides. An export also stops at 2 GiB of files held, or twice the document if that
|
||||
decoded, as a mask, to apply one, or to undo a CMYK file's inversion, and not when it is written as it is), an image is skipped, and so is one whose mask is past it, which would otherwise come out showing what the mask hides. An export also stops at 2 GiB of files held, or twice the document if that
|
||||
is more, since they stay in memory until it ends.
|
||||
Indexed Decode values select palette entries after rounding and clipping. Color-key masks compare the original samples.
|
||||
Missing, null or malformed Decode arrays keep the default mapping.
|
||||
|
||||
@@ -185,7 +185,7 @@ fn image(doc: &Document, s: &Stream, budget: &mut ImageBudget) -> Result<(&'stat
|
||||
}
|
||||
let mut space = if mask { Space::Gray } else { space(doc, s.dict.get(b"ColorSpace"))? };
|
||||
let (samples, bpc) = if jpeg {
|
||||
if !budget.pay_jpeg((w as u64).saturating_mul(h as u64).saturating_mul(space.components() as u64)) {
|
||||
if !budget.pay_jpeg((w as u64).saturating_mul(h as u64).saturating_mul(space.components() as u64), &s.raw) {
|
||||
return Err(FilterError::Spent.to_string());
|
||||
}
|
||||
// Grey, RGB or CMYK samples (YCCK is converted); a JPEG that doesn't decode, or an
|
||||
@@ -545,7 +545,7 @@ fn gray_plane(doc: &Document, m: &Stream, w: usize, h: usize, budget: &mut Image
|
||||
let mut px = match f.last().map(Vec::as_slice) {
|
||||
Some(b"DCTDecode" | b"DCT") if f.len() == 1 => {
|
||||
// Past the allowance the image goes too: `image` sees this one refused.
|
||||
if jpeg_size(&m.raw)? != (mw, mh) || !budget.pay_jpeg((mw as u64).saturating_mul(mh as u64)) {
|
||||
if !budget.pay_jpeg((mw as u64).saturating_mul(mh as u64), &m.raw) {
|
||||
return None;
|
||||
}
|
||||
jpeg_of_size(&m.raw, mw, mh)?.into_luma8().into_raw()
|
||||
|
||||
@@ -2312,8 +2312,8 @@ fn flate_images_past_the_exports_allowance_are_left_out_with_the_reason() {
|
||||
assert_eq!(budget.refused(), 1);
|
||||
}
|
||||
|
||||
/// A JPEG is written as it is, at no cost, unless a mask means decoding it: that pays for its
|
||||
/// samples, and past the allowance the image goes, with the reason, as it would come out without
|
||||
/// A JPEG is written as it is, at no cost, unless something means decoding it, here a mask: that
|
||||
/// pays for its samples, and past the allowance the image goes, with the reason, as it would come out without
|
||||
/// its transparency otherwise.
|
||||
#[test]
|
||||
fn a_jpeg_that_has_to_be_decoded_shares_the_allowance() {
|
||||
@@ -2341,6 +2341,7 @@ fn a_jpeg_that_has_to_be_decoded_shares_the_allowance() {
|
||||
mask.set(b"BitsPerComponent".to_vec(), Object::Int(8));
|
||||
mask.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
|
||||
let mask = doc.add(Object::Stream(Stream::from_raw(mask, vec![0x80; 16])));
|
||||
let doc_jpeg_len = jpeg.len();
|
||||
let masked = doc.add(Object::Stream(Stream::from_raw(image_dict(Some((b"SMask", Object::Ref(mask)))), jpeg.clone())));
|
||||
let plain = doc.add(Object::Stream(Stream::from_raw(image_dict(None), jpeg)));
|
||||
|
||||
@@ -2350,7 +2351,8 @@ fn a_jpeg_that_has_to_be_decoded_shares_the_allowance() {
|
||||
let mut budget = ImageBudget::with_work(u64::MAX);
|
||||
assert!(crate::extract::image_file_within(&doc, masked, &mut budget).is_ok());
|
||||
let cost = u64::MAX - budget.work_left();
|
||||
assert_eq!(cost, 16 + 16 + 2 * 4 * 4 * 3, "the mask's bytes and pixels, and the JPEG's samples twice");
|
||||
let read = 4 * doc_jpeg_len as u64;
|
||||
assert_eq!(cost, 16 + 16 + 2 * 4 * 4 * 3 + read, "the mask's bytes and pixels, the JPEG's samples twice and its reading");
|
||||
let mut budget = ImageBudget::with_work(cost - 1);
|
||||
let refused = crate::extract::image_file_within(&doc, masked, &mut budget).unwrap_err();
|
||||
assert!(refused.contains("more image decoding than"), "{refused}");
|
||||
@@ -2372,6 +2374,7 @@ fn a_jpeg_mask_pays_for_its_samples() {
|
||||
mask.set(b"ColorSpace".to_vec(), Object::name("DeviceGray"));
|
||||
mask.set(b"Filter".to_vec(), Object::name("DCTDecode"));
|
||||
let mut doc = Document::new_empty();
|
||||
let read = 4 * jpeg.len() as u64;
|
||||
let mask = doc.add(Object::Stream(Stream::from_raw(mask, jpeg)));
|
||||
let mut image = Dict::new();
|
||||
image.set(b"Subtype".to_vec(), Object::name("Image"));
|
||||
@@ -2383,9 +2386,9 @@ fn a_jpeg_mask_pays_for_its_samples() {
|
||||
let image = doc.add(Object::Stream(Stream::from_raw(image, vec![0x80])));
|
||||
let mut budget = ImageBudget::with_work(u64::MAX);
|
||||
assert!(crate::extract::image_file_within(&doc, image, &mut budget).is_ok());
|
||||
assert_eq!(u64::MAX - budget.work_left(), 2 * 4 * 4 + 2, "its samples twice, and the image's byte and pixel");
|
||||
assert_eq!(u64::MAX - budget.work_left(), 2 * 4 * 4 + read + 2, "its samples twice, its reading, and the image's byte and pixel");
|
||||
// Just short of what the mask costs: it isn't decoded, and takes the image with it.
|
||||
let mut budget = ImageBudget::with_work(2 * 4 * 4 - 1);
|
||||
let mut budget = ImageBudget::with_work(2 * 4 * 4 + read - 1);
|
||||
let refused = crate::extract::image_file_within(&doc, image, &mut budget).unwrap_err();
|
||||
assert!(refused.contains("more image decoding than"), "{refused}");
|
||||
assert_eq!((budget.work_left(), budget.refused()), (0, 1));
|
||||
@@ -2433,6 +2436,14 @@ fn a_jpeg_larger_than_its_dictionary_says_isnt_decoded() {
|
||||
image.set(b"SMask".to_vec(), Object::Ref(mask));
|
||||
let image = doc.add(Object::Stream(Stream::from_raw(image, vec![0x80])));
|
||||
assert_eq!(decodes(&doc, image), 0);
|
||||
// Reading the header to find that out is paid for: a mask can be mostly tables, shared by
|
||||
// every image of a file, and with nothing left it isn't read.
|
||||
let mut budget = ImageBudget::with_work(u64::MAX);
|
||||
assert!(crate::extract::image_file_within(&doc, image, &mut budget).is_ok());
|
||||
assert_eq!(u64::MAX - budget.work_left(), 2 + 4 * jpeg.len() as u64 + 2, "its one declared sample twice, its bytes, the image's byte and pixel");
|
||||
let mut budget = ImageBudget::with_work(0);
|
||||
assert!(crate::extract::image_file_within(&doc, image, &mut budget).is_err());
|
||||
assert_eq!(budget.refused(), 1, "the image goes, with its mask unread");
|
||||
}
|
||||
|
||||
/// A mask past the allowance can't be left out as an unreadable one is, or the image comes out
|
||||
|
||||
@@ -105,7 +105,7 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going
|
||||
pictures too: about
|
||||
1,000 scanned pages' worth, plus more for each byte of the file, enough to decode a stream
|
||||
about 8 times at Flate's best. Images past it stay as they are, and `Report::images_refused`
|
||||
counts them.
|
||||
counts those the image step read. Streams it only stores with Flate (below) aren't counted.
|
||||
- **Discard objects:** thumbnails, alternate images, document tags, print settings.
|
||||
- **Clean up:** Flate for streams with no filter; for streams with an outer ASCII85 or ASCIIHex
|
||||
layer, that layer is removed when the result is smaller (the inner filters keep their encoded
|
||||
|
||||
@@ -490,7 +490,7 @@ pub(crate) fn pixels(doc: &Document, s: &Stream, n: usize, budget: &mut ImageBud
|
||||
// never get here: the colour space check excludes them. The JPEG's own size must be the
|
||||
// dictionary's: `jpeg_samples` refuses another from its header, before any pixels.
|
||||
let ycc = n == 3 && ycc_encoded(doc, s)?;
|
||||
if !budget.pay_jpeg(u64::try_from(bytes).ok()?) {
|
||||
if !budget.pay_jpeg(u64::try_from(bytes).ok()?, &s.raw) {
|
||||
return None;
|
||||
}
|
||||
let v = jpeg_samples(&s.raw, usize::try_from(w).ok()?, usize::try_from(h).ok()?, n, ycc)?;
|
||||
@@ -1559,10 +1559,11 @@ mod dimension_tests {
|
||||
dict.set(b"ColorSpace".to_vec(), Object::name("DeviceRGB"));
|
||||
let jpeg = image_stream(&dict, &[120; 16 * 8 * 3], 3, 16, 8, Compression::Jpeg(60), false).expect("a JPEG");
|
||||
let doc = Document::new_empty();
|
||||
let mut budget = ImageBudget::with_work(2 * 16 * 8 * 3);
|
||||
let cost = 2 * 16 * 8 * 3 + 4 * jpeg.raw.len() as u64;
|
||||
let mut budget = ImageBudget::with_work(cost);
|
||||
assert_eq!(pixels(&doc, &jpeg, 3, &mut budget).map(|p| p.len()), Some(16 * 8 * 3));
|
||||
assert_eq!((budget.work_left(), budget.refused()), (0, 0), "exactly that");
|
||||
let mut budget = ImageBudget::with_work(2 * 16 * 8 * 3 - 1);
|
||||
let mut budget = ImageBudget::with_work(cost - 1);
|
||||
assert!(pixels(&doc, &jpeg, 3, &mut budget).is_none());
|
||||
assert_eq!((budget.work_left(), budget.refused()), (0, 1));
|
||||
}
|
||||
|
||||
@@ -7189,7 +7189,7 @@ status = "shipped"
|
||||
commands = ["export.all_images"]
|
||||
tools = ["doc_export_all_images"]
|
||||
evidence = ["crates/create/src/tests.rs::ccitt_fax_images_export_as_their_picture", "crates/create/src/tests.rs::group_3_ccitt_masks_and_parameters_by_reference_export_as_their_picture", "crates/create/src/tests.rs::images_export_as_jpeg_unchanged_and_others_as_png", "crates/create/src/tests.rs::indexed_and_one_bit_images_decode", "crates/create/src/tests.rs::jpeg_images_with_a_soft_mask_export_as_png_with_alpha", "crates/create/src/tests.rs::soft_masks_of_another_size_and_stencil_and_colour_key_masks_become_alpha", "crates/create/src/tests.rs::soft_mask_decode_ranges_become_exported_alpha", "crates/create/src/tests.rs::soft_mask_decode_maps_packed_and_jpeg_masks_before_resizing", "crates/automation/tests/it/automation.rs::exporting_all_images_through_tools", "crates/ui-egui/tests/it/export.rs::export_dialogs_write_images_and_text", "crates/create/src/tests.rs::lossless_decode_maps_each_component_before_colour_conversion", "crates/create/src/tests.rs::lossless_indexed_decode_rounds_in_index_units_then_clamps", "crates/create/src/tests.rs::jbig2_images_export_as_their_picture", "crates/create/src/tests.rs::jbig2_globals_are_decoded_within_a_limit_of_their_own", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/create/src/tests.rs::symbol_coded_jbig2_images_export_as_their_picture", "crates/create/src/tests.rs::jbig2_images_behind_another_filter_export_as_their_picture"]
|
||||
notes = "each image once (page resources and nested forms), up to what one export may decode of JBIG2 and hold of files (2 GiB or twice the document), images past that left out with the reason; JPEG unchanged unless it has transparency, others PNG (gray/RGB/CMYK/ICC/Indexed, 1–16 bpc, stencil masks); transparency as PNG alpha: /SMask (scaled to the image size, /Matte un-blended), stencil /Mask and colour-key /Mask, JPEG images included (#645); JPX masks are left out (image exported opaque); JPX images and separations reported as skipped; minimum size filter. Lossless images apply per-component Decode; Indexed values round and clip before palette lookup. Missing, null or malformed Decode arrays keep the default mapping. Soft-mask Decode intervals preserve opacity before resizing, including indirect arrays and endpoints. Still limited: 16-bit samples use high bytes; general JPEG image Decode, ICC transforms, and calibrated colors remain unsupported. CCITT fax images (Group 3 and 4, with their /DecodeParms, behind other filters too) export as their picture, masks included; byte-aligned Group 3 is read by the end-of-line codes when the data starts with one and on bytes otherwise, and one cut short is reported. Still limited: byte-aligned Group 3 with codes but none before the first row (not as T.4 sends them) is reported, or at /K 0 may export wrong. JBIG2 images (with the /JBIG2Globals of their own /DecodeParms entry, within 32 MiB, behind other filters too) export as their picture, masks included; the JBIG2 page has to be the image's size (the renderer draws the whole page into the image)."
|
||||
notes = "each image once (page resources and nested forms), up to what one export may decode and hold of files (2 GiB or twice the document), images past that left out with the reason; JPEG unchanged unless it has transparency, others PNG (gray/RGB/CMYK/ICC/Indexed, 1–16 bpc, stencil masks); transparency as PNG alpha: /SMask (scaled to the image size, /Matte un-blended), stencil /Mask and colour-key /Mask, JPEG images included (#645); JPX masks are left out (image exported opaque); JPX images and separations reported as skipped; minimum size filter. Lossless images apply per-component Decode; Indexed values round and clip before palette lookup. Missing, null or malformed Decode arrays keep the default mapping. Soft-mask Decode intervals preserve opacity before resizing, including indirect arrays and endpoints. Still limited: 16-bit samples use high bytes; general JPEG image Decode, ICC transforms, and calibrated colors remain unsupported. CCITT fax images (Group 3 and 4, with their /DecodeParms, behind other filters too) export as their picture, masks included; byte-aligned Group 3 is read by the end-of-line codes when the data starts with one and on bytes otherwise, and one cut short is reported. Still limited: byte-aligned Group 3 with codes but none before the first row (not as T.4 sends them) is reported, or at /K 0 may export wrong. JBIG2 images (with the /JBIG2Globals of their own /DecodeParms entry, within 32 MiB, behind other filters too) export as their picture, masks included; the JBIG2 page has to be the image's size (the renderer draws the whole page into the image)."
|
||||
|
||||
[[feature]]
|
||||
id = "create.export-html"
|
||||
@@ -7409,7 +7409,7 @@ status = "partial"
|
||||
commands = ["optimize.advanced", "optimize.reduce"]
|
||||
tools = ["doc_optimize", "doc_reduce"]
|
||||
evidence = ["crates/optimize/src/images.rs::flate_images_past_the_runs_allowance_stay_as_they_are", "crates/optimize/tests/optimize.rs::eight_bit_black_and_white_images_become_one_bit", "crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/optimize/tests/optimize.rs::group_3_fax_images_become_group_4", "crates/optimize/tests/optimize.rs::rgb_and_two_colour_indexed_black_and_white_images_become_one_bit", "crates/optimize/tests/optimize.rs::black_and_white_rgb_used_beyond_the_page_content_read_stays_rgb", "crates/optimize/tests/optimize.rs::black_and_white_rgb_drawn_only_where_the_optimizer_reads_becomes_grey", "crates/optimize/tests/optimize.rs::black_and_white_rgb_in_resources_its_page_tree_and_a_form_share_becomes_grey", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy", "crates/filters/src/mq.rs::the_annex_h_test_sequence_encodes_to_its_bytes", "crates/filters/src/jbig2.rs::bitmaps_round_trip", "crates/filters/src/jbig2.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/optimize/tests/optimize.rs::black_and_white_images_become_jbig2", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/filters/src/jbig2/symbols.rs::repeated_marks_are_stored_once", "crates/filters/src/jbig2/symbols.rs::scattered_marks_round_trip", "crates/optimize/tests/optimize.rs::rendered_text_becomes_jbig2_symbols", "crates/optimize/tests/optimize.rs::jbig2_scans_are_averaged_and_stored_again", "crates/optimize/src/images.rs::jbig2_images_past_the_runs_allowance_stay_as_they_are"]
|
||||
notes = "All of a run's images share one allowance of decoding, filters, decoders and pixels alike (about 1,000 scanned pages' worth, more for a longer file); those past it stay as they are and are counted in the report, which doc_optimize returns and the panel doesn't show yet. CCITT Group 4, lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size does it too; doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Packed to one bit first, then stored the same way (colour-key masks aside): eight-bit grey of only black and white, eight-bit DeviceRGB of only black and white (as DeviceGray; calibrated RGB, RGB with a profile, RGB drawn or listed where /DefaultGray or /DefaultRGB is in force or referred to from anywhere but the page and form resources read (an annotation's appearance or a pattern drawing it, directly or through resources or a form shared with a page; another image's /Alternates), and RGB whose soft mask has a /Matte aside), and an indexed image of 2, 4 or 8 bits that uses only palette entries 0 and 1 (keeping its palette), unless the grey or colour settings make it smaller still, as they can a dithered scan downsampled to JPEG, its soft mask counted. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. Group 3 fax images are stored again as Group 4 when that is smaller (damaged Group 3 with end-of-line codes as the decoder recovers it, as it reports no damage there; byte-aligned Group 3 with end-of-line codes but none before the first row, not as T.4 sends them, as PdfCraft draws it, which at /K 0 may be other rows than were coded). CCITT fax images are decoded to be downsampled (optimize.downsample-images), and a resampled image is stored as Group 4 or 1-bit Flate, whichever is smaller. Lossless JBIG2 with bounded exact-copy symbol coding is available too, with image export; lossy JBIG2 remains unsupported. JBIG2 images are read too: averaged like any (under Retain stored as JBIG2 again), and, not resampled, stored again as the compression asks when that is smaller (as JBIG2 symbols where its own coding is a generic region, say); a symbol-coded scan, averaged, loses its exact repeats and is often larger, and then stays as it is; one whose JBIG2 page isn't its size is left alone. Not yet: inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples."
|
||||
notes = "All of a run's images share one allowance of decoding, filters, decoders and pixels alike (about 1,000 scanned pages' worth, more for a longer file); those past it stay as they are, those the image pass read being counted in the report, which doc_optimize and doc_reduce return and the panel doesn't show yet. CCITT Group 4, lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (on by default, so Reduce File Size does it too; doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Packed to one bit first, then stored the same way (colour-key masks aside): eight-bit grey of only black and white, eight-bit DeviceRGB of only black and white (as DeviceGray; calibrated RGB, RGB with a profile, RGB drawn or listed where /DefaultGray or /DefaultRGB is in force or referred to from anywhere but the page and form resources read (an annotation's appearance or a pattern drawing it, directly or through resources or a form shared with a page; another image's /Alternates), and RGB whose soft mask has a /Matte aside), and an indexed image of 2, 4 or 8 bits that uses only palette entries 0 and 1 (keeping its palette), unless the grey or colour settings make it smaller still, as they can a dithered scan downsampled to JPEG, its soft mask counted. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. Group 3 fax images are stored again as Group 4 when that is smaller (damaged Group 3 with end-of-line codes as the decoder recovers it, as it reports no damage there; byte-aligned Group 3 with end-of-line codes but none before the first row, not as T.4 sends them, as PdfCraft draws it, which at /K 0 may be other rows than were coded). CCITT fax images are decoded to be downsampled (optimize.downsample-images), and a resampled image is stored as Group 4 or 1-bit Flate, whichever is smaller. Lossless JBIG2 with bounded exact-copy symbol coding is available too, with image export; lossy JBIG2 remains unsupported. JBIG2 images are read too: averaged like any (under Retain stored as JBIG2 again), and, not resampled, stored again as the compression asks when that is smaller (as JBIG2 symbols where its own coding is a generic region, say); a symbol-coded scan, averaged, loses its exact repeats and is often larger, and then stays as it is; one whose JBIG2 page isn't its size is left alone. Not yet: inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples."
|
||||
|
||||
[[feature]]
|
||||
id = "optimize.only-if-smaller"
|
||||
|
||||
Reference in New Issue
Block a user