M11: JBIG2 symbol coding: tests for padding, the 32-bit range and the mark rules

Review found behaviour the tests let break unnoticed (each was a
surviving mutant, and each new test fails on it):
- black padding bits: a run reaching past the row would stop symbol
  coding on many scans;
- Annex A's 32-bit range and PREV at exactly 256, where a decision would
  go uncoded: coordinates past 4,435 on a 12,000-pixel-wide and a
  10,050-row page, and the contexts of 4436's coding;
- marks joined at a corner are one mark; 256 pixels a side is a symbol,
  257 is left to the generic region;
- the run and row caps, at and past them.

The roadmap's status line for this change now comes first, and the
"about a third smaller than Group 4 on scans" figure is replaced by what
was measured: 27-33% on the same pages with scan-like noise added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
This commit is contained in:
Grzegorz Martens
2026-10-29 22:37:07 +01:00
co-authored by Claude Opus 5.5
parent 9edd08ce07
commit 0d40594cf9
5 changed files with 100 additions and 10 deletions
+1 -1
View File
@@ -7385,7 +7385,7 @@ status = "partial"
commands = ["optimize.advanced", "optimize.reduce"]
tools = ["doc_optimize", "doc_reduce"]
evidence = ["crates/filters/src/ccitt.rs::bitmaps_round_trip", "crates/filters/src/ccitt.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/ccitt.rs::what_it_cannot_encode_is_an_error", "crates/filters/src/mq.rs::the_annex_h_test_sequence_encodes_to_its_bytes", "crates/filters/src/jbig2.rs::bitmaps_round_trip", "crates/filters/src/jbig2.rs::a_page_of_text_like_marks_round_trips_and_shrinks", "crates/filters/src/jbig2/symbols.rs::repeated_marks_are_stored_once", "crates/filters/src/jbig2/symbols.rs::scattered_marks_round_trip", "crates/optimize/tests/optimize.rs::black_and_white_images_become_jbig2", "crates/optimize/tests/optimize.rs::rendered_text_becomes_jbig2_symbols", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_jbig2", "crates/optimize/tests/optimize.rs::black_and_white_images_become_ccitt_group_4", "crates/automation/tests/it/automation.rs::doc_optimize_stores_black_and_white_images_as_ccitt_group_4", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy"]
notes = "CCITT Group 4 (the default, so Reduce File Size does it too) or JBIG2, both lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. JBIG2 stores each distinct mark once in a symbol dictionary and places its exact copies with a text region, larger and lone marks in a generic region (template 0, typical prediction), or the whole image as a generic region, whichever is smaller: on rendered 300 dpi text 28% of the generic region's size and 17% of Group 4's; on scans, whose copies of a letter are seldom identical, the generic region, about a third smaller than Group 4. A document below PDF 1.4 gets /Version 1.4. Not yet: JBIG2 refinement coding of near-copies (where lossless gains on scans would come from), a symbol dictionary shared between images (/JBIG2Globals), JBIG2's lossy mode, monochrome downsampling, inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples (JBIG2 takes rows of up to 1,048,576). Redaction doesn't decode JBIG2 or Group 4 yet: a mark over part of such an image removes the whole image (nothing leaks). Export reads JBIG2, but nothing that exports an image reads Group 4 yet: Export ▸ Images and Save Image As refuse it, Copy Image fails, and Word, PowerPoint and HTML export leave the picture out."
notes = "CCITT Group 4 (the default, so Reduce File Size does it too) or JBIG2, both lossless: PDF Optimizer ▸ Images ▸ Monochrome Images (doc_optimize mono) stores a 1-bit image of one colour component (grey, indexed, a separation) or an image mask, kept with no filter or general-purpose ones, as Group 4 when that is smaller as stored; the samples decode to the same bits, so colour space, decode array and masks stay. Group 4 wins on scans; on clean computer-drawn 1-bit art Flate can be smaller, and the image then stays. JBIG2 stores each distinct mark once in a symbol dictionary and places its exact copies with a text region, larger and lone marks in a generic region (template 0, typical prediction), or the whole image as a generic region, whichever is smaller: on rendered 300 dpi text 28% of the generic region's size and 17% of Group 4's; on scans, whose copies of a letter are seldom identical, nearly always the generic region (27–33% smaller than Group 4 on the same pages with scan-like noise added). A document below PDF 1.4 gets /Version 1.4. Not yet: JBIG2 refinement coding of near-copies (where lossless gains on scans would come from), a symbol dictionary shared between images (/JBIG2Globals), JBIG2's lossy mode, monochrome downsampling, inline images, soft masks and /Mask images, Group 4 for rows wider than 32,767 samples (JBIG2 takes rows of up to 1,048,576). Redaction doesn't decode JBIG2 or Group 4 yet: a mark over part of such an image removes the whole image (nothing leaks). Export reads JBIG2, but nothing that exports an image reads Group 4 yet: Export ▸ Images and Save Image As refuse it, Copy Image fails, and Word, PowerPoint and HTML export leave the picture out."
[[feature]]
id = "optimize.only-if-smaller"