M11: monochrome downsampling: a tie goes to the dark sample, after the decode array

A 2 × 2 tie always gave sample 0, black in grey and the painted part of
an image mask with the default decode array. With an inverting one,
[1 0], 0 is white or unpainted, and in a separation 0 is no ink, so a
line half a new pixel wide could vanish. dark_is_one says which sample
is dark from the colour space and decode array: 0 in grey and of a
mask, 1 in a separation or a DeviceN of one colourant, each the other
way round when the decode array inverts; an indexed image keeps entry
0, as its palette, not the decode array, says which is darker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z

crates/optimize/README.md

@@ -43,8 +43,10 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going
   (resampled with it there, kept as it is here), as they can a dithered scan by downsampling it to
   JPEG.
   - **Downsampling:** one whose effective resolution is above `above_ppi` is averaged down to
-    `target_ppi`: each new pixel takes the sample covering more of its area, a 0 (black in grey)
-    on a tie, so a dark line half a new pixel wide stays. Not one with a soft mask, whose own size
+    `target_ppi`: each new pixel takes the sample covering more of its area, and the dark one on a
+    tie, so a dark line half a new pixel wide stays: black in grey and the painted part of an image
+    mask (a 0, or a 1 with an inverting decode array), full ink in a separation (a 1, or a 0), and
+    entry 0 of an indexed image, whose palette says which is darker. Not one with a soft mask, whose own size
     would have to follow. On a 600 ppi text page, Group 4 at 300 ppi is half the size.
   - **Compression:** CCITT Group 4 (`pdfcraft_filters::encode_g4`), or 1-bit Flate when that is
     smaller, which only a resampled image or one packed from eight bits may be: a 1-bit image that

crates/optimize/src/images.rs

@@ -621,11 +621,34 @@ fn one_bit(doc: &Document, s: &Stream, eight_bit: bool) -> Option<(Vec<u8>, u32,
     Some((packed, w, h, true))
 }
 
+/// Whether sample 1 is the dark one of the 1-bit image `d` describes, after its decode array
+/// (`[0 1]` when it has none of two numbers): the painted part of an image mask and black in grey
+/// decode to 0, and full ink in a separation (or a DeviceN of one colourant) to 1. An indexed image
+/// says 0, as its palette, not its decode array, says which entry is darker.
+fn dark_is_one(doc: &Document, d: &Dict) -> bool {
+    let number = |v: Option<&Object>| v.and_then(|v| doc.resolve(v).as_f64());
+    let (d0, d1) = match d.get(b"Decode").map(|v| doc.resolve(v)).as_deref() {
+        Some(Object::Array(a)) => number(a.first()).zip(number(a.get(1))).unwrap_or((0.0, 1.0)),
+        _ => (0.0, 1.0),
+    };
+    if matches!(d.get(b"ImageMask"), Some(Object::Bool(true))) {
+        return d1 < d0;
+    }
+    match d.get(b"ColorSpace").map(|cs| doc.resolve(cs)).as_deref() {
+        Some(Object::Array(a)) => match a.first().and_then(Object::as_name) {
+            Some(b"Indexed" | b"I") => false,
+            Some(b"Separation" | b"DeviceN") => d1 > d0,
+            _ => d1 < d0,
+        },
+        _ => d1 < d0,
+    }
+}
+
 /// `w` × `h` 1-bit rows (each starting on a new byte) averaged down to `nw` × `nh`, no larger:
-/// each new pixel takes the sample that covers more of its area, and a 0 when the two tie. A 0 is
-/// black in grey and the painted part of an image mask, so a dark line half a new pixel wide stays.
-/// `None` when the rows are short or the new size isn't smaller in both directions or empty.
-pub(crate) fn downsample_bits(data: &[u8], w: u32, h: u32, nw: u32, nh: u32) -> Option<Vec<u8>> {
+/// each new pixel takes the sample that covers more of its area, and the dark one when the two tie
+/// (1 if `dark_is_one`, see [`dark_is_one`]), so a dark line half a new pixel wide stays. `None`
+/// when the rows are short or the new size isn't smaller in both directions or empty.
+pub(crate) fn downsample_bits(data: &[u8], w: u32, h: u32, nw: u32, nh: u32, dark_is_one: bool) -> Option<Vec<u8>> {
     if nw == 0 || nh == 0 || nw > w || nh > h {
         return None;
     }
@@ -656,8 +679,10 @@ pub(crate) fn downsample_bits(data: &[u8], w: u32, h: u32, nw: u32, nh: u32) ->
             let covered: u64 = (x0 / nw..x1.div_ceil(nw).min(w))
                 .map(|x| overlap(x0, x1, x * nw, (x + 1) * nw) * usize::try_from(x).ok().and_then(|x| ones.get(x)).copied().unwrap_or(0))
                 .sum();
-            // More than half of w·h, which can't overflow (w·h < 2^64 for u32 sizes).
-            if covered > w * h / 2 {
+            // Against what the 0 samples cover: covered is at most w·h, which can't overflow
+            // (w·h < 2^64 for u32 sizes).
+            let zeros = w * h - covered;
+            if covered > zeros || (dark_is_one && covered == zeros) {
                 let i = usize::try_from(i).ok()?;
                 *out_row.get_mut(i / 8)? |= 0x80 >> (i % 8);
             }
@@ -700,7 +725,7 @@ fn mono(doc: &Document, s: &Stream, ppi: f64, settings: &MonoSettings) -> Option
     if (retain || fax_alone) && !resampled {
         return None;
     }
-    let data = if resampled { downsample_bits(&data, w, h, nw, nh)? } else { data };
+    let data = if resampled { downsample_bits(&data, w, h, nw, nh, dark_is_one(doc, d))? } else { data };
     let mut dict = d.clone();
     for key in [&b"Filter"[..], b"DecodeParms", b"Length", b"DL"] {
         dict.remove(key);
@@ -874,10 +899,11 @@ mod tests {
 
 #[cfg(test)]
 mod downsample_tests {
-    use super::downsample_bits;
+    use super::{dark_is_one, downsample_bits};
+    use pdfcraft_cos::{Dict, Document, Object};
 
     /// [`downsample_bits`] the slow way: each new pixel's area against every source pixel's.
-    fn naive(data: &[u8], w: u32, h: u32, nw: u32, nh: u32) -> Vec<u8> {
+    fn naive(data: &[u8], w: u32, h: u32, nw: u32, nh: u32, dark_one: bool) -> Vec<u8> {
         let (stride, nstride) = ((w as usize).div_ceil(8), (nw as usize).div_ceil(8));
         let (w, h, nw, nh) = (u64::from(w), u64::from(h), u64::from(nw), u64::from(nh));
         let mut out = vec![0u8; nstride * nh as usize];
@@ -893,7 +919,7 @@ mod downsample_tests {
                         }
                     }
                 }
-                if 2 * ones > w * h {
+                if 2 * ones > w * h || (dark_one && 2 * ones == w * h) {
                     out[j as usize * nstride + i as usize / 8] |= 0x80 >> (i % 8);
                 }
             }
@@ -904,28 +930,65 @@ mod downsample_tests {
     #[test]
     fn each_new_pixel_takes_the_sample_covering_more_of_it() {
         // 4 × 2 to 2 × 1: three 1s of four on the left, none on the right.
-        assert_eq!(downsample_bits(&[0b1100_0000, 0b1000_0000], 4, 2, 2, 1), Some(vec![0b1000_0000]));
-        // A tie gives 0: black in grey, so a black line half a new pixel wide stays (the second
-        // of four white columns is black); a white one that thin on black doesn't.
-        assert_eq!(downsample_bits(&[0b1011_0000, 0b1011_0000], 4, 2, 2, 1), Some(vec![0b0100_0000]));
-        assert_eq!(downsample_bits(&[0b0100_0000, 0b0100_0000], 4, 2, 2, 1), Some(vec![0]));
+        assert_eq!(downsample_bits(&[0b1100_0000, 0b1000_0000], 4, 2, 2, 1, false), Some(vec![0b1000_0000]));
+        // A tie gives the dark sample: 0, black in grey, so a black line half a new pixel wide
+        // stays (the second of four white columns is black); a white one that thin on black doesn't.
+        assert_eq!(downsample_bits(&[0b1011_0000, 0b1011_0000], 4, 2, 2, 1, false), Some(vec![0b0100_0000]));
+        assert_eq!(downsample_bits(&[0b0100_0000, 0b0100_0000], 4, 2, 2, 1, false), Some(vec![0]));
+        // With 1 the dark sample (an inverting decode array, a separation), that line is a 1.
+        assert_eq!(downsample_bits(&[0b0100_0000, 0b0100_0000], 4, 2, 2, 1, true), Some(vec![0b1000_0000]));
+        assert_eq!(downsample_bits(&[0b1011_0000, 0b1011_0000], 4, 2, 2, 1, true), Some(vec![0b1100_0000]));
         // The same size gives the same bits; padding bits are 0.
-        assert_eq!(downsample_bits(&[0b1010_1111], 3, 1, 3, 1), Some(vec![0b1010_0000]));
+        assert_eq!(downsample_bits(&[0b1010_1111], 3, 1, 3, 1, false), Some(vec![0b1010_0000]));
         // 3 to 2 across: the middle column counts half to each new one.
-        assert_eq!(downsample_bits(&[0b1100_0000], 3, 1, 2, 1), Some(vec![0b1000_0000]));
+        assert_eq!(downsample_bits(&[0b1100_0000], 3, 1, 2, 1, false), Some(vec![0b1000_0000]));
+    }
+
+    /// The dark sample: 0 in grey and of an image mask, 1 in a separation, each the other way round
+    /// with an inverting decode array; 0 for an indexed image, whatever its decode array.
+    #[test]
+    fn the_dark_sample_follows_the_colour_space_and_decode_array() {
+        let doc = Document::new_empty();
+        let image = |entries: &[(&str, Object)]| {
+            let mut d = Dict::new();
+            for (k, v) in entries {
+                d.set(k.as_bytes().to_vec(), v.clone());
+            }
+            d
+        };
+        let inverted = ("Decode", Object::Array(vec![1.into(), 0.into()]));
+        let grey = ("ColorSpace", Object::name("DeviceGray"));
+        let mask = ("ImageMask", Object::Bool(true));
+        let separation =
+            ("ColorSpace", Object::Array(vec![Object::name("Separation"), Object::name("Spot"), Object::name("DeviceCMYK"), Object::Null]));
+        let indexed = ("ColorSpace", Object::Array(vec![Object::name("Indexed"), Object::name("DeviceGray"), 1.into(), Object::Null]));
+        for (entries, dark_one) in [
+            (vec![grey.clone()], false),
+            (vec![grey, inverted.clone()], true),
+            (vec![mask.clone()], false),
+            (vec![mask, inverted.clone()], true),
+            (vec![separation.clone()], true),
+            (vec![separation, inverted.clone()], false),
+            (vec![indexed.clone()], false),
+            (vec![indexed, inverted], false),
+            (vec![("ColorSpace", Object::name("DeviceGray")), ("Decode", Object::Array(vec![Object::name("x"), 0.into()]))], false),
+        ] {
+            assert_eq!(dark_is_one(&doc, &image(&entries)), dark_one, "{entries:?}");
+        }
     }
 
     #[test]
     fn what_it_cannot_resample_is_none() {
-        assert_eq!(downsample_bits(&[0; 2], 4, 2, 5, 1), None, "wider");
-        assert_eq!(downsample_bits(&[0; 2], 4, 2, 2, 3), None, "taller");
-        assert_eq!(downsample_bits(&[0; 2], 4, 2, 0, 1), None);
-        assert_eq!(downsample_bits(&[0; 1], 4, 2, 2, 1), None, "short rows");
-        assert_eq!(downsample_bits(&[], u32::MAX, u32::MAX, 1, 1), None);
+        assert_eq!(downsample_bits(&[0; 2], 4, 2, 5, 1, false), None, "wider");
+        assert_eq!(downsample_bits(&[0; 2], 4, 2, 2, 3, false), None, "taller");
+        assert_eq!(downsample_bits(&[0; 2], 4, 2, 0, 1, false), None);
+        assert_eq!(downsample_bits(&[0; 1], 4, 2, 2, 1, false), None, "short rows");
+        assert_eq!(downsample_bits(&[], u32::MAX, u32::MAX, 1, 1, true), None);
     }
 
     proptest::proptest! {
-        /// Any bitmap, to any size no larger, gives what the slow way gives; all 1s stay all 1s.
+        /// Any bitmap, to any size no larger, with either sample the dark one, gives what the slow
+        /// way gives; all 1s stay all 1s.
         #[test]
         fn it_matches_the_slow_way(
             w in 1u32..40,
@@ -933,14 +996,15 @@ mod downsample_tests {
             nw_part in 0.0f64..1.0,
             nh_part in 0.0f64..1.0,
             seed in proptest::collection::vec(0u8..=255, 1..64),
+            dark_one: bool,
         ) {
             let (nw, nh) = (1 + (f64::from(w - 1) * nw_part) as u32, 1 + (f64::from(h - 1) * nh_part) as u32);
             let len = (w as usize).div_ceil(8) * h as usize;
             let data: Vec<u8> = seed.iter().cycle().take(len).copied().collect();
-            proptest::prop_assert_eq!(downsample_bits(&data, w, h, nw, nh), Some(naive(&data, w, h, nw, nh)));
-            let white = downsample_bits(&vec![0xFF; len], w, h, nw, nh).unwrap();
-            proptest::prop_assert_eq!(white, naive(&vec![0xFF; len], w, h, nw, nh));
-            proptest::prop_assert!(naive(&vec![0xFF; len], w, h, nw, nh).iter().any(|b| *b != 0));
+            proptest::prop_assert_eq!(downsample_bits(&data, w, h, nw, nh, dark_one), Some(naive(&data, w, h, nw, nh, dark_one)));
+            let white = downsample_bits(&vec![0xFF; len], w, h, nw, nh, dark_one).unwrap();
+            proptest::prop_assert_eq!(white, naive(&vec![0xFF; len], w, h, nw, nh, dark_one));
+            proptest::prop_assert!(naive(&vec![0xFF; len], w, h, nw, nh, dark_one).iter().any(|b| *b != 0));
         }
     }
 }

crates/optimize/tests/optimize.rs

@@ -792,7 +792,7 @@ fn eight_bit_black_and_white_images_become_one_bit() {
 /// Group 4 (alone and behind ASCIIHexDecode), an image mask (its inverting decode array stays) and
 /// eight-bit grey of only black and white, all drawn at 600 ppi, are stored at half the size (as
 /// Group 4, or 1-bit Flate when that is smaller, as it is on this clean text), each new pixel the
-/// majority of the 2 × 2 it covers (a tie black). Left at their size: an image at 400 ppi (stored
+/// majority of the 2 × 2 it covers (a tie black, or for the mask painted). Left at their size: an image at 400 ppi (stored
 /// again as Group 4), a Group 4 one at 400 ppi (not even decoded), one behind ASCIIHexDecode at
 /// 400 ppi (stored again as Group 4 alone), and one with a soft mask. The page looks the same at
 /// screen size. With `downsample` off nothing is resampled; with Retain, Flate stays Flate, CCITT
@@ -867,18 +867,24 @@ fn black_and_white_images_above_450_ppi_are_averaged_down_to_300() {
         let d = stream(doc, r).dict;
         (d.int(b"Width").unwrap(), d.int(b"Height").unwrap())
     };
-    // Each new pixel: 1 when more than two of the 2 × 2 it covers are.
+    // Each new pixel: 1 when more than two of the 2 × 2 it covers are, or two when 1 is the dark
+    // sample (the mask's decode array inverts it, so its painted part is 1).
     let stride = (w as usize).div_ceil(8);
     let bit = |x: usize, y: usize| text[y * stride + x / 8] & (0x80 >> (x % 8)) != 0;
-    let mut halved = vec![0u8; (w as usize / 2).div_ceil(8) * h as usize / 2];
-    for y in 0..h as usize / 2 {
-        for x in 0..w as usize / 2 {
-            let ones = [(0, 0), (1, 0), (0, 1), (1, 1)].iter().filter(|(dx, dy)| bit(2 * x + dx, 2 * y + dy)).count();
-            if ones > 2 {
-                halved[y * (w as usize / 2).div_ceil(8) + x / 8] |= 0x80 >> (x % 8);
+    let halve = |dark_one: bool| {
+        let mut out = vec![0u8; (w as usize / 2).div_ceil(8) * h as usize / 2];
+        for y in 0..h as usize / 2 {
+            for x in 0..w as usize / 2 {
+                let ones = [(0, 0), (1, 0), (0, 1), (1, 1)].iter().filter(|(dx, dy)| bit(2 * x + dx, 2 * y + dy)).count();
+                if ones > 2 || (dark_one && ones == 2) {
+                    out[y * (w as usize / 2).div_ceil(8) + x / 8] |= 0x80 >> (x % 8);
+                }
             }
         }
-    }
+        out
+    };
+    let (halved, halved_mask) = (halve(false), halve(true));
+    assert_ne!(halved, halved_mask, "the picture has ties");
 
     // The grey settings left alone too: an eight-bit image would otherwise take them when smaller.
     let mut off = doc.clone();
@@ -909,7 +915,7 @@ fn black_and_white_images_above_450_ppi_are_averaged_down_to_300() {
         let s = stream(&doc, i);
         assert_eq!((size(&doc, i), s.dict.int(b"BitsPerComponent")), ((600, 240), if i == mask { None } else { Some(1) }));
         assert!(matches!(s.dict.name(b"Filter"), Some(b"CCITTFaxDecode" | b"FlateDecode")), "{i:?}");
-        assert_eq!(doc.decoded_image(&s).unwrap(), halved, "{i:?}");
+        assert_eq!(doc.decoded_image(&s).unwrap(), if i == mask { halved_mask.clone() } else { halved.clone() }, "{i:?}");
     }
     assert_eq!(stream(&doc, mask).dict.get(b"Decode"), Some(&decode), "the decode array stays");
     for i in [at_400, soft, hex_at_400] {

parity/acrobat-features.toml

@@ -7361,7 +7361,7 @@ status = "partial"
 commands = ["optimize.advanced"]
 tools = ["doc_optimize"]
 evidence = ["crates/optimize/tests/optimize.rs::images_are_measured_where_drawn_and_downsampled", "crates/optimize/tests/optimize.rs::settings_choose_what_happens", "crates/automation/tests/it/automation.rs::optimizing_through_tools", "crates/optimize/tests/optimize.rs::a_page_picture_printed_as_one_pixel_strips_is_merged_into_one_image", "crates/optimize/tests/optimize.rs::images_that_do_not_tile_a_picture_are_left_alone", "crates/optimize/tests/optimize.rs::a_deeply_nested_form_is_measured_at_its_largest_use", "crates/optimize/tests/optimize.rs::images_are_not_resampled_when_some_uses_could_not_be_measured", "crates/optimize/tests/optimize.rs::image_resolution_uses_the_pages_physical_unit", "crates/optimize/tests/optimize.rs::optimization_retains_an_image_below_threshold_on_a_scaled_page", "crates/optimize/tests/optimize.rs::shared_forms_are_measured_in_each_pages_physical_units", "crates/optimize/tests/optimize.rs::black_and_white_images_above_450_ppi_are_averaged_down_to_300", "crates/optimize/src/images.rs::it_matches_the_slow_way", "crates/automation/tests/it/automation.rs::doc_optimize_averages_black_and_white_scans_down", "crates/ui-egui/tests/it/create.rs::the_pdf_optimizer_dialog_saves_an_optimized_copy"]
-notes = "colour and grayscale, bicubic, effective resolution measured where drawn (every use, with page-local UserUnit scaling, through nested forms up to 12 deep; when a use can't be measured, nothing is resampled); image fragments stacked edge to edge are merged first (side-by-side tiles not yet); monochrome (1-bit, CCITT fax and eight-bit black-and-white grey) averaged by area to 300 ppi above 450 ppi by default, a tie black, then stored as Group 4 or 1-bit Flate (Monochrome Images ▸ Downsample to; doc_optimize mono); not monochrome images with a soft mask, and Acrobat's bicubic and subsampling choices for monochrome aren't offered"
+notes = "colour and grayscale, bicubic, effective resolution measured where drawn (every use, with page-local UserUnit scaling, through nested forms up to 12 deep; when a use can't be measured, nothing is resampled); image fragments stacked edge to edge are merged first (side-by-side tiles not yet); monochrome (1-bit, CCITT fax and eight-bit black-and-white grey) averaged by area to 300 ppi above 450 ppi by default, a tie to the dark sample (black in grey, painted in a mask, ink in a separation, after the decode array), then stored as Group 4 or 1-bit Flate (Monochrome Images ▸ Downsample to; doc_optimize mono); not monochrome images with a soft mask, and Acrobat's bicubic and subsampling choices for monochrome aren't offered"
 
 [[feature]]
 id = "optimize.recompress-images"