Checkboxes with an indirect /AP /N state dictionary are drawn (#180) (#181)

hayro-syntax recovered a stream with a wrong /Length by scanning ahead
for the bytes `stream`. An indirect appearance-state dictionary
(`<< /Yes … /Off … >>`, as Quartz writes a checkbox's `/AP /N`) was
read as whatever stream came later in the file. That dictionary has no
`/BBox`, so the appearance was dropped and the widget was skipped.

`stream` now has to be the next token. A wrong `/Length` still
recovers, and the renderer already selects the `/AS` state once `/N`
is a dictionary.

Fixes #180

Co-authored-by: Eri Bastos <eb@fluxmail.xyz>
Co-authored-by: Brandon Thomas <echelon@gmail.com>

crates/render/src/raster.rs

@@ -590,6 +590,40 @@ trailer << /Root 1 0 R >>
         assert_eq!(px(75, 25), vec![255, 255, 255, 255], "NoView annotation must not be drawn");
     }
 
+    /// Quartz writes `/AP /N` as an indirect dictionary of states, and the next object in
+    /// the file is often some other stream. That dictionary must not be read as that stream,
+    /// or the checkbox (whose real appearance is `/AS`) is skipped.
+    #[test]
+    fn indirect_appearance_state_dict_is_not_another_stream() {
+        let pdf = b"%PDF-1.7
+1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj
+2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj
+3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Annots [4 0 R] >> endobj
+4 0 obj << /Type /Annot /Subtype /Widget /FT /Btn /T (cb) /V /Yes /AS /Yes /Rect [10 10 30 30]
+   /AP 9 0 R >> endobj
+9 0 obj << /N 10 0 R >> endobj
+10 0 obj << /Yes 5 0 R /Off 6 0 R >> endobj
+11 0 obj << /Length 24 >> stream
+0 0 1 rg 0 0 20 20 re f
+endstream
+endobj
+5 0 obj << /Type /XObject /Subtype /Form /BBox [0 0 20 20] /Length 24 >> stream
+1 0 0 rg 0 0 20 20 re f
+endstream
+endobj
+6 0 obj << /Type /XObject /Subtype /Form /BBox [0 0 20 20] /Length 24 >> stream
+0 1 0 rg 0 0 20 20 re f
+endstream
+endobj
+trailer << /Root 1 0 R >>
+%%EOF";
+        let mut r = PageRenderer::new(Arc::new(pdf.to_vec()), RenderConfig::default());
+        let p = r.render(RenderRequest { page: 0, kind: RequestKind::Pixels, tile: None, scale: 1.0, tag: 0 });
+        assert!(p.error.is_none(), "{:?}", p.error);
+        let px = |x: u32, y: u32| p.rgba[((y * p.width + x) * 4) as usize..][..4].to_vec();
+        assert_eq!(px(20, 80), vec![255, 0, 0, 255], "checkbox must show the /Yes appearance, not the decoy stream");
+    }
+
     #[test]
     fn hiding_comments_keeps_fields() {
         let pdf = b"%PDF-1.7

vendor/README.md

@@ -12,6 +12,6 @@ the fix. Vendoring copyleft code is never allowed (plan/adr/0001).
 | hayro | 0.7.1 | Apache-2.0 OR MIT | (1) `RenderSettings::{x_offset, y_offset}` to render one tile of a large page; (2) images over `MAX_IMAGE_PIXELS` (2^28) are skipped (a fuzzed `/W 4294967295` hung in resampling); (3) `tiling_cell_scale` keeps a tiling pattern's cell pixmap within 3000 px a side whatever its `/XStep`/`/YStep` (a fuzzed step far beyond `/BBox` allocated 2 GB); (4) stroke widths clamped to 4 × (canvas width + height) device pixels (a fuzzed `/LW 9223372036854775807` allocated 10 GB in stroke expansion) | (1) `raster::tests::tiles_match_full_render`; (2) `raster::tests::absurd_image_dimensions_are_skipped`; (3) `raster::tests::tiling_cells_stay_small_whatever_the_step`; (4) `raster::tests::absurd_predictor_columns_and_line_widths_render` |
 | hayro-jbig2 | 0.3.0 | Apache-2.0 OR MIT | (1) at most 2^28 pixels per bitmap, page bitmap, halftone grid and grayscale image (each side could be 65535: 4.3 billion pixels decoded one by one); (2) at most 2^26 symbol pixels per symbol dictionary (a fuzzed dictionary decoded for minutes) | (1) `raster::tests::jbig2_regions_of_billions_of_pixels_are_refused`; (2) the nightly `cargo xtask fuzz` finding, which now finishes in about a second |
 | lopdf | 0.45.0 | MIT | (1) PNG predictor rows longer than the data are refused before their buffers are allocated (a fuzzed `/Columns 4294967295` allocated two 4 GiB rows); (2) predictor row widths use saturating arithmetic | (1) `inspect::tests::png_predictor_rows_longer_than_the_data_are_refused_before_allocating` |
-| hayro-syntax | 0.7.2 | Apache-2.0 OR MIT | (1) Page-tree cycle guard in `resolve_pages` (visited set + depth cap): a `/Kids` loop among compressed objects overflowed the stack and aborted the process; (2) `ccitt_size_ok`: CCITT images wider than 2^20 columns or over 2^28 pixels are not decoded (a fuzzed `/Columns 4294967295` allocated 4 GiB); (3) Flate/LZW predictor row widths are saturating `usize`, a zero width is refused and a row longer than the data is not allocated (a fuzzed `/Columns 9223372036854775807` wrapped to a 2^61-byte allocation and aborted); (4) the inline-image end-of-data check (`EI`) doesn't nest inside its own look-ahead and examines at most 256 candidates and steps (50 images with "EI" in their data ran for over ten minutes) | (1) `cos/tests/writer.rs::page_tree_loops_in_object_streams_do_not_crash_readers`; (2) `raster::tests::absurd_mask_and_fax_sizes_are_refused_before_decoding`; (3) `raster::tests::absurd_predictor_columns_and_line_widths_render`; (4) `raster::tests::inline_images_with_ei_in_their_data_parse_in_linear_time` |
+| hayro-syntax | 0.7.2 | Apache-2.0 OR MIT | (1) Page-tree cycle guard in `resolve_pages` (visited set + depth cap): a `/Kids` loop among compressed objects overflowed the stack and aborted the process; (2) `ccitt_size_ok`: CCITT images wider than 2^20 columns or over 2^28 pixels are not decoded (a fuzzed `/Columns 4294967295` allocated 4 GiB); (3) Flate/LZW predictor row widths are saturating `usize`, a zero width is refused and a row longer than the data is not allocated (a fuzzed `/Columns 9223372036854775807` wrapped to a 2^61-byte allocation and aborted); (4) the inline-image end-of-data check (`EI`) doesn't nest inside its own look-ahead and examines at most 256 candidates and steps (50 images with "EI" in their data ran for over ten minutes); (5) a stream whose `/Length` is wrong is recovered only when `stream` is the next token — scanning ahead to a later object's `stream` made an indirect appearance-state dictionary (Quartz checkboxes: `/AP /N` → `<< /Yes … /Off … >>`) render as a non-form, so the widget was skipped | (1) `cos/tests/writer.rs::page_tree_loops_in_object_streams_do_not_crash_readers`; (2) `raster::tests::absurd_mask_and_fax_sizes_are_refused_before_decoding`; (3) `raster::tests::absurd_predictor_columns_and_line_widths_render`; (4) `raster::tests::inline_images_with_ei_in_their_data_parse_in_linear_time`; (5) `raster::tests::indirect_appearance_state_dict_is_not_another_stream` |
 
 Temporary: hayro is the bootstrap renderer and lopdf the bootstrap inspector (ADR-0004); M2.6 replaces them with our own devices and `cos`/`model`.

vendor/hayro-syntax/src/object/stream.rs

@@ -302,8 +302,12 @@ fn parse_proper<'a>(r: &mut Reader<'a>, dict: &Dict<'a>) -> Option<Stream<'a>> {
 }
 
 fn parse_fallback<'a>(r: &mut Reader<'a>, dict: &Dict<'a>) -> Option<Stream<'a>> {
-    let stream_offset = find_needle(r.tail()?, b"stream")?;
-    r.read_bytes(stream_offset)?;
+    // PdfCraft patch: `stream` must be the next token. Scanning ahead to a later
+    // `stream` made a dictionary look like a stream — in particular an indirect
+    // `/AP /N` appearance-state dictionary (`<< /Yes 5 0 R /Off 6 0 R >>`, as Quartz
+    // writes checkboxes). The widget then had no form to draw, so the box disappeared.
+    // A wrong `/Length` still recovers, because that keyword sits directly after the dict.
+    r.skip_white_spaces_and_comments();
     r.forward_tag(b"stream")?;
 
     r.forward_tag(b"\n")
@@ -385,4 +389,13 @@ mod tests {
 
         assert_eq!(stream.data, b"abcdefghij");
     }
+
+    /// A dictionary followed by some other object's stream is not itself a stream.
+    /// `parse_fallback` used to latch onto that later `stream` keyword.
+    #[test]
+    fn dictionary_is_not_a_later_objects_stream() {
+        let data = b"<< /Yes 5 0 R /Off 6 0 R >>\nendobj\n11 0 obj << /Length 5 >> stream\nhello\nendstream";
+        let mut r = Reader::new(data);
+        assert!(r.read_with_context::<Stream<'_>>(&ReaderContext::dummy()).is_none());
+    }
 }