From 9c875656de6967b54abe3e818dd102f7f13de609 Mon Sep 17 00:00:00 2001 From: Piohrun Date: Thu, 15 Oct 2026 08:20:31 +0200 Subject: [PATCH] fix(render, ui): form-heavy documents draw and fill fast (spot-colour images, widget placement, Fields panel) (#1691) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(render): convert spot-colour images once per pixel value (vendored hayro-interpret patch 28) An image whose pixels hold at most 16 bits of samples (an 8-bit Separation or DeviceN image, a 1-, 2- or 4-bit gray or indexed one) is now converted to RGB through a table of every possible pixel value instead of pixel by pixel. A Separation or DeviceN tint transform is a PDF function, often a PostScript calculator program, and the image decoder ran it once per pixel. Fillable character sheets draw their background art as an 8-bit DeviceN image: one with a 3304 x 4105 background took 3.8 s and 430 MB to draw each page (Poppler: 0.27 s), when opening it and again after every field was filled. It now draws in 0.11 s and 100 MB, pixel for pixel the same. The table decodes each sample through the same `decode_sample` as the per-pixel path, keeps `fix_image_length`'s rules for short and overlong data (the pixels past the data are converted as before), and is used only when the image has more pixels than the table has entries. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01KnNgjPEDfKJM1wj17Q8Za8 * fix(render): place widgets without /P in one pass over the pages Opening a document, the inspector finds the page of a form field whose widget has no /P by searching every page's /Annots for it, once per field: O(fields x annotations). A form of 8,000 such fields took 1.17 s to open instead of 0.27 s. The pages' /Annots are now read once, on the first such field, into a map from annotation to page. A widget that several pages list is now placed on the first of them, as pdfcraft-forms places it; the search took whichever page a HashMap visited first. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01KnNgjPEDfKJM1wj17Q8Za8 * fix(ui): the Fields panel keeps up with large forms Every frame the Fields panel searched the form for each row by name, once to sort the rows into tab order and again to find what a click selects: O(fields²) a frame. It also laid out, drew and built the tooltip of every row, scrolled into view or not. With 8,000 fields the app used 13.4 s of CPU for 20 pointer moves with the panel open (0.2 s with it closed). The rows are now worked out by `field_rows`, which pairs each row with the first form field of its name through one map a frame, as the engine pairs them, and a row out of view is allocated and keeps its clicks, keyboard focus and accessibility label but isn't drawn and builds no tooltip. The same 20 moves now take 0.86 s. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01KnNgjPEDfKJM1wj17Q8Za8 * docs: roadmap entries for faster form-heavy documents Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01KnNgjPEDfKJM1wj17Q8Za8 * fix(render): convert each lookup-table entry on its own (vendored hayro-interpret patch 28) Patch 28 converted the whole table of possible pixel values in one batch. Separation and DeviceN `convert_f32` concatenate each tint result, and a Type 4 program that returns the wrong number of components is only warned about, so a malformed result for a value the image never uses shifted every later entry. A 32 x 32 image whose only sample (255) is red drew black because tint 0 returned one component (PR #1691 review). `convert_each` now converts the table one entry at a time, so a malformed entry changes only itself. The largest table (65,536 entries, a two-ink DeviceN image over DeviceCMYK) takes about 2 ms longer. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 Co-authored-by: Brandon Thomas --- ROADMAP.md | 1 + crates/render/src/inspect.rs | 97 +++++++++-- crates/render/src/raster.rs | 148 +++++++++++++++++ crates/ui-egui/src/panels.rs | 218 ++++++++++++++++++++----- docs/roadmap.md | 2 + vendor/README.md | 2 +- vendor/hayro-interpret/src/x_object.rs | 170 ++++++++++++++++--- 7 files changed, 566 insertions(+), 72 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index a975629455..242afd39db 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -155,6 +155,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-14 (M5, PR #1457 branch refresh):** Merged upstream main and reconciled roadmap entries while preserving comment submission shortcuts. Estimates unchanged. - **2026-10-12 (M3, page-label navigation):** The page box displays logical labels and keeps the current page when its unchanged value is submitted, including duplicate labels. Editable-control regressions also cover numeric fallback, invalid input, active typing and focused same-document replacement with changed labels or fewer pages. Clean file-watcher reload preserves the chosen scrolling mode, preventing a Single Page reader from jumping to page 1 during replacement. Estimates unchanged. +- **2026-10-13 (community, M2/M6, form-heavy documents):** An image whose pixels hold at most 16 bits of samples (an 8-bit Separation or DeviceN spot-colour image, a 1-4-bit gray or indexed one) is converted once per possible pixel value instead of once per pixel (vendored hayro-interpret patch 28). A fillable character sheet whose 3304 × 4105 DeviceN background ran a PostScript tint transform per pixel drew each page in 3.8 s and 430 MB, on opening and again after every field was filled; it now draws in 0.11 s and 100 MB, pixel for pixel the same. The inspector places widgets without `/P` in one pass over the pages (a form of 8,000 such fields opened in 1.17 s), and the Fields panel pairs its rows with the form through one map a frame and draws only the rows in view (8,000 fields: about 15× less CPU a frame with the panel open). Regressions fail before each fix. - **2026-10-13 (M3, native-opening candidate):** Chosen tools survive preparation, conversion and password retries; startup intake waits for pending preparation. Regression tests cover completion, cancellation and document identity. Estimates unchanged. - **2026-10-12 (M10, Windows printing without PowerShell, #1107):** craft-print now prints on Windows in-process: printers from EnumPrinters/GetDefaultPrinter, jobs drawn as vectors by Windows.Data.Pdf into the XPS print API with a driver-validated PrintTicket (exact paper, orientation, no scaling), so printing works where App Control for Business or AppLocker run PowerShell in Constrained Language Mode. Portrait and landscape sheets still print as one job (turned with /Rotate); a sheet size the driver lacks is refused rather than shrunk. Pins craft-libs f03ec27 (craft-libs#28); deny.toml allows the storytold/winprint.rs fork and adds a webpki-root-certs licence exception and a RUSTSEC-2024-0370 ignore for winprint's build-dependencies. An earlier craft-print revision was checked in Windows Sandbox under enforced WDAC: printers listed, CLI and desktop jobs spooled, and no powershell.exe in the 4688 audit. The final pinned revision passed craft-libs Windows CI, including a real paused-queue job; that Sandbox run has not been repeated. Estimates unchanged. - **2026-10-12 (M4, #1310, #1311, #1312):** Undo back to the opened file left it marked unsaved, so recovery and close treated unchanged work as dirty. After a save, Revert followed the restored snapshot's older base and the next save dropped the latest saved revision. The last saved file is remembered on its own: undo and redo are clean only when they land on it, Revert opens it, and a later save appends to it. Estimates unchanged. diff --git a/crates/render/src/inspect.rs b/crates/render/src/inspect.rs index a3af3876b0..eefc2e923d 100644 --- a/crates/render/src/inspect.rs +++ b/crates/render/src/inspect.rs @@ -5,6 +5,7 @@ //! remains the compatibility fallback. When neither can inspect a renderable file, panels //! are empty and `warnings` says why. +use std::cell::OnceCell; use std::collections::{BTreeMap, HashMap, HashSet}; use std::sync::Arc; @@ -499,13 +500,17 @@ struct Inspector<'a> { /// Named destinations (`/Names /Dests` tree), keyed by raw string bytes, built once. /// Looking each name up by walking the tree was O(links × names): minutes on manuals. named: HashMap, &'a Object>, + /// The page of each annotation (the first page that lists it), built when a field's widget + /// has no `/P`. Searching every page's `/Annots` for each such field was O(fields × + /// annotations): a second on a form of 8,000 fields. + annotation_pages: OnceCell>, } impl<'a> Inspector<'a> { fn new(doc: &'a dyn Structure) -> Self { let page_ids = doc.page_ids(); let page_index = page_ids.iter().copied().enumerate().map(|(index, id)| (id, index)).collect(); - let mut me = Self { doc, page_ids, page_index, named: HashMap::new() }; + let mut me = Self { doc, page_ids, page_index, named: HashMap::new(), annotation_pages: OnceCell::new() }; let mut named = HashMap::new(); if let Some(tree) = doc.catalog().ok().and_then(|c| c.get(b"Names").ok()).and_then(|o| me.dict(o)).and_then(|n| n.get(b"Dests").ok()).and_then(|o| me.dict(o)) @@ -1184,17 +1189,25 @@ impl<'a> Inspector<'a> { out.push(Field { name: full, kind, value, page, tooltip: self.text(d, b"TU"), has_actions, rect }); } + /// The first page whose `/Annots` lists the field or one of its kids. fn find_widget_page(&self, field: &Object, kids: &[Object]) -> Option { - let targets: HashSet = - std::iter::once(field).chain(kids.iter()).filter_map(|o| if let Object::Reference(id) = o { Some(*id) } else { None }).collect(); - if targets.is_empty() { - return None; + let pages = self.annotation_pages.get_or_init(|| self.annotation_pages()); + std::iter::once(field).chain(kids).filter_map(|o| if let Object::Reference(id) = o { pages.get(id).copied() } else { None }).min() + } + + /// Each annotation listed in a page's `/Annots`, with the first page that lists it. + fn annotation_pages(&self) -> HashMap { + let mut pages = HashMap::new(); + for (idx, &pid) in self.page_ids.iter().enumerate() { + let Some(annots) = self.doc.get_dictionary(pid).ok().and_then(|d| d.get(b"Annots").ok()) else { continue }; + let Ok(annots) = self.resolve(annots).as_array() else { continue }; + for a in annots { + if let Object::Reference(id) = a { + pages.entry(*id).or_insert(idx); + } + } } - self.page_index.iter().find_map(|(&pid, &idx)| { - let annots = self.doc.get_dictionary(pid).ok()?.get(b"Annots").ok()?; - let arr = self.resolve(annots).as_array().ok()?; - arr.iter().any(|a| matches!(a, Object::Reference(id) if targets.contains(id))).then_some(idx) - }) + pages } /// The topmost field above widget `id` (itself without a `/Parent`), or `None` when it or an @@ -2062,6 +2075,70 @@ trailer << /Root 1 0 R >> assert_eq!(listed, [("alpha", Some("one"), Some(0)), ("beta", None, Some(0))], "{:?}", info.fields); } + /// A `Structure` that counts the objects read through it. + struct CountingReads<'a> { + doc: &'a Document, + reads: std::cell::Cell, + } + + impl Structure for CountingReads<'_> { + fn trailer(&self) -> &Dictionary { + &self.doc.trailer + } + fn get_object(&self, id: ObjectId) -> lopdf::Result<&Object> { + self.reads.set(self.reads.get() + 1); + self.doc.get_object(id) + } + fn encrypted(&self) -> bool { + false + } + } + + /// Widgets without `/P` are placed on the page that lists them with one pass over the pages' + /// `/Annots`, not a search of every page per field: that was O(fields × annotations), a + /// second to open a form of 8,000 fields. A widget two pages list is on the first of them. + #[test] + fn widgets_without_a_page_entry_are_placed_in_one_pass() { + const PAGES: usize = 100; + const PER_PAGE: usize = 10; + let field = |p: usize, k: usize| 4 + PAGES + p * PER_PAGE + k; + let mut pdf = String::from("%PDF-1.7\n1 0 obj << /Type /Catalog /Pages 2 0 R /AcroForm 3 0 R >> endobj\n"); + let kids: Vec = (0..PAGES).map(|p| format!("{} 0 R", 4 + p)).collect(); + pdf += &format!("2 0 obj << /Type /Pages /Kids [{}] /Count {PAGES} >> endobj\n", kids.join(" ")); + let fields: Vec = (0..PAGES).flat_map(|p| (0..PER_PAGE).map(move |k| format!("{} 0 R", field(p, k)))).collect(); + pdf += &format!("3 0 obj << /Fields [{}] >> endobj\n", fields.join(" ")); + for p in 0..PAGES { + let mut annots: Vec = (0..PER_PAGE).map(|k| format!("{} 0 R", field(p, k))).collect(); + // The last page lists the first page's first widget too. + if p == PAGES - 1 { + annots.push(format!("{} 0 R", field(0, 0))); + } + pdf += &format!("{} 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [{}] >> endobj\n", 4 + p, annots.join(" ")); + } + for p in 0..PAGES { + for k in 0..PER_PAGE { + pdf += &format!("{} 0 obj << /Type /Annot /Subtype /Widget /FT /Tx /T (f{p}_{k}) /Rect [10 10 90 30] >> endobj\n", field(p, k)); + } + } + pdf += "trailer << /Root 1 0 R >>\n%%EOF"; + let doc = Document::load_mem(pdf.as_bytes()).expect("loads"); + let counting = CountingReads { doc: &doc, reads: std::cell::Cell::new(0) }; + let mut info = DocInfo { + pages: (0..PAGES) + .map(|_| PageInfo { width: 200.0, height: 200.0, crop: [0.0, 0.0, 200.0, 200.0], rotation: 0, label: String::new() }) + .collect(), + ..Default::default() + }; + Inspector::new(&counting).fill(&mut info); + let placed: Vec<_> = info.fields.iter().map(|f| (f.name.clone(), f.page)).collect(); + let want: Vec<_> = (0..PAGES).flat_map(|p| (0..PER_PAGE).map(move |k| (format!("f{p}_{k}"), Some(p)))).collect(); + assert_eq!(placed, want); + // A handful of reads per page and field; searching the pages for each field read the + // page dictionaries about 50,000 times. + let reads = counting.reads.get(); + assert!(reads < 20 * (PAGES + PAGES * PER_PAGE), "{reads} object reads"); + } + #[test] fn lazy_structure_matches_the_compatibility_inspector() { // Each fixture as lopdf writes it, and with its objects in object streams. diff --git a/crates/render/src/raster.rs b/crates/render/src/raster.rs index 4ad7196629..d1425717d6 100644 --- a/crates/render/src/raster.rs +++ b/crates/render/src/raster.rs @@ -5579,6 +5579,154 @@ endstream endobj assert_eq!(rgb_at(&out.rgba, w, 100, 35), [255, 0, 0]); } + /// Render `pdf`'s first page at 72 dpi with no time limit beyond the test's own. + fn render_first_page(pdf: Vec) -> RenderedPage { + let mut r = PageRenderer::new(Arc::new(pdf), RenderConfig::default()); + let out = r.render(RenderRequest { page: 0, kind: RequestKind::Pixels, tile: None, scale: 1.0, tag: 0, preview: false }); + assert!(out.error.is_none(), "{:?}", out.error); + out + } + + /// A spot-colour image runs its tint transform once per possible sample, not once per pixel + /// (vendored hayro-interpret patch 28). Fillable character sheets draw their background art + /// as an 8-bit DeviceN image whose tint transform is a PostScript calculator program: run for + /// each of a 3304 × 4105 background's pixels, every page took four seconds and 430 MB to + /// draw, again after each field was filled. This image (2048 × 2048, a 9,000-operator + /// program) took about 40 seconds that way; from a 256-entry table it takes milliseconds. + #[test] + fn spot_colour_images_run_their_tint_transform_once_per_sample_value() { + const SIDE: usize = 2048; + // 8,990 operators that change nothing (the calculator allows 10,000 a run), then + // tint t → RGB (1 − t, t, 0). + let program = format!("{{ {}dup 1 exch sub exch 0 }}", "dup pop ".repeat(4495)); + // Each row: tint 0 on the left half, tint 1 on the right. + let row: Vec = [vec![0u8; SIDE / 2], vec![255u8; SIDE / 2]].concat(); + let data = row.repeat(SIDE); + let image = format!( + "<< /Type /XObject /Subtype /Image /Width {SIDE} /Height {SIDE} /ColorSpace [/Separation /Spot /DeviceRGB 6 0 R] /BitsPerComponent 8 /Length {} >>", + data.len() + ); + let function = format!("<< /FunctionType 4 /Domain [0 1] /Range [0 1 0 1 0 1] /Length {} >>", program.len()); + let pdf = + image_cache_pdf("q 130 0 0 70 0 0 cm /A Do Q", "<< /XObject << /A 5 0 R >> >>", &[(&image, &data), (&function, program.as_bytes())]); + let (tx, rx) = std::sync::mpsc::channel(); + std::thread::spawn(move || { + let _ = tx.send(render_first_page(pdf)); + }); + let out = rx.recv_timeout(std::time::Duration::from_secs(10)).expect("a spot-colour image must not run its tint transform for every pixel"); + let w = out.width as usize; + assert_eq!(rgb_at(&out.rgba, w, 10, 35), [255, 0, 0], "tint 0"); + assert_eq!(rgb_at(&out.rgba, w, 120, 35), [0, 255, 0], "tint 1"); + } + + /// Images converted through patch 28's table keep the colour of every sample: the samples of + /// a pixel in order, each through its own `/Decode` range, rows that end mid-byte, and an image + /// whose data stops part-way through a row (that row is padded with decoded zeros, as before). + #[test] + fn images_converted_through_a_table_keep_every_samples_colour() { + let close = |got: [u8; 3], want: [u8; 3], at: (usize, usize)| { + assert!(got.iter().zip(want).all(|(g, w)| g.abs_diff(w) <= 1), "pixel {at:?}: got {got:?}, want {want:?}"); + }; + // DeviceN of two inks at 4 bits (256 possible pixels) shown as RGB (a, b, 0), the first + // ink's range reversed. 32 × 16 pixels, each drawn 4 pt square. + let mut data = Vec::new(); + for y in 0..16 { + for x in 0..32 { + let (a, b) = (x % 16, (x / 16 + 2 * y) % 16); + data.push(((a << 4) | b) as u8); + } + } + let image = format!( + "<< /Type /XObject /Subtype /Image /Width 32 /Height 16 /ColorSpace [/DeviceN [/A /B] /DeviceRGB 6 0 R] /BitsPerComponent 4 /Decode [1 0 0 1] /Length {} >>", + data.len() + ); + let pdf = image_cache_pdf( + "q 128 0 0 64 0 0 cm /A Do Q", + "<< /XObject << /A 5 0 R >> >>", + &[(&image, &data), ("<< /FunctionType 4 /Domain [0 1 0 1] /Range [0 1 0 1 0 1] /Length 5 >>", b"{ 0 }")], + ); + let out = render_first_page(pdf); + let w = out.width as usize; + for y in 0..16 { + for x in 0..32 { + let (a, b) = (x % 16, (x / 16 + 2 * y) % 16); + // The 64 pt image sits at the bottom of the 70 pt page. + close(rgb_at(&out.rgba, w, 4 * x + 2, 6 + 4 * y + 2), [(255 - 17 * a) as u8, (17 * b) as u8, 0], (x, y)); + } + } + + // DeviceGray at 2 bits, reversed: 31 pixels (62 bits, so each row ends mid-byte) by 6. + let sample = |x: usize, y: usize| (x + y) % 4; + let mut data = Vec::new(); + for y in 0..6 { + let mut row = [0u8; 8]; + for x in 0..31 { + row[x / 4] |= (sample(x, y) as u8) << (6 - 2 * (x % 4)); + } + data.extend_from_slice(&row); + } + let image = format!( + "<< /Type /XObject /Subtype /Image /Width 31 /Height 6 /ColorSpace /DeviceGray /BitsPerComponent 2 /Decode [1 0] /Length {} >>", + data.len() + ); + let pdf = image_cache_pdf("q 124 0 0 60 0 0 cm /A Do Q", "<< /XObject << /A 5 0 R >> >>", &[(&image, &data)]); + let out = render_first_page(pdf); + let w = out.width as usize; + for y in 0..6 { + for x in 0..31 { + // The 60 pt image at the bottom of the page: rows 10 pt high. + close(rgb_at(&out.rgba, w, 4 * x + 2, 10 + 10 * y + 5), [(255 - 85 * sample(x, y)) as u8; 3], (x, y)); + } + } + + // DeviceGray at 8 bits through the range [0.2 0.8] (so not drawn from the raw bytes): + // 31 × 12 declared, data for 8 rows and the first 12 pixels of the 9th. + let sample = |x: usize, y: usize| (x * 7 + y * 3) % 256; + let mut data = Vec::new(); + for y in 0..9 { + let row: Vec = (0..31).map(|x| sample(x, y) as u8).collect(); + data.extend_from_slice(if y < 8 { &row[..] } else { &row[..12] }); + } + let image = format!( + "<< /Type /XObject /Subtype /Image /Width 31 /Height 12 /ColorSpace /DeviceGray /BitsPerComponent 8 /Decode [0.2 0.8] /Length {} >>", + data.len() + ); + let pdf = image_cache_pdf("q 124 0 0 60 0 0 cm /A Do Q", "<< /XObject << /A 5 0 R >> >>", &[(&image, &data)]); + let out = render_first_page(pdf); + let w = out.width as usize; + for y in 0..9 { + for x in 0..31 { + // The rest of the 9th row is decoded zero (black), not sample 0 (0.2 under this range). + let gray = if y < 8 || x < 12 { (51.0 + 0.6 * sample(x, y) as f64).round() as u8 } else { 0 }; + // The rows present keep the declared rows' height: 60 pt / 12. + close(rgb_at(&out.rgba, w, 4 * x + 2, 10 + 5 * y + 2), [gray; 3], (x, y)); + } + } + } + + /// Patch 28's table converts each entry on its own: a tint transform that gives the wrong + /// number of components for a sample value the image never uses leaves the colours of the + /// values it does use alone. Converted as one batch, the one component tint 0 gives here + /// shifted every later entry, and tint 1's red came out black (PR #1691 review). + #[test] + fn a_malformed_tint_for_an_unused_sample_keeps_the_used_colours() { + // Tint 0 gives one component instead of the three `/Range` declares; any other tint red. + let program = b"{ dup 0 eq { pop 0 } { pop 1 0 0 } ifelse }"; + // 32 × 32 = 1,024 pixels, more than the table's 256 entries, all sample 255 (tint 1). + let data = [255u8; 32 * 32]; + let centre = |space: &str| { + let image = format!( + "<< /Type /XObject /Subtype /Image /Width 32 /Height 32 /ColorSpace [{space} /DeviceRGB 6 0 R] /BitsPerComponent 8 /Length {} >>", + data.len() + ); + let function = format!("<< /FunctionType 4 /Domain [0 1] /Range [0 1 0 1 0 1] /Length {} >>", program.len()); + let pdf = image_cache_pdf("q 130 0 0 70 0 0 cm /A Do Q", "<< /XObject << /A 5 0 R >> >>", &[(&image, &data), (&function, program)]); + let out = render_first_page(pdf); + rgb_at(&out.rgba, out.width as usize, 65, 35) + }; + assert_eq!([centre("/Separation /Spot"), centre("/DeviceN [/Spot]")], [[255, 0, 0]; 2], "Separation, then one-ink DeviceN"); + } + /// A Japanese CID font that isn't embedded (Adobe-Japan1, as `HeiseiMin-W3` with /// `UniJIS-UCS2-H` in #260's test file) drew nothing: hayro's substitutes for fonts that /// aren't embedded are Latin-only. With craft-fonts (the build input release builds embed), diff --git a/crates/ui-egui/src/panels.rs b/crates/ui-egui/src/panels.rs index cf71f99a68..dc989d660e 100644 --- a/crates/ui-egui/src/panels.rs +++ b/crates/ui-egui/src/panels.rs @@ -1239,9 +1239,7 @@ fn fields( empty(ui, t, "text-cursor-input", "This document has no form fields."); return None; } - let rank = |name: &str| form.iter().find(|f| f.name == name).and_then(|f| f.widgets.iter().map(|w| w.tab).min()).unwrap_or(usize::MAX); - let mut ordered: Vec<_> = info.fields.iter().collect(); - ordered.sort_by_key(|f| (f.page, rank(&f.name))); + let rows = field_rows(&info.fields, form); if preparing { ui.label(egui::RichText::new(tl!("Tab order: move a field with its arrows.")).small().color(t.text_muted)); ui.label(egui::RichText::new(tl!("Shift- or Ctrl/Command-click to select several fields.")).small().color(t.text_muted)); @@ -1255,10 +1253,7 @@ fn fields( } }); } - let mut pages: Vec> = info.fields.iter().map(|f| f.page).collect(); - pages.sort(); - pages.dedup(); - for p in pages { + for (p, page_rows) in rows { // A field's /P can name a page the document doesn't have (#1307); it reads as unplaced. let label = p .and_then(|p| info.pages.get(p)) @@ -1266,11 +1261,7 @@ fn fields( .unwrap_or_else(|| tl!("Unplaced").to_string()); ui.add_space(4.0); ui.label(egui::RichText::new(label).font(theme::semibold(12.5)).color(t.text_muted)); - for f in ordered.iter().copied().filter(|f| f.page == p) { - let key = form - .iter() - .find(|field| field.name == f.name) - .and_then(|field| field.widgets.iter().position(|w| w.page == f.page).map(|wi| (field.name.clone(), wi))); + for (f, key) in page_rows { let selected = key.as_ref().is_some_and(|key| selection.selected.as_ref() == Some(key) || selection.also.contains(key)); let icon = match f.kind { FieldKind::Text => "text-cursor-input", @@ -1284,28 +1275,25 @@ fn fields( }; let (rect, resp) = ui.allocate_exact_size(vec2(ui.available_width(), 30.0), Sense::click()); resp.widget_info(|| egui::WidgetInfo::selected(egui::WidgetType::Button, true, selected, &f.name)); - let press = theme::Press::track(ui, &resp); - press.wash(ui, rect, 6, selected); - let body = rect.translate(press.offset()); - icons::paint(ui, Rect::from_min_size(body.min + vec2(8.0, 7.0), vec2(16.0, 16.0)), icon, 15.0, Color32::from_rgb(0x8E, 0x4E, 0xE6)); - let value = - f.value.as_ref().map(|v| if v.chars().count() > 18 { format!("{}…", v.chars().take(17).collect::()) } else { v.clone() }); - let value_font = theme::regular(11.5); - let value_w = value.as_ref().map(|v| crate::widgets::text_width(ui, v, &value_font)).unwrap_or(0.0); - let arrows = if preparing && f.page.is_some() { 56.0 } else { 0.0 }; - let max_w = (rect.width() - 32.0 - value_w - arrows - 16.0).max(0.0); - crate::widgets::paint_left(ui, body.left_center() + vec2(32.0, 0.0), &f.name, theme::regular(13.0), t.text, max_w); - if let Some(v) = value { - let x = if arrows > 0.0 { 64.0 } else { 8.0 }; - ui.painter().text(body.right_center() - vec2(x, 0.0), Align2::RIGHT_CENTER, v, value_font, t.text_faint); - } - let kind = format!("{:?}", f.kind); - let mut tip = crate::i18n::fmt(tl!("{kind} field"), &[("kind", tl!(&kind))]); - if let Some(tt) = &f.tooltip { - tip.push_str(&format!(" — {tt}")); - } - if f.has_actions { - tip.push_str(&format!("\n{}", tl!("Has JavaScript actions (run in M6)"))); + // A row scrolled out of view still takes clicks and keyboard focus, but isn't drawn: + // laying out every row of a large form each frame took a tenth of a second a frame. + let visible = ui.is_rect_visible(rect); + if visible { + let press = theme::Press::track(ui, &resp); + press.wash(ui, rect, 6, selected); + let body = rect.translate(press.offset()); + icons::paint(ui, Rect::from_min_size(body.min + vec2(8.0, 7.0), vec2(16.0, 16.0)), icon, 15.0, Color32::from_rgb(0x8E, 0x4E, 0xE6)); + let value = + f.value.as_ref().map(|v| if v.chars().count() > 18 { format!("{}…", v.chars().take(17).collect::()) } else { v.clone() }); + let value_font = theme::regular(11.5); + let value_w = value.as_ref().map(|v| crate::widgets::text_width(ui, v, &value_font)).unwrap_or(0.0); + let arrows = if preparing && f.page.is_some() { 56.0 } else { 0.0 }; + let max_w = (rect.width() - 32.0 - value_w - arrows - 16.0).max(0.0); + crate::widgets::paint_left(ui, body.left_center() + vec2(32.0, 0.0), &f.name, theme::regular(13.0), t.text, max_w); + if let Some(v) = value { + let x = if arrows > 0.0 { 64.0 } else { 8.0 }; + ui.painter().text(body.right_center() - vec2(x, 0.0), Align2::RIGHT_CENTER, v, value_font, t.text_faint); + } } if preparing && f.page.is_some() { let up = Rect::from_center_size(rect.right_center() - vec2(44.0, 0.0), vec2(22.0, 22.0)); @@ -1315,9 +1303,11 @@ fn fields( { let b = ui.interact(r, ui.id().with((&f.name, earlier)), Sense::click()); b.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, format!("{tip}: {}", f.name))); - let bpress = theme::Press::track(ui, &b); - bpress.wash(ui, r, 4, false); - icons::paint(ui, r.shrink(3.0).translate(bpress.offset()), icon, 15.0, t.icon); + if visible { + let bpress = theme::Press::track(ui, &b); + bpress.wash(ui, r, 4, false); + icons::paint(ui, r.shrink(3.0).translate(bpress.offset()), icon, 15.0, t.icon); + } if theme::hand(b).on_hover_text(tip).clicked() { *edit = Some(pdfcraft_engine::Edit::MoveInTabOrder { name: f.name.clone(), earlier }); } @@ -1336,7 +1326,10 @@ fn fields( { selection.select(key, ui.input(|i| i.modifiers.shift || i.modifiers.command)); } - if theme::hand(resp).on_hover_text(tip).clicked() + let resp = theme::hand(resp); + // Only a row in view can be hovered: its tip isn't built for the others. + let resp = if visible { resp.on_hover_text(field_tip(f)) } else { resp }; + if resp.clicked() && let (Some(p), Some(r)) = (f.page, f.rect) { *nav = Some(Nav::Flash(p, r)); @@ -1346,6 +1339,46 @@ fn fields( properties } +/// A Fields panel row's tooltip: the field's kind, its own tooltip and whether it has scripts. +fn field_tip(f: &pdfcraft_render::Field) -> String { + let kind = format!("{:?}", f.kind); + let mut tip = crate::i18n::fmt(tl!("{kind} field"), &[("kind", tl!(&kind))]); + if let Some(tt) = &f.tooltip { + tip.push_str(&format!(" — {tt}")); + } + if f.has_actions { + tip.push_str(&format!("\n{}", tl!("Has JavaScript actions (run in M6)"))); + } + tip +} + +/// One page's rows of the Fields panel: each field, with the form field and widget a click on +/// it selects (`None` when the form has no widget of it on that page). +type FieldRows<'a> = Vec<(&'a pdfcraft_render::Field, Option<(String, usize)>)>; + +/// The Fields panel's rows: by page ("Unplaced" first), each page's in tab order. A row is paired +/// with the first form field of its name, as the engine pairs them: one map, built once a frame. +/// Searching the form for each row, and again inside the sort, was O(fields²) every frame. +fn field_rows<'a>(fields: &'a [pdfcraft_render::Field], form: &[pdfcraft_engine::FormField]) -> Vec<(Option, FieldRows<'a>)> { + let mut by_name: std::collections::HashMap<&str, &pdfcraft_engine::FormField> = std::collections::HashMap::with_capacity(form.len()); + for field in form { + by_name.entry(field.name.as_str()).or_insert(field); + } + let rank = |name: &str| by_name.get(name).and_then(|f| f.widgets.iter().map(|w| w.tab).min()).unwrap_or(usize::MAX); + let mut ordered: Vec<&pdfcraft_render::Field> = fields.iter().collect(); + ordered.sort_by_cached_key(|f| (f.page, rank(&f.name))); + let mut rows: Vec<(Option, FieldRows<'a>)> = Vec::new(); + for f in ordered { + let key = + by_name.get(f.name.as_str()).and_then(|field| field.widgets.iter().position(|w| w.page == f.page).map(|wi| (field.name.clone(), wi))); + match rows.last_mut() { + Some((page, page_rows)) if *page == f.page => page_rows.push((f, key)), + _ => rows.push((f.page, vec![(f, key)])), + } + } + rows +} + pub fn human_size(n: usize) -> String { match n { n if n >= 1 << 20 => format!("{:.1} MB", n as f64 / (1u64 << 20) as f64), @@ -1388,6 +1421,115 @@ mod page_moves { } } +#[cfg(test)] +mod field_rows { + use super::field_rows; + use std::sync::Arc; + + /// A PDF of `objects` (numbered from 1; object 1 the catalog) with a valid xref table. + fn pdf(objects: &[String]) -> Vec { + let mut out = b"%PDF-1.7\n".to_vec(); + let mut offsets = Vec::new(); + for (i, o) in objects.iter().enumerate() { + offsets.push(out.len()); + out.extend(format!("{} 0 obj\n{o}\nendobj\n", i + 1).bytes()); + } + let xref = out.len(); + out.extend(format!("xref\n0 {}\n0000000000 65535 f \n", objects.len() + 1).bytes()); + for o in offsets { + out.extend(format!("{o:010} 00000 n \n").bytes()); + } + out.extend(format!("trailer\n<< /Size {} /Root 1 0 R >>\nstartxref\n{xref}\n%%EOF\n", objects.len() + 1).bytes()); + out + } + + /// The fields the engine reads from `bytes`: the inspector's list and the form. + fn open(bytes: Vec) -> (Vec, Arc>) { + let mut session = pdfcraft_engine::Session::new(); + let id = session.open("form.pdf", None, Arc::new(bytes), None).expect("opens"); + let doc = session.get(id).expect("open"); + (doc.info.fields.clone(), doc.form.clone()) + } + + /// Rows go by page, the unplaced first, each page's in tab order (here its annotation order, + /// not the `/Fields` order), and a row selects the first form field of its name. + #[test] + fn rows_follow_pages_and_tab_order() { + let widget = |name: &str, page: Option| match page { + Some(p) => format!("<< /Type /Annot /Subtype /Widget /FT /Tx /T ({name}) /Rect [10 10 90 30] /P {} 0 R >>", 3 + p), + None => format!("<< /Type /Annot /Subtype /Widget /FT /Tx /T ({name}) /Rect [10 10 90 30] >>"), + }; + let bytes = pdf(&[ + "<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [5 0 R 7 0 R 8 0 R 9 0 R 10 0 R 6 0 R] >> >>".into(), + "<< /Type /Pages /Kids [3 0 R 4 0 R] /Count 2 >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [6 0 R 5 0 R] >>".into(), + "<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [7 0 R 8 0 R 9 0 R] >>".into(), + widget("a1", Some(0)), + widget("a2", Some(0)), + widget("b", Some(1)), + widget("twin", Some(1)), + widget("twin", Some(1)), + widget("loose", None), + ]); + let (fields, form) = open(bytes); + // A page's rows as (field name, the form field and widget a click selects). + type Named<'a> = Vec<(&'a str, Option<(String, usize)>)>; + let rows: Vec<(Option, Named)> = field_rows(&fields, &form) + .into_iter() + .map(|(page, rows)| (page, rows.into_iter().map(|(f, key)| (f.name.as_str(), key)).collect())) + .collect(); + let key = |name: &str| Some((name.to_string(), 0)); + assert_eq!( + rows, + [ + (None, vec![("loose", key("loose"))]), + (Some(0), vec![("a2", key("a2")), ("a1", key("a1"))]), + (Some(1), vec![("b", key("b")), ("twin", key("twin")), ("twin", key("twin"))]), + ] + ); + } + + /// The panel's rows for a large form take milliseconds: looking each row's field up in the + /// form, there and inside the sort, was O(fields²) every frame. + #[test] + fn rows_for_a_large_form_are_quick() { + const PAGES: usize = 150; + const PER_PAGE: usize = 100; + let mut objects = vec![String::new(), String::new()]; + let page = |p: usize| 3 + p; + let field = |p: usize, k: usize| 3 + PAGES + p * PER_PAGE + k; + let mut all = Vec::new(); + for p in 0..PAGES { + let annots: Vec = (0..PER_PAGE).map(|k| format!("{} 0 R", field(p, k))).collect(); + all.extend(annots.iter().cloned()); + objects.push(format!("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 200 200] /Annots [{}] >>", annots.join(" "))); + } + for p in 0..PAGES { + for k in 0..PER_PAGE { + objects.push(format!("<< /Type /Annot /Subtype /Widget /FT /Tx /T (field{p}_{k}) /Rect [10 10 90 30] /P {} 0 R >>", page(p))); + } + } + // `/Fields` in no particular order, so the rows must be sorted. + let mut seed = 12345u64; + for i in (1..all.len()).rev() { + seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + all.swap(i, (seed >> 33) as usize % (i + 1)); + } + objects[0] = format!("<< /Type /Catalog /Pages 2 0 R /AcroForm << /Fields [{}] >> >>", all.join(" ")); + let kids: Vec = (0..PAGES).map(|p| format!("{} 0 R", page(p))).collect(); + objects[1] = format!("<< /Type /Pages /Kids [{}] /Count {PAGES} >>", kids.join(" ")); + let (fields, form) = open(pdf(&objects)); + assert_eq!(fields.len(), PAGES * PER_PAGE); + let (tx, rx) = std::sync::mpsc::channel(); + std::thread::spawn(move || { + let rows = field_rows(&fields, &form); + let _ = tx.send((rows.len(), rows.iter().map(|(_, r)| r.len()).sum::())); + }); + let counted = rx.recv_timeout(std::time::Duration::from_secs(3)).expect("the Fields panel's rows must not take seconds a frame"); + assert_eq!(counted, (PAGES, PAGES * PER_PAGE)); + } +} + #[cfg(test)] mod label_column { use super::ellipsized_prefix; diff --git a/docs/roadmap.md b/docs/roadmap.md index bccae8026a..22a24b81c7 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,5 +1,6 @@ # PdfCraft roadmap: milestones and what's next +> **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-13 · **Change:** minor (M2/M6: spot-colour images convert once per pixel value; widget placement and the Fields panel scale with the field count; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-13 · **Last updated:** 2026-10-13 · **Change:** minor (M3 candidate: native opens retain chosen tools through password retries and defer startup intake while preparation is pending; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M4 #1310 #1311 #1312: undo, revert and the next save keep the last saved revision; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-12 · **Last updated:** 2026-10-12 · **Change:** minor (M13: CLI check visits each directory once through aliases and cycles; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) @@ -166,6 +167,7 @@ M2 (renderer), M7 (content editing), M10 (OCR and Office), M11 (Preflight and st |---|---|---| | 2026-10-12 | minor | M4 #1310 #1311 #1312: undo back to the opened file is clean, Revert returns to the last saved file, and the next save keeps that revision. Estimates unchanged. | | 2026-10-14 | minor | M5: reconciled PR #1457 with upstream main, preserving comment submission shortcuts and upstream roadmap progress. Estimates unchanged. | +| 2026-10-13 | minor | M2/M6: images with at most 16 bits of samples a pixel (8-bit Separation/DeviceN, 1-4-bit gray or indexed) convert once per pixel value (hayro-interpret patch 28), so spot-colour form backgrounds draw in milliseconds instead of seconds; widgets without `/P` are placed in one pass; the Fields panel no longer searches the form per row and draws only the rows in view. Estimates unchanged. | | 2026-10-12 | minor | M13: CLI check directory identity, with overlap and symlink regressions; estimates unchanged | | 2026-10-12 | minor | M4: shared Type 3 glyph resources survive extraction, with save/reopen and independent raster regressions; estimates unchanged | | 2026-10-12 | minor | M2: packed TIFF-predictor RGB row boundaries, with decoding and page-render regressions; estimates unchanged | diff --git a/vendor/README.md b/vendor/README.md index c433193f3a..33248d09d4 100644 --- a/vendor/README.md +++ b/vendor/README.md @@ -7,7 +7,7 @@ the fix. Vendoring copyleft code is never allowed (plan/adr/0001). | Crate | Version | Licence | Patches | Test | |---|---|---|---|---| -| hayro-interpret | 0.7.0 | Apache-2.0 OR MIT | (1) select `/AP /N` appearance states via `/AS` (checkboxes, radios, stateful icons) and honour the NoView flag; (2) Type3 Unicode fallback to `/Encoding` glyph names, then printable ASCII; (3) `Type3Glyph::advance_width`; (4) `InterpreterSettings::ocg_overrides` for viewer layer toggles; (5) soft masks whose group lacks `/CS` were dropped (Chrome gradient text); (6) paint nesting cap `MAX_PAINT_NESTING` for tiling patterns and Type 3 glyphs (self-referencing content overflowed the stack); (7) CID width ranges in `/W` and `/W2` clamped to `MAX_CID` (a `0 4294967295 w` range hung); (8) `InterpreterSettings::hide_comments` skips markup annotations but keeps widgets and links (View ▸ Hide all comments); (9) `image_size_ok`: images and masks over 2^28 pixels are not decoded (a fuzzed `/W 4294967295` stencil mask allocated 4 GiB); (10) `MAX_NESTED_PAINTS`: nested paints (forms, Type 3 glyphs and tiling patterns inside one another) share a per-page budget of 50,000 (a glyph showing eight glyphs of its own font, sixteen deep, hung); (11) depends on skrifa 0.47 instead of 0.42 (upstream read-fonts fixed a Type 1 tokenizer that recursed through runs of numbers: a fuzzed font hung or overflowed the stack), with the one API change (`Charset::glyph_id` returns `Option`); (12) annotation appearances whose `/BBox` or `/Matrix` collapses to a line or a point are skipped (mapping them onto `/Rect` divided by zero and handed the device infinite or NaN matrices); (13) a Highlight annotation without a usable normal appearance is drawn from its `/QuadPoints`, `/C` and `/CA` with Multiply blending, as pdfcraft-annot draws highlights (it was not drawn at all); (14) simple-font `/Widths` (`read_widths`): entries only for codes 0–255 (one entry per code below `/FirstChar` was allocated: `4294967295` asked for 34 GB), an entry that isn't a number leaves its code missing instead of ending the array, and a `/LastChar` below `/FirstChar` gives no widths instead of discarding the font for a standard one; (15) the appearance box is mapped onto `/Rect` corner to corner: the translation now accounts for the scale (an appearance whose box didn't start at the origin and differed in size from `/Rect` was drawn shifted); (16) a clip set by `W`/`W*` takes effect after whichever path-painting operator ends the path (`S`, `f`, `B`, … as well as `n`; ISO 32000-2 §8.5.4): it was applied only on `n`, so `re W* S` clipped nothing and the clip stayed pending for a later, unrelated `n`; (17) `InterpreterSettings::cancelled`: once set, interpretation stops before the next content operator (a render pool sets it for every worker when dropped, after an edit or on closing, and for one worker when the watchdog gives up on its render, so renders whose answers nobody can receive stop instead of running to the end); (18) `MAX_PAGE_CONTENT`: the content a page holds at once (its contents, and the forms, tiling patterns and Type 3 glyph procedures being painted) stays within 512 MiB (`decode_content`; once it runs out, `InterpreterWarning::ContentTruncated` is reported (once per page by pdfcraft-render) so a truncated page is not a clean render; each level gives its bytes back when it finishes): forms nest fifty deep and each level kept its content while the next ran, so a form painting itself around a stream inflating to 256 MiB would have held over 12 GB; and Type 0 sampled functions take at most `MAX_SAMPLE_ENTRIES` (2^21) table points and `MAX_SAMPLE_VALUES` (2^23) samples, read no further than `/Size` asks (256 MiB of 1-bit samples asked for over 8 GB); (19) mesh shadings (types 4–7) are sampled only inside the area being drawn (`ShadingPattern::encode_within`, which the vendored hayro renderer calls with that area plus a pixel), within 8 visits per pixel of it plus 4 per triangle (`TriangleSampler`), and patches are triangulated one at a time: points far off the page were sampled over their whole bounding box, up to 65536 × 65536 entries (a fuzzed 147-byte patch stream; a synthetic page passed 6 GB in six seconds), ten thousand overlapping triangles ran for minutes, and every patch became 722 triangles before any limit; meshes hold at most `MAX_MESH_PATCHES` (2^16) patches and `MAX_MESH_TRIANGLES` (2^20) triangles, a lattice needs two to 2^19 vertices per row (`/VerticesPerRow 0` pushed empty rows for ever; one huge row read the whole stream), and a page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25) mesh triangles in all (a mesh painted a thousand times was triangulated a thousand times); each cut logs a warning; (20) PDF function construction is limited to 64 levels and 10,000 total nodes with cycle detection; calculator parsing and evaluation share procedure depth/work budgets, integer division/bitshift/rotation/index are panic-free, and fixed-size stack errors propagate; (21) a Type 0 font drawn with a substitute that isn't CID-keyed (it isn't embedded, or its program doesn't load) draws `.notdef` for a CID of an Adobe character collection (Japan1, GB1, CNS1, Korea1) that it can't find by Unicode: hayro took the CID for a glyph index of the substitute, which numbers its glyphs its own way, so the character was drawn as an unrelated one (𠮷 as "Ｑ" in BIZ UDPGothic); (22) a Type 0 font drawn with a substitute finds a character given with a variation selector (`/ToUnicode`, or Adobe-Japan1-UCS2, which maps the JIS X 0208 forms of the kanji JIS X 0213:2004 redrew that way: CID 1247 is 噂 U+E0100) through the substitute's variation sequences, else as the character alone: such a CID found no glyph and was drawn as an unrelated one; (23) backport of hayro 0.8's `gradient` module: `EncodedShadingPattern::as_svg_gradient` turns an axial or radial shading into stops of an sRGB gradient (within 0.01 per channel, a stop at every stitching-function bound via `Function::stitching_bounds`) when a native gradient draws it exactly (extended past the box drawn in; a radial one with no background colour); (24) `DecodedImageCache`: decoded image pixels shared across renders through the interpreter cache within a 64 MiB budget (an image-heavy page decoded its thousands of images again per tile and per scale), keyed by every decode input (`DecodedImageKey`: the stream's place and dictionary bytes, its object, mask/stencil/raster output, interpolation, decode target size, preview or final); images under a transfer function and inline images are not cached, an image over the budget is never kept, and a poisoned lock decodes uncached; (25) `Device::actual_text` hands the device a marked content sequence's `/ActualText` (inline or named in `/Properties`), so text extraction keeps the Thai, Indic and Arabic clusters Chrome marks this way (their shaped tone marks and vowels have no Unicode of their own); (26) non-embedded simple fonts drawn with a standard-font stand-in build the Latin Extended-A letters the stand-in lacks (`/Gbreve`, `/Idotaccent`, `/Scedilla`, `/aogonek`, `/ccaron`…) from the base letter and accent it has (`StandardKind::composite_glyph`, `ACCENTED_LETTERS`): PDFium's Foxit stand-ins hold only the Standard/WinAnsi/Mac Roman glyphs, so iText letters with Times New Roman unembedded showed blanks for Ğ, İ, Ş, ğ and ş; (27) an embedded TrueType font without a base encoding reads its codes as WinAnsi for text extraction (`EmbeddedKind::code_to_unicode_name`), matching pdftotext's output, instead of Standard: OCR text layers of scans write WinAnsi codes with no `/Encoding`, and Standard has no ü, ö or ß and reads é as Ø (glyph selection unchanged) | (1) `raster::tests::widget_appearance_states`; (2, 3) `xtask text-oracle` on corpus/pdfjs issue918; (4) `raster::tests::layer_override_hides_content`; (5) `raster::tests::soft_mask_without_group_colour_space_is_applied`; (6) `raster::tests::self_referencing_tiling_pattern_terminates`, `raster::tests::self_referencing_type3_glyph_terminates`; (7) `raster::tests::huge_cid_width_ranges_terminate`; (8) `raster::tests::hiding_comments_keeps_fields`; (9) `raster::tests::absurd_mask_and_fax_sizes_are_refused_before_decoding`; (10) `raster::tests::type3_glyphs_that_fan_out_into_themselves_finish`; (11) `raster::tests::type1_font_programs_with_long_runs_of_numbers_load_quickly`; (12) `raster::tests::degenerate_annotation_appearances_are_skipped`; (13) `raster::tests::highlights_without_an_appearance_are_drawn`; (14) `raster::tests::malformed_simple_font_widths_keep_later_widths_and_the_font`, `raster::tests::huge_simple_font_first_char_terminates`; (15) `raster::tests::annotation_appearance_box_maps_onto_rect`; (16) `raster::tests::clipping_path_applies_after_any_painting_operator`; (17) `raster::tests::dropping_the_pool_cancels_its_renders`, `raster::tests::the_watchdog_stops_the_render_it_gives_up_on`; (18) `raster::tests::page_contents_and_nested_forms_share_one_budget`, `raster::tests::content_painted_again_and_again_keeps_drawing`, `raster::tests::sampled_functions_with_billions_of_samples_are_refused`; (19) `raster::tests::mesh_shadings_far_off_the_page_render`, `raster::tests::many_overlapping_mesh_triangles_finish`, `raster::tests::mesh_shadings_with_many_patches_or_no_rows_finish`; (20) `raster::tests::function_limits_calculator_arithmetic_and_depth`, `raster::tests::function_limits_public_construction_work_and_depth`, `raster::tests::function_limits_stitching_cycles_are_refused_and_shared_children_work`, `raster::tests::function_limits_invalid_calculator_keeps_the_page_renderable`; (21) `raster::tests::cids_a_substitute_lacks_are_not_drawn_as_other_glyphs`; (26) `raster::tests::accented_letters_the_standard_stand_ins_lack_are_built_from_their_parts` |; (22) `raster::tests::variation_sequences_draw_their_kanji_in_a_substitute`; (23) `raster::tests::axial_and_radial_shadings_draw_as_native_gradients`; (24) `raster::tests::tiles_match_full_render`, `raster::tests::skipping_paths_off_the_canvas_changes_no_pixel` (both decode through the cache), `raster::tests::decoded_images_with_identical_dictionaries_keep_their_own_pixels`, `raster::tests::decoded_images_under_a_transfer_function_are_not_shared`; (25) `text::tests::actual_text_replaces_the_glyphs_it_marks_in_every_writing_direction`, `text::tests::actual_text_follows_the_pdf_rules_for_marked_content`; (27) `text::tests::truetype_without_an_encoding_reads_its_codes_as_win_ansi` +| hayro-interpret | 0.7.0 | Apache-2.0 OR MIT | (1) select `/AP /N` appearance states via `/AS` (checkboxes, radios, stateful icons) and honour the NoView flag; (2) Type3 Unicode fallback to `/Encoding` glyph names, then printable ASCII; (3) `Type3Glyph::advance_width`; (4) `InterpreterSettings::ocg_overrides` for viewer layer toggles; (5) soft masks whose group lacks `/CS` were dropped (Chrome gradient text); (6) paint nesting cap `MAX_PAINT_NESTING` for tiling patterns and Type 3 glyphs (self-referencing content overflowed the stack); (7) CID width ranges in `/W` and `/W2` clamped to `MAX_CID` (a `0 4294967295 w` range hung); (8) `InterpreterSettings::hide_comments` skips markup annotations but keeps widgets and links (View ▸ Hide all comments); (9) `image_size_ok`: images and masks over 2^28 pixels are not decoded (a fuzzed `/W 4294967295` stencil mask allocated 4 GiB); (10) `MAX_NESTED_PAINTS`: nested paints (forms, Type 3 glyphs and tiling patterns inside one another) share a per-page budget of 50,000 (a glyph showing eight glyphs of its own font, sixteen deep, hung); (11) depends on skrifa 0.47 instead of 0.42 (upstream read-fonts fixed a Type 1 tokenizer that recursed through runs of numbers: a fuzzed font hung or overflowed the stack), with the one API change (`Charset::glyph_id` returns `Option`); (12) annotation appearances whose `/BBox` or `/Matrix` collapses to a line or a point are skipped (mapping them onto `/Rect` divided by zero and handed the device infinite or NaN matrices); (13) a Highlight annotation without a usable normal appearance is drawn from its `/QuadPoints`, `/C` and `/CA` with Multiply blending, as pdfcraft-annot draws highlights (it was not drawn at all); (14) simple-font `/Widths` (`read_widths`): entries only for codes 0–255 (one entry per code below `/FirstChar` was allocated: `4294967295` asked for 34 GB), an entry that isn't a number leaves its code missing instead of ending the array, and a `/LastChar` below `/FirstChar` gives no widths instead of discarding the font for a standard one; (15) the appearance box is mapped onto `/Rect` corner to corner: the translation now accounts for the scale (an appearance whose box didn't start at the origin and differed in size from `/Rect` was drawn shifted); (16) a clip set by `W`/`W*` takes effect after whichever path-painting operator ends the path (`S`, `f`, `B`, … as well as `n`; ISO 32000-2 §8.5.4): it was applied only on `n`, so `re W* S` clipped nothing and the clip stayed pending for a later, unrelated `n`; (17) `InterpreterSettings::cancelled`: once set, interpretation stops before the next content operator (a render pool sets it for every worker when dropped, after an edit or on closing, and for one worker when the watchdog gives up on its render, so renders whose answers nobody can receive stop instead of running to the end); (18) `MAX_PAGE_CONTENT`: the content a page holds at once (its contents, and the forms, tiling patterns and Type 3 glyph procedures being painted) stays within 512 MiB (`decode_content`; once it runs out, `InterpreterWarning::ContentTruncated` is reported (once per page by pdfcraft-render) so a truncated page is not a clean render; each level gives its bytes back when it finishes): forms nest fifty deep and each level kept its content while the next ran, so a form painting itself around a stream inflating to 256 MiB would have held over 12 GB; and Type 0 sampled functions take at most `MAX_SAMPLE_ENTRIES` (2^21) table points and `MAX_SAMPLE_VALUES` (2^23) samples, read no further than `/Size` asks (256 MiB of 1-bit samples asked for over 8 GB); (19) mesh shadings (types 4–7) are sampled only inside the area being drawn (`ShadingPattern::encode_within`, which the vendored hayro renderer calls with that area plus a pixel), within 8 visits per pixel of it plus 4 per triangle (`TriangleSampler`), and patches are triangulated one at a time: points far off the page were sampled over their whole bounding box, up to 65536 × 65536 entries (a fuzzed 147-byte patch stream; a synthetic page passed 6 GB in six seconds), ten thousand overlapping triangles ran for minutes, and every patch became 722 triangles before any limit; meshes hold at most `MAX_MESH_PATCHES` (2^16) patches and `MAX_MESH_TRIANGLES` (2^20) triangles, a lattice needs two to 2^19 vertices per row (`/VerticesPerRow 0` pushed empty rows for ever; one huge row read the whole stream), and a page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25) mesh triangles in all (a mesh painted a thousand times was triangulated a thousand times); each cut logs a warning; (20) PDF function construction is limited to 64 levels and 10,000 total nodes with cycle detection; calculator parsing and evaluation share procedure depth/work budgets, integer division/bitshift/rotation/index are panic-free, and fixed-size stack errors propagate; (21) a Type 0 font drawn with a substitute that isn't CID-keyed (it isn't embedded, or its program doesn't load) draws `.notdef` for a CID of an Adobe character collection (Japan1, GB1, CNS1, Korea1) that it can't find by Unicode: hayro took the CID for a glyph index of the substitute, which numbers its glyphs its own way, so the character was drawn as an unrelated one (𠮷 as "Ｑ" in BIZ UDPGothic); (22) a Type 0 font drawn with a substitute finds a character given with a variation selector (`/ToUnicode`, or Adobe-Japan1-UCS2, which maps the JIS X 0208 forms of the kanji JIS X 0213:2004 redrew that way: CID 1247 is 噂 U+E0100) through the substitute's variation sequences, else as the character alone: such a CID found no glyph and was drawn as an unrelated one; (23) backport of hayro 0.8's `gradient` module: `EncodedShadingPattern::as_svg_gradient` turns an axial or radial shading into stops of an sRGB gradient (within 0.01 per channel, a stop at every stitching-function bound via `Function::stitching_bounds`) when a native gradient draws it exactly (extended past the box drawn in; a radial one with no background colour); (24) `DecodedImageCache`: decoded image pixels shared across renders through the interpreter cache within a 64 MiB budget (an image-heavy page decoded its thousands of images again per tile and per scale), keyed by every decode input (`DecodedImageKey`: the stream's place and dictionary bytes, its object, mask/stencil/raster output, interpolation, decode target size, preview or final); images under a transfer function and inline images are not cached, an image over the budget is never kept, and a poisoned lock decodes uncached; (25) `Device::actual_text` hands the device a marked content sequence's `/ActualText` (inline or named in `/Properties`), so text extraction keeps the Thai, Indic and Arabic clusters Chrome marks this way (their shaped tone marks and vowels have no Unicode of their own); (26) non-embedded simple fonts drawn with a standard-font stand-in build the Latin Extended-A letters the stand-in lacks (`/Gbreve`, `/Idotaccent`, `/Scedilla`, `/aogonek`, `/ccaron`…) from the base letter and accent it has (`StandardKind::composite_glyph`, `ACCENTED_LETTERS`): PDFium's Foxit stand-ins hold only the Standard/WinAnsi/Mac Roman glyphs, so iText letters with Times New Roman unembedded showed blanks for Ğ, İ, Ş, ğ and ş; (27) an embedded TrueType font without a base encoding reads its codes as WinAnsi for text extraction (`EmbeddedKind::code_to_unicode_name`), matching pdftotext's output, instead of Standard: OCR text layers of scans write WinAnsi codes with no `/Encoding`, and Standard has no ü, ö or ß and reads é as Ø (glyph selection unchanged); (28) an image whose pixels hold at most 16 bits of samples (an 8-bit Separation or DeviceN image, a 1-, 2- or 4-bit gray or indexed one) is converted once per possible pixel value through a lookup table instead of once per pixel (`rgb_via_lookup`): a spot-colour tint transform is often a PostScript calculator program, run for every pixel, so a fillable character sheet's 3304 × 4105 DeviceN background took four seconds and 430 MB to draw a page; each entry is converted on its own (`convert_each`), so a tint transform giving the wrong number of components for a value the image never uses cannot shift the colours of the values it does | (1) `raster::tests::widget_appearance_states`; (2, 3) `xtask text-oracle` on corpus/pdfjs issue918; (4) `raster::tests::layer_override_hides_content`; (5) `raster::tests::soft_mask_without_group_colour_space_is_applied`; (6) `raster::tests::self_referencing_tiling_pattern_terminates`, `raster::tests::self_referencing_type3_glyph_terminates`; (7) `raster::tests::huge_cid_width_ranges_terminate`; (8) `raster::tests::hiding_comments_keeps_fields`; (9) `raster::tests::absurd_mask_and_fax_sizes_are_refused_before_decoding`; (10) `raster::tests::type3_glyphs_that_fan_out_into_themselves_finish`; (11) `raster::tests::type1_font_programs_with_long_runs_of_numbers_load_quickly`; (12) `raster::tests::degenerate_annotation_appearances_are_skipped`; (13) `raster::tests::highlights_without_an_appearance_are_drawn`; (14) `raster::tests::malformed_simple_font_widths_keep_later_widths_and_the_font`, `raster::tests::huge_simple_font_first_char_terminates`; (15) `raster::tests::annotation_appearance_box_maps_onto_rect`; (16) `raster::tests::clipping_path_applies_after_any_painting_operator`; (17) `raster::tests::dropping_the_pool_cancels_its_renders`, `raster::tests::the_watchdog_stops_the_render_it_gives_up_on`; (18) `raster::tests::page_contents_and_nested_forms_share_one_budget`, `raster::tests::content_painted_again_and_again_keeps_drawing`, `raster::tests::sampled_functions_with_billions_of_samples_are_refused`; (19) `raster::tests::mesh_shadings_far_off_the_page_render`, `raster::tests::many_overlapping_mesh_triangles_finish`, `raster::tests::mesh_shadings_with_many_patches_or_no_rows_finish`; (20) `raster::tests::function_limits_calculator_arithmetic_and_depth`, `raster::tests::function_limits_public_construction_work_and_depth`, `raster::tests::function_limits_stitching_cycles_are_refused_and_shared_children_work`, `raster::tests::function_limits_invalid_calculator_keeps_the_page_renderable`; (21) `raster::tests::cids_a_substitute_lacks_are_not_drawn_as_other_glyphs`; (26) `raster::tests::accented_letters_the_standard_stand_ins_lack_are_built_from_their_parts` |; (22) `raster::tests::variation_sequences_draw_their_kanji_in_a_substitute`; (23) `raster::tests::axial_and_radial_shadings_draw_as_native_gradients`; (24) `raster::tests::tiles_match_full_render`, `raster::tests::skipping_paths_off_the_canvas_changes_no_pixel` (both decode through the cache), `raster::tests::decoded_images_with_identical_dictionaries_keep_their_own_pixels`, `raster::tests::decoded_images_under_a_transfer_function_are_not_shared`; (25) `text::tests::actual_text_replaces_the_glyphs_it_marks_in_every_writing_direction`, `text::tests::actual_text_follows_the_pdf_rules_for_marked_content`; (27) `text::tests::truetype_without_an_encoding_reads_its_codes_as_win_ansi`; (28) `raster::tests::spot_colour_images_run_their_tint_transform_once_per_sample_value`, `raster::tests::images_converted_through_a_table_keep_every_samples_colour`, `raster::tests::a_malformed_tint_for_an_unused_sample_keeps_the_used_colours` | hayro-interpret | 0.7.0 | Apache-2.0 OR MIT | (1) select `/AP /N` appearance states via `/AS` (checkboxes, radios, stateful icons) and honour the NoView flag; (2) Type3 Unicode fallback to `/Encoding` glyph names, then printable ASCII; (3) `Type3Glyph::advance_width`; (4) `InterpreterSettings::ocg_overrides` for viewer layer toggles; (5) soft masks whose group lacks `/CS` were dropped (Chrome gradient text); (6) paint nesting cap `MAX_PAINT_NESTING` for tiling patterns and Type 3 glyphs (self-referencing content overflowed the stack); (7) CID width ranges in `/W` and `/W2` clamped to `MAX_CID` (a `0 4294967295 w` range hung); (8) `InterpreterSettings::hide_comments` skips markup annotations but keeps widgets and links (View ▸ Hide all comments); (9) `image_size_ok`: images and masks over 2^28 pixels are not decoded (a fuzzed `/W 4294967295` stencil mask allocated 4 GiB); (10) `MAX_NESTED_PAINTS`: nested paints (forms, Type 3 glyphs and tiling patterns inside one another) share a per-page budget of 50,000 (a glyph showing eight glyphs of its own font, sixteen deep, hung); (11) depends on skrifa 0.47 instead of 0.42 (upstream read-fonts fixed a Type 1 tokenizer that recursed through runs of numbers: a fuzzed font hung or overflowed the stack), with the one API change (`Charset::glyph_id` returns `Option`); (12) annotation appearances whose `/BBox` or `/Matrix` collapses to a line or a point are skipped (mapping them onto `/Rect` divided by zero and handed the device infinite or NaN matrices); (13) a Highlight annotation without a usable normal appearance is drawn from its `/QuadPoints`, `/C` and `/CA` with Multiply blending, as pdfcraft-annot draws highlights (it was not drawn at all); (14) simple-font `/Widths` (`read_widths`): entries only for codes 0–255 (one entry per code below `/FirstChar` was allocated: `4294967295` asked for 34 GB), an entry that isn't a number leaves its code missing instead of ending the array, and a `/LastChar` below `/FirstChar` gives no widths instead of discarding the font for a standard one; (15) the appearance box is mapped onto `/Rect` corner to corner: the translation now accounts for the scale (an appearance whose box didn't start at the origin and differed in size from `/Rect` was drawn shifted); (16) a clip set by `W`/`W*` takes effect after whichever path-painting operator ends the path (`S`, `f`, `B`, … as well as `n`; ISO 32000-2 §8.5.4): it was applied only on `n`, so `re W* S` clipped nothing and the clip stayed pending for a later, unrelated `n`; (17) `InterpreterSettings::cancelled`: once set, interpretation stops before the next content operator (a render pool sets it for every worker when dropped, after an edit or on closing, and for one worker when the watchdog gives up on its render, so renders whose answers nobody can receive stop instead of running to the end); (18) `MAX_PAGE_CONTENT`: the content a page holds at once (its contents, and the forms, tiling patterns and Type 3 glyph procedures being painted) stays within 512 MiB (`decode_content`; once it runs out, `InterpreterWarning::ContentTruncated` is reported (once per page by pdfcraft-render) so a truncated page is not a clean render; each level gives its bytes back when it finishes): forms nest fifty deep and each level kept its content while the next ran, so a form painting itself around a stream inflating to 256 MiB would have held over 12 GB; and Type 0 sampled functions take at most `MAX_SAMPLE_ENTRIES` (2^21) table points and `MAX_SAMPLE_VALUES` (2^23) samples, read no further than `/Size` asks (256 MiB of 1-bit samples asked for over 8 GB); (19) mesh shadings (types 4–7) are sampled only inside the area being drawn (`ShadingPattern::encode_within`, which the vendored hayro renderer calls with that area plus a pixel), within 8 visits per pixel of it plus 4 per triangle (`TriangleSampler`), and patches are triangulated one at a time: points far off the page were sampled over their whole bounding box, up to 65536 × 65536 entries (a fuzzed 147-byte patch stream; a synthetic page passed 6 GB in six seconds), ten thousand overlapping triangles ran for minutes, and every patch became 722 triangles before any limit; meshes hold at most `MAX_MESH_PATCHES` (2^16) patches and `MAX_MESH_TRIANGLES` (2^20) triangles, a lattice needs two to 2^19 vertices per row (`/VerticesPerRow 0` pushed empty rows for ever; one huge row read the whole stream), and a page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25) mesh triangles in all (a mesh painted a thousand times was triangulated a thousand times); each cut logs a warning; (20) PDF function construction is limited to 64 levels and 10,000 total nodes with cycle detection; calculator parsing and evaluation share procedure depth/work budgets, integer division/bitshift/rotation/index are panic-free, and fixed-size stack errors propagate; (21) a Type 0 font drawn with a substitute that isn't CID-keyed (it isn't embedded, or its program doesn't load) draws `.notdef` for a CID of an Adobe character collection (Japan1, GB1, CNS1, Korea1) that it can't find by Unicode: hayro took the CID for a glyph index of the substitute, which numbers its glyphs its own way, so the character was drawn as an unrelated one (𠮷 as "Ｑ" in BIZ UDPGothic); (22) a Type 0 font drawn with a substitute finds a character given with a variation selector (`/ToUnicode`, or Adobe-Japan1-UCS2, which maps the JIS X 0208 forms of the kanji JIS X 0213:2004 redrew that way: CID 1247 is 噂 U+E0100) through the substitute's variation sequences, else as the character alone: such a CID found no glyph and was drawn as an unrelated one; (23) backport of hayro 0.8's `gradient` module: `EncodedShadingPattern::as_svg_gradient` turns an axial or radial shading into stops of an sRGB gradient (within 0.01 per channel, a stop at every stitching-function bound via `Function::stitching_bounds`) when a native gradient draws it exactly (extended past the box drawn in; a radial one with no background colour); (24) `DecodedImageCache`: decoded image pixels shared across renders through the interpreter cache within a 64 MiB budget (an image-heavy page decoded its thousands of images again per tile and per scale), keyed by every decode input (`DecodedImageKey`: the stream's place and dictionary bytes, its object, mask/stencil/raster output, interpolation, decode target size, preview or final); images under a transfer function and inline images are not cached, an image over the budget is never kept, and a poisoned lock decodes uncached; (25) `Device::actual_text` hands the device a marked content sequence's `/ActualText` (inline or named in `/Properties`), so text extraction keeps the Thai, Indic and Arabic clusters Chrome marks this way (their shaped tone marks and vowels have no Unicode of their own); (26) non-embedded simple fonts drawn with a standard-font stand-in build the Latin Extended-A letters the stand-in lacks (`/Gbreve`, `/Idotaccent`, `/Scedilla`, `/aogonek`, `/ccaron`…) from the base letter and accent it has (`StandardKind::composite_glyph`, `ACCENTED_LETTERS`): PDFium's Foxit stand-ins hold only the Standard/WinAnsi/Mac Roman glyphs, so iText letters with Times New Roman unembedded showed blanks for Ğ, İ, Ş, ğ and ş; (27) text in a Type 3 font is drawn filled whatever the text rendering mode, except mode 3 (invisible): only that mode has any effect on Type 3 fonts (ISO 32000-1 §9.3.6). Stroking ran the glyph procedure a second time, so `2 Tr` made bitmap fonts look bold, `1 Tr` painted an uncoloured glyph in the stroking colour, and `7 Tr` hid it | (1) `raster::tests::widget_appearance_states`; (2, 3) `xtask text-oracle` on corpus/pdfjs issue918; (4) `raster::tests::layer_override_hides_content`; (5) `raster::tests::soft_mask_without_group_colour_space_is_applied`; (6) `raster::tests::self_referencing_tiling_pattern_terminates`, `raster::tests::self_referencing_type3_glyph_terminates`; (7) `raster::tests::huge_cid_width_ranges_terminate`; (8) `raster::tests::hiding_comments_keeps_fields`; (9) `raster::tests::absurd_mask_and_fax_sizes_are_refused_before_decoding`; (10) `raster::tests::type3_glyphs_that_fan_out_into_themselves_finish`; (11) `raster::tests::type1_font_programs_with_long_runs_of_numbers_load_quickly`; (12) `raster::tests::degenerate_annotation_appearances_are_skipped`; (13) `raster::tests::highlights_without_an_appearance_are_drawn`; (14) `raster::tests::malformed_simple_font_widths_keep_later_widths_and_the_font`, `raster::tests::huge_simple_font_first_char_terminates`; (15) `raster::tests::annotation_appearance_box_maps_onto_rect`; (16) `raster::tests::clipping_path_applies_after_any_painting_operator`; (17) `raster::tests::dropping_the_pool_cancels_its_renders`, `raster::tests::the_watchdog_stops_the_render_it_gives_up_on`; (18) `raster::tests::page_contents_and_nested_forms_share_one_budget`, `raster::tests::content_painted_again_and_again_keeps_drawing`, `raster::tests::sampled_functions_with_billions_of_samples_are_refused`; (19) `raster::tests::mesh_shadings_far_off_the_page_render`, `raster::tests::many_overlapping_mesh_triangles_finish`, `raster::tests::mesh_shadings_with_many_patches_or_no_rows_finish`; (20) `raster::tests::function_limits_calculator_arithmetic_and_depth`, `raster::tests::function_limits_public_construction_work_and_depth`, `raster::tests::function_limits_stitching_cycles_are_refused_and_shared_children_work`, `raster::tests::function_limits_invalid_calculator_keeps_the_page_renderable`; (21) `raster::tests::cids_a_substitute_lacks_are_not_drawn_as_other_glyphs`; (26) `raster::tests::accented_letters_the_standard_stand_ins_lack_are_built_from_their_parts` |; (22) `raster::tests::variation_sequences_draw_their_kanji_in_a_substitute`; (23) `raster::tests::axial_and_radial_shadings_draw_as_native_gradients`; (24) `raster::tests::tiles_match_full_render`, `raster::tests::skipping_paths_off_the_canvas_changes_no_pixel` (both decode through the cache), `raster::tests::decoded_images_with_identical_dictionaries_keep_their_own_pixels`, `raster::tests::decoded_images_under_a_transfer_function_are_not_shared`; (25) `text::tests::actual_text_replaces_the_glyphs_it_marks_in_every_writing_direction`, `text::tests::actual_text_follows_the_pdf_rules_for_marked_content`; (27) `raster::tests::text_rendering_modes_other_than_invisible_draw_type3_glyphs_as_filled` | hayro-interpret | 0.7.0 | Apache-2.0 OR MIT | (1) select `/AP /N` appearance states via `/AS` (checkboxes, radios, stateful icons) and honour the NoView flag; (2) Type3 Unicode fallback to `/Encoding` glyph names, then printable ASCII; (3) `Type3Glyph::advance_width`; (4) `InterpreterSettings::ocg_overrides` for viewer layer toggles; (5) soft masks whose group lacks `/CS` were dropped (Chrome gradient text); (6) paint nesting cap `MAX_PAINT_NESTING` for tiling patterns and Type 3 glyphs (self-referencing content overflowed the stack); (7) CID width ranges in `/W` and `/W2` clamped to `MAX_CID` (a `0 4294967295 w` range hung); (8) `InterpreterSettings::hide_comments` skips markup annotations but keeps widgets and links (View ▸ Hide all comments); (9) `image_size_ok`: images and masks over 2^28 pixels are not decoded (a fuzzed `/W 4294967295` stencil mask allocated 4 GiB); (10) `MAX_NESTED_PAINTS`: nested paints (forms, Type 3 glyphs and tiling patterns inside one another) share a per-page budget of 50,000 (a glyph showing eight glyphs of its own font, sixteen deep, hung); (11) depends on skrifa 0.47 instead of 0.42 (upstream read-fonts fixed a Type 1 tokenizer that recursed through runs of numbers: a fuzzed font hung or overflowed the stack), with the one API change (`Charset::glyph_id` returns `Option`); (12) annotation appearances whose `/BBox` or `/Matrix` collapses to a line or a point are skipped (mapping them onto `/Rect` divided by zero and handed the device infinite or NaN matrices); (13) a Highlight annotation without a usable normal appearance is drawn from its `/QuadPoints`, `/C` and `/CA` with Multiply blending, as pdfcraft-annot draws highlights (it was not drawn at all); (14) simple-font `/Widths` (`read_widths`): entries only for codes 0–255 (one entry per code below `/FirstChar` was allocated: `4294967295` asked for 34 GB), an entry that isn't a number leaves its code missing instead of ending the array, and a `/LastChar` below `/FirstChar` gives no widths instead of discarding the font for a standard one; (15) the appearance box is mapped onto `/Rect` corner to corner: the translation now accounts for the scale (an appearance whose box didn't start at the origin and differed in size from `/Rect` was drawn shifted); (16) a clip set by `W`/`W*` takes effect after whichever path-painting operator ends the path (`S`, `f`, `B`, … as well as `n`; ISO 32000-2 §8.5.4): it was applied only on `n`, so `re W* S` clipped nothing and the clip stayed pending for a later, unrelated `n`; (17) `InterpreterSettings::cancelled`: once set, interpretation stops before the next content operator (a render pool sets it for every worker when dropped, after an edit or on closing, and for one worker when the watchdog gives up on its render, so renders whose answers nobody can receive stop instead of running to the end); (18) `MAX_PAGE_CONTENT`: the content a page holds at once (its contents, and the forms, tiling patterns and Type 3 glyph procedures being painted) stays within 512 MiB (`decode_content`; once it runs out, `InterpreterWarning::ContentTruncated` is reported (once per page by pdfcraft-render) so a truncated page is not a clean render; each level gives its bytes back when it finishes): forms nest fifty deep and each level kept its content while the next ran, so a form painting itself around a stream inflating to 256 MiB would have held over 12 GB; and Type 0 sampled functions take at most `MAX_SAMPLE_ENTRIES` (2^21) table points and `MAX_SAMPLE_VALUES` (2^23) samples, read no further than `/Size` asks (256 MiB of 1-bit samples asked for over 8 GB); (19) mesh shadings (types 4–7) are sampled only inside the area being drawn (`ShadingPattern::encode_within`, which the vendored hayro renderer calls with that area plus a pixel), within 8 visits per pixel of it plus 4 per triangle (`TriangleSampler`), and patches are triangulated one at a time: points far off the page were sampled over their whole bounding box, up to 65536 × 65536 entries (a fuzzed 147-byte patch stream; a synthetic page passed 6 GB in six seconds), ten thousand overlapping triangles ran for minutes, and every patch became 722 triangles before any limit; meshes hold at most `MAX_MESH_PATCHES` (2^16) patches and `MAX_MESH_TRIANGLES` (2^20) triangles, a lattice needs two to 2^19 vertices per row (`/VerticesPerRow 0` pushed empty rows for ever; one huge row read the whole stream), and a page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25) mesh triangles in all (a mesh painted a thousand times was triangulated a thousand times); each cut logs a warning; (20) PDF function construction is limited to 64 levels and 10,000 total nodes with cycle detection; calculator parsing and evaluation share procedure depth/work budgets, integer division/bitshift/rotation/index are panic-free, and fixed-size stack errors propagate; (21) a Type 0 font drawn with a substitute that isn't CID-keyed (it isn't embedded, or its program doesn't load) draws `.notdef` for a CID of an Adobe character collection (Japan1, GB1, CNS1, Korea1) that it can't find by Unicode: hayro took the CID for a glyph index of the substitute, which numbers its glyphs its own way, so the character was drawn as an unrelated one (𠮷 as "Ｑ" in BIZ UDPGothic); (22) a Type 0 font drawn with a substitute finds a character given with a variation selector (`/ToUnicode`, or Adobe-Japan1-UCS2, which maps the JIS X 0208 forms of the kanji JIS X 0213:2004 redrew that way: CID 1247 is 噂 U+E0100) through the substitute's variation sequences, else as the character alone: such a CID found no glyph and was drawn as an unrelated one; (23) backport of hayro 0.8's `gradient` module: `EncodedShadingPattern::as_svg_gradient` turns an axial or radial shading into stops of an sRGB gradient (within 0.01 per channel, a stop at every stitching-function bound via `Function::stitching_bounds`) when a native gradient draws it exactly (extended past the box drawn in; a radial one with no background colour); (24) `DecodedImageCache`: decoded image pixels shared across renders through the interpreter cache within a 64 MiB budget (an image-heavy page decoded its thousands of images again per tile and per scale), keyed by every decode input (`DecodedImageKey`: the stream's place and dictionary bytes, its object, mask/stencil/raster output, interpolation, decode target size, preview or final); images under a transfer function and inline images are not cached, an image over the budget is never kept, and a poisoned lock decodes uncached; (25) Lab colour spaces honour their own `/WhitePoint`: L\*a\*b\* goes to XYZ relative to that white, is Bradford-adapted to D65 and converted to sRGB (ISO 32000-2 §8.6.5.4) instead of through `LAB.icc` as if every white were D50 (a D65 blue came out 45 levels too red; the profile is removed); components normalised to [0, 1] (`manual_scale`, an `/Indexed` palette over Lab) map onto L\* 0..100 and the a\*/b\* `/Range` | (1) `raster::tests::widget_appearance_states`; (2, 3) `xtask text-oracle` on corpus/pdfjs issue918; (4) `raster::tests::layer_override_hides_content`; (5) `raster::tests::soft_mask_without_group_colour_space_is_applied`; (6) `raster::tests::self_referencing_tiling_pattern_terminates`, `raster::tests::self_referencing_type3_glyph_terminates`; (7) `raster::tests::huge_cid_width_ranges_terminate`; (8) `raster::tests::hiding_comments_keeps_fields`; (9) `raster::tests::absurd_mask_and_fax_sizes_are_refused_before_decoding`; (10) `raster::tests::type3_glyphs_that_fan_out_into_themselves_finish`; (11) `raster::tests::type1_font_programs_with_long_runs_of_numbers_load_quickly`; (12) `raster::tests::degenerate_annotation_appearances_are_skipped`; (13) `raster::tests::highlights_without_an_appearance_are_drawn`; (14) `raster::tests::malformed_simple_font_widths_keep_later_widths_and_the_font`, `raster::tests::huge_simple_font_first_char_terminates`; (15) `raster::tests::annotation_appearance_box_maps_onto_rect`; (16) `raster::tests::clipping_path_applies_after_any_painting_operator`; (17) `raster::tests::dropping_the_pool_cancels_its_renders`, `raster::tests::the_watchdog_stops_the_render_it_gives_up_on`; (18) `raster::tests::page_contents_and_nested_forms_share_one_budget`, `raster::tests::content_painted_again_and_again_keeps_drawing`, `raster::tests::sampled_functions_with_billions_of_samples_are_refused`; (19) `raster::tests::mesh_shadings_far_off_the_page_render`, `raster::tests::many_overlapping_mesh_triangles_finish`, `raster::tests::mesh_shadings_with_many_patches_or_no_rows_finish`; (20) `raster::tests::function_limits_calculator_arithmetic_and_depth`, `raster::tests::function_limits_public_construction_work_and_depth`, `raster::tests::function_limits_stitching_cycles_are_refused_and_shared_children_work`, `raster::tests::function_limits_invalid_calculator_keeps_the_page_renderable`; (21) `raster::tests::cids_a_substitute_lacks_are_not_drawn_as_other_glyphs` |; (22) `raster::tests::variation_sequences_draw_their_kanji_in_a_substitute`; (23) `raster::tests::axial_and_radial_shadings_draw_as_native_gradients`; (24) `raster::tests::tiles_match_full_render`, `raster::tests::skipping_paths_off_the_canvas_changes_no_pixel` (both decode through the cache), `raster::tests::decoded_images_with_identical_dictionaries_keep_their_own_pixels`, `raster::tests::decoded_images_under_a_transfer_function_are_not_shared`; (25) `raster::tests::lab_honours_the_space_white_point`, `raster::tests::indexed_palettes_over_lab_keep_their_colours` diff --git a/vendor/hayro-interpret/src/x_object.rs b/vendor/hayro-interpret/src/x_object.rs index 68d97d5555..d3654871a7 100644 --- a/vendor/hayro-interpret/src/x_object.rs +++ b/vendor/hayro-interpret/src/x_object.rs @@ -667,23 +667,38 @@ fn decode_raster( ctx.bits_per_component, )?; - let mut f32_data = apply_decode_array( + // PdfCraft patch (28): an image with few possible pixel values converts each of them once. + let mut rgb_data = if let Some(rgb) = rgb_via_lookup( &components, - &ctx.color_space, - ctx.bits_per_component, - &ctx.decode_arr, - )?; - - fix_image_length(&mut f32_data, ctx.width, &mut height, 0.0, &ctx.color_space)?; - - let mut rgb_data = get_rgb_data( - &f32_data, ctx.width, height, ctx.scale_factors, &ctx.color_space, + ctx.bits_per_component, + &ctx.decode_arr, obj.interpolate, - ); + ) { + height = rgb.height; + Some(rgb) + } else { + let mut f32_data = apply_decode_array( + &components, + &ctx.color_space, + ctx.bits_per_component, + &ctx.decode_arr, + )?; + + fix_image_length(&mut f32_data, ctx.width, &mut height, 0.0, &ctx.color_space)?; + + get_rgb_data( + &f32_data, + ctx.width, + height, + ctx.scale_factors, + &ctx.color_space, + obj.interpolate, + ) + }; if let Some(transfer_function) = &obj.transfer_function && let Some(rgb_data) = &mut rgb_data @@ -1034,27 +1049,136 @@ fn apply_decode_array( bits_per_component: u8, decode: &[(f32, f32)], ) -> Option> { - let interpolate = |n: f32, d_min: f32, d_max: f32| { - interpolate( - n, - 0.0, - 2.0_f32.powi(bits_per_component as i32) - 1.0, - d_min, - d_max, - ) - }; - let mut decoded_arr = vec![]; for pixel in components.chunks(color_space.num_components() as usize) { - for (component, (d_min, d_max)) in pixel.iter().zip(decode) { - decoded_arr.push(interpolate(*component as f32, *d_min, *d_max)); + for (component, range) in pixel.iter().zip(decode) { + decoded_arr.push(decode_sample(*component, bits_per_component, *range)); } } Some(decoded_arr) } +/// One sample mapped through its `/Decode` range (ISO 32000-2 §8.9.5.2). PdfCraft patch (28): +/// shared by [`apply_decode_array`] and [`rgb_via_lookup`], so both decode alike. +fn decode_sample(sample: u16, bits_per_component: u8, (d_min, d_max): (f32, f32)) -> f32 { + interpolate( + sample as f32, + 0.0, + 2.0_f32.powi(bits_per_component as i32) - 1.0, + d_min, + d_max, + ) +} + +/// PdfCraft patch (28): the most bits a pixel's samples may hold together for +/// [`rgb_via_lookup`] (a table of 65,536 colours). +const MAX_LOOKUP_BITS: usize = 16; + +/// PdfCraft patch (28): the RGB of an image whose pixels hold at most [`MAX_LOOKUP_BITS`] bits +/// of samples, converted once per possible pixel value rather than once per pixel, at the +/// height `fix_image_length` gives. `None` when the table would not pay off (the image has no +/// more pixels than it has entries) or does not apply; the caller then converts every pixel. +/// +/// The colour of a pixel depends only on its own samples, so the result is the per-pixel one, +/// except where a tint transform gives the wrong number of components. The per-pixel batch then +/// shifts the colours after that pixel. The table converts each entry on its own +/// ([`convert_each`]), so a wrong-sized entry changes only itself, and a value the image never +/// uses cannot change the colour of one it does. +/// A Separation or DeviceN tint transform is a PDF function, often a PostScript calculator +/// program: run once per pixel, an 8-bit spot-colour background of 3304 × 4105 took four +/// seconds and 430 MB to draw; looked up from 256 entries it takes milliseconds. +#[allow(clippy::too_many_arguments)] +fn rgb_via_lookup( + components: &[u16], + width: u32, + height: u32, + scale_factors: (f32, f32), + cs: &ColorSpace, + bits_per_component: u8, + decode: &[(f32, f32)], + interpolate: bool, +) -> Option { + let n = cs.num_components() as usize; + let bpc = usize::from(bits_per_component); + let bits = bpc.checked_mul(n)?; + // `apply_decode_array` decodes only the samples `/Decode` has a range for: leave a short + // array to it, and every sample size it doesn't read. + if n == 0 || bits == 0 || bits > MAX_LOOKUP_BITS || bpc > 16 || decode.len() < n { + return None; + } + // The length rules of `fix_image_length`: extra samples are dropped; missing ones shorten + // the image to the rows begun, the last row padded with decoded zeros. + let row_len = usize::try_from(width).ok()?.checked_mul(n)?; + let wanted = row_len.checked_mul(usize::try_from(height).ok()?)?; + let (height, used) = if wanted <= components.len() { + (height, wanted) + } else { + (u32::try_from(components.len().div_ceil(row_len)).ok()?, components.len()) + }; + let pixels = usize::try_from(width).ok()?.checked_mul(usize::try_from(height).ok()?)?; + let total = pixels.checked_mul(n)?; + let entries = 1usize << bits; + if pixels <= entries { + return None; + } + + // Every possible pixel value, its samples in order, decoded as `apply_decode_array` would. + let mask = (1usize << bpc) - 1; + let mut table_input = Vec::with_capacity(entries * n); + for code in 0..entries { + for (k, range) in decode.iter().take(n).enumerate() { + let sample = ((code >> (bpc * (n - 1 - k))) & mask) as u16; + table_input.push(decode_sample(sample, bits_per_component, *range)); + } + } + let mut table = vec![0; entries * 3]; + convert_each(cs, &table_input, n, &mut table); + + let mut data = vec![0; pixels.checked_mul(3)?]; + let whole = used / n; + let samples = components.get(..whole * n)?; + for (pixel, rgb) in samples.chunks_exact(n).zip(data.chunks_exact_mut(3)) { + let code = pixel.iter().fold(0usize, |code, s| (code << bpc) | (usize::from(*s) & mask)); + if let Some(colour) = table.get(code * 3..code * 3 + 3) { + rgb.copy_from_slice(colour); + } + } + // The pixels past the samples present (at most a row): converted as before, padding and all. + if whole < pixels { + let tail: Vec = (whole * n..total) + .map(|i| match (components.get(i), decode.get(i % n)) { + (Some(s), Some(range)) if i < used => decode_sample(*s, bits_per_component, *range), + _ => 0.0, + }) + .collect(); + let _ = cs.convert_f32(&tail, data.get_mut(whole * 3..)?, false); + } + + Some(RgbData { + data, + width, + height, + interpolate, + scale_factors, + }) +} + +/// PdfCraft patch (28): `input`, `n` decoded samples a pixel, converted into `output` (RGB) one +/// pixel at a time. A Separation or DeviceN tint transform may give more or fewer components +/// than its alternate space takes (a Type 4 program's count is only warned about): converted as +/// one batch, such a result shifts every pixel after it; converted alone, it affects only its +/// own. A failed conversion leaves black, as in `get_rgb_data`. +fn convert_each(cs: &ColorSpace, input: &[f32], n: usize, output: &mut [u8]) { + if n == 0 { + return; + } + for (pixel, rgb) in input.chunks_exact(n).zip(output.chunks_exact_mut(3)) { + let _ = cs.convert_f32(pixel, rgb, false); + } +} + /// PdfCraft patch: whether an image or mask of `width` × `height` may be decoded (at most /// 2^28 pixels, the renderer's `MAX_IMAGE_PIXELS`). pub fn image_size_ok(width: u32, height: u32) -> bool {