diff --git a/ROADMAP.md b/ROADMAP.md index b1283e775..1db693573 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -172,6 +172,8 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md ## Progress log +- **2026-10-22 (M2, #3107 / PR #3438):** Inline-image parsing falls back to the existing EI heuristic when missing declared samples, headers or terminators exceed the content-window budget, instead of waiting for an impossible allocation. Exact-length parsing for available samples and ordinary large streamed images is retained. Estimates unchanged. + - **2026-10-19 (M14, #3373):** The main menu and its long submenus scroll in short windows, keeping their commands reachable. Estimates unchanged. - **2026-10-19 (M3, #3387):** Save-prompt shortcuts yield to later consent and utility dialogs; a synthetic UI regression checks that an edited document stays open. Estimates unchanged. diff --git a/docs/roadmap.md b/docs/roadmap.md index 5d73511e9..757162983 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -111,7 +111,7 @@ total matches the parity estimate. |---|---|---|---|---|---| | M0 | Skeleton: workspace, xtask gates, CI | 15–30 | 98% | 1–2 | GitHub workflow, `deny.toml` (licence audit of every dependency), parity checklist (826 features, `xtask parity`) done. Missing: remaining crate stubs, testkit/oracle crates | | M1 | COS: filters, crypt, parser, xref, writer | 120–200 | 80% | 20–40 | Done:
- filters and crypt: every standard-security revision R2–R6 (RC4, AES-128/256), SASLprep, permissions, creating encryption;
- cos: parse and repair (including invalid xref object-number ranges), decoded-stream limits including unfiltered data, decrypt on load, re-encrypt on save, incremental and full writing.
Corpus: open/edit/save passes on 958 files, and all 7 password-protected files open.
Full saves now pack objects into compressed object streams. An unchanged incremental save of a classic table writes a zero-length cross-reference subsection instead of a section with no subsections (#1339). A full rewrite keeps unknown trailer entries, including objects reachable only from them (#1338). Fuzzing runs nightly (`xtask fuzz`). Raw-deflate fallback preserves decoding-limit errors, including through tolerant COS stream decoding. Missing: ≥ 250 tests, own image codecs | -| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. | +| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. The renderer keeps its one-point crop-size floor in physical points after UserUnit scaling, with regression coverage for crop origins, rotations and f32 reciprocal rounding. Existing unit policy and nearly-zero fallback are retained. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings | | M3 | Viewer app (native + web) | 80–150 | 90% | 10–20 | Acrobat-style shell, find, select, panels, tiles, web build, UI control channel for agents (opt-in). Missing: 60 fps test on a 500-page document, snapshot tests of every panel
Done since: fit visible, document title in the window; Linux middle-button scrolling; shared three-state theme preferences in the toolbar, View menu and control channel; numeric Linux portal events and idle wakeups completing #881, with native appearance inheritance retained (#446); late browser startup downloads preserve the current document and Save target (#171). Selected-text Copy in Edit and the canvas context menu is covered by focused regressions and native clipboard checks (#1225); document copy restrictions apply to the floating action too. #1364 candidate: absolute local GoToR links ask before target access; headless callers explicitly opt in. Native-opening candidate: chosen tools survive preparation and password retries; startup intake waits for pending preparation. Page-label navigation displays logical labels, retains the current page on unchanged submission and refreshes focused same-document replacements; clean file-watcher reload preserves the chosen scrolling mode, preventing a Single Page reader from jumping to page 1 during replacement. Synthetic control regressions cover these behaviors. #3149: page-list changes discard render demand based on old geometry; actual raster dimensions are checked against the GPU limit before upload. Synthetic crash and continued-rendering regressions cover the replacement path. Save-prompt shortcuts yield to dialogs above them; stacked link-consent UI regression covers Escape and Return. | diff --git a/vendor/hayro-syntax/src/content/mod.rs b/vendor/hayro-syntax/src/content/mod.rs index a18e91817..4dfc9eb6b 100644 --- a/vendor/hayro-syntax/src/content/mod.rs +++ b/vendor/hayro-syntax/src/content/mod.rs @@ -245,7 +245,12 @@ impl<'a> UntypedIter<'a> { // PdfCraft patch: an unfiltered inline image's byte length is determined by // its parameters (and `/L` or `/Length` when present and consistent), so // `EI` inside its samples does not cut the image short (#3107). - match unfiltered_image_end(stream_data, &dict, self.complete) { + match unfiltered_image_end( + stream_data, + &dict, + self.complete, + start_offset.saturating_sub(self.committed), + ) { UnfilteredEnd::Found(end_data, end_after_ei) => { let image_data = stream_data.get(..end_data)?; let stream = Stream::new(image_data, dict.clone()); @@ -971,33 +976,37 @@ fn unfiltered_image_len(dict: &Dict<'_>) -> Option { } /// PdfCraft patch: locate the deterministic `EI` terminator of an unfiltered inline image (#3107). -fn unfiltered_image_end(data: &[u8], dict: &Dict<'_>, complete: bool) -> UnfilteredEnd { +fn unfiltered_image_end(data: &[u8], dict: &Dict<'_>, complete: bool, header_bytes: usize) -> UnfilteredEnd { let Some(end_data) = unfiltered_image_len(dict) else { return UnfilteredEnd::Fallback; }; + // An inline image shares the content window's budget, not the larger decoded-image + // budget. Include its dictionary/header and the shortest whitespace + EI + delimiter + // look-ahead, or one more byte beyond any separator whitespace already present. + // Missing bytes that cannot fit must use the existing heuristic, not pin the window + // and discard following content. Already-available exact ends still win. + let missing_end = if complete + || end_data + .checked_add(4) + .and_then(|minimum| data.len().checked_add(1).map(|next| minimum.max(next))) + .and_then(|needed| needed.checked_add(header_bytes)) + .is_none_or(|needed| needed > crate::MAX_DECODED_STREAM) + { + UnfilteredEnd::Fallback + } else { + UnfilteredEnd::NeedsMore + }; let Some(after) = data.get(end_data..) else { - return if complete { - UnfilteredEnd::Fallback - } else { - UnfilteredEnd::NeedsMore - }; + return missing_end; }; let Some(&first) = after.first() else { - return if complete { - UnfilteredEnd::Fallback - } else { - UnfilteredEnd::NeedsMore - }; + return missing_end; }; if !is_white_space_character(first) { return UnfilteredEnd::Fallback; } let Some(separator) = after.iter().position(|&b| !is_white_space_character(b)) else { - return if complete { - UnfilteredEnd::Fallback - } else { - UnfilteredEnd::NeedsMore - }; + return missing_end; }; let Some(rest) = after.get(separator..) else { return UnfilteredEnd::Fallback; @@ -1006,11 +1015,7 @@ fn unfiltered_image_end(data: &[u8], dict: &Dict<'_>, complete: bool) -> Unfilte return UnfilteredEnd::Fallback; } let Some(&second) = rest.get(1) else { - return if complete { - UnfilteredEnd::Fallback - } else { - UnfilteredEnd::NeedsMore - }; + return missing_end; }; if second != b'I' { return UnfilteredEnd::Fallback; @@ -1022,13 +1027,14 @@ fn unfiltered_image_end(data: &[u8], dict: &Dict<'_>, complete: bool) -> Unfilte Some(&b) if !is_regular_character(b) => UnfilteredEnd::Found(end_data, end_after_ei), Some(_) => UnfilteredEnd::Fallback, None if complete => UnfilteredEnd::Found(end_data, end_after_ei), - None => UnfilteredEnd::NeedsMore, + None => missing_end, } } #[cfg(test)] mod tests { use super::{ContentTokenizer, UntypedIter}; + use crate::reader::ReaderExt; fn operators(data: &[u8]) -> Vec> { let mut tokenizer = UntypedIter::new(data); @@ -1216,6 +1222,63 @@ BT /F1 14 Tf 72 550 Td (After) Tj ET"; } } + #[test] + fn oversized_unfiltered_inline_image_does_not_hold_following_content() { + for length in [crate::MAX_DECODED_STREAM - 1, crate::MAX_DECODED_STREAM, crate::MAX_DECODED_STREAM + 1] { + // The next BI supplies the existing EI heuristic's bounded look-ahead. The + // stream is not exhausted: an impossible declaration must not pin this window. + let data = format!( + "BI /W {length} /H 1 /CS /G /BPC 8 ID x EI BT (After) Tj ET \ + BI /W 1 /H 1 /CS /G /BPC 8 ID y EI Q" + ); + let mut tokenizer = UntypedIter::new_window(data.as_bytes(), false); + for expected in [b"BI".as_slice(), b"BT", b"Tj", b"ET"] { + let instruction = tokenizer.next_instruction().expect("following content survives"); + assert_eq!(instruction.operator.as_ref(), expected, "declared length {length}"); + } + } + } + + #[test] + fn unfiltered_inline_image_budget_includes_header_and_terminator() { + let mut reader = crate::reader::Reader::new(b"/W 1 /H 1 /CS /G /BPC 8 ID "); + let inline = reader + .read_without_context::>() + .expect("synthetic inline dictionary"); + let dict = inline.get_dict(); + let fits = crate::MAX_DECODED_STREAM - 5; // One sample + whitespace/EI/delimiter. + assert!(matches!(super::unfiltered_image_end(b"", dict, false, fits), super::UnfilteredEnd::NeedsMore)); + assert!(matches!(super::unfiltered_image_end(b"x ", dict, false, fits), super::UnfilteredEnd::Fallback)); + assert!(matches!(super::unfiltered_image_end(b"x EI", dict, false, fits + 1), super::UnfilteredEnd::Fallback)); + assert!(matches!(super::unfiltered_image_end(b"x EI", dict, true, usize::MAX), super::UnfilteredEnd::Found(1, 4))); + for header in [fits + 1, usize::MAX] { + assert!(matches!(super::unfiltered_image_end(b"", dict, false, header), super::UnfilteredEnd::Fallback)); + // The guard is only for missing bytes, never a reason to reinterpret available samples. + assert!(matches!(super::unfiltered_image_end(b"x EI Q", dict, false, header), super::UnfilteredEnd::Found(1, 4))); + } + } + + #[test] + fn large_unfiltered_inline_image_still_waits_for_its_samples() { + // A valid image larger than the interpreter's initial 64 KiB window must + // still use its exact length rather than execute operators in its samples. + let length = (64 << 10) + 1; + let mut data = format!("BI /W {length} /H 1 /CS /G /BPC 8 ID ").into_bytes(); + let sample_start = data.len(); + data.extend_from_slice(b"0EI Q BT (Injected) Tj ET q"); + data.resize(sample_start + length, b'0'); + data.extend_from_slice(b" EI BT (After) Tj ET"); + let mut partial = UntypedIter::new_window(&data[..64 << 10], false); + assert!(partial.next_instruction().is_none()); + assert!(partial.needs_more); + assert_eq!(partial.committed, 0); + assert_eq!(operators(&data), [b"BI".to_vec(), b"BT".to_vec(), b"Tj".to_vec(), b"ET".to_vec()]); + let expected = slice_instructions(&data); + for piece in [4096, 64 << 10, data.len()] { + assert_eq!(windowed_instructions(&data, || piece), expected, "piece size {piece}"); + } + } + #[test] fn unfiltered_inline_image_variants_and_fallbacks() { // ImageMask with 1-bit row padding (width 10 -> 2 bytes per row, height 2 -> 4 bytes)