diff --git a/ROADMAP.md b/ROADMAP.md index 9cfc7f3ea..b23c200bf 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -156,6 +156,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md | 8 | 1.0 polish: performance, right-to-left, missing languages, keyboard-only (M14) | 55โ€“95 | ## Progress log +- **2026-10-12 (M2, inline-image boundaries):** Unfiltered inline images with known sample layouts keep pixel bytes that look like the `EI` terminator when content is rewritten. Regression tests cover row padding, colour-space and parameter aliases, masks and malformed sizes; classic and packed saved PDFs retain the pixels in independent readers and renderers. Filtered and resource-defined layouts retain the existing tolerant heuristic. Estimates unchanged. - **2026-10-12 (M3, drawing markups):** While a comment is being drawn, the pointer in a band along any edge of the view (or just past it) scrolls the view up, down, left or right, so points beyond what is shown can be reached. A polygon, cloud or run of connected lines takes back its last point with Backspace, Delete or Undo (the first point's removal cancels it). Edge scrolling has unit tests of its maths only; it has not been exercised end to end in the app by an automated test. - **2026-10-12 (gap 6, #1319, #1305, #1324):** Editing an XFA form keeps a radio item's stored text when it contains a space, so a later layout still selects it. Replace Pages on a split selection replaces those pages and leaves the gap; editing the dialog range still replaces the span. Marking a figure decorative rewrites only that figure's content stream, so a header, added text and unknown stream entries stay. Estimates unchanged. - **2026-10-15 (M14, gap 24):** `misc.localization` and `misc.installers` move from planned to partial. Localization: 22 catalogs ship, but Chinese shows tofu in releases, Telugu has no release font and Arabic is not mirrored. Installers: macOS and Windows releases are signed, Linux packages are not. The Acrobat menu diff itself remains open. Estimates unchanged. diff --git a/crates/content/README.md b/crates/content/README.md index d9d8af10c..6ab984dd4 100644 --- a/crates/content/README.md +++ b/crates/content/README.md @@ -12,3 +12,8 @@ let m = Matrix::from_operands(&op.operands); // cm / Tm arithmetic, invert, bbo The parser is tolerant (unreadable bytes are skipped and counted, never fatal) and never panics. `redact` and (later) `edit` build their interpreters on it; the graphics-state tracker and the `Patch` API of architecture ยง6 grow here. + +Unfiltered inline images with built-in or Indexed colour spaces, and image masks, use their +sample layout (including per-row byte padding) to locate `EI`. This preserves pixel bytes +that look like the terminator. Filtered images and resource-defined colour spaces retain +the tolerant delimiter heuristic; damaged or truncated layouts fall back to it too. diff --git a/crates/content/src/lib.rs b/crates/content/src/lib.rs index 4a8f077fe..32334e047 100644 --- a/crates/content/src/lib.rs +++ b/crates/content/src/lib.rs @@ -175,12 +175,83 @@ fn inline_image(data: &[u8], pos: usize) -> Result<(Dict, Vec, usize), usize } // One whitespace byte separates ID from the data. let begin = (lx.pos + 1).min(data.len()); - Ok(match end_of_image(data, begin) { + let end = unfiltered_image_end(data, begin, &dict).or_else(|| end_of_image(data, begin)); + Ok(match end { Some((end_data, end)) => (dict, data[begin..end_data].to_vec(), end), None => (dict, data[begin..].to_vec(), data.len()), }) } +/// An unfiltered image's samples determine its length, including byte padding per row. +/// Binary samples may themselves contain whitespace + `EI` + whitespace, so use the known +/// length when the actual terminator follows it. Unknown or damaged layouts keep the heuristic. +fn unfiltered_image_end(data: &[u8], begin: usize, dict: &Dict) -> Option<(usize, usize)> { + for key in [b"Filter".as_slice(), b"F"] { + if let Some(filter) = dict.get(key) + && !matches!(filter, Object::Null) + && !matches!(filter, Object::Array(filters) if filters.is_empty()) + { + return None; + } + } + for (full, short) in + [(b"Width".as_slice(), b"W".as_slice()), (b"Height", b"H"), (b"ColorSpace", b"CS"), (b"BitsPerComponent", b"BPC"), (b"ImageMask", b"IM")] + { + if let (Some(full), Some(short)) = (dict.get(full), dict.get(short)) + && full != short + { + return None; + } + } + let get = |full: &[u8], short: &[u8]| dict.get(full).or_else(|| dict.get(short)); + let width = usize::try_from(get(b"Width", b"W")?.as_int()?).ok().filter(|n| *n > 0)?; + let height = usize::try_from(get(b"Height", b"H")?.as_int()?).ok().filter(|n| *n > 0)?; + let mask = match get(b"ImageMask", b"IM") { + None | Some(Object::Null | Object::Bool(false)) => false, + Some(Object::Bool(true)) => true, + _ => return None, + }; + let bits = match get(b"BitsPerComponent", b"BPC") { + None | Some(Object::Null) => None, + Some(value) => Some(value.as_int()?), + }; + let (components, bits) = if mask { + if bits.is_some_and(|bits| bits != 1) { + return None; + } + (1usize, 1usize) + } else { + let (components, max_bits) = match get(b"ColorSpace", b"CS")? { + Object::Name(name) => match name.as_slice() { + b"DeviceGray" | b"G" => (1, 16), + b"DeviceRGB" | b"RGB" => (3, 16), + b"DeviceCMYK" | b"CMYK" => (4, 16), + _ => return None, + }, + Object::Array(space) if space.len() == 4 && matches!(space.first().and_then(Object::as_name), Some(b"Indexed" | b"I")) => (1, 8), + _ => return None, + }; + let bits = match bits? { + bits @ (1 | 2 | 4 | 8 | 16) => bits as usize, + _ => return None, + }; + if bits > max_bits { + return None; + } + (components, bits) + }; + let row = width.checked_mul(components)?.checked_mul(bits)?.div_ceil(8); + let end_data = begin.checked_add(row.checked_mul(height)?)?; + let after = data.get(end_data..)?; + if !after.first().is_some_and(|b| is_ws(*b)) { + return None; + } + let separator = after.iter().position(|b| !is_ws(*b))?; + let ei = end_data.checked_add(separator)?; + let end = ei.checked_add(2)?; + if data.get(ei..end) == Some(b"EI") && data.get(end).is_none_or(|b| is_ws(*b) || is_delim(*b)) { Some((end_data, end)) } else { None } +} + /// Where inline image data starting at `begin` ends: (end of the data, position after `EI`). The /// data ends at whitespace + `EI` + (whitespace, a delimiter or the end). fn end_of_image(data: &[u8], begin: usize) -> Option<(usize, usize)> { diff --git a/crates/content/src/tests.rs b/crates/content/src/tests.rs index 99f40d4e3..18e0b8cbc 100644 --- a/crates/content/src/tests.rs +++ b/crates/content/src/tests.rs @@ -38,6 +38,171 @@ fn inline_images_stay_whole() { assert_eq!(again.ops[2].inline, p.ops[2].inline); } +#[test] +fn unfiltered_inline_image_keeps_whitespace_bounded_ei_pixels() { + let src = b"q 4 0 0 1 0 0 cm BI /W 4 /H 1 /BPC 8 /CS /G ID EI \nEI Q"; + let parsed = parse(src); + assert_eq!(parsed.ops[2].inline.as_ref().unwrap().1, b" EI "); + assert_eq!(parsed.ops.iter().map(|op| op.op.as_slice()).collect::>(), [b"q".as_slice(), b"cm", b"BI", b"Q"]); + let rewritten = serialize_ops(&parsed.ops); + assert_eq!(parse(&rewritten).ops[2].inline, parsed.ops[2].inline); +} + +fn inline_pixels(parameters: &[u8], pixels: &[u8]) -> Parsed { + let mut src = b"q BI ".to_vec(); + src.extend_from_slice(parameters); + src.extend_from_slice(b" ID "); + src.extend_from_slice(pixels); + src.extend_from_slice(b"\nEI Q"); + parse(&src) +} + +#[test] +fn unfiltered_inline_image_lengths_use_components_and_parameter_aliases() { + for (parameters, pixels) in [ + (b"/W 4 /H 1 /CS /G /BPC 8".as_slice(), b" EI ".as_slice()), + (b"/W 4 /H 1 /CS /G /BPC 8 /Filter null", b" EI "), + (b"/W 4 /H 1 /CS /G /BPC 8 /F []", b" EI "), + (b"/W 4 /Width 4 /H 1 /CS /G /BPC 8", b" EI "), + (b"/Width 4 /Height 1 /ColorSpace /DeviceGray /BitsPerComponent 8", b" EI "), + (b"/W 2 /H 1 /CS /RGB /BPC 8", b"\0 EI \xff"), + (b"/Width 2 /Height 1 /ColorSpace /DeviceRGB /BitsPerComponent 8", b"\0 EI \xff"), + (b"/W 1 /H 1 /CS /CMYK /BPC 8", b" EI "), + (b"/Width 1 /Height 1 /ColorSpace /DeviceCMYK /BitsPerComponent 8", b" EI "), + ] { + let parsed = inline_pixels(parameters, pixels); + assert_eq!(parsed.skipped, 0); + assert_eq!(parsed.ops.iter().map(|op| op.op.as_slice()).collect::>(), [b"q".as_slice(), b"BI", b"Q"]); + assert_eq!(parsed.ops[1].inline.as_ref().unwrap().1, pixels); + assert_eq!(parse(&serialize_ops(&parsed.ops)).ops[1].inline, parsed.ops[1].inline); + } +} + +#[test] +fn unfiltered_inline_image_rows_are_byte_aligned_for_every_sample_depth() { + // Two rows of two bytes each. Padding belongs to each row, not only the final row. + for (bits, width) in [(1, 9), (2, 5), (4, 3), (8, 2), (16, 1)] { + let parameters = format!("/W {width} /H 2 /BPC {bits} /CS /G"); + let parsed = inline_pixels(parameters.as_bytes(), b" EI "); + assert_eq!(parsed.ops[1].inline.as_ref().unwrap().1, b" EI ", "bits={bits}"); + assert_eq!(parsed.ops.len(), 3); + } +} + +#[test] +fn unfiltered_inline_masks_default_to_one_bit_and_indexed_images_have_one_component() { + for parameters in [b"/W 9 /H 2 /IM true".as_slice(), b"/Width 9 /Height 2 /ImageMask true /BitsPerComponent 1"] { + let parsed = inline_pixels(parameters, b" EI "); + assert_eq!(parsed.ops[1].inline.as_ref().unwrap().1, b" EI "); + assert_eq!(parsed.ops.len(), 3); + } + for name in ["Indexed", "I"] { + // A valid 256-entry RGB lookup table; samples are indices regardless of the base space. + let parameters = format!("/W 4 /H 1 /BPC 8 /CS [/{name} /DeviceRGB 255 <{}>]", "00".repeat(768)); + let parsed = inline_pixels(parameters.as_bytes(), b" EI "); + assert_eq!(parsed.ops[1].inline.as_ref().unwrap().1, b" EI "); + assert_eq!(parsed.ops.len(), 3); + // Indexed samples permit at most 8 bits, even though device spaces permit 16. + let invalid = parameters.replace("/W 4", "/W 2").replace("/BPC 8", "/BPC 16"); + let parsed = inline_pixels(invalid.as_bytes(), b" EI "); + assert!(parsed.ops[1].inline.as_ref().unwrap().1.is_empty()); + } +} + +#[test] +fn unfiltered_inline_image_terminator_must_follow_its_samples() { + for separator in [b" ".as_slice(), b"\r\n", b"\0\t\x0c\n"] { + let mut src = b"BI /W 4 /H 1 /BPC 8 /CS /G ID EI ".to_vec(); + src.extend_from_slice(separator); + src.extend_from_slice(b"EI/Q q"); + let parsed = parse(&src); + assert_eq!(parsed.ops[0].inline.as_ref().unwrap().1, b" EI "); + assert_eq!(&src[parsed.ops[0].span.clone()], &src[..src.len() - b"/Q q".len()]); + } + let parsed = parse(b"BI /W 4 /H 1 /BPC 8 /CS /G ID EI \nEI"); + assert_eq!(parsed.ops[0].inline.as_ref().unwrap().1, b" EI "); + // EI is a keyword: a prefix of a larger token is not a terminator. + let parsed = parse(b"BI /W 4 /H 1 /BPC 8 /CS /G ID abcd EIx\nEI Q"); + assert_eq!(parsed.ops[0].inline.as_ref().unwrap().1, b"abcd EIx"); +} + +#[test] +fn filtered_unknown_and_damaged_inline_images_keep_the_existing_fallback() { + for parameters in [ + b"/W 4 /H 1 /BPC 8 /CS /G /F /A85".as_slice(), + b"/W 4 /H 1 /BPC 8 /CS /ResourceColor", + b"/W 0 /H 1 /BPC 8 /CS /G", + b"/W -1 /H 1 /BPC 8 /CS /G", + b"/W 4 /H -1 /BPC 8 /CS /G", + b"/W 4 /H 1 /BPC 3 /CS /G", + b"/W 4 /H 1 /IM true /BPC 8", + b"/W 4 /H 1 /IM true /BPC /Bad", + b"/Width 4 /W 1 /H 1 /BPC 8 /CS /G", + b"/W 4 /H 1 /BPC 8 /CS /G /ImageMask false /IM true", + b"/W 9223372036854775807 /H 9223372036854775807 /BPC 16 /CS /CMYK", + b"/W 9223372036854775807 /H 2 /BPC 8 /CS /G", + ] { + let parsed = inline_pixels(parameters, b"! EI !!~>"); + assert_eq!(parsed.ops[1].inline.as_ref().unwrap().1, b"!", "{}", String::from_utf8_lossy(parameters)); + } + // A short payload still uses the tolerant recovery, including when there is no EI. + for src in [b"BI /W 4 /H 1 /BPC 8 /CS /G ID abc\nEI Q".as_slice(), b"BI /W 4 /H 1 /BPC 8 /CS /G ID abc"] { + assert_eq!(parse(src).ops[0].inline.as_ref().unwrap().1, b"abc"); + } +} + +fn inline_pdf(content: &[u8], object_streams: bool) -> Vec { + use pdfcraft_cos::{Document, SaveOptions, Stream, write_full}; + + let mut doc = Document::new_empty(); + let pages = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Pages").unwrap(); + let contents = doc.add(Object::Stream(Stream::flate(Dict::new(), content))); + let page = doc.add(Object::Dict( + [ + (b"Type".to_vec(), Object::name("Page")), + (b"Parent".to_vec(), Object::Ref(pages)), + (b"MediaBox".to_vec(), Object::Array([0, 0, 4, 1].map(Object::Int).to_vec())), + (b"Resources".to_vec(), Object::Dict(Dict::new())), + (b"Contents".to_vec(), Object::Ref(contents)), + ] + .into_iter() + .collect(), + )); + doc.update_dict(pages, |dict| { + dict.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(page)])); + dict.set(b"Count".to_vec(), Object::Int(1)); + }) + .unwrap(); + write_full(&doc, &SaveOptions { object_streams, ..SaveOptions::default() }).unwrap() +} + +#[test] +fn rewritten_inline_image_pixels_survive_saving_and_reopening_a_pdf() { + use pdfcraft_cos::{Document, Object}; + use std::sync::Arc; + + let original = b"q 4 0 0 1 0 0 cm BI /W 4 /H 1 /BPC 8 /CS /G ID EI \nEI Q"; + let rewritten = serialize_ops(&parse(original).ops); + for object_streams in [false, true] { + let bytes = inline_pdf(&rewritten, object_streams); + let doc = Document::open(Arc::new(bytes.clone())).unwrap(); + assert!(doc.repair_log().is_empty()); + let pages = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Pages").unwrap(); + let pages = doc.get(pages); + let page = doc.resolve(&pages.as_dict().unwrap().get(b"Kids").unwrap().as_array().unwrap()[0]); + let content = doc.resolve(page.as_dict().unwrap().get(b"Contents").unwrap()); + let Object::Stream(stream) = &*content else { panic!("page content stream") }; + assert_eq!(parse(&stream.decoded_strict().unwrap()).ops[2].inline.as_ref().unwrap().1, b" EI "); + // Opt-in artifacts for independent readers/renderers; synthetic PDFs are never committed. + if let Some(dir) = std::env::var_os("PDFCRAFT_INLINE_FIXTURE_DIR") { + let dir = std::path::PathBuf::from(dir); + std::fs::create_dir_all(&dir).unwrap(); + std::fs::write(dir.join("original.pdf"), inline_pdf(original, false)).unwrap(); + std::fs::write(dir.join(if object_streams { "rewritten-packed.pdf" } else { "rewritten-classic.pdf" }), bytes).unwrap(); + } + } +} + #[test] fn damage_is_skipped_not_fatal() { let p = parse(b"1 0 0 RG ) ] 10 20 m 30 40 l S (unterminated"); diff --git a/docs/roadmap.md b/docs/roadmap.md index 01ba25378..479189b75 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -178,6 +178,7 @@ M2 (renderer), M7 (content editing), M10 (OCR and Office), M11 (Preflight and st | Date | Change | Summary | |---|---|---| +| 2026-10-12 | minor | M2: preserve whitespace-bounded EI bytes in known unfiltered inline images; row padding, masks and aliases tested, with independent saved-PDF pixel checks. Estimates unchanged. | | 2026-10-12 | minor | M1 #1338: a full rewrite keeps unknown trailer entries and objects reachable only from them. Estimates unchanged. | | 2026-10-12 | minor | M7 #1443: adding a word to a multi-line paragraph keeps its line breaks. Composite subset fonts whose `/W` leaves out default-width glyphs (Chrome, Skia) no longer measure those letters as missing, and paragraphs drawn turned on the page group into rows along their own baselines. Estimates unchanged. | | 2026-10-14 | minor | M14: Italian catalog completed (2,528 entries) and dropped from the โ‰ˆ 2,000โ€“2,140-entry group; estimates unchanged |