diff --git a/Cargo.lock b/Cargo.lock index 93c047eef..6ce24d013 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4235,6 +4235,7 @@ dependencies = [ "pdfcraft-annot", "pdfcraft-content", "pdfcraft-cos", + "pdfcraft-filters", "pdfcraft-model", "pdfcraft-render", "thiserror 2.0.21", diff --git a/ROADMAP.md b/ROADMAP.md index d82a53528..3b3a53159 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,6 +4,10 @@ > **Last reviewed:** 2026-10-23 | **Last updated:** 2026-10-23 | **Change:** minor (M8, #3205 follow-up: labelled Italian international phone numbers with a slash match whole; estimates unchanged) > **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M7, #2636 partial: pasting added text boxes and images in the engine and through `content_copy`; estimates unchanged) +> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M11: image optimization accounts for page-local UserUnit; estimates unchanged) +> **Last reviewed:** 2026-10-23 | **Last updated:** 2026-10-23 | **Change:** minor (M10: Word table cells use the widths of their own grid columns; estimates unchanged) +> **Last reviewed:** 2026-10-23 | **Last updated:** 2026-10-23 | **Change:** minor (M10: table spans retain text from covered slots in Office exports; estimates unchanged) +> **Last reviewed:** 2026-10-23 · **Last updated:** 2026-10-23 · **Change:** minor (M10: image export preserves soft-mask Decode opacity ranges; estimates unchanged) > **Last reviewed:** 2026-10-21 · **Last updated:** 2026-10-21 · **Change:** minor (M3, #3808: the Pages inspector has an independent 50–150% thumbnail size slider and buttons; selection and current-page following are retained, with headless ui.set coverage. Estimates unchanged.) > **Last reviewed:** 2026-10-20 | **Last updated:** 2026-10-20 | **Change:** minor (M11 #3507: image merging preserves content stream boundaries; estimates unchanged) > **Last reviewed:** 2026-10-20 | **Last updated:** 2026-10-20 | **Change:** minor (M10, #3532 partial: vector printing retains existing appearances with absent Subtype; estimates unchanged) @@ -56,6 +60,7 @@ > **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** minor (closed issues synced: #774, #756 and the #799–#821 batch; earlier: localized shortcut labels, #496; readiness table with hours per audience; full number as weighted sum 49%; earlier: second look: ready 40% → 47%, mainstream 55% added; core-workflow gate applied → alpha) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS) > **Last reviewed:** 2026-10-18 - **Last updated:** 2026-10-18 - **Change:** minor (M6 #2892: Push-button normal labels are editable per widget through Options and form_set_props, independently of field names; rendering, undo/redo and incremental/full save/reopen are covered. Estimates unchanged) > **Last reviewed:** 2026-10-18 - **Last updated:** 2026-10-18 - **Change:** minor (M12 #2589: Active XFA skips PDF-container fallback scripts while retaining requested messages and saved scripts; ordinary/static/unlaid-out documents and explicit console/field calls keep their existing behavior. Estimates unchanged) +> **Last reviewed:** 2026-10-18 - **Last updated:** 2026-10-18 - **Change:** minor (M7 #2592: Styled fallback fonts batch up to 240 glyphs per actual face and retain source styles. Budget refusal and distinct same-BaseFont descriptor weights have mutation-checked regressions; coverage and complex mixed runs remain limited. Estimates unchanged) > **Last reviewed:** 2026-10-18 - **Last updated:** 2026-10-18 - **Change:** minor (M7 #3187: Added-image automation normalizes extreme signed rotation angles before combining them with the current orientation; integer bounds, history, pixels, save/reopen and MCP have regressions. Estimates unchanged) > **Last reviewed:** 2026-10-18 - **Last updated:** 2026-10-18 - **Change:** minor (M12 #2569: Growable multiline XFA text reflows after edits, with undo/redo, save/reopen and print coverage; oversized fields still become read-only page text. Structured-data synchronization warnings retain accepted whitespace without relayout from stale data. Estimates unchanged) > **Last reviewed:** 2026-10-18 - **Last updated:** 2026-10-18 - **Change:** minor (M12 #2572: Every repeated page-master button resolves its own field to the shared XFA click script, including print buttons. Estimates unchanged) @@ -196,6 +201,15 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-23 (M7, #4280):** Edit text names each standard stand-in font after its face (`PCEdTimesBold`, ...) instead of one shared `PCEdHelv`. Rewriting a bold paragraph whose subset lacks an accent no longer repoints the paragraphs set earlier in a regular stand-in at Times-Bold, and the editor's frame stays on the paragraph being edited. Pages saved by older builds keep their shared name and are not repaired. Estimates unchanged. - **2026-10-23 (M8, #3205 follow-up):** Phone-pattern redaction matches complete labelled Italian geographic numbers such as `Tel. +39 0426/123456`, with synthetic matcher and mark/apply/save/reopen coverage; unlabelled international slash numbers remain unsupported. - **2026-10-23 (M7, #2636 partial):** A new `PasteContent` edit pastes an added text box or image copied in the same document or another, keeping its style, picture, turns, flips and crop, as one undo step; `content_copy` drives it headlessly. A paste lands beside earlier copies (18 pt steps) and stays on the page. A pasted image reuses its picture object in its own document and after the first paste into another; there a colour profile is dropped and CMYK colours stored losslessly are turned to RGB. The desktop keys and menus follow in separate PRs. Estimates unchanged. +- **2026-10-23 (M11, optimizer resolution):** Image downsampling includes each page's `/UserUnit` scale, preserving images already below the physical resolution threshold, including shared nested forms. + Synthetic measurement and optimization regressions cover direct and indirect units and page-local defaults. + Estimates unchanged. +- **2026-10-23 (M10, Word table widths):** Word export sizes each cell from the grid columns it spans, keeping unequal columns and cells after a merged cell consistent with the table grid. +- **2026-10-23 (M10, table continuation text):** Office exporters retain text from table slots covered by another cell's span as paragraph text. + Synthetic regressions check every input fragment survives exactly once in Word, HTML, RTF and PowerPoint, while ordinary superscripts stay in their table cells. +- **2026-10-23 (M10, image export):** Soft-mask Decode intervals preserve partial opacity in Export all images and Save Image As. + Synthetic regressions cover direct and indirect ranges, inverted/subrange alpha, packed masks, JPEG masks and unchanged image component Decode. + - **2026-10-22 (M4, #3916 partial):** Document edits and Undo/Redo cancel an Organize page drag. Pointer-release regressions check exact page order and save/reopen. - **2026-10-22 (M14, macOS association recovery candidate):** Successful desktop open/save/sign can queue recovery of an existing per-file binding to the running PDFCraft bundle through NSWorkspace. Metadata-only watcher changes retain undo and avoid false overwrite prompts; real same-length/same-mtime edits remain protected. Rebased on 0.12.0 and reuses #3042's shared atomic metadata preservation; the bounded recovery parser does not narrow save limits. Prior native Preview TEST-signature and Finder workflows passed on macOS 27.0.1; other versions remain unqualified. Estimates unchanged. - **2026-10-13 (M7, free image rotation):** A selected page image has a rotate knob (Shift: 15 degree steps) and `image_edit` a `rotate_by` action; the angle is written as one undoable `cm`. Snapping to page edges while moving is in #1740; resize snapping is not done. Estimates unchanged. @@ -254,6 +268,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-18 (M7, #3212):** Headers, Bates numbers, watermarks and added text stay visible after unfinished marked content, with original content preserved. - **2026-10-18 (M12, #2589):** Active XFA skips PDF-container fallback scripts while retaining requested messages and saved scripts; ordinary/static/unlaid-out documents and explicit console/field calls keep their existing behavior. Estimates unchanged. +- **2026-10-18 (M7, #2592):** Styled fallback fonts batch up to 240 glyphs per actual face and retain source styles. Budget refusal and distinct same-BaseFont descriptor weights have mutation-checked regressions; coverage and complex mixed runs remain limited. Estimates unchanged. - **2026-10-18 (M7, #3187):** Added-image automation normalizes extreme signed rotation angles before combining them with the current orientation; integer bounds, history, pixels, save/reopen and MCP have regressions. Estimates unchanged. - **2026-10-18 (M12, #2569):** Growable multiline XFA text reflows after edits, with undo/redo, save/reopen and print coverage; oversized fields still become read-only page text. Structured-data synchronization warnings retain accepted whitespace without relayout from stale data. Estimates unchanged. - **2026-10-18 (M12, #2572):** Every repeated page-master button resolves its own field to the shared XFA click script, including print buttons. Estimates unchanged. @@ -647,6 +662,10 @@ Newest first: the date, what moved, and the new overall estimate. | Date | Change | Summary | |---|---|---| | 2026-10-23 | minor | M8, #3205 follow-up: phone-pattern redaction includes the +39 country code on labelled Italian geographic numbers with a slash; dates, protocol identifiers and whole-run guards remain. Estimates unchanged. | +| 2026-10-23 | minor | M11: image downsampling measures physical resolution after page-local UserUnit scaling, including shared nested forms; synthetic measurement and optimization regressions. Estimates unchanged. | +| 2026-10-23 | minor | M10: Word table cells use the grid columns they span, including cells after a merged cell; synthetic export regression and CLI reproduction. Estimates unchanged. | +| 2026-10-23 | minor | M10: covered table slots retain their text in paragraph flow across Office exporters; emitted superscripts remain in their cells. Synthetic source and CLI regressions; estimates unchanged. | +| 2026-10-23 | minor | M10: Export all images and Save Image As apply soft-mask Decode intervals to PNG alpha before resizing, including inverted/subrange, indirect, packed and JPEG masks. Estimates unchanged. | | 2026-10-22 | minor | M4, #3916 partial: document edits and Undo/Redo cancel Organize drags. Pointer-release and save/reopen regressions check page order. Estimates unchanged. | | 2026-10-20 | minor | M4: initial-view window and interface options resolve indirect boolean ViewerPreferences values; synthetic true/false and save/reopen regressions retain defaults for malformed entries. Estimates unchanged. | | 2026-10-20 | minor | M6, #3502: form detection preserves comments across content streams; estimates unchanged. | @@ -692,8 +711,15 @@ Newest first: the date, what moved, and the new overall estimate. - **2026-10-13 (M7):** Edit a PDF ▸ Add content: Header and footer, Watermark and Background open an Add / Update / Remove menu (Update and Remove enabled only when the document has that mark), as Acrobat's Watermark control does. The catalogue gains `ToolItem::actions`, so any tool can list more commands (#1192). - **2026-10-18 (M8, #1350):** Published the history-preserving integration; simplified the inherited-form resource guard for Rust 1.96 Clippy. Formatting, 64 redaction tests, scoped Clippy, layering, assets and the scoped wasm check pass locally. CI remains pending; estimates unchanged. + +- 2026-10-18: M11 / #3227: mark target-version compatibility partial; document the unchecked newer-content limitation. + +- **2026-10-18 (PR #3227 integration):** M11: integrate upstream b0e5c6eb; retain partial target-version compatibility claims and effective-version reporting, correcting the borrowed version parser lifetime. Scope and estimates unchanged. + - **2026-10-12 (i18n):** Albanian (`sq`, Shqip) added: 2,343-entry catalog, month and weekday names, Preferences entry; not yet reviewed by a native speaker. - **2026-10-21 (M4, #3865):** A new bookmark records where the reader is: the top-left of the selected text, or the top of the window, as `/XYZ left top null` (zoom kept), instead of the page top (`Edit::AddBookmark` takes `left` and `top`; `bookmark_add` takes `top` and `left`; `PageInfo::xyz_for_view`). - **2026-10-18 (renderer, hayro patches 19/35):** A mesh shading (types 4–7) is sampled over its own triangles instead of the whole path: a BricsCAD plan with 310 one-triangle `sh` fills took 2 s per 1024 px tile at every zoom (and hit the 20 s watchdog on a busy machine); now about 70 ms, so zooming into dense topology drawings stays fluid. Estimates unchanged. + +- **2026-10-22 (PR #3227 integration):** M11: integrate upstream `80b7d215`; retain partial target-version compatibility and upstream content-boundary and image-size regression coverage. Scope and estimates unchanged. diff --git a/apps/pdfcraft-cli/src/main.rs b/apps/pdfcraft-cli/src/main.rs index d0f23242d..3a46280c6 100644 --- a/apps/pdfcraft-cli/src/main.rs +++ b/apps/pdfcraft-cli/src/main.rs @@ -688,6 +688,10 @@ fn render(args: &[String]) -> Result<(), CliError> { pdfcraft_render::RenderWarning::ContentTruncated => { let _ = writeln!(std::io::stderr().lock(), "warning: page {page}: content past the page's safety budget was skipped"); } + pdfcraft_render::RenderWarning::ColourFallback => { + let _ = + writeln!(std::io::stderr().lock(), "warning: page {page}: a colour that could not be computed was painted in a fallback colour"); + } } } Ok(()) diff --git a/crates/automation/src/lib.rs b/crates/automation/src/lib.rs index 2a4541512..208370f15 100644 --- a/crates/automation/src/lib.rs +++ b/crates/automation/src/lib.rs @@ -1216,7 +1216,7 @@ impl Automation { } fn doc_optimize(&mut self, a: &Args) -> Result { - use pdfcraft_engine::optimize::{Compression, ImageSettings, Preset}; + use pdfcraft_engine::optimize::{Compatibility, Compression, ImageSettings, Preset}; let id = self.doc(a)?.id; let before = self.doc(a)?.bytes.len(); let path = self.resolve(a.str("path")?, true)?; @@ -1224,6 +1224,10 @@ impl Automation { None => Preset::Standard.settings(), Some(p) => Preset::from_name(p).ok_or_else(|| ToolError::InvalidArgs(format!("unknown preset {p:?} (standard, mobile)")))?.settings(), }; + if let Some(c) = a.opt_str("compatibility")? { + settings.compatibility = Compatibility::from_name(c) + .ok_or_else(|| ToolError::InvalidArgs(format!("unknown compatibility {c:?} (retain, 1.4, 1.5, 1.6, 1.7, 2.0)")))?; + } let images = |key: &str, base: ImageSettings| -> Result { let Some(v) = a.get(key) else { return Ok(base) }; let mut s = base; @@ -1257,6 +1261,8 @@ impl Automation { ("discard_tags", &mut settings.discard_tags), ("discard_print_settings", &mut settings.discard_print_settings), ("flate_unencoded", &mut settings.flate_unencoded), + ("object_streams", &mut settings.object_streams), + ("lzw_to_flate", &mut settings.lzw_to_flate), ("remove_invalid_links", &mut settings.remove_invalid_links), ("remove_unreferenced_dests", &mut settings.remove_unreferenced_dests), ] { @@ -1279,9 +1285,24 @@ impl Automation { }; let (bytes, r) = self.session.optimized_bytes(id, &settings, &discard).map_err(failed)?; write_atomic(&path, &bytes)?; + fn valid_version(v: &[u8]) -> Option<&str> { + if matches!(v, [b'1'..=b'9', b'.', b'0'..=b'9']) { std::str::from_utf8(v).ok() } else { None } + } + let catalog_version = (settings.compatibility == Compatibility::Retain) + .then(|| { + let pw = self.doc(a).ok().and_then(|d| d.password.as_deref()); + let doc = pdfcraft_cos::Document::open_with_password(bytes.clone(), pw).ok()?; + let root = doc.root()?; + doc.get(root).as_dict()?.name(b"Version").and_then(valid_version).map(str::to_owned) + }) + .flatten(); + let header_version = bytes.strip_prefix(b"%PDF-").and_then(|b| b.get(..3)).and_then(valid_version); + let pdf_version = catalog_version.as_deref().or(header_version); let o = &r.optimize; Ok(json!({ "path": path.to_string_lossy(), + "pdf_version": pdf_version, + "object_streams": settings.effective_object_streams(), "bytes_before": before, "bytes_after": bytes.len(), "merged_images": o.merged_images, @@ -1295,6 +1316,7 @@ impl Automation { "alternate_images": o.alternate_images, "tags_removed": o.tags_removed, "streams_compressed": o.streams_compressed, + "lzw_streams": o.lzw_streams, "invalid_links": o.invalid_links, "invalid_bookmarks": o.invalid_bookmarks, "unreferenced_dests": o.unreferenced_dests, diff --git a/crates/automation/src/tools.rs b/crates/automation/src/tools.rs index 2d2cf41f3..ebc5796dc 100644 --- a/crates/automation/src/tools.rs +++ b/crates/automation/src/tools.rs @@ -1391,13 +1391,14 @@ pub fn tools() -> Vec { .with(schema(json!({ "doc": doc() }), &["doc"])), t("doc_open_revision", "Open a revision", "Open saved revision `revision` (1 = the oldest) of a document as a new, unsaved document, to see the file as it was then.") .with(schema(json!({ "doc": doc(), "revision": { "type": "integer", "minimum": 1 } }), &["doc", "revision"])), - t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.") + t("doc_optimize", "Optimize PDF", "Write an optimized copy to `path` (Acrobat's PDF Optimizer). preset: standard (default; Reduce File Size's choices) or mobile (images to 96 ppi above 144, JPEG high quality); the options below change it. compatibility: retain (default; keeps the document's version, or 1.5 when object streams are written), 1.4, 1.5, 1.6, 1.7 or 2.0. merge_fragments (default true) first puts a page picture drawn as many stacked image strips (as some PDF printers write it) back together into one image, so it can be resampled (merged_images, fragments_merged). color / gray: { downsample, ppi, above_ppi, compression: jpeg|zip|retain, quality 1–100 } (defaults: downsample to 150 ppi above 225, JPEG 60). Images are measured where pages draw them and replaced only when smaller. discard_*: thumbnails (default true), alternate_images (true), tags, print_settings; flate_unencoded (true); object_streams (default true; always off for 1.4); lzw_to_flate (true) decodes LZW streams and stores them with Flate instead, leaving any it cannot decode unchanged (lzw_streams counts the streams converted); remove_invalid_links and remove_unreferenced_dests (true); images and forms the pages list but never draw are always dropped (unused_xobjects). discard: Remove Hidden Information categories (metadata, attachments, comments, form-fields, hidden-text, hidden-layers, bookmarks, links-actions-scripts, private-data). Signed documents are refused. The open document is unchanged.") .cmd("optimize.advanced") .with(schema( json!({ "doc": doc(), "path": { "type": "string" }, "preset": { "type": "string", "enum": ["standard", "mobile"] }, + "compatibility": { "type": "string", "enum": ["retain", "1.4", "1.5", "1.6", "1.7", "2.0"] }, "merge_fragments": { "type": "boolean" }, "color": { "type": "object" }, "gray": { "type": "object" }, @@ -1406,6 +1407,8 @@ pub fn tools() -> Vec { "discard_tags": { "type": "boolean" }, "discard_print_settings": { "type": "boolean" }, "flate_unencoded": { "type": "boolean" }, + "object_streams": { "type": "boolean" }, + "lzw_to_flate": { "type": "boolean" }, "remove_invalid_links": { "type": "boolean" }, "remove_unreferenced_dests": { "type": "boolean" }, "discard": { "type": "array", "items": { "type": "string" } }, diff --git a/crates/automation/tests/it/automation.rs b/crates/automation/tests/it/automation.rs index 443328af7..7e073b757 100644 --- a/crates/automation/tests/it/automation.rs +++ b/crates/automation/tests/it/automation.rs @@ -3003,6 +3003,43 @@ fn editing_existing_text_through_tools() { assert_eq!(p["lines"].as_array().map(Vec::len), Some(2), "{p}"); } +#[test] +fn styled_cyrillic_fallback_is_batched_through_tools_and_reopen() { + if pdfcraft_fonts::document_japanese_font().is_none() { + eprintln!("skipping: built without craft-fonts"); + return; + } + let dir = workdir("styled-fallback"); + std::fs::write( + dir.join("mixed.pdf"), + helvetica_page( + "BT /F1 12 Tf 14.4 TL 72 700 Td (The quick brown fox jumps over the lazy dog) Tj T* (and keeps running through the long field.) Tj ET", + ), + ) + .unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({"path": "mixed.pdf"}))["doc"].as_u64().unwrap(); + let replacement = "The quick brown fox jumps over the lazy dog шалом\nand keeps running through the long field."; + ok(&mut a, "text_edit", json!({"doc":doc,"page":1,"paragraph":1,"text":replacement,"width":520})); + ok(&mut a, "doc_save", json!({"doc":doc,"path":"saved.pdf"})); + let saved = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join("saved.pdf")).unwrap())).unwrap(); + let page = pdfcraft_model::pages(&saved).swap_remove(0); + let resources = saved.dict(page.dict.get(b"Resources").unwrap()).unwrap(); + let fonts = saved.dict(resources.get(b"Font").unwrap()).unwrap(); + let fallback_count = fonts.iter().filter(|(name, _)| name.starts_with(b"PCJp")).count(); + assert_eq!(fallback_count, 1, "one shared fallback font for the new word"); + ok(&mut a, "edit_undo", json!({"doc":doc})); + assert!(!page_text(&mut a, doc)[0].contains("шалом")); + ok(&mut a, "edit_redo", json!({"doc":doc})); + assert!(page_text(&mut a, doc)[0].contains("шалом")); + let reopened = ok(&mut a, "doc_open", json!({"path":"saved.pdf"}))["doc"].as_u64().unwrap(); + let lines = ok(&mut a, "text_lines", json!({"doc":reopened,"page":1})); + let runs = lines["lines"].as_array().unwrap(); + assert!(runs.iter().any(|l| l["text"].as_str().is_some_and(|t| t.starts_with("The quick")) && l["font"] == "Helvetica"), "{lines}"); + assert_eq!(runs.iter().filter(|l| l["text"] == "шалом").count(), 1, "{lines}"); + assert!(page_text(&mut a, reopened)[0].contains("шалом")); +} + /// #2593: a paragraph whose own font cannot show a character is written from PdfCraft's fallback /// faces. The Cyrillic the craft-fonts faces lack (Uzbek қ ғ ҳ, Kazakh ә ө ү ұ һ) comes from the /// bundled Inter face, so the edit is not refused even without `CRAFT_FONTS_DIR`, and the words @@ -5876,6 +5913,28 @@ fn optimizing_through_tools() { assert!(reduced["bytes_after"].as_u64().unwrap() < reduced["bytes_before"].as_u64().unwrap()); assert_eq!(reduced["written"], true); assert_eq!(std::fs::metadata(dir.join("reduced.pdf")).unwrap().len(), reduced["bytes_after"].as_u64().unwrap()); + + // Target PDF 1.4 (disables object streams) and PDF 2.0 (with object_streams toggled off). + let v14 = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "v14.pdf", "compatibility": "1.4" })); + assert_eq!((v14["pdf_version"].as_str(), v14["object_streams"].as_bool()), (Some("1.4"), Some(false)), "{v14}"); + let v14_bytes = std::fs::read(dir.join("v14.pdf")).unwrap(); + assert!(v14_bytes.starts_with(b"%PDF-1.4\n") && !v14_bytes.windows(7).any(|w| w == b"/ObjStm")); + + let v20 = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "v20.pdf", "compatibility": "2.0", "object_streams": false })); + assert_eq!((v20["pdf_version"].as_str(), v20["object_streams"].as_bool()), (Some("2.0"), Some(false)), "{v20}"); + assert!(std::fs::read(dir.join("v20.pdf")).unwrap().starts_with(b"%PDF-2.0\n")); + + // In retain mode, a catalog /Version override is reported as the effective PDF version. + let mut cos14 = pdfcraft_cos::Document::open(std::sync::Arc::new(v14_bytes)).unwrap(); + let root = cos14.root().unwrap(); + cos14.update_dict(root, |c| c.set(b"Version".to_vec(), pdfcraft_cos::Object::name("1.6"))).unwrap(); + let cat16 = pdfcraft_cos::write_full(&cos14, &pdfcraft_cos::SaveOptions { object_streams: false, ..Default::default() }).unwrap(); + std::fs::write(dir.join("cat16.pdf"), &cat16).unwrap(); + let cat_doc = ok(&mut a, "doc_open", json!({ "path": "cat16.pdf" }))["doc"].as_u64().unwrap(); + let retained = ok(&mut a, "doc_optimize", json!({ "doc": cat_doc, "path": "retained.pdf", "compatibility": "retain", "object_streams": false })); + assert_eq!(retained["pdf_version"].as_str(), Some("1.6"), "{retained}"); + + assert!(matches!(a.call("doc_optimize", &json!({ "doc": doc, "path": "x.pdf", "compatibility": "1.3" })), Err(ToolError::InvalidArgs(_)))); assert!(matches!( a.call("doc_optimize", &json!({ "doc": doc, "path": "x.pdf", "color": { "compression": "gif" } })), Err(ToolError::InvalidArgs(_)) @@ -5883,6 +5942,56 @@ fn optimizing_through_tools() { assert!(matches!(a.call("doc_optimize", &json!({ "doc": doc, "path": "x.pdf", "discard": ["everything"] })), Err(ToolError::InvalidArgs(_)))); } +/// `data` as LZW with one 9-bit literal code per byte (clear-table first, EOD last): valid for +/// fewer than 250 bytes, before the code width grows. +fn lzw_literals(data: &[u8]) -> Vec { + let codes: Vec = std::iter::once(256).chain(data.iter().map(|&b| u16::from(b))).chain([257]).collect(); + let bits: Vec = codes.iter().flat_map(|c| (0..9).rev().map(move |i| c >> i & 1 == 1)).collect(); + bits.chunks(8).map(|byte| byte.iter().enumerate().fold(0u8, |acc, (i, &b)| acc | (u8::from(b) << (7 - i)))).collect() +} + +#[test] +fn lzw_streams_are_stored_with_flate_through_tools() { + use pdfcraft_cos::{Dict, Document, Object, SaveOptions, Stream, write_full}; + let dir = workdir("optimize-lzw"); + let ops = b"0.2 0.4 0.8 rg 72 72 200 100 re f 0 0 1 RG 72 300 m 400 500 l S".to_vec(); + let mut doc = Document::new_empty(); + let pages = doc.get(doc.root().unwrap()).as_dict().unwrap().reference(b"Pages").unwrap(); + let lzw = Stream::from_raw(Dict::from_iter([(b"Filter".to_vec(), Object::name("LZWDecode"))]), lzw_literals(&ops)); + assert_eq!(lzw.decoded_strict().unwrap(), ops); + let content = doc.add(Object::Stream(lzw)); + let page = doc.add(Object::Dict(Dict::from_iter([ + (b"Type".to_vec(), Object::name("Page")), + (b"Parent".to_vec(), Object::Ref(pages)), + (b"MediaBox".to_vec(), Object::Array(vec![0.into(), 0.into(), 612.into(), 792.into()])), + (b"Contents".to_vec(), Object::Ref(content)), + ]))); + doc.update_dict(pages, |d| { + d.set(b"Kids".to_vec(), Object::Array(vec![Object::Ref(page)])); + d.set(b"Count".to_vec(), Object::Int(1)); + }) + .unwrap(); + std::fs::write(dir.join("lzw.pdf"), write_full(&doc, &SaveOptions::default()).unwrap()).unwrap(); + let mut a = auto(&dir); + let doc = ok(&mut a, "doc_open", json!({ "path": "lzw.pdf" }))["doc"].as_u64().unwrap(); + let filters = |name: &str| { + let saved = pdfcraft_cos::Document::open(std::sync::Arc::new(std::fs::read(dir.join(name)).unwrap())).unwrap(); + let catalog = saved.get(saved.root().unwrap()).as_dict().cloned().unwrap(); + let pages = saved.get(catalog.reference(b"Pages").unwrap()).as_dict().cloned().unwrap(); + let page = saved.get(pages.get(b"Kids").and_then(|k| k.as_array()?.first()?.as_ref()).unwrap()).as_dict().cloned().unwrap(); + match &*saved.get(page.reference(b"Contents").unwrap()) { + Object::Stream(s) => (s.dict.get(b"Filter").cloned(), s.decoded_strict().unwrap()), + _ => panic!("the page content is not a stream"), + } + }; + let r = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "flate.pdf" })); + assert_eq!(r["lzw_streams"].as_u64(), Some(1), "{r}"); + assert_eq!(filters("flate.pdf"), (Some(Object::name("FlateDecode")), ops.clone())); + let off = ok(&mut a, "doc_optimize", json!({ "doc": doc, "path": "kept.pdf", "lzw_to_flate": false })); + assert_eq!(off["lzw_streams"].as_u64(), Some(0), "{off}"); + assert_eq!(filters("kept.pdf"), (Some(Object::name("LZWDecode")), ops)); +} + /// A one-page PDF whose page picture is drawn as `rows` grey images one pixel row high, 600 px /// wide on a 72 pt square (600 ppi), as some PDF printers write it. fn printed_in_strips(rows: usize) -> Vec { diff --git a/crates/cos/src/document.rs b/crates/cos/src/document.rs index 892c8e04d..c7f2cb015 100644 --- a/crates/cos/src/document.rs +++ b/crates/cos/src/document.rs @@ -897,6 +897,11 @@ impl Document { &self.version } + /// Set the PDF header version (`"1.4"`, `"1.7"`, `"2.0"`, …) written by a full save. + pub fn set_version(&mut self, version: impl Into) { + self.version = version.into(); + } + pub fn trailer(&self) -> &Dict { &self.trailer } diff --git a/crates/create/README.md b/crates/create/README.md index 499081059..d1561a633 100644 --- a/crates/create/README.md +++ b/crates/create/README.md @@ -27,5 +27,7 @@ Every function returns a `pdfcraft_cos::Document`; the caller writes it. Image export and Save Image As apply each lossless image's Decode pair before converting its color components. Indexed Decode values select palette entries after rounding and clipping. Color-key masks compare the original samples. Missing, null or malformed Decode arrays keep the default mapping. -The existing 16-bit path still uses each sample's high byte. JPEG and mask Decode handling remain limited. +Soft-mask Decode intervals map samples to opacity before the mask is resized, including reversed ranges and indirect entries. +The existing 16-bit path still uses each sample's high byte. +JPEG image Decode handling remains limited. ICC transforms and calibrated color conversion remain outside this export path. diff --git a/crates/create/src/extract.rs b/crates/create/src/extract.rs index 115876d9c..7828952ed 100644 --- a/crates/create/src/extract.rs +++ b/crates/create/src/extract.rs @@ -465,7 +465,7 @@ fn stencil_mask(doc: &Document, s: &Stream, w: usize, h: usize) -> Option Option> { let (mw, mh) = size(m.dict.int(b"Width")?, m.dict.int(b"Height")?)?; @@ -491,14 +491,11 @@ fn gray_plane(doc: &Document, m: &Stream, w: usize, h: usize) -> Option> unpack(&m.decoded().ok()?, mw, mh, 1, bpc)?.into_iter().map(|v| (v as u32 * 255 / max) as u8).collect() } }; - let invert = m - .dict - .get(b"Decode") - .map(|d| doc.resolve(d)) - .and_then(|d| d.as_array().map(|a| a.first().and_then(Object::as_f64) > a.get(1).and_then(Object::as_f64))) - .unwrap_or(false); - if invert { - px.iter_mut().for_each(|v| *v = 255 - *v); + if let Some([lo, hi]) = decode(doc, m, 1).and_then(|pairs| pairs.first().copied()) { + for value in &mut px { + let t = f64::from(*value) / 255.0; + *value = (((1.0 - t) * lo + t * hi).clamp(0.0, 1.0) * 255.0).round() as u8; + } } if (mw, mh) == (w, h) { return Some(px); diff --git a/crates/create/src/tests.rs b/crates/create/src/tests.rs index b6c7969c2..3c9aa42a0 100644 --- a/crates/create/src/tests.rs +++ b/crates/create/src/tests.rs @@ -682,6 +682,74 @@ fn jpeg_images_with_a_soft_mask_export_as_png_with_alpha() { assert_eq!((out.images[0].extension, &out.images[0].data), ("jpg", &real_jpeg())); } +#[test] +/// Both image export paths apply mask intervals independently of the image's component Decode. +fn soft_mask_decode_ranges_become_exported_alpha() { + for indirect in [false, true] { + for (range, expected) in [ + ([0.25, 0.75], [64, 128, 191]), + ([0.75, 0.25], [191, 127, 64]), + ([1.0, 0.0], [255, 127, 0]), + ([-1.0, 2.0], [0, 129, 255]), + ([0.5, 0.5], [128, 128, 128]), + ] { + let mut doc = from_images(&[("flat.png".into(), png_bytes(false))]).unwrap(); + with_image(&mut doc, |doc| { + let endpoints = range.into_iter().map(|v| { + let value = Object::Real(v); + if indirect { Object::Ref(doc.add(value)) } else { value } + }); + let decode = Object::Array(endpoints.collect()); + let decode = if indirect { Object::Ref(doc.add(decode)) } else { decode }; + let mut mask = image_dict(3, 1, "DeviceGray", 8); + mask.set(b"Decode".to_vec(), decode); + let mask = doc.add(Object::Stream(Stream::from_raw(mask, vec![0, 128, 255]))); + let mut image = image_dict(3, 1, "DeviceRGB", 8); + image.set(b"SMask".to_vec(), Object::Ref(mask)); + image.set(b"Decode".to_vec(), decode_array(&[1.0, 0.0, 0.0, 1.0, 0.0, 1.0])); + Stream::from_raw(image, [0, 64, 255].repeat(3)) + }); + let doc = reopen(&doc); + let out = extract_images(&doc, &[0], 0); + assert!(out.skipped.is_empty(), "{:?}", out.skipped); + assert_eq!(out.images.len(), 1); + let pixels = exported_png_pixels(&out.images[0].data); + assert_eq!(alphas(&pixels), expected, "Decode {range:?}, indirect={indirect}"); + assert!(pixels.as_chunks::<4>().0.iter().all(|p| p[..3] == [255, 64, 255]), "image component Decode stays independent"); + let (ext, bytes) = image_file(&doc, out.images[0].object).unwrap(); + assert_eq!(ext, "png"); + assert_eq!(exported_png_pixels(&bytes), pixels, "Save Image As and Export all images agree"); + } + } +} + +#[test] +/// Packed and JPEG samples map to opacity before resizing to their image's dimensions. +fn soft_mask_decode_maps_packed_and_jpeg_masks_before_resizing() { + for jpeg in [false, true] { + let mut doc = from_images(&[("flat.png".into(), png_bytes(false))]).unwrap(); + with_image(&mut doc, |doc| { + let mask = if jpeg { + let mut mask = image_dict(4, 2, "DeviceGray", 8); + mask.set(b"Filter".to_vec(), Object::name("DCTDecode")); + mask.set(b"Decode".to_vec(), decode_array(&[0.25, 0.25])); + Stream::from_raw(mask, real_jpeg()) + } else { + let mut mask = image_dict(2, 1, "DeviceGray", 1); + mask.set(b"Decode".to_vec(), decode_array(&[-1.0, 2.0])); + Stream::from_raw(mask, vec![0b0100_0000]) + }; + let mask = doc.add(Object::Stream(mask)); + let mut image = image_dict(4, 2, "DeviceRGB", 8); + image.set(b"SMask".to_vec(), Object::Ref(mask)); + Stream::from_raw(image, vec![10; 24]) + }); + // Clipping before resizing differs from resizing first, then clipping to [0, 255]. + let expected = if jpeg { vec![64; 8] } else { vec![0, 64, 191, 255, 0, 64, 191, 255] }; + assert_eq!(alphas(&exported_rgba(&reopen(&doc)).1), expected, "jpeg={jpeg}"); + } +} + #[test] fn soft_masks_of_another_size_and_stencil_and_colour_key_masks_become_alpha() { // A 2×1 soft mask on a 4×2 image is scaled to the image's size. diff --git a/crates/edit/src/tests.rs b/crates/edit/src/tests.rs index 76b63d82e..3ad46e864 100644 --- a/crates/edit/src/tests.rs +++ b/crates/edit/src/tests.rs @@ -1731,6 +1731,214 @@ fn japanese_replacement_over_240_unique_glyphs_is_split_across_fallback_fonts() assert_eq!(glyphs, 300); } +#[test] +fn styled_paragraph_batches_cyrillic_without_changing_source_fonts() { + if without_craft_fonts("styled_paragraph_batches_cyrillic_without_changing_source_fonts") { + return; + } + let mut doc = text_page( + "BT /F1 12 Tf 14.4 TL 1 0 0 1 72 700 Tm (The quick brown fox jumps over the lazy dog) Tj T* (and keeps running through the long field.) Tj ET", + ); + let replacement = "The quick brown fox jumps over the lazy dog шалом\nand keeps running through the long field."; + text::rewrite_block(&mut doc, 0, 0, Some(replacement), &text::BlockStyle { width: Some(520.0), ..Default::default() }).unwrap(); + let doc = reopen(&doc); + assert_eq!(fallback_fonts(&doc).len(), 1, "the five letters must share one font"); + let lines = text::text_lines(&doc, 0).unwrap(); + assert!(lines.iter().any(|l| l.base_font == "Helvetica" && l.text.starts_with("The quick brown fox")), "{lines:?}"); + let fallback: Vec<_> = lines.iter().filter(|l| l.font.starts_with("PCJp")).collect(); + assert_eq!(fallback.len(), 1, "one word must be one text run: {fallback:?}"); + assert_eq!(fallback[0].text, "шалом"); + assert_eq!(lines.iter().map(|l| l.text.as_str()).collect::().replace(' ', ""), replacement.replace([' ', '\n'], "")); +} + +#[test] +fn styled_paragraph_over_240_fallback_glyphs_splits_and_refuses_atomically() { + let replacement = format!("{}\nLatin", covered_japanese_chars(300)); + let mut doc = text_page("BT /F1 12 Tf 14.4 TL 72 700 Td (Original) Tj T* (Latin) Tj ET"); + let original = write_incremental(&doc, &SaveOptions::default()).unwrap(); + if without_craft_fonts("styled_paragraph_over_240_fallback_glyphs_splits_and_refuses_atomically") { + assert!(text::replace_block(&mut doc, 0, 0, &replacement).is_err()); + assert_eq!(write_incremental(&doc, &SaveOptions::default()).unwrap(), original); + return; + } + text::rewrite_block(&mut doc, 0, 0, Some(&replacement), &text::BlockStyle { width: Some(14000.0), ..Default::default() }).unwrap(); + let reopened = reopen(&doc); + let fonts = fallback_fonts(&reopened); + assert_eq!(fonts.len(), 2, "300 glyphs need two fonts, not an error at glyph 241"); + let counts: Vec<_> = fonts.iter().map(|(_, f)| reopened.dict(f.get(b"CharProcs").unwrap()).unwrap().len()).collect(); + assert_eq!(counts.iter().sum::(), 300); + assert!(counts.iter().all(|n| *n <= 240)); + assert_eq!(text::text_lines(&reopened, 0).unwrap().iter().map(|l| l.text.as_str()).collect::(), replacement.replace('\n', "")); + let before = write_incremental(&doc, &SaveOptions::default()).unwrap(); + let snapshot = doc.clone(); + let err = text::replace_block(&mut doc, 0, 0, &format!("{replacement}\u{10ffff}")).unwrap_err(); + assert!(err.to_string().contains("U+10FFFF"), "{err}"); + assert_eq!(write_incremental(&doc, &SaveOptions::default()).unwrap(), before); + assert_eq!(write_incremental(&snapshot, &SaveOptions::default()).unwrap(), before); +} + +#[test] +fn styled_fallback_batches_keep_face_choices_and_range_styles() { + if without_craft_fonts("styled_fallback_batches_keep_face_choices_and_range_styles") { + return; + } + if !pdfcraft_fonts::CRAFT_FONTS.iter().any(|f| f.family == "BIZ UDPGothic" && f.style == "Bold") { + eprintln!("skipping: no bold Gothic face"); + return; + } + let mut doc = text_page("BT /F1 12 Tf 72 700 Td (Original) Tj ET"); + let range = |range, family, bold| text::RangeStyle { + range, + family: Some((family, bold, false)), + resource: None, + size: None, + color: None, + underline: None, + char_spacing: None, + scale: None, + }; + let style = text::BlockStyle { + width: Some(520.0), + ranges: vec![ + range(3..6, added::Family::Times, false), + range(6..9, added::Family::Helvetica, true), + range(9..11, added::Family::Times, false), + ], + ..Default::default() + }; + let original = write_incremental(&doc, &SaveOptions::default()).unwrap(); + let err = text::rewrite_block(&mut doc, 0, 0, Some("見見見ш見ш\nLatin\u{10ffff}"), &style).unwrap_err(); + assert!(err.to_string().contains("U+10FFFF"), "{err}"); + assert_eq!(write_incremental(&doc, &SaveOptions::default()).unwrap(), original, "all faces must validate before writing fallback objects"); + text::rewrite_block(&mut doc, 0, 0, Some("見見見ш見ш\nLatin"), &style).unwrap(); + let doc = reopen(&doc); + let fonts = fallback_fonts(&doc); + let mut summary: Vec<_> = fonts + .iter() + .map(|(_, f)| { + let face = pdfcraft_fonts::pdf::Metrics::from_dict(&doc, f).base_font; + let count = doc.dict(f.get(b"CharProcs").unwrap()).unwrap().len(); + (face, count) + }) + .collect(); + summary.sort(); + assert_eq!( + summary, + [ + ("BIZUDMincho-Regular".into(), 1), + ("BIZUDPGothic-Bold".into(), 1), + ("BIZUDPGothic-Regular".into(), 2), + ("ShipporiMincho-Regular".into(), 1) + ] + ); + assert_eq!(text::text_lines(&doc, 0).unwrap().iter().map(|l| l.text.as_str()).collect::(), "見見見ш見шLatin"); +} + +#[test] +fn styled_fallback_budget_is_enforced_before_objects_are_written() { + let glyphs = covered_japanese_chars(65); + let mut doc = text_page("BT /F1 12 Tf 72 700 Td (Original) Tj ET"); + let original = write_incremental(&doc, &SaveOptions::default()).unwrap(); + let snapshot = doc.clone(); + let style = text::BlockStyle { width: Some(14000.0), ..Default::default() }; + let over_budget = format!("{glyphs}\nLatin"); + let err = text::rewrite_block_with_limit(&mut doc, 0, 0, Some(&over_budget), &style, 1).unwrap_err().to_string(); + assert_eq!(write_incremental(&doc, &SaveOptions::default()).unwrap(), original, "refusal must add no objects"); + assert_eq!(write_incremental(&snapshot, &SaveOptions::default()).unwrap(), original); + if without_craft_fonts("styled_fallback_budget_is_enforced_before_objects_are_written") { + assert!(err.contains("CRAFT_FONTS_DIR"), "{err}"); + return; + } + assert!(err.contains("more than 64 fallback glyphs"), "{err}"); + // One glyph per font gives a small, real budget of 64 distinct glyphs. At the + // boundary a repeated glyph is free: this is not a limit on text length. + let at_budget: String = glyphs.chars().take(64).chain(glyphs.chars().take(1)).collect(); + text::rewrite_block_with_limit(&mut doc, 0, 0, Some(&format!("{at_budget}\nLatin")), &style, 1).unwrap(); + let doc = reopen(&doc); + assert_eq!(fallback_fonts(&doc).len(), 64); + assert_eq!(text::text_lines(&doc, 0).unwrap().iter().map(|l| l.text.as_str()).collect::(), format!("{at_budget}Latin")); +} + +#[test] +fn styled_fallback_same_base_font_and_character_keep_both_weights() { + if without_craft_fonts("styled_fallback_same_base_font_and_character_keep_both_weights") { + return; + } + if !pdfcraft_fonts::CRAFT_FONTS.iter().any(|f| f.family == "BIZ UDPGothic" && f.style == "Bold") { + eprintln!("skipping: no bold Gothic face"); + return; + } + let mut doc = text_page("BT /F1 12 Tf 72 700 Td (alpha ) Tj /F2 12 Tf (beta ) Tj ET"); + let source = doc.get(pdfcraft_cos::ObjRef::new(5, 0)).as_dict().unwrap().clone(); + for (object, weight) in [(5, 400), (6, 700)] { + let mut font = source.clone(); + // The same BaseFont can have different descriptor weights. Changing a + // RangeStyle family would also rename BaseFont and miss this cache key. + font.set(b"BaseFont".to_vec(), Object::name("NeutralFace")); + let mut descriptor = Dict::new(); + descriptor.set(b"FontName".to_vec(), Object::name("NeutralFace")); + descriptor.set(b"FontWeight".to_vec(), Object::Int(weight)); + font.set(b"FontDescriptor".to_vec(), Object::Dict(descriptor)); + doc.set(pdfcraft_cos::ObjRef::new(object, 0), Object::Dict(font)); + } + let lines = text::text_lines(&doc, 0).unwrap(); + assert_eq!(lines.len(), 2); + assert_eq!(lines[0].base_font, lines[1].base_font); + assert_eq!((lines[0].bold, lines[1].bold), (false, true)); + // Unchanged words anchor each inserted copy of the same letter in its own source style. + let block = text::text_blocks(&doc, 0).unwrap().remove(0); + let mut inherited = Vec::new(); + for (range, source) in block.edited_spans("alpha ш beta ш") { + for _ in "alpha ш beta ш".get(range).unwrap().chars().filter(|ch| *ch == 'ш') { + inherited.push(lines[source].bold); + } + } + assert_eq!(inherited, [false, true], "the fixture must exercise both weights of the same BaseFont"); + text::replace_block(&mut doc, 0, 0, "alpha ш beta ш").unwrap(); + let doc = reopen(&doc); + let fonts = fallback_fonts(&doc); + let mut names: Vec<_> = fonts.iter().map(|(_, f)| pdfcraft_fonts::pdf::Metrics::from_dict(&doc, f).base_font).collect(); + names.sort(); + assert_eq!(names, ["BIZUDPGothic-Bold", "BIZUDPGothic-Regular"]); + let outlines: Vec<_> = fonts + .iter() + .map(|(_, f)| { + let procs = doc.dict(f.get(b"CharProcs").unwrap()).unwrap(); + assert_eq!(procs.len(), 1, "each weight needs just the shared letter"); + stream_bytes(&doc, procs.iter().next().unwrap().1).unwrap() + }) + .collect(); + assert_ne!(outlines[0], outlines[1], "weight must change the outline, not only the descriptor"); + let lines = text::text_lines(&doc, 0).unwrap(); + assert_eq!(lines.iter().map(|l| l.text.as_str()).collect::(), "alpha ш beta ш"); +} + +#[test] +fn styled_fallback_shards_keep_text_advances_and_alignment() { + if without_craft_fonts("styled_fallback_shards_keep_text_advances_and_alignment") { + return; + } + let source = "BT /F1 12 Tf 0.5 Tc 5 Tw 80 Tz 14.4 TL 72 700 Td (Original row) Tj T* (Second row) Tj ET"; + let style = + text::BlockStyle { width: Some(520.0), align: Some(added::Align::Center), char_spacing: Some(0.7), scale: Some(80.0), ..Default::default() }; + let mut whole = text_page(source); + let mut split = text_page(source); + let replacement = "Latin шалом ш\nSecond row"; + text::rewrite_block_with_limit(&mut whole, 0, 0, Some(replacement), &style, 240).unwrap(); + text::rewrite_block_with_limit(&mut split, 0, 0, Some(replacement), &style, 2).unwrap(); + let (whole, split) = (reopen(&whole), reopen(&split)); + assert_eq!(fallback_fonts(&whole).len(), 1); + assert_eq!(fallback_fonts(&split).len(), 3); + let (one, many) = (text::text_lines(&whole, 0).unwrap(), text::text_lines(&split, 0).unwrap()); + assert_eq!(one.len(), many.len(), "{one:?} vs {many:?}"); + for (a, b) in one.iter().zip(&many) { + assert_eq!(a.text, b.text); + for (a, b) in a.rect.iter().zip(b.rect) { + assert!((a - b).abs() < 0.002, "sharding must preserve alignment and advances: {a} vs {b}"); + } + } +} + /// A replacement whose last glyph no face draws (in a later font) changes nothing: neither the /// document nor a snapshot taken before the edit, which is what undo restores. #[test] diff --git a/crates/edit/src/text.rs b/crates/edit/src/text.rs index 2047f6a6f..55f2e7ad1 100644 --- a/crates/edit/src/text.rs +++ b/crates/edit/src/text.rs @@ -2252,6 +2252,19 @@ fn type3_font( .map_err(|_| EditError::Invalid("replacement text has too many unique characters".into()))?; glyphs.push(Type3Glyph { code, ch, path, width }); } + Ok(write_type3_glyphs(doc, fonts_res, face, hebrew, glyphs, per_font)) +} + +/// Write already validated outlines; callers can prepare every face before adding any objects. +fn write_type3_glyphs( + doc: &mut Document, + fonts_res: &mut Dict, + face: &CraftFont, + hebrew: bool, + glyphs: Vec, + per_font: usize, +) -> Type3Fallback { + let per_font = per_font.clamp(1, MAX_TYPE3_GLYPHS); let mut codes = HashMap::with_capacity(glyphs.len()); let mut widths = Vec::with_capacity(glyphs.len()); let mut names = Vec::new(); @@ -2267,7 +2280,7 @@ fn type3_font( } names.push(add_type3_font(doc, fonts_res, face, hebrew, group)); } - Ok(Type3Fallback { family: face.family, names, widths, codes }) + Type3Fallback { family: face.family, names, widths, codes } } /// Adds one generated font whose codes 1 to `glyphs.len()` show `glyphs`, and returns its resource @@ -2915,7 +2928,7 @@ fn rewrite_block_from( && style.family.is_none() && style.bold.is_none()) { - return rewrite_styled_block(doc, page, &b, &lines, text, style); + return rewrite_styled_block(doc, page, &b, &lines, text, style, max_glyphs); } let streams = content_streams(doc, &p.dict)?; let pieces = Pieces::join(&streams.iter().map(|(_, d)| d.as_slice()).collect::>()); diff --git a/crates/edit/src/text_regions.rs b/crates/edit/src/text_regions.rs index e2fdff202..d4ebe4c44 100644 --- a/crates/edit/src/text_regions.rs +++ b/crates/edit/src/text_regions.rs @@ -2,6 +2,18 @@ use super::*; +// The draft is capped at 4 MiB; also bound the outline data prepared across all fallback faces. +// A glyph budget worth 64 full batches (15,360 glyphs with the production batch size). +// Lowering the internal batch size for tests lowers the budget by the same ratio. +const MAX_STYLED_FALLBACK_BATCHES: usize = 64; + +struct FallbackGroup { + face: &'static CraftFont, + seen: std::collections::HashSet, + glyphs: Vec, + positions: Vec, +} + fn same_plane(a: &TextLine, b: &TextLine) -> bool { let at = Matrix(a.origin.tm).then(&Matrix(a.origin.ctm)); let bt = Matrix(b.origin.tm).then(&Matrix(b.origin.ctm)); @@ -400,6 +412,7 @@ pub(super) fn rewrite_styled_block( lines: &[TextLine], text: Option<&str>, style: &BlockStyle, + max_glyphs: usize, ) -> Result { let first = block.lines.first().and_then(|i| lines.get(*i)).ok_or_else(|| EditError::Invalid("the paragraph has no source runs".into()))?; let draft = match text { @@ -542,7 +555,11 @@ pub(super) fn rewrite_styled_block( // Each tuple is a character, its source style and an encodable resource name. let mut chars = Vec::new(); let mut substituted = None; - let mut fallback_names: std::collections::BTreeMap<(String, char), String> = std::collections::BTreeMap::new(); + let mut fallback_groups: Vec = Vec::new(); + let mut fallback_names: std::collections::BTreeMap<(String, bool, char), usize> = std::collections::BTreeMap::new(); + let mut fallback_glyphs = 0_usize; + let per_font = max_glyphs.clamp(1, MAX_TYPE3_GLYPHS); + let fallback_budget = per_font.saturating_mul(MAX_STYLED_FALLBACK_BATCHES); let mut full_faces: HashMap = HashMap::new(); let mut programs: HashMap>>> = HashMap::new(); for (range, index) in styled_spans { @@ -614,24 +631,40 @@ pub(super) fn rewrite_styled_block( } else { let family = source_family(&source.base_font); if needs_type3(&s) { - let key = (source.base_font.rsplit('+').next().unwrap_or(&source.base_font).to_owned(), ch); - if let Some(name) = fallback_names.get(&key) { - chars.push((ch, index, name.clone())); - continue; + let key = (source.base_font.clone(), source.bold, ch); + let group_index = if let Some(index) = fallback_names.get(&key) { + *index + } else { + // Keep the old per-character face choice: a serif Japanese character + // and a Cyrillic character may need different faces even in one style. + // Hebrew uses the separate RTL path before reaching this writer. + let (faces, _) = type3_faces(&s, family, source.bold)?; + let face = + fallback_face(&faces, &[ch], |face, ch| japanese_glyph_from(face, ch).is_ok()).ok_or_else(|| no_fallback_font(&s))?; + let index = fallback_groups.iter().position(|group| std::ptr::eq(group.face, face)).unwrap_or_else(|| { + let index = fallback_groups.len(); + fallback_groups.push(FallbackGroup { face, seen: Default::default(), glyphs: Vec::new(), positions: Vec::new() }); + index + }); + fallback_names.insert(key, index); + index + }; + let group = fallback_groups.get_mut(group_index).ok_or_else(|| EditError::Invalid("the fallback face is missing".into()))?; + if group.seen.insert(ch) { + if fallback_glyphs >= fallback_budget { + return Err(EditError::Invalid(format!( + "the replacement needs more than {fallback_budget} fallback glyphs; edit a smaller paragraph or load a complete font" + ))); + } + let (path, width) = type3_glyph(group.face, ch)?.ok_or_else(|| no_glyph_for(group.face, ch))?; + let code = u8::try_from((group.glyphs.len() % per_font).saturating_add(1)) + .map_err(|_| EditError::Invalid("replacement text has too many unique characters".into()))?; + group.glyphs.push(Type3Glyph { code, ch, path, width }); + fallback_glyphs = fallback_glyphs.saturating_add(1); } - if fallback_names.len() >= MAX_TYPE3_GLYPHS { - return Err(EditError::Invalid("the replacement has too many fallback glyphs".into())); - } - let fallback = type3_font(doc, &mut fonts, &s, family, source.bold, MAX_TYPE3_GLYPHS)?; - let name = fallback.names.first().cloned().ok_or_else(no_japanese_font)?; - let dict = fonts - .get(name.as_bytes()) - .and_then(|f| doc.resolve(f).as_dict().cloned()) - .ok_or_else(|| EditError::Invalid("the fallback font could not be loaded".into()))?; - metrics.insert(name.clone(), Metrics::from_dict(doc, &dict)); - substituted = Some(fallback.family.to_string()); - fallback_names.insert(key, name.clone()); - name + group.positions.push(chars.len()); + // Filled from the prepared font/code map before measuring or drawing. + String::new() } else { let base = family.base_font(source.bold, source.italic); let name = format!("PCEd{}", base.replace('-', "")); @@ -649,6 +682,24 @@ pub(super) fn rewrite_styled_block( chars.push((ch, index, name)); } } + // All fallback outlines have now succeeded. Batch by actual face, preserving source + // style indices; the existing writer then coalesces adjacent equal-font/style characters. + for group in fallback_groups { + let fallback = write_type3_glyphs(doc, &mut fonts, group.face, false, group.glyphs, per_font); + for name in &fallback.names { + let font = fonts + .get(name.as_bytes()) + .and_then(|f| doc.dict(f)) + .ok_or_else(|| EditError::Invalid("the fallback font could not be loaded".into()))?; + metrics.insert(name.clone(), Metrics::from_dict(doc, &font)); + } + for position in group.positions { + let item = chars.get_mut(position).ok_or_else(|| EditError::Invalid("the fallback character is missing".into()))?; + let (font, _) = fallback.codes.get(&item.0).ok_or_else(|| EditError::Invalid("the fallback glyph is missing".into()))?; + item.2 = fallback.names.get(*font).cloned().ok_or_else(|| EditError::Invalid("the fallback font is missing".into()))?; + } + substituted = Some(fallback.family.to_owned()); + } let size_of = |source: &TextLine, index: usize| { overrides.get(&index).and_then(|r| r.size).or(style.size).unwrap_or(source.size).max(0.1) / source.origin.k.max(1e-6) }; diff --git a/crates/engine/src/optimizer.rs b/crates/engine/src/optimizer.rs index 444c428ca..a3becf514 100644 --- a/crates/engine/src/optimizer.rs +++ b/crates/engine/src/optimizer.rs @@ -68,7 +68,11 @@ impl Session { cos: editor.cos.clone(), settings: settings.clone(), discard: discard.to_vec(), - opts: SaveOptions { mod_date: self.now().map(pdfcraft_cos::pdf_date), ..SaveOptions::default() }, + opts: SaveOptions { + mod_date: self.now().map(pdfcraft_cos::pdf_date), + object_streams: settings.effective_object_streams(), + ..SaveOptions::default() + }, }) } } diff --git a/crates/export/src/lib.rs b/crates/export/src/lib.rs index c8dc65951..1303254d6 100644 --- a/crates/export/src/lib.rs +++ b/crates/export/src/lib.rs @@ -256,7 +256,7 @@ pub(crate) fn tables(blocks: &[Block]) -> (Vec, Vec) { } let right = used.iter().map(|&i| blocks[i].rect[2]).fold(f64::MIN, f64::max) + COL_TOL; let mut rows_out = Vec::new(); - let mut slots: Vec> = vec![None; ncols]; + let mut slots: Vec)>> = vec![None; ncols]; for row in run.drain(..) { for slot in slots.iter_mut() { *slot = None; @@ -266,20 +266,28 @@ pub(crate) fn tables(blocks: &[Block]) -> (Vec
, Vec) { let Some(slot) = edges.iter().position(|e| (*e - b.rect[0]).abs() <= COL_TOL) else { continue }; let span = edges[slot + 1..].iter().filter(|e| **e > b.rect[0] + COL_TOL && **e < b.rect[2] - COL_TOL).count() + 1; let cell = Cell { text: b.text.trim().to_string(), size: b.size, bold: b.bold, italic: b.italic, color: b.color, span }; - match &mut slots[slot] { - Some(c) => { + match slots.get_mut(slot) { + Some(Some((c, sources))) => { c.text.push(' '); c.text.push_str(&cell.text); + sources.push(bi); } - None => slots[slot] = Some(cell), + Some(empty @ None) => *empty = Some((cell, vec![bi])), + None => continue, } - consumed[bi] = true; } let mut materialized = Vec::with_capacity(ncols); let mut column = 0; while column < ncols { - match slots[column].take() { - Some(mut cell) => { + match slots.get_mut(column).and_then(Option::take) { + Some((mut cell, sources)) => { + // A spanning cell can skip later slots. Only consume source blocks + // when their text is emitted, leaving covered text in paragraph flow. + for bi in sources { + if let Some(taken) = consumed.get_mut(bi) { + *taken = true; + } + } cell.span = cell.span.min(ncols - column).max(1); column += cell.span; materialized.push(cell); @@ -517,17 +525,18 @@ fn docx_table(t: &Table) -> String { s.push_str(""); for row in &t.rows { s.push_str(""); + let mut column = 0usize; for c in row { let span = if c.span > 1 { format!("", c.span) } else { String::new() }; - let w: i64 = widths.iter().take(c.span.min(widths.len())).sum(); + let w: i64 = widths.iter().skip(column).take(c.span).sum(); s.push_str(&format!( "{span}{}", run_xml(&c.text, c.size, c.bold, c.italic, c.color) )); + column = column.saturating_add(c.span); } // Pad the grid so every row covers all columns (Word rejects short rows). - let used: usize = row.iter().map(|c| c.span).sum(); - for _ in used..t.cols.len() { + for _ in column..t.cols.len() { s.push_str(""); } s.push_str(""); @@ -1040,6 +1049,27 @@ mod tests { assert!(xml.contains("")); } + #[test] + fn word_table_cell_widths_follow_the_columns_they_span() { + let cell = |text: &str, span| Cell { text: text.into(), size: 11.0, bold: false, italic: false, color: [0.0; 3], span }; + let table = Table { + rect: [0.0, 0.0, 300.0, 100.0], + cols: vec![0.0, 50.0, 150.0], + rows: vec![ + vec![cell("First", 1), cell("Second", 1), cell("Third", 1)], + vec![cell("First", 1), cell("Last two", 2)], + vec![cell("First two", 2), cell("Third", 1)], + vec![cell("All", 3)], + vec![cell("First", 1)], + ], + }; + let xml = docx_table(&table); + assert!(xml.contains(""), "{xml}"); + let widths: Vec> = + xml.split("").skip(1).map(|row| row.split(""), "the prize table was lost: {h}"); assert!(h.contains("30,000") && h.contains("15,000"), "{h}"); + let (tables, consumed) = tables(&p.blocks); + assert!(consumed.iter().all(|used| *used), "ordinary table continuations stay in their cells"); + let cells: Vec<&str> = tables.iter().flat_map(|t| &t.rows).flatten().map(|c| c.text.as_str()).collect(); + assert!(["st", "nd", "rd", "th"].iter().all(|suffix| cells.contains(suffix))); } #[test] diff --git a/crates/optimize/Cargo.toml b/crates/optimize/Cargo.toml index 184ee9697..3b3bbac75 100644 --- a/crates/optimize/Cargo.toml +++ b/crates/optimize/Cargo.toml @@ -12,6 +12,7 @@ workspace = true [dependencies] pdfcraft-cos = { workspace = true } +pdfcraft-filters = { workspace = true } pdfcraft-content = { workspace = true } pdfcraft-model = { workspace = true } pdfcraft-annot = { workspace = true } diff --git a/crates/optimize/README.md b/crates/optimize/README.md index 3d517eef3..d061d2cad 100644 --- a/crates/optimize/README.md +++ b/crates/optimize/README.md @@ -21,7 +21,9 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going side-by-side tiles, rotated strips, masks. Content arrays use the shared space boundary policy, so trailing comments, split operands and graphics state keep their meaning after merging. - **Images:** the effective resolution of each image is measured where pages draw it (the CTM - through form XObjects, up to 12 deep; the smallest over all uses). If a use can't be + through form XObjects, up to 12 deep; the smallest over all uses). + Page-local `/UserUnit` scaling is included, so large-format pages are measured in physical inches. + If a use can't be measured (forms nested deeper, or drawn too many times), no image is resampled. Colour and grayscale images above a threshold are resampled with a bicubic (Catmull-Rom) filter to a target resolution and stored as JPEG (Acrobat's quality steps) or ZIP; soft masks are resampled with their image. @@ -30,10 +32,23 @@ let report = optimize_with_progress(&mut doc, &settings, &mut |stage| keep_going - **Discard objects:** thumbnails, alternate images, document tags, print settings. - **Clean up:** Flate for streams with no filter; for streams with an outer ASCII85 or ASCIIHex layer, that layer is removed when the result is smaller (the inner filters keep their encoded - bytes). Streams with `/DecodeParms`, images and object streams are left alone. + bytes; streams with `/DecodeParms`, images and object streams keep their ASCII layer. LZW-compressed + streams, content and images alike, are decoded and stored with Flate instead (Acrobat's "In + streams that use LZW encoding, use Flate instead"): a predictor carries over to Flate without + `EarlyChange`, filters applied before the LZW one go with it and those after it keep their + bytes. A stream whose LZW data doesn't decode cleanly is left as it is. Not yet: inline images. The engine runs Remove Hidden Information (`pdfcraft-redact`) for the user-data categories, then this, then merges identical objects and writes a full, compressed save (`pdfcraft_engine::optimizer::OptimizeJob`, which the UI runs on a worker thread with a progress bar). JPEG decoding and encoding and resampling come from the `image` crate. + +## Target PDF version: partial compatibility + +The target selects the header version, clears the catalog `/Version`, controls object +streams (disabled for PDF 1.4), and checks encryption requirements. It does **not** +detect or convert content newer than the target: optional content, JPEG 2000 images, +16-bit images, OpenType fonts and PDF 2.0 features can remain. A PDF 1.4 header +therefore does not guarantee compatibility with PDF 1.4 readers or validators. +`optimize.compatibility-level` remains partial until content compatibility is checked. diff --git a/crates/optimize/src/images.rs b/crates/optimize/src/images.rs index 896542285..d66cc3318 100644 --- a/crates/optimize/src/images.rs +++ b/crates/optimize/src/images.rs @@ -158,15 +158,18 @@ pub fn effective_resolutions(doc: &Document, pages: &[ObjRef]) -> HashMap