diff --git a/AGENTS.md b/AGENTS.md index 0665ad1d3b..1cb3e1c34a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -71,6 +71,7 @@ Anything else is forbidden, including assets whose licence you cannot prove from - Font assets shared by the Crafting Apps, including every Japanese font, live in [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts), not here. **Never commit a font file to this repository**; adding a font means adding it to craft-fonts. (The Latin UI fonts already in `assets/fonts/` — Inter, JetBrains Mono, Dancing Script — stay.) Team members: the shared rules are in [craftrules `standards/fonts.md`](https://github.com/storytold/craftrules/blob/main/standards/fonts.md) (internal). - craft-fonts is an **optional build input**, never a Cargo dependency: `CRAFT_FONTS_DIR= cargo build` makes `crates/fonts/build.rs` embed its fonts as `pdfcraft_fonts::CRAFT_FONTS`. Without it PdfCraft builds, tests and runs, but has no Japanese face (Japanese UI text shows replacement boxes, and writing Japanese into a PDF returns a clear error). Release builds always set it (`.github/workflows/release.yml`), and `ATTRIBUTION.toml` lists it as a `[[build_input]]`. - The desktop app may read one font **already installed on the user's machine** at runtime, as the last interface fallback for characters no embedded face has (`crates/ui-egui/src/system_fonts.rs`). That font is never embedded, shipped, committed or copied, and never appears in published screenshots (`PDFCRAFT_SYSTEM_FONTS=0`, which `cargo xtask screenshots` sets). It is not a way around §1.2: nothing from it may enter a PDF, a package or the repository. +- The page viewer may also draw a CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that a PDF does **not embed** with one face already installed on the machine (`crates/render/src/system_font.rs`), **on screen only**. `RenderConfig::system_fonts` is off by default and is set only for the pool that draws the viewer (`display_config` in `crates/engine/src/lib.rs`), so exports, printing, OCR and the automation tools never use it. Like the interface font it is read at runtime and never embedded, shipped, committed or copied; Noto CJK and Source Han are never candidates (§1.1), nor is any Adobe family; and `PDFCRAFT_SYSTEM_FONTS=0` turns it off. It is not a way around §1.2 either: nothing drawn with it may be saved or printed. ### 1.5 Procedure for adding or changing an asset diff --git a/ROADMAP.md b/ROADMAP.md index 27c8e1b6bb..a984ca53ef 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -190,6 +190,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md | 8 | 1.0 polish: performance, right-to-left, missing languages, keyboard-only (M14) | 55–95 | ## Progress log +- **2026-10-23 (M2, #234, continuation of #1567):** Reconciled installed CJK viewer fallback with current rendering changes; Windows preserves installed Song/Hei/Kai families and bounds font reads. GBK-EUC-H CIDFontType2 regression coverage includes text extraction; viewer fallback also survives reload with retained preferences. System fonts remain display-only. No overall readiness estimate changed. - **2026-10-22 (M14, macOS association recovery candidate):** Successful desktop open/save/sign can queue recovery of an existing per-file binding to the running PDFCraft bundle through NSWorkspace. Metadata-only watcher changes retain undo and avoid false overwrite prompts; real same-length/same-mtime edits remain protected. Rebased on 0.12.0 and reuses #3042's shared atomic metadata preservation; the bounded recovery parser does not narrow save limits. Prior native Preview TEST-signature and Finder workflows passed on macOS 27.0.1; other versions remain unqualified. Estimates unchanged. - **2026-10-13 (M7, free image rotation):** A selected page image has a rotate knob (Shift: 15 degree steps) and `image_edit` a `rotate_by` action; the angle is written as one undoable `cm`. Snapping to page edges while moving is in #1740; resize snapping is not done. Estimates unchanged. @@ -298,6 +299,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md - **2026-10-22 (M14, JPEG and TIFF export):** Export ▸ Image as JPEG or TIFF refuses a raster that is not exactly width × height pixels, as PNG export has since #2425. A raster a pixel short panicked inside the JPEG and TIFF encoders, and a partial pixel at the end was dropped and the rest written as a valid image. Rendered pages are always the right size, so what exports is unchanged. Estimates unchanged. - **2026-10-13 (G14, Vietnamese localization):** Added Vietnamese (`vi`, Tiếng Việt) as an interface and date language with a 2,591-entry catalog (`plural_none`) covering 100% of the current localizable key union (plus `theme`/`Appearance`). Locale tags `vi` / `vi-VN` / `vi_VN.UTF-8` resolve; calendar months use numeric `Tháng 1`…`12` / `tháng 1`…`12`. Native-speaker community review recorded. The twelve-language localization mean moves from ≈59% to ≈68%; weighted readiness ≈49% → ≈50%. Status stays `partial`: engine/OS details, document contents, filenames, CLI/MCP and external help stay outside the catalog by design. +- **2026-10-12 (M2, installed CJK fonts):** A CID font of an Adobe CJK collection (Japan1, GB1, CNS1, Korea1) that a PDF doesn't embed, such as the Kozuka Gothic of an Acrobat text box or the SongtiTC of a Pages export, drew nothing unless craft-fonts had a Japanese face. After craft-fonts, it now draws with a face already installed on the machine (PingFang, Heiti, Microsoft JhengHei / YaHei, Malgun and the like; read at runtime, never bundled, and used only for the on-screen viewer, never for exports, printing or OCR; Noto CJK and Source Han excluded; `PDFCRAFT_SYSTEM_FONTS=0` turns it off). The face of a collection with several regions (PingFang SC, TC) is chosen by region. Checked on macOS with a synthetic Traditional Chinese, Simplified Chinese and Korean page; the Windows and Linux font lists are untested, and a machine with none of the listed faces still draws nothing. - **2026-10-13 (M3, typed zoom):** The right rail's zoom percentage is an editable field: type 150, 150% or 62.5 % and press Enter, Escape discards, and nonsense keeps the current zoom. The control channel's zoom option shares its parser, which now refuses zero, negative and non-finite values. - **2026-10-13 (M4, indirect page rotation):** Rotation queries and edits resolve indirect integer values on pages and their page-tree ancestors. Turning a page already at 270° by 90° now saves it at 0°, including when its rotation is inherited. Synthetic full/incremental save-and-reopen tests cover nested inheritance, sibling preservation, negative/large angles and invalid-value fallback. Estimates unchanged. - **2026-10-14 (M1, Flate fallback):** Raw deflate whose first bytes resemble a zlib header no longer loses a decoding-limit error during fallback. Strict and tolerant filter regressions and a COS stream regression cover refusal at the caller limit and successful decoding within it. No parity status or estimate changes. diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index f257865d3a..5a698fd7d7 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -1074,11 +1074,19 @@ fn set_default_layer_state(cos: &mut pdfcraft_cos::Document, layers: &[Layer]) { cos.set(root_ref, Object::Dict(root)); } +/// The configuration of the pool that draws the viewer: the document's own, plus installed CJK +/// fonts for fonts a PDF does not embed (`RenderConfig::system_fonts`). Exports, printing, OCR and +/// the automation tools render with the document's configuration itself, so nothing drawn with an +/// installed font is saved or printed (AGENTS.md §1.2, §1.4). +fn display_config(config: &RenderConfig) -> RenderConfig { + RenderConfig { system_fonts: true, ..config.clone() } +} + /// Render `doc` with the layer visibility in its `info.layers`. fn use_layer_choices(doc: &mut Document) { let overrides: Vec<(i32, i32, bool)> = doc.info.layers.iter().map(|l| (l.id.0 as i32, l.id.1 as i32, l.visible)).collect(); doc.config.layers = Arc::new(overrides); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } /// Apply a set-layer-visibility action to `layers`, one change at a time, so a toggle flips the @@ -3699,7 +3707,7 @@ impl Session { let sig_cache = Arc::new(pdfcraft_sign::DigestCache::default()); let signatures = editor.as_ref().map(|e| signatures_of(&e.cos, &bytes, trust, &sig_cache)).unwrap_or_default(); let display = display_bytes(editor.as_mut(), &bytes); - let renderer = RenderPool::new(display.clone(), render_threads(), config.clone()); + let renderer = RenderPool::new(display.clone(), render_threads(), display_config(&config)); let (form, adopted) = editor .as_ref() .map(|e| { @@ -4124,7 +4132,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; Ok(()) } @@ -4169,7 +4177,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; doc.fonts_stale = true; Ok(()) @@ -4196,7 +4204,7 @@ impl Session { /// After a refresh failed: draw the display bytes it left (the original, once released), /// so the page still shows a document. fn show_released(doc: &mut Document) { - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } /// Rebuild working bytes, inspection and renderer from the current edit state. @@ -4246,7 +4254,7 @@ impl Session { let signatures = signatures_of(&editor.cos, &bytes, &doc.trust, &doc.sig_cache); doc.display = display_bytes(Some(editor), &bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.bytes = bytes; Ok(()) } @@ -4407,7 +4415,7 @@ impl Session { { doc.config.reading_colors = old.config.reading_colors; doc.config.hide_comments = old.config.hide_comments; - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } // Caches keyed by the edit generation (edit lines, images, measurements) must see a new // one; there is nothing new to autosave. @@ -4994,7 +5002,7 @@ impl Session { return false; } doc.config.hide_comments = hide; - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.generation += 1; true } @@ -5007,7 +5015,7 @@ impl Session { return false; } doc.config.reading_colors = colors; - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); doc.generation += 1; true } @@ -5049,7 +5057,7 @@ impl Session { let signatures = if doc.signatures.is_empty() { Vec::new() } else { signatures_of(&editor.cos, &doc.bytes, &doc.trust, &doc.sig_cache) }; doc.display = display_bytes(Some(editor), &doc.bytes); doc.signatures = Arc::new(signatures); - doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), doc.config.clone()); + doc.renderer = RenderPool::new(doc.display.clone(), render_threads(), display_config(&doc.config)); } } diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs index 3012a1f3fe..547aa4a35a 100644 --- a/crates/engine/src/tests.rs +++ b/crates/engine/src/tests.rs @@ -438,6 +438,80 @@ fn edits_update_view_data_and_mark_dirty() { assert_eq!(page_texts(&s, id), ["Page 1", "Page 3"], "renderer refreshed"); } +/// A font of an Adobe CJK collection that a PDF does not embed is drawn with an installed face on +/// the viewer's pool only. An export of the same page, which can be saved, has nothing drawn with +/// an installed font (AGENTS.md §1.2, §1.4). Skipped where no such face is installed. +#[test] +fn installed_cjk_fonts_draw_in_the_viewer_but_not_in_an_export() { + let content = "BT /F1 40 Tf 5 15 Td <7E419AD45B57> Tj ET"; + let pdf = format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 60] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj +4 0 obj << /Length {} >> stream +{content} +endstream endobj +5 0 obj << /Type /Font /Subtype /Type0 /BaseFont /MSung-Light /Encoding /UniCNS-UCS2-H /DescendantFonts [6 0 R] >> endobj +6 0 obj << /Type /Font /Subtype /CIDFontType0 /BaseFont /MSung-Light /CIDSystemInfo << /Registry (Adobe) /Ordering (CNS1) /Supplement 2 >> /FontDescriptor 7 0 R /DW 1000 >> endobj +7 0 obj << /Type /FontDescriptor /FontName /MSung-Light /Flags 6 /FontBBox [0 -141 1000 859] /ItalicAngle 0 /Ascent 859 /Descent -141 /CapHeight 700 /StemV 80 >> endobj +trailer << /Root 1 0 R >> +%%EOF", + content.len() + ); + let mut s = Session::new(); + let id = s.open("cns1.pdf", None, Arc::new(pdf.into_bytes()), None).unwrap(); + let doc = s.get(id).unwrap(); + doc.renderer.set_queue(vec![pdfcraft_render::RenderRequest { + page: 0, + kind: pdfcraft_render::RequestKind::Pixels, + scale: 1.0, + ..Default::default() + }]); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(30); + let viewer = loop { + if let Some(page) = doc.renderer.try_recv() { + assert!(page.error.is_none(), "{:?}", page.error); + break page; + } + assert!(std::time::Instant::now() < deadline, "page render did not finish"); + std::thread::yield_now(); + }; + let inked = |rgba: &[u8]| rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count(); + // Whether this machine has a face that can draw the page at all, asked of a renderer with the + // fallback switched on by hand, so a viewer pool that lost it fails below instead of skipping. + let probe = pdfcraft_render::PageRenderer::new(doc.display.clone(), pdfcraft_render::RenderConfig { system_fonts: true, ..Default::default() }) + .render(pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }); + assert!(probe.error.is_none(), "{:?}", probe.error); + if inked(&probe.rgba) < 300 { + eprintln!("no installed Traditional Chinese face (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); + return; + } + assert!(inked(&viewer.rgba) > 300, "the viewer's pool draws it with the installed face ({} dark pixels)", inked(&viewer.rgba)); + let png = export::Exporter::new(doc).png(0, 72.0).unwrap(); + let exported = image::load_from_memory(&png).unwrap().into_rgba8(); + assert!(inked(exported.as_raw()) < 50, "an export draws nothing with an installed font ({} dark pixels)", inked(exported.as_raw())); + + // Reload rebuilds the pool when retaining a view preference. That path must keep + // the display-only fallback too; otherwise Chinese disappears after a file refresh. + let bytes = doc.bytes.clone(); + assert!(s.set_hide_comments(id, true)); + s.reload(id, bytes).unwrap(); + let doc = s.get(id).unwrap(); + assert!(doc.comments_hidden()); + doc.renderer.set_queue(vec![pdfcraft_render::RenderRequest { page: 0, scale: 1.0, ..Default::default() }]); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(30); + let reloaded = loop { + if let Some(page) = doc.renderer.try_recv() { + assert!(page.error.is_none(), "{:?}", page.error); + break page; + } + assert!(std::time::Instant::now() < deadline, "reloaded page render did not finish"); + std::thread::yield_now(); + }; + assert!(inked(&reloaded.rgba) > 300, "reload must retain visible Chinese text"); +} + #[test] fn metadata_edit_keeps_the_renderer_without_changing_page_content() { let (mut s, id) = session_with(1); diff --git a/crates/render/Cargo.toml b/crates/render/Cargo.toml index 28202a38b4..71467dfeb2 100644 --- a/crates/render/Cargo.toml +++ b/crates/render/Cargo.toml @@ -13,6 +13,8 @@ pdfcraft-cos = { workspace = true } pdfcraft-fonts = { workspace = true } hayro = { workspace = true } hayro-syntax = { workspace = true } +# Picks the face of an installed font collection (the version pdfcraft-fonts already uses). +skrifa = "0.44" lopdf = { workspace = true } kurbo = { workspace = true } thiserror = { workspace = true } @@ -27,7 +29,5 @@ bytemuck = { version = "1" } getrandom = { version = "0.4", features = ["wasm_js"] } [dev-dependencies] -# Checks the Japanese substitute's character coverage; the version pdfcraft-fonts already uses. -skrifa = "0.44" # Builds the decompression-bomb streams in the raster tests. flate2 = { workspace = true } diff --git a/crates/render/src/lib.rs b/crates/render/src/lib.rs index cc1e4af2e3..87a7e7731d 100644 --- a/crates/render/src/lib.rs +++ b/crates/render/src/lib.rs @@ -13,6 +13,7 @@ mod pixels; mod raster; mod reading; mod structure; +mod system_font; pub mod text; pub mod transition; diff --git a/crates/render/src/raster.rs b/crates/render/src/raster.rs index 1f8d2f19ef..0a2fc29e04 100644 --- a/crates/render/src/raster.rs +++ b/crates/render/src/raster.rs @@ -71,15 +71,23 @@ pub struct RenderConfig { /// The page clears to transparent instead of white (or the reading background): for a page /// holding only what is laid over another raster, such as a comment while it is dragged. pub transparent: bool, + /// Draw a CID font of an Adobe CJK collection that the PDF does not embed with one face already + /// installed on this machine (see `system_font`). Off by default and on only for the pool that + /// draws the viewer: exports, printing and OCR render without it, so nothing drawn with an + /// installed font is saved or printed (AGENTS.md §1.2, §1.4). + pub system_fonts: bool, } impl RenderConfig { fn settings(&self) -> InterpreterSettings { let standard = InterpreterSettings::default().font_resolver; + let system_fonts = self.system_fonts; InterpreterSettings { ocg_overrides: self.layers.clone(), hide_comments: self.hide_comments, - font_resolver: Arc::new(move |query| japanese_fallback(query).or_else(|| standard(query))), + font_resolver: Arc::new(move |query| { + japanese_fallback(query).or_else(|| system_fonts.then(|| system_fallback(query)).flatten()).or_else(|| standard(query)) + }), ..InterpreterSettings::default() } } @@ -112,6 +120,14 @@ fn japanese_fallback(query: &FontQuery) -> Option<(FontData, u32)> { Some((Arc::new(face.bytes), 0)) } +/// An installed face for a CID font of an Adobe CJK collection that the PDF doesn't embed and +/// craft-fonts has no face for (Traditional and Simplified Chinese, Korean, and Japanese in a +/// build without craft-fonts). `None` for every other font. +fn system_fallback(query: &FontQuery) -> Option<(FontData, u32)> { + let FontQuery::Fallback(f) = query else { return None }; + crate::system_font::fallback(f) +} + #[derive(Debug, PartialEq)] enum JapaneseFace { Mincho, @@ -7683,6 +7699,76 @@ trailer << /Root 1 0 R >> } } + /// Chinese and Korean CID fonts that aren't embedded (Adobe-CNS1 `MSung-Light`, Adobe-GB1 + /// `STSong-Light`, Adobe-Korea1 `HYGoThic-Medium`; Acrobat also writes Japanese `KozGoPr6N` + /// text boxes without the font) drew nothing, because craft-fonts has only Japanese faces. + /// They now draw in a face already installed on the machine, read at runtime. A machine + /// without such a face draws nothing, as before, so each check is skipped there. + #[test] + fn non_embedded_chinese_and_korean_cid_fonts_draw_with_an_installed_face() { + use hayro::hayro_interpret::font::FallbackFontQuery; + use hayro::hayro_interpret::hayro_cmap::CharacterCollection; + // (base font, collection, ordering, encoding, encoded text, descendant subtype) + let cases = [ + ("MSung-Light", CidFamily::AdobeCNS1, "CNS1", "UniCNS-UCS2-H", "7E419AD45B57", "CIDFontType0"), // 繁體字 + ("STSong-Light", CidFamily::AdobeGB1, "GB1", "UniGB-UCS2-H", "72698D448DF7", "CIDFontType0"), // 物资跷 + ("HYGoThic-Medium", CidFamily::AdobeKorea1, "Korea1", "UniKS-UCS2-H", "D55CAE00", "CIDFontType0"), // 한글 + // The real Windows book uses non-embedded TrueType CID fonts and a legacy GBK CMap, + // without ToUnicode. These bytes encode 中文; they are not Unicode code points. + ("SimSun", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("SimHei", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ("KaiTi", CidFamily::AdobeGB1, "GB1", "GBK-EUC-H", "D6D0CEC4", "CIDFontType2"), + ]; + for (base_font, family, ordering, encoding, code, subtype) in cases { + let query = FontQuery::Fallback(FallbackFontQuery { + post_script_name: Some(base_font.into()), + character_collection: Some(CharacterCollection { family, supplement: 2 }), + ..FallbackFontQuery::default() + }); + if super::system_fallback(&query).is_none() { + eprintln!("no installed face for {base_font} (or PDFCRAFT_SYSTEM_FONTS=0): skipped"); + continue; + } + let content = format!("BT /F1 40 Tf 5 15 Td <{code}> Tj ET"); + let pdf = format!( + "%PDF-1.7 +1 0 obj << /Type /Catalog /Pages 2 0 R >> endobj +2 0 obj << /Type /Pages /Kids [3 0 R] /Count 1 >> endobj +3 0 obj << /Type /Page /Parent 2 0 R /MediaBox [0 0 200 60] /Contents 4 0 R /Resources << /Font << /F1 5 0 R >> >> >> endobj +4 0 obj << /Length {} >> stream +{content} +endstream endobj +5 0 obj << /Type /Font /Subtype /Type0 /BaseFont /{base_font} /Encoding /{encoding} /DescendantFonts [6 0 R] >> endobj +6 0 obj << /Type /Font /Subtype /{subtype} /BaseFont /{base_font} /CIDSystemInfo << /Registry (Adobe) /Ordering ({ordering}) /Supplement 2 >> + /FontDescriptor 7 0 R /DW 1000 >> endobj +7 0 obj << /Type /FontDescriptor /FontName /{base_font} /Flags 6 /FontBBox [0 -141 1000 859] /ItalicAngle 0 /Ascent 859 /Descent -141 /CapHeight 700 /StemV 80 >> endobj +trailer << /Root 1 0 R >> +%%EOF", + content.len() + ); + let pdf = Arc::new(pdf.into_bytes()); + let ink = |config: RenderConfig| { + let mut r = PageRenderer::new(pdf.clone(), config); + let p = r.render(RenderRequest { page: 0, kind: RequestKind::Pixels, tile: None, scale: 1.0, tag: 0, ..Default::default() }); + assert!(p.error.is_none(), "{base_font}: {:?}", p.error); + p.rgba.as_chunks::<4>().0.iter().filter(|c| c[0] < 128).count() + }; + // Two to four glyphs at 40 pt cover several hundred dark pixels; a blank run has none. + let shown = ink(RenderConfig { system_fonts: true, ..RenderConfig::default() }); + assert!(shown > 300, "{base_font}: the text is drawn ({shown} dark pixels)"); + // Off by default: exports, printing and OCR render without an installed font, so + // nothing drawn with one can be saved (AGENTS.md §1.2, §1.4). + let saved = ink(RenderConfig::default()); + assert!(saved < 50, "{base_font}: not drawn without system_fonts ({saved} dark pixels)"); + if encoding == "GBK-EUC-H" { + let mut renderer = PageRenderer::new(pdf, RenderConfig { system_fonts: true, ..Default::default() }); + let text = renderer.render(RenderRequest { page: 0, kind: RequestKind::Text, ..Default::default() }); + assert!(text.error.is_none(), "{base_font}: {:?}", text.error); + assert_eq!(text.text.unwrap().plain_text().trim(), "中文", "the fallback preserves GBK text mapping"); + } + } + } + /// Half-width katakana are common in Japanese documents set in a non-embedded Mincho font /// such as HeiseiMin-W3. The Mincho substitute must have those glyphs: one that lacks them /// (Shippori Mincho) draws them as .notdef. diff --git a/crates/render/src/system_font.rs b/crates/render/src/system_font.rs new file mode 100644 index 0000000000..626b3d10e8 --- /dev/null +++ b/crates/render/src/system_font.rs @@ -0,0 +1,438 @@ +//! Installed CJK faces for CID fonts a PDF doesn't embed. +//! +//! Acrobat and other tools often write Chinese, Japanese or Korean text with one of the Adobe +//! collections (Adobe-Japan1, GB1, CNS1, Korea1) and a font name such as `KozGoPr6N-Medium` or +//! `MSung-Light` without embedding the font. hayro maps those CIDs to Unicode itself, but it needs +//! a face to draw them with; without one the text is blank. The craft-fonts Japanese faces come +//! first (see `japanese_fallback` in `raster.rs`); this module is the last resort: one face +//! already installed on this machine, used for the viewer only (`RenderConfig::system_fonts`, off +//! for exports, printing and OCR). It is read at runtime and never embedded or shipped +//! (AGENTS.md §1.4), Noto CJK and Source Han are left out on purpose (§1.1), and +//! `PDFCRAFT_SYSTEM_FONTS=0` turns it off, as for the interface font. + +use std::collections::HashMap; +use std::io::Read as _; +use std::path::{Path, PathBuf}; +use std::sync::{Arc, Mutex, OnceLock, PoisonError}; + +use hayro::hayro_interpret::font::{FallbackFontQuery, FontData}; +use hayro::hayro_interpret::hayro_cmap::CidFamily; +use skrifa::MetadataProvider as _; +use skrifa::string::StringId; + +/// Larger files are not read: a font path is still untrusted input. CJK system faces are big +/// collections (PingFang is tens of megabytes). +const MAX_BYTES: u64 = 96 << 20; +/// Faces tried in a collection (`.ttc`). +const MAX_FACES: u32 = 32; +/// Downloaded font assets looked at per kind on macOS. +const MAX_ASSETS: usize = 512; + +/// The writing system a character collection is for. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +enum Script { + Japanese, + SimplifiedChinese, + TraditionalChinese, + Korean, +} + +impl Script { + fn of(family: &CidFamily) -> Option { + match family { + CidFamily::AdobeJapan1 => Some(Self::Japanese), + CidFamily::AdobeGB1 => Some(Self::SimplifiedChinese), + CidFamily::AdobeCNS1 => Some(Self::TraditionalChinese), + CidFamily::AdobeKorea1 => Some(Self::Korean), + CidFamily::AdobeIdentity | CidFamily::Custom { .. } => None, + } + } + + /// A character the face must have to be worth using. Each is common and not shared by the + /// other scripts' faces: 物 (Japanese; also Chinese, so it only rules out Latin-only faces), + /// 欢 (Simplified only), 說 (Traditional only), 한 (Hangul). + fn probe(self) -> char { + match self { + Self::Japanese => '\u{7269}', + Self::SimplifiedChinese => '\u{6B22}', + Self::TraditionalChinese => '\u{8AAA}', + Self::Korean => '\u{D55C}', + } + } +} + +/// One installed file and the words (in a face's family name) that pick the face for the right +/// region out of a collection that holds several (PingFang SC, TC and HK share one file). +struct Candidate { + path: PathBuf, + region: &'static [&'static str], +} + +/// Keep named Windows body, heading and calligraphic faces distinct. A cache keyed only +/// by script and weight would make the first Chinese font on a page stand in for all three. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +enum ChineseFamily { + Default, + Song, + Hei, + Kai, +} + +impl ChineseFamily { + fn of(name: &str) -> Self { + let name = name.to_ascii_lowercase(); + if ["simsun", "stsong", "songti", "宋体"].iter().any(|hint| name.contains(hint)) { + Self::Song + } else if ["simhei", "stheiti", "黑体"].iter().any(|hint| name.contains(hint)) { + Self::Hei + } else if ["kaiti", "stkaiti", "楷体"].iter().any(|hint| name.contains(hint)) { + Self::Kai + } else { + Self::Default + } + } +} + +/// The installed face to draw a CID font of `query`'s collection with, as `(data, face index)`. +/// `None` when it is turned off, the collection isn't one of the four, or nothing fits. +pub(crate) fn fallback(query: &FallbackFontQuery) -> Option<(FontData, u32)> { + if cfg!(target_arch = "wasm32") || std::env::var_os("PDFCRAFT_SYSTEM_FONTS").is_some_and(|v| v == "0") { + return None; + } + let script = Script::of(&query.character_collection.as_ref()?.family)?; + let bold = query.is_bold || query.font_weight >= 600; + let family = ChineseFamily::of(query.post_script_name.as_deref().unwrap_or_default()); + /// A face file and the index of the chosen face in it, or `None` when nothing fits. + type Face = Option<(Arc>, u32)>; + type Faces = HashMap<(Script, bool, ChineseFamily), Face>; + static CACHE: Mutex> = Mutex::new(None); + let mut cache = CACHE.lock().unwrap_or_else(PoisonError::into_inner); + let found = cache.get_or_insert_with(HashMap::new).entry((script, bold, family)).or_insert_with(|| { + candidates(script, family, bold).iter().find_map(|c| { + read(c, script.probe(), bold).inspect(|(_, index)| { + log::info!("CJK font fallback for {script:?}/{family:?}: {} (face {index})", c.path.display()); + }) + }) + }); + found.as_ref().map(|(bytes, index)| (Arc::clone(bytes) as FontData, *index)) +} + +fn read(candidate: &Candidate, probe: char, bold: bool) -> Option<(Arc>, u32)> { + let bytes = file_bytes(&candidate.path)?; + let index = pick_face(&bytes, probe, candidate.region, bold)?; + Some((bytes, index)) +} + +/// The bytes of an installed font file, read once for the whole process and shared: a collection +/// such as `PingFang.ttc` is tens of megabytes and is the candidate of several scripts and both +/// weights, so the per-script cache keeps only a handle and a face index. A file that cannot be +/// used is remembered too, so it is not tried again for every script or family. +fn file_bytes(path: &Path) -> Option>> { + /// The bytes of each file tried, or `None` for one that cannot be used. + type Files = HashMap>>>; + static FILES: Mutex> = Mutex::new(None); + let mut files = FILES.lock().unwrap_or_else(PoisonError::into_inner); + files.get_or_insert_with(HashMap::new).entry(path.to_path_buf()).or_insert_with(|| read_file(path, MAX_BYTES).map(Arc::new)).clone() +} + +/// A regular file of at most `max` bytes, or `None`: a font path is untrusted input. +fn read_file(path: &Path, max: u64) -> Option> { + // Reject special files before opening them (in particular a FIFO can block on open). + let meta = std::fs::metadata(path).ok()?; + if !meta.is_file() || meta.len() > max { + return None; + } + let file = std::fs::File::open(path).ok()?; + let meta = file.metadata().ok()?; + if !meta.is_file() || meta.len() > max { + return None; + } + // Bound the actual read too: the file can grow after the metadata check. + let mut bytes = Vec::new(); + file.take(max.saturating_add(1)).read_to_end(&mut bytes).ok()?; + (bytes.len() as u64 <= max).then_some(bytes) +} + +/// The best face of the file that parses and maps `probe`: the region's face when the file has +/// one, then the regular (or, for bold text, the semibold or bold) weight, then the first. +fn pick_face(bytes: &[u8], probe: char, region: &[&str], bold: bool) -> Option { + let mut best: Option<(u32, u32)> = None; + for index in 0..MAX_FACES { + let Ok(font) = skrifa::FontRef::from_index(bytes, index) else { continue }; + if font.charmap().map(probe).is_none() { + continue; + } + let names = |id: StringId| font.localized_strings(id).map(|s| s.chars().collect::()).collect::>(); + let family = names(StringId::FAMILY_NAME).join(" "); + let style = names(StringId::SUBFAMILY_NAME).join(" ").to_ascii_lowercase(); + let mut score = 1; + if region.iter().any(|r| family.split_whitespace().any(|w| w == *r)) { + score += 4; + } + let weighty = style.contains("bold") || style.contains("semibold") || style.contains("heavy"); + let plain = style.is_empty() || style.contains("regular") || style.contains("normal") || style.contains("medium") && !bold; + if (bold && weighty) || (!bold && plain && !weighty) { + score += 2; + } + if best.is_none_or(|(_, s)| score > s) { + best = Some((index, score)); + } + } + best.map(|(index, _)| index) +} + +fn candidates(script: Script, family: ChineseFamily, bold: bool) -> Vec { + let region: &'static [&'static str] = match script { + Script::Japanese => &["JP", "Japanese"], + Script::SimplifiedChinese => &["SC", "GB"], + Script::TraditionalChinese => &["TC", "HK", "CNS"], + Script::Korean => &["KR", "Korean"], + }; + let files: Vec = if cfg!(windows) { + let dir = windows_fonts(); + let names: &[&str] = match script { + Script::Japanese => &["YuGothR.ttc", "meiryo.ttc", "msgothic.ttc"], + Script::SimplifiedChinese => match family { + ChineseFamily::Song => &["simsun.ttc", "msyh.ttc", "msjh.ttc"], + ChineseFamily::Hei => &["simhei.ttf", "msyh.ttc", "simsun.ttc", "msjh.ttc"], + ChineseFamily::Kai => &["simkai.ttf", "simsun.ttc", "msyh.ttc", "msjh.ttc"], + ChineseFamily::Default if bold => &["msyhbd.ttc", "msyh.ttc", "simsun.ttc", "msjh.ttc"], + ChineseFamily::Default => &["msyh.ttc", "simsun.ttc", "msjh.ttc"], + }, + Script::TraditionalChinese => &["msjh.ttc", "mingliu.ttc", "msyh.ttc"], + Script::Korean => &["malgun.ttf", "gulim.ttc"], + }; + names.iter().map(|f| dir.join(f)).collect() + } else if cfg!(target_os = "macos") { + let names: &[&str] = match script { + Script::Japanese => &["ヒラギノ角ゴシック W3.ttc", "Hiragino Sans GB.ttc", "PingFang.ttc"], + Script::SimplifiedChinese => &["PingFang.ttc", "Hiragino Sans GB.ttc", "STHeiti Light.ttc"], + Script::TraditionalChinese => &["PingFang.ttc", "STHeiti Light.ttc", "Hiragino Sans GB.ttc"], + Script::Korean => &["AppleSDGothicNeo.ttc", "AppleGothic.ttf", "PingFang.ttc"], + }; + // The system folder first, then the downloaded font assets: on recent macOS a face such as + // PingFang lives in `/System/Library/AssetsV2/com_apple_MobileAsset_Font*/.asset/AssetData`. + let mut dirs = vec![PathBuf::from("/System/Library/Fonts")]; + dirs.extend(macos_font_asset_dirs().iter().cloned()); + names.iter().flat_map(|f| dirs.iter().map(move |d| d.join(f))).collect() + } else { + let names: &[&str] = match script { + Script::Korean => { + &["truetype/nanum/NanumGothic.ttf", "nanum/NanumGothic.ttf", "truetype/baekmuk/gulim.ttf", "truetype/unfonts-core/UnDotum.ttf"] + } + Script::Japanese => &[ + "truetype/fonts-japanese-gothic.ttf", + "ipa-gothic/ipag.ttf", + "truetype/takao-gothic/TakaoGothic.ttf", + "truetype/vlgothic/VL-Gothic-Regular.ttf", + ], + Script::SimplifiedChinese | Script::TraditionalChinese => &[ + "truetype/wqy/wqy-microhei.ttc", + "wenquanyi/wqy-microhei/wqy-microhei.ttc", + "wqy-microhei/wqy-microhei.ttc", + "truetype/wqy/wqy-zenhei.ttc", + "wenquanyi/wqy-zenhei/wqy-zenhei.ttc", + "wqy-zenhei/wqy-zenhei.ttc", + "truetype/droid/DroidSansFallbackFull.ttf", + "google-droid-sans-fonts/DroidSansFallbackFull.ttf", + "truetype/arphic/uming.ttc", + "arphic-uming/uming.ttc", + ], + }; + ["/usr/share/fonts", "/usr/local/share/fonts"].iter().flat_map(|dir| names.iter().map(move |f| Path::new(dir).join(f))).collect() + }; + files.into_iter().map(|path| Candidate { path, region }).collect() +} + +/// The `AssetData` folders of the downloaded font assets of macOS, found once. The ids in their +/// names change with each asset version, so they are listed rather than named; at most +/// [`MAX_ASSETS`] are looked at. +fn macos_font_asset_dirs() -> &'static [PathBuf] { + static DIRS: OnceLock> = OnceLock::new(); + DIRS.get_or_init(|| { + let root = Path::new("/System/Library/AssetsV2"); + let Ok(kinds) = std::fs::read_dir(root) else { return Vec::new() }; + let mut dirs = Vec::new(); + for kind in kinds.flatten().filter(|k| k.file_name().to_string_lossy().starts_with("com_apple_MobileAsset_Font")) { + let Ok(assets) = std::fs::read_dir(kind.path()) else { continue }; + for asset in assets.flatten().take(MAX_ASSETS) { + let data = asset.path().join("AssetData"); + if data.is_dir() { + dirs.push(data); + } + } + } + dirs + }) +} + +fn windows_fonts() -> PathBuf { + let dir = std::env::var_os("WINDIR").or_else(|| std::env::var_os("SystemRoot")).map_or_else(|| PathBuf::from(r"C:\Windows"), PathBuf::from); + dir.join("Fonts") +} + +#[cfg(test)] +mod tests { + use super::*; + + const SCRIPTS: [Script; 4] = [Script::Japanese, Script::SimplifiedChinese, Script::TraditionalChinese, Script::Korean]; + + #[test] + fn chinese_family_hints_include_subset_and_localized_names() { + for name in ["SimSun", "ABCDEF+SimSun", "STSong-Light", "宋体"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Song, "{name}"); + } + for name in ["SimHei", "STHeiti", "黑体"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Hei, "{name}"); + } + for name in ["KaiTi", "STKaiti", "楷体"] { + assert_eq!(ChineseFamily::of(name), ChineseFamily::Kai, "{name}"); + } + assert_eq!(ChineseFamily::of("Helvetica"), ChineseFamily::Default); + } + + #[test] + fn broken_and_missing_files_are_skipped() { + assert_eq!(pick_face(b"", '\u{7269}', &[], false), None); + assert_eq!(pick_face(b"not a font at all", '\u{7269}', &[], false), None); + assert_eq!(pick_face(&[0u8; 4096], '\u{7269}', &[], false), None); + let missing = Candidate { path: PathBuf::from("definitely/not/here.ttc"), region: &[] }; + assert!(read(&missing, '\u{7269}', false).is_none()); + // A directory is not a font. + let dir = Candidate { path: std::env::temp_dir(), region: &[] }; + assert!(read(&dir, '\u{7269}', false).is_none()); + } + + /// The files are bounded by `max`: a larger one is not read. + #[test] + fn a_file_over_the_cap_is_not_read() { + let inter = Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../assets/fonts/Inter-Regular.ttf")); + assert!(read_file(inter, MAX_BYTES).is_some()); + assert!(read_file(inter, 1024).is_none()); + } + + /// Two faces drawn from one installed file (two scripts, or regular and bold) hold the same + /// bytes, not a copy each; a file that can't be used is remembered as unusable. + #[test] + fn one_installed_file_is_read_once_and_shared() { + let inter = Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/../../assets/fonts/Inter-Regular.ttf")); + let first = file_bytes(inter).expect("readable"); + let second = file_bytes(inter).expect("readable"); + assert!(Arc::ptr_eq(&first, &second), "one allocation for the file"); + let missing = Path::new("definitely/not/here.ttc"); + assert!(file_bytes(missing).is_none() && file_bytes(missing).is_none()); + } + + #[test] + fn a_latin_face_is_rejected_for_every_script() { + let inter = include_bytes!("../../../assets/fonts/Inter-Regular.ttf"); + for script in SCRIPTS { + assert_eq!(pick_face(inter, script.probe(), &[], false), None, "{script:?}"); + } + assert_eq!(pick_face(inter, 'A', &[], false), Some(0)); + } + + #[test] + fn candidates_are_absolute_font_files_without_adobe_designs() { + for script in SCRIPTS { + let list = candidates(script, ChineseFamily::Default, false); + assert!(!list.is_empty(), "{script:?}"); + for c in list { + assert!(c.path.is_absolute(), "{}", c.path.display()); + assert!(c.path.extension().is_some_and(|e| e == "ttf" || e == "ttc"), "{}", c.path.display()); + let name = c.path.to_string_lossy().to_lowercase(); + assert!( + !["notosanscjk", "notoserifcjk", "sourcehan", "source-han", "adobe", "kozuka", "minion"].iter().any(|b| name.contains(b)), + "{name}" + ); + } + } + } + + #[test] + fn only_the_four_adobe_collections_have_a_script() { + assert_eq!(Script::of(&CidFamily::AdobeCNS1), Some(Script::TraditionalChinese)); + assert_eq!(Script::of(&CidFamily::AdobeGB1), Some(Script::SimplifiedChinese)); + assert_eq!(Script::of(&CidFamily::AdobeKorea1), Some(Script::Korean)); + assert_eq!(Script::of(&CidFamily::AdobeJapan1), Some(Script::Japanese)); + assert_eq!(Script::of(&CidFamily::AdobeIdentity), None); + } + + /// A real Windows GBK book names these faces. Replacing both with YaHei hides the + /// distinction between serif body text and sans headings even when the exact face exists. + #[cfg(windows)] + #[test] + fn windows_preserves_installed_chinese_document_families() { + use hayro::hayro_interpret::hayro_cmap::CharacterCollection; + for (name, file) in [("SimSun", "simsun.ttc"), ("SimHei", "simhei.ttf"), ("KaiTi", "simkai.ttf")] { + if !windows_fonts().join(file).is_file() { + continue; + } + let query = FallbackFontQuery { + post_script_name: Some(name.into()), + character_collection: Some(CharacterCollection { family: CidFamily::AdobeGB1, supplement: 4 }), + ..FallbackFontQuery::default() + }; + let (bytes, index) = fallback(&query).expect("installed Chinese font"); + let font = skrifa::FontRef::from_index(bytes.as_ref().as_ref(), index).unwrap(); + let names: Vec = font.localized_strings(StringId::POSTSCRIPT_NAME).map(|n| n.chars().collect()).collect(); + assert!(names.iter().any(|n| n.eq_ignore_ascii_case(name)), "{name} resolved to {names:?}"); + } + } + + /// On a Mac the face for Traditional Chinese is the TC one of PingFang, not the first of the + /// collection (which is another region's), and it maps a character only Traditional has. + #[cfg(target_os = "macos")] + #[test] + fn macos_picks_the_regional_pingfang_face() { + // Found where this macOS keeps it: the system folder, or the downloaded font assets. + let path = candidates(Script::TraditionalChinese, ChineseFamily::Default, false) + .into_iter() + .map(|c| c.path) + .find(|p| p.file_name().is_some_and(|n| n == "PingFang.ttc") && p.is_file()); + let Some(bytes) = path.and_then(|p| std::fs::read(p).ok()) else { + eprintln!("PingFang is not installed here: skipped"); + return; + }; + for (script, region) in [(Script::TraditionalChinese, "TC"), (Script::SimplifiedChinese, "SC")] { + let index = pick_face(&bytes, script.probe(), &[region], false).expect("PingFang covers it"); + let font = skrifa::FontRef::from_index(&bytes, index).unwrap(); + let family: String = font.localized_strings(StringId::FAMILY_NAME).next().unwrap().chars().collect(); + assert!(family.contains(region), "{script:?} got {family:?}"); + } + } + /// A face that this macOS keeps only in its downloaded font assets (PingFang, on recent + /// versions) is among the candidates. The assets are searched here on their own, not by the + /// code under test; where there are none, nothing is checked. + #[cfg(target_os = "macos")] + #[test] + fn macos_candidates_include_a_pingfang_that_lives_in_the_font_assets() { + fn find(dir: &Path, name: &str, depth: u32) -> Option { + for entry in std::fs::read_dir(dir).ok()?.flatten() { + let path = entry.path(); + if path.file_name().is_some_and(|n| n == name) && path.is_file() { + return Some(path); + } + if depth > 0 + && path.is_dir() + && !path.is_symlink() + && let Some(found) = find(&path, name, depth - 1) + { + return Some(found); + } + } + None + } + let Some(asset) = find(Path::new("/System/Library/AssetsV2"), "PingFang.ttc", 3) else { + eprintln!("no PingFang in the font assets here: skipped"); + return; + }; + for script in [Script::SimplifiedChinese, Script::TraditionalChinese] { + assert!( + candidates(script, ChineseFamily::Default, false).iter().any(|c| c.path == asset), + "{script:?}: {} is not a candidate", + asset.display() + ); + } + } +} diff --git a/docs/development.md b/docs/development.md index 1c6cce6a95..4851f77eef 100644 --- a/docs/development.md +++ b/docs/development.md @@ -133,7 +133,7 @@ copies retain their current preferences locations. | `PDFCRAFT_TITLEBAR` | Linux: `app` draws PdfCraft's own title bar (window buttons in the tab strip, in the order of GNOME's `button-layout`, and resize edges); `system` keeps the frame winit or the desktop draws; unset, PdfCraft draws its own when the Wayland compositor draws no frames (GNOME), and leaves it to the desktop otherwise (KDE, Sway, Hyprland, X11). Where PdfCraft draws it, Preferences ▸ Use system title bar switches back to winit's frame without a restart; `system` also hides that option | | `PDFCRAFT_BUILD_LABEL` | Build time: a label for a local or patched build (letters, digits, inner dots), shown after the version in `--version`, About and the log (`0.7.1 (titlebar)`). `packaging/linux/package.sh` also puts it in the file names and the .rpm/.deb release (`1.titlebar`), which upgrades over the release of the same version | | `CRAFT_FONTS_DIR` | Build time: a [craft-fonts](https://github.com/storytold/craft-fonts) checkout to embed (Japanese fonts) | -| `PDFCRAFT_SYSTEM_FONTS` | `0` stops the desktop app from using an installed font for characters its embedded fonts lack (`cargo xtask screenshots` sets it) | +| `PDFCRAFT_SYSTEM_FONTS` | `0` disables installed-font fallback for both interface text and non-embedded CJK page text in the desktop viewer (`cargo xtask screenshots` sets it). Document fallback is display-only: exports, printing, OCR and automation retain the bundled-font path. | | `FONTCONFIG_FILE` | On Linux/Unix, the Fontconfig configuration file used to discover installed UI fallback fonts (default: `/etc/fonts/fonts.conf`); relative filenames are resolved under `FONTCONFIG_PATH` | | `FONTCONFIG_PATH` | Directory for a relative `FONTCONFIG_FILE` (default: `/etc/fonts`; the first entry of a search path is used) | diff --git a/docs/roadmap.md b/docs/roadmap.md index 92d91503bc..2b290575c8 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -140,6 +140,12 @@ Help ▸ Check for updates is manual only. ## Milestones +M2 update (2026-10-23, #234 / continuation of #1567): the desktop viewer uses installed CJK +faces for non-embedded CID fonts, including Windows GBK-EUC-H books. Windows Song/Hei/Kai +families are retained when installed, and reload retains the fallback. This is display-only; +export, printing, OCR, automation and machines without a candidate retain their limitations. +The M2 readiness and remaining-work estimates below are unchanged. + "Plan size" is the original single-agent sizing. "Done" is the estimated share of that milestone's *acceptance criteria* that are met, not lines of code. "Remaining" is re-measured on 2026-10-10 at the measured rate (see [target-app-parity.md](target-app-parity.md#effort-and-calibration)); the @@ -149,7 +155,7 @@ total matches the parity estimate. |---|---|---|---|---|---| | M0 | Skeleton: workspace, xtask gates, CI | 15–30 | 98% | 1–2 | GitHub workflow, `deny.toml` (licence audit of every dependency), parity checklist (826 features, `xtask parity`) done. Missing: remaining crate stubs, testkit/oracle crates | | M1 | COS: filters, crypt, parser, xref, writer | 120–200 | 80% | 20–40 | Done:
- filters and crypt: every standard-security revision R2–R6 (RC4, AES-128/256), SASLprep, permissions, creating encryption;
- cos: parse and repair (including invalid xref object-number ranges), decoded-stream limits including unfiltered data, decrypt on load, re-encrypt on save, incremental and full writing.
Corpus: open/edit/save passes on 958 files, and all 7 password-protected files open.
Full saves now pack objects into compressed object streams. An unchanged incremental save of a classic table writes a zero-length cross-reference subsection instead of a section with no subsections (#1339). A full rewrite keeps unknown trailer entries, including objects reachable only from them (#1338). Fuzzing runs nightly (`xtask fuzz`). Raw-deflate fallback preserves decoding-limit errors, including through tolerant COS stream decoding. Done since: #2658: macOS atomic saves retain destination extended attributes, including Finder tags, and reject partial resource-fork reads after size changes. Metadata failures retain the original PDF and unsaved edit. Missing: ≥ 250 tests, own image codecs | -| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). | +| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Forms without Resources use page Properties for ActualText and layer visibility; own dictionaries stop fallback. Oversized missing inline-image samples, including header and terminator overhead, no longer pin the content window (#3107 / PR #3438); normal large streamed images retain deterministic sample boundaries. Radial luminosity fades have synthetic coverage for the existing #2132 mask guard, including per-pixel references, antialiased clips, previews, scales and tiles; no additional production fix was needed. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. Missing spacing glyphs in non-embedded Japanese CID substitutes render blank without changing PDF advances or Unicode (#3601). The desktop viewer can use installed CJK fonts, including Windows GBK Song/Hei/Kai faces, without changing export or print font policy (#234, continuation of #1567). | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. The renderer keeps its one-point crop-size floor in physical points after UserUnit scaling, with regression coverage for crop origins, rotations and f32 reciprocal rounding. Existing unit policy and nearly-zero fallback are retained. | | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings | | M3 | Viewer app (native + web) | 80–150 | 90% | 10–20 | Acrobat-style shell, find, select, panels, tiles, web build, UI control channel for agents (opt-in). #3808: the Pages inspector offers a 50–150% thumbnail slider and buttons, also available through `ui.set`. Missing: 60 fps test on a 500-page document, snapshot tests of every panel
Done since: fit visible, document title in the window; Linux middle-button scrolling; shared three-state theme preferences in the toolbar, View menu and control channel; numeric Linux portal events and idle wakeups completing #881, with native appearance inheritance retained (#446); late browser startup downloads preserve the current document and Save target (#171). Selected-text Copy in Edit and the canvas context menu is covered by focused regressions and native clipboard checks (#1225); document copy restrictions apply to the floating action too. #1364 candidate: absolute local GoToR links ask before target access; headless callers explicitly opt in. Native-opening candidate: chosen tools survive preparation and password retries; startup intake waits for pending preparation. Page-label navigation displays logical labels, retains the current page on unchanged submission and refreshes focused same-document replacements; clean file-watcher reload preserves the chosen scrolling mode, preventing a Single Page reader from jumping to page 1 during replacement. Synthetic control regressions cover these behaviors. #3149: page-list changes discard render demand based on old geometry; actual raster dimensions are checked against the GPU limit before upload. Synthetic crash and continued-rendering regressions cover the replacement path. Save-prompt shortcuts yield to dialogs above them; stacked link-consent UI regression covers Escape and Return. Comment hover follows individual marked lines and original annotation paint order (#3541/#3620), independently of spatial inspection sorting; synthetic rotation, differing-upper-edge overlap and unchanged-document regressions. Optional page gaps join page edges without cropping PDF margins; current-document toggle, persisted default and UI control options are covered by viewer tests. Gap changes in Organize Pages preserve document scroll position, and both comparison panes follow menu and control choices. | diff --git a/parity/acrobat-features.toml b/parity/acrobat-features.toml index 45dda39822..9afb691b3c 100644 --- a/parity/acrobat-features.toml +++ b/parity/acrobat-features.toml @@ -782,8 +782,8 @@ milestone = "M2" status = "partial" commands = [] tools = [] -evidence = ["crates/render/src/raster.rs::huge_cid_width_ranges_terminate", "crates/render/src/raster.rs::non_embedded_japanese_cid_fonts_draw_with_a_craft_fonts_face", "crates/render/src/raster.rs::mincho_substitute_covers_half_width_katakana", "crates/render/src/text.rs::identity_to_unicode_names_give_text", "crates/fonts/src/pdf.rs::identity_to_unicode_names_read_codes_as_unicode"] -notes = "CID-to-Unicode tables are M2.4; a non-embedded Adobe-Japan1 font is drawn in a craft-fonts Japanese face (Mincho or Gothic by name; BIZ UDMincho for Mincho, which covers half-width katakana) when built with craft-fonts; non-embedded GB1, CNS1 and Korea1 fonts still draw nothing (no allowed face); a `/ToUnicode` given as the name `/Identity-H` or `/Identity-V` (WSCAD) reads codes as Unicode for selection, search and editing, without control characters" +evidence = ["crates/render/src/raster.rs::huge_cid_width_ranges_terminate", "crates/render/src/raster.rs::non_embedded_japanese_cid_fonts_draw_with_a_craft_fonts_face", "crates/render/src/raster.rs::mincho_substitute_covers_half_width_katakana", "crates/render/src/raster.rs::non_embedded_chinese_and_korean_cid_fonts_draw_with_an_installed_face", "crates/engine/src/tests.rs::installed_cjk_fonts_draw_in_the_viewer_but_not_in_an_export", "crates/render/src/text.rs::identity_to_unicode_names_give_text", "crates/fonts/src/pdf.rs::identity_to_unicode_names_read_codes_as_unicode"] +notes = "CID-to-Unicode tables are M2.4; a non-embedded Adobe-Japan1 font is drawn in a craft-fonts Japanese face (Mincho or Gothic by name; BIZ UDMincho for Mincho, which covers half-width katakana) when built with craft-fonts; the desktop viewer can fall back to an installed CJK face for GB1, CNS1, Korea1 and Japan1, including GBK-EUC-H TrueType CID fonts without ToUnicode. Windows preserves installed SimSun, SimHei and KaiTi families. System faces are display-only, disabled by PDFCRAFT_SYSTEM_FONTS=0, and not used by export, printing, OCR or automation; machines without a candidate and web builds remain unsupported. A `/ToUnicode` given as the name `/Identity-H` or `/Identity-V` (WSCAD) reads codes as Unicode for selection, search and editing, without control characters" [[feature]] id = "view.render-image-codecs"