Merge PR #3348: M2: preserve word spaces around TOC leaders (#3086)

ROADMAP.md: kept both progress-log entries, newest first.

ROADMAP.md

@@ -174,6 +174,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
 
 - **2026-10-20 (M6, calculation order page):** Prepare a form ▸ Calculation order, a page of its own, lists the calculated fields in the order they calculate with their formulas and values, moved with Up/Down arrows (as Acrobat's Set Field Calculation Order), Alt+↑/↓ or by dragging. A card says what is wrong with the order and holds Fix order; a loop has a card of its own that opens one of its formulas. On the page, each calculated field shows its step, and arrows show what the field in focus reads and what reads it. Estimates unchanged.
 - **2026-10-20 (M6, calculation order):** The order a form's fields calculate in (`/CO`) is read, checked, fixed and set headlessly with `form_calculation_order`: it says which fields read one calculated later (they show stale values), which read each other in a loop, and which the PDF's order leaves out; Fix order moves only what must move, in one undo step, and the form recalculates. Editing a formula no longer moves its field to the end of the order, and a field the order lists twice calculates once. Estimates unchanged.
+- **2026-10-20 (M2, PR #3348 review):** Tracking samples retain short repeated punctuation (up to four glyphs), keeping tracked ellipses intact and searchable while longer TOC leaders still preserve title word spaces. Synthetic regressions cover isolated and surrounded ellipses. Estimates unchanged.
 - **2026-10-19 (M6, renames keep formulas):** Renaming a field renames it in every calculation that reads it, in the same undo step: `AFSimple_Calculate` field lists, simplified field notation (the comment and the JavaScript after it) and `getField` calls naming it in one string literal; the rest of each script is kept as written. A sum used to drop the renamed field, notation read it as 0 and a script kept a stale value. Field Properties says how many formulas changed and `form_set_props` lists them as `formulas_updated`. Names built at run time and other events' scripts are not rewritten. Estimates unchanged.
 
 - **2026-10-19 (M14, #3373):** The main menu and its long submenus scroll in short windows, keeping their commands reachable. Estimates unchanged.
@@ -185,6 +186,7 @@ Ranked; detail and the full milestone table in [docs/roadmap.md](docs/roadmap.md
 - **2026-10-18 (M5, PR #1971 review):** Nested FDF annotation values that identify a source `/Page` or `/Pages` dictionary become null at every copy depth. A synthetic three-page PDF saved and reopened before export verifies that destination actions cannot pull source page content into the FDF, while unrelated nested metadata and shared references remain intact. FDF import semantics, copied-graph limits and estimates are unchanged.
 - **2026-10-15 (M5, PR #1971 refresh):** Nested FDF annotation values retain shared or cyclic copied graphs. Discovery borrows direct values and records annotation positions with lazy array owners; retained entries are budgeted before cloning. Synthetic stress fixtures include repeated aliases and saved-PDF cache eviction. Limits cover the copied graph; source parsing and page discovery retain their existing limits, with no total source-memory bound. FDF import semantics and estimates are unchanged.
 - **2026-10-13 (M5, nested FDF values):** Comment export copies and renumbers nested indirect values, retaining geometry, colours, borders and shared graphs. Graph/data limits return an error before the destination is written; synthetic saved-PDF and automation exchange regressions. Estimates unchanged.
+- **2026-10-18 (M2, #3086):** Text extraction estimates letter tracking without repeated punctuation gaps so dotted TOC leaders do not swallow word spaces. Synthetic PDF regressions cover leader lengths, phrase search and selection; strongly tracked text stays intact. Estimates unchanged.
 
 - **2026-10-18 (M6, #2892):** Push-button normal labels are editable per widget through Options and form_set_props, independently of field names; rendering, undo/redo and incremental/full save/reopen are covered. Estimates unchanged.
 

crates/render/src/text.rs

@@ -1636,8 +1636,25 @@ fn arrange(glyphs: &[TextGlyph]) -> (Vec<usize>, Vec<u32>, Vec<bool>) {
             place_glyphs(&glyphs, &height, &segs[si], &mut sc);
             let h = segs[si].h;
             // Word gaps relative to this line's typical letter gap (handles tracking).
+            // Long runs of repeated punctuation belong to leaders, not letter tracking: a TOC leader
+            // must not raise the threshold above the word gaps in its title (#3086).
             sc.gaps.clear();
-            sc.gaps.extend(sc.placed.windows(2).map(|w| glyphs[w[1]].rect[0] - glyphs[w[0]].rect[2]));
+            sc.gaps.extend(sc.placed.windows(2).enumerate().filter_map(|(w, pair)| {
+                let &[p, c] = pair else { return None };
+                let (p, c) = (glyphs.get(p)?, glyphs.get(c)?);
+                if p.text == c.text && p.text.chars().all(|ch| !ch.is_alphanumeric() && !ch.is_whitespace()) {
+                    // Keep short tracked punctuation, including an ellipsis with a final period.
+                    // Treat five or more matching glyphs as a leader. Three neighbours on each
+                    // side suffice to classify this pair, bounding work even on long runs.
+                    let same = |&&i: &&usize| glyphs.get(i).is_some_and(|g| g.text == p.text);
+                    let before = sc.placed.get(..w).unwrap_or_default().iter().rev().take(3).take_while(same).count();
+                    let after = sc.placed.get(w.saturating_add(2)..).unwrap_or_default().iter().take(3).take_while(same).count();
+                    if before + after >= 3 {
+                        return None;
+                    }
+                }
+                Some(c.rect[0] - p.rect[2])
+            }));
             sc.gaps.sort_by(f32::total_cmp);
             let typical = sc.gaps.get(sc.gaps.len() / 3).copied().unwrap_or(0.0).max(0.0);
             let threshold = (typical + h * WORD_GAP).max(h * WORD_GAP);
@@ -2409,6 +2426,92 @@ mod tests {
         assert_eq!(layout(v).plain_text(), "CHAPTER FOUR");
     }
 
+    #[test]
+    fn toc_leaders_do_not_hide_word_gaps() {
+        use lopdf::{Document, Object, Stream, dictionary};
+        for dots in [0, 5, 10, 20, 40, 60] {
+            for (words, expected) in [
+                ("(Was)-333(ist)-333(OTRS?)", "Was ist OTRS?"),
+                ("(Das)-333(OTRS)-333(Web-Interface)", "Das OTRS Web-Interface"),
+                ("(Neuen)-333(Zugang)-333(beantragen)", "Neuen Zugang beantragen"),
+            ] {
+                // The issue's LaTeX TJ spacing, without embedding the corpus PDF or a font asset.
+                let mut doc = Document::with_version("1.7");
+                let font = doc.add_object(dictionary! { "Type" => "Font", "Subtype" => "Type1", "BaseFont" => "Times-Roman" });
+                let operators = format!("BT /F1 10 Tf 10 100 Td [{words}-441{}-1663(7)] TJ ET", "-500(.)".repeat(dots));
+                let content = doc.add_object(Stream::new(dictionary! {}, operators.into_bytes()));
+                let pages = doc.new_object_id();
+                let page = doc.add_object(dictionary! {
+                    "Type" => "Page", "Parent" => pages,
+                    "MediaBox" => vec![0.into(), 0.into(), 800.into(), Object::Integer(200)],
+                    "Resources" => dictionary! { "Font" => dictionary! { "F1" => font } }, "Contents" => content
+                });
+                doc.objects.insert(pages, dictionary! { "Type" => "Pages", "Kids" => vec![Object::Reference(page)], "Count" => 1 }.into());
+                let catalog = doc.add_object(dictionary! { "Type" => "Catalog", "Pages" => pages });
+                doc.trailer.set("Root", catalog);
+                let mut bytes = Vec::new();
+                doc.save_to(&mut bytes).unwrap();
+                let pdf = Pdf::new(std::sync::Arc::new(bytes)).unwrap();
+                let text = extract_page(&pdf, 0, &InterpreterSettings::default()).unwrap();
+                assert_eq!(text.plain_text(), format!("{expected} {}7", ". ".repeat(dots)), "{dots} leader dots");
+                let hits = text.find_opts(expected, true, true);
+                assert_eq!(hits.len(), 1, "{expected:?}, {dots} leader dots");
+                assert_eq!(text.text_of(hits[0].clone()), expected);
+                assert_eq!(text.line_rects(hits[0].clone()).len(), 1);
+                assert_eq!(text.find(&expected.to_lowercase()), hits);
+                assert!(text.find(&expected.replace(' ', "")).is_empty());
+            }
+        }
+    }
+
+    #[test]
+    fn short_tracked_ellipses_remain_searchable() {
+        for dots in [3, 4] {
+            let mut v = Vec::new();
+            for i in 0..dots {
+                let x = 10.0 + i as f32 * 4.0;
+                v.push(g(".", x, 10.0, x + 2.0));
+            }
+            let ellipsis = ".".repeat(dots);
+            let text = layout(v.clone());
+            assert_eq!(text.plain_text(), ellipsis);
+            assert_eq!(text.find(&ellipsis), vec![0..dots]);
+            v.insert(0, g("A", 0.0, 10.0, 6.0));
+            let x = 12.0 + dots as f32 * 4.0;
+            v.push(g("B", x, 10.0, x + 6.0));
+            let text = layout(v);
+            let expected = format!("A {ellipsis} B");
+            assert_eq!(text.plain_text(), expected);
+            assert_eq!(text.find(&expected), vec![0..dots + 2]);
+        }
+    }
+
+    #[test]
+    fn leaders_preserve_tracking_in_letters_numbers_and_punctuation() {
+        for advance in [9.0, 12.0] {
+            for leader in [".", "-", "…"] {
+                let mut v = Vec::new();
+                word(&mut v, "CHAPTER-1", 10.0, 10.0, advance);
+                let x = 10.0 + 9.0 * advance + 5.0;
+                word(&mut v, "1.4", x, 10.0, advance);
+                let mut x = x + 3.0 * advance + 5.0;
+                for _ in 0..60 {
+                    v.push(g(leader, x, 10.0, x + 2.5));
+                    x += 11.0;
+                }
+                let text = layout(v);
+                assert_eq!(text.plain_text(), format!("CHAPTER-1 1.4 {}", vec![leader; 60].join(" ")));
+                assert_eq!(text.find("chapter-1 1.4"), vec![0..12]);
+            }
+        }
+        // Decimal punctuation and repeated letters/digits still use their original tracking estimate.
+        for value in ["1.4", "1111", "llll"] {
+            let mut v = Vec::new();
+            word(&mut v, value, 10.0, 10.0, 12.0);
+            assert_eq!(layout(v).plain_text(), value);
+        }
+    }
+
     #[test]
     fn rtl_runs_come_out_in_logical_order_and_cjk_has_no_spaces() {
         let mut v = Vec::new();

docs/roadmap.md

@@ -1,6 +1,10 @@
 # PdfCraft roadmap: milestones and what's next
 
+> **Last reviewed:** 2026-10-20 · **Last updated:** 2026-10-20 · **Change:** patch (PR #3348 review: retain short tracked punctuation in gap estimates; estimates unchanged)
+
 > **Last reviewed:** 2026-10-18 · **Last updated:** 2026-10-18 · **Change:** minor (M5, PR #1971: bounded nested FDF export excludes source page/page-tree dictionaries; refreshed against upstream 4db6ac4a; estimates unchanged)
+> **Last reviewed:** 2026-10-18 · **Last updated:** 2026-10-18 · **Change:** patch (M2 #3086: dotted TOC leaders no longer hide word spaces; estimates unchanged)
+
 > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M14/#2383 GPU-limit candidate; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
 > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M7: the Edit text caret is as tall as the row it is on; estimates unchanged) · **Target:** Adobe Acrobat Pro (Acrobat DC, continuous track 26.002.21931, macOS)
 > **Last reviewed:** 2026-10-15 · **Last updated:** 2026-10-15 · **Change:** minor (M6: CSV tab-containing headers; estimates unchanged)
@@ -113,7 +117,7 @@ total matches the parity estimate.
 |---|---|---|---|---|---|
 | M0 | Skeleton: workspace, xtask gates, CI | 15–30 | 98% | 1–2 | GitHub workflow, `deny.toml` (licence audit of every dependency), parity checklist (826 features, `xtask parity`) done. Missing: remaining crate stubs, testkit/oracle crates |
 | M1 | COS: filters, crypt, parser, xref, writer | 120–200 | 80% | 20–40 | Done:<br>- filters and crypt: every standard-security revision R2–R6 (RC4, AES-128/256), SASLprep, permissions, creating encryption;<br>- cos: parse and repair (including invalid xref object-number ranges), decoded-stream limits including unfiltered data, decrypt on load, re-encrypt on save, incremental and full writing.<br>Corpus: open/edit/save passes on 958 files, and all 7 password-protected files open.<br>Full saves now pack objects into compressed object streams. An unchanged incremental save of a classic table writes a zero-length cross-reference subsection instead of a section with no subsections (#1339). A full rewrite keeps unknown trailer entries, including objects reachable only from them (#1338). Fuzzing runs nightly (`xtask fuzz`). Raw-deflate fallback preserves decoding-limit errors, including through tolerant COS stream decoding. Missing: ≥ 250 tests, own image codecs |
-| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. |
+| M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. Page marks, added content and print output retain small page-unit transform coefficients when serialized. Dotted TOC leaders no longer inflate letter tracking or hide word spaces (#3086); phrase search and selection are covered. Short tracked ellipses retain their gap samples and remain searchable. |
 | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings. Packed TIFF-predictor RGB rows retain later rows in decoding and rendering. The renderer keeps its one-point crop-size floor in physical points after UserUnit scaling, with regression coverage for crop origins, rotations and f32 reciprocal rounding. Existing unit policy and nearly-zero fallback are retained. |
 | M2 | Model, render, text | 200–350 | 15% | 130–230 | hayro bootstrap renderer (vendored patches); inspector predictor scratch is bounded by available input, and JPX sample rescaling handles 1–32 bits safely. Text extraction reaches word-F1 0.98 against pdftotext. Short same-line redraws split before placement rather than interleaving. Missing: model crate, fonts, DisplayList, renderer independent of hayro. Inspector page-label traversal and output are resource-bounded, with physical-number fallback warnings |
 | M3 | Viewer app (native + web) | 80–150 | 90% | 10–20 | Acrobat-style shell, find, select, panels, tiles, web build, UI control channel for agents (opt-in). Missing: 60 fps test on a 500-page document, snapshot tests of every panel<br>Done since: fit visible, document title in the window; Linux middle-button scrolling; shared three-state theme preferences in the toolbar, View menu and control channel; numeric Linux portal events and idle wakeups completing #881, with native appearance inheritance retained (#446); late browser startup downloads preserve the current document and Save target (#171). Selected-text Copy in Edit and the canvas context menu is covered by focused regressions and native clipboard checks (#1225); document copy restrictions apply to the floating action too. #1364 candidate: absolute local GoToR links ask before target access; headless callers explicitly opt in. Native-opening candidate: chosen tools survive preparation and password retries; startup intake waits for pending preparation. Page-label navigation displays logical labels, retains the current page on unchanged submission and refreshes focused same-document replacements; clean file-watcher reload preserves the chosen scrolling mode, preventing a Single Page reader from jumping to page 1 during replacement. Synthetic control regressions cover these behaviors. #3149: page-list changes discard render demand based on old geometry; actual raster dimensions are checked against the GPU limit before upload. Synthetic crash and continued-rendering regressions cover the replacement path. Save-prompt shortcuts yield to dialogs above them; stacked link-consent UI regression covers Escape and Return. |

parity/acrobat-features.toml

@@ -1394,8 +1394,8 @@ milestone = "M3"
 status = "shipped"
 commands = ["edit.find"]
 tools = ["text_find"]
-evidence = ["crates/ui-egui/tests/ui.rs::find_counts_matches_across_pages", "crates/ui-egui/tests/ui.rs::find_reports_no_matches", "crates/automation/tests/automation.rs::open_inspect_render_and_find", "crates/render/src/text.rs::identity_v_columns_are_searchable_in_reading_order"]
-notes = "Horizontal and WMode 1 vertical text are grouped independently, including a horizontal heading above right-to-left vertical columns. Complex interleaved flows still use a geometric reading-order heuristic."
+evidence = ["crates/ui-egui/tests/ui.rs::find_counts_matches_across_pages", "crates/ui-egui/tests/ui.rs::find_reports_no_matches", "crates/automation/tests/automation.rs::open_inspect_render_and_find", "crates/render/src/text.rs::identity_v_columns_are_searchable_in_reading_order", "crates/render/src/text.rs::toc_leaders_do_not_hide_word_gaps", "crates/render/src/text.rs::short_tracked_ellipses_remain_searchable"]
+notes = "Horizontal and WMode 1 vertical text are grouped independently, including a horizontal heading above right-to-left vertical columns. Dotted TOC leaders preserve word spaces for phrase search (#3086), while short tracked ellipses remain intact and searchable; sparse lines and condensed OCR gaps below the existing height-based threshold remain limitations. Complex interleaved flows still use a geometric reading-order heuristic."
 
 [[feature]]
 id = "view.find-next-previous"