#2305 fixed WinAnsi 0xAD in `crates/fonts`, and closed with two loose ends it named: the
user-visible extraction path goes through hayro, not that table, so it still reported the
invisible character; and the same Annex D.2 note makes 0xA0 the `space` glyph. This finishes
both, and makes the two tables in the product agree about the same byte.
WinAnsiEncoding gives codes 0xA0 and 0xAD the glyph names `space` and `hyphen` -- the names it
also gives 0x20 and 0x2D -- and MacRomanEncoding gives 0xCA `space`. The tables called them
`nbspace` and `sfthyphen`, whose Adobe Glyph List values U+00A0 and U+00AD are invisible
formatting characters, so extraction hid ink the reader can see: "well-known" drawn with the
0xAD byte matched no search and no redaction pattern, and a non-breaking space did not match the
space a reader typed.
Evidence. Two sources transcribe the table the same way (an Annex D implementation's docs, and
the WinAnsi.enc encoding vector R ships, which gives `/space` at octal 240 and `/hyphen` at
255), and a black-box check settles the rest: on a synthetic fixture pdftotext reads the
base-encoded codes as `a-b a b`, and keeps U+00AD for a code an explicit `/Differences` names
`sfthyphen`. We now do both. Over the 989-file pdf.js corpus the soft hyphens we extract fall
from 117 to 13 and the non-breaking spaces from 30 to 26; everything left comes from `/ToUnicode`
maps, which say what they mean and are not touched.
Scope. The base table only, in the text direction only.
- A `/Differences` entry naming `sfthyphen` or `nbspace` is a deliberate statement about one
code and keeps its glyph list value, as pdftotext does.
- Glyph lookup is untouched, so a font program's own `sfthyphen` program is still found first.
In hayro that is why the alias sits in the Unicode path rather than in the generated encoding
table: `code_to_ps_name` is shared with lookup.
- Writing is unaffected: `encode` takes the lowest code showing a character, so ' ' and '-'
still write 0x20 and 0x2D, and a caller holding a literal U+00A0 still writes 0xA0.
Tests. `fonts` pins both tables and the whole decode/encode path including the `/Differences`
case; `render` pins the extraction a reader actually gets, over WinAnsi, MacRoman and
`/Differences`. Each was watched failing with each cell and each match arm reverted in turn, so
no assertion is passing by accident. fonts 70, render 296+4, redact 49, a11y 43 and edit 156 are
green, with fmt, clippy, `xtask layers` (37 crates) and `xtask assets` clean.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* M2: Add word and character counts with optional content exclusions
Retain the complete reviewed change, including prior integration resolutions,
on current upstream.
M2: Add document word counts and semantic exclusions
* Refresh catalog hashes after merging main
---------
Co-authored-by: LabDennis <341377186+LabDennis@users.noreply.github.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* fix(text): read a TrueType font without an encoding as WinAnsi (fixes#1468)
* fix(text): describe the WinAnsi reading by pdftotext's output
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Flattening deleted a cloudy comment it could not draw, and a refused text-box restyle or resize left the new properties in the file. Arrow defaults were stored on plain lines. A custom match ending on a separator included the next glyph, and the vertical-text redaction error claimed an overlap it had not checked.
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M3: use the macOS Finder label in the tab-menu regression
The UI already uses Reveal in Finder on macOS, but the regression test
queried Reveal in File Explorer on every platform. Select the platform's
label while preserving the disabled-action and inactive-tab assertions.
Co-Authored-By: Codex <noreply@openai.com>
* M3: match Greek final sigma in case-insensitive search
Fold capital, normal and final sigma to the same character so Greek words
match in either case. Cover exact search, whole words, glyph ranges and
bounded matches without changing the existing Turkish-I behavior.
Record the search fix and the macOS test portability follow-up in the
roadmap. Workspace tests, all-target Clippy, formatting, wasm, layering
and a synthetic CLI before/after comparison pass on macOS.
Co-Authored-By: Codex <noreply@openai.com>
---------
Co-authored-by: nevicia2010-hub <259836477+nevicia2010-hub@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
* Text extraction: honour /ActualText, so Thai from Chrome PDFs is searchable
Chrome (Skia) writes /ActualText around each Thai, Indic or Arabic cluster
whose shaped glyphs (a tone mark moved clear of a vowel, a ligature) have no
Unicode of their own. Text extraction ignored it, so those marks were lost:
"น้ำหนัก" came out as "นาหนัก", "เป็น" as "เปน", and searching for any
such word found nothing.
- hayro-interpret patch (25): Device::actual_text hands the device a
marked content sequence's /ActualText, inline or named in /Properties.
- TextDevice turns the glyphs of a replacement into one glyph carrying the
replacement text, boxed around them. The outermost replacement wins, an
empty one stands for no text, and one spanning several lines or writing
directions keeps per-glyph text so selection and lines still work.
- Canva draws text twice (outline or shadow) and marks only one drawing:
the unmarked redraw at the same places takes the same replacement, so the
existing redraw detection still drops the copy.
Checked on 1,114 real-world PDFs against the previous build, with pdftotext
as the reference: 957 extract byte-identical text, 157 change and all of
them read closer to the reference (exact matches 3 -> 122), none got worse,
none newly fail, and total extraction time is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Text extraction: bound the work of the ActualText redraw dedupe
`TextDevice::finish()` compared every replacement with every unmatched glyph
in up to 4096 grid cells, so a hostile page stacking ~10^5 glyphs and ~10^5
replacements at one spot meant ~10^10 comparisons (a hang). It now charges
each visited cell and each candidate run against a per-page budget
(`MAX_REDRAW_WORK`, 2^22) and stops deduplicating once it is spent; redraws
found so far are still merged. A regression test feeds 100,000 replacements and
100,000 non-matching glyphs at one place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: git-monchiant <4667243+git-monchiant@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M2: overlapping text runs stay whole, laid out in arrange
Two text runs that overprint each other on one baseline were merged into one segment and sorted by x, so their letters came out interleaved ("ADleplhtaa ebpestialo gna zmemtaa"). Runs are now ranges of the filtered glyph list that arrange() already works on. Glyphs of runs that collide (overlap by more than a quarter of the line height) stay whole and are placed by their left edges; runs that only interleave along the line keep glyph-by-glyph order. Glyph placement happens when a segment is emitted, into scratch storage that every segment reuses.
layout() is unchanged: it still calls arrange() and takes the original glyphs by the returned indices. Turned text needs no separate path, because arrange_flow() already lays it out in its own baseline frame.
Tests: overprinted runs at 0 and 37 degrees, a run drawn over a line, a combining mark drawn last, and runs that only touch stay as they were or come out whole.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: a run that redraws an earlier run is printed once
Doubled, shadowed or fake-bold text draws a whole line again at (almost) the same place. The per-glyph fake-bold filter only looks four glyphs back, so a redraw of a whole run got through and printed twice.
After runs are identified, a run is dropped when an earlier retained run has the same glyph texts over the same places (each glyph within a fifth of its height). Runs are keyed by a fingerprint of their glyph texts, with each text hashed with its length so glyph boundaries count, and by a grid cell. Every earlier candidate with the same key is checked, so a run that does not match cannot hide a later redraw. Dropping a run compacts the segment list only; glyphs are never removed from the input, so the returned indices are still mapped through the fake-bold filter's `kept`.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: merge zero-width marks into a segment in one pass
Each zero-width mark was inserted with a search from the start of the placed glyphs and a Vec::insert, so a segment of m marks and n glyphs cost O(m·n) comparisons and shifts. Marks are visited in x order, and the position each one goes to never moves back, so a single cursor over the placed glyphs puts them all in. The output is unchanged, including when a colliding component leaves the placed order non-monotonic in x, because the cursor stops at the same first glyph whose x is greater than the mark's.
A 16000-mark segment takes 225 ms, the same as main (608 ms with the per-mark insert).
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: a dense overprint joins its runs instead of sweeping every pair
The collision sweep compared each glyph with every later glyph that started before its right edge. When many glyphs start at one x, as in a heavily overprinted line, that is quadratic in the glyphs of the segment.
The sweep now has a budget of COLLIDE_PAIRS_PER_GLYPH (16) pair tests per glyph of the segment. Ordinary text overlaps each glyph with about one neighbour, so it never reaches the budget and its output is unchanged. A segment that needs more is dense: its text runs are joined into one component and placed run by run, which keeps every run whole. The fallback is one pass over the runs, so the work for a page is O(glyphs) pair tests plus the sorts.
On a page of 2000 overprinting runs (12000 glyphs) the sweep took 157 ms, against 53 ms on main; with the budget it takes 65 ms. The output is identical to the sweep's, since every run in these pages overprints every other.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: look up forced word breaks by binary search
Each glyph of a segment was checked against every forced word break with breaks.contains(), so a segment with many overlapping rows cost O(glyphs x breaks). The breaks are sorted once, after collision placement, and looked up by binary search.
The lookup returns the same answers, so the output is unchanged. The new test has 60 overlapping rows, each of which starts a word.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: find each word gap's glyph by a position table, not a scan
For every word gap, the glyph that takes the space was located by scanning the segment's output (idx.iter().position), so a long line with many gaps cost O(glyphs x gaps). This was already on main; the change makes it linear.
Each glyph's place in its segment's output is now written into a page-sized table as the segment is emitted, and the gap reads it. The table is never cleared: a segment writes every entry it reads before reading it, and the glyphs of a segment are a permutation of the same set it wrote. The space goes to the same glyph as before, so the output is unchanged.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: check a redraw against at most 32 earlier runs
A run is checked against every earlier run with the same fingerprint and grid cell. Runs that share a key but do not redraw each other are all kept in that chain, so a page of N such runs costs O(N²) checks. The chain is new in this change, not in main.
Each run now checks at most REDRAW_CANDIDATES (32) candidates, newest first. A run past the limit is kept, as it would be if no earlier run matched. Runs that redraw one of the newest candidates are still dropped, and a test checks both sides of the limit: a redraw of the newest of 40 look-alike runs is found, and a redraw of the oldest is kept.
On 4096 non-redrawing runs (69 632 glyphs, all kept by both versions) the check cost 200 ms, and the bound brings the page back to 286 ms, against 272 ms on main. Output is identical.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The bookmark filter, the Comments panel filter and the command palette
lowercased with str::to_lowercase, which follows no locale: "İSTANBUL"
became "i̇stanbul" (a combining dot added) and "ı" has no case pair, so
"istanbul" never found the bookmark "İSTANBUL ŞUBESİ" and "imzalayın" never
found a comment "Lütfen İMZALAYIN".
They now fold with pdfcraft_render::text::fold_case, the folding Find and
Search & Redact use (made public here): I, i, İ and ı count as one letter
and every other character folds to its own lowercase.
A line's glyphs are sorted left to right, but the space for a word gap was put before whichever of the two glyphs came later in the content stream. When the right-hand run was drawn first, the space landed inside the left run: 20.0 19.4 came out as "20. 019.4". The same rule put the space one letter early between right-to-left words drawn left to right.
Gaps are now marked by position on the line and reversed together with right-to-left runs.
As in Acrobat, holding Alt (Option on macOS) when a drag starts selects only
the text inside the rectangle drawn, instead of everything between two
points in reading order. It works with the Select tool, the text markup
tools and Redact, so a column can be copied, highlighted or redacted.
- render: PageText::glyphs_in (glyph centre inside the rectangle),
glyph_rects (one box per run, never covering unselected text) and
column_text: rows top to bottom, side-by-side pieces separated by a tab,
right to left in right-to-left rows, reading order for vertical text.
- ui-egui: a selection is a reading-order run or a column rectangle; the
rectangle is outlined while dragging and the cursor is a crosshair.
⇧-click clears a column selection (it has no anchor glyph).
- Edit ▸ Column select (edit.column_select, quick tool "column-select")
does it with a plain drag. It is the way in on Linux desktops whose
window manager takes Alt-drag for moving windows (Xfce, Cinnamon, MATE,
KDE Plasma 5 by default), where the app never sees the drag.
- automation: text_extract takes rect for the same extraction headless.
- "Column select" translated in all 15 catalogs (Arabic included); their ATTRIBUTION.toml
hashes updated.
- parity: new view.column-select entry with platform notes and gaps
(no selection across pages, no ⇧-click extension).
Windows: winit hands an Alt release to the system menu loop only when the
window has a native menu, which PdfCraft's does not, so Alt-drag cannot
leave the window in menu mode.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Case-insensitive search folded the query and the page with
str::to_lowercase, which follows no locale: U+0130 (İ) becomes "i" plus a
combining dot above (two characters) and U+0131 (ı) has no case pair. So
"istanbul" never found "İSTANBUL", and "ırmak"/"kış" never found
"IRMAK"/"KIŞ" (or the reverse). The same PageText::find serves Find,
Search & Redact (phrases and word lists) and the automation text search,
so Turkish words could also be left unredacted.
Fold I, i, İ and ı to "i" and every other character with its own
lowercase, one character for one character. Case-sensitive search is
unchanged.
The phrase search collapsed runs of whitespace in the query but not in the page text it was matching against, so a document drawn with two space glyphs in a row like the issue example never matched the phrase. Single words still matched, which is why it looked like only some files were affected.
Collapse whitespace while the text is flattened, the same way the query is normalised. The regression test fails on the old code because the phrase find returns an empty range, and it also covers tabs and single word queries.
Fixes#814.
Co-authored-by: Big Pickle <noreply@opencode.ai>
* M2: text turned at any angle reads as lines along its own baseline
The text layer sorted every glyph into the nearest of four quarter turns
and laid it out by its axis-aligned box. Text at 30-60 degrees lost its
word gaps, split into one line per word in reverse order, or interleaved
two neighbouring lines letter by letter; text set along a curve came out
in fragments.
Each glyph now keeps its drawn em box as a quad and its writing
direction. Glyphs within the layout's word-gap tolerance (sin 0.15) of a
quarter turn take the existing path unchanged. Other glyphs are laid out
in a frame along their own baseline, one flow per direction; runs that
continue each other while turning (stamps, seals) are unrolled into one
line. Turned text keeps the place in the reading order that its first
glyph had before.
Horizontal, quarter-turn, skewed and near-axis text is bit-identical
(text, rect, line_of, space_before) over pdf_engine_test.pdf, the
crates' test PDFs and 300 synthetic pages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M2: look up a turned glyph's direction among directions only, not every curved run
Curved runs can number about half the glyphs, which made grouping quadratic on hostile pages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
A glyph whose ToUnicode is several characters ("ffi", Arabic lam-alef, a
letter with its marks) was split into one glyph per character, with the
glyph box cut into equal parts along view x. Turned text and vertical
columns put the parts across the line, so "office" at 90° read
"o ce\nf\nf\ni"; narrow ligatures put two "f"s close enough to be dropped
as fake bold ("of ice"); right-to-left runs reversed the characters inside
the ligature ("سالم" for "سلام").
The glyph now keeps its text and its own box, as PDFium reports every
character of a ligature with the whole glyph box.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Text: find the word and the line around a glyph
PageText gets word_at(i) and line_at(i). Each returns the first and
last glyph of the word or the line around glyph i, or None when i is
past the end of the page's text. A word stops at a word gap, a blank
glyph or a line end. A line is the run of glyphs with the same line
number.
The canvas's double-click used its own loop for this, which indexed the
glyph, line and gap arrays directly. It now calls word_at, which reads
them with get(). Double-clicking selects the same words as before.
Test: render words_and_lines_around_a_glyph, on two lines with a blank
glyph, a word gap and a line break.
* Issue #295: three clicks select the line, four clicks the page
With the Select tool, quick clicks on text widen the selection like
Acrobat: two clicks select a word (as before), three the line, four or
more all the text on the page. A third click used to clear the
selection.
egui counts only up to triple clicks and reports a fourth quick click
as another triple, so the view keeps its own count of the current run.
A single click resets it and a pause ends it.
With the markup tools (Highlight and the like) and Redact, a
double-click already turns the word into a mark at once. They stop at
the word, so more clicks don't stack line or page marks.
Tests: ui-egui quick_clicks_select_a_word_then_the_line_then_the_page
and quick_clicks_with_the_highlighter_mark_only_the_word.
parity: view.text-select cites the tests. The notes say what is
missing: paragraph selection, and Ctrl/⌘+A across the whole document
(it selects the current page).
* Clippy: as_chunks in the signature preview test (follow-up to #355)
Clippy 1.99 rejects chunks_exact with a constant chunk size
(chunks_exact_to_as_chunks), so `cargo clippy --workspace --all-targets
-- -D warnings` fails on main since #355. Use as_chunks, as #360 did
for the test above it.
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The app, crates, binaries, bundle id, packaging, icons, screenshots and docs are now
PdfCraft / pdfcraft (repo storytold/pdfcraft). On first launch the desktop app moves the
PrintCraft settings and crash-recovery folders to the new name, and both apps read settings
saved under the old storage key, so upgrades keep recent files, preferences and unsaved work.
The signing test fixtures keep their PrintCraft subject names (they are binary).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The text reading order uses `let … else`, `while let` and `get_mut` instead of `expect`, and
named destinations bind their key with an or-pattern instead of `unreachable!`. The crate now
denies the panic lints.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
