mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-10-27 03:27:21 -06:00
A page that draws a discretionary hyphen through WinAnsiEncoding's 0xAD extracts, in our map, as U+00AD SOFT HYPHEN. That character is invisible, so text the reader can plainly see does not match a search for it, and `redact`'s pattern matching -- which runs on exactly this map -- does not match it either. The same silent-failure shape as reading Symbol as Latin: the text is there, the tool cannot see it. Annex D.2 gives codes 0x2D and 0xAD the same glyph name, so the byte a Windows codepage calls a soft hyphen draws, and should extract as, an ordinary hyphen. Changed in the reading table only: writing is unaffected, because win_ansi_byte maps 0xA0..0xFF by identity, so '-' still takes the canonical 0x2D and a caller holding a literal U+00AD still writes 0xAD. The round-trip test could no longer hold for that code -- two codes now mean one character -- so it states the aliasing instead of asserting a bijection, and a new test pins the behaviour and the reason. **This needs the owner's reading of the spec before merging.** The evidence is not unanimous: a PDF library's documentation cites Annex D.2 for the glyph name `hyphen` at 0xAD, and Poppler extracts U+002D, but the vendored hayro maps 0xAD to the glyph name `sfthyphen` and reports U+00AD, normalising to `hyphen` only when looking the glyph up in the font program. Two implementations disagree and I cannot read the normative table from here. Note this does not change `pdfcraft-cli text`, which maps through hayro, not this table; the user-visible extraction path carries the same question separately. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>