* chore: resolve SonarCloud maintainability issues
* fix: restore compiling refactors, revert behaviour changes, run cargo fmt
- fontdb::norm: keep the closure for filter (char::is_alphanumeric takes
self, not &self, so it does not fit Iterator::filter).
- story.rs: write_mark_rpr needs &mut self (rev_attrs, rpr_change).
- Revert the viewport meta change and the return 0 added to
workspace_version/sha256, which changed behaviour or masked failures.
- cargo fmt.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Import Document in odt page_parts after dropping glob re-export
* fix: apply_kind_properties binds ContentControl; cell_ref keeps char closures
* fix(engine): pass case helpers by value (clippy needless borrows)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan Taylor <dylan@craftmaker.dev>
Each of these languages accepted every word in capitals of up to six letters and every word
with a digit on its own, so turning off Ignore words in UPPERCASE or Ignore words that contain
numbers had no effect on them for short words. The check now takes the options, as the Spanish
one does; the direct is_correct/is_correct_with keep the defaults.
Co-authored-by: Federico Olivares <FedeOlivares@users.noreply.github.com>
Co-authored-by: aiongg <67543387+aiongg@users.noreply.github.com>
crates/proof/src/lang.rs: main's English-dialect helpers kept; code() doc lists `de`.
crates/proof/src/lib.rs: hostile-input test loops over main's languages plus De.
crates/ui-egui/src/options.rs: language_name maps EnGb/EnCa and De to no dictionary name.
ROADMAP.md: main's status lines; both progress-log and revision rows kept.
docs/gaps.md: main's status line; gap 8 row is main's with German hyphenation added; revision rows unioned.
docs/layout-parity.md: main's status line, Tabs and Drop caps rows; the PR's Hyphenation row; revision rows unioned.
docs/localization-parity.md: main's status line; revision rows unioned.
* #1281: English dialects in proofing — en-GB, en-CA, en-AU… flag each other's spellings (#1045)
English was one proofing language: en-GB, en-CA and en-AU all proofed like
US English, and US documents accepted British spellings. Now ProofLang has
three Englishes, picked by the tag's region:
- En (en-US, other regions, no language): British spellings flagged
(colour → color).
- EnGb (en-GB, en-AU, en-NZ, en-IE, en-ZA, en-IN…): American spellings
flagged (color → colour, center → centre, analyze → analyse); -ize and
-ise both right (Oxford spelling).
- EnCa (en-CA): British -our/-re/-ence, American -ize, tire, program.
Data: assets/spelling/en-variants.txt, our own ~280 American/British
pairs with a policy per English (MIT OR Apache-2.0, ATTRIBUTION.md row).
Inflections by rule (crates/proof/src/en_variants.rs). A dialect's form
is right when the shared list knows it or its other spelling, so
"realisation" is right in en-GB. No system dictionary needed.
Suggestions put the dialect's own form first. Review › Language lists
English (Australia) and (Canada); the Language dialog adds India,
Ireland, New Zealand and South Africa (names in every catalog, not
native-reviewed). An English interface on an en-GB/en-CA/en-AU… region
writes new documents in that English. Options › Language shows the
document's language instead of a fixed "English (United States)".
Tests: per-dialect accept/flag/suggest, tag mapping, no-panic proptests,
engine end to end through review.language, DOCX w:lang round trip,
region-based editing language, language menu.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P8veLveuUsJJm6UEcQPEuW
* #1281: review — pin the #40 words (judgement, defence, licence, programme, catalogue) in en-GB and en-US
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012WEhZXSxzS8orX7f9J3qpr
* #1281: review — harness picks every proofing language in Review › Language and checks it is ticked
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012WEhZXSxzS8orX7f9J3qpr
* #1281: review — pin more British spellings in en-GB (goitre, sceptre, enrol, fulfil, instalment, skilful, wilful, encyclopaedia…)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012WEhZXSxzS8orX7f9J3qpr
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: aiongg <aiongtaigi@gmail.com>
Review follow-up: the directives that name one flag read the same emoji
form, so they now skip the variation selector like PFX/SFX and stem flags.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RLA-ES, the Spanish Hunspell dictionaries in LibreOffice and most Linux
distributions, writes one flag as U+260E followed by U+FE0F, in the rules
(PFX ☎️ Y 3) and on stems (banco/S☎️). parse_aff rejected it as "not one
character", so the whole dictionary failed to load. The variation selector
is now dropped wherever flags are read, with a regression test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeRabbit (#3252): reload_dictionary() wrote DICT and DICT_IN_MEMORY
under two separate locks. If install_dictionary() ran between them,
the final state could hold an in-memory dictionary with the flag left
false, so installed_path() would report a same-named disk file
instead of None.
Combined both into one DictState struct behind a single RwLock in
both ru and cs, so install_dictionary() and reload_dictionary() each
update the dictionary and its origin in one critical section.
Didn't add a dedicated concurrency test: genuinely proving a race's
absence needs real multi-threaded stress testing, and the previous
commit on this PR already found that tests directly exercising
install_dictionary()/reload_dictionary() concurrency in this crate's
shared global state cause real, reproducible flakiness in unrelated
tests. The fix is correct by construction (one lock acquisition around
both field writes) and all 222 existing proof tests, run 15 times in
a row, stay green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CodeRabbit (#3252): installed_path() searched the filesystem
unconditionally, so a dictionary given directly with install_dictionary
(a user-picked one, or a test fixture) was reported as the file of a
same-named dictionary that happened to also be installed, even though
spelling was actually using the in-memory one, not that file.
Added a DICT_IN_MEMORY flag to both modules, set in install_dictionary
and cleared in reload_dictionary; installed_path() now returns None
immediately when it's set, before even searching the filesystem.
Tried adding a dedicated test that calls install_dictionary() and
asserts installed_path() is None. It made dictionaries_folder_is_
searched_first flaky (~15-35% of runs across repeated local testing):
ru's and cs's own test modules already call install_dictionary()
directly in several tests of their own, sharing the same process-wide
static, and dictionaries_folder_is_searched_first's exact-path
assertion for Russian/Czech could observe whichever one last touched
it. Reverted the dedicated test; the production fix alone never
touches that shared state (it's a pure read-only check) and caused no
flakiness by itself across 25 repeated runs. Instead, narrowed
dictionaries_folder_is_searched_first's Russian/Czech assertion to the
language only, not the exact path, since unlike the other four
languages, installed_path() for these two is now sensitive to
concurrently-running tests' legitimate in-memory installs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CodeRabbit (#3252): set_dictionaries_folder() reloaded every other
proofed language but not Russian or Czech, so after a folder change
installed_dictionaries() would report a path from the new folder
while spelling kept using the stale cached dictionary.
Also added Czech's test fixture to the shared dictionaries-folder
test's temp folder (the same content czech_proofing_through_the_
review_commands installs directly): since that folder-change test
now reloads Czech too, it was clearing a concurrently running Czech
test's installed dictionary with nothing to replace it, intermittently
emptying it mid-test. Same fixture content either way keeps the race
harmless, exactly as it already does for Polish/Portuguese/Indonesian.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
installed_dictionaries() never included Russian or Czech at all,
because neither crates/proof/src/ru nor crates/proof/src/cs had an
installed_path() function for it to call (every other proofed
Hunspell language — Polish, Portuguese, Italian, Croatian, Turkish,
Indonesian — already did). File > Options > Proofing's dictionaries
list iterates installed_dictionaries() directly, so Russian and Czech
never showed up there at all, installed or not, matching the report.
Added installed_path() to both modules, mirroring the existing
pattern (crate::hunspell::find(NAMES).map(|(_, dic)| dic)), and added
both to installed_dictionaries().
The actual spell-checking dispatch (is_correct_in_with) already
routed Russian and Czech correctly — confirmed by installing the real
system ru_RU Hunspell dictionary and checking it directly
(ru::is_correct("молоко") true, ru::is_correct("малоко") false), so
that part of the pipeline isn't what's broken. The reporter's other
symptom (Russian text in the document not being checked at all) did
not reproduce with the real dictionary installed; I could not pin
that part down further without more specific repro details from
their setup.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Proofing: say when a dictionary is installed but unreadable, with file and reason (#2830)
* proof test: use SHIFT_JIS as the unsupported encoding (ISO 8859-1 is decoded since #2904)
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* #1741: WIP
* #1741: list items no longer get the sentence-capital grammar squiggle
The English "Capitalize the first word of a sentence" rule skips the first
word of a list paragraph (own or style numbering); later sentences are still
checked. check_text_with and paragraph_issues take a list_item flag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* #1741: review fixes
Thread the capital-first flag into the Polish, Croatian, Italian, Turkish and
Portuguese grammar rules; test the drawn squiggles through layout() for own,
style and num 0 numbering; tidy the proof test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* #1741: main's Vietnamese proofing test passes the new list-item argument
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* #1741: the third call in main's Vietnamese proofing test
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Dmitrii Khlapov <19991457+dkhlapov@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* #1694: -ically adverbs are no longer flagged as misspelled
English spelling accepts "-ally" adverbs built on an "-ic" adjective in the
word list (specific → specifically, scientific → scientifically), as Word does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* #1694: only -ic adjectives take -ally
Review (FredApps): the "-ally" → stem rule applied to every known word,
so "desktopally" was accepted. The rule is now "-ically" → "-ic"
(specifically → specific); other words don't take "-ally".
Test: non_ic_words_do_not_take_ally (desktop, house, final, green).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P8veLveuUsJJm6UEcQPEuW
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Review follow-ups: the German patterns' MIT notice goes into NOTICE with
the other pattern sets; the pattern reader skips other commands'
arguments, so hyph-de-1996.tex's \message{} line no longer adds junk
patterns and exception words; a load test for the German set; the
status bar names a bare "de" tag; the docs list the languages with
their own hyphenation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runs in German (de, de-DE, de-AT, de-CH) hyphenated with the English
patterns, giving breaks such as "unterlieg-ende" and "rechtli-cher".
They now use hyph-utf8's German patterns for the reformed spelling
(MIT, bundled unmodified as assets/hyphenation/de.pat). German gets a
ProofLang of its own that, like Bulgarian, hyphenates only: no spelling
or grammar marks yet (#1564). Portuguese, Italian and German share one
pattern lookup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Indonesian spelling reads the installed Hunspell id_ID dictionary at runtime
(Debian/Ubuntu hunspell-id, LibreOffice dict-id, the Dictionaries folder or
WORDCRAFT_DICTIONARIES), like Czech, Polish and Portuguese: nothing is bundled,
and without a dictionary Indonesian text gets no spelling marks.
- ProofLang::Id for id, id-ID, ind and the legacy in tag
- suggestions reuse the dictionary-driven Czech ranking (MAP/REP/TRY/KEY of
the loaded dictionary), nothing Indonesian hard-coded
- hyphenated reduplication accepted whole or by parts (kupu-kupu, anak-anak);
hyphenated clitics after capitals/abbreviations (KTP-nya, se-Indonesia)
- generic grammar only: repeated word (offers anak-anak first), space before
punctuation, double space, with Indonesian messages
- Review > Language, status bar, Set Proofing Language, File > Options >
Proofing list Indonesian; name translated in every catalog
- tests use a hand-written synthetic dictionary; engine end-to-end test
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Same pattern as Portuguese (#797) and Polish (#346): the cs_CZ
dictionary is read from the system at runtime (e.g. Debian/Ubuntu
hunspell-cs), nothing is bundled. Suggestions use the dictionary's own
MAP/REP/TRY/KEY; grammar stays generic (repeated word, spacing), with
digit ratios like "3 : 1" exempt. The Czech interface now writes new
documents (and opened .txt/.md) in cs-CZ.
Engine fixes the Czech dictionary needs, also used by pl/pt:
- PFX/SFX header lines may carry a trailing comment (cs_CZ has
"SFX y Y 80 #…"); rule lines are left literal;
- NOSUGGEST stems are accepted when typed but never offered;
- KEY is parsed for neighbour-key suggestions.
Tests use a hand-written synthetic Czech dictionary only; an
end-to-end engine test mirrors the Portuguese one.
Co-authored-by: Losminek <301412321+Losminek@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
A document typed in Vietnamese under the default editing language (en-US) was
covered in red underlines, every Vietnamese word checked as English. A
paragraph that reads as Vietnamese (words with Vietnamese-only letters: the
breve and horn letters, đ, ĩ ũ, the tone-marked vowels of Latin Extended
Additional, or stacked combining marks) is now treated like any language
without proofing tools: no spelling or grammar marks, as Word's automatic
language detection does. A Vietnamese name in English text (Nguyễn) is no
longer flagged either; French and Portuguese, which share â ê ô, are unaffected.
Automatic hyphenation also leaves Vietnamese paragraphs and words whole
(single syllables never take a hyphen); the English patterns happen not to
split these words today, and the guard keeps it so.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Italian proofing: po', qual è, È and un/un' grammar rules; more elisions in spelling
- grammar (works without a dictionary): un pò / un po -> un po'; qual'è -> qual è;
E' -> È; un'altro -> un altro and un altra -> un'altra (also nessun, ciascun,
qualcun, alcun), gender from the ending with masculine -ista/-ma/-cida... left alone;
more emphatic doubles allowed (pian piano, poco poco, così così...)
- spelling: more elisions (dov'è, cos'è, anch'io, senz'altro, nessun'altra, n'ha...),
truncations po'/mo'/be' accepted, pò and qual'è left to grammar (no double marks)
- tests: rule tests, grammar proptests, Italian suggestion proptest, It in mixed-span proptest
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Contributors: credit lupant by their GitHub profile name
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Proofing: status bar flags a language whose dictionary isn't installed (#1167)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* vi: missing-dictionary status strings
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Russian proofing: spelling, suggestions, grammar and word division
Runs tagged ru get Russian tools instead of none:
- spelling against an installed ru_RU Hunspell dictionary (not bundled;
LibreOffice dict-ru folders searched; KOI8-R and Windows-1251 decoded),
with ё optional, hyphenated words and solid compounds understood, не with
adjectives, participles and verbal nouns, prefixed verbs, abbreviations
with endings and camel case left alone, plus our supplement of modern
words with their declensions (assets/spelling/ru-extra.txt);
- suggestions tuned for Russian: е/ё, letter groups, common confusions,
missing spaces, typos and a stem scan;
- grammar rules with Russian messages (repeated words, spaces, sentence
capitals with Russian abbreviations, likely missing commas before
clause openers, doubled punctuation);
- Russian word division from the spelling rules, written as code.
* Options: Russian in the dictionaries list (ProofLang::Ru arm)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Магда М.Г. <mail@mgmagda.ru>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
File › Options › Proofing gets a Dictionaries folder: a folder of Hunspell
.aff/.dic files searched before the system locations, saved with the settings
and applied at start-up. An app started from Finder or the Dock doesn't see
WORDCRAFT_DICTIONARIES, so this is how it finds a dictionary kept in a folder
of the user's own. Changing it reloads the Polish and Portuguese dictionaries
and checks the text again. The pane lists the dictionary file each language
found, or that none was found, so missing spelling marks are explained.
Agents set it with file.options {"dictionariesFolder": path | "auto"}; a
missing folder or a relative path is refused and changes nothing.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* proof: Turkish spelling, suggestions, grammar and hyphenation
* Turkish: dictionary tests skip when no Turkish Hunspell dictionary is installed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The bundled Moby list already knows many British forms (judgement,
defence, programme) but still flagged others: honour and its inflections,
colourful, colourless, rigour, glamour, prioritise, privatise,
synthesise, paediatric and a few more.
Adds those base forms to assets/spelling/en-extra.txt (inflections come
from the existing rules) and a regression test. A separate en-GB
dictionary chosen from w:lang is still open on the issue.
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* Italian proofing: it-IT spelling, suggestions and grammar (#951)
* Italian proofing: Review Language menu and editing language
* Italian proofing: interface test for editing language
* Italian proofing: find the Swiss it_CH dictionary too
* Italian proofing: TeX hyphenation patterns (MIT option)
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Runs in Bulgarian (`w:lang` bg, bg-BG) are hyphenated by the Bulgarian syllable rules
(crates/proof/src/bg.rs, clean-room from the spelling rules; no pattern file or word list):
vowel-vowel splits, one consonant goes with the next syllable, a consonant cluster splits after
its first consonant except that an obstruent before р/л starts the next syllable (до-бре,
та-бли-ца), й stays with its vowel, дж/дз and ьо are never split.
- ProofLang::Bg (bg, bul): hyphenation only. It isn't "proofed", so like Other there are no
spelling or grammar marks until there is a Bulgarian dictionary.
- Latin-script words in a Bulgarian run are hyphenated as English, as in any other language.
- The layout already takes each word's language from its run (ProofRun), so nothing changes there.
Tests: the rules (срич-ко-пре-на-ся-не, та-бли-ца, шо-фьор, хо-джа, строй-ка), tag resolution,
hyphenate_word_in with the paragraph limits (Бъл-га-рия), and a laid-out bg-BG paragraph whose
hyphenated lines all break at Bulgarian points.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M15: Polish interface, and Polish spelling, grammar and hyphenation
Interface
- Polish is the tenth interface language ("Polski", pl.tsv, 1,045 entries);
pl, pl-PL and pl_PL.UTF-8 system locales select it automatically.
- Right-click menu labels now go through translation in every language.
- Review > Language is a menu (English US/UK, Polski, Do not check spelling
or grammar); the status bar shows the language at the caret.
- With a Polish interface, new blank documents and opened .txt/.md files
default to pl-PL.
Proofing
- Each run is checked in its own language (w:lang, resolved through
styles): squiggles, F7, review.issues, suggestions and hyphenation.
Text in languages without proofing tools is no longer flagged as English.
- Polish spelling from the SJP.PL Hunspell dictionary (Apache-2.0 option),
read by a new Rust affix engine (crates/proof/src/affix.rs); 875 KB asset,
~4.5M word forms. Regenerate with the pl_dictionary example.
- Polish suggestions: case, missing diacritics, rz/ż, ó/u, ch/h confusions,
missing spaces (napewno -> na pewno), edit distance.
- Conservative Polish grammar rules: repeated words, spacing, doubled
punctuation, sentence capitals, missing comma before że/który/ponieważ/
aby/żeby/jeśli…, we/ze forms.
- Polish hyphenation with the hyph-utf8 hyph-pl.tex patterns (MIT option);
pattern keys widened to 20 letters, TeX exceptions parsed; the heuristic
no longer treats ł, ż, ś… as vowels.
review.language validates and normalizes tags, can set noProof alone and
the document default; format.state reports lang and noProof.
Tests for every part, docs and ATTRIBUTION.md/NOTICE updated;
cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: rustfmt the test module order left unformatted by #391
cargo fmt --check failed on main after #391 added mod tests_grid.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #346: read Polish spelling from the installed Hunspell dictionary
- Polish spelling uses the Hunspell pl_PL dictionary installed on the
system (OS packages, Homebrew, LibreOffice, ~/.local/share/hunspell, or
WORDCRAFT_DICTIONARIES) instead of a bundled 870 KB SJP.PL file;
without one, Polish text gets no spelling marks, like Word without the
language's proofing tools. .aff/.dic are size-capped and decoded from
UTF-8 or ISO 8859-2 (crates/proof/src/hunspell.rs).
- The dictionary can be replaced at runtime (install_dictionary) or
re-searched (reload_dictionary); an empty one accepts every word.
- Tests use small synthetic dictionaries written for them, so they don't
depend on what the machine has installed.
- The bundled pl.dic, its ATTRIBUTION/NOTICE entries and the generator
example are removed; docs describe the installed dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* #100: affix — accept FLAG UTF-8 and DEFAULT flag modes
Per-character flag modes are what the reader already parses; only the
multi-character encodings (long, num) stay errors. VERO pt_BR declares
FLAG UTF-8.
* #100: affix — widen flag masks to u128 (128 flag capacity)
VERO pt_BR needs 106 distinct flags; the u64 bitmask capped at 64.
Serialization is unaffected: WCAFF1 stores flag characters and
re-derives the masks on load.
* #100: affix — support FORBIDDENWORD
VERO pt_BR marks spellings it must reject with a forbidden flag (ý).
Stems carrying it are listed but never correct and never inflect.
* #100: affix — continuation classes in check
SFX A 0 able/B: the form derived by A carries flag B, licensing a
further suffix of class B (workable -> workables) while the bare stem
cannot take one. form_has_flag walks the chain depth-bounded
(MAX_CHAIN = 3); with no continuations declared it degenerates to the
stem lookup, so existing dictionaries are unaffected. VERO pt_BR uses
26 continuation rules.
* #100: affix — continuation classes in forms
Enumerate chained suffix forms (workable -> workables) under the same
MAX_CHAIN bound the checker walks, so every form the checker accepts
is listed and vice versa.
* #100: affix — round-trip continuations and FORBIDDENWORD through WCAFF1
The compact format keeps its WCAFF1 magic: continuation classes ride
on the rule's appended field (as in the .aff source) and the forbidden
flag becomes a FORBIDDENWORD line. Old files read as before.
* #100: affix — strip a UTF-8 BOM from .aff/.dic input
VERO's files open with a BOM; glued to the count line it parsed as a
stem (one garbage entry per load) and to SET it hid the directive.
* #782: affix — the module doc still rejected what the code now reads
Continuation classes and FORBIDDENWORD are supported; the header said
Rejected for both.
* #100: pt — Portuguese spelling, following the Polish module
ProofLang::Pt resolves pt/pt-BR/pt-PT/por tags; the pt module reads the
installed Hunspell pt_BR dictionary (VERO) through the affix engine —
FLAG UTF-8 flags, continuation classes and FORBIDDENWORD spellings all
exercised by the synthetic test dictionary. Hyphenated compounds are
checked part by part (guarda-roupa, ex-marido); without a dictionary
installed, Portuguese text gets no marks. Spelling message in
Portuguese; suggestions and grammar land next.
* #100: pt — Portuguese suggestions
Rounds of rising cost, driven by the dictionary's own tables: missing
accents from MAP (coraçao -> coração, saude -> saúde), the REP
confusions (seje -> seja, excessão -> exceção), a missing space after a
short function word (nãoé -> não é, dagua -> da água), one typo over
TRY, and an accent-folding stem scan — the Polish rounds, retuned for
Portuguese.
* #100: pt — Portuguese grammar flags
Conservative rules with Portuguese messages: repeated words (except
the emphatic doubles Portuguese allows), space before punctuation and
double spaces, sentence capitals knowing the Portuguese abbreviations
and dialogue dashes, and doubled punctuation. No comma rule: que opens
a clause only sometimes, so a wrong flag would cost more than a missed
one.
* #100: pt — Portuguese hyphenation
hyph-utf8's hyph-pt.tex bundled unmodified (BSD 3-clause, ATTRIBUTION
row with source and hash): 426 Liang patterns and 2 exception words;
falls back to the generic heuristic outside their reach.
* #100: pt — Review › Language offers Português (Brasil), and a pt-br interface writes pt-BR
The menu's fourth entry sets the run language; the status bar names
Portuguese; editing_lang is now allowed for pt-br (wordcraft-proof
proofs it), so new documents under a Portuguese interface are
Portuguese by default.
* #100: pt — engine test: Portuguese proofing through the review commands
review.language pt-BR normalizes the tag, review.issues reports
Portuguese spelling and grammar with the lang field, F7 and Change
apply a suggestion, Add to Dictionary works for Portuguese.
* #100: pt — Portuguese in the mixed-language proofing tests and docs
check_text proofs a Portuguese span with the Portuguese message; the
never-panic loops and the module docs cover the third language.
* #100: docs — Portuguese proofing in the gap, parity and roadmap docs
Gap 8 narrowed (English, Polish and Portuguese), localization parity
gains a Portuguese row and a revision entry, target-app-parity's
Proofing row moves 40% -> 42%, and the ROADMAP progress log records
what landed.
* #100: hunspell — LibreOffice extension folders follow the language
The search paths hardcoded LibreOffice's dict-pl; a pt_BR user whose
only dictionary is LibreOffice's dict-pt-BR extension — on Windows the
only realistic on-disk source — got silently no spelling. Both the
primary-language folder and the dashed full name are now tried, so
Polish keeps dict-pl and Portuguese finds dict-pt-BR.
* #100: pt — review fixes
Suggestion ties break toward the change nearest the word's start
(saúde before saudê, proven against a fixture that has both); the
Language menu's label is the English source "Portuguese (Brazil)" with
translations in all twelve catalogs (as Polish is); the attribution
counts pt.pat's 2 exception words; the pt fixture drops a duplicate
stem; docs/roadmap.md catches up with the landing; the throwaway spike
is gone.
* Review fixes for #782: two-suffix chain limit, module doc
- MAX_CHAIN = 2 counts suffixes in a row, matching Hunspell's two-suffix limit
(the old bound of 3 allowed up to four); test pins that a third is rejected.
- Module doc: say probes allocate small scratch strings instead of claiming none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #797: NOTICE for pt.pat, no indexing in the LibreOffice folder helper
- NOTICE reproduces hyph-pt.tex's BSD 3-Clause copyright notice, conditions and
disclaimer, which the licence requires for binary redistributions.
- hunspell::lo_ext_dirs builds its list without indexing (never-crash rule).
- Catalogs: main already translates "Portuguese (Brazil)" in every language, so the
duplicate rows the branch added are dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: witkowski01 <konradwitkowski01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M15: Polish interface, and Polish spelling, grammar and hyphenation
Interface
- Polish is the tenth interface language ("Polski", pl.tsv, 1,045 entries);
pl, pl-PL and pl_PL.UTF-8 system locales select it automatically.
- Right-click menu labels now go through translation in every language.
- Review > Language is a menu (English US/UK, Polski, Do not check spelling
or grammar); the status bar shows the language at the caret.
- With a Polish interface, new blank documents and opened .txt/.md files
default to pl-PL.
Proofing
- Each run is checked in its own language (w:lang, resolved through
styles): squiggles, F7, review.issues, suggestions and hyphenation.
Text in languages without proofing tools is no longer flagged as English.
- Polish spelling from the SJP.PL Hunspell dictionary (Apache-2.0 option),
read by a new Rust affix engine (crates/proof/src/affix.rs); 875 KB asset,
~4.5M word forms. Regenerate with the pl_dictionary example.
- Polish suggestions: case, missing diacritics, rz/ż, ó/u, ch/h confusions,
missing spaces (napewno -> na pewno), edit distance.
- Conservative Polish grammar rules: repeated words, spacing, doubled
punctuation, sentence capitals, missing comma before że/który/ponieważ/
aby/żeby/jeśli…, we/ze forms.
- Polish hyphenation with the hyph-utf8 hyph-pl.tex patterns (MIT option);
pattern keys widened to 20 letters, TeX exceptions parsed; the heuristic
no longer treats ł, ż, ś… as vowels.
review.language validates and normalizes tags, can set noProof alone and
the document default; format.state reports lang and noProof.
Tests for every part, docs and ATTRIBUTION.md/NOTICE updated;
cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: rustfmt the test module order left unformatted by #391
cargo fmt --check failed on main after #391 added mod tests_grid.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #346: read Polish spelling from the installed Hunspell dictionary
- Polish spelling uses the Hunspell pl_PL dictionary installed on the
system (OS packages, Homebrew, LibreOffice, ~/.local/share/hunspell, or
WORDCRAFT_DICTIONARIES) instead of a bundled 870 KB SJP.PL file;
without one, Polish text gets no spelling marks, like Word without the
language's proofing tools. .aff/.dic are size-capped and decoded from
UTF-8 or ISO 8859-2 (crates/proof/src/hunspell.rs).
- The dictionary can be replaced at runtime (install_dictionary) or
re-searched (reload_dictionary); an empty one accepts every word.
- Tests use small synthetic dictionaries written for them, so they don't
depend on what the machine has installed.
- The bundled pl.dic, its ATTRIBUTION/NOTICE entries and the generator
example are removed; docs describe the installed dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* #100: affix — accept FLAG UTF-8 and DEFAULT flag modes
Per-character flag modes are what the reader already parses; only the
multi-character encodings (long, num) stay errors. VERO pt_BR declares
FLAG UTF-8.
* #100: affix — widen flag masks to u128 (128 flag capacity)
VERO pt_BR needs 106 distinct flags; the u64 bitmask capped at 64.
Serialization is unaffected: WCAFF1 stores flag characters and
re-derives the masks on load.
* #100: affix — support FORBIDDENWORD
VERO pt_BR marks spellings it must reject with a forbidden flag (ý).
Stems carrying it are listed but never correct and never inflect.
* #100: affix — continuation classes in check
SFX A 0 able/B: the form derived by A carries flag B, licensing a
further suffix of class B (workable -> workables) while the bare stem
cannot take one. form_has_flag walks the chain depth-bounded
(MAX_CHAIN = 3); with no continuations declared it degenerates to the
stem lookup, so existing dictionaries are unaffected. VERO pt_BR uses
26 continuation rules.
* #100: affix — continuation classes in forms
Enumerate chained suffix forms (workable -> workables) under the same
MAX_CHAIN bound the checker walks, so every form the checker accepts
is listed and vice versa.
* #100: affix — round-trip continuations and FORBIDDENWORD through WCAFF1
The compact format keeps its WCAFF1 magic: continuation classes ride
on the rule's appended field (as in the .aff source) and the forbidden
flag becomes a FORBIDDENWORD line. Old files read as before.
* #100: affix — strip a UTF-8 BOM from .aff/.dic input
VERO's files open with a BOM; glued to the count line it parsed as a
stem (one garbage entry per load) and to SET it hid the directive.
* #782: affix — the module doc still rejected what the code now reads
Continuation classes and FORBIDDENWORD are supported; the header said
Rejected for both.
* Review fixes for #782: two-suffix chain limit, module doc
- MAX_CHAIN = 2 counts suffixes in a row, matching Hunspell's two-suffix limit
(the old bound of 3 allowed up to four); test pins that a third is rejected.
- Module doc: say probes allocate small scratch strings instead of claiming none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: witkowski01 <konradwitkowski01@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Persist the custom dictionary across sessions (#40)
* Review fixes for #796: session-only Ignore All, desktop-only persistence, capped atomic I/O
- review.ignoreAll no longer adds to (and saves) the custom dictionary: ignored words go to a
separate session-only list, as Ignore All does in Word.
- Only the desktop app wires the dictionary file. Headless MCP and `wordcraft-cli run` keep
words in memory, so an agent never reads or changes the person's dictionary (and nothing
writes outside an automation write root, #769).
- Loading reads through `take(1 MB)`, so a huge file is never read whole; lines that aren't a
single word (spaces, control characters, invalid UTF-8, over 64 characters) are skipped.
- review.addToDictionary applies the same word rule (proof::custom_word), so a word added is
never dropped on the next load.
- Saving merges the words on disk first (another process's additions survive; words just
removed stay out), creates the folder, and replaces the file atomically (temp + rename).
The file I/O lives in the proof crate, so the engine doesn't touch std::fs itself.
- Tests for each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The five tests that assert linear layout/export/hyphenation by comparing
the cost of an n-sized and a larger input (layout: unequal formats, hidden
separators with auto hyphenation, hidden float anchors; pdf: fragmented
hidden headings; proof: hyphenating a long word) timed both with wall
clock, fastest of three. On a loaded machine (CI, shared dev boxes, the
test harness running other tests in parallel) a preempted big run, timer
resolution on the small run, or a tight 4x/8.0 limit (hyphenation measured
~7.2 when linear) made them fail now and then.
Each now uses a shared cost_ratio helper (test-only, no production change):
- CPU time of the test thread from /proc/thread-self/schedstat on Linux, so
time spent waiting for a CPU is left out (wall clock elsewhere);
- the small input runs `scale` times per timing so both timings are long
against the clock's resolution;
- small and big runs alternate, five rounds after a warm-up, cheapest kept;
- 8x the input everywhere: linear ~8x, quadratic ~64x, limit 24x.
Measured under load (load avg ~15-35 on 32 cores): 60 linear ratios in
7.7-8.8. With the old quadratic code injected locally each test fails:
unequal formats (uncapped style bucket scan) ~90-104x, hidden separators
(hyphenator rescan per point) ~36x, float anchors (run walk restarted per
anchor) ~188x, PDF headings (left-out range scan per char) ~37x,
hyphenation 50-65x.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* M15: Polish interface, and Polish spelling, grammar and hyphenation
Interface
- Polish is the tenth interface language ("Polski", pl.tsv, 1,045 entries);
pl, pl-PL and pl_PL.UTF-8 system locales select it automatically.
- Right-click menu labels now go through translation in every language.
- Review > Language is a menu (English US/UK, Polski, Do not check spelling
or grammar); the status bar shows the language at the caret.
- With a Polish interface, new blank documents and opened .txt/.md files
default to pl-PL.
Proofing
- Each run is checked in its own language (w:lang, resolved through
styles): squiggles, F7, review.issues, suggestions and hyphenation.
Text in languages without proofing tools is no longer flagged as English.
- Polish spelling from the SJP.PL Hunspell dictionary (Apache-2.0 option),
read by a new Rust affix engine (crates/proof/src/affix.rs); 875 KB asset,
~4.5M word forms. Regenerate with the pl_dictionary example.
- Polish suggestions: case, missing diacritics, rz/ż, ó/u, ch/h confusions,
missing spaces (napewno -> na pewno), edit distance.
- Conservative Polish grammar rules: repeated words, spacing, doubled
punctuation, sentence capitals, missing comma before że/który/ponieważ/
aby/żeby/jeśli…, we/ze forms.
- Polish hyphenation with the hyph-utf8 hyph-pl.tex patterns (MIT option);
pattern keys widened to 20 letters, TeX exceptions parsed; the heuristic
no longer treats ł, ż, ś… as vowels.
review.language validates and normalizes tags, can set noProof alone and
the document default; format.state reports lang and noProof.
Tests for every part, docs and ATTRIBUTION.md/NOTICE updated;
cargo xtask ci passes (fmt, clippy, test, assets, layers, wasm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: rustfmt the test module order left unformatted by #391
cargo fmt --check failed on main after #391 added mod tests_grid.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Review fixes for #346: read Polish spelling from the installed Hunspell dictionary
- Polish spelling uses the Hunspell pl_PL dictionary installed on the
system (OS packages, Homebrew, LibreOffice, ~/.local/share/hunspell, or
WORDCRAFT_DICTIONARIES) instead of a bundled 870 KB SJP.PL file;
without one, Polish text gets no spelling marks, like Word without the
language's proofing tools. .aff/.dic are size-capped and decoded from
UTF-8 or ISO 8859-2 (crates/proof/src/hunspell.rs).
- The dictionary can be replaced at runtime (install_dictionary) or
re-searched (reload_dictionary); an empty one accepts every word.
- Tests use small synthetic dictionaries written for them, so they don't
depend on what the machine has installed.
- The bundled pl.dic, its ATTRIBUTION/NOTICE entries and the generator
example are removed; docs describe the installed dictionary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M15: Polish catalog covers the interface text merged from main
The merge of main (98456ac) brought ~575 new interface strings (charts,
SmartArt, content controls, Developer tab, AutoCorrect, outline view,
protection, compare, print, accessibility, references and Language/
Hyphenation dialogs, Draw additions). pl.tsv lacked them, so
polish_covers_every_key_of_every_other_catalog and
every_tab_and_command_is_translated failed. All are now translated.
cargo xtask ci: all 6 steps pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
File › Options is now a pane list (General, Display, Proofing, Save,
Language, Accessibility, Advanced, Agents), one function per pane in
crates/ui-egui/src/options.rs. Every setting shown is wired:
- General: theme, interface size (#475), mini toolbar on selection, dark page, community
button, user name and initials (comments use the initials).
- Display: tab characters, spaces, paragraph marks and hidden text shown
with Show/Hide ¶ off; rulers; print the page colour; update fields
before printing.
- Proofing: ignore words in capitals / with numbers / web and file
addresses (proof::Options, passed through layout), check spelling and
mark grammar as you type, custom dictionary add/remove (kept in ui.json).
- Save: AutoSave. Language: interface language. Accessibility: Read Aloud
speed and citations. Advanced: typing replaces selection, Insert key
overtype, overtype (text.overtype), recent documents count, measurement
units (dialogs, Layout ribbon indents and rulers).
file.options reads and sets the engine-side settings headlessly and only
opens the page when given no setting. ui.json loads leniently: a bad value
falls back to its default instead of dropping every setting.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The English speller only knew the 1990s Moby word list plus a small
hard-coded EXTRA array, so everyday modern words (desktop, laptop,
download, upload, spreadsheet, database, screenshot, inbox, offline,
workplace…) were flagged. Inflection, doubled-consonant and prefix rules
also only looked stems up in Moby, so even supplemented words failed in
inflected forms (blogging, uploader).
- New supplement assets/spelling/en-extra.txt: ~220 lowercase base forms,
our own list (MIT OR Apache-2.0), replacing the EXTRA array.
- All stem lookups (inflections, doubled consonant, prefixes) consult the
dictionary and the supplement; suggestions include supplement words.
- Regression tests: common modern words and inflections accepted,
misspellings still flagged, "dekstop" suggests "desktop"; the list
stays sorted and lowercase.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Tables: stop repeating a header row that leaves no room for the next row
A repeating header row plus a row that could not fit below it on an empty
page made place_table loop: after each page break the header was placed
again, so the page was never "at top" when the row was retried, split_row
had nothing to split, and the loop ran its 1,000 iterations per row. A
20-row table with a 640 pt header came out as 19,002 pages.
After a page break the header now repeats only when the row (or its first
split) still fits below it; otherwise the row starts at the top of the
page, as Word does when a header row would fill the page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: give runs with different links or decorations their own style
Builder::style memoised StyleRuns by a key that folded strike, double
strike and "has a link" into one boolean and left out the underline
colour and baseline position. The display list reads all of these from
the shared StyleRun, so in a paragraph of like-formatted links every link
pointed at the first one's URL in the PDF (and an external link after an
internal one became a bookmark jump), double strike drew as single strike
after a struck run, and underline colours were copied.
The key is now the whole resolved format, so runs share a StyleRun only
when nothing the display list reads differs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF and No Markup: leave tracked deletions out of the final text
PDF export defaults to no markup, but it laid the document out with every
deleted run in place and only switched off the strike and colour, so the
PDF printed deleted text as ordinary text ("Keep DELETEDTEXT INSERTED"):
neither the original nor the final document, and a leak of text the
author removed. The No Markup view did the same on screen.
A new LayoutOptions::hide_deleted lays deleted runs out like hidden text
(zero-width markers, so offsets, carets and hit testing keep working). PDF
export sets it whenever markup is off, the session layout sets it when
markup is hidden, and the markup toggles relayout as Show/Hide marks does.
The display list also never draws a deletion as plain text when markup is
off, whatever layout it is given.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tables: repeat the header on every page and put the next row under it, as Word does
The earlier fix for the runaway loop dropped the repeated header row on any
page where the header and the next row did not fit together. Word never
does that: measured on Word for Mac, it moves the row to a new page, repeats
the header there and places the row under it even when the row then runs
into the bottom margin (a 300 pt header with 400 pt rows gives one header +
row per page; a 640 pt header with 100 pt rows does the same).
place_table now remembers when the page holds only the repeated header
rows. A row that cannot split is placed there anyway instead of breaking
the page again, which is both Word's behaviour and what keeps the loop from
spinning: every pass now places something. Rows of text taller than the
room under the header still split across pages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF: keep tracked deletions out of bookmarks and heading tags
Without markup the PDF pages show the final text, but bookmark titles and
the titles of heading structure tags were still built from plain_text(),
which includes deleted runs. A heading "Intro SECRET Part" with "SECRET "
deleted printed as "Intro Part" yet bookmarked as "Intro SECRET Part", and
a fully deleted heading still got a bookmark: text the author removed
leaked into the outline and the tag tree.
Paragraph::final_text() gives the text with deletions left out, and the
exporter titles headings with it unless markup is included. A heading
whose text is all deleted has an empty title and so no bookmark.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* No Markup: deleted note references take no number and print no endnote
With deletions hidden, a tracked-deleted footnote or endnote reference
disappears from the text, but note_numbers() still counted it. Deleting
the first of two footnote references left the surviving one numbered 2,
and a deleted endnote reference still printed its endnote after the last
paragraph, because the endnote list is taken from the same map.
note_numbers() now skips references inside tracked deletions when the
layout hides them, so the remaining notes number from 1 and endnotes whose
reference was deleted are not emitted. With markup shown nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* No Markup: a deleted floating object leaves no wrap area behind
With deletions hidden, a tracked-deleted floating picture or shape was
not drawn, but place_para had already added its wrap exclusion from
p.objects without looking at the anchor, so the final text still flowed
around an invisible object (indented beside a square-wrapped shape).
Floating objects whose anchor is a tracked deletion are now skipped when
the layout hides deletions; deleted_objects() is shared with note
numbering, which needs the same test. With markup shown nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* No Markup: a deleted soft hyphen no longer breaks the word with a hyphen
With deletions hidden, a tracked-deleted soft hyphen became a zero-width
marker, but hyphenation_points() reads the paragraph's raw text and
accepted marker clusters, so the line could still break there and draw
"ab-" in a narrow column although the final text has no soft hyphen.
When the layout hides deletions, clusters inside a tracked deletion are
no longer hyphenation points (soft hyphens or dictionary points alike).
With markup shown nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* No Markup: tracked deletions add no line-break opportunities
Break opportunities came from running UAX #14 over the paragraph's raw
text, deleted runs included. With deletions hidden, a deleted soft hyphen
or space still let the line break there: "xx ab<del>SHY</del>cdef" in a
narrow column came out as "xx ab" / "cdef" instead of moving the word
"abcdef" to the next line whole.
When the layout hides deletions, the opportunities are computed over the
text without them and mapped back to offsets in the paragraph. With
markup shown, or with no deletions, the computation is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hidden text: no hyphenation points or line breaks in text that is not shown
Hidden text that is not shown (the default, and always in print and PDF)
is laid out as zero-width markers, but break opportunities came from UAX #14
over the paragraph's raw text and hyphenation_points() accepted marker
clusters, so a hidden space or soft hyphen still let the line break there:
"xx ab<hidden SHY>cdef" in a narrow column came out "xx ab-" / "cdef".
Measured in Word for Mac, the hidden soft hyphen and the hidden space are
no place to break: "xx" / "abcdef".
left_out() is now the one test for text the layout leaves out (hidden text
not shown, tracked deletions without markup). The run loop records the
ranges it leaves out, and both the break opportunities and the hyphenation
points come from that, so deletions behave exactly as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hidden text: a floating object anchored in it leaves no wrap area behind
With hidden text not shown, a floating picture or shape anchored in a
hidden run was not drawn (the run is a zero-width marker, so no object
cluster places it), but place_para had already added its wrap exclusion,
so the text still flowed around an invisible object. Measured in Word for
Mac: a square-wrapped picture anchored in hidden text is not printed and
the text runs the full width.
deleted_objects() becomes left_out_objects(), which resolves each anchor's
formatting through the styles (hidden text often comes from a character
style) and applies the same left_out() test as the paragraph layout. Note
numbering keeps counting hidden reference marks, as Word does, so it asks
only about deletions; its behaviour is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hidden text: print the footnote of a hidden reference mark, as Word does
A footnote whose reference mark is hidden text was numbered but never
printed: the hidden run became zero-width markers without recording the
note, so place_para never put the footnote at the foot of the page.
Measured in Word for Mac with hidden text not printed, a hidden footnote or
endnote reference still takes its number (the next visible footnote is 2)
and both notes are printed. Endnotes already were, and the numbering
already matched.
A hidden reference mark now records its note like a visible one, so the
footnote goes to the foot of the page holding the line it sits on. A
deleted reference in the final text still records nothing, as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF: keep hidden text out of bookmarks and heading tags
The PDF never prints hidden text (its layout does not show it), but the
bookmark and heading-tag titles were built from final_text(), which drops
tracked deletions only. A heading "Intro SECRET Part" with "SECRET " hidden
printed as "Intro Part" yet bookmarked as "Intro SECRET Part", and a heading
that is all hidden text still got a bookmark for a heading nobody can see.
Paragraph::text_where() generalises final_text() to any per-run test, and
the exporter titles headings with the runs that are neither hidden (resolved
through the styles, since a character style can hide text) nor, without
markup, deleted. A heading that is all hidden has an empty title and so no
bookmark. Word's PDF export from AppleScript writes no outline, so there is
no Word answer to match; this follows what the pages show, as for deletions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* PDF: titles leave out what the layout left out, table formatting included
Bookmark and heading-tag titles resolved hidden text from the paragraph
and run formatting alone, but the layout also applies the table style's
conditional formatting (header row, first column, total row) beneath the
runs. A Heading 1 in a table header row whose style hides it printed
nothing yet still got a bookmark, and a partly hidden heading leaked the
hidden part into its tag.
ParaLayout now records the byte ranges it laid out as nothing (left_out),
resolved exactly as laid out, and the exporter titles a heading from the
text outside them (plus tracked deletions without markup, as before), so
the titles cannot disagree with the pages. text_where() gives way to
text_without() over byte ranges; final_text() is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hyphenation: hyphenate words as laid out, without hidden or deleted text
Automatic hyphenation looked up the words of the paragraph's raw text and
only dropped points that fell inside text the layout leaves out. Words and
the minimum-letters limits still came from the raw text: with "accep"
hidden in "acceptable", the visible "table" could break as "t-able" (the
point after "accept"), and a hidden space split "foot<space>ball" into two
words, so the visible "football" never hyphenated.
Soft hyphens and dictionary points are now found in the same text as laid
out that the UAX #14 break opportunities already use, and mapped back to
paragraph offsets. With hidden text shown and markup on, that text is the
paragraph's own and nothing changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout, PDF: skip left-out text in one pass, not a scan per char
Building the text as laid out tested every char against every left-out
range, and Paragraph::text_without did the same for PDF titles. Hidden
text in many runs (formatting that alternates, a common result of imports)
made both quadratic: a 100,000-char hidden heading in one-char runs took
13.9 s to convert with the release CLI, 400,000 chars minutes.
The layout now merges adjacent left-out runs as it records them and walks
the ordered ranges with a cursor; text_without sorts its ranges by start
(they may come in any order and overlap) and walks them the same way. Both
are linear: 0.20 s for that heading, 0.81 s at 400,000 chars, with the same
output. No left-out text still means no filtering at all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Hyphenation: check the limits of each point in constant time
hyphen_points() found each candidate point's letter run by scanning out
from the point and counted the letters on each side by rescanning the run,
and the syllable heuristic rebuilt the rest of the word as a string for
every position. Both are quadratic in the word's length. Ordinary words are
short, but with hidden separators the layout now hyphenates the visible
word, so "hyphenation " repeated 8,192 times with every space hidden is one
90,112-letter word: 2.0 s to convert with the release CLI.
Letter and uppercase counts before each char, the letter run of each char
and the heuristic's next vowel and byte offsets are now computed once, so
every point is checked in constant time: that document converts in 0.09 s.
The output is unchanged: the old functions stay in the tests as the
reference, compared over awkward words and every combination of limits, and
a hyphenated document's PDF text is identical. No maximum word length is
imposed: the work is linear, so a cap would only change output.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: find the formatting of every object anchor in one walk
left_out_objects() asked props_of_char() for each floating object's anchor,
and props_of_char() scans the runs from the start, so a paragraph with many
anchors in their own runs cost a scan per anchor, quadratic, and paid it on
every layout because it runs before the paragraph cache is consulted:
40,000 hidden anchors took 0.69 s, against 0.05 s for 10,000.
The runs and the object offsets are both in order, so one walk now finds
each anchor's run, resolving a run's formatting once; past the last run it
is the paragraph mark, as props_of_char() gives. With hidden text shown and
markup on nothing can be left out and no work is done at all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: find a run's style without formatting it as a string
#152 keyed each run's style by its whole format's Debug string, built for every
shaped piece; on a long document with many runs that doubled layout time (3000
paragraphs x 40 linked runs: 12.7 s vs 6.4 s for the old key). Key by a hash of
the commonly varying fields and confirm by comparing formats: 3.8 s. A capped
bucket scan keeps formats that never compare equal (NaN) linear.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Layout: floats hidden in table cells take no room, table style included
Test the layout_box path the merge extended (a float anchored in hidden text in a
table cell is not placed and leaves no wrap area), and resolve the anchors'
formatting with the table style's character formatting under the runs' own, as
the cell's text is laid out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Tests: linear-time checks compare 8x the input against a 24x limit
At 4x with an 8x limit a busy machine could push a linear layout over the line (8.8x seen once in a full workspace run); at 8x a linear layout takes about 8x and a quadratic one about 64x, so 24x stays far from both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
