From 2666031c35ce28277868fbf1dcc991d166e19dd6 Mon Sep 17 00:00:00 2001 From: Valentin Klein Date: Sun, 11 Oct 2026 17:14:31 +0200 Subject: [PATCH] Spelling: common words like "desktop" are recognised (#520) (#522) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The English speller only knew the 1990s Moby word list plus a small hard-coded EXTRA array, so everyday modern words (desktop, laptop, download, upload, spreadsheet, database, screenshot, inbox, offline, workplace…) were flagged. Inflection, doubled-consonant and prefix rules also only looked stems up in Moby, so even supplemented words failed in inflected forms (blogging, uploader). - New supplement assets/spelling/en-extra.txt: ~220 lowercase base forms, our own list (MIT OR Apache-2.0), replacing the EXTRA array. - All stem lookups (inflections, doubled consonant, prefixes) consult the dictionary and the supplement; suggestions include supplement words. - Regression tests: common modern words and inflections accepted, misspellings still flagged, "dekstop" suggests "desktop"; the list stays sorted and lowercase. Co-authored-by: Claude Opus 5.5 Co-authored-by: Brandon Thomas --- ATTRIBUTION.md | 1 + assets/spelling/en-extra.txt | 224 +++++++++++++++++++++++++++++++++++ crates/proof/src/lib.rs | 149 +++++++++++------------ docs/localization-parity.md | 3 +- docs/target-app-parity.md | 5 +- 5 files changed, 296 insertions(+), 86 deletions(-) create mode 100644 assets/spelling/en-extra.txt diff --git a/ATTRIBUTION.md b/ATTRIBUTION.md index fec7816fc..2117de034 100644 --- a/ATTRIBUTION.md +++ b/ATTRIBUTION.md @@ -10,6 +10,7 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra |---|---|---|---|---| | `assets/hyphenation/en-us.pat` | DesignCraft contributors (generated) | trained from the Moby list by DesignCraft's patgen | Public Domain (CC0-1.0) | Our own Liang hyphenation patterns (no TeX patterns) | | `assets/hyphenation/en-us.dic` | Grady Ward (Moby Hyphenator II word list); converted by DesignCraft contributors | https://www.gutenberg.org/ebooks/3204 (file `mhyph.txt`, SHA-256 `eeb30474c86b8af3469035ec1a0913e35905325ca885290db9dfed9881e230ac`), via DesignCraft | Public Domain | ~165k words with break points; drives spelling and hyphenation | +| `assets/spelling/en-extra.txt` | WordCraft contributors | original word list written for this repository (common modern words the Moby list lacks); no other dictionary copied | MIT OR Apache-2.0 | ~220 lowercase English base forms supplementing spelling (#520) | | `assets/app-icon/wordcraft-1024.png` | WordCraft contributors | original artwork drawn in SVG (`assets/app-icon/wordcraft.svg`), rendered by `packaging/icons.sh` | MIT OR Apache-2.0 | Placeholder app icon until the WordCraft mascot drawing lands | | `packaging/macos/dmg/background.svg` | @XusBadia | original work for this repository, derived from the WordCraft app icon (`assets/app-icon/wordcraft.svg`, referenced via ``, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0, like the code ([`LICENSE-MIT`](LICENSE-MIT), [`LICENSE-APACHE`](LICENSE-APACHE)) | WordCraft macOS DMG window background (2026-10-08); used for the Finder window of the macOS DMG, rendered to `background.tiff` by `packaging/macos/dmg/generate.py` | | `packaging/macos/dmg/background.tiff` | @XusBadia | Rendered from `packaging/macos/dmg/background.svg` by `packaging/macos/dmg/generate.py` | MIT OR Apache-2.0, like the code ([`LICENSE-MIT`](LICENSE-MIT), [`LICENSE-APACHE`](LICENSE-APACHE)) | 1x + 2x HiDPI TIFF for the macOS DMG window; includes the app icon (see its row) and LittleCMS's built-in sRGB profile ("No copyright, use freely") | diff --git a/assets/spelling/en-extra.txt b/assets/spelling/en-extra.txt new file mode 100644 index 000000000..41567c7d7 --- /dev/null +++ b/assets/spelling/en-extra.txt @@ -0,0 +1,224 @@ +# English spelling supplement: common words the public-domain Moby list (1990s) lacks. +# Written by WordCraft contributors (MIT OR Apache-2.0); see ATTRIBUTION.md. +# One lowercase base form per line, sorted. Plurals, -ed, -ing, -er, -ly… come from the +# inflection rules in crates/proof/src/lib.rs, so list "desktop", not "desktops". +airbag +airfare +artcraft +autocomplete +autocorrect +autofill +autoformat +autosave +backend +bitcoin +bitmap +blockchain +blog +blogger +bluetooth +captcha +caregiver +carpool +cellphone +chatbot +chatroom +checkbox +checklist +childcare +clickable +configurable +copyedit +coronavirus +covid +crowdfunding +crowdsource +crypto +cryptocurrency +customizable +customization +cyber +cybersecurity +cyberspace +database +dataset +daycare +desktop +download +downloadable +downsize +dropdown +earbud +ebook +ecommerce +email +emoji +emoticon +esports +facebook +favicon +fintech +firewall +flowchart +foodie +freeware +frontend +fundraiser +gameplay +gamification +geeky +geolocation +gigabit +gigabyte +github +globalization +google +groupthink +hackathon +hardcode +hashtag +healthcare +homepage +homeschool +hotspot +hyperlink +hypertext +inbox +incentivize +infographic +instagram +internet +interoperability +intranet +ipad +iphone +javascript +kickstart +kilobyte +laptop +lifestyle +linkedin +linux +livestream +lockdown +login +logout +lowercase +makeover +malware +megabyte +megapixel +meme +metadata +microchip +microsoft +middleware +mindset +multicultural +multiplayer +multitask +nerdy +netbook +netflix +newsfeed +offboard +offline +onboard +online +onscreen +outsource +paralegal +passcode +petabyte +phishing +photobomb +playdate +playlist +plugin +podcast +popup +prequel +printout +prioritization +prioritize +ransomware +recyclable +ringtone +roadmap +rollout +runtime +scalability +screencast +screensaver +screenshot +scrollbar +selfie +signup +sitemap +skillset +smartphone +smartwatch +smiley +snowboard +spellcheck +spellchecker +spreadsheet +spyware +stakeholder +standup +startup +subfolder +sunscreen +superfood +superuser +tagline +takeout +techie +telecommute +teleconference +telehealth +telemedicine +terabyte +textbox +timeline +timestamp +toolbar +toolkit +touchpad +touchscreen +trackball +trackpad +upcycle +upload +uppercase +upsell +upskill +usability +username +videoconference +videogame +vlog +vlogger +voicemail +walkthrough +webcam +webcast +webinar +weblog +webmaster +webpage +website +whatsapp +whiteboard +wifi +wiki +wikipedia +wireframe +wordcraft +workaround +workflow +workforce +workplace +workspace +workstation +youtube diff --git a/crates/proof/src/lib.rs b/crates/proof/src/lib.rs index 8570b7a0c..9ccfac92a 100644 --- a/crates/proof/src/lib.rs +++ b/crates/proof/src/lib.rs @@ -1,8 +1,10 @@ //! WordCraft proofing. //! //! - **Spelling** against the public-domain Moby word list (~160k words, the same list that -//! drives hyphenation), with inflections (`-s`, `-es`, `-ed`, `-ing`, `-ly`, `'s`…), a user -//! dictionary and ignore lists; suggestions by edit distance. +//! drives hyphenation) plus our supplement of modern words it lacks +//! (`assets/spelling/en-extra.txt`: `desktop`, `email`, `download`…), with inflections (`-s`, +//! `-es`, `-ed`, `-ing`, `-ly`, `'s`…), a user dictionary and ignore lists; suggestions by edit +//! distance. //! - **Grammar** checks that are cheap and reliable: repeated words, `a`/`an`, capital letter //! after a sentence end, spaces before punctuation, doubled spaces. //! - **Hyphenation** ([`hyphen`]): dictionary, Liang patterns, heuristic. @@ -51,86 +53,23 @@ pub fn user_dictionary() -> Vec { /// Common words the source list lacks (modern vocabulary, contractions). const COMMON: &str = "a an the and or but if of to in on at by for with from as is am are was were be been being it its we us our you your he him his she her they them their i me my mine this that these those there here what which who whom whose when where why how all any both each few more most other some such no nor not only own same so than too very can will just should now do does did done has have had having may might must shall would could up down out over under again further then once off into onto upon about above below between through during before after until while because per via vs etc ok yes yeah oh ah hi hey bye one two three four five six seven eight nine ten first last next new old get got go goes went gone see saw seen say said make made take took come came give gave know knew think thought look want need use used find found tell told ask asked feel felt try left keep let put mean set run show turn move live play pay hear hold bring begin seem help talk start might"; -const EXTRA: &[&str] = &[ - "don't", - "doesn't", - "didn't", - "can't", - "won't", - "isn't", - "aren't", - "wasn't", - "weren't", - "shouldn't", - "wouldn't", - "couldn't", - "i'm", - "i've", - "i'd", - "i'll", - "you're", - "you've", - "you'll", - "you'd", - "we're", - "we've", - "we'll", - "they're", - "they've", - "they'll", - "it's", - "that's", - "there's", - "here's", - "what's", - "let's", - "email", - "emails", - "online", - "website", - "websites", - "internet", - "software", - "app", - "apps", - "smartphone", - "blog", - "login", - "logout", - "username", - "wifi", - "toolbar", - "workflow", - "workflows", - "dataset", - "datasets", - "metadata", - "startup", - "startups", - "covid", - "selfie", - "podcast", - "podcasts", - "hashtag", - "wordcraft", - "artcraft", - "discord", - "github", - "rust", - "ok", - "okay", - "unlabelled", - "labelled", - "colour", - "colours", - "favourite", - "centre", - "organise", -]; +/// Our supplement of common words the Moby list lacks (`desktop`, `email`, `download`…): one +/// lowercase base form per line, `#` comments. Inflections come from [`known_core`]. +const EXTRA_EN: &str = include_str!("../../../assets/spelling/en-extra.txt"); + +fn extra_words() -> &'static HashSet<&'static str> { + static E: OnceLock> = OnceLock::new(); + E.get_or_init(|| EXTRA_EN.lines().map(str::trim).filter(|l| !l.is_empty() && !l.starts_with('#')).collect()) +} + +/// A base form in the dictionary or the supplement. +fn base_known(d: &dict::Dictionary, w: &str) -> bool { + d.get(w).is_some() || extra_words().contains(w) +} fn known_core(w: &str) -> bool { let d = dict::Dictionary::en_us(); - if d.get(w).is_some() || EXTRA.contains(&w) || COMMON.split(' ').any(|c| c == w) { + if base_known(d, w) || COMMON.split(' ').any(|c| c == w) { return true; } if user_words().read().unwrap_or_else(|e| e.into_inner()).contains(w) { @@ -157,7 +96,7 @@ fn known_core(w: &str) -> bool { { for a in adds { let s = format!("{stem}{a}"); - if d.get(&s).is_some() || EXTRA.contains(&s.as_str()) { + if base_known(d, &s) { return true; } } @@ -167,7 +106,7 @@ fn known_core(w: &str) -> bool { && b.len() >= 3 && b.last() == b.get(b.len() - 2) && let Some(s) = stem.get(..stem.len() - 1) - && d.get(s).is_some() + && base_known(d, s) { return true; } @@ -177,7 +116,7 @@ fn known_core(w: &str) -> bool { for pre in ["un", "re", "non", "pre", "over", "under", "co", "mis"] { if let Some(rest) = w.strip_prefix(pre) && rest.len() >= 3 - && d.get(rest).is_some() + && base_known(d, rest) { return true; } @@ -220,7 +159,7 @@ pub fn suggest(word: &str, max: usize) -> Vec { let d = dict::Dictionary::en_us(); let mut scored: Vec<(usize, i64, String)> = Vec::new(); let target: Vec = lower.chars().collect(); - for (w, _) in d.iter() { + for w in d.iter().map(|(w, _)| w).chain(extra_words().iter().copied()) { let wn = w.chars().count(); if wn + 2 < n || wn > n + 2 { continue; @@ -460,6 +399,50 @@ mod tests { assert!(suggest("Wrold", 3).contains(&"World".to_string())); } + /// #520: common modern words (and their inflections) are known; misspellings still flagged. + #[test] + fn common_modern_words() { + for w in [ + "desktop", + "Desktops", + "laptop", + "download", + "downloaded", + "uploading", + "email", + "emailing", + "website", + "online", + "offline", + "smartphones", + "login", + "username", + "spreadsheet", + "database", + "screenshots", + "blogging", + "inbox", + "workplace", + "uploader", + ] { + assert!(is_correct(w), "{w}"); + } + for w in ["dekstop", "laptp", "dowload", "spreadshet"] { + assert!(!is_correct(w), "{w}"); + } + assert!(suggest("dekstop", 5).contains(&"desktop".to_string()), "{:?}", suggest("dekstop", 5)); + assert!(check_spelling("My desktop and laptop download email online.").is_empty()); + } + + /// The supplement stays a sorted list of lowercase base forms. + #[test] + fn supplement_is_sorted_lowercase() { + let words: Vec<&str> = EXTRA_EN.lines().filter(|l| !l.is_empty() && !l.starts_with('#')).collect(); + assert!(words.len() > 100); + assert!(words.windows(2).all(|p| p[0] < p[1]), "not sorted / duplicate"); + assert!(words.iter().all(|w| w.chars().all(|c| c.is_lowercase() || c == '\'' || c == '-')), "not lowercase"); + } + #[test] fn spelling_issues_in_text() { let t = "This sentense has a mispelled word, see https://example.com/xyzq."; diff --git a/docs/localization-parity.md b/docs/localization-parity.md index 166afbd3c..53f896ad2 100644 --- a/docs/localization-parity.md +++ b/docs/localization-parity.md @@ -1,6 +1,6 @@ # Localization parity with Microsoft Word -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-10 · **Change:** major (first version; catalogs measured from `crates/ui-egui/src/i18n`, Word's languages from its bundle listing) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** trivial (English spelling knows common modern words, #520; previously major: first version; catalogs measured from `crates/ui-egui/src/i18n`, Word's languages from its bundle listing) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 Interface languages, script support and proofing per language. How translation works (catalog format, adding a language, clean-room rule): `crates/ui-egui/src/i18n/mod.rs`. @@ -72,6 +72,7 @@ Estonian spelling, grammar, dates and document templates are not added by the in | Date | Change | Summary | |---|---|---| +| 2026-10-11 | trivial | English spelling (#520): our supplement `assets/spelling/en-extra.txt` adds ~220 common modern words the Moby list lacks ("desktop", "download", "email"…) | | 2026-10-10 | minor | Estonian interface catalog, locale selection, saved preference and font coverage checks | | 2026-10-10 | minor | Serbian (Cyrillic and Latin) interface merged (#250) | | 2026-10-10 | major | First version: twelve-language table, catalog coverage measured, Word's 30 UI languages and 120 proofing tools listed | diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index befa05e78..64a676f86 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -1,6 +1,6 @@ # WordCraft parity with Microsoft Word -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** trivial (all 12 bibliography styles, #383; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; replaces the parity tables that lived in ROADMAP.md) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-11 · **Change:** trivial (English spelling knows common modern words, #520; previously trivial: all 12 bibliography styles, #383; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; replaces the parity tables that lived in ROADMAP.md) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4 The authoritative assessment of how close WordCraft is to Microsoft Word. [`ROADMAP.md`](../ROADMAP.md) summarizes it; [`gaps.md`](gaps.md) itemizes every shortfall; the deep checklists are @@ -66,7 +66,7 @@ full parity for the area. | Footnotes and endnotes | 3 | 60% | 60% | 5–8 | Placed and editable, numbers from Word files (#103). Long notes don't continue onto the next page; no continuation separator; no note options per section. | | References (TOC, citations, bibliography, captions, index, TOA) | 5 | 65% | 60% | 12–20 | Working first versions, Zotero integration (#189), Zotero/Mendeley `ADDIN` fields round-trip. All 12 of Word's bibliography styles (APA, MLA, Chicago, IEEE, Harvard, ISO 690 author-date and numerical, Turabian, GB/T 7714, GOST name and title sort, SIST02; #383); source manager depth; bibliography sources not yet in DOCX (#169 open). | | Review (proofing UI, comments, track changes, compare, protect) | 8 | 72% | 75% | 15–25 | Comment balloons, tracked insert/delete incl. paragraph marks (#244, #125), compare, restrict editing. Formatting revisions (#41): kept in DOCX, recorded while tracking, accepted/rejected, "Formatted: …" balloons and change bars. Missing: move tracking, insert/delete balloons, Translate, Block Authors, modern comment threads/mentions. Lowered: these gaps are confirmed by source (no `rPrChange` anywhere). | -| Proofing (spelling, grammar, thesaurus, languages) | 5 | 35% | (in Review) | 25–45 | English only: one dictionary, a rule-based grammar checker. Word ships 120 proofing tools for ~50 languages plus Editor (style refinements, similarity). Issues #25, #40, #100. | +| Proofing (spelling, grammar, thesaurus, languages) | 5 | 35% | (in Review) | 25–45 | English only: one dictionary (Moby list plus our supplement of modern words such as "desktop", #520), a rule-based grammar checker. Word ships 120 proofing tools for ~50 languages plus Editor (style refinements, similarity). Issues #25, #40, #100. | | Mailings | 2 | 70% | 70% | 6–10 | Mail merge from CSV, fields, rules (IF, SKIPIF, NEXT, NEXTIF, evaluated alike in preview and merge), fields in headers/footers (one section per letter), preview, envelopes, labels. Select Recipients takes a typed list or CSV (#247); no Excel/Outlook/Contacts data sources, no email merge. | | Pictures, shapes, text boxes, WordArt | 6 | 60% | 60% | 20–30 | Insert, crop, recolour, styles, floating placement like Word (#136), wrap (square/top-bottom/behind/front), text boxes edit and overflow (#46), VML text boxes (#242), free rotation and flips with a rotation handle (#332). Missing: tight/through contour wrap, rotated text-box text, group (#267 open), shape effects, WordArt, connectors, Drawing Canvas, Icons, online pictures, screenshot. | | Equations | 1 | 75% | 20% (with Draw) | 4–8 | OMML read/write, OpenType MATH layout, in-place editor with an Equation tab (#191). Missing: line breaking of long display equations, ink equations. | @@ -249,6 +249,7 @@ can be used without committing Word output. | Date | Change | Summary | |---|---|---| +| 2026-10-11 | trivial | English spelling (#520): common modern words ("desktop", "download", "email"…) recognised via our supplement list; Proofing row updated, no percentage change | | 2026-10-11 | trivial | References: all 12 of Word's bibliography styles (eight more: Harvard, ISO 690 ×2, Turabian, GB/T 7714, GOST ×2, SIST02) (#383); no percentage change | | 2026-10-11 | trivial | Mail merge fixes (#427–#430): SKIPIF drops matching records, NEXT advances later fields, IF previews, header/footer fields merge; Mailings row updated, no percentage change | | 2026-10-11 | trivial | Formatting revisions (#41): Review row updated; no percentage change |