Compare commits

..
Author SHA1 Message Date
Dylan M. Taylor e15899c8a1 fonts: cut names read from a font's name table to 128 characters
A name record can hold 64K characters, and every face copies its family into
each localized name and every variable-font named instance (with their
localized names again), so a small font could claim hundreds of megabytes of
name strings before any later check.

name() now keeps at most MAX_NAME_CHARS characters, the limit localized names
already used, so every copy stays small.
2026-11-02 20:36:38 -05:00
Brandon ThomasandClaude Opus 5.5 b47079fc79 Release 0.15.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 19:25:56 -05:00
Dylan M. Taylor 866c1f9cac Merge pull request #5208 from benandez/feat/cp4-language-packs
CP4.3–CP4.5: Language packs — proof.packs/installPack/removePack, Options › Language packs, missing-pack bar, About › Language data
2026-11-02 17:32:04 -05:00
Dylan M. Taylor 0bb9add551 Merge main into language packs PR 2026-11-02 16:53:31 -05:00
Dylan M. Taylor 0a5b4221fd Merge pull request #5205 from benandez/feat/cp2-scowl
CP2.3: English spelling reads SCOWL 2026 (#40, #520, #1281, #1045)
2026-11-02 16:52:47 -05:00
Dylan M. Taylor 52a8a5db76 Merge main into SCOWL spelling PR 2026-11-02 16:23:38 -05:00
Dylan M. Taylor 421bc8cdcd Merge pull request #5202 from benandez/feat/cp1-craft-proof
CP1.3: proofing moves to craft-proof (craft-libs) and craft-i18n language-data
2026-11-02 16:23:16 -05:00
Dylan M. Taylor 3ce9a84f81 Merge main into proofing move PR 2026-11-02 15:57:35 -05:00
FredApps bbe3547dd1 Merge pull request #1104 from naxvog/formats/alt-chunks
Files: documents keep their embedded alternative format chunks (w:altChunk), and MHT opens
2026-11-02 19:05:39 +01:00
FredApps 389fa90c29 Merge remote-tracking branch 'origin/main' into pr1104
# Conflicts:
#	ROADMAP.md
2026-11-02 18:47:20 +01:00
FredApps 84b4623e14 Merge pull request #4471 from totoz64/feat/spelling-review-dialog
Review: add a Word-style Proofing task pane with clickable suggestions
2026-11-02 18:40:33 +01:00
FredApps e2bc9e017e Merge remote-tracking branch 'origin/main' into pr1104
# Conflicts:
#	docs/file-format-parity.md
2026-11-02 18:25:52 +01:00
41d46625e0 DOCX: VML horizontal rules take their line (#3956)
* DOCX: VML horizontal rules take their line

A web page's <hr> pasted into Word is a VML rule: `v:rect` with
`o:hr="t"`, often `width:0`. WordCraft took the width as written and
drew a 1 pt dot where Word draws a line across the page.

Measured on Word with a synthetic document: a rule with no width and
no `o:hrpct` spans the line inside the paragraph's indents; `o:hrpct`
is a share of it in tenths of a percent (500 = half); `o:hrpct="0"`
keeps the shape's own width; `o:hralign` places a shorter one (left,
centre, right); `fillcolor` and `stroked` apply as to any VML shape.
WordCraft now matches each case within 0.1 pt.

- wordcraft-doc: ShapeExtra::hr (HorizontalRule: share, alignment,
  Word's standard look).
- wordcraft-docx: `o:hr`, `o:hrpct`, `o:hralign`, `o:hrstd` read; a rule
  is written back as the same VML rule (DrawingML has none), and Word
  opens the re-saved file with the same lines.
- wordcraft-layout: a rule takes its whole line and is drawn at its
  share; measured for a table column it asks for no width, so it
  follows its column. (Word draws a rule in a cell at the page's text
  width, centred on the cell and past its edges; it stays in the cell
  here.)

docs/file-format-parity.md is updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Horizontal rules: only an inline, shown rule is one

Review: the rule arm of the writer ignored the shape's placement, so a
rule imported with position:absolute or visibility:hidden came back
inline and visible. Only a rule in the text and shown is read as a rule
now (a pasted <hr> is one); a positioned or hidden one stays a plain
shape, which keeps both when saved. The writer checks the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Horizontal rules: a see-through fill is kept; column measuring asks for the right width

Review: `write_hr` wrote only the fill's colour, so a rule with `v:fill
opacity` came back opaque after a save; the opacity now goes back as `v:fill`.

Review: measured for a table column, every rule asked for no width, so one of
its own width (`o:hrpct="0"`) could end up clipped by a narrow column; it now
asks for its width. AutoFit to contents measures on a line of its own width
that the rule took for a real line, so a rule sharing its line made the
column as wide as it could go; any measuring width now counts.

`Placed::Shape` boxes its extras: with the rule, `Placed` went past clippy's
large_enum_variant limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Horizontal rules: placed in what the line has left; rotation and flips kept on save

Review: a whole-line rule ignores a first-line indent or a line narrowed by a
float. Measured in Word, it does too: the width stays the share of the
paragraph's width and the rule runs past the margin, or under a float on its
right. What did differ was the placement: Word centres and right-aligns a rule
in what the line has left, with a first-line indent moving the end of that room
along, and a right-aligned rule beside a float was drawn off the page here. The
display now places it that way (`ParaLayout::rule_room`).

Review: a save dropped a rule's rotation and flips; they now go back in its
style, as Word writes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: VML horizontal-rule revision row at the top of the history

---------

Co-authored-by: Daniele Mori <Dinasty@hotmail.it>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-02 10:55:20 -05:00
Fin WarmanandBrandon Thomas 4f2d0da04c Shapes: align soft-edge paint and caret opacity (#1600)
* Shapes: a solid fill keeps its transparency

A shape's fill colour can carry a:alpha, its opacity. WordCraft dropped
it, so a translucent white wash laid over a picture was drawn opaque and
hid the picture. The transparency is now read into the shape, drawn on
screen and in the PDF, and written back.

* Fix #1600 freeform and soft-edge fill opacity

* #1600: wrap canonical caret sampler signature

* #1600: format canonical integration

* #1600: assert canonical DOCX alpha round-trip

* #1600: document canonical shape opacity API

* Adapt alpha fixture to rectangle render bounds

* docs: one file-format-parity row for soft-edge fill opacity, status line

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-02 10:53:00 -05:00
242e850aaa #3660: refuse a package that declares too many zip entries before zip parses them (#5260)
* #3660: refuse a package that declares too many zip entries before zip parses them

zip::ZipArchive::new parses and keeps every central directory header
the end record declares before its caller can look at len(). A ZIP64
end record can declare one per 46 bytes of the file: a 47 MB package of
one stored file listed 1,000,000 times takes zip 3.0.0 about half a
second and a large transient allocation, and len() then says 1, since
the copies share a name, so the docx reader's MAX_ENTRIES cap never
fires. ODT and spreadsheet packages had no cap at all.

wordcraft_doc::zip_guard::check finds the end records the way zip finds
them (backwards; the ZIP64 record from the locator's offset on; a plain
record when it has no entries or a central header before it) and refuses
a package declaring more than the reader's limit. The docx package
reader, the ODT reader, the spreadsheet reader (mail-merge data) and the
template-title probe call it first, with 20,000 entries as the limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* formats: keep the extensions doc line on EXTENSIONS

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-02 04:58:18 -05:00
Hamza NaveedandBrandon Thomas ee8622ddb3 Count the runs proofing text walks, instead of timing it (#5185) (#5190)
* Count the runs proofing text walks, instead of timing it (#5185)

`proof_text_walks_the_runs_once` was the one linear-time test here that timed
its walk, and the ratio crept over its 24x limit when the crate's tests ran in
parallel -- 28.2, 26.4, 25.6 -- from cache and memory contention rather than
from any regression.

The file's own module doc says why the others don't have this problem: "The
cost is the steps the lookups' walks count (crate::work), not a timing, so a
busy machine can't make a linear lookup look quadratic or the other way round."
It then says this one is timed because "proofing text, whose walk is the
document model's" counts no steps -- but `proof_text` does the walk itself,
advancing a runs iterator alongside the text, so it can count the runs it looks
at like every other walk here. Nothing in `wordcraft-doc` had to change.

    before   8x the runs took 28.2x the time (limit 24), under load
    after    50000 steps for n, 400000 for 8n, ratio 8.0 (limit 12)

The count is the same on any machine under any load, so the flakiness is gone
rather than made less likely: raising the limit to 32x would have left the same
failure available on a busier machine. It is also about 100x faster -- 0.03 s
against three rounds of eight repetitions -- and `work::steps` is `#[cfg(test)]`
inside, so release builds are untouched.

The test still catches what it was written for. Putting the quadratic walk back
(the run for each character found from the first run, every time) gives:

    proof_text: 1250025001 steps for n, 80000200001 for 8n, ratio 64.0
    panicked: 8x the input took 64.0x the steps: not linear

Verified under the reported condition: three runs of the whole crate suite in
parallel, 559 tests, all green.

* linear tests: proofing text is counted now, not timed

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-02 04:55:59 -05:00
Martin BallandClaude Opus 5.5 72327bdd36 CP4.4: pin language-packs-1 index
craft-i18n's first Language pack release is published
(https://github.com/storytold/craft-i18n/releases/tag/language-packs-1, 61 packs,
built from craft-i18n main 4504e288). PACK_INDEX now pins its index.json sha256
instead of the all-zero placeholder. Docs say which languages it covers and which
wait on craft-proof (compounding) or are excluded on licence grounds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 09:40:07 +00:00
Martin BallandClaude Opus 5.5 fbe68547f4 CP1.3: review — CLAUDE.md describes no-data proofing the same as AGENTS.md (English falls back to Hunspell or the Moby list; pattern hyphenation off)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 09:28:04 +00:00
NaxvogandClaude Opus 5.5 868fffa6fe Files: a Word chunk that asks to keep its source formatting keeps it (w:matchSrc)
An alternative format chunk holding a Word document with w:matchSrc keeps its
own styles: a style the document already has comes along under a new id, the
chunk's document defaults (and Word's, where they say nothing) go into its
base styles, and its unstyled paragraphs use its own Normal. Without
w:matchSrc the document's styles win, as before. Adopting another document
now also renumbers table styles and the revisions of table, row and cell
property changes.

A document whose chunk shares style names with it (Normal with an exact line
height, an italic Subtitle) ran a page longer than in Word: the chunk's tables
took the document's styles. It now keeps its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 07:04:31 +01:00
NaxvogandClaude Opus 5.5 89c0d516ae Files: documents keep their embedded alternative format chunks, and MHT opens
Word brings a document's alternative format chunks (w:altChunk) in where they
stand when it opens the file; WordCraft left them out, so a document made of
one (as generators and mail-merge tools write them) opened empty.

- docx: read_with/ReadOptions with a ChunkImporter for formats other than
  WordprocessingML; nested Word documents are read here (at most 4 deep,
  256 chunks and 512 MB of chunk bytes a file, XML depth counted on).
- doc: Document::adopt (moved from paste) brings another document's media,
  lists, styles, notes, comments and revisions along under new ids; pictures
  in groups and charts are relinked. Insert > Text from File uses it and now
  keeps the file's pictures and lists.
- engine: chunks in MHT, HTML (in their charset), RTF and plain text go
  through the format importers.
- formats: MHT web archives open (MIME, quoted-printable/base64, charsets,
  pictures by Content-Location or cid). HTML reads style element rules
  (simple selectors, cascade, budgets), paragraph margins, indents, line
  height, cell widths and borders; an opened page takes Word's web-page
  defaults (Times New Roman 12 pt, single lines, black headings, automatic
  spacing). html::decode knows the single-byte Windows code pages.

docxide-pdf suite: croatian_regulations_altchunk goes from one empty page to
Word's 8 pages, SSIM 2.7% -> 78.7%; no other document changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 07:04:31 +01:00
NetterSilverandClaude Opus 5.5 3e2c79407d review.compare: keep text boxes of both documents in the result (#5248)
Fixes #5235

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 00:24:32 -05:00
ReesandBrandon Thomas bf9e9fb8ae DOCX: keep alternate-content table rows and cells (#5221)
* Fix #5219: retain alternate-content table rows and cells

* Re-wrap the collect() doc comment

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 23:17:19 -05:00
Martin BallandClaude Opus 5.5 1376a5476b CP4.3: restack on #5205 9ffd30d; craft-proof at storytold/craft-libs#495 89fa1559 (contains #485), font store and catalog at #486 762a06a4 (store-lock race fix)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:23:48 +00:00
Martin BallandClaude Opus 5.5 34525f1cb1 CP4.3: review — regression test: a same-length index refresh still changes index_generation, so the missing-pack bar works it out again (CodeRabbit)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:04:00 +00:00
Martin BallandClaude Opus 5.5 905570a90b CP4.3: restack on #5205 3a666d3 — craft-proof at storytold/craft-libs#485 8f16fe5a (on #483 6e45fda); Cargo.toml and Cargo.lock rebuilt from the base, so main's version and other pins stay
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:04:00 +00:00
Martin BallandClaude Opus 5.5 03c50f8af2 CP4.3: restack on #5205 — craft-proof #485 bdd434bc and font store #486 83c2e7b from storytold/craft-libs, craft-i18n 059cbf9; SCOWL notice in About; no "(no dictionary)" or bar for a language with no pack; Swiss German and Portugal's Portuguese keep their names
- craft-proof now names ja, zh and th in missing_dictionary: packs::missing_dictionary hides it
  for a pack language when no pack for it is listed (status bar and Options).
- de-CH and pt-PT are pack languages in craft-proof (de-ch, pt-pt): names, hyphenation and
  grammar stay German and Portuguese (tests); pack names fall back to the region-aware name.
- About › Language data shows craft_proof::scowl::notice() when SCOWL is embedded (test).
- Flatpak manifest's craft-i18n commit follows craft-i18n.rev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:04:00 +00:00
Martin BallandClaude Opus 5.5 a8250e6472 CP4.3: restack — drop the ROADMAP rows the rebase doubled
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:04:00 +00:00
Martin BallandClaude Opus 5.5 1146691b90 CP4.3: review — craft-libs at #485 f88bd48 / #486 83c2e7b; test-source warning in proof.packs, the CLI and the bar; About › Language data a fixed-height list; Cancel reaches a stalled download
- proof.packs reports index_override; the CLI prints the test source; the bar's Install tooltip
  says it turns on downloading of fonts and Language packs, and names a test source.
- Docs: Install works once language-packs-1 is published (until then only a test index); with a
  test index the sha256 only catches a broken download.
- About › Language data keeps its size: a fixed-height scrolling list, licence texts read again
  only when the dictionaries change.
- The engine's Cancel is the font store's own flag (no snapshot), so a cancel stops a fetch or a
  stalled download too.
- Index text shown without control, bidi or format characters; long names cut; the bar works out
  what to offer again when the list changes (index generation, not its length).
- ui_shot's store is a temporary folder (or storeDir); tests reset the backend even on panic; a
  fixture pack proves per-run proofing and hyphenation from a pack.
- The default-build check follows craft-proof's carry-over (de, pl, pt, it, ru embedded until packs
  ship; CRAFT_PROOF_EMBED=en is English only); the Polish hyphenation check runs again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:04:00 +00:00
Martin BallandClaude Opus 5.5 c1490c5ba9 CP4.3: review — a click from the control channel never turns on downloading; installs and refreshes run in the background in the app; parked documents proof again after a pack change
- Install, Show available languages and Refresh turn on the font store's downloading only on the
  person's own click (input_from_control, as in #959); a faked click says why instead.
- proof.installPack and proof.packs {refresh} run on the engine's thread in the app ("started"),
  with progress, results and errors in proof.packs; the CLI and headless MCP stay synchronous.
- A session's layout is keyed on the dictionaries' generation, so parked documents proof again.
- proof.removePack refuses while that pack installs; pack commands that need a locale are left out
  of Search; options text names Refresh; licence tooltip says Source; errors go through tl!;
  the bar rescans at most once a second while typing; English names for every pack language and
  sr-latn; About has one Close button; xtask doc comment back on cmd_i18n_report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:04:00 +00:00
Martin BallandClaude Opus 5.5 82418896f0 CP4.3: layout test checks Polish hyphenation only when the Polish patterns are embedded (a Language build); otherwise they come from the pack
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:04:00 +00:00
Martin BallandClaude Opus 5.5 efb7d8ef20 CP4.3: craft-proof pinned to storytold/craft-libs#485's rebased head (6e8eb65), font store to #486 (d342a50); Pack languages report their tag
- `review.issues` and `file.options` report a run's language with `ProofLang::tag()`, so a
  Pack language reports its locale, not "".
- The test-source warning reads craft-font-store's `language_pack_index_overridden()`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:04:00 +00:00
Martin BallandClaude Opus 5.5 2ef046bad9 CP4.3: craft-libs pinned to Lane 4's pushed heads; test-source warning, upstream source links, code-drawn bar close button
- craft-proof (benandez/craft-libs 6ae4f03, storytold/craft-libs#485) and craft-font-store /
  craft-font-catalog (630e42f, #486) by git rev instead of local paths.
- ProofLang::Pack: the bar offers every language with a pack (zxx/und never).
- Options › Language warns when CRAFT_LANGUAGE_PACK_INDEX supplies a test index.
- Pack `source` (upstream link) when the index gives it, else the archive.
- The bar's close button is the code-drawn close icon: "✕" had no glyph in the interface fonts
  and egui panicked drawing it.
- `cargo xtask language` allows our own ru-extra, which stays embedded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:03:59 +00:00
Martin BallandClaude Opus 5.5 e5d46bb242 CP4.3: Language packs: proof.packs, proof.installPack, proof.removePack; Options › Language list, missing-pack bar, About › Language data
- engine `packs`: the packs Backend (the font store on the desktop and CLI, a fixed list in tests
  and ui_shot), the pinned index, installs and removals through craft-font-store's
  `language-pack` kind, craft-proof reading the store's `language-packs/` folder at once.
  Commands open nothing and never turn downloading on (only an Install click in the app does);
  confined automation can't write the packs folder outside its write root.
- ui-egui `language_packs`: File › Options › Language › Language packs (name, size, licence,
  source, Install/Remove, progress, errors), the missing-pack bar (dismissible per document,
  never for a language without a pack), About › Language data (full licence texts).
- CP4.4: `cargo xtask language` (in `ci`): the default build embeds only en-* Language data.
- CP4.5: docs (help, MCP, control protocol, gaps #8, localization parity, roadmap).
- Path deps to Lane 4's worktrees while developing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 01:03:59 +00:00
Martin BallandClaude Opus 5.5 9ffd30d968 CP2.3: review — NOTICE carries the SCOWL, Titze and Ispell notices; gap 8 records SCOWL English; status lines for localization-tr and architecture
CodeRabbit on #5205:
- NOTICE: the SCOWL English dictionaries (en_US, en_GB-ise, en_CA) with
  Kevin Atkinson's SCOWLv2 notice, Benjamin Titze's notice and Geoff
  Kuenning's Ispell BSD licence for the affix files, as their licences ask of
  distributions.
- docs/gaps.md: gap 8's evidence says each English checks its own SCOWL
  2026 dictionary, with the #40 and #520 regression tests; #40 (closed) is no
  longer listed as open. Status line and revision history.

Also: docs/localization-tr.md's craft-proof path change (3110dcf02, #5202)
and docs/architecture.md's no-data wording (b7dad64d9) get their status lines
and revision rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:39:15 +00:00
Martin BallandClaude Opus 5.5 96c6c19b9e CP2.3: review — pin craft-proof to craft-libs#483's re-review fix (British -ize only for real -ize words)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:31:38 +00:00
Martin BallandClaude Opus 5.5 919b8a5838 CP2.3: Cargo.toml — the craft-proof pin needs no follow-up (merge commits keep #483's head in storytold main)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:31:38 +00:00
Martin BallandClaude Opus 5.5 211377f550 CP2.3: review — pin craft-proof to craft-libs#483's review fixes (each English checks its own SCOWL dictionary) and craft-i18n to #114's merge
craft-proof moves to 0675d58: an English no longer takes another English's
SCOWL words (en-US flagged colourblind, en-GB colorblind), British English
keeps -ize, and English suggestions drop junk splits and names.
craft-i18n.rev and the Flatpak source move to storytold/craft-i18n#114's merge
commit (059cbf9: judgment/judgement right in every English).
No wordcraft test depended on the dialects crossing
(engine 1120 + 43, layout 578 pass). localization-parity's English row says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:31:38 +00:00
Martin BallandClaude Opus 5.5 09e9522a35 CP2.3: English spelling reads SCOWL 2026; regression tests for #40, #520, #1281, #1045
craft-proof moves to storytold/craft-libs#483's head (e6e948f, on #482's
9e5862a): each English checks against its own SCOWL rel-2026.02.25 dictionary
through the affix engine (ordinals via COMPOUNDRULE), with en-variants.txt as
the dialect policy. Lookup order: dictionaries folder, [English pack], SCOWL,
system Hunspell, the built-in Moby list; Moby stays for hyphenation.
craft-i18n.rev and the Flatpak source move to storytold/craft-i18n#109's merge
commit (efb1127), which adds the SCOWL files.

Typed-text tests pin British spellings in en-GB (#40), modern words (#520),
Canadian/Australian/NZ/Indian English (#1281, #1045) and ordinals; they skip
without the Language data, as the other data tests do. Docs: typing-parity,
localization-parity, ROADMAP progress log.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:31:38 +00:00
Martin BallandClaude Opus 5.5 b7dad64d94 CP1.3: review — no-data proofing described right, NOTICE names craft-i18n's hyphenation.pat files, Flatpak dictionaries folder, printing checkout without credentials, data gate asks the variant table too
CodeRabbit on #5202:
- AGENTS.md, ATTRIBUTION.md, README.md, docs/architecture.md: without
  CRAFT_I18N_DIR English spelling falls back to an installed Hunspell English
  or craft-proof's built-in Moby list, and only pattern hyphenation is off
  (Bulgarian, Turkish and Russian hyphenate by rule); they said English
  wasn't spell-checked. (CLAUDE.md carries the same sentence; left for the
  maintainer, as agents don't edit it.)
- NOTICE: craft-i18n's patterns are language-data/<lang>/hyphenation.pat,
  not en-us.pat, pl.pat, pt.pat, de.pat, it.pat.
- docs/spelling-dictionaries.md: the Flatpak's dictionaries folder is
  ~/.var/app/ai.storyteller.wordcraft/data/wordcraft/dictionaries; the host's
  folders are outside its sandbox.
- printing.yml: the repository checkout doesn't persist credentials.
- tests.rs no_language_data: also probes the American/British variant table,
  so a craft-i18n checkout missing some files skips the dialect tests
  instead of failing them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-02 00:31:32 +00:00
Martin BallandClaude Opus 5.5 b545bc2757 CP1.3: Printing checks out craft-fonts too, so the CJK interface languages from craft-i18n have glyphs in the UI tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:39:14 +00:00
Martin BallandClaude Opus 5.5 36400f69eb CP1.3: review — no_language_data says what it probes (the embedded en-us list, present exactly when CRAFT_I18N_DIR's data is)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:12 +00:00
Martin BallandClaude Opus 5.5 6bb3f085aa CP1.3: rebase onto main — remove the crates/proof files #5000 added (affix_suggest, slot, spelling, tr/test_dict; already in craft-proof) and resolve craft-proof from git in Cargo.lock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:12 +00:00
Martin BallandClaude Opus 5.5 9d5b98eb18 CP1.3: review — six engine tests run again without the Language data now that craft-proof's built-in English is the Moby list
Polish/Portuguese/Czech review commands, the custom dictionary file and the two typing squiggle
tests pass without CRAFT_I18N_DIR at 48ed6f8. English dialects, regions' English and Hyphenation
Options still need the variant table and patterns, so they keep the skip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:12 +00:00
Martin BallandClaude Opus 5.5 452d38da0b CP1.3: pin craft-proof 48ed6f8
storytold/craft-libs#482's new head: without Language data English spells from the built-in
Moby list plus en-extra with edit-1 suggestions, and a system English Hunspell loads leniently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:12 +00:00
Martin BallandClaude Opus 5.5 3181e502c5 CP1.3: review — the Printing workflow checks out craft-i18n before its app and UI tests, as the gate does
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:12 +00:00
Martin BallandClaude Opus 5.5 8c3a792b49 CP1.3: review — engine tests that need the Language data skip with a note naming CRAFT_I18N_DIR when none is embedded, as craft-proof's do
cargo test -p wordcraft-engine now passes without CRAFT_I18N_DIR (10 tests return early) and
still runs them all with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:12 +00:00
Martin BallandClaude Opus 5.5 c967f934ca CP1.3: review — gaps and localization-parity point at craft-proof (craft-libs) and craft-i18n language-data instead of the removed crates/proof and assets/spelling
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:12 +00:00
Martin BallandClaude Opus 5.5 d6cb183e6d CP1.3: craft-proof pinned by git to storytold/craft-libs#482's head (961162a) instead of a local path
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:03 +00:00
Martin BallandClaude Opus 5.5 449d2c2392 CP1.3: craft-i18n pinned to the merge of its Language data review fixes (storytold/craft-i18n#107, 062a637)
craft-proof reads hyphenation patterns as language-data/<locale>/hyphenation.pat,
which #107 introduced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:03 +00:00
Martin BallandClaude Opus 5.5 4ea6fea512 CP1.3: craft-i18n pinned to its Language data merge (storytold/craft-i18n#106, 5e78e5b)
craft-i18n.rev and the flatpak source move from the feat/cp1-language-data
commit to the merge on craft-i18n main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:03 +00:00
ccc8780b5e CP1.3: spelling-dictionaries lists WordCraft's own folder on macOS and Linux
craft-proof now searches it there too (ported from storytold/craft-libs#31).

Co-Authored-By: Michał Marcin Brzuchalski <michal.brzuchalski@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:03 +00:00
Martin BallandClaude Opus 5.5 3110dcf02b CP1.3: proofing comes from craft-proof (craft-libs), its data from craft-i18n
- wordcraft-proof is now `package = "craft-proof"` from craft-libs (local
  path for now; TODO in Cargo.toml for the git+rev pin after the craft-libs
  PR merges). `wordcraft_proof::` paths are unchanged.
- crates/proof and assets/{spelling,hyphenation} deleted with their
  ATTRIBUTION.md rows; one row now points at craft-i18n's Language data;
  NOTICE paths updated.
- craft-i18n.rev (and the flatpak source) moved to the craft-i18n commit that
  adds language-data/ (CI sets CRAFT_I18N_REQUIRED=1, so the pin must have it).
- xtask layers: proof removed. Engine tests carry their own copy of the
  synthetic Czech fixture (they reached into crates/proof/src).
- Docs: architecture.md, CLAUDE.md/AGENTS.md layer table, README build note
  (CRAFT_I18N_DIR), localization-tr path, ROADMAP progress log.

Before/after wordcraft-cli on a 17-language proofing document: issues,
suggestions, manual hyphenation, PDF text and PDF bytes identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FypgCX8LsYYE8fsj4byBm2
2026-11-01 20:13:03 +00:00
fd7f6621a1 Markdown export: a list inside a block quote stays in the quote (#5156)
* Markdown export: a list inside a block quote stays in the quote

`> - one / > - two` was imported as quoted list paragraphs (the quote kind on each item) but
the list branch of the exporter ignored that kind and wrote `- one / - two`, so every save as
Markdown moved the list out of its quote. A quoted list item now takes `> ` on each of its
lines, nested levels and continuation lines included. The same paragraphs come back from a
.docx (Quote style with numbering), so the quote now survives that round trip too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Markdown export: a list after a quoted list stays outside the quote

* fmt

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:48:32 -05:00
8a191fbf11 Shapes: arrowheads keep their kind and size and are drawn at Word's sizes (#5192)
Word draws an arrowhead 2, 3 or 5 times the line's width long and wide
(small, medium, large; `a:headEnd`/`a:tailEnd` `len`/`w`), a line under
2 points as a 2-point one. Every arrowhead was drawn one size and saved
as a medium triangle: the kind, width and length are now read (DrawingML
and VML), kept and written back, and the size is drawn as Word does.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:46:16 -05:00
499fc6289e Columns: the line between them is drawn for selected text, as tall as the section's text (#5178) (#5183)
* #5178: the line between columns is drawn for a section starting mid-page

Columns for selected text are a continuous section in the middle of a page, and
the line between columns was only added when a page started, down to the page
bottom. The lines are now drawn per section on each page, from the top of its
text there to the bottom of its tallest column, when the page or the section ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #5178: the line between columns spans the text's lines, not its paragraph spacing

Review: the line started above the section's first line (its space before) and
ended below its last (its space after). It now runs from the highest text line or
table cell of the section on the page to the lowest, falling back on the flow's
extent when there are none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #5178: the column lines find the section's text by place, and draw nothing without it

Review: the section's items were the page's items after an index, which objects
behind the text (inserted at the front) shift; the flow's extent drew a line
where the section had only spacing; overflow lines of exact-height cells counted.
The text is now the page's top-level body lines and table cells below the
section's top; lines in cells count by their cell's box; no text, no line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #5178: the column lines take the section's text by its blocks

Review: a floating table of an earlier section placed lower on the page would
pass a filter by place. The section's text is now the page's items of top-level
blocks from the section's first block on the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:39:35 -05:00
935aaf7147 Layout tests: compare pairs of layouts drawn with the same faces (#5167)
The numeral and automatic-space tests compare two layouts of one text. Tests
running alongside share the font database, and one loading a font between the
two layouts can change the face a character falls back to: U+06DD drew in
Segoe UI in one layout and Noto Sans Arabic in the other on Windows, failing
arabic_mode_keeps_arabic_indic_digits_as_typed and
context_mode_follows_run_direction now and then, and
spaces_punctuation_hangul_and_latin_alone_get_nothing failed the same way. The
pairs are now laid out again until both use the same faces.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:36:14 -05:00
1bd71f72bc .doc import: the default font is the stylesheet's standard font, so Word 365 files open in Aptos (#5066) (#5069)
* #5066: .doc default font is the stylesheet's standard font

The .doc reader took the default character font from the font table's
first entry. A Word 365 .doc keeps Times New Roman there and names Aptos as
the stylesheet's standard font (Stshif.rgftcStandardChpStsh), so its text
came out in Times New Roman. The defaults are now the standard ASCII, East
Asian and complex-script (ftcBi) fonts, else entry 0 as before; runs'
sprmCRgFtc1 and sprmCFtcBi are read too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #5066: the complex-script standard font is read only when the header has it

ftcBi is optional in the Stshi, and the latent style data that follows it starts
with cbLSD (4): a header that leaves ftcBi out no longer makes cbLSD the
complex-script font.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #5066: a header ending with ftcBi = 4 keeps it

Review: a 4 at ftcBi's offset was taken for cbLSD even when the header ends
there, so a complex-script standard font of index 4 was lost. It is read as
left out only when more data follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:24:47 -05:00
Brandon ThomasandClaude Opus 5.5 bf80a31d10 docs: merge the Tables row duplicated by the #5145/#5161 doc merge
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 14:23:38 -05:00
a64d9748f2 Tables: a row isn't split to leave a picture cell empty beside its text; it moves on whole, as in Word (#4272) (#5161)
When a row's first part on a page that already holds something would leave
a cell empty because that cell's inline picture (one line, never split) or
first line runs past the room, the row moves on whole instead of leaving the
other cells' text beside the empty cell. An at-least row grown taller than its
minimum by its picture used to pass the #1742 check (room >= declared minimum)
and then split, the picture going on alone. At the top of a page such a row
still splits.

Co-authored-by: Harrison Smith <harrison@hsmith.dev>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:23:01 -05:00
c395760959 Tables: a middle or bottom cell of a row split across pages lines up on the first page (#5145)
Measured in Word for Windows: when a table row breaks across pages, a cell
aligned to the middle or bottom whose text all fits in the part of the row
left on the page is aligned within that part, as tall as the other cells'
text there (with its space after), and a cell whose text doesn't fit runs
from the top of the cell with the rest on the next page. WordCraft kept the
offset the whole row's height gave the cell, so a short centred label
beside a long cell went to the top of the next page and left its cell empty
on the first, and a long centred cell started far down the page.

The vertical offset each cell's content got is kept with the row, so a split
moves the content back to the top before it decides what fits.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:21:25 -05:00
6ab1023ed6 Layout: tab leader dots spaced and placed as in Word (#5144)
* Layout: tab leader dots spaced and placed as in Word

Measured in Word for Windows with six fonts at four sizes: Word spaces leader
dots by the dot's advance at the font size in whole 600-dpi pixels (the size
itself in whole pixels), so a TH SarabunPSK dot at 16 pt (2.59 pt) is drawn
every 2.64 pt and a Tahoma one at 22 pt every 6.6 pt. The dots sit on the
page's grid of that pitch, from the first place after the text before the
tab to the last one that ends before the text after it. WordCraft drew them
at the unrounded advance, half a dot after the text and 1.2 dots short of
the number, so a table of contents line had a dot or two more than Word's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Layout: each leader dot's place comes from its grid index

The leader's dots were placed by adding the pitch to the last place, so on
a long leader the rounding of each addition added up. From 72 to 528 pt at
2.64 pt it came to more than the 0.001 pt allowed at the end, and the last
dot, which ends exactly on 528 pt, was left out. `leader_xs` now works out
each place as its grid index times the pitch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Layout: name the leader slack and glyph limit

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Layout: leader dots use the font size rounded to whole pixels

Word rounds the font size to the nearest 600-dpi pixel before it rounds the
dot's advance (Arial 14 pt: 116.7 px, so 117, dots every 3.96 pt); taking the
floor drew them 3.84 pt apart, as measured at 11, 14, 17, 20 and 26 pt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* leader_xs: saturate the glyph range end

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:19:41 -05:00
639b2342c2 Lists: a level above counts its first number at the first deeper item (#5108)
* Lists: a level above counts its first number at the first deeper item

Word counts a list level that hasn't been used yet as started at its first
number when a deeper item comes first: after 10.1-10.4 with no level-1 item
before them, the next level-1 item is 11. WordCraft showed 10 in the deeper
labels but didn't count it, so the next level-1 item repeated it ("10.
Qualifications", then "10. Penalties" in a tender document whose sub-items
come first).

Counters::next_label now starts each unstarted level above the one it counts,
at that num's start override if it has one, else the level's start. Measured
in Word with ListFormat.ListString: 1.1, 1.2, 2., 3.; (1), (2), 2. when the
deeper text doesn't show level 1; 1.1.1., 1.2., 2. from level 3.
a_list_s_own_level_definitions_win expected the old count; its level 0 item
after level 1 items is now 2., as in Word.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Lists: record the counting fix in the progress log and gaps

AGENTS.md asks for the progress log and the gap list when work lands. The
ROADMAP progress log has the entry; gaps.md records that no ranked gap
changed. Each with its status line and revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Lists: test a num's start override for the level above

The deeper-item-first test now also gives the num a start override for
level 0 (10) under level-1 items: 10.1., 10.2., then 11. It fails if the
override is ignored for the level above (1.1., 1.2., 2.).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test a start override of 5 and a restart after a level-1 item

Review: 5.1., 6., 6.1. with the num's start override, and level 2 restarting below the next level-1 item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:18:14 -05:00
VoiceLessQandBrandon Thomas 4a3586c686 Paragraph side borders lie outside the text (#5036) (#5088)
* #5036: paragraph side borders lie beyond their distance from the text

push_para centred the left and right rules on the distance, so a 3 pt border
at no distance reached 1.5 pt into the text and, drawn after the lines, covered
the first letter. In a heading whose border matches its shading the letter
looked cropped, on the page and in the ribbon's style tiles. All four lines now
lie outside the distance, as the top and bottom ones did. Top and bottom reach
the corners and the shading fills out to the outer edges, so no seam shows
where rules and shading meet.

* #5036: pin the right border on its own in the side borders test

The test only compared the right border with the end of the top rule, and both come from the same value, so a wrong right edge still passed. Check it against the line's own bound.

* docs: progress lines for #5088

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:14:59 -05:00
ba9c4000ed Keep repeated spaces in LaTeX export (#5051)
Found while testing crates/formats/src/latex/export.rs:411-444 (escape_text, `c => o.push(c)` for ' ')

Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:13:08 -05:00
Fin WarmanandBrandon Thomas e5f2885ecc Use the body boundary for header margin anchors (#2485)
* T280: anchor header drawings below tall headers

* #2485: share rendered header boundary with pins and obstacles

* docs: layout-parity status line for #2485

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
2026-11-01 14:11:56 -05:00
totoz64 8686fa45ec M7.REVIEW: use normal shared CI for the Proofing pane
Remove the feature-only preview workflow from this feature PR. Retain all Rust implementation and regressions unchanged; document direct manual rendering and the exact-source historical previews.

Credit: @totoz64
2026-11-01 12:53:38 +01:00
totoz64 3abe452b7c M7.SYNC: reconcile Proofing pane with current main
Preserve upstream changes and keep shared documentation headers aligned with main; retain feature status in its dedicated document.

Credit: @totoz64
2026-10-31 17:32:49 +01:00
totoz64 7c20316dd5 M7.SYNC: refresh Proofing pane against main 0.13.2 2026-10-31 09:42:33 +01:00
totoz64 6e96b5470f PROOF.API: preserve explicit-position suggestion lookup 2026-10-30 10:44:42 +01:00
totoz64 c077c155e4 PROOF.UI: refresh Proofing pane against main 2026-10-30 10:20:06 +01:00
totoz64 55dce8eebf ci: avoid persisting proofing-preview checkout credentials
Set persist-credentials: false for preview tests and join the two newly added documentation blockquotes as requested by review. No Rust source change.

Validation: independently reviewed exact workflow and Markdown diffs and tree hashes. Cargo fmt passed on the unchanged Rust source; cargo xtask ci was attempted but blocked by missing local dlltool.exe. Fresh hosted checks are required for this revision.
2026-10-27 17:13:05 +01:00
totoz64 e490c13cdb M6: sync Proofing pane with main module extractions
Merge main 436d444ef0. Port proofing selection/navigation and app/control/pane hooks to extracted modules, retain per-window state and stale-target guards, preserve upstream menus, and extend preview path filters. Keep the original pane implementation and targeted tests, with explicit imports after test relocation.

Workspace formatting passes. Local cargo xtask ci is blocked during Clippy by missing dlltool.exe in the Windows GNU toolchain; no local test success is claimed. Hosted navigation, UI, rendering and full checks on this tree remain required.
2026-10-27 16:57:57 +01:00
totoz64 0cd06c786c M7: normalize regression module spacing
Remove the extra blank line before the new navigation regression module's closing brace. No behavior change.

Local cargo xtask ci was attempted but cargo is absent; git diff --check passes. Hosted checks must validate this new SHA.
2026-10-26 16:57:22 +01:00
totoz64 62f9050848 M7: resolve exact proofing selections and address pane review
Give the exact selected issue priority over overlapping grammar when querying the focus, including reversed selections. Keep point lookup for explicit other positions. Add command and pane regressions for overlapping repeated misspellings, correction and undo.

Include command/window/document/canvas integration paths in preview workflow triggers. Preserve historical three-platform evidence while recording f126's successful gate/previews/Windows/Linux and outstanding macOS validation after pre-run cancellations.

These address the three CodeRabbit findings on official #4471. Local cargo xtask ci was attempted but cargo is absent; git diff --check passes. New hosted checks must validate this SHA before any claim of success.
2026-10-26 16:53:47 +01:00
totoz64 f1264f3047 M7: synchronize Proofing pane with main's test import repair
Upstream #4401 merged the baseline Page/Rgb import fix. Keep main's exact tests_linear.rs and merge its current head, removing the equivalent temporary import patch from the final official diff. No proofing/UI behavior changes.

The last fully validated pane revision is 056b18b; the subsequent Read Aloud alignment and this merge need hosted checks and new PNG inspection. Local cargo xtask ci was attempted but cargo is absent; git diff --check passes.
2026-10-26 15:51:02 +01:00
totoz64 057350ef6e M7: keep the proofing Read Aloud control at the context edge
The 056b18b preview confirms full-width context and visible actions, but the text child still shrinks to its contents and pulls the Read Aloud control inward. Keep that child at its allocated width so the control aligns at the right edge as observed in Word.

All five workflows passed at parent 056b18b; both light/dark artifacts were downloaded and inspected. This alignment revision needs new hosted checks and preview inspection. Local cargo xtask ci was attempted but cargo is absent; git diff --check passes.
2026-10-26 15:48:32 +01:00
totoz64 056b18b714 M7: match proofing pane geometry after rendered comparison
Inspect the successful 20205a5 light/dark artifacts against the observed Word arrangement. Fill the sentence frame across the pane and bound the suggestion list to roughly three cards, retaining all results through scrolling and keeping ignore/dictionary actions below the list visible.

The direct-correction/undo regression now clicks the first visible engine suggestion rather than relying on an offscreen fourth card. Update comparison/progress records. Normalize the baseline test file's line endings so its final diff contains only the two missing imports.

Parent 20205a5 passed full CI, three-platform Printing/UI checks, targeted proofing tests and preview generation. This layout revision needs new hosted validation and inspected PNGs. Local cargo xtask ci was attempted but cargo is absent; git diff --check passes.
2026-10-26 12:57:39 +01:00
totoz64 20205a5051 M7: repair baseline test imports and publish proofing previews
The full gate found missing Page/Rgb imports in main's tests_linear.rs (also tracked by upstream #4401). Add the imports already validated with the French contribution. No layout behavior changes.

The preview run rendered both images and passed all six correction tests, but upload searched the workspace root. Cargo tests run in the crate directory; collect the PNGs from crates/ui-egui instead.

At parent 574cef, Printing passed on Linux, macOS and Windows, including UI tests and browser compatibility. Local cargo xtask ci was attempted but cargo is absent; git diff --check passes. This revision needs new hosted validation.
2026-10-26 11:11:10 +01:00
totoz64 574cef34c9 M7: use current egui accessibility roles for proofing controls
Replace two deprecated WidgetType aliases with Role::Button, as required by the hosted Windows and macOS Clippy checks.

Local cargo xtask ci was attempted but cargo is not installed on this machine. git diff --check passes. Hosted CI and the preview workflow must validate this revision.
2026-10-26 10:27:57 +01:00
totoz64 9e4adaf320 M7: apply the hosted rustfmt corrections to the proofing pane
Apply the six formatting hunks reported by the exact-head Printing/CI runs. No behavior change. cargo xtask ci was attempted locally but cargo is absent; hosted checks must validate this revision.
2026-10-26 10:21:10 +01:00
totoz64 f6f9d65b0e M7: follow the observed Word for Windows proofing pane
Observe F7 in the installed Word with synthetic text, then replace the central correction form with a right-side task pane. Show sentence context, directly clickable suggestion cards and ignore/dictionary links. Use the existing pane host, palette and original icons; copy no Microsoft assets.

Keep per-window state and clear targets on document switch. Adapt UI tests, pseudo-localization exemptions and light/dark previews. Record observed measurements and remaining feature differences. Local cargo xtask ci was attempted but cargo is absent; exact-head hosted checks and visual comparison remain required.
2026-10-26 10:16:49 +01:00
totoz64 b2787bf36e M7: add a persistent language-aware spelling review dialog
Keep compact context menus, but show a correction form for user spelling/editor commands. Advance without wrapping after change, ignore or dictionary actions; reject stale targets and retain programmatic command results.

Add navigation/UI regressions and light/dark preview CI. cargo xtask ci was attempted on the preparation PC but cargo is absent; hosted validation is required before integration.
2026-10-26 10:00:13 +01:00
217 changed files with 8739 additions and 60501 deletions
+22
View File
@@ -6,6 +6,7 @@ on:
pull_request:
paths:
- 'Cargo*'
- 'craft-i18n.rev'
- 'apps/wordcraft/**'
- 'apps/wordcraft-web/**'
- 'crates/ui-egui/**'
@@ -14,6 +15,7 @@ on:
branches: [main]
paths:
- 'Cargo*'
- 'craft-i18n.rev'
- 'apps/wordcraft/**'
- 'apps/wordcraft-web/**'
- 'crates/ui-egui/**'
@@ -41,6 +43,26 @@ jobs:
CARGO_INCREMENTAL: '0'
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
# The interface catalogs and craft-proof's Language data come from craft-i18n at the commit in
# craft-i18n.rev, so these tests build with the same data as the gate (ci.yml).
- name: Check out craft-i18n
uses: storytold/craft-common-actions/actions/checkout-craft-i18n@main
# The catalogs include CJK interface languages, so the UI tests need the suite's fonts, as in
# the gate (same commit as ci.yml and release.yml).
- name: Check out craft-fonts
uses: storytold/craft-common-actions/actions/checkout-craft-fonts@main
with:
ref: b076417b5880bd4154bd22f00695b1d28c6bea83
- name: Build with craft-fonts
shell: bash
run: |
echo "CRAFT_FONTS_DIR=$GITHUB_WORKSPACE/craft-fonts" >> "$GITHUB_ENV"
echo "CRAFT_FONTS_REQUIRED=1" >> "$GITHUB_ENV"
- name: Install Linux build and software-rendering dependencies
if: runner.os == 'Linux'
+3 -2
View File
@@ -20,7 +20,7 @@ People trust WordCraft with their writing; a crash loses their work. **This outr
- Never copy, trace, screenshot-crop or recreate icons, images, templates, themes, clip art, fonts, cursors, sounds or any other asset from Microsoft Word/Office or any Microsoft, Adobe, Avid or Autodesk product. Not even "temporarily".
- Every asset must be **original** (drawn in code — prefer this: `crates/ui-egui/src/icons.rs`), **public domain / CC0**, **OSI open source**, or **Creative Commons that allows redistribution**, or licensed open source by a contributor who made it.
- **Every asset file has a row in [`ATTRIBUTION.md`](ATTRIBUTION.md)** (author, source, licence). `cargo xtask assets` (part of `cargo xtask ci`) fails otherwise. Add the row in the same commit as the file.
- Fonts: new font files go to [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts), never this repo; it's an optional build input (`CRAFT_FONTS_DIR`). Documents asking for proprietary fonts (Calibri, Aptos, Cambria…) are rendered with installed system fonts or metric-compatible open substitutes (`crates/fonts/src/word.rs`) — never bundle proprietary fonts.
- Fonts: new font files go to [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts), never this repo; it's an optional build input (`CRAFT_FONTS_DIR`). Spelling word lists and hyphenation patterns live in [`storytold/craft-i18n`](https://github.com/storytold/craft-i18n) `language-data/`, never this repo; they are embedded from `CRAFT_I18N_DIR` (pinned by `craft-i18n.rev`, which CI checks out). Without it the app builds, but English spelling falls back to an installed Hunspell English or the built-in Moby word list (with no American/British variant table) and pattern hyphenation is off (Bulgarian, Turkish and Russian still hyphenate by rule), so proofing tests need it. Documents asking for proprietary fonts (Calibri, Aptos, Cambria…) are rendered with installed system fonts or metric-compatible open substitutes (`crates/fonts/src/word.rs`) — never bundle proprietary fonts.
- Templates, sample documents and themes are our own text and colours (`crates/engine/src/sample.rs`, `cmd/design.rs`). Demo images are public domain, with sources recorded.
## Clean room
@@ -34,11 +34,12 @@ People trust WordCraft with their writing; a crash loses their work. **This outr
| L0 | `geom` | units, rects, measurement parsing |
| L1 | `doc`, `fonts` | document model + edit primitives; font DB, shaping, Word font substitution |
| L2 | `layout`, `docx`, `formats` | line breaking, pagination, tables, headers/footers, hit testing; OOXML I/O; TXT/MD/HTML/RTF/ODT |
| L3 | `render`, `pdf`, `proof` | vello_cpu rasteriser; krilla PDF; proofing |
| L3 | `render`, `pdf` | vello_cpu rasteriser; krilla PDF |
| L4 | `engine` | Session, undo, **command registry**, Word feature catalog, samples, I/O dispatch |
| L5 | `mcp` | MCP server (headless or bridged to the app) |
| L6 | `ui-egui` | Word-style egui front end (swappable) |
| apps | `apps/wordcraft`, `apps/wordcraft-cli`, `apps/wordcraft-web` | binaries |
| external | `craft-proof` (craft-libs, used as `wordcraft-proof`, sits at L1) | proofing: spelling, grammar, hyphenation; its word lists and patterns are craft-i18n's `language-data/` (`CRAFT_I18N_DIR`) |
- **Everything is a command.** User-visible behaviour = a `CommandSpec` in `crates/engine/src/cmd/*.rs` (id, label, ribbon location, shortcut, params doc, `enabled`, `run`) + tests. UI-only commands (`ui.*`) live in `crates/ui-egui/src/lib.rs`. The ribbon, shortcuts, command search, CLI, control channel and MCP all dispatch by id. Programmatic calls never open dialogs.
- **Parity is measured:** `crates/engine/src/catalog.rs` lists Word's ribbon/menu features with command ids. `cargo xtask parity` writes `docs/parity-checklist.md`; a test enforces a floor that only rises. Implement missing ids to raise it.
+1 -9
View File
@@ -10,15 +10,6 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra
|---|---|---|---|---|
| `egui` dependency: `epaint_default_fonts/fonts/egui-icons.ttf` | John Slegers (emoji-icon-font) | https://github.com/storytold/egui/blob/173cc7375d35dc5fbc5b5d0994fdf787f547c5d1/crates/epaint_default_fonts/fonts/egui-icons.txt; original https://github.com/jslegers/emoji-icon-font | MIT (full copyright and permission text in NOTICE) | Small egui special-icon subset; embedded by default_fonts, retained in FontDefinitions::default() |
| `egui` dependency: `epaint_default_fonts/fonts/emoji-icon-font.ttf` | John Slegers | https://github.com/storytold/egui/blob/173cc7375d35dc5fbc5b5d0994fdf787f547c5d1/crates/epaint_default_fonts/fonts/emoji-icon-font-mit-license.txt; original https://github.com/jslegers/emoji-icon-font | MIT (full copyright and permission text in NOTICE) | Parent monochrome icon/emoji face embedded by monochrome_emoji_fonts; separate from the egui-icons subset |
| `assets/hyphenation/en-us.pat` | DesignCraft contributors (generated) | trained from the Moby list by DesignCraft's patgen | Public Domain (CC0-1.0) | Our own Liang hyphenation patterns (no TeX patterns) |
| `assets/hyphenation/en-us.dic` | Grady Ward (Moby Hyphenator II word list); converted by DesignCraft contributors | https://www.gutenberg.org/ebooks/3204 (file `mhyph.txt`, SHA-256 `eeb30474c86b8af3469035ec1a0913e35905325ca885290db9dfed9881e230ac`), via DesignCraft | Public Domain | ~165k words with break points; drives spelling and hyphenation |
| `assets/spelling/en-extra.txt` | WordCraft contributors | original word list written for this repository (common modern words the Moby list lacks); no other dictionary copied | MIT OR Apache-2.0 | ~240 lowercase English base forms supplementing spelling (#520), including British spellings (#40) |
| `assets/spelling/en-variants.txt` | WordCraft contributors | original list written for this repository from general knowledge of English spelling; no other dictionary or variant list copied | MIT OR Apache-2.0 | ~280 American/British spelling pairs (-or/-our, -er/-re, -ize/-ise, -yze/-yse, -ense/-ence, doubled l, ae/oe…) with a policy per English: en-US, en-CA and the British-spelling Englishes (#1281, #1045) |
| `assets/spelling/ru-extra.txt` | WordCraft contributors | original word list written for this repository (common modern Russian words older Hunspell dictionaries lack, with their declension classes); no other dictionary copied | MIT OR Apache-2.0 | ~200 Russian base forms supplementing spelling; their forms are generated by `crates/proof/src/ru/extra.rs` |
| `assets/hyphenation/pl.pat` | Hanna Kołodziejska, Bogusław Jackowski, Marek Ryćko (Polish TeX hyphenation patterns, 1987–1995); hyph-utf8 package | https://github.com/hyphenation/tex-hyphen (`hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-pl.tex`, SHA-256 `83ddbbd21bc06a44bc4f4a4c49c38782e809cff5f026b764c044daa29e818843`), bundled unmodified | MIT (the file is offered under MIT or Knuth's public-domain terms; WordCraft uses MIT, whose notice is in the file's header) | 4,053 Liang patterns and 20 exception words; Polish hyphenation |
| `assets/hyphenation/pt.pat` | Pedro J. de Rezende, J. Joao Dias Almeida, Leonardo Araujo, Aline Benevides (Portuguese TeX hyphenation patterns, 1987–2024); hyph-utf8 package | https://github.com/hyphenation/tex-hyphen (`hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-pt.tex`, SHA-256 `c110a39e501db372d1d755e1c64464321a2efa7d8e9617f66001000389b941f2`), bundled unmodified | BSD 3-clause (the file's licence header names it and carries the notice) | 426 Liang patterns and 2 exception words; Portuguese hyphenation |
| `assets/hyphenation/it.pat` | Claudio Beccari (Italian TeX hyphenation patterns, 2008–2014); hyph-utf8 package | https://github.com/hyphenation/tex-hyphen (`hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-it.tex`, SHA-256 `6ce56ed6ed2ca7c0688838e366e6595e0429f984f4fc6e96ec045cebb1e76a47`), bundled unmodified | MIT (the file is offered under LPPL 1.3+ or MIT; WordCraft uses MIT, whose notice is in the file's header) | 355 Liang patterns, no exception words; Italian hyphenation |
| `assets/hyphenation/de.pat` | Deutschsprachige Trennmustermannschaft: Stephan Hennig, Werner Lemberg, Günter Milde, Sander van Geloven, Georg Pfeiffer, Gisbert W. Selke, Tobias Wendorf, Keno Wehr (German TeX hyphenation patterns, reformed spelling, 2013–2024); hyph-utf8 package | https://github.com/hyphenation/tex-hyphen (`hyph-utf8/tex/generic/hyph-utf8/patterns/tex/hyph-de-1996.tex`, SHA-256 `374ad1ce3263f8a2791ec070ef9b516c532dae64456f3fb656e617e8ae53a9d3`), bundled unmodified | MIT (the notice is in the file's header) | 36,709 Liang patterns, no exception words; German hyphenation (de-DE, de-AT, de-CH) |
| `assets/app-icon/wordcraft-1024.png` | WordCraft contributors | original artwork drawn in SVG (`assets/app-icon/wordcraft.svg`), rendered by `packaging/icons.sh` | MIT OR Apache-2.0 | Placeholder app icon until the WordCraft mascot drawing lands |
| `packaging/macos/dmg/background.svg` | @XusBadia | original work for this repository, derived from the WordCraft app icon (`assets/app-icon/wordcraft.svg`, referenced via `<image>`, not copied); text is outlined Inter / JetBrains Mono glyph paths | MIT OR Apache-2.0, like the code ([`LICENSE-MIT`](LICENSE-MIT), [`LICENSE-APACHE`](LICENSE-APACHE)) | WordCraft macOS DMG window background (2026-10-08); used for the Finder window of the macOS DMG, rendered to `background.tiff` by `packaging/macos/dmg/generate.py` |
| `packaging/macos/dmg/background.tiff` | @XusBadia | Rendered from `packaging/macos/dmg/background.svg` by `packaging/macos/dmg/generate.py` | MIT OR Apache-2.0, like the code ([`LICENSE-MIT`](LICENSE-MIT), [`LICENSE-APACHE`](LICENSE-APACHE)) | 1x + 2x HiDPI TIFF for the macOS DMG window; includes the app icon (see its row) and LittleCMS's built-in sRGB profile ("No copyright, use freely") |
@@ -70,3 +61,4 @@ Generated-in-code art is original and has no file to list: the UI icon set (`cra
| `docs/brand/artcraft-mark.svg` | ArtCraft Team | ArtCraft brand kit (project owner) | ArtCraft trademark, `docs/brand/LICENSE-brand.txt` | Not open source; usable unmodified only |
| craft-fonts (optional build input, not files in this repo): BIZ UDPGothic, BIZ UDMincho, Shippori Mincho, Noto Sans CJK SC, Noto Sans Arabic | see https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md | embedded only when built with `CRAFT_FONTS_DIR` (all official releases) | OFL-1.1 | Japanese and Chinese UI, Japanese document fallback |
| craft-i18n (optional build input, not files in this repo): the interface translations | see https://github.com/storytold/craft-i18n/blob/main/ATTRIBUTION.md, which credits every translator of WordCraft's former in-repo catalogs (`crates/ui-egui/src/i18n/*.tsv`, migrated at `wordcraft@806a945`) | embedded only when built with `CRAFT_I18N_DIR`, at the commit in `craft-i18n.rev` (all official releases) | MIT OR Apache-2.0 | Interface languages other than English; clean-room translations from the English interface labels |
| craft-i18n Language data (optional build input, not files in this repo): the spelling word lists and hyphenation patterns (`language-data/`: the Moby Hyphenator II list and our patterns for English, `en-extra.txt`, `en-variants.txt`, `ru-extra.txt`, and the hyph-utf8 German, Polish, Portuguese and Italian patterns) | see the Language data section of https://github.com/storytold/craft-i18n/blob/main/ATTRIBUTION.md (authors, upstream sources with SHA-256, licences); moved there from `assets/hyphenation/` and `assets/spelling/` byte for byte | embedded by craft-proof (storytold/craft-libs) only when built with `CRAFT_I18N_DIR`, at the commit in `craft-i18n.rev` (all official releases) | Public Domain / CC0-1.0, MIT, BSD-3-Clause, MIT OR Apache-2.0 (per file; see NOTICE) | Spelling and hyphenation; without it English spelling falls back to an installed Hunspell English or craft-proof's built-in Moby list, and pattern hyphenation is off |
+3 -2
View File
@@ -20,7 +20,7 @@ People trust WordCraft with their writing; a crash loses their work. **This outr
- Never copy, trace, screenshot-crop or recreate icons, images, templates, themes, clip art, fonts, cursors, sounds or any other asset from Microsoft Word/Office or any Microsoft, Adobe, Avid or Autodesk product. Not even "temporarily".
- Every asset must be **original** (drawn in code — prefer this: `crates/ui-egui/src/icons.rs`), **public domain / CC0**, **OSI open source**, or **Creative Commons that allows redistribution**, or licensed open source by a contributor who made it.
- **Every asset file has a row in [`ATTRIBUTION.md`](ATTRIBUTION.md)** (author, source, licence). `cargo xtask assets` (part of `cargo xtask ci`) fails otherwise. Add the row in the same commit as the file.
- Fonts: new font files go to [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts), never this repo; it's an optional build input (`CRAFT_FONTS_DIR`). Documents asking for proprietary fonts (Calibri, Aptos, Cambria…) are rendered with installed system fonts or metric-compatible open substitutes (`crates/fonts/src/word.rs`) — never bundle proprietary fonts.
- Fonts: new font files go to [`storytold/craft-fonts`](https://github.com/storytold/craft-fonts), never this repo; it's an optional build input (`CRAFT_FONTS_DIR`). Spelling word lists and hyphenation patterns live in [`storytold/craft-i18n`](https://github.com/storytold/craft-i18n) `language-data/`, never this repo; they are embedded from `CRAFT_I18N_DIR` (pinned by `craft-i18n.rev`, which CI checks out). Without it the app builds, but English spelling falls back to an installed Hunspell English or the built-in Moby word list (with no American/British variant table) and pattern hyphenation is off (Bulgarian, Turkish and Russian still hyphenate by rule), so proofing tests need it. Documents asking for proprietary fonts (Calibri, Aptos, Cambria…) are rendered with installed system fonts or metric-compatible open substitutes (`crates/fonts/src/word.rs`) — never bundle proprietary fonts.
- Templates, sample documents and themes are our own text and colours (`crates/engine/src/sample.rs`, `cmd/design.rs`). Demo images are public domain, with sources recorded.
## Clean room
@@ -34,11 +34,12 @@ People trust WordCraft with their writing; a crash loses their work. **This outr
| L0 | `geom` | units, rects, measurement parsing |
| L1 | `doc`, `fonts` | document model + edit primitives; font DB, shaping, Word font substitution |
| L2 | `layout`, `docx`, `formats` | line breaking, pagination, tables, headers/footers, hit testing; OOXML I/O; TXT/MD/HTML/RTF/ODT |
| L3 | `render`, `pdf`, `proof` | vello_cpu rasteriser; krilla PDF; proofing |
| L3 | `render`, `pdf` | vello_cpu rasteriser; krilla PDF |
| L4 | `engine` | Session, undo, **command registry**, Word feature catalog, samples, I/O dispatch |
| L5 | `mcp` | MCP server (headless or bridged to the app) |
| L6 | `ui-egui` | Word-style egui front end (swappable) |
| apps | `apps/wordcraft`, `apps/wordcraft-cli`, `apps/wordcraft-web` | binaries |
| external | `craft-proof` (craft-libs, used as `wordcraft-proof`, sits at L1) | proofing: spelling, grammar, hyphenation; its word lists and patterns are craft-i18n's `language-data/` (`CRAFT_I18N_DIR`) |
- **Everything is a command.** User-visible behaviour = a `CommandSpec` in `crates/engine/src/cmd/*.rs` (id, label, ribbon location, shortcut, params doc, `enabled`, `run`) + tests. UI-only commands (`ui.*`) live in `crates/ui-egui/src/lib.rs`. The ribbon, shortcuts, command search, CLI, control channel and MCP all dispatch by id. Programmatic calls never open dialogs.
- **Parity is measured:** `crates/engine/src/catalog.rs` lists Word's ribbon/menu features with command ids. `cargo xtask parity` writes `docs/parity-checklist.md`; a test enforces a floor that only rises. Implement missing ids to raise it.
Generated
+139 -32
View File
@@ -1011,6 +1011,45 @@ dependencies = [
"libc",
]
[[package]]
name = "craft-atomic"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=762a06a4c6b48b522b7d4de84a87917ef90265e6#762a06a4c6b48b522b7d4de84a87917ef90265e6"
dependencies = [
"rustix 1.1.5",
]
[[package]]
name = "craft-font-catalog"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=762a06a4c6b48b522b7d4de84a87917ef90265e6#762a06a4c6b48b522b7d4de84a87917ef90265e6"
dependencies = [
"fontique",
"skrifa 0.47.0",
"thiserror 2.0.21",
"windows-registry",
]
[[package]]
name = "craft-font-store"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=762a06a4c6b48b522b7d4de84a87917ef90265e6#762a06a4c6b48b522b7d4de84a87917ef90265e6"
dependencies = [
"craft-atomic",
"craft-font-catalog",
"flate2",
"log",
"ruzstd",
"serde",
"serde_json",
"sha2 0.10.9",
"skrifa 0.47.0",
"tar",
"thiserror 2.0.21",
"toml",
"ureq",
]
[[package]]
name = "craft-fonts-build"
version = "0.1.0"
@@ -1061,6 +1100,17 @@ dependencies = [
"winsafe",
]
[[package]]
name = "craft-proof"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=89fa1559e94378ecef85cd7e4681400f4df053e0#89fa1559e94378ecef85cd7e4681400f4df053e0"
dependencies = [
"log",
"miniz_oxide 0.9.1",
"toml",
"unicode-normalization",
]
[[package]]
name = "craft-ui"
version = "0.1.0"
@@ -1801,6 +1851,28 @@ dependencies = [
"bytemuck",
]
[[package]]
name = "fontique"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6688bc1294fe7117d788937b6c53480169b29c566954af490830d4c09da9516a"
dependencies = [
"hashbrown 0.17.1",
"linebender_resource_handle",
"memmap2",
"objc2 0.6.5",
"objc2-core-foundation",
"objc2-core-text",
"objc2-foundation 0.3.2",
"parlance",
"read-fonts 0.41.0",
"roxmltree",
"smallvec",
"windows",
"windows-core",
"yeslogic-fontconfig-sys",
]
[[package]]
name = "foreign-types"
version = "0.5.0"
@@ -3334,6 +3406,16 @@ dependencies = [
"objc2-foundation 0.2.2",
]
[[package]]
name = "objc2-core-text"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0cde0dfb48d25d2b4862161a4d5fcc0e3c24367869ad306b0c9ec0073bfed92d"
dependencies = [
"bitflags 2.13.2",
"objc2-core-foundation",
]
[[package]]
name = "objc2-encode"
version = "4.1.0"
@@ -3605,6 +3687,12 @@ dependencies = [
"windows-link",
]
[[package]]
name = "parlance"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "73072a9470554c8e794778acece2312661218aaabb635600957d952cf01c716e"
[[package]]
name = "pdf-writer"
version = "0.15.0"
@@ -4516,6 +4604,12 @@ dependencies = [
"wait-timeout",
]
[[package]]
name = "ruzstd"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69e4ce4a039f42bf25920b95667e4d789e98749ab11bbb8d2c2936f788421bb5"
[[package]]
name = "ryu"
version = "1.0.23"
@@ -5723,6 +5817,7 @@ dependencies = [
"rustls-platform-verifier",
"ureq-proto",
"utf8-zero",
"webpki-roots",
]
[[package]]
@@ -6106,6 +6201,15 @@ dependencies = [
"rustls-pki-types",
]
[[package]]
name = "webpki-roots"
version = "1.0.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f5f280bbb2b287bc8e7de7e3852945a0ff50d1bbcb5ed42b6c7a568c546df171"
dependencies = [
"rustls-pki-types",
]
[[package]]
name = "weezl"
version = "0.1.12"
@@ -6648,12 +6752,13 @@ dependencies = [
[[package]]
name = "wordcraft"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"arboard",
"chrono",
"clipboard-win",
"craft-print",
"craft-proof",
"craft-update",
"dispatch2",
"eframe",
@@ -6676,14 +6781,13 @@ dependencies = [
"wordcraft-doc",
"wordcraft-engine",
"wordcraft-fonts",
"wordcraft-proof",
"wordcraft-ui-egui",
"zbus",
]
[[package]]
name = "wordcraft-cli"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"serde_json",
"sys-locale",
@@ -6696,11 +6800,11 @@ dependencies = [
[[package]]
name = "wordcraft-control-key"
version = "0.14.0"
version = "0.15.0"
[[package]]
name = "wordcraft-doc"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"base64 0.23.1",
"log",
@@ -6714,7 +6818,7 @@ dependencies = [
[[package]]
name = "wordcraft-docbin"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"cfb",
"chrono",
@@ -6731,7 +6835,7 @@ dependencies = [
[[package]]
name = "wordcraft-docx"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"aes",
"base64 0.23.1",
@@ -6755,10 +6859,13 @@ dependencies = [
[[package]]
name = "wordcraft-engine"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"cfb",
"chrono",
"craft-font-catalog",
"craft-font-store",
"craft-proof",
"encoding_rs",
"getrandom 0.3.4",
"image",
@@ -6778,7 +6885,6 @@ dependencies = [
"wordcraft-geom",
"wordcraft-layout",
"wordcraft-pdf",
"wordcraft-proof",
"wordcraft-render",
"zeroize",
"zip 3.0.0",
@@ -6786,7 +6892,7 @@ dependencies = [
[[package]]
name = "wordcraft-fonts"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"brotli-decompressor",
"harfrust 0.13.3",
@@ -6800,7 +6906,7 @@ dependencies = [
[[package]]
name = "wordcraft-formats"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"cfb",
"chardetng",
@@ -6819,7 +6925,7 @@ dependencies = [
[[package]]
name = "wordcraft-geom"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"image",
"serde",
@@ -6827,8 +6933,9 @@ dependencies = [
[[package]]
name = "wordcraft-layout"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"craft-proof",
"image",
"js-sys",
"kurbo",
@@ -6842,12 +6949,11 @@ dependencies = [
"wordcraft-fonts",
"wordcraft-geom",
"wordcraft-metafile",
"wordcraft-proof",
]
[[package]]
name = "wordcraft-mcp"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"log",
"serde_json",
@@ -6859,7 +6965,7 @@ dependencies = [
[[package]]
name = "wordcraft-metafile"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"kurbo",
"thiserror 2.0.21",
@@ -6867,7 +6973,7 @@ dependencies = [
[[package]]
name = "wordcraft-pdf"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"chrono",
"craft-pdf",
@@ -6886,19 +6992,9 @@ dependencies = [
"wordcraft-metafile",
]
[[package]]
name = "wordcraft-proof"
version = "0.14.0"
dependencies = [
"log",
"miniz_oxide 0.9.1",
"proptest",
"unicode-normalization",
]
[[package]]
name = "wordcraft-render"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"image",
"log",
@@ -6912,7 +7008,7 @@ dependencies = [
[[package]]
name = "wordcraft-ui-egui"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"cfb",
"chrono",
@@ -6943,7 +7039,7 @@ dependencies = [
[[package]]
name = "wordcraft-web"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"eframe",
"egui",
@@ -6962,7 +7058,7 @@ dependencies = [
[[package]]
name = "wordcraft-zotero"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"log",
"proptest",
@@ -7125,11 +7221,22 @@ checksum = "9440ea3e5aeabb0ac63af70daf835274065238cdd0cec83418f417eae38bacee"
[[package]]
name = "xtask"
version = "0.14.0"
version = "0.15.0"
dependencies = [
"serde_json",
]
[[package]]
name = "yeslogic-fontconfig-sys"
version = "6.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d8b8abf912b9a29ff112e1671c97c33636903d13a69712037190e6805af4f76"
dependencies = [
"dlib",
"once_cell",
"pkg-config",
]
[[package]]
name = "yoke"
version = "0.8.4"
+11 -2
View File
@@ -4,7 +4,7 @@ members = ["crates/*", "apps/*", "xtask"]
default-members = ["crates/*", "apps/wordcraft", "apps/wordcraft-cli", "xtask"]
[workspace.package]
version = "0.14.0"
version = "0.15.0"
edition = "2024"
license = "MIT OR Apache-2.0"
rust-version = "1.95"
@@ -21,7 +21,16 @@ wordcraft-docbin = { path = "crates/docbin" }
wordcraft-formats = { path = "crates/formats" }
wordcraft-render = { path = "crates/render" }
wordcraft-pdf = { path = "crates/pdf" }
wordcraft-proof = { path = "crates/proof" }
# Proofing is craft-libs' craft-proof (it was crates/proof); its Language data is craft-i18n's
# language-data/, embedded from CRAFT_I18N_DIR (craft-i18n.rev). The package keeps the
# `wordcraft-proof` name here so `wordcraft_proof::` paths are unchanged.
# craft-proof is the head of storytold/craft-libs#495 (review fixes), which contains #485 (CP4.2,
# Language packs, on #483 SCOWL English and #482); the font store's `language-pack` kind is #486
# (CP4.1). GitHub serves PR heads from storytold/craft-libs, and merged with merge commits they
# stay in its main.
wordcraft-proof = { package = "craft-proof", git = "https://github.com/storytold/craft-libs", rev = "89fa1559e94378ecef85cd7e4681400f4df053e0", version = "=0.1.0" }
craft-font-store = { git = "https://github.com/storytold/craft-libs", rev = "762a06a4c6b48b522b7d4de84a87917ef90265e6", version = "=0.1.0" }
craft-font-catalog = { git = "https://github.com/storytold/craft-libs", rev = "762a06a4c6b48b522b7d4de84a87917ef90265e6", version = "=0.1.0" }
wordcraft-engine = { path = "crates/engine" }
wordcraft-mcp = { path = "crates/mcp" }
wordcraft-zotero = { path = "crates/zotero" }
+70 -6
View File
@@ -20,13 +20,77 @@ list (author, source, license) is in ATTRIBUTION.md.
https://github.com/storytold/craft-fonts/blob/main/ATTRIBUTION.md (BIZ UDPGothic, BIZ UDMincho,
Shippori Mincho), under the SIL Open Font License 1.1. Their licence texts ship with the
packages as OFL-<family>.txt. Builds without CRAFT_FONTS_DIR do not include them.
- English word list (assets/hyphenation/en-us.dic) — derived from the Moby Hyphenator II list by
- Language data (craft-i18n): the word lists, dictionaries and hyphenation patterns below are embedded at build
time from https://github.com/storytold/craft-i18n (language-data/, CRAFT_I18N_DIR at the commit
in craft-i18n.rev; all official releases) by craft-proof (storytold/craft-libs). Builds without
CRAFT_I18N_DIR do not include them.
- English word list (craft-i18n language-data/en-us/en-us.dic) — derived from the Moby Hyphenator II list by
Grady Ward, public domain (Project Gutenberg eBook #3204). The hyphenation patterns
(assets/hyphenation/en-us.pat) were trained from it and are released as CC0-1.0.
- Polish hyphenation patterns (assets/hyphenation/pl.pat) — hyph-pl.tex from the hyph-utf8
(craft-i18n language-data/en-us/hyphenation.pat) were trained from it and are released as CC0-1.0.
- English spelling dictionaries (craft-i18n language-data/en-us, en-gb and en-ca: spelling.aff and
spelling.dic) — the SCOWL rel-2026.02.25 Hunspell dictionaries en_US, en_GB-ise and en_CA from
https://wordlist.aspell.net, bundled unmodified. Their upstream notice (embedded too, for the
About window) is language-data/en-us/LICENCE-SCOWL.txt in craft-i18n. The word lists:
Copyright 2000-2026 by Kevin Atkinson
Permission to use, copy, modify, distribute, and sell any part of SCOWLv2, or
word lists created from it, is hereby granted without fee, provided that the
above copyright notice appears in all copies and that both the above
copyright notice and this notice appear in supporting documentation. Kevin
Atkinson makes no representations about the suitability of this database for
any purpose. It is provided "as is" without express or implied warranty.
The Australian data:
Copyright 2016 by Benjamin Titze
Permission to use, copy, modify, distribute and sell this array, the
associated software, and its documentation for any purpose is hereby
granted without fee, provided that the above copyright notice appears
in all copies and that both that copyright notice and this permission
notice appear in supporting documentation. Benjamin Titze makes no
representations about the suitability of this array for any
purpose. It is provided "as is" without express or implied warranty.
The affix files, a modified version of Geoff Kuenning's Ispell english.aff:
Copyright 1993, Geoff Kuenning, Granada Hills, CA
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. All modifications to the source code must be clearly marked as
such. Binary redistributions based on modified source code
must be clearly marked as modified versions in the documentation
and/or other materials provided with the distribution.
(clause 4 removed with permission from Geoff Kuenning)
5. The name of Geoff Kuenning may not be used to endorse or promote
products derived from this software without specific prior
written permission.
THIS SOFTWARE IS PROVIDED BY GEOFF KUENNING AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL GEOFF KUENNING OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
- Polish hyphenation patterns (craft-i18n language-data/pl/hyphenation.pat) — hyph-pl.tex from the hyph-utf8
package, Copyright (C) 1987-1995 Hanna Kołodziejska, Bogusław Jackowski, Marek Ryćko, bundled
unmodified and used under the MIT License; the copyright and permission notice is in the file.
- Portuguese hyphenation patterns (assets/hyphenation/pt.pat) — hyph-pt.tex from the hyph-utf8
- Portuguese hyphenation patterns (craft-i18n language-data/pt/hyphenation.pat) — hyph-pt.tex from the hyph-utf8
package, bundled unmodified and used under the BSD 3-Clause License:
Copyright (C) 1987, 1994, 1996, 2015 Pedro J. de Rezende, 1996, 2015 J. Joao Dias Almeida,
@@ -52,7 +116,7 @@ list (author, source, license) is in ATTRIBUTION.md.
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE.
- German hyphenation patterns (assets/hyphenation/de.pat) — hyph-de-1996.tex (reformed spelling)
- German hyphenation patterns (craft-i18n language-data/de/hyphenation.pat) — hyph-de-1996.tex (reformed spelling)
from the hyph-utf8 package, bundled unmodified and used under the MIT License; the copyright
and permission notice is in the file:
@@ -74,7 +138,7 @@ list (author, source, license) is in ATTRIBUTION.md.
FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
- Italian hyphenation patterns (assets/hyphenation/it.pat) — hyph-it.tex from the hyph-utf8
- Italian hyphenation patterns (craft-i18n language-data/it/hyphenation.pat) — hyph-it.tex from the hyph-utf8
package, Copyright (C) 2008-2011 Claudio Beccari, bundled unmodified and used under the MIT
License (offered under LPPL 1.3+ or MIT); the copyright and permission notice is in the file:
+14 -1
View File
@@ -140,6 +140,19 @@ cargo run --release -p wordcraft -- --sample # the desktop app with the s
cargo run --release -p wordcraft -- report.docx # open a document
```
The interface translations and the spelling and hyphenation data come from
[craft-i18n](https://github.com/storytold/craft-i18n), an optional build input. To build with them,
check it out at the commit in `craft-i18n.rev` and point `CRAFT_I18N_DIR` at it (an absolute
path); without it WordCraft builds with an English interface, English spelling falls back to an
installed Hunspell English or craft-proof's built-in word list, and only Bulgarian, Turkish and
Russian hyphenate (by rule):
```sh
git clone https://github.com/storytold/craft-i18n ../craft-i18n
git -C ../craft-i18n checkout "$(cat craft-i18n.rev)"
CRAFT_I18N_DIR="$PWD/../craft-i18n" cargo run --release -p wordcraft -- --sample
```
Command line:
```sh
@@ -194,7 +207,7 @@ covers every target starting with it (`wordcraft*=debug`). The logger is
| Layer | Crate | Job |
|---|---|---|
| L0 | `wordcraft-geom` | units and measurements |
| L1 | `wordcraft-doc`, `wordcraft-fonts`, `wordcraft-proof` | document model and editing; fonts and shaping; spelling, grammar, hyphenation |
| L1 | `wordcraft-doc`, `wordcraft-fonts`, `wordcraft-proof` ([`craft-proof`](https://github.com/storytold/craft-libs/tree/main/crates/proof) from craft-libs) | document model and editing; fonts and shaping; spelling, grammar, hyphenation (data: craft-i18n `language-data/`, `CRAFT_I18N_DIR`) |
| L2 | `wordcraft-layout`, `wordcraft-docx`, `wordcraft-formats` | line breaking, pagination, tables, notes, hit testing; OOXML; ODT/RTF/HTML/Markdown/TXT |
| L3 | `wordcraft-render`, `wordcraft-pdf` | rasteriser (vello_cpu); PDF ([craft-pdf](docs/shared-pdf.md), Krilla) |
| L4 | `wordcraft-engine` | session, undo, 422 commands, document feature catalog |
+36 -2
View File
File diff suppressed because one or more lines are too long
+6
View File
@@ -758,6 +758,12 @@ fn parity_markdown(p: &Value) -> String {
fn main() -> ExitCode {
let args: Vec<OsString> = std::env::args_os().skip(1).collect();
// Language packs installed in the suite's font store are used; `proof.installPack` installs
// only while the person has turned downloading on in the app.
wordcraft_engine::packs::use_font_store(false);
if let Some(source) = wordcraft_engine::packs::index_override() {
eprintln!("wordcraft-cli: Language packs come from a test source: {source}");
}
match run(&args) {
Ok(()) | Err(Failure::StdoutClosed) => ExitCode::SUCCESS,
Err(Failure::Error(e)) => {
+2 -1
View File
@@ -9,7 +9,8 @@ use wordcraft_engine::Session;
use wordcraft_ui_egui::async_saves::{AsyncSaves, SaveOutcome};
use wordcraft_ui_egui::{Inbox, Services, WordApp};
const DOC_EXTS: &[&str] = &["docx", "docm", "dotx", "dotm", "doc", "dot", "odt", "rtf", "txt", "md", "html", "htm", "tex", "json", "xml"];
const DOC_EXTS: &[&str] =
&["docx", "docm", "dotx", "dotm", "doc", "dot", "odt", "rtf", "txt", "md", "html", "htm", "mht", "mhtml", "tex", "json", "xml"];
const IMAGE_EXTS: &[&str] = &["png", "jpg", "jpeg", "gif", "webp", "bmp", "wmf", "emf"];
const RECIPIENT_EXTS: &[&str] = wordcraft_engine::cmd::mailings::RECIPIENT_EXTENSIONS;
mod font_store;
+2
View File
@@ -490,6 +490,8 @@ fn main() -> eframe::Result {
app.set_startup_update_check(true);
load_prefs(&mut app);
load_user_dictionary(&mut app);
// Language packs from the suite's font store; an Install click is the person's own.
wordcraft_engine::packs::use_font_store(true);
// A region chosen in File › Options replaces the system's for the first blank
// document too (#838).
if !sample && app.session.prefs.region.is_some() {
File diff suppressed because it is too large Load Diff
Binary file not shown.
File diff suppressed because it is too large Load Diff
-431
View File
@@ -1,431 +0,0 @@
% title: Hyphenation patterns for Italian
% copyright: Copyright (C) 2008-2011 Claudio Beccari
% notice: This file is part of the hyph-utf8 package.
% See http://www.hyphenation.org/tex for more information.
% language:
% name: Italian
% tag: it
% version: 4.9 2014/04/22
% authors:
% -
% name: Claudio Beccari
% contact: claudio.beccari (at) gmail.com
% licence:
% - This file is available under any of the following licences:
% -
% name: LPPL
% version: 1.3
% or_later: true
% url: http://www.latex-project.org/lppl.txt
% status: maintained
% maintainer: Claudio Beccari, e-mail claudio dot beccari at gmail dot com
% -
% name: MIT
% url: https://opensource.org/licenses/MIT
% text: >
% Permission is hereby granted, free of charge, to any person
% obtaining a copy of this software and associated documentation
% files (the "Software"), to deal in the Software without
% restriction, including without limitation the rights to use,
% copy, modify, merge, publish, distribute, sublicense, and/or sell
% copies of the Software, and to permit persons to whom the
% Software is furnished to do so, subject to the following
% conditions:
%
% The above copyright notice and this permission notice shall be
% included in all copies or substantial portions of the Software.
%
% THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
% EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
% OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
% NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
% HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
% WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
% FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
% OTHER DEALINGS IN THE SOFTWARE.
% hyphenmins:
% typesetting:
% left: 2
% right: 2
% changes:
% - 2014-04-22 - Add few patterns involving `h'
% - 2011-08-16 - Change the licence from GNU LGPL into LPPL v1.3.
% - 2010-05-24 - Fix for Italian patterns for proper hyphenation of -ich and Ljubljana.
% - 2008-06-09 - Import of original ithyph.tex into hyph-utf8 package.
% - 2008-03-08 - (last change in ithyph.tex)
% texlive:
% encoding: ascii
% babelname: italian
% legacy_patterns: ithyph.tex
% message: Italian hyphenation patterns
% description: |-
% Hyphenation patterns for Italian in ASCII encoding.
% Compliant with the Recommendation UNI 6461 on hyphenation
% issued by the Italian Standards Institution
% (Ente Nazionale di Unificazione UNI).
% ==========================================
%
% These hyphenation patterns for the Italian language are supposed to comply
% with the Recommendation UNI 6461 on hyphenation issued by the Italian
% Standards Institution (Ente Nazionale di Unificazione UNI). No guarantee
% or declaration of fitness to any particular purpose is given and any
% liability is disclaimed.
%
\patterns{
.a3p2n % After the Garzanti dictionary: a-pnea, a-pnoi-co,...
.anti1
.anti3m2n
.bio1
.ca4p3s
.circu2m1
.contro1
.di2s3cine
.e2x1eu
.fran2k3
.free3
.li3p2sa
.narco1
.opto1
.orto3p2
.para1
.ph2l
.ph2r
.poli3p2
.pre1
.p2s
.re1i2scr
.sha2re3
.tran2s3c
.tran2s3d
.tran2s3l
.tran2s3n
.tran2s3p
.tran2s3r
.tran2s3t
.su2b3lu
.su2b3r
.wa2g3n
.wel2t1
2'2
a1ia
a1ie
a1io
a1iu
a1uo
a1ya
2at.
e1iu
e2w
o1ia
o1ie
o1io
o1iu
1b
2bb
2bc
2bd
2bf
2bm
2bn
2bp
2bs
2bt
2bv
b2l
b2r
2b.
2b'
1c
2cb
2cc
2cd
2cf
2ck
2cm
2cn
2cq
2cs
2ct
2cz
2chh
c2h
2ch.
2ch'.
2ch''.
2chb
ch2r
2chn
c2l
c2r
2c.
2c'
.c2
1d
2db
2dd
2dg
2dl
2dm
2dn
2dp
d2r
2ds
2dt
2dv
2dw
2d.
2d'
.d2
1f
2fb
2fg
2ff
2fn
f2l
f2r
2fs
2ft
2f.
2f'
1g
2gb
2gd
2gf
2gg
g2h
g2l
2gm
g2n
2gp
g2r
2gs
2gt
2gv
2gw
2gz
2gh2t
2g.
2g'
.h2
1h
2hb
2hd
2hh
hi3p2n
h2l
2hm
2hn
2hr
2hv
2h.
2h'
.j2
1j
2j.
2j'
.k2
1k
2kg
2kf
k2h
2kk
k2l
2km
k2r
2ks
2kt
2k.
2k'
1l
2lb
2lc
2ld
2l3f2
2lg
l2h
l2j
2lk
2ll
2lm
2ln
2lp
2lq
2lr
2ls
2lt
2lv
2lw
2lz
2l.
2l'.
2l''
1m
2mb
2mc
2mf
2ml
2mm
2mn
2mp
2mq
2mr
2ms
2mt
2mv
2mw
2m.
2m'
1n
2nb
2nc
2nd
2nf
2ng
2nk
2nl
2nm
2nn
2np
2nq
2nr
2ns
n2s3fer
2nt
2nv
2nz
n2g3n
2nheit
2n.
2n'
1p
2pd
p2h
p2l
2pn
3p2ne
2pp
p2r
2ps
3p2sic
2pt
2pz
2p.
2p'
1q
2qq
2q.
2q'
1r
2rb
2rc
2rd
2rf
r2h
2rg
2rk
2rl
2rm
2rn
2rp
2rq
2rr
2rs
2rt
r2t2s3
2rv
2rx
2rw
2rz
2r.
2r'
1s2
2shm
2sh.
2sh'
2s3s
s4s3m
2s3p2n
2stb
2stc
2std
2stf
2stg
2stm
2stn
2stp
2sts
2stt
2stv
2sz
4s.
4s'.
4s''
.t2
1t
2tb
2tc
2td
2tf
2tg
t2h
2th.
t2l
2tm
2tn
2tp
t2r
t2s
3t2sch
2tt
t2t3s
2tv
2tw
t2z
2tzk
tz2s
2t.
2t'.
2t''
1v
2vc
v2l
v2r
2vv
2v.
2v'.
2v''
1w
w2h
wa2r
2w1y
2w.
2w'
1x
2xb
2xc
2xf
2xh
2xm
2xp
2xt
2xw
2x.
2x'
y1ou
y1i
1z
2zb
2zd
2zl
2zn
2zp
2zt
2zs
2zv
2zz
2z.
2z'.
2z''
.z2
} % Pattern end
File diff suppressed because it is too large Load Diff
-503
View File
@@ -1,503 +0,0 @@
% title: Hyphenation patterns for Portuguese
% copyright: Copyright (C) 1987, 1994, 1996, 2015 Pedro J. de Rezende, 1996, 2015 J. Joao Dias Almeida, 2024 Leonardo Araujo and Aline Benevides
% notice: This file is part of the hyph-utf8 package.
% See http://www.hyphenation.org/tex for more information.
% language:
% name: Portuguese
% tag: pt
% version: 1.4 2024-07-13
% authors:
% -
% name: Pedro J. de Rezende
% contact: rezende (at) ic.unicamp.br
% -
% name: J. Joao Dias Almeida
% contact: jj (at) di.uminho.pt
% -
% name: Leonardo Araujo
% contact: leolca (at) gmail.com
% -
% name: Aline Benevides
% contact: benevides.aline12 (at) gmail.com
% licence:
% name: BSD 3-clause licence
% url: https://opensource.org/licenses/BSD-3-Clause
% text: >
% Redistribution and use in source and binary forms, with or without
% modification, are permitted provided that the following conditions
% are met:
% * Redistributions of source code must retain the above copyright
% notice, this list of conditions and the following disclaimer.
% * Redistributions in binary form must reproduce the above copyright
% notice, this list of conditions and the following disclaimer in the
% documentation and/or other materials provided with the
% distribution.
% * Neither the name of the University of Campinas, of the University
% of Minho nor the names of its contributors may be used to endorse
% or promote products derived from this software without specific
% prior written permission.
%
% THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
% "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
% LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
% A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL PEDRO J. DE
% REZENDE OR J.JOAO DIAS ALMEIDA BE LIABLE FOR ANY DIRECT, INDIRECT,
% INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
% BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
% OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
% AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
% LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY
% WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
% POSSIBILITY OF SUCH DAMAGE.
% hyphenmins:
% typesetting:
% left: 2
% right: 3
% changes:
% - Version 1.4 Release date: 13/07/2024 Leonardo Araujo and Aline Benevides
% - Version 1.3 Release date: 12/08/2015 Pedro J. de Rezende and J. Joao Dias Almeida
% - Version 1.2 Release date: 07/21/1996 Pedro J. de Rezende and J. Joao Dias Almeida
% - Version 1.1 Release date: 04/12/1994 Pedro J. de Rezende
% - Version 1.0 Release date: 02/13/1987 Pedro J. de Rezende
% texlive:
% synonyms:
% - portuges
% encoding: ec
% babelname: portuguese
% legacy_patterns: pthyph.tex
% message: Portuguese hyphenation patterns
% description: Hyphenation patterns for Portuguese in T1/EC and UTF-8 encodings.
%
\patterns{
1b2l
1b2r
1ba
1be
1bi
1bo
1bu
1bá
1bâ
1bã
1bé
1bí
1bó
1bú
1bê
1bõ
1c2h
1c2l
1c2r
1ca
1ce
1ci
1co
1cu
1cá
1câ
1cã
1cé
1cí
1có
1cú
1cê
1cõ
1ça
1çe
1çi
1ço
1çu
1çá
1çâ
1çã
1çé
1çí
1çó
1çú
1çê
1çõ
1d2l
1d2r
1da
1de
1di
1do
1du
1dá
1dâ
1dã
1dé
1dí
1dó
1dú
1dê
1dõ
1f2l
1f2r
1fa
1fe
1fi
1fo
1fu
1fá
1fâ
1fã
1fé
1fí
1fó
1fú
1fê
1fõ
1g2l
1g2r
1ga
1ge
1gi
1go
1gu
1gu4a
1gu4e
1gu4i
1gu4o
1gá
1gâ
1gã
1gé
1gí
1gó
1gú
1gê
1gõ
1ja
1je
1ji
1jo
1ju
1já
1jâ
1jã
1jé
1jí
1jó
1jú
1jê
1jõ
1k2l
1k2r
1ka
1ke
1ki
1ko
1ku
1ká
1kâ
1kã
1ké
1kí
1kó
1kú
1kê
1kõ
1l2h
1la
1le
1li
1lo
1lu
1lá
1lâ
1lã
1lé
1lí
1ló
1lú
1lê
1lõ
1ma
1me
1mi
1mo
1mu
1má
1mâ
1mã
1mé
1mí
1mó
1mú
1mê
1mõ
1n2h
1na
1ne
1ni
1no
1nu
1ná
1nâ
1nã
1né
1ní
1nó
1nú
1nê
1nõ
1p2l
1p2r
1pa
1pe
1pi
1po
1pu
1pá
1pâ
1pã
1pé
1pí
1pó
1pú
1pê
1põ
1qu4a
1qu4e
1qu4i
1qu4o
1ra
1re
1ri
1ro
1ru
1rá
1râ
1rã
1ré
1rí
1ró
1rú
1rê
1rõ
1sa
1se
1si
1so
1su
1sá
1sâ
1sã
1sé
1sí
1só
1sú
1sê
1sõ
1t2l
1t2r
1ta
1te
1ti
1to
1tu
1tá
1tâ
1tã
1té
1tí
1tó
1tú
1tê
1tõ
1v2l
1v2r
1va
1ve
1vi
1vo
1vu
1vá
1vâ
1vã
1vé
1ví
1vó
1vú
1vê
1võ
1w2l
1w2r
1xa
1xe
1xi
1xo
1xu
1xá
1xâ
1xã
1xé
1xí
1xó
1xú
1xê
1xõ
1za
1ze
1zi
1zo
1zu
1zá
1zâ
1zã
1zé
1zí
1zó
1zú
1zê
1zõ
a3a
a3e
a3o
c3c
e3a
e3e
e3o
i3a
i3e
i3i
i3o
i3â
i3ê
i3ô
o3a
o3e
o3o
r3r
s3s
u3a
u3e
u3o
u3u
.p2si
.p2sí
.g2no
.g2nó
.g2nô
t2c
1p2neu
.t2m
.p2t
su2b3r
su2b3l
.su3b4li
a1ir.
u1ir.
1qu
1vô
1lô
1cô
1gô
1bô
1tô
1rô
1pô
a1é
a1í
a1ó
a1ú
e1á
e1â
e1ã
e1é
e1ê
e1í
e1ó
é1o
e1ú
i1á
i1ã
í1a
i1é
i1í
i1ó
í1o
i1u
i1ú
o1á
o1é
o1í
o1ó
u1á
u1ã
u1â
u1í
ú1o
1qu2á
1qu2â
1qu2í
1gu2í
a1ind
a1i1nh
e1imp
e1inc
e1inf
e1ing
e1ins
e1int
e1inv
u1iz.
a1iz.
4a.
4e.
4o.
1gu2á
1gu2ã
1qu2ã
.m2n
c2za
.s2
1p2seu1d
1dô
1fô
1mô
1nô
1sô
1zô
tu1i
tu2it
tu2id
bu1i
nu1i
o1in
u1in
su1i
í1e
ju1i
fu1i
du1i
do1im
au1i
u1i1ç
u1ê
1gu2ê
1qu2ê
1çô
u1é
1gu2é
1qu2é
1xô
a1â
a1ã
a1ô
e1ô
.ne4o
o1ã
o1ê
o1im
o2i1na
pro1i1b
co2ima
.t2
1-
}
\hyphenation{% Do NOT make any alterations to this list! --- PdR
hard-ware
soft-ware
}
-241
View File
@@ -1,241 +0,0 @@
# English spelling supplement: common words the public-domain Moby list (1990s) lacks.
# Written by WordCraft contributors (MIT OR Apache-2.0); see ATTRIBUTION.md.
# One lowercase base form per line, sorted. Plurals, -ed, -ing, -er, -ly… come from the
# inflection rules in crates/proof/src/lib.rs, so list "desktop", not "desktops".
airbag
airfare
artcraft
autocomplete
autocorrect
autofill
autoformat
autosave
backend
bitcoin
bitmap
blockchain
blog
blogger
bluetooth
captcha
caregiver
carpool
cellphone
chatbot
chatroom
checkbox
checklist
childcare
clamour
clickable
colourful
colourisation
colourise
colourless
configurable
copyedit
coronavirus
covid
crowdfunding
crowdsource
crypto
cryptocurrency
customizable
customization
cyber
cybersecurity
cyberspace
database
dataset
daycare
desktop
download
downloadable
downsize
dropdown
earbud
ebook
ecommerce
email
emoji
emoticon
endeavour
esports
facebook
favicon
fintech
firewall
flowchart
foodie
freeware
frontend
fundraiser
gameplay
gamification
geeky
geolocation
gigabit
gigabyte
github
glamour
globalization
google
groupthink
hackathon
hardcode
hashtag
healthcare
homepage
homeschool
honour
honourable
honourably
hotspot
hyperlink
hypertext
inbox
incentivize
infographic
instagram
internet
interoperability
intranet
ipad
iphone
javascript
kickstart
kilobyte
laptop
lifestyle
linkedin
linux
livestream
lockdown
login
logout
lowercase
makeover
malware
manoeuvrable
megabyte
megapixel
meme
metadata
microchip
microsoft
middleware
mindset
multicultural
multiplayer
multitask
nerdy
netbook
netflix
newsfeed
offboard
offline
onboard
online
onscreen
outsource
paediatric
paralegal
passcode
petabyte
phishing
photobomb
playdate
playlist
plugin
podcast
popup
prequel
printout
prioritise
prioritization
prioritize
privatise
ransomware
recyclable
rigour
ringtone
roadmap
rollout
runtime
scalability
screencast
screensaver
screenshot
scrollbar
selfie
signup
sitemap
skillset
smartphone
smartwatch
smiley
snowboard
spellcheck
spellchecker
spreadsheet
spyware
stakeholder
standup
startup
subfolder
sunscreen
superfood
superuser
synthesise
tagline
takeout
techie
telecommute
teleconference
telehealth
telemedicine
terabyte
textbox
timeline
timestamp
toolbar
toolkit
touchpad
touchscreen
trackball
trackpad
travelogue
upcycle
upload
uppercase
upsell
upskill
usability
username
videoconference
videogame
vlog
vlogger
voicemail
walkthrough
webcam
webcast
webinar
weblog
webmaster
webpage
website
whatsapp
whiteboard
wifi
wiki
wikipedia
wireframe
wordcraft
workaround
workflow
workforce
workplace
workspace
workstation
youtube
-305
View File
@@ -1,305 +0,0 @@
# English spelling variants: American and British forms of the same word, for the English
# dialects (#1281, #1045). Written by WordCraft contributors from general knowledge of English
# spelling (MIT OR Apache-2.0); no other word list copied. See ATTRIBUTION.md.
#
# Each line: POLICY AMERICAN BRITISH
# POLICY is three letters, for American English (en-US), Canadian English (en-CA) and the
# British-spelling Englishes (en-GB, en-AU, en-NZ, en-IE, en-ZA, en-IN…), in that order:
# u only the American form is right there (the British one is flagged, American suggested)
# b only the British form is right there
# * both are right
# Inflections (colours, centred, organising, organisation…) come from the rules in
# crates/proof/src/en_variants.rs for -or/-our, -er/-re and -ize/-ise style pairs; other pairs
# get only a plural, so list their other forms on lines of their own. When in doubt a form is
# accepted: a false alarm is worse than a missed variant.
# -or / -our
ubb arbor arbour
ubb ardor ardour
ubb armor armour
ubb armory armoury
ubb behavior behaviour
ubb behavioral behavioural
ubb candor candour
ubb clamor clamour
ubb color colour
ubb demeanor demeanour
ubb discolor discolour
ubb dishonor dishonour
ubb endeavor endeavour
ubb favor favour
ubb favorite favourite
ubb favoritism favouritism
ubb fervor fervour
ubb flavor flavour
ubb harbor harbour
ubb honor honour
ubb humor humour
ubb labor labour
ubb misbehavior misbehaviour
ubb neighbor neighbour
ubb neighborhood neighbourhood
ubb neighborly neighbourly
ubb odor odour
ubb parlor parlour
ubb rancor rancour
ubb rigor rigour
ubb rumor rumour
ubb savior saviour
ubb savor savour
ubb savory savoury
ubb splendor splendour
ubb succor succour
ubb tumor tumour
ubb unfavorable unfavourable
ubb valor valour
ubb vapor vapour
ubb vigor vigour
# -er / -re
ubb caliber calibre
ubb center centre
ubb centimeter centimetre
ubb epicenter epicentre
ubb fiber fibre
ubb kilometer kilometre
ubb liter litre
ubb luster lustre
ubb maneuver manoeuvre
ubb meager meagre
ubb millimeter millimetre
ubb miter mitre
ubb ocher ochre
ubb saber sabre
ubb sepulcher sepulchre
ubb somber sombre
ubb specter spectre
*bb theater theatre
# A meter is also a measuring device in British English.
u** meter metre
# -ize / -ise (British English accepts both: Oxford spelling uses -ize)
uu* agonize agonise
uu* apologize apologise
uu* authorize authorise
uu* capitalize capitalise
uu* categorize categorise
uu* centralize centralise
uu* characterize characterise
uu* civilize civilise
uu* colonize colonise
uu* commercialize commercialise
uu* computerize computerise
uu* criticize criticise
uu* crystallize crystallise
uu* customize customise
uu* decentralize decentralise
uu* demonize demonise
uu* deodorize deodorise
uu* destabilize destabilise
uu* digitize digitise
uu* dramatize dramatise
uu* economize economise
uu* emphasize emphasise
uu* energize energise
uu* equalize equalise
uu* familiarize familiarise
uu* fantasize fantasise
uu* fertilize fertilise
uu* finalize finalise
uu* formalize formalise
uu* fossilize fossilise
uu* galvanize galvanise
uu* generalize generalise
uu* globalize globalise
uu* harmonize harmonise
uu* hospitalize hospitalise
uu* humanize humanise
uu* hypnotize hypnotise
uu* hypothesize hypothesise
uu* idealize idealise
uu* idolize idolise
uu* immobilize immobilise
uu* immortalize immortalise
uu* immunize immunise
uu* industrialize industrialise
uu* initialize initialise
uu* italicize italicise
uu* itemize itemise
uu* jeopardize jeopardise
uu* legalize legalise
uu* legitimize legitimise
uu* liberalize liberalise
uu* localize localise
uu* magnetize magnetise
uu* marginalize marginalise
uu* maximize maximise
uu* mechanize mechanise
uu* memorize memorise
uu* mesmerize mesmerise
uu* minimize minimise
uu* mobilize mobilise
uu* modernize modernise
uu* monopolize monopolise
uu* moralize moralise
uu* nationalize nationalise
uu* naturalize naturalise
uu* neutralize neutralise
uu* normalize normalise
uu* optimize optimise
uu* organize organise
uu* ostracize ostracise
uu* oxidize oxidise
uu* pasteurize pasteurise
uu* patronize patronise
uu* penalize penalise
uu* personalize personalise
uu* plagiarize plagiarise
uu* polarize polarise
uu* popularize popularise
uu* pressurize pressurise
uu* prioritize prioritise
uu* privatize privatise
uu* publicize publicise
uu* pulverize pulverise
uu* randomize randomise
uu* rationalize rationalise
uu* realize realise
uu* recognize recognise
uu* reorganize reorganise
uu* revolutionize revolutionise
uu* satirize satirise
uu* scrutinize scrutinise
uu* sensitize sensitise
uu* socialize socialise
uu* specialize specialise
uu* stabilize stabilise
uu* standardize standardise
uu* sterilize sterilise
uu* stigmatize stigmatise
uu* subsidize subsidise
uu* summarize summarise
uu* symbolize symbolise
uu* sympathize sympathise
uu* synchronize synchronise
uu* synthesize synthesise
uu* systematize systematise
uu* tantalize tantalise
uu* terrorize terrorise
uu* theorize theorise
uu* trivialize trivialise
uu* urbanize urbanise
uu* utilize utilise
uu* vandalize vandalise
uu* verbalize verbalise
uu* victimize victimise
uu* visualize visualise
uu* vocalize vocalise
uu* westernize westernise
# -yze / -yse
u*b analyze analyse
u*b catalyze catalyse
u*b dialyze dialyse
u*b electrolyze electrolyse
u*b hydrolyze hydrolyse
u*b paralyze paralyse
u*b psychoanalyze psychoanalyse
# -ense / -ence
ubb defense defence
ubb offense offence
ubb pretense pretence
ubb defenseless defenceless
# In British English "license" and "practise" are the verbs, so only American English flags.
u** license licence
u** practice practise
# Doubled l (American English accepts both)
*bb canceled cancelled
*bb canceling cancelling
*bb channeled channelled
*bb channeling channelling
*bb counseled counselled
*bb counseling counselling
*bb counselor counsellor
*bb counselors counsellors
*bb dueling duelling
*bb equaled equalled
*bb equaling equalling
*bb fueled fuelled
*bb fueling fuelling
*bb jeweler jeweller
*bb jewelers jewellers
*bb jewelry jewellery
*bb labeled labelled
*bb labeling labelling
*bb leveled levelled
*bb leveling levelling
*bb marveled marvelled
*bb marveling marvelling
*bb marvelous marvellous
*bb modeled modelled
*bb modeling modelling
*bb quarreled quarrelled
*bb quarreling quarrelling
*bb signaled signalled
*bb signaling signalling
*bb totaled totalled
*bb totaling totalling
*bb traveled travelled
*bb traveler traveller
*bb travelers travellers
*bb traveling travelling
*bb tunneled tunnelled
*bb tunneling tunnelling
*bb woolen woollen
# -og / -ogue
*bb catalog catalogue
*bb cataloged catalogued
*bb cataloging cataloguing
# ae / oe
u*b anemia anaemia
u*b anemic anaemic
u*b anesthesia anaesthesia
u*b anesthetic anaesthetic
**b archeology archaeology
**b archeologist archaeologist
u*b diarrhea diarrhoea
u*b edema oedema
u*b esophagus oesophagus
u*b estrogen oestrogen
u*b gynecologist gynaecologist
u*b gynecology gynaecology
u*b hematology haematology
u*b hemoglobin haemoglobin
u*b hemorrhage haemorrhage
u*b leukemia leukaemia
u*b orthopedic orthopaedic
u*b pediatric paediatric
u*b pediatrician paediatrician
u*b pediatrics paediatrics
# Other words
u*b aluminum aluminium
u*b cozy cosy
ubb mold mould
ubb molded moulded
ubb molding moulding
ubb moldy mouldy
ubb molt moult
ubb pajamas pyjamas
ubb plow plough
ubb plowed ploughed
ubb plowing ploughing
u** program programme
u*b skeptic sceptic
u*b skeptical sceptical
u*b skepticism scepticism
ubb smolder smoulder
ubb smoldering smouldering
# "Tire" and "curb" are also verbs everywhere, so only American English flags.
uu* tire tyre
uu* curb kerb
-214
View File
@@ -1,214 +0,0 @@
# Russian spelling supplement: common words older Russian Hunspell dictionaries lack.
# Written by WordCraft contributors (MIT OR Apache-2.0); see ATTRIBUTION.md.
# One word per line: the base form, a TAB, then its declension, from which
# crates/proof/src/ru/extra.rs makes every form:
# m masculine noun ending in a consonant (маркетплейс, -а, -у, -ом, -е, -ы/-и, -ов/-ей…)
# f feminine noun in -а (инфографика, -и, -е, -у, -ой…; genitive plural without ending)
# fia feminine noun in -ия (интеграция, -ии, -ию, -ией, -ий…)
# fs feminine noun in -ь (соцсеть, -и, -ью, -ей, -ям…)
# nie neuter noun in -ие (согласие, -ия, -ию, -ием, -ий…)
# no neuter noun in -о (юрлицо, -а, -у, -ом, -е; genitive plural without ending)
# adj adjective or participle in -ый, -ий, -ой (бонусный, газоиспользующий)
# ovat verb in -овать with its conjugation and participles (колеровать, администрировать)
# = the forms themselves, space-separated (verbs, nouns with a fleeting vowel)
# A word alone on its line doesn't decline (онлайн, медиа). Capitalized words are names and match
# only when written with a capital.
# Commerce, marketing and the web
маркетплейс m
селлер m
фулфилмент m
ритейл m
ритейлер m
дистрибьютор m
дропшиппинг m
мерчандайзер m
эквайринг m
промокод m
кешбэк m
кэшбэк m
лендинг m
лид m
тренд m
бренд m
брендбук m
ребрендинг m
таргетинг m
геотаргетинг m
ретаргетинг m
таргетолог m
маркетолог m
копирайтер m
рерайтер m
контент m
блогер m
инфлюенсер m
хештег m
хэштег m
репост m
лайк m
подкаст m
стриминг m
вебинар m
чат m
бот m
чат-бот m
стартап m
франчайзинг m
аутсорсинг m
аутстаффинг m
коворкинг m
дедлайн m
фидбэк m
фидбек m
кейс m
скриншот m
скринкаст m
баннер m
виджет m
дашборд m
сервис m
сайт m
скан m
токен m
аккаунт m
логин m
пароль m
смартфон m
гаджет m
планшет m
ноутбук m
апгрейд m
апдейт m
бэкап m
бэкенд m
фронтенд m
датасет m
промпт m
интерфейс m
хостинг m
домен m
сниппет m
трафик m
юзер m
стикер m
чекбокс m
инпут m
плейсхолдер m
тултип m
гипсокартон m
пеноплекс m
праймер m
колер m
колорант m
Яндекс m
Битрикс m
Вебвизор m
Дзен m
инфографика f
аналитика f
оплата f
предоплата f
постоплата f
доплата f
переплата f
недоплата f
маркировка = маркировка маркировки маркировке маркировку маркировкой маркировкою маркировок маркировкам маркировками маркировках
отгрузка = отгрузка отгрузки отгрузке отгрузку отгрузкой отгрузкою отгрузок отгрузкам отгрузками отгрузках
колеровка = колеровка колеровки колеровке колеровку колеровкой колеровкою колеровок колеровкам колеровками колеровках
рассылка = рассылка рассылки рассылке рассылку рассылкой рассылкою рассылок рассылкам рассылками рассылках
выгрузка = выгрузка выгрузки выгрузке выгрузку выгрузкой выгрузкою выгрузок выгрузкам выгрузками выгрузках
интеграция fia
пагинация fia
госрегистрация fia
геолокация fia
монетизация fia
кастомизация fia
персонализация fia
сегментация fia
верификация fia
валидация fia
авторизация fia
аутентификация fia
оптимизация fia
визуализация fia
соцсеть fs
согласие nie
домовладение nie
рождение nie
паллетирование nie
неисполнение nie
непредоставление nie
бонусный adj
маркетинговый adj
мотивационный adj
гипоаллергенный adj
тиксотропный adj
сополимерный adj
покупочный adj
таргетированный adj
брендированный adj
кастомный adj
онлайновый adj
офлайновый adj
колеруемый adj
колерованный adj
заколерованный adj
газоиспользующий adj
колеровать ovat
заколеровать ovat
администрировать ovat
модерировать ovat
отмодерировать ovat
таргетировать ovat
монетизировать ovat
брендировать ovat
кастомизировать ovat
детектировать ovat
верифицировать ovat
валидировать ovat
генерировать ovat
сгенерировать ovat
мониторить = мониторить мониторю мониторишь мониторит мониторим мониторите мониторят мониторил мониторила мониторили мониторя мониторинг
гуглить = гуглить гуглю гуглишь гуглит гуглим гуглите гуглят гуглил гуглила гуглили
скачивать = скачивать скачиваю скачиваешь скачивает скачиваем скачиваете скачивают скачивал скачивала скачивали скачивается скачиваются
оценивание nie
медучреждение nie
юрлицо no
учётка = учётка учётки учётке учётку учёткой учёток учёткам учётками учётках учетка учетки учетке учетку учеткой учеток учеткам учетками учетках
перелинковка = перелинковка перелинковки перелинковке перелинковку перелинковкой перелинковок
поклейка = поклейка поклейки поклейке поклейку поклейкой поклеек
детекция fia
досудебный adj
категорийный adj
оркестратор m
оффер m
трекер m
бриф m
гайд m
монтажёр m
тенант m
мерчант m
амбассадор m
Роскомнадзор m
Директ m
Маркет m
вовлечённость fs
вовлеченность fs
укрывистость fs
# Words that don't decline
онлайн
офлайн
оффлайн
медиа
демо
франчайзи
киноа
смс
пн
вт
ср
чт
пт
сб
вс
+1 -1
View File
@@ -1 +1 @@
9205108cfdced5177706264038848b68b2422d2b
059cbf95c2a3eb95d637ca8584faca1129d17f48
+2 -1
View File
@@ -9,7 +9,8 @@ description = "WordCraft document model: stories, paragraphs, runs, tables, sect
workspace = true
[features]
# Test inputs other crates' tests share (`cfb_guard::repeating_difat_sample`); turned on only as a
# Test inputs other crates' tests share (`cfb_guard::repeating_difat_sample`,
# `zip_guard::zip64_entry_count_sample`); turned on only as a
# dev-dependency, so never built into the app.
test-support = []
+163
View File
@@ -48,6 +48,131 @@ impl LineCap {
}
}
/// An arrowhead's width across its line or length along it (ST_LineEndWidth and
/// ST_LineEndLength, ECMA-376 §20.1.10.39–40): 2, 3 or 5 times the line's width, as Word draws it.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum ArrowSize {
Small,
#[default]
Medium,
Large,
}
impl ArrowSize {
/// Read a DrawingML size (`sm`, `med`, `lg`); absent or unknown values are medium.
pub fn from_ooxml(value: &str) -> Self {
match value {
"sm" => Self::Small,
"lg" => Self::Large,
_ => Self::Medium,
}
}
/// The DrawingML spelling.
pub fn ooxml(self) -> &'static str {
match self {
Self::Small => "sm",
Self::Medium => "med",
Self::Large => "lg",
}
}
/// Read a VML width (`narrow`, `medium`, `wide`) or length (`short`, `medium`, `long`).
pub fn from_vml(value: &str) -> Self {
match value {
"narrow" | "short" => Self::Small,
"wide" | "long" => Self::Large,
_ => Self::Medium,
}
}
/// How many [`arrow_unit`]s the arrowhead measures.
pub fn factor(self) -> f32 {
match self {
Self::Small => 2.0,
Self::Medium => 3.0,
Self::Large => 5.0,
}
}
}
/// The kind of an arrowhead (ST_LineEndType, ECMA-376 §20.1.10.33), kept for saving; every kind
/// is drawn as a triangle.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum ArrowKind {
#[default]
Triangle,
Stealth,
Diamond,
Oval,
Arrow,
}
impl ArrowKind {
/// Read a DrawingML end type other than `none`; unknown values are triangles.
pub fn from_ooxml(value: &str) -> Self {
match value {
"stealth" => Self::Stealth,
"diamond" => Self::Diamond,
"oval" => Self::Oval,
"arrow" => Self::Arrow,
_ => Self::Triangle,
}
}
/// The DrawingML spelling.
pub fn ooxml(self) -> &'static str {
match self {
Self::Triangle => "triangle",
Self::Stealth => "stealth",
Self::Diamond => "diamond",
Self::Oval => "oval",
Self::Arrow => "arrow",
}
}
/// Read a VML arrow (`block`, `classic`, `diamond`, `oval`, `open`) other than `none`.
pub fn from_vml(value: &str) -> Self {
match value {
"classic" => Self::Stealth,
"diamond" => Self::Diamond,
"oval" => Self::Oval,
"open" => Self::Arrow,
_ => Self::Triangle,
}
}
}
/// An arrowhead's kind and size (`a:headEnd` / `a:tailEnd`). The default is a medium triangle.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct ArrowStyle {
pub kind: ArrowKind,
pub width: ArrowSize,
pub length: ArrowSize,
}
impl ArrowStyle {
pub fn is_default(&self) -> bool {
*self == Self::default()
}
/// The arrowhead's length along its line and width across it, for a line `line_width` wide.
pub fn dims(self, line_width: f32) -> (f32, f32) {
let unit = arrow_unit(line_width);
(self.length.factor() * unit, self.width.factor() * unit)
}
}
/// What an arrowhead's size is a multiple of: the line's width, but at least 2 points (Word draws
/// a thinner line's arrowheads as for a 2-point line).
pub fn arrow_unit(line_width: f32) -> f32 {
let w = if line_width.is_finite() { line_width.clamp(0.25, 200.0) } else { 0.75 };
w.max(2.0)
}
/// The rest of a shape's properties. Empty (the default) for most shapes.
#[derive(Clone, Debug, PartialEq, Default, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
@@ -69,6 +194,11 @@ pub struct ShapeExtra {
pub arrow_start: bool,
#[serde(skip_serializing_if = "is_false")]
pub arrow_end: bool,
/// The kind and size of the start and end arrowheads, when there are any.
#[serde(skip_serializing_if = "ArrowStyle::is_default")]
pub start_style: ArrowStyle,
#[serde(skip_serializing_if = "ArrowStyle::is_default")]
pub end_style: ArrowStyle,
/// A rounded rectangle's corner radius (`roundRect`'s `adj`, ECMA-376 §20.1.10.56): in
/// 100000ths of the shorter side, at most [`MAX_CORNER`]; `None` draws [`DEFAULT_CORNER`].
#[serde(skip_serializing_if = "Option::is_none")]
@@ -83,6 +213,39 @@ pub struct ShapeExtra {
/// main colour (the shape's `stroke`).
#[serde(skip_serializing_if = "Option::is_none")]
pub outline_fill: Option<crate::fill::Fill>,
/// A horizontal rule (VML `o:hr`: a web page's `<hr>` pasted into Word): it takes its whole
/// line, and is drawn across [`HorizontalRule::pct`] of it or at the shape's own width.
#[serde(skip_serializing_if = "Option::is_none")]
pub hr: Option<HorizontalRule>,
}
/// How a horizontal rule sits on its line (VML `o:hrpct`, `o:hralign`), as Word draws one:
/// across `pct` percent of the line's width (the paragraph's, inside its indents), or at the
/// shape's own width when `pct` is 0; `align` places it when it is narrower than the line.
#[derive(Clone, Copy, Debug, PartialEq, Default, Serialize, Deserialize)]
#[serde(default, rename_all = "camelCase")]
pub struct HorizontalRule {
pub pct: f32,
pub align: crate::props::Align,
/// Word's standard look (`o:hrstd`), kept for saving.
pub standard: bool,
}
impl HorizontalRule {
/// The rule's width on a line `line` wide, for a shape `own` wide.
pub fn width(&self, line: f32, own: f32) -> f32 {
let line = line.max(0.0);
if self.pct.is_finite() && self.pct > 0.0 { line * (self.pct.min(100.0) / 100.0) } else { own.clamp(0.0, line) }
}
/// Where a rule `w` wide starts on a line from `x`, `line` wide.
pub fn left(&self, x: f32, line: f32, w: f32) -> f32 {
match self.align {
crate::props::Align::Center => x + (line - w) / 2.0,
crate::props::Align::Right => x + line - w,
_ => x,
}
}
}
fn is_zero(v: &u32) -> bool {
+363 -2
View File
@@ -205,11 +205,15 @@ struct BoxBudget {
costs: BTreeMap<u32, usize>,
/// Boxes left without a story because the paste's budget couldn't hold theirs.
skipped: usize,
/// A box the fragment has no story for may show this document's story of that id (a
/// fragment copied from this document without its parts). Off for another document's
/// fragment, whose ids mean nothing here.
local: bool,
}
impl BoxBudget {
fn new() -> BoxBudget {
BoxBudget { stories: MAX_PASTED_BOXES, bytes: MAX_PASTED_BOX_BYTES, active: Vec::new(), costs: BTreeMap::new(), skipped: 0 }
BoxBudget { stories: MAX_PASTED_BOXES, bytes: MAX_PASTED_BOX_BYTES, active: Vec::new(), costs: BTreeMap::new(), skipped: 0, local: true }
}
}
@@ -385,6 +389,167 @@ fn restyle_block(b: &mut Block, from: &str, to: &Option<String>, depth: usize) -
}
}
/// What [`Document::adopt_with`] renumbers in another document's content: media keys, lists,
/// note and comment stories, comment ids, text boxes (one it doesn't bring is dropped), revisions
/// (after ours, at `rev`; an id past the `revs` it brings is dropped) and styles (those kept under new ids; unstyled paragraphs take
/// `default_style` when it's given).
struct Relink<'a> {
media: &'a BTreeMap<String, String>,
nums: &'a BTreeMap<u32, u32>,
notes: &'a BTreeMap<u32, u32>,
comments: &'a BTreeMap<u32, u32>,
boxes: &'a BTreeMap<u32, Part>,
/// Added to permission range ids: ours come first.
perms: u32,
rev: u32,
revs: u32,
styles: &'a BTreeMap<String, String>,
default_style: Option<&'a str>,
}
impl Relink<'_> {
fn block(&self, b: &mut Block) {
// Its own stack, like `each_para_mut`: tables however deeply nested are relinked (#1339).
let mut todo = vec![b];
while let Some(b) = todo.pop() {
match b {
Block::Para(p) => self.para(p),
Block::Table(t) => {
self.style(&mut t.props.style);
self.shift_change(&mut t.props.fmt_change);
if let Some(c) = t.props.fmt_change.as_mut() {
self.style(&mut c.old.style);
}
for row in t.rows.iter_mut() {
self.shift_change(&mut row.props.fmt_change);
for cell in row.cells.iter_mut() {
self.shift_change(&mut cell.props.fmt_change);
todo.extend(cell.blocks.iter_mut().map(Arc::make_mut));
}
}
}
}
}
}
fn style(&self, id: &mut Option<String>) {
if let Some(new) = id.as_ref().and_then(|i| self.styles.get(i)) {
*id = Some(new.clone());
}
}
/// A paragraph's style: unstyled ones take `default_style` when it's given.
fn para_style(&self, id: &mut Option<String>) {
match (&*id, self.default_style) {
(None, Some(d)) => *id = Some(d.to_string()),
_ => self.style(id),
}
}
fn num(&self, n: &mut Option<crate::props::NumRef>) {
if let Some(n) = n.as_mut()
&& n.num != 0
{
n.num = self.nums.get(&n.num).copied().unwrap_or(0);
}
}
/// Our id for the other document's revision `r`, if it has one.
fn rev_id(&self, r: u32) -> Option<u32> {
if r < self.revs { self.rev.checked_add(r) } else { None }
}
fn shift_change<T>(&self, c: &mut Option<Box<crate::props::PropChange<T>>>) {
match (c.as_ref().and_then(|c| self.rev_id(c.rev)), c.as_mut()) {
(Some(v), Some(x)) => x.rev = v,
_ => *c = None,
}
}
fn para(&self, p: &mut Paragraph) {
self.num(&mut p.props.numbering);
self.para_style(&mut p.props.style);
for r in &mut p.runs {
self.style(&mut r.props.style);
}
self.style(&mut p.mark.style);
for c in p.runs.iter_mut().map(|r| &mut r.props).chain([&mut p.mark]) {
c.ins = c.ins.and_then(|r| self.rev_id(r));
c.del = c.del.and_then(|r| self.rev_id(r));
self.shift_change(&mut c.fmt_change);
// A tracked change's old formatting names styles and lists as the live one does.
if let Some(f) = c.fmt_change.as_mut() {
self.style(&mut f.old.style);
}
}
self.shift_change(&mut p.props.fmt_change);
if let Some(f) = p.props.fmt_change.as_mut() {
self.para_style(&mut f.old.style);
self.num(&mut f.old.numbering);
}
if let Some(s) = p.section.as_deref_mut() {
self.shift_change(&mut s.fmt_change);
}
match (p.props.num_change.as_ref().and_then(|c| self.rev_id(c.rev)), p.props.num_change.as_mut()) {
(Some(v), Some(c)) => c.rev = v,
_ => p.props.num_change = None,
}
for o in &mut p.objects {
self.object(o, 0);
}
p.touch();
}
fn object(&self, o: &mut InlineObject, depth: usize) {
// An id that names nothing here becomes one that names nothing at all.
let map = |m: &BTreeMap<u32, u32>, id: &mut u32| *id = m.get(id).copied().unwrap_or(u32::MAX);
match o {
InlineObject::Image { media, .. } => {
if let Some(k) = self.media.get(media.as_str()) {
*media = k.clone();
}
}
InlineObject::Graphic { graphic, .. } => {
let mapped = |m: &str| self.media.get(m).filter(|k| k.as_str() != m);
if graphic.items.iter().any(|i| matches!(i, crate::graphic::GraphicItem::Image { media, .. } if mapped(media).is_some())) {
for i in &mut Arc::make_mut(graphic).items {
if let crate::graphic::GraphicItem::Image { media, .. } = i
&& let Some(k) = self.media.get(media.as_str())
{
*media = k.clone();
}
}
}
}
InlineObject::Group { children, .. } if depth < MAX_BOX_DEPTH => children.iter_mut().for_each(|c| self.object(&mut c.obj, depth + 1)),
InlineObject::NoteRef { id, .. } => map(self.notes, id),
InlineObject::CommentStart { id } | InlineObject::CommentEnd { id } => map(self.comments, id),
InlineObject::PermStart { id, .. } | InlineObject::PermEnd { id } => *id = id.saturating_add(self.perms),
_ => {}
}
for story in o.text_box_slots() {
if story.is_some_and(|id| !self.boxes.contains_key(&id)) {
*story = None;
}
}
}
}
/// One past the highest permission range id in `blocks`.
fn perm_end<'a>(blocks: impl Iterator<Item = &'a Arc<Block>>) -> u32 {
let mut end = 0u32;
for b in blocks {
each_para(b, 0, &mut |p| {
for o in &p.objects {
if let InlineObject::PermStart { id, .. } | InlineObject::PermEnd { id } = o {
end = end.max(id.saturating_add(1));
}
}
});
}
end
}
/// Text box story ids of a paragraph's shapes.
fn box_ids(p: &Paragraph) -> impl Iterator<Item = u32> + '_ {
p.objects.iter().flat_map(InlineObject::text_boxes)
@@ -862,6 +1027,186 @@ impl Document {
Fragment { blocks, parts }
}
/// Another document's body as a fragment of this one, to insert: the media, lists, styles,
/// notes, comments and tracked changes it refers to come along (lists, notes, comments and
/// revisions renumbered so they can't collide with ours; a style we already have by id stays
/// ours), and its text boxes ride in [`Fragment::parts`].
pub fn adopt(&mut self, src: Document) -> Fragment {
self.adopt_with(src, false)
}
/// [`Self::adopt`]; with `keep_styles` its content keeps its own formatting (Word's
/// `w:matchSrc`): a style whose id we already have comes along under a new id, and its
/// document defaults go into its base styles, which its unstyled paragraphs then use.
pub fn adopt_with(&mut self, src: Document, keep_styles: bool) -> Fragment {
self.adopt_in(src, keep_styles, &mut 0)
}
/// [`Self::adopt_with`], its permission range ids from at least `perms` on (and past ours);
/// `perms` moves past them.
fn adopt_in(&mut self, mut src: Document, keep_styles: bool, perms: &mut u32) -> Fragment {
let mut renamed = BTreeMap::new();
let mut default_style = None;
if keep_styles {
if src.styles.get("Normal").is_none() {
src.styles.upsert(crate::styles::Style { id: "Normal".into(), name: "Normal".into(), ..Default::default() });
}
// Ids given out here are taken too: two styles never get the same new one.
let mut given = std::collections::HashSet::new();
let room = src.styles.styles.len().saturating_mul(2).saturating_add(self.styles.styles.len()).saturating_add(1);
for st in &src.styles.styles {
if self.styles.get(&st.id).is_some() {
let fresh = (1..=room)
.map(|n| format!("{}{n}", st.id))
.find(|id| self.styles.get(id).is_none() && src.styles.get(id).is_none() && !given.contains(id));
if let Some(fresh) = fresh {
given.insert(fresh.clone());
renamed.insert(st.id.clone(), fresh);
}
}
}
default_style = Some(renamed.get("Normal").cloned().unwrap_or_else(|| "Normal".into()));
}
let restyle = |id: &mut Option<String>| {
if let Some(new) = id.as_ref().and_then(|i| renamed.get(i)) {
*id = Some(new.clone());
}
};
let mut media = BTreeMap::new();
for (key, data) in &src.media {
let ext = key.rsplit_once('.').map(|(_, e)| e).unwrap_or("png");
media.insert(key.clone(), self.add_media(data.to_vec(), ext));
}
let next_abs = self.numbering.abstracts.iter().map(|a| a.id.saturating_add(1)).max().unwrap_or(0);
let mut abs_map = BTreeMap::new();
for (i, a) in src.numbering.abstracts.iter().enumerate() {
let id = next_abs.saturating_add(i as u32);
abs_map.insert(a.id, id);
let mut a = crate::numbering::AbstractNum { id, ..a.clone() };
a.levels.iter_mut().for_each(|l| restyle(&mut l.style));
self.numbering.abstracts.push(a);
}
let next_num = self.numbering.nums.iter().map(|n| n.id.saturating_add(1)).max().unwrap_or(1).max(1);
let mut nums = BTreeMap::new();
for (i, n) in src.numbering.nums.iter().enumerate() {
let id = next_num.saturating_add(i as u32);
nums.insert(n.id, id);
let abstract_id = abs_map.get(&n.abstract_id).copied().unwrap_or(n.abstract_id);
let mut n = crate::numbering::Num { id, abstract_id, ..n.clone() };
n.level_overrides.iter_mut().for_each(|(_, l)| restyle(&mut l.style));
self.numbering.nums.push(n);
}
let renumber = |n: &mut Option<crate::props::NumRef>| {
if let Some(n) = n.as_mut()
&& n.num != 0
{
n.num = nums.get(&n.num).copied().unwrap_or(0);
}
};
// What a style in the other document doesn't set comes from its defaults there, and
// Word's own (no space, single lines) where those don't say.
let mut para_defaults = src.styles.default_para.clone();
para_defaults.space_before.get_or_insert(0.0);
para_defaults.space_after.get_or_insert(0.0);
para_defaults.line_spacing.get_or_insert(crate::props::LineSpacing::Multiple(1.0));
for st in &src.styles.styles {
let mut st = st.clone();
// Our defaults stay ours.
st.default = false;
if keep_styles {
if let Some(new) = renamed.get(&st.id) {
st.name = format!("{} ({new})", st.name);
st.id = new.clone();
}
restyle(&mut st.based_on);
restyle(&mut st.next);
restyle(&mut st.linked);
// A character style's runs get the defaults through their paragraph's style.
if st.based_on.is_none() && st.kind != crate::styles::StyleKind::Character {
let mut chr = src.styles.default_chr.clone();
chr.overlay(&st.chr);
st.chr = chr;
let mut para = para_defaults.clone();
para.overlay(&st.para);
st.para = para;
}
}
if self.styles.get(&st.id).is_none() {
renumber(&mut st.para.numbering);
self.styles.upsert(st);
}
}
let rev = u32::try_from(self.revisions.len()).unwrap_or(u32::MAX);
self.revisions.extend(src.revisions.iter().cloned());
// Notes and comments get stories (and comments ids) of their own here; their text is
// relinked below, once every new id is known.
let mut notes = BTreeMap::new();
for (id, part) in src.parts.iter().filter(|(_, p)| matches!(p.kind, PartKind::Footnote | PartKind::Endnote)) {
notes.insert(*id, self.add_part(part.kind, Vec::new()));
}
let mut comments = BTreeMap::new();
let mut next_comment = self.comments.keys().next_back().map_or(0, |k| k.saturating_add(1));
for (id, c) in &src.comments {
if src.parts.get(&c.part).is_some_and(|p| p.kind == PartKind::Comment) {
notes.insert(c.part, self.add_part(PartKind::Comment, Vec::new()));
comments.insert(*id, next_comment);
next_comment = next_comment.saturating_add(1);
}
}
let boxes: BTreeMap<u32, Part> = src.parts.iter().filter(|(_, p)| p.kind == PartKind::TextBox).map(|(k, p)| (*k, p.clone())).collect();
// Its permission ranges are numbered after ours, so no end of theirs closes one of ours.
let first_perm = perm_end(self.body.iter().chain(self.parts.values().flat_map(|p| p.blocks.iter()))).max(*perms);
*perms = first_perm.saturating_add(perm_end(src.body.iter().chain(src.parts.values().flat_map(|p| p.blocks.iter()))));
let link = Relink {
media: &media,
nums: &nums,
notes: &notes,
comments: &comments,
boxes: &boxes,
perms: first_perm,
rev,
revs: u32::try_from(src.revisions.len()).unwrap_or(u32::MAX),
styles: &renamed,
default_style: default_style.as_deref(),
};
let mut parts = boxes.clone();
for part in parts.values_mut() {
part.blocks.iter_mut().for_each(|b| link.block(Arc::make_mut(b)));
}
// A note's or comment's text boxes get stories here now; the body's go with the fragment.
let mut budget = BoxBudget::new();
for (old, new) in &notes {
let Some(part) = src.parts.get(old) else { continue };
let mut blocks: Vec<Block> = part.blocks.iter().map(|b| (**b).clone()).collect();
blocks.iter_mut().for_each(|b| link.block(b));
self.adopt_text_boxes(&mut blocks, &parts, 0, &mut budget);
if let Some(p) = self.parts.get_mut(new) {
p.blocks = blocks.into_iter().map(Arc::new).collect();
}
}
for (id, c) in &src.comments {
if let (Some(new), Some(part)) = (comments.get(id), notes.get(&c.part)) {
let parent = c.parent.and_then(|p| comments.get(&p).copied());
// Its durable id is unique in its own document only: the writer gives the copy one.
self.comments.insert(*new, crate::Comment { parent, part: *part, durable_id: None, ..c.clone() });
}
}
let mut blocks: Vec<Block> = src.body.iter().map(|b| (**b).clone()).collect();
blocks.iter_mut().for_each(|b| link.block(b));
Fragment { blocks, parts }
}
/// [`Self::adopt_with`] as blocks to place in a story directly (a reader building one): its
/// text boxes already have stories of their own here. `perms` is the first permission range
/// id the reader hasn't given out (ranges it holds aren't in this document yet); it moves past
/// the adopted ones.
pub fn adopt_blocks(&mut self, src: Document, keep_styles: bool, perms: &mut u32) -> Vec<Block> {
let Fragment { mut blocks, parts } = self.adopt_in(src, keep_styles, perms);
let mut budget = BoxBudget::new();
self.adopt_text_boxes(&mut blocks, &parts, 0, &mut budget);
blocks
}
/// Copies of the text box stories shapes in `blocks` show, nested boxes included.
fn text_box_parts(&self, blocks: &[Block]) -> BTreeMap<u32, Part> {
let mut out = BTreeMap::new();
@@ -971,7 +1316,7 @@ impl Document {
budget.skipped = budget.skipped.saturating_add(1);
return None;
}
let src = parts.get(&old).or_else(|| self.parts.get(&old).filter(|p| p.kind == PartKind::TextBox))?;
let src = parts.get(&old).or_else(|| self.parts.get(&old).filter(|p| budget.local && p.kind == PartKind::TextBox))?;
// Charge what the copy really allocates before making any of it: the block slots, and the
// blocks holding text boxes (deep-copied below). The rest are shared `Arc`s, however many
// boxes show them, so they cost nothing more. Worked out once per paste.
@@ -1012,6 +1357,22 @@ impl Document {
Some(id)
}
/// The blocks of `frag`, a fragment of another document, with every text box given a new
/// story of its own here copied from the fragment's parts (as a paste does), to place them
/// in this document; a box the fragment has no story for keeps none.
pub fn adopt_fragment(&mut self, frag: &Fragment) -> Vec<Block> {
let mut blocks = frag.blocks.clone();
let mut budget = BoxBudget { local: false, ..BoxBudget::new() };
self.adopt_text_boxes(&mut blocks, &frag.parts, 0, &mut budget);
if budget.skipped > 0 {
log::warn!(
"{} text boxes are left empty; their stories are past what one copy brings ({MAX_PASTED_BOXES} stories, {MAX_PASTED_BOX_BYTES} bytes)",
budget.skipped
);
}
blocks
}
/// Insert a fragment at `pos`; returns the position after the pasted content.
pub fn insert_fragment(&mut self, pos: &Pos, frag: &Fragment) -> Result<Pos> {
// Tables that would nest deeper than the limit here become their paragraphs (#1596).
+227
View File
@@ -664,3 +664,230 @@ fn a_table_nested_deeper_than_the_paragraph_walk_is_never_removed_whole() {
d.delete_range(&Pos { story: StoryRef::Body, path: first, off: 0 }, &Pos::body(2, 0)).unwrap();
assert!(d.body.get(1).is_some_and(|b| b.as_table().is_some()), "a table with paragraphs the walk never saw was removed");
}
#[test]
fn adopting_a_document_renumbers_what_it_refers_to() {
use crate::para::NoteKind;
use crate::{Comment, Revision, RevisionKind};
// Ours already has a footnote, a comment, a list and a revision with the ids the other's use.
let mut ours = doc(&["ours"]);
let our_note = ours.add_part(PartKind::Footnote, vec![para_block(Paragraph::with_text("our note", CharProps::default()))]);
ours.comments.insert(0, Comment { part: our_note, ..Comment::default() });
let our_list = ours.numbering.add_list(crate::numbering::ListKind::Bullet);
ours.revisions.push(Revision { kind: RevisionKind::Insert, ..Revision::default() });
let mut theirs = doc(&["theirs"]);
let note = theirs.add_part(PartKind::Footnote, vec![para_block(Paragraph::with_text("their note", CharProps::default()))]);
let said = theirs.add_part(PartKind::Comment, vec![para_block(Paragraph::with_text("their comment", CharProps::default()))]);
theirs.comments.insert(0, Comment { author: "Them".into(), part: said, durable_id: Some("0A0B0C0D".into()), ..Comment::default() });
let list = theirs.numbering.add_list(crate::numbering::ListKind::Bullet);
assert_eq!(list, our_list);
theirs.revisions.push(Revision { kind: RevisionKind::Delete, ..Revision::default() });
let mut style = crate::styles::Style { id: "TheirList".into(), name: "Their List".into(), ..Default::default() };
style.para.numbering = Some(crate::props::NumRef { num: list, level: 0 });
theirs.styles.upsert(style);
let p = theirs.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
p.props.numbering = Some(crate::props::NumRef { num: list, level: 0 });
p.runs[0].props.del = Some(0);
// An insertion naming a revision it doesn't have.
p.runs[0].props.ins = Some(7);
let props = p.props_of_char(0).clone();
p.insert_object(0, InlineObject::NoteRef { kind: NoteKind::Footnote, id: note, custom: String::new() }, &props).unwrap();
p.insert_object(0, InlineObject::CommentStart { id: 0 }, &props).unwrap();
let frag = ours.adopt(theirs);
let Some(Block::Para(p)) = frag.blocks.first() else { panic!() };
let new_list = p.props.numbering.unwrap().num;
assert_ne!(new_list, our_list);
assert!(ours.numbering.num(new_list).is_some());
assert_eq!(ours.styles.get("TheirList").and_then(|s| s.para.numbering).map(|n| n.num), Some(new_list));
// Their deletion is their revision, after ours.
let del = p.runs.iter().find_map(|r| r.props.del).unwrap();
assert_eq!(ours.revisions.get(del as usize).map(|r| r.kind), Some(RevisionKind::Delete));
assert!(p.runs.iter().all(|r| r.props.ins.is_none()));
// Their note and comment come along under ids of their own.
let (note_id, comment_id) = p.objects.iter().fold((None, None), |(n, c), o| match o {
InlineObject::NoteRef { id, .. } => (Some(*id), c),
InlineObject::CommentStart { id } => (n, Some(*id)),
_ => (n, c),
});
let text = |id: u32| {
ours.parts
.get(&id)
.map(|p| p.blocks.iter().map(|b| if let Block::Para(p) = &**b { p.text.clone() } else { String::new() }).collect::<String>())
};
assert_eq!(text(note_id.unwrap()).as_deref(), Some("their note"));
let comment = ours.comments.get(&comment_id.unwrap()).unwrap();
assert_eq!((comment.author.as_str(), text(comment.part).as_deref()), ("Them", Some("their comment")));
// Its durable id is the other document's: the writer gives the copy one of its own.
assert_eq!(comment.durable_id, None);
assert_eq!(text(our_note).as_deref(), Some("our note"));
}
#[test]
fn a_kept_character_style_leaves_what_it_doesnt_set_to_the_paragraph_style() {
use crate::styles::{Style, StyleKind, StyleSheet};
let mut ours = doc(&["ours"]);
let mut theirs = doc(&["theirs"]);
theirs.styles = StyleSheet::empty();
theirs.styles.default_chr = CharProps { size: Some(9.0), ..Default::default() };
let big = CharProps { size: Some(20.0), ..Default::default() };
theirs.styles.upsert(Style { id: "Big".into(), name: "Big".into(), chr: big, ..Default::default() });
theirs.styles.upsert(Style { id: "Link".into(), name: "Link".into(), kind: StyleKind::Character, ..Default::default() });
let p = theirs.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
p.props.style = Some("Big".into());
p.runs[0].props.style = Some("Link".into());
let frag = ours.adopt_with(theirs, true);
let Some(Block::Para(p)) = frag.blocks.first() else { panic!() };
assert_eq!(ours.styles.resolve_char(p.props.style.as_deref(), &p.runs[0].props).size, 20.0);
}
#[test]
fn kept_styles_renamed_on_adoption_are_renamed_in_list_levels_too() {
use crate::styles::Style;
let mut ours = doc(&["ours"]);
ours.styles.upsert(Style { id: "Heading1".into(), name: "heading 1".into(), ..Default::default() });
let mut theirs = doc(&["theirs"]);
theirs.styles.upsert(Style { id: "Heading1".into(), name: "heading 1".into(), ..Default::default() });
let list = theirs.numbering.add_list(crate::numbering::ListKind::Numbered);
let abs = theirs.numbering.nums.iter().find(|n| n.id == list).map(|n| n.abstract_id).unwrap();
let a = theirs.numbering.abstracts.iter_mut().find(|a| a.id == abs).unwrap();
a.levels[0].style = Some("Heading1".into());
let level = a.levels[0].clone();
theirs.numbering.nums.iter_mut().find(|n| n.id == list).unwrap().level_overrides.push((1, level));
let before = ours.numbering.abstracts.len();
ours.adopt_with(theirs, true);
let copied = &ours.numbering.abstracts[before..];
let styles: Vec<_> = copied
.iter()
.flat_map(|a| a.levels.iter())
.chain(ours.numbering.nums.iter().flat_map(|n| n.level_overrides.iter().map(|(_, l)| l)))
.filter_map(|l| l.style.clone())
.collect();
assert_eq!(styles.len(), 2, "{styles:?}");
for st in &styles {
assert_ne!(st, "Heading1");
assert!(ours.styles.get(st).is_some(), "{st}");
}
}
#[test]
fn an_adopted_section_breaks_tracked_change_names_its_revision() {
use crate::{Revision, RevisionKind, SectionProps};
let mut ours = doc(&["ours"]);
ours.revisions.push(Revision { kind: RevisionKind::Insert, ..Revision::default() });
let mut theirs = doc(&["theirs"]);
theirs.revisions.push(Revision { kind: RevisionKind::Format, ..Revision::default() });
let p = theirs.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
p.section = Some(Box::new(SectionProps { fmt_change: crate::props::PropChange::boxed(0, SectionProps::default()), ..SectionProps::default() }));
let frag = ours.adopt(theirs);
let Some(Block::Para(p)) = frag.blocks.first() else { panic!() };
let rev = p.section.as_ref().and_then(|s| s.fmt_change.as_ref()).map(|c| c.rev);
assert_eq!(rev.and_then(|r| ours.revisions.get(r as usize)).map(|r| r.kind), Some(RevisionKind::Format));
}
#[test]
fn an_adopted_tracked_format_change_keeps_its_old_styles_and_list() {
use crate::props::{NumRef, ParaProps, PropChange};
use crate::{Revision, RevisionKind};
for keep in [false, true] {
let mut ours = doc(&["ours"]);
let our_list = ours.numbering.add_list(crate::numbering::ListKind::Bullet);
let mut theirs = doc(&["theirs"]);
theirs.revisions.push(Revision { kind: RevisionKind::Format, ..Revision::default() });
let list = theirs.numbering.add_list(crate::numbering::ListKind::Bullet);
assert_eq!(list, our_list);
let p = theirs.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
// Live and old props name the same styles (ours has them too) and their list.
p.props.style = Some("Quote".into());
p.props.numbering = Some(NumRef { num: list, level: 0 });
let old = ParaProps { style: Some("Quote".into()), numbering: Some(NumRef { num: list, level: 0 }), ..ParaProps::default() };
p.props.fmt_change = PropChange::boxed(0, old);
p.runs[0].props.style = Some("Strong".into());
p.runs[0].props.fmt_change = PropChange::boxed(0, CharProps { style: Some("Strong".into()), ..CharProps::default() });
let frag = ours.adopt_with(theirs, keep);
let Some(Block::Para(p)) = frag.blocks.first() else { panic!() };
let old = &p.props.fmt_change.as_ref().unwrap().old;
assert_eq!((&old.style, old.numbering), (&p.props.style, p.props.numbering), "keep_styles {keep}");
assert_ne!(old.numbering.map(|n| n.num), Some(our_list));
let run = &p.runs[0].props;
assert_eq!(run.fmt_change.as_ref().unwrap().old.style, run.style, "keep_styles {keep}");
}
}
#[test]
fn adopted_styles_never_become_our_defaults() {
use crate::styles::{Style, StyleKind};
for keep in [false, true] {
let mut ours = doc(&["ours"]);
let mut theirs = doc(&["theirs"]);
// A localized default character style, and one whose id we have too.
theirs.styles.upsert(Style { id: "Absatz".into(), kind: StyleKind::Character, default: true, ..Default::default() });
theirs.styles.upsert(Style { id: "Strong".into(), kind: StyleKind::Character, default: true, ..Default::default() });
ours.adopt_with(theirs, keep);
assert!(ours.styles.styles.iter().all(|s| !s.default), "keep_styles {keep}");
}
}
#[test]
fn adopted_permission_ranges_keep_ids_of_their_own() {
let perm = |id, editor: &str| InlineObject::PermStart { id, group: String::new(), editor: editor.into() };
let mut ours = doc(&["ours"]);
ours.insert_object(&Pos::body(0, 4), InlineObject::PermEnd { id: 0 }, &CharProps::default()).unwrap();
ours.insert_object(&Pos::body(0, 0), perm(0, "us"), &CharProps::default()).unwrap();
// Theirs: a closed range and one left open, numbered from 0 like ours.
let mut theirs = doc(&["theirs"]);
theirs.insert_object(&Pos::body(0, 6), perm(1, "them"), &CharProps::default()).unwrap();
theirs.insert_object(&Pos::body(0, 6), InlineObject::PermEnd { id: 0 }, &CharProps::default()).unwrap();
theirs.insert_object(&Pos::body(0, 0), perm(0, "them"), &CharProps::default()).unwrap();
let frag = ours.adopt(theirs);
let Some(Block::Para(p)) = frag.blocks.first() else { panic!() };
let ids: Vec<u32> = p
.objects
.iter()
.filter_map(|o| match o {
InlineObject::PermStart { id, .. } | InlineObject::PermEnd { id } => Some(*id),
_ => None,
})
.collect();
assert_eq!(ids.len(), 3);
assert!(!ids.contains(&0), "{ids:?}");
assert_eq!(ids[0], ids[1], "a range's start and end still pair: {ids:?}");
assert_ne!(ids[1], ids[2]);
}
#[test]
fn an_adopted_notes_text_box_gets_a_story_of_its_own() {
use crate::para::NoteKind;
// Ours has a text box under the id their note's text box has.
let mut ours = doc(&["ours"]);
ours.add_part(PartKind::Footnote, Vec::new());
let our_box = text_box(&mut ours, &Pos::body(0, 0), "our box");
let mut theirs = doc(&["theirs"]);
let note = theirs.add_part(PartKind::Footnote, vec![para_block(Paragraph::with_text("note", CharProps::default()))]);
let their_box = theirs.add_part(PartKind::TextBox, vec![para_block(Paragraph::with_text("their box", CharProps::default()))]);
assert_eq!(their_box, our_box);
theirs
.insert_object(&Pos { story: StoryRef::Part(note), path: Path::top(0), off: 0 }, box_shape(Some(their_box)), &CharProps::default())
.unwrap();
theirs
.insert_object(&Pos::body(0, 0), InlineObject::NoteRef { kind: NoteKind::Footnote, id: note, custom: String::new() }, &CharProps::default())
.unwrap();
let blocks = ours.adopt(theirs).blocks;
let Some(Block::Para(p)) = blocks.first() else { panic!() };
let new_note = p.objects.iter().find_map(|o| if let InlineObject::NoteRef { id, .. } = o { Some(*id) } else { None }).unwrap();
let story = ours.parts[&new_note]
.blocks
.iter()
.find_map(|b| if let Block::Para(p) = &**b { p.objects.iter().find_map(|o| o.text_boxes().first().copied()) } else { None })
.unwrap();
assert_ne!(story, our_box);
let part = &ours.parts[&story];
assert_eq!(part.kind, PartKind::TextBox);
assert!(matches!(part.blocks.first().map(|b| &**b), Some(Block::Para(p)) if p.text == "their box"));
}
+1
View File
@@ -48,6 +48,7 @@ pub mod symbol_fonts;
pub mod table;
pub mod wordart;
pub mod wrap;
pub mod zip_guard;
use std::collections::BTreeMap;
use std::sync::Arc;
+54 -2
View File
@@ -338,6 +338,23 @@ impl Counters {
*f = LevelFlags { started: false, first: Some(*start) };
}
}
// A level above that hasn't counted yet counts its first number here, as in Word: after
// 1.1 and 1.2 with no level-1 item before them, the next level-1 item is 2, whether or not
// the deeper level's text shows it.
for (k, (v, f)) in st.iter_mut().zip(fl.iter_mut()).enumerate().take(lv) {
if f.started {
continue;
}
let start = match self.seen.entry((num, k as u8)) {
std::collections::hash_map::Entry::Vacant(e) => {
e.insert(true);
n.start_overrides.iter().find(|(l, _)| *l as usize == k).map(|(_, s)| *s)
}
std::collections::hash_map::Entry::Occupied(_) => None,
};
*v = start.or(f.first).unwrap_or_else(|| level_def(k).map_or(1, |l| l.start));
*f = LevelFlags { started: true, first: None };
}
if let (Some(v), Some(f)) = (st.get_mut(lv), fl.get_mut(lv)) {
*v = if f.started { v.saturating_add(1) } else { f.first.unwrap_or(def.start) };
*f = LevelFlags { started: true, first: None };
@@ -426,8 +443,9 @@ mod tests {
assert_eq!(c.next_label(&n, id, 2).unwrap().0, "1.02");
assert_eq!(c.next_label(&n, id, 1).unwrap().0, "");
assert_eq!(c.next_label(&n, id, 2).unwrap().0, "2.01");
// Levels it doesn't define still come from the abstract list.
assert_eq!(c.next_label(&n, id, 0).unwrap().0, "1.");
// Levels it doesn't define still come from the abstract list; level 0 took its first number
// with the first level-1 item, so this is its second.
assert_eq!(c.next_label(&n, id, 0).unwrap().0, "2.");
assert_eq!(n.level(id, 2).map(|l| l.format), Some(NumFormat::DecimalZero));
}
@@ -476,6 +494,40 @@ mod tests {
assert_eq!(l(2), "(1)", "level 1 restarts level 3");
}
#[test]
fn a_deeper_item_first_counts_the_levels_above_it() {
// Measured in Word: a level above that hasn't counted yet takes its first number at the
// first deeper item, whether or not that item's text shows it.
let labels_with = |starts: [u32; 3], texts: [&str; 3], overrides: &[(u8, u32)], seq: &[u8]| {
let mut n = Numbering::default();
let id = n.add_list(ListKind::Legal);
if let Some(a) = n.abstracts.first_mut() {
for (l, (s, t)) in a.levels.iter_mut().zip(starts.into_iter().zip(texts)) {
l.start = s;
l.text = t.into();
}
}
if let Some(num) = n.nums.iter_mut().find(|x| x.id == id) {
num.start_overrides = overrides.to_vec();
}
let mut c = Counters::default();
seq.iter().map(|lv| c.next_label(&n, id, *lv).unwrap().0).collect::<Vec<_>>()
};
let labels = |starts: [u32; 3], texts: [&str; 3], seq: &[u8]| labels_with(starts, texts, &[], seq);
let std = ["%1.", "%1.%2.", "%1.%2.%3."];
assert_eq!(labels([1, 1, 1], std, &[1, 1, 0, 0]), ["1.1.", "1.2.", "2.", "3."]);
assert_eq!(labels([10, 1, 1], std, &[1, 1, 1, 1, 0, 0]), ["10.1.", "10.2.", "10.3.", "10.4.", "11.", "12."]);
assert_eq!(labels([1, 1, 1], ["%1.", "(%2)", "%1.%2.%3."], &[1, 1, 0, 0]), ["(1)", "(2)", "2.", "3."]);
assert_eq!(labels([1, 1, 1], std, &[2, 1, 0, 1]), ["1.1.1.", "1.2.", "2.", "2.1."]);
assert_eq!(labels([1, 1, 1], ["%1.", "%1.%2.", "(%3)"], &[2, 1, 0]), ["(1)", "1.2.", "2."]);
assert_eq!(labels([1, 1, 1], std, &[0, 2, 1, 0]), ["1.", "1.1.1.", "1.2.", "2."]);
// The num's own start for the level above counts in its place, as the level's start does.
assert_eq!(labels_with([1, 1, 1], std, &[(0, 10)], &[1, 1, 0]), ["10.1.", "10.2.", "11."]);
assert_eq!(labels_with([1, 1, 1], std, &[(0, 5)], &[1, 0, 1]), ["5.1.", "6.", "6.1."]);
// A level-1 item after the first deeper ones restarts level 2 below it, as usual.
assert_eq!(labels([1, 1, 1], std, &[1, 1, 0, 1, 1, 0, 2]), ["1.1.", "1.2.", "2.", "2.1.", "2.2.", "3.", "3.1.1."]);
}
#[test]
fn a_level_starting_at_zero_counts_up() {
let mut n = Numbering::default();
+19
View File
@@ -448,6 +448,25 @@ fn box_para(story: u32, id: u32) -> Blocks {
vec![para_block(p)]
}
#[test]
fn adopting_relinks_lists_in_deeply_nested_tables() {
// As #1339: a list paragraph 20 tables down takes the list it is brought in with.
let mut ours = Document::from_text("ours");
let our_list = ours.numbering.add_list(numbering::ListKind::Bullet);
let mut theirs = Document::from_text("theirs");
let list = theirs.numbering.add_list(numbering::ListKind::Bullet);
assert_eq!(list, our_list);
let mut p = Paragraph::with_text("item", CharProps::default());
p.props.numbering = Some(props::NumRef { num: list, level: 0 });
theirs.body = nested_tables(20, vec![para_block(p)]);
let mut nums = Vec::new();
for b in &ours.adopt(theirs).blocks {
edit::each_para(b, 0, &mut |p| nums.extend(p.props.numbering.map(|n| n.num)));
}
assert_eq!(nums.len(), 1);
assert_ne!(nums[0], our_list);
}
#[test]
fn text_boxes_shown_from_deeply_nested_tables_are_kept() {
// #1339: the paragraph walk stopped 16 tables down, so a box shown only from deeper was taken
+191
View File
@@ -0,0 +1,191 @@
//! A bytes-only check of a ZIP package's declared entry count before the `zip` crate opens it
//! (#3660). Every reader of `.docx`, `.odt` and spreadsheet packages calls [`check`] first.
//!
//! `zip::ZipArchive::new` reserves room for, and parses, as many central directory headers as the
//! end record declares before its caller can look at `len()`. A ZIP64 end record can declare
//! millions of entries for a file of a few hundred megabytes (only 46 bytes each have to fit), and
//! each parsed entry costs several times that in memory. Here the end records are found the way
//! `zip` finds them (APPNOTE.TXT §4.3.14 to §4.3.16), and a package declaring more than the
//! reader's limit is refused before `zip` parses anything. Anything else that is wrong is left for
//! `zip` to report.
const EOCD: &[u8; 4] = b"PK\x05\x06";
const EOCD64_LOCATOR: &[u8; 4] = b"PK\x06\x07";
const EOCD64: &[u8; 4] = b"PK\x06\x06";
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
/// End records looked at, at most. A package has one; a few more can turn up inside stored data.
const MAX_END_RECORDS: usize = 16;
/// Refuse `bytes` when the end record `zip` would use declares more than `max_entries` entries.
///
/// `zip` searches the file backwards for an end of central directory record. A ZIP64 one points
/// through its locator at the ZIP64 end record, which `zip` looks for from the locator's offset
/// on: every such record there is checked, and as `zip` may fall back to an earlier end record
/// when one doesn't parse, the search goes on. A plain one is used when it declares no entries, or
/// a central directory header lies between its directory offset and itself: it is the last one
/// looked at.
pub fn check(bytes: &[u8], max_entries: u64) -> Result<(), String> {
let u16_at = |at: usize| bytes.get(at..at.checked_add(2)?).map(|b| u16::from_le_bytes([b[0], b[1]]));
let u32_at = |at: usize| bytes.get(at..at.checked_add(4)?).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]));
let u64_at = |at: usize| bytes.get(at..at.checked_add(8)?).and_then(|b| <[u8; 8]>::try_from(b).ok()).map(u64::from_le_bytes);
let refuse = |n: u64| -> Result<(), String> {
if n > max_entries { Err(format!("the package declares {n} entries, more than {max_entries}")) } else { Ok(()) }
};
let mut end = bytes.len();
let mut seen = 0;
while let Some(at) = bytes.get(..end).and_then(|b| b.windows(4).rposition(|w| w == EOCD)) {
end = at;
seen += 1;
if seen > MAX_END_RECORDS {
return Err(format!("more than {MAX_END_RECORDS} end of central directory records"));
}
// `zip` skips a record whose comment runs past the end of the file.
let Some(comment) = u16_at(at + 20) else { continue };
if at + 22 + usize::from(comment) > bytes.len() {
continue;
}
let (Some(on_disk), Some(total), Some(cd_offset)) = (u16_at(at + 8), u16_at(at + 10), u32_at(at + 16)) else { continue };
let zip64 = total == u16::MAX || cd_offset == u32::MAX;
if zip64
&& let Some(locator) = at.checked_sub(20)
&& bytes.get(locator..locator + 4) == Some(EOCD64_LOCATOR.as_slice())
{
let (Some(eocd64), Some(disks)) = (u64_at(locator + 8), u32_at(locator + 16)) else { continue };
let Some(from) = usize::try_from(eocd64).ok().filter(|&f| f < locator) else { continue };
if disks > 1 {
continue;
}
let records = bytes.get(from..locator).unwrap_or_default().windows(4).enumerate().filter(|(_, w)| *w == EOCD64);
for (i, _) in records {
let p = from + i;
refuse(u64_at(p + 24).unwrap_or(0).max(u64_at(p + 32).unwrap_or(0)))?;
}
continue;
}
if total == 0 {
return Ok(());
}
let cd = cd_offset as usize;
if cd < at && bytes.get(cd..at).is_some_and(|b| b.windows(4).any(|w| w == CENTRAL_HEADER)) {
return refuse(u64::from(on_disk.max(total)));
}
}
Ok(())
}
/// A package of one empty stored file, `a`, whose central directory lists it `entries` times (at
/// most as many as fit in 64 MiB) under a ZIP64 end record that declares them all: `zip` 3.0.0
/// parses and keeps every one of them before its caller can count them. Regression input for the
/// readers' tests: built only for tests, this crate's and (through the `test-support` feature,
/// which they turn on as a dev-dependency) the readers', never into the app.
#[cfg(any(test, feature = "test-support"))]
pub fn zip64_entry_count_sample(entries: u64) -> Vec<u8> {
// Local file header (APPNOTE.TXT §4.3.7): version 10, stored, no data, name "a".
let mut f = b"PK\x03\x04".to_vec();
f.extend_from_slice(&[10, 0, 0, 0, 0, 0, 0, 0, 0x21, 0]); // version, flags, method, time, date
f.extend_from_slice(&[0; 12]); // crc, sizes
f.extend_from_slice(&[1, 0, 0, 0]); // name and extra lengths
f.push(b'a');
let cd_at = f.len() as u64;
// Its central header (§4.3.12), 47 bytes.
let mut header = CENTRAL_HEADER.to_vec();
header.extend_from_slice(&[10, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0x21, 0]); // versions, flags, method, time, date
header.extend_from_slice(&[0; 12]); // crc, sizes
header.extend_from_slice(&[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); // name, extra, comment, disk, attributes
header.extend_from_slice(&[0; 4]); // local header offset
header.push(b'a');
let count = entries.min((64 << 20) / header.len() as u64);
for _ in 0..count {
f.extend_from_slice(&header);
}
let cd_len = f.len() as u64 - cd_at;
let eocd64_at = f.len() as u64;
// ZIP64 end of central directory record (§4.3.14), 56 bytes.
f.extend_from_slice(EOCD64);
f.extend_from_slice(&44u64.to_le_bytes()); // size of the rest of the record
f.extend_from_slice(&[45, 0, 45, 0]); // versions
f.extend_from_slice(&[0; 8]); // this disk, disk with the central directory
f.extend_from_slice(&entries.to_le_bytes()); // entries on this disk
f.extend_from_slice(&entries.to_le_bytes()); // entries
f.extend_from_slice(&cd_len.to_le_bytes());
f.extend_from_slice(&cd_at.to_le_bytes());
// Its locator (§4.3.15), 20 bytes.
f.extend_from_slice(EOCD64_LOCATOR);
f.extend_from_slice(&0u32.to_le_bytes());
f.extend_from_slice(&eocd64_at.to_le_bytes());
f.extend_from_slice(&1u32.to_le_bytes());
// End of central directory record (§4.3.16), 22 bytes, each count and offset "see ZIP64".
f.extend_from_slice(EOCD);
f.extend_from_slice(&[0; 4]);
f.extend_from_slice(&[0xFF; 12]);
f.extend_from_slice(&[0; 2]);
f
}
#[cfg(test)]
mod tests {
use super::{CENTRAL_HEADER, EOCD, check, zip64_entry_count_sample};
/// A plain package's tail: one central header at 0, then an end record declaring `entries`.
fn plain(entries: u16, comment: &[u8]) -> Vec<u8> {
let mut f = CENTRAL_HEADER.to_vec();
f.resize(46, 0);
f.extend_from_slice(EOCD);
f.extend_from_slice(&[0, 0, 0, 0]);
f.extend_from_slice(&entries.to_le_bytes());
f.extend_from_slice(&entries.to_le_bytes());
f.extend_from_slice(&46u32.to_le_bytes());
f.extend_from_slice(&0u32.to_le_bytes());
f.extend_from_slice(&(comment.len() as u16).to_le_bytes());
f.extend_from_slice(comment);
f
}
#[test]
fn a_zip64_end_record_declaring_too_many_entries_is_refused() {
// #3660: `zip` would reserve and parse them all before the reader's cap.
let sample = zip64_entry_count_sample(20_001);
assert!(check(&sample, 20_000).is_err_and(|e| e.contains("20001 entries")));
assert!(check(&zip64_entry_count_sample(5), 20_000).is_ok());
// Junk before the package (a self-extracting stub) doesn't hide it.
let mut prefixed = vec![0x55u8; 4096];
prefixed.extend_from_slice(&sample);
assert!(check(&prefixed, 20_000).is_err());
}
#[test]
fn a_plain_end_record_is_checked_as_zip_reads_it() {
assert!(check(&plain(3, b""), 20_000).is_ok());
assert!(check(&plain(30_000, b"a comment"), 20_000).is_err_and(|e| e.contains("30000 entries")));
// A record whose comment runs past the end is not the one `zip` uses.
let mut cut = plain(30_000, b"");
cut.truncate(cut.len() - 2);
cut.extend_from_slice(&500u16.to_le_bytes());
assert!(check(&cut, 20_000).is_ok());
assert!(check(b"not a package", 20_000).is_ok());
assert!(check(b"", 20_000).is_ok());
}
#[test]
fn a_fake_end_record_after_the_real_one_does_not_hide_it() {
// `zip` skips a last record with no central header before it and uses the one before.
let mut f = zip64_entry_count_sample(1_000_000);
f.extend_from_slice(EOCD);
f.extend_from_slice(&[0, 0, 0, 0]);
f.extend_from_slice(&7u16.to_le_bytes());
f.extend_from_slice(&7u16.to_le_bytes());
f.extend_from_slice(&0u32.to_le_bytes());
f.extend_from_slice(&((f.len() - 12) as u32).to_le_bytes());
f.extend_from_slice(&0u16.to_le_bytes());
assert!(check(&f, 20_000).is_err());
}
#[test]
fn many_end_records_are_refused() {
// Each declares one entry and has no central header before it, so `zip` would try them all.
let record = [0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0];
let f: Vec<u8> = (0..20).flat_map(|_| EOCD.iter().copied().chain(record)).collect();
assert!(check(&f, 20_000).is_err_and(|e| e.contains("end of central directory records")));
}
}
+21 -4
View File
@@ -20,7 +20,9 @@ const C_ICO: u16 = 0x2A42; // palette colour
const C_CR: u16 = 0x6870; // explicit COLORREF
const C_HPS: u16 = 0x4A43; // half-points
const C_HPS_POS: u16 = 0x4845; // super/subscript by half-points
const C_FTC0: u16 = 0x4A4F; // font index
const C_FTC0: u16 = 0x4A4F; // font index (ASCII)
const C_FTC1: u16 = 0x4A50; // East Asian font index (sprmCRgFtc1)
const C_FTC_BI: u16 = 0x4A5E; // complex-script font index (sprmCFtcBi)
const C_HIGHLIGHT: u16 = 0x2A0C;
const C_ISTD: u16 = 0x4A30; // character style
@@ -254,6 +256,16 @@ pub(crate) fn apply_char(p: &mut CharProps, prl: &Prl, base: &CharProps, fonts:
p.font = Some(name.clone());
}
}
C_FTC1 => {
if let Some(name) = u16_of(prl).and_then(|i| fonts.get(i as usize)) {
p.font_ea = Some(name.clone());
}
}
C_FTC_BI => {
if let Some(name) = u16_of(prl).and_then(|i| fonts.get(i as usize)) {
p.font_cs = Some(name.clone());
}
}
C_HIGHLIGHT => {
if let Some(h) = HL.get(prl.operand.first().copied().unwrap_or(0) as usize) {
p.highlight = Some(*h);
@@ -417,10 +429,15 @@ pub(crate) fn apply_para(p: &mut ParaProps, prl: &Prl) {
/// Build the model stylesheet: resolve each raw style's chain (base → this) and the ids we
/// reference from paragraphs.
pub(crate) fn stylesheet(raw: &[crate::stsh::RawStyle], fonts: &[String]) -> StyleSheet {
pub(crate) fn stylesheet(raw: &[crate::stsh::RawStyle], fonts: &[String], standard: crate::stsh::StandardFonts) -> StyleSheet {
let mut sheet = StyleSheet::empty();
// Word's base character properties: font index 0 of the table, 10 pt ([MS-DOC] sprmCHps).
sheet.default_chr.font = fonts.first().cloned();
// Word's base character properties: the stylesheet's standard fonts (Aptos in a Word 365
// file, whose font table still starts with Times New Roman, #5066), else font index 0 of the
// table; 10 pt ([MS-DOC] sprmCHps).
let font = |i: Option<u16>| i.and_then(|i| fonts.get(usize::from(i))).cloned();
sheet.default_chr.font = font(standard.ascii).or_else(|| fonts.first().cloned());
sheet.default_chr.font_ea = font(standard.east_asian);
sheet.default_chr.font_cs = font(standard.bidi);
if sheet.default_chr.size.is_none() {
sheet.default_chr.size = Some(10.0);
}
+2 -1
View File
@@ -233,6 +233,7 @@ pub fn read_with_password(bytes: &[u8], password: Option<&str>) -> Result<Docume
Some((f, l)) => stsh::parse(&table, f, l)?,
None => Vec::new(),
};
let standard_fonts = fib.pair(fib::pair::STSHF).map(|(f, l)| stsh::standard_fonts(&table, f, l)).unwrap_or_default();
let chpx_bins = match fib.pair(fib::pair::PLCF_BTE_CHPX) {
Some((f, l)) => Bins::parse(&table, f, l)?,
None => Bins::parse(&[], 0, 0)?,
@@ -247,7 +248,7 @@ pub fn read_with_password(bytes: &[u8], password: Option<&str>) -> Result<Docume
doc.core = core;
}
doc.body.clear();
doc.styles = fmt::stylesheet(&raw_styles, &fonts);
doc.styles = fmt::stylesheet(&raw_styles, &fonts, standard_fonts);
doc.numbering = list::parse(&table, &fib, &fonts);
let sheet: StyleSheet = doc.styles.clone();
let sects = sections::parse(&word, &table, &fib);
+35
View File
@@ -20,6 +20,41 @@ pub(crate) struct RawStyle {
pub(crate) chpx: Vec<u8>,
}
/// The stylesheet's standard fonts, the ones text takes when neither its style nor its own
/// formatting names a font: font-table indexes of the ASCII, East Asian and other
/// (non-East-Asian, non-ASCII) fonts (`Stshif.rgftcStandardChpStsh`, [MS-DOC] §2.9.273) and the
/// complex-script font (`Stshi.ftcBi`, §2.9.272). `None` where the header is too short for one.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub(crate) struct StandardFonts {
pub(crate) ascii: Option<u16>,
pub(crate) east_asian: Option<u16>,
pub(crate) other: Option<u16>,
pub(crate) bidi: Option<u16>,
}
/// `StshiLsd.cbLSD`: the size of a latent style's `LSD` ([MS-DOC] `StshiLsd`).
const LSD_SIZE: u16 = 4;
/// The [`StandardFonts`] in the header (`Stshi`) of the `STSH` at `table[fc..fc+lcb]`: Word 365
/// writes Aptos there while the font table's first entry is still Times New Roman (#5066).
pub(crate) fn standard_fonts(table: &[u8], fc: u32, lcb: u32) -> StandardFonts {
let Some(stsh) = (fc as usize).checked_add(lcb as usize).and_then(|end| table.get(fc as usize..end)) else { return StandardFonts::default() };
let cb_stshi = stsh.get(0..2).map_or(0, |b| u16::from_le_bytes([b[0], b[1]]) as usize);
let stshi = stsh.get(2..).map_or(&[][..], |s| s.get(..cb_stshi.min(s.len())).unwrap_or(&[]));
// Stshif: cstd, cbSTDBaseInFile, flags, stiMaxWhenSaved, istdMaxFixedWhenSaved,
// nVerBuiltInNamesWhenSaved (2 bytes each), then rgftcStandardChpStsh (3 × 2); ftcBi follows.
let at = |o: usize| stshi.get(o..o + 2).map(|b| u16::from_le_bytes([b[0], b[1]]));
// ftcBi is optional, and the latent style data (`StshiLsd`, whose `cbLSD` MUST be 4) follows
// it: a 4 right after it says it is there, a 4 in its place with more data after says it was
// left out. A header that ends right after the field (as Word 97's) has it, a 4 included.
let bidi = match (at(18), at(20)) {
(b, Some(LSD_SIZE) | None) => b,
(Some(LSD_SIZE), Some(_)) => None,
(b, _) => b,
};
StandardFonts { ascii: at(12), east_asian: at(14), other: at(16), bidi }
}
/// Parse the `STSH` at `table[fc..fc+lcb]`.
pub(crate) fn parse(table: &[u8], fc: u32, lcb: u32) -> Result<Vec<RawStyle>, DocbinError> {
let fc = fc as usize;
+75 -2
View File
@@ -421,7 +421,17 @@ fn std_bytes_next(stk: u8, istd_base: u16, istd_next: u16, name: &str, papx: &[u
/// An STSH with the given styles.
fn stsh_bytes(styles: &[Vec<u8>]) -> Vec<u8> {
let stshi = [(styles.len() as u16).to_le_bytes(), 10u16.to_le_bytes()].concat(); // cstd, cbSTDBaseInFile
stsh_bytes_with(styles, None)
}
/// An STSH with the given styles; with `standard`, a full Stshif and ftcBi giving the standard
/// fonts' indexes (ASCII, East Asian, other, complex script), as Word writes them.
fn stsh_bytes_with(styles: &[Vec<u8>], standard: Option<[u16; 4]>) -> Vec<u8> {
let mut stshi = [(styles.len() as u16).to_le_bytes(), 10u16.to_le_bytes()].concat(); // cstd, cbSTDBaseInFile
if let Some(fonts) = standard {
stshi.extend_from_slice(&[0u8; 8]); // flags, stiMaxWhenSaved, istdMaxFixedWhenSaved, nVerBuiltInNamesWhenSaved
stshi.extend(fonts.iter().flat_map(|f| f.to_le_bytes()));
}
let mut v = (stshi.len() as u16).to_le_bytes().to_vec();
v.extend_from_slice(&stshi);
for s in styles {
@@ -532,6 +542,8 @@ struct RichExtra {
prm: u16,
alt_fonts: Vec<Option<&'static str>>,
style_next: Vec<u16>,
/// The stylesheet's standard fonts (ASCII, East Asian, other, complex script).
standard_fonts: Option<[u16; 4]>,
}
fn rich_doc(spec: &RichSpec) -> Vec<u8> {
@@ -571,7 +583,7 @@ fn rich_doc_with(spec: &RichSpec, extra: &RichExtra) -> Vec<u8> {
.enumerate()
.map(|(i, s)| std_bytes_next(s.0, s.1, extra.style_next.get(i).copied().unwrap_or(0x0FFF), s.2, &s.3, &s.4))
.collect();
let stsh = stsh_bytes(&styles);
let stsh = stsh_bytes_with(&styles, extra.standard_fonts);
let fonts: Vec<(&str, Option<&str>)> = spec.fonts.iter().enumerate().map(|(i, n)| (*n, extra.alt_fonts.get(i).copied().flatten())).collect();
let ffn = ffn_bytes_alt(&fonts);
let clx_at = 0x40;
@@ -654,6 +666,67 @@ fn font_and_size_from_style_and_direct() {
assert!(p.runs.first().is_some_and(|r| r.props.font.as_deref() == Some("Arial")), "run font: {:?}", p.runs);
}
/// #5066: the default font is the stylesheet's standard font (Aptos in a Word 365 file), not
/// the font table's first entry (still Times New Roman there); the East Asian and complex-script
/// standard fonts are the defaults for those scripts, and runs name theirs with sprmCRgFtc1 and
/// sprmCFtcBi.
#[test]
fn the_default_font_is_the_stylesheets_standard_font() {
let spec = |runs: Vec<(u32, Vec<u8>)>| RichSpec {
text: "Hello\r",
styles: vec![(1, 0x0FFF, "Normal", Vec::new(), grpprl(&[(0x4A43, &24u16.to_le_bytes())]))],
fonts: vec!["Times New Roman", "Symbol", "Arial", "Aptos", "MS Mincho", "Arial Unicode"],
chpx_runs: runs,
papx_entries: vec![(0, 0, Vec::new())],
};
let with = |standard: Option<[u16; 4]>, runs| {
read(&rich_doc_with(&spec(runs), &RichExtra { standard_fonts: standard, ..Default::default() })).expect("opens")
};
let doc = with(Some([3, 4, 3, 2]), vec![(0, Vec::new())]);
let d = &doc.styles.default_chr;
assert_eq!((d.font.as_deref(), d.font_ea.as_deref(), d.font_cs.as_deref()), (Some("Aptos"), Some("MS Mincho"), Some("Arial")));
let p = para0(&doc);
let resolved = doc.styles.resolve_char(p.props.style.as_deref(), &p.runs[0].props);
assert_eq!(resolved.font, "Aptos", "Normal names no font, so the text takes the standard one");
// A run's own East Asian and complex-script fonts.
let doc = with(Some([3, 4, 3, 2]), vec![(0, grpprl(&[(0x4A50, &5u16.to_le_bytes()), (0x4A5E, &3u16.to_le_bytes())]))]);
let run = &para0(&doc).runs[0].props;
assert_eq!((run.font_ea.as_deref(), run.font_cs.as_deref()), (Some("Arial Unicode"), Some("Aptos")));
// Indexes past the font table, or a header too short to hold them, keep entry 0.
for standard in [Some([99, 99, 99, 99]), None] {
let doc = with(standard, vec![(0, Vec::new())]);
let d = &doc.styles.default_chr;
assert_eq!((d.font.as_deref(), d.font_ea.as_deref(), d.font_cs.as_deref()), (Some("Times New Roman"), None, None), "{standard:?}");
}
}
/// #5066: `ftcBi` is optional in the stylesheet header, and the latent style data (`cbLSD`, 4,
/// then the LSDs) follows it: a header that leaves it out doesn't make `cbLSD` the complex-script
/// font.
#[test]
fn the_complex_script_standard_font_is_read_only_when_present() {
let header = |tail: &[u16]| -> Vec<u8> {
// cstd, cbSTDBaseInFile, flags, stiMaxWhenSaved, istdMaxFixedWhenSaved,
// nVerBuiltInNamesWhenSaved; ASCII, East Asian, other.
let words = [&[0u16, 10, 0, 1, 0, 0, 3, 4, 3][..], tail].concat();
let stshi: Vec<u8> = words.iter().flat_map(|w| w.to_le_bytes()).collect();
[(stshi.len() as u16).to_le_bytes().to_vec(), stshi].concat()
};
let bidi = |tail: &[u16]| {
let b = header(tail);
crate::stsh::standard_fonts(&b, 0, b.len() as u32).bidi
};
assert_eq!(bidi(&[2]), Some(2), "a header that ends with it (Word 97)");
assert_eq!(bidi(&[4]), Some(4), "font 4, at the end of the header");
assert_eq!(bidi(&[2, 4, 0, 0]), Some(2), "followed by the latent style data");
assert_eq!(bidi(&[4, 4, 0, 0]), Some(4), "font 4, followed by the latent style data");
assert_eq!(bidi(&[4, 0, 0]), None, "left out: cbLSD is in its place");
assert_eq!(bidi(&[]), None, "a header too short for it");
let b = header(&[4, 0, 0]);
let fonts = crate::stsh::standard_fonts(&b, 0, b.len() as u32);
assert_eq!((fonts.ascii, fonts.east_asian, fonts.other), (Some(3), Some(4), Some(3)), "the other fonts are read all the same");
}
#[test]
fn paragraph_props_and_heading_style() {
let spec = RichSpec {
+1 -1
View File
@@ -76,7 +76,7 @@ pub fn sniff_xml(bytes: &[u8]) -> XmlKind {
/// Read a Flat OPC file.
pub fn read(bytes: &[u8]) -> Result<Document, DocxError> {
crate::read::read_opened(&unpack(bytes)?)
crate::read::read_opened(&unpack(bytes)?, None)
}
/// The data element of the part being read.
+23 -8
View File
@@ -32,7 +32,7 @@ pub use crypt::{
};
pub use flat::{XmlKind, sniff_xml, write as write_flat};
pub use fonts::{EmbeddedFont, embedded_fonts};
pub use read::read;
pub use read::{AltChunk, ChunkImporter, ReadOptions, read};
pub use smart_art::smart_art_items;
pub use wordml2003_write::write as write_wordml2003;
pub use write::{write, write_as};
@@ -56,11 +56,19 @@ pub fn read_xml(bytes: &[u8]) -> Result<wordcraft_doc::Document, DocxError> {
/// The title (`dc:title`) in a package's core properties part (`docProps/core.xml`, ECMA-376
/// Part 2 §11), trimmed; `None` when the part can't be read or has no title. Only the package's
/// directory and at most `max_part` bytes of that part are read: for listing templates by name
/// without reading the whole package.
pub fn package_title<R: std::io::Read + std::io::Seek>(package: R, max_part: u64) -> Option<String> {
/// directory and at most `max_part` bytes of that part are inflated: for listing templates by
/// name without unpacking the whole package. A package over 64 MiB, or one declaring more entries
/// than `MAX_ENTRIES` (20,000), has none (#3660).
pub fn package_title<R: std::io::Read>(package: R, max_part: u64) -> Option<String> {
use std::io::Read;
let mut zip = zip::ZipArchive::new(package).ok()?;
const MAX_PACKAGE: u64 = 64 << 20;
let mut bytes = Vec::new();
package.take(MAX_PACKAGE + 1).read_to_end(&mut bytes).ok()?;
if bytes.len() as u64 > MAX_PACKAGE {
return None;
}
wordcraft_doc::zip_guard::check(&bytes, package::MAX_ENTRIES as u64).ok()?;
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).ok()?;
let part = zip.by_name("docProps/core.xml").ok()?;
let mut xml = Vec::new();
part.take(max_part).read_to_end(&mut xml).ok()?;
@@ -84,11 +92,18 @@ pub(crate) const CANVAS_SPIN_EXT: &str = "urn:wordcraft:canvas-xfrm";
/// `password` first. An encrypted package without a password gives
/// [`DocxError::PasswordRequired`]; a wrong one, [`DocxError::WrongPassword`].
pub fn read_with_password(bytes: &[u8], password: Option<&str>) -> Result<wordcraft_doc::Document, DocxError> {
read_with(bytes, &ReadOptions { password, ..Default::default() })
}
/// Read a package with options: the password of an encrypted one ([`read_with_password`]) and
/// an importer for `w:altChunk` content in formats other than WordprocessingML (see
/// [`ChunkImporter`]).
pub fn read_with(bytes: &[u8], opts: &ReadOptions) -> Result<wordcraft_doc::Document, DocxError> {
if is_encrypted(bytes) {
let package = zeroize::Zeroizing::new(decrypt(bytes, password)?);
return read::read_package(&package);
let package = zeroize::Zeroizing::new(decrypt(bytes, opts.password)?);
return read::read_package(&package, opts.chunks);
}
read(bytes)
read::read_package(bytes, opts.chunks)
}
/// Which kind of WordprocessingML package to write. The main part's content type differs per
+24 -1
View File
@@ -47,6 +47,7 @@ pub mod rt {
pub const DIAGRAM_COLORS: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/diagramColors";
pub const CUSTOM_XML: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/customXml";
pub const GLOSSARY: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/glossaryDocument";
pub const AF_CHUNK: &str = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/aFChunk";
}
/// Content type of a glossary document part (ECMA-376 Part 1 §11.3.8).
@@ -212,6 +213,8 @@ impl Package {
}
pub fn open_limited(bytes: &[u8], max_part: u64, max_total: u64) -> Result<Package, DocxError> {
// Before `zip` parses every entry the end record declares (#3660).
wordcraft_doc::zip_guard::check(bytes, MAX_ENTRIES as u64).map_err(DocxError::Limit)?;
let mut zip = zip::ZipArchive::new(Cursor::new(bytes)).map_err(|e| DocxError::Zip(e.to_string()))?;
if zip.len() > MAX_ENTRIES {
return Err(DocxError::Limit(format!("{} zip entries", zip.len())));
@@ -252,6 +255,11 @@ impl Package {
Ok(Package::new(files))
}
/// The bytes of all its parts.
pub fn size(&self) -> u64 {
self.files.values().map(|f| f.len() as u64).sum()
}
/// A package from parts already in memory (a Flat OPC file, a converted document), by part
/// name without the leading `/`.
pub fn from_parts(files: BTreeMap<String, Vec<u8>>) -> Package {
@@ -524,6 +532,21 @@ mod tests {
assert!(folded.iter().all(|w| w.contains("2 zip entries")), "{folded:?}");
}
#[test]
fn a_package_declaring_too_many_entries_is_refused_before_zip_reads_them() {
// #3660: a ZIP64 end record declaring more than MAX_ENTRIES.
let bomb = wordcraft_doc::zip_guard::zip64_entry_count_sample(MAX_ENTRIES as u64 + 1);
assert!(matches!(Package::open(&bomb), Err(DocxError::Limit(e)) if e.contains("declares")));
assert_eq!(crate::package_title(std::io::Cursor::new(bomb), 1 << 20), None);
// A real package still opens, and its title still reads.
let z = zip_of(&[(
"docProps/core.xml",
br#"<cp:coreProperties xmlns:cp="http://schemas.openxmlformats.org/package/2006/metadata/core-properties" xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:title>Plan</dc:title></cp:coreProperties>"#.to_vec(),
)]);
assert!(Package::open(&z).is_ok());
assert_eq!(crate::package_title(std::io::Cursor::new(z), 1 << 20).as_deref(), Some("Plan"));
}
#[test]
fn decompression_bomb_is_rejected() {
let z = zip_of(&[("word/document.xml", vec![b' '; 2_000_000])]);
@@ -587,7 +610,7 @@ mod tests {
("word/media/pic.png", b"\x89PNG picture bytes".to_vec()),
]);
let pkg = Package::open(&z).unwrap();
let doc = crate::read::read_opened(&pkg).unwrap();
let doc = crate::read::read_opened(&pkg, None).unwrap();
let held = doc.media.values().next().expect("the picture is read");
assert!(Arc::ptr_eq(held, pkg.get_shared("word/media/pic.png").unwrap()), "the same bytes, not a copy");
assert_eq!(pkg.get("/WORD/media/PIC.png"), Some(&b"\x89PNG picture bytes"[..]), "lookups are unchanged");
+26 -11
View File
@@ -1,6 +1,7 @@
//! DOCX → [`Document`].
pub(crate) mod chart;
mod chunk;
pub(crate) mod diagram;
mod drawing_color;
mod embed;
@@ -13,7 +14,7 @@ mod shape_style;
mod story;
mod wordart;
use std::collections::{BTreeMap, HashMap, HashSet};
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use wordcraft_doc::control::{ControlEvent, ControlKind, control_events, pair_events};
@@ -46,6 +47,9 @@ pub(crate) use meta::PROTECTION_HASH_ATTRS;
pub(crate) use styles::LOWER_NAMES;
use toc::{is_toc, toc_control_to_toc, toc_field_to_heading, unstyle_toc_links};
use chunk::ChunkState;
pub use chunk::{AltChunk, ChunkImporter, ReadOptions};
/// Most fonts read from a font table, and the longest name (hostile files).
const MAX_TABLE_FONTS: usize = 2000;
const MAX_FONT_NAME: usize = 256;
@@ -87,12 +91,12 @@ pub(crate) struct Reader<'p> {
pub bookmarks: HashMap<String, String>,
/// Restrict Editing exception ranges (`w:permStart`): the file's id → ours.
pub perms: HashMap<String, u32>,
/// The next permission range id to give out (chunks' ranges take theirs from it too).
pub next_perm: u32,
/// Chart and diagram parts by path, parsed once.
graphic_parts: HashMap<String, Option<Arc<El>>>,
/// Charts and diagrams by (kind, part path, width bits, height bits), built once.
graphics: HashMap<(GraphicKind, String, u32, u32), Arc<Graphic>>,
/// Graphic work left before charts and diagrams are left empty (see [`MAX_GRAPHIC_WORK`]).
graphic_budget: usize,
/// Drawing ids connectors are glued to (`a:stCxn`, `a:endCxn`), and whether any shape's id
/// was kept while reading (those not glued to are dropped at the end).
pub glued: HashSet<u32>,
@@ -103,6 +107,8 @@ pub(crate) struct Reader<'p> {
content_types: Option<ContentTypes>,
/// Parts kept for charts, diagrams and OLE objects (see `embed`).
embedded: EmbeddedManifest,
/// What alternative format chunks may still bring in, and how (see [`chunk`]).
chunks: ChunkState<'p>,
/// Linked text boxes as read: (`wps:txbx/@id` or `wps:linkedTxbx/@id`, `seq` (0 for the box
/// holding the text), text box part). See [`link_text_boxes`].
pub txbx_links: Vec<(u32, u32, u32)>,
@@ -125,17 +131,24 @@ pub fn read(bytes: &[u8]) -> Result<Document, DocxError> {
if crate::is_encrypted(bytes) {
return crate::read_with_password(bytes, None);
}
read_package(bytes)
read_package(bytes, None)
}
/// [`read`] for bytes known not to be an encrypted package.
pub(crate) fn read_package(bytes: &[u8]) -> Result<Document, DocxError> {
read_opened(&Package::open(bytes)?)
/// [`read`] for bytes known not to be an encrypted package; `chunks` imports alternative format
/// chunks in formats other than WordprocessingML.
pub(crate) fn read_package(bytes: &[u8], chunks: Option<ChunkImporter>) -> Result<Document, DocxError> {
read_opened(&Package::open(bytes)?, chunks)
}
/// Read a package whose parts are already unpacked (from a zip, a Flat OPC file or a converted
/// Word 2003 XML document).
pub(crate) fn read_opened(pkg: &Package) -> Result<Document, DocxError> {
pub(crate) fn read_opened(pkg: &Package, chunks: Option<ChunkImporter>) -> Result<Document, DocxError> {
read_opened_in(pkg, ChunkState::new(chunks)).map(|(doc, ..)| doc)
}
/// [`read_opened`] with what chunks may still bring in; also gives the chunks, chunk bytes and
/// graphic work left of that.
fn read_opened_in(pkg: &Package, chunks: ChunkState) -> Result<(Document, usize, u64, usize), DocxError> {
let root_rels = pkg.rels("");
let main = root_rels.by_type(rt::OFFICE_DOC).filter(|r| !r.external).map(|r| r.target.clone()).unwrap_or_else(|| "word/document.xml".to_string());
let Some(root) = pkg.xml_strict(&main)? else {
@@ -164,9 +177,9 @@ pub(crate) fn read_opened(pkg: &Package) -> Result<Document, DocxError> {
comments_ended: HashSet::new(),
bookmarks: HashMap::new(),
perms: HashMap::new(),
next_perm: 0,
graphic_parts: HashMap::new(),
graphics: HashMap::new(),
graphic_budget: MAX_GRAPHIC_WORK,
glued: HashSet::new(),
shape_ids: false,
main: main.clone(),
@@ -175,6 +188,7 @@ pub(crate) fn read_opened(pkg: &Package) -> Result<Document, DocxError> {
txbx_links: Vec::new(),
theme_fills: Default::default(),
group_fill: None,
chunks,
};
r.pc.major_font = r.doc.settings.major_font.clone();
r.pc.minor_font = r.doc.settings.minor_font.clone();
@@ -267,7 +281,7 @@ pub(crate) fn read_opened(pkg: &Package) -> Result<Document, DocxError> {
if let Some(body) = root.child("w:body") {
let mut sc = StoryCtx::default();
let mut blocks = Blocks::new();
r.read_blocks(&mut sc, body, &rels, &mut blocks, 0);
r.read_blocks(&mut sc, body, &rels, &mut blocks, r.chunks.depth);
r.flush_pending(&mut sc, &mut blocks);
r.doc.body = blocks;
unstyle_toc_links(&mut r.doc);
@@ -321,9 +335,10 @@ pub(crate) fn read_opened(pkg: &Package) -> Result<Document, DocxError> {
if r.shape_ids {
keep_glued_ids(&mut r.doc, &r.glued);
}
let (left, bytes, graphic) = (r.chunks.left, r.chunks.bytes, r.chunks.graphic);
let mut doc = r.doc;
doc.ensure_nonempty();
Ok(doc)
Ok((doc, left, bytes, graphic))
}
/// Most move ranges open at once, and most move names tracked, per file.
+187
View File
@@ -0,0 +1,187 @@
//! `w:altChunk` (ECMA-376 §17.17.2.1): content kept in another format (HTML, MHT, RTF, plain
//! text, another WordprocessingML document) that Word brings into the document where the chunk
//! stands when it opens the file. A nested WordprocessingML package is read here; other formats
//! go to the caller's [`ChunkImporter`] (the format crates sit beside this one), and without one
//! the chunk is left out.
use std::sync::Arc;
use wordcraft_doc::{Block, Blocks, Document};
use super::story::StoryCtx;
use super::{Reader, part_of};
use crate::package::{ContentTypes, MAX_PART, MAX_TOTAL, Package, Rels, rt};
use crate::xml::El;
/// Most chunks brought in from one file (nested documents' chunks included).
pub(crate) const MAX_CHUNKS: usize = 256;
/// Deepest a WordprocessingML chunk may nest in another.
pub(crate) const MAX_CHUNK_NESTING: u8 = 4;
/// An alternative format chunk's content, as a [`ChunkImporter`] gets it.
pub struct AltChunk<'a> {
/// The part's content type (`application/xhtml+xml`, `message/rfc822`, `application/rtf`,
/// `text/plain`…), empty when the package doesn't give one.
pub content_type: &'a str,
/// The part's name (`word/afchunk.mht`): its extension tells the format when the content type
/// doesn't.
pub part: &'a str,
pub bytes: &'a [u8],
}
/// Turns a chunk the docx reader can't read itself into a document whose body takes the chunk's
/// place; `None` leaves the chunk out.
pub type ChunkImporter<'a> = &'a dyn Fn(&AltChunk) -> Option<Document>;
/// How [`crate::read_with`] reads a package.
#[derive(Clone, Copy, Default)]
pub struct ReadOptions<'a> {
/// The password of an encrypted package.
pub password: Option<&'a str>,
/// Imports `w:altChunk` content in formats other than WordprocessingML.
pub chunks: Option<ChunkImporter<'a>>,
}
impl Reader<'_> {
/// Bring the chunk `e` names into `out` where it stands (`depth`: of the XML around it).
pub(crate) fn read_alt_chunk(&mut self, sc: &mut StoryCtx, e: &El, rels: &Rels, out: &mut Blocks, depth: usize) {
let Some(part) = e.attr("r:id").and_then(|id| part_of(rels, id, rt::AF_CHUNK)) else { return };
let pkg = self.pkg;
let Some(bytes) = pkg.get(&part) else {
log::warn!("docx: alternative format chunk {part} is missing");
return;
};
// Every chunk's bytes count against one budget for the whole file, however often the same
// part is named and however deep chunks nest.
if self.chunks.left == 0 || bytes.len() as u64 > self.chunks.bytes {
log::warn!("docx: more than {MAX_CHUNKS} alternative format chunks or {MAX_TOTAL} bytes of them; the rest are left out");
return;
}
self.chunks.left -= 1;
self.chunks.bytes -= bytes.len() as u64;
let ct = self.content_types.get_or_insert_with(|| ContentTypes::read(pkg)).of(&part).unwrap_or("").to_string();
let chunk = AltChunk { content_type: &ct, part: &part, bytes };
// A WordprocessingML chunk is a package; one that isn't Word's fails to read (logged).
let is_docx = bytes.starts_with(b"PK\x03\x04");
let doc = if is_docx { self.read_nested(bytes, depth) } else { self.chunks.import.and_then(|f| f(&chunk)) };
let Some(mut doc) = doc else {
log::warn!("docx: alternative format chunk {part} ({ct}) is in a format WordCraft can't bring in");
return;
};
// The chunk flows in the section it stands in; its own section breaks don't come along.
for b in &mut doc.body {
if let Block::Para(p) = &**b
&& p.section.is_some()
&& let Block::Para(p) = Arc::make_mut(b)
{
p.section = None;
}
}
// `w:matchSrc`: a Word document's content keeps its own styles (other formats' formatting
// is their own already).
let keep = is_docx && e.child("w:altChunkPr").and_then(|p| p.child("w:matchSrc")).is_some_and(crate::units::on_off);
let mut blocks = self.doc.adopt_blocks(doc, keep, &mut self.next_perm);
// Bookmarks and comment marks read before the chunk stay before it.
self.flush_pending_before(sc, out, &mut blocks);
out.extend(blocks.into_iter().map(Arc::new));
}
/// A WordprocessingML chunk: its document, read with the same importer and what is left of
/// the budgets, its XML depth counting on from `depth`.
fn read_nested(&mut self, bytes: &[u8], depth: usize) -> Option<Document> {
if self.chunks.nesting >= MAX_CHUNK_NESTING {
log::warn!("docx: alternative format chunks nest deeper than {MAX_CHUNK_NESTING}; the inner ones are left out");
return None;
}
let pkg =
Package::open_limited(bytes, MAX_PART, self.chunks.bytes).map_err(|e| log::warn!("docx: unreadable nested document chunk: {e}")).ok()?;
self.chunks.bytes = self.chunks.bytes.saturating_sub(pkg.size());
let state = ChunkState { nesting: self.chunks.nesting + 1, depth: depth + 1, ..self.chunks };
let (doc, left, bytes, graphic) =
super::read_opened_in(&pkg, state).map_err(|e| log::warn!("docx: unreadable nested document chunk: {e}")).ok()?;
self.chunks.left = left;
self.chunks.bytes = bytes;
self.chunks.graphic = graphic;
Some(doc)
}
}
/// What a reader may still bring in from chunks.
#[derive(Clone, Copy)]
pub(crate) struct ChunkState<'a> {
/// Chunks left in the file's budget ([`MAX_CHUNKS`]).
pub left: usize,
/// Chunk bytes (nested packages unpacked) left in the file's budget.
pub bytes: u64,
/// Graphic work left in the file's budget ([`super::MAX_GRAPHIC_WORK`]): nested documents'
/// charts and diagrams count against it too.
pub graphic: usize,
/// How deep in nested WordprocessingML chunks this reader is.
pub nesting: u8,
/// The XML depth its stories start at (a nested document's: that of the chunk naming it).
pub depth: usize,
pub import: Option<ChunkImporter<'a>>,
}
impl<'a> ChunkState<'a> {
pub fn new(import: Option<ChunkImporter<'a>>) -> Self {
ChunkState { left: MAX_CHUNKS, bytes: MAX_TOTAL, graphic: super::MAX_GRAPHIC_WORK, nesting: 0, depth: 0, import }
}
}
#[cfg(test)]
mod tests {
use std::io::Write;
use wordcraft_doc::{Block, Document, InlineObject};
use super::ChunkState;
use crate::package::Package;
/// A package whose body is `body`, with the document relationship `rel` (id, type, target) to
/// the part `(name, bytes)`.
fn package(body: &str, (id, ty, target): (&str, &str, &str), (name, bytes): (&str, &[u8])) -> Vec<u8> {
let ns = r#"xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main""#;
let rels =
|inner: &str| format!(r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">{inner}</Relationships>"#);
let rel = |id: &str, ty: &str, target: &str| {
format!(r#"<Relationship Id="{id}" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/{ty}" Target="{target}"/>"#)
};
let files: [(&str, Vec<u8>); 4] = [
("_rels/.rels", rels(&rel("rId1", "officeDocument", "word/document.xml")).into_bytes()),
("word/document.xml", format!(r#"<w:document {ns}><w:body>{body}</w:body></w:document>"#).into_bytes()),
("word/_rels/document.xml.rels", rels(&rel(id, ty, target)).into_bytes()),
(name, bytes.to_vec()),
];
let mut zw = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
for (name, bytes) in files {
zw.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
zw.write_all(&bytes).unwrap();
}
zw.finish().unwrap().into_inner()
}
/// Whether the body's first paragraph's chart is drawn.
fn drawn(d: &Document) -> bool {
let Some(Block::Para(p)) = d.body.first().map(|b| &**b) else { panic!("no paragraph") };
p.objects.iter().any(|o| matches!(o, InlineObject::Graphic { graphic, .. } if !graphic.items.is_empty()))
}
#[test]
fn a_nested_documents_charts_count_against_the_files_graphic_budget() {
let chart = r#"<c:chartSpace xmlns:c="http://schemas.openxmlformats.org/drawingml/2006/chart"><c:chart><c:plotArea><c:barChart><c:ser><c:val><c:numLit><c:ptCount val="2"/><c:pt idx="0"><c:v>1</c:v></c:pt><c:pt idx="1"><c:v>2</c:v></c:pt></c:numLit></c:val></c:ser></c:barChart></c:plotArea></c:chart></c:chartSpace>"#;
let drawing = r#"<w:p><w:r><w:drawing><wp:inline><wp:extent cx="2743200" cy="1828800"/><wp:docPr id="5" name="G"/><a:graphic><a:graphicData uri="http://schemas.openxmlformats.org/drawingml/2006/chart"><c:chart xmlns:c="http://schemas.openxmlformats.org/drawingml/2006/chart" r:id="rIdChart"/></a:graphicData></a:graphic></wp:inline></w:drawing></w:r></w:p>"#;
let inner = package(drawing, ("rIdChart", "chart", "charts/chart1.xml"), ("word/charts/chart1.xml", chart.as_bytes()));
let outer = package(r#"<w:altChunk r:id="rIdC"/>"#, ("rIdC", "aFChunk", "afchunk.docx"), ("word/afchunk.docx", &inner));
let pkg = Package::open(&outer).unwrap();
let read = |graphic| {
let (d, .., left) = super::super::read_opened_in(&pkg, ChunkState { graphic, ..ChunkState::new(None) }).unwrap();
(drawn(&d), left)
};
let (shown, left) = read(super::super::MAX_GRAPHIC_WORK);
assert!(shown);
assert!(left < super::super::MAX_GRAPHIC_WORK, "the nested chart's work is the file's");
// With the file's budget spent, the nested document doesn't get one of its own.
assert_eq!(read(0), (false, 0));
}
}
+15 -17
View File
@@ -1,6 +1,6 @@
use super::{
Arc, BTreeMap, Blocks, Comment, DocxError, El, HashMap, HashSet, MAX_PARTS, MAX_SOURCES, MAX_TEMPLATE_PATH, NoteKind, NumFormat, Paragraph,
PartKind, Reader, Rels, Seen, StoryCtx, Zoom, ZoomFit, int, math, on_off, para_block, props, tw, u32_of,
Arc, Blocks, Comment, DocxError, El, HashMap, MAX_PARTS, MAX_SOURCES, MAX_TEMPLATE_PATH, NoteKind, NumFormat, Paragraph, PartKind, Reader, Rels,
Seen, StoryCtx, Zoom, ZoomFit, int, math, on_off, para_block, props, tw, u32_of,
};
impl Reader<'_> {
@@ -120,7 +120,7 @@ impl Reader<'_> {
let mut sc = StoryCtx::default();
let mut blocks = Blocks::new();
self.current_note = Some((kind, id));
self.read_blocks(&mut sc, n, &rels, &mut blocks, 0);
self.read_blocks(&mut sc, n, &rels, &mut blocks, self.chunks.depth);
self.flush_pending(&mut sc, &mut blocks);
self.current_note = None;
if let Some(p) = self.doc.parts.get_mut(&id)
@@ -152,7 +152,7 @@ impl Reader<'_> {
let mut sc = StoryCtx::default();
let mut blocks = Blocks::new();
self.in_note_sep = true;
self.read_blocks(&mut sc, n, rels, &mut blocks, 0);
self.read_blocks(&mut sc, n, rels, &mut blocks, self.chunks.depth);
self.flush_pending(&mut sc, &mut blocks);
self.in_note_sep = false;
if blocks.is_empty() {
@@ -188,11 +188,7 @@ impl Reader<'_> {
let rels = self.pkg.rels(path);
let mut by_para: HashMap<String, u32> = HashMap::new();
let mut next_id = 0u32;
let mut entries: BTreeMap<u32, Comment> = BTreeMap::new();
// Ids given before this part, looked up rather than scanned for every generated id: a
// scan per comment made many non-numeric ids quadratic (#3692). Ids given here are in
// `entries`.
let given: HashSet<u32> = self.comment_map.values().copied().collect();
// Straight into the document, beside the comments chunks in notes brought in.
for c in root.children("w:comment") {
if self.doc.parts.len() >= MAX_PARTS {
break;
@@ -202,23 +198,26 @@ impl Reader<'_> {
continue;
}
let id = match file_id.parse::<u32>() {
Ok(v) if !entries.contains_key(&v) => v,
Ok(v) if !self.doc.comments.contains_key(&v) => v,
_ => {
while entries.contains_key(&next_id) || given.contains(&next_id) {
// Every id handed out so far is a key of `doc.comments` (inserted below).
while self.doc.comments.contains_key(&next_id) {
next_id += 1;
}
next_id
}
};
// Taken before its text is read: a chunk there brings comments of its own.
self.doc.comments.insert(id, Comment::default());
let mut sc = StoryCtx::default();
let mut blocks = Blocks::new();
self.read_blocks(&mut sc, c, &rels, &mut blocks, 0);
self.read_blocks(&mut sc, c, &rels, &mut blocks, self.chunks.depth);
self.flush_pending(&mut sc, &mut blocks);
let part = self.doc.add_part(PartKind::Comment, blocks);
if let Some(pid) = c.children("w:p").last().and_then(|p| p.attr("w14:paraId")) {
by_para.insert(pid.to_string(), id);
}
entries.insert(
self.doc.comments.insert(
id,
Comment {
author: c.attr("w:author").unwrap_or("").to_string(),
@@ -239,7 +238,7 @@ impl Reader<'_> {
let Some(cid) = e.attr("w15:paraId").and_then(|p| by_para.get(p)).copied() else { continue };
let parent = e.attr("w15:paraIdParent").and_then(|p| by_para.get(p)).copied().filter(|p| *p != cid);
let done = e.attr("w15:done").is_some_and(|v| v == "1" || v == "true");
if let Some(c) = entries.get_mut(&cid) {
if let Some(c) = self.doc.comments.get_mut(&cid) {
c.parent = parent;
c.resolved = done;
}
@@ -258,7 +257,7 @@ impl Reader<'_> {
for e in x.els().filter(|e| e.local() == "commentId") {
let Some(cid) = local(e, "paraId").and_then(|p| by_para.get(&p).copied()) else { continue };
let Some(durable) = local(e, "durableId").filter(|d| crate::write::durable_id_ok(d)) else { continue };
if let Some(c) = entries.get_mut(&cid) {
if let Some(c) = self.doc.comments.get_mut(&cid) {
c.durable_id = Some(durable.to_ascii_uppercase());
by_durable.insert(durable.to_ascii_uppercase(), cid);
}
@@ -270,12 +269,11 @@ impl Reader<'_> {
for e in x.els().filter(|e| e.local() == "commentExtensible") {
let Some(cid) = local(e, "durableId").and_then(|d| by_durable.get(&d.to_ascii_uppercase()).copied()) else { continue };
let date = local(e, "dateUtc").filter(|d| d.len() <= 40 && d.chars().all(|c| c.is_ascii_alphanumeric() || ":-.+".contains(c)));
if let (Some(c), Some(date)) = (entries.get_mut(&cid), date) {
if let (Some(c), Some(date)) = (self.doc.comments.get_mut(&cid), date) {
c.date_utc = Some(date);
}
}
}
self.doc.comments = entries;
Ok(())
}
+1 -1
View File
@@ -144,7 +144,7 @@ impl Reader<'_> {
sc.in_hf = true;
let mut blocks = Blocks::new();
self.hf_depth += 1;
self.read_blocks(&mut sc, &root, &rels, &mut blocks, 0);
self.read_blocks(&mut sc, &root, &rels, &mut blocks, self.chunks.depth);
self.hf_depth = self.hf_depth.saturating_sub(1);
self.flush_pending(&mut sc, &mut blocks);
if let Some(p) = self.doc.parts.get_mut(&id)
+7 -3
View File
@@ -166,9 +166,9 @@ fn children_of_choice(ac: &El) -> Option<&El> {
ac.children("mc:Choice").find(|c| ok(c)).or_else(|| ac.child("mc:Fallback")).or_else(|| ac.child("mc:Choice"))
}
/// Gather `name` children (rows, cells), looking through `w:sdt`/`w:customXml` wrappers; each
/// with the content controls that open before it and how many close after it. `pending` holds
/// controls opened but not yet given to an element.
/// Gather `name` children (rows, cells), looking through content controls, custom XML and the
/// selected alternate-content branch; each with the content controls that open before it and how
/// many close after it. `pending` holds controls opened but not yet given to an element.
fn collect<'a>(e: &'a El, name: &str, out: &mut Vec<(&'a El, ControlWrap)>, pending: &mut Vec<ContentControl>, depth: usize) {
if depth > 8 {
return;
@@ -194,6 +194,10 @@ fn collect<'a>(e: &'a El, name: &str, out: &mut Vec<(&'a El, ControlWrap)>, pend
}
} else if k.name == "w:customXml" {
collect(k, name, out, pending, depth + 1);
} else if k.name == "mc:AlternateContent"
&& let Some(branch) = children_of_choice(k)
{
collect(branch, name, out, pending, depth + 1);
}
}
}
+25 -2
View File
@@ -51,6 +51,7 @@ impl Reader<'_> {
}
}
"w:sdt" => self.read_block_sdt(sc, e, rels, out, depth),
"w:altChunk" => self.read_alt_chunk(sc, e, rels, out, depth),
"w:customXml" | "w:smartTag" | "w:ins" | "w:moveTo" => self.read_blocks(sc, e, rels, out, depth + 1),
// Math is allowed straight in a story (EG_BlockLevelElts takes EG_RunLevelElements,
// which takes EG_MathContent): it gets a paragraph of its own (#1273).
@@ -135,6 +136,21 @@ impl Reader<'_> {
sc.pending.clear();
}
/// Markers read before content placed whole (a chunk's `next`) stay before it: at the end of
/// the paragraph before, or after a table or with none, at the start of the content's first
/// paragraph, as a paragraph read there would take them (with a table first, the next
/// paragraph read takes them).
pub fn flush_pending_before(&mut self, sc: &mut StoryCtx, out: &mut Blocks, next: &mut [Block]) {
if out.last().is_some_and(|b| matches!(**b, Block::Para(_))) {
return self.flush_pending(sc, out);
}
if let Some(Block::Para(p)) = next.first_mut() {
for o in sc.pending.drain(..).rev() {
let _ = p.insert_object(0, o, &CharProps::default());
}
}
}
pub(super) fn marker(&mut self, e: &El) -> Option<InlineObject> {
let id = e.attr("w:id").map(str::trim).unwrap_or("").to_string();
match e.name.as_str() {
@@ -161,8 +177,15 @@ impl Reader<'_> {
if self.perms.len() >= 100_000 {
return None;
}
let n = self.perms.len() as u32;
let id = *self.perms.entry(id).or_insert(n);
let id = match self.perms.get(&id) {
Some(n) => *n,
None => {
let n = self.next_perm;
self.next_perm = n.saturating_add(1);
self.perms.insert(id, n);
n
}
};
let text = |a: &str| e.attr(a).filter(|v| v.len() <= 256).unwrap_or("").to_string();
let (group, editor) = (text("w:edGrp"), text("w:ed"));
if group.is_empty() && editor.is_empty() {
+23 -7
View File
@@ -218,8 +218,8 @@ impl Reader<'_> {
if let Some(g) = self.graphics.get(&key) {
return g.clone();
}
let items = if self.graphic_budget == 0 { Vec::new() } else { self.graphic_items(kind, &key.1, w, h) };
self.graphic_budget = self.graphic_budget.saturating_sub(graphic_work(&items));
let items = if self.chunks.graphic == 0 { Vec::new() } else { self.graphic_items(kind, &key.1, w, h) };
self.chunks.graphic = self.chunks.graphic.saturating_sub(graphic_work(&items));
// A chart or SmartArt that is exactly what WordCraft writes for its model can be edited
// (and is written from the model); any other stays as it was read.
let chart = match kind {
@@ -499,10 +499,26 @@ impl Reader<'_> {
.and_then(|g| preset_adj(g, "adj"))
.and_then(|v| u32::try_from(v.clamp(0, i64::from(wordcraft_doc::connector::MAX_CORNER))).ok());
}
// Arrowheads: any end type but `none` (ECMA-376 §20.1.10.33) is drawn as a triangle.
let arrow = |tag: &str| ln.and_then(|l| l.child(tag)).and_then(|e| e.attr("type")).is_some_and(|t| t != "none");
extra.arrow_start = arrow("a:headEnd");
extra.arrow_end = arrow("a:tailEnd");
// Arrowheads: any end type but `none` (ECMA-376 §20.1.10.33) is drawn as a triangle of its
// width and length.
let arrow = |tag: &str| {
let e = ln.and_then(|l| l.child(tag))?;
let kind = e.attr("type").filter(|t| *t != "none")?;
let size = |name: &str| wordcraft_doc::connector::ArrowSize::from_ooxml(e.attr(name).unwrap_or("med"));
Some(wordcraft_doc::connector::ArrowStyle {
kind: wordcraft_doc::connector::ArrowKind::from_ooxml(kind),
width: size("w"),
length: size("len"),
})
};
if let Some(style) = arrow("a:headEnd") {
extra.arrow_start = true;
extra.start_style = style;
}
if let Some(style) = arrow("a:tailEnd") {
extra.arrow_end = true;
extra.end_style = style;
}
// A connector's glued ends (ECMA-376 §20.1.2.2.36, §20.1.2.2.13).
if let Some(cn) = wsp.child("wps:cNvCnPr") {
let end = |tag: &str| {
@@ -540,7 +556,7 @@ impl Reader<'_> {
let mut blocks = Blocks::new();
// Shapes in the text aren't members of a group the box may be in.
let group = self.group_fill.take();
self.read_blocks(&mut inner, content, rels, &mut blocks, 0);
self.read_blocks(&mut inner, content, rels, &mut blocks, self.chunks.depth);
self.group_fill = group;
self.flush_pending(&mut inner, &mut blocks);
self.doc.add_part(PartKind::TextBox, blocks)
+34 -4
View File
@@ -295,11 +295,41 @@ impl Reader<'_> {
}
let kind = kind?;
let mut extra = ShapeExtra { cap, ..Default::default() };
// A horizontal rule (`o:hr`, a pasted `<hr>`): across `o:hrpct` tenths of a percent of its
// line, or its own width (`o:hrpct="0"`); a width of 0 with no percentage is the whole
// line. `o:hralign` places a shorter one (Word, measured: 50 % centred, left, right).
// Only a rule in the text, shown: a positioned or hidden one stays a plain shape, which
// keeps its placement and visibility when saved.
if attr("o:hr").is_some_and(|v| matches!(v.trim(), "t" | "true" | "1")) && float.wrap == Wrap::Inline && !float.hidden {
let tenths = attr("o:hrpct").and_then(|v| v.trim().parse::<f32>().ok()).filter(|p| p.is_finite() && *p > 0.0);
let pct = tenths.map_or(if w > 0.0 { 0.0 } else { 100.0 }, |t| (t / 10.0).min(100.0));
let align = match attr("o:hralign").map(|a| a.trim().to_ascii_lowercase()).as_deref() {
Some("center") => wordcraft_doc::props::Align::Center,
Some("right") => wordcraft_doc::props::Align::Right,
_ => wordcraft_doc::props::Align::Left,
};
let standard = attr("o:hrstd").is_some_and(|v| matches!(v.trim(), "t" | "true" | "1"));
extra.hr = Some(wordcraft_doc::connector::HorizontalRule { pct, align, standard });
}
if kind == ShapeKind::Line {
// Arrowheads (`v:stroke` startarrow / endarrow): any but `none` is drawn.
let arrow = |name: &str| shape.child("v:stroke").and_then(|s| s.attr(name)).is_some_and(|v| v != "none");
extra.arrow_start = arrow("startarrow");
extra.arrow_end = arrow("endarrow");
// Arrowheads (`v:stroke` startarrow / endarrow, with their width and length): any but
// `none` is drawn.
use wordcraft_doc::connector::{ArrowKind, ArrowSize, ArrowStyle};
let stroke = shape.child("v:stroke");
let arrow = |end: &str| {
let s = stroke?;
let kind = s.attr(&format!("{end}arrow")).filter(|v| *v != "none")?;
let size = |what: &str| ArrowSize::from_vml(s.attr(&format!("{end}arrow{what}")).unwrap_or("medium"));
Some(ArrowStyle { kind: ArrowKind::from_vml(kind), width: size("width"), length: size("length") })
};
if let Some(style) = arrow("start") {
extra.arrow_start = true;
extra.start_style = style;
}
if let Some(style) = arrow("end") {
extra.arrow_end = true;
extra.end_style = style;
}
}
if shape.local() == "line" {
// A line runs `from` → `to` (points, offsets from where its style places it). A preset
+1 -1
View File
@@ -33,7 +33,7 @@ const MAX_STORIES: usize = 50_000;
/// Read a Word 2003 XML Document.
pub fn read(bytes: &[u8]) -> Result<Document, DocxError> {
crate::read::read_opened(&convert(bytes)?)
crate::read::read_opened(&convert(bytes)?, None)
}
/// Parts the converted package gets relationships for.
+4 -5
View File
@@ -382,11 +382,10 @@ fn line_dash(w: &mut W, dash: &wordcraft_doc::line_dash::LineDash) {
/// A shape's `a:effectLst` (ECMA-376 Part 1 §20.1.8.26), children in schema order.
/// A line's arrowheads (`a:headEnd`, `a:tailEnd`, ECMA-376 §20.1.8.38 / §20.1.8.57).
fn arrow_ends(w: &mut W, extra: &wordcraft_doc::connector::ShapeExtra) {
if extra.arrow_start {
w.empty("a:headEnd", &[("type", "triangle")]);
}
if extra.arrow_end {
w.empty("a:tailEnd", &[("type", "triangle")]);
for (on, tag, style) in [(extra.arrow_start, "a:headEnd", extra.start_style), (extra.arrow_end, "a:tailEnd", extra.end_style)] {
if on {
w.empty(tag, &[("type", style.kind.ooxml()), ("w", style.width.ooxml()), ("len", style.length.ooxml())]);
}
}
}
+71
View File
@@ -115,6 +115,9 @@ impl Writer<'_> {
// ink keep their original source and related parts.
InlineObject::Graphic { .. } => self.write_graphic(w, o, props, rels, depth),
InlineObject::Image { .. } => self.write_image(w, o, props, rels, depth),
InlineObject::Shape { extra, float, .. } if extra.hr.is_some() && float.wrap == Wrap::Inline && !float.hidden => {
self.write_hr(w, o, props)
}
InlineObject::Shape { .. } => self.write_shape(w, o, props, rels, depth),
InlineObject::Group { .. } => self.write_group(w, o, props, rels, depth),
InlineObject::Ruby { base, ruby, align, size, raise, base_size, lang, props: ruby_props } => {
@@ -277,6 +280,74 @@ impl Writer<'_> {
self.rev_close(w, props);
}
/// A horizontal rule goes back as the VML rule it was read from: DrawingML has none, and a
/// fixed-size shape would lose its line-wide width.
fn write_hr(&mut self, w: &mut W, o: &InlineObject, props: &CharProps) {
let InlineObject::Shape { w: sw, h: sh, fill, stroke, stroke_width, extra, float, .. } = o else { return };
let Some(hr) = extra.hr else { return };
let align = match hr.align {
wordcraft_doc::props::Align::Center => "center",
wordcraft_doc::props::Align::Right => "right",
_ => "left",
};
let width = if hr.pct > 0.0 { "0".to_string() } else { format!("{:.2}pt", sw.max(0.0)) };
let mut style = format!("width:{width};height:{:.2}pt", sh.max(0.0));
let spin = float.spin();
if spin.deg != 0.0 {
style.push_str(&format!(";rotation:{}", spin.deg));
}
match (spin.flip_h, spin.flip_v) {
(true, true) => style.push_str(";flip:x y"),
(true, false) => style.push_str(";flip:x"),
(false, true) => style.push_str(";flip:y"),
(false, false) => {}
}
let pct = format!("{}", (hr.pct * 10.0).round().max(0.0) as u32);
let fill_hex = fill.as_ref().map(|f| format!("#{}", f.main_color().hex()));
let stroke_hex = stroke.map(|c| format!("#{}", c.hex()));
let weight = format!("{stroke_width:.2}pt");
let mut attrs: Vec<(&str, &str)> = vec![
("xmlns:v", "urn:schemas-microsoft-com:vml"),
("xmlns:o", "urn:schemas-microsoft-com:office:office"),
("style", &style),
("o:hrpct", &pct),
("o:hralign", align),
("o:hr", "t"),
];
if hr.standard {
attrs.push(("o:hrstd", "t"));
}
if let Some(f) = &fill_hex {
attrs.push(("fillcolor", f));
} else {
attrs.push(("filled", "f"));
}
match &stroke_hex {
Some(s) => {
attrs.push(("strokecolor", s));
attrs.push(("strokeweight", &weight));
}
None => attrs.push(("stroked", "f")),
}
self.rev_open(w, props);
w.open("w:r", &[]);
self.rpr(w, props);
w.open("w:pict", &[]);
// A see-through fill keeps its opacity, which `fillcolor` can't say.
match fill.as_ref() {
Some(wordcraft_doc::fill::Fill::Solid { transparency, .. }) if *transparency > 0.0 => {
let opacity = format!("{:.4}", (1.0 - transparency / 100.0).clamp(0.0, 1.0));
w.open("v:rect", &attrs);
w.empty("v:fill", &[("opacity", &opacity)]);
w.close("v:rect");
}
_ => w.empty("v:rect", &attrs),
}
w.close("w:pict");
w.close("w:r");
self.rev_close(w, props);
}
fn write_shape(&mut self, w: &mut W, o: &InlineObject, props: &CharProps, rels: &mut PartRels, depth: usize) {
let InlineObject::Shape { kind, w: sw, h: sh, float, effects, freeform, extra, .. } = o else { return };
self.rev_open(w, props);
+365
View File
@@ -0,0 +1,365 @@
//! Alternative format chunks (`w:altChunk`, ECMA-376 §17.17.2.1): content in another format that
//! is brought in where the chunk stands. All markup here is synthetic.
use std::io::Write;
use wordcraft_doc::{Block, Document, InlineObject, StoryRef};
use wordcraft_docx::{AltChunk, ReadOptions, read, read_with};
const NS: &str = r#"xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships""#;
const ROOT_RELS: &str = r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/></Relationships>"#;
const DOCX_MAIN: &str = "application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml";
fn p(text: &str) -> String {
format!(r#"<w:p><w:r><w:t xml:space="preserve">{text}</w:t></w:r></w:p>"#)
}
/// A package whose body is `body`, with the chunk `afchunk.{ext}` (content type `ct`) as `rIdC`.
fn package(body: &str, chunk: Option<(&str, &str, &[u8])>) -> Vec<u8> {
package_styled(body, chunk, None)
}
/// [`package`] with a styles part holding `styles`.
fn package_styled(body: &str, chunk: Option<(&str, &str, &[u8])>, styles: Option<&str>) -> Vec<u8> {
let doc = format!(r#"<w:document {NS}><w:body>{body}<w:sectPr/></w:body></w:document>"#);
let mut files: Vec<(String, Vec<u8>)> = vec![("_rels/.rels".into(), ROOT_RELS.into()), ("word/document.xml".into(), doc.into_bytes())];
let mut types = String::from(
r#"<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="xml" ContentType="application/xml"/>"#,
);
let mut rels = String::new();
if let Some((ext, ct, bytes)) = chunk {
types.push_str(&format!(r#"<Default Extension="{ext}" ContentType="{ct}"/>"#));
rels.push_str(&format!(r#"<Relationship Id="rIdC" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/aFChunk" Target="/word/afchunk.{ext}"/>"#));
files.push((format!("word/afchunk.{ext}"), bytes.to_vec()));
}
if let Some(styles) = styles {
rels.push_str(
r#"<Relationship Id="rIdS" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/styles" Target="styles.xml"/>"#,
);
files.push(("word/styles.xml".into(), format!(r#"<w:styles {NS}>{styles}</w:styles>"#).into_bytes()));
}
if !rels.is_empty() {
files.push((
"word/_rels/document.xml.rels".into(),
format!(r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">{rels}</Relationships>"#).into_bytes(),
));
}
types.push_str(r#"<Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>"#);
files.push(("[Content_Types].xml".into(), types.into_bytes()));
let mut zw = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
for (name, bytes) in files {
zw.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
zw.write_all(&bytes).unwrap();
}
zw.finish().unwrap().into_inner()
}
const CHUNK: &str = r#"<w:altChunk r:id="rIdC"/>"#;
fn body_lines(d: &Document) -> Vec<String> {
d.plain_text(StoryRef::Body).split('\n').map(str::to_string).collect()
}
#[test]
fn a_chunk_comes_in_through_the_importer_where_it_stands() {
let html = b"<p>Chunk one</p><p>Chunk two</p>";
let bytes = package(&format!("{}{CHUNK}{}", p("Before"), p("After")), Some(("htm", "application/xhtml+xml", html)));
let seen = std::cell::RefCell::new(Vec::new());
let import = |c: &AltChunk| {
seen.borrow_mut().push((c.content_type.to_string(), c.part.to_string(), c.bytes.to_vec()));
Some(Document::from_text("Chunk one\nChunk two"))
};
let d = read_with(&bytes, &ReadOptions { chunks: Some(&import), ..Default::default() }).unwrap();
assert_eq!(body_lines(&d), ["Before", "Chunk one", "Chunk two", "After"]);
assert_eq!(*seen.borrow(), [("application/xhtml+xml".to_string(), "word/afchunk.htm".to_string(), html.to_vec())]);
// Without an importer the chunk is left out.
assert_eq!(body_lines(&read(&bytes).unwrap()), ["Before", "After"]);
}
#[test]
fn a_bookmark_opened_before_a_chunk_at_the_start_of_the_body_is_kept() {
let body = format!(r#"<w:bookmarkStart w:id="0" w:name="Here"/>{CHUNK}<w:bookmarkEnd w:id="0"/>{}"#, p("After"));
let bytes = package(&body, Some(("htm", "application/xhtml+xml", b"<p>Chunk</p>")));
let import = |_: &AltChunk| Some(Document::from_text("Chunk"));
let d = read_with(&bytes, &ReadOptions { chunks: Some(&import), ..Default::default() }).unwrap();
assert_eq!(body_lines(&d), ["Chunk", "After"]);
let marks = d.bookmarks();
assert_eq!(marks.len(), 1);
assert_eq!((marks[0].0.as_str(), marks[0].1.path.last(), marks[0].1.off), ("Here", 0, 0));
}
#[test]
fn a_bookmark_opened_between_a_table_and_a_chunk_starts_the_chunk() {
let table = format!("<w:tbl><w:tr><w:tc>{}</w:tc></w:tr></w:tbl>", p("Cell"));
let body = format!(r#"{table}<w:bookmarkStart w:id="0" w:name="Here"/>{CHUNK}<w:bookmarkEnd w:id="0"/>{}"#, p("After"));
let bytes = package(&body, Some(("htm", "application/xhtml+xml", b"<p>Chunk</p>")));
let import = |_: &AltChunk| Some(Document::from_text("Chunk"));
let d = read_with(&bytes, &ReadOptions { chunks: Some(&import), ..Default::default() }).unwrap();
let marks = d.bookmarks();
assert_eq!(marks.len(), 1);
assert_eq!((marks[0].1.path.clone(), marks[0].1.off), (wordcraft_doc::Path::top(1), 0), "not in the table's cell");
}
#[test]
fn permission_ranges_of_chunks_and_their_host_stay_apart() {
let inner = package(r#"<w:p><w:permStart w:id="0" w:ed="them"/><w:r><w:t>Inner</w:t></w:r><w:permEnd w:id="0"/></w:p>"#, None);
// The host's range 0 is open across two chunks whose own ranges are 0 too.
let body = format!(
r#"<w:p><w:permStart w:id="0" w:edGrp="everyone"/><w:r><w:t>Host</w:t></w:r></w:p>{CHUNK}{CHUNK}<w:p><w:r><w:t>After</w:t></w:r><w:permEnd w:id="0"/></w:p>"#
);
let d = read(&package(&body, Some(("docx", DOCX_MAIN, &inner)))).unwrap();
assert_eq!(body_lines(&d), ["Host", "Inner", "Inner", "After"]);
let check = |d: &Document| {
let ranges = d.perm_ranges();
let mut ids: Vec<u32> = ranges.iter().map(|r| r.id).collect();
ids.sort_unstable();
ids.dedup();
assert_eq!(ids.len(), 3, "{ranges:?}");
let host = ranges.iter().find(|r| r.group == "everyone").unwrap();
assert_eq!((host.start.path.last(), host.end.path.last()), (0, 3));
assert!(ranges.iter().filter(|r| r.editor == "them").all(|r| r.start.path == r.end.path));
};
check(&d);
check(&read(&wordcraft_docx::write(&d).unwrap()).unwrap());
}
/// A package whose body is `body`, with story parts (relationship type, part name, XML) and the
/// Word document `chunk` as `rIdC` from the main part and from each story part.
fn package_with_stories(body: &str, stories: &[(&str, &str, String)], chunk: Option<&[u8]>) -> Vec<u8> {
let rel = |id: &str, ty: &str, target: &str| {
format!(r#"<Relationship Id="{id}" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/{ty}" Target="{target}"/>"#)
};
let rels =
|inner: String| format!(r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships">{inner}</Relationships>"#);
let chunk_rel = if chunk.is_some() { rel("rIdC", "aFChunk", "/word/afchunk.docx") } else { String::new() };
let doc = format!(r#"<w:document {NS}><w:body>{body}<w:sectPr/></w:body></w:document>"#);
let mut main_rels = chunk_rel.clone();
let mut files: Vec<(String, Vec<u8>)> = vec![("_rels/.rels".into(), ROOT_RELS.into()), ("word/document.xml".into(), doc.into_bytes())];
for (i, (ty, name, xml)) in stories.iter().enumerate() {
main_rels.push_str(&rel(&format!("rIdS{i}"), ty, name));
files.push((format!("word/{name}"), xml.clone().into_bytes()));
files.push((format!("word/_rels/{name}.rels"), rels(chunk_rel.clone()).into_bytes()));
}
files.push(("word/_rels/document.xml.rels".into(), rels(main_rels).into_bytes()));
if let Some(c) = chunk {
files.push(("word/afchunk.docx".into(), c.to_vec()));
}
let mut zw = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
for (name, bytes) in files {
zw.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
zw.write_all(&bytes).unwrap();
}
zw.finish().unwrap().into_inner()
}
/// A paragraph `text` that comment `id` (in the file) is anchored around.
fn commented(id: u32, text: &str) -> String {
format!(
r#"<w:p><w:commentRangeStart w:id="{id}"/><w:r><w:t>{text}</w:t></w:r><w:commentRangeEnd w:id="{id}"/><w:r><w:commentReference w:id="{id}"/></w:r></w:p>"#
)
}
/// A comments part: (file id, author, content).
fn comments_part(list: &[(u32, &str, String)]) -> (&'static str, &'static str, String) {
let body: String = list.iter().map(|(id, who, content)| format!(r#"<w:comment w:id="{id}" w:author="{who}">{content}</w:comment>"#)).collect();
("comments", "comments.xml", format!("<w:comments {NS}>{body}</w:comments>"))
}
/// The authors of the comments anchored in the stories whose text contains `text`, and whether
/// every comment has text of its own and a distinct id.
fn anchored(d: &Document, text: &str) -> Vec<String> {
let mut stories: Vec<&[std::sync::Arc<Block>]> = vec![&d.body];
stories.extend(d.parts.values().map(|p| p.blocks.as_slice()));
let mut out = Vec::new();
for blocks in stories {
for b in blocks {
let Block::Para(p) = &**b else { continue };
if !p.text.contains(text) {
continue;
}
for o in &p.objects {
if let InlineObject::CommentStart { id } = o {
out.push(d.comments.get(id).map_or("?".to_string(), |c| c.author.clone()));
}
}
}
}
out
}
/// A Word document with one comment by "Them" around "Inner".
fn commented_chunk() -> Vec<u8> {
package_with_stories(&commented(0, "Inner"), &[comments_part(&[(0, "Them", p("Theirs"))])], None)
}
#[test]
fn a_chunk_in_a_footnote_keeps_its_comments_beside_the_documents() {
let notes = format!(r#"<w:footnotes {NS}><w:footnote w:id="1"><w:altChunk r:id="rIdC"/></w:footnote></w:footnotes>"#);
let body = format!(r#"{}<w:p><w:r><w:footnoteReference w:id="1"/></w:r></w:p>"#, commented(0, "Host"));
let stories = [("footnotes", "footnotes.xml", notes), comments_part(&[(0, "Us", p("Ours"))])];
let d = read(&package_with_stories(&body, &stories, Some(&commented_chunk()))).unwrap();
assert_eq!(d.comments.len(), 2, "{:?}", d.comments);
assert_eq!(anchored(&d, "Host"), ["Us"]);
assert_eq!(anchored(&d, "Inner"), ["Them"]);
}
#[test]
fn a_chunk_in_a_comment_keeps_its_comments_beside_the_documents() {
let inside = format!(r#"{}<w:altChunk r:id="rIdC"/>"#, p("Ours"));
let body = format!("{}{}", commented(0, "Host"), commented(1, "Second"));
let stories = [comments_part(&[(0, "Us", inside), (1, "Us too", p("Ours too"))])];
let d = read(&package_with_stories(&body, &stories, Some(&commented_chunk()))).unwrap();
assert_eq!(d.comments.len(), 3, "{:?}", d.comments);
assert_eq!(anchored(&d, "Host"), ["Us"]);
assert_eq!(anchored(&d, "Second"), ["Us too"]);
assert_eq!(anchored(&d, "Inner"), ["Them"]);
}
#[test]
fn a_wordprocessing_chunk_is_read_without_an_importer_and_flows_in_its_section() {
let inner = package(&format!("{}{}", p("Inner one"), p("Inner two")), None);
let bytes = package(&format!("{}{CHUNK}{}", p("Before"), p("After")), Some(("docx", DOCX_MAIN, &inner)));
let d = read(&bytes).unwrap();
assert_eq!(body_lines(&d), ["Before", "Inner one", "Inner two", "After"]);
// The inner document's own section doesn't come along.
assert_eq!(d.sections().len(), 1);
}
/// Styles a matchSrc chunk brings under new ids get distinct ones: with `A` and `A1` ours and
/// `A` to `A10` the chunk's, its `A` and `A1` both renamed don't land on one id (the second then
/// lost, its paragraphs taking the first's formatting), and keep their formatting through a save.
#[test]
fn match_src_gives_each_renamed_style_its_own_id() {
let style = |id: &str, half_points: u32| {
format!(r#"<w:style w:type="paragraph" w:styleId="{id}"><w:name w:val="{id}"/><w:rPr><w:sz w:val="{half_points}"/></w:rPr></w:style>"#)
};
let ours: String = ["A", "A1"].iter().map(|id| style(id, 22)).collect();
let mut theirs = style("A", 20) + &style("A1", 28);
for n in 2..=10 {
theirs += &style(&format!("A{n}"), 16);
}
let para = |id: &str, text: &str| format!(r#"<w:p><w:pPr><w:pStyle w:val="{id}"/></w:pPr><w:r><w:t>{text}</w:t></w:r></w:p>"#);
let inner = package_styled(&(para("A", "ten") + &para("A1", "fourteen")), None, Some(&theirs));
let body = format!(r#"{}<w:altChunk r:id="rIdC"><w:altChunkPr><w:matchSrc/></w:altChunkPr></w:altChunk>"#, p("Ours"));
let d = read(&package_styled(&body, Some(("docx", DOCX_MAIN, &inner)), Some(&ours))).unwrap();
let check = |d: &Document| {
let look = |text: &str| {
d.body
.iter()
.find_map(|b| match &**b {
Block::Para(p) if p.text == text => {
Some((p.props.style.clone(), d.styles.resolve_char(p.props.style.as_deref(), p.props_of_char(0)).size))
}
_ => None,
})
.unwrap()
};
let ((ten, ten_size), (fourteen, fourteen_size)) = (look("ten"), look("fourteen"));
assert_ne!(ten, fourteen);
assert_eq!((ten_size, fourteen_size), (10.0, 14.0));
};
check(&d);
check(&read(&wordcraft_docx::write(&d).unwrap()).unwrap());
}
#[test]
fn match_src_keeps_a_word_chunks_own_formatting() {
// Both define Normal and Subtitle differently, and the chunk's defaults differ from ours.
let ours = r#"<w:docDefaults><w:rPrDefault><w:rPr><w:sz w:val="22"/></w:rPr></w:rPrDefault><w:pPrDefault><w:pPr><w:spacing w:after="200"/></w:pPr></w:pPrDefault></w:docDefaults><w:style w:type="paragraph" w:default="1" w:styleId="Normal"><w:name w:val="Normal"/><w:pPr><w:spacing w:after="248" w:line="248" w:lineRule="exact"/></w:pPr></w:style><w:style w:type="paragraph" w:styleId="Subtitle"><w:name w:val="Subtitle"/><w:rPr><w:i/></w:rPr></w:style>"#;
let theirs = r#"<w:docDefaults><w:rPrDefault><w:rPr><w:sz w:val="24"/></w:rPr></w:rPrDefault></w:docDefaults><w:style w:type="paragraph" w:default="1" w:styleId="Normal"><w:name w:val="Normal"/></w:style><w:style w:type="paragraph" w:styleId="Subtitle"><w:name w:val="Subtitle"/><w:basedOn w:val="Normal"/></w:style>"#;
let inner = package_styled(
&format!(r#"{}<w:p><w:pPr><w:pStyle w:val="Subtitle"/></w:pPr><w:r><w:t>Sub</w:t></w:r></w:p>"#, p("Plain")),
None,
Some(theirs),
);
for match_src in [true, false] {
let pr = if match_src { "<w:altChunkPr><w:matchSrc/></w:altChunkPr>" } else { "" };
let body = format!(r#"{}<w:altChunk r:id="rIdC">{pr}</w:altChunk>"#, p("Ours"));
let d = read(&package_styled(&body, Some(("docx", DOCX_MAIN, &inner)), Some(ours))).unwrap();
let look = |i: usize| match &*d.body[i] {
Block::Para(p) => {
let para = d.styles.resolve_para(&p.props);
let chr = d.styles.resolve_char(p.props.style.as_deref(), p.props_of_char(0));
(para.space_after, chr.size, chr.italic)
}
b => panic!("{b:?}"),
};
// Ours stays ours either way.
assert_eq!(look(0), (12.4, 11.0, false));
if match_src {
// The chunk's own Normal (its 12 pt, Word's no space after) and its Subtitle (not italic).
assert_eq!(look(1), (0.0, 12.0, false));
assert_eq!(look(2), (0.0, 12.0, false));
} else {
// Without it the chunk takes our styles (our Subtitle stands on our defaults).
assert_eq!(look(1), (12.4, 11.0, false));
assert_eq!(look(2), (10.0, 11.0, true));
}
}
}
#[test]
fn nested_wordprocessing_chunks_stop_at_a_depth() {
let mut bytes = package(&p("Deepest"), None);
for level in 0..8 {
bytes = package(&format!("{}{CHUNK}", p(&format!("Level {level}"))), Some(("docx", DOCX_MAIN, &bytes)));
}
let lines = body_lines(&read(&bytes).unwrap());
assert_eq!(lines.first().map(String::as_str), Some("Level 7"));
assert!(!lines.iter().any(|l| l == "Deepest"), "{lines:?}");
assert!(lines.len() >= 4, "{lines:?}");
}
#[test]
fn a_chunks_pictures_come_with_it() {
let bytes = package(&format!("{}{CHUNK}", p("Before")), Some(("htm", "text/html", b"<p>x</p>")));
let import = |_: &AltChunk| {
let mut d = Document::new();
let key = d.add_media(vec![1, 2, 3], "png");
let mut para = wordcraft_doc::Paragraph::new();
let image = InlineObject::Image { media: key, w: 10.0, h: 10.0, alt: String::new(), float: Default::default(), crop: [0.0; 4], ole: None };
para.insert_object(0, image, &Default::default()).unwrap();
d.body = vec![std::sync::Arc::new(Block::Para(para))];
Some(d)
};
let d = read_with(&bytes, &ReadOptions { chunks: Some(&import), ..Default::default() }).unwrap();
let keys: Vec<String> = d
.body
.iter()
.filter_map(|b| match &**b {
Block::Para(p) => p.objects.iter().find_map(|o| match o {
InlineObject::Image { media, .. } => Some(media.clone()),
_ => None,
}),
_ => None,
})
.collect();
assert_eq!(keys.len(), 1);
assert_eq!(d.media.get(&keys[0]).map(|m| m.as_slice()), Some(&[1u8, 2, 3][..]));
}
#[test]
fn a_file_brings_in_a_bounded_number_of_chunks() {
let bytes = package(&CHUNK.repeat(300), Some(("htm", "text/html", b"<p>x</p>")));
let calls = std::cell::Cell::new(0);
let import = |_: &AltChunk| {
calls.set(calls.get() + 1);
Some(Document::from_text("x"))
};
let d = read_with(&bytes, &ReadOptions { chunks: Some(&import), ..Default::default() }).unwrap();
assert_eq!(calls.get(), 256);
assert_eq!(body_lines(&d).len(), 256);
}
#[test]
fn hostile_chunks_never_panic() {
for (ext, ct, bytes) in [("docx", DOCX_MAIN, &b"PK\x03\x04 not a zip"[..]), ("htm", "text/html", b""), ("bin", "", b"\x00\xff")] {
let bytes = package(&format!("{CHUNK}{CHUNK}{}", p("x")), Some((ext, ct, bytes)));
let import = |_: &AltChunk| None;
let d = read_with(&bytes, &ReadOptions { chunks: Some(&import), ..Default::default() }).unwrap();
assert_eq!(body_lines(&d), ["x"]);
}
// A chunk naming a missing part or another relationship type is left out.
let bytes = package(&format!(r#"<w:altChunk r:id="nope"/><w:altChunk/>{}"#, p("y")), None);
assert_eq!(body_lines(&read(&bytes).unwrap()), ["y"]);
}
+199
View File
@@ -48,6 +48,76 @@ fn read_body(body: &str) -> Document {
wordcraft_docx::read(&docx(body, &[], &[])).unwrap()
}
/// Markup compatibility replaces an AlternateContent wrapper with one selected branch,
/// including where a table expects rows or cells (ECMA-376 Part 3).
#[test]
fn table_alternate_content_keeps_selected_rows_and_cells() {
let cell = |text: &str| format!("<w:tc><w:p><w:r><w:t>{text}</w:t></w:r></w:p></w:tc>");
let row = |text: &str| format!("<w:tr>{}</w:tr>", cell(text));
let table =
|rows: &str, columns: usize| format!(r#"<w:tbl><w:tblGrid>{}</w:tblGrid>{rows}</w:tbl>"#, r#"<w:gridCol w:w="2400"/>"#.repeat(columns));
let alternate = |chosen: &str, fallback: &str, requires: &str| {
format!(
r#"<mc:AlternateContent><mc:Choice Requires="{requires}">{chosen}</mc:Choice><mc:Fallback>{fallback}</mc:Fallback></mc:AlternateContent>"#
)
};
for (requires, use_choice) in [("wps", true), ("alias", true), ("unknown", false), ("wps unknown", false)] {
for cells in [false, true] {
let choice = if cells { cell("Chosen cell") } else { row("Chosen row") };
let fallback = if cells { cell("Fallback cell") } else { row("Fallback row") };
let selected = if use_choice { &choice } else { &fallback };
let middle = alternate(&choice, &fallback, requires);
let (actual, expected) = if cells {
(
format!("<w:tr>{}{middle}{}</w:tr>", cell("Before"), cell("After")),
format!("<w:tr>{}{selected}{}</w:tr>", cell("Before"), cell("After")),
)
} else {
(format!("{}{middle}{}", row("Before"), row("After")), format!("{}{selected}{}", row("Before"), row("After")))
};
let columns = if cells { 3 } else { 1 };
let expected = read_body(&table(&expected, columns));
let wrapped = format!(
r#"<w:customXml xmlns:alias="http://schemas.microsoft.com/office/word/2010/wordprocessingShape" xmlns:unknown="urn:original:unsupported">{}</w:customXml>"#,
table(&actual, columns)
);
let mut actual = read_body(&wrapped);
for round in 0..=2 {
assert_eq!(actual.body, expected.body, "requires={requires}, cells={cells}, round={round}");
if round < 2 {
actual = wordcraft_docx::read(&wordcraft_docx::write(&actual).unwrap()).unwrap();
}
}
}
}
}
#[test]
fn table_alternate_content_keeps_nested_row_and_cell_controls() {
let wrap =
|tag: &str, content: &str| format!(r#"<w:sdt><w:sdtPr><w:tag w:val="{tag}"/></w:sdtPr><w:sdtContent>{content}</w:sdtContent></w:sdt>"#);
let alternate =
|content: &str| format!(r#"<mc:AlternateContent><mc:Choice Requires="wps">{content}</mc:Choice><mc:Fallback/></mc:AlternateContent>"#);
let cell = "<w:tc><w:p><w:r><w:t>Controlled cell</w:t></w:r></w:p></w:tc>";
let oracle_cell = wrap("cell", cell);
let actual_cell = alternate(&wrap("cell", &alternate(cell)));
let oracle_row = wrap("row", &format!("<w:tr>{oracle_cell}</w:tr>"));
let actual_row = wrap("row", &alternate(&format!("<w:tr>{actual_cell}</w:tr>")));
let expected = read_body(&format!("<w:tbl>{oracle_row}</w:tbl>"));
let mut actual = read_body(&format!("<w:tbl>{}</w:tbl>", alternate(&actual_row)));
for round in 0..=2 {
assert_eq!(actual.body, expected.body, "round={round}");
let table = actual.body.iter().find_map(|b| b.as_table()).unwrap();
assert_eq!(table.rows[0].controls.open[0].tag, "row");
assert_eq!(table.rows[0].controls.close, 1);
assert_eq!(table.rows[0].cells[0].controls.open[0].tag, "cell");
assert_eq!(table.rows[0].cells[0].controls.close, 1);
if round < 2 {
actual = wordcraft_docx::read(&wordcraft_docx::write(&actual).unwrap()).unwrap();
}
}
}
#[test]
fn repeated_style_ids_use_last() {
let styles = format!(
@@ -2475,6 +2545,52 @@ fn vml_lines_and_rectangles() {
));
}
/// An arrowhead's kind, width and length (DrawingML `a:headEnd`/`a:tailEnd` `type`/`w`/`len`, VML
/// `startarrow`/`startarrowwidth`/`startarrowlength`) are read and saved: they set the size Word
/// draws it at. An end without `w`/`len` is medium.
#[test]
fn arrowheads_keep_their_kind_and_size() {
use wordcraft_doc::connector::{ArrowKind, ArrowSize, ArrowStyle};
let style = |kind, width, length| ArrowStyle { kind, width, length };
let body = [
anchored_wsp(
"Line",
1270000,
0,
r#"<a:prstGeom prst="line"><a:avLst/></a:prstGeom><a:ln w="12700"><a:solidFill><a:srgbClr val="000000"/></a:solidFill><a:headEnd type="stealth" w="sm" len="lg"/><a:tailEnd type="diamond" w="lg" len="sm"/></a:ln>"#,
"",
"<wps:cNvCnPr/>",
),
anchored_wsp("Plain", 1270000, 0, r#"<a:prstGeom prst="line"><a:avLst/></a:prstGeom><a:ln><a:solidFill><a:srgbClr val="000000"/></a:solidFill><a:tailEnd type="triangle"/></a:ln>"#, "", "<wps:cNvCnPr/>"),
r#"<w:r><w:pict><v:line style="position:absolute" from="10pt,20pt" to="40pt,20pt"><v:stroke startarrow="classic" startarrowwidth="narrow" startarrowlength="long" endarrow="oval" endarrowwidth="wide" endarrowlength="short"/></v:line></w:pict></w:r>"#.to_string(),
]
.concat();
let ends = |d: &Document| -> Vec<(bool, ArrowStyle, bool, ArrowStyle)> {
paras(d)[0]
.objects
.iter()
.map(|o| match o {
InlineObject::Shape { extra, .. } => (extra.arrow_start, extra.start_style, extra.arrow_end, extra.end_style),
o => panic!("{o:?}"),
})
.collect()
};
let expected = vec![
(true, style(ArrowKind::Stealth, ArrowSize::Small, ArrowSize::Large), true, style(ArrowKind::Diamond, ArrowSize::Large, ArrowSize::Small)),
(false, ArrowStyle::default(), true, ArrowStyle::default()),
(true, style(ArrowKind::Stealth, ArrowSize::Small, ArrowSize::Large), true, style(ArrowKind::Oval, ArrowSize::Large, ArrowSize::Small)),
];
let d = read_body(&format!("<w:p>{body}</w:p>"));
assert_eq!(ends(&d), expected);
let bytes = wordcraft_docx::write(&d).unwrap();
let mut xml = String::new();
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes.as_slice())).unwrap();
std::io::Read::read_to_string(&mut zip.by_name("word/document.xml").unwrap(), &mut xml).unwrap();
assert!(xml.contains(r#"<a:headEnd type="stealth" w="sm" len="lg"/>"#), "{xml}");
assert!(xml.contains(r#"<a:tailEnd type="diamond" w="lg" len="sm"/>"#), "{xml}");
assert_eq!(ends(&wordcraft_docx::read(&bytes).unwrap()), expected);
}
/// The right arrow's shapetype as Word writes it (ECMA-376 Part 4 `v:shapetype`, `o:spt` 13).
const VML_RIGHT_ARROW_TYPE: &str = r#"<v:shapetype id="_x0000_t13" coordsize="21600,21600" o:spt="13" adj="16200,5400" path="m@0,l@0@1,0@1,0@2@0@2@0,21600,21600,10800xe"><v:stroke joinstyle="miter"/><v:formulas><v:f eqn="val #0"/><v:f eqn="val #1"/></v:formulas><v:path o:connecttype="custom"/></v:shapetype>"#;
@@ -3976,6 +4092,30 @@ fn equation_in_alternate_content_is_read() {
assert!(!p.plain_text().contains("[picture]"));
}
/// A fill's `a:alpha` is its opacity: a translucent white wash over a picture stayed opaque and
/// hid the picture.
#[test]
fn a_shape_fill_keeps_its_transparency() {
let fill =
r#"<a:prstGeom prst="rect"><a:avLst/></a:prstGeom><a:solidFill><a:srgbClr val="FFFFFF"><a:alpha val="30980"/></a:srgbClr></a:solidFill>"#;
let d = read_body(&format!("<w:p>{}</w:p>", anchored_wsp("Wash", 1270000, 635000, fill, "", "<wps:cNvSpPr/>")));
let transparency = |d: &Document| match &paras(d)[0].objects[0] {
InlineObject::Shape { fill: Some(wordcraft_doc::fill::Fill::Solid { color, transparency }), .. } => (Some(*color), *transparency),
o => panic!("{o:?}"),
};
const FILL_ALPHA: u32 = 30_980;
const ALPHA_SCALE: f32 = 100_000.0;
// Canonical Fill stores the percentage computed from opacity, including f32 rounding.
let expected = 100.0 - (FILL_ALPHA as f32 / ALPHA_SCALE) * 100.0;
assert_eq!(transparency(&d), (Some(Rgb(255, 255, 255)), expected));
let bytes = wordcraft_docx::write(&d).unwrap();
let parts = unzip(&bytes);
let xml = String::from_utf8_lossy(parts.get("word/document.xml").unwrap());
assert!(xml.contains(&format!("<a:alpha val=\"{FILL_ALPHA}\"/>")), "{xml}");
let again = wordcraft_docx::read(&bytes).unwrap();
assert_eq!(transparency(&again), (Some(Rgb(255, 255, 255)), expected));
}
/// #1500: gradient fills and transparency on shapes (the cases in the issue), from the shape's
/// own properties, the theme's fill styles (`a:fillRef` 3 and 1003, as Word's cover pages use),
/// a group's fill (`a:grpFill`), an outline gradient and VML's `v:fill`. Each is read as the
@@ -4172,3 +4312,62 @@ fn vml_textbox_style_anchor_and_fit() {
assert_eq!(tb.body.anchor, wordcraft_doc::props::VAlign::Center);
assert!(tb.body.auto_fit);
}
/// A VML horizontal rule (`o:hr`, a web page's `<hr>` pasted into Word) is read as one: the whole
/// line when it has no width and no `o:hrpct`, a share of it (`o:hrpct`, tenths of a percent), or
/// its own width (`o:hrpct="0"`), placed by `o:hralign`. It saves back as the same VML rule.
#[test]
fn vml_horizontal_rules_are_read_and_kept() {
use wordcraft_doc::connector::HorizontalRule;
use wordcraft_doc::props::Align;
let hr = |attrs: &str, style: &str| format!(r#"<w:p><w:r><w:pict><v:rect style="{style}" {attrs}/></w:pict></w:r></w:p>"#);
let body = [
hr(r#"o:hralign="center" o:hrstd="t" o:hr="t""#, "width:0;height:1.5pt"),
hr(r#"o:hrpct="500" o:hralign="right" o:hr="t""#, "width:0;height:1.5pt"),
hr(r#"o:hrpct="0" o:hralign="center" o:hr="t""#, "width:100pt;height:3pt"),
hr("", "width:100pt;height:3pt"),
]
.concat();
let rules = |d: &Document| -> Vec<(Option<HorizontalRule>, f32)> {
paras(d)
.iter()
.filter_map(|p| p.objects.iter().find_map(|o| if let InlineObject::Shape { extra, w, .. } = o { Some((extra.hr, *w)) } else { None }))
.collect()
};
let d = read_body(&body);
let got = rules(&d);
assert_eq!(
got,
[
(Some(HorizontalRule { pct: 100.0, align: Align::Center, standard: true }), 0.0),
(Some(HorizontalRule { pct: 50.0, align: Align::Right, standard: false }), 0.0),
(Some(HorizontalRule { pct: 0.0, align: Align::Center, standard: false }), 100.0),
(None, 100.0),
]
);
let again = wordcraft_docx::read(&wordcraft_docx::write(&d).unwrap()).unwrap();
assert_eq!(rules(&again)[..3], got[..3]);
// A rule's rotation and flips are kept.
let spin = |d: &Document| paras(d)[0].objects.iter().find_map(|o| o.frame()).map(|(_, _, f)| f.spin());
let d = read_body(&hr(r#"o:hrpct="500" o:hr="t""#, "width:0;height:1.5pt;rotation:45;flip:x y"));
assert_eq!(spin(&d), Some(wordcraft_geom::Spin::new(45.0, true, true)));
let again = wordcraft_docx::read(&wordcraft_docx::write(&d).unwrap()).unwrap();
assert_eq!(spin(&again), spin(&d));
// A see-through rule stays see-through.
let fill = |d: &Document| paras(d)[0].objects.iter().find_map(|o| if let InlineObject::Shape { fill, .. } = o { fill.clone() } else { None });
let d = read_body(
r##"<w:p><w:r><w:pict><v:rect style="width:0;height:1.5pt" o:hr="t" fillcolor="#ff0000"><v:fill opacity="0.25"/></v:rect></w:pict></w:r></w:p>"##,
);
let Some(wordcraft_doc::fill::Fill::Solid { transparency, .. }) = fill(&d) else { panic!("{:?}", fill(&d)) };
assert!((transparency - 75.0).abs() < 0.01, "{transparency}");
let again = wordcraft_docx::read(&wordcraft_docx::write(&d).unwrap()).unwrap();
assert_eq!(fill(&again), fill(&d));
// A positioned or hidden rule stays a plain shape, keeping its placement and visibility.
let body = [
hr(r#"o:hralign="center" o:hr="t""#, "position:absolute;margin-left:10pt;margin-top:20pt;width:100pt;height:2pt"),
hr(r#"o:hralign="center" o:hr="t""#, "visibility:hidden;width:0;height:2pt"),
]
.concat();
let d = read_body(&body);
assert_eq!(rules(&d).iter().map(|r| r.0).collect::<Vec<_>>(), [None, None]);
}
+1
View File
@@ -1,3 +1,4 @@
mod alt_chunks;
mod bidi;
mod bidi_runs;
mod cell_margin_inheritance;
+3
View File
@@ -41,6 +41,9 @@ js-sys = "0.3"
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
getrandom = { workspace = true }
# Language packs (CP4.3): downloaded, verified and unpacked by the suite's font store.
craft-font-store = { workspace = true }
craft-font-catalog = { workspace = true }
[target.'cfg(unix)'.dependencies]
# O_NONBLOCK, so a path swapped for a named pipe can't block `open` (#3666).
+2
View File
@@ -28,6 +28,7 @@ pub mod moves;
pub mod objects;
pub mod options;
pub mod outline;
pub mod packs;
pub mod page;
pub mod para;
pub mod paste;
@@ -84,6 +85,7 @@ pub fn registry() -> Registry {
v.extend(review::specs());
v.extend(file::specs());
v.extend(options::specs());
v.extend(packs::specs());
v.extend(design::specs());
v.extend(references::specs());
v.extend(fields::specs());
+2 -1
View File
@@ -134,7 +134,8 @@ pub fn specs() -> Vec<CommandSpec> {
CommandSpec::new("insert.textFromFile", "Text from File", "Insert › Text › Object", |s, v| {
let path = p::req_str(v, "path")?;
let doc = crate::io::open_path(std::path::Path::new(path)).map_err(CmdError::Failed)?;
let mut frag = doc.body_fragment();
// The file's pictures, lists and styles come along (its own keys and ids mean nothing here).
let mut frag = s.doc.adopt(doc);
super::paste::unlink_sections(&mut frag.blocks);
let at = delete_selection(s)?;
let end = s.doc.insert_fragment(&at, &frag)?;
+1 -1
View File
@@ -235,7 +235,7 @@ pub fn current(s: &Session) -> Value {
"userTemplatesFolder": pr.user_templates_folder,
"dictionaries": wordcraft_proof::installed_dictionaries()
.into_iter()
.map(|(lang, dic)| json!({"language": lang.code(), "file": dic.map(|d| d.to_string_lossy().into_owned())}))
.map(|(lang, dic)| json!({"language": lang.tag(), "file": dic.map(|d| d.to_string_lossy().into_owned())}))
.collect::<Vec<_>>(),
})
}
+423
View File
@@ -0,0 +1,423 @@
//! Language packs as commands (CP4.3): list, install and remove, for the CLI, MCP and the control
//! channel. They open nothing and never turn downloading on: an install needs the font store's
//! consent setting, which only a person in the app turns on (File › Options › Language).
//! See [`crate::packs`].
use serde_json::{Value, json};
use crate::packs;
use crate::{CmdError, CmdResult, CommandSpec, Session, p};
pub fn specs() -> Vec<CommandSpec> {
vec![
CommandSpec::new("proof.packs", "Language Packs", "File › Options › Language", list)
.params(r#"{"refresh"?: bool (fetch the pinned index again; only while downloading is on; in the app it runs in the background: see "refreshing")} — opens nothing. Returns {"packs": [{"locale", "name", "size" (bytes), "licence" (SPDX), "class", "source" (where the data comes from), "archive" (the download), "spelling", "hyphenation", "installed"}], "installed": [{"locale", "name", "licence"}], "installing": [{"locale", "done", "total"} (bytes)], "refreshing": bool, "errors": {locale or "" (the list): message} (the last failure of each), "index_override": string|null (a test index in use instead of the pinned one), "network": bool (downloading on), "available": bool (a font store to install into)}"#)
.pure(),
CommandSpec::new("proof.installPack", "Install Language Pack", "File › Options › Language", install)
.params(r#"{"locale": string (a pack's locale from proof.packs, like nl or pt-pt)} — downloads, verifies (sha256) and installs it; spelling and hyphenation in that language use it at once. Needs downloading on (only the person in the app can turn it on). Opens nothing. In the app it runs in the background and returns {"started": true}: follow it with proof.packs ("installing", then "installed" or "errors"); elsewhere it returns when installed."#)
.pure()
.when(available),
CommandSpec::new("proof.removePack", "Remove Language Pack", "File › Options › Language", remove)
.params(r#"{"locale": string} — removes the installed pack. Opens nothing."#)
.pure()
.when(available),
]
}
fn available(_: &Session) -> Option<&'static str> {
if cfg!(target_arch = "wasm32") { Some("Language packs aren't available in the browser") } else { None }
}
fn locale_param(v: &Value) -> Result<String, CmdError> {
let raw = p::str(v, "locale").ok_or_else(|| CmdError::Params("`locale` (a language tag like nl) is required".into()))?;
packs::parse_locale(raw)
}
fn list(_: &mut Session, v: &Value) -> CmdResult {
if p::bool(v, "refresh") == Some(true) {
packs::start_refresh().map_err(CmdError::Failed)?;
if let Some(e) = packs::error("") {
return Err(CmdError::Failed(e));
}
}
let index = packs::available();
let installed = packs::installed();
let backend = packs::backend();
let rows: Vec<Value> = index
.iter()
.map(|p| {
json!({
"locale": p.locale, "name": p.name, "size": p.size, "licence": p.licence, "class": p.class,
"source": p.source_link(), "archive": p.url, "spelling": p.spelling, "hyphenation": p.hyphenation,
"installed": installed.iter().any(|i| i.locale == p.locale),
})
})
.collect();
Ok(json!({
"packs": rows,
"installed": installed.iter().map(|i| json!({"locale": i.locale, "name": i.name, "licence": i.licence})).collect::<Vec<_>>(),
"installing": packs::installing(),
"refreshing": packs::fetching(),
"errors": packs::errors().into_iter().map(|(l, e)| (l, Value::String(e))).collect::<serde_json::Map<String, Value>>(),
"network": backend.as_ref().is_some_and(|b| b.network_on()),
// The packs come from a test index (`CRAFT_LANGUAGE_PACK_INDEX`), not the pinned one.
"index_override": packs::index_override(),
"available": backend.is_some(),
}))
}
fn install(s: &mut Session, v: &Value) -> CmdResult {
let locale = locale_param(v)?;
if packs::is_installed(&locale) {
return Ok(json!({"locale": locale, "installed": true, "already": true}));
}
if packs::start_install(&locale).map_err(CmdError::Failed)? {
return Ok(json!({"locale": locale, "started": true}));
}
let done =
packs::installed().into_iter().find(|p| p.locale == locale).ok_or_else(|| CmdError::Failed(format!("the {locale} pack didn't install")))?;
// Spelling and hyphenation in that language change.
s.relayout();
Ok(json!({"locale": done.locale, "name": done.name, "licence": done.licence, "installed": true}))
}
fn remove(s: &mut Session, v: &Value) -> CmdResult {
let locale = locale_param(v)?;
packs::remove(&locale).map_err(CmdError::Failed)?;
s.relayout();
Ok(json!({"locale": locale, "installed": false}))
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use std::sync::atomic::Ordering;
use super::*;
use crate::packs::{Backend, FixedBackend};
/// Tests that set the process-wide backend take turns.
static LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
/// Puts the process back to no backend and no background thread when dropped, even when
/// the test panicked.
pub(crate) struct Reset;
impl Drop for Reset {
fn drop(&mut self) {
packs::set_background(None);
packs::set_backend(None);
}
}
fn with_backend<T>(b: FixedBackend, f: impl FnOnce(&mut Session, &FixedBackend) -> T) -> T {
let _g = LOCK.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
let _reset = Reset;
let b = Arc::new(b);
packs::set_backend(Some(b.clone() as Arc<dyn Backend>));
let mut s = Session::new(crate::sample::sample_document());
f(&mut s, &b)
}
#[test]
fn lists_installs_and_removes_without_opening_anything() {
with_backend(FixedBackend::sample(), |s, b| {
let r = s.run("proof.packs", &json!({})).unwrap();
assert_eq!(r["packs"].as_array().unwrap().len(), 6);
assert_eq!(r["network"], false);
assert_eq!(r["index_override"], Value::Null, "the pinned index");
let nl = r["packs"].as_array().unwrap().iter().find(|p| p["locale"] == "nl").unwrap();
assert_eq!(nl["licence"], "BSD-3-Clause");
assert_eq!(nl["installed"], false);
// Downloading is off: an agent can't install, and can't turn it on.
let e = s.run("proof.installPack", &json!({"locale": "nl"})).unwrap_err();
assert!(e.to_string().contains("downloading is turned off"), "{e}");
assert!(packs::installed().is_empty());
b.network.store(true, Ordering::Relaxed);
let r = s.run("proof.installPack", &json!({"locale": "NL"})).unwrap();
assert_eq!(r["locale"], "nl");
assert!(packs::is_installed("nl"));
let again = s.run("proof.installPack", &json!({"locale": "nl"})).unwrap();
assert_eq!(again["already"], true);
let r = s.run("proof.packs", &json!({})).unwrap();
assert_eq!(r["installed"][0]["locale"], "nl");
s.run("proof.removePack", &json!({"locale": "nl"})).unwrap();
assert!(!packs::is_installed("nl"));
assert!(s.run("proof.removePack", &json!({"locale": "nl"})).is_err(), "not installed");
assert!(s.ui_requests.is_empty(), "no dialog opens");
});
}
/// The bar's missing-pack cache keys on `index_generation`, so a refresh that returns a list
/// of the same length must still change it (CodeRabbit on #5208).
#[test]
fn a_refresh_of_the_same_length_still_changes_the_index_generation() {
with_backend(FixedBackend::sample(), |s, b| {
b.network.store(true, Ordering::Relaxed);
let n = packs::available_len();
let before = packs::index_generation();
s.run("proof.packs", &json!({"refresh": true})).unwrap();
assert_eq!(packs::available_len(), n, "same length");
assert_ne!(packs::index_generation(), before);
});
}
#[test]
fn unknown_and_hostile_locales_are_errors() {
with_backend(FixedBackend::sample(), |s, b| {
b.network.store(true, Ordering::Relaxed);
for bad in [json!({}), json!({"locale": 5}), json!({"locale": "../../etc"}), json!({"locale": "x".repeat(10_000)}), json!({"locale": ""})]
{
assert!(s.run("proof.installPack", &bad).is_err(), "{bad}");
assert!(s.run("proof.removePack", &bad).is_err(), "{bad}");
}
let e = s.run("proof.installPack", &json!({"locale": "tlh"})).unwrap_err();
assert!(e.to_string().contains("no language pack for tlh"), "{e}");
assert!(packs::installed().is_empty());
});
}
#[test]
fn an_agent_backend_cannot_turn_downloading_on() {
let b = FixedBackend { agent: true, ..FixedBackend::sample() };
assert!(b.enable_network().is_err());
assert!(!b.network_on());
}
#[test]
fn without_a_store_nothing_is_available() {
let _g = LOCK.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
let _reset = Reset;
packs::set_backend(None);
let mut s = Session::new(crate::sample::sample_document());
let r = s.run("proof.packs", &json!({})).unwrap();
assert_eq!(r["packs"], json!([]));
assert_eq!(r["available"], false);
assert!(s.run("proof.installPack", &json!({"locale": "nl"})).is_err());
assert!(s.run("proof.packs", &json!({"refresh": true})).is_err());
}
#[test]
fn the_bar_offers_only_languages_with_a_pack() {
with_backend(FixedBackend::sample(), |s, b| {
s.run("document.setText", &json!({"text": "Hallo wereld\nDzień dobry\nGuten Tag\nHello\nWitam"})).unwrap();
assert!(packs::missing(s).is_empty(), "all English");
for (para, lang) in [(0, "nl-NL"), (1, "pl-PL"), (2, "de-DE"), (4, "pl")] {
s.run("caret.set", &json!({"pos": {"block": para, "off": 0}})).unwrap();
s.run("select.paragraph", &json!({})).unwrap();
s.run("review.language", &json!({"lang": lang})).unwrap();
}
let langs = packs::document_languages(s);
assert!(langs.iter().any(|l| l == "nl-NL"), "{langs:?}");
let missing: Vec<String> = packs::missing(s).into_iter().map(|p| p.locale).collect();
// German has no pack: never offered.
assert_eq!(missing, ["nl", "pl"]);
b.network.store(true, Ordering::Relaxed);
s.run("proof.installPack", &json!({"locale": "pl"})).unwrap();
let missing: Vec<String> = packs::missing(s).into_iter().map(|p| p.locale).collect();
assert_eq!(missing, ["nl"]);
// Text marked "no language" (`zxx`) never asks for a pack.
assert!(!packs::proofable("zxx"));
});
}
#[test]
fn tags_map_to_the_most_specific_pack() {
let mut packs = FixedBackend::sample().packs;
packs.push(Pack { locale: "pt-pt".into(), ..packs[0].clone() });
packs.push(Pack { locale: "pt".into(), ..packs[0].clone() });
assert_eq!(packs::pack_for_tag("pt-PT", &packs).map(|p| p.locale.as_str()), Some("pt-pt"));
assert_eq!(packs::pack_for_tag("pt_BR", &packs).map(|p| p.locale.as_str()), Some("pt"));
assert_eq!(packs::pack_for_tag("nl-BE", &packs).map(|p| p.locale.as_str()), Some("nl"));
assert_eq!(packs::pack_for_tag("en-GB", &packs), None);
assert_eq!(packs::pack_for_tag("", &packs), None);
assert!(packs::valid_locale("sr-latn"));
assert!(!packs::valid_locale("sr--latn") && !packs::valid_locale("-x") && !packs::valid_locale("x/y"));
}
/// An install or a removal in one window proofs every open document again, parked ones too:
/// their layouts are keyed on the dictionaries' generation, not only their own revision.
#[test]
fn installing_proofs_parked_documents_again() {
with_backend(FixedBackend::sample(), |s, b| {
let mut parked = Session::new(crate::sample::sample_document());
let before = parked.layout();
assert!(std::sync::Arc::ptr_eq(&before, &parked.layout()), "unchanged: the same layout");
b.network.store(true, Ordering::Relaxed);
s.run("proof.installPack", &json!({"locale": "pl"})).unwrap();
assert!(!std::sync::Arc::ptr_eq(&before, &parked.layout()), "the parked document is laid out (and proofed) again");
let after = parked.layout();
s.run("proof.removePack", &json!({"locale": "pl"})).unwrap();
assert!(!std::sync::Arc::ptr_eq(&after, &parked.layout()));
});
}
/// A backend whose installs wait until the test lets them finish.
struct Slow {
inner: FixedBackend,
go: std::sync::Mutex<std::sync::mpsc::Receiver<()>>,
}
impl Backend for Slow {
fn index(&self) -> Result<Vec<Pack>, String> {
self.inner.index()
}
fn fetch_index(&self, c: &packs::Cancel) -> Result<Vec<Pack>, String> {
self.inner.fetch_index(c)
}
fn network_on(&self) -> bool {
self.inner.network_on()
}
fn enable_network(&self) -> Result<(), String> {
self.inner.enable_network()
}
fn install(&self, pack: &Pack, cancel: &packs::Cancel, progress: &mut dyn FnMut(u64, u64)) -> Result<(), String> {
progress(1, pack.size);
let _ = self.go.lock().unwrap().recv();
self.inner.install(pack, cancel, progress)
}
fn remove(&self, locale: &str) -> Result<(), String> {
self.inner.remove(locale)
}
fn packs_dir(&self) -> Option<std::path::PathBuf> {
None
}
fn installed(&self) -> Vec<packs::Installed> {
self.inner.installed()
}
}
/// In the app an install runs on a thread, whoever asks: the command returns at once, and
/// `proof.packs` shows it running, then installed. A removal of the same pack meanwhile is
/// refused.
#[test]
fn in_the_app_installs_run_in_the_background() {
let _g = LOCK.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
let _reset = Reset;
let (tx, rx) = std::sync::mpsc::channel();
let slow = Arc::new(Slow { inner: FixedBackend::sample(), go: std::sync::Mutex::new(rx) });
slow.inner.network.store(true, Ordering::Relaxed);
packs::set_backend(Some(slow.clone() as Arc<dyn Backend>));
let notified = Arc::new(std::sync::atomic::AtomicUsize::new(0));
let n = notified.clone();
packs::set_background(Some(Arc::new(move || {
n.fetch_add(1, Ordering::Relaxed);
})));
let mut s = Session::new(crate::sample::sample_document());
let before = packs::finished();
let r = s.run("proof.installPack", &json!({"locale": "pl"})).unwrap();
assert_eq!(r["started"], true);
let list = s.run("proof.packs", &json!({})).unwrap();
assert_eq!(list["installing"][0]["locale"], "pl");
let e = s.run("proof.removePack", &json!({"locale": "pl"})).unwrap_err();
assert!(e.to_string().contains("being installed"), "{e}");
tx.send(()).unwrap();
for _ in 0..500 {
if packs::finished() != before {
break;
}
std::thread::sleep(std::time::Duration::from_millis(10));
}
assert_ne!(packs::finished(), before);
assert!(notified.load(Ordering::Relaxed) > 0, "the window is told");
let list = s.run("proof.packs", &json!({})).unwrap();
assert_eq!(list["installing"], json!([]));
assert_eq!(list["installed"][0]["locale"], "pl");
assert!(s.ui_requests.is_empty(), "no dialog opens");
}
/// A pack read from its folder proofs and hyphenates its language's runs, and leaves the
/// English runs to English: a tiny Lithuanian pack (`ProofLang::Pack`, no rules of its own).
#[test]
fn an_installed_pack_proofs_and_hyphenates_its_runs() {
let _g = LOCK.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
let _reset = Reset;
let dir = std::env::temp_dir().join(format!("wordcraft-pack-fixture-{}", std::process::id()));
let lt = dir.join("lt");
std::fs::create_dir_all(&lt).unwrap();
std::fs::write(lt.join("pack.toml"), "locale = \"lt\"\nname = \"Lietuvių\"\nlicence = \"MIT\"\nclass = \"Embeddable\"\n").unwrap();
std::fs::write(lt.join("spelling.aff"), "SET UTF-8\n").unwrap();
std::fs::write(lt.join("spelling.dic"), "2\nlabas\nrytas\n").unwrap();
// One break, laba-rytas; the other letters only make up the patterns' alphabet.
std::fs::write(lt.join("hyphenation.pat"), "a1r b l s t y z\n").unwrap();
wordcraft_proof::packs::set_packs_dir(Some(dir.clone()));
let text = "labas rytaz labarytas typography beautifull";
let mut s = Session::new(wordcraft_doc::Document::from_text(text));
let eng = text.find("typography").unwrap();
s.doc.format_range(&wordcraft_doc::Pos::body(0, 0), &wordcraft_doc::Pos::body(0, eng), &|c| c.lang = Some("lt-LT".into())).unwrap();
s.doc.format_range(&wordcraft_doc::Pos::body(0, eng), &wordcraft_doc::Pos::body(0, text.len()), &|c| c.lang = Some("en-US".into())).unwrap();
s.doc.settings.auto_hyphenation = true;
s.relayout();
let issues = s.run("review.issues", &json!({})).map(|v| v.to_string()).unwrap_or_default();
let layout = s.layout();
let ends: Vec<usize> = layout
.pages
.iter()
.flat_map(|p| &p.items)
.filter_map(|i| if let wordcraft_layout::Placed::Lines { para, .. } = i { Some(para) } else { None })
.flat_map(|para| para.hyph_after.iter().filter_map(|&k| para.clusters.get(k as usize).map(|c| c.end)).collect::<Vec<_>>())
.collect();
wordcraft_proof::packs::set_packs_dir(None);
let _ = std::fs::remove_dir_all(&dir);
let laba = text.find("labarytas").unwrap() + "laba".len();
assert!(ends.contains(&laba), "the pack's pattern breaks laba-rytas: {ends:?}");
assert!(issues.contains("rytaz"), "the Lithuanian misspelling, from the pack: {issues}");
assert!(issues.contains("beautifull"), "the English one: {issues}");
assert!(!issues.contains("\"labas\"") && !issues.contains("typography"), "{issues}");
}
/// Japanese, Chinese and Thai have no Hunspell dictionary anywhere: without a pack for them
/// in the list the status bar doesn't say "(no dictionary)" and the bar offers nothing; once
/// the list has one, both do.
#[test]
fn a_language_with_no_pack_and_no_dictionary_asks_for_nothing() {
with_backend(FixedBackend::sample(), |s, _| {
for tag in ["ja-JP", "zh-CN", "th-TH"] {
assert_eq!(packs::missing_dictionary(Some(tag)), None, "{tag}");
}
s.run("document.setText", &json!({"text": "日本語のテキスト"})).unwrap();
s.run("select.all", &json!({})).unwrap();
s.run("review.language", &json!({"lang": "ja-JP"})).unwrap();
assert!(packs::missing(s).is_empty(), "no Japanese pack: nothing offered");
});
let mut b = FixedBackend::sample();
let ja = Pack { locale: "ja".into(), name: "日本語".into(), ..b.packs[0].clone() };
b.packs.push(ja);
with_backend(b, |s, _| {
assert!(packs::missing_dictionary(Some("ja-JP")).is_some(), "a Japanese pack is available");
s.run("document.setText", &json!({"text": "日本語のテキスト"})).unwrap();
s.run("select.all", &json!({})).unwrap();
s.run("review.language", &json!({"lang": "ja-JP"})).unwrap();
assert_eq!(packs::missing(s).into_iter().map(|p| p.locale).collect::<Vec<_>>(), ["ja"]);
});
}
/// Swiss German and European Portuguese are pack languages in craft-proof (`de-ch`, `pt-pt`:
/// their spelling is their own) but hyphenate and check grammar as German and Portuguese.
#[test]
fn swiss_german_and_portugal_portuguese_keep_their_languages_rules() {
use wordcraft_proof::ProofLang;
let de_ch = ProofLang::from_tag(Some("de-CH"));
let pt_pt = ProofLang::from_tag(Some("pt-PT"));
assert_eq!(de_ch.tag(), "de-ch");
assert_eq!(pt_pt.tag(), "pt-pt");
assert!(packs::proofable("de-CH") && packs::proofable("pt-PT"));
for word in ["Zusammenarbeit", "Rechtschreibung"] {
let german = wordcraft_proof::hyphen::word_breaks_in(word, ProofLang::De).0;
assert!(!german.is_empty(), "German hyphenates {word}");
assert_eq!(wordcraft_proof::hyphen::word_breaks_in(word, de_ch).0, german, "Swiss German hyphenates {word} as German");
}
for word in ["computador", "informação"] {
assert_eq!(
wordcraft_proof::hyphen::word_breaks_in(word, pt_pt).0,
wordcraft_proof::hyphen::word_breaks_in(word, ProofLang::Pt).0,
"{word}"
);
}
let messages = |lang| wordcraft_proof::check_grammar_in("Ele disse que que viria .", lang).into_iter().map(|i| i.message).collect::<Vec<_>>();
assert!(!messages(ProofLang::Pt).is_empty());
assert_eq!(messages(pt_pt), messages(ProofLang::Pt), "Portugal text gets the Portuguese grammar checks");
}
use crate::packs::Pack;
}
+3
View File
@@ -374,6 +374,9 @@ fn line_numbering_options() {
/// there breaks the word.
#[test]
fn hyphenation_options_and_manual_hyphenation() {
if crate::tests::no_language_data("hyphenation_options_and_manual_hyphenation") {
return;
}
let mut s =
Session::new(wordcraft_doc::Document::from_text(&"a extraordinarily be uncharacteristic of responsibilities the considerations ".repeat(12)));
let r = s.run("layout.hyphenation", &json!({"zone": 36, "caps": false, "limit": 3})).unwrap();
+4 -103
View File
@@ -8,7 +8,6 @@
//! program's HTML or RTF; Merge Formatting keeps only its emphasis (#687).
//! Plain text has no formatting to keep, so Paste inserts it as Keep Text Only does.
use std::collections::BTreeMap;
use std::sync::Arc;
use serde_json::{Value, json};
@@ -401,35 +400,6 @@ pub(crate) fn each_para(b: &mut Block, depth: usize, f: &mut dyn FnMut(&mut Para
}
}
/// Point a tracked formatting change at its renumbered revision (dropped when it has none).
fn remap_change<T>(c: &mut Option<Box<wordcraft_doc::props::PropChange<T>>>, rev: &dyn Fn(&mut Option<u32>)) {
let mut r = c.as_ref().map(|c| c.rev);
rev(&mut r);
match (r, c.as_mut()) {
(Some(v), Some(x)) => x.rev = v,
_ => *c = None,
}
}
/// [`remap_change`] for the table, row and cell formatting changes in `b` (bounded depth).
fn remap_table_changes(b: &mut Block, depth: usize, rev: &dyn Fn(&mut Option<u32>)) {
if depth > MAX_DEPTH {
return;
}
if let Block::Table(t) = b {
remap_change(&mut t.props.fmt_change, rev);
for row in &mut t.rows {
remap_change(&mut row.props.fmt_change, rev);
for cell in &mut row.cells {
remap_change(&mut cell.props.fmt_change, rev);
for cb in &mut cell.blocks {
remap_table_changes(Arc::make_mut(cb), depth + 1, rev);
}
}
}
}
}
/// Section breaks brought in from another document keep their page, but their header and
/// footer ids are that document's: clear them (and the different first page that went with
/// them) so the sections link to the previous section's headers and footers, as in Word.
@@ -448,81 +418,12 @@ pub(crate) fn unlink_sections(blocks: &mut [Block]) {
/// Turn an imported document into a fragment of the session's document: bring over the media,
/// lists and styles it refers to (renumbered so they can't collide with ours).
fn adopt(s: &mut Session, src: Document) -> Fragment {
let mut media = BTreeMap::new();
for (key, data) in &src.media {
let ext = key.rsplit_once('.').map(|(_, e)| e).unwrap_or("png");
media.insert(key.clone(), s.doc.add_media(data.to_vec(), ext));
}
let next_abs = s.doc.numbering.abstracts.iter().map(|a| a.id.saturating_add(1)).max().unwrap_or(0);
let mut abs_map = BTreeMap::new();
for (i, a) in src.numbering.abstracts.iter().enumerate() {
let id = next_abs.saturating_add(i as u32);
abs_map.insert(a.id, id);
s.doc.numbering.abstracts.push(wordcraft_doc::numbering::AbstractNum { id, ..a.clone() });
}
let next_num = s.doc.numbering.nums.iter().map(|n| n.id.saturating_add(1)).max().unwrap_or(1).max(1);
let mut num_map = BTreeMap::new();
for (i, n) in src.numbering.nums.iter().enumerate() {
let id = next_num.saturating_add(i as u32);
num_map.insert(n.id, id);
let abstract_id = abs_map.get(&n.abstract_id).copied().unwrap_or(n.abstract_id);
s.doc.numbering.nums.push(wordcraft_doc::numbering::Num { id, abstract_id, ..n.clone() });
}
for st in &src.styles.styles {
if s.doc.styles.get(&st.id).is_none() {
s.doc.styles.upsert(st.clone());
}
}
// Tracked changes the source carries (RTF revisions) point at its own revisions: add them
// after ours and renumber.
let rev_base = u32::try_from(s.doc.revisions.len()).unwrap_or(u32::MAX);
let src_revs = src.revisions.len();
s.doc.revisions.extend(src.revisions.iter().cloned());
let rev = |i: &mut Option<u32>| {
*i = i.filter(|v| (*v as usize) < src_revs).and_then(|v| rev_base.checked_add(v));
};
let mut blocks: Vec<Block> = src.body.iter().map(|b| (**b).clone()).collect();
let mut frag = s.doc.adopt(src);
// Pasted pictures fit our text width, like Insert › Pictures (layout keeps their size).
let max_w = crate::io::text_width(&s.doc);
for b in &mut blocks {
for b in &mut frag.blocks {
crate::io::fit_block_pictures(b, max_w);
}
for b in &mut blocks {
each_para(b, 0, &mut |p| {
if let Some(n) = p.props.numbering.as_mut()
&& n.num != 0
{
n.num = num_map.get(&n.num).copied().unwrap_or(0);
}
for r in &mut p.runs {
rev(&mut r.props.ins);
rev(&mut r.props.del);
remap_change(&mut r.props.fmt_change, &rev);
}
rev(&mut p.mark.ins);
rev(&mut p.mark.del);
remap_change(&mut p.mark.fmt_change, &rev);
remap_change(&mut p.props.fmt_change, &rev);
let mut n = p.props.num_change.as_ref().map(|c| c.rev);
rev(&mut n);
match (n, p.props.num_change.as_mut()) {
(Some(v), Some(c)) => c.rev = v,
_ => p.props.num_change = None,
}
for o in &mut p.objects {
if let wordcraft_doc::InlineObject::Image { media: key, .. } = o
&& let Some(k) = media.get(key.as_str())
{
*key = k.clone();
}
}
});
}
for b in &mut blocks {
remap_table_changes(b, 0, &rev);
}
unlink_sections(&mut blocks);
// Text box stories the imported shapes show; pasting gives each a new story of its own.
let parts = src.parts.into_iter().filter(|(_, p)| p.kind == wordcraft_doc::PartKind::TextBox).collect();
Fragment { blocks, parts }
unlink_sections(&mut frag.blocks);
frag
}
+1
View File
@@ -80,6 +80,7 @@ pub fn specs() -> Vec<CommandSpec> {
CommandSpec::new("review.changes", "Reviewing Pane", "Review › Tracking", list_changes).pure(),
CommandSpec::new("review.spelling", "Spelling & Grammar", "Review › Proofing", next_issue)
.key(super::mac_or("F7 / Alt+F7 / Mod+Alt+L", "F7 / Alt+F7"))
.params(r#"{"wrap"?: bool (default true; false stops at the end of the current story)}"#)
.pure(),
CommandSpec::new("review.issues", "Proofing Issues", "Review › Proofing", all_issues).pure(),
CommandSpec::new("review.suggestions", "Spelling Suggestions", "Review › Proofing", suggestions).params(r#"{"pos"?: Pos}"#).pure(),
+30 -8
View File
@@ -55,7 +55,16 @@ fn para_issues(s: &Session, story: StoryRef, path: &wordcraft_doc::Path) -> Vec<
/// The issue under a position.
pub(super) fn issue_at(s: &Session, at: &Pos) -> Option<(Pos, Pos, wordcraft_proof::Issue)> {
para_issues(s, at.story, &at.path).into_iter().find(|(a, b, _)| a.off <= at.off && at.off <= b.off)
let issues = para_issues(s, at.story, &at.path);
let selected = s.sel.ordered();
// F7 can select a spelling issue inside a larger grammar issue. Keep that
// exact target for either selection direction before falling back to a point.
if at == &s.sel.focus
&& let Some(issue) = issues.iter().find(|(a, b, _)| a == &selected.0 && b == &selected.1)
{
return Some(issue.clone());
}
issues.into_iter().find(|(a, b, _)| a.off <= at.off && at.off <= b.off)
}
fn issue_json(s: &Session, a: &Pos, b: &Pos, i: &wordcraft_proof::Issue) -> Value {
@@ -64,20 +73,28 @@ fn issue_json(s: &Session, a: &Pos, b: &Pos, i: &wordcraft_proof::Issue) -> Valu
json!({
"start": pos_json(a), "end": pos_json(b), "text": word,
"kind": if i.kind == wordcraft_proof::IssueKind::Spelling { "spelling" } else { "grammar" },
"message": i.message, "suggestions": sugg, "lang": i.lang.code(),
"message": i.message, "suggestions": sugg, "lang": i.lang.tag(),
})
}
/// F7: select the next issue after the caret and return it with suggestions.
pub(super) fn next_issue(s: &mut Session, _: &Value) -> CmdResult {
pub(super) fn next_issue(s: &mut Session, v: &Value) -> CmdResult {
let caret = s.sel.ordered().1;
let story = caret.story;
let paths = s.doc.para_paths(story);
let start = paths.iter().position(|p| *p == caret.path).unwrap_or(0);
for k in 0..paths.len() {
let Some(path) = paths.get((start + k) % paths.len().max(1)) else { break };
let wrap = p::bool(v, "wrap").unwrap_or(true);
// Scan forward first, including the rest of the caret's paragraph. A second pass
// optionally visits the earlier part; never select the first typo repeatedly in
// a single-paragraph story. Dialogs disable wrapping so Ignore Once can finish.
for (path, before) in paths
.iter()
.skip(start)
.map(|path| (path, false))
.chain(paths.iter().take(if wrap { start.saturating_add(1) } else { 0 }).map(|path| (path, true)))
{
for (a, b, i) in para_issues(s, story, path) {
if k == 0 && a.off < caret.off && start + k < paths.len() && paths.len() > 1 {
if *path == caret.path && (a.off < caret.off) != before {
continue;
}
let r = issue_json(s, &a, &b, &i);
@@ -103,8 +120,13 @@ pub(super) fn all_issues(s: &mut Session, _: &Value) -> CmdResult {
}
pub(super) fn suggestions(s: &mut Session, v: &Value) -> CmdResult {
let at = v.get("pos").and_then(crate::cmd::parse_pos).unwrap_or_else(|| s.sel.focus.clone());
match issue_at(s, &at) {
// Explicit positions are point lookups, even when they equal the selection focus.
// Only an implicit or unparseable position keeps the exact selected issue.
let issue = match v.get("pos").and_then(crate::cmd::parse_pos) {
Some(at) => para_issues(s, at.story, &at.path).into_iter().find(|(a, b, _)| a.off <= at.off && at.off <= b.off),
None => issue_at(s, &s.sel.focus),
};
match issue {
Some((a, b, i)) => Ok(issue_json(s, &a, &b, &i)),
None => Ok(Value::Null),
}
+63
View File
@@ -1058,3 +1058,66 @@ fn rtf_tracked_changes_accept_and_reject_after_opening_and_round_trip() {
}
}
}
#[cfg(test)]
mod spelling_navigation_tests {
use super::*;
use wordcraft_doc::{Document, Pos, StoryRef};
use crate::Selection;
#[test]
fn spelling_review_dialog_navigation_moves_forward_and_wraps_only_when_requested() {
for text in ["The brwn fox and brwn dog.", "The brwn fox.\nThe brwn dog."] {
let mut s = Session::new(Document::from_text(text));
let first = s.run("review.spelling", &json!({"wrap": false})).unwrap();
assert_eq!(first["text"], "brwn");
let first_selection = s.sel.clone();
let second = s.run("review.spelling", &json!({"wrap": false})).unwrap();
assert_eq!(second["text"], "brwn");
assert_ne!(s.sel, first_selection);
assert_eq!(s.run("review.spelling", &json!({"wrap": false})).unwrap()["done"], true);
assert_eq!(s.run("review.spelling", &json!({})).unwrap()["start"], first["start"]);
}
}
#[test]
fn spelling_review_dialog_navigation_resolves_selected_spelling_before_overlapping_grammar() {
for reverse in [false, true] {
let mut s = Session::new(Document::from_text("A sentense sentense."));
s.sel = Selection::caret(Pos::body(0, 11));
let selected = s.run("review.spelling", &json!({"wrap": false})).unwrap();
assert_eq!(selected["kind"], "spelling");
assert_eq!(selected["start"]["off"], 11);
assert_eq!(selected["text"], "sentense");
let issues = s.run("review.issues", &json!({})).unwrap();
assert!(issues.as_array().unwrap().iter().any(|i| i["kind"] == "grammar" && i["text"] == "sentense sentense"));
if reverse {
std::mem::swap(&mut s.sel.anchor, &mut s.sel.focus);
}
assert_eq!(s.run("review.suggestions", &json!({})).unwrap(), selected);
let before_lookup = s.sel.clone();
let focus = s.sel.focus.off;
let explicit = s.run("review.suggestions", &json!({"pos": {"block": 0, "off": focus}})).unwrap();
assert_eq!(explicit["kind"], "grammar", "an explicit focus is a point lookup in either selection direction");
assert_eq!(explicit["text"], "sentense sentense");
assert_eq!(explicit["start"]["off"], 2);
assert_eq!(explicit["end"]["off"], 19);
assert_eq!(s.sel, before_lookup, "suggestion lookups do not move the selection");
for pos in [serde_json::Value::Null, json!({"off": "invalid"})] {
assert_eq!(
s.run("review.suggestions", &json!({"pos": pos})).unwrap(),
selected,
"unparseable positions preserve implicit selection lookup"
);
}
assert_eq!(s.run("review.suggestions", &json!({})).unwrap(), selected);
let elsewhere = s.run("review.suggestions", &json!({"pos": {"block": 0, "off": 5}})).unwrap();
assert_eq!(elsewhere["kind"], "spelling");
assert_eq!(elsewhere["start"]["off"], 2, "an explicit other position keeps point lookup");
s.run("review.applySuggestion", &json!({"text": "sentence"})).unwrap();
assert_eq!(s.doc.plain_text(StoryRef::Body), "A sentense sentence.");
s.run("edit.undo", &json!({})).unwrap();
assert_eq!(s.doc.plain_text(StoryRef::Body), "A sentense sentense.");
}
}
}
+105 -19
View File
@@ -1,6 +1,8 @@
use super::{
Arc, Block, CharProps, CmdError, CmdResult, Document, Paragraph, RevisionKind, Selection, Session, StoryRef, Value, json, p, para_block,
Arc, Block, CharProps, CmdError, CmdResult, Document, InlineObject, Paragraph, RevisionKind, Selection, Session, StoryRef, Value, json, p,
para_block,
};
use wordcraft_doc::para::OBJ;
/// Compare and Combine take the same settings.
pub(super) const COMPARE_PARAMS: &str = r#"{"path"?: string (revised document), "text"?: string (revised text), "original"?: string (original document; default: this one), "author"?: string (label changes with), "caseChanges"?: bool, "whiteSpace"?: bool, "formatting"?: bool (default true: differences count), "level"?: "word|character", "showIn"?: "original|revised|new" (revised/new: an untitled document), "moves"|"comments"|"tables"|"headersFooters"|"footnotes"|"textBoxes"|"fields"?: bool (accepted; not honoured yet except tables, always compared)}"#;
@@ -64,7 +66,13 @@ pub(super) fn compare(s: &mut Session, v: &Value) -> CmdResult {
let author = p::str(v, "author").map(str::trim).filter(|a| !a.is_empty()).map(str::to_string).unwrap_or_else(|| s.author.clone());
let date = crate::cmd::now_iso();
let old: Vec<Arc<Block>> = s.doc.body.clone();
let new: Vec<Arc<Block>> = revised.body.clone();
let new: Vec<Arc<Block>> = if revised.parts.values().any(|p| p.kind == wordcraft_doc::PartKind::TextBox) {
// The revised document's text boxes get stories of their own in the result, so its
// boxes never show a story of the original's (or none at all).
s.doc.adopt_fragment(&revised.body_fragment()).into_iter().map(Arc::new).collect()
} else {
revised.body.clone()
};
s.doc.revisions.push(wordcraft_doc::Revision { kind: RevisionKind::Insert, author: author.clone(), date: date.clone(), move_name: None });
let ins = (s.doc.revisions.len() - 1) as u32;
s.doc.revisions.push(wordcraft_doc::Revision { kind: RevisionKind::Delete, author: author.clone(), date: date.clone(), move_name: None });
@@ -86,6 +94,16 @@ pub(super) fn compare(s: &mut Session, v: &Value) -> CmdResult {
let (a, b) = (old.get(i), new.get(j));
// The revised block, with its formatting changes tracked.
if let (Some(Block::Para(pa)), Some(Block::Para(pb))) = (a.map(|x| &**x), b.map(|x| &**x)) {
if !same_objects(&s.doc, pa, pb) {
// Same text, but objects (text boxes, pictures…) differ: diffed, so the
// original's objects aren't lost.
out.push(para_block(word_diff(&s.doc, pa, pb, ins, del, &opts, &mut budget)));
changes += 1;
i += 1;
j += 1;
k += 1;
continue;
}
let mut p = pb.clone();
if opts.formatting && pa.text == pb.text {
let before = s.doc.revisions.len();
@@ -111,7 +129,9 @@ pub(super) fn compare(s: &mut Session, v: &Value) -> CmdResult {
for x in 0..dels.max(inss) {
let (a, b) = (if x < dels { old.get(i + x) } else { None }, if x < inss { new.get(j + x) } else { None });
match (a.map(|x| &**x), b.map(|x| &**x)) {
(Some(Block::Para(pa)), Some(Block::Para(pb))) => out.push(para_block(word_diff(pa, pb, ins, del, &opts, &mut budget))),
(Some(Block::Para(pa)), Some(Block::Para(pb))) => {
out.push(para_block(word_diff(&s.doc, pa, pb, ins, del, &opts, &mut budget)))
}
_ => {
if let Some(x) = a {
out.push(Arc::new(mark_block(x, &|c| c.del = Some(del), 0)));
@@ -308,30 +328,97 @@ fn flatten_region(out: &mut Vec<Op>, region: &[(Op, usize, Kind)]) {
out.extend(std::iter::repeat_n(Op::Ins, inss));
}
/// Words with the spaces after them ("The ", "cat "), or single characters.
fn tokens(s: &str, characters: bool) -> Vec<String> {
if characters {
return s.chars().map(String::from).collect();
/// A token of a changed paragraph: a word (or character), or an inline object without text
/// of its own (a text box anchor, a picture, a bookmark…), which is compared and kept whole.
enum Tok<'a> {
Text(String),
Obj(&'a InlineObject),
}
/// What a comparison sees of a token: the text as the options say, an object with the text of
/// the text boxes it shows instead of their story ids (each document numbers its own stories).
#[derive(PartialEq)]
enum Key {
Text(String),
Obj(Box<InlineObject>, Vec<String>),
}
fn obj_key(doc: &Document, o: &InlineObject) -> Key {
let texts = o.text_boxes().into_iter().map(|id| doc.plain_text(StoryRef::Part(id))).collect();
let mut o = o.clone();
for story in o.text_box_slots() {
*story = None;
}
Key::Obj(Box::new(o), texts)
}
/// Whether two paragraphs hold the same objects, as compared.
fn same_objects(doc: &Document, a: &Paragraph, b: &Paragraph) -> bool {
a.objects.len() == b.objects.len() && a.objects.iter().zip(&b.objects).all(|(x, y)| x == y || obj_key(doc, x) == obj_key(doc, y))
}
/// Words with the spaces after them ("The ", "cat "), or single characters, and the objects
/// between them. Objects with text (fields, equations…) count as their text.
fn tokens(p: &Paragraph, characters: bool) -> Vec<Tok<'_>> {
fn push(v: &mut Vec<Tok<'_>>, cur: &mut String, c: char, characters: bool) {
cur.push(c);
if characters || c == ' ' {
v.push(Tok::Text(std::mem::take(cur)));
}
}
let mut v = Vec::new();
let mut cur = String::new();
for c in s.chars() {
cur.push(c);
if c == ' ' {
v.push(std::mem::take(&mut cur));
let mut objects = p.objects.iter();
for c in p.text.chars() {
if c != OBJ {
push(&mut v, &mut cur, c, characters);
continue;
}
match objects.next() {
Some(o) if o.plain_text().is_empty() => {
if !cur.is_empty() {
v.push(Tok::Text(std::mem::take(&mut cur)));
}
v.push(Tok::Obj(o));
}
Some(o) => o.plain_text().chars().for_each(|c| push(&mut v, &mut cur, c, characters)),
None => {
// A marker without its object (a broken paragraph): kept as a visible
// placeholder, so the loss shows in the result instead of vanishing.
log::warn!("compare: an object marker without its object");
push(&mut v, &mut cur, char::REPLACEMENT_CHARACTER, characters);
}
}
}
if !cur.is_empty() {
v.push(cur);
v.push(Tok::Text(cur));
}
v
}
fn word_diff(a: &Paragraph, b: &Paragraph, ins: u32, del: u32, opts: &CompareOptions, budget: &mut u64) -> Paragraph {
/// Add a token at the end of `p` with `props`.
fn put(p: &mut Paragraph, t: Option<&Tok<'_>>, props: &CharProps) {
let at = p.len();
match t {
Some(Tok::Text(t)) => {
let _ = p.insert_text(at, t, props);
}
Some(Tok::Obj(o)) => {
let _ = p.insert_object(at, (*o).clone(), props);
}
None => {}
}
}
fn word_diff(doc: &Document, a: &Paragraph, b: &Paragraph, ins: u32, del: u32, opts: &CompareOptions, budget: &mut u64) -> Paragraph {
let (at, bt) = (a.plain_text(), b.plain_text());
let chars = opts.characters && at.chars().count().max(bt.chars().count()) <= MAX_CHAR_DIFF;
let (ta, tb) = (tokens(&at, chars), tokens(&bt, chars));
let ops = lcs_ops(&ta.iter().map(|t| opts.key(t)).collect::<Vec<_>>(), &tb.iter().map(|t| opts.key(t)).collect::<Vec<_>>(), budget);
let (ta, tb) = (tokens(a, chars), tokens(b, chars));
let key = |t: &Tok<'_>| match t {
Tok::Text(t) => Key::Text(opts.key(t)),
Tok::Obj(o) => obj_key(doc, o),
};
let ops = lcs_ops(&ta.iter().map(key).collect::<Vec<_>>(), &tb.iter().map(key).collect::<Vec<_>>(), budget);
// Only at character granularity: a matched whole word almost never happens by coincidence,
// but a matched single character very often does (digits, punctuation), so only there does
// a short interior match need folding back into its surrounding edit.
@@ -341,20 +428,19 @@ fn word_diff(a: &Paragraph, b: &Paragraph, ins: u32, del: u32, opts: &CompareOpt
let base = a.runs.first().map(|r| r.props.clone()).unwrap_or_default();
let (mut i, mut j) = (0, 0);
for op in ops {
let at = p.len();
match op {
Op::Same => {
// Equal as compared (perhaps up to case or spacing): the revised text.
let _ = p.insert_text(at, tb.get(j).map(String::as_str).unwrap_or(""), &base);
put(&mut p, tb.get(j), &base);
i += 1;
j += 1;
}
Op::Del => {
let _ = p.insert_text(at, ta.get(i).map(String::as_str).unwrap_or(""), &CharProps { del: Some(del), ..base.clone() });
put(&mut p, ta.get(i), &CharProps { del: Some(del), ..base.clone() });
i += 1;
}
Op::Ins => {
let _ = p.insert_text(at, tb.get(j).map(String::as_str).unwrap_or(""), &CharProps { ins: Some(ins), ..base.clone() });
put(&mut p, tb.get(j), &CharProps { ins: Some(ins), ..base.clone() });
j += 1;
}
}
+80
View File
@@ -80,6 +80,86 @@ fn compare_at_word_level_still_matches_a_whole_unchanged_word() {
);
}
#[test]
fn compare_keeps_text_boxes_of_the_original() {
// Text boxes are anchored in body paragraphs: one after changed text, one alone in an
// otherwise empty paragraph, one after text the revised document keeps. Their anchors must
// survive the compare (marked deleted, since the revised text has no boxes), so no text is lost.
for show_in in ["original", "revised", "new"] {
let mut s = Session::new(Document::from_text("Body one.\n\nKeep."));
let mut boxes = Vec::new();
for (para, off, text) in [(0usize, "Body one.".len(), "Alpha boxword gamma."), (1, 0, "Vml onlyword here."), (2, "Keep.".len(), "Kept box.")]
{
s.run("caret.set", &json!({"pos": Pos::body(para, off)})).unwrap();
let id = s.run("insert.textBox", &json!({"text": text})).unwrap()["story"].as_u64().unwrap() as u32;
boxes.push((id, text));
}
s.run("caret.set", &json!({"pos": Pos::body(0, 0)})).unwrap();
s.run("review.compare", &json!({"text": "Totally new first.\nSecond new line.\nKeep.", "showIn": show_in})).unwrap();
for (id, text) in &boxes {
let anchor = s.doc.text_box_anchor(*id).unwrap_or_else(|| panic!("{show_in}: the box \"{text}\" lost its anchor"));
assert_eq!(s.doc.plain_text(StoryRef::Part(*id)), *text);
let p = s.doc.para(anchor.story, &anchor.path).unwrap();
assert!(
p.props_of_char(anchor.off - wordcraft_doc::para::OBJ.len_utf8()).del.is_some(),
"{show_in}: the box \"{text}\" is not marked deleted"
);
}
s.run("review.rejectAll", &json!({})).unwrap();
assert_eq!(s.doc.plain_text(StoryRef::Body), "Body one.\n\nKeep.");
for (id, text) in &boxes {
assert!(s.doc.text_box_anchor(*id).is_some(), "{show_in}: rejecting lost the box \"{text}\"");
assert_eq!(s.doc.plain_text(StoryRef::Part(*id)), *text);
}
}
}
#[test]
fn compare_with_a_revised_file_keeps_its_text_boxes_backed() {
// Text boxes in both documents, the revised one from a file: every anchor in the result
// shows a text box story of the result, and both documents' boxes are there, marked.
fn with_boxes(text: &str, boxes: &[(usize, usize, &str)]) -> Session {
let mut s = Session::new(Document::from_text(text));
for (para, off, text) in boxes {
s.run("caret.set", &json!({"pos": Pos::body(*para, *off)})).unwrap();
s.run("insert.textBox", &json!({"text": text})).unwrap();
}
s
}
/// Each text box anchored in the body: its story's text, and whether its anchor is deleted
/// or inserted.
fn boxes(d: &Document) -> Vec<(String, bool, bool)> {
let mut out = Vec::new();
for path in d.para_paths(StoryRef::Body) {
let p = d.para(StoryRef::Body, &path).unwrap();
for (off, o) in p.objects_with_offsets() {
for id in o.text_boxes() {
let part = d.parts.get(&id).unwrap_or_else(|| panic!("an anchor shows story {id}, which the result lacks"));
assert_eq!(part.kind, wordcraft_doc::PartKind::TextBox, "story {id}");
let c = p.props_of_char(off);
out.push((d.plain_text(StoryRef::Part(id)), c.del.is_some(), c.ins.is_some()));
}
}
}
out.sort();
out
}
let rev = with_boxes("Body one.\nNew middle.\nKeep.", &[(1, "New middle.".len(), "Revised box."), (2, "Keep.".len(), "Kept box.")]);
let path = std::env::temp_dir().join(format!("wc-compare-boxes-{}.docx", std::process::id()));
crate::io::save_path(&path, &rev.doc).unwrap();
let t = |t: &str, del, ins| (t.to_string(), del, ins);
for show_in in ["original", "revised", "new"] {
for (then, left) in [("review.acceptAll", "Revised box."), ("review.rejectAll", "Original box.")] {
let mut s = with_boxes("Body one.\nOld middle.\nKeep.", &[(1, "Old middle.".len(), "Original box."), (2, "Keep.".len(), "Kept box.")]);
s.run("review.compare", &json!({"path": path.to_string_lossy(), "showIn": show_in})).unwrap();
assert_eq!(boxes(&s.doc), [t("Kept box.", false, false), t("Original box.", true, false), t("Revised box.", false, true)], "{show_in}");
s.run(then, &json!({})).unwrap();
assert_eq!(boxes(&s.doc), [t("Kept box.", false, false), t(left, false, false)], "{show_in} {then}");
}
}
let _ = std::fs::remove_file(&path);
}
#[test]
fn compare_bounds_total_diff_work() {
// Many long, slightly different paragraphs: each pair alone fits one comparison table,
+47 -4
View File
@@ -18,6 +18,8 @@ pub const OPEN_EXTS: &[&str] = &[
"markdown",
"html",
"htm",
"mht",
"mhtml",
"rtf",
"odt",
"tex",
@@ -201,7 +203,7 @@ fn load_embedded_fonts(doc: &Document) {
fn read_bytes(name: &str, bytes: &[u8], password: Option<&str>, page: &wordcraft_doc::SectionProps) -> Result<(Document, bool), String> {
let ext = ext_of(name);
if matches!(ext.as_str(), "docx" | "docm" | "dotx" | "dotm" | "doc" | "dot") && wordcraft_docx::is_encrypted(bytes) {
let mut doc = wordcraft_docx::read_with_password(bytes, password).map_err(|e| e.to_string())?;
let mut doc = read_docx(bytes, password).map_err(|e| e.to_string())?;
opened(&mut doc);
return Ok((doc, true));
}
@@ -230,6 +232,42 @@ fn read_bytes(name: &str, bytes: &[u8], password: Option<&str>, page: &wordcraft
Ok((doc, false))
}
/// Read a Word package (decrypted with `password` if it is encrypted), bringing in its
/// alternative format chunks (`w:altChunk`) as Word does when it opens one: see [`import_chunk`].
pub(crate) fn read_docx(bytes: &[u8], password: Option<&str>) -> Result<Document, wordcraft_docx::DocxError> {
wordcraft_docx::read_with(bytes, &wordcraft_docx::ReadOptions { password, chunks: Some(&import_chunk) })
}
/// An alternative format chunk in a format the docx reader leaves to the format importers, as a
/// document: an MHT web archive, HTML, RTF or plain text, told by its content type, its part's
/// extension or what it holds. Its pictures fit the text width, as when the file is opened.
fn import_chunk(c: &wordcraft_docx::AltChunk) -> Option<Document> {
use wordcraft_formats::{html, mht, rtf, txt};
let full = c.content_type.to_ascii_lowercase();
let ct = full.split(';').next().unwrap_or("").trim();
let charset = full.split(';').skip(1).find_map(|p| p.trim().strip_prefix("charset=")).map(|v| v.trim_matches('"'));
let ext = ext_of(c.part);
let b = c.bytes;
let mut doc = if matches!(ct, "message/rfc822" | "application/x-mimearchive")
|| ct.starts_with("multipart/")
|| matches!(ext.as_str(), "mht" | "mhtml")
|| mht::is_mht(b)
{
mht::import(b)
} else if ct.contains("rtf") || ext == "rtf" || b.starts_with(b"{\\rtf") {
rtf::import(b).ok()?
} else if ct.contains("html") || matches!(ext.as_str(), "html" | "htm" | "xhtml") {
html::import_page(&html::decode_as(b, charset), &|_| None)
} else if ct == "text/plain" || ext == "txt" {
txt::import(b)
} else {
return None;
};
// shortcut: pictures fit the chunk's default page, not the section it lands in; fit them there if wide pictures in chunks overflow narrow sections.
fit_pictures(&mut doc);
Some(doc)
}
/// The narrowest text width of the document's sections, points: what a picture must fit.
pub(crate) fn text_width(doc: &Document) -> f32 {
doc.sections().iter().map(|(_, sp)| sp.text_width()).fold(f32::INFINITY, f32::min).clamp(36.0, 4000.0)
@@ -270,7 +308,7 @@ pub(crate) fn fit_pictures(doc: &mut Document) {
/// Does the file hold a different document format than its Word or RTF extension says? An RTF
/// file saved as `.doc` (as accounting systems and report generators often save their
/// documents), a `.docx` package renamed `.doc`, a Word 97-2003 file renamed `.docx`, a Word XML
/// Document (2003 or Flat OPC) or a web page saved as `.doc` (#2853): Word opens
/// Document (2003 or Flat OPC) or a web page (HTML or MHT) saved as `.doc` (#2853): Word opens
/// each by what it holds, so they go through [`open_by_content`] instead of failing.
fn signature_contradicts(ext: &str, bytes: &[u8]) -> bool {
let expected = match ext {
@@ -289,6 +327,8 @@ fn signature_contradicts(ext: &str, bytes: &[u8]) -> bool {
"xml"
} else if looks_like_web_page(bytes) {
"html"
} else if wordcraft_formats::mht::is_mht(bytes) {
"mht"
} else {
return false;
};
@@ -346,9 +386,9 @@ fn open_by_content(
page: &wordcraft_doc::SectionProps,
) -> Result<(Document, bool), String> {
let (mut doc, encrypted) = if bytes.starts_with(OLE_SIG) && wordcraft_docx::is_encrypted(bytes) {
(wordcraft_docx::read_with_password(bytes, password).map_err(|e| e.to_string())?, true)
(read_docx(bytes, password).map_err(|e| e.to_string())?, true)
} else if bytes.starts_with(ZIP_SIG) {
(wordcraft_docx::read(bytes).map_err(|e| format!("{name}: not a document WordCraft can open ({e})"))?, false)
(read_docx(bytes, None).map_err(|e| format!("{name}: not a document WordCraft can open ({e})"))?, false)
} else if bytes.starts_with(OLE_SIG) {
open_binary_doc(bytes, password)
.map_err(|e| if needs_password(&e) || wrong_password(&e) { e } else { format!("{name}: not a document WordCraft can open ({e})") })?
@@ -359,6 +399,9 @@ fn open_by_content(
} else if matches!(ext, "doc" | "dot" | "docx" | "docm" | "dotx" | "dotm" | "rtf") && looks_like_web_page(bytes) {
let opened = crate::io_ext::open("html", bytes, page).unwrap_or_else(|| Err(format!("{name}: can't open web pages")));
(opened?, false)
} else if matches!(ext, "doc" | "dot" | "docx" | "docm" | "dotx" | "dotm" | "rtf") && wordcraft_formats::mht::is_mht(bytes) {
let opened = crate::io_ext::open("mht", bytes, page).unwrap_or_else(|| Err(format!("{name}: can't open web archives")));
(opened?, false)
} else if wordcraft_formats::txt::is_text(bytes) {
(wordcraft_formats::txt::import_on(bytes, page), false)
} else if ext.is_empty() {
+1 -1
View File
@@ -6,7 +6,7 @@ use wordcraft_doc::Document;
/// start on `page`.
pub fn open(ext: &str, bytes: &[u8], page: &wordcraft_doc::SectionProps) -> Option<Result<Document, String>> {
match ext {
"docx" | "docm" | "dotx" | "dotm" => Some(wordcraft_docx::read(bytes).map_err(|e| e.to_string())),
"docx" | "docm" | "dotx" | "dotm" => Some(crate::io::read_docx(bytes, None).map_err(|e| e.to_string())),
"doc" | "dot" => Some(wordcraft_docbin::read(bytes).map_err(|e| e.to_string())),
// Word XML Document (Flat OPC) or Word 2003 XML, told apart by the root element.
"xml" => Some(wordcraft_docx::read_xml(bytes).map_err(|e| e.to_string())),
+1
View File
@@ -16,6 +16,7 @@ mod io_ext;
pub mod keymap;
pub(crate) mod locale;
pub mod math_gallery;
pub mod packs;
pub(crate) mod prefs;
pub mod print;
pub mod recovery;
+730
View File
@@ -0,0 +1,730 @@
//! Language packs (CP4.3): spelling dictionaries and hyphenation patterns for languages beyond
//! the embedded English, one pack per locale, downloaded and verified by the font store
//! (craft-libs `craft-font-store`, item kind `language-pack`) and read by `craft-proof` from the
//! store's `language-packs/` folder.
//!
//! The commands (`proof.packs`, `proof.installPack`, `proof.removePack`), Options › Language,
//! the missing-pack bar and About › Language data all go through this module. It never asks
//! anything: an install needs the font store's single consent setting (downloading on), which only
//! a person in the app can turn on ([`Backend::enable_network`], called from an Install click).
//! Agents (MCP, the CLI, the control channel) install only while it is on.
//!
//! The [`Backend`] is set once at start-up by the desktop app and the CLI ([`use_font_store`]).
//! Without one (tests, the web build, `ui_shot`) nothing is available and nothing is downloaded.
use std::collections::BTreeSet;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex, PoisonError, RwLock};
use serde::Serialize;
use crate::{CmdError, Session};
#[cfg(not(target_arch = "wasm32"))]
mod store;
/// Longest locale tag taken, in bytes (BCP 47 tags in practice stay under 20).
pub const MAX_LOCALE_LEN: usize = 35;
/// Most packs an index may list that are read.
const MAX_PACKS: usize = 1000;
/// Paragraphs of a document looked at for its languages (the missing-pack bar).
const MAX_PARAGRAPHS_SCANNED: usize = 20_000;
/// Distinct languages of a document collected.
const MAX_DOC_LANGUAGES: usize = 64;
/// A pack the pinned index lists.
#[derive(Clone, Debug, PartialEq, Eq, Serialize)]
pub struct Pack {
/// Lowercase BCP 47 (`nl`, `pt-pt`).
pub locale: String,
/// The language's own name (`Nederlands`).
pub name: String,
/// The craft-i18n commit the pack was built from.
pub version: String,
/// The archive.
pub url: String,
/// The upstream folder the data comes from, at its pinned commit (the index's optional
/// `source`), shown before install; the archive when the index gives none.
pub source: Option<String>,
/// Bytes of the archive.
pub size: u64,
/// SPDX expression.
pub licence: String,
/// `Embeddable` or `Download-only`.
pub class: String,
/// Has a spelling dictionary.
pub spelling: bool,
/// Has hyphenation patterns.
pub hyphenation: bool,
}
/// An installed pack.
#[derive(Clone, Debug, PartialEq, Eq, Serialize)]
pub struct Installed {
pub locale: String,
pub name: String,
pub licence: String,
pub path: PathBuf,
}
/// Stops a download. On the desktop it is the font store's own flag, which a download watches
/// even while the server sends nothing.
#[derive(Clone, Debug, Default)]
pub struct Cancel(#[cfg(not(target_arch = "wasm32"))] pub(crate) craft_font_store::Cancel, #[cfg(target_arch = "wasm32")] Arc<AtomicBool>);
impl Cancel {
pub fn new() -> Self {
Self::default()
}
#[cfg(not(target_arch = "wasm32"))]
pub fn cancel(&self) {
self.0.cancel();
}
#[cfg(not(target_arch = "wasm32"))]
pub fn is_cancelled(&self) -> bool {
self.0.is_cancelled()
}
#[cfg(target_arch = "wasm32")]
pub fn cancel(&self) {
self.0.store(true, Ordering::Relaxed);
}
#[cfg(target_arch = "wasm32")]
pub fn is_cancelled(&self) -> bool {
self.0.load(Ordering::Relaxed)
}
}
/// Where packs come from: the font store on the desktop, a fixed list in tests and `ui_shot`.
pub trait Backend: Send + Sync {
/// The pinned index as last fetched (no request); empty when it never was.
fn index(&self) -> Result<Vec<Pack>, String>;
/// Fetch the pinned index (a request; only while downloading is on).
fn fetch_index(&self, cancel: &Cancel) -> Result<Vec<Pack>, String>;
/// Whether the font store may download.
fn network_on(&self) -> bool;
/// Turn downloading on. Only from the person's own action in the app (an Install click):
/// the commands never call it, and a backend for an agent refuses.
fn enable_network(&self) -> Result<(), String>;
/// Download, verify and unpack `pack` into the packs folder. `progress` gets (done, total) bytes.
fn install(&self, pack: &Pack, cancel: &Cancel, progress: &mut dyn FnMut(u64, u64)) -> Result<(), String>;
/// Remove the installed pack for `locale`.
fn remove(&self, locale: &str) -> Result<(), String>;
/// The installed-packs folder (`<font store>/language-packs`).
fn packs_dir(&self) -> Option<PathBuf>;
/// The packs installed now.
fn installed(&self) -> Vec<Installed> {
installed_from_proof()
}
/// The test index in use instead of the pinned one (`CRAFT_LANGUAGE_PACK_INDEX`), if any.
fn index_override(&self) -> Option<String> {
None
}
}
static BACKEND: RwLock<Option<Arc<dyn Backend>>> = RwLock::new(None);
/// The index last read, so the bar and the list don't read the store's cache every frame.
static INDEX: Mutex<Option<Vec<Pack>>> = Mutex::new(None);
/// Bumped whenever [`INDEX`] changes.
static INDEX_GENERATION: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(0);
fn set_index(index: Option<Vec<Pack>>) {
*INDEX.lock().unwrap_or_else(PoisonError::into_inner) = index;
INDEX_GENERATION.fetch_add(1, Ordering::Relaxed);
}
/// Changes whenever the list of available packs does (a fetch, another backend).
pub fn index_generation() -> u64 {
INDEX_GENERATION.load(Ordering::Relaxed)
}
/// Use `backend` for packs from now on (`None`: no packs), and point proofing at its folder.
pub fn set_backend(backend: Option<Arc<dyn Backend>>) {
let dir = backend.as_ref().and_then(|b| b.packs_dir());
*BACKEND.write().unwrap_or_else(PoisonError::into_inner) = backend;
set_index(None);
jobs().errors.clear();
wordcraft_proof::packs::set_packs_dir(dir);
}
/// Use the suite's font store for packs (`<font store>/language-packs`), as the person in the app
/// (`user`: an Install click may turn downloading on) or as an agent (the CLI, headless MCP: it
/// never can). The pack index is the pinned one, or `CRAFT_LANGUAGE_PACK_INDEX` (a URL or a path,
/// for testing packs before they are published), which the font store reads itself. Returns
/// whether there is a store folder.
#[cfg(not(target_arch = "wasm32"))]
pub fn use_font_store(user: bool) -> bool {
match store::FontStoreBackend::new(user) {
Some(b) => {
set_backend(Some(Arc::new(b)));
true
}
None => false,
}
}
/// [`use_font_store`] with the store in `dir` instead of the person's (`ui_shot`, tests).
#[cfg(not(target_arch = "wasm32"))]
pub fn use_font_store_in(dir: PathBuf, user: bool) {
set_backend(Some(Arc::new(store::FontStoreBackend::in_dir(dir, user))));
}
/// The backend in use.
pub fn backend() -> Option<Arc<dyn Backend>> {
BACKEND.read().unwrap_or_else(PoisonError::into_inner).clone()
}
fn installed_from_proof() -> Vec<Installed> {
wordcraft_proof::packs::installed().into_iter().map(|p| Installed { locale: p.locale, name: p.name, licence: p.licence, path: p.path }).collect()
}
/// The test source packs come from instead of the pinned index (`CRAFT_LANGUAGE_PACK_INDEX`):
/// Options › Language says so.
pub fn index_override() -> Option<String> {
backend().and_then(|b| b.index_override())
}
/// The packs installed now (none without a backend).
pub fn installed() -> Vec<Installed> {
backend().map(|b| b.installed()).unwrap_or_default()
}
/// The available packs: the index as last read, read from the store's cache the first time.
pub fn available() -> Vec<Pack> {
let Some(b) = backend() else { return Vec::new() };
let mut cached = INDEX.lock().unwrap_or_else(PoisonError::into_inner);
if let Some(i) = cached.as_ref() {
return i.clone();
}
let index = clean_index(b.index().unwrap_or_default());
*cached = Some(index.clone());
drop(cached);
INDEX_GENERATION.fetch_add(1, Ordering::Relaxed);
index
}
/// How many packs [`available`] lists, without copying them (the window asks every frame).
pub fn available_len() -> usize {
if let Some(i) = INDEX.lock().unwrap_or_else(PoisonError::into_inner).as_ref() {
return i.len();
}
available().len()
}
/// Fetch the index again (downloading must be on).
pub fn refresh(cancel: &Cancel) -> Result<Vec<Pack>, String> {
let b = backend().ok_or_else(no_store)?;
if !b.network_on() {
return Err(network_off());
}
let index = clean_index(b.fetch_index(cancel)?);
set_index(Some(index.clone()));
Ok(index)
}
/// Drop packs with a malformed locale, duplicates and English (embedded), and cap the list.
fn clean_index(mut packs: Vec<Pack>) -> Vec<Pack> {
packs.truncate(MAX_PACKS);
let mut seen = BTreeSet::new();
packs.retain(|p| valid_locale(&p.locale) && !is_embedded(&p.locale) && seen.insert(p.locale.clone()));
packs.sort_by(|a, b| a.locale.cmp(&b.locale));
packs
}
/// English is embedded in every build; it never comes as a pack.
fn is_embedded(locale: &str) -> bool {
locale == "en" || locale.starts_with("en-")
}
/// A lowercase BCP 47-shaped tag: letters, digits and `-`, starting with a letter.
pub fn valid_locale(locale: &str) -> bool {
!locale.is_empty()
&& locale.len() <= MAX_LOCALE_LEN
&& locale.starts_with(|c: char| c.is_ascii_lowercase())
&& locale.chars().all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-')
&& !locale.contains("--")
&& !locale.ends_with('-')
}
/// A locale from a command's params: trimmed, lowercased, `_` as `-`.
pub fn parse_locale(raw: &str) -> Result<String, CmdError> {
let l = raw.trim().replace('_', "-").to_ascii_lowercase();
if valid_locale(&l) { Ok(l) } else { Err(CmdError::Params(format!("`locale` must be a language tag like nl or pt-PT, not {raw:?}"))) }
}
fn no_store() -> String {
if cfg!(target_arch = "wasm32") {
"Language packs aren't available in the browser".into()
} else {
"there is no font store folder for language packs".into()
}
}
fn network_off() -> String {
"downloading is turned off: turn it on in WordCraft (File › Options › Language, or an Install button there)".into()
}
/// The pack for `locale` in the index.
pub fn find(locale: &str) -> Option<Pack> {
available().into_iter().find(|p| p.locale == locale)
}
/// Whether a pack is installed for `locale`.
pub fn is_installed(locale: &str) -> bool {
installed().iter().any(|p| p.locale == locale)
}
/// Install the pack for `locale` (downloading must be on): the commands and the Install button.
/// Proofing picks it up at once.
pub fn install(locale: &str, cancel: &Cancel, progress: &mut dyn FnMut(u64, u64)) -> Result<Installed, String> {
let b = backend().ok_or_else(no_store)?;
let pack = find(locale).ok_or_else(|| format!("there is no language pack for {locale}"))?;
if !b.network_on() {
return Err(network_off());
}
if let Some(dir) = b.packs_dir() {
crate::file_access::check_write(&dir.to_string_lossy())?;
}
b.install(&pack, cancel, progress)?;
reload(&b);
b.installed().into_iter().find(|p| p.locale == locale).ok_or_else(|| format!("the {locale} pack didn't install"))
}
/// Remove the installed pack for `locale`.
pub fn remove(locale: &str) -> Result<(), String> {
let b = backend().ok_or_else(no_store)?;
if jobs().installs.iter().any(|(l, ..)| l == locale) {
return Err(format!("the {locale} pack is being installed: cancel it first"));
}
if !b.installed().iter().any(|p| p.locale == locale) {
return Err(format!("no language pack is installed for {locale}"));
}
if let Some(dir) = b.packs_dir() {
crate::file_access::check_write(&dir.to_string_lossy())?;
}
b.remove(locale)?;
reload(&b);
Ok(())
}
/// What the app does with a long pack request (an install or a fetch of the list): `None` runs it
/// in the caller (the CLI, headless MCP, tests); `Some(notify)` runs it on a thread, so the window
/// keeps drawing whoever asked (the person, the control channel, MCP), and calls `notify` on each
/// change (the window repaints). Set once by the desktop app.
pub type Notify = Arc<dyn Fn() + Send + Sync>;
static BACKGROUND: RwLock<Option<Notify>> = RwLock::new(None);
/// Installs and fetches running on a thread, and how the finished ones ended.
#[derive(Default)]
struct Jobs {
/// Running installs by locale: (bytes done, bytes in all, its Cancel).
installs: Vec<(String, u64, u64, Cancel)>,
fetching: bool,
/// The last failure of each locale's install ("" for the list), until it is tried again.
errors: Vec<(String, String)>,
/// Bumped as each job ends: the window relays out and shows the result.
finished: u64,
}
static JOBS: Mutex<Jobs> = Mutex::new(Jobs { installs: Vec::new(), fetching: false, errors: Vec::new(), finished: 0 });
fn jobs() -> std::sync::MutexGuard<'static, Jobs> {
JOBS.lock().unwrap_or_else(PoisonError::into_inner)
}
/// Run long pack requests on a thread (`Some`: the desktop app), or in the caller (`None`).
pub fn set_background(notify: Option<Notify>) {
*BACKGROUND.write().unwrap_or_else(PoisonError::into_inner) = notify;
}
fn background() -> Option<Notify> {
if cfg!(target_arch = "wasm32") {
return None;
}
BACKGROUND.read().unwrap_or_else(PoisonError::into_inner).clone()
}
/// Whether installs and fetches run on a thread ([`set_background`]).
pub fn runs_in_background() -> bool {
background().is_some()
}
/// A running install: (locale, bytes done, bytes in all).
#[derive(Clone, Debug, PartialEq, Eq, Serialize)]
pub struct Progress {
pub locale: String,
pub done: u64,
pub total: u64,
}
/// The installs running now.
pub fn installing() -> Vec<Progress> {
jobs().installs.iter().map(|(l, d, t, _)| Progress { locale: l.clone(), done: *d, total: *t }).collect()
}
/// Whether the list is being fetched.
pub fn fetching() -> bool {
jobs().fetching
}
/// The last failure of `locale`'s install ("" for the list), until it is tried again.
pub fn error(locale: &str) -> Option<String> {
jobs().errors.iter().find(|(l, _)| l == locale).map(|(_, e)| e.clone())
}
/// Every recorded failure, (locale or "" for the list, message).
pub fn errors() -> Vec<(String, String)> {
jobs().errors.clone()
}
/// Counts finished jobs: when it changes, the window lays the documents out again.
pub fn finished() -> u64 {
jobs().finished
}
/// Stop `locale`'s running install.
pub fn cancel(locale: &str) {
if let Some((_, _, _, c)) = jobs().installs.iter().find(|(l, ..)| l == locale) {
c.cancel();
}
}
fn set_error(locale: &str, e: Option<String>) {
let mut j = jobs();
j.errors.retain(|(l, _)| l != locale);
if let Some(e) = e {
j.errors.push((locale.to_string(), e));
}
}
/// Start installing `locale` ([`install`]): on a thread when [`runs_in_background`], else now.
/// What's wrong up front (no pack, downloading off, outside the write root) is an error at once.
/// Returns whether it is still running (`false`: done, or it already was running).
pub fn start_install(locale: &str) -> Result<bool, String> {
let b = backend().ok_or_else(no_store)?;
let pack = find(locale).ok_or_else(|| format!("there is no language pack for {locale}"))?;
if !b.network_on() {
return Err(network_off());
}
if let Some(dir) = b.packs_dir() {
crate::file_access::check_write(&dir.to_string_lossy())?;
}
let Some(notify) = background() else {
let r = install(locale, &Cancel::new(), &mut |_, _| {});
set_error(locale, r.as_ref().err().cloned());
return r.map(|_| false);
};
let cancel = Cancel::new();
{
let mut j = jobs();
if j.installs.iter().any(|(l, ..)| l == locale) {
return Ok(true);
}
j.installs.push((locale.to_string(), 0, pack.size, cancel.clone()));
j.errors.retain(|(l, _)| l != locale);
}
let key = locale.to_string();
let locale = locale.to_string();
let started = spawn(move || {
let progress_notify = notify.clone();
let r = install(&locale, &cancel, &mut |done, total| {
if let Some(job) = jobs().installs.iter_mut().find(|(l, ..)| *l == locale) {
job.1 = done;
job.2 = total;
}
progress_notify();
});
let mut j = jobs();
j.installs.retain(|(l, ..)| *l != locale);
if let Err(e) = r {
j.errors.push((locale, e));
}
j.finished = j.finished.wrapping_add(1);
drop(j);
notify();
});
if let Err(e) = started {
jobs().installs.retain(|(l, ..)| *l != key);
return Err(e);
}
Ok(true)
}
/// Start fetching the list again ([`refresh`]): on a thread when [`runs_in_background`], else now.
/// Returns whether it is still running.
pub fn start_refresh() -> Result<bool, String> {
let b = backend().ok_or_else(no_store)?;
if !b.network_on() {
return Err(network_off());
}
let Some(notify) = background() else {
let r = refresh(&Cancel::new());
set_error("", r.as_ref().err().cloned());
return r.map(|_| false);
};
{
let mut j = jobs();
if j.fetching {
return Ok(true);
}
j.fetching = true;
j.errors.retain(|(l, _)| !l.is_empty());
}
let started = spawn(move || {
let r = refresh(&Cancel::new());
let mut j = jobs();
j.fetching = false;
if let Err(e) = r {
j.errors.push((String::new(), e));
}
j.finished = j.finished.wrapping_add(1);
drop(j);
notify();
});
if let Err(e) = started {
jobs().fetching = false;
return Err(e);
}
Ok(true)
}
#[cfg(not(target_arch = "wasm32"))]
fn spawn(f: impl FnOnce() + Send + 'static) -> Result<(), String> {
std::thread::Builder::new().name("language-pack".into()).spawn(f).map(|_| ()).map_err(|e| format!("couldn't start the download: {e}"))
}
#[cfg(target_arch = "wasm32")]
fn spawn(_: impl FnOnce() + Send + 'static) -> Result<(), String> {
Err(no_store())
}
/// Have proofing look at the packs folder again (its dictionaries are reloaded).
fn reload(b: &Arc<dyn Backend>) {
if wordcraft_proof::packs::packs_dir() == b.packs_dir() {
wordcraft_proof::packs::reload();
} else {
wordcraft_proof::packs::set_packs_dir(b.packs_dir());
}
}
/// The full licence texts for About › Language data: the embedded English data, then each
/// installed pack.
pub fn licence_texts() -> Vec<(String, String)> {
let mut texts = wordcraft_proof::packs::licence_texts();
// SCOWL's notices (with the Ispell BSD licence of its affix files), when English spells from
// the embedded SCOWL data; after the Moby and supplement entries.
if let Some(notice) = wordcraft_proof::scowl::notice() {
let at = texts.len().min(2);
texts.insert(at, (SCOWL_TITLE.into(), notice.into()));
}
texts
}
/// About › Language data's title for SCOWL's notices.
pub const SCOWL_TITLE: &str = "English spelling dictionaries (SCOWL)";
/// The pack locale for a document language tag: the whole tag (`pt-pt`), else its language
/// (`nl` for `nl-NL`), whichever the index has. `None` for English and tags with no pack.
pub fn pack_for_tag<'a>(tag: &str, packs: &'a [Pack]) -> Option<&'a Pack> {
let tag = tag.trim().replace('_', "-").to_ascii_lowercase();
let base = tag.split('-').next().unwrap_or("");
if base.is_empty() || base == "en" {
return None;
}
packs.iter().find(|p| p.locale == tag).or_else(|| packs.iter().find(|p| p.locale == base))
}
/// The languages (`w:lang`, resolved through the styles) of the body text, in first-seen order.
pub fn document_languages(s: &Session) -> Vec<String> {
let mut out: Vec<String> = Vec::new();
for path in s.doc.para_paths(wordcraft_doc::StoryRef::Body).into_iter().take(MAX_PARAGRAPHS_SCANNED) {
let Some(p) = s.doc.para(wordcraft_doc::StoryRef::Body, &path) else { continue };
if p.text.trim().is_empty() {
continue;
}
let style = p.props.style.as_deref();
for (range, c) in p.run_ranges() {
let Some(text) = p.text.get(range) else { continue };
if !text.chars().any(char::is_alphabetic) {
continue;
}
let rc = s.doc.styles.resolve_char_in(style, None, c);
if rc.no_proof {
continue;
}
let tag = rc.lang.unwrap_or_else(|| "en-US".into());
if !out.contains(&tag) {
out.push(tag);
if out.len() >= MAX_DOC_LANGUAGES {
return out;
}
}
}
}
out
}
/// Whether proofing reads a pack for text in `tag`: every language craft-proof can name, its own
/// (Polish…) and any other (`ProofLang::Pack`, read from the pack alone); not `und`, `zxx` or a
/// malformed tag. The one place the bar's language rule lives.
pub fn proofable(tag: &str) -> bool {
wordcraft_proof::ProofLang::from_tag(Some(tag)) != wordcraft_proof::ProofLang::Other
}
/// The Hunspell dictionary `tag`'s language needs and doesn't have, for the status bar's
/// "(no dictionary)": craft-proof's [`wordcraft_proof::missing_dictionary`], except for a
/// language craft-proof has no rules of its own for (`ProofLang::Pack`: Japanese, Chinese,
/// Thai…) when no pack for it is available either, as no Hunspell dictionary may exist at all.
pub fn missing_dictionary(tag: Option<&str>) -> Option<&'static str> {
let name = wordcraft_proof::missing_dictionary(tag)?;
let lang = wordcraft_proof::ProofLang::from_tag(tag);
if lang.pack_locale().is_some() && pack_for_tag(tag.unwrap_or(""), &available()).is_none() {
return None;
}
Some(name)
}
/// The packs the document's text asks for that are available and not installed: what the
/// missing-pack bar offers. Never a language without a pack, nor one proofing can't use.
pub fn missing(s: &Session) -> Vec<Pack> {
let packs = available();
if packs.is_empty() {
return Vec::new();
}
let installed = installed();
let mut out: Vec<Pack> = Vec::new();
for tag in document_languages(s) {
if proofable(&tag)
&& let Some(p) = pack_for_tag(&tag, &packs)
&& !installed.iter().any(|i| i.locale == p.locale)
&& !out.iter().any(|o| o.locale == p.locale)
{
out.push(p.clone());
}
}
out
}
/// A backend with a fixed list and installs that only record themselves: tests and `ui_shot`.
#[derive(Default)]
pub struct FixedBackend {
pub packs: Vec<Pack>,
pub network: AtomicBool,
/// Agents can't turn the network on.
pub agent: bool,
pub installed: Mutex<Vec<Installed>>,
}
impl Pack {
/// Where the data comes from: the upstream source when the index names it, else the archive.
pub fn source_link(&self) -> &str {
self.source.as_deref().filter(|s| !s.is_empty()).unwrap_or(&self.url)
}
}
impl FixedBackend {
/// A few packs shaped like the published ones, for screenshots and tests.
pub fn sample() -> Self {
let pack = |locale: &str, name: &str, size: u64, licence: &str, class: &str| Pack {
locale: locale.into(),
name: name.into(),
version: "0000000".into(),
url: format!("https://github.com/storytold/craft-i18n/releases/download/language-packs-1/{locale}.tar.zst"),
source: Some("https://github.com/LibreOffice/dictionaries".into()),
size,
licence: licence.into(),
class: class.into(),
spelling: true,
hyphenation: true,
};
FixedBackend {
packs: vec![
pack("ca", "Català", 1_480_000, "GPL-2.0-or-later OR LGPL-2.1-or-later", "Download-only"),
pack("el", "Ελληνικά", 2_310_000, "MPL-1.1 OR GPL-2.0-or-later OR LGPL-2.1-or-later", "Embeddable"),
pack("fr", "Français", 1_120_000, "MPL-2.0", "Embeddable"),
pack("nl", "Nederlands", 1_950_000, "BSD-3-Clause", "Embeddable"),
pack("pl", "Polski", 4_620_000, "GPL-2.0-or-later OR LGPL-2.1-or-later OR MPL-1.1 OR Apache-2.0 OR CC-BY-4.0", "Embeddable"),
pack("sk", "Slovenčina", 2_040_000, "GPL-2.0-only OR LGPL-2.1-only OR MPL-1.1", "Download-only"),
],
..Default::default()
}
}
}
impl Backend for FixedBackend {
fn index(&self) -> Result<Vec<Pack>, String> {
Ok(self.packs.clone())
}
fn fetch_index(&self, _: &Cancel) -> Result<Vec<Pack>, String> {
Ok(self.packs.clone())
}
fn network_on(&self) -> bool {
self.network.load(Ordering::Relaxed)
}
fn enable_network(&self) -> Result<(), String> {
if self.agent {
return Err("only the user can turn on downloading".into());
}
self.network.store(true, Ordering::Relaxed);
Ok(())
}
fn install(&self, pack: &Pack, cancel: &Cancel, progress: &mut dyn FnMut(u64, u64)) -> Result<(), String> {
if cancel.is_cancelled() {
return Err("cancelled".into());
}
progress(pack.size, pack.size);
let mut i = self.installed.lock().unwrap_or_else(PoisonError::into_inner);
i.retain(|p| p.locale != pack.locale);
i.push(Installed { locale: pack.locale.clone(), name: pack.name.clone(), licence: pack.licence.clone(), path: PathBuf::new() });
Ok(())
}
fn remove(&self, locale: &str) -> Result<(), String> {
self.installed.lock().unwrap_or_else(PoisonError::into_inner).retain(|p| p.locale != locale);
Ok(())
}
fn packs_dir(&self) -> Option<PathBuf> {
None
}
fn installed(&self) -> Vec<Installed> {
self.installed.lock().unwrap_or_else(PoisonError::into_inner).clone()
}
}
#[cfg(test)]
mod tests {
/// CP4.4 (`cargo xtask language`): a build embeds English Language data and nothing it
/// wasn't asked to. A Language build names the locales it embeds in `CRAFT_PROOF_EMBED`
/// (`en` alone: English only); the default embeds the carry-over locales too, until Language
/// packs are published and pinned (craft-proof's `DEFAULT_EMBED`; TODO(CP4.4): English only
/// then).
/// About › Language data shows SCOWL's notices (the Ispell BSD licence among them) whenever
/// English spells from the embedded SCOWL data.
#[test]
fn about_shows_the_scowl_notice_when_scowl_is_embedded() {
let texts = super::licence_texts();
match wordcraft_proof::scowl::notice() {
Some(notice) => {
let entry = texts.iter().find(|(t, _)| t == super::SCOWL_TITLE).map(|(_, x)| x.as_str());
assert_eq!(entry, Some(notice));
assert!(notice.contains("Ispell") || notice.contains("ispell"), "the Ispell notice is part of it");
}
None => assert!(texts.iter().all(|(t, _)| t != super::SCOWL_TITLE), "no SCOWL data: no SCOWL entry"),
}
}
#[test]
fn the_default_build_embeds_only_english() {
const CARRY_OVER: [&str; 5] = ["de", "pl", "pt", "it", "ru"];
let named: Vec<String> = std::env::var("CRAFT_PROOF_EMBED")
.unwrap_or_default()
.split(',')
.map(|l| l.trim().to_ascii_lowercase())
.filter(|l| !l.is_empty())
.collect();
let allowed = |base: &str| base == "en" || if named.is_empty() { CARRY_OVER.contains(&base) } else { named.iter().any(|n| n == base) };
let embedded = wordcraft_proof::packs::embedded_locales();
let extra: Vec<&&str> = embedded.iter().filter(|l| !allowed(l.split('-').next().unwrap_or(l))).collect();
assert!(extra.is_empty(), "embedded beyond English and {named:?} (or the carry-over by default): {extra:?}");
}
}
+104
View File
@@ -0,0 +1,104 @@
//! The font store as the packs [`Backend`](super::Backend): its folder, its consent setting, the
//! pinned pack index and verified installs (craft-libs `craft-font-store`, kind `language-pack`).
use std::path::PathBuf;
use std::sync::{Mutex, PoisonError};
use craft_font_catalog::FontEnv;
use craft_font_store::{FontStore, Network, PackEntry, PackIndex, PackIndexPin};
use super::{Cancel, Pack};
/// The pack index this build trusts: craft-i18n's first Language pack release, language-packs-1
/// (61 packs, built from craft-i18n `4504e288`), pinned by the sha256 of its `index.json`.
/// `CRAFT_LANGUAGE_PACK_INDEX` (read by the font store itself) points WordCraft at a local or test
/// index instead, for trying packs before they are released.
pub const PACK_INDEX: PackIndexPin = PackIndexPin {
url: "https://github.com/storytold/craft-i18n/releases/download/language-packs-1/index.json",
sha256: "531465b50126d0594bb89a9c4512f7ae67d581259636040a410190641ec1428c",
};
pub(super) struct FontStoreBackend {
store: FontStore,
/// The index last read or fetched; installs take their entry (url, size, sha256) from it.
index: Mutex<Option<PackIndex>>,
}
impl FontStoreBackend {
pub(super) fn new(user: bool) -> Option<Self> {
let env = FontEnv::from_process();
let store = if user { FontStore::for_user(&env)? } else { FontStore::for_agent(&env)? };
Some(Self { store, index: Mutex::new(None) })
}
/// A store in `dir`.
pub(super) fn in_dir(dir: PathBuf, user: bool) -> Self {
let caller = if user { craft_font_store::Caller::User } else { craft_font_store::Caller::Agent };
Self { store: FontStore::new(dir, caller), index: Mutex::new(None) }
}
fn keep(&self, index: PackIndex) -> Vec<Pack> {
let packs = index.packs.iter().map(pack).collect();
*self.index.lock().unwrap_or_else(PoisonError::into_inner) = Some(index);
packs
}
}
fn pack(e: &PackEntry) -> Pack {
Pack {
locale: e.locale.clone(),
name: e.name.clone(),
version: e.version.clone(),
url: e.url.clone(),
// TODO(CP4): the index's optional `source` once craft-font-store's PackEntry carries it.
source: None,
size: e.size,
licence: e.licence.clone(),
class: e.class.clone(),
spelling: e.spelling,
hyphenation: e.hyphenation,
}
}
impl super::Backend for FontStoreBackend {
fn index(&self) -> Result<Vec<Pack>, String> {
Ok(self.store.cached_language_pack_index(&PACK_INDEX).map(|i| self.keep(i)).unwrap_or_default())
}
fn fetch_index(&self, cancel: &Cancel) -> Result<Vec<Pack>, String> {
let index = self.store.fetch_language_pack_index(&PACK_INDEX, &cancel.0).map_err(|e| e.to_string())?;
Ok(self.keep(index))
}
fn network_on(&self) -> bool {
self.store.network() == Network::On
}
fn enable_network(&self) -> Result<(), String> {
self.store.enable_network().map_err(|e| e.to_string())
}
fn install(&self, pack: &Pack, cancel: &Cancel, progress: &mut dyn FnMut(u64, u64)) -> Result<(), String> {
let index = {
let held = self.index.lock().unwrap_or_else(PoisonError::into_inner).clone();
match held {
Some(i) => i,
None => self.store.cached_language_pack_index(&PACK_INDEX).ok_or("the list of Language packs hasn't been fetched")?,
}
};
let entry = index.pack(&pack.locale).ok_or_else(|| format!("there is no language pack for {}", pack.locale))?.clone();
self.store.install_language_pack(&index, &entry, &cancel.0, &mut |p| progress(p.done, p.total)).map(|_| ()).map_err(|e| e.to_string())
}
fn remove(&self, locale: &str) -> Result<(), String> {
self.store.remove_language_pack(locale).map_err(|e| e.to_string())
}
fn index_override(&self) -> Option<String> {
craft_font_store::language_pack_index_overridden()
}
fn packs_dir(&self) -> Option<PathBuf> {
Some(self.store.language_packs_dir())
}
}
+6 -5
View File
@@ -27,7 +27,10 @@ impl Session {
let deleted = (hide_deleted, self.prefs.markup.hidden_deletions(), self.revisions_in_balloons());
let outline = (mode == ViewMode::Outline).then(|| self.view.outline.view());
let shown = (self.view.marks || self.prefs.marks.hidden, self.view.field_codes);
let (rev, proofing, numeral) = (self.rev, self.view.proofing, self.view.numeral);
// A dictionary or Language pack installed, removed or edited (in this document's window or
// any other) proofs it again, open or parked.
let rev = (self.rev, wordcraft_proof::dictionary_generation());
let (proofing, numeral) = (self.view.proofing, self.view.numeral);
let fits = |e: &Option<LayoutEntry>| {
e.as_ref().is_some_and(|(r, w, h, m, sp, _, pf, o, hd, n, t, sh)| {
*r == rev
@@ -74,10 +77,8 @@ impl Session {
};
let l = Arc::new(wordcraft_layout::layout(&self.doc, &mut self.cache, &opts));
let [newest, _] = std::mem::take(&mut self.layout);
self.layout = [
Some((self.rev, width, height, mode, spacing, l.clone(), self.view.proofing, outline, deleted, self.view.numeral, typing, shown)),
newest,
];
self.layout =
[Some((rev, width, height, mode, spacing, l.clone(), self.view.proofing, outline, deleted, self.view.numeral, typing, shown)), newest];
l
}
+1 -1
View File
@@ -258,7 +258,7 @@ impl Session {
}
self.rev = rev;
for slot in &mut self.layout {
if slot.as_ref().is_some_and(|l| l.0 != rev) {
if slot.as_ref().is_some_and(|l| l.0.0 != rev) {
*slot = None;
}
}
+2 -2
View File
@@ -56,12 +56,12 @@ impl EditSnapshot {
/// block.
pub(super) type RunSnapshot = (Document, Selection, Vec<Arc<Undo>>, Vec<Arc<Undo>>, bool, u64, Option<ColumnBlock>);
/// A layout made and what it was made for: (revision, width, read height, view, read line
/// A layout made and what it was made for: ((revision, proofing dictionaries' generation), width, read height, view, read line
/// spacing, layout, proofing, outline, (deletions left out, deletions marked Hidden, body
/// deletions in balloons), numeral display, the word being typed, (hidden text shown, field
/// codes shown)).
pub(super) type LayoutEntry = (
u64,
(u64, u64),
f32,
f32,
ViewMode,
+164 -2
View File
@@ -1,6 +1,6 @@
// Reuse our hand-written Czech proof fixture; its install helper resolves this parent import.
// A copy of craft-proof's hand-written Czech fixture (`crates/proof/src/cs/test_dict.rs` in
// craft-libs, which a dependency's tests can't share); its install helper resolves this parent import.
use wordcraft_proof::cs::install_dictionary;
#[path = "../../proof/src/cs/test_dict.rs"]
mod czech_test_dict;
use serde_json::json;
@@ -18,6 +18,22 @@ mod revision_checks;
mod typing;
mod wordart;
/// True, after saying so on stderr, when this build lacks the Language data the proofing tests
/// need (craft-proof built without `CRAFT_I18N_DIR`). The probes are craft-proof's public
/// `english_source` (the embedded English dictionary) and its American/British variant table
/// (`colour` → `color`): a craft-i18n checkout without some of the files embeds only the rest,
/// so both are asked. Without them English still spells from craft-proof's built-in word list,
/// but there are no hyphenation patterns and no variant table, so the tests that check those
/// return early, as craft-proof's own do; with the data (CI, releases) they all run.
pub(crate) fn no_language_data(test: &str) -> bool {
let missing = wordcraft_proof::english_source(wordcraft_proof::ProofLang::En) != Some(wordcraft_proof::EnglishSource::Embedded)
|| wordcraft_proof::en_variants::preferred("colour", wordcraft_proof::ProofLang::En).as_deref() != Some("color");
if missing {
eprintln!("{test}: skipped, needs the Language data (set CRAFT_I18N_DIR to a craft-i18n checkout)");
}
missing
}
fn s() -> Session {
Session::new(wordcraft_doc::Document::new())
}
@@ -8665,6 +8681,9 @@ fn pairs(v: &[(&str, &str)]) -> Vec<(String, String)> {
/// it and Change fixes it.
#[test]
fn english_dialects_through_the_review_commands() {
if no_language_data("english_dialects_through_the_review_commands") {
return;
}
let mut s = s();
run(&mut s, "document.setText", json!({"text": "The colour of the center."}));
// US English by default: the British spelling is flagged, the American one isn't.
@@ -8698,6 +8717,9 @@ fn english_dialects_through_the_review_commands() {
/// and is proofed the same way when it comes back.
#[test]
fn english_dialects_survive_a_docx_round_trip() {
if no_language_data("english_dialects_survive_a_docx_round_trip") {
return;
}
use std::io::Read;
let mut s = s();
run(&mut s, "document.setText", json!({"text": "Colour and color. Centre and center."}));
@@ -8726,6 +8748,9 @@ fn english_dialects_survive_a_docx_round_trip() {
/// system, #1045); another region's language leaves US English.
#[test]
fn new_documents_follow_the_regions_english() {
if no_language_data("new_documents_follow_the_regions_english") {
return;
}
let mut s = s();
s.system_region = wordcraft_geom::Region::from_tag("en_GB.UTF-8");
assert_eq!(s.region_english().as_deref(), Some("en-GB"));
@@ -11599,3 +11624,140 @@ fn export_png_rejects_a_page_number_that_does_not_fit() {
assert!(!target.exists(), "no file is written for an invalid page");
let _ = std::fs::remove_dir_all(&dir);
}
/// A Word package whose body holds `body` and, as `rIdC`, the chunk part `word/afchunk.{ext}`.
#[cfg(not(target_arch = "wasm32"))]
fn docx_with_chunk(body: &str, ext: &str, content_type: &str, chunk: &[u8]) -> Vec<u8> {
use std::io::Write;
let files: [(String, Vec<u8>); 4] = [
(
"[Content_Types].xml".into(),
format!(r#"<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="{ext}" ContentType="{content_type}"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>"#).into_bytes(),
),
("_rels/.rels".into(), br#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/></Relationships>"#.to_vec()),
(
"word/_rels/document.xml.rels".into(),
format!(r#"<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rIdC" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/aFChunk" Target="afchunk.{ext}"/></Relationships>"#).into_bytes(),
),
(
"word/document.xml".into(),
format!(r#"<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"><w:body>{body}</w:body></w:document>"#).into_bytes(),
),
];
let mut zw = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
for (name, bytes) in files.iter().map(|(n, b)| (n.as_str(), b.as_slice())).chain([(format!("word/afchunk.{ext}").as_str(), chunk)]) {
zw.start_file(name, zip::write::SimpleFileOptions::default()).unwrap();
zw.write_all(bytes).unwrap();
}
zw.finish().unwrap().into_inner()
}
/// A Single File Web Page under a Word or RTF name opens as one, like the web pages of #2853.
#[test]
fn a_single_file_web_page_saved_as_doc_opens_by_content() {
let mht = "MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=\"B\"\r\n\r\n--B\r\nContent-Type: text/html; charset=\"utf-8\"\r\n\r\n<p>Izvje\u{161}taj</p>\r\n--B--\r\n";
for name in ["report.doc", "report.docx", "report.rtf"] {
let doc = crate::io::open_bytes(name, mht.as_bytes()).unwrap_or_else(|e| panic!("{name}: {e}"));
assert_eq!(doc.plain_text(StoryRef::Body), "Izvje\u{161}taj", "{name}");
}
}
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn alt_chunks_open_in_place_as_word_brings_them_in() {
let png = wordcraft_formats::model::base64_encode(&tiny_png());
let mht = format!(
"MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=\"B\"\r\n\r\n--B\r\nContent-Location: file:///C:/doc.htm\r\n\
Content-Transfer-Encoding: quoted-printable\r\nContent-Type: text/html; charset=\"utf-8\"\r\n\r\n\
<style>p.c {{ text-align: center }}</style><p class=3D\"c\">=C4=8Clanak 1.</p><p><img src=3D\"doc_files/a.png\"></p>\r\n\
--B\r\nContent-Location: file:///C:/doc_files/a.png\r\nContent-Transfer-Encoding: base64\r\nContent-Type: image/png\r\n\r\n{png}\r\n--B--\r\n"
);
let p = |t: &str| format!(r#"<w:p><w:r><w:t>{t}</w:t></w:r></w:p>"#);
let chunk = r#"<w:altChunk r:id="rIdC"/>"#;
let bytes = docx_with_chunk(&format!("{}{chunk}{}", p("Before"), p("After")), "mht", "message/rfc822", mht.as_bytes());
let doc = crate::io::open_bytes("chunk.docx", &bytes).unwrap();
let text = doc.plain_text(StoryRef::Body);
assert!(text.starts_with("Before\nČlanak 1.\n") && text.ends_with("After"), "{text:?}");
let para = doc.para(StoryRef::Body, &wordcraft_doc::Path::top(1)).unwrap();
assert_eq!(para.props.align, Some(wordcraft_doc::Align::Center));
// Web page text: Times New Roman unless the page says otherwise.
assert_eq!(para.props_of_char(0).font.as_deref(), Some("Times New Roman"));
// The picture came along, its bytes under the key it shows.
let key = doc
.body
.iter()
.find_map(|b| match &**b {
wordcraft_doc::Block::Para(p) => p.objects.iter().find_map(|o| match o {
wordcraft_doc::InlineObject::Image { media, .. } => Some(media.clone()),
_ => None,
}),
_ => None,
})
.expect("the chunk's picture");
assert_eq!(doc.media.get(&key).map(|m| m.to_vec()), Some(tiny_png()));
// HTML (in its content type's charset), RTF and plain-text chunks come in through their
// importers too, and an MHT chunk is known by what it holds.
for (ext, ct, data, want) in [
("htm", "text/html; charset=windows-1250", &b"<p>\xc8lanak chunk</p>"[..], "Članak chunk"),
("rtf", "application/rtf", br"{\rtf1\ansi Rich chunk\par}", "Rich chunk"),
("txt", "text/plain", b"Plain chunk", "Plain chunk"),
("bin", "", mht.as_bytes(), "Članak 1."),
] {
let bytes = docx_with_chunk(&format!("{chunk}{}", p("End")), ext, ct, data);
let text = crate::io::open_bytes("chunk.docx", &bytes).unwrap().plain_text(StoryRef::Body);
assert!(text.starts_with(want) && text.ends_with("End"), "{ext}: {text:?}");
}
}
#[cfg(not(target_arch = "wasm32"))]
#[test]
fn text_from_file_brings_the_files_pictures() {
let dir = scratch_dir("text-from-file-pictures");
let mut src = Session::new(wordcraft_doc::Document::new());
let list = src.doc.numbering.add_list(wordcraft_doc::numbering::ListKind::Bullet);
// A list item, then a paragraph (a single paragraph would take the caret's formatting).
src.doc.split_paragraph(&Pos::body(0, 0)).unwrap();
src.doc.para_mut(StoryRef::Body, &wordcraft_doc::Path::top(0)).unwrap().props.numbering =
Some(wordcraft_doc::props::NumRef { num: list, level: 0 });
let key = src.doc.add_media(tiny_png(), "png");
let image =
wordcraft_doc::InlineObject::Image { media: key, w: 20.0, h: 20.0, alt: String::new(), float: Default::default(), crop: [0.0; 4], ole: None };
src.doc.para_mut(StoryRef::Body, &wordcraft_doc::Path::top(0)).unwrap().insert_object(0, image, &Default::default()).unwrap();
let path = dir.join("pictures.docx");
std::fs::write(&path, wordcraft_docx::write(&src.doc).unwrap()).unwrap();
let mut s = s();
// A picture and a list of our own first, so the file's key and list id would point at ours.
let ours = s.doc.add_media(vec![9, 9, 9], "png");
let our_list = s.doc.numbering.add_list(wordcraft_doc::numbering::ListKind::Numbered);
run(&mut s, "insert.textFromFile", json!({"path": path.to_string_lossy()}));
let _ = std::fs::remove_dir_all(&dir);
let shown: Vec<String> = s
.doc
.body
.iter()
.filter_map(|b| match &**b {
wordcraft_doc::Block::Para(p) => p.objects.iter().find_map(|o| match o {
wordcraft_doc::InlineObject::Image { media, .. } => Some(media.clone()),
_ => None,
}),
_ => None,
})
.collect();
assert_eq!(shown.len(), 1);
assert_ne!(shown[0], ours);
assert_eq!(s.doc.media.get(&shown[0]).map(|m| m.to_vec()), Some(tiny_png()));
let listed: Vec<u32> = s
.doc
.body
.iter()
.filter_map(|b| match &**b {
wordcraft_doc::Block::Para(p) => p.props.numbering.map(|n| n.num),
_ => None,
})
.collect();
assert_eq!(listed.len(), 1);
assert_ne!(listed[0], our_list);
assert!(s.doc.numbering.level(listed[0], 0).is_some_and(|l| l.format == wordcraft_doc::section::NumFormat::Bullet));
}
+79
View File
@@ -3631,3 +3631,82 @@ fn backspace_cannot_join_into_locked_content_control() {
assert!(s.run("text.backspace", &json!({})).is_err());
assert_eq!(s.doc, before);
}
// English spelling from SCOWL 2026 (CP2): each English reads its own SCOWL dictionary, with
// `en-variants.txt` deciding the other dialect's spellings. Typed text, as a person types it.
/// The spelling words typed text gets in `lang` (the whole document's language), with the first
/// suggestion of each.
fn typed_spelling_in(lang: &str, keys: &str) -> Vec<(String, String)> {
let mut s = s();
run(&mut s, "review.language", json!({"lang": lang, "default": true}));
typ(&mut s, keys);
let issues = s.run("review.issues", &json!({})).unwrap();
issues
.as_array()
.unwrap()
.iter()
.filter(|i| i["kind"] == "spelling")
.map(|i| (i["text"].as_str().unwrap().to_string(), i["suggestions"][0].as_str().unwrap_or("").to_string()))
.collect()
}
/// #40: the British spellings the issue lists are right in en-GB, and the American ones are
/// flagged with the British form first.
#[test]
fn issue_40_british_spellings_are_right_in_en_gb() {
if super::no_language_data("issue_40_british_spellings_are_right_in_en_gb") {
return;
}
let none: Vec<(String, String)> = Vec::new();
let text = "The judgement on the defence licence put the programme in the catalogue. ";
assert_eq!(typed_spelling_in("en-GB", text), none);
assert_eq!(typed_spelling_in("en-GB", "Its colour and centre, organised and organized, travelled. "), none);
let flagged = typed_spelling_in("en-GB", "The color of the center. ");
assert_eq!(flagged, [("color".to_string(), "colour".to_string()), ("center".to_string(), "centre".to_string())]);
}
/// #520: common modern words are right (SCOWL plus our supplement), and real misspellings of
/// them are still caught.
#[test]
fn issue_520_common_words_are_right() {
if super::no_language_data("issue_520_common_words_are_right") {
return;
}
let none: Vec<(String, String)> = Vec::new();
assert_eq!(typed_spelling_in("en-US", "My desktop email has downloads, a smartphone and a website. "), none);
// (`recieved` would be AutoCorrected as it is typed.)
let flagged = typed_spelling_in("en-US", "My desktpo emial. ");
assert_eq!(flagged.iter().map(|f| f.0.as_str()).collect::<Vec<_>>(), ["desktpo", "emial"]);
assert_eq!(flagged[0].1, "desktop");
}
/// #1281 and #1045: Canadian and Australian English (and the other British-spelling Englishes)
/// proof by their own rules; ordinals are words in every English.
#[test]
fn issues_1281_1045_english_dialects_proof_typed_text() {
if super::no_language_data("issues_1281_1045_english_dialects_proof_typed_text") {
return;
}
let none: Vec<(String, String)> = Vec::new();
// Canada: British -our and -re, American -ize.
assert_eq!(typed_spelling_in("en-CA", "We organize the colour in the centre. "), none);
assert_eq!(
typed_spelling_in("en-CA", "We organise the color. "),
[("organise".to_string(), "organize".to_string()), ("color".to_string(), "colour".to_string())]
);
// Australia, New Zealand, India: British spelling.
for lang in ["en-AU", "en-NZ", "en-IN"] {
assert_eq!(typed_spelling_in(lang, "We analyse the colour of the theatre. "), none, "{lang}");
assert_eq!(typed_spelling_in(lang, "We analyze it. "), [("analyze".to_string(), "analyse".to_string())], "{lang}");
}
// US English flags British spellings with the American form first.
assert_eq!(typed_spelling_in("en-US", "The colour. "), [("colour".to_string(), "color".to_string())]);
// SCOWL's ordinals (COMPOUNDRULE) are words, even where numbers are checked.
let mut s = s();
run(&mut s, "file.options", json!({"ignoreNumbers": false}));
typ(&mut s, "The 21st and 112th times, not the 21th. ");
let issues = s.run("review.issues", &json!({})).unwrap();
let words: Vec<&str> = issues.as_array().unwrap().iter().filter(|i| i["kind"] == "spelling").filter_map(|i| i["text"].as_str()).collect();
assert_eq!(words, ["21th"]);
}
+1 -1
View File
@@ -35,7 +35,7 @@ pub(crate) use names::face_names;
pub use names::{LocalName, base_style, name_language_matches, system_font_dirs};
use names::{MAX_FILE_FACES, enumerate_faces, make_face};
#[cfg(test)]
use names::{file_face_names, font_faces, mini_font, name};
use names::{MAX_NAME_CHARS, file_face_names, font_faces, mini_font, name};
#[cfg(test)]
pub(crate) use names::{last_resort_face, legacy_family};
+10 -3
View File
@@ -61,8 +61,16 @@ pub(super) struct CatalogEntry {
/// The longest name a document's font table gives an embedded font that faces take on.
pub(super) const MAX_FAMILY_NAME: usize = 128;
/// The longest name read from a font's `name` table, in characters. A record can hold 64K
/// characters, and each face copies its family into its localized names and into every named
/// instance, so a small font could claim hundreds of megabytes of names. Real names run to a
/// few dozen characters.
pub(super) const MAX_NAME_CHARS: usize = 128;
pub(super) fn name(font: &skrifa::FontRef<'_>, ids: &[StringId]) -> Option<String> {
ids.iter().find_map(|id| font.localized_strings(*id).english_or_first().map(|s| s.to_string()).filter(|s| !s.is_empty()))
ids.iter().find_map(|id| {
font.localized_strings(*id).english_or_first().map(|s| s.chars().take(MAX_NAME_CHARS).collect::<String>()).filter(|s| !s.is_empty())
})
}
/// The family's names in other languages: localized typographic family names (name ID 16) stand
@@ -70,7 +78,6 @@ pub(super) fn name(font: &skrifa::FontRef<'_>, ids: &[StringId]) -> Option<Strin
/// font has no typographic names). Bounded: a damaged font can list any number of records.
fn local_names(f: &skrifa::FontRef<'_>, family: &str, id1: Option<&str>) -> Vec<LocalName> {
const MAX_NAMES: usize = 64;
const MAX_CHARS: usize = 128;
let typographic = f.localized_strings(StringId::TYPOGRAPHIC_FAMILY_NAME).next().is_some();
let legacy = if typographic { id1 } else { Some(family) };
let mut out: Vec<LocalName> = Vec::new();
@@ -81,7 +88,7 @@ fn local_names(f: &skrifa::FontRef<'_>, family: &str, id1: Option<&str>) -> Vec<
if lang == "en" || lang.starts_with("en-") {
continue;
}
let name: String = s.chars().take(MAX_CHARS).collect::<String>().trim().to_string();
let name: String = s.chars().take(MAX_NAME_CHARS).collect::<String>().trim().to_string();
if name.is_empty() || name.eq_ignore_ascii_case(english) || out.iter().any(|n| n.lang == lang && n.name == name) {
continue;
}
+34 -4
View File
@@ -245,15 +245,24 @@ fn postscript_names_select_faces() {
/// A variable font holding just `name` and `fvar` tables: family "Var Test", one `wght` axis
/// and `instances` named instances, each with its own style name.
fn variable_font(instances: u16) -> Vec<u8> {
let mut names: Vec<(u16, String)> = vec![(1, "Var Test".into()), (2, "Regular".into())];
names.extend((0..instances).map(|i| (256 + i, format!("Style {i}"))));
variable_font_named("Var Test", &[], instances)
}
/// [`variable_font`] with family `family`, and family names in other languages `local`
/// (Windows language ids).
fn variable_font_named(family: &str, local: &[(u16, &str)], instances: u16) -> Vec<u8> {
let mut names: Vec<(u16, u16, String)> = vec![(0x409, 1, family.into()), (0x409, 2, "Regular".into())];
names.extend(local.iter().map(|(lang, n)| (*lang, 1, n.to_string())));
names.extend((0..instances).map(|i| (0x409, 256 + i, format!("Style {i}"))));
// Records sorted by language, then name id.
names.sort_by_key(|(lang, id, _)| (*lang, *id));
let mut name = vec![0, 0];
name.extend_from_slice(&(names.len() as u16).to_be_bytes());
name.extend_from_slice(&(6 + 12 * names.len() as u16).to_be_bytes());
let mut strings = Vec::new();
for (id, text) in &names {
for (lang, id, text) in &names {
let utf16: Vec<u8> = text.encode_utf16().flat_map(u16::to_be_bytes).collect();
for v in [3, 1, 0x409, *id, utf16.len() as u16, strings.len() as u16] {
for v in [3, 1, *lang, *id, utf16.len() as u16, strings.len() as u16] {
name.extend_from_slice(&v.to_be_bytes());
}
strings.extend(utf16);
@@ -357,6 +366,27 @@ fn the_draw_that_turns_hinting_off_is_redrawn_unhinted_for_the_outline_cache() {
assert_eq!(db.outline(&face, gid).elements(), tripping.path.elements());
}
#[test]
fn long_names_are_cut_before_faces_copy_them() {
// A name record can hold 64K characters, and every named instance copies the family and its
// localized names: the names are cut when read, so the copies stay small.
let long = "W".repeat(14_000);
let local = "Ж".repeat(14_000);
let font = variable_font_named(&long, &[(0x419, &local)], 32);
let faces = font_faces(&skrifa::FontRef::new(&font).unwrap(), 0);
assert_eq!(faces.len(), 32);
for f in &faces {
assert_eq!(f.family.chars().count(), MAX_NAME_CHARS);
assert!(f.legacy.chars().count() <= MAX_NAME_CHARS + " Style 31".len(), "{}", f.legacy.len());
assert_eq!(f.local.len(), 1);
assert_eq!(f.local[0].name.chars().count(), MAX_NAME_CHARS);
assert_eq!(f.local[0].english, f.family);
}
// Short names are kept whole.
let faces = font_faces(&skrifa::FontRef::new(&variable_font_named("Var Test", &[(0x419, "Вар Тест")], 2)).unwrap(), 0);
assert_eq!((faces[0].family.as_str(), faces[0].local[0].name.as_str()), ("Var Test", "Вар Тест"));
}
#[test]
fn named_instances_and_collection_members_are_capped() {
// #3716: a file's faces are capped before they are built, however the file multiplies them.
+10 -4
View File
@@ -8,18 +8,22 @@
//! (`rb`/`rt`/`rp`: furigana and other phonetic guides, #1681), and the CSS
//! properties `color`, `background-color`, `font-weight`, `font-style`, `font-size`,
//! `font-family`, `text-decoration`, `vertical-align`, `text-align`, `white-space` and page
//! breaks. Scripts and styles are skipped; nesting is capped.
//! breaks, and on paragraphs `margin`, `text-indent` and `line-height`. CSS comes from `style`
//! attributes and from `style` elements' rules with simple selectors (`p`, `.note`, `p.note`),
//! applied in cascade order (specificity, then source order, then the `style` attribute).
//! Scripts are skipped; nesting is capped.
//!
//! Export writes clean semantic HTML with inline CSS, pictures as `data:` URIs and ruby as
//! `<ruby>`.
use std::sync::Arc;
use wordcraft_doc::props::LineSpacing;
use wordcraft_doc::props::VAlign;
use wordcraft_doc::{Align, Document};
use crate::model::{
self, Cell, FBlock, FTable, Flow, Fmt, Inline, Kind, ListInfo, Meta, Para, base64_encode, data_uri, make_img, mime_of, parse_color,
self, Cell, FBlock, FTable, Flow, Fmt, Inline, Kind, ListInfo, Meta, Para, Spacing, base64_encode, data_uri, make_img, mime_of, parse_color,
};
mod builder;
@@ -32,9 +36,11 @@ mod math_ruby;
pub use builder::ImageLoader;
pub use css::css_len_pt;
pub use entity::{decode, decode_entity, unescape};
pub use entity::{decode, decode_as, decode_entity, unescape};
pub use export::{esc, export, export_flow, export_fragment, inlines_html};
pub use import::{import, import_on, import_with, parse, parse_with};
#[cfg(test)]
use import::parse_in;
pub use import::{PAGE_FONT, import, import_on, import_page, import_with, parse, parse_with};
#[cfg(test)]
mod tests;
+423 -37
View File
@@ -1,6 +1,9 @@
use super::css::{apply_style, css_len_pt};
use super::import::collapse_ws;
use super::{Align, Arc, Cell, FBlock, FTable, Flow, Fmt, Inline, Kind, ListInfo, Meta, Para, VAlign, data_uri, make_img, model, parse_color};
use super::css::{apply_style, css_len_pt, strip_important};
use super::import::{PAGE_FONT, PAGE_HEADING_SIZES, collapse_ws};
use super::{
Align, Arc, Cell, FBlock, FTable, Flow, Fmt, Inline, Kind, LineSpacing, ListInfo, Meta, Para, Spacing, VAlign, data_uri, make_img, model,
parse_color,
};
/// Open elements kept at most (deeper start tags are ignored).
const MAX_STACK: usize = 256;
@@ -28,6 +31,12 @@ struct El {
name: String,
kind: ElKind,
fmt: Fmt,
/// Margins, indents and line height its CSS states (blocks only; they don't inherit; on a
/// page's spans, their margins).
spacing: Spacing,
/// A page's `p` or heading: margins its CSS leaves out (or writes invalidly) are HTML's
/// automatic spacing.
auto_margins: bool,
align: Option<Align>,
preserve: bool,
page_break: bool,
@@ -77,20 +86,37 @@ fn apply_font_attributes(attrs: &[(String, String)], fmt: &mut Fmt) {
}
}
/// Whether an element's `border` attribute or `style` declares a visible border.
fn declares_border(attrs: &[(String, String)], is_table: bool) -> bool {
/// Whether an element's `border` attribute or its CSS declares a visible border.
fn declares_border(attrs: &[(String, String)], css: &str, is_table: bool) -> bool {
if is_table && attr(attrs, "border").is_some_and(|b| b.trim().parse::<u32>().map(|n| n > 0).unwrap_or(b.trim().is_empty())) {
return true;
}
let Some(style) = attr(attrs, "style") else { return false };
style.split(';').any(|decl| {
let Some((name, value)) = decl.split_once(':') else { return false };
let name = name.trim().to_ascii_lowercase();
if !matches!(name.as_str(), "border" | "border-top" | "border-right" | "border-bottom" | "border-left") {
return false;
// Each side's last declaration decides (the `style` attribute's come after the sheet's).
let mut sides = [false; 4];
for (name, value) in decls(css) {
let set = match name.as_str() {
"border" => 0..4,
"border-top" => 0..1,
"border-right" => 1..2,
"border-bottom" => 2..3,
"border-left" => 3..4,
_ => continue,
};
let value = value.to_ascii_lowercase();
let on = !value.is_empty() && !value.split_whitespace().any(|tok| matches!(tok, "none" | "hidden" | "0" | "0px" | "0pt"));
if let Some(s) = sides.get_mut(set) {
s.fill(on);
}
let value = value.trim().to_ascii_lowercase();
!value.is_empty() && !value.split_whitespace().any(|tok| matches!(tok, "none" | "hidden" | "0" | "0px" | "0pt"))
}
sides.contains(&true)
}
/// A declaration block's declarations, at most [`MAX_DECLS`]: (property in lower case, value
/// without `!important`).
pub(super) fn decls(css: &str) -> impl Iterator<Item = (String, &str)> {
css.split(';').take(MAX_DECLS).filter_map(|d| {
let (k, v) = d.split_once(':')?;
Some((k.trim().to_ascii_lowercase(), strip_important(v)))
})
}
@@ -116,6 +142,12 @@ pub(super) struct Builder<'r> {
pre_tail: bool,
meta: Meta,
title: Option<String>,
/// Rules of the page's `style` elements.
pub(super) sheet: Sheet,
/// Read as Word opens a web page ([`import_page`]), not as a pasted fragment.
page: bool,
/// What is left of [`MAX_CSS_WORK`].
css_work: usize,
}
fn is_void(n: &str) -> bool {
@@ -196,12 +228,310 @@ fn closes_p(n: &str) -> bool {
)
}
/// The paragraph spacing a block's CSS states: `margin` (and its sides) as space before / after
/// and left / right indents, `text-indent` as the first-line indent, `line-height` as line
/// spacing (a number or percentage: a multiple; a length: exactly). `auto` and values that don't
/// parse state nothing.
fn block_spacing(css: &str) -> Spacing {
let mut sp = Spacing::default();
// A length with a unit, or 0: a bare number, a percentage or a keyword states nothing.
let len = |v: &str| match v.trim() {
"0" => Some(0.0),
v if v.ends_with('%') || v.parse::<f32>().is_ok() || !v.starts_with(|c: char| c.is_ascii_digit() || matches!(c, '.' | '-' | '+')) => None,
v => css_len_pt(v).filter(|x| x.abs() <= 1584.0),
};
for (k, v) in decls(css) {
match k.as_str() {
"margin" => {
let parts: Vec<Option<f32>> = v.split_whitespace().map(len).collect();
let [top, right, bottom, left] = match parts.as_slice() {
[a] => [*a; 4],
[a, b] => [*a, *b, *a, *b],
[a, b, c] => [*a, *b, *c, *b],
[a, b, c, d] => [*a, *b, *c, *d],
_ => continue,
};
sp.before = top.map(|x| x.max(0.0)).or(sp.before);
sp.after = bottom.map(|x| x.max(0.0)).or(sp.after);
sp.left = left.or(sp.left);
sp.right = right.or(sp.right);
}
"margin-top" => sp.before = len(v).map(|x| x.max(0.0)).or(sp.before),
"margin-bottom" => sp.after = len(v).map(|x| x.max(0.0)).or(sp.after),
"margin-left" => sp.left = len(v).or(sp.left),
"margin-right" => sp.right = len(v).or(sp.right),
"text-indent" => sp.first = len(v).or(sp.first),
"line-height" => {
let lv = v.to_ascii_lowercase();
let multiple = |m: f32| (m.is_finite() && m > 0.0).then(|| LineSpacing::Multiple(m.clamp(0.25, 10.0)));
sp.line = if let Some(pc) = lv.strip_suffix('%') {
pc.trim().parse::<f32>().ok().and_then(|p| multiple(p / 100.0))
} else if let Ok(m) = lv.parse::<f32>() {
multiple(m)
} else if lv == "normal" {
Some(LineSpacing::Multiple(1.0))
} else {
len(&lv).filter(|x| *x > 0.0).map(LineSpacing::Exactly)
}
.or(sp.line);
}
_ => {}
}
}
sp
}
/// A rule of a `style` element with a simple selector: an element name, classes or both.
#[derive(Clone, Debug)]
struct Rule {
tag: Option<String>,
classes: Vec<String>,
/// Classes count ten, an element name one.
specificity: usize,
/// Shared by the selectors of one block.
decls: Arc<str>,
/// How many declarations `decls` holds.
count: usize,
}
/// Most rules kept from a page's style elements.
const MAX_RULES: usize = 10_000;
/// Most declarations read for one element (its rules' and its `style` attribute's).
const MAX_DECLS: usize = 512;
/// Most declarations read from one `style` attribute.
const INLINE_DECLS: usize = 64;
/// Rule matching (rules looked at plus bytes gathered) one page may spend.
const MAX_CSS_WORK: usize = 64 << 20;
/// Most bytes of declarations gathered from the rules one element matches.
const MAX_RULE_CSS: usize = 64 * 1024;
/// Most of an element's classes looked up.
const MAX_CLASSES: usize = 32;
/// A page's style rules, found by the element name or a class they need.
#[derive(Default)]
pub(super) struct Sheet {
rules: Vec<Rule>,
/// Rules naming classes, by their first class.
by_class: std::collections::HashMap<String, Vec<usize>>,
/// Rules naming only an element, by its name.
by_tag: std::collections::HashMap<String, Vec<usize>>,
/// `*` rules.
any: Vec<usize>,
}
impl Sheet {
fn push(&mut self, r: Rule) {
let i = self.rules.len();
match (r.classes.first(), &r.tag) {
(Some(c), _) => self.by_class.entry(c.clone()).or_default().push(i),
(None, Some(t)) => self.by_tag.entry(t.clone()).or_default().push(i),
(None, None) => self.any.push(i),
}
self.rules.push(r);
}
}
/// Read the rules of a style sheet into `out`. At-rules (`@media`, `@page`…) and rules whose
/// selectors aren't simple (descendants, pseudo-classes, attributes…) are skipped.
pub(super) fn parse_sheet(css: &str, out: &mut Sheet) {
// Comments go, as do the `<!-- -->` old pages hide their styles in.
let mut text = String::with_capacity(css.len());
let mut rest = css;
while let Some(i) = rest.find("/*") {
text.push_str(rest.get(..i).unwrap_or(""));
rest = rest.get(i + 2..).and_then(|r| r.find("*/").and_then(|j| r.get(j + 2..))).unwrap_or("");
}
text.push_str(rest);
let text = text.replace("<!--", " ").replace("-->", " ");
let mut rest = text.as_str();
while let Some(open) = rest.find('{') {
// A statement at-rule before it (`@charset "x";`, `@import url(a);`) isn't a selector.
let selectors = rest.get(..open).unwrap_or("").rsplit(';').next().unwrap_or("").trim();
let body = rest.get(open + 1..).unwrap_or("");
// The block's end, counting nested braces (an at-rule's own rules).
let mut depth = 1usize;
let mut end = body.len();
for (i, c) in body.char_indices() {
match c {
'{' => depth += 1,
'}' => {
depth -= 1;
if depth == 0 {
end = i;
break;
}
}
_ => {}
}
}
let decls = body.get(..end).unwrap_or("");
rest = body.get(end + 1..).unwrap_or("");
if selectors.starts_with('@') || decls.contains('{') || decls.len() > MAX_RULE_CSS {
continue;
}
let (count, decls) = (decls.split(';').count(), Arc::<str>::from(decls));
for sel in selectors.split(',') {
if out.rules.len() >= MAX_RULES {
return;
}
if let Some((tag, classes)) = simple_selector(sel.trim()) {
let specificity = classes.len() * 10 + usize::from(tag.is_some());
out.push(Rule { tag, classes, specificity, decls: decls.clone(), count });
}
}
}
}
/// `p`, `.note`, `p.note.wide` or `*`: (element name, classes). Class names may start with a
/// digit (`.000003`): browsers drop such a rule, but Word's HTML import applies it, and
/// converters write them.
fn simple_selector(sel: &str) -> Option<(Option<String>, Vec<String>)> {
if sel == "*" {
return Some((None, Vec::new()));
}
let mut parts = sel.split('.');
let tag = parts.next()?;
let tag = if tag.is_empty() {
None
} else if tag.chars().all(|c| c.is_ascii_alphanumeric()) && tag.starts_with(|c: char| c.is_ascii_alphabetic()) {
Some(tag.to_ascii_lowercase())
} else {
return None;
};
let ident = |c: &str| !c.is_empty() && c.chars().all(|ch| ch.is_alphanumeric() || ch == '_' || ch == '-');
let classes: Vec<String> = parts.map(str::to_string).collect();
if classes.iter().any(|c| !ident(c)) || (tag.is_none() && classes.is_empty()) {
return None;
}
Some((tag, classes))
}
impl Builder<'_> {
/// An element's CSS: the declarations of the style sheet rules it matches, in cascade order,
/// then its `style` attribute (later declarations win).
fn css_of(&mut self, name: &str, attrs: &[(String, String)]) -> String {
// The `style` attribute's first declarations always fit in what is read of the result.
let inline: Vec<&str> = attr(attrs, "style").unwrap_or("").split(';').take(INLINE_DECLS).collect();
let inline = inline.join(";");
let sheet = &self.sheet;
if sheet.rules.is_empty() || self.css_work == 0 {
return important_last(&inline);
}
let mut classes: Vec<&str> = attr(attrs, "class").map(|c| c.split_whitespace().take(MAX_CLASSES).collect()).unwrap_or_default();
classes.sort_unstable();
classes.dedup();
let mut candidates: Vec<usize> = sheet.any.clone();
candidates.extend(sheet.by_tag.get(name).into_iter().flatten());
for c in &classes {
candidates.extend(sheet.by_class.get(*c).into_iter().flatten());
}
let mut hits: Vec<(usize, usize)> = candidates
.iter()
.filter_map(|&i| sheet.rules.get(i).map(|r| (i, r)))
.filter(|(_, r)| r.tag.as_deref().is_none_or(|t| t == name) && r.classes.iter().all(|c| classes.contains(&c.as_str())))
.map(|(i, r)| (r.specificity, i))
.collect();
hits.sort_unstable();
let mut css = String::new();
let mut count = 0;
for (_, i) in hits {
let Some(r) = sheet.rules.get(i) else { continue };
if css.len() + r.decls.len() > MAX_RULE_CSS || count + r.count > MAX_DECLS - INLINE_DECLS {
break;
}
css.push_str(&r.decls);
css.push(';');
count += r.count;
}
// A page whose rules cost too much to apply reads on without them.
self.css_work = self.css_work.saturating_sub(candidates.len() + css.len());
css.push_str(&inline);
important_last(&css)
}
}
/// `css`'s declarations in order, the `!important` ones after the others: they win over normal
/// ones wherever they come from (CSS Cascade 5 §6.4), an inline one over the sheet's.
fn important_last(css: &str) -> String {
let (important, normal): (Vec<&str>, Vec<&str>) = css.split(';').filter(|d| !d.trim().is_empty()).partition(|d| strip_important(d) != d.trim());
normal.into_iter().chain(important).collect::<Vec<_>>().join(";")
}
/// The borders a cell's CSS gives it (`border`, `border-top`…: style, width and colour in any
/// order; `windowtext` is the automatic colour). `None` when it states none.
fn cell_borders(css: &str) -> Option<wordcraft_doc::props::Borders> {
use wordcraft_doc::props::{Border, BorderStyle, Borders};
let mut b = Borders::default();
let mut any = false;
for (k, v) in decls(css) {
let sides: &[u8] = match k.as_str() {
"border" => b"tlbr",
"border-top" => b"t",
"border-left" => b"l",
"border-bottom" => b"b",
"border-right" => b"r",
_ => continue,
};
// CSS's default width is `medium`.
let (mut style, mut width, mut color) = (None, 2.25_f32, None);
for tok in v.split_whitespace() {
let lt = tok.to_ascii_lowercase();
match lt.as_str() {
"none" | "hidden" => style = Some(BorderStyle::None),
"solid" => style = Some(BorderStyle::Single),
"double" => style = Some(BorderStyle::Double),
"dotted" => style = Some(BorderStyle::Dotted),
"dashed" => style = Some(BorderStyle::Dashed),
"thin" => width = 0.75,
"medium" => width = 2.25,
"thick" => width = 3.75,
"windowtext" => {}
_ => match css_len_pt(tok).filter(|_| tok.starts_with(|c: char| c.is_ascii_digit() || c == '.')) {
Some(w) => width = w.clamp(0.0, 12.0),
None => color = parse_color(tok).or(color),
},
}
}
// CSS draws no border without a style, or at no width.
let style = if width > 0.0 { style.unwrap_or(BorderStyle::None) } else { BorderStyle::None };
let border = Border { style, width, color, space: 0.0 };
for side in sides {
let slot = match side {
b't' => &mut b.top,
b'l' => &mut b.left,
b'b' => &mut b.bottom,
_ => &mut b.right,
};
*slot = Some(border);
}
any = true;
}
any.then_some(b)
}
/// Most columns given widths in one table.
const MAX_TABLE_COLS: usize = 63;
/// A cell's width, points: its CSS `width`, else its `width` attribute (pixels); percentages
/// and widths that don't parse give none.
fn cell_width(css: &str, attrs: &[(String, String)]) -> Option<f32> {
let from_css = decls(css).filter(|(k, v)| k == "width" && !v.ends_with('%')).filter_map(|(_, v)| css_len_pt(v));
let from_attr = || {
attr(attrs, "width")
.map(str::trim)
.filter(|v| !v.ends_with('%'))
.and_then(|v| v.trim_end_matches("px").trim().parse::<f32>().ok())
.map(|px| px * 0.75)
};
from_css.last().or_else(from_attr).filter(|w| w.is_finite() && *w > 1.0 && *w <= 1584.0)
}
fn attr<'a>(attrs: &'a [(String, String)], name: &str) -> Option<&'a str> {
attrs.iter().find(|(k, _)| k == name).map(|(_, v)| v.as_str())
}
impl<'r> Builder<'r> {
pub(super) fn new(images: ImageLoader<'r>) -> Builder<'r> {
pub(super) fn new(images: ImageLoader<'r>, page: bool) -> Builder<'r> {
Builder {
images,
containers: vec![Vec::new()],
@@ -215,11 +545,19 @@ impl<'r> Builder<'r> {
pre_tail: false,
meta: Meta::default(),
title: None,
sheet: Sheet::default(),
page,
css_work: MAX_CSS_WORK,
}
}
fn fmt(&self) -> Fmt {
self.stack.last().map(|e| e.fmt.clone()).unwrap_or_default()
match self.stack.last() {
Some(e) => e.fmt.clone(),
// A web page's text is Times New Roman 12 pt unless its CSS says otherwise.
None if self.page => Fmt { font: Some(PAGE_FONT.into()), size: Some(12.0), ..Fmt::default() },
None => Fmt::default(),
}
}
fn preserve(&self) -> bool {
@@ -237,6 +575,15 @@ impl<'r> Builder<'r> {
let mut list_level: Option<(bool, usize)> = None;
let mut li_at: Option<usize> = None;
let mut lists = 0usize;
let mut spaced = false;
// A page's first span stating margins (see below).
let span = self
.stack
.iter()
.rev()
.take_while(|e| e.kind == ElKind::Inline)
.find(|e| e.spacing.before.is_some() || e.spacing.after.is_some())
.map(|e| (e.spacing.before, e.spacing.after));
for (i, e) in self.stack.iter().enumerate().rev() {
match e.kind {
ElKind::Cell => {
@@ -270,6 +617,27 @@ impl<'r> Builder<'r> {
if p.align.is_none() && e.align.is_some() && e.kind != ElKind::Inline {
p.align = e.align;
}
// Margins belong to the paragraph's own block, not to those around it.
if !spaced && e.kind != ElKind::Inline {
p.spacing = e.spacing;
if self.page {
// Word opens a page's paragraphs with single lines; a margin a span
// states replaces the one its paragraph states, and one the paragraph
// leaves out is automatic spacing.
let (top, bottom) = span.unwrap_or_default();
let sp = &mut p.spacing;
sp.before_auto = sp.before.is_none() && e.auto_margins;
sp.after_auto = sp.after.is_none() && e.auto_margins;
sp.before = sp.before.map(|own| top.unwrap_or(own));
sp.after = sp.after.map(|own| bottom.unwrap_or(own));
sp.line = sp.line.or(Some(LineSpacing::Multiple(1.0)));
}
spaced = true;
}
}
// Page text without a block of its own (straight in a cell) is single-spaced too.
if self.page && !spaced {
p.spacing.line = Some(LineSpacing::Multiple(1.0));
}
if let (Some((o, _)), Some(li)) = (list_level, li_at) {
p.list = Some(ListInfo { ordered: o, level: lists.saturating_sub(1).min(8) as u8, start: 1 });
@@ -310,7 +678,11 @@ impl<'r> Builder<'r> {
}
}
if p.inlines.len() == 1 && p.text() == "\u{00A0}" {
// `<p>&nbsp;</p>`: an empty paragraph on purpose.
// `<p>&nbsp;</p>`: an empty paragraph on purpose, whose line is as high as the
// space's formatting makes it.
if let Some(Inline::Text(_, f)) = p.inlines.first() {
p.mark = Some(Box::new(f.clone()));
}
p.inlines.clear();
self.para_cont = false;
self.container().push(FBlock::Para(p));
@@ -482,6 +854,10 @@ impl<'r> Builder<'r> {
let mut ft = FTable { rows, widths: Vec::new(), borderless };
if !ft.rows.is_empty() {
ft.insert_covered();
// Columns keep the widths their cells give when every column has one.
if ft.cols() <= MAX_TABLE_COLS {
ft.widths_from_cells();
}
self.container().push(FBlock::Table(ft));
}
}
@@ -625,13 +1001,15 @@ impl<'r> Builder<'r> {
true
}
fn element_style(&self, name: &str, attrs: &[(String, String)], kind: ElKind) -> ElementStyle {
/// `css`: the element's declarations in cascade order ([`Builder::css_of`]).
fn element_style(&self, name: &str, attrs: &[(String, String)], kind: ElKind, css: &str) -> ElementStyle {
let mut fmt = self.fmt();
let mut align = None;
let mut preserve = self.preserve();
let mut page_break = false;
let mut page_break_after = false;
match name {
// A header cell is bold, as in browsers and Word, unless its own CSS says otherwise.
"b" | "strong" | "th" => fmt.bold = true,
"i" | "em" | "cite" | "var" | "dfn" | "address" => fmt.italic = true,
"u" | "ins" => fmt.underline = true,
@@ -662,9 +1040,16 @@ impl<'r> Builder<'r> {
if name == "center" {
align = Some(Align::Center);
}
if let Some(style) = attr(attrs, "style") {
apply_style(style, &mut fmt, &mut align, &mut preserve, &mut page_break, &mut page_break_after);
if self.page
&& let ElKind::Heading(n) = kind
{
// A page's headings are bold, black (unless they inherit a colour) and sized as
// browsers show them.
fmt.bold = true;
fmt.color.get_or_insert(wordcraft_doc::Rgb(0, 0, 0));
fmt.size = PAGE_HEADING_SIZES.get(usize::from(n.clamp(1, 6)) - 1).copied();
}
apply_style(css, &mut fmt, &mut align, &mut preserve, &mut page_break, &mut page_break_after);
if kind == ElKind::Pre {
preserve = true;
}
@@ -703,9 +1088,10 @@ impl<'r> Builder<'r> {
if kind != ElKind::Inline {
self.flush();
}
let ElementStyle { mut fmt, align, preserve, page_break, page_break_after } = self.element_style(name, attrs, kind);
let css = self.css_of(name, attrs);
let ElementStyle { mut fmt, align, preserve, page_break, page_break_after } = self.element_style(name, attrs, kind, &css);
match kind {
ElKind::Table => self.tables.push(TableB { rows: Vec::new(), row: None, bordered: declares_border(attrs, true) }),
ElKind::Table => self.tables.push(TableB { rows: Vec::new(), row: None, bordered: declares_border(attrs, &css, true) }),
ElKind::Tr => {
if let Some(t) = self.tables.last_mut()
&& let Some(r) = t.row.take()
@@ -716,7 +1102,7 @@ impl<'r> Builder<'r> {
ElKind::Cell => {
let tb = self.find_open(|e| e.kind == ElKind::Tr, |e| e.kind == ElKind::Table).is_none();
if let Some(t) = self.tables.last_mut()
&& declares_border(attrs, false)
&& declares_border(attrs, &css, false)
{
t.bordered = true;
}
@@ -728,6 +1114,7 @@ impl<'r> Builder<'r> {
t.row = Some(Vec::new());
}
let num = |k: &str| attr(attrs, k).and_then(|v| v.trim().parse::<u32>().ok()).map(|v| v.clamp(1, 63));
let colspan = num("colspan").unwrap_or(1);
let shading = attr(attrs, "bgcolor").and_then(parse_color).or(fmt.background);
let mut valign = attr(attrs, "valign").and_then(|v| match v.trim().to_ascii_lowercase().as_str() {
"top" => Some(VAlign::Top),
@@ -735,36 +1122,35 @@ impl<'r> Builder<'r> {
"bottom" => Some(VAlign::Bottom),
_ => None,
});
if let Some(st) = attr(attrs, "style") {
for decl in st.split(';') {
if let Some((k, v)) = decl.split_once(':')
&& k.trim().eq_ignore_ascii_case("vertical-align")
{
match v.trim().to_ascii_lowercase().as_str() {
"top" => valign = Some(VAlign::Top),
"middle" | "center" => valign = Some(VAlign::Center),
"bottom" => valign = Some(VAlign::Bottom),
_ => {}
}
}
for (_, v) in decls(&css).filter(|(k, _)| k == "vertical-align") {
match v.to_ascii_lowercase().as_str() {
"top" => valign = Some(VAlign::Top),
"middle" | "center" => valign = Some(VAlign::Center),
"bottom" => valign = Some(VAlign::Bottom),
_ => {}
}
}
self.cells.push(Cell {
blocks: Vec::new(),
colspan: num("colspan").unwrap_or(1),
colspan,
rowspan: attr(attrs, "rowspan").and_then(|v| v.trim().parse::<u32>().ok()).map(|v| v.clamp(1, 1000)).unwrap_or(1),
covered: false,
header: name == "th",
shading,
borders: None,
borders: cell_borders(&css),
valign,
width: if colspan == 1 { cell_width(&css, attrs) } else { None },
});
fmt.background = None;
self.containers.push(Vec::new());
}
_ => {}
}
self.stack.push(El { name: name.to_string(), kind, fmt, align, preserve, page_break, page_break_after, used: false });
// The page's own margins (`body`, `html`) aren't its paragraphs'.
let spacing =
if (kind != ElKind::Inline || self.page) && !matches!(name, "body" | "html") { block_spacing(&css) } else { Spacing::default() };
let auto_margins = self.page && (name == "p" || matches!(kind, ElKind::Heading(_)));
self.stack.push(El { name: name.to_string(), kind, fmt, spacing, auto_margins, align, preserve, page_break, page_break_after, used: false });
}
pub(super) fn end(&mut self, name: &str) {
+12 -5
View File
@@ -1,3 +1,5 @@
use super::builder::decls;
use super::import::PAGE_FONT;
use super::{Align, Fmt, model, parse_color};
/// A CSS length (or font-size keyword) in points; `None` when it is not one.
@@ -33,7 +35,7 @@ pub fn css_len_pt(v: &str) -> Option<f32> {
}
}
fn strip_important(v: &str) -> &str {
pub(super) fn strip_important(v: &str) -> &str {
let s = v.trim();
if let Some(pos) = s.rfind('!') {
let after = s[pos + 1..].trim();
@@ -53,10 +55,7 @@ pub(super) fn apply_style(
page_break: &mut bool,
page_break_after: &mut bool,
) {
for decl in css.split(';').take(64) {
let Some((k, v)) = decl.split_once(':') else { continue };
let k = k.trim().to_ascii_lowercase();
let v = strip_important(v);
for (k, v) in decls(css) {
let lv = v.to_ascii_lowercase();
match k.as_str() {
"font-weight" => f.bold = lv.starts_with("bold") || lv.parse::<u32>().is_ok_and(|w| w >= 600),
@@ -94,6 +93,14 @@ pub(super) fn apply_style(
f.code = true;
} else if !fam.is_empty() && !matches!(fam.to_ascii_lowercase().as_str(), "serif" | "sans-serif" | "inherit" | "initial") {
f.font = model::kept_font(&fam).or(f.font.take());
} else if f.font.is_some() {
// A generic family replaces a face the text has (a page's Times New Roman);
// text without one keeps the document's font.
match fam.to_ascii_lowercase().as_str() {
"serif" => f.font = Some(PAGE_FONT.into()),
"sans-serif" => f.font = Some("Arial".into()),
_ => {}
}
}
}
"vertical-align" => {
+30 -8
View File
@@ -274,17 +274,39 @@ fn sniff_charset(head: &str) -> Option<&str> {
/// Decode HTML bytes: BOM, then a `charset` in the first 2 KB, else UTF-8 with a legacy fallback.
pub fn decode(b: &[u8]) -> String {
decode_as(b, None)
}
/// [`decode`] in the `charset` its container states (a MIME part's, a content type's), when it
/// states one. Single-byte Windows code pages (`windows-1250`…, `iso-8859-1`, `us-ascii` as 1252)
/// are decoded as such, other labels by their WHATWG decoders.
pub fn decode_as(b: &[u8], charset: Option<&str>) -> String {
if b.starts_with(&[0xFF, 0xFE]) || b.starts_with(&[0xFE, 0xFF]) || b.starts_with(&[0xEF, 0xBB, 0xBF]) {
return crate::txt::decode(b);
}
let head = String::from_utf8_lossy(b.get(..b.len().min(2048)).unwrap_or(b)).to_ascii_lowercase();
if let Some(label) = sniff_charset(&head)
&& let Some(s) = crate::txt::decode_as(b, label)
{
return s;
let named = charset.map(str::to_ascii_lowercase).or_else(|| sniff_charset(&head).map(str::to_string));
let code_page = match named.as_deref().map(str::trim) {
Some("iso-8859-1" | "latin1" | "us-ascii") => Some(1252),
Some(cs) => cs
.strip_prefix("windows-")
.or_else(|| cs.strip_prefix("cp"))
.and_then(|n| n.parse::<u16>().ok())
.filter(|cp| wordcraft_doc::encoding::SINGLE_BYTE_CODE_PAGES.contains(cp)),
None => None,
};
// Other labels (ISO 8859, KOI8, Shift_JIS, GBK, Big5, EUC-KR…) by their WHATWG decoders; a
// UTF-16 one without a BOM means UTF-8, as in browsers.
let other = named
.as_deref()
.and_then(|l| encoding_rs::Encoding::for_label(l.trim().as_bytes()))
.filter(|e| ![encoding_rs::UTF_8, encoding_rs::UTF_16LE, encoding_rs::UTF_16BE].contains(e));
match (code_page, other) {
(Some(cp), _) => b.iter().map(|&c| wordcraft_doc::encoding::windows_byte(cp, c)).collect(),
(None, Some(enc)) => enc.decode_without_bom_handling(b).0.into_owned(),
(None, None) => match std::str::from_utf8(b) {
Ok(s) => s.to_string(),
Err(_) => crate::txt::decode(b),
},
}
if let Ok(s) = std::str::from_utf8(b) {
return s.to_string();
}
crate::txt::decode(b)
}
+32 -6
View File
@@ -1,4 +1,4 @@
use super::builder::{Builder, ImageLoader};
use super::builder::{Builder, ImageLoader, parse_sheet};
use super::lexer::{Lexer, Tok};
use super::math_ruby::{read_math, read_ruby};
use super::{Document, Flow, Inline, model};
@@ -10,6 +10,10 @@ pub fn parse(s: &str) -> Flow {
/// [`parse`], loading pictures that aren't `data:` URIs through `images`.
pub fn parse_with(s: &str, images: ImageLoader) -> Flow {
parse_in(s, images, false)
}
pub(super) fn parse_in(s: &str, images: ImageLoader, page: bool) -> Flow {
// HTML normalizes input newlines before tokenization, not after decoding character references.
let normalized = if s.contains('\r') {
let mut out = String::with_capacity(s.len());
@@ -23,8 +27,10 @@ pub fn parse_with(s: &str, images: ImageLoader) -> Flow {
} else {
std::borrow::Cow::Borrowed(s)
};
let mut lx = Lexer { s: &normalized, pos: 0 };
let mut b = Builder::new(images);
// The lexer's offsets (a style element's text is cut out by them) are into the normalized text.
let s: &str = &normalized;
let mut lx = Lexer { s, pos: 0 };
let mut b = Builder::new(images, page);
let mut guard = 0usize;
while let Some(t) = lx.next() {
guard += 1;
@@ -52,7 +58,11 @@ pub fn parse_with(s: &str, images: ImageLoader) -> Flow {
continue;
}
if matches!(name.as_str(), "script" | "style" | "noscript" | "template" | "textarea") && !self_close {
lx.skip_past_end_tag(&name);
let from = lx.pos;
let to = lx.skip_past_end_tag(&name);
if name == "style" {
parse_sheet(s.get(from..to).unwrap_or(""), &mut b.sheet);
}
continue;
}
b.start(&name, &attrs, self_close);
@@ -91,9 +101,25 @@ pub fn import_with(s: &str, images: ImageLoader) -> Document {
model::to_doc(&parse_with(s, images))
}
/// [`import_with`] onto `page` (its size and margins), which tables size to: HTML gives no page
/// A web page opened as a document, as Word opens one (a file, or a document's HTML
/// `w:altChunk`): its text is Times New Roman 12 pt on single lines, headings bold and black at
/// the sizes browsers give them, and a `p` or heading whose CSS leaves a margin out (or writes it
/// invalidly) gets HTML's automatic spacing there; a margin a span states replaces the one its
/// paragraph states (both as Word's import does). [`import`] reads a pasted fragment, which takes
/// the document's own formatting instead (but its CSS still applies).
pub fn import_page(s: &str, images: ImageLoader) -> Document {
model::begin_import();
model::to_doc(&parse_in(s, images, true))
}
/// [`import_page`] onto `page` (its size and margins), which tables size to: HTML gives no page
/// size, so the caller picks it (the region's paper, #1381).
pub fn import_on(s: &str, images: ImageLoader, page: &wordcraft_doc::SectionProps) -> Document {
model::begin_import();
model::to_doc_in(&parse_with(s, images), page.clone())
model::to_doc_in(&parse_in(s, images, true), page.clone())
}
/// The font of a web page's text when its CSS names none.
pub const PAGE_FONT: &str = "Times New Roman";
/// Heading sizes on a web page, `h1`–`h6`, points.
pub(super) const PAGE_HEADING_SIZES: [f32; 6] = [24.0, 18.0, 13.5, 12.0, 10.0, 7.5];
+7 -3
View File
@@ -34,7 +34,9 @@ impl Lexer<'_> {
}
/// Skip past the next matching end tag `</tag...>` (ASCII case-insensitive), or to the end.
pub(super) fn skip_past_end_tag(&mut self, tag: &str) {
/// Skip past the end tag of raw text element `tag`; returns where that end tag starts (the
/// end of the input when there is none).
pub(super) fn skip_past_end_tag(&mut self, tag: &str) -> usize {
let prefix = format!("</{tag}");
let p = prefix.as_bytes();
while self.pos < self.s.len() {
@@ -42,17 +44,19 @@ impl Lexer<'_> {
let found = hay.windows(p.len().max(1)).position(|w| w.eq_ignore_ascii_case(p));
let Some(k) = found else {
self.pos = self.s.len();
return;
return self.pos;
};
let after_tag = k + p.len();
let next_byte = hay.get(after_tag);
if next_byte.is_none_or(|b| b.is_ascii_whitespace() || *b == b'>' || *b == b'/') {
let at = self.pos + k;
self.pos += after_tag;
self.skip_past(">");
return;
return at;
}
self.pos += k + 2;
}
self.s.len()
}
pub(super) fn next(&mut self) -> Option<Tok> {
+192
View File
@@ -7,6 +7,27 @@ fn numeric_entity_rejects_sign() {
assert_eq!(unescape("a&#65;&#x41;b"), "aAAb");
}
#[test]
fn important_declarations_win_the_cascade() {
let color = |html: &str| fmt_of(paras(&parse(html))[0]).color;
let red = Some(wordcraft_doc::Rgb(255, 0, 0));
let blue = Some(wordcraft_doc::Rgb(0, 0, 255));
assert_eq!(color("<style>p { color: red }</style><p style='color: blue'>x</p>"), blue);
assert_eq!(color("<style>p { color: red !important }</style><p style='color: blue'>x</p>"), red);
assert_eq!(color("<style>p { color: red !important }</style><p style='color: blue !important'>x</p>"), blue);
assert_eq!(color("<p style='color: red !important; color: blue'>x</p>"), red);
}
#[test]
fn a_declared_charset_decodes_the_page() {
// ISO 8859-2 (Central European): 0xA9 Š, 0xB9 š; named by the caller or the page.
assert_eq!(decode_as(&[0xA9, b' ', 0xB9], Some("iso-8859-2")), "Š š");
assert!(decode_as(b"<meta charset=iso-8859-2><p>\xA9</p>", None).contains("<p>Š</p>"));
assert_eq!(decode_as(&[0x8A], Some("windows-1252")), "Š");
// A page that says UTF-16 without a BOM is read as UTF-8, as browsers do.
assert_eq!(decode_as("é".as_bytes(), Some("utf-16")), "é");
}
fn texts(f: &Flow) -> Vec<String> {
f.blocks
.iter()
@@ -33,6 +54,9 @@ fn borderless_tables_get_no_grid_style() {
assert_eq!(table_style("<table border=1><tr><td>a</table>").as_deref(), Some("TableGrid"));
assert_eq!(table_style("<table style='border:1px solid #000'><tr><td>a</table>").as_deref(), Some("TableGrid"));
assert_eq!(table_style("<table><tr><td style='border:1px solid #999'>a</table>").as_deref(), Some("TableGrid"));
// The last declaration decides: a `style` attribute's over the style sheet's.
assert_eq!(table_style("<style>table { border: 1px solid }</style><table><tr><td>a</table>").as_deref(), Some("TableGrid"));
assert_eq!(table_style("<style>table { border: 1px solid }</style><table style='border:none'><tr><td>a</table>").as_deref(), Some(""));
}
#[test]
@@ -522,3 +546,171 @@ fn empty_comments_do_not_swallow_the_rest() {
assert_eq!(doc.plain_text(Default::default()), "ab\nc", "{html}");
}
}
fn paras(f: &Flow) -> Vec<&Para> {
f.blocks.iter().filter_map(|b| if let FBlock::Para(p) = b { Some(p) } else { None }).collect()
}
fn fmt_of(p: &Para) -> Fmt {
p.inlines.iter().find_map(|i| if let Inline::Text(_, f) = i { Some(f.clone()) } else { None }).unwrap_or_default()
}
#[test]
fn style_sheet_rules_apply_in_cascade_order() {
let f = parse(
"<style><!-- p { color: #00ff00; font-size: 10pt } .big { font-size: 20pt } p.big { font-weight: bold } \
/* a comment { } */ @media print { p { color: #ff0000 } } div p, li:hover { font-style: italic } .000003 { text-align: center } --></style>\
<p>plain</p><p class='big x'>big</p><p class=big style='font-size: 8pt'>inline wins</p><p class=000003>digits</p>",
);
let ps = paras(&f);
let (plain, big, inline, digits) = (fmt_of(ps[0]), fmt_of(ps[1]), fmt_of(ps[2]), fmt_of(ps[3]));
assert_eq!((plain.color, plain.size, plain.bold, plain.italic), (Some(wordcraft_doc::Rgb(0, 255, 0)), Some(10.0), false, false));
assert_eq!((big.size, big.bold), (Some(20.0), true));
assert_eq!(inline.size, Some(8.0));
assert_eq!(ps[3].align, Some(Align::Center));
assert_eq!(digits.size, Some(10.0));
}
#[test]
fn paragraph_css_states_spacing_indents_and_line_height() {
let f = parse(
"<style>p.a { margin: 6pt 1in 3pt; text-indent: -18pt; line-height: 150% }</style>\
<p class=a>a</p><p style='margin-top: 14,4px; margin-bottom: auto; line-height: 20pt'>b</p><p>c</p>",
);
let ps = paras(&f);
assert_eq!(
ps[0].spacing,
Spacing {
before: Some(6.0),
after: Some(3.0),
left: Some(72.0),
right: Some(72.0),
first: Some(-18.0),
line: Some(LineSpacing::Multiple(1.5)),
..Spacing::default()
}
);
// Values that don't parse (a comma decimal, `auto`) state nothing.
assert_eq!(ps[1].spacing, Spacing { line: Some(LineSpacing::Exactly(20.0)), ..Spacing::default() });
assert_eq!(ps[2].spacing, Spacing::default());
}
#[test]
fn a_page_opens_as_word_opens_one() {
let html = "<style>p.k { margin-bottom: 13px } span.z { margin: 0in }</style>\
<p>first</p><h1>Head</h1><p class=k><span class=z>kept</span></p><p class=k>own</p><div>bare</div><p>&nbsp;</p>";
let f = parse_in(html, &|_| None, true);
let ps = paras(&f);
// Times New Roman 12 pt on single lines; headings bold and black at a browser's sizes.
let first = fmt_of(ps[0]);
assert_eq!((first.font.as_deref(), first.size), (Some(PAGE_FONT), Some(12.0)));
assert_eq!(ps[0].spacing.line, Some(LineSpacing::Multiple(1.0)));
let cell = parse_in("<table><tr><td>cell</td></tr></table>", &|_| None, true);
let FBlock::Table(t) = &cell.blocks[0] else { panic!() };
let FBlock::Para(cp) = &t.rows[0][0].blocks[0] else { panic!() };
assert_eq!(cp.spacing.line, Some(LineSpacing::Multiple(1.0)));
let head = fmt_of(ps[1]);
assert_eq!((head.bold, head.size, head.color), (true, Some(24.0), Some(wordcraft_doc::Rgb(0, 0, 0))));
// A colour the heading inherits stays.
// A generic family names a face on a page; pasted text keeps the document's font.
let generic = parse_in("<p style='font-family: sans-serif'>a</p><p style='font-family: serif'>b</p>", &|_| None, true);
let fonts: Vec<Option<String>> = paras(&generic).iter().map(|p| fmt_of(p).font.map(|f| f.to_string())).collect();
assert_eq!(fonts, [Some("Arial".to_string()), Some(PAGE_FONT.to_string())]);
assert_eq!(fmt_of(paras(&parse("<p style='font-family: sans-serif'>a</p>"))[0]).font, None);
let inherited = parse_in("<body style='color: #000080'><h1>Head</h1></body>", &|_| None, true);
assert_eq!(fmt_of(paras(&inherited)[0]).color, Some(wordcraft_doc::Rgb(0, 0, 0x80)));
// Automatic spacing where a p or heading leaves its margins out (layout drops it at the
// top of the page and of a cell).
let auto = |p: &Para| (p.spacing.before_auto, p.spacing.after_auto, p.spacing.before, p.spacing.after);
assert_eq!(auto(ps[0]), (true, true, None, None));
assert_eq!(auto(ps[1]), (true, true, None, None));
// A span's margin replaces the one its paragraph states; the one it leaves out stays automatic.
assert_eq!(auto(ps[2]), (true, false, None, Some(0.0)));
assert_eq!(auto(ps[3]), (true, false, None, Some(9.75)));
// Other blocks get no automatic spacing.
assert_eq!(auto(ps[4]), (false, false, None, None));
// The document says so, and the empty paragraph's mark keeps its size.
let d = model::to_doc(&f);
let props = |i: usize| match &*d.body[i] {
wordcraft_doc::Block::Para(p) => p.clone(),
b => panic!("{b:?}"),
};
assert_eq!((props(0).props.space_before_auto, props(0).props.space_after_auto), (Some(true), Some(true)));
assert_eq!((props(2).props.space_after_auto, props(2).props.space_after), (None, Some(0.0)));
assert_eq!(props(5).mark.size, Some(12.0));
// `<p>&nbsp;</p>` keeps the size of its space for its line.
assert!(ps[5].inlines.is_empty());
assert_eq!(ps[5].mark.as_ref().and_then(|m| m.size), Some(12.0));
// A fragment (a paste) keeps the document's own formatting.
let pasted = parse(html);
assert_eq!(fmt_of(paras(&pasted)[0]).font, None);
assert_eq!(paras(&pasted)[0].spacing, Spacing::default());
}
#[test]
fn cells_give_their_columns_widths() {
let f = parse(
"<style>td.n { width: 42px }</style><table border=1><tr><td colspan=3>t</td></tr><tr><td class=n>1</td><td width=200>a</td><td style='width: 2in'>b</td></tr></table>",
);
let FBlock::Table(t) = &f.blocks[0] else { panic!("{:?}", f.blocks) };
assert_eq!(t.widths, [31.5, 150.0, 144.0]);
// A column without a width leaves them all to the layout.
let f = parse("<table><tr><td width=100>a</td><td>b</td></tr></table>");
let FBlock::Table(t) = &f.blocks[0] else { panic!() };
assert!(t.widths.is_empty());
// A cell beside a cell spanning rows from above gives its own column its width.
let f =
parse("<table><tr><td rowspan=2>a</td><td width=200>b</td></tr><tr><td width=300>c</td></tr><tr><td width=100>d</td><td>e</td></tr></table>");
let FBlock::Table(t) = &f.blocks[0] else { panic!() };
assert_eq!(t.widths, [75.0, 150.0]);
}
#[test]
fn cells_take_their_css_borders() {
use wordcraft_doc::props::{Border, BorderStyle};
let f = parse(
"<style>td.b { border-top: solid windowtext 1px; border-bottom: 2pt double #ff0000; border-left: none; border-right: 0px solid; vertical-align: bottom }</style><table><tr><td class=b>a</td><td>b</td></tr></table>",
);
let FBlock::Table(t) = &f.blocks[0] else { panic!() };
// The style sheet's other cell properties apply too.
assert_eq!(t.rows[0][0].valign, Some(wordcraft_doc::props::VAlign::Bottom));
let b = t.rows[0][0].borders.expect("borders");
assert_eq!(b.top, Some(Border::single(0.75)));
assert_eq!(b.bottom, Some(Border { style: BorderStyle::Double, width: 2.0, color: Some(wordcraft_doc::Rgb(255, 0, 0)), space: 0.0 }));
assert_eq!(b.left.map(|l| l.style), Some(BorderStyle::None));
// A border of no width is none (OOXML has no zero-width line).
assert_eq!(b.right.map(|r| r.style), Some(BorderStyle::None));
assert_eq!(t.rows[0][1].borders, None);
}
#[test]
fn style_sheet_edge_cases() {
let f = parse_in(
"<style>@charset \"utf-8\"; @import url(a.css); p.a { color: #0000ff } body { margin: 20px } p.b { margin: 10 5% medium }</style>\
<body><p class=a>a</p>bare<p class=b>b</p></body>",
&|_| None,
true,
);
let ps = paras(&f);
// A rule right after statement at-rules still applies.
assert_eq!(fmt_of(ps[0]).color, Some(wordcraft_doc::Rgb(0, 0, 255)));
// The body's margins aren't the paragraphs' indents.
assert_eq!((ps[1].spacing.left, ps[1].spacing.before), (None, None));
// Bare numbers, percentages and keywords state no margin: automatic spacing stays.
assert_eq!((ps[2].spacing.before_auto, ps[2].spacing.left), (true, None));
// The `style` attribute wins even over a sheet with more declarations than are read.
let many = (0..600).map(|i| format!("x-{i}: 1")).collect::<Vec<_>>().join(";");
let f = parse(&format!("<style>p {{ {many}; color: #ff0000 }}</style><p style='color: #00ff00'>c</p>"));
assert_eq!(fmt_of(paras(&f)[0]).color, Some(wordcraft_doc::Rgb(0, 255, 0)));
}
#[test]
fn hostile_style_sheets_never_panic() {
// One declaration block shared by many selectors isn't copied for each.
let selectors = vec!["p"; 20_000].join(",");
let huge = format!("<style>{selectors} {{ color: red; {} }}</style><p>x</p>", "x: y;".repeat(200_000));
let _ = parse_in(&huge, &|_| None, true);
let many = ".a { color: red } ".repeat(20_000);
for css in ["{", "}", "p {", "@media { p { x } ", "p { color: red", "/* open", ".{}", "p..a{}", "\u{0}.é{x:y}", many.as_str()] {
let _ = parse_in(&format!("<style>{css}</style><p class='a é'>x</p><td style='width: 1e40px'>y</td>"), &|_| None, true);
}
}
+2
View File
@@ -430,6 +430,8 @@ pub fn escape_text(s: &str) -> String {
'>' => o.push_str("\\textgreater{}"),
'|' => o.push_str("\\textbar{}"),
'\u{A0}' => o.push('~'),
// TeX collapses a run of spaces into one; a control space keeps each further one.
' ' if prev == ' ' => o.push_str("\\ "),
// An opening bracket here would become an optional item or line-break argument.
'[' if !line_has_text => o.push_str("{[}"),
// Keep TeX from joining `--`, ``` `` ``` and `''` into dashes and quotes.
+5 -1
View File
@@ -485,7 +485,11 @@ impl<'a> Parser<'a> {
fn command(&mut self, name: &str) {
if let Some(sym) = text_symbol(name) {
if name == "item" {
if name == " " {
// A control space is never collapsed into the space before it.
let f = self.fmt();
self.para.push_text(sym, &f);
} else if name == "item" {
self.item();
} else if matches!(name, "centering" | "raggedleft" | "raggedright") {
let a = match name {
+13 -5
View File
@@ -1,6 +1,7 @@
//! WordCraft import/export for the "other" text formats: plain text, Markdown (CommonMark subset
//! plus GFM tables and strikethrough), HTML, RTF, OpenDocument Text and LaTeX — and, read-only,
//! spreadsheet cells for mail-merge recipient lists ([`sheet`]: .xlsx/.xlsm/.xls/.ods).
//! plus GFM tables and strikethrough), HTML (and MHT web archives, read-only), RTF, OpenDocument
//! Text and LaTeX — and, read-only, spreadsheet cells for mail-merge recipient lists ([`sheet`]:
//! .xlsx/.xlsm/.xls/.ods).
//!
//! Every format maps to a small flow model ([`model::Flow`]: paragraphs with a kind, list
//! membership, alignment and formatted spans; tables of cells), which [`model::to_doc`] and
@@ -14,6 +15,7 @@ pub mod html;
pub mod latex;
pub mod markdown;
pub mod mathml;
pub mod mht;
pub mod model;
pub mod odt;
pub mod rtf;
@@ -22,8 +24,12 @@ pub mod txt;
use wordcraft_doc::Document;
/// Most entries an ODT or spreadsheet package may declare, as many as a Word package
/// (`wordcraft_docx::package::MAX_ENTRIES`); checked before `zip` parses them (#3660).
pub const MAX_ZIP_ENTRIES: u64 = 20_000;
/// Extensions this crate handles.
pub const EXTENSIONS: &[&str] = &["txt", "text", "md", "markdown", "html", "htm", "xhtml", "rtf", "odt", "tex", "latex", "ltx"];
pub const EXTENSIONS: &[&str] = &["txt", "text", "md", "markdown", "html", "htm", "xhtml", "mht", "mhtml", "rtf", "odt", "tex", "latex", "ltx"];
/// Import `bytes` in the format named by `ext` (no dot, any case). `None` = not our format.
pub fn import(ext: &str, bytes: &[u8]) -> Option<Result<Document, String>> {
@@ -31,7 +37,8 @@ pub fn import(ext: &str, bytes: &[u8]) -> Option<Result<Document, String>> {
let r = match ext.as_str() {
"txt" | "text" => Ok(txt::import(bytes)),
"md" | "markdown" => Ok(markdown::import(&txt::decode(bytes))),
"html" | "htm" | "xhtml" => Ok(html::import(&html::decode(bytes))),
"html" | "htm" | "xhtml" => Ok(html::import_page(&html::decode(bytes), &|_| None)),
"mht" | "mhtml" => Ok(mht::import(bytes)),
"rtf" => rtf::import(bytes),
"odt" => odt::import(bytes),
"tex" | "latex" | "ltx" => Ok(latex::import(bytes)),
@@ -44,7 +51,7 @@ pub fn import(ext: &str, bytes: &[u8]) -> Option<Result<Document, String>> {
}
/// [`import`] onto `page` (its size and margins) for the formats whose files give no page size:
/// plain text, Markdown and HTML start on it, and their tables size to it (#1381). The other
/// plain text, Markdown, HTML and MHT start on it, and their tables size to it (#1381). The other
/// formats keep the page their file gives.
pub fn import_on(ext: &str, bytes: &[u8], page: &wordcraft_doc::SectionProps) -> Option<Result<Document, String>> {
let ext = ext.trim_start_matches('.').to_ascii_lowercase();
@@ -52,6 +59,7 @@ pub fn import_on(ext: &str, bytes: &[u8], page: &wordcraft_doc::SectionProps) ->
"txt" | "text" => txt::import_on(bytes, page),
"md" | "markdown" => markdown::import_on(&txt::decode(bytes), page),
"html" | "htm" | "xhtml" => html::import_on(&html::decode(bytes), &|_| None, page),
"mht" | "mhtml" => mht::import_on(bytes, page),
_ => return import(&ext, bytes),
};
doc.ensure_nonempty();
+9 -2
View File
@@ -380,9 +380,16 @@ pub(super) fn write_blocks(blocks: &[FBlock], out: &mut Vec<String>) {
// The bullet marker can turn a short run of hyphens into a rule.
if thematic_break(&format!("{marker} {body}")) { format!("\\{body}") } else { body }
};
out.push(format!("{pad}{marker} {body}").trim_end().to_string());
let line = format!("{pad}{marker} {body}").trim_end().to_string();
// A list inside a block quote stays in it: every line of the item, its
// continuation lines included, takes the quote marker.
let line = if p.kind == Kind::Quote { format!("> {}", line.replace('\n', "\n> ")) } else { line };
out.push(line);
prev_list = true;
let next_is_list = matches!(blocks.get(i + 1), Some(FBlock::Para(q)) if q.list.is_some());
// A list going into or out of a block quote is a new block: without the
// blank line, an unquoted item would be read back into the quote.
let quoted = |k: Kind| k == Kind::Quote;
let next_is_list = matches!(blocks.get(i + 1), Some(FBlock::Para(q)) if q.list.is_some() && quoted(q.kind) == quoted(p.kind));
if !next_is_list {
out.push(String::new());
}
+19
View File
@@ -558,6 +558,25 @@ fn ordered_list_keeps_start_number() {
assert_eq!(export(&import("5. a\n6. b")).trim_end(), "5. a\n6. b");
}
/// A list inside a block quote stays in the quote: its items were imported as quoted list
/// paragraphs but written back without `>`, so the list moved out of the quote.
#[test]
fn a_list_inside_a_quote_stays_quoted() {
let src = "> Intro\n>\n> - one\n> - two\n> - nested\n> 1. first\n\nAfter.\n";
let want = "> Intro\n\n> - one\n> - two\n> - nested\n> 1. first\n\nAfter.";
let md = export(&import(src));
assert_eq!(md.trim_end(), want);
assert_eq!(export(&import(&md)).trim_end(), want, "and stays so when read back");
// An item's continuation line is quoted too.
assert_eq!(export(&import("> - a\\\n> b\n")).trim_end(), "> - a\\\n> b");
// A list outside a quote is unchanged.
assert_eq!(export(&import("- one\n- two\n")).trim_end(), "- one\n- two");
// A list after a quoted list stays outside the quote.
let md = export(&import("> - a\n\n2. b\n"));
assert_eq!(md.trim_end(), "> - a\n\n2. b");
assert_eq!(export(&import(&md)).trim_end(), "> - a\n\n2. b", "and stays so when read back");
}
#[test]
fn lines_after_line_break_stay_in_paragraph() {
use wordcraft_doc::{CharProps, Document, Paragraph, StoryRef, para_block};
+444
View File
@@ -0,0 +1,444 @@
//! MHT / MHTML (Single File Web Page): a web page and its pictures in one MIME message (RFC 2557
//! `multipart/related`, RFC 2045 transfer encodings). The page is the part the message's `start`
//! parameter names, else its first HTML part; it is read by [`crate::html`], with `img` sources
//! found among the other parts by `Content-Location` (also relative to the page's) or `cid:`
//! `Content-ID`, and the page opens as Word opens one ([`crate::html::import_page`]). Word keeps
//! HTML `w:altChunk`s in this form too.
use std::collections::HashMap;
use std::sync::Arc;
use wordcraft_doc::Document;
/// Most parts read from one message (nested multiparts included).
const MAX_PARTS: usize = 10_000;
/// Deepest `multipart` nesting followed.
const MAX_NESTING: usize = 8;
/// Longest header block looked at for one part.
const MAX_HEADER: usize = 64 * 1024;
/// Longest boundary (RFC 2046 §5.1.1).
const MAX_BOUNDARY: usize = 70;
/// Picture bytes one page may load, counting every `img` that names a part.
const PICTURE_BUDGET: usize = 1 << 30;
/// One leaf part of the message: its headers (names lower-cased) and decoded body.
struct Part {
content_type: String,
location: Option<String>,
id: Option<String>,
/// Shared by every `img` that names it.
body: Arc<Vec<u8>>,
}
impl Part {
fn mime(&self) -> String {
self.content_type.split(';').next().unwrap_or("").trim().to_ascii_lowercase()
}
}
/// Does `bytes` look like a MIME message (an `.mht` file)?
pub fn is_mht(bytes: &[u8]) -> bool {
let head = String::from_utf8_lossy(bytes.get(..bytes.len().min(4096)).unwrap_or(bytes)).to_ascii_lowercase();
head.starts_with("mime-version:") || head.starts_with("content-type: multipart/") || head.contains("\ncontent-type: multipart/related")
}
/// Import an MHT file. A message without an HTML part reads as its text (an empty document
/// when there's none).
pub fn import(bytes: &[u8]) -> Document {
import_on(bytes, &wordcraft_doc::SectionProps::default())
}
/// [`import`] onto `page`: like HTML, an MHT file gives no page size (#1381).
pub fn import_on(bytes: &[u8], page: &wordcraft_doc::SectionProps) -> Document {
let mut parts = Vec::new();
let (headers, body) = split_headers(bytes);
let start = param(header(&headers, "content-type").unwrap_or(""), "start").map(|s| s.trim_matches(['<', '>']).to_string());
collect(&headers, body, &mut parts, 0);
let root = start
.as_deref()
.and_then(|s| parts.iter().position(|p| p.id.as_deref() == Some(s)))
.or_else(|| parts.iter().position(|p| matches!(p.mime().as_str(), "text/html" | "application/xhtml+xml")));
let Some(root) = root.and_then(|i| parts.get(i)) else {
let text = parts.iter().find(|p| p.mime() == "text/plain").map(|p| p.body.as_slice()).unwrap_or(&[]);
return crate::txt::import_on(text, page);
};
let html = crate::html::decode_as(&root.body, param(&root.content_type, "charset"));
let base = root.location.as_deref().unwrap_or("");
// Parts by `Content-Location` (lower case) and `Content-ID`; the first of each wins.
let mut by_location = HashMap::new();
let mut by_id = HashMap::new();
for p in &parts {
if let Some(l) = &p.location {
by_location.entry(l.to_ascii_lowercase()).or_insert(p);
}
if let Some(i) = &p.id {
by_id.entry(i.as_str()).or_insert(p);
}
}
let left = std::cell::Cell::new(PICTURE_BUDGET);
let loader = |src: &str| -> Option<Arc<Vec<u8>>> {
let src = src.trim();
let part = match src.get(..4).filter(|s| s.eq_ignore_ascii_case("cid:")).and(src.get(4..)) {
Some(cid) => by_id.get(cid),
// By its location as written, or resolved against the page's.
None => by_location.get(&src.to_ascii_lowercase()).or_else(|| by_location.get(&resolve(base, src).to_ascii_lowercase())),
}?;
left.set(left.get().checked_sub(part.body.len())?);
Some(part.body.clone())
};
crate::html::import_on(&html, &loader, page)
}
/// `reference` resolved against the page's location `base`, as RFC 2557 §5 asks (RFC 3986
/// §5.2.2): a reference with a scheme stays as it is, `//host/…` takes the base's scheme, `/…`
/// its scheme and host, an empty path its path and query, anything else is merged with its path
/// (§5.2.3); `.` and `..` segments go (§5.2.4) and fragments are dropped. Nothing is fetched.
fn resolve(base: &str, reference: &str) -> String {
let (b, r) = (UriParts::of(base), UriParts::of(reference));
let t = if r.scheme.is_some() {
UriParts { path: remove_dots(&r.path), ..r }
} else if r.authority.is_some() {
UriParts { scheme: b.scheme, path: remove_dots(&r.path), ..r }
} else if r.path.is_empty() {
UriParts { query: r.query.or(b.query), ..b }
} else if r.path.starts_with('/') {
UriParts { path: remove_dots(&r.path), query: r.query, ..b }
} else {
let merged = if b.authority.is_some() && b.path.is_empty() {
format!("/{}", r.path)
} else {
format!("{}{}", b.path.rfind('/').and_then(|i| b.path.get(..=i)).unwrap_or(""), r.path)
};
UriParts { path: remove_dots(&merged), query: r.query, ..b }
};
t.to_string()
}
/// A URI's parts (RFC 3986 §3): `scheme:`, `//authority`, path and `?query`; the fragment left out.
struct UriParts<'a> {
scheme: Option<&'a str>,
authority: Option<&'a str>,
path: String,
query: Option<&'a str>,
}
impl<'a> UriParts<'a> {
fn of(uri: &'a str) -> Self {
let uri = uri.split('#').next().unwrap_or("");
// A one-letter scheme is a Windows drive (`C:/…`), not a scheme.
let scheme = uri.split_once(':').map(|(k, _)| k).filter(|k| {
k.len() > 1 && k.starts_with(|c: char| c.is_ascii_alphabetic()) && k.chars().all(|c| c.is_ascii_alphanumeric() || "+-.".contains(c))
});
let rest = scheme.and_then(|k| uri.get(k.len() + 1..)).unwrap_or(uri);
let (authority, rest) = match rest.strip_prefix("//") {
Some(r) => {
let end = r.find(['/', '?']).unwrap_or(r.len());
(r.get(..end), r.get(end..).unwrap_or(""))
}
None => (None, rest),
};
let (path, query) = match rest.split_once('?') {
Some((p, q)) => (p, Some(q)),
None => (rest, None),
};
UriParts { scheme, authority, path: path.to_string(), query }
}
}
impl std::fmt::Display for UriParts<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
if let Some(s) = self.scheme {
write!(f, "{s}:")?;
}
if let Some(a) = self.authority {
write!(f, "//{a}")?;
}
f.write_str(&self.path)?;
if let Some(q) = self.query {
write!(f, "?{q}")?;
}
Ok(())
}
}
/// `path` without its `.` and `..` segments (RFC 3986 §5.2.4); `..` stops at the root.
fn remove_dots(path: &str) -> String {
let mut out: Vec<&str> = Vec::new();
let segments: Vec<&str> = path.split('/').collect();
for (i, seg) in segments.iter().enumerate() {
match *seg {
"." => {}
".." => {
if out.len() > 1 || out.first().is_some_and(|f| !f.is_empty()) {
out.pop();
}
}
s => out.push(s),
}
if i + 1 == segments.len() && matches!(*seg, "." | "..") {
out.push("");
}
}
out.join("/")
}
/// Split a part into its unfolded header lines and its body (after the first empty line).
fn split_headers(b: &[u8]) -> (Vec<(String, String)>, &[u8]) {
// Whichever blank line comes first ends the headers.
let blank = [find_bytes(b, b"\r\n\r\n").map(|i| (i, 4)), find_bytes(b, b"\n\n").map(|i| (i, 2))].into_iter().flatten().min();
let (head, body) = match blank {
Some((i, n)) => (b.get(..i).unwrap_or(&[]), b.get(i + n..).unwrap_or(&[])),
// Headers only, or a body without them.
None if b.first().is_some_and(|c| c.is_ascii_alphabetic()) && find_bytes(b, b":").is_some() => (b, &[][..]),
None => (&[][..], b),
};
let head = String::from_utf8_lossy(head.get(..head.len().min(MAX_HEADER)).unwrap_or(head));
let mut headers: Vec<(String, String)> = Vec::new();
for line in head.lines() {
if line.starts_with([' ', '\t']) {
if let Some((_, v)) = headers.last_mut() {
v.push(' ');
v.push_str(line.trim());
}
} else if let Some((k, v)) = line.split_once(':') {
headers.push((k.trim().to_ascii_lowercase(), v.trim().to_string()));
}
}
(headers, body)
}
fn header<'a>(h: &'a [(String, String)], name: &str) -> Option<&'a str> {
h.iter().find(|(k, _)| k == name).map(|(_, v)| v.as_str())
}
/// A `; name=value` parameter of a header value (quotes removed), name case-insensitive.
fn param<'a>(value: &'a str, name: &str) -> Option<&'a str> {
value.split(';').skip(1).find_map(|p| {
let (k, v) = p.split_once('=')?;
k.trim().eq_ignore_ascii_case(name).then(|| v.trim().trim_matches('"'))
})
}
/// Gather the leaf parts of a (possibly multipart) entity.
fn collect(headers: &[(String, String)], body: &[u8], out: &mut Vec<Part>, depth: usize) {
if out.len() >= MAX_PARTS {
return;
}
let ct = header(headers, "content-type").unwrap_or("text/plain").to_string();
if ct.trim_start().to_ascii_lowercase().starts_with("multipart/") {
let Some(boundary) = param(&ct, "boundary").filter(|b| !b.is_empty() && b.len() <= MAX_BOUNDARY) else { return };
if depth >= MAX_NESTING {
return;
}
for chunk in multipart_bodies(body, boundary.as_bytes()) {
let (h, b) = split_headers(chunk);
collect(&h, b, out, depth + 1);
if out.len() >= MAX_PARTS {
return;
}
}
return;
}
let enc = header(headers, "content-transfer-encoding").unwrap_or("").trim().to_ascii_lowercase();
let body = match enc.as_str() {
"quoted-printable" => quoted_printable(body),
"base64" => crate::model::base64_decode(&String::from_utf8_lossy(body)).unwrap_or_default(),
_ => body.to_vec(),
};
let clean = |s: &str| s.trim().trim_matches(['<', '>']).to_string();
out.push(Part {
content_type: ct,
location: header(headers, "content-location").map(clean),
id: header(headers, "content-id").map(clean),
body: Arc::new(body),
});
}
/// The bodies between `--boundary` lines, up to the closing `--boundary--`.
fn multipart_bodies<'a>(body: &'a [u8], boundary: &[u8]) -> Vec<&'a [u8]> {
let mut delim = b"--".to_vec();
delim.extend_from_slice(boundary);
let mut out = Vec::new();
// Delimiters sit at line starts.
let mut starts = Vec::new();
let mut i = 0;
while let Some(k) = body.get(i..).and_then(|rest| find_bytes(rest, &delim)) {
let at = i + k;
if at == 0 || body.get(at - 1) == Some(&b'\n') {
starts.push(at);
if starts.len() > MAX_PARTS {
break;
}
}
i = at + delim.len();
}
for w in starts.windows(2) {
let (Some(&a), Some(&b)) = (w.first(), w.get(1)) else { continue };
let Some(seg) = body.get(a + delim.len()..b) else { continue };
if seg.starts_with(b"--") {
break;
}
// The rest of the delimiter line, then the part; the line break before the next
// delimiter belongs to it.
let seg = match find_bytes(seg, b"\n") {
Some(n) => seg.get(n + 1..).unwrap_or(&[]),
None => &[],
};
let seg = seg.strip_suffix(b"\r\n").or_else(|| seg.strip_suffix(b"\n")).unwrap_or(seg);
out.push(seg);
if out.len() >= MAX_PARTS {
break;
}
}
out
}
fn find_bytes(hay: &[u8], needle: &[u8]) -> Option<usize> {
if needle.is_empty() {
return None;
}
hay.windows(needle.len()).position(|w| w == needle)
}
/// Decode quoted-printable (RFC 2045 §6.7): `=XX` bytes and `=` soft line breaks; anything
/// malformed is kept as it stands.
fn quoted_printable(b: &[u8]) -> Vec<u8> {
let hex = |c: u8| (c as char).to_digit(16).and_then(|d| u8::try_from(d).ok());
let mut out = Vec::with_capacity(b.len());
let mut i = 0;
while let Some(&c) = b.get(i) {
if c != b'=' {
out.push(c);
i += 1;
continue;
}
match (b.get(i + 1).copied(), b.get(i + 2).copied()) {
(Some(b'\r'), Some(b'\n')) => i += 3,
(Some(b'\n'), _) => i += 2,
(Some(h), Some(l)) if let (Some(h), Some(l)) = (hex(h), hex(l)) => {
out.push((h << 4) | l);
i += 3;
}
_ => {
out.push(c);
i += 1;
}
}
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use wordcraft_doc::{Block, InlineObject};
const PNG_1X1: &str = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==";
fn message(html_part: &str) -> String {
format!(
"MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=\"----=_B\"\r\n\r\n\
------=_B\r\n{html_part}\r\n\
------=_B\r\nContent-Location: file:///C:/page_files/dot.png\r\nContent-Transfer-Encoding: base64\r\nContent-Type: image/png\r\n\r\n{PNG_1X1}\r\n\
------=_B--\r\n"
)
}
#[test]
fn reads_the_quoted_printable_page_and_its_pictures() {
let page = "Content-Location: file:///C:/page.htm\r\nContent-Transfer-Encoding: quoted-printable\r\nContent-Type: text/html; charset=\"utf-8\"\r\n\r\n\
<html><body><p class=3D\"x\">Dobar dan, gospo=C4=91o, ovo je =\r\njedan red.</p><p><img src=3D\"page_files/dot.png\" width=3D\"10\" height=3D\"10\"></p></body></html>";
let d = import(message(page).as_bytes());
let text = d.plain_text(wordcraft_doc::StoryRef::Body);
assert!(text.contains("Dobar dan, gospođo, ovo je jedan red."), "{text:?}");
let pictures =
d.body.iter().filter(|b| matches!(&***b, Block::Para(p) if p.objects.iter().any(|o| matches!(o, InlineObject::Image { .. })))).count();
assert_eq!(pictures, 1);
assert_eq!(d.media.len(), 1);
}
#[test]
fn pictures_named_relative_to_the_page_resolve_as_uris() {
// The page in `reports/`, its picture in `images/` (RFC 2557 §5: relative URIs resolve
// against the page's Content-Location).
let m = |src: &str| {
format!(
"MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=b\r\n\r\n\
--b\r\nContent-Location: https://example.test/reports/page.html\r\nContent-Type: text/html\r\n\r\n<p>x</p><img src=\"{src}\">\r\n\
--b\r\nContent-Location: https://example.test/images/logo.png\r\nContent-Transfer-Encoding: base64\r\nContent-Type: image/png\r\n\r\n{PNG_1X1}\r\n--b--\r\n"
)
};
for src in [
"../images/logo.png",
"/images/logo.png",
"./../images/./logo.png",
"//example.test/images/logo.png",
"https://example.test/images/logo.png",
] {
assert_eq!(import(m(src).as_bytes()).media.len(), 1, "{src}");
}
// Nothing past the root: `..` there stays at the root.
assert_eq!(import(m("../../../images/logo.png").as_bytes()).media.len(), 1);
assert_eq!(import(m("images/logo.png").as_bytes()).media.len(), 0);
assert_eq!(resolve("https://h/a/b/c.html", "../d/./e.png#f"), "https://h/a/d/e.png");
assert_eq!(resolve("file:///C:/x/page.htm", "page_files/dot.png"), "file:///C:/x/page_files/dot.png");
assert_eq!(resolve("", "dot.png"), "dot.png");
// A host with no path, and a base whose query holds slashes (RFC 3986 §5.2.3: the path
// merges, not the query; an authority with an empty path gets its `/`).
assert_eq!(resolve("https://example.test", "logo.png"), "https://example.test/logo.png");
assert_eq!(resolve("https://example.test/page.html?redirect=/other/", "logo.png"), "https://example.test/logo.png");
assert_eq!(resolve("https://h/a/p.html?q", "?r"), "https://h/a/p.html?r");
assert_eq!(resolve("https://h/a/p.html?q", "x.png?v=1"), "https://h/a/x.png?v=1");
assert_eq!(resolve("https://h/a/p.html", "mailto:x@y"), "mailto:x@y");
for page in ["https://example.test", "https://example.test/page.html?redirect=/other/"] {
let m = format!(
"MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=b\r\n\r\n\
--b\r\nContent-Location: {page}\r\nContent-Type: text/html\r\n\r\n<p>x</p><img src=\"logo.png\">\r\n\
--b\r\nContent-Location: https://example.test/logo.png\r\nContent-Transfer-Encoding: base64\r\nContent-Type: image/png\r\n\r\n{PNG_1X1}\r\n--b--\r\n"
);
assert_eq!(import(m.as_bytes()).media.len(), 1, "{page}");
}
}
#[test]
fn the_part_charset_decides_the_text() {
let page = "Content-Type: text/html; charset=windows-1250\r\nContent-Transfer-Encoding: quoted-printable\r\n\r\n<p>=E8lanak</p>";
let d = import(message(page).as_bytes());
assert_eq!(d.plain_text(wordcraft_doc::StoryRef::Body).trim(), "članak");
}
#[test]
fn the_start_part_is_the_page_and_cid_names_pictures() {
let m = format!(
"MIME-Version: 1.0\nContent-Type: multipart/related; type=\"text/html\"; start=\"<page>\"; boundary=b\n\n\
--b\nContent-Type: text/html\n\n<p>Not this one</p>\n\
--b\nContent-ID: <page>\nContent-Type: text/html\n\n<p>This one</p><img src=\"cid:dot\">\n\
--b\nContent-ID: <dot>\nContent-Transfer-Encoding: base64\nContent-Type: image/png\n\n{PNG_1X1}\n--b--\n"
);
let d = import(m.as_bytes());
assert_eq!(d.plain_text(wordcraft_doc::StoryRef::Body).trim(), "This one");
assert_eq!(d.media.len(), 1);
// Without a page, a message reads as its text.
let d = import(b"MIME-Version: 1.0\nContent-Type: multipart/mixed; boundary=b\n\n--b\nContent-Type: text/plain\n\nJust text\n--b--\n");
assert_eq!(d.plain_text(wordcraft_doc::StoryRef::Body).trim(), "Just text");
assert!(is_mht(m.as_bytes()) && !is_mht(b"<html><p>x</p>"));
}
#[test]
fn hostile_messages_never_panic() {
for m in [
&b""[..],
b"MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=\"\"\r\n\r\n",
b"MIME-Version: 1.0\r\nContent-Type: multipart/related; boundary=x\r\n\r\n--x\r\n--x--",
b"MIME-Version: 1.0\nContent-Type: multipart/related; boundary=x\n\n--x\nContent-Transfer-Encoding: quoted-printable\nContent-Type: text/html\n\n=\n=Z=4",
b"Content-Type: text/html\r\nContent-Transfer-Encoding: base64\r\n\r\n!!!!",
] {
let _ = import(m);
}
let mut deep = String::from("MIME-Version: 1.0\r\n");
for i in 0..64 {
deep.push_str(&format!("Content-Type: multipart/related; boundary=b{i}\r\n\r\n--b{i}\r\n"));
}
let _ = import(deep.as_bytes());
}
}
+3
View File
@@ -148,6 +148,7 @@ fn flow_paras_in(doc: &Document, p: &Paragraph, tracked: bool) -> Vec<Para> {
mark_ins: None,
mark_del: None,
fmt_change: None,
mark: None,
};
let in_code = kind == Kind::Code;
let mut k = 0usize;
@@ -179,6 +180,7 @@ fn flow_paras_in(doc: &Document, p: &Paragraph, tracked: bool) -> Vec<Para> {
mark_ins: None,
mark_del: None,
fmt_change: None,
mark: None,
};
let prev = std::mem::replace(&mut out, next);
if prev.inlines.is_empty() {
@@ -320,6 +322,7 @@ fn flow_table(doc: &Document, t: &Table, depth: usize, tracked: bool) -> FTable
shading: c.props.shading,
borders: c.props.borders,
valign: c.props.valign,
width: None,
});
}
rows.push(cells);
+9
View File
@@ -100,6 +100,12 @@ fn para_props(p: &Para, list_num: Option<u32>) -> ParaProps {
out.indent_left = sp.left;
out.indent_right = sp.right;
out.indent_first = sp.first;
if sp.before_auto {
out.space_before_auto = Some(true);
}
if sp.after_auto {
out.space_after_auto = Some(true);
}
if p.page_break {
out.page_break_before = Some(true);
}
@@ -177,6 +183,9 @@ impl<'a> Builder<'a> {
let old = para_props(&c.old, old_num);
out.props.fmt_change = PropChange::boxed(self.revision(RevisionKind::Format, &c.mark), old);
}
if let Some(f) = &p.mark {
out.mark = self.budget(char_props(f));
}
if p.keep_next {
out.props.keep_next = Some(true);
}
+25 -1
View File
@@ -220,6 +220,9 @@ pub struct Spacing {
pub right: Option<f32>,
/// First-line indent; negative = hanging.
pub first: Option<f32>,
/// HTML's automatic space before / after (see `ParaProps::space_before_auto`).
pub before_auto: bool,
pub after_auto: bool,
}
#[derive(Clone, Debug, Default, PartialEq)]
@@ -244,6 +247,8 @@ pub struct Para {
/// Tracked formatting change of the paragraph: the paragraph's formatting before it (kind,
/// list, alignment, page break, spacing and borders of `old`; its inlines are unused).
pub fmt_change: Option<Box<FmtChange<Para>>>,
/// Formatting of an empty paragraph's mark (what sizes its line), when the source gives it.
pub mark: Option<Box<Fmt>>,
}
impl Para {
@@ -337,11 +342,14 @@ pub struct Cell {
/// The cell's own borders (RTF `\clbrdr…`), drawn whatever the table's style.
pub borders: Option<wordcraft_doc::props::Borders>,
pub valign: Option<VAlign>,
/// The width, points, the source gives a cell spanning one column (HTML): see
/// [`FTable::widths_from_cells`].
pub width: Option<f32>,
}
impl Default for Cell {
fn default() -> Self {
Cell { blocks: Vec::new(), colspan: 1, rowspan: 1, covered: false, header: false, shading: None, borders: None, valign: None }
Cell { blocks: Vec::new(), colspan: 1, rowspan: 1, covered: false, header: false, shading: None, borders: None, valign: None, width: None }
}
}
@@ -372,6 +380,22 @@ impl FTable {
pub fn cols(&self) -> usize {
self.rows.iter().map(|r| r.iter().map(|c| c.colspan.clamp(1, 63) as usize).sum::<usize>()).max().unwrap_or(0)
}
/// Grid column widths from the cells that give one (after [`Self::insert_covered`], so a
/// cell's column counts those spanned from rows above), when every column has one.
pub fn widths_from_cells(&mut self) {
let mut widths = vec![None; self.cols()];
for row in &self.rows {
let mut g = 0;
for c in row {
if let (Some(w), Some(slot)) = (c.width, widths.get_mut(g)) {
slot.get_or_insert(w);
}
g += c.colspan.clamp(1, 63) as usize;
}
}
self.widths = widths.into_iter().collect::<Option<Vec<f32>>>().unwrap_or_default();
}
/// After an importer that gives `rowspan` but no placeholder cells (HTML): insert covered
/// cells into the rows below each spanning cell.
pub fn insert_covered(&mut self) {
+1
View File
@@ -115,6 +115,7 @@ fn parse_full(bytes: &[u8]) -> Result<(Flow, SectionProps), String> {
/// [`parse_full`], plus whether change tracking is on (`None`: the document has no tracked changes).
fn parse_tracked(bytes: &[u8]) -> Result<(Flow, SectionProps, Option<bool>, Option<String>, page_parts::Parts), String> {
wordcraft_doc::zip_guard::check(bytes, crate::MAX_ZIP_ENTRIES).map_err(|e| format!("ODT: {e}"))?;
let mut z = zip::ZipArchive::new(Cursor::new(bytes)).map_err(|e| format!("ODT: not a valid package ({e})"))?;
if let Some(mt) = read_entry(&mut z, "mimetype") {
let mt = String::from_utf8_lossy(&mt);
+9 -1
View File
@@ -261,5 +261,13 @@ fn spacing(st: &St) -> model::Spacing {
} else {
Some(LineSpacing::AtLeast(pt(sl)))
};
model::Spacing { before: gap(st.sb), after: gap(st.sa), line, left: st.li.map(pt), right: st.ri.map(pt), first: st.fi.map(pt) }
model::Spacing {
before: gap(st.sb),
after: gap(st.sa),
line,
left: st.li.map(pt),
right: st.ri.map(pt),
first: st.fi.map(pt),
..Default::default()
}
}
+9
View File
@@ -201,6 +201,7 @@ struct Package<'a> {
impl<'a> Package<'a> {
fn open(bytes: &'a [u8]) -> Result<Self, String> {
wordcraft_doc::zip_guard::check(bytes, crate::MAX_ZIP_ENTRIES)?;
let zip = zip::ZipArchive::new(Cursor::new(bytes)).map_err(|e| format!("not a valid spreadsheet package ({e})"))?;
Ok(Package { zip, total: 0 })
}
@@ -401,6 +402,14 @@ mod tests {
use super::*;
use std::io::Write;
#[test]
fn a_package_declaring_too_many_entries_is_refused() {
// #3660: refused before `zip` reserves and parses them.
let bomb = wordcraft_doc::zip_guard::zip64_entry_count_sample(crate::MAX_ZIP_ENTRIES + 1);
assert!(sheet_names(&bomb).is_err_and(|e| e.contains("declares")));
assert!(crate::odt::parse(&bomb).is_err_and(|e| e.contains("declares")));
}
#[test]
fn negative_zero_formats_as_zero() {
assert_eq!(number(-0.0), "0");
+8
View File
@@ -1011,3 +1011,11 @@ fn html_title_outside_head_is_metadata_not_body() {
assert_eq!(d.core.title, "T", "{html}");
}
}
#[test]
fn latex_export_keeps_repeated_spaces() {
let tex = String::from_utf8(crate::export("tex", &Document::from_text("a b c")).unwrap().unwrap()).unwrap();
assert!(!tex.contains(" "), "{tex}");
let d = crate::import("tex", tex.as_bytes()).unwrap().unwrap();
assert_eq!(para_texts(&d), vec!["a b c".to_string()]);
}
+26 -14
View File
@@ -6,8 +6,8 @@ use super::page::PageFrame;
use super::placed::Placed;
use super::table_flow::below;
use super::{
Anchor, Arc, Block, Blocks, Border, Document, Float, FloatAlign, HashMap, ParaLayout, Paragraph, Path, Rect, SectionProps, StoryRef, TableFloat,
TextDirection, Wrap, WrapText, para, table,
Anchor, Arc, Block, Blocks, Border, Document, Float, FloatAlign, HashMap, ParaLayout, Paragraph, Path, Rect, StoryRef, TableFloat, TextDirection,
Wrap, WrapText, para, table,
};
/// Cell identity is independent of optional table-style formatting.
@@ -281,7 +281,8 @@ pub(crate) fn layout_box(ctx: &mut Ctx, input: BoxInput<'_>) -> BoxLayout {
continue;
}
if let Some(pin) = ctx.pin.as_ref().filter(|pin| in_cell && pin.story == story && float.outside_cell && page_anchored(float)) {
let page = PageFrame { sect: &pin.sect, origin: (0.0, 0.0), number: pin.number, mirror: pin.mirror };
let page =
PageFrame { sect: &pin.sect, margin_top: pin.margin_top, origin: (0.0, 0.0), number: pin.number, mirror: pin.mirror };
pinned.insert(oi, float_rect(Some(page), (0.0, width), para_top, w, h, float));
continue;
}
@@ -460,15 +461,22 @@ fn box_float_table_rect(
_ => None,
}
};
let page_y = |s: &SectionProps| match f.v_rel {
Anchor::Page => Some((0.0, s.page_h)),
Anchor::Margin => Some((s.margin_top, s.text_height())),
_ => None,
let page_y = |fr: &PageFrame| {
let s = fr.sect;
match f.v_rel {
Anchor::Page => Some((0.0, s.page_h)),
Anchor::Margin => {
let mut reference = s.clone();
reference.margin_top = fr.margin_top.unwrap_or(s.margin_top);
Some((reference.margin_top, reference.text_height()))
}
_ => None,
}
};
let h_area = frame.and_then(|fr| page_x(&fr).map(|(a, e)| (a - fr.origin.0, e))).unwrap_or((0.0, width));
// In a cell an offset from the page or margin has nothing to measure from: the table stands
// where the text is.
let (v_area, y_off) = match frame.and_then(|fr| page_y(fr.sect).map(|(a, e)| (a - fr.origin.1, e))) {
let (v_area, y_off) = match frame.and_then(|fr| page_y(&fr).map(|(a, e)| (a - fr.origin.1, e))) {
Some(area) => (area, f.y),
None if matches!(f.v_rel, Anchor::Page | Anchor::Margin) => ((y, 0.0), 0.0),
None => ((y, 0.0), f.y),
@@ -621,15 +629,17 @@ pub(crate) fn push_para(
let left = x + li;
let right = x + width - ri;
let (top, bottom) = (y - e.top - e.up, y + h + e.bottom);
// The outer edges of the side borders.
let sides = pl.rp.borders.unwrap_or_default();
let (lx, rx) = (left - border_room(sides.left), right + border_room(sides.right));
if let Some(c) = pl.rp.shading {
let pad = pl.rp.borders.as_ref().map(|b| b.left.map(|l| l.space).unwrap_or(4.0)).unwrap_or(0.0);
items.push(Placed::Fill { rect: Rect::new(left - pad, top, right - left + pad * 2.0, bottom - top), color: c });
let (sl, sr) = (lx.min(left - pad), rx.max(right + pad));
items.push(Placed::Fill { rect: Rect::new(sl, top, sr - sl, bottom - top), color: c });
}
items.push(Placed::Lines { story, path: Path(path.to_vec()), para: pl.clone(), l0, l1, x, y, turn: TextDirection::Horizontal });
if let Some(b) = &pl.rp.borders {
let sp = |o: &Option<Border>| o.map(|b| b.space).unwrap_or(0.0);
let (lx, rx) = (left - sp(&b.left), right + sp(&b.right));
// The top and bottom lines lie beyond their distance from the text.
// All four lines lie beyond their distance from the text; top and bottom reach the corners.
if let Some(t) = b.top.filter(|t| e.top > 0.0 && t.is_visible()) {
let ty = y - t.space - t.width / 2.0;
items.push(Placed::Rule { x0: lx, y0: ty, x1: rx, y1: ty, border: t });
@@ -639,10 +649,12 @@ pub(crate) fn push_para(
items.push(Placed::Rule { x0: lx, y0: by, x1: rx, y1: by, border: t });
}
if let Some(t) = b.left.filter(Border::is_visible) {
items.push(Placed::Rule { x0: lx, y0: top, x1: lx, y1: bottom, border: t });
let bx = lx + t.width / 2.0;
items.push(Placed::Rule { x0: bx, y0: top, x1: bx, y1: bottom, border: t });
}
if let Some(t) = b.right.filter(Border::is_visible) {
items.push(Placed::Rule { x0: rx, y0: top, x1: rx, y1: bottom, border: t });
let bx = rx - t.width / 2.0;
items.push(Placed::Rule { x0: bx, y0: top, x1: bx, y1: bottom, border: t });
}
}
y + h
+11 -1
View File
@@ -89,6 +89,8 @@ struct ParaEnvInput<'a> {
pub(crate) struct Pin {
pub(crate) story: StoryRef,
pub(crate) sect: SectionProps,
/// Rendered header Margin boundary; absent in footers and height probes.
pub(crate) margin_top: Option<f32>,
pub(crate) number: u32,
pub(crate) mirror: bool,
pub(crate) behind: Vec<Placed>,
@@ -108,7 +110,14 @@ pub(crate) fn page_anchored(f: &Float) -> bool {
/// Run `f`, a layout of header or footer `id` for page `number` of `sect`, with the floats in its
/// table cells that are placed on the page put there (see [`Pin`]). Returns `f`'s result and those
/// floats' items, in page coordinates: behind the header's text, and in front.
pub(crate) fn with_pin<T>(ctx: &mut Ctx, id: u32, sect: &SectionProps, number: u32, f: impl FnOnce(&mut Ctx) -> T) -> (T, Vec<Placed>, Vec<Placed>) {
pub(crate) fn with_pin<T>(
ctx: &mut Ctx,
id: u32,
sect: &SectionProps,
number: u32,
margin_top: Option<f32>,
f: impl FnOnce(&mut Ctx) -> T,
) -> (T, Vec<Placed>, Vec<Placed>) {
let tables = ctx.doc.parts.get(&id).is_some_and(|p| p.blocks.iter().any(|b| matches!(**b, Block::Table(_))));
if !tables {
return (f(ctx), Vec::new(), Vec::new());
@@ -116,6 +125,7 @@ pub(crate) fn with_pin<T>(ctx: &mut Ctx, id: u32, sect: &SectionProps, number: u
let pin = Pin {
story: StoryRef::Part(id),
sect: sect.clone(),
margin_top,
number,
mirror: ctx.mirror,
behind: Vec::new(),
+2
View File
@@ -23,6 +23,8 @@ mod paint;
pub(crate) use glyphs::author_order;
pub use glyphs::{CHECK_BOX_INSET, check_box, text_color, upright_in_vertical};
#[cfg(test)]
pub(crate) use lines::{leader_pitch, leader_xs};
pub use lines::{mark_text_width, section_break_label};
pub(crate) use objects::inline_rect;
pub use objects::seg_path;

Some files were not shown because too many files have changed in this diff Show More