Compare commits

...
64 Commits
Author SHA1 Message Date
FredApps 6ef13b2cc8 Merge pull request #4209 from marcoarnulfo/fix/save-as-document-folder
Save As starts in the folder of the open file
2026-11-18 22:16:38 +07:00
FredApps 6f288b926e Tests: make file-dialog scratch directories absolute
If TMPDIR is relative, scratch() used to return a relative path, but Save As
sends an absolute folder. Make the scratch path absolute so the assertions match.
2026-11-18 21:40:42 +07:00
FredApps 9ba48e0032 Tests: unsaved Save As still has no folder, and uses the first words as its name
#5793 landed on main after this PR: a new document's Save As name is its first
words, not Document1.docx. The folder is still none, which is what #4143 tests.
2026-11-18 21:10:53 +07:00
FredApps 8834e602e3 Merge remote-tracking branch 'origin/main' into fix/save-as-document-folder 2026-11-18 21:08:11 +07:00
FredApps 64683a7d0b Merge pull request #5912 from batistalucasv/fix/5886-wayland-resize
Wayland: 600 pt minimum width for GNOME split window tiling (#5886)
2026-11-18 21:05:40 +07:00
FredApps bce64c0eb9 Compare Documents: name the unlabeled Original, Revised and author fields
The Compare dialog's "Another file" radio, Revised path box and author
box had no accessible name. After the status bar's two-pass layout,
Harness::run settles on a fully laid-out dialog and the accessibility
check failed. Existing catalog strings only; no layout change.
2026-11-18 20:20:34 +07:00
FredApps 497d658574 Merge remote-tracking branch 'origin/main' into fix/5886-wayland-resize 2026-11-18 19:34:59 +07:00
squalorso 4e8c9d5531 #3172: the Paste button's name has no macOS key on Windows and Linux
Home › Paste is named "Paste" for screen readers. Its hint adds the key as the platform writes it, so the name no longer carries macOS's ⌘V on Windows and Linux. The status bar's Go To hint uses the platform's key too. A menu button's KeyTip is looked up from the English tip, so it still appears in other interface languages.

See the pull request for the discussion.
2026-11-18 19:32:22 +07:00
squalorso 3ab2e53a89 Screen readers: headings and list items come one level too deep (#3173)
Headings and list items now use AccessKit's zero-based level, so a Heading 1 reaches UI Automation as level 1 and a first-level list item as level 1. They previously came one level too deep.

See the pull request for the discussion.
2026-11-18 18:57:56 +07:00
Bryan Mark 07c0a89396 IME: the composition is drawn in the text's font and size, and CJK in it no longer shows boxes
The IME composition is drawn in the typing style's face, size and baseline, with size capped from the font atlas. CJK in the composition no longer shows boxes. Installing fonts in one window no longer drops another window's faces. This does not change the IME popup position.

See the pull request for the discussion.
2026-11-18 18:16:08 +07:00
FredApps c1e2e6ba38 Merge pull request #4964 from tsw2k/fix/table-narrow-column-floor
fix(engine): a drag between narrow columns leaves them as they are
2026-11-18 17:35:02 +07:00
FredApps a0e14f3b5f Merge remote-tracking branch 'origin/main' into pr-4964 2026-11-18 17:10:08 +07:00
FredApps 3ff3f39727 Merge pull request #4219 from ovezthaking/fix/builtin-styles-bcs-ics-4198
fix(styles): pair w:bCs and w:iCs with w:b and w:i in built-in styles (#4198)
2026-11-18 17:09:47 +07:00
FredApps 705f2bee27 Merge remote-tracking branch 'origin/main' into fix/table-narrow-column-floor 2026-11-18 16:57:19 +07:00
FredApps be56b5f1bb Merge remote-tracking branch 'origin/main' into fix/builtin-styles-bcs-ics-4198 2026-11-18 16:35:11 +07:00
naxvog 7831216b2c Borders: lines meet in closed corners, as in Word
Border lines meet in closed corners, as in Word, including after the side-border geometry from #5036.

Originally authored by @naxvog in #1757.
2026-11-18 16:33:13 +07:00
NetterSilver 73b999ec89 Say in the placeholder when a picture is too large to decode
When a picture is too large to decode, the placeholder now says so, instead of a plain broken box.

Originally authored by @Silvertree2010 in #5638.
2026-11-18 16:30:16 +07:00
pakiat 1e1bd7e002 Fields: PDFs and prints show body PAGEREF pages, as Word fills them in
PDFs and prints show body PAGEREF pages as Word fills them in, including PAGEREF in text boxes and notes, with a second pass once the pages are known.

Originally authored by @pakiat in #5129.
2026-11-18 16:29:51 +07:00
pakiat 5bf3ffb920 Tables: percentage cell widths size their columns whichever rows give them
Percentage cell widths size their columns from whichever rows give them, taking the largest when rows disagree, sharing a merged cell's percentage, and keeping a grid that already matches.

Originally authored by @pakiat in #5074.
2026-11-18 16:29:27 +07:00
pakiat 32b4ab5724 Paragraph dialog: Special box sets First line and Hanging indents
The Paragraph dialog's Special box sets First line and Hanging indents, keeps By when the kind changes, leaves a mixed selection blank, and limits By to 0–22 inches.

Originally authored by @pakiat in #4922.
2026-11-18 16:29:00 +07:00
Thân Minh Nghĩa c3be9f094f #2702: a missing font is drawn in its font table's altName, as Word does
A missing font is drawn in its font table altName, as Word does. The DOCX reader half lives in craft-libs#1426; this is the fonts, layout and settings half.

Originally authored by @Nghiathan13 in #2711.
2026-11-18 16:28:30 +07:00
Bryan Mark c3fe680a0f Fonts: fallback no longer keeps unneeded font families in memory
Font fallback records the file stamp with the first coverage read, so a rescan does not keep loading families that are no longer needed, and a replaced file still invalidates the cache.

Originally authored by @arcanii in #2345.
2026-11-18 16:28:07 +07:00
FredApps b0d3165ede Merge pull request #5096 from dfgodvan/fix/no-break-in-url-query
Layout: no line break after the '?' of a query, nor after a slash before an underscore
2026-11-18 16:24:30 +07:00
batistalucasv 15d315e3f8 Wayland: 600 pt minimum width for GNOME split window tiling (#5886) 2026-11-17 21:49:07 -03:00
Martin BallandClaude Opus 5.5 22a73d9f2d #778: progress log: file formats, codecs and protocols come from craft-libs (#6962)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 21:31:05 +00:00
dfd51a3cdc #778: land the picture-probe and MCP cutovers (#6955, #6956) on main (#6960)
* #778: take craft-libs integration/wordcraft-778 at beeea924

Every craft-libs git dependency (workspace, ui-egui, wordcraft, wordcraft-web,
krilla patch) moves from 453b0da782 to beeea92451 (craft-libs main 0649a07aa
plus open PRs #2061-#2064). Cargo.lock: only craft-libs sources move, plus
windows-sys edge changes inside craft-libs crates; no crates.io versions change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: docx stacking-order test follows the shared reader's one-more z

craft-libs 5f453efa9 (craft-docx: a relativeHeight of 0 stays the lowest
stacking order) reads Float::z as one more than the file's relativeHeight and
writes stored orders one less, so a file's 0 is no longer "none". The test
expected the old values; relative order and the round trip are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: take craft-libs integration/wordcraft-778 at 24a15211

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* deps: pin craft-libs to the merged shared cutover prerequisites

* #778: pin craft-libs main at 93dac78c (with #2061-#2064)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: picture probes use craft-image-codecs (DIB wrapping, JPEG budget, PNG/WebP) (#6955)

* #778: picture: wrap clipboard DIBs with craft-image-codecs' bmp_probe

probe_dib's local dib_to_bmp and dib_layout are replaced by
craft_image_codecs::bmp_probe::{dib_to_bmp, dib_layout}. The shared copies are
the same code; the only change is that they reserve their output fallibly. A
failed reservation in dib_to_bmp (the BMP's bytes can't be allocated) is
Refusal::TooLarge with the size the DIB header declares, the refusal a picture
too large to hold already gets; dib_layout leaves the DIB unchanged then.

The local dib_to_bmp name stays as a thin wrapper returning Result<_, Refusal>.
The old code is kept in the tests as an oracle: wrapping and layout are
byte-equal on thousands of crafted DIBs (core, info, V4, V5 headers, bit
fields, alpha with BI_RGB, every truncation), and probe_dib agrees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: picture: budget JPEG coefficients with craft-image-codecs

probe's JPEG held-coefficient estimate (jpeg_coefficients, frame_coefficients,
worst_coefficients) is replaced by craft_image_codecs::jpeg_worst_case_coefficients,
which returns the larger of that same estimate and the shared held_coefficients.
So the charge can only grow, never shrink.

It grows for: a SOF0/SOF1 8-bit first frame whose first scan has fewer
components than the frame (the decoder holds the coefficients until the other
scans); and a first-frame SOF2 with sampling factors above 4 (the old rule
clamped them to 4). A real baseline JPEG with all components in its first scan
is charged nothing, as before. With the 512 MiB budget, a 4:4:4 non-interleaved
baseline JPEG is now refused from about 59.6 Mpx instead of 67 Mpx; 4:2:0 and
4:2:2 stay under the RGBA8 conversion and are unchanged.

The old code is kept in the tests as an oracle: the new charge is never less
(crafted headers, every prefix, random damage), equal where the shared rule adds
nothing, and one test pins a case where it grows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: picture: read PNG and WebP probes with craft-image-codecs

The local png_chunks header read is replaced by
craft_image_codecs::png_probe::pixel_model, and the VP8X alpha check in probe
by webp_probe::has_alpha (an error reads as no alpha, as non-RIFF input did).

PNG: for every valid colour type and bit depth, with and without tRNS and an
iCCP chunk, the size, bytes a pixel and alpha are what the old read gave
(PixelModel::bytes_per_pixel is channels * 2 for 16 bits, else channels, with a
palette as RGB and tRNS adding a channel to grey, RGB and palette). pixel_model
is stricter, so these are now refused (probe: Refusal::Broken, dimensions:
None) where the old read measured them: an invalid colour type and bit depth
pair, a zero or past-i32::MAX side, and more than 65,536 chunks before the
image data. None is a valid PNG.

WebP: a bare lossless (VP8L) picture whose header sets alpha_is_used now reports
alpha; the old rule only knew the VP8X flag.

The old PNG read and WebP rule are kept in the tests as oracles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: picture tests follow craft-image-codecs' DIB fixes

The shared dib_to_bmp no longer adds 12 appended mask bytes to the pixel
offset of 52/56-byte BI_BITFIELDS headers, and dib_layout treats the bytes
after the header as a mask copy only when the payload is longer than the
pixel rows need. The differential test excludes exactly those inputs by
named predicate, asserts the new output for them, and stays byte-equal
elsewhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: dib_header_dims uses the shared fallible dib_to_bmp

Calls the shared probe directly so a failed allocation cannot recurse through the refusal wrapper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* #778: WordCraft's MCP stdio server runs on craft-mcp-server (#6956)

The JSON-RPC framing, bounded line reading, initialize, ping, tools/* and
resources/* now come from craft_mcp_server::Server; WordCraft keeps its tools,
resources, instructions and backends. The old server.rs stays as a test-only
oracle for a differential test. click/key schemas list every argument the app
accepts, since the shared check refuses unlisted ones.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-17 22:19:16 +01:00
c39b0e5755 doc: cut legacy encoding tables over to craft-codepage (#6785)
* doc: reuse shared legacy codepage tables with parity tests

* Regenerate codepage dependency ordering in Cargo lockfile

* #778: Cargo.lock: wordcraft-doc depends on craft-codepage after merging main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Martin Ball <martin.ball@ucscaregroup.co.uk>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 22:19:05 +01:00
d2bfcd4d16 #778: speech: use craft-ssip for the SSIP wire helpers (#6951)
Message encoding, reply framing and final-code parsing now come from the shared crate;
socket discovery, timings, conversation and events stay here.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-17 22:18:55 +01:00
Martin BallandClaude Opus 5.5 1531007d5e #778: plain-text detection uses craft-mime detect (#6950)
The local is_text/decode/legacy copies in formats::txt are replaced by a re-export of
craft_mime::detect. The old code stays as a test oracle, with differential and random-buffer
tests showing identical output. chardetng stays only as a dev-dependency of that oracle.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:47:52 -05:00
30584c81ad #778: pin craft-libs main with the shared cutover prerequisites (#6954)
* #778: take craft-libs integration/wordcraft-778 at beeea924

Every craft-libs git dependency (workspace, ui-egui, wordcraft, wordcraft-web,
krilla patch) moves from 453b0da782 to beeea92451 (craft-libs main 0649a07aa
plus open PRs #2061-#2064). Cargo.lock: only craft-libs sources move, plus
windows-sys edge changes inside craft-libs crates; no crates.io versions change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: docx stacking-order test follows the shared reader's one-more z

craft-libs 5f453efa9 (craft-docx: a relativeHeight of 0 stays the lowest
stacking order) reads Float::z as one more than the file's relativeHeight and
writes stored orders one less, so a file's 0 is no longer "none". The test
expected the old values; relative order and the round trip are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: take craft-libs integration/wordcraft-778 at 24a15211

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* deps: pin craft-libs to the merged shared cutover prerequisites

* #778: pin craft-libs main at 93dac78c (with #2061-#2064)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-17 15:45:46 -05:00
Aleksander HelgakerandClaude Fable 5.2 8331f2e68d Fonts: tell where a missing font can be downloaded (Get the Missing Fonts) (#6826)
* Fonts: tell where a missing font can be downloaded (Get the Missing Fonts)

A document opened from a file that is missing fonts their publisher gives
away (Microsoft's Aptos family, Cascadia Code and Mono) brings up a box
naming each font, the font drawn instead, the publisher's download page and
how to install the download. Once per document; never for a new document or
one a script opened. "Don't show this again" turns it off (File > Options >
General turns it back on); the status bar's missing-fonts menu opens it by
hand (ui.dialog fontDownload).

font.missing reports the same page as "download", so agents get it without
the box. WordCraft links to the page only: nothing is downloaded or bundled.

The desktop and web apps switch the box on at start-up
(tell_about_font_downloads); tests and ui_shot don't, so what they show
doesn't depend on the fonts installed where they run.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Get the Missing Fonts: the box with nothing to download is one row tall

Its Close button's far-side layout took the rest of the window's height.
The button row is now a row of its own, shared by both forms of the box.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Floating windows: a flat title bar and room around the content

Dialogs, the Read Aloud player and the built-in file browser had a grey
band behind a centred title and 6 pt margins. The title bar is now the
window's own colour with no band or line under it, and the title and the
content have 18 pt of room. The close box follows the new title bar height.

egui fills the front window's title bar with the colour of an open menu
button, so show_window makes that colour the window's while the title bar
is drawn and puts it back before the content is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Get the Missing Fonts: a headline, a card to each download, numbered steps

The box was a plain table and a paragraph of steps. It now opens with the
warning mark and how many fonts are missing; lists each download once
(Aptos and Aptos Display come in one) with the fonts drawn instead, the
publisher and a Download button that opens the publisher's page; and gives
the two steps that add the download, with Add Font… beside its step.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Fonts: the download pages are a JSON list (font_downloads.json)

The list of fonts their publisher gives away, and each publisher's page,
was a table in the code. It is now crates/fonts/src/font_downloads.json,
built into the app: adding a font is a new entry there. Entries without a
family, a publisher or an https page are left out, and a list that isn't
JSON has no downloads rather than failing.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Get the Missing Fonts: licence notice, and fonts offered with limits

The box says that fonts come with licence terms and that following the
publisher's terms is up to the user. A font whose publisher limits who may
use it is marked "restricted" in font_downloads.json, says so on its card
and in font.missing ("restricted"), and has a card of its own.

The list gains Selawik (Microsoft's open-source font) and, marked
restricted, Segoe UI Variable, Segoe Fluent Icons and Segoe MDL2 Assets
from Microsoft's Windows design downloads page. Plain Segoe UI is not
offered for download and is not matched.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Get the Missing Fonts: the downloads scroll in a short window

With many downloads in a short window the box ran past the bottom of the
screen, leaving the steps, Add Font…, Don't show this again and Close out
of reach. The cards now scroll in the room the window has left, between
96 and 320 pt, and the rest of the box stays in view.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Get the Missing Fonts: all above the Close row scrolls in a short window

Only the download cards scrolled, within fixed limits, so in a window
shorter than about 480 pt the opt-out and Close could still fall off
screen. Everything above the row with Don't show this again and Close now
scrolls in the room the window has left, and that row stays on screen.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Floating windows: the flat title bar and wider margins leave this change

Reverts 2d39c54e and its revision-history row. The restyle changes every
dialog, the Read Aloud player and the file browser, and is to be reviewed
on its own.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Get the Missing Fonts: the headline counts the fonts that can be downloaded

The box lists only the missing fonts with a download, so "1 font in this
document isn't installed" was wrong for a document missing others too. It
now says "1 missing font can be downloaded", and the line under it no longer
mixes one and many.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-11-17 21:05:13 +01:00
Martin BallandClaude Opus 5.5 1d71d904d2 #778: a clipboard DIB's size is read by craft-image-codecs' BMP probe (#6949)
`dib_header_dims` read a DIB header's width and height by hand. It now
wraps the header's first 12 bytes as a BMP and asks
`craft_image_codecs::bmp_probe::dimensions`. Header sizes other than 12
and 40 or more are still refused here, since the probe reads them as
info headers. The old reader stays as a test oracle.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 14:47:31 -05:00
Martin BallandClaude Opus 5.5 e28c25639b #778: the ZIP entry-count guard delegates to craft-zip (#6948)
* #778: zip_guard delegates to craft-zip's entry-count guard

wordcraft_doc::zip_guard::check and zip64_entry_count_sample now call craft-zip, which also
treats a central-directory size of u32::MAX as a ZIP64 marker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: Cargo.lock: wordcraft-doc depends on craft-zip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: zip_guard: craft-zip test-support as a dev-dependency

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 14:47:27 -05:00
Martin BallandClaude Opus 5.5 7b3c2efb1f CI: skip every performance test under coverage instrumentation (#6873)
SonarCloud's coverage shards run tests built with -Cinstrument-coverage
on shared runners. Tests that assert a wall-clock limit or a time
ratio swing there: the megabyte-word test took 5.7-23 s against a
15 s limit, and three linear-time tests read 24-31x against 24x. Each
one that flakes turns main red after merging.

Ignore under cfg(coverage) every test whose purpose is performance
(linear time, bounded, capped, fast). The gate CI still runs them all
uninstrumented. Behaviour tests that only carry a generous time guard
(fuzz, never-panics, clipboard, regex, render, recent fonts) and the
SSIP speech timeout tests keep running, so their checks stay covered.


Claude-Session: https://claude.ai/code/session_01FypgCXtSjB6e5jmy9Uq7u8

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 20:29:14 +01:00
Michael MuandClaude Sonnet 5.5 8b212e6665 KeyTips: a compact button's badge no longer covers its icon (#4497) (#4503)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-17 16:42:25 +01:00
Diego GueriniandClaude Opus 5.5 26b3ae2cc2 inspect_document marks the page fields of headers and footers (#4467) (#4474)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 16:26:33 +01:00
Martin BallandClaude Opus 5.5 e784a2aaed Tests: review.language's no-proofing case uses zxx, not fr-FR (#6944)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 16:12:46 +01:00
pakiatandClaude Opus 5.5 65212d6856 Shapes: lines move by their body and resize by their ends (#4393)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:59:23 +01:00
Thân Minh Nghĩa 5c8a6e1d75 #3917: Text Outline and Shape Outline › Weight show a line sample for each weight (#3933) 2026-11-17 15:45:51 +01:00
Thân Minh NghĩaandClaude Opus 5.5 83f18db12d #2786: Home › Bullets: Define New Bullet… dialog and Change List Level › (#2860)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:29:27 +01:00
Martin Ball d4b5c68833 Merge branch 'main' into fix/no-break-in-url-query 2026-11-17 09:24:47 +01:00
Martin Ball b69dc6a34f Merge branch 'main' into fix/no-break-in-url-query 2026-11-17 05:36:10 +01:00
tsw2k 210fe76f0c fix(engine): a drag between narrow columns leaves them as they are
A column a file gave below the 6 pt drag minimum was forced up to it by
any drag of its border, even one of no distance: columns of 3 pt and 4 pt
became 6 pt and 1 pt, and a column of a point or less is one import reads
as no width, giving the whole table equal columns. A column already under
the minimum is no longer narrowed further, nor widened by a drag with no
room for it. Whole-column and selected-row drags both go through
share_out.

The import part of this change (keeping columns over 1 pt and under 6 pt)
moved to craft-libs with RTF and ODT import (storytold/craft-libs#1865).
2026-11-16 11:47:53 +00:00
dfgodvan 6574fedb87 Line breaks: a question mark keeps its break in CJK text, a path after a space wraps whole
Review follow-up. The break after '?' is dropped only before an ASCII letter,
digit or underscore, so a '?' in CJK text keeps the break UAX #14 gives it.
The break before a slash that starts a word now also takes '/_': with the
break after '/' before '_' gone, "caption /_back" had no break left there
and split mid-word when it did not fit.
2026-11-14 19:15:17 +03:00
dfgodvan 26b805a4b0 Line breaks: no break after the '?' of a query, nor after '/' before '_'
Word keeps "/portal/_back/?type=..." whole: a web address goes to the next line whole, and
a long one breaks only where it no longer fits. UAX #14 allows a break after the '?' of a
query and after a slash before an underscore, and the slash rule that keeps "and/or" whole
covered only a letter or digit after the slash.
2026-11-14 19:15:04 +03:00
Oliwer Urbaniak 329ba1640e Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198 2026-11-11 20:53:20 +01:00
Oliwer Urbaniak e6185e6dcf Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198 2026-11-04 03:44:32 +01:00
Oliwer Urbaniak a7254fb5a5 Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198 2026-11-02 00:57:42 +01:00
Oliwer Urbaniak a54093f6f8 Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198
# Conflicts:
#	ROADMAP.md
#	contributors/people.toml
#	docs/file-format-parity.md
2026-10-30 17:55:35 +01:00
Marco Arnulfo 1113013e87 Save As starts in the working folder for a file opened by a relative path (#4143) 2026-10-30 04:41:52 +01:00
Oliwer Urbaniak 0bcf52115b Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198
# Conflicts:
#	ROADMAP.md
2026-10-30 02:55:23 +01:00
Oliwer Urbaniak 33968d3b81 Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198
# Conflicts:
#	crates/ui-egui/src/dialogs/dispatch.rs
2026-10-28 22:14:15 +01:00
Oliwer Urbaniak c8631193ad Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198
# Conflicts:
#	crates/engine/src/cmd/para.rs
#	crates/ui-egui/src/dialogs.rs
2026-10-27 02:54:02 +01:00
Oliwer Urbaniak 69dc7928fd Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198
# Conflicts:
#	ROADMAP.md
2026-10-26 18:38:37 +01:00
Oliwer Urbaniak 93c200779f Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198
# Conflicts:
#	ROADMAP.md
2026-10-25 18:43:31 +01:00
Oliwer Urbaniak baca390e6c contributors: add ovezthaking 2026-10-25 18:00:20 +01:00
Oliwer Urbaniak b86fd70cdd Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198
# Conflicts:
#	ROADMAP.md
#	docs/file-format-parity.md
2026-10-25 16:59:33 +01:00
Oliwer Urbaniak ca36c75433 Merge remote-tracking branch 'upstream/main' into fix/builtin-styles-bcs-ics-4198
# Conflicts:
#	docs/file-format-parity.md
2026-10-25 16:39:06 +01:00
Oliwer Urbaniak b4d15331f1 fix(styles): seed style chain fold with document default_chr in modify_style 2026-10-25 02:11:08 +02:00
Oliwer Urbaniak 0f41c6ca54 fix(styles): omit unchanged bold/italic in Modify Style dialog and respect inherited cs overrides 2026-10-25 02:03:32 +02:00
Oliwer Urbaniak baf9840472 docs: add docstrings to touched style functions and tests 2026-10-25 01:54:11 +02:00
Oliwer Urbaniak 776c9a91ca fix(styles): preserve distinct complex-script overrides in styles.modify 2026-10-25 01:48:11 +02:00
Oliwer Urbaniak cbf93700e6 fix(styles): pair w:bCs and w:iCs with w:b and w:i in built-in styles (#4198) 2026-10-25 01:30:42 +02:00
Marco Arnulfo d414891cc0 Save As starts in the folder of the open file (fixes #4143) 2026-10-24 23:47:12 +02:00
148 changed files with 8677 additions and 1587 deletions
Generated
+60 -42
View File
@@ -1060,7 +1060,7 @@ dependencies = [
[[package]]
name = "craft-atomic"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"rustix 1.1.5",
]
@@ -1068,17 +1068,17 @@ dependencies = [
[[package]]
name = "craft-cfb"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-codepage"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-dds"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"rayon",
]
@@ -1086,7 +1086,7 @@ dependencies = [
[[package]]
name = "craft-doc-binary"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"chrono",
"craft-cfb",
@@ -1102,7 +1102,7 @@ dependencies = [
[[package]]
name = "craft-docx"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"base64 0.23.1",
"chrono",
@@ -1122,7 +1122,7 @@ dependencies = [
[[package]]
name = "craft-drawingml"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-xml",
"kurbo",
@@ -1133,7 +1133,7 @@ dependencies = [
[[package]]
name = "craft-flow"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-wordmodel",
"image",
@@ -1143,7 +1143,7 @@ dependencies = [
[[package]]
name = "craft-font-catalog"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"fontique",
"harfrust 0.13.3",
@@ -1158,7 +1158,7 @@ dependencies = [
[[package]]
name = "craft-font-store"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"cap-primitives",
"craft-atomic",
@@ -1179,12 +1179,12 @@ dependencies = [
[[package]]
name = "craft-fonts-build"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-html"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-codepage",
"craft-flow",
@@ -1197,7 +1197,7 @@ dependencies = [
[[package]]
name = "craft-i18n"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"fixed_decimal",
"icu_decimal",
@@ -1220,7 +1220,7 @@ dependencies = [
[[package]]
name = "craft-image-codecs"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-dds",
"craft-jpeg",
@@ -1247,7 +1247,7 @@ dependencies = [
[[package]]
name = "craft-jpeg"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"thiserror 2.0.21",
]
@@ -1255,7 +1255,7 @@ dependencies = [
[[package]]
name = "craft-latex"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-flow",
"craft-math",
@@ -1266,7 +1266,7 @@ dependencies = [
[[package]]
name = "craft-markdown"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-flow",
"craft-html",
@@ -1276,16 +1276,24 @@ dependencies = [
[[package]]
name = "craft-math"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-xml",
"serde",
]
[[package]]
name = "craft-mcp-server"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"serde_json",
]
[[package]]
name = "craft-metadata"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"brotli-decompressor 6.3.1",
"chrono",
@@ -1297,7 +1305,7 @@ dependencies = [
[[package]]
name = "craft-metafile"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"encoding_rs",
"image",
@@ -1308,7 +1316,7 @@ dependencies = [
[[package]]
name = "craft-mime"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"chardetng",
"chrono",
@@ -1319,7 +1327,7 @@ dependencies = [
[[package]]
name = "craft-odf"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"aes",
"base64 0.23.1",
@@ -1345,7 +1353,7 @@ dependencies = [
[[package]]
name = "craft-ods"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-odf",
"craft-sheet",
@@ -1356,7 +1364,7 @@ dependencies = [
[[package]]
name = "craft-offcrypto"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"aes",
"base64 0.23.1",
@@ -1373,7 +1381,7 @@ dependencies = [
[[package]]
name = "craft-opc"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-xml",
"craft-zip",
@@ -1384,7 +1392,7 @@ dependencies = [
[[package]]
name = "craft-pdf"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"krilla",
"thiserror 2.0.21",
@@ -1393,7 +1401,7 @@ dependencies = [
[[package]]
name = "craft-print"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-winprint-gdi",
"log",
@@ -1407,7 +1415,7 @@ dependencies = [
[[package]]
name = "craft-proof"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"log",
"miniz_oxide 0.9.1",
@@ -1418,7 +1426,7 @@ dependencies = [
[[package]]
name = "craft-rtf"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-codepage",
"craft-flow",
@@ -1431,12 +1439,17 @@ dependencies = [
[[package]]
name = "craft-sheet"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-ssip"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-tiff"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"serde",
"thiserror 2.0.21",
@@ -1445,7 +1458,7 @@ dependencies = [
[[package]]
name = "craft-ui"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-fonts-build",
"craft-i18n",
@@ -1465,7 +1478,7 @@ dependencies = [
[[package]]
name = "craft-update"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"chrono",
"dunce",
@@ -1492,7 +1505,7 @@ dependencies = [
[[package]]
name = "craft-web-api"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"js-sys",
"serde_json",
@@ -1503,7 +1516,7 @@ dependencies = [
[[package]]
name = "craft-web-host"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"wasm-bindgen",
"web-sys",
@@ -1512,7 +1525,7 @@ dependencies = [
[[package]]
name = "craft-window"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"egui",
"objc2 0.6.5",
@@ -1531,7 +1544,7 @@ dependencies = [
[[package]]
name = "craft-wordmodel"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"base64 0.23.1",
"craft-math",
@@ -1545,7 +1558,7 @@ dependencies = [
[[package]]
name = "craft-xls"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-cfb",
"craft-sheet",
@@ -1558,7 +1571,7 @@ dependencies = [
[[package]]
name = "craft-xlsx"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-drawingml",
"craft-opc",
@@ -1570,7 +1583,7 @@ dependencies = [
[[package]]
name = "craft-xml"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"quick-xml 0.41.0",
"serde",
@@ -1580,7 +1593,7 @@ dependencies = [
[[package]]
name = "craft-zip"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"flate2",
"thiserror 2.0.21",
@@ -3416,7 +3429,7 @@ dependencies = [
[[package]]
name = "krilla"
version = "0.8.2"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"base64 0.22.1",
"bumpalo",
@@ -7385,6 +7398,8 @@ name = "wordcraft-doc"
version = "0.18.1"
dependencies = [
"base64 0.23.1",
"craft-codepage",
"craft-zip",
"log",
"proptest",
"serde",
@@ -7435,6 +7450,7 @@ dependencies = [
"craft-font-store",
"craft-mime",
"craft-proof",
"craft-ssip",
"craft-zip",
"encoding_rs",
"getrandom 0.3.4",
@@ -7468,6 +7484,7 @@ dependencies = [
"harfrust 0.13.3",
"kurbo",
"log",
"serde_json",
"skrifa 0.47.0",
"unicode-bidi",
"unicode-script",
@@ -7538,6 +7555,7 @@ dependencies = [
name = "wordcraft-mcp"
version = "0.18.1"
dependencies = [
"craft-mcp-server",
"log",
"serde_json",
"wordcraft-control-key",
+33 -29
View File
@@ -24,19 +24,21 @@ wordcraft-pdf = { path = "crates/pdf" }
# Proofing is craft-libs' craft-proof (it was crates/proof); its Language data is craft-i18n's
# language-data/, embedded from CRAFT_I18N_DIR (craft-i18n.rev). The package keeps the
# `wordcraft-proof` name here so `wordcraft_proof::` paths are unchanged.
# craft-proof is storytold/craft-libs main at 453b0da782. That commit has #590 (ca-ES-valencia
# craft-proof is storytold/craft-libs main at 93dac78c. That tree has #590 (ca-ES-valencia
# reads the ca-valencia pack), #589 (French elision suggestions first), #497 (flag-based
# compounding), #496 (review follow-ups, en-AU) and the merged #482–#490/#495 stack; the font
# store's `language-pack` kind is #486 (CP4.1).
wordcraft-proof = { package = "craft-proof", git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-font-store = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-font-catalog = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
wordcraft-proof = { package = "craft-proof", git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-font-store = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-font-catalog = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-ssip = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
wordcraft-engine = { path = "crates/engine" }
wordcraft-mcp = { path = "crates/mcp" }
wordcraft-zotero = { path = "crates/zotero" }
wordcraft-control-key = { path = "crates/control-key" }
wordcraft-ui-egui = { path = "crates/ui-egui" }
craft-metafile = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-codepage = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-metafile = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
# foundation
kurbo = { version = "0.13", features = ["serde"] }
@@ -67,35 +69,37 @@ wuff = "0.2.9"
brotli-decompressor = "5"
vello_cpu = "0.2"
image = { version = "0.25", default-features = false }
craft-image-codecs = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-tiff = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-pdf = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", default-features = false, features = ["write"] }
craft-image-codecs = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-tiff = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-pdf = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", default-features = false, features = ["write"] }
# craft-update: app-specific pretend versions apply to startup and manual checks.
craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", features = ["egui"] }
craft-wordmodel = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-docx = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-rtf = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", features = ["doc"] }
craft-doc-binary = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-flow = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-odf = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", features = ["text"] }
craft-markdown = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-html = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", features = ["flow"] }
craft-latex = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-math = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-mime = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", features = ["detect"] }
craft-sheet = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-ods = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-xls = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-xlsx = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-xml = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-zip = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", features = ["egui"] }
craft-wordmodel = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-docx = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-mcp-server = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-rtf = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", features = ["doc"] }
craft-doc-binary = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-flow = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-odf = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", features = ["text"] }
craft-markdown = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-html = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", features = ["flow"] }
craft-latex = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-math = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-mime = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", features = ["detect"] }
craft-sheet = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-ods = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-xls = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-xlsx = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-xml = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-zip = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
# zlib streams (compressed metafile pictures in `.doc` drawings); already in the lock as a dependency of other crates.
miniz_oxide = "0.9"
cfb = "0.15"
# Legacy text encodings (GBK/GB18030, Big5, Shift_JIS, EUC-KR, single-byte code pages) for plain
# text and CSV recipient lists (#778): encoding_rs ((Apache-2.0 OR MIT) AND BSD-3-Clause) and its
# detector chardetng (Apache-2.0 OR MIT), both pure Rust (Firefox's).
# text and CSV recipient lists (#778): encoding_rs ((Apache-2.0 OR MIT) AND BSD-3-Clause), pure
# Rust (Firefox's). Detection is craft-mime's `detect` feature.
encoding_rs = "0.8"
# chardetng (Apache-2.0 OR MIT): only the formats test oracle for the old detector uses it directly.
chardetng = "1"
# Password-protected (agile-encrypted) Word documents: pure-Rust RustCrypto primitives.
aes = "0.9"
@@ -153,7 +157,7 @@ debug = 0
# Required by craft-pdf: the backend rejects writers that failed during unwinding.
[patch.crates-io]
krilla = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc" }
krilla = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a" }
ecolor = { git = "https://github.com/storytold/egui.git", rev = "6d518fc799681571963842f95b55a6b68431b15a" }
emath = { git = "https://github.com/storytold/egui.git", rev = "6d518fc799681571963842f95b55a6b68431b15a" }
epaint = { git = "https://github.com/storytold/egui.git", rev = "6d518fc799681571963842f95b55a6b68431b15a" }
+21 -2
View File
File diff suppressed because one or more lines are too long
+2 -2
View File
@@ -18,8 +18,8 @@ wordcraft-engine = { workspace = true }
wordcraft-ui-egui = { workspace = true }
# The host messages (storytold/craft-libs): one contract and one bridge for the web builds.
# The page command API (storytold/craft-libs#413): window.wordcraft over the control channel.
craft-web-api = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "0.1.0" }
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "0.1.0" }
craft-web-api = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "0.1.0" }
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "0.1.0" }
egui = { workspace = true }
log = { workspace = true }
serde_json = { workspace = true }
+1
View File
@@ -81,6 +81,7 @@ pub fn start() {
// Pasted pictures wait apart from dropped files: the app inserts one only when the page is the edit target (#1349).
app.services.pasted = Some(pasted);
app.autosave = false;
app.tell_about_font_downloads();
// Preferences from an earlier visit (#2132), as the desktop app reads `ui.json`.
let stored = stored_prefs();
crate::prefs::restore(&mut app, stored.as_deref());
+1 -1
View File
@@ -86,7 +86,7 @@ clipboard-win = { version = "5.4", features = ["std"] }
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
# At the same craft-libs commit as the workspace's crates: `printer_queues`, the printer list that
# asks no driver (#5710, storytold/craft-libs#618), and `submit_cancellable` (storytold/craft-libs#622).
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc" }
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a" }
rfd = { workspace = true }
# Waits for an `rfd::AsyncFileDialog` on a background thread (the dialogs don't block the UI, #94);
# rfd itself already depends on it.
+2 -1
View File
@@ -649,7 +649,7 @@ fn main() -> eframe::Result {
.with_title("WordCraft")
.with_app_id("ai.storyteller.wordcraft")
.with_inner_size([1440.0, 920.0])
.with_min_inner_size([760.0, 480.0])
.with_min_inner_size(wordcraft_ui_egui::window_geometry::MIN_SIZE)
.with_drag_and_drop(true)
.with_fullsize_content_view(true)
.with_titlebar_shown(false)
@@ -728,6 +728,7 @@ fn main() -> eframe::Result {
app.replace_startup_document(doc);
}
app.watch_desktop_theme(&cc.egui_ctx);
app.tell_about_font_downloads();
if prefs_enabled()
&& let Some(dir) = wordcraft_control_key::settings_dir()
{
+4
View File
@@ -187,6 +187,10 @@ sync_github_name = true
[people.pakiat]
display_name = "OtorO"
[people.ovezthaking]
real_name = "Oliwer Urbaniak"
sync_github_name = true
[people.TobyWu666]
display_name = "tobywu"
+4 -1
View File
@@ -13,10 +13,12 @@ workspace = true
# `zip_guard::zip64_entry_count_sample`) and the step count of
# `work::count`; turned on only as a dev-dependency, so never built into
# the app.
test-support = []
test-support = ["craft-zip/test-support"]
[dependencies]
craft-codepage = { workspace = true }
wordcraft-geom = { workspace = true }
craft-zip = { workspace = true }
base64 = { workspace = true }
log = { workspace = true }
serde = { workspace = true }
@@ -25,4 +27,5 @@ thiserror = { workspace = true }
unicode-segmentation = { workspace = true }
[dev-dependencies]
craft-zip = { workspace = true, features = ["test-support"] }
proptest = { workspace = true }
+44 -337
View File
@@ -1,345 +1,52 @@
//! Windows-1252 (the byte encoding legacy Word text pieces use) and the other single-byte Windows
//! code pages RTF files name with `\ansicpg` or a font's `\fcharset` (Cyrillic 1251, Central
//! European 1250, Greek, Turkish, Hebrew, Arabic, Baltic, Vietnamese, Thai), Mac OS Roman
//! (10000, RTF `\mac`), and the East Asian multibyte code pages (932 Shift-JIS, 936 GBK, 949 UHC,
//! 950 Big5), whose bytes decode as runs rather than one at a time.
//! Legacy character decoding delegates to craft-codepage; document fallback policy stays here.
pub use craft_codepage::{
MAC_ROMAN_CODE_PAGE, MULTIBYTE_CODE_PAGES, OEM_CODE_PAGES, SINGLE_BYTE_CODE_PAGES, cp1252, is_multibyte_code_page, multibyte_encoding_name,
oem_byte, windows_byte,
};
/// Windows-1252 code points for 0x80..=0x9F.
const CP1252: [u16; 32] = [
0x20AC, 0x0081, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0x008D, 0x017D, 0x008F, 0x0090, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0x009D, 0x017E, 0x0178,
];
/// One Windows-1252 byte as a char.
pub fn cp1252(b: u8) -> char {
if (0x80..0xA0).contains(&b) {
CP1252.get((b - 0x80) as usize).and_then(|u| char::from_u32(u32::from(*u))).unwrap_or('\u{FFFD}')
} else {
b as char
}
/// Word's supported single-byte document code pages (symbol-font policy is separate).
pub fn is_byte_code_page(cp: u16) -> bool {
SINGLE_BYTE_CODE_PAGES.contains(&cp) || OEM_CODE_PAGES.contains(&cp) || cp == MAC_ROMAN_CODE_PAGE
}
/// Windows-874 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP874: [u16; 128] = [
0x20AC, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2026, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x0E01, 0x0E02, 0x0E03,
0x0E04, 0x0E05, 0x0E06, 0x0E07, 0x0E08, 0x0E09, 0x0E0A, 0x0E0B, 0x0E0C, 0x0E0D, 0x0E0E, 0x0E0F, 0x0E10, 0x0E11, 0x0E12, 0x0E13, 0x0E14, 0x0E15,
0x0E16, 0x0E17, 0x0E18, 0x0E19, 0x0E1A, 0x0E1B, 0x0E1C, 0x0E1D, 0x0E1E, 0x0E1F, 0x0E20, 0x0E21, 0x0E22, 0x0E23, 0x0E24, 0x0E25, 0x0E26, 0x0E27,
0x0E28, 0x0E29, 0x0E2A, 0x0E2B, 0x0E2C, 0x0E2D, 0x0E2E, 0x0E2F, 0x0E30, 0x0E31, 0x0E32, 0x0E33, 0x0E34, 0x0E35, 0x0E36, 0x0E37, 0x0E38, 0x0E39,
0x0E3A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x0E3F, 0x0E40, 0x0E41, 0x0E42, 0x0E43, 0x0E44, 0x0E45, 0x0E46, 0x0E47, 0x0E48, 0x0E49, 0x0E4A, 0x0E4B,
0x0E4C, 0x0E4D, 0x0E4E, 0x0E4F, 0x0E50, 0x0E51, 0x0E52, 0x0E53, 0x0E54, 0x0E55, 0x0E56, 0x0E57, 0x0E58, 0x0E59, 0x0E5A, 0x0E5B, 0xFFFD, 0xFFFD,
0xFFFD, 0xFFFD,
];
/// Windows-1250 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1250: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0xFFFD, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0x0160, 0x2039, 0x015A, 0x0164, 0x017D, 0x0179, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0x0161, 0x203A, 0x015B, 0x0165, 0x017E, 0x017A, 0x00A0, 0x02C7, 0x02D8, 0x0141,
0x00A4, 0x0104, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x015E, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x017B, 0x00B0, 0x00B1, 0x02DB, 0x0142, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x0105, 0x015F, 0x00BB, 0x013D, 0x02DD, 0x013E, 0x017C, 0x0154, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x0139, 0x0106, 0x00C7,
0x010C, 0x00C9, 0x0118, 0x00CB, 0x011A, 0x00CD, 0x00CE, 0x010E, 0x0110, 0x0143, 0x0147, 0x00D3, 0x00D4, 0x0150, 0x00D6, 0x00D7, 0x0158, 0x016E,
0x00DA, 0x0170, 0x00DC, 0x00DD, 0x0162, 0x00DF, 0x0155, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x013A, 0x0107, 0x00E7, 0x010D, 0x00E9, 0x0119, 0x00EB,
0x011B, 0x00ED, 0x00EE, 0x010F, 0x0111, 0x0144, 0x0148, 0x00F3, 0x00F4, 0x0151, 0x00F6, 0x00F7, 0x0159, 0x016F, 0x00FA, 0x0171, 0x00FC, 0x00FD,
0x0163, 0x02D9,
];
/// Windows-1251 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1251: [u16; 128] = [
0x0402, 0x0403, 0x201A, 0x0453, 0x201E, 0x2026, 0x2020, 0x2021, 0x20AC, 0x2030, 0x0409, 0x2039, 0x040A, 0x040C, 0x040B, 0x040F, 0x0452, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0x0459, 0x203A, 0x045A, 0x045C, 0x045B, 0x045F, 0x00A0, 0x040E, 0x045E, 0x0408,
0x00A4, 0x0490, 0x00A6, 0x00A7, 0x0401, 0x00A9, 0x0404, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x0407, 0x00B0, 0x00B1, 0x0406, 0x0456, 0x0491, 0x00B5,
0x00B6, 0x00B7, 0x0451, 0x2116, 0x0454, 0x00BB, 0x0458, 0x0405, 0x0455, 0x0457, 0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417,
0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421, 0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429,
0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433, 0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B,
0x043C, 0x043D, 0x043E, 0x043F, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B, 0x044C, 0x044D,
0x044E, 0x044F,
];
/// Windows-1253 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1253: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x0385, 0x0386, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0xFFFD, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x2015, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x0384, 0x00B5,
0x00B6, 0x00B7, 0x0388, 0x0389, 0x038A, 0x00BB, 0x038C, 0x00BD, 0x038E, 0x038F, 0x0390, 0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397,
0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, 0xFFFD, 0x03A3, 0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9,
0x03AA, 0x03AB, 0x03AC, 0x03AD, 0x03AE, 0x03AF, 0x03B0, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB,
0x03BC, 0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C2, 0x03C3, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x03C9, 0x03CA, 0x03CB, 0x03CC, 0x03CD,
0x03CE, 0xFFFD,
];
/// Windows-1254 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1254: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0xFFFD, 0xFFFD, 0x0178, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x00C7,
0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, 0x011E, 0x00D1, 0x00D2, 0x00D3, 0x00D4, 0x00D5, 0x00D6, 0x00D7, 0x00D8, 0x00D9,
0x00DA, 0x00DB, 0x00DC, 0x0130, 0x015E, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x00EC, 0x00ED, 0x00EE, 0x00EF, 0x011F, 0x00F1, 0x00F2, 0x00F3, 0x00F4, 0x00F5, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x0131,
0x015F, 0x00FF,
];
/// Windows-1255 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1255: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x20AA, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00D7, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00F7, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x05B0, 0x05B1, 0x05B2, 0x05B3, 0x05B4, 0x05B5, 0x05B6, 0x05B7,
0x05B8, 0x05B9, 0x05BA, 0x05BB, 0x05BC, 0x05BD, 0x05BE, 0x05BF, 0x05C0, 0x05C1, 0x05C2, 0x05C3, 0x05F0, 0x05F1, 0x05F2, 0x05F3, 0x05F4, 0xFFFD,
0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x05D0, 0x05D1, 0x05D2, 0x05D3, 0x05D4, 0x05D5, 0x05D6, 0x05D7, 0x05D8, 0x05D9, 0x05DA, 0x05DB,
0x05DC, 0x05DD, 0x05DE, 0x05DF, 0x05E0, 0x05E1, 0x05E2, 0x05E3, 0x05E4, 0x05E5, 0x05E6, 0x05E7, 0x05E8, 0x05E9, 0x05EA, 0xFFFD, 0xFFFD, 0x200E,
0x200F, 0xFFFD,
];
/// Windows-1256 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1256: [u16; 128] = [
0x20AC, 0x067E, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0679, 0x2039, 0x0152, 0x0686, 0x0698, 0x0688, 0x06AF, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x06A9, 0x2122, 0x0691, 0x203A, 0x0153, 0x200C, 0x200D, 0x06BA, 0x00A0, 0x060C, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x06BE, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x061B, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x061F, 0x06C1, 0x0621, 0x0622, 0x0623, 0x0624, 0x0625, 0x0626, 0x0627,
0x0628, 0x0629, 0x062A, 0x062B, 0x062C, 0x062D, 0x062E, 0x062F, 0x0630, 0x0631, 0x0632, 0x0633, 0x0634, 0x0635, 0x0636, 0x00D7, 0x0637, 0x0638,
0x0639, 0x063A, 0x0640, 0x0641, 0x0642, 0x0643, 0x00E0, 0x0644, 0x00E2, 0x0645, 0x0646, 0x0647, 0x0648, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0649, 0x064A, 0x00EE, 0x00EF, 0x064B, 0x064C, 0x064D, 0x064E, 0x00F4, 0x064F, 0x0650, 0x00F7, 0x0651, 0x00F9, 0x0652, 0x00FB, 0x00FC, 0x200E,
0x200F, 0x06D2,
];
/// Windows-1257 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1257: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0xFFFD, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0x00A8, 0x02C7, 0x00B8, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0x00AF, 0x02DB, 0xFFFD, 0x00A0, 0xFFFD, 0x00A2, 0x00A3,
0x00A4, 0xFFFD, 0x00A6, 0x00A7, 0x00D8, 0x00A9, 0x0156, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00C6, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00F8, 0x00B9, 0x0157, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00E6, 0x0104, 0x012E, 0x0100, 0x0106, 0x00C4, 0x00C5, 0x0118, 0x0112,
0x010C, 0x00C9, 0x0179, 0x0116, 0x0122, 0x0136, 0x012A, 0x013B, 0x0160, 0x0143, 0x0145, 0x00D3, 0x014C, 0x00D5, 0x00D6, 0x00D7, 0x0172, 0x0141,
0x015A, 0x016A, 0x00DC, 0x017B, 0x017D, 0x00DF, 0x0105, 0x012F, 0x0101, 0x0107, 0x00E4, 0x00E5, 0x0119, 0x0113, 0x010D, 0x00E9, 0x017A, 0x0117,
0x0123, 0x0137, 0x012B, 0x013C, 0x0161, 0x0144, 0x0146, 0x00F3, 0x014D, 0x00F5, 0x00F6, 0x00F7, 0x0173, 0x0142, 0x015B, 0x016B, 0x00FC, 0x017C,
0x017E, 0x02D9,
];
/// Windows-1258 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1258: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0xFFFD, 0x2039, 0x0152, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0xFFFD, 0x203A, 0x0153, 0xFFFD, 0xFFFD, 0x0178, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x00C5, 0x00C6, 0x00C7,
0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x0300, 0x00CD, 0x00CE, 0x00CF, 0x0110, 0x00D1, 0x0309, 0x00D3, 0x00D4, 0x01A0, 0x00D6, 0x00D7, 0x00D8, 0x00D9,
0x00DA, 0x00DB, 0x00DC, 0x01AF, 0x0303, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0301, 0x00ED, 0x00EE, 0x00EF, 0x0111, 0x00F1, 0x0323, 0x00F3, 0x00F4, 0x01A1, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x01B0,
0x20AB, 0x00FF,
];
/// Mac OS Roman (code page 10000, RTF `\mac`) code points for 0x80..=0xFF, with the euro sign at
/// 0xDB (Mac OS 8.5 and later).
const MAC_ROMAN: [u16; 128] = [
0x00C4, 0x00C5, 0x00C7, 0x00C9, 0x00D1, 0x00D6, 0x00DC, 0x00E1, 0x00E0, 0x00E2, 0x00E4, 0x00E3, 0x00E5, 0x00E7, 0x00E9, 0x00E8, 0x00EA, 0x00EB,
0x00ED, 0x00EC, 0x00EE, 0x00EF, 0x00F1, 0x00F3, 0x00F2, 0x00F4, 0x00F6, 0x00F5, 0x00FA, 0x00F9, 0x00FB, 0x00FC, 0x2020, 0x00B0, 0x00A2, 0x00A3,
0x00A7, 0x2022, 0x00B6, 0x00DF, 0x00AE, 0x00A9, 0x2122, 0x00B4, 0x00A8, 0x2260, 0x00C6, 0x00D8, 0x221E, 0x00B1, 0x2264, 0x2265, 0x00A5, 0x00B5,
0x2202, 0x2211, 0x220F, 0x03C0, 0x222B, 0x00AA, 0x00BA, 0x03A9, 0x00E6, 0x00F8, 0x00BF, 0x00A1, 0x00AC, 0x221A, 0x0192, 0x2248, 0x2206, 0x00AB,
0x00BB, 0x2026, 0x00A0, 0x00C0, 0x00C3, 0x00D5, 0x0152, 0x0153, 0x2013, 0x2014, 0x201C, 0x201D, 0x2018, 0x2019, 0x00F7, 0x25CA, 0x00FF, 0x0178,
0x2044, 0x20AC, 0x2039, 0x203A, 0xFB01, 0xFB02, 0x2021, 0x00B7, 0x201A, 0x201E, 0x2030, 0x00C2, 0x00CA, 0x00C1, 0x00CB, 0x00C8, 0x00CD, 0x00CE,
0x00CF, 0x00CC, 0x00D3, 0x00D4, 0xF8FF, 0x00D2, 0x00DA, 0x00DB, 0x00D9, 0x0131, 0x02C6, 0x02DC, 0x00AF, 0x02D8, 0x02D9, 0x02DA, 0x00B8, 0x02DD,
0x02DB, 0x02C7,
];
/// IBM PC (OEM) code page 437 code points for 0x80..=0xFF (the Unicode Consortium's CP437
/// mapping file).
const CP437: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x00EC, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x00FF, 0x00D6, 0x00DC, 0x00A2, 0x00A3, 0x00A5, 0x20A7, 0x0192, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x00AA, 0x00BA, 0x00BF, 0x2310, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x03B1, 0x00DF, 0x0393, 0x03C0, 0x03A3, 0x03C3, 0x00B5, 0x03C4, 0x03A6, 0x0398, 0x03A9, 0x03B4,
0x221E, 0x03C6, 0x03B5, 0x2229, 0x2261, 0x00B1, 0x2265, 0x2264, 0x2320, 0x2321, 0x00F7, 0x2248, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 850 (DOS Latin-1, Western European) code points for 0x80..=0xFF (the Unicode
/// Consortium's CP850 mapping file).
const CP850: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x00EC, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x00FF, 0x00D6, 0x00DC, 0x00F8, 0x00A3, 0x00D8, 0x00D7, 0x0192, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x00AA, 0x00BA, 0x00BF, 0x00AE, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x00C0, 0x00A9, 0x2563, 0x2551, 0x2557, 0x255D, 0x00A2, 0x00A5, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x00E3, 0x00C3,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x00F0, 0x00D0, 0x00CA, 0x00CB, 0x00C8, 0x0131, 0x00CD, 0x00CE, 0x00CF, 0x2518,
0x250C, 0x2588, 0x2584, 0x00A6, 0x00CC, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x00D2, 0x00F5, 0x00D5, 0x00B5, 0x00FE, 0x00DE, 0x00DA, 0x00DB, 0x00D9,
0x00FD, 0x00DD, 0x00AF, 0x00B4, 0x00AD, 0x00B1, 0x2017, 0x00BE, 0x00B6, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x00B7, 0x00B9, 0x00B3, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 852 (DOS Latin-2, Central European) code points for 0x80..=0xFF (the Unicode Consortium's CP852
/// mapping file).
const CP852: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x016F, 0x0107, 0x00E7, 0x0142, 0x00EB, 0x0150, 0x0151, 0x00EE, 0x0179, 0x00C4, 0x0106, 0x00C9, 0x0139,
0x013A, 0x00F4, 0x00F6, 0x013D, 0x013E, 0x015A, 0x015B, 0x00D6, 0x00DC, 0x0164, 0x0165, 0x0141, 0x00D7, 0x010D, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x0104, 0x0105, 0x017D, 0x017E, 0x0118, 0x0119, 0x00AC, 0x017A, 0x010C, 0x015F, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x011A, 0x015E, 0x2563, 0x2551, 0x2557, 0x255D, 0x017B, 0x017C, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x0102, 0x0103,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x0111, 0x0110, 0x010E, 0x00CB, 0x010F, 0x0147, 0x00CD, 0x00CE, 0x011B, 0x2518,
0x250C, 0x2588, 0x2584, 0x0162, 0x016E, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x0143, 0x0144, 0x0148, 0x0160, 0x0161, 0x0154, 0x00DA, 0x0155, 0x0170,
0x00FD, 0x00DD, 0x0163, 0x00B4, 0x00AD, 0x02DD, 0x02DB, 0x02C7, 0x02D8, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x02D9, 0x0171, 0x0158, 0x0159,
0x25A0, 0x00A0,
];
/// OEM code page 866 (DOS Cyrillic) code points for 0x80..=0xFF (the Unicode Consortium's CP866
/// mapping file).
const CP866: [u16; 128] = [
0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417, 0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421,
0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429, 0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433,
0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B, 0x043C, 0x043D, 0x043E, 0x043F, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B,
0x044C, 0x044D, 0x044E, 0x044F, 0x0401, 0x0451, 0x0404, 0x0454, 0x0407, 0x0457, 0x040E, 0x045E, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x2116, 0x00A4,
0x25A0, 0x00A0,
];
/// OEM code page 737 (DOS Greek) code points for 0x80..=0xFF (the Unicode Consortium's CP737
/// mapping file).
const CP737: [u16; 128] = [
0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397, 0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, 0x03A3,
0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB, 0x03BC,
0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C3, 0x03C2, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x03C9, 0x03AC, 0x03AD, 0x03AE, 0x03CA, 0x03AF, 0x03CC, 0x03CD, 0x03CB, 0x03CE, 0x0386, 0x0388,
0x0389, 0x038A, 0x038C, 0x038E, 0x038F, 0x00B1, 0x2265, 0x2264, 0x03AA, 0x03AB, 0x00F7, 0x2248, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 857 (DOS Turkish; U+FFFD where the byte is unassigned) code points for 0x80..=0xFF (the Unicode Consortium's CP857
/// mapping file).
const CP857: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x0131, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x0130, 0x00D6, 0x00DC, 0x00F8, 0x00A3, 0x00D8, 0x015E, 0x015F, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x011E, 0x011F, 0x00BF, 0x00AE, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x00C0, 0x00A9, 0x2563, 0x2551, 0x2557, 0x255D, 0x00A2, 0x00A5, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x00E3, 0x00C3,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x00BA, 0x00AA, 0x00CA, 0x00CB, 0x00C8, 0xFFFD, 0x00CD, 0x00CE, 0x00CF, 0x2518,
0x250C, 0x2588, 0x2584, 0x00A6, 0x00CC, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x00D2, 0x00F5, 0x00D5, 0x00B5, 0xFFFD, 0x00D7, 0x00DA, 0x00DB, 0x00D9,
0x00EC, 0x00FF, 0x00AF, 0x00B4, 0x00AD, 0x00B1, 0xFFFD, 0x00BE, 0x00B6, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x00B7, 0x00B9, 0x00B3, 0x00B2,
0x25A0, 0x00A0,
];
/// The glyphs the OEM code pages show for 0x01..=0x1F (smileys, card suits, arrows), as
/// Windows' Alt+numpad entry produces them rather than the control characters.
const OEM_GLYPHS: [u16; 31] = [
0x263A, 0x263B, 0x2665, 0x2666, 0x2663, 0x2660, 0x2022, 0x25D8, 0x25CB, 0x25D9, 0x2642, 0x2640, 0x266A, 0x266B, 0x263C, 0x25BA, 0x25C4, 0x2195,
0x203C, 0x00B6, 0x00A7, 0x25AC, 0x21A8, 0x2191, 0x2193, 0x2192, 0x2190, 0x221F, 0x2194, 0x25B2, 0x25BC,
];
/// The OEM code pages [`oem_byte`] decodes.
pub const OEM_CODE_PAGES: [u16; 6] = [437, 737, 850, 852, 857, 866];
/// One byte of an OEM code page in [`OEM_CODE_PAGES`] (any other code page reads as 437) as a
/// char, with the graphic glyphs for 0x01..=0x1F and 0x7F (⌂); 0x00 stays NUL.
pub fn oem_byte(code_page: u16, b: u8) -> char {
let table: &[u16; 128] = match code_page {
737 => &CP737,
850 => &CP850,
852 => &CP852,
857 => &CP857,
866 => &CP866,
_ => &CP437,
};
let u = match b {
0x01..=0x1F => OEM_GLYPHS.get(usize::from(b - 1)).copied(),
0x7F => Some(0x2302),
0x80..=0xFF => table.get(usize::from(b - 0x80)).copied(),
_ => Some(u16::from(b)),
};
u.and_then(|u| char::from_u32(u32::from(u))).unwrap_or('\u{FFFD}')
/// Decode document text without applying OEM control glyphs or symbol-font remapping.
pub fn code_page_byte(cp: u16, b: u8) -> char {
if b >= 0x80 && OEM_CODE_PAGES.contains(&cp) { oem_byte(cp, b) } else { windows_byte(cp, b) }
}
/// The single-byte Windows code pages [`windows_byte`] decodes (1252 included).
pub const SINGLE_BYTE_CODE_PAGES: [u16; 10] = [874, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258];
/// Mac OS Roman, RTF's `\mac` character set.
pub const MAC_ROMAN_CODE_PAGE: u16 = 10000;
/// One byte of a single-byte code page as a char: a Windows one in [`SINGLE_BYTE_CODE_PAGES`] or
/// Mac OS Roman ([`MAC_ROMAN_CODE_PAGE`]); any other code page reads as Windows-1252.
pub fn windows_byte(code_page: u16, b: u8) -> char {
let table: &[u16; 128] = match code_page {
874 => &CP874,
1250 => &CP1250,
1251 => &CP1251,
1253 => &CP1253,
1254 => &CP1254,
1255 => &CP1255,
1256 => &CP1256,
1257 => &CP1257,
1258 => &CP1258,
MAC_ROMAN_CODE_PAGE => &MAC_ROMAN,
_ => return cp1252(b),
};
if b < 0x80 {
return b as char;
}
table.get(usize::from(b - 0x80)).and_then(|u| char::from_u32(u32::from(*u))).unwrap_or('\u{FFFD}')
/// Whether Word's document import accepts this encoding.
pub fn is_text_code_page(cp: u16) -> bool {
is_byte_code_page(cp) || is_multibyte_code_page(cp)
}
/// Whether [`code_page_byte`] decodes `code_page`: a Windows one ([`SINGLE_BYTE_CODE_PAGES`]), an
/// OEM (DOS) one ([`OEM_CODE_PAGES`]) or Mac OS Roman. Document formats (RTF) name all three.
pub fn is_byte_code_page(code_page: u16) -> bool {
SINGLE_BYTE_CODE_PAGES.contains(&code_page) || OEM_CODE_PAGES.contains(&code_page) || code_page == MAC_ROMAN_CODE_PAGE
}
/// A byte of 8-bit document text in `code_page` (see [`is_byte_code_page`]) as a char. Below 0x80
/// it is ASCII in every code page (text, so an OEM page's glyphs for control codes don't apply).
pub fn code_page_byte(code_page: u16, b: u8) -> char {
if b >= 0x80 && OEM_CODE_PAGES.contains(&code_page) { oem_byte(code_page, b) } else { windows_byte(code_page, b) }
}
/// The East Asian multibyte code pages a document format names with `\ansicpg` or a font's `\cpg`:
/// 932 Shift-JIS (Japanese), 936 GBK (Simplified Chinese), 949 UHC (Korean) and 950 Big5
/// (Traditional Chinese).
pub const MULTIBYTE_CODE_PAGES: [u16; 4] = [932, 936, 949, 950];
/// Whether `code_page` is one of the East Asian multibyte code pages ([`MULTIBYTE_CODE_PAGES`]):
/// its bytes decode as a run, a lead byte together with its trail byte.
pub fn is_multibyte_code_page(code_page: u16) -> bool {
multibyte_encoding_name(code_page).is_some()
}
/// The WHATWG encoding name of an East Asian multibyte code page (see [`MULTIBYTE_CODE_PAGES`]),
/// for `encoding_rs::Encoding::for_label`; `None` for the single-byte code pages, whose bytes
/// decode one at a time, and anything else.
pub fn multibyte_encoding_name(code_page: u16) -> Option<&'static str> {
Some(match code_page {
932 => "shift_jis",
936 => "gbk",
949 => "euc-kr",
950 => "big5",
_ => return None,
})
}
/// Whether a document format can name `code_page` for its text: a single-byte Windows, DOS or Mac
/// one ([`is_byte_code_page`]) or an East Asian multibyte one ([`is_multibyte_code_page`]).
pub fn is_text_code_page(code_page: u16) -> bool {
is_byte_code_page(code_page) || is_multibyte_code_page(code_page)
}
/// The code page of an RTF or Word font charset (`\fcharset`): 204 Russian → 1251, 238 Eastern
/// European → 1250, 77 Mac → 10000, 254 PC / 255 OEM → 437, 128 Japanese → 932, 129 Korean → 949,
/// 134 Simplified Chinese → 936, 136 Traditional Chinese → 950… `None` for the ANSI and default
/// charsets (the document's code page applies) and symbol fonts.
/// Preserve Word's existing symbol-font selection policy, not the shared symbol extension.
pub fn charset_code_page(charset: i32) -> Option<u16> {
Some(match charset {
77 => MAC_ROMAN_CODE_PAGE,
128 => 932,
129 => 949,
134 => 936,
136 => 950,
161 => 1253,
162 => 1254,
163 => 1258,
177 => 1255,
178 => 1256,
186 => 1257,
204 => 1251,
222 => 874,
238 => 1250,
254 | 255 => 437,
_ => return None,
})
if charset == 2 { None } else { craft_codepage::charset_code_page(charset) }
}
#[cfg(test)]
#[allow(dead_code)]
#[path = "encoding_legacy_tests.rs"]
mod legacy;
#[cfg(test)]
mod parity {
use super::*;
#[test]
fn every_supported_byte_and_fallback_matches_original() {
let pages = [0, 42, 437, 737, 850, 852, 857, 866, 874, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 10000, 9999, u16::MAX];
for cp in pages {
for b in 0..=u8::MAX {
assert_eq!(windows_byte(cp, b), legacy::windows_byte(cp, b), "windows {cp}/{b}");
assert_eq!(oem_byte(cp, b), legacy::oem_byte(cp, b), "oem {cp}/{b}");
assert_eq!(code_page_byte(cp, b), legacy::code_page_byte(cp, b), "text {cp}/{b}");
assert_eq!(cp1252(b), legacy::cp1252(b));
}
}
for cp in 0..=u16::MAX {
assert_eq!(is_byte_code_page(cp), legacy::is_byte_code_page(cp));
assert_eq!(is_text_code_page(cp), legacy::is_text_code_page(cp));
assert_eq!(multibyte_encoding_name(cp), legacy::multibyte_encoding_name(cp));
}
for charset in -1..=256 {
assert_eq!(charset_code_page(charset), legacy::charset_code_page(charset));
}
}
}
#[cfg(test)]
+428
View File
@@ -0,0 +1,428 @@
// Frozen pre-cutover implementation, test-only parity oracle. Original authors retained in file history.
//! Windows-1252 (the byte encoding legacy Word text pieces use) and the other single-byte Windows
//! code pages RTF files name with `\ansicpg` or a font's `\fcharset` (Cyrillic 1251, Central
//! European 1250, Greek, Turkish, Hebrew, Arabic, Baltic, Vietnamese, Thai), Mac OS Roman
//! (10000, RTF `\mac`), and the East Asian multibyte code pages (932 Shift-JIS, 936 GBK, 949 UHC,
//! 950 Big5), whose bytes decode as runs rather than one at a time.
/// Windows-1252 code points for 0x80..=0x9F.
const CP1252: [u16; 32] = [
0x20AC, 0x0081, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0x008D, 0x017D, 0x008F, 0x0090, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0x009D, 0x017E, 0x0178,
];
/// One Windows-1252 byte as a char.
pub fn cp1252(b: u8) -> char {
if (0x80..0xA0).contains(&b) {
CP1252.get((b - 0x80) as usize).and_then(|u| char::from_u32(u32::from(*u))).unwrap_or('\u{FFFD}')
} else {
b as char
}
}
/// Windows-874 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP874: [u16; 128] = [
0x20AC, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2026, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x0E01, 0x0E02, 0x0E03,
0x0E04, 0x0E05, 0x0E06, 0x0E07, 0x0E08, 0x0E09, 0x0E0A, 0x0E0B, 0x0E0C, 0x0E0D, 0x0E0E, 0x0E0F, 0x0E10, 0x0E11, 0x0E12, 0x0E13, 0x0E14, 0x0E15,
0x0E16, 0x0E17, 0x0E18, 0x0E19, 0x0E1A, 0x0E1B, 0x0E1C, 0x0E1D, 0x0E1E, 0x0E1F, 0x0E20, 0x0E21, 0x0E22, 0x0E23, 0x0E24, 0x0E25, 0x0E26, 0x0E27,
0x0E28, 0x0E29, 0x0E2A, 0x0E2B, 0x0E2C, 0x0E2D, 0x0E2E, 0x0E2F, 0x0E30, 0x0E31, 0x0E32, 0x0E33, 0x0E34, 0x0E35, 0x0E36, 0x0E37, 0x0E38, 0x0E39,
0x0E3A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x0E3F, 0x0E40, 0x0E41, 0x0E42, 0x0E43, 0x0E44, 0x0E45, 0x0E46, 0x0E47, 0x0E48, 0x0E49, 0x0E4A, 0x0E4B,
0x0E4C, 0x0E4D, 0x0E4E, 0x0E4F, 0x0E50, 0x0E51, 0x0E52, 0x0E53, 0x0E54, 0x0E55, 0x0E56, 0x0E57, 0x0E58, 0x0E59, 0x0E5A, 0x0E5B, 0xFFFD, 0xFFFD,
0xFFFD, 0xFFFD,
];
/// Windows-1250 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1250: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0xFFFD, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0x0160, 0x2039, 0x015A, 0x0164, 0x017D, 0x0179, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0x0161, 0x203A, 0x015B, 0x0165, 0x017E, 0x017A, 0x00A0, 0x02C7, 0x02D8, 0x0141,
0x00A4, 0x0104, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x015E, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x017B, 0x00B0, 0x00B1, 0x02DB, 0x0142, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x0105, 0x015F, 0x00BB, 0x013D, 0x02DD, 0x013E, 0x017C, 0x0154, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x0139, 0x0106, 0x00C7,
0x010C, 0x00C9, 0x0118, 0x00CB, 0x011A, 0x00CD, 0x00CE, 0x010E, 0x0110, 0x0143, 0x0147, 0x00D3, 0x00D4, 0x0150, 0x00D6, 0x00D7, 0x0158, 0x016E,
0x00DA, 0x0170, 0x00DC, 0x00DD, 0x0162, 0x00DF, 0x0155, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x013A, 0x0107, 0x00E7, 0x010D, 0x00E9, 0x0119, 0x00EB,
0x011B, 0x00ED, 0x00EE, 0x010F, 0x0111, 0x0144, 0x0148, 0x00F3, 0x00F4, 0x0151, 0x00F6, 0x00F7, 0x0159, 0x016F, 0x00FA, 0x0171, 0x00FC, 0x00FD,
0x0163, 0x02D9,
];
/// Windows-1251 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1251: [u16; 128] = [
0x0402, 0x0403, 0x201A, 0x0453, 0x201E, 0x2026, 0x2020, 0x2021, 0x20AC, 0x2030, 0x0409, 0x2039, 0x040A, 0x040C, 0x040B, 0x040F, 0x0452, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0x0459, 0x203A, 0x045A, 0x045C, 0x045B, 0x045F, 0x00A0, 0x040E, 0x045E, 0x0408,
0x00A4, 0x0490, 0x00A6, 0x00A7, 0x0401, 0x00A9, 0x0404, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x0407, 0x00B0, 0x00B1, 0x0406, 0x0456, 0x0491, 0x00B5,
0x00B6, 0x00B7, 0x0451, 0x2116, 0x0454, 0x00BB, 0x0458, 0x0405, 0x0455, 0x0457, 0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417,
0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421, 0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429,
0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433, 0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B,
0x043C, 0x043D, 0x043E, 0x043F, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B, 0x044C, 0x044D,
0x044E, 0x044F,
];
/// Windows-1253 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1253: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x0385, 0x0386, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0xFFFD, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x2015, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x0384, 0x00B5,
0x00B6, 0x00B7, 0x0388, 0x0389, 0x038A, 0x00BB, 0x038C, 0x00BD, 0x038E, 0x038F, 0x0390, 0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397,
0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, 0xFFFD, 0x03A3, 0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9,
0x03AA, 0x03AB, 0x03AC, 0x03AD, 0x03AE, 0x03AF, 0x03B0, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB,
0x03BC, 0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C2, 0x03C3, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x03C9, 0x03CA, 0x03CB, 0x03CC, 0x03CD,
0x03CE, 0xFFFD,
];
/// Windows-1254 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1254: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0xFFFD, 0xFFFD, 0x0178, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x00C7,
0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, 0x011E, 0x00D1, 0x00D2, 0x00D3, 0x00D4, 0x00D5, 0x00D6, 0x00D7, 0x00D8, 0x00D9,
0x00DA, 0x00DB, 0x00DC, 0x0130, 0x015E, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x00EC, 0x00ED, 0x00EE, 0x00EF, 0x011F, 0x00F1, 0x00F2, 0x00F3, 0x00F4, 0x00F5, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x0131,
0x015F, 0x00FF,
];
/// Windows-1255 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1255: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x20AA, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00D7, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00F7, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x05B0, 0x05B1, 0x05B2, 0x05B3, 0x05B4, 0x05B5, 0x05B6, 0x05B7,
0x05B8, 0x05B9, 0x05BA, 0x05BB, 0x05BC, 0x05BD, 0x05BE, 0x05BF, 0x05C0, 0x05C1, 0x05C2, 0x05C3, 0x05F0, 0x05F1, 0x05F2, 0x05F3, 0x05F4, 0xFFFD,
0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x05D0, 0x05D1, 0x05D2, 0x05D3, 0x05D4, 0x05D5, 0x05D6, 0x05D7, 0x05D8, 0x05D9, 0x05DA, 0x05DB,
0x05DC, 0x05DD, 0x05DE, 0x05DF, 0x05E0, 0x05E1, 0x05E2, 0x05E3, 0x05E4, 0x05E5, 0x05E6, 0x05E7, 0x05E8, 0x05E9, 0x05EA, 0xFFFD, 0xFFFD, 0x200E,
0x200F, 0xFFFD,
];
/// Windows-1256 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1256: [u16; 128] = [
0x20AC, 0x067E, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0679, 0x2039, 0x0152, 0x0686, 0x0698, 0x0688, 0x06AF, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x06A9, 0x2122, 0x0691, 0x203A, 0x0153, 0x200C, 0x200D, 0x06BA, 0x00A0, 0x060C, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x06BE, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x061B, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x061F, 0x06C1, 0x0621, 0x0622, 0x0623, 0x0624, 0x0625, 0x0626, 0x0627,
0x0628, 0x0629, 0x062A, 0x062B, 0x062C, 0x062D, 0x062E, 0x062F, 0x0630, 0x0631, 0x0632, 0x0633, 0x0634, 0x0635, 0x0636, 0x00D7, 0x0637, 0x0638,
0x0639, 0x063A, 0x0640, 0x0641, 0x0642, 0x0643, 0x00E0, 0x0644, 0x00E2, 0x0645, 0x0646, 0x0647, 0x0648, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0649, 0x064A, 0x00EE, 0x00EF, 0x064B, 0x064C, 0x064D, 0x064E, 0x00F4, 0x064F, 0x0650, 0x00F7, 0x0651, 0x00F9, 0x0652, 0x00FB, 0x00FC, 0x200E,
0x200F, 0x06D2,
];
/// Windows-1257 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1257: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0xFFFD, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0x00A8, 0x02C7, 0x00B8, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0x00AF, 0x02DB, 0xFFFD, 0x00A0, 0xFFFD, 0x00A2, 0x00A3,
0x00A4, 0xFFFD, 0x00A6, 0x00A7, 0x00D8, 0x00A9, 0x0156, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00C6, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00F8, 0x00B9, 0x0157, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00E6, 0x0104, 0x012E, 0x0100, 0x0106, 0x00C4, 0x00C5, 0x0118, 0x0112,
0x010C, 0x00C9, 0x0179, 0x0116, 0x0122, 0x0136, 0x012A, 0x013B, 0x0160, 0x0143, 0x0145, 0x00D3, 0x014C, 0x00D5, 0x00D6, 0x00D7, 0x0172, 0x0141,
0x015A, 0x016A, 0x00DC, 0x017B, 0x017D, 0x00DF, 0x0105, 0x012F, 0x0101, 0x0107, 0x00E4, 0x00E5, 0x0119, 0x0113, 0x010D, 0x00E9, 0x017A, 0x0117,
0x0123, 0x0137, 0x012B, 0x013C, 0x0161, 0x0144, 0x0146, 0x00F3, 0x014D, 0x00F5, 0x00F6, 0x00F7, 0x0173, 0x0142, 0x015B, 0x016B, 0x00FC, 0x017C,
0x017E, 0x02D9,
];
/// Windows-1258 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1258: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0xFFFD, 0x2039, 0x0152, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0xFFFD, 0x203A, 0x0153, 0xFFFD, 0xFFFD, 0x0178, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x00C5, 0x00C6, 0x00C7,
0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x0300, 0x00CD, 0x00CE, 0x00CF, 0x0110, 0x00D1, 0x0309, 0x00D3, 0x00D4, 0x01A0, 0x00D6, 0x00D7, 0x00D8, 0x00D9,
0x00DA, 0x00DB, 0x00DC, 0x01AF, 0x0303, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0301, 0x00ED, 0x00EE, 0x00EF, 0x0111, 0x00F1, 0x0323, 0x00F3, 0x00F4, 0x01A1, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x01B0,
0x20AB, 0x00FF,
];
/// Mac OS Roman (code page 10000, RTF `\mac`) code points for 0x80..=0xFF, with the euro sign at
/// 0xDB (Mac OS 8.5 and later).
const MAC_ROMAN: [u16; 128] = [
0x00C4, 0x00C5, 0x00C7, 0x00C9, 0x00D1, 0x00D6, 0x00DC, 0x00E1, 0x00E0, 0x00E2, 0x00E4, 0x00E3, 0x00E5, 0x00E7, 0x00E9, 0x00E8, 0x00EA, 0x00EB,
0x00ED, 0x00EC, 0x00EE, 0x00EF, 0x00F1, 0x00F3, 0x00F2, 0x00F4, 0x00F6, 0x00F5, 0x00FA, 0x00F9, 0x00FB, 0x00FC, 0x2020, 0x00B0, 0x00A2, 0x00A3,
0x00A7, 0x2022, 0x00B6, 0x00DF, 0x00AE, 0x00A9, 0x2122, 0x00B4, 0x00A8, 0x2260, 0x00C6, 0x00D8, 0x221E, 0x00B1, 0x2264, 0x2265, 0x00A5, 0x00B5,
0x2202, 0x2211, 0x220F, 0x03C0, 0x222B, 0x00AA, 0x00BA, 0x03A9, 0x00E6, 0x00F8, 0x00BF, 0x00A1, 0x00AC, 0x221A, 0x0192, 0x2248, 0x2206, 0x00AB,
0x00BB, 0x2026, 0x00A0, 0x00C0, 0x00C3, 0x00D5, 0x0152, 0x0153, 0x2013, 0x2014, 0x201C, 0x201D, 0x2018, 0x2019, 0x00F7, 0x25CA, 0x00FF, 0x0178,
0x2044, 0x20AC, 0x2039, 0x203A, 0xFB01, 0xFB02, 0x2021, 0x00B7, 0x201A, 0x201E, 0x2030, 0x00C2, 0x00CA, 0x00C1, 0x00CB, 0x00C8, 0x00CD, 0x00CE,
0x00CF, 0x00CC, 0x00D3, 0x00D4, 0xF8FF, 0x00D2, 0x00DA, 0x00DB, 0x00D9, 0x0131, 0x02C6, 0x02DC, 0x00AF, 0x02D8, 0x02D9, 0x02DA, 0x00B8, 0x02DD,
0x02DB, 0x02C7,
];
/// IBM PC (OEM) code page 437 code points for 0x80..=0xFF (the Unicode Consortium's CP437
/// mapping file).
const CP437: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x00EC, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x00FF, 0x00D6, 0x00DC, 0x00A2, 0x00A3, 0x00A5, 0x20A7, 0x0192, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x00AA, 0x00BA, 0x00BF, 0x2310, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x03B1, 0x00DF, 0x0393, 0x03C0, 0x03A3, 0x03C3, 0x00B5, 0x03C4, 0x03A6, 0x0398, 0x03A9, 0x03B4,
0x221E, 0x03C6, 0x03B5, 0x2229, 0x2261, 0x00B1, 0x2265, 0x2264, 0x2320, 0x2321, 0x00F7, 0x2248, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 850 (DOS Latin-1, Western European) code points for 0x80..=0xFF (the Unicode
/// Consortium's CP850 mapping file).
const CP850: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x00EC, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x00FF, 0x00D6, 0x00DC, 0x00F8, 0x00A3, 0x00D8, 0x00D7, 0x0192, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x00AA, 0x00BA, 0x00BF, 0x00AE, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x00C0, 0x00A9, 0x2563, 0x2551, 0x2557, 0x255D, 0x00A2, 0x00A5, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x00E3, 0x00C3,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x00F0, 0x00D0, 0x00CA, 0x00CB, 0x00C8, 0x0131, 0x00CD, 0x00CE, 0x00CF, 0x2518,
0x250C, 0x2588, 0x2584, 0x00A6, 0x00CC, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x00D2, 0x00F5, 0x00D5, 0x00B5, 0x00FE, 0x00DE, 0x00DA, 0x00DB, 0x00D9,
0x00FD, 0x00DD, 0x00AF, 0x00B4, 0x00AD, 0x00B1, 0x2017, 0x00BE, 0x00B6, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x00B7, 0x00B9, 0x00B3, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 852 (DOS Latin-2, Central European) code points for 0x80..=0xFF (the Unicode Consortium's CP852
/// mapping file).
const CP852: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x016F, 0x0107, 0x00E7, 0x0142, 0x00EB, 0x0150, 0x0151, 0x00EE, 0x0179, 0x00C4, 0x0106, 0x00C9, 0x0139,
0x013A, 0x00F4, 0x00F6, 0x013D, 0x013E, 0x015A, 0x015B, 0x00D6, 0x00DC, 0x0164, 0x0165, 0x0141, 0x00D7, 0x010D, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x0104, 0x0105, 0x017D, 0x017E, 0x0118, 0x0119, 0x00AC, 0x017A, 0x010C, 0x015F, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x011A, 0x015E, 0x2563, 0x2551, 0x2557, 0x255D, 0x017B, 0x017C, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x0102, 0x0103,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x0111, 0x0110, 0x010E, 0x00CB, 0x010F, 0x0147, 0x00CD, 0x00CE, 0x011B, 0x2518,
0x250C, 0x2588, 0x2584, 0x0162, 0x016E, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x0143, 0x0144, 0x0148, 0x0160, 0x0161, 0x0154, 0x00DA, 0x0155, 0x0170,
0x00FD, 0x00DD, 0x0163, 0x00B4, 0x00AD, 0x02DD, 0x02DB, 0x02C7, 0x02D8, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x02D9, 0x0171, 0x0158, 0x0159,
0x25A0, 0x00A0,
];
/// OEM code page 866 (DOS Cyrillic) code points for 0x80..=0xFF (the Unicode Consortium's CP866
/// mapping file).
const CP866: [u16; 128] = [
0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417, 0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421,
0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429, 0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433,
0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B, 0x043C, 0x043D, 0x043E, 0x043F, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B,
0x044C, 0x044D, 0x044E, 0x044F, 0x0401, 0x0451, 0x0404, 0x0454, 0x0407, 0x0457, 0x040E, 0x045E, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x2116, 0x00A4,
0x25A0, 0x00A0,
];
/// OEM code page 737 (DOS Greek) code points for 0x80..=0xFF (the Unicode Consortium's CP737
/// mapping file).
const CP737: [u16; 128] = [
0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397, 0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, 0x03A3,
0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB, 0x03BC,
0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C3, 0x03C2, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x03C9, 0x03AC, 0x03AD, 0x03AE, 0x03CA, 0x03AF, 0x03CC, 0x03CD, 0x03CB, 0x03CE, 0x0386, 0x0388,
0x0389, 0x038A, 0x038C, 0x038E, 0x038F, 0x00B1, 0x2265, 0x2264, 0x03AA, 0x03AB, 0x00F7, 0x2248, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 857 (DOS Turkish; U+FFFD where the byte is unassigned) code points for 0x80..=0xFF (the Unicode Consortium's CP857
/// mapping file).
const CP857: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x0131, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x0130, 0x00D6, 0x00DC, 0x00F8, 0x00A3, 0x00D8, 0x015E, 0x015F, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x011E, 0x011F, 0x00BF, 0x00AE, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x00C0, 0x00A9, 0x2563, 0x2551, 0x2557, 0x255D, 0x00A2, 0x00A5, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x00E3, 0x00C3,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x00BA, 0x00AA, 0x00CA, 0x00CB, 0x00C8, 0xFFFD, 0x00CD, 0x00CE, 0x00CF, 0x2518,
0x250C, 0x2588, 0x2584, 0x00A6, 0x00CC, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x00D2, 0x00F5, 0x00D5, 0x00B5, 0xFFFD, 0x00D7, 0x00DA, 0x00DB, 0x00D9,
0x00EC, 0x00FF, 0x00AF, 0x00B4, 0x00AD, 0x00B1, 0xFFFD, 0x00BE, 0x00B6, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x00B7, 0x00B9, 0x00B3, 0x00B2,
0x25A0, 0x00A0,
];
/// The glyphs the OEM code pages show for 0x01..=0x1F (smileys, card suits, arrows), as
/// Windows' Alt+numpad entry produces them rather than the control characters.
const OEM_GLYPHS: [u16; 31] = [
0x263A, 0x263B, 0x2665, 0x2666, 0x2663, 0x2660, 0x2022, 0x25D8, 0x25CB, 0x25D9, 0x2642, 0x2640, 0x266A, 0x266B, 0x263C, 0x25BA, 0x25C4, 0x2195,
0x203C, 0x00B6, 0x00A7, 0x25AC, 0x21A8, 0x2191, 0x2193, 0x2192, 0x2190, 0x221F, 0x2194, 0x25B2, 0x25BC,
];
/// The OEM code pages [`oem_byte`] decodes.
pub const OEM_CODE_PAGES: [u16; 6] = [437, 737, 850, 852, 857, 866];
/// One byte of an OEM code page in [`OEM_CODE_PAGES`] (any other code page reads as 437) as a
/// char, with the graphic glyphs for 0x01..=0x1F and 0x7F (⌂); 0x00 stays NUL.
pub fn oem_byte(code_page: u16, b: u8) -> char {
let table: &[u16; 128] = match code_page {
737 => &CP737,
850 => &CP850,
852 => &CP852,
857 => &CP857,
866 => &CP866,
_ => &CP437,
};
let u = match b {
0x01..=0x1F => OEM_GLYPHS.get(usize::from(b - 1)).copied(),
0x7F => Some(0x2302),
0x80..=0xFF => table.get(usize::from(b - 0x80)).copied(),
_ => Some(u16::from(b)),
};
u.and_then(|u| char::from_u32(u32::from(u))).unwrap_or('\u{FFFD}')
}
/// The single-byte Windows code pages [`windows_byte`] decodes (1252 included).
pub const SINGLE_BYTE_CODE_PAGES: [u16; 10] = [874, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258];
/// Mac OS Roman, RTF's `\mac` character set.
pub const MAC_ROMAN_CODE_PAGE: u16 = 10000;
/// One byte of a single-byte code page as a char: a Windows one in [`SINGLE_BYTE_CODE_PAGES`] or
/// Mac OS Roman ([`MAC_ROMAN_CODE_PAGE`]); any other code page reads as Windows-1252.
pub fn windows_byte(code_page: u16, b: u8) -> char {
let table: &[u16; 128] = match code_page {
874 => &CP874,
1250 => &CP1250,
1251 => &CP1251,
1253 => &CP1253,
1254 => &CP1254,
1255 => &CP1255,
1256 => &CP1256,
1257 => &CP1257,
1258 => &CP1258,
MAC_ROMAN_CODE_PAGE => &MAC_ROMAN,
_ => return cp1252(b),
};
if b < 0x80 {
return b as char;
}
table.get(usize::from(b - 0x80)).and_then(|u| char::from_u32(u32::from(*u))).unwrap_or('\u{FFFD}')
}
/// Whether [`code_page_byte`] decodes `code_page`: a Windows one ([`SINGLE_BYTE_CODE_PAGES`]), an
/// OEM (DOS) one ([`OEM_CODE_PAGES`]) or Mac OS Roman. Document formats (RTF) name all three.
pub fn is_byte_code_page(code_page: u16) -> bool {
SINGLE_BYTE_CODE_PAGES.contains(&code_page) || OEM_CODE_PAGES.contains(&code_page) || code_page == MAC_ROMAN_CODE_PAGE
}
/// A byte of 8-bit document text in `code_page` (see [`is_byte_code_page`]) as a char. Below 0x80
/// it is ASCII in every code page (text, so an OEM page's glyphs for control codes don't apply).
pub fn code_page_byte(code_page: u16, b: u8) -> char {
if b >= 0x80 && OEM_CODE_PAGES.contains(&code_page) { oem_byte(code_page, b) } else { windows_byte(code_page, b) }
}
/// The East Asian multibyte code pages a document format names with `\ansicpg` or a font's `\cpg`:
/// 932 Shift-JIS (Japanese), 936 GBK (Simplified Chinese), 949 UHC (Korean) and 950 Big5
/// (Traditional Chinese).
pub const MULTIBYTE_CODE_PAGES: [u16; 4] = [932, 936, 949, 950];
/// Whether `code_page` is one of the East Asian multibyte code pages ([`MULTIBYTE_CODE_PAGES`]):
/// its bytes decode as a run, a lead byte together with its trail byte.
pub fn is_multibyte_code_page(code_page: u16) -> bool {
multibyte_encoding_name(code_page).is_some()
}
/// The WHATWG encoding name of an East Asian multibyte code page (see [`MULTIBYTE_CODE_PAGES`]),
/// for `encoding_rs::Encoding::for_label`; `None` for the single-byte code pages, whose bytes
/// decode one at a time, and anything else.
pub fn multibyte_encoding_name(code_page: u16) -> Option<&'static str> {
Some(match code_page {
932 => "shift_jis",
936 => "gbk",
949 => "euc-kr",
950 => "big5",
_ => return None,
})
}
/// Whether a document format can name `code_page` for its text: a single-byte Windows, DOS or Mac
/// one ([`is_byte_code_page`]) or an East Asian multibyte one ([`is_multibyte_code_page`]).
pub fn is_text_code_page(code_page: u16) -> bool {
is_byte_code_page(code_page) || is_multibyte_code_page(code_page)
}
/// The code page of an RTF or Word font charset (`\fcharset`): 204 Russian → 1251, 238 Eastern
/// European → 1250, 77 Mac → 10000, 254 PC / 255 OEM → 437, 128 Japanese → 932, 129 Korean → 949,
/// 134 Simplified Chinese → 936, 136 Traditional Chinese → 950… `None` for the ANSI and default
/// charsets (the document's code page applies) and symbol fonts.
pub fn charset_code_page(charset: i32) -> Option<u16> {
Some(match charset {
77 => MAC_ROMAN_CODE_PAGE,
128 => 932,
129 => 949,
134 => 936,
136 => 950,
161 => 1253,
162 => 1254,
163 => 1258,
177 => 1255,
178 => 1256,
186 => 1257,
204 => 1251,
222 => 874,
238 => 1250,
254 | 255 => 437,
_ => return None,
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn single_byte_code_pages() {
let decode = |cp: u16, bytes: &[u8]| bytes.iter().map(|&b| windows_byte(cp, b)).collect::<String>();
assert_eq!(decode(1251, &[0xD1, 0xF7, 0xE5, 0xF2, 0x20, 0xB9]), "Счет №");
assert_eq!(decode(1251, &[0xA8, 0xB8]), "Ёё");
assert_eq!(decode(1250, &[0xA3, 0xF3, 0x9F]), "Łóź");
assert_eq!(decode(1253, &[0xC1, 0xE2]), "Αβ");
assert_eq!(decode(1252, &[0x80, 0xE9]), "ێ");
assert_eq!(decode(9999, &[0xE9]), "é");
// Mac OS Roman (#1653): accents, the euro sign at 0xDB, typographic quotes.
assert_eq!(decode(10000, &[0x80, 0x8E, 0xDB, 0xF1, 0xF2, 0xF3, 0xD2, 0xD3, 0x41]), "Äé€ÒÚÛ“”A");
assert_eq!(charset_code_page(77), Some(10000));
assert_eq!(charset_code_page(254), Some(437));
assert_eq!(charset_code_page(255), Some(437));
assert_eq!(charset_code_page(204), Some(1251));
assert_eq!(charset_code_page(0), None);
for cp in SINGLE_BYTE_CODE_PAGES {
for b in 0..=255u8 {
let _ = windows_byte(cp, b);
}
}
}
#[test]
fn windows_1255_holam_haser_for_vav() {
assert_eq!(windows_byte(1255, 0xCA), '\u{05BA}');
}
/// #1653: one decoder for document text in any code page RTF names: Windows, DOS or Mac.
#[test]
fn document_text_in_windows_dos_and_mac_code_pages() {
let decode = |cp: u16, bytes: &[u8]| bytes.iter().map(|&b| code_page_byte(cp, b)).collect::<String>();
assert_eq!(decode(437, &[0x82, 0xA0, 0x41, 0x01]), "éáA\u{1}", "DOS text; ASCII stays ASCII");
assert_eq!(decode(850, &[0x9D, 0xE9]), "ØÚ");
assert_eq!(decode(10000, &[0x8E]), "é");
assert_eq!(decode(1251, &[0xC4]), "Д");
for cp in [437, 850, 866, 1250, 1252, 10000] {
assert!(is_byte_code_page(cp), "{cp}");
}
assert!(!is_byte_code_page(932) && !is_byte_code_page(0) && !SINGLE_BYTE_CODE_PAGES.contains(&10000));
}
/// #1653: the East Asian charsets and code pages RTF names are multibyte: their bytes decode
/// as runs (a lead byte and its trail byte together), not one at a time.
#[test]
fn east_asian_code_pages_are_multibyte() {
assert_eq!(charset_code_page(128), Some(932), "Japanese");
assert_eq!(charset_code_page(129), Some(949), "Korean");
assert_eq!(charset_code_page(134), Some(936), "Simplified Chinese");
assert_eq!(charset_code_page(136), Some(950), "Traditional Chinese");
assert_eq!(charset_code_page(255), Some(437), "OEM is single-byte");
assert_eq!(MULTIBYTE_CODE_PAGES, [932, 936, 949, 950]);
for cp in MULTIBYTE_CODE_PAGES {
assert!(is_multibyte_code_page(cp) && is_text_code_page(cp), "{cp}");
assert!(!is_byte_code_page(cp) && !SINGLE_BYTE_CODE_PAGES.contains(&cp), "{cp}");
assert!(multibyte_encoding_name(cp).is_some(), "{cp}");
}
assert!(is_text_code_page(1251) && is_text_code_page(437) && is_text_code_page(10000));
assert!(!is_text_code_page(0) && !is_text_code_page(9999));
}
#[test]
fn oem_code_pages() {
let decode = |cp: u16, bytes: &[u8]| bytes.iter().map(|&b| oem_byte(cp, b)).collect::<String>();
assert_eq!(decode(437, &[0x82, 0xE9, 0x01, 0x7F]), "éΘ☺⌂");
assert_eq!(decode(850, &[0x9B, 0xE9]), "øÚ");
assert_eq!(decode(852, &[0xA5, 0x88, 0xBE, 0xA9]), "ąłżę");
assert_eq!(decode(866, &[0x8F, 0xE0, 0xA8, 0xA2, 0xA5, 0xE2, 0xF0, 0xFC]), "ПриветЁ№");
assert_eq!(decode(737, &[0x80, 0x99, 0xE0]), "Αβω");
assert_eq!(decode(857, &[0x8D, 0x98, 0xA6, 0x9F]), "ıİĞş");
assert_eq!(oem_byte(857, 0xD5), '\u{FFFD}', "unassigned in 857");
assert_eq!(decode(9999, &[0xE9]), "Θ", "unknown pages read as 437");
for cp in OEM_CODE_PAGES {
for b in 0..=255u8 {
let _ = oem_byte(cp, b);
}
}
}
}
+11
View File
@@ -128,6 +128,15 @@ pub struct Settings {
/// draws one that isn't installed in a serif face.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub roman_fonts: Vec<String>,
/// Fonts the font table classes as modern (fixed pitch, `w:family w:val="modern"`): Word draws
/// one that isn't installed, and whose alternate isn't either, in a serif face (#2702). Not
/// the East Asian ones (`w:charset` 80, 81, 86, 88: MS Gothic, SimHei…).
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub modern_fonts: Vec<String>,
/// Alternate names the document's font table gives its fonts (`w:font/w:altName`), as (font,
/// alternate): Word draws a font that isn't installed in its alternate when that one is.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub font_alt_names: Vec<(String, String)>,
/// `w:doNotUseHTMLParagraphAutoSpacing`: the space between two paragraphs is the first's space
/// after plus the second's space before, not the larger of the two.
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
@@ -325,6 +334,8 @@ impl Default for Settings {
protect_hash: Vec::new(),
compat_mode: COMPAT_MODE_CURRENT,
roman_fonts: Vec::new(),
modern_fonts: Vec::new(),
font_alt_names: Vec::new(),
sum_paragraph_spacing: false,
do_not_expand_shift_return: false,
allow_space_of_same_style_in_table: false,
+53 -17
View File
@@ -585,7 +585,7 @@ impl StyleSheet {
outline_level: Some(i as u8),
..para()
},
chr: CharProps { font: Some(font.into()), size: Some(size), italic, color: color(c), ..chr() },
chr: CharProps { font: Some(font.into()), size: Some(size), italic, italic_cs: italic, color: color(c), ..chr() },
priority: Some(9 + n as u32),
quick: n <= 3,
..p_style()
@@ -623,17 +623,27 @@ impl StyleSheet {
builtin: true,
..Default::default()
};
add(c_style("SubtleEmphasis", "Subtle Emphasis", 19, CharProps { italic: Some(true), color: color(Rgb(0x40, 0x40, 0x40)), ..chr() }));
add(c_style("Emphasis", "Emphasis", 20, CharProps { italic: Some(true), ..chr() }));
add(c_style("IntenseEmphasis", "Intense Emphasis", 21, CharProps { italic: Some(true), color: color(HEADING_BLUE), ..chr() }));
add(c_style("Strong", "Strong", 22, CharProps { bold: Some(true), ..chr() }));
add(c_style(
"SubtleEmphasis",
"Subtle Emphasis",
19,
CharProps { italic: Some(true), italic_cs: Some(true), color: color(Rgb(0x40, 0x40, 0x40)), ..chr() },
));
add(c_style("Emphasis", "Emphasis", 20, CharProps { italic: Some(true), italic_cs: Some(true), ..chr() }));
add(c_style(
"IntenseEmphasis",
"Intense Emphasis",
21,
CharProps { italic: Some(true), italic_cs: Some(true), color: color(HEADING_BLUE), ..chr() },
));
add(c_style("Strong", "Strong", 22, CharProps { bold: Some(true), bold_cs: Some(true), ..chr() }));
add(Style {
id: "Quote".into(),
name: "Quote".into(),
based_on: Some("Normal".into()),
next: Some("Normal".into()),
para: ParaProps { align: Some(Align::Center), space_before: Some(8.0), ..para() },
chr: CharProps { italic: Some(true), color: color(Rgb(0x40, 0x40, 0x40)), ..chr() },
chr: CharProps { italic: Some(true), italic_cs: Some(true), color: color(Rgb(0x40, 0x40, 0x40)), ..chr() },
priority: Some(29),
quick: true,
..p_style()
@@ -653,7 +663,7 @@ impl StyleSheet {
borders: Some(Borders { top: Some(blue_line), bottom: Some(blue_line), ..Default::default() }),
..para()
},
chr: CharProps { italic: Some(true), color: color(HEADING_BLUE), ..chr() },
chr: CharProps { italic: Some(true), italic_cs: Some(true), color: color(HEADING_BLUE), ..chr() },
priority: Some(30),
quick: true,
..p_style()
@@ -663,9 +673,14 @@ impl StyleSheet {
"IntenseReference",
"Intense Reference",
32,
CharProps { bold: Some(true), small_caps: Some(true), spacing: Some(0.25), color: color(HEADING_BLUE), ..chr() },
CharProps { bold: Some(true), bold_cs: Some(true), small_caps: Some(true), spacing: Some(0.25), color: color(HEADING_BLUE), ..chr() },
));
add(c_style(
"BookTitle",
"Book Title",
33,
CharProps { bold: Some(true), bold_cs: Some(true), italic: Some(true), italic_cs: Some(true), spacing: Some(0.25), ..chr() },
));
add(c_style("BookTitle", "Book Title", 33, CharProps { bold: Some(true), italic: Some(true), spacing: Some(0.25), ..chr() }));
add(Style {
id: "ListParagraph".into(),
name: "List Paragraph".into(),
@@ -685,7 +700,7 @@ impl StyleSheet {
based_on: Some("Normal".into()),
next: Some("Normal".into()),
para: ParaProps { space_after: Some(10.0), line_spacing: Some(LineSpacing::Multiple(1.0)), ..para() },
chr: CharProps { italic: Some(true), size: Some(9.0), color: color(Rgb(0x0E, 0x28, 0x41)), ..chr() },
chr: CharProps { italic: Some(true), italic_cs: Some(true), size: Some(9.0), color: color(Rgb(0x0E, 0x28, 0x41)), ..chr() },
priority: Some(35),
..p_style()
});
@@ -925,9 +940,9 @@ fn table_styles() -> Vec<Style> {
"Plain Table 1",
TableStyleParts {
borders: Some(Borders::all(thin(Rgb(0xBF, 0xBF, 0xBF)))),
header_chr: CharProps { bold: Some(true), ..CharProps::default() },
header_chr: CharProps { bold: Some(true), bold_cs: Some(true), ..CharProps::default() },
band_fill: Some(Rgb(0xF2, 0xF2, 0xF2)),
first_col_chr: CharProps { bold: Some(true), ..CharProps::default() },
first_col_chr: CharProps { bold: Some(true), bold_cs: Some(true), ..CharProps::default() },
..Default::default()
},
),
@@ -936,8 +951,8 @@ fn table_styles() -> Vec<Style> {
"Grid Table 1 Light",
TableStyleParts {
borders: Some(Borders::all(thin(Rgb(0x99, 0x99, 0x99)))),
header_chr: CharProps { bold: Some(true), ..CharProps::default() },
first_col_chr: CharProps { bold: Some(true), ..CharProps::default() },
header_chr: CharProps { bold: Some(true), bold_cs: Some(true), ..CharProps::default() },
first_col_chr: CharProps { bold: Some(true), bold_cs: Some(true), ..CharProps::default() },
..Default::default()
},
),
@@ -949,9 +964,9 @@ fn table_styles() -> Vec<Style> {
TableStyleParts {
borders: Some(Borders::all(thin(light))),
header_fill: Some(dark),
header_chr: CharProps { bold: Some(true), color: Some(TextColor::Rgb(Rgb::WHITE)), ..CharProps::default() },
header_chr: CharProps { bold: Some(true), bold_cs: Some(true), color: Some(TextColor::Rgb(Rgb::WHITE)), ..CharProps::default() },
band_fill: Some(light),
first_col_chr: CharProps { bold: Some(true), ..CharProps::default() },
first_col_chr: CharProps { bold: Some(true), bold_cs: Some(true), ..CharProps::default() },
..Default::default()
},
));
@@ -961,7 +976,7 @@ fn table_styles() -> Vec<Style> {
TableStyleParts {
borders: Some(Borders::box_(thin(dark))),
header_fill: Some(dark),
header_chr: CharProps { bold: Some(true), color: Some(TextColor::Rgb(Rgb::WHITE)), ..CharProps::default() },
header_chr: CharProps { bold: Some(true), bold_cs: Some(true), color: Some(TextColor::Rgb(Rgb::WHITE)), ..CharProps::default() },
total_border_top: Some(thin(dark)),
..Default::default()
},
@@ -1113,6 +1128,7 @@ mod tests {
assert!(s.table_style("RedGrid").is_some());
}
/// Derived style ids strip non-alphanumeric characters and append a counter when taken.
#[test]
fn new_ids_unique() {
let s = StyleSheet::builtin();
@@ -1120,4 +1136,24 @@ mod tests {
assert_eq!(s.new_id("Normal"), "Normal1");
assert_eq!(s.new_id("!!!"), "Style");
}
/// #4198: Word and LibreOffice make a complex-script (Arabic, Hebrew…) run bold or italic only
/// through `w:bCs` / `w:iCs`, so every built-in style and table-style part that sets `bold` or
/// `italic` sets `bold_cs` or `italic_cs` alongside it.
#[test]
fn builtin_styles_pair_complex_script_bold_and_italic() {
let s = StyleSheet::builtin();
for st in &s.styles {
assert_eq!(st.chr.bold_cs, st.chr.bold, "{}: chr.bold", st.id);
assert_eq!(st.chr.italic_cs, st.chr.italic, "{}: chr.italic", st.id);
for part in TablePart::ALL {
let f = st.table_part(part);
assert_eq!(f.chr.bold_cs, f.chr.bold, "{} {part:?}: bold", st.id);
assert_eq!(f.chr.italic_cs, f.chr.italic, "{} {part:?}: italic", st.id);
}
}
let blue = s.get("GridTable4AccentBlue").unwrap();
assert_eq!((blue.table_part(TablePart::FirstRow).chr.bold, blue.table_part(TablePart::FirstRow).chr.bold_cs), (Some(true), Some(true)));
assert_eq!((blue.table_part(TablePart::FirstCol).chr.bold, blue.table_part(TablePart::FirstCol).chr.bold_cs), (Some(true), Some(true)));
}
}
+4
View File
@@ -144,6 +144,7 @@ fn parts_and_media() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn many_distinct_media_add_in_linear_time() {
// An import with tens of thousands of small distinct images (same length, differing only at
// the end) used to compare each against every earlier one.
@@ -645,6 +646,7 @@ fn para_of_objects(objs: Vec<InlineObject>) -> Paragraph {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn object_walks_are_linear_in_the_paragraph() {
// #3614: loops of object_offsets + object_at recounted the objects before each one, so a
// paragraph of 40,000 markers took seconds per walk.
@@ -691,6 +693,7 @@ fn object_walks_are_linear_in_the_paragraph() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn deleting_across_many_lone_control_markers_is_not_quadratic() {
// A millisecond now; the old delete took 18 s at 1,500 markers and 114 s at 3,000.
let n = 3_000;
@@ -725,6 +728,7 @@ fn control_markers_kept_are_sorted() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn lone_table_open_markers_linear_time() {
let mut d = Document::new();
let mut t = Table::new(1, 1, 100.0);
+145 -119
View File
@@ -1,131 +1,21 @@
//! A bytes-only check of a ZIP package's declared entry count before the `zip` crate opens it
//! (#3660). Every reader of `.docx`, `.odt` and spreadsheet packages calls [`check`] first.
//!
//! `zip::ZipArchive::new` reserves room for, and parses, as many central directory headers as the
//! end record declares before its caller can look at `len()`. A ZIP64 end record can declare
//! millions of entries for a file of a few hundred megabytes (only 46 bytes each have to fit), and
//! each parsed entry costs several times that in memory. Here the end records are found the way
//! `zip` finds them (APPNOTE.TXT §4.3.14 to §4.3.16), and a package declaring more than the
//! reader's limit is refused before `zip` parses anything. Anything else that is wrong is left for
//! `zip` to report.
const EOCD: &[u8; 4] = b"PK\x05\x06";
const EOCD64_LOCATOR: &[u8; 4] = b"PK\x06\x07";
const EOCD64: &[u8; 4] = b"PK\x06\x06";
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
/// End records looked at, at most. A package has one; a few more can turn up inside stored data.
const MAX_END_RECORDS: usize = 16;
//! A bytes-only check of a ZIP package's declared entry count, run before anything opens the ZIP
//! (#3660). The logic lives in `craft-zip`; this wraps its error as a `String`.
/// Refuse `bytes` when the end record `zip` would use declares more than `max_entries` entries.
///
/// `zip` searches the file backwards for an end of central directory record. A ZIP64 one points
/// through its locator at the ZIP64 end record, which `zip` looks for from the locator's offset
/// on: every such record there is checked, and as `zip` may fall back to an earlier end record
/// when one doesn't parse, the search goes on. A plain one is used when it declares no entries, or
/// a central directory header lies between its directory offset and itself: it is the last one
/// looked at.
pub fn check(bytes: &[u8], max_entries: u64) -> Result<(), String> {
let u16_at = |at: usize| bytes.get(at..at.checked_add(2)?).map(|b| u16::from_le_bytes([b[0], b[1]]));
let u32_at = |at: usize| bytes.get(at..at.checked_add(4)?).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]));
let u64_at = |at: usize| bytes.get(at..at.checked_add(8)?).and_then(|b| <[u8; 8]>::try_from(b).ok()).map(u64::from_le_bytes);
let refuse = |n: u64| -> Result<(), String> {
if n > max_entries { Err(format!("the package declares {n} entries, more than {max_entries}")) } else { Ok(()) }
};
let mut end = bytes.len();
let mut seen = 0;
while let Some(at) = bytes.get(..end).and_then(|b| b.windows(4).rposition(|w| w == EOCD)) {
end = at;
seen += 1;
if seen > MAX_END_RECORDS {
return Err(format!("more than {MAX_END_RECORDS} end of central directory records"));
}
// `zip` skips a record whose comment runs past the end of the file.
let Some(comment) = u16_at(at + 20) else { continue };
if at + 22 + usize::from(comment) > bytes.len() {
continue;
}
let (Some(on_disk), Some(total), Some(cd_offset)) = (u16_at(at + 8), u16_at(at + 10), u32_at(at + 16)) else { continue };
let zip64 = total == u16::MAX || cd_offset == u32::MAX;
if zip64
&& let Some(locator) = at.checked_sub(20)
&& bytes.get(locator..locator + 4) == Some(EOCD64_LOCATOR.as_slice())
{
let (Some(eocd64), Some(disks)) = (u64_at(locator + 8), u32_at(locator + 16)) else { continue };
let Some(from) = usize::try_from(eocd64).ok().filter(|&f| f < locator) else { continue };
if disks > 1 {
continue;
}
let records = bytes.get(from..locator).unwrap_or_default().windows(4).enumerate().filter(|(_, w)| *w == EOCD64);
for (i, _) in records {
let p = from + i;
refuse(u64_at(p + 24).unwrap_or(0).max(u64_at(p + 32).unwrap_or(0)))?;
}
continue;
}
if total == 0 {
return Ok(());
}
let cd = cd_offset as usize;
if cd < at && bytes.get(cd..at).is_some_and(|b| b.windows(4).any(|w| w == CENTRAL_HEADER)) {
return refuse(u64::from(on_disk.max(total)));
}
}
Ok(())
craft_zip::check_entry_count(bytes, max_entries).map_err(|e| e.to_string())
}
/// A package of one empty stored file, `a`, whose central directory lists it `entries` times (at
/// most as many as fit in 64 MiB) under a ZIP64 end record that declares them all: `zip` 3.0.0
/// parses and keeps every one of them before its caller can count them. Regression input for the
/// readers' tests: built only for tests, this crate's and (through the `test-support` feature,
/// which they turn on as a dev-dependency) the readers', never into the app.
/// A ZIP64 package declaring `entries` entries: regression input for the readers' tests.
#[cfg(any(test, feature = "test-support"))]
pub fn zip64_entry_count_sample(entries: u64) -> Vec<u8> {
// Local file header (APPNOTE.TXT §4.3.7): version 10, stored, no data, name "a".
let mut f = b"PK\x03\x04".to_vec();
f.extend_from_slice(&[10, 0, 0, 0, 0, 0, 0, 0, 0x21, 0]); // version, flags, method, time, date
f.extend_from_slice(&[0; 12]); // crc, sizes
f.extend_from_slice(&[1, 0, 0, 0]); // name and extra lengths
f.push(b'a');
let cd_at = f.len() as u64;
// Its central header (§4.3.12), 47 bytes.
let mut header = CENTRAL_HEADER.to_vec();
header.extend_from_slice(&[10, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0x21, 0]); // versions, flags, method, time, date
header.extend_from_slice(&[0; 12]); // crc, sizes
header.extend_from_slice(&[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); // name, extra, comment, disk, attributes
header.extend_from_slice(&[0; 4]); // local header offset
header.push(b'a');
let count = entries.min((64 << 20) / header.len() as u64);
for _ in 0..count {
f.extend_from_slice(&header);
}
let cd_len = f.len() as u64 - cd_at;
let eocd64_at = f.len() as u64;
// ZIP64 end of central directory record (§4.3.14), 56 bytes.
f.extend_from_slice(EOCD64);
f.extend_from_slice(&44u64.to_le_bytes()); // size of the rest of the record
f.extend_from_slice(&[45, 0, 45, 0]); // versions
f.extend_from_slice(&[0; 8]); // this disk, disk with the central directory
f.extend_from_slice(&entries.to_le_bytes()); // entries on this disk
f.extend_from_slice(&entries.to_le_bytes()); // entries
f.extend_from_slice(&cd_len.to_le_bytes());
f.extend_from_slice(&cd_at.to_le_bytes());
// Its locator (§4.3.15), 20 bytes.
f.extend_from_slice(EOCD64_LOCATOR);
f.extend_from_slice(&0u32.to_le_bytes());
f.extend_from_slice(&eocd64_at.to_le_bytes());
f.extend_from_slice(&1u32.to_le_bytes());
// End of central directory record (§4.3.16), 22 bytes, each count and offset "see ZIP64".
f.extend_from_slice(EOCD);
f.extend_from_slice(&[0; 4]);
f.extend_from_slice(&[0xFF; 12]);
f.extend_from_slice(&[0; 2]);
f
}
pub use craft_zip::zip64_entry_count_sample;
#[cfg(test)]
mod tests {
use super::{CENTRAL_HEADER, EOCD, check, zip64_entry_count_sample};
use super::{check, zip64_entry_count_sample};
const EOCD: &[u8; 4] = b"PK\x05\x06";
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
/// A plain package's tail: one central header at 0, then an end record declaring `entries`.
fn plain(entries: u16, comment: &[u8]) -> Vec<u8> {
@@ -188,4 +78,140 @@ mod tests {
let f: Vec<u8> = (0..20).flat_map(|_| EOCD.iter().copied().chain(record)).collect();
assert!(check(&f, 20_000).is_err_and(|e| e.contains("end of central directory records")));
}
/// The pre-cutover implementation, kept as a differential oracle.
mod old {
const EOCD: &[u8; 4] = b"PK\x05\x06";
const EOCD64_LOCATOR: &[u8; 4] = b"PK\x06\x07";
const EOCD64: &[u8; 4] = b"PK\x06\x06";
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
/// End records looked at, at most. A package has one; a few more can turn up inside stored data.
const MAX_END_RECORDS: usize = 16;
/// Refuse `bytes` when the end record `zip` would use declares more than `max_entries` entries.
///
/// `zip` searches the file backwards for an end of central directory record. A ZIP64 one points
/// through its locator at the ZIP64 end record, which `zip` looks for from the locator's offset
/// on: every such record there is checked, and as `zip` may fall back to an earlier end record
/// when one doesn't parse, the search goes on. A plain one is used when it declares no entries, or
/// a central directory header lies between its directory offset and itself: it is the last one
/// looked at.
pub fn check(bytes: &[u8], max_entries: u64) -> Result<(), String> {
let u16_at = |at: usize| bytes.get(at..at.checked_add(2)?).map(|b| u16::from_le_bytes([b[0], b[1]]));
let u32_at = |at: usize| bytes.get(at..at.checked_add(4)?).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]));
let u64_at = |at: usize| bytes.get(at..at.checked_add(8)?).and_then(|b| <[u8; 8]>::try_from(b).ok()).map(u64::from_le_bytes);
let refuse = |n: u64| -> Result<(), String> {
if n > max_entries { Err(format!("the package declares {n} entries, more than {max_entries}")) } else { Ok(()) }
};
let mut end = bytes.len();
let mut seen = 0;
while let Some(at) = bytes.get(..end).and_then(|b| b.windows(4).rposition(|w| w == EOCD)) {
end = at;
seen += 1;
if seen > MAX_END_RECORDS {
return Err(format!("more than {MAX_END_RECORDS} end of central directory records"));
}
// `zip` skips a record whose comment runs past the end of the file.
let Some(comment) = u16_at(at + 20) else { continue };
if at + 22 + usize::from(comment) > bytes.len() {
continue;
}
let (Some(on_disk), Some(total), Some(cd_offset)) = (u16_at(at + 8), u16_at(at + 10), u32_at(at + 16)) else { continue };
let zip64 = total == u16::MAX || cd_offset == u32::MAX;
if zip64
&& let Some(locator) = at.checked_sub(20)
&& bytes.get(locator..locator + 4) == Some(EOCD64_LOCATOR.as_slice())
{
let (Some(eocd64), Some(disks)) = (u64_at(locator + 8), u32_at(locator + 16)) else { continue };
let Some(from) = usize::try_from(eocd64).ok().filter(|&f| f < locator) else { continue };
if disks > 1 {
continue;
}
let records = bytes.get(from..locator).unwrap_or_default().windows(4).enumerate().filter(|(_, w)| *w == EOCD64);
for (i, _) in records {
let p = from + i;
refuse(u64_at(p + 24).unwrap_or(0).max(u64_at(p + 32).unwrap_or(0)))?;
}
continue;
}
if total == 0 {
return Ok(());
}
let cd = cd_offset as usize;
if cd < at && bytes.get(cd..at).is_some_and(|b| b.windows(4).any(|w| w == CENTRAL_HEADER)) {
return refuse(u64::from(on_disk.max(total)));
}
}
Ok(())
}
}
fn sentinel_sample() -> Vec<u8> {
// ZIP64 sample whose plain end record claims one entry and a real directory offset but
// sets the directory size to u32::MAX, the ZIP64 marker `zip` honours.
let mut f = zip64_entry_count_sample(30_000);
let n = f.len();
f[n - 14..n - 12].copy_from_slice(&1u16.to_le_bytes());
f[n - 12..n - 10].copy_from_slice(&1u16.to_le_bytes());
f[n - 10..n - 6].copy_from_slice(&u32::MAX.to_le_bytes());
f[n - 6..n - 2].copy_from_slice(&31u32.to_le_bytes());
f
}
fn corpus() -> Vec<Vec<u8>> {
let mut seed = 0x2545_F491_4F6C_DD1Du64;
let mut rnd = |n: usize| -> Vec<u8> {
(0..n)
.map(|_| {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
(seed >> 24) as u8
})
.collect()
};
let mut truncated = zip64_entry_count_sample(10);
truncated.truncate(truncated.len() - 5);
vec![
vec![],
plain(3, b""),
plain(30_000, b"c"),
zip64_entry_count_sample(1),
zip64_entry_count_sample(10),
zip64_entry_count_sample(30_000),
zip64_entry_count_sample(5_000_000),
truncated,
rnd(1),
rnd(100),
rnd(5000),
]
}
#[test]
fn agrees_with_the_old_implementation_except_for_the_sentinel() {
for (i, input) in corpus().iter().enumerate() {
for max in [0u64, 9, 10, 20_000] {
assert_eq!(check(input, max).is_ok(), old::check(input, max).is_ok(), "input {i}, max {max}");
}
}
}
#[test]
fn a_directory_size_sentinel_alone_cannot_bypass_the_count_limit() {
let f = sentinel_sample();
assert!(old::check(&f, 20_000).is_ok(), "the old check missed it");
assert!(check(&f, 20_000).is_err_and(|e| e.contains("30000 entries")));
}
#[test]
fn no_truncation_of_a_zip64_package_panics() {
for f in [zip64_entry_count_sample(10), sentinel_sample()] {
for end in 0..=f.len().min(4096) {
let _ = check(&f[..end], 5);
}
for start in f.len().saturating_sub(300)..f.len() {
let _ = check(&f[start..], 5);
}
}
}
}
@@ -27,6 +27,7 @@ fn cpu_cost(mut f: impl FnMut()) -> f64 {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn many_comments_save_without_quadratic_id_searches() {
for imported in [true, false] {
let small = comments(4_000, imported);
@@ -74,6 +74,7 @@ fn cost_ratio(scale: usize, mut small: impl FnMut(), mut big: impl FnMut()) -> (
/// looked for each paragraph's markers in the whole set of markers, once per paragraph, so a
/// document with 32,000 controls took four seconds to save where it takes a tenth of one now.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn a_document_with_many_inline_content_controls_saves_in_linear_time() {
let scale = 4;
let small = inline_controls(8_000);
@@ -94,6 +95,7 @@ fn a_document_with_many_inline_content_controls_saves_in_linear_time() {
/// The same for the controls written around whole paragraphs: the writer compared every control
/// with every other one to find the ones that clash, so the save was quadratic in their number.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn a_document_with_many_block_content_controls_saves_in_linear_time() {
let scale = 4;
let small = block_controls(8_000);
+27 -2
View File
@@ -1280,10 +1280,12 @@ fn anchor_stacking_order_is_read_and_kept() {
paras(d)[0].objects.iter().filter_map(|o| if let InlineObject::Shape { float, .. } = o { Some(float.z) } else { None }).collect()
};
let d = read_body(&body);
assert_eq!(zs(&d), [251660288, 251657214, 0]);
// The shared reader keeps one more than the file's `relativeHeight` (so a file's 0 is the
// lowest order, not "none") and the writer writes one less.
assert_eq!(zs(&d), [251660289, 251657215, 0]);
let again = wordcraft_docx::read(&wordcraft_docx::write(&d).unwrap()).unwrap();
let z = zs(&again);
assert_eq!(&z[..2], &[251660288, 251657214]);
assert_eq!(&z[..2], &[251660289, 251657215]);
assert!(z[2] > 0, "an object without one gets an order on saving: {z:?}");
}
@@ -4633,6 +4635,29 @@ fn our_banded_table_styles_write_their_band_sizes() {
assert_eq!(back.styles.table_style("GridTable4AccentBlue").map(|t| t.parts.band_size), Some(Some(1)));
}
/// #4198: built-in styles and table styles write `w:bCs` beside `w:b` and `w:iCs` beside `w:i` in
/// `word/styles.xml`, so Word and LibreOffice draw Arabic and Hebrew headers and emphasis bold or
/// italic just as WordCraft does.
#[test]
fn builtin_styles_write_bcs_and_ics_with_b_and_i() {
let d = Document::new();
let xml = text(&unzip(&wordcraft_docx::write(&d).unwrap()), "word/styles.xml");
let start = xml.find(r#"w:styleId="GridTable4AccentBlue""#).expect("Grid Table 4 – Blue is written");
let style = &xml[start..start + xml[start..].find("</w:style>").unwrap()];
assert!(
style.contains(r#"<w:tblStylePr w:type="firstRow"><w:rPr><w:b/><w:bCs/><w:color w:val="FFFFFF"/></w:rPr>"#),
"firstRow has w:bCs beside w:b: {style}"
);
assert!(
style.contains(r#"<w:tblStylePr w:type="firstCol"><w:rPr><w:b/><w:bCs/></w:rPr></w:tblStylePr>"#),
"firstCol has w:bCs beside w:b: {style}"
);
for rpr in xml.split("<w:rPr>").skip(1).filter_map(|s| s.split("</w:rPr>").next()) {
assert_eq!(rpr.contains("<w:b/>"), rpr.contains("<w:bCs/>"), "unpaired w:b / w:bCs in {rpr}");
assert_eq!(rpr.contains("<w:i/>"), rpr.contains("<w:iCs/>"), "unpaired w:i / w:iCs in {rpr}");
}
}
#[test]
fn vml_shape_wrap_side_and_anchor_case_insensitivity() {
let body = r##"<w:p>
+2
View File
@@ -152,6 +152,7 @@ fn many_fields_and_huge_attribute_values() {
/// A paragraph with 2,000,000 attributes deflates to ~15 KB. Opening it is refused with a
/// limit error, or the extras are dropped and the text survives (#3702, WC-06).
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn millions_of_attributes_on_one_paragraph_are_capped() {
let attrs: String = (0..2_000_000).map(|i| format!(" a{i}=\"\"")).collect();
let x = format!("<w:document {W_NS}><w:body><w:p{attrs}><w:r><w:t>kept</w:t></w:r></w:p></w:body></w:document>");
@@ -222,6 +223,7 @@ fn malformed_or_hostile_bibliography_never_panics() {
/// A file with a huge number of bibliography sources (duplicates and distinct ones) reads in
/// linear time and keeps at most 10,000 of them.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn hostile_bibliography_source_count_is_capped() {
let doc = format!("<w:document {W_NS}><w:body><w:p><w:r><w:t>body</w:t></w:r></w:p></w:body></w:document>");
let mut sources = String::from(r#"<b:Sources xmlns:b="http://schemas.openxmlformats.org/officeDocument/2006/bibliography" StyleName="MLA">"#);
+4
View File
@@ -48,6 +48,10 @@ craft-font-catalog = { workspace = true }
# The local time zone's offset for Insert ▸ Date & Time and date fields (#1707); the web asks the browser's Date.
chrono = { workspace = true, features = ["clock"] }
[target.'cfg(all(unix, not(target_os = "macos"), not(target_arch = "wasm32")))'.dependencies]
# Speech Dispatcher's wire framing for Read Aloud (speech/ssip.rs).
craft-ssip = { workspace = true }
[target.'cfg(unix)'.dependencies]
# O_NONBLOCK, so a path swapped for a named pipe can't block `open` (#3666).
libc = { workspace = true }
+1
View File
@@ -871,6 +871,7 @@ fn citation_source_lookups_keep_first_duplicates_case_and_style_order() {
#[cfg(not(target_arch = "wasm32"))]
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn citation_refresh_source_lookup_work_scales_with_document_size() {
fn doc(n: usize) -> Session {
let mut p = Paragraph::new();
+24 -7
View File
@@ -20,9 +20,12 @@ pub const SNAP: f32 = 12.0;
/// Largest offset a routed connector gets, points.
const MAX_OFFSET: f32 = 4000.0;
/// `shape.connect` is offered for what it runs on: a floating connector or line in the body text.
fn has_connector(s: &Session) -> Option<&'static str> {
match super::objects::selected(s) {
Some((_, InlineObject::Shape { kind, .. })) if kind.is_connector() => None,
Some((pos, InlineObject::Shape { kind, float, .. })) if kind.is_open() => {
(pos.story != StoryRef::Body || float.wrap == Wrap::Inline).then_some("only floating connectors in the body text can be glued")
}
_ => Some("select a connector first"),
}
}
@@ -31,7 +34,7 @@ pub fn specs() -> Vec<CommandSpec> {
vec![
CommandSpec::new("shape.connect", "Connect Shapes", "Shape Format › Insert Shapes", connect)
.params(
r#"{"start"?: end, "end"?: end} end: {"path": [..], "off": n, "site"?: n} (glue to that body shape's connection site; nearest one when omitted) | {"x": pt, "y": pt, "page"?: n} (drag the end there: it glues to a shape's site within 12 pt, else stays free there) | null (unglue)"#,
r#"{"start"?: end, "end"?: end} end: {"path": [..], "off": n, "site"?: n} (glue to that body shape's connection site; nearest one when omitted) | {"x": pt, "y": pt, "page"?: n} (drag the end there: it glues to a shape's site within 12 pt, else stays free there) | null (unglue). A plain line's ends only move to points: lines don't glue"#,
)
.when(has_connector),
]
@@ -186,13 +189,23 @@ fn ensure_id(s: &mut Session, pos: &Pos) -> Result<u32, CmdError> {
Ok(id)
}
/// `shape.connect`: glue, move or unglue the selected connector's ends.
/// A floating line or connector's start and end on its page: its frame's ends (flips
/// included), turned with the frame by `deg` about its centre.
pub fn page_ends(r: wordcraft_geom::Rect, flip_h: bool, flip_v: bool, deg: f32) -> ((f32, f32), (f32, f32)) {
let (a, b) = ends(r.x, r.y, r.w, r.h, flip_h, flip_v);
let turn = wordcraft_geom::Spin::new(deg, false, false);
let (cx, cy) = (r.x + r.w / 2.0, r.y + r.h / 2.0);
(turn.apply(cx, cy, a.0, a.1), turn.apply(cx, cy, b.0, b.1))
}
/// `shape.connect`: glue, move or unglue the selected connector's ends, or move a line's.
fn connect(s: &mut Session, v: &Value) -> CmdResult {
let (pos, obj) = super::objects::selected(s).ok_or_else(|| CmdError::Disabled("select a connector first".into()))?;
let InlineObject::Shape { kind, float, .. } = &obj else { return Err(CmdError::Disabled("select a connector first".into())) };
if !kind.is_connector() {
if !kind.is_open() {
return Err(CmdError::Disabled("select a connector first".into()));
}
let line = !kind.is_connector();
if pos.story != StoryRef::Body || float.wrap == Wrap::Inline {
return Err(CmdError::Disabled("only floating connectors in the body text can be glued".into()));
}
@@ -200,7 +213,7 @@ fn connect(s: &mut Session, v: &Value) -> CmdResult {
let layout = s.layout();
let hit = layout.object(&pos, s.page_hint).ok_or_else(|| CmdError::Failed("the connector isn't laid out".into()))?;
let r = hit.rect;
let (mut p0, mut p1) = ends(r.x, r.y, r.w, r.h, fh, fv);
let (mut p0, mut p1) = page_ends(r, fh, fv, hit.spin.deg);
let mut glue: [Option<Option<ConnEnd>>; 2] = [None, None];
for (i, key) in ["start", "end"].into_iter().enumerate() {
let Some(e) = v.get(key) else { continue };
@@ -209,6 +222,9 @@ fn connect(s: &mut Session, v: &Value) -> CmdResult {
continue;
}
let here = if i == 0 { p0 } else { p1 };
if line && e.get("path").is_some() {
return Err(CmdError::Params(format!("`{key}`: a line's ends move to points; lines don't glue to shapes")));
}
let (target, site) = if e.get("path").is_some() {
let at = body_pos(e).ok_or_else(|| CmdError::Params(format!("`{key}`: no shape there")))?;
let th = layout
@@ -234,9 +250,9 @@ fn connect(s: &mut Session, v: &Value) -> CmdResult {
if page != hit.page {
return Err(CmdError::Params("a connector's ends stay on its page".into()));
}
// The nearest site of a shape (not a connector) within reach.
// The nearest site of a shape (not a connector) within reach; a line's end stays free.
let mut best: Option<(Pos, u32, f32)> = None;
if let Some(pg) = layout.pages.get(page) {
if !line && let Some(pg) = layout.pages.get(page) {
for it in &pg.items {
let wordcraft_layout::Placed::Object { rect, spin, story: StoryRef::Body, path, off, .. } = it else { continue };
let at = Pos { story: StoryRef::Body, path: path.clone(), off: *off };
@@ -316,6 +332,7 @@ mod tests {
/// Drawing commands must not search every page again for each glued connector.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn rerouting_many_pages_does_not_rescan_all_objects_per_connector() {
use wordcraft_doc::{Document, Paragraph, para_block};
let mut doc = Document::new();
+88 -2
View File
@@ -651,6 +651,77 @@ fn info(s: &mut Session, _: &Value) -> CmdResult {
}))
}
/// Most fields [`story_fields`] lists for one story, and how deep it goes into nested tables.
const MAX_STORY_FIELDS: usize = 200;
const MAX_STORY_DEPTH: usize = 8;
/// The fields of a story's paragraphs, tables included: the code, the result stored with it, and
/// whether the page shows a value of its own (#4467). A header or footer's text carries the
/// stored result of a page field (`1` for a PAGE, the page count when it was inserted for a
/// NUMPAGES); each page shows its own, as the layout draws it. A STYLEREF counts too: the layout
/// looks its text up from the page it lands on. False when a limit cut the list short.
fn story_fields(blocks: &wordcraft_doc::Blocks, depth: usize, out: &mut Vec<Value>) -> bool {
if depth > MAX_STORY_DEPTH {
return false;
}
for b in blocks {
match &**b {
Block::Para(p) => {
for o in &p.objects {
if let wordcraft_doc::para::InlineObject::Field { instr, result, .. } = o {
if out.len() >= MAX_STORY_FIELDS {
return false;
}
out.push(field_json(instr, result));
}
}
}
Block::Table(t) => {
for cell in t.rows.iter().flat_map(|r| &r.cells) {
if !story_fields(&cell.blocks, depth + 1, out) {
return false;
}
}
}
}
}
true
}
/// The range fields of `story` (a `FieldStart` … `FieldEnd` pair: what the DOCX reader makes of a
/// field whose result has tracked changes of its own or spans paragraphs, and of every TOC),
/// listed as [`story_fields`] lists inline ones, after them; the result is the text between the
/// markers as it is, at most [`MAX_RANGE_RESULT`] characters of it (`resultTruncated` when it is
/// cut: a bibliography or a long TOC). False when a limit cut the list short.
fn range_fields(doc: &Document, story: StoryRef, out: &mut Vec<Value>) -> bool {
for r in doc.field_ranges(story) {
if out.len() >= MAX_STORY_FIELDS {
return false;
}
let from = wordcraft_doc::Pos { off: r.start.off.saturating_add(wordcraft_doc::para::OBJ.len_utf8()), ..r.start.clone() };
let text = doc.copy_range(&from, &r.end).plain_text();
let mut chars = text.chars().filter(|c| *c != wordcraft_doc::para::OBJ);
let result: String = chars.by_ref().take(MAX_RANGE_RESULT).collect();
let mut field = field_json(&r.instr, &result);
if chars.next().is_some() {
field["resultTruncated"] = json!(true);
}
out.push(field);
}
true
}
/// Longest result `document.inspect` gives a range field, in characters.
const MAX_RANGE_RESULT: usize = 2000;
/// A field as `document.inspect` lists it: its code, its stored result, and whether each page
/// shows a value of its own.
fn field_json(instr: &str, result: &str) -> Value {
let name = wordcraft_layout::fields::field_name(instr);
let per_page = matches!(name.as_str(), "PAGE" | "NUMPAGES" | "SECTIONPAGES" | "SECTION" | "PAGEREF" | "STYLEREF");
json!({"code": instr.trim(), "result": result, "perPage": per_page})
}
/// Structure for agents: blocks with text and formatting summary, selection, stats.
fn inspect(s: &mut Session, v: &Value) -> CmdResult {
let with_text = p::bool(v, "text").unwrap_or(true);
@@ -702,8 +773,23 @@ fn inspect(s: &mut Session, v: &Value) -> CmdResult {
}
}
let l = s.layout();
let parts: Vec<Value> =
s.doc.parts.iter().map(|(id, p)| json!({"id": id, "kind": p.kind, "text": s.doc.plain_text(StoryRef::Part(*id))})).collect();
let parts: Vec<Value> = s
.doc
.parts
.iter()
.map(|(id, p)| {
let mut o = json!({"id": id, "kind": p.kind, "text": s.doc.plain_text(StoryRef::Part(*id))});
let mut fields = Vec::new();
let complete = story_fields(&p.blocks, 0, &mut fields) && range_fields(&s.doc, StoryRef::Part(*id), &mut fields);
if !fields.is_empty() {
o["fields"] = json!(fields);
}
if !complete {
o["fieldsTruncated"] = json!(true);
}
o
})
.collect();
Ok(json!({
"blocks": blocks,
"parts": parts,
+15 -2
View File
@@ -24,7 +24,7 @@ pub fn specs() -> Vec<CommandSpec> {
CommandSpec::new("font.remove", "Remove Added Font", "Home › Font", remove).params(r#"{"family": string}"#).pure(),
CommandSpec::new("font.loadSystem", "Load System Installed Fonts…", "Home › Font", load_system).pure(),
CommandSpec::new("font.missing", "Missing Fonts", "Status Bar", missing)
.params(r#"{} → {"missing": [{"font": string, "shownAs": [string] (the families drawn instead), "metricCompatible": bool (its substitute measures like the font)}]} — the text the current view shows"#)
.params(r#"{} → {"missing": [{"font": string, "shownAs": [string] (the families drawn instead), "metricCompatible": bool (its substitute measures like the font), "download"?: {"name": string, "publisher": string, "url": string, "restricted": bool (the publisher limits who may use it: check its licence terms)} (its publisher's download page, when it offers the font at no charge)}]} — the text the current view shows"#)
.pure(),
]
}
@@ -136,6 +136,9 @@ pub struct MissingFont {
/// Fallback faces for characters the substitute lacks don't count: the font itself might
/// lack them too (an emoji in Calibri text).
pub metric_compatible: bool,
/// Its publisher's download page, when the publisher offers the font at no charge
/// ([`wordcraft_fonts::download`]): installing it shows the document as it was written.
pub download: Option<&'static wordcraft_fonts::download::FontDownload>,
}
/// The family text set in `font` is drawn in when `font` isn't installed (None when it is), as
@@ -174,6 +177,7 @@ pub fn missing_fonts(doc: &Document, layout: &DocLayout) -> Vec<MissingFont> {
// Judged by the name layout looked up, spaces and all: "Arial " isn't Arial to it, so
// its text is in a substitute even where Arial is installed.
metric_compatible: shown_as(doc, &recorded).is_some_and(|s| !warns(font, &s)),
download: wordcraft_fonts::download::download_for(font),
});
if !m.shown_as.contains(&family) {
m.shown_as.push(family);
@@ -216,7 +220,16 @@ fn missing(s: &mut Session, _: &Value) -> CmdResult {
(_, n) if n > NAMED => format!("Not installed: {named} and {} more", n - NAMED),
_ => format!("Not installed: {named}"),
};
let fonts: Vec<Value> = found.iter().map(|f| json!({"font": f.font, "shownAs": f.shown_as, "metricCompatible": f.metric_compatible})).collect();
let fonts: Vec<Value> = found
.iter()
.map(|f| {
let mut font = json!({"font": f.font, "shownAs": f.shown_as, "metricCompatible": f.metric_compatible});
if let Some(d) = f.download {
font["download"] = json!({"name": d.name, "publisher": d.publisher, "url": d.url, "restricted": d.restricted});
}
font
})
.collect();
Ok(json!({"missing": fonts}))
}
+98
View File
@@ -157,6 +157,104 @@ fn default_level(i: usize) -> Level {
levels_for(ListKind::Numbered).into_iter().nth(i).unwrap_or_default()
}
/// Define New Bullet (#2786): the selected paragraphs go into a list of their own whose bullet is
/// `char`, drawn in `font` (with `size`, `bold`, `italic` and `color` when given) and aligned by
/// `align` at every level they use. A list of its own, so
/// no other list using the same character changes, and it never toggles a bullet off the way
/// `para.bullets` does. Each paragraph keeps its level (level 1 when it wasn't in a list).
pub fn define_bullet(s: &mut Session, v: &Value) -> CmdResult {
let ch = match v.get("char") {
None => '★',
Some(c) => {
let c = c.as_str().ok_or_else(|| CmdError::Params("char must be one character".into()))?;
let mut it = c.chars();
match (it.next(), it.next()) {
(Some(c), None) if !c.is_control() => c,
_ => return Err(CmdError::Params("char must be one character".into())),
}
}
};
let font = match v.get("font") {
None => None,
Some(f) => {
let f = f.as_str().map(str::trim).ok_or_else(|| CmdError::Params("font must be a family name".into()))?;
if f.len() > 128 {
return Err(CmdError::Params("font name too long".into()));
}
(!f.is_empty()).then(|| f.to_string())
}
};
let size = match v.get("size") {
None => None,
Some(n) => match n.as_f64() {
Some(pt) if (1.0..=1638.0).contains(&pt) => Some(pt as f32),
_ => return Err(CmdError::Params("size must be 1 to 1638 points".into())),
},
};
let flag = |k: &str| -> Result<Option<bool>, CmdError> {
match v.get(k) {
None => Ok(None),
Some(b) => b.as_bool().map(Some).ok_or_else(|| CmdError::Params(format!("{k} must be true or false"))),
}
};
let (bold, italic) = (flag("bold")?, flag("italic")?);
let color = match v.get("color") {
None => None,
Some(c) => {
let c = c.as_str().ok_or_else(|| CmdError::Params("color must be RRGGBB or auto".into()))?;
Some(TextColor::parse(c).ok_or_else(|| CmdError::Params("color must be RRGGBB or auto".into()))?)
}
};
let align = match p::str(v, "align") {
None if v.get("align").is_none() => None,
Some("left") => Some(Align::Left),
Some("center") => Some(Align::Center),
Some("right") => Some(Align::Right),
_ => return Err(CmdError::Params("align must be left, center or right".into())),
};
// A paragraph's level, from its own numbering or its style's (List Bullet 2 numbers its
// paragraphs at level 2 without any of their own).
let styles = s.doc.styles.clone();
let level_of = |pp: &wordcraft_doc::props::ParaProps| styles.resolve_para(pp).numbering.filter(|n| n.num != 0).map_or(0, |n| n.level);
let story = s.sel.focus.story;
let used: std::collections::BTreeSet<u8> =
super::selected_paths(s).iter().filter_map(|p| s.doc.para(story, p)).map(|x| level_of(&x.props)).collect();
let mut levels = levels_for(ListKind::BulletChar(ch));
for l in used {
if let Some(lv) = levels.get_mut(usize::from(l)) {
lv.text = ch.to_string();
lv.chr.font = font.clone();
if size.is_some() {
lv.chr.size = size;
}
if bold.is_some() {
lv.chr.bold = bold;
}
if italic.is_some() {
lv.chr.italic = italic;
}
if color.is_some() {
lv.chr.color = color;
}
if let Some(a) = align {
lv.align = a;
}
}
}
let num = Arc::make_mut(&mut s.doc.numbering).try_add_levels(None, levels)?;
super::para::fmt(s, &|pp| {
let level = level_of(pp);
pp.numbering = Some(NumRef { num, level });
pp.indent_left = None;
pp.indent_first = None;
pp.indent_left_chars = None;
pp.indent_first_chars = None;
if pp.style.is_none() || pp.style.as_deref() == Some("Normal") {
pp.style = Some("ListParagraph".into());
}
})
}
fn get(s: &mut Session, v: &Value) -> CmdResult {
let num =
p::u64(v, "num").map(|n| n.min(u32::MAX as u64) as u32).or_else(|| caret_list(s)).ok_or_else(|| CmdError::Failed("not in a list".into()))?;
+3 -2
View File
@@ -21,8 +21,9 @@ pub(super) fn bounds(s: &mut Session, v: &Value) -> CmdResult {
}
let (w0, h0) = obj_size(&obj);
let min = min_size(&obj);
let w = p::f32(v, "width").unwrap_or(w0).clamp(min, MAX_OFFSET);
let h = p::f32(v, "height").unwrap_or(h0).clamp(min, MAX_OFFSET);
// A size not given stays as it is: a move must not give a level line (0 high) a height.
let w = p::f32(v, "width").map_or(w0, |w| w.clamp(min, MAX_OFFSET));
let h = p::f32(v, "height").map_or(h0, |h| h.clamp(min, MAX_OFFSET));
if (w, h) != (w0, h0) {
with_obj(s, |o| o.set_size(w, h))?;
}
+105
View File
@@ -152,6 +152,111 @@ fn click_resize_and_drag(insert: &str, params: Value) {
assert_eq!((object_area(&mut s).rect.x, object_area(&mut s).rect.y), (250.0, 300.0), "{insert}");
}
/// Moving keeps a size it isn't given: a level line (0 high, as Word files store a rule)
/// moved by its body stays level, though a height given to it is still at least the minimum.
#[test]
fn moving_a_level_line_keeps_it_level() {
let line = InlineObject::Shape {
kind: wordcraft_doc::para::ShapeKind::Line,
w: 200.0,
h: 0.0,
fill: None,
stroke: None,
stroke_width: 1.0,
float: wordcraft_doc::para::Float { wrap: Wrap::InFrontOfText, x: 50.0, y: 100.0, ..Default::default() },
story: None,
freeform: None,
effects: Default::default(),
extra: Default::default(),
};
let mut s = Session::new(wordcraft_doc::Document::new());
s.doc.insert_object(&Pos::body(0, 0), line, &Default::default()).unwrap();
s.run("select.range", &json!({"anchor": Pos::body(0, 0), "focus": Pos::body(0, wordcraft_doc::para::OBJ.len_utf8())})).unwrap();
s.run("arrange.bounds", &json!({"page": 0, "x": 150, "y": 300})).unwrap();
let o = s.layout().object(&Pos::body(0, 0), 0).unwrap();
assert_eq!((o.rect.x, o.rect.y, o.rect.w, o.rect.h), (150.0, 300.0, 200.0, 0.0));
s.run("arrange.bounds", &json!({"height": 0})).unwrap();
assert_eq!(s.layout().object(&Pos::body(0, 0), 0).unwrap().rect.h, min_size(&selected(&s).unwrap().1));
}
/// Connect Shapes is offered for the lines and connectors it moves: floating ones in the body
/// text, not one In Line with Text or in a header.
#[test]
fn connect_is_offered_only_for_floating_lines_in_the_body() {
use wordcraft_doc::para::ShapeKind;
let shape = |kind, wrap| InlineObject::Shape {
kind,
w: 200.0,
h: 0.0,
fill: None,
stroke: None,
stroke_width: 1.0,
float: wordcraft_doc::para::Float { wrap, x: 50.0, y: 100.0, ..Default::default() },
story: None,
freeform: None,
effects: Default::default(),
extra: Default::default(),
};
for kind in [ShapeKind::Line, ShapeKind::StraightConnector] {
for (wrap, offered) in [(Wrap::InFrontOfText, true), (Wrap::Inline, false)] {
let mut s = Session::new(wordcraft_doc::Document::new());
s.doc.insert_object(&Pos::body(0, 0), shape(kind, wrap), &Default::default()).unwrap();
s.run("select.range", &json!({"anchor": Pos::body(0, 0), "focus": Pos::body(0, wordcraft_doc::para::OBJ.len_utf8())})).unwrap();
assert_eq!(s.why_disabled("shape.connect").is_none(), offered, "{kind:?} {wrap:?}");
assert_eq!(s.run("shape.connect", &json!({"end": {"x": 300.0, "y": 150.0}})).is_ok(), offered, "{kind:?} {wrap:?}: runs when offered");
}
}
}
/// Moving one end of a slanted, flipped line leaves the other where the page shows it: the ends
/// `shape.connect` reads and writes are the drawn ones (checked on the rendered page).
#[test]
fn moving_an_end_of_a_flipped_slanted_line_keeps_the_other() {
use wordcraft_doc::para::ShapeKind;
// Flipped vertically, a line in the frame (300, 600) 100 × 100 runs from its bottom-left
// (300, 700) to its top-right (400, 600), as in Word.
let line = InlineObject::Shape {
kind: ShapeKind::Line,
w: 100.0,
h: 100.0,
fill: None,
stroke: Some(wordcraft_doc::props::Rgb::BLACK),
stroke_width: 3.0,
float: wordcraft_doc::para::Float {
wrap: Wrap::InFrontOfText,
h_rel: wordcraft_doc::para::Anchor::Page,
v_rel: wordcraft_doc::para::Anchor::Page,
x: 300.0,
y: 600.0,
flip_v: true,
..Default::default()
},
story: None,
freeform: None,
effects: Default::default(),
extra: Default::default(),
};
let mut s = Session::new(wordcraft_doc::Document::new());
s.doc.insert_object(&Pos::body(0, 0), line, &Default::default()).unwrap();
s.run("select.range", &json!({"anchor": Pos::body(0, 0), "focus": Pos::body(0, wordcraft_doc::para::OBJ.len_utf8())})).unwrap();
let inked = |s: &Session, x: f32, y: f32| {
let l = s.export_layout();
let r = wordcraft_render::render_page(&s.doc, &l.pages[0], 1.0, &Default::default());
let dark = |px: i64, py: i64| {
let i = ((py * i64::from(r.width) + px) * 4) as usize;
r.pixels.get(i..i + 3).is_some_and(|p| p.iter().all(|&c| c < 128))
};
(-3..=3).any(|dx| (-3..=3).any(|dy| dark(x as i64 + dx, y as i64 + dy)))
};
assert!(inked(&s, 302.0, 698.0) && inked(&s, 398.0, 602.0), "drawn from bottom-left to top-right");
assert!(!inked(&s, 302.0, 602.0), "not from the top-left");
// Its end goes to (450, 650): the start stays at (300, 700).
s.run("shape.connect", &json!({"end": {"x": 450.0, "y": 650.0}})).unwrap();
assert!(inked(&s, 303.0, 699.0), "the start stays where it was drawn");
assert!(inked(&s, 447.0, 651.0), "the end is where it was dragged");
assert!(!inked(&s, 398.0, 602.0), "the end left its old place");
}
/// Ctrl+drag: a copy lands where dropped and is selected; the original stays, the clipboard
/// is untouched, and it is one undo step.
#[test]
+13 -1
View File
@@ -107,6 +107,7 @@ pub(super) fn create_style(s: &mut Session, v: &Value) -> CmdResult {
Ok(json!({"id": id}))
}
/// Modify a style's character and paragraph formatting, display name or next-paragraph style.
pub(super) fn modify_style(s: &mut Session, v: &Value) -> CmdResult {
let id = style_id(s, p::req_str(v, "style")?)?;
let chr: Option<CharProps> = v.get("chr").map(|c| serde_json::from_value(c.clone())).transpose().map_err(|e| CmdError::Params(e.to_string()))?;
@@ -114,8 +115,19 @@ pub(super) fn modify_style(s: &mut Session, v: &Value) -> CmdResult {
v.get("para").map(|c| serde_json::from_value(c.clone())).transpose().map_err(|e| CmdError::Params(e.to_string()))?;
let next = p::str(v, "next").map(str::to_string);
let name = p::str(v, "name").map(str::to_string);
let d = &s.doc.styles.default_chr;
let (eff_b, eff_b_cs, eff_i, eff_i_cs) =
s.doc.styles.chain(&id).into_iter().fold((d.bold, d.bold_cs, d.italic, d.italic_cs), |(b, b_cs, i, i_cs), st| {
(st.chr.bold.or(b), st.chr.bold_cs.or(b_cs), st.chr.italic.or(i), st.chr.italic_cs.or(i_cs))
});
let st = Arc::make_mut(&mut s.doc.styles).get_mut(&id).ok_or_else(|| CmdError::Params("no such style".into()))?;
if let Some(c) = chr {
if let Some(mut c) = chr {
if eff_b_cs.filter(|&cs| Some(cs) != eff_b).is_none() {
c.bold_cs = c.bold_cs.or(c.bold);
}
if eff_i_cs.filter(|&cs| Some(cs) != eff_i).is_none() {
c.italic_cs = c.italic_cs.or(c.italic);
}
st.chr.overlay(&c);
}
if let Some(pp) = para {
+2
View File
@@ -17,6 +17,8 @@ mod properties;
mod toc;
use fields::caption;
#[cfg(test)]
use fields::output_fields_with;
pub use fields::{
CHAPTER_SEPARATORS, REFERENCE_NOT_FOUND, output_fields, refresh_date_fields, seq_format, update_fields, update_fields_in, update_seq,
};
+13 -4
View File
@@ -198,8 +198,9 @@ pub const REFERENCE_NOT_FOUND: &str = "Error! Reference source not found.";
/// comment or text box is found too, and a REF to it shows that text as `stored` holds it, its
/// own fields not yet refreshed, as Word's output does (measured in Word for Windows). The
/// document's bookmarks are looked up once, and each bookmark's text read once, however many
/// fields refer to it.
fn ref_results(doc: &wordcraft_doc::Document, stored: &wordcraft_doc::Document) -> Vec<FieldResult> {
/// fields refer to it. `error` is what a REF to a missing bookmark shows ([`REFERENCE_NOT_FOUND`]
/// in English).
fn ref_results(doc: &wordcraft_doc::Document, stored: &wordcraft_doc::Document, error: &str) -> Vec<FieldResult> {
let mut out = Vec::new();
let mut starts: std::collections::HashMap<String, Pos> = std::collections::HashMap::new();
for (name, start) in doc.bookmarks().into_iter().chain(part_bookmarks(doc)) {
@@ -220,7 +221,7 @@ fn ref_results(doc: &wordcraft_doc::Document, stored: &wordcraft_doc::Document)
let f = FieldCode::parse(instr);
let target = f.args.first().map(|s| s.split('\\').next().unwrap_or("").trim_matches('"')).unwrap_or("");
let Some(start) = starts.get(target) else {
out.push((StoryRef::Body, path.clone(), k, REFERENCE_NOT_FOUND.to_string()));
out.push((StoryRef::Body, path.clone(), k, error.to_string()));
continue;
};
let text = texts.entry(target.to_string()).or_insert_with(|| {
@@ -281,7 +282,15 @@ fn changed(doc: &wordcraft_doc::Document, mut results: Vec<FieldResult>) -> Vec<
/// formulas keep their stored results (measured with Word for Windows, printing and exporting
/// fields whose stored results were wrong). The document itself keeps its stored results.
/// `None` when every such field already shows its fresh result, so nothing is copied then.
/// A `REF` to a missing bookmark shows Word's error in the interface language
/// ([`crate::set_reference_error`]).
pub fn output_fields(doc: &wordcraft_doc::Document) -> Option<wordcraft_doc::Document> {
let error = crate::locale::reference_error();
output_fields_with(doc, error.as_deref().unwrap_or(REFERENCE_NOT_FOUND))
}
/// [`output_fields`], with `ref_error` for a `REF` to a missing bookmark.
pub(crate) fn output_fields_with(doc: &wordcraft_doc::Document, ref_error: &str) -> Option<wordcraft_doc::Document> {
// Numbers first, then the fields that read them: a STYLEREF to a heading holding a SEQ field,
// and a REF to a bookmarked caption, show the new number.
let seq = changed(doc, seq_results(doc));
@@ -293,7 +302,7 @@ pub fn output_fields(doc: &wordcraft_doc::Document) -> Option<wordcraft_doc::Doc
if !styles.is_empty() {
apply_results(out.get_or_insert_with(|| doc.clone()), styles).ok()?;
}
let refs = changed(out.as_ref().unwrap_or(doc), ref_results(out.as_ref().unwrap_or(doc), doc));
let refs = changed(out.as_ref().unwrap_or(doc), ref_results(out.as_ref().unwrap_or(doc), doc, ref_error));
if !refs.is_empty() {
apply_results(out.get_or_insert_with(|| doc.clone()), refs).ok()?;
}
+28
View File
@@ -874,6 +874,34 @@ fn output_shows_words_error_for_a_reference_to_a_missing_bookmark() {
assert_eq!(fields(&s).iter().map(|f| f.1.as_str()).collect::<Vec<_>>(), ["stale", "stale"], "the document keeps it");
}
/// Word prints a REF's missing-bookmark error in its interface language: output takes the text
/// the shell sets ([`crate::set_reference_error`]), and a REF to a bookmark the document has
/// still shows the bookmark's text.
#[test]
fn a_missing_references_error_is_in_the_interface_language() {
let mut s = Session::new(wordcraft_doc::Document::from_text("Target"));
s.run("select.text", &json!({"text": "Target"})).unwrap();
s.run("insert.bookmark", &json!({"name": "Here"})).unwrap();
s.run("caret.docEnd", &json!({})).unwrap();
for instr in ["REF NoSuchMark \\h", "REF Here \\h"] {
s.run("text.insert", &json!({"text": " "})).unwrap();
s.run("insert.field", &json!({"instr": instr, "result": "stale"})).unwrap();
}
let french = "Erreur ! Source du renvoi introuvable.";
let out = output_fields_with(&s.doc, french).expect("both results change");
let shown: Vec<String> = out
.para(StoryRef::Body, &wordcraft_doc::Path::top(0))
.unwrap()
.objects
.iter()
.filter_map(|o| match o {
InlineObject::Field { result, .. } => Some(result.clone()),
_ => None,
})
.collect();
assert_eq!(shown, [french, "Target"]);
}
/// A REF showing the bookmarked paragraph's number (`\r`, `\n`, `\w`), which output doesn't
/// compute, keeps its stored number when printed or exported rather than show the paragraph's
/// text; a plain REF to the same bookmark is filled in.
+1
View File
@@ -799,6 +799,7 @@ fn deleting_a_comment_in_a_parent_cycle_ends() {
/// A very long reply chain (as a hostile file could hold) is deleted in one pass over the
/// comments, not one pass per reply: 50,000 replies would otherwise freeze the app.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn deleting_a_long_reply_chain_stays_fast() {
let mut s = Session::new(Document::new());
s.view.proofing = false;
+8 -4
View File
@@ -305,12 +305,16 @@ fn share_out(w: &mut [f32], i: usize, dx: f32, share: Share) -> Option<f32> {
Share::Proportional if rest > 0.0 && narrowest > 0.0 => Share::Proportional,
_ => Share::LeftOnly,
};
let least = MIN_COL - left;
// A column already narrower than `MIN_COL` (a file can give one) is not narrowed further,
// nor widened by a drag that has no room for it: forcing it to `MIN_COL` would take its
// neighbour below a point, which import reads as no width and gives the whole table equal
// columns.
let least = (MIN_COL - left).min(0.0);
let d = match share {
Share::Neighbour => dx.clamp(least, (w.get(i + 1).copied().unwrap_or(0.0) - MIN_COL).max(least)),
Share::Neighbour => dx.clamp(least, (w.get(i + 1).copied().unwrap_or(0.0) - MIN_COL).max(0.0)),
// As far as the narrowest column on the right, shrunk in proportion, stays `MIN_COL`.
Share::Proportional => dx.clamp(least, (rest * (1.0 - (MIN_COL / narrowest).min(1.0))).max(least)),
Share::LeftOnly => (left + dx).clamp(MIN_COL, MAX_LEN) - left,
Share::Proportional => dx.clamp(least, (rest * (1.0 - (MIN_COL / narrowest).min(1.0))).max(0.0)),
Share::LeftOnly => (left + dx).clamp(MIN_COL.min(left), MAX_LEN.max(left)) - left,
};
if let Some(x) = w.get_mut(i) {
*x = left + d;
@@ -237,6 +237,29 @@ fn ctrl_shares_a_border_drag_among_the_columns_right_of_it() {
assert_eq!(share_out(&mut w, 3, 1.0, Share::Proportional), None);
}
/// Columns a file gives narrower than the minimum (3 pt and 4 pt) aren't forced to it by a drag
/// between them: that took the right one to 1 pt, which the next import read as no width.
#[test]
fn a_drag_between_narrow_columns_leaves_them_as_they_are() {
for share in [Share::Neighbour, Share::Proportional, Share::LeftOnly] {
for dx in [0.0, -2.0, 2.0] {
let mut w = [3.0, 4.0, 100.0];
share_out(&mut w, 0, dx, share);
assert!(w.iter().all(|x| *x > 1.0), "{share:?} by {dx}: {w:?}");
if share == Share::Neighbour {
assert_eq!(w, [3.0, 4.0, 100.0], "by {dx}: no room on either side");
}
}
}
// A narrow column still widens when its neighbour has room.
let mut w = [3.0, 100.0];
assert_eq!(share_out(&mut w, 0, 10.0, Share::Neighbour), Some(10.0));
assert_eq!(w, [13.0, 90.0]);
// Nothing moves for no drag.
let mut w = [3.0, 100.0];
assert_eq!(share_out(&mut w, 0, 0.0, Share::Neighbour), Some(0.0));
}
/// The table's left edge, grabbed from just outside it, moves the table in or out: its
/// indent changes and the first column makes up the difference, so the other borders stay.
/// A centred or right-aligned table keeps its alignment, as in Word: only the first column
+3 -4
View File
@@ -316,10 +316,9 @@ pub fn specs() -> Vec<CommandSpec> {
sel_result(s)
})
.params(r#"{"value": n}"#),
CommandSpec::new("para.defineBullet", "Define New Bullet", "Home › Paragraph › Bullets", |s, v| {
s.run("para.bullets", &json!({"kind": p::str(v, "char").unwrap_or("★")}))
})
.params(r#"{"char": string}"#),
CommandSpec::new("para.defineBullet", "Define New Bullet", "Home › Paragraph › Bullets", crate::cmd::lists::define_bullet).params(
r#"{"char"?: one character (default ★), "font"?: family (the bullet's own font; "" = the text's), "size"?: pt, "bold"?: bool, "italic"?: bool, "color"?: "RRGGBB" | "auto", "align"?: "left|center|right"} (the selected paragraphs get a list of their own; each keeps its level)"#,
),
CommandSpec::new("para.defineNumber", "Define New Number Format", "Home › Paragraph › Numbering", |s, v| {
let fmt = wordcraft_doc::section::NumFormat::from_ooxml(p::str(v, "format").unwrap_or("decimal"));
let text = p::str(v, "text").unwrap_or("%1.").to_string();
+2 -2
View File
@@ -47,8 +47,8 @@ fn theme(s: &Session) -> Vec<Rgb> {
}
/// The text box story the selection formats as a whole: the selected shape's, or the one the
/// caret is in.
fn box_story(s: &Session) -> Option<u32> {
/// caret is in. The ribbon reads the current effects from the same place.
pub fn box_story(s: &Session) -> Option<u32> {
if let Some((_, InlineObject::Shape { story: Some(id), .. })) = super::objects::object_selection(s) {
return Some(*id);
}
+7 -1
View File
@@ -103,7 +103,13 @@ pub fn save(ext: &str, doc: &Document, opts: &crate::io::ExportOptions) -> Optio
pub fn render_png(doc: &Document, page: usize, scale: f32, opts: &crate::io::ExportOptions) -> Result<Vec<u8>, String> {
let out = crate::cmd::references::output_fields(doc);
let doc = out.as_ref().unwrap_or(doc);
let lo = wordcraft_layout::LayoutOptions { comment_balloons: opts.markup && opts.comments, numeral: opts.numeral, ..Default::default() };
let lo = wordcraft_layout::LayoutOptions {
comment_balloons: opts.markup && opts.comments,
numeral: opts.numeral,
output_page_refs: true,
bookmark_error: crate::locale::bookmark_error(),
..Default::default()
};
let l = wordcraft_layout::layout(doc, &mut wordcraft_layout::LayoutCache::new(), &lo);
let p = l.pages.get(page).ok_or_else(|| format!("no page {}", page + 1))?;
if opts.strict {
+1
View File
@@ -34,6 +34,7 @@ use serde_json::Value;
pub use io::Password;
pub use keymap::KeyMap;
pub use locale::{set_bookmark_error, set_reference_error};
pub use ribbon::RibbonLayout;
pub use session::{
CellRange, ColumnBlock, DocInfo, DocState, EditSnapshot, FindState, MathEdit, Prefs, Selection, Session, StylesPaneDock, UNDO_LIST_MAX,
+39 -1
View File
@@ -1,4 +1,4 @@
use std::sync::OnceLock;
use std::sync::{Arc, OnceLock, RwLock};
/// The system locale (BCP 47 tag) the shell found at startup, for Numeral › System.
pub(super) static SYSTEM_LOCALE: OnceLock<String> = OnceLock::new();
@@ -13,3 +13,41 @@ pub fn set_system_locale(tag: &str) {
pub(crate) fn system_locale() -> Option<&'static str> {
SYSTEM_LOCALE.get().map(String::as_str)
}
/// What output prints for a page reference to a missing bookmark, in the interface language
/// ([`set_bookmark_error`]); `None` until the shell sets it (English).
static BOOKMARK_ERROR: RwLock<Option<Arc<str>>> = RwLock::new(None);
/// Tell the engine what PDFs, prints and page images show for a page reference to a bookmark the
/// document doesn't have, in the interface language: Word prints "Error! Bookmark not defined."
/// in its own. The shell calls this when its language is set; headless sessions print English.
pub fn set_bookmark_error(text: &str) {
let mut slot = BOOKMARK_ERROR.write().unwrap_or_else(std::sync::PoisonError::into_inner);
if slot.as_deref() != Some(text) {
*slot = Some(Arc::from(text));
}
}
/// The text [`set_bookmark_error`] recorded, for output's layout options.
pub(crate) fn bookmark_error() -> Option<Arc<str>> {
BOOKMARK_ERROR.read().unwrap_or_else(std::sync::PoisonError::into_inner).clone()
}
/// What output prints for a `REF` to a missing bookmark, in the interface language
/// ([`set_reference_error`]); `None` until the shell sets it (English).
static REFERENCE_ERROR: RwLock<Option<Arc<str>>> = RwLock::new(None);
/// Tell the engine what PDFs, prints and page images show for a `REF` to a bookmark the document
/// doesn't have, in the interface language: Word prints "Error! Reference source not found." in
/// its own. The shell calls this when its language is set; headless sessions print English.
pub fn set_reference_error(text: &str) {
let mut slot = REFERENCE_ERROR.write().unwrap_or_else(std::sync::PoisonError::into_inner);
if slot.as_deref() != Some(text) {
*slot = Some(Arc::from(text));
}
}
/// The text [`set_reference_error`] recorded, for output's `REF` results.
pub(crate) fn reference_error() -> Option<Arc<str>> {
REFERENCE_ERROR.read().unwrap_or_else(std::sync::PoisonError::into_inner).clone()
}
+7 -1
View File
@@ -74,6 +74,9 @@ impl Session {
typing_word: typing,
filename: filename.clone(),
filepath: filepath.clone(),
// The editor shows a body page reference as stored until fields are updated.
output_page_refs: false,
bookmark_error: None,
};
let l = Arc::new(wordcraft_layout::layout(&self.doc, &mut self.cache, &opts));
let [newest, _] = std::mem::take(&mut self.layout);
@@ -155,7 +158,8 @@ impl Session {
layout.map(|l| (copy.doc, copy.sel, l))
}
/// A layout for output (PDF, images, print): no proofing marks, print view.
/// A layout for output (PDF, images, print): no proofing marks, print view, and page
/// references in the body showing their bookmarks' pages, as Word prints them.
pub fn export_layout(&self) -> Arc<DocLayout> {
self.export_layout_with(false)
}
@@ -177,6 +181,8 @@ impl Session {
numeral: self.view.numeral.resolve(SYSTEM_LOCALE.get().map(String::as_str)),
filename: Some(Arc::from(fn_name.as_str())),
filepath: Some(Arc::from(fn_path.as_str())),
output_page_refs: true,
bookmark_error: crate::locale::bookmark_error(),
..Default::default()
};
Arc::new(wordcraft_layout::layout(doc, &mut LayoutCache::new(), &opts))
+72 -77
View File
@@ -13,6 +13,7 @@
//! client halts reading on it ([`Outcome::Halted`]), and so does one that never began
//! ([`Outcome::NotStarted`]).
use craft_ssip::{Lines, final_code, message_data};
use std::io::{ErrorKind, Read, Write};
use std::net::{TcpStream, ToSocketAddrs};
use std::os::unix::net::UnixStream;
@@ -23,8 +24,8 @@ use std::time::{Duration, Instant};
const POLL: Duration = Duration::from_millis(30);
/// Longest wait for a reply to a command (the server answers at once; this only bounds a stuck one).
const REPLY_TIMEOUT: Duration = if cfg!(test) { Duration::from_secs(1) } else { Duration::from_secs(5) };
/// Longest reply line kept (a server sends short lines; this bounds a broken one).
const MAX_LINE: usize = 4096;
/// Most bytes of one sentence sent as message data: [`super::MAX_UTTERANCE`] chars of up to 4 bytes each.
const MAX_MESSAGE_BYTES: usize = 4 * super::MAX_UTTERANCE;
/// Speech Dispatcher's TCP port when `SPEECHD_ADDRESS` names none.
const DEFAULT_PORT: u16 = 6560;
/// How long a freshly started server gets to open its socket (it loads its speech modules first).
@@ -286,63 +287,6 @@ fn start_server(command: &mut std::process::Command, until: Instant, changed: &d
stopped
}
/// The sentence as SSIP message data: one line (a line break is a space when read aloud anyway),
/// a leading `.` doubled so it can't end the message, and the terminating `.` line.
pub(super) fn message_data(text: &str) -> String {
let line: String = text.chars().map(|c| if c == '\r' || c == '\n' { ' ' } else { c }).collect();
let dot = if line.starts_with('.') { "." } else { "" };
format!("{dot}{line}\r\n.\r\n")
}
/// Reads reply lines. Bytes wait in `pending` until their line is complete, so a character split
/// across reads stays whole, and a line longer than [`MAX_LINE`] is an error, never a growing buffer.
struct Lines<R> {
reader: R,
pending: Vec<u8>,
}
impl<R: Read> Lines<R> {
fn new(reader: R) -> Self {
Lines { reader, pending: Vec::new() }
}
/// The next complete line without its line ending, or `Ok(None)` when none is complete after
/// one read (a timeout, or part of a line): the caller looks at its deadline and whether reading
/// moved on between calls, however slowly the bytes trickle in.
fn next(&mut self) -> std::io::Result<Option<String>> {
if let Some(line) = self.complete_line() {
return Ok(Some(line));
}
if self.pending.len() > MAX_LINE {
return Err(ErrorKind::InvalidData.into());
}
let mut buf = [0u8; 512];
match self.reader.read(&mut buf) {
Ok(0) => Err(ErrorKind::UnexpectedEof.into()),
Ok(n) => {
self.pending.extend_from_slice(buf.get(..n).unwrap_or_default());
Ok(self.complete_line())
}
Err(e) if matches!(e.kind(), ErrorKind::WouldBlock | ErrorKind::TimedOut | ErrorKind::Interrupted) => Ok(None),
Err(e) => Err(e),
}
}
/// The first complete line in `pending`, taken out of it.
fn complete_line(&mut self) -> Option<String> {
let end = self.pending.iter().position(|&b| b == b'\n')?;
let line: Vec<u8> = self.pending.drain(..=end).collect();
Some(String::from_utf8_lossy(&line).trim_end_matches(['\r', '\n']).to_string())
}
}
/// The three-digit code of a final reply line (`225 OK MESSAGE QUEUED`); `None` for a
/// continuation line (`225-12`).
fn final_code(line: &str) -> Option<u16> {
let code = line.get(..3)?.parse().ok()?;
(line.as_bytes().get(3) == Some(&b' ') || line.len() == 3).then_some(code)
}
/// A `2xx` reply, or an error naming the code.
fn ok(code: u16) -> std::io::Result<()> {
if (200..300).contains(&code) { Ok(()) } else { Err(std::io::Error::other(format!("Speech Dispatcher refused a command ({code})"))) }
@@ -378,7 +322,7 @@ impl<W: Write, R: Read> Conversation<'_, W, R> {
fn reply(&mut self) -> std::io::Result<u16> {
let start = Instant::now();
loop {
if let Some(line) = self.lines.next()? {
if let Some(line) = self.lines.next_line()? {
match final_code(&line) {
Some(BEGIN) => self.begun = true,
Some(code @ (END | CANCEL | PAUSE)) => self.event = Some(code),
@@ -425,7 +369,7 @@ impl<W: Write, R: Read> Conversation<'_, W, R> {
Some(t) if t.elapsed() > limit => return Err(std::io::Error::new(ErrorKind::TimedOut, "the sentence didn't end")),
_ => {}
}
if let Some(line) = self.lines.next()? {
if let Some(line) = self.lines.next_line()? {
match final_code(&line) {
Some(BEGIN) => {
began.get_or_insert_with(Instant::now);
@@ -444,7 +388,7 @@ impl<W: Write, R: Read> Conversation<'_, W, R> {
if (self.changed)() {
return Err(moved_on());
}
self.lines.next()?;
self.lines.next_line()?;
}
Ok(())
}
@@ -474,7 +418,7 @@ impl<W: Write, R: Read> Conversation<'_, W, R> {
self.event = None;
self.begun = false;
ok(self.command("SPEAK")?)?;
self.w.write_all(message_data(req.text).as_bytes())?;
self.w.write_all(message_data(req.text, MAX_MESSAGE_BYTES)?.as_bytes())?;
ok(self.reply()?)?;
match self.fate(limit)? {
Fate::End => return Ok(Outcome::Done),
@@ -520,6 +464,7 @@ pub(super) fn converse(stream: &Stream, req: &Request<'_>, changed: &dyn Fn() ->
#[cfg(test)]
mod tests {
use super::*;
use craft_ssip::MAX_LINE;
use std::io::{BufRead, BufReader};
#[test]
@@ -549,11 +494,11 @@ mod tests {
#[test]
fn document_text_is_always_message_data() {
// `spd-say -e` would run this line as an SSIP command; here it's spoken text.
assert_eq!(message_data("!-!SET SELF RATE 50"), "!-!SET SELF RATE 50\r\n.\r\n");
assert_eq!(message_data("!-!SET SELF RATE 50", 64).unwrap(), "!-!SET SELF RATE 50\r\n.\r\n");
// A line of a single `.` would end the message early, and a line break would start a
// new line that could be one.
assert_eq!(message_data(".\r\nQUIT"), ".. QUIT\r\n.\r\n");
assert_eq!(message_data("a\n.\nb"), "a . b\r\n.\r\n");
assert_eq!(message_data(".\r\nQUIT", 64).unwrap(), ".. QUIT\r\n.\r\n");
assert_eq!(message_data("a\n.\nb", 64).unwrap(), "a . b\r\n.\r\n");
assert_eq!(final_code("225 OK MESSAGE QUEUED"), Some(225));
assert_eq!(final_code("225-17"), None);
assert_eq!(final_code("x"), None);
@@ -582,11 +527,11 @@ mod tests {
fn a_character_split_by_a_read_timeout_stays_whole() {
// `é` is C3 A9; the read times out (the empty read) between its two bytes.
let mut lines = Lines::new(Reads(vec![b"249-Caf\xC3".to_vec(), Vec::new(), b"\xA9\tfr\tnone\r\n702 END\r\n".to_vec()].into()));
assert_eq!(lines.next().unwrap(), None, "half a line");
assert_eq!(lines.next().unwrap(), None, "the timeout");
assert_eq!(lines.next().unwrap().as_deref(), Some("249-Café\tfr\tnone"));
assert_eq!(lines.next().unwrap().as_deref(), Some("702 END"));
assert_eq!(lines.next().unwrap(), None);
assert_eq!(lines.next_line().unwrap(), None, "half a line");
assert_eq!(lines.next_line().unwrap(), None, "the timeout");
assert_eq!(lines.next_line().unwrap().as_deref(), Some("249-Café\tfr\tnone"));
assert_eq!(lines.next_line().unwrap().as_deref(), Some("702 END"));
assert_eq!(lines.next_line().unwrap(), None);
}
#[test]
@@ -597,14 +542,13 @@ mod tests {
let err = loop {
calls += 1;
assert!(calls < 100, "never gave up");
match lines.next() {
match lines.next_line() {
Ok(None) => {}
Ok(Some(l)) => panic!("{l}"),
Err(e) => break e,
}
};
assert_eq!(err.kind(), ErrorKind::InvalidData);
assert!(lines.pending.len() <= MAX_LINE + 512);
}
#[test]
@@ -612,10 +556,61 @@ mod tests {
// One byte per read and never a line end: every call returns, so the caller can look at
// its deadline and at whether reading moved on.
let mut lines = Lines::new(Reads(vec![b"2".to_vec(), b"0".to_vec(), b"0".to_vec(), b" OK\r\n".to_vec()].into()));
assert_eq!(lines.next().unwrap(), None);
assert_eq!(lines.next().unwrap(), None);
assert_eq!(lines.next().unwrap(), None);
assert_eq!(lines.next().unwrap().as_deref(), Some("200 OK"));
assert_eq!(lines.next_line().unwrap(), None);
assert_eq!(lines.next_line().unwrap(), None);
assert_eq!(lines.next_line().unwrap(), None);
assert_eq!(lines.next_line().unwrap().as_deref(), Some("200 OK"));
}
/// The sentence encoder this module had before it moved to `craft-ssip`, kept as the oracle.
fn old_message_data(text: &str) -> String {
let line: String = text.chars().map(|c| if c == '\r' || c == '\n' { ' ' } else { c }).collect();
let dot = if line.starts_with('.') { "." } else { "" };
format!("{dot}{line}\r\n.\r\n")
}
#[test]
fn shared_message_data_matches_the_old_encoder() {
let near_cap = "é".repeat(super::super::MAX_UTTERANCE);
let at_cap = "𝄞".repeat(super::super::MAX_UTTERANCE);
assert_eq!(at_cap.len(), MAX_MESSAGE_BYTES, "the cap fits the worst case");
for text in ["", ".", "..x", "a\r\nb", "\n.", "!-!SET SELF RATE 50", "\r\n.\r\n", near_cap.as_str(), at_cap.as_str()] {
assert_eq!(message_data(text, MAX_MESSAGE_BYTES).unwrap(), old_message_data(text), "{text:?}");
}
}
/// A server that answers the first command with `reply` instead of a code.
fn bad_server(mut s: UnixStream, reply: Vec<u8>) -> std::thread::JoinHandle<()> {
std::thread::spawn(move || {
let mut line = String::new();
let _ = BufReader::new(s.try_clone().unwrap()).read_line(&mut line);
let _ = s.write_all(&reply);
// Hold the socket open so the client sees the reply, not a hang-up.
std::thread::sleep(Duration::from_millis(1500));
})
}
#[test]
fn a_reply_code_with_a_sign_is_not_a_code() {
// `+22` parsed as a number before; now it's no reply at all, so the wait ends in a timeout.
let (client, srv) = UnixStream::pair().unwrap();
let h = bad_server(srv, b"+22 OK\r\n".to_vec());
let req = Request { text: "Hello.", rate: 0, lang: None };
let err = converse(&Stream::Unix(client), &req, &|| false).unwrap_err();
assert_eq!(err.kind(), ErrorKind::TimedOut);
h.join().unwrap();
}
#[test]
fn an_overlong_reply_line_is_an_error() {
let (client, srv) = UnixStream::pair().unwrap();
let mut junk = vec![b'x'; MAX_LINE + 100];
junk.extend_from_slice(b"\r\n");
let h = bad_server(srv, junk);
let req = Request { text: "Hello.", rate: 0, lang: None };
let err = converse(&Stream::Unix(client), &req, &|| false).unwrap_err();
assert_eq!(err.kind(), ErrorKind::InvalidData);
h.join().unwrap();
}
/// How the stand-in server ends a message.
+1
View File
@@ -102,6 +102,7 @@ fn tidy_matches_replacing_until_nothing_changes() {
/// Nested empty brackets are removed in one pass: 40,000 pairs took seconds when each pass
/// removed one level (#3680).
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn tidy_is_linear_on_nested_brackets() {
let n = 40_000;
let started = std::time::Instant::now();
+7 -1
View File
@@ -193,7 +193,13 @@ pub fn strict_report(
/// The PDF export's options for `opts`: the one place they are built, shared by the export and
/// its strict check.
pub(crate) fn pdf_options(opts: &ExportOptions) -> wordcraft_pdf::PdfOptions {
wordcraft_pdf::PdfOptions { include_markup: opts.markup, comments: opts.markup && opts.comments, numeral: opts.numeral, ..Default::default() }
wordcraft_pdf::PdfOptions {
include_markup: opts.markup,
comments: opts.markup && opts.comments,
numeral: opts.numeral,
bookmark_error: crate::locale::bookmark_error(),
..Default::default()
}
}
/// What a page image shows: the display options `file.exportPng` renders with.
+245 -2
View File
@@ -1052,6 +1052,7 @@ fn rep_mode(text: &str, q: &str, with: &str, extra: serde_json::Value) -> String
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn hostile_find_patterns_are_bounded() {
let mut s = s();
run(&mut s, "document.setText", json!({"text": &"a".repeat(5000)}));
@@ -4928,6 +4929,50 @@ fn toc_page_numbers_are_pageref_fields() {
assert_eq!(text(&s), expected);
}
/// Output (PDF, print) shows a table of contents' page numbers where the headings now are, as
/// Word fills its `PAGEREF`s in when it prints or exports; the editor shows them as stored until
/// the table is updated, and the document keeps them.
#[test]
fn output_shows_toc_page_numbers_where_the_headings_are() {
let mut s = s();
run(&mut s, "references.toc", json!({}));
run(&mut s, "caret.docEnd", json!({}));
for name in ["Alpha", "Bravo"] {
run(&mut s, "text.pageBreak", json!({}));
run(&mut s, "text.insert", json!({"text": name}));
run(&mut s, "para.heading1", json!({}));
run(&mut s, "text.newParagraph", json!({}));
}
run(&mut s, "references.updateToc", json!({}));
// Another page before Alpha moves the headings to pages 3 and 4.
let alpha = s
.doc
.body
.iter()
.position(|b| b.as_para().is_some_and(|p| p.props.style.as_deref() == Some("Heading1") && p.text.contains("Alpha")))
.unwrap();
s.sel = crate::Selection::caret(Pos::body(alpha, 0));
run(&mut s, "text.pageBreak", json!({}));
let toc_numbers = |l: &wordcraft_layout::DocLayout| -> Vec<String> {
l.pages[0]
.items
.iter()
.flat_map(|it| match it {
wordcraft_layout::Placed::Lines { para, .. } => para.shown.iter().map(|s| s.1.clone()).collect(),
_ => vec![],
})
.collect()
};
assert_eq!(s.export_layout().pages.len(), 4);
assert_eq!(toc_numbers(&s.layout()), ["2", "3"]);
assert_eq!(toc_numbers(&s.export_layout()), ["3", "4"]);
assert!(text(&s).contains("Alpha\t2"), "the document keeps the stored numbers");
// A page image is laid out as the print job is: it shows the same numbers.
let png = crate::io_ext::render_png(&s.doc, 0, 1.0, &Default::default()).unwrap();
let printed = wordcraft_render::render_page(&s.doc, &s.export_layout().pages[0], 1.0, &Default::default()).to_png();
assert!(png == printed, "the page image and the printed page differ");
}
/// Headings numbered through a list carry their numbers, and the level's separator, into the
/// TOC entries.
#[test]
@@ -8548,6 +8593,83 @@ fn colour_commands_take_any_custom_hex() {
assert_eq!(s.doc.settings.page_color, wordcraft_doc::Rgb::parse("123ABC"));
}
/// #2786: Define New Bullet puts the selection in a list of its own: the character, its font and
/// its alignment at the levels the paragraphs use, each paragraph keeping its level. A list that
/// already uses the same character is left alone, defining again never toggles the bullet off,
/// it survives a DOCX round trip, and bad params are errors that change nothing.
#[test]
fn define_bullet_gives_the_selection_a_bullet_of_its_own() {
use wordcraft_doc::props::Align;
let mut s = Session::new(wordcraft_doc::Document::from_text("One\nTwo\nThree"));
let num_at = |s: &Session, i: usize| s.doc.para_at(&Pos::body(i, 0)).unwrap().props.numbering.unwrap();
// "Three" already has a ♦ list, defined the ordinary way.
s.sel = crate::Selection::caret(Pos::body(2, 0));
run(&mut s, "para.bullets", json!({"kind": "♦"}));
let other = num_at(&s, 2).num;
let other_level = s.doc.numbering.level(other, 0).unwrap().clone();
// "Two" sits at level 2 of a bulleted list before the definition.
s.sel = crate::Selection::caret(Pos::body(1, 0));
run(&mut s, "para.bullets", json!({}));
run(&mut s, "para.listLevel", json!({"level": 1}));
s.sel = crate::Selection { anchor: Pos::body(0, 0), focus: Pos::body(1, 3) };
run(&mut s, "para.defineBullet", json!({"char": "♦", "font": "Segoe UI Symbol", "align": "center"}));
let (one, two) = (num_at(&s, 0), num_at(&s, 1));
assert_eq!(one.num, two.num, "one list for the selection");
assert_ne!(one.num, other, "not the existing ♦ list");
assert_eq!((one.level, two.level), (0, 1), "each paragraph keeps its level");
for lvl in [0, 1] {
let l = s.doc.numbering.level(one.num, lvl).unwrap();
assert_eq!((l.text.as_str(), l.chr.font.as_deref(), l.align), ("♦", Some("Segoe UI Symbol"), Align::Center), "level {lvl}");
}
assert_eq!(s.doc.numbering.level(other, 0).unwrap(), &other_level, "the other ♦ list is unchanged");
// Defining the same bullet again keeps it (para.bullets would toggle it off).
run(&mut s, "para.defineBullet", json!({"char": "♦"}));
assert_eq!(s.doc.numbering.level(num_at(&s, 0).num, 0).unwrap().text, "♦");
// DOCX keeps the character, its font and the alignment.
run(&mut s, "para.defineBullet", json!({"char": "✓", "font": "Segoe UI Symbol", "align": "right", "size": 14, "bold": true, "color": "C00000"}));
let back = wordcraft_docx::read(&wordcraft_docx::write(&s.doc).unwrap()).unwrap();
let n = back.para_at(&Pos::body(0, 0)).unwrap().props.numbering.unwrap();
let l = back.numbering.level(n.num, 0).unwrap();
assert_eq!((l.text.as_str(), l.chr.font.as_deref(), l.align), ("✓", Some("Segoe UI Symbol"), Align::Right));
// The Font dialog's size, bold and colour reach the bullet too.
assert_eq!((l.chr.size, l.chr.bold), (Some(14.0), Some(true)));
assert_eq!(l.chr.color, wordcraft_doc::props::TextColor::parse("C00000"));
// A paragraph numbered only by its style keeps the style's level (review of #2860).
let mut s2 = Session::new(wordcraft_doc::Document::from_text("Styled"));
let styled = Arc::make_mut(&mut s2.doc.numbering).try_add_list(wordcraft_doc::ListKind::Bullet).unwrap();
Arc::make_mut(&mut s2.doc.styles).get_mut("ListParagraph").unwrap().para.numbering = Some(wordcraft_doc::props::NumRef { num: styled, level: 1 });
s2.doc.para_mut(wordcraft_doc::StoryRef::Body, &wordcraft_doc::Path::top(0)).unwrap().props.style = Some("ListParagraph".into());
assert!(s2.doc.para_at(&Pos::body(0, 0)).unwrap().props.numbering.is_none(), "no numbering of its own");
s2.sel = crate::Selection::caret(Pos::body(0, 0));
run(&mut s2, "para.defineBullet", json!({"char": "♦"}));
let n = s2.doc.para_at(&Pos::body(0, 0)).unwrap().props.numbering.unwrap();
assert_eq!(n.level, 1, "the style's level is kept");
assert_eq!(s2.doc.numbering.level(n.num, 1).unwrap().text, "♦");
// No params: ★, as before.
s.sel = crate::Selection::caret(Pos::body(2, 0));
run(&mut s, "para.defineBullet", json!({}));
assert_eq!(s.doc.numbering.level(num_at(&s, 2).num, 0).unwrap().text, "★");
let doc = s.doc.clone();
let long = "x".repeat(200);
for junk in [
json!({"char": ""}),
json!({"char": "ab"}),
json!({"char": 5}),
json!({"char": "\u{7}"}),
json!({"char": "•", "align": "middle"}),
json!({"char": "•", "align": 1}),
json!({"char": "•", "font": 3}),
json!({"char": "•", "font": long}),
json!({"char": "•", "size": 0}),
json!({"char": "•", "size": "big"}),
json!({"char": "•", "bold": "yes"}),
json!({"char": "•", "color": "purple"}),
] {
assert!(s.run("para.defineBullet", &junk).is_err(), "{junk}");
}
assert_eq!(s.doc, doc);
}
/// #4685: Home › Paragraph › Shading on text selected inside a paragraph shades that text, as in
/// Word, so No Color takes off a text shading a file gave it (an orange "highlight" from a .docx is
/// `w:shd`, which Text Highlight Color › No Color leaves, in Word too). A caret or a whole
@@ -10329,8 +10451,9 @@ fn review_language_validates_tags_and_marks_text_unchecked() {
assert_eq!(run(&mut s, "format.state", json!({}))["lang"], "pl");
run(&mut s, "review.language", json!({}));
assert_eq!(run(&mut s, "format.state", json!({}))["lang"], "en-US");
// A language without proofing tools: no marks.
run(&mut s, "review.language", json!({"lang": "fr-FR"}));
// A language without proofing tools: no marks. `zxx` (no linguistic content) never has
// them; a real language such as fr-FR does wherever its system Hunspell dictionary is installed.
run(&mut s, "review.language", json!({"lang": "zxx"}));
assert_eq!(run(&mut s, "review.issues", json!({})), json!([]));
}
@@ -12083,6 +12206,7 @@ fn inspect_document_removes_the_note_of_a_hidden_reference() {
/// #1344: many hidden pieces (index entries are hidden text) are removed quickly, and the
/// found list is capped.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn inspect_document_removes_thousands_of_hidden_pieces() {
use wordcraft_doc::props::CharProps;
use wordcraft_doc::{Paragraph, para_block};
@@ -13104,6 +13228,125 @@ fn scale_font_sizes_reaches_char_table_and_cs_sizes_and_unset_default() {
assert_eq!(grid.regions.get(&TablePart::Band2Horz).unwrap().chr.size, Some(10.0));
}
/// #4467: `document.inspect` lists a footer's fields and marks the ones each page shows its own
/// value of, so a caller knows "1 of 1" is a stored result and not what page 2 shows.
#[test]
fn inspect_marks_page_fields_in_headers_and_footers() {
let mut s = s();
let id = run(&mut s, "insert.pageNumber", json!({"format": "x of y"}))["story"].as_u64().unwrap();
let doc = run(&mut s, "document.inspect", json!({}));
let part = doc["parts"].as_array().unwrap().iter().find(|p| p["id"] == id).unwrap().clone();
assert_eq!(part["text"], "Page 1 of 1");
assert_eq!(part["fields"], json!([{"code": "PAGE", "result": "1", "perPage": true}, {"code": "NUMPAGES", "result": "1", "perPage": true}]));
}
/// #4467 review: a STYLEREF in a footer's table cell is listed as per page too (the layout looks
/// it up on each page), and a part with more fields than `document.inspect` lists says so.
#[test]
fn inspect_lists_cell_fields_and_marks_a_cut_list() {
use wordcraft_doc::{Block, CharProps, InlineObject, Paragraph, Table, para_block};
let field =
|instr: &str| InlineObject::Field { instr: instr.into(), result: "x".into(), locked: false, dirty: false, code: false, form: String::new() };
let mut s = s();
let id = run(&mut s, "insert.pageNumber", json!({}))["story"].as_u64().unwrap();
let key = u32::try_from(id).unwrap();
let mut cell = Paragraph::new();
cell.insert_object(0, field("STYLEREF \"Heading 1\""), &CharProps::default()).unwrap();
let mut table = Table::new(1, 1, 200.0);
table.rows[0].cells[0].blocks = vec![para_block(cell)];
s.doc.parts.get_mut(&key).unwrap().blocks.push(std::sync::Arc::new(Block::Table(table)));
let part = |s: &mut Session| {
let doc = run(s, "document.inspect", json!({}));
doc["parts"].as_array().unwrap().iter().find(|p| p["id"] == id).unwrap().clone()
};
let p = part(&mut s);
assert_eq!(p["fields"][1], json!({"code": "STYLEREF \"Heading 1\"", "result": "x", "perPage": true}));
assert!(p.get("fieldsTruncated").is_none(), "{p}");
let mut many = Paragraph::new();
for _ in 0..250 {
many.insert_object(0, field("AUTHOR"), &CharProps::default()).unwrap();
}
s.doc.parts.get_mut(&key).unwrap().blocks.push(para_block(many));
let p = part(&mut s);
assert_eq!(p["fields"].as_array().unwrap().len(), 200);
assert_eq!(p["fieldsTruncated"], true);
}
/// #4467 review: a field kept as a range (`FieldStart` … `FieldEnd`, as the DOCX reader keeps a
/// PAGE whose result has tracked changes, or a TOC spanning paragraphs) is listed too, its result
/// the text between the markers.
#[test]
fn inspect_lists_range_fields() {
use wordcraft_doc::{CharProps, InlineObject, Paragraph, para_block};
let start = |instr: &str| InlineObject::FieldStart { instr: instr.into(), locked: false, dirty: false };
let mut s = s();
let id = run(&mut s, "insert.pageNumber", json!({}))["story"].as_u64().unwrap();
let key = u32::try_from(id).unwrap();
let c = CharProps::default();
let mut page = Paragraph::new();
page.insert_object(0, start("PAGE"), &c).unwrap();
page.insert_text(page.len(), "7 ", &c).unwrap();
page.insert_object(page.len(), InlineObject::FieldEnd, &c).unwrap();
let mut first = Paragraph::new();
first.insert_object(0, start("TOC \\o \"1-3\""), &c).unwrap();
first.insert_text(first.len(), "One", &c).unwrap();
let mut second = Paragraph::with_text("Two", c.clone());
second.insert_object(second.len(), InlineObject::FieldEnd, &c).unwrap();
// A result longer than `document.inspect` gives: cut, and marked.
let mut long = Paragraph::new();
long.insert_object(0, start("BIBLIOGRAPHY"), &c).unwrap();
long.insert_text(long.len(), &"x".repeat(2500), &c).unwrap();
long.insert_object(long.len(), InlineObject::FieldEnd, &c).unwrap();
let blocks = &mut s.doc.parts.get_mut(&key).unwrap().blocks;
blocks.extend([para_block(page), para_block(first), para_block(second), para_block(long)]);
let doc = run(&mut s, "document.inspect", json!({}));
let part = doc["parts"].as_array().unwrap().iter().find(|p| p["id"] == id).unwrap().clone();
let fields = part["fields"].as_array().unwrap();
assert!(fields.contains(&json!({"code": "PAGE", "result": "7 ", "perPage": true})), "spaces kept: {fields:?}");
assert!(fields.contains(&json!({"code": "TOC \\o \"1-3\"", "result": "One\nTwo", "perPage": false})), "{fields:?}");
let bib = fields.iter().find(|f| f["code"] == "BIBLIOGRAPHY").unwrap();
assert_eq!((bib["result"].as_str().map(str::len), &bib["resultTruncated"]), (Some(2000), &json!(true)));
assert!(fields.iter().filter(|f| f["code"] != "BIBLIOGRAPHY").all(|f| f.get("resultTruncated").is_none()));
assert!(part.get("fieldsTruncated").is_none(), "{part}");
}
/// #4467 review: a field name in lower case is a page field all the same, and a field in a table
/// nested deeper than `document.inspect` walks (8 tables) is left out with `fieldsTruncated`.
#[test]
fn inspect_reads_field_names_in_any_case_and_bounds_nested_tables() {
use wordcraft_doc::{Block, CharProps, InlineObject, Paragraph, Table, para_block};
let field =
|instr: &str| InlineObject::Field { instr: instr.into(), result: "1".into(), locked: false, dirty: false, code: false, form: String::new() };
// A field `depth` tables deep.
let nested = |depth: usize| {
let mut p = Paragraph::new();
p.insert_object(0, field("page"), &CharProps::default()).unwrap();
let mut block = para_block(p);
for _ in 0..depth {
let mut t = Table::new(1, 1, 200.0);
t.rows[0].cells[0].blocks = vec![block];
block = std::sync::Arc::new(Block::Table(t));
}
block
};
for (depth, listed) in [(0, true), (8, true), (9, false)] {
let mut s = s();
let id = run(&mut s, "insert.pageNumber", json!({}))["story"].as_u64().unwrap();
let key = u32::try_from(id).unwrap();
s.doc.parts.get_mut(&key).unwrap().blocks = vec![nested(depth)];
let doc = run(&mut s, "document.inspect", json!({}));
let part = doc["parts"].as_array().unwrap().iter().find(|p| p["id"] == id).unwrap().clone();
if listed {
assert_eq!(part["fields"], json!([{"code": "page", "result": "1", "perPage": true}]), "{depth}");
assert!(part.get("fieldsTruncated").is_none(), "{depth}: {part}");
} else {
assert!(part.get("fields").is_none(), "{depth}: {part}");
assert_eq!(part["fieldsTruncated"], true, "{depth}");
}
}
}
/// #3676: a 1-based page parameter becomes a 0-based index without wrapping.
/// On 32-bit and wasm the old cast truncated 2^32 to 0 and `0 - 1` panicked.
#[test]
+36
View File
@@ -328,3 +328,39 @@ fn a_latin_link_inserted_into_reopened_hebrew_is_saved_left_to_right() {
assert_eq!(s.typing_props().rtl, None);
assert_eq!(s.typing_props().lang_bidi.as_deref(), Some("he-IL"), "the bidi language stays");
}
/// #4198: inserting a styled table, typing Arabic into the header row with `para.rtl` and
/// saving as DOCX keeps `w:bCs` beside `w:b` in the table style's header row and first column.
#[test]
fn arabic_table_header_row_and_modified_styles_carry_complex_script_bold_and_italic() {
let mut s = Session::new(wordcraft_doc::Document::new());
run(&mut s, "insert.table", json!({"rows": 2, "cols": 2, "style": "Grid Table 4 – Blue"}));
run(&mut s, "para.rtl", json!({"value": true}));
run(&mut s, "text.insert", json!({"text": "العنوان"}));
let bytes = crate::io::save_bytes("x.docx", &s.doc).unwrap();
let reopened = crate::io::open_bytes("x.docx", &bytes).unwrap();
let blue = reopened.styles.table_style("GridTable4AccentBlue").unwrap();
assert_eq!((blue.parts.header_chr.bold, blue.parts.header_chr.bold_cs), (Some(true), Some(true)));
assert_eq!((blue.parts.first_col_chr.bold, blue.parts.first_col_chr.bold_cs), (Some(true), Some(true)));
// Modifying a style's bold or italic without an explicit complex-script value formats both
// scripts, while an explicit `boldCs` / `italicCs` still wins.
run(&mut s, "styles.modify", json!({"style": "Heading 1", "chr": {"bold": true, "italic": true}}));
let h1 = s.doc.styles.get("Heading1").unwrap();
assert_eq!((h1.chr.bold, h1.chr.bold_cs, h1.chr.italic, h1.chr.italic_cs), (Some(true), Some(true), Some(true), Some(true)));
run(&mut s, "styles.modify", json!({"style": "Heading 1", "chr": {"bold": true, "boldCs": false, "italic": false}}));
let h1 = s.doc.styles.get("Heading1").unwrap();
assert_eq!((h1.chr.bold, h1.chr.bold_cs, h1.chr.italic, h1.chr.italic_cs), (Some(true), Some(false), Some(false), Some(false)));
run(&mut s, "styles.modify", json!({"style": "Heading 1", "chr": {"bold": true}}));
let h1 = s.doc.styles.get("Heading1").unwrap();
assert_eq!((h1.chr.bold, h1.chr.bold_cs), (Some(true), Some(false)), "omitted boldCs keeps a distinct complex-script override");
run(&mut s, "styles.modify", json!({"style": "TOC Heading", "chr": {"font": "Georgia", "bold": true}}));
let toc = s.doc.styles.get("TOCHeading").unwrap();
assert_eq!(toc.chr.bold_cs, None, "child style keeps inheriting parent's distinct complex-script override");
let styles = std::sync::Arc::make_mut(&mut s.doc.styles);
styles.default_chr.italic = Some(true);
styles.default_chr.italic_cs = Some(false);
run(&mut s, "styles.modify", json!({"style": "Normal", "chr": {"italic": true}}));
let normal = s.doc.styles.get("Normal").unwrap();
assert_eq!(normal.chr.italic_cs, None, "style keeps inheriting document default's distinct complex-script override");
}
+21
View File
@@ -371,6 +371,27 @@ fn missing_fonts_from_command_search_say_so_in_the_status_line() {
assert!(s.status.ends_with(" and 35 more") && s.status.len() < 300, "{}", s.status);
}
/// A missing font its publisher gives away says where to get it; others don't.
#[test]
fn a_missing_font_its_publisher_gives_away_names_the_download_page() {
if wordcraft_fonts::FontDb::global().families().iter().any(|f| f.eq_ignore_ascii_case("Aptos")) {
return; // Installed here: not missing.
}
let mut s = Session::new(doc_of(vec![para_in("Text", "Aptos"), para_in("Text", "Wordcraft Nowhere Face")]));
let layout = s.layout();
let m = crate::cmd::font::missing_fonts(&s.doc, &layout);
let aptos = m.iter().find(|f| f.font == "Aptos").unwrap();
assert_eq!(aptos.download.map(|d| d.publisher.as_str()), Some("Microsoft"));
assert!(m.iter().find(|f| f.font == "Wordcraft Nowhere Face").unwrap().download.is_none());
let out = s.run("font.missing", &json!({})).unwrap();
let listed = out["missing"].as_array().unwrap();
let aptos = listed.iter().find(|f| f["font"] == "Aptos").unwrap();
assert!(aptos["download"]["url"].as_str().unwrap().starts_with("https://www.microsoft.com/"), "{aptos}");
assert_eq!(aptos["download"]["publisher"], "Microsoft");
assert_eq!(aptos["download"]["restricted"], false);
assert!(listed.iter().find(|f| f["font"] == "Wordcraft Nowhere Face").unwrap().get("download").is_none());
}
#[test]
fn a_huge_font_name_is_kept_short() {
let name = format!("Wordcraft {}", "é".repeat(100_000));
+57
View File
@@ -141,6 +141,63 @@ fn dragging_a_connector_end_glues_it_to_the_nearest_site() {
run(&mut s, "arrange.nudge", json!({"dx": 1}));
}
/// A plain line's ends move to points the way a connector's free ends do, but never glue: not
/// even dropped on a shape's connection site, nor by reference. A turned line's ends are where
/// it is drawn, and moving one leaves the other there.
#[test]
fn dragging_a_line_end_moves_it_freely() {
let float =
|x: f32, y: f32, rot: f32| Float { wrap: Wrap::InFrontOfText, h_rel: Anchor::Page, v_rel: Anchor::Page, x, y, rot, ..Default::default() };
let shape = |kind, w, h, float, extra| InlineObject::Shape {
kind,
w,
h,
fill: None,
stroke: None,
stroke_width: 1.0,
float,
story: None,
freeform: None,
effects: Default::default(),
extra,
};
let mut p = Paragraph::with_text("Body ", CharProps::default());
for o in [
shape(ShapeKind::Rectangle, 72.0, 36.0, float(100.0, 200.0, 0.0), ShapeExtra { id: 1, ..Default::default() }),
shape(ShapeKind::Line, 200.0, 0.0, float(300.0, 300.0, 0.0), ShapeExtra::default()),
shape(ShapeKind::Line, 200.0, 0.0, float(300.0, 500.0, 90.0), ShapeExtra::default()),
] {
let end = p.len();
p.insert_object(end, o, &CharProps::default()).unwrap();
}
let mut d = Document::new();
d.body = vec![para_block(p)];
let mut s = Session::new(d);
let line_ends = |s: &mut Session, off: usize| {
let pos = Pos::body(0, off);
let hit = s.layout().object(&pos, 0).unwrap();
let Some(InlineObject::Shape { float, extra, .. }) = s.doc.para_at(&pos).and_then(|p| p.object_at(pos.off)).cloned() else { panic!() };
let (a, b) = crate::cmd::connectors::page_ends(hit.rect, float.flip_h, float.flip_v, hit.spin.deg);
let round = |p: (f32, f32)| ((p.0 * 100.0).round() / 100.0, (p.1 * 100.0).round() / 100.0);
(round(a), round(b), extra)
};
select(&mut s, 8);
run(&mut s, "shape.connect", json!({"end": {"x": 500.0, "y": 350.0}}));
assert_eq!(line_ends(&mut s, 8).0, (300.0, 300.0), "the start stays");
assert_eq!(line_ends(&mut s, 8).1, (500.0, 350.0));
// Right on the rectangle's left site (100, 218): a connector would glue; the line doesn't.
run(&mut s, "shape.connect", json!({"start": {"x": 101.0, "y": 217.0}}));
let (a, _, extra) = line_ends(&mut s, 8);
assert_eq!((a, extra.start, extra.end), ((101.0, 217.0), None, None));
assert!(s.run("shape.connect", &json!({"end": {"path": [0], "off": 5}})).is_err(), "a line doesn't glue by reference");
// Turned a quarter: drawn from (400, 400) to (400, 600). Its end moves down; its start stays.
select(&mut s, 11);
assert_eq!(line_ends(&mut s, 11).0, (400.0, 400.0));
run(&mut s, "shape.connect", json!({"end": {"x": 400.0, "y": 650.0}}));
let (a, b, _) = line_ends(&mut s, 11);
assert_eq!((a, b), ((400.0, 400.0), (400.0, 650.0)));
}
#[test]
fn wordart_is_inserted_styled_and_bent() {
let mut s = Session::new(Document::new());
+1
View File
@@ -21,3 +21,4 @@ brotli-decompressor = { workspace = true }
unicode-bidi = { workspace = true }
unicode-script = { workspace = true }
log = { workspace = true }
serde_json = { workspace = true }
+149
View File
@@ -0,0 +1,149 @@
//! Fonts a document may ask for that their publisher gives away, and where: when one isn't
//! installed, the app can say where to get it instead of only naming the substitute.
//!
//! Only a publisher's own page is listed, for fonts it offers for download at no charge. The app
//! links to the page; it never downloads, bundles or redistributes the fonts, and following the
//! publisher's licence terms is the user's part. A font whose publisher limits who may use it
//! is marked [`FontDownload::restricted`], so the app can say so beside the link. Most Office
//! fonts (Calibri, Cambria, Consolas, Segoe UI…) come only with Windows or Office, so they are
//! not here.
//!
//! The list is `font_downloads.json` beside this file, built into the app: adding a font is a
//! new entry there.
/// The list: `{"downloads": [{"family", "publisher", "url", "restricted"?, "note"?}]}`.
const LIST: &str = include_str!("font_downloads.json");
/// Most entries read from the list.
const MAX_DOWNLOADS: usize = 1024;
/// Where a font family can be downloaded at no charge.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FontDownload {
/// The name of the download: the family or collection the page offers. It covers the
/// family itself and every name that continues it after a space ("Aptos" covers "Aptos
/// Display", "Aptos Narrow", "Aptos Mono"…).
pub name: String,
/// Who publishes the font and hosts the page.
pub publisher: String,
/// The publisher's download page.
pub url: String,
/// The publisher limits who may use the font or what for (its design resources for one
/// platform, say): worth saying beside the link.
pub restricted: bool,
}
/// The entries of `list` that name a family, a publisher and an `https` page; others are left
/// out, and a list that isn't JSON has none.
fn parse(list: &str) -> Vec<FontDownload> {
let Ok(json) = serde_json::from_str::<serde_json::Value>(list) else {
log::warn!("font downloads: the list isn't JSON");
return Vec::new();
};
let entries = json.get("downloads").and_then(|d| d.as_array()).map(Vec::as_slice).unwrap_or_default();
entries
.iter()
.take(MAX_DOWNLOADS)
.filter_map(|e| {
let field = |k: &str| e.get(k).and_then(|v| v.as_str()).map(str::trim).filter(|v| !v.is_empty());
let (name, publisher, url) = (field("family")?, field("publisher")?, field("url")?);
let restricted = e.get("restricted").and_then(|v| v.as_bool()).unwrap_or(false);
url.starts_with("https://").then(|| FontDownload { name: name.into(), publisher: publisher.into(), url: url.into(), restricted })
})
.collect()
}
/// The built-in list, read once.
fn downloads() -> &'static [FontDownload] {
static DOWNLOADS: std::sync::OnceLock<Vec<FontDownload>> = std::sync::OnceLock::new();
DOWNLOADS.get_or_init(|| parse(LIST))
}
/// Where `family` can be downloaded free of charge from its publisher, if it can. The name is
/// matched whatever its case, with a charset suffix ("Aptos CE") or alternates ("Aptos;Arial")
/// as a document may write it.
pub fn download_for(family: &str) -> Option<&'static FontDownload> {
family.split(';').map(str::trim).filter(|f| !f.is_empty()).find_map(|f| {
let f = crate::word::charset_base_family(f).trim();
downloads().iter().find(|d| {
// Compared as bytes: `get` on a `str` would refuse a split inside a character.
let (name, f) = (d.name.as_bytes(), f.as_bytes());
f.get(..name.len()).is_some_and(|head| head.eq_ignore_ascii_case(name)) && matches!(f.get(name.len()), None | Some(b' '))
})
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn microsoft_fonts_offered_free_of_charge_have_a_download_page() {
let aptos = download_for("Aptos").unwrap();
assert_eq!(aptos.publisher, "Microsoft");
assert!(aptos.url.starts_with("https://www.microsoft.com/"));
for family in ["aptos", "Aptos Display", "APTOS NARROW", "Aptos Mono", "Aptos Serif", " Aptos ", "Aptos;Arial", "Aptos CE"] {
assert_eq!(download_for(family), Some(aptos), "{family}");
}
assert_eq!(download_for("Cascadia Code").map(|d| d.name.as_str()), Some("Cascadia Code"));
assert_eq!(download_for("Cascadia Mono PL").map(|d| d.name.as_str()), Some("Cascadia Mono"));
assert_eq!(download_for("Selawik Semibold").map(|d| (d.name.as_str(), d.restricted)), Some(("Selawik", false)));
assert!(!aptos.restricted);
}
/// Fonts offered with limits on their use are listed, and say so. Plain Segoe UI isn't
/// offered for download, and the Segoe UI Variable entry doesn't stand in for it.
#[test]
fn fonts_with_limited_licences_are_marked() {
for family in ["Segoe UI Variable", "Segoe UI Variable Text", "segoe fluent icons", "Segoe MDL2 Assets"] {
let d = download_for(family).unwrap_or_else(|| panic!("{family}"));
assert!(d.restricted, "{family}");
assert!(d.url.starts_with("https://learn.microsoft.com/"), "{}", d.url);
}
for family in ["Segoe UI", "Segoe UI Semibold", "Segoe UI Symbol", "Segoe Print"] {
assert_eq!(download_for(family), None, "{family}");
}
}
#[test]
fn other_fonts_have_none() {
// Fonts that come only with Windows or Office, look-alike names, and junk.
for family in ["Calibri", "Cambria", "Consolas", "Arial", "Aptosa", "Apt", "", ";", "Aptosé", "é", "Cascadia"] {
assert_eq!(download_for(family), None, "{family}");
}
}
#[test]
fn every_page_is_https_on_the_publishers_site() {
// Every entry of the file is read: none is left out for a missing field.
let entries = serde_json::from_str::<serde_json::Value>(LIST).unwrap()["downloads"].as_array().unwrap().len();
assert_eq!(downloads().len(), entries);
assert!(entries >= 3);
for d in downloads() {
let hosts = ["https://www.microsoft.com/", "https://learn.microsoft.com/", "https://github.com/microsoft/"];
assert!(hosts.iter().any(|h| d.url.starts_with(h)), "{}", d.url);
}
}
#[test]
fn a_broken_list_has_no_downloads() {
assert!(parse("not json").is_empty());
assert!(parse("{}").is_empty());
assert!(parse(r#"{"downloads": 7}"#).is_empty());
// Entries without a family, a publisher or an https page are left out.
let list = r#"{"downloads": [
{"family": "A", "publisher": "P", "url": "http://plain.example"},
{"family": "", "publisher": "P", "url": "https://x.example"},
{"publisher": "P", "url": "https://x.example"},
{"family": 3, "publisher": "P", "url": "https://x.example"},
"junk",
{"family": " Kept ", "publisher": "P", "url": "https://x.example"}
]}"#;
let read = parse(list);
assert_eq!(read.len(), 1);
assert_eq!(read[0].name, "Kept");
assert!(!read[0].restricted, "not restricted unless it says so");
let marked = parse(r#"{"downloads": [{"family": "A", "publisher": "P", "url": "https://x.example", "restricted": true}]}"#);
assert!(marked[0].restricted);
}
}
+50
View File
@@ -0,0 +1,50 @@
{
"about": "Fonts a document may ask for that their publisher offers for download at no charge, and the publisher's own download page. WordCraft links to the page; it never downloads, bundles or redistributes the fonts, and the user is responsible for the publisher's licence terms. 'family' also covers every name that continues it after a space: 'Aptos' covers 'Aptos Display'. 'restricted': true marks a font whose publisher limits who may use it or what for; the app then says so beside the download.",
"downloads": [
{
"family": "Aptos",
"publisher": "Microsoft",
"url": "https://www.microsoft.com/download/details.aspx?id=106087",
"note": "Microsoft Download Center, 'Microsoft Aptos Fonts': the whole Aptos family."
},
{
"family": "Cascadia Code",
"publisher": "Microsoft",
"url": "https://github.com/microsoft/cascadia-code/releases",
"note": "Microsoft's own repository; SIL Open Font License."
},
{
"family": "Cascadia Mono",
"publisher": "Microsoft",
"url": "https://github.com/microsoft/cascadia-code/releases",
"note": "Comes in the Cascadia Code release."
},
{
"family": "Selawik",
"publisher": "Microsoft",
"url": "https://github.com/microsoft/fonts",
"note": "Microsoft's open-source stand-in for Segoe UI; SIL Open Font License."
},
{
"family": "Segoe UI Variable",
"publisher": "Microsoft",
"url": "https://learn.microsoft.com/windows/apps/design/downloads",
"restricted": true,
"note": "Microsoft's Windows design downloads page. Offered with the design resources for Windows apps; plain 'Segoe UI' is not offered for download and is not matched by this entry."
},
{
"family": "Segoe Fluent Icons",
"publisher": "Microsoft",
"url": "https://learn.microsoft.com/windows/apps/design/downloads",
"restricted": true,
"note": "Microsoft's Windows design downloads page."
},
{
"family": "Segoe MDL2 Assets",
"publisher": "Microsoft",
"url": "https://learn.microsoft.com/windows/apps/design/downloads",
"restricted": true,
"note": "Microsoft's Windows design downloads page."
}
]
}
+2 -2
View File
@@ -26,11 +26,11 @@ mod names;
use face::{BUNDLED, FontBytes};
pub use face::{FALLBACK_FAMILY, FaceRef, FontFace, bundled};
#[cfg(test)]
use fallback_data::{Coverage, RUN_FALLBACK_CACHE_CHARS, SYS_FALLBACK_MAX_FILE, file_coverage, ranges_hold};
#[cfg(not(target_arch = "wasm32"))]
use fallback_data::{CoverageIndex, SysFallback};
use fallback_data::{FallbackCache, RunFallbackCache};
#[cfg(test)]
use fallback_data::{RUN_FALLBACK_CACHE_CHARS, coverage_ranges, ranges_hold};
#[cfg(not(target_arch = "wasm32"))]
use names::CatalogEntry;
pub(crate) use names::face_names;
+57 -45
View File
@@ -1,6 +1,6 @@
use super::face::{FALLBACK_FAMILY, FontFace};
#[cfg(not(target_arch = "wasm32"))]
use super::fallback_data::{CoverageIndex, IndexedFile, SYS_MISSES_MAX, SYSTEM_FALLBACKS, coverage_ranges, ranges_hold, read_fallback_file};
use super::fallback_data::{CoverageIndex, IndexedFile, SYS_MISSES_MAX, SYSTEM_FALLBACKS, file_stamp, ranges_hold, read_fallback_file};
use super::fallback_data::{FALLBACK_CACHE_MAX, RTL_FALLBACKS, RUN_FALLBACK_CACHE_CHARS, RUN_FALLBACK_CACHE_MAX, arabic_block, fallback_order};
use super::{Arc, FontDb};
#[cfg(not(target_arch = "wasm32"))]
@@ -235,7 +235,7 @@ impl FontDb {
}
/// Load a system font covering `c` (preferred fallback families first, then any cataloged
/// file under 40 MB, found through the coverage index). Returns true if one was loaded.
/// file under 40 MB, both found through the coverage index). Returns true if one was loaded.
/// Misses are remembered.
#[cfg(not(target_arch = "wasm32"))]
fn system_fallback(&self, c: char) -> bool {
@@ -257,6 +257,11 @@ impl FontDb {
if self.is_loaded(fam) || !cataloged.iter().any(|f| f.eq_ignore_ascii_case(fam)) {
continue;
}
// A family none of whose files maps `c` is not loaded for it: it would stay loaded
// and help nothing (an emoji loaded every CJK family a Mac has first, 500 MB).
if !self.family_maps(fam, c) {
continue;
}
if !self.sys.lock().unwrap_or_else(std::sync::PoisonError::into_inner).tried.insert(fam) {
continue;
}
@@ -276,8 +281,47 @@ impl FontDb {
false
}
/// Load the first cataloged font file (in path order) that maps `c`, reading each file not
/// yet indexed as the search reaches it. Returns whether a loaded face now covers `c`.
/// Does a file of the installed `family` map `c`? Answered by the coverage index, which reads
/// a file's character maps the first time it is asked about it.
#[cfg(not(target_arch = "wasm32"))]
pub(super) fn family_maps(&self, family: &str, c: char) -> bool {
let paths = self.family_paths(family);
let mut index = self.coverage_index();
paths.iter().any(|p| index.file_holds(p, u32::from(c)))
}
/// The coverage index, its file list brought up to date with the catalog.
#[cfg(not(target_arch = "wasm32"))]
fn coverage_index(&self) -> std::sync::MutexGuard<'_, CoverageIndex> {
let mut index = self.coverage.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
let generation = self.catalog_gen.load(Ordering::Acquire);
if index.generation != Some(generation) {
// A new scan (or the first search): keep what is known of files still installed,
// unless a file was replaced since its character maps were read.
let mut known: HashMap<std::path::PathBuf, IndexedFile> =
std::mem::take(&mut index.files).into_iter().map(|f| (f.path.clone(), f)).collect();
let mut paths: Vec<std::path::PathBuf> =
self.catalog.read().unwrap_or_else(std::sync::PoisonError::into_inner).iter().map(|e| e.path.clone()).collect();
paths.sort();
paths.dedup();
let files = paths.into_iter().map(|path| match known.remove(&path) {
Some(mut file) => {
if file.ranges.is_some() && file.stamp != file_stamp(&file.path) {
file.ranges = None;
}
file
}
None => IndexedFile { used: index.loaded.contains(&path), path, ranges: None, stamp: None },
});
index.files = files.collect();
index.generation = Some(generation);
}
index
}
/// Load the first cataloged font file (in path order) that maps `c`, reading the character
/// maps of each file not yet indexed as the search reaches it. Returns whether a loaded face
/// now covers `c`.
#[cfg(not(target_arch = "wasm32"))]
fn indexed_fallback(&self, c: char) -> bool {
self.load_indexed(&[c], false)
@@ -294,49 +338,22 @@ impl FontDb {
self.load_indexed(chars, true);
}
/// Load the cataloged font files (in path order) that map every char of `chars`, reading each
/// file not yet indexed as the search reaches it, until a loaded face covers them all; with
/// `whole_family`, each such file's family is loaded with it (whatever of it isn't loaded
/// yet). A collection's faces only together may map them: the search then goes on. Returns
/// whether a loaded face now covers them all.
/// Load the cataloged font files (in path order) that map every char of `chars`, reading the
/// character maps of each file not yet indexed as the search reaches it, until a loaded face
/// covers them all; with `whole_family`, each such file's family is loaded with it (whatever
/// of it isn't loaded yet). A collection's faces only together may map them: the search then
/// goes on. Returns whether a loaded face now covers them all.
#[cfg(not(target_arch = "wasm32"))]
fn load_indexed(&self, chars: &[char], whole_family: bool) -> bool {
let covered = |db: &FontDb| db.read_faces().iter().any(|f| chars.iter().all(|c| f.covers(*c)));
let cps: Vec<u32> = chars.iter().map(|c| u32::from(*c)).collect();
let maps_all = |ranges: &[(u32, u32)]| cps.iter().all(|cp| ranges_hold(ranges, *cp));
let mut index = self.coverage.lock().unwrap_or_else(std::sync::PoisonError::into_inner);
let generation = self.catalog_gen.load(Ordering::Acquire);
if index.generation != Some(generation) {
// A new scan (or the first search): keep what is known of files still installed.
let mut known: HashMap<std::path::PathBuf, IndexedFile> =
std::mem::take(&mut index.files).into_iter().map(|f| (f.path.clone(), f)).collect();
let mut paths: Vec<std::path::PathBuf> =
self.catalog.read().unwrap_or_else(std::sync::PoisonError::into_inner).iter().map(|e| e.path.clone()).collect();
paths.sort();
paths.dedup();
let files = paths
.into_iter()
.map(|path| known.remove(&path).unwrap_or_else(|| IndexedFile { used: index.loaded.contains(&path), path, ranges: None }));
index.files = files.collect();
index.generation = Some(generation);
}
let mut index = self.coverage_index();
let CoverageIndex { files, reads, .. } = &mut *index;
for i in 0..files.len() {
let Some(file) = files.get_mut(i).filter(|f| !f.used) else { continue };
let mut data = None;
match &file.ranges {
Some(r) if !maps_all(r) => continue,
Some(_) => {}
None => {
*reads += 1;
data = read_fallback_file(&file.path);
let ranges = data.as_deref().map(coverage_ranges).unwrap_or_default();
let hit = maps_all(&ranges);
file.ranges = Some(ranges);
if !hit {
continue;
}
}
if !maps_all(file.ranges(reads)) {
continue;
}
// The file maps them: load it (once; its faces are loaded fonts from now on).
file.used = true;
@@ -351,17 +368,12 @@ impl FontDb {
load.push(f.path.clone());
}
let fonts: Vec<(Vec<u8>, Option<String>)> = load.iter().filter_map(|p| read_fallback_file(p)).map(|b| (b, None)).collect();
*reads += fonts.len();
if self.add_fonts(fonts) > 0 && covered(self) {
return true;
}
continue;
}
if data.is_none() {
*reads += 1;
data = read_fallback_file(&file.path);
}
if let Some(data) = data
if let Some(data) = read_fallback_file(&file.path)
&& self.add_font(data) > 0
&& covered(self)
{
+127 -50
View File
@@ -1,7 +1,9 @@
use super::face::{FALLBACK_FAMILY, FontFace};
use super::{Arc, HashMap};
#[cfg(not(target_arch = "wasm32"))]
use super::{FileRef, MetadataProvider};
use super::MetadataProvider;
use super::face::{FALLBACK_FAMILY, FontFace};
#[cfg(not(target_arch = "wasm32"))]
use super::names::{TableReader, mini_font};
use super::{Arc, HashMap};
#[cfg(not(target_arch = "wasm32"))]
#[derive(Default)]
@@ -17,14 +19,16 @@ pub(super) struct SysFallback {
#[cfg(not(target_arch = "wasm32"))]
pub(super) const SYS_MISSES_MAX: usize = 65_536;
/// Installed font files larger than this are not searched for fallback characters.
/// Installed font files larger than this are not loaded for a fallback character the coverage
/// index finds in them (a preferred fallback family loads whatever its size).
#[cfg(not(target_arch = "wasm32"))]
const SYS_FALLBACK_MAX_FILE: u64 = 40 << 20;
pub(super) const SYS_FALLBACK_MAX_FILE: u64 = 40 << 20;
/// Which characters each cataloged font file maps (#605). Before it, every distinct character no
/// loaded font covered read and parsed every installed font file again: 500 distinct Tangut
/// characters froze the app for most of a minute. Now each file is read once, when a search
/// first gets to it, and later searches answer from its ranges, reading only a file that matches.
/// characters froze the app for most of a minute. Now each file's character maps are read once,
/// when a search first gets to it, and later searches answer from its ranges, loading only a file
/// that matches. The system fallback asks it too before loading a preferred family.
#[cfg(not(target_arch = "wasm32"))]
#[derive(Default)]
pub(super) struct CoverageIndex {
@@ -32,23 +36,58 @@ pub(super) struct CoverageIndex {
pub(super) generation: Option<u32>,
/// The cataloged font files, sorted.
pub(super) files: Vec<IndexedFile>,
/// Font files read by fallback searches (each at most once to index it, once more to load it).
/// Font files whose character maps fallback searches read (each once, and again only when
/// a rescan finds it replaced).
pub(super) reads: usize,
/// Files loaded by family name ([`FontDb::load_cataloged`]), whether or not they are indexed
/// yet: a search never loads them a second time.
pub(super) loaded: std::collections::HashSet<std::path::PathBuf>,
}
#[cfg(not(target_arch = "wasm32"))]
impl CoverageIndex {
/// Does the cataloged file at `path` map `cp`? Its character maps are read the first time.
pub(super) fn file_holds(&mut self, path: &std::path::Path, cp: u32) -> bool {
let Ok(i) = self.files.binary_search_by(|f| f.path.as_path().cmp(path)) else { return false };
let Some(file) = self.files.get_mut(i) else { return false };
ranges_hold(file.ranges(&mut self.reads), cp)
}
}
#[cfg(not(target_arch = "wasm32"))]
pub(super) struct IndexedFile {
pub(super) path: std::path::PathBuf,
/// The code points the file's faces map, as sorted, disjoint inclusive ranges (`None` until
/// the file is read; empty when it can't be read or is too large).
/// its character maps are read; empty when they can't be).
pub(super) ranges: Option<Box<[(u32, u32)]>>,
/// The file's size and modification time when its character maps were read: a rescan
/// drops the ranges of a file replaced since.
pub(super) stamp: Option<(u64, std::time::SystemTime)>,
/// Loaded for a character it covers: its faces are searched as loaded fonts from then on.
pub(super) used: bool,
}
#[cfg(not(target_arch = "wasm32"))]
impl IndexedFile {
/// The code points the file maps. Its character maps are read the first time (counted in
/// `reads`), and its stamp kept with them, so a rescan keeps them unless the file is replaced.
pub(super) fn ranges(&mut self, reads: &mut usize) -> &[(u32, u32)] {
let (path, stamp) = (&self.path, &mut self.stamp);
self.ranges.get_or_insert_with(|| {
*reads += 1;
*stamp = file_stamp(path);
file_coverage(path)
})
}
}
/// The size and modification time of the file at `path` (`None` when it can't be read).
#[cfg(not(target_arch = "wasm32"))]
pub(super) fn file_stamp(path: &std::path::Path) -> Option<(u64, std::time::SystemTime)> {
let meta = std::fs::metadata(path).ok()?;
Some((meta.len(), meta.modified().ok()?))
}
/// Does `ranges` (sorted, disjoint, inclusive) hold `cp`?
#[cfg(not(target_arch = "wasm32"))]
pub(super) fn ranges_hold(ranges: &[(u32, u32)], cp: u32) -> bool {
@@ -56,56 +95,94 @@ pub(super) fn ranges_hold(ranges: &[(u32, u32)], cp: u32) -> bool {
ranges.get(i).is_some_and(|r| r.0 <= cp)
}
/// The code points mapped by the faces of `data` (a font file or collection), as sorted,
/// disjoint, inclusive ranges. Bounded however damaged the file: at most 256 faces, a budget of
/// mappings read across them, and a cap on the ranges kept.
/// The code points a font file's faces map, gathered face by face into sorted, disjoint,
/// inclusive ranges. Bounded however damaged the file: a budget of mappings read across its
/// faces, and a cap on the ranges kept.
#[cfg(not(target_arch = "wasm32"))]
pub(super) fn coverage_ranges(data: &[u8]) -> Box<[(u32, u32)]> {
const MAX_FACES: u32 = 256;
const MAX_MAPPINGS: usize = 4 * 0x11_0000;
const MAX_RANGES: usize = 1 << 17;
fn compact(r: &mut Vec<(u32, u32)>) {
r.sort_unstable();
let mut out: Vec<(u32, u32)> = Vec::with_capacity(r.len());
for &(s, e) in r.iter() {
#[derive(Default)]
pub(super) struct Coverage {
ranges: Vec<(u32, u32)>,
mappings: usize,
}
#[cfg(not(target_arch = "wasm32"))]
impl Coverage {
pub(super) const MAX_MAPPINGS: usize = 4 * 0x11_0000;
pub(super) const MAX_RANGES: usize = 1 << 17;
/// Add the code points `f` maps. False once a bound is reached: later faces are left out.
pub(super) fn add(&mut self, f: &skrifa::FontRef<'_>) -> bool {
f.charmap().mappings().all(|(cp, _)| self.push(cp))
}
/// Add one mapped code point. False once a bound is reached.
pub(super) fn push(&mut self, cp: u32) -> bool {
if self.mappings >= Self::MAX_MAPPINGS {
return false;
}
self.mappings += 1;
match self.ranges.last_mut() {
Some(last) if last.1.checked_add(1) == Some(cp) => last.1 = cp,
Some(last) if last.0 <= cp && cp <= last.1 => {}
_ => {
if self.ranges.len() >= Self::MAX_RANGES {
self.compact();
// A compaction must free a quarter of the cap, or a crafted file could make
// every later mapping sort all the ranges again.
if self.ranges.len() > Self::MAX_RANGES / 4 * 3 {
return false;
}
}
self.ranges.push((cp, cp));
}
}
true
}
fn compact(&mut self) {
self.ranges.sort_unstable();
let mut out: Vec<(u32, u32)> = Vec::with_capacity(self.ranges.len());
for &(s, e) in self.ranges.iter() {
match out.last_mut() {
Some(last) if s <= last.1.saturating_add(1) => last.1 = last.1.max(e),
_ => out.push((s, e)),
}
}
*r = out;
self.ranges = out;
}
let count = match FileRef::new(data) {
Ok(FileRef::Font(_)) => 1,
Ok(FileRef::Collection(c)) => c.len().min(MAX_FACES),
Err(_) => 0,
};
let mut ranges: Vec<(u32, u32)> = Vec::new();
let mut budget = MAX_MAPPINGS;
'faces: for i in 0..count {
let Ok(f) = skrifa::FontRef::from_index(data, i) else { continue };
for (cp, _) in f.charmap().mappings() {
budget = match budget.checked_sub(1) {
Some(b) => b,
None => break 'faces,
};
match ranges.last_mut() {
Some(last) if last.1.checked_add(1) == Some(cp) => last.1 = cp,
Some(last) if last.0 <= cp && cp <= last.1 => {}
_ => {
if ranges.len() >= MAX_RANGES {
compact(&mut ranges);
if ranges.len() >= MAX_RANGES {
break 'faces;
}
}
ranges.push((cp, cp));
}
}
pub(super) fn finish(mut self) -> Box<[(u32, u32)]> {
self.compact();
self.ranges.into_boxed_slice()
}
}
/// The code points the faces of the font file at `path` map, as sorted, disjoint, inclusive
/// ranges, read from their character maps (`cmap`) alone: Apple Color Emoji's 180 MB cost a few
/// kilobytes. Empty when the file can't be read.
#[cfg(not(target_arch = "wasm32"))]
pub(super) fn file_coverage(path: &std::path::Path) -> Box<[(u32, u32)]> {
/// The longest character map read (a format 12 map of every code point is 13 MB).
const MAX_CMAP: u32 = 16 << 20;
let mut coverage = Coverage::default();
let Some(mut file) = TableReader::open(path) else { return coverage.finish() };
// The faces of a collection often share one map: each is read once.
let mut read: Vec<(u32, u32)> = Vec::new();
for face in 0..file.faces.len() {
let Some(at) = file.locate(face, b"cmap") else { continue };
if read.contains(&at) {
continue;
}
read.push(at);
let Some(cmap) = file.read(at, MAX_CMAP) else { continue };
// A font holding just the map, to read it as the font itself would.
let font = mini_font(&[(*b"cmap", cmap)]);
let Ok(f) = skrifa::FontRef::new(&font) else { continue };
if !coverage.add(&f) {
break;
}
}
compact(&mut ranges);
ranges.into_boxed_slice()
coverage.finish()
}
/// An installed font file's bytes, `None` when it can't be read or is larger than
+79 -34
View File
@@ -145,43 +145,15 @@ const COLOR_BITMAP_TABLES: [&[u8; 4]; 2] = [b"sbix", b"CBDT"];
#[cfg(not(target_arch = "wasm32"))]
pub(super) fn file_face_names(path: &std::path::Path) -> Vec<Found> {
use crate::ui_fallback::OUTLINE_TABLES;
use std::io::{Read, Seek, SeekFrom};
/// Caps on what a (possibly damaged) file can make the scan read.
const MAX_FACES: u32 = 256;
/// The longest `name`, `OS/2` or `fvar` table read.
const MAX_TABLE: u32 = 1 << 20;
let Ok(mut file) = std::fs::File::open(path) else { return vec![] };
let mut read_at = |offset: u64, len: usize| -> Option<Vec<u8>> {
let mut buf = vec![0; len];
file.seek(SeekFrom::Start(offset)).ok()?;
file.read_exact(&mut buf).ok()?;
Some(buf)
};
let be32 = |b: &[u8], at: usize| b.get(at..at + 4).and_then(|s| s.try_into().ok()).map(u32::from_be_bytes);
let Some(head) = read_at(0, 12) else { return vec![] };
// A collection lists where each face's table directory starts.
let starts: Vec<u32> = if head.starts_with(b"ttcf") {
let n = be32(&head, 8).unwrap_or(0).min(MAX_FACES) as usize;
read_at(12, n * 4).map(|b| b.as_chunks::<4>().0.iter().map(|c| u32::from_be_bytes(*c)).collect()).unwrap_or_default()
} else {
vec![0]
};
starts
.into_iter()
.filter_map(|start| {
let dir = read_at(start.into(), 12)?;
let tables = u16::from_be_bytes(dir.get(4..6)?.try_into().ok()?) as usize;
let records = read_at(u64::from(start) + 12, tables * 16)?;
if !records.as_chunks::<16>().0.iter().any(|r| OUTLINE_TABLES.iter().chain(&COLOR_BITMAP_TABLES).any(|t| r.starts_with(*t))) {
let Some(mut file) = TableReader::open(path) else { return vec![] };
(0..file.faces.len())
.filter_map(|face| {
if !OUTLINE_TABLES.iter().chain(&COLOR_BITMAP_TABLES).any(|t| file.locate(face, t).is_some()) {
return None;
}
let mut table = |tag: &[u8; 4]| -> Option<Vec<u8>> {
let rec = records.as_chunks::<16>().0.iter().find(|r| r.starts_with(tag))?;
let (offset, len) = (be32(rec, 8)?, be32(rec, 12)?);
if len > MAX_TABLE {
return None;
}
read_at(offset.into(), len as usize)
};
let mut table = |tag: &[u8; 4]| file.locate(face, tag).and_then(|at| file.read(at, MAX_TABLE));
let name = table(b"name")?;
// Variable fonts: the named instances; OS/2: the code pages (East Asian fonts).
let mut tables: Vec<([u8; 4], Vec<u8>)> = table(b"OS/2").map(|t| (*b"OS/2", t)).into_iter().collect();
@@ -195,6 +167,79 @@ pub(super) fn file_face_names(path: &std::path::Path) -> Vec<Found> {
.collect()
}
/// A font file (or collection) read table by table instead of whole: the font folder scan and
/// the fallback coverage index open hundreds of files, many of them megabytes long (CJK
/// collections run to 20 MB and more, Apple Color Emoji to 180 MB).
#[cfg(not(target_arch = "wasm32"))]
pub(super) struct TableReader {
file: std::fs::File,
/// Each face's table records (tag, checksum, offset, length), for the faces whose table
/// directory could be read.
pub(super) faces: Vec<Vec<[u8; 16]>>,
}
#[cfg(not(target_arch = "wasm32"))]
impl TableReader {
/// Most faces read from a (possibly damaged) collection.
const MAX_FACES: u32 = 256;
/// Open the font file at `path` and read its table directories; `None` when it can't be read.
pub(super) fn open(path: &std::path::Path) -> Option<TableReader> {
let file = std::fs::File::open(path).ok()?;
let mut reader = TableReader { file, faces: Vec::new() };
let head = reader.read_at(0, 12)?;
// A collection lists where each face's table directory starts.
let starts: Vec<u32> = if head.starts_with(b"ttcf") {
let n = be32(&head, 8).unwrap_or(0).min(Self::MAX_FACES) as usize;
reader.read_at(12, n * 4).map(|b| b.as_chunks::<4>().0.iter().map(|c| u32::from_be_bytes(*c)).collect()).unwrap_or_default()
} else {
vec![0]
};
for start in starts {
if let Some(records) = reader.directory(start) {
reader.faces.push(records);
}
}
Some(reader)
}
/// The table records of the face whose table directory starts at `start`.
fn directory(&mut self, start: u32) -> Option<Vec<[u8; 16]>> {
let dir = self.read_at(start.into(), 12)?;
let tables = u16::from_be_bytes(dir.get(4..6)?.try_into().ok()?) as usize;
let records = self.read_at(u64::from(start) + 12, tables * 16)?;
Some(records.as_chunks::<16>().0.to_vec())
}
/// Where face `face`'s table `tag` is: (offset, length).
pub(super) fn locate(&self, face: usize, tag: &[u8; 4]) -> Option<(u32, u32)> {
let rec = self.faces.get(face)?.iter().find(|r| r.starts_with(tag))?;
Some((be32(rec, 8)?, be32(rec, 12)?))
}
/// The table at `(offset, length)`; `None` when it is longer than `max_len` or can't be read.
pub(super) fn read(&mut self, (offset, len): (u32, u32), max_len: u32) -> Option<Vec<u8>> {
if len > max_len {
return None;
}
self.read_at(offset.into(), len as usize)
}
fn read_at(&mut self, offset: u64, len: usize) -> Option<Vec<u8>> {
use std::io::{Read, Seek, SeekFrom};
let mut buf = vec![0; len];
self.file.seek(SeekFrom::Start(offset)).ok()?;
self.file.read_exact(&mut buf).ok()?;
Some(buf)
}
}
/// The big-endian `u32` at `at` in `b`.
#[cfg(not(target_arch = "wasm32"))]
fn be32(b: &[u8], at: usize) -> Option<u32> {
b.get(at..at.checked_add(4)?).and_then(|s| s.try_into().ok()).map(u32::from_be_bytes)
}
/// A font file holding just `tables` (sorted by tag), each padded to four bytes.
#[cfg(not(target_arch = "wasm32"))]
pub(super) fn mini_font(tables: &[([u8; 4], Vec<u8>)]) -> Vec<u8> {
+1
View File
@@ -450,6 +450,7 @@ fn long_names_are_cut_before_faces_copy_them() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn named_instances_and_collection_members_are_capped() {
// #3716: a file's faces are capped before they are built, however the file multiplies them.
let font = variable_font(300);
+150 -15
View File
@@ -305,13 +305,11 @@ fn fallback_cache_is_dropped_when_a_face_loads_by_its_legacy_family_name() {
assert!(!cache.map.contains_key(&(missing, latin.id())), "answers from before the load are dropped");
}
#[test]
fn fallback_reads_each_installed_font_file_once_however_many_characters_miss() {
// #605: every distinct character no font covers read every installed font file again.
let dir = font_dir("coverage");
// The installed Regular also maps U+30000 (no bundled font does): its last cmap group, the
// single character U+1F916, moved there (still the highest, so the groups stay sorted).
let mut regular = renamed("SourceSans3-Regular.ttf");
/// A bundled Source Sans 3 Regular renamed `family` (13 characters) that also maps U+30000 (no
/// bundled font does): its last cmap group, the single character U+1F916, moved there (still the
/// highest, so the groups stay sorted).
fn mapping_u30000(family: &str) -> Vec<u8> {
let mut regular = renamed_to("SourceSans3-Regular.ttf", family);
let (from, to) = ([0x0001_F916_u32.to_be_bytes(); 2].concat(), [0x0003_0000_u32.to_be_bytes(); 2].concat());
let mut patched = 0;
while let Some(at) = regular.windows(from.len()).position(|w| w == from) {
@@ -319,7 +317,14 @@ fn fallback_reads_each_installed_font_file_once_however_many_characters_miss() {
patched += 1;
}
assert!(patched > 0);
std::fs::write(dir.join("Sysfont-Regular.ttf"), regular).unwrap();
regular
}
#[test]
fn fallback_reads_each_installed_font_file_once_however_many_characters_miss() {
// #605: every distinct character no font covers read every installed font file again.
let dir = font_dir("coverage");
std::fs::write(dir.join("Sysfont-Regular.ttf"), mapping_u30000(FAMILY)).unwrap();
let db = FontDb::with_font_dirs(vec![dir]);
let latin = db.face(FALLBACK_FAMILY, "Regular");
let found = db.fallback_for('\u{30000}', latin.id()).expect("the installed font maps it");
@@ -333,7 +338,7 @@ fn fallback_reads_each_installed_font_file_once_however_many_characters_miss() {
}
let index = db.coverage.lock().unwrap();
assert_eq!(index.files.len(), 2, "Regular and Bold are cataloged");
assert_eq!(index.reads, 2, "each file read once");
assert_eq!(index.reads, 2, "each file's character map read once");
}
/// A bundled Source Sans 3 `file` renamed `family` whose last cmap group (the single character
@@ -421,6 +426,45 @@ fn a_family_partly_loaded_by_a_character_fallback_is_loaded_whole_for_a_run() {
assert_eq!(db.read_faces().iter().filter(|f| f.family == "Sysfont Famly").count(), 3);
}
#[test]
fn a_preferred_fallback_family_loads_only_when_it_maps_the_character() {
// A character no loaded font covers loaded every installed family in the fallback list up
// to one that covers it, and kept them all: an emoji loaded every CJK family a Mac has,
// about 500 MB. Malgun Gothic comes before Apple Symbols in that list.
let dir = font_dir("preferred");
std::fs::write(dir.join("Malgun.ttf"), renamed_to("SourceSans3-Regular.ttf", "Malgun Gothic")).unwrap();
std::fs::write(dir.join("Symbols.ttf"), mapping_u30000("Apple Symbols")).unwrap();
let db = FontDb::with_font_dirs(vec![dir.clone()]);
let latin = db.face(FALLBACK_FAMILY, "Regular");
let found = db.fallback_for('\u{30000}', latin.id()).expect("Apple Symbols maps it");
assert_eq!(found.family, "Apple Symbols");
assert!(!db.is_loaded("Malgun Gothic"), "a family that lacks the character isn't loaded for it");
assert_eq!(db.coverage.lock().unwrap().reads, 2, "only the two families' character maps were read");
// The fallback order is unchanged: when both map it, the first in the list is chosen.
std::fs::write(dir.join("Malgun.ttf"), mapping_u30000("Malgun Gothic")).unwrap();
let db = FontDb::with_font_dirs(vec![dir]);
let found = db.fallback_for('\u{30000}', latin.id()).expect("both map it");
assert_eq!(found.family, "Malgun Gothic");
assert!(!db.is_loaded("Apple Symbols"));
}
#[test]
fn a_preferred_fallback_family_in_a_very_large_file_is_still_found() {
// The coverage index reads character maps, not whole files, so a preferred family is found
// whatever its size (PingFang is 75 MB, Apple Color Emoji 180 MB): here a font padded past
// the limit on files the index loads by itself.
let dir = font_dir("large");
let mut data = mapping_u30000("Apple Symbols");
data.resize(data.len() + SYS_FALLBACK_MAX_FILE as usize, 0);
std::fs::write(dir.join("Symbols.ttf"), &data).unwrap();
drop(data);
let db = FontDb::with_font_dirs(vec![dir.clone()]);
let latin = db.face(FALLBACK_FAMILY, "Regular");
let found = db.fallback_for('\u{30000}', latin.id()).map(|f| f.family.clone());
std::fs::remove_file(dir.join("Symbols.ttf")).unwrap();
assert_eq!(found.as_deref(), Some("Apple Symbols"));
}
#[test]
fn screen_hinting_runs_a_tricky_fonts_instructions_with_the_monochrome_target() {
// FreeType's "tricky" fonts (old CJK fonts listed by name, such as DynaLab's "cpop") assemble
@@ -455,20 +499,111 @@ fn screen_hinting_runs_a_tricky_fonts_instructions_with_the_monochrome_target()
}
#[test]
fn coverage_ranges_hold_exactly_the_mapped_characters() {
fn coverage_holds_exactly_the_mapped_characters() {
let dir = font_dir("coverage-ranges");
let data = renamed("SourceSans3-Regular.ttf");
let ranges = coverage_ranges(&data);
let ranges = file_coverage(&dir.join("Sysfont-Regular.ttf"));
assert!(ranges.windows(2).all(|w| w[0].1 + 1 < w[1].0), "sorted, disjoint and merged");
let font = skrifa::FontRef::new(&data).unwrap();
let mapped: Vec<u32> = font.charmap().mappings().map(|(cp, _)| cp).collect();
assert!(ranges.len() < mapped.len() / 4, "ranges, not a set of characters");
assert!(mapped.iter().all(|cp| ranges_hold(&ranges, *cp)));
for cp in [0x0, 0x9, 0x0E01, 0x17000, 0x10_FFFF] {
for cp in [0x0, 0x9, 0x0E01, 0x17000, 0x3_0000, 0x10_FFFF] {
assert_eq!(ranges_hold(&ranges, cp), font.charmap().map(cp).is_some(), "U+{cp:04X}");
}
// Damaged data indexes as nothing.
assert!(coverage_ranges(b"ttcf\0\x01\0\0\xff\xff\xff\xff").is_empty());
assert!(coverage_ranges(&data[..100]).is_empty());
// A collection: what its faces map together.
std::fs::write(dir.join("both.ttc"), collection(&[data.clone(), mapping_u30000(FAMILY)])).unwrap();
let both = file_coverage(&dir.join("both.ttc"));
assert!(ranges_hold(&both, 0x3_0000) && mapped.iter().all(|cp| ranges_hold(&both, *cp)));
// Damaged, cut short, not a font or missing: nothing.
std::fs::write(dir.join("short.ttf"), &data[..100]).unwrap();
for name in ["damaged.ttf", "short.ttf", "readme.txt", "missing.ttf"] {
assert!(file_coverage(&dir.join(name)).is_empty(), "{name}");
}
}
#[test]
fn coverage_stops_when_a_compaction_frees_too_little() {
// A crafted file whose faces interleave their code points made every mapping past the cap
// sort all the ranges again, freeing one: a compaction must free a quarter of the cap.
let cap = u32::try_from(Coverage::MAX_RANGES).unwrap();
let mut c = Coverage::default();
// One face maps every third code point, one range short of the cap; the next face maps the
// code points after them: the first fills the cap, the second compacts it and frees one.
assert!((0..cap - 1).all(|i| c.push(3 * i)));
assert!(c.push(1));
assert!(!c.push(4), "a compaction that frees one range ends the file's coverage");
// A compaction that frees enough goes on.
let mut c = Coverage::default();
assert!((0..cap / 2).all(|i| c.push(2 * i)) && (0..cap / 2).all(|i| c.push(2 * i + 1)));
assert!(c.push(cap + 1), "the compaction merges every range");
assert_eq!(&*c.finish(), &[(0, cap - 1), (cap + 1, cap + 1)]);
}
#[test]
fn a_rescan_reads_a_replaced_font_files_character_maps_again() {
// A font updated in place since its character maps were read: what the update maps counts.
let dir = font_dir("replaced");
let db = FontDb::with_font_dirs(vec![dir.clone()]);
assert!(!db.family_maps(FAMILY, '\u{30000}'));
// The update is a few bytes longer, so it differs however coarse the file times are.
let mut data = mapping_u30000(FAMILY);
data.extend_from_slice(&[0; 4]);
std::fs::write(dir.join("Sysfont-Regular.ttf"), data).unwrap();
assert!(!db.family_maps(FAMILY, '\u{30000}'), "the index answers until the folders are scanned again");
db.load_system_fonts();
assert!(db.family_maps(FAMILY, '\u{30000}'));
assert_eq!(db.coverage.lock().unwrap().reads, 3, "the unchanged Bold file isn't read again");
}
#[test]
fn a_rescan_keeps_the_character_maps_a_fallback_search_read() {
// Files indexed by a fallback search, not only by a family lookup, keep their character maps
// through a rescan: #605's hundreds of files aren't read again after each font folder change.
let dir = font_dir("rescan-fallback");
std::fs::write(dir.join("Sysfont-Regular.ttf"), mapping_u30000(FAMILY)).unwrap();
let db = FontDb::with_font_dirs(vec![dir]);
let latin = db.face(FALLBACK_FAMILY, "Regular");
assert!(db.fallback_for('\u{30000}', latin.id()).is_some(), "the installed font maps it");
assert_eq!(db.coverage.lock().unwrap().reads, 2, "the Bold file and then the Regular");
db.load_system_fonts();
// Nothing maps it: the search reaches the Bold file, which isn't loaded, and asks its ranges.
assert!(db.fallback_for('\u{F0001}', latin.id()).is_none());
assert_eq!(db.coverage.lock().unwrap().reads, 2, "the rescan kept both files' character maps");
}
#[test]
fn coverage_of_installed_fonts_is_what_their_loaded_faces_map() {
// Read from the character maps alone, as the whole file maps. Uses installed fonts when
// there are some, collections among them.
let paths: Vec<&Path> = [
"/System/Library/Fonts/Helvetica.ttc",
"/System/Library/Fonts/Hiragino Sans GB.ttc",
"/System/Library/Fonts/Supplemental/Arial Unicode.ttf",
"C:\\Windows\\Fonts\\arial.ttf",
"C:\\Windows\\Fonts\\msyh.ttc",
"/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
"/usr/share/fonts/noto/NotoSans-Regular.ttf",
]
.iter()
.map(Path::new)
.filter(|p| p.exists())
.collect();
if paths.is_empty() {
eprintln!("no known font installed: nothing to check");
}
for path in paths {
let data = std::fs::read(path).unwrap();
let faces = match FileRef::new(&data).unwrap() {
FileRef::Font(_) => 1,
FileRef::Collection(c) => c.len(),
};
let mut whole = Coverage::default();
for i in 0..faces {
assert!(whole.add(&skrifa::FontRef::from_index(&data, i).unwrap()));
}
assert_eq!(file_coverage(path), whole.finish(), "{}", path.display());
}
}
/// A bundled font renamed by a `name` table holding just its family names: (Windows language id,
+1
View File
@@ -8,6 +8,7 @@
#![forbid(unsafe_code)]
pub mod color;
pub mod download;
mod fontdb;
pub mod math;
mod script;
+70 -12
View File
@@ -40,6 +40,23 @@ pub fn has_named_substitutes(family: &str) -> bool {
const SANS: &[&str] = &["Carlito", "Helvetica Neue", "Arial", "Liberation Sans", "Noto Sans", "DejaVu Sans", "Source Sans 3"];
/// Serif faces known by name with no open clone of their own (lowercase): they take the generic
/// serif list, but as fonts we know, not by a guess from the name ([`has_own_substitutes`]).
const KNOWN_SERIFS: &[&str] = &[
"cambria",
"cambria math",
"georgia",
"garamond",
"constantia",
"baskerville old face",
"century",
"century schoolbook",
"bookman old style",
"sitka text",
"sitka",
"aptos serif",
];
/// [`substitutes`], and whether they were chosen for this family by name.
fn substitutes_by_name(family: &str) -> (&'static [&'static str], bool) {
if family.contains(';') {
@@ -149,18 +166,7 @@ fn substitutes_by_name(family: &str) -> (&'static [&'static str], bool) {
"courier new" | "courier" | "consolas" | "lucida console" | "cascadia code" | "cascadia mono" => (MONO, true),
"symbol" | "wingdings" | "wingdings 2" | "wingdings 3" | "segoe ui symbol" | "webdings" => (SYMBOL, true),
// Serif faces with no open clone of their own: the generic serif list, as for any font.
"cambria"
| "cambria math"
| "georgia"
| "garamond"
| "constantia"
| "baskerville old face"
| "century"
| "century schoolbook"
| "bookman old style"
| "sitka text"
| "sitka"
| "aptos serif" => (SERIF, false),
_ if KNOWN_SERIFS.contains(&f.as_str()) => (SERIF, false),
_ if f.contains("mono") || f.contains("code") => (MONO, false),
_ if f.contains("serif") && !f.contains("sans") => (SERIF, false),
_ => (SANS, false),
@@ -207,6 +213,22 @@ pub fn charset_base_family(family: &str) -> &str {
.map_or(family, str::trim_end)
}
/// Whether `family` (or the font its charset name stands for) has substitutes of its own: Times
/// New Roman, Book Antiqua, Courier New, the symbol, Hebrew and Persian fonts, or a serif face we
/// know (Georgia, Garamond…), rather than the generic list a font we don't know gets, also when
/// its name only suggests one ("… Mono", "… Serif").
pub fn has_own_substitutes(family: &str) -> bool {
let base = charset_base_family(family);
has_named_substitutes(base) || KNOWN_SERIFS.contains(&base.to_ascii_lowercase().as_str())
}
/// Whether `family` has substitutes of its own ([`has_own_substitutes`]) and one of them is
/// installed, so [`resolve`] draws it in a face chosen for it rather than the generic fallback.
pub fn has_installed_substitute(family: &str) -> bool {
let base = charset_base_family(family);
has_own_substitutes(base) && substitute(FontDb::global(), base).is_some()
}
/// Fonts made to measure alike (the same advance widths and line height), so text drawn in one
/// wraps and paginates as in the others: Carlito for Calibri, Caladea for Cambria, Liberation and
/// Croscore fonts for Arial, Times New Roman and Courier New, the Palatino look-alikes for Palatino
@@ -848,6 +870,42 @@ mod tests {
assert_eq!(substitutes(charset_base_family("Times New Roman Cyr")), substitutes("Times New Roman"));
}
/// #2702 review: an alternate counts as drawable only when one of its own substitutes is
/// installed; a font without a list of its own never does.
#[test]
fn installed_substitutes_are_the_fonts_own_and_installed() {
let db = FontDb::global();
for f in ["Times New Roman", "Wingdings", "Courier New", "Book Antiqua"] {
let any = substitutes(f).iter().any(|s| db.has_family(s));
assert_eq!(has_installed_substitute(f), any, "{f}");
}
for f in ["No Such Brand Font", ""] {
assert!(!has_installed_substitute(f), "{f}");
}
}
#[test]
fn fonts_known_by_name_have_their_own_substitutes() {
for f in [
"Times New Roman",
"times",
"Times New Roman Cyr",
"Book Antiqua",
"Palatino Linotype",
"Courier New",
"Arial",
"Georgia",
"Wingdings",
"David",
] {
assert!(has_own_substitutes(f), "{f}");
}
// A name that only suggests a list ("… Mono", "… Serif") isn't a font we know.
for f in ["No Such Brand Font", "NewCenturySchlbk", "", "Brand Serif", "Foo Mono", "Source Code Pro"] {
assert!(!has_own_substitutes(f), "{f}");
}
}
#[test]
fn pcl6_symbol_has_symbols_line_metrics() {
assert_eq!(symbol_font_metrics("Symbol (PCL6)"), symbol_font_metrics("Symbol"));
+1 -1
View File
@@ -33,7 +33,6 @@ craft-zip = { workspace = true }
wordcraft-geom = { workspace = true, features = ["image"] }
craft-metafile = { workspace = true }
encoding_rs = { workspace = true }
chardetng = { workspace = true }
log = { workspace = true }
chrono = { workspace = true }
# Password-protected ODT (ODF 1.3 Part 2 §3.4): AES-256-CBC, SHA-1/SHA-256, PBKDF2 (HMAC-SHA-1).
@@ -41,3 +40,4 @@ chrono = { workspace = true }
wordcraft-doc = { workspace = true, features = ["test-support"] }
craft-image-codecs = { workspace = true }
proptest = { workspace = true }
chardetng = { workspace = true }
+4
View File
@@ -488,6 +488,7 @@ fn code_blocks_and_rules() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn deep_nesting_is_bounded() {
let html = "<div><table><tr><td>".repeat(5000) + "x";
let d = import("html", html.as_bytes()).unwrap().unwrap();
@@ -566,6 +567,7 @@ fn latex_export_carries_no_commands_from_the_document() {
/// past them for the last text made a megabyte of labels take tens of seconds), and labels
/// separated by text are all kept.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn latex_labels_in_a_row_are_read_in_linear_time() {
let started = std::time::Instant::now();
for src in ["\\label{a} ".repeat(100_000) + "x", "\\label{a}".repeat(100_000) + "x", "\\hypertarget{a}{} ".repeat(100_000)] {
@@ -743,6 +745,7 @@ fn decorative_picture_exports_with_empty_alt_text() {
/// Hostile Markdown and HTML that rescanned the rest of the text from every `[`, `![`, `<` or `&`
/// took quadratic time: a few hundred KB froze the importer for tens of seconds.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn hostile_importer_input_takes_linear_time() {
use std::time::{Duration, Instant};
let cases: [(&str, String); 5] = [
@@ -764,6 +767,7 @@ fn hostile_importer_input_takes_linear_time() {
/// bracket scan over them looking for a closing run through the rest of the text: superlinear
/// time on hostile Markdown.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn unmatched_backtick_runs_take_linear_time() {
use std::time::{Duration, Instant};
// Runs of 1, 2, … 2000 backticks (2 MB), none of them closed.
+5 -88
View File
@@ -10,95 +10,12 @@ pub use wordcraft_doc::encoding::cp1252;
use crate::model::{FBlock, ListCounter, from_doc};
fn utf16(b: &[u8], le: bool) -> String {
let units: Vec<u16> = b.as_chunks::<2>().0.iter().map(|c| if le { u16::from_le_bytes(*c) } else { u16::from_be_bytes(*c) }).collect();
String::from_utf16_lossy(&units)
}
#[cfg(test)]
mod legacy_oracle_tests;
/// UTF-16 without a BOM, told by its zero bytes: `Some(true)` little-endian, `Some(false)`
/// big-endian.
fn unmarked_utf16(b: &[u8]) -> Option<bool> {
if b.len() < 4 || !b.len().is_multiple_of(2) {
return None;
}
let sample = b.get(..b.len().min(512))?;
let even_zero = sample.iter().step_by(2).filter(|x| **x == 0).count();
let odd_zero = sample.iter().skip(1).step_by(2).filter(|x| **x == 0).count();
let half = sample.len() / 2;
if odd_zero * 10 >= half * 4 && even_zero * 10 < half {
return Some(true);
}
if even_zero * 10 >= half * 4 && odd_zero * 10 < half {
return Some(false);
}
None
}
/// Does `b` look like plain text, so a file with an extension WordCraft doesn't know (a
/// playlist, a log, a config file…) can open as text the way Word's text import opens it?
///
/// Text has a BOM, or reads as UTF-16 by its zero bytes, or has no NUL byte and next to no
/// other control characters in its first 64 KB. PDFs are refused by their signature (they
/// start as text). An empty file is text.
pub fn is_text(b: &[u8]) -> bool {
if b.starts_with(&[0xEF, 0xBB, 0xBF]) || b.starts_with(&[0xFF, 0xFE]) || b.starts_with(&[0xFE, 0xFF]) {
return true;
}
if b.starts_with(b"%PDF-") {
return false;
}
let sample = b.get(..b.len().min(64 << 10)).unwrap_or(b);
// Tab, line feed, form feed, carriage return, DOS end-of-file (^Z) and escape appear in text;
// other controls in more than 1 of 100 characters mean binary.
let odd = |c: u32| c == 0 || (c < 0x20 && !matches!(c, 0x09 | 0x0A | 0x0C | 0x0D | 0x1A | 0x1B)) || c == 0x7F;
if let Some(le) = unmarked_utf16(sample) {
let units = sample.as_chunks::<2>().0;
let n = units.iter().filter(|c| odd(u32::from(if le { u16::from_le_bytes(**c) } else { u16::from_be_bytes(**c) }))).count();
return n.saturating_mul(100) <= units.len();
}
if sample.contains(&0) {
return false;
}
sample.iter().filter(|c| odd(u32::from(**c))).count().saturating_mul(100) <= sample.len()
}
/// Decode text bytes: BOMs first, then UTF-16 without BOM (by its zero bytes), UTF-8, and the
/// legacy encoding the bytes look like ([`legacy`]) as the last resort.
pub fn decode(b: &[u8]) -> String {
if let Some(rest) = b.strip_prefix(&[0xEF, 0xBB, 0xBF]) {
return String::from_utf8_lossy(rest).into_owned();
}
if let Some(rest) = b.strip_prefix(&[0xFF, 0xFE]) {
return utf16(rest, true);
}
if let Some(rest) = b.strip_prefix(&[0xFE, 0xFF]) {
return utf16(rest, false);
}
if let Some(le) = unmarked_utf16(b) {
return utf16(b, le);
}
match std::str::from_utf8(b) {
Ok(s) => s.to_string(),
Err(_) => legacy(b),
}
}
/// Most bytes looked at to guess a legacy encoding.
const SNIFF: usize = 1 << 20;
/// Bytes that aren't UTF-8 in the legacy encoding they look like (chardetng, the detector
/// Firefox uses for unlabelled pages): GBK/GB18030 for Simplified Chinese, Big5, Shift_JIS or
/// EUC-JP, EUC-KR, or a single-byte Windows / ISO code page; Windows-1252 as before when the
/// guess is a single-byte Western one.
pub fn legacy(b: &[u8]) -> String {
let mut det = chardetng::EncodingDetector::new(chardetng::Iso2022JpDetection::Deny);
det.feed(b.get(..SNIFF).unwrap_or(b), b.len() <= SNIFF);
let enc = det.guess(None, chardetng::Utf8Detection::Deny);
if enc == encoding_rs::WINDOWS_1252 {
return b.iter().map(|c| cp1252(*c)).collect();
}
enc.decode_without_bom_handling(b).0.into_owned()
}
// Detection is shared with the other Craft apps. WordCraft's `cp1252` and encoding_rs agree on
// every byte (pinned by `legacy_oracle_tests`), so a Windows-1252 guess needs no local mapping.
pub use craft_mime::detect::{decode, is_text, legacy};
/// Decode bytes in a named encoding (a WHATWG label such as `gbk`, `gb18030`, `big5`,
/// `shift_jis`, `euc-kr`, `utf-16le`, `windows-1251`); `None` when the label isn't known.
@@ -0,0 +1,210 @@
//! The pre-cutover local implementations, kept as an oracle: the shared `craft_mime::detect`
//! functions plus the cp1252 wrapper must agree with them byte for byte.
use super::cp1252;
fn utf16(b: &[u8], le: bool) -> String {
let units: Vec<u16> = b.as_chunks::<2>().0.iter().map(|c| if le { u16::from_le_bytes(*c) } else { u16::from_be_bytes(*c) }).collect();
String::from_utf16_lossy(&units)
}
/// UTF-16 without a BOM, told by its zero bytes: `Some(true)` little-endian, `Some(false)`
/// big-endian.
fn unmarked_utf16(b: &[u8]) -> Option<bool> {
if b.len() < 4 || !b.len().is_multiple_of(2) {
return None;
}
let sample = b.get(..b.len().min(512))?;
let even_zero = sample.iter().step_by(2).filter(|x| **x == 0).count();
let odd_zero = sample.iter().skip(1).step_by(2).filter(|x| **x == 0).count();
let half = sample.len() / 2;
if odd_zero * 10 >= half * 4 && even_zero * 10 < half {
return Some(true);
}
if even_zero * 10 >= half * 4 && odd_zero * 10 < half {
return Some(false);
}
None
}
/// Does `b` look like plain text, so a file with an extension WordCraft doesn't know (a
/// playlist, a log, a config file…) can open as text the way Word's text import opens it?
///
/// Text has a BOM, or reads as UTF-16 by its zero bytes, or has no NUL byte and next to no
/// other control characters in its first 64 KB. PDFs are refused by their signature (they
/// start as text). An empty file is text.
fn is_text(b: &[u8]) -> bool {
if b.starts_with(&[0xEF, 0xBB, 0xBF]) || b.starts_with(&[0xFF, 0xFE]) || b.starts_with(&[0xFE, 0xFF]) {
return true;
}
if b.starts_with(b"%PDF-") {
return false;
}
let sample = b.get(..b.len().min(64 << 10)).unwrap_or(b);
// Tab, line feed, form feed, carriage return, DOS end-of-file (^Z) and escape appear in text;
// other controls in more than 1 of 100 characters mean binary.
let odd = |c: u32| c == 0 || (c < 0x20 && !matches!(c, 0x09 | 0x0A | 0x0C | 0x0D | 0x1A | 0x1B)) || c == 0x7F;
if let Some(le) = unmarked_utf16(sample) {
let units = sample.as_chunks::<2>().0;
let n = units.iter().filter(|c| odd(u32::from(if le { u16::from_le_bytes(**c) } else { u16::from_be_bytes(**c) }))).count();
return n.saturating_mul(100) <= units.len();
}
if sample.contains(&0) {
return false;
}
sample.iter().filter(|c| odd(u32::from(**c))).count().saturating_mul(100) <= sample.len()
}
/// Decode text bytes: BOMs first, then UTF-16 without BOM (by its zero bytes), UTF-8, and the
/// legacy encoding the bytes look like ([`legacy`]) as the last resort.
fn decode(b: &[u8]) -> String {
if let Some(rest) = b.strip_prefix(&[0xEF, 0xBB, 0xBF]) {
return String::from_utf8_lossy(rest).into_owned();
}
if let Some(rest) = b.strip_prefix(&[0xFF, 0xFE]) {
return utf16(rest, true);
}
if let Some(rest) = b.strip_prefix(&[0xFE, 0xFF]) {
return utf16(rest, false);
}
if let Some(le) = unmarked_utf16(b) {
return utf16(b, le);
}
match std::str::from_utf8(b) {
Ok(s) => s.to_string(),
Err(_) => legacy(b),
}
}
/// Most bytes looked at to guess a legacy encoding.
const SNIFF: usize = 1 << 20;
/// Bytes that aren't UTF-8 in the legacy encoding they look like (chardetng, the detector
/// Firefox uses for unlabelled pages): GBK/GB18030 for Simplified Chinese, Big5, Shift_JIS or
/// EUC-JP, EUC-KR, or a single-byte Windows / ISO code page; Windows-1252 as before when the
/// guess is a single-byte Western one.
fn legacy(b: &[u8]) -> String {
let mut det = chardetng::EncodingDetector::new(chardetng::Iso2022JpDetection::Deny);
det.feed(b.get(..SNIFF).unwrap_or(b), b.len() <= SNIFF);
let enc = det.guess(None, chardetng::Utf8Detection::Deny);
if enc == encoding_rs::WINDOWS_1252 {
return b.iter().map(|c| cp1252(*c)).collect();
}
enc.decode_without_bom_handling(b).0.into_owned()
}
fn samples() -> Vec<Vec<u8>> {
let mut v: Vec<Vec<u8>> = vec![vec![]];
for b in 0..=255u8 {
v.push(vec![b]);
}
for b in 0x80..=0x9Fu8 {
v.push([b"caf".as_slice(), &[b], b" ok text here"].concat());
v.push(vec![b, b'a', b'b']);
}
// Western text with each byte Windows-1252 leaves undefined: the case the cp1252 wrapper is for.
for b in [0x81u8, 0x8D, 0x8F, 0x90, 0x9D] {
v.push([b"caf\xE9 au lait ".as_slice(), &[b], b" ok"].concat());
v.push([b"Le caf\xE9 de la gare ".as_slice(), &[b]].concat());
v.push([b"r\xE9sum\xE9 ".as_slice(), &[b]].concat());
}
v.push("héllo wörld € naïve".as_bytes().to_vec());
v.push([&[0xEF, 0xBB, 0xBF][..], "héllo".as_bytes()].concat());
v.push([&[0xEF, 0xBB, 0xBF][..], &[0xE9, 0x80]].concat());
for (bom, le) in [(&[0xFF, 0xFE][..], true), (&[0xFE, 0xFF][..], false)] {
let t = "héllo wörld 日本語 😀";
v.push([bom, &t.encode_utf16().flat_map(|u| if le { u.to_le_bytes() } else { u.to_be_bytes() }).collect::<Vec<u8>>()[..]].concat());
v.push([bom, b"h\0i\0!"].concat());
let raw: Vec<u8> = "hello world, plain ascii".encode_utf16().flat_map(|u| if le { u.to_le_bytes() } else { u.to_be_bytes() }).collect();
v.push(raw);
let raw: Vec<u8> =
"héllo wörld, mostly ascii \u{20AC}".encode_utf16().flat_map(|u| if le { u.to_le_bytes() } else { u.to_be_bytes() }).collect();
v.push(raw);
}
v.push(vec![0xC3, 0x28, 0xA0, 0xA1, 0xE2, 0x82]);
v.push(b"#EXTM3U\r\n#EXTINF:1,x\r\n".to_vec());
v.push(b"%PDF-1.7\n%\xE2\xE3".to_vec());
v.push(b"\x89PNG\r\n\x1a\n\0\0\0\rIHDR".to_vec());
v.push(vec![1, 0, 2, 0, 3]);
for (enc, text) in [
(encoding_rs::GBK, "这是一个用简体中文写的文本文件,用来测试编码检测。"),
(encoding_rs::BIG5, "這是一個用繁體中文寫的文字檔案,用來測試編碼偵測。"),
(encoding_rs::SHIFT_JIS, "これは日本語で書かれたテキストファイルです。エンコーディングの検出をテストします。"),
(encoding_rs::EUC_KR, "이것은 한국어로 작성된 텍스트 파일입니다. 인코딩 감지를 테스트합니다."),
(encoding_rs::WINDOWS_1251, "Это текстовый файл на русском языке для проверки определения кодировки."),
] {
let (b, _, _) = enc.encode(text);
v.push(b.into_owned());
}
// Past the 1 MiB sniff limit, and a 2 MiB input.
let mut big = vec![b'a'; 1 << 20];
big.extend_from_slice(&[0xE9; 3]);
v.push(big);
let mut big = "héllo wörld ".repeat((2 << 20) / 13).into_bytes();
big.push(0xE9);
v.push(big);
v.push(vec![0x80; 2 << 20]);
v
}
fn assert_same(b: &[u8]) {
let head: Vec<u8> = b.iter().take(24).copied().collect();
assert_eq!(is_text(b), super::is_text(b), "is_text {head:02x?}");
assert_eq!(legacy(b), super::legacy(b), "legacy {head:02x?}");
assert_eq!(decode(b), super::decode(b), "decode {head:02x?}");
}
#[test]
fn shared_detect_matches_the_old_local_copies() {
for b in samples() {
assert_same(&b);
}
}
#[test]
fn cp1252_agrees_with_encoding_rs_on_every_byte() {
// Why a Windows-1252 guess needs no local mapping: both give U+0081, U+008D, U+008F, U+0090
// and U+009D for the bytes the code page leaves undefined.
for b in 0..=255u8 {
let one = [b];
let (s, _) = encoding_rs::WINDOWS_1252.decode_without_bom_handling(&one);
assert_eq!(s.chars().collect::<Vec<_>>(), vec![cp1252(b)], "byte {b:#x}");
}
}
#[test]
fn random_buffers_never_panic_and_match() {
let mut s: u64 = 0x2545_F491_4F6C_DD1D;
let mut next = || {
s = s.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1_442_695_040_888_963_407);
(s >> 33) as u32
};
for i in 0..3000 {
let len = (next() % 700) as usize;
let mode = i % 4;
let b: Vec<u8> = (0..len)
.map(|_| {
let r = next();
match mode {
0 => r as u8,
1 => 0x80 | (r as u8 & 0x1F), // 0x80..=0x9F
2 => {
if r % 3 == 0 {
0
} else {
(r >> 8) as u8 % 0x80
}
}
_ => {
if r % 5 == 0 {
0x80 + (r >> 8) as u8 % 0x80
} else {
b'a' + (r >> 8) as u8 % 26
}
}
}
})
.collect();
assert_same(&b);
}
}
+713 -198
View File
@@ -9,7 +9,8 @@ use std::borrow::Cow;
use craft_image_codecs::{
ChannelLayout, CodecError, DecodeOptions, EncodeOptions, Format as CodecFormat, Image as CodecImage, Limits as CodecLimits, RasterColorType,
SampleType, decode_as_with, detect, encode as codec_encode, inspect_raster, probe_size,
SampleType, bmp_probe, decode_as_with, detect, encode as codec_encode, inspect_raster, jpeg_worst_case_coefficients, png_probe, probe_size,
webp_probe,
};
use image::{DynamicImage, ImageBuffer, ImageFormat};
@@ -147,7 +148,7 @@ pub fn encode_jpeg(rgb: &[u8], width: u32, height: u32, quality: u8) -> Result<V
pub fn dimensions(bytes: &[u8]) -> Option<(u32, u32)> {
let format = detect(bytes)?;
match format {
CodecFormat::Png => png_chunks(bytes).map(|(w, h, _, _)| (w, h)),
CodecFormat::Png => png_probe::pixel_model(bytes).ok().map(|m| (m.width, m.height)),
CodecFormat::Jpeg | CodecFormat::Tiff | CodecFormat::Gif | CodecFormat::Bmp => {
inspect_raster(bytes).ok().flatten().map(|i| (i.width, i.height))
}
@@ -197,7 +198,7 @@ pub fn probe(bytes: &[u8]) -> Result<Probe<'_>, Refusal> {
let format = from_codec_format(codec_fmt).ok_or(Refusal::NotAPicture)?;
let held = match format {
ImageFormat::Png | ImageFormat::Gif | ImageFormat::WebP | ImageFormat::Bmp => 0,
ImageFormat::Jpeg => jpeg_coefficients(bytes),
ImageFormat::Jpeg => jpeg_worst_case_coefficients(bytes),
ImageFormat::Tiff => {
if !tiff_compression(bytes).is_some_and(|c| TIFF_COMPRESSIONS.contains(&c)) {
return Err(Refusal::Unsupported("TIFF pictures with JPEG or an unknown compression can't be read"));
@@ -207,14 +208,17 @@ pub fn probe(bytes: &[u8]) -> Result<Probe<'_>, Refusal> {
_ => return Err(Refusal::NotAPicture),
};
if format == ImageFormat::Png {
let (width, height, native, alpha) = png_chunks(bytes).ok_or_else(|| Refusal::Broken("its PNG header can't be read".into()))?;
// Measured from `IHDR` and the chunk headers: an ICC profile (`iCCP`) isn't inflated, though the
// decoder's own header read inflates one up to its 512 MiB limit.
let m = png_probe::pixel_model(bytes).map_err(|_| Refusal::Broken("its PNG header can't be read".into()))?;
let (width, height, native, alpha) = (m.width, m.height, u64::from(m.bytes_per_pixel()), m.has_alpha());
let (width, height, peak) = sized(width, height, native, 0)?;
return Ok(Probe { bytes: Cow::Borrowed(bytes), format, width, height, peak, alpha });
}
if format == ImageFormat::WebP {
let (width, height) = probe_size(bytes).map_err(broken)?;
let (width, height, peak) = sized(width, height, 4, 0)?;
let alpha = bytes.get(12..16) == Some(b"VP8X") && bytes.get(20).is_some_and(|b| b & 0x10 != 0);
let alpha = webp_probe::has_alpha(bytes).unwrap_or(false);
return Ok(Probe { bytes: Cow::Borrowed(bytes), format, width, height, peak, alpha });
}
let info = inspect_raster(bytes).map_err(broken)?.ok_or_else(|| Refusal::Broken("its header can't be read".into()))?;
@@ -225,84 +229,26 @@ pub fn probe(bytes: &[u8]) -> Result<Probe<'_>, Refusal> {
Ok(Probe { bytes: Cow::Borrowed(bytes), format, width, height, peak, alpha })
}
/// A PNG's width, height, the bytes a pixel its decoder gives (samples under 8 bits widened to 8,
/// a palette to RGB, `tRNS` to an alpha channel) and whether it has transparency, from `IHDR` and
/// the chunk headers before the image data. The decoder's own header read inflates an ICC profile
/// (`iCCP`) up to its 512 MiB default limit, for a picture that may only be measured.
fn png_chunks(bytes: &[u8]) -> Option<(u32, u32, u64, bool)> {
let be = |at: usize| bytes.get(at..at.checked_add(4)?).and_then(|b| <[u8; 4]>::try_from(b).ok()).map(u32::from_be_bytes);
// The 8-byte signature, then IHDR: length, type, width, height, bit depth, colour type.
if bytes.get(12..16) != Some(b"IHDR".as_slice()) {
return None;
}
let (width, height) = (be(16)?, be(20)?);
let (depth, color) = (*bytes.get(24)?, *bytes.get(25)?);
let mut trns = false;
let mut at = 8usize;
loop {
let len = usize::try_from(be(at)?).ok()?;
match bytes.get(at.checked_add(4)?..at.checked_add(8)?)? {
b"IDAT" => break,
b"IEND" => return None,
b"tRNS" => trns = true,
_ => {}
}
at = at.checked_add(12)?.checked_add(len)?;
}
let (channels, alpha) = match color {
0 => (1 + u64::from(trns), trns),
2 | 3 => (3 + u64::from(trns), trns),
4 => (2, true),
6 => (4, true),
_ => return None,
};
Some((width, height, channels * if depth == 16 { 2 } else { 1 }, alpha))
}
/// Prepend a 14-byte `BITMAPFILEHEADER` to a normalized DIB so `craft-image-codecs` can inspect and decode it as BMP.
fn dib_to_bmp(dib: &[u8]) -> Vec<u8> {
let u16_at = |at: usize| dib.get(at..at + 2).and_then(|b| b.try_into().ok()).map(u16::from_le_bytes);
let u32_at = |at: usize| dib.get(at..at + 4).and_then(|b| b.try_into().ok()).map(u32::from_le_bytes);
let bi_size = u32_at(0).unwrap_or(40);
let offset = if bi_size == 12 {
let bpp = u16_at(10).unwrap_or(24);
let palette = if bpp <= 8 { (1u32 << bpp) * 3 } else { 0 };
14 + bi_size + palette
} else {
let bpp = u16_at(14).unwrap_or(24);
let compression = u32_at(16).unwrap_or(0);
let clr_used = u32_at(32).unwrap_or(0);
let palette_entries = if clr_used != 0 {
clr_used
} else if bpp <= 8 {
1u32 << bpp
} else {
0
};
let masks = if compression == 3 && bi_size >= 40 { 12 } else { 0 };
14u32.saturating_add(bi_size).saturating_add(masks).saturating_add(palette_entries.saturating_mul(4))
};
let file_size = u32::try_from(14 + dib.len()).unwrap_or(u32::MAX);
let mut bmp = Vec::with_capacity(14 + dib.len());
bmp.extend_from_slice(b"BM");
bmp.extend_from_slice(&file_size.to_le_bytes());
bmp.extend_from_slice(&[0u8; 4]);
bmp.extend_from_slice(&offset.to_le_bytes());
bmp.extend_from_slice(dib);
bmp
/// [`bmp_probe::dib_to_bmp`], its failure (the BMP's bytes can't be allocated) as the refusal a
/// picture too large to hold gets, with the size its header declares (0 × 0 when it can't be read).
fn dib_to_bmp(dib: &[u8]) -> Result<Vec<u8>, Refusal> {
bmp_probe::dib_to_bmp(dib).map_err(|_| {
let (width, height) = dib_header_dims(dib).unwrap_or((0, 0));
Refusal::TooLarge { width, height }
})
}
/// Read a DIB's header: a BMP without its 14-byte file header, as the Windows clipboard holds
/// one (`CF_DIB`, `CF_DIBV5`). Its layout is first made the one the decoder reads (see
/// [`dib_layout`]).
/// [`bmp_probe::dib_layout`]).
///
/// # Errors
///
/// Returns `Refusal::Broken` for an invalid DIB header and `Refusal::TooLarge` on size arithmetic
/// overflow.
pub fn probe_dib(dib: Vec<u8>) -> Result<Probe<'static>, Refusal> {
let dib = dib_layout(dib);
let bmp = dib_to_bmp(&dib);
let dib = bmp_probe::dib_layout(dib);
let bmp = dib_to_bmp(&dib)?;
let (width, height, native, alpha) = match inspect_raster(&bmp) {
Ok(Some(info)) => {
let native = u64::from(info.channels.max(1)) * u64::from(info.bits_per_sample.max(8) / 8);
@@ -320,19 +266,16 @@ pub fn probe_dib(dib: Vec<u8>) -> Result<Probe<'static>, Refusal> {
Ok(Probe { bytes: Cow::Owned(bmp), format: ImageFormat::Bmp, width, height, peak, alpha })
}
/// A DIB's width and height, read by the shared BMP probe. Header sizes other than 12 (core) and
/// 40 or more (info) are refused here: the probe would read them as info headers.
fn dib_header_dims(dib: &[u8]) -> Option<(u32, u32)> {
let bi_size = u32::from_le_bytes(dib.get(0..4)?.try_into().ok()?);
if bi_size == 12 {
let w = u16::from_le_bytes(dib.get(4..6)?.try_into().ok()?);
let h = u16::from_le_bytes(dib.get(6..8)?.try_into().ok()?);
Some((u32::from(w), u32::from(h)))
} else if bi_size >= 40 {
let w = i32::from_le_bytes(dib.get(4..8)?.try_into().ok()?);
let h = i32::from_le_bytes(dib.get(8..12)?.try_into().ok()?);
Some((u32::try_from(w).ok()?, h.unsigned_abs()))
} else {
None
if bi_size != 12 && bi_size < 40 {
return None;
}
// Only the first 12 header bytes are read, so only they are wrapped: this runs for DIBs too
// big to decode, which must not be copied whole.
craft_image_codecs::bmp_probe::dimensions(&bmp_probe::dib_to_bmp(dib.get(..12).unwrap_or(dib)).ok()?)
}
/// A `width` × `height` picture of `native` bytes a pixel, with its peak (see [`peak_bytes`]).
@@ -477,64 +420,6 @@ fn peak_bytes(width: u32, height: u32, native: u64, held: u64) -> Option<u64> {
Some(decoding.max(converting).max(premultiplying))
}
/// Bytes of DCT coefficients a progressive JPEG's decoder keeps for the whole picture while it
/// reads the scans: zune-jpeg holds an `i16` per coefficient of every component, in whole MCUs.
/// 0 for a baseline JPEG, decoded an MCU row at a time. Each frame header before the first scan
/// counts and the largest wins; headers this can't follow are charged as the costliest frame.
fn jpeg_coefficients(bytes: &[u8]) -> u64 {
let mut at = 2;
let mut most = 0u64;
loop {
if bytes.get(at) != Some(&0xFF) {
return worst_coefficients(bytes).max(most);
}
while bytes.get(at) == Some(&0xFF) {
at += 1;
}
let Some(&code) = bytes.get(at) else { return worst_coefficients(bytes).max(most) };
at += 1;
match code {
// Markers without a length: start of image, TEM, restarts.
0xD8 | 0x01 | 0xD0..=0xD7 => continue,
// End of image, start of scan: every frame header is behind.
0xD9 | 0xDA => return most,
_ => {}
}
let len = match (bytes.get(at), bytes.get(at + 1)) {
(Some(&hi), Some(&lo)) => usize::from(u16::from_be_bytes([hi, lo])),
_ => 0,
};
let Some(segment) = bytes.get(at + 2..at + len).filter(|_| len >= 2) else { return worst_coefficients(bytes).max(most) };
// SOF2, SOF6, SOF10 and SOF14 are the progressive frames.
if matches!(code, 0xC2 | 0xC6 | 0xCA | 0xCE) {
let Some(held) = frame_coefficients(segment) else { return worst_coefficients(bytes).max(most) };
most = most.max(held);
}
at += len;
}
}
/// [`jpeg_coefficients`] of one progressive frame header (after its length): the components'
/// blocks per MCU (horizontal × vertical sampling) times the MCUs that cover the picture.
fn frame_coefficients(sof: &[u8]) -> Option<u64> {
let height = u64::from(u16::from_be_bytes([*sof.get(1)?, *sof.get(2)?]));
let width = u64::from(u16::from_be_bytes([*sof.get(3)?, *sof.get(4)?]));
let n = usize::from(*sof.get(5)?);
let sampling: Vec<(u64, u64)> =
sof.get(6..6 + 3 * n)?.as_chunks::<3>().0.iter().map(|&[_, s, _]| (u64::from(s >> 4).clamp(1, 4), u64::from(s & 15).clamp(1, 4))).collect();
let (h_max, v_max) = sampling.iter().fold((1, 1), |(h, v), &(a, b)| (h.max(a), v.max(b)));
let mcus = width.div_ceil(8 * h_max).checked_mul(height.div_ceil(8 * v_max))?;
sampling.iter().try_fold(0u64, |sum, &(h, v)| sum.checked_add(mcus.checked_mul(2 * 64 * h * v)?))
}
/// The coefficients of the costliest progressive frame of the size the image crate reads from
/// `bytes`: four components, each sampled 4 × 4, in MCUs of 32 pixels a side.
fn worst_coefficients(bytes: &[u8]) -> u64 {
let Some((w, h)) = dimensions(bytes) else { return 0 };
let mcus = u64::from(w).div_ceil(32) * u64::from(h).div_ceil(32);
mcus.saturating_mul(2 * 64 * 16 * 4)
}
/// The compression of a classic TIFF's first image (the one the image crate decodes): 1 when the
/// tag is missing. None when the header can't be read, or when the tag appears more than once:
/// the decoder keeps the last copy, so a harmless first one must not hide a JPEG one after it.
@@ -542,41 +427,72 @@ fn tiff_compression(bytes: &[u8]) -> Option<u16> {
craft_tiff::probe::first_image_compression(bytes)
}
/// A DIB with a V4 or V5 header in the layout the decoder reads. A 32-bit one whose alpha mask
/// is set but whose compression says `BI_RGB` is marked `BI_BITFIELDS` (with the usual colour
/// masks when it gives none), so its alpha is kept as Windows programs mean it; arboard makes the
/// same change. Bit fields then need the colour masks after the header too, where Windows puts a
/// copy and the decoder skips them; a DIB without that copy (arboard writes one) gets it.
fn dib_layout(mut dib: Vec<u8>) -> Vec<u8> {
let u32_at = |d: &[u8], at: usize| d.get(at..at + 4).and_then(|b| b.try_into().ok()).map(u32::from_le_bytes);
let Some(size) = u32_at(&dib, 0).filter(|&s| s >= 108).and_then(|s| usize::try_from(s).ok()) else { return dib };
let put = |d: &mut Vec<u8>, at: usize, v: u32| {
if let Some(b) = d.get_mut(at..at + 4) {
b.copy_from_slice(&v.to_le_bytes());
}
};
if dib.get(14..16) == Some(&[32, 0]) && u32_at(&dib, 16) == Some(0) && u32_at(&dib, 52) == Some(0xFF00_0000) {
put(&mut dib, 16, 3);
if (40..52).step_by(4).all(|at| u32_at(&dib, at) == Some(0)) {
put(&mut dib, 40, 0x00FF_0000);
put(&mut dib, 44, 0x0000_FF00);
put(&mut dib, 48, 0x0000_00FF);
}
}
if u32_at(&dib, 16) == Some(3)
&& size <= dib.len()
&& let Some(masks) = dib.get(40..52).map(<[u8]>::to_vec)
&& dib.get(size..size + 12) != Some(masks.as_slice())
{
dib.splice(size..size, masks);
}
dib
}
#[cfg(test)]
mod tests {
use super::*;
// Hand-written reader the shared probe replaced; kept as the oracle.
fn dib_header_dims_old(dib: &[u8]) -> Option<(u32, u32)> {
let bi_size = u32::from_le_bytes(dib.get(0..4)?.try_into().ok()?);
if bi_size == 12 {
let w = u16::from_le_bytes(dib.get(4..6)?.try_into().ok()?);
let h = u16::from_le_bytes(dib.get(6..8)?.try_into().ok()?);
Some((u32::from(w), u32::from(h)))
} else if bi_size >= 40 {
let w = i32::from_le_bytes(dib.get(4..8)?.try_into().ok()?);
let h = i32::from_le_bytes(dib.get(8..12)?.try_into().ok()?);
Some((u32::try_from(w).ok()?, h.unsigned_abs()))
} else {
None
}
}
fn hdr(size: u32, w: i32, h: i32, len: usize) -> Vec<u8> {
let mut v = [0u8; 124].to_vec();
v.resize(124.max(len), 0);
v[0..4].copy_from_slice(&size.to_le_bytes());
v[4..8].copy_from_slice(&w.to_le_bytes());
v[8..12].copy_from_slice(&h.to_le_bytes());
v.truncate(len);
v
}
#[test]
fn dib_dims_match_the_hand_written_reader() {
let sizes = [0u32, 11, 12, 13, 16, 39, 40, 41, 52, 108, 124, u32::MAX];
let dims = [(0, 0), (1, 1), (3, -2), (0, 5), (7, i32::MIN), (i32::MIN, 1), (-1, 1), (65536, 70000), (i32::MAX, i32::MAX)];
for &s in &sizes {
for &(w, h) in &dims {
for len in 0..=130 {
let d = hdr(s, w, h, len);
assert_eq!(dib_header_dims(&d), dib_header_dims_old(&d), "size {s} w {w} h {h} len {len}");
}
}
}
assert_eq!(dib_header_dims(&hdr(40, 3, -2, 40)), Some((3, 2)));
let mut core = hdr(12, 3, 0, 12);
core[6..8].copy_from_slice(&2u16.to_le_bytes());
assert_eq!(dib_header_dims(&core), Some((3, 2)));
}
#[test]
fn dib_dims_never_panic_on_junk() {
let mut x = 0x2545_f491_4f6c_dd1du64;
let mut buf = [0u8; 64];
for _ in 0..2000 {
for b in buf.iter_mut() {
x ^= x << 13;
x ^= x >> 7;
x ^= x << 17;
*b = x as u8;
}
for n in 0..=buf.len() {
assert_eq!(dib_header_dims(&buf[..n]), dib_header_dims_old(&buf[..n]));
let _ = probe_dib(buf[..n].to_vec());
}
}
}
fn encode(img: impl Into<DynamicImage>, format: ImageFormat) -> Vec<u8> {
let dyn_img = img.into();
let codec_img = dynamic_to_codec(&dyn_img).unwrap();
@@ -709,33 +625,120 @@ mod tests {
assert_eq!(probe(&baseline).unwrap().peak_bytes(), 64 * 64 * 8);
}
/// A JPEG's headers up to the first scan: frames `(marker, precision, w, h, components)`, then
/// a scan of `scan` components (none: no scan, the end of the image).
fn jpeg_headers(frames: &[(u8, u8, u16, u16, &[u8])], scan: Option<u8>) -> Vec<u8> {
let mut b = vec![0xFF, 0xD8, 0xFF, 0xE0, 0, 4, 0, 0];
for &(marker, prec, w, h, comps) in frames {
b.extend_from_slice(&[0xFF, marker]);
#[expect(clippy::cast_possible_truncation, reason = "Fixtures hold at most four components.")]
let (n, len) = ((comps.len() / 3) as u8, (8 + comps.len()) as u16);
b.extend(len.to_be_bytes());
b.push(prec);
b.extend(h.to_be_bytes());
b.extend(w.to_be_bytes());
b.push(n);
b.extend_from_slice(comps);
}
if let Some(n) = scan {
b.extend_from_slice(&[0xFF, 0xDA, 0, 6 + 2 * n, n]);
(1..=n).for_each(|c| b.extend_from_slice(&[c, 0]));
b.extend_from_slice(&[0, 63, 0]);
}
b.extend_from_slice(&[0xFF, 0xD9]);
b
}
const C444: &[u8] = &[1, 0x11, 0, 2, 0x11, 0, 3, 0x11, 0];
const C420: &[u8] = &[1, 0x22, 0, 2, 0x11, 0, 3, 0x11, 0];
const GREY: &[u8] = &[1, 0x11, 0];
/// The JPEG budget before the cutover to the shared rule, as the oracle: the shared charge is
/// never less, and is equal wherever it adds nothing.
#[test]
fn coefficients_follow_sampling_and_mcu_padding() {
// P, height, width, components (id, sampling, table).
let sof = |w: u16, h: u16, comps: &[u8]| {
let mut s = vec![8];
s.extend(h.to_be_bytes());
s.extend(w.to_be_bytes());
#[expect(clippy::cast_possible_truncation, reason = "These fixtures contain at most four three-byte component records.")]
s.push((comps.len() / 3) as u8);
s.extend(comps);
s
};
// Four components, no subsampling: 2 bytes a coefficient, 8 a pixel.
assert_eq!(frame_coefficients(&sof(64, 64, &[1, 0x11, 0, 2, 0x11, 0, 3, 0x11, 0, 4, 0x11, 0])), Some(64 * 64 * 8));
// 4:2:0: MCUs of 16 × 16 holding 4 + 1 + 1 blocks; a 1-pixel-wide picture still takes one.
assert_eq!(frame_coefficients(&sof(1, 16, &[1, 0x22, 0, 2, 0x11, 0, 3, 0x11, 0])), Some(128 * 6));
let mut short = sof(1, 16, &[1, 0x22, 0, 2, 0x11, 0]);
short[5] = 3;
assert_eq!(frame_coefficients(&short), None, "three components declared, two there");
// A marker walk over SOI, an APP0 segment, SOF2, then the scan.
let mut jpeg = vec![0xFF, 0xD8, 0xFF, 0xE0, 0, 4, 0, 0, 0xFF, 0xC2];
let f = sof(64, 64, &[1, 0x11, 0]);
#[expect(clippy::cast_possible_truncation, reason = "This fixture's nine-byte frame header and length field fit u16.")]
jpeg.extend(((f.len() + 2) as u16).to_be_bytes());
jpeg.extend(&f);
jpeg.extend([0xFF, 0xDA]);
assert_eq!(jpeg_coefficients(&jpeg), 64 * 64 * 2);
fn jpeg_charge_is_never_less_than_the_local_one_it_replaced() {
let wide = &[1, 0xFF, 0, 2, 0xFF, 0, 3, 0xFF, 0];
let cmyk = &[1, 0x11, 0, 2, 0x11, 0, 3, 0x11, 0, 4, 0x11, 0];
let zero = &[1, 0x00, 0, 2, 0x00, 0, 3, 0x00, 0];
// (case, whether the shared rule is expected to charge more).
let cases: Vec<(Vec<u8>, bool)> = vec![
(jpeg_headers(&[(0xC0, 8, 640, 480, C420)], Some(3)), false),
(jpeg_headers(&[(0xC0, 8, 640, 480, GREY)], Some(1)), false),
(jpeg_headers(&[(0xC0, 8, 640, 480, C444)], Some(1)), true),
(jpeg_headers(&[(0xC1, 8, 640, 480, C420)], Some(2)), true),
(jpeg_headers(&[(0xC0, 8, 640, 480, C444)], None), false),
(jpeg_headers(&[(0xC2, 8, 640, 480, C420)], Some(1)), false),
(jpeg_headers(&[(0xC2, 8, 640, 480, C444)], Some(3)), false),
(jpeg_headers(&[(0xC2, 12, 640, 480, C444)], Some(1)), false),
(jpeg_headers(&[(0xC2, 8, 640, 480, cmyk)], Some(1)), false),
(jpeg_headers(&[(0xC2, 8, 640, 480, wide)], None), true),
(jpeg_headers(&[(0xC2, 8, 640, 480, zero)], None), false),
(jpeg_headers(&[(0xC6, 8, 640, 480, C444)], None), false),
(jpeg_headers(&[(0xCA, 8, 640, 480, C444)], None), false),
(jpeg_headers(&[(0xCE, 8, 640, 480, C444)], None), false),
(jpeg_headers(&[(0xC0, 8, 64, 64, GREY), (0xC2, 8, 6000, 4000, C444), (0xC2, 8, 100, 100, C444)], Some(1)), false),
(jpeg_headers(&[(0xC2, 8, 65535, 65535, cmyk)], None), false),
(jpeg_headers(&[], None), false),
(vec![0xFF, 0xD8], false),
(vec![], false),
(b"not a jpeg at all".to_vec(), false),
];
let mut grew = 0;
for (b, expect_more) in &cases {
let (old, new) = (oracle_jpeg_coefficients(b), jpeg_worst_case_coefficients(b));
assert!(new >= old, "{b:02x?}: {new} < {old}");
assert_eq!(new > old, *expect_more, "{b:02x?}: {old} -> {new}");
grew += usize::from(new > old);
}
assert_eq!(grew, 3);
// Every prefix of a frame with a cut header, and pseudo-random damage: never less.
let full = jpeg_headers(&[(0xC2, 8, 3000, 2000, C420), (0xC0, 8, 100, 100, C444)], Some(1));
for n in 0..=full.len() {
assert!(jpeg_worst_case_coefficients(&full[..n]) >= oracle_jpeg_coefficients(&full[..n]), "prefix {n}");
}
let mut seed = 0x9e37_79b9u32;
for _ in 0..3000 {
let mut f = full.clone();
for _ in 0..3 {
seed = seed.wrapping_mul(1_664_525).wrapping_add(1_013_904_223);
let (i, v) = ((seed >> 8) as usize % f.len(), (seed >> 24) as u8);
f[i] = v;
}
assert!(jpeg_worst_case_coefficients(&f) >= oracle_jpeg_coefficients(&f), "{f:02x?}");
}
}
#[test]
fn a_baseline_jpeg_with_a_short_first_scan_is_charged_its_coefficients() {
// 3-component 4:4:4 SOF0 of 8000 x 8000 (64 Mpx) whose first scan has one component: the
// decoder holds 2 bytes a coefficient of all three until the other scans, 6 bytes a pixel.
let b = jpeg_headers(&[(0xC0, 8, 8000, 8000, C444)], Some(1));
let (old, new) = (oracle_jpeg_coefficients(&b), jpeg_worst_case_coefficients(&b));
assert_eq!((old, new), (0, 8000 * 8000 * 6));
// 3 + 6 bytes a pixel beside the output passes the 512 MiB budget; it fit as 7 (the RGBA8 conversion).
let (before, after) = (peak_bytes(8000, 8000, 3, old), peak_bytes(8000, 8000, 3, new));
assert_eq!((before, after), (Some(8000 * 8000 * 8), Some(8000 * 8000 * 9)));
assert!(before.is_some_and(|p| p <= crate::MAX_DECODED_BYTES) && after.is_some_and(|p| p > crate::MAX_DECODED_BYTES));
// The same through `probe`, on a real JPEG whose headers are made so.
let mut jpeg = encode(image::RgbImage::from_pixel(16, 16, image::Rgb([200, 30, 30])), ImageFormat::Jpeg);
let sof = jpeg.windows(2).position(|m| m == [0xFF, 0xC0]).unwrap();
jpeg[sof + 5..sof + 7].copy_from_slice(&8000u16.to_be_bytes());
jpeg[sof + 7..sof + 9].copy_from_slice(&8000u16.to_be_bytes());
(0..3).for_each(|c| jpeg[sof + 11 + 3 * c] = 0x11);
let sos = jpeg.windows(2).position(|m| m == [0xFF, 0xDA]).unwrap();
jpeg.splice(sos..sos + 14, [0xFF, 0xDA, 0, 8, 1, 1, 0, 0, 63, 0]);
let p = probe(&jpeg).unwrap();
assert_eq!((p.width, p.height, p.peak_bytes()), (8000, 8000, 8000 * 8000 * 9));
assert!(!p.fits(crate::MAX_DECODED_BYTES));
}
#[test]
fn jpeg_coefficients_follow_a_marker_walk() {
// SOI, an APP0 segment, SOF2, then the scan.
let jpeg = jpeg_headers(&[(0xC2, 8, 64, 64, GREY)], Some(1));
assert_eq!(jpeg_worst_case_coefficients(&jpeg), 64 * 64 * 2);
// 4:2:0: MCUs of 16 x 16 holding 4 + 1 + 1 blocks; a 1-pixel-wide picture still takes one.
assert_eq!(jpeg_worst_case_coefficients(&jpeg_headers(&[(0xC2, 8, 1, 16, C420)], None)), 128 * 6);
}
/// A little-endian TIFF header whose first IFD holds just a compression tag.
@@ -839,4 +842,516 @@ mod tests {
assert!(probe_dib(d).and_then(|p| p.decode(1 << 20)).is_err());
}
}
// The local JPEG budget before it moved to `craft_image_codecs::jpeg_worst_case_coefficients` (#778), kept as the oracle.
/// Bytes of DCT coefficients a progressive JPEG's decoder keeps for the whole picture while it
/// reads the scans: zune-jpeg holds an `i16` per coefficient of every component, in whole MCUs.
/// 0 for a baseline JPEG, decoded an MCU row at a time. Each frame header before the first scan
/// counts and the largest wins; headers this can't follow are charged as the costliest frame.
fn oracle_jpeg_coefficients(bytes: &[u8]) -> u64 {
let mut at = 2;
let mut most = 0u64;
loop {
if bytes.get(at) != Some(&0xFF) {
return oracle_worst_coefficients(bytes).max(most);
}
while bytes.get(at) == Some(&0xFF) {
at += 1;
}
let Some(&code) = bytes.get(at) else { return oracle_worst_coefficients(bytes).max(most) };
at += 1;
match code {
// Markers without a length: start of image, TEM, restarts.
0xD8 | 0x01 | 0xD0..=0xD7 => continue,
// End of image, start of scan: every frame header is behind.
0xD9 | 0xDA => return most,
_ => {}
}
let len = match (bytes.get(at), bytes.get(at + 1)) {
(Some(&hi), Some(&lo)) => usize::from(u16::from_be_bytes([hi, lo])),
_ => 0,
};
let Some(segment) = bytes.get(at + 2..at + len).filter(|_| len >= 2) else { return oracle_worst_coefficients(bytes).max(most) };
// SOF2, SOF6, SOF10 and SOF14 are the progressive frames.
if matches!(code, 0xC2 | 0xC6 | 0xCA | 0xCE) {
let Some(held) = oracle_frame_coefficients(segment) else { return oracle_worst_coefficients(bytes).max(most) };
most = most.max(held);
}
at += len;
}
}
/// `oracle_jpeg_coefficients` of one progressive frame header (after its length): the components'
/// blocks per MCU (horizontal × vertical sampling) times the MCUs that cover the picture.
fn oracle_frame_coefficients(sof: &[u8]) -> Option<u64> {
let height = u64::from(u16::from_be_bytes([*sof.get(1)?, *sof.get(2)?]));
let width = u64::from(u16::from_be_bytes([*sof.get(3)?, *sof.get(4)?]));
let n = usize::from(*sof.get(5)?);
let sampling: Vec<(u64, u64)> = sof
.get(6..6 + 3 * n)?
.as_chunks::<3>()
.0
.iter()
.map(|&[_, s, _]| (u64::from(s >> 4).clamp(1, 4), u64::from(s & 15).clamp(1, 4)))
.collect();
let (h_max, v_max) = sampling.iter().fold((1, 1), |(h, v), &(a, b)| (h.max(a), v.max(b)));
let mcus = width.div_ceil(8 * h_max).checked_mul(height.div_ceil(8 * v_max))?;
sampling.iter().try_fold(0u64, |sum, &(h, v)| sum.checked_add(mcus.checked_mul(2 * 64 * h * v)?))
}
/// The coefficients of the costliest progressive frame of the size the image crate reads from
/// `bytes`: four components, each sampled 4 × 4, in MCUs of 32 pixels a side.
fn oracle_worst_coefficients(bytes: &[u8]) -> u64 {
let Some((w, h)) = dimensions(bytes) else { return 0 };
let mcus = u64::from(w).div_ceil(32) * u64::from(h).div_ceil(32);
mcus.saturating_mul(2 * 64 * 16 * 4)
}
// The local DIB wrapping before it moved to `craft_image_codecs::bmp_probe` (#778), kept as the oracle.
/// Prepend a 14-byte `BITMAPFILEHEADER` to a normalized DIB so `craft-image-codecs` can inspect and decode it as BMP.
fn oracle_dib_to_bmp(dib: &[u8]) -> Vec<u8> {
let u16_at = |at: usize| dib.get(at..at + 2).and_then(|b| b.try_into().ok()).map(u16::from_le_bytes);
let u32_at = |at: usize| dib.get(at..at + 4).and_then(|b| b.try_into().ok()).map(u32::from_le_bytes);
let bi_size = u32_at(0).unwrap_or(40);
let offset = if bi_size == 12 {
let bpp = u16_at(10).unwrap_or(24);
let palette = if bpp <= 8 { (1u32 << bpp) * 3 } else { 0 };
14 + bi_size + palette
} else {
let bpp = u16_at(14).unwrap_or(24);
let compression = u32_at(16).unwrap_or(0);
let clr_used = u32_at(32).unwrap_or(0);
let palette_entries = if clr_used != 0 {
clr_used
} else if bpp <= 8 {
1u32 << bpp
} else {
0
};
let masks = if compression == 3 && bi_size >= 40 { 12 } else { 0 };
14u32.saturating_add(bi_size).saturating_add(masks).saturating_add(palette_entries.saturating_mul(4))
};
let file_size = u32::try_from(14 + dib.len()).unwrap_or(u32::MAX);
let mut bmp = Vec::with_capacity(14 + dib.len());
bmp.extend_from_slice(b"BM");
bmp.extend_from_slice(&file_size.to_le_bytes());
bmp.extend_from_slice(&[0u8; 4]);
bmp.extend_from_slice(&offset.to_le_bytes());
bmp.extend_from_slice(dib);
bmp
}
/// A DIB with a V4 or V5 header in the layout the decoder reads. A 32-bit one whose alpha mask
/// is set but whose compression says `BI_RGB` is marked `BI_BITFIELDS` (with the usual colour
/// masks when it gives none), so its alpha is kept as Windows programs mean it; arboard makes the
/// same change. Bit fields then need the colour masks after the header too, where Windows puts a
/// copy and the decoder skips them; a DIB without that copy (arboard writes one) gets it.
fn oracle_dib_layout(mut dib: Vec<u8>) -> Vec<u8> {
let u32_at = |d: &[u8], at: usize| d.get(at..at + 4).and_then(|b| b.try_into().ok()).map(u32::from_le_bytes);
let Some(size) = u32_at(&dib, 0).filter(|&s| s >= 108).and_then(|s| usize::try_from(s).ok()) else { return dib };
let put = |d: &mut Vec<u8>, at: usize, v: u32| {
if let Some(b) = d.get_mut(at..at + 4) {
b.copy_from_slice(&v.to_le_bytes());
}
};
if dib.get(14..16) == Some(&[32, 0]) && u32_at(&dib, 16) == Some(0) && u32_at(&dib, 52) == Some(0xFF00_0000) {
put(&mut dib, 16, 3);
if (40..52).step_by(4).all(|at| u32_at(&dib, at) == Some(0)) {
put(&mut dib, 40, 0x00FF_0000);
put(&mut dib, 44, 0x0000_FF00);
put(&mut dib, 48, 0x0000_00FF);
}
}
if u32_at(&dib, 16) == Some(3)
&& size <= dib.len()
&& let Some(masks) = dib.get(40..52).map(<[u8]>::to_vec)
&& dib.get(size..size + 12) != Some(masks.as_slice())
{
dib.splice(size..size, masks);
}
dib
}
/// The size and peak `probe_dib` reported before the cutover, as `(width, height, peak, alpha, bytes)`.
fn oracle_probe_dib(dib: Vec<u8>) -> Result<(u32, u32, u64, bool, Vec<u8>), Refusal> {
let dib = oracle_dib_layout(dib);
let bmp = oracle_dib_to_bmp(&dib);
let (width, height, native, alpha) = match inspect_raster(&bmp) {
Ok(Some(info)) => {
let native = u64::from(info.channels.max(1)) * u64::from(info.bits_per_sample.max(8) / 8);
(info.width, info.height, native, matches!(info.color_type, RasterColorType::GrayAlpha | RasterColorType::Rgba))
}
Ok(None) => return Err(Refusal::Broken("its DIB header can't be read".into())),
Err(CodecError::LimitExceeded(_)) => {
let (w, h) = dib_header_dims(&dib).ok_or_else(|| Refusal::Broken("its DIB header can't be read".into()))?;
(w, h, 4, false)
}
Err(e) => return Err(broken(e)),
};
let (width, height, peak) = sized(width, height, native, 0)?;
Ok((width, height, peak, alpha, bmp))
}
/// A DIB with a header of `size` bytes (12, 40, 52, 56, 64, 108 or 124), `bpp`, `compression`
/// and `clr_used`, followed by `tail` bytes.
fn dib_with(size: u32, bpp: u16, compression: u32, clr_used: u32, alpha_mask: u32, tail: &[u8]) -> Vec<u8> {
let mut d = vec![0u8; usize::try_from(size).unwrap()];
d[0..4].copy_from_slice(&size.to_le_bytes());
if size == 12 {
d[4..6].copy_from_slice(&3u16.to_le_bytes());
d[6..8].copy_from_slice(&2u16.to_le_bytes());
d[8..10].copy_from_slice(&1u16.to_le_bytes());
d[10..12].copy_from_slice(&bpp.to_le_bytes());
} else {
d[4..8].copy_from_slice(&3i32.to_le_bytes());
d[8..12].copy_from_slice(&2i32.to_le_bytes());
d[12..14].copy_from_slice(&1u16.to_le_bytes());
d[14..16].copy_from_slice(&bpp.to_le_bytes());
d[16..20].copy_from_slice(&compression.to_le_bytes());
d[32..36].copy_from_slice(&clr_used.to_le_bytes());
if size >= 56 {
d[52..56].copy_from_slice(&alpha_mask.to_le_bytes());
}
if size >= 52 {
for (at, m) in [(40, 0x00FF_0000u32), (44, 0x0000_FF00), (48, 0x0000_00FF)] {
d[at..at + 4].copy_from_slice(&m.to_le_bytes());
}
}
}
d.extend_from_slice(tail);
d
}
fn crafted_dibs() -> Vec<Vec<u8>> {
let px = [[0u8, 0, 255, 128]; 6].concat();
let masks = [0x00u8, 0, 0xFF, 0, 0, 0xFF, 0, 0, 0xFF, 0, 0, 0];
let mut v = Vec::new();
for size in [12u32, 40, 52, 56, 64, 108, 124] {
for bpp in [1u16, 4, 8, 16, 24, 32] {
for compression in [0u32, 1, 2, 3, 6] {
for clr_used in [0u32, 2, 300, u32::MAX] {
for alpha_mask in [0u32, 0xFF00_0000] {
for tail in [&[][..], &px[..], &masks[..], &[&masks[..], &px[..]].concat()[..]] {
v.push(dib_with(size, bpp, compression, clr_used, alpha_mask, tail));
}
}
}
}
}
}
// Clipboard layouts: the V5 helper's three, and a lying header size.
let q = [[0, 0, 255, 128], [255, 0, 0, 255]];
for (bitfields, copy) in [(false, false), (true, false), (true, true)] {
v.push(dib(2, 1, 0xFF00_0000, bitfields, copy, &q));
v.push(dib(2, 1, 0, bitfields, copy, &q));
}
v.push(dib(60_000, 60_000, 0xFF00_0000, false, false, &[]));
let mut lying = dib(2, 1, 0xFF00_0000, true, false, &q);
lying[0..4].copy_from_slice(&u32::MAX.to_le_bytes());
v.push(lying);
v
}
// craft-image-codecs fixed two DIB behaviours after the cutover (craft-libs #2062); the oracle
// keeps the old ones, so these predicates name the inputs where the two must differ.
fn le32(d: &[u8], at: usize) -> Option<u32> {
d.get(at..at + 4).and_then(|b| b.try_into().ok()).map(u32::from_le_bytes)
}
/// A 52- or 56-byte `BI_BITFIELDS` header: its masks are inside the header, so the pixel offset
/// no longer counts 12 appended mask bytes. Returns the new, independently computed offset.
fn v2v3_bitfields_offset(d: &[u8]) -> Option<u32> {
let size = le32(d, 0).unwrap_or(40);
if !matches!(size, 52 | 56) || le32(d, 16).unwrap_or(0) != 3 {
return None;
}
let bpp = d.get(14..16).and_then(|b| b.try_into().ok()).map_or(24, u16::from_le_bytes);
let clr_used = le32(d, 32).unwrap_or(0);
let entries = if clr_used != 0 {
clr_used
} else if bpp <= 8 {
1u32 << bpp
} else {
0
};
Some(14u32.saturating_add(size).saturating_add(entries.saturating_mul(4)))
}
/// A V4/V5 bit-fields DIB whose 12 bytes after the header equal the masks but whose payload is
/// no longer than the pixel rows need: those bytes are pixels, not a copy, so a copy is inserted.
/// Returns the masks the shared code inserts at `size`.
fn exact_payload_equal_to_masks(d: &[u8]) -> Option<(usize, [u8; 12])> {
let size = usize::try_from(le32(d, 0).filter(|&s| s >= 108)?).ok()?;
if size > d.len() {
return None;
}
let promote = d.get(14..16) == Some(&[32, 0][..]) && le32(d, 16) == Some(0) && le32(d, 52) == Some(0xFF00_0000);
let mut masks: [u8; 12] = d.get(40..52)?.try_into().ok()?;
if promote && masks == [0; 12] {
masks = [0x00, 0x00, 0xFF, 0x00, 0x00, 0xFF, 0x00, 0x00, 0xFF, 0x00, 0x00, 0x00];
}
if !(promote || le32(d, 16) == Some(3)) || d.get(size..size + 12) != Some(masks.as_slice()) {
return None;
}
let w = u64::try_from(i32::from_le_bytes(d.get(4..8)?.try_into().ok()?)).ok()?;
let h = u64::from(i32::from_le_bytes(d.get(8..12)?.try_into().ok()?).unsigned_abs());
let bpp = u64::from(u16::from_le_bytes(d.get(14..16)?.try_into().ok()?));
let need = usize::try_from(w.checked_mul(bpp)?.div_ceil(32).checked_mul(4)?.checked_mul(h)?).ok()?;
(d.len() - size <= need).then_some((size, masks))
}
#[test]
fn dib_wrapping_is_byte_equal_to_the_local_one_it_replaced() {
let (mut n, mut v2v3, mut exact) = (0, 0, 0);
for d in crafted_dibs() {
// Every truncation point of the header region, then the whole.
let cuts = (0..=d.len().min(140)).chain([d.len()]);
for cut in cuts {
let d = &d[..cut];
n += 1;
let (new, old) = (dib_to_bmp(d).unwrap(), oracle_dib_to_bmp(d));
if let Some(offset) = v2v3_bitfields_offset(d) {
// New behaviour: only the offset differs, and it no longer counts the 12 mask bytes.
v2v3 += 1;
let mut expected = old;
expected[10..14].copy_from_slice(&offset.to_le_bytes());
assert_eq!(new, expected, "wrap, 52/56-byte bit fields, cut {cut}");
} else {
assert_eq!(new, old, "wrap, cut {cut}");
}
let (new, old) = (bmp_probe::dib_layout(d.to_vec()), oracle_dib_layout(d.to_vec()));
if let Some((size, masks)) = exact_payload_equal_to_masks(d) {
// New behaviour: the bytes after the header are pixels, so a mask copy is inserted.
exact += 1;
let mut expected = old;
expected.splice(size..size, masks);
assert_eq!(new, expected, "layout, exact payload equal to the masks, cut {cut}");
} else {
assert_eq!(new, old, "layout, cut {cut}");
}
}
}
assert!(n > 100_000, "{n}");
// The exclusions are small and present: a predicate that matched nothing would hide a regression.
eprintln!("DIB corpus: {n} cases, {v2v3} excluded as 52/56-byte bit fields, {exact} as exact payload equal to the masks");
assert!(v2v3 > 0 && exact > 0 && v2v3 + exact < n / 10, "{v2v3} {exact}");
}
#[test]
fn probe_dib_is_unchanged_by_the_shared_wrapping() {
for d in crafted_dibs() {
for cut in [0, 4, 11, 12, 14, 40, 52, 100, 124, 130, d.len()] {
let d = d[..cut.min(d.len())].to_vec();
// The two DIB fixes change the bytes handed to the decoder; the wrapping test above pins
// them, so skip those inputs here.
if v2v3_bitfields_offset(&d).is_some() || exact_payload_equal_to_masks(&d).is_some() {
continue;
}
let new = probe_dib(d.clone());
match (new, oracle_probe_dib(d.clone())) {
(Ok(p), Ok((w, h, peak, alpha, bmp))) => {
assert_eq!((p.width, p.height, p.peak, p.alpha), (w, h, peak, alpha), "{d:02x?}");
assert_eq!(&*p.bytes, bmp.as_slice());
}
(Err(a), Err(b)) => assert_eq!(a, b, "{d:02x?}"),
(a, b) => panic!("{d:02x?}: {:?} vs {:?}", a.map(|p| p.width), b.map(|o| o.0)),
}
}
}
}
// The local PNG header read before it moved to `craft_image_codecs::png_probe` (#778), kept as the oracle.
/// A PNG's width, height, the bytes a pixel its decoder gives (samples under 8 bits widened to 8,
/// a palette to RGB, `tRNS` to an alpha channel) and whether it has transparency, from `IHDR` and
/// the chunk headers before the image data. The decoder's own header read inflates an ICC profile
/// (`iCCP`) up to its 512 MiB default limit, for a picture that may only be measured.
fn oracle_png_chunks(bytes: &[u8]) -> Option<(u32, u32, u64, bool)> {
let be = |at: usize| bytes.get(at..at.checked_add(4)?).and_then(|b| <[u8; 4]>::try_from(b).ok()).map(u32::from_be_bytes);
// The 8-byte signature, then IHDR: length, type, width, height, bit depth, colour type.
if bytes.get(12..16) != Some(b"IHDR".as_slice()) {
return None;
}
let (width, height) = (be(16)?, be(20)?);
let (depth, color) = (*bytes.get(24)?, *bytes.get(25)?);
let mut trns = false;
let mut at = 8usize;
loop {
let len = usize::try_from(be(at)?).ok()?;
match bytes.get(at.checked_add(4)?..at.checked_add(8)?)? {
b"IDAT" => break,
b"IEND" => return None,
b"tRNS" => trns = true,
_ => {}
}
at = at.checked_add(12)?.checked_add(len)?;
}
let (channels, alpha) = match color {
0 => (1 + u64::from(trns), trns),
2 | 3 => (3 + u64::from(trns), trns),
4 => (2, true),
6 => (4, true),
_ => return None,
};
Some((width, height, channels * if depth == 16 { 2 } else { 1 }, alpha))
}
/// WordCraft's WebP alpha rule before the cutover: the `VP8X` flag, nothing else.
fn oracle_webp_alpha(bytes: &[u8]) -> bool {
bytes.get(12..16) == Some(b"VP8X") && bytes.get(20).is_some_and(|b| b & 0x10 != 0)
}
fn new_png(b: &[u8]) -> Option<(u32, u32, u64, bool)> {
png_probe::pixel_model(b).ok().map(|m| (m.width, m.height, u64::from(m.bytes_per_pixel()), m.has_alpha()))
}
/// A PNG with the given IHDR fields and chunks before a (junk) `IDAT`.
fn png_with(width: u32, height: u32, depth: u8, color: u8, before_idat: &[(&[u8; 4], Vec<u8>)]) -> Vec<u8> {
let chunk = |out: &mut Vec<u8>, kind: &[u8; 4], data: &[u8]| {
out.extend((data.len() as u32).to_be_bytes());
out.extend_from_slice(kind);
out.extend_from_slice(data);
out.extend_from_slice(&[0; 4]);
};
let mut b = b"\x89PNG\r\n\x1a\n".to_vec();
let mut ihdr = [width.to_be_bytes(), height.to_be_bytes()].concat();
ihdr.extend_from_slice(&[depth, color, 0, 0, 0]);
chunk(&mut b, b"IHDR", &ihdr);
for (k, d) in before_idat {
chunk(&mut b, k, d);
}
chunk(&mut b, b"IDAT", &[1, 2, 3]);
chunk(&mut b, b"IEND", &[]);
b
}
#[test]
fn png_pixel_model_matches_the_local_header_read_for_every_valid_header() {
let valid: [(u8, &[u8]); 5] = [(0, &[1, 2, 4, 8, 16]), (2, &[8, 16]), (3, &[1, 2, 4, 8]), (4, &[8, 16]), (6, &[8, 16])];
let mut n = 0;
for (color, depths) in valid {
for &depth in depths {
for (trns, icc) in [(false, false), (true, false), (true, true), (false, true)] {
let mut extra: Vec<(&[u8; 4], Vec<u8>)> = vec![];
if icc {
extra.push((b"iCCP", vec![0xAB; 300]));
}
if trns {
extra.push((b"tRNS", vec![0, 0]));
}
let b = png_with(5, 7, depth, color, &extra);
assert!(new_png(&b).is_some(), "colour {color} depth {depth}");
assert_eq!(new_png(&b), oracle_png_chunks(&b), "colour {color} depth {depth} trns {trns}");
// And `probe` reports what the old read gave it: the size and the peak.
let (w, h, native, alpha) = oracle_png_chunks(&b).unwrap();
let p = probe(&b).unwrap();
assert_eq!((p.width, p.height, p.has_alpha(), Some(p.peak_bytes())), (w, h, alpha, peak_bytes(w, h, native, 0)));
assert_eq!(dimensions(&b), Some((5, 7)));
n += 1;
}
}
}
assert_eq!(n, 4 * 15);
// Real encoder output, every layout the image types give.
for img in [
DynamicImage::ImageLuma8(image::GrayImage::new(3, 2)),
DynamicImage::ImageLumaA8(image::GrayAlphaImage::new(3, 2)),
DynamicImage::ImageRgb8(image::RgbImage::new(3, 2)),
DynamicImage::ImageRgba8(image::RgbaImage::new(3, 2)),
DynamicImage::ImageRgb16(image::ImageBuffer::new(3, 2)),
DynamicImage::ImageRgba16(image::ImageBuffer::new(3, 2)),
] {
let b = encode(img, ImageFormat::Png);
assert_eq!(new_png(&b), oracle_png_chunks(&b));
}
}
/// What the old read accepted and `pixel_model` refuses is a header the decoder can't use
/// either; `probe` then says the PNG header can't be read (`Broken`) and `dimensions` is None.
#[test]
fn png_pixel_model_is_stricter_only_on_headers_that_are_not_valid_pngs() {
let ok = png_with(5, 7, 8, 2, &[]);
let mut cases: Vec<(&str, Vec<u8>)> = vec![
("depth 3 for RGB", png_with(5, 7, 3, 2, &[])),
("depth 16 for palette", png_with(5, 7, 16, 3, &[])),
("depth 4 for RGBA", png_with(5, 7, 4, 6, &[])),
("zero width", png_with(0, 7, 8, 2, &[])),
("zero height", png_with(5, 0, 8, 2, &[])),
("width past i32::MAX", png_with(1 << 31, 1, 8, 2, &[])),
("height past i32::MAX", png_with(1, u32::MAX, 8, 2, &[])),
];
let many: Vec<(&[u8; 4], Vec<u8>)> = (0..70_000).map(|_| (b"tEXt", Vec::new())).collect();
cases.push(("70,000 chunks before the image data", png_with(5, 7, 8, 2, &many)));
for (what, b) in &cases {
assert!(oracle_png_chunks(b).is_some(), "{what}: the old read accepted it");
assert_eq!(new_png(b), None, "{what}");
assert_eq!(dimensions(b), None, "{what}");
assert!(matches!(probe(b), Err(Refusal::Broken(_))), "{what}");
assert!(!header_has_alpha(b), "{what}");
}
// Everything else the old read refused (no IHDR, unknown colour, a wrong IHDR length that
// throws its chunk walk off, no image data) is still refused.
let mut short_ihdr = ok.clone();
short_ihdr[8..12].copy_from_slice(&12u32.to_be_bytes());
for b in [png_with(5, 7, 8, 5, &[]), b"\x89PNG\r\n\x1a\n".to_vec(), ok[..33].to_vec(), short_ihdr] {
assert_eq!(new_png(&b), None);
assert_eq!(oracle_png_chunks(&b), None);
}
// Truncations and damage: where the new read answers, it answers as the old one did.
let full = png_with(5, 7, 8, 6, &[(b"tRNS", vec![0]), (b"gAMA", vec![0; 4])]);
for n in 0..=full.len() {
if new_png(&full[..n]).is_some() {
assert_eq!(new_png(&full[..n]), oracle_png_chunks(&full[..n]), "prefix {n}");
}
}
let mut seed = 0x9e37_79b9u32;
for _ in 0..3000 {
let mut f = full.clone();
for _ in 0..3 {
seed = seed.wrapping_mul(1_664_525).wrapping_add(1_013_904_223);
let (i, v) = ((seed >> 8) as usize % f.len(), (seed >> 24) as u8);
f[i] = v;
}
if new_png(&f).is_some() {
assert_eq!(new_png(&f), oracle_png_chunks(&f), "{f:02x?}");
}
}
}
/// A RIFF WebP with one chunk.
fn riff_webp(kind: &[u8; 4], payload: &[u8]) -> Vec<u8> {
let mut b = b"RIFF".to_vec();
b.extend((4 + 8 + payload.len() as u32).to_le_bytes());
b.extend_from_slice(b"WEBP");
b.extend_from_slice(kind);
b.extend((payload.len() as u32).to_le_bytes());
b.extend_from_slice(payload);
b
}
#[test]
fn webp_alpha_follows_vp8x_and_now_vp8l() {
// VP8X: the flag, as before.
for flags in [0u8, 0x10, 0x20, 0x30] {
let mut payload = vec![flags, 0, 0, 0];
payload.extend_from_slice(&[0, 0, 0, 0, 0, 0]); // 1 x 1 canvas
let b = riff_webp(b"VP8X", &payload);
assert_eq!(webp_probe::has_alpha(&b).unwrap(), oracle_webp_alpha(&b), "flags {flags:#x}");
assert_eq!(oracle_webp_alpha(&b), flags & 0x10 != 0);
}
// A bare lossless picture (VP8L) says so in its header: 1 x 1, `alpha_is_used` (bit 28).
let with = riff_webp(b"VP8L", &[0x2f, 0, 0, 0, 0x10, 0, 0, 0]);
let without = riff_webp(b"VP8L", &[0x2f, 0, 0, 0, 0x00, 0, 0, 0]);
assert!(!oracle_webp_alpha(&with), "the old rule never saw it");
assert!(webp_probe::has_alpha(&with).unwrap());
assert!(!webp_probe::has_alpha(&without).unwrap());
// A real encoder's lossless RGBA WebP.
let enc = encode(image::RgbaImage::from_pixel(4, 4, image::Rgba([9, 8, 7, 100])), ImageFormat::WebP);
assert_eq!(probe(&enc).unwrap().has_alpha(), webp_probe::has_alpha(&enc).unwrap());
// Lossy VP8 never has alpha; and input that isn't a RIFF WebP reads as none, as before.
assert!(!webp_probe::has_alpha(&riff_webp(b"VP8 ", &[0; 10])).unwrap());
for junk in [&b"not a webp at all"[..], b"", b"RIFF", b"RIFF\0\0\0\0WEBPVP8L"] {
assert!(!webp_probe::has_alpha(junk).unwrap_or(false));
}
}
}
+43 -20
View File
@@ -7,7 +7,7 @@ use super::placed::Placed;
use super::table_flow::below;
use super::{
Anchor, Arc, Block, Blocks, Border, Document, Float, FloatAlign, HashMap, ParaLayout, Paragraph, Path, Rect, StoryRef, TableFloat, TextDirection,
Wrap, WrapText, para, table,
Wrap, WrapText, display, para, table,
};
/// Cell identity is independent of optional table-style formatting.
@@ -325,6 +325,7 @@ pub(crate) fn layout_box(ctx: &mut Ctx, input: BoxInput<'_>) -> BoxLayout {
if below {
edges.bottom = 0.0;
}
edges.joined = [above, below];
let lead = pl.lines.first().map_or(0.0, |l| l.top.max(0.0)) + edges.top;
if above {
edges.up = (y + lead - prev_end).max(0.0);
@@ -638,19 +639,26 @@ fn border_room(b: Option<Border>) -> f32 {
/// How a piece of a paragraph meets the paragraphs around it: the room taken by its top and
/// bottom borders (0: none drawn there — the paragraph goes on from or into one of its border
/// group, or it is split there) and how far its shading and side borders reach up to join the
/// paragraph above in its group.
/// paragraph above in its group, and whether it goes on from the paragraph above and into the one
/// below in that group (its side lines don't run on past those ends).
#[derive(Clone, Copy, Default)]
pub(crate) struct Edges {
pub(crate) top: f32,
pub(crate) bottom: f32,
pub(crate) up: f32,
pub(crate) joined: [bool; 2],
}
impl Edges {
/// A paragraph on its own: its top and bottom borders at its ends.
pub(crate) fn own(pl: &ParaLayout, l0: usize, l1: usize) -> Edges {
let b = pl.rp.borders.unwrap_or_default();
Edges { top: if l0 == 0 { border_room(b.top) } else { 0.0 }, bottom: if l1 >= pl.lines.len() { border_room(b.bottom) } else { 0.0 }, up: 0.0 }
Edges {
top: if l0 == 0 { border_room(b.top) } else { 0.0 },
bottom: if l1 >= pl.lines.len() { border_room(b.bottom) } else { 0.0 },
up: 0.0,
joined: [false; 2],
}
}
}
@@ -685,23 +693,38 @@ pub(crate) fn push_para(
}
items.push(Placed::Lines { story, path: Path(path.to_vec()), para: pl.clone(), l0, l1, x, y, turn: TextDirection::Horizontal });
if let Some(b) = &pl.rp.borders {
// All four lines lie beyond their distance from the text; top and bottom reach the corners.
if let Some(t) = b.top.filter(|t| e.top > 0.0 && t.is_visible()) {
let ty = y - t.space - t.width / 2.0;
items.push(Placed::Rule { x0: lx, y0: ty, x1: rx, y1: ty, border: t });
}
if let Some(t) = b.bottom.filter(|t| e.bottom > 0.0 && t.is_visible()) {
let by = y + h + t.space + t.width / 2.0;
items.push(Placed::Rule { x0: lx, y0: by, x1: rx, y1: by, border: t });
}
if let Some(t) = b.left.filter(Border::is_visible) {
let bx = lx + t.width / 2.0;
items.push(Placed::Rule { x0: bx, y0: top, x1: bx, y1: bottom, border: t });
}
if let Some(t) = b.right.filter(Border::is_visible) {
let bx = rx - t.width / 2.0;
items.push(Placed::Rule { x0: bx, y0: top, x1: bx, y1: bottom, border: t });
}
// All four lines lie beyond their distance from the text (`lx`, `rx`: the side lines'
// outer edges); `box_rules` runs the top and bottom over the side lines to those edges.
let half = |o: Option<Border>| o.filter(Border::is_visible).map_or(0.0, |t| t.width / 2.0);
let (cl, cr) = (lx + half(b.left), rx - half(b.right));
let tb = b.top.filter(|t| e.top > 0.0 && t.is_visible());
let bb = b.bottom.filter(|t| e.bottom > 0.0 && t.is_visible());
let ty = tb.map_or(top, |t| y - t.space - t.width / 2.0);
let by = bb.map_or(bottom, |t| y + h + t.space + t.width / 2.0);
box_rules(items, (cl, ty, cr, by), [tb, bb, b.left, b.right], e.joined);
}
y + h
}
/// The visible lines of a border box with corners `(x0, y0)` and `(x1, y1)` (`[top, bottom, left,
/// right]`), ending square as Word draws them: a top or bottom line runs on over half the side
/// line it meets, or half its own width where there is none, so single-line boxes close at the
/// corners; a side line runs on to the outer edge of the top or bottom line it meets, or by half
/// its own width where there is none, unless the box goes on there (`joined`: at the top, at the
/// bottom).
pub(crate) fn box_rules(items: &mut Vec<Placed>, (x0, y0, x1, y1): (f32, f32, f32, f32), sides: [Option<Border>; 4], joined: [bool; 2]) {
let [top, bottom, left, right] = sides.map(|b| b.filter(Border::is_visible));
let half = |t: &Border| display::rule_width(t) / 2.0;
for (t, y) in [(top, y0), (bottom, y1)] {
if let Some(t) = t {
let over = |side: Option<Border>| half(&side.unwrap_or(t));
items.push(Placed::Rule { x0: x0 - over(left), y0: y, x1: x1 + over(right), y1: y, border: t });
}
}
for (t, x) in [(left, x0), (right, x1)] {
if let Some(t) = t {
let end = |o: Option<Border>, joined: bool| if joined { 0.0 } else { half(&o.unwrap_or(t)) };
items.push(Placed::Rule { x0: x, y0: y0 - end(top, joined[0]), x1: x, y1: y1 + end(bottom, joined[1]), border: t });
}
}
}
+7
View File
@@ -107,6 +107,9 @@ pub(crate) fn env_hash(doc: &Document, opts: &LayoutOptions) -> u64 {
outline.hash(&mut h);
opts.filename.hash(&mut h);
opts.filepath.hash(&mut h);
// Output shows page references the editor leaves stored (a missing bookmark's error too).
opts.output_page_refs.hash(&mut h);
opts.bookmark_error.hash(&mut h);
h.finish()
}
@@ -187,3 +190,7 @@ impl Totals {
/// The most extra layout passes for page totals in the body: a total that changes the page
/// count (its digits pushing text to another page) is tried again, this many times at most.
pub(crate) const MAX_TOTAL_PASSES: usize = 3;
/// The most extra layout passes for output's body `PAGEREF`s: a page number whose digits move a
/// bookmark to another page is tried again, this many times at most.
pub(crate) const MAX_PAGE_REF_PASSES: usize = 3;
+1
View File
@@ -29,6 +29,7 @@ pub use lines::{mark_text_width, section_break_label};
pub(crate) use objects::inline_rect;
pub use objects::seg_path;
pub use options::{BalloonMode, ChangeBar, CommentDisplay, DeleteMark, DisplayOptions, FormattingMarks, InsertMark, MarkupOptions};
pub(crate) use page::rule_width;
pub use page::{CHANGE_BAR, ItemDisplay, MOVE_COLOR, object_display, page_display, page_layers, revision_color};
pub use paint::{Paint, Stroke, paint_for};
+34 -10
View File
@@ -326,16 +326,8 @@ fn draw_glyph_runs(ctx: &LineContext<'_, '_, '_>, out: &mut Vec<Draw>) {
let uy = base - st.shift + st.size * 0.12;
let ucolor = rc.underline_color.unwrap_or(color);
if underline == Underline::Words {
for kk in run_start..end_k {
let Some(c) = pl.clusters.get(kk) else { continue };
if c.kind != ClKind::Text {
continue;
}
let a = x + line.cl_left(kk).unwrap_or(0.0);
let b = x + line.cl_right(kk).unwrap_or(a - x);
if a < x0 - 0.01 || b > x1 + 0.01 {
continue; // drawn with its own span
}
let words = word_spans(pl, line, (run_start, end_k), x, (x0, x1));
for (a, b) in words {
out.push(Draw::Line { x0: a, y0: uy, x1: b, y1: uy, width: thick, color: ucolor, stroke: Stroke::Solid, alpha, text_bg });
}
} else {
@@ -403,6 +395,38 @@ fn emphasis_mark(mark: wordcraft_doc::props::EmphasisMark, cx: f32, base: f32, s
}
}
/// The words among clusters `k0..k1` that lie inside `(x0, x1)`, as their left and right edges:
/// a Words underline draws one line per word, since flat-ended lines per letter would show seams
/// where they meet.
fn word_spans(pl: &ParaLayout, line: &crate::para::Line, (k0, k1): (usize, usize), x: f32, (x0, x1): (f32, f32)) -> Vec<(f32, f32)> {
let mut words: Vec<(f32, f32)> = Vec::new();
// A space ends a word even when condensed spacing leaves it no width.
let mut apart = true;
for k in k0..k1 {
let Some(c) = pl.clusters.get(k) else { continue };
// A bookmark or comment anchor inside a word takes no room and doesn't part it.
if c.kind == ClKind::Marker {
continue;
}
if c.kind != ClKind::Text {
apart = true;
continue;
}
let a = x + line.cl_left(k).unwrap_or(0.0);
let b = x + line.cl_right(k).unwrap_or(a - x);
if a < x0 - 0.01 || b > x1 + 0.01 {
apart = true;
continue; // drawn with its own span
}
match words.last_mut() {
Some(w) if !apart && a <= w.1 + 0.01 && b >= w.0 - 0.01 => *w = (w.0.min(a), w.1.max(b)),
_ => words.push((a, b)),
}
apart = false;
}
words
}
fn comment_backgrounds(ctx: &LineContext<'_, '_, '_>, out: &mut Vec<Draw>) {
let Some(para) = ctx.para.filter(|_| ctx.opts.markup && ctx.opts.revisions.comments) else { return };
let mut open: Option<usize> = None;
+10 -13
View File
@@ -422,27 +422,24 @@ fn border_stroke(s: BorderStyle) -> Stroke {
}
}
/// The width a border line is drawn at.
pub(crate) fn rule_width(b: &Border) -> f32 {
if b.style == BorderStyle::Thick { b.width.max(1.5) } else { b.width.max(0.25) }
}
/// A border line from `(x0, y0)` to `(x1, y1)` (paragraph rules and character borders).
fn rule(x0: f32, y0: f32, x1: f32, y1: f32, b: &Border, alpha: f32) -> Draw {
Draw::Line {
x0,
y0,
x1,
y1,
width: if b.style == BorderStyle::Thick { b.width.max(1.5) } else { b.width.max(0.25) },
color: b.color.unwrap_or(Rgb::BLACK),
stroke: border_stroke(b.style),
alpha,
text_bg: None,
}
Draw::Line { x0, y0, x1, y1, width: rule_width(b), color: b.color.unwrap_or(Rgb::BLACK), stroke: border_stroke(b.style), alpha, text_bg: None }
}
/// Closed box around one line segment of a character-border group, widened by `space`.
// Note: border drawn outside text advance, reserve width in line breaking if overlap shows.
pub(super) fn char_box(b: &Border, x0: f32, x1: f32, top: f32, bottom: f32, alpha: f32, out: &mut Vec<Draw>) {
let (l, r) = (x0 - b.space, x1 + b.space);
out.push(rule(l, top, r, top, b, alpha));
out.push(rule(l, bottom, r, bottom, b, alpha));
// The top and bottom lines run on over half the side lines, closing the corners.
let half = rule_width(b) / 2.0;
out.push(rule(l - half, top, r + half, top, b, alpha));
out.push(rule(l - half, bottom, r + half, bottom, b, alpha));
out.push(rule(l, top, l, bottom, b, alpha));
out.push(rule(r, top, r, bottom, b, alpha));
}
+25 -1
View File
@@ -1,6 +1,6 @@
//! Field results computed at layout time (page numbers) and note numbering.
use std::collections::HashMap;
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use wordcraft_doc::section::NumFormat;
@@ -31,6 +31,17 @@ pub struct FieldCtx {
pub bookmark_pages: Arc<BookmarkPages>,
/// A key for caches standing for `bookmark_pages` (0 when it's empty).
pub bookmarks_key: u64,
/// Output ([`crate::LayoutOptions::output_page_refs`]) with `bookmark_pages` found for the
/// whole document: a `PAGEREF` to a bookmark that is neither there nor in `bookmarks` shows
/// `bookmark_error` instead of its result. Off while the pages aren't known yet (the first
/// pass shows the stored results, the closest guess).
pub output: bool,
/// The names of all the document's bookmarks, for output (see `output`): one in a story that
/// `bookmark_pages` doesn't place (a header, a comment) keeps the field's result.
pub bookmarks: Arc<HashSet<String>>,
/// What output shows for a `PAGEREF` to a missing bookmark ([`BOOKMARK_NOT_DEFINED`] when
/// empty).
pub bookmark_error: Arc<str>,
/// Where a paragraph runs onto later pages: from text offset `.0` on, its fields are on page
/// `.1` (ascending offsets). Empty except during the layout of a body paragraph split across
/// pages, for its later pages (#1352); nothing else laid out meanwhile sees it.
@@ -295,6 +306,15 @@ pub fn field_text_on(instr: &str, result: &str, ctx: &FieldCtx, page: u32) -> (S
"SECTION" => (field_number(instr, fmt.unwrap_or(NumFormat::Decimal), ctx.section), true),
"PAGEREF" => match pageref_page(instr, ctx) {
Some((n, page_format)) => (field_number(instr, fmt.unwrap_or(page_format), n), true),
// Printed or exported, a reference to a bookmark the document lacks shows Word's
// error (measured in Word for Windows).
None if ctx.output
&& !has_switch(instr, 'p')
&& pageref_target(instr).is_some_and(|t| !ctx.bookmark_pages.contains_key(t) && !ctx.bookmarks.contains(t)) =>
{
let text = if ctx.bookmark_error.is_empty() { BOOKMARK_NOT_DEFINED } else { &ctx.bookmark_error };
(text.to_string(), true)
}
None => (result.to_string(), false),
},
"TITLE" if result.is_empty() => (apply_case_switch(instr, ctx.title.to_string()), false),
@@ -319,6 +339,10 @@ pub fn field_text_on(instr: &str, result: &str, ctx: &FieldCtx, page: u32) -> (S
}
}
/// What Word prints for a `PAGEREF` to a bookmark the document doesn't have, in English (Word
/// prints it in its interface language: see [`crate::LayoutOptions::bookmark_error`]).
pub const BOOKMARK_NOT_DEFINED: &str = "Error! Bookmark not defined.";
/// Extract the bookmark target from a `PAGEREF` instruction.
fn pageref_target(instr: &str) -> Option<&str> {
let mut parts = instr.split_whitespace();
+9 -3
View File
@@ -116,8 +116,9 @@ fn with_page_map<T>(ctx: &mut Ctx, map: &mut Vec<(usize, u32)>, f: impl FnOnce(&
/// drawn by its first and last paragraph.
#[derive(Clone, Copy)]
struct Rooms {
/// Joined to the paragraph above.
/// Joined to the paragraph above, and to the one below.
above: bool,
below: bool,
top: f32,
bottom: f32,
}
@@ -127,7 +128,7 @@ impl Rooms {
let above = joined(ctx.doc, &ctx.doc.body, block, block.checked_sub(1), None);
let below = joined(ctx.doc, &ctx.doc.body, block, Some(block + 1), None);
let own = Edges::own(pl, 0, pl.lines.len());
Rooms { above, top: if above { 0.0 } else { own.top }, bottom: if below { 0.0 } else { own.bottom } }
Rooms { above, below, top: if above { 0.0 } else { own.top }, bottom: if below { 0.0 } else { own.bottom } }
}
}
@@ -528,7 +529,12 @@ impl<'a> Flow<'a> {
pb.bottom = pb.body_floor().max(placed_bottom).min(pb.orig_bottom);
}
let mut items = Vec::new();
let mut edges = Edges { top: if l0 == 0 { rooms.top } else { 0.0 }, bottom: if l1 >= n { rooms.bottom } else { 0.0 }, up: 0.0 };
let mut edges = Edges {
top: if l0 == 0 { rooms.top } else { 0.0 },
bottom: if l1 >= n { rooms.bottom } else { 0.0 },
up: 0.0,
joined: [l0 == 0 && rooms.above, l1 >= n && rooms.below],
};
if l0 == 0
&& rooms.above
&& let Some((page, col, end)) = pb.para_end
+121 -20
View File
@@ -236,23 +236,83 @@ pub(crate) fn headers_footers_refer_to_pages(doc: &Document) -> bool {
})
}
/// Where each bookmark in the body starts: the number and number format of the page it is placed
/// on (with two of the same name, the first in the document).
/// Whether a story laid out with the body (the body itself, text boxes, footnotes and endnotes)
/// shows an unlocked `PAGEREF` field, which output resolves from the paginated document
/// ([`crate::LayoutOptions::output_page_refs`]); a locked one keeps its result, so it needs no
/// extra pass. Headers and footers are filled in after pagination and need none either.
pub(crate) fn flow_refers_to_pages(doc: &Document) -> bool {
use wordcraft_doc::PartKind;
let refers = |story: StoryRef| {
doc.para_paths(story).iter().filter_map(|path| doc.para(story, path)).any(|p| {
p.objects.iter().any(|o| matches!(o, InlineObject::Field { instr, locked, .. } if !*locked && fields::field_name(instr) == "PAGEREF"))
})
};
refers(StoryRef::Body)
|| doc
.parts
.iter()
.filter(|(_, part)| matches!(part.kind, PartKind::TextBox | PartKind::Footnote | PartKind::Endnote))
.any(|(id, _)| refers(StoryRef::Part(*id)))
}
/// The names of all the document's bookmarks, in any story: a `PAGEREF` to one of them that
/// [`bookmark_pages`] can't place keeps its result rather than showing Word's error.
pub(crate) fn bookmark_names(doc: &Document) -> HashSet<String> {
let mut names = HashSet::new();
let stories = std::iter::once(StoryRef::Body).chain(doc.parts.keys().map(|id| StoryRef::Part(*id)));
for story in stories {
for path in doc.para_paths(story) {
let Some(p) = doc.para(story, &path) else { continue };
for o in &p.objects {
if let InlineObject::BookmarkStart { name } = o {
names.insert(name.clone());
}
}
}
}
names
}
/// Where each bookmark in the body, a footnote or an endnote, or in a text box anchored in one of
/// them, starts: the number and number format of the page it is placed on (with two of the same
/// name, the first found). A bookmark in a text box is on the page of the box's anchor (measured
/// in Word for Windows); one in a note, on the page that shows that part of the note.
pub(crate) fn bookmark_pages(doc: &Document, pages: &[Page], sections: &[(usize, &SectionProps)]) -> fields::BookmarkPages {
use wordcraft_doc::PartKind;
// Each paragraph's bookmarks: name, offset, and its line in the paragraph layout last looked
// up (the pieces of a paragraph usually share one).
let mut wanted: HashMap<Path, Vec<(String, usize, Option<(usize, usize)>)>> = HashMap::new();
let mut wanted: Wanted = HashMap::new();
let mut seen = HashSet::new();
for path in doc.para_paths(StoryRef::Body) {
let Some(p) = doc.para(StoryRef::Body, &path) else { continue };
if !p.objects.iter().any(|o| matches!(o, InlineObject::BookmarkStart { .. })) {
continue;
let mut stories = vec![StoryRef::Body];
let mut notes: Vec<u32> =
doc.parts.iter().filter(|(_, part)| matches!(part.kind, PartKind::Footnote | PartKind::Endnote)).map(|(id, _)| *id).collect();
notes.sort_unstable();
stories.extend(notes.into_iter().map(StoryRef::Part));
for &story in &stories {
for path in doc.para_paths(story) {
let Some(p) = doc.para(story, &path) else { continue };
if !p.objects.iter().any(|o| matches!(o, InlineObject::BookmarkStart { .. })) {
continue;
}
for (off, o) in p.object_offsets().into_iter().zip(&p.objects) {
if let InlineObject::BookmarkStart { name } = o
&& seen.insert(name.as_str())
{
wanted.entry(story).or_default().entry(path.clone()).or_default().push((name.clone(), off, None));
}
}
}
for (off, o) in p.object_offsets().into_iter().zip(&p.objects) {
if let InlineObject::BookmarkStart { name } = o
&& seen.insert(name.as_str())
{
wanted.entry(path.clone()).or_default().push((name.clone(), off, None));
}
if doc.parts.values().any(|part| part.kind == PartKind::TextBox) {
let mut boxes = HashSet::new();
for &story in &stories {
for path in doc.para_paths(story) {
let Some(p) = doc.para(story, &path) else { continue };
for (off, o) in p.object_offsets().into_iter().zip(&p.objects) {
for id in o.text_boxes() {
box_bookmarks(doc, id, (story, &path, off), &mut boxes, &mut wordcraft_doc::BoxBudget::default(), &mut seen, &mut wanted);
}
}
}
}
}
@@ -263,8 +323,9 @@ pub(crate) fn bookmark_pages(doc: &Document, pages: &[Page], sections: &[(usize,
}
let format = sections.get(page.section).map(|(_, s)| s.page_num_format).unwrap_or_default();
for it in &page.items {
let Placed::Lines { story: StoryRef::Body, path, para, l0, l1, .. } = it else { continue };
let Some(marks) = wanted.get_mut(path) else { continue };
let Placed::Lines { story, path, para, l0, l1, .. } = it else { continue };
let Some(paras) = wanted.get_mut(story) else { continue };
let Some(marks) = paras.get_mut(path) else { continue };
let layout = Arc::as_ptr(para) as usize;
marks.retain_mut(|(name, off, line)| {
let li = match *line {
@@ -282,21 +343,61 @@ pub(crate) fn bookmark_pages(doc: &Document, pages: &[Page], sections: &[(usize,
false
});
if marks.is_empty() {
wanted.remove(path);
paras.remove(path);
if paras.is_empty() {
wanted.remove(story);
}
}
}
}
out
}
/// A key for caches standing for `marks` (0 for none).
pub(crate) fn marks_key(marks: &fields::BookmarkPages) -> u64 {
if marks.is_empty() {
return 0;
/// The bookmarks [`bookmark_pages`] looks for, by story and paragraph: name, offset, and the line
/// in the paragraph layout last looked up.
type Wanted = HashMap<StoryRef, HashMap<Path, Vec<(String, usize, Option<(usize, usize)>)>>>;
/// Add the bookmarks of text box `id`, and of the boxes inside it, to `wanted` at `anchor` (the
/// story, paragraph and offset of the outermost box). A box shown twice, or inside itself, counts
/// once, and the boxes inside one go as deep as layout shows them ([`wordcraft_doc::BoxBudget`]): a
/// bookmark in a box nested deeper has no page.
fn box_bookmarks<'d>(
doc: &'d Document,
id: u32,
anchor: (StoryRef, &Path, usize),
boxes: &mut HashSet<u32>,
budget: &mut wordcraft_doc::BoxBudget,
seen: &mut HashSet<&'d str>,
wanted: &mut Wanted,
) {
if boxes.contains(&id) || !budget.enter(id) {
return;
}
boxes.insert(id);
let story = StoryRef::Part(id);
for path in doc.para_paths(story) {
let Some(p) = doc.para(story, &path) else { continue };
for o in &p.objects {
if let InlineObject::BookmarkStart { name } = o
&& seen.insert(name.as_str())
{
wanted.entry(anchor.0).or_default().entry(anchor.1.clone()).or_default().push((name.clone(), anchor.2, None));
}
for inner in o.text_boxes() {
box_bookmarks(doc, inner, anchor, boxes, budget, seen, wanted);
}
}
}
budget.leave();
}
/// A key for caches standing for `marks`: 0 while the pages aren't known (`None`), something else
/// once they are, even with no bookmarks (output then shows missing ones as errors).
pub(crate) fn marks_key(marks: Option<&fields::BookmarkPages>) -> u64 {
let Some(marks) = marks else { return 0 };
let mut v: Vec<_> = marks.iter().collect();
v.sort_by(|a, b| a.0.cmp(b.0));
hash_of(&v)
hash_of(&v).max(1)
}
pub(crate) fn headers_footers(ctx: &mut Ctx, pages: &mut [Page], sections: &[(usize, &SectionProps)], totals: &Totals) {
+25 -1
View File
@@ -7,7 +7,7 @@ use wordcraft_doc::props::TextDirection;
use wordcraft_doc::{Document, Path, Pos, StoryRef};
use wordcraft_geom::{Point, Rect};
use crate::para::{PARA_MARK_WIDTH, ParaLayout};
use crate::para::{PARA_MARK_WIDTH, ParaLayout, StyleRun};
use crate::{DocLayout, Page, Placed};
/// Caret geometry on a page: a bar from (`x`, `top`) to (`x + width`, `top + height`). It is
@@ -727,6 +727,30 @@ impl DocLayout {
None
}
/// The style run text typed at `pos` would take (the character's before it, or at the start
/// of a paragraph the first character's) and the baseline it sits on, on the page: `pos`'s
/// line's, raised or lowered by the run's shift (superscript, `w:position`). An input method's
/// composition is drawn with them, as Word draws it in the text's own font and size. `None` in
/// turned text, whose lines aren't horizontal.
pub fn typing_style(&self, pos: &Pos, page_hint: usize) -> Option<(StyleRun, f32)> {
for (_, it) in self.pieces(pos.story, &pos.path, page_hint) {
let Placed::Lines { para, l0, l1, y, turn, .. } = it else { continue };
let li = para.line_of(pos.off);
if li < *l0 || li >= *l1 {
continue;
}
if *turn != TextDirection::Horizontal {
return None;
}
let first = para.lines.get(*l0)?;
let line = para.lines.get(li)?;
let at = |off: usize| para.clusters.iter().find(|c| c.start <= off && off < c.end).and_then(|c| para.styles.get(c.style as usize));
let run = pos.off.checked_sub(1).and_then(at).or_else(|| at(pos.off)).or_else(|| para.styles.first())?;
return Some((run.clone(), y + line.baseline - first.top - run.shift));
}
None
}
/// The position one line up (`dir < 0`) or down from `pos`, keeping `goal_x` (page x).
pub fn vertical(&self, pos: &Pos, goal_x: Option<f32>, dir: i32, page_hint: usize) -> Option<(Pos, f32)> {
let c = self.caret_on(pos, page_hint)?;
+29 -2
View File
@@ -73,7 +73,7 @@ mod table_flow;
use boxes::{BOX_INSET_X, BOX_INSET_Y};
use boxes::{BoxInput, BoxLayout, Edges, fit_box, layout_box, push_para};
pub use cache::LayoutCache;
use cache::{MAX_TOTAL_PASSES, Totals};
use cache::{MAX_PAGE_REF_PASSES, MAX_TOTAL_PASSES, Totals};
#[cfg(test)]
use ctx::picture_outline;
use ctx::{Ctx, Pass};
@@ -234,6 +234,14 @@ pub struct LayoutOptions {
pub filename: Option<Arc<str>>,
/// Document full path for FILENAME \p fields ([`FieldCtx::filepath`]).
pub filepath: Option<Arc<str>>,
/// Output (PDF, print, page images): `PAGEREF` fields in the body show the page their
/// bookmark is on, as Word fills them in when it prints or exports. The editor leaves this
/// off: there a body `PAGEREF` keeps its result until fields are updated, as in Word.
pub output_page_refs: bool,
/// What output shows for a `PAGEREF` to a bookmark the document doesn't have: Word prints
/// "Error! Bookmark not defined." in its interface language. `None` is the English text,
/// [`fields::BOOKMARK_NOT_DEFINED`].
pub bookmark_error: Option<Arc<str>>,
}
/// The word being typed ([`LayoutOptions::typing_word`]): the paragraph by its revision
@@ -344,8 +352,15 @@ pub fn layout(doc: &Document, cache: &mut LayoutCache, opts: &LayoutOptions) ->
let mut totals = Totals::first_guess(doc.sections().len());
let mut passes = 0;
let (mut early, mut rewraps) = (EarlyFloats::new(), 0);
// Output's `PAGEREF`s in the body, text boxes and notes ([`LayoutOptions::output_page_refs`])
// show the pages the pass before found for their bookmarks, until those hold (bounded, like
// the totals). The first pass, with no pages found yet, shows their stored results.
let page_refs = opts.output_page_refs && headers::flow_refers_to_pages(doc);
let mut marks: Option<Arc<fields::BookmarkPages>> = None;
let mut mark_passes = 0;
let mut l = loop {
match layout_pass(doc, cache, opts, &note_pages, &totals, passes < MAX_TOTAL_PASSES, (&early, rewraps < MAX_REWRAP_PASSES)) {
match layout_pass(doc, cache, opts, (&note_pages, marks.as_ref()), &totals, passes < MAX_TOTAL_PASSES, (&early, rewraps < MAX_REWRAP_PASSES))
{
Pass::Again(t) => {
totals = t;
passes += 1;
@@ -364,6 +379,18 @@ pub fn layout(doc: &Document, cache: &mut LayoutCache, opts: &LayoutOptions) ->
continue;
}
}
if page_refs && mark_passes < MAX_PAGE_REF_PASSES {
let found = headers::bookmark_pages(doc, &l.pages, &doc.sections());
if marks.as_deref() != Some(&found) {
marks = Some(Arc::new(found));
mark_passes += 1;
// The page numbers shown can move text to other pages: the page totals and
// footnotes numbered per page get their passes to settle again.
passes = 0;
note_passes = 1;
continue;
}
}
break l;
}
}
+2 -6
View File
@@ -1,4 +1,4 @@
use super::boxes::{BoxInput, layout_box};
use super::boxes::{BoxInput, box_rules, layout_box};
use super::ctx::{Ctx, with_pin};
use super::floats::{EarlyFloats, WrapArea};
use super::notes::{NOTE_SEP, NOTE_SEP_W, NoteBox, NoteSeps, v_span};
@@ -112,11 +112,7 @@ pub(crate) fn page_border_rules(s: &SectionProps) -> Vec<Placed> {
if let Some(b) = &s.page_borders {
let sp = |e: &Option<Border>| e.map(|x| x.space).unwrap_or(24.0);
let (x0, x1, y0, y1) = (sp(&b.left), s.page_w - sp(&b.right), sp(&b.top), s.page_h - sp(&b.bottom));
for (e, a, bb) in [(b.top, (x0, y0), (x1, y0)), (b.bottom, (x0, y1), (x1, y1)), (b.left, (x0, y0), (x0, y1)), (b.right, (x1, y0), (x1, y1))] {
if let Some(e) = e.filter(Border::is_visible) {
v.push(Placed::Rule { x0: a.0, y0: a.1, x1: bb.0, y1: bb.1, border: e });
}
}
box_rules(&mut v, (x0, y0, x1, y1), [b.top, b.bottom, b.left, b.right], [false; 2]);
}
v
}
+20 -13
View File
@@ -150,6 +150,7 @@ impl<'a> Builder<'a> {
};
let st = StyleRun {
face,
requested_face: r.face,
size,
synth_bold,
synth_italic,
@@ -214,7 +215,9 @@ impl<'a> Builder<'a> {
if let Some(f) = missing.latin.filter(|_| asks_latin) {
self.note_substituted(f, drawn);
}
if let Some(f) = missing.east_asian.filter(|_| !in_ea_font && sub.chars().any(east_asian)) {
// A missing East Asian font has no face of its own: CJK text drawn in the East Asian
// face is in its font table alternate (#2702) and still asks for it.
if let Some(f) = missing.east_asian.filter(|_| sub.chars().any(east_asian)) {
self.note_substituted(f, drawn);
}
}
@@ -248,25 +251,29 @@ impl<'a> Builder<'a> {
let rc_cs: Option<Arc<ResolvedChar>> = text.chars().any(|c| rc.uses_complex(c)).then(|| Arc::new(rc.complex()));
let family = lookup_family(self.env.doc, &rc.font);
let resolved = wordcraft_fonts::word::resolve(family, rc.bold, rc.italic);
// The requested font is missing when it was substituted, or when another family stands in
// for it: its font table alternate (#2702), which may itself be installed.
let requested_missing = resolved.substituted || !family.eq_ignore_ascii_case(&rc.font);
let primary = resolved.face;
// Symbol-font characters are laid out as their Unicode equivalents.
let sym = SymbolRun::of(&rc.font, &resolved);
let show = |c: char| sym.map_or(c, |m| m.show(c));
// A font isn't installed when it was substituted, or looked up as a missing roman font.
// A font isn't installed when it was substituted, or looked up as a missing roman font or
// its font table alternate (#2702).
let resolved_cs = rc_cs.as_ref().map(|r| {
let family = lookup_family(self.env.doc, &r.font);
let resolved = wordcraft_fonts::word::resolve(family, r.bold, r.italic);
(resolved.face, resolved.substituted || family != r.font)
(resolved.face, resolved.substituted || !family.eq_ignore_ascii_case(&r.font))
});
let primary_cs = resolved_cs.map(|(face, _)| face);
// Chinese, Japanese and Korean characters use the run's East Asian font (`w:eastAsia`) when
// it is installed and has them.
let wanted_ea = rc
.font_ea
.as_deref()
.filter(|f| !f.eq_ignore_ascii_case(&rc.font) && text.chars().any(east_asian))
.map(|f| (f, wordcraft_fonts::word::resolve(f, rc.bold, rc.italic)));
let primary_ea = wanted_ea.filter(|(_, r)| !r.substituted).map(|(_, r)| r.face);
// it, or the font table alternate it is drawn in (#2702), is installed and has them.
let wanted_ea = rc.font_ea.as_deref().filter(|f| !f.eq_ignore_ascii_case(&rc.font) && text.chars().any(east_asian)).map(|f| {
let family = lookup_family(self.env.doc, f);
let resolved = wordcraft_fonts::word::resolve(family, rc.bold, rc.italic);
(f, resolved, resolved.substituted || !family.eq_ignore_ascii_case(f))
});
let primary_ea = wanted_ea.filter(|(_, r, _)| !r.substituted).map(|(_, r, _)| r.face);
// The levels by byte of `text`, from `lbase` on.
let (levels, lbase): (&[u8], usize) = match dir {
Dir::Para => (&self.levels, base),
@@ -278,7 +285,7 @@ impl<'a> Builder<'a> {
// A missing Thai font: one face for the run's Thai and Latin text (#1549). A run with its
// own complex-script font keeps that font for the Thai letters.
let run_face: Option<FaceRef> = if primary_cs.is_none_or(|f| f.id() == primary.id()) {
thai_run_face(&rc.font, resolved.substituted, primary, text, rc.bold, rc.italic)
thai_run_face(&rc.font, requested_missing, primary, text, rc.bold, rc.italic)
} else {
None
};
@@ -314,9 +321,9 @@ impl<'a> Builder<'a> {
context: &context,
});
let missing = MissingFonts {
latin: (resolved.substituted || family != rc.font).then_some(rc.font.as_str()),
latin: requested_missing.then_some(rc.font.as_str()),
complex: rc_cs.as_ref().filter(|_| resolved_cs.is_some_and(|(_, missing)| missing)).map(|r| r.font.as_str()),
east_asian: wanted_ea.filter(|(_, r)| r.substituted).map(|(f, _)| f),
east_asian: wanted_ea.filter(|(_, _, missing)| *missing).map(|(f, _, _)| f),
names_east_asian: wanted_ea.is_some(),
};
self.note_missing_fonts(text, &segs, (primary, primary_cs, primary_ea), &missing);
+46 -16
View File
@@ -124,23 +124,52 @@ pub(crate) fn remap_range(map_starts: &[usize], orig_of: &[usize], orig_len: usi
/// as roman (measured in Word; Calibri otherwise, which our own substitutes already match).
pub(super) const MISSING_ROMAN_FONT: &str = "Cambria";
/// The family to look `font` up as: [`MISSING_ROMAN_FONT`] for a missing roman font, else `font`.
/// A missing Thai font keeps its name: it is drawn in a Thai face, not in Cambria (#1549).
/// A font with substitutes of its own keeps them: Times New Roman is roman in every font table,
/// and where it isn't installed Liberation Serif has its metrics; Word never meets it missing.
pub(super) fn lookup_family<'f>(doc: &Document, font: &'f str) -> &'f str {
/// The family to look `font` up as. A font that isn't installed is drawn, as Word draws it
/// (measured in Word 2024, #2702):
/// 1. in the alternate the font table names for it (`w:altName`), when that one can be drawn:
/// installed, or a font one of whose own substitutes is installed. That comes before the
/// family class: a roman font with an installed alternate is drawn in the alternate;
/// 2. in [`MISSING_ROMAN_FONT`] when the font table classes it as roman, or as modern with an
/// alternate that can't be drawn either (a modern font with no alternate, and a swiss one with
/// or without, take the generic substitutes, as Calibri does in Word). Not for a Thai font
/// (drawn in a Thai face, #1549), a symbol font, or a font with substitutes of its own (Times
/// New Roman is roman in every font table, and where it isn't installed Liberation Serif has
/// its metrics; Word never meets it missing);
/// 3. else as `font`, which [`wordcraft_fonts::word::resolve`] substitutes.
pub(super) fn lookup_family<'f>(doc: &'f Document, font: &'f str) -> &'f str {
let settings = &doc.settings;
// A fallback list (`Liberation Serif;Times New Roman`) is in a class when one of its
// candidates is.
let listed = |list: &[String]| {
!list.is_empty() && font.split(';').map(str::trim).filter(|f| !f.is_empty()).any(|f| list.iter().any(|r| r.eq_ignore_ascii_case(f)))
};
let alternate = settings.font_alt_names.iter().find(|(f, _)| f.eq_ignore_ascii_case(font)).map(|(_, alt)| alt.as_str());
let roman = listed(&settings.roman_fonts);
if alternate.is_none() && !roman {
return font;
}
let db = wordcraft_fonts::FontDb::global();
let roman = !doc.settings.roman_fonts.is_empty()
&& font.split(';').map(str::trim).filter(|f| !f.is_empty()).any(|f| doc.settings.roman_fonts.iter().any(|r| r.eq_ignore_ascii_case(f)));
if roman
let installed = |f: &str| db.has_family(f) || db.has_family(wordcraft_fonts::word::charset_base_family(f));
// A fallback list is there when one of its candidates is.
let missing = if font.contains(';') { wordcraft_fonts::word::resolve(font, false, false).substituted } else { !installed(font) };
if !missing {
return font;
}
// Calibri has no list of its own because the generic sans list is its (Carlito first).
let drawable =
|f: &&str| !f.is_empty() && (installed(f) || wordcraft_fonts::word::has_installed_substitute(f) || f.eq_ignore_ascii_case("Calibri"));
// ECMA-376 calls the alternate a comma-separated list, but Word reads the whole value as one
// name (MS-OE376 2.1.264), so a value with a comma names a font that isn't there.
if let Some(alt) = alternate.map(str::trim).filter(drawable) {
return alt;
}
let modern_alternate_missing = alternate.is_some() && listed(&settings.modern_fonts);
// Thai, symbol and fonts with substitutes of their own keep their name: those substitutes
// draw them, Cambria wouldn't.
if (roman || modern_alternate_missing)
&& !wordcraft_fonts::word::is_thai_family(font)
&& !wordcraft_doc::symbol_fonts::is_symbol_font(font)
&& !wordcraft_fonts::word::has_named_substitutes(font)
&& if font.contains(';') {
wordcraft_fonts::word::resolve(font, false, false).substituted
} else {
!db.has_family(font) && !db.has_family(wordcraft_fonts::word::charset_base_family(font))
}
{
MISSING_ROMAN_FONT
} else {
@@ -150,9 +179,10 @@ pub(super) fn lookup_family<'f>(doc: &Document, font: &'f str) -> &'f str {
/// One face for a run in a Thai font that isn't installed (#1549): Word substitutes one font
/// (Cordia New) for the whole run, so the Latin letters, digits and ฿ keep the Thai letters' face
/// and size. `None` unless `font` is a Thai family, it was substituted, and its substitute
/// `primary` lacks some of the run's Thai or Latin characters; then the first face covering all of
/// them, if one does. Other families keep Latin text in their own substitute, as Word does.
/// and size. `None` unless `font` is a Thai family, it was substituted (or drawn in its font
/// table alternate, #2702), and its stand-in `primary` lacks some of the run's Thai or Latin
/// characters; then the first face covering all of them, if one does. Other families keep Latin
/// text in their own substitute, as Word does.
pub(crate) fn thai_run_face(font: &str, substituted: bool, primary: FaceRef, text: &str, bold: bool, italic: bool) -> Option<FaceRef> {
if !substituted || !wordcraft_fonts::word::is_thai_family(font) {
return None;
+3
View File
@@ -67,6 +67,9 @@ pub(super) fn undrawable_bullet(text: &str, lvl: &Level, rc: &ResolvedChar) -> b
#[derive(Clone, Debug)]
pub struct StyleRun {
pub face: FaceRef,
/// The face the run's font resolves to, which `face` replaces where it lacks a character (a
/// fallback face, the East Asian font for CJK): the face new text typed in the run asks for.
pub requested_face: FaceRef,
pub size: f32,
pub synth_bold: bool,
pub synth_italic: bool,
+20 -6
View File
@@ -411,7 +411,7 @@ impl<'a> Shaped<'a> {
/// A field: a legacy form field, its code in braces, or its result.
fn field(&mut self, cx: &Cx, (k, o): (usize, &InlineObject), slot: Slot, rc: &Arc<ResolvedChar>) {
let InlineObject::Field { instr, result, code, .. } = o else { return };
let InlineObject::Field { instr, result, code, locked, .. } = o else { return };
let env = cx.env;
let form = if env.field_codes != *code { None } else { o.legacy_form() };
if let Some(wordcraft_doc::legacy_form::LegacyForm::CheckBox { size, .. }) = form {
@@ -426,6 +426,9 @@ impl<'a> Shaped<'a> {
shaded.shading = Some(FIELD_SHADING);
}
self.b.shape_atomic(&field_code_text(instr), slot.start, slot.end, &Arc::new(shaded));
} else if *locked && crate::fields::field_name(instr) == "PAGEREF" {
// A locked page reference keeps its result, as Word prints and exports it.
self.b.shape_atomic(result, slot.start, slot.end, &linked(rc, instr));
} else {
self.field_result(cx, (instr, result), slot, rc);
}
@@ -812,13 +815,24 @@ fn break_text<'a>(source: &'a str, skip: &[std::ops::Range<usize>]) -> (std::bor
}
}
/// A character that continues a word or a path segment after a slash: a letter, a digit or `_`.
fn word_char(c: char) -> bool {
c.is_alphanumeric() || c == '_'
}
fn break_positions(text: &str, rp: &wordcraft_doc::resolve::ResolvedPara) -> Vec<usize> {
let mut found = Vec::new();
for (i, o) in unicode_linebreak::linebreaks(text) {
// Word keeps "and/or" and web addresses whole: no break right after a slash (a word
// too long for the line still breaks anywhere).
let after_slash =
text.get(..i).is_some_and(|t| t.ends_with('/')) && text.get(i..).and_then(|t| t.chars().next()).is_some_and(char::is_alphanumeric);
// Word keeps "and/or" and web addresses whole: no break right after a slash before a letter,
// digit or underscore ("/_back"), nor after the question mark of a query ("/?type="), which
// takes an ASCII one (a question mark in CJK text keeps its break). A word too long for
// the line still breaks anywhere.
let next = text.get(i..).and_then(|t| t.chars().next());
let after_slash = match text.get(..i).and_then(|t| t.chars().next_back()) {
Some('/') => next.is_some_and(word_char),
Some('?') => next.is_some_and(|c| c.is_ascii_alphanumeric() || c == '_'),
_ => false,
};
if (o == unicode_linebreak::BreakOpportunity::Allowed || i < text.len()) && !after_slash {
found.push(i);
}
@@ -830,7 +844,7 @@ fn break_positions(text: &str, rp: &wordcraft_doc::resolve::ResolvedPara) -> Vec
.match_indices('/')
.filter(|&(i, _)| {
text.get(..i).and_then(|t| t.chars().next_back()).is_some_and(|c| c.is_whitespace() && !matches!(c, '\u{a0}' | '\u{2007}' | '\u{202f}'))
&& text.get(i + 1..).and_then(|t| t.chars().next()).is_some_and(char::is_alphanumeric)
&& text.get(i + 1..).and_then(|t| t.chars().next()).is_some_and(word_char)
})
.map(|(i, _)| i)
.collect();
+21 -9
View File
@@ -2,9 +2,12 @@ use std::collections::VecDeque;
use super::cache::{LayoutCache, Totals, env_hash, hash_of};
use super::ctx::{Ctx, Pass, with_pct_base};
use super::fields::{BOOKMARK_NOT_DEFINED, BookmarkPages};
use super::floats::{EarlyFloats, WrapArea, early_covered, left_out_objects};
use super::flow::{OutlinePass, place_outline_para, place_para};
use super::headers::{HfLinks, bookmark_pages, headers_footers, headers_footers_refer_to_pages, marks_key, mirror_even_pages, section_side};
use super::headers::{
HfLinks, bookmark_names, bookmark_pages, headers_footers, headers_footers_refer_to_pages, marks_key, mirror_even_pages, section_side,
};
use super::notes::{MAX_NOTE_PAGES, NoteSeps, note_info, place_endnotes};
use super::page::{PageBuilder, body_top_for, foot_top_for, item_bottom};
use super::placed::Placed;
@@ -60,12 +63,13 @@ fn reset_cache(doc: &Document, cache: &mut LayoutCache, opts: &LayoutOptions) {
cache.used.clear();
}
/// The walk state at the start of a pass, with the note references the body shows.
/// The walk state at the start of a pass, with the note references the body shows and the pages
/// of the bookmarks its `PAGEREF`s show (`None` until output has found them).
fn new_ctx<'a>(
doc: &'a Document,
cache: &'a mut LayoutCache,
opts: &'a LayoutOptions,
(note_pages, totals): (&HashMap<u32, usize>, &Totals),
(note_pages, marks, totals): (&HashMap<u32, usize>, Option<&Arc<BookmarkPages>>, &Totals),
) -> Ctx<'a> {
let notes = note_info(doc, opts.hides_deleted(StoryRef::Body), note_pages);
let notes_hash = hash_of(&{
@@ -89,6 +93,11 @@ fn new_ctx<'a>(
author: doc.core.creator.as_str().into(),
filename: opts.filename.clone().unwrap_or_else(|| "".into()),
filepath: opts.filepath.clone().unwrap_or_else(|| "".into()),
bookmark_pages: marks.cloned().unwrap_or_default(),
bookmarks_key: marks_key(marks.map(|m| &**m)),
output: opts.output_page_refs && marks.is_some(),
bookmarks: if opts.output_page_refs { Arc::new(bookmark_names(doc)) } else { Arc::default() },
bookmark_error: opts.bookmark_error.clone().unwrap_or_else(|| BOOKMARK_NOT_DEFINED.into()),
..Default::default()
},
notes_hash,
@@ -482,8 +491,10 @@ fn fill_headers_footers(
ctx.track_totals = false;
if headers_footers_refer_to_pages(ctx.doc) {
let marks = bookmark_pages(ctx.doc, pages, sections);
ctx.fields.bookmarks_key = marks_key(&marks);
ctx.fields.bookmarks_key = marks_key(Some(&marks));
ctx.fields.bookmark_pages = Arc::new(marks);
// The pages are known now: output shows a reference to a missing bookmark as Word does.
ctx.fields.output = ctx.opts.output_page_refs;
}
headers_footers(ctx, pages, sections, &found);
if ctx.doc.settings.mirror_margins {
@@ -515,20 +526,21 @@ fn index_lines(pages: &[Page]) -> HashMap<(StoryRef, Path), Vec<(usize, usize)>>
}
/// One layout pass, numbering footnotes restarted on each page by `note_pages` ([`note_info`]),
/// with `totals` assumed for the page totals in the body. With `may_repeat`, a pass whose body
/// shows totals that turn out different asks to be done again. Pages start with the wrap areas
/// of the `early` floats; with `may_rewrap`, a pass that finds others asks to be done again.
/// with `totals` assumed for the page totals in the body and `marks` for the pages its `PAGEREF`s
/// show. With `may_repeat`, a pass whose body shows totals that turn out different asks to be
/// done again. Pages start with the wrap areas of the `early` floats; with `may_rewrap`, a pass
/// that finds others asks to be done again.
pub(crate) fn layout_pass(
doc: &Document,
cache: &mut LayoutCache,
opts: &LayoutOptions,
note_pages: &HashMap<u32, usize>,
(note_pages, marks): (&HashMap<u32, usize>, Option<&Arc<BookmarkPages>>),
totals: &Totals,
may_repeat: bool,
(early, may_rewrap): (&EarlyFloats, bool),
) -> Pass {
reset_cache(doc, cache, opts);
let mut ctx = new_ctx(doc, cache, opts, (note_pages, totals));
let mut ctx = new_ctx(doc, cache, opts, (note_pages, marks, totals));
let hf = Arc::clone(&ctx.hf);
let sections = &hf.sections;
let mode = Mode::of(opts.view);
+144 -3
View File
@@ -378,7 +378,9 @@ struct TableFrame {
x: f32,
}
// Nested measurement needs the same preferred weights as layout before sharing span minima.
/// The table's column widths before autofit: its grid, or equal shares of `avail` without one,
/// then the cells' absolute and percentage widths. Nested measurement needs the same preferred
/// weights as layout before sharing span minima.
fn preferred_grid(t: &Table, avail: f32) -> Vec<f32> {
let ncols = grid_cols(t).max(1);
let mut grid: Vec<f32> = if t.grid.len() == ncols {
@@ -395,6 +397,7 @@ fn preferred_grid(t: &Table, avail: f32) -> Vec<f32> {
vec![avail / ncols as f32; ncols]
};
apply_cell_widths(t, &mut grid);
apply_pct_widths(t, &mut grid, avail);
grid
}
@@ -723,6 +726,9 @@ struct TableRows<'a> {
total: f32,
}
/// Place the laid-out cells of every row: vertical alignment, turned text, borders and the cell
/// rectangles, clipped where the row's exact height or a fixed table's column is too small for
/// the cell's text.
fn finish_table_rows(ctx: &mut Ctx, input: TableRows<'_>) -> TableLayout {
let TableRows { story, table: t, path, depth, cell_text, rows, heights, bottoms, edges, x, total } = input;
// Grid columns holding a vertical-merge continuation, per row.
@@ -835,7 +841,7 @@ fn finish_table_rows(ctx: &mut Ctx, input: TableRows<'_>) -> TableLayout {
let (x0, x1, y0, y1) = (c.x, c.x + c.w, 0.0, rh);
// Horizontal edges run on over half of each visible vertical edge, so the corners
// close the way Word draws them instead of leaving a notch at thick borders.
let half = |o: Option<Border>| o.filter(Border::is_visible).map_or(0.0, |e| e.width / 2.0);
let half = |o: Option<Border>| o.filter(Border::is_visible).map_or(0.0, |e| crate::display::rule_width(&e) / 2.0);
let (hx0, hx1) = (x0 - half(b.left), x1 + half(b.right));
let top_vis = c.vmerge != VMerge::Continue;
if top_vis && let Some(e) = b.top.filter(Border::is_visible) {
@@ -858,7 +864,9 @@ fn finish_table_rows(ctx: &mut Ctx, input: TableRows<'_>) -> TableLayout {
row: ri,
cell: ci,
story,
clip: exact && c.h > span_h + 0.01,
// A fixed table's cell too narrow for any text (its lines are laid out 4 pt wide)
// clips them, as Word does, rather than letting them run past its edges.
clip: (exact && c.h > span_h + 0.01) || (t.props.fixed && c.w - c.margins[1] - c.margins[3] < 4.0),
});
let top = c.margins[0] + c.borders.top.filter(Border::is_visible).map_or(0.0, |b| b.width.clamp(0.0, 12.0));
parts.push(CellPart { top, items: first..items.len(), end, align });
@@ -1535,6 +1543,121 @@ fn apply_cell_widths(t: &Table, grid: &mut [f32]) {
grid.copy_from_slice(&widths);
}
/// Word sizes each column by the percentage its cells ask for, whichever rows ask: cells without a
/// preferred width take no part, so a header row of percentages sizes the automatic rows below it.
/// The percentages are shares of the table's width, scaled up when they come short of 100%; past
/// 100% the columns take their shares left to right until the width runs out
/// (`overfull_shares`). A table without a
/// width of its own takes `avail` when autofit and its grid's width when fixed. Where rows ask
/// different percentages of a column it takes the largest. A merged cell's percentage is shared
/// by the columns under it that ask none of their own, in their grid proportions (Word shares it
/// by their content), and makes no column narrower than it asks. Measured in Word. Columns no cell
/// sizes and absolute widths keep the grid.
fn apply_pct_widths(t: &Table, grid: &mut [f32], avail: f32) {
let mut pcts: Vec<Option<f32>> = vec![None; grid.len()];
let mut merged: Vec<(usize, usize, f32)> = Vec::new();
for row in &t.rows {
let mut g = row.first_grid();
for cell in &row.cells {
if cell.props.width.is_some() {
return;
}
if let Some(p) = cell.props.width_pct {
if !p.is_finite() || p <= 0.0 {
return;
}
if cell.span() == 1 {
let Some(slot) = pcts.get_mut(g) else { return };
*slot = Some(slot.map_or(p, |q| q.max(p)));
} else {
merged.push((g, cell.span(), p));
}
}
g = g.saturating_add(cell.span());
}
}
for (g, span, p) in merged {
let Some(cols) = pcts.get_mut(g..g.saturating_add(span)) else { return };
let Some(weights) = grid.get(g..g.saturating_add(span)) else { return };
let own: f32 = cols.iter().flatten().sum();
let free: f32 = cols.iter().zip(weights).filter(|(c, _)| c.is_none()).map(|(_, w)| *w).sum();
let rest = (p - own).max(0.0);
if free > 0.0 {
for (c, w) in cols.iter_mut().zip(weights) {
if c.is_none() {
*c = Some(rest * w / free);
}
}
} else if own > 0.0 && rest > 0.0 {
// Every column asks its own, short of the merged cell's: they grow to it together.
cols.iter_mut().flatten().for_each(|c| *c *= p / own);
}
}
let Some(pcts) = pcts.into_iter().collect::<Option<Vec<f32>>>() else { return };
if pcts.iter().any(|p| *p <= 0.0) {
return;
}
let sum: f32 = pcts.iter().sum();
let own_width = t.props.width_pct.is_some_and(|p| p > 0.0) || t.props.width.is_some_and(|w| w > 0.0);
// A table with its own width is scaled to it afterwards; any positive total will do. A fixed
// table without one keeps its grid's width.
let total = if own_width || t.props.fixed { grid.iter().sum() } else { avail };
if !sum.is_finite() || sum <= 0.0 || !total.is_finite() || total <= 0.0 {
return;
}
if sum <= 100.0 {
let shares: Vec<f32> = pcts.iter().map(|p| total * (p / sum)).collect();
// A grid Word wrote from these same percentages is within its rounding of them (whole
// twips, and percentages in fiftieths): it is kept exactly. Rebuilt, a column could come
// out a few hundredths of a point narrower and wrap its text differently (#6575).
let rounding = 0.05 + total * 0.0002;
if grid.iter().zip(&shares).all(|(g, s)| (g - s).abs() <= rounding) {
return;
}
grid.copy_from_slice(&shares);
return;
}
// A grid too narrow for the floor in every column shares its width out instead, so the table
// keeps it.
let floor = if t.props.fixed { FIXED_PCT_MIN_COLUMN.min(total / pcts.len() as f32) } else { 0.0 };
grid.copy_from_slice(&overfull_shares(&pcts, total, floor));
}
/// Over 100%, as Word lays it out: the columns take their shares left to right until the table's
/// width runs out; the column it runs out in takes what is left, and those after it nothing. In an
/// autofit table autofit then widens them to their longest words at the others' expense. In a
/// fixed table they keep `floor` each, their text clipped, and the shares are of what is left
/// once those are set aside.
fn overfull_shares(pcts: &[f32], total: f32, floor: f32) -> Vec<f32> {
let mut starved = 0;
loop {
let room = (total - floor * starved as f32).max(0.0);
let mut used = 0.0;
let mut short = 0;
let widths: Vec<f32> = pcts
.iter()
.map(|p| {
let w = (room * p / 100.0).min(room - used);
if w < floor || w <= 0.0 {
short += 1;
return floor;
}
used += w;
w
})
.collect();
// More columns starved than were set aside for: set aside for them and share again.
if short <= starved || starved >= pcts.len() {
return widths;
}
starved = short;
}
}
/// The narrowest a fixed table's columns get when the percentages before them fill the table
/// (measured in Word: 2 pt, with or without a table width).
const FIXED_PCT_MIN_COLUMN: f32 = 2.0;
/// Word's autofit for a table whose widths may follow its content (`tblLayout` not fixed).
/// - With no preferred width anywhere (the table's and every cell's automatic) the columns come
/// from their content alone: each its widest line if they all fit in `avail`, else shared out
@@ -1594,6 +1717,24 @@ mod width_tests {
assert_eq!(grid, [28.0, 111.0, 40.0, 81.0]);
}
/// #5074 review (and #6575): a grid Word wrote from the cells' percentages is kept exactly,
/// while one that is off by more than Word's rounding is rebuilt from them.
#[test]
fn a_grid_matching_its_percentages_is_kept() {
let mut t = Table::new(1, 3, 300.0);
t.props.width = Some(300.0);
for (cell, p) in t.rows[0].cells.iter_mut().zip([33.32, 33.34, 33.34]) {
cell.props.width = None;
cell.props.width_pct = Some(p);
}
let mut grid = [99.95, 100.05, 100.0];
apply_pct_widths(&t, &mut grid, 468.0);
assert_eq!(grid, [99.95, 100.05, 100.0], "kept");
let mut stale = [90.0, 110.0, 100.0];
apply_pct_widths(&t, &mut stale, 468.0);
assert!(stale.iter().zip([99.96, 100.02, 100.02]).all(|(g, w)| (g - w).abs() < 0.01), "rebuilt: {stale:?}");
}
#[test]
fn autofit_exhausts_donors() {
crate::test_fonts();
+506 -3
View File
@@ -923,6 +923,7 @@ fn a_long_unbreakable_word_lays_out_in_linear_time() {
/// A word a megabyte long lays out in a fraction of a second: it used to take half a minute.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn a_megabyte_long_word_lays_out_quickly() {
let d = long_word_beside_a_float(1_000_000);
let t = std::time::Instant::now();
@@ -1143,6 +1144,7 @@ fn a_word_with_alternating_advances_lays_out_in_linear_time() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn hidden_separators_with_automatic_hyphenation_lay_out_in_linear_time() {
crate::test_fonts();
// "hyphenation " n times with every space hidden: one joined word of 11n letters reaches the
@@ -1175,6 +1177,7 @@ fn page_break_paragraphs(n: usize) -> Document {
/// the section list by walking the whole body, so a document of n paragraphs paid n body-length
/// scans: a two-megabyte file took half a minute before the first page was drawn.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn a_document_of_many_paragraphs_lays_out_in_linear_time() {
let (t1, tk, ratio) = layout_time_ratio(page_break_paragraphs, 2_500, &LayoutOptions::default());
eprintln!("t(n) {t1:.4} s, t(8n) {tk:.4} s, ratio {ratio:.1}");
@@ -1182,6 +1185,7 @@ fn a_document_of_many_paragraphs_lays_out_in_linear_time() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn many_hidden_float_anchors_lay_out_in_linear_time() {
crate::test_fonts();
// n square-wrapped shapes, each anchored in its own hidden run (bold alternating, so runs do not
@@ -1425,6 +1429,93 @@ fn a_missing_roman_font_is_drawn_as_word_draws_it() {
assert_ne!(face(false), cambria);
}
#[test]
fn a_missing_font_is_drawn_in_its_font_table_alternate() {
crate::test_fonts();
// Measured in Word 2024 and Word for Windows 16.0.20527 (#2702): a font that isn't installed is
// drawn in the alternate its font table names (`w:altName`), before the roman rule; an
// alternate that can't be drawn either leaves the family class in charge: Cambria for a roman
// or modern font, the generic substitutes (Calibri in Word) for a swiss one.
#[derive(Clone, Copy)]
enum Family {
Swiss,
Roman,
Modern,
}
let face_of = |alt: Option<&str>, family: Family| {
let mut d = Document::from_text("Lanterns");
if let Some(alt) = alt {
d.settings.font_alt_names = vec![("No Such Brand Font".into(), alt.into())];
}
match family {
Family::Roman => d.settings.roman_fonts = vec!["No Such Brand Font".into()],
Family::Modern => d.settings.modern_fonts = vec!["No Such Brand Font".into()],
Family::Swiss => {}
}
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
p.runs[0].props.font = Some("No Such Brand Font".into());
let l = lay(&d);
let Some(Placed::Lines { para, .. }) = l.pages[0].items.iter().find(|i| matches!(i, Placed::Lines { .. })) else { panic!("no text") };
let first = para.clusters.first().expect("a cluster");
para.styles[first.style as usize].face
};
let face = |alt: Option<&str>, roman: bool| face_of(alt, if roman { Family::Roman } else { Family::Swiss });
let resolve = |f: &str| wordcraft_fonts::word::resolve(f, false, false).face;
let (courier, times, calibri, cambria) = (resolve("Courier New"), resolve("Times New Roman"), resolve("Calibri"), resolve("Cambria"));
let generic = face(None, false);
// Serif and sans faces are bundled, so these differ on any machine.
assert_ne!(times, generic);
assert_ne!(cambria, generic);
// A, B: a missing font with an alternate.
assert_eq!(face(Some("Times New Roman"), false), times);
assert_eq!(face(Some("Courier New"), false), courier);
// go_math (docxide-pdf corpus): Museo Sans 300 → Calibri.
assert_eq!(face(Some("Calibri"), false), calibri);
// D: the alternate beats the roman rule.
assert_eq!(face(Some("Times New Roman"), true), times);
assert_eq!(face(Some("Courier New"), true), courier);
// E: an alternate that is missing too, with no substitutes of its own: the roman rule, or the
// generic substitutes for a font that isn't roman.
assert_eq!(face(Some("No Such Alternate"), true), cambria);
assert_eq!(face(Some("No Such Alternate"), false), generic);
// Word reads a value with a comma as one name, not a list (MS-OE376 2.1.264): no font is
// called that, so the roman rule or the generic substitutes draw it.
assert_eq!(face(Some("No Such Alternate, Times New Roman"), false), generic);
assert_eq!(face(Some("No Such Alternate, Times New Roman"), true), cambria);
// A modern font: its installed alternate draws it, a missing alternate leaves it in Cambria,
// and with no alternate at all it takes the generic substitutes (Calibri in Word).
assert_eq!(face_of(Some("Courier New"), Family::Modern), courier);
assert_eq!(face_of(Some("No Such Alternate"), Family::Modern), cambria);
assert_eq!(face_of(None, Family::Modern), generic);
// An alternate that names another missing font with an alternate of its own: the chain isn't
// followed.
let mut d = Document::from_text("Lanterns");
d.settings.font_alt_names = vec![("No Such Brand Font".into(), "No Such Alternate".into()), ("No Such Alternate".into(), "Courier New".into())];
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
p.runs[0].props.font = Some("No Such Brand Font".into());
let l = lay(&d);
let Some(Placed::Lines { para, .. }) = l.pages[0].items.iter().find(|i| matches!(i, Placed::Lines { .. })) else { panic!("no text") };
let first = para.clusters.first().expect("a cluster");
assert_eq!(para.styles[first.style as usize].face, generic, "the chain isn't followed");
}
#[test]
fn an_installed_font_ignores_its_font_table_alternate() {
// The alternate is only for a font that isn't there: an installed (here bundled) font keeps its
// own face.
let font = wordcraft_fonts::FALLBACK_FAMILY;
let mut d = Document::from_text("Lanterns");
d.settings.font_alt_names = vec![(font.into(), "Times New Roman".into())];
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
p.runs[0].props.font = Some(font.into());
let l = lay(&d);
let Some(Placed::Lines { para, .. }) = l.pages[0].items.iter().find(|i| matches!(i, Placed::Lines { .. })) else { panic!("no text") };
let first = para.clusters.first().expect("a cluster");
let own = wordcraft_fonts::word::resolve(font, false, false).face;
assert_ne!(own, wordcraft_fonts::word::resolve("Times New Roman", false, false).face);
assert_eq!(para.styles[first.style as usize].face, own);
}
#[test]
fn a_missing_roman_font_with_its_own_substitute_keeps_it() {
crate::test_fonts();
@@ -3529,7 +3620,51 @@ fn character_border_draws_one_box_around_run() {
assert_eq!(horiz.len(), 2);
assert_eq!(vert.len(), 2);
assert!(horiz.iter().any(|h| (h.1 - top).abs() < 0.01) && horiz.iter().any(|h| (h.1 - bottom).abs() < 0.01), "{lines:?}");
assert!(horiz.iter().all(|h| (h.0 - gx0).abs() < 0.01 && (h.2 - gx1).abs() < 0.01), "{lines:?} {gx0} {gx1}");
// Top and bottom run on over half the 0.5 pt side lines, closing the corners.
assert!(horiz.iter().all(|h| (h.0 - (gx0 - 0.25)).abs() < 0.01 && (h.2 - (gx1 + 0.25)).abs() < 0.01), "{lines:?} {gx0} {gx1}");
}
#[test]
fn words_underline_draws_one_line_per_word() {
let mut d = Document::from_text("ab cd");
d.format_range(&Pos::body(0, 0), &Pos::body(0, 5), &|c| c.underline = Some(wordcraft_doc::props::Underline::Words)).unwrap();
assert_eq!(border_lines(&d).len(), 2, "{:?}", border_lines(&d));
// A space condensed to no width still parts the words.
d.format_range(&Pos::body(0, 0), &Pos::body(0, 5), &|c| c.spacing = Some(-4.0)).unwrap();
assert_eq!(border_lines(&d).len(), 2, "{:?}", border_lines(&d));
// A bookmark between two letters doesn't part them.
let mut d = Document::from_text("ab cd");
d.insert_object(&Pos::body(0, 1), InlineObject::BookmarkStart { name: "x".into() }, &Default::default()).unwrap();
let end = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap().text.len();
d.format_range(&Pos::body(0, 0), &Pos::body(0, end), &|c| c.underline = Some(wordcraft_doc::props::Underline::Words)).unwrap();
assert_eq!(border_lines(&d).len(), 2, "{:?}", border_lines(&d));
}
#[test]
fn paragraph_border_lines_meet_in_square_corners() {
use wordcraft_doc::props::{Border, Borders};
let mut d = Document::from_text("Boxed");
let (thick, thin) = (Some(Border::single(3.0)), Some(Border::single(1.0)));
d.para_mut(StoryRef::Body, &Path::top(0)).unwrap().props.borders = Some(Borders { top: thick, left: thin, bottom: thick, ..Borders::default() });
let lines = border_lines(&d);
let [left] = lines.iter().filter(|l| l.0 == l.2).collect::<Vec<_>>()[..] else { panic!("{lines:?}") };
let horiz: Vec<_> = lines.iter().filter(|l| l.1 == l.3).collect();
assert_eq!(horiz.len(), 2, "{lines:?}");
// Word: the 3 pt lines run on over half the 1 pt left line, and end half their own width past
// the right edge, where there is no line (docxide-pdf's samtale and english_town_council_report).
let right = d.last_section.page_w - d.last_section.margin_right;
for h in &horiz {
assert!((h.0 - (left.0 - 0.5)).abs() < 0.01 && (h.2 - (right + 1.5)).abs() < 0.01, "{lines:?} {right}");
}
// The left line runs on to the top and bottom lines' outer edges; with no top line it ends square.
assert!((left.1 - (horiz[0].1.min(horiz[1].1) - 1.5)).abs() < 0.01 && (left.3 - (horiz[0].1.max(horiz[1].1) + 1.5)).abs() < 0.01, "{lines:?}");
d.para_mut(StoryRef::Body, &Path::top(0)).unwrap().props.borders = Some(Borders { left: thin, bottom: thick, ..Borders::default() });
let lines = border_lines(&d);
let [left] = lines.iter().filter(|l| l.0 == l.2).collect::<Vec<_>>()[..] else { panic!("{lines:?}") };
let [bottom] = lines.iter().filter(|l| l.1 == l.3).collect::<Vec<_>>()[..] else { panic!("{lines:?}") };
assert!((left.3 - (bottom.1 + 1.5)).abs() < 0.01, "{lines:?}");
let first = lay(&d).pages[0].items.iter().find_map(|i| if let Placed::Lines { y, .. } = i { Some(*y) } else { None }).unwrap();
assert!((left.1 - (first - 0.5)).abs() < 0.01, "{lines:?} {first}");
}
#[test]
@@ -7927,6 +8062,72 @@ fn no_line_break_right_after_a_slash() {
assert!(lines > 10, "{lines} lines");
}
/// The text of every laid-out line.
fn body_lines(l: &DocLayout, text: &str) -> Vec<String> {
l.pages
.iter()
.flat_map(|p| &p.items)
.filter_map(|it| {
if let Placed::Lines { para, l0, l1, .. } = it {
Some(para.lines[*l0..*l1].iter().map(|line| text[line.start..line.stop].to_string()))
} else {
None
}
})
.flatten()
.collect()
}
#[test]
fn no_line_break_inside_a_query_address() {
// Word keeps "/?type=" and "/_back" whole (measured: a request address went to the next
// line whole, a long one breaking only where it no longer fits).
for addr in ["https://example.org/portal/_back/?type=objects_export&action=get", "https://example.org/portal/_back_office/reports/_export_all"] {
for k in 0..60 {
let text = format!("{}{addr} end", "a ".repeat(k));
let lines = body_lines(&lay(&Document::from_text(&text)), &text);
for (a, b) in lines.iter().zip(&lines[1..]) {
assert!(!a.ends_with('?') && !a.ends_with('/'), "{k}: a line ends inside the address: {a:?} / {b:?}");
}
}
}
}
/// A path that starts after a space ("caption /_back") wraps whole: UAX #14 has no break between
/// the space and the slash, so the break before the slash is the only one there.
#[test]
fn a_path_after_a_space_wraps_whole() {
let text = "enregistrement /_back_office_reports";
let mut wrapped = 0;
// Down to the width where the path still fits on a line (it no longer does at 425 pt).
for m in 100..=420 {
let mut d = Document::from_text(text);
d.last_section.margin_right = m as f32;
let lines = body_lines(&lay(&d), text);
if lines.len() < 2 {
continue; // fits on one line
}
wrapped += 1;
assert_eq!(lines, ["enregistrement ", "/_back_office_reports"], "the path is split at margin {m}");
}
assert!(wrapped > 50, "{wrapped} widths wrapped");
}
/// A question mark in CJK text keeps the break after it: only a query's `?` before an ASCII
/// letter, digit or underscore loses it.
#[test]
fn a_question_mark_in_cjk_text_keeps_its_break() {
let text = "你好?世界".repeat(40);
let mut ended = false;
for m in 300..=440 {
let mut d = Document::from_text(&text);
d.last_section.margin_right = m as f32;
let lines = body_lines(&lay(&d), &text);
ended |= lines[..lines.len() - 1].iter().any(|l| l.ends_with('?'));
}
assert!(ended, "no line ever ends after the question mark");
}
/// #2963: a word after a slash ("caption /заполняется") wraps whole, as in Word. UAX #14 forbids a
/// break between the space and the slash, and the break after the slash is dropped (Word keeps
/// "and/or" whole), so the slash word had no break before it and split between two letters.
@@ -10564,6 +10765,87 @@ fn east_asian_text_takes_the_asian_font_and_latin_text_the_latin_font() {
}
}
/// Text typed after a raised or lowered character sits where that character does: an input
/// method's composition there is drawn on the line's baseline shifted as the run is (positive up),
/// as the committed text is drawn, not on the line's own baseline.
#[test]
fn typing_style_takes_the_runs_baseline_shift() {
use wordcraft_doc::props::VertAlign;
let mut d = Document::from_text("x2");
d.format_range(&Pos::body(0, 1), &Pos::body(0, 2), &|c| c.vert_align = Some(VertAlign::Superscript)).unwrap();
let l = lay(&d);
let (plain, line) = l.typing_style(&Pos::body(0, 1), 0).expect("after the x");
let (raised, base) = l.typing_style(&Pos::body(0, 2), 0).expect("after the superscript 2");
assert_eq!(plain.shift, 0.0);
assert!(raised.shift > 0.0, "a superscript is raised: {}", raised.shift);
assert!((line - raised.shift - base).abs() < 0.01, "line {line}, shift {}, baseline {base}", raised.shift);
}
/// Text typed after a character the run's font lacks asks for the run's own font, not the fallback
/// face that drew that character: an input method's composition after a Chinese character in a
/// Latin font draws its letters as the committed letters are drawn.
#[test]
fn typing_style_asks_for_the_runs_font_after_a_fallback_character() {
let text = "A\u{4f60}";
let l = lay(&Document::from_text(text));
let para = l.pages[0].items.iter().find_map(|it| if let Placed::Lines { para, .. } = it { Some(para) } else { None }).unwrap();
let face_at = |off: usize| para.clusters.iter().find(|c| c.start == off).map(|c| para.styles[c.style as usize].face.id());
let (latin, han) = (face_at(0), face_at(1));
if latin == han {
eprintln!("skipped: the body font has the character, or no installed font does");
return;
}
let (run, _) = l.typing_style(&Pos::body(0, text.len()), 0).expect("after the Chinese character");
assert_eq!(Some(run.face.id()), han, "the character before the caret is drawn in a fallback face");
assert_eq!(Some(run.requested_face.id()), latin, "text typed there asks for the run's own font");
}
#[test]
fn a_missing_east_asian_font_is_drawn_in_its_font_table_alternate() {
crate::test_fonts();
// #2702 for the run's East Asian font (`w:eastAsia`): one that isn't installed is drawn in the
// alternate its font table names, when that one is installed and has the characters. Take an
// installed CJK font other than the one fallback would pick anyway, so the test tells them apart.
let resolve = |f: &str| wordcraft_fonts::word::resolve(f, false, false);
let fallback = wordcraft_fonts::FontDb::global().fallback_for('你', 0).map(|f| f.family.clone());
let Some(asian) = wordcraft_fonts::FontDb::global().east_asian_families().into_iter().find(|f| {
let r = resolve(f);
Some(f) != fallback.as_ref() && !r.substituted && r.face.covers('你') && r.face.covers('好')
}) else {
eprintln!("skipped: no second Chinese font installed");
return;
};
let missing = "No Such East Asian Font 2702";
let text = "Hello 你好";
let faces = |alternate: bool| {
let mut d = Document::from_text(text);
if alternate {
d.settings.font_alt_names = vec![(missing.into(), asian.clone())];
}
d.format_range(&Pos::body(0, 0), &Pos::body(0, text.len()), &|c| {
c.font = Some(wordcraft_doc::styles::BODY_FONT.into());
c.font_ea = Some(missing.into());
})
.unwrap();
let l = lay(&d);
let para =
l.pages.iter().flat_map(|p| p.items.iter()).find_map(|it| if let Placed::Lines { para, .. } = it { Some(para) } else { None }).unwrap();
let drawn = para
.clusters
.iter()
.filter(|c| text.get(c.start..c.end).and_then(|s| s.chars().next()).is_some_and(|ch| !ch.is_ascii()))
.map(|c| para.styles[c.style as usize].face.id())
.collect::<Vec<_>>();
(drawn, l.substituted_fonts())
};
let asian_face = resolve(&asian).face.id();
let (with, reported) = faces(true);
assert!(!with.is_empty() && with.iter().all(|f| *f == asian_face), "你好 in the alternate {asian}");
// Drawn in an installed alternate, the font asked for is still missing (#4770).
assert!(reported.iter().any(|(f, _)| f == missing), "{reported:?}");
assert!(faces(false).0.iter().all(|f| *f != asian_face), "without the alternate, fallback picks another font");
}
/// A page total in the footer ("Page X of Y") is filled in after pagination, so it doesn't make
/// the body lay out a second time; one in the body still does.
#[test]
@@ -10581,7 +10863,7 @@ fn footer_page_total_needs_one_body_pass() {
d.last_section.footers.default = Some(id);
let first = |d: &Document| {
let totals = Totals::first_guess(d.sections().len());
layout_pass(d, &mut LayoutCache::new(), &LayoutOptions::default(), &HashMap::new(), &totals, true, (&EarlyFloats::new(), true))
layout_pass(d, &mut LayoutCache::new(), &LayoutOptions::default(), (&HashMap::new(), None), &totals, true, (&EarlyFloats::new(), true))
};
assert!(matches!(first(&d), Pass::Done(l) if l.pages.len() == 3), "the footer's total asked for another pass");
// The same field in the body is only known once every page is laid out.
@@ -12241,6 +12523,219 @@ fn cell_widths_override_grid() {
}
}
#[test]
fn percentage_cell_widths_that_disagree_or_span_columns() {
// Measured in Word (A4, 451.3 pt between the margins; 468 pt here), autofit tables 100% wide:
// rows asking 30/10/10/30 and 10/30/10/30, in either order, give 30/30/10/30 (each column its
// largest); a merged cell asking 50% over the first two columns, beside 10% and 40%, gives
// those two 50% between them; a merged 20% over columns asking 40% and 10% changes nothing;
// and a nested table's percentages are of its own width.
let cell_text = |s: &str| vec![wordcraft_doc::para_block(Paragraph::with_text(s, Default::default()))];
let table = |rows: &[&[(Option<f32>, u32)]]| {
let mut t = Table::new(rows.len(), 4, 460.3);
t.grid = vec![107.3, 37.2, 31.95, 283.85];
t.props.width_pct = Some(100.0);
for (row, spec) in t.rows.iter_mut().zip(rows) {
row.cells.truncate(spec.len());
for (cell, (pct, span)) in row.cells.iter_mut().zip(spec.iter()) {
cell.props.width = None;
cell.props.width_pct = *pct;
cell.props.span = *span;
cell.blocks = cell_text("A");
}
}
t
};
let starts_at = |t: Table, prefix: &[u32], row: u32| {
let mut d = Document::from_text("After");
d.insert_block(StoryRef::Body, &Path::top(0), wordcraft_doc::Block::Table(t)).unwrap();
let l = lay(&d);
let x: Vec<f32> = (0..4)
.map(|c| {
let path: Vec<u32> = prefix.iter().copied().chain([row, c, 0]).collect();
let [(x, _, _)] = para_at(&l, &path)[..] else { panic!("cell {c} missing") };
x
})
.collect();
x.iter().map(|v| v - x[0]).collect::<Vec<_>>()
};
let near = |got: Vec<f32>, want: [f32; 4], what: &str| {
assert!(got.iter().zip(want).all(|(g, w)| (g - w).abs() < 0.01), "{what}: columns start at {got:?}, not {want:?}");
};
let at = |shares: [f32; 3]| [0.0, shares[0] * 468.0, shares[1] * 468.0, shares[2] * 468.0];
let p = |v: f32| (Some(v), 1);
let auto = (None, 1);
let open = [auto, auto, auto, auto];
near(
starts_at(table(&[&[p(30.0), p(10.0), p(10.0), p(30.0)], &[p(10.0), p(30.0), p(10.0), p(30.0)], &open]), &[0], 2),
at([0.3, 0.6, 0.7]),
"rows disagree",
);
near(
starts_at(table(&[&[p(10.0), p(30.0), p(10.0), p(30.0)], &[p(30.0), p(10.0), p(10.0), p(30.0)], &open]), &[0], 2),
at([0.3, 0.6, 0.7]),
"the other order",
);
let merged = starts_at(table(&[&[(Some(50.0), 2), p(10.0), p(40.0)], &open]), &[0], 1);
near(vec![merged[0], merged[2], merged[3]].into_iter().chain([0.0]).collect(), [0.0, 0.5 * 468.0, 0.6 * 468.0, 0.0], "merged 50%");
near(
starts_at(table(&[&[p(40.0), p(10.0), p(10.0), p(40.0)], &[(Some(20.0), 2), auto, auto], &open]), &[0], 2),
at([0.4, 0.5, 0.6]),
"a smaller merged cell",
);
// The nested table fills the outer table's only cell, less the cell's margins.
let inner = table(&[&[p(40.0), p(10.0), p(10.0), p(40.0)], &open]);
let mut outer = Table::new(1, 1, 468.0);
outer.props.width_pct = Some(100.0);
outer.rows[0].cells[0].props.width_pct = Some(100.0);
outer.rows[0].cells[0].blocks = vec![std::sync::Arc::new(wordcraft_doc::Block::Table(inner)), wordcraft_doc::para_block(Paragraph::new())];
let nested = starts_at(outer, &[0, 0, 0, 0], 1);
let w = nested[3] / 0.6;
near(nested, [0.0, 0.4 * w, 0.5 * w, 0.6 * w], "nested");
assert!(w > 400.0 && w <= 468.0, "the nested table is {w} wide");
}
#[test]
fn percentage_cell_widths_size_the_columns_whichever_rows_ask() {
// Measured in Word (A4, 451.3 pt between the margins; 468 pt here): a 40/10/10/40 header row
// over rows without cell widths lays out 40/10/10/40 of the table, from the grid
// 107.3/37.2/32/283.9 pt a document saved. The same with the row in the middle, in a fixed
// table, with the table 100% wide or without a width; 144 pt for 40% of a 360 pt table; and
// 20/5/5/20 scaled up to fill the table.
let starts = |t: Table| {
let mut d = Document::from_text("After");
d.insert_block(StoryRef::Body, &Path::top(0), wordcraft_doc::Block::Table(t)).unwrap();
let l = lay(&d);
let x: Vec<f32> = (0..4)
.map(|c| {
let [(x, _, _)] = para_at(&l, &[0, 2, c, 0])[..] else { panic!("cell {c} missing") };
x
})
.collect();
x.iter().map(|v| v - x[0]).collect::<Vec<_>>()
};
let table = |pct_row: usize, pcts: [f32; 4]| {
let mut t = Table::new(4, 4, 460.3);
t.grid = vec![107.3, 37.2, 31.95, 283.85];
for (r, row) in t.rows.iter_mut().enumerate() {
for (cell, p) in row.cells.iter_mut().zip(pcts) {
cell.props.width = None;
cell.props.width_pct = (r == pct_row).then_some(p);
cell.blocks = vec![wordcraft_doc::para_block(Paragraph::with_text("A", Default::default()))];
}
}
t
};
let near = |got: Vec<f32>, want: [f32; 4]| {
assert!(got.iter().zip(want).all(|(g, w)| (g - w).abs() < 0.01), "columns start at {got:?}, not {want:?}");
};
let of = |w: f32| [0.0, w * 0.4, w * 0.5, w * 0.6];
near(starts(table(0, [40.0, 10.0, 10.0, 40.0])), of(468.0));
near(starts(table(1, [40.0, 10.0, 10.0, 40.0])), of(468.0));
near(starts(table(0, [20.0, 5.0, 5.0, 20.0])), of(468.0));
// A fixed table without a width of its own keeps its grid's 460.3 pt; 100% wide, it takes
// the text width.
let mut fixed = table(0, [40.0, 10.0, 10.0, 40.0]);
fixed.props.fixed = true;
near(starts(fixed.clone()), of(460.3));
fixed.props.width_pct = Some(100.0);
near(starts(fixed), of(468.0));
let mut full = table(0, [40.0, 10.0, 10.0, 40.0]);
full.props.width_pct = Some(100.0);
near(starts(full), of(468.0));
let mut narrow = table(0, [20.0, 5.0, 5.0, 20.0]);
narrow.props.width = Some(360.0);
near(starts(narrow), of(360.0));
// A row that starts one column in gives its percentages to the columns it covers: 10/10/40
// there agree with the header row's 10/10/40 over the last three columns.
let mut shifted = table(0, [40.0, 10.0, 10.0, 40.0]);
shifted.rows[1].props.grid_before = 1;
shifted.rows[1].cells.remove(0);
for (cell, p) in shifted.rows[1].cells.iter_mut().zip([10.0, 10.0, 40.0]) {
cell.props.width_pct = Some(p);
}
near(starts(shifted), of(468.0));
// Past 100% Word gives every column but the last its share of the table and the last what
// is left: 44/11/11/44 starts the columns at 44%, 55% and 66%. At 60/20/20/60 nothing is left,
// so the last column takes its longest word and the other three give up the same width.
near(starts(table(0, [44.0, 11.0, 11.0, 44.0])), [0.0, 0.44, 0.55, 0.66].map(|f| f * 468.0));
let over = starts(table(0, [60.0, 20.0, 20.0, 60.0]));
let lost = [0.6 * 468.0 - over[1], 0.2 * 468.0 - (over[2] - over[1]), 0.2 * 468.0 - (over[3] - over[2])];
assert!(lost[0] > 1.0 && lost.iter().all(|l| (l - lost[0]).abs() < 0.01), "columns start at {over:?}");
// A fixed table gives the last column 2 pt and clips its text to the cell instead; the
// others share the rest of its 460.3 pt (Word: 275.5, 91.5, 91 and 2 pt).
let mut fixed_over = table(0, [60.0, 20.0, 20.0, 60.0]);
fixed_over.props.fixed = true;
for row in &mut fixed_over.rows {
row.cells[3].blocks = vec![wordcraft_doc::para_block(Paragraph::with_text("Lorem ipsum", Default::default()))];
}
near(starts(fixed_over.clone()), [0.0, 0.6, 0.8, 1.0].map(|f| f * (460.3 - 2.0)));
let mut d = Document::from_text("After");
d.insert_block(StoryRef::Body, &Path::top(0), wordcraft_doc::Block::Table(fixed_over)).unwrap();
let l = lay(&d);
let cells: Vec<(usize, f32, bool)> = l.pages[0]
.items
.iter()
.filter_map(|i| if let Placed::Cell { cell, rect, clip, row: 0, .. } = i { Some((*cell, rect.w, *clip)) } else { None })
.collect();
assert!(cells.iter().any(|&(c, w, clip)| c == 3 && (w - 2.0).abs() < 0.01 && clip), "the last cell is 2 pt and clipped: {cells:?}");
assert!(cells.iter().all(|&(c, _, clip)| c == 3 || !clip), "only the last cell is clipped: {cells:?}");
// When the columns before the last already pass 100% (80/40/40/20), Word gives them their
// shares left to right until the width runs out, and the table keeps its width: in a fixed
// table the third and fourth columns get 2 pt, the shares being of 460.3 - 4 pt; in an
// autofit table they get their longest words, the first two giving up the same width.
let widths = |t: Table| {
let mut d = Document::from_text("After");
d.insert_block(StoryRef::Body, &Path::top(0), wordcraft_doc::Block::Table(t)).unwrap();
let l = lay(&d);
let mut w: Vec<(usize, f32)> = l.pages[0]
.items
.iter()
.filter_map(|i| if let Placed::Cell { cell, rect, row: 0, .. } = i { Some((*cell, rect.w)) } else { None })
.collect();
w.sort_by_key(|c| c.0);
w.into_iter().map(|c| c.1).collect::<Vec<f32>>()
};
let mut fixed_far = table(0, [80.0, 40.0, 40.0, 20.0]);
fixed_far.props.fixed = true;
let got = widths(fixed_far);
let room = 460.3 - 4.0;
let want = [0.8 * room, 0.2 * room, 2.0, 2.0];
assert!(got.iter().zip(want).all(|(g, w)| (g - w).abs() < 0.01), "fixed 80/40/40/20: {got:?}, not {want:?}");
assert!((got.iter().sum::<f32>() - 460.3).abs() < 0.01, "fixed 80/40/40/20 is {} wide", got.iter().sum::<f32>());
// A fixed grid narrower than 2 pt a column keeps its width: the columns share it.
let mut fixed_narrow = table(0, [80.0, 40.0, 40.0, 20.0]);
fixed_narrow.props.fixed = true;
fixed_narrow.grid = vec![1.5; 4];
let got = widths(fixed_narrow);
assert!(got.iter().all(|w| (w - 1.5).abs() < 0.01), "a fixed 6 pt grid of 80/40/40/20: {got:?}");
let got = widths(table(0, [80.0, 40.0, 40.0, 20.0]));
assert!((got.iter().sum::<f32>() - 468.0).abs() < 0.01, "autofit 80/40/40/20 is {} wide: {got:?}", got.iter().sum::<f32>());
let lost = [0.8 * 468.0 - got[0], 0.2 * 468.0 - got[1]];
assert!(lost[0] > 1.0 && (lost[0] - lost[1]).abs() < 0.01 && got[2] > 1.0 && got[3] > 1.0, "autofit 80/40/40/20: {got:?}");
// A smaller percentage in another row of a column changes nothing: the column takes its
// largest. A merged cell's 20% goes to the two columns under it, which ask none.
let mut disagreeing = table(0, [40.0, 10.0, 10.0, 40.0]);
disagreeing.rows[1].cells[0].props.width_pct = Some(10.0);
near(starts(disagreeing), of(468.0));
let mut spanned = table(0, [40.0, 10.0, 10.0, 40.0]);
spanned.rows[0].cells.remove(2);
spanned.rows[0].cells[1].props.span = 2;
spanned.rows[0].cells[1].props.width_pct = Some(20.0);
let got = starts(spanned);
assert!((got[1] - 0.4 * 468.0).abs() < 0.01 && (got[3] - 0.6 * 468.0).abs() < 0.01, "merged 20%: columns start at {got:?}");
// A column no cell sizes and an absolute width alongside keep the grid.
let grid = [0.0, 107.3, 144.5, 176.45];
let mut unsized_column = table(0, [40.0, 10.0, 10.0, 40.0]);
unsized_column.rows[0].cells[3].props.width_pct = None;
let mut absolute = table(0, [40.0, 10.0, 10.0, 40.0]);
absolute.rows[1].cells[0].props.width = Some(100.0);
for t in [unsized_column, absolute] {
let got = starts(t);
let k = got[3] / grid[3];
near(got, grid.map(|g| g * k));
}
}
#[test]
fn ambiguous_widths_keep_grid() {
crate::test_fonts();
@@ -14382,7 +14877,15 @@ fn last_rewrap_pass_keeps_early_floats_clear_of_text() {
let one_page = picture_above_anchor(&"Earlier text flows beside the picture placed above its anchor. ".repeat(3), 4, 72.0);
for d in [one_page, early_float_across_pages()] {
let totals = Totals::first_guess(d.sections().len());
let pass = layout_pass(&d, &mut LayoutCache::new(), &LayoutOptions::default(), &HashMap::new(), &totals, false, (&EarlyFloats::new(), false));
let pass = layout_pass(
&d,
&mut LayoutCache::new(),
&LayoutOptions::default(),
(&HashMap::new(), None),
&totals,
false,
(&EarlyFloats::new(), false),
);
let Pass::Done(l) = pass else { panic!("a last pass is done") };
let (under, n) = text_under_shapes(&l);
assert_eq!(under, None);
+51
View File
@@ -167,6 +167,57 @@ fn a_missing_thai_font_draws_the_whole_run_in_one_thai_face() {
}
}
/// Body paragraph 0 of `text` at 16 pt, its run set in `font` whose font table alternate
/// (`w:altName`) is `alt`, and the face of every cluster: (cluster start, face).
fn cluster_faces_with_alternate(text: &str, font: &str, alt: &str) -> Vec<(usize, wordcraft_fonts::FaceRef)> {
let mut d = Document::from_text(text);
d.settings.font_alt_names = vec![(font.into(), alt.into())];
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
p.runs[0].props.font = Some(font.into());
p.runs[0].props.size = Some(16.0);
let l = lay(&d);
let pl = para_layout(&l, 0);
pl.clusters.iter().map(|c| (c.start, pl.styles[c.style as usize].face)).collect()
}
#[test]
fn a_missing_thai_font_with_a_latin_alternate_still_draws_the_run_in_one_face() {
if !thai_font_installed() {
eprintln!("skipped: no Thai font installed");
return;
}
// #2702 draws a missing font in its font table alternate. When the font is Thai and the
// installed alternate has no Thai letters, the run still takes one face for its Thai and
// Latin text (#1549), not the alternate for the Latin and per-letter fallback for the Thai.
let alt = "Source Sans 3";
let alt_face = wordcraft_fonts::word::resolve(alt, false, false).face;
assert!(!alt_face.covers('\u{0E19}'), "{} has no Thai, so the run must move", alt_face.family);
let db = wordcraft_fonts::FontDb::global();
// One face for the run needs a Thai font with Latin letters and digits too.
let thai_latin = thai_latin_font_installed();
if !thai_latin {
eprintln!("skipped the one-face cases: no Thai font with Latin letters installed");
}
for font in ["TH Sarabun New", "Angsana New"] {
if !thai_latin || db.has_family(font) {
continue;
}
let faces = cluster_faces_with_alternate(THAI_RUN, font, alt);
let first = faces[0].1;
let names: Vec<&str> = faces.iter().map(|(_, f)| f.family.as_str()).collect();
assert!(faces.iter().all(|(_, f)| f.id() == first.id()), "{font} (alternate {alt}): one face for the run, not {names:?}");
assert!(first.covers('\u{0E19}') && first.covers('D') && first.covers('1') && first.covers('\u{0E3F}'), "{font} → {}", first.family);
// A Latin-only run in the same font is drawn in the alternate, as #2702 has it.
let latin = cluster_faces_with_alternate("Lanterns 1,250", font, alt);
assert!(latin.iter().all(|(_, f)| f.id() == alt_face.id()), "{font}: Latin text in {alt}");
}
// A missing Latin font with the same alternate isn't Thai: its Latin text stays in the
// alternate and only the Thai letters fall back.
let faces = cluster_faces_with_alternate("Hello \u{0E2A}\u{0E27}\u{0E31}\u{0E2A}\u{0E14}\u{0E35}", "No Such Latin Font 2702", alt);
assert!(faces.iter().filter(|(at, _)| *at < 5).all(|(_, f)| f.id() == alt_face.id()), "Hello stays in {alt}");
assert!(faces.iter().any(|(at, f)| *at >= 6 && f.id() != alt_face.id()), "the Thai letters fall back");
}
#[test]
fn a_thai_substitute_without_latin_hands_the_run_to_one_face_covering_it() {
crate::test_fonts();
+1
View File
@@ -117,6 +117,7 @@ fn resolve_uses_the_given_system_locale() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn context_mode_on_a_very_long_digit_run_stays_linear() {
crate::test_fonts();
// Every digit used to scan back over all the digits and spaces before it (O(n²)).
+332
View File
@@ -345,6 +345,296 @@ fn page_references_in_footers_show_their_bookmarks_page() {
assert_eq!(texts, ["9"]);
}
/// Output (PDF, print, page images) shows each body `PAGEREF` its bookmark's page, counted from
/// the section's restart, as Word fills them in when it prints or exports; the editor keeps the
/// stored result until fields are updated, and a locked reference keeps it in output too.
/// Measured in Word for Microsoft 365: references stored as "9" to a heading on the third page
/// print and export as 3, or 2 when the second page starts a section numbered from 1; a locked
/// one stays 9, in the body and in a footer.
#[test]
fn page_references_in_the_body_show_their_bookmarks_page_in_output() {
let field = |instr: &str, locked: bool| InlineObject::Field {
instr: instr.into(),
result: "9".into(),
locked,
dirty: false,
code: false,
form: String::new(),
};
let mut d = Document::from_text("See page \nLocked \nBlank\nHeading");
for (i, instr, locked) in [(0, "PAGEREF _Toc1 \\h", false), (1, "PAGEREF _Toc1", true)] {
let p = d.para_mut(StoryRef::Body, &Path::top(i)).unwrap();
let n = p.len();
p.insert_object(n, field(instr, locked), &Default::default()).unwrap();
}
for i in [2, 3] {
d.para_mut(StoryRef::Body, &Path::top(i)).unwrap().props.page_break_before = Some(true);
}
let heading = d.para_mut(StoryRef::Body, &Path::top(3)).unwrap();
heading.insert_object(0, InlineObject::BookmarkStart { name: "_Toc1".into() }, &Default::default()).unwrap();
let refs = |l: &DocLayout| -> Vec<String> {
l.pages[0]
.items
.iter()
.flat_map(|it| if let Placed::Lines { para, .. } = it { para.shown.iter().map(|s| s.1.clone()).collect() } else { vec![] })
.collect()
};
let output = LayoutOptions { output_page_refs: true, ..Default::default() };
// Only an unlocked reference calls for the extra pass: a locked one keeps its result.
assert!(crate::headers::flow_refers_to_pages(&d));
let mut all_locked = d.clone();
if let Some(InlineObject::Field { locked, .. }) = all_locked.para_mut(StoryRef::Body, &Path::top(0)).unwrap().objects.last_mut() {
*locked = true;
}
assert!(!crate::headers::flow_refers_to_pages(&all_locked));
let l = lay(&d);
assert_eq!(l.pages.len(), 3);
assert_eq!(refs(&l), ["9", "9"]);
assert_eq!(refs(&layout(&d, &mut LayoutCache::new(), &output)), ["3", "9"]);
// The second page starts a section that numbers its pages from 1.
let first = d.last_section.clone();
d.para_mut(StoryRef::Body, &Path::top(1)).unwrap().section = Some(Box::new(first));
d.last_section.page_num_start = Some(1);
let l = layout(&d, &mut LayoutCache::new(), &output);
assert_eq!(l.pages.iter().map(|p| p.number).collect::<Vec<_>>(), [1, 1, 2]);
assert_eq!(refs(&l), ["2", "9"]);
assert_eq!(refs(&lay(&d)), ["9", "9"]);
// A footer's references: the unlocked one shows the page, the locked one keeps its result.
let mut fp = Paragraph::new();
fp.insert_object(0, field("PAGEREF _Toc1", false), &Default::default()).unwrap();
fp.insert_object(fp.len(), field("PAGEREF _Toc1", true), &Default::default()).unwrap();
d.last_section.footers.default = Some(d.add_part(PartKind::Footer, vec![para_block(fp)]));
assert_eq!(footer_shown(&lay(&d)).last().map(String::as_str), Some("2/9"));
}
/// Printed or exported, a reference to a bookmark the document doesn't have shows Word's error,
/// whether or not the document has other bookmarks (measured in Word for Windows: "Error!
/// Bookmark not defined."); the editor keeps the stored result, and so does a reference that
/// shows "above" or "below" (`\p`).
#[test]
fn a_page_reference_to_a_missing_bookmark_shows_words_error_in_output() {
let field =
|instr: &str| InlineObject::Field { instr: instr.into(), result: "9".into(), locked: false, dirty: false, code: false, form: String::new() };
let output = LayoutOptions { output_page_refs: true, ..Default::default() };
for other_mark in [false, true] {
let mut d = Document::from_text("See \nAbove \nTarget");
for (i, instr) in [(0, "PAGEREF NoSuchMark \\h"), (1, "PAGEREF NoSuchMark \\p")] {
let p = d.para_mut(StoryRef::Body, &Path::top(i)).unwrap();
let n = p.len();
p.insert_object(n, field(instr), &Default::default()).unwrap();
}
if other_mark {
let p = d.para_mut(StoryRef::Body, &Path::top(2)).unwrap();
p.insert_object(0, InlineObject::BookmarkStart { name: "Elsewhere".into() }, &Default::default()).unwrap();
}
let shown = |l: &DocLayout| -> Vec<String> {
l.pages[0]
.items
.iter()
.flat_map(|it| if let Placed::Lines { para, .. } = it { para.shown.iter().map(|s| s.1.clone()).collect() } else { vec![] })
.collect()
};
assert_eq!(shown(&layout(&d, &mut LayoutCache::new(), &output)), [crate::fields::BOOKMARK_NOT_DEFINED, "9"], "other bookmark: {other_mark}");
assert_eq!(shown(&lay(&d)), ["9", "9"], "the editor, other bookmark: {other_mark}");
// One cache for the editor's layout and then output's: output doesn't reuse the
// editor's paragraphs, and the editor doesn't reuse output's after it.
let mut cache = LayoutCache::new();
let editor = LayoutOptions::default();
assert_eq!(shown(&layout(&d, &mut cache, &editor)), ["9", "9"]);
assert_eq!(
shown(&layout(&d, &mut cache, &output)),
[crate::fields::BOOKMARK_NOT_DEFINED, "9"],
"a shared cache, other bookmark: {other_mark}"
);
assert_eq!(shown(&layout(&d, &mut cache, &editor)), ["9", "9"], "the editor after output, other bookmark: {other_mark}");
}
}
/// Printed or exported, a reference to a bookmark in a text box shows the page of the box's
/// anchor (measured in Word for Windows), not "Error! Bookmark not defined.".
#[test]
fn a_page_reference_to_a_bookmark_in_a_text_box_shows_its_anchors_page_in_output() {
use wordcraft_doc::para::{Float, ShapeKind, Wrap};
let mut d = Document::from_text("See \nSecond");
d.para_mut(StoryRef::Body, &Path::top(1)).unwrap().props.page_break_before = Some(true);
let mut inner = Paragraph::with_text("Caption", Default::default());
inner.insert_object(0, InlineObject::BookmarkStart { name: "Boxed".into() }, &Default::default()).unwrap();
let story = d.add_part(PartKind::TextBox, vec![para_block(inner)]);
let shape = InlineObject::Shape {
kind: ShapeKind::TextBox,
w: 100.0,
h: 40.0,
fill: None,
stroke: None,
stroke_width: 0.0,
float: Float { wrap: Wrap::Inline, ..Default::default() },
story: Some(story),
freeform: None,
effects: Default::default(),
extra: Default::default(),
};
d.para_mut(StoryRef::Body, &Path::top(1)).unwrap().insert_object(0, shape, &Default::default()).unwrap();
let field =
InlineObject::Field { instr: "PAGEREF Boxed \\h".into(), result: "9".into(), locked: false, dirty: false, code: false, form: String::new() };
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
let n = p.len();
p.insert_object(n, field, &Default::default()).unwrap();
let output = layout(&d, &mut LayoutCache::new(), &LayoutOptions { output_page_refs: true, ..Default::default() });
let shown: Vec<String> = output.pages[0]
.items
.iter()
.flat_map(|it| match it {
Placed::Lines { story: StoryRef::Body, para, .. } => para.shown.iter().map(|s| s.1.clone()).collect(),
_ => vec![],
})
.collect();
assert_eq!(shown, ["2"]);
}
/// The field results shown on page `page` by the lines of `story`.
fn shown_in(l: &DocLayout, page: usize, story: StoryRef) -> Vec<String> {
l.pages[page]
.items
.iter()
.flat_map(|it| match it {
Placed::Lines { story: s, para, .. } if *s == story => para.shown.iter().map(|s| s.1.clone()).collect(),
_ => vec![],
})
.collect()
}
/// A page reference field showing `result` until it is updated.
fn pageref(instr: &str, result: &str) -> InlineObject {
InlineObject::Field { instr: instr.into(), result: result.into(), locked: false, dirty: false, code: false, form: String::new() }
}
/// Three pages, a bookmark "Body" on the third; a text box on the first page and a footnote on
/// the second refer to it, the footnote also to a bookmark that doesn't exist, and holds the
/// bookmark "InNote"; the header holds the bookmark "InHeader". With `body_refs` the first
/// paragraph refers to "InNote" and "InHeader" too. Returns the document and the box's and the
/// note's stories.
fn refs_outside_the_body(body_refs: bool) -> (Document, StoryRef, StoryRef) {
use wordcraft_doc::para::{Float, ShapeKind, Wrap};
let mut d = Document::from_text("Intro \nSecond\nTarget");
for i in [1, 2] {
d.para_mut(StoryRef::Body, &Path::top(i)).unwrap().props.page_break_before = Some(true);
}
d.para_mut(StoryRef::Body, &Path::top(2))
.unwrap()
.insert_object(0, InlineObject::BookmarkStart { name: "Body".into() }, &Default::default())
.unwrap();
let mut boxed = Paragraph::with_text("Box ", Default::default());
boxed.insert_object(boxed.len(), pageref("PAGEREF Body \\h", "9"), &Default::default()).unwrap();
let box_story = d.add_part(PartKind::TextBox, vec![para_block(boxed)]);
let shape = InlineObject::Shape {
kind: ShapeKind::TextBox,
w: 100.0,
h: 40.0,
fill: None,
stroke: None,
stroke_width: 0.0,
float: Float { wrap: Wrap::Inline, ..Default::default() },
story: Some(box_story),
freeform: None,
effects: Default::default(),
extra: Default::default(),
};
d.para_mut(StoryRef::Body, &Path::top(0)).unwrap().insert_object(0, shape, &Default::default()).unwrap();
let mut note = Paragraph::with_text("Note ", Default::default());
note.insert_object(0, InlineObject::BookmarkStart { name: "InNote".into() }, &Default::default()).unwrap();
for f in [pageref("PAGEREF Body \\h", "9"), pageref("PAGEREF Gone \\h", "9")] {
note.insert_object(note.len(), f, &Default::default()).unwrap();
}
let note_id = d.add_part(PartKind::Footnote, vec![para_block(note)]);
d.insert_object(&Pos::body(1, 6), InlineObject::NoteRef { kind: NoteKind::Footnote, id: note_id, custom: String::new() }, &Default::default())
.unwrap();
let mut hp = Paragraph::with_text("Header", Default::default());
hp.insert_object(0, InlineObject::BookmarkStart { name: "InHeader".into() }, &Default::default()).unwrap();
d.last_section.headers.default = Some(d.add_part(PartKind::Header, vec![para_block(hp)]));
if body_refs {
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
for f in [pageref("PAGEREF InNote \\h", "9"), pageref("PAGEREF InHeader \\h", "9")] {
p.insert_object(p.len(), f, &Default::default()).unwrap();
}
}
(d, StoryRef::Part(box_story), StoryRef::Part(note_id))
}
/// Printed or exported, page references in text boxes and notes show their bookmarks' pages too,
/// also when the body has none (the extra pass is for them as well); a bookmark in a note is on
/// the page that shows the note; a reference to a bookmark that exists where pages aren't found
/// (a header) keeps its result, and only one to a bookmark nowhere in the document shows Word's
/// error. The editor keeps every stored result.
#[test]
fn page_references_outside_the_body_show_pages_in_output() {
let output = LayoutOptions { output_page_refs: true, ..Default::default() };
for body_refs in [false, true] {
let (d, boxed, note) = refs_outside_the_body(body_refs);
assert!(crate::headers::flow_refers_to_pages(&d));
let l = layout(&d, &mut LayoutCache::new(), &output);
assert_eq!(l.pages.len(), 3);
assert_eq!(shown_in(&l, 0, boxed), ["3"], "the text box, body refs: {body_refs}");
assert_eq!(shown_in(&l, 1, note), ["3", crate::fields::BOOKMARK_NOT_DEFINED], "the footnote, body refs: {body_refs}");
let body: Vec<String> = if body_refs { vec!["2".into(), "9".into()] } else { vec![] };
assert_eq!(shown_in(&l, 0, StoryRef::Body), body, "body refs: {body_refs}");
let editor = lay(&d);
assert_eq!(shown_in(&editor, 0, boxed), ["9"]);
assert_eq!(shown_in(&editor, 1, note), ["9", "9"]);
}
}
/// Output shows a reference to a missing bookmark in the text it is given for the interface
/// language (Word prints its error in its own), English without one.
#[test]
fn a_missing_bookmarks_error_is_in_the_interface_language() {
let (d, _, note) = refs_outside_the_body(false);
let french: Arc<str> = "Erreur ! Signet non défini.".into();
let l =
layout(&d, &mut LayoutCache::new(), &LayoutOptions { output_page_refs: true, bookmark_error: Some(french.clone()), ..Default::default() });
assert_eq!(shown_in(&l, 1, note), ["3", &*french]);
}
/// The first pass of output, before any bookmark's page is known, shows the stored results (the
/// closest guess: a table of contents isn't stretched by errors), not Word's error.
#[test]
fn outputs_first_pass_shows_stored_page_references() {
let mut d = Document::from_text("See ");
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
p.insert_object(p.len(), pageref("PAGEREF Gone \\h", "9"), &Default::default()).unwrap();
let output = LayoutOptions { output_page_refs: true, ..Default::default() };
let totals = crate::cache::Totals::first_guess(d.sections().len());
let first = crate::pass::layout_pass(
&d,
&mut LayoutCache::new(),
&output,
(&std::collections::HashMap::new(), None),
&totals,
true,
(&crate::floats::EarlyFloats::new(), true),
);
let crate::ctx::Pass::Done(l) = first else { panic!("one page needs no other pass") };
assert_eq!(shown_in(&l, 0, StoryRef::Body), ["9"]);
assert_eq!(shown_in(&layout(&d, &mut LayoutCache::new(), &output), 0, StoryRef::Body), [crate::fields::BOOKMARK_NOT_DEFINED]);
}
/// Output's page references can change the page count after the totals settled (here many
/// references to a missing bookmark grow from their stored "9" to Word's error, adding pages):
/// the body's `NUMPAGES` still shows the pages the document ends with.
#[test]
fn output_page_references_that_change_the_page_count_leave_totals_right() {
const REFS: usize = 300;
let mut d = Document::from_text(&" ".repeat(REFS));
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
for off in (1..=REFS).rev() {
p.insert_object(off, pageref("PAGEREF Gone \\h", "9"), &Default::default()).unwrap();
}
p.insert_object(0, pageref("NUMPAGES", "9"), &Default::default()).unwrap();
let stored = lay(&d).pages.len();
let l = layout(&d, &mut LayoutCache::new(), &LayoutOptions { output_page_refs: true, ..Default::default() });
assert!(l.pages.len() > stored, "{} pages, {stored} with the stored results", l.pages.len());
assert_eq!(shown_in(&l, 0, StoryRef::Body).first(), Some(&l.pages.len().to_string()));
}
/// The fields' text in each page's footer, joined with "/".
fn footer_shown(l: &DocLayout) -> Vec<String> {
l.pages
@@ -771,3 +1061,45 @@ fn table_cell_page_field_uses_the_section_number_format() {
.collect();
assert_eq!(shown, "one");
}
/// Printed or exported, a reference to a bookmark in a text box nested in text boxes shows the
/// outermost box's anchor page as deep as layout shows the boxes ([`wordcraft_doc::MAX_TEXT_BOX_DEPTH`]
/// levels); one nested deeper, which layout doesn't show, keeps its stored result.
#[test]
fn a_page_reference_into_nested_text_boxes_stops_where_layout_does() {
use wordcraft_doc::para::{Float, ShapeKind, Wrap};
let shape = |story: u32| InlineObject::Shape {
kind: ShapeKind::TextBox,
w: 100.0,
h: 40.0,
fill: None,
stroke: None,
stroke_width: 0.0,
float: Float { wrap: Wrap::Inline, ..Default::default() },
story: Some(story),
freeform: None,
effects: Default::default(),
extra: Default::default(),
};
let deepest = wordcraft_doc::MAX_TEXT_BOX_DEPTH + 1;
let mut d = Document::from_text("See \nSecond");
d.para_mut(StoryRef::Body, &Path::top(1)).unwrap().props.page_break_before = Some(true);
// From the deepest box out: box `level` holds bookmark "Level{level}" and the box inside it.
let mut inner: Option<u32> = None;
for level in (1..=deepest).rev() {
let mut p = Paragraph::with_text("Box", Default::default());
p.insert_object(0, InlineObject::BookmarkStart { name: format!("Level{level}") }, &Default::default()).unwrap();
if let Some(id) = inner {
p.insert_object(p.len(), shape(id), &Default::default()).unwrap();
}
inner = Some(d.add_part(PartKind::TextBox, vec![para_block(p)]));
}
d.para_mut(StoryRef::Body, &Path::top(1)).unwrap().insert_object(0, shape(inner.unwrap()), &Default::default()).unwrap();
let p = d.para_mut(StoryRef::Body, &Path::top(0)).unwrap();
for level in [1, wordcraft_doc::MAX_TEXT_BOX_DEPTH, deepest] {
let n = p.len();
p.insert_object(n, pageref(&format!("PAGEREF Level{level} \\h"), "9"), &Default::default()).unwrap();
}
let output = layout(&d, &mut LayoutCache::new(), &LayoutOptions { output_page_refs: true, ..Default::default() });
assert_eq!(shown_in(&output, 0, StoryRef::Body), ["2", "2", "9"]);
}
+1
View File
@@ -13,5 +13,6 @@ wordcraft-doc = { workspace = true }
wordcraft-engine = { workspace = true }
wordcraft-render = { workspace = true }
wordcraft-control-key = { workspace = true }
craft-mcp-server = { workspace = true }
serde_json = { workspace = true }
log = { workspace = true }
+64 -129
View File
@@ -1,22 +1,16 @@
//! JSON-RPC 2.0 framing and the MCP lifecycle / tools / resources methods.
//! WordCraft's MCP server: the shared `craft_mcp_server` core (JSON-RPC framing, the lifecycle,
//! `tools/*` and `resources/*`) around WordCraft's own tools, resources and instructions.
use std::io::{BufRead, Read, Write};
use craft_mcp_server::{Contents, Info, Options, Tools};
use serde_json::{Value, json};
use crate::backend::Backend;
use crate::tools::{call_tool, tool_definitions};
/// The MCP revision we implement.
pub const PROTOCOL_VERSION: &str = "2025-06-18";
const SUPPORTED_VERSIONS: &[&str] = &[PROTOCOL_VERSION, "2025-03-26", "2024-11-05"];
const PARSE_ERROR: i64 = -32700;
const INVALID_REQUEST: i64 = -32600;
const METHOD_NOT_FOUND: i64 = -32601;
const INVALID_PARAMS: i64 = -32602;
const INTERNAL_ERROR: i64 = -32603;
const RESOURCE_NOT_FOUND: i64 = -32002;
pub use craft_mcp_server::PROTOCOL_VERSION;
const INSTRUCTIONS: &str = "WordCraft is a word processor (a Microsoft Word clone). Every action is a command: find ids and \
parameters with list_commands and run them with execute (or several with batch). Positions are \
@@ -74,149 +68,90 @@ pub(crate) fn read_bounded_line(reader: &mut impl BufRead, max: usize) -> std::i
String::from_utf8(buf).map(BoundedLine::Line).map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))
}
/// WordCraft's tools, resources and instructions over one backend, for the shared server.
struct WordcraftTools {
backend: Box<dyn Backend>,
}
impl Tools for WordcraftTools {
fn instructions(&self) -> String {
format!(
"{INSTRUCTIONS} Backend: {}. Community: https://discord.gg/artcraft · https://getartcraft.com/apps/wordcraft · https://github.com/storytold/wordcraft",
self.backend.describe()
)
}
fn list(&mut self) -> Vec<Value> {
match tool_definitions() {
Value::Array(tools) => tools,
_ => Vec::new(),
}
}
fn call(&mut self, name: &str, args: &Value) -> Value {
call_tool(self.backend.as_mut(), name, args).to_value()
}
fn resources(&self) -> Vec<Value> {
vec![
json!({"uri": DOC_URI, "name": "document", "title": "Active document", "description": "Blocks, text, formatting, sections, parts and selection of the document (document.inspect)", "mimeType": "application/json"}),
json!({"uri": COMMANDS_URI, "name": "commands", "title": "Commands", "description": "Every command: id, label, ribbon location, shortcut, params, enabled (engine.commands)", "mimeType": "application/json"}),
]
}
fn read_resource(&mut self, uri: &str) -> Option<Result<Contents, String>> {
let method = match uri {
DOC_URI => "document.inspect",
COMMANDS_URI => "engine.commands",
_ => return None,
};
Some(
self.backend
.call(method, json!({}))
.map(|v| Contents::Text { mime_type: "application/json".into(), text: serde_json::to_string_pretty(&v).unwrap_or_default() }),
)
}
}
/// An MCP server bound to one backend.
pub struct Server {
backend: Box<dyn Backend>,
initialized: bool,
}
fn response(id: Value, result: Value) -> Value {
json!({"jsonrpc": "2.0", "id": id, "result": result})
}
fn error(id: Value, code: i64, message: impl Into<String>) -> Value {
json!({"jsonrpc": "2.0", "id": id, "error": {"code": code, "message": message.into()}})
inner: craft_mcp_server::Server<WordcraftTools>,
}
impl Server {
pub fn new(backend: Box<dyn Backend>) -> Self {
Self { backend, initialized: false }
let info = Info { name: "wordcraft", title: "WordCraft", version: env!("CARGO_PKG_VERSION"), max_line: MAX_REQUEST_LINE };
let options = Options {
website_url: Some("https://getartcraft.com/apps/wordcraft".into()),
// Clients read every tool failure from the result, as before the shared core.
tool_errors_as_results: true,
strict_request_ids: true,
};
Self { inner: craft_mcp_server::Server::new(info, WordcraftTools { backend }).with_options(options) }
}
pub fn backend(&mut self) -> &mut dyn Backend {
self.backend.as_mut()
self.inner.tools().backend.as_mut()
}
/// Whether the client has sent `notifications/initialized`.
pub fn is_initialized(&self) -> bool {
self.initialized
self.inner.is_initialized()
}
/// Serve newline-delimited JSON-RPC until `input` closes. Logs go to stderr only (stdout is
/// the protocol stream).
pub fn serve(&mut self, mut input: impl BufRead, mut output: impl Write) -> std::io::Result<()> {
loop {
let reply = match read_bounded_line(&mut input, MAX_REQUEST_LINE) {
Ok(BoundedLine::Eof) => break,
Ok(BoundedLine::Line(line)) => self.handle_line(&line),
Ok(BoundedLine::TooLong) => Some(error(Value::Null, INVALID_REQUEST, "request line too long").to_string()),
Err(e) if e.kind() == std::io::ErrorKind::InvalidData => {
Some(error(Value::Null, PARSE_ERROR, "request line is not UTF-8").to_string())
}
Err(e) => return Err(e),
};
if let Some(reply) = reply {
output.write_all(reply.as_bytes())?;
output.write_all(b"\n")?;
output.flush()?;
}
}
Ok(())
pub fn serve(&mut self, input: impl BufRead, output: impl Write) -> std::io::Result<()> {
self.inner.serve(input, output)
}
/// Handle one line; returns the reply line (None for notifications and blank lines).
pub fn handle_line(&mut self, line: &str) -> Option<String> {
let line = line.trim();
if line.is_empty() {
return None;
}
let reply = match serde_json::from_str::<Value>(line) {
Ok(Value::Array(batch)) => {
// Batches were removed in 2025-06-18; still answer older clients sensibly.
if batch.is_empty() {
Some(error(Value::Null, INVALID_REQUEST, "empty batch"))
} else {
let replies: Vec<Value> = batch.into_iter().filter_map(|m| self.handle(m)).collect();
(!replies.is_empty()).then_some(Value::Array(replies))
}
}
Ok(msg) => self.handle(msg),
Err(e) => Some(error(Value::Null, PARSE_ERROR, format!("parse error: {e}"))),
};
reply.map(|r| r.to_string())
self.inner.handle_line(line)
}
/// Handle one JSON-RPC message; `None` for notifications and responses.
pub fn handle(&mut self, msg: Value) -> Option<Value> {
let Value::Object(o) = &msg else { return Some(error(Value::Null, INVALID_REQUEST, "message must be an object")) };
let id = o.get("id").cloned();
let Some(method) = o.get("method").and_then(Value::as_str) else {
// A response to a server→client request (we send none): ignore. Anything else is invalid.
if o.contains_key("result") || o.contains_key("error") {
return None;
}
return Some(error(id.unwrap_or(Value::Null), INVALID_REQUEST, "missing `method`"));
};
let params = o.get("params").cloned().unwrap_or(Value::Null);
let Some(id) = id else {
self.notification(method);
return None;
};
if !(id.is_string() || id.is_number()) {
return Some(error(Value::Null, INVALID_REQUEST, "`id` must be a string or number"));
}
Some(match self.request(method, &params) {
Ok(r) => response(id, r),
Err((code, m)) => error(id, code, m),
})
}
fn notification(&mut self, method: &str) {
match method {
"notifications/initialized" => self.initialized = true,
"notifications/cancelled" | "notifications/progress" | "notifications/roots/list_changed" => {}
other => log::debug!("ignoring notification {other}"),
}
}
fn request(&mut self, method: &str, params: &Value) -> Result<Value, (i64, String)> {
match method {
"initialize" => {
let asked = params.get("protocolVersion").and_then(Value::as_str).unwrap_or(PROTOCOL_VERSION);
let version = if SUPPORTED_VERSIONS.contains(&asked) { asked } else { PROTOCOL_VERSION };
Ok(json!({
"protocolVersion": version,
"capabilities": {"tools": {}, "resources": {}},
"serverInfo": {"name": "wordcraft", "title": "WordCraft", "version": env!("CARGO_PKG_VERSION"), "websiteUrl": "https://getartcraft.com/apps/wordcraft"},
"instructions": format!(
"{INSTRUCTIONS} Backend: {}. Community: https://discord.gg/artcraft · https://getartcraft.com/apps/wordcraft · https://github.com/storytold/wordcraft",
self.backend.describe()
),
}))
}
"ping" => Ok(json!({})),
"tools/list" => Ok(json!({"tools": tool_definitions()})),
"tools/call" => {
let name = params.get("name").and_then(Value::as_str).ok_or((INVALID_PARAMS, "missing tool `name`".to_string()))?;
let args = params.get("arguments").cloned().unwrap_or(Value::Null);
Ok(call_tool(self.backend.as_mut(), name, &args).to_value())
}
"resources/list" => Ok(json!({"resources": [
{"uri": DOC_URI, "name": "document", "title": "Active document", "description": "Blocks, text, formatting, sections, parts and selection of the document (document.inspect)", "mimeType": "application/json"},
{"uri": COMMANDS_URI, "name": "commands", "title": "Commands", "description": "Every command: id, label, ribbon location, shortcut, params, enabled (engine.commands)", "mimeType": "application/json"},
]})),
"resources/templates/list" => Ok(json!({"resourceTemplates": []})),
"resources/read" => {
let uri = params.get("uri").and_then(Value::as_str).ok_or((INVALID_PARAMS, "missing `uri`".to_string()))?;
let v = match uri {
DOC_URI => self.backend.call("document.inspect", json!({})),
COMMANDS_URI => self.backend.call("engine.commands", json!({})),
_ => return Err((RESOURCE_NOT_FOUND, format!("resource not found: {uri}"))),
}
.map_err(|e| (INTERNAL_ERROR, e))?;
Ok(json!({"contents": [{"uri": uri, "mimeType": "application/json", "text": serde_json::to_string_pretty(&v).unwrap_or_default()}]}))
}
other => Err((METHOD_NOT_FOUND, format!("method not found: {other}"))),
}
self.inner.handle(msg)
}
}
+8 -2
View File
@@ -4,6 +4,9 @@ use serde_json::{Value, json};
use crate::{Headless, Server};
mod differential;
mod oracle;
fn call(s: &mut Server, id: u64, method: &str, params: Value) -> Value {
let line = json!({"jsonrpc": "2.0", "id": id, "method": method, "params": params}).to_string();
let r = s.handle_line(&line).expect("reply");
@@ -169,7 +172,9 @@ fn render_and_screenshot_through_the_app_use_no_temp_file() {
let calls = std::sync::Arc::new(std::sync::Mutex::new(Vec::new()));
let mut s = Server::new(Box::new(FakeApp { calls: calls.clone() }));
for (tool, method) in [("render_page", "ui.render"), ("screenshot", "ui.screenshot")] {
let r = call(&mut s, 10, "tools/call", json!({"name": tool, "arguments": {"page": 1}}));
// `screenshot` takes no arguments (the shared check refuses ones its schema does not list).
let args = if tool == "render_page" { json!({"page": 1}) } else { json!({}) };
let r = call(&mut s, 10, "tools/call", json!({"name": tool, "arguments": args}));
let content = &r["result"]["content"];
assert_eq!(r["result"]["isError"], false, "{tool}: {r}");
assert_eq!(content[0]["type"], "image", "{tool}");
@@ -310,7 +315,8 @@ fn over_long_request_line_is_rejected_and_serving_continues() {
let text = String::from_utf8(out).unwrap();
let lines: Vec<&str> = text.lines().collect();
assert_eq!(lines.len(), 2, "{text}");
assert!(lines[0].contains("too long"), "{}", lines[0]);
// The shared core words it "request too large" (was "request line too long"); same -32600.
assert!(lines[0].contains("too large") && lines[0].contains("-32600"), "{}", lines[0]);
assert!(lines[1].contains(r#""id":5"#), "{}", lines[1]);
}
+452
View File
@@ -0,0 +1,452 @@
//! The shared server against the hand-written one it replaced (`oracle`): the same scripted
//! JSON-RPC session goes through both and the replies must be equal, after normalising only the
//! differences listed in `normalise`. Differences that cannot be normalised away are pinned by
//! their own tests below. Document ids (a process-wide counter) are zeroed before comparing.
use serde_json::{Value, json};
use super::oracle;
use crate::{Headless, Server};
/// One input line of the session; `free_text` marks a tool call whose failure text is the
/// validator's own words (both sides must still say `isError: true`).
struct Step {
line: Vec<u8>,
free_text: bool,
}
fn req(id: Value, method: &str, params: Value) -> Vec<u8> {
json!({"jsonrpc": "2.0", "id": id, "method": method, "params": params}).to_string().into_bytes()
}
fn step(line: Vec<u8>) -> Step {
Step { line, free_text: false }
}
fn call(id: u64, name: &str, args: Value) -> Step {
step(req(json!(id), "tools/call", json!({"name": name, "arguments": args})))
}
fn call_free(id: u64, name: &str, args: Value) -> Step {
Step { free_text: true, ..call(id, name, args) }
}
fn script() -> Vec<Step> {
let mut s = Vec::new();
// initialize with each supported version, an unknown one, an odd one, and none.
for (i, v) in [json!("2025-06-18"), json!("2025-03-26"), json!("2024-11-05"), json!("1999-01-01"), json!(7), Value::Null].into_iter().enumerate()
{
s.push(step(req(json!(i + 1), "initialize", json!({"protocolVersion": v, "capabilities": {}}))));
}
s.push(step(req(json!(20), "initialize", json!({}))));
s.push(step(req(json!(21), "initialize", Value::Null)));
s.push(step(br#"{"jsonrpc":"2.0","method":"notifications/initialized"}"#.to_vec()));
s.push(step(req(json!(30), "ping", json!({}))));
s.push(step(req(json!(31), "tools/list", json!({}))));
s.push(step(req(json!(32), "resources/templates/list", json!({}))));
// tools: a realistic session, an image, an error result, a doc-addressed call.
for (i, (name, args)) in [
("new_document", json!({})),
("new_document", json!({"template": "sample"})),
("type_text", json!({"text": "Hello there"})),
("type_text", json!({"text": "a\nb", "paragraphs": true, "doc": null})),
("select_text", json!({"text": "Hello", "occurrence": 1})),
("execute", json!({"command": "format.bold"})),
("execute", json!({"command": "no.such.command"})),
("execute", json!({"command": "text.insert", "params": {"text": "x"}, "doc": 99})),
("batch", json!({"commands": [{"command": "caret.docEnd"}, {"command": "text.insert", "params": {"text": "!"}}]})),
("batch", json!({"commands": [{"command": "no.such"}, {"command": "caret.docEnd"}], "keepGoing": true})),
("insert_table", json!({"values": [["a", 1], ["b", 2]], "header": true})),
("list_commands", json!({"query": "bold"})),
("list_documents", json!({})),
("get_text", json!({})),
("inspect_document", json!({})),
("render_page", json!({"page": 1, "scale": 0.2})),
("render_page", json!({"page": 99})),
("screenshot", json!({})),
("click", json!({"x": 1, "y": 2})),
("key", json!({"key": "B", "cmd": true})),
("parity", json!({})),
("save_document", json!({"path": "/nonexistent-dir-wc/x.docx"})),
]
.into_iter()
.enumerate()
{
s.push(call(100 + i as u64, name, args));
}
// bad arguments: both sides fail with isError; the words are the validator's own.
s.push(call_free(200, "execute", json!({})));
s.push(call_free(201, "execute", json!({"command": 5})));
s.push(call_free(203, "insert_table", json!({"values": "x"})));
s.push(call_free(204, "insert_table", json!({"values": null})));
s.push(call_free(205, "get_text", json!({"doc": "1"})));
s.push(call_free(206, "render_page", json!({"doc": 1.5})));
s.push(call_free(207, "open_document", json!({})));
s.push(call_free(208, "execute", json!("not an object")));
// A missing or null `arguments` is `{}` to the shared server (it was null).
for (id, params) in [(209, json!({"name": "execute"})), (210, json!({"name": "execute", "arguments": null}))] {
s.push(Step { free_text: true, ..step(req(json!(id), "tools/call", params)) });
}
s.push(step(req(json!(211), "tools/call", json!({"name": "nope", "arguments": {}}))));
s.push(step(req(json!(212), "tools/call", json!({}))));
s.push(step(req(json!(213), "tools/call", json!({"name": 5}))));
s.push(step(req(json!(214), "tools/call", Value::Null)));
// odd ids.
for id in [json!("abc"), json!(0), json!(-3), json!(1.5), json!(u64::MAX), json!(null), json!(true), json!([1]), json!({"a": 1})] {
s.push(step(req(id, "ping", json!({}))));
}
s.push(step(br#"{"jsonrpc":"2.0","method":"ping"}"#.to_vec()));
// resources.
s.push(step(req(json!(300), "resources/list", json!({}))));
s.push(step(req(json!(301), "resources/read", json!({"uri": "wordcraft://document"}))));
s.push(step(req(json!(302), "resources/read", json!({"uri": "wordcraft://commands"}))));
s.push(step(req(json!(303), "resources/read", json!({"uri": "wordcraft://nothing"}))));
// unknown methods, malformed messages, notifications.
s.push(step(req(json!(310), "nope", json!({}))));
s.push(step(req(json!(311), "tools/nope", Value::Null)));
for raw in [
&br#"{"jsonrpc":"2.0","method":"notifications/cancelled","params":{"requestId":1}}"#[..],
br#"{"jsonrpc":"2.0","method":"notifications/progress"}"#,
br#"{"jsonrpc":"2.0","method":"notifications/whatever"}"#,
br#"{"jsonrpc":"2.0","id":1,"result":{}}"#,
br#"{"jsonrpc":"2.0","id":1,"error":{"code":1,"message":"x"}}"#,
br#"{"jsonrpc":"2.0","id":7}"#,
br#"{"jsonrpc":"2.0","id":7,"method":5}"#,
br#"{"id":8,"method":"ping"}"#,
br#"[]"#,
br#"42"#,
br#""text""#,
br#"null"#,
br#"{not json"#,
br#" "#,
b"",
] {
s.push(step(raw.to_vec()));
}
// batches: mixed requests and notifications, only notifications, bad members.
s.push(step(
json!([
{"jsonrpc": "2.0", "id": 400, "method": "ping"},
{"jsonrpc": "2.0", "method": "notifications/initialized"},
{"jsonrpc": "2.0", "id": 401, "method": "tools/list"},
5,
{"jsonrpc": "2.0", "id": 402, "method": "nope"}
])
.to_string()
.into_bytes(),
));
s.push(step(br#"[{"jsonrpc":"2.0","method":"notifications/initialized"}]"#.to_vec()));
// CRLF line ending, leading and trailing whitespace.
s.push(step(b" {\"jsonrpc\":\"2.0\",\"id\":500,\"method\":\"ping\"}\r".to_vec()));
s
}
/// The documented differences, removed from a reply before comparing. Anything else must match.
fn normalise(mut v: Value, free_text: bool) -> Value {
if let Value::Array(items) = v {
return Value::Array(items.into_iter().map(|i| normalise(i, free_text)).collect());
}
// 1. `capabilities` gains `listChanged: false` on tools and resources.
if let Some(c) = v.pointer_mut("/result/capabilities").and_then(Value::as_object_mut) {
for cap in c.values_mut() {
if let Some(o) = cap.as_object_mut() {
o.remove("listChanged");
}
}
}
// 2. Error texts are reworded (too long/large, non-UTF-8, unknown method, unknown resource);
// the codes are compared.
if let Some(e) = v.get_mut("error").and_then(Value::as_object_mut) {
let code = e.get("code").and_then(Value::as_i64).unwrap_or(0);
if [-32601, -32002].contains(&code) || e.get("message").and_then(Value::as_str).is_some_and(|m| m.contains("too ") || m.contains("UTF-8")) {
e.insert("message".into(), json!("<reworded>"));
}
}
// 3. Document ids come from a process-wide counter, so two sessions in one process differ.
if let Some(text) = v.pointer_mut("/result/content/0/text")
&& let Some(Ok(mut j)) = text.as_str().map(serde_json::from_str::<Value>)
{
zero_ids(&mut j);
*text = json!(j.to_string());
}
// 4. A tool call refused before the tool runs: the validator's own words.
if free_text && v.pointer("/result/isError") == Some(&json!(true)) {
v["result"]["content"] = json!("<validator text>");
}
v
}
fn zero_ids(v: &mut Value) {
match v {
Value::Object(o) => {
for (k, x) in o.iter_mut() {
if k == "id" && x.is_number() {
*x = json!(0);
} else {
zero_ids(x);
}
}
}
Value::Array(a) => a.iter_mut().for_each(zero_ids),
_ => {}
}
}
fn run_oracle(input: &[u8]) -> Vec<Value> {
let mut server = oracle::Server::new(Box::new(Headless::default()));
let mut out = Vec::new();
server.serve(std::io::Cursor::new(input.to_vec()), &mut out).unwrap();
lines(out)
}
fn run_shared(input: &[u8]) -> Vec<Value> {
let mut server = Server::new(Box::new(Headless::default()));
let mut out = Vec::new();
server.serve(std::io::Cursor::new(input.to_vec()), &mut out).unwrap();
lines(out)
}
fn lines(out: Vec<u8>) -> Vec<Value> {
String::from_utf8(out).unwrap().lines().map(|l| serde_json::from_str(l).unwrap_or_else(|e| panic!("{e}: {l}"))).collect()
}
/// Both servers answer a line at a time, so each step is compared on its own reply (or lack of one).
fn compare(steps: &[Step]) {
let mut old = oracle::Server::new(Box::new(Headless::default()));
let mut new = Server::new(Box::new(Headless::default()));
for (i, s) in steps.iter().enumerate() {
let text = String::from_utf8_lossy(&s.line).into_owned();
let (a, b) = (old.handle_line(&text), new.handle_line(&text));
let parse = |r: Option<String>| r.map(|r| normalise(serde_json::from_str::<Value>(&r).unwrap(), s.free_text));
assert_eq!(parse(b), parse(a), "step {i}: {text}");
}
assert_eq!(old.is_initialized(), new.is_initialized());
}
#[test]
fn shared_server_answers_the_scripted_session_like_the_old_one() {
compare(&script());
}
/// The same session as a byte stream through `serve`, with an oversize line and a line that is
/// not UTF-8 in the middle: same replies, same order.
#[test]
fn serve_matches_the_old_server_with_an_oversize_and_a_non_utf8_line() {
let mut input = Vec::new();
for (i, s) in script().iter().enumerate() {
if i == 20 {
input.extend(vec![b'a'; (1 << 20) + 1]);
input.push(b'\n');
input.extend_from_slice(b"{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"ping\",\"x\":\"\xff\xfe\"}\n");
// Exactly at the limit is accepted (and is a parse error); one over is refused.
input.extend(vec![b' '; 1 << 20]);
input.push(b'\n');
input.extend(vec![b'{'; 1 << 20]);
input.push(b'\n');
}
input.extend_from_slice(&s.line);
input.push(b'\n');
}
// An unterminated last line still counts.
input.extend_from_slice(br#"{"jsonrpc":"2.0","id":9,"method":"ping"}"#);
let (old, new) = (run_oracle(&input), run_shared(&input));
let norm = |v: Vec<Value>| v.into_iter().map(|r| normalise(r, false)).collect::<Vec<_>>();
// The script's free-text failures are only compared in `compare`; drop their texts here.
let strip = |v: Vec<Value>| -> Vec<Value> {
v.into_iter()
.map(|mut r| {
if r.pointer("/result/isError") == Some(&json!(true))
&& r["result"]["content"][0]["text"].as_str().is_some_and(|t| !t.starts_with('{') && !t.starts_with('['))
{
r["result"]["content"] = json!("<text>");
}
r
})
.collect()
};
let (old, new) = (strip(norm(old)), strip(norm(new)));
assert_eq!(new.len(), old.len());
for (i, (n, o)) in new.iter().zip(&old).enumerate() {
assert_eq!(n, o, "reply {i}");
}
}
/// Difference 1 (accepted): the oversize and non-UTF-8 replies keep their codes, the words change.
#[test]
fn refused_lines_keep_their_codes_and_say_why() {
let mut input = vec![b'a'; (1 << 20) + 1];
input.extend_from_slice(b"\n\xff\n");
let old = run_oracle(&input);
let new = run_shared(&input);
assert_eq!(old[0]["error"]["code"], -32600);
assert_eq!(new[0]["error"]["code"], -32600);
assert_eq!(old[0]["error"]["message"], "request line too long");
assert_eq!(new[0]["error"]["message"], "request too large");
assert_eq!(old[1]["error"]["code"], -32700);
assert_eq!(new[1]["error"]["code"], -32700);
assert!(new[1]["error"]["message"].as_str().unwrap().contains("UTF-8"));
assert_eq!(new[0]["id"], Value::Null);
}
/// Difference 2 (accepted): a request with no `uri` is "unknown resource" -32002, not -32602
/// (the shared core hands the tools an empty uri); an unknown uri keeps -32002.
#[test]
fn resources_read_without_a_uri_is_not_found() {
let mut old = oracle::Server::new(Box::new(Headless::default()));
let mut new = Server::new(Box::new(Headless::default()));
let line = req(json!(1), "resources/read", json!({}));
let line = String::from_utf8(line).unwrap();
let o: Value = serde_json::from_str(&old.handle_line(&line).unwrap()).unwrap();
let n: Value = serde_json::from_str(&new.handle_line(&line).unwrap()).unwrap();
assert_eq!(o["error"]["code"], -32602);
assert_eq!(n["error"]["code"], -32002);
}
/// Difference 3 (accepted): the shared core serves `prompts/list` (empty) where the old server
/// said "method not found"; WordCraft has no prompts, and `prompts/get` stays not found.
#[test]
fn prompts_list_is_empty_and_prompts_get_is_not_found() {
let mut new = Server::new(Box::new(Headless::default()));
let r = new.handle(json!({"jsonrpc": "2.0", "id": 1, "method": "prompts/list"})).unwrap();
assert_eq!(r["result"], json!({"prompts": []}));
let r = new.handle(json!({"jsonrpc": "2.0", "id": 2, "method": "prompts/get", "params": {"name": "x"}})).unwrap();
assert_eq!(r["error"]["code"], -32601);
// And capabilities do not advertise prompts or completions.
let r = new.handle(json!({"jsonrpc": "2.0", "id": 3, "method": "initialize", "params": {}})).unwrap();
assert_eq!(r["result"]["capabilities"], json!({"tools": {"listChanged": false}, "resources": {"listChanged": false}}));
}
/// Difference 4 (accepted hardening): a batch of more than 1000 messages is refused whole; the old
/// server answered every member.
#[test]
fn batches_are_capped_at_a_thousand() {
let batch = |n: usize| Value::Array((0..n).map(|i| json!({"jsonrpc": "2.0", "id": i, "method": "ping"})).collect()).to_string();
let mut old = oracle::Server::new(Box::new(Headless::default()));
let mut new = Server::new(Box::new(Headless::default()));
let (a, b) = (old.handle_line(&batch(1000)).unwrap(), new.handle_line(&batch(1000)).unwrap());
assert_eq!(a, b);
let a: Value = serde_json::from_str(&old.handle_line(&batch(1001)).unwrap()).unwrap();
let b: Value = serde_json::from_str(&new.handle_line(&batch(1001)).unwrap()).unwrap();
assert_eq!(a.as_array().map(Vec::len), Some(1001));
assert_eq!(b["error"]["code"], -32600);
}
/// Difference 5 (accepted, stricter): arguments the tool's schema does not list, or of another
/// type, are a tool error before the tool runs. The old server passed them on and ignored them.
#[test]
fn unknown_and_mistyped_arguments_are_tool_errors() {
let mut new = Server::new(Box::new(Headless::default()));
for (name, args, needle) in [
("type_text", json!({"text": "x", "bogus": 1}), "unknown argument `bogus`"),
("get_text", json!({"x": 1}), "unknown argument `x`"),
("type_text", json!({"text": 5}), "must be of type"),
("select_text", json!({"text": "x", "occurrence": "1"}), "must be of type"),
] {
let r = new.handle(json!({"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": name, "arguments": args}})).unwrap();
assert_eq!(r["result"]["isError"], true, "{r}");
assert!(r["result"]["content"][0]["text"].as_str().unwrap().contains(needle), "{r}");
assert!(r.get("error").is_none(), "a tool failure is a result, not a JSON-RPC error: {r}");
}
}
/// Difference 6 (accepted, stricter): a required argument that is missing is refused before the
/// tool runs, where `type_text` and `select_text` used to treat a missing `text` as "". The
/// schemas have always listed `text` as required, so the call was never valid.
#[test]
fn a_missing_required_argument_is_refused_where_the_tool_defaulted_it() {
let mut old = oracle::Server::new(Box::new(Headless::default()));
let mut new = Server::new(Box::new(Headless::default()));
for name in ["type_text", "select_text"] {
let line = String::from_utf8(req(json!(1), "tools/call", json!({"name": name, "arguments": {}}))).unwrap();
let o: Value = serde_json::from_str(&old.handle_line(&line).unwrap()).unwrap();
let n: Value = serde_json::from_str(&new.handle_line(&line).unwrap()).unwrap();
// type_text {} used to type nothing and succeed; select_text {} failed to find "".
assert_eq!(o["result"]["isError"], name == "select_text", "{name}: {o}");
assert_eq!(n["result"]["isError"], true, "{name}: {n}");
assert!(n["result"]["content"][0]["text"].as_str().unwrap().contains("missing required argument `text`"), "{n}");
}
}
/// Every argument the tools read, and every one the documentation shows, passes the check: the
/// schemas list what the tools accept, so no call that worked now fails.
#[test]
fn every_documented_call_passes_the_argument_check() {
use craft_mcp_server::check_arguments;
let defs = crate::tools::tool_definitions();
let defs = defs.as_array().unwrap();
for (name, args) in [
("list_commands", json!({"query": "x"})),
("execute", json!({"command": "insert.table", "params": {"rows": 3, "cols": 4}, "doc": 3})),
("insert_table", json!({"values": [["Month", "Cases"], ["January", 12]], "header": true, "style": "TableGrid", "doc": 1})),
("batch", json!({"commands": [{"command": "a"}], "keepGoing": true, "doc": 0})),
("new_document", json!({"template": "letter"})),
("open_document", json!({"path": "a.docx", "password": "p"})),
("save_document", json!({"path": "a.pdf", "doc": 2})),
("type_text", json!({"text": "x", "paragraphs": true, "doc": 2})),
("select_text", json!({"text": "x", "occurrence": 2, "doc": 2})),
("get_text", json!({"doc": 2})),
("inspect_document", json!({"doc": 2})),
("render_page", json!({"page": 1, "scale": 0.5, "doc": 2})),
("list_documents", json!({})),
("parity", json!({})),
("screenshot", json!({})),
("click", json!({"x": 1, "y": 2, "count": 2, "button": "right", "shift": true, "alt": true, "cmd": true, "ctrl": true, "command": true})),
("key", json!({"key": "B", "shift": true, "alt": true, "cmd": true, "ctrl": true, "command": true})),
("ui_inspect", json!({})),
] {
assert_eq!(check_arguments(defs, name, &args), Ok(()), "{name}");
}
// Every tool is covered above.
assert_eq!(defs.len(), 18);
}
/// Hostile and random input never panics or breaks the stream: every reply is one JSON line.
#[test]
fn random_lines_never_panic() {
let mut x: u64 = 0x9e37_79b9_7f4a_7c15;
let mut next = move || {
x ^= x << 13;
x ^= x >> 7;
x ^= x << 17;
x
};
let valid: Vec<Vec<u8>> = script().into_iter().map(|s| s.line).filter(|l| !l.is_empty()).collect();
let toks: [&[u8]; 14] =
[b"{", b"}", b"[", b"]", b"\"", b":", b",", b"null", b"\"method\"", b"\"id\"", b"\"params\"", b"\"tools/call\"", b"\"arguments\"", b"\xff"];
let mut input = Vec::new();
for _ in 0..1500 {
let mut line = match next() % 3 {
// Random bytes.
0 => (0..next() % 64).map(|_| (next() & 0xff) as u8).collect(),
// JSON-ish tokens.
1 => (0..next() % 20).flat_map(|_| toks[(next() % 14) as usize].to_vec()).collect(),
// A valid request, mutated: bytes flipped, cut, or doubled.
_ => {
let mut l = valid[(next() % valid.len() as u64) as usize].clone();
for _ in 0..next() % 4 {
if l.is_empty() {
break;
}
let at = (next() % l.len() as u64) as usize;
match next() % 3 {
0 => l[at] = (next() & 0xff) as u8,
1 => l.truncate(at),
_ => l.insert(at, toks[(next() % 14) as usize][0]),
}
}
l
}
};
line.retain(|&b| b != b'\n');
// Keep random lines from naming tools that touch the disk.
if String::from_utf8_lossy(&line).contains("save_document") || String::from_utf8_lossy(&line).contains("open_document") {
continue;
}
input.extend(line);
input.push(b'\n');
}
let replies = run_shared(&input);
for r in replies {
assert!(r.is_object() || r.is_array(), "{r}");
}
}
+174
View File
@@ -0,0 +1,174 @@
//! The hand-written MCP server core WordCraft had before it moved to `craft_mcp_server`, kept
//! verbatim as the oracle the differential test compares the shared server against.
use std::io::{BufRead, Write};
use serde_json::{Value, json};
use crate::backend::Backend;
use crate::server::{BoundedLine, MAX_REQUEST_LINE, read_bounded_line};
use crate::server::{COMMANDS_URI, DOC_URI};
use crate::tools::{call_tool, tool_definitions};
/// The MCP revision we implement.
pub(crate) const PROTOCOL_VERSION: &str = "2025-06-18";
const SUPPORTED_VERSIONS: &[&str] = &[PROTOCOL_VERSION, "2025-03-26", "2024-11-05"];
const PARSE_ERROR: i64 = -32700;
const INVALID_REQUEST: i64 = -32600;
const METHOD_NOT_FOUND: i64 = -32601;
const INVALID_PARAMS: i64 = -32602;
const INTERNAL_ERROR: i64 = -32603;
const RESOURCE_NOT_FOUND: i64 = -32002;
const INSTRUCTIONS: &str = "WordCraft is a word processor (a Microsoft Word clone). Every action is a command: find ids and \
parameters with list_commands and run them with execute (or several with batch). Positions are \
{story:\"body\", path:[block,(row,cell,block)...], off:byteOffset}. Typical flow: new_document → type_text \
\"Hello\" → execute text.newParagraph → execute para.style {style:\"Heading 1\"} → select_text {text:\"Hello\"} → \
execute format.bold → inspect_document to verify → render_page to look at the result → save {path:\"out.docx\"}. \
Several documents can be open: new_document and open_document open another one and make it active; \
list_documents gives their ids, and `doc` on execute, batch and the other document tools works on one \
without switching.";
/// An MCP server bound to one backend.
pub(crate) struct Server {
backend: Box<dyn Backend>,
initialized: bool,
}
fn response(id: Value, result: Value) -> Value {
json!({"jsonrpc": "2.0", "id": id, "result": result})
}
fn error(id: Value, code: i64, message: impl Into<String>) -> Value {
json!({"jsonrpc": "2.0", "id": id, "error": {"code": code, "message": message.into()}})
}
impl Server {
pub fn new(backend: Box<dyn Backend>) -> Self {
Self { backend, initialized: false }
}
/// Whether the client has sent `notifications/initialized`.
pub fn is_initialized(&self) -> bool {
self.initialized
}
/// Serve newline-delimited JSON-RPC until `input` closes. Logs go to stderr only (stdout is
/// the protocol stream).
pub fn serve(&mut self, mut input: impl BufRead, mut output: impl Write) -> std::io::Result<()> {
loop {
let reply = match read_bounded_line(&mut input, MAX_REQUEST_LINE) {
Ok(BoundedLine::Eof) => break,
Ok(BoundedLine::Line(line)) => self.handle_line(&line),
Ok(BoundedLine::TooLong) => Some(error(Value::Null, INVALID_REQUEST, "request line too long").to_string()),
Err(e) if e.kind() == std::io::ErrorKind::InvalidData => {
Some(error(Value::Null, PARSE_ERROR, "request line is not UTF-8").to_string())
}
Err(e) => return Err(e),
};
if let Some(reply) = reply {
output.write_all(reply.as_bytes())?;
output.write_all(b"\n")?;
output.flush()?;
}
}
Ok(())
}
/// Handle one line; returns the reply line (None for notifications and blank lines).
pub fn handle_line(&mut self, line: &str) -> Option<String> {
let line = line.trim();
if line.is_empty() {
return None;
}
let reply = match serde_json::from_str::<Value>(line) {
Ok(Value::Array(batch)) => {
// Batches were removed in 2025-06-18; still answer older clients sensibly.
if batch.is_empty() {
Some(error(Value::Null, INVALID_REQUEST, "empty batch"))
} else {
let replies: Vec<Value> = batch.into_iter().filter_map(|m| self.handle(m)).collect();
(!replies.is_empty()).then_some(Value::Array(replies))
}
}
Ok(msg) => self.handle(msg),
Err(e) => Some(error(Value::Null, PARSE_ERROR, format!("parse error: {e}"))),
};
reply.map(|r| r.to_string())
}
/// Handle one JSON-RPC message; `None` for notifications and responses.
pub fn handle(&mut self, msg: Value) -> Option<Value> {
let Value::Object(o) = &msg else { return Some(error(Value::Null, INVALID_REQUEST, "message must be an object")) };
let id = o.get("id").cloned();
let Some(method) = o.get("method").and_then(Value::as_str) else {
// A response to a server→client request (we send none): ignore. Anything else is invalid.
if o.contains_key("result") || o.contains_key("error") {
return None;
}
return Some(error(id.unwrap_or(Value::Null), INVALID_REQUEST, "missing `method`"));
};
let params = o.get("params").cloned().unwrap_or(Value::Null);
let Some(id) = id else {
self.notification(method);
return None;
};
if !(id.is_string() || id.is_number()) {
return Some(error(Value::Null, INVALID_REQUEST, "`id` must be a string or number"));
}
Some(match self.request(method, &params) {
Ok(r) => response(id, r),
Err((code, m)) => error(id, code, m),
})
}
fn notification(&mut self, method: &str) {
match method {
"notifications/initialized" => self.initialized = true,
"notifications/cancelled" | "notifications/progress" | "notifications/roots/list_changed" => {}
other => log::debug!("ignoring notification {other}"),
}
}
fn request(&mut self, method: &str, params: &Value) -> Result<Value, (i64, String)> {
match method {
"initialize" => {
let asked = params.get("protocolVersion").and_then(Value::as_str).unwrap_or(PROTOCOL_VERSION);
let version = if SUPPORTED_VERSIONS.contains(&asked) { asked } else { PROTOCOL_VERSION };
Ok(json!({
"protocolVersion": version,
"capabilities": {"tools": {}, "resources": {}},
"serverInfo": {"name": "wordcraft", "title": "WordCraft", "version": env!("CARGO_PKG_VERSION"), "websiteUrl": "https://getartcraft.com/apps/wordcraft"},
"instructions": format!(
"{INSTRUCTIONS} Backend: {}. Community: https://discord.gg/artcraft · https://getartcraft.com/apps/wordcraft · https://github.com/storytold/wordcraft",
self.backend.describe()
),
}))
}
"ping" => Ok(json!({})),
"tools/list" => Ok(json!({"tools": tool_definitions()})),
"tools/call" => {
let name = params.get("name").and_then(Value::as_str).ok_or((INVALID_PARAMS, "missing tool `name`".to_string()))?;
let args = params.get("arguments").cloned().unwrap_or(Value::Null);
Ok(call_tool(self.backend.as_mut(), name, &args).to_value())
}
"resources/list" => Ok(json!({"resources": [
{"uri": DOC_URI, "name": "document", "title": "Active document", "description": "Blocks, text, formatting, sections, parts and selection of the document (document.inspect)", "mimeType": "application/json"},
{"uri": COMMANDS_URI, "name": "commands", "title": "Commands", "description": "Every command: id, label, ribbon location, shortcut, params, enabled (engine.commands)", "mimeType": "application/json"},
]})),
"resources/templates/list" => Ok(json!({"resourceTemplates": []})),
"resources/read" => {
let uri = params.get("uri").and_then(Value::as_str).ok_or((INVALID_PARAMS, "missing `uri`".to_string()))?;
let v = match uri {
DOC_URI => self.backend.call("document.inspect", json!({})),
COMMANDS_URI => self.backend.call("engine.commands", json!({})),
_ => return Err((RESOURCE_NOT_FOUND, format!("resource not found: {uri}"))),
}
.map_err(|e| (INTERNAL_ERROR, e))?;
Ok(json!({"contents": [{"uri": uri, "mimeType": "application/json", "text": serde_json::to_string_pretty(&v).unwrap_or_default()}]}))
}
other => Err((METHOD_NOT_FOUND, format!("method not found: {other}"))),
}
}
}
+9 -3
View File
@@ -145,7 +145,7 @@ pub fn tool_definitions() -> Value {
tool(
"inspect_document",
"Inspect document",
"Structure: blocks with text, styles, formatting runs, lists, tables, sections, parts, selection, pages.",
"Structure: blocks with text, styles, formatting runs, lists, tables, sections, parts (headers, footers, notes: their text and fields; a field with perPage true, such as PAGE, NUMPAGES or STYLEREF, shows its own value on each page, so its stored result is not what every page shows: render_page shows it; fieldsTruncated means a part has more than the 200 fields listed; a range field's result, such as a TOC's, is cut at 2000 characters, with resultTruncated), selection, pages.",
with_doc(obj(json!({}), &[])),
true
),
@@ -169,14 +169,20 @@ pub fn tool_definitions() -> Value {
"click",
"Click",
"Click at window coordinates (desktop app only).",
obj(json!({"x": n("x"), "y": n("y"), "count": n("1-3 clicks"), "shift": {"type": "boolean"}, "cmd": {"type": "boolean"}}), &["x", "y"]),
obj(
json!({"x": n("x"), "y": n("y"), "count": n("1-3 clicks"), "button": s("left (default) or right"), "shift": {"type": "boolean"}, "alt": {"type": "boolean"}, "cmd": {"type": "boolean"}, "ctrl": {"type": "boolean"}, "command": {"type": "boolean"}}),
&["x", "y"]
),
false
),
tool(
"key",
"Press key",
"Press a key with modifiers (desktop app only), e.g. {key:\"B\", cmd:true}.",
obj(json!({"key": s("Key name"), "shift": {"type": "boolean"}, "alt": {"type": "boolean"}, "cmd": {"type": "boolean"}}), &["key"]),
obj(
json!({"key": s("Key name"), "shift": {"type": "boolean"}, "alt": {"type": "boolean"}, "cmd": {"type": "boolean"}, "ctrl": {"type": "boolean"}, "command": {"type": "boolean"}}),
&["key"]
),
false
),
tool(
+2 -1
View File
@@ -9,7 +9,8 @@ use super::{
/// Lay out `doc` and write it as PDF. Without markup the layout is the final text: tracked
/// deletions are left out, as in Word's "No Markup" view. With `comments` each page carries the
/// markup area with the comment balloons.
/// markup area with the comment balloons. Page references in the body show their bookmarks'
/// pages, as Word's export fills them in.
pub fn export(doc: &Document, opts: &PdfOptions) -> Result<Vec<u8>, PdfError> {
write::guard(|| {
let lay = layout(doc, &mut LayoutCache::new(), &opts.layout_options());
+9 -1
View File
@@ -50,6 +50,10 @@ pub struct PdfOptions {
/// How [`export`] shows digits (the Numeral setting; callers resolve `System`, see
/// [`NumeralMode::resolve`]). [`export_layout`] keeps the layout it is given.
pub numeral: NumeralMode,
/// What [`export`] shows for a page reference to a bookmark the document doesn't have, in the
/// interface language (`None`: Word's English "Error! Bookmark not defined."). [`export_layout`]
/// keeps the layout it is given.
pub bookmark_error: Option<std::sync::Arc<str>>,
}
/// How many characters of the watermark text the PDF draws.
@@ -70,12 +74,15 @@ impl PdfOptions {
}
}
/// How [`crate::export`] lays a document out for these options.
/// How [`crate::export`] lays a document out for these options. Page references in the body
/// show their bookmarks' pages, as Word's export fills them in.
pub fn layout_options(&self) -> wordcraft_layout::LayoutOptions {
wordcraft_layout::LayoutOptions {
hide_deleted: !self.include_markup,
comment_balloons: self.comments,
numeral: self.numeral,
output_page_refs: true,
bookmark_error: self.bookmark_error.clone(),
..Default::default()
}
}
@@ -98,6 +105,7 @@ impl Default for PdfOptions {
links: true,
paper: None,
numeral: NumeralMode::Arabic,
bookmark_error: None,
}
}
}
+24
View File
@@ -390,6 +390,7 @@ fn cost_ratio(scale: usize, mut small: impl FnMut(), mut big: impl FnMut()) -> (
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn long_fragmented_hidden_headings_export_in_linear_time() {
// Headings of n chars in one-char runs: all hidden with bold alternating, and hidden and visible
// alternating. Layout and the bookmark/tag titles once scanned every left-out range for every
@@ -1917,6 +1918,7 @@ fn comments_print_in_balloons_beside_scaled_pages() {
/// references) exports in time proportional to its length: each link looked its bookmark up by
/// walking the whole document, so NIH's 67-page protocol template took 10.6 s (0.9 s now).
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn internal_links_export_in_linear_time() {
let doc = |n: usize| {
let mut d = Document::new();
@@ -3121,6 +3123,7 @@ fn only_safe_link_targets_become_annotations() {
/// without one), found in one pass: a long run of glyphs sharing a cluster used to rescan the rest
/// of the run for every glyph.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn cluster_ends_match_the_next_larger_cluster_in_linear_time() {
let brute = |cl: &[usize], len: usize| -> Vec<usize> {
(0..cl.len()).map(|i| cl.iter().skip(i + 1).copied().find(|n| *n > cl[i]).unwrap_or(len)).collect::<Vec<_>>()
@@ -3153,6 +3156,27 @@ fn baseline_is_the_most_shared_y() {
assert_eq!(baseline(&[(1, 0.0, -0.0), (2, 3.0, 0.0)]), Some(0.0));
}
/// The exported PDF shows a body page reference's page and Word's error for a missing bookmark,
/// as Word's export does, where the document stores "9" for both.
#[test]
fn exported_page_references_show_the_page_or_words_error() {
let field =
|instr: &str| InlineObject::Field { instr: instr.into(), result: "9".into(), locked: false, dirty: false, code: false, form: String::new() };
let mut d = Document::from_text("See \nMissing \nTarget");
for (i, instr) in [(0, "PAGEREF Target"), (1, "PAGEREF NoSuchMark")] {
let p = d.para_mut(wordcraft_doc::StoryRef::Body, &wordcraft_doc::Path::top(i)).unwrap();
let n = p.len();
p.insert_object(n, field(instr), &CharProps::default()).unwrap();
}
let target = d.para_mut(wordcraft_doc::StoryRef::Body, &wordcraft_doc::Path::top(2)).unwrap();
target.props.page_break_before = Some(true);
target.insert_object(0, InlineObject::BookmarkStart { name: "Target".into() }, &CharProps::default()).unwrap();
let text = extract_text(&export(&d, &PdfOptions::default()).unwrap());
let first = squash(&text[0]);
assert!(first.contains("See 2"), "{first}");
assert!(first.contains("Missing Error! Bookmark not defined."), "{first}");
}
#[test]
fn the_author_is_left_out_when_personal_information_is_removed() {
let mut d = Document::from_text("Private");
+88 -5
View File
@@ -880,11 +880,18 @@ fn draw_image(ctx: &mut RenderContext, fr: &Frame, it: &Draw) {
draw_vector(ctx, &v, &r, crop, *alpha, view);
return;
}
let Some(pm) = pixmap_for(doc, media, *fx) else {
ctx.set_transform(view);
ctx.set_paint(color(fr.opts.ink(Rgb(0xD0, 0xD0, 0xD0)), 1.0));
ctx.fill_rect(&r);
return;
let pm = match pixmap_for(doc, media, *fx) {
Ok(pm) => pm,
Err(refusal) => {
ctx.set_transform(view);
ctx.set_paint(color(fr.opts.ink(Rgb(0xD0, 0xD0, 0xD0)), 1.0));
ctx.fill_rect(&r);
// Say why when the picture is fine but too large to decode (#5637).
if let wordcraft_geom::picture::Refusal::TooLarge { width, height } = refusal {
draw_box_label(ctx, fr, view, &r, &too_large_label(width, height));
}
return;
}
};
let (pw, ph) = (pm.width().max(1) as f64, pm.height().max(1) as f64);
let [cl, ct, vis_w, vis_h] = wordcraft_layout::crop_window(*crop);
@@ -921,6 +928,82 @@ fn path_extent_ok(view: Affine, path: &kurbo::BezPath) -> bool {
path.is_finite() && [b.x0, b.y0, b.x1, b.y1].iter().all(|v| v.is_finite() && v.abs() <= MAX_PATH_PIXELS)
}
/// A label's lines, shaped once: each glyph with its face, and each line's width at 1 pt.
struct BoxLabel {
lines: Vec<(Vec<(wordcraft_fonts::FaceRef, u32)>, f64)>,
widest: f64,
}
/// Shapes `lines` in the UI face (with fallbacks).
fn shape_box_label(lines: &[&str]) -> BoxLabel {
let face = wordcraft_fonts::word::resolve("Source Sans 3", false, false).face;
let db = FontDb::global();
let lines: Vec<_> = lines
.iter()
.map(|line| {
let glyphs: Vec<_> = line
.chars()
.map(|ch| {
let f = if face.glyph_for(ch) == 0 {
db.fallback_for(ch, face.id()).map_or(face, |f| wordcraft_fonts::FaceRef::of(&f))
} else {
face
};
(f, f.glyph_for(ch))
})
.collect();
let width = glyphs.iter().map(|(f, gid)| f.advance(*gid) / f.upem.max(1.0)).sum();
(glyphs, width)
})
.collect();
let widest = lines.iter().map(|(_, w)| *w).fold(0.0, f64::max);
BoxLabel { lines, widest }
}
/// The message in the box of a picture too large to decode, shaped once per picture size (pages
/// are redrawn often).
fn too_large_label(width: u32, height: u32) -> Arc<BoxLabel> {
static C: std::sync::OnceLock<Mutex<HashMap<(u32, u32), Arc<BoxLabel>>>> = std::sync::OnceLock::new();
let cache = C.get_or_init(|| Mutex::new(HashMap::new()));
if let Some(l) = lock(cache).get(&(width, height)) {
return l.clone();
}
let size = format!("{width} × {height} pixels");
let label = Arc::new(shape_box_label(&["Picture too large to display", &size, "Resize it to show it"]));
let mut c = lock(cache);
// A handful of refused pictures at most; start over rather than grow without bound.
if c.len() >= 64 {
c.clear();
}
c.insert((width, height), label.clone());
label
}
/// Writes `label` centred in the box `r` (drawn under `view`), at most 10 pt and shrunk to fit;
/// nothing when even 3 pt text wouldn't fit.
fn draw_box_label(ctx: &mut RenderContext, fr: &Frame, view: Affine, r: &kurbo::Rect, label: &BoxLabel) {
let db = FontDb::global();
let pad = 4.0;
let n = label.lines.len() as f64;
let size = 10.0f64.min((r.width() - 2.0 * pad) / label.widest.max(1e-9)).min((r.height() - 2.0 * pad) / (1.2 * n).max(1e-9));
if !(size.is_finite() && size >= 3.0) {
return;
}
ctx.set_paint(color(fr.opts.ink(Rgb(0x50, 0x50, 0x50)), 1.0));
let top = r.center().y - 1.2 * size * n / 2.0;
for (i, (glyphs, w)) in label.lines.iter().enumerate() {
let baseline = top + 1.2 * size * (i as f64 + 0.8);
let mut x = r.center().x - w * size / 2.0;
for (f, gid) in glyphs {
let k = size / f.upem.max(1.0);
ctx.set_transform(view * Affine::translate((x, baseline)) * Affine::scale(k));
ctx.fill_path(&db.outline(f, *gid));
x += f.advance(*gid) * k;
}
}
ctx.set_transform(view);
}
fn draw_path(ctx: &mut RenderContext, fr: &Frame, it: &Draw) {
let Draw::Path { segs, fill, stroke, stroke_width, dash, cap } = it else { return };
let path = wordcraft_layout::display::seg_path(segs);
+20 -9
View File
@@ -1,3 +1,5 @@
use wordcraft_geom::picture::Refusal;
use super::{Affine, Arc, Document, FontDb, HashMap, Item, Mutex, Page, Picture, Pixmap, PlacedItem, RenderContext, Seg, Shape, Weak, kurbo, peniko};
use craft_metafile::{ClipPath, Tile};
use std::collections::HashSet;
@@ -51,8 +53,11 @@ pub(crate) fn lock<T>(m: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
m.lock().unwrap_or_else(std::sync::PoisonError::into_inner)
}
fn images() -> &'static Mutex<MediaCache<Option<Arc<Pixmap>>>> {
static C: std::sync::OnceLock<Mutex<MediaCache<Option<Arc<Pixmap>>>>> = std::sync::OnceLock::new();
/// A decoded picture, or why it isn't drawn.
pub(crate) type Decoded = Result<Arc<Pixmap>, Refusal>;
fn images() -> &'static Mutex<MediaCache<Decoded>> {
static C: std::sync::OnceLock<Mutex<MediaCache<Decoded>>> = std::sync::OnceLock::new();
C.get_or_init(|| Mutex::new(MediaCache { map: HashMap::new(), bytes: 0 }))
}
@@ -75,11 +80,17 @@ pub fn decode_pixmap(bytes: &[u8]) -> Option<Pixmap> {
/// [`decode_pixmap`] with a picture's brightness and contrast applied (#1711, #1766), under the
/// same decode limits.
pub fn decode_pixmap_fx(bytes: &[u8], fx: wordcraft_doc::picfx::PicFx) -> Option<Pixmap> {
decode_pixmap_checked(bytes, fx).ok()
}
/// [`decode_pixmap_fx`], with why a picture isn't decoded (#5637).
fn decode_pixmap_checked(bytes: &[u8], fx: wordcraft_doc::picfx::PicFx) -> Result<Pixmap, Refusal> {
// The shared probe also charges what the decoder holds beside its output (#3644).
if !wordcraft_geom::picture::probe(bytes).is_ok_and(|p| p.fits(wordcraft_geom::MAX_DECODED_BYTES)) {
return None;
let probe = wordcraft_geom::picture::probe(bytes)?;
if !probe.fits(wordcraft_geom::MAX_DECODED_BYTES) {
return Err(Refusal::TooLarge { width: probe.width, height: probe.height });
}
decode_pixmap_within(bytes, fx, wordcraft_geom::MAX_DECODED_BYTES)
decode_pixmap_within(bytes, fx, wordcraft_geom::MAX_DECODED_BYTES).ok_or_else(|| Refusal::Broken("it could not be decoded".into()))
}
/// [`decode_pixmap_fx`] with an explicit RGBA byte budget (a small one in tests).
@@ -534,7 +545,7 @@ pub(crate) fn draw_page_picture(ctx: &mut RenderContext, doc: &Document, page: &
let Some(pic) = &doc.settings.page_picture else { return };
let Some(bytes) = doc.media.get(&pic.media) else { return };
let Some((nw, nh)) = wordcraft_geom::picture::background_size(bytes).map(|(w, h)| (f64::from(w), f64::from(h))) else { return };
let Some(pm) = pixmap_for(doc, &pic.media, wordcraft_doc::picfx::PicFx::default()) else { return };
let Ok(pm) = pixmap_for(doc, &pic.media, wordcraft_doc::picfx::PicFx::default()) else { return };
let (pw, ph) = (pm.width().max(1) as f64, pm.height().max(1) as f64);
let (w, h) = (page.w as f64, (page.h as f64).min(1e6));
if !(w.is_finite() && h.is_finite() && w > 0.0 && h > 0.0) {
@@ -568,8 +579,8 @@ pub(crate) fn draw_page_picture(ctx: &mut RenderContext, doc: &Document, page: &
ctx.reset_paint_transform();
}
pub(crate) fn pixmap_for(doc: &Document, key: &str, fx: wordcraft_doc::picfx::PicFx) -> Option<Arc<Pixmap>> {
let bytes = doc.media.get(key)?;
pub(crate) fn pixmap_for(doc: &Document, key: &str, fx: wordcraft_doc::picfx::PicFx) -> Decoded {
let Some(bytes) = doc.media.get(key) else { return Err(Refusal::NotAPicture) };
// Corrected pixels are cached apart from the picture's own (the same picture can be shown
// both ways).
let fx = fx.clamped();
@@ -578,7 +589,7 @@ pub(crate) fn pixmap_for(doc: &Document, key: &str, fx: wordcraft_doc::picfx::Pi
if let Some(v) = lock(images()).get(&cache_key, bytes) {
return v;
}
let pm = decode_pixmap_fx(bytes, fx).map(|p| Arc::new(cacheable(p)));
let pm = decode_pixmap_checked(bytes, fx).map(|p| Arc::new(cacheable(p)));
let size = pm.as_ref().map_or(0, |p| p.width() as usize * p.height() as usize * 4);
lock(images()).insert(&cache_key, bytes, pm.clone(), size);
pm
+79
View File
@@ -2100,3 +2100,82 @@ fn geometry_bytes_counts_dash_arrays_and_text_records() {
let empty = crate::media::geometry_bytes(&pic(vec![]));
assert!(bare - empty >= std::mem::size_of::<Text>(), "the boxed Text struct is charged: {} bytes", bare - empty);
}
/// The placeholder for a picture `bytes` shown 200 × 150 pt, one row per pixel row: `Some(true)`
/// for dark (text) pixels, `Some(false)` for the grey fill, `None` for anything else.
fn placeholder_pixels(bytes: Vec<u8>) -> Vec<Vec<Option<bool>>> {
let mut d = Document::new();
let media = d.add_media(bytes, "png");
let mut p = wordcraft_doc::Paragraph::new();
let obj = wordcraft_doc::para::InlineObject::Image {
media,
w: 200.0,
h: 150.0,
alt: String::new(),
float: Default::default(),
crop: [0.0; 4],
fx: Default::default(),
ole: None,
};
p.insert_object(0, obj, &Default::default()).unwrap();
d.body = vec![wordcraft_doc::para_block(p)];
let l = layout(&d, &mut LayoutCache::new(), &LayoutOptions::default());
let opts = RenderOptions::default();
let rect = page_display(&d, &l.pages[0], &opts.display)
.iter()
.find_map(|i| match i {
Draw::Image { rect, .. } => Some(*rect),
_ => None,
})
.expect("a picture on the page");
let img = render_page(&d, &l.pages[0], 1.0, &opts);
((rect.y.ceil() as u32)..((rect.y + rect.h).floor() as u32))
.map(|y| {
((rect.x.ceil() as u32)..((rect.x + rect.w).floor() as u32))
.map(|x| {
let px = img.pixel(x, y);
if px[..3] == [0xD0; 3] {
Some(false)
} else if px[..3].iter().all(|c| *c < 0x90) {
Some(true)
} else {
None
}
})
.collect()
})
.collect()
}
/// #5637: a picture too large to decode is not just a grey box: the box says why, in three
/// upright lines centred in it.
#[test]
fn a_picture_too_large_to_decode_says_so_in_its_box() {
let huge = ihdr_only_png(8333, 8333);
assert!(decode_pixmap(&huge).is_none(), "the decode budget refuses it");
let grid = placeholder_pixels(huge);
let (h, w) = (grid.len() as f64, grid[0].len() as f64);
assert!(grid.iter().flatten().any(|p| *p == Some(false)), "the grey placeholder is drawn");
let inked: Vec<bool> = grid.iter().map(|row| row.contains(&Some(true))).collect();
let rows: Vec<usize> = (0..inked.len()).filter(|y| inked[*y]).collect();
assert!(!rows.is_empty(), "the placeholder carries a message, not a plain grey box");
// Three lines of text: three bands of inked rows with blank rows between them.
let bands = (0..inked.len()).filter(|y| inked[*y] && (*y == 0 || !inked[*y - 1])).count();
assert_eq!(bands, 3, "three lines of text, inked rows {rows:?}");
// Centred both ways; glyphs flipped about their baselines would sit several pixels low.
let (top, bottom) = (rows[0] as f64, *rows.last().unwrap() as f64);
assert!(((top + bottom) / 2.0 - h / 2.0).abs() < 3.0, "centred vertically: rows {top}..{bottom} of {h}");
let cols: Vec<usize> = (0..grid[0].len()).filter(|x| grid.iter().any(|row| row[*x] == Some(true))).collect();
let (left, right) = (cols[0] as f64, *cols.last().unwrap() as f64);
assert!(((left + right) / 2.0 - w / 2.0).abs() < 3.0, "centred horizontally: columns {left}..{right} of {w}");
// The first line starts with the stem of an upright "P": its leftmost inked column runs at
// least 5 px down from the line's top.
let first = rows[0]..rows.iter().zip(&rows[1..]).find(|(a, b)| **b != **a + 1).map_or(rows.len(), |(a, _)| *a + 1);
let lead = (0..grid[0].len()).find(|x| grid[first.clone()].iter().any(|row| row[*x] == Some(true))).unwrap();
let stem = grid[first.clone()].iter().filter(|row| row[lead] == Some(true) || row.get(lead + 1) == Some(&Some(true))).count();
assert!(stem >= 5, "the first line starts with an upright P stem ({stem} px)");
// A damaged picture still gets its plain grey placeholder.
let grid = placeholder_pixels(b"\x89PNG\r\n\x1a\njunk".to_vec());
assert!(grid.iter().flatten().any(|p| *p == Some(false)), "a damaged picture keeps its grey placeholder");
assert!(!grid.iter().flatten().any(|p| *p == Some(true)), "with no size message");
}
+3 -3
View File
@@ -9,14 +9,14 @@ description = "WordCraft egui front end (Word-style ribbon UI); swappable: nothi
workspace = true
[dependencies]
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc" }
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a" }
# Interface translations: lookup, fallback, plurals and system-locale matching over the craft-i18n
# catalogs that `build.rs` embeds from `CRAFT_I18N_DIR` (storytold/craft-i18n, pinned in
# `craft-i18n.rev`). Same craft-libs rev as craft-ui; it brings lazy per-locale parsing
# (storytold/craft-libs#247), ICU4X plural rules and bidi isolates around inserted values.
craft-i18n = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc" }
craft-i18n = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a" }
# One native window per open document: the window list, the Window menu and the egui viewports.
craft-window = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", features = ["egui"] }
craft-window = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", features = ["egui"] }
wordcraft-geom = { workspace = true, features = ["image"] }
wordcraft-doc = { workspace = true }
wordcraft-fonts = { workspace = true }
+11 -6
View File
@@ -317,12 +317,14 @@ impl Builder<'_> {
let node = match (heading, list) {
(Some(l), _) => {
let mut n = Node::new(Role::Heading);
n.set_level(l as usize + 1);
// AccessKit counts levels from zero, as the outline level does: a Heading 1 is
// level 0, which the Windows adapter reports to UIA as level 1.
n.set_level(l as usize);
n
}
(None, Some(num)) => {
let mut n = Node::new(Role::ListItem);
n.set_level(num.level as usize + 1);
n.set_level(num.level as usize);
n
}
(None, None) => Node::new(Role::Paragraph),
@@ -677,6 +679,9 @@ mod tests {
("text.insert", json!({"text": "Title here"})),
("para.style", json!({"style": "Heading1"})),
("text.newParagraph", json!({})),
("text.insert", json!({"text": "Section here"})),
("para.style", json!({"style": "Heading2"})),
("text.newParagraph", json!({})),
("para.style", json!({"style": "Normal"})),
("text.insert", json!({"text": "Read this "})),
("insert.link", json!({"url": "https://example.com", "text": "link"})),
@@ -740,16 +745,16 @@ mod tests {
.collect::<Vec<_>>()
};
// AccessKit's levels are zero-based: Heading 1 is level 0, Heading 2 level 1.
let headings = of(Role::Heading);
assert_eq!(headings.len(), 1);
assert_eq!(nodes[&headings[0]].level(), Some(1));
assert_eq!(texts(Role::Heading), ["Title here\n"]);
assert_eq!(headings.iter().map(|h| nodes[h].level()).collect::<Vec<_>>(), [Some(0), Some(1)]);
assert_eq!(texts(Role::Heading), ["Title here\n", "Section here\n"]);
assert!(texts(Role::Paragraph).contains(&"Read this link\n".to_string()), "{:?}", texts(Role::Paragraph));
assert_eq!(texts(Role::Link), ["link"]);
assert_eq!(nodes[&of(Role::Link)[0]].url(), Some("https://example.com"));
assert_eq!(of(Role::List).len(), 1, "one list");
assert_eq!(texts(Role::ListItem), ["First item\n"]);
assert_eq!(nodes[&of(Role::ListItem)[0]].level(), Some(1));
assert_eq!(nodes[&of(Role::ListItem)[0]].level(), Some(0), "a first-level list item");
assert_eq!(of(Role::ListMarker).len(), 1, "the bullet");
assert_eq!(of(Role::Table).len(), 1);
assert_eq!(of(Role::Row).len(), 2);
+6
View File
@@ -110,6 +110,8 @@ impl WordApp {
integrated_titlebar: false,
mac_titlebar: chrome::MacTitleBar::default(),
updates_rt: updates_ui::Runtime::default(),
font_notice_seen: Vec::new(),
font_notices: false,
packs_rt: Default::default(),
control_rx: None,
shots: crate::control::Shots::default(),
@@ -249,6 +251,10 @@ impl WordApp {
crate::language_packs::tick(self);
let lang = i18n::Lang::from_pref(&self.ui.language);
i18n::set_current(lang);
// PDFs and prints show a page reference or a REF to a missing bookmark in the
// interface's language, as Word prints them in its own.
wordcraft_engine::set_bookmark_error(i18n::tr(lang, "Error! Bookmark not defined."));
wordcraft_engine::set_reference_error(i18n::tr(lang, "Error! Reference source not found."));
crate::rtl::sync(ctx);
// New documents are written in the interface's language when WordCraft can proof it
// (a Polish interface: Polish spelling, grammar and hyphenation from the first word).
+30 -18
View File
@@ -165,29 +165,12 @@ impl WordApp {
|| system_cjk != self.fonts_system_cjk
|| system_arabic != self.fonts_system_arabic
{
// Every window's borrowed faces: the fonts are the app's.
let mut faces = self.canvas.ime_faces.clone();
for w in self.other_windows() {
for face in &w.canvas.ime_faces {
if !faces.iter().any(|f| f.id() == face.id()) {
faces.push(face.clone());
}
}
}
theme::install_fonts_language_arabic(
ctx,
lang.prefers_hans(),
lang.prefers_korean(),
lang.prefers_hant(),
system_cjk,
system_arabic,
&faces,
);
self.fonts_hans = lang.prefers_hans();
self.fonts_korean = lang.prefers_korean();
self.fonts_hant = lang.prefers_hant();
self.fonts_system_cjk = system_cjk;
self.fonts_system_arabic = system_arabic;
self.apply_ui_fonts(ctx);
// Mod with -, = and 0 are Word shortcuts (optional hyphen, subscript, paragraph spacing);
// egui's keyboard zoom would also scale the whole window on them. Zoom is View › Zoom.
// egui quits on Command-Q, which is Ctrl+Q off the Mac: Word's Reset Paragraph
@@ -202,4 +185,33 @@ impl WordApp {
self.styled = true;
}
}
/// Install the interface fonts [`Self::install_ui_fonts`] chose, with every window's borrowed
/// faces and each window's composition face. The fonts are the app's, one set for every
/// window: fonts made from one window's faces would draw another window's caption or
/// comments as boxes, and its composition in the wrong face.
pub(crate) fn apply_ui_fonts(&self, ctx: &egui::Context) {
let mut fonts = theme::font_definitions_language_arabic(
self.fonts_hans,
self.fonts_korean,
self.fonts_hant,
self.fonts_system_cjk,
self.fonts_system_arabic,
);
// Every window's borrowed faces: the fonts are the app's.
let mut faces = self.canvas.ime_faces.clone();
for w in self.other_windows() {
for face in &w.canvas.ime_faces {
if !faces.iter().any(|f| f.id() == face.id()) {
faces.push(face.clone());
}
}
}
theme::add_fallback_faces(&mut fonts, &faces);
theme::set_composition_face(&mut fonts, self.windows.shown(), self.canvas.ime_face);
for (window, w) in self.other_windows_by_key() {
theme::set_composition_face(&mut fonts, window, w.canvas.ime_face);
}
ctx.set_fonts(fonts);
}
}

Some files were not shown because too many files have changed in this diff Show More