Compare commits

..
Author SHA1 Message Date
Martin BallandClaude Opus 5.5 9fd9a12957 Merge main into cutover/shared-codepage; craft-codepage at craft-libs main 93dac78c
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 20:56:27 +00:00
Martin BallandClaude Opus 5.5 1531007d5e #778: plain-text detection uses craft-mime detect (#6950)
The local is_text/decode/legacy copies in formats::txt are replaced by a re-export of
craft_mime::detect. The old code stays as a test oracle, with differential and random-buffer
tests showing identical output. chardetng stays only as a dev-dependency of that oracle.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:47:52 -05:00
30584c81ad #778: pin craft-libs main with the shared cutover prerequisites (#6954)
* #778: take craft-libs integration/wordcraft-778 at beeea924

Every craft-libs git dependency (workspace, ui-egui, wordcraft, wordcraft-web,
krilla patch) moves from 453b0da782 to beeea92451 (craft-libs main 0649a07aa
plus open PRs #2061-#2064). Cargo.lock: only craft-libs sources move, plus
windows-sys edge changes inside craft-libs crates; no crates.io versions change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: docx stacking-order test follows the shared reader's one-more z

craft-libs 5f453efa9 (craft-docx: a relativeHeight of 0 stays the lowest
stacking order) reads Float::z as one more than the file's relativeHeight and
writes stored orders one less, so a file's 0 is no longer "none". The test
expected the old values; relative order and the round trip are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: take craft-libs integration/wordcraft-778 at 24a15211

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* deps: pin craft-libs to the merged shared cutover prerequisites

* #778: pin craft-libs main at 93dac78c (with #2061-#2064)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-17 15:45:46 -05:00
Martin BallandClaude Opus 5.5 4bb5407a51 #778: Cargo.lock: wordcraft-doc depends on craft-codepage after merging main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 20:40:15 +00:00
Martin Ball 55f1a7857f Merge remote-tracking branch 'origin/main' into HEAD
; Conflicts:
;	Cargo.lock
2026-11-17 20:40:06 +00:00
Aleksander HelgakerandClaude Fable 5.2 8331f2e68d Fonts: tell where a missing font can be downloaded (Get the Missing Fonts) (#6826)
* Fonts: tell where a missing font can be downloaded (Get the Missing Fonts)

A document opened from a file that is missing fonts their publisher gives
away (Microsoft's Aptos family, Cascadia Code and Mono) brings up a box
naming each font, the font drawn instead, the publisher's download page and
how to install the download. Once per document; never for a new document or
one a script opened. "Don't show this again" turns it off (File > Options >
General turns it back on); the status bar's missing-fonts menu opens it by
hand (ui.dialog fontDownload).

font.missing reports the same page as "download", so agents get it without
the box. WordCraft links to the page only: nothing is downloaded or bundled.

The desktop and web apps switch the box on at start-up
(tell_about_font_downloads); tests and ui_shot don't, so what they show
doesn't depend on the fonts installed where they run.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Get the Missing Fonts: the box with nothing to download is one row tall

Its Close button's far-side layout took the rest of the window's height.
The button row is now a row of its own, shared by both forms of the box.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Floating windows: a flat title bar and room around the content

Dialogs, the Read Aloud player and the built-in file browser had a grey
band behind a centred title and 6 pt margins. The title bar is now the
window's own colour with no band or line under it, and the title and the
content have 18 pt of room. The close box follows the new title bar height.

egui fills the front window's title bar with the colour of an open menu
button, so show_window makes that colour the window's while the title bar
is drawn and puts it back before the content is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Get the Missing Fonts: a headline, a card to each download, numbered steps

The box was a plain table and a paragraph of steps. It now opens with the
warning mark and how many fonts are missing; lists each download once
(Aptos and Aptos Display come in one) with the fonts drawn instead, the
publisher and a Download button that opens the publisher's page; and gives
the two steps that add the download, with Add Font… beside its step.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Fonts: the download pages are a JSON list (font_downloads.json)

The list of fonts their publisher gives away, and each publisher's page,
was a table in the code. It is now crates/fonts/src/font_downloads.json,
built into the app: adding a font is a new entry there. Entries without a
family, a publisher or an https page are left out, and a list that isn't
JSON has no downloads rather than failing.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Get the Missing Fonts: licence notice, and fonts offered with limits

The box says that fonts come with licence terms and that following the
publisher's terms is up to the user. A font whose publisher limits who may
use it is marked "restricted" in font_downloads.json, says so on its card
and in font.missing ("restricted"), and has a card of its own.

The list gains Selawik (Microsoft's open-source font) and, marked
restricted, Segoe UI Variable, Segoe Fluent Icons and Segoe MDL2 Assets
from Microsoft's Windows design downloads page. Plain Segoe UI is not
offered for download and is not matched.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GBVQLv63guWCuWUM3SctPP

* Get the Missing Fonts: the downloads scroll in a short window

With many downloads in a short window the box ran past the bottom of the
screen, leaving the steps, Add Font…, Don't show this again and Close out
of reach. The cards now scroll in the room the window has left, between
96 and 320 pt, and the rest of the box stays in view.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Get the Missing Fonts: all above the Close row scrolls in a short window

Only the download cards scrolled, within fixed limits, so in a window
shorter than about 480 pt the opt-out and Close could still fall off
screen. Everything above the row with Don't show this again and Close now
scrolls in the room the window has left, and that row stays on screen.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Floating windows: the flat title bar and wider margins leave this change

Reverts 2d39c54e and its revision-history row. The restyle changes every
dialog, the Read Aloud player and the file browser, and is to be reviewed
on its own.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

* Get the Missing Fonts: the headline counts the fonts that can be downloaded

The box lists only the missing fonts with a download, so "1 font in this
document isn't installed" was wrong for a document missing others too. It
now says "1 missing font can be downloaded", and the line under it no longer
mixes one and many.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-11-17 21:05:13 +01:00
Martin BallandClaude Opus 5.5 1d71d904d2 #778: a clipboard DIB's size is read by craft-image-codecs' BMP probe (#6949)
`dib_header_dims` read a DIB header's width and height by hand. It now
wraps the header's first 12 bytes as a BMP and asks
`craft_image_codecs::bmp_probe::dimensions`. Header sizes other than 12
and 40 or more are still refused here, since the probe reads them as
info headers. The old reader stays as a test oracle.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 14:47:31 -05:00
Martin BallandClaude Opus 5.5 e28c25639b #778: the ZIP entry-count guard delegates to craft-zip (#6948)
* #778: zip_guard delegates to craft-zip's entry-count guard

wordcraft_doc::zip_guard::check and zip64_entry_count_sample now call craft-zip, which also
treats a central-directory size of u32::MAX as a ZIP64 marker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: Cargo.lock: wordcraft-doc depends on craft-zip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* #778: zip_guard: craft-zip test-support as a dev-dependency

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 14:47:27 -05:00
Martin BallandClaude Opus 5.5 7b3c2efb1f CI: skip every performance test under coverage instrumentation (#6873)
SonarCloud's coverage shards run tests built with -Cinstrument-coverage
on shared runners. Tests that assert a wall-clock limit or a time
ratio swing there: the megabyte-word test took 5.7-23 s against a
15 s limit, and three linear-time tests read 24-31x against 24x. Each
one that flakes turns main red after merging.

Ignore under cfg(coverage) every test whose purpose is performance
(linear time, bounded, capped, fast). The gate CI still runs them all
uninstrumented. Behaviour tests that only carry a generous time guard
(fuzz, never-panics, clipboard, regex, render, recent fonts) and the
SSIP speech timeout tests keep running, so their checks stay covered.


Claude-Session: https://claude.ai/code/session_01FypgCXtSjB6e5jmy9Uq7u8

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 20:29:14 +01:00
Martin Ball 9e2691290f Merge remote-tracking branch 'origin/main' into repin/6785
; Conflicts:
;	Cargo.toml
2026-11-17 16:53:27 +00:00
Dylan M. Taylor cd839d35b6 Regenerate codepage dependency ordering in Cargo lockfile 2026-11-15 12:00:40 -05:00
Dylan M. Taylor ae022437c8 doc: reuse shared legacy codepage tables with parity tests 2026-11-15 09:40:33 -05:00
53 changed files with 1875 additions and 640 deletions
Generated
+45 -42
View File
@@ -1060,7 +1060,7 @@ dependencies = [
[[package]]
name = "craft-atomic"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"rustix 1.1.5",
]
@@ -1068,17 +1068,17 @@ dependencies = [
[[package]]
name = "craft-cfb"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-codepage"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-dds"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"rayon",
]
@@ -1086,7 +1086,7 @@ dependencies = [
[[package]]
name = "craft-doc-binary"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"chrono",
"craft-cfb",
@@ -1102,7 +1102,7 @@ dependencies = [
[[package]]
name = "craft-docx"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"base64 0.23.1",
"chrono",
@@ -1122,7 +1122,7 @@ dependencies = [
[[package]]
name = "craft-drawingml"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-xml",
"kurbo",
@@ -1133,7 +1133,7 @@ dependencies = [
[[package]]
name = "craft-flow"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-wordmodel",
"image",
@@ -1143,7 +1143,7 @@ dependencies = [
[[package]]
name = "craft-font-catalog"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"fontique",
"harfrust 0.13.3",
@@ -1158,7 +1158,7 @@ dependencies = [
[[package]]
name = "craft-font-store"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"cap-primitives",
"craft-atomic",
@@ -1179,12 +1179,12 @@ dependencies = [
[[package]]
name = "craft-fonts-build"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-html"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-codepage",
"craft-flow",
@@ -1197,7 +1197,7 @@ dependencies = [
[[package]]
name = "craft-i18n"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"fixed_decimal",
"icu_decimal",
@@ -1220,7 +1220,7 @@ dependencies = [
[[package]]
name = "craft-image-codecs"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-dds",
"craft-jpeg",
@@ -1247,7 +1247,7 @@ dependencies = [
[[package]]
name = "craft-jpeg"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"thiserror 2.0.21",
]
@@ -1255,7 +1255,7 @@ dependencies = [
[[package]]
name = "craft-latex"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-flow",
"craft-math",
@@ -1266,7 +1266,7 @@ dependencies = [
[[package]]
name = "craft-markdown"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-flow",
"craft-html",
@@ -1276,7 +1276,7 @@ dependencies = [
[[package]]
name = "craft-math"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-xml",
"serde",
@@ -1285,7 +1285,7 @@ dependencies = [
[[package]]
name = "craft-metadata"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"brotli-decompressor 6.3.1",
"chrono",
@@ -1297,7 +1297,7 @@ dependencies = [
[[package]]
name = "craft-metafile"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"encoding_rs",
"image",
@@ -1308,7 +1308,7 @@ dependencies = [
[[package]]
name = "craft-mime"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"chardetng",
"chrono",
@@ -1319,7 +1319,7 @@ dependencies = [
[[package]]
name = "craft-odf"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"aes",
"base64 0.23.1",
@@ -1345,7 +1345,7 @@ dependencies = [
[[package]]
name = "craft-ods"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-odf",
"craft-sheet",
@@ -1356,7 +1356,7 @@ dependencies = [
[[package]]
name = "craft-offcrypto"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"aes",
"base64 0.23.1",
@@ -1373,7 +1373,7 @@ dependencies = [
[[package]]
name = "craft-opc"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-xml",
"craft-zip",
@@ -1384,7 +1384,7 @@ dependencies = [
[[package]]
name = "craft-pdf"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"krilla",
"thiserror 2.0.21",
@@ -1393,7 +1393,7 @@ dependencies = [
[[package]]
name = "craft-print"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-winprint-gdi",
"log",
@@ -1407,7 +1407,7 @@ dependencies = [
[[package]]
name = "craft-proof"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"log",
"miniz_oxide 0.9.1",
@@ -1418,7 +1418,7 @@ dependencies = [
[[package]]
name = "craft-rtf"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-codepage",
"craft-flow",
@@ -1431,12 +1431,12 @@ dependencies = [
[[package]]
name = "craft-sheet"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
[[package]]
name = "craft-tiff"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"serde",
"thiserror 2.0.21",
@@ -1445,7 +1445,7 @@ dependencies = [
[[package]]
name = "craft-ui"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-fonts-build",
"craft-i18n",
@@ -1465,7 +1465,7 @@ dependencies = [
[[package]]
name = "craft-update"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"chrono",
"dunce",
@@ -1492,7 +1492,7 @@ dependencies = [
[[package]]
name = "craft-web-api"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"js-sys",
"serde_json",
@@ -1503,7 +1503,7 @@ dependencies = [
[[package]]
name = "craft-web-host"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"wasm-bindgen",
"web-sys",
@@ -1512,7 +1512,7 @@ dependencies = [
[[package]]
name = "craft-window"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"egui",
"objc2 0.6.5",
@@ -1531,7 +1531,7 @@ dependencies = [
[[package]]
name = "craft-wordmodel"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"base64 0.23.1",
"craft-math",
@@ -1545,7 +1545,7 @@ dependencies = [
[[package]]
name = "craft-xls"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-cfb",
"craft-sheet",
@@ -1558,7 +1558,7 @@ dependencies = [
[[package]]
name = "craft-xlsx"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"craft-drawingml",
"craft-opc",
@@ -1570,7 +1570,7 @@ dependencies = [
[[package]]
name = "craft-xml"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"quick-xml 0.41.0",
"serde",
@@ -1580,7 +1580,7 @@ dependencies = [
[[package]]
name = "craft-zip"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"flate2",
"thiserror 2.0.21",
@@ -3416,7 +3416,7 @@ dependencies = [
[[package]]
name = "krilla"
version = "0.8.2"
source = "git+https://github.com/storytold/craft-libs?rev=453b0da782ccef39be9a6dac8d27c80b553ef1cc#453b0da782ccef39be9a6dac8d27c80b553ef1cc"
source = "git+https://github.com/storytold/craft-libs?rev=93dac78c93a7f1fd66e2022edeb089233b1dc45a#93dac78c93a7f1fd66e2022edeb089233b1dc45a"
dependencies = [
"base64 0.22.1",
"bumpalo",
@@ -7385,6 +7385,8 @@ name = "wordcraft-doc"
version = "0.18.1"
dependencies = [
"base64 0.23.1",
"craft-codepage",
"craft-zip",
"log",
"proptest",
"serde",
@@ -7468,6 +7470,7 @@ dependencies = [
"harfrust 0.13.3",
"kurbo",
"log",
"serde_json",
"skrifa 0.47.0",
"unicode-bidi",
"unicode-script",
+31 -29
View File
@@ -24,19 +24,20 @@ wordcraft-pdf = { path = "crates/pdf" }
# Proofing is craft-libs' craft-proof (it was crates/proof); its Language data is craft-i18n's
# language-data/, embedded from CRAFT_I18N_DIR (craft-i18n.rev). The package keeps the
# `wordcraft-proof` name here so `wordcraft_proof::` paths are unchanged.
# craft-proof is storytold/craft-libs main at 453b0da782. That commit has #590 (ca-ES-valencia
# craft-proof is storytold/craft-libs main at 93dac78c. That tree has #590 (ca-ES-valencia
# reads the ca-valencia pack), #589 (French elision suggestions first), #497 (flag-based
# compounding), #496 (review follow-ups, en-AU) and the merged #482–#490/#495 stack; the font
# store's `language-pack` kind is #486 (CP4.1).
wordcraft-proof = { package = "craft-proof", git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-font-store = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-font-catalog = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
wordcraft-proof = { package = "craft-proof", git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-font-store = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-font-catalog = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
wordcraft-engine = { path = "crates/engine" }
wordcraft-mcp = { path = "crates/mcp" }
wordcraft-zotero = { path = "crates/zotero" }
wordcraft-control-key = { path = "crates/control-key" }
wordcraft-ui-egui = { path = "crates/ui-egui" }
craft-metafile = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-codepage = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-metafile = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
# foundation
kurbo = { version = "0.13", features = ["serde"] }
@@ -67,35 +68,36 @@ wuff = "0.2.9"
brotli-decompressor = "5"
vello_cpu = "0.2"
image = { version = "0.25", default-features = false }
craft-image-codecs = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-tiff = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-pdf = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", default-features = false, features = ["write"] }
craft-image-codecs = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-tiff = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-pdf = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", default-features = false, features = ["write"] }
# craft-update: app-specific pretend versions apply to startup and manual checks.
craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", features = ["egui"] }
craft-wordmodel = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-docx = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-rtf = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", features = ["doc"] }
craft-doc-binary = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-flow = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-odf = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", features = ["text"] }
craft-markdown = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-html = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", features = ["flow"] }
craft-latex = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-math = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-mime = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0", features = ["detect"] }
craft-sheet = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-ods = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-xls = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-xlsx = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-xml = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-zip = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "=0.1.0" }
craft-update = { git = "https://github.com/storytold/craft-libs", version = "=0.1.0", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", features = ["egui"] }
craft-wordmodel = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-docx = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-rtf = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", features = ["doc"] }
craft-doc-binary = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-flow = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-odf = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", features = ["text"] }
craft-markdown = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-html = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", features = ["flow"] }
craft-latex = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-math = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-mime = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0", features = ["detect"] }
craft-sheet = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-ods = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-xls = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-xlsx = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-xml = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
craft-zip = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "=0.1.0" }
# zlib streams (compressed metafile pictures in `.doc` drawings); already in the lock as a dependency of other crates.
miniz_oxide = "0.9"
cfb = "0.15"
# Legacy text encodings (GBK/GB18030, Big5, Shift_JIS, EUC-KR, single-byte code pages) for plain
# text and CSV recipient lists (#778): encoding_rs ((Apache-2.0 OR MIT) AND BSD-3-Clause) and its
# detector chardetng (Apache-2.0 OR MIT), both pure Rust (Firefox's).
# text and CSV recipient lists (#778): encoding_rs ((Apache-2.0 OR MIT) AND BSD-3-Clause), pure
# Rust (Firefox's). Detection is craft-mime's `detect` feature.
encoding_rs = "0.8"
# chardetng (Apache-2.0 OR MIT): only the formats test oracle for the old detector uses it directly.
chardetng = "1"
# Password-protected (agile-encrypted) Word documents: pure-Rust RustCrypto primitives.
aes = "0.9"
@@ -153,7 +155,7 @@ debug = 0
# Required by craft-pdf: the backend rejects writers that failed during unwinding.
[patch.crates-io]
krilla = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc" }
krilla = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a" }
ecolor = { git = "https://github.com/storytold/egui.git", rev = "6d518fc799681571963842f95b55a6b68431b15a" }
emath = { git = "https://github.com/storytold/egui.git", rev = "6d518fc799681571963842f95b55a6b68431b15a" }
epaint = { git = "https://github.com/storytold/egui.git", rev = "6d518fc799681571963842f95b55a6b68431b15a" }
+5 -1
View File
File diff suppressed because one or more lines are too long
+2 -2
View File
@@ -18,8 +18,8 @@ wordcraft-engine = { workspace = true }
wordcraft-ui-egui = { workspace = true }
# The host messages (storytold/craft-libs): one contract and one bridge for the web builds.
# The page command API (storytold/craft-libs#413): window.wordcraft over the control channel.
craft-web-api = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "0.1.0" }
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", version = "0.1.0" }
craft-web-api = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "0.1.0" }
craft-web-host = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", version = "0.1.0" }
egui = { workspace = true }
log = { workspace = true }
serde_json = { workspace = true }
+1
View File
@@ -81,6 +81,7 @@ pub fn start() {
// Pasted pictures wait apart from dropped files: the app inserts one only when the page is the edit target (#1349).
app.services.pasted = Some(pasted);
app.autosave = false;
app.tell_about_font_downloads();
// Preferences from an earlier visit (#2132), as the desktop app reads `ui.json`.
let stored = stored_prefs();
crate::prefs::restore(&mut app, stored.as_deref());
+1 -1
View File
@@ -86,7 +86,7 @@ clipboard-win = { version = "5.4", features = ["std"] }
[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
# At the same craft-libs commit as the workspace's crates: `printer_queues`, the printer list that
# asks no driver (#5710, storytold/craft-libs#618), and `submit_cancellable` (storytold/craft-libs#622).
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc" }
craft-print = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a" }
rfd = { workspace = true }
# Waits for an `rfd::AsyncFileDialog` on a background thread (the dialogs don't block the UI, #94);
# rfd itself already depends on it.
+1
View File
@@ -728,6 +728,7 @@ fn main() -> eframe::Result {
app.replace_startup_document(doc);
}
app.watch_desktop_theme(&cc.egui_ctx);
app.tell_about_font_downloads();
if prefs_enabled()
&& let Some(dir) = wordcraft_control_key::settings_dir()
{
+4 -1
View File
@@ -13,10 +13,12 @@ workspace = true
# `zip_guard::zip64_entry_count_sample`) and the step count of
# `work::count`; turned on only as a dev-dependency, so never built into
# the app.
test-support = []
test-support = ["craft-zip/test-support"]
[dependencies]
craft-codepage = { workspace = true }
wordcraft-geom = { workspace = true }
craft-zip = { workspace = true }
base64 = { workspace = true }
log = { workspace = true }
serde = { workspace = true }
@@ -25,4 +27,5 @@ thiserror = { workspace = true }
unicode-segmentation = { workspace = true }
[dev-dependencies]
craft-zip = { workspace = true, features = ["test-support"] }
proptest = { workspace = true }
+44 -337
View File
@@ -1,345 +1,52 @@
//! Windows-1252 (the byte encoding legacy Word text pieces use) and the other single-byte Windows
//! code pages RTF files name with `\ansicpg` or a font's `\fcharset` (Cyrillic 1251, Central
//! European 1250, Greek, Turkish, Hebrew, Arabic, Baltic, Vietnamese, Thai), Mac OS Roman
//! (10000, RTF `\mac`), and the East Asian multibyte code pages (932 Shift-JIS, 936 GBK, 949 UHC,
//! 950 Big5), whose bytes decode as runs rather than one at a time.
//! Legacy character decoding delegates to craft-codepage; document fallback policy stays here.
pub use craft_codepage::{
MAC_ROMAN_CODE_PAGE, MULTIBYTE_CODE_PAGES, OEM_CODE_PAGES, SINGLE_BYTE_CODE_PAGES, cp1252, is_multibyte_code_page, multibyte_encoding_name,
oem_byte, windows_byte,
};
/// Windows-1252 code points for 0x80..=0x9F.
const CP1252: [u16; 32] = [
0x20AC, 0x0081, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0x008D, 0x017D, 0x008F, 0x0090, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0x009D, 0x017E, 0x0178,
];
/// One Windows-1252 byte as a char.
pub fn cp1252(b: u8) -> char {
if (0x80..0xA0).contains(&b) {
CP1252.get((b - 0x80) as usize).and_then(|u| char::from_u32(u32::from(*u))).unwrap_or('\u{FFFD}')
} else {
b as char
}
/// Word's supported single-byte document code pages (symbol-font policy is separate).
pub fn is_byte_code_page(cp: u16) -> bool {
SINGLE_BYTE_CODE_PAGES.contains(&cp) || OEM_CODE_PAGES.contains(&cp) || cp == MAC_ROMAN_CODE_PAGE
}
/// Windows-874 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP874: [u16; 128] = [
0x20AC, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2026, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x0E01, 0x0E02, 0x0E03,
0x0E04, 0x0E05, 0x0E06, 0x0E07, 0x0E08, 0x0E09, 0x0E0A, 0x0E0B, 0x0E0C, 0x0E0D, 0x0E0E, 0x0E0F, 0x0E10, 0x0E11, 0x0E12, 0x0E13, 0x0E14, 0x0E15,
0x0E16, 0x0E17, 0x0E18, 0x0E19, 0x0E1A, 0x0E1B, 0x0E1C, 0x0E1D, 0x0E1E, 0x0E1F, 0x0E20, 0x0E21, 0x0E22, 0x0E23, 0x0E24, 0x0E25, 0x0E26, 0x0E27,
0x0E28, 0x0E29, 0x0E2A, 0x0E2B, 0x0E2C, 0x0E2D, 0x0E2E, 0x0E2F, 0x0E30, 0x0E31, 0x0E32, 0x0E33, 0x0E34, 0x0E35, 0x0E36, 0x0E37, 0x0E38, 0x0E39,
0x0E3A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x0E3F, 0x0E40, 0x0E41, 0x0E42, 0x0E43, 0x0E44, 0x0E45, 0x0E46, 0x0E47, 0x0E48, 0x0E49, 0x0E4A, 0x0E4B,
0x0E4C, 0x0E4D, 0x0E4E, 0x0E4F, 0x0E50, 0x0E51, 0x0E52, 0x0E53, 0x0E54, 0x0E55, 0x0E56, 0x0E57, 0x0E58, 0x0E59, 0x0E5A, 0x0E5B, 0xFFFD, 0xFFFD,
0xFFFD, 0xFFFD,
];
/// Windows-1250 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1250: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0xFFFD, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0x0160, 0x2039, 0x015A, 0x0164, 0x017D, 0x0179, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0x0161, 0x203A, 0x015B, 0x0165, 0x017E, 0x017A, 0x00A0, 0x02C7, 0x02D8, 0x0141,
0x00A4, 0x0104, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x015E, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x017B, 0x00B0, 0x00B1, 0x02DB, 0x0142, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x0105, 0x015F, 0x00BB, 0x013D, 0x02DD, 0x013E, 0x017C, 0x0154, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x0139, 0x0106, 0x00C7,
0x010C, 0x00C9, 0x0118, 0x00CB, 0x011A, 0x00CD, 0x00CE, 0x010E, 0x0110, 0x0143, 0x0147, 0x00D3, 0x00D4, 0x0150, 0x00D6, 0x00D7, 0x0158, 0x016E,
0x00DA, 0x0170, 0x00DC, 0x00DD, 0x0162, 0x00DF, 0x0155, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x013A, 0x0107, 0x00E7, 0x010D, 0x00E9, 0x0119, 0x00EB,
0x011B, 0x00ED, 0x00EE, 0x010F, 0x0111, 0x0144, 0x0148, 0x00F3, 0x00F4, 0x0151, 0x00F6, 0x00F7, 0x0159, 0x016F, 0x00FA, 0x0171, 0x00FC, 0x00FD,
0x0163, 0x02D9,
];
/// Windows-1251 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1251: [u16; 128] = [
0x0402, 0x0403, 0x201A, 0x0453, 0x201E, 0x2026, 0x2020, 0x2021, 0x20AC, 0x2030, 0x0409, 0x2039, 0x040A, 0x040C, 0x040B, 0x040F, 0x0452, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0x0459, 0x203A, 0x045A, 0x045C, 0x045B, 0x045F, 0x00A0, 0x040E, 0x045E, 0x0408,
0x00A4, 0x0490, 0x00A6, 0x00A7, 0x0401, 0x00A9, 0x0404, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x0407, 0x00B0, 0x00B1, 0x0406, 0x0456, 0x0491, 0x00B5,
0x00B6, 0x00B7, 0x0451, 0x2116, 0x0454, 0x00BB, 0x0458, 0x0405, 0x0455, 0x0457, 0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417,
0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421, 0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429,
0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433, 0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B,
0x043C, 0x043D, 0x043E, 0x043F, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B, 0x044C, 0x044D,
0x044E, 0x044F,
];
/// Windows-1253 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1253: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x0385, 0x0386, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0xFFFD, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x2015, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x0384, 0x00B5,
0x00B6, 0x00B7, 0x0388, 0x0389, 0x038A, 0x00BB, 0x038C, 0x00BD, 0x038E, 0x038F, 0x0390, 0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397,
0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, 0xFFFD, 0x03A3, 0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9,
0x03AA, 0x03AB, 0x03AC, 0x03AD, 0x03AE, 0x03AF, 0x03B0, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB,
0x03BC, 0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C2, 0x03C3, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x03C9, 0x03CA, 0x03CB, 0x03CC, 0x03CD,
0x03CE, 0xFFFD,
];
/// Windows-1254 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1254: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0xFFFD, 0xFFFD, 0x0178, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x00C7,
0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, 0x011E, 0x00D1, 0x00D2, 0x00D3, 0x00D4, 0x00D5, 0x00D6, 0x00D7, 0x00D8, 0x00D9,
0x00DA, 0x00DB, 0x00DC, 0x0130, 0x015E, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x00EC, 0x00ED, 0x00EE, 0x00EF, 0x011F, 0x00F1, 0x00F2, 0x00F3, 0x00F4, 0x00F5, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x0131,
0x015F, 0x00FF,
];
/// Windows-1255 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1255: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x20AA, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00D7, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00F7, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x05B0, 0x05B1, 0x05B2, 0x05B3, 0x05B4, 0x05B5, 0x05B6, 0x05B7,
0x05B8, 0x05B9, 0x05BA, 0x05BB, 0x05BC, 0x05BD, 0x05BE, 0x05BF, 0x05C0, 0x05C1, 0x05C2, 0x05C3, 0x05F0, 0x05F1, 0x05F2, 0x05F3, 0x05F4, 0xFFFD,
0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x05D0, 0x05D1, 0x05D2, 0x05D3, 0x05D4, 0x05D5, 0x05D6, 0x05D7, 0x05D8, 0x05D9, 0x05DA, 0x05DB,
0x05DC, 0x05DD, 0x05DE, 0x05DF, 0x05E0, 0x05E1, 0x05E2, 0x05E3, 0x05E4, 0x05E5, 0x05E6, 0x05E7, 0x05E8, 0x05E9, 0x05EA, 0xFFFD, 0xFFFD, 0x200E,
0x200F, 0xFFFD,
];
/// Windows-1256 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1256: [u16; 128] = [
0x20AC, 0x067E, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0679, 0x2039, 0x0152, 0x0686, 0x0698, 0x0688, 0x06AF, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x06A9, 0x2122, 0x0691, 0x203A, 0x0153, 0x200C, 0x200D, 0x06BA, 0x00A0, 0x060C, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x06BE, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x061B, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x061F, 0x06C1, 0x0621, 0x0622, 0x0623, 0x0624, 0x0625, 0x0626, 0x0627,
0x0628, 0x0629, 0x062A, 0x062B, 0x062C, 0x062D, 0x062E, 0x062F, 0x0630, 0x0631, 0x0632, 0x0633, 0x0634, 0x0635, 0x0636, 0x00D7, 0x0637, 0x0638,
0x0639, 0x063A, 0x0640, 0x0641, 0x0642, 0x0643, 0x00E0, 0x0644, 0x00E2, 0x0645, 0x0646, 0x0647, 0x0648, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0649, 0x064A, 0x00EE, 0x00EF, 0x064B, 0x064C, 0x064D, 0x064E, 0x00F4, 0x064F, 0x0650, 0x00F7, 0x0651, 0x00F9, 0x0652, 0x00FB, 0x00FC, 0x200E,
0x200F, 0x06D2,
];
/// Windows-1257 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1257: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0xFFFD, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0x00A8, 0x02C7, 0x00B8, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0x00AF, 0x02DB, 0xFFFD, 0x00A0, 0xFFFD, 0x00A2, 0x00A3,
0x00A4, 0xFFFD, 0x00A6, 0x00A7, 0x00D8, 0x00A9, 0x0156, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00C6, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00F8, 0x00B9, 0x0157, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00E6, 0x0104, 0x012E, 0x0100, 0x0106, 0x00C4, 0x00C5, 0x0118, 0x0112,
0x010C, 0x00C9, 0x0179, 0x0116, 0x0122, 0x0136, 0x012A, 0x013B, 0x0160, 0x0143, 0x0145, 0x00D3, 0x014C, 0x00D5, 0x00D6, 0x00D7, 0x0172, 0x0141,
0x015A, 0x016A, 0x00DC, 0x017B, 0x017D, 0x00DF, 0x0105, 0x012F, 0x0101, 0x0107, 0x00E4, 0x00E5, 0x0119, 0x0113, 0x010D, 0x00E9, 0x017A, 0x0117,
0x0123, 0x0137, 0x012B, 0x013C, 0x0161, 0x0144, 0x0146, 0x00F3, 0x014D, 0x00F5, 0x00F6, 0x00F7, 0x0173, 0x0142, 0x015B, 0x016B, 0x00FC, 0x017C,
0x017E, 0x02D9,
];
/// Windows-1258 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1258: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0xFFFD, 0x2039, 0x0152, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0xFFFD, 0x203A, 0x0153, 0xFFFD, 0xFFFD, 0x0178, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x00C5, 0x00C6, 0x00C7,
0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x0300, 0x00CD, 0x00CE, 0x00CF, 0x0110, 0x00D1, 0x0309, 0x00D3, 0x00D4, 0x01A0, 0x00D6, 0x00D7, 0x00D8, 0x00D9,
0x00DA, 0x00DB, 0x00DC, 0x01AF, 0x0303, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0301, 0x00ED, 0x00EE, 0x00EF, 0x0111, 0x00F1, 0x0323, 0x00F3, 0x00F4, 0x01A1, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x01B0,
0x20AB, 0x00FF,
];
/// Mac OS Roman (code page 10000, RTF `\mac`) code points for 0x80..=0xFF, with the euro sign at
/// 0xDB (Mac OS 8.5 and later).
const MAC_ROMAN: [u16; 128] = [
0x00C4, 0x00C5, 0x00C7, 0x00C9, 0x00D1, 0x00D6, 0x00DC, 0x00E1, 0x00E0, 0x00E2, 0x00E4, 0x00E3, 0x00E5, 0x00E7, 0x00E9, 0x00E8, 0x00EA, 0x00EB,
0x00ED, 0x00EC, 0x00EE, 0x00EF, 0x00F1, 0x00F3, 0x00F2, 0x00F4, 0x00F6, 0x00F5, 0x00FA, 0x00F9, 0x00FB, 0x00FC, 0x2020, 0x00B0, 0x00A2, 0x00A3,
0x00A7, 0x2022, 0x00B6, 0x00DF, 0x00AE, 0x00A9, 0x2122, 0x00B4, 0x00A8, 0x2260, 0x00C6, 0x00D8, 0x221E, 0x00B1, 0x2264, 0x2265, 0x00A5, 0x00B5,
0x2202, 0x2211, 0x220F, 0x03C0, 0x222B, 0x00AA, 0x00BA, 0x03A9, 0x00E6, 0x00F8, 0x00BF, 0x00A1, 0x00AC, 0x221A, 0x0192, 0x2248, 0x2206, 0x00AB,
0x00BB, 0x2026, 0x00A0, 0x00C0, 0x00C3, 0x00D5, 0x0152, 0x0153, 0x2013, 0x2014, 0x201C, 0x201D, 0x2018, 0x2019, 0x00F7, 0x25CA, 0x00FF, 0x0178,
0x2044, 0x20AC, 0x2039, 0x203A, 0xFB01, 0xFB02, 0x2021, 0x00B7, 0x201A, 0x201E, 0x2030, 0x00C2, 0x00CA, 0x00C1, 0x00CB, 0x00C8, 0x00CD, 0x00CE,
0x00CF, 0x00CC, 0x00D3, 0x00D4, 0xF8FF, 0x00D2, 0x00DA, 0x00DB, 0x00D9, 0x0131, 0x02C6, 0x02DC, 0x00AF, 0x02D8, 0x02D9, 0x02DA, 0x00B8, 0x02DD,
0x02DB, 0x02C7,
];
/// IBM PC (OEM) code page 437 code points for 0x80..=0xFF (the Unicode Consortium's CP437
/// mapping file).
const CP437: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x00EC, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x00FF, 0x00D6, 0x00DC, 0x00A2, 0x00A3, 0x00A5, 0x20A7, 0x0192, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x00AA, 0x00BA, 0x00BF, 0x2310, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x03B1, 0x00DF, 0x0393, 0x03C0, 0x03A3, 0x03C3, 0x00B5, 0x03C4, 0x03A6, 0x0398, 0x03A9, 0x03B4,
0x221E, 0x03C6, 0x03B5, 0x2229, 0x2261, 0x00B1, 0x2265, 0x2264, 0x2320, 0x2321, 0x00F7, 0x2248, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 850 (DOS Latin-1, Western European) code points for 0x80..=0xFF (the Unicode
/// Consortium's CP850 mapping file).
const CP850: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x00EC, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x00FF, 0x00D6, 0x00DC, 0x00F8, 0x00A3, 0x00D8, 0x00D7, 0x0192, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x00AA, 0x00BA, 0x00BF, 0x00AE, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x00C0, 0x00A9, 0x2563, 0x2551, 0x2557, 0x255D, 0x00A2, 0x00A5, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x00E3, 0x00C3,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x00F0, 0x00D0, 0x00CA, 0x00CB, 0x00C8, 0x0131, 0x00CD, 0x00CE, 0x00CF, 0x2518,
0x250C, 0x2588, 0x2584, 0x00A6, 0x00CC, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x00D2, 0x00F5, 0x00D5, 0x00B5, 0x00FE, 0x00DE, 0x00DA, 0x00DB, 0x00D9,
0x00FD, 0x00DD, 0x00AF, 0x00B4, 0x00AD, 0x00B1, 0x2017, 0x00BE, 0x00B6, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x00B7, 0x00B9, 0x00B3, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 852 (DOS Latin-2, Central European) code points for 0x80..=0xFF (the Unicode Consortium's CP852
/// mapping file).
const CP852: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x016F, 0x0107, 0x00E7, 0x0142, 0x00EB, 0x0150, 0x0151, 0x00EE, 0x0179, 0x00C4, 0x0106, 0x00C9, 0x0139,
0x013A, 0x00F4, 0x00F6, 0x013D, 0x013E, 0x015A, 0x015B, 0x00D6, 0x00DC, 0x0164, 0x0165, 0x0141, 0x00D7, 0x010D, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x0104, 0x0105, 0x017D, 0x017E, 0x0118, 0x0119, 0x00AC, 0x017A, 0x010C, 0x015F, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x011A, 0x015E, 0x2563, 0x2551, 0x2557, 0x255D, 0x017B, 0x017C, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x0102, 0x0103,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x0111, 0x0110, 0x010E, 0x00CB, 0x010F, 0x0147, 0x00CD, 0x00CE, 0x011B, 0x2518,
0x250C, 0x2588, 0x2584, 0x0162, 0x016E, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x0143, 0x0144, 0x0148, 0x0160, 0x0161, 0x0154, 0x00DA, 0x0155, 0x0170,
0x00FD, 0x00DD, 0x0163, 0x00B4, 0x00AD, 0x02DD, 0x02DB, 0x02C7, 0x02D8, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x02D9, 0x0171, 0x0158, 0x0159,
0x25A0, 0x00A0,
];
/// OEM code page 866 (DOS Cyrillic) code points for 0x80..=0xFF (the Unicode Consortium's CP866
/// mapping file).
const CP866: [u16; 128] = [
0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417, 0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421,
0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429, 0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433,
0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B, 0x043C, 0x043D, 0x043E, 0x043F, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B,
0x044C, 0x044D, 0x044E, 0x044F, 0x0401, 0x0451, 0x0404, 0x0454, 0x0407, 0x0457, 0x040E, 0x045E, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x2116, 0x00A4,
0x25A0, 0x00A0,
];
/// OEM code page 737 (DOS Greek) code points for 0x80..=0xFF (the Unicode Consortium's CP737
/// mapping file).
const CP737: [u16; 128] = [
0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397, 0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, 0x03A3,
0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB, 0x03BC,
0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C3, 0x03C2, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x03C9, 0x03AC, 0x03AD, 0x03AE, 0x03CA, 0x03AF, 0x03CC, 0x03CD, 0x03CB, 0x03CE, 0x0386, 0x0388,
0x0389, 0x038A, 0x038C, 0x038E, 0x038F, 0x00B1, 0x2265, 0x2264, 0x03AA, 0x03AB, 0x00F7, 0x2248, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 857 (DOS Turkish; U+FFFD where the byte is unassigned) code points for 0x80..=0xFF (the Unicode Consortium's CP857
/// mapping file).
const CP857: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x0131, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x0130, 0x00D6, 0x00DC, 0x00F8, 0x00A3, 0x00D8, 0x015E, 0x015F, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x011E, 0x011F, 0x00BF, 0x00AE, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x00C0, 0x00A9, 0x2563, 0x2551, 0x2557, 0x255D, 0x00A2, 0x00A5, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x00E3, 0x00C3,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x00BA, 0x00AA, 0x00CA, 0x00CB, 0x00C8, 0xFFFD, 0x00CD, 0x00CE, 0x00CF, 0x2518,
0x250C, 0x2588, 0x2584, 0x00A6, 0x00CC, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x00D2, 0x00F5, 0x00D5, 0x00B5, 0xFFFD, 0x00D7, 0x00DA, 0x00DB, 0x00D9,
0x00EC, 0x00FF, 0x00AF, 0x00B4, 0x00AD, 0x00B1, 0xFFFD, 0x00BE, 0x00B6, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x00B7, 0x00B9, 0x00B3, 0x00B2,
0x25A0, 0x00A0,
];
/// The glyphs the OEM code pages show for 0x01..=0x1F (smileys, card suits, arrows), as
/// Windows' Alt+numpad entry produces them rather than the control characters.
const OEM_GLYPHS: [u16; 31] = [
0x263A, 0x263B, 0x2665, 0x2666, 0x2663, 0x2660, 0x2022, 0x25D8, 0x25CB, 0x25D9, 0x2642, 0x2640, 0x266A, 0x266B, 0x263C, 0x25BA, 0x25C4, 0x2195,
0x203C, 0x00B6, 0x00A7, 0x25AC, 0x21A8, 0x2191, 0x2193, 0x2192, 0x2190, 0x221F, 0x2194, 0x25B2, 0x25BC,
];
/// The OEM code pages [`oem_byte`] decodes.
pub const OEM_CODE_PAGES: [u16; 6] = [437, 737, 850, 852, 857, 866];
/// One byte of an OEM code page in [`OEM_CODE_PAGES`] (any other code page reads as 437) as a
/// char, with the graphic glyphs for 0x01..=0x1F and 0x7F (⌂); 0x00 stays NUL.
pub fn oem_byte(code_page: u16, b: u8) -> char {
let table: &[u16; 128] = match code_page {
737 => &CP737,
850 => &CP850,
852 => &CP852,
857 => &CP857,
866 => &CP866,
_ => &CP437,
};
let u = match b {
0x01..=0x1F => OEM_GLYPHS.get(usize::from(b - 1)).copied(),
0x7F => Some(0x2302),
0x80..=0xFF => table.get(usize::from(b - 0x80)).copied(),
_ => Some(u16::from(b)),
};
u.and_then(|u| char::from_u32(u32::from(u))).unwrap_or('\u{FFFD}')
/// Decode document text without applying OEM control glyphs or symbol-font remapping.
pub fn code_page_byte(cp: u16, b: u8) -> char {
if b >= 0x80 && OEM_CODE_PAGES.contains(&cp) { oem_byte(cp, b) } else { windows_byte(cp, b) }
}
/// The single-byte Windows code pages [`windows_byte`] decodes (1252 included).
pub const SINGLE_BYTE_CODE_PAGES: [u16; 10] = [874, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258];
/// Mac OS Roman, RTF's `\mac` character set.
pub const MAC_ROMAN_CODE_PAGE: u16 = 10000;
/// One byte of a single-byte code page as a char: a Windows one in [`SINGLE_BYTE_CODE_PAGES`] or
/// Mac OS Roman ([`MAC_ROMAN_CODE_PAGE`]); any other code page reads as Windows-1252.
pub fn windows_byte(code_page: u16, b: u8) -> char {
let table: &[u16; 128] = match code_page {
874 => &CP874,
1250 => &CP1250,
1251 => &CP1251,
1253 => &CP1253,
1254 => &CP1254,
1255 => &CP1255,
1256 => &CP1256,
1257 => &CP1257,
1258 => &CP1258,
MAC_ROMAN_CODE_PAGE => &MAC_ROMAN,
_ => return cp1252(b),
};
if b < 0x80 {
return b as char;
}
table.get(usize::from(b - 0x80)).and_then(|u| char::from_u32(u32::from(*u))).unwrap_or('\u{FFFD}')
/// Whether Word's document import accepts this encoding.
pub fn is_text_code_page(cp: u16) -> bool {
is_byte_code_page(cp) || is_multibyte_code_page(cp)
}
/// Whether [`code_page_byte`] decodes `code_page`: a Windows one ([`SINGLE_BYTE_CODE_PAGES`]), an
/// OEM (DOS) one ([`OEM_CODE_PAGES`]) or Mac OS Roman. Document formats (RTF) name all three.
pub fn is_byte_code_page(code_page: u16) -> bool {
SINGLE_BYTE_CODE_PAGES.contains(&code_page) || OEM_CODE_PAGES.contains(&code_page) || code_page == MAC_ROMAN_CODE_PAGE
}
/// A byte of 8-bit document text in `code_page` (see [`is_byte_code_page`]) as a char. Below 0x80
/// it is ASCII in every code page (text, so an OEM page's glyphs for control codes don't apply).
pub fn code_page_byte(code_page: u16, b: u8) -> char {
if b >= 0x80 && OEM_CODE_PAGES.contains(&code_page) { oem_byte(code_page, b) } else { windows_byte(code_page, b) }
}
/// The East Asian multibyte code pages a document format names with `\ansicpg` or a font's `\cpg`:
/// 932 Shift-JIS (Japanese), 936 GBK (Simplified Chinese), 949 UHC (Korean) and 950 Big5
/// (Traditional Chinese).
pub const MULTIBYTE_CODE_PAGES: [u16; 4] = [932, 936, 949, 950];
/// Whether `code_page` is one of the East Asian multibyte code pages ([`MULTIBYTE_CODE_PAGES`]):
/// its bytes decode as a run, a lead byte together with its trail byte.
pub fn is_multibyte_code_page(code_page: u16) -> bool {
multibyte_encoding_name(code_page).is_some()
}
/// The WHATWG encoding name of an East Asian multibyte code page (see [`MULTIBYTE_CODE_PAGES`]),
/// for `encoding_rs::Encoding::for_label`; `None` for the single-byte code pages, whose bytes
/// decode one at a time, and anything else.
pub fn multibyte_encoding_name(code_page: u16) -> Option<&'static str> {
Some(match code_page {
932 => "shift_jis",
936 => "gbk",
949 => "euc-kr",
950 => "big5",
_ => return None,
})
}
/// Whether a document format can name `code_page` for its text: a single-byte Windows, DOS or Mac
/// one ([`is_byte_code_page`]) or an East Asian multibyte one ([`is_multibyte_code_page`]).
pub fn is_text_code_page(code_page: u16) -> bool {
is_byte_code_page(code_page) || is_multibyte_code_page(code_page)
}
/// The code page of an RTF or Word font charset (`\fcharset`): 204 Russian → 1251, 238 Eastern
/// European → 1250, 77 Mac → 10000, 254 PC / 255 OEM → 437, 128 Japanese → 932, 129 Korean → 949,
/// 134 Simplified Chinese → 936, 136 Traditional Chinese → 950… `None` for the ANSI and default
/// charsets (the document's code page applies) and symbol fonts.
/// Preserve Word's existing symbol-font selection policy, not the shared symbol extension.
pub fn charset_code_page(charset: i32) -> Option<u16> {
Some(match charset {
77 => MAC_ROMAN_CODE_PAGE,
128 => 932,
129 => 949,
134 => 936,
136 => 950,
161 => 1253,
162 => 1254,
163 => 1258,
177 => 1255,
178 => 1256,
186 => 1257,
204 => 1251,
222 => 874,
238 => 1250,
254 | 255 => 437,
_ => return None,
})
if charset == 2 { None } else { craft_codepage::charset_code_page(charset) }
}
#[cfg(test)]
#[allow(dead_code)]
#[path = "encoding_legacy_tests.rs"]
mod legacy;
#[cfg(test)]
mod parity {
use super::*;
#[test]
fn every_supported_byte_and_fallback_matches_original() {
let pages = [0, 42, 437, 737, 850, 852, 857, 866, 874, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258, 10000, 9999, u16::MAX];
for cp in pages {
for b in 0..=u8::MAX {
assert_eq!(windows_byte(cp, b), legacy::windows_byte(cp, b), "windows {cp}/{b}");
assert_eq!(oem_byte(cp, b), legacy::oem_byte(cp, b), "oem {cp}/{b}");
assert_eq!(code_page_byte(cp, b), legacy::code_page_byte(cp, b), "text {cp}/{b}");
assert_eq!(cp1252(b), legacy::cp1252(b));
}
}
for cp in 0..=u16::MAX {
assert_eq!(is_byte_code_page(cp), legacy::is_byte_code_page(cp));
assert_eq!(is_text_code_page(cp), legacy::is_text_code_page(cp));
assert_eq!(multibyte_encoding_name(cp), legacy::multibyte_encoding_name(cp));
}
for charset in -1..=256 {
assert_eq!(charset_code_page(charset), legacy::charset_code_page(charset));
}
}
}
#[cfg(test)]
+428
View File
@@ -0,0 +1,428 @@
// Frozen pre-cutover implementation, test-only parity oracle. Original authors retained in file history.
//! Windows-1252 (the byte encoding legacy Word text pieces use) and the other single-byte Windows
//! code pages RTF files name with `\ansicpg` or a font's `\fcharset` (Cyrillic 1251, Central
//! European 1250, Greek, Turkish, Hebrew, Arabic, Baltic, Vietnamese, Thai), Mac OS Roman
//! (10000, RTF `\mac`), and the East Asian multibyte code pages (932 Shift-JIS, 936 GBK, 949 UHC,
//! 950 Big5), whose bytes decode as runs rather than one at a time.
/// Windows-1252 code points for 0x80..=0x9F.
const CP1252: [u16; 32] = [
0x20AC, 0x0081, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0x008D, 0x017D, 0x008F, 0x0090, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0x009D, 0x017E, 0x0178,
];
/// One Windows-1252 byte as a char.
pub fn cp1252(b: u8) -> char {
if (0x80..0xA0).contains(&b) {
CP1252.get((b - 0x80) as usize).and_then(|u| char::from_u32(u32::from(*u))).unwrap_or('\u{FFFD}')
} else {
b as char
}
}
/// Windows-874 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP874: [u16; 128] = [
0x20AC, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2026, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x0E01, 0x0E02, 0x0E03,
0x0E04, 0x0E05, 0x0E06, 0x0E07, 0x0E08, 0x0E09, 0x0E0A, 0x0E0B, 0x0E0C, 0x0E0D, 0x0E0E, 0x0E0F, 0x0E10, 0x0E11, 0x0E12, 0x0E13, 0x0E14, 0x0E15,
0x0E16, 0x0E17, 0x0E18, 0x0E19, 0x0E1A, 0x0E1B, 0x0E1C, 0x0E1D, 0x0E1E, 0x0E1F, 0x0E20, 0x0E21, 0x0E22, 0x0E23, 0x0E24, 0x0E25, 0x0E26, 0x0E27,
0x0E28, 0x0E29, 0x0E2A, 0x0E2B, 0x0E2C, 0x0E2D, 0x0E2E, 0x0E2F, 0x0E30, 0x0E31, 0x0E32, 0x0E33, 0x0E34, 0x0E35, 0x0E36, 0x0E37, 0x0E38, 0x0E39,
0x0E3A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x0E3F, 0x0E40, 0x0E41, 0x0E42, 0x0E43, 0x0E44, 0x0E45, 0x0E46, 0x0E47, 0x0E48, 0x0E49, 0x0E4A, 0x0E4B,
0x0E4C, 0x0E4D, 0x0E4E, 0x0E4F, 0x0E50, 0x0E51, 0x0E52, 0x0E53, 0x0E54, 0x0E55, 0x0E56, 0x0E57, 0x0E58, 0x0E59, 0x0E5A, 0x0E5B, 0xFFFD, 0xFFFD,
0xFFFD, 0xFFFD,
];
/// Windows-1250 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1250: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0xFFFD, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0x0160, 0x2039, 0x015A, 0x0164, 0x017D, 0x0179, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0x0161, 0x203A, 0x015B, 0x0165, 0x017E, 0x017A, 0x00A0, 0x02C7, 0x02D8, 0x0141,
0x00A4, 0x0104, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x015E, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x017B, 0x00B0, 0x00B1, 0x02DB, 0x0142, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x0105, 0x015F, 0x00BB, 0x013D, 0x02DD, 0x013E, 0x017C, 0x0154, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x0139, 0x0106, 0x00C7,
0x010C, 0x00C9, 0x0118, 0x00CB, 0x011A, 0x00CD, 0x00CE, 0x010E, 0x0110, 0x0143, 0x0147, 0x00D3, 0x00D4, 0x0150, 0x00D6, 0x00D7, 0x0158, 0x016E,
0x00DA, 0x0170, 0x00DC, 0x00DD, 0x0162, 0x00DF, 0x0155, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x013A, 0x0107, 0x00E7, 0x010D, 0x00E9, 0x0119, 0x00EB,
0x011B, 0x00ED, 0x00EE, 0x010F, 0x0111, 0x0144, 0x0148, 0x00F3, 0x00F4, 0x0151, 0x00F6, 0x00F7, 0x0159, 0x016F, 0x00FA, 0x0171, 0x00FC, 0x00FD,
0x0163, 0x02D9,
];
/// Windows-1251 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1251: [u16; 128] = [
0x0402, 0x0403, 0x201A, 0x0453, 0x201E, 0x2026, 0x2020, 0x2021, 0x20AC, 0x2030, 0x0409, 0x2039, 0x040A, 0x040C, 0x040B, 0x040F, 0x0452, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0x0459, 0x203A, 0x045A, 0x045C, 0x045B, 0x045F, 0x00A0, 0x040E, 0x045E, 0x0408,
0x00A4, 0x0490, 0x00A6, 0x00A7, 0x0401, 0x00A9, 0x0404, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x0407, 0x00B0, 0x00B1, 0x0406, 0x0456, 0x0491, 0x00B5,
0x00B6, 0x00B7, 0x0451, 0x2116, 0x0454, 0x00BB, 0x0458, 0x0405, 0x0455, 0x0457, 0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417,
0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421, 0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429,
0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433, 0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B,
0x043C, 0x043D, 0x043E, 0x043F, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B, 0x044C, 0x044D,
0x044E, 0x044F,
];
/// Windows-1253 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1253: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x0385, 0x0386, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0xFFFD, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x2015, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x0384, 0x00B5,
0x00B6, 0x00B7, 0x0388, 0x0389, 0x038A, 0x00BB, 0x038C, 0x00BD, 0x038E, 0x038F, 0x0390, 0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397,
0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, 0xFFFD, 0x03A3, 0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9,
0x03AA, 0x03AB, 0x03AC, 0x03AD, 0x03AE, 0x03AF, 0x03B0, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB,
0x03BC, 0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C2, 0x03C3, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x03C9, 0x03CA, 0x03CB, 0x03CC, 0x03CD,
0x03CE, 0xFFFD,
];
/// Windows-1254 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1254: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0xFFFD, 0xFFFD, 0x0178, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x00C3, 0x00C4, 0x00C5, 0x00C6, 0x00C7,
0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x00CC, 0x00CD, 0x00CE, 0x00CF, 0x011E, 0x00D1, 0x00D2, 0x00D3, 0x00D4, 0x00D5, 0x00D6, 0x00D7, 0x00D8, 0x00D9,
0x00DA, 0x00DB, 0x00DC, 0x0130, 0x015E, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x00E3, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x00EC, 0x00ED, 0x00EE, 0x00EF, 0x011F, 0x00F1, 0x00F2, 0x00F3, 0x00F4, 0x00F5, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x0131,
0x015F, 0x00FF,
];
/// Windows-1255 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1255: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x20AA, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00D7, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00F7, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x05B0, 0x05B1, 0x05B2, 0x05B3, 0x05B4, 0x05B5, 0x05B6, 0x05B7,
0x05B8, 0x05B9, 0x05BA, 0x05BB, 0x05BC, 0x05BD, 0x05BE, 0x05BF, 0x05C0, 0x05C1, 0x05C2, 0x05C3, 0x05F0, 0x05F1, 0x05F2, 0x05F3, 0x05F4, 0xFFFD,
0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x05D0, 0x05D1, 0x05D2, 0x05D3, 0x05D4, 0x05D5, 0x05D6, 0x05D7, 0x05D8, 0x05D9, 0x05DA, 0x05DB,
0x05DC, 0x05DD, 0x05DE, 0x05DF, 0x05E0, 0x05E1, 0x05E2, 0x05E3, 0x05E4, 0x05E5, 0x05E6, 0x05E7, 0x05E8, 0x05E9, 0x05EA, 0xFFFD, 0xFFFD, 0x200E,
0x200F, 0xFFFD,
];
/// Windows-1256 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1256: [u16; 128] = [
0x20AC, 0x067E, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0679, 0x2039, 0x0152, 0x0686, 0x0698, 0x0688, 0x06AF, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x06A9, 0x2122, 0x0691, 0x203A, 0x0153, 0x200C, 0x200D, 0x06BA, 0x00A0, 0x060C, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x06BE, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x061B, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x061F, 0x06C1, 0x0621, 0x0622, 0x0623, 0x0624, 0x0625, 0x0626, 0x0627,
0x0628, 0x0629, 0x062A, 0x062B, 0x062C, 0x062D, 0x062E, 0x062F, 0x0630, 0x0631, 0x0632, 0x0633, 0x0634, 0x0635, 0x0636, 0x00D7, 0x0637, 0x0638,
0x0639, 0x063A, 0x0640, 0x0641, 0x0642, 0x0643, 0x00E0, 0x0644, 0x00E2, 0x0645, 0x0646, 0x0647, 0x0648, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0649, 0x064A, 0x00EE, 0x00EF, 0x064B, 0x064C, 0x064D, 0x064E, 0x00F4, 0x064F, 0x0650, 0x00F7, 0x0651, 0x00F9, 0x0652, 0x00FB, 0x00FC, 0x200E,
0x200F, 0x06D2,
];
/// Windows-1257 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1257: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0xFFFD, 0x201E, 0x2026, 0x2020, 0x2021, 0xFFFD, 0x2030, 0xFFFD, 0x2039, 0xFFFD, 0x00A8, 0x02C7, 0x00B8, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0xFFFD, 0x2122, 0xFFFD, 0x203A, 0xFFFD, 0x00AF, 0x02DB, 0xFFFD, 0x00A0, 0xFFFD, 0x00A2, 0x00A3,
0x00A4, 0xFFFD, 0x00A6, 0x00A7, 0x00D8, 0x00A9, 0x0156, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00C6, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00F8, 0x00B9, 0x0157, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00E6, 0x0104, 0x012E, 0x0100, 0x0106, 0x00C4, 0x00C5, 0x0118, 0x0112,
0x010C, 0x00C9, 0x0179, 0x0116, 0x0122, 0x0136, 0x012A, 0x013B, 0x0160, 0x0143, 0x0145, 0x00D3, 0x014C, 0x00D5, 0x00D6, 0x00D7, 0x0172, 0x0141,
0x015A, 0x016A, 0x00DC, 0x017B, 0x017D, 0x00DF, 0x0105, 0x012F, 0x0101, 0x0107, 0x00E4, 0x00E5, 0x0119, 0x0113, 0x010D, 0x00E9, 0x017A, 0x0117,
0x0123, 0x0137, 0x012B, 0x013C, 0x0161, 0x0144, 0x0146, 0x00F3, 0x014D, 0x00F5, 0x00F6, 0x00F7, 0x0173, 0x0142, 0x015B, 0x016B, 0x00FC, 0x017C,
0x017E, 0x02D9,
];
/// Windows-1258 code points for 0x80..=0xFF (U+FFFD where the byte is unassigned).
const CP1258: [u16; 128] = [
0x20AC, 0xFFFD, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0xFFFD, 0x2039, 0x0152, 0xFFFD, 0xFFFD, 0xFFFD, 0xFFFD, 0x2018,
0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0xFFFD, 0x203A, 0x0153, 0xFFFD, 0xFFFD, 0x0178, 0x00A0, 0x00A1, 0x00A2, 0x00A3,
0x00A4, 0x00A5, 0x00A6, 0x00A7, 0x00A8, 0x00A9, 0x00AA, 0x00AB, 0x00AC, 0x00AD, 0x00AE, 0x00AF, 0x00B0, 0x00B1, 0x00B2, 0x00B3, 0x00B4, 0x00B5,
0x00B6, 0x00B7, 0x00B8, 0x00B9, 0x00BA, 0x00BB, 0x00BC, 0x00BD, 0x00BE, 0x00BF, 0x00C0, 0x00C1, 0x00C2, 0x0102, 0x00C4, 0x00C5, 0x00C6, 0x00C7,
0x00C8, 0x00C9, 0x00CA, 0x00CB, 0x0300, 0x00CD, 0x00CE, 0x00CF, 0x0110, 0x00D1, 0x0309, 0x00D3, 0x00D4, 0x01A0, 0x00D6, 0x00D7, 0x00D8, 0x00D9,
0x00DA, 0x00DB, 0x00DC, 0x01AF, 0x0303, 0x00DF, 0x00E0, 0x00E1, 0x00E2, 0x0103, 0x00E4, 0x00E5, 0x00E6, 0x00E7, 0x00E8, 0x00E9, 0x00EA, 0x00EB,
0x0301, 0x00ED, 0x00EE, 0x00EF, 0x0111, 0x00F1, 0x0323, 0x00F3, 0x00F4, 0x01A1, 0x00F6, 0x00F7, 0x00F8, 0x00F9, 0x00FA, 0x00FB, 0x00FC, 0x01B0,
0x20AB, 0x00FF,
];
/// Mac OS Roman (code page 10000, RTF `\mac`) code points for 0x80..=0xFF, with the euro sign at
/// 0xDB (Mac OS 8.5 and later).
const MAC_ROMAN: [u16; 128] = [
0x00C4, 0x00C5, 0x00C7, 0x00C9, 0x00D1, 0x00D6, 0x00DC, 0x00E1, 0x00E0, 0x00E2, 0x00E4, 0x00E3, 0x00E5, 0x00E7, 0x00E9, 0x00E8, 0x00EA, 0x00EB,
0x00ED, 0x00EC, 0x00EE, 0x00EF, 0x00F1, 0x00F3, 0x00F2, 0x00F4, 0x00F6, 0x00F5, 0x00FA, 0x00F9, 0x00FB, 0x00FC, 0x2020, 0x00B0, 0x00A2, 0x00A3,
0x00A7, 0x2022, 0x00B6, 0x00DF, 0x00AE, 0x00A9, 0x2122, 0x00B4, 0x00A8, 0x2260, 0x00C6, 0x00D8, 0x221E, 0x00B1, 0x2264, 0x2265, 0x00A5, 0x00B5,
0x2202, 0x2211, 0x220F, 0x03C0, 0x222B, 0x00AA, 0x00BA, 0x03A9, 0x00E6, 0x00F8, 0x00BF, 0x00A1, 0x00AC, 0x221A, 0x0192, 0x2248, 0x2206, 0x00AB,
0x00BB, 0x2026, 0x00A0, 0x00C0, 0x00C3, 0x00D5, 0x0152, 0x0153, 0x2013, 0x2014, 0x201C, 0x201D, 0x2018, 0x2019, 0x00F7, 0x25CA, 0x00FF, 0x0178,
0x2044, 0x20AC, 0x2039, 0x203A, 0xFB01, 0xFB02, 0x2021, 0x00B7, 0x201A, 0x201E, 0x2030, 0x00C2, 0x00CA, 0x00C1, 0x00CB, 0x00C8, 0x00CD, 0x00CE,
0x00CF, 0x00CC, 0x00D3, 0x00D4, 0xF8FF, 0x00D2, 0x00DA, 0x00DB, 0x00D9, 0x0131, 0x02C6, 0x02DC, 0x00AF, 0x02D8, 0x02D9, 0x02DA, 0x00B8, 0x02DD,
0x02DB, 0x02C7,
];
/// IBM PC (OEM) code page 437 code points for 0x80..=0xFF (the Unicode Consortium's CP437
/// mapping file).
const CP437: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x00EC, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x00FF, 0x00D6, 0x00DC, 0x00A2, 0x00A3, 0x00A5, 0x20A7, 0x0192, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x00AA, 0x00BA, 0x00BF, 0x2310, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x03B1, 0x00DF, 0x0393, 0x03C0, 0x03A3, 0x03C3, 0x00B5, 0x03C4, 0x03A6, 0x0398, 0x03A9, 0x03B4,
0x221E, 0x03C6, 0x03B5, 0x2229, 0x2261, 0x00B1, 0x2265, 0x2264, 0x2320, 0x2321, 0x00F7, 0x2248, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 850 (DOS Latin-1, Western European) code points for 0x80..=0xFF (the Unicode
/// Consortium's CP850 mapping file).
const CP850: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x00EC, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x00FF, 0x00D6, 0x00DC, 0x00F8, 0x00A3, 0x00D8, 0x00D7, 0x0192, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x00AA, 0x00BA, 0x00BF, 0x00AE, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x00C0, 0x00A9, 0x2563, 0x2551, 0x2557, 0x255D, 0x00A2, 0x00A5, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x00E3, 0x00C3,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x00F0, 0x00D0, 0x00CA, 0x00CB, 0x00C8, 0x0131, 0x00CD, 0x00CE, 0x00CF, 0x2518,
0x250C, 0x2588, 0x2584, 0x00A6, 0x00CC, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x00D2, 0x00F5, 0x00D5, 0x00B5, 0x00FE, 0x00DE, 0x00DA, 0x00DB, 0x00D9,
0x00FD, 0x00DD, 0x00AF, 0x00B4, 0x00AD, 0x00B1, 0x2017, 0x00BE, 0x00B6, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x00B7, 0x00B9, 0x00B3, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 852 (DOS Latin-2, Central European) code points for 0x80..=0xFF (the Unicode Consortium's CP852
/// mapping file).
const CP852: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x016F, 0x0107, 0x00E7, 0x0142, 0x00EB, 0x0150, 0x0151, 0x00EE, 0x0179, 0x00C4, 0x0106, 0x00C9, 0x0139,
0x013A, 0x00F4, 0x00F6, 0x013D, 0x013E, 0x015A, 0x015B, 0x00D6, 0x00DC, 0x0164, 0x0165, 0x0141, 0x00D7, 0x010D, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x0104, 0x0105, 0x017D, 0x017E, 0x0118, 0x0119, 0x00AC, 0x017A, 0x010C, 0x015F, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x011A, 0x015E, 0x2563, 0x2551, 0x2557, 0x255D, 0x017B, 0x017C, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x0102, 0x0103,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x0111, 0x0110, 0x010E, 0x00CB, 0x010F, 0x0147, 0x00CD, 0x00CE, 0x011B, 0x2518,
0x250C, 0x2588, 0x2584, 0x0162, 0x016E, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x0143, 0x0144, 0x0148, 0x0160, 0x0161, 0x0154, 0x00DA, 0x0155, 0x0170,
0x00FD, 0x00DD, 0x0163, 0x00B4, 0x00AD, 0x02DD, 0x02DB, 0x02C7, 0x02D8, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x02D9, 0x0171, 0x0158, 0x0159,
0x25A0, 0x00A0,
];
/// OEM code page 866 (DOS Cyrillic) code points for 0x80..=0xFF (the Unicode Consortium's CP866
/// mapping file).
const CP866: [u16; 128] = [
0x0410, 0x0411, 0x0412, 0x0413, 0x0414, 0x0415, 0x0416, 0x0417, 0x0418, 0x0419, 0x041A, 0x041B, 0x041C, 0x041D, 0x041E, 0x041F, 0x0420, 0x0421,
0x0422, 0x0423, 0x0424, 0x0425, 0x0426, 0x0427, 0x0428, 0x0429, 0x042A, 0x042B, 0x042C, 0x042D, 0x042E, 0x042F, 0x0430, 0x0431, 0x0432, 0x0433,
0x0434, 0x0435, 0x0436, 0x0437, 0x0438, 0x0439, 0x043A, 0x043B, 0x043C, 0x043D, 0x043E, 0x043F, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x0440, 0x0441, 0x0442, 0x0443, 0x0444, 0x0445, 0x0446, 0x0447, 0x0448, 0x0449, 0x044A, 0x044B,
0x044C, 0x044D, 0x044E, 0x044F, 0x0401, 0x0451, 0x0404, 0x0454, 0x0407, 0x0457, 0x040E, 0x045E, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x2116, 0x00A4,
0x25A0, 0x00A0,
];
/// OEM code page 737 (DOS Greek) code points for 0x80..=0xFF (the Unicode Consortium's CP737
/// mapping file).
const CP737: [u16; 128] = [
0x0391, 0x0392, 0x0393, 0x0394, 0x0395, 0x0396, 0x0397, 0x0398, 0x0399, 0x039A, 0x039B, 0x039C, 0x039D, 0x039E, 0x039F, 0x03A0, 0x03A1, 0x03A3,
0x03A4, 0x03A5, 0x03A6, 0x03A7, 0x03A8, 0x03A9, 0x03B1, 0x03B2, 0x03B3, 0x03B4, 0x03B5, 0x03B6, 0x03B7, 0x03B8, 0x03B9, 0x03BA, 0x03BB, 0x03BC,
0x03BD, 0x03BE, 0x03BF, 0x03C0, 0x03C1, 0x03C3, 0x03C2, 0x03C4, 0x03C5, 0x03C6, 0x03C7, 0x03C8, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x2561,
0x2562, 0x2556, 0x2555, 0x2563, 0x2551, 0x2557, 0x255D, 0x255C, 0x255B, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x255E, 0x255F,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x2567, 0x2568, 0x2564, 0x2565, 0x2559, 0x2558, 0x2552, 0x2553, 0x256B, 0x256A, 0x2518,
0x250C, 0x2588, 0x2584, 0x258C, 0x2590, 0x2580, 0x03C9, 0x03AC, 0x03AD, 0x03AE, 0x03CA, 0x03AF, 0x03CC, 0x03CD, 0x03CB, 0x03CE, 0x0386, 0x0388,
0x0389, 0x038A, 0x038C, 0x038E, 0x038F, 0x00B1, 0x2265, 0x2264, 0x03AA, 0x03AB, 0x00F7, 0x2248, 0x00B0, 0x2219, 0x00B7, 0x221A, 0x207F, 0x00B2,
0x25A0, 0x00A0,
];
/// OEM code page 857 (DOS Turkish; U+FFFD where the byte is unassigned) code points for 0x80..=0xFF (the Unicode Consortium's CP857
/// mapping file).
const CP857: [u16; 128] = [
0x00C7, 0x00FC, 0x00E9, 0x00E2, 0x00E4, 0x00E0, 0x00E5, 0x00E7, 0x00EA, 0x00EB, 0x00E8, 0x00EF, 0x00EE, 0x0131, 0x00C4, 0x00C5, 0x00C9, 0x00E6,
0x00C6, 0x00F4, 0x00F6, 0x00F2, 0x00FB, 0x00F9, 0x0130, 0x00D6, 0x00DC, 0x00F8, 0x00A3, 0x00D8, 0x015E, 0x015F, 0x00E1, 0x00ED, 0x00F3, 0x00FA,
0x00F1, 0x00D1, 0x011E, 0x011F, 0x00BF, 0x00AE, 0x00AC, 0x00BD, 0x00BC, 0x00A1, 0x00AB, 0x00BB, 0x2591, 0x2592, 0x2593, 0x2502, 0x2524, 0x00C1,
0x00C2, 0x00C0, 0x00A9, 0x2563, 0x2551, 0x2557, 0x255D, 0x00A2, 0x00A5, 0x2510, 0x2514, 0x2534, 0x252C, 0x251C, 0x2500, 0x253C, 0x00E3, 0x00C3,
0x255A, 0x2554, 0x2569, 0x2566, 0x2560, 0x2550, 0x256C, 0x00A4, 0x00BA, 0x00AA, 0x00CA, 0x00CB, 0x00C8, 0xFFFD, 0x00CD, 0x00CE, 0x00CF, 0x2518,
0x250C, 0x2588, 0x2584, 0x00A6, 0x00CC, 0x2580, 0x00D3, 0x00DF, 0x00D4, 0x00D2, 0x00F5, 0x00D5, 0x00B5, 0xFFFD, 0x00D7, 0x00DA, 0x00DB, 0x00D9,
0x00EC, 0x00FF, 0x00AF, 0x00B4, 0x00AD, 0x00B1, 0xFFFD, 0x00BE, 0x00B6, 0x00A7, 0x00F7, 0x00B8, 0x00B0, 0x00A8, 0x00B7, 0x00B9, 0x00B3, 0x00B2,
0x25A0, 0x00A0,
];
/// The glyphs the OEM code pages show for 0x01..=0x1F (smileys, card suits, arrows), as
/// Windows' Alt+numpad entry produces them rather than the control characters.
const OEM_GLYPHS: [u16; 31] = [
0x263A, 0x263B, 0x2665, 0x2666, 0x2663, 0x2660, 0x2022, 0x25D8, 0x25CB, 0x25D9, 0x2642, 0x2640, 0x266A, 0x266B, 0x263C, 0x25BA, 0x25C4, 0x2195,
0x203C, 0x00B6, 0x00A7, 0x25AC, 0x21A8, 0x2191, 0x2193, 0x2192, 0x2190, 0x221F, 0x2194, 0x25B2, 0x25BC,
];
/// The OEM code pages [`oem_byte`] decodes.
pub const OEM_CODE_PAGES: [u16; 6] = [437, 737, 850, 852, 857, 866];
/// One byte of an OEM code page in [`OEM_CODE_PAGES`] (any other code page reads as 437) as a
/// char, with the graphic glyphs for 0x01..=0x1F and 0x7F (⌂); 0x00 stays NUL.
pub fn oem_byte(code_page: u16, b: u8) -> char {
let table: &[u16; 128] = match code_page {
737 => &CP737,
850 => &CP850,
852 => &CP852,
857 => &CP857,
866 => &CP866,
_ => &CP437,
};
let u = match b {
0x01..=0x1F => OEM_GLYPHS.get(usize::from(b - 1)).copied(),
0x7F => Some(0x2302),
0x80..=0xFF => table.get(usize::from(b - 0x80)).copied(),
_ => Some(u16::from(b)),
};
u.and_then(|u| char::from_u32(u32::from(u))).unwrap_or('\u{FFFD}')
}
/// The single-byte Windows code pages [`windows_byte`] decodes (1252 included).
pub const SINGLE_BYTE_CODE_PAGES: [u16; 10] = [874, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258];
/// Mac OS Roman, RTF's `\mac` character set.
pub const MAC_ROMAN_CODE_PAGE: u16 = 10000;
/// One byte of a single-byte code page as a char: a Windows one in [`SINGLE_BYTE_CODE_PAGES`] or
/// Mac OS Roman ([`MAC_ROMAN_CODE_PAGE`]); any other code page reads as Windows-1252.
pub fn windows_byte(code_page: u16, b: u8) -> char {
let table: &[u16; 128] = match code_page {
874 => &CP874,
1250 => &CP1250,
1251 => &CP1251,
1253 => &CP1253,
1254 => &CP1254,
1255 => &CP1255,
1256 => &CP1256,
1257 => &CP1257,
1258 => &CP1258,
MAC_ROMAN_CODE_PAGE => &MAC_ROMAN,
_ => return cp1252(b),
};
if b < 0x80 {
return b as char;
}
table.get(usize::from(b - 0x80)).and_then(|u| char::from_u32(u32::from(*u))).unwrap_or('\u{FFFD}')
}
/// Whether [`code_page_byte`] decodes `code_page`: a Windows one ([`SINGLE_BYTE_CODE_PAGES`]), an
/// OEM (DOS) one ([`OEM_CODE_PAGES`]) or Mac OS Roman. Document formats (RTF) name all three.
pub fn is_byte_code_page(code_page: u16) -> bool {
SINGLE_BYTE_CODE_PAGES.contains(&code_page) || OEM_CODE_PAGES.contains(&code_page) || code_page == MAC_ROMAN_CODE_PAGE
}
/// A byte of 8-bit document text in `code_page` (see [`is_byte_code_page`]) as a char. Below 0x80
/// it is ASCII in every code page (text, so an OEM page's glyphs for control codes don't apply).
pub fn code_page_byte(code_page: u16, b: u8) -> char {
if b >= 0x80 && OEM_CODE_PAGES.contains(&code_page) { oem_byte(code_page, b) } else { windows_byte(code_page, b) }
}
/// The East Asian multibyte code pages a document format names with `\ansicpg` or a font's `\cpg`:
/// 932 Shift-JIS (Japanese), 936 GBK (Simplified Chinese), 949 UHC (Korean) and 950 Big5
/// (Traditional Chinese).
pub const MULTIBYTE_CODE_PAGES: [u16; 4] = [932, 936, 949, 950];
/// Whether `code_page` is one of the East Asian multibyte code pages ([`MULTIBYTE_CODE_PAGES`]):
/// its bytes decode as a run, a lead byte together with its trail byte.
pub fn is_multibyte_code_page(code_page: u16) -> bool {
multibyte_encoding_name(code_page).is_some()
}
/// The WHATWG encoding name of an East Asian multibyte code page (see [`MULTIBYTE_CODE_PAGES`]),
/// for `encoding_rs::Encoding::for_label`; `None` for the single-byte code pages, whose bytes
/// decode one at a time, and anything else.
pub fn multibyte_encoding_name(code_page: u16) -> Option<&'static str> {
Some(match code_page {
932 => "shift_jis",
936 => "gbk",
949 => "euc-kr",
950 => "big5",
_ => return None,
})
}
/// Whether a document format can name `code_page` for its text: a single-byte Windows, DOS or Mac
/// one ([`is_byte_code_page`]) or an East Asian multibyte one ([`is_multibyte_code_page`]).
pub fn is_text_code_page(code_page: u16) -> bool {
is_byte_code_page(code_page) || is_multibyte_code_page(code_page)
}
/// The code page of an RTF or Word font charset (`\fcharset`): 204 Russian → 1251, 238 Eastern
/// European → 1250, 77 Mac → 10000, 254 PC / 255 OEM → 437, 128 Japanese → 932, 129 Korean → 949,
/// 134 Simplified Chinese → 936, 136 Traditional Chinese → 950… `None` for the ANSI and default
/// charsets (the document's code page applies) and symbol fonts.
pub fn charset_code_page(charset: i32) -> Option<u16> {
Some(match charset {
77 => MAC_ROMAN_CODE_PAGE,
128 => 932,
129 => 949,
134 => 936,
136 => 950,
161 => 1253,
162 => 1254,
163 => 1258,
177 => 1255,
178 => 1256,
186 => 1257,
204 => 1251,
222 => 874,
238 => 1250,
254 | 255 => 437,
_ => return None,
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn single_byte_code_pages() {
let decode = |cp: u16, bytes: &[u8]| bytes.iter().map(|&b| windows_byte(cp, b)).collect::<String>();
assert_eq!(decode(1251, &[0xD1, 0xF7, 0xE5, 0xF2, 0x20, 0xB9]), "Счет №");
assert_eq!(decode(1251, &[0xA8, 0xB8]), "Ёё");
assert_eq!(decode(1250, &[0xA3, 0xF3, 0x9F]), "Łóź");
assert_eq!(decode(1253, &[0xC1, 0xE2]), "Αβ");
assert_eq!(decode(1252, &[0x80, 0xE9]), "ێ");
assert_eq!(decode(9999, &[0xE9]), "é");
// Mac OS Roman (#1653): accents, the euro sign at 0xDB, typographic quotes.
assert_eq!(decode(10000, &[0x80, 0x8E, 0xDB, 0xF1, 0xF2, 0xF3, 0xD2, 0xD3, 0x41]), "Äé€ÒÚÛ“”A");
assert_eq!(charset_code_page(77), Some(10000));
assert_eq!(charset_code_page(254), Some(437));
assert_eq!(charset_code_page(255), Some(437));
assert_eq!(charset_code_page(204), Some(1251));
assert_eq!(charset_code_page(0), None);
for cp in SINGLE_BYTE_CODE_PAGES {
for b in 0..=255u8 {
let _ = windows_byte(cp, b);
}
}
}
#[test]
fn windows_1255_holam_haser_for_vav() {
assert_eq!(windows_byte(1255, 0xCA), '\u{05BA}');
}
/// #1653: one decoder for document text in any code page RTF names: Windows, DOS or Mac.
#[test]
fn document_text_in_windows_dos_and_mac_code_pages() {
let decode = |cp: u16, bytes: &[u8]| bytes.iter().map(|&b| code_page_byte(cp, b)).collect::<String>();
assert_eq!(decode(437, &[0x82, 0xA0, 0x41, 0x01]), "éáA\u{1}", "DOS text; ASCII stays ASCII");
assert_eq!(decode(850, &[0x9D, 0xE9]), "ØÚ");
assert_eq!(decode(10000, &[0x8E]), "é");
assert_eq!(decode(1251, &[0xC4]), "Д");
for cp in [437, 850, 866, 1250, 1252, 10000] {
assert!(is_byte_code_page(cp), "{cp}");
}
assert!(!is_byte_code_page(932) && !is_byte_code_page(0) && !SINGLE_BYTE_CODE_PAGES.contains(&10000));
}
/// #1653: the East Asian charsets and code pages RTF names are multibyte: their bytes decode
/// as runs (a lead byte and its trail byte together), not one at a time.
#[test]
fn east_asian_code_pages_are_multibyte() {
assert_eq!(charset_code_page(128), Some(932), "Japanese");
assert_eq!(charset_code_page(129), Some(949), "Korean");
assert_eq!(charset_code_page(134), Some(936), "Simplified Chinese");
assert_eq!(charset_code_page(136), Some(950), "Traditional Chinese");
assert_eq!(charset_code_page(255), Some(437), "OEM is single-byte");
assert_eq!(MULTIBYTE_CODE_PAGES, [932, 936, 949, 950]);
for cp in MULTIBYTE_CODE_PAGES {
assert!(is_multibyte_code_page(cp) && is_text_code_page(cp), "{cp}");
assert!(!is_byte_code_page(cp) && !SINGLE_BYTE_CODE_PAGES.contains(&cp), "{cp}");
assert!(multibyte_encoding_name(cp).is_some(), "{cp}");
}
assert!(is_text_code_page(1251) && is_text_code_page(437) && is_text_code_page(10000));
assert!(!is_text_code_page(0) && !is_text_code_page(9999));
}
#[test]
fn oem_code_pages() {
let decode = |cp: u16, bytes: &[u8]| bytes.iter().map(|&b| oem_byte(cp, b)).collect::<String>();
assert_eq!(decode(437, &[0x82, 0xE9, 0x01, 0x7F]), "éΘ☺⌂");
assert_eq!(decode(850, &[0x9B, 0xE9]), "øÚ");
assert_eq!(decode(852, &[0xA5, 0x88, 0xBE, 0xA9]), "ąłżę");
assert_eq!(decode(866, &[0x8F, 0xE0, 0xA8, 0xA2, 0xA5, 0xE2, 0xF0, 0xFC]), "ПриветЁ№");
assert_eq!(decode(737, &[0x80, 0x99, 0xE0]), "Αβω");
assert_eq!(decode(857, &[0x8D, 0x98, 0xA6, 0x9F]), "ıİĞş");
assert_eq!(oem_byte(857, 0xD5), '\u{FFFD}', "unassigned in 857");
assert_eq!(decode(9999, &[0xE9]), "Θ", "unknown pages read as 437");
for cp in OEM_CODE_PAGES {
for b in 0..=255u8 {
let _ = oem_byte(cp, b);
}
}
}
}
+4
View File
@@ -144,6 +144,7 @@ fn parts_and_media() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn many_distinct_media_add_in_linear_time() {
// An import with tens of thousands of small distinct images (same length, differing only at
// the end) used to compare each against every earlier one.
@@ -645,6 +646,7 @@ fn para_of_objects(objs: Vec<InlineObject>) -> Paragraph {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn object_walks_are_linear_in_the_paragraph() {
// #3614: loops of object_offsets + object_at recounted the objects before each one, so a
// paragraph of 40,000 markers took seconds per walk.
@@ -691,6 +693,7 @@ fn object_walks_are_linear_in_the_paragraph() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn deleting_across_many_lone_control_markers_is_not_quadratic() {
// A millisecond now; the old delete took 18 s at 1,500 markers and 114 s at 3,000.
let n = 3_000;
@@ -725,6 +728,7 @@ fn control_markers_kept_are_sorted() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn lone_table_open_markers_linear_time() {
let mut d = Document::new();
let mut t = Table::new(1, 1, 100.0);
+145 -119
View File
@@ -1,131 +1,21 @@
//! A bytes-only check of a ZIP package's declared entry count before the `zip` crate opens it
//! (#3660). Every reader of `.docx`, `.odt` and spreadsheet packages calls [`check`] first.
//!
//! `zip::ZipArchive::new` reserves room for, and parses, as many central directory headers as the
//! end record declares before its caller can look at `len()`. A ZIP64 end record can declare
//! millions of entries for a file of a few hundred megabytes (only 46 bytes each have to fit), and
//! each parsed entry costs several times that in memory. Here the end records are found the way
//! `zip` finds them (APPNOTE.TXT §4.3.14 to §4.3.16), and a package declaring more than the
//! reader's limit is refused before `zip` parses anything. Anything else that is wrong is left for
//! `zip` to report.
const EOCD: &[u8; 4] = b"PK\x05\x06";
const EOCD64_LOCATOR: &[u8; 4] = b"PK\x06\x07";
const EOCD64: &[u8; 4] = b"PK\x06\x06";
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
/// End records looked at, at most. A package has one; a few more can turn up inside stored data.
const MAX_END_RECORDS: usize = 16;
//! A bytes-only check of a ZIP package's declared entry count, run before anything opens the ZIP
//! (#3660). The logic lives in `craft-zip`; this wraps its error as a `String`.
/// Refuse `bytes` when the end record `zip` would use declares more than `max_entries` entries.
///
/// `zip` searches the file backwards for an end of central directory record. A ZIP64 one points
/// through its locator at the ZIP64 end record, which `zip` looks for from the locator's offset
/// on: every such record there is checked, and as `zip` may fall back to an earlier end record
/// when one doesn't parse, the search goes on. A plain one is used when it declares no entries, or
/// a central directory header lies between its directory offset and itself: it is the last one
/// looked at.
pub fn check(bytes: &[u8], max_entries: u64) -> Result<(), String> {
let u16_at = |at: usize| bytes.get(at..at.checked_add(2)?).map(|b| u16::from_le_bytes([b[0], b[1]]));
let u32_at = |at: usize| bytes.get(at..at.checked_add(4)?).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]));
let u64_at = |at: usize| bytes.get(at..at.checked_add(8)?).and_then(|b| <[u8; 8]>::try_from(b).ok()).map(u64::from_le_bytes);
let refuse = |n: u64| -> Result<(), String> {
if n > max_entries { Err(format!("the package declares {n} entries, more than {max_entries}")) } else { Ok(()) }
};
let mut end = bytes.len();
let mut seen = 0;
while let Some(at) = bytes.get(..end).and_then(|b| b.windows(4).rposition(|w| w == EOCD)) {
end = at;
seen += 1;
if seen > MAX_END_RECORDS {
return Err(format!("more than {MAX_END_RECORDS} end of central directory records"));
}
// `zip` skips a record whose comment runs past the end of the file.
let Some(comment) = u16_at(at + 20) else { continue };
if at + 22 + usize::from(comment) > bytes.len() {
continue;
}
let (Some(on_disk), Some(total), Some(cd_offset)) = (u16_at(at + 8), u16_at(at + 10), u32_at(at + 16)) else { continue };
let zip64 = total == u16::MAX || cd_offset == u32::MAX;
if zip64
&& let Some(locator) = at.checked_sub(20)
&& bytes.get(locator..locator + 4) == Some(EOCD64_LOCATOR.as_slice())
{
let (Some(eocd64), Some(disks)) = (u64_at(locator + 8), u32_at(locator + 16)) else { continue };
let Some(from) = usize::try_from(eocd64).ok().filter(|&f| f < locator) else { continue };
if disks > 1 {
continue;
}
let records = bytes.get(from..locator).unwrap_or_default().windows(4).enumerate().filter(|(_, w)| *w == EOCD64);
for (i, _) in records {
let p = from + i;
refuse(u64_at(p + 24).unwrap_or(0).max(u64_at(p + 32).unwrap_or(0)))?;
}
continue;
}
if total == 0 {
return Ok(());
}
let cd = cd_offset as usize;
if cd < at && bytes.get(cd..at).is_some_and(|b| b.windows(4).any(|w| w == CENTRAL_HEADER)) {
return refuse(u64::from(on_disk.max(total)));
}
}
Ok(())
craft_zip::check_entry_count(bytes, max_entries).map_err(|e| e.to_string())
}
/// A package of one empty stored file, `a`, whose central directory lists it `entries` times (at
/// most as many as fit in 64 MiB) under a ZIP64 end record that declares them all: `zip` 3.0.0
/// parses and keeps every one of them before its caller can count them. Regression input for the
/// readers' tests: built only for tests, this crate's and (through the `test-support` feature,
/// which they turn on as a dev-dependency) the readers', never into the app.
/// A ZIP64 package declaring `entries` entries: regression input for the readers' tests.
#[cfg(any(test, feature = "test-support"))]
pub fn zip64_entry_count_sample(entries: u64) -> Vec<u8> {
// Local file header (APPNOTE.TXT §4.3.7): version 10, stored, no data, name "a".
let mut f = b"PK\x03\x04".to_vec();
f.extend_from_slice(&[10, 0, 0, 0, 0, 0, 0, 0, 0x21, 0]); // version, flags, method, time, date
f.extend_from_slice(&[0; 12]); // crc, sizes
f.extend_from_slice(&[1, 0, 0, 0]); // name and extra lengths
f.push(b'a');
let cd_at = f.len() as u64;
// Its central header (§4.3.12), 47 bytes.
let mut header = CENTRAL_HEADER.to_vec();
header.extend_from_slice(&[10, 0, 10, 0, 0, 0, 0, 0, 0, 0, 0x21, 0]); // versions, flags, method, time, date
header.extend_from_slice(&[0; 12]); // crc, sizes
header.extend_from_slice(&[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]); // name, extra, comment, disk, attributes
header.extend_from_slice(&[0; 4]); // local header offset
header.push(b'a');
let count = entries.min((64 << 20) / header.len() as u64);
for _ in 0..count {
f.extend_from_slice(&header);
}
let cd_len = f.len() as u64 - cd_at;
let eocd64_at = f.len() as u64;
// ZIP64 end of central directory record (§4.3.14), 56 bytes.
f.extend_from_slice(EOCD64);
f.extend_from_slice(&44u64.to_le_bytes()); // size of the rest of the record
f.extend_from_slice(&[45, 0, 45, 0]); // versions
f.extend_from_slice(&[0; 8]); // this disk, disk with the central directory
f.extend_from_slice(&entries.to_le_bytes()); // entries on this disk
f.extend_from_slice(&entries.to_le_bytes()); // entries
f.extend_from_slice(&cd_len.to_le_bytes());
f.extend_from_slice(&cd_at.to_le_bytes());
// Its locator (§4.3.15), 20 bytes.
f.extend_from_slice(EOCD64_LOCATOR);
f.extend_from_slice(&0u32.to_le_bytes());
f.extend_from_slice(&eocd64_at.to_le_bytes());
f.extend_from_slice(&1u32.to_le_bytes());
// End of central directory record (§4.3.16), 22 bytes, each count and offset "see ZIP64".
f.extend_from_slice(EOCD);
f.extend_from_slice(&[0; 4]);
f.extend_from_slice(&[0xFF; 12]);
f.extend_from_slice(&[0; 2]);
f
}
pub use craft_zip::zip64_entry_count_sample;
#[cfg(test)]
mod tests {
use super::{CENTRAL_HEADER, EOCD, check, zip64_entry_count_sample};
use super::{check, zip64_entry_count_sample};
const EOCD: &[u8; 4] = b"PK\x05\x06";
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
/// A plain package's tail: one central header at 0, then an end record declaring `entries`.
fn plain(entries: u16, comment: &[u8]) -> Vec<u8> {
@@ -188,4 +78,140 @@ mod tests {
let f: Vec<u8> = (0..20).flat_map(|_| EOCD.iter().copied().chain(record)).collect();
assert!(check(&f, 20_000).is_err_and(|e| e.contains("end of central directory records")));
}
/// The pre-cutover implementation, kept as a differential oracle.
mod old {
const EOCD: &[u8; 4] = b"PK\x05\x06";
const EOCD64_LOCATOR: &[u8; 4] = b"PK\x06\x07";
const EOCD64: &[u8; 4] = b"PK\x06\x06";
const CENTRAL_HEADER: &[u8; 4] = b"PK\x01\x02";
/// End records looked at, at most. A package has one; a few more can turn up inside stored data.
const MAX_END_RECORDS: usize = 16;
/// Refuse `bytes` when the end record `zip` would use declares more than `max_entries` entries.
///
/// `zip` searches the file backwards for an end of central directory record. A ZIP64 one points
/// through its locator at the ZIP64 end record, which `zip` looks for from the locator's offset
/// on: every such record there is checked, and as `zip` may fall back to an earlier end record
/// when one doesn't parse, the search goes on. A plain one is used when it declares no entries, or
/// a central directory header lies between its directory offset and itself: it is the last one
/// looked at.
pub fn check(bytes: &[u8], max_entries: u64) -> Result<(), String> {
let u16_at = |at: usize| bytes.get(at..at.checked_add(2)?).map(|b| u16::from_le_bytes([b[0], b[1]]));
let u32_at = |at: usize| bytes.get(at..at.checked_add(4)?).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]]));
let u64_at = |at: usize| bytes.get(at..at.checked_add(8)?).and_then(|b| <[u8; 8]>::try_from(b).ok()).map(u64::from_le_bytes);
let refuse = |n: u64| -> Result<(), String> {
if n > max_entries { Err(format!("the package declares {n} entries, more than {max_entries}")) } else { Ok(()) }
};
let mut end = bytes.len();
let mut seen = 0;
while let Some(at) = bytes.get(..end).and_then(|b| b.windows(4).rposition(|w| w == EOCD)) {
end = at;
seen += 1;
if seen > MAX_END_RECORDS {
return Err(format!("more than {MAX_END_RECORDS} end of central directory records"));
}
// `zip` skips a record whose comment runs past the end of the file.
let Some(comment) = u16_at(at + 20) else { continue };
if at + 22 + usize::from(comment) > bytes.len() {
continue;
}
let (Some(on_disk), Some(total), Some(cd_offset)) = (u16_at(at + 8), u16_at(at + 10), u32_at(at + 16)) else { continue };
let zip64 = total == u16::MAX || cd_offset == u32::MAX;
if zip64
&& let Some(locator) = at.checked_sub(20)
&& bytes.get(locator..locator + 4) == Some(EOCD64_LOCATOR.as_slice())
{
let (Some(eocd64), Some(disks)) = (u64_at(locator + 8), u32_at(locator + 16)) else { continue };
let Some(from) = usize::try_from(eocd64).ok().filter(|&f| f < locator) else { continue };
if disks > 1 {
continue;
}
let records = bytes.get(from..locator).unwrap_or_default().windows(4).enumerate().filter(|(_, w)| *w == EOCD64);
for (i, _) in records {
let p = from + i;
refuse(u64_at(p + 24).unwrap_or(0).max(u64_at(p + 32).unwrap_or(0)))?;
}
continue;
}
if total == 0 {
return Ok(());
}
let cd = cd_offset as usize;
if cd < at && bytes.get(cd..at).is_some_and(|b| b.windows(4).any(|w| w == CENTRAL_HEADER)) {
return refuse(u64::from(on_disk.max(total)));
}
}
Ok(())
}
}
fn sentinel_sample() -> Vec<u8> {
// ZIP64 sample whose plain end record claims one entry and a real directory offset but
// sets the directory size to u32::MAX, the ZIP64 marker `zip` honours.
let mut f = zip64_entry_count_sample(30_000);
let n = f.len();
f[n - 14..n - 12].copy_from_slice(&1u16.to_le_bytes());
f[n - 12..n - 10].copy_from_slice(&1u16.to_le_bytes());
f[n - 10..n - 6].copy_from_slice(&u32::MAX.to_le_bytes());
f[n - 6..n - 2].copy_from_slice(&31u32.to_le_bytes());
f
}
fn corpus() -> Vec<Vec<u8>> {
let mut seed = 0x2545_F491_4F6C_DD1Du64;
let mut rnd = |n: usize| -> Vec<u8> {
(0..n)
.map(|_| {
seed ^= seed << 13;
seed ^= seed >> 7;
seed ^= seed << 17;
(seed >> 24) as u8
})
.collect()
};
let mut truncated = zip64_entry_count_sample(10);
truncated.truncate(truncated.len() - 5);
vec![
vec![],
plain(3, b""),
plain(30_000, b"c"),
zip64_entry_count_sample(1),
zip64_entry_count_sample(10),
zip64_entry_count_sample(30_000),
zip64_entry_count_sample(5_000_000),
truncated,
rnd(1),
rnd(100),
rnd(5000),
]
}
#[test]
fn agrees_with_the_old_implementation_except_for_the_sentinel() {
for (i, input) in corpus().iter().enumerate() {
for max in [0u64, 9, 10, 20_000] {
assert_eq!(check(input, max).is_ok(), old::check(input, max).is_ok(), "input {i}, max {max}");
}
}
}
#[test]
fn a_directory_size_sentinel_alone_cannot_bypass_the_count_limit() {
let f = sentinel_sample();
assert!(old::check(&f, 20_000).is_ok(), "the old check missed it");
assert!(check(&f, 20_000).is_err_and(|e| e.contains("30000 entries")));
}
#[test]
fn no_truncation_of_a_zip64_package_panics() {
for f in [zip64_entry_count_sample(10), sentinel_sample()] {
for end in 0..=f.len().min(4096) {
let _ = check(&f[..end], 5);
}
for start in f.len().saturating_sub(300)..f.len() {
let _ = check(&f[start..], 5);
}
}
}
}
@@ -27,6 +27,7 @@ fn cpu_cost(mut f: impl FnMut()) -> f64 {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn many_comments_save_without_quadratic_id_searches() {
for imported in [true, false] {
let small = comments(4_000, imported);
@@ -74,6 +74,7 @@ fn cost_ratio(scale: usize, mut small: impl FnMut(), mut big: impl FnMut()) -> (
/// looked for each paragraph's markers in the whole set of markers, once per paragraph, so a
/// document with 32,000 controls took four seconds to save where it takes a tenth of one now.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn a_document_with_many_inline_content_controls_saves_in_linear_time() {
let scale = 4;
let small = inline_controls(8_000);
@@ -94,6 +95,7 @@ fn a_document_with_many_inline_content_controls_saves_in_linear_time() {
/// The same for the controls written around whole paragraphs: the writer compared every control
/// with every other one to find the ones that clash, so the save was quadratic in their number.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn a_document_with_many_block_content_controls_saves_in_linear_time() {
let scale = 4;
let small = block_controls(8_000);
+4 -2
View File
@@ -1280,10 +1280,12 @@ fn anchor_stacking_order_is_read_and_kept() {
paras(d)[0].objects.iter().filter_map(|o| if let InlineObject::Shape { float, .. } = o { Some(float.z) } else { None }).collect()
};
let d = read_body(&body);
assert_eq!(zs(&d), [251660288, 251657214, 0]);
// The shared reader keeps one more than the file's `relativeHeight` (so a file's 0 is the
// lowest order, not "none") and the writer writes one less.
assert_eq!(zs(&d), [251660289, 251657215, 0]);
let again = wordcraft_docx::read(&wordcraft_docx::write(&d).unwrap()).unwrap();
let z = zs(&again);
assert_eq!(&z[..2], &[251660288, 251657214]);
assert_eq!(&z[..2], &[251660289, 251657215]);
assert!(z[2] > 0, "an object without one gets an order on saving: {z:?}");
}
+2
View File
@@ -152,6 +152,7 @@ fn many_fields_and_huge_attribute_values() {
/// A paragraph with 2,000,000 attributes deflates to ~15 KB. Opening it is refused with a
/// limit error, or the extras are dropped and the text survives (#3702, WC-06).
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn millions_of_attributes_on_one_paragraph_are_capped() {
let attrs: String = (0..2_000_000).map(|i| format!(" a{i}=\"\"")).collect();
let x = format!("<w:document {W_NS}><w:body><w:p{attrs}><w:r><w:t>kept</w:t></w:r></w:p></w:body></w:document>");
@@ -222,6 +223,7 @@ fn malformed_or_hostile_bibliography_never_panics() {
/// A file with a huge number of bibliography sources (duplicates and distinct ones) reads in
/// linear time and keeps at most 10,000 of them.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn hostile_bibliography_source_count_is_capped() {
let doc = format!("<w:document {W_NS}><w:body><w:p><w:r><w:t>body</w:t></w:r></w:p></w:body></w:document>");
let mut sources = String::from(r#"<b:Sources xmlns:b="http://schemas.openxmlformats.org/officeDocument/2006/bibliography" StyleName="MLA">"#);
+1
View File
@@ -871,6 +871,7 @@ fn citation_source_lookups_keep_first_duplicates_case_and_style_order() {
#[cfg(not(target_arch = "wasm32"))]
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn citation_refresh_source_lookup_work_scales_with_document_size() {
fn doc(n: usize) -> Session {
let mut p = Paragraph::new();
+1
View File
@@ -332,6 +332,7 @@ mod tests {
/// Drawing commands must not search every page again for each glued connector.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn rerouting_many_pages_does_not_rescan_all_objects_per_connector() {
use wordcraft_doc::{Document, Paragraph, para_block};
let mut doc = Document::new();
+15 -2
View File
@@ -24,7 +24,7 @@ pub fn specs() -> Vec<CommandSpec> {
CommandSpec::new("font.remove", "Remove Added Font", "Home › Font", remove).params(r#"{"family": string}"#).pure(),
CommandSpec::new("font.loadSystem", "Load System Installed Fonts…", "Home › Font", load_system).pure(),
CommandSpec::new("font.missing", "Missing Fonts", "Status Bar", missing)
.params(r#"{} → {"missing": [{"font": string, "shownAs": [string] (the families drawn instead), "metricCompatible": bool (its substitute measures like the font)}]} — the text the current view shows"#)
.params(r#"{} → {"missing": [{"font": string, "shownAs": [string] (the families drawn instead), "metricCompatible": bool (its substitute measures like the font), "download"?: {"name": string, "publisher": string, "url": string, "restricted": bool (the publisher limits who may use it: check its licence terms)} (its publisher's download page, when it offers the font at no charge)}]} — the text the current view shows"#)
.pure(),
]
}
@@ -136,6 +136,9 @@ pub struct MissingFont {
/// Fallback faces for characters the substitute lacks don't count: the font itself might
/// lack them too (an emoji in Calibri text).
pub metric_compatible: bool,
/// Its publisher's download page, when the publisher offers the font at no charge
/// ([`wordcraft_fonts::download`]): installing it shows the document as it was written.
pub download: Option<&'static wordcraft_fonts::download::FontDownload>,
}
/// The family text set in `font` is drawn in when `font` isn't installed (None when it is), as
@@ -174,6 +177,7 @@ pub fn missing_fonts(doc: &Document, layout: &DocLayout) -> Vec<MissingFont> {
// Judged by the name layout looked up, spaces and all: "Arial " isn't Arial to it, so
// its text is in a substitute even where Arial is installed.
metric_compatible: shown_as(doc, &recorded).is_some_and(|s| !warns(font, &s)),
download: wordcraft_fonts::download::download_for(font),
});
if !m.shown_as.contains(&family) {
m.shown_as.push(family);
@@ -216,7 +220,16 @@ fn missing(s: &mut Session, _: &Value) -> CmdResult {
(_, n) if n > NAMED => format!("Not installed: {named} and {} more", n - NAMED),
_ => format!("Not installed: {named}"),
};
let fonts: Vec<Value> = found.iter().map(|f| json!({"font": f.font, "shownAs": f.shown_as, "metricCompatible": f.metric_compatible})).collect();
let fonts: Vec<Value> = found
.iter()
.map(|f| {
let mut font = json!({"font": f.font, "shownAs": f.shown_as, "metricCompatible": f.metric_compatible});
if let Some(d) = f.download {
font["download"] = json!({"name": d.name, "publisher": d.publisher, "url": d.url, "restricted": d.restricted});
}
font
})
.collect();
Ok(json!({"missing": fonts}))
}
+1
View File
@@ -799,6 +799,7 @@ fn deleting_a_comment_in_a_parent_cycle_ends() {
/// A very long reply chain (as a hostile file could hold) is deleted in one pass over the
/// comments, not one pass per reply: 50,000 replies would otherwise freeze the app.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn deleting_a_long_reply_chain_stays_fast() {
let mut s = Session::new(Document::new());
s.view.proofing = false;
+1
View File
@@ -102,6 +102,7 @@ fn tidy_matches_replacing_until_nothing_changes() {
/// Nested empty brackets are removed in one pass: 40,000 pairs took seconds when each pass
/// removed one level (#3680).
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn tidy_is_linear_on_nested_brackets() {
let n = 40_000;
let started = std::time::Instant::now();
+2
View File
@@ -1052,6 +1052,7 @@ fn rep_mode(text: &str, q: &str, with: &str, extra: serde_json::Value) -> String
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn hostile_find_patterns_are_bounded() {
let mut s = s();
run(&mut s, "document.setText", json!({"text": &"a".repeat(5000)}));
@@ -12161,6 +12162,7 @@ fn inspect_document_removes_the_note_of_a_hidden_reference() {
/// #1344: many hidden pieces (index entries are hidden text) are removed quickly, and the
/// found list is capped.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn inspect_document_removes_thousands_of_hidden_pieces() {
use wordcraft_doc::props::CharProps;
use wordcraft_doc::{Paragraph, para_block};
+21
View File
@@ -371,6 +371,27 @@ fn missing_fonts_from_command_search_say_so_in_the_status_line() {
assert!(s.status.ends_with(" and 35 more") && s.status.len() < 300, "{}", s.status);
}
/// A missing font its publisher gives away says where to get it; others don't.
#[test]
fn a_missing_font_its_publisher_gives_away_names_the_download_page() {
if wordcraft_fonts::FontDb::global().families().iter().any(|f| f.eq_ignore_ascii_case("Aptos")) {
return; // Installed here: not missing.
}
let mut s = Session::new(doc_of(vec![para_in("Text", "Aptos"), para_in("Text", "Wordcraft Nowhere Face")]));
let layout = s.layout();
let m = crate::cmd::font::missing_fonts(&s.doc, &layout);
let aptos = m.iter().find(|f| f.font == "Aptos").unwrap();
assert_eq!(aptos.download.map(|d| d.publisher.as_str()), Some("Microsoft"));
assert!(m.iter().find(|f| f.font == "Wordcraft Nowhere Face").unwrap().download.is_none());
let out = s.run("font.missing", &json!({})).unwrap();
let listed = out["missing"].as_array().unwrap();
let aptos = listed.iter().find(|f| f["font"] == "Aptos").unwrap();
assert!(aptos["download"]["url"].as_str().unwrap().starts_with("https://www.microsoft.com/"), "{aptos}");
assert_eq!(aptos["download"]["publisher"], "Microsoft");
assert_eq!(aptos["download"]["restricted"], false);
assert!(listed.iter().find(|f| f["font"] == "Wordcraft Nowhere Face").unwrap().get("download").is_none());
}
#[test]
fn a_huge_font_name_is_kept_short() {
let name = format!("Wordcraft {}", "é".repeat(100_000));
+1
View File
@@ -21,3 +21,4 @@ brotli-decompressor = { workspace = true }
unicode-bidi = { workspace = true }
unicode-script = { workspace = true }
log = { workspace = true }
serde_json = { workspace = true }
+149
View File
@@ -0,0 +1,149 @@
//! Fonts a document may ask for that their publisher gives away, and where: when one isn't
//! installed, the app can say where to get it instead of only naming the substitute.
//!
//! Only a publisher's own page is listed, for fonts it offers for download at no charge. The app
//! links to the page; it never downloads, bundles or redistributes the fonts, and following the
//! publisher's licence terms is the user's part. A font whose publisher limits who may use it
//! is marked [`FontDownload::restricted`], so the app can say so beside the link. Most Office
//! fonts (Calibri, Cambria, Consolas, Segoe UI…) come only with Windows or Office, so they are
//! not here.
//!
//! The list is `font_downloads.json` beside this file, built into the app: adding a font is a
//! new entry there.
/// The list: `{"downloads": [{"family", "publisher", "url", "restricted"?, "note"?}]}`.
const LIST: &str = include_str!("font_downloads.json");
/// Most entries read from the list.
const MAX_DOWNLOADS: usize = 1024;
/// Where a font family can be downloaded at no charge.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct FontDownload {
/// The name of the download: the family or collection the page offers. It covers the
/// family itself and every name that continues it after a space ("Aptos" covers "Aptos
/// Display", "Aptos Narrow", "Aptos Mono"…).
pub name: String,
/// Who publishes the font and hosts the page.
pub publisher: String,
/// The publisher's download page.
pub url: String,
/// The publisher limits who may use the font or what for (its design resources for one
/// platform, say): worth saying beside the link.
pub restricted: bool,
}
/// The entries of `list` that name a family, a publisher and an `https` page; others are left
/// out, and a list that isn't JSON has none.
fn parse(list: &str) -> Vec<FontDownload> {
let Ok(json) = serde_json::from_str::<serde_json::Value>(list) else {
log::warn!("font downloads: the list isn't JSON");
return Vec::new();
};
let entries = json.get("downloads").and_then(|d| d.as_array()).map(Vec::as_slice).unwrap_or_default();
entries
.iter()
.take(MAX_DOWNLOADS)
.filter_map(|e| {
let field = |k: &str| e.get(k).and_then(|v| v.as_str()).map(str::trim).filter(|v| !v.is_empty());
let (name, publisher, url) = (field("family")?, field("publisher")?, field("url")?);
let restricted = e.get("restricted").and_then(|v| v.as_bool()).unwrap_or(false);
url.starts_with("https://").then(|| FontDownload { name: name.into(), publisher: publisher.into(), url: url.into(), restricted })
})
.collect()
}
/// The built-in list, read once.
fn downloads() -> &'static [FontDownload] {
static DOWNLOADS: std::sync::OnceLock<Vec<FontDownload>> = std::sync::OnceLock::new();
DOWNLOADS.get_or_init(|| parse(LIST))
}
/// Where `family` can be downloaded free of charge from its publisher, if it can. The name is
/// matched whatever its case, with a charset suffix ("Aptos CE") or alternates ("Aptos;Arial")
/// as a document may write it.
pub fn download_for(family: &str) -> Option<&'static FontDownload> {
family.split(';').map(str::trim).filter(|f| !f.is_empty()).find_map(|f| {
let f = crate::word::charset_base_family(f).trim();
downloads().iter().find(|d| {
// Compared as bytes: `get` on a `str` would refuse a split inside a character.
let (name, f) = (d.name.as_bytes(), f.as_bytes());
f.get(..name.len()).is_some_and(|head| head.eq_ignore_ascii_case(name)) && matches!(f.get(name.len()), None | Some(b' '))
})
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn microsoft_fonts_offered_free_of_charge_have_a_download_page() {
let aptos = download_for("Aptos").unwrap();
assert_eq!(aptos.publisher, "Microsoft");
assert!(aptos.url.starts_with("https://www.microsoft.com/"));
for family in ["aptos", "Aptos Display", "APTOS NARROW", "Aptos Mono", "Aptos Serif", " Aptos ", "Aptos;Arial", "Aptos CE"] {
assert_eq!(download_for(family), Some(aptos), "{family}");
}
assert_eq!(download_for("Cascadia Code").map(|d| d.name.as_str()), Some("Cascadia Code"));
assert_eq!(download_for("Cascadia Mono PL").map(|d| d.name.as_str()), Some("Cascadia Mono"));
assert_eq!(download_for("Selawik Semibold").map(|d| (d.name.as_str(), d.restricted)), Some(("Selawik", false)));
assert!(!aptos.restricted);
}
/// Fonts offered with limits on their use are listed, and say so. Plain Segoe UI isn't
/// offered for download, and the Segoe UI Variable entry doesn't stand in for it.
#[test]
fn fonts_with_limited_licences_are_marked() {
for family in ["Segoe UI Variable", "Segoe UI Variable Text", "segoe fluent icons", "Segoe MDL2 Assets"] {
let d = download_for(family).unwrap_or_else(|| panic!("{family}"));
assert!(d.restricted, "{family}");
assert!(d.url.starts_with("https://learn.microsoft.com/"), "{}", d.url);
}
for family in ["Segoe UI", "Segoe UI Semibold", "Segoe UI Symbol", "Segoe Print"] {
assert_eq!(download_for(family), None, "{family}");
}
}
#[test]
fn other_fonts_have_none() {
// Fonts that come only with Windows or Office, look-alike names, and junk.
for family in ["Calibri", "Cambria", "Consolas", "Arial", "Aptosa", "Apt", "", ";", "Aptosé", "é", "Cascadia"] {
assert_eq!(download_for(family), None, "{family}");
}
}
#[test]
fn every_page_is_https_on_the_publishers_site() {
// Every entry of the file is read: none is left out for a missing field.
let entries = serde_json::from_str::<serde_json::Value>(LIST).unwrap()["downloads"].as_array().unwrap().len();
assert_eq!(downloads().len(), entries);
assert!(entries >= 3);
for d in downloads() {
let hosts = ["https://www.microsoft.com/", "https://learn.microsoft.com/", "https://github.com/microsoft/"];
assert!(hosts.iter().any(|h| d.url.starts_with(h)), "{}", d.url);
}
}
#[test]
fn a_broken_list_has_no_downloads() {
assert!(parse("not json").is_empty());
assert!(parse("{}").is_empty());
assert!(parse(r#"{"downloads": 7}"#).is_empty());
// Entries without a family, a publisher or an https page are left out.
let list = r#"{"downloads": [
{"family": "A", "publisher": "P", "url": "http://plain.example"},
{"family": "", "publisher": "P", "url": "https://x.example"},
{"publisher": "P", "url": "https://x.example"},
{"family": 3, "publisher": "P", "url": "https://x.example"},
"junk",
{"family": " Kept ", "publisher": "P", "url": "https://x.example"}
]}"#;
let read = parse(list);
assert_eq!(read.len(), 1);
assert_eq!(read[0].name, "Kept");
assert!(!read[0].restricted, "not restricted unless it says so");
let marked = parse(r#"{"downloads": [{"family": "A", "publisher": "P", "url": "https://x.example", "restricted": true}]}"#);
assert!(marked[0].restricted);
}
}
+50
View File
@@ -0,0 +1,50 @@
{
"about": "Fonts a document may ask for that their publisher offers for download at no charge, and the publisher's own download page. WordCraft links to the page; it never downloads, bundles or redistributes the fonts, and the user is responsible for the publisher's licence terms. 'family' also covers every name that continues it after a space: 'Aptos' covers 'Aptos Display'. 'restricted': true marks a font whose publisher limits who may use it or what for; the app then says so beside the download.",
"downloads": [
{
"family": "Aptos",
"publisher": "Microsoft",
"url": "https://www.microsoft.com/download/details.aspx?id=106087",
"note": "Microsoft Download Center, 'Microsoft Aptos Fonts': the whole Aptos family."
},
{
"family": "Cascadia Code",
"publisher": "Microsoft",
"url": "https://github.com/microsoft/cascadia-code/releases",
"note": "Microsoft's own repository; SIL Open Font License."
},
{
"family": "Cascadia Mono",
"publisher": "Microsoft",
"url": "https://github.com/microsoft/cascadia-code/releases",
"note": "Comes in the Cascadia Code release."
},
{
"family": "Selawik",
"publisher": "Microsoft",
"url": "https://github.com/microsoft/fonts",
"note": "Microsoft's open-source stand-in for Segoe UI; SIL Open Font License."
},
{
"family": "Segoe UI Variable",
"publisher": "Microsoft",
"url": "https://learn.microsoft.com/windows/apps/design/downloads",
"restricted": true,
"note": "Microsoft's Windows design downloads page. Offered with the design resources for Windows apps; plain 'Segoe UI' is not offered for download and is not matched by this entry."
},
{
"family": "Segoe Fluent Icons",
"publisher": "Microsoft",
"url": "https://learn.microsoft.com/windows/apps/design/downloads",
"restricted": true,
"note": "Microsoft's Windows design downloads page."
},
{
"family": "Segoe MDL2 Assets",
"publisher": "Microsoft",
"url": "https://learn.microsoft.com/windows/apps/design/downloads",
"restricted": true,
"note": "Microsoft's Windows design downloads page."
}
]
}
+1
View File
@@ -450,6 +450,7 @@ fn long_names_are_cut_before_faces_copy_them() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn named_instances_and_collection_members_are_capped() {
// #3716: a file's faces are capped before they are built, however the file multiplies them.
let font = variable_font(300);
+1
View File
@@ -8,6 +8,7 @@
#![forbid(unsafe_code)]
pub mod color;
pub mod download;
mod fontdb;
pub mod math;
mod script;
+1 -1
View File
@@ -33,7 +33,6 @@ craft-zip = { workspace = true }
wordcraft-geom = { workspace = true, features = ["image"] }
craft-metafile = { workspace = true }
encoding_rs = { workspace = true }
chardetng = { workspace = true }
log = { workspace = true }
chrono = { workspace = true }
# Password-protected ODT (ODF 1.3 Part 2 §3.4): AES-256-CBC, SHA-1/SHA-256, PBKDF2 (HMAC-SHA-1).
@@ -41,3 +40,4 @@ chrono = { workspace = true }
wordcraft-doc = { workspace = true, features = ["test-support"] }
craft-image-codecs = { workspace = true }
proptest = { workspace = true }
chardetng = { workspace = true }
+4
View File
@@ -488,6 +488,7 @@ fn code_blocks_and_rules() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn deep_nesting_is_bounded() {
let html = "<div><table><tr><td>".repeat(5000) + "x";
let d = import("html", html.as_bytes()).unwrap().unwrap();
@@ -566,6 +567,7 @@ fn latex_export_carries_no_commands_from_the_document() {
/// past them for the last text made a megabyte of labels take tens of seconds), and labels
/// separated by text are all kept.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn latex_labels_in_a_row_are_read_in_linear_time() {
let started = std::time::Instant::now();
for src in ["\\label{a} ".repeat(100_000) + "x", "\\label{a}".repeat(100_000) + "x", "\\hypertarget{a}{} ".repeat(100_000)] {
@@ -743,6 +745,7 @@ fn decorative_picture_exports_with_empty_alt_text() {
/// Hostile Markdown and HTML that rescanned the rest of the text from every `[`, `![`, `<` or `&`
/// took quadratic time: a few hundred KB froze the importer for tens of seconds.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn hostile_importer_input_takes_linear_time() {
use std::time::{Duration, Instant};
let cases: [(&str, String); 5] = [
@@ -764,6 +767,7 @@ fn hostile_importer_input_takes_linear_time() {
/// bracket scan over them looking for a closing run through the rest of the text: superlinear
/// time on hostile Markdown.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn unmatched_backtick_runs_take_linear_time() {
use std::time::{Duration, Instant};
// Runs of 1, 2, … 2000 backticks (2 MB), none of them closed.
+5 -88
View File
@@ -10,95 +10,12 @@ pub use wordcraft_doc::encoding::cp1252;
use crate::model::{FBlock, ListCounter, from_doc};
fn utf16(b: &[u8], le: bool) -> String {
let units: Vec<u16> = b.as_chunks::<2>().0.iter().map(|c| if le { u16::from_le_bytes(*c) } else { u16::from_be_bytes(*c) }).collect();
String::from_utf16_lossy(&units)
}
#[cfg(test)]
mod legacy_oracle_tests;
/// UTF-16 without a BOM, told by its zero bytes: `Some(true)` little-endian, `Some(false)`
/// big-endian.
fn unmarked_utf16(b: &[u8]) -> Option<bool> {
if b.len() < 4 || !b.len().is_multiple_of(2) {
return None;
}
let sample = b.get(..b.len().min(512))?;
let even_zero = sample.iter().step_by(2).filter(|x| **x == 0).count();
let odd_zero = sample.iter().skip(1).step_by(2).filter(|x| **x == 0).count();
let half = sample.len() / 2;
if odd_zero * 10 >= half * 4 && even_zero * 10 < half {
return Some(true);
}
if even_zero * 10 >= half * 4 && odd_zero * 10 < half {
return Some(false);
}
None
}
/// Does `b` look like plain text, so a file with an extension WordCraft doesn't know (a
/// playlist, a log, a config file…) can open as text the way Word's text import opens it?
///
/// Text has a BOM, or reads as UTF-16 by its zero bytes, or has no NUL byte and next to no
/// other control characters in its first 64 KB. PDFs are refused by their signature (they
/// start as text). An empty file is text.
pub fn is_text(b: &[u8]) -> bool {
if b.starts_with(&[0xEF, 0xBB, 0xBF]) || b.starts_with(&[0xFF, 0xFE]) || b.starts_with(&[0xFE, 0xFF]) {
return true;
}
if b.starts_with(b"%PDF-") {
return false;
}
let sample = b.get(..b.len().min(64 << 10)).unwrap_or(b);
// Tab, line feed, form feed, carriage return, DOS end-of-file (^Z) and escape appear in text;
// other controls in more than 1 of 100 characters mean binary.
let odd = |c: u32| c == 0 || (c < 0x20 && !matches!(c, 0x09 | 0x0A | 0x0C | 0x0D | 0x1A | 0x1B)) || c == 0x7F;
if let Some(le) = unmarked_utf16(sample) {
let units = sample.as_chunks::<2>().0;
let n = units.iter().filter(|c| odd(u32::from(if le { u16::from_le_bytes(**c) } else { u16::from_be_bytes(**c) }))).count();
return n.saturating_mul(100) <= units.len();
}
if sample.contains(&0) {
return false;
}
sample.iter().filter(|c| odd(u32::from(**c))).count().saturating_mul(100) <= sample.len()
}
/// Decode text bytes: BOMs first, then UTF-16 without BOM (by its zero bytes), UTF-8, and the
/// legacy encoding the bytes look like ([`legacy`]) as the last resort.
pub fn decode(b: &[u8]) -> String {
if let Some(rest) = b.strip_prefix(&[0xEF, 0xBB, 0xBF]) {
return String::from_utf8_lossy(rest).into_owned();
}
if let Some(rest) = b.strip_prefix(&[0xFF, 0xFE]) {
return utf16(rest, true);
}
if let Some(rest) = b.strip_prefix(&[0xFE, 0xFF]) {
return utf16(rest, false);
}
if let Some(le) = unmarked_utf16(b) {
return utf16(b, le);
}
match std::str::from_utf8(b) {
Ok(s) => s.to_string(),
Err(_) => legacy(b),
}
}
/// Most bytes looked at to guess a legacy encoding.
const SNIFF: usize = 1 << 20;
/// Bytes that aren't UTF-8 in the legacy encoding they look like (chardetng, the detector
/// Firefox uses for unlabelled pages): GBK/GB18030 for Simplified Chinese, Big5, Shift_JIS or
/// EUC-JP, EUC-KR, or a single-byte Windows / ISO code page; Windows-1252 as before when the
/// guess is a single-byte Western one.
pub fn legacy(b: &[u8]) -> String {
let mut det = chardetng::EncodingDetector::new(chardetng::Iso2022JpDetection::Deny);
det.feed(b.get(..SNIFF).unwrap_or(b), b.len() <= SNIFF);
let enc = det.guess(None, chardetng::Utf8Detection::Deny);
if enc == encoding_rs::WINDOWS_1252 {
return b.iter().map(|c| cp1252(*c)).collect();
}
enc.decode_without_bom_handling(b).0.into_owned()
}
// Detection is shared with the other Craft apps. WordCraft's `cp1252` and encoding_rs agree on
// every byte (pinned by `legacy_oracle_tests`), so a Windows-1252 guess needs no local mapping.
pub use craft_mime::detect::{decode, is_text, legacy};
/// Decode bytes in a named encoding (a WHATWG label such as `gbk`, `gb18030`, `big5`,
/// `shift_jis`, `euc-kr`, `utf-16le`, `windows-1251`); `None` when the label isn't known.
@@ -0,0 +1,210 @@
//! The pre-cutover local implementations, kept as an oracle: the shared `craft_mime::detect`
//! functions plus the cp1252 wrapper must agree with them byte for byte.
use super::cp1252;
fn utf16(b: &[u8], le: bool) -> String {
let units: Vec<u16> = b.as_chunks::<2>().0.iter().map(|c| if le { u16::from_le_bytes(*c) } else { u16::from_be_bytes(*c) }).collect();
String::from_utf16_lossy(&units)
}
/// UTF-16 without a BOM, told by its zero bytes: `Some(true)` little-endian, `Some(false)`
/// big-endian.
fn unmarked_utf16(b: &[u8]) -> Option<bool> {
if b.len() < 4 || !b.len().is_multiple_of(2) {
return None;
}
let sample = b.get(..b.len().min(512))?;
let even_zero = sample.iter().step_by(2).filter(|x| **x == 0).count();
let odd_zero = sample.iter().skip(1).step_by(2).filter(|x| **x == 0).count();
let half = sample.len() / 2;
if odd_zero * 10 >= half * 4 && even_zero * 10 < half {
return Some(true);
}
if even_zero * 10 >= half * 4 && odd_zero * 10 < half {
return Some(false);
}
None
}
/// Does `b` look like plain text, so a file with an extension WordCraft doesn't know (a
/// playlist, a log, a config file…) can open as text the way Word's text import opens it?
///
/// Text has a BOM, or reads as UTF-16 by its zero bytes, or has no NUL byte and next to no
/// other control characters in its first 64 KB. PDFs are refused by their signature (they
/// start as text). An empty file is text.
fn is_text(b: &[u8]) -> bool {
if b.starts_with(&[0xEF, 0xBB, 0xBF]) || b.starts_with(&[0xFF, 0xFE]) || b.starts_with(&[0xFE, 0xFF]) {
return true;
}
if b.starts_with(b"%PDF-") {
return false;
}
let sample = b.get(..b.len().min(64 << 10)).unwrap_or(b);
// Tab, line feed, form feed, carriage return, DOS end-of-file (^Z) and escape appear in text;
// other controls in more than 1 of 100 characters mean binary.
let odd = |c: u32| c == 0 || (c < 0x20 && !matches!(c, 0x09 | 0x0A | 0x0C | 0x0D | 0x1A | 0x1B)) || c == 0x7F;
if let Some(le) = unmarked_utf16(sample) {
let units = sample.as_chunks::<2>().0;
let n = units.iter().filter(|c| odd(u32::from(if le { u16::from_le_bytes(**c) } else { u16::from_be_bytes(**c) }))).count();
return n.saturating_mul(100) <= units.len();
}
if sample.contains(&0) {
return false;
}
sample.iter().filter(|c| odd(u32::from(**c))).count().saturating_mul(100) <= sample.len()
}
/// Decode text bytes: BOMs first, then UTF-16 without BOM (by its zero bytes), UTF-8, and the
/// legacy encoding the bytes look like ([`legacy`]) as the last resort.
fn decode(b: &[u8]) -> String {
if let Some(rest) = b.strip_prefix(&[0xEF, 0xBB, 0xBF]) {
return String::from_utf8_lossy(rest).into_owned();
}
if let Some(rest) = b.strip_prefix(&[0xFF, 0xFE]) {
return utf16(rest, true);
}
if let Some(rest) = b.strip_prefix(&[0xFE, 0xFF]) {
return utf16(rest, false);
}
if let Some(le) = unmarked_utf16(b) {
return utf16(b, le);
}
match std::str::from_utf8(b) {
Ok(s) => s.to_string(),
Err(_) => legacy(b),
}
}
/// Most bytes looked at to guess a legacy encoding.
const SNIFF: usize = 1 << 20;
/// Bytes that aren't UTF-8 in the legacy encoding they look like (chardetng, the detector
/// Firefox uses for unlabelled pages): GBK/GB18030 for Simplified Chinese, Big5, Shift_JIS or
/// EUC-JP, EUC-KR, or a single-byte Windows / ISO code page; Windows-1252 as before when the
/// guess is a single-byte Western one.
fn legacy(b: &[u8]) -> String {
let mut det = chardetng::EncodingDetector::new(chardetng::Iso2022JpDetection::Deny);
det.feed(b.get(..SNIFF).unwrap_or(b), b.len() <= SNIFF);
let enc = det.guess(None, chardetng::Utf8Detection::Deny);
if enc == encoding_rs::WINDOWS_1252 {
return b.iter().map(|c| cp1252(*c)).collect();
}
enc.decode_without_bom_handling(b).0.into_owned()
}
fn samples() -> Vec<Vec<u8>> {
let mut v: Vec<Vec<u8>> = vec![vec![]];
for b in 0..=255u8 {
v.push(vec![b]);
}
for b in 0x80..=0x9Fu8 {
v.push([b"caf".as_slice(), &[b], b" ok text here"].concat());
v.push(vec![b, b'a', b'b']);
}
// Western text with each byte Windows-1252 leaves undefined: the case the cp1252 wrapper is for.
for b in [0x81u8, 0x8D, 0x8F, 0x90, 0x9D] {
v.push([b"caf\xE9 au lait ".as_slice(), &[b], b" ok"].concat());
v.push([b"Le caf\xE9 de la gare ".as_slice(), &[b]].concat());
v.push([b"r\xE9sum\xE9 ".as_slice(), &[b]].concat());
}
v.push("héllo wörld € naïve".as_bytes().to_vec());
v.push([&[0xEF, 0xBB, 0xBF][..], "héllo".as_bytes()].concat());
v.push([&[0xEF, 0xBB, 0xBF][..], &[0xE9, 0x80]].concat());
for (bom, le) in [(&[0xFF, 0xFE][..], true), (&[0xFE, 0xFF][..], false)] {
let t = "héllo wörld 日本語 😀";
v.push([bom, &t.encode_utf16().flat_map(|u| if le { u.to_le_bytes() } else { u.to_be_bytes() }).collect::<Vec<u8>>()[..]].concat());
v.push([bom, b"h\0i\0!"].concat());
let raw: Vec<u8> = "hello world, plain ascii".encode_utf16().flat_map(|u| if le { u.to_le_bytes() } else { u.to_be_bytes() }).collect();
v.push(raw);
let raw: Vec<u8> =
"héllo wörld, mostly ascii \u{20AC}".encode_utf16().flat_map(|u| if le { u.to_le_bytes() } else { u.to_be_bytes() }).collect();
v.push(raw);
}
v.push(vec![0xC3, 0x28, 0xA0, 0xA1, 0xE2, 0x82]);
v.push(b"#EXTM3U\r\n#EXTINF:1,x\r\n".to_vec());
v.push(b"%PDF-1.7\n%\xE2\xE3".to_vec());
v.push(b"\x89PNG\r\n\x1a\n\0\0\0\rIHDR".to_vec());
v.push(vec![1, 0, 2, 0, 3]);
for (enc, text) in [
(encoding_rs::GBK, "这是一个用简体中文写的文本文件,用来测试编码检测。"),
(encoding_rs::BIG5, "這是一個用繁體中文寫的文字檔案,用來測試編碼偵測。"),
(encoding_rs::SHIFT_JIS, "これは日本語で書かれたテキストファイルです。エンコーディングの検出をテストします。"),
(encoding_rs::EUC_KR, "이것은 한국어로 작성된 텍스트 파일입니다. 인코딩 감지를 테스트합니다."),
(encoding_rs::WINDOWS_1251, "Это текстовый файл на русском языке для проверки определения кодировки."),
] {
let (b, _, _) = enc.encode(text);
v.push(b.into_owned());
}
// Past the 1 MiB sniff limit, and a 2 MiB input.
let mut big = vec![b'a'; 1 << 20];
big.extend_from_slice(&[0xE9; 3]);
v.push(big);
let mut big = "héllo wörld ".repeat((2 << 20) / 13).into_bytes();
big.push(0xE9);
v.push(big);
v.push(vec![0x80; 2 << 20]);
v
}
fn assert_same(b: &[u8]) {
let head: Vec<u8> = b.iter().take(24).copied().collect();
assert_eq!(is_text(b), super::is_text(b), "is_text {head:02x?}");
assert_eq!(legacy(b), super::legacy(b), "legacy {head:02x?}");
assert_eq!(decode(b), super::decode(b), "decode {head:02x?}");
}
#[test]
fn shared_detect_matches_the_old_local_copies() {
for b in samples() {
assert_same(&b);
}
}
#[test]
fn cp1252_agrees_with_encoding_rs_on_every_byte() {
// Why a Windows-1252 guess needs no local mapping: both give U+0081, U+008D, U+008F, U+0090
// and U+009D for the bytes the code page leaves undefined.
for b in 0..=255u8 {
let one = [b];
let (s, _) = encoding_rs::WINDOWS_1252.decode_without_bom_handling(&one);
assert_eq!(s.chars().collect::<Vec<_>>(), vec![cp1252(b)], "byte {b:#x}");
}
}
#[test]
fn random_buffers_never_panic_and_match() {
let mut s: u64 = 0x2545_F491_4F6C_DD1D;
let mut next = || {
s = s.wrapping_mul(6_364_136_223_846_793_005).wrapping_add(1_442_695_040_888_963_407);
(s >> 33) as u32
};
for i in 0..3000 {
let len = (next() % 700) as usize;
let mode = i % 4;
let b: Vec<u8> = (0..len)
.map(|_| {
let r = next();
match mode {
0 => r as u8,
1 => 0x80 | (r as u8 & 0x1F), // 0x80..=0x9F
2 => {
if r % 3 == 0 {
0
} else {
(r >> 8) as u8 % 0x80
}
}
_ => {
if r % 5 == 0 {
0x80 + (r >> 8) as u8 % 0x80
} else {
b'a' + (r >> 8) as u8 % 26
}
}
}
})
.collect();
assert_same(&b);
}
}
+69 -10
View File
@@ -320,19 +320,16 @@ pub fn probe_dib(dib: Vec<u8>) -> Result<Probe<'static>, Refusal> {
Ok(Probe { bytes: Cow::Owned(bmp), format: ImageFormat::Bmp, width, height, peak, alpha })
}
/// A DIB's width and height, read by the shared BMP probe. Header sizes other than 12 (core) and
/// 40 or more (info) are refused here: the probe would read them as info headers.
fn dib_header_dims(dib: &[u8]) -> Option<(u32, u32)> {
let bi_size = u32::from_le_bytes(dib.get(0..4)?.try_into().ok()?);
if bi_size == 12 {
let w = u16::from_le_bytes(dib.get(4..6)?.try_into().ok()?);
let h = u16::from_le_bytes(dib.get(6..8)?.try_into().ok()?);
Some((u32::from(w), u32::from(h)))
} else if bi_size >= 40 {
let w = i32::from_le_bytes(dib.get(4..8)?.try_into().ok()?);
let h = i32::from_le_bytes(dib.get(8..12)?.try_into().ok()?);
Some((u32::try_from(w).ok()?, h.unsigned_abs()))
} else {
None
if bi_size != 12 && bi_size < 40 {
return None;
}
// Only the first 12 header bytes are read, so only they are wrapped: this runs for DIBs too
// big to decode, which must not be copied whole.
craft_image_codecs::bmp_probe::dimensions(&dib_to_bmp(dib.get(..12).unwrap_or(dib)))
}
/// A `width` × `height` picture of `native` bytes a pixel, with its peak (see [`peak_bytes`]).
@@ -577,6 +574,68 @@ fn dib_layout(mut dib: Vec<u8>) -> Vec<u8> {
mod tests {
use super::*;
// Hand-written reader the shared probe replaced; kept as the oracle.
fn dib_header_dims_old(dib: &[u8]) -> Option<(u32, u32)> {
let bi_size = u32::from_le_bytes(dib.get(0..4)?.try_into().ok()?);
if bi_size == 12 {
let w = u16::from_le_bytes(dib.get(4..6)?.try_into().ok()?);
let h = u16::from_le_bytes(dib.get(6..8)?.try_into().ok()?);
Some((u32::from(w), u32::from(h)))
} else if bi_size >= 40 {
let w = i32::from_le_bytes(dib.get(4..8)?.try_into().ok()?);
let h = i32::from_le_bytes(dib.get(8..12)?.try_into().ok()?);
Some((u32::try_from(w).ok()?, h.unsigned_abs()))
} else {
None
}
}
fn hdr(size: u32, w: i32, h: i32, len: usize) -> Vec<u8> {
let mut v = [0u8; 124].to_vec();
v.resize(124.max(len), 0);
v[0..4].copy_from_slice(&size.to_le_bytes());
v[4..8].copy_from_slice(&w.to_le_bytes());
v[8..12].copy_from_slice(&h.to_le_bytes());
v.truncate(len);
v
}
#[test]
fn dib_dims_match_the_hand_written_reader() {
let sizes = [0u32, 11, 12, 13, 16, 39, 40, 41, 52, 108, 124, u32::MAX];
let dims = [(0, 0), (1, 1), (3, -2), (0, 5), (7, i32::MIN), (i32::MIN, 1), (-1, 1), (65536, 70000), (i32::MAX, i32::MAX)];
for &s in &sizes {
for &(w, h) in &dims {
for len in 0..=130 {
let d = hdr(s, w, h, len);
assert_eq!(dib_header_dims(&d), dib_header_dims_old(&d), "size {s} w {w} h {h} len {len}");
}
}
}
assert_eq!(dib_header_dims(&hdr(40, 3, -2, 40)), Some((3, 2)));
let mut core = hdr(12, 3, 0, 12);
core[6..8].copy_from_slice(&2u16.to_le_bytes());
assert_eq!(dib_header_dims(&core), Some((3, 2)));
}
#[test]
fn dib_dims_never_panic_on_junk() {
let mut x = 0x2545_f491_4f6c_dd1du64;
let mut buf = [0u8; 64];
for _ in 0..2000 {
for b in buf.iter_mut() {
x ^= x << 13;
x ^= x >> 7;
x ^= x << 17;
*b = x as u8;
}
for n in 0..=buf.len() {
assert_eq!(dib_header_dims(&buf[..n]), dib_header_dims_old(&buf[..n]));
let _ = probe_dib(buf[..n].to_vec());
}
}
}
fn encode(img: impl Into<DynamicImage>, format: ImageFormat) -> Vec<u8> {
let dyn_img = img.into();
let codec_img = dynamic_to_codec(&dyn_img).unwrap();
+4
View File
@@ -923,6 +923,7 @@ fn a_long_unbreakable_word_lays_out_in_linear_time() {
/// A word a megabyte long lays out in a fraction of a second: it used to take half a minute.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn a_megabyte_long_word_lays_out_quickly() {
let d = long_word_beside_a_float(1_000_000);
let t = std::time::Instant::now();
@@ -1143,6 +1144,7 @@ fn a_word_with_alternating_advances_lays_out_in_linear_time() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn hidden_separators_with_automatic_hyphenation_lay_out_in_linear_time() {
crate::test_fonts();
// "hyphenation " n times with every space hidden: one joined word of 11n letters reaches the
@@ -1175,6 +1177,7 @@ fn page_break_paragraphs(n: usize) -> Document {
/// the section list by walking the whole body, so a document of n paragraphs paid n body-length
/// scans: a two-megabyte file took half a minute before the first page was drawn.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn a_document_of_many_paragraphs_lays_out_in_linear_time() {
let (t1, tk, ratio) = layout_time_ratio(page_break_paragraphs, 2_500, &LayoutOptions::default());
eprintln!("t(n) {t1:.4} s, t(8n) {tk:.4} s, ratio {ratio:.1}");
@@ -1182,6 +1185,7 @@ fn a_document_of_many_paragraphs_lays_out_in_linear_time() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn many_hidden_float_anchors_lay_out_in_linear_time() {
crate::test_fonts();
// n square-wrapped shapes, each anchored in its own hidden run (bold alternating, so runs do not
+1
View File
@@ -117,6 +117,7 @@ fn resolve_uses_the_given_system_locale() {
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn context_mode_on_a_very_long_digit_run_stays_linear() {
crate::test_fonts();
// Every digit used to scan back over all the digits and spaces before it (O(n²)).
+3
View File
@@ -390,6 +390,7 @@ fn cost_ratio(scale: usize, mut small: impl FnMut(), mut big: impl FnMut()) -> (
}
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn long_fragmented_hidden_headings_export_in_linear_time() {
// Headings of n chars in one-char runs: all hidden with bold alternating, and hidden and visible
// alternating. Layout and the bookmark/tag titles once scanned every left-out range for every
@@ -1917,6 +1918,7 @@ fn comments_print_in_balloons_beside_scaled_pages() {
/// references) exports in time proportional to its length: each link looked its bookmark up by
/// walking the whole document, so NIH's 67-page protocol template took 10.6 s (0.9 s now).
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn internal_links_export_in_linear_time() {
let doc = |n: usize| {
let mut d = Document::new();
@@ -3121,6 +3123,7 @@ fn only_safe_link_targets_become_annotations() {
/// without one), found in one pass: a long run of glyphs sharing a cluster used to rescan the rest
/// of the run for every glyph.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn cluster_ends_match_the_next_larger_cluster_in_linear_time() {
let brute = |cl: &[usize], len: usize| -> Vec<usize> {
(0..cl.len()).map(|i| cl.iter().skip(i + 1).copied().find(|n| *n > cl[i]).unwrap_or(len)).collect::<Vec<_>>()
+3 -3
View File
@@ -9,14 +9,14 @@ description = "WordCraft egui front end (Word-style ribbon UI); swappable: nothi
workspace = true
[dependencies]
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc" }
craft-ui = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a" }
# Interface translations: lookup, fallback, plurals and system-locale matching over the craft-i18n
# catalogs that `build.rs` embeds from `CRAFT_I18N_DIR` (storytold/craft-i18n, pinned in
# `craft-i18n.rev`). Same craft-libs rev as craft-ui; it brings lazy per-locale parsing
# (storytold/craft-libs#247), ICU4X plural rules and bidi isolates around inserted values.
craft-i18n = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc" }
craft-i18n = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a" }
# One native window per open document: the window list, the Window menu and the egui viewports.
craft-window = { git = "https://github.com/storytold/craft-libs", rev = "453b0da782ccef39be9a6dac8d27c80b553ef1cc", features = ["egui"] }
craft-window = { git = "https://github.com/storytold/craft-libs", rev = "93dac78c93a7f1fd66e2022edeb089233b1dc45a", features = ["egui"] }
wordcraft-geom = { workspace = true, features = ["image"] }
wordcraft-doc = { workspace = true }
wordcraft-fonts = { workspace = true }
+2
View File
@@ -110,6 +110,8 @@ impl WordApp {
integrated_titlebar: false,
mac_titlebar: chrome::MacTitleBar::default(),
updates_rt: updates_ui::Runtime::default(),
font_notice_seen: Vec::new(),
font_notices: false,
packs_rt: Default::default(),
control_rx: None,
shots: crate::control::Shots::default(),
+1
View File
@@ -129,6 +129,7 @@ impl WordApp {
let window = self.windows.shown();
let moved = |app: &WordApp| app.windows.shown() != window;
content_controls::popup(self, ctx);
crate::font_download::tick(self);
dialogs::show(self, ctx);
if moved(self) {
return;
+4
View File
@@ -853,6 +853,10 @@ pub fn status_bar(app: &mut WordApp, ui: &mut Ui) {
ui.label(crate::i18n::tn((count - SHOWN) as u64, "…and {n} more", "…and {n} more"));
}
ui.separator();
if missing.iter().any(|f| f.download.is_some()) && ui.button(tl!("Get the Missing Fonts…")).clicked() {
let _ = app.run("ui.dialog", json!({"name": "fontDownload"}));
ui.close();
}
if ui.button(tl!("Add Font…")).clicked() {
let _ = app.run("font.add", json!({}));
ui.close();
+11
View File
@@ -226,6 +226,17 @@ fn handle_in(app: &mut WordApp, doc: u64, method: &str, p: &Value) -> Outcome {
/// drag, key, wheel and text input, screenshots and renders, resize, focus and quit, and unknown
/// ids as commands. It refuses closing unsaved work and document operations without a document.
pub fn handle(app: &mut WordApp, ctx: &egui::Context, req: &ControlRequest) -> Outcome {
let before = app.session.document_id();
let outcome = handle_request(app, ctx, req);
// A document a script opened doesn't bring up the missing-fonts box.
let after = app.session.document_id();
if after != before {
crate::font_download::skip(app, after);
}
outcome
}
fn handle_request(app: &mut WordApp, ctx: &egui::Context, req: &ControlRequest) -> Outcome {
let p = &req.params;
if DOC_METHODS.contains(&req.method.as_str()) {
match addressed(app, p) {
+8
View File
@@ -321,6 +321,12 @@ pub enum Dialog {
#[serde(skip)]
note: String,
},
/// The document's missing fonts that can be downloaded, and how ([`crate::font_download`]).
FontDownload {
fonts: Vec<crate::font_download::Row>,
/// "Don't show this again".
dont_show: bool,
},
/// Tabs: 0 About, 1 Contributors, 2 Models.
About {
tab: u8,
@@ -550,6 +556,7 @@ impl Dialog {
Dialog::Icons { .. } => "icons",
Dialog::PasteSpecial { .. } => "pasteSpecial",
Dialog::About { .. } => "about",
Dialog::FontDownload { .. } => "fontDownload",
Dialog::SaveChanges { .. } => "saveChanges",
Dialog::DiscardRecovery { .. } => "discardRecovery",
Dialog::RecipientList { .. } => "recipientList",
@@ -803,6 +810,7 @@ impl Dialog {
"pasteSpecial" => Dialog::paste_special(app, &json!({})),
"renameDocument" => Dialog::RenameDocument { name: app.title_stem(), message: String::new(), focus: true },
"about" => Dialog::About { tab: 0 },
"fontDownload" => crate::font_download::open(app),
"contributors" => Dialog::About { tab: 1 },
"models" => Dialog::About { tab: 2 },
"languageData" => Dialog::About { tab: 3 },
+1
View File
@@ -188,6 +188,7 @@ pub fn show(app: &mut WordApp, ctx: &egui::Context) {
Dialog::Icons { .. } => "Icons",
Dialog::PasteSpecial { .. } => "Paste Special",
Dialog::About { .. } => "About WordCraft",
Dialog::FontDownload { .. } => "Get the Missing Fonts",
Dialog::SaveChanges { .. } => "WordCraft",
Dialog::DiscardRecovery { .. } => "Discard recovery copy",
Dialog::RecipientList { .. } => "Recipient List",
+1
View File
@@ -40,6 +40,7 @@ pub(super) fn body(app: &mut WordApp, ui: &mut Ui, d: &mut Dialog) -> bool {
Dialog::Icons { query, category, selected } => icons_body(app, ui, query, category, selected),
Dialog::PasteSpecial { .. } => paste_special_dialog(app, ui, d),
Dialog::About { .. } => about_dialog(ui, d),
Dialog::FontDownload { fonts, dont_show } => crate::font_download::body(app, ui, fonts, dont_show),
Dialog::DiscardRecovery { .. } => discard_recovery_dialog(app, ui, d),
Dialog::SaveChanges { .. } => save_changes_dialog(app, ui, d),
Dialog::RecipientList { fields, rows, message, filter } => recipient_list(app, ui, fields, rows, message, filter),
+308
View File
@@ -0,0 +1,308 @@
//! "Get the missing fonts": a document opened from a file that asks for fonts their publisher
//! gives away (Microsoft's Aptos, the default in recent Word documents) is told about once, in a
//! box that names each font, the font drawn in its place, the publisher's download page and how
//! to install the download. "Don't show this again" turns the box off (File › Options › General
//! turns it back on); the status bar's missing-fonts menu opens it by hand.
//!
//! The list is `font.missing`'s ([`wordcraft_engine::cmd::font::missing_fonts`]), so agents read
//! the same fonts and pages without the box. WordCraft only links to the page: it never
//! downloads or bundles the fonts.
use serde::Serialize;
use crate::WordApp;
use crate::dialogs::Dialog;
use crate::theme::{Tokens, semibold};
/// Most documents the box has been considered for that are remembered: beyond that the oldest
/// are forgotten (a document shown again after hundreds of others may be told about twice).
const REMEMBERED: usize = 256;
/// Most fonts the box lists: a hostile file can name thousands.
const LISTED: usize = 12;
/// The height the box takes around what scrolls: the title bar, the row with Close and the
/// window's margins.
const ROOM_AROUND_SCROLL: f32 = 190.0;
/// The least height what scrolls gets, however short the window.
const SCROLL_MIN: f32 = 80.0;
/// One missing font the box lists.
#[derive(Clone, Debug, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct Row {
/// The font the document asks for.
pub font: String,
/// The families its text is drawn in instead.
pub shown_as: Vec<String>,
/// The download the font comes in (a family: "Aptos" holds Aptos Display too).
pub download: String,
/// Who publishes the font and hosts the page.
pub publisher: String,
/// The publisher's download page.
pub url: String,
/// The publisher limits who may use the font: the box says so beside the download.
pub restricted: bool,
}
/// The shown document's missing fonts that can be downloaded.
pub(crate) fn rows(app: &mut WordApp) -> Vec<Row> {
let layout = app.session.layout();
let missing = app.cached_missing_fonts(&layout).0;
missing
.iter()
.filter_map(|f| {
let d = f.download?;
Some(Row {
font: f.font.clone(),
shown_as: f.shown_as.clone(),
download: d.name.clone(),
publisher: d.publisher.clone(),
url: d.url.clone(),
restricted: d.restricted,
})
})
.take(LISTED)
.collect()
}
/// The box for the shown document (`ui.dialog {"name": "fontDownload"}`), listing nothing when
/// none of its missing fonts can be downloaded.
pub(crate) fn open(app: &mut WordApp) -> Dialog {
Dialog::FontDownload { fonts: rows(app), dont_show: !app.ui.font_download_notice }
}
/// The document `id` has been considered for the box, and is not considered again.
fn remember(app: &mut WordApp, id: u64) {
if app.font_notice_seen.len() >= REMEMBERED {
app.font_notice_seen.remove(0);
}
app.font_notice_seen.push(id);
}
/// A document a script opened is not told about: programmatic calls never open dialogs.
pub(crate) fn skip(app: &mut WordApp, id: u64) {
if !app.font_notice_seen.contains(&id) {
remember(app, id);
}
}
impl WordApp {
/// Bring the box up by itself when a document opened from a file is missing fonts that can be
/// downloaded. The desktop and web apps call this once at start-up; unit tests and `ui_shot`
/// don't, so what they show doesn't depend on the fonts installed where they run.
pub fn tell_about_font_downloads(&mut self) {
self.font_notices = true;
}
}
/// Show the box once for a document opened from a file, when it is missing fonts that can be
/// downloaded and the box hasn't been turned off. Called every frame; it waits for an open
/// dialog or the File tab to close rather than replace them.
pub(crate) fn tick(app: &mut WordApp) {
let id = app.session.document_id();
if !app.font_notices || app.font_notice_seen.contains(&id) || app.dialog.is_some() || app.ui.backstage {
return;
}
remember(app, id);
// A new document has no file: its fonts are WordCraft's choice, not a document's from elsewhere.
if !app.ui.font_download_notice || app.session.path.is_none() {
return;
}
let fonts = rows(app);
if !fonts.is_empty() {
app.dialog = Some(Dialog::FontDownload { fonts, dont_show: false });
}
}
/// How to install a downloaded font on this system, after the files are unpacked.
fn system_install_steps() -> &'static str {
if cfg!(target_arch = "wasm32") {
""
} else if cfg!(target_os = "windows") {
tl!("Or install it for every app: select the font files, right-click and choose Install, then restart WordCraft.")
} else if cfg!(target_os = "macos") {
tl!("Or install it for every app: double-click each font file and choose Install in Font Book, then restart WordCraft.")
} else {
tl!("Or install it for every app: copy the font files to the .local/share/fonts folder in your home folder, then restart WordCraft.")
}
}
/// The Close button (Escape too) at the far side, in a row of its own: the far-side layout
/// then takes one row's height and not the rest of the window's.
fn close_row(ui: &mut egui::Ui) -> bool {
let mut close = ui.input(|i| i.key_pressed(egui::Key::Escape));
ui.horizontal(|ui| {
ui.with_layout(crate::rtl::far_side(ui, egui::Align::Center), |ui| {
close |= ui.add(crate::widgets::primary_button(ui, tl!("Close"))).clicked();
});
});
close
}
/// The fonts one download brings: listed once, however many of its fonts are missing.
struct Group<'a> {
/// Its missing fonts.
fonts: Vec<&'a str>,
/// The families they are drawn in instead, each once.
shown_as: Vec<&'a str>,
publisher: &'a str,
url: &'a str,
/// The publisher limits who may use them.
restricted: bool,
}
/// `fonts` by download page, in the order first met.
fn groups(fonts: &[Row]) -> Vec<Group<'_>> {
let mut out: Vec<Group> = Vec::new();
for f in fonts {
let at = match out.iter().position(|g| g.url == f.url && g.publisher == f.publisher && g.restricted == f.restricted) {
Some(at) => at,
None => {
out.push(Group { fonts: Vec::new(), shown_as: Vec::new(), publisher: &f.publisher, url: &f.url, restricted: f.restricted });
out.len() - 1
}
};
if let Some(g) = out.get_mut(at) {
g.fonts.push(&f.font);
for s in &f.shown_as {
if !g.shown_as.contains(&s.as_str()) {
g.shown_as.push(s);
}
}
}
}
out
}
/// A step's number in a filled circle.
fn step_badge(ui: &mut egui::Ui, n: usize, t: &Tokens) {
let (r, _) = ui.allocate_exact_size(egui::vec2(20.0, 20.0), egui::Sense::hover());
ui.painter().circle_filled(r.center(), 10.0, t.accent);
ui.painter().text(r.center(), egui::Align2::CENTER_CENTER, n.to_string(), semibold(12.0), t.on_accent);
}
/// The box's content: a headline, a card to a download with its button, the two steps that
/// add the downloaded font (Add Font… beside the step that uses it), then the opt-out and Close.
/// Returns true to close.
pub(crate) fn body(app: &mut WordApp, ui: &mut egui::Ui, fonts: &[Row], dont_show: &mut bool) -> bool {
ui.set_width(520.0);
if fonts.is_empty() {
ui.label(tl!("None of the fonts this document is missing can be downloaded free of charge from their publisher."));
ui.add_space(8.0);
return close_row(ui);
}
let t = Tokens::get(ui.ctx());
let mut open_url = None;
let mut add = false;
// Everything above the row with Close scrolls when the window is too short for it, so
// that row is on screen and the rest in reach however many downloads there are.
let room = (ui.ctx().content_rect().height() - ROOM_AROUND_SCROLL).max(SCROLL_MIN);
egui::ScrollArea::vertical().id_salt("font_downloads").max_height(room).show(ui, |ui| {
ui.horizontal(|ui| {
let (r, _) = ui.allocate_exact_size(egui::vec2(34.0, 34.0), egui::Sense::hover());
crate::icons::paint(ui.painter(), r, "fontMissing", t.icon, t.accent);
ui.add_space(4.0);
ui.vertical(|ui| {
ui.spacing_mut().item_spacing.y = 3.0;
// `fonts` holds only the missing fonts that can be downloaded: the document may
// be missing others, so the headline counts downloads, not missing fonts.
let headline = crate::i18n::tn(fonts.len() as u64, "{n} missing font can be downloaded", "{n} missing fonts can be downloaded");
ui.label(egui::RichText::new(headline).font(semibold(15.0)));
let note =
tl!("Text in a missing font is shown in another font, so it may not look as it was written. The downloads are free of charge.");
ui.label(egui::RichText::new(note).color(t.text_dim));
});
});
ui.add_space(12.0);
for g in groups(fonts) {
egui::Frame::NONE
.fill(t.panel)
.stroke(egui::Stroke::new(1.0, t.border))
.corner_radius(8)
.inner_margin(egui::Margin::symmetric(14, 10))
.show(ui, |ui| {
ui.set_width(ui.available_width());
ui.horizontal(|ui| {
ui.vertical(|ui| {
ui.spacing_mut().item_spacing.y = 2.0;
// A label to a name: each is the document's own word, not interface text.
ui.horizontal_wrapped(|ui| {
ui.spacing_mut().item_spacing.x = 0.0;
for (i, font) in g.fonts.iter().enumerate() {
if i > 0 {
ui.label(egui::RichText::new(", ").font(semibold(14.0)));
}
ui.label(egui::RichText::new(*font).font(semibold(14.0)));
}
});
let shown = crate::i18n::fmt(
tl!("Shown as {fonts} · {publisher}"),
&[("fonts", &g.shown_as.join(", ")), ("publisher", g.publisher)],
);
ui.label(egui::RichText::new(shown).color(t.text_dim));
if g.restricted {
let limits = tl!("Its licence limits who may use it: read the terms first.");
ui.label(egui::RichText::new(limits).small().color(t.text_dim));
}
});
ui.with_layout(crate::rtl::far_side(ui, egui::Align::Center), |ui| {
let tip = crate::i18n::fmt(tl!("Download from {publisher}"), &[("publisher", g.publisher)]);
if ui.add(crate::widgets::primary_button(ui, tl!("Download"))).on_hover_text(format!("{tip}\n{}", g.url)).clicked() {
open_url = Some(g.url.to_string());
}
});
});
});
ui.add_space(6.0);
}
if open_url.is_some() {
// The publisher's page, in the browser: WordCraft downloads nothing itself.
app.canvas.open_url = open_url;
}
// Whose part the licence is: WordCraft only points at the publisher's page.
let terms = tl!("Fonts come with licence terms. Read the publisher's terms on the download page: following them is up to you.");
ui.label(egui::RichText::new(terms).small().color(t.text_dim));
ui.add_space(12.0);
ui.label(egui::RichText::new(tl!("Then add it to WordCraft")).font(semibold(14.0)));
ui.add_space(4.0);
ui.horizontal(|ui| {
step_badge(ui, 1, &t);
ui.label(tl!("Download the font. If it comes as a .zip file, unpack it."));
});
ui.add_space(2.0);
ui.horizontal(|ui| {
step_badge(ui, 2, &t);
ui.label(tl!("Pick the font files:"));
add = ui.button(tl!("Add Font…")).clicked();
});
let system = system_install_steps();
if !system.is_empty() {
ui.add_space(6.0);
ui.label(egui::RichText::new(system).small().color(t.text_dim));
}
});
ui.add_space(12.0);
ui.separator();
ui.add_space(8.0);
// The downloads are the accent buttons; Close (Escape too) is plain beside them.
let mut close = ui.input(|i| i.key_pressed(egui::Key::Escape));
ui.horizontal(|ui| {
if ui.checkbox(dont_show, tl!("Don't show this again")).changed() {
// Kept at once, however the box is closed.
app.ui.font_download_notice = !*dont_show;
}
ui.with_layout(crate::rtl::far_side(ui, egui::Align::Center), |ui| {
close |= ui.button(tl!("Close")).clicked();
});
});
if add {
// The file picker replaces the box; the added font lays the document out again.
let _ = app.run("font.add", serde_json::json!({}));
return true;
}
close
}
#[cfg(test)]
mod tests;
+253
View File
@@ -0,0 +1,253 @@
use egui_kittest::kittest::Queryable;
use serde_json::json;
use wordcraft_engine::Session;
use super::*;
/// Aptos is installed on this machine: nothing is missing, so there is nothing to tell.
fn aptos_installed() -> bool {
wordcraft_fonts::FontDb::global().families().iter().any(|f| f.eq_ignore_ascii_case("Aptos"))
}
/// A document with text in Aptos (which Microsoft gives away) and in a font nobody has,
/// opened from a file when `opened`.
fn app(opened: bool) -> WordApp {
let mut a = WordApp::new(Session::new(wordcraft_doc::Document::new()), crate::Services::default());
a.session.run("format.font", &json!({"name": "Aptos"})).unwrap();
a.session.run("text.insert", &json!({"text": "Lanterns"})).unwrap();
a.session.run("text.newParagraph", &json!({})).unwrap();
a.session.run("format.font", &json!({"name": "Wordcraft Nowhere Face"})).unwrap();
a.session.run("text.insert", &json!({"text": "Harbour"})).unwrap();
if opened {
a.session.path = Some("letter.docx".into());
}
a.tell_about_font_downloads();
a
}
fn harness(app: WordApp) -> egui_kittest::Harness<'static, WordApp> {
harness_sized(app, egui::vec2(1440.0, 900.0))
}
fn harness_sized(app: WordApp, size: egui::Vec2) -> egui_kittest::Harness<'static, WordApp> {
let mut h = egui_kittest::Harness::builder().with_size(size).build_ui_state(
|ui, app: &mut WordApp| {
let ctx = ui.ctx().clone();
app.logic(&ctx);
app.ui(ui);
},
app,
);
h.run_steps(4);
h
}
fn shown(h: &egui_kittest::Harness<'static, WordApp>) -> Option<&'static str> {
h.state().dialog.as_ref().map(Dialog::name)
}
#[test]
fn an_opened_document_missing_a_downloadable_font_is_told_about_once() {
if aptos_installed() {
return;
}
let mut h = harness(app(true));
let Some(Dialog::FontDownload { fonts, dont_show }) = h.state().dialog.clone() else { panic!("no box: {:?}", shown(&h)) };
// Only the font that can be downloaded, with what it is drawn in and Microsoft's page.
assert_eq!(fonts.len(), 1, "{fonts:?}");
assert_eq!((fonts[0].font.as_str(), fonts[0].publisher.as_str()), ("Aptos", "Microsoft"));
assert!(fonts[0].url.starts_with("https://www.microsoft.com/"), "{}", fonts[0].url);
assert!(!fonts[0].shown_as.is_empty() && !fonts[0].shown_as.iter().any(|f| f == "Aptos"), "{:?}", fonts[0].shown_as);
assert!(!dont_show);
// The box says so: the font, its substitute and publisher, the download, how to add it,
// and the opt-out.
// The headline counts what can be downloaded, not every font the document is missing.
assert!(h.query_by_label("1 missing font can be downloaded").is_some());
assert!(h.query_by_label_contains("isn't installed").is_none(), "other fonts may be missing too");
assert!(h.query_by_label("Aptos").is_some());
assert!(h.query_by_label(format!("Shown as {} · Microsoft", fonts[0].shown_as.join(", ")).as_str()).is_some());
assert!(h.query_by_label_contains("following them is up to you").is_some(), "whose part the licence is");
assert!(h.query_by_label_contains("limits who may use it").is_none(), "Aptos has no such limit");
assert!(h.query_by_label("Pick the font files:").is_some());
assert!(h.query_by_role_and_label(egui::accesskit::Role::Button, "Add Font…").is_some());
assert!(h.query_by_label("Don't show this again").is_some());
// Download opens the publisher's page in the browser; nothing is downloaded here.
h.get_by_role_and_label(egui::accesskit::Role::Button, "Download").click();
let mut opened = false;
for _ in 0..4 {
h.step();
opened |= h.output().platform_output.commands.iter().any(|c| matches!(c, egui::OutputCommand::OpenUrl(u) if u.url == fonts[0].url));
}
assert!(opened, "the page is opened");
assert_eq!(shown(&h), Some("fontDownload"), "and the box stays for the next step");
// Closed, it stays closed for this document.
// The box's own Close button, below the title bar's.
let close =
h.query_all_by_role_and_label(egui::accesskit::Role::Button, "Close").max_by(|a, b| a.rect().center().y.total_cmp(&b.rect().center().y));
close.expect("a Close button").click();
h.run_steps(4);
assert_eq!(shown(&h), None);
assert!(h.state().ui.font_download_notice, "closing alone keeps the box on");
}
#[test]
fn the_box_comes_up_only_where_the_host_asked_for_it() {
let mut a = app(true);
a.font_notices = false;
let h = harness(a);
assert_eq!(shown(&h), None, "headless: tests and ui_shot");
}
#[test]
fn a_new_document_is_not_told_about() {
let h = harness(app(false));
assert_eq!(shown(&h), None, "its fonts are WordCraft's choice");
}
#[test]
fn dont_show_this_again_turns_the_box_off() {
if aptos_installed() {
return;
}
let mut h = harness(app(true));
assert_eq!(shown(&h), Some("fontDownload"));
h.get_by_label("Don't show this again").click();
h.run_steps(2);
assert!(!h.state().ui.font_download_notice, "kept at once");
// Saved between runs, and read back.
let saved = serde_json::to_value(h.state().prefs()).unwrap();
assert_eq!(saved["fontDownloadNotice"], json!(false), "{saved}");
// The next opened document isn't told about.
let mut a = app(true);
a.ui.font_download_notice = false;
let h = harness(a);
assert_eq!(shown(&h), None);
}
#[test]
fn the_box_waits_for_an_open_dialog_and_can_be_opened_by_hand() {
if aptos_installed() {
return;
}
let mut a = app(true);
a.dialog = Dialog::open("wordCount", &mut a);
let mut h = harness(a);
assert_eq!(shown(&h), Some("wordCount"), "the open dialog stays");
h.state_mut().dialog = None;
h.run_steps(3);
assert_eq!(shown(&h), Some("fontDownload"), "then the box");
h.state_mut().dialog = None;
h.run_steps(3);
assert_eq!(shown(&h), None);
// The status bar's menu (`ui.dialog`) opens it again, even with the box turned off.
h.state_mut().ui.font_download_notice = false;
h.state_mut().run("ui.dialog", json!({"name": "fontDownload"})).unwrap();
let Some(Dialog::FontDownload { fonts, dont_show }) = h.state().dialog.clone() else { panic!("no box") };
assert_eq!(fonts.len(), 1);
assert!(dont_show, "the box shows the setting");
}
#[test]
fn a_document_a_script_opened_is_not_told_about() {
let mut a = app(true);
let id = a.session.document_id();
skip(&mut a, id);
skip(&mut a, id);
assert_eq!(a.font_notice_seen, vec![id]);
let h = harness(a);
assert_eq!(shown(&h), None);
}
#[test]
fn a_document_with_nothing_to_download_says_so_when_asked() {
let mut a = WordApp::new(Session::new(wordcraft_doc::Document::from_text("Lanterns")), crate::Services::default());
a.session.run("select.all", &json!({})).unwrap();
a.session.run("format.font", &json!({"name": "Wordcraft Nowhere Face"})).unwrap();
a.session.path = Some("letter.docx".into());
a.tell_about_font_downloads();
let mut h = harness(a);
assert_eq!(shown(&h), None, "a font nobody gives away brings up no box");
h.state_mut().run("ui.dialog", json!({"name": "fontDownload"})).unwrap();
h.run_steps(3);
assert!(h.query_by_label_contains("None of the fonts").is_some());
}
/// Fonts that come in one download share a card and a Download button.
#[test]
fn fonts_of_one_download_are_listed_together() {
let row = |font: &str, shown: &str, url: &str| Row {
font: font.into(),
shown_as: vec![shown.into()],
download: font.into(),
publisher: "Microsoft".into(),
url: url.into(),
restricted: false,
};
let fonts =
[row("Aptos", "Carlito", "https://a"), row("Cascadia Code", "Liberation Mono", "https://c"), row("Aptos Display", "Carlito", "https://a")];
let g = groups(&fonts);
assert_eq!(g.len(), 2);
assert_eq!(
(g[0].fonts.as_slice(), g[0].shown_as.as_slice(), g[0].url),
(["Aptos", "Aptos Display"].as_slice(), ["Carlito"].as_slice(), "https://a")
);
assert_eq!((g[1].fonts.as_slice(), g[1].url), (["Cascadia Code"].as_slice(), "https://c"));
}
/// As many downloads as the box lists, in a short window and a very short one: what is above
/// the row with Close scrolls, so the opt-out and Close stay on screen and the box in the window.
#[test]
fn many_downloads_in_a_short_window_leave_the_opt_out_and_close_on_screen() {
for height in [600.0, 400.0] {
let mut a = WordApp::new(Session::new(wordcraft_doc::Document::from_text("Lanterns")), crate::Services::default());
let fonts: Vec<Row> = (0..LISTED)
.map(|i| Row {
font: format!("Harbour Face {i}"),
shown_as: vec!["Carlito".into()],
download: format!("Harbour Face {i}"),
publisher: "Lantern Works".into(),
url: format!("https://example.org/{i}"),
restricted: false,
})
.collect();
a.dialog = Some(Dialog::FontDownload { fonts, dont_show: false });
let h = harness_sized(a, egui::vec2(1000.0, height));
// The steps are still there, to scroll to.
assert!(h.query_by_label("Add Font…").is_some());
for label in ["Don't show this again", "Close"] {
// Close is also the window's close box: every match is on screen.
let rects: Vec<egui::Rect> = h.query_all_by_label(label).map(|n| n.rect()).collect();
assert!(!rects.is_empty(), "no {label}");
for r in rects {
assert!(r.min.y >= 0.0 && r.max.y <= height, "{label} is at {r:?}, outside the {height} pt window");
}
}
}
}
/// A font whose publisher limits its use says so on its card, and has a card of its own.
#[test]
fn a_font_with_a_limited_licence_says_so() {
let mut a = WordApp::new(Session::new(wordcraft_doc::Document::from_text("Lanterns")), crate::Services::default());
a.session.run("select.all", &json!({})).unwrap();
a.session.run("format.font", &json!({"name": "Segoe UI Variable"})).unwrap();
let layout = a.session.layout();
if !wordcraft_engine::cmd::font::missing_fonts(&a.session.doc, &layout).iter().any(|f| f.font == "Segoe UI Variable") {
return; // Installed here: not missing.
}
a.dialog = Dialog::open("fontDownload", &mut a);
let Some(Dialog::FontDownload { fonts, .. }) = a.dialog.clone() else { panic!("no box") };
assert!(fonts.iter().any(|f| f.font == "Segoe UI Variable" && f.restricted), "{fonts:?}");
let h = harness(a);
assert!(h.query_by_label_contains("limits who may use it").is_some());
}
#[test]
fn only_so_many_documents_are_remembered() {
let mut a = app(false);
for id in 0..(REMEMBERED as u64 + 50) {
skip(&mut a, id);
}
assert_eq!(a.font_notice_seen.len(), REMEMBERED);
assert_eq!(a.font_notice_seen.last(), Some(&(REMEMBERED as u64 + 49)));
}
+2
View File
@@ -306,6 +306,7 @@ fn untranslated_interface_text() {
"tableProperties",
"pasteSpecial",
"about",
"fontDownload",
"contributors",
"models",
"languageData",
@@ -752,6 +753,7 @@ fn the_arabic_interface_is_mirrored_and_never_panics() {
"enclose",
"pasteSpecial",
"about",
"fontDownload",
"contributors",
"models",
"insertMergeField",
+10
View File
@@ -43,6 +43,7 @@ pub mod equation_tab;
#[cfg(not(target_arch = "wasm32"))]
pub mod file_browser;
pub mod file_dialogs;
pub mod font_download;
pub mod format_shape;
pub mod frame;
pub mod guides;
@@ -435,6 +436,9 @@ pub struct UiState {
/// spaces, or characters without spaces. Right-click the count in the status bar, or File ›
/// Options › Display. Junk saved values read as words.
pub status_count: StatusCount,
/// File › Options › General: tell about a document's missing fonts that can be downloaded
/// ([`font_download`]); the box's "Don't show this again" turns it off.
pub font_download_notice: bool,
/// How far a mouse-wheel notch scrolls (#6146).
#[serde(default, alias = "scrollSpeed")]
pub scroll_speed: ScrollSpeed,
@@ -540,6 +544,7 @@ impl Default for UiState {
dictionary: Vec::new(),
display_server: DisplayServer::Auto,
status_count: StatusCount::default(),
font_download_notice: true,
scroll_speed: ScrollSpeed::default(),
touchpad_scroll_speed: ScrollSpeed::default(),
}
@@ -690,6 +695,11 @@ pub struct WordApp {
/// Focus (#1678): the ribbon is dropped down from the Focus bar, over the page.
pub(crate) focus_ribbon: bool,
pub(crate) updates_rt: updates_ui::Runtime,
/// The documents (by id) the missing-fonts download box has been considered for
/// ([`font_download::tick`]), in every window.
pub(crate) font_notice_seen: Vec<u64>,
/// Whether that box comes up by itself ([`WordApp::tell_about_font_downloads`]).
pub(crate) font_notices: bool,
/// Language pack installs, the missing-pack bar ([`language_packs`]).
pub(crate) packs_rt: language_packs::Runtime,
control_rx: Option<std::sync::mpsc::Receiver<ControlRequest>>,
+1
View File
@@ -250,6 +250,7 @@ fn general(app: &mut WordApp, ui: &mut Ui) {
let _ = app.run("view.darkMode", json!({"value": dark_page}));
}
ui.checkbox(&mut app.ui.show_discord, tl!("Show the community button in the title bar"));
ui.checkbox(&mut app.ui.font_download_notice, tl!("Tell me when a document's missing fonts can be downloaded"));
default_app_button(app, ui);
ui.add_space(8.0);
section(ui, "Start-up");
+3
View File
@@ -175,6 +175,8 @@ impl WordApp {
mac_titlebar: _,
control_rx: _,
updates_rt: _,
font_notice_seen: _,
font_notices: _,
packs_rt: _,
// The control channel's screenshots and input go to the active window, whichever it is.
shots: _,
@@ -315,6 +317,7 @@ impl WordApp {
print_backgrounds: _,
update_fields_before_print: _,
dictionary: _,
font_download_notice: _,
// Where each task pane sits, and their order.
styles_pane_dock: _,
styles_pane_float: _,
+2
View File
@@ -56,6 +56,7 @@ fn listing_stops_at_the_field_budget() {
// Finding 16: rich text with alternating formatting is built in one pass, not run by run with
// every earlier run merged again.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn rich_text_with_many_runs_is_linear() {
let html: String = "<i>a</i>b".repeat(50_000);
let t = Instant::now();
@@ -82,6 +83,7 @@ fn call(b: &mut Bridge, s: &mut Session, method: &str, args: Value) -> Result<Va
// Finding 32: Document_convert checks each distinct id once against one list of the fields.
#[test]
#[cfg_attr(coverage, ignore = "coverage instrumentation skews the measured time")]
fn convert_with_one_id_repeated_is_one_check() {
let mut doc = Document::from_text("");
// Checking each id against a fresh list of the fields took 81 s with 300 of them; with 1,000
+2 -1
View File
File diff suppressed because one or more lines are too long
+2 -1
View File
File diff suppressed because one or more lines are too long