aleleba/wordcraft
1
0
Fork 0
mirror of https://github.com/storytold/wordcraft.git synced 2026-10-24 10:27:44 -06:00
Code Issues Packages Projects Releases Wiki Activity

PDF export: Hebrew and Arabic text extracts in reading order (#1383) (#1508)

Browse Source 
* #1383: WIP

* Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Dmitrii Khlapov <19991457+dkhlapov@users.noreply.github.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
This commit is contained in:
Dmitrii Khlapov
2026-10-14 14:07:08 -04:00
committed by GitHub
co-authored by Dmitrii Khlapov Brandon Thomas
parent b1e8ae07e7
commit 5e8c5b3c4e
5 changed files with 378 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ROADMAP.md
+3 -1
View File
@@ -2,7 +2,7 @@
**Stage: alpha** · next: beta, ~15 points (60% → 75% ready for real work) and ~150–230 h away
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** trivial (progress log: ODT sections, #1379; previously trivial: progress log: HTML table cells and region paper for opened HTML, #1381; previously trivial: progress log: `.doc` table shading and the extra column, #1380; previously trivial: progress log: `.doc` table shading, columns and merged rows, #1380; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** trivial (progress log: ODT sections, #1379; previously trivial: progress log: Hebrew and Arabic text in exported PDFs copies and searches as whole words, #1383; previously trivial: progress log: HTML table cells and region paper for opened HTML, #1381; previously trivial: progress log: `.doc` table shading and the extra column, #1380; previously trivial: progress log: `.doc` table shading, columns and merged rows, #1380; previously trivial: progress log: privacy options on save, #1062; previously trivial: progress log: equation pictures in theme colours on Windows, #1002; previously trivial: symbol fonts, #776; previously trivial: the save prompt answers from the keyboard, #861; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw tab ink, #307, and Draw Table and Eraser, #303, landed: catalog 402/434; previously minor: readiness table with hours per audience; full number back to the additive weighted sum, 60%; mainstream and essentials numbers added; alpha gate checked; previously major: full re-measure against Word for Mac 16.113.4; restructured to the craftrules progress-docs standard) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4
> **Printing update:** 2026-10-12 · direct desktop printing through craft-print; details below.
@@ -153,6 +153,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm
| 2026-10-13 | Copying an equation also puts MathML in HTML on the clipboard beside the linear text like `y=2x_s` (#816; Word/LibreOffice paste not yet verified; OMML is #1297). HTML export writes equations as MathML too, and opening or pasting HTML with MathML gives equations back |
| 2026-10-13 | The Equation tab's galleries, structure buttons and symbols keep the theme's colours on Windows: their pictures share a few textures written in place (no wipe of pictures already on screen, room for the built-ins at high display scales), instead of hundreds of small textures freed in bulk, which on DirectX 12 showed inverted, red or teal tiles (#1002) |
| 2026-10-14 | ODT files open with their sections (#1379): a paragraph that switches to another page style starts a new section on a new page, with that page's size, orientation, margins and columns and, when the file asks, page numbering restarting; a two-column `text:section` becomes a continuous two-column section. Before, the whole document was one portrait section, so landscape pages, columns and restarts were lost on open and on save to DOCX |
| 2026-10-14 | Hebrew and Arabic in exported PDFs copy and search as whole words (#1383): right-to-left glyphs are written left to right as they appear, the order Chrome and Edge (PDFium) and poppler read text back from (checked by hand), instead of in reading order, which they reversed (`שלום` came back as `םולש`, Arabic in pieces). Lam-alef stays whole and vowel marks follow their letters, though Chrome and poppler can still break a fully vowelled word with spaces. The page looks the same. Readers that take glyphs strictly in file order get only plain Hebrew runs right, through `/ReversedChars`; screen readers are unverified |
| 2026-10-14 | HTML import (#1381): header cells (`<th>`) are bold unless their own CSS says otherwise, and `text-align` on a cell, its row, row group or table aligns the cell's paragraphs (a paragraph's own alignment wins), so `wordcraft-cli convert page.html out.docx` writes the bold header and `w:jc` right-aligned cells. HTML, Markdown and plain text, which give no page size, open on the region's paper (A4 in Italy, Letter in the US) like a new blank document, in the app and in `wordcraft-cli`; documents with their own page size keep it |
| 2026-10-14 | Custom page sizes (#1433): Layout › Size ends with More Paper Sizes…, which opens Page Setup's new Paper group — a Paper size list (the named sizes, or Custom size) and Width and Height in the measurement unit; OK sets the size only when it changed, and a wider-than-tall size comes out landscape. `layout.size` now clamps each side to Word's 0.1"–22" and refuses a page the margins don't fit (the margins rule: 36 pt of text room), and Page Setup checks the size and margins together before applying either |
| 2026-10-13 | Highlighted text reads on a dark page (View › Switch Modes, #1252): highlights and character shading are dimmed, keeping their hue (yellow turns a dim olive, as in Word), and the text on them is drawn light (coloured text below 4.5:1 is drawn white, and underlines and strikethroughs follow the same rule), instead of white text on bright yellow; light page, print and exports unchanged |
@@ -325,6 +326,7 @@ Ranked, to beta (detail and remaining milestones in [docs/roadmap.md](docs/roadm
| 2026-10-13 | trivial | Progress log: clicks on ribbon chrome keep typing in the document (#1093) |
| 2026-10-13 | trivial | Progress log: dragged shapes wrap in their new column (#884) |
| 2026-10-14 | trivial | Progress log: ODT sections on import (#1379) |
| 2026-10-14 | trivial | Progress log: right-to-left text in exported PDFs extracts as whole words (#1383) |
| 2026-10-14 | trivial | Progress log: HTML header cells, cell alignment and region paper for opened HTML/Markdown/text (#1381) |
| 2026-10-14 | trivial | Progress log: custom page size from More Paper Sizes… and Page Setup (#1433) |
| 2026-10-13 | trivial | Progress log: highlighted text on a dark page (#1252) |
crates/pdf/src/lib.rs
+119 -8
View File
@@ -241,6 +241,103 @@ fn glyph_outlines(face: &FaceRef, size: f32, glyphs: &[(u32, f32, f32)]) -> Opti
pb.finish()
}
/// Whether a run's glyphs already run left to right, as PDF text extractors (PDFium in Chrome and
/// Edge, pdf.js, poppler) take them to: they read a run's glyphs as written left to right and turn
/// right-to-left words back into reading order themselves. Layout gives right-to-left text in
/// reading order.
fn left_to_right(glyphs: &[(u32, f32, f32)]) -> bool {
glyphs.windows(2).all(|w| matches!(w, [a, b] if a.1 <= b.1))
}
/// Per-glyph text ranges for clusters of several glyphs (a letter and its vowel marks) when each
/// glyph is one of the cluster's characters (through the font's character map, one glyph at most
/// left to take the character left over), so a viewer
/// reversing a right-to-left word puts every mark back after its letter. Other clusters, such
/// as a ligature drawn as two glyphs, keep their shared range.
fn split_clusters(
text: &str,
glyphs: &[(u32, f32, f32)],
ranges: &[std::ops::Range<usize>],
rev: &HashMap<u32, char>,
) -> Vec<std::ops::Range<usize>> {
let mut out = ranges.to_vec();
let mut i = 0;
while i < ranges.len() {
let Some(r) = ranges.get(i) else { break };
let n = ranges.get(i..).map_or(1, |rest| rest.iter().take_while(|x| *x == r).count()).max(1);
let chars: Vec<(usize, char)> = text.get(r.clone()).map(|t| t.char_indices().map(|(k, c)| (r.start + k, c)).collect()).unwrap_or_default();
if n > 1 && chars.len() == n {
// Glyphs the character map knows take their characters; one left over (a letter's
// contextual form, which the map doesn't list) takes the one character left.
let mut used = vec![false; n];
let mut of: Vec<Option<usize>> = (i..i + n)
.map(|g| {
let c = rev.get(&glyphs.get(g)?.0)?;
let k = chars.iter().enumerate().position(|(k, (_, ch))| ch == c && !used.get(k).copied().unwrap_or(true))?;
*used.get_mut(k)? = true;
Some(k)
})
.collect();
let free: Vec<usize> = (0..n).filter(|k| !used.get(*k).copied().unwrap_or(true)).collect();
if let ([k], [slot]) = (free.as_slice(), of.iter_mut().filter(|o| o.is_none()).collect::<Vec<_>>().as_mut_slice()) {
**slot = Some(*k);
}
let parts: Option<Vec<std::ops::Range<usize>>> = of
.iter()
.map(|k| {
let (a, ch) = chars.get((*k)?)?;
Some(*a..a + ch.len_utf8())
})
.collect();
if let (Some(parts), Some(dst)) = (parts, out.get_mut(i..i + n)) {
dst.clone_from_slice(&parts);
}
}
i += n;
}
out
}
/// Indices of a run's glyphs in visual order, left to right: its clusters (consecutive glyphs
/// sharing a range in `clusters`, such as a letter and its marks, or a lam-alef ligature) sorted
/// by their leftmost glyph and kept together. Inside a cluster, glyphs go in reverse order of
/// their own text (`parts`) when it is right to left, so a viewer reversing the word reads the
/// letter before its marks.
fn visual_order(text: &str, glyphs: &[(u32, f32, f32)], clusters: &[std::ops::Range<usize>], parts: &[std::ops::Range<usize>]) -> Vec<usize> {
let mut groups: Vec<(f32, Vec<usize>)> = Vec::new();
for (i, (_, x, _)) in glyphs.iter().enumerate() {
match groups.last_mut() {
Some((left, idx)) if i > 0 && clusters.get(i).is_some() && clusters.get(i) == clusters.get(i - 1) => {
*left = left.min(*x);
idx.push(i);
}
_ => groups.push((*x, vec![i])),
}
}
// Stable: clusters at the same x keep their order, as do glyphs sharing their text.
groups.sort_by(|a, b| a.0.total_cmp(&b.0));
groups
.into_iter()
.flat_map(|(_, mut idx)| {
let rtl = idx.first().and_then(|&g| text.get(clusters.get(g)?.clone())).is_some_and(|t| t.chars().any(wordcraft_fonts::is_rtl));
if rtl {
idx.sort_by_key(|&g| std::cmp::Reverse(parts.get(g).map_or(0, |r| r.start)));
}
idx
})
.collect()
}
/// The y most of a run's glyphs share: its baseline, which raised or lowered marks leave.
fn baseline(glyphs: &[(u32, f32, f32)]) -> Option<f32> {
let mut counts: HashMap<u32, usize> = HashMap::new();
for (_, _, y) in glyphs {
*counts.entry(y.to_bits()).or_insert(0) += 1;
}
// Ties go to the lower y bits, so the result doesn't depend on hash order.
counts.into_iter().max_by_key(|&(bits, n)| (n, std::cmp::Reverse(bits))).map(|(bits, _)| f32::from_bits(bits))
}
/// A chart's or diagram's alt text: its own, else what it is ("chart", "diagram").
fn graphic_alt(alt: &str, kind: GraphicKind) -> &str {
if alt.trim().is_empty() { kind.noun() } else { alt }
@@ -1148,7 +1245,9 @@ impl Exporter<'_> {
if !size.is_finite() || size <= 0.0 || size > 5000.0 {
return;
}
let Some(&(_, x0, y0)) = glyphs.first() else { return };
if glyphs.is_empty() {
return;
}
if !glyphs.iter().all(|(_, x, y)| ok(*x) && ok(*y)) {
return;
}
@@ -1162,21 +1261,33 @@ impl Exporter<'_> {
let valid = known.len() == glyphs.len() && known.iter().all(|r| r.start < r.end && text.get(r.clone()).is_some());
let (txt, ranges) = if valid { (text.to_string(), known.to_vec()) } else { self.map_text(face, glyphs, text) };
let upem = face.upem.max(1.0) as f32;
let kg: Vec<KrillaGlyph> = glyphs
// Right-to-left text is written left to right as it appears (see `left_to_right`); glyphs
// already in that order are written as they are.
let (order, ranges) = if left_to_right(glyphs) {
((0..glyphs.len()).collect(), ranges)
} else {
let parts = split_clusters(&txt, glyphs, &ranges, &self.cmap(face));
(visual_order(&txt, glyphs, &ranges, &parts), parts)
};
let placed: Vec<(u32, f32, f32, std::ops::Range<usize>)> =
order.iter().filter_map(|&i| Some((glyphs.get(i).copied()?, ranges.get(i).cloned()?))).map(|((g, x, y), r)| (g, x, y, r)).collect();
let Some(&(_, x0, y0, _)) = placed.first() else { return };
// The run's baseline: the y most of its glyphs share (marks sit higher or lower).
let base = baseline(glyphs).unwrap_or(y0);
let kg: Vec<KrillaGlyph> = placed
.iter()
.zip(ranges)
.enumerate()
.map(|(k, ((g, x, y), r))| {
let adv = match glyphs.get(k + 1) {
Some((_, nx, _)) => (nx - x) / size,
.map(|(k, (g, x, y, r))| {
let adv = match placed.get(k + 1) {
Some((_, nx, _, _)) => (nx - x) / size,
None => face.advance(*g) as f32 / upem,
};
KrillaGlyph::new(GlyphId::new(*g), adv, 0.0, (y0 - y) / size, 0.0, r, None)
KrillaGlyph::new(GlyphId::new(*g), adv, 0.0, (y0 - y) / size, 0.0, r.clone(), None)
})
.collect();
let pushed = italic;
if italic {
s.push_transform(&Transform::from_row(1.0, 0.0, -0.21, 1.0, 0.21 * y0, 0.0));
s.push_transform(&Transform::from_row(1.0, 0.0, -0.21, 1.0, 0.21 * base, 0.0));
}
s.set_fill(Some(fill(color, alpha)));
s.set_stroke(if bold {
crates/pdf/src/tests.rs
+250
View File
@@ -1023,3 +1023,253 @@ fn internal_links_export_in_linear_time() {
let ratio = fastest(&big) / fastest(&small).max(1e-9);
assert!(ratio < 9.0, "4x the paragraphs and links took {ratio:.1}x as long (quadratic would be ~16x)");
}
/// The reporter's four paragraphs from #1383 (Hebrew, Arabic and mixed right-to-left paragraphs,
/// and a Latin one), then right-to-left words with a lam-alef ligature and with vowel marks.
fn rtl_sample() -> Document {
let mut d = Document::from_text("שלום עולם\nمرحبا بالعالم\nייפוי כח מס' 123 – Power of Attorney\nHello world\nالسلام مَرْحَبًا שָׁלוֹם");
// Right to left: all but "Hello world".
for i in [0, 1, 2, 4] {
let at = wordcraft_doc::Pos::body(i, 0);
d.format_paragraphs(&at, &at, &|p: &mut wordcraft_doc::ParaProps| p.bidi = Some(true)).unwrap();
}
for (i, rtl) in [true, true, true, false, true].into_iter().enumerate() {
let Some(Block::Para(p)) = d.body.get(i).map(|b| &**b) else { panic!("paragraph {i}") };
assert_eq!(p.props.bidi == Some(true), rtl, "paragraph {i}");
}
d
}
/// Each glyph's Unicode text in content-stream order, with its baseline y and its outline's left
/// and right edges (none for a space); `None` text for a glyph without a ToUnicode entry of its
/// own (a cluster's extra glyph, covered by ActualText).
type StreamGlyph = (Option<String>, f64, Option<(f64, f64)>);
fn stream_glyphs(bytes: &[u8]) -> Vec<StreamGlyph> {
use hayro_interpret::font::Glyph;
use hayro_interpret::{
BlendMode, ClipPath, Context, Device, GlyphDrawMode, Image, InterpreterCache, InterpreterSettings, Paint, PathDrawMode, SoftMask,
interpret_page,
};
use hayro_syntax::Pdf;
struct Units(Vec<StreamGlyph>);
impl Device<'_> for Units {
fn set_soft_mask(&mut self, _: Option<SoftMask<'_>>) {}
fn set_blend_mode(&mut self, _: BlendMode) {}
fn draw_path(&mut self, _: &kurbo::BezPath, _: kurbo::Affine, _: &Paint<'_>, _: &PathDrawMode) {}
fn push_clip_path(&mut self, _: &ClipPath) {}
fn push_transparency_group(&mut self, _: f32, _: Option<SoftMask<'_>>, _: BlendMode) {}
fn draw_glyph(&mut self, g: &Glyph<'_>, t: kurbo::Affine, gt: kurbo::Affine, _: &Paint<'_>, mode: &GlyphDrawMode) {
if matches!(mode, GlyphDrawMode::Stroke(_)) {
return;
}
let text = match g.as_unicode() {
Some(hayro_cmap::BfString::Char(c)) => Some(c.to_string()),
Some(hayro_cmap::BfString::String(s)) => Some(s),
None => None,
};
let edges = match g {
Glyph::Outline(o) if !o.outline().elements().is_empty() => {
use kurbo::Shape;
let b = (t * gt * o.outline()).bounding_box();
Some((b.x0, b.x1))
}
_ => None,
};
self.0.push((text, (t * gt * kurbo::Point::ZERO).y, edges));
}
fn draw_image(&mut self, _: Image<'_, '_>, _: kurbo::Affine) {}
fn pop_clip_path(&mut self) {}
fn pop_transparency_group(&mut self) {}
}
let pdf = Pdf::new(bytes.to_vec()).expect("parse");
let cache = InterpreterCache::new();
let mut out = Units(Vec::new());
for page in pdf.pages().iter() {
let mut ctx = Context::new(kurbo::Affine::IDENTITY, kurbo::Rect::new(0.0, 0.0, 1.0, 1.0), &cache, pdf.xref(), InterpreterSettings::default());
interpret_page(page, &mut ctx, &mut out);
}
out.0
}
/// Words as PDF viewers' text extraction (PDFium in Chrome and Edge, pdf.js, poppler) reads them:
/// glyphs are taken to be in visual order, so a word in a right-to-left script is read with its
/// glyphs reversed. Words end at spaces, at a jump back to the left, and where the
/// baseline moves (more than a raised mark does).
fn viewer_words(bytes: &[u8]) -> Vec<String> {
let mut words = Vec::new();
let mut word: Vec<String> = Vec::new();
let mut flush = |word: &mut Vec<String>| {
if word.concat().chars().any(wordcraft_fonts::is_rtl) {
word.reverse();
}
if !word.is_empty() {
words.push(word.concat());
}
word.clear();
};
let mut last_y = None;
let mut last_edges: Option<(f64, f64)> = None;
for (text, y, edges) in stream_glyphs(bytes) {
// A new line, or a letter wholly left of the last one (a jump back): a new word. Hebrew
// and Arabic vowel marks sit anywhere over or under their letter.
let mark = text.as_deref().is_some_and(|t| t.chars().all(|c| matches!(c, '\u{591}'..='\u{5C7}' | '\u{64B}'..='\u{65F}' | '\u{670}')));
let edges = edges.filter(|_| !mark);
let apart = matches!((last_edges, edges), (Some((l0, _)), Some((_, x1))) if x1 < l0);
if last_y.is_some_and(|l: f64| (l - y).abs() > 6.0) || apart {
flush(&mut word);
}
if !mark {
last_y = Some(y);
}
if edges.is_some() {
last_edges = edges;
}
let Some(text) = text else { continue };
if text.chars().all(char::is_whitespace) {
flush(&mut word);
} else {
word.push(text);
}
}
flush(&mut word);
words
}
/// Right-to-left text reads back whole (#1383): Chrome, Edge and other PDF viewers take a run's
/// glyphs to be written left to right as they appear, and reverse right-to-left words to get
/// their reading order. Written in reading order instead, `שלום` came back as `םולש`, and Arabic
/// came apart.
#[test]
fn right_to_left_text_extracts_in_logical_order() {
let d = rtl_sample();
let words = viewer_words(&export(&d, &PdfOptions::default()).unwrap());
for w in ["שלום", "עולם", "مرحبا", "بالعالم", "ייפוי", "כח", "123", "Power", "Attorney", "Hello", "world", "السلام", "مَرْحَبًا", "שָׁלוֹם"]
{
assert!(words.iter().any(|x| x == w), "{w:?} not extracted intact from {words:?}");
}
}
/// Every glyph's outline on the page, as a rounded bounding box in page space (sorted): what the
/// page looks like, whatever order the glyphs were written in.
fn glyph_boxes(bytes: &[u8]) -> Vec<[i64; 4]> {
use hayro_interpret::font::Glyph;
use hayro_interpret::{
BlendMode, ClipPath, Context, Device, GlyphDrawMode, Image, InterpreterCache, InterpreterSettings, Paint, PathDrawMode, SoftMask,
interpret_page,
};
use hayro_syntax::Pdf;
use kurbo::Shape;
struct Boxes(Vec<[i64; 4]>);
impl Device<'_> for Boxes {
fn set_soft_mask(&mut self, _: Option<SoftMask<'_>>) {}
fn set_blend_mode(&mut self, _: BlendMode) {}
fn draw_path(&mut self, _: &kurbo::BezPath, _: kurbo::Affine, _: &Paint<'_>, _: &PathDrawMode) {}
fn push_clip_path(&mut self, _: &ClipPath) {}
fn push_transparency_group(&mut self, _: f32, _: Option<SoftMask<'_>>, _: BlendMode) {}
fn draw_glyph(&mut self, g: &Glyph<'_>, t: kurbo::Affine, gt: kurbo::Affine, _: &Paint<'_>, mode: &GlyphDrawMode) {
if matches!(mode, GlyphDrawMode::Stroke(_)) {
return;
}
// Spaces have no outline: nothing to see.
if let Glyph::Outline(o) = g
&& !o.outline().elements().is_empty()
{
let b = (t * gt * o.outline()).bounding_box();
let r = |v: f64| (v * 10.0).round() as i64;
self.0.push([r(b.x0), r(b.y0), r(b.x1), r(b.y1)]);
}
}
fn draw_image(&mut self, _: Image<'_, '_>, _: kurbo::Affine) {}
fn pop_clip_path(&mut self) {}
fn pop_transparency_group(&mut self) {}
}
let pdf = Pdf::new(bytes.to_vec()).expect("parse");
let cache = InterpreterCache::new();
let mut out = Boxes(Vec::new());
for page in pdf.pages().iter() {
let mut ctx = Context::new(kurbo::Affine::IDENTITY, kurbo::Rect::new(0.0, 0.0, 1.0, 1.0), &cache, pdf.xref(), InterpreterSettings::default());
interpret_page(page, &mut ctx, &mut out);
}
out.0.sort_unstable();
out.0
}
/// Writing right-to-left runs in reading order doesn't move a glyph: the page draws the same
/// glyph outlines at the same places as layout put them.
#[test]
fn right_to_left_glyphs_stay_where_layout_put_them() {
use kurbo::Shape;
let d = rtl_sample();
let bytes = export(&d, &PdfOptions::default()).unwrap();
let got = glyph_boxes(&bytes);
// Layout's glyphs, drawn as their outlines (y down from the page top; PDF is y up).
let lay = layout(&d, &mut LayoutCache::new(), &LayoutOptions::default());
let page = &lay.pages[0];
let h = f64::from(page.h);
let db = wordcraft_fonts::FontDb::global();
let mut want = Vec::new();
for item in wordcraft_layout::display::page_display(&d, page, &DisplayOptions::default()) {
let Draw::Glyphs { face, size, glyphs, .. } = item else { continue };
let scale = f64::from(size) / face.upem.max(1.0);
for (g, x, y) in glyphs {
let mut o = (*db.outline(&face, g)).clone();
if o.elements().is_empty() {
continue;
}
o.apply_affine(
kurbo::Affine::new([1.0, 0.0, 0.0, -1.0, 0.0, h])
* kurbo::Affine::translate((f64::from(x), f64::from(y)))
* kurbo::Affine::scale(scale),
);
let b = o.bounding_box();
let r = |v: f64| (v * 10.0).round() as i64;
want.push([r(b.x0), r(b.y0), r(b.x1), r(b.y1)]);
}
}
want.sort_unstable();
assert!(!want.is_empty());
let close = got.len() == want.len() && got.iter().zip(&want).all(|(a, b)| a.iter().zip(b).all(|(p, q)| (p - q).abs() <= 2));
assert!(close, "glyphs moved:\n got {got:?}\nwant {want:?}");
}
/// A separately formatted Latin run inside a right-to-left paragraph is written as it was, left
/// to right, and the Hebrew around it still reads back whole.
#[test]
fn latin_run_in_right_to_left_paragraph_reads_back_whole() {
let mut p = Paragraph::with_text("שלום ", CharProps::default());
p.props.bidi = Some(true);
p.insert_text(p.len(), "Latin words", &CharProps { bold: Some(true), ..Default::default() }).unwrap();
p.insert_text(p.len(), " עולם", &CharProps::default()).unwrap();
let mut d = Document::new();
d.body = vec![para_block(p)];
let words = viewer_words(&export(&d, &PdfOptions::default()).unwrap());
for w in ["שלום", "Latin", "words", "עולם"] {
assert!(words.iter().any(|x| x == w), "{w:?} not extracted intact from {words:?}");
}
}
/// Glyphs already running left to right keep their order: Latin runs are written as before.
#[test]
fn left_to_right_glyphs_keep_their_order() {
let glyphs = [(1, 0.0, 10.0), (2, 5.0, 10.0), (3, 5.0, 10.0), (4, 9.0, 10.0)];
assert!(left_to_right(&glyphs));
}
/// Right-to-left glyphs (reading order, drawn right to left) are written left to right a
/// cluster at a time. A letter's vowel mark takes its own character and goes before the letter,
/// so a viewer reversing the word reads letter then mark; a lam-alef drawn as two glyphs keeps
/// its glyphs together in shaping order, so no word comes apart.
#[test]
fn right_to_left_clusters_are_written_whole_left_to_right() {
// Reading order: shin with qamats (mark glyph 2 drawn at x 28, then the letter, glyph 1, at
// x 30); lam-alef as two glyphs at x 20 and 16; a one-glyph lam-alef at x 8; final mem at 0.
let text = "שָלالاם";
let glyphs = [(2, 28.0, 8.0), (1, 30.0, 10.0), (3, 20.0, 10.0), (4, 16.0, 10.0), (5, 8.0, 10.0), (6, 0.0, 10.0)];
let ranges = [0..4, 0..4, 4..8, 4..8, 8..12, 12..14];
let rev: std::collections::HashMap<u32, char> = [(1, 'ש'), (2, '\u{5B8}'), (6, 'ם')].into_iter().collect();
assert!(!left_to_right(&glyphs));
let parts = split_clusters(text, &glyphs, &ranges, &rev);
assert_eq!(parts, [2..4, 0..2, 4..8, 4..8, 8..12, 12..14]);
// Glyph by glyph by x this would be [5, 4, 3, 2, 0, 1]: the lam-alef's glyphs swapped.
assert_eq!(visual_order(text, &glyphs, &ranges, &parts), [5, 4, 2, 3, 0, 1]);
}
docs/file-format-parity.md
+3 -2
View File
@@ -1,6 +1,6 @@
# File-format parity with Microsoft Word
> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** trivial (ODT sections on import, #1379; previously trivial: HTML header cells bold, cell `text-align` kept, and HTML/Markdown/text open on the region's paper, #1381; previously trivial: `.doc` table cell shading and the extra column, #1380; previously trivial: `.doc` tables from Word 2000+: cell shading, no extra column, merged rows, #1380; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: equations as MathML in HTML export, copy and paste, #816; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats
> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** trivial (ODT sections on import, #1379; previously trivial: PDF export: Hebrew and Arabic text extracts as whole words, #1383; previously trivial: HTML header cells bold, cell `text-align` kept, and HTML/Markdown/text open on the region's paper, #1381; previously trivial: `.doc` table cell shading and the extra column, #1380; previously trivial: `.doc` tables from Word 2000+: cell shading, no extra column, merged rows, #1380; previously trivial: VML text boxes keep their internal margins, fit to their text and vertical alignment; hidden drawings not shown; previously trivial: equations as MathML in HTML export, copy and paste, #816; previously trivial: symbol-font text exported as Unicode, #776; previously trivial: text-box margins, Resize shape to fit text and freeforms in groups, #743; previously trivial: formatted paste from the system clipboard's HTML and RTF, #687; previously trivial: independent complex-script formatting inheritance and round-trip regression tests; previously trivial: import/export fixes for TXT, RTF, ODT, HTML and LaTeX, #433–#440; previously major: first version; every format Word reads or writes, measured from the readers and writers on origin/main) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4, plus Word for Windows' extra formats
Every format Word opens or saves, what WordCraft does with it, and how it's tested. Fidelity is the
share of a typical real-world file's content and formatting that survives (read: shows correctly;
@@ -22,7 +22,7 @@ Weights are the share of a Word user's file traffic (estimate). Read/write: ✅
| `.docx` Word Document (ECMA-376 Transitional) | 70% | R/W | ✅ | ✅ | ~65% | 70–110 | `crates/docx/tests` (roundtrip 34, fixtures 29, malformed 8, bidi 5, citations 5); synthetic fixtures only | Word opens our files. Untested on a real-world corpus. Details below |
| `.docx` Strict Open XML | 1% | R/W | ✅ (namespace mapping in `xml.rs`) | ❌ | ~60% | 2–4 | one fixture | Writing Strict is rare |
| `.docm` / `.dotx` / `.dotm` | 3% | R/W | ✅ | ✅ | ~65% | incl. above | `macro_packages.rs` | Macros and signatures kept on save (#172); macros never run |
| PDF export | 8% | W | — | ✅ | ~80% | 6–10 | `crates/pdf/src/tests.rs` | krilla: embedded subsets, selectable text incl. field results (#133), links, outline, tagging. Fonts it can't subset become outlines (#132). Print Markup (#589): comments in balloons beside each page (scaled to its paper) and tracked changes; HTML/Markdown/text copies end with a comment list. Horizontal and vertical lines inside groups are drawn (#1474). No PDF/A option, no "best for printing/online" choice |
| PDF export | 8% | W | — | ✅ | ~80% | 6–10 | `crates/pdf/src/tests.rs` | krilla: embedded subsets, selectable text incl. field results (#133); Hebrew and Arabic copy and search as whole words (right-to-left glyphs written in visual order, #1383), links, outline, tagging. Fonts it can't subset become outlines (#132). Print Markup (#589): comments in balloons beside each page (scaled to its paper) and tracked changes; HTML/Markdown/text copies end with a comment list. Horizontal and vertical lines inside groups are drawn (#1474). No PDF/A option, no "best for printing/online" choice |
| `.doc` Word 97-2003 (and `.dot`) | 8% | R/W | 🟡 | ❌ | ~60% | 25–40 | `crates/docbin/src/tests.rs`, `tests/drawing.rs` (56) | Spec-based reader ([MS-DOC]): text, formatting, styles, sections, headers/footers, tables, lists, notes, fields, bookmarks; inline and floating pictures (PNG, JPEG, DIB, TIFF, EMF, WMF; Macintosh PICT as a sized placeholder) with crop, alt text, position and wrap; basic shapes, lines, groups and text boxes ([MS-ODRAW]). Table cell shading from every SHD record (`sprmTDefTableShd`, `sprmTDefTableShdRaw` and their 2nd/3rd ranges, with the Word 97 `Shd80` as fallback; a clear pattern shows its background colour); the row-end mark is not a cell, and each table has its own grid and properties (#1380). Horizontally merged rows stored as fewer, wider cells come with #1123. Freeform shapes, z-order and Word 6/95 or encrypted files are dropped or refused. No writer |
| PDF open (PDF Reflow, Word for Windows) | 1% | R | ❌ | — | 0% | 30–50 | — | Converting a PDF into an editable document. A sibling app (PdfCraft) may supply the parser |
| `.odt` OpenDocument Text | 2% | R/W | 🟡 | 🟡 | ~50% | 8–12 | `crates/formats/src/tests.rs`, `crates/formats/src/odt.rs` unit tests, `crates/engine/tests/odt_sections.rs` | Styles, lists (#87), tables, pictures, headers/footers, page breaks (#86), page size, orientation and margins (#810), sections: one per master-page change on a paragraph or table (next page, with its own page size, orientation, margins, columns and page-numbering restart) and per columned `text:section` (continuous, with its columns and separator line) (#1379). No footnotes, comments, tracked changes, fields or RTL |
@@ -170,6 +170,7 @@ raise readiness estimates. No Word-derived files are fixtures.
| 2026-10-12 | trivial | Direct run-level `w:contentPart` InkML keeps its relationship, content type and bytes through DOCX edits and repeated saves; nested group/canvas ink and native rendering remain missing |
| 2026-10-13 | trivial | HTML export writes equations as MathML instead of linear text, and HTML import (open and paste) reads `<math>` back into equations; a copy holding an equation puts that HTML on the system clipboard: MathML in HTML on the clipboard (Word/LibreOffice paste not yet verified; OMML is #1297) (#816) |
| 2026-10-14 | trivial | ODT import reads sections: a paragraph naming another master page starts a new-page section with that page's size, orientation, margins and columns, and its `style:page-number` restart; a columned `text:section` is a continuous section with its columns (#1379) |
| 2026-10-14 | trivial | PDF export writes right-to-left runs' glyphs in visual order, so Chrome, Edge (PDFium) and poppler extract Hebrew and Arabic as whole words (vowelled words keep their letters in order but may extract with spaces inside), and Ctrl+F finds them (#1383). `/ReversedChars` (for strict file-order readers) covers plain runs and vowelled runs whose marks split into their own glyphs, not runs with ligatures or mixed directions |
| 2026-10-14 | trivial | HTML: `<th>` bold, cell `text-align` (own or inherited) reaches the cell's paragraphs; HTML, Markdown and plain text open on the region's paper, in the app and `wordcraft-cli` (#1381) |
| 2026-10-13 | trivial | PDF export draws horizontal and vertical lines inside groups (#1474) |
| 2026-10-14 | trivial | RTF import keeps sections: page size and orientation, margins, columns, break type, page numbering and each section's headers and footers (#1378) |
docs/gaps.md
+3 -2
View File
@@ -1,6 +1,6 @@
# Where WordCraft falls short of Word
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** trivial (#19 narrowed: ODT sections on import, #1379; previously trivial: Word 365 cover page positions and closed freeforms, #743; previously trivial: RTF import keeps sections, #1378; previously trivial: sample-text formulas on Enter, #1067; previously trivial: #29: copied equations as MathML, OMML open as #1297, #816; previously trivial: the Font list no longer keeps every previewed font in memory, #848; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: beta distance restated: ~15 points, ~150–230 h; previously: first version; every known shortfall from the 2026-10-10 re-measure, ranked) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** trivial (#19 narrowed: ODT sections on import, #1379; previously trivial: #16: PDF export's right-to-left text extracts as whole words, #1383; previously trivial: Word 365 cover page positions and closed freeforms, #743; previously trivial: RTF import keeps sections, #1378; previously trivial: sample-text formulas on Enter, #1067; previously trivial: #29: copied equations as MathML, OMML open as #1297, #816; previously trivial: the Font list no longer keeps every previewed font in memory, #848; previously trivial: table column and row markers on the rulers, #700; previously trivial: table borders and the resize handle drag on the page, #217; previously trivial: formatting revisions landed, #41; previously trivial: Draw Table and Eraser landed, #303; previously minor: beta distance restated: ~15 points, ~150–230 h; previously: first version; every known shortfall from the 2026-10-10 re-measure, ranked) · **Target:** Microsoft Word (Microsoft 365) for Mac 16.113.4
> **Printing update:** 2026-10-12 · direct desktop printing through craft-print; details below.
@@ -30,7 +30,7 @@ Native save reliability: synthetic cases on macOS reproduced loss through the pr
| 12 | **Draw tab / ink**: pens, pencil, highlighter, eraser, Select and Hide Ink landed (#307); Lasso Select, Add Pen, Ink to Shape and Ink Replay landed (#395); missing: Ink to Math, arrows in Ink to Shape, pressure, rendering/editing Word's own ink (drawing-level and direct run-level InkML content parts now survive DOCX save) | | [parity-checklist.md](parity-checklist.md) (Draw 9/11) | Pen and tablet users | 4–8 | [hardware-parity.md](hardware-parity.md) |
| 14 | **`.doc` can't be written; Word 6/95 files refused** | | `crates/docbin` is read-only | Users exchanging files with old Word or legacy systems | 25–40 | [file-format-parity.md](file-format-parity.md) |
| 15 | **References depth**: sources not saved in DOCX (#169 open); no EndNote/Mendeley desktop integration | | Sources live only in the session (#169 open) | Students and researchers | 6–10 | [target-app-parity.md](target-app-parity.md) |
| 16 | **Right-to-left completeness**: kashida, RTL in HTML/ODT/RTF; eight older RTL bug reports to re-verify after #207 (#215, #211, #199, #66, #63, #48, #19). RTL tables and sections landed (#362) | | [layout-parity.md](layout-parity.md) | Arabic, Persian, Hebrew writers | 7–11 | [layout-parity.md](layout-parity.md) |
| 16 | **Right-to-left completeness**: kashida, RTL in HTML/ODT/RTF; eight older RTL bug reports to re-verify after #207 (#215, #211, #199, #66, #63, #48, #19). RTL tables and sections landed (#362); PDF export's Hebrew and Arabic text copies and searches as whole words (#1383) | | [layout-parity.md](layout-parity.md) | Arabic, Persian, Hebrew writers | 7–11 | [layout-parity.md](layout-parity.md) |
| 17 | **East Asian typography**: Asian Typography options (Phonetic Guide landed, #288; Enclose Characters landed, #297; document grid landed, #391; vertical text landed, #491) | | Catalog misses `para.asianTypography` | Chinese, Japanese and Korean documents | 11–19 | [layout-parity.md](layout-parity.md) |
| 18 | **Screen-reader access** to the document canvas: exposed, not yet tried with real screen readers | | The canvas describes paragraphs, headings, lists, tables, links, pictures and the caret to AccessKit (#488), tested headlessly; no NVDA, VoiceOver or Orca run yet | Blind and low-vision users; public-sector procurement | 3–6 | [ui-parity.md](ui-parity.md) |
| 19 | **RTF and ODT depth**: notes, comments, revisions, sections (RTF import done, #1378; RTF export and ODT left), RTL. ODT sections landed on import (#1379): page-style changes and columned `text:section`s | | ~1,500 lines each in `crates/formats` | Users exchanging with LibreOffice or older tools | 18–27 | [file-format-parity.md](file-format-parity.md) |
@@ -85,6 +85,7 @@ The same gaps grouped the way the parity documents are, for agents working in on
| 2026-10-12 | minor | Italian proofing landed (it-IT spelling from the installed Hunspell dictionary, accent-first suggestions, conservative grammar); gap #8 narrowed to the remaining languages |
| 2026-10-12 | trivial | #2 narrowed: direct run-level `w:contentPart` InkML survives DOCX edits and repeated saves; nested group/canvas ink and native rendering remain |
| 2026-10-14 | trivial | #19 narrowed: ODT import keeps sections (master-page changes, columned `text:section`s, page-numbering restarts) (#1379) |
| 2026-10-14 | trivial | #16: PDF export writes right-to-left glyphs in visual order, so Hebrew and Arabic extract, copy and search as whole words (#1383) |
| 2026-10-14 | trivial | RTF import keeps sections (#1378): gap 19 narrowed to RTF section export and ODT sections |
| 2026-10-11 | trivial | Table column and row markers on the rulers landed (#700): the ruler's column markers are off the UI/UX gaps |
| 2026-10-11 | trivial | UI/UX gaps: the table context menu landed, its items following the selection as in Word (Delete Cells, Rows, Columns or Table, Merge and Split Cells, Distribute, AutoFit, Text Direction, Insert Caption…, Table Properties…); objects' menus and Border Styles remain |