Type: a click in a Devanagari syllable goes to its nearer end, and shaping is tested

A syllable drawn with several glyphs that share one cluster, such as a
pre-base i matra and its consonant (कि), had a caret stop at the edge
of each glyph. A click just right of the matra, in the left half of
the syllable, put the caret after the syllable. byte_in_line now takes
one extent for each cluster, and type on a path compares the click
with the middle of the cluster, from the start of its first glyph to
the end of its last.

New tests with an installed OpenType Devanagari font check that क्ष
forms one conjunct glyph, that र्क draws the reph after its base, that
कि draws the matra before its consonant, that Ka with a nukta matches
the precomposed क़, and that each syllable is one cluster with the
arrow keys stopping at its ends. They print SKIPPED when no such font
is installed.

Docs: docs/localization-parity.md (Hindi row), docs/target-app-parity.md.

crates/text/src/lib.rs

@@ -304,16 +304,20 @@ pub fn caret_position(layout: &TextLayout, byte: usize) -> (Point, Point) {
 /// Byte offset nearest to the text-space point `p` (for clicking/dragging with the Type tool).
 pub fn hit_byte(layout: &TextLayout, p: Point) -> usize {
     if layout.on_path {
-        let best = layout.glyphs.iter().min_by(|a, b| {
+        let best = layout.glyphs.iter().enumerate().min_by(|(_, a), (_, b)| {
             let ca = a.origin + dir(a.angle) * (a.advance * 0.5);
             let cb = b.origin + dir(b.angle) * (b.advance * 0.5);
             (p - ca).hypot2().total_cmp(&(p - cb).hypot2())
         });
-        return match best {
-            Some(g) if ((p - g.origin).dot(dir(g.angle)) < g.advance * 0.5) != g.rtl => g.byte,
-            Some(g) => g.byte + g.len,
-            None => layout.lines.first().map_or(0, |l| l.start),
-        };
+        let Some((i, g)) = best else { return layout.lines.first().map_or(0, |l| l.start) };
+        // `first` and `last` are the first and last glyphs of the cluster. A click before the middle of
+        // the cluster, measured along the first glyph's direction, goes to the cluster's start.
+        let same = |o: &&PositionedGlyph| o.byte == g.byte && o.len == g.len;
+        let first = layout.glyphs[..=i].iter().rev().take_while(same).last().unwrap_or(g);
+        let last = layout.glyphs[i..].iter().take_while(same).last().unwrap_or(g);
+        let along = dir(first.angle);
+        let span = (last.origin + dir(last.angle) * last.advance - first.origin).dot(along);
+        return if ((p - first.origin).dot(along) < span * 0.5) != g.rtl { g.byte } else { g.byte + g.len };
     }
     let p = layout.logical_point(p);
     // Nearest line band (vertical distance first, then distance to the line's column).
@@ -349,13 +353,20 @@ pub fn hit_byte(layout: &TextLayout, p: Point) -> usize {
 pub fn byte_in_line(layout: &TextLayout, li: usize, x: f64) -> usize {
     let Some(line) = layout.lines.get(li) else { return 0 };
     let glyphs = &layout.glyphs[line.glyph_start..line.glyph_end];
-    let nearest = glyphs
+    // A cluster drawn with several adjacent glyphs (a Devanagari pre-base matra and its consonant,
+    // a consonant and its reph) has one caret stop at each end of its glyphs taken together.
+    let mut clusters: Vec<(f64, f64, &PositionedGlyph)> = Vec::with_capacity(glyphs.len());
+    for g in glyphs {
+        let left = layout.logical_point(g.origin).x;
+        let right = left + g.advance;
+        match clusters.last_mut() {
+            Some((l, r, c)) if c.byte == g.byte && c.len == g.len => (*l, *r) = (l.min(left), r.max(right)),
+            _ => clusters.push((left, right, g)),
+        }
+    }
+    let nearest = clusters
         .iter()
-        .flat_map(|g| {
-            let left = layout.logical_point(g.origin).x;
-            let right = left + g.advance;
-            if g.rtl { [(right, g.byte), (left, g.byte + g.len)] } else { [(left, g.byte), (right, g.byte + g.len)] }
-        })
+        .flat_map(|&(left, right, g)| if g.rtl { [(right, g.byte), (left, g.byte + g.len)] } else { [(left, g.byte), (right, g.byte + g.len)] })
         .min_by(|a, b| (a.0 - x).abs().total_cmp(&(b.0 - x).abs()));
     nearest.map_or(line.start, |(_, byte)| byte)
 }
@@ -518,6 +529,8 @@ mod tests_bidi;
 #[cfg(test)]
 mod tests_combos;
 #[cfg(test)]
+mod tests_devanagari;
+#[cfg(test)]
 mod tests_embed;
 #[cfg(test)]
 mod tests_fit;

crates/text/src/tests_devanagari.rs

@@ -0,0 +1,158 @@
+//! Devanagari shaping: conjuncts, reph, pre-base matras and nukta forms, with their source clusters
+//! and caret stops. The expected forms come from Microsoft's OpenType Devanagari shaping specification
+//! (https://learn.microsoft.com/en-us/typography/script-development/devanagari) and the Unicode
+//! Standard's Devanagari section (https://www.unicode.org/versions/latest/core-spec/chapter-12/).
+//! No font is committed: the tests use an installed face and print SKIPPED when none is found.
+use super::*;
+use kurbo::{BezPath, Point};
+use std::sync::Arc;
+use vectorcraft_doc::{CharStyle, TextKind};
+use vectorcraft_geom::PathData;
+
+fn devanagari_test_face(test: &str) -> Option<Arc<FontFace>> {
+    let db = FontDb::global();
+    // The face must have OpenType (GSUB) Devanagari lookups. An AAT face such as macOS's Devanagari MT
+    // shapes with `morx`, and हिन्दी comes out as three clusters because the conjunct न्दी is split in two.
+    let opentype = |face: &FontFace| {
+        use skrifa::raw::TableProvider;
+        face.skrifa()
+            .and_then(|f| f.gsub().ok())
+            .and_then(|gsub| gsub.script_list().ok())
+            .is_some_and(|list| list.script_records().iter().any(|r| matches!(&r.script_tag().to_be_bytes(), b"dev2" | b"deva")))
+    };
+    let face = ["Kohinoor Devanagari", "ITF Devanagari", "Devanagari Sangam MN", "Noto Sans Devanagari", "Nirmala UI", "Mangal"]
+        .into_iter()
+        .filter_map(|name| db.face(name, "Regular").filter(|face| face.family == name))
+        .find(|face| {
+            ['क', 'ष', 'र', 'ह', 'न', 'द', '\u{094D}', '\u{093F}', '\u{0940}', '\u{093C}'].iter().all(|&c| face.covers(c)) && opentype(face)
+        });
+    if let Some(face) = &face {
+        let path = face.path().map_or_else(|| "bundled font".to_string(), |p| p.display().to_string());
+        eprintln!("ASSERTIONS RUN: {test}; Devanagari font {} {} ({path})", face.family, face.style);
+    } else {
+        eprintln!("SKIPPED: {test}; no installed candidate font shapes Devanagari through GSUB");
+    }
+    face
+}
+
+fn devanagari_layout(face: &FontFace, text: &str) -> TextLayout {
+    let st = CharStyle { font_family: face.family.clone(), font_style: face.style.clone(), size: 20.0, ..CharStyle::default() };
+    layout(FontDb::global(), &TextObject::point(Point::ZERO, text, st))
+}
+
+fn source_clusters(layout: &TextLayout) -> Vec<(usize, usize)> {
+    let mut clusters: Vec<_> = layout.glyphs.iter().map(|glyph| (glyph.byte, glyph.len)).collect();
+    clusters.dedup();
+    clusters
+}
+
+/// The left and right edges of the glyphs of the cluster that starts at `byte`.
+fn cluster_extent(layout: &TextLayout, byte: usize) -> (f64, f64) {
+    let glyphs = layout.glyphs.iter().filter(|g| g.byte == byte);
+    glyphs.fold((f64::INFINITY, f64::NEG_INFINITY), |(left, right), g| (left.min(g.origin.x), right.max(g.origin.x + g.advance)))
+}
+
+#[test]
+fn ka_virama_ssa_forms_one_conjunct_glyph() {
+    let Some(face) = devanagari_test_face("ka_virama_ssa_forms_one_conjunct_glyph") else { return };
+    // `akhn` substitutes Ka + halant + Ssa with the KaSsa ligature.
+    let l = devanagari_layout(&face, "क्ष");
+    assert_eq!(l.glyphs.len(), 1, "{:?}", l.glyphs);
+    let g = &l.glyphs[0];
+    assert_eq!((g.byte, g.len), (0, "क्ष".len()));
+    assert!(!['क', 'ष', '\u{094D}'].iter().any(|&c| face.glyph_for(c) == g.gid), "conjunct drawn with a nominal glyph: {}", g.gid);
+}
+
+#[test]
+fn reph_is_drawn_after_its_base_consonant() {
+    let Some(face) = devanagari_test_face("reph_is_drawn_after_its_base_consonant") else { return };
+    // Ra + halant before the base consonant becomes the reph (`rphf`), reordered after the base.
+    let s = "र्क";
+    let l = devanagari_layout(&face, s);
+    let gids: Vec<u32> = l.glyphs.iter().map(|g| g.gid).collect();
+    assert_eq!(gids.len(), 2, "{:?}", l.glyphs);
+    assert_eq!(gids[0], face.glyph_for('क'), "the base consonant comes first: {gids:?}");
+    assert!(![face.glyph_for('र'), face.glyph_for('\u{094D}'), face.glyph_for('क')].contains(&gids[1]), "no reph glyph: {gids:?}");
+    assert!(l.glyphs[1].origin.x >= l.glyphs[0].origin.x);
+    assert_eq!(source_clusters(&l), vec![(0, s.len())]);
+}
+
+#[test]
+fn pre_base_matra_is_drawn_before_its_consonant() {
+    let Some(face) = devanagari_test_face("pre_base_matra_is_drawn_before_its_consonant") else { return };
+    let s = "कि";
+    let l = devanagari_layout(&face, s);
+    assert_eq!(l.glyphs.len(), 2, "{:?}", l.glyphs);
+    let (matra, ka) = (&l.glyphs[0], &l.glyphs[1]);
+    assert_eq!(ka.gid, face.glyph_for('क'));
+    assert_ne!(matra.gid, ka.gid);
+    assert!(matra.origin.x < ka.origin.x, "the i matra is drawn to the left: {:?}", l.glyphs);
+    assert_eq!(source_clusters(&l), vec![(0, s.len())]);
+    // The caret stops at the syllable's ends: before the matra, and after the consonant.
+    let (left, right) = cluster_extent(&l, 0);
+    assert!((caret_position(&l, 0).0.x - left).abs() < 1e-9);
+    assert!((caret_position(&l, s.len()).0.x - right).abs() < 1e-9);
+}
+
+#[test]
+fn syllables_map_to_their_source_bytes_and_caret_stops() {
+    let Some(face) = devanagari_test_face("syllables_map_to_their_source_bytes_and_caret_stops") else { return };
+    // हि + न्दी: the second syllable's matra stays with the conjunct it follows.
+    let s = "हिन्दी";
+    let l = devanagari_layout(&face, s);
+    let second = "हि".len();
+    assert_eq!(source_clusters(&l), vec![(0, second), (second, s.len() - second)]);
+    // The first syllable's i matra is drawn before its consonant.
+    assert_eq!(l.glyphs[1].gid, face.glyph_for('ह'), "{:?}", l.glyphs);
+    assert!(l.glyphs[0].origin.x < l.glyphs[1].origin.x, "{:?}", l.glyphs);
+    // Left and Right arrows pass over the same syllables the shaper made.
+    assert_eq!(edit::next_cluster(s, 0), second);
+    assert_eq!(edit::next_cluster(s, second), s.len());
+    assert_eq!(edit::prev_cluster(s, s.len()), second);
+    assert_eq!(caret_horizontal(&l, 0, true), second);
+    assert_eq!(caret_horizontal(&l, second, true), s.len());
+    assert_eq!(caret_horizontal(&l, s.len(), false), second);
+    assert!((caret_position(&l, second).0.x - cluster_extent(&l, 0).1).abs() < 1e-9);
+}
+
+#[test]
+fn nukta_forms_match_the_precomposed_letters() {
+    let Some(face) = devanagari_test_face("nukta_forms_match_the_precomposed_letters") else { return };
+    // U+0958 is canonically equivalent to Ka + nukta and is drawn the same way.
+    let composed = devanagari_layout(&face, "\u{0958}");
+    let decomposed = devanagari_layout(&face, "\u{0915}\u{093C}");
+    let shape = |l: &TextLayout| l.glyphs.iter().map(|g| (g.gid, g.advance, g.origin)).collect::<Vec<_>>();
+    assert_eq!(shape(&composed), shape(&decomposed));
+    assert_eq!(source_clusters(&decomposed), vec![(0, "\u{0915}\u{093C}".len())]);
+    // The nukta joins its consonant: one glyph takes up room on the line.
+    assert_eq!(decomposed.glyphs.iter().filter(|g| g.advance > 0.0).count(), 1, "{:?}", decomposed.glyphs);
+}
+
+#[test]
+fn a_click_in_a_syllable_goes_to_its_nearer_end() {
+    let Some(face) = devanagari_test_face("a_click_in_a_syllable_goes_to_its_nearer_end") else { return };
+    let s = "कि";
+    let l = devanagari_layout(&face, s);
+    let (left, right) = cluster_extent(&l, 0);
+    // This point lies just past the matra, still in the left half of the syllable.
+    let matra_end = l.glyphs[0].origin.x + l.glyphs[0].advance;
+    let near_left = matra_end + 0.1;
+    assert!(near_left < (left + right) * 0.5, "the matra takes most of the syllable: {:?}", l.glyphs);
+    let y = l.lines[0].baseline;
+    assert_eq!(hit_byte(&l, Point::new(near_left, y)), 0);
+    assert_eq!(hit_byte(&l, Point::new(right - 0.1, y)), s.len());
+
+    // Clicks at the same places on a path go to the same ends.
+    let mut line = BezPath::new();
+    line.move_to((0.0, 50.0));
+    line.line_to((500.0, 50.0));
+    let st = CharStyle { font_family: face.family.clone(), font_style: face.style.clone(), size: 20.0, ..CharStyle::default() };
+    let mut t = TextObject::point(Point::ZERO, s, st);
+    t.kind = TextKind::OnPath { path: PathData::from_bezpath(&line), start: 0.0, end: None };
+    let p = layout(FontDb::global(), &t);
+    assert!(p.on_path && p.glyphs.len() == 2, "{:?}", p.glyphs);
+    let x0 = p.glyphs[0].origin.x;
+    let matra_end = x0 + p.glyphs[0].advance;
+    assert_eq!(hit_byte(&p, Point::new(matra_end + 0.1, 50.0)), 0);
+    assert_eq!(hit_byte(&p, Point::new(p.glyphs[1].origin.x + p.glyphs[1].advance - 0.1, 50.0)), s.len());
+}

docs/localization-parity.md

@@ -26,7 +26,7 @@ Chinese by `imachus`). None is recorded as reviewed by a native speaker; the cat
 | English | `en` | source, 100% | all | Latin | — | full | — |
 | Simplified Chinese (Mandarin) | `zh-hans` | 3,647 rows; menus, panels, dialogs complete (tests); 85% overall | messages in English (1 of 604) | CJK UI font from craft-fonts (BIZ UDPGothic) when embedded, else a system font; IME with marked text at the caret; vertical type partial ([type-parity.md](type-parity.md)) | no | partial | 2–3 |
 | Spanish | `es` | 4,009 / 4,020 (99.7%) | all (604 messages) | Latin | no | full | — (review) |
-| Hindi | `hi` | 0 | none | Devanagari: the document's text shapes (HarfRust), untested for Devanagari, but the egui interface does no complex shaping | no | none | 20–30 |
+| Hindi | `hi` | 0 | none | Devanagari: the document's text shapes (HarfRust), with tests of conjuncts, reph, pre-base matras, nukta forms and caret stops that run when an OpenType Devanagari font is installed, but the egui interface does no complex shaping | no | none | 20–30 |
 | Arabic | `ar` | 0 | none | document text: bidi layout and Paragraph Direction; interface: no right-to-left layout, no Arabic shaping in egui | no | none | 25–40 |
 | French | `fr` | 4,020 / 4,020 (100%) | all | Latin | no | full | — (review) |
 | Portuguese | `pt-br` | 4,707 catalog entries: menus, `tl!` literals and messages complete (tests); one|other plurals | all | Latin | contributor is a Brazilian Portuguese speaker; further native review welcome | full | — (review) |