3 people 517a4b9256 Sort: order Arabic text by its letters, not its vowel marks or digit script (#1940) ...

* M12.AR7: sort Arabic text by its letters, not its vowel marks or digit script

* M12.AR7: order marked Arabic text the way Excel does

Checked compare_text against Excel 16.0 build 20527 (en-US) via COM:
Range.Sort, =SORT() and the < / = operators over ~300 strings. Fixes the
mismatches inside this change's scope:

- The tie-break compared raw code points, so an RLM-prefixed word sorted
  after every vowelled form of it, "‏12" after "١٢", and a tatweel
  form after a shadda form. Ignorable characters are now skipped at the
  tie-break too.
- Marks are compared letter by letter (an unmarked letter first), so a
  mark on an earlier letter outranks the digit or letter after it, as in
  Excel ("ب١" < "بَ1", "بپ" < "بَپ").
- Marks rank in Excel's measured order (sukun before shadda), and the
  Quranic marks Excel skips (U+0610–061A, U+06D8–06ED except 06DD/06DE/
  06DF/06E0/06E5/06E6/06E9) are skipped by letter order.
- Excel also ignores bidi embeddings/overrides/isolates, BOM, soft hyphen,
  CGJ and word joiner; they now are ignored too (ZWSP is not, as in Excel).

Equality is unchanged. Regression tests use the orders Excel produced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* M12.AR7: compare the mark level of Arabic text without allocating

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>

2026-11-09 15:49:26 +01:00

..

src

Sort: order Arabic text by its letters, not its vowel marks or digit script (#1940)

2026-11-09 15:49:26 +01:00

Cargo.toml

Use chrono instead of hand-rolled UTC conversion (#2121)

2026-10-23 11:48:02 -04:00