Fall back only when missing sample or terminator bytes cannot fit the
content-window budget, including retained header and separator overhead.
Preserve available deterministic boundaries and ordinary large streams.
Add oversized, exact-budget, overflow and large-stream regressions.
* M2: a JP2 image in a colour space the decoder refuses is drawn by its component count (#3306)
The veraPDF fixture's JPX header names CIEJab (enumerated colour space 19),
which hayro-jpeg2000 refuses, so the image failed and the page drew blank.
When the decoder refuses the image and its colr boxes name such a space, it is
decoded again from a copy naming greyscale, sRGB or CMYK by component count,
as MuPDF and Poppler draw it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: copy the JP2 stream only when a colour enum needs replacing (#3306)
Review follow-up on #3363: a stream the decoder refused for another reason was
copied for nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* Renumber the JP2 colour-space patch (21): #3378 takes hayro-syntax (20)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* fix(parse): accept spaces between stream and its line break
Microsoft Reporting Services writes `stream \r\n`, a space before the line break.
hayro-syntax and pdfcraft-cos took the stream data to start at the space, so the
content stream failed to inflate: the page rendered blank (white, or black with
inverted colours) and an edited save corrupted the stream. Spaces and tabs are now
skipped, but only when a line break follows.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Number the stream end-of-line patch (20), as vendor/README does
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M2: a tiling pattern drawn once shows its cell at full scale (#3349)
PowerPoint writes a picture fill as a tiling pattern with /XStep 2147480100.
Fitting that step into the 3000 px cell pixmap shrank the picture below a
pixel. When what is painted can only show the first cell, the cell's /BBox is
drawn at full scale with a transparent border and padded instead of tiled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a tiling cell whose scale isn't finite keeps the old path (#3349)
Review follow-up on #3356: a /BBox 1e-40 wide passes the zero-area check and
gives an infinite cell scale, which would send non-finite geometry to the cell
renderer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a stroke's miter counts in the one-cell check (#3349)
Review follow-up on #3356: shows_one_cell took the line width as a stroke's
reach; a miter join reaches half the width times the miter limit. The reach
formula patch (9) uses is now one function, stroke_reach, used by both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
* M2: a skewed stroke's reach uses the transform's largest stretch (#3349)
Review on #3356: under a skew, a direction between the axes stretches
further than either axis, so the larger axis stretch could place a miter
tip short of the next cell when it reached it, and the one-cell path left
that cell unpainted. The reach now uses the spectral norm.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Merges left four hayro-interpret rows; the current one lacked the
transparency-group, Type 3 rendering-mode and Lab white-point patches (each
numbered 25, 27 or 28 on a branch where another patch took that number).
They are now (41), (42) and (43) in the one row, with their tests.
Claude-Session: https://claude.ai/code/session_01Gv2WwdgjmBz33GWkwgNn3Q
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Seven tests failed at random on slow or busy machines, main included
(FreeBSD run 38834256439 at 5fdd820b). They now count the work or wait
for the state they check; no assertion is loosened and nothing retries.
- spot_colour_images_run_their_tint_transform_once_per_sample_value
counts tint-transform runs (exactly 256, not one per pixel) through
vendored hayro-interpret patch (34), color::tint_transform_runs, a
thread-local counter, instead of a 10 s deadline.
- shadings_with_large_sampled_functions_paint_quickly: no counter tells
the flat table from the hash map (both read every sample per build),
so the build each paint does is timed against reading its samples on
the same thread, fastest of 8 rounds each, and must stay under 10x
(flat table 1.1x, hash map 109x), instead of a 10 s deadline.
- page_contents_and_nested_forms_share_one_budget: the budget is
checked by its warnings; the deadline is only a hang guard, now 600 s.
- The pen-stroke tests hold finished renders back from the view
(DocView::hold_rasters, for tests) while they look at the settling
stroke, then wait by state for its raster.
- files_and_quit_from_the_operating_system waits for the tab before
deleting the file the open worker reads.
- image_pixels_follow_the_frame_while_a_corner_is_dragged waits for the
page raster on screen before each drag.
Product fix (#1858): a released drawing stayed painted only until any
raster of its page arrived. In full screen its edit is applied a frame
later, so a raster already under way could end the preview first. A
settling drawing is now marked committed when its AddAnnotation lands,
only rasters after that end its preview, and a failed edit drops it.
* fix(render): sampled functions index a flat sample array (vendored hayro-interpret patch 31)
A Type 0 sampled function kept its samples in a HashMap keyed by each point of
its /Size grid (two SmallVecs per key, SipHash), and the renderer builds a
shading's functions again every time it is painted, so every paint hashed one
insert per sample. A 2,097,152-sample ramp painted 30 times took 38 s; the
pdf.js corpus's bug1721218 (InDesign) spent most of a 1.5 s page on it.
The samples now stay in stream order in one Vec, and a grid point is found by
strides (first input fastest). The interpolation keeps the same recursion and
arithmetic, so results are bit for bit the same, and a point the stream ended
before still gives no value. The repeated-paint page draws in well under a
second. The maximum sample value no longer computes 2^32 in a u32 (a panic with
overflow checks on at 32 bits per sample).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MoEx9Nj8KC4WKDcf5YaokQ
* fix(render): one decode of an image serves every size it is drawn at (vendored hayro-interpret patch 32)
The decoded-image cache (patch 24) keyed each decode by the size the image is
drawn at, but only the JPX filter decodes a reduced resolution for a size:
every other filter decodes the whole image. So a JPEG or Flate scan missed the
cache at every zoom step, was decoded again, and was kept once per zoom level.
The key now carries the target size only when JPX decodes the image, its
/SMask or its /Mask (read from /Filter or /F, erring towards keeping the
target). `InterpreterCache::decoded_image_decodes` counts the cache's decodes;
the new test draws a Flate image at five sizes and sees one decode, and a
copy whose soft mask is JPX-encoded still decodes per size. Both assertions
fail with the old keying.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MoEx9Nj8KC4WKDcf5YaokQ
* fix(render): patch meshes copy their vertices' colours less (vendored hayro-interpret patch 33)
A Coons or tensor-product patch is drawn as 722 triangles over a 20 x 20 grid
of points, and every vertex's colours (a SmallVec) were produced or copied up
to five times: each grid cell interpolated its four corners itself (1,444
interpolations a patch for 400 points), cloned all six vertices of its two
triangles, and the mesh sampler cloned each vertex again to move the triangle
into device space. Profiling page 8 of the annual report in #1584 put a
quarter of its time in those clones.
Each grid point's colours are now interpolated once, a cell's second triangle
takes the vertices its first copied, and the sampler moves a patch's freshly
made triangles instead of copying them (`TriangleSampler::sample_owned`; a
free-form triangle mesh, which the shading keeps, is still copied). The same
inputs give the same values: rendering the first three pages of every pdf.js
corpus file is byte for byte the same. Release build, Linux: report page 8
773 -> 696 ms, personwithdog.pdf 149 -> 129 ms, issue13520.pdf 104 -> 87 ms.
The new test pins the colours near each corner, at the centre and between them
for a Coons and a tensor-product patch (taken before the change); swapping two
vertices' grid points makes it fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MoEx9Nj8KC4WKDcf5YaokQ
* docs: roadmap entries for faster sampled functions, image decodes and patch meshes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MoEx9Nj8KC4WKDcf5YaokQ
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
#2305 fixed WinAnsi 0xAD in `crates/fonts`, and closed with two loose ends it named: the
user-visible extraction path goes through hayro, not that table, so it still reported the
invisible character; and the same Annex D.2 note makes 0xA0 the `space` glyph. This finishes
both, and makes the two tables in the product agree about the same byte.
WinAnsiEncoding gives codes 0xA0 and 0xAD the glyph names `space` and `hyphen` -- the names it
also gives 0x20 and 0x2D -- and MacRomanEncoding gives 0xCA `space`. The tables called them
`nbspace` and `sfthyphen`, whose Adobe Glyph List values U+00A0 and U+00AD are invisible
formatting characters, so extraction hid ink the reader can see: "well-known" drawn with the
0xAD byte matched no search and no redaction pattern, and a non-breaking space did not match the
space a reader typed.
Evidence. Two sources transcribe the table the same way (an Annex D implementation's docs, and
the WinAnsi.enc encoding vector R ships, which gives `/space` at octal 240 and `/hyphen` at
255), and a black-box check settles the rest: on a synthetic fixture pdftotext reads the
base-encoded codes as `a-b a b`, and keeps U+00AD for a code an explicit `/Differences` names
`sfthyphen`. We now do both. Over the 989-file pdf.js corpus the soft hyphens we extract fall
from 117 to 13 and the non-breaking spaces from 30 to 26; everything left comes from `/ToUnicode`
maps, which say what they mean and are not touched.
Scope. The base table only, in the text direction only.
- A `/Differences` entry naming `sfthyphen` or `nbspace` is a deliberate statement about one
code and keeps its glyph list value, as pdftotext does.
- Glyph lookup is untouched, so a font program's own `sfthyphen` program is still found first.
In hayro that is why the alias sits in the Unicode path rather than in the generated encoding
table: `code_to_ps_name` is shared with lookup.
- Writing is unaffected: `encode` takes the lowest code showing a character, so ' ' and '-'
still write 0x20 and 0x2D, and a caller holding a literal U+00A0 still writes 0xA0.
Tests. `fonts` pins both tables and the whole decode/encode path including the `/Differences`
case; `render` pins the extraction a reader actually gets, over WinAnsi, MacRoman and
`/Differences`. Each was watched failing with each cell and each match arm reverted in turn, so
no assertion is passing by accident. fonts 70, render 296+4, redact 49, a11y 43 and edit 156 are
green, with fmt, clippy, `xtask layers` (37 crates) and `xtask assets` clean.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* M3: release Ctrl zoom for pixel-only Wayland wheel events
Use the reported scroll source to avoid unterminated finger gestures for wheel, wheel-tilt and continuous input. Exercise the production policy through egui, including modifier release and preserved finger momentum.
Assisted-by: OpenAI Codex
* M3: keep Cargo lockfile dependency ordering canonical
Record the dependency ordering produced by the completed workspace and wasm gates.
Assisted-by: OpenAI Codex
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(render): keep annotations readable under reading colours (#1548)
Reading colours remapped annotation paints as line art: the yellow
highlight fill is luminance-light, so it was mapped toward the dark
background, and Multiply then darkened the remapped-white words under
it into invisibility.
Two changes in hayro:
- The /Annots loop in interpret_page is bracketed by a new
begin_annotations/end_annotations Device signal (patch 26), and the
renderer's reading-colours mapping leaves annotation paints alone.
Reading colours replace document colours, never the colours a
comment was given — the Acrobat model.
- The page background is painted inside the page transparency group
(patch 27), as the initial backdrop, so a Multiply highlight blends
with the background too. With the background outside the group the
band kept its own flat colour and the words vanished into it. White
pages are unaffected: multiplying with white is the identity.
Regression test reading_colors_keep_highlight_annotations_readable
covers both annotation paths (/AP and the no-/AP /QuadPoints
fallback) under WHITE_ON_BLACK; parity entry a11y.replace-page-colors
updated with the behaviour and evidence.
Co-Authored-By: GLM <noreply@z.ai>
* contributors: credit @edurbs as Eduardo Soares
The contributor's own entry, chosen by them (real and display name).
Co-Authored-By: GLM <noreply@z.ai>
* #1557: record the vendored hayro patches in vendor/README.md
Main already uses hayro-interpret patches (26)-(28), so the annotation hooks become
hayro-interpret (29) and the renderer's parts hayro (17) and (18).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: GLM <noreply@z.ai>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
On macOS the tab strip is the title bar: a transparent title bar over a
full-size content view. AppKit asks the view under the pointer whether a
press there can move the window (`mouseDownCanMoveWindow`), and NSView
answers YES for a view that isn't opaque. winit's view didn't override it,
so AppKit moved the whole window for any drag that began in the 28 pt band
before egui saw it, and the tab reordering from #1551 never got the drag.
vendor/winit now answers with the window's `movable_by_window_background`
attribute (NO unless asked). The empty strip still moves the window through
`ViewportCommand::StartDrag`, and double-clicking it still maximizes. Patch
(12) in vendor/README.md, kept in the build by
`tests/winit_macos_title_bar_drag.rs`.
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Highlight appearances written by other editors are a transparency group whose
content is filled with `/BM /Multiply`. Every `/Group` form got a layer of its
own, so the Multiply blended with that layer's transparent backdrop and the
layer then covered the page as an opaque block.
Vendored hayro-interpret patch (28): a group that is neither isolated nor
knockout composites its elements with the backdrop (ISO 32000-2 §11.4.7), so
when it is entered at full opacity with no soft mask and Normal blending there
is nothing for a layer to add, and its content is drawn straight onto the page.
`/I` and `/K` are read from the group dictionary. An isolated group keeps its
blank backdrop; a knockout group or one entered at partial opacity, under a
soft mask or with a blend mode keeps its layer.
Test: `raster::tests::a_multiply_highlight_inside_a_transparency_group_keeps_the_text`
(the appearance drawn as an annotation and from page content, with `/I false
/K false` spelled out and with no group at all; an isolated group as a control;
a group entered at half opacity still composited as one). It fails before the
fix with the text pixel yellow instead of black. parity: view.render-transparency
cites it and says what is still approximated.
Claude-Session: https://claude.ai/code/session_01RUNQSPVXx3dDBabRZHAnjz
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* M2: clip intersection starts its row walk at the first shared row
clip::intersect in vello_common, the step every clipped path passes
through, began its walk over strip rows at the earlier of the two paths'
first rows. Each path therefore visited every row above it, although rows
where one path has no strips produce no output. Every page is clipped (the
crop box is a clip), so this ran for every stroke of every render: on the
A1 engineering sample it was the hottest function in a full-page render.
Start at the later first row, as vello_common 0.1 does. The strips and the
pixels are unchanged; a test counts the rows the walk visits and fails
without the change.
vello_common 0.0.8 is vendored with that one-line PdfCraft patch (marked
in the source), its unit test, the vendor/README.md row and an
ATTRIBUTION.toml entry for its reference image. Cargo.lock records it as a
path crate.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* Refresh catalog hashes after merging main
---------
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Apply the minimum after page-unit scaling without inflating small raw crops. Retain the existing unit policy and nearly-zero fallback, with regression coverage for origins, rotations and reciprocal rounding.
Co-authored-by: sanyamsingh001 <314051937+sanyamsingh001@users.noreply.github.com>
Co-authored-by: Codex <codex@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
The vendored hayro-syntax passed each filter an output limit, but
ASCII85 ignored it: every `z` expands one byte to four, so a Flate
stage at its limit could be grown another 4x past every budget.
ASCII85 now writes only what fits under the limit, stops with a
warning once it is reached (like RunLength, LZW and Flate), and caps
its starting capacity at the limit.
Test: decompression_bombs_stop_at_the_stream_limit gets a
[/FlateDecode /ASCII85Decode] bomb; it decoded 301,989,892 bytes
before and stops at MAX_DECODED_STREAM now.
Co-authored-by: skylar taylor-barrick <hi@skylar.cc>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
ISO 32000-2 Table 11 defines /Rows 0 (the default) as "height not
predetermined", but the vendored hayro-syntax decoder took it as zero
rows, so scanned pages written that way rendered blank. Treat 0 like an
absent /Rows and fall back to the image height.
Adds a synthetic regression test: an 8x4 all-black Group 4 stream,
hand-encoded from ITU-T T.6, rendered with /Rows absent, 4 and 0. It
fails without the patch.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(render): convert spot-colour images once per pixel value (vendored hayro-interpret patch 28)
An image whose pixels hold at most 16 bits of samples (an 8-bit Separation or
DeviceN image, a 1-, 2- or 4-bit gray or indexed one) is now converted to RGB
through a table of every possible pixel value instead of pixel by pixel.
A Separation or DeviceN tint transform is a PDF function, often a PostScript
calculator program, and the image decoder ran it once per pixel. Fillable
character sheets draw their background art as an 8-bit DeviceN image: one with
a 3304 x 4105 background took 3.8 s and 430 MB to draw each page (Poppler: 0.27 s),
when opening it and again after every field was filled. It now draws in 0.11 s
and 100 MB, pixel for pixel the same.
The table decodes each sample through the same `decode_sample` as the per-pixel
path, keeps `fix_image_length`'s rules for short and overlong data (the pixels
past the data are converted as before), and is used only when the image has
more pixels than the table has entries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnNgjPEDfKJM1wj17Q8Za8
* fix(render): place widgets without /P in one pass over the pages
Opening a document, the inspector finds the page of a form field whose widget
has no /P by searching every page's /Annots for it, once per field: O(fields x
annotations). A form of 8,000 such fields took 1.17 s to open instead of
0.27 s. The pages' /Annots are now read once, on the first such field, into a
map from annotation to page.
A widget that several pages list is now placed on the first of them, as
pdfcraft-forms places it; the search took whichever page a HashMap visited
first.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnNgjPEDfKJM1wj17Q8Za8
* fix(ui): the Fields panel keeps up with large forms
Every frame the Fields panel searched the form for each row by name, once to
sort the rows into tab order and again to find what a click selects: O(fields²)
a frame. It also laid out, drew and built the tooltip of every row, scrolled
into view or not. With 8,000 fields the app used 13.4 s of CPU for 20 pointer
moves with the panel open (0.2 s with it closed).
The rows are now worked out by `field_rows`, which pairs each row with the
first form field of its name through one map a frame, as the engine pairs
them, and a row out of view is allocated and keeps its clicks, keyboard focus
and accessibility label but isn't drawn and builds no tooltip. The same 20
moves now take 0.86 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnNgjPEDfKJM1wj17Q8Za8
* docs: roadmap entries for faster form-heavy documents
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KnNgjPEDfKJM1wj17Q8Za8
* fix(render): convert each lookup-table entry on its own (vendored hayro-interpret patch 28)
Patch 28 converted the whole table of possible pixel values in one batch.
Separation and DeviceN `convert_f32` concatenate each tint result, and a Type 4
program that returns the wrong number of components is only warned about, so a
malformed result for a value the image never uses shifted every later entry. A
32 x 32 image whose only sample (255) is red drew black because tint 0 returned
one component (PR #1691 review).
`convert_each` now converts the table one entry at a time, so a malformed entry
changes only itself. The largest table (65,536 entries, a two-ink DeviceN image
over DeviceCMYK) takes about 2 ms longer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* fix(text): read a TrueType font without an encoding as WinAnsi (fixes#1468)
* fix(text): describe the WinAnsi reading by pdftotext's output
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Follow up #1237 by deriving Bradford scaling ratios from the source and
D65 white points in LMS. Preserve CalRGB and the merged Indexed/Range
handling. Add independent neutral/chromatic, identity, asymmetric Range,
and degenerate-cone render regressions.
Co-authored-by: Codex <noreply@openai.com>
A PDF whose fonts aren't embedded is drawn with PDFium's Foxit stand-ins
for the standard 14 fonts. Those carry only the 230-odd glyphs of the
Standard, WinAnsi and Mac Roman encodings, so a letter a font names beyond
them in its /Differences mapped to .notdef and drew nothing. iText 2
writes Turkish exactly this way (Times New Roman left unembedded,
/Gbreve /Idotaccent /Scedilla /gbreve /scedilla over Ð Ý Þ ð þ), and such
letters showed blanks for Ğ, İ, Ş, ğ and ş while ı, ç, ö and ü drew.
StandardKind now builds a Latin Extended-A letter the stand-in lacks from
the base letter and accent it does have, as Type 1 seac composites and
the standard-14 AFM composites are built: the accent is centred on the
letter, raised over capitals and letters with ascenders (and moved along
the slant in italics), cedillas hang below and ogoneks from the right
edge. Composite glyph ids sit above any font's glyph count, one per
character code; their width comes from the standard metrics of the name.
Test: raster::tests::accented_letters_the_standard_stand_ins_lack_are_built_from_their_parts
(fails without the patch: Gbreve drew no pixel).
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* M2: snap shrunk interpolated images to whole pixels so scans stay sharp
A scanned page (one 300 dpi image with /Interpolate true) was shrunk to a
ceil-sized raster and then sampled a second time at a fractional scale and
offset, which softened every edge next to Acrobat. An axis-aligned shrunk
interpolated image is now shrunk once to its rounded device rectangle and
drawn 1:1 at the rounded corner. Rotated, skewed, flipped, Type 3 and
non-interpolated images are unchanged.
On a synthetic scan page the mean error against an area-average reference
falls from 3.4 to 1.6 grey levels at 96 dpi (3.0 to 1.5 at 144 dpi).
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* vendor: list the hayro snap patch as (14) in the Patches and Tests columns; correct gaps.md patch counts
The patch description had been appended to the Tests column of one of the
two hayro rows. gaps.md now counts the distinct patches vendor/README.md
lists (26 hayro-interpret, 15 hayro, 3 hayro-jbig2, 17 hayro-syntax).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Revisions 5-6 of the standard security handler key the file on the
password processed with SASLprep (ISO 32000-2 7.6.4.3.3, Algorithm 2.A
step a). pdfcraft-crypt writes and reads it so, but the vendored
hayro-syntax skipped the step ("TODO: Do the above.") and used the UTF-8
bytes as typed. A password that SASLprep changes (a decomposed "cafe" +
U+0301, a no-break space, fullwidth letters typed by an IME in full-width
mode) was wrong for the renderer: Protect with such an open password was
rolled back ("the password is incorrect"), and an AES-256 file with such a
password did not open in a session with it, although pdfcraft-cos
accepted it.
- hayro-syntax (vendored patch 16) tries an R5/R6 password in its SASLprep
form (as pdfcraft-crypt computes it: as given if SASLprep refuses it,
UTF-8, at most 127 bytes), then, when that differs, as given.
stringprep, already in the tree through pdfcraft-crypt, is an optional
dependency enabled by the std feature; without std nothing changes.
- Tests: hayro-syntax crypto::pdfcraft_password_tests (candidate list and
R5/R6 dictionaries built from prepared passwords) and two engine tests
(Protect and reopen, and pdfcraft-cos-encrypted files) for the three
cases, with precomposed "caf\u{E9}" as the control. The dictionary test
and both engine tests fail without the fix.
Fixes#1296.
* Text extraction: honour /ActualText, so Thai from Chrome PDFs is searchable
Chrome (Skia) writes /ActualText around each Thai, Indic or Arabic cluster
whose shaped glyphs (a tone mark moved clear of a vowel, a ligature) have no
Unicode of their own. Text extraction ignored it, so those marks were lost:
"น้ำหนัก" came out as "นาหนัก", "เป็น" as "เปน", and searching for any
such word found nothing.
- hayro-interpret patch (25): Device::actual_text hands the device a
marked content sequence's /ActualText, inline or named in /Properties.
- TextDevice turns the glyphs of a replacement into one glyph carrying the
replacement text, boxed around them. The outermost replacement wins, an
empty one stands for no text, and one spanning several lines or writing
directions keeps per-glyph text so selection and lines still work.
- Canva draws text twice (outline or shadow) and marks only one drawing:
the unmarked redraw at the same places takes the same replacement, so the
existing redraw detection still drops the copy.
Checked on 1,114 real-world PDFs against the previous build, with pdftotext
as the reference: 957 extract byte-identical text, 157 change and all of
them read closer to the reference (exact matches 3 -> 122), none got worse,
none newly fail, and total extraction time is unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Text extraction: bound the work of the ActualText redraw dedupe
`TextDevice::finish()` compared every replacement with every unmatched glyph
in up to 4096 grid cells, so a hostile page stacking ~10^5 glyphs and ~10^5
replacements at one spot meant ~10^10 comparisons (a hang). It now charges
each visited cell and each candidate run against a per-page budget
(`MAX_REDRAW_WORK`, 2^22) and stops deduplicating once it is spent; redraws
found so far are still merged. A regression test feeds 100,000 replacements and
100,000 non-matching glyphs at one place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: git-monchiant <4667243+git-monchiant@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* test: give faux-redaction audit its own workdir (parallel-test flake)
* hayro: ColorReplacement type and luminance mapping (reading colours, #897)
* fmt: engine dates (upstream Greek localization miss)
* hayro: apply ColorReplacement to solid text and line-art paints (#897)
* render: ReadingColors view setting flows into RenderSettings (#897)
* engine: per-document reading colours state (#897)
* automation: view_reading_colors tool; page_render honours it (#897)
* ui: Reading colours preference (Replace Document Colors, #897)
* ui: seed reading colours on revision and signed-version opens (#897)
* parity: a11y.replace-page-colors shipped (#897)
* render: map stencil-mask paints through reading colours (#897)
An ImageMask stencil's solid paint was baked into the raster bytes raw,
bypassing the reading-colours hook every other solid fill goes through:
under white-on-black, CCITT scans, JBIG2 text and stencil logos drew
black ink on the black mapped background and vanished. Route it through
replaced(paint, false) — stencils are line art in disguise — and cover
it with an ImageMask fixture test sampling mapped ink on a mapped
background.
* render: print previews render with reading colours off (#897)
Print previews went through the document's render pool, whose config now
carries the reading colours: a white-on-black user saw black sheets in
the Print dialog while the actual print stayed uncoloured. The request
tag is the per-request channel that already crosses the render boundary,
so PRINT_TAG now lives in pdfcraft_render and the renderer honours it by
rendering previews with reading colours off — a preview's job is
fidelity to the printout. The main view stays recoloured. Covered on
both render paths (inline PageRenderer and pool worker).
* ui: seed reading colours on created-document opens (#897)
open_created (comment summaries) and open_created_bytes (images, text,
blank pages, clipboard) pushed new DocViews without seeding the reading
colours, unlike every other open path. A created document now starts
with the app-wide preference too; the seed-on-open test drives both
paths.
* docs: reading-colours flag echo and display-fidelity parity notes (#897)
The view_reading_colors description now says a flag that differs from
the mode's preset default makes the echoed mode read custom, and the
a11y.replace-page-colors parity notes record that print previews and
view snapshots render with the document's true colours: reading colours
apply to the reading view only.
* Merge follow-ups: reading colours beside preview renders and the new catalogs
- Renderer::new carries both color_replacement and the new preview flag
- our raster tests construct RenderRequest with preview: false
- the preference dialog grew past the window: scroll the combo into view
and give the fixture a valid xref so no repair banner covers the dialog
- checkboxes use the crate's square-box widget (clippy disallows egui's)
- Kazakh and Turkish catalogs gain the ten reading-colours strings
* Reading colours: record the hayro patch in vendor/README.md; add missing translations
vendor/README.md gains hayro patch (13): `RenderSettings::color_replacement` /
`ColorReplacement`, the `render_pdf` signature change and the removed
`[dev-dependencies.log]`, with its tests. Adds the ten reading-colours labels
to el, es, pl, te and zh-hant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix: recognize paragraph drafts as unsaved work (Issue #811)
* fix(color): honour the /Lab colour space /WhitePoint (#1203)
The old path fed every Lab value to the bundled ICC transform as if the
space's white point were D50, so a /Lab space with any other /WhitePoint
(e.g. D65 [0.9505 1 1.089]) rendered off by several levels per channel
and a D65 blue came out 45 levels too red.
Map L*a*b* to XYZ in the space's own white coordinates (ISO 32000-2
§8.6.5.4), Bradford-adapt it to D65 and convert to sRGB directly, as
Poppler does. Missing or degenerate white points fall back to a flat
white and hostile components are clamped without panicking.
Add a render regression that checks the five reported flat patches of
the D65 white-point fixture against their CIE sRGB values.
* Lab: map normalised Indexed palette components onto L*/Range; drop unused LAB.icc
An /Indexed palette over Lab passes its entries as byte/255 (manual_scale),
which the new Lab conversion read as raw L*a*b* and drew near black. Map
them onto L* 0..100 and the a*/b* /Range, with a regression test. Record the
patch as hayro-interpret (25) in vendor/README.md, and remove LAB.icc (no
longer referenced) with its ATTRIBUTION, NOTICE and assets README entries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Main gates: rustfmt tests/viewer.rs; a type alias for the refresh-steps array (clippy type_complexity)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(x11): prefer server DPI over panel physical DPI in winit
Without Xft DPI, use the X server screen dimensions before the output physical density. Xorg can run a dense panel at 96 DPI; EDID sizes alone make only winit apps appear zoomed.
Supersedes #908: preserves Xft.dpi priority, configured HiDPI screens, and WINIT_X11_SCALE_FACTOR override (including randr). Falls back to RandR output density when server dimensions are missing.
Adds default_dpi_factor() with 4 unit tests plus vendored-copy guard test in apps/pdfcraft/tests/winit_x11_dpi.rs.
* vendor/README: record the winit X11 DPI patch as (11)
---------
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Heavy pages: cull off-tile images, cache decodes, preview backdrops, calmer queues
A large CAD sheet (page 24 of the benchmark file: ~5050 pt wide, 2104
images, 9 MB of content) cost a full page interpret per tile: 4.3 s for a
1024 px tile at 1000%, 10.5 s for the whole page, and a storm of renders on
every pan frame. Tiles now cost 0.6 s, the sheet is covered by a handful of
large tiles, and gestures no longer queue work faster than workers finish it.
- Vendored hayro patch (10): skip images whose stream-size box misses the
canvas before decoding them, so a tile no longer decodes, resamples and
rasterizes every image on its page. Pixel-identical; the skip regression
test now covers images too.
- Vendored hayro-interpret patch (23): decoded image pixels shared across
renders through the interpreter cache, keyed by stream and decode target
size within a 64 MiB budget.
- Preview renders (hayro patch 11, RenderRequest::preview through to the
device): backdrops, thumbnails, grid cells and signature-drag backgrounds
skip images under 16 device px. Tiles, whole pages, print and export always
draw everything. Previews may differ by those pixels; finals don't.
- Viewer: zoom/pan settle holds apply only to pages whose raster proved slow
(slow_pages, 500 ms), so fast pages render with no added latency; tile sides
grow with zoom and page size (giant sheets need ~8 tiles, not ~28), queued
center-first and keyed by exact coordinates; Edit-text outlines capped.
- vendor/README.md records the new patches with their tests.
Co-Authored-By: Muse Spark <noreply@meta.com>
* Decoded-image cache: key every decode input, survive a poisoned lock
The key was the dictionary's bytes and the target size, so two images with byte-identical
dictionaries (uncompressed, one size) shared pixels, and an image drawn under a transfer
function reused pixels decoded without it. DecodedImageKey now holds the dictionary's place
in the document (its data follows it), its bytes and object, mask/stencil/raster output,
interpolation, the target size and preview or final quality; images under a transfer
function and inline images (named colour spaces, dictionaries in content streams) are not
cached, and an image over the budget is never kept. A poisoned lock decodes uncached
instead of panicking. Soft masks: the per-renderer cache can't carry a preview mask into a
final render (documented). Regression tests for identical dictionaries, preview then final
through one cache, and transfer functions.
---------
Co-authored-by: Muse Spark <noreply@meta.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* Fix#624: preserve hard edges for non-interpolated images
Select nearest-neighbour for Hayro's image pre-minification when /Interpolate is absent or false, retaining Catmull-Rom for true. Add a synthetic regression covering default, false, and true behaviour and record the vendored patch.
Co-authored-by: Codex <noreply@openai.com>
* M2: test non-interpolated minification with exact colours
The #624 regression used alternating black and white columns. Nearest 8→4 sampling reads only white columns from that pattern, so an image skipped entirely still passed, and the check only asked for near-black or near-white pixels.
Replace it with 2×2 colour blocks whose exact device output is asserted, with /Interpolate omitted equal to false, and /Interpolate true still blending. Add exact cases for gray, a non-integer 10→6 minification, and a matched soft mask. All pass on this head. On origin/main the same block image renders with blended mid-tones (for example 12eb12ff where pure green is expected), so these tests fail without the #624 change.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: plan nearest-neighbour pre-resize without filter limits
Non-interpolated images are pre-resized with nearest-neighbour, which copies source pixels and needs no filter tables, crossed intermediate or scratch. The preflight still applied the Catmull-Rom checks to it: a source wider than 2^20 pixels, or a filter plan over 128 MiB, was refused, and once such a source was wider than the backend's u16 limit the image was skipped.
resampling_byte_len and image_resampling_size take the image's interpolation flag. Nearest keeps the source and target buffer bounds and refuses any scratch it reports. Filtered images keep their limits unchanged, and the existing image_resampling_size tests pass true.
Measured: a 1.5M-pixel-wide RGB row drawn at 750×4 device pixels renders 0 non-white pixels on origin/main (the image is skipped) and all 3000 pixels with this commit. The 1.1M-pixel gray case is covered by wide_non_interpolated_sources_draw_after_shrinking.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: keep non-interpolated Type 3 glyph images nearest
Minified images inside Type 3 glyphs were drawn with ImageQuality::High, a bicubic sampler, whatever their /Interpolate. A non-interpolated image had already been pre-resized with nearest-neighbour, and bicubic sampling of that result blended it again. High is now used only for interpolated images, as elsewhere.
Before this commit the new test saw an interior glyph pixel of [28, 28, 27] (not a palette colour) for a 4.6 × 2.3 px glyph of a palette image; after it, every interior pixel is a palette colour. /Interpolate true still blends.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: nearest-sample masked minified images straight into premultiplied RGBA
A non-interpolated image with a matching soft mask used to build a full-resolution RGBA copy, nearest-resize it, then premultiply the target. For a minified image most of that copy was discarded.
nearest_premultiplied_rgba reads colour and mask from the same source pixel, selected with PicScale's nearest index (same fixed-point centre sampling as ResampleNearestPlan) and premultiplies only the target pixels. It falls back to the previous path if its target buffer cannot be allocated. Interpolated images, unmasked images and magnification are unchanged.
New test: a ten-column gray ramp with a soft mask drawn six pixels wide must pick the same columns for colour and mask.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: record the #624 image patch in vendor/README.md
The hayro row listed the #624 regression test but not the patch itself. Add patch item (10) covering the nearest pre-resize and its planning, the Type 3 bicubic restriction and the premultiplied soft-mask path, and list every regression test that covers them.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
InDesign writes fades as luminosity soft masks whose group is one
extended axial `sh` over the whole page. hayro 0.7 sampled every such
shading pixel by pixel through the shading function, so a report page
with 40 of them took 10-15 s at 144 dpi.
Backport hayro 0.8.0's `gradient` module: an axial or radial shading
that a native gradient draws exactly becomes sRGB stops (within 0.01
per channel, a stop at every stitching bound), painted as a vello_cpu
gradient. Upstream's half-pixel shift is left out: it suits vello_cpu
0.3 but moves 0.0.8's gradients off pixel centres.
Page 4 of the reported file: 10.8 s -> 1.0 s.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* M2: stream page content past the 256 MiB cap (item 1 of storytold/pdfcraft#623)
Page content was decoded whole, and the decode stopped at
MAX_DECODED_STREAM (256 MiB), so the last ~10 MB of the #623 sample
was dropped without a warning. Top-level page content is now read a
window at a time, so the decoded content is never all in memory:
- the content tokenizer reads windows and holds back an instruction
that runs past one; Reader records reads that touch the end of the
data, including reads made through clones, so an instruction is
only decided once it has been seen whole;
- the Flate decoder is resumable and keeps its own 32 KiB history, so
it pauses at any output size and back-references do not depend on
what the caller has kept;
- page content may decode to max(256 MiB, 64 x its compressed size),
counting each distinct stream once; content that reaches that
budget is cut and the page reports ContentTruncated, as it did not
before;
- forms, tiling patterns, Type 3 glyph procedures and generic streams
keep their existing budgets and caps.
Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
* M2: Avoid duplicate page content raw reads
* M2: Document page content byte reuse
---------
Co-authored-by: Claude Haiku 5.5 <noreply@anthropic.com>
* M13: report unavailable symbolic-link fixtures in hostile-argument tests
Only include the planted link escape when symlink creation succeeds. On
non-elevated Windows the fixture can fail with OS error 1314; exercising
its nonexistent path otherwise produces false confinement leaks. Keep
all remaining traversal and absolute-path cases and report the skip.
Co-authored-by: Codex (GPT-6) <noreply@openai.com>
* M3: return renderer initialization errors to the OpenGL startup fallback
Guard renderer creation with the existing scoped GPU-error handler and
return CreateRenderer instead of invoking wgpu's default panic handler.
This lets Auto startup reach its existing OpenGL retry before app creation.
Add a real-device invalid/valid pipeline regression and Windows startup
reproduction notes. The host-specific DX12 shader compiler error itself
is not fixed; explicit renderer choices and the non-winit path are unchanged.
Co-authored-by: Codex (GPT-6) <noreply@openai.com>
* Leave the symlink-test fix to #949, which covers the same ground
---------
Co-authored-by: Codex (GPT-6) <noreply@openai.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Revisions 2–4 of the standard security handler take the password in
PDFDocEncoding (ISO 32000-2 §7.6.4.3.2, Algorithm 2). PdfCraft authenticates
an encrypted file twice, and neither did that:
- pdfcraft-crypt took Latin-1 and dropped every other character, so "ı"
(0x9A) and "€" (0xA0) vanished and "ılık€" became "lk";
- the renderer (vendored hayro-syntax, "TODO: Convert to PDFDocEncoding")
used the UTF-8 bytes, so any non-ASCII password failed, even "Çok güzel",
which Latin-1 and PDFDocEncoding write alike;
- the engine handed the renderer an owner-recovered user password as Latin-1
characters, which the renderer then read as UTF-8 again.
Files that qpdf or Acrobat protected with such passwords could not be
opened with the right password.
- pdfcraft-crypt encodes R2–R4 passwords in PDFDocEncoding and, for a
password with characters PDFDocEncoding lacks (ğ, ş, İ), also tries its
UTF-8 bytes, which qpdf writes for those. The PDFDocEncoding table moves
here from pdfcraft-cos (which depends on this crate) and cos uses it.
- hayro-syntax (vendored patch 14) tries an R2–R4 password as given, then
in PDFDocEncoding.
- The engine passes a recovered user password to the renderer as UTF-8 when
its bytes are UTF-8 and as PDFDocEncoding otherwise, so the renderer gets
exactly those bytes back.
Co-authored-by: Brandon Thomas <echelon@gmail.com>
Skipping a direct array or dictionary, and the lenient repair of a
dictionary that has an object where a key belongs, recursed once per
level with no bound, so deep nesting in a trailer entry or a content
stream overflowed the stack. Count the depth on the Reader and fail
beyond 128 levels like any other malformed object.
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
A compressed object's container was resolved with a fresh cycle set and
without checking that it is an ordinary object, and indirect /Length
chains had no depth limit, so self-containing or chained lookups
overflowed the stack. Pass the caller's set on, require an in-file
container and cap the chain at 16 lookups. Object-stream members were
each parsed to the end of the decoded stream, so repeated or nested
offsets kept many copies of one large object: parse each member within
its own extent and skip repeated offsets.
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* M2: pages with /UserUnit are shown at their size in points, as in Acrobat
/UserUnit gives the size of a page's user-space unit in points; PdfCraft
ignored it everywhere except the measuring tools, so a /UserUnit 2 page
was shown, extracted, printed and edited at half its size, and the
measuring tools disagreed with the rest of the app.
The value is now read as Acrobat reads it (the page's own entry, a number
from 1 up, at most 75,000; anything else counts as 1) in two places:
- vendored hayro-syntax Page (patch 13): render_dimensions and
initial_transform give the page in points, so rendering, tiles,
PageInfo and the text layer follow; a page without it keeps exactly
the transform it had.
- pdfcraft-model Page: user_unit(), and display_size/view_matrix in
points, so print, header/footer, watermark, background, added content
and measuring agree with the screen. The measure crate drops its own
arithmetic. Added items record the /UserUnit they were written for;
items without the record (written before) read back converted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* M2: an added item's recorded /UserUnit is clamped like the page's
An infinite or huge recorded /UserUnit made the scale factor 0 and collapsed the item.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* macOS: IME text committed after the marked text is cleared isn't lost (winit#4650)
winit 0.30.13 reports insertText as Ime::Commit only while marked text is
left, and after any commit it swallows the next key. Real paths where that
loses text:
- input methods that end a composition with unmarkText, then insertText
(rust-windowing/winit#3925): the committed text was dropped;
- Apple Korean 2-Set, for the key typed after a syllable: on 한 then 5 it
calls setMarkedText("한"), insertText("한"), setMarkedText(""),
insertText("5"), so the 5 (or ;, or any key that ends the syllable) was
dropped;
- a commit made outside a key press (a candidate clicked with the mouse,
dictation): no key follows to reset the state, so the next key (for
example Backspace) was swallowed.
vendor/winit now commits when a composition was started (pending_commit,
set in setMarkedText, cleared by the commit and when IME is disallowed);
other insertText calls still reach the app as KeyboardInput. A commit
outside keyDown clears the marked text and returns to the ground state.
validAttributesForMarkedText lists the underline and marked-clause
attributes, which some input methods need before they compose.
This is rust-windowing/winit#4650, the 0.30 backport of #4651 (merged on
winit master), plus the part of #4748 that fixes#4651's out-of-key-press
case (#4748's other parts are not included). One correction: upstream
spells the underline attribute "NSUnderlineStyle", but AppKit's
NSUnderlineStyleAttributeName is "NSUnderline" (printed on macOS 14), so
the AppKit value is used.
Checked with the real input methods on macOS 14.7.1 (a scratch winit app fed
key events through AppKit): with the patch 한+5 and 한+; commit the 5 and ;,
unmarkText-then-insertText commits, and Backspace after a commit outside a
key press is delivered; ABC typing, held keys, Japanese (日本, あ), Cangjie
(日) and Korean composition give the same events as before. A test fails if
a re-vendored winit loses the patch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* macOS: the first Apple Korean key in a fresh PdfCraft composes (winit#4744)
On the first key after PdfCraft starts, Apple Korean 2-Set can answer
interpretKeyEvents with insertText and the raw compatibility jamo instead of
setMarkedText. winit sends that key to the app as an ordinary key, and the
next keys compose without it: g k s Space typed ㅎㅏㄴ instead of 한 (Search,
form fields, comments; rust-windowing/winit#3095).
vendor/winit now ports rust-windowing/winit#4744: keyDown interprets that
one key a second time, synchronously, before any KeyboardInput is sent, and
only when every condition holds: Apple Korean, IME allowed and not yet
composing, no pending commit or marked text, no Command/Control/Option, not
a repeat, the key window's first responder, the only callback is one
insertText of that same single jamo, and focus and input source unchanged
afterwards. The decision is ImeStartup (ime_startup.rs, copied with its
seven tests); no timer, no key queue, no text rewriting. It sits on top of
the previous macOS IME change (pending_commit, in_key_down); the retry runs
inside in_key_down. Command maps to ModifiersState::SUPER in winit 0.30
(META on winit master).
Checked with the real input method on macOS 14.7.1 (scratch winit app, fresh
process each time): without the patch g k s Space gave ㅎ as a key, then
ㅏ and ㄴ committed separately, 3 runs out of 3; with it, ㅎ → 하 → 한 → 한
committed, 3 out of 3. All other traces (Japanese, Cangjie, ABC, held
keys, later Korean keys) are unchanged. ImeStartup's tests pass with
rustc --test. A test fails if a re-vendored winit loses the patch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* macOS: IME text committed after the marked text is cleared isn't lost (winit#4650)
winit 0.30.13 reports insertText as Ime::Commit only while marked text is
left, and after any commit it swallows the next key. Real paths where that
loses text:
- input methods that end a composition with unmarkText, then insertText
(rust-windowing/winit#3925): the committed text was dropped;
- Apple Korean 2-Set, for the key typed after a syllable: on 한 then 5 it
calls setMarkedText("한"), insertText("한"), setMarkedText(""),
insertText("5"), so the 5 (or ;, or any key that ends the syllable) was
dropped;
- a commit made outside a key press (a candidate clicked with the mouse,
dictation): no key follows to reset the state, so the next key (for
example Backspace) was swallowed.
vendor/winit now commits when a composition was started (pending_commit,
set in setMarkedText, cleared by the commit and when IME is disallowed);
other insertText calls still reach the app as KeyboardInput. A commit
outside keyDown clears the marked text and returns to the ground state.
validAttributesForMarkedText lists the underline and marked-clause
attributes, which some input methods need before they compose.
This is rust-windowing/winit#4650, the 0.30 backport of #4651 (merged on
winit master), plus the part of #4748 that fixes#4651's out-of-key-press
case (#4748's other parts are not included). One correction: upstream
spells the underline attribute "NSUnderlineStyle", but AppKit's
NSUnderlineStyleAttributeName is "NSUnderline" (printed on macOS 14), so
the AppKit value is used.
Checked with the real input methods on macOS 14.7.1 (a scratch winit app fed
key events through AppKit): with the patch 한+5 and 한+; commit the 5 and ;,
unmarkText-then-insertText commits, and Backspace after a commit outside a
key press is delivered; ABC typing, held keys, Japanese (日本, あ), Cangjie
(日) and Korean composition give the same events as before. A test fails if
a re-vendored winit loses the patch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* macOS: Space in Apple Korean with no syllable in progress types one space (winit#4478)
With Apple Korean 2-Set on macOS 14, Space with no syllable in progress
(a second space, or a space after a digit or punctuation) makes AppKit
call setMarkedText(" "), insertText(" "), then setMarkedText(""). winit
reports the insertText as Ime::Commit(" "), but the trailing empty
setMarkedText returns its state to the ground state, so keyDown also sends
the key as KeyboardInput with text " ". egui inserts both, and PdfCraft's
text fields got two spaces (the double space of alacritty/alacritty#8079).
vendor/winit now records that the key being handled committed IME text and
then doesn't also send it as KeyboardInput, unless AppKit forwards it as a
command through doCommandBySelector (Enter after a syllable still commits
the syllable and sends Enter). keyDown consumes the record on every key,
so keys typed after IME is disallowed are unaffected.
rust-windowing/winit#4478 aims at the same double space, but its changes
(clearing the marked text on commit, forwarding a later insertText, and
dropping doCommandBySelector's guard) don't reach this call order, and the
key loss it also fixes is already fixed by the previous change
(pending_commit). This is the narrow form for the call order observed
here; it should go upstream with #4478.
Checked with the real input methods on macOS 14.7.1 (scratch winit app, on top
of the previous change): the duplicate KeyboardInput Space is gone and
nothing else changes (Korean syllables, 한+Enter, 한+5, Japanese, Cangjie,
ABC, held keys, keys after IME is disallowed). A test fails if a
re-vendored winit loses the patch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep #678's emoji-picker branch: view.rs is main's plus this PR's Space fix (54b3d3c)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
* macOS: IME text committed after the marked text is cleared isn't lost (winit#4650)
winit 0.30.13 reports insertText as Ime::Commit only while marked text is
left, and after any commit it swallows the next key. Real paths where that
loses text:
- input methods that end a composition with unmarkText, then insertText
(rust-windowing/winit#3925): the committed text was dropped;
- Apple Korean 2-Set, for the key typed after a syllable: on 한 then 5 it
calls setMarkedText("한"), insertText("한"), setMarkedText(""),
insertText("5"), so the 5 (or ;, or any key that ends the syllable) was
dropped;
- a commit made outside a key press (a candidate clicked with the mouse,
dictation): no key follows to reset the state, so the next key (for
example Backspace) was swallowed.
vendor/winit now commits when a composition was started (pending_commit,
set in setMarkedText, cleared by the commit and when IME is disallowed);
other insertText calls still reach the app as KeyboardInput. A commit
outside keyDown clears the marked text and returns to the ground state.
validAttributesForMarkedText lists the underline and marked-clause
attributes, which some input methods need before they compose.
This is rust-windowing/winit#4650, the 0.30 backport of #4651 (merged on
winit master), plus the part of #4748 that fixes#4651's out-of-key-press
case (#4748's other parts are not included). One correction: upstream
spells the underline attribute "NSUnderlineStyle", but AppKit's
NSUnderlineStyleAttributeName is "NSUnderline" (printed on macOS 14), so
the AppKit value is used.
Checked with the real input methods on macOS 14.7.1 (a scratch winit app fed
key events through AppKit): with the patch 한+5 and 한+; commit the 5 and ;,
unmarkText-then-insertText commits, and Backspace after a commit outside a
key press is delivered; ABC typing, held keys, Japanese (日本, あ), Cangjie
(日) and Korean composition give the same events as before. A test fails if
a re-vendored winit loses the patch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* macOS: emoji from the emoji picker reach text fields (winit#4749)
The macOS emoji picker (Fn/Globe-E, Control-Command-Space) inserts its text
with insertText:replacementRange: outside any key press and without a
composition. winit 0.30.13 reports insertText only as part of a key press or
a composition, so an emoji picked into Search, a form field or a comment
was dropped.
vendor/winit now reports text inserted outside a key press, with IME
allowed and no composition, as Ime::Commit, as rust-windowing/winit#4749
(the 0.30 backport of #4748) does. Builds on the pending_commit and
in_key_down state of the previous macOS IME change.
#4748's selectedRange change, which lets dictation start, is left out:
checked on macOS 14.7.1, a valid selection switches on press-and-hold for
every held key in a winit window, so holding e kept sending e key events
under an accent menu, and the accent picked with 1 arrived as a separate
commit after "eeee…1". Without it, held keys repeat as before.
Checked with a scratch winit app on macOS 14.7.1: an insertText outside a key
press now arrives as Ime::Commit; held keys, ABC, Japanese, Cangjie and
Korean typing give the same events as before. A test fails if a re-vendored
winit loses the patch or gains the selectedRange change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Brandon Thomas <echelon@gmail.com>
jonathanwxh-cell