mirror of
https://github.com/storytold/pdfcraft.git
synced 2026-10-25 18:47:22 -06:00
sonar-fix/enforce-https
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
4541974ef9 |
Reading colours: replace document colours for reading comfort (#897) (#1270)
* test: give faux-redaction audit its own workdir (parallel-test flake) * hayro: ColorReplacement type and luminance mapping (reading colours, #897) * fmt: engine dates (upstream Greek localization miss) * hayro: apply ColorReplacement to solid text and line-art paints (#897) * render: ReadingColors view setting flows into RenderSettings (#897) * engine: per-document reading colours state (#897) * automation: view_reading_colors tool; page_render honours it (#897) * ui: Reading colours preference (Replace Document Colors, #897) * ui: seed reading colours on revision and signed-version opens (#897) * parity: a11y.replace-page-colors shipped (#897) * render: map stencil-mask paints through reading colours (#897) An ImageMask stencil's solid paint was baked into the raster bytes raw, bypassing the reading-colours hook every other solid fill goes through: under white-on-black, CCITT scans, JBIG2 text and stencil logos drew black ink on the black mapped background and vanished. Route it through replaced(paint, false) — stencils are line art in disguise — and cover it with an ImageMask fixture test sampling mapped ink on a mapped background. * render: print previews render with reading colours off (#897) Print previews went through the document's render pool, whose config now carries the reading colours: a white-on-black user saw black sheets in the Print dialog while the actual print stayed uncoloured. The request tag is the per-request channel that already crosses the render boundary, so PRINT_TAG now lives in pdfcraft_render and the renderer honours it by rendering previews with reading colours off — a preview's job is fidelity to the printout. The main view stays recoloured. Covered on both render paths (inline PageRenderer and pool worker). * ui: seed reading colours on created-document opens (#897) open_created (comment summaries) and open_created_bytes (images, text, blank pages, clipboard) pushed new DocViews without seeding the reading colours, unlike every other open path. A created document now starts with the app-wide preference too; the seed-on-open test drives both paths. * docs: reading-colours flag echo and display-fidelity parity notes (#897) The view_reading_colors description now says a flag that differs from the mode's preset default makes the echoed mode read custom, and the a11y.replace-page-colors parity notes record that print previews and view snapshots render with the document's true colours: reading colours apply to the reading view only. * Merge follow-ups: reading colours beside preview renders and the new catalogs - Renderer::new carries both color_replacement and the new preview flag - our raster tests construct RenderRequest with preview: false - the preference dialog grew past the window: scroll the combo into view and give the fixture a valid xref so no repair banner covers the dialog - checkboxes use the crate's square-box widget (clippy disallows egui's) - Kazakh and Turkish catalogs gain the ten reading-colours strings * Reading colours: record the hayro patch in vendor/README.md; add missing translations vendor/README.md gains hayro patch (13): `RenderSettings::color_replacement` / `ColorReplacement`, the `render_pdf` signature change and the removed `[dev-dependencies.log]`, with its tests. Adds the ten reading-colours labels to el, es, pl, te and zh-hant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Brandon Thomas <echelon@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
bd4736d39c |
M2: bound page-label traversal and output (#480)
* hayro-syntax: fix float mantissa overflow on high-precision coordinates (#199) jsPDF outputs double-precision floating-point coordinates with 16+ decimal places in /MediaBox. In hayro-syntax, read_inner accumulated float mantissa digits into u64 using wrapping arithmetic. Integer mantissas exceeding u64::MAX wrapped modulo 2^64, severely corrupting page dimensions (e.g., 3874.96 pt became 185.61 pt, rendering as a warped ribbon). Accumulation now uses checked arithmetic: trailing fractional digits exceeding u64 capacity are safely ignored without incrementing decimal_shift. This preserves full IEEE-754 f64 precision while preventing overflow for numbers with arbitrary numbers of decimal places. Includes: - Unit test in hayro-syntax with reported values and a 50-digit fraction. - Synthetic PDF regression test in pdfcraft-render verifying inspect(). - Vendored patch (6) documentation in vendor/README.md. Fixes #199 * M2: bound page-label work and output before applying labels Share bounded PDF Roman and repeated-letter formatters between inspection and editing. Bound tree traversal, raw and decoded prefixes, individual labels and aggregate label output, with checked numbering arithmetic. Stage inspection labels before applying them and retain physical page numbers with a warning when optional labels cannot be processed. Reject unsupported label edits before changing the document tree. Add synthetic limit, cycle, Unicode, numbering and atomic-edit regressions. Co-Authored-By: OpenAI Codex <codex@openai.com> * Never crash: mesh shadings can't exhaust memory or run for minutes `cargo xtask fuzz` (with the dash, JBIG2 and decompression fixes in place) found a page whose Coons patch shading (type 6), a 147-byte stream with 1-bit coordinates decoded onto +-40000, took the renderer past 3 GB. hayro-interpret samples mesh shadings (types 4-7) pixel by pixel over each triangle's device-space bounding box into a map, with nothing but the u16 cast bounding it: up to 65536 x 65536 entries for one triangle. A synthetic single patch passed 6 GB in six seconds; ten thousand triangles that each cover half a 1000 x 1000 page ran for minutes. Two rounds of review of earlier versions found more: every patch was cut into 722 triangles (about 150 KB) before anything limited them; a lattice mesh with `/VerticesPerRow 0` pushed empty rows forever, and one with 4294967295 read the whole stream into one row; a mesh painted a thousand times was triangulated a thousand times; and the first caps cut content without a word. Vendored hayro-interpret patch (18): - `ShadingPattern::encode_within(area)`: the vendored hayro renderer passes the area it will actually draw (the path's bounding box on the canvas, which it already computes) plus a pixel for texels at a fractional edge, and triangles are sampled only inside it, so the map holds at most that area's pixels. - `TriangleSampler`: at most 8 visits per pixel of that area plus 4 per triangle in it. A triangle's bounding box is about twice its area and a sub-pixel triangle's is at most 2 x 2 pixels, so tiling meshes and small dense ones (a patch in a few pixels is 722 triangles) still shade completely; overlapping ones stop. - Coons and tensor-product patches are triangulated one at a time. - Meshes hold at most `MAX_MESH_PATCHES` (2^16; a 200 x 200 gradient mesh is 40 thousand) patches and `MAX_MESH_TRIANGLES` (2^20) triangles, and a lattice needs two (ISO 32000-2 8.7.4.5.6) to 2^19 vertices per row. - A page samples at most `MAX_PAGE_MESH_TRIANGLES` (2^25; the 200 x 200 mesh is 29 million) mesh triangles in all, reset per page like the existing paint budget. - Each of these cuts logs a warning. `encode()` keeps its signature; with no area it samples within a fixed 2^24 visits. Synthetic regression tests in render: - `mesh_shadings_far_off_the_page_render`: a Coons patch and a triangle mesh decoded onto +-40000 render and the rest of the page draws; a patch on the page, the part of the huge one that covers it, and a patch filling a 4 x 4 pt rectangle as a pattern are all shaded. - `many_overlapping_mesh_triangles_finish`: 10,000 triangles each covering half of a 1000 x 1000 page finish, and are drawn. - `mesh_shadings_with_many_patches_or_no_rows_finish`: 40,000 patches beside the page, and a lattice with `/VerticesPerRow 0`. Red on main (out of memory, killed at 3.9 GB). Each part is pinned by a mutation: no visit budget stalls past 60 s; no clip leaves the covering patch unshaded; no per-triangle allowance leaves the small patch unshaded; triangulating all patches first runs out of memory (killed at 5.8 GB); no rows check runs out of memory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Never crash: decompression bombs can't exhaust the renderer's memory The renderer's parser (vendored hayro-syntax) decoded Flate, LZW and RunLength streams with no output limit. A 2 MB page whose content stream inflates to 2 GB, or whose 8 x 8 image does, took `pdfcraft-cli render` and `text` past 3 GB in under a second, and a few more MB of file scale it further. On the desktop that is the first-page render of an emailed PDF, and the allocation-failure abort takes every open document with it. Our own layers already bound single streams (lopdf's load-time decoding at 256 MiB, cos at 1 GiB), so `info` stays at a few MB; the renderer path didn't. Random mutation almost never makes a high-ratio stream, so `cargo xtask fuzz` doesn't find this; found by hand while triaging the fuzz run. Vendored hayro-syntax patch (6): - `decode_limit` gives each filter an output limit, and Flate (both flate2 and the fallback inflater), LZW and RunLength never write past it (the buffer can still grow to under twice it): `MAX_DECODED_STREAM`, 256 MiB, lopdf's figure; or, for the last filter of an image whose size, bits per component and components are known, twice its pixel bytes (a PNG predictor tag byte per byte at most), capped at 1 GiB. Earlier filters in a chain produce compressed data and get the general limit. - `Stream::decoded_within(max)` for callers that budget. - A page's `/Contents` array shares one `MAX_DECODED_STREAM` (naming one bomb twenty times decoded 5 GB), reserving exactly what it appends. Vendored hayro-interpret patch (18): - The content a page holds at once (its contents, and the forms, tiling patterns and Type 3 glyph procedures being painted) stays within `MAX_PAGE_CONTENT`, 512 MiB, reset per page like the existing paint budget. Forms nest fifty deep and each level kept its content while the next ran: a form painting itself around a 256 MiB stream would have held over 12 GB. Each level gives its bytes back when it is dropped, so content painted again and again keeps drawing, and a stream without filters (borrowed from the file) costs nothing. - Type 0 sampled functions read every sample into a 4-byte value and then a table entry: 256 MiB of 1-bit samples asked for over 8 GB. They now take at most 2^21 table points and 2^23 values (the largest real tables, DeviceN tint transforms such as 33^4 points x 3 outputs, are 1.2 million and 3.6 million), counted with checked arithmetic, and read no further than `/Size` asks. `/Size 0` no longer underflows building the default `/Encode`. Three rounds of independent review found the aggregate cases, buffer doubling past the limit, a chain/predictor shape the first image limit would have truncated, the Type 0 amplifier, and a first, cumulative budget that would have dropped legitimately repeated content, and a first Type 0 cap low enough to refuse real tint transforms; all fixed here. Measured in a release build (render; OS peak commit): - content stream inflating to 2 GB: 389 MB, 0.3 s - 8 x 8 image inflating to 2 GB: 4 MB, 0.2 s - /Contents naming it twenty times: 516 MB, 0.3 s - a form painting itself around it: 644 MB, 0.4 s Synthetic regression tests in render (flate2 is a new dev-dependency there, already in the workspace): - `decompression_bombs_stop_at_the_stream_limit`: 288 MiB bombs through flate2, the fallback inflater (a stored block with a wrong length check, which flate2 refuses and the fallback accepts), LZW and RunLength each decode exactly 256 MiB, and at most 128 bytes as an 8 x 8 image. - `page_contents_and_nested_forms_share_one_budget`: three /Contents references decode one stream's worth; a self-painting form finishes. - `content_painted_again_and_again_keeps_drawing`: a 2 MiB form painted 300 times draws the 300th time. - `sampled_functions_with_billions_of_samples_are_refused`. Ten mutations each turn them red: no limit at all; no limit in flate2, the fallback, LZW, RunLength or for images; an unshared /Contents limit; no page budget; no refund on drop; no Type 0 sample cap. (Taking a form out of the budget no longer compiles: its content is the budget guard.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * M2: normalize JPEG2000 samples without overflowing Use bounded integer sample scaling for 1-32-bit JPX images and reject empty dimensions before chunking. Cover public decoding and continued page rendering with synthetic regressions. Co-Authored-By: OpenAI Codex <codex@openai.com> * Never crash: decompression cut at its limit is logged, not silent; vendor README renumbered after #575 --------- Co-authored-by: philkwek <philkwek03@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: James Locke <336218001+jcl-bit@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Brandon Thomas <echelon@gmail.com> |
||
|
|
e88def3465 |
Never crash: decompression bombs can't exhaust the renderer's memory (#411)
* Never crash: decompression bombs can't exhaust the renderer's memory The renderer's parser (vendored hayro-syntax) decoded Flate, LZW and RunLength streams with no output limit. A 2 MB page whose content stream inflates to 2 GB, or whose 8 x 8 image does, took `pdfcraft-cli render` and `text` past 3 GB in under a second, and a few more MB of file scale it further. On the desktop that is the first-page render of an emailed PDF, and the allocation-failure abort takes every open document with it. Our own layers already bound single streams (lopdf's load-time decoding at 256 MiB, cos at 1 GiB), so `info` stays at a few MB; the renderer path didn't. Random mutation almost never makes a high-ratio stream, so `cargo xtask fuzz` doesn't find this; found by hand while triaging the fuzz run. Vendored hayro-syntax patch (6): - `decode_limit` gives each filter an output limit, and Flate (both flate2 and the fallback inflater), LZW and RunLength never write past it (the buffer can still grow to under twice it): `MAX_DECODED_STREAM`, 256 MiB, lopdf's figure; or, for the last filter of an image whose size, bits per component and components are known, twice its pixel bytes (a PNG predictor tag byte per byte at most), capped at 1 GiB. Earlier filters in a chain produce compressed data and get the general limit. - `Stream::decoded_within(max)` for callers that budget. - A page's `/Contents` array shares one `MAX_DECODED_STREAM` (naming one bomb twenty times decoded 5 GB), reserving exactly what it appends. Vendored hayro-interpret patch (18): - The content a page holds at once (its contents, and the forms, tiling patterns and Type 3 glyph procedures being painted) stays within `MAX_PAGE_CONTENT`, 512 MiB, reset per page like the existing paint budget. Forms nest fifty deep and each level kept its content while the next ran: a form painting itself around a 256 MiB stream would have held over 12 GB. Each level gives its bytes back when it is dropped, so content painted again and again keeps drawing, and a stream without filters (borrowed from the file) costs nothing. - Type 0 sampled functions read every sample into a 4-byte value and then a table entry: 256 MiB of 1-bit samples asked for over 8 GB. They now take at most 2^21 table points and 2^23 values (the largest real tables, DeviceN tint transforms such as 33^4 points x 3 outputs, are 1.2 million and 3.6 million), counted with checked arithmetic, and read no further than `/Size` asks. `/Size 0` no longer underflows building the default `/Encode`. Three rounds of independent review found the aggregate cases, buffer doubling past the limit, a chain/predictor shape the first image limit would have truncated, the Type 0 amplifier, and a first, cumulative budget that would have dropped legitimately repeated content, and a first Type 0 cap low enough to refuse real tint transforms; all fixed here. Measured in a release build (render; OS peak commit): - content stream inflating to 2 GB: 389 MB, 0.3 s - 8 x 8 image inflating to 2 GB: 4 MB, 0.2 s - /Contents naming it twenty times: 516 MB, 0.3 s - a form painting itself around it: 644 MB, 0.4 s Synthetic regression tests in render (flate2 is a new dev-dependency there, already in the workspace): - `decompression_bombs_stop_at_the_stream_limit`: 288 MiB bombs through flate2, the fallback inflater (a stored block with a wrong length check, which flate2 refuses and the fallback accepts), LZW and RunLength each decode exactly 256 MiB, and at most 128 bytes as an 8 x 8 image. - `page_contents_and_nested_forms_share_one_budget`: three /Contents references decode one stream's worth; a self-painting form finishes. - `content_painted_again_and_again_keeps_drawing`: a 2 MiB form painted 300 times draws the 300th time. - `sampled_functions_with_billions_of_samples_are_refused`. Ten mutations each turn them red: no limit at all; no limit in flate2, the fallback, LZW, RunLength or for images; an unshared /Contents limit; no page budget; no refund on drop; no Type 0 sample cap. (Taking a form out of the budget no longer compiles: its content is the budget guard.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * M2: normalize JPEG2000 samples without overflowing Use bounded integer sample scaling for 1-32-bit JPX images and reject empty dimensions before chunking. Cover public decoding and continued page rendering with synthetic regressions. Co-Authored-By: OpenAI Codex <codex@openai.com> * Never crash: decompression cut at its limit is logged, not silent; vendor README renumbered after #575 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Streetblock <github@streetblock.de> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Brandon Thomas <echelon@gmail.com> |
||
|
|
63363ec438 |
M3: rasters reach egui without a copy on the UI thread (#555)
* M3: rasters reach egui without a copy on the UI thread Every page, tile and thumbnail raster was copied pixel by pixel on the UI thread (`ColorImage::from_rgba_premultiplied`), because egui's `Color32` is 4-byte aligned and a `Vec<u8>` isn't. That is 0.5 ms per 1024² tile, 5.4 ms for a 33 MB page and 36 ms for a 197 MB page, against a 16.7 ms frame. hayro, vendored patch 7: `render_size` and `render_into` render into a buffer the caller allocates. `render` is `render_into` on a new pixmap, so its output is unchanged. pdfcraft-render renders straight into `Pixels`, the same premultiplied RGBA bytes kept in one 4-byte word per pixel. `Pixels` dereferences to `[u8]`, so every reader of `RenderedPage::rgba` compiles unchanged. The canvas reinterprets the words as `Color32`s and hands them to egui as they are. Pixels are identical: all 33 renders of #260's test file hash the same. The UI-thread copy is gone, and rendering a 6182×8000 page takes 67 ms instead of 71–76 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * M3: render_into leaves a buffer of the wrong length untouched instead of reaching vello's assert * M3: regression test for render_into with a wrong-length buffer --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Brandon Thomas <echelon@gmail.com> |
||
|
cyfung1031andClaude Opus 5.5
|
d6c2a694ec |
M3: rendered pixels are moved out of the pixmap, not copied (#404)
`render_page` copied every raster out of vello_cpu's pixmap with `data_as_u8_slice().to_vec()`. That is a second buffer the size of the raster (up to 256 MB for a 64 MP page), alive alongside the first for as long as the copy runs. `Pixmap::take` hands over the pixel vector, and `bytemuck` reinterprets its four one-byte channels as bytes in place: the same bytes, in the same buffer. On a 6182×8000 px page, peak RSS goes from about 425 MB to 246 MB and render time from 102 ms to 81 ms. Pixels are identical: every page of #260's test file, at two scales and as a tile, hashes the same before and after. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Chen TaoandClaude Opus 5.5
|
5f570eab01 |
M2.6: non-embedded Mincho fonts draw half-width katakana (#484)
Since #368, a non-embedded Adobe-Japan1 font with a Mincho name (HeiseiMin-W3, MS-Mincho, ...) is drawn in document_japanese_font(), i.e. Shippori Mincho. Shippori Mincho has none of the 63 half-width katakana (U+FF61-U+FF9F), which are common in Japanese documents. hayro maps the CID through Adobe-Japan1-UCS2 to Unicode, finds no glyph in the face's cmap, and falls back to drawing the CID as a glyph index: half-width katakana came out as unrelated glyphs (ligatures such as fi and fl). The Mincho substitute is now BIZ UDMincho Regular, which covers half-width katakana (and was designed as a metric-compatible replacement for MS Mincho), falling back to document_japanese_font() when it isn't built in. The face used for Japanese text written into PDFs is unchanged. Without craft-fonts nothing changes. Tests: raster::tests::mincho_substitute_covers_half_width_katakana (with CRAFT_FONTS_DIR: the Mincho substitute's cmap maps U+FF61-U+FF9F and 日本; it lacked all 63 half-width katakana before; skips without craft-fonts). skrifa 0.44 is a dev-dependency of pdfcraft-render for it, the version pdfcraft-fonts already uses. The parity entry view.render-cid-fonts cites the test. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3b8a274f52 |
M2.6: non-embedded Japanese CID fonts draw in a craft-fonts face (#368)
#260 (page 2): text in a Japanese CID font that the PDF doesn't embed (HeiseiMin-W3 with UniJIS-UCS2-H, Adobe-Japan1) drew nothing, while MuPDF, Poppler, PDFium and Acrobat draw it in a Mincho face. hayro asks its font resolver for a substitute with the CID collection in the query, but the renderer never installed one, so hayro's default answered with a Latin standard-14 font that has no Japanese glyphs. Release builds have embedded Japanese faces from craft-fonts all along; the renderer just didn't use them. RenderConfig::settings now installs a resolver: for an Adobe-Japan1 fallback query it returns a craft-fonts face (Shippori Mincho for Mincho names, BIZ UDPGothic Regular or Bold for Gothic names, the descriptor's serif flag otherwise); every other query keeps hayro's default. hayro already maps such fonts through the Adobe-Japan1-UCS2 CMap to Unicode and the face's cmap. Without craft-fonts nothing changes. Other CJK collections are untouched: the pinned craft-fonts has only Japanese faces, and its later Chinese face is Noto CJK, which AGENTS.md §1.1 rules out. pdfcraft-render now depends on pdfcraft-fonts (L3 to L2). Tests: raster::tests::non_embedded_japanese_cid_fonts_draw_with_a_craft_fonts_face (with CRAFT_FONTS_DIR: 日本 in HeiseiMin-W3 and HeiseiKakuGo-W5 now draws; it drew 0 dark pixels before; skips without craft-fonts, as the Linux CI job has them) and raster::tests::japanese_font_names_pick_mincho_or_gothic. The parity entry view.render-cid-fonts cites the test and notes what is still missing. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Brandon Thomas <echelon@gmail.com> |
||
|
Brandon ThomasandClaude Opus 5.5
|
d3db99818f |
Rename PrintCraft to PdfCraft (#174)
The app, crates, binaries, bundle id, packaging, icons, screenshots and docs are now PdfCraft / pdfcraft (repo storytold/pdfcraft). On first launch the desktop app moves the PrintCraft settings and crash-recovery folders to the new name, and both apps read settings saved under the old storage key, so upgrades keep recent files, preferences and unsaved work. The signing test fixtures keep their PrintCraft subject names (they are binary). Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brandon Thomas
|
108fdc108c | more progress, roadmap, wip | ||
|
Brandon Thomas
|
f0b5f8ff55 | one-shot first ver |