Storing LZW streams with Flate decoded image streams outside the allowance,
with a limit for each stream and no total, and nothing asked whether to go
on: a 593 KB file of 100 images took 113 s with none refused, and images
refused by the image pass were still rewritten. The pass now pays for what
it decodes and for storing and checking it, at 8 steps a byte since Flate
takes up to 78 ns a byte, and streams past that stay LZW.
An image is counted as refused only when it stayed as it was: one packed
before the allowance ran out on its second read was rewritten. doc_reduce
reports images_refused as doc_optimize does, and the app's action says so
in its log. A refused image no longer reads as an unsupported filter:
FilterError::Spent says what happened.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From a review of the spellings an indexed image can take:
- The palette was read for every 1-bit indexed image, resampled or not, and twice each. Images
share a table, so 1,000 images sharing one with 16 MiB of raw data took 16 to 60 s, none of
them resampled. It is now read only for an image about to be resampled, and a table over
4 KiB is refused before it is copied or decoded, since no palette needs that.
- A colour key can hide the dark sample a tie goes to, so a thin rule of the other was lost: a
white rule on a hidden black went from 255 to the backdrop's 128. An image with a colour key
now keeps its resolution. Grey had this since #5426 and is covered too.
- `/Indexed` by reference, and a base of `[/DeviceRGB]`, are the same colour space and are read
so. `/I`, an inline abbreviation, and a real hival aren't, and such an image is left alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
From CodeRabbit's review: default colour spaces apply to an indexed colour space's
device base, so one in force where an annotation's appearance (or a pattern, or
another image's /Alternates) draws the image may change which palette entry is darker,
and the tie a resampled image keeps. Only black-and-white RGB went through
used_elsewhere; indexed images now do too when downsampling is on, and one used beyond
the page and form resources the walk read keeps its resolution, as such RGB keeps its
colour space.
A grey image stays out of this check, since sample 0 stays dark in a grey space, and #5426's rule
for grey is unchanged: the test now has a grey image drawn on the page and in an annotation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AmsdeUCdTaD72jkDw2pJKY
#5461 made Document::decoded_image read JBIG2, page exactly the image's size, and
kept the optimizer's monochrome step off it until it could weigh JBIG2 again. Now
one_bit reads JBIG2 last after any filters, within the raster's allowance, and mono
treats it as it does any black-and-white image:
- above the threshold it is averaged, and stored as the compression asks, the
smallest of what that may store; under Retain it is stored as JBIG2 again, its kind
of compression kept;
- not resampled, it is stored again as the compression asks when that is smaller: a
generic region becomes symbols with JBIG2 asked for, and JBIG2 behind ASCIIHex is
stored again alone (as Group 4 where that is smaller);
- a JBIG2 image stored again refers to no /JBIG2Globals; one whose page isn't its
size isn't read, and stays.
jbig2_images_are_left_as_they_are becomes jbig2_scans_are_averaged_and_stored_again:
a JBIG2 scan at 600 ppi, alone, behind ASCIIHex and with globals, averages to the
same samples as its Flate copy under each compression (Retain keeping JBIG2); at 300
ppi a generic region of rendered text stays under Group 4 (larger) and Retain and
becomes symbols under JBIG2 (under half the size); behind ASCIIHex it is stored again
alone, as Group 4 where the hex is spaced out; a taller page stays.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5426 left indexed images at their resolution rather than guess which sample is
dark. The palette says: dark_is_one now maps samples 0 and 1 through the decode array
to palette entries (rounded and clipped to hival, ISO 32000-2 §8.9.5.2) and compares
how light those entries are (`lightness`: the grey, or Rec. 601 luma over DeviceRGB).
Only DeviceGray and DeviceRGB bases count, the table a string or a stream (within
4 KiB), directly or by reference: there the palette's values are the colour, and
under /DefaultGray or /DefaultRGB no polarity is asked for. Other bases (CMYK,
calibrated, ICC-based, a resource name), two entries alike, an entry the table
doesn't hold or a hival past 255 still say nothing; separations and DeviceN still
aren't averaged, their tint transforms unread. #5426's tone guard applies as to any
image.
#5426's indexed thin-rule test now checks what it was for: with the palette either
way round and the decode array either way, the averaged image holds the averaged
DeviceGray one's samples (flipped where its samples are) and renders the same. A unit
test covers each base, both orders, an inverting decode array, the weights of luma,
the table by reference and in a stream, and every refusal; the older polarity test's
doc says what is known now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
decode_jbig2 sits next to decode_ccitt, through hayro-jbig2 as Export ▸ Images
used it, and shares CCITT's row sink. The JBIG2 page has to be the image's size:
PdfCraft's renderer draws the whole page into the image (black box: a 4-row page
behind /Height 2 fills the image box with all four rows), so the rows of a taller
page aren't the image's. The decoder decodes a page whole, so the page is checked
before it is decoded: that size, within the caller's bound as rows of one bit, and
at most 2^28 pixels (MAX_JBIG2_PIXELS; the decoder keeps a bit a pixel, about 34 MB
there).
Document::decoded_image_within decodes a JBIG2 image as it does a fax image: the
filters before it undone first, then the decoder with the /JBIG2Globals of the
image's own /DecodeParms entry (/Crypt counts in the position; fax_params and the
globals now share last_filter_parms). Globals have a limit of their own,
MAX_JBIG2_GLOBALS (32 MiB, moved from Export), and globals that can't be read are an
error rather than ignored.
Export ▸ Images reads JBIG2 through it, so its own decoder, row sink and DecodeParms
reader go, and hayro-jbig2 leaves pdfcraft-create. JBIG2 behind another filter now
exports, and so copies (Copy Image uses the same function); a page taller than the
image, which Export cut to the image's rows, now skips the image, as one of another
width did. doc_export_all_images' description no longer lists JBIG2 inside other
filters as skipped.
The optimizer's monochrome step read images through decoded_image_within and
relied on JBIG2 failing there: it now leaves JBIG2 images alone explicitly, as
before (a resampled one would come out Flate under Retain), and the README and
parity note list them as not yet handled.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Includes #4325#1952#4180#2529#4320#4805, preserving their original histories.
Keep paragraph alignment and hard/soft wrap metadata together; retain CCITT/downsampling/RGB ownership guards with bounded JBIG2 export and exact-copy symbol encoding.
Combined interaction regressions added. Validation pending.
Packing makes an RGB image grey for every use at once, but /DefaultGray and
/DefaultRGB were checked only where the walk reads: page content and forms.
An annotation's appearance, a pattern, a Type3 glyph or a soft mask's group
drawing the same image can have other default colour spaces, so its colours
could change there.
The walk now records, by object number, every reference in the resources it
reads (each /XObject entry, drawn or not, an /XObject dictionary under its
resources, resources under a page, page-tree node or form), reading the
resources of every page and of every node above one, each page and node a
root. An image the walk reached stays covered unless an object reachable
from the trailer refers to it, or to an object on its way from a page, other
than through those references, or it is held by one that isn't covered; an
eight-bit DeviceRGB image that isn't covered stays RGB. So resources, an
/XObject dictionary or a form a page shares with an appearance keep it RGB,
as does a reference with another generation, while LibreOffice's layout (one
resources object for a page, its /Pages node, and a form inside it that
names it again) still packs. The scan runs only when packing is on and there
is such an image.
The images drawn or listed where the defaults are in force are kept by
object number too, so one named with another generation there stays RGB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the previous commit:
- Nothing checked the segment numbers the region split assigns: a
mutant numbering every region 2 decoded. The many-marks test now
reads each segment's number, and has a bar too tall to be a symbol,
so the generic region is numbered after the last text region.
- The decoder's 32,767-symbol limit was checked against all distinct
marks, before either plan, so an image with more distinct marks lost
the second plan, which codes only the repeated ones. The limit now
applies to each plan (MAX_CODED_SYMBOLS); finding marks keeps its
memory bound of 65,536 (MAX_SYMBOLS), as before the previous commit.
A test has 32,768 distinct marks above a page of repeated text, which
encode_jbig2 still symbol codes through the second plan.
- The dense-page optimizer test now checks that it takes several text
regions, and a compile-time assert keeps segment numbers within a
byte's reach.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A review found that a dense page came out as a stream PdfCraft's own
renderer refuses, so the page drew blank after optimizing: hayro-jbig2
decodes a text region of at most 10,000 symbol instances, and refuses
32,768 symbols or more. The stream was valid (Poppler decoded it), but
every mark went into one text region and up to 65,536 symbols were kept.
A 300 dpi letter page of the optimizer test's own text crossed it.
A page with more than 10,000 marks is now placed by several text
regions, each over a band of rows and all referring to the one symbol
dictionary, which each region but the last keeps retained (7.2.4). Past
32,767 distinct symbols the image is coded generically.
New tests: 25,000 copies placed by three regions, which decode; the
symbol caps decoded, not just reached; and in the optimizer, a whole
letter page of rendered text that looks the same as JBIG2 symbols (it
drew blank before). The READMEs say "still looked at" for the caps, and
that the scans figure is the six pages' total (20% on one page).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
pdfcraft_filters::encode_jbig2 stores each distinct mark of a 1-bit
image (8-connected black pixels, at most 256 a side) once, as a symbol
in a symbol dictionary, and places its exact copies with an immediate
lossless text region that refers to it. Larger marks, and in a second
plan marks that occur once, go to a generic region over their box. It
keeps whichever of that and the generic coding (encode_jbig2_generic,
unchanged) is smaller. Only exact copies share a symbol, so the stream
decodes to the same bits.
Written from the JBIG committee's ISO/IEC 14492 Final Committee Draft:
6.4 and 6.5 (text region and symbol dictionary decoding), 7.2, 7.4.2
and 7.4.3 (segment headers and syntax) and Annex A (the integer coders,
checked against its worked example and Table A.1).
On six pages of rendered 300 dpi text the output is 28% of the generic
coding's size and 17% of Group 4's, at about 100 ms a page; hayro-jbig2
and Poppler (pdfimages) both decode it to the original bits. Scans,
whose copies of a letter are seldom identical, gain little: that needs
refinement coding. A dictionary shared between pages (/JBIG2Globals)
is a further step.
The optimizer's Monochrome Images > JBIG2 (lossless) and doc_optimize
mono jbig2 use it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
group_3_fax_images_become_group_4 coded its rows only off bytes. It now
also stores again rows on bytes: Modified Huffman (no end-of-line codes),
and codes whose fill ends each on a byte, as TIFF writes them, at /K 0 and
1 without /EndOfLine, which reading by that flag misread (the test fails
with that rule). Each becomes Group 4, smaller, decoding to the picture.
The page is compared with the picture in Flate rather than with itself
before: PdfCraft's bootstrap renderer draws TIFF's layout (the fill
ending each code on a byte) as other rows, at every /K and with
/EndOfLine or not (black box), so storing it as Group 4 corrects what
PdfCraft shows rather than keeping it. (Redaction removes such images
whole, for that reason, since #4384's "redaction removes Group 3 on
bytes whole".)
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Packing compared an indexed image's decode array, and RGB's pairs, as
written: [0 255.0], an entry by reference or a real among RGB's pairs
stopped it, though they mean the same as integers. Each entry is
resolved and read as a number now, as is_identity_decode and
dark_is_one read them; RGB's pair is written back direct.
- A test holds that Retain stores nothing again it doesn't resample: a
Group 3 halftone above 450 ppi, which isn't averaged down, stays Group
3 byte for byte (the guard's || as && passed every test).
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Mutation testing found three behaviours no test held:
- eight-bit black-and-white in calibrated grey, an ICC profile of one
component or a separation is packed, each keeping its colour space,
and calibrated RGB of only 0 and 1 isn't (taking every array colour
space for an indexed one passed every test);
- mixed Group 3 (/K 1, with end-of-line codes) is stored again as
Group 4 (taking /K above 0 for Group 4 passed);
- a resampled Group 4 image of vertical stripes is stored as 1-bit Flate,
which is smaller (ruling Flate out for any CCITT image passed).
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Packed to one bit, black-and-white DeviceRGB became DeviceGray. Where
/DefaultGray or /DefaultRGB is in force (ISO 32000-2 §8.6.5.6), that
drew it otherwise: through DefaultGray as grey, or no longer through
DefaultRGB. The page walk now notes the images drawn, on any use, where
a page's or form's resources (an enclosing one's included) name
either, and those stay RGB. When a use couldn't be followed, every
image counts as such.
- One whose soft mask has a /Matte stays RGB too: the matte has one
number for each of the parent's components, three, and grey has one.
- The optimize README, the mono-compression parity note and doc_reduce's
description said Group 3 to Group 4 wasn't done yet, that only images
packed from eight bits could become Flate, and that the colour space
and decode array always stay. They now match the code.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Monochrome Images only took eight-bit grey of only black and white
besides 1-bit images, so the same scan written as RGB, or as an indexed
image of two palette entries, stayed at eight bits (and RGB went to the
colour settings, as JPEG). Now these are packed to one bit first:
- eight-bit DeviceRGB whose every pixel is black or white becomes
DeviceGray; a decode array the same for each component becomes that
pair, another stops it. Calibrated RGB and RGB with a profile stay.
- an indexed image of 2, 4 or 8 bits that uses only palette entries 0
and 1 keeps its palette, whatever the two colours; a decode array
other than the default stops it.
Colour-key masks still stop all of them, and Retain leaves them to the
grey and colour settings. They are decoded through /DecodeParms given
by reference too (Document::with_direct_parms), and when downsampled a
tie goes to the dark sample of the image as packed: packed RGB is grey,
with grey's decode pair.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
A CCITT fax image that wasn't resampled was left as it was, so Group 3
images (the 1-D and 2-D codings fax software writes) stayed Group 3.
Group 4 codes each row against the one above and is usually much
smaller. Now only Group 4 alone skips decoding; Group 3 is stored again
as Group 4 when that is smaller as stored, and Retain still keeps it.
It reads an image's CCITT parameters (its /DecodeParms entry with Table
11's defaults) through pdfcraft_cos::Document::fax_params, from #4384.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
Mutation testing found behaviour no test held, now covered:
- a Group 4 halftone above 450 ppi isn't resampled and stays byte for
byte, alone by default and behind ASCIIHexDecode with Retain;
- under Retain a resampled Flate scan stays Flate, though Group 4 would
be smaller (ragged bands);
- sixteen-bit grey of only 0x0000 and 0xFFFF keeps its bytes;
- a busy Group 4 image behind ASCIIHexDecode, whose code is over three
times its raster, decodes within the fax bound and is stored again as
Group 4 alone. Redaction's busy-fax test is that busy again: since the
bound for other images is three times the raster, the old checkerboard
fitted it, and dropping the fax bound passed.
The redaction README says three times, and doc_optimize's description
says what Retain keeps: CCITT stays CCITT, anything else becomes Flate.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Halftones and dithers: most of their new pixels cover both black and
white, and taking the sample that covers more turned their tones
solid (a 50% screen black, a 25% one white). An image whose new pixels
mostly cover both is now left at its resolution; text covers both only
along its edges and is still averaged down. downsample_bits says how
many new pixels cover both.
- A counter for each column: an image wider than a fax row
(MAX_CCITT_COLUMNS) isn't averaged, so a strip of 80 million columns
can't make the counters 640 MB.
- A fax image's decoding is bounded at about a byte a pixel, as
redaction's is (decoded_image_within), not 1 GiB; behind a Flate bomb
it is left alone.
- An indexed image's tie went to sample 0, which an inverting decode
array maps to entry 1, though the docs say entry 0. It goes to the
sample that decodes to entry 0 now (ISO 32000-2 §8.9.5.2).
Each has a test that fails without it.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A 2 × 2 tie always gave sample 0, black in grey and the painted part of
an image mask with the default decode array. With an inverting one,
[1 0], 0 is white or unpainted, and in a separation 0 is no ink, so a
line half a new pixel wide could vanish. dark_is_one says which sample
is dark from the colour space and decode array: 0 in grey and of a
mask, 1 in a separation or a DeviceN of one colourant, each the other
way round when the decode array inverts; an indexed image keeps entry
0, as its palette, not the decode array, says which is darker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
The PDF Optimizer and Reduce File Size downsampled colour and grey images
but not black-and-white ones. Settings::mono is now MonoSettings {
downsample, target_ppi, above_ppi, compression }, 300 ppi above 450 ppi
and CCITT Group 4 by default (Monochrome Images ▸ Downsample to;
doc_optimize mono: { downsample, ppi, above_ppi, compression }).
A 1-bit image or image mask (no filter, general-purpose filters, or CCITT
fax after any of them) and eight-bit grey of only black and white are averaged by
area: each new pixel takes the sample covering more of it, a tie 0, so a
dark line half a new pixel wide stays. They are stored as Group 4, or as
1-bit Flate when that is smaller; with Retain a resampled image keeps its
kind of compression. Images with a soft mask keep their size.
CCITT decoding moves from Export ▸ Images into pdfcraft-filters
(decode_ccitt, CcittParams, through hayro-ccitt), and
pdfcraft_cos::Document::decoded_image reads a fax image's /DecodeParms
for it (undoing filters before CCITT, such as ASCII85Decode, with any
/DecodeParms given by reference made direct first), so export
and the optimizer share one reader, and Group 4 scans can be
downsampled. A 600 ppi text page went from 79 KB to 40 KB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
- A noisy 640 × 800 soft mask under 1200 × 1500 text: resampled to the
image's new size it shrinks only a little, by less than the grey
settings' JPEG adds over the packed image, so packing wins; not
counting the new mask passed every test before.
- The hex-text half of the bomb test also stores eight-bit
black-and-white, which must pack, as its comment says.
- The optimize README, the mono-compression parity note and both
progress logs say the soft mask counts.
- The Copy Image parity note no longer lists CCITT as missing: it
copies through Export's decoder, which reads CCITT now.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
- Eight-bit black-and-white text under a noisy soft mask: the grey
settings win, as they shrink the mask far more than packing saves.
Counting the packed image without its mask passed every test before.
- The packing test checks the report's byte totals exactly.
- Export reads Group 3 with end-of-line codes, with rows on a byte
(EncodedByteAlign), with both, and mixed (/K 1, each row tagged
one-dimensional) with end-of-line codes and without. Decoding /K 1 as
one-dimensional, or ignoring EncodedByteAlign, now fails a test.
/EndOfLine is passed to the decoder, but no input here tells it apart:
the decoder accepts end-of-line codes either way, as ISO 32000-2
(Table 11) requires.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
An eight-bit grey image of only black and white was packed to one bit
and stored, and the grey settings never saw it. For a dithered scan that
is bigger than what Reduce File Size made of it before, downsampled to
JPEG: 1-bit Flate can't shrink the dots (a 1200 × 1500 dither at 300 ppi
was 157 KB packed, 67 KB as before). Both are made now, and the smaller
as stored wins, counting a soft mask resampled with the image. Clean
text still packs to one bit. The packing also decodes no further than
its samples can need, as the other image steps do now.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Packing eight-bit black-and-white grey to one bit decoded with
Stream::decoded, which ignores /DecodeParms given by reference. With PNG
predictor 10 each row then kept its tag byte, a 0, so the shifted rows
still passed as black and white and were packed and stored skewed. It
now decodes through Document::with_direct_parms (#4304), and the test
of images with their parameters by reference covers this case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
An eight-bit grey image whose samples are all 0 or 255 is packed to one
bit per pixel and then stored as Monochrome Images asks (Group 4, or
JBIG2 with #4320), when that is smaller as stored; its decode array
stays. Indexed images, a colour-key /Mask and Retain are left alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
When the optimizer stores an image as JBIG2 (PDF 1.4), it raises the
catalog's /Version for an older file. It read /Version only when it was
direct: a catalog saying 1.7 by reference looked like no version at all,
and was set to /1.4, lowering the document. It is now resolved first.
Found in pre-push review.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
pdfcraft_filters::encode_jbig2_generic writes 1-bit rows as a JBIG2
stream for JBIG2Decode (embedded organisation: a page information
segment and an immediate lossless generic region, no globals): template
0 with its nominal AT pixels, typical prediction, and the MQ arithmetic
coder (mq.rs), written from the JBIG committee's ISO/IEC 14492 Final
Committee Draft. The coder reproduces its Annex H.2 test sequence byte
for byte; output round-trips through hayro-jbig2, and Poppler renders
it the same as the original.
The optimizer's Monochrome Images gets JBIG2 (lossless) beside CCITT
Group 4 (still the default), doc_optimize mono jbig2; a document below
PDF 1.4 gets /Version /1.4 in its catalog. On 300 dpi text JBIG2 is
about a third smaller than Group 4.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
The optimizer's images decode no further than twice their samples plus
a byte a row, meant for ASCIIHex text under Flate; but hex text broken
into lines, as it usually is, is a little more than twice, so such an
image was left alone as if it were a decompression bomb (a 300 dpi
letter scan stayed Flate). It is three bytes for each now, a separator
for every two digits, and a test stores both images as hex text in
lines of 64 digits.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A test now covers what qualifies and what doesn't: an ICC profile of
one component and a DeviceN of one colourant convert, three components
and two colourants (more than a bit a pixel) don't; an image mask
converts with /BitsPerComponent 1 given, not with 8; an image of exactly
80 million pixels converts, one row more doesn't. The byte totals are
checked exactly. Each of those rules could be broken before with no
test failing (cargo mutants). one_component's comment now says /G
counts with DeviceGray, as the code does.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
A 1-bit image was decoded for Group 4 up to MAX_DECODED (1 GiB), so a
few kilobytes of Flate could cost a gigabyte, and the same held for the
samples of a colour or grey image read to resample it (pixels). Both now
stop at twice the samples plus a byte a row and 4 KiB, as redaction
does; data past that can only be a decompression bomb, and the image is
left alone.
From reviews with the pdfcraft-review skill.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Document::with_direct_parms resolved an indirect /DecodeParms and its
array entries but not the numbers inside (/Predictor 9 0 R), which
Dict::int doesn't follow. A 1-bit image with such a predictor was
decoded without it, its rows shifted by their tag bytes, and Group 4 of
that stored in its place; resampling read images the same way. It now
keeps, of each parameters dictionary, the numbers and flags the decoders
read, resolved, and /JBIG2Globals as a reference. Only those small
values are copied (an entry that is no dictionary is null), so a file
naming one big object from every key and filter can't have it copied a
hundred times.
The change is #4384's, moved down so that every PR stacked on this one
reads such parameters.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P6yUtvWGPsJpMLyh6pxSXG
Stream::decoded reads only direct parameter dictionaries, so an image
whose /DecodeParms is a reference (whole, or as array entries) was
decoded without its predictor. For a 1-bit image with a PNG predictor
the rows came out shifted by their tag bytes, Group 4 of that was still
smaller, and the picture was replaced by a skewed one; resampling and
merging image strips read the same way.
pdfcraft_cos::Document::with_direct_parms gives the stream with the
parameters its filters use made direct: only the entries at the
filters' places, and none for a chain longer than any decoder takes,
so at most MAX_CHAIN objects are copied. The optimizer's pixels() and
the Group 4 step decode through it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
A bare /Indexed, /I or /CalGray is no colour space on its own, so it can
only name a resource, which might be RGB: such an image is now left
alone instead of being read as one bit per pixel. doc_optimize refuses a
mono that isn't an object or a compression that isn't a known name,
instead of reading it as the default (from CodeRabbit's review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
pdfcraft_filters::encode_g4 encodes packed 1-bit rows as Group 4 (ITU-T
T.6, /K -1) through the fax crate (MIT, already in the tree via tiff),
with rows capped at 32,767 samples so its 16-bit positions can't
overflow; round trips are checked against hayro-ccitt.
The optimizer (Settings::mono, CCITT G4 by default, so Reduce File Size
too) re-encodes a 1-bit image of one colour component, or an image
mask, stored with no filter or general-purpose ones, when that is
smaller as stored. The samples decode to the same bits, so colour
space, decode array and masks stay. PDF Optimizer gets a Monochrome
Images row and doc_optimize a mono option. optimize.mono-compression
goes from planned to partial (JBIG2 and monochrome downsampling not
yet).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BfMGHQNgdq5yf5tKZHGR3z
Clean Up removes named destinations that nothing in the document refers
to. referenced_names counted the /D of GoTo, GoToR and GoToE actions
alike. The /D of a GoToR or GoToE action names a destination in another
document, so a local destination with the same name was kept although
nothing in the document used it.
Only a GoTo's /D now counts as a use. A /Dest key and a GoTo's /D are
read as before.
Test: links with a GoTo, a GoToR and a GoToE action name three
destinations; optimize() keeps the one the GoTo uses and removes the
other two.
ROADMAP.md: kept both progress lines.
crates/optimize/README.md: PR's decoder sentence plus the target-version section from main.
Cargo.lock: auto-merged.
Conflicts: ATTRIBUTION.toml: took main's side (hashes recomputed after the merges). crates/optimize/src/lib.rs, crates/automation/src/lib.rs, tools.rs: Settings and tool schema keep object_streams (#3227) and lzw_to_flate (#4181); doc_optimize description names both options. crates/optimize/tests/optimize.rs: both sides' imports and tests kept.
Conflicts: docs/roadmap.md: all status lines kept, M10 row from main, M11 row with #3227's two sentences. parity/acrobat-features.toml: save-with-object-streams entry from #3227 with its test paths under tests/it/.
