mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-25 02:30:15 -06:00
Claude
f6f5e6f1b7
The import probe of a non-raw image decoded it (at a 64 px fit, which for
JPEG still entropy-decodes every block and for PNG inflates everything)
only to learn its stored size and orientation: ~95 % of import CPU.
lightcraft-codecs gets `read_header(bytes) -> Header { format, width,
height, orientation }`: JPEG from its markers (EXIF orientation), PNG from
its chunks (IHDR, eXIf anywhere), TIFF from the first IFD (tiff crate,
no strips read), WebP from its RIFF chunks / bitstream header (image-webp),
PSD from its header and image resources. GIF, BMP and JPEG XL keep the
small decode. `files::probe_bytes` uses it; raw handling is unchanged.
The TIFF colour-layout check is shared with the decoder (`layout`).
Measured (release, 4 cores, throwaway benchmark, per file, bytes in
memory): 200 generated 4000x3000 JPEGs (3.6 MB): 72.7 ms -> 2.1 ms
(read_header alone 1.05 ms; the rest is the content hash and metadata);
20 PNGs (30.8 MB): 236-265 ms -> 9.0-9.6 ms.
Corrupt files (behaviour note): the header readers also refuse what a
decode refuses before the pixels (unsupported layout, compression, frame
type, bad IHDR, ...) and truncated files: a JPEG whose markers don't reach
its EOI, a PNG cut before the chunk after its IDAT run, TIFF strips/tiles
that start past the end, a WebP image chunk or PSD merged data past the
end. So truncated files (interrupted copies, partial downloads) are still
marked `error` in the import review, as before; tests check that whenever
the old decode accepted a (cut or byte-mutated) file the header agrees.
Changed: damage *inside* compressed data that is all present (flipped or
garbled bytes) was refused at scan and is now imported, and shows as
unreadable ("!" in the grid) when rendered. Also: tiny JPEGs (< 128 px)
were decoded leniently by the old probe, so a truncated one imported; it
is now refused. A TIFF truncated inside its last strip is not detected
(StripByteCounts are often overstated, so only strip starts are checked).
Tests: header readers vs decode on generated JPEG/PNG/TIFF/WebP/PSD/BMP
with and without EXIF/TIFF orientation; truncations and random mutations
agree with decode; hostile headers (zero/absurd sizes, bad lengths,
truncation, garbage) are errors; read_header added to the fuzz tests;
probe_bytes oriented dims match the decode-based values.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01912WaK6Rmew3wxZuPyX7E8
116 lines
7.2 KiB
Markdown
116 lines
7.2 KiB
Markdown
# lightcraft-codecs (L1)
|
||
|
||
Standard image formats for LightCraft: format sniffing, decoding to **linear light** with colour
|
||
management, fast thumbnails, and encoders with metadata embedding. Pure Rust, no C dependencies,
|
||
builds for `wasm32-unknown-unknown`.
|
||
|
||
## API at a glance
|
||
|
||
```rust
|
||
let fmt: Option<Format> = sniff(&bytes); // JPEG/PNG/TIFF/WebP/AVIF/HEIF/JXL/GIF/BMP/PSD/raw
|
||
let d: Decoded = decode(&bytes, DecodeOptions::default())?;
|
||
// d.image: Rgb32f, linear light in the source primaries (d.space: SourceSpace)
|
||
// d.alpha, d.bit_depth, d.float, d.orientation (EXIF value, NOT applied), d.exif/xmp/icc (raw blobs)
|
||
let working: Rgb32f = to_working(&d); // linear Rec.2020 D65 (Bradford)
|
||
let fast = decode(&bytes, DecodeOptions::fit(2048, 2048))?; // JPEG: DCT-domain 1/2..1/8 scaling
|
||
let t: Thumbnail = decode_thumbnail(&bytes, 256)?; // embedded EXIF/MPF preview or scaled decode
|
||
let h: Header = read_header(&bytes)?; // stored width/height + orientation, no pixels decoded
|
||
|
||
let icc = icc::write_named(NamedSpace::DisplayP3); // or icc::write_matrix_trc(&space, &Trc::…)
|
||
let meta = EncodeMeta { icc: Some(&icc), exif: Some(&exif), xmp: Some(&xmp) };
|
||
encode_jpeg(&EncodeImage::rgba8(&img), 90, ChromaSubsampling::S444, &meta)?;
|
||
encode_png(&EncodeImage::new(w, h, 4, Samples::U16(&px)), &meta)?;
|
||
encode_tiff(&EncodeImage::new(w, h, 3, Samples::F32(&px)), TiffCompression::Deflate, &meta)?;
|
||
encode_webp_lossless(&img, &meta)?;
|
||
encode_avif(&img, 70, 6, &meta)?; // native + feature `avif`
|
||
```
|
||
|
||
Thumbnail fallback decoding limits the source to 64 million pixels by default.
|
||
Non-JPEG decoders allocate at source resolution before resizing, even for a tiny
|
||
thumbnail. Use `decode_thumbnail_with` and `ThumbnailOptions::max_pixels` to set
|
||
a different source budget. Oversized sources return `Error::TooLarge`; embedded
|
||
JPEG previews can still be used without decoding the full source image.
|
||
|
||
`Format::RawTiffLike` (DNG, CR2, NEF, ARW, PEF, ORF, RW2, SRW, …) and `Format::RawOther` (CR3, RAF,
|
||
CRW, MRW, X3F) are detected so the engine can route them to `lightcraft-raw`; `decode` returns
|
||
`Error::Unsupported` for them.
|
||
|
||
## Formats
|
||
|
||
| Format | Decode | Encode | Notes |
|
||
|---|---|---|---|
|
||
| JPEG | yes (zune-jpeg; jpeg-decoder for DCT scaling, CMYK/YCCK, 12-bit, non-interleaved scans) | yes (jpeg-encoder, baseline, 4:4:4/4:2:2/4:2:0) | EXIF (APP1), XMP (APP1, ≤ 64 KiB), ICC (APP2, multi-chunk), Adobe APP14, MPF previews |
|
||
| PNG | yes, 1–16-bit, palette, tRNS, interlaced | 8/16-bit, gray/GA/RGB/RGBA | iCCP, eXIf, iTXt XMP; sRGB/cICP/cHRM/gAMA honoured when no ICC |
|
||
| TIFF | 8/16-bit int (32/64-bit int kept to 16 bits), 16/32/64-bit float, gray/RGB/CMYK/palette, alpha (assoc./unassoc.), planar, LZW/Deflate/PackBits/JPEG | 8/16-bit, 32-bit float; none/LZW/Deflate/PackBits | ICC (34675), XMP (700), Orientation; first IFD only; EXIF not written (use `lightcraft-tiff`) |
|
||
| WebP | lossy + lossless, alpha, first frame of animations | lossless only | ICCP/EXIF/XMP chunks |
|
||
| GIF / BMP | yes (first GIF frame) | — | via `image` |
|
||
| PSD / PSB | merged composite: 8/16/32-bit gray, RGB, CMYK, indexed, duotone (as gray); raw/RLE | — | ICC (1039), EXIF (1058), XMP (1060); ZIP-compressed composite, Lab and 1-bit unsupported |
|
||
| JPEG XL | yes (jxl-oxide, feature `jxl`, default on) | — | enum colour → rendered straight to linear Rec.2020; ICC → our ICC path; orientation applied by the decoder (reported as 1) |
|
||
| AVIF | **no** | yes (ravif/rav1e, native only, feature `avif`) | 8-bit sRGB, EXIF; no ICC in the muxer |
|
||
| HEIC/HEIF | **no** (sniff only) | — | see gaps |
|
||
|
||
## Colour
|
||
|
||
Decoders never guess silently; `SourceSpace::origin` says where the interpretation came from:
|
||
`IccMatrixTrc` (profile applied exactly: TRC linearization + colorants → XYZ D50), `IccCms` (LUT-based
|
||
or CMYK profile converted by the CMS straight to linear Rec.2020), `Container` (PNG chunks, JXL enum,
|
||
EXIF `R03` Adobe RGB hint), `Untagged` (assumed sRGB; untagged float data assumed linear),
|
||
`IccUnsupported` (profile present but unusable → **sRGB fallback**), `Naive` (CMYK without profile).
|
||
|
||
`decode_jpeg_with_fallback(bytes, opts, space)` lets an enclosing RAW container supply the colour
|
||
space of a JPEG with no ICC or EXIF metadata. It applies the transfer curve before linear-light
|
||
resizing; a JPEG carrying its own metadata retains its existing interpretation.
|
||
|
||
ICC profiles are parsed with `moxcms` (v2/v4, `curv`/`para` TRCs, `chad`); sRGB, Display P3,
|
||
Adobe RGB (1998), ProPhoto (ROMM) and Rec.2020 are recognised by colorants. `icc::write_matrix_trc`
|
||
emits v4 display profiles (D50 PCS, Bradford `chad`) for export embedding.
|
||
|
||
## Dependencies and licences
|
||
|
||
All pure Rust, verified with `cargo info` / `cargo tree`:
|
||
|
||
| Crate | Licence | Use |
|
||
|---|---|---|
|
||
| zune-jpeg, zune-core | MIT OR Apache-2.0 OR Zlib | JPEG decode |
|
||
| jpeg-decoder | MIT OR Apache-2.0 | scaled / CMYK JPEG decode |
|
||
| jpeg-encoder | (MIT OR Apache-2.0) AND IJG | JPEG encode (IJG: permissive, attribution in docs) |
|
||
| png | MIT OR Apache-2.0 | PNG |
|
||
| tiff | MIT | TIFF |
|
||
| image-webp | MIT OR Apache-2.0 | WebP |
|
||
| image (gif, bmp only) | MIT OR Apache-2.0 | GIF/BMP |
|
||
| moxcms | BSD-3-Clause OR Apache-2.0 | ICC parse/write, CMS |
|
||
| jxl-oxide | MIT OR Apache-2.0 | JPEG XL decode |
|
||
| ravif (+ rav1e, avif-serialize) | BSD-3-Clause / BSD-2-Clause | AVIF encode (native only; heavy to compile) |
|
||
|
||
Deliberately **not** used: anything AGPL from imazen (heic, zenjpeg, jxl-encoder, rav1d-safe),
|
||
jpegxl-rs / imagequant (GPL / C), dav1d / libheif (C).
|
||
|
||
## Performance (Apple M-series, release, synthetic 24 MP JPEG q90 4:2:0)
|
||
|
||
Run `cargo test --release -p lightcraft-codecs --test bench -- --ignored --nocapture`.
|
||
|
||
| Operation | Time |
|
||
|---|---|
|
||
| full decode → linear `Rgb32f` | ~92 ms (≈ 260 MP/s) |
|
||
| decode fit 2048 (DCT 1/2) | ~81 ms |
|
||
| thumbnail 256, scaled decode (DCT 1/8) | ~54 ms (entropy decoding bound) |
|
||
| thumbnail 256 from embedded EXIF/MPF preview | ~0.6 ms |
|
||
| `to_working` matrix | ~23 ms |
|
||
| PNG / TIFF-deflate encode (fast levels) | ~430 ms / ~900 ms |
|
||
|
||
## Gaps / limitations
|
||
|
||
- **AVIF and HEIC decode**: no permissively licensed, pure-Rust AV1/HEVC decoder exists today
|
||
(dav1d/libheif are C; rav1d-safe and imazen's heic are AGPL). Files are sniffed and rejected with
|
||
`Error::Unsupported`; a raw-style embedded-preview path could be added for HEIC thumbnails.
|
||
- JPEG: arithmetic coding unsupported (as in both decoders); extended XMP (> 64 KiB) neither read
|
||
nor written; EXIF > 64 KiB not written. The encoder uses standard Huffman tables (interleaved
|
||
baseline) for maximum compatibility.
|
||
- Thumbnails under 15 ms need an embedded preview ≥ the requested edge (EXIF IFD1 or MPF). Most
|
||
cameras store a 160 px EXIF thumbnail: use `decode_thumbnail_with` with `min_embedded_edge: 160`
|
||
for an instant placeholder, otherwise the DCT-scaled path takes ~50 ms for 24 MP.
|
||
- TIFF: first image only; EXIF is not re-emitted on encode; Lab/uncompressed-YCbCr unsupported.
|
||
- PSD: only the merged composite (no layers); ZIP-compressed composites unsupported.
|
||
- Panics inside third-party decoders are caught (`catch_unwind`) on unwinding targets; on wasm32
|
||
(`panic = abort`) we rely on the property tests of our own code plus the decoders' own hardening.
|