Export for Screens: a ZIP export deflates its files

With `zip: true`, document.exportForScreens deflated nothing: every file
went into the archive stored. Each file is now deflated, or stored when
deflating does not make it smaller, as file.package's archive does. The
file list, the names and the reply's fields stay the same; `bytes` and
the archive's contents change. Entries keep the fixed 1980-01-01 date, so
the same files still give the same archive.

The deflating writer moves from package.rs into fileio/zip.rs as
`zip::deflate`, beside `zip::store`, and file.package calls it there with
its own date and error text. The test reader `zip::entries` reads
deflated entries too. The `zip` option's text in docs/mcp.md,
docs/control-protocol.md and the command's params says the files are
deflated.
This commit is contained in:
dotCooCoo
2026-10-29 10:03:27 +07:00
committed by bflatastic
parent 71a67601a5
commit 3b01769d46
7 changed files with 141 additions and 62 deletions
+1 -1
View File
@@ -54,7 +54,7 @@ pub fn specs() -> Vec<CommandSpec> {
"Export for Screens",
["File", "Export"],
None,
"{folder?, zip?: false (one store-only .zip of every file: no folder → {name, dataBase64, bytes, files: [name…]}; with a folder it is written there → {path, bytes, files}), artboards?: [index…] | range?: \"1-3\" (default all), assets?: [asset id…] (instead of artboards: each asset's art alone, cropped to its bounds and named after the asset; see assets.list), layers?: [layer id…] (instead of artboards: each layer's art alone, sublayers included, named after the layer), selection?: one|each (instead of artboards: the selected art as one file, or a file per selected object, named after it), fullDocument?: false (one file per format instead: a PDF of every artboard, other formats the bounds of all visible art, named after the document), includeBleed?: false (artboards grown by the document's bleed), exportPreset?: (a preset of document.exportPresets.list: its settings, under these params), subfolders?: false (each row's files in a sub-folder: its folder, else its size for raster (1x, 2x, 100w…) or its format (SVG, PDF)), folders?: none|rows (as subfolders)|format (a folder per extension: png, svg, pdf…)|item (a folder per artboard, asset, layer or object)|itemFormat (item/extension) (default: rows with subfolders or a preset, else none), name?: (a file name pattern instead of the item's name, with <filename> for the document, <name> for the artboard, asset, layer or object, <index> from 1, #### for the counter zero-padded to the run of #, <format>, <scale>, <suffix> (else the row's suffix ends the name), YYYYMMDD, YYYY-MM-DD, YYYY, <date>, <time>, <YYYY>, <MM>, <DD>, <hh>, <mm> and <ss>; a trailing .ext is dropped, as in `COMPANY-Logo-<filename>-####-YYYYMMDD`; different files it names alike get -2, -3…), counterStart?: 1, counterPer?: item (every file of an item has its number)|file, date?: YYYY-MM-DD (the date the pattern uses; default today, UTC), preset?: mobile (PNG 1x, 2x, 3x) | density (PNG 0.75x–4x in ldpi…xxxhdpi sub-folders) instead of formats (turns subfolders on), formats?: [{format: png|png8|jpg|webp|gif|tiff|bmp|tga|psd|svg|svgz|pdf|eps|ai|vectorcraft, scale?: 1 | \"2x\" | \"100w\" | \"100h\" | \"72ppi\", width?: px | height?: px | ppi? (raster only; width, then height, then ppi win over scale), suffix?: (raster default: @2x, @100w, @100h, none at 1x; vector formats drop size suffixes), folder?: (its sub-folder), quality?: (jpg 0–100), …the format's options (antiAlias, background, preset (pdf)…)}], settings?: {png|png8|jpg|webp|gif|tiff|bmp|tga|psd|svg|svgz|pdf|eps|ai|vectorcraft: {…options for every row of that format}} (rows' own options win), prefix?, antiAlias?: none|art|type (raster rows without their own), openLocation?: (remembered; the app shows the files after exporting)} one file per artboard and format (a PDF holds its artboard alone; artboards with the same name, in any case, get -2, -3…; an unnamed one is Artboard-N) → {files: [path…]}; no folder → {files: [{name, dataBase64}]} (names include sub-folders: 2x/Icon@2x.png); every reply has warnings: [the encoders' warnings for the files, each once: an image that couldn't be decoded, type written in the fallback font…]. eps, ai and vectorcraft rows write a document of the item alone (its one artboard). The params (but zip and assets) are remembered in the document (document.exportSettings; not an undo step)",
"{folder?, zip?: false (one .zip of every file, each deflated or stored, whichever is smaller: no folder → {name, dataBase64, bytes, files: [name…]}; with a folder it is written there → {path, bytes, files}), artboards?: [index…] | range?: \"1-3\" (default all), assets?: [asset id…] (instead of artboards: each asset's art alone, cropped to its bounds and named after the asset; see assets.list), layers?: [layer id…] (instead of artboards: each layer's art alone, sublayers included, named after the layer), selection?: one|each (instead of artboards: the selected art as one file, or a file per selected object, named after it), fullDocument?: false (one file per format instead: a PDF of every artboard, other formats the bounds of all visible art, named after the document), includeBleed?: false (artboards grown by the document's bleed), exportPreset?: (a preset of document.exportPresets.list: its settings, under these params), subfolders?: false (each row's files in a sub-folder: its folder, else its size for raster (1x, 2x, 100w…) or its format (SVG, PDF)), folders?: none|rows (as subfolders)|format (a folder per extension: png, svg, pdf…)|item (a folder per artboard, asset, layer or object)|itemFormat (item/extension) (default: rows with subfolders or a preset, else none), name?: (a file name pattern instead of the item's name, with <filename> for the document, <name> for the artboard, asset, layer or object, <index> from 1, #### for the counter zero-padded to the run of #, <format>, <scale>, <suffix> (else the row's suffix ends the name), YYYYMMDD, YYYY-MM-DD, YYYY, <date>, <time>, <YYYY>, <MM>, <DD>, <hh>, <mm> and <ss>; a trailing .ext is dropped, as in `COMPANY-Logo-<filename>-####-YYYYMMDD`; different files it names alike get -2, -3…), counterStart?: 1, counterPer?: item (every file of an item has its number)|file, date?: YYYY-MM-DD (the date the pattern uses; default today, UTC), preset?: mobile (PNG 1x, 2x, 3x) | density (PNG 0.75x–4x in ldpi…xxxhdpi sub-folders) instead of formats (turns subfolders on), formats?: [{format: png|png8|jpg|webp|gif|tiff|bmp|tga|psd|svg|svgz|pdf|eps|ai|vectorcraft, scale?: 1 | \"2x\" | \"100w\" | \"100h\" | \"72ppi\", width?: px | height?: px | ppi? (raster only; width, then height, then ppi win over scale), suffix?: (raster default: @2x, @100w, @100h, none at 1x; vector formats drop size suffixes), folder?: (its sub-folder), quality?: (jpg 0–100), …the format's options (antiAlias, background, preset (pdf)…)}], settings?: {png|png8|jpg|webp|gif|tiff|bmp|tga|psd|svg|svgz|pdf|eps|ai|vectorcraft: {…options for every row of that format}} (rows' own options win), prefix?, antiAlias?: none|art|type (raster rows without their own), openLocation?: (remembered; the app shows the files after exporting)} one file per artboard and format (a PDF holds its artboard alone; artboards with the same name, in any case, get -2, -3…; an unnamed one is Artboard-N) → {files: [path…]}; no folder → {files: [{name, dataBase64}]} (names include sub-folders: 2x/Icon@2x.png); every reply has warnings: [the encoders' warnings for the files, each once: an image that couldn't be decoded, type written in the fallback font…]. eps, ai and vectorcraft rows write a document of the item alone (its one artboard). The params (but zip and assets) are remembered in the document (document.exportSettings; not an undo step)",
has_doc,
screens::export_for_screens
),
+1 -1
View File
@@ -820,7 +820,7 @@ fn encode_and_write(export: ScreenExport, writer: &mut dyn FnMut(&str, &[u8]) ->
}
let names: Vec<&str> = out.kept.iter().map(|(n, _)| n.as_str()).collect();
if o.zip {
let zip = super::zip::store(&out.kept).map_err(|e| bad(C, e))?;
let zip = super::zip::deflate(&out.kept, super::zip::FIXED_TIME).map_err(|e| bad(C, e))?;
let zip_name = safe_file_name(&format!("{}{}.zip", naming.prefix, naming.filename));
return match folder {
Some(dir) => {
@@ -634,6 +634,67 @@ fn zip_archives_store_their_files() {
assert!(super::zip::entries(&broken).is_err(), "the CRC catches a changed byte");
}
/// The compression method of each file in `zip` (0: stored, 8: deflated), from its central directory.
fn methods(zip: &[u8]) -> Vec<u16> {
let u16_at = |at: usize| usize::from(u16::from_le_bytes([zip[at], zip[at + 1]]));
let end = zip.len() - 22;
let mut at = u32::from_le_bytes(zip[end + 16..end + 20].try_into().unwrap()) as usize;
(0..u16_at(end + 10))
.map(|_| {
let method = u16_at(at + 10) as u16;
at += 46 + u16_at(at + 28) + u16_at(at + 30) + u16_at(at + 32);
method
})
.collect()
}
#[test]
fn zip_archives_deflate_what_gets_smaller() {
// A xorshift run: bytes that deflating makes larger.
let mut x = 0x9E37_79B9u32;
let noise: Vec<u8> = (0..4096)
.map(|_| {
x ^= x << 13;
x ^= x >> 17;
x ^= x << 5;
x as u8
})
.collect();
let text = "<rect x=\"5\" y=\"5\" width=\"20\" height=\"20\"/>\n".repeat(100).into_bytes();
let files = [("art.svg", text.clone()), ("sub/ünï.png", noise.clone()), ("empty", vec![])];
let zip = super::zip::deflate(&files, super::zip::FIXED_TIME).unwrap();
assert_eq!(methods(&zip), [8, 0, 0], "the text deflated, the noise and the empty file stored");
assert!(zip.windows(noise.len()).any(|w| w == noise.as_slice()), "a stored file's bytes are in the archive as they are");
assert!(zip.len() < super::zip::store(&files).unwrap().len());
let back = super::zip::entries(&zip).unwrap();
assert_eq!(back.len(), 3);
for ((n, b), (m, c)) in files.iter().zip(&back) {
assert_eq!((*n, b), (m.as_str(), c));
}
assert_eq!(super::zip::deflate(&files, super::zip::FIXED_TIME).unwrap(), zip, "the same files, the same archive");
}
#[test]
fn zip_exports_deflate_their_files() {
let mut s = session(1);
for i in 0..40 {
s.execute("shape.rectangle", &json!({"x": i % 30, "y": i % 20, "width": 8, "height": 6})).unwrap();
}
let p = json!({"formats": [{"format": "svg"}, {"format": "png"}]});
let plain = files(&export(&mut s, p.clone()));
let mut q = p;
q["zip"] = json!(true);
let r = export(&mut s, q);
let zip = b64(&r);
assert_eq!(super::zip::entries(&zip).unwrap(), plain, "every file comes back byte for byte");
let svg = plain.iter().position(|(n, _)| n.ends_with(".svg")).unwrap();
let png = plain.iter().position(|(n, _)| n.ends_with(".png")).unwrap();
let methods = methods(&zip);
assert_eq!((methods[svg], methods[png]), (8, 0), "the SVG deflated, the PNG stored");
assert!(zip.len() < super::zip::store(&plain).unwrap().len(), "smaller than the stored archive");
assert_eq!(r["bytes"].as_u64(), Some(zip.len() as u64));
}
#[test]
fn the_document_remembers_its_export_settings() {
let mut s = session(2);
+73 -11
View File
@@ -1,16 +1,18 @@
//! A store-only (uncompressed) ZIP archive: several exported files as one download. The files are
//! already compressed (PNG, JPEG, PDF…), so storing them keeps the archive fast to write and read.
//! ZIP archives: several exported files as one download. [`store`] writes them uncompressed, and
//! [`deflate`] deflates each file that gets smaller and stores the others.
//!
//! [`Archive`] reads the files of a ZIP archive, stored or deflated, as PKWARE's APPNOTE.TXT
//! describes the format (<https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT>).
use std::io::Read as _;
use std::io::{Read as _, Write as _};
use flate2::Crc;
/// The DOS date of every entry: 1980-01-01, the earliest a ZIP date can say (no clock on the web,
/// and a fixed date keeps archives of the same files identical).
const DOS_DATE: u16 = (1 << 5) | 1;
/// The MS-DOS time and date [`store`] gives every entry: midnight on [`DOS_DATE`].
pub const FIXED_TIME: (u16, u16) = (0, DOS_DATE);
/// General purpose flag: file names are UTF-8.
const UTF8_NAMES: u16 = 1 << 11;
/// Version needed to extract: 2.0.
@@ -70,6 +72,57 @@ pub fn store<N: AsRef<str>, B: AsRef<[u8]>>(files: &[(N, B)]) -> Result<Vec<u8>,
Ok(out)
}
/// The archive of `files` (`(name, bytes)`; `/` separates folders in a name), each deflated, or
/// stored when deflating doesn't make it smaller, every entry dated `(time, date)` (MS-DOS format).
/// Fails past the same limits as [`store`].
pub fn deflate<N: AsRef<str>, B: AsRef<[u8]>>(files: &[(N, B)], (time, date): (u16, u16)) -> Result<Vec<u8>, String> {
let too_big = || "too much to zip: at most 65535 files and 4 GB".to_string();
let (mut out, mut central) = (Vec::new(), Vec::new());
for (name, bytes) in files {
let (name, bytes) = (name.as_ref(), bytes.as_ref());
let mut crc = Crc::new();
crc.update(bytes);
let mut enc = flate2::write::DeflateEncoder::new(Vec::new(), flate2::Compression::default());
enc.write_all(bytes).map_err(|e| e.to_string())?;
let deflated = enc.finish().map_err(|e| e.to_string())?;
let (method, data): (u16, &[u8]) = if deflated.len() < bytes.len() { (8, &deflated) } else { (0, bytes) };
let offset = u32::try_from(out.len()).map_err(|_| too_big())?;
let (packed, size) = (u32::try_from(data.len()).map_err(|_| too_big())?, u32::try_from(bytes.len()).map_err(|_| too_big())?);
let name_len = u16::try_from(name.len()).map_err(|_| format!("file name too long to zip: {name}"))?;
// Version 2.0, UTF-8 names, method, time, date, CRC-32, sizes, name length, no extra field.
let common =
[&VERSION.to_le_bytes()[..], &UTF8_NAMES.to_le_bytes(), &method.to_le_bytes(), &time.to_le_bytes(), &date.to_le_bytes()].concat();
let sums = [crc.sum().to_le_bytes(), packed.to_le_bytes(), size.to_le_bytes()].concat();
out.extend_from_slice(&0x0403_4b50u32.to_le_bytes());
out.extend_from_slice(&common);
out.extend_from_slice(&sums);
out.extend_from_slice(&name_len.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
out.extend_from_slice(name.as_bytes());
out.extend_from_slice(data);
central.extend_from_slice(&0x0201_4b50u32.to_le_bytes());
central.extend_from_slice(&VERSION.to_le_bytes());
central.extend_from_slice(&common);
central.extend_from_slice(&sums);
central.extend_from_slice(&name_len.to_le_bytes());
// No extra field or comment, disk 0, no attributes.
central.extend_from_slice(&[0; 12]);
central.extend_from_slice(&offset.to_le_bytes());
central.extend_from_slice(name.as_bytes());
}
let count = u16::try_from(files.len()).map_err(|_| too_big())?;
let (start, len) = (u32::try_from(out.len()).map_err(|_| too_big())?, u32::try_from(central.len()).map_err(|_| too_big())?);
out.extend_from_slice(&central);
out.extend_from_slice(&0x0605_4b50u32.to_le_bytes());
out.extend_from_slice(&[0; 4]);
out.extend_from_slice(&count.to_le_bytes());
out.extend_from_slice(&count.to_le_bytes());
out.extend_from_slice(&len.to_le_bytes());
out.extend_from_slice(&start.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
Ok(out)
}
/// Does `bytes` start with a ZIP archive's first local file header?
pub fn is_zip(bytes: &[u8]) -> bool {
bytes.starts_with(&0x0403_4b50u32.to_le_bytes())
@@ -209,8 +262,8 @@ impl<'a> Archive<'a> {
}
}
/// The entries of a store-only archive written by [`store`] (read through its central directory,
/// checking every CRC) → `(name, bytes)`.
/// The entries of an archive written by [`store`] or [`deflate`] (read through its central
/// directory, checking every CRC and size) → `(name, bytes)`.
#[cfg(test)]
pub fn entries(zip: &[u8]) -> Result<Vec<(String, Vec<u8>)>, String> {
let u16_at = |at: usize| zip.get(at..at + 2).map(|b| u16::from_le_bytes([b[0], b[1]])).ok_or("truncated");
@@ -223,10 +276,12 @@ pub fn entries(zip: &[u8]) -> Result<Vec<(String, Vec<u8>)>, String> {
let mut at = u32_at(end + 16)? as usize;
let mut out = vec![];
for _ in 0..count {
if u32_at(at)? != 0x0201_4b50 || u16_at(at + 10)? != 0 {
return Err("not a stored central record".into());
let method = u16_at(at + 10)?;
if u32_at(at)? != 0x0201_4b50 || (method != 0 && method != 8) {
return Err("not a stored or deflated central record".into());
}
let (crc, size, name_len) = (u32_at(at + 16)?, u32_at(at + 20)? as usize, u16_at(at + 28)? as usize);
let (crc, packed, name_len) = (u32_at(at + 16)?, u32_at(at + 20)? as usize, u16_at(at + 28)? as usize);
let size = u32_at(at + 24)? as usize;
let extra = u16_at(at + 30)? as usize + u16_at(at + 32)? as usize;
let local = u32_at(at + 42)? as usize;
let name = String::from_utf8(zip.get(at + 46..at + 46 + name_len).ok_or("truncated")?.to_vec()).map_err(|e| e.to_string())?;
@@ -234,11 +289,18 @@ pub fn entries(zip: &[u8]) -> Result<Vec<(String, Vec<u8>)>, String> {
return Err(format!("{name}: no local header"));
}
let data = local + 30 + u16_at(local + 26)? as usize + u16_at(local + 28)? as usize;
let bytes = zip.get(data..data + size).ok_or("truncated")?.to_vec();
let data = zip.get(data..data + packed).ok_or("truncated")?;
let bytes = if method == 8 {
let mut inflated = Vec::new();
flate2::read::DeflateDecoder::new(data).read_to_end(&mut inflated).map_err(|e| format!("{name}: {e}"))?;
inflated
} else {
data.to_vec()
};
let mut c = Crc::new();
c.update(&bytes);
if c.sum() != crc {
return Err(format!("{name}: CRC mismatch"));
if c.sum() != crc || bytes.len() != size {
return Err(format!("{name}: CRC or size mismatch"));
}
out.push((name, bytes));
at += 46 + name_len + extra;
+2 -47
View File
@@ -5,7 +5,6 @@
//! Without a folder (the web, agents) the same files come back as a zip archive.
use std::collections::{BTreeMap, BTreeSet, HashSet};
use std::io::Write;
use std::path::{Path, PathBuf};
use vectorcraft_doc::links::hash_bytes;
@@ -550,53 +549,9 @@ fn dos_time() -> (u16, u16) {
}
/// A zip archive of `files` ((path with `/` separators, bytes)), each deflated, or stored when
/// that isn't smaller.
/// that isn't smaller, every entry dated now.
pub(crate) fn zip(files: &[(String, &[u8])]) -> Result<Vec<u8>> {
let too_big = || bad(C, "too large for a zip archive: package to a folder");
let io = |e: std::io::Error| EngineError::Other(e.to_string());
let (time, date) = dos_time();
let (mut out, mut central) = (Vec::new(), Vec::new());
for (name, bytes) in files {
let mut crc = flate2::Crc::new();
crc.update(bytes);
let mut enc = flate2::write::DeflateEncoder::new(Vec::new(), flate2::Compression::default());
enc.write_all(bytes).map_err(io)?;
let deflated = enc.finish().map_err(io)?;
let (method, data): (u16, &[u8]) = if deflated.len() < bytes.len() { (8, &deflated) } else { (0, bytes) };
let offset = u32::try_from(out.len()).map_err(|_| too_big())?;
let (packed, size) = (u32::try_from(data.len()).map_err(|_| too_big())?, u32::try_from(bytes.len()).map_err(|_| too_big())?);
let name_len = u16::try_from(name.len()).map_err(|_| too_big())?;
// Version 2.0, UTF-8 names, method, time, date, CRC-32, sizes, name length, no extra field.
let common = [&20u16.to_le_bytes()[..], &0x0800u16.to_le_bytes(), &method.to_le_bytes(), &time.to_le_bytes(), &date.to_le_bytes()].concat();
let sums = [crc.sum().to_le_bytes(), packed.to_le_bytes(), size.to_le_bytes()].concat();
out.extend_from_slice(&0x0403_4b50u32.to_le_bytes());
out.extend_from_slice(&common);
out.extend_from_slice(&sums);
out.extend_from_slice(&name_len.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
out.extend_from_slice(name.as_bytes());
out.extend_from_slice(data);
central.extend_from_slice(&0x0201_4b50u32.to_le_bytes());
central.extend_from_slice(&20u16.to_le_bytes());
central.extend_from_slice(&common);
central.extend_from_slice(&sums);
central.extend_from_slice(&name_len.to_le_bytes());
// No extra field or comment, disk 0, no attributes.
central.extend_from_slice(&[0; 12]);
central.extend_from_slice(&offset.to_le_bytes());
central.extend_from_slice(name.as_bytes());
}
let count = u16::try_from(files.len()).map_err(|_| too_big())?;
let (start, len) = (u32::try_from(out.len()).map_err(|_| too_big())?, u32::try_from(central.len()).map_err(|_| too_big())?);
out.extend_from_slice(&central);
out.extend_from_slice(&0x0605_4b50u32.to_le_bytes());
out.extend_from_slice(&[0; 4]);
out.extend_from_slice(&count.to_le_bytes());
out.extend_from_slice(&count.to_le_bytes());
out.extend_from_slice(&len.to_le_bytes());
out.extend_from_slice(&start.to_le_bytes());
out.extend_from_slice(&0u16.to_le_bytes());
Ok(out)
super::fileio::zip::deflate(files, dos_time()).map_err(|_| bad(C, "too large for a zip archive: package to a folder"))
}
#[cfg(test)]