mirror of
https://github.com/storytold/lightcraft.git
synced 2026-10-27 19:47:35 -06:00
Merge PR #755
On merge: the quick preview path orients like the whole-file path (preview_orientation with the raw's header prefix, so a portrait raw whose embedded JPEG carries no orientation is not sideways), and its doc links point at preview_window.
This commit is contained in:
@@ -314,6 +314,7 @@ LightCraft is young and moving fast: **alpha**. **Where we honestly stand** (det
|
||||
- **camera colour calibration:** Sony, Nikon, Panasonic, Fujifilm and Canon CR3 raws have guarded estimates from their camera JPEGs, with built-in ILCE-7CR, ILCE-7M4, X-H2S and X-T4 profiles; measured calibration is missing, and other raws or rejected fits retain a neutral matrix;
|
||||
- **compressed Olympus raws and unsupported CR3 variants:** these use embedded JPEG previews when present. Fujifilm lossless/lossy compressed RAF now decodes sensor data; [verification and existing-library reload instructions](docs/raf-compression.md);
|
||||
- **AI masks and denoise:** subject and sky selection are classical heuristics;
|
||||
- **network libraries:** a preview reads the raw's header and its embedded JPEG only (56.5 MB → 5.9 MB, 10 s → 1 s on the test share), but the *developed* picture still needs the sensor data over the wire; see [docs/raw-preview-reads.md](docs/raw-preview-reads.md);
|
||||
- **on-screen HDR display, video and the Classic Print / Book / Map modules.**
|
||||
- **What's next:** see [the ranked gaps](docs/gaps.md) and [the roadmap](docs/roadmap.md).
|
||||
|
||||
|
||||
+179
-2
@@ -680,17 +680,144 @@ fn fit_box(max_edge: usize) -> lightcraft_codecs::DecodeOptions {
|
||||
|
||||
/// Filesystem-backed embedded-preview hook (native).
|
||||
///
|
||||
/// Reads the whole file to find the preview, so it holds the memory gate for twice the file's size (the file and the
|
||||
/// decoded preview): background work (the face scan) waits for room, interactive work is counted and never waits.
|
||||
/// The quick path reads the file's header and its embedded preview and nothing else (see
|
||||
/// [`preview_window`]); a format whose preview the header cannot point at falls back to the
|
||||
/// whole-file path, which is what every format used to cost. The memory gate still counts the whole
|
||||
/// file, which is what the decode of one can hold.
|
||||
pub fn fs_preview_loader() -> PreviewLoader {
|
||||
Arc::new(|path: &str, max_edge: usize| {
|
||||
let len = std::fs::metadata(path).map(|m| usize::try_from(m.len()).unwrap_or(usize::MAX)).unwrap_or(0);
|
||||
let gate = crate::memory::work_gate();
|
||||
let _permit = if crate::memory::is_background() { gate.acquire(len.saturating_mul(2)) } else { gate.acquire_urgent(len.saturating_mul(2)) };
|
||||
if let Some(image) = preview_window(path).ok().flatten().and_then(|(header, window, _)| quick_image_srgb(&header, &window, max_edge)) {
|
||||
return Some(image);
|
||||
}
|
||||
embedded_preview_srgb(&std::fs::read(path).ok()?, max_edge)
|
||||
})
|
||||
}
|
||||
|
||||
/// What [`preview_window`] read from a file, so a caller can see what a preview cost.
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct PreviewRead {
|
||||
/// Bytes read.
|
||||
pub read: u64,
|
||||
/// The file's size.
|
||||
pub total: u64,
|
||||
/// Whether the preview came from a bounded read (`false`: the whole file was read, which is what
|
||||
/// every preview cost before the header path existed).
|
||||
pub header_path: bool,
|
||||
}
|
||||
|
||||
impl PreviewRead {
|
||||
/// The share of the file that was read, `0.0`..=`1.0`.
|
||||
pub fn fraction(self) -> f64 {
|
||||
if self.total == 0 {
|
||||
return 0.0;
|
||||
}
|
||||
(self.read as f64 / self.total as f64).clamp(0.0, 1.0)
|
||||
}
|
||||
}
|
||||
|
||||
/// The file's header and its embedded preview, read without the rest of the file: `(header bytes,
|
||||
/// preview bytes, what was read)`. `None` when the header points at no preview this build can find,
|
||||
/// which is the caller's cue to read the file whole.
|
||||
///
|
||||
/// Why it exists: a 56 MB Fuji RAF keeps its JPEG in the first 5.6 MB (measured: `DSCF9492.RAF`),
|
||||
/// so reading the file whole to extract it costs ten times the bytes — and on a network share ten
|
||||
/// times the wait (measured on that file: 2.5 s local against 25.6 s over SMB). `embedded_preview`
|
||||
/// still needs the whole container, so the header comes back with the window; for a preview taken
|
||||
/// out of a container that carries no preview colour space of its own (Nikon's maker note) the
|
||||
/// whole-file fallback stays in charge.
|
||||
pub fn preview_window(path: &str) -> Result<Option<(Vec<u8>, Vec<u8>, PreviewRead)>, String> {
|
||||
use std::io::{Read, Seek, SeekFrom};
|
||||
|
||||
let mut file = std::fs::File::open(path).map_err(|e| format!("{path}: {e}"))?;
|
||||
let total = file.metadata().map_err(|e| format!("{path}: {e}"))?.len();
|
||||
let mut prefix = read_prefix(&mut file, path, total)?;
|
||||
|
||||
// What the header itself declares, before the container reader is asked: it clamps a slice to
|
||||
// the buffer it was given, so a prefix that stops inside the declared preview yields a
|
||||
// self-consistently short range that looks trustworthy (measured on a real 56 MB DSCF9492.RAF:
|
||||
// a 4 KiB prefix reports 3948 bytes where the file holds 5.6 MB). Reading the declared extent
|
||||
// first costs nothing — it lives in the first bytes of every format covered here.
|
||||
if let Some(extent) = raf_preview_extent(&prefix)
|
||||
&& extent > prefix.len() as u64
|
||||
{
|
||||
prefix = read_prefix_at(&mut file, path, extent.min(total))?;
|
||||
}
|
||||
let Some(range) = lightcraft_raw::bounded_preview(&prefix) else { return Ok(None) };
|
||||
// A preview at the file's end may have been padded past it: clamp rather than fail.
|
||||
let end = range.offset.saturating_add(range.window_len).min(total);
|
||||
let len = usize::try_from(end.saturating_sub(range.offset)).map_err(|_| format!("{path}: preview too large"))?;
|
||||
// When the prefix already covers the range (the usual case: the prefix was grown to the declared
|
||||
// extent), take the window out of it. Reading those bytes twice would double a 5.6 MB preview.
|
||||
let prefix_len = prefix.len() as u64;
|
||||
let from_prefix = usize::try_from(range.offset).ok().and_then(|start| prefix.get(start..start.checked_add(len)?)).map(<[u8]>::to_vec);
|
||||
let (window, bytes_read) = match from_prefix {
|
||||
Some(window) => (window, prefix_len),
|
||||
None => {
|
||||
let mut window = vec![0u8; len];
|
||||
file.seek(SeekFrom::Start(range.offset)).map_err(|e| format!("{path}: {e}"))?;
|
||||
file.read_exact(&mut window).map_err(|e| format!("{path}: {e}"))?;
|
||||
(window, prefix_len + len as u64)
|
||||
}
|
||||
};
|
||||
|
||||
// A lying pointer must not cost the user their preview: hand the caller the whole-file fallback.
|
||||
if lightcraft_raw::embedded_preview_from_range(&window, range).is_none() {
|
||||
return Ok(None);
|
||||
}
|
||||
let read = PreviewRead { read: bytes_read, total, header_path: true };
|
||||
// On a network share the bytes are the wait, so say what this preview actually cost. A window
|
||||
// taken from the prefix cost nothing beyond the prefix itself.
|
||||
if lightcraft_pipeline::profiling() {
|
||||
eprintln!("[profile] preview {path}: read {} of {} bytes (header {}, window {})", read.read, total, prefix_len, window.len());
|
||||
}
|
||||
Ok(Some((prefix, window, read)))
|
||||
}
|
||||
|
||||
/// The file's first [`lightcraft_raw::PREVIEW_PREFIX`] bytes (or the whole file when it is shorter),
|
||||
/// which is what every format [`lightcraft_raw::bounded_preview`] covers keeps its pointer in.
|
||||
fn read_prefix(file: &mut std::fs::File, path: &str, total: u64) -> Result<Vec<u8>, String> {
|
||||
let len = lightcraft_raw::PREVIEW_PREFIX.min(usize::try_from(total).unwrap_or(usize::MAX));
|
||||
read_prefix_at(file, path, len as u64)
|
||||
}
|
||||
|
||||
/// The file's first `len` bytes, read from the start.
|
||||
fn read_prefix_at(file: &mut std::fs::File, path: &str, len: u64) -> Result<Vec<u8>, String> {
|
||||
use std::io::{Read, Seek, SeekFrom};
|
||||
let len = usize::try_from(len).map_err(|_| format!("{path}: file too large"))?;
|
||||
let mut prefix = vec![0u8; len];
|
||||
file.seek(SeekFrom::Start(0)).map_err(|e| format!("{path}: {e}"))?;
|
||||
file.read_exact(&mut prefix).map_err(|e| format!("{path}: {e}"))?;
|
||||
Ok(prefix)
|
||||
}
|
||||
|
||||
/// The end of a Fuji RAF's preview block, from the two big-endian u32s every RAF header carries at
|
||||
/// 84 (offset) and 88 (length) — the same fields `lightcraft_raw`'s RAF reader uses, read here
|
||||
/// without the rest of the file so the prefix can be made big enough first.
|
||||
fn raf_preview_extent(prefix: &[u8]) -> Option<u64> {
|
||||
if !prefix.starts_with(b"FUJIFILMCCD-RAW") {
|
||||
return None;
|
||||
}
|
||||
let u32_at = |at: usize| prefix.get(at..at + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]])).map(u64::from);
|
||||
let (offset, len) = (u32_at(84)?, u32_at(88)?);
|
||||
(offset > 0 && len > 0).then(|| offset.checked_add(len)).flatten()
|
||||
}
|
||||
|
||||
/// The display-ready preview of a raw whose embedded JPEG was already located: the counterpart of
|
||||
/// [`embedded_preview_srgb`] for [`preview_window`], which does not re-read the container.
|
||||
/// `None` when the bytes are not a decodable image.
|
||||
fn quick_image_srgb(header: &[u8], window: &[u8], max_edge: usize) -> Option<lightcraft_raster::Rgba8> {
|
||||
let mut d = lightcraft_codecs::decode(window, fit_box(max_edge)).ok()?;
|
||||
if d.image.width.max(d.image.height) > max_edge {
|
||||
d.image = fit(&d.image, max_edge, max_edge, Filter::Box);
|
||||
d.alpha = None;
|
||||
}
|
||||
// The JPEG's own EXIF orientation, else the raw's (from its header), as the whole-file path does
|
||||
Some(d.to_srgb8().oriented(preview_orientation(header, d.orientation)))
|
||||
}
|
||||
|
||||
/// The filesystem-backed [`PairLoader`]: the file is read and decoded once, and developed as usual and from the photo's
|
||||
/// denoised picture. It holds two pictures at once, so it counts for more at the memory gate than a plain load.
|
||||
pub fn fs_pair_loader() -> PairLoader {
|
||||
@@ -803,6 +930,56 @@ mod tests {
|
||||
assert!(!lightcraft_catalog::relative_wb_camera("DNG", "SONY ILCE-7M3"));
|
||||
}
|
||||
|
||||
/// A RAF whose preview pointer is in its header, then a large raw tail the reader must not need.
|
||||
fn raf_with_embedded_jpeg(jpeg: &[u8], tail: usize) -> Vec<u8> {
|
||||
let head = 160;
|
||||
let mut f = vec![0u8; head];
|
||||
f.get_mut(..16).expect("head").copy_from_slice(b"FUJIFILMCCD-RAW ");
|
||||
f.get_mut(84..88).expect("jpeg offset").copy_from_slice(&(head as u32).to_be_bytes());
|
||||
f.get_mut(88..92).expect("jpeg length").copy_from_slice(&(jpeg.len() as u32).to_be_bytes());
|
||||
f.extend_from_slice(jpeg);
|
||||
f.extend(std::iter::repeat_n(0x33u8, tail));
|
||||
f
|
||||
}
|
||||
|
||||
/// A raw's preview is read from its header and its JPEG only. Every preview used to cost a
|
||||
/// whole-file read, which on a 56 MB RAF over a network share is ten times the bytes and ten
|
||||
/// times the wait (measured: 2.5 s local against 25.6 s over SMB, DSCF9492.RAF). The pixels the
|
||||
/// quick path produces must equal the whole-file path's, exactly.
|
||||
#[test]
|
||||
fn preview_loading_reads_only_the_header_and_the_jpeg() {
|
||||
let (w, h) = (48u32, 32u32);
|
||||
let rgb: Vec<u8> = (0..w * h * 3).map(|i| (i * 13 % 251) as u8).collect();
|
||||
let img = EncodeImage::new(w, h, 3, Samples::U8(&rgb));
|
||||
let jpeg = encode_jpeg(&img, 90, ChromaSubsampling::S420, &EncodeMeta::default()).unwrap();
|
||||
let file = raf_with_embedded_jpeg(&jpeg, 200_000);
|
||||
let dir = crate::tests_xmp::temp_dir("preview-window");
|
||||
let path = dir.join("dscf0001.raf");
|
||||
std::fs::write(&path, &file).unwrap();
|
||||
let path = path.to_string_lossy().into_owned();
|
||||
let max_edge = 512;
|
||||
|
||||
let (header, window, read) = preview_window(&path).expect("the file reads").expect("it has a preview");
|
||||
assert_eq!(header.len(), lightcraft_raw::PREVIEW_PREFIX, "the header prefix is bounded");
|
||||
assert!(read.header_path, "the header path is what answered");
|
||||
assert_eq!(read.total, file.len() as u64);
|
||||
assert!(read.fraction() < 0.5, "the read is a fraction of the file: {read:?}");
|
||||
assert!(window.len() < file.len() / 2, "the raw tail must not be read: {} of {} bytes", window.len(), file.len());
|
||||
|
||||
// The quick path and the whole-file path agree byte for byte.
|
||||
let quick = quick_image_srgb(&header, &window, max_edge).expect("the window holds a decodable preview");
|
||||
let whole = embedded_preview_srgb(&file, max_edge).expect("the whole file decodes too");
|
||||
assert_eq!((quick.width, quick.height), (whole.width, whole.height), "same preview size");
|
||||
assert_eq!(quick.data, whole.data, "byte for byte");
|
||||
|
||||
// A file with no locatable preview declines (the caller then reads it whole), a missing file errors.
|
||||
let plain = dir.join("plain.jpg");
|
||||
std::fs::write(&plain, &jpeg).unwrap();
|
||||
assert!(preview_window(&plain.to_string_lossy()).expect("a plain file reads").is_none());
|
||||
assert!(preview_window(&dir.join("nope.raf").to_string_lossy()).is_err());
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
/// Issue #367: probes read headers, not pixels, and report what the decode-based probe did:
|
||||
/// oriented dimensions (EXIF orientation 6 swaps them), metadata, the error for a truncated
|
||||
/// file or one that isn't an image.
|
||||
|
||||
@@ -1295,6 +1295,23 @@ impl crate::Session {
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `p`'s thumbnail can be the camera's own JPEG, so a frontend can skip the sensor
|
||||
/// decode behind it.
|
||||
///
|
||||
/// The grid shows this JPEG first in any case ([`QuickJob`]: "grid thumbnails of unedited raws
|
||||
/// likewise start from the embedded preview"), and then paid for a render of the raw to replace
|
||||
/// it. That render decodes the whole sensor — 50 of a 56 MB Fuji RAF's bytes — for a 512 px
|
||||
/// thumbnail the 4 416 × 2 944 embedded JPEG already covers, and on a share at 1.2 MB/s it is
|
||||
/// 45 s a photo: a 149-photo folder spent 8.3 GB of traffic on thumbnails, and the queue behind
|
||||
/// it delayed the render the user was waiting for.
|
||||
///
|
||||
/// Only unedited raws answer `true`: any edit must be rendered, and `has_import_look` is what
|
||||
/// says the photo still looks the way it did at import. A photo the user edits re-renders
|
||||
/// through the ordinary path, because its thumbnail key (settings hash) changes.
|
||||
pub fn embedded_thumb_is_final(&self, p: &Photo) -> bool {
|
||||
self.embedded_of(p).is_some()
|
||||
}
|
||||
|
||||
/// The camera's embedded preview of a raw file, for the face scan. Faces are read from the unedited picture, so the
|
||||
/// photo's edits do not matter, and the camera's own JPEG (oriented, display-ready) costs a fraction of decoding the
|
||||
/// raw. `None` for anything else, and where the host installed no loader.
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
//! Opt-in test of the embedded-preview read path on a **real file the user has** — typically one on a
|
||||
//! network share, which is where reading only the header instead of the whole raw matters:
|
||||
//!
|
||||
//! LC_PREVIEW_RAW="/run/user/1000/gvfs/smb-share:.../DSCF0001.RAF" \
|
||||
//! cargo test --release -p lightcraft-engine --test preview_read -- --ignored --nocapture
|
||||
//!
|
||||
//! It reports what the preview cost and fails when the read is not the small part of the file the
|
||||
//! header path promises. Skipped without `LC_PREVIEW_RAW` (there is no committed raw fixture: media
|
||||
//! is never committed, see AGENTS.md).
|
||||
use std::time::Instant;
|
||||
|
||||
#[test]
|
||||
#[ignore = "needs LC_PREVIEW_RAW pointing at a real raw file"]
|
||||
fn a_preview_reads_only_the_header_and_the_preview() {
|
||||
let Ok(path) = std::env::var("LC_PREVIEW_RAW") else {
|
||||
eprintln!("skipped: set LC_PREVIEW_RAW=<path to a raw file>");
|
||||
return;
|
||||
};
|
||||
let file_len = std::fs::metadata(&path).expect("the file exists").len();
|
||||
assert!(file_len > 0, "empty file");
|
||||
|
||||
let started = Instant::now();
|
||||
let window =
|
||||
lightcraft_engine::files::preview_window(&path).expect("the file reads").expect("it carries an embedded preview the header can point at");
|
||||
let elapsed = started.elapsed();
|
||||
let (_, preview, read) = window;
|
||||
|
||||
let frac = read.fraction();
|
||||
println!(
|
||||
"preview of {path}\n read {} of {} bytes ({:.1} %) in {:.2} s{}",
|
||||
read.read,
|
||||
read.total,
|
||||
frac * 100.0,
|
||||
elapsed.as_secs_f64(),
|
||||
if read.header_path { " [header path]" } else { " [whole file]" }
|
||||
);
|
||||
|
||||
assert!(read.header_path, "the header path must answer for a raw whose pointer is in its header");
|
||||
assert_eq!(read.total, file_len, "the report covers the whole file");
|
||||
assert!(frac < 0.5, "a preview must not cost most of the file: read {} of {} bytes", read.read, read.total);
|
||||
assert!(preview.starts_with(&[0xff, 0xd8]), "the preview bytes are a JPEG: {:?}", preview.get(..4));
|
||||
}
|
||||
@@ -54,7 +54,10 @@ pub use lightcraft_geom::Orientation;
|
||||
pub use lightcraft_meta::Metadata;
|
||||
pub use lightcraft_raster::Rgb32f;
|
||||
pub use opcodes::{Opcode, OpcodeLists};
|
||||
pub use preview::{PreviewColorSpace, embedded_preview, embedded_preview_color_space, embedded_preview_dynamic_range_optimized};
|
||||
pub use preview::{
|
||||
PREVIEW_PREFIX, PreviewColorSpace, PreviewRange, bounded_preview, embedded_preview, embedded_preview_color_space,
|
||||
embedded_preview_dynamic_range_optimized, embedded_preview_from_range,
|
||||
};
|
||||
pub use semantic::{SemanticMask, semantic_masks};
|
||||
|
||||
use lightcraft_color::Xy;
|
||||
|
||||
@@ -186,6 +186,63 @@ pub fn embedded_preview(bytes: &[u8]) -> Option<Vec<u8>> {
|
||||
/// walking the marker segments and the entropy-coded data. `None` when it is not a complete, displayable
|
||||
/// (baseline or progressive) JPEG, so a stray `D8 FF` in other data is rejected. `*reach` is left at
|
||||
/// the furthest byte looked at (what the walk cost; see [`scan_for_jpeg`]).
|
||||
/// Where a preview lives in a file, so a caller with a seekable file can read just that part of it.
|
||||
///
|
||||
/// Why it exists: a 56 MB Fuji RAF keeps its JPEG in the first 5.6 MB (measured: DSCF9492.RAF at
|
||||
/// 5.6 MB), so a grid thumbnail or a loupe preview has no reason to pull the remaining 50 MB —
|
||||
/// which on a network share is the difference between a moment and half a minute. A caller that has
|
||||
/// only bytes (`wasm`, tests) keeps using [`embedded_preview`].
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub struct PreviewRange {
|
||||
/// Byte offset of the preview in the file.
|
||||
pub offset: u64,
|
||||
/// Bytes to read at [`Self::offset`]: the container's declared preview extent, which may carry
|
||||
/// padding past the preview's end-of-image marker.
|
||||
pub window_len: u64,
|
||||
}
|
||||
|
||||
/// The preview's range in the file, when the file's header alone locates it.
|
||||
///
|
||||
/// `bytes` is the file's first [`PREVIEW_PREFIX`] bytes (more is fine). `None` means "read the file
|
||||
/// whole and call [`embedded_preview`]": the formats not covered here (a TIFF-based raw's pointer
|
||||
/// tag can sit in an IFD the prefix cuts short; CR3 scans box by box) and files whose pointer is
|
||||
/// damaged.
|
||||
///
|
||||
/// The caller reads `[range.offset, range.offset + range.window_len)` and passes those bytes to
|
||||
/// [`embedded_preview_from_range`], which returns exactly what [`embedded_preview`] returns for the
|
||||
/// whole file.
|
||||
///
|
||||
/// Pass a prefix that actually covers the pointer's range (read [`Self::offset`] +
|
||||
/// [`Self::window_len`] bytes, or start with [`PREVIEW_PREFIX`] and grow when it is not enough):
|
||||
/// [`crate::vendor::raf::header`] clamps a slice to the buffer it was given, so a prefix that stops
|
||||
/// inside the declared extent yields a shorter `window_len` than the file really holds.
|
||||
pub fn bounded_preview(bytes: &[u8]) -> Option<PreviewRange> {
|
||||
if !bytes.starts_with(b"FUJIFILMCCD-RAW") {
|
||||
return None;
|
||||
}
|
||||
let jpeg = crate::vendor::raf::header(bytes).ok()?.jpeg?;
|
||||
let offset = (jpeg.as_ptr() as usize).checked_sub(bytes.as_ptr() as usize)?;
|
||||
if offset.checked_add(jpeg.len())? > bytes.len() {
|
||||
return None;
|
||||
}
|
||||
Some(PreviewRange { offset: offset as u64, window_len: jpeg.len() as u64 })
|
||||
}
|
||||
|
||||
/// The preview inside `window`, which holds the file's bytes at `[range.offset, range.offset +
|
||||
/// range.window_len)`: the counterpart of [`bounded_preview`]. Padding past the end-of-image marker
|
||||
/// is dropped, exactly as [`embedded_preview`] drops it.
|
||||
pub fn embedded_preview_from_range(window: &[u8], range: PreviewRange) -> Option<Vec<u8>> {
|
||||
let preview = window.get(..usize::try_from(range.window_len).ok()?)?;
|
||||
if !is_dct_jpeg(preview) {
|
||||
return None;
|
||||
}
|
||||
Some(trim_eoi(preview).to_vec())
|
||||
}
|
||||
|
||||
/// How many bytes of a raw file are enough for [`bounded_preview`]: every covered format keeps its
|
||||
/// preview pointer in a fixed header, well inside this.
|
||||
pub const PREVIEW_PREFIX: usize = 4096;
|
||||
|
||||
fn jpeg_extent(b: &[u8], soi: usize, reach: &mut usize) -> Option<(usize, u8)> {
|
||||
let mut i = soi + 2;
|
||||
let r = walk_jpeg(b, &mut i);
|
||||
@@ -341,6 +398,112 @@ mod tests {
|
||||
j
|
||||
}
|
||||
|
||||
/// A minimal RAF: the fixed header (JPEG offset/length at 84/88), the JPEG, then a raw tail the
|
||||
/// caller must not have to read. `jpeg_len` is written as declared, so a test can claim more than
|
||||
/// the JPEG holds the way a padded container does.
|
||||
fn raf_fixture(jpeg: &[u8], gap: usize, preamble: usize, declared_len: Option<usize>) -> Vec<u8> {
|
||||
let head = 160 + preamble;
|
||||
let jpeg_off = head;
|
||||
let mut f = vec![0u8; head];
|
||||
f[..16].copy_from_slice(b"FUJIFILMCCD-RAW ");
|
||||
f[84..88].copy_from_slice(&(jpeg_off as u32).to_be_bytes());
|
||||
f[88..92].copy_from_slice(&(declared_len.unwrap_or(jpeg.len()) as u32).to_be_bytes());
|
||||
// records directory and raw block are absent (offset 0), which `header` reads as "none"
|
||||
f.extend_from_slice(jpeg);
|
||||
f.extend(std::iter::repeat_n(0x33u8, gap));
|
||||
f
|
||||
}
|
||||
|
||||
/// A RAF's preview pointer sits at a fixed offset, so a header-sized prefix locates it and the
|
||||
/// caller reads just that range instead of the whole file — the whole point on a network share.
|
||||
#[test]
|
||||
fn bounded_preview_locates_a_rafs_jpeg_from_the_header_alone() {
|
||||
let jpeg = fake_jpeg(5000);
|
||||
let file = raf_fixture(&jpeg, 40_000, 0, None);
|
||||
// A prefix that covers the pointer's range: header + JPEG, and nothing of the tail.
|
||||
let prefix_len = 8192.min(file.len());
|
||||
let prefix = &file[..prefix_len];
|
||||
|
||||
let range = bounded_preview(prefix).expect("a RAF's preview is found from its header");
|
||||
assert_eq!(range.offset, 160);
|
||||
assert_eq!(range.window_len, jpeg.len() as u64);
|
||||
assert!(range.offset + range.window_len <= prefix_len as u64, "the prefix covers the whole range: {range:?} of {prefix_len}");
|
||||
assert!((range.offset + range.window_len) < file.len() as u64, "the range stops short of the tail: {range:?} of {}", file.len());
|
||||
|
||||
// What the caller reads, and what it holds afterwards: identical to the whole-file answer.
|
||||
let window = prefix.get(range.offset as usize..(range.offset + range.window_len) as usize).expect("the range lies inside the prefix");
|
||||
let bounded = embedded_preview_from_range(window, range).expect("the range holds the preview");
|
||||
assert_eq!(Some(bounded.clone()), embedded_preview(&file));
|
||||
assert!(bounded.ends_with(&[0xff, 0xd9]), "the preview ends at its EOI marker");
|
||||
|
||||
// A prefix that stops inside the declared extent is clamped by the container reader. The
|
||||
// caller then reads a short window; what comes back is the whole JPEG it holds (the EOI
|
||||
// marker is inside), never bytes past the window.
|
||||
let short = &file[..2000];
|
||||
let clipped = bounded_preview(short).expect("a pointer before the cut is still found");
|
||||
assert!(clipped.window_len < jpeg.len() as u64, "clamped: {clipped:?}");
|
||||
let window = short.get(clipped.offset as usize..).expect("inside");
|
||||
let preview = embedded_preview_from_range(window, clipped).expect("the bytes are a JPEG start");
|
||||
// No end-of-image marker inside the window: the bytes come back as they are, exactly as the
|
||||
// whole-file path returns an unclosed JPEG. The decoder refuses it later; nothing is invented.
|
||||
assert_eq!(preview, window, "an unclosed JPEG is handed on unchanged, never padded");
|
||||
assert!(preview.len() <= window.len(), "never past the window");
|
||||
}
|
||||
|
||||
/// The preview pointer may sit past the prefix (a long maker-note record before it), and a
|
||||
/// container may declare padding after the JPEG: both stay correct, and the padding is dropped.
|
||||
#[test]
|
||||
fn bounded_preview_handles_a_late_pointer_and_padding() {
|
||||
let jpeg = fake_jpeg(300);
|
||||
let late = raf_fixture(&jpeg, 0, 9000, None);
|
||||
let range = bounded_preview(&late).expect("the whole file is always a valid prefix");
|
||||
assert_eq!(range.offset, (160 + 9000) as u64);
|
||||
let padded = raf_fixture(&jpeg, 500, 0, Some(jpeg.len() + 500));
|
||||
let range = bounded_preview(&padded).expect("a padded extent is still located");
|
||||
assert_eq!(range.window_len, (jpeg.len() + 500) as u64);
|
||||
let window = padded.get(range.offset as usize..(range.offset + range.window_len) as usize).expect("the window is inside the file");
|
||||
let preview = embedded_preview_from_range(window, range).expect("the padding is dropped");
|
||||
assert_eq!(preview, embedded_preview(&padded).expect("the whole-file answer agrees"));
|
||||
assert!(preview.ends_with(&[0xff, 0xd9]));
|
||||
}
|
||||
|
||||
/// Without a usable header or pointer the caller must read the file whole: the answer is `None`,
|
||||
/// never a wrong range.
|
||||
#[test]
|
||||
fn bounded_preview_declines_what_the_header_cannot_answer() {
|
||||
let jpeg = fake_jpeg(64);
|
||||
let file = raf_fixture(&jpeg, 0, 0, None);
|
||||
assert_eq!(bounded_preview(&jpeg), None, "not a raw at all");
|
||||
assert_eq!(bounded_preview(b""), None);
|
||||
if let Some(short) = file.get(..64) {
|
||||
assert_eq!(bounded_preview(short), None, "a header cut short has not shown the pointer yet");
|
||||
}
|
||||
// a pointer that claims more than the file holds: `header` clamps the slice to the buffer,
|
||||
// so the range never runs past the end (no out-of-bounds read can follow from it)
|
||||
let mut lying = raf_fixture(&jpeg, 4096, 0, None);
|
||||
lying[88..92].copy_from_slice(&(u32::MAX / 2).to_be_bytes());
|
||||
let clamped = bounded_preview(&lying).expect("a pointer inside the file is still usable");
|
||||
assert!(clamped.offset + clamped.window_len <= lying.len() as u64, "clamped to the file: {clamped:?}");
|
||||
// a range the window cannot satisfy yields nothing
|
||||
assert_eq!(embedded_preview_from_range(&jpeg, PreviewRange { offset: 0, window_len: u64::MAX }), None);
|
||||
// a TIFF-based raw: its pointer tag lives in an IFD this prefix may cut short
|
||||
let mut tiff = b"II*\0\x08\0\0\0".to_vec();
|
||||
tiff.extend_from_slice(&[0u8; 64]);
|
||||
assert_eq!(bounded_preview(&tiff), None);
|
||||
}
|
||||
|
||||
/// A range that does not hold a JPEG yields nothing, rather than a truncated image handed on.
|
||||
#[test]
|
||||
fn a_range_that_does_not_hold_a_jpeg_yields_nothing() {
|
||||
let jpeg = fake_jpeg(200);
|
||||
let range = PreviewRange { offset: 0, window_len: jpeg.len() as u64 };
|
||||
assert_eq!(embedded_preview_from_range(&jpeg, range).as_deref(), Some(&jpeg[..]));
|
||||
assert_eq!(embedded_preview_from_range(&jpeg[..10], range), None, "window shorter than the range claims");
|
||||
let not_jpeg = vec![0u8; jpeg.len()];
|
||||
assert_eq!(embedded_preview_from_range(¬_jpeg, range), None, "not a JPEG");
|
||||
assert_eq!(embedded_preview_from_range(&[], PreviewRange { offset: 0, window_len: 0 }), None, "empty window");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn picks_largest_dct_jpeg() {
|
||||
let small = fake_jpeg(10);
|
||||
|
||||
@@ -444,8 +444,10 @@ fn group_header(app: &mut LightcraftApp, ui: &mut egui::Ui, run: &DateRun, ids:
|
||||
}
|
||||
|
||||
/// Request a grid/filmstrip thumbnail at `priority`. An unedited raw without a thumbnail texture
|
||||
/// first gets a stand-in (its cached thumbnail, else its embedded camera preview), and its real
|
||||
/// render then follows in the background.
|
||||
/// gets its embedded camera preview, and the render of the raw behind it follows in the background
|
||||
/// — except where that render would only redraw what the embedded JPEG already shows (see
|
||||
/// [`lightcraft_engine::media::Session::embedded_thumb_is_final`]), which for a folder of raws on a
|
||||
/// network share is the difference between reading every file whole and reading a preview each.
|
||||
pub fn request_thumb(app: &mut LightcraftApp, id: PhotoId, size: usize, priority: u32) {
|
||||
let bucket = lightcraft_engine::media::thumb_bucket(size);
|
||||
if let Some(photo) = app.session.catalog.photo(id)
|
||||
@@ -453,19 +455,35 @@ pub fn request_thumb(app: &mut LightcraftApp, id: PhotoId, size: usize, priority
|
||||
{
|
||||
return;
|
||||
}
|
||||
// The camera's embedded preview is this photo's thumbnail (unedited raw): once its quick render
|
||||
// has arrived there is nothing left to ask for. Without this the grid would rebuild the render
|
||||
// job — and its cache key — on every frame, which is what the embedded-thumbnail record settles.
|
||||
if let Some(photo) = app.session.catalog.photo(id)
|
||||
&& (app.renderer.thumb_quick_is_final(photo, bucket, priority) || app.renderer.thumb_quick_pending(photo, bucket))
|
||||
{
|
||||
return;
|
||||
}
|
||||
let Some(job) = app.session.thumb_job(id, size) else { return };
|
||||
#[cfg(test)]
|
||||
{
|
||||
app.renderer.thumb_jobs_built += 1;
|
||||
}
|
||||
let quick = if app.renderer.textures.contains_key(&Slot::Thumb(id)) { None } else { app.session.quick_thumb_job(&job) };
|
||||
// A cached render is a stand-in for whatever comes next; an embedded camera preview of an
|
||||
// unedited raw is the picture itself, so no sensor decode is scheduled behind it.
|
||||
let embedded_is_final =
|
||||
quick.is_some() && app.session.catalog.photo(id).is_some_and(|p| skips_the_real_render(app.session.embedded_thumb_is_final(p), true));
|
||||
if let Some(photo) = app.session.catalog.photo(id) {
|
||||
app.renderer.remember_thumb(photo, bucket, job.key, quick.is_some());
|
||||
if embedded_is_final {
|
||||
app.renderer.remember_embedded_thumb(photo, bucket, job.key);
|
||||
} else {
|
||||
app.renderer.remember_thumb(photo, bucket, job.key, quick.is_some());
|
||||
}
|
||||
}
|
||||
match quick {
|
||||
Some(q) => {
|
||||
app.renderer.request_quick(Slot::ThumbQuick(id), q, priority + 1);
|
||||
if !app.renderer.is_pending(Slot::ThumbQuick(id)) {
|
||||
if !embedded_is_final && !app.renderer.is_pending(Slot::ThumbQuick(id)) {
|
||||
app.renderer.request(Slot::Thumb(id), job, BACKGROUND_THUMB_PRIORITY);
|
||||
}
|
||||
}
|
||||
@@ -473,6 +491,19 @@ pub fn request_thumb(app: &mut LightcraftApp, id: PhotoId, size: usize, priority
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a photo's grid thumbnail is already the picture it should show, so the render of the raw
|
||||
/// behind it is skipped. `embedded_is_final` is the engine's answer
|
||||
/// ([`lightcraft_engine::media::Session::embedded_thumb_is_final`]: the photo is an unedited raw
|
||||
/// whose camera JPEG the viewer is showing) and `from_embedded` says the stand-in actually came from
|
||||
/// the embedded preview rather than a cached render of the file.
|
||||
///
|
||||
/// Skipping matters where the file is expensive to read: decoding a 56 MB Fuji RAF to draw a 512 px
|
||||
/// thumbnail — which the 4 416 × 2 944 embedded JPEG already covers — cost 45 s a photo over a
|
||||
/// 1.2 MB/s share, and the queue it built delayed the render the user was waiting for.
|
||||
pub(crate) fn skips_the_real_render(embedded_is_final: bool, from_embedded: bool) -> bool {
|
||||
embedded_is_final && from_embedded
|
||||
}
|
||||
|
||||
/// Rendered thumbnails replacing embedded previews: after everything on screen.
|
||||
pub const BACKGROUND_THUMB_PRIORITY: u32 = 3;
|
||||
|
||||
|
||||
@@ -177,6 +177,8 @@ pub struct Renderer {
|
||||
/// Last thumbnail inputs; weak identities do not retain photo histories or force deep clones.
|
||||
/// (photo, size bucket, job key, whether the request went through an embedded stand-in)
|
||||
thumb_inputs: HashMap<PhotoId, (std::sync::Weak<lightcraft_catalog::Photo>, usize, u64, bool)>,
|
||||
/// Photos whose grid thumbnail is the camera's embedded preview itself (no render behind it).
|
||||
embedded_thumbs: HashMap<PhotoId, (std::sync::Weak<lightcraft_catalog::Photo>, usize, u64)>,
|
||||
pool: JobPool<Slot, RenderResult>,
|
||||
/// Jobs run elsewhere (see [`RenderOffload`]); `queue` holds the ones not started yet.
|
||||
offload: Option<Box<dyn RenderOffload>>,
|
||||
@@ -254,6 +256,7 @@ impl Renderer {
|
||||
preview_generation: None,
|
||||
pool: JobPool::new(threads),
|
||||
thumb_inputs: HashMap::new(),
|
||||
embedded_thumbs: HashMap::new(),
|
||||
offload: None,
|
||||
queue: Vec::new(),
|
||||
seq: 0,
|
||||
@@ -298,6 +301,30 @@ impl Renderer {
|
||||
}
|
||||
|
||||
/// Remember the inputs of the thumbnail just requested (`quick`: through a stand-in).
|
||||
/// Whether the camera's embedded preview *is* this photo's thumbnail: its quick render arrived
|
||||
/// (so `Slot::ThumbQuick` holds the picture) and no edit has moved the photo on to a settings
|
||||
/// hash of its own. The grid stops here instead of decoding the raw behind a thumbnail the
|
||||
/// embedded JPEG already covers — see `grid::skips_the_real_render`.
|
||||
pub fn thumb_quick_is_final(&self, photo: &Arc<lightcraft_catalog::Photo>, bucket: usize, priority: u32) -> bool {
|
||||
let Some((old, size, key)) = self.embedded_thumbs.get(&photo.id) else { return false };
|
||||
old.as_ptr() == Arc::as_ptr(photo) && *size == bucket && !self.request_needed(Slot::ThumbQuick(photo.id), *key, priority)
|
||||
}
|
||||
|
||||
/// Whether this photo's embedded-preview thumbnail is being fetched right now: the grid has
|
||||
/// asked, and neither the picture nor a render behind it is owed.
|
||||
pub fn thumb_quick_pending(&self, photo: &Arc<lightcraft_catalog::Photo>, bucket: usize) -> bool {
|
||||
let Some((old, size, _, quick)) = self.thumb_inputs.get(&photo.id) else { return false };
|
||||
old.as_ptr() == Arc::as_ptr(photo) && *size == bucket && *quick && self.pending.contains_key(&Slot::ThumbQuick(photo.id))
|
||||
}
|
||||
|
||||
/// Remember that the camera's embedded preview is this photo's final thumbnail, so the grid's
|
||||
/// per-frame requests stop at it (the quick render's key, which [`Self::thumb_quick_is_final`]
|
||||
/// compares against).
|
||||
pub fn remember_embedded_thumb(&mut self, photo: &Arc<lightcraft_catalog::Photo>, bucket: usize, key: u64) {
|
||||
self.remember_thumb(photo, bucket, key, true);
|
||||
self.embedded_thumbs.insert(photo.id, (Arc::downgrade(photo), bucket, key));
|
||||
}
|
||||
|
||||
pub fn remember_thumb(&mut self, photo: &Arc<lightcraft_catalog::Photo>, bucket: usize, key: u64, quick: bool) {
|
||||
if self.thumb_inputs.len() >= 1024 && !self.thumb_inputs.contains_key(&photo.id) {
|
||||
self.thumb_inputs.retain(|_, (p, _, _, _)| p.strong_count() > 0);
|
||||
@@ -1032,8 +1059,9 @@ mod thumbnail_tests {
|
||||
|
||||
#[test]
|
||||
fn raw_quick_path_frames_build_no_jobs() {
|
||||
// an unedited raw: the grid shows its embedded preview first (held here until released),
|
||||
// then queues the real thumbnail behind everything on screen
|
||||
// an unedited raw: the grid shows its embedded preview (held here until released) and stops
|
||||
// there — the camera JPEG at 4 416 × 2 944 already covers a 512 px thumbnail, so decoding the
|
||||
// whole raw behind it only costs the file read (45 s a photo on a 1.2 MB/s share)
|
||||
let gate = Arc::new(std::sync::Mutex::new(()));
|
||||
let held = gate.lock().unwrap();
|
||||
let mut s = Session::new();
|
||||
@@ -1061,14 +1089,30 @@ mod thumbnail_tests {
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
app.renderer.poll(&ctx, &mut app.session);
|
||||
}
|
||||
// the stand-in is shown: one more job queues the real thumbnail at background priority,
|
||||
// and the grid asking at its own (higher) priority every frame does not rebuild it
|
||||
// the embedded preview is the thumbnail: no real render is queued behind it, and the grid
|
||||
// asking every frame neither rebuilds the job nor re-requests the preview
|
||||
for _ in 0..180 {
|
||||
request(&mut app);
|
||||
}
|
||||
assert_eq!(app.renderer.thumb_jobs_built, 2);
|
||||
assert_eq!(app.renderer.queue.len(), 1);
|
||||
assert_eq!(app.renderer.pending.get(&Slot::Thumb(PhotoId(1))).map(|p| p.1), Some(crate::panels::grid::BACKGROUND_THUMB_PRIORITY));
|
||||
assert_eq!(app.renderer.thumb_jobs_built, 1, "an unedited raw's embedded preview is the thumbnail");
|
||||
assert!(app.renderer.queue.is_empty(), "nothing is decoded behind it");
|
||||
assert!(!app.renderer.is_pending(Slot::Thumb(PhotoId(1))));
|
||||
|
||||
// an edit takes the embedded preview out of the picture: the real thumbnail is rendered then
|
||||
let mut edited = (*photo()).clone();
|
||||
edited.kind = lightcraft_catalog::MediaKind::Raw;
|
||||
edited.develop = Arc::new({
|
||||
let mut d = edited.camera_defaults();
|
||||
d.light.exposure = 0.5;
|
||||
d
|
||||
});
|
||||
let (mut app, _, _ctx) = fixture_with(Session::new(), edited);
|
||||
app.renderer.set_offload(Box::new(Blocked));
|
||||
for _ in 0..180 {
|
||||
request(&mut app);
|
||||
}
|
||||
assert_eq!(app.renderer.thumb_jobs_built, 1);
|
||||
assert!(app.renderer.is_pending(Slot::Thumb(PhotoId(1))) || !app.renderer.queue.is_empty(), "an edited raw renders its thumbnail");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -155,3 +155,20 @@ fn unloaded_thumbnails_show_their_selection() {
|
||||
}
|
||||
assert!(checked >= 3, "checked {checked}");
|
||||
}
|
||||
|
||||
/// A folder of unedited raws on a slow share: the grid must not decode every raw to draw a 512 px
|
||||
/// thumbnail the embedded camera JPEG already covers. Measured on a 56 MB Fuji RAF over a 1.2 MB/s
|
||||
/// share, that decode is 45 s a photo, and 149 of them queued in front of the render the user asked
|
||||
/// for. The embedded JPEG stays a *stand-in* — replaced by a real render — whenever it is not the
|
||||
/// picture the photo should show.
|
||||
#[test]
|
||||
fn an_unedited_raws_thumbnail_skips_the_sensor_decode() {
|
||||
use crate::panels::grid::skips_the_real_render;
|
||||
// unedited raw, embedded camera JPEG on screen: nothing behind it to render
|
||||
assert!(skips_the_real_render(true, true));
|
||||
// a cached render of the file was the stand-in, even for an unedited raw: the real one follows
|
||||
assert!(!skips_the_real_render(true, false));
|
||||
// edited photo, or a raw with no embedded JPEG: render it
|
||||
assert!(!skips_the_real_render(false, true));
|
||||
assert!(!skips_the_real_render(false, false));
|
||||
}
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,82 @@
|
||||
# Reading a raw's preview without reading the raw
|
||||
|
||||
A grid thumbnail and a loupe preview of an unedited raw both show the camera's embedded JPEG, not a
|
||||
decode of the sensor data. That JPEG lives near the *start* of the file, but the rest of the file is
|
||||
the sensor data — for a 24 MP Fuji RAF, **50 of 56 MB**. Until this change the preview loader called
|
||||
`std::fs::read` on the whole file and threw the sensor data away, so every preview paid for bytes
|
||||
nobody looked at.
|
||||
|
||||
That is free on a local SSD and expensive on a network share, which is where photographers keep
|
||||
libraries. Measured on one real file (`DSCF9492.RAF`, 56.5 MB) over a GVFS/SMB share on Wi-Fi:
|
||||
|
||||
| | read | time |
|
||||
|---|---|---|
|
||||
| whole file | 59 277 632 B (100 %) | ~10 s |
|
||||
| header + preview | **5 857 257 B (9.9 %)** | **0.96 s** |
|
||||
|
||||
## How it works
|
||||
|
||||
1. `read_prefix` reads the first `PREVIEW_PREFIX` (4 KiB); every format whose preview is locatable
|
||||
from the header keeps its pointer well inside that.
|
||||
2. For Fuji RAF the header's two big-endian `u32`s at **84** (offset) and **88** (length) are the
|
||||
same fields `lightcraft_raw`'s RAF reader uses. When they declare more than the prefix holds, the
|
||||
prefix is re-read once at exactly that extent: `lightcraft_raw::bounded_preview` reports the range
|
||||
its buffer covers (`slice` clamps to `b.len()`), so a short prefix yields a *plausible but
|
||||
truncated* range — a real 4 KiB prefix reports 3 948 bytes where the file holds 5.6 MB.
|
||||
3. `embedded_preview_from_range` checks the window really starts with a JPEG SOI and trims it at the
|
||||
last EOI marker, exactly as `embedded_preview` trims the whole-file answer.
|
||||
4. When the prefix already covers the range — the usual case, since it was grown to it — the window
|
||||
is a slice of the prefix. Reading it again would double a 5.6 MB preview.
|
||||
|
||||
Formats whose preview pointer is not in the header (CR3 scans boxes; a TIFF pointer tag can sit in
|
||||
an IFD the prefix cuts short; CRW/MRW/X3F are found by scanning) keep the whole-file path, so
|
||||
nothing regresses: `preview_window` returns `None` and the loader reads the file as before. A pointer
|
||||
that lies, a short read or a file that changes underneath us ends in the same fallback — the user's
|
||||
preview is never lost to a damaged header.
|
||||
|
||||
`LIGHTCRAFT_PROFILE=1` prints what each preview cost:
|
||||
|
||||
```
|
||||
[profile] preview …/DSCF9492.RAF: read 5857257 of 59277632 bytes (header 5857257, window 5857109)
|
||||
```
|
||||
|
||||
## The grid thumbnail is the embedded JPEG too
|
||||
|
||||
A 512 px grid thumbnail of an unedited raw now keeps the camera's own JPEG instead of rendering the
|
||||
sensor behind it. The grid showed that JPEG first in any case and then queued a decode of the whole
|
||||
raw to replace it — for a folder of 149 Fuji RAFs that is 8.3 GB of traffic, 45 s a photo at
|
||||
1.2 MB/s, and a queue that delayed the render the user was actually waiting for (measured: the first
|
||||
loupe render of a cold photo finished 174 s after the click, most of it spent behind thumbnails).
|
||||
|
||||
- `Session::embedded_thumb_is_final(photo)` says whether the photo is an unedited raw whose camera
|
||||
JPEG the viewer is showing (the same "unedited raw" gate as the preview path: `has_import_look`).
|
||||
- `grid::skips_the_real_render(embedded_is_final, from_embedded)` is the decision, unit-tested:
|
||||
a **cached render** as the stand-in, an **edited photo** and a **raw with no embedded JPEG** all
|
||||
still render, because their thumbnail must show the pipeline's output rather than the camera's.
|
||||
- The thumbnail key still changes with the settings hash, so editing a photo re-renders its
|
||||
thumbnail through the ordinary background path.
|
||||
|
||||
Measured on the real folder: idle for a minute, the app read 12 previews (**26.9 MB**) and decoded
|
||||
**zero** raws for thumbnails, against 638 MB of whole-file reads for the same 12 photos before. The
|
||||
render queue stayed empty.
|
||||
|
||||
## Tests
|
||||
|
||||
- `crates/raw/src/preview.rs`: the header-only range matches `embedded_preview` byte for byte; a
|
||||
clamped prefix, a padded extent, a late pointer, a lying pointer and a non-JPEG window are all
|
||||
covered.
|
||||
- `crates/engine/src/files.rs`: on a synthetic RAF with a 200 KB tail, the quick path and the
|
||||
whole-file path produce identical pixels, and the read stays under half the file.
|
||||
- `crates/engine/tests/preview_read.rs` (opt-in, `#[ignore]`): the same measurement on a **real**
|
||||
file, typically on a share:
|
||||
|
||||
```sh
|
||||
LC_PREVIEW_RAW="/mnt/nas/…/DSCF0001.RAF" \
|
||||
cargo test --release -p lightcraft-engine --test preview_read -- --ignored --nocapture
|
||||
```
|
||||
|
||||
No raw fixture is committed (media never is, see `AGENTS.md`).
|
||||
|
||||
A network share is still slower than a local disk — the preview itself is 5.6 MB — and the sensor
|
||||
data still has to come over the wire for the *developed* picture, which is a separate path
|
||||
(`fs_pair_loader`, the full decode). This change removes the preview's share of it, nothing more.
|
||||
Reference in New Issue
Block a user