diff --git a/README.md b/README.md
index 1cf517475..2a6e72be3 100644
--- a/README.md
+++ b/README.md
@@ -224,9 +224,10 @@ LightCraft is young and moving fast — see the [roadmap](ROADMAP.md) for estima
| RAW: DNG, CR2, ARW, NEF (uncompressed) | ✅ · compressed NEF, CR3, RAF, ORF, RW2, PEF… 🚧 |
| Detail: sharpening, luminance + colour noise reduction | ✅ · AI Denoise, Super Resolution ⬜ |
| Remove / Heal / Clone spots (auto source) | ✅ · content-aware fill (PatchMatch), Red Eye 🚧 |
-| Export: JPEG / PNG / TIFF / WebP / AVIF, sizing, file-size limit, output sharpening, naming, batch | ✅ · metadata policy, watermark, DNG ⬜ |
-| MCP server, CLI, control channel | ✅ |
-| Optics, Geometry/Upright, GPU pipeline, persistent library | 🚧 |
+| Export: JPEG / PNG / TIFF / WebP / AVIF, sizing, file-size limit, output sharpening, naming, batch, metadata policy, text watermark | ✅ · DNG export, image watermark ⬜ |
+| Library persistence (crash-safe op log + snapshots), disk thumbnail cache, import with duplicate detection | ✅ |
+| MCP server (headless or live app, persistent libraries), CLI, control channel | ✅ |
+| Optics, Geometry/Upright, XMP sidecars, GPU pipeline | 🚧 |
| Web build (same UI in the browser via WASM; in-memory imports, export downloads) | ✅ · persistence, workers 🚧 |
diff --git a/ROADMAP.md b/ROADMAP.md
index 584ed3e7c..91d7f54df 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -11,7 +11,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
| M0 | Skeleton + visual shell | workspace, xtask CI + layering, geom/color/raster, develop model, pipeline v0, catalog v0, engine commands, Lightroom-look UI (grid, loupe, filmstrip, Edit panel), control channel, MCP, web build | 3–5 | 🚧 (UI, pipeline, engine, control channel ✅; MCP/CLI, web 🚧) |
| M1 | Library core | import (JPEG/PNG/TIFF/WebP), EXIF/XMP, persistent catalog (op log + snapshots), albums, ratings/flags/labels, filter/search/sort, thumbnail cache, 100k-photo grid | 6–10 | 🚧 (import, codecs, EXIF/XMP, albums, search ✅; persistence 🚧) |
| M2 | Pipeline v1 (quality) | WB temp/tint, profiles, local tone mapping (highlights/shadows), curves, HSL, point colour, colour grading, texture/clarity/dehaze, vignette, grain, B&W, auto tone/WB, histogram, before/after | 10–15 | ⬜ |
-| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW ✅) |
+| M3 | RAW I | TIFF/DNG (LJ92, deflate, tiles, opcodes), demosaic (AHD/PPG/bilinear), highlight recovery, DNG colour model, CR2, NEF, ARW, embedded previews | 10–15 | 🚧 (DNG, CR2, ARW, NEF uncompressed, embedded previews ✅; NEF Huffman ⬜) |
| M4 | Crop, geometry, optics | crop tool + overlays, straighten, Upright (auto/level/vertical/full/guided), manual transforms, CA, defringe, manual lens corrections | 6–10 | ⬜ |
| M5 | Performance | source pyramids, wgpu compute pipeline (CPU oracle), draft/full renders, prefetch, budgets (16 ms slider updates on 24 MP) | 10–15 | ⬜ |
| M6 | Masking | brush, linear/radial gradients, colour/luminance/depth range, add/subtract/intersect/invert, all local adjustments, masks panel | 8–12 | ⬜ |
@@ -19,7 +19,7 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
| M8 | Heal / Remove | content-aware remove (PatchMatch), heal, clone, brush spots, visualize spots, red/pet eye | 6–10 | ⬜ |
| M9 | Presets, profiles, versions, sync | preset browser + amount, create/import presets, profile browser, versions, history, copy/paste/sync settings | 5–8 | ⬜ |
| M10 | Export & share | export dialog (JPEG/PNG/TIFF/DNG/AVIF/JXL/original), sizing, sharpening, metadata, watermark, naming, batch jobs, XMP sidecars, HDR export | 6–10 | ⬜ |
-| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | ⬜ |
+| M11 | RAW II | CR3, RAF (X-Trans), ORF, RW2, PEF, SRW, 3FR, IIQ + long tail; camera calibration DB; HEIC/AVIF/JXL import | 20–35 | 🚧 (RAF uncompressed, RW2 packed, PEF, ORF uncompressed ✅) |
| M12 | AI & smart features | subject/sky/background/people/object masks, semantic search, faces/People (permissively licensed models, pure-Rust inference) | 20–40 | ⬜ |
| M13 | Merge | HDR merge (deghost), panorama (projections, boundary warp, fill edges), HDR panorama | 10–15 | ⬜ |
| M14 | Video | import/playback/trim via FilmCraft crates, global edits + presets on video, video export | 6–10 | ⬜ |
@@ -44,6 +44,30 @@ projects (DrawCraft reached its first four milestones in ≈ 4½ h) and are revi
HEVC (HEIC).
- **Look parity** with Adobe's default rendering is subjective tuning against our own reference targets.
+## Raw format coverage and known gaps
+
+Decoded (CC0 corpus from raw.pixls.us, `cargo xtask corpus --download`, `crates/raw/tests/corpus.rs`): DNG (uncompressed,
+LJ92, Deflate, float, linear), CR2, ARW (uncompressed, ARW2, LJ92), NEF/NRW uncompressed, RAF uncompressed (Bayer and
+X-Trans), RW2 packed 12/14-bit, PEF (uncompressed and Huffman), ORF uncompressed (16-bit and 12-bit packed). Every
+supported container also yields its embedded JPEG preview (CR3 too), and the engine shows that preview for raw variants
+it can't decode yet.
+
+Not decoded yet — preview only (no permissively licensed description; black-box analysis incomplete):
+- **Nikon Huffman NEF** (lossless / lossy). Black-box findings so far: maker note `0x0096` holds per-parity predictor
+ seeds (2048 for 14-bit lossless); ≈ 9 bits per pixel; the category-0 code word is a 6-bit rotation of `011111`. The
+ Huffman tables are not stored in the files, and an automatic table search (beam search over prefix codes scored by
+ smoothness and bit rate, validated on Pentax files whose tables *are* stored) did not converge.
+- **Panasonic RW2 raw format 4** (quantised): the block layout is known (0x4000-byte chunks rotated by 0x1ff8; 128-bit
+ blocks of two 12-bit seeds plus four groups of a 2-bit scale and three 8-bit codes; scales 0/1 are ×1/×2 differences
+ from the same-colour pixel two to the left), the reconstruction rule for scales 2/3 is not established.
+- **Olympus compressed ORF**, **Fujifilm compressed RAF**, **Canon CR3/CRX** (M11.1), **Canon sRAW/mRAW**, lossy DNG.
+
+**Camera colour matrices:** non-DNG raws use the documented neutral fallback (camera RGB ≈ linear sRGB, flagged
+`matrix_is_fallback`) with the file's as-shot white-balance multipliers. Clean sources to evaluate next: manufacturer
+matrices stored in the files themselves (Olympus ImageProcessing `ColorMatrix`, Pentax/Panasonic equivalents) and our
+own chart-based calibration (M11.4). Adobe matrices are never used.
+
## Log
- 2026-09-30: roadmap created; M0 in progress; research docs (Lightroom reference, Rust imaging ecosystem) complete.
- 2026-09-30 (later): app running with the full Lightroom-style UI; pipeline v0; DNG/CR2/ARW; README showcase. ≈ 8 h elapsed.
+- 2026-10-01: RAW II formats: RAF (uncompressed Bayer + X-Trans), RW2 (packed), PEF (incl. Huffman), ORF (uncompressed); embedded previews for every container incl. CR3; raw corpus test with 37 CC0 samples.
diff --git a/apps/lightcraft-web/src/web.rs b/apps/lightcraft-web/src/web.rs
index 21817cda9..c902a2589 100644
--- a/apps/lightcraft-web/src/web.rs
+++ b/apps/lightcraft-web/src/web.rs
@@ -89,6 +89,9 @@ fn services(store: MemStore, ctx: egui::Context) -> Services {
open_picker(store.clone(), ctx.clone());
Vec::new() // files arrive asynchronously and are imported on a later frame
})),
+ // Preset files: browser pickers are asynchronous; not wired on the web yet.
+ pick_preset_files: None,
+ save_preset_file: None,
write: Some(Box::new(download)),
png: Some(Box::new(|img: &lightcraft_raster::Rgba8| {
lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(img), &lightcraft_codecs::EncodeMeta::default()).unwrap_or_default()
diff --git a/apps/lightcraft/src/main.rs b/apps/lightcraft/src/main.rs
index 5b39ee4e6..0d10bab7e 100644
--- a/apps/lightcraft/src/main.rs
+++ b/apps/lightcraft/src/main.rs
@@ -78,13 +78,37 @@ fn services() -> Services {
Services {
pick_files: Some(Box::new(|| {
rfd::FileDialog::new()
- .add_filter("Photos", &["jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "nef", "arw", "psd", "jxl", "gif", "bmp"])
+ .add_filter(
+ "Photos",
+ &[
+ "jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "pef", "psd",
+ "jxl", "gif", "bmp",
+ ],
+ )
.pick_files()
.unwrap_or_default()
.into_iter()
.map(|p| p.to_string_lossy().to_string())
.collect()
})),
+ pick_preset_files: Some(Box::new(|| {
+ rfd::FileDialog::new()
+ .set_title("Import Presets")
+ .add_filter("Presets", &["lcpreset", "xmp"])
+ .pick_files()
+ .unwrap_or_default()
+ .into_iter()
+ .map(|p| p.to_string_lossy().to_string())
+ .collect()
+ })),
+ save_preset_file: Some(Box::new(|name: &str| {
+ rfd::FileDialog::new()
+ .set_title("Export Presets")
+ .add_filter("LightCraft Preset", &["lcpreset"])
+ .set_file_name(name)
+ .save_file()
+ .map(|p| p.to_string_lossy().to_string())
+ })),
write: Some(Box::new(|p: &str, b: &[u8]| {
if let Some(dir) = std::path::Path::new(p).parent().filter(|d| !d.as_os_str().is_empty()) {
std::fs::create_dir_all(dir).map_err(|e| e.to_string())?;
diff --git a/crates/engine/src/cmd/mod.rs b/crates/engine/src/cmd/mod.rs
index 3b3cd5e24..3e1b8c540 100644
--- a/crates/engine/src/cmd/mod.rs
+++ b/crates/engine/src/cmd/mod.rs
@@ -8,7 +8,9 @@ mod develop;
mod edit;
mod library;
mod masks;
+mod preset_files;
mod query;
+mod xmp;
use serde::Serialize;
use serde_json::Value;
@@ -101,6 +103,8 @@ pub fn command_specs() -> &'static [CommandSpec] {
v.extend(develop::specs());
v.extend(masks::specs());
v.extend(query::specs());
+ v.extend(xmp::specs());
+ v.extend(preset_files::specs());
v
})
}
diff --git a/crates/engine/src/cmd/preset_files.rs b/crates/engine/src/cmd/preset_files.rs
new file mode 100644
index 000000000..71f32af32
--- /dev/null
+++ b/crates/engine/src/cmd/preset_files.rs
@@ -0,0 +1,86 @@
+//! Preset files: import `.lcpreset` / XMP presets, export `.lcpreset`.
+
+use serde_json::{Value, json};
+
+use super::{CommandSpec, always, bad, cmd, str_param};
+use crate::presets::{LCPRESET_EXT, expand_preset_paths, parse_preset_file, to_lcpreset};
+use crate::{Result, Session};
+
+fn strings(p: &Value, key: &str) -> Option> {
+ p.get(key).and_then(Value::as_array).map(|a| a.iter().filter_map(Value::as_str).map(str::to_string).collect())
+}
+
+fn import(s: &mut Session, p: &Value) -> Result {
+ let paths = strings(p, "paths").filter(|v| !v.is_empty()).ok_or_else(|| bad("preset.import", "no paths"))?;
+ let mut imported = Vec::new();
+ let mut failed = Vec::new();
+ let mut skipped = 0usize;
+ for f in expand_preset_paths(&paths) {
+ let parsed = std::fs::read(&f).map_err(|e| e.to_string()).and_then(|b| parse_preset_file(&f, &b));
+ match parsed {
+ Ok(list) => {
+ let n = list.len();
+ let ids = s.add_presets(list);
+ skipped += n - ids.len();
+ for id in ids {
+ if let Some(pr) = s.presets.iter().find(|x| x.id == id) {
+ imported.push(json!({"id": pr.id, "name": pr.name, "group": pr.group}));
+ }
+ }
+ }
+ Err(e) => failed.push(json!([f, e])),
+ }
+ }
+ Ok(json!({"imported": imported, "skipped": skipped, "failed": failed}))
+}
+
+fn export(s: &mut Session, p: &Value) -> Result {
+ let path = str_param(p, "path").filter(|x| !x.trim().is_empty()).ok_or_else(|| bad("preset.export", "missing path"))?;
+ let mut path = std::path::PathBuf::from(path);
+ if path.extension().is_none() {
+ path.set_extension(LCPRESET_EXT);
+ }
+ let ids = strings(p, "ids");
+ let group = str_param(p, "group");
+ let chosen: Vec<_> = s
+ .presets
+ .iter()
+ .filter(|x| match (&ids, group) {
+ (Some(ids), _) => ids.contains(&x.id),
+ (None, Some(g)) => x.group == g,
+ (None, None) => !x.builtin,
+ })
+ .cloned()
+ .collect();
+ if chosen.is_empty() {
+ return Err(bad("preset.export", "no presets to export (create one first, or pass ids/group)"));
+ }
+ if let Some(dir) = path.parent().filter(|d| !d.as_os_str().is_empty()) {
+ std::fs::create_dir_all(dir).map_err(|e| crate::EngineError::Other(format!("{}: {e}", dir.display())))?;
+ }
+ std::fs::write(&path, to_lcpreset(&chosen)).map_err(|e| crate::EngineError::Other(format!("{}: {e}", path.display())))?;
+ Ok(json!({"path": path.display().to_string(), "count": chosen.len()}))
+}
+
+pub fn specs() -> Vec {
+ vec![
+ cmd!(
+ "preset.import",
+ "Import Presets",
+ [],
+ None,
+ "{paths: [.lcpreset / .xmp file or folder]} — groups preserved; XMP presets map crs: fields (docs/xmp-interop.md) → {imported: [{id,name,group}], skipped, failed}",
+ always,
+ import
+ ),
+ cmd!(
+ "preset.export",
+ "Export Presets",
+ [],
+ None,
+ "{path, ids?: [presetId], group?: name} (default: all user presets) → {path, count}; writes a .lcpreset JSON file",
+ always,
+ export
+ ),
+ ]
+}
diff --git a/crates/engine/src/cmd/xmp.rs b/crates/engine/src/cmd/xmp.rs
new file mode 100644
index 000000000..7b7d4325e
--- /dev/null
+++ b/crates/engine/src/cmd/xmp.rs
@@ -0,0 +1,93 @@
+//! XMP sidecar commands: save/read metadata to/from files, sidecar preferences.
+
+use lightcraft_catalog::Op;
+use serde_json::{Value, json};
+
+use super::{CommandSpec, always, bad, cmd, has_selection, str_param};
+use crate::sidecar::SidecarNaming;
+use crate::{Result, Session};
+
+fn save(s: &mut Session, p: &Value) -> Result {
+ let mut written = Vec::new();
+ let mut failed = Vec::new();
+ for id in s.targets(p) {
+ match s.save_sidecar(id) {
+ Ok(path) => written.push(path.display().to_string()),
+ Err(e) => failed.push(json!({"id": id.0, "error": e.to_string()})),
+ }
+ }
+ Ok(json!({"written": written, "failed": failed}))
+}
+
+fn read(s: &mut Session, p: &Value) -> Result {
+ let mut ops = Vec::new();
+ let mut read = Vec::new();
+ let mut failed = Vec::new();
+ for id in s.targets(p) {
+ match s.read_sidecar_op(id) {
+ Ok(Some((op, from))) => {
+ ops.push(op);
+ read.push(json!({"id": id.0, "from": from.display().to_string()}));
+ }
+ Ok(None) => failed.push(json!({"id": id.0, "error": "no XMP sidecar or embedded XMP"})),
+ Err(e) => failed.push(json!({"id": id.0, "error": e.to_string()})),
+ }
+ }
+ if !ops.is_empty() {
+ s.commit("Read Metadata from File", Op::Batch { ops })?;
+ }
+ Ok(json!({"read": read, "failed": failed}))
+}
+
+fn prefs(s: &mut Session, p: &Value) -> Result {
+ if let Some(a) = p.get("autoWrite").and_then(Value::as_bool) {
+ s.xmp.auto_write = a;
+ }
+ if let Some(n) = str_param(p, "naming") {
+ s.xmp.naming = match n {
+ "stem" => SidecarNaming::Stem,
+ "full" => SidecarNaming::Full,
+ other => return Err(bad("library.xmpPreferences", format!("unknown naming `{other}` (stem|full)"))),
+ };
+ }
+ if p.as_object().is_some_and(|o| !o.is_empty()) {
+ s.save_prefs()?;
+ }
+ Ok(serde_json::to_value(s.xmp).unwrap_or_default())
+}
+
+pub fn specs() -> Vec {
+ vec![
+ cmd!(
+ "photo.saveMetadataToFile",
+ "Save Metadata to File",
+ ["Photo"],
+ Some("Cmd+S"),
+ "{ids?} — writes each photo's XMP sidecar (metadata + develop settings) next to the original → {written: [path], failed}",
+ has_selection,
+ save
+ ),
+ cmd!(
+ "photo.readMetadataFromFile",
+ "Read Metadata from File",
+ ["Photo"],
+ None,
+ "{ids?} — reads each photo's XMP sidecar (or a raw/DNG's embedded XMP): metadata and develop settings, one undo step → {read, failed}",
+ has_selection,
+ read
+ ),
+ cmd!(
+ "library.xmpPreferences",
+ "XMP Sidecar Preferences",
+ [],
+ None,
+ "{autoWrite?: bool, naming?: stem (IMG_1.xmp, default) | full (IMG_1.CR3.xmp)} → current preferences (saved with the library)",
+ always,
+ prefs
+ ),
+ cmd!("library.toggleAutoWriteXmp", "Automatically Write Changes into XMP", ["File"], None, "{} → {autoWrite}", always, |s, _| {
+ let on = !s.xmp.auto_write;
+ prefs(s, &json!({"autoWrite": on}))
+ }),
+ ]
+}
diff --git a/crates/engine/src/crs.rs b/crates/engine/src/crs.rs
new file mode 100644
index 000000000..d35641e7e
--- /dev/null
+++ b/crates/engine/src/crs.rs
@@ -0,0 +1,446 @@
+//! Interchange: camera-raw-settings (`crs:`) develop fields from XMP → our develop settings.
+//!
+//! Other raw developers store their edits as `crs:` properties in XMP sidecars, in DNG-embedded
+//! XMP and in XMP preset files. This module maps the commonly used, documented-by-observation
+//! fields to a *partial* [`DevelopSettings`](lightcraft_develop::DevelopSettings) JSON object
+//! (only the fields present in the packet), which is then merged like a preset. Our pipeline is
+//! not theirs, so the result is a best-effort approximation of the look, not a pixel match.
+//! The full table is in `docs/xmp-interop.md`.
+//!
+//! Implemented from the public XMP specification (ISO 16684-1) and black-box observation of what
+//! each field means; no third-party code or preset content was used.
+
+use std::collections::BTreeMap;
+
+use lightcraft_develop::{MIXER_BANDS, Preset};
+use serde_json::{Map, Value, json};
+
+/// Properties keyed `prefix:name` (as produced by [`lightcraft_meta::parse_xmp`]).
+pub type Props = BTreeMap>;
+
+/// Band names as they appear in `crs:` field names, in our mixer order.
+const CRS_BANDS: [&str; 8] = ["Red", "Orange", "Yellow", "Green", "Aqua", "Blue", "Purple", "Magenta"];
+
+/// `crs:` fields that describe the packet rather than an adjustment.
+const NON_ADJUSTMENT: &[&str] = &[
+ "crs:Version",
+ "crs:ProcessVersion",
+ "crs:Name",
+ "crs:ShortName",
+ "crs:Group",
+ "crs:UUID",
+ "crs:PresetType",
+ "crs:SupportsAmount",
+ "crs:SupportsColor",
+ "crs:SupportsMonochrome",
+ "crs:SupportsHighDynamicRange",
+ "crs:SupportsNormalDynamicRange",
+ "crs:SupportsSceneReferred",
+ "crs:SupportsOutputReferred",
+ "crs:CameraModelRestriction",
+ "crs:Copyright",
+ "crs:ContactInfo",
+ "crs:HasSettings",
+ "crs:AlreadyApplied",
+ "crs:RawFileName",
+ "crs:HasCrop",
+];
+
+/// True if the packet carries any `crs:` adjustment (not just bookkeeping fields).
+pub fn has_adjustments(props: &Props) -> bool {
+ if props.get("crs:AlreadyApplied").and_then(|v| v.first()).is_some_and(|s| s.eq_ignore_ascii_case("true")) {
+ // The pixels already contain these settings (e.g. an exported/rendered file).
+ return false;
+ }
+ props.keys().any(|k| k.starts_with("crs:") && !NON_ADJUSTMENT.contains(&k.split('/').next().unwrap_or(k)))
+}
+
+fn first<'a>(props: &'a Props, k: &str) -> Option<&'a str> {
+ props.get(k).and_then(|v| v.first()).map(|s| s.trim()).filter(|s| !s.is_empty())
+}
+
+fn num(props: &Props, k: &str) -> Option {
+ let s = first(props, k)?;
+ let s = s.strip_prefix('+').unwrap_or(s);
+ if let Some((n, d)) = s.split_once('/') {
+ let (n, d) = (n.trim().parse::().ok()?, d.trim().parse::().ok()?);
+ return (d != 0.0).then_some(n / d).filter(|v| v.is_finite());
+ }
+ s.parse::().ok().filter(|v| v.is_finite())
+}
+
+fn boolean(props: &Props, k: &str) -> Option {
+ first(props, k).map(|s| s.eq_ignore_ascii_case("true") || s == "1")
+}
+
+/// Insert `v` at the dotted `path` of a JSON object, creating intermediate objects.
+fn put(out: &mut Value, path: &str, v: Value) {
+ let mut cur = out;
+ let mut parts = path.split('.').peekable();
+ while let Some(p) = parts.next() {
+ let Value::Object(m) = cur else { return };
+ if parts.peek().is_none() {
+ m.insert(p.to_string(), v);
+ return;
+ }
+ cur = m.entry(p.to_string()).or_insert_with(|| Value::Object(Map::new()));
+ }
+}
+
+/// Relative temperature (−100..100, used for rendered files) → Kelvin, matching the UI's scale
+/// (a mired shift around 6500 K).
+pub fn rel_to_kelvin(r: f64) -> f64 {
+ 1e6 / (1e6 / 6500.0 - r.clamp(-100.0, 100.0) * 0.8)
+}
+
+/// Parse a point curve stored as an `rdf:Seq` of `"x, y"` strings in 0..255.
+fn curve(props: &Props, k: &str) -> Option {
+ let items = props.get(k)?;
+ let mut pts = Vec::new();
+ for it in items {
+ let (x, y) = it.split_once(',')?;
+ let (x, y) = (x.trim().parse::().ok()?, y.trim().parse::().ok()?);
+ pts.push(json!({"x": (x / 255.0).clamp(0.0, 1.0), "y": (y / 255.0).clamp(0.0, 1.0)}));
+ }
+ if pts.len() < 2 {
+ return None;
+ }
+ // A straight 0,0 → 255,255 line is the identity: store it as "no curve".
+ let identity = pts.len() == 2 && items.iter().all(|it| it.split(',').map(|v| v.trim()).collect::>().windows(2).all(|w| w[0] == w[1]));
+ Some(if identity { json!([]) } else { Value::Array(pts) })
+}
+
+/// Map the `crs:` fields of an XMP packet to a partial develop-settings JSON object.
+///
+/// `raw`: the target is a raw file (absolute Kelvin white balance) — `Some(false)` prefers the
+/// relative `Incremental*` white balance used for rendered files, `None` (presets) takes whichever
+/// is present, absolute first.
+pub fn to_partial(props: &Props, raw: Option) -> Value {
+ let mut out = Value::Object(Map::new());
+ let o = &mut out;
+ let n = |o: &mut Value, crs: &str, path: &str| {
+ if let Some(v) = num(props, &format!("crs:{crs}")) {
+ put(o, path, json!(v));
+ }
+ };
+ // ---- Light (process version 2012+ names)
+ n(o, "Exposure2012", "light.exposure");
+ n(o, "Contrast2012", "light.contrast");
+ n(o, "Highlights2012", "light.highlights");
+ n(o, "Shadows2012", "light.shadows");
+ n(o, "Whites2012", "light.whites");
+ n(o, "Blacks2012", "light.blacks");
+ // ---- Presence
+ n(o, "Texture", "effects.texture");
+ n(o, "Clarity2012", "effects.clarity");
+ n(o, "Dehaze", "effects.dehaze");
+ n(o, "Vibrance", "color.vibrance");
+ n(o, "Saturation", "color.saturation");
+
+ // ---- White balance
+ let mode = first(props, "crs:WhiteBalance").map(|m| match m.to_ascii_lowercase().replace(' ', "").as_str() {
+ "asshot" => "asShot",
+ "auto" => "auto",
+ "daylight" => "daylight",
+ "cloudy" => "cloudy",
+ "shade" => "shade",
+ "tungsten" => "tungsten",
+ "fluorescent" => "fluorescent",
+ "flash" => "flash",
+ _ => "custom",
+ });
+ let abs = num(props, "crs:Temperature").map(|t| (t, num(props, "crs:Tint")));
+ let rel = num(props, "crs:IncrementalTemperature")
+ .or(num(props, "crs:IncrementalTint").map(|_| 0.0))
+ .map(|t| (rel_to_kelvin(t), num(props, "crs:IncrementalTint")));
+ let wb = match raw {
+ Some(true) => abs,
+ Some(false) => rel.or(abs),
+ None => abs.or(rel),
+ };
+ match (mode, wb) {
+ (Some(m), _) if m != "custom" => put(o, "wb.mode", json!(m)),
+ (m, Some((t, tint))) => {
+ put(o, "wb.mode", json!(m.unwrap_or("custom")));
+ put(o, "wb.temp", json!(t));
+ if let Some(tint) = tint {
+ put(o, "wb.tint", json!(tint));
+ }
+ }
+ (Some(m), None) => put(o, "wb.mode", json!(m)),
+ (None, None) => {}
+ }
+
+ // ---- Treatment + B&W mix
+ if let Some(bw) = boolean(props, "crs:ConvertToGrayscale") {
+ put(o, "treatment", json!(if bw { "bw" } else { "color" }));
+ }
+ for (band, crs) in MIXER_BANDS.iter().zip(CRS_BANDS) {
+ n(o, &format!("HueAdjustment{crs}"), &format!("mixer.{band}.hue"));
+ n(o, &format!("SaturationAdjustment{crs}"), &format!("mixer.{band}.sat"));
+ n(o, &format!("LuminanceAdjustment{crs}"), &format!("mixer.{band}.lum"));
+ n(o, &format!("GrayMixer{crs}"), &format!("bw_mix.{band}"));
+ }
+
+ // ---- Tone curve: parametric regions + point curves
+ n(o, "ParametricShadows", "curve.shadows");
+ n(o, "ParametricDarks", "curve.darks");
+ n(o, "ParametricLights", "curve.lights");
+ n(o, "ParametricHighlights", "curve.highlights");
+ n(o, "ParametricShadowSplit", "curve.split_shadows");
+ n(o, "ParametricMidtoneSplit", "curve.split_mid");
+ n(o, "ParametricHighlightSplit", "curve.split_highlights");
+ for (crs, ch) in
+ [("ToneCurvePV2012", "master"), ("ToneCurvePV2012Red", "red"), ("ToneCurvePV2012Green", "green"), ("ToneCurvePV2012Blue", "blue")]
+ {
+ if let Some(c) = curve(props, &format!("crs:{crs}")) {
+ put(o, &format!("curve.{ch}"), c);
+ }
+ }
+
+ // ---- Color grading (split toning fields are shared with the older split-toning panel)
+ n(o, "SplitToningShadowHue", "grading.shadows.hue");
+ n(o, "SplitToningShadowSaturation", "grading.shadows.sat");
+ n(o, "ColorGradeShadowLum", "grading.shadows.lum");
+ n(o, "SplitToningHighlightHue", "grading.highlights.hue");
+ n(o, "SplitToningHighlightSaturation", "grading.highlights.sat");
+ n(o, "ColorGradeHighlightLum", "grading.highlights.lum");
+ n(o, "ColorGradeMidtoneHue", "grading.midtones.hue");
+ n(o, "ColorGradeMidtoneSat", "grading.midtones.sat");
+ n(o, "ColorGradeMidtoneLum", "grading.midtones.lum");
+ n(o, "ColorGradeGlobalHue", "grading.global.hue");
+ n(o, "ColorGradeGlobalSat", "grading.global.sat");
+ n(o, "ColorGradeGlobalLum", "grading.global.lum");
+ n(o, "ColorGradeBlending", "grading.blending");
+ n(o, "SplitToningBalance", "grading.balance");
+
+ // ---- Detail
+ n(o, "Sharpness", "detail.sharpen_amount");
+ n(o, "SharpenRadius", "detail.sharpen_radius");
+ n(o, "SharpenDetail", "detail.sharpen_detail");
+ n(o, "SharpenEdgeMasking", "detail.sharpen_masking");
+ n(o, "LuminanceSmoothing", "detail.nr_luminance");
+ n(o, "LuminanceNoiseReductionDetail", "detail.nr_detail");
+ n(o, "LuminanceNoiseReductionContrast", "detail.nr_contrast");
+ n(o, "ColorNoiseReduction", "detail.nr_color");
+ n(o, "ColorNoiseReductionDetail", "detail.nr_color_detail");
+ n(o, "ColorNoiseReductionSmoothness", "detail.nr_color_smoothness");
+
+ // ---- Effects: post-crop vignette + grain
+ n(o, "PostCropVignetteAmount", "vignette.amount");
+ n(o, "PostCropVignetteMidpoint", "vignette.midpoint");
+ n(o, "PostCropVignetteRoundness", "vignette.roundness");
+ n(o, "PostCropVignetteFeather", "vignette.feather");
+ n(o, "PostCropVignetteHighlightContrast", "vignette.highlights");
+ if let Some(st) = num(props, "crs:PostCropVignetteStyle") {
+ let style = match st as i64 {
+ 2 => "colorPriority",
+ 3 => "paintOverlay",
+ _ => "highlightPriority",
+ };
+ put(o, "vignette.style", json!(style));
+ }
+ n(o, "GrainAmount", "grain.amount");
+ n(o, "GrainSize", "grain.size");
+ n(o, "GrainFrequency", "grain.roughness");
+
+ // ---- Optics (manual corrections; lens profiles are ours, only the switch carries over)
+ if let Some(b) = boolean(props, "crs:LensProfileEnable") {
+ put(o, "optics.lens_profile", json!(b));
+ }
+ if let Some(b) = boolean(props, "crs:AutoLateralCA") {
+ put(o, "optics.remove_ca", json!(b));
+ }
+ n(o, "LensManualDistortionAmount", "optics.distortion");
+ n(o, "VignetteAmount", "optics.vignetting");
+ n(o, "VignetteMidpoint", "optics.vignetting_midpoint");
+ n(o, "DefringePurpleAmount", "optics.defringe_purple_amount");
+ n(o, "DefringePurpleHueLo", "optics.defringe_purple_hue_lo");
+ n(o, "DefringePurpleHueHi", "optics.defringe_purple_hue_hi");
+ n(o, "DefringeGreenAmount", "optics.defringe_green_amount");
+ n(o, "DefringeGreenHueLo", "optics.defringe_green_hue_lo");
+ n(o, "DefringeGreenHueHi", "optics.defringe_green_hue_hi");
+
+ // ---- Geometry
+ n(o, "PerspectiveVertical", "geometry.vertical");
+ n(o, "PerspectiveHorizontal", "geometry.horizontal");
+ n(o, "PerspectiveRotate", "geometry.rotate");
+ n(o, "PerspectiveScale", "geometry.scale");
+ n(o, "PerspectiveAspect", "geometry.aspect");
+ n(o, "PerspectiveX", "geometry.offset_x");
+ n(o, "PerspectiveY", "geometry.offset_y");
+ if let Some(u) = num(props, "crs:PerspectiveUpright") {
+ let mode = match u as i64 {
+ 1 => "auto",
+ 2 => "level",
+ 3 => "vertical",
+ 4 => "full",
+ 5 => "guided",
+ _ => "off",
+ };
+ put(o, "geometry.upright", json!(mode));
+ }
+
+ // ---- Crop (normalized edges of the unrotated image + straighten angle)
+ match boolean(props, "crs:HasCrop") {
+ Some(true) => {
+ let e = |k: &str, d: f64| num(props, &format!("crs:Crop{k}")).unwrap_or(d).clamp(0.0, 1.0);
+ let (l, t, r, b) = (e("Left", 0.0), e("Top", 0.0), e("Right", 1.0), e("Bottom", 1.0));
+ if r > l && b > t {
+ put(
+ o,
+ "crop.geometry",
+ json!({"rect": {"x0": l, "y0": t, "x1": r, "y1": b}, "angle": num(props, "crs:CropAngle").unwrap_or(0.0).clamp(-45.0, 45.0)}),
+ );
+ }
+ }
+ Some(false) => put(o, "crop.geometry", json!({"rect": {"x0": 0.0, "y0": 0.0, "x1": 1.0, "y1": 1.0}, "angle": 0.0})),
+ None => {}
+ }
+ out
+}
+
+/// Read an XMP preset (`crs:` fields + `crs:Name` / `crs:Group`) into one of our presets.
+/// Returns `None` when the packet has no adjustments we understand.
+pub fn preset_from_xmp(xmp: &str, fallback_name: &str) -> Option {
+ let d = lightcraft_meta::parse_xmp(xmp).ok()?;
+ let props = &d.properties;
+ let settings = to_partial(props, None);
+ if settings.as_object().is_none_or(Map::is_empty) {
+ return None;
+ }
+ let name = first(props, "crs:Name").unwrap_or(fallback_name).to_string();
+ let group = first(props, "crs:Group").unwrap_or("Imported Presets").to_string();
+ let uuid = first(props, "crs:UUID").map(|u| u.to_ascii_lowercase());
+ let id = format!("user.xmp.{}", uuid.unwrap_or_else(|| crate::presets::slug(&format!("{group}-{name}"))));
+ Some(Preset { id, name, group, settings, favorite: false, builtin: false })
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use lightcraft_develop::{DevelopSettings, Upright, VignetteStyle, WbMode, apply_partial};
+
+ /// A hand-written sidecar in attribute form (as many tools write it).
+ const SIDECAR: &str = r#"
+
+
+
+ 0, 064, 52192, 205255, 255
+
+
+ 0, 0255, 255
+
+
+
+"#;
+
+ fn props(x: &str) -> Props {
+ lightcraft_meta::parse_xmp(x).unwrap().properties
+ }
+
+ #[test]
+ fn maps_common_fields() {
+ let p = props(SIDECAR);
+ assert!(has_adjustments(&p));
+ let partial = to_partial(&p, Some(true));
+ let s = apply_partial(&DevelopSettings::for_raw(5000.0, 0.0), &partial, 1.0);
+ assert_eq!(s.wb.mode, WbMode::Custom);
+ assert_eq!((s.wb.temp, s.wb.tint), (5150.0, 12.0));
+ assert_eq!(s.light.exposure, 0.65);
+ assert_eq!((s.light.contrast, s.light.highlights, s.light.shadows, s.light.whites, s.light.blacks), (-14.0, -58.0, 41.0, 9.0, -17.0));
+ assert_eq!((s.effects.texture, s.effects.clarity, s.effects.dehaze), (6.0, 11.0, 4.0));
+ assert_eq!((s.color.vibrance, s.color.saturation), (19.0, -3.0));
+ assert_eq!(s.mixer.orange.hue, -7.0);
+ assert_eq!(s.mixer.blue.sat, -22.0);
+ assert_eq!(s.mixer.aqua.lum, 13.0);
+ assert_eq!((s.curve.shadows, s.curve.highlights, s.curve.split_mid), (5.0, -8.0, 55.0));
+ assert_eq!(s.curve.master.len(), 4);
+ assert!((s.curve.master[1].x - 64.0 / 255.0).abs() < 1e-9 && (s.curve.master[1].y - 52.0 / 255.0).abs() < 1e-9);
+ assert!(s.curve.red.is_empty(), "identity channel curve maps to no curve");
+ assert_eq!((s.grading.shadows.hue, s.grading.shadows.sat), (210.0, 14.0));
+ assert_eq!((s.grading.highlights.hue, s.grading.highlights.sat), (40.0, 9.0));
+ assert_eq!((s.grading.midtones.hue, s.grading.midtones.sat), (120.0, 6.0));
+ assert_eq!((s.grading.balance, s.grading.blending), (20.0, 70.0));
+ assert_eq!((s.detail.sharpen_amount, s.detail.sharpen_radius, s.detail.sharpen_detail, s.detail.sharpen_masking), (55.0, 1.2, 30.0, 12.0));
+ assert_eq!((s.detail.nr_luminance, s.detail.nr_color), (18.0, 25.0));
+ assert_eq!(s.vignette.amount, -21.0);
+ assert_eq!(s.vignette.style, VignetteStyle::ColorPriority);
+ assert_eq!((s.grain.amount, s.grain.size, s.grain.roughness), (15.0, 30.0, 60.0));
+ assert_eq!(s.geometry.upright, Upright::Level);
+ assert_eq!(s.geometry.vertical, -10.0);
+ assert!(s.optics.remove_ca);
+ assert_eq!(s.treatment, lightcraft_develop::Treatment::Color);
+ let c = s.crop.geometry;
+ assert_eq!((c.rect.x0, c.rect.y0, c.rect.x1, c.rect.y1, c.angle), (0.05, 0.1, 0.8, 0.9, 1.5));
+ }
+
+ #[test]
+ fn only_present_fields_are_in_the_partial() {
+ let x = r#""#;
+ let partial = to_partial(&props(x), None);
+ assert_eq!(partial, json!({"color": {"vibrance": 30.0}, "grain": {"amount": 10.0}}));
+ }
+
+ #[test]
+ fn element_form_bw_and_incremental_wb() {
+ let x = r#"
+
+ True
+ -35
+ 25
+ -6
+ 4800
+ "#;
+ let p = props(x);
+ let rendered = apply_partial(&DevelopSettings::default(), &to_partial(&p, Some(false)), 1.0);
+ assert_eq!(rendered.treatment, lightcraft_develop::Treatment::Bw);
+ assert_eq!(rendered.bw_mix.blue, -35.0);
+ assert!((rendered.wb.temp - rel_to_kelvin(25.0)).abs() < 1e-6 && rendered.wb.temp > 6500.0);
+ assert_eq!(rendered.wb.tint, -6.0);
+ let raw = apply_partial(&DevelopSettings::default(), &to_partial(&p, Some(true)), 1.0);
+ assert_eq!(raw.wb.temp, 4800.0);
+ }
+
+ #[test]
+ fn named_wb_and_already_applied() {
+ let x = r#""#;
+ let p = props(x);
+ assert_eq!(to_partial(&p, Some(true)), json!({"wb": {"mode": "daylight"}}));
+ assert!(!has_adjustments(&p));
+ let bookkeeping = r#""#;
+ assert!(!has_adjustments(&props(bookkeeping)));
+ }
+
+ #[test]
+ fn xmp_preset_with_name_and_group() {
+ // Written for this test: a minimal preset-shaped packet with localized name/group.
+ let x = r#"
+
+ Punchy Test
+ My Looks
+ "#;
+ let p = preset_from_xmp(x, "file").unwrap();
+ assert_eq!((p.name.as_str(), p.group.as_str(), p.id.as_str()), ("Punchy Test", "My Looks", "user.xmp.0123abcd"));
+ assert_eq!(p.settings, json!({"light": {"contrast": 25.0}}));
+ assert!(!p.builtin);
+ assert!(preset_from_xmp("", "f").is_none());
+ }
+}
diff --git a/crates/engine/src/files.rs b/crates/engine/src/files.rs
index 5645341e1..c4dfa0772 100644
--- a/crates/engine/src/files.rs
+++ b/crates/engine/src/files.rs
@@ -92,7 +92,26 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result {
let m = lightcraft_meta::extract(bytes);
let (meta, captured) = meta_of(&m);
if lightcraft_raw::probe(bytes).is_some() {
- let raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;
+ let raw = match lightcraft_raw::decode(bytes) {
+ Ok(r) => r,
+ Err(lightcraft_raw::RawError::Unsupported(why)) => {
+ // a raw variant we can't decode yet: describe it from its embedded preview
+ let (w, h) = embedded_preview_size(bytes).ok_or(format!("unsupported raw ({why}) without an embedded preview"))?;
+ return Ok(ProbeInfo {
+ width: w,
+ height: h,
+ format: ext_upper(name),
+ kind: MediaKind::Raw,
+ file_size: bytes.len() as u64,
+ captured,
+ meta,
+ as_shot_wb: None,
+ content_hash,
+ ..Default::default()
+ });
+ }
+ Err(e) => return Err(e.to_string()),
+ };
let (mut w, mut h) = (raw.crop.width.max(1) as u32, raw.crop.height.max(1) as u32);
if w <= 1 || h <= 1 {
(w, h) = (raw.active_area.width as u32, raw.active_area.height as u32);
@@ -114,6 +133,7 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result {
meta,
as_shot_wb,
content_hash,
+ xmp: lightcraft_meta::embedded(bytes).xmp,
});
}
let fmt = lightcraft_codecs::sniff(bytes).ok_or("unrecognized file format")?;
@@ -142,13 +162,21 @@ pub fn probe_bytes(name: &str, bytes: &[u8]) -> Result {
as_shot_wb: None,
content_hash,
embedded_lens: None,
+ xmp: None,
})
}
/// Decode a file into a linear Rec.2020 image no larger than `max_edge`, oriented.
pub fn load_bytes(bytes: &[u8], max_edge: usize) -> Result<(Rgb32f, SourceInfo), String> {
if lightcraft_raw::probe(bytes).is_some() {
- let mut raw = lightcraft_raw::decode(bytes).map_err(|e| e.to_string())?;
+ let mut raw = match lightcraft_raw::decode(bytes) {
+ Ok(r) => r,
+ Err(lightcraft_raw::RawError::Unsupported(why)) => {
+ // show the camera's embedded JPEG (rendered, not raw) until the variant is supported
+ return load_embedded_preview(bytes, max_edge).ok_or(format!("unsupported raw ({why}) without an embedded preview"));
+ }
+ Err(e) => return Err(e.to_string()),
+ };
// Embedded lens corrections are applied by the pipeline ("Enable Profile Corrections"), not baked in.
let lens = embedded_lens(&raw);
raw.opcodes.list3.retain(|op| !is_lens_opcode(op));
@@ -178,6 +206,34 @@ pub fn load_bytes(bytes: &[u8], max_edge: usize) -> Result<(Rgb32f, SourceInfo),
Ok((img.oriented(Orientation::from_exif(d.orientation)), SourceInfo::default()))
}
+/// Orientation for an embedded preview: its own EXIF orientation when it has one, else the raw file's.
+fn preview_orientation(raw_bytes: &[u8], jpeg_orientation: u16) -> Orientation {
+ if jpeg_orientation > 1 {
+ return Orientation::from_exif(jpeg_orientation);
+ }
+ lightcraft_meta::extract(raw_bytes).orientation.unwrap_or(Orientation::Normal)
+}
+
+/// Oriented size of the embedded preview of a raw file.
+fn embedded_preview_size(bytes: &[u8]) -> Option<(u32, u32)> {
+ let jpeg = lightcraft_raw::embedded_preview(bytes)?;
+ let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(64, 64)).ok()?;
+ let (mut w, mut h) = (d.source_width, d.source_height);
+ if preview_orientation(bytes, d.orientation).swaps_axes() {
+ std::mem::swap(&mut w, &mut h);
+ }
+ Some((w, h))
+}
+
+/// The embedded preview of a raw file as a working-space image no larger than `max_edge`, oriented.
+pub fn load_embedded_preview(bytes: &[u8], max_edge: usize) -> Option<(Rgb32f, SourceInfo)> {
+ let jpeg = lightcraft_raw::embedded_preview(bytes)?;
+ let d = lightcraft_codecs::decode(&jpeg, lightcraft_codecs::DecodeOptions::fit(max_edge as u32, max_edge as u32)).ok()?;
+ let img = d.to_working();
+ let img = if img.width.max(img.height) > max_edge { fit(&img, max_edge, max_edge, Filter::Mitchell) } else { img };
+ Some((img.oriented(preview_orientation(bytes, d.orientation)), SourceInfo::default()))
+}
+
/// Filesystem-backed hooks (native). On the web the host installs bytes-based hooks instead.
pub fn fs_hooks() -> (FileLoader, FileProbe) {
let loader: FileLoader = Arc::new(|path: &str, max_edge: usize| {
@@ -200,3 +256,36 @@ impl crate::Session {
self
}
}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use lightcraft_codecs::{ChromaSubsampling, EncodeImage, EncodeMeta, Samples, encode_jpeg};
+
+ /// A CR3-shaped file (not decodable yet) whose only content is a `PRVW` preview box.
+ fn cr3_with_preview(w: u32, h: u32) -> Vec {
+ let px: Vec = (0..w * h).flat_map(|i| [(i % 251) as u8, 128, 200]).collect();
+ let jpeg = encode_jpeg(&EncodeImage::new(w, h, 3, Samples::U8(&px)), 90, ChromaSubsampling::S444, &EncodeMeta::default()).unwrap();
+ let mut b = b"\0\0\0\x18ftypcrx \0\0\0\x01crx isom".to_vec();
+ b.extend_from_slice(&((24 + jpeg.len()) as u32).to_be_bytes());
+ b.extend_from_slice(b"PRVW\0\0\0\0\0\x01");
+ b.extend_from_slice(&(w as u16).to_be_bytes());
+ b.extend_from_slice(&(h as u16).to_be_bytes());
+ b.extend_from_slice(b"\0\x01");
+ b.extend_from_slice(&(jpeg.len() as u32).to_be_bytes());
+ b.extend_from_slice(&jpeg);
+ b
+ }
+
+ #[test]
+ fn unsupported_raw_falls_back_to_embedded_preview() {
+ let b = cr3_with_preview(48, 32);
+ let p = probe_bytes("x.cr3", &b).unwrap();
+ assert_eq!((p.width, p.height, p.kind, p.format.as_str()), (48, 32, MediaKind::Raw, "CR3"));
+ let (img, src) = load_bytes(&b, 24).unwrap();
+ assert_eq!((img.width, img.height), (24, 16));
+ assert!(!src.raw);
+ // no preview at all: a clear error, not a panic
+ assert!(load_bytes(b"\0\0\0\x18ftypcrx \0\0\0\x01", 24).is_err());
+ }
+}
diff --git a/crates/engine/src/import.rs b/crates/engine/src/import.rs
index f02028555..3aac02034 100644
--- a/crates/engine/src/import.rs
+++ b/crates/engine/src/import.rs
@@ -8,7 +8,9 @@
//! 4. Skip **duplicates by content** (same bytes already in the library, or twice in this batch).
//! 5. *Add* in place (the photo points at the original file) or *copy* into the library's
//! `Originals/YYYY/YYYY-MM-DD/` folder (names made unique) and point at the copy.
-//! 6. Commit all new photos as one undoable op.
+//! 6. Read each photo's XMP sidecar (or a raw/DNG file's embedded XMP): the sidecar wins for
+//! metadata and develop settings are restored (see [`crate::sidecar`]).
+//! 7. Commit all new photos as one undoable op.
use std::collections::HashMap;
use std::path::{Path, PathBuf};
@@ -21,8 +23,8 @@ use crate::media::ProbeInfo;
/// File extensions LightCraft imports (lower case).
pub const EXTENSIONS: &[&str] = &[
- "jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "arw", "raf", "orf", "rw2", "pef", "psd", "jxl", "gif", "bmp", "heic",
- "avif",
+ "jpg", "jpeg", "png", "tif", "tiff", "webp", "dng", "cr2", "cr3", "nef", "nrw", "arw", "raf", "orf", "rw2", "pef", "psd", "jxl", "gif", "bmp",
+ "heic", "avif",
];
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
@@ -51,6 +53,8 @@ pub struct ImportReport {
pub failed: Vec<(String, String)>,
/// Files found after expanding folders.
pub scanned: usize,
+ /// Photos whose metadata/develop settings were read from an XMP sidecar (or embedded XMP).
+ pub sidecars: usize,
}
pub fn is_supported(path: &Path) -> bool {
@@ -224,6 +228,20 @@ pub fn import(s: &mut Session, paths: &[String], mode: ImportMode) -> crate::Res
// built-in (file-embedded) lens corrections are on by default, like the camera intended
std::sync::Arc::make_mut(&mut p.develop).optics.lens_profile = true;
}
+ let raw = p.kind == lightcraft_catalog::MediaKind::Raw;
+ let packet = crate::sidecar::find_sidecar(&path, s.xmp.naming)
+ .and_then(|f| std::fs::read_to_string(f).ok())
+ .or_else(|| info.xmp.clone().filter(|_| raw));
+ if let Some(x) = packet {
+ match crate::sidecar::parse_sidecar(&x, raw) {
+ Ok(sc) if sc != crate::sidecar::SidecarData::default() => {
+ crate::sidecar::merge_into(&mut p, &sc, &now);
+ report.sidecars += 1;
+ }
+ Ok(_) => {}
+ Err(e) => log::warn!("import {path}: XMP: {e}"),
+ }
+ }
report.imported.push(id.0);
ops.push(Op::AddPhoto { photo: Box::new(p) });
}
diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs
index 01ae5da90..f40c07570 100644
--- a/crates/engine/src/lib.rs
+++ b/crates/engine/src/lib.rs
@@ -11,6 +11,7 @@
#![forbid(unsafe_code)]
pub mod cmd;
+pub mod crs;
pub mod demo;
pub mod export;
pub mod files;
@@ -18,6 +19,7 @@ pub mod import;
pub mod library;
pub mod media;
pub mod presets;
+pub mod sidecar;
mod view;
use std::sync::Arc;
@@ -90,6 +92,8 @@ pub struct Session {
pub active_mask: Option,
/// The persistent library this session writes to (`None` = in-memory only).
pub library: Option,
+ /// XMP sidecar preferences (persisted with the library).
+ pub xmp: sidecar::XmpPrefs,
}
impl Default for Session {
@@ -121,6 +125,7 @@ impl Session {
depth: 0,
active_mask: None,
library: None,
+ xmp: sidecar::XmpPrefs::default(),
}
}
@@ -137,6 +142,7 @@ impl Session {
(spec.enabled)(self).map_err(|why| EngineError::Disabled(id.to_string(), why))?;
let empty = Value::Object(Default::default());
let params = if params.is_null() { &empty } else { params };
+ let log_start = self.pending_log.len();
self.depth += 1;
let r = (spec.run)(self, params);
self.depth -= 1;
@@ -146,6 +152,9 @@ impl Session {
self.journal.drain(..1000);
}
}
+ if r.is_ok() && self.depth == 0 && self.xmp.auto_write && self.interaction.is_none() && self.pending_log.len() > log_start {
+ self.auto_write_sidecars(&self.pending_log[log_start..]);
+ }
if self.depth == 0 && self.library.is_some() {
// Make the command durable before reporting success (on failure the ops stay pending,
// are retried after the next command, and `library.info` reports the error).
@@ -311,3 +320,5 @@ mod tests;
mod tests_import;
#[cfg(test)]
mod tests_library;
+#[cfg(test)]
+mod tests_xmp;
diff --git a/crates/engine/src/library.rs b/crates/engine/src/library.rs
index 1ec5c4f41..e9979354c 100644
--- a/crates/engine/src/library.rs
+++ b/crates/engine/src/library.rs
@@ -5,6 +5,7 @@
//! LightCraft Library/
//! catalog.snap catalog.log (lightcraft-catalog journal)
//! presets.json view.json (user presets + favourites; last source/filter/sort/selection)
+//! prefs.json (library preferences: XMP sidecars)
//! thumbs/ (rendered thumbnail cache, safe to delete)
//! Originals/ (photos imported with "copy into library")
//! ```
@@ -72,6 +73,12 @@ impl Library {
}
}
+#[derive(Default, Serialize, Deserialize)]
+#[serde(default)]
+struct PrefsFile {
+ xmp: crate::sidecar::XmpPrefs,
+}
+
fn presets_json(presets: &[Preset]) -> String {
let f = PresetsFile {
user: presets.iter().filter(|p| !p.builtin).cloned().collect(),
@@ -115,6 +122,9 @@ impl Session {
}
}
}
+ // preferences
+ self.xmp =
+ std::fs::read(dir.join("prefs.json")).ok().and_then(|b| serde_json::from_slice::(&b).ok()).map(|p| p.xmp).unwrap_or_default();
// view state
if let Ok(bytes) = std::fs::read(dir.join("view.json"))
&& let Ok(v) = serde_json::from_slice::(&bytes)
@@ -189,6 +199,13 @@ impl Session {
Ok(())
}
+ /// Save the library preferences (no-op for in-memory sessions).
+ pub fn save_prefs(&self) -> Result<()> {
+ let Some(lib) = &self.library else { return Ok(()) };
+ let v = serde_json::to_vec_pretty(&PrefsFile { xmp: self.xmp }).unwrap_or_default();
+ write_atomic(&lib.dir, "prefs.json", &v).map_err(|e| EngineError::Other(format!("prefs: {e}")))
+ }
+
/// Compact the log into a snapshot now.
pub fn compact_library(&mut self) -> Result<()> {
self.persist()?;
diff --git a/crates/engine/src/media.rs b/crates/engine/src/media.rs
index 7c3285283..b4a4f6e6a 100644
--- a/crates/engine/src/media.rs
+++ b/crates/engine/src/media.rs
@@ -307,6 +307,8 @@ pub struct ProbeInfo {
pub content_hash: Option,
/// Lens corrections embedded in the file (DNG opcodes).
pub embedded_lens: Option,
+ /// The file's embedded XMP packet (raw/DNG files), for develop settings stored inside the file.
+ pub xmp: Option,
}
pub type FileProbe = Arc Result + Send + Sync>;
diff --git a/crates/engine/src/presets.rs b/crates/engine/src/presets.rs
index 86bbd0f97..f7994f061 100644
--- a/crates/engine/src/presets.rs
+++ b/crates/engine/src/presets.rs
@@ -1,8 +1,140 @@
-//! Built-in presets and profiles. All values are our own (no third-party preset content).
+//! Built-in presets and profiles (all values are our own; no third-party preset content), and
+//! preset files: our `.lcpreset` JSON (import + export, groups preserved) and XMP presets
+//! (`crs:` fields, read only — see [`crate::crs`]).
+
+use std::path::Path;
use lightcraft_develop::Preset;
-use serde::Serialize;
-use serde_json::json;
+use serde::{Deserialize, Serialize};
+use serde_json::{Value, json};
+
+use crate::Session;
+
+/// File extension of LightCraft preset files.
+pub const LCPRESET_EXT: &str = "lcpreset";
+/// The `format` tag of a `.lcpreset` file.
+pub const LCPRESET_FORMAT: &str = "lightcraft.preset";
+
+/// A `.lcpreset` file: one or more presets with their groups.
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
+pub struct PresetFile {
+ pub format: String,
+ pub version: u32,
+ pub presets: Vec,
+}
+
+/// Lower-case ASCII slug for ids (`"Warm & Soft"` → `warm-soft`).
+pub fn slug(s: &str) -> String {
+ let mut out = String::new();
+ for c in s.chars() {
+ if c.is_ascii_alphanumeric() {
+ out.push(c.to_ascii_lowercase());
+ } else if !out.ends_with('-') && !out.is_empty() {
+ out.push('-');
+ }
+ }
+ let out = out.trim_end_matches('-').to_string();
+ if out.is_empty() { "preset".into() } else { out }
+}
+
+/// Serialise presets as a `.lcpreset` file (favourite/built-in flags are not exported).
+pub fn to_lcpreset(presets: &[Preset]) -> String {
+ let presets = presets.iter().map(|p| Preset { favorite: false, builtin: false, ..p.clone() }).collect();
+ serde_json::to_string_pretty(&PresetFile { format: LCPRESET_FORMAT.into(), version: 1, presets }).unwrap_or_default()
+}
+
+/// Read a preset file by name: `.lcpreset` (a [`PresetFile`], a bare preset object or an array of
+/// presets) or `.xmp` (a `crs:` preset).
+pub fn parse_preset_file(name: &str, bytes: &[u8]) -> Result, String> {
+ let stem = Path::new(name).file_stem().map(|s| s.to_string_lossy().to_string()).unwrap_or_else(|| "Preset".into());
+ let ext = Path::new(name).extension().map(|e| e.to_string_lossy().to_ascii_lowercase()).unwrap_or_default();
+ let text = String::from_utf8_lossy(bytes);
+ let text = text.trim_start_matches('\u{feff}');
+ if ext == "xmp" || (ext != LCPRESET_EXT && text.trim_start().starts_with('<')) {
+ return crate::crs::preset_from_xmp(text, &stem).map(|p| vec![p]).ok_or_else(|| "no develop settings in this XMP file".into());
+ }
+ let v: Value = serde_json::from_str(text).map_err(|e| format!("not a preset file: {e}"))?;
+ let list = match &v {
+ Value::Object(o) if o.contains_key("presets") => {
+ let f: PresetFile = serde_json::from_value(v.clone()).map_err(|e| e.to_string())?;
+ if f.format != LCPRESET_FORMAT {
+ return Err(format!("unknown preset format `{}`", f.format));
+ }
+ f.presets
+ }
+ Value::Array(_) => serde_json::from_value(v).map_err(|e| e.to_string())?,
+ _ => vec![serde_json::from_value(v).map_err(|e| e.to_string())?],
+ };
+ let mut out = Vec::new();
+ for mut p in list {
+ if !p.settings.is_object() {
+ return Err(format!("preset `{}` has no settings object", p.name));
+ }
+ p.builtin = false;
+ p.favorite = false;
+ if p.name.trim().is_empty() {
+ p.name = stem.clone();
+ }
+ if p.group.trim().is_empty() {
+ p.group = "Imported Presets".into();
+ }
+ out.push(p);
+ }
+ Ok(out)
+}
+
+/// Expand files/folders (recursively) into preset files (`.lcpreset`, `.xmp`).
+pub fn expand_preset_paths(paths: &[String]) -> Vec {
+ fn walk(p: &Path, out: &mut Vec, top: bool) {
+ if p.is_dir() {
+ let Ok(rd) = std::fs::read_dir(p) else { return };
+ let mut v: Vec<_> = rd.flatten().map(|e| e.path()).collect();
+ v.sort();
+ for c in v {
+ if !c.file_name().is_some_and(|n| n.to_string_lossy().starts_with('.')) {
+ walk(&c, out, false);
+ }
+ }
+ } else {
+ let ext = p.extension().map(|e| e.to_string_lossy().to_ascii_lowercase()).unwrap_or_default();
+ if top || ext == LCPRESET_EXT || ext == "xmp" {
+ out.push(p.to_string_lossy().to_string());
+ }
+ }
+ }
+ let mut out = Vec::new();
+ for p in paths {
+ walk(Path::new(p), &mut out, true);
+ }
+ out
+}
+
+impl Session {
+ /// Add imported presets. Identical presets (same name, group and settings) already present are
+ /// skipped; id clashes get a fresh id. Returns the ids added.
+ pub fn add_presets(&mut self, presets: Vec) -> Vec {
+ let mut added = Vec::new();
+ for mut p in presets {
+ if self.presets.iter().any(|q| q.name == p.name && q.group == p.group && q.settings == p.settings) {
+ continue;
+ }
+ if p.id.trim().is_empty() || p.id.starts_with("lc.") || self.presets.iter().any(|q| q.id == p.id) {
+ let base = format!("user.{}", slug(&format!("{}-{}", p.group, p.name)));
+ let mut id = base.clone();
+ let mut n = 2;
+ while self.presets.iter().any(|q| q.id == id) {
+ id = format!("{base}-{n}");
+ n += 1;
+ }
+ p.id = id;
+ }
+ p.builtin = false;
+ added.push(p.id.clone());
+ self.presets.push(p);
+ }
+ added
+ }
+}
fn p(group: &str, id: &str, name: &str, settings: serde_json::Value) -> Preset {
Preset { id: format!("lc.{id}"), name: name.into(), group: group.into(), settings, favorite: false, builtin: true }
@@ -147,4 +279,94 @@ mod tests {
assert_ne!(out, base, "{} had no effect", pr.id);
}
}
+
+ #[test]
+ fn slugs() {
+ assert_eq!(slug("Warm & Soft!"), "warm-soft");
+ assert_eq!(slug(" "), "preset");
+ assert_eq!(slug("B&W/Film 2"), "b-w-film-2");
+ }
+
+ fn dir(tag: &str) -> std::path::PathBuf {
+ let d = std::env::temp_dir().join(format!("lc-presets-{tag}-{}", std::process::id()));
+ let _ = std::fs::remove_dir_all(&d);
+ std::fs::create_dir_all(&d).unwrap();
+ d
+ }
+
+ #[test]
+ fn lcpreset_export_import_roundtrip_keeps_groups() {
+ let d = dir("rt");
+ let mut s = Session::with_demo();
+ s.execute("develop.set", &json!({"control": "light.exposure", "value": 0.6})).unwrap();
+ s.execute("preset.create", &json!({"name": "Bright", "group": "Travel"})).unwrap();
+ s.execute("develop.set", &json!({"control": "effects.clarity", "value": 30})).unwrap();
+ s.execute("preset.create", &json!({"name": "Crisp", "group": "Street"})).unwrap();
+ let mine: Vec = s.presets.iter().filter(|p| !p.builtin).cloned().collect();
+ assert_eq!(mine.len(), 2);
+ let path = d.join("mine");
+ let r = s.execute("preset.export", &json!({"path": path.to_string_lossy()})).unwrap();
+ assert_eq!(r["count"], 2);
+ let file = d.join("mine.lcpreset");
+ assert!(file.is_file());
+ let one = d.join("travel.lcpreset");
+ s.execute("preset.export", &json!({"path": one.to_string_lossy(), "group": "Travel"})).unwrap();
+ assert!(s.execute("preset.export", &json!({"path": one.to_string_lossy(), "group": "Nope"})).is_err());
+
+ // a fresh session imports both files from the folder; the duplicate is skipped
+ let mut t = Session::new();
+ let r = t.execute("preset.import", &json!({"paths": [d.to_string_lossy()]})).unwrap();
+ assert_eq!(r["imported"].as_array().unwrap().len(), 2, "{r}");
+ assert_eq!(r["skipped"], 1, "{r}");
+ for p in &mine {
+ let q = t.presets.iter().find(|q| q.name == p.name).unwrap();
+ assert_eq!((&q.group, &q.settings, q.builtin), (&p.group, &p.settings, false));
+ }
+ // importing into the session that has them: everything is skipped, nothing duplicated
+ let n = s.presets.len();
+ let r = s.execute("preset.import", &json!({"paths": [file.to_string_lossy()]})).unwrap();
+ assert_eq!((r["skipped"].as_u64(), s.presets.len()), (Some(2), n));
+ let _ = std::fs::remove_dir_all(&d);
+ }
+
+ #[test]
+ fn id_clashes_get_fresh_ids_and_bad_files_are_reported() {
+ let d = dir("clash");
+ let builtin = builtin().remove(0);
+ let mut clash = builtin.clone();
+ clash.settings = json!({"light": {"exposure": 1.0}});
+ std::fs::write(d.join("a.lcpreset"), serde_json::to_string(&clash).unwrap()).unwrap();
+ std::fs::write(d.join("b.lcpreset"), "{not json").unwrap();
+ std::fs::write(d.join("c.lcpreset"), r#"{"format": "other", "version": 1, "presets": []}"#).unwrap();
+ std::fs::write(d.join("notes.txt"), "ignored inside folders").unwrap();
+ let mut s = Session::new();
+ let r = s.execute("preset.import", &json!({"paths": [d.to_string_lossy()]})).unwrap();
+ assert_eq!(r["failed"].as_array().unwrap().len(), 2, "{r}");
+ let id = r["imported"][0]["id"].as_str().unwrap();
+ assert_ne!(id, builtin.id);
+ assert!(id.starts_with("user."));
+ assert!(s.presets.iter().find(|p| p.id == builtin.id).unwrap().builtin, "built-in untouched");
+ let _ = std::fs::remove_dir_all(&d);
+ }
+
+ #[test]
+ fn xmp_preset_file_imports_and_applies() {
+ let d = dir("xmp");
+ // Hand-written for this test (not a third-party preset).
+ let x = r#"
+
+ Test Group
+ "#;
+ std::fs::write(d.join("Soft Lift.xmp"), x).unwrap();
+ let mut s = Session::with_demo();
+ let r = s.execute("preset.import", &json!({"paths": [d.join("Soft Lift.xmp").to_string_lossy()]})).unwrap();
+ let id = r["imported"][0]["id"].as_str().unwrap().to_string();
+ assert_eq!(r["imported"][0]["name"], "Soft Lift", "file name when crs:Name is missing");
+ assert_eq!(r["imported"][0]["group"], "Test Group");
+ s.execute("preset.apply", &json!({"id": id})).unwrap();
+ let dv = s.develop_of(s.active().unwrap()).unwrap();
+ assert_eq!((dv.light.exposure, dv.light.shadows, dv.grain.amount), (0.3, 20.0, 12.0));
+ let _ = std::fs::remove_dir_all(&d);
+ }
}
diff --git a/crates/engine/src/sidecar.rs b/crates/engine/src/sidecar.rs
new file mode 100644
index 000000000..c84c3eb11
--- /dev/null
+++ b/crates/engine/src/sidecar.rs
@@ -0,0 +1,369 @@
+//! XMP sidecars: `.xmp` (default) or `.xmp` next to the original.
+//!
+//! **Write** ([`Session::save_sidecar`], `photo.saveMetadataToFile`, or automatically after every
+//! change when [`XmpPrefs::auto_write`] is on): rating, colour label, title, caption, copyright,
+//! creator, keywords, capture time and GPS in the standard namespaces, plus our complete develop
+//! settings (`lc:settings`, JSON), the pick/reject flag (`lc:flag`) and location (`lc:location`).
+//!
+//! **Read** (on import, and `photo.readMetadataFromFile`): the sidecar wins for metadata; develop
+//! settings are restored from `lc:settings` when present, else mapped from interoperable `crs:`
+//! fields ([`crate::crs`]). For raw/DNG files without a sidecar the file's embedded XMP is used.
+
+use std::path::{Path, PathBuf};
+use std::sync::Arc;
+
+use lightcraft_catalog::{ColorLabel, Flag, MediaKind, Op, Photo, PhotoId, Source};
+use lightcraft_develop::DevelopSettings;
+use serde::{Deserialize, Serialize};
+use serde_json::Value;
+
+use crate::{EngineError, Result, Session};
+
+/// Sidecar file naming.
+#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "lowercase")]
+pub enum SidecarNaming {
+ /// `IMG_0001.xmp` (the common convention; shared by `IMG_0001.CR3` and `IMG_0001.JPG`).
+ #[default]
+ Stem,
+ /// `IMG_0001.CR3.xmp` (unambiguous when raw+JPEG pairs live side by side).
+ Full,
+}
+
+/// Library preferences for XMP sidecars.
+#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(default, rename_all = "camelCase")]
+pub struct XmpPrefs {
+ /// Write the sidecar after every change to a photo's metadata or develop settings.
+ pub auto_write: bool,
+ pub naming: SidecarNaming,
+}
+
+/// The sidecar path for an original, by naming convention.
+pub fn sidecar_path(original: &str, naming: SidecarNaming) -> PathBuf {
+ match naming {
+ SidecarNaming::Stem => Path::new(original).with_extension("xmp"),
+ SidecarNaming::Full => PathBuf::from(format!("{original}.xmp")),
+ }
+}
+
+/// An existing sidecar for `original`: the preferred naming first, then the other one (also
+/// accepting an upper-case `.XMP`).
+pub fn find_sidecar(original: &str, naming: SidecarNaming) -> Option {
+ let other = if naming == SidecarNaming::Stem { SidecarNaming::Full } else { SidecarNaming::Stem };
+ [naming, other]
+ .into_iter()
+ .flat_map(|n| {
+ let p = sidecar_path(original, n);
+ let upper = p.with_extension("XMP");
+ [p, upper]
+ })
+ .find(|p| p.is_file())
+}
+
+/// What a sidecar (or embedded XMP packet) says about a photo. `None` = not stated.
+#[derive(Clone, Debug, Default, PartialEq)]
+pub struct SidecarData {
+ pub rating: Option,
+ pub flag: Option,
+ pub label: Option>,
+ pub title: Option,
+ pub caption: Option,
+ pub copyright: Option,
+ pub creator: Option,
+ pub location: Option,
+ pub keywords: Option>,
+ pub develop: Option,
+}
+
+#[derive(Clone, Debug, PartialEq)]
+pub enum DevelopPatch {
+ /// Our own complete settings (`lc:settings`).
+ Full(Box),
+ /// Mapped `crs:` fields (partial JSON, merged like a preset).
+ Partial(Value),
+}
+
+/// Parse a sidecar / XMP packet. `raw` selects absolute (raw) vs relative white balance for `crs:`.
+pub fn parse_sidecar(xmp: &str, raw: bool) -> std::result::Result {
+ let d = lightcraft_meta::parse_xmp(xmp).map_err(|e| e.to_string())?;
+ let m = &d.metadata;
+ let lc = |k: &str| d.properties.get(&format!("lc:{k}")).and_then(|v| v.first()).cloned();
+ let mut out = SidecarData {
+ title: m.title.clone(),
+ caption: m.caption.clone(),
+ copyright: m.copyright.clone(),
+ creator: m.artist.clone(),
+ location: lc("location"),
+ keywords: (!m.keywords.is_empty()).then(|| m.keywords.clone()),
+ ..Default::default()
+ };
+ if let Some(r) = m.rating {
+ if r < 0 {
+ out.flag = Some(Flag::Reject);
+ out.rating = Some(0);
+ } else {
+ out.rating = Some(r.clamp(0, 5) as u8);
+ }
+ }
+ if let Some(f) = lc("flag").and_then(|f| Flag::parse(&f)) {
+ out.flag = Some(f);
+ }
+ if let Some(l) = &m.label {
+ out.label = Some(ColorLabel::parse(l.trim()));
+ }
+ let full = d.lc_settings.as_deref().and_then(|j| serde_json::from_str::(j).ok()).and_then(|v| DevelopSettings::from_json(&v).ok());
+ out.develop = match full {
+ Some(s) => Some(DevelopPatch::Full(Box::new(s))),
+ None if crate::crs::has_adjustments(&d.properties) => Some(DevelopPatch::Partial(crate::crs::to_partial(&d.properties, Some(raw)))),
+ None => None,
+ };
+ Ok(out)
+}
+
+/// Apply sidecar data to a photo record (the sidecar wins). Returns true if develop changed.
+pub fn merge_into(p: &mut Photo, sc: &SidecarData, now: &str) -> bool {
+ if let Some(r) = sc.rating {
+ p.rating = r;
+ }
+ if let Some(f) = sc.flag {
+ p.flag = f;
+ }
+ if let Some(l) = sc.label {
+ p.label = l;
+ }
+ let m = &mut p.meta;
+ for (src, dst) in [
+ (&sc.title, &mut m.title),
+ (&sc.caption, &mut m.caption),
+ (&sc.copyright, &mut m.copyright),
+ (&sc.creator, &mut m.creator),
+ (&sc.location, &mut m.location),
+ ] {
+ if let Some(v) = src {
+ *dst = v.clone();
+ }
+ }
+ if let Some(k) = &sc.keywords {
+ m.keywords = k.clone();
+ }
+ let develop = match &sc.develop {
+ Some(DevelopPatch::Full(s)) => (**s).clone(),
+ Some(DevelopPatch::Partial(v)) => lightcraft_develop::apply_partial(&p.develop, v, 1.0),
+ None => return false,
+ };
+ if develop == *p.develop {
+ return false;
+ }
+ p.develop = Arc::new(develop);
+ p.edited = Some(now.to_string());
+ true
+}
+
+fn label_name(l: ColorLabel) -> String {
+ format!("{l:?}")
+}
+
+/// The sidecar packet for a photo.
+pub fn sidecar_packet(p: &Photo) -> String {
+ let nz = |s: &str| (!s.trim().is_empty()).then(|| s.to_string());
+ let meta = lightcraft_meta::Metadata {
+ software: Some("LightCraft".into()),
+ title: nz(&p.meta.title),
+ caption: nz(&p.meta.caption),
+ copyright: nz(&p.meta.copyright),
+ artist: nz(&p.meta.creator),
+ keywords: p.meta.keywords.clone(),
+ rating: Some(p.rating.min(5) as i8),
+ label: p.label.map(label_name),
+ capture_time: p.captured.as_deref().and_then(lightcraft_meta::DateTime::parse_iso),
+ gps: p.meta.gps.map(|(latitude, longitude)| lightcraft_meta::Gps { latitude, longitude, altitude: None }),
+ ..Default::default()
+ };
+ let settings = serde_json::to_string(&*p.develop).unwrap_or_default();
+ let flag = match p.flag {
+ Flag::None => "none",
+ Flag::Pick => "pick",
+ Flag::Reject => "reject",
+ };
+ let mut lc = vec![("settings", settings.as_str()), ("flag", flag)];
+ if !p.meta.location.trim().is_empty() {
+ lc.push(("location", p.meta.location.as_str()));
+ }
+ lightcraft_meta::write_xmp_lc(&meta, &lc)
+}
+
+/// Write `data` to `path` atomically (temp file + rename).
+fn write_atomic(path: &Path, data: &[u8]) -> std::io::Result<()> {
+ let tmp = path.with_extension(format!("xmp.tmp-{}", std::process::id()));
+ std::fs::write(&tmp, data)?;
+ std::fs::rename(&tmp, path).inspect_err(|_| {
+ let _ = std::fs::remove_file(&tmp);
+ })
+}
+
+fn file_path(p: &Photo) -> Option<&str> {
+ match &p.source {
+ Source::File { path } => Some(path),
+ Source::Demo { .. } => None,
+ }
+}
+
+/// The XMP packet that describes a photo on disk: its sidecar, else (raw/DNG) the embedded XMP.
+pub fn read_packet(original: &str, kind: MediaKind, naming: SidecarNaming) -> Option<(String, PathBuf)> {
+ if let Some(sc) = find_sidecar(original, naming)
+ && let Ok(s) = std::fs::read_to_string(&sc)
+ {
+ return Some((s, sc));
+ }
+ if kind == MediaKind::Raw {
+ let bytes = std::fs::read(original).ok()?;
+ let x = lightcraft_meta::embedded(&bytes).xmp?;
+ return Some((x, PathBuf::from(original)));
+ }
+ None
+}
+
+/// Photo ids touched by an op (for auto-write).
+pub(crate) fn op_photos(op: &Op, out: &mut Vec) {
+ match op {
+ Op::SetRating { id, .. } | Op::SetFlag { id, .. } | Op::SetLabel { id, .. } | Op::SetDevelop { id, .. } | Op::SetMeta { id, .. } => {
+ if !out.contains(id) {
+ out.push(*id);
+ }
+ }
+ Op::Batch { ops } => ops.iter().for_each(|o| op_photos(o, out)),
+ _ => {}
+ }
+}
+
+impl Session {
+ /// Write the photo's XMP sidecar; returns its path.
+ pub fn save_sidecar(&self, id: PhotoId) -> Result {
+ let p = self.catalog.photo(id).ok_or(lightcraft_catalog::CatalogError::NoPhoto(id))?;
+ let orig = file_path(p).ok_or_else(|| EngineError::Other(format!("{} is not a file on disk", p.file_name)))?;
+ let path = sidecar_path(orig, self.xmp.naming);
+ write_atomic(&path, sidecar_packet(p).as_bytes()).map_err(|e| EngineError::Other(format!("{}: {e}", path.display())))?;
+ Ok(path)
+ }
+
+ /// The op that applies a photo's sidecar (or embedded XMP) to the catalog, if there is one.
+ pub fn read_sidecar_op(&self, id: PhotoId) -> Result> {
+ let p = self.catalog.photo(id).ok_or(lightcraft_catalog::CatalogError::NoPhoto(id))?;
+ let Some(orig) = file_path(p) else { return Ok(None) };
+ let Some((packet, from)) = read_packet(orig, p.kind, self.xmp.naming) else { return Ok(None) };
+ let sc = parse_sidecar(&packet, p.kind == MediaKind::Raw).map_err(|e| EngineError::Other(format!("{}: {e}", from.display())))?;
+ let mut q = (**p).clone();
+ let develop_changed = merge_into(&mut q, &sc, &(self.clock)());
+ let mut ops = vec![
+ Op::SetRating { id, rating: q.rating },
+ Op::SetFlag { id, flag: q.flag },
+ Op::SetLabel { id, label: q.label },
+ Op::SetMeta { id, meta: Box::new(q.meta.clone()) },
+ ];
+ if develop_changed {
+ ops.extend(self.develop_op(id, (*q.develop).clone(), "Read Metadata from File"));
+ }
+ Ok(Some((Op::Batch { ops }, from)))
+ }
+
+ /// Auto-write: sidecars for photos changed by `ops` (errors are logged, not returned).
+ pub(crate) fn auto_write_sidecars(&self, ops: &[Op]) {
+ let mut ids = Vec::new();
+ ops.iter().for_each(|o| op_photos(o, &mut ids));
+ for id in ids {
+ if self.catalog.photo(id).is_some_and(|p| file_path(p).is_some())
+ && let Err(e) = self.save_sidecar(id)
+ {
+ log::warn!("auto-write XMP: {e}");
+ }
+ }
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use lightcraft_catalog::Meta;
+
+ fn photo() -> Photo {
+ let mut p = Photo::new(PhotoId(7), Source::File { path: "/x/IMG_1.jpg".into() }, "IMG_1.jpg", "JPEG", 40, 30, "2026-01-01T00:00:00");
+ p.rating = 4;
+ p.flag = Flag::Pick;
+ p.label = Some(ColorLabel::Purple);
+ p.captured = Some("2025-06-07T08:09:10".into());
+ p.meta = Meta {
+ title: "Dune & sky".into(),
+ caption: "Line \nline two".into(),
+ copyright: "© 2026 Me".into(),
+ creator: "Ann Lee".into(),
+ location: "Namib".into(),
+ keywords: vec!["desert".into(), "red".into()],
+ gps: Some((-24.75, 15.3)),
+ ..Default::default()
+ };
+ let mut d = DevelopSettings::default();
+ d.light.exposure = 0.75;
+ d.effects.clarity = 22.0;
+ d.crop.geometry.angle = 2.5;
+ d.masks.push(lightcraft_develop::Mask { id: 1, ..Default::default() });
+ p.develop = Arc::new(d);
+ p
+ }
+
+ #[test]
+ fn naming() {
+ assert_eq!(sidecar_path("/a/b/IMG_1.CR3", SidecarNaming::Stem), PathBuf::from("/a/b/IMG_1.xmp"));
+ assert_eq!(sidecar_path("/a/b/IMG_1.CR3", SidecarNaming::Full), PathBuf::from("/a/b/IMG_1.CR3.xmp"));
+ }
+
+ #[test]
+ fn packet_roundtrip_restores_everything() {
+ let p = photo();
+ let x = sidecar_packet(&p);
+ let sc = parse_sidecar(&x, false).unwrap();
+ assert_eq!(sc.rating, Some(4));
+ assert_eq!(sc.flag, Some(Flag::Pick));
+ assert_eq!(sc.label, Some(Some(ColorLabel::Purple)));
+ let mut q = Photo::new(PhotoId(7), p.source.clone(), "IMG_1.jpg", "JPEG", 40, 30, "2026-01-01T00:00:00");
+ q.captured = p.captured.clone();
+ q.meta.gps = p.meta.gps;
+ assert!(merge_into(&mut q, &sc, "now"));
+ assert_eq!(q.develop, p.develop);
+ assert_eq!(q.meta, p.meta);
+ assert_eq!((q.rating, q.flag, q.label), (p.rating, p.flag, p.label));
+ assert_eq!(q.edited.as_deref(), Some("now"));
+ }
+
+ #[test]
+ fn foreign_sidecar_rating_reject_label_and_crs() {
+ let x = r#"
+ "#;
+ let sc = parse_sidecar(x, true).unwrap();
+ assert_eq!((sc.rating, sc.flag, sc.label), (Some(0), Some(Flag::Reject), Some(Some(ColorLabel::Green))));
+ let mut p = photo();
+ let before = p.develop.clone();
+ assert!(merge_into(&mut p, &sc, "t"));
+ assert_eq!(p.develop.light.exposure, -0.4);
+ assert_eq!(p.develop.color.vibrance, 12.0);
+ assert_eq!(p.develop.effects.clarity, before.effects.clarity, "crs partial keeps unrelated settings");
+ assert_eq!(p.meta.title, "Dune & sky", "fields the sidecar doesn't state are kept");
+ }
+
+ #[test]
+ fn op_photo_collection() {
+ let mut v = Vec::new();
+ op_photos(
+ &Op::Batch {
+ ops: vec![
+ Op::SetRating { id: PhotoId(1), rating: 2 },
+ Op::SetFlag { id: PhotoId(1), flag: Flag::Pick },
+ Op::SetLabel { id: PhotoId(3), label: None },
+ ],
+ },
+ &mut v,
+ );
+ assert_eq!(v, vec![PhotoId(1), PhotoId(3)]);
+ }
+}
diff --git a/crates/engine/src/tests_xmp.rs b/crates/engine/src/tests_xmp.rs
new file mode 100644
index 000000000..2997d7937
--- /dev/null
+++ b/crates/engine/src/tests_xmp.rs
@@ -0,0 +1,246 @@
+//! XMP sidecars end to end: save → delete library → re-import restores edits; auto-write; foreign
+//! `crs:` sidecars; DNG-embedded XMP; read metadata from file (undoable).
+
+use std::path::{Path, PathBuf};
+
+use lightcraft_catalog::{ColorLabel, Flag, PhotoId};
+use serde_json::json;
+
+use crate::Session;
+
+fn temp_dir(tag: &str) -> PathBuf {
+ let d = std::env::temp_dir().join(format!("lc-xmp-{tag}-{}", std::process::id()));
+ let _ = std::fs::remove_dir_all(&d);
+ std::fs::create_dir_all(&d).unwrap();
+ d
+}
+
+fn write_png(path: &Path, seed: u8) {
+ let (w, h) = (40usize, 24usize);
+ let data: Vec<[u8; 4]> = (0..w * h).map(|i| [(i % w * 6) as u8, (i / w * 9) as u8, seed, 255]).collect();
+ let img = lightcraft_raster::Rgba8 { width: w, height: h, data };
+ let bytes = lightcraft_codecs::encode_png(&lightcraft_codecs::EncodeImage::rgba8(&img), &lightcraft_codecs::EncodeMeta::default()).unwrap();
+ std::fs::write(path, bytes).unwrap();
+}
+
+fn only(s: &Session) -> PhotoId {
+ s.catalog.photos().next().unwrap().id
+}
+
+#[test]
+fn save_delete_library_reimport_restores_edits() {
+ let src = temp_dir("roundtrip-src");
+ let lib = temp_dir("roundtrip-lib");
+ write_png(&src.join("dune.png"), 1);
+ write_png(&src.join("plain.png"), 2);
+ let mut s = Session::new().with_fs();
+ s.open_library(&lib, false).unwrap();
+ s.execute("library.import", &json!({"paths": [src.join("dune.png").to_string_lossy()]})).unwrap();
+ let id = only(&s);
+ s.execute("library.select", &json!({"ids": [id.0]})).unwrap();
+ s.execute("photo.rate", &json!({"rating": 4})).unwrap();
+ s.execute("photo.pick", &json!({})).unwrap();
+ s.execute("photo.label", &json!({"label": "blue"})).unwrap();
+ s.execute("photo.setMeta", &json!({"title": "Dune", "caption": "Evening light", "copyright": "© me", "keywords": ["sand", "dusk"]})).unwrap();
+ s.execute("develop.set", &json!({"control": "light.exposure", "value": 0.8})).unwrap();
+ s.execute("develop.set", &json!({"control": "effects.clarity", "value": 25})).unwrap();
+ s.execute("mask.add", &json!({"kind": "radial"})).unwrap();
+ let edited = s.catalog.photo(id).unwrap().clone();
+ let r = s.execute("photo.saveMetadataToFile", &json!({})).unwrap();
+ assert_eq!(r["written"][0], src.join("dune.xmp").to_string_lossy().as_ref(), "{r}");
+ assert!(src.join("dune.xmp").is_file());
+ assert!(!src.join("plain.xmp").exists());
+ drop(s);
+
+ // delete the library; a fresh library re-imports the folder
+ std::fs::remove_dir_all(&lib).unwrap();
+ let mut s = Session::new().with_fs();
+ s.open_library(&lib, false).unwrap();
+ let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
+ assert_eq!(r["imported"].as_array().unwrap().len(), 2, "{r}");
+ assert_eq!(r["sidecars"], 1, "{r}");
+ let p = s.catalog.photos().find(|p| p.file_name == "dune.png").unwrap().clone();
+ assert_eq!(p.develop, edited.develop, "develop settings restored exactly");
+ assert_eq!(p.develop.masks.len(), 1);
+ assert_eq!((p.rating, p.flag, p.label), (4, Flag::Pick, Some(ColorLabel::Blue)));
+ assert_eq!(p.meta.title, "Dune");
+ assert_eq!(p.meta.caption, "Evening light");
+ assert_eq!(p.meta.copyright, "© me");
+ assert_eq!(p.meta.keywords, vec!["sand".to_string(), "dusk".to_string()]);
+ assert!(p.edited.is_some());
+ let plain = s.catalog.photos().find(|p| p.file_name == "plain.png").unwrap();
+ assert!(plain.develop.is_unedited() && plain.rating == 0);
+ let _ = std::fs::remove_dir_all(&src);
+ let _ = std::fs::remove_dir_all(&lib);
+}
+
+#[test]
+fn auto_write_and_naming_preference() {
+ let src = temp_dir("auto");
+ let lib = temp_dir("auto-lib");
+ write_png(&src.join("a.png"), 3);
+ let mut s = Session::new().with_fs();
+ s.open_library(&lib, false).unwrap();
+ s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
+ s.execute("photo.rate", &json!({"rating": 2})).unwrap();
+ assert!(!src.join("a.xmp").exists(), "auto-write is off by default");
+
+ let v = s.execute("library.xmpPreferences", &json!({"autoWrite": true, "naming": "full"})).unwrap();
+ assert_eq!(v, json!({"autoWrite": true, "naming": "full"}));
+ s.execute("photo.rate", &json!({"rating": 5})).unwrap();
+ let sidecar = src.join("a.png.xmp");
+ assert!(sidecar.is_file());
+ assert!(std::fs::read_to_string(&sidecar).unwrap().contains("5"));
+ // a slider drag writes once, at the end
+ s.execute("develop.beginInteraction", &json!({"label": "Exposure"})).unwrap();
+ s.execute("develop.set", &json!({"control": "light.exposure", "value": 1.5})).unwrap();
+ let exposure = |p: &Path| match crate::sidecar::parse_sidecar(&std::fs::read_to_string(p).unwrap(), false).unwrap().develop {
+ Some(crate::sidecar::DevelopPatch::Full(d)) => d.light.exposure,
+ other => panic!("{other:?}"),
+ };
+ assert_eq!(exposure(&sidecar), 0.0, "not written mid-drag");
+ s.execute("develop.endInteraction", &json!({})).unwrap();
+ assert_eq!(exposure(&sidecar), 1.5);
+ // undo rewrites too
+ s.execute("edit.undo", &json!({})).unwrap();
+ assert_eq!(exposure(&sidecar), 0.0);
+
+ // the preference persists with the library
+ drop(s);
+ let mut s = Session::new().with_fs();
+ s.open_library(&lib, false).unwrap();
+ assert!(s.xmp.auto_write);
+ assert_eq!(s.xmp.naming, crate::sidecar::SidecarNaming::Full);
+ let v = s.execute("library.toggleAutoWriteXmp", &json!({})).unwrap();
+ assert_eq!(v["autoWrite"], false);
+ let _ = std::fs::remove_dir_all(&src);
+ let _ = std::fs::remove_dir_all(&lib);
+}
+
+/// Hand-written sidecar in the style other raw developers use (attribute form, `crs:` fields).
+const FOREIGN: &str = r#"
+
+
+
+ harbour
+
+
+
+"#;
+
+#[test]
+fn foreign_crs_sidecar_on_import_and_read_from_file() {
+ let src = temp_dir("foreign");
+ write_png(&src.join("harbour.png"), 4);
+ std::fs::write(src.join("harbour.xmp"), FOREIGN).unwrap();
+ let mut s = Session::new().with_fs();
+ let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
+ assert_eq!(r["sidecars"], 1);
+ let p = s.catalog.photos().next().unwrap().clone();
+ assert_eq!((p.rating, p.label), (3, Some(ColorLabel::Red)));
+ assert_eq!(p.meta.keywords, vec!["harbour".to_string()]);
+ let d = &p.develop;
+ assert_eq!((d.light.exposure, d.light.shadows, d.color.vibrance, d.effects.clarity), (1.1, 35.0, 15.0, 10.0));
+ assert!(d.wb.temp > 6500.0 && d.wb.tint == -5.0, "relative WB for a rendered file: {:?}", d.wb);
+ assert_eq!((d.grading.shadows.hue, d.grading.shadows.sat), (220.0, 18.0));
+ assert_eq!((d.crop.geometry.rect.x0, d.crop.geometry.rect.x1), (0.1, 0.9));
+ assert!(s.render_now(p.id, 32, 32).is_ok());
+
+ // change things, then read the sidecar again: one undo step restores the edits
+ s.execute("library.select", &json!({"ids": [p.id.0]})).unwrap();
+ s.execute("develop.reset", &json!({})).unwrap();
+ s.execute("photo.rate", &json!({"rating": 1})).unwrap();
+ let r = s.execute("photo.readMetadataFromFile", &json!({})).unwrap();
+ assert_eq!(r["read"].as_array().unwrap().len(), 1, "{r}");
+ let q = s.catalog.photo(p.id).unwrap();
+ assert_eq!(q.rating, 3);
+ assert_eq!(q.develop.light.exposure, 1.1);
+ s.execute("edit.undo", &json!({})).unwrap();
+ let q = s.catalog.photo(p.id).unwrap();
+ assert_eq!((q.rating, q.develop.light.exposure), (1, 0.0));
+ // no sidecar → reported, not an error
+ std::fs::remove_file(src.join("harbour.xmp")).unwrap();
+ let r = s.execute("photo.readMetadataFromFile", &json!({})).unwrap();
+ assert_eq!(r["failed"].as_array().unwrap().len(), 1);
+ let _ = std::fs::remove_dir_all(&src);
+}
+
+fn synthetic_dng(xmp: &str) -> Vec {
+ use lightcraft_raw::*;
+ let (w, h) = (32usize, 24usize);
+ let cfa = Cfa::bayer("RGGB").unwrap();
+ let data: Vec = (0..w * h).map(|i| 256 + ((i % w) * 300 + (i / w) * 200) as u16).collect();
+ let raw = RawImage {
+ format: RawFormat::Dng,
+ width: w,
+ height: h,
+ cpp: 1,
+ data: RawData::U16(data),
+ cfa: Some(cfa),
+ bits: 14,
+ black: BlackLevel::uniform(256.0),
+ white: vec![16000.0],
+ active_area: Rect::new(0, 0, w, h),
+ crop: Rect::new(0, 0, w, h),
+ orientation: Orientation::Normal,
+ color: ColorData {
+ illuminant: [17, 21],
+ color_matrix: [
+ Some(lightcraft_color::Mat3([[0.9, 0.2, -0.15], [-0.3, 1.25, 0.08], [0.02, -0.12, 0.85]])),
+ Some(lightcraft_color::Mat3([[0.7, 0.3, -0.1], [-0.35, 1.3, 0.1], [0.05, -0.2, 1.0]])),
+ ],
+ as_shot_neutral: Some([0.5, 1.0, 0.7]),
+ ..Default::default()
+ },
+ wb_multipliers: None,
+ linearized: false,
+ opcodes: OpcodeLists::default(),
+ metadata: lightcraft_meta::Metadata::default(),
+ };
+ write_dng(&raw, &DngWriteOptions { xmp: Some(xmp.to_string()), ..Default::default() }).unwrap()
+}
+
+#[test]
+fn dng_embedded_crs_settings_are_read_on_import() {
+ let src = temp_dir("dng");
+ let x = r#"
+
+ "#;
+ std::fs::write(src.join("synth.dng"), synthetic_dng(x)).unwrap();
+ let mut s = Session::new().with_fs();
+ let r = s.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
+ assert_eq!((r["imported"].as_array().unwrap().len(), r["sidecars"].as_u64()), (1, Some(1)), "{r}");
+ let p = s.catalog.photos().next().unwrap();
+ assert_eq!(p.kind, lightcraft_catalog::MediaKind::Raw);
+ assert_eq!((p.develop.wb.temp, p.develop.wb.tint), (4300.0, 7.0));
+ assert_eq!((p.develop.light.contrast, p.develop.effects.dehaze), (30.0, 12.0));
+ assert_eq!(p.develop.detail.sharpen_amount, 40.0, "raw defaults kept for fields the XMP doesn't set");
+ assert_eq!(p.rating, 2);
+
+ // a sidecar next to the DNG takes precedence over the embedded XMP
+ let id = p.id;
+ s.execute("library.select", &json!({"ids": [id.0]})).unwrap();
+ s.execute("develop.set", &json!({"control": "light.contrast", "value": -10})).unwrap();
+ s.execute("photo.saveMetadataToFile", &json!({})).unwrap();
+ let mut s2 = Session::new().with_fs();
+ s2.execute("library.import", &json!({"paths": [src.to_string_lossy()]})).unwrap();
+ assert_eq!(s2.catalog.photos().next().unwrap().develop.light.contrast, -10.0);
+ let _ = std::fs::remove_dir_all(&src);
+}
+
+#[test]
+fn demo_photos_have_no_sidecar() {
+ let mut s = Session::with_demo();
+ let r = s.execute("photo.saveMetadataToFile", &json!({})).unwrap();
+ assert_eq!(r["written"].as_array().unwrap().len(), 0);
+ assert_eq!(r["failed"].as_array().unwrap().len(), 1);
+}
diff --git a/crates/meta/src/container.rs b/crates/meta/src/container.rs
index 87bf51149..7e6c7270d 100644
--- a/crates/meta/src/container.rs
+++ b/crates/meta/src/container.rs
@@ -26,6 +26,14 @@ pub struct Embedded {
/// Sniff the container and collect its metadata blocks. Unknown containers yield an empty result.
pub fn embedded(bytes: &[u8]) -> Embedded {
+ if bytes.starts_with(b"FUJIFILMCCD-RAW") {
+ // Fujifilm RAF: metadata lives in the embedded JPEG whose (offset, length) are big-endian u32s at byte 84
+ let be = |i: usize| bytes.get(i..i + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize);
+ return match (be(84), be(88)) {
+ (Some(o), Some(l)) if o > 0 => bytes.get(o..o.saturating_add(l).min(bytes.len())).map(jpeg_segments).unwrap_or_default(),
+ _ => Embedded::default(),
+ };
+ }
if bytes.starts_with(&[0xff, 0xd8]) {
jpeg_segments(bytes)
} else if bytes.starts_with(b"\x89PNG\r\n\x1a\n") {
@@ -294,6 +302,20 @@ mod tests {
assert_eq!(embedded(b"not an image"), Embedded::default());
}
+ #[test]
+ fn raf_reads_its_embedded_jpeg() {
+ let j = sample_jpeg();
+ let mut raf = b"FUJIFILMCCD-RAW 0201FF000000TEST".to_vec();
+ raf.resize(120, 0);
+ raf[84..88].copy_from_slice(&120u32.to_be_bytes());
+ raf[88..92].copy_from_slice(&(j.len() as u32).to_be_bytes());
+ raf.extend_from_slice(&j);
+ assert_eq!(crate::extract(&raf).iso, Some(400));
+ for n in 0..raf.len() {
+ let _ = crate::extract(&raf[..n]);
+ }
+ }
+
#[test]
fn truncated_containers_do_not_panic() {
let j = sample_jpeg();
diff --git a/crates/meta/src/lib.rs b/crates/meta/src/lib.rs
index 71284677d..cfc678c75 100644
--- a/crates/meta/src/lib.rs
+++ b/crates/meta/src/lib.rs
@@ -20,7 +20,7 @@ pub use datetime::DateTime;
pub use exif::{from_tiff, read_exif, strip_exif_header, try_read_exif, write_exif};
pub use iptc::parse_iptc;
pub use lightcraft_geom::Orientation;
-pub use xmp::{LC_NS, XmpData, XmpError, parse_xmp, write_xmp};
+pub use xmp::{CRS_NS, LC_NS, XmpData, XmpError, parse_xmp, write_xmp, write_xmp_lc};
use serde::{Deserialize, Serialize};
diff --git a/crates/meta/src/xmp.rs b/crates/meta/src/xmp.rs
index 3fa3c7e31..06b6569a8 100644
--- a/crates/meta/src/xmp.rs
+++ b/crates/meta/src/xmp.rs
@@ -9,7 +9,7 @@
//! LightCraft's full develop settings as an opaque JSON string in `lc:settings`.
use crate::{DateTime, Flash, Gps, Metadata, Orientation, parse_number};
-use quick_xml::escape::escape;
+use quick_xml::escape::{escape, partial_escape};
use quick_xml::events::Event;
use std::collections::BTreeMap;
@@ -30,8 +30,13 @@ const NAMESPACES: &[(&str, &str)] = &[
("lr", "http://ns.adobe.com/lightroom/1.0/"),
("Iptc4xmpCore", "http://iptc.org/std/Iptc4xmpCore/1.0/xmlns/"),
("lc", LC_NS),
+ // Read-only: develop settings written by other raw developers (interchange; see docs/xmp-interop.md).
+ ("crs", CRS_NS),
];
+/// The camera-raw-settings namespace URI (prefix `crs`), read for interchange only.
+pub const CRS_NS: &str = "http://ns.adobe.com/camera-raw-settings/1.0/";
+
/// XMP read failure.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum XmpError {
@@ -405,6 +410,12 @@ fn sfrac(v: f64) -> String {
/// Serialise the interchange subset of `meta` (plus `lc_settings`, an opaque JSON string, in `lc:settings`)
/// as a complete XMP packet.
pub fn write_xmp(meta: &Metadata, lc_settings: Option<&str>) -> String {
+ let lc: Vec<(&str, &str)> = lc_settings.map(|s| ("settings", s)).into_iter().collect();
+ write_xmp_lc(meta, &lc)
+}
+
+/// Like [`write_xmp`], with any number of simple `lc:` properties (e.g. `settings`, `flag`).
+pub fn write_xmp_lc(meta: &Metadata, lc: &[(&str, &str)]) -> String {
let mut simple: Vec<(&str, String)> = Vec::new();
let mut push = |k: &'static str, v: Option| {
if let Some(v) = v {
@@ -441,20 +452,25 @@ pub fn write_xmp(meta: &Metadata, lc_settings: Option<&str>) -> String {
push("exif:GPSAltitudeRef", Some(if a < 0.0 { "1" } else { "0" }.into()));
}
}
- push("lc:settings", lc_settings.map(str::to_string));
let mut x = String::new();
x.push_str("\n");
x.push_str("\n");
x.push_str(" \n");
x.push_str(" \n");
for (k, v) in &simple {
x.push_str(&format!(" <{k}>{}\n", escape(v.as_str())));
}
+ for (k, v) in lc {
+ if !k.is_empty() && k.chars().all(|c| c.is_ascii_alphanumeric() || c == '_') {
+ // element content: quotes needn't be escaped, which keeps the JSON readable
+ x.push_str(&format!(" {}\n", partial_escape(*v)));
+ }
+ }
let array = |x: &mut String, k: &str, kind: &str, items: &[String], lang: bool| {
if items.is_empty() {
return;
@@ -599,6 +615,19 @@ mod tests {
}
}
+ #[test]
+ fn lc_extras_and_crs_prefix() {
+ let x = write_xmp_lc(&Metadata::default(), &[("settings", "{\"a\":1}"), ("flag", "pick"), ("bad name", "x")]);
+ assert!(!x.contains("xmlns:crs"), "we never write crs:");
+ let d = parse_xmp(&x).unwrap();
+ assert_eq!(d.lc_settings.as_deref(), Some("{\"a\":1}"));
+ assert_eq!(d.properties["lc:flag"], vec!["pick".to_string()]);
+ assert!(!x.contains("bad name"));
+ let foreign = r#""#;
+ assert_eq!(parse_xmp(foreign).unwrap().properties["crs:Exposure2012"], vec!["+0.5".to_string()]);
+ }
+
#[test]
fn gps_coords() {
assert_eq!(parse_gps_coord("45,30N"), Some(45.5));
diff --git a/crates/raw/examples/colstats.rs b/crates/raw/examples/colstats.rs
new file mode 100644
index 000000000..d536274cd
--- /dev/null
+++ b/crates/raw/examples/colstats.rs
@@ -0,0 +1,17 @@
+//! Debug helper: per-column / per-row statistics of the decoded raw (outliers above the bit range).
+fn main() {
+ let a: Vec = std::env::args().skip(1).collect();
+ let r = lightcraft_raw::decode(&std::fs::read(&a[0]).unwrap()).unwrap();
+ let lightcraft_raw::RawData::U16(d) = &r.data else { return };
+ let lim = (1u32 << r.bits) as u16;
+ let (w, h) = (r.width, r.height);
+ let bad_cols: Vec<(usize, usize)> = (0..w).map(|x| (x, (0..h).filter(|&y| d[y * w + x] >= lim).count())).filter(|c| c.1 > 0).collect();
+ let bad_rows: Vec<(usize, usize)> = (0..h).map(|y| (y, (0..w).filter(|&x| d[y * w + x] >= lim).count())).filter(|c| c.1 > 0).collect();
+ println!("cols with >= {lim}: {} {:?}", bad_cols.len(), &bad_cols[..bad_cols.len().min(20)]);
+ println!("rows with >= {lim}: {} {:?}", bad_rows.len(), &bad_rows[..bad_rows.len().min(10)]);
+ let mx = d.iter().filter(|&&v| v < lim).max();
+ println!(
+ "max in range: {mx:?}; right-edge col means: {:?}",
+ (w - 8..w).map(|x| (0..h).map(|y| d[y * w + x] as u64).sum::() / h as u64).collect::>()
+ );
+}
diff --git a/crates/raw/examples/rawdump.rs b/crates/raw/examples/rawdump.rs
new file mode 100644
index 000000000..2b12a64d7
--- /dev/null
+++ b/crates/raw/examples/rawdump.rs
@@ -0,0 +1,14 @@
+//! Debug helper: `rawdump IN OUT` writes the decoded sensor data as `u32 width, u32 height` (little-endian)
+//! followed by `width × height × cpp` little-endian `u16` samples, for analysis in external tools.
+fn main() {
+ let a: Vec = std::env::args().skip(1).collect();
+ let r = lightcraft_raw::decode(&std::fs::read(&a[0]).expect("read")).expect("decode");
+ let lightcraft_raw::RawData::U16(d) = &r.data else { panic!("float data") };
+ let mut out = Vec::with_capacity(8 + d.len() * 2);
+ out.extend_from_slice(&(r.width as u32).to_le_bytes());
+ out.extend_from_slice(&(r.height as u32).to_le_bytes());
+ for v in d {
+ out.extend_from_slice(&v.to_le_bytes());
+ }
+ std::fs::write(&a[1], out).expect("write");
+}
diff --git a/crates/raw/src/lib.rs b/crates/raw/src/lib.rs
index da4f4e414..2803d4b5c 100644
--- a/crates/raw/src/lib.rs
+++ b/crates/raw/src/lib.rs
@@ -10,8 +10,12 @@
//! and produces camera → linear Rec.2020 D65 matrices.
//!
//! Formats: DNG (uncompressed, lossless JPEG, Deflate incl. floating point, tiled/stripped, CFA and LinearRaw),
-//! Canon CR2, Nikon NEF (uncompressed), Sony ARW (uncompressed, lossless). See the crate README for sources and
-//! gaps. The decoders never panic on malformed input.
+//! Canon CR2, Nikon NEF/NRW (uncompressed), Sony ARW (uncompressed, ARW2, lossless), Fujifilm RAF (uncompressed Bayer
+//! and X-Trans), Panasonic RW2 (packed 12/14-bit), Pentax PEF (uncompressed, Huffman), Olympus ORF (uncompressed).
+//! [`embedded_preview`] covers all of them plus CR3. Variants we can't decode yet (Nikon Huffman NEF, Panasonic
+//! quantised RW2, compressed ORF/RAF, CR3) return [`RawError::Unsupported`]; each vendor module documents its sources
+//! (public specifications, tag-name documentation, black-box analysis of CC0 samples) and gaps. Non-DNG files carry no
+//! colour matrix: [`color`] falls back to a documented neutral model. The decoders never panic on malformed input.
#![forbid(unsafe_code)]
pub mod color;
@@ -80,7 +84,10 @@ pub enum RawFormat {
impl RawFormat {
/// Whether [`decode`] supports this container (possibly not every compression inside it).
pub fn is_supported(self) -> bool {
- matches!(self, RawFormat::Dng | RawFormat::Cr2 | RawFormat::Nef | RawFormat::Nrw | RawFormat::Arw)
+ matches!(
+ self,
+ RawFormat::Dng | RawFormat::Cr2 | RawFormat::Nef | RawFormat::Nrw | RawFormat::Arw | RawFormat::Raf | RawFormat::Rw2 | RawFormat::Pef
+ )
}
}
@@ -141,6 +148,10 @@ pub fn decode(bytes: &[u8]) -> Result {
RawFormat::Cr2 => vendor::cr2::decode(bytes),
RawFormat::Nef | RawFormat::Nrw => vendor::nef::decode(bytes),
RawFormat::Arw => vendor::arw::decode(bytes),
+ RawFormat::Raf => vendor::raf::decode(bytes),
+ RawFormat::Rw2 => vendor::rw2::decode(bytes),
+ RawFormat::Pef => vendor::pef::decode(bytes),
+ RawFormat::Orf => vendor::orf::decode(bytes),
other => Err(RawError::Unsupported(format!("{other:?} files are not decoded yet"))),
}
}
@@ -487,7 +498,7 @@ mod tests {
assert_eq!(probe(b"hello world, not a raw"), None);
assert_eq!(probe(b"\0\0\0\x18ftypcrx \0\0\0\x01"), Some(RawFormat::Cr3));
assert_eq!(probe(b"FUJIFILMCCD-RAW 0201"), Some(RawFormat::Raf));
- assert!(matches!(decode(b"FUJIFILMCCD-RAW 0201"), Err(RawError::Unsupported(_))));
+ assert!(decode(b"FUJIFILMCCD-RAW 0201").is_err());
assert_eq!(decode(b"junk"), Err(RawError::NotRaw));
}
}
diff --git a/crates/raw/src/preview.rs b/crates/raw/src/preview.rs
index 8b7aa21e0..050870ce1 100644
--- a/crates/raw/src/preview.rs
+++ b/crates/raw/src/preview.rs
@@ -33,6 +33,16 @@ fn is_dct_jpeg(b: &[u8]) -> bool {
}
fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>) {
+ // whole JPEG files stored as an undefined-type tag value (e.g. Panasonic `JpgFromRaw` 0x002e)
+ for e in &ifd.entries {
+ if matches!(e.value, lightcraft_tiff::Value::Undefined(_))
+ && e.count() > 1024
+ && let Some(s) = data.get(e.offset as usize..(e.offset.saturating_add(e.count() as u64) as usize).min(data.len()))
+ && s.starts_with(&[0xff, 0xd8])
+ {
+ out.push(s);
+ }
+ }
if let (Some(off), Some(len)) = (ifd.u64(t::JPEG_INTERCHANGE_FORMAT), ifd.u64(t::JPEG_INTERCHANGE_FORMAT_LENGTH)) {
let off = off.saturating_add(base);
if let Some(s) = data.get(off as usize..(off.saturating_add(len) as usize).min(data.len())) {
@@ -53,6 +63,13 @@ fn candidates<'a>(data: &'a [u8], ifd: &Ifd, base: u64, out: &mut Vec<&'a [u8]>)
/// The largest embedded JPEG preview, if any.
pub fn embedded_preview(bytes: &[u8]) -> Option> {
+ if bytes.starts_with(b"FUJIFILMCCD-RAW") {
+ let j = crate::vendor::raf::header(bytes).ok()?.jpeg?;
+ return is_dct_jpeg(j).then(|| trim_eoi(j).to_vec());
+ }
+ if crate::probe(bytes) == Some(crate::RawFormat::Cr3) {
+ return cr3_preview(bytes).map(|j| trim_eoi(j).to_vec());
+ }
let tiff = Tiff::parse(bytes).ok()?;
let mut found: Vec<&[u8]> = Vec::new();
for ifd in tiff.all_ifds() {
@@ -72,11 +89,45 @@ pub fn embedded_preview(bytes: &[u8]) -> Option> {
}
}
}
- found.into_iter().filter(|s| is_dct_jpeg(s)).max_by_key(|s| s.len()).map(|s| {
- // trim trailing garbage after EOI when the length over-reports
- let end = s.windows(2).rposition(|w| w == [0xff, 0xd9]).map(|p| p + 2).unwrap_or(s.len());
- s[..end].to_vec()
- })
+ // Olympus: CameraSettings preview
+ if let Some(p) = crate::vendor::orf::preview(bytes) {
+ found.push(p);
+ }
+ found.into_iter().filter(|s| is_dct_jpeg(s)).max_by_key(|s| s.len()).map(|s| trim_eoi(s).to_vec())
+}
+
+/// Canon CR3 (ISO base media file): the `PRVW` box (Laurent Clévy's CR3 notes; layout confirmed on a CC0
+/// sample) is `u32 size, "PRVW", u32 0, u16 ?, u16 width, u16 height, u16 ?, u32 jpeg length, JPEG`; the smaller
+/// `THMB` box has the same shape. Returns the larger valid one.
+fn cr3_preview(bytes: &[u8]) -> Option<&[u8]> {
+ let mut best: Option<&[u8]> = None;
+ for tag in [b"PRVW", b"THMB"] {
+ let mut from = 0;
+ while let Some(i) = bytes.get(from..).and_then(|s| s.windows(4).position(|w| w == tag)).map(|p| p + from) {
+ from = i + 4;
+ let Some(start) = i.checked_sub(4) else { continue };
+ let be32 = |at: usize| bytes.get(at..at + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize);
+ let (Some(size), Some(len)) = (be32(start), be32(start + 20)) else { continue };
+ let j = start + 24;
+ if len < 4 || j + len > start + size.max(24) || j + len > bytes.len() {
+ continue;
+ }
+ let jpeg = &bytes[j..j + len];
+ if is_dct_jpeg(jpeg) {
+ if best.is_none_or(|b| b.len() < jpeg.len()) {
+ best = Some(jpeg);
+ }
+ break;
+ }
+ }
+ }
+ best
+}
+
+/// Trim trailing garbage after the last EOI when a stored length over-reports.
+fn trim_eoi(s: &[u8]) -> &[u8] {
+ let end = s.windows(2).rposition(|w| w == [0xff, 0xd9]).map(|p| p + 2).unwrap_or(s.len());
+ &s[..end]
}
#[cfg(test)]
@@ -117,5 +168,16 @@ mod tests {
let bytes = TiffWriter::default().write(&[ifd0]).unwrap();
assert_eq!(embedded_preview(&bytes).unwrap(), big);
assert!(embedded_preview(b"nope").is_none());
+ // CR3: a PRVW box after the ftyp
+ let mut cr3 = b"\0\0\0\x18ftypcrx \0\0\0\x01crx isom".to_vec();
+ let j = fake_jpeg(300);
+ cr3.extend_from_slice(&((24 + j.len()) as u32).to_be_bytes());
+ cr3.extend_from_slice(b"PRVW\0\0\0\0\0\x01\x06\x54\x04\x38\0\x01");
+ cr3.extend_from_slice(&(j.len() as u32).to_be_bytes());
+ cr3.extend_from_slice(&j);
+ assert_eq!(embedded_preview(&cr3).unwrap(), j);
+ for n in 0..cr3.len() {
+ let _ = embedded_preview(&cr3[..n]);
+ }
}
}
diff --git a/crates/raw/src/vendor/mod.rs b/crates/raw/src/vendor/mod.rs
index f15e64159..74a4461bd 100644
--- a/crates/raw/src/vendor/mod.rs
+++ b/crates/raw/src/vendor/mod.rs
@@ -3,6 +3,10 @@
pub mod arw;
pub mod cr2;
pub mod nef;
+pub mod orf;
+pub mod pef;
+pub mod raf;
+pub mod rw2;
use crate::{BlackLevel, Rect};
use std::ops::Range;
@@ -30,13 +34,17 @@ pub(crate) fn black_from_columns(data: &[u16], width: usize, cols: Range,
/// otherwise the full `bits` range.
pub(crate) fn white_from_data(data: &[u16], bits: u32) -> f32 {
let full = ((1u32 << bits.clamp(1, 16)) - 1) as f32;
- let Some(&mx) = data.iter().max() else { return full };
+ // samples above the nominal range are padding / invalid (e.g. 0xFFFF fill), not saturation
+ let Some(&mx) = data.iter().filter(|&&v| v as f32 <= full).max() else { return full };
let band = (mx / 256).max(1);
let near = mx.saturating_sub(band);
let below = near.saturating_sub(band);
let step = (data.len() / 2_000_000).max(1);
let (mut top, mut under) = (0usize, 0usize);
for &v in data.iter().step_by(step) {
+ if v as f32 > full {
+ continue;
+ }
if v >= near {
top += 1;
} else if v >= below {
@@ -48,7 +56,7 @@ pub(crate) fn white_from_data(data: &[u16], bits: u32) -> f32 {
// plateau: use its lower edge so everything at saturation maps to ≥ 1.0
near as f32
} else {
- full.max(mx as f32)
+ full
}
}
diff --git a/crates/raw/src/vendor/nef.rs b/crates/raw/src/vendor/nef.rs
index b82f3453a..bcd8dd1ac 100644
--- a/crates/raw/src/vendor/nef.rs
+++ b/crates/raw/src/vendor/nef.rs
@@ -1,5 +1,166 @@
-use crate::{RawError, RawImage, Result};
+//! Nikon NEF / NRW — uncompressed variants only.
+//!
+//! Sources: TIFF 6.0 (the raw image is a standard CFA SubIFD), Laurent Clévy's NEF structure notes (prose: IFD
+//! layout, SubIFDs, maker note header) and the ExifTool Nikon tag-name documentation (`0x000c` WB_RBLevels,
+//! `0x003d` BlackLevel). Nikon's Huffman-compressed variants (compression 34713: lossless, lossy type 1/2) are
+//! **not** implemented: the only descriptions we found are derived from GPL code, which our clean-room rules
+//! forbid. The coding is T.81-style (difference category + additional bits, as in Pentax PEF), but unlike PEF the
+//! Huffman tables are not stored in the files and our black-box table search has not converged (see ROADMAP.md).
+//! They are reported as [`RawError::Unsupported`]; their embedded previews still work.
-pub(crate) fn decode(_bytes: &[u8]) -> Result {
- Err(RawError::Unsupported("nef: not yet".into()))
+use super::white_from_data;
+use crate::tiffraw::{Packing, read_image};
+use crate::{BlackLevel, Cfa, ColorData, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result};
+use lightcraft_geom::Orientation;
+use lightcraft_tiff::tags::{self as t, photometric};
+use lightcraft_tiff::{Ifd, Tiff, makernote};
+
+const WB_RB_LEVELS: u16 = 0x000c;
+const BLACK_LEVEL: u16 = 0x003d;
+
+fn raw_ifd(tiff: &Tiff) -> Option<&Ifd> {
+ tiff.all_ifds()
+ .into_iter()
+ .filter(|i| i.u16(t::PHOTOMETRIC) == Some(photometric::CFA))
+ .max_by_key(|i| i.u64(t::IMAGE_WIDTH).unwrap_or(0) * i.u64(t::IMAGE_LENGTH).unwrap_or(0))
+}
+
+/// Width without the optically masked columns some bodies append on the right: trailing columns (at most 64)
+/// whose mean is below 1% of the white level while the image interior is brighter. Kept even for CFA phase.
+pub(crate) fn trailing_masked_columns(d: &[u16], w: usize, h: usize, white: f32) -> usize {
+ if w < 128 || h == 0 {
+ return w;
+ }
+ let step = (h / 256).max(1);
+ let col_mean = |x: usize| (0..h).step_by(step).map(|y| d[y * w + x] as f64).sum::() / h.div_ceil(step) as f64;
+ let interior = (w / 4..w * 3 / 4).step_by(w / 64).map(col_mean).sum::() / 32.0;
+ let dark = (white as f64 * 0.01).min(interior * 0.1);
+ let mut aw = w;
+ while aw > w - 64 && col_mean(aw - 1) < dark {
+ aw -= 1;
+ }
+ aw & !1
+}
+
+pub(crate) fn decode(bytes: &[u8]) -> Result {
+ let tiff = Tiff::parse(bytes)?;
+ let ifd0 = &tiff.ifds[0];
+ let raw = raw_ifd(&tiff).ok_or_else(|| RawError::Unsupported("NEF without a CFA image IFD".into()))?;
+ let info = raw.image()?;
+ if info.compression == t::compression::NIKON {
+ return Err(RawError::Unsupported("Nikon Huffman-compressed NEF (no clean-room description available)".into()));
+ }
+ let (w, h) = (info.width as usize, info.height as usize);
+ let bits = info.bits() as u32;
+ let row_samples = w * info.samples_per_pixel as usize;
+ let chunks = info.chunks(bytes.len() as u64);
+ let rows_in_first = chunks.first().map(|c| c.height as usize).unwrap_or(h).max(1);
+ let bytes_per_row = chunks.first().map(|c| c.len as usize / rows_in_first).unwrap_or(0);
+ let packing = if bytes_per_row >= row_samples * 2 {
+ Packing::Word16
+ } else if bits == 12 && bytes_per_row * 8 >= row_samples * 12 && bytes_per_row * 8 < row_samples * 13 {
+ Packing::Msb
+ } else if info.compression == 1 && bits != 8 && bits != 16 {
+ return Err(RawError::Unsupported(format!("NEF uncompressed packing ({bytes_per_row} bytes per {row_samples}-sample row)")));
+ } else {
+ Packing::Msb
+ };
+ let data = read_image(bytes, &info, tiff.order, packing)?;
+ let RawData::U16(ref samples) = data else { return Err(RawError::Unsupported("float NEF".into())) };
+
+ let make = ifd0.string(t::MAKE).unwrap_or_default();
+ let mn =
+ tiff.exif().and_then(|e| e.get(t::MAKER_NOTE)).and_then(|e| makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make));
+ let black = match mn.as_ref().and_then(|m| m.ifd.f64s(BLACK_LEVEL)).as_deref() {
+ Some([a, b, c, d]) if [a, b, c, d].iter().all(|v| **v < 16384.0) => {
+ BlackLevel { repeat_rows: 2, repeat_cols: 2, values: vec![*a as f32, *b as f32, *c as f32, *d as f32], ..Default::default() }
+ }
+ _ => BlackLevel::uniform(0.0),
+ };
+ let wb = mn
+ .as_ref()
+ .and_then(|m| m.ifd.f64s(WB_RB_LEVELS))
+ .filter(|v| v.len() >= 2 && v[0] > 0.1 && v[1] > 0.1 && v[0] < 10.0 && v[1] < 10.0)
+ .map(|v| [v[0] as f32, 1.0, v[1] as f32]);
+ let cfa = match (raw.u64s(t::CFA_REPEAT_PATTERN_DIM).as_deref(), raw.bytes(t::CFA_PATTERN_EP)) {
+ (Some([2, 2]), Some(p)) if p.len() == 4 && p.iter().all(|&c| c <= 2) => Cfa { width: 2, height: 2, pattern: p.to_vec() },
+ _ => Cfa::bayer("RGGB").expect("static"),
+ };
+ let white = white_from_data(samples, bits);
+ let active_w = trailing_masked_columns(samples, w, h, white);
+ let mut metadata = lightcraft_meta::from_tiff(&tiff);
+ metadata.width = Some(active_w as u32);
+ metadata.height = Some(h as u32);
+ let img = RawImage {
+ format: RawFormat::Nef,
+ width: w,
+ height: h,
+ cpp: 1,
+ data,
+ cfa: Some(cfa),
+ bits,
+ black,
+ white: vec![white],
+ active_area: Rect::new(0, 0, active_w, h),
+ crop: Rect::new(0, 0, active_w, h),
+ orientation: Orientation::from_exif(ifd0.u16(t::ORIENTATION).unwrap_or(1)),
+ color: ColorData::default(),
+ wb_multipliers: wb,
+ linearized: false,
+ opcodes: OpcodeLists::default(),
+ metadata,
+ };
+ img.validate()?;
+ Ok(img)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use lightcraft_tiff::{ByteOrder, IfdBuilder, ImageData, TiffWriter, Value};
+
+ fn nef(compression: u16, bits: u16, strips: Vec>, w: u32, h: u32, rps: u32) -> Vec {
+ let mut raw = IfdBuilder::new();
+ raw.set(t::NEW_SUBFILE_TYPE, Value::Long(vec![0]));
+ raw.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
+ raw.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
+ raw.set(t::BITS_PER_SAMPLE, Value::Short(vec![bits]));
+ raw.set(t::COMPRESSION, Value::Short(vec![compression]));
+ raw.set(t::PHOTOMETRIC, Value::Short(vec![photometric::CFA]));
+ raw.set(t::CFA_REPEAT_PATTERN_DIM, Value::Short(vec![2, 2]));
+ raw.set(t::CFA_PATTERN_EP, Value::Byte(vec![2, 1, 1, 0]));
+ raw.set_image(ImageData::Strips { rows_per_strip: rps, strips });
+ let mut ifd0 = IfdBuilder::new();
+ ifd0.set(t::MAKE, Value::Ascii("NIKON CORPORATION".into()));
+ ifd0.set(t::MODEL, Value::Ascii("NIKON TEST".into()));
+ ifd0.add_sub_ifd(raw);
+ TiffWriter::new(ByteOrder::Big, false).write(&[ifd0]).unwrap()
+ }
+
+ #[test]
+ fn uncompressed_word16_and_packed12() {
+ let (w, h) = (16usize, 6usize);
+ let px: Vec = (0..w * h).map(|i| (i * 131 % 4096) as u16).collect();
+ let words: Vec = px.iter().flat_map(|v| v.to_be_bytes()).collect();
+ let bytes = nef(1, 12, words.chunks(w * 2 * 3).map(|c| c.to_vec()).collect(), w as u32, h as u32, 3);
+ assert_eq!(crate::probe(&bytes), Some(RawFormat::Nef));
+ let r = crate::decode(&bytes).unwrap();
+ assert_eq!(r.data, RawData::U16(px.clone()));
+ assert_eq!(r.cfa.as_ref().unwrap().name(), "BGGR");
+ // 12-bit MSB packed
+ let mut packed = Vec::new();
+ for pair in px.chunks(2) {
+ let (a, b) = (pair[0] as u32, pair[1] as u32);
+ let v = (a << 12) | b;
+ packed.extend_from_slice(&[(v >> 16) as u8, (v >> 8) as u8, v as u8]);
+ }
+ let bytes = nef(1, 12, vec![packed], w as u32, h as u32, h as u32);
+ assert_eq!(crate::decode(&bytes).unwrap().data, RawData::U16(px));
+ }
+
+ #[test]
+ fn compressed_is_unsupported() {
+ let bytes = nef(34713, 14, vec![vec![0; 64]], 8, 8, 8);
+ assert!(matches!(crate::decode(&bytes), Err(RawError::Unsupported(_))));
+ }
}
diff --git a/crates/raw/src/vendor/orf.rs b/crates/raw/src/vendor/orf.rs
new file mode 100644
index 000000000..2e2eebb38
--- /dev/null
+++ b/crates/raw/src/vendor/orf.rs
@@ -0,0 +1,223 @@
+//! Olympus ORF — uncompressed variants.
+//!
+//! Sources: TIFF 6.0 (the `IIRO`/`MMOR` container is a TIFF with a different magic number), the ExifTool
+//! Olympus tag-name documentation (maker-note sub-directories `0x2020` CameraSettings — `0x0101/0x0102`
+//! PreviewImageStart/Length — and `0x2040` ImageProcessing — `0x0100` WB_RBLevels, `0x0600` BlackLevel2,
+//! `0x0612–0x0615` CropLeft/Top/Width/Height) and our own black-box analysis of CC0 samples from raw.pixls.us
+//! (E-1, E-400, XZ-2):
+//!
+//! - 16 bits per sample: little-endian words; some bodies (E-1, E-400) store 12-bit values in the top bits (the low
+//! four bits zero in more than 99% of samples), which we shift down.
+//! - 12-bit packed (XZ-2): each row is a sequence of little-endian 32-bit words read MSB-first (found by testing
+//! candidate bit orders for the smoothest image).
+//! - Olympus's compressed ORF (most interchangeable-lens bodies since ~2008) is not decoded: no permissively
+//! licensed description exists. It reports [`RawError::Unsupported`]; the embedded preview still works.
+//! - The files carry no CFA tag. The green diagonal is found from the data (the diagonal whose two samples differ
+//! least); bodies with greens on the main diagonal (E-1, E-400) are GRBG, the others (XZ-2) RGGB — verified by
+//! colour renders.
+
+use super::white_from_data;
+use crate::tiffraw::{Packing, read_image};
+use crate::unpack::unpack_msb;
+use crate::{BlackLevel, Cfa, ColorData, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result};
+use lightcraft_geom::Orientation;
+use lightcraft_tiff::image::chunk_bytes;
+use lightcraft_tiff::makernote::MakerNote;
+use lightcraft_tiff::{Ifd, Tiff, Value, makernote, tags as t};
+use rayon::prelude::*;
+
+pub(crate) const CAMERA_SETTINGS: u16 = 0x2020;
+const IMAGE_PROCESSING: u16 = 0x2040;
+pub(crate) const PREVIEW_START: u16 = 0x0101;
+pub(crate) const PREVIEW_LENGTH: u16 = 0x0102;
+const WB_RB: u16 = 0x0100;
+const BLACK: u16 = 0x0600;
+const CROP: [u16; 4] = [0x0612, 0x0613, 0x0614, 0x0615];
+
+/// The Olympus maker note.
+pub(crate) fn maker_note(bytes: &[u8], tiff: &Tiff) -> Option {
+ let make = tiff.find(t::MAKE).and_then(|e| e.value.as_str()).unwrap_or_default().to_string();
+ let e = tiff.exif().and_then(|e| e.get(t::MAKER_NOTE))?;
+ makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make)
+}
+
+/// A maker-note sub-directory: an IFD pointer (offset relative to the note's base) in new-style notes, or the
+/// IFD stored inline as an undefined blob (offsets relative to the base) in old-style ones.
+pub(crate) fn sub_ifd(bytes: &[u8], mn: &MakerNote, tag: u16) -> Option {
+ let e = mn.ifd.get(tag)?;
+ let at = match &e.value {
+ Value::Undefined(_) | Value::Byte(_) => e.offset,
+ _ => mn.base.checked_add(mn.ifd.u64(tag)?)?,
+ };
+ let opts = lightcraft_tiff::ParseOptions { max_ifds: 4, max_depth: 1, follow_children: false, ..Default::default() };
+ lightcraft_tiff::parse_ifd_at(bytes, at, mn.order, mn.base, false, &opts).ok().map(|(i, _)| i)
+}
+
+/// Unpack one row of 12-bit samples stored as little-endian 32-bit words read MSB-first.
+pub(crate) fn unpack_row_le32_msb(src: &[u8], bits: u32, out: &mut [u16]) {
+ let swapped: Vec = src
+ .chunks(4)
+ .flat_map(|c| {
+ let mut w = [0u8; 4];
+ w[..c.len()].copy_from_slice(c);
+ [w[3], w[2], w[1], w[0]]
+ })
+ .collect();
+ unpack_msb(&swapped, bits, out);
+}
+
+/// GRBG when the greens sit on the main diagonal of the 2×2 cell at the sensor origin, else RGGB.
+pub(crate) fn cfa_from_data(d: &[u16], w: usize, a: Rect) -> Cfa {
+ let (x0, y0) = (a.x + ((a.width / 8) & !1), a.y + ((a.height / 8) & !1));
+ let (x1, y1) = (a.x + a.width * 7 / 8, a.y + a.height * 7 / 8);
+ let (mut main, mut anti) = (0u64, 0u64);
+ for y in (y0..y1.saturating_sub(1)).step_by(16) {
+ for x in (x0..x1.saturating_sub(1)).step_by(8) {
+ let (x, y) = (x & !1, y & !1);
+ let p = |dx: usize, dy: usize| d[(y + dy) * w + x + dx] as i64;
+ main += (p(0, 0) - p(1, 1)).unsigned_abs();
+ anti += (p(1, 0) - p(0, 1)).unsigned_abs();
+ }
+ }
+ Cfa::bayer(if main < anti { "GRBG" } else { "RGGB" }).expect("static")
+}
+
+pub(crate) fn decode(bytes: &[u8]) -> Result {
+ let tiff = Tiff::parse(bytes)?;
+ let ifd0 = &tiff.ifds[0];
+ let info = ifd0.image()?;
+ let (w, h) = (info.width as usize, info.height as usize);
+ let n = w.checked_mul(h).filter(|n| *n > 0 && *n <= crate::MAX_SAMPLES).ok_or(RawError::Limit("image too large"))?;
+ let chunks = info.chunks(bytes.len() as u64);
+ let total: u64 = chunks.iter().map(|c| c.len).sum();
+ let (mut data, bits) = if info.compression != 1 {
+ return Err(RawError::Unsupported(format!("ORF compression {}", info.compression)));
+ } else if total >= (n as u64) * 2 {
+ let d = read_image(bytes, &info, tiff.order, Packing::Word16)?;
+ (d, 16)
+ } else if total * 8 >= (n as u64) * 12 && total * 8 < (n as u64) * 13 && chunks.len() == 1 {
+ let src = chunk_bytes(bytes, &chunks[0]).ok_or_else(|| RawError::Corrupt("ORF strip outside file".into()))?;
+ let stride = src.len() / h;
+ let mut d = vec![0u16; n];
+ d.par_chunks_mut(w).enumerate().for_each(|(y, row)| unpack_row_le32_msb(&src[y * stride..(y + 1) * stride], 12, row));
+ (RawData::U16(d), 12)
+ } else {
+ return Err(RawError::Unsupported("Olympus compressed ORF".into()));
+ };
+ let RawData::U16(ref mut samples) = data else { return Err(RawError::Unsupported("float ORF".into())) };
+ let bits = if bits == 16 && samples.iter().step_by(7).filter(|v| *v & 15 != 0).count() * 700 <= n {
+ // 12-bit values stored in the top bits
+ samples.par_iter_mut().for_each(|v| *v >>= 4);
+ 12
+ } else if bits == 16 {
+ let mx = samples.iter().step_by(31).max().copied().unwrap_or(0);
+ if mx < 4096 {
+ 12
+ } else if mx < 16384 {
+ 14
+ } else {
+ 16
+ }
+ } else {
+ bits
+ };
+
+ let mn = maker_note(bytes, &tiff);
+ let ip = mn.as_ref().and_then(|m| sub_ifd(bytes, m, IMAGE_PROCESSING));
+ let active = match ip.as_ref().map(|i| CROP.map(|tag| i.u64(tag).map(|v| v as usize))) {
+ Some([Some(x), Some(y), Some(cw), Some(ch)]) if cw > 0 && ch > 0 && x + cw <= w && y + ch <= h => Rect::new(x, y, cw, ch),
+ _ => Rect::new(0, 0, w, h),
+ };
+ let cfa = cfa_from_data(samples, w, active);
+ let black = match ip.as_ref().and_then(|i| i.f64s(BLACK)).as_deref() {
+ Some(v @ [_, _, _, _]) => {
+ let a = cfa.shifted(active.x, active.y);
+ let values = a.pattern.iter().map(|&c| [v[0], (v[1] + v[2]) / 2.0, v[3]][c as usize] as f32).collect();
+ BlackLevel { repeat_rows: 2, repeat_cols: 2, values, ..Default::default() }
+ }
+ _ => BlackLevel::uniform(0.0),
+ };
+ let wb = ip
+ .as_ref()
+ .and_then(|i| i.f64s(WB_RB))
+ .filter(|v| v.len() >= 2 && v[0] > 0.0 && v[1] > 0.0)
+ .map(|v| [(v[0] / 256.0) as f32, 1.0, (v[1] / 256.0) as f32]);
+ let white = white_from_data(samples, bits);
+ let mut metadata = lightcraft_meta::from_tiff(&tiff);
+ metadata.width = Some(active.width as u32);
+ metadata.height = Some(active.height as u32);
+ let img = RawImage {
+ format: RawFormat::Orf,
+ width: w,
+ height: h,
+ cpp: 1,
+ data,
+ cfa: Some(cfa),
+ bits,
+ black,
+ white: vec![white],
+ active_area: active,
+ crop: Rect::new(0, 0, active.width, active.height),
+ orientation: Orientation::from_exif(ifd0.u16(t::ORIENTATION).unwrap_or(1)),
+ color: ColorData::default(),
+ wb_multipliers: wb,
+ linearized: false,
+ opcodes: OpcodeLists::default(),
+ metadata,
+ };
+ img.validate()?;
+ Ok(img)
+}
+
+/// The large preview JPEG referenced by CameraSettings `PreviewImageStart/Length` (relative to the note base).
+pub(crate) fn preview(bytes: &[u8]) -> Option<&[u8]> {
+ let tiff = Tiff::parse(bytes).ok()?;
+ let mn = maker_note(bytes, &tiff)?;
+ let cs = sub_ifd(bytes, &mn, CAMERA_SETTINGS)?;
+ let start = mn.base.checked_add(cs.u64(PREVIEW_START)?)? as usize;
+ let len = cs.u64(PREVIEW_LENGTH)? as usize;
+ bytes.get(start..start.checked_add(len)?.min(bytes.len()))
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use lightcraft_tiff::{ByteOrder, IfdBuilder, ImageData, TiffWriter};
+
+ fn orf(w: u32, h: u32, bits: u16, strip: Vec) -> Vec {
+ let mut ifd = IfdBuilder::new();
+ ifd.set(t::IMAGE_WIDTH, Value::Long(vec![w]));
+ ifd.set(t::IMAGE_LENGTH, Value::Long(vec![h]));
+ ifd.set(t::BITS_PER_SAMPLE, Value::Short(vec![bits]));
+ ifd.set(t::COMPRESSION, Value::Short(vec![1]));
+ ifd.set(t::PHOTOMETRIC, Value::Short(vec![1]));
+ ifd.set(t::MAKE, Value::Ascii("OLYMPUS IMAGING CORP.".into()));
+ ifd.set_image(ImageData::Strips { rows_per_strip: h, strips: vec![strip] });
+ let mut b = TiffWriter::new(ByteOrder::Little, false).write(&[ifd]).unwrap();
+ b[2] = b'R';
+ b[3] = b'O';
+ b
+ }
+
+ #[test]
+ fn word16_shifted_and_packed12() {
+ let (w, h) = (16usize, 4usize);
+ let px: Vec = (0..w * h).map(|i| ((i * 211) % 4096) as u16).collect();
+ let words: Vec = px.iter().flat_map(|v| (v << 4).to_le_bytes()).collect();
+ let bytes = orf(w as u32, h as u32, 16, words);
+ assert_eq!(crate::probe(&bytes), Some(RawFormat::Orf));
+ let r = crate::decode(&bytes).unwrap();
+ assert_eq!((r.data.clone(), r.bits), (RawData::U16(px.clone()), 12));
+ // 12-bit: MSB-first stream, every 32-bit word byte-swapped
+ let mut be = Vec::new();
+ for p in px.chunks(2) {
+ let v = (p[0] as u32) << 12 | p[1] as u32;
+ be.extend_from_slice(&[(v >> 16) as u8, (v >> 8) as u8, v as u8]);
+ }
+ let le: Vec = be.chunks(4).flat_map(|c| [c[3], c[2], c[1], c[0]]).collect();
+ let bytes = orf(w as u32, h as u32, 12, le);
+ assert_eq!(crate::decode(&bytes).unwrap().data, RawData::U16(px));
+ let bytes = orf(w as u32, h as u32, 16, vec![0; 40]);
+ assert!(matches!(crate::decode(&bytes), Err(RawError::Unsupported(_))));
+ }
+}
diff --git a/crates/raw/src/vendor/pef.rs b/crates/raw/src/vendor/pef.rs
new file mode 100644
index 000000000..57034d772
--- /dev/null
+++ b/crates/raw/src/vendor/pef.rs
@@ -0,0 +1,336 @@
+//! Pentax PEF — uncompressed and Pentax Huffman-compressed (compression 65535).
+//!
+//! Sources: TIFF 6.0 (container), ITU-T T.81 (JPEG) Annex H / F.1.2.1 for the lossless "difference category +
+//! additional bits" coding, the ExifTool Pentax tag-name documentation (maker-note `0x0200` BlackPoint, `0x0201`
+//! WhitePoint i.e. WB levels, `0x0220` HuffmanTable) and our own black-box analysis of CC0 samples from
+//! raw.pixls.us (K10D, K-5 II s, K-3):
+//!
+//! - The Huffman table is stored in the file (maker note `0x0220`, in the maker note's byte order): a `u16` `d`,
+//! 12 further bytes, then `n = d + 12` `u16` codes left-aligned in 12 bits, then `n` code lengths (bytes).
+//! Symbol `i` is the difference category `i` (T.81 Table H.2: the number of additional bits).
+//! - The bit stream (MSB-first, no byte stuffing, continuous across rows) codes one difference per pixel; the
+//! additional bits follow T.81 F.1.2.1 (a leading 0 means a negative value).
+//! - Prediction: each pixel is predicted from the previous same-colour pixel in its row (two to the left); the
+//! first two pixels of a row are predicted from the first two pixels of the previous row of the same parity
+//! (starting at 0). Found by trying candidate predictors and checking the decoded rows are continuous.
+//! - Maker note `0x0038`/`0x0039`: the image area's left/top and width/height (verified against the data).
+
+use super::{black_from_columns, white_from_data};
+
+use crate::tiffraw::{Packing, read_image};
+use crate::{BlackLevel, Cfa, ColorData, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result};
+use lightcraft_geom::Orientation;
+use lightcraft_tiff::image::chunk_bytes;
+use lightcraft_tiff::{ByteOrder, Tiff, makernote, tags as t};
+
+const CROP_ORIGIN: u16 = 0x0038;
+const CROP_SIZE: u16 = 0x0039;
+const BLACK_POINT: u16 = 0x0200;
+const WB_LEVELS: u16 = 0x0201;
+const HUFFMAN: u16 = 0x0220;
+
+/// A canonical-by-table Huffman decoder with a 12-bit lookup.
+pub(crate) struct Huffman {
+ /// Indexed by the next 12 bits: (symbol, length); length 0 = invalid.
+ lut: Vec<(u8, u8)>,
+}
+
+impl Huffman {
+ /// From `(left-aligned 12-bit code, length)` per symbol.
+ pub fn new(codes: &[(u16, u8)]) -> Result {
+ let mut lut = vec![(0u8, 0u8); 4096];
+ for (sym, &(code, len)) in codes.iter().enumerate() {
+ if len == 0 {
+ continue;
+ }
+ if len > 12 || sym > 16 {
+ return Err(RawError::Unsupported(format!("PEF Huffman code of length {len}")));
+ }
+ let first = (code >> (12 - len) << (12 - len)) as usize;
+ for e in &mut lut[first..first + (1 << (12 - len))] {
+ if e.1 == 0 {
+ *e = (sym as u8, len);
+ }
+ }
+ }
+ Ok(Huffman { lut })
+ }
+}
+
+/// MSB-first bit reader that yields zeros past the end.
+pub(crate) struct Bits<'a> {
+ src: &'a [u8],
+ pos: usize,
+ acc: u64,
+ n: u32,
+}
+
+impl<'a> Bits<'a> {
+ pub fn new(src: &'a [u8]) -> Self {
+ Bits { src, pos: 0, acc: 0, n: 0 }
+ }
+ #[inline]
+ fn fill(&mut self) {
+ while self.n <= 56 {
+ let b = self.src.get(self.pos).copied().unwrap_or(0);
+ self.pos += 1;
+ self.acc |= (b as u64) << (56 - self.n);
+ self.n += 8;
+ }
+ }
+ #[inline]
+ pub fn peek(&mut self, k: u32) -> u32 {
+ if self.n < k {
+ self.fill();
+ }
+ (self.acc >> (64 - k)) as u32
+ }
+ #[inline]
+ pub fn skip(&mut self, k: u32) {
+ self.acc <<= k;
+ self.n -= k;
+ }
+ #[inline]
+ pub fn get(&mut self, k: u32) -> u32 {
+ if k == 0 {
+ return 0;
+ }
+ let v = self.peek(k);
+ self.skip(k);
+ v
+ }
+ pub fn overrun(&self) -> bool {
+ self.pos > self.src.len() + 8
+ }
+}
+
+/// Decode one difference: Huffman category then T.81 additional bits.
+#[inline]
+pub(crate) fn diff(bits: &mut Bits, h: &Huffman) -> Option {
+ let (sym, len) = h.lut[bits.peek(12) as usize];
+ if len == 0 {
+ return None;
+ }
+ bits.skip(len as u32);
+ let s = sym as u32;
+ if s == 0 {
+ return Some(0);
+ }
+ if s > 16 {
+ return None;
+ }
+ let v = bits.get(s) as i32;
+ Some(if v < 1 << (s - 1) { v - (1 << s) + 1 } else { v })
+}
+
+/// Parse the maker-note Huffman table.
+pub(crate) fn table(b: &[u8], order: ByteOrder) -> Result {
+ let u16_at = |i: usize| b.get(i..i + 2).map(|s| order.u16([s[0], s[1]]));
+ let d = u16_at(0).ok_or_else(|| RawError::Corrupt("PEF Huffman table too short".into()))? as usize;
+ let n = d + 12;
+ if n > 17 || b.len() < 14 + 3 * n {
+ return Err(RawError::Corrupt(format!("PEF Huffman table: {n} symbols in {} bytes", b.len())));
+ }
+ let codes: Vec<(u16, u8)> = (0..n).map(|i| (u16_at(14 + 2 * i).unwrap_or(0) & 0x0fff, b[14 + 2 * n + i])).collect();
+ Huffman::new(&codes)
+}
+
+/// Decode a `w × h` Huffman-coded image.
+pub(crate) fn decode_huffman(src: &[u8], h: &Huffman, w: usize, hgt: usize, bits_per: u32) -> Result> {
+ let mut out = vec![0u16; w * hgt];
+ let mut bits = Bits::new(src);
+ let mut vpred = [[0i32; 2]; 2];
+ let max = (1i32 << bits_per.min(16)) - 1;
+ for y in 0..hgt {
+ let mut hpred = [0i32; 2];
+ let row = &mut out[y * w..(y + 1) * w];
+ for (x, o) in row.iter_mut().enumerate() {
+ let d = diff(&mut bits, h).ok_or_else(|| RawError::Corrupt(format!("PEF: invalid Huffman code at row {y}")))?;
+ let p = if x < 2 {
+ vpred[y & 1][x] += d;
+ hpred[x] = vpred[y & 1][x];
+ hpred[x]
+ } else {
+ hpred[x & 1] += d;
+ hpred[x & 1]
+ };
+ *o = p.clamp(0, max) as u16;
+ }
+ if bits.overrun() {
+ return Err(RawError::Corrupt(format!("PEF: data ends at row {y} of {hgt}")));
+ }
+ }
+ Ok(out)
+}
+
+/// The image area without all-dark leading/trailing columns (masked or empty), kept at even offsets.
+fn dark_trimmed(d: &[u16], w: usize, h: usize) -> Rect {
+ if w < 256 || h == 0 {
+ return Rect::new(0, 0, w, h);
+ }
+ let step = (h / 256).max(1);
+ let col_mean = |x: usize| (0..h).step_by(step).map(|y| d[y * w + x] as f64).sum::() / h.div_ceil(step) as f64;
+ let interior = (w / 4..w * 3 / 4).step_by(w / 64).map(col_mean).sum::() / 32.0;
+ let dark = interior * 0.05;
+ let mut left = 0;
+ while left < 64 && col_mean(left) < dark {
+ left += 1;
+ }
+ let mut right = w;
+ while right > w - 64 && col_mean(right - 1) < dark {
+ right -= 1;
+ }
+ let left = left.next_multiple_of(2);
+ Rect::new(left, 0, (right - left) & !1, h)
+}
+
+pub(crate) fn decode(bytes: &[u8]) -> Result {
+ let tiff = Tiff::parse(bytes)?;
+ let ifd0 = &tiff.ifds[0];
+ let info = ifd0.image()?;
+ let (w, hgt) = (info.width as usize, info.height as usize);
+ let bits = info.bits() as u32;
+ if w == 0 || hgt == 0 || w.saturating_mul(hgt) > crate::MAX_SAMPLES {
+ return Err(RawError::Limit("image too large"));
+ }
+ let make = ifd0.string(t::MAKE).unwrap_or_default();
+ let mn =
+ tiff.exif().and_then(|e| e.get(t::MAKER_NOTE)).and_then(|e| makernote::parse_makernote(bytes, e.offset, e.count() as u64, tiff.order, &make));
+ let data = match info.compression {
+ 65535 => {
+ let mn = mn.as_ref().ok_or_else(|| RawError::Corrupt("compressed PEF without maker note".into()))?;
+ let tb = mn.ifd.bytes(HUFFMAN).ok_or_else(|| RawError::Unsupported("compressed PEF without a Huffman table".into()))?;
+ let huff = table(tb, mn.order)?;
+ let chunks = info.chunks(bytes.len() as u64);
+ let first = chunks.first().ok_or_else(|| RawError::Corrupt("PEF without image data".into()))?;
+ let mut c = *first;
+ c.len = chunks.iter().map(|c| c.len).sum::().max(first.len);
+ let src = chunk_bytes(bytes, &c)
+ .or_else(|| bytes.get(first.offset as usize..))
+ .ok_or_else(|| RawError::Corrupt("PEF data outside file".into()))?;
+ RawData::U16(decode_huffman(src, &huff, w, hgt, bits)?)
+ }
+ 1 => {
+ let strip: u64 = info.chunks(bytes.len() as u64).iter().map(|c| c.len).sum();
+ let packing = if strip >= (w * hgt * 2) as u64 { Packing::Word16 } else { Packing::Msb };
+ read_image(bytes, &info, tiff.order, packing)?
+ }
+ c => return Err(RawError::Unsupported(format!("PEF compression {c}"))),
+ };
+ let RawData::U16(ref samples) = data else { return Err(RawError::Unsupported("float PEF".into())) };
+
+ let pair = |tag: u16| mn.as_ref().and_then(|m| m.ifd.u64s(tag)).filter(|v| v.len() == 2).map(|v| (v[0] as usize, v[1] as usize));
+ let active = match (pair(CROP_ORIGIN), pair(CROP_SIZE)) {
+ (Some((x, y)), Some((cw, ch))) if cw > 0 && ch > 0 && x + cw <= w && y + ch <= hgt => Rect::new(x, y, cw, ch),
+ _ => dark_trimmed(samples, w, hgt),
+ };
+ let cfa = match (ifd0.u64s(t::CFA_REPEAT_PATTERN_DIM).as_deref(), ifd0.bytes(t::CFA_PATTERN_EP)) {
+ (Some([2, 2]), Some(p)) if p.len() == 4 && p.iter().all(|&c| c <= 2) => Cfa { width: 2, height: 2, pattern: p.to_vec() },
+ _ => Cfa::bayer("BGGR").expect("static"),
+ };
+ let black = match mn.as_ref().and_then(|m| m.ifd.f64s(BLACK_POINT)).as_deref() {
+ Some(v @ [_, _, _, _]) => {
+ // RGGB-ordered levels → per CFA position at the active area origin
+ let a = cfa.shifted(active.x, active.y);
+ let values = a.pattern.iter().map(|&c| [v[0], (v[1] + v[2]) / 2.0, v[3]][c as usize] as f32).collect();
+ BlackLevel { repeat_rows: 2, repeat_cols: 2, values, ..Default::default() }
+ }
+ _ if active.x >= 4 => black_from_columns(samples, w, 0..active.x.saturating_sub(2), 0..hgt, active),
+ _ => BlackLevel::uniform(0.0),
+ };
+ let wb = mn.as_ref().and_then(|m| m.ifd.f64s(WB_LEVELS)).filter(|v| v.len() == 4 && v.iter().all(|x| *x > 0.0)).map(|v| {
+ let g = (v[1] + v[2]) / 2.0;
+ [(v[0] / g) as f32, 1.0, (v[3] / g) as f32]
+ });
+ let white = white_from_data(samples, bits);
+ let mut metadata = lightcraft_meta::from_tiff(&tiff);
+ metadata.width = Some(active.width as u32);
+ metadata.height = Some(active.height as u32);
+ let img = RawImage {
+ format: RawFormat::Pef,
+ width: w,
+ height: hgt,
+ cpp: 1,
+ data,
+ cfa: Some(cfa),
+ bits,
+ black,
+ white: vec![white],
+ active_area: active,
+ crop: Rect::new(0, 0, active.width, active.height),
+ orientation: Orientation::from_exif(ifd0.u16(t::ORIENTATION).unwrap_or(1)),
+ color: ColorData::default(),
+ wb_multipliers: wb,
+ linearized: false,
+ opcodes: OpcodeLists::default(),
+ metadata,
+ };
+ img.validate()?;
+ Ok(img)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ /// Our own prefix code for categories 0..=14: lengths 2,2,3,3,4,5,..., codes assigned canonically.
+ fn test_table() -> (Vec, Vec<(u16, u8)>) {
+ let lens: [u8; 15] = [3, 2, 2, 3, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 12];
+ let mut order: Vec = (0..15).collect();
+ order.sort_by_key(|&i| (lens[i], i));
+ let mut codes = vec![(0u16, 0u8); 15];
+ let (mut code, mut prev) = (0u32, lens[order[0]]);
+ for (k, &i) in order.iter().enumerate() {
+ if k > 0 {
+ code = (code + 1) << (lens[i] - prev);
+ }
+ prev = lens[i];
+ codes[i] = ((code << (12 - lens[i])) as u16, lens[i]);
+ }
+ let mut b = vec![0u8, 3];
+ b.extend_from_slice(&[0; 12]);
+ for c in &codes {
+ b.extend_from_slice(&c.0.to_be_bytes());
+ }
+ b.extend(codes.iter().map(|c| c.1));
+ (b, codes)
+ }
+
+ fn encode(px: &[u16], w: usize, codes: &[(u16, u8)]) -> Vec {
+ let mut bits: Vec = Vec::new();
+ let put = |v: u32, n: u8, bits: &mut Vec| (0..n).rev().for_each(|i| bits.push(v >> i & 1 == 1));
+ let mut vpred = [[0i32; 2]; 2];
+ for (y, row) in px.chunks(w).enumerate() {
+ let mut hpred = [0i32; 2];
+ for (x, &v) in row.iter().enumerate() {
+ let pred = if x < 2 { vpred[y & 1][x] } else { hpred[x & 1] };
+ let d = v as i32 - pred;
+ if x < 2 {
+ vpred[y & 1][x] = v as i32;
+ }
+ hpred[x & 1] = v as i32;
+ let s = if d == 0 { 0 } else { 32 - d.unsigned_abs().leading_zeros() } as u8;
+ let (c, l) = codes[s as usize];
+ put((c >> (12 - l)) as u32, l, &mut bits);
+ if s > 0 {
+ let extra = if d < 0 { (d - 1) as u32 & ((1 << s) - 1) } else { d as u32 };
+ put(extra, s, &mut bits);
+ }
+ }
+ }
+ bits.chunks(8).map(|c| c.iter().enumerate().fold(0u8, |a, (i, &b)| a | (b as u8) << (7 - i))).collect()
+ }
+
+ #[test]
+ fn huffman_round_trip() {
+ let (tb, codes) = test_table();
+ let h = table(&tb, ByteOrder::Big).unwrap();
+ let (w, hgt) = (37usize, 9usize);
+ let px: Vec = (0..w * hgt).map(|i| ((i * 7919) % 16384) as u16).collect();
+ let src = encode(&px, w, &codes);
+ assert_eq!(decode_huffman(&src, &h, w, hgt, 14).unwrap(), px);
+ // truncated data is reported, not a panic
+ assert!(decode_huffman(&src[..src.len() / 3], &h, w, hgt, 14).is_err());
+ assert!(table(&tb[..20], ByteOrder::Big).is_err());
+ }
+}
diff --git a/crates/raw/src/vendor/raf.rs b/crates/raw/src/vendor/raf.rs
new file mode 100644
index 000000000..45dd3e93c
--- /dev/null
+++ b/crates/raw/src/vendor/raf.rs
@@ -0,0 +1,313 @@
+//! Fujifilm RAF — uncompressed Bayer and X-Trans.
+//!
+//! Sources: the ExifTool FujiFilm tag-name documentation (RAF header record tags `0x0100` RawImageFullSize,
+//! `0x0110` RawImageCropTopLeft, `0x0111` RawImageCroppedSize, `0x0131` XTransLayout; raw-IFD tags `0xf001`
+//! width, `0xf002` height, `0xf003` bits, `0xf007` strip offset, `0xf008` strip byte count, `0xf00a` black
+//! levels, `0xf00e` WB_GRBLevels) and our own black-box analysis of CC0 samples from raw.pixls.us:
+//!
+//! - The file starts with a 16-byte magic, version and camera name; at byte 84 eight big-endian `u32`s give the
+//! embedded JPEG (offset, length), the header-record directory (offset, length) and the raw block (offset,
+//! length). The record directory is a `u32` count then `(u16 tag, u16 size, data)` records, big-endian.
+//! - In current bodies the raw block is a little-endian TIFF whose IFD0 has a `0xf000` IFD pointer to the raw
+//! IFD; offsets in it are relative to the raw block.
+//! - Sample packing, found by testing candidate bit orders for the smoothest image: 16-bit little-endian words;
+//! 12-bit LSB-first bit stream; 14-bit (and other depths) stored as little-endian 32-bit words read MSB-first.
+//! `0xf005`, when non-zero, is the row stride in 32-bit words.
+//! - CFA: the X-Trans layout record (36 bytes, 0 = R, 1 = G, 2 = B) is stored in reverse order and, reversed,
+//! is anchored at raw pixel (0, 0) — verified on X-Trans I and III samples by minimising the difference
+//! between horizontally / vertically adjacent "green" samples over all 72 orientations/phases, and by colour
+//! renders. Bayer bodies (X-A series) use RGGB at (0, 0), verified by colour renders.
+//! - Fujifilm's compressed RAF variants are reported as unsupported (their embedded preview still works).
+
+use super::white_from_data;
+use crate::unpack::{unpack_lsb, unpack_msb};
+use crate::{BlackLevel, Cfa, ColorData, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result};
+use lightcraft_geom::Orientation;
+use lightcraft_tiff::{ByteOrder, Ifd, Tiff};
+use rayon::prelude::*;
+
+const CROP_TOP_LEFT: u16 = 0x0110;
+const CROPPED_SIZE: u16 = 0x0111;
+const XTRANS_LAYOUT: u16 = 0x0131;
+
+/// The fixed-position header pointers.
+pub(crate) struct Header<'a> {
+ pub jpeg: Option<&'a [u8]>,
+ pub records: Vec<(u16, &'a [u8])>,
+ pub raw: Option<&'a [u8]>,
+}
+
+fn be32(b: &[u8], at: usize) -> Option {
+ b.get(at..at + 4).map(|s| u32::from_be_bytes([s[0], s[1], s[2], s[3]]) as usize)
+}
+
+fn slice(b: &[u8], off: Option, len: Option) -> Option<&[u8]> {
+ let (o, l) = (off?, len?);
+ if o == 0 || l == 0 {
+ return None;
+ }
+ b.get(o..o.checked_add(l)?.min(b.len())).filter(|s| !s.is_empty())
+}
+
+pub(crate) fn header(b: &[u8]) -> Result> {
+ if !b.starts_with(b"FUJIFILMCCD-RAW") || b.len() < 108 {
+ return Err(RawError::NotRaw);
+ }
+ let jpeg = slice(b, be32(b, 84), be32(b, 88));
+ let raw = slice(b, be32(b, 100), be32(b, 104));
+ let mut records = Vec::new();
+ if let Some(dir) = slice(b, be32(b, 92), be32(b, 96)) {
+ let n = be32(dir, 0).unwrap_or(0).min(4096);
+ let mut p = 4;
+ for _ in 0..n {
+ let Some(h) = dir.get(p..p + 4) else { break };
+ let (tag, size) = (u16::from_be_bytes([h[0], h[1]]), u16::from_be_bytes([h[2], h[3]]) as usize);
+ let Some(d) = dir.get(p + 4..p + 4 + size) else { break };
+ records.push((tag, d));
+ p += 4 + size;
+ }
+ }
+ Ok(Header { jpeg, records, raw })
+}
+
+fn record<'a>(h: &Header<'a>, tag: u16) -> Option<&'a [u8]> {
+ h.records.iter().find(|r| r.0 == tag).map(|r| r.1)
+}
+
+/// A record holding two big-endian `u16`s.
+fn pair(h: &Header, tag: u16) -> Option<(usize, usize)> {
+ let d = record(h, tag).filter(|d| d.len() >= 4)?;
+ Some((u16::from_be_bytes([d[0], d[1]]) as usize, u16::from_be_bytes([d[2], d[3]]) as usize))
+}
+
+fn raw_ifd(raw: &[u8]) -> Result {
+ let tiff = Tiff::parse(raw).map_err(|_| RawError::Unsupported("RAF without a raw IFD (older FinePix layout)".into()))?;
+ let off = tiff.ifds.first().and_then(|i| i.u64(0xf000)).ok_or_else(|| RawError::Unsupported("RAF without a raw IFD".into()))?;
+ let (ifd, _) = lightcraft_tiff::parse_ifd_at(raw, off, ByteOrder::Little, 0, false, &Default::default())?;
+ Ok(ifd)
+}
+
+/// The CFA: X-Trans from the (reversed) layout record, else RGGB.
+fn cfa(h: &Header) -> Cfa {
+ match record(h, XTRANS_LAYOUT) {
+ Some(l) if l.len() == 36 && l.iter().all(|&c| c <= 2) => Cfa { width: 6, height: 6, pattern: l.iter().rev().copied().collect() },
+ _ => Cfa::bayer("RGGB").expect("static"),
+ }
+}
+
+/// Unpack one row of `bits`-bit samples in Fujifilm's packing.
+pub(crate) fn unpack_row(src: &[u8], bits: u32, out: &mut [u16]) {
+ match bits {
+ 16 => out.iter_mut().zip(src.chunks_exact(2)).for_each(|(o, c)| *o = u16::from_le_bytes([c[0], c[1]])),
+ 12 => unpack_lsb(src, 12, out),
+ _ => {
+ let swapped: Vec = src
+ .chunks(4)
+ .flat_map(|c| {
+ let mut w = [0u8; 4];
+ w[..c.len()].copy_from_slice(c);
+ [w[3], w[2], w[1], w[0]]
+ })
+ .collect();
+ unpack_msb(&swapped, bits, out)
+ }
+ }
+}
+
+pub(crate) fn decode(bytes: &[u8]) -> Result {
+ let h = header(bytes)?;
+ let raw = h.raw.ok_or_else(|| RawError::Corrupt("RAF without raw data".into()))?;
+ let ifd = raw_ifd(raw)?;
+ let get = |t: u16| ifd.u64(t).map(|v| v as usize);
+ let (w, hgt) = (get(0xf001).unwrap_or(0), get(0xf002).unwrap_or(0));
+ let bits = get(0xf003).unwrap_or(0) as u32;
+ let (off, len) = (get(0xf007).unwrap_or(0), get(0xf008).unwrap_or(0));
+ if w == 0 || hgt == 0 || !(8..=16).contains(&bits) {
+ return Err(RawError::Corrupt(format!("RAF raw IFD: {w}x{hgt}, {bits} bits")));
+ }
+ let n = w.checked_mul(hgt).filter(|n| *n <= crate::MAX_SAMPLES).ok_or(RawError::Limit("image too large"))?;
+ // samples of any depth may be stored in 16-bit words (e.g. 14-bit X-T20); otherwise packed at `bits`
+ let store = if len as u64 >= (n as u64) * 2 { 16 } else { bits };
+ let packed_row = if store == 16 { w * 2 } else { (w * store as usize).div_ceil(8) };
+ let stride = match get(0xf005).unwrap_or(0) {
+ 0 => packed_row,
+ words => (words * 4).max(packed_row),
+ };
+ if (len as u64) < (stride as u64) * (hgt as u64 - 1) + packed_row as u64 {
+ return Err(RawError::Unsupported(format!("Fujifilm compressed RAF ({len} bytes for {w}x{hgt} {bits}-bit)")));
+ }
+ let src = raw.get(off..off.saturating_add(len).min(raw.len())).ok_or_else(|| RawError::Corrupt("RAF strip outside file".into()))?;
+ let mut data = vec![0u16; n];
+ data.par_chunks_mut(w).enumerate().for_each(|(y, row)| {
+ let s = src.get(y * stride..(y * stride + packed_row).min(src.len())).unwrap_or(&[]);
+ unpack_row(s, store, row);
+ });
+
+ let cfa = cfa(&h);
+ let active = match (pair(&h, CROP_TOP_LEFT), pair(&h, CROPPED_SIZE)) {
+ (Some((top, left)), Some((ch, cw))) if ch > 0 && cw > 0 => Rect::new(left, top, cw, ch).clipped(w, hgt),
+ _ => Rect::new(0, 0, w, hgt),
+ };
+ let active = if active.width == 0 || active.height == 0 { Rect::new(0, 0, w, hgt) } else { active };
+ // black levels: one per CFA position, anchored at (0, 0) of the raw data; re-anchor at the active area
+ let black = match ifd.f64s(0xf00a) {
+ Some(v) if v.len() == cfa.pattern.len() && v.iter().any(|b| *b != v[0]) => {
+ let (cw, chh) = (cfa.width, cfa.height);
+ let values = (0..chh * cw).map(|i| v[((i / cw + active.y) % chh) * cw + (i % cw + active.x) % cw] as f32).collect();
+ BlackLevel { repeat_rows: chh, repeat_cols: cw, values, ..Default::default() }
+ }
+ Some(v) if !v.is_empty() => BlackLevel::uniform(v[0] as f32),
+ _ => BlackLevel::uniform(0.0),
+ };
+ let wb =
+ ifd.f64s(0xf00e).filter(|v| v.len() >= 3 && v.iter().take(3).all(|x| *x > 0.0)).map(|v| [(v[1] / v[0]) as f32, 1.0, (v[2] / v[0]) as f32]);
+ let white = white_from_data(&data, bits);
+
+ let mut metadata = h.jpeg.map(lightcraft_meta::extract).unwrap_or_default();
+ metadata.width = Some(active.width as u32);
+ metadata.height = Some(active.height as u32);
+ let orientation = metadata.orientation.unwrap_or(Orientation::Normal);
+ let img = RawImage {
+ format: RawFormat::Raf,
+ width: w,
+ height: hgt,
+ cpp: 1,
+ data: RawData::U16(data),
+ cfa: Some(cfa),
+ bits,
+ black,
+ white: vec![white],
+ active_area: active,
+ crop: Rect::new(0, 0, active.width, active.height),
+ orientation,
+ color: ColorData::default(),
+ wb_multipliers: wb,
+ linearized: false,
+ opcodes: OpcodeLists::default(),
+ metadata,
+ };
+ img.validate()?;
+ Ok(img)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ /// Build a minimal RAF: header, records, raw block (LE TIFF whose IFD0 points to a 0xf000 raw IFD).
+ pub(crate) fn raf(w: u32, h: u32, bits: u32, strip: Vec, layout: Option<[u8; 36]>, jpeg: &[u8]) -> Vec {
+ let mut block = b"II*\0\x08\0\0\0".to_vec();
+ let sub_off: u32 = 8 + 2 + 12 + 4;
+ block.extend_from_slice(&1u16.to_le_bytes());
+ block.extend_from_slice(&[0x00, 0xf0, 13, 0, 1, 0, 0, 0]);
+ block.extend_from_slice(&sub_off.to_le_bytes());
+ block.extend_from_slice(&0u32.to_le_bytes());
+ let n = 7u32;
+ let wb_off = sub_off + 2 + 12 * n + 4;
+ let strip_off = wb_off + 12;
+ let entries: [(u16, u32, u32); 7] = [
+ (0xf001, 1, w),
+ (0xf002, 1, h),
+ (0xf003, 1, bits),
+ (0xf007, 1, strip_off),
+ (0xf008, 1, strip.len() as u32),
+ (0xf00a, 1, 64),
+ (0xf00e, 3, wb_off),
+ ];
+ block.extend_from_slice(&(n as u16).to_le_bytes());
+ for (tag, count, v) in entries {
+ block.extend_from_slice(&tag.to_le_bytes());
+ block.extend_from_slice(&4u16.to_le_bytes());
+ block.extend_from_slice(&count.to_le_bytes());
+ block.extend_from_slice(&v.to_le_bytes());
+ }
+ block.extend_from_slice(&0u32.to_le_bytes());
+ for v in [300u32, 600, 450] {
+ block.extend_from_slice(&v.to_le_bytes());
+ }
+ block.extend_from_slice(&strip);
+ let mut recs: Vec = Vec::new();
+ let mut nrec = 0u32;
+ let mut add = |tag: u16, d: &[u8]| {
+ recs.extend_from_slice(&tag.to_be_bytes());
+ recs.extend_from_slice(&(d.len() as u16).to_be_bytes());
+ recs.extend_from_slice(d);
+ nrec += 1;
+ };
+ add(0x0100, &[(h >> 8) as u8, h as u8, (w >> 8) as u8, w as u8]);
+ add(0x0110, &[0, 2, 0, 4]);
+ add(0x0111, &[0, (h - 2) as u8, 0, (w - 4) as u8]);
+ if let Some(l) = layout {
+ add(0x0131, &l);
+ }
+ let mut dir = nrec.to_be_bytes().to_vec();
+ dir.extend_from_slice(&recs);
+ let mut out = b"FUJIFILMCCD-RAW 0201FF000000TEST".to_vec();
+ out.resize(108, 0);
+ let jpeg_off = 160u32;
+ let dir_off = jpeg_off + jpeg.len() as u32;
+ let raw_off = dir_off + dir.len() as u32;
+ for (i, v) in [jpeg_off, jpeg.len() as u32, dir_off, dir.len() as u32, raw_off, block.len() as u32].iter().enumerate() {
+ out[84 + i * 4..88 + i * 4].copy_from_slice(&v.to_be_bytes());
+ }
+ out.resize(160, 0);
+ out.extend_from_slice(jpeg);
+ out.extend_from_slice(&dir);
+ out.extend_from_slice(&block);
+ out
+ }
+
+ fn pack(px: &[u16], bits: u32) -> Vec {
+ match bits {
+ 16 => px.iter().flat_map(|v| v.to_le_bytes()).collect(),
+ 12 => {
+ let mut o = Vec::new();
+ for p in px.chunks(2) {
+ let v = p[0] as u32 | (p[1] as u32) << 12;
+ o.extend_from_slice(&[v as u8, (v >> 8) as u8, (v >> 16) as u8]);
+ }
+ o
+ }
+ _ => {
+ // MSB-first stream, then byte-swap every 32-bit word
+ let mut bitsv: Vec = Vec::new();
+ let (mut acc, mut n) = (0u64, 0u32);
+ for &v in px {
+ acc = (acc << bits) | v as u64;
+ n += bits;
+ while n >= 8 {
+ bitsv.push((acc >> (n - 8)) as u8);
+ n -= 8;
+ }
+ }
+ if n > 0 {
+ bitsv.push((acc << (8 - n)) as u8);
+ }
+ bitsv.resize(bitsv.len().div_ceil(4) * 4, 0);
+ bitsv.chunks(4).flat_map(|c| [c[3], c[2], c[1], c[0]]).collect()
+ }
+ }
+ }
+
+ #[test]
+ fn decodes_all_packings() {
+ let (w, h) = (16u32, 6u32);
+ for (bits, store) in [(12u32, 12u32), (14, 14), (14, 16), (16, 16)] {
+ let px: Vec = (0..w * h).map(|i| ((i * 977) % (1 << bits.min(14))) as u16).collect();
+ let layout: [u8; 36] = std::array::from_fn(|i| Cfa::xtrans().pattern[35 - i]);
+ let bytes = raf(w, h, bits, pack(&px, store), Some(layout), b"\xff\xd8\xff\xd9");
+ assert_eq!(crate::probe(&bytes), Some(RawFormat::Raf));
+ let r = crate::decode(&bytes).unwrap_or_else(|e| panic!("{bits}: {e}"));
+ assert_eq!(r.data, RawData::U16(px), "{bits}-bit");
+ assert_eq!(r.cfa, Some(Cfa::xtrans()));
+ assert_eq!(r.active_area, Rect::new(4, 2, 12, 4));
+ assert_eq!(r.black.mean(), 64.0);
+ assert_eq!(r.wb_multipliers, Some([2.0, 1.0, 1.5]));
+ }
+ let bytes = raf(w, h, 16, pack(&[0; 96], 16), None, b"");
+ assert_eq!(crate::decode(&bytes).unwrap().cfa.unwrap().name(), "RGGB");
+ // too little data for the size: compressed
+ let bytes = raf(w, h, 14, vec![0; 20], None, b"");
+ assert!(matches!(crate::decode(&bytes), Err(RawError::Unsupported(_))));
+ }
+}
diff --git a/crates/raw/src/vendor/rw2.rs b/crates/raw/src/vendor/rw2.rs
new file mode 100644
index 000000000..47b9e5436
--- /dev/null
+++ b/crates/raw/src/vendor/rw2.rs
@@ -0,0 +1,251 @@
+//! Panasonic RW2 — the packed (unquantised) variant.
+//!
+//! Sources: the ExifTool PanasonicRaw tag-name documentation (`0x0002/0x0003` sensor size, `0x0004–0x0007`
+//! sensor borders, `0x0009` CFA pattern, `0x000a` bits per sample, `0x000e` linearity limit, `0x001c–0x001e`
+//! black levels, `0x0024–0x0026` WB levels, `0x0017` ISO, `0x002d` raw format, `0x002e` JpgFromRaw, `0x0118` raw data
+//! offset) and our own black-box analysis of CC0 samples from raw.pixls.us (DC-G9 12-bit, DC-GH5S 14-bit):
+//!
+//! - The raw data is a sequence of 0x4000-byte chunks; each chunk is stored rotated: its logical content is
+//! bytes `0x1ff8..0x4000` followed by bytes `0..0x1ff8`. (Found by trying every 8-byte rotation and keeping
+//! the one under which the block at each chunk seam is as smooth as its neighbours.)
+//! - The logical stream is made of 16-byte blocks, each a little-endian 128-bit word read LSB-first from bit 0:
+//! 12-bit data packs 10 samples (8 padding bits at the top), 14-bit data 9 samples (2 padding bits at the top).
+//! Found from the per-bit-position probability of a 1 (padding is always 0; the top bits of 12-bit samples and
+//! the two lowest bits of the GH5S's 14-bit samples are almost always 0), confirmed by image smoothness and by
+//! the decoded masked-free dark level matching the recorded black levels.
+//! - Pixels fill rows left to right, rows top to bottom.
+//!
+//! Panasonic's quantised variant (raw format 4: 14 pixels per block, two 12-bit seeds plus four groups of a 2-bit
+//! scale and three 8-bit codes) is not decoded: our black-box analysis identified the block layout but not the
+//! reconstruction rule for every scale, and no permissively licensed description exists. Those files report
+//! [`RawError::Unsupported`]; their embedded preview still works.
+
+use crate::{BlackLevel, Cfa, ColorData, OpcodeLists, RawData, RawError, RawFormat, RawImage, Rect, Result};
+use lightcraft_geom::Orientation;
+use lightcraft_tiff::{Tiff, tags as t};
+use rayon::prelude::*;
+
+const SENSOR_WIDTH: u16 = 0x0002;
+const SENSOR_HEIGHT: u16 = 0x0003;
+const BORDER_TOP: u16 = 0x0004;
+const BORDER_LEFT: u16 = 0x0005;
+const BORDER_BOTTOM: u16 = 0x0006;
+const BORDER_RIGHT: u16 = 0x0007;
+const CFA_PATTERN: u16 = 0x0009;
+const BITS: u16 = 0x000a;
+const LINEARITY_LIMIT: u16 = 0x000e;
+const BLACK_RGB: [u16; 3] = [0x001c, 0x001d, 0x001e];
+const WB_RGB: [u16; 3] = [0x0024, 0x0025, 0x0026];
+const RAW_FORMAT: u16 = 0x002d;
+const ISO: u16 = 0x0017;
+const RAW_OFFSET: u16 = 0x0118;
+
+const CHUNK: usize = 0x4000;
+const ROTATE: usize = 0x1ff8;
+
+/// The logical byte stream of the raw data (chunks un-rotated). A trailing partial chunk is kept as is.
+pub(crate) fn unrotate(src: &[u8]) -> Vec {
+ let mut out = Vec::with_capacity(src.len());
+ for c in src.chunks(CHUNK) {
+ if c.len() == CHUNK {
+ out.extend_from_slice(&c[ROTATE..]);
+ out.extend_from_slice(&c[..ROTATE]);
+ } else {
+ out.extend_from_slice(c);
+ }
+ }
+ out
+}
+
+/// Samples per 16-byte block and the bit position of the first sample (kept for other layouts).
+fn layout(bits: u32) -> Option<(usize, u32)> {
+ match bits {
+ 12 => Some((10, 0)),
+ 14 => Some((9, 0)),
+ _ => None,
+ }
+}
+
+/// Unpack `out.len()` samples from the logical stream.
+pub(crate) fn unpack(stream: &[u8], bits: u32, out: &mut [u16]) -> Result<()> {
+ let (per, start) = layout(bits).ok_or_else(|| RawError::Unsupported(format!("RW2 packed {bits}-bit data")))?;
+ let mask = (1u128 << bits) - 1;
+ out.par_chunks_mut(per * 1024).enumerate().for_each(|(ci, o)| {
+ let first_block = ci * 1024;
+ for (bi, px) in o.chunks_mut(per).enumerate() {
+ let at = (first_block + bi) * 16;
+ let Some(b) = stream.get(at..at + 16) else {
+ px.fill(0);
+ continue;
+ };
+ let word = u128::from_le_bytes(b.try_into().expect("16 bytes"));
+ for (i, p) in px.iter_mut().enumerate() {
+ *p = ((word >> (start + i as u32 * bits)) & mask) as u16;
+ }
+ }
+ });
+ Ok(())
+}
+
+fn cfa(code: u16) -> Cfa {
+ let s = match code {
+ 2 => "GRBG",
+ 3 => "GBRG",
+ 4 => "BGGR",
+ _ => "RGGB",
+ };
+ Cfa::bayer(s).expect("static")
+}
+
+pub(crate) fn decode(bytes: &[u8]) -> Result {
+ let tiff = Tiff::parse(bytes)?;
+ let ifd0 = &tiff.ifds[0];
+ let get = |tag: u16| ifd0.u64(tag).map(|v| v as usize);
+ let (w, h) = (get(SENSOR_WIDTH).unwrap_or(0), get(SENSOR_HEIGHT).unwrap_or(0));
+ let bits = get(BITS).unwrap_or(12) as u32;
+ let format = get(RAW_FORMAT).unwrap_or(0);
+ if w == 0 || h == 0 {
+ return Err(RawError::Corrupt("RW2 without sensor size".into()));
+ }
+ let n = w.checked_mul(h).filter(|n| *n <= crate::MAX_SAMPLES).ok_or(RawError::Limit("image too large"))?;
+ if format != 5 {
+ return Err(RawError::Unsupported(format!("Panasonic RW2 raw format {format} (quantised)")));
+ }
+ let (per, _) = layout(bits).ok_or_else(|| RawError::Unsupported(format!("RW2 packed {bits}-bit data")))?;
+ let off = get(RAW_OFFSET).or_else(|| get(t::STRIP_OFFSETS)).ok_or_else(|| RawError::Corrupt("RW2 without raw data offset".into()))?;
+ let src = bytes.get(off..).ok_or_else(|| RawError::Corrupt("RW2 raw data outside file".into()))?;
+ let need = n.div_ceil(per) * 16;
+ if src.len() < need {
+ return Err(RawError::Corrupt(format!("RW2 raw data truncated ({} of {need} bytes)", src.len())));
+ }
+ let stream = unrotate(&src[..(need.div_ceil(CHUNK) * CHUNK).min(src.len())]);
+ let mut data = vec![0u16; n];
+ unpack(&stream, bits, &mut data)?;
+
+ let active = match (get(BORDER_TOP), get(BORDER_LEFT), get(BORDER_BOTTOM), get(BORDER_RIGHT)) {
+ (Some(tp), Some(l), Some(b), Some(r)) if b > tp && r > l => Rect::new(l, tp, r - l, b - tp).clipped(w, h),
+ _ => Rect::new(0, 0, w, h),
+ };
+ let active = if active.width == 0 || active.height == 0 { Rect::new(0, 0, w, h) } else { active };
+ let cfa = cfa(ifd0.u16(CFA_PATTERN).unwrap_or(1));
+ let black = match BLACK_RGB.map(|tag| ifd0.f64(tag)) {
+ [Some(r), Some(g), Some(b)] => {
+ // per CFA position, anchored at the active area
+ let a = cfa.shifted(active.x, active.y);
+ let values = a.pattern.iter().map(|&c| [r, g, b][c as usize] as f32).collect();
+ BlackLevel { repeat_rows: 2, repeat_cols: 2, values, ..Default::default() }
+ }
+ _ => BlackLevel::uniform(0.0),
+ };
+ let white = ifd0.f64(LINEARITY_LIMIT).filter(|v| *v > 0.0).map(|v| v as f32).unwrap_or(((1u32 << bits) - 1) as f32);
+ let wb = match WB_RGB.map(|tag| ifd0.f64(tag)) {
+ [Some(r), Some(g), Some(b)] if r > 0.0 && g > 0.0 && b > 0.0 => Some([(r / g) as f32, 1.0, (b / g) as f32]),
+ _ => None,
+ };
+ let mut metadata = lightcraft_meta::from_tiff(&tiff);
+ metadata.width = Some(active.width as u32);
+ metadata.height = Some(active.height as u32);
+ if metadata.iso.is_none() {
+ metadata.iso = ifd0.u32(ISO).filter(|v| *v > 0);
+ }
+ let img = RawImage {
+ format: RawFormat::Rw2,
+ width: w,
+ height: h,
+ cpp: 1,
+ data: RawData::U16(data),
+ cfa: Some(cfa),
+ bits,
+ black,
+ white: vec![white],
+ active_area: active,
+ crop: Rect::new(0, 0, active.width, active.height),
+ orientation: Orientation::from_exif(ifd0.u16(t::ORIENTATION).unwrap_or(1)),
+ color: ColorData::default(),
+ wb_multipliers: wb,
+ linearized: false,
+ opcodes: OpcodeLists::default(),
+ metadata,
+ };
+ img.validate()?;
+ Ok(img)
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use lightcraft_tiff::{ByteOrder, IfdBuilder, TiffWriter, Value};
+
+ /// Inverse of the decoder: pack, then rotate every whole chunk.
+ pub(crate) fn encode(px: &[u16], bits: u32) -> Vec {
+ let (per, start) = layout(bits).unwrap();
+ let mut stream = Vec::new();
+ for blk in px.chunks(per) {
+ let mut word = 0u128;
+ for (i, &v) in blk.iter().enumerate() {
+ word |= (v as u128) << (start + i as u32 * bits);
+ }
+ stream.extend_from_slice(&word.to_le_bytes());
+ }
+ stream.resize(stream.len().div_ceil(CHUNK) * CHUNK, 0);
+ let mut out = Vec::with_capacity(stream.len());
+ for c in stream.chunks(CHUNK) {
+ // logical = file[ROTATE..] ++ file[..ROTATE] => file = logical[CHUNK-ROTATE..] ++ logical[..CHUNK-ROTATE]
+ out.extend_from_slice(&c[CHUNK - ROTATE..]);
+ out.extend_from_slice(&c[..CHUNK - ROTATE]);
+ }
+ out
+ }
+
+ pub(crate) fn rw2(w: u16, h: u16, bits: u16, format: u16, payload: &[u8]) -> Vec {
+ let build = |off: u32| {
+ let mut ifd = IfdBuilder::new();
+ for (tag, v) in [(SENSOR_WIDTH, w), (SENSOR_HEIGHT, h), (BORDER_TOP, 2), (BORDER_LEFT, 4), (BORDER_BOTTOM, h), (BORDER_RIGHT, w)] {
+ ifd.set(tag, Value::Short(vec![v]));
+ }
+ ifd.set(CFA_PATTERN, Value::Short(vec![1]));
+ ifd.set(BITS, Value::Short(vec![bits]));
+ ifd.set(RAW_FORMAT, Value::Short(vec![format]));
+ ifd.set(LINEARITY_LIMIT, Value::Short(vec![(1 << bits) - 1]));
+ for (i, tag) in BLACK_RGB.iter().enumerate() {
+ ifd.set(*tag, Value::Short(vec![100 + i as u16]));
+ }
+ for (tag, v) in WB_RGB.iter().zip([512u16, 256, 384]) {
+ ifd.set(*tag, Value::Short(vec![v]));
+ }
+ ifd.set(t::MAKE, Value::Ascii("Panasonic".into()));
+ ifd.set(RAW_OFFSET, Value::Long(vec![off]));
+ let mut b = TiffWriter::new(ByteOrder::Little, false).write(&[ifd]).unwrap();
+ b[2] = 0x55;
+ b
+ };
+ let head = build(0);
+ let off = head.len().div_ceil(16) * 16;
+ let mut b = build(off as u32);
+ b.resize(off, 0);
+ b.extend_from_slice(payload);
+ b
+ }
+
+ #[test]
+ fn packed_12_and_14_bit() {
+ let (w, h) = (90usize, 200usize); // > one chunk of data
+ for bits in [12u32, 14] {
+ let px: Vec = (0..w * h).map(|i| ((i * 2654435761usize) >> 7) as u16 & ((1 << bits) - 1)).collect();
+ let bytes = rw2(w as u16, h as u16, bits as u16, 5, &encode(&px, bits));
+ assert_eq!(crate::probe(&bytes), Some(RawFormat::Rw2));
+ let r = crate::decode(&bytes).unwrap_or_else(|e| panic!("{bits}: {e}"));
+ assert_eq!(r.data, RawData::U16(px), "{bits}-bit");
+ assert_eq!(r.active_area, Rect::new(4, 2, w - 4, h - 2));
+ assert_eq!(r.cfa.as_ref().unwrap().name(), "RGGB");
+ assert_eq!(r.black.at(0, 0, 0, 1), 100.0);
+ assert_eq!(r.black.at(1, 0, 0, 1), 101.0);
+ assert_eq!(r.black.at(1, 1, 0, 1), 102.0);
+ assert_eq!(r.wb_multipliers, Some([2.0, 1.0, 1.5]));
+ }
+ let bytes = rw2(28, 2, 12, 4, &[0; 64]);
+ assert!(matches!(crate::decode(&bytes), Err(RawError::Unsupported(_))));
+ let bytes = rw2(280, 20, 12, 5, &[0; 64]);
+ assert!(matches!(crate::decode(&bytes), Err(RawError::Corrupt(_))));
+ }
+}
diff --git a/crates/raw/tests/corpus.rs b/crates/raw/tests/corpus.rs
new file mode 100644
index 000000000..e8f055cff
--- /dev/null
+++ b/crates/raw/tests/corpus.rs
@@ -0,0 +1,84 @@
+//! Corpus test over `corpus/raw/**` (git-ignored CC0 samples from raw.pixls.us, fetched with
+//! `cargo xtask corpus --download`; `LIGHTCRAFT_CORPUS` overrides the corpus root). Skips cleanly when absent.
+//!
+//! Every file must be recognised, carry an embedded JPEG preview (DNG: optional), and either decode to a valid image or report
+//! `Unsupported` for one of the variants we know we don't decode yet. Prints decode times
+//! (`cargo test -p lightcraft-raw --release --test corpus -- --nocapture`).
+
+use lightcraft_raw::{RawError, RawFormat, decode, embedded_preview, probe};
+use std::path::{Path, PathBuf};
+use std::time::Instant;
+
+fn corpus_root() -> PathBuf {
+ std::env::var_os("LIGHTCRAFT_CORPUS").map(PathBuf::from).unwrap_or_else(|| Path::new(env!("CARGO_MANIFEST_DIR")).join("../../corpus"))
+}
+
+/// Variants known not to decode yet (see the crate docs): matched against the lower-case file name.
+const KNOWN_UNSUPPORTED: &[&str] = &[
+ "cr3-", // CR3 / CRX (M11.1)
+ "raf-fuji-xt20-compressed", // Fujifilm compressed RAF
+ "rw2-panasonic-gh5.", // Panasonic raw format 4 (quantised)
+ "rw2-panasonic-gx80", // "
+ "rw2-panasonic-g9-b", // "
+ "orf-olympus-em", // Olympus compressed ORF
+ "nef-nikon-d5100-lossless", // Nikon Huffman NEF
+ "nef-nikon-d7000-lossy", // "
+ "sraw", // Canon sRAW / mRAW
+ "-lossy.dng", // lossy (DCT) DNG
+];
+
+#[test]
+fn corpus_raw_decodes() {
+ let dir = corpus_root().join("raw");
+ let Ok(rd) = std::fs::read_dir(&dir) else {
+ eprintln!("skip: {} absent", dir.display());
+ return;
+ };
+ let mut paths: Vec = rd.flatten().map(|e| e.path()).filter(|p| p.is_file()).collect();
+ paths.sort();
+ let (mut ok, mut unsupported) = (0, 0);
+ for p in paths {
+ let name = p.file_name().unwrap().to_string_lossy().to_lowercase();
+ if name.ends_with(".part") || name.ends_with(".txt") || name.ends_with(".md") {
+ continue;
+ }
+ let bytes = std::fs::read(&p).unwrap();
+ let fmt = probe(&bytes).unwrap_or_else(|| panic!("{name}: not recognised"));
+ let t0 = Instant::now();
+ let preview = embedded_preview(&bytes);
+ let tp = t0.elapsed().as_secs_f64() * 1e3;
+ // DNG previews are optional (and some carry only an uncompressed RGB thumbnail); vendor raws always embed a JPEG
+ if let Some(p) = &preview {
+ assert!(p.starts_with(&[0xff, 0xd8]) && p.ends_with(&[0xff, 0xd9]), "{name}: preview is not a JPEG");
+ } else {
+ assert_eq!(fmt, RawFormat::Dng, "{name}: no embedded preview");
+ }
+ let preview_kb = preview.as_ref().map_or(0, |p| p.len() / 1024);
+ let t1 = Instant::now();
+ match decode(&bytes) {
+ Ok(img) => {
+ let dt = t1.elapsed().as_secs_f64() * 1e3;
+ img.validate().unwrap();
+ assert!(img.white_at(0) > img.black.mean(), "{name}: white {} <= black {}", img.white_at(0), img.black.mean());
+ let mp = (img.width * img.height) as f64 / 1e6;
+ eprintln!(
+ "{name:44} {fmt:?} {}x{} {}-bit {:?}: decode {dt:.0} ms ({:.0} MP/s), preview {} KB in {tp:.1} ms",
+ img.width,
+ img.height,
+ img.bits,
+ img.cfa.as_ref().map(|c| c.name()),
+ mp / (dt / 1e3),
+ preview_kb
+ );
+ ok += 1;
+ }
+ Err(RawError::Unsupported(why)) => {
+ assert!(KNOWN_UNSUPPORTED.iter().any(|k| name.contains(k)), "{name}: unexpectedly unsupported: {why}");
+ eprintln!("{name:44} {fmt:?} unsupported ({why}); preview {preview_kb} KB in {tp:.1} ms");
+ unsupported += 1;
+ }
+ Err(e) => panic!("{name}: {e}"),
+ }
+ }
+ eprintln!("corpus/raw: {ok} decoded, {unsupported} known-unsupported (preview only)");
+}
diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs
index 0244b4dfc..42f58fcc9 100644
--- a/crates/ui-egui/src/lib.rs
+++ b/crates/ui-egui/src/lib.rs
@@ -24,6 +24,8 @@ pub use control::{ControlRequest, ControlResponse};
pub use state::UiState;
pub type PickFiles = Box Vec>;
+/// A save dialog: suggested file name → chosen path (`None` = cancelled).
+pub type SaveFile = Box Option>;
pub type WriteFn = Box Result<(), String>>;
pub type PngEncode = Box Vec>;
@@ -32,6 +34,10 @@ pub type PngEncode = Box Vec>;
pub struct Services {
/// Show an open dialog for photos; returns paths.
pub pick_files: Option,
+ /// Open dialog for preset files (`.lcpreset`, `.xmp`).
+ pub pick_preset_files: Option,
+ /// Save dialog for an exported `.lcpreset` file.
+ pub save_preset_file: Option,
pub write: Option,
/// PNG encoder (the host links an image encoder; the UI crate stays codec-free).
pub png: Option,
diff --git a/crates/ui-egui/src/menus.rs b/crates/ui-egui/src/menus.rs
index 5b760cd0e..1dd7101f5 100644
--- a/crates/ui-egui/src/menus.rs
+++ b/crates/ui-egui/src/menus.rs
@@ -56,6 +56,8 @@ pub const UI_COMMANDS: &[UiCommand] = &[
("dialog.copySettings", "Choose Edit Settings to Copy…", Some("Cmd+Shift+C"), "Photo"),
("dialog.export", "Export…", Some("Cmd+Shift+E"), "File"),
("file.addPhotos", "Add Photos…", Some("Cmd+Shift+I"), "File"),
+ ("file.importPresets", "Import Presets…", None, "File"),
+ ("file.exportPresets", "Export Presets…", None, "File"),
("app.about", "About LightCraft", None, "LightCraft"),
("app.shortcuts", "Keyboard Shortcuts", Some("Cmd+/"), "Help"),
("app.export", "Export Now", None, ""),
@@ -297,6 +299,59 @@ pub fn run_ui_command(app: &mut LightcraftApp, id: &str, p: &Value) -> Option {
+ let paths = match p.get("paths").and_then(Value::as_array) {
+ Some(a) => a.iter().filter_map(Value::as_str).map(str::to_string).collect(),
+ None => match app.services.pick_preset_files.as_mut() {
+ Some(f) => f(),
+ None => return Some(Err("no file dialog on this platform".into())),
+ },
+ };
+ if paths.is_empty() {
+ return Some(Ok(Value::Null));
+ }
+ let r = app.session.execute("preset.import", &json!({"paths": paths})).map_err(|e| e.to_string());
+ if let Ok(v) = &r {
+ let n = v["imported"].as_array().map_or(0, Vec::len);
+ let failed = v["failed"].as_array().map_or(0, Vec::len);
+ let msg = match (n, failed) {
+ (0, 0) => "No new presets".to_string(),
+ (n, 0) => format!("Imported {n} preset{}", if n == 1 { "" } else { "s" }),
+ (n, f) => format!("Imported {n} preset{}, {f} file{} not readable", if n == 1 { "" } else { "s" }, if f == 1 { "" } else { "s" }),
+ };
+ app.toast(&ctx, msg);
+ if n > 0 {
+ app.ui.presets = true;
+ }
+ }
+ return Some(r);
+ }
+ "file.exportPresets" => {
+ let group = p.get("group").and_then(Value::as_str).map(str::to_string);
+ let path = match p.get("path").and_then(Value::as_str) {
+ Some(x) => Some(x.to_string()),
+ None => {
+ let name = format!("{}.lcpreset", group.as_deref().unwrap_or("LightCraft Presets"));
+ match app.services.save_preset_file.as_mut() {
+ Some(f) => f(&name),
+ None => return Some(Err("no file dialog on this platform".into())),
+ }
+ }
+ };
+ let Some(path) = path else { return Some(Ok(Value::Null)) };
+ let mut params = json!({"path": path});
+ if let Some(g) = group {
+ params["group"] = json!(g);
+ }
+ if let Some(ids) = p.get("ids") {
+ params["ids"] = ids.clone();
+ }
+ let r = app.session.execute("preset.export", ¶ms).map_err(|e| e.to_string());
+ if let Ok(v) = &r {
+ app.toast(&ctx, format!("Exported {} preset{}", v["count"], if v["count"] == 1 { "" } else { "s" }));
+ }
+ return Some(r);
+ }
"app.export" => crate::control::export_active(app, p),
_ => return None,
};
@@ -307,6 +362,7 @@ pub fn ui_enabled(app: &LightcraftApp, id: &str) -> bool {
match id {
s if s.starts_with("panel.") || s.starts_with("tool.") || s.starts_with("section.") => app.session.active().is_some() || s == "panel.close",
"app.export" | "dialog.export" | "dialog.createPreset" | "dialog.copySettings" => app.session.active().is_some(),
+ "file.exportPresets" => app.session.presets.iter().any(|p| !p.builtin),
_ => true,
}
}
diff --git a/crates/ui-egui/src/panels/presets.rs b/crates/ui-egui/src/panels/presets.rs
index 6d891cc5c..7e1abd36a 100644
--- a/crates/ui-egui/src/panels/presets.rs
+++ b/crates/ui-egui/src/panels/presets.rs
@@ -1,5 +1,5 @@
//! The Presets column (opens to the left of the Edit panel): grouped presets with hover preview,
-//! amount slider, create/favourite.
+//! amount slider, create/favourite, import/export of preset files.
use std::collections::BTreeMap;
@@ -31,6 +31,16 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
{
app.ui.dialog = Some(crate::state::Dialog::CreatePreset { name: String::new(), group: "User Presets".into() });
}
+ let more = icon_button(&mut hdr, "presetMore", Icon::More, vec2(28.0, 28.0), false, true, "More preset options");
+ egui::Popup::menu(&more).show(|ui| {
+ if ui.button("Import Presets…").clicked() {
+ let _ = app.run("file.importPresets", json!({}));
+ }
+ let any_user = app.session.presets.iter().any(|p| !p.builtin);
+ if ui.add_enabled(any_user, egui::Button::new("Export User Presets…")).clicked() {
+ let _ = app.run("file.exportPresets", json!({}));
+ }
+ });
divider(ui);
// amount slider (applies to the last applied preset)
let amt_id = egui::Id::new("preset-amount");
@@ -76,6 +86,11 @@ pub fn show(app: &mut LightcraftApp, ui: &mut egui::Ui) {
if resp.clicked() {
ui.data_mut(|d| d.insert_temp(open_id, !open));
}
+ resp.context_menu(|ui| {
+ if ui.button("Export Group…").clicked() {
+ let _ = app.run("file.exportPresets", json!({"group": g}));
+ }
+ });
if !open {
continue;
}
diff --git a/docs/xmp-interop.md b/docs/xmp-interop.md
new file mode 100644
index 000000000..cfdc6ac3e
--- /dev/null
+++ b/docs/xmp-interop.md
@@ -0,0 +1,93 @@
+# XMP sidecars and interchange
+
+LightCraft keeps its catalog as the source of truth, and can also store each photo's metadata and develop settings in
+an XMP sidecar next to the original. Sidecars let edits travel with the files (backups, another LightCraft library,
+other tools), and let LightCraft pick up edits made elsewhere.
+
+## Sidecar files
+
+| | |
+|---|---|
+| Name | `.xmp` by default (`IMG_0001.CR3` → `IMG_0001.xmp`); `.xmp` (`IMG_0001.CR3.xmp`) with `naming: "full"`. Reading accepts either (preferred first) and `.XMP`. |
+| Write | `photo.saveMetadataToFile {ids?}` (Photo ▸ Save Metadata to File, ⌘S), or automatically after every change with `library.xmpPreferences {autoWrite: true}` (File ▸ Automatically Write Changes into XMP). Slider drags are written once, when the drag ends; undo/redo rewrite the sidecar. Written atomically (temp file + rename). |
+| Read | On import (`library.import`, the report counts `sidecars`), and `photo.readMetadataFromFile {ids?}` (one undo step). For raw/DNG files without a sidecar, the XMP embedded in the file is used. |
+| Preferences | `library.xmpPreferences {autoWrite?, naming?: stem\|full}`, stored in the library's `prefs.json`. |
+
+What we write (standard namespaces, so other tools can read the metadata):
+
+| Field | XMP property |
+|---|---|
+| Rating 0–5 | `xmp:Rating` |
+| Colour label | `xmp:Label` (`Red`, `Yellow`, `Green`, `Blue`, `Purple`) |
+| Title / caption / copyright / creator | `dc:title` / `dc:description` / `dc:rights` / `dc:creator` |
+| Keywords | `dc:subject` |
+| Capture time, GPS | `exif:DateTimeOriginal`, `photoshop:DateCreated`, `exif:GPSLatitude`/`GPSLongitude` |
+| Pick / reject flag | `lc:flag` (`pick`, `reject`, `none`) |
+| Location | `lc:location` |
+| Develop settings | `lc:settings` — our complete `DevelopSettings` as JSON (exact round trip, incl. masks, spots, crop) |
+
+`lc:` is `http://ns.lightcraft.app/lc/1.0/`.
+
+Reading merges into the catalog with the **sidecar winning** for every field it states; fields it doesn't state are
+kept. `xmp:Rating="-1"` (the XMP convention for rejected) sets the reject flag. Develop settings come from
+`lc:settings` when present (exact); otherwise from the `crs:` fields below (approximate).
+
+## Reading `crs:` develop fields
+
+Many raw developers store edits as `crs:` properties (`http://ns.adobe.com/camera-raw-settings/1.0/`) in sidecars, in
+DNG files and in XMP presets. LightCraft reads the common ones and maps them to its own controls. We implemented this
+from the public XMP specification and by observing what each field does; no third-party code or preset files were used.
+Our pipeline renders differently, so **values carry over but the look is approximate**.
+
+We read these fields; we never write them. Only fields in the packet are applied: the result is a partial settings
+object that gets merged like a preset, so everything else keeps its current or default value. Packets marked
+`crs:AlreadyApplied="True"` are skipped, because those pixels already contain the edit. Only process-version 2012+ field
+names are read (e.g. `Exposure2012`, not the older `Exposure`).
+
+| `crs:` field(s) | LightCraft control | Notes |
+|---|---|---|
+| `Exposure2012` | `light.exposure` | EV, 1:1 |
+| `Contrast2012`, `Highlights2012`, `Shadows2012`, `Whites2012`, `Blacks2012` | `light.contrast` … `light.blacks` | −100..100, 1:1 |
+| `WhiteBalance` | `wb.mode` | `As Shot`, `Auto`, `Daylight`, `Cloudy`, `Shade`, `Tungsten`, `Fluorescent`, `Flash`; other names → custom |
+| `Temperature`, `Tint` | `wb.temp`, `wb.tint` | Kelvin / tint for raw files (and presets) |
+| `IncrementalTemperature`, `IncrementalTint` | `wb.temp`, `wb.tint` | rendered files: −100..100 on our relative scale (mired shift around 6500 K, same as the Temp slider) |
+| `Vibrance`, `Saturation` | `color.vibrance`, `color.saturation` | 1:1 |
+| `Texture`, `Clarity2012`, `Dehaze` | `effects.texture`, `effects.clarity`, `effects.dehaze` | 1:1 |
+| `HueAdjustment`, `SaturationAdjustment`, `LuminanceAdjustment` | `mixer..hue/sat/lum` | bands Red, Orange, Yellow, Green, Aqua, Blue, Purple, Magenta |
+| `ConvertToGrayscale` | `treatment` | `True` → B&W |
+| `GrayMixer` | `bw_mix.` | |
+| `ParametricShadows`, `ParametricDarks`, `ParametricLights`, `ParametricHighlights` | `curve.shadows/darks/lights/highlights` | |
+| `ParametricShadowSplit`, `ParametricMidtoneSplit`, `ParametricHighlightSplit` | `curve.split_shadows/split_mid/split_highlights` | |
+| `ToneCurvePV2012`, `ToneCurvePV2012Red/Green/Blue` | `curve.master/red/green/blue` | `"x, y"` points in 0..255 → 0..1; a straight 0→255 line = no curve |
+| `SplitToningShadowHue/Saturation`, `SplitToningHighlightHue/Saturation` | `grading.shadows/highlights.hue/sat` | |
+| `ColorGradeShadowLum`, `ColorGradeHighlightLum` | `grading.shadows/highlights.lum` | |
+| `ColorGradeMidtoneHue/Sat/Lum`, `ColorGradeGlobalHue/Sat/Lum` | `grading.midtones/global.*` | |
+| `ColorGradeBlending`, `SplitToningBalance` | `grading.blending`, `grading.balance` | |
+| `Sharpness`, `SharpenRadius`, `SharpenDetail`, `SharpenEdgeMasking` | `detail.sharpen_*` | |
+| `LuminanceSmoothing`, `LuminanceNoiseReductionDetail`, `LuminanceNoiseReductionContrast` | `detail.nr_luminance/nr_detail/nr_contrast` | |
+| `ColorNoiseReduction`, `ColorNoiseReductionDetail`, `ColorNoiseReductionSmoothness` | `detail.nr_color/nr_color_detail/nr_color_smoothness` | |
+| `PostCropVignetteAmount/Midpoint/Roundness/Feather/HighlightContrast` | `vignette.amount/midpoint/roundness/feather/highlights` | |
+| `PostCropVignetteStyle` | `vignette.style` | 1 highlight priority, 2 colour priority, 3 paint overlay |
+| `GrainAmount`, `GrainSize`, `GrainFrequency` | `grain.amount`, `grain.size`, `grain.roughness` | |
+| `LensProfileEnable`, `AutoLateralCA` | `optics.lens_profile`, `optics.remove_ca` | the switch only; lens profiles are our own |
+| `LensManualDistortionAmount`, `VignetteAmount`, `VignetteMidpoint` | `optics.distortion`, `optics.vignetting`, `optics.vignetting_midpoint` | |
+| `DefringePurple/GreenAmount/HueLo/HueHi` | `optics.defringe_*` | |
+| `PerspectiveVertical/Horizontal/Rotate/Scale/Aspect/X/Y` | `geometry.vertical/horizontal/rotate/scale/aspect/offset_x/offset_y` | |
+| `PerspectiveUpright` | `geometry.upright` | 0 off, 1 auto, 2 level, 3 vertical, 4 full, 5 guided |
+| `HasCrop`, `CropLeft/Top/Right/Bottom`, `CropAngle` | `crop.geometry` | normalized edges → rect; angle in degrees; `HasCrop="False"` → no crop |
+
+Values pass through our control specs, so anything outside our slider ranges gets clamped.
+
+**Not mapped:** camera profiles and looks (`CameraProfile`, `Look`; we have our own profile set), local adjustments
+(masks, gradients, brushes), spot removal, red eye, lens blur, process-version 2010 field names, and AI features.
+
+## Preset files
+
+| | |
+|---|---|
+| Ours: `.lcpreset` | JSON `{"format": "lightcraft.preset", "version": 1, "presets": [{id, name, group, settings}]}`, where `settings` is a partial develop-settings object (only the groups the preset includes). A file can hold one preset or many, and every preset keeps its group. Import also accepts a bare preset object or an array of them. |
+| Export | `preset.export {path, ids?, group?}`: all user presets by default, or the given ids or one group. In the app: File ▸ Export Presets…, Presets panel ▸ ⋯ ▸ Export User Presets…, or right-click a group ▸ Export Group…. |
+| Import | `preset.import {paths}`: files or folders (recursive), `.lcpreset` and `.xmp`. In the app: File ▸ Import Presets…, or Presets panel ▸ ⋯ ▸ Import Presets…. A preset that's already there (same name, group and settings) is skipped. If an id clashes, the import gets a fresh `user.*` id, and built-in presets are never replaced. |
+| XMP presets | Read with the `crs:` table above, with `crs:Name` as the name (falling back to the file name) and `crs:Group` as the group (falling back to "Imported Presets"). Only the fields the preset sets are included, so applying it leaves everything else alone and the Amount slider scales it like any other preset. We only read XMP presets; we don't write them. |
+
+LightCraft ships no third-party presets. Its built-in presets are its own values (`crates/engine/src/presets.rs`).
diff --git a/xtask/src/main.rs b/xtask/src/main.rs
index 0218d4a96..6a537e081 100644
--- a/xtask/src/main.rs
+++ b/xtask/src/main.rs
@@ -23,7 +23,7 @@ commands:
--serve serves it on http://127.0.0.1: (default 8080)
ci fmt --check, clippy -D warnings, test, layers, assets, wasm (stops at first failure)
corpus [--download]
- show where test corpora live; --download fetches PngSuite into corpus/pngsuite
+ show where test corpora live; --download fetches PngSuite and CC0 raw samples (raw.pixls.us) into corpus/
stats [--exact] count tests and lines per crate (--exact: ask the test harness via `-- --list`)
";
@@ -197,7 +197,86 @@ fn cmd_ci() -> Result<(), String> {
}
/// CC0 raw samples (raw.pixls.us). Small, representative set; extend freely (CC0 only).
-const RAW_SAMPLES: &[(&str, &str)] = &[];
+/// CC0 raw samples from raw.pixls.us (each verified CC0 on the site; files matched by their published SHA-256).
+/// One per format / compression variant we decode or deliberately report as unsupported (preview only).
+const RAW_SAMPLES: &[(&str, &str)] = &[
+ (
+ "arw-sony-a7m3-compressed.arw",
+ "https://raw.pixls.us/getfile.php/2414/nice/Sony%20-%20ILCE-7M3%20-%2014bit%2014bit%20compressed%20%283:2%29.ARW",
+ ),
+ (
+ "arw-sony-a7m3-uncompressed.arw",
+ "https://raw.pixls.us/getfile.php/2418/nice/Sony%20-%20ILCE-7M3%20-%2014bit%2014bit%20uncompressed%20%283:2%29.ARW",
+ ),
+ ("arw-sony-a7m4-14bit.arw", "https://raw.pixls.us/getfile.php/6936/nice/Sony%20-%20ILCE-7M4%20-%2014bit%20%283:2%29.ARW"),
+ ("cr2-canon-5d3-sraw2.cr2", "https://raw.pixls.us/getfile.php/773/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%20sRAW2%20%28sRAW%29.CR2"),
+ ("cr2-canon-5d3.cr2", "https://raw.pixls.us/getfile.php/771/nice/Canon%20-%20EOS%205D%20Mark%20III.CR2"),
+ ("cr2-canon-80d.cr2", "https://raw.pixls.us/getfile.php/1294/nice/Canon%20-%20EOS%2080D%20-%20RAW%20%283:2%29.CR2"),
+ ("cr3-canon-m50-craw.cr3", "https://raw.pixls.us/getfile.php/2663/nice/Canon%20-%20EOS%20M50%20-%20CRAW%20%283:2%29.CR3"),
+ (
+ "dng-adobe-canon-5d3-linear-lj92.dng",
+ "https://raw.pixls.us/getfile.php/1032/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%20Lossless%20JPEG%20compression%2C%20rgb%20%283:2%29.DNG",
+ ),
+ (
+ "dng-adobe-canon-5d3-lj92.dng",
+ "https://raw.pixls.us/getfile.php/1024/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%2016bit%2016bit%20Lossless%20JPEG%20compression%20%283:2%29.DNG",
+ ),
+ (
+ "dng-adobe-canon-5d3-lossy.dng",
+ "https://raw.pixls.us/getfile.php/1023/nice/Adobe%20DNG%20Converter%20-%20Canon%20EOS%205D%20Mark%20III%20-%20Lossy%20JPEG%20compression%20%283:2%29.DNG",
+ ),
+ (
+ "dng-canon-5d3-14bit-small.dng",
+ "https://raw.pixls.us/getfile.php/2204/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2014bit%2014bit%20%282.3471882640587%29.dng",
+ ),
+ ("dng-canon-5d3-16bit-169.dng", "https://raw.pixls.us/getfile.php/2649/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2016bit%20%2816:9%29.dng"),
+ ("dng-canon-5d3-16bit.dng", "https://raw.pixls.us/getfile.php/885/nice/Canon%20-%20EOS%205D%20Mark%20III%20-%2016bit%2016bit%20RAW.dng"),
+ ("dng-google-pixel2xl.dng", "https://raw.pixls.us/getfile.php/2206/nice/Google%20-%20Pixel%202%20XL%20-%2016bit%20%284:3%29.dng"),
+ ("dng-ricoh-gr3.dng", "https://raw.pixls.us/getfile.php/3115/nice/Ricoh%20-%20GR%20III%20-%2014bit%20%283:2%29.DNG"),
+ (
+ "nef-nikon-d5100-lossless.nef",
+ "https://raw.pixls.us/getfile.php/1597/nice/Nikon%20-%20D5100%20-%2014bit%2014bit%20compressed%20%28Lossless%29%20%283:2%29.nef",
+ ),
+ (
+ "nef-nikon-d5100-uncompressed.nef",
+ "https://raw.pixls.us/getfile.php/1598/nice/Nikon%20-%20D5100%20-%2014bit%2014bit%20uncompressed%20%283:2%29.nef",
+ ),
+ (
+ "nef-nikon-d7000-lossy12.nef",
+ "https://raw.pixls.us/getfile.php/961/nice/Nikon%20-%20D7000%20-%2012bit%2012bit%20compressed%20%28Lossy%20%28type%202%29%29%20%283:2%29.NEF",
+ ),
+ (
+ "nrw-nikon-b700-uncompressed.nrw",
+ "https://raw.pixls.us/getfile.php/1621/nice/Nikon%20-%20COOLPIX%20B700%20-%2012bit%2012bit%20uncompressed%20%284:3%29.NRW",
+ ),
+ ("orf-olympus-e1.orf", "https://raw.pixls.us/getfile.php/1800/nice/Olympus%20-%20E-1%20-%2016bit%20%284:3%29.ORF"),
+ ("orf-olympus-e400.orf", "https://raw.pixls.us/getfile.php/2151/nice/Olympus%20-%20E-400%20-%2016bit%20%284:3%29.ORF"),
+ ("orf-olympus-em1.orf", "https://raw.pixls.us/getfile.php/1051/nice/Olympus%20-%20E-M1%20-%2016bit%20%284:3%29.orf"),
+ ("orf-olympus-em10iii.orf", "https://raw.pixls.us/getfile.php/1787/nice/Olympus%20-%20E-M10%20Mark%20III%20-%2016bit%20%284:3%29.ORF"),
+ ("orf-olympus-xz2.orf", "https://raw.pixls.us/getfile.php/1432/nice/Olympus%20-%20XZ-2%20-%2012bit%20%284:3%29.orf"),
+ ("pef-pentax-k10d.pef", "https://raw.pixls.us/getfile.php/2239/nice/Pentax%20-%20K10D%20-%2012bit%2012bit%20compressed%20%283:2%29.PEF"),
+ ("pef-pentax-k3.pef", "https://raw.pixls.us/getfile.php/1075/nice/Pentax%20-%20K-3%20-%2014bit%20%283:2%29.PEF"),
+ ("pef-pentax-k5iis.pef", "https://raw.pixls.us/getfile.php/1198/nice/Pentax%20-%20K-5%20II%20s%20-%2014bit%20%283:2%29.PEF"),
+ (
+ "raf-fuji-xa2-12bit-bayer.raf",
+ "https://raw.pixls.us/getfile.php/2883/nice/Fujifilm%20-%20X-A2%20-%2012bit%2012bit%20uncompressed%20%283:2%29.RAF",
+ ),
+ (
+ "raf-fuji-xa5-14bit-bayer.raf",
+ "https://raw.pixls.us/getfile.php/2526/nice/Fujifilm%20-%20X-A5%20-%2014bit%2014bit%20uncompressed%20%283:2%29.RAF",
+ ),
+ ("raf-fuji-xe1-12bit.raf", "https://raw.pixls.us/getfile.php/3098/nice/Fujifilm%20-%20X-E1%20-%2012bit%2012bit%20uncompressed%20%283:2%29.RAF"),
+ ("raf-fuji-xt20-14bit.raf", "https://raw.pixls.us/getfile.php/1177/nice/Fujifilm%20-%20X-T20%20-%2014bit%2014bit%20uncompressed%20%283:2%29.RAF"),
+ (
+ "raf-fuji-xt20-compressed.raf",
+ "https://raw.pixls.us/getfile.php/1178/nice/Fujifilm%20-%20X-T20%20-%2014bit%2014bit%20compressed%20%283:2%29.RAF",
+ ),
+ ("rw2-panasonic-g9-b.rw2", "https://raw.pixls.us/getfile.php/2348/nice/Panasonic%20-%20DC-G9%20-%204:3.RW2"),
+ ("rw2-panasonic-g9.rw2", "https://raw.pixls.us/getfile.php/2585/nice/Panasonic%20-%20DC-G9%20-%204:3.RW2"),
+ ("rw2-panasonic-gh5.rw2", "https://raw.pixls.us/getfile.php/1517/nice/Panasonic%20-%20DC-GH5%20-%204:3.RW2"),
+ ("rw2-panasonic-gh5s.rw2", "https://raw.pixls.us/getfile.php/2603/nice/Panasonic%20-%20DC-GH5S%20-%204:3.RW2"),
+ ("rw2-panasonic-gx80.rw2", "https://raw.pixls.us/getfile.php/1569/nice/Panasonic%20-%20DMC-GX80%20-%204:3.RW2"),
+];
fn cmd_corpus(download: bool) -> Result<(), String> {
let corpus = root().join("corpus");