From 1a301fc2d85a4d19f1d482d9dd62ddd6059dc55c Mon Sep 17 00:00:00 2001
From: cezar <55374787+czarist@users.noreply.github.com>
Date: Wed, 11 Nov 2026 07:02:05 +0000
Subject: [PATCH] LR-EDIT-LENSBLUR: camera-like Lens Blur with AI subject focus
and per-mask bokeh
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Part of the Lens Blur row (`LR-EDIT-LENSBLUR`, ⬜ → 🟡: no depth map yet). Related: #377 (SAM 3 subject selection, used here for the Lens Blur focus).
## Problem
Lens Blur settings existed but nothing rendered them: there was no way to add a shallow depth of field, keep the subject sharp while the rest falls out of focus, or blur only part of a photo through a mask.
## Change
**Edit → Lens Blur** (`section.lensBlur`, ⌘6)
- **Apply**, **Amount**, **Boost** (brighter bokeh highlights) and **Transition** (how gradually the blur grows away from what is in focus, so the subject doesn't look cut out).
- **In Focus**: *Subject* (a mask) or a fixed shape: **Auto / Round / Horizontal / Vertical**, buttons drawn in code that work on the light themes.
- **Move & Resize** (tool `lensBlurShape`) drags the ellipse on the photo and stretches it by its handles. Each drag is one undo step; a shape button puts the ellipse back in the middle.
- **Find Subject** (`lensBlur.findSubject`) runs only when clicked, never automatically on Apply.
- It starts from the focus point the camera recorded, when the file has one (Sony ILCE-1 and ILCE-7SM3, the models `lightcraft_meta` reads today). Otherwise it starts from the subject estimate.
- SAM 3 keeps the prominent people and animals: those at the focus point and those of a similar size. When the focus point falls on neither, it falls back to an object click at that point.
- The result is a normal Object mask ("Lens Blur Subject") that can be refined in Masking.
- It runs on the background segment worker, and on remote SAM 3 too (`Operation::Subject`). While the mask is pending, the UI shows a spinner and the render keeps the subject estimate in focus, so the photo is never blurred whole.
- Without the model, it returns the model-missing error, changes nothing, and the UI offers the SAM 3 download.
- **Brush Refinement**: Focus and Blur brushes painted on the photo (`lensBlur.brushStroke`, `lensBlur.brushReset`), with Size, Feather, Flow, Amount and Auto Mask.
- While Apply is off, In Focus, the shapes, Move & Resize and Find Subject are disabled.
**Masking: a Lens Blur block in every mask** (`section:maskLensBlur`, under Refine Edges)
- **Blur: Selection / Outside**. Outside blurs what the mask leaves out, without inverting the mask's other adjustments.
- **Amount**, **Boost** and **Transition**, per mask. The header's eye switches the block off and keeps its values; the header folds the block.
- Stored as `lens_blur`, `lens_blur_outside`, `lens_blur_boost`, `lens_blur_transition` and `lens_blur_off` on `LocalAdjustments`, set through `mask.adjust`.
**Rendering** (`crates/pipeline/src/lens_blur.rs`)
- Runs in the scene-linear stage after denoise; its settings are part of `pre_nr_key`.
- Disc (bokeh) kernel at four levels on coarse grids, with a normalized convolution weighted by the circle of confusion, so sharp areas don't bleed halos into the blur. Transition uses a chamfer distance to the in-focus area.
- The radius is relative to the long edge, so the blur doesn't depend on resolution. The work stays bounded on large exports: the disc kernel has a fixed size, and each level's coarse grid stays within a cell budget.
- Zoomed windows, both 8-bit and deep (16-bit/float), read around them and are cut back to the request. The grids follow the whole output's origin, and the automatic subject estimate is made once from the whole photo. A window therefore matches the whole render.
- GPU path: read back, apply on the CPU, upload (`crates/gpu/src/render.rs`).
- Hostile values (NaN or infinite sizes, out-of-range areas and focus points, oversized stroke lists) are sanitized or rejected, never a panic.
All new strings are in the 17 locales.
**Not in this PR:**
- Real depth. The falloff is the image distance from the focus, not scene depth; a depth model (e.g. Depth Anything V2 Small on candle) would be a separate PR.
- Rotating the ellipse.
- Aperture shapes.
- Focus points for more camera makes and models.
- Find Subject doesn't resume by itself after the SAM 3 download (the Masking tools do). Like the existing Object / Describe jobs, it shows no row in the activity stack.
## How it was checked
- [x] Full `cargo xtask ci` on Windows 11 x86_64 after the latest merge of `main` (465c8722):
- fmt, clippy, heif, parity, layers, assets and wasm pass;
- every Lens Blur test passes;
- the only failures are five tests outside this change that fail on Windows on `main` too: `export::tests::sixteen_bit_zip_tiff_is_never_larger_than_uncompressed`, the two GPU out-of-memory tests in `gpu::ctx`, `dialog_role::settings_is_a_dialog_too` (needs craft-fonts) and `folder_drop::dont_ask_again_moves_the_next_drop_at_once`.
- [x] Tests that fail without the change:
- **pipeline**:
- the `lens_blur` tests: shapes and moved or hostile areas, blur growth away from the focus, the fallback while an AI mask is pending, brushes, masks blurring their selection or outside with Transition, hostile input;
- `large_image_blur_work_is_bounded`: on a 60 MP export with the strongest global and per-mask settings, the disc kernel stays a fixed size and every level's coarse grid stays within its cell budget, including an offset render window;
- window/full-render parity at U8 and U16, including the automatic subject.
- **engine**: brushes with undo; Find Subject without the model changes nothing; deleting the focus mask clears the focus; a partial effect preset sets `lens_blur`; with the `sam` feature, Find Subject fills the waiting mask.
- **segment**: `prominent_subjects`, and remote `Subject` validation.
- **catalog**: `op_variants_are_versioned` with the new format version.
- **ui-egui**: the brushes painting on the photo, the shape buttons, the controls disabled while Apply is off, Move & Resize with undo and redo, the mask block (side, sliders, eye, folding).
- [x] User-visible change: ran the app and looked at the result (dark and light themes).
## Screenshots
**Before / after**
**Edit → Lens Blur** and **the mask's Lens Blur block**
## Merging
- **Overlap:** #1713 (Clearfall; a draft since 2026-10-14 that conflicts with main) implements the same row on the CPU. It has aperture shapes and XMP roundtrip, which this PR lacks. This PR adds the per-mask block, SAM 3 Find Subject from the camera's focus point, the GPU path and window parity. Only one should land; happy to port what #1713 has that this lacks.
- **Existing edits and libraries:** unchanged. Lens Blur is off by default and every new field is omitted from the JSON at its default, so existing settings keep their JSON and hashes.
- **Shared numbers taken:** catalog format `VERSION` 12 (`crates/catalog/src/journal.rs`). main already uses 10 (`Mask.on_photo`) and 11 (`EmbeddedLens.source`), so Lens Blur takes one past them. If another pull request lands with 12 first, I'll take the next one when I merge main. No render-cache, look or process version change.
- **Where it came from:** own implementation; nothing derived from GPL code or Adobe data. The camera focus point comes from the existing `lightcraft_meta` reader; subject selection uses SAM 3 through the existing segment crate.
- **New dependencies, `unsafe`, build scripts, network access, spawned processes, deleted files, CI or `xtask` gate changes:** none. The remote SAM 3 protocol gains one request (`Operation::Subject`). An older worker rejects it with an error, as `docs/ai-masks.md` notes.
## Checklist
- [x] No new panic path in non-test code
- [x] New text on screen goes through `tr()` with keys in all `locales/*.json`; ⌘6 opens the new section (KEY-SECTIONS)
- [x] `docs/parity-checklist.md` rows updated (LR-EDIT-LENSBLUR 🟡, LR-MASK-SLIDERS, KEY-SECTIONS); `docs/gaps.md` has no Lens Blur entry
- [x] New code has a caller
- [x] No new assets (the shape icons are drawn in code)
- [x] Ready to merge
## Summary by CodeRabbit
* **New Features**
* Added Lens Blur with adjustable strength, highlight boost, and transition controls, plus Round, Wide, and Tall focus shapes.
* Keep a subject or selected mask in focus, find people or animals automatically, and refine focus or blur with brushes. Move and resize the focus area directly on the photo.
* Apply Lens Blur to individual masks, including selections or areas outside them.
* **Bug Fixes**
* Improved Lens Blur rendering for cropped views and preserved focus-mask assignments correctly when masks are edited.
* **Documentation**
* Updated feature and control guides, with Lens Blur interface translations added across multiple languages.
---
crates/catalog/src/journal.rs | 3 +-
crates/catalog/src/tests_format_version.rs | 6 +-
crates/develop/src/controls.rs | 5 +
crates/develop/src/lib.rs | 8 +
crates/develop/src/settings.rs | 170 ++-
crates/engine/src/cmd/develop.rs | 147 +++
crates/engine/src/cmd/mask_effects.rs | 22 +-
crates/engine/src/cmd/masks.rs | 11 +-
crates/engine/src/segment.rs | 118 ++
crates/engine/src/segment/worker.rs | 12 +
crates/engine/src/tests.rs | 78 ++
crates/gpu/src/render.rs | 9 +-
crates/pipeline/src/lens_blur.rs | 1109 +++++++++++++++++
crates/pipeline/src/lib.rs | 15 +-
crates/pipeline/src/masks.rs | 6 +
crates/pipeline/src/output.rs | 62 +-
crates/pipeline/src/tests_window.rs | 69 +-
crates/segment/src/detector.rs | 107 ++
crates/segment/src/lib.rs | 13 +
crates/segment/src/remote.rs | 11 +-
crates/segment/tests/it/remote.rs | 6 +
crates/ui-egui/locales/cs.json | 28 +
crates/ui-egui/locales/de.json | 28 +
crates/ui-egui/locales/es.json | 28 +
crates/ui-egui/locales/fi.json | 28 +
crates/ui-egui/locales/fr.json | 28 +
crates/ui-egui/locales/id.json | 28 +
crates/ui-egui/locales/it.json | 28 +
crates/ui-egui/locales/ja.json | 28 +
crates/ui-egui/locales/ka.json | 28 +
crates/ui-egui/locales/pl.json | 29 +
crates/ui-egui/locales/pt-br.json | 28 +
crates/ui-egui/locales/ru.json | 28 +
crates/ui-egui/locales/sv.json | 29 +
crates/ui-egui/locales/uk.json | 28 +
crates/ui-egui/locales/vi.json | 28 +
crates/ui-egui/locales/zh-hans.json | 28 +
crates/ui-egui/locales/zh-hant.json | 28 +
crates/ui-egui/src/menus/ui_commands.rs | 1 +
crates/ui-egui/src/panels/detail.rs | 10 +
crates/ui-egui/src/panels/detail/lens_blur.rs | 148 +++
crates/ui-egui/src/panels/detail/masks.rs | 2 +-
crates/ui-egui/src/panels/edit.rs | 6 +
crates/ui-egui/src/panels/edit/lens_blur.rs | 257 ++++
crates/ui-egui/src/panels/masking.rs | 57 +-
crates/ui-egui/src/state.rs | 7 +-
crates/ui-egui/src/tests/masking.rs | 154 +++
docs/ai-masks.md | 17 +-
docs/parity-checklist.md | 13 +-
docs/target-app-parity.md | 3 +-
50 files changed, 3090 insertions(+), 50 deletions(-)
create mode 100644 crates/pipeline/src/lens_blur.rs
create mode 100644 crates/ui-egui/src/panels/detail/lens_blur.rs
create mode 100644 crates/ui-egui/src/panels/edit/lens_blur.rs
diff --git a/crates/catalog/src/journal.rs b/crates/catalog/src/journal.rs
index 75ec07e59f..fc93ea9f13 100644
--- a/crates/catalog/src/journal.rs
+++ b/crates/catalog/src/journal.rs
@@ -64,6 +64,7 @@
//! | 10 | masks stay on the photo under lens corrections, Upright and Transform | `Mask.on_photo` (develop settings) |
//! | 11 | lens database profiles named in the Optics panel | `EmbeddedLens.source` |
//! | 12 | history steps and versions share segmentation grids (#2154) | `Catalog.grids`, `SegMask.grid` |
+//! | 13 | Lens Blur | `DevelopSettings.lens_blur`, `Mask.adjust.lens_blur` |
//!
//! Rules:
//! - **Bump [`VERSION`]** (and add a row above) in the change that adds an [`Op`] variant or a
@@ -88,7 +89,7 @@ pub const LOG: &str = "catalog.log";
const FORMAT: &str = "lightcraft-catalog";
/// The catalog format this build writes (and the newest it reads). See the module docs →
/// *Format versions*; bump it whenever an [`Op`] variant or a serialized field is added.
-pub const VERSION: u32 = 12;
+pub const VERSION: u32 = 13;
/// When [`Journal::wants_snapshot`] says it's time to compact the log.
#[derive(Clone, Copy, Debug)]
diff --git a/crates/catalog/src/tests_format_version.rs b/crates/catalog/src/tests_format_version.rs
index 612ee4586b..248933a4a2 100644
--- a/crates/catalog/src/tests_format_version.rs
+++ b/crates/catalog/src/tests_format_version.rs
@@ -238,9 +238,9 @@ fn op_variants_are_versioned() {
Op::SetHistoryCursor { .. } => 8,
}
}
- /// v12: `Catalog.grids` and `SegMask.grid`, shared by a snapshot's masks (#2154) (v11: `EmbeddedLens.source`; v10: `Mask.on_photo`; v9:
- /// `Photo.preview_checked` and `Op::SetContent.preview_checked`).
- const NEWEST_FIELDS: u32 = 12;
+ /// v13: Lens Blur in develop settings and mask adjustments (v12: `Catalog.grids` and `SegMask.grid`; v11: `EmbeddedLens.source`;
+ /// v10: `Mask.on_photo`; v9: `Photo.preview_checked` and `Op::SetContent.preview_checked`).
+ const NEWEST_FIELDS: u32 = 13;
let newest = since(&Op::SetHistoryCursor { id: PhotoId(1), cursor: None });
assert_eq!(newest.max(NEWEST_FIELDS), VERSION, "the newest op's or fields' version must be the current format version");
}
diff --git a/crates/develop/src/controls.rs b/crates/develop/src/controls.rs
index 9d384f57c4..589bcc6f19 100644
--- a/crates/develop/src/controls.rs
+++ b/crates/develop/src/controls.rs
@@ -28,6 +28,7 @@ pub enum Section {
Calibration,
PointColor,
RedEye,
+ LensBlur,
}
impl Section {
@@ -49,6 +50,7 @@ impl Section {
Section::Calibration => "Calibration",
Section::PointColor => "Point Color",
Section::RedEye => "Red Eye",
+ Section::LensBlur => "Lens Blur",
}
}
}
@@ -286,6 +288,9 @@ controls! {
"geometry.scale" => geometry.scale, "Scale", Geometry, 50, 150, 100, 1, 0, Plain;
"geometry.offsetX" => geometry.offset_x, "Offset X", Geometry, -100, 100, 0, 0.1, 1, Centered;
"geometry.offsetY" => geometry.offset_y, "Offset Y", Geometry, -100, 100, 0, 0.1, 1, Centered;
+ "lensBlur.amount" => lens_blur.amount, "Blur Amount", LensBlur, 0, 100, 0, 1, 0, Plain;
+ "lensBlur.boost" => lens_blur.boost, "Boost", LensBlur, 0, 100, 0, 1, 0, Plain;
+ "lensBlur.transition" => lens_blur.transition, "Transition", LensBlur, 0, 100, 50, 1, 0, Plain;
"calibration.shadowsTint" => calibration.shadows_tint, "Shadows Tint", Calibration, -100, 100, 0, 1, 0, Tint;
"calibration.redHue" => calibration.red_hue, "Red Hue", Calibration, -100, 100, 0, 1, 0, Gradient { from: "#e0306a", to: "#e08a30" };
"calibration.redSat" => calibration.red_sat, "Red Saturation", Calibration, -100, 100, 0, 1, 0, Gradient { from: "#7a7a7a", to: "#e03a3a" };
diff --git a/crates/develop/src/lib.rs b/crates/develop/src/lib.rs
index cb5acab6f5..0e90df716a 100644
--- a/crates/develop/src/lib.rs
+++ b/crates/develop/src/lib.rs
@@ -200,6 +200,14 @@ impl DevelopSettings {
}
}
+ /// The middle of the crop, in normalized photo coordinates (the photo's middle when the crop
+ /// isn't a usable rectangle).
+ pub fn crop_center(&self) -> lightcraft_geom::Point {
+ let r = self.crop.geometry.rect;
+ let (x, y) = ((r.x0 + r.x1) / 2.0, (r.y0 + r.y1) / 2.0);
+ if x.is_finite() && y.is_finite() { lightcraft_geom::Point::new(x, y) } else { lightcraft_geom::Point::new(0.5, 0.5) }
+ }
+
/// Next free mask id: one past the highest, or, when that is `u32::MAX` (ids come from files,
/// pastes and agents), the lowest one no mask uses.
pub fn next_mask_id(&self) -> u32 {
diff --git a/crates/develop/src/settings.rs b/crates/develop/src/settings.rs
index 7b0226d9e7..a4e6eab147 100644
--- a/crates/develop/src/settings.rs
+++ b/crates/develop/src/settings.rs
@@ -898,6 +898,25 @@ pub struct LocalAdjustments {
pub noise: f64,
pub moire: f64,
pub defringe: f64,
+ /// Lens Blur inside the mask, 0..100 (`lightcraft_pipeline::lens_blur`): disc (bokeh) blur
+ /// whose radius follows the mask's strength. Omitted while 0, so masks without it keep their
+ /// JSON (and hash).
+ #[serde(skip_serializing_if = "is_zero")]
+ pub lens_blur: f64,
+ /// The mask's Lens Blur block switched off (its values kept).
+ #[serde(skip_serializing_if = "std::ops::Not::not")]
+ pub lens_blur_off: bool,
+ /// Blur what the mask leaves out instead of what it selects (the mask's other adjustments
+ /// still go where it selects).
+ #[serde(skip_serializing_if = "std::ops::Not::not")]
+ pub lens_blur_outside: bool,
+ /// Brighter bokeh highlights in the mask's blur, 0..100.
+ #[serde(skip_serializing_if = "is_zero")]
+ pub lens_blur_boost: f64,
+ /// How gradually the mask's blur grows away from what stays sharp, 0..100 (0: it follows the
+ /// mask's own edge).
+ #[serde(skip_serializing_if = "is_zero")]
+ pub lens_blur_transition: f64,
/// Colour overlay (hue degrees, saturation 0..100); sat 0 = none.
pub color_hue: f64,
pub color_sat: f64,
@@ -925,6 +944,11 @@ impl Default for LocalAdjustments {
noise: 0.0,
moire: 0.0,
defringe: 0.0,
+ lens_blur: 0.0,
+ lens_blur_off: false,
+ lens_blur_outside: false,
+ lens_blur_boost: 0.0,
+ lens_blur_transition: 0.0,
color_hue: 0.0,
color_sat: 0.0,
amount: 100.0,
@@ -1266,13 +1290,157 @@ impl Default for RedEye {
}
}
-#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize)]
+/// Lens Blur: a shallow depth of field added after the fact (`lightcraft_pipeline::lens_blur`).
+/// What stays sharp is the photo's subject, or the selection of the mask [`LensBlur::focus_mask`]
+/// names; everything else gets disc-shaped (bokeh) blur up to [`LensBlur::amount`].
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(default)]
pub struct LensBlur {
pub enabled: bool,
+ /// Blur of what is out of focus, 0..100 (100: a disc of 3.5 % of the long edge).
pub amount: f64,
+ /// Reserved for a depth map (not rendered yet).
pub focal_distance: f64,
+ /// Reserved for a depth map (not rendered yet).
pub focal_range: f64,
+ /// How much brighter the out-of-focus highlights turn into bokeh discs, 0..100. Omitted while
+ /// 0, so settings without it keep their JSON (and hash).
+ #[serde(skip_serializing_if = "is_zero")]
+ pub boost: f64,
+ /// The id of the mask whose selection stays in focus (any mask, shown or hidden); `None`, or a
+ /// mask that no longer exists: the photo's subject.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub focus_mask: Option,
+ /// Brush Refinement, Focus brush: strokes painted in focus (their strength: how far; erase
+ /// strokes take it back). Omitted while empty.
+ #[serde(skip_serializing_if = "Vec::is_empty")]
+ pub focus_strokes: Vec,
+ /// Brush Refinement, Blur brush: strokes painted out of focus, over the focus and the Focus
+ /// brush. Omitted while empty.
+ #[serde(skip_serializing_if = "Vec::is_empty")]
+ pub blur_strokes: Vec,
+ /// How gradually the blur grows away from what is in focus, 0..100: 0 a sharp cut at the
+ /// focus's edge, higher a longer ramp (a stand-in for depth until there is a depth map).
+ /// Omitted at its default, so settings without it keep their JSON (and hash).
+ #[serde(skip_serializing_if = "LensBlur::is_default_transition")]
+ pub transition: f64,
+ /// What stays in focus without a focus mask: the subject estimate, or a fixed shape in the
+ /// middle of the framing. Omitted while automatic.
+ #[serde(skip_serializing_if = "FocusShape::is_auto")]
+ pub shape: FocusShape,
+ /// Where a fixed shape sits and how large it is, once moved or resized on the photo; `None`:
+ /// the shape's preset, in the middle of the crop.
+ #[serde(skip_serializing_if = "Option::is_none")]
+ pub area: Option,
+}
+
+/// A fixed focus shape on the photo: an upright ellipse.
+#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)]
+pub struct FocusArea {
+ /// Normalized (uncropped, oriented) photo coordinates, as mask shapes.
+ pub center: Point,
+ /// Half-width and half-height, in units of the photo's long edge.
+ pub rx: f64,
+ pub ry: f64,
+}
+
+impl FocusArea {
+ /// Smallest and largest radius (long-edge units).
+ pub const MIN_RADIUS: f64 = 0.02;
+ pub const MAX_RADIUS: f64 = 1.0;
+
+ /// Usable values: finite, the radii within range, the centre no farther than a photo's width
+ /// outside it. A hostile value falls back to the middle and the default size.
+ pub fn sanitized(self) -> FocusArea {
+ let r = |v: f64, d: f64| if v.is_finite() { v.clamp(Self::MIN_RADIUS, Self::MAX_RADIUS) } else { d };
+ let c = |v: f64| if v.is_finite() { v.clamp(-1.0, 2.0) } else { 0.5 };
+ FocusArea { center: Point::new(c(self.center.x), c(self.center.y)), rx: r(self.rx, 0.22), ry: r(self.ry, 0.22) }
+ }
+}
+
+impl LensBlur {
+ /// The fixed focus shape in effect, if any: the moved / resized [`LensBlur::area`], else the
+ /// shape's preset centred on `crop_center`. `None` for [`FocusShape::Auto`].
+ pub fn focus_area(&self, crop_center: Point) -> Option {
+ let (rx, ry) = self.shape.radii()?;
+ Some(self.area.unwrap_or(FocusArea { center: crop_center, rx, ry }).sanitized())
+ }
+}
+
+/// The focus area [`LensBlur`] uses when no mask says what is in focus.
+#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "lowercase")]
+pub enum FocusShape {
+ /// A circle.
+ Round,
+ /// An oval lying down (landscape).
+ Wide,
+ /// An oval standing up (portrait).
+ Tall,
+ /// The photo's subject, estimated (person mattes, else a saliency heuristic). Also what a
+ /// shape this build doesn't know reads as (so it is last: serde's `other`).
+ #[default]
+ #[serde(other)]
+ Auto,
+}
+
+impl FocusShape {
+ pub const ALL: [FocusShape; 4] = [FocusShape::Auto, FocusShape::Round, FocusShape::Wide, FocusShape::Tall];
+
+ /// Whether this is the default (Auto: no fixed shape), for skipping it in the JSON.
+ fn is_auto(&self) -> bool {
+ *self == FocusShape::Auto
+ }
+
+ /// The fixed shape's half-width and half-height, in units of the photo's long edge; `None`
+ /// for [`FocusShape::Auto`].
+ pub fn radii(self) -> Option<(f64, f64)> {
+ match self {
+ FocusShape::Auto => None,
+ FocusShape::Round => Some((0.22, 0.22)),
+ FocusShape::Wide => Some((0.32, 0.17)),
+ FocusShape::Tall => Some((0.17, 0.32)),
+ }
+ }
+
+ /// The name shown on the shape's button (translated by the UI).
+ pub fn label(self) -> &'static str {
+ match self {
+ FocusShape::Auto => "Auto",
+ FocusShape::Round => "Round",
+ FocusShape::Wide => "Horizontal",
+ FocusShape::Tall => "Vertical",
+ }
+ }
+}
+
+impl LensBlur {
+ /// [`LensBlur::transition`]'s default.
+ pub const DEFAULT_TRANSITION: f64 = 50.0;
+
+ /// Whether `v` is [`Self::DEFAULT_TRANSITION`], for skipping it in the JSON.
+ fn is_default_transition(v: &f64) -> bool {
+ *v == Self::DEFAULT_TRANSITION
+ }
+}
+
+impl Default for LensBlur {
+ /// Off, with the default Transition and everything else empty.
+ fn default() -> Self {
+ Self {
+ enabled: false,
+ amount: 0.0,
+ focal_distance: 0.0,
+ focal_range: 0.0,
+ boost: 0.0,
+ focus_mask: None,
+ focus_strokes: Vec::new(),
+ blur_strokes: Vec::new(),
+ transition: Self::DEFAULT_TRANSITION,
+ shape: FocusShape::Auto,
+ area: None,
+ }
+ }
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize)]
diff --git a/crates/engine/src/cmd/develop.rs b/crates/engine/src/cmd/develop.rs
index 5403ad6eb2..5223710ed6 100644
--- a/crates/engine/src/cmd/develop.rs
+++ b/crates/engine/src/cmd/develop.rs
@@ -8,6 +8,9 @@ use serde_json::{Value, json};
use super::{CommandSpec, always, bad, bool_or, cmd, f64_or, f64_req, has_active, has_clipboard, has_selection, index_req, ok, str_param};
use crate::{Result, Session, media::SourceLevel};
+/// Strokes one Lens Blur refinement brush keeps (a document can't grow without bound).
+const MAX_LENS_BLUR_STROKES: usize = 2000;
+
fn active(s: &Session, c: &str) -> Result {
s.active().ok_or_else(|| bad(c, "no active photo"))
}
@@ -482,6 +485,150 @@ pub fn specs() -> Vec {
Ok(())
})
}),
+ cmd!(
+ "lensBlur.brushStroke",
+ "Lens Blur Brush Stroke",
+ [],
+ None,
+ "{mode: \"focus\"|\"blur\", points: [[x,y],…] normalized, size?: fraction of long edge (0.03), feather?: 0..100, flow?: 1..100, amount?: 1..100 (how far the stroke refines), erase?: bool, autoMask?: bool, pressure?: [p,…] 0..1 per point} — Lens Blur Brush Refinement: Focus strokes stay sharp, Blur strokes are blurred",
+ has_active,
+ |s, p| {
+ const C: &str = "lensBlur.brushStroke";
+ let focus = match str_param(p, "mode") {
+ Some("focus") => true,
+ Some("blur") => false,
+ _ => return Err(bad(C, "`mode` must be \"focus\" or \"blur\"")),
+ };
+ let points: Vec = p
+ .get("points")
+ .and_then(Value::as_array)
+ .ok_or_else(|| bad(C, "missing points"))?
+ .iter()
+ .filter_map(|q| Some(Point::new(q.get(0)?.as_f64()?, q.get(1)?.as_f64()?)))
+ .filter(|q| q.x.is_finite() && q.y.is_finite())
+ .collect();
+ if points.is_empty() {
+ return Err(bad(C, "a stroke needs at least one point"));
+ }
+ let d = lightcraft_develop::BrushStroke::default();
+ let num = |key: &str, def: f64, lo: f64, hi: f64| {
+ let v = f64_or(p, key, def);
+ if v.is_finite() { v.clamp(lo, hi) } else { def }
+ };
+ let stroke = lightcraft_develop::BrushStroke {
+ pressure: super::masks::pressure_list(p.get("pressure"), points.len())?,
+ points,
+ size: num("size", d.size, 0.001, 0.5),
+ feather: num("feather", d.feather, 0.0, 100.0),
+ flow: num("flow", d.flow, 1.0, 100.0),
+ density: num("amount", d.density, 1.0, 100.0),
+ erase: bool_or(p, "erase", false),
+ auto_mask: bool_or(p, "autoMask", false),
+ };
+ edit(s, C, if focus { "Lens Blur Focus Brush" } else { "Lens Blur Blur Brush" }, |d| {
+ let strokes = if focus { &mut d.lens_blur.focus_strokes } else { &mut d.lens_blur.blur_strokes };
+ if strokes.len() >= MAX_LENS_BLUR_STROKES {
+ return Err(bad(C, format!("at most {MAX_LENS_BLUR_STROKES} strokes per brush: Reset the brush first")));
+ }
+ strokes.push(stroke);
+ Ok(())
+ })
+ }
+ ),
+ cmd!(
+ "lensBlur.findSubject",
+ "Find Subject",
+ [],
+ None,
+ "{} — selects the photo's subject with an Object mask (SAM 3) clicked where the camera focused, when the file records it, else at the likeliest subject; Lens Blur is switched on and keeps that mask in focus → {mask, point: [x,y], from: \"camera\"|\"photo\", pending?}",
+ has_active,
+ |s, _| {
+ const C: &str = "lensBlur.findSubject";
+ let id = active(s, C)?;
+ let d = s.develop_of(id).unwrap_or_default();
+ let camera = s.catalog.photo(id).and_then(|p| p.meta.focus.clone());
+ let (point, from) = match camera {
+ // stored on the upright photo: the user's Rotate / Flip on top, as masks are
+ Some(f) => {
+ let r = d.orientation.map_norm_rect(f.rect);
+ (Point::new((r.x0 + r.x1) / 2.0, (r.y0 + r.y1) / 2.0), "camera")
+ }
+ None => {
+ let src = s.source_now(id, SourceLevel::Thumb).map_err(|e| bad(C, e))?;
+ let info = s.source_info(id);
+ let p = lightcraft_pipeline::lens_blur::subject_point(&src, &d, info.mattes.as_deref())
+ .ok_or_else(|| bad(C, "the photo has no picture to look for a subject in"))?;
+ (p, "photo")
+ }
+ };
+ if !(point.x.is_finite() && point.y.is_finite()) {
+ return Err(bad(C, "the camera's focus point is not usable"));
+ }
+ let point = Point::new(point.x.clamp(0.0, 1.0), point.y.clamp(0.0, 1.0));
+ // an Object waiting for the subject (the photo is analyzed meanwhile); the
+ // segmenter fills it with the people / animals the focus falls on, or the object
+ // at the point
+ let mut out = s.execute("mask.add", &json!({"kind": "object", "name": "Lens Blur Subject"}))?;
+ let mask = out
+ .get("activeMask")
+ .and_then(Value::as_u64)
+ .and_then(|v| u32::try_from(v).ok())
+ .ok_or_else(|| bad(C, "the subject mask was not created"))?;
+ let focus = (from == "camera").then_some(point);
+ if let Err(e) = s.segment_subject_later(id, mask, 0, point, focus) {
+ // nothing half-made: the empty mask goes again
+ let _ = s.execute("edit.undo", &json!({}));
+ return Err(crate::EngineError::Other(e));
+ }
+ out["pending"] = json!(s.segmenter.background);
+ edit(s, C, "Lens Blur Subject", |d| {
+ // a mask that only says what is in focus: no adjustments of its own
+ if let Some(m) = d.masks.iter_mut().find(|m| m.id == mask) {
+ m.adjust = lightcraft_develop::LocalAdjustments::default();
+ }
+ let b = &mut d.lens_blur;
+ b.enabled = true;
+ if b.amount <= 0.0 || !b.amount.is_finite() {
+ b.amount = 50.0;
+ }
+ b.focus_mask = Some(mask);
+ d.set_section_enabled("lensBlur", true);
+ Ok(())
+ })?;
+ if let Some(o) = out.as_object_mut() {
+ o.insert("mask".into(), json!(mask));
+ o.insert("point".into(), json!([point.x, point.y]));
+ o.insert("from".into(), json!(from));
+ }
+ Ok(out)
+ }
+ ),
+ cmd!(
+ "lensBlur.brushReset",
+ "Reset Lens Blur Brush",
+ [],
+ None,
+ "{mode?: \"focus\"|\"blur\"} — removes that brush's strokes (both brushes without `mode`)",
+ has_active,
+ |s, p| {
+ const C: &str = "lensBlur.brushReset";
+ let (focus, blur) = match str_param(p, "mode") {
+ None => (true, true),
+ Some("focus") => (true, false),
+ Some("blur") => (false, true),
+ Some(_) => return Err(bad(C, "`mode` must be \"focus\" or \"blur\"")),
+ };
+ edit(s, C, "Reset Lens Blur Brush", |d| {
+ if focus {
+ d.lens_blur.focus_strokes.clear();
+ }
+ if blur {
+ d.lens_blur.blur_strokes.clear();
+ }
+ Ok(())
+ })
+ }
+ ),
cmd!("develop.sectionEnabled", "Toggle Section", [], None, "{section: string, enabled: bool}", has_active, |s, p| {
let sec = str_param(p, "section").ok_or_else(|| bad("develop.sectionEnabled", "missing section"))?.to_string();
let on = bool_or(p, "enabled", true);
diff --git a/crates/engine/src/cmd/mask_effects.rs b/crates/engine/src/cmd/mask_effects.rs
index d791b42606..529566f3bb 100644
--- a/crates/engine/src/cmd/mask_effects.rs
+++ b/crates/engine/src/cmd/mask_effects.rs
@@ -33,6 +33,7 @@ pub const FIELDS: &[&str] = &[
"noise",
"moire",
"defringe",
+ "lens_blur",
"color_hue",
"color_sat",
];
@@ -118,10 +119,11 @@ pub fn get(a: &LocalAdjustments, k: &str) -> f64 {
serde_json::to_value(a).ok().and_then(|v| v.get(k).and_then(Value::as_f64)).unwrap_or(0.0)
}
-/// `a` with field `k` set to `v` (unchanged for an unknown name).
+/// `a` with field `k` set to `v` (unchanged for an unknown name). A [`FIELDS`] slider counts as
+/// known also while its JSON leaves it out at 0 (`lens_blur`).
pub fn set(a: &LocalAdjustments, k: &str, v: f64) -> LocalAdjustments {
let Ok(Value::Object(mut o)) = serde_json::to_value(a) else { return *a };
- if !o.contains_key(k) || !v.is_finite() {
+ if !(o.contains_key(k) || FIELDS.contains(&k)) || !v.is_finite() {
return *a;
}
o.insert(k.into(), json!(v));
@@ -707,6 +709,22 @@ mod tests {
assert!(!e.matches(&LocalAdjustments { amount: 100.0, ..out }), "Amount counts when carried");
}
+ /// A partial effect sets Lens Blur on a mask without any yet (its JSON leaves `lens_blur` out
+ /// at 0); a name no slider has still changes nothing.
+ #[test]
+ fn partial_effects_set_lens_blur_from_zero() {
+ let e = MaskEffect {
+ name: "Bokeh".into(),
+ adjust: LocalAdjustments { lens_blur: 60.0, ..Default::default() },
+ fields: vec!["lens_blur".into()],
+ ..Default::default()
+ };
+ let out = e.apply(&LocalAdjustments::default());
+ assert_eq!(out.lens_blur, 60.0);
+ assert!(e.matches(&out));
+ assert_eq!(set(&LocalAdjustments::default(), "bogus", 5.0), LocalAdjustments::default());
+ }
+
#[test]
fn hue_without_saturation_still_matches() {
let e = MaskEffect {
diff --git a/crates/engine/src/cmd/masks.rs b/crates/engine/src/cmd/masks.rs
index 06168989a6..e4017f4003 100644
--- a/crates/engine/src/cmd/masks.rs
+++ b/crates/engine/src/cmd/masks.rs
@@ -162,7 +162,11 @@ fn masks_edit(s: &mut Session, c: &str, label: &str, f: impl FnOnce(&mut Vec Vec {
"Set Mask Adjustments",
[],
None,
- "{id?, values: {exposure, contrast, highlights, shadows, whites, blacks, temp, tint, texture, clarity, dehaze, hue, saturation, sharpness, noise, moire, defringe, color_hue, color_sat, amount}}",
+ "{id?, values: {exposure, contrast, highlights, shadows, whites, blacks, temp, tint, texture, clarity, dehaze, hue, saturation, sharpness, noise, moire, defringe, lens_blur, lens_blur_off, lens_blur_outside, lens_blur_boost, lens_blur_transition, color_hue, color_sat, amount}}",
has_active,
|s, p| {
let vals = p.get("values").cloned().ok_or_else(|| bad("mask.adjust", "missing values"))?;
@@ -1151,6 +1155,9 @@ pub(crate) fn clamp_local(mut a: LocalAdjustments) -> LocalAdjustments {
] {
c(v, -100.0, 100.0);
}
+ c(&mut a.lens_blur, 0.0, 100.0);
+ c(&mut a.lens_blur_boost, 0.0, 100.0);
+ c(&mut a.lens_blur_transition, 0.0, 100.0);
c(&mut a.color_hue, 0.0, 360.0);
c(&mut a.color_sat, 0.0, 100.0);
c(&mut a.amount, 0.0, 200.0);
@@ -1159,7 +1166,7 @@ pub(crate) fn clamp_local(mut a: LocalAdjustments) -> LocalAdjustments {
/// `mask.brushStroke`'s `pressure`: one number 0..1 per point (clamped), at most one per point; absent or `null`
/// is no pressure data.
-fn pressure_list(v: Option<&Value>, points: usize) -> Result> {
+pub(super) fn pressure_list(v: Option<&Value>, points: usize) -> Result> {
let Some(v) = v.filter(|v| !v.is_null()) else { return Ok(Vec::new()) };
let list = v.as_array().ok_or_else(|| bad("mask.brushStroke", "pressure must be a list of numbers 0..1"))?;
if list.len() > points {
diff --git a/crates/engine/src/segment.rs b/crates/engine/src/segment.rs
index 0f99636122..9c479cb176 100644
--- a/crates/engine/src/segment.rs
+++ b/crates/engine/src/segment.rs
@@ -64,6 +64,9 @@ pub(crate) enum Tag {
NewPrompt { photo: PhotoId, mask: Option, op: String, name: Option, text: String },
/// A zoomed-in pass for a component, computed from `shape` (without its detail).
Detail { photo: PhotoId, mask: u32, comp: usize, shape: MaskShape },
+ /// Find Subject for component `comp` of mask `mask` (an Object waiting for it): the people /
+ /// animals found become a "person" Describe selection; none, the object at `point`.
+ Subject { photo: PhotoId, mask: u32, comp: usize, point: Point },
}
/// The model input: 8-bit RGB.
@@ -573,6 +576,35 @@ impl Session {
}
}
+ /// Find Subject for component `comp` of mask `mask` on photo `id` (an Object without clicks):
+ /// the photo's main people / animals ([`worker::Kind::Subject`]), those the camera's `focus`
+ /// falls on when the file records it; when the subject is neither, the object at `point`. In
+ /// background mode [`Session::segment_poll`] applies the result; otherwise it is applied
+ /// before this returns.
+ pub(crate) fn segment_subject_later(&mut self, id: PhotoId, mask: u32, comp: usize, point: Point, focus: Option) -> Result<(), String> {
+ self.segmenter.model_dir()?;
+ #[cfg(feature = "sam")]
+ {
+ let kind = worker::Kind::Subject { focus: focus.map(|p| (p.x as f32, p.y as f32)) };
+ let tag = Tag::Subject { photo: id, mask, comp, point };
+ if self.segmenter.background {
+ return self.segment_queue(id, kind, tag, None);
+ }
+ let seg = self.segment_wait(id, kind, None)?;
+ let mut polled = Polled::default();
+ self.apply_outcome(worker::Outcome { tag, result: Ok(seg), superseded: false, notice: None }, &mut polled);
+ match polled.messages.into_iter().next() {
+ Some(e) => Err(e),
+ None => Ok(()),
+ }
+ }
+ #[cfg(not(feature = "sam"))]
+ {
+ let _ = (id, mask, comp, point, focus);
+ Err("AI masks are not available in this build".into())
+ }
+ }
+
/// Start a zoomed-in pass for component `comp` of mask `mask` on photo `id` (an Object or
/// Describe selection): the photo around the selection is rendered at a higher resolution
/// and segmented again. In background mode [`Session::segment_poll`] applies the result;
@@ -709,6 +741,51 @@ impl Session {
Err(e) => polled.messages.push(e.to_string()),
}
}
+ Tag::Subject { photo, mask, comp, point } => {
+ let found = match o.result {
+ Ok(found) => found,
+ Err(e) => return polled.messages.push(e),
+ };
+ let Some(d) = self.develop_of(photo) else { return };
+ let mut d = (*d).clone();
+ let Some(c) = d.masks.iter_mut().find(|m| m.id == mask).and_then(|m| m.components.get_mut(comp)) else { return };
+ // still the Object waiting for it (not clicked or replaced meanwhile)
+ let MaskShape::Object { hint, seg: None, edge, .. } = &c.shape else { return };
+ if !hint.is_empty() {
+ return;
+ }
+ let edge = *edge;
+ let seg = match found {
+ Some(seg) => seg,
+ // neither a person nor an animal: the object at the point
+ None if self.segmenter.background => {
+ if let Err(e) = self.segment_clicks_later(photo, mask, comp, vec![point], Vec::new()) {
+ polled.messages.push(e);
+ }
+ return;
+ }
+ None => {
+ match self.segment_clicks(photo, &clicks_of(&[point], &[])) {
+ Ok(seg) => {
+ c.shape = MaskShape::Object { hint: vec![point], exclude: Vec::new(), seg: Some(seg), detail: Vec::new(), edge }
+ }
+ Err(e) => return polled.messages.push(e),
+ }
+ if let Err(e) = self.set_develop(photo, d, "Find Subject") {
+ polled.messages.push(e.to_string());
+ } else {
+ polled.changed = true;
+ }
+ return;
+ }
+ };
+ let text = lightcraft_segment::SUBJECT_CONCEPTS.join(", ");
+ c.shape = MaskShape::Prompt { text, seg: Some(seg), detail: Vec::new(), edge };
+ match self.set_develop(photo, d, "Find Subject") {
+ Ok(()) => polled.changed = true,
+ Err(e) => polled.messages.push(e.to_string()),
+ }
+ }
Tag::Detail { photo, mask, comp, shape } => {
let patch = match o.result {
Ok(Some(p)) => p,
@@ -750,6 +827,47 @@ mod tests {
assert!(s.segment_poll().messages.is_empty(), "once");
}
+ /// Find Subject's result: people / animals found become a Describe selection of the waiting
+ /// Object (one undo step), not after it was clicked meanwhile; nothing found queues the object
+ /// at the point (here: no model, so a message, and the mask still waits).
+ #[cfg(feature = "sam")]
+ #[test]
+ fn find_subject_results_fill_the_waiting_mask() {
+ use lightcraft_develop::{Mask, MaskComponent, MaskOp};
+ let mut s = Session::with_demo();
+ s.segmenter.dir = Some(std::env::temp_dir().join(format!("lc-no-sam3-subject-{}", std::process::id())));
+ let id = s.active().unwrap();
+ let waiting = || MaskShape::Object { hint: vec![], exclude: vec![], seg: None, detail: vec![], edge: 0.0 };
+ let mut d = (*s.develop_of(id).unwrap()).clone();
+ d.masks.push(Mask {
+ id: 3,
+ components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: waiting() }],
+ ..Default::default()
+ });
+ s.set_develop(id, d, "test").unwrap();
+ let point = Point::new(0.4, 0.5);
+ let send = |s: &mut Session, result| {
+ let tag = Tag::Subject { photo: id, mask: 3, comp: 0, point };
+ s.segmenter.results.0.send(worker::Outcome { tag, result, superseded: false, notice: None }).unwrap();
+ s.segment_poll()
+ };
+ let shape = |s: &Session| s.develop_of(id).unwrap().masks[0].components[0].shape.clone();
+ // nothing found, no model to click with: said so, and the mask still waits
+ let polled = send(&mut s, Ok(None));
+ assert!(!polled.messages.is_empty() && shape(&s) == waiting(), "{:?}", polled.messages);
+ // people found: a Describe selection of them
+ let polled = send(&mut s, Ok(Some(seg_box(20, 5, 5, 15, 15))));
+ assert!(polled.changed && polled.messages.is_empty());
+ let MaskShape::Prompt { text, seg: Some(_), .. } = shape(&s) else { panic!("{:?}", shape(&s)) };
+ assert_eq!(text, "person, animal");
+ // a result for a mask that changed meanwhile is dropped
+ let before = shape(&s);
+ assert!(!send(&mut s, Ok(Some(seg_box(20, 0, 0, 4, 4)))).changed);
+ assert_eq!(shape(&s), before);
+ // an error reaches the user
+ assert_eq!(send(&mut s, Err("boom".into())).messages, vec!["boom".to_string()]);
+ }
+
fn seg_box(side: usize, x0: usize, y0: usize, x1: usize, y1: usize) -> SegMask {
let l: Vec =
(0..side * side).map(|i| if (x0..x1).contains(&(i % side)) && (y0..y1).contains(&(i / side)) { 8.0 } else { -8.0 }).collect();
diff --git a/crates/engine/src/segment/worker.rs b/crates/engine/src/segment/worker.rs
index efbb1c21c4..1e4bf6e8d9 100644
--- a/crates/engine/src/segment/worker.rs
+++ b/crates/engine/src/segment/worker.rs
@@ -42,6 +42,12 @@ pub enum Kind {
Prepare,
Clicks(Vec),
Text(String),
+ /// The photo's subject among its people and animals (`Sam3::segment_subject`); `focus`: where
+ /// the camera focused (normalized), when the file records it. No result: the subject is
+ /// something else.
+ Subject {
+ focus: Option<(f32, f32)>,
+ },
/// A zoomed-in pass over `region` (normalized) of the photo, rendered by the job's `render`.
Detail {
prompt: Prompt,
@@ -420,6 +426,7 @@ fn remote_job(state: &mut State, shared: &Shared, job: &mut Job, endpoint: &str)
let operation = match std::mem::replace(&mut job.kind, Kind::Prepare) {
Kind::Prepare => Operation::Prepare,
Kind::Text(text) => Operation::Text(text),
+ Kind::Subject { focus } => Operation::Subject(focus.map(|(x, y)| [x, y])),
Kind::Clicks(clicks) => Operation::Clicks(clicks.iter().map(|c| [c.x, c.y, if c.positive { 1.0 } else { 0.0 }]).collect()),
Kind::Detail { prompt, region: r } => {
region = Some(r);
@@ -480,6 +487,11 @@ fn run_job(state: &mut State, shared: &Shared, job: &mut Job, placement: Placeme
let (m, enc) = encoded(state, shared, job, placement)?;
Ok(text_logits(m, enc, &text)?.map(|l| SegMask::from_logits(side, &l)))
}
+ Kind::Subject { focus } => {
+ let (m, enc) = encoded(state, shared, job, placement)?;
+ // (the reference score threshold, as Describe uses)
+ Ok(m.segment_subject(enc, focus, 0.5)?.map(|p| SegMask::from_logits(side, &p.logits)))
+ }
#[cfg(test)]
#[allow(clippy::panic)]
Kind::Panic => panic!("a test panic"),
diff --git a/crates/engine/src/tests.rs b/crates/engine/src/tests.rs
index 632814334d..64641f9edf 100644
--- a/crates/engine/src/tests.rs
+++ b/crates/engine/src/tests.rs
@@ -385,6 +385,84 @@ fn permanently_deleting_explicit_targets_keeps_the_surviving_selection() {
assert_eq!(s.catalog.to_snapshot(), snapshot);
}
+/// `lensBlur.brushStroke` adds a stroke to the Focus or the Blur brush (clamped, undoable); hostile
+/// input is an error; `lensBlur.brushReset` clears one brush or both.
+#[test]
+fn lens_blur_brush_strokes_and_reset() {
+ let mut s = demo();
+ let id = s.active().unwrap();
+ s.execute(
+ "lensBlur.brushStroke",
+ &json!({"mode": "focus", "points": [[0.2, 0.3], [0.25, 0.3]], "size": 9.0, "amount": 40, "feather": f64::NAN, "autoMask": true}),
+ )
+ .unwrap();
+ s.execute("lensBlur.brushStroke", &json!({"mode": "blur", "points": [[0.7, 0.6]]})).unwrap();
+ let d = s.develop_of(id).unwrap();
+ let (f, b) = (&d.lens_blur.focus_strokes, &d.lens_blur.blur_strokes);
+ assert_eq!((f.len(), b.len()), (1, 1));
+ assert_eq!(f[0].points.len(), 2);
+ assert!(f[0].size == 0.5 && f[0].density == 40.0 && f[0].feather == lightcraft_develop::BrushStroke::default().feather && f[0].auto_mask);
+ for bad in [
+ json!({"mode": "sideways", "points": [[0.5, 0.5]]}),
+ json!({"mode": "focus", "points": []}),
+ json!({"mode": "focus"}),
+ json!({"mode": "blur", "points": [[0.5, 0.5]], "pressure": [0.5, 0.5]}),
+ ] {
+ assert!(s.execute("lensBlur.brushStroke", &bad).is_err(), "{bad}");
+ }
+ s.execute("edit.undo", &json!({})).unwrap();
+ assert!(s.develop_of(id).unwrap().lens_blur.blur_strokes.is_empty(), "a stroke is one undo step");
+ s.execute("lensBlur.brushStroke", &json!({"mode": "blur", "points": [[0.7, 0.6]]})).unwrap();
+ s.execute("lensBlur.brushReset", &json!({"mode": "blur"})).unwrap();
+ let d = s.develop_of(id).unwrap();
+ assert!(d.lens_blur.blur_strokes.is_empty() && d.lens_blur.focus_strokes.len() == 1);
+ assert!(s.execute("lensBlur.brushReset", &json!({"mode": "nope"})).is_err());
+ s.execute("lensBlur.brushReset", &json!({})).unwrap();
+ assert!(s.develop_of(id).unwrap().lens_blur.focus_strokes.is_empty());
+}
+
+/// `lensBlur.findSubject` without the SAM 3 model (or in a build without AI masks) is an error that
+/// leaves the photo as it was: no empty mask, the blur not switched on.
+#[test]
+fn find_subject_without_the_model_changes_nothing() {
+ let mut s = demo();
+ let id = s.active().unwrap();
+ // (never the machine's own model: a folder that has none)
+ s.segmenter.dir = Some(std::env::temp_dir().join(format!("lc-no-sam3-{}", std::process::id())));
+ let before = s.develop_of(id).unwrap();
+ let e = s.execute("lensBlur.findSubject", &json!({})).unwrap_err();
+ assert!(e.to_string().contains("SAM 3") || e.to_string().contains("AI masks"), "{e}");
+ let after = s.develop_of(id).unwrap();
+ assert!(after.masks.is_empty() && !after.lens_blur.enabled);
+ assert_eq!(*before, *after);
+}
+
+/// Deleting the mask that is the Lens Blur focus clears the focus, so a new mask that gets the same
+/// id doesn't become the focus; undo brings both back.
+#[test]
+fn deleting_the_focus_mask_clears_the_lens_blur_focus() {
+ let mut s = demo();
+ let id = s.active().unwrap();
+ let radial = json!({"kind": "radial", "center": [0.5, 0.5], "rx": 0.2, "ry": 0.2});
+ s.execute("mask.add", &radial).unwrap();
+ let mid = s.develop_of(id).unwrap().masks[0].id;
+ s.execute("develop.merge", &json!({"settings": {"lens_blur": {"enabled": true, "amount": 50, "focus_mask": mid}}})).unwrap();
+ s.execute("mask.delete", &json!({"id": mid})).unwrap();
+ assert_eq!(s.develop_of(id).unwrap().lens_blur.focus_mask, None);
+ // a new mask takes the freed id, and stays an ordinary mask
+ s.execute("mask.add", &radial).unwrap();
+ let d = s.develop_of(id).unwrap();
+ assert_eq!(d.masks[0].id, mid);
+ assert_eq!(d.lens_blur.focus_mask, None);
+ // undoing the add and the delete brings the focus back with its mask
+ s.execute("edit.undo", &json!({})).unwrap();
+ s.execute("edit.undo", &json!({})).unwrap();
+ assert_eq!(s.develop_of(id).unwrap().lens_blur.focus_mask, Some(mid));
+ // other mask edits keep it
+ s.execute("mask.adjust", &json!({"values": {"exposure": 0.5}})).unwrap();
+ assert_eq!(s.develop_of(id).unwrap().lens_blur.focus_mask, Some(mid));
+}
+
/// `mask.brushStroke {connect: true}` (a ⇧-click) starts from the last point of the mask's last stroke: one straight
/// stroke when the brush is unchanged, pieces whose size, feather and flow step from the last stroke's to the new
/// values when it changed, and a plain dab when there is nothing to connect to.
diff --git a/crates/gpu/src/render.rs b/crates/gpu/src/render.rs
index bc079edb11..a392ee6ee1 100644
--- a/crates/gpu/src/render.rs
+++ b/crates/gpu/src/render.rs
@@ -1054,7 +1054,14 @@ fn linear(cx: &mut Cx<'_>, sampled: &Buf, info: &SourceInfo, plan: &Plan<'_>, ho
eyed
}
};
- denoise(cx, img, plan, colour_noise)
+ let img = denoise(cx, img, plan, colour_noise);
+ if !lightcraft_pipeline::lens_blur::renders(s) {
+ return img;
+ }
+ // Lens Blur (global or a mask's): on the CPU, like the CPU renderer's (`lens_blur::apply`)
+ let mut host_img = cx.read_rgb(&img, w, h);
+ lightcraft_pipeline::lens_blur::apply_with(&mut host_img, s, &plan.frame, plan.px_per_long, plan.mattes.as_deref(), plan.lens_subject.as_deref());
+ cx.gpu.upload(rgb_words(&host_img))
}
/// Defringe (`optics::defringe`) of the white-balanced image.
diff --git a/crates/pipeline/src/lens_blur.rs b/crates/pipeline/src/lens_blur.rs
new file mode 100644
index 0000000000..f518eb5de7
--- /dev/null
+++ b/crates/pipeline/src/lens_blur.rs
@@ -0,0 +1,1109 @@
+//! Lens Blur: a shallow depth of field added after the fact.
+//!
+//! Without a depth map, the scene has two depths: what is in focus (the photo's subject, or the
+//! selection of the mask [`LensBlur::focus_mask`](lightcraft_develop::LensBlur) names) and
+//! everything else. A pixel's circle of confusion is `R · (1 − focus)`, `R` set by Blur Amount
+//! relative to the long edge, so previews and exports match.
+//!
+//! The blur is gathered from [`LEVELS`] disc blurs of growing radius, blended per pixel by its
+//! circle of confusion. Each disc blur runs on a grid coarse enough that the disc spans about
+//! [`DISC_CELLS`] cells (its cost doesn't grow with the radius) and is a normalized convolution
+//! weighted by how far out of focus each pixel is, so the sharp subject doesn't bleed into the
+//! blurred background around it (no halo). Boost brightens highlights before they are spread, so
+//! they turn into bright discs (bokeh), as a lens does with scene-linear light.
+//!
+//! Brush Refinement paints the focus by hand: Focus brush strokes stay sharp, Blur brush strokes
+//! are blurred whatever the focus says.
+//!
+//! A mask can blur too (Lens Blur among its local adjustments): there the mask's strength is the
+//! circle of confusion, so what it fully selects gets the whole blur, its feathered edge less, and
+//! what it leaves out stays sharp and doesn't bleed into the blur.
+//!
+//! Amount is capped at 3.5% of the output's long edge. The disc convolution runs on a grid whose
+//! disc radius is at most [`DISC_CELLS`] cells and whose size is capped by [`MAX_CELLS`] per level,
+//! so increasing the output resolution or blur radius does not make the convolution kernel grow.
+//! The full-resolution passes remain linear in the output pixel count.
+//!
+//! It runs on the scene-linear image (white-balanced and denoised, before exposure: a blur is
+//! linear, so exposure commutes with it), on the CPU, for both renderers.
+
+use lightcraft_develop::{BrushStroke, DevelopSettings, Mask, MaskShape};
+use lightcraft_raster::{Plane, Rgb32f};
+
+use crate::geometry::Frame;
+use crate::masks::{self, Mattes};
+
+/// The largest circle of confusion (Blur Amount 100), as a fraction of the long edge.
+pub const MAX_RADIUS: f64 = 0.035;
+/// How far from the focus the blur reaches its full size at Transition 100, as a fraction of the
+/// long edge (a stand-in for depth: what is near the subject in the picture is usually near it in
+/// the scene too, as a lens renders a floor or a wall running away from it).
+pub const MAX_TRANSITION: f64 = 0.15;
+/// The blur right at the focus's edge, as a share of the full blur (with a transition): the
+/// background around a subject is a little soft at once, not cut out sharp.
+const EDGE_BLUR: f32 = 0.1;
+/// The feather (0..100) of a fixed focus shape's edge ([`lightcraft_develop::FocusShape`]).
+const SHAPE_FEATHER: f64 = 40.0;
+/// Most cells along the long edge of the grid the distance to the focus is measured on.
+const DISTANCE_CELLS: usize = 384;
+/// Disc blurs the per-pixel circle of confusion is interpolated between.
+const LEVELS: usize = 4;
+/// Radius, in grid cells, a level's disc blur works at.
+const DISC_CELLS: f32 = 8.0;
+/// Most cells a level's grid holds (bounds the memory of a full-size export).
+const MAX_CELLS: usize = 3_000_000;
+/// Weight of in-focus pixels in the blurred levels: enough to keep a level defined where only
+/// in-focus pixels are near, too little to bleed them into the background.
+const FOCUS_WEIGHT: f32 = 1e-4;
+/// The brightest highlights grow this much brighter at Boost 100 before they are spread.
+const BOOST_GAIN: f32 = 4.0;
+
+/// Whether `s` renders a lens blur (switched on, with some amount, and its section's eye open).
+pub fn active(s: &DevelopSettings) -> bool {
+ let b = &s.lens_blur;
+ b.enabled && b.amount.is_finite() && b.amount > 0.0 && s.section_enabled("lensBlur")
+}
+
+/// Whether [`apply`] changes anything: the global lens blur, or a mask's.
+pub fn renders(s: &DevelopSettings) -> bool {
+ active(s) || blurring_masks(s).next().is_some()
+}
+
+/// The largest circle of confusion (radius in output pixels) at `px_per_long` output pixels per
+/// long edge; 0 when [`active`] is false.
+pub fn radius_px(s: &DevelopSettings, px_per_long: f64) -> f32 {
+ if !active(s) {
+ return 0.0;
+ }
+ let r = s.lens_blur.amount.clamp(0.0, 100.0) / 100.0 * MAX_RADIUS * px_per_long;
+ if r.is_finite() { r.max(0.0) as f32 } else { 0.0 }
+}
+
+/// The masks that blur what they select, or what they leave out (the Lens Blur block among a
+/// mask's local adjustments): shown, with components, the block on and some blur.
+fn blurring_masks(s: &DevelopSettings) -> impl Iterator {
+ s.masks
+ .iter()
+ .filter(|m| m.visible && !m.components.is_empty() && !m.adjust.lens_blur_off && m.adjust.lens_blur.is_finite() && m.adjust.lens_blur > 0.0)
+}
+
+/// The circle of confusion (radius in output pixels) where mask `m` fully selects.
+fn mask_radius_px(m: &Mask, px_per_long: f64) -> f32 {
+ let r = m.adjust.lens_blur.clamp(0.0, 100.0) / 100.0 * MAX_RADIUS * px_per_long;
+ if r.is_finite() { r.max(0.0) as f32 } else { 0.0 }
+}
+
+/// The distance from the focus (output pixels) at which the blur is full; 0: a sharp cut.
+pub fn transition_px(s: &DevelopSettings, px_per_long: f64) -> f32 {
+ if !active(s) {
+ return 0.0;
+ }
+ transition_of(s.lens_blur.transition, px_per_long)
+}
+
+/// [`transition_px`] for a Transition slider value `t` (0..100).
+fn transition_of(t: f64, px_per_long: f64) -> f32 {
+ if !t.is_finite() || t <= 0.0 {
+ return 0.0;
+ }
+ let d = t.min(100.0) / 100.0 * MAX_TRANSITION * px_per_long;
+ if d.is_finite() { d.max(0.0) as f32 } else { 0.0 }
+}
+
+/// A Boost slider value (0..100) as 0..1.
+fn boost_of(b: f64) -> f32 {
+ if b.is_finite() { (b.clamp(0.0, 100.0) / 100.0) as f32 } else { 0.0 }
+}
+
+/// The farthest any lens blur of `s` reads (output pixels): the global one's (its blur, and how far
+/// it looks for the focus) or a mask's.
+pub fn reach_px(s: &DevelopSettings, px_per_long: f64) -> f32 {
+ let global = radius_px(s, px_per_long) + transition_px(s, px_per_long);
+ blurring_masks(s).map(|m| mask_radius_px(m, px_per_long) + transition_of(m.adjust.lens_blur_transition, px_per_long)).fold(global, f32::max)
+}
+
+/// `win` (a window of the `full_w × full_h` output) grown by the blur's reach, so a zoomed view
+/// blurs its edges with the pixels around them, as the whole render does (the render is cut back
+/// to `win`, see [`crate::Plan::keep`]).
+pub fn window_for_reads(s: &DevelopSettings, full_w: usize, full_h: usize, px_per_long: f64, win: crate::PixelWindow) -> crate::PixelWindow {
+ let r = reach_px(s, px_per_long);
+ if r <= 0.0 {
+ return win;
+ }
+ let m = (r * 1.25).ceil() as usize + 2;
+ let (x0, y0) = (win.x.saturating_sub(m), win.y.saturating_sub(m));
+ let x1 = win.x.saturating_add(win.w).saturating_add(m).min(full_w.max(1));
+ let y1 = win.y.saturating_add(win.h).saturating_add(m).min(full_h.max(1));
+ crate::PixelWindow { x: x0, y: y0, w: x1.saturating_sub(x0).max(1), h: y1.saturating_sub(y0).max(1) }
+}
+
+/// What [`apply`] reads from `s`, for render cache keys (empty when no lens blur renders).
+pub fn key(s: &DevelopSettings) -> String {
+ let mut k = String::new();
+ if active(s) {
+ k = match focus_mask(s) {
+ Some(m) => format!("{:?}|{m:?}", s.lens_blur),
+ None => format!("{:?}", s.lens_blur),
+ };
+ }
+ for m in blurring_masks(s) {
+ k.push_str(&format!("|{m:?}"));
+ }
+ // a mask's luminance / colour ranges select on the exposed image
+ if !k.is_empty() {
+ k.push_str(&format!("|{}", s.light.exposure.to_bits()));
+ }
+ k
+}
+
+/// The mask that says what is in focus, if the settings name one that exists and selects something.
+/// An AI mask whose selection is still being computed (Find Subject on a slow machine) doesn't
+/// count yet: until it is, the subject estimate stays in focus rather than nothing.
+fn focus_mask(s: &DevelopSettings) -> Option<&Mask> {
+ let id = s.lens_blur.focus_mask?;
+ s.masks.iter().find(|m| m.id == id && !m.components.is_empty() && !awaiting_ai(m))
+}
+
+/// Whether every component of `m` is an AI selection (Object / Describe) not computed yet.
+pub fn awaiting_ai(m: &Mask) -> bool {
+ m.components.iter().all(|c| match &c.shape {
+ MaskShape::Object { seg, detail, .. } | MaskShape::Prompt { seg, detail, .. } => seg.is_none() && detail.is_empty(),
+ _ => false,
+ })
+}
+
+/// Smoothstep: 0 below `e0`, 1 above `e1`, a smooth S-curve between.
+#[inline]
+fn smooth(e0: f32, e1: f32, x: f32) -> f32 {
+ let t = ((x - e0) / (e1 - e0)).clamp(0.0, 1.0);
+ t * t * (3.0 - 2.0 * t)
+}
+
+/// What is in focus, 0..1 per pixel of `img` (scene-linear, before exposure, output resolution).
+pub fn focus(img: &Rgb32f, s: &DevelopSettings, frame: &Frame, mattes: Option<&Mattes>) -> Plane {
+ focus_with(img, s, frame, mattes, None)
+}
+
+/// [`focus`], taking the automatic subject estimate from `subject` (made from the whole photo,
+/// see [`subject_estimate`]) when there is one, so a window of the output gets the same focus as
+/// the whole render.
+fn focus_with(img: &Rgb32f, s: &DevelopSettings, frame: &Frame, mattes: Option<&Mattes>, subject: Option<&SubjectEstimate>) -> Plane {
+ let (w, h) = (img.width, img.height);
+ let log_l = img.map(crate::local::log_lum);
+ let mut a = match (focus_mask(s), s.lens_blur.focus_area(s.crop_center())) {
+ (Some(m), _) => masks::evaluate_one(m, frame, w, h, img, &log_l, s.light.exposure as f32, mattes),
+ // a fixed shape: where it was moved to, else the middle of the crop (in photo
+ // coordinates, so it stays put in a zoomed view)
+ (None, Some(area)) => {
+ let shape = MaskShape::Radial { center: area.center, rx: area.rx, ry: area.ry, angle: 0.0, feather: SHAPE_FEATHER, invert: false };
+ masks::shape_alpha(&shape, frame, w, h, img, &log_l, 0.0, mattes)
+ }
+ (None, None) if let Some(est) = subject => est.sample(frame, w, h),
+ (None, None) => {
+ let mut a = masks::shape_alpha(&MaskShape::Subject, frame, w, h, img, &log_l, 0.0, mattes);
+ // the saliency heuristic gives a flat centre about 0.5: in focus too (person mattes
+ // are 0 or 1 and stay so)
+ a.data.iter_mut().for_each(|v| *v = smooth(0.25, 0.55, *v));
+ a
+ }
+ };
+ // Brush Refinement: the Focus brush adds focus, the Blur brush takes it away (last word)
+ let b = &s.lens_blur;
+ let brush = |strokes: &[BrushStroke]| {
+ (!strokes.is_empty()).then(|| masks::shape_alpha(&MaskShape::Brush { strokes: strokes.to_vec() }, frame, w, h, img, &log_l, 0.0, mattes))
+ };
+ if let Some(f) = brush(&b.focus_strokes) {
+ a.data.iter_mut().zip(&f.data).for_each(|(v, f)| *v = v.max(*f));
+ }
+ if let Some(bl) = brush(&b.blur_strokes) {
+ a.data.iter_mut().zip(&bl.data).for_each(|(v, b)| *v *= 1.0 - b.clamp(0.0, 1.0));
+ }
+ a.data.iter_mut().for_each(|v| *v = if v.is_finite() { v.clamp(0.0, 1.0) } else { 0.0 });
+ a
+}
+
+/// Long edge of the small render [`subject_point`] looks at.
+const SUBJECT_EDGE: usize = 256;
+
+/// Long edge of the small render [`subject_estimate`] makes of the whole photo.
+const ESTIMATE_EDGE: usize = 384;
+
+/// The automatic subject estimate (the saliency heuristic, for a photo without a person matte)
+/// made once from the whole, uncropped photo at [`ESTIMATE_EDGE`]: every render of the photo,
+/// whole or a window of it, samples the same estimate, so a zoomed view keeps the focus the
+/// whole render has (the heuristic reads its neighbourhood, which a window cuts off).
+#[derive(Debug)]
+pub struct SubjectEstimate {
+ plane: Plane,
+ frame: Frame,
+}
+
+impl SubjectEstimate {
+ /// The estimate at each pixel of a `w × h` output of `frame` (bilinear).
+ fn sample(&self, frame: &Frame, w: usize, h: usize) -> Plane {
+ let (pw, ph) = (self.plane.width, self.plane.height);
+ let mut out = Plane::new(w, h);
+ if pw == 0 || ph == 0 || self.plane.data.len() != pw * ph {
+ return out;
+ }
+ let to_norm = frame.out_to_norm(w, h);
+ let to_proxy = self.frame.norm_to_out(pw, ph);
+ let at = |x: usize, y: usize| self.plane.data.get(y.min(ph - 1) * pw + x.min(pw - 1)).copied().unwrap_or(0.0);
+ for (i, v) in out.data.iter_mut().enumerate() {
+ let n = to_norm.apply(lightcraft_geom::Point::new((i % w) as f64 + 0.5, (i / w) as f64 + 0.5));
+ let p = to_proxy.apply(n);
+ let (fx, fy) = ((p.x - 0.5).clamp(0.0, (pw - 1) as f64), (p.y - 0.5).clamp(0.0, (ph - 1) as f64));
+ if !(fx.is_finite() && fy.is_finite()) {
+ continue;
+ }
+ let (x0, y0) = (fx as usize, fy as usize);
+ let (tx, ty) = ((fx - x0 as f64) as f32, (fy - y0 as f64) as f32);
+ let top = at(x0, y0) + (at(x0 + 1, y0) - at(x0, y0)) * tx;
+ let bot = at(x0, y0 + 1) + (at(x0 + 1, y0 + 1) - at(x0, y0 + 1)) * tx;
+ *v = top + (bot - top) * ty;
+ }
+ out
+ }
+}
+
+/// The [`SubjectEstimate`] the lens blur of `s` focuses on, when it focuses on the automatic
+/// subject estimate (switched on, no focus mask, no fixed shape) of a photo without a person
+/// matte; `None` otherwise (the focus then needs no whole-photo data).
+pub fn subject_estimate(src: &Rgb32f, s: &DevelopSettings, mattes: Option<&Mattes>) -> Option {
+ if !active(s) || focus_mask(s).is_some() || s.lens_blur.focus_area(s.crop_center()).is_some() || mattes.is_some_and(Mattes::selects_subject) {
+ return None;
+ }
+ if src.width == 0 || src.height == 0 || src.data.len() != src.width.saturating_mul(src.height) {
+ return None;
+ }
+ let frame = Frame::new(src.width, src.height, s, false);
+ let (w, h) = frame.fit(ESTIMATE_EDGE, ESTIMATE_EDGE);
+ if w == 0 || h == 0 {
+ return None;
+ }
+ let img = frame.sample(src, w, h);
+ let log_l = img.map(crate::local::log_lum);
+ let mut plane = masks::shape_alpha(&MaskShape::Subject, &frame, w, h, &img, &log_l, 0.0, None);
+ // the saliency heuristic gives a flat centre about 0.5: in focus too
+ plane.data.iter_mut().for_each(|v| *v = smooth(0.25, 0.55, *v));
+ Some(SubjectEstimate { plane, frame })
+}
+
+/// Where the photo's subject most likely is, for a photo whose camera recorded no focus point: the
+/// peak of the subject estimate (person mattes, else the saliency heuristic) over the whole,
+/// uncropped photo, in normalized mask coordinates. `None` for an empty source.
+pub fn subject_point(src: &Rgb32f, s: &DevelopSettings, mattes: Option<&Mattes>) -> Option {
+ if src.width == 0 || src.height == 0 || src.data.len() != src.width.saturating_mul(src.height) {
+ return None;
+ }
+ let frame = Frame::new(src.width, src.height, s, false);
+ let (w, h) = frame.fit(SUBJECT_EDGE, SUBJECT_EDGE);
+ if w == 0 || h == 0 {
+ return None;
+ }
+ let img = frame.sample(src, w, h);
+ let log_l = img.map(crate::local::log_lum);
+ let a = masks::shape_alpha(&MaskShape::Subject, &frame, w, h, &img, &log_l, 0.0, mattes);
+ let (i, _) = a.data.iter().enumerate().filter(|(_, v)| v.is_finite()).max_by(|x, y| x.1.total_cmp(y.1))?;
+ let n = frame.out_to_norm(w, h).apply(lightcraft_geom::Point::new((i % w) as f64 + 0.5, (i / w) as f64 + 0.5));
+ (n.x.is_finite() && n.y.is_finite()).then(|| lightcraft_geom::Point::new(n.x.clamp(0.0, 1.0), n.y.clamp(0.0, 1.0)))
+}
+
+/// Blur what is out of focus in `img` (scene-linear, before exposure, at output resolution;
+/// `frame` and `px_per_long` describe that output), then what each mask with a Lens Blur
+/// adjustment selects (in mask order). Does nothing when no lens blur renders.
+pub fn apply(img: &mut Rgb32f, s: &DevelopSettings, frame: &Frame, px_per_long: f64, mattes: Option<&Mattes>) {
+ apply_with(img, s, frame, px_per_long, mattes, None);
+}
+
+/// [`apply`] with the photo's [`SubjectEstimate`] (renders pass [`crate::Plan::lens_subject`]).
+pub fn apply_with(
+ img: &mut Rgb32f,
+ s: &DevelopSettings,
+ frame: &Frame,
+ px_per_long: f64,
+ mattes: Option<&Mattes>,
+ subject: Option<&SubjectEstimate>,
+) {
+ let (w, h) = (img.width, img.height);
+ if w == 0 || h == 0 || img.data.len() != w.saturating_mul(h) {
+ return;
+ }
+ // a zoomed view's place in the whole output: its coarse grids line up with the whole render's
+ let at = |v: f64| if v.is_finite() && v > 0.0 { v.round() as usize } else { 0 };
+ let origin = frame.view.map_or((0, 0), |v| (at(v.x), at(v.y)));
+ let r = radius_px(s, px_per_long);
+ if r >= 0.5 {
+ let focus = crate::timed("lens blur focus", || focus_with(img, s, frame, mattes, subject));
+ if focus.data.len() == img.data.len() {
+ // the blur grows with the distance from what is in focus, as with depth
+ let ramp = crate::timed("lens blur transition", || transition_ramp(&focus, transition_px(s, px_per_long), px_per_long, origin));
+ let coc: Vec = focus.data.iter().enumerate().map(|(i, f)| (1.0 - f) * ramp.as_ref().map_or(1.0, |r| r.at(i))).collect();
+ blur(img, &coc, r, boost_of(s.lens_blur.boost), origin);
+ }
+ }
+ let ev = s.light.exposure as f32;
+ for m in blurring_masks(s) {
+ let r = mask_radius_px(m, px_per_long);
+ if r < 0.5 {
+ continue;
+ }
+ // the mask's strength (its Amount included) is how far out of focus a pixel is; blurring
+ // the outside, it is how far the mask stays below its Amount
+ let log_l = img.map(crate::local::log_lum);
+ let alpha = crate::timed("lens blur mask", || masks::evaluate_one(m, frame, w, h, img, &log_l, ev, mattes));
+ if alpha.data.len() != img.data.len() {
+ continue;
+ }
+ let a = &m.adjust;
+ let top = if a.amount.is_finite() { (a.amount / 100.0).clamp(0.0, 1.0) as f32 } else { 0.0 };
+ let mut coc: Vec = alpha
+ .data
+ .iter()
+ .map(|v| {
+ let v = if v.is_finite() { v.clamp(0.0, 1.0) } else { 0.0 };
+ if a.lens_blur_outside { (top - v).max(0.0) } else { v }
+ })
+ .collect();
+ // the blur grows away from what stays sharp, as the global one's does
+ let sharp = Plane { width: w, height: h, data: coc.iter().map(|c| 1.0 - c).collect() };
+ if let Some(ramp) = transition_ramp(&sharp, transition_of(a.lens_blur_transition, px_per_long), px_per_long, origin) {
+ coc.iter_mut().enumerate().for_each(|(i, c)| *c *= ramp.at(i));
+ }
+ blur(img, &coc, r, boost_of(a.lens_blur_boost), origin);
+ }
+}
+
+/// How much of the full blur each pixel gets for its distance from the focus: [`EDGE_BLUR`] at
+/// the focus's edge, growing smoothly to all of it `d` pixels away.
+struct Ramp {
+ /// Distance from the focus, in grid cells.
+ dist: Plane,
+ /// Pixels per cell.
+ g: usize,
+ w: usize,
+ d: f32,
+ /// Where pixel (0, 0) sits in the grid (the grid follows the whole output's cells).
+ shift: (usize, usize),
+}
+
+impl Ramp {
+ /// The share of the full blur pixel `i` (row-major in the window) gets.
+ fn at(&self, i: usize) -> f32 {
+ let (x, y) = (i % self.w.max(1) + self.shift.0, i / self.w.max(1) + self.shift.1);
+ let g = self.g as f32;
+ // bilinear in the grid, cell centres at +0.5
+ let (fx, fy) = (((x as f32 + 0.5) / g - 0.5).max(0.0), ((y as f32 + 0.5) / g - 0.5).max(0.0));
+ let (gw, gh) = (self.dist.width, self.dist.height);
+ let (x0, y0) = ((fx as usize).min(gw - 1), (fy as usize).min(gh - 1));
+ let (x1, y1) = ((x0 + 1).min(gw - 1), (y0 + 1).min(gh - 1));
+ let (tx, ty) = (fx - x0 as f32, fy - y0 as f32);
+ let v = |x: usize, y: usize| self.dist.data.get(y * gw + x).copied().unwrap_or(f32::MAX);
+ let top = v(x0, y0) + (v(x1, y0) - v(x0, y0)) * tx;
+ let bot = v(x0, y1) + (v(x1, y1) - v(x0, y1)) * tx;
+ let px = (top + (bot - top) * ty) * g;
+ EDGE_BLUR + (1.0 - EDGE_BLUR) * smooth(0.0, self.d, px)
+ }
+}
+
+/// The [`Ramp`] for `focus` over `d` pixels; `None` for a sharp cut (`d` 0) or nothing in focus.
+/// Distances are chamfer distances (steps of 1 and √2 cells) on a grid of about
+/// [`DISTANCE_CELLS`] along the photo's long edge (`px_per_long`: the cell size follows the
+/// photo's scale, not the rendered window's, and its cells are the whole output's: `origin` is
+/// where a window's pixel (0, 0) sits in it, so a zoomed view measures as the whole render does):
+/// what is in focus are the cells mostly in focus.
+fn transition_ramp(focus: &Plane, d: f32, px_per_long: f64, origin: (usize, usize)) -> Option {
+ let (w, h) = (focus.width, focus.height);
+ // (a NaN distance is no transition)
+ if d.is_nan() || d <= 0.5 || w == 0 || h == 0 || focus.data.len() != w * h {
+ return None;
+ }
+ let per_cell = px_per_long / DISTANCE_CELLS as f64;
+ let g = if per_cell.is_finite() { (per_cell.ceil() as usize).clamp(1, w.max(h)) } else { 1 };
+ let shift = (origin.0 % g, origin.1 % g);
+ // the grid covers the window shifted into the whole output's cells
+ let (gw, gh) = ((w + shift.0).div_ceil(g), (h + shift.1).div_ceil(g));
+ let mut dist = Plane::new(gw, gh);
+ let mut any = false;
+ for cy in 0..gh {
+ for cx in 0..gw {
+ let (mut sum, mut n) = (0.0f32, 0usize);
+ for vy in cy * g..((cy + 1) * g).min(h + shift.1) {
+ for vx in cx * g..((cx + 1) * g).min(w + shift.0) {
+ let (Some(x), Some(y)) = (vx.checked_sub(shift.0), vy.checked_sub(shift.1)) else { continue };
+ sum += focus.data.get(y * w + x).copied().unwrap_or(0.0);
+ n += 1;
+ }
+ }
+ let inside = n > 0 && sum / n as f32 > 0.5;
+ any |= inside;
+ if let Some(c) = dist.data.get_mut(cy * gw + cx) {
+ *c = if inside { 0.0 } else { f32::MAX };
+ }
+ }
+ }
+ if !any {
+ return None;
+ }
+ let diag = std::f32::consts::SQRT_2;
+ let at = |d: &Plane, x: isize, y: isize| {
+ if x < 0 || y < 0 || x >= gw as isize || y >= gh as isize {
+ f32::MAX
+ } else {
+ d.data.get(y as usize * gw + x as usize).copied().unwrap_or(f32::MAX)
+ }
+ };
+ // forward then backward chamfer passes
+ for y in 0..gh as isize {
+ for x in 0..gw as isize {
+ let c = at(&dist, x, y)
+ .min(at(&dist, x - 1, y) + 1.0)
+ .min(at(&dist, x, y - 1) + 1.0)
+ .min(at(&dist, x - 1, y - 1) + diag)
+ .min(at(&dist, x + 1, y - 1) + diag);
+ if let Some(v) = dist.data.get_mut(y as usize * gw + x as usize) {
+ *v = c;
+ }
+ }
+ }
+ for y in (0..gh as isize).rev() {
+ for x in (0..gw as isize).rev() {
+ let c = at(&dist, x, y)
+ .min(at(&dist, x + 1, y) + 1.0)
+ .min(at(&dist, x, y + 1) + 1.0)
+ .min(at(&dist, x + 1, y + 1) + diag)
+ .min(at(&dist, x - 1, y + 1) + diag);
+ if let Some(v) = dist.data.get_mut(y as usize * gw + x as usize) {
+ *v = c;
+ }
+ }
+ }
+ Some(Ramp { dist, g, w, d, shift })
+}
+
+/// Blur `img` by a circle of confusion `coc · r` pixels per pixel (`coc` 0..1, one per pixel),
+/// highlights brightened by `boost` (0..1) before they are spread.
+/// `origin`: where `img`'s pixel (0, 0) sits in the whole output (a zoomed view's window), so the
+/// coarse grids line up with the whole render's.
+fn blur(img: &mut Rgb32f, coc: &[f32], r: f32, boost: f32, origin: (usize, usize)) {
+ let w = img.width;
+ // highlights brightened before they are spread (the bokeh), the rest as it is
+ let gain = |c: [f32; 3]| 1.0 + BOOST_GAIN * boost * smooth(0.5, 1.0, lightcraft_color::luminance_2020(c));
+ let levels: Vec = (1..=LEVELS).map(|k| crate::timed("lens blur level", || level(img, coc, k, r, &gain, origin))).collect();
+ // each pixel between the two levels around its circle of confusion (level 0: the image)
+ let src = img.clone();
+ crate::for_rows(&mut img.data, w, |y, row| {
+ for (x, px) in row.iter_mut().enumerate() {
+ let i = y * w + x;
+ let t = coc.get(i).copied().unwrap_or(0.0) * LEVELS as f32;
+ let k = (t.floor().max(0.0) as usize).min(LEVELS - 1);
+ let fr = (t - k as f32).clamp(0.0, 1.0);
+ let at = |k: usize| match k {
+ 0 => src.data.get(i).copied().unwrap_or(*px),
+ k => levels.get(k - 1).map_or(*px, |l| l.sample(x, y)),
+ };
+ let (a, b) = (at(k), at(k + 1));
+ *px = std::array::from_fn(|c| a[c] + (b[c] - a[c]) * fr);
+ }
+ });
+}
+
+/// One disc blur, on a grid `f` times coarser than the image.
+struct Level {
+ img: Rgb32f,
+ f: usize,
+ /// Where image pixel (0, 0) sits on the grid's pixels (the whole output's cells).
+ shift: (usize, usize),
+}
+
+impl Level {
+ /// The blurred value at image pixel `(x, y)`.
+ fn sample(&self, x: usize, y: usize) -> [f32; 3] {
+ let f = self.f as f32;
+ self.img.sample_bilinear(((x + self.shift.0) as f32 + 0.5) / f, ((y + self.shift.1) as f32 + 0.5) / f)
+ }
+}
+
+/// Level `k` of [`LEVELS`] (radius `k / LEVELS · r` pixels): the disc blur of the pixels at least
+/// that far out of focus.
+fn level(img: &Rgb32f, coc: &[f32], k: usize, r: f32, gain: &(impl Fn([f32; 3]) -> f32 + Sync), origin: (usize, usize)) -> Level {
+ let (w, h) = (img.width, img.height);
+ let radius = r * k as f32 / LEVELS as f32;
+ let (f, shift, lw, lh) = level_grid(w, h, radius, origin);
+ let (grid_w, grid_h) = (w.saturating_add(shift.0), h.saturating_add(shift.1));
+ // a pixel counts in this level when its own circle of confusion reaches it
+ let (lo, hi) = ((k - 1) as f32 / LEVELS as f32, k as f32 / LEVELS as f32);
+ let mut cells = vec![[0f32; 4]; lw * lh];
+ crate::for_rows(&mut cells, lw, |cy, row| {
+ for (cx, cell) in row.iter_mut().enumerate() {
+ let mut acc = [0f32; 4];
+ for vy in cy * f..((cy + 1) * f).min(grid_h) {
+ for vx in cx * f..((cx + 1) * f).min(grid_w) {
+ let (Some(x), Some(y)) = (vx.checked_sub(shift.0), vy.checked_sub(shift.1)) else { continue };
+ let i = y * w + x;
+ let (Some(c), Some(d)) = (img.data.get(i), coc.get(i)) else { continue };
+ // (a non-finite pixel would spread over the whole disc: left out)
+ if !c.iter().all(|v| v.is_finite()) {
+ continue;
+ }
+ let wt = FOCUS_WEIGHT + smooth(lo, hi, *d);
+ let g = gain(*c) * wt;
+ acc = [acc[0] + c[0] * g, acc[1] + c[1] * g, acc[2] + c[2] * g, acc[3] + wt];
+ }
+ }
+ *cell = acc;
+ }
+ });
+ Level { img: disc(&cells, lw, lh, radius / f as f32), f, shift }
+}
+
+/// Grid scale and dimensions for one disc level, bounded even when a window's grid is offset.
+fn level_grid(w: usize, h: usize, radius: f32, origin: (usize, usize)) -> (usize, (usize, usize), usize, usize) {
+ let at_least = ((w as f64 * h as f64) / MAX_CELLS as f64).sqrt().ceil() as usize;
+ let max_edge = w.max(h).max(1);
+ let mut f = ((radius / DISC_CELLS).ceil() as usize).max(at_least).clamp(1, max_edge);
+ loop {
+ let shift = (origin.0 % f, origin.1 % f);
+ let (lw, lh) = (w.saturating_add(shift.0).div_ceil(f), h.saturating_add(shift.1).div_ceil(f));
+ if lw.saturating_mul(lh) <= MAX_CELLS || f == max_edge {
+ return (f, shift, lw, lh);
+ }
+ f = f.saturating_add(1);
+ }
+}
+
+/// The normalized disc blur of radius `rho` cells of weighted sums `cells` (`[r·w, g·w, b·w, w]`,
+/// `lw × lh`): per row of the disc, a span read from that row's prefix sums, its two end cells
+/// weighted by the fraction of them inside.
+fn disc(cells: &[[f32; 4]], lw: usize, lh: usize, rho: f32) -> Rgb32f {
+ let stride = lw + 1;
+ let mut pre = vec![[0f64; 4]; stride * lh];
+ crate::for_rows(&mut pre, stride, |y, p| {
+ let Some(row) = cells.get(y * lw..(y + 1) * lw) else { return };
+ let mut acc = [0f64; 4];
+ for (x, c) in row.iter().enumerate() {
+ for ch in 0..4 {
+ acc[ch] += c[ch] as f64;
+ }
+ if let Some(slot) = p.get_mut(x + 1) {
+ *slot = acc;
+ }
+ }
+ });
+ // (a disc past the grid reads nothing more)
+ let rho = if rho.is_finite() { rho.clamp(0.0, lw.max(lh) as f32 + 1.0) } else { 0.0 };
+ let n = rho.ceil() as isize;
+ let spans: Vec<(isize, isize, f64)> = (-n..=n)
+ .filter_map(|dy| {
+ let q = rho * rho - (dy * dy) as f32;
+ (q >= 0.0).then(|| {
+ let hw = q.sqrt();
+ (dy, hw.floor() as isize, (hw - hw.floor()) as f64)
+ })
+ })
+ .collect();
+ let mut out = Rgb32f::new(lw, lh);
+ let (lwi, lhi) = (lw as isize, lh as isize);
+ crate::for_rows(&mut out.data, lw, |y, row| {
+ for (x, px) in row.iter_mut().enumerate() {
+ let x = x as isize;
+ let mut acc = [0f64; 4];
+ for &(dy, hw, fr) in &spans {
+ let yy = y as isize + dy;
+ if yy < 0 || yy >= lhi {
+ continue;
+ }
+ let base = yy as usize * stride;
+ let (a, b) = ((x - hw).max(0), (x + hw + 1).min(lwi));
+ if a < b
+ && let (Some(pa), Some(pb)) = (pre.get(base + a as usize), pre.get(base + b as usize))
+ {
+ for ch in 0..4 {
+ acc[ch] += pb[ch] - pa[ch];
+ }
+ }
+ if fr > 0.0 {
+ for xx in [x - hw - 1, x + hw + 1] {
+ if xx >= 0
+ && xx < lwi
+ && let Some(c) = cells.get(yy as usize * lw + xx as usize)
+ {
+ for ch in 0..4 {
+ acc[ch] += fr * c[ch] as f64;
+ }
+ }
+ }
+ }
+ }
+ *px = if acc[3] > 0.0 { std::array::from_fn(|c| (acc[c] / acc[3]) as f32) } else { [0.0; 3] };
+ }
+ });
+ out
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use lightcraft_develop::{MaskComponent, MaskOp};
+ use lightcraft_geom::Point;
+
+ /// Settings with the lens blur switched on at `amount`.
+ fn on(amount: f64) -> DevelopSettings {
+ let mut s = DevelopSettings::default();
+ s.lens_blur.enabled = true;
+ s.lens_blur.amount = amount;
+ s
+ }
+
+ /// A frame for a `w × h` source rendered at its own size.
+ fn frame(s: &DevelopSettings, w: usize, h: usize) -> Frame {
+ Frame::new(w, h, s, true)
+ }
+
+ /// A radial mask around the centre (radius `r`, long-edge units).
+ fn centre_mask(id: u32, r: f64) -> Mask {
+ let mut m = Mask { id, ..Mask::default() };
+ m.components.push(MaskComponent {
+ name: None,
+ op: MaskOp::Add,
+ invert: false,
+ shape: MaskShape::Radial { center: Point::new(0.5, 0.5), rx: r, ry: r, angle: 0.0, feather: 0.0, invert: false },
+ });
+ m
+ }
+
+ /// A black image with one bright point at `(px, py)` and white stripes in the left third.
+ fn scene(w: usize, h: usize, px: usize, py: usize) -> Rgb32f {
+ let mut img = Rgb32f::from_fn(w, h, |x, _| if x < w / 3 && x % 4 < 2 { [0.6; 3] } else { [0.02; 3] });
+ img.set(px, py, [4.0; 3]);
+ img
+ }
+
+ /// On a 60 MP export, the strongest global and per-mask settings keep a fixed-size disc kernel
+ /// and every level's coarse grid within its cell budget, including an offset render window.
+ #[test]
+ fn large_image_blur_work_is_bounded() {
+ let (w, h) = (9500, 6316);
+ let long_edge = w.max(h) as f64;
+ let s = on(100.0);
+ let r = radius_px(&s, long_edge);
+ assert!((r - (MAX_RADIUS * long_edge) as f32).abs() < 1e-3);
+
+ let mut mask = Mask::default();
+ mask.adjust.lens_blur = 100.0;
+ assert!((mask_radius_px(&mask, long_edge) - r).abs() < 1e-3);
+
+ let mut total_cells = 0usize;
+ for k in 1..=LEVELS {
+ let radius = r * k as f32 / LEVELS as f32;
+ let (f, _, lw, lh) = level_grid(w, h, radius, (17, 29));
+ let cells = lw.saturating_mul(lh);
+ assert!(cells <= MAX_CELLS, "level {k}: {lw} × {lh} = {cells}");
+ total_cells = total_cells.saturating_add(cells);
+ assert!(radius / f as f32 <= DISC_CELLS);
+ }
+ assert!(total_cells <= LEVELS * MAX_CELLS);
+
+ let (_, _, lw, lh) = level_grid(w, h, r, (w - 1, h - 1));
+ assert!(lw.saturating_mul(lh) <= MAX_CELLS, "offset window: {lw} × {lh}");
+ }
+
+ /// Switched off, at Amount 0 or with the section's eye closed, the photo is untouched.
+ #[test]
+ fn off_amount_zero_or_section_hidden_changes_nothing() {
+ let img = scene(64, 48, 50, 10);
+ for s in [DevelopSettings::default(), on(0.0), {
+ let mut s = on(80.0);
+ s.set_section_enabled("lensBlur", false);
+ s
+ }] {
+ let mut out = img.clone();
+ apply(&mut out, &s, &frame(&s, 64, 48), 64.0, None);
+ assert_eq!(out, img);
+ assert_eq!(key(&s), "");
+ }
+ }
+
+ /// An out-of-focus point of light spreads into a disc of the expected radius and keeps its energy.
+ #[test]
+ fn out_of_focus_point_becomes_a_disc_and_keeps_its_energy() {
+ let (w, h) = (200, 200);
+ let mut s = on(100.0);
+ // in focus: a small disc in the middle; the point sits far outside it
+ s.masks.push(centre_mask(7, 0.05));
+ s.lens_blur.focus_mask = Some(7);
+ let mut img = Rgb32f::filled(w, h, [0.0; 3]);
+ img.set(40, 40, [100.0; 3]);
+ let before: f32 = img.data.iter().map(|c| c[1]).sum();
+ apply(&mut img, &s, &frame(&s, w, h), w as f64, None);
+ let r = radius_px(&s, w as f64);
+ assert!((r - 7.0).abs() < 1e-3, "{r}");
+ // spread over a disc: inside it lit, well outside it dark, the centre no longer a spike
+ let at = |x: usize, y: usize| img.get(x, y)[1];
+ assert!(at(40, 40) < 10.0, "{}", at(40, 40));
+ for (x, y) in [(44, 40), (40, 36), (37, 43)] {
+ assert!(at(x, y) > 0.2, "({x}, {y}) {}", at(x, y));
+ }
+ for (x, y) in [(52, 40), (40, 28), (50, 50)] {
+ assert!(at(x, y) < 0.05, "({x}, {y}) {}", at(x, y));
+ }
+ let after: f32 = img.data.iter().map(|c| c[1]).sum();
+ assert!((after - before).abs() / before < 0.15, "{before} → {after}");
+ }
+
+ /// What is in focus stays sharp, and its brightness doesn't bleed into the blurred background as a halo.
+ #[test]
+ fn the_focus_stays_sharp_and_does_not_bleed_into_the_background() {
+ let (w, h) = (160, 160);
+ let mut s = on(100.0);
+ s.masks.push(centre_mask(3, 0.2));
+ s.lens_blur.focus_mask = Some(3);
+ // a bright subject filling the focus (a disc of 30 px; the focus is 32 px), grey stripes
+ // around it
+ let mut img = Rgb32f::from_fn(w, h, |x, y| {
+ let d = ((x as f32 + 0.5 - 80.0).powi(2) + (y as f32 + 0.5 - 80.0).powi(2)).sqrt();
+ if d < 30.0 {
+ [2.0; 3]
+ } else if x % 4 < 2 {
+ [0.3; 3]
+ } else {
+ [0.1; 3]
+ }
+ });
+ let src = img.clone();
+ apply(&mut img, &s, &frame(&s, w, h), w as f64, None);
+ // the subject is untouched
+ assert_eq!(img.get(80, 80), src.get(80, 80));
+ assert_eq!(img.get(60, 70), src.get(60, 70));
+ // the background's stripes are smoothed out…
+ let row: Vec = (4..40).map(|x| img.get(x, 10)[1]).collect();
+ let (lo, hi) = row.iter().fold((f32::MAX, f32::MIN), |(a, b), v| (a.min(*v), b.max(*v)));
+ assert!(hi - lo < 0.04, "{lo}..{hi}");
+ // …to their mean, also 5 px from the bright subject (within the blur's reach): no halo
+ let near = img.get(45, 80)[1];
+ assert!((near - 0.2).abs() < 0.04, "{near}");
+ }
+
+ /// Boost brightens the blurred highlights only: a mid-grey background stays as it is.
+ #[test]
+ fn boost_brightens_bokeh_only() {
+ let (w, h) = (120, 120);
+ let mut s = on(60.0);
+ s.masks.push(centre_mask(1, 0.05));
+ s.lens_blur.focus_mask = Some(1);
+ let mut img = Rgb32f::filled(w, h, [0.1; 3]);
+ img.set(20, 20, [8.0; 3]);
+ let (mut plain, mut boosted) = (img.clone(), img.clone());
+ apply(&mut plain, &s, &frame(&s, w, h), w as f64, None);
+ s.lens_blur.boost = 100.0;
+ apply(&mut boosted, &s, &frame(&s, w, h), w as f64, None);
+ assert!(boosted.get(22, 20)[0] > plain.get(22, 20)[0] * 2.0, "{} vs {}", boosted.get(22, 20)[0], plain.get(22, 20)[0]);
+ // a mid-grey background stays as it is
+ assert!((boosted.get(100, 100)[0] - plain.get(100, 100)[0]).abs() < 1e-4);
+ }
+
+ /// A small preview and a full-size export blur alike (the radius is relative to the long edge).
+ #[test]
+ fn previews_and_exports_blur_alike() {
+ // the same photo at two sizes: the blur scales with the long edge
+ let s = on(100.0);
+ let big = scene(240, 180, 200, 30);
+ let small = Rgb32f::from_fn(120, 90, |x, y| {
+ let p = |dx, dy| big.get(2 * x + dx, 2 * y + dy);
+ let q = [p(0, 0), p(1, 0), p(0, 1), p(1, 1)];
+ std::array::from_fn(|c| q.iter().map(|v| v[c]).sum::() / 4.0)
+ });
+ let (mut a, mut b) = (big.clone(), small.clone());
+ apply(&mut a, &s, &frame(&s, 240, 180), 240.0, None);
+ apply(&mut b, &s, &frame(&s, 120, 90), 120.0, None);
+ let mut diff = 0.0;
+ for y in 0..90 {
+ for x in 0..120 {
+ let d = a.get(2 * x, 2 * y)[1] - b.get(x, y)[1];
+ diff += d.abs();
+ }
+ }
+ let mean = diff / (120.0 * 90.0);
+ assert!(mean < 0.02, "{mean}");
+ }
+
+ /// Brush Refinement: a Focus stroke keeps what it covers sharp, a Blur stroke blurs what it
+ /// covers even inside the focus.
+ #[test]
+ fn focus_and_blur_brushes_refine_the_focus() {
+ let (w, h) = (160, 120);
+ let img = Rgb32f::from_fn(w, h, |x, _| if x % 4 < 2 { [0.3; 3] } else { [0.1; 3] });
+ let mut s = on(100.0);
+ // in focus: the middle
+ s.masks.push(centre_mask(5, 0.15));
+ s.lens_blur.focus_mask = Some(5);
+ let dab = |x: f64, y: f64| BrushStroke { points: vec![Point::new(x, y)], size: 0.08, feather: 0.0, ..BrushStroke::default() };
+ // Focus brush on the top-left corner (out of focus otherwise), Blur brush on the centre
+ s.lens_blur.focus_strokes.push(dab(0.1, 0.12));
+ s.lens_blur.blur_strokes.push(dab(0.5, 0.5));
+ let mut out = img.clone();
+ apply(&mut out, &s, &frame(&s, w, h), w as f64, None);
+ let stripes = |o: &Rgb32f, cx: usize, cy: usize| {
+ let v: Vec = (cx - 4..cx + 4).map(|x| o.get(x, cy)[1]).collect();
+ v.iter().fold(f32::MIN, |a, b| a.max(*b)) - v.iter().fold(f32::MAX, |a, b| a.min(*b))
+ };
+ // the corner the Focus brush painted keeps its stripes; one it didn't is smooth
+ assert!(stripes(&out, 16, 14) > 0.15, "{}", stripes(&out, 16, 14));
+ assert!(stripes(&out, 140, 100) < 0.05, "{}", stripes(&out, 140, 100));
+ // the centre is in focus, but the Blur brush wins
+ assert!(stripes(&out, 80, 60) < 0.05, "{}", stripes(&out, 80, 60));
+ // without the strokes, the centre stays sharp
+ let mut plain = s.clone();
+ plain.lens_blur.focus_strokes.clear();
+ plain.lens_blur.blur_strokes.clear();
+ let mut out = img.clone();
+ apply(&mut out, &plain, &frame(&plain, w, h), w as f64, None);
+ assert!(stripes(&out, 80, 60) > 0.15);
+ assert_ne!(key(&plain), key(&s));
+ }
+
+ /// Without a focus mask, a fixed shape keeps a circle, a wide or a tall oval sharp in the
+ /// middle: a wide one reaches farther sideways than up, a tall one the other way round.
+ #[test]
+ fn fixed_focus_shapes() {
+ use lightcraft_develop::FocusShape;
+ let (w, h) = (300, 300);
+ let s0 = on(100.0);
+ let fr = frame(&s0, w, h);
+ let img = Rgb32f::filled(w, h, [0.2; 3]);
+ let alpha = |shape: FocusShape| {
+ let mut s = s0.clone();
+ s.lens_blur.shape = shape;
+ focus(&img, &s, &fr, None)
+ };
+ // sideways and upwards from the centre, 0.2 of the long edge away
+ let side = |a: &Plane| a.get(150 + 60, 150);
+ let up = |a: &Plane| a.get(150, 150 - 60);
+ let (round, wide, tall) = (alpha(FocusShape::Round), alpha(FocusShape::Wide), alpha(FocusShape::Tall));
+ for a in [&round, &wide, &tall] {
+ assert!(a.get(150, 150) > 0.99, "the middle is in focus");
+ assert!(a.get(5, 5) < 0.01, "a corner isn't");
+ }
+ assert!((side(&round) - up(&round)).abs() < 0.05);
+ assert!(side(&wide) > 0.9 && up(&wide) < 0.1, "{} {}", side(&wide), up(&wide));
+ assert!(up(&tall) > 0.9 && side(&tall) < 0.1, "{} {}", side(&tall), up(&tall));
+ // moved and resized: the area follows; hostile values are tamed
+ let mut s = s0.clone();
+ s.lens_blur.shape = FocusShape::Round;
+ s.lens_blur.area = Some(lightcraft_develop::FocusArea { center: Point::new(0.2, 0.2), rx: 0.1, ry: 0.1 });
+ let a = focus(&img, &s, &fr, None);
+ assert!(a.get(60, 60) > 0.99 && a.get(150, 150) < 0.01, "moved to the top left");
+ s.lens_blur.area = Some(lightcraft_develop::FocusArea { center: Point::new(f64::NAN, 9e9), rx: f64::INFINITY, ry: -3.0 });
+ let a = focus(&img, &s, &fr, None);
+ assert!(a.data.iter().all(|v| v.is_finite()));
+ // an unknown shape in the JSON reads as Auto (never an error)
+ let b: lightcraft_develop::LensBlur = serde_json::from_value(serde_json::json!({"shape": "hexagon"})).unwrap();
+ assert_eq!(b.shape, FocusShape::Auto);
+ }
+
+ /// With a Transition the blur grows with the distance from the focus, as a lens renders what
+ /// is near the subject: soft right beside it, full far away. At 0 it is a sharp cut.
+ #[test]
+ fn the_blur_grows_away_from_the_focus() {
+ let (w, h) = (400, 200);
+ // wide stripes: how much of them a blur leaves says how large it is
+ let img = Rgb32f::from_fn(w, h, |x, _| if (x / 12) % 2 == 0 { [0.3; 3] } else { [0.1; 3] });
+ let mut s = on(100.0);
+ // in focus: a disc left of centre
+ let mut m = centre_mask(2, 0.08);
+ if let MaskShape::Radial { center, .. } = &mut m.components[0].shape {
+ *center = Point::new(0.2, 0.5);
+ }
+ s.masks.push(m);
+ s.lens_blur.focus_mask = Some(2);
+ // how much detail is left (stripe contrast) around x, on the middle row
+ let detail = |o: &Rgb32f, x: usize| {
+ let v: Vec = (x - 12..x + 12).map(|x| o.get(x, 100)[1]).collect();
+ v.iter().fold(f32::MIN, |a, b| a.max(*b)) - v.iter().fold(f32::MAX, |a, b| a.min(*b))
+ };
+ let render = |t: f64| {
+ let mut s = s.clone();
+ s.lens_blur.transition = t;
+ let mut o = img.clone();
+ apply(&mut o, &s, &frame(&s, w, h), w as f64, None);
+ o
+ };
+ let (soft, cut) = (render(100.0), render(0.0));
+ // the focus disc ends at x ≈ 112 (the ramp is 60 px long): just past it, then farther
+ let (near, mid, far) = (detail(&soft, 130), detail(&soft, 150), detail(&soft, 330));
+ assert!(near > mid + 0.02 && mid > far + 0.02, "{near} > {mid} > {far}");
+ // a sharp cut blurs right beside the focus as much as far away
+ let cut_near = detail(&cut, 130);
+ assert!((cut_near - detail(&cut, 330)).abs() < 0.02 && near > cut_near + 0.05, "{cut_near} vs {near}");
+ // the focus itself stays sharp either way
+ assert!(detail(&soft, 70) > 0.19 && detail(&cut, 70) > 0.19);
+ }
+
+ /// A focus mask whose AI selection isn't computed yet (Find Subject still running) leaves the
+ /// subject estimate in focus: the photo is never blurred all over meanwhile.
+ #[test]
+ fn a_pending_ai_focus_mask_falls_back_to_the_subject_estimate() {
+ let (w, h) = (160, 120);
+ let img = Rgb32f::from_fn(w, h, |x, _| if x % 4 < 2 { [0.3; 3] } else { [0.1; 3] });
+ let mut s = on(100.0);
+ let mut m = Mask { id: 9, ..Mask::default() };
+ let pending = MaskShape::Object { hint: vec![], exclude: vec![], seg: None, detail: vec![], edge: 0.0 };
+ m.components.push(MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: pending });
+ assert!(awaiting_ai(&m));
+ s.masks.push(m);
+ s.lens_blur.focus_mask = Some(9);
+ let mut subject = s.clone();
+ subject.lens_blur.focus_mask = None;
+ let (mut a, mut b) = (img.clone(), img.clone());
+ apply(&mut a, &s, &frame(&s, w, h), w as f64, None);
+ apply(&mut b, &subject, &frame(&subject, w, h), w as f64, None);
+ assert_eq!(a, b, "the same as no focus mask");
+ // the middle (the estimate's subject) keeps its stripes
+ let v: Vec = (76..84).map(|x| a.get(x, 60)[1]).collect();
+ assert!(v.iter().fold(f32::MIN, |p, q| p.max(*q)) - v.iter().fold(f32::MAX, |p, q| p.min(*q)) > 0.1, "{v:?}");
+ }
+
+ /// A mask with Lens Blur blurs what it selects and nothing else; hidden, it blurs nothing.
+ #[test]
+ fn a_mask_blurs_only_what_it_selects() {
+ let (w, h) = (160, 120);
+ // vertical stripes everywhere
+ let img = Rgb32f::from_fn(w, h, |x, _| if x % 4 < 2 { [0.3; 3] } else { [0.1; 3] });
+ let mut s = DevelopSettings::default();
+ // the left half (a linear gradient with a short ramp in the middle)
+ let mut m = Mask { id: 2, ..Mask::default() };
+ m.components.push(MaskComponent {
+ name: None,
+ op: MaskOp::Add,
+ invert: false,
+ shape: MaskShape::Linear { start: Point::new(0.45, 0.5), end: Point::new(0.55, 0.5) },
+ });
+ m.adjust.lens_blur = 100.0;
+ s.masks.push(m);
+ assert!(renders(&s) && !active(&s) && !key(&s).is_empty());
+ let mut out = img.clone();
+ apply(&mut out, &s, &frame(&s, w, h), w as f64, None);
+ // left: the stripes are smoothed to their mean
+ let left: Vec = (8..50).map(|x| out.get(x, 60)[1]).collect();
+ let (lo, hi) = left.iter().fold((f32::MAX, f32::MIN), |(a, b), v| (a.min(*v), b.max(*v)));
+ assert!(hi - lo < 0.05 && (lo + hi) / 2.0 > 0.15 && (lo + hi) / 2.0 < 0.25, "{lo}..{hi}");
+ // right: untouched
+ for x in 100..150 {
+ assert_eq!(out.get(x, 60), img.get(x, 60), "x {x}");
+ }
+ // the window grows by the mask's blur
+ let win = crate::PixelWindow { x: 60, y: 40, w: 20, h: 20 };
+ assert!(window_for_reads(&s, w, h, w as f64, win).w > 20);
+ // hidden or at 0: nothing
+ for change in [|m: &mut Mask| m.visible = false, |m: &mut Mask| m.adjust.lens_blur = 0.0, |m: &mut Mask| m.adjust.lens_blur_off = true] {
+ let mut t = s.clone();
+ change(&mut t.masks[0]);
+ let mut out = img.clone();
+ apply(&mut out, &t, &frame(&t, w, h), w as f64, None);
+ assert_eq!(out, img);
+ assert!(!renders(&t) && key(&t).is_empty());
+ }
+ }
+
+ /// A mask's Lens Blur block can blur what the mask leaves out, and its Transition makes that
+ /// blur grow away from what stays sharp.
+ #[test]
+ fn a_mask_blurs_its_outside_gradually() {
+ let (w, h) = (320, 120);
+ let img = Rgb32f::from_fn(w, h, |x, _| if x % 4 < 2 { [0.3; 3] } else { [0.1; 3] });
+ let mut s = DevelopSettings::default();
+ // the left third selected
+ let mut m = Mask { id: 2, ..Mask::default() };
+ m.components.push(MaskComponent {
+ name: None,
+ op: MaskOp::Add,
+ invert: false,
+ shape: MaskShape::Linear { start: Point::new(0.30, 0.5), end: Point::new(0.34, 0.5) },
+ });
+ m.adjust.lens_blur = 100.0;
+ m.adjust.lens_blur_outside = true;
+ s.masks.push(m);
+ // how much of the stripes is left around column x
+ let spread = |out: &Rgb32f, x: usize| {
+ let v: Vec = (x..x + 8).map(|x| out.get(x, 60)[1]).collect();
+ v.iter().fold(f32::MIN, |a, b| a.max(*b)) - v.iter().fold(f32::MAX, |a, b| a.min(*b))
+ };
+ let mut sharp_cut = img.clone();
+ apply(&mut sharp_cut, &s, &frame(&s, w, h), w as f64, None);
+ // the selection stays sharp, the rest is blurred
+ for x in 8..80 {
+ assert_eq!(sharp_cut.get(x, 60), img.get(x, 60), "x {x}");
+ }
+ assert!(spread(&sharp_cut, 150) < 0.05 && spread(&sharp_cut, 290) < 0.05);
+ // with a Transition, near the selection keeps more of the stripes than far from it
+ s.masks[0].adjust.lens_blur_transition = 100.0;
+ s.masks[0].adjust.lens_blur_boost = 50.0;
+ let mut gradual = img.clone();
+ apply(&mut gradual, &s, &frame(&s, w, h), w as f64, None);
+ assert!(spread(&gradual, 112) > spread(&sharp_cut, 112) + 0.02, "{} vs {}", spread(&gradual, 112), spread(&sharp_cut, 112));
+ assert!(spread(&gradual, 112) > spread(&gradual, 290), "{} vs {}", spread(&gradual, 112), spread(&gradual, 290));
+ // the window reads farther with the transition
+ let win = crate::PixelWindow { x: 200, y: 40, w: 20, h: 20 };
+ let mut no_t = s.clone();
+ no_t.masks[0].adjust.lens_blur_transition = 0.0;
+ assert!(window_for_reads(&s, w, h, w as f64, win).w > window_for_reads(&no_t, w, h, w as f64, win).w);
+ // at Amount 0 the outside isn't blurred either
+ s.masks[0].adjust.amount = 0.0;
+ let mut out = img.clone();
+ apply(&mut out, &s, &frame(&s, w, h), w as f64, None);
+ assert_eq!(out, img);
+ // hostile values
+ s.masks[0].adjust.amount = f64::NAN;
+ s.masks[0].adjust.lens_blur_transition = f64::INFINITY;
+ s.masks[0].adjust.lens_blur_boost = f64::NAN;
+ let mut out = img.clone();
+ apply(&mut out, &s, &frame(&s, w, h), w as f64, None);
+ assert!(out.data.iter().all(|p| p.iter().all(|v| v.is_finite())));
+ }
+
+ /// The subject point lands on a person matte when the photo carries one, and is always inside
+ /// the photo; an empty source has none.
+ #[test]
+ fn subject_point_finds_the_person_matte() {
+ use crate::masks::{MatteKind, Mattes};
+ let s = DevelopSettings::default();
+ let src = Rgb32f::filled(300, 200, [0.2; 3]);
+ // a person in the top-right corner, far from the centre the heuristic favours
+ let mut person = lightcraft_raster::Image::::new(300, 200);
+ for y in 10..60 {
+ for x in 220..290 {
+ person.set(x, y, 255);
+ }
+ }
+ let mut mattes = Mattes::default();
+ mattes.push(MatteKind::Person, person);
+ let p = subject_point(&src, &s, Some(&mattes)).unwrap();
+ assert!(p.x > 0.72 && p.x < 0.97 && p.y > 0.04 && p.y < 0.3, "{p:?}");
+ // without mattes: somewhere in the photo (the heuristic's centre-weighted peak)
+ let p = subject_point(&src, &s, None).unwrap();
+ assert!((0.0..=1.0).contains(&p.x) && (0.0..=1.0).contains(&p.y), "{p:?}");
+ assert!(subject_point(&Rgb32f::new(0, 0), &s, None).is_none());
+ }
+
+ /// Hostile settings and images: no panic, no NaN.
+ #[test]
+ fn hostile_input_never_panics() {
+ for amount in [f64::NAN, f64::INFINITY, -5.0, 1e9] {
+ let mut s = on(amount);
+ s.lens_blur.boost = f64::NAN;
+ s.lens_blur.focus_mask = Some(99);
+ for (w, h) in [(1, 1), (1, 7), (9, 1), (33, 5)] {
+ let mut img = Rgb32f::filled(w, h, [0.5, f32::NAN, f32::INFINITY]);
+ apply(&mut img, &s, &frame(&s, w, h), 1e6, None);
+ let mut img = Rgb32f::filled(w, h, [0.5; 3]);
+ apply(&mut img, &s, &frame(&s, w, h), w.max(h) as f64, None);
+ assert!(img.data.iter().flatten().all(|v| v.is_finite()), "{amount} {w}×{h}");
+ }
+ }
+ // a mismatched buffer is left alone
+ let s = on(50.0);
+ let mut img = Rgb32f { width: 10, height: 10, data: vec![[0.5; 3]; 7] };
+ apply(&mut img, &s, &frame(&s, 10, 10), 10.0, None);
+ assert_eq!(img.data.len(), 7);
+ }
+}
diff --git a/crates/pipeline/src/lib.rs b/crates/pipeline/src/lib.rs
index 34bd402336..ef2d2359ca 100644
--- a/crates/pipeline/src/lib.rs
+++ b/crates/pipeline/src/lib.rs
@@ -32,6 +32,7 @@ pub mod curve;
pub mod dust;
pub mod finish;
pub mod geometry;
+pub mod lens_blur;
pub mod light_v3;
// generated (`light_v3_params.rs`), kept in its own compact layout
#[rustfmt::skip]
@@ -444,6 +445,8 @@ pub struct Plan<'a> {
pub keep: Option,
/// The source's segmentation mattes ([`SourceInfo::mattes`]).
pub mattes: Option>,
+ /// The whole photo's subject estimate a lens blur focuses on ([`lens_blur::subject_estimate`]).
+ pub lens_subject: Option>,
/// The whole frame and its output size (a windowed render's [`Self::frame`] is part of it).
pub full: (geometry::Frame, usize, usize),
}
@@ -487,8 +490,10 @@ pub fn plan<'a>(src: &Rgb32f, info: &SourceInfo, s: &'a DevelopSettings, req: &R
let (pw, ph) = (((full_w as f64 * k).round() as usize).max(1), ((full_h as f64 * k).round() as usize).max(1));
fixed_air = Some(local::frame_airlight(src, info, s, &frame, pw, ph));
}
- // (spots grow the rendered window of 8-bit renders; a grown window is cut back to the request)
- let mut work = if req.depth == OutputDepth::U8 { spots::window_for_reads(s, &frame, full_w, full_h, px_per_long, win) } else { win };
+ // (spots grow the rendered window of 8-bit renders, the lens blur that of every
+ // render; a grown window is cut back to the request)
+ let work = if req.depth == OutputDepth::U8 { spots::window_for_reads(s, &frame, full_w, full_h, px_per_long, win) } else { win };
+ let mut work = lens_blur::window_for_reads(s, full_w, full_h, px_per_long, work);
if local::joint_colour_nr(s)
&& let Some(params) = local::nrc_params(s, detail_scale)
{
@@ -522,6 +527,8 @@ pub fn plan<'a>(src: &Rgb32f, info: &SourceInfo, s: &'a DevelopSettings, req: &R
// and the local tone mapping
local_tone::enabled(info, s),
src_long,
+ // and the lens blur (after noise reduction, but a draft must not reuse planes without it)
+ lens_blur::key(s),
));
let lin_key = hash_of((
pre_nr_key,
@@ -530,6 +537,7 @@ pub fn plan<'a>(src: &Rgb32f, info: &SourceInfo, s: &'a DevelopSettings, req: &R
// the process picks the colour filter (Update to Current Process, undo)
local::joint_colour_nr(s),
));
+ let lens_subject = lens_blur::subject_estimate(src, s, info.mattes.as_deref()).map(Arc::new);
let noise_key = hash_of((pre_nr_key, [d.nr_luminance, d.nr_detail].map(f64::to_bits), detail_scale.to_bits()));
Plan {
settings,
@@ -548,6 +556,7 @@ pub fn plan<'a>(src: &Rgb32f, info: &SourceInfo, s: &'a DevelopSettings, req: &R
fixed_air,
keep,
mattes: info.mattes.clone(),
+ lens_subject,
full,
}
}
@@ -585,6 +594,7 @@ pub fn colour_noise(src: &Rgb32f, info: &SourceInfo, p: &Plan<'_>) -> Option, info: &SourceInfo, s: &DevelopSettings, req: &Rende
colour_noise = self::colour_noise(src_img, info, &plan).map(|n| (plan.noise_key, n));
}
local::denoise(&mut img, s, detail_scale, colour_noise.as_ref().map(|c| &c.1));
+ lens_blur::apply_with(&mut img, s, frame, px_per_long, plan.mattes.as_deref(), plan.lens_subject.as_deref());
Arc::new(img)
}
};
diff --git a/crates/pipeline/src/masks.rs b/crates/pipeline/src/masks.rs
index b40e3dcd97..dfbf00326a 100644
--- a/crates/pipeline/src/masks.rs
+++ b/crates/pipeline/src/masks.rs
@@ -93,6 +93,12 @@ impl Mattes {
self.mattes.iter().filter(|(k, _)| kinds.contains(k)).map(|(_, m)| m).collect()
}
+ /// Whether a mask of the photo's Subject comes from these mattes (a person, or a person's
+ /// parts) rather than from the saliency heuristic.
+ pub fn selects_subject(&self) -> bool {
+ !self.for_shape(&MaskShape::Subject).is_empty()
+ }
+
/// The mattes a shape uses instead of its heuristic (empty: none fits).
fn for_shape(&self, shape: &MaskShape) -> Vec<&Image> {
use MatteKind::*;
diff --git a/crates/pipeline/src/output.rs b/crates/pipeline/src/output.rs
index b514c25503..c3fa176eb6 100644
--- a/crates/pipeline/src/output.rs
+++ b/crates/pipeline/src/output.rs
@@ -346,6 +346,30 @@ pub struct DeepImage {
}
impl DeepImage {
+ /// The `w × h` part of the image at (`x`, `y`), clamped to the image (at least 1 × 1 when the
+ /// image has pixels). Samples are RGB, three per pixel, row-major.
+ pub fn crop(&self, x: usize, y: usize, w: usize, h: usize) -> DeepImage {
+ let x = x.min(self.width.saturating_sub(1));
+ let y = y.min(self.height.saturating_sub(1));
+ let w = w.min(self.width.saturating_sub(x)).max(usize::from(self.width > 0));
+ let h = h.min(self.height.saturating_sub(y)).max(usize::from(self.height > 0));
+ fn cut(v: &[T], width: usize, x: usize, y: usize, w: usize, h: usize) -> Vec {
+ let mut out = Vec::with_capacity(w.saturating_mul(h).saturating_mul(3));
+ for row in y..y + h {
+ let start = row.saturating_mul(width).saturating_add(x).saturating_mul(3);
+ if let Some(r) = v.get(start..start.saturating_add(w * 3)) {
+ out.extend_from_slice(r);
+ }
+ }
+ out
+ }
+ let samples = match &self.samples {
+ DeepSamples::U16(v) => DeepSamples::U16(cut(v, self.width, x, y, w, h)),
+ DeepSamples::F32(v) => DeepSamples::F32(cut(v, self.width, x, y, w, h)),
+ };
+ DeepImage { width: w, height: h, space: self.space, samples }
+ }
+
/// The same image reduced to 8 bits, display-encoded with the space's curve.
pub fn to_rgba8(&self) -> lightcraft_raster::Rgba8 {
let mut out = lightcraft_raster::Rgba8::new(self.width, self.height);
@@ -364,27 +388,6 @@ impl DeepImage {
}
out
}
-
- /// Crop to `[x0, x0+w) × [y0, y0+h)` (clamped to the image).
- pub fn crop(&self, x0: usize, y0: usize, w: usize, h: usize) -> Self {
- let x0 = x0.min(self.width);
- let y0 = y0.min(self.height);
- let w = w.min(self.width - x0);
- let h = h.min(self.height - y0);
- fn rows(v: &[T], width: usize, x0: usize, y0: usize, w: usize, h: usize) -> Vec {
- let mut out = Vec::with_capacity(3 * w * h);
- for y in y0..y0 + h {
- let at = 3 * (y * width + x0);
- out.extend_from_slice(v.get(at..at + 3 * w).unwrap_or_default());
- }
- out
- }
- let samples = match &self.samples {
- DeepSamples::U16(v) => DeepSamples::U16(rows(v, self.width, x0, y0, w, h)),
- DeepSamples::F32(v) => DeepSamples::F32(rows(v, self.width, x0, y0, w, h)),
- };
- Self { width: w, height: h, space: self.space, samples }
- }
}
trait IntoRgba {
@@ -401,6 +404,23 @@ impl IntoRgba for [u8; 3] {
mod tests {
use super::*;
+ /// A crop keeps the right pixels and never reads past the image, whatever the request.
+ #[test]
+ fn deep_crop_keeps_the_window_and_clamps() {
+ // 4 × 3, pixel (x, y) holds x + 10 y in every channel
+ let v: Vec = (0..3).flat_map(|y| (0..4).flat_map(move |x| [x + 10 * y; 3])).collect();
+ let d = DeepImage { width: 4, height: 3, space: OutputSpace::Srgb, samples: DeepSamples::U16(v) };
+ let c = d.crop(1, 1, 2, 2);
+ assert_eq!((c.width, c.height), (2, 2));
+ assert_eq!(c.samples, DeepSamples::U16(vec![11, 11, 11, 12, 12, 12, 21, 21, 21, 22, 22, 22]));
+ for (x, y, w, h) in [(9, 9, 5, 5), (0, 0, usize::MAX, usize::MAX), (3, 2, 0, 0)] {
+ let c = d.crop(x, y, w, h);
+ let DeepSamples::U16(s) = &c.samples else { panic!("a U16 image crops to U16") };
+ assert!(c.width >= 1 && c.height >= 1 && c.width <= 4 && c.height <= 3);
+ assert_eq!(s.len(), c.width * c.height * 3, "{x} {y} {w} {h}");
+ }
+ }
+
#[test]
fn trcs_invert() {
for o in OutputSpace::ALL {
diff --git a/crates/pipeline/src/tests_window.rs b/crates/pipeline/src/tests_window.rs
index e5fb6ea801..7fa71a5f25 100644
--- a/crates/pipeline/src/tests_window.rs
+++ b/crates/pipeline/src/tests_window.rs
@@ -6,7 +6,7 @@ use lightcraft_develop::{DevelopSettings, controls};
use lightcraft_geom::{Orientation, Rect};
use lightcraft_raster::{Rgb32f, Rgba8};
-use crate::{PixelWindow, RenderRequest, SourceInfo, render};
+use crate::{OutputDepth, PixelWindow, RenderRequest, SourceInfo, render};
const W: usize = 960;
const H: usize = 640;
@@ -422,6 +422,73 @@ fn auto_mask_strokes_match_the_whole_render() {
check_noisy("auto mask", &s, PixelWindow { x: 400, y: 300, w: 300, h: 250 }, 2, 0.1);
}
+// Given a lens blur, a window (zoomed view) blurs right up to its edges as the whole render does:
+// it is rendered with the blur's reach around it
+#[test]
+fn lens_blur_matches_the_whole_render_up_to_the_window_edges() {
+ lens_blur_window_matches(OutputDepth::U8);
+ lens_blur_window_matches(OutputDepth::U16);
+}
+
+/// A high-bit-depth window grows for the lens blur too, and is cut back to the request.
+fn lens_blur_window_matches(depth: OutputDepth) {
+ use lightcraft_develop::{Mask, MaskComponent, MaskOp, MaskShape};
+ use lightcraft_geom::Point;
+ let radial = MaskShape::Radial { center: Point::new(0.5, 0.5), rx: 0.15, ry: 0.2, angle: 0.0, feather: 30.0, invert: false };
+ let mut s = DevelopSettings {
+ masks: vec![Mask {
+ id: 4,
+ components: vec![MaskComponent { name: None, op: MaskOp::Add, invert: false, shape: radial }],
+ ..Default::default()
+ }],
+ ..Default::default()
+ };
+ s.lens_blur.enabled = true;
+ s.lens_blur.amount = 80.0;
+ s.lens_blur.boost = 40.0;
+ s.lens_blur.focus_mask = Some(4);
+ let deep = |win: Option| {
+ let r = render(&scene(), &SourceInfo::default(), &s, &RenderRequest { window: win, depth, ..RenderRequest::fit(W, H) });
+ if let Some(d) = &r.deep {
+ assert_eq!((d.width, d.height), (r.image.width, r.image.height));
+ }
+ r.image
+ };
+ let (f, w) = (deep(None), deep(Some(MID)));
+ assert_eq!((w.width, w.height), (MID.w, MID.h));
+ let (max, mean) = compare_inset(&f, &w, MID, 0);
+ assert!(max <= 6 && mean <= 0.4, "max {max}, mean {mean:.3}");
+ // and the blur shows: the whole render differs from the one without it
+ let mut off = s.clone();
+ off.lens_blur.enabled = false;
+ let (_, mean) = compare_inset(&full(&off), &f, PixelWindow { x: 0, y: 0, w: f.width, h: f.height }, 0);
+ assert!(mean > 1.0, "{mean:.3}");
+}
+
+/// With the automatic subject focus (no focus mask, no fixed shape, no person matte) a window
+/// focuses where the whole render does: the estimate comes from the whole photo, not from the
+/// pixels the window holds.
+#[test]
+fn lens_blur_on_the_estimated_subject_matches_the_whole_render() {
+ let mut s = DevelopSettings::default();
+ s.lens_blur.enabled = true;
+ s.lens_blur.amount = 80.0;
+ // a sharp cut: the window then grows only by the blur's radius, less than the estimate reads
+ s.lens_blur.transition = 0.0;
+ for depth in [OutputDepth::U8, OutputDepth::U16] {
+ let r = |win: Option| {
+ render(&scene(), &SourceInfo::default(), &s, &RenderRequest { window: win, depth, ..RenderRequest::fit(W, H) }).image
+ };
+ // a window at the middle and one at an edge, where the estimate's neighbourhood is cut off
+ for win in [MID, PixelWindow { x: 0, y: 0, w: 300, h: 220 }] {
+ let (f, w) = (r(None), r(Some(win)));
+ assert_eq!((w.width, w.height), (win.w, win.h));
+ let (max, mean) = compare_inset(&f, &w, win, 0);
+ assert!(max <= 6 && mean <= 0.4, "{depth:?} {win:?}: max {max}, mean {mean:.3}");
+ }
+ }
+}
+
// Given the joint colour NR, a window is filtered with the whole frame's noise model: a quiet part of
// the frame does not make a window of its noisy part (or the other way round) smooth differently
#[test]
diff --git a/crates/segment/src/detector.rs b/crates/segment/src/detector.rs
index c302aafe76..3099af44af 100644
--- a/crates/segment/src/detector.rs
+++ b/crates/segment/src/detector.rs
@@ -333,10 +333,117 @@ pub fn merge(out: &Output, threshold: f32) -> Option {
merged.map(|logits| Prediction { logits, score: best })
}
+/// Share of the largest candidate's area another needs to count among the photo's main subjects
+/// (people in front, not the crowd behind them).
+pub const PROMINENT: f32 = 0.25;
+
+/// Which candidates are the photo's subject, given each one's area (cells it selects) and, when the
+/// camera recorded where it focused, whether it covers that point. The focus decides: the
+/// candidates it falls on, with the others about as large either way (a group at the same
+/// distance); a focus on none of them means the subject is something else (empty: the caller
+/// selects what is at the point).
+/// Without a focus: the largest, and the others about as large.
+pub fn pick_prominent(areas: &[usize], on_focus: Option<&[bool]>) -> Vec {
+ let largest = |ks: &mut dyn Iterator| ks.filter_map(|k| areas.get(k).copied()).max().unwrap_or(0);
+ let near = |a: usize, of: usize| of > 0 && a as f32 >= PROMINENT * of as f32;
+ match on_focus {
+ Some(hit) => {
+ let focused: Vec = (0..areas.len()).filter(|k| hit.get(*k).copied().unwrap_or(false) && areas[*k] > 0).collect();
+ if focused.is_empty() {
+ return Vec::new();
+ }
+ // about as large as what is in focus, either way: much larger is nearer the camera
+ // than the focus, so out of focus too
+ let of = largest(&mut focused.iter().copied());
+ let alike = |a: usize| near(a, of) && (a as f32) * PROMINENT <= of as f32;
+ (0..areas.len()).filter(|k| focused.contains(k) || alike(areas[*k])).collect()
+ }
+ None => {
+ let of = largest(&mut (0..areas.len()));
+ (0..areas.len()).filter(|k| near(areas[*k], of)).collect()
+ }
+ }
+}
+
+/// The instances of several detector outputs (one per concept: "person", "animal") above
+/// `threshold` that [`pick_prominent`] keeps, each with its share of its concept's semantic mask,
+/// merged into one mask. `focus`: where the camera focused (normalized). `None` when nothing is kept.
+pub fn subject(outs: &[Output], threshold: f32, focus: Option<(f32, f32)>) -> Option {
+ let s = |x: f32| 1.0 / (1.0 + (-x).exp());
+ // (mask, score, concept)
+ let mut cands: Vec<(Vec, f32, usize)> = Vec::new();
+ for (c, out) in outs.iter().enumerate() {
+ let presence = s(out.presence);
+ for (i, l) in out.logits.iter().enumerate() {
+ let p = s(*l) * presence;
+ if p > threshold
+ && let Some(m) = out.masks.i(i).ok().and_then(|t| to_vec(&t).ok())
+ {
+ cands.push((m, p, c));
+ }
+ }
+ }
+ let n = cands.first()?.0.len();
+ let side = (n as f64).sqrt() as usize;
+ if side == 0 || side * side != n || cands.iter().any(|c| c.0.len() != n) {
+ return None;
+ }
+ let areas: Vec = cands.iter().map(|c| c.0.iter().filter(|l| **l > 0.0).count()).collect();
+ let cell = focus.filter(|(x, y)| x.is_finite() && y.is_finite()).map(|(x, y)| {
+ let at = |v: f32| ((v.clamp(0.0, 1.0) * side as f32) as usize).min(side - 1);
+ at(y) * side + at(x)
+ });
+ let hits: Option> = cell.map(|i| cands.iter().map(|c| c.0.get(i).is_some_and(|l| *l > 0.0)).collect());
+ let keep = pick_prominent(&areas, hits.as_deref());
+ if keep.is_empty() {
+ return None;
+ }
+ let best = keep.iter().filter_map(|k| cands.get(*k)).map(|c| c.1).fold(0f32, f32::max);
+ let mut merged: Option> = None;
+ for (c, out) in outs.iter().enumerate() {
+ let mut masks: Vec> = keep.iter().filter_map(|k| cands.get(*k)).filter(|x| x.2 == c).map(|x| x.0.clone()).collect();
+ if masks.is_empty() {
+ continue;
+ }
+ share(&mut masks, &out.semantic);
+ for m in masks {
+ merged = Some(match merged {
+ Some(acc) => acc.iter().zip(&m).map(|(a, b)| a.max(*b)).collect(),
+ None => m,
+ });
+ }
+ }
+ merged.map(|logits| Prediction { logits, score: best })
+}
+
#[cfg(test)]
mod tests {
use super::*;
+ /// A group in front is kept, the crowd behind isn't; the camera's focus decides: on a person,
+ /// that person and the others as large; on nobody, nothing (the object there is the subject).
+ #[test]
+ fn prominent_subjects() {
+ // four people in front, two small ones far behind
+ let areas = [900, 1000, 800, 950, 60, 40];
+ assert_eq!(pick_prominent(&areas, None), vec![0, 1, 2, 3]);
+ // focused on the first: the group
+ let hit = [true, false, false, false, false, false];
+ assert_eq!(pick_prominent(&areas, Some(&hit)), vec![0, 1, 2, 3]);
+ // focused on someone small behind: that one and whoever is about as large, not the
+ // (much larger, so nearer and out of focus) group in front
+ let hit = [false, false, false, false, true, false];
+ assert_eq!(pick_prominent(&areas, Some(&hit)), vec![4, 5]);
+ // a product in front of people: the focus is on none of them
+ assert!(pick_prominent(&areas, Some(&[false; 6])).is_empty());
+ // nobody at all, or empty masks
+ assert!(pick_prominent(&[], None).is_empty());
+ assert!(pick_prominent(&[0, 0], None).is_empty());
+ assert!(pick_prominent(&[0, 5], Some(&[true, false])).is_empty());
+ // a mismatched focus list is read as "not on it"
+ assert_eq!(pick_prominent(&[10, 9], Some(&[true])), vec![0, 1]);
+ }
+
/// Two people side by side on a row of cells, each only weakly sure of themself and both fading where
/// they touch; the semantic mask is sure of both and runs across. Each takes the semantic logits where
/// it is the strongest instance, a third region nobody is near keeps its own.
diff --git a/crates/segment/src/lib.rs b/crates/segment/src/lib.rs
index fba28b00e9..98a47d9888 100644
--- a/crates/segment/src/lib.rs
+++ b/crates/segment/src/lib.rs
@@ -81,6 +81,10 @@ pub const MAX_CLICKS: usize = 64;
#[cfg(not(target_arch = "wasm32"))]
const _: () = assert!(MASK_SIDE == tracker::LOW_RES && INPUT == vit::IMAGE);
+/// What a photo's subject usually is, as concept prompts ([`Sam3::segment_subject`]); anything else
+/// (an object, a landscape's feature) is selected at the focus point instead.
+pub const SUBJECT_CONCEPTS: &[&str] = &["person", "animal"];
+
/// The files of a model directory.
pub const WEIGHTS_FILE: &str = "model.safetensors";
@@ -247,6 +251,15 @@ impl Sam3 {
Ok(detector::merge(&out, threshold))
}
+ /// The photo's subject among its people and animals ([`detector::subject`]): the ones the
+ /// camera's focus (`focus`, normalized; `None` when the file doesn't record it) falls on, or
+ /// without it the most prominent, with whoever is about as large. `None` when the subject is
+ /// neither (a focus on something else, or nobody in the photo): select what is at the point.
+ pub fn segment_subject(&mut self, enc: &mut Encoded, focus: Option<(f32, f32)>, threshold: f32) -> Result