mirror of
https://github.com/storytold/effectcraft.git
synced 2026-10-26 19:18:14 -06:00
M12.6: 3D Camera Tracker effect + UI
Effect > Perspective > 3D Camera Tracker (Shot Type, Horizontal Angle of View, Show/Render Track Points, Track Point Size, Target Size, Advanced: Solve Method, Method Used, Average Error, Detailed Analysis, Auto-delete Points Across Time, Hide Warning Banner); tracks and solve stored in the effect with keys (frame edits re-track, setting / deleted-point edits only re-solve). Background job (Analyzing in background (step 1 of 2) / Solving camera banner, cancel, one undo step). Commands: track.camera (replaces the stub), camera.analyze/cancel/solveStatus/points/selectPoints/ createFromSolve/create/setGroundPlane/deletePoints; the one-node 3D Tracker Camera keyed per frame and text / solids / nulls / shadow catcher + light on the target plane, Create Multiple. Viewer: depth-sized coloured targets, hover target plane, click / shift / marquee selection, AE right-click menu, Delete; Effect Controls Analyze / Cancel / Create Camera; Tracker panel Track Camera. Solver: adaptive noise thresholds, denser keyframes, quick tripod check, faster Cholesky and parallel Schur complement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Brandon Thomas
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
bab4f88c22
commit
cba8d7e737
Generated
+1
@@ -1420,6 +1420,7 @@ dependencies = [
|
||||
"rayon",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"web-time",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||
+2
-2
@@ -28,10 +28,10 @@ once.
|
||||
Measured feature by feature in [docs/parity.md](docs/parity.md) (end of 2 October 2026):
|
||||
|
||||
- **≈ 79% of After Effects' features, weighted by importance** (87% of the essentials, 67% of
|
||||
professional daily-use features, 21% of the long tail); 257 of 298 effects, 16 of them on the GPU.
|
||||
professional daily-use features, 21% of the long tail); 258 of 298 effects, 16 of them on the GPU.
|
||||
- **≈ 100 agent-hours of work remain by the audit's estimate.** With five Claude Opus 5.5 agents in
|
||||
parallel that is **≈ 8–10 hours of wall-clock time at the pace measured so far, up to ≈ 25 hours**
|
||||
by the audit's conservative per-feature figures.
|
||||
- Hardest remaining pieces: Roto Brush, the 3D camera tracker, Advanced 3D, a scripting object model.
|
||||
- Hardest remaining pieces: Roto Brush, Advanced 3D, a scripting object model (the 3D Camera Tracker landed).
|
||||
|
||||
Come tell us what matters most to you on [Discord](https://discord.gg/artcraft).
|
||||
|
||||
@@ -0,0 +1,291 @@
|
||||
//! 3D Camera Tracker (Effect ▸ Perspective ▸ 3D Camera Tracker; Animation ▸ Track Camera).
|
||||
//!
|
||||
//! The analysis runs as a background job in the engine (`camera.analyze`): step 1 tracks
|
||||
//! features through the clip ([`effectcraft_track::camtrack::TrackAnalyzer`]), step 2 solves
|
||||
//! the camera ([`effectcraft_track::camtrack::solve`]). Both results are stored as JSON in the
|
||||
//! instance's hidden parameters — **Tracks** (with **Tracks Key**: what the frames were made
|
||||
//! from, so source / In-Out / time edits invalidate them) and **Solve** (with **Solve Key**: the
|
||||
//! Shot Type, Angle of View, Solve Method and deleted points it was solved with, so changing
|
||||
//! those re-solves without re-tracking). **Method Used** and **Average Error** report the solve.
|
||||
//!
|
||||
//! The effect passes its input through; with **Render Track Points** on it draws the solved
|
||||
//! points (as the viewer shows them) into the frame. The viewer overlay, target and the Create
|
||||
//! commands live in the engine and the UI.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
|
||||
use effectcraft_keyframe::Value;
|
||||
use effectcraft_project::ParamUi;
|
||||
use effectcraft_raster::{Image, Px};
|
||||
use effectcraft_track::camtrack::{CameraSolve, CameraTracks, ShotType, SolveMethod, SolveSettings, linalg, point_color};
|
||||
|
||||
use crate::warp_stab::param;
|
||||
use crate::{Buf, EffectCtx, EffectSpec, Params, num, p, popup, slider};
|
||||
|
||||
/// The effect's id.
|
||||
pub const ID: &str = "ec.perspective.cameratracker";
|
||||
|
||||
/// Hidden parameters (see the module docs).
|
||||
pub const TRACKS: &str = "tracks";
|
||||
pub const TRACKS_KEY: &str = "tracksKey";
|
||||
pub const SOLVE: &str = "solve";
|
||||
pub const SOLVE_KEY: &str = "solveKey";
|
||||
/// Deleted track point ids (JSON array).
|
||||
pub const DELETED: &str = "deleted";
|
||||
/// The "3D Tracker Camera" layer the instance created (layer id, 0 = none).
|
||||
pub const CAMERA: &str = "camera";
|
||||
pub const METHOD_USED: &str = "advanced/methodUsed";
|
||||
pub const AVERAGE_ERROR: &str = "advanced/averageError";
|
||||
|
||||
pub const SHOT_TYPES: [&str; 3] = ["Fixed Angle of View", "Variable Zoom", "Specify Angle of View"];
|
||||
pub const SHOW_POINTS: [&str; 2] = ["2D Source", "3D Solved"];
|
||||
pub const SOLVE_METHODS: [&str; 4] = ["Auto Detect", "Typical", "Mostly Flat Scene", "Tripod Pan"];
|
||||
|
||||
pub fn specs() -> Vec<EffectSpec> {
|
||||
vec![EffectSpec {
|
||||
id: ID,
|
||||
name: "3D Camera Tracker",
|
||||
category: "Perspective",
|
||||
params: vec![
|
||||
p("shotType", "Shot Type", Value::Enum(0), popup(&SHOT_TYPES)),
|
||||
p("horizontalAngleOfView", "Horizontal Angle of View", num(40.0), slider(1.0, 170.0, 10.0, 120.0, 1)),
|
||||
p("showTrackPoints", "Show Track Points", Value::Enum(1), popup(&SHOW_POINTS)),
|
||||
p("renderTrackPoints", "Render Track Points", Value::Bool(false), ParamUi::Checkbox),
|
||||
p("trackPointSize", "Track Point Size", num(100.0), slider(0.0, 1000.0, 0.0, 300.0, 0)),
|
||||
p("targetSize", "Target Size", num(100.0), slider(1.0, 1000.0, 10.0, 300.0, 0)),
|
||||
p("advanced/solveMethod", "Solve Method", Value::Enum(0), popup(&SOLVE_METHODS)),
|
||||
p(METHOD_USED, "Method Used", Value::Str(String::new()), ParamUi::Hidden),
|
||||
p(AVERAGE_ERROR, "Average Error", num(0.0), ParamUi::Hidden),
|
||||
p("advanced/detailedAnalysis", "Detailed Analysis", Value::Bool(false), ParamUi::Checkbox),
|
||||
p("advanced/autoDeletePoints", "Auto-delete Points Across Time", Value::Bool(true), ParamUi::Checkbox),
|
||||
p("advanced/hideWarningBanner", "Hide Warning Banner", Value::Bool(false), ParamUi::Checkbox),
|
||||
p(TRACKS, "Tracks", Value::Str(String::new()), ParamUi::Hidden),
|
||||
p(TRACKS_KEY, "Tracks Key", Value::Str(String::new()), ParamUi::Hidden),
|
||||
p(SOLVE, "Solve", Value::Str(String::new()), ParamUi::Hidden),
|
||||
p(SOLVE_KEY, "Solve Key", Value::Str(String::new()), ParamUi::Hidden),
|
||||
p(DELETED, "Deleted Points", Value::Str(String::new()), ParamUi::Hidden),
|
||||
p(CAMERA, "Camera Layer", num(0.0), ParamUi::Hidden),
|
||||
],
|
||||
render,
|
||||
gpu: false,
|
||||
float: true,
|
||||
}]
|
||||
}
|
||||
|
||||
fn f(params: &Params, id: &str) -> f64 {
|
||||
param(params, id).map(Value::as_f64).unwrap_or(0.0)
|
||||
}
|
||||
fn e(params: &Params, id: &str) -> u32 {
|
||||
param(params, id).map(Value::as_enum).unwrap_or(0)
|
||||
}
|
||||
fn b(params: &Params, id: &str) -> bool {
|
||||
param(params, id).map(Value::as_bool).unwrap_or(false)
|
||||
}
|
||||
fn s<'a>(params: &'a Params, id: &str) -> &'a str {
|
||||
match param(params, id) {
|
||||
Some(Value::Str(s)) => s,
|
||||
_ => "",
|
||||
}
|
||||
}
|
||||
|
||||
/// Deleted point ids of an instance (stored as a JSON array of numbers).
|
||||
pub fn deleted(params: &Params) -> Vec<u32> {
|
||||
s(params, DELETED).trim().trim_start_matches('[').trim_end_matches(']').split(',').filter_map(|v| v.trim().parse().ok()).collect()
|
||||
}
|
||||
|
||||
/// The solve settings of an instance.
|
||||
pub fn settings(params: &Params) -> SolveSettings {
|
||||
let mut del = deleted(params);
|
||||
del.sort_unstable();
|
||||
del.dedup();
|
||||
SolveSettings {
|
||||
shot: ShotType::ALL.get(e(params, "shotType") as usize).copied().unwrap_or_default(),
|
||||
hfov: f(params, "horizontalAngleOfView"),
|
||||
method: SolveMethod::ALL.get(e(params, "advanced/solveMethod") as usize).copied().unwrap_or_default(),
|
||||
deleted: del,
|
||||
}
|
||||
}
|
||||
|
||||
/// What a solve depends on besides the tracks.
|
||||
pub fn solve_key(st: &SolveSettings) -> String {
|
||||
let hfov = if st.shot == ShotType::SpecifyAngle { format!("{:.4}", st.hfov) } else { String::new() };
|
||||
format!("{:?}|{hfov}|{:?}|{:?}", st.shot, st.method, st.deleted)
|
||||
}
|
||||
|
||||
fn fnv(bytes: &[u8]) -> u64 {
|
||||
let mut h: u64 = 0xcbf2_9ce4_8422_2325;
|
||||
for b in bytes {
|
||||
h ^= *b as u64;
|
||||
h = h.wrapping_mul(0x0100_0000_01b3);
|
||||
}
|
||||
h
|
||||
}
|
||||
|
||||
type Cache<T> = Mutex<HashMap<u64, Arc<T>>>;
|
||||
|
||||
fn cached<T>(c: &'static OnceLock<Cache<T>>, json: &str, parse: impl Fn(&str) -> Option<T>) -> Option<Arc<T>> {
|
||||
let key = fnv(json.as_bytes());
|
||||
let c = c.get_or_init(Default::default);
|
||||
if let Some(a) = c.lock().unwrap_or_else(|e| e.into_inner()).get(&key) {
|
||||
return Some(a.clone());
|
||||
}
|
||||
let a = Arc::new(parse(json)?);
|
||||
let mut m = c.lock().unwrap_or_else(|e| e.into_inner());
|
||||
if m.len() > 8 {
|
||||
m.clear();
|
||||
}
|
||||
m.insert(key, a.clone());
|
||||
Some(a)
|
||||
}
|
||||
|
||||
/// The stored feature tracks of an instance (`None` when not analysed).
|
||||
pub fn tracks(params: &Params) -> Option<Arc<CameraTracks>> {
|
||||
static C: OnceLock<Cache<CameraTracks>> = OnceLock::new();
|
||||
let j = s(params, TRACKS);
|
||||
if j.is_empty() {
|
||||
return None;
|
||||
}
|
||||
cached(&C, j, CameraTracks::from_json).filter(|t| !t.is_empty())
|
||||
}
|
||||
|
||||
/// The stored camera solve of an instance (`None` when not solved).
|
||||
pub fn solve(params: &Params) -> Option<Arc<CameraSolve>> {
|
||||
static C: OnceLock<Cache<CameraSolve>> = OnceLock::new();
|
||||
let j = s(params, SOLVE);
|
||||
if j.is_empty() {
|
||||
return None;
|
||||
}
|
||||
cached(&C, j, CameraSolve::from_json).filter(|t| !t.is_empty())
|
||||
}
|
||||
|
||||
/// Screen size (layer pixels) of a target drawn for a point at depth `z`, relative to the
|
||||
/// median depth of the frame's points.
|
||||
pub fn target_radius(size_pct: f64, z: f64, median_z: f64) -> f64 {
|
||||
(6.0 * size_pct / 100.0 * (median_z / z.max(1e-9))).clamp(1.5, 40.0)
|
||||
}
|
||||
|
||||
/// The solved points visible on frame `k`: (id, layer pixel position, camera depth).
|
||||
pub fn projected(solve: &CameraSolve, k: usize) -> Vec<(u32, [f64; 2], f64)> {
|
||||
let Some(cam) = solve.frames.get(k) else { return vec![] };
|
||||
solve
|
||||
.visible(k)
|
||||
.filter_map(|p| {
|
||||
let c = cam.to_cam(p.pos);
|
||||
cam.project(solve.size, p.pos).map(|q| (p.id, q, c[2]))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Median of the depths.
|
||||
pub fn median_depth(v: &[(u32, [f64; 2], f64)]) -> f64 {
|
||||
let mut z: Vec<f64> = v.iter().map(|x| x.2).collect();
|
||||
if z.is_empty() {
|
||||
return 1.0;
|
||||
}
|
||||
z.sort_by(f64::total_cmp);
|
||||
z[z.len() / 2]
|
||||
}
|
||||
|
||||
fn render(ctx: &EffectCtx, mut buf: Buf) -> Buf {
|
||||
if !b(ctx.params, "renderTrackPoints") {
|
||||
return buf;
|
||||
}
|
||||
let Some(sol) = solve(ctx.params) else { return buf };
|
||||
if (sol.size[0] - ctx.layer_size[0]).abs() > 0.5 || (sol.size[1] - ctx.layer_size[1]).abs() > 0.5 {
|
||||
return buf;
|
||||
}
|
||||
let Some(k) = sol.frame_at(ctx.time) else { return buf };
|
||||
let pts = projected(&sol, k);
|
||||
let med = median_depth(&pts);
|
||||
let size = f(ctx.params, "trackPointSize");
|
||||
let sc = if buf.scale > 0.0 { buf.scale } else { 1.0 };
|
||||
for (id, q, z) in pts {
|
||||
let c = point_color(id);
|
||||
let r = target_radius(size, z, med) * sc;
|
||||
let at = [q[0] * sc + buf.offset[0], q[1] * sc + buf.offset[1]];
|
||||
ring(&mut buf.img, at, r, (r * 0.25).max(1.0), [c[0], c[1], c[2], 1.0]);
|
||||
disc(&mut buf.img, at, (r * 0.2).max(0.8), [c[0], c[1], c[2], 1.0]);
|
||||
}
|
||||
buf
|
||||
}
|
||||
|
||||
fn lerp_px(a: Px, b: Px, t: f32) -> Px {
|
||||
[a[0] + (b[0] - a[0]) * t, a[1] + (b[1] - a[1]) * t, a[2] + (b[2] - a[2]) * t, a[3] + (b[3] - a[3]) * t]
|
||||
}
|
||||
|
||||
fn cover(img: &mut Image, c: [f64; 2], r_out: f64, cov: impl Fn(f64) -> f64, col: Px) {
|
||||
let (x0, x1) = ((c[0] - r_out - 1.0).floor().max(0.0) as i64, (c[0] + r_out + 1.0).ceil() as i64);
|
||||
let (y0, y1) = ((c[1] - r_out - 1.0).floor().max(0.0) as i64, (c[1] + r_out + 1.0).ceil() as i64);
|
||||
for y in y0..y1.min(img.height as i64) {
|
||||
for x in x0..x1.min(img.width as i64) {
|
||||
let d = (x as f64 + 0.5 - c[0]).hypot(y as f64 + 0.5 - c[1]);
|
||||
let a = cov(d).clamp(0.0, 1.0) as f32;
|
||||
if a > 0.0 {
|
||||
let i = img.idx(x as u32, y as u32);
|
||||
img.data[i] = lerp_px(img.data[i], col, a);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn disc(img: &mut Image, c: [f64; 2], r: f64, col: Px) {
|
||||
cover(img, c, r, |d| r + 0.5 - d, col);
|
||||
}
|
||||
|
||||
fn ring(img: &mut Image, c: [f64; 2], r: f64, w: f64, col: Px) {
|
||||
cover(img, c, r + w, |d| (w * 0.5 + 0.5 - (d - r).abs()).min(1.0), col);
|
||||
}
|
||||
|
||||
/// Points of a target disc (canonical coordinates) for drawing: `n` points on the circle of
|
||||
/// radius `size / 2` in the plane through `center` with normal `normal`.
|
||||
pub fn target_circle(center: linalg::V3, normal: linalg::V3, radius: f64, n: usize) -> Vec<linalg::V3> {
|
||||
let a = if normal[0].abs() < 0.9 { [1.0, 0.0, 0.0] } else { [0.0, 1.0, 0.0] };
|
||||
let u = linalg::normalize(linalg::cross(normal, a));
|
||||
let v = linalg::cross(normal, u);
|
||||
(0..n)
|
||||
.map(|i| {
|
||||
let t = i as f64 / n as f64 * std::f64::consts::TAU;
|
||||
linalg::add(center, linalg::add(linalg::scale(u, radius * t.cos()), linalg::scale(v, radius * t.sin())))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::EffectEnv;
|
||||
use effectcraft_track::camtrack::{SolvedFrame, SolvedPoint};
|
||||
|
||||
fn params_with(solve: &CameraSolve, render: bool) -> Params {
|
||||
let mut p = Params::default();
|
||||
p.values.insert(SOLVE.into(), Value::Str(solve.to_json()));
|
||||
p.values.insert("renderTrackPoints".into(), Value::Bool(render));
|
||||
p.values.insert("trackPointSize".into(), Value::Scalar(100.0));
|
||||
p
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_track_points_draws_targets_and_passes_through_otherwise() {
|
||||
let sol = CameraSolve {
|
||||
version: 1,
|
||||
size: [64.0, 48.0],
|
||||
start: 0.0,
|
||||
frame_duration: 1.0 / 30.0,
|
||||
frames: vec![SolvedFrame { rot: [0.0; 3], center: [0.0; 3], focal: 50.0, solved: true }],
|
||||
points: vec![SolvedPoint { id: 3, pos: [0.0, 0.0, 1.0], error: 0.1, first: 0, last: 0 }],
|
||||
..Default::default()
|
||||
};
|
||||
let img = Image::new(64, 48);
|
||||
for on in [false, true] {
|
||||
let params = params_with(&sol, on);
|
||||
let ctx = EffectCtx { params: ¶ms, time: 0.0, layer_size: [64.0, 48.0], seed: 0, adjustment: false, env: EffectEnv::default() };
|
||||
let out = render(&ctx, Buf { img: img.clone(), offset: [0.0; 2], scale: 1.0 });
|
||||
let px = out.img.data[out.img.idx(32, 24)];
|
||||
assert_eq!(px[3] > 0.5, on);
|
||||
}
|
||||
let st = settings(&Params::default());
|
||||
assert_eq!(st.shot, ShotType::FixedAngle);
|
||||
assert!(solve_key(&st).contains("FixedAngle"));
|
||||
}
|
||||
}
|
||||
@@ -8,6 +8,7 @@
|
||||
|
||||
pub mod audio_fx;
|
||||
mod blur2;
|
||||
pub mod camera_tracker;
|
||||
mod channel;
|
||||
mod channel2;
|
||||
mod color2;
|
||||
@@ -354,6 +355,7 @@ pub fn registry() -> &'static [EffectSpec] {
|
||||
v.extend(paint::specs());
|
||||
v.extend(puppet::specs());
|
||||
v.extend(warp_stab::specs());
|
||||
v.extend(camera_tracker::specs());
|
||||
v.sort_by(|a, b| a.category.cmp(b.category).then(a.name.cmp(b.name)));
|
||||
for s in v.iter_mut() {
|
||||
if GPU_EFFECTS.contains(&s.id) {
|
||||
@@ -494,6 +496,8 @@ pub const TIME_DEPENDENT: &[&str] = &[
|
||||
"ec.time.pixelmotionblur",
|
||||
// Each frame gets its own stabilizing warp.
|
||||
warp_stab::ID,
|
||||
// Render Track Points draws each frame's solved points.
|
||||
camera_tracker::ID,
|
||||
];
|
||||
|
||||
/// See [`TIME_DEPENDENT`].
|
||||
|
||||
@@ -0,0 +1,640 @@
|
||||
//! 3D Camera Tracker runtime: the analysis job (`camera.analyze`, `track.camera`), keeping
|
||||
//! analyses valid, and turning a solve into composition layers (`camera.createFromSolve`).
|
||||
//!
|
||||
//! The job renders the layer's input to the effect (`Renderer::layer_input`) for every frame
|
||||
//! between the layer's In and Out points on a background thread and tracks features through
|
||||
//! them (step 1 of 2, "Analyzing in background"), then solves the camera (step 2, "Solving
|
||||
//! camera"). When only the solve settings changed (Shot Type, Angle of View, Solve Method,
|
||||
//! deleted points) the stored tracks are re-solved without re-tracking. Results are written into
|
||||
//! the effect's hidden parameters as one undo step; edits that change the layer's frames clear
|
||||
//! them and queue the effect again ([`invalidate`], run after every edit), like After Effects.
|
||||
//!
|
||||
//! Composition space: the solve's canonical frame maps to comp world by
|
||||
//! [`CameraSolve::world`] with the zoom of the first solved frame (focal length × the layer's
|
||||
//! scale in the comp), so the "3D Tracker Camera" reproduces the footage's view and created
|
||||
//! layers sit on the selected points.
|
||||
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use effectcraft_effects::camera_tracker::{self as ct, AVERAGE_ERROR, METHOD_USED, SOLVE, SOLVE_KEY, TRACKS, TRACKS_KEY};
|
||||
use effectcraft_keyframe::{Keyframe, Value};
|
||||
use effectcraft_project::{AutoOrient, GroupKind, ItemId, ItemKind, Layer, LayerId, Project, PropGroup, Uid};
|
||||
use effectcraft_render::{EvalCtx, ExprHost, FootageSource, LayerCache, RenderOpts, Renderer};
|
||||
use effectcraft_time::Tick;
|
||||
use effectcraft_track::camtrack::linalg::{self, Similarity, V3};
|
||||
use effectcraft_track::camtrack::{AnalyzeOpts, CameraSolve, CameraTracks, SolveSettings, Target, TrackAnalyzer};
|
||||
use serde::Serialize;
|
||||
|
||||
use crate::tracking::lock;
|
||||
use crate::warp::{analysis_times, signature};
|
||||
use crate::{Event, Session};
|
||||
|
||||
/// Live analysis progress (serde for agents / the control channel).
|
||||
#[derive(Clone, Debug, Default, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct CameraProgress {
|
||||
/// 1 = tracking features, 2 = solving the camera.
|
||||
pub step: u8,
|
||||
pub done: u64,
|
||||
pub total: u64,
|
||||
pub elapsed: f64,
|
||||
pub fps: f64,
|
||||
pub finished: bool,
|
||||
pub cancelled: bool,
|
||||
pub error: Option<String>,
|
||||
/// Re-solving stored tracks (no tracking step).
|
||||
pub solve_only: bool,
|
||||
}
|
||||
|
||||
impl CameraProgress {
|
||||
/// The viewer banner text, as After Effects words it.
|
||||
pub fn banner(&self) -> String {
|
||||
match self.step {
|
||||
2 => "Solving camera".into(),
|
||||
_ => {
|
||||
let pc = (100 * self.done).checked_div(self.total).unwrap_or(0);
|
||||
format!("Analyzing in background (step 1 of 2): {pc}%")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct CamResult {
|
||||
/// New tracks (None for a re-solve).
|
||||
tracks: Option<CameraTracks>,
|
||||
solve: Result<CameraSolve, String>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
pub struct CameraShared {
|
||||
pub cancel: AtomicBool,
|
||||
pub state: Mutex<CameraProgress>,
|
||||
result: Mutex<Option<CamResult>>,
|
||||
}
|
||||
|
||||
/// A running (or finished, not yet polled) camera analysis.
|
||||
pub struct CameraJob {
|
||||
pub shared: Arc<CameraShared>,
|
||||
thread: Option<std::thread::JoinHandle<()>>,
|
||||
pub comp: ItemId,
|
||||
pub layer: LayerId,
|
||||
pub effect: Uid,
|
||||
/// The layer signature the tracks are made from, and the solve settings key.
|
||||
key: String,
|
||||
solve_key: String,
|
||||
}
|
||||
|
||||
impl CameraJob {
|
||||
pub fn is_finished(&self) -> bool {
|
||||
lock(&self.shared.state).finished
|
||||
}
|
||||
pub fn progress(&self) -> CameraProgress {
|
||||
lock(&self.shared.state).clone()
|
||||
}
|
||||
}
|
||||
|
||||
struct CamWork {
|
||||
project: Arc<Project>,
|
||||
footage: Arc<dyn FootageSource>,
|
||||
expr: Option<Arc<dyn ExprHost>>,
|
||||
cache: Arc<LayerCache>,
|
||||
comp: ItemId,
|
||||
layer: LayerId,
|
||||
effect_index: usize,
|
||||
size: [f64; 2],
|
||||
detailed: bool,
|
||||
times: Vec<Tick>,
|
||||
/// Stored tracks to re-solve (skips step 1).
|
||||
tracks: Option<Arc<CameraTracks>>,
|
||||
settings: SolveSettings,
|
||||
}
|
||||
|
||||
pub(crate) fn is_tracker(g: &PropGroup) -> bool {
|
||||
matches!(&g.kind, GroupKind::Effect { effect } if effect == ct::ID)
|
||||
}
|
||||
|
||||
fn str_of(g: &PropGroup, m: &str) -> String {
|
||||
match g.prop(m).map(|p| &p.value) {
|
||||
Some(Value::Str(s)) => s.clone(),
|
||||
_ => String::new(),
|
||||
}
|
||||
}
|
||||
|
||||
/// What a tracker's tracks are made from: the layer's frames and Detailed Analysis.
|
||||
fn tracks_key(project: &Project, cid: ItemId, l: &Layer, g: &PropGroup) -> String {
|
||||
let detailed = effectcraft_effects::warp_stab::param(&static_params(g), "advanced/detailedAnalysis").is_some_and(Value::as_bool);
|
||||
format!("{}{}", signature(project, cid, l), if detailed { "+detailed" } else { "" })
|
||||
}
|
||||
|
||||
/// Parameters of an instance from static values (the tracker's settings are not animated).
|
||||
pub fn static_params(g: &PropGroup) -> effectcraft_effects::Params {
|
||||
effectcraft_effects::flatten_params(g, &mut |p| p.value.clone())
|
||||
}
|
||||
|
||||
fn clear(g: &mut PropGroup, all: bool) {
|
||||
let mut ids = vec![SOLVE, SOLVE_KEY, METHOD_USED];
|
||||
if all {
|
||||
ids.extend([TRACKS, TRACKS_KEY]);
|
||||
}
|
||||
for m in ids {
|
||||
if let Some(p) = g.prop_mut(m) {
|
||||
p.value = Value::Str(String::new());
|
||||
p.keys.clear();
|
||||
}
|
||||
}
|
||||
if let Some(p) = g.prop_mut(AVERAGE_ERROR) {
|
||||
p.value = Value::Scalar(0.0);
|
||||
}
|
||||
}
|
||||
|
||||
/// Clear analyses that no longer match, after an edit: tracks made from other frames (and their
|
||||
/// solve), and solves made with other settings. Returns the trackers that need (re-)analysis:
|
||||
/// cleared ones and newly applied, unanalysed ones.
|
||||
pub(crate) fn invalidate(before: &Project, after: &mut Project) -> Vec<(ItemId, LayerId, Uid)> {
|
||||
let mut todo = vec![];
|
||||
let mut clears = vec![];
|
||||
for (cid, it) in &after.items {
|
||||
let ItemKind::Comp(c) = &it.kind else { continue };
|
||||
for l in &c.layers {
|
||||
let Some(fx) = l.effects() else { continue };
|
||||
if !fx.groups().any(is_tracker) {
|
||||
continue;
|
||||
}
|
||||
let old_layer = before.comp(*cid).and_then(|bc| bc.layer(l.id));
|
||||
for g in fx.groups().filter(|g| is_tracker(g)) {
|
||||
let tracked = !str_of(g, TRACKS).is_empty();
|
||||
if tracked && str_of(g, TRACKS_KEY) != tracks_key(after, *cid, l, g) {
|
||||
clears.push((*cid, l.id, g.uid, true));
|
||||
todo.push((*cid, l.id, g.uid));
|
||||
} else if tracked && str_of(g, SOLVE_KEY) != ct::solve_key(&ct::settings(&static_params(g))) {
|
||||
clears.push((*cid, l.id, g.uid, false));
|
||||
todo.push((*cid, l.id, g.uid));
|
||||
} else if !tracked && old_layer.and_then(|ol| ol.props.find_group(g.uid)).is_none() {
|
||||
todo.push((*cid, l.id, g.uid));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for (cid, lid, uid, all) in clears {
|
||||
if let Some(g) = after.comp_mut(cid).and_then(|c| c.layer_mut(lid)).and_then(|l| l.props.find_group_mut(uid)) {
|
||||
clear(g, all);
|
||||
}
|
||||
}
|
||||
todo
|
||||
}
|
||||
|
||||
fn input_frame(r: &Renderer, ctx: &EvalCtx, layer: &Layer, effect_index: usize) -> Option<(Arc<effectcraft_raster::Image>, [f64; 2])> {
|
||||
let buf = r.layer_input(ctx, layer, effect_index)?;
|
||||
if (buf.scale - 1.0).abs() > 1e-9 && buf.scale > 0.0 {
|
||||
let w = (buf.img.width as f64 / buf.scale).round().max(1.0) as u32;
|
||||
let h = (buf.img.height as f64 / buf.scale).round().max(1.0) as u32;
|
||||
let off = [buf.offset[0] / buf.scale, buf.offset[1] / buf.scale];
|
||||
return Some((Arc::new(effectcraft_raster::resample(&buf.img, w, h)), off));
|
||||
}
|
||||
Some((Arc::new(buf.img.clone()), buf.offset))
|
||||
}
|
||||
|
||||
fn run_work(w: CamWork, shared: &CameraShared) {
|
||||
let t0 = web_time::Instant::now();
|
||||
let fail = |msg: &str| {
|
||||
let mut s = lock(&shared.state);
|
||||
s.error = Some(msg.to_string());
|
||||
s.finished = true;
|
||||
};
|
||||
let cancelled = || {
|
||||
let mut s = lock(&shared.state);
|
||||
s.cancelled = true;
|
||||
s.finished = true;
|
||||
};
|
||||
let Some(comp) = w.project.comp(w.comp) else { return fail("composition missing") };
|
||||
let Some(layer) = comp.layer(w.layer) else { return fail("layer missing") };
|
||||
let lt0 = layer.layer_time(w.times[0]).seconds();
|
||||
let fd = if w.times.len() > 1 { layer.layer_time(w.times[1]).seconds() - lt0 } else { comp.frame_duration().seconds() };
|
||||
let (tracks, fresh) = match w.tracks.clone() {
|
||||
Some(t) => (t, false),
|
||||
None => {
|
||||
let mut r = Renderer::new(&w.project, w.footage.as_ref(), RenderOpts { scale: 1.0, ..Default::default() });
|
||||
r.expr = w.expr.as_deref();
|
||||
r.cache = Some(&w.cache);
|
||||
let r = &r;
|
||||
let ctx0 = EvalCtx::new(&w.project, w.comp, comp, Tick::ZERO);
|
||||
let ctx0 = EvalCtx { expr: w.expr.as_deref(), ..ctx0 };
|
||||
let frame_at = |t: Tick| input_frame(r, &ctx0.at(t), layer, w.effect_index);
|
||||
{
|
||||
let mut s = lock(&shared.state);
|
||||
s.step = 1;
|
||||
s.total = w.times.len() as u64;
|
||||
}
|
||||
let mut an = TrackAnalyzer::new(w.size, AnalyzeOpts { detailed: w.detailed });
|
||||
let mut next = frame_at(w.times[0]);
|
||||
for k in 0..w.times.len() {
|
||||
if shared.cancel.load(Ordering::Relaxed) {
|
||||
return cancelled();
|
||||
}
|
||||
let Some(cur) = next.take() else { return fail(&format!("no frame at {:.3} s", w.times[k].seconds())) };
|
||||
let ((), nf) = rayon::join(
|
||||
|| an.push(&effectcraft_track::Frame { img: &cur.0, offset: cur.1 }),
|
||||
|| if k + 1 < w.times.len() { frame_at(w.times[k + 1]) } else { None },
|
||||
);
|
||||
next = nf;
|
||||
let mut s = lock(&shared.state);
|
||||
s.done = k as u64 + 1;
|
||||
s.elapsed = t0.elapsed().as_secs_f64();
|
||||
s.fps = if s.elapsed > 0.0 { s.done as f64 / s.elapsed } else { 0.0 };
|
||||
}
|
||||
(Arc::new(an.finish(lt0, fd)), true)
|
||||
}
|
||||
};
|
||||
lock(&shared.state).step = 2;
|
||||
let solve = effectcraft_track::camtrack::solve(&tracks, &w.settings, Some(&shared.cancel)).map_err(|e| e.0);
|
||||
if shared.cancel.load(Ordering::Relaxed) {
|
||||
return cancelled();
|
||||
}
|
||||
*lock(&shared.result) = Some(CamResult { tracks: fresh.then(|| (*tracks).clone()), solve });
|
||||
let mut s = lock(&shared.state);
|
||||
s.elapsed = t0.elapsed().as_secs_f64();
|
||||
s.finished = true;
|
||||
}
|
||||
|
||||
impl Session {
|
||||
pub fn is_camera_analyzing(&self) -> bool {
|
||||
self.camera_job.as_ref().is_some_and(|j| !j.is_finished())
|
||||
}
|
||||
|
||||
/// Progress of the running (or just finished) analysis.
|
||||
pub fn camera_progress(&self) -> Option<CameraProgress> {
|
||||
self.camera_job.as_ref().map(|j| j.progress())
|
||||
}
|
||||
|
||||
/// The tracker being analysed: (comp, layer, effect uid).
|
||||
pub fn camera_target(&self) -> Option<(ItemId, LayerId, Uid)> {
|
||||
self.camera_job.as_ref().map(|j| (j.comp, j.layer, j.effect))
|
||||
}
|
||||
|
||||
/// Cancel the running analysis (nothing is written).
|
||||
pub fn stop_camera(&mut self) -> bool {
|
||||
match &self.camera_job {
|
||||
Some(j) if !j.is_finished() => {
|
||||
j.shared.cancel.store(true, Ordering::Relaxed);
|
||||
true
|
||||
}
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Start analysing 3D Camera Tracker `effect` on `layer` (re-solving the stored tracks when
|
||||
/// they are still valid). `wait`: block until done (wasm32 always does). Returns the number
|
||||
/// of frames.
|
||||
pub fn start_camera(&mut self, comp: ItemId, layer: LayerId, effect: Uid, wait: bool) -> Result<usize, String> {
|
||||
if self.is_camera_analyzing() {
|
||||
return Err("a 3D Camera Tracker analysis is already running".into());
|
||||
}
|
||||
self.poll_camera_job();
|
||||
self.camera_pending.retain(|x| *x != (comp, layer, effect));
|
||||
let c = self.project.comp(comp).ok_or("no composition")?;
|
||||
let l = c.layer(layer).ok_or("no layer")?;
|
||||
let fx = l.effects().ok_or("the layer has no effects")?;
|
||||
let index = fx.groups().position(|g| g.uid == effect).ok_or("no such effect")?;
|
||||
let g = fx.groups().nth(index).filter(|g| is_tracker(g)).ok_or("not a 3D Camera Tracker")?;
|
||||
if l.switches.adjustment {
|
||||
return Err("3D Camera Tracker needs footage, not an adjustment layer".into());
|
||||
}
|
||||
let times = analysis_times(c, l);
|
||||
if times.len() < 2 {
|
||||
return Err("the layer must be at least two frames long".into());
|
||||
}
|
||||
let (w, h) = effectcraft_render::source_size(&self.project, l);
|
||||
let size = if w == 0 { [c.width as f64, c.height as f64] } else { [w as f64, h as f64] };
|
||||
let params = static_params(g);
|
||||
let key = tracks_key(&self.project, comp, l, g);
|
||||
let settings = ct::settings(¶ms);
|
||||
let solve_key = ct::solve_key(&settings);
|
||||
let tracks = ct::tracks(¶ms).filter(|_| str_of(g, TRACKS_KEY) == key);
|
||||
let n = times.len();
|
||||
let work = CamWork {
|
||||
project: self.project.clone(),
|
||||
footage: self.footage.clone(),
|
||||
expr: self.expr.clone(),
|
||||
cache: self.layer_cache.clone(),
|
||||
comp,
|
||||
layer,
|
||||
effect_index: index,
|
||||
size,
|
||||
detailed: params.values.iter().any(|(k, v)| k.ends_with("detailedAnalysis") && v.as_bool()),
|
||||
times,
|
||||
tracks: tracks.clone(),
|
||||
settings,
|
||||
};
|
||||
let wait = wait || cfg!(target_arch = "wasm32");
|
||||
let shared = Arc::new(CameraShared::default());
|
||||
{
|
||||
let mut st = lock(&shared.state);
|
||||
st.solve_only = tracks.is_some();
|
||||
st.step = if tracks.is_some() { 2 } else { 1 };
|
||||
st.total = n as u64;
|
||||
}
|
||||
let thread = if wait {
|
||||
run_work(work, &shared);
|
||||
None
|
||||
} else {
|
||||
let sh = shared.clone();
|
||||
Some(std::thread::Builder::new().name("camera-track".into()).spawn(move || run_work(work, &sh)).map_err(|e| e.to_string())?)
|
||||
};
|
||||
self.camera_job = Some(CameraJob { shared, thread, comp, layer, effect, key, solve_key });
|
||||
self.events.push(Event::ProjectChanged { revision: self.revision });
|
||||
self.poll_camera_job();
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// Write a finished analysis into its effect (one undo step) and drop the job.
|
||||
fn poll_camera_job(&mut self) -> bool {
|
||||
let Some(job) = &self.camera_job else { return false };
|
||||
if !job.is_finished() {
|
||||
return false;
|
||||
}
|
||||
let mut job = self.camera_job.take().expect("job");
|
||||
if let Some(t) = job.thread.take() {
|
||||
let _ = t.join();
|
||||
}
|
||||
let st = job.progress();
|
||||
let result = lock(&job.shared.result).take();
|
||||
match (result, &st.error) {
|
||||
(Some(res), None) if !st.cancelled => {
|
||||
let (cid, lid, uid) = (job.comp, job.layer, job.effect);
|
||||
let (key, skey) = (job.key.clone(), job.solve_key.clone());
|
||||
let tracks_json = res.tracks.as_ref().map(CameraTracks::to_json);
|
||||
let (solve_json, method, avg, msg, failed) = match &res.solve {
|
||||
Ok(s) => (
|
||||
s.to_json(),
|
||||
s.method_used.label().to_string(),
|
||||
s.average_error,
|
||||
format!(
|
||||
"3D Camera Tracker: solved {} frame(s), {} points, average error {:.2} px ({:.1} s)",
|
||||
s.frames.len(),
|
||||
s.points.len(),
|
||||
s.average_error,
|
||||
st.elapsed
|
||||
),
|
||||
false,
|
||||
),
|
||||
Err(e) => (String::new(), String::new(), 0.0, format!("3D Camera Tracker: {e}"), true),
|
||||
};
|
||||
let r = self.edit(if st.solve_only { "3D Camera Tracker Solve" } else { "3D Camera Tracker Analysis" }, None, |p, _| {
|
||||
let g = p
|
||||
.comp_mut(cid)
|
||||
.and_then(|c| c.layer_mut(lid))
|
||||
.and_then(|l| l.props.find_group_mut(uid))
|
||||
.ok_or_else(|| crate::EngineError::Other("the 3D Camera Tracker was removed during analysis".into()))?;
|
||||
let mut set = |m: &str, v: Value| {
|
||||
if let Some(pr) = g.prop_mut(m) {
|
||||
pr.value = v;
|
||||
}
|
||||
};
|
||||
if let Some(t) = &tracks_json {
|
||||
set(TRACKS, Value::Str(t.clone()));
|
||||
set(TRACKS_KEY, Value::Str(key.clone()));
|
||||
}
|
||||
set(SOLVE, Value::Str(solve_json.clone()));
|
||||
set(SOLVE_KEY, Value::Str(skey.clone()));
|
||||
set(METHOD_USED, Value::Str(method.clone()));
|
||||
set(AVERAGE_ERROR, Value::Scalar(avg));
|
||||
Ok(())
|
||||
});
|
||||
let (msg, failed) = match r {
|
||||
Ok(()) => (msg, failed),
|
||||
Err(e) => (format!("3D Camera Tracker: {e}"), true),
|
||||
};
|
||||
self.events.push(Event::Toast { message: msg, error: failed });
|
||||
}
|
||||
_ => {
|
||||
let msg = match &st.error {
|
||||
Some(e) => format!("3D Camera Tracker analysis failed: {e}"),
|
||||
None => "3D Camera Tracker analysis cancelled".into(),
|
||||
};
|
||||
self.events.push(Event::Toast { message: msg, error: st.error.is_some() });
|
||||
self.bump();
|
||||
}
|
||||
}
|
||||
true
|
||||
}
|
||||
|
||||
/// Frontends call this every frame: finishes analyses and (with `auto`) starts queued
|
||||
/// (re-)analyses in the background, like After Effects.
|
||||
pub fn poll_camera(&mut self, auto: bool) -> bool {
|
||||
let changed = self.poll_camera_job();
|
||||
if auto && self.camera_job.is_none() {
|
||||
while let Some((c, l, e)) = self.camera_pending.first().copied() {
|
||||
self.camera_pending.remove(0);
|
||||
if self.start_camera(c, l, e, false).is_ok() {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
changed
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- solve → composition
|
||||
|
||||
/// A tracker's solve placed in its composition.
|
||||
pub struct Placed {
|
||||
pub solve: Arc<CameraSolve>,
|
||||
/// Canonical → comp world.
|
||||
pub world: Similarity,
|
||||
/// Comp pixels per layer pixel.
|
||||
pub scale: f64,
|
||||
/// Comp times of the analysed frames.
|
||||
pub times: Vec<Tick>,
|
||||
pub comp_size: [f64; 2],
|
||||
}
|
||||
|
||||
impl Placed {
|
||||
/// Comp-world position of a canonical point.
|
||||
pub fn to_world(&self, x: V3) -> V3 {
|
||||
self.world.apply(x)
|
||||
}
|
||||
/// Canonical position of a comp-world point.
|
||||
pub fn from_world(&self, x: V3) -> V3 {
|
||||
let r = &self.world;
|
||||
linalg::scale(linalg::mtv(&r.r, linalg::sub(x, r.t)), 1.0 / r.s)
|
||||
}
|
||||
/// The 3D Tracker Camera on frame `k`: (position, orientation degrees, zoom).
|
||||
pub fn camera(&self, k: usize) -> (V3, V3, f64) {
|
||||
let c = &self.solve.frames[k];
|
||||
let m = linalg::mm(&self.world.r, &linalg::mt(&c.r()));
|
||||
(self.to_world(c.center), linalg::euler_xyz(&m), c.focal * self.scale)
|
||||
}
|
||||
/// Analysed frame at comp time `t`.
|
||||
pub fn frame_at(&self, t: Tick) -> usize {
|
||||
self.times.iter().enumerate().min_by_key(|(_, x)| (x.0 - t.0).abs()).map(|(i, _)| i).unwrap_or(0).min(self.solve.frames.len().saturating_sub(1))
|
||||
}
|
||||
}
|
||||
|
||||
/// Comp pixels per layer pixel of `layer` (its 2D transform at its In point).
|
||||
fn layer_scale(project: &Project, cid: ItemId, layer: &Layer) -> f64 {
|
||||
let Some(comp) = project.comp(cid) else { return 1.0 };
|
||||
let ctx = EvalCtx::new(project, cid, comp, layer.in_point.max(Tick::ZERO));
|
||||
let m = ctx.world_matrix(layer).0;
|
||||
let s = m[0][0].hypot(m[1][0]);
|
||||
if s.is_finite() && s > 1e-9 { s } else { 1.0 }
|
||||
}
|
||||
|
||||
/// The solve of tracker `uid` on `lid`, placed in the comp.
|
||||
pub fn placed(project: &Project, cid: ItemId, lid: LayerId, uid: Uid) -> Option<Placed> {
|
||||
let comp = project.comp(cid)?;
|
||||
let layer = comp.layer(lid)?;
|
||||
let g = layer.props.find_group(uid).filter(|g| is_tracker(g))?;
|
||||
let solve = ct::solve(&static_params(g))?;
|
||||
let scale = layer_scale(project, cid, layer);
|
||||
let k0 = solve.first_solved()?;
|
||||
let comp_size = [comp.width as f64, comp.height as f64];
|
||||
let world = solve.world(comp_size, solve.frames[k0].focal * scale);
|
||||
let mut times = analysis_times(comp, layer);
|
||||
times.truncate(solve.frames.len());
|
||||
Some(Placed { solve, world, scale, times, comp_size })
|
||||
}
|
||||
|
||||
/// What to create from a target (`camera.createFromSolve`).
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||
pub enum CreateKind {
|
||||
Text,
|
||||
Solid,
|
||||
Null,
|
||||
ShadowCatcher,
|
||||
/// Only the camera.
|
||||
Camera,
|
||||
}
|
||||
|
||||
impl CreateKind {
|
||||
pub fn from_name(s: &str) -> Option<CreateKind> {
|
||||
match s.to_ascii_lowercase().replace([' ', '-', '_'], "").as_str() {
|
||||
"text" => Some(CreateKind::Text),
|
||||
"solid" => Some(CreateKind::Solid),
|
||||
"null" => Some(CreateKind::Null),
|
||||
"shadowcatcher" | "shadow" => Some(CreateKind::ShadowCatcher),
|
||||
"camera" => Some(CreateKind::Camera),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Rotation (layer → world) of a layer lying on a plane with unit `normal` (towards the camera),
|
||||
/// its x axis along `right` projected on the plane: the layer's front (−z) faces the camera.
|
||||
pub fn plane_orientation(normal: V3, right: V3) -> V3 {
|
||||
let z = linalg::scale(normal, -1.0);
|
||||
let mut x = linalg::sub(right, linalg::scale(z, linalg::dot(right, z)));
|
||||
if linalg::norm(x) < 1e-9 {
|
||||
x = if z[0].abs() < 0.9 { [1.0, 0.0, 0.0] } else { [0.0, 1.0, 0.0] };
|
||||
x = linalg::sub(x, linalg::scale(z, linalg::dot(x, z)));
|
||||
}
|
||||
let x = linalg::normalize(x);
|
||||
let y = linalg::cross(z, x);
|
||||
let m = linalg::from_cols(x, y, z);
|
||||
norm_angles(linalg::euler_xyz(&m))
|
||||
}
|
||||
|
||||
fn norm_angles(a: V3) -> V3 {
|
||||
a.map(|v| {
|
||||
let r = v.rem_euclid(360.0);
|
||||
if (r - 360.0).abs() < 1e-9 { 0.0 } else { r }
|
||||
})
|
||||
}
|
||||
|
||||
/// Key `path` on `l` at every `(time, value)`.
|
||||
pub(crate) fn key_all(l: &mut Layer, path: &str, keys: &[(Tick, Value)]) {
|
||||
if let Some(pr) = l.props.prop_mut(path) {
|
||||
pr.keys = keys.iter().map(|(t, v)| Keyframe::new(*t, v.clone())).collect();
|
||||
if let Some((_, v)) = keys.first() {
|
||||
pr.value = v.clone();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Write the solved camera path into camera layer `l` (one-node, keyed on every analysed frame;
|
||||
/// Zoom keyed for Variable Zoom).
|
||||
pub(crate) fn write_camera(l: &mut Layer, pl: &Placed) {
|
||||
l.auto_orient = AutoOrient::Off;
|
||||
let n = pl.times.len().min(pl.solve.frames.len());
|
||||
let mut pos = Vec::with_capacity(n);
|
||||
let mut ori = Vec::with_capacity(n);
|
||||
let mut zoom = Vec::with_capacity(n);
|
||||
let mut prev: Option<V3> = None;
|
||||
for k in 0..n {
|
||||
let (p, o, z) = pl.camera(k);
|
||||
// Unwrap the angles so per-frame keys interpolate the short way.
|
||||
let o = match prev {
|
||||
None => norm_angles(o),
|
||||
Some(q) => [0, 1, 2].map(|i| q[i] + (o[i] - q[i] + 180.0).rem_euclid(360.0) - 180.0),
|
||||
};
|
||||
prev = Some(o);
|
||||
let t = l.layer_time(pl.times[k]);
|
||||
pos.push((t, Value::Vec3(p)));
|
||||
ori.push((t, Value::Vec3(o)));
|
||||
zoom.push((t, Value::Scalar(z)));
|
||||
}
|
||||
key_all(l, "transform/position", &pos);
|
||||
key_all(l, "transform/orientation", &ori);
|
||||
let variable = pl.solve.frames.iter().any(|f| (f.focal - pl.solve.frames[0].focal).abs() > 1e-6 * f.focal);
|
||||
if variable {
|
||||
key_all(l, "cameraOptions/zoom", &zoom);
|
||||
} else if let Some((_, z)) = zoom.first()
|
||||
&& let Some(pr) = l.props.prop_mut("cameraOptions/zoom")
|
||||
{
|
||||
pr.keys.clear();
|
||||
pr.value = z.clone();
|
||||
}
|
||||
if let Some((_, z)) = zoom.first()
|
||||
&& let Some(pr) = l.props.prop_mut("cameraOptions/focusDistance")
|
||||
{
|
||||
pr.value = z.clone();
|
||||
}
|
||||
// A one-node camera has no Point of Interest.
|
||||
if let Some(tr) = l.props.sub_mut("transform") {
|
||||
tr.children.retain(|c| !matches!(c, effectcraft_project::Node::Prop(p) if p.match_id == "poi"));
|
||||
}
|
||||
for rot in ["transform/rotationX", "transform/rotationY", "transform/rotation"] {
|
||||
if let Some(pr) = l.props.prop_mut(rot) {
|
||||
pr.keys.clear();
|
||||
pr.value = Value::Scalar(0.0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The target for `ids` on frame `k`: when fewer than three points are given, the nearest
|
||||
/// solved neighbours (in 3D) complete the plane.
|
||||
pub fn target_for(solve: &CameraSolve, ids: &[u32], k: usize) -> Option<Target> {
|
||||
let mut use_ids: Vec<u32> = ids.iter().copied().filter(|i| solve.point(*i).is_some()).collect();
|
||||
if use_ids.is_empty() {
|
||||
return None;
|
||||
}
|
||||
if use_ids.len() < 3 {
|
||||
let c = solve.point(use_ids[0])?.pos;
|
||||
let mut others: Vec<(f64, u32)> = solve.visible(k).filter(|p| !use_ids.contains(&p.id)).map(|p| (linalg::norm(linalg::sub(p.pos, c)), p.id)).collect();
|
||||
others.sort_by(|a, b| a.0.total_cmp(&b.0));
|
||||
for (_, id) in others.into_iter().take(6usize.saturating_sub(use_ids.len())) {
|
||||
use_ids.push(id);
|
||||
}
|
||||
let mut t = solve.target(&use_ids, k)?;
|
||||
// The target sits on the chosen point(s).
|
||||
let pts: Vec<V3> = ids.iter().filter_map(|i| solve.point(*i)).map(|p| p.pos).collect();
|
||||
t.center = linalg::scale(pts.iter().fold([0.0; 3], |a, b| linalg::add(a, *b)), 1.0 / pts.len() as f64);
|
||||
return Some(t);
|
||||
}
|
||||
solve.target(&use_ids, k)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn plane_orientation_faces_the_normal() {
|
||||
let n = linalg::normalize([0.2, -0.9, -0.3]);
|
||||
let o = plane_orientation(n, [1.0, 0.0, 0.0]);
|
||||
let m = linalg::from_euler_xyz(o);
|
||||
// Layer −z (its front) is the normal.
|
||||
let front = linalg::mv(&m, [0.0, 0.0, -1.0]);
|
||||
assert!(linalg::norm(linalg::sub(front, n)) < 1e-9);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,574 @@
|
||||
//! 3D Camera Tracker commands: Animation ▸ Track Camera (`track.camera`), the effect's Analyze /
|
||||
//! Cancel / Create Camera buttons, the viewer's track point selection and right-click menu
|
||||
//! (Set Ground Plane and Origin, Create Text / Solid / Null / Shadow Catcher … and Camera,
|
||||
//! Create Multiple …, Delete Selected Points) and the status / points queries.
|
||||
|
||||
use effectcraft_color::Label;
|
||||
use effectcraft_effects::camera_tracker::{self as ct, CAMERA, DELETED, SOLVE};
|
||||
use effectcraft_keyframe::{Justify, TextDoc, Value as KV};
|
||||
use effectcraft_project::build;
|
||||
use effectcraft_project::{ItemId, ItemKind, Layer, LayerId, LayerSource, LightKind, Project, Solid, Uid};
|
||||
use effectcraft_track::camtrack::linalg::{self, V3};
|
||||
use effectcraft_track::camtrack::{ShotType, SolveMethod, Target, point_color};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use super::{CommandSpec, b_p, bad, f_p, has_comp, layer_p, str_p};
|
||||
use crate::camera_track::{CreateKind, Placed, is_tracker, placed, plane_orientation, static_params, target_for, write_camera};
|
||||
use crate::{EngineError, Result, Session, cmd, query};
|
||||
|
||||
pub fn specs() -> Vec<CommandSpec> {
|
||||
vec![
|
||||
cmd!(
|
||||
"track.camera",
|
||||
"Track Camera",
|
||||
["Animation"],
|
||||
None,
|
||||
"{layer?, shotType?: fixed|variable|specify, aov?: degrees, solveMethod?: auto|typical|flat|tripod, detailed?, wait?}",
|
||||
can_track,
|
||||
track_camera
|
||||
),
|
||||
cmd!("camera.analyze", "Analyze", [], None, "{layer?, effect?: uid|name|index, wait?: block until done}", has_comp, analyze),
|
||||
cmd!("camera.cancel", "Cancel", [], None, "{}", is_analyzing, |s, _| Ok(json!({"stopped": s.stop_camera()}))),
|
||||
query!("camera.solveStatus", "3D Camera Tracker Status", "{layer?, effect?}", status),
|
||||
query!("camera.points", "3D Camera Tracker Points", "{layer?, effect?, time?: comp seconds} → visible solved points", points),
|
||||
cmd!("camera.selectPoints", "Select Track Points", [], None, "{points: [id], add?, toggle?}", has_comp, select_points),
|
||||
cmd!(
|
||||
"camera.createFromSolve",
|
||||
"Create from Camera Solve",
|
||||
[],
|
||||
None,
|
||||
"{kind: text|solid|null|shadowCatcher|camera, points?: [id] (default: selected), target?: {center, normal, size?} (comp world), multiple?, layer?, effect?}",
|
||||
has_comp,
|
||||
create_from_solve
|
||||
),
|
||||
cmd!("camera.create", "Create Camera", [], None, "{layer?, effect?}", has_comp, |s, p| create_from_solve(s, &with_kind(p, "camera"))),
|
||||
cmd!("camera.setGroundPlane", "Set Ground Plane and Origin", [], None, "{points?: [id] (default: selected), layer?, effect?}", has_comp, set_ground),
|
||||
cmd!("camera.deletePoints", "Delete Selected Points", [], None, "{points?: [id] (default: selected), layer?, effect?, wait?}", has_comp, delete_points),
|
||||
]
|
||||
}
|
||||
|
||||
fn with_kind(p: &Value, kind: &str) -> Value {
|
||||
let mut q = p.clone();
|
||||
if !q.is_object() {
|
||||
q = json!({});
|
||||
}
|
||||
q["kind"] = json!(kind);
|
||||
q
|
||||
}
|
||||
|
||||
fn can_track(s: &Session) -> std::result::Result<(), String> {
|
||||
has_comp(s)?;
|
||||
let c = s.active_comp().ok_or("no composition is open")?;
|
||||
match s.state.selected_layers.first().and_then(|l| c.layer(*l)) {
|
||||
Some(l) if l.has_video() && !l.switches.adjustment && !l.is_camera() && !l.is_light() => Ok(()),
|
||||
Some(_) => Err("Track Camera needs a footage or composition layer".into()),
|
||||
None => Err("select a layer to track".into()),
|
||||
}
|
||||
}
|
||||
|
||||
fn is_analyzing(s: &Session) -> std::result::Result<(), String> {
|
||||
if s.is_camera_analyzing() { Ok(()) } else { Err("no 3D Camera Tracker analysis is running".into()) }
|
||||
}
|
||||
|
||||
/// (comp, layer, effect uid) of a 3D Camera Tracker: `layer` / `effect` (uid, name or 1-based
|
||||
/// index), else a selected one, else the first in the comp.
|
||||
fn ct_p(s: &Session, p: &Value, cmd: &str) -> Result<(ItemId, LayerId, Uid)> {
|
||||
let cid = super::comp_id(s, p)?;
|
||||
let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?;
|
||||
let candidates: Vec<LayerId> = match p.get("layer") {
|
||||
Some(v) => vec![super::resolve_layer(comp, v).ok_or_else(|| bad(cmd, format!("no layer {v}")))?],
|
||||
None => {
|
||||
let mut v: Vec<LayerId> = s.state.selected_props.iter().map(|(l, _)| *l).collect();
|
||||
v.extend(s.state.selected_layers.iter().copied());
|
||||
v.extend(comp.layers.iter().map(|l| l.id));
|
||||
v
|
||||
}
|
||||
};
|
||||
for lid in candidates {
|
||||
let Some(fx) = comp.layer(lid).and_then(Layer::effects) else { continue };
|
||||
let g = match p.get("effect") {
|
||||
Some(Value::Number(n)) => {
|
||||
let n = n.as_u64().unwrap_or(0);
|
||||
fx.groups().find(|g| g.uid == n).or_else(|| fx.groups().nth((n as usize).saturating_sub(1)))
|
||||
}
|
||||
Some(Value::String(name)) => fx.groups().find(|g| &g.name == name),
|
||||
_ => s
|
||||
.state
|
||||
.selected_props
|
||||
.iter()
|
||||
.find_map(|(l, u)| (*l == lid).then(|| fx.groups().find(|g| g.uid == *u && is_tracker(g))).flatten())
|
||||
.or_else(|| fx.groups().find(|g| is_tracker(g))),
|
||||
};
|
||||
if let Some(g) = g.filter(|g| is_tracker(g)) {
|
||||
return Ok((cid, lid, g.uid));
|
||||
}
|
||||
if p.get("layer").is_some() {
|
||||
break;
|
||||
}
|
||||
}
|
||||
Err(bad(cmd, "no 3D Camera Tracker found (apply Animation ▸ Track Camera first)"))
|
||||
}
|
||||
|
||||
fn shot_p(p: &Value) -> Result<Option<ShotType>> {
|
||||
match p.get("shotType") {
|
||||
None => Ok(None),
|
||||
Some(Value::Number(n)) => ShotType::ALL.get(n.as_u64().unwrap_or(99) as usize).copied().map(Some).ok_or_else(|| bad("track.camera", "shotType: 0..2")),
|
||||
Some(Value::String(v)) => ShotType::from_name(v).map(Some).ok_or_else(|| bad("track.camera", "shotType: fixed|variable|specify")),
|
||||
Some(_) => Err(bad("track.camera", "shotType: fixed|variable|specify")),
|
||||
}
|
||||
}
|
||||
|
||||
fn method_p(p: &Value) -> Result<Option<SolveMethod>> {
|
||||
match p.get("solveMethod") {
|
||||
None => Ok(None),
|
||||
Some(Value::Number(n)) => {
|
||||
SolveMethod::ALL.get(n.as_u64().unwrap_or(99) as usize).copied().map(Some).ok_or_else(|| bad("track.camera", "solveMethod: 0..3"))
|
||||
}
|
||||
Some(Value::String(v)) => SolveMethod::from_name(v).map(Some).ok_or_else(|| bad("track.camera", "solveMethod: auto|typical|flat|tripod")),
|
||||
Some(_) => Err(bad("track.camera", "solveMethod: auto|typical|flat|tripod")),
|
||||
}
|
||||
}
|
||||
|
||||
/// Animation ▸ Track Camera / the Tracker panel button: apply the 3D Camera Tracker to the layer
|
||||
/// (reusing one it already has), set the given options and start the analysis.
|
||||
fn track_camera(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let (cid, lid) = layer_p(s, p, "track.camera")?;
|
||||
let shot = shot_p(p)?;
|
||||
let method = method_p(p)?;
|
||||
let aov = f_p(p, "aov").or_else(|| f_p(p, "angleOfView"));
|
||||
if aov.is_some_and(|a| !(1.0..=170.0).contains(&a)) {
|
||||
return Err(bad("track.camera", "aov: 1..170 degrees"));
|
||||
}
|
||||
let detailed = b_p(p, "detailed");
|
||||
let existing = s.project.comp(cid).and_then(|c| c.layer(lid)).and_then(Layer::effects).and_then(|fx| fx.groups().find(|g| is_tracker(g)).map(|g| g.uid));
|
||||
let uid = match existing {
|
||||
Some(u) => u,
|
||||
None => {
|
||||
let r = s.execute("effect.apply", json!({"effect": ct::ID, "layers": [lid.0], "comp": cid.0}))?;
|
||||
r["effects"].get(0).and_then(Value::as_u64).ok_or_else(|| EngineError::Other("the effect could not be applied".into()))?
|
||||
}
|
||||
};
|
||||
if shot.is_some() || method.is_some() || aov.is_some() || detailed.is_some() {
|
||||
s.edit("3D Camera Tracker Settings", None, |proj, _| {
|
||||
let g = proj
|
||||
.comp_mut(cid)
|
||||
.and_then(|c| c.layer_mut(lid))
|
||||
.and_then(|l| l.props.find_group_mut(uid))
|
||||
.ok_or_else(|| EngineError::Other("no 3D Camera Tracker".into()))?;
|
||||
let mut set = |m: &str, v: KV| {
|
||||
if let Some(pr) = g.prop_mut(m) {
|
||||
pr.keys.clear();
|
||||
pr.value = v;
|
||||
}
|
||||
};
|
||||
if let Some(sh) = shot {
|
||||
set("shotType", KV::Enum(ShotType::ALL.iter().position(|x| *x == sh).unwrap_or(0) as u32));
|
||||
}
|
||||
if let Some(a) = aov {
|
||||
set("horizontalAngleOfView", KV::Scalar(a));
|
||||
if shot.is_none() {
|
||||
set("shotType", KV::Enum(2));
|
||||
}
|
||||
}
|
||||
if let Some(m) = method {
|
||||
set("advanced/solveMethod", KV::Enum(SolveMethod::ALL.iter().position(|x| *x == m).unwrap_or(0) as u32));
|
||||
}
|
||||
if let Some(d) = detailed {
|
||||
set("advanced/detailedAnalysis", KV::Bool(d));
|
||||
}
|
||||
Ok(())
|
||||
})?;
|
||||
}
|
||||
// Select the effect so the viewer shows its track points.
|
||||
s.state.selected_props = vec![(lid, uid)];
|
||||
let wait = b_p(p, "wait").unwrap_or(false);
|
||||
let n = s.start_camera(cid, lid, uid, wait).map_err(EngineError::Other)?;
|
||||
let mut out = json!({"layer": lid.0, "effect": uid, "frames": n, "running": s.is_camera_analyzing(), "progress": s.camera_progress()});
|
||||
if wait {
|
||||
out["status"] = status(s, &json!({"layer": lid.0, "effect": uid, "comp": cid.0}))?;
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn analyze(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let (cid, lid, uid) = ct_p(s, p, "camera.analyze")?;
|
||||
let wait = b_p(p, "wait").unwrap_or(false);
|
||||
let n = s.start_camera(cid, lid, uid, wait).map_err(EngineError::Other)?;
|
||||
let mut out = json!({"layer": lid.0, "effect": uid, "frames": n, "running": s.is_camera_analyzing(), "progress": s.camera_progress()});
|
||||
if wait {
|
||||
out["status"] = status(s, &json!({"layer": lid.0, "effect": uid, "comp": cid.0}))?;
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
fn status(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
s.poll_camera(false);
|
||||
let mut out = json!({"running": s.is_camera_analyzing(), "progress": s.camera_progress(), "banner": s.camera_progress().map(|p| p.banner())});
|
||||
let Ok((cid, lid, uid)) = ct_p(s, p, "camera.solveStatus") else { return Ok(out) };
|
||||
let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?;
|
||||
let layer = comp.layer(lid).ok_or(EngineError::NoComp)?;
|
||||
let g = layer.props.find_group(uid).ok_or(EngineError::NoComp)?;
|
||||
let params = static_params(g);
|
||||
out["layer"] = json!(lid.0);
|
||||
out["effect"] = json!(uid);
|
||||
out["pending"] = json!(s.camera_pending.contains(&(cid, lid, uid)));
|
||||
let tracks = ct::tracks(¶ms);
|
||||
out["analyzed"] = json!(tracks.is_some());
|
||||
if let Some(t) = &tracks {
|
||||
out["tracks"] = json!(t.tracks.len());
|
||||
out["frames"] = json!(t.frames);
|
||||
}
|
||||
let settings = ct::settings(¶ms);
|
||||
out["shotType"] = json!(settings.shot.label());
|
||||
out["solveMethod"] = json!(settings.method.label());
|
||||
out["deleted"] = json!(settings.deleted.len());
|
||||
let solve = ct::solve(¶ms);
|
||||
out["solved"] = json!(solve.is_some());
|
||||
if let Some(sv) = &solve {
|
||||
let f = sv.focal();
|
||||
out["methodUsed"] = json!(sv.method_used.label());
|
||||
out["averageError"] = json!(sv.average_error);
|
||||
out["points"] = json!(sv.points.len());
|
||||
out["solvedFrames"] = json!(sv.frames.iter().filter(|c| c.solved).count());
|
||||
out["focalLength"] = json!(f);
|
||||
out["horizontalAngleOfView"] = json!(sv.hfov(f));
|
||||
out["groundPlane"] = json!(sv.ground.is_some());
|
||||
if let Some(pl) = placed(&s.project, cid, lid, uid) {
|
||||
let k = pl.frame_at(s.time_of(cid));
|
||||
let (pos, ori, zoom) = pl.camera(k);
|
||||
out["frame"] = json!(k);
|
||||
out["camera"] = json!({"position": pos, "orientation": ori, "zoom": zoom});
|
||||
out["visible"] = json!(sv.visible(k).count());
|
||||
}
|
||||
}
|
||||
let cam = params.get(CAMERA).map(|v| v.as_f64() as u64).unwrap_or(0);
|
||||
out["cameraLayer"] = if cam > 0 && comp.layer(LayerId(cam)).is_some_and(Layer::is_camera) { json!(cam) } else { Value::Null };
|
||||
out["selected"] = json!(s.state.camera_points);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// The visible solved points at the current (or given) time, with their comp positions.
|
||||
fn points(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let (cid, lid, uid) = ct_p(s, p, "camera.points")?;
|
||||
let pl = placed(&s.project, cid, lid, uid).ok_or_else(|| bad("camera.points", "the camera is not solved yet"))?;
|
||||
let t = f_p(p, "time").map(effectcraft_time::Tick::from_seconds_f64).unwrap_or_else(|| s.time_of(cid));
|
||||
let k = pl.frame_at(t);
|
||||
let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?;
|
||||
let layer = comp.layer(lid).ok_or(EngineError::NoComp)?;
|
||||
let ctx = effectcraft_render::EvalCtx::new(&s.project, cid, comp, t);
|
||||
let m = ctx.world_matrix(layer);
|
||||
let list: Vec<Value> = ct::projected(&pl.solve, k)
|
||||
.into_iter()
|
||||
.map(|(id, q, z)| {
|
||||
let c = m.apply(effectcraft_geom::vec3(q[0], q[1], 0.0));
|
||||
let sp = pl.solve.point(id).copied().unwrap_or_default();
|
||||
let col = point_color(id);
|
||||
json!({"id": id, "comp": [c.x, c.y], "layer": q, "depth": z, "world": pl.to_world(sp.pos), "error": sp.error, "color": col, "selected": s.state.camera_points.contains(&id)})
|
||||
})
|
||||
.collect();
|
||||
Ok(json!({"frame": k, "points": list}))
|
||||
}
|
||||
|
||||
fn ids_p(s: &Session, p: &Value) -> Vec<u32> {
|
||||
match p.get("points") {
|
||||
Some(Value::Array(a)) => a.iter().filter_map(Value::as_u64).map(|v| v as u32).collect(),
|
||||
Some(Value::Number(n)) => n.as_u64().map(|v| vec![v as u32]).unwrap_or_default(),
|
||||
_ => s.state.camera_points.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
fn select_points(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let ids: Vec<u32> = match p.get("points") {
|
||||
Some(Value::Array(a)) => a.iter().filter_map(Value::as_u64).map(|v| v as u32).collect(),
|
||||
_ => vec![],
|
||||
};
|
||||
if b_p(p, "toggle").unwrap_or(false) {
|
||||
for id in ids {
|
||||
match s.state.camera_points.iter().position(|x| *x == id) {
|
||||
Some(i) => {
|
||||
s.state.camera_points.remove(i);
|
||||
}
|
||||
None => s.state.camera_points.push(id),
|
||||
}
|
||||
}
|
||||
} else if b_p(p, "add").unwrap_or(false) {
|
||||
for id in ids {
|
||||
if !s.state.camera_points.contains(&id) {
|
||||
s.state.camera_points.push(id);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
s.state.camera_points = ids;
|
||||
}
|
||||
s.bump();
|
||||
Ok(json!({"selected": s.state.camera_points}))
|
||||
}
|
||||
|
||||
/// The target of a create / ground command in canonical solve coordinates.
|
||||
fn target_p(s: &Session, p: &Value, pl: &Placed, k: usize, cmd: &str) -> Result<Target> {
|
||||
if let Some(t) = p.get("target") {
|
||||
let v3 = |k: &str| -> Option<V3> {
|
||||
let a = t.get(k)?.as_array()?;
|
||||
Some([a.first()?.as_f64()?, a.get(1)?.as_f64()?, a.get(2).and_then(Value::as_f64).unwrap_or(0.0)])
|
||||
};
|
||||
let (c, n) = (v3("center").ok_or_else(|| bad(cmd, "target.center: [x, y, z]"))?, v3("normal").unwrap_or([0.0, -1.0, 0.0]));
|
||||
let size = t.get("size").and_then(Value::as_f64).unwrap_or(100.0);
|
||||
let nc = linalg::normalize(linalg::mtv(&pl.world.r, n));
|
||||
return Ok(Target { center: pl.from_world(c), normal: nc, size: size / pl.world.s });
|
||||
}
|
||||
let ids = ids_p(s, p);
|
||||
if ids.is_empty() {
|
||||
return Err(bad(cmd, "select track points (or give `points` / `target`)"));
|
||||
}
|
||||
target_for(&pl.solve, &ids, k).ok_or_else(|| bad(cmd, "those points do not define a plane"))
|
||||
}
|
||||
|
||||
fn next_name(c: &effectcraft_project::Comp, base: &str) -> String {
|
||||
let n = (1..).find(|i| !c.layers.iter().any(|l| l.name == format!("{base} {i}"))).unwrap_or(1);
|
||||
format!("{base} {n}")
|
||||
}
|
||||
|
||||
fn place(l: &mut Layer, pos: V3, ori: V3, src: &Layer) {
|
||||
l.switches.three_d = true;
|
||||
l.in_point = src.in_point;
|
||||
l.out_point = src.out_point;
|
||||
if let Some(pr) = l.props.prop_mut("transform/position") {
|
||||
pr.value = KV::Vec3(pos);
|
||||
}
|
||||
if let Some(pr) = l.props.prop_mut("transform/orientation") {
|
||||
pr.value = KV::Vec3(ori);
|
||||
}
|
||||
}
|
||||
|
||||
/// Insert `l` above the tracked layer.
|
||||
fn insert_above(proj: &mut Project, cid: ItemId, above: LayerId, l: Layer) -> Result<LayerId> {
|
||||
let comp = proj.comp_mut(cid).ok_or(EngineError::NoComp)?;
|
||||
let at = comp.layers.iter().position(|x| x.id == above).unwrap_or(0);
|
||||
let id = l.id;
|
||||
comp.layers.insert(at, l);
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
fn add_solid(proj: &mut Project, comp: &effectcraft_project::Comp, name: &str, size: u32, color: [f32; 3]) -> Layer {
|
||||
let folder = proj.folder_named("Solids").unwrap_or_else(|| proj.add_item("Solids", Label::Yellow, None, ItemKind::Folder));
|
||||
let sid = proj.add_item(name, Label::Red, Some(folder), ItemKind::Solid(Solid { color, width: size, height: size, pixel_aspect: 1.0 }));
|
||||
build::layer(proj, comp, name, LayerSource::Solid { item: sid }, (size, size), None)
|
||||
}
|
||||
|
||||
/// Create Text / Solid / Null / Shadow Catcher (and Light) on the target, plus the 3D Tracker
|
||||
/// Camera when the tracker has none yet — one undo step.
|
||||
fn create_from_solve(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let cmd = "camera.createFromSolve";
|
||||
let (cid, lid, uid) = ct_p(s, p, cmd)?;
|
||||
let kind = CreateKind::from_name(str_p(p, "kind").unwrap_or("null")).ok_or_else(|| bad(cmd, "kind: text|solid|null|shadowCatcher|camera"))?;
|
||||
let pl = placed(&s.project, cid, lid, uid).ok_or_else(|| bad(cmd, "the camera is not solved yet (analysis still running or failed)"))?;
|
||||
let k = pl.frame_at(s.time_of(cid));
|
||||
let multiple = b_p(p, "multiple").unwrap_or(false);
|
||||
let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?.clone();
|
||||
let src = comp.layer(lid).ok_or(EngineError::NoComp)?.clone();
|
||||
let g = src.props.find_group(uid).ok_or(EngineError::NoComp)?;
|
||||
let params = static_params(g);
|
||||
let size_pct = effectcraft_effects::warp_stab::param(¶ms, "targetSize").map(|v| v.as_f64()).unwrap_or(100.0) / 100.0;
|
||||
let existing_cam = params.get(CAMERA).map(|v| v.as_f64() as u64).filter(|id| *id > 0 && comp.layer(LayerId(*id)).is_some_and(Layer::is_camera));
|
||||
// Targets: one, or one per selected point (Create Multiple …) sharing the selection's plane.
|
||||
let targets: Vec<Target> = if kind == CreateKind::Camera {
|
||||
vec![]
|
||||
} else {
|
||||
let t = target_p(s, p, &pl, k, cmd)?;
|
||||
if multiple {
|
||||
let ids = ids_p(s, p);
|
||||
ids.iter().filter_map(|i| pl.solve.point(*i)).map(|sp| Target { center: sp.pos, ..t }).collect()
|
||||
} else {
|
||||
vec![t]
|
||||
}
|
||||
};
|
||||
// Camera right vector on this frame (world), to align created layers with the view.
|
||||
let cam = &pl.solve.frames[k];
|
||||
let right = linalg::mv(&pl.world.r, linalg::mtv(&cam.r(), [1.0, 0.0, 0.0]));
|
||||
let label = match (kind, multiple) {
|
||||
(CreateKind::Text, false) => "Create Text and Camera",
|
||||
(CreateKind::Solid, false) => "Create Solid and Camera",
|
||||
(CreateKind::Null, false) => "Create Null and Camera",
|
||||
(CreateKind::ShadowCatcher, _) => "Create Shadow Catcher, Camera and Light",
|
||||
(CreateKind::Text, true) => "Create Multiple Text Layers and Camera",
|
||||
(CreateKind::Solid, true) => "Create Multiple Solids and Camera",
|
||||
(CreateKind::Null, true) => "Create Multiple Nulls and Camera",
|
||||
(CreateKind::Camera, _) => "Create Camera",
|
||||
};
|
||||
let (camera, created) = s.edit(label, None, |proj, st| {
|
||||
let mut created = vec![];
|
||||
// The camera: reuse the tracker's, else create "3D Tracker Camera" at the top.
|
||||
let camera = match existing_cam {
|
||||
Some(id) if kind != CreateKind::Camera => LayerId(id),
|
||||
Some(id) => {
|
||||
let l = proj.comp_mut(cid).and_then(|c| c.layer_mut(LayerId(id))).ok_or(EngineError::NoComp)?;
|
||||
write_camera(l, &pl);
|
||||
LayerId(id)
|
||||
}
|
||||
None => {
|
||||
let mut l = build::layer(proj, &comp, "3D Tracker Camera", LayerSource::Camera, (comp.width, comp.height), None);
|
||||
l.in_point = src.in_point;
|
||||
l.out_point = src.out_point;
|
||||
write_camera(&mut l, &pl);
|
||||
let id = l.id;
|
||||
proj.comp_mut(cid).ok_or(EngineError::NoComp)?.layers.insert(0, l);
|
||||
if let Some(pr) = proj.comp_mut(cid).and_then(|c| c.layer_mut(lid)).and_then(|l| l.props.find_group_mut(uid)).and_then(|g| g.prop_mut(CAMERA)) {
|
||||
pr.value = KV::Scalar(id.0 as f64);
|
||||
}
|
||||
id
|
||||
}
|
||||
};
|
||||
for t in &targets {
|
||||
let center = pl.to_world(t.center);
|
||||
let normal = linalg::normalize(linalg::mv(&pl.world.r, t.normal));
|
||||
let ori = plane_orientation(normal, right);
|
||||
let size = (t.size * pl.world.s * size_pct).max(1.0);
|
||||
let cur = proj.comp(cid).ok_or(EngineError::NoComp)?.clone();
|
||||
match kind {
|
||||
CreateKind::Text => {
|
||||
let mut l = build::layer(proj, &cur, "Text", LayerSource::Text, (comp.width, comp.height), None);
|
||||
let doc = TextDoc { text: "Text".into(), justify: Justify::Center, size: (size * 0.35).clamp(4.0, 2000.0), ..Default::default() };
|
||||
if let Some(pr) = l.props.prop_mut("text/sourceText") {
|
||||
pr.value = KV::Text(Box::new(doc));
|
||||
}
|
||||
place(&mut l, center, ori, &src);
|
||||
created.push(insert_above(proj, cid, lid, l)?);
|
||||
}
|
||||
CreateKind::Solid => {
|
||||
let name = next_name(&cur, "Track Solid");
|
||||
let c = point_color((created.len() as u32).wrapping_add(7));
|
||||
let mut l = add_solid(proj, &cur, &name, size.round().max(1.0) as u32, c);
|
||||
place(&mut l, center, ori, &src);
|
||||
created.push(insert_above(proj, cid, lid, l)?);
|
||||
}
|
||||
CreateKind::Null => {
|
||||
let name = next_name(&cur, "Track Null");
|
||||
let mut l = build::layer(proj, &cur, &name, LayerSource::Null, (comp.width, comp.height), None);
|
||||
place(&mut l, center, ori, &src);
|
||||
if let Some(pr) = l.props.prop_mut("transform/scale") {
|
||||
pr.value = KV::Vec3([size, size, size]);
|
||||
}
|
||||
created.push(insert_above(proj, cid, lid, l)?);
|
||||
}
|
||||
CreateKind::ShadowCatcher => {
|
||||
let name = next_name(&cur, "Shadow Catcher");
|
||||
let mut l = add_solid(proj, &cur, &name, (size * 4.0).round().max(1.0) as u32, [1.0, 1.0, 1.0]);
|
||||
place(&mut l, center, ori, &src);
|
||||
if let Some(pr) = l.props.prop_mut("materialOptions/acceptsShadows") {
|
||||
pr.value = KV::Enum(2);
|
||||
}
|
||||
if let Some(pr) = l.props.prop_mut("materialOptions/castsShadows") {
|
||||
pr.value = KV::Enum(0);
|
||||
}
|
||||
let sc = created.len();
|
||||
created.push(insert_above(proj, cid, lid, l)?);
|
||||
let cur = proj.comp(cid).ok_or(EngineError::NoComp)?.clone();
|
||||
let lname = next_name(&cur, "Light");
|
||||
let mut light = build::layer(proj, &cur, &lname, LayerSource::Light { kind: LightKind::Point }, (comp.width, comp.height), None);
|
||||
light.in_point = src.in_point;
|
||||
light.out_point = src.out_point;
|
||||
let lp = linalg::add(center, linalg::add(linalg::scale(normal, size * 2.0), linalg::scale(right, size)));
|
||||
if let Some(pr) = light.props.prop_mut("transform/position") {
|
||||
pr.value = KV::Vec3(lp);
|
||||
}
|
||||
if let Some(pr) = light.props.prop_mut("lightOptions/castsShadows") {
|
||||
pr.value = KV::Bool(true);
|
||||
}
|
||||
let _ = sc;
|
||||
created.push(insert_above(proj, cid, created[created.len() - 1], light)?);
|
||||
}
|
||||
CreateKind::Camera => {}
|
||||
}
|
||||
}
|
||||
st.selected_layers = if created.is_empty() { vec![camera] } else { created.clone() };
|
||||
st.selected_props.clear();
|
||||
Ok((camera, created))
|
||||
})?;
|
||||
let t0 = targets.first().map(
|
||||
|t| json!({"center": pl.to_world(t.center), "normal": linalg::normalize(linalg::mv(&pl.world.r, t.normal)), "size": t.size * pl.world.s * size_pct}),
|
||||
);
|
||||
Ok(json!({"camera": camera.0, "layers": created.iter().map(|l| l.0).collect::<Vec<_>>(), "target": t0}))
|
||||
}
|
||||
|
||||
fn set_ground(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let cmd = "camera.setGroundPlane";
|
||||
let (cid, lid, uid) = ct_p(s, p, cmd)?;
|
||||
let pl = placed(&s.project, cid, lid, uid).ok_or_else(|| bad(cmd, "the camera is not solved yet"))?;
|
||||
let k = pl.frame_at(s.time_of(cid));
|
||||
let ids = ids_p(s, p);
|
||||
if ids.is_empty() {
|
||||
return Err(bad(cmd, "select track points (or give `points`)"));
|
||||
}
|
||||
let t = target_for(&pl.solve, &ids, k).ok_or_else(|| bad(cmd, "those points do not define a plane"))?;
|
||||
let mut sv = (*pl.solve).clone();
|
||||
let mut g = sv.ground_from(&ids, k).ok_or_else(|| bad(cmd, "those points do not define a plane"))?;
|
||||
g.origin = t.center;
|
||||
sv.ground = Some(g);
|
||||
let json = sv.to_json();
|
||||
s.edit("Set Ground Plane and Origin", None, |proj, _| {
|
||||
let pr = proj
|
||||
.comp_mut(cid)
|
||||
.and_then(|c| c.layer_mut(lid))
|
||||
.and_then(|l| l.props.find_group_mut(uid))
|
||||
.and_then(|g| g.prop_mut(SOLVE))
|
||||
.ok_or_else(|| EngineError::Other("no 3D Camera Tracker".into()))?;
|
||||
pr.value = KV::Str(json);
|
||||
Ok(())
|
||||
})?;
|
||||
Ok(json!({"origin": g.origin, "normal": g.normal}))
|
||||
}
|
||||
|
||||
/// Delete points (and, with Auto-delete Points Across Time, the points of other tracks on the
|
||||
/// same 3D feature), then re-solve.
|
||||
fn delete_points(s: &mut Session, p: &Value) -> Result<Value> {
|
||||
let cmd = "camera.deletePoints";
|
||||
let (cid, lid, uid) = ct_p(s, p, cmd)?;
|
||||
let ids = ids_p(s, p);
|
||||
if ids.is_empty() {
|
||||
return Err(bad(cmd, "select track points (or give `points`)"));
|
||||
}
|
||||
let comp = s.project.comp(cid).ok_or(EngineError::NoComp)?;
|
||||
let g = comp.layer(lid).and_then(|l| l.props.find_group(uid)).ok_or(EngineError::NoComp)?;
|
||||
let params = static_params(g);
|
||||
let mut del = ct::deleted(¶ms);
|
||||
let auto = effectcraft_effects::warp_stab::param(¶ms, "advanced/autoDeletePoints").is_some_and(KV::as_bool);
|
||||
let mut add = ids.clone();
|
||||
if auto && let Some(sv) = ct::solve(¶ms) {
|
||||
// Same feature tracked again at other times: solved within 1 % of its depth.
|
||||
let cam0 = sv.first_solved().map(|k| sv.frames[k].center).unwrap_or([0.0; 3]);
|
||||
for id in &ids {
|
||||
let Some(pt) = sv.point(*id) else { continue };
|
||||
let r = linalg::norm(linalg::sub(pt.pos, cam0)) * 0.01;
|
||||
for q in &sv.points {
|
||||
let disjoint = q.last < pt.first || q.first > pt.last;
|
||||
if q.id != *id && disjoint && linalg::norm(linalg::sub(q.pos, pt.pos)) < r {
|
||||
add.push(q.id);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for id in add {
|
||||
if !del.contains(&id) {
|
||||
del.push(id);
|
||||
}
|
||||
}
|
||||
del.sort_unstable();
|
||||
let json = serde_json::to_string(&del).unwrap_or_default();
|
||||
s.edit("Delete Track Points", None, |proj, _| {
|
||||
let pr = proj
|
||||
.comp_mut(cid)
|
||||
.and_then(|c| c.layer_mut(lid))
|
||||
.and_then(|l| l.props.find_group_mut(uid))
|
||||
.and_then(|g| g.prop_mut(DELETED))
|
||||
.ok_or_else(|| EngineError::Other("no 3D Camera Tracker".into()))?;
|
||||
pr.value = KV::Str(json);
|
||||
Ok(())
|
||||
})?;
|
||||
s.state.camera_points.retain(|x| !del.contains(x));
|
||||
// Re-solve now (After Effects re-solves after deleting points).
|
||||
let wait = b_p(p, "wait").unwrap_or(false);
|
||||
if !s.is_camera_analyzing() {
|
||||
s.start_camera(cid, lid, uid, wait).map_err(EngineError::Other)?;
|
||||
}
|
||||
Ok(json!({"deleted": del, "running": s.is_camera_analyzing()}))
|
||||
}
|
||||
@@ -4,6 +4,7 @@
|
||||
mod anim;
|
||||
pub mod anim_tools;
|
||||
mod animation;
|
||||
mod camera_cmds;
|
||||
mod comp;
|
||||
mod comp_more;
|
||||
mod edit;
|
||||
@@ -129,6 +130,7 @@ pub fn command_specs() -> &'static [CommandSpec] {
|
||||
v.extend(track::specs());
|
||||
v.extend(mask_interp::specs());
|
||||
v.extend(warp_cmds::specs());
|
||||
v.extend(camera_cmds::specs());
|
||||
v.extend(paint::specs());
|
||||
v.extend(puppet::specs());
|
||||
v.extend(settings::specs());
|
||||
|
||||
@@ -59,7 +59,6 @@ pub fn specs() -> Vec<CommandSpec> {
|
||||
stub!("keys.audioToKeyframes", "Convert Audio to Keyframes", ["Animation", "Keyframe Assistant"], "{}"),
|
||||
stub!("keys.rpfCameraImport", "RPF Camera Import", ["Animation", "Keyframe Assistant"], "{}"),
|
||||
stub!("text.animatorFontAxes", "Variable Font Axes", ["Animation", "Animate Text"], "{}"),
|
||||
stub!("track.camera", "Track Camera", ["Animation"], "{}"),
|
||||
// Workspaces and panels that don't exist yet.
|
||||
stub!("window.unavailablePanel", "Panel", [], "{panel}"),
|
||||
]
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
//! (compositions are `Arc`s, so untouched comps are shared).
|
||||
|
||||
pub mod autosave;
|
||||
pub mod camera_track;
|
||||
pub mod commands;
|
||||
pub mod config;
|
||||
pub mod demo;
|
||||
@@ -41,6 +42,7 @@ pub use effectcraft_project as project;
|
||||
pub use effectcraft_render as render;
|
||||
pub use effectcraft_text as text;
|
||||
pub use effectcraft_time as time;
|
||||
pub use effectcraft_track as track;
|
||||
pub use render_queue::{ExportJob, ExportResult, Exporter, JobState};
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
@@ -170,6 +172,9 @@ pub struct EditorState {
|
||||
/// Tracker panel ▸ Method in mask mode (Track Mask).
|
||||
#[serde(default)]
|
||||
pub mask_track_method: mask_track::MaskMethod,
|
||||
/// 3D Camera Tracker: the selected track points (ids) in the viewer.
|
||||
#[serde(default)]
|
||||
pub camera_points: Vec<u32>,
|
||||
/// Mask Interpolation panel options.
|
||||
#[serde(default)]
|
||||
pub mask_interp: commands::mask_interp::MaskInterpOptions,
|
||||
@@ -263,6 +268,10 @@ pub struct Session {
|
||||
pub warp_job: Option<warp::WarpJob>,
|
||||
/// Warp Stabilizers waiting for (re-)analysis: (comp, layer, effect uid).
|
||||
pub warp_pending: Vec<(ItemId, LayerId, Uid)>,
|
||||
/// The running (or finished, not yet polled) 3D Camera Tracker analysis.
|
||||
pub camera_job: Option<camera_track::CameraJob>,
|
||||
/// 3D Camera Trackers waiting for (re-)analysis: (comp, layer, effect uid).
|
||||
pub camera_pending: Vec<(ItemId, LayerId, Uid)>,
|
||||
pub events: Vec<Event>,
|
||||
/// Commands executed: (id, params).
|
||||
pub journal: Vec<(String, Value)>,
|
||||
@@ -308,6 +317,8 @@ impl Default for Session {
|
||||
mask_job: None,
|
||||
warp_job: None,
|
||||
warp_pending: vec![],
|
||||
camera_job: None,
|
||||
camera_pending: vec![],
|
||||
events: vec![],
|
||||
journal: vec![],
|
||||
layer_cache: Arc::new(LayerCache::default()),
|
||||
@@ -374,6 +385,12 @@ impl Session {
|
||||
self.warp_pending.push(w);
|
||||
}
|
||||
}
|
||||
// So are 3D Camera Tracker tracks (and solves made with other settings).
|
||||
for w in camera_track::invalidate(&before, &mut p) {
|
||||
if !self.camera_pending.contains(&w) {
|
||||
self.camera_pending.push(w);
|
||||
}
|
||||
}
|
||||
let same = merge.is_some() && merge.map(str::to_string) == self.history.merge_key;
|
||||
if !same {
|
||||
self.history.undo.push((label.to_string(), before));
|
||||
@@ -551,6 +568,9 @@ impl Session {
|
||||
self.stop_warp();
|
||||
self.warp_job = None;
|
||||
self.warp_pending.clear();
|
||||
self.stop_camera();
|
||||
self.camera_job = None;
|
||||
self.camera_pending.clear();
|
||||
p.fix_next_id();
|
||||
self.project = Arc::new(p);
|
||||
self.history = History::default();
|
||||
@@ -574,6 +594,8 @@ mod tests_3d;
|
||||
#[cfg(test)]
|
||||
mod tests_anim_tools;
|
||||
#[cfg(test)]
|
||||
mod tests_camera_track;
|
||||
#[cfg(test)]
|
||||
mod tests_effects;
|
||||
#[cfg(test)]
|
||||
mod tests_fidelity;
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
//! The 3D Camera Tracker end to end through the commands, on synthetic footage rendered in-test:
|
||||
//! textured planes (a floor, a back wall and a box face) seen through a known moving camera.
|
||||
|
||||
use effectcraft_effects::camera_tracker as ct;
|
||||
use effectcraft_geom::vec3;
|
||||
use effectcraft_project::LayerId;
|
||||
use effectcraft_raster::Image;
|
||||
use effectcraft_render::EvalCtx;
|
||||
use effectcraft_render::three_d::camera::active_camera;
|
||||
use effectcraft_track::camtrack::linalg::{self, M3, V3};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::Session;
|
||||
use crate::tests_track::{H, W, frame_time, setup, texture};
|
||||
|
||||
const F: f64 = 260.0;
|
||||
|
||||
/// The true camera on frame `f`: (world → camera rotation, centre).
|
||||
fn true_camera(f: u32) -> (M3, V3) {
|
||||
let t = f as f64 / 24.0;
|
||||
let c = [-1.0 + 2.0 * t, -0.3 + 0.2 * (t * 3.0).sin(), 0.5 * t];
|
||||
let fwd = linalg::normalize(linalg::sub([0.2 * t, 0.4, 8.0], c));
|
||||
let right = linalg::normalize(linalg::cross([0.0, 1.0, 0.0], fwd));
|
||||
([right, linalg::cross(fwd, right), fwd], c)
|
||||
}
|
||||
|
||||
/// Planes: (point, normal, u axis, v axis, half extent or 0 = infinite).
|
||||
const PLANES: [(V3, V3, V3, V3, f64); 3] = [
|
||||
([0.0, 1.5, 0.0], [0.0, -1.0, 0.0], [1.0, 0.0, 0.0], [0.0, 0.0, 1.0], 0.0),
|
||||
([0.0, 0.0, 10.0], [0.0, 0.0, -1.0], [1.0, 0.0, 0.0], [0.0, 1.0, 0.0], 0.0),
|
||||
([-1.2, 0.2, 6.0], [0.3, 0.0, -1.0], [0.957_826, 0.0, 0.287_348], [0.0, 1.0, 0.0], 1.1),
|
||||
];
|
||||
|
||||
fn scene_frame(f: u32) -> Image {
|
||||
let (r, c) = true_camera(f);
|
||||
let mut img = Image::new(W, H);
|
||||
for y in 0..H {
|
||||
for x in 0..W {
|
||||
let d = linalg::mtv(&r, [(x as f64 + 0.5 - W as f64 / 2.0) / F, (y as f64 + 0.5 - H as f64 / 2.0) / F, 1.0]);
|
||||
let mut best = (f64::INFINITY, 0.2f32);
|
||||
for (i, (p0, n, ua, va, ext)) in PLANES.iter().enumerate() {
|
||||
let den = linalg::dot(*n, d);
|
||||
if den.abs() < 1e-9 {
|
||||
continue;
|
||||
}
|
||||
let tt = linalg::dot(*n, linalg::sub(*p0, c)) / den;
|
||||
if tt <= 0.1 || tt >= best.0 {
|
||||
continue;
|
||||
}
|
||||
let hit = linalg::add(c, linalg::scale(d, tt));
|
||||
let (u, v) = (linalg::dot(linalg::sub(hit, *p0), *ua), linalg::dot(linalg::sub(hit, *p0), *va));
|
||||
if *ext > 0.0 && (u.abs() > *ext || v.abs() > *ext) {
|
||||
continue;
|
||||
}
|
||||
best = (tt, texture(u * 40.0, v * 40.0, 11 + i as u32));
|
||||
}
|
||||
img.set(x, y, [best.1, best.1, best.1, 1.0]);
|
||||
}
|
||||
}
|
||||
img
|
||||
}
|
||||
|
||||
/// The comp with the clip tracked (blocking) with `params`.
|
||||
fn tracked(params: Value) -> (Session, LayerId) {
|
||||
let (mut s, clip, _) = setup(scene_frame);
|
||||
s.execute("layer.select", json!({"layers": [clip.0]})).unwrap();
|
||||
assert!(s.is_enabled("track.camera"));
|
||||
let mut p = params;
|
||||
p["wait"] = json!(true);
|
||||
let r = s.execute_checked("track.camera", p).unwrap();
|
||||
assert_eq!(r["frames"], 25, "{r}");
|
||||
assert!(s.camera_job.is_none());
|
||||
(s, clip)
|
||||
}
|
||||
|
||||
fn status(s: &mut Session) -> Value {
|
||||
s.execute("camera.solveStatus", json!({})).unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn track_camera_solves_and_creates_layers_on_the_plane() {
|
||||
let (mut s, clip) = tracked(json!({}));
|
||||
let st = status(&mut s);
|
||||
assert_eq!(st["solved"], true, "{st}");
|
||||
let err = st["averageError"].as_f64().unwrap();
|
||||
assert!(err < 0.5, "average error {err}");
|
||||
let f = st["focalLength"].as_f64().unwrap();
|
||||
assert!((f - F).abs() / F < 0.05, "focal {f}");
|
||||
assert_ne!(st["methodUsed"], "Tripod Pan");
|
||||
// Method Used / Average Error are reported in the effect.
|
||||
let g = s.active_comp().unwrap().layer(clip).unwrap().effects().unwrap().groups().next().unwrap().clone();
|
||||
assert!(matches!(g.prop(ct::METHOD_USED).map(|p| &p.value), Some(effectcraft_keyframe::Value::Str(m)) if !m.is_empty()));
|
||||
|
||||
// Pick floor points on frame 12 (their 3D positions lie on y = 1.5 in the true scene: the
|
||||
// ones seen below the horizon at the bottom of the frame).
|
||||
s.execute("time.set", json!({"time": frame_time(12).seconds()})).unwrap();
|
||||
let pts = s.execute("camera.points", json!({})).unwrap();
|
||||
let list = pts["points"].as_array().unwrap();
|
||||
assert!(list.len() > 30, "{}", list.len());
|
||||
let floor: Vec<u64> = list.iter().filter(|p| p["comp"][1].as_f64().unwrap() > H as f64 * 0.8).map(|p| p["id"].as_u64().unwrap()).take(8).collect();
|
||||
assert!(floor.len() >= 3, "{floor:?}");
|
||||
s.execute("camera.selectPoints", json!({"points": floor})).unwrap();
|
||||
assert_eq!(s.state.camera_points.len(), floor.len());
|
||||
|
||||
// Create Solid and Camera: one undo step, camera + solid.
|
||||
let r = s.execute_checked("camera.createFromSolve", json!({"kind": "solid"})).unwrap();
|
||||
let cam = LayerId(r["camera"].as_u64().unwrap());
|
||||
let solid = LayerId(r["layers"][0].as_u64().unwrap());
|
||||
let comp = s.active_comp().unwrap().clone();
|
||||
assert!(comp.layer(cam).unwrap().is_camera());
|
||||
assert_eq!(comp.layer(cam).unwrap().name, "3D Tracker Camera");
|
||||
assert_eq!(comp.layers[0].id, cam);
|
||||
let sl = comp.layer(solid).unwrap();
|
||||
assert!(sl.switches.three_d);
|
||||
assert!(sl.name.starts_with("Track Solid"));
|
||||
// The camera is keyed on every frame.
|
||||
assert_eq!(comp.layer(cam).unwrap().props.prop("transform/position").unwrap().keys.len(), 25);
|
||||
|
||||
// Every solved point projects through the created camera exactly where the solve puts it,
|
||||
// and onto its tracked position (the footage fills the comp, so layer pixels are comp
|
||||
// pixels): within 1 px for 95 % of the observations (the rest are the solve's outliers:
|
||||
// features on occlusion edges), 0.5 px on average.
|
||||
let cid = s.active_comp_id().unwrap();
|
||||
let uid = g.uid;
|
||||
let pl = crate::camera_track::placed(&s.project, cid, clip, uid).unwrap();
|
||||
let tracks = ct::tracks(&crate::camera_track::static_params(&g)).unwrap();
|
||||
let mut errs = vec![];
|
||||
for k in [0u32, 6, 12, 18, 24] {
|
||||
let ctx = EvalCtx::new(&s.project, cid, &comp, frame_time(k));
|
||||
let camst = active_camera(&ctx);
|
||||
for sp in pl.solve.visible(k as usize) {
|
||||
let Some(obs) = tracks.track(sp.id).and_then(|t| t.at(k)) else { continue };
|
||||
let w = pl.to_world(sp.pos);
|
||||
let q = camst.project(W as f64, H as f64, vec3(w[0], w[1], w[2])).unwrap();
|
||||
let own = pl.solve.frames[k as usize].project(pl.solve.size, sp.pos).unwrap();
|
||||
assert!((q.x - own[0]).hypot(q.y - own[1]) < 0.01, "camera layer disagrees with the solve");
|
||||
errs.push((q.x - obs[0]).hypot(q.y - obs[1]));
|
||||
}
|
||||
}
|
||||
assert!(errs.len() > 100, "{}", errs.len());
|
||||
errs.sort_by(f64::total_cmp);
|
||||
let p95 = errs[errs.len() * 95 / 100];
|
||||
let mean = errs.iter().sum::<f64>() / errs.len() as f64;
|
||||
assert!(p95 < 1.0 && mean < 0.5, "p95 {p95} px, mean {mean} px");
|
||||
|
||||
// The solid lies on the selected points' plane.
|
||||
let tr = sl.props.sub("transform").unwrap();
|
||||
let pos = tr.get("position").unwrap().value.as_vec3();
|
||||
let ori = tr.get("orientation").unwrap().value.as_vec3();
|
||||
let normal = linalg::mv(&linalg::from_euler_xyz(ori), [0.0, 0.0, -1.0]);
|
||||
let size = r["target"]["size"].as_f64().unwrap();
|
||||
for id in &s.state.camera_points {
|
||||
let w = pl.to_world(pl.solve.point(*id).unwrap().pos);
|
||||
let d = linalg::dot(linalg::sub(w, pos), normal).abs();
|
||||
assert!(d < 0.05 * size.max(1.0) + 1.0, "point {id} is {d} px off the plane");
|
||||
}
|
||||
// The solid faces the camera.
|
||||
let ctx = EvalCtx::new(&s.project, cid, &comp, frame_time(12));
|
||||
let eye = active_camera(&ctx).eye;
|
||||
assert!(linalg::dot(normal, linalg::sub([eye.x, eye.y, eye.z], pos)) > 0.0);
|
||||
|
||||
// Undo removes both layers; redo brings them back.
|
||||
let before = comp.layers.len();
|
||||
assert!(s.undo());
|
||||
assert_eq!(s.active_comp().unwrap().layers.len(), before - 2);
|
||||
assert!(s.redo());
|
||||
assert_eq!(s.active_comp().unwrap().layers.len(), before);
|
||||
|
||||
// A second create reuses the camera: Create Multiple Nulls and Camera.
|
||||
let r = s.execute_checked("camera.createFromSolve", json!({"kind": "null", "multiple": true})).unwrap();
|
||||
assert_eq!(r["camera"].as_u64().unwrap(), cam.0);
|
||||
assert_eq!(r["layers"].as_array().unwrap().len(), floor.len());
|
||||
// Text and Shadow Catcher (+ light).
|
||||
let r = s.execute_checked("camera.createFromSolve", json!({"kind": "text"})).unwrap();
|
||||
assert_eq!(r["layers"].as_array().unwrap().len(), 1);
|
||||
let r = s.execute_checked("camera.createFromSolve", json!({"kind": "shadowCatcher"})).unwrap();
|
||||
let ids: Vec<LayerId> = r["layers"].as_array().unwrap().iter().map(|v| LayerId(v.as_u64().unwrap())).collect();
|
||||
let comp = s.active_comp().unwrap();
|
||||
assert!(comp.layer(ids[0]).unwrap().name.starts_with("Shadow Catcher"));
|
||||
assert_eq!(comp.layer(ids[0]).unwrap().props.prop("materialOptions/acceptsShadows").unwrap().value.as_enum(), 2);
|
||||
assert!(comp.layer(ids[1]).unwrap().is_light());
|
||||
assert!(s.execute_checked("camera.createFromSolve", json!({"kind": "teapot"})).is_err());
|
||||
|
||||
// Set Ground Plane and Origin: the target becomes the origin; a null created there sits at
|
||||
// (0, 0, 0) lying on the X-Z plane (its front facing up, −y).
|
||||
s.execute_checked("camera.setGroundPlane", json!({})).unwrap();
|
||||
assert_eq!(status(&mut s)["groundPlane"], true);
|
||||
let r = s.execute_checked("camera.createFromSolve", json!({"kind": "null"})).unwrap();
|
||||
let null = LayerId(r["layers"][0].as_u64().unwrap());
|
||||
let tr = s.active_comp().unwrap().layer(null).unwrap().props.sub("transform").unwrap().clone();
|
||||
let pos = tr.get("position").unwrap().value.as_vec3();
|
||||
assert!(linalg::norm(pos) < 1e-6, "{pos:?}");
|
||||
let up = linalg::mv(&linalg::from_euler_xyz(tr.get("orientation").unwrap().value.as_vec3()), [0.0, 0.0, -1.0]);
|
||||
assert!(linalg::norm(linalg::sub(up, [0.0, -1.0, 0.0])) < 1e-6, "{up:?}");
|
||||
|
||||
// An explicit target (comp world) places the layer there, its front facing the normal.
|
||||
let r = s
|
||||
.execute_checked("camera.createFromSolve", json!({"kind": "solid", "target": {"center": [10.0, 0.0, 20.0], "normal": [0.0, 0.0, -1.0], "size": 80.0}}))
|
||||
.unwrap();
|
||||
let l = s.active_comp().unwrap().layer(LayerId(r["layers"][0].as_u64().unwrap())).unwrap().clone();
|
||||
let tr = l.props.sub("transform").unwrap();
|
||||
let pos = tr.get("position").unwrap().value.as_vec3();
|
||||
assert!(linalg::norm(linalg::sub(pos, [10.0, 0.0, 20.0])) < 1e-6, "{pos:?}");
|
||||
let front = linalg::mv(&linalg::from_euler_xyz(tr.get("orientation").unwrap().value.as_vec3()), [0.0, 0.0, -1.0]);
|
||||
assert!(linalg::norm(linalg::sub(front, [0.0, 0.0, -1.0])) < 1e-6, "{front:?}");
|
||||
|
||||
// Serde keeps the analysis.
|
||||
let back = effectcraft_project::Project::from_json(&s.project.to_json()).unwrap();
|
||||
let g2 = back.comp(cid).unwrap().layer(clip).unwrap().props.find_group(uid).unwrap();
|
||||
let sv = ct::solve(&crate::camera_track::static_params(g2)).unwrap();
|
||||
assert!(sv.ground.is_some());
|
||||
assert_eq!(sv.points.len(), pl.solve.points.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalidation_resolve_cancel_and_undo() {
|
||||
let (mut s, clip) = tracked(json!({"shotType": "specify", "aov": 2.0 * (W as f64 / 2.0 / F).atan().to_degrees()}));
|
||||
let st = status(&mut s);
|
||||
assert_eq!(st["solved"], true, "{st}");
|
||||
assert_eq!(st["shotType"], "Specify Angle of View");
|
||||
assert!((st["focalLength"].as_f64().unwrap() - F).abs() < 0.01 * F);
|
||||
let cid = s.active_comp_id().unwrap();
|
||||
let uid = st["effect"].as_u64().unwrap();
|
||||
// Undo removes the analysis (one step); redo restores it.
|
||||
assert!(s.undo());
|
||||
assert_eq!(status(&mut s)["analyzed"], false);
|
||||
assert!(s.redo());
|
||||
assert_eq!(status(&mut s)["solved"], true);
|
||||
|
||||
// Changing the Solve Method keeps the tracks and only re-solves.
|
||||
s.execute_checked("prop.set", json!({"layer": clip.0, "path": "effects/#1/advanced/solveMethod", "value": 1})).unwrap();
|
||||
let st = status(&mut s);
|
||||
assert_eq!(st["analyzed"], true);
|
||||
assert_eq!(st["solved"], false);
|
||||
assert_eq!(st["pending"], true);
|
||||
s.execute_checked("camera.analyze", json!({"wait": true})).unwrap();
|
||||
let st = status(&mut s);
|
||||
assert_eq!(st["solved"], true, "{st}");
|
||||
assert_eq!(st["methodUsed"], "Typical");
|
||||
|
||||
// Deleting points re-solves without them.
|
||||
let pts = s.execute("camera.points", json!({})).unwrap();
|
||||
let del: Vec<u64> = pts["points"].as_array().unwrap().iter().take(3).map(|p| p["id"].as_u64().unwrap()).collect();
|
||||
s.execute_checked("camera.deletePoints", json!({"points": del, "wait": true})).unwrap();
|
||||
s.poll_camera(false);
|
||||
let st = status(&mut s);
|
||||
assert_eq!(st["solved"], true, "{st}");
|
||||
assert!(st["deleted"].as_u64().unwrap() >= 3);
|
||||
let g = s.project.comp(cid).unwrap().layer(clip).unwrap().props.find_group(uid).unwrap().clone();
|
||||
let sv = ct::solve(&crate::camera_track::static_params(&g)).unwrap();
|
||||
assert!(sv.points.iter().all(|p| !del.contains(&(p.id as u64))));
|
||||
|
||||
// Trimming the layer changes its frames: tracks and solve are cleared and queued.
|
||||
s.edit("trim", None, |p, _| {
|
||||
p.comp_mut(cid).unwrap().layer_mut(clip).unwrap().in_point = frame_time(2);
|
||||
Ok(())
|
||||
})
|
||||
.unwrap();
|
||||
let st = status(&mut s);
|
||||
assert_eq!(st["analyzed"], false);
|
||||
assert_eq!(st["pending"], true);
|
||||
|
||||
// A background analysis can be cancelled: nothing is written.
|
||||
s.execute_checked("camera.analyze", json!({})).unwrap();
|
||||
assert!(s.is_camera_analyzing() || s.camera_job.is_some());
|
||||
let banner = status(&mut s)["banner"].as_str().map(str::to_string);
|
||||
if s.is_camera_analyzing() {
|
||||
assert!(banner.unwrap().starts_with("Analyzing in background (step 1 of 2)"));
|
||||
}
|
||||
s.execute("camera.cancel", json!({})).ok();
|
||||
let t0 = std::time::Instant::now();
|
||||
while s.camera_job.is_some() {
|
||||
assert!(t0.elapsed().as_secs() < 120);
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
s.poll_camera(false);
|
||||
}
|
||||
// (When the job finished before the cancel landed it wrote a valid analysis.)
|
||||
let st = status(&mut s);
|
||||
assert_eq!(st["running"], false);
|
||||
assert!(s.execute_checked("track.camera", json!({"layer": clip.0, "shotType": "bogus"})).is_err());
|
||||
assert!(s.execute_checked("track.camera", json!({"layer": clip.0, "aov": 500})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn effect_parameters_match_after_effects() {
|
||||
let spec = effectcraft_effects::registry().iter().find(|e| e.id == ct::ID).unwrap();
|
||||
assert_eq!(spec.name, "3D Camera Tracker");
|
||||
assert_eq!(spec.category, "Perspective");
|
||||
let names: Vec<&str> = spec.params.iter().filter(|p| !matches!(p.ui, effectcraft_project::ParamUi::Hidden)).map(|p| p.name).collect();
|
||||
for n in [
|
||||
"Shot Type",
|
||||
"Horizontal Angle of View",
|
||||
"Show Track Points",
|
||||
"Render Track Points",
|
||||
"Track Point Size",
|
||||
"Target Size",
|
||||
"Solve Method",
|
||||
"Detailed Analysis",
|
||||
"Auto-delete Points Across Time",
|
||||
"Hide Warning Banner",
|
||||
] {
|
||||
assert!(names.contains(&n), "{n}");
|
||||
}
|
||||
// The Animation menu entry is a real command now.
|
||||
let c = crate::find_command("track.camera").unwrap();
|
||||
assert_eq!(c.label, "Track Camera");
|
||||
}
|
||||
@@ -331,7 +331,7 @@ fn implemented_stubs_are_enabled_and_workspaces_are_frontend() {
|
||||
assert!(s.drain_events().iter().any(|e| matches!(e, Event::Frontend { command, .. } if command == id)));
|
||||
}
|
||||
// What remains a stub is still disabled.
|
||||
for id in ["view.displayColorManagement", "file.createProxy", "track.camera"] {
|
||||
for id in ["view.displayColorManagement", "file.createProxy"] {
|
||||
assert!(!s.is_enabled(id), "{id}");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -14,3 +14,4 @@ effectcraft-raster = { workspace = true }
|
||||
rayon = { workspace = true }
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
web-time = { workspace = true }
|
||||
|
||||
@@ -173,6 +173,9 @@ pub fn adjust(cams: &mut [BaCam], pts: &mut [V3], obs: &[BaObs], o: &BaOpts) ->
|
||||
}
|
||||
let mut cur = cost(cams, pts, obs, o);
|
||||
let initial = cur;
|
||||
let t0 = web_time::Instant::now();
|
||||
let mut tries = 0usize;
|
||||
let dbg = std::env::var_os("EC_CAMTRACK_DEBUG").is_some() && l.n > 200;
|
||||
let mut lambda = 1e-4;
|
||||
let mut iters = 0;
|
||||
if l.n == 0 && o.fix_points {
|
||||
@@ -231,7 +234,10 @@ pub fn adjust(cams: &mut [BaCam], pts: &mut [V3], obs: &[BaObs], o: &BaOpts) ->
|
||||
for (ia, ja) in &jc {
|
||||
let wv =
|
||||
[w * (ja[0] * jx[0][0] + ja[1] * jx[1][0]), w * (ja[0] * jx[0][1] + ja[1] * jx[1][1]), w * (ja[0] * jx[0][2] + ja[1] * jx[1][2])];
|
||||
match ps.w.iter_mut().find(|e| e.0 == *ia) {
|
||||
// Each observation of a point is on another camera: only the shared focal
|
||||
// length repeats.
|
||||
let found = if Some(*ia) == l.shared { ps.w.iter_mut().find(|e| e.0 == *ia) } else { None };
|
||||
match found {
|
||||
Some(e) => e.1 = add(e.1, wv),
|
||||
None => ps.w.push((*ia, wv)),
|
||||
}
|
||||
@@ -294,31 +300,67 @@ pub fn adjust(cams: &mut [BaCam], pts: &mut [V3], obs: &[BaObs], o: &BaOpts) ->
|
||||
// ---- Try steps with increasing damping.
|
||||
let mut accepted = false;
|
||||
for _try in 0..12 {
|
||||
tries += 1;
|
||||
let mut s = base_s.clone();
|
||||
let mut rhs = base_bc.clone();
|
||||
for i in 0..n {
|
||||
let d = s[i * n + i];
|
||||
s[i * n + i] = d + lambda * d.max(1e-6 * mean_diag);
|
||||
}
|
||||
// Schur complement over the points.
|
||||
let mut vinv: Vec<Option<M3>> = vec![None; npts];
|
||||
// Schur complement over the points: S −= W V⁻¹ Wᵀ, rhs −= W V⁻¹ bp (in parallel
|
||||
// chunks of points when the reduced system is small enough to copy per thread).
|
||||
let vinv: Vec<Option<M3>> = if o.fix_points {
|
||||
vec![None; npts]
|
||||
} else {
|
||||
systems
|
||||
.par_iter()
|
||||
.map(|ps| {
|
||||
let mut v = ps.v;
|
||||
let vd = (v[0][0] + v[1][1] + v[2][2]) / 3.0;
|
||||
for k in 0..3 {
|
||||
v[k][k] += lambda * v[k][k].max(1e-6 * vd.max(1e-12)) + 1e-12;
|
||||
}
|
||||
inv3(&v)
|
||||
})
|
||||
.collect()
|
||||
};
|
||||
if !o.fix_points {
|
||||
for (j, ps) in systems.iter().enumerate() {
|
||||
let mut v = ps.v;
|
||||
let vd = (v[0][0] + v[1][1] + v[2][2]) / 3.0;
|
||||
for k in 0..3 {
|
||||
v[k][k] += lambda * v[k][k].max(1e-6 * vd.max(1e-12)) + 1e-12;
|
||||
}
|
||||
let Some(vi) = inv3(&v) else { continue };
|
||||
vinv[j] = Some(vi);
|
||||
let vb = mv(&vi, ps.bp);
|
||||
let wv: Vec<V3> = ps.w.iter().map(|(_, w)| mv(&vi, *w)).collect();
|
||||
for (a, (ia, wa)) in ps.w.iter().enumerate() {
|
||||
rhs[*ia] -= dot(*wa, vb);
|
||||
for (ib, wb) in &ps.w {
|
||||
s[ia * n + ib] -= dot(wv[a], *wb);
|
||||
let reduce = |range: std::ops::Range<usize>, s: &mut [f64], rhs: &mut [f64]| {
|
||||
for j in range {
|
||||
let (Some(vi), ps) = (&vinv[j], &systems[j]) else { continue };
|
||||
let vb = mv(vi, ps.bp);
|
||||
let wv: Vec<V3> = ps.w.iter().map(|(_, w)| mv(vi, *w)).collect();
|
||||
for (a, (ia, wa)) in ps.w.iter().enumerate() {
|
||||
rhs[*ia] -= dot(*wa, vb);
|
||||
for (ib, wb) in &ps.w[a..] {
|
||||
let d = dot(wv[a], *wb);
|
||||
s[ia * n + ib] -= d;
|
||||
if ia != ib {
|
||||
s[ib * n + ia] -= d;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
let chunks = rayon::current_num_threads().clamp(1, 8);
|
||||
if n <= 1200 && npts >= 64 && chunks > 1 {
|
||||
let per = npts.div_ceil(chunks);
|
||||
let parts: Vec<(Vec<f64>, Vec<f64>)> = (0..chunks)
|
||||
.into_par_iter()
|
||||
.map(|t| {
|
||||
let (mut ds, mut dr) = (vec![0.0; n * n], vec![0.0; n]);
|
||||
reduce(t * per..((t + 1) * per).min(npts), &mut ds, &mut dr);
|
||||
(ds, dr)
|
||||
})
|
||||
.collect();
|
||||
for (ds, dr) in parts {
|
||||
s.par_iter_mut().zip(ds.par_iter()).for_each(|(x, y)| *x += y);
|
||||
for (x, y) in rhs.iter_mut().zip(dr) {
|
||||
*x += y;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
reduce(0..npts, &mut s, &mut rhs);
|
||||
}
|
||||
}
|
||||
let mut dc = rhs;
|
||||
@@ -363,9 +405,18 @@ pub fn adjust(cams: &mut [BaCam], pts: &mut [V3], obs: &[BaObs], o: &BaOpts) ->
|
||||
let gain = cur - c2;
|
||||
cur = c2;
|
||||
lambda = (lambda / 3.0).max(1e-9);
|
||||
accepted = gain > 1e-10 * cur.max(1e-12);
|
||||
accepted = gain > 1e-6 * cur.max(1e-12);
|
||||
if !accepted {
|
||||
// Converged (tiny improvement).
|
||||
if dbg {
|
||||
eprintln!(
|
||||
"ba n={} pts={} obs={} iters={iters} tries={tries} {:.2}s cost {initial:.1} -> {cur:.1}",
|
||||
l.n,
|
||||
npts,
|
||||
obs.len(),
|
||||
t0.elapsed().as_secs_f64()
|
||||
);
|
||||
}
|
||||
return BaReport { initial_cost: initial, cost: cur, iterations: iters };
|
||||
}
|
||||
break;
|
||||
@@ -376,6 +427,9 @@ pub fn adjust(cams: &mut [BaCam], pts: &mut [V3], obs: &[BaObs], o: &BaOpts) ->
|
||||
break;
|
||||
}
|
||||
}
|
||||
if dbg {
|
||||
eprintln!("ba n={} pts={} obs={} iters={iters} tries={tries} {:.2}s cost {initial:.1} -> {cur:.1}", l.n, npts, obs.len(), t0.elapsed().as_secs_f64());
|
||||
}
|
||||
BaReport { initial_cost: initial, cost: cur, iterations: iters }
|
||||
}
|
||||
|
||||
|
||||
@@ -89,7 +89,7 @@ impl TrackAnalyzer {
|
||||
let (cx, cy) = ((p[0] / cell) as usize, (p[1] / cell) as usize);
|
||||
!occ.get(cy * gw + cx).copied().unwrap_or(true)
|
||||
};
|
||||
let opts = CornerOpts { max_features: want - self.active.len().min(want), min_distance: md, quality: 0.005, window: 2, border: 8 };
|
||||
let opts = CornerOpts { max_features: want - self.active.len().min(want), min_distance: md, quality: 0.001, window: 2, border: 8 };
|
||||
for p in good_features(&pyr, &opts, Some(&free)) {
|
||||
let l = pyr.to_layer(p);
|
||||
let id = self.tracks.len() as u32;
|
||||
|
||||
@@ -298,6 +298,23 @@ pub fn sym_eigen(a: &[f64], n: usize) -> (Vec<f64>, Vec<f64>) {
|
||||
(vals, vecs)
|
||||
}
|
||||
|
||||
/// Dot product, four lanes (lets the compiler vectorise).
|
||||
fn dot_n(a: &[f64], b: &[f64]) -> f64 {
|
||||
let mut acc = [0.0f64; 4];
|
||||
let (ca, cb) = (a.chunks_exact(4), b.chunks_exact(4));
|
||||
let (ra, rb) = (ca.remainder(), cb.remainder());
|
||||
for (x, y) in ca.zip(cb) {
|
||||
for k in 0..4 {
|
||||
acc[k] += x[k] * y[k];
|
||||
}
|
||||
}
|
||||
let mut s = acc[0] + acc[1] + acc[2] + acc[3];
|
||||
for (x, y) in ra.iter().zip(rb) {
|
||||
s += x * y;
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
/// Solve `A x = b` for a symmetric positive definite `A` (`n × n`, row-major) in place by
|
||||
/// Cholesky factorisation. Returns false when `A` is not positive definite.
|
||||
pub fn cholesky_solve(a: &mut [f64], n: usize, b: &mut [f64]) -> bool {
|
||||
@@ -313,16 +330,9 @@ pub fn cholesky_solve(a: &mut [f64], n: usize, b: &mut [f64]) -> bool {
|
||||
a[j * n + j] = d;
|
||||
let (head, tail) = a.split_at_mut((j + 1) * n);
|
||||
let rowj = &head[j * n..j * n + j];
|
||||
// Rows below j are independent.
|
||||
use rayon::prelude::*;
|
||||
let work = |row: &mut [f64]| {
|
||||
let s: f64 = row[j] - row[..j].iter().zip(rowj).map(|(a, b)| a * b).sum::<f64>();
|
||||
row[j] = s / d;
|
||||
};
|
||||
if n - j > 192 {
|
||||
tail.par_chunks_mut(n).for_each(work);
|
||||
} else {
|
||||
tail.chunks_mut(n).for_each(work);
|
||||
// Rows below j (sequential: per-column parallel dispatch costs more than it saves).
|
||||
for row in tail.chunks_mut(n) {
|
||||
row[j] = (row[j] - dot_n(&row[..j], rowj)) / d;
|
||||
}
|
||||
}
|
||||
// Forward: L y = b.
|
||||
|
||||
@@ -13,7 +13,10 @@
|
||||
//! the other keyframes are added incrementally (**resection** by robust pose refinement,
|
||||
//! **triangulation** of new tracks by the linear method of H&Z §12.2); a sparse
|
||||
//! **Levenberg–Marquardt bundle adjustment** with the Schur complement over the points
|
||||
//! ([`bundle`]) refines everything; the frames between keyframes are resected last.
|
||||
//! ([`bundle`]) refines everything; the frames between keyframes are resected last. Inlier
|
||||
//! thresholds start from the analysis resolution and are re-estimated from the residuals
|
||||
//! (median absolute residual) once the keyframes are solved, so features on moving objects,
|
||||
//! occlusion edges and screen-locked overlays drop out.
|
||||
//! 3. **Focal length**: *Specify Angle of View* fixes it; *Fixed Angle of View* searches it
|
||||
//! (the reprojection error of the keyframe reconstruction over a log-spaced range of focal
|
||||
//! lengths, refined by golden-section search) and then adjusts it in the bundle adjustment
|
||||
@@ -22,7 +25,8 @@
|
||||
//! 4. **Tripod pan** shots (no camera translation) are solved as a pure rotation: frame-to-frame
|
||||
//! rotations from the tracked rays (orthogonal Procrustes) refined by the same bundle
|
||||
//! adjustment with the camera centres fixed. *Auto Detect* solves both ways and keeps the
|
||||
//! tripod solution when it explains the tracks as well as the general one.
|
||||
//! tripod solution when it explains the tracks as well as the general one (a quick rotation fit
|
||||
//! at the general solution's focal length skips the tripod search when it clearly fails).
|
||||
//!
|
||||
//! The camera model has square pixels, the principal point at the layer centre and no lens
|
||||
//! distortion. Camera space is x right, y down, z forward (After Effects' convention); a camera
|
||||
|
||||
@@ -37,8 +37,9 @@ struct Data {
|
||||
tracks: Vec<(u32, usize, Vec<[f64; 2]>)>,
|
||||
/// Per frame: (track index, position).
|
||||
by_frame: Vec<Vec<(u32, [f64; 2])>>,
|
||||
/// Tracking noise scale (pixels).
|
||||
noise: f64,
|
||||
/// Tracking noise scale (pixels): starts from the analysis resolution and is re-estimated
|
||||
/// from the residuals once the keyframes are solved.
|
||||
noise_bits: std::sync::atomic::AtomicU64,
|
||||
}
|
||||
|
||||
impl Data {
|
||||
@@ -58,23 +59,31 @@ impl Data {
|
||||
}
|
||||
tracks.push((tr.id, tr.start as usize, pts));
|
||||
}
|
||||
Data { n, w: t.size[0], tracks, by_frame, noise: t.factor.max(1.0) }
|
||||
let noise = (0.5 * t.factor).max(0.5);
|
||||
Data { n, w: t.size[0], tracks, by_frame, noise_bits: std::sync::atomic::AtomicU64::new(noise.to_bits()) }
|
||||
}
|
||||
fn at(&self, ti: u32, k: usize) -> Option<[f64; 2]> {
|
||||
let (_, s, p) = &self.tracks[ti as usize];
|
||||
k.checked_sub(*s).and_then(|i| p.get(i).copied())
|
||||
}
|
||||
fn noise(&self) -> f64 {
|
||||
f64::from_bits(self.noise_bits.load(Ordering::Relaxed))
|
||||
}
|
||||
fn set_noise(&self, v: f64) {
|
||||
self.noise_bits.store(v.to_bits(), Ordering::Relaxed);
|
||||
}
|
||||
fn inlier_thr(&self) -> f64 {
|
||||
4.0 * self.noise
|
||||
4.0 * self.noise()
|
||||
}
|
||||
fn huber(&self) -> f64 {
|
||||
1.0 * self.noise
|
||||
self.noise()
|
||||
}
|
||||
}
|
||||
|
||||
/// Parallax-based keyframes: a new keyframe when the median feature motion since the last
|
||||
/// exceeds 3 % of the width, when fewer than 60 % of its features survive, or before the shared
|
||||
/// features drop below 24. The first and last frames with features are keyframes.
|
||||
/// exceeds 1.5 % of the width, when fewer than 70 % of its features survive, after 12 frames, or
|
||||
/// before the shared features drop below 24. The first and last frames with features are
|
||||
/// keyframes.
|
||||
fn keyframes(d: &Data) -> Vec<usize> {
|
||||
let Some(first) = (0..d.n).find(|k| d.by_frame[*k].len() >= 8) else { return vec![] };
|
||||
let mut kf = vec![first];
|
||||
@@ -97,7 +106,7 @@ fn keyframes(d: &Data) -> Vec<usize> {
|
||||
disp.sort_by(f64::total_cmp);
|
||||
let med = disp.get(disp.len() / 2).copied().unwrap_or(0.0);
|
||||
let ratio = disp.len() as f64 / base.len().max(1) as f64;
|
||||
if med > 0.03 * d.w || ratio < 0.6 || disp.len() < 24 {
|
||||
if med > 0.015 * d.w || ratio < 0.7 || disp.len() < 24 || j - last >= 12 {
|
||||
last = j;
|
||||
kf.push(j);
|
||||
}
|
||||
@@ -405,7 +414,7 @@ fn init_pair(d: &Data, a: usize, b: usize, f: f64, use_e: bool, use_h: bool) ->
|
||||
}
|
||||
let x1: Vec<[f64; 2]> = sh.iter().map(|s| [s.1[0] / f, s.1[1] / f]).collect();
|
||||
let x2: Vec<[f64; 2]> = sh.iter().map(|s| [s.2[0] / f, s.2[1] / f]).collect();
|
||||
let thr = 2.0 * d.noise / f;
|
||||
let thr = 2.0 * d.noise() / f;
|
||||
let mut cands: Vec<Pose> = vec![];
|
||||
if use_e && let Some((e, inl)) = geometry::ransac_essential(&x1, &x2, thr, 400, (a * 7919 + b) as u64) {
|
||||
let (p1, p2): (Vec<[f64; 2]>, Vec<[f64; 2]>) = (0..x1.len()).filter(|i| inl[*i]).map(|i| (x1[i], x2[i])).unzip();
|
||||
@@ -415,7 +424,7 @@ fn init_pair(d: &Data, a: usize, b: usize, f: f64, use_e: bool, use_h: bool) ->
|
||||
}
|
||||
if use_h {
|
||||
let (s1, s2): (Vec<[f64; 2]>, Vec<[f64; 2]>) = sh.iter().map(|s| (s.1, s.2)).unzip();
|
||||
if let Some(fit) = ransac(Model::Homography, &s1, &s2, 2.0 * d.noise, 500, (a * 31 + b) as u64) {
|
||||
if let Some(fit) = ransac(Model::Homography, &s1, &s2, 2.0 * d.noise(), 500, (a * 31 + b) as u64) {
|
||||
let h = fit.h.0;
|
||||
// Normalised: K⁻¹ H K with K = diag(f, f, 1).
|
||||
let hn = [[h[0][0], h[0][1], h[0][2] / f], [h[1][0], h[1][1], h[1][2] / f], [h[2][0] * f, h[2][1] * f, h[2][2]]];
|
||||
@@ -651,6 +660,12 @@ pub fn solve(tracks: &CameraTracks, s: &SolveSettings, cancel: Option<&AtomicBoo
|
||||
let try_general = matches!(s.method, SolveMethod::Auto | SolveMethod::Typical | SolveMethod::MostlyFlat);
|
||||
let try_tripod = matches!(s.method, SolveMethod::Auto | SolveMethod::TripodPan);
|
||||
|
||||
let t0 = web_time::Instant::now();
|
||||
let lap = |what: &str| {
|
||||
if debug() {
|
||||
eprintln!("{what}: {:.2} s", t0.elapsed().as_secs_f64());
|
||||
}
|
||||
};
|
||||
let mut results: Vec<(Attempt, f64)> = vec![];
|
||||
if try_general {
|
||||
let f = match fixed_f {
|
||||
@@ -666,7 +681,20 @@ pub fn solve(tracks: &CameraTracks, s: &SolveSettings, cancel: Option<&AtomicBoo
|
||||
results.push((a, f));
|
||||
}
|
||||
}
|
||||
if try_tripod && !cancelled() {
|
||||
lap("general");
|
||||
// Auto Detect: a quick tripod check at the general solve's focal length; when a rotation
|
||||
// clearly cannot explain the tracks, the tripod search is skipped.
|
||||
let tripod_plausible = match (s.method, results.first()) {
|
||||
(SolveMethod::Auto, Some((g, f))) => {
|
||||
let quick = tripod(&d, &mini, *f, &tcfg, cancel).map(|a| a.err).unwrap_or(f64::INFINITY);
|
||||
if debug() {
|
||||
eprintln!("quick tripod check: {quick:.3} vs general {:.3}", g.err);
|
||||
}
|
||||
quick <= 3.0 * g.err + d.noise()
|
||||
}
|
||||
_ => true,
|
||||
};
|
||||
if try_tripod && tripod_plausible && !cancelled() {
|
||||
let f = match fixed_f {
|
||||
Some(f) => f,
|
||||
None => search_focal(d.w, &|f| tripod(&d, &mini, f, &Cfg { per_frame_f: variable, ..tcfg }, cancel).map(|a| a.err).unwrap_or(f64::INFINITY)),
|
||||
@@ -696,7 +724,7 @@ pub fn solve(tracks: &CameraTracks, s: &SolveSettings, cancel: Option<&AtomicBoo
|
||||
if debug() {
|
||||
eprintln!("auto: general {eg:.4} tripod {et:.4} baseline/depth {:.4}", base / med);
|
||||
}
|
||||
if et <= (eg * 1.25).max(eg + 0.05 * d.noise) || base / med < 0.01 { t } else { g }
|
||||
if et <= (eg * 1.25).max(eg + 0.05 * d.noise()) || base / med < 0.01 { t } else { g }
|
||||
}
|
||||
(Some(g), None) => g,
|
||||
(None, Some(t)) => t,
|
||||
@@ -717,15 +745,44 @@ pub fn solve(tracks: &CameraTracks, s: &SolveSettings, cancel: Option<&AtomicBoo
|
||||
(None, true) => FocalMode::PerCamera(F_PRIOR),
|
||||
(None, false) => FocalMode::Shared,
|
||||
};
|
||||
lap("tripod");
|
||||
// Keyframes: adjust with the focal length free, reject, re-triangulate, adjust again.
|
||||
let kf_frames: Vec<usize> = r.cams.keys().copied().collect();
|
||||
r.adjust(&d, &kf_frames, &cfg, focal, 40);
|
||||
r.reject(&d);
|
||||
// Re-estimate the tracking noise from the inlier residuals (robustly, from their median:
|
||||
// for 2D Gaussian noise of deviation σ the median distance is 1.1774 σ), then tighten the
|
||||
// thresholds: observations of moving objects and occlusion edges become outliers.
|
||||
let mut res: Vec<f64> = vec![];
|
||||
for (k, c) in &r.cams {
|
||||
for (ti, uv) in &d.by_frame[*k] {
|
||||
if r.outliers.contains(&(*ti, *k)) {
|
||||
continue;
|
||||
}
|
||||
if let Some(e) = r.pts.get(ti).and_then(|x| bundle::residual(c, *x, *uv)) {
|
||||
res.push(e[0].hypot(e[1]));
|
||||
}
|
||||
}
|
||||
}
|
||||
if res.len() > 20 {
|
||||
res.sort_by(f64::total_cmp);
|
||||
let sigma = res[res.len() / 2] / 1.1774;
|
||||
let floor = 0.05 * tracks.factor.max(1.0);
|
||||
d.set_noise((1.5 * sigma).clamp(floor, d.noise()));
|
||||
if debug() {
|
||||
eprintln!("noise: σ {sigma:.3} → {:.3}", d.noise());
|
||||
}
|
||||
r.outliers.clear();
|
||||
r.reject(&d);
|
||||
}
|
||||
r.triangulate(&d, kind, 1.0, 1.0);
|
||||
r.adjust(&d, &kf_frames, &cfg, focal, 40);
|
||||
r.reject(&d);
|
||||
r.adjust(&d, &kf_frames, &cfg, focal, 20);
|
||||
if cancelled() {
|
||||
return err("cancelled");
|
||||
}
|
||||
lap("keyframe adjustment");
|
||||
// Every other frame: resect from the points (nearest registered camera first).
|
||||
let mut todo: Vec<usize> = (0..d.n).filter(|k| !r.cams.contains_key(k)).collect();
|
||||
todo.sort_by_key(|k| r.cams.keys().map(|c| c.abs_diff(*k)).min().unwrap_or(usize::MAX));
|
||||
@@ -746,6 +803,7 @@ pub fn solve(tracks: &CameraTracks, s: &SolveSettings, cancel: Option<&AtomicBoo
|
||||
if cancelled() {
|
||||
return err("cancelled");
|
||||
}
|
||||
lap("resection");
|
||||
// Points for the tracks that only lived between keyframes, then the final adjustment over
|
||||
// every frame (when that stays affordable).
|
||||
r.triangulate(&d, kind, 1.0, 1.0);
|
||||
@@ -786,6 +844,7 @@ pub fn solve(tracks: &CameraTracks, s: &SolveSettings, cancel: Option<&AtomicBoo
|
||||
}
|
||||
(m, _) => m,
|
||||
};
|
||||
lap("final adjustment");
|
||||
Ok(finish(&d, tracks, s, r, method, kind))
|
||||
}
|
||||
|
||||
|
||||
@@ -358,3 +358,81 @@ fn perf_long_clip() {
|
||||
eprintln!("300 frames, {} tracks: {:?} err {:.3} focal {:.1} in {:?}", tr.tracks.len(), s.method_used, s.average_error, s.focal(), t0.elapsed());
|
||||
assert!(s.average_error < 0.5);
|
||||
}
|
||||
|
||||
/// Solve tracks saved from the app (`EC_CAMTRACK_FILE=tracks.json cargo test -p effectcraft-track
|
||||
/// --release solve_saved_tracks -- --ignored --nocapture`): a debugging aid.
|
||||
#[test]
|
||||
#[ignore]
|
||||
fn solve_saved_tracks() {
|
||||
let Some(path) = std::env::var_os("EC_CAMTRACK_FILE") else { return };
|
||||
let mut tr = CameraTracks::from_json(&std::fs::read_to_string(path).unwrap()).unwrap();
|
||||
if let Some(maxy) = std::env::var("EC_CAMTRACK_MAXY").ok().and_then(|v| v.parse::<f32>().ok()) {
|
||||
tr.tracks.retain(|t| t.pts.iter().all(|p| p[1] < maxy));
|
||||
}
|
||||
let shot = if std::env::var_os("EC_CAMTRACK_VARIABLE").is_some() { ShotType::VariableZoom } else { ShotType::FixedAngle };
|
||||
let t0 = std::time::Instant::now();
|
||||
// With the true cameras (`[{pos, poi, zoom}]` per frame, two-node), how consistent are the
|
||||
// tracks themselves?
|
||||
if let Some(tp) = std::env::var_os("EC_CAMTRACK_TRUTH") {
|
||||
let v: Vec<serde_json::Value> = serde_json::from_str(&std::fs::read_to_string(tp).unwrap()).unwrap();
|
||||
let g = |x: &serde_json::Value| -> V3 { [x[0].as_f64().unwrap(), x[1].as_f64().unwrap(), x[2].as_f64().unwrap()] };
|
||||
let cams: Vec<(super::geometry::Pose, f64)> = v
|
||||
.iter()
|
||||
.map(|c| {
|
||||
let (pos, poi) = (g(&c["pos"]), g(&c["poi"]));
|
||||
let fwd = normalize(sub(poi, pos));
|
||||
let d0 = [0.0, 1.0, 0.0];
|
||||
let down = normalize(sub(d0, scale(fwd, dot(d0, fwd))));
|
||||
let right = cross(down, fwd);
|
||||
(super::geometry::Pose { r: [right, down, fwd], c: pos }, c["zoom"].as_f64().unwrap())
|
||||
})
|
||||
.collect();
|
||||
let (w, h) = (tr.size[0], tr.size[1]);
|
||||
let mut means = vec![];
|
||||
let mut drop_ids = vec![];
|
||||
for t in &tr.tracks {
|
||||
let views: Vec<(super::geometry::Pose, [f64; 2])> = t
|
||||
.pts
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(i, p)| {
|
||||
let (pose, z) = cams[t.start as usize + i];
|
||||
(pose, [(p[0] as f64 - w / 2.0) / z, (p[1] as f64 - h / 2.0) / z])
|
||||
})
|
||||
.collect();
|
||||
let Some(x) = super::geometry::triangulate(&views) else { continue };
|
||||
let e: Vec<f64> = views.iter().filter_map(|(p, o)| super::geometry::reproj2(p, x, *o).map(|e| e.sqrt() * cams[0].1)).collect();
|
||||
means.push((e.iter().sum::<f64>() / e.len().max(1) as f64, t.pts[0]));
|
||||
if std::env::var_os("EC_CAMTRACK_TRUTH_FILTER").is_some() && means.last().unwrap().0 > 1.0 {
|
||||
drop_ids.push(t.id);
|
||||
}
|
||||
}
|
||||
tr.tracks.retain(|t| !drop_ids.contains(&t.id));
|
||||
means.sort_by(|a, b| a.0.total_cmp(&b.0));
|
||||
eprintln!(
|
||||
"true-camera track errors (px): median {:.2}; quartiles {:?}",
|
||||
means[means.len() / 2].0,
|
||||
[means[means.len() / 4].0, means[means.len() * 3 / 4].0]
|
||||
);
|
||||
for m in means.iter().step_by(6) {
|
||||
eprintln!(" {:.2} at {:?}", m.0, m.1);
|
||||
}
|
||||
}
|
||||
let mut st = settings(shot, SolveMethod::Auto);
|
||||
if let Some(a) = std::env::var("EC_CAMTRACK_AOV").ok().and_then(|v| v.parse::<f64>().ok()) {
|
||||
st.shot = ShotType::SpecifyAngle;
|
||||
st.hfov = a;
|
||||
}
|
||||
let s = solve(&tr, &st, None).expect("solve");
|
||||
eprintln!(
|
||||
"{} frames, {} tracks: {:?} err {:.3} focal {:.1} (hfov {:.1}) points {} in {:?}",
|
||||
tr.frames,
|
||||
tr.tracks.len(),
|
||||
s.method_used,
|
||||
s.average_error,
|
||||
s.focal(),
|
||||
s.hfov(s.focal()),
|
||||
s.points.len(),
|
||||
t0.elapsed()
|
||||
);
|
||||
}
|
||||
|
||||
@@ -727,6 +727,11 @@ impl EffectcraftApp {
|
||||
self.session.poll_warp(true);
|
||||
ctx.request_repaint_after(std::time::Duration::from_millis(100));
|
||||
}
|
||||
// 3D Camera Tracker: likewise (tracking, then solving).
|
||||
if self.session.camera_job.is_some() || !self.session.camera_pending.is_empty() {
|
||||
self.session.poll_camera(true);
|
||||
ctx.request_repaint_after(std::time::Duration::from_millis(100));
|
||||
}
|
||||
self.apply_prefs(&ctx);
|
||||
self.handle_events(&ctx);
|
||||
self.tick_autosave(&ctx);
|
||||
|
||||
@@ -0,0 +1,438 @@
|
||||
//! 3D Camera Tracker in the Composition viewer and Effect Controls:
|
||||
//!
|
||||
//! - while the effect is selected, the solved 3D track points are drawn as coloured targets
|
||||
//! sized by depth (or the 2D source tracks with Show Track Points ▸ 2D Source);
|
||||
//! - hovering between three points shows the target plane they span (a disc in perspective);
|
||||
//! click a point (Shift adds / toggles) or drag a marquee to select points, click inside a
|
||||
//! triangle to select its three points; the selection's target is drawn;
|
||||
//! - right-click: Set Ground Plane and Origin, Create Text / Solid / Null / Shadow Catcher …
|
||||
//! and Camera, Create Multiple …, Delete Selected Points; Delete removes the selection;
|
||||
//! - the "Analyzing in background (step 1 of 2)" / "Solving camera" banner;
|
||||
//! - the Effect Controls block with Analyze / Cancel / Create Camera, Method Used and Average
|
||||
//! Error.
|
||||
//!
|
||||
//! Everything is dispatched to the `camera.*` commands.
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use effectcraft_engine::Session;
|
||||
use effectcraft_engine::effects::camera_tracker as ct;
|
||||
use effectcraft_engine::geom::{Mat3, vec2 as gv2};
|
||||
use effectcraft_engine::keyframe::Value;
|
||||
use effectcraft_engine::project::{Comp, Layer, LayerId, PropGroup, Uid};
|
||||
use effectcraft_engine::render::EvalCtx;
|
||||
use effectcraft_engine::track::camtrack::{CameraSolve, CameraTracks, Target, point_color};
|
||||
use egui::{Align2, Color32, Pos2, Rect, Sense, Stroke, pos2, vec2};
|
||||
use serde_json::json;
|
||||
|
||||
use super::viewer::ViewerMap;
|
||||
use crate::theme::Tokens;
|
||||
use crate::{EffectcraftApp, widgets};
|
||||
|
||||
/// The tracker whose points the viewer shows.
|
||||
struct Active {
|
||||
layer: LayerId,
|
||||
uid: Uid,
|
||||
solve: Option<Arc<CameraSolve>>,
|
||||
tracks: Option<Arc<CameraTracks>>,
|
||||
show_3d: bool,
|
||||
point_size: f64,
|
||||
target_size: f64,
|
||||
/// Layer pixels → comp pixels.
|
||||
l2c: Mat3,
|
||||
frame: usize,
|
||||
}
|
||||
|
||||
/// A drawn point: (id, screen position, screen radius).
|
||||
type Hit = (u32, Pos2, f32);
|
||||
|
||||
fn tracker_group(l: &Layer, uid: Uid) -> Option<&PropGroup> {
|
||||
l.props.find_group(uid).filter(|g| matches!(&g.kind, effectcraft_engine::project::GroupKind::Effect { effect } if effect == ct::ID))
|
||||
}
|
||||
|
||||
/// The selected 3D Camera Tracker effect on an active layer (After Effects shows the points
|
||||
/// while the effect is selected).
|
||||
fn active(s: &Session, comp: &Comp, ectx: &EvalCtx, l2c: &dyn Fn(&EvalCtx, &Layer) -> Mat3) -> Option<Active> {
|
||||
let time = ectx.time;
|
||||
for (lid, uid) in &s.state.selected_props {
|
||||
let Some(l) = comp.layer(*lid).filter(|l| l.is_active_at(time)) else { continue };
|
||||
let Some(g) = tracker_group(l, *uid) else { continue };
|
||||
let params = effectcraft_engine::camera_track::static_params(g);
|
||||
let val = |id: &str| effectcraft_engine::effects::warp_stab::param(¶ms, id).cloned();
|
||||
let solve = ct::solve(¶ms);
|
||||
let tracks = ct::tracks(¶ms);
|
||||
let lt = l.layer_time(time).seconds();
|
||||
let frame = solve.as_ref().and_then(|sv| sv.frame_at(lt)).or_else(|| {
|
||||
let t = tracks.as_ref()?;
|
||||
(t.frame_duration > 0.0).then(|| ((lt - t.start) / t.frame_duration).round().clamp(0.0, t.frames.saturating_sub(1) as f64) as usize)
|
||||
});
|
||||
return Some(Active {
|
||||
layer: *lid,
|
||||
uid: *uid,
|
||||
solve,
|
||||
tracks,
|
||||
show_3d: val("showTrackPoints").map(|v| v.as_enum()).unwrap_or(1) == 1,
|
||||
point_size: val("trackPointSize").map(|v| v.as_f64()).unwrap_or(100.0),
|
||||
target_size: val("targetSize").map(|v| v.as_f64()).unwrap_or(100.0),
|
||||
l2c: l2c(ectx, l),
|
||||
frame: frame.unwrap_or(0),
|
||||
});
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn to_screen(a: &Active, map: &ViewerMap, q: [f64; 2]) -> Pos2 {
|
||||
let c = a.l2c.apply(gv2(q[0], q[1]));
|
||||
map.to_screen([c.x, c.y])
|
||||
}
|
||||
|
||||
fn col(id: u32) -> Color32 {
|
||||
let c = point_color(id);
|
||||
Color32::from_rgb((c[0] * 255.0) as u8, (c[1] * 255.0) as u8, (c[2] * 255.0) as u8)
|
||||
}
|
||||
|
||||
/// Comp-pixel scale of the layer → comp mapping.
|
||||
fn l2c_scale(m: &Mat3) -> f32 {
|
||||
m.0[0][0].hypot(m.0[1][0]) as f32
|
||||
}
|
||||
|
||||
/// Draw the target disc (concentric rings in perspective) of `t` on frame `k`.
|
||||
fn draw_target(p: &egui::Painter, a: &Active, map: &ViewerMap, sv: &CameraSolve, t: &Target) {
|
||||
let Some(cam) = sv.frames.get(a.frame) else { return };
|
||||
let r = t.size * 0.5 * a.target_size / 100.0;
|
||||
for (i, frac) in [1.0, 0.66, 0.33].into_iter().enumerate() {
|
||||
let ring = ct::target_circle(t.center, t.normal, r * frac, 48);
|
||||
let pts: Vec<Pos2> = ring.iter().filter_map(|x| cam.project(sv.size, *x)).map(|q| to_screen(a, map, q)).collect();
|
||||
if pts.len() < 3 {
|
||||
return;
|
||||
}
|
||||
let fill = if i % 2 == 0 { Color32::from_rgba_unmultiplied(0xd0, 0x30, 0x30, 70) } else { Color32::from_rgba_unmultiplied(0xff, 0xff, 0xff, 50) };
|
||||
p.add(egui::Shape::convex_polygon(pts.clone(), fill, Stroke::new(1.0, Color32::from_rgb(0xe0, 0x40, 0x40))));
|
||||
}
|
||||
if let Some(c) = cam.project(sv.size, t.center) {
|
||||
let c = to_screen(a, map, c);
|
||||
p.circle_filled(c, 3.0, Color32::from_rgb(0xff, 0xd0, 0x40));
|
||||
}
|
||||
}
|
||||
|
||||
/// The three drawn points forming a triangle around `hp` (nearest first).
|
||||
fn triangle_at(hits: &[Hit], hp: Pos2) -> Option<[u32; 3]> {
|
||||
let mut near: Vec<&Hit> = hits.iter().collect();
|
||||
near.sort_by(|x, y| x.1.distance_sq(hp).total_cmp(&y.1.distance_sq(hp)));
|
||||
let near: Vec<&Hit> = near.into_iter().take(8).collect();
|
||||
let inside = |a: Pos2, b: Pos2, c: Pos2| {
|
||||
let s = |p: Pos2, q: Pos2| (q.x - p.x) * (hp.y - p.y) - (q.y - p.y) * (hp.x - p.x);
|
||||
let (d1, d2, d3) = (s(a, b), s(b, c), s(c, a));
|
||||
!((d1 < 0.0 || d2 < 0.0 || d3 < 0.0) && (d1 > 0.0 || d2 > 0.0 || d3 > 0.0))
|
||||
};
|
||||
for i in 0..near.len() {
|
||||
for j in i + 1..near.len() {
|
||||
for k in j + 1..near.len() {
|
||||
let (a, b, c) = (near[i], near[j], near[k]);
|
||||
// Skip slivers.
|
||||
let area = ((b.1 - a.1).x * (c.1 - a.1).y - (b.1 - a.1).y * (c.1 - a.1).x).abs();
|
||||
if area > 30.0 && inside(a.1, b.1, c.1) {
|
||||
return Some([a.0, b.0, c.0]);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
fn run(app: &mut EffectcraftApp, ctx: &egui::Context, cmd: &str, p: serde_json::Value) {
|
||||
if let Err(e) = crate::menus::invoke(app, ctx, cmd, p) {
|
||||
app.ui.status = e;
|
||||
}
|
||||
}
|
||||
|
||||
/// The viewer overlay and its interaction (call after the viewer's own gestures so it takes
|
||||
/// the clicks while a 3D Camera Tracker is selected).
|
||||
pub fn viewer_hook(
|
||||
app: &mut EffectcraftApp,
|
||||
ui: &mut egui::Ui,
|
||||
painter: &egui::Painter,
|
||||
map: &ViewerMap,
|
||||
ectx: &EvalCtx,
|
||||
l2c: &dyn Fn(&EvalCtx, &Layer) -> Mat3,
|
||||
) {
|
||||
let ctx = ui.ctx().clone();
|
||||
let comp = ectx.comp;
|
||||
banner(app, painter, map, ectx, l2c);
|
||||
if !app.ui.viewer.show_layer_controls {
|
||||
return;
|
||||
}
|
||||
let Some(a) = active(&app.session, comp, ectx, l2c) else { return };
|
||||
let selected = app.session.state.camera_points.clone();
|
||||
let lscale = l2c_scale(&a.l2c) * map.zoom;
|
||||
let mut hits: Vec<Hit> = vec![];
|
||||
match (&a.solve, a.show_3d) {
|
||||
(Some(sv), true) => {
|
||||
let pts = ct::projected(sv, a.frame);
|
||||
let med = ct::median_depth(&pts);
|
||||
for (id, q, z) in pts {
|
||||
let sp = to_screen(&a, map, q);
|
||||
if !map.area.expand(20.0).contains(sp) {
|
||||
continue;
|
||||
}
|
||||
let r = (ct::target_radius(a.point_size, z, med) as f32 * lscale).clamp(2.0, 30.0);
|
||||
let c = col(id);
|
||||
let sel = selected.contains(&id);
|
||||
if sel {
|
||||
painter.circle_filled(sp, r, c);
|
||||
painter.circle_stroke(sp, r + 1.5, Stroke::new(1.5, Color32::WHITE));
|
||||
} else {
|
||||
painter.circle_stroke(sp, r, Stroke::new((r * 0.3).clamp(1.0, 3.0), c));
|
||||
painter.circle_filled(sp, (r * 0.25).max(1.0), c);
|
||||
}
|
||||
hits.push((id, sp, r));
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
// 2D source tracks (also shown before the solve finishes).
|
||||
if let Some(tr) = &a.tracks {
|
||||
for t in &tr.tracks {
|
||||
let Some(q) = t.at(a.frame as u32) else { continue };
|
||||
let sp = to_screen(&a, map, q);
|
||||
let c = col(t.id);
|
||||
let r = (3.0 * a.point_size as f32 / 100.0).clamp(1.5, 12.0);
|
||||
painter.line_segment([sp - vec2(r, r), sp + vec2(r, r)], Stroke::new(1.2, c));
|
||||
painter.line_segment([sp - vec2(r, -r), sp + vec2(r, -r)], Stroke::new(1.2, c));
|
||||
hits.push((t.id, sp, r));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
let Some(sv) = a.solve.clone() else { return };
|
||||
if app.ui.tool != crate::state::Tool::Selection {
|
||||
return;
|
||||
}
|
||||
// Interaction over the image.
|
||||
let resp = ui.interact(map.area, egui::Id::new("viewer-camera-tracker"), Sense::click_and_drag());
|
||||
app.auto.add("viewer.cameraTracker", map.area, "3D Camera Tracker points");
|
||||
for (id, sp, r) in hits.iter().take(400) {
|
||||
app.auto.add(&format!("viewer.cameraTracker.point.{id}"), Rect::from_center_size(*sp, vec2(r * 2.0 + 4.0, r * 2.0 + 4.0)), "track point");
|
||||
}
|
||||
let hover = resp.hover_pos();
|
||||
let point_at = |p: Pos2| hits.iter().filter(|h| h.1.distance(p) <= h.2 + 3.0).min_by(|x, y| x.1.distance(p).total_cmp(&y.1.distance(p))).map(|h| h.0);
|
||||
let tri = hover.filter(|p| point_at(*p).is_none()).and_then(|p| triangle_at(&hits, p));
|
||||
let mid = egui::Id::new("viewer-camera-tracker-marquee");
|
||||
let marquee: Option<Pos2> = ui.data(|d| d.get_temp(mid));
|
||||
// Hover target / the selection's target.
|
||||
if marquee.is_none() {
|
||||
if let Some(t3) = tri
|
||||
&& let Some(t) = effectcraft_engine::camera_track::target_for(&sv, &t3, a.frame)
|
||||
{
|
||||
let ps: Vec<Pos2> = t3.iter().filter_map(|i| hits.iter().find(|h| h.0 == *i)).map(|h| h.1).collect();
|
||||
if ps.len() == 3 {
|
||||
painter.add(egui::Shape::closed_line(ps, Stroke::new(1.0, Color32::from_white_alpha(200))));
|
||||
}
|
||||
draw_target(painter, &a, map, &sv, &t);
|
||||
} else if !selected.is_empty()
|
||||
&& let Some(t) = effectcraft_engine::camera_track::target_for(&sv, &selected, a.frame)
|
||||
{
|
||||
draw_target(painter, &a, map, &sv, &t);
|
||||
}
|
||||
}
|
||||
let mods = ui.input(|i| i.modifiers);
|
||||
if resp.drag_started()
|
||||
&& let Some(p) = ui.input(|i| i.pointer.press_origin())
|
||||
{
|
||||
ui.data_mut(|d| d.insert_temp(mid, p));
|
||||
}
|
||||
if let (Some(start), Some(p)) = (marquee, resp.interact_pointer_pos()) {
|
||||
let r = Rect::from_two_pos(start, p);
|
||||
let t = app.tokens;
|
||||
painter.rect_filled(r, 0.0, t.accent.gamma_multiply(0.12));
|
||||
painter.rect_stroke(r, 0.0, Stroke::new(1.0, t.accent), egui::StrokeKind::Middle);
|
||||
if resp.drag_stopped() {
|
||||
let ids: Vec<u32> = hits.iter().filter(|h| r.contains(h.1)).map(|h| h.0).collect();
|
||||
ui.data_mut(|d| d.remove::<Pos2>(mid));
|
||||
run(app, &ctx, "camera.selectPoints", json!({"points": ids, "add": mods.shift}));
|
||||
}
|
||||
} else if resp.drag_stopped() {
|
||||
ui.data_mut(|d| d.remove::<Pos2>(mid));
|
||||
}
|
||||
if resp.clicked()
|
||||
&& let Some(p) = resp.interact_pointer_pos()
|
||||
{
|
||||
match (point_at(p), triangle_at(&hits, p)) {
|
||||
(Some(id), _) => run(app, &ctx, "camera.selectPoints", json!({"points": [id], "toggle": mods.shift})),
|
||||
(None, Some(t3)) => run(app, &ctx, "camera.selectPoints", json!({"points": t3, "add": mods.shift})),
|
||||
(None, None) => run(app, &ctx, "camera.selectPoints", json!({"points": []})),
|
||||
}
|
||||
}
|
||||
// Right-click: the points under the cursor (selection, else the hovered triangle).
|
||||
let cid = egui::Id::new("viewer-camera-tracker-menu-points");
|
||||
if resp.secondary_clicked() {
|
||||
let pts: Vec<u32> = if !selected.is_empty() {
|
||||
selected.clone()
|
||||
} else if let Some(p) = resp.interact_pointer_pos() {
|
||||
point_at(p).map(|i| vec![i]).or_else(|| triangle_at(&hits, p).map(|t| t.to_vec())).unwrap_or_default()
|
||||
} else {
|
||||
vec![]
|
||||
};
|
||||
ui.data_mut(|d| d.insert_temp(cid, pts));
|
||||
}
|
||||
let base = json!({"layer": a.layer.0, "effect": a.uid});
|
||||
resp.context_menu(|ui| {
|
||||
let pts: Vec<u32> = ui.data(|d| d.get_temp(cid)).unwrap_or_default();
|
||||
let with = |kind: Option<&str>, multiple: bool| {
|
||||
let mut q = base.clone();
|
||||
q["points"] = json!(pts);
|
||||
if let Some(k) = kind {
|
||||
q["kind"] = json!(k);
|
||||
}
|
||||
if multiple {
|
||||
q["multiple"] = json!(true);
|
||||
}
|
||||
q
|
||||
};
|
||||
let some = !pts.is_empty();
|
||||
let many = pts.len() > 1;
|
||||
let mut pick: Option<(&str, serde_json::Value)> = None;
|
||||
if ui.add_enabled(some, egui::Button::new("Set Ground Plane and Origin")).clicked() {
|
||||
pick = Some(("camera.setGroundPlane", with(None, false)));
|
||||
}
|
||||
ui.separator();
|
||||
for (label, kind) in [
|
||||
("Create Text and Camera", "text"),
|
||||
("Create Solid and Camera", "solid"),
|
||||
("Create Null and Camera", "null"),
|
||||
("Create Shadow Catcher, Camera and Light", "shadowCatcher"),
|
||||
] {
|
||||
if ui.add_enabled(some, egui::Button::new(label)).clicked() {
|
||||
pick = Some(("camera.createFromSolve", with(Some(kind), false)));
|
||||
}
|
||||
}
|
||||
ui.separator();
|
||||
for (label, kind) in
|
||||
[("Create Multiple Text Layers and Camera", "text"), ("Create Multiple Solids and Camera", "solid"), ("Create Multiple Nulls and Camera", "null")]
|
||||
{
|
||||
if ui.add_enabled(many, egui::Button::new(label)).clicked() {
|
||||
pick = Some(("camera.createFromSolve", with(Some(kind), true)));
|
||||
}
|
||||
}
|
||||
ui.separator();
|
||||
if ui.add_enabled(some, egui::Button::new("Delete Selected Points")).clicked() {
|
||||
pick = Some(("camera.deletePoints", with(None, false)));
|
||||
}
|
||||
if let Some((cmd, q)) = pick {
|
||||
ui.close();
|
||||
run(app, &ctx, cmd, q);
|
||||
}
|
||||
});
|
||||
if resp.hovered()
|
||||
&& !selected.is_empty()
|
||||
&& ui.input(|i| i.key_pressed(egui::Key::Delete) || i.key_pressed(egui::Key::Backspace))
|
||||
&& !ctx.egui_wants_keyboard_input()
|
||||
{
|
||||
run(app, &ctx, "camera.deletePoints", json!({"layer": a.layer.0, "effect": a.uid, "points": selected}));
|
||||
}
|
||||
}
|
||||
|
||||
/// "Analyzing in background (step 1 of 2)" / "Solving camera" over the analysed layer, and a
|
||||
/// warning when the solve failed (unless Hide Warning Banner).
|
||||
fn banner(app: &mut EffectcraftApp, painter: &egui::Painter, map: &ViewerMap, ectx: &EvalCtx, l2c: &dyn Fn(&EvalCtx, &Layer) -> Mat3) {
|
||||
let comp = ectx.comp;
|
||||
let time = ectx.time;
|
||||
let cid = ectx.comp_id;
|
||||
let draw = |app: &mut EffectcraftApp, layer: &Layer, text: String, color: Color32, id: &str| {
|
||||
let m = l2c(ectx, layer);
|
||||
let (w, h) = effectcraft_engine::render::source_size(&app.session.project, layer);
|
||||
let c = m.apply(gv2(w as f64 / 2.0, h as f64 / 2.0));
|
||||
let at = map.to_screen([c.x, c.y]);
|
||||
let galley = painter.layout_no_wrap(text.clone(), Tokens::medium(13.0), Color32::WHITE);
|
||||
let r = Rect::from_center_size(at, galley.size() + vec2(28.0, 14.0)).intersect(map.area);
|
||||
painter.rect_filled(r, 3.0, color);
|
||||
painter.galley(pos2(r.center().x - galley.size().x / 2.0, r.center().y - galley.size().y / 2.0), galley, Color32::WHITE);
|
||||
app.auto.add(id, r, &text);
|
||||
};
|
||||
if let Some((cc, cl, _)) = app.session.camera_target()
|
||||
&& cc == cid
|
||||
&& let Some(pr) = app.session.camera_progress()
|
||||
&& let Some(layer) = comp.layer(cl)
|
||||
&& layer.is_active_at(time)
|
||||
{
|
||||
draw(app, layer, pr.banner(), Color32::from_rgba_unmultiplied(0x1d, 0x4f, 0x9c, 0xe0), "viewer.cameraBanner");
|
||||
return;
|
||||
}
|
||||
// A failed solve: tracks without a solve while nothing is running or queued.
|
||||
for l in comp.layers.iter().filter(|l| l.is_active_at(time)) {
|
||||
let Some(fx) = l.effects() else { continue };
|
||||
for g in fx.groups().filter(|g| matches!(&g.kind, effectcraft_engine::project::GroupKind::Effect { effect } if effect == ct::ID)) {
|
||||
let params = effectcraft_engine::camera_track::static_params(g);
|
||||
let hide = effectcraft_engine::effects::warp_stab::param(¶ms, "advanced/hideWarningBanner").is_some_and(Value::as_bool);
|
||||
let solve_key = matches!(g.prop(ct::SOLVE_KEY).map(|p| &p.value), Some(Value::Str(s)) if !s.is_empty());
|
||||
let failed = solve_key && ct::tracks(¶ms).is_some() && ct::solve(¶ms).is_none();
|
||||
if failed && !hide && !app.session.camera_pending.iter().any(|(_, ll, u)| *ll == l.id && *u == g.uid) {
|
||||
draw(app, l, "Unable to solve camera".into(), Color32::from_rgba_unmultiplied(0x9c, 0x2a, 0x1d, 0xe0), "viewer.cameraWarning");
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Effect Controls: Analyze / Cancel, Create Camera and the solve report.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn editor(
|
||||
app: &mut EffectcraftApp,
|
||||
ui: &mut egui::Ui,
|
||||
p: &egui::Painter,
|
||||
layer: &Layer,
|
||||
g: &PropGroup,
|
||||
r: Rect,
|
||||
actions: &mut Vec<(String, serde_json::Value)>,
|
||||
) {
|
||||
let t = app.tokens;
|
||||
let running = app.session.camera_target().is_some_and(|(_, l, u)| l == layer.id && u == g.uid) && app.session.is_camera_analyzing();
|
||||
let busy = app.session.is_camera_analyzing();
|
||||
let me = json!({"layer": layer.id.0, "effect": g.uid});
|
||||
let b1 = Rect::from_min_size(pos2(r.min.x + 24.0, r.min.y + 4.0), vec2(84.0, 22.0));
|
||||
let b2 = Rect::from_min_size(pos2(b1.max.x + 8.0, b1.min.y), vec2(84.0, 22.0));
|
||||
let b3 = Rect::from_min_size(pos2(b2.max.x + 8.0, b1.min.y), vec2(110.0, 22.0));
|
||||
if widgets::text_button(ui, b1, "Analyze", false, &t, egui::Id::new(("ct-analyze", g.uid))).clicked() && !busy {
|
||||
actions.push(("camera.analyze".into(), me.clone()));
|
||||
}
|
||||
app.auto.add(&format!("effectControls.cameraTracker.{}.analyze", g.uid), b1, "Analyze");
|
||||
if running {
|
||||
if widgets::text_button(ui, b2, "Cancel", false, &t, egui::Id::new(("ct-cancel", g.uid))).clicked() {
|
||||
actions.push(("camera.cancel".into(), json!({})));
|
||||
}
|
||||
app.auto.add(&format!("effectControls.cameraTracker.{}.cancel", g.uid), b2, "Cancel");
|
||||
}
|
||||
let params = effectcraft_engine::camera_track::static_params(g);
|
||||
let solve = ct::solve(¶ms);
|
||||
if solve.is_some() {
|
||||
if widgets::text_button(ui, b3, "Create Camera", false, &t, egui::Id::new(("ct-camera", g.uid))).clicked() {
|
||||
actions.push(("camera.create".into(), me.clone()));
|
||||
}
|
||||
app.auto.add(&format!("effectControls.cameraTracker.{}.createCamera", g.uid), b3, "Create Camera");
|
||||
}
|
||||
let status = match app.session.camera_progress() {
|
||||
Some(pr) if running => pr.banner(),
|
||||
_ => match &solve {
|
||||
Some(sv) => format!("Method Used: {} Average Error: {:.2} pixels", sv.method_used.label(), sv.average_error),
|
||||
None if ct::tracks(¶ms).is_some() && app.session.camera_pending.iter().any(|(_, l, u)| *l == layer.id && *u == g.uid) => {
|
||||
"Waiting to solve".into()
|
||||
}
|
||||
None if ct::tracks(¶ms).is_some() => "Unable to solve camera: try another Solve Method or Shot Type".into(),
|
||||
None if app.session.camera_pending.iter().any(|(_, l, u)| *l == layer.id && *u == g.uid) => "Waiting to analyze".into(),
|
||||
None => "Not analyzed: click Analyze".into(),
|
||||
},
|
||||
};
|
||||
p.text(pos2(r.min.x + 24.0, r.min.y + 38.0), Align2::LEFT_CENTER, status, Tokens::ui(11.5), t.text_dim);
|
||||
if let Some(sv) = &solve {
|
||||
let f = sv.focal();
|
||||
p.text(
|
||||
pos2(r.min.x + 24.0, r.min.y + 56.0),
|
||||
Align2::LEFT_CENTER,
|
||||
format!("{} solved points · Horizontal Angle of View {:.1}°", sv.points.len(), sv.hfov(f)),
|
||||
Tokens::ui(11.5),
|
||||
t.text_faint,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Height of [`editor`].
|
||||
pub const EDITOR_HEIGHT: f32 = 68.0;
|
||||
@@ -545,6 +545,7 @@ fn editor_height(effect: &str, width: f32) -> f32 {
|
||||
match effect {
|
||||
"ec.color.curves" => 34.0 + curves_size(width) + 26.0,
|
||||
effectcraft_engine::effects::warp_stab::ID => 50.0,
|
||||
effectcraft_engine::effects::camera_tracker::ID => super::camera_tracker_ui::EDITOR_HEIGHT,
|
||||
"ec.color.levels" | "ec.color.levelsic" => 34.0 + 80.0 + 58.0,
|
||||
"ec.color.autolevels" | "ec.color.autocontrast" | "ec.color.autocolor" => 24.0 + 80.0 + 12.0,
|
||||
_ => 0.0,
|
||||
@@ -920,6 +921,7 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, rect: Rect) {
|
||||
match effect.as_str() {
|
||||
"ec.color.curves" => curves_editor(app, ui, &bp, &layer, g, &ectx, er, &mut actions),
|
||||
effectcraft_engine::effects::warp_stab::ID => warp_editor(app, ui, &bp, &layer, g, er, &mut actions),
|
||||
effectcraft_engine::effects::camera_tracker::ID => super::camera_tracker_ui::editor(app, ui, &bp, &layer, g, er, &mut actions),
|
||||
"ec.color.levels" | "ec.color.levelsic" => levels_editor(app, ui, &bp, &layer, g, effect, &ectx, er, &mut actions),
|
||||
_ => histogram_only(app, ui, &bp, g, er),
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! Panel bodies.
|
||||
|
||||
pub mod anim_tools;
|
||||
pub mod camera_tracker_ui;
|
||||
pub mod comp_settings;
|
||||
pub mod dialogs;
|
||||
pub mod dialogs_3d;
|
||||
|
||||
@@ -130,11 +130,14 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, rect: Rect) {
|
||||
let cur = current(app);
|
||||
let can_track = has_comp && source.is_some();
|
||||
|
||||
// Track Camera / Warp Stabilizer (not available) and Track Motion / Stabilize Motion.
|
||||
// Track Camera / Warp Stabilizer and Track Motion / Stabilize Motion.
|
||||
let r1 = Rect::from_min_size(pos2(x0, y), vec2(half, 22.0));
|
||||
let r2 = Rect::from_min_size(pos2(x0 + half + 6.0, y), vec2(half, 22.0));
|
||||
let src_p = source.map(|l| json!({"layer": l.0})).unwrap_or_else(|| json!({}));
|
||||
button(app, ui, r1, "Track Camera", false, "tracker.trackCamera");
|
||||
let can_camera = app.session.is_enabled("track.camera") && !app.session.is_camera_analyzing();
|
||||
if button(app, ui, r1, "Track Camera", can_camera, "tracker.trackCamera") {
|
||||
run(app, &ctx, "track.camera", json!({}));
|
||||
}
|
||||
let can_warp = app.session.is_enabled("track.warpStabilizer") && !app.session.is_warp_analyzing();
|
||||
if button(app, ui, r2, "Warp Stabilizer", can_warp, "tracker.warpStabilizer") {
|
||||
run(app, &ctx, "track.warpStabilizer", json!({}));
|
||||
|
||||
@@ -1144,6 +1144,9 @@ pub fn show(app: &mut EffectcraftApp, ui: &mut egui::Ui, rect: Rect) {
|
||||
app.session.open_comp(*item);
|
||||
}
|
||||
}
|
||||
// 3D Camera Tracker points, targets and their menu (on top of the viewer's gestures while
|
||||
// the effect is selected), and its analysis banner.
|
||||
super::camera_tracker_ui::viewer_hook(app, ui, &painter, &map, &ectx, &|c, l| l2c(c, l).0);
|
||||
// Effect point controls and crosshair/eyedropper picks (on top of the viewer's gestures).
|
||||
crate::panels::effect_controls::viewer_hook(app, ui, &painter, &map, &ectx, &|c, l| l2c(c, l).0);
|
||||
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
//! Headless checks for the 3D Camera Tracker UI (egui_kittest, UI logic only): the viewer's track
|
||||
//! points and selection, the Effect Controls buttons and the Tracker panel's Track Camera button.
|
||||
//! The solve is a synthetic one written into the effect (the analysis itself is tested in the
|
||||
//! engine and track crates).
|
||||
|
||||
use effectcraft_engine::Session;
|
||||
use effectcraft_engine::effects::camera_tracker as ct;
|
||||
use effectcraft_engine::track::camtrack::{CameraSolve, ShotType, SolveMethod, SolvedFrame, SolvedPoint};
|
||||
use effectcraft_ui_egui::EffectcraftApp;
|
||||
use effectcraft_ui_egui::dock::PanelKind;
|
||||
use egui_kittest::Harness;
|
||||
use serde_json::json;
|
||||
|
||||
const W: f64 = 640.0;
|
||||
const H: f64 = 360.0;
|
||||
|
||||
fn app() -> (EffectcraftApp, u64, u64) {
|
||||
let mut s = Session::default();
|
||||
s.execute("comp.new", json!({"name": "Shot", "width": W, "height": H, "duration": 2})).unwrap();
|
||||
let plate = s.execute("layer.newSolid", json!({"name": "Plate", "color": "#406080"})).unwrap()["layer"].as_u64().unwrap();
|
||||
let r = s.execute("effect.apply", json!({"effect": ct::ID, "layers": [plate]})).unwrap();
|
||||
let uid = r["effects"][0].as_u64().unwrap();
|
||||
// A synthetic solve: a static camera (focal 600) looking at a 6 × 4 grid of points on z = 1.
|
||||
let frames = 60;
|
||||
let solve = CameraSolve {
|
||||
version: 1,
|
||||
size: [W, H],
|
||||
start: 0.0,
|
||||
frame_duration: 1.0 / 30.0,
|
||||
shot: ShotType::FixedAngle,
|
||||
method_used: SolveMethod::Typical,
|
||||
average_error: 0.21,
|
||||
frames: vec![SolvedFrame { rot: [0.0; 3], center: [0.0; 3], focal: 600.0, solved: true }; frames],
|
||||
points: (0..24)
|
||||
.map(|i| SolvedPoint {
|
||||
id: i,
|
||||
pos: [((i % 6) as f64 - 2.5) * 0.15, ((i / 6) as f64 - 1.5) * 0.12, 1.0 + 0.01 * (i % 3) as f64],
|
||||
error: 0.2,
|
||||
first: 0,
|
||||
last: frames as u32 - 1,
|
||||
})
|
||||
.collect(),
|
||||
ground: None,
|
||||
};
|
||||
s.execute("prop.set", json!({"layer": plate, "path": "effects/#1/solve", "value": solve.to_json()})).unwrap();
|
||||
s.camera_pending.clear();
|
||||
s.state.selected_props = vec![(effectcraft_engine::project::LayerId(plate), uid)];
|
||||
s.execute("layer.select", json!({"layers": [plate]})).unwrap();
|
||||
s.state.selected_props = vec![(effectcraft_engine::project::LayerId(plate), uid)];
|
||||
(EffectcraftApp::new(s), plate, uid)
|
||||
}
|
||||
|
||||
fn ids(h: &Harness<'_, EffectcraftApp>) -> Vec<String> {
|
||||
h.state().auto.previous.iter().chain(h.state().auto.elements.iter()).map(|e| e.id.clone()).collect()
|
||||
}
|
||||
|
||||
fn rect(h: &Harness<'_, EffectcraftApp>, id: &str) -> egui::Rect {
|
||||
let e = h.state().auto.find(id).unwrap_or_else(|| panic!("no {id}"));
|
||||
egui::Rect::from_min_size(egui::pos2(e.rect[0], e.rect[1]), egui::vec2(e.rect[2], e.rect[3]))
|
||||
}
|
||||
|
||||
fn click(h: &mut Harness<'_, EffectcraftApp>, at: egui::Pos2, modifiers: egui::Modifiers) {
|
||||
h.input_mut().events.push(egui::Event::ModifiersChanged(modifiers));
|
||||
h.input_mut().events.push(egui::Event::PointerMoved(at));
|
||||
h.input_mut().events.push(egui::Event::PointerButton { pos: at, button: egui::PointerButton::Primary, pressed: true, modifiers });
|
||||
h.input_mut().events.push(egui::Event::PointerButton { pos: at, button: egui::PointerButton::Primary, pressed: false, modifiers });
|
||||
h.run_steps(3);
|
||||
h.input_mut().events.push(egui::Event::ModifiersChanged(Default::default()));
|
||||
h.step();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn viewer_points_selection_and_effect_controls() {
|
||||
let (a, plate, uid) = app();
|
||||
let mut h = Harness::builder().with_size(egui::vec2(1600.0, 1000.0)).build_eframe(|_| a);
|
||||
h.state_mut().show_panel(PanelKind::Composition);
|
||||
h.run_steps(4);
|
||||
let all = ids(&h);
|
||||
assert!(all.iter().any(|i| i == "viewer.cameraTracker"), "track point overlay registered");
|
||||
let mut pts: Vec<&String> = all.iter().filter(|i| i.starts_with("viewer.cameraTracker.point.")).collect();
|
||||
pts.sort();
|
||||
pts.dedup();
|
||||
let n = pts.len();
|
||||
assert_eq!(n, 24, "every visible point is drawn");
|
||||
// Click a point: it is selected; Shift-click another adds it.
|
||||
let r = rect(&h, "viewer.cameraTracker.point.7");
|
||||
click(&mut h, r.center(), Default::default());
|
||||
assert_eq!(h.state().session.state.camera_points, vec![7]);
|
||||
let r = rect(&h, "viewer.cameraTracker.point.9");
|
||||
click(&mut h, r.center(), egui::Modifiers::SHIFT);
|
||||
assert_eq!(h.state().session.state.camera_points, vec![7, 9]);
|
||||
// Marquee over the first row selects its six points.
|
||||
let (a0, a5) = (rect(&h, "viewer.cameraTracker.point.0"), rect(&h, "viewer.cameraTracker.point.5"));
|
||||
let from = a0.left_top() - egui::vec2(6.0, 6.0);
|
||||
let to = a5.right_bottom() + egui::vec2(6.0, 6.0);
|
||||
h.input_mut().events.push(egui::Event::PointerMoved(from));
|
||||
h.input_mut().events.push(egui::Event::PointerButton { pos: from, button: egui::PointerButton::Primary, pressed: true, modifiers: Default::default() });
|
||||
h.step();
|
||||
for i in 1..=8 {
|
||||
h.input_mut().events.push(egui::Event::PointerMoved(from + (to - from) * (i as f32 / 8.0)));
|
||||
h.step();
|
||||
}
|
||||
h.input_mut().events.push(egui::Event::PointerButton { pos: to, button: egui::PointerButton::Primary, pressed: false, modifiers: Default::default() });
|
||||
h.run_steps(3);
|
||||
let mut sel = h.state().session.state.camera_points.clone();
|
||||
sel.sort();
|
||||
assert_eq!(sel, vec![0, 1, 2, 3, 4, 5]);
|
||||
// The right-click menu's Create Solid and Camera (via its command).
|
||||
let ctx = h.ctx.clone();
|
||||
effectcraft_ui_egui::menus::invoke(h.state_mut(), &ctx, "camera.createFromSolve", json!({"kind": "solid"})).unwrap();
|
||||
let comp = h.state().session.active_comp().unwrap();
|
||||
assert!(comp.layers.iter().any(|l| l.name == "3D Tracker Camera" && l.is_camera()));
|
||||
assert!(comp.layers.iter().any(|l| l.name.starts_with("Track Solid")));
|
||||
|
||||
// Effect Controls: Analyze and Create Camera.
|
||||
h.state_mut().session.execute("layer.select", json!({"layers": [plate]})).unwrap();
|
||||
h.state_mut().show_panel(PanelKind::EffectControls);
|
||||
h.run_steps(4);
|
||||
let all = ids(&h);
|
||||
for want in [format!("effectControls.cameraTracker.{uid}.analyze"), format!("effectControls.cameraTracker.{uid}.createCamera")] {
|
||||
assert!(all.contains(&want), "missing {want}");
|
||||
}
|
||||
// Tracker panel: Track Camera is available for the selected layer.
|
||||
h.state_mut().show_panel(PanelKind::Tracker);
|
||||
h.run_steps(3);
|
||||
assert!(ids(&h).iter().any(|i| i == "tracker.trackCamera"));
|
||||
assert!(h.state().session.is_enabled("track.camera"));
|
||||
}
|
||||
|
||||
/// Step the UI until background frame renders have landed.
|
||||
fn settle(h: &mut Harness<'_, EffectcraftApp>) {
|
||||
for _ in 0..600 {
|
||||
h.step();
|
||||
if h.state().frames.inflight() == 0 && h.state().frames.last_ms.lock().map(|v| *v > 0.0).unwrap_or(false) {
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(std::time::Duration::from_millis(20));
|
||||
}
|
||||
for _ in 0..4 {
|
||||
h.step();
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[ignore = "analyses the demo's 3D Showcase and renders with wgpu; run with --ignored (set EC_SNAPSHOT_DIR to keep PNGs)"]
|
||||
fn snapshot_showcase_track() {
|
||||
let dir = std::env::var("EC_SNAPSHOT_DIR").ok();
|
||||
let mut s = Session::default();
|
||||
s.execute("file.openDemoProject", json!({})).unwrap();
|
||||
s.execute("comp.new", json!({"name": "Camera Track", "width": 1920, "height": 1080, "duration": 4, "frameRate": 29.97})).unwrap();
|
||||
s.execute("layer.addItem", json!({"item": "3D Showcase"})).unwrap();
|
||||
s.execute("layer.select", json!({"layers": ["#1"]})).unwrap();
|
||||
s.execute("track.camera", json!({"layer": "#1", "wait": true})).unwrap();
|
||||
s.execute("time.set", json!({"time": 2.0})).unwrap();
|
||||
let mut h = Harness::builder().with_size(egui::vec2(1600.0, 1000.0)).with_pixels_per_point(1.0).wgpu().build_eframe(|_| EffectcraftApp::new(s));
|
||||
h.state_mut().show_panel(PanelKind::Composition);
|
||||
settle(&mut h);
|
||||
let pts: Vec<String> = ids(&h).into_iter().filter(|i| i.starts_with("viewer.cameraTracker.point.")).collect();
|
||||
assert!(pts.len() > 10, "{}", pts.len());
|
||||
// Hover between points to show a target, then select three and show the selection's target.
|
||||
let r = rect(&h, &pts[0]);
|
||||
h.input_mut().events.push(egui::Event::PointerMoved(r.center() + egui::vec2(12.0, 12.0)));
|
||||
h.run_steps(2);
|
||||
if let Some(d) = &dir {
|
||||
h.render().expect("render").save(format!("{d}/camera_tracker_hover.png")).unwrap();
|
||||
}
|
||||
let sel: Vec<u64> = pts.iter().take(5).filter_map(|i| i.rsplit('.').next()?.parse().ok()).collect();
|
||||
h.state_mut().session.execute("camera.selectPoints", json!({"points": sel})).unwrap();
|
||||
h.state_mut().session.execute("camera.createFromSolve", json!({"kind": "solid"})).unwrap();
|
||||
h.state_mut().session.execute("layer.select", json!({"layers": ["#3"]})).unwrap();
|
||||
settle(&mut h);
|
||||
if let Some(d) = &dir {
|
||||
h.render().expect("render").save(format!("{d}/camera_tracker_solid.png")).unwrap();
|
||||
}
|
||||
}
|
||||
@@ -146,6 +146,34 @@ effectcraft-cli exec warp.analyze '{"layer":"#1","wait":true}' shaky.ecproj --sa
|
||||
effectcraft-cli exec warp.status '{"layer":"#1"}' shaky.ecproj --json
|
||||
```
|
||||
|
||||
### 3D Camera Tracker
|
||||
|
||||
Animation ▸ Track Camera and the Tracker panel's button (`track.camera {layer?, shotType?:
|
||||
fixed|variable|specify, aov?, solveMethod?: auto|typical|flat|tripod, detailed?, wait?}`) apply
|
||||
Effect ▸ Perspective ▸ 3D Camera Tracker (or reuse the layer's) and analyse it in the background:
|
||||
"Analyzing in background (step 1 of 2)" tracks features, "Solving camera" solves.
|
||||
`camera.analyze {layer?, effect?, wait?}` re-runs it, `camera.cancel` stops it without writing
|
||||
anything. `camera.solveStatus` reports `progress`/`banner`, `analyzed`, `solved`, `methodUsed`,
|
||||
`averageError` (pixels), `focalLength`, `horizontalAngleOfView`, the current frame's `camera`
|
||||
(position, orientation, zoom) and `groundPlane`. `camera.points {time?}` lists the solved points
|
||||
visible now with their `id`, `comp` position, `depth`, `world` position and `error`.
|
||||
|
||||
Select points with `camera.selectPoints {points, add?, toggle?}` (the viewer's click / Shift-click /
|
||||
marquee), then the right-click menu's commands: `camera.setGroundPlane {points?}` (Set Ground Plane
|
||||
and Origin), `camera.createFromSolve {kind: text|solid|null|shadowCatcher, points? | target:
|
||||
{center, normal, size?}, multiple?}` (Create Text / Solid / Null / Shadow Catcher, Camera and Light,
|
||||
Create Multiple …), `camera.deletePoints {points?, wait?}` (re-solves; with Auto-delete Points
|
||||
Across Time the same feature's other tracks go too) and `camera.create` (the Create Camera button).
|
||||
Each is one undo step; the first create adds the one-node "3D Tracker Camera" keyed on every frame.
|
||||
Changing the layer's frames clears the analysis; changing `effects/#1/shotType`,
|
||||
`effects/#1/horizontalAngleOfView`, `effects/#1/advanced/solveMethod` re-solves the stored tracks.
|
||||
|
||||
```sh
|
||||
effectcraft-cli run shot.ecproj track.camera '{"layer":"#1","wait":true}' \
|
||||
camera.points '{}' --json
|
||||
effectcraft-cli run shot.ecproj camera.createFromSolve '{"kind":"solid","points":[12,40,77]}' --save
|
||||
```
|
||||
|
||||
### Text: styles and editing
|
||||
|
||||
Source Text holds character style runs and per-paragraph settings. `layer.setText` changes the
|
||||
|
||||
@@ -176,6 +176,22 @@ auto-scale; cached per analysis and settings) and warps each frame; Synthesize E
|
||||
borders from neighbouring frames read with `EffectHost::self_at`. The effect lives in the
|
||||
effects crate, hence the `effects → track` edge.
|
||||
|
||||
**3D Camera Tracker** (`effects::camera_tracker`, `engine::camera_track`, `effectcraft-track`'s
|
||||
`camtrack`): `camera.analyze` / `track.camera` render the layer's input to the effect on a
|
||||
background thread and run structure from motion in two steps. Step 1 follows Shi–Tomasi features
|
||||
through the clip with pyramidal Lucas–Kanade (forward–backward checked, re-detected where the frame
|
||||
has none) into long 2D tracks. Step 2 solves the camera: parallax keyframes; a two-view start from
|
||||
the essential matrix (normalised 8-point in RANSAC) or a planar homography decomposition; incremental
|
||||
resection and triangulation; a sparse Levenberg–Marquardt bundle adjustment (Schur complement over
|
||||
the points, Huber loss) with the focal length fixed, shared, or per frame; a log-spaced and
|
||||
golden-section focal search for Fixed Angle of View / Variable Zoom; a rotation-only model for tripod
|
||||
pans, chosen by Auto Detect when it explains the tracks as well. Tracks and solve are stored as JSON
|
||||
in hidden effect parameters with keys: changing the layer's frames clears both, changing Shot Type,
|
||||
Angle of View, Solve Method or deleting points only re-solves. The solve's canonical frame maps to
|
||||
comp space so the first frame's camera is the default comp camera (or so a chosen ground plane is
|
||||
the X-Z plane at the origin); `camera.createFromSolve` keys a one-node "3D Tracker Camera" on every
|
||||
frame and places text, solids, nulls or a shadow catcher and light on the target plane.
|
||||
|
||||
## 5. Expressions
|
||||
|
||||
Expressions are JavaScript, run by boa, with After Effects' object model (`thisComp`, `thisLayer`,
|
||||
|
||||
+5
-5
@@ -15,7 +15,7 @@ rendered, does not count as done.
|
||||
| P1 (professional daily use) | ≈ 38% |
|
||||
| P2 (long tail) | ≈ 17% |
|
||||
| Features done / partial / missing | 18 / 60 / 14 of 92 |
|
||||
| **Effects** | **257 of 298** After Effects 2026 effects by name (86%); ≈ 70% allowing for simplified implementations |
|
||||
| **Effects** | **258 of 298** After Effects 2026 effects by name (87%); ≈ 70% allowing for simplified implementations |
|
||||
| Remaining work | ≈ **167 agent-hours** (139 h of features + ≈ 20% for 1:1 polish against After Effects) |
|
||||
| Wall-clock estimate | ≈ **42–50 hours** with five agents working in parallel and one integrating; ≈ 31–35 h for P0 + P1 only |
|
||||
|
||||
@@ -73,7 +73,7 @@ library groundwork for mask tracking and Warp Stabilizer.
|
||||
| Remaining (audit-scale agent-hours) | ≈ 167 | ≈ 100 |
|
||||
| Wall clock, five parallel agents | 42–50 h | **≈ 8–10 h at today's measured pace; ≈ 25 h by the audit's conservative figures** |
|
||||
|
||||
What is left is concentrated in large systems: Roto Brush and Refine Edge, the 3D camera tracker,
|
||||
What is left is concentrated in large systems: Roto Brush and Refine Edge,
|
||||
Advanced 3D (models, PBR, image-based lighting), the Warp Stabilizer and mask-tracking UI on top of
|
||||
the new tracking library, a JavaScript scripting object model, SVG/PSD import, WebM, a disk cache,
|
||||
the missing effect categories (3D Channel, Immersive Video, OCIO), native macOS menus, and
|
||||
@@ -93,13 +93,13 @@ wasm threads.
|
||||
| Animation | 65% | 10.4 | puppet, Wiggler/Smoother/Motion Sketch (motion-path handles and the graph editor transform box landed in M5.8) |
|
||||
| Text | ≈ 85% | 1.5 | vertical type is basic (upright characters, no tate-chu-yoko), no OpenType feature panel, no text on 3D bevels; per-character styles, paragraph settings, on-canvas editing and the `sourceText` style API landed (M9.9–M9.10) |
|
||||
| Web | 60% | 2.5 | threads, browser storage, audio |
|
||||
| 3D | 58% | 20.5 | Advanced 3D (models, PBR, image-based light), 3D camera tracker, multi-view layouts |
|
||||
| 3D | 62% | 18.5 | Advanced 3D (models, PBR, image-based light), multi-view layouts (the 3D Camera Tracker landed in M12.6) |
|
||||
| Effects | 58% | 13.6 | Effect Controls widgets (angle dial, point crosshair, eyedropper, curves), GPU versions of more effects, 41 missing effects |
|
||||
| Interface | 55% | 10.4 | drag-to-dock and floating panels, preferences, native macOS menus (viewer rulers/snapping/channels/snapshots landed in M0.13) |
|
||||
| Project | 53% | 8.1 | 8/16/32-bit pipeline and colour management, auto-save, proxies, folder moves |
|
||||
| Masks & roto | 52% | 10.0 | Roto Brush, variable-width mask feather points (mask tracking and Mask Interpolation landed in M6.6) |
|
||||
| Preview | 55% | 6.1 | GPU 3D and adjustment layers, disk cache (region of interest, snapshots, exposure and Fast Previews landed in M0.13) |
|
||||
| Tracking | ≈ 70% | 4.0 | 3D camera tracker, face tracking, Subspace Warp's mesh warp and rolling-shutter repair (point tracker, mask tracking and Warp Stabilizer landed in M6.x / M12.5) |
|
||||
| Tracking | ≈ 85% | 2.5 | face tracking, Subspace Warp's mesh warp, rolling-shutter repair, lens distortion in the camera solve (point tracker, mask tracking, Warp Stabilizer and the 3D Camera Tracker landed in M6.x / M12.5 / M12.6) |
|
||||
| Paint | 0% | 4.0 | Brush, Clone Stamp, Eraser |
|
||||
|
||||
## Effects still missing
|
||||
@@ -107,7 +107,7 @@ wasm threads.
|
||||
Immersive Video (all 12), 3D Channel (all 8, including Cryptomatte), Color Stabilizer and the five
|
||||
OCIO transforms, CC Flo Motion, Liquify, Rolling Shutter Repair, Compressor,
|
||||
Distortion and Gate (audio), Camera-Shake Deblur, CC Radial Blur, CC Hair, Particle Playground,
|
||||
Keylight (our Screen Key stands in), Color Profile Converter and the 3D Camera Tracker. Boris FX
|
||||
Keylight (our Screen Key stands in) and Color Profile Converter. Boris FX
|
||||
Mocha and Cineware are third-party and not counted.
|
||||
|
||||
## Highest-value gaps, in order
|
||||
|
||||
Reference in New Issue
Block a user