mirror of
https://github.com/storytold/figcraft.git
synced 2026-10-27 20:06:53 -06:00
M9: avoid read-only snapshot rewrites and duplicate buffers
Validated formatting, workspace Clippy, workspace tests excluding the two pre-existing localized startup assertion failures tracked in #1017, layers, assets, WASM and plugins. The startup tests are unchanged in this independent fix.
This commit is contained in:
@@ -1,5 +1,10 @@
|
||||
# FigCraft Roadmap
|
||||
|
||||
M9 / M7.4 server memory follow-up: read-only disconnects retain their durable client-id journal
|
||||
without rebuilding the image-bearing snapshot. Snapshot encoding appends into its header buffer,
|
||||
avoiding a second full payload copy while preserving the existing disk format. See
|
||||
[`docs/server-memory-retention.md`](docs/server-memory-retention.md).
|
||||
|
||||
**Stage: alpha** · next: beta, ~26 points and ~162–292 h away (one Opus 5.5 agent; ~33–67 h wall-clock with 5 agents)
|
||||
|
||||
> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-17 · **Change:** minor (community batch #753: Selection colors, smart-selection spacing, a Present tab; language table re-measured from the catalogs; community fixes from #524 up (55 PRs: `.fig` import fidelity, paste from other apps, the Actions palette, editor fixes, server limits) and a boolean-op crash guard; number variables in the text content picker (#629); frames and sections convert into each other (#719); bounded canvas pan overscan; community fixes from #478 up and a control-channel fix (#589); bounded canvas pan overscan; Typography spacing icons support scrubbing (#373); font scans skip named pipes; system theme on first run; recent files as cards with thumbnails on the home screen; community fixes: lost images reported on loading (#207), SVG import budgets (#190), Leptos code (#223), full screen (#227), server snapshots readable again (#188), owner-only preferences (#200), ruzstd 0.9.1 (#182), layer cycle repair (#205), batch rollback, Direct3D 12 on Windows, plugin metering (#192); vector edit mode regions: hover hatch, selection and drag; SVG/PDF exports refuse over-depth documents (#208); bounded MCP/control framing, connections and queues (#201); plugin log limits (#191); live-session revert guard (#198); attached MCP no longer replays after a lost reply (#194); repeat command recursion fix; the Layout panel's constrain-proportions link scales the other side, #220; core editor leftovers: component playground, auto gap handles, variables inside styles, inner shadows and background blur on groups, command-made text readable on dark pages, a one-step frame/group convert, delays on mouse triggers and manual overlay offsets, wrapped-line alignment; `.fig` export: instance sizes, variables with modes, shared styles, variable-width strokes; starter templates, a Templates tab and Getting started tips on the home screen; your online files; everyday editing partials: number fields, outline options, Hide when publishing, cursors, panel widths, dark pages; sharing with a join code; text box sizes measured against documents re-saved by other software; live sessions per document; .fig export checked by opening written files in other software; a public figcraft-server: open hosting, owner tokens, per-address limits, idle expiry, moderation, install.sh; experimental .fig export; keyboard shortcut parity suite and bindings; Simplified Chinese draft catalog; Latin American Spanish overlay; prototyping tail: gamepad triggers, widgets in prototypes, mode lookups; plugin directory on figcraft-server; Japanese and Korean draft catalogs and CJK UI fonts; plugin widgets and the plugin panel tail; French, German and Brazilian Portuguese draft catalogs; cursor chat and spotlight in live sessions; layers and assets panels behaviour suite and fixes; vector multi-edit and edit tail; dev resources links; extended collections tail; branch version history and finer list merges; headline percentages recomputed from the area table: feature breadth ~84%, core ~94%, mainstream ~50%; text lists, hanging punctuation and decoration styles; comments and version history behaviour suite and fixes; canvas navigation, guides and grids behaviour suite and fixes; export and inspect behaviour suite and fixes; .fig import: editable vector networks and more layer kinds; shapes and vector behaviour suite and fixes; paint and effects behaviour suite and fixes; frames, groups, sections and masks behaviour suite and fixes; styles and variables behaviour suite and fixes; prototyping behaviour suite and fixes; components behaviour suite and fixes; embedded image colour profiles; layers panel keyboard: F2 rename, Shift+F10 context menu, keyboard box selection, 10k-layer QA; per-side stroke weight variables; text behaviour suite: CJK line breaking, metrics, vertical trim; selection and transform behaviour suite and fixes; `.fig` import, best effort; Windows QA pass with eight fixes; auto layout behaviour suite and fixes; Display P3: gamut warning and soft-proof (true P3 blocked upstream); several windows; localization layer with a draft Spanish catalog; hosting figcraft-server: wss in the desktop app, server releases, deployment guide; plugin panels and settings; canvas fixes: boolean hit testing on the result, group drags, frame outlines and slices; code connect; stretch and scatter brushes, pen pressure; text on a path and pattern fills; team libraries on the server and swap library; branches: create, update from main, review and merge; animated GIF fills and TIFF images; Open recent, outlines for the selection and mask outlines: menu breadth 100%; extended collections, variables import/export, gradient stop and variant bindings; feature breadth ~80%; ready ~49%) · **Target:** Figma Design (desktop app 126.10.7)
|
||||
|
||||
@@ -127,13 +127,14 @@ fn u64_at(b: &[u8], at: usize) -> Option<u64> {
|
||||
|
||||
/// A snapshot file's bytes.
|
||||
pub fn encode_snapshot(generation: u64, state: &SessionState) -> Result<Vec<u8>, String> {
|
||||
let body = message::encode(state).map_err(|e| e.to_string())?;
|
||||
let mut out = Vec::with_capacity(SNAP_HEADER + body.len());
|
||||
let mut out = Vec::with_capacity(SNAP_HEADER);
|
||||
out.extend_from_slice(SNAP_MAGIC);
|
||||
out.push(FORMAT);
|
||||
out.extend_from_slice(&generation.to_le_bytes());
|
||||
out.extend_from_slice(&crc(&body).to_le_bytes());
|
||||
out.extend_from_slice(&body);
|
||||
out.extend_from_slice(&0u32.to_le_bytes());
|
||||
let mut out = message::encode_into(state, out).map_err(|e| e.to_string())?;
|
||||
let sum = crc(&out[SNAP_HEADER..]);
|
||||
out[13..SNAP_HEADER].copy_from_slice(&sum.to_le_bytes());
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
|
||||
@@ -1385,14 +1385,18 @@ impl Backend for Store {
|
||||
}
|
||||
|
||||
fn idle(&mut self, file: &str, server: &mut Server) -> bool {
|
||||
let dirty = self.files.get_mut(file).and_then(|e| e.log.as_mut()).is_some_and(|log| {
|
||||
let _ = flush_log(file, log);
|
||||
log.dirty
|
||||
let compact = self.files.get_mut(file).is_some_and(|entry| {
|
||||
let snapshot_seq = entry.meta.seq;
|
||||
entry.log.as_mut().is_some_and(|log| {
|
||||
// Issued client ids and refusals are durable in the log; only edits need a
|
||||
// new image-bearing snapshot. Still try a snapshot if syncing the log failed.
|
||||
flush_log(file, log).is_err() || (log.dirty && server.seq() > snapshot_seq)
|
||||
})
|
||||
});
|
||||
if dirty {
|
||||
if compact {
|
||||
self.compact(file, server);
|
||||
}
|
||||
// It closes anyway, everything in its snapshot: a compaction cut short needs no report.
|
||||
// It closes anyway, with its state in the snapshot and durable log.
|
||||
self.cut_short.retain(|f| f != file);
|
||||
if let Some(e) = self.files.get_mut(file) {
|
||||
e.log = None;
|
||||
|
||||
@@ -127,6 +127,34 @@ fn a_file_survives_a_crash_and_compacts() {
|
||||
assert_eq!(store.list(&|_| None)[0].seq, seq, "the description follows");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_only_sessions_keep_client_ids_without_rewriting_the_snapshot() {
|
||||
let dir = TempDir::new("read-only-reconnects");
|
||||
let c = StoreConfig { compact_after: 1000, ..cfg(dir.path()) };
|
||||
let (id, original) = {
|
||||
let mut store = Store::open(dir.path(), c.clone()).unwrap();
|
||||
let (id, _, _) = publish(&mut store, "Read only");
|
||||
let bytes = fs::read(dir.path().join(FILES).join(&id).join(SNAPSHOT)).unwrap();
|
||||
(id, bytes)
|
||||
};
|
||||
let mut last_client = 0;
|
||||
for _ in 0..6 {
|
||||
let mut store = Store::open(dir.path(), c.clone()).unwrap();
|
||||
let mut server = store.open(&id).unwrap();
|
||||
let client = join(&mut server, Access::View);
|
||||
assert!(client.id() > last_client, "issued ids survive closing and reopening the store");
|
||||
last_client = client.id();
|
||||
store.changed(&id, &mut server);
|
||||
// Idle must sync even if no periodic flush ran before the connection closed.
|
||||
assert!(store.idle(&id, &mut server));
|
||||
assert_eq!(store.compactions(), 0, "a client id does not require rebuilding the image-bearing snapshot");
|
||||
assert_eq!(fs::read(dir.path().join(FILES).join(&id).join(SNAPSHOT)).unwrap(), original);
|
||||
let log = disk::read_log(&fs::read(dir.path().join(FILES).join(&id).join(LOG)).unwrap()).unwrap();
|
||||
assert!(!log.entries.is_empty());
|
||||
assert_eq!(server.seq(), 0);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_long_log_is_compacted_while_open() {
|
||||
let dir = TempDir::new("compact");
|
||||
@@ -241,6 +269,19 @@ fn temporary_directories_are_removed() {
|
||||
assert!(!path.exists());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn snapshot_encoding_keeps_the_existing_framing_and_checksum() {
|
||||
let state = Server::new(doc("Snapshot framing"), "", figcraft_collab::Limits::default()).state().unwrap();
|
||||
let body = figcraft_collab::message::encode(&state).unwrap();
|
||||
let mut legacy = b"FCSN\x01".to_vec();
|
||||
legacy.extend_from_slice(&37u64.to_le_bytes());
|
||||
legacy.extend_from_slice(&crc32fast::hash(&body).to_le_bytes());
|
||||
legacy.extend_from_slice(&body);
|
||||
let encoded = disk::encode_snapshot(37, &state).unwrap();
|
||||
assert_eq!(encoded, legacy);
|
||||
assert_eq!(disk::decode_snapshot(&encoded).unwrap(), (37, state));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn random_bytes_never_panic_the_readers() {
|
||||
let mut rng = Rng::new(7);
|
||||
|
||||
@@ -11,6 +11,16 @@ use crate::message::{
|
||||
use crate::sim::Rng;
|
||||
use crate::{Error, Replica, diff};
|
||||
|
||||
#[test]
|
||||
fn encoding_into_a_prefix_keeps_the_wire_body() {
|
||||
let msg = moved_layer();
|
||||
let prefix = b"container header".to_vec();
|
||||
let bytes = crate::message::encode_into(&msg, prefix.clone()).unwrap();
|
||||
assert_eq!(&bytes[..prefix.len()], prefix);
|
||||
assert_eq!(&bytes[prefix.len()..], encode(&msg).unwrap());
|
||||
assert_eq!(decode::<ClientMsg>(&bytes[prefix.len()..], bytes.len()).unwrap(), msg);
|
||||
}
|
||||
|
||||
/// A typical edit: one layer dragged (its transform).
|
||||
fn moved_layer() -> ClientMsg {
|
||||
let mut doc = Document::new("f");
|
||||
|
||||
@@ -732,11 +732,13 @@ pub fn utc_date(secs: u64) -> String {
|
||||
|
||||
/// The binary form of `msg`: [`WIRE_VERSION`], then postcard.
|
||||
pub fn encode<T: Serialize>(msg: &T) -> Result<Vec<u8>, Error> {
|
||||
let body = postcard::to_allocvec(msg).map_err(|e| Error::Encode(e.to_string()))?;
|
||||
let mut out = Vec::with_capacity(body.len() + 1);
|
||||
out.push(WIRE_VERSION);
|
||||
out.extend(body);
|
||||
Ok(out)
|
||||
encode_into(msg, Vec::new())
|
||||
}
|
||||
|
||||
/// Append the binary form of `msg` to an owned prefix, without a second payload-sized buffer.
|
||||
pub fn encode_into<T: Serialize>(msg: &T, mut prefix: Vec<u8>) -> Result<Vec<u8>, Error> {
|
||||
prefix.push(WIRE_VERSION);
|
||||
postcard::to_extend(msg, prefix).map_err(|e| Error::Encode(e.to_string()))
|
||||
}
|
||||
|
||||
/// Read a binary message of at most `max_bytes` (untrusted input: never panics, allocates at most
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
# Server memory follow-up
|
||||
|
||||
Joining a persisted session issues a client id and makes its journal dirty, even if nobody edits
|
||||
the document. Previously, the last client leaving caused a full snapshot rewrite for that id.
|
||||
For image-heavy documents this creates large temporary allocations and unnecessary disk writes;
|
||||
an allocator can retain the freed memory after the room closes.
|
||||
|
||||
On idle, the server now flushes the journal and only compacts when the document sequence is newer
|
||||
than the stored snapshot, or when flushing failed and a snapshot must be attempted as a fallback.
|
||||
Issued ids and refusals remain durable in the journal. Existing size/count compaction thresholds
|
||||
remain in effect, and edited documents still compact when the last participant leaves.
|
||||
|
||||
Snapshot encoding also appends the versioned postcard body directly to the snapshot header's
|
||||
buffer, then fills the checksum. It no longer holds a separate complete encoded body alongside
|
||||
the finished snapshot. The bytes, generation, checksum and file format are unchanged.
|
||||
|
||||
Regression coverage uses synthetic documents: six read-only sessions across store reopenings
|
||||
must preserve increasing client ids without rewriting the snapshot; exact legacy framing and
|
||||
checksum must match; the binary message body must be unchanged when appended to a prefix.
|
||||
Existing crash recovery, damaged-input and edit compaction tests continue to apply.
|
||||
|
||||
This reduces unnecessary temporary allocations. It does not promise that the operating system
|
||||
immediately reclaims every freed allocation, or that all large-document memory costs are solved.
|
||||
Reference in New Issue
Block a user