GitHub Issue

v0.4+ candidate: live-fork via memfd-backed source RAM + uffd_wp (deferred from v0.3)

Discovered On May 18, 2026
Primary Metric open
**Status:** Deferred from v0.3. The design and scaffolding are in the repo (see "What's already here" below) so the work can be picked up cleanly if/when the cost-benefit changes. The reason for deferral is in "Why deferred" below. ## Goal Cut BRANCH pause-window to ~30 ms regardless of source memory size, by replacing "pause source, write full memory.bin, resume" with "pause source, register WP on its memory, resume; children inherit a memfd view and the WP handler resolves source's post-fork writes lazily." Motivating measurement: `bench/pause-window/RESULTS-v0.2.md`. Today's pause is storage-bound: 163 ms (tmpfs) to 4.26 s (SATA SSD) for 513 MiB, scaling linearly with source memory. v0.2.5's prewarm ([PR #100](https://github.com/deeplethe/forkd/pull/100)) flattens the cold/warm ratio but doesn't change the absolute floor. ## What's already here (scaffolding from v0.3 cycle) | Artifact | Purpose | Notes | |---|---|---| | `docs/design/userfaultfd.md` | Full architectural design | Honest about the open question on source-divergence sync | | `crates/forkd-uffd/` | Workspace crate with Firecracker UDS handshake parser | Linux-only; tested over `socketpair(2)`; no event loop yet | | `crates/forkd-vmm/src/lib.rs` `MemoryBackend::Userfault` | Enum variant + ForkOpts field | `bail!`s in `restore_many_with` so no caller can rely on it | | `firecracker-patch/v0.3-memfd-backend.patch` | First-cut patch against Firecracker v1.10.1 | Not compile-tested; reviewed against real upstre...
View Raw Thread

Developer & User Discourse

WaylandYang • May 19, 2026
**Update 2026-05-19 — narrowing scope.** v0.3 phase 1 shipped 143× on vanilla Firecracker (no fork required). The `firecracker-patch/` directory that originally drafted a `MemoryBackend::Memfd` patch has been removed (PR opened).

Concretely, we're now explicitly NOT taking the memfd route, for four reasons:

1. **Phase 1 cleared ~85 % of the original target headroom** (30 ms target → 205 ms achieved on 4 GiB SSD) without any Firecracker change.
2. **memfd's value-add isn't sharing** — `mmap(MAP_PRIVATE)` of `memory.bin` across N children already gives kernel-CoW fan-out for free. memfd's real value is letting uffd_wp track source writes cleanly, but that's only useful for the live-fork architecture in `docs/design/userfaultfd.md`, which has its own deferred open questions.
3. **Fork maintenance is real cost**: own musl-via-docker CI, rebase on every upstream tag (~quarterly), track CVEs, weaken the "vanilla Firecracker" trust story users rely on. CodeSandbox went this route and hasn't...
WaylandYang • May 24, 2026
Draft RFC + implementation plan now live: #156

8-week phased plan: PoC → integrate into `forkd-uffd` → pause-window bench → hardening (write-heavy / NUMA / pre-5.7 fallback) → launch.

Target: BRANCH pause < 10 ms by removing the synchronous memory write entirely. Switch source RAM to memfd, arm `UFFDIO_WRITEPROTECT`, async dirty-page copier on uffd handler.

Comments / prior-art pointers especially welcome on the open questions in the doc — particularly the behavior of `UFFD_WP` on memfd-backed VMAs under `KVM_RUN`.
WaylandYang • May 27, 2026
Surface RFC posted at #174 — `DESIGN-v0.4-USER-API.md`.

Pins the CLI / REST / SDK shape before implementation starts so we don't re-litigate `--live` vs `--mode live` vs `--wp` mid-coding. Companion to the existing kernel-mechanism doc (`DESIGN-v0.4.md`), use-case inventory (`DESIGN-v0.4-USE-CASES.md`), and FC integration spike (`DESIGN-v0.4-PHASE3-SPIKE.md`).

Five open questions tagged for review in the doc; the most consequential is fail-vs-silent-fallback when the kernel doesn't support `uffd_wp` on memfd. Feedback welcome here or on the PR.
WaylandYang • Jun 4, 2026
Closing — v0.4 live BRANCH ships end-to-end across REST, CLI, SDKs, doctor, docs, and bench (Phase 7.1–7.5, PRs #204–#210). User-facing surface complete; release artifacts published.

**What landed:**
- Phase 7.1 — REST canonical `mode` field on BRANCH (#204)
- Phase 7.2 — CLI `--live` / `--no-wait` (#205)
- Phase 7.3 — Python / TypeScript / MCP SDK `mode` / `wait` / `live_fork` (#206)
- Phase 7.4 — `forkd doctor` uffd_wp + memfd_create capability checks (#207)
- Phase 7.5 — bench harness + RESULTS-v0.4.md on a clean source (#210)
- docs sweep — README + API.md + CHANGELOG + DESIGN-v0.4.md (#208)
- `forkd fork --live-fork` for local-boot path (#211)

**Bench (1.5 GiB python-numpy, Intel i7-12700, ext4 HDD):**

| mode | pause p50 | pause p90 | RT p50 |
|---|---:|---:|---:|
| live | 56 ms | 64 ms | 13.7 s (sync) / 69 ms (async) |
| diff | 202 ms | 418 ms | 13.5 s |
| full | 13 550 ms | 14 268 ms | 13.6 s |

**3.6× faster pause vs Diff; 200× faster RT with `wait: false`.** Full data + CSV...