Functionality and performance of OffloadingConnector (KV eviction to RAM/disk) for extending KV cache capacity on the Qwen3.8-27B stack.

Two follow-ups from running this in anger. **Edge 6 escalated from ops-hygiene to a production outage.** The engine died once (host OOM, most likely — nothing in the engine log), and then `restart: unless-stopped` did the rest: every automatic restart leaves the previous incarnation's mmap file behind, the next boot hits `OSError: [Errno 14] Bad address` in `shared_offload_region.py` `madvise`, and the container loops. We found it **70 restarts and ~3 hours later** with 70 ghost files in `/dev/shm` (sparse, so they fit — until reservation fails). A boot-time unlink of stale `vllm_offload_*.mmap` not owned by a live engine would turn this whole class into a non-event; we've deployed a cron watchdog as a stopgap (detect `restarting` + `Bad address` → wipe ghosts → restart). **KVarN KV (`kvarn_k4v2_g128`) + OffloadingConnector: store path works, load path never hits.** Same protocol as the bf16 runs in the report: eviction traffic flows (41 GB GPU→CPU over a full ladder+pyramid program ...

Answers to all three open questions, one shipped fix for the outage class, and one debt I am keeping. Your follow-up comment upgraded this report from useful to load-bearing — a 70-restart ghost loop found in production is exactly the kind of evidence a boot-time fix needs to justify itself. ## Edge 6: shipped in `60daef8` Both launchers now unlink stale `/dev/shm/vllm_offload_*.mmap` regions **that no live process maps** before boot — the liveness check is a `/proc/[pid]/maps` scan, so a second engine's live region on the same host is left alone. Your restart-loop scenario (engine dies, `restart: unless-stopped` re-enters, ghost blocks `madvise`, loop) becomes one deleted file and a log line. One honest caveat for your exact topology: inside a container with `ipc: host`, the scan can only see the container's own processes, so if you run **several engines sharing the host's `/dev/shm` across namespaces**, set `VLLM_OFFLOAD_KEEP_SHM=1` and keep your cron watchdog — the cleanup cannot ...

The traced cell I owed you is done — your "store path works, load path never hits" is **reproduced on this card, mechanism found, and it is neither unliftable-by-design nor a hash miss**. It's an allocator geometry problem, and your own bf16-vs-KVarN split falls out of it exactly. Fixes and a boot-time warning are on main; the full story so you can check it against your VM. ## The reproduction KVarN `CTX=huge` + your exact `TieringOffloadingSpec` connector config (4 GiB CPU tier), instrumented at `CPUOffloadingManager.{lookup,prepare_store,complete_store,touch}` and the connector's key builder. Protocol: document A (~23.5k tokens) → B → C (fill the GPU pool, evict A) → A again. Result: `A first=22.2s, resend=22.3s` — full recompute, `CPU_to_GPU` zero. Your symptom, on bare metal, no VM in the picture. ## The mechanism, from the trace The store side is *perfect*: A's chunks are stored and `complete_store(success=True)` lands for every key (in one capture the very key the resend late...