ROIpad ← Back to Search
github.com › issue comment

Comment on: Field report: OffloadingConnector (KV eviction to RAM/disk) works on this stack — 8 sharp edges and one open question

Repo: syv-ai/qwen38-27b-rtx3090 by mhenrichsen
Posted: Aug 25, 2026
Answers to all three open questions, one shipped fix for the outage class, and one debt I am keeping. Your follow-up comment upgraded this report from useful to load-bearing — a 70-restart ghost loop found in production is exactly the kind of evidence a boot-time fix needs to justify itself. ## Edge 6: shipped in `60daef8` Both launchers now unlink stale `/dev/shm/vllm_offload_*.mmap` regions **that no live process maps** before boot — the liveness check is a `/proc/[pid]/maps` scan, so a second engine's live region on the same host is left alone. Your restart-loop scenario (engine dies, `restart: unless-stopped` re-enters, ghost blocks `madvise`, loop) becomes one deleted file and a log line. One honest caveat for your exact topology: inside a container with `ipc: host`, the scan can only see the container's own processes, so if you run **several engines sharing the host's `/dev/shm` across namespaces**, set `VLLM_OFFLOAD_KEEP_SHM=1` and keep your cron watchdog — the cleanup cannot ...
Other Comments / Reviews