ROIpad ← Back to Search
github.com › issue comment

Comment on: Field report: OffloadingConnector (KV eviction to RAM/disk) works on this stack — 8 sharp edges and one open question

Repo: syv-ai/qwen38-27b-rtx3090 by alexander99lab
Posted: Aug 24, 2026
Two follow-ups from running this in anger. **Edge 6 escalated from ops-hygiene to a production outage.** The engine died once (host OOM, most likely — nothing in the engine log), and then `restart: unless-stopped` did the rest: every automatic restart leaves the previous incarnation's mmap file behind, the next boot hits `OSError: [Errno 14] Bad address` in `shared_offload_region.py` `madvise`, and the container loops. We found it **70 restarts and ~3 hours later** with 70 ghost files in `/dev/shm` (sparse, so they fit — until reservation fails). A boot-time unlink of stale `vllm_offload_*.mmap` not owned by a live engine would turn this whole class into a non-event; we've deployed a cron watchdog as a stopgap (detect `restarting` + `Bad address` → wipe ghosts → restart). **KVarN KV (`kvarn_k4v2_g128`) + OffloadingConnector: store path works, load path never hits.** Same protocol as the bf16 runs in the report: eviction traffic flows (41 GB GPU→CPU over a full ladder+pyramid program ...
Other Comments / Reviews