ROIpad ← Back to Search
github.com › issue comment

Comment on: Field report: OffloadingConnector (KV eviction to RAM/disk) works on this stack — 8 sharp edges and one open question

Repo: syv-ai/qwen38-27b-rtx3090 by mhenrichsen
Posted: Aug 26, 2026
The traced cell I owed you is done — your "store path works, load path never hits" is **reproduced on this card, mechanism found, and it is neither unliftable-by-design nor a hash miss**. It's an allocator geometry problem, and your own bf16-vs-KVarN split falls out of it exactly. Fixes and a boot-time warning are on main; the full story so you can check it against your VM. ## The reproduction KVarN `CTX=huge` + your exact `TieringOffloadingSpec` connector config (4 GiB CPU tier), instrumented at `CPUOffloadingManager.{lookup,prepare_store,complete_store,touch}` and the connector's key builder. Protocol: document A (~23.5k tokens) → B → C (fill the GPU pool, evict A) → A again. Result: `A first=22.2s, resend=22.3s` — full recompute, `CPU_to_GPU` zero. Your symptom, on bare metal, no VM in the picture. ## The mechanism, from the trace The store side is *perfect*: A's chunks are stored and `complete_store(success=True)` lands for every key (in one capture the very key the resend late...
Other Comments / Reviews