Comment on: Dual RTX 3090 (TP=2) vs single: a controlled A/B on your harness — +16-35%, DFlash2 residency, and the KV_MEM pin
Repo: syv-ai/qwen38-27b-rtx3090 by mhenrichsen
This is the best-run field report this repo has received — a genuine controlled A/B, spread stated, negative results kept, and the two findings that touch deliberate calibrations flagged instead of papered over. Both of those findings are correct, and both are now fixed on main. Point by point, then what I owe you, then @dpcartwright's T=15 result, which I think is the most interesting datapoint in the thread.
## 4.3 — the KV_MEM pin: you are right, and the launcher now skips it at TP>1
The pin is a single-card constant (the 24 GiB transient margin is sharp — 396 MiB dead / 436 MiB fine, gotcha 39), and vLLM applies `--kv-cache-memory` per worker, so at TP=2 it strands ~8 GiB per card exactly as you measured. As of `60921ed` the launcher detects `--tensor-parallel-size`/`-tp` in `EXTRA_ARGS` and, unless you exported a `KV_MEM` yourself, sizes the pool from `GPU_UTIL` with a note — your suggested fix, taken as suggested. Your unpin A/B (302,223 tokens for deltas inside spread, boots c...
GitHub Issue
Parent Entity
State: Open • Comments: 4
SaaS Metrics