Comment on: w8a16 (int8 GPTQ) target: Marlin repack CUDA IMA (wedges card until reboot); every fallback route fails — Conch 23 tok/s, fp16 blocked by bf16-only spec-decode kernels

This is an unusually good report — five attempts with the blame isolated by controls, and a working-vs-broken vLLM version to bisect against. Taking your two asks in reverse order, because I can act on the second one and not the first. ## Ask 2: fail at startup. Done, in `7db2d1f`. You were right that this is a real bug independent of the checkpoint, and I can name the line. **The split-KV verify attention this repo adds is bf16-only, hardcoded, in two places** — `patches/spec-decode-attn.patch`: ```python qs = (q * scale).to(tl.bfloat16) # line 217 acc = acc * alpha[:, None] + tl.dot(p.to(tl.bfloat16), v).to(...) # line 236 ``` With `--dtype float16`, `v` is fp16 and the left operand of that `tl.dot` is bf16, which is your row 4 exactly: `AssertionError: Both operands must be same dtype. Got bf16 and fp16` at `spec_decode_attn.py:220`, surfaced as a `CompilationError`. The patch header does say *"bf16 KV cache only"*, but that reads as a KV-d...