Comment on: w8a16 (int8 GPTQ) target: Marlin repack CUDA IMA (wedges card until reboot); every fallback route fails — Conch 23 tok/s, fp16 blocked by bf16-only spec-decode kernels
Repo: syv-ai/qwen38-27b-rtx3090 by ahnguyen17
**Root cause found and worked around on our 170HX — the IMA is sm80 VMM corruption under allocation churn, not a kernel pointer bug.** Full story below; patch linked at the end.
## What actually faults
With your `7db2d1f` fail-fast and `expandable_segments:False`, the w8a16 target + W4A16 DFlash2 drafter still Xid-31'd during `gptq_marlin_repack` — but the faulting frame **drifted between runs** (once `empty_cache`, once a `.to(int32)` cast, once mid-repack), always with `Xid 31 FAULT_INFO_TYPE_REGION_VIOLATION … VIRT_WRITE` at a different VA. Classic async-fault signature: the Python traceback is a bystander.
Controlled experiments (5 reboots, sadly the card wedges per attempt):
| Control | Result |
|---|---|
| Bit-exact torch reimplementation of the repack, real weights, **sm86 (3090)**, `compute-sanitizer` | **0 errors** |
| Full engine-faithful drafter load on **sm80**, 10 GB allocator pressure | clean |
| Same chain on **sm80**, production occupancy (~27 GB resident) | **Xid 3...
GitHub Issue
SaaS Metrics