Comment on: w8a16 (int8 GPTQ) target: Marlin repack CUDA IMA (wedges card until reboot); every fallback route fails — Conch 23 tok/s, fp16 blocked by bf16-only spec-decode kernels
Repo: syv-ai/qwen38-27b-rtx3090 by mhenrichsen
This is the best piece of debugging this tracker has seen — five reboots on a card that wedges per attempt, a bit-exact CPU reimplementation as the control, and the discipline to notice the faulting frame *drifting* and conclude the code in the traceback is a bystander. "When vanilla elementwise kernels fault, the page mappings under them are guilty" is the sentence the next GA100 owner needs, and it is now **gotcha 41** on main (`60daef8`), with your patch linked and the `expandable_segments` independent-trigger warning kept next to it. Your 170HX numbers are in the README's new "Results from other hardware" section, method-flagged (133.7 on the shipped target — the first sm80 datapoint, level with the 3090 — and 97.8 on your w8a16 with the workaround).
## Taking you up on the candidate patch
Your in-tree diagnosis points at a fix I can actually build and half-test from here: keep the repack's Python-side intermediates small — chunked unpack/pack over bounded column slices, int32 st...
GitHub Issue
SaaS Metrics