Comment on: Don't work on vulkan device
Repo: TheTom/turboquant_plus by TheTom
turbo3 currently only supports Metal, CUDA, and ROCm/HIP backends. the Vulkan backend doesn't have a SET_ROWS kernel for the turbo3 quant type yet.
since you have an RX 7900 XTX, ROCm would be your best path — apollosenvy has a working ROCm PR here: https://github.com/TheTom/llama-cpp-turboquant/pull/5
that said, ROCm turbo3 still has some quality issues being worked out (head_dim=128 models especially). head_dim=256 models like Qwen3.5 should work.
GitHub Issue
SaaS Metrics