Gemini Executive Synthesis

turboquant_plus compilation issues with CUDA on specific GPU architectures.

Technical Positioning

Compatibility and support for modern GPU hardware in AI/ML development.

SaaS Insight & Market Implications

This issue highlights a critical compatibility problem: turboquant_plus failing to compile with CUDA due to an 'Unsupported gpu architecture 'compute_120a'' error on an rtx 5060ti. This indicates a significant developer pain point in deploying AI/ML tools on modern hardware, particularly within WSL2 environments. The error suggests outdated CUDA toolkit configurations or a lack of support for newer GPU architectures. For SaaS providers in the AI/ML infrastructure space, ensuring broad and up-to-date hardware compatibility is paramount. Failure to support current GPU generations directly impedes adoption and creates significant friction for users attempting to leverage their compute resources. This necessitates continuous updates to build systems and CUDA dependencies.

Proprietary Technical Taxonomy

Raw Developer Origin & Technical Request

GitHub Issue Mar 31, 2026

Repo: TheTom/turboquant_plus

Unsupported gpu architecture 'compute_120a'

WSL2 - rtx 5060ti
Ubuntu.

# Windows (CUDA, use Developer Command Prompt or WSL2)
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

[ 7%] Building CUDA object ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/conv-transpose-1d.cu.o
nvcc fatal : Unsupported gpu architecture 'compute_120a'
gmake[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:227: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/conv2d-transpose.cu.o] Error 1
nvcc fatal : Unsupported gpu architecture 'compute_120a'
[ 10%] Built target ggml-cpu
nvcc fatal : Unsupported gpu architecture 'compute_120a'
gmake[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:257: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/convert.cu.o] Error 1
gmake[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:242: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/conv2d.cu.o] Error 1
nvcc fatal : Unsupported gpu architecture 'compute_120a'
gmake[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:212: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/conv2d-dw.cu.o] Error 1
gmake[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:287: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/cpy.cu.o] Error 1
nvcc fatal : Unsupported gpu architecture 'compute_120a'
gmake[2]: *** [ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/build.make:272: ggml/src/ggml-cuda/CMakeFiles/ggml-cuda.dir/count-equal.cu.o] Error 1
nvcc fatal : Unsupported gpu architecture 'compute_120a'
gmake[2]: **...

View Raw Source

Developer Debate & Comments

zekrom-vale • Mar 31, 2026

Looks like a WSL2 specific issue I got it to work on Ubuntu 24.04 LTS directly in a docker container. Or you are using a cuda version that does not have 120a support try upgrading to CUDA 12.8 or higher. I use cuda:13.1.1-devel-ubuntu24.04 for my container that has everything I need to build it and it works fine. (Cuda version 13.1.1) [Dockerfile.txt](https://github.com/user-attachments/files/26391116/Dockerfile.txt)

Ezzz-dev • Apr 1, 2026

I am using Cuda 13.2 as instructed by NVIDIA in WSL2

Adjacent Repository Pain Points

Other highly discussed features and pain points extracted from TheTom/turboquant_plus.

Quality validation: perplexity, KL divergence, and NIAH benchmarks

Extracted Positioning

turbo3 quantization for LLM KV cache compression

Achieving 4.6x compression with quality (perplexity, KL divergence, NIAH) comparable to q8_0 (within 2% PPL) and superior to q4_0, while maintaining high inference speed.

Top Replies

TheTom • Mar 25, 2026

## CRITICAL: Perplexity test reveals quality failure | Cache | PPL | vs f16 | |-------|-----|--------| | f16 | 6.121 | baseline | | q8_0 | 6.111 | -0.16% | | q4_0 | 6.142 | +0.34% | | **turbo3** | ...

TheTom • Mar 25, 2026

## Root causes found ### 1. V cache in rotated space Python verification: dequant output has cosine=0.02 with input (garbage). After inverse rotation: cosine=0.987 (correct). V cache values MUST be...

TheTom • Mar 25, 2026

## QUALITY FIXED ✅ Perplexity with inverse rotation restored in dequant: | Cache | PPL | vs q8_0 | |-------|-----|---------| | f16 | 6.121 | — | | q8_0 | 6.111 | baseline | | q4_0 | 6.142 | +0.5% ...

Experiment: Fused Q·Centroid compressed attention for turbo3 decode

Extracted Positioning

`turbo3` decode performance for LLM inference on Apple Silicon (M1, M2 Pro, M5 Max), specifically addressing the 'decode cliff' at increasing context depths.

Achieving flat, high-performance `turbo3` decode ratios (0.90x+ of `q8_0`) across all context depths on Apple Silicon, minimizing performance degradation from memory access patterns.

Top Replies

TheTom • Mar 27, 2026

## M2 Pro Results: Bit-Arithmetic Dequant **Hardware:** Apple M2 Pro, Apple8 (1008), has_tensor=false, 32GB **Model:** Qwen2.5-7B-Instruct-Q4_K_M **Build:** experiment/m1-m2-decode-comparison (auto...

TheTom • Mar 27, 2026

## M2 Pro Results Update: Batched Extract IS a Win True baseline comparison (same branch chain, same build): | Depth | q8_0 | Main (const LUT) | Batched extract | Bit-arithmetic | |-------|------|-...

TheTom • Mar 27, 2026

## BREAKTHROUGH: Profiling isolation identifies exact bottleneck Added TURBO_PROFILE_MODE (0-4) to strip away dequant layers one at a time. ### M5 Max vs M2 Pro at 8K context decode: | Mode | What ...

Don't work on vulkan device

Extracted Positioning

TurboQuant (`-ctk turbo3 -ctv turbo3`) integration with Vulkan devices for LLM inference.

Achieving broad hardware compatibility for TurboQuant, specifically extending to Vulkan-enabled AMD GPUs.

Top Replies

TheTom • Mar 28, 2026

turbo3 currently only supports Metal, CUDA, and ROCm/HIP backends. the Vulkan backend doesn't have a SET_ROWS kernel for the turbo3 quant type yet. since you have an RX 7900 XTX, ROCm would be your...

ogbinar • Mar 30, 2026

i hope the rocm issues get fixed i'm interested to try this out!

TheTom • Mar 31, 2026

Yeah we really need some more rcom support from the community. i only have so many viable devices to play with at home

No Difference in tokens/sec - Ministral3 8B Q5_K_M

Extracted Positioning

TurboQuant (turbo3 and turbo4) performance optimization for LLM inference, specifically on Apple M1 hardware.

Achieving superior LLM inference speed (tokens/sec) through TurboQuant optimizations on Apple Silicon (M1).

Top Replies

MrMuhannadObeidat • Mar 29, 2026

I missed the part where you highlight the fact that tokens/sec may actually degrade with the added compression of KV cache. I tried with turbo3, do not see noticeable degradation but certainly see ...

zrlhk • Mar 31, 2026

这个只是对kv缓存压缩，所以只是提升了最大推理上下文的大小。对模型量化压缩和推理速度，是没有提升的。原来10G显存，如果是一个9b模型，最大上下文128k可能就OOM了。现在压缩后，就可以支持128k上下文了。

zekrom-vale • Mar 31, 2026

You can just use it like Ollama or LM studio without the slow development or wrapper overheads. I use llama cpp directly and use it with router mode and many llms configured with models.ini and int...

Engineering findings: K/V norm disparity + MSE > Prod + outlier mixed precision

Extracted Positioning

TurboQuant's quantization strategy, specifically regarding K/V norm disparity, attention quantization methods (MSE vs. Prod), and outlier detection (dynamic vs. fixed).

Advancing TurboQuant's quantization efficacy to achieve lower perplexity (PPL) and higher compression (lower average bit rates) through refined techniques.

Top Replies

TheTom • Mar 28, 2026

this is great work, thanks for sharing. the K/V norm disparity data across models is something we hadn't quantified — 182x ratio on Qwen2.5-1.5B is wild. that directly informs the head_dim=128 qual...

TheTom • Mar 28, 2026

update on this: we ran a full turbo4 investigation this week and your MSE > Prod finding is now independently confirmed on three setups: 1. our Metal (M5 Max): QJL ablation on turbo4 shows removing...

Frequently Asked Questions

Market intelligence mapped to turboquant_plus compilation issues with CUDA on specific GPU architectures..

How is turboquant_plus compilation issues with CUDA on specific GPU architectures. positioned in the market?

Based on our AI analysis of the original developer request, its primary technical positioning is: Compatibility and support for modern GPU hardware in AI/ML development.

Are engineers actively discussing turboquant_plus compilation issues with CUDA on specific GPU architectures.?

Yes, we have tracked 1 direct responses and active debates regarding this specific topic originating from GitHub Issue.

Which technical concepts are associated with turboquant_plus compilation issues with CUDA on specific GPU architectures.?

Our proprietary extraction maps turboquant_plus compilation issues with CUDA on specific GPU architectures. to adjacent architectural concepts including Unsupported gpu architecture 'compute_120a', WSL2, rtx 5060ti, Ubuntu.

Engagement Signals

Replies

open

Issue Status

Cross-Market Term Frequency

Quantifies the cross-market adoption of foundational terms like CUDA and Ubuntu by tracking occurrence frequency across active SaaS architectures and enterprise developer debates.