ROIpad ← Back to Search
github.com › issue comment

Comment on: Experiment: Fused Q·Centroid compressed attention for turbo3 decode

Repo: TheTom/turboquant_plus by TheTom
Posted: Mar 27, 2026
## BREAKTHROUGH: Profiling isolation identifies exact bottleneck Added TURBO_PROFILE_MODE (0-4) to strip away dequant layers one at a time. ### M5 Max vs M2 Pro at 8K context decode: | Mode | What | M5 (% ceil) | M2 (% ceil) | |------|------|------------|------------| | 1 | No-op ceiling | 78.9 (100%) | 24.5 (100%) | | 2 | + norm read | 75.1 (95%) | 22.1 (90%) | | 4 | + all byte reads | 75.2 (95%) | 21.9 (89%) | | 3 | + qs extraction + LUT | 64.9 (82%) | 16.4 (67%) | | 0 | + signs + full LUT | 59.2 (75%) | 14.0 (57%) | | q8_0 | baseline | 78.8 | 22.1 | ### Key findings: 1. **No-op turbo3 is FASTER than q8_0 on M2** (24.5 vs 22.1) — compressed cache = less bandwidth. The format is not the problem. 2. **Constant memory LUT is 2x worse on M2 than M5:** - Mode 4→3 (LUT cost): M5 loses 13.7%, M2 loses 25.1% - Mode 3→0 (signs+more LUT): M5 loses another 8.6%, M2 loses another 14.7% 3. **Byte reading is NOT the bottleneck** — Mode 4 (all reads, no LUT) only costs 10% on both. 4....
GitHub Issue