Comment on: Experiment: Fused Q·Centroid compressed attention for turbo3 decode
Repo: TheTom/turboquant_plus by TheTom
## BREAKTHROUGH: Profiling isolation identifies exact bottleneck
Added TURBO_PROFILE_MODE (0-4) to strip away dequant layers one at a time.
### M5 Max vs M2 Pro at 8K context decode:
| Mode | What | M5 (% ceil) | M2 (% ceil) |
|------|------|------------|------------|
| 1 | No-op ceiling | 78.9 (100%) | 24.5 (100%) |
| 2 | + norm read | 75.1 (95%) | 22.1 (90%) |
| 4 | + all byte reads | 75.2 (95%) | 21.9 (89%) |
| 3 | + qs extraction + LUT | 64.9 (82%) | 16.4 (67%) |
| 0 | + signs + full LUT | 59.2 (75%) | 14.0 (57%) |
| q8_0 | baseline | 78.8 | 22.1 |
### Key findings:
1. **No-op turbo3 is FASTER than q8_0 on M2** (24.5 vs 22.1) — compressed cache = less bandwidth. The format is not the problem.
2. **Constant memory LUT is 2x worse on M2 than M5:**
- Mode 4→3 (LUT cost): M5 loses 13.7%, M2 loses 25.1%
- Mode 3→0 (signs+more LUT): M5 loses another 8.6%, M2 loses another 14.7%
3. **Byte reading is NOT the bottleneck** — Mode 4 (all reads, no LUT) only costs 10% on both.
4....
GitHub Issue
Parent Entity
State: Open • Comments: 6
SaaS Metrics