Comment on: Review spiritbuun's CUDA fork for portable optimizations

## buun's latest results (2026-03-26) ### Decode is FLAT across all context lengths This is the key finding — decode stays constant at ~30 tok/s from 4K to 128K: | Context | Prefill | Decode | Notes | |---------|---------|--------|-------| | 4K | 1123 | 30.03 | | | 32K | 980 | 29.83 | | | 65K | 847 | 29.79 | **q8_0 OOMs here (~28GB)** | | 98K | 748 | 29.86 | | | 128K | 671 | 29.89 | Full context window | Our Metal: decode degrades 0.91x → 0.72x over the same range. buun's CUDA stays flat. **This is the optimization we need to port.** ### All turbo configs beat q8_0 quality | Config | Compression | PPL | vs q8_0 | Prefill | Decode | |--------|------------|-----|---------|---------|--------| | q8_0 (baseline) | 2.0x | 5.8375 | — | 1133 | 31.04 | | **LA-1 turbo3** | **3.5x** | **5.7690** | **-1.17%** | 1128 | 30.25 | | LA-5 turbo3 | 4.2x | 5.8091 | -0.49% | 1119 | 30.03 | | turbo3 uniform | 4.9x | 5.8323 | -0.09% | 1125 | 30.04 | | turbo4 | 3.8x | 5.8186 | -0.32% | 588 | 29.43 | ###...