ROIpad ← Back to Search
github.com › issue comment

Comment on: Engineering findings: K/V norm disparity + MSE > Prod + outlier mixed precision

Repo: TheTom/turboquant_plus by TheTom
Posted: Mar 28, 2026
update on this: we ran a full turbo4 investigation this week and your MSE > Prod finding is now independently confirmed on three setups: 1. our Metal (M5 Max): QJL ablation on turbo4 shows removing QJL improves PPL from 6.1894 to 6.1756. QJL actively hurts. 2. buun's CUDA (RTX 3090): turbo4 degrades from -0.28% at 2K to +3.69% at 64K. QJL noise accumulates with context. 3. your GPT-2 data: Prod at +300% PPL vs MSE at +7.6%. we've dropped QJL from turbo4 entirely and fixed the dequant path (byte-aligned packing, direct extraction). turbo4 now matches turbo3 speed and quality. the QJL bit was pure waste. next step for us is looking at asymmetric K/V using your norm disparity data as a starting point.
GitHub Issue
Other Comments / Reviews