Precision handling within the `h3_gqa_causal_bf16` kernel, specifically the intermediate BF16 rounding during scaled query processing for Qwen causal GQA.
Raw Developer Origin & Technical Request
GitHub Issue
Aug 11, 2026
## Problem
`h3_gqa_causal_bf16` scales each BF16 query value in F32, but then rounds the product back to BF16 before the F32 QK contraction:
```metal
shared_query[d] = h3_bf16_to_f32(h3_f32_to_bf16(
h3_bf16_to_f32(query[q_base + d]) * args.scale));
```
`shared_query` is a `threadgroup float` array and the dot product uses F32 `fma`, so this intermediate BF16 round loses precision without reducing storage or bandwidth.
```text
Current: BF16 Q -> F32 scale -> BF16 round -> F32 QK contraction
Expected: BF16 Q -> F32 scale --------------> F32 QK contraction
```
The lost bits perturb the attention scores before softmax. This path is used by the Qwen3-VL prompt encoder's causal GQA kernel; the DiT attention path uses MPSGraph SDPA and is not affected by this specific expression.
## Reproduction and evidence
I exercised the real Metal kernel on an Apple M5 with deterministic synthetic BF16 inputs at the production head width:
- sequence: 32
- query heads: 8
- KV heads: 2
- head dimension: 128
- scale: `1 / sqrt(128)`
- 64 trials, 2,097,152 output elements
Each GPU result was compared at the kernel's BF16 output boundary with an explicit CPU attention reference that keeps the scaled query in F32.
| Metric versus F32 reference | Current BF16-rounded Q | F32-scaled Q | Change |
|---|---:|---:|---:|
| Maximum absolute error | 0.0924071 | 0.00390625 | 95.77% lower |
| RMSE | 0.00194536 | 0.000868307 | 55.37% lower |
| BF16 output mismatch rate | 21.6468% | 0.007486% | 99.9...
Developer Debate & Comments
No active discussions extracted for this entry yet.
Adjacent Repository Pain Points
Other highly discussed features and pain points extracted from antirez/h3.c.
Frequently Asked Questions
Market intelligence mapped to Precision handling within the `h3_gqa_causal_bf16` kernel, specifically the intermediate BF16 rounding during scaled query processing for Qwen causal GQA..
How is Precision handling within the `h3_gqa_causal_bf16` kernel, specifically the intermediate BF16 rounding during scaled query processing for Qwen causal GQA. positioned in the market?
What architecture is tied to Precision handling within the `h3_gqa_causal_bf16` kernel, specifically the intermediate BF16 rounding during scaled query processing for Qwen causal GQA.?
Engagement Signals
Cross-Market Term Frequency
Quantifies the cross-market adoption of foundational terms like scale and sequence by tracking occurrence frequency across active SaaS architectures and enterprise developer debates.
SaaS Metrics