Image-generation model support: memory-architecture suggestions & gotchas from shipping MLX diffusion pipelines

Hi — first, thanks for nativ (and for mlx-vlm/mlx-audio, which several of our Swift-MLX ports build on). We've spent the last year shipping MLX diffusion/video pipelines on Apple silicon (FLUX-class image models, Wan-family video, multi-component 3D texture pipelines), and since the README lists dedicated image-generation model support as coming soon — and the Swift side of the app (ImageGeneration feature, `/v1/images/*` clients, diffusion capability detection in `LocalModelDiscovery`) already looks ready for it — we wanted to share the things that bit us hardest on the serving/memory side, in case any of it saves you time. Offering these as suggestions, not asks; happy to elaborate or contribute on any of them. ## The big one: weights-based memory estimates break for diffusion Nativ's "likely fits in memory" estimate (params × bytes/param + headroom) works well for LLMs, but for diffusion pipelines **activation memory dominates and is invisible to that estimate**. Two properties we've measured repeatedly: - **Activation peak is roughly dtype-independent.** Quantizing the transformer shrinks resident weights a lot but barely moves the transient peak (attention/scratch buffers are computed in fp16/fp32 regardless). On a large video DiT we measured ~12–15 GB of activation whether the weights were bf16, int8, or int4. A weights-only estimate will green-light an int4 pipeline that OOMs at generation time. - **The peak is resolution-dependent**, driven mostly by VAE de...