Insight for: Image-generation model support: memory-architecture suggestions & gotchas from shipping MLX diffusion pipelines

Accurate memory estimation for image/video generation models (diffusion pipelines) on Apple Silicon.

Analyzed: Jul 22, 2026

Nativ's current memory estimation, effective for LLMs, fails critically for diffusion pipelines due to dominant and underestimated activation memory. This leads to Out-Of-Memory (OOM) errors despite seemingly sufficient resources, frustrating users attempting image/video generation. The issue is compounded by activation peak being largely dtype-independent and resolution-dependent, rendering weight-based estimates unreliable. Addressing this requires a sophisticated memory prediction model that accounts for activation memory, crucial for Nativ to credibly support generative AI workloads on Apple Silicon. Without accurate resource management, Nativ risks user dissatisfaction and limited adoption for advanced MLX applications.

MLX diffusion/video pipelines Apple silicon FLUX-class image models Wan-family video 3D texture pipelines ImageGeneration feature /v1/images/* clients diffusion capability detection LocalModelDiscovery weights-based memory estimates LLMs diffusion pipelines activation memory params × bytes/param + headroom activation peak dtype-independent quantizing the transformer resident weights transient peak attention/scratch buffers fp16/fp32 video DiT bf16 int8 int4 weights-only estimate OOMs generation time resolution-dependent VAE

GitHub Issue

Parent Entity

Image-generation model support: memory-architecture suggestions & gotchas from shipping MLX diffusion pipelines

State: Open