Comment on: I did get it working, with a lot of pain, if your interested here's a readme I had claud crank out capturing the gotchas.

This helped me a ton! Managed to get it running, and wanted to add to the numbers: ## Performance Notes ### Expected Performance by Hardware | Machine | RAM | Bandwidth | Expected tok/s | |---------|-----|-----------|---------------| | M3 Max (reference) | 48 GB | ~400 GB/s | 4.4 | | M4 Max | 64 GB | ~546 GB/s | 5.0-5.5+ | | M1 Max | 64 GB | ~400 GB/s | 2.4-2.9+ | Tested on a MBP 16" fully loaded M1 Max series