MPS (Metal Performance Shaders)

MPS is Apple's high-level Metal-based compute library, exposed in PyTorch as the mps device backend. Calling model.to("mps") runs on the Apple Silicon GPU through MPS kernels.

MPS is workable for inference of small models but historically incomplete: many ops fall back to CPU, FP16 is supported but BF16 is not on older silicon, and large allocations sometimes hit RuntimeError: MPS backend out of memory even with available unified memory due to the 80% allocation limit.

For local LLM inference, llama.cpp's native Metal kernels and MLX-LM both outperform PyTorch MPS by 1.5–3×. Use MPS for quick PyTorch experiments; use llama.cpp or MLX for production.

A developer prototyping a fine-tuned Llama 3.1 8B classifier on an M2 MacBook Pro loads it with model.to("mps") in PyTorch and gets fast enough iteration for a Jupyter notebook — a few tokens per second is fine when you're debugging prompts, not benchmarking throughput. But when they move to serving it for a small team, they hit intermittent MPS backend out of memory errors on a 16 GB machine even though Activity Monitor shows 6 GB free, because MPS won't allocate past roughly 80% of unified memory regardless of what's actually available. Switching the same GGUF-converted weights to llama.cpp's Metal backend not only fixes the OOM ceiling but also roughly doubles tokens/sec, because llama.cpp's hand-written Metal kernels avoid the CPU-fallback ops that silently tax PyTorch's MPS path for anything outside standard matmuls.