MLX (Apple)

MLX is Apple's open-source array framework optimized for Apple Silicon. The Apple equivalent of PyTorch + CUDA, with first-party Metal kernels and Apple Neural Engine integration. Two key surfaces: MLX-LM (the Python LLM-inference library) and MLX Swift (the iOS/macOS native bindings used in App Store-shipping apps).

What MLX does well: unified-memory aware model loading (no GPU/CPU copy overhead on Apple Silicon), ANE delegation for compatible ops, native quantization formats (MLX-4bit, MLX-8bit) tuned for Apple Silicon's memory bandwidth. Tok/s on M3 Max / M3 Ultra is competitive with consumer NVIDIA at the same VRAM tier when the workload is bandwidth-bound (most LLM inference is).

What MLX doesn't do: cross-platform deployment (it's Apple-only), CUDA quant formats (no AWQ/GPTQ/EXL2 — convert to MLX format first), full PyTorch ecosystem parity. The model coverage is good but lags Hugging Face mainline by 2-6 weeks for new architectures. For Apple Silicon production deployments, MLX-LM is the operator default; for cross-platform Mac+Linux+Windows deployments, llama.cpp Metal is the more portable fallback.

An operator on an M3 Max 64GB comparing local inference options finds that a 4-bit MLX build of a 34B model, loaded through MLX-LM, delivers noticeably better tokens/sec than the same model run through llama.cpp's Metal backend, because MLX's unified-memory-aware loading avoids a CPU-to-GPU copy step that llama.cpp's more general-purpose path still pays on some operations. They convert the model from Hugging Face safetensors to MLX format with mlx_lm.convert --hf-path ... -q, a one-time step. The trade-off shows up later: a new model architecture drops on Hugging Face and llama.cpp's GGUF conversion script supports it within days, while MLX-LM support lags several weeks — so for bleeding-edge model coverage they keep a GGUF fallback, and use MLX only for unified-memory throughput once a converter exists.