Apple Silicon: M2 / M3 / M4 local AI
For: Mac owners on M2, M3, or M4 silicon with 16GB or more unified memory. By the end: A 14-32B-class model running on your Mac at usable speed, with the right runtime + quant for your chip and memory size.
Apple Silicon is genuinely good at local AI inference and genuinely different from running on NVIDIA. Unified memory means a 64GB M-series chip behaves more like a "64GB GPU" than a "16GB GPU + 48GB RAM" — but memory bandwidth, not capacity, becomes the constraint at large model sizes. This path moves you from a clean Ollama install to MLX-LM as the production runtime, with realistic expectations per chip tier.
Every chip figure on this page is an auditable spec from our hardware database; throughput claims are bandwidth-derived heuristics, not benchmarks — measured tokens-per-second lives on the linked benchmark pages and our methodology.
Pick by bandwidth, not by name
Apple Silicon for local AI, by memory bandwidth
| Chip | Bandwidth | Unified memory | Model ceiling (~mem − 8 GB) | Year |
|---|---|---|---|---|
| Apple M4 (base) | 120 GB/s | 16 GB | ~8 GB · 7B–14B | 2024 |
| Apple M4 Pro | 273 GB/s | 48 GB | ~40 GB · 32B–70B (4-bit) | 2024 |
| Apple M1 Max | 400 GB/s | 32 GB | ~24 GB · up to 32B | 2021 |
| Apple M2 Max | 400 GB/s | 64 GB | ~56 GB · 70B (4-bit) | 2023 |
| Apple M3 Max | 400 GB/s | 96 GB | ~88 GB · 70B comfortably | 2023 |
| Apple M4 Max | 546 GB/s | 128 GB | ~120 GB · 70B+ / multi-model | 2024 |
| Apple M1 Ultra | 800 GB/s | 128 GB | ~120 GB · 70B+ / multi-model | 2022 |
| Apple M2 Ultra | 800 GB/s | 192 GB | ~184 GB · 100B+ / 70B FP16 | 2023 |
| Apple M3 Ultra | 800 GB/s | 192 GB | ~184 GB · 100B+ / 70B FP16 | 2025 |
| Apple M4 Ultra | 1100 GB/s | 256 GB | ~248 GB · 100B+ | 2025 |
Source: RunLocalAI hardware database. Bandwidth sets speed; unified memory sets the model ceiling. iPhone/iPad-class chips (8 GB) are excluded — at (8 − 8) GB usable they don't run serious local models. Measured tok/s per model: see /benchmarks.
Honest picks · what we'd check
Which Mac should you buy for local AI
Four picks by budget and goal. Each links to the full hardware verdict and an Amazon search. The honest part is the "not for" line — telling you when not to spend is the whole point.
Cheapest viable entry
Mac Mini M4
120 GB/s · 16 GB · ~8 GB ceiling
MSRP ~$599
The cheapest way into real local AI. Runs 7B-class daily drivers and a tight 14B Q4 at usable speed on the Metal backend.
Not for 32B-class and up — for that, step up to the M4 Pro.
Portable daily driver
MacBook Air M4
120 GB/s · 16 GB · ~8 GB ceiling
MSRP ~$999
Same chip class as the Mini in a fanless laptop. Ideal if you need local inference on the move for 7B–14B work.
Fanless: it throttles on sustained generation. For long sessions the Mini or Studio hold clocks better.
Value sweet spot · 32B-class
Mac Mini M4 Pro
273 GB/s · 48 GB · ~40 GB ceiling
MSRP ~$1,399
The pick for most people. 273 GB/s and 48 GB clear 32B-class comfortably and fit 70B in 4-bit — at a desktop price.
Skip it only if you truly need 70B at FP16 or several large models resident at once.
No-compromise · 70B+
Mac Studio M3 Ultra
800 GB/s · 192 GB · ~184 GB ceiling
MSRP ~$4,999
800 GB/s and up to 192 GB (512 GB top spec) run 70B-class at FP16 and 100B+ in 4-bit — the only single-box consumer path there.
Overkill below a confirmed >32B need. The M4 Pro covers everyone else for a quarter of the price.
Already own an M-series Mac with 16 GB+? Start with Milestone 2 on what you have before spending a cent. A used M2 Max (400 GB/s, 64 GB) has the same memory bandwidth as a new M3 Max and runs the same 32B–70B models, often at well under half the price — check Apple Certified Refurbished. Want more bandwidth without the Ultra's price? The Mac Studio M4 Max (546 GB/s, 64 GB, ~$1,999) sits between the M4 Pro and the Ultra. For the pure shopping question, see our best Mac for local AI guide.
Some links above are affiliate links. We may earn a commission at no extra cost to you. How we make money.
Identify your chip and memory honestly
Base M4 runs at 120 GB/s. M4 Pro is 273 GB/s. The Max tier is 400 GB/s (M1 / M2 / M3 Max) up to 546 GB/s (M4 Max). Ultra is 800 GB/s (M1 / M2 / M3 Ultra), up to 1,100 GB/s on the M4 Ultra. Tokens per second on a memory-bound LLM scales roughly with bandwidth, so an M4 Max (546 GB/s) at 64GB can outpace an M3 Max (400 GB/s) at 96GB on a model that fits both. Memory size sets your model ceiling; bandwidth sets your speed.
Heuristic: a chip with X GB of unified memory can comfortably load models up to about (X − 8)GB. The OS needs the rest.
Two phases matter. Prompt processing (prefill) is compute-bound — Apple Silicon's relative weakness, so expect slower time-to-first-token on long prompts, RAG, and agent loops than a discrete NVIDIA card. Generation (decode) is bandwidth-bound — Apple's strength, and what the GB/s figures above predict.
Install Ollama with the Metal backend
Ollama on macOS auto-uses Metal — no driver hunting, no CUDA install, no kernel pin. The starter installation is literally one binary. Pull a 7B Q4 model, run "ollama run", check that it generates faster than 15 tok/s. If it doesn't, you have a thermal or background-process problem, not a configuration one.
One caveat: the fanless MacBook Air M4 will throttle on sustained generation, where the actively-cooled Mac Mini and Mac Studio hold their clocks. That's expectation-setting, not a defect.
Move up to a 14B-32B model
This is the milestone where Apple Silicon starts to look different from a 4090. A 32B model in Q4 is ~18GB; an M2/M3/M4 Pro with 24GB+ unified memory can comfortably run it. A 4090 cannot. This is the "unified memory makes big models accessible" advantage in practice.
Throughput will be lower than a 4090 on the same model. That's fine. The relevant question is whether the model fits and runs at usable speed for your tasks.
Switch to MLX-LM for the production path
MLX is Apple's own ML framework. MLX-LM is the LLM- specific runtime built on top. Same model in MLX-4bit (or MLX-8bit) is typically 10-30% faster than the GGUF equivalent through Ollama, and uses Apple's compute framework rather than llama.cpp's. This is the upgrade from "easy mode" to "the best inference Apple Silicon can give you today."
Convert or download MLX-quantized weights. The model community on Hugging Face has MLX variants for most popular models; if not, MLX includes a one-line conversion tool from the original safetensors.
Pick the right MLX quant for your chip
MLX uses its own quant format (mlx-4bit, mlx-8bit). The heuristic is the same as GGUF: 4-bit is default, 8-bit for accuracy-critical workloads, FP16 for development (compare-vs-ground-truth) work. Don't run FP16 in production on an M-series; the bandwidth math doesn't favor it.
Front the local server with a real client
LM Studio is a Mac-native app and the easiest path. Open WebUI runs in a Docker container and gives you the ChatGPT-style web UI. Either one points at the mlx-lm OpenAI-compatible server. Pick one.
Decide if you want to push to the chip's limit
If you have an M2/M3/M4 Ultra with 128GB+ unified memory, 70B-class models in MLX-4bit are practical. The Apple Silicon AI stack page documents the recipe end-to-end. For multi-Mac setups (rare, but possible), the multi- machine cluster stack walks layer-sharding via Exo over Thunderbolt.
The Mac Studio M3 Ultra configures up to 512GB of unified memory (~504GB usable by the (X − 8) rule) — enough to hold very large models, or several models resident at once, that no single consumer NVIDIA card can fit.
For everyone else: stop at 32B-class. The Mac is a beautiful daily driver for that size; pushing past it costs more in waiting than it gives in capability.
Frequently asked questions
How much unified memory do I need to run local AI on a Mac?
Use the rule of thumb that a Mac with X GB of unified memory comfortably loads models up to about (X−8) GB — the OS needs the rest. So 16 GB runs 7B-class models (and a tight 14B Q4); 48 GB clears 32B-class (a 32B Q4 is ~18 GB); 128 GB+ reaches 70B-class in MLX-4bit. See the M4 Pro for the value tier. the M4 Pro.
Is the base M4 chip fast enough for local LLMs?
Yes for 7B–14B-class work. The base M4 runs at 120 GB/s memory bandwidth with 16 GB of unified memory, and decode speed on a memory-bound model scales with bandwidth. It handles a daily-driver 7B and a tight 14B Q4 well, but 32B-class is better suited to the M4 Pro and up. See the Mac Mini M4. the Mac Mini M4.
M4 Pro vs M4 Max vs Ultra for local AI — what's the real difference?
Bandwidth and memory ceiling. The M4 Pro is 273 GB/s, the M4 Max 546 GB/s, the M3 Ultra 800 GB/s, and the M4 Ultra 1,100 GB/s. Generation speed on a memory-bound model scales roughly with bandwidth; unified memory size sets the largest model you can load. See the M4 Max. the M4 Max.
Should I use Ollama or MLX on a Mac?
Start with Ollama — it auto-uses the Metal backend with no driver setup. Move to MLX-LM for the production path: the same model in MLX-4bit is typically 10–30% faster than the GGUF equivalent through Ollama, because it uses Apple's own compute framework. Measure on your own chip. See the MLX-LM review. the MLX-LM review.
Can a Mac run a 70B model locally?
Yes, on an Ultra-class Mac with 128 GB+ unified memory, running 70B in MLX-4bit (~40 GB of weights). The Mac Studio M3 Ultra reaches 192 GB (and configures up to 512 GB), giving real headroom for 70B-plus or several resident models. See the Mac Studio M3 Ultra. the Mac Studio M3 Ultra.
Is Apple Silicon as fast as an RTX 4090 for local AI?
Not on raw tokens-per-second — a 4090 generates faster on a model that fits its 24 GB. But unified memory lets a Mac load models the 4090 cannot fit at all (a 32B Q4 is ~18 GB and runs comfortably on 24 GB+). The honest tradeoff is throughput vs capacity. See milestone 3 below. milestone 3 below.
Next recommended step
The reference recipe: M-series + MLX-LM + Open WebUI + agent layer. Setup commands, expected outcome, and failure modes.