01. macOS AI Landscape

macOS runs local AI through three distinct stacks, and knowing which one you are using matters more than people admit. The first stack is Metal, Apple's GPU framework. Any model that uses Apple's native ML tools routes through Metal for GPU acceleration. The second stack is CUDA-style compute via llama.cpp compiled for ARM64, which runs on the CPU but can dispatch to Metal for some operations. The third stack is higher-level wrappers like Ollama and LM Studio that abstract the underlying runtime.

Most users encounter the landscape through Ollama because it has the simplest installation story: brew install ollama. Behind that single command, Ollama downloads a quantized model, selects a runtime (llama.cpp by default), and starts an API server on port 11434. That works fine until you try to run a 7B model on an M1 with 8 GB of unified memory and wonder why the fan spins up and the response time is 40 tokens per second. The answer lives in the architecture, not the settings.

The tools are improving fast. As of early 2026, Ollama has native Metal support, MLX models run 2–4× faster on Apple Silicon than equivalent llama.cpp builds, and LM Studio provides a GUI that makes model management less error-prone for teams that do not live in the terminal. The rest of this course maps the full terrain.

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.