08. Mac Model Selection Guide
Model selection on Apple Silicon is not just about capability—it is about fitting within your RAM and achieving acceptable throughput. The two variables are model size (parameters × quantization) and hardware (chip generation and RAM).
A practical selection table for Apple Silicon:
| Model | Parameters | Quantization | RAM Needed | Target Chip |
|---|---|---|---|---|
| Phi-3.5-mini | 3.8B | Q4_K_M | 2.4 GB | M1+ |
| Llama 3.2 1B | 1B | Q4_K_M | 0.7 GB | M1+ |
| Llama 3.2 3B | 3B | Q4_K_M | 1.9 GB | M1+ |
| Llama 3.2 7B | 7B | Q4_K_M | 4.9 GB | M1 Pro+ |
| Mistral 7B | 7B | Q4_K_M | 4.9 GB | M1 Pro+ |
| Llama 3.1 8B | 8B | Q4_K_M | 5.4 GB | M2 Pro+ |
| Qwen 2.5 14B | 14B | Q4_K_M | 8.8 GB | M2 Pro 24GB+ |
| Llama 3.1 70B | 70B | Q4_K_M | 43 GB | M2 Ultra 128GB |
The 8 GB machines (M1/M2 base) are viable for 1B–3B models. They can load a 7B model but the memory headroom is too tight for comfortable use at longer context lengths.
The 16 GB machines (M1/M2 Pro base, M3 base) handle 7B models well at short contexts and 3B models at long contexts. This is the sweet spot for most developers.
The 24 GB+ machines (M3 Pro, M3 Max, M4 Pro) run 13B–14B models comfortably and can push into 30B territory with careful quantization selection.
Ollama model search:
# Search available models
ollama list
ollama search llama # requires internet, lists from registry
# Pull the model
ollama pull llama3.2:3b
MLX model selection via the community hub:
# Find model size and recommended hardware
from mlx_lm import ls
models = ls()
# Returns list of model names with sizes
# mlx-community/Llama-3.2-3B-Instruct-4bit ~1.8GB
Identify three models from Ollama that fit in your available RAM using the Q4_K_M calculation from Chapter 7. Pull the smallest one and run it. Note the time-to-first-token and tokens per second.