How do you fix "Apple Silicon: RuntimeError: MPS backend out of memory"?

**1. Raise the MPS memory ratio:** ```bash # Allow MPS to use up to 80% of unified memory export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 # Or set an absolute cap (in GB) export PYTORCH_MPS_MEMORY_LIMIT_MB=49152 ``` `HIGH_WATERMARK_RATIO=0.0` removes the soft cap entirely; macOS will swap if needed but the allocation will succeed. **2. Use a smaller / quantized model.** PyTorch's MPS backend doesn't support 4-bit GGUF; for quantized inference on Apple Silicon, use llama.cpp or MLX-LM directly: ```bash brew install llama.cpp llama-cli -m model.Q4_K_M.gguf -p "Hello" # Or MLX pip install mlx-lm mlx_lm.generate --model mlx-community/Llama-3.1-8B-Instruct-4bit --prompt "Hello" ``` **3. Free unified memory** by closing other apps. macOS shows actual GPU memory usage in Activity Monitor → GPU tab. **4. Restart the Python process** after a previous large allocation — MPS allocator pools don't always release back to the OS until process exit.

Metal / Apple Silicon

Verified by owner

Apple Silicon: RuntimeError: MPS backend out of memory

RuntimeError: MPS backend out of memory (MPS allocated: ... GB, other allocations: ... GB, max allowed: ... GB)

By Eruo Fredoline · Last verified Oct 7, 2026

Cause

macOS imposes a per-process limit on how much unified memory can be allocated to the GPU — by default ~67% of total system RAM. Jobs that try to allocate more (e.g. loading a large model into MPS) hit this limit even though there's free RAM available.

A separate cause: the MPS allocator pools memory and can fragment, OOMing on a 10 GB allocation while the system has 30 GB free.

Solution

1. Raise the MPS memory ratio:

# Allow MPS to use up to 80% of unified memory
export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0
# Or set an absolute cap (in GB)
export PYTORCH_MPS_MEMORY_LIMIT_MB=49152

HIGH_WATERMARK_RATIO=0.0 removes the soft cap entirely; macOS will swap if needed but the allocation will succeed.

2. Use a smaller / quantized model. PyTorch's MPS backend doesn't support 4-bit GGUF; for quantized inference on Apple Silicon, use llama.cpp or MLX-LM directly:

brew install llama.cpp
llama-cli -m model.Q4_K_M.gguf -p "Hello"

# Or MLX
pip install mlx-lm
mlx_lm.generate --model mlx-community/Llama-3.1-8B-Instruct-4bit --prompt "Hello"

3. Free unified memory by closing other apps. macOS shows actual GPU memory usage in Activity Monitor → GPU tab.

4. Restart the Python process after a previous large allocation — MPS allocator pools don't always release back to the OS until process exit.

Cloud GPU rental

Model too big for your Mac's memory? Rent a GPU by the hour

If the model does not fit in your Mac's unified memory even with a smaller quant and a shorter context, you can rent a GPU with more memory by the hour. Rented GPUs run CUDA or ROCm, not Metal or MLX, so you would run the model there with llama.cpp, Ollama or vLLM.

RunPod

billed per GPU-hour

Hourly GPU pods, in community-cloud and secure-cloud tiers.

The two tiers run on different hosts and terms. Check which tier a pod is on before a long job.

Rent on RunPod

Vast.ai

billed per GPU-hour

Marketplace of host-listed GPUs; price and reliability vary by host.

Host quality varies. Filter for verified hosts and check the host's reliability before a long run.

Rent on Vast.ai

Referral links: RunLocalAI may earn a commission or account credits if you sign up and rent, at no extra cost to you. Hourly rates and which GPUs are listed change all the time, so check the provider's page before you start. Editorial opinions are independent of the referral relationship; see how we make money.

Related errors

Did this fix it?

If your case was different, email Contact support with what you saw and we'll update the page. If it worked but took different commands on your platform, we want to know that too.