Local AI errors & fixes
60 common errors when running AI locally — with verified causes and solutions. Paste your error message into Google and you should land on the right page.
Solutions tagged
Network / downloads4 entries
Open WebUI: Failed to fetch from /ollama (cannot reach Ollama backend)
Failed to fetch from /ollama / WebUI says "Connection failed: Could not connect to Ollama"
HuggingFace download is extremely slow or stalls
(downloads at 100 KB/s instead of saturating bandwidth)
HuggingFace: 403 Forbidden when downloading a gated model
401 Client Error: Unauthorized for url: https://huggingface.co/...
Ollama can't bind port 11434 — already in use
Error: listen tcp 127.0.0.1:11434: bind: address already in use
CUDA / NVIDIA4 entries
vLLM install picks the wrong CUDA wheel
ImportError: libcudart.so.12: cannot open shared object file (typical when the cu124 wheel of vLLM lands on a cu118-only...
RuntimeError: CUDA error: device-side assert triggered
RuntimeError: CUDA error: device-side assert triggered
CUDA runtime version doesn't match the installed driver
RuntimeError: The detected CUDA version (12.4) mismatches the version that was used to compile PyTorch (12.1). Please ma...
PyTorch: CUDA error: no kernel image is available for execution on the device
RuntimeError: CUDA error: no kernel image is available for execution on the device
Configuration11 entries
Docker: could not select device driver "" with capabilities: [[gpu]]
could not select device driver "" with capabilities: [[gpu]]
Ollama: Error: model 'X' not found
Error: model 'X' not found, try pulling it first
Ollama: bind: address already in use (port 11434)
Error: listen tcp 127.0.0.1:11434: bind: address already in use
Ollama: connection refused on localhost:11434
Error: connect ECONNREFUSED 127.0.0.1:11434
Token generation slows as conversation gets longer
(no error — tok/s drops from 50 to 5 as context fills)
Ollama truncates input — default context length is only 2048
(no error — long inputs get silently truncated)
Slow tokens/sec on capable GPU (silent CPU fallback)
(no error — output is correct but tok/s is 5-10× slower than expected)
Ollama: listen tcp 127.0.0.1:11434 bind: address already in use
Error: listen tcp 127.0.0.1:11434: bind: address already in use
Very slow first token / OOM only at long prompts
(no error — TTFT goes from 200ms at 2K context to 30+ seconds at 64K context)
LM Studio generation much slower than expected
(no error — tok/s reads e.g. 4 tok/s on hardware that should do 40 tok/s)
Windows DirectML model runs on CPU instead of GPU
(no error — onnxruntime falls back to CPUExecutionProvider despite DirectML wheel installed)
Driver issues9 entries
WSL2: nvidia-smi works but PyTorch sees no CUDA / libcuda.so missing
OSError: libcuda.so.1: cannot open shared object file: No such file or directory
PyTorch CUDA error: driver version is insufficient for CUDA runtime
RuntimeError: CUDA error: CUDA driver version is insufficient for CUDA runtime version
WSL2: torch.cuda.is_available() returns False
torch.cuda.is_available() == False and "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver" in...
NCCL error: peer to peer not supported (multi-GPU)
NCCL error: unhandled system error / peer to peer access not supported between GPU{0} and GPU{1}
CUDA driver version is insufficient for CUDA runtime version
CUDA driver version is insufficient for CUDA runtime version
WSL2 GPU not detected — nvidia-smi missing or empty
Command 'nvidia-smi' not found, or NVIDIA-SMI failed because it couldn't communicate with the NVIDIA driver
nvidia-smi: command not found
nvidia-smi: command not found
Docker container can't see GPU — nvidia-container-toolkit missing
could not select device driver "nvidia" with capabilities: [[gpu]]
Docker: could not select device driver "nvidia"
docker: Error response from daemon: could not select device driver "nvidia" with capabilities: [[gpu]].
Out of memory8 entries
Process killed (OOM killer) when loading large model
Killed
CUDA out of memory when loading a model
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate X.XX GiB
CUDA OOM that only happens at long context (KV cache blowup)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate
vLLM AsyncEngineDeadError after large batch / OOM
AsyncEngineDeadError: Background loop has errored already
Ollama: model requires more system memory than is available
Error: model requires more system memory than is available
SGLang: RadixAttention KV cache overflow / out of memory
RuntimeError: KV cache pool full (RadixAttention) — increase --mem-fraction-static or reduce --max-running-requests
Out of memory specifically at long context lengths
torch.cuda.OutOfMemoryError or 'cannot allocate KV cache' at >32K tokens
vLLM: No available KV cache blocks
RuntimeError: No available KV cache blocks
ROCm / AMD4 entries
ROCm: HIP error: invalid device — no GPU detected
HIP error: invalid device function / hipErrorNoDevice
ROCm: HIP error: invalid device function
HIP error: invalid device function
ROCm: hipErrorInvalidDeviceFunction on RX 7000-series
HIP error: invalid device function / hipErrorInvalidDeviceFunction (typical wording when HSA_OVERRIDE_GFX_VERSION is uns...
ROCm: HSA_STATUS_ERROR_INVALID_DEVICE — GPU not detected
HSA_STATUS_ERROR_INVALID_DEVICE or rocminfo shows no agents
Model format / GGUF3 entries
llama.cpp: error loading model — bad magic / unsupported GGUF
llama_model_load: error loading model: failed to load model 'X': bad magic / unsupported GGUF version
llama.cpp: failed to mmap GGUF file
llama_model_load: error loading model: failed to open ... or mmap
Failed to load model: GGUF version mismatch
llama_model_load: error loading model: this GGUF file is version X but llama.cpp supports up to version Y
Tokenizer mismatches6 entries
GGUF model outputs garbage — tokenizer / chat-template mismatch
(no error — generation is fluent gibberish, repeats one token, or emits raw special tokens like <|im_start|>)
Model produces gibberish or repeats one token forever
(no error — output is garbled like 'the the the' or random unicode)
Quantized model produces garbage / never stops generating
(no error — output is incoherent, repeats, or generates until max tokens)
Model loaded but tokenizer vocab size mismatch
Vocab size mismatch: model has X tokens, tokenizer has Y
TypeError: 'NoneType' object is not subscriptable in tokenizer
TypeError: 'NoneType' object is not subscriptable
OSError: Can't load tokenizer for ... / no file named tokenizer.json
OSError: Can't load tokenizer for '...'. If you were trying to load it from 'https://huggingface.co/models'
Build / compile failures5 entries
llama.cpp build fails: nvcc not found / CUDA toolkit missing
make: nvcc: No such file or directory
llama.cpp build fails: nvcc not found
GGML_USE_CUDA defined but nvcc not found in PATH
llama.cpp CUDA build: unsupported GNU version! gcc versions later than X are not supported
error: unsupported GNU version! gcc versions later than 13 are not supported
exllamav2 ImportError: cannot import name 'ExLlamaV2' / undefined symbol
ImportError: cannot import name 'ExLlamaV2' from 'exllamav2'
flash-attn install fails on Windows / no precompiled wheel
ERROR: Could not build wheels for flash-attn
Metal / Apple Silicon5 entries
Apple Silicon: RuntimeError: MPS backend out of memory
RuntimeError: MPS backend out of memory (MPS allocated: ... GB, other allocations: ... GB, max allowed: ... GB)
Metal Allocator: out of memory on Apple Silicon
[METAL] Metal Allocator: out of memory (Allocation size X exceeds available)
MLX / Metal: command buffer execution failed
[MLX][ERROR] Metal command buffer execution failed
Metal allocation failed — Apple Silicon OOM under unified memory pressure
metal::MetalCommandQueue allocation failed or [MPS] OOM
MLX: Memory pressure detected — consider reducing batch size
Warning: Memory pressure detected. Consider reducing the batch size.
Quantization issues1 entry
Hit an error we don't have?
We add ~5 new errors per month based on what readers report.
Email Contact support with the literal error message and what you tried. If it's a common one we'll write it up; if it's something only you hit, we'll often help directly.