CUDA runtime version doesn't match the installed driver

Environment: NVIDIA Linux/WSL2 hosts running vLLM, PyTorch wheels built against a specific CUDA toolkit, or llama.cpp compiled with flash-attention.

Severity: high — the runtime refuses to start.

The PyTorch / vLLM / flash-attention wheel you installed was compiled against a different CUDA toolkit than the one your driver exposes. Driver and runtime CUDA versions don't have to be identical, but the wheel often hard-checks the toolkit it was built against.

• Mixed installs: pip install vllm pulled the cu121 wheel onto a system whose driver only supports up to CUDA 12.0
• Driver upgraded after PyTorch was installed (now ahead of the wheel)
• flash-attention pre-built wheel for cu124 dropped onto a cu121 environment
• Conda environment leaked a different libcudart.so onto LD_LIBRARY_PATH

1. Pin to your driver's CUDA via the matching wheel index (most reliable):

# Check what your driver supports
nvidia-smi  # right column "CUDA Version" is the MAX
# Reinstall PyTorch with that exact toolkit
pip install --force-reinstall torch torchvision \
  --index-url https://download.pytorch.org/whl/cu121

2. Pin via Docker so toolkit + wheel + driver are always consistent:

docker run --gpus all --rm \
  nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04 \
  bash -c "pip install vllm && python -c 'import vllm'"

3. Downgrade flash-attention to a build matching your toolkit:

pip install --force-reinstall \
  flash-attn==2.5.8 --no-build-isolation

4. Or upgrade the NVIDIA driver (preferred when no app-side pin exists):

sudo apt install nvidia-driver-550  # or latest stable
sudo reboot

5. Verify the resolved versions match:

python -c "import torch; print(torch.version.cuda, torch.cuda.is_available())"

If on Windows + WSL2, never install Linux NVIDIA drivers inside WSL — only update the Windows host driver. The WSL2 distro inherits the host's libcuda.so.1.