Docker: could not select device driver "nvidia"

Environment: Docker / Docker Desktop on Linux or WSL2 trying to run GPU containers (e.g. vLLM, Ollama, tgi).

Severity: high — GPU containers won't start.

• nvidia-container-toolkit not installed
• Toolkit installed but Docker daemon not restarted (config not picked up)
• Docker rootless mode without the toolkit's rootless adjustments
• WSL2 Docker Desktop with Windows host driver missing
• docker run --gpus all used on a Docker version older than 19.03 (the flag didn't exist)

1. Install nvidia-container-toolkit (Ubuntu/Debian):

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update && sudo apt install -y nvidia-container-toolkit

2. Configure Docker runtime + restart daemon:

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

3. Verify GPU is visible inside a container:

docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

4. Rootless Docker users: the toolkit needs an extra step:

nvidia-ctk runtime configure --runtime=docker --config=$HOME/.config/docker/daemon.json
systemctl --user restart docker

5. WSL2 Docker Desktop: install the toolkit inside the WSL distro, not on Windows. The Windows host driver passes through; you only need the Linux-side toolkit + Docker engine.

Caveat: --gpus is a Docker-CLI shorthand. With Docker Compose, switch to the deploy.resources.reservations.devices syntax. With podman, use --device nvidia.com/gpu=all and the CDI workflow instead.