How to verify CUDA setup for AI workloads
CUDA drivers installed, NVIDIA GPU present on the system
What this does
Confirms that the CUDA driver and runtime are correctly installed and that a GPU is available for compute workloads. This is the primary gate before running any local AI model that requires GPU acceleration.
Steps
Query the GPU with nvidia-smi. This talks directly to the kernel driver and reports GPU model, memory usage, and driver version.
nvidia-smiExpected output: Table listing GPU with name, memory, driver version, and CUDA version.
Check the nvcc compiler version.
nvcc --versionExpected output:
Cuda compilation tools release 12.xwith build date.Run a minimal CUDA compute test.
python3 -c "import torch; print('CUDA available:', torch.cuda.is_available()); print('Device:', torch.cuda.get_device_name(0))"Expected output:
CUDA available: Truefollowed by GPU name.
- Record the local run evidence. Save the exact command, runtime or package version, model name if applicable, and observed output so the result can be reproduced later.
Verification
nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv
# Expected: header row followed by GPU name, driver version, total memory
Common failures
nvidia-smi: command not found— The NVIDIA driver is not installed or not in PATH. Reinstall the driver package.nvcc: command not found— CUDA Toolkit not in PATH. Source/etc/profile.d/cuda.shor add/usr/local/cuda/binto PATH.cudaGetDeviceCount returned 0— Kernel module not loaded. Runsudo modprobe nvidia.- Mismatch between driver and runtime version — Downgrade the Toolkit or upgrade the driver.
Operator checkpoint
Before treating this as solved, write down the local runtime, model or package version, hardware/backend if relevant, and the verification output. This keeps the guide useful as a Will-It-Run style decision instead of a one-off command transcript.