Why Linux for AI — Local AI on Linux (Chapter 1)

Linux dominates AI infrastructure because every major ML framework treats it as the primary target. CUDA, ROCm, cuDNN, and TensorRT are developed and tested on Linux first. The Linux kernel exposes GPU devices through /dev/nvidia* and /dev/dri/* in ways that container runtimes can bind-mount or pass through with zero configuration overhead.

The GPU vendor driver stacks integrate directly with the kernel ABI. On Ubuntu, nvidia-driver-535 installs a kernel module that lives at /usr/lib/modules/$(uname -r)/kernel/drivers/video/nvidia.ko. On Fedora, the same package uses DKMS to rebuild the module against the running kernel after every kernel update. If you run uname -r and it returns 6.8.0-45-generic, the driver must match that exact kernel ABI. Mismatches are the leading cause of GPU initialization failures.

Linux gives you nvidia-smi from the command line, /proc/driver/nvidia/ for diagnostics, and nvidia-ctk on newer stacks. On no other OS can you cat /proc/driver/nvidia/gpus/0000:01:00.0/information to retrieve GPU serial numbers programmatically. This matters for multi-GPU racks where you need to identify which card is which from SSH.

Containers are first-class citizens on Linux. docker run --gpus all works because the NVIDIA container runtime hooks into the Docker daemon via --hooks-dir=/usr/share/containers/oci/hooks.d/. On Windows or macOS, this requires a Linux VM under the hood. The performance penalty is real and measurable.

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.