ROCm for AMD GPUs — Local AI on Linux (Chapter 4)

ROCm (Radeon Open Compute) is AMD's open-source GPU compute stack. It runs on Linux only. The install process differs significantly from NVIDIA because ROCm exposes the GPU through /dev/kfd (Kernel FD) and the user-space runtime uses HSA Runtime.

Check your GPU's supported OS and ROCm version before installing. The AMD GPU list at rocm.docs.amd.com shows which architectures are supported by which ROCm versions. For example, an RX 7900 XTX (gfx1100) requires ROCm 6.1 or later. An RX 6800 XT (gfx1030) works with ROCm 5.x and 6.x.

Install ROCm on Ubuntu 22.04:

sudo apt install rocm-runtime rocm-hip-runtime rocminfo
sudo usermod -aG video $USER
sudo usermod -aG render $USER
reboot

Verify after reboot:

/opt/rocm/bin/rocminfo
# Expected snippet:
# ROCm Platform Agent
#   Name:                    gfx1030
#   Marketing Name:          AMD Radeon RX 6800 XT
#   Vendor Name:             AMD
#   Type:                    GPU
#  supports: HSA, KFD

Check compute devices:

/opt/rocm/opencl/bin/clinfo
# Expected: Number of platforms: 1
#   Platform Name: AMD Accelerated Parallel Processing
#   Number of devices: 1

Failure mode: rocminfo returns No devices found but the driver loaded. The user was not added to the video and render groups before running the check. Re-run sudo usermod -aG video $USER && sudo usermod -aG render $USER and log out/in.

Failure mode: rocminfo fails with hsa api call failure [-1] and Could not load amdhsak64.so. The ROCm user-space libraries are not in the linker path. Add export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH to your shell profile.

Failure mode: ROCm 6.1 installed but llama.cpp only has HIP support for gfx1030 compiled in. You compiled llama.cpp expecting HIP backend support and hipgetopt was not found. Check rocminfo for your agent name and confirm it matches the architecture llama.cpp was compiled for. Not all ROCm versions support all architectures.