ROCm: hipErrorInvalidDeviceFunction on RX 7000-series

Environment: AMD RDNA3 cards (RX 7900 XTX, RX 7800 XT, W7900) on Linux with ROCm 6.x running vLLM, PyTorch ROCm, or llama.cpp HIP backend.

Severity: high — kernels fail to launch on this device.

• The pre-built kernel is pinned to a different gfx target (e.g. compiled for gfx1030 / RDNA2 but you're on gfx1100)
• ROCm version is older than the GPU (RX 7900 XTX requires ROCm 5.7+; full support arrives in 6.x)
• Mixed PyTorch ROCm wheel + system ROCm drift
• Multi-GPU host with a mix of RDNA2 + RDNA3 cards and ROCR_VISIBLE_DEVICES not set

1. Set HSA_OVERRIDE_GFX_VERSION to make ROCm dispatch RDNA2 kernels onto your RDNA3 card (works because RDNA3 ISA is mostly backward-compat for compute):

# RX 7900 XTX / 7800 XT (gfx1100)
export HSA_OVERRIDE_GFX_VERSION=11.0.0
# RX 6000-series (gfx1030)
export HSA_OVERRIDE_GFX_VERSION=10.3.0

Add to ~/.bashrc or your systemd service env-file.

2. Upgrade ROCm (preferred over override on RDNA3):

sudo amdgpu-install --uninstall
sudo amdgpu-install -y --usecase=rocm --rocmrelease=6.3

3. Reinstall PyTorch with matching ROCm wheel:

pip install --force-reinstall torch \
  --index-url https://download.pytorch.org/whl/rocm6.3

4. Pin the visible device if you have mixed-gen GPUs:

export ROCR_VISIBLE_DEVICES=0
export HIP_VISIBLE_DEVICES=0

5. Verify:

python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"

If you're on RDNA1 (RX 5000) or Polaris (RX 580), ROCm 6 dropped support. Use the Vulkan backend in llama.cpp instead — pre-built binaries with -DGGML_VULKAN=ON work on every GPU with a Vulkan 1.2 driver.