← /pulse/vllm-0-24-device-ids-breaking
WARNINGRUNTIME UPDATE·2026-07-10

vLLM 0.24.0 breaking change: CUDA_VISIBLE_DEVICES no longer set

▼ WHAT HAPPENED

vLLM v0.24.0 (2026-06-29) stops setting CUDA_VISIBLE_DEVICES internally. GPU selection moves to a new device_ids argument, with the old ROCm behavior on a deprecation window rather than removed outright. Existing multi-GPU launch configs that relied on vLLM managing device visibility can silently break - the process may grab the wrong GPUs, oversubscribe a card that another service is using, or fail to start with no obvious error pointing at the cause. The release also adds MiniMax-M3 and DiffusionGemma support and makes Model Runner V2 handle quantized models by default, which changes the execution path for most Q4/Q8 deployments even if you never touch the new flag.

▼ OPERATOR ANGLE

Before upgrading a multi-GPU vLLM deployment to 0.24.0, audit every launch script and systemd unit for CUDA_VISIBLE_DEVICES assumptions and migrate to device_ids. ROCm users: the old path still works but is deprecated - migrate now rather than during the removal release. Pin 0.23.x until you have tested the new argument on your exact topology.

▼ ENTITIES REFERENCED

[pulse item] · runlocalai.co/pulse/vllm-0-24-device-ids-breaking