INFORUNTIME UPDATE·2026-10-20
ExLlamaV3 1.6.0-1.6.3: preliminary ROCm, faster CPU offload, less VRAM overhead
▼ WHAT HAPPENED
ExLlamaV3 (turboderp-org/exllamav3) shipped five releases between October 3 and October 17, 2026.
v1.6.0 (Oct 7) adds preliminary ROCm support, limited to ROCm 10 and gfx1100+ GPUs. No earlier ExLlamaV3 release notes mention ROCm. The same release lists faster offloading on AVX2 CPUs, smarter default CPU core allocation and pinning, reduced system memory overhead, and fixed vision towers for GLM4 and Qwen3-VL.
v1.6.1 (Oct 10) brings a reduced, flat VRAM overhead for GDN/KDA/Mamba drafting, support for recurrent draft models, and a new hybrid drafting mode (draft model plus long n-gram). DFlash switches to sliding attention to cut VRAM overhead. The notes also list a workaround for a Triton issue that broke kernel autotune on mixed-architecture setups, Turing (sm_75) optimizations, AVX2 optimizations, and many ROCm fixes and optimizations.
v1.6.2 (Oct 14) adds faster CPU offloading, support for TP models with DFlash2 drafters, less VRAM overhead for GDN/KDA/MLA layers, and ROCm fixes with faster attention on gfx11*.
v1.6.3 (Oct 17) adds support for BailingMoeV3ForCausalLM (Ling-3.0, 3.1).
Just before the 1.6 series, v1.5.4 (Oct 3) added options for token-embedding quantization and disk streaming, and updated the Hugging Face Transformers integration module to Transformers 5.
The release notes give no speed or memory figures for any of these changes. We have not benchmarked these releases.
▼ OPERATOR ANGLE
Treat the ROCm path as experimental. The notes call it preliminary, and v1.6.1 and v1.6.2 already carried more ROCm fixes. On an AMD card, first confirm you are on ROCm 10 and that your GPU reports a gfx1100 or newer target, for example with `rocminfo`. Keep a working llama.cpp setup as a fallback. If you use GDN/KDA/Mamba or DFlash drafting close to your VRAM limit, or rely on CPU offloading, update to v1.6.2 or later. Measure memory use and tokens/s on your own models before and after, since the notes give no figures. Turing (sm_75) users may also want v1.6.1 or later. Upgrading from v1.5.3 or earlier also brings the Transformers 5 integration, so if you use that path, test in a separate environment and pin versions.
▼ ENTITIES REFERENCED
[pulse item] · runlocalai.co/pulse/exllamav3-1-6-preliminary-rocm-faster-cpu-offload-lower-vram-overhead