INFOMODEL RELEASE·2026-10-20
IBM Granite Switch 4.1 8B and 30B: chat, decision and safety adapters
▼ WHAT HAPPENED
IBM released Granite Switch 4.1 in 8B and 30B sizes. Both model cards give a release date of October 14, 2026. IBM's Hugging Face blog post is dated October 15, 2026.
Each model is a dense Granite 4.1 base (8B or 30B) with two activated LoRA (aLoRA) adapters, switched on by control tokens in the chat template. `decision` returns a softmax probability distribution over option labels; the blog names `v1/systemone` as an example use. `guardian-core` detects safety risks. Speech-to-text uses `granite-speech-5.0-470m-turboctc` (470M parameters, English). IBM calls it embedded, but neither checkpoint holds speech weights; the card describes an ASR cascade. It lists limits: English only, lowercase unpunctuated transcripts, and no joint training with the LLM.
The cards list a 131,072-token context and an Apache 2.0 licence. The Hub shows the 8B as "9B"; its safetensors total about 8.78 billion parameters. As of October 20, 2026, IBM publishes BF16 safetensors only: about 17.6 GB for the 8B and 60 GB for the 30B. The ibm-granite organisation has no GGUF or quantized builds of Granite Switch.
The cards name Hugging Face Transformers and vLLM as backends. The blog says the model runs in vLLM with a plug-in and has native inference support in llama.cpp. The cards do not list llama.cpp, but llama.cpp's own repo shows support. It merged the Granite Switch architecture on August 10, 2026 (PR #25107). Release b11700 on October 16 added decision support (PR #30676), so llama-server can serve granite-switch-4.1-8b for chat and on `/v1/systemone`. The PR author converted the 8B with `convert_hf_to_gguf.py` and ran it in BF16 and Q4_K_M. The PR does not mention the 30B or speech input.
▼ OPERATOR ANGLE
Treat Granite Switch as a vLLM or Transformers model first. The vLLM route needs the Granite Switch plug-in, so install it from the Granite Switch GitHub repo and pin the vLLM and Transformers versions the card lists. llama.cpp is now an option for chat and decisions: convert IBM's safetensors yourself with a build at b11700 or later, and try the 8B before you rely on the 30B. Prefer your own conversion to third-party GGUFs. Do not assume Ollama or LM Studio can load it until their release notes name Granite Switch. For speech-to-text, stay on the vLLM path with the audio extra, and leave room for the separate Granite Speech model, since it is not inside the checkpoint. Check the file sizes against your GPU memory before downloading. We have not benchmarked either size.
▼ ENTITIES REFERENCED
[pulse item] · runlocalai.co/pulse/ibm-granite-switch-4-1-8b-30b-decision-guardian-adapters