Nemotron 3.5 Bolt 74B-A9B 24GB: 2-bit VLM; NVIDIA GGUF recipes target 32GB+

NVIDIA released Nemotron 3.5 Bolt 74B-A9B 24GB on Hugging Face on October 15, 2026. It is an open-weight vision-language model with 74B total parameters and 9B active. The model card describes a hybrid of Mamba-2, attention and mixture-of-experts layers. Input is text, images and video; output is text. Context goes up to 256K tokens. The license is the OpenMDW License Agreement v1.1. NVIDIA says the model fits in 24GB through native 2-bit quantization-aware training. All routed and shared experts are IQ2-XXS, at 2.0625 bits per parameter. Mamba layers stay in NVFP4, and the embedding and output layers in Q8_0. NVIDIA publishes its own GGUF. On the Hub, the main file is 22.02 GB and the `mmproj` vision projector is 1.73 GB. The GGUF card names llama.cpp as the runtime and says to build the latest version from source. Its deployment table has two recipes and no 24GB one. The 28GB recipe uses 65,536 tokens of context and a q8_0 KV cache, for RTX Spark 64GB or RTX discrete GPUs with 32GB or more. The 32GB recipe uses 262,144 tokens, for RTX Spark 128GB or RTX discrete GPUs with 48GB or more. Both recipes also load a separate LiLiCorr drafter GGUF (998 MB) for speculative decoding. The card lists NVIDIA Ada Lovelace and Blackwell as supported GPU architectures, on Windows and Linux. NVIDIA's blog lists support in llama.cpp, Ollama, LM Studio, Unsloth and vLLM. The base (non-GGUF) card documents vLLM. The blog also says a smaller Bolt E4B 4GB model is coming October 23.