Granite 4.1 3B Instruct
Granite 4.1 3B Instruct is the smallest of IBM's Granite 4.1 language models, released April 29, 2026 under Apache 2.0. It is a dense, decoder-only transformer — 3.4B parameters, 40 layers, grouped-query attention, RoPE, SwiGLU, tied embeddings — with a 131,072-token (128K) context window. IBM pretrained the family on roughly 15T tokens in a five-phase pipeline, then post-trained with supervised fine-tuning on ~4.1M curated samples and on-policy GRPO reinforcement learning targeting tool calling and instruction following. Model-card scores: 67.02 MMLU, 82.30 IFEval, 86.88 GSM8K, 81.71 HumanEval, 60.80 BFCL v3. It supports twelve languages, and IBM lists optimization for llama.cpp, vLLM, and SGLang at launch — runtimes IBM says enable flexible deployment across cloud and local environments — with availability on Ollama, LM Studio, OpenRouter, and Unsloth. As the family's smallest model, it is the natural pick when local memory is tight.
Positioning
This is the Granite you deploy on hardware that can't take the 8B: fanless mini PCs, older laptops, or as a sidecar model inside a larger application. IBM's post-training gives it unusually strong instruction discipline for the size — 82.30 IFEval and 60.80 BFCL v3 are genuinely good 3B numbers — so it earns its keep as a local function-calling and RAG-extraction engine.
Know what you're giving up. ArenaHard collapses to 37.80 versus 68.98 on the 8B, and a SimpleQA score of 3.68 means it knows almost nothing off the top of its head — treat it as a text processor that must be fed context, never as a knowledge oracle. IBM's own RULER numbers for the base model also fall to 58.0 at 128K, so keep real-world contexts well short of the maximum. Note too that the family's 512K long-context training stage skipped the 3B entirely.
Against Qwen3 4B and Gemma 3 4B, Granite's case is the clean Apache 2.0 license, the 128K window, and enterprise-flavored tool training; Qwen3 will beat it on math and reasoning-heavy work. Run it for constrained pipelines — classification, extraction, structured tool calls — at the edge. If you can spare the memory for the 8B, spend it: that model is a class apart for barely more RAM.
Overview
Granite 4.1 3B Instruct is the smallest of IBM's Granite 4.1 language models, released April 29, 2026 under Apache 2.0. It is a dense, decoder-only transformer — 3.4B parameters, 40 layers, grouped-query attention, RoPE, SwiGLU, tied embeddings — with a 131,072-token (128K) context window. IBM pretrained the family on roughly 15T tokens in a five-phase pipeline, then post-trained with supervised fine-tuning on ~4.1M curated samples and on-policy GRPO reinforcement learning targeting tool calling and instruction following. Model-card scores: 67.02 MMLU, 82.30 IFEval, 86.88 GSM8K, 81.71 HumanEval, 60.80 BFCL v3. It supports twelve languages, and IBM lists optimization for llama.cpp, vLLM, and SGLang at launch — runtimes IBM says enable flexible deployment across cloud and local environments — with availability on Ollama, LM Studio, OpenRouter, and Unsloth. As the family's smallest model, it is the natural pick when local memory is tight.
Family & lineage
How this model relates to others in its lineage. Family members share architecture and training-data roots; parent / children edges record direct distillation or fine-tune relationships.
Strengths
Weaknesses
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|
Get the model
HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of Granite 4.1 3B Instruct.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
Can I use Granite 4.1 3B Instruct commercially?
What's the context length of Granite 4.1 3B Instruct?
Source: huggingface.co/ibm-granite/granite-4.1-3b
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify Granite 4.1 3B Instruct runs on your specific hardware before committing money.