Granite 4.1 8B Instruct
Granite 4.1 8B Instruct is the mid-size model in IBM's Granite 4.1 family: a dense, decoder-only transformer with 8.79B parameters (40 layers, grouped-query attention, RoPE, SwiGLU) and a 131,072-token context window, released April 29, 2026 under Apache 2.0. The family was pretrained on roughly 15T tokens; the 8B additionally went through staged long-context training out to 512K tokens, though the shipped configuration is 128K. Post-training combined supervised fine-tuning on ~4.1M curated samples with on-policy GRPO reinforcement learning (DAPO loss), and IBM reports the result matches or surpasses its previous Granite 4.0-H-Small, a 32B-A9B MoE. Model-card scores: 73.84 MMLU, 87.06 IFEval, 68.98 ArenaHard, 92.49 GSM8K, 68.27 BFCL v3, 85.37 HumanEval. Twelve supported languages, FP8 variants optimized for vLLM, and llama.cpp, vLLM, and SGLang support at launch.
Positioning
The model to take seriously in this family, and the default Granite for local use. An 8B dense model that IBM credibly claims matches or surpasses its own Granite 4.0-H-Small — a 32B-A9B MoE from seven months earlier — is real progress, and the numbers back the enterprise focus: 87.06 IFEval and 68.27 BFCL v3 make it one of the strongest instruction-following and tool-calling models you can run on a single consumer GPU under a clean Apache 2.0 license. Artificial Analysis also flagged the family as notably token-efficient among non-reasoning models, with the 8B standing out — it answers without padding, which matters when you pay in local tokens per second.
Honest caveats: there is no reasoning mode, so Qwen3 8B with thinking enabled will beat it on hard math and multi-step logic, and a SimpleQA of 4.82 means minimal built-in world knowledge — design around retrieval from day one. The chat register is businesslike, not companionable.
Run it for local agents, RAG backends, and structured-output pipelines on 8-12GB of VRAM or Apple Silicon. Skip it for open-ended reasoning or knowledge Q&A without retrieval. At 760K+ Hugging Face downloads in the past month alone, the community verdict is quietly positive.
Overview
Granite 4.1 8B Instruct is the mid-size model in IBM's Granite 4.1 family: a dense, decoder-only transformer with 8.79B parameters (40 layers, grouped-query attention, RoPE, SwiGLU) and a 131,072-token context window, released April 29, 2026 under Apache 2.0. The family was pretrained on roughly 15T tokens; the 8B additionally went through staged long-context training out to 512K tokens, though the shipped configuration is 128K. Post-training combined supervised fine-tuning on ~4.1M curated samples with on-policy GRPO reinforcement learning (DAPO loss), and IBM reports the result matches or surpasses its previous Granite 4.0-H-Small, a 32B-A9B MoE. Model-card scores: 73.84 MMLU, 87.06 IFEval, 68.98 ArenaHard, 92.49 GSM8K, 68.27 BFCL v3, 85.37 HumanEval. Twelve supported languages, FP8 variants optimized for vLLM, and llama.cpp, vLLM, and SGLang support at launch.
Family & lineage
How this model relates to others in its lineage. Family members share architecture and training-data roots; parent / children edges record direct distillation or fine-tune relationships.
Strengths
Weaknesses
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|
Get the model
HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of Granite 4.1 8B Instruct.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
Can I use Granite 4.1 8B Instruct commercially?
What's the context length of Granite 4.1 8B Instruct?
Source: huggingface.co/ibm-granite/granite-4.1-8b
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify Granite 4.1 8B Instruct runs on your specific hardware before committing money.