Jamba 1.5 Large
Jamba flagship at 398B total / 94B active. Frontier hybrid-architecture model with 256k context.
Our verdict
Positioning
Jamba 1.5 Large is AI21 Labs' 398B-parameter hybrid architecture combining Mamba state-space layers with traditional Transformer attention. This hybrid design — the largest production SSM+Transformer model available — targets operators who want frontier-scale capability with the unique memory-efficiency properties of state-space models. AI21 positions Jamba 1.5 as a genuine alternative to pure-Transformer architectures at scale, delivering competitive reasoning and coding performance with dramatically lower KV-cache memory pressure for long-context workloads.
Strengths
- Hybrid Mamba+Transformer architecture. State-space layers handle long-range dependencies efficiently; Transformer layers provide the attention precision needed for reasoning. The KV-cache savings are real — especially at 128K+ context lengths where pure-Transformer KV caches become prohibitive.
- Linearly-scaling long-context handling. AI21's paper lists the KV cache at 256K context as 9 GB for Jamba-1.5-Large against 252 GB for LLaMA-3.1 405B, with one attention layer for every seven Mamba layers.
- 398B total parameters with efficient serving characteristics. Jamba is a large model that serves like a meaningfully smaller one at long context — the hybrid architecture's memory savings compound as context grows.
- AI21's production engineering culture. AI21 builds for deployment. Jamba's architecture choices reflect practical serving considerations, not just benchmark-chasing. Operators will appreciate the thoughtful engineering.
- Strong general intelligence benchmarks. Competitive with similarly-sized models on MMLU, HellaSwag, and general knowledge tasks. The hybrid architecture doesn't sacrifice quality for efficiency.
- Jamba Open Model License. Open weights. Commercial use is allowed under the license terms, but an organization with more than 50 million US dollars in annual revenue may not use it commercially under this license.
Limitations
- Serving ecosystem is immature. Hybrid Mamba+Transformer architectures require explicit support in serving frameworks. vLLM is AI21's recommended path and lists Jamba as a supported architecture. llama.cpp has merged Jamba support, and community GGUFs of Jamba 1.5 Large are on Hugging Face, with Q4_K_M at about 240 GB.
- Massive hardware requirements for the full model. 398B parameters at FP16 require ~800 GB. Q4 requires ~240 GB, which needs the 256 GB configuration of the Mac Studio M3 Ultra. AI21's own path is a single node of 8 80GB GPUs with ExpertsInt8.
- Attention layers still require KV cache. The hybrid design reduces but doesn't eliminate KV cache — Transformer layers in the architecture still need cache space. The savings are meaningful but not total.
- Single-GPU consumer paths don't exist. RTX 5090 at 32 GB can't run this at any usable quant. This is a datacenter or high-end workstation model.
- Quality on short-context tasks may not justify the hardware. For workloads under 4K context (typical chat), Jamba's long-context advantage is irrelevant and the model doesn't outperform similarly-sized dense Transformers. The architecture's strength emerges at scale.
Real-world performance
- vs Mistral Medium 3.5: Different architectural bets. Mistral Medium 3.5 is a dense 128B model with a 256k context window; Jamba is hybrid SSM+Transformer. Pick Jamba for long-context workloads with memory constraints; pick Mistral for general-purpose frontier serving.
- vs Qwen 3 235B-A22B: Qwen 3 is more accessible (fits Mac Studio at Q4) and has Apache 2.0 licensing. Jamba's architectural uniqueness (long-context efficiency) is the reason to pick it over Qwen.
Should you run this locally?
Yes if your workload is long-context heavy (128K+ tokens), you have a single node of 8 80GB GPUs, which is the path AI21 documents, and you're willing to invest engineering time in the serving stack. The long-context efficiency is genuinely best-in-class.
No if your context lengths are under 8K (the architecture advantage is wasted), you need mature tooling (pick Llama or Qwen), or you lack the hardware (8 80GB GPUs, per AI21's card).
How it compares
- vs Qwen 3 235B-A22B: Qwen 3 is more accessible and production-hardened; Jamba wins uniquely on long-context memory profile.
- vs [Mamba SSM-only architectures]: Jamba's hybrid design combines Mamba's efficiency with Transformer attention quality. Pure SSM models are more efficient but lower quality.
Run this yourself
- Datacenter: 8× H100 SXM with ExpertsInt8 in vLLM. AI21's card fits prompts up to 220K tokens on 8 80GB GPUs with the default vLLM configuration.
- Workstation: The 256 GB Mac Studio M3 Ultra can hold the ~240 GB Q4_K_M weights from a community GGUF.
- Cloud rental: Runpod / Lambda H100 clusters. AI21's documented path needs 8 80GB GPUs on one node.
- Vendor: huggingface.co/ai21labs/AI21-Jamba-Large-1.5
Overview
Jamba flagship at 398B total / 94B active. Frontier hybrid-architecture model with 256k context.
How to run it
Jamba 1.5 Large is AI21's SSM-hybrid model (398B total, 94B active via MoE). The SSM backbone means KV cache overhead is lower than pure attention — practical for long-context workloads. AI21 recommends vLLM with its ExpertsInt8 quantization on a single node of 8 80GB GPUs, because the model is too large to load at FP16 or BF16 on that node. Q4_K_M arithmetic is ~241 GB, so RTX A6000 48GB, RTX 4090 24GB and dual RTX 3090 cannot hold it. AI21's paper lists the KV cache at 256K context as 9 GB, so the weights, not the cache, set the memory limit.
Hardware guidance
AI21's documented path is a single node of 8 80GB GPUs with ExpertsInt8 in vLLM. VRAM math: 398B total, 94B active. All 398B must be loaded because experts are selected per token. Q4_K_M ≈ 241 GB for weights, FP16 ≈ 796 GB. KV cache at 256K context: 9 GB per AI21's paper. RTX 3090 24GB, RTX A6000 48GB, RTX 6000 Ada 48GB, Mac Studio M4 Max 64GB and A10 24GB cannot hold these weights.
What breaks first
- Mamba kernels. AI21's card installs
mamba-ssmandcausal-conv1dfor the optimized kernels, and mamba-ssm's CUDA build needs CUDA 11.6+. If they will not install,use_mamba_kernels=Falseruns the model without them at the cost of extra latency. 2. No official Ollama model. Ollama's library has no official Jamba 1.5 model. Its search lists only user uploads of Jamba Reasoning 3B. 3. Training data cutoff. AI21's card gives a knowledge cutoff of March 5, 2024. RAG or web grounding needed for current information.
Runtime recommendation
vLLM is AI21's recommended path and lists Jamba as a supported architecture. Use ExpertsInt8 on 8 80GB GPUs. llama.cpp has a Jamba architecture and MLX-LM has a Jamba model file. Community GGUFs of Jamba 1.5 Large exist on Hugging Face, with Q4_K_M at about 240 GB, more than any consumer GPU can hold. Ollama's library has no official Jamba 1.5 model.
Common beginner mistakes
Mistake: Assuming Jamba runs like a standard Llama. Fix: Jamba is SSM-hybrid — different architecture. Benchmark your workload specifically. Mistake: Planning to run Jamba Large on an RTX 4090. Fix: Q4_K_M weights are ~241 GB against 24 GB of VRAM. AI21's documented path is 8 80GB GPUs. Mistake: Setting 256K context and expecting it to work. Fix: AI21's card fits prompts up to 220K tokens on 8 80GB GPUs with the default vLLM configuration, and shows a vLLM configuration for the full 256K at the cost of added latency at high loads. Mistake: Using automatic mixed precision (AMP) without testing. Fix: SSM precision sensitivity differs from attention. Custom quants may produce numerical instability.
Family & lineage
How this model relates to others in its lineage. Family members share architecture and training-data roots; parent / children edges record direct distillation or fine-tune relationships.
Strengths
- 256k context at frontier scale
- Hybrid architecture
Weaknesses
- Cluster-only deployment
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|---|---|
| Q4_K_M | 240.0 GB | 260 GB |
At Q4 Jamba 1.5 Large needs about 260 GB of memory by our estimate, more than a 32 GB graphics card holds. Renting a GPU with enough memory (or several GPUs) by the hour is the practical way to try it before you plan hardware for it.
Referral links: RunLocalAI may earn a commission or account credits if you sign up and rent, at no extra cost to you. Hourly rates and which GPUs are listed change all the time, so check the provider's page before you start. How we make money.
Get the model
HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of Jamba 1.5 Large.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
What's the minimum VRAM to run Jamba 1.5 Large?
Can I use Jamba 1.5 Large commercially?
What's the context length of Jamba 1.5 Large?
Source: huggingface.co/ai21labs/AI21-Jamba-Large-1.5
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify Jamba 1.5 Large runs on your specific hardware before committing money.