Falcon Mamba 7B
TII's Mamba (state-space) architecture model. Linear inference cost; the architectural alternative to attention-based models.
Overview
TII's Mamba (state-space) architecture model. Linear inference cost; the architectural alternative to attention-based models.
Strengths
- Linear inference cost
- SSM architecture variety
Weaknesses
- Long prompts need sequential prefill to keep memory flat
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|---|---|
| Q4_K_M | 4.2 GB | 6 GB |
Get the model
HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of Falcon Mamba 7B.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
What's the minimum VRAM to run Falcon Mamba 7B?
Can I use Falcon Mamba 7B commercially?
Source: huggingface.co/tiiuae/falcon-mamba-7b
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify Falcon Mamba 7B runs on your specific hardware before committing money.