Llama 3.2 1B Instruct
True edge-tier Llama. Runs on a phone or Raspberry Pi. Useful for classification, simple summarization, and on-device agents.
Positioning
A 1B model exists for one job: routing or classification inside agent loops, where you need decisions in 5–10 ms on minimal hardware. As a chat model, it's clearly the bottom of the useful spectrum — fine for trivial queries, struggles with anything multi-step.
Strengths
- Under 1 GB at Q4_K_M — runs on Raspberry Pi 5 with NPU, mobile devices, anywhere with at least 2 GB free RAM.
- Conversational tone holds up better than Phi 1.5 at similar parameter count.
- Same permissive license as the rest of the Llama family.
Limitations
- Multi-step reasoning fails frequently — pick a 3B+ for anything beyond a one-turn answer.
- Hallucinates on factual questions more aggressively than expected; needs RAG or strict refusal prompting.
- No structured-output reliability — JSON mode is unstable.
Real-world performance on RTX 4090
- Q4_K_M (0.8 GB): 220–280 tok/s decode, TTFT under 30 ms
- Q5_K_M (0.95 GB): 200–250 tok/s
- Q8_0 (1.3 GB): 170–210 tok/s
Should you run this locally?
Yes, for routing layers in agent stacks (intent classification, query rewriting, tool selection), or for genuinely low-spec edge devices. No, for any standalone chat or assistant role.
How it compares
- vs Llama 3.2 3B → 3B is much more capable; only pick 1B when memory or latency forces it.
- vs Qwen 2.5 1.5B → Qwen 1.5B is meaningfully smarter at similar footprint; preferred for new edge work.
- vs Phi-3.5 Mini (3.8B) → not the same class; Phi is for "small but capable", 1B is "tiny but functional".
Run this yourself
ollama pull llama3.2:1b-instruct-q4_K_M
ollama run llama3.2:1b-instruct-q4_K_M
›Why this rating
6.0/10 — the smallest useful Llama. Below this size, models stop being general-purpose and become routing/classification helpers. Loses points to Qwen 2.5 1.5B which is more capable at near-equal footprint.
Overview
True edge-tier Llama. Runs on a phone or Raspberry Pi. Useful for classification, simple summarization, and on-device agents.
Family & lineage
How this model relates to others in its lineage. Family members share architecture and training-data roots; parent / children edges record direct distillation or fine-tune relationships.
Strengths
- Mobile-class footprint
- Fast on CPU
- 128K context
Weaknesses
- Limited reasoning
- Hallucinations more common
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|---|---|
| Q4_K_M | 0.8 GB | 2 GB |
| Q8_0 | 1.3 GB | 2 GB |
Get the model
Ollama
One-line install
ollama run llama3.2:1bRead our Ollama review →HuggingFace
Original weights
Source repository — direct quantization required.
Benchmarks
Real measurements on real hardware. Numbers ship with the runner version, quant, and date.
| Hardware | Provenance | Quant | Ctx | Tokens / sec | TTFT | Date |
|---|---|---|---|---|---|---|
| NVIDIA GeForce RTX 3080 16GB (Mobile) | EditorialM | Q4_K_M | 4K | 189.5tok/s | 359 ms | Jun 2, 26 |
What to do next
Got this model running on real hardware? Share what you measured — the form arrives with the model pre-selected.
Hardware that runs this
Cards with enough VRAM for at least one quantization of Llama 3.2 1B Instruct.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
What's the minimum VRAM to run Llama 3.2 1B Instruct?
Can I use Llama 3.2 1B Instruct commercially?
What's the context length of Llama 3.2 1B Instruct?
How do I install Llama 3.2 1B Instruct with Ollama?
Source: huggingface.co/meta-llama/Llama-3.2-1B-Instruct
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify Llama 3.2 1B Instruct runs on your specific hardware before committing money.