Qwen 3 8B
Qwen 3 at the 8B scale. Direct head-to-head against Llama 3.1 8B on most benchmarks; usually wins on coding and structured output.
Positioning
Qwen 3 8B introduces a hybrid "thinking" / "non-thinking" toggle into the 7B class. In non-thinking mode it's a tier with Qwen 2.5 7B; in thinking mode it produces visible chain-of-thought and lifts hard-task performance closer to 14B-class models at the cost of latency.
Strengths
- Hybrid reasoning toggle —
/thinkand/no_thinkper turn lets you pay for reasoning only when needed. - Improved tool-use over Qwen 2.5 — function-call format more standardized.
- Strong multilingual carryover from the 2.5 generation.
Limitations
- Thinking-mode output is verbose — tokens-per-answer roughly doubles, eating speed.
- Some prompt-injection vectors specific to the
/thinktoggle that haven't been fully audited. - License remains Qwen-flavored with usage caps.
Real-world performance on RTX 4090
- Q4_K_M (5.0 GB): 95–115 tok/s decode (non-thinking); 90–110 tok/s thinking but 2× output
- Q5_K_M (5.9 GB): 85–100 tok/s
- Q8_0 (8.4 GB): 65–82 tok/s
Should you run this locally?
Yes, for users who want the best 8B-class capability and are willing to use thinking mode selectively for hard prompts. No, for users who don't need reasoning — Qwen 2.5 7B is simpler and similar speed.
How it compares
- vs Qwen 2.5 7B → Qwen 3 8B with thinking mode wins on reasoning; without thinking, near-equal. Pick Qwen 3 if reasoning matters.
- vs Llama 3.1 8B → Qwen 3 8B wins on raw capability; Llama wins on instruction polish + ecosystem maturity.
- vs QwQ 32B → QwQ is the dedicated reasoning specialist at 32B; Qwen 3 8B's thinking mode is a poor man's QwQ at lighter VRAM.
- vs Phi-4 14B → Phi-4 has cleaner reasoning at higher VRAM; Qwen 3 8B fits in less memory.
Run this yourself
ollama pull qwen3:8b
ollama run qwen3:8b
# Toggle reasoning per turn:
# /think — enable chain-of-thought
# /no_think — disable
›Why this rating
8.5/10 — Qwen 3's hybrid reasoning mode in an 8B body. Strong as a 7B-class chat model, with a "thinking" mode that pushes it materially beyond Qwen 2.5 7B on reasoning tasks. Loses points only on ecosystem maturity vs Llama 3.1 8B.
Overview
Qwen 3 at the 8B scale. Direct head-to-head against Llama 3.1 8B on most benchmarks; usually wins on coding and structured output.
Family & lineage
How this model relates to others in its lineage. Family members share architecture and training-data roots; parent / children edges record direct distillation or fine-tune relationships.
Strengths
- Best 8B coder
- Apache 2.0
- Thinking mode
Weaknesses
- More verbose with thinking enabled
Prompting kit
Tested patterns for getting the most out of Qwen 3 8B locally. Local models are pickier about prompt structure than cloud models — what works on Claude or GPT-5 often fails here.
Recommended system prompt
You are Qwen, a helpful assistant created by Alibaba Cloud. Answer directly and concisely. For multi-step problems, work through your reasoning before giving the final answer.
Quirks to know
- •Small Qwen 3 sibling for 8GB-VRAM rigs. Same /think + /no_think mode toggle as the larger Qwen 3 models.
- •Native 32K context, extendable to 128K with YaRN scaling.
- •ChatML template — same as all Qwen 3 family models.
- •Multilingual: 119 languages per the model card. Quality is materially lower than Qwen 3 32B on lower-resource languages — anchor system prompts in the target language explicitly for best results.
- •Tool-call reliability lower than 32B sibling — use a strict JSON schema validator runtime-side and re-prompt on parse failures.
Chat template
Same template as Qwen 3 32B and Qwen 3 30B-A3B.
Tool calling
Hermes-style. Reliability degrades vs the 32B per the model card; constrain output strictly.
Sampler settings
- temperature
- 0.7
- top_p
- 0.8
- top_k
- 20
Same vendor-recommended defaults as Qwen 3 32B. /think mode: temperature 0.6, top_p 0.95.
Reviewed quality benchmarks
First-party rows were run by RunLocalAI; reviewed community rows are labeled in the data. Every row links to the raw test-run log.
| Benchmark | Quant | Runtime / Hardware | Score | Raw log |
|---|---|---|---|---|
HumanEval+ tested 2026-05-29 | Q4_K_M | ollama-0.24 rtx-3080-16gb-mobile | 2.4/100 | Gist → |
Q4_K_M note:First-party HumanEval+ on RTX 3080 Laptop. Earlier probe failed on this thinking model (max_tokens=5 → empty visible content); re-run with thinking-tolerant probe.
Want to verify? Every row links to its Gist with full stdout and stderr of the run. The runner script is in the public repo (scripts/run-humaneval-plus.ts) — reproducible end-to-end. Browse all coding scores at /benchmarks/coding.
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|---|---|
| Q4_K_M | 4.8 GB | 6 GB |
| Q8_0 | 8.2 GB | 10 GB |
Get the model
Ollama
One-line install
ollama run qwen3:8bRead our Ollama review →HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of Qwen 3 8B.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
What's the minimum VRAM to run Qwen 3 8B?
Can I use Qwen 3 8B commercially?
What's the context length of Qwen 3 8B?
How do I install Qwen 3 8B with Ollama?
Compare against other models
Curated head-to-head decisions where Qwen 3 8B is one of the contenders. For arbitrary pairings use /model-battle.
Source: huggingface.co/Qwen/Qwen3-8B
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify Qwen 3 8B runs on your specific hardware before committing money.