Qwen 2.5 Math 72B
Largest Qwen 2.5 Math. Datacenter-tier math specialist; eclipsed by R1 distills for general reasoning.
Positioning
Qwen 2.5 Math 72B is a dense 72-billion-parameter model from Alibaba, released under the Qwen License. It is the largest model in the Qwen 2.5 Math family, purpose-built for mathematical reasoning. With a 4,096-token context window, it is designed for datacenter deployment where high-throughput, specialized math capabilities are needed. In the open-weight landscape, it stands as a dedicated math specialist, though it has been eclipsed by more recent reasoning-focused models like DeepSeek-R1 distills for general reasoning tasks.
Strengths
- Dedicated math specialist: Purpose-built for mathematical reasoning, making it a strong candidate for tasks requiring precise computation and formal logic.
- Dense architecture: As a dense 72B model, it uses all parameters for every forward pass, providing consistent capacity without the routing overhead of mixture-of-experts models.
- Permissive commercial license: The Qwen License allows commercial use, making it suitable for enterprise deployment in math-intensive applications.
- Large parameter count: With 72B parameters, it offers substantial representational capacity for complex mathematical problem-solving.
Limitations
- Datacenter-only deployment: At FP16, the model requires ~144 GB of disk space, and even at Q4_K_M it needs ~40.5 GB plus significant overhead for KV cache and framework, placing it firmly in datacenter-class hardware (multi-GPU setups).
- Short context window: The 4,096-token context is limiting for tasks requiring long-form reasoning or multi-step derivations with extensive intermediate steps.
- Eclipsed for general reasoning: While strong in math, it has been surpassed by newer reasoning-focused models (e.g., R1 distills) for broader reasoning tasks.
- No community benchmarks available: We do not have independent, community-reported benchmark results for this model. Operators should treat vendor-published metrics as best-case and verify performance in their own environment.
What it takes to run this locally
Quantized sizes (disk):
- FP16: ~144 GB
- Q8_0: ~77 GB
- Q6_K: ~59.4 GB
- Q5_K_M: ~51.3 GB
- Q4_K_M: ~40.5 GB
- Q3_K_M: ~35.1 GB
- Q2_K: ~23.4 GB
Add approximately 30–50% for KV cache and framework overhead at typical context lengths. This model requires datacenter-class hardware: multiple high-memory GPUs (e.g., 2–4× A100 80GB or H100) or a multi-GPU workstation with sufficient VRAM. Single consumer GPUs (12–24 GB) cannot run even the smallest quantized version due to memory constraints.
Should you run this locally?
Yes if: You need a dedicated math specialist for datacenter-scale inference, have access to multi-GPU hardware (A100/H100 clusters), and require a permissive commercial license for math-focused applications.
No if: You are working with limited hardware (single consumer GPU), need long-context reasoning, or require a general-purpose reasoning model. For most local setups, smaller math-tuned models or newer reasoning distills are more practical.
Catalog cross-links
- Qwen 2.5 Math 7B
- DeepSeek-R1 Distill Qwen 32B
- Qwen 2.5 72B
Overview
Largest Qwen 2.5 Math. Datacenter-tier math specialist; eclipsed by R1 distills for general reasoning.
How to run it
Qwen 2.5 Math 72B is Alibaba's math-specialized 72B dense model. Run at Q4_K_M via Ollama (ollama pull qwen2.5-math:72b) or llama.cpp with -ngl 999 -fa -c 8192. Q4_K_M file size ~41 GB on disk. Minimum VRAM: 48 GB — RTX A6000 (48GB) at Q4_K_M for 4K context. RTX 4090 24GB: Q3_K_M with KV offload. Recommended: A100 80GB at AWQ-INT4 for serving. Throughput: ~12-20 tok/s on A6000 at Q4_K_M; ~25-40 tok/s on A100. Qwen 2.5 architecture — well-supported in llama.cpp, vLLM, and Ollama. Math specialization means this model was trained with math-specific data and reasoning formats. Use for: theorem proving, mathematical reasoning, step-by-step problem solving, competition math. Not ideal for: general chat, creative writing, coding (use Qwen 2.5 Coder instead). The math tuning may make the model verbose on non-math tasks. Context: 32K advertised; practical at Q4 on 48 GB is 8-16K.
Hardware guidance
Minimum: RTX 3090 24GB at Q3_K_M with KV offload (4K context). Recommended: RTX A6000 48GB at Q4_K_M (8K context). Optimal: A100 80GB at AWQ-INT4. VRAM math: 72B dense, Q4_K_M ≈ 41 GB. KV cache at 8K: ~10 GB. Total: ~51 GB at 8K. Single A6000 48GB: borderline at 8K — trim to 4K or offload KV. RTX 4090 24GB: Q3_K_M ≈ 31 GB + KV offload. RTX 5090 32GB: Q4_K_M ≈ 41 GB — must offload KV to RAM. Mac Studio M4 Max 64GB: Q4_K_M at 5-10 tok/s. Dual RTX 4090 48 GB: Q4_K_M at 8K context — viable. Cloud: A100 at $5-10/hr.
What breaks first
- Math formatting expectations. Qwen 2.5 Math expects problems formatted with specific delimiters (e.g., "Problem:" / "Solution:"). Free-form prompts may trigger verbose, off-target outputs. 2. General-task quality degradation. The math specialization narrows the model's general knowledge. Non-math factual queries may hallucinate more than base Qwen 2.5 72B. 3. Quantization sensitivity on math. Mathematical reasoning degrades more sharply at Q3 than general language tasks. The precise numerical weights matter for arithmetic fidelity. Use Q4_K_M minimum for math. 4. Chain-of-thought explosion. The model is trained to produce detailed CoT — generation length can be 3-5× longer than non-math models for the same problem. Budget extra tokens in your pipeline.
Runtime recommendation
Common beginner mistakes
Mistake: Using high temperature for math. Fix: Temperature >0.3 introduces randomness that corrupts arithmetic. Use temp=0 for deterministic math outputs. Mistake: Expecting Qwen 2.5 Math to be good at coding. Fix: Math tuning ≠ code tuning. Use Qwen 3 Coder 32B for code. Mistake: Truncating CoT output mid-generation. Fix: Qwen 2.5 Math's CoT is verbose — configure max_tokens high enough (4K+) to avoid cutting off the solution. Mistake: Using Q3 quantization for math. Fix: Math is precision-sensitive. Q4_K_M minimum; Q8 if VRAM permits. Q3 loses ~5-10% on math benchmarks vs Q4.
Family & lineage
How this model relates to others in its lineage. Family members share architecture and training-data roots; parent / children edges record direct distillation or fine-tune relationships.
Strengths
- Math-tuned at 72B
Weaknesses
- R1 distill 70B is sharper at general reasoning
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|---|---|
| Q4_K_M | 41.0 GB | 48 GB |
Get the model
HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of Qwen 2.5 Math 72B.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
What's the minimum VRAM to run Qwen 2.5 Math 72B?
Can I use Qwen 2.5 Math 72B commercially?
What's the context length of Qwen 2.5 Math 72B?
Source: huggingface.co/Qwen/Qwen2.5-Math-72B-Instruct
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify Qwen 2.5 Math 72B runs on your specific hardware before committing money.