Every GPU ranked for local AI inference
One screen. Every catalog GPU sorted by tier, with estimated tok/s for the four canonical model sizes (7B, 14B, 32B, 70B at Q4_K_M). Measurements where we have them, bandwidth-derived estimates where we don't — every cell labeled so you know what you're reading. Methodology at /methodology.
| Tier | GPU | VRAM | BW (GB/s) | Launch MSRP | 7B Q4 | 14B Q4 | 32B Q4 | 70B Q4 | Rating |
|---|---|---|---|---|---|---|---|---|---|
| A | NVIDIA GeForce RTX 4090 nvidia · 2022 | 24 GB | 1008 | $1,599 | 112–157 | 59–83 | 26–36 | — | 9.4 |
| A | NVIDIA GeForce RTX 5090 nvidia · 2025 | 32 GB | 1792 | $1,999 | 199–279 | 105–148 | 46–64 | — | — |
| A | Apple Mac Studio (M3 Ultra) apple · 2025 | 0 GB | ? | $4,999 | ? | ? | ? | ? | — |
| A | MacBook Pro 16" M4 Max apple · 2024 | 0 GB | ? | $3,999 | ? | ? | ? | ? | — |
| B | AMD Radeon RX 7900 XTX amd · 2022 | 24 GB | 960 | $999 | 107–149 | 56–79 | 25–34 | — | 7.8 |
| B | NVIDIA GeForce RTX 4080 Super nvidia · 2024 | 16 GB | 736 | $999 | 82–114 | 43–61 | — | — | 7.2 |
| B | NVIDIA GeForce RTX 5080 nvidia · 2025 | 16 GB | 960 | $999 | 107–149 | 56–79 | — | — | — |
| B | Lenovo Legion 5 Pro Gen 7 (RTX 3080 16GB) nvidia · 2022 | 16 GB | ? | $2,299 | ? | ? | ? | ? | — |
| B | NVIDIA GeForce RTX 3090 nvidia · 2020 | 24 GB | ? | $1,499 | ? | ? | ? | ? | — |
| B | NVIDIA GeForce RTX 3090 Ti nvidia · 2022 | 24 GB | ? | $1,999 | ? | ? | ? | ? | — |
| B | NVIDIA GeForce RTX 4070 Ti Super nvidia · 2024 | 16 GB | ? | $799 | ? | ? | ? | ? | — |
| B | NVIDIA GeForce RTX 4080 nvidia · 2022 | 16 GB | ? | $1,199 | ? | ? | ? | ? | — |
| B | NVIDIA GeForce RTX 5070 Ti nvidia · 2025 | 16 GB | ? | $749 | ? | ? | ? | ? | — |
| C | AMD Radeon RX 9060 XT amd · 2026 | 16 GB | 640 | $449 | 71–100 | 38–53 | — | — | 7.5 |
| C | AMD Radeon RX 7800 XT amd · 2023 | 16 GB | ? | $499 | ? | ? | ? | ? | — |
| C | AMD Radeon RX 7900 XT amd · 2022 | 20 GB | ? | $899 | ? | ? | ? | ? | — |
| C | AMD Radeon RX 9070 amd · 2025 | 16 GB | ? | $549 | ? | ? | ? | ? | — |
| C | AMD Radeon RX 9070 XT amd · 2025 | 16 GB | ? | $599 | ? | ? | ? | ? | — |
| C | NVIDIA GeForce RTX 3080 12GB nvidia · 2022 | 12 GB | ? | $799 | ? | ? | ? | ? | — |
| C | NVIDIA GeForce RTX 4060 Ti 16GB nvidia · 2023 | 16 GB | ? | $499 | ? | ? | ? | ? | — |
| C | NVIDIA GeForce RTX 4070 nvidia · 2023 | 12 GB | ? | $599 | ? | ? | ? | ? | — |
| C | NVIDIA GeForce RTX 4070 Super nvidia · 2024 | 12 GB | ? | $599 | ? | ? | ? | ? | — |
| C | NVIDIA GeForce RTX 4070 Ti nvidia · 2023 | 12 GB | ? | $799 | ? | ? | ? | ? | — |
| C | NVIDIA GeForce RTX 5060 Ti 16GB nvidia · 2025 | 16 GB | ? | $429 | ? | ? | ? | ? | — |
| C | NVIDIA GeForce RTX 5070 nvidia · 2025 | 12 GB | ? | $549 | ? | ? | ? | ? | — |
| D | NVIDIA GeForce RTX 3060 12GB nvidia · 2021 | 12 GB | 360 | $329 | 40–56 | 21–30 | — | — | — |
| D | AMD Radeon RX 7600 XT amd · 2024 | 16 GB | ? | $329 | ? | ? | ? | ? | — |
| D | AMD Radeon RX 7700 XT amd · 2023 | 12 GB | ? | $449 | ? | ? | ? | ? | — |
| D | Intel Arc A770 16GB intel · 2022 | 16 GB | ? | $349 | ? | ? | ? | ? | — |
| D | Intel Arc B570 intel · 2025 | 10 GB | ? | $219 | ? | ? | ? | ? | — |
| D | Intel Arc B580 intel · 2024 | 12 GB | ? | $249 | ? | ? | ? | ? | — |
| D | NVIDIA GeForce RTX 3070 nvidia · 2020 | 8 GB | ? | $499 | ? | ? | ? | ? | — |
| D | NVIDIA GeForce RTX 3080 10GB nvidia · 2020 | 10 GB | ? | $699 | ? | ? | ? | ? | — |
| D | NVIDIA GeForce RTX 4060 nvidia · 2023 | 8 GB | ? | $299 | ? | ? | ? | ? | — |
| D | NVIDIA GeForce RTX 4060 Ti 8GB nvidia · 2023 | 8 GB | ? | $399 | ? | ? | ? | ? | — |
| D | NVIDIA GeForce RTX 5060 nvidia · 2025 | 8 GB | ? | $299 | ? | ? | ? | ? | — |
| D | NVIDIA GeForce RTX 5060 Ti 8GB nvidia · 2025 | 8 GB | ? | $379 | ? | ? | ? | ? | — |
| M | NVIDIA GeForce RTX 3080 16GB (Mobile) nvidia · 2022 | 16 GB | 512 | ? | 57–80 | 30–42 | — | — | — |
| M | ASUS ROG Strix Scar 18 (RTX 5090 Mobile) nvidia · 2025 | 24 GB | ? | $3,999 | ? | ? | ? | ? | — |
| M | Framework Laptop 16 (RX 7700S) amd · 2024 | 8 GB | ? | $1,699 | ? | ? | ? | ? | — |
| M | NVIDIA GeForce RTX 4090 Mobile nvidia · 2023 | 16 GB | ? | ? | ? | ? | ? | ? | — |
| M | NVIDIA GeForce RTX 5090 Mobile nvidia · 2025 | 24 GB | ? | ? | ? | ? | ? | ? | — |
| M | Razer Blade 16 (2025, RTX 5090 Mobile) nvidia · 2025 | 24 GB | ? | $4,499 | ? | ? | ? | ? | — |
| S | NVIDIA RTX 5000 PRO Blackwell 48GB nvidia · 2026 | 48 GB | 960 | $5,499 | 107–149 | 56–79 | 25–34 | 11–16 | 8.5 |
| S | AMD Instinct MI210 amd · 2022 | 64 GB | ? | $8,500 | ? | ? | ? | ? | — |
| S | AMD Instinct MI250X amd · 2021 | 128 GB | ? | $13,000 | ? | ? | ? | ? | — |
| S | AMD Instinct MI300A (APU) amd · 2023 | 128 GB | ? | ? | ? | ? | ? | ? | — |
| S | AMD Instinct MI300X amd · 2023 | 192 GB | ? | $15,000 | ? | ? | ? | ? | — |
| S | AMD Instinct MI325X amd · 2024 | 256 GB | ? | $20,000 | ? | ? | ? | ? | — |
| S | AMD Instinct MI355X amd · 2025 | 288 GB | ? | $25,000 | ? | ? | ? | ? | — |
| S | Intel Gaudi 2 intel · 2022 | 96 GB | ? | $8,000 | ? | ? | ? | ? | — |
| S | Intel Gaudi 3 intel · 2024 | 128 GB | ? | $18,000 | ? | ? | ? | ? | — |
| S | NVIDIA A100 40GB nvidia · 2020 | 40 GB | ? | $11,000 | ? | ? | ? | ? | — |
| S | NVIDIA A100 80GB SXM nvidia · 2020 | 80 GB | ? | $17,000 | ? | ? | ? | ? | — |
| S | NVIDIA A40 nvidia · 2020 | 48 GB | ? | $5,500 | ? | ? | ? | ? | — |
| S | NVIDIA B200 nvidia · 2024 | 192 GB | ? | $40,000 | ? | ? | ? | ? | — |
| S | NVIDIA DGX Spark (Project Digits) nvidia · 2025 | 0 GB | ? | $3,999 | ? | ? | ? | ? | — |
| S | NVIDIA GB200 NVL72 nvidia · 2024 | 13824 GB | ? | ? | ? | ? | ? | ? | — |
| S | NVIDIA H100 NVL nvidia · 2023 | 188 GB | ? | $60,000 | ? | ? | ? | ? | — |
| S | NVIDIA H100 PCIe nvidia · 2022 | 80 GB | ? | $25,000 | ? | ? | ? | ? | — |
| S | NVIDIA H100 SXM nvidia · 2022 | 80 GB | ? | $30,000 | ? | ? | ? | ? | — |
| S | NVIDIA H200 nvidia · 2024 | 141 GB | ? | $31,000 | ? | ? | ? | ? | — |
| S | NVIDIA L4 nvidia · 2023 | 24 GB | ? | $2,500 | ? | ? | ? | ? | — |
| S | NVIDIA L40 nvidia · 2022 | 48 GB | ? | $8,000 | ? | ? | ? | ? | — |
| S | NVIDIA L40S nvidia · 2023 | 48 GB | ? | $8,500 | ? | ? | ? | ? | — |
| S | NVIDIA RTX 5000 Ada Generation nvidia · 2023 | 32 GB | ? | $4,000 | ? | ? | ? | ? | — |
| S | NVIDIA RTX 6000 Ada Generation nvidia · 2022 | 48 GB | ? | $6,799 | ? | ? | ? | ? | — |
| S | NVIDIA RTX A5000 nvidia · 2021 | 24 GB | ? | $2,500 | ? | ? | ? | ? | — |
| S | NVIDIA RTX A6000 (Ampere) nvidia · 2020 | 48 GB | ? | $4,650 | ? | ? | ? | ? | — |
| S | NVIDIA RTX PRO 6000 Blackwell nvidia · 2025 | 96 GB | ? | $8,499 | ? | ? | ? | ? | — |
| E | Apple M3 Max apple · 2023 | 0 GB | ? | ? | ? | ? | ? | ? | 8.5 |
| E | AMD Ryzen AI 9 HX 370 (Strix Point) amd · 2024 | 0 GB | ? | $1,599 | ? | ? | ? | ? | — |
| E | Apple A17 Pro apple · 2023 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple A18 Pro apple · 2024 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple M1 Max apple · 2021 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple M1 Ultra apple · 2022 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple M2 Max apple · 2023 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple M2 Ultra apple · 2023 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple M3 Ultra apple · 2025 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple M4 (iPad Pro) apple · 2024 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple M4 Max apple · 2024 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Apple M4 Pro apple · 2024 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Google Tensor G4 google · 2024 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Intel Core Ultra 7 258V (Lunar Lake) intel · 2024 | 0 GB | ? | $1,199 | ? | ? | ? | ? | — |
| E | Qualcomm Snapdragon 8 Elite qualcomm · 2024 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Qualcomm Snapdragon 8 Gen 3 qualcomm · 2023 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Qualcomm Snapdragon X Elite qualcomm · 2024 | 0 GB | ? | ? | ? | ? | ? | ? | — |
| E | Qualcomm Snapdragon X Plus qualcomm · 2024 | 0 GB | ? | ? | ? | ? | ? | ? | — |
Estimates use the formula tok/s ≈ memory_bandwidth_GBps ÷ model_weights_GB × efficiency — the dominant constraint for autoregressive decode. The 50-70% efficiency band reflects realistic Ollama / llama.cpp / vLLM runtime overhead. See /methodology for the full derivation.
The Launch MSRP column is the manufacturer's list price at release, not a live price — hover a figure for the release year and, on older cards, a used-market note. A figure marked est. is our own estimate, checked within the last 90 days. Check the current price before you buy.
Got a rig? Run a benchmark and turn an estimate into a measured cell. Every measurement improves the table for the next reader.
Lowest $/tok-s pick for each model size
For each model tier, the catalog card with the lowest cost-per-tok/s among cards that fit. Computed from our planning price estimate ÷ estimated tok/s midpoint; each pick names the price basis used. A pick changes here when we revise an estimate or new hardware shifts the value frontier.
Caveat: $/tok-s is a derived estimate stacked on the bandwidth formula. For workloads where you have a measured benchmark in the table above, trust the measured number first; for unmeasured combinations, this is the ranked best-guess for buyer decisions. Price figures are our planning estimates (launch MSRP, or a used-market estimate for older cards), not live prices. Check the current price before you buy.
Choosing a GPU for your workload
The hierarchy answers "which is fastest" — but the right card for you depends on which model size you actually want to run. Budget bands are 2026 editorial estimates, not live prices. The four most common operator decisions:
- 7B Q4 (autocomplete, single-model chat) — any card with ≥6 GB VRAM works. The decision shifts to price + power draw + cross-vendor preference. Top D-tier cards (Arc B580, RTX 3060) deliver useful tok/s at <$300.
- 14B Q4 (coding assistant, mid-size chat) — ≥11 GB VRAM minimum. C-tier (RTX 4070 / RX 7800 XT) is the value sweet spot at $400-600.
- 32B Q4 (full coding agent, multi-model) — ≥22 GB VRAM. B-tier 24 GB cards are the canonical buy: RTX 3090 used, RX 7900 XTX new, RTX 4090 if budget allows.
- 70B Q4 (frontier-class local) — ≥48 GB VRAM. Single-card: RTX 6000 Ada / L40S / Mac Studio M3 Ultra. Multi-card: dual 3090 / dual 4090. Workstation tier or above.
Need it more personalized? Use /choose-my-gpu for a 9-input recommender, or /will-it-run to validate a specific model + GPU combination.
Amazon search links — we may earn a small commission at no extra cost to you. Prices change; check the listing before you buy. How we make money.