Choosing a GPU for local AI in 2026

The two numbers that decide everything

Most GPU comparisons get distracted by gaming benchmarks, ray-tracing performance, or the MSRP. For local AI, those numbers are mostly irrelevant. The two numbers that actually determine whether a card is good for running LLMs are VRAM capacity and memory bandwidth, in that order.

VRAM is the gate. If a model doesn't fit, it doesn't run — full stop. You can spill into system RAM via CPU offload, but you'll go from 40 tokens/sec to 2-3 tokens/sec. That's the difference between "this is a useful tool" and "I'd rather use ChatGPT." So the first question is always: how much VRAM does the card have?

Memory bandwidth is the speed limit. Once a model fits in VRAM, the speed at which it generates tokens is almost entirely bound by how fast the GPU can read its own weights from memory. Tokens-per-second scales roughly linearly with bandwidth. The RTX 5090 has 1,792 GB/s; the RTX 5060 has about 280 GB/s. That's a 6× gap in real inference speed even on a model that fits comfortably on both.

Compute (TFLOPS) matters for prefill — the time before the first token appears. But for sustained generation, it's the bandwidth that determines whether you're waiting or working.

Tier breakdown

Below are six tiers based on VRAM capacity and price-per-VRAM-GB. Each tier corresponds to a different class of model you can comfortably run. Prices shown are launch MSRPs with their year, from our catalog. Street and used prices move, so check the current price before you buy.

Entry tier — 8 GB cards

Examples: RTX 5060 (launch MSRP $299 (2025)), RTX 4060 (launch MSRP $299 (2023)), RTX 3060 8GB, RX 7600 XT 16GB (launch MSRP $329 (2024)) (the better pick if you're shopping at this price).

What runs: 7B models in 4-bit quantization with reduced context. You can chat with Llama 3.1 8B or Qwen 3 8B reasonably well. You cannot run 14B-class models comfortably.

Our verdict: Skip the 8 GB tier entirely if AI is your primary use case. The performance gap from 8 GB to 12 GB is much larger than the price gap, and 8 GB cards cap your future as the field moves toward larger models. The single exception is the RX 7600 XT — one of the cheapest 16 GB cards on the new market, which is a different conversation entirely.

Mid tier — 12 GB cards

Examples: RTX 3060 12GB (launch MSRP $329 (2021)) (the value floor, new or used), RTX 4070 Super (launch MSRP $599 (2024)), RTX 5070 (launch MSRP $549 (2025)).

What runs: 14B models in 4-bit with usable context. 7B models with full context and high-quality quantization. Borderline 32B with severe quant compression.

Our verdict: The RTX 3060 12GB is the cheapest serious entry into local AI and remains a coherent buy in 2026 even though it's a 2021 card. The RTX 4070 Super is the mid-range pick if you're buying new — it has 1.5× the bandwidth of the 3060 and runs 14B models meaningfully faster. The RTX 5070 has the same VRAM as both with newer architecture but minimal real-world advantage; only buy at MSRP, never above.

High mid tier — 16 GB cards (the sweet spot)

Examples: RTX 4060 Ti 16GB (launch MSRP $499 (2023)), RTX 5060 Ti 16GB (launch MSRP $429 (2025)), RX 9070 XT (launch MSRP $599 (2025)), RX 7800 XT (launch MSRP $499 (2023)), RTX 4080 Super (launch MSRP $999 (2024)), RTX 5080 (launch MSRP $999 (2025)).

What runs: 14B comfortably with full context. 32B models in 4-bit with reasonable context. 70B-class models become possible with aggressive offload (slow).

Our verdict: This tier is where the genuine "local AI is fun" experience starts. The RTX 4060 Ti 16GB is the value pick — its memory bandwidth is mediocre but the VRAM-per-dollar is unmatched on the new market. If your budget stretches further, the RTX 5080 gives you the same 16 GB but with 2-3× the bandwidth and 2026-current architecture. The RX 9070 XT is the AMD pick if you're on Linux and want to skip the NVIDIA tax — same 16 GB, comparable bandwidth, and a lower launch price than the 5080.

Don't get confused by: the RTX 4060 Ti 8GB. Despite the same name, it's a materially different card with half the VRAM. Always verify "16GB" is in the listing.

Enthusiast tier — 24 GB cards

Examples: RTX 3090 (used; the champion of VRAM per dollar), RTX 3090 Ti (used), RTX 4090 (launch MSRP $1,599 (2022)), RX 7900 XTX (launch MSRP $999 (2022)).

What runs: 32B models comfortably. 70B in 4-bit does not fit on one 24 GB card (Q4_K_M weights are about 42 GB), so it needs partial CPU offload or a second card. MoE models like Qwen 3 30B-A3B run beautifully here.

Our verdict: If we had to recommend a single card for serious local AI in 2026, it would be a used RTX 3090. You get the same 24 GB as the RTX 4090 for much less money, with about 25 percent slower inference. The bandwidth (936 GB/s) is excellent. The community has stress-tested this card for four years.

The RTX 4090 is faster but the price premium over the 3090 is hard to justify unless you specifically need the speed. The RX 7900 XTX is the sleeper pick on Linux — same 24 GB, comparable bandwidth, and a much lower launch price than the 4090. ROCm support is now solid for inference (training is still rougher).

Top tier — 32 GB single card

Example: RTX 5090 (launch MSRP $1,999 (2025)). It often sells above its launch price.

What runs: 32B in 4-bit with long context. Practical multi-model workflows where you keep a primary 32B + a draft model loaded simultaneously. 70B in 4-bit still does not fit fully (Q4_K_M weights are about 42 GB), so it needs partial CPU offload, and 100B-class MoE models in 4-bit need offload too.

Our verdict: The RTX 5090 is the only consumer card that crosses 24 GB, and its price reflects that exclusivity tax. If you specifically need 32 GB on a single card — for very long contexts or multi-model setups — there's no competitor. Otherwise, two used RTX 3090s give you 48 GB across two cards via tensor parallelism, with the catch that not every workflow supports multi-card.

Workstation tier — 48-96 GB

Examples: RTX PRO 6000 Blackwell (96 GB), RTX 6000 Ada (48 GB), RTX A6000 (48 GB, used).

What runs: 70B in 4-bit on a 48 GB card (tight on context). On the 96 GB card, 70B in 8-bit or 120B-class models in 4-bit. FP16 70B (about 140 GB of weights) and the largest MoE models (Qwen 3 235B-A22B) still need offload or more than one card.

Our verdict: The used RTX A6000 is genuinely interesting for serious tinkerers — buying it used puts 48 GB on one card in reach without paying for a new workstation card. The RTX PRO 6000 Blackwell is for production users who need 96 GB and full warranty support; for hobbyists its price is hard to justify over multi-3090 setups.

Apple Silicon — the unified-memory plot twist

Apple's M-series chips are unique in that the GPU and CPU share the same memory pool. There is no PCIe transfer cost, no fragmentation between VRAM and system RAM. A Mac Studio with M3 Ultra and 256 GB or 512 GB of unified memory can load models that no consumer NVIDIA card can hold.

The relevant Apple chips for local AI:

• M4 Max with 64-128 GB. 546 GB/s memory bandwidth. Runs 70B Q4 comfortably. The premium AI laptop pick.
• M3 Ultra with 96-512 GB. 819 GB/s bandwidth. Mac Studio configurations top out at 512 GB unified — enough to run 671B-class MoE models in 4-bit.
• M4 Pro with 24-48 GB. 273 GB/s. Solid for 14-32B class.

The catch is the software ecosystem. MLX (Apple's native ML framework) performs excellently — sometimes 70-90 percent of CUDA throughput on equivalent hardware — but the breadth of supported tools and quantization formats is narrower. llama.cpp on Metal is mature; vLLM and ExLlamaV2 don't run on Apple. Image-generation tooling (ComfyUI, Stable Diffusion WebUI) lags significantly behind NVIDIA.

Our verdict: If you primarily run LLMs (not image gen), and you value quiet/low-power, M4 Max with 64 GB is the simplest path to "I can run any consumer-class model" without dealing with discrete GPUs. The configurations to look for: M4 Max (16-core CPU, 40-core GPU) with 64 GB, or an M3 Ultra Mac Studio with 256 GB.

The used market math

VRAM per dollar on the used market in 2026 is dominated by NVIDIA's 30-series. A used RTX 3090 has the same 24 GB as a new RTX 4090 and costs a fraction of a new RTX 5090; the trade is about 25 percent slower inference and an older architecture. Used prices move week to week, so compare current listings rather than any number you read here.

The used cards we still actively recommend in 2026:

• RTX 3090 — the VRAM-per-dollar champion
• RTX 3090 Ti — slightly faster, same VRAM
• RTX 3060 12GB — entry-level value floor
• RTX A6000 — 48 GB on one card without a new workstation card's price

We do not recommend used RTX 4090s — their used prices haven't dropped enough below new to justify the warranty risk. We do not recommend used 8 GB cards of any era for AI use.

The mistakes to avoid

1. Buying for compute over VRAM. The RTX 5060 has more compute than an RTX 3060 12GB, but the 3060 runs more models because of the 4 GB VRAM advantage. Always optimize for VRAM first.
2. Assuming "more recent" means "better for AI." Generation matters less than VRAM and bandwidth. A used RTX 3090 outperforms a new RTX 5070 for AI specifically.
3. Believing memory-bandwidth-blind benchmarks. Gaming benchmarks measure compute. AI benchmarks should measure memory bandwidth. They tell different stories.
4. Going AMD on Windows for AI. ROCm Windows support has improved but still trails Linux substantially. If you must run Windows, NVIDIA still wins for AI.
5. Skipping cooling consideration on laptops. Laptop GPUs throttle under sustained inference load. A "RTX 4090 Mobile 16GB" laptop with bad thermals will underperform an RTX 4070 Ti Super desktop in real use. Read sustained-load reviews.
6. Buying for what you might run someday. Buy for the model you will run this month. The field moves fast enough that hardware bought "for future 200B models" will be eclipsed by something better before those models matter to you.

The buy-or-wait calculation in 2026

NVIDIA's RTX 60-series is rumored for late 2027. AMD's RDNA 5 (RX 10000-series) is rumored for the same window. Apple's Mac Studio with M5 Max and M5 Ultra went on sale in September 2026 (our news item); the M4 Max and M3 Ultra picks on this page are now the previous generation.

If you're shopping now, the RTX 50 series and RX 9070 line are on sale, the used 30-series is plentiful, and Apple's M5 generation is the current Mac option. If you wait until late 2027 hoping for the next generation, you'll wait through six months of supply constraint and pay launch premiums when it ships. Buy now for the model size you actually want to run, with the assumption that you'll repeat this calculation in 18 months.

Where to go from here

• Check exactly what your hardware can run with our compatibility tool: Will it run? — find what your GPU can run
• Browse the full hardware database with bandwidth specs and benchmarks: Hardware directory
• See benchmarks tagged by source (owner-run, community, vendor): Benchmarks
• Read our methodology page for how we calculate tokens-per-second predictions

Frequently asked