What can NVIDIA GeForce RTX 5090 run?
Build: RTX 5090 + Ryzen 9 9950X + 64GB DDR5
Runs comfortably128 models
Full-VRAM resident, with room for context. No compromises.
Quant: Q4_K_MContext: 8,192VRAM: 2.4 GBHeadroom: 29.6 GBTTFT: instant3215tok/sEstimated
Quant: FP16Context: 8,192VRAM: 19.8 GBHeadroom: 12.2 GBTTFT: fastollama run llama3.1:8b73tok/sEstimated
ollama run llama3.1:8bQuant: Q4_K_MContext: 2,048VRAM: 24.5 GBHeadroom: 7.5 GBTTFT: noticeableollama run qwen3:30b64tok/sEstimated
ollama run qwen3:30bQuant: Q4_K_MContext: 8,192VRAM: 23.9 GBHeadroom: 8.1 GBTTFT: noticeableollama run qwen2.5-coder:32b60tok/sEstimated
ollama run qwen2.5-coder:32bQuant: Q4_K_MContext: 2,048VRAM: 25.8 GBHeadroom: 6.2 GBTTFT: noticeableollama run qwen3:32b60tok/sEstimated
ollama run qwen3:32bQuant: Q4_K_MContext: 2,048VRAM: 24.6 GBHeadroom: 7.4 GBTTFT: noticeableollama run gemma4:31b62tok/sEstimated
ollama run gemma4:31bQuant: Q8_0Context: 8,192VRAM: 14.4 GBHeadroom: 17.6 GBTTFT: fastollama run qwen3:8b137tok/sEstimated
ollama run qwen3:8bQuant: Q4_K_MContext: 2,048VRAM: 25.8 GBHeadroom: 6.2 GBTTFT: noticeableollama run deepseek-r1:32b60tok/sEstimated
ollama run deepseek-r1:32bQuant: Q8_0Context: 8,192VRAM: 24.6 GBHeadroom: 7.4 GBTTFT: fastollama run qwen3:14b78tok/sEstimated
ollama run qwen3:14bQuant: Q8_0Context: 8,192VRAM: 6.9 GBHeadroom: 25.1 GBTTFT: instantollama run llama3.2:3b365tok/sEstimated
ollama run llama3.2:3bQuant: Q4_K_MContext: 8,192VRAM: 3.6 GBHeadroom: 28.4 GBTTFT: instant1135tok/sEstimated
Quant: Q8_0Context: 8,192VRAM: 10.8 GBHeadroom: 21.2 GBTTFT: fastollama run qwen2.5:7b157tok/sEstimated
ollama run qwen2.5:7bRuns with tradeoffs22 models
Tight VRAM, partial CPU offload, or context-limited.
Quant: Q5_K_MContext: 8,192VRAM: 53.8 GBHeadroom: 16.6 GBTTFT: noticeable- • Partial CPU offload: ~41% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run llama3.3:70b2tok/sEstimated
- • Partial CPU offload: ~41% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run llama3.3:70bQuant: Q4_K_MContext: 2,048VRAM: 52.6 GBHeadroom: 17.8 GBTTFT: noticeable- • Partial CPU offload: ~39% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run deepseek-r1:70b2tok/sEstimated
- • Partial CPU offload: ~39% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run deepseek-r1:70bQuant: Q4_K_MContext: 8,192VRAM: 28.5 GBHeadroom: 3.5 GBTTFT: fast- • Tight VRAM fit — only 3.5 GB headroom left for context growth
ollama run mistral-small:24b80tok/sEstimated
- • Tight VRAM fit — only 3.5 GB headroom left for context growth
ollama run mistral-small:24bQuant: Q4_K_MContext: 2,048VRAM: 52.6 GBHeadroom: 17.8 GBTTFT: noticeable- • Partial CPU offload: ~39% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run llama3.1:70b2tok/sEstimated
- • Partial CPU offload: ~39% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run llama3.1:70bQuant: Q4_K_MContext: 2,048VRAM: 52.6 GBHeadroom: 17.8 GBTTFT: noticeable- • Partial CPU offload: ~39% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run nemotron:70b2tok/sEstimated
- • Partial CPU offload: ~39% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run nemotron:70bQuant: Q4_K_MContext: 2,048VRAM: 53.9 GBHeadroom: 16.5 GBTTFT: noticeable- • Partial CPU offload: ~41% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run qwen2.5:72b2tok/sEstimated
- • Partial CPU offload: ~41% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run qwen2.5:72bQuant: Q4_K_MContext: 2,048VRAM: 52.6 GBHeadroom: 17.8 GBTTFT: noticeable- • Partial CPU offload: ~39% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run hermes3:70b2tok/sEstimated
- • Partial CPU offload: ~39% of layers run on CPU
- • CPU is the bottleneck — upgrading RAM bandwidth helps more than VRAM here
ollama run hermes3:70bQuant: Q4_K_MContext: 8,192VRAM: 28.5 GBHeadroom: 3.5 GBTTFT: fast- • Tight VRAM fit — only 3.5 GB headroom left for context growth
ollama run dolphin-mistral:24b80tok/sEstimated
- • Tight VRAM fit — only 3.5 GB headroom left for context growth
ollama run dolphin-mistral:24bWhat if you upgraded?
Hypothetical scenarios. We re-ran the compatibility engine for each.
+32 GB system RAM
Check the current price
Adds 32 GB to your CPU-offload working set. Helps when models don't quite fit in VRAM.
Unlocks: 25 new tradeoff
- • Llama 4 Scout
- • Llama 3.3 70B Instruct
- • DeepSeek R1 Distill Llama 70B
- • Mistral Small 3 24B
Upgrade to NVIDIA A100 40GB
Launch MSRP $11,000 (2020). Check the current price.
40 GB VRAM (vs your 32 GB) plus a bandwidth jump from ~1792 GB/s to ~? GB/s.
Unlocks: 6 new comfortable
- • Mistral Small 3 24B
- • Dolphin 3.0 Mistral 24B
- • Gemma 4 Turkish 26B (4B active)
- • Command R 35B
Add a second NVIDIA GeForce RTX 5090
Launch MSRP $1,999 (2025). Check the current price.
Tensor parallelism splits the model across both cards, effectively doubling VRAM. Bandwidth doesn't double — runs ~1.5× the single-card speed in practice.
Unlocks: 18 new comfortable
- • Llama 3.3 70B Instruct
- • DeepSeek R1 Distill Llama 70B
- • Mistral Small 3 24B
- • Llama 3.1 70B Instruct
Some links above are affiliate links. We may earn a commission at no extra cost to you. How we make money.
Won't runtop 5 popular models
Need more memory than you have. Shown for orientation.
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
—
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
—
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
—
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
—
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
—
Even with CPU offload, needs more memory than your VRAM (32 GB) + 60% of system RAM (38 GB) combined.
Not enough VRAM for these models? Rent a GPU by the hour
The models above need more memory than 32 GB VRAM + 64 GB system RAM. You can rent a GPU with enough VRAM by the hour to run one of them, or to test it before you buy hardware. For daily use, owning a card usually costs less over time.
RunPod
billed per GPU-hourHourly GPU pods, in community-cloud and secure-cloud tiers.
The two tiers run on different hosts and terms. Check which tier a pod is on before a long job.
Rent on RunPodVast.ai
billed per GPU-hourMarketplace of host-listed GPUs; price and reliability vary by host.
Host quality varies. Filter for verified hosts and check the host's reliability before a long run.
Rent on Vast.aiReferral links: RunLocalAI may earn a commission or account credits if you sign up and rent, at no extra cost to you. Hourly rates and which GPUs are listed change all the time, so check the provider's page before you start. Editorial opinions are independent of the referral relationship; see how we make money.
How to read these numbers
Want a specific benchmark we don't have? Email Contact support and we'll prioritize it.