What can NVIDIA GeForce RTX 5080 run locally?

133 models fit comfortably on NVIDIA GeForce RTX 5080 with 32 GB system RAM. 94 more run with tradeoffs. Top pick: Qwen 3 0.6B.

What's the recommended runner for this hardware?

llama.cpp / Ollama (CUDA). This is the runner that gives best tokens-per-second on NVIDIA GeForce RTX 5080 based on windows support and the GPU backend.

What can NVIDIA GeForce RTX 5080 run for chat?

Build: NVIDIA GeForce RTX 5080 + — + 32 GB RAM (windows)

Memory: 16 GB VRAM + 32 GB system RAM

Runner: llama.cpp / Ollama (CUDA)

Any Chat Coding Agents Reasoning Vision Long context Creative

Runs comfortably
133 models

Ranked by fit for chat use case + predicted speed. Click a row for VRAM breakdown.

#1Qwen 3 0.6B

0.6B

qwen

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 2.4 GBHeadroom: 13.6 GBTTFT: instant

1723

tok/s

Estimated

Weights

0.30 GB

KV cache

0.30 GB

Activations

0.02 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~27 ms (instant)

Model details →

#2Qwen 3 8B

qwen

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 10.8 GBHeadroom: 5.2 GBTTFT: fast

ollama run qwen3:8b

129

tok/s

Estimated

Weights

4.80 GB

KV cache

4.00 GB

Activations

0.25 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~366 ms (fast)

Model details →

#3Llama 3.2 3B Instruct

llama

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 6.9 GBHeadroom: 9.1 GBTTFT: fast

ollama run llama3.2:3b

196

tok/s

Estimated

Weights

3.40 GB

KV cache

1.50 GB

Activations

0.18 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~137 ms (fast)

Model details →

#4Qwen 3 1.7B

1.7B

qwen

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 3.6 GBHeadroom: 12.4 GBTTFT: instant

608

tok/s

Estimated

Weights

0.90 GB

KV cache

0.85 GB

Activations

0.05 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~78 ms (instant)

Model details →

#5Dolphin 3.0 8B

dolphin

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 11.1 GBHeadroom: 4.9 GBTTFT: fast

ollama run dolphin3:8b

129

tok/s

Estimated

Weights

5.00 GB

KV cache

4.00 GB

Activations

0.26 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~366 ms (fast)

Model details →

#6TinyLlama 1.1B Chat v1.0

1.1B

llama

Commercial OK

Quant: Q4_K_MContext: 2,048VRAM: 2.6 GBHeadroom: 13.4 GBTTFT: instant

940

tok/s

Estimated

Weights

0.60 GB

KV cache

0.14 GB

Activations

0.03 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~50 ms (instant)

Model details →

#7Gemma 2 2B Instruct

gemma

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 4.0 GBHeadroom: 12.0 GBTTFT: instant

517

tok/s

Estimated

Weights

1.10 GB

KV cache

1.00 GB

Activations

0.06 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~91 ms (instant)

Model details →

#8Mistral 7B Instruct v0.3

mistral

Commercial OK

Quant: Q5_K_MContext: 8,192VRAM: 10.7 GBHeadroom: 5.3 GBTTFT: fast

ollama run mistral:7b

130

tok/s

Estimated

Weights

5.10 GB

KV cache

3.50 GB

Activations

0.26 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~320 ms (fast)

Model details →

#9Trendyol LLM 7B Chat v0.1

llama

Quant: Q4_K_MContext: 4,096VRAM: 7.6 GBHeadroom: 8.4 GBTTFT: fast

148

tok/s

Estimated

Weights

3.90 GB

KV cache

1.75 GB

Activations

0.20 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~320 ms (fast)

Model details →

#10Kumru 2B

2.4B

mistral

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 4.6 GBHeadroom: 11.4 GBTTFT: fast

ollama run alibayram/kumru:latest

431

tok/s

Estimated

Weights

1.50 GB

KV cache

1.20 GB

Activations

0.08 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~110 ms (fast)

Model details →Reference measurement: 443.7 tok/s at 2,048 context · ollama-local-api · settings may differ →

#11Salamandra 7B Instruct

llama

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 9.4 GBHeadroom: 6.6 GBTTFT: fast

148

tok/s

Estimated

Weights

3.90 GB

KV cache

3.50 GB

Activations

0.20 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~320 ms (fast)

Model details →

#12TinyLlama 1.1B Chat v0.3 AWQ

1.1B

other

Commercial OK

Quant: Q4_K_MContext: 2,048VRAM: 2.6 GBHeadroom: 13.4 GBTTFT: instant

940

tok/s

Estimated

Weights

0.60 GB

KV cache

0.14 GB

Activations

0.03 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~50 ms (instant)

Model details →

Runs with tradeoffs
94 models

Tight VRAM, partial CPU offload, or context-limited.

DeepSeek V2 Lite Chat

15.7B

deepseek

Commercial OK

Quant: Q4_K_MContext: 2,048VRAM: 12.8 GBHeadroom: 3.2 GBTTFT: fast

• Tight VRAM fit — only 3.2 GB headroom left for context growth

431

tok/s

Estimated

Weights

8.60 GB

KV cache

1.96 GB

Activations

0.43 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~110 ms (fast)

Model details →

Gemma 2 9B Instruct

gemma

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 12.4 GBHeadroom: 3.6 GBTTFT: fast

• Tight VRAM fit — only 3.6 GB headroom left for context growth

ollama run gemma2:9b

115

tok/s

Estimated

Weights

5.80 GB

KV cache

4.50 GB

Activations

0.30 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~411 ms (fast)

Model details →

Bielik 11B v3.0 Instruct GGUF

11B

llama

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 13.7 GBHeadroom: 2.3 GBTTFT: noticeable

• Tight VRAM fit — only 2.3 GB headroom left for context growth

tok/s

Estimated

Weights

6.10 GB

KV cache

5.50 GB

Activations

0.31 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~503 ms (noticeable)

Model details →

Mistral Nemo 12B Instruct

12B

mistral

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 15.7 GBHeadroom: 0.3 GBTTFT: noticeable

• Tight VRAM fit — only 0.3 GB headroom left for context growth

ollama run mistral-nemo:12b

tok/s

Estimated

Weights

7.50 GB

KV cache

6.00 GB

Activations

0.38 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~549 ms (noticeable)

Model details →

Gemma 3 12B

12B

gemma

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 15.5 GBHeadroom: 0.5 GBTTFT: noticeable

• Tight VRAM fit — only 0.5 GB headroom left for context growth

ollama run gemma3:12b

tok/s

Estimated

Weights

7.30 GB

KV cache

6.00 GB

Activations

0.37 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~549 ms (noticeable)

Model details →

Stable LM 2 12B

12B

other

Quant: Q4_K_MContext: 4,096VRAM: 12.4 GBHeadroom: 3.6 GBTTFT: noticeable

• Tight VRAM fit — only 3.6 GB headroom left for context growth

tok/s

Estimated

Weights

7.20 GB

KV cache

3.00 GB

Activations

0.36 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~549 ms (noticeable)

Model details →

OpenThaiGPT 1.0.0 Beta 13B Chat

13B

llama

Commercial OK

Quant: Q4_K_MContext: 4,096VRAM: 12.6 GBHeadroom: 3.4 GBTTFT: noticeable

• Tight VRAM fit — only 3.4 GB headroom left for context growth

tok/s

Estimated

Weights

7.20 GB

KV cache

3.25 GB

Activations

0.36 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~594 ms (noticeable)

Model details →

Qwen 3 14B

14B

qwen

Commercial OK

Quant: Q4_K_MContext: 2,048VRAM: 12.4 GBHeadroom: 3.6 GBTTFT: noticeable

• Tight VRAM fit — only 3.6 GB headroom left for context growth

ollama run qwen3:14b

tok/s

Estimated

Weights

8.40 GB

KV cache

1.75 GB

Activations

0.42 GB

Runtime

1.80 GB

Time to first token (prefill, 512-token prompt): ~640 ms (noticeable)

Model details →

What if you upgraded?

Hypothetical scenarios. We re-ran the compatibility engine for each.

+32 GB system RAM

~$80–150

Doubles your CPU-offload working set. Helps when models don't quite fit in VRAM.

Unlocks: 9 new comfortable, 112 new tradeoff

• Gemma 3 270M
• SmolLM2 135M Instruct
• SmolLM2 360M Instruct
• VBART Large (Turkish Summarization)

Shop this upgrade↗

Upgrade to NVIDIA RTX 2080 Ti 22GB (China-mod)

~$350

22 GB VRAM (vs your 16 GB) plus a bandwidth jump from ~960 GB/s to ~616 GB/s.

Unlocks: 36 new comfortable

• Gemma 4 12B
• Qwen3.5 9B
• Qwen 3 14B
• Gemma 3 270M

Shop this upgrade↗

Add a second NVIDIA GeForce RTX 5080

~$1199

Tensor parallelism splits the model across both cards, effectively doubling VRAM. Bandwidth doesn't double — runs ~1.5× the single-card speed in practice.

Unlocks: 83 new comfortable

• Qwen 3 30B-A3B
• Qwen 2.5 Coder 32B Instruct
• Qwen3.6 27B
• Qwen 3 32B

Shop this upgrade↗

Some links above are affiliate links. We may earn a commission at no extra cost to you. How we make money.

Won't run
top 5 popular models

Need more memory than you have. Shown for orientation.

DeepSeek V4 Pro (1.6T MoE)

1600B

deepseek

Commercial OK