What can NVIDIA GeForce RTX 5090 Mobile run locally?

117 models fit comfortably on NVIDIA GeForce RTX 5090 Mobile with 32 GB system RAM. 32 more run with tradeoffs. Top pick: Hermes 3 Llama 3.1 8B.

What's the recommended runner for this hardware?

llama.cpp / Ollama (CUDA). This is the runner that gives best tokens-per-second on NVIDIA GeForce RTX 5090 Mobile based on windows support and the GPU backend.

What can NVIDIA GeForce RTX 5090 Mobile run for creative?

Build: NVIDIA GeForce RTX 5090 Mobile + — + 32 GB RAM (windows)

Memory: 24 GB VRAM + 32 GB system RAM

Runner: llama.cpp / Ollama (CUDA)

Any Chat Coding Agents Reasoning Vision Long context Creative

Runs comfortably
117 models

Ranked by fit for creative use case + predicted speed. Click a row for VRAM breakdown.

#1Hermes 3 Llama 3.1 8B

hermes

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 14.7 GBHeadroom: 9.3 GB

ollama run hermes3:8b

tok/s

Estimated

Weights

8.50 GB

KV cache

4.00 GB

Activations

0.43 GB

Runtime

1.80 GB

Model details →

#2Gemma 2 9B Instruct

gemma

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 16.6 GBHeadroom: 7.4 GB

ollama run gemma2:9b

tok/s

Estimated

Weights

9.80 GB

KV cache

4.50 GB

Activations

0.50 GB

Runtime

1.80 GB

Model details →

#3Dolphin 3.0 Llama 3.2 3B

dolphin

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 5.2 GBHeadroom: 18.8 GB

341

tok/s

Estimated

Weights

1.80 GB

KV cache

1.50 GB

Activations

0.10 GB

Runtime

1.80 GB

Model details →

#4Hermes 3 Llama 3.2 3B

hermes

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 5.2 GBHeadroom: 18.8 GB

341

tok/s

Estimated

Weights

1.80 GB

KV cache

1.50 GB

Activations

0.10 GB

Runtime

1.80 GB

Model details →

#5Gemma 4 E4B (Effective 4B)

gemma

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 8.4 GBHeadroom: 15.6 GB

ollama run gemma4:e4b

145

tok/s

Estimated

Weights

4.40 GB

KV cache

2.00 GB

Activations

0.23 GB

Runtime

1.80 GB

Model details →

#6Gemma 2 2B Instruct

gemma

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 4.0 GBHeadroom: 20.0 GB

511

tok/s

Estimated

Weights

1.10 GB

KV cache

1.00 GB

Activations

0.06 GB

Runtime

1.80 GB

Model details →

#7Gemma 3 4B

gemma

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 8.4 GBHeadroom: 15.6 GB

ollama run gemma3:4b

145

tok/s

Estimated

Weights

4.40 GB

KV cache

2.00 GB

Activations

0.23 GB

Runtime

1.80 GB

Model details →

#8Gemma 4 E2B (Effective 2B)

gemma

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 5.1 GBHeadroom: 18.9 GB

ollama run gemma4:e2b

291

tok/s

Estimated

Weights

2.20 GB

KV cache

1.00 GB

Activations

0.12 GB

Runtime

1.80 GB

Model details →

#9CodeGemma 7B

gemma

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 9.7 GBHeadroom: 14.3 GB

ollama run codegemma:7b

146

tok/s

Estimated

Weights

4.20 GB

KV cache

3.50 GB

Activations

0.22 GB

Runtime

1.80 GB

Model details →

#10Turkish Gemma 9B T1

gemma

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 11.6 GBHeadroom: 12.4 GB

114

tok/s

Estimated

Weights

5.00 GB

KV cache

4.50 GB

Activations

0.26 GB

Runtime

1.80 GB

Model details →

#11YTU Turkish Gemma 9B v0.1

9.2B

gemma

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 12.5 GBHeadroom: 11.5 GB

ollama run alibayram/turkish-gemma-9b-v0.1:latest

111

tok/s

Estimated

Weights

5.80 GB

KV cache

4.60 GB

Activations

0.30 GB

Runtime

1.80 GB

Model details →

#12PaliGemma 2 3B

gemma

Commercial OK

Quant: BF16Context: 8,192VRAM: 9.6 GBHeadroom: 14.4 GB

103

tok/s

Estimated

Weights

6.00 GB

KV cache

1.50 GB

Activations

0.31 GB

Runtime

1.80 GB

Model details →

Runs with tradeoffs
32 models

Tight VRAM, partial CPU offload, or context-limited.

Jamba 1.5 Mini

52B

other

Commercial OK

Quant: Q4_K_MContext: 2,048VRAM: 39.8 GBHeadroom: 3.4 GB

• Partial CPU offload: ~40% of layers run on CPU

tok/s

Estimated

Weights

30.00 GB

KV cache

6.50 GB

Activations

1.50 GB

Runtime

1.80 GB

Model details →

Gemma 4 26B MoE

26B

gemma

Commercial OK

Quant: Q4_K_MContext: 2,048VRAM: 21.9 GBHeadroom: 2.1 GB

• Tight VRAM fit — only 2.1 GB headroom left for context growth

ollama run gemma4:26b-moe

tok/s

Estimated

Weights

16.00 GB

KV cache

3.25 GB

Activations

0.80 GB

Runtime

1.80 GB

Model details →

Gemma 4 Turkish 26B (4B active)

26B

gemma

Commercial OK

Quant: Q4_K_MContext: 2,048VRAM: 20.1 GBHeadroom: 3.9 GB

• Tight VRAM fit — only 3.9 GB headroom left for context growth

tok/s

Estimated

Weights

14.30 GB

KV cache

3.25 GB

Activations

0.72 GB

Runtime

1.80 GB

Model details →

Gemma 3 27B

27B

gemma

Commercial OK

Quant: Q4_K_MContext: 2,048VRAM: 22.0 GBHeadroom: 2.0 GB

• Tight VRAM fit — only 2.0 GB headroom left for context growth

ollama run gemma3:27b

tok/s

Estimated

Weights

16.00 GB

KV cache

3.38 GB

Activations

0.80 GB

Runtime

1.80 GB

Model details →

MedGemma 27B

27B

gemma

Quant: Q4_K_MContext: 2,048VRAM: 22.0 GBHeadroom: 2.0 GB

• Tight VRAM fit — only 2.0 GB headroom left for context growth

tok/s

Estimated

Weights

16.00 GB

KV cache

3.38 GB

Activations

0.80 GB

Runtime

1.80 GB

Model details →

Llama 3.2 11B Vision Instruct

11B

llama

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 20.4 GBHeadroom: 3.6 GB

• Tight VRAM fit — only 3.6 GB headroom left for context growth

ollama run llama3.2-vision:11b

tok/s

Estimated

Weights

12.50 GB

KV cache

5.50 GB

Activations

0.63 GB

Runtime

1.80 GB

Model details →

Gemma 4 31B Dense

31B

gemma

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 36.2 GBHeadroom: 7.0 GB

• Partial CPU offload: ~34% of layers run on CPU

ollama run gemma4:31b

tok/s

Estimated

Weights

18.00 GB

KV cache

15.50 GB

Activations

0.91 GB

Runtime

1.80 GB

Model details →

PaliGemma 2 10B

10B

gemma

Commercial OK

Quant: BF16Context: 8,192VRAM: 27.8 GBHeadroom: 15.4 GB

• Partial CPU offload: ~14% of layers run on CPU

tok/s

Estimated

Weights

20.00 GB

KV cache

5.00 GB

Activations

1.01 GB

Runtime

1.80 GB

Model details →

What if you upgraded?

Hypothetical scenarios. We re-ran the compatibility engine for each.

+32 GB system RAM

Check the current price

Adds 32 GB to your CPU-offload working set. Helps when models don't quite fit in VRAM.

Unlocks: 13 new comfortable, 48 new tradeoff

• Qwen 3 0.6B
• Gemma 3 270M
• SmolLM2 135M Instruct
• TinyLlama 1.1B Chat v1.0

Shop this upgrade↗

Upgrade to NVIDIA GeForce RTX 5090

Launch MSRP $1,999 (2025). Check the current price.

32 GB VRAM (vs your 24 GB) plus a bandwidth jump from ~? GB/s to ~1792 GB/s.

Unlocks: 37 new comfortable

• Qwen 3 0.6B
• Qwen 3 30B-A3B
• Qwen 2.5 Coder 32B Instruct
• Qwen 3 32B

Shop this upgrade↗

Add a second NVIDIA GeForce RTX 5090 Mobile

Check the current price

Tensor parallelism splits the model across both cards, effectively doubling VRAM. Bandwidth doesn't double — runs ~1.5× the single-card speed in practice.

Unlocks: 45 new comfortable

• Qwen 3 0.6B
• Qwen 3 30B-A3B
• Qwen 2.5 Coder 32B Instruct
• Qwen 3 32B

Shop this upgrade↗

Some links above are affiliate links. We may earn a commission at no extra cost to you. How we make money.

Won't run
top 5 popular models

Need more memory than you have. Shown for orientation.

DeepSeek V4 Pro (1.6T MoE)

1600B

deepseek

Commercial OK