What can NVIDIA GB200 NVL72 run locally?

166 models fit comfortably on NVIDIA GB200 NVL72 with 32 GB system RAM. 0 more run with tradeoffs. Top pick: DeepSeek V4 Flash (284B MoE).

What's the recommended runner for this hardware?

llama.cpp / Ollama (CUDA). This is the runner that gives best tokens-per-second on NVIDIA GB200 NVL72 based on windows support and the GPU backend.

What can NVIDIA GB200 NVL72 run for long context?

Build: NVIDIA GB200 NVL72 + — + 32 GB RAM (windows)

Memory: 13824 GB VRAM + 32 GB system RAM

Runner: llama.cpp / Ollama (CUDA)

Any Chat Coding Agents Reasoning Vision Long context Creative

Runs comfortably
166 models

Ranked by fit for long context use case + predicted speed. Click a row for VRAM breakdown.

#1DeepSeek V4 Flash (284B MoE)

284B

deepseek

Commercial OK

Quant: Q5_K_MContext: 8,192VRAM: 357.0 GBHeadroom: 13467.0 GB

582

tok/s

Weights

195.25 GB

KV cache

142.00 GB

Activations

17.95 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#2DeepSeek V4 Pro (1.6T MoE)

1600B

deepseek

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 1824.3 GBHeadroom: 11999.7 GB

176

tok/s

Weights

966.00 GB

KV cache

800.00 GB

Activations

56.49 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#3Nemotron 3 Nano (30B-A3B)

30B

other

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 58.5 GBHeadroom: 13765.5 GB

ollama run nemotron3:nano

163

tok/s

Weights

31.88 GB

KV cache

15.00 GB

Activations

9.79 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#4Nemotron 3 Super (120B-A12B)

120B

other

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 146.1 GBHeadroom: 13677.9 GB

ollama run nemotron3:super

tok/s

Weights

72.45 GB

KV cache

60.00 GB

Activations

11.81 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#5Phi-3.5 Mini Instruct

3.8B

phi

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 16.1 GBHeadroom: 13807.9 GB

ollama run phi3.5:3.8b

1288

tok/s

Weights

4.04 GB

KV cache

1.90 GB

Activations

8.39 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#6Gemma 4 E4B (Effective 4B)

gemma

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 16.5 GBHeadroom: 13807.5 GB

ollama run gemma4:e4b

1224

tok/s

Weights

4.25 GB

KV cache

2.00 GB

Activations

8.40 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#7Qwen 2.5 7B Instruct

qwen

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 18.3 GBHeadroom: 13805.7 GB

ollama run qwen2.5:7b

699

tok/s

Weights

7.44 GB

KV cache

0.47 GB

Activations

8.56 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#8Qwen 3 8B

qwen

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 22.9 GBHeadroom: 13801.1 GB

ollama run qwen3:8b

612

tok/s

Weights

8.50 GB

KV cache

4.00 GB

Activations

8.62 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#9Qwen 3.5 235B-A17B (MoE)

397B

qwen

Commercial OK

Quant: Q4_K_MContext: 8,192VRAM: 460.2 GBHeadroom: 13363.8 GB

507

tok/s

Weights

239.69 GB

KV cache

198.50 GB

Activations

20.18 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#10Mistral Nemo 12B Instruct

12B

mistral

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 29.4 GBHeadroom: 13794.6 GB

ollama run mistral-nemo:12b

408

tok/s

Weights

12.75 GB

KV cache

6.00 GB

Activations

8.83 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#11Llama 3.1 8B Instruct

llama

Commercial OK

Quant: FP16Context: 8,192VRAM: 27.9 GBHeadroom: 13796.1 GB

ollama run llama3.1:8b

325

tok/s

Weights

16.00 GB

KV cache

1.07 GB

Activations

8.99 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

#12Qwen 3 14B

14B

qwen

Commercial OK

Quant: Q8_0Context: 8,192VRAM: 32.6 GBHeadroom: 13791.4 GB

ollama run qwen3:14b

350

tok/s

Weights

14.88 GB

KV cache

7.00 GB

Activations

8.94 GB

Runtime

1.80 GB

Model details →Run-on benchmark page →

What if you upgraded?

Hypothetical scenarios. We re-ran the compatibility engine for each.

+32 GB system RAM

~$80–150

Doubles your CPU-offload working set. Helps when models don't quite fit in VRAM.

Unlocks: 17 new comfortable

• Gemma 3 1B
• Llama 3.2 1B Instruct
• Gemma 4 E2B (Effective 2B)
• Whisper Large v3

Shop this upgrade↗

Add a second NVIDIA GB200 NVL72

see current pricing

Tensor parallelism splits the model across both cards, effectively doubling VRAM. Bandwidth doesn't double — runs ~1.5× the single-card speed in practice.

Unlocks: 17 new comfortable

• Gemma 3 1B
• Llama 3.2 1B Instruct
• Gemma 4 E2B (Effective 2B)
• Whisper Large v3

Shop this upgrade↗

Some links above are affiliate links. We may earn a commission at no extra cost to you. How we make money.

Won't run
top 3 popular models

Need more memory than you have. Shown for orientation.

Qwen 3.6 35B-A3B (MTP)

35B

qwen

Commercial OK

Even with CPU offload, needs more memory than your VRAM (13824 GB) + 60% of system RAM (19 GB) combined.

—

Qwen 3.6 27B (MTP)

27B

qwen

Commercial OK

Even with CPU offload, needs more memory than your VRAM (13824 GB) + 60% of system RAM (19 GB) combined.

—

Ring-2.6-1T

1000B

other

Commercial OK

Even with CPU offload, needs more memory than your VRAM (13824 GB) + 60% of system RAM (19 GB) combined.

—

How to read these numbers

Measured — we ran this exact combo on owner hardware.

Extrapolated — predicted from a measured benchmark on similar-bandwidth hardware.

Estimated — pure formula based on VRAM bandwidth and model architecture.

Full methodology →

Want a specific benchmark we don't have? Email support@runlocalai.co and we'll prioritize it.

Runs comfortably166 models

What if you upgraded?

+32 GB system RAM

Add a second NVIDIA GB200 NVL72

Won't runtop 3 popular models

How to read these numbers

Runs comfortably
166 models

Won't run
top 3 popular models