Custom build engine

Describe your build — any GPUs, CPU, RAM, OS, runtime, use case. We'll compute effective VRAM honestly, recommend a runtime, and tell you which models fit comfortably, which are borderline, and which aren't practical.

Total VRAM ≠ pooled VRAM. We never sum VRAM unless the silicon truly pools (Apple unified memory). We always explain why effective is lower than total.

Calculations follow the RunLocalAI Will-It-Run Framework: effective VRAM, model working set, runtime constraints, fit tiers, and measured-vs-estimated evidence labels.

Describe your build

Add GPUs, set CPU/RAM/OS, optionally pick a runtime + use case. URL updates as you change fields — share a build by copying the URL.

GPUs in your build

CPU class

System RAM (GB)

Use case

Your skill level

Runtime preference

Build summary

Total VRAM

320 GB

Effective VRAM

~265 GB

range 250-272 GB

Topology

single node multi gpu

pcie

Setup difficulty

advanced

speed penalty ~18%

Why effective VRAM is lower than total

4× NVIDIA H100 SXM = 320 GB total VRAM, but without NVLink, cross-card bandwidth is PCIe-bound (~32 GB/s vs NVLink ~112 GB/s). With tensor-parallelism, each card holds ~1/4 of the model weights and replicates activations + KV cache. After 15% TP overhead, effective model capacity is ~265 GB. Largest single tensor on one card is ~78 GB.

Measured evidence on this hardware

Publicly inspectable measured rows for the selected hardware slug(s). Exact measured rows calibrate the fit table instead of leaving it as pure VRAM estimation.

No publicly inspectable benchmark rows are attached to this exact hardware yet. The engine will still calculate fit and runtime, but speed rows will remain estimated.

Recommended runtime

Best engine for this topology + skill level + use case.

vLLM

primary

involved

Tensor-parallel across NVLink/PCIe — works on every recent consumer + datacenter pair. AWQ-INT4 + 70B fits dual 3090 / dual 4090 cleanly.

ExLlamaV2

alternative

involved

Single-stream king. EXL2 4.0bpw + 70B fits dual 3090 with NVLink and beats vLLM on solo-user throughput.

llama.cpp

alternative

moderate

Layer-split via --tensor-split is the experimentation-friendly path. Worse throughput than vLLM but easier to debug.

WORKLOAD PROFILE

FITS

Kimi K1.5 @ AWQ-INT4, 8K context on NVIDIA H100 SXM

0 GB265 GBVRAM ceiling

Weights115 GB

KV cache100 GB

Activations5.8 GB

Runtime1.8 GB

Headroom42 GB

ESTIMATED DECODE RATE

20 tok/s

Bandwidth-derived estimate · efficiency 0.70. Real-world rates land within ±20% on well-tuned runtimes.

Models that fit your build

313 models considered. Categorized by headroom at the recommended quant + a sensible context for your use case.

Comfortable

24 models · ≥15% headroom

Model	Params	Quant	VRAM est.	Context	Evidence	Note
Kimi K1.5	200B	AWQ-INT4	220.8 GB	8,192	No measured row yet	Fits cleanly at AWQ-INT4 + 8,192 ctx with 17% headroom.
GLM-5 Pro	144B	AWQ-INT4	158.1 GB	8,192	No measured row yet	Comfortable fit with 40% headroom — room to extend context or run alongside other workloads.
WizardLM-2 8x22B	141B	Q4_K_M	158.7 GB	8,192	No measured row yet	Comfortable fit with 40% headroom — room to extend context or run alongside other workloads.
Mixtral 8x22B Instruct	141B	Q4_K_M	158.7 GB	8,192	No measured row yet	Comfortable fit with 40% headroom — room to extend context or run alongside other workloads.
DBRX Base	132B	Q4_K_M	144.8 GB	8,192	No measured row yet	Comfortable fit with 45% headroom — room to extend context or run alongside other workloads.
DBRX Instruct	132B	AWQ-INT4	144.8 GB	8,192	No measured row yet	Comfortable fit with 45% headroom — room to extend context or run alongside other workloads.
Mistral Large 2 (123B)	123B	Q4_K_M	138.2 GB	8,192	No measured row yet	Comfortable fit with 48% headroom — room to extend context or run alongside other workloads.
Nemotron 3 Super (120B-A12B)	120B	Q4_K_M	135.6 GB	8,192	No measured row yet	Comfortable fit with 49% headroom — room to extend context or run alongside other workloads.
Llama 4 Scout	109B	Q5_K_M	136.4 GB	8,192	No measured row yet	Comfortable fit with 49% headroom — room to extend context or run alongside other workloads.
Sarvam 105B	105B	Q4_K_M	113.2 GB	8,192	No measured row yet	Comfortable fit with 57% headroom — room to extend context or run alongside other workloads.
Sarvam 105B FP8	105B	Q4_K_M	113.2 GB	8,192	No measured row yet	Comfortable fit with 57% headroom — room to extend context or run alongside other workloads.
Command R+ (Aug 2024)	104B	AWQ-INT4	115 GB	8,192	No measured row yet	Comfortable fit with 57% headroom — room to extend context or run alongside other workloads.
Command R+ 104B	104B	Q4_K_M	115 GB	8,192	No measured row yet	Comfortable fit with 57% headroom — room to extend context or run alongside other workloads.
Llama 3.2 90B Vision	90B	AWQ-INT4	99.6 GB	8,192	No measured row yet	Comfortable fit with 62% headroom — room to extend context or run alongside other workloads.
Llama 3.2 90B Vision Instruct	90B	Q4_K_M	98.6 GB	8,192	No measured row yet	Comfortable fit with 63% headroom — room to extend context or run alongside other workloads.
InternVL 2.5 78B	78B	Q4_K_M	86.3 GB	8,192	No measured row yet	Comfortable fit with 67% headroom — room to extend context or run alongside other workloads.
Qwen 2.5 Math 72B	72B	Q4_K_M	61.1 GB	4,096	No measured row yet	Comfortable fit with 77% headroom — room to extend context or run alongside other workloads.
Molmo 72B	72B	Q4_K_M	61.1 GB	4,096	No measured row yet	Comfortable fit with 77% headroom — room to extend context or run alongside other workloads.
Qwen 2.5-VL 72B	72B	AWQ-INT4	80.1 GB	8,192	No measured row yet	Comfortable fit with 70% headroom — room to extend context or run alongside other workloads.
Qwen 2.5 72B Instruct	72B	Q5_K_M	87.5 GB	8,192	No measured row yet	Comfortable fit with 67% headroom — room to extend context or run alongside other workloads.
Dolphin 3 Llama 3.3 70B	70B	AWQ-INT4	77 GB	8,192	No measured row yet	Comfortable fit with 71% headroom — room to extend context or run alongside other workloads.
Tulu 3 70B	70B	Q4_K_M	77 GB	8,192	No measured row yet	Comfortable fit with 71% headroom — room to extend context or run alongside other workloads.
Hermes 3 Llama 3.1 70B	70B	Q4_K_M	77 GB	8,192	No measured row yet	Comfortable fit with 71% headroom — room to extend context or run alongside other workloads.
Hermes 4 Llama 3.3 70B	70B	AWQ-INT4	77 GB	8,192	No measured row yet	Comfortable fit with 71% headroom — room to extend context or run alongside other workloads.

Borderline

4 models · tight, may need quant downgrade

Model	Params	Quant	VRAM est.	Context	Evidence	Note
DeepSeek Coder V2 236B	236B	Q4_K_M	258.7 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 2% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
DeepSeek V2.5 236B	236B	Q4_K_M	258.7 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 2% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
K-EXAONE 236B A23B	236B	Q4_K_M	254.3 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 4% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
GLM-5	200B	Q4_K_M	226 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 15% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.

Not practical

16 models · oversize for this build

Model	Params	Quant	VRAM est.	Context	Evidence	Note
Qwen 3 235B-A22B	235B	Q4_K_M	266.6 GB	8,192	No measured row yet	~266.6 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 1%. Drop quant or move to a larger build.
Llama 3.1 Nemotron Ultra 253B	253B	Q4_K_M	277.7 GB	8,192	No measured row yet	~277.7 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 5%. Drop quant or move to a larger build.
DeepSeek V4 Flash (284B MoE)	284B	Q4_K_M	312.1 GB	8,192	No measured row yet	~312.1 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 18%. Drop quant or move to a larger build.
Hunyuan 3.0 (Hy3)	295B	Q3_K_M	298.5 GB	8,192	No measured row yet	~298.5 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 13%. Drop quant or move to a larger build.
Hunyuan Large 389B MoE	389B	Q4_K_M	425.5 GB	8,192	No measured row yet	~425.5 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 61%. Drop quant or move to a larger build.
Qwen 3.5 235B-A17B (MoE)	397B	Q4_K_M	435.8 GB	8,192	No measured row yet	~435.8 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 64%. Drop quant or move to a larger build.
Jamba 1.5 Large	398B	Q4_K_M	440.5 GB	8,192	No measured row yet	~440.5 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 66%. Drop quant or move to a larger build.
Llama 4 Maverick	400B	Q4_K_M	452 GB	8,192	No measured row yet	~452.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 71%. Drop quant or move to a larger build.
Llama 4 405B	405B	AWQ-INT4	444 GB	8,192	No measured row yet	~444.0 GB needed at AWQ-INT4 + 8,192 ctx — overshoots effective VRAM by 68%. Drop quant or move to a larger build.
MiniMax-M3	428B	Q3_K_M	433.1 GB	8,192	No measured row yet	~433.1 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 63%. Drop quant or move to a larger build.
Nemotron 3 Ultra (550B-A55B)	550B	Q3_K_M	556.5 GB	8,192	No measured row yet	~556.5 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 110%. Drop quant or move to a larger build.
DeepSeek R1 (671B reasoning)	671B	Q4_K_M	734.5 GB	8,192	No measured row yet	~734.5 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 177%. Drop quant or move to a larger build.
DeepSeek V3 (671B MoE)	671B	Q4_K_M	734.5 GB	8,192	No measured row yet	~734.5 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 177%. Drop quant or move to a larger build.
Mistral Medium 3.5 (675B MoE)	675B	Q4_K_M	744.9 GB	8,192	No measured row yet	~744.9 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 181%. Drop quant or move to a larger build.
DeepSeek V4	745B	AWQ-INT4	818.8 GB	8,192	No measured row yet	~818.8 GB needed at AWQ-INT4 + 8,192 ctx — overshoots effective VRAM by 209%. Drop quant or move to a larger build.
GLM-5.2	753B	Q3_K_M	762 GB	8,192	No measured row yet	~762.0 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 188%. Drop quant or move to a larger build.

Multi-GPU buying guide →

NVLink vs PCIe, tensor- vs pipeline-parallel, mixed-card honesty.

Hardware combinations →

Curated multi-GPU / cluster setups with effective-VRAM math.

Setup path-finder →

OS + runtime install commands for your stack.

Compatibility matrix →

Runtime × OS × hardware support truth table.

Shopping a full build instead of a single card?

If you're sizing a fresh AI build (not just a card to drop into an existing system), the build-budget walkthroughs cover the whole BOM honestly: AI PC build under $1,000 or AI PC build under $2,000 cover the realistic 2026 budget tiers.

Vertical-fit shopping? AI PC for students covers the budget + portability tradeoffs; AI PC for developers covers the coding workflow specifics; AI PC for small business covers the document-RAG / always-on machine.

Form-factor first? See best laptop for local AI, best Mac for local AI, best mini PC for local AI, or best used GPU for local AI.