Custom build engine

Describe your build — any GPUs, CPU, RAM, OS, runtime, use case. We'll compute effective VRAM honestly, recommend a runtime, and tell you which models fit comfortably, which are borderline, and which aren't practical.

Total VRAM ≠ pooled VRAM. We never sum VRAM unless the silicon truly pools (Apple unified memory). We always explain why effective is lower than total.

Calculations follow the RunLocalAI Will-It-Run Framework: effective VRAM, model working set, runtime constraints, fit tiers, and measured-vs-estimated evidence labels.

Describe your build

Add GPUs, set CPU/RAM/OS, optionally pick a runtime + use case. URL updates as you change fields — share a build by copying the URL.

GPUs in your build

CPU class

System RAM (GB)

Use case

Your skill level

Runtime preference

Interconnect (force override)

Split strategy (force override)

Multi-node deployment

Allow CPU offload as fallback

Build summary

Total VRAM

48 GB

Effective VRAM

~33 GB

range 28-36 GB

Topology

mixed gpu

pcie

Setup difficulty

advanced

speed penalty ~35%

Why effective VRAM is lower than total

Mixed-GPU (asymmetric) configuration. Tensor-parallel doesn't work cleanly because TP requires identical cards — your faster card stalls waiting on the slower one every layer. Use llama.cpp's layer-split with manual --tensor-split tuning to distribute layers by VRAM ratio. Effective capacity ~33 GB after layer-split overhead, but the slowest card (22 GB effective) bottlenecks single-tensor operations.

Measured evidence on this hardware

Publicly inspectable measured rows for the selected hardware slug(s). Exact measured rows calibrate the fit table instead of leaving it as pure VRAM estimation.

No publicly inspectable benchmark rows are attached to this exact hardware yet. The engine will still calculate fit and runtime, but speed rows will remain estimated.

Recommended runtime

Best engine for this topology + skill level + use case.

llama.cpp (layer-split)

primary

involved

Mixed-GPU configurations need llama.cpp's --tensor-split flag with manual ratio tuning by VRAM. vLLM's tensor-parallel requires identical cards and won't run cleanly here.

Ollama

alternative

moderate

Inherits llama.cpp's layer-split path with friendlier UX. OLLAMA_GPU_OVERHEAD and per-card env vars do most of what manual flags do.

WORKLOAD PROFILE

FITS

Qwen 2.5 Coder 32B Instruct @ Q4_K_M, 8K context on NVIDIA GeForce RTX 4090

0 GB33 GBVRAM ceiling

Weights19 GB

KV cache2.1 GB

Activations1.0 GB

Runtime1.8 GB

Headroom9.1 GB

ESTIMATED DECODE RATE

29 tok/s

Bandwidth-derived estimate · efficiency 0.55. Real-world rates land within ±20% on well-tuned runtimes.

Models that fit your build

177 models considered. Categorized by headroom at the recommended quant + a sensible context for your use case.

Comfortable

24 models · ≥15% headroom

Model	Params	Quant	VRAM est.	Context	Evidence	Note
Qwen 2.5 Coder 32B Instruct	32B	Q4_K_M	22.1 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 33% headroom.
Mistral Small 3 24B	24B	Q4_K_M	26.7 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 19% headroom.
Dolphin 3.0 Mistral 24B	24B	Q4_K_M	26.7 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 19% headroom.
Codestral 22B	22B	Q4_K_M	24.7 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 25% headroom.
Gemma 4 Turkish 26B (4B active)	20B	Q4_K_M	25.2 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 24% headroom.
DeepSeek Coder V2 Lite (16B)	16B	Q4_K_M	18 GB	8,192	No measured row yet	Comfortable fit with 46% headroom — room to extend context or run alongside other workloads.
DeepSeek MoE 16B Base	16B	Q4_K_M	14 GB	4,096	No measured row yet	Comfortable fit with 58% headroom — room to extend context or run alongside other workloads.
StarCoder 2 15B	16B	Q4_K_M	17.4 GB	8,192	No measured row yet	Comfortable fit with 47% headroom — room to extend context or run alongside other workloads.
DeepSeek V2 Lite Chat	16B	Q4_K_M	16.9 GB	8,192	No measured row yet	Comfortable fit with 49% headroom — room to extend context or run alongside other workloads.
DeepSeek R1 Distill Qwen 14B	14B	Q4_K_M	15.8 GB	8,192	No measured row yet	Comfortable fit with 52% headroom — room to extend context or run alongside other workloads.
Qwen 2.5 Coder 14B Instruct	14B	Q4_K_M	15.8 GB	8,192	No measured row yet	Comfortable fit with 52% headroom — room to extend context or run alongside other workloads.
Qwen 2.5 14B Instruct	14B	Q8_0	23.5 GB	8,192	No measured row yet	Fits cleanly at Q8_0 + 8,192 ctx with 29% headroom.
Qwen 3 14B	14B	Q8_0	22.8 GB	8,192	No measured row yet	Fits cleanly at Q8_0 + 8,192 ctx with 31% headroom.
Phi-4 Reasoning 14B	14B	Q4_K_M	18.7 GB	8,192	No measured row yet	Comfortable fit with 43% headroom — room to extend context or run alongside other workloads.
Phi-4 14B	14B	Q8_0	22.8 GB	8,192	No measured row yet	Fits cleanly at Q8_0 + 8,192 ctx with 31% headroom.
GLM-4V 9B	14B	INT4	16.2 GB	8,192	No measured row yet	Comfortable fit with 51% headroom — room to extend context or run alongside other workloads.
OLMo 2 13B	14B	Q4_K_M	11.6 GB	4,096	No measured row yet	Comfortable fit with 65% headroom — room to extend context or run alongside other workloads.
Stable LM 2 12B	12B	Q4_K_M	10.6 GB	4,096	No measured row yet	Comfortable fit with 68% headroom — room to extend context or run alongside other workloads.
Gemma 3 12B	12B	Q8_0	19.7 GB	8,192	No measured row yet	Comfortable fit with 40% headroom — room to extend context or run alongside other workloads.
Mistral Nemo 12B Instruct	12B	Q8_0	19.7 GB	8,192	No measured row yet	Comfortable fit with 40% headroom — room to extend context or run alongside other workloads.
Pixtral 12B	12B	Q8_0	19.7 GB	8,192	No measured row yet	Comfortable fit with 40% headroom — room to extend context or run alongside other workloads.
Trendyol LLM Asure 12B	12B	Q4_K_M	10.9 GB	8,192	No measured row yet	Comfortable fit with 67% headroom — room to extend context or run alongside other workloads.
Llama 3.2 11B Vision Instruct	11B	Q8_0	18.6 GB	8,192	No measured row yet	Comfortable fit with 44% headroom — room to extend context or run alongside other workloads.
Llama 3.2 11B Vision	11B	Q4_K_M	13.7 GB	8,192	No measured row yet	Comfortable fit with 58% headroom — room to extend context or run alongside other workloads.

Borderline

7 models · tight, may need quant downgrade

Model	Params	Quant	VRAM est.	Context	Evidence	Note
Yi 1.5 34B	34B	Q4_K_M	29.5 GB	4,096	No measured row yet	Tight fit at Q4_K_M — only 11% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Aya Expanse 32B	32B	AWQ-INT4	32.8 GB	8,192	No measured row yet	Tight fit at AWQ-INT4 — only 1% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Qwen3.8 27B	27B	Q4_K_M	30.8 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 7% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Qwen 3.6 27B (MTP)	27B	Q5_K_M	33 GB	8,192	No measured row yet	Tight fit at Q5_K_M — only 0% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Gemma 3 27B	27B	Q4_K_M	30.3 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 8% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
MedGemma 27B	27B	Q4_K_M	30.3 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 8% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
InternVL 2.5 26B	26B	Q4_K_M	29.8 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 10% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.

Not practical

16 models · oversize for this build

Model	Params	Quant	VRAM est.	Context	Evidence	Note
Qwen 3 30B-A3B	30B	Q4_K_M	33.9 GB	8,192	No measured row yet	~33.9 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 3%. Drop quant or move to a larger build.
Gemma 4 31B Dense	31B	Q4_K_M	34.4 GB	8,192	No measured row yet	~34.4 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 4%. Drop quant or move to a larger build.
Omni 31B Turkish Reasoning	31B	Q4_K_M	33.5 GB	8,192	No measured row yet	~33.5 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 1%. Drop quant or move to a larger build.
Qwen 2.5 32B Instruct	32B	Q4_K_M	36 GB	8,192	No measured row yet	~36.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 9%. Drop quant or move to a larger build.
Qwen 3 32B	32B	Q4_K_M	36 GB	8,192	No measured row yet	~36.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 9%. Drop quant or move to a larger build.
QwQ 32B Preview	32B	Q4_K_M	36 GB	8,192	No measured row yet	~36.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 9%. Drop quant or move to a larger build.
DeepSeek R1 Distill Qwen 32B	32B	Q4_K_M	36 GB	8,192	No measured row yet	~36.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 9%. Drop quant or move to a larger build.
EXAONE 3.5 32B	32B	AWQ-INT4	36 GB	8,192	No measured row yet	~36.0 GB needed at AWQ-INT4 + 8,192 ctx — overshoots effective VRAM by 9%. Drop quant or move to a larger build.
Phind CodeLlama 34B v2	34B	Q4_K_M	38 GB	8,192	No measured row yet	~38.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 15%. Drop quant or move to a larger build.
Command R 35B	35B	Q4_K_M	39.6 GB	8,192	No measured row yet	~39.6 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 20%. Drop quant or move to a larger build.
Aya 23 35B	35B	Q4_K_M	39.6 GB	8,192	No measured row yet	~39.6 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 20%. Drop quant or move to a larger build.
Mixtral 8x7B Instruct	47B	Q4_K_M	52.9 GB	8,192	No measured row yet	~52.9 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 60%. Drop quant or move to a larger build.
Jamba 1.5 Mini	52B	Q4_K_M	58.6 GB	8,192	No measured row yet	~58.6 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 77%. Drop quant or move to a larger build.
DeepSeek R1 Distill Llama 70B	70B	Q4_K_M	77 GB	8,192	No measured row yet	~77.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 133%. Drop quant or move to a larger build.
Llama 3.1 70B Instruct	70B	Q4_K_M	77 GB	8,192	No measured row yet	~77.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 133%. Drop quant or move to a larger build.
Llama 3.3 70B Instruct	70B	Q4_K_M	44.7 GB	8,192	No measured row yet	~44.7 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 35%. Drop quant or move to a larger build.

Rent instead

Not enough VRAM for this model? Rent a GPU by the hour

This build can't hold Qwen 3 30B-A3B. By our estimate, a rented GPU with at least 34 GB runs it now, without buying new hardware.

Referral links: RunLocalAI may earn a commission or account credits if you sign up and rent, at no extra cost to you. Hourly rates and which GPUs are listed change all the time, so check the provider's page before you start. How we make money.

Multi-GPU buying guide →

NVLink vs PCIe, tensor- vs pipeline-parallel, mixed-card honesty.

Hardware combinations →

Curated multi-GPU / cluster setups with effective-VRAM math.

Setup path-finder →

OS + runtime install commands for your stack.

Compatibility matrix →

Runtime × OS × hardware support truth table.

Shopping a full build instead of a single card?

If you're sizing a fresh AI build (not just a card to drop into an existing system), the build-budget walkthroughs cover the whole BOM honestly: AI PC build under $1,000 or AI PC build under $2,000 cover the realistic 2026 budget tiers.

Vertical-fit shopping? AI PC for students covers the budget + portability tradeoffs; AI PC for developers covers the coding workflow specifics; AI PC for small business covers the document-RAG / always-on machine.

Form-factor first? See best laptop for local AI, best Mac for local AI, best mini PC for local AI, or best used GPU for local AI.