Custom build engine

Describe your build — any GPUs, CPU, RAM, OS, runtime, use case. We'll compute effective VRAM honestly, recommend a runtime, and tell you which models fit comfortably, which are borderline, and which aren't practical.

Total VRAM ≠ pooled VRAM. We never sum VRAM unless the silicon truly pools (Apple unified memory). We always explain why effective is lower than total.

Calculations follow the RunLocalAI Will-It-Run Framework: effective VRAM, model working set, runtime constraints, fit tiers, and measured-vs-estimated evidence labels.

Describe your build

Add GPUs, set CPU/RAM/OS, optionally pick a runtime + use case. URL updates as you change fields — share a build by copying the URL.

GPUs in your build

CPU class

System RAM (GB)

Use case

Your skill level

Runtime preference

Build summary

Total VRAM

48 GB

Effective VRAM

~46 GB

range 42-46 GB

Topology

single gpu

none

Setup difficulty

beginner

speed penalty ~0%

Why effective VRAM is lower than total

Single NVIDIA RTX 4090 48GB (China-mod) — 48 GB VRAM minus ~1.8 GB runtime/driver overhead = ~46 GB usable for weights + KV cache + activations. The remaining uncertainty band covers OS display use and background CUDA allocations.

Measured evidence on this hardware

Publicly inspectable measured rows for the selected hardware slug(s). Exact measured rows calibrate the fit table instead of leaving it as pure VRAM estimation.

No publicly inspectable benchmark rows are attached to this exact hardware yet. The engine will still calculate fit and runtime, but speed rows will remain estimated.

Coding agents — agentic tool-call burst

Workload-specific bottleneck. Where this kind of work actually breaks first, and what to budget for.

Bottleneck: kv cache

Coding agents emit 5-15 tool calls per task. Each call carries the full agent system prompt + context. KV-cache budget for that prompt × concurrent requests is the limit. The decode side is well-served by any modern card; the prefill side bottlenecks first.

Budget for

•32K context with KV-cache room to spare (~3-4 GB on 4090 AWQ-INT4)
•Prefix cache: prefer SGLang for >5 tool calls / task
•Decode latency: aim for >40 tok/s sustained

Recommended runtime

Best engine for this topology + skill level + use case.

vLLM

primary

involved

AWQ-INT4 path fits 32B-class models on a 24 GB card with concurrent users. The production-default for self-hosted coding agents and multi-user serving.

ExLlamaV2

alternative

involved

Single-stream throughput king on consumer NVIDIA. EXL2 4.65bpw on a 4090 hits the highest tok/s in this class.

WORKLOAD PROFILE

FITS

Phind CodeLlama 34B v2 @ Q4_K_M, 8K context on NVIDIA RTX 4090 48GB (China-mod)

0 GB48 GBVRAM ceiling

Weights20 GB

KV cache17 GB

Activations1.0 GB

Runtime1.8 GB

Headroom8.2 GB

ESTIMATED DECODE RATE

35 tok/s

Bandwidth-derived estimate · efficiency 0.70. Real-world rates land within ±20% on well-tuned runtimes.

Models that fit your build

63 models considered (filtered by coding). Categorized by headroom at the recommended quant + a sensible context for your use case.

Comfortable

24 models · ≥15% headroom

Model	Params	Quant	VRAM est.	Context	Evidence	Note
Phind CodeLlama 34B v2	34B	Q4_K_M	38 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 17% headroom.
DeepSeek Coder V3	33B	AWQ-INT4	36.5 GB	8,192	No measured row yet	Fits cleanly at AWQ-INT4 + 8,192 ctx with 21% headroom.
Qwen 2.5 32B Instruct	32B	Q4_K_M	36 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 22% headroom.
DeepSeek R1 Distill Qwen 3 32B	32B	AWQ-INT4	36 GB	8,192	No measured row yet	Fits cleanly at AWQ-INT4 + 8,192 ctx with 22% headroom.
Qwen 3 Coder 32B	32B	AWQ-INT4	36 GB	8,192	No measured row yet	Fits cleanly at AWQ-INT4 + 8,192 ctx with 22% headroom.
Qwen 2.5 Coder 32B Instruct	32B	Q8_0	37.9 GB	8,192	No measured row yet	Fits cleanly at Q8_0 + 8,192 ctx with 18% headroom.
Qwen3 Swallow 32B RL v0.2	32B	Q4_K_M	34.5 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 25% headroom.
Gemma 4 31B Dense	31B	Q4_K_M	34.4 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 25% headroom.
Qwen 3 30B-A3B	30B	Q4_K_M	33.9 GB	8,192	No measured row yet	Fits cleanly at Q4_K_M + 8,192 ctx with 26% headroom.
Sarvam 30B	30B	Q4_K_M	24.8 GB	4,096	No measured row yet	Comfortable fit with 46% headroom — room to extend context or run alongside other workloads.
Devstral Small 2 24B	24B	Q4_K_M	26.7 GB	8,192	No measured row yet	Comfortable fit with 42% headroom — room to extend context or run alongside other workloads.
Codestral 22B	22B	Q8_0	35.2 GB	8,192	No measured row yet	Fits cleanly at Q8_0 + 8,192 ctx with 24% headroom.
DeepSeek V3 Lite (16B MoE)	16B	Q4_K_M	18 GB	8,192	No measured row yet	Comfortable fit with 61% headroom — room to extend context or run alongside other workloads.
DeepSeek Coder V2 Lite (16B)	16B	Q4_K_M	18 GB	8,192	No measured row yet	Comfortable fit with 61% headroom — room to extend context or run alongside other workloads.
StarCoder 2 15B	15B	Q4_K_M	17 GB	8,192	No measured row yet	Comfortable fit with 63% headroom — room to extend context or run alongside other workloads.
Qwen 2.5 14B Instruct	14B	Q8_0	23.5 GB	8,192	No measured row yet	Comfortable fit with 49% headroom — room to extend context or run alongside other workloads.
Qwen 2.5 Coder 14B Instruct	14B	Q4_K_M	15.8 GB	8,192	No measured row yet	Comfortable fit with 66% headroom — room to extend context or run alongside other workloads.
Qwen 3 14B	14B	Q8_0	22.8 GB	8,192	No measured row yet	Comfortable fit with 51% headroom — room to extend context or run alongside other workloads.
Yi Coder 9B	9B	Q4_K_M	10.2 GB	8,192	No measured row yet	Comfortable fit with 78% headroom — room to extend context or run alongside other workloads.
OpenCoder 8B	8B	Q4_K_M	13 GB	16,384	No measured row yet	Comfortable fit with 72% headroom — room to extend context or run alongside other workloads.
Llama 3.1 8B Instruct	8B	FP16	19.1 GB	16,384	No measured row yet	Comfortable fit with 59% headroom — room to extend context or run alongside other workloads.
Qwen 3 8B	8B	Q8_0	16.6 GB	16,384	No measured row yet	Comfortable fit with 64% headroom — room to extend context or run alongside other workloads.
DeepSeek R1 Distill Llama 8B	8B	Q4_K_M	13 GB	16,384	No measured row yet	Comfortable fit with 72% headroom — room to extend context or run alongside other workloads.
Gervásio 8B PTPT	8B	Q4_K_M	6.6 GB	4,096	No measured row yet	Comfortable fit with 86% headroom — room to extend context or run alongside other workloads.

Borderline

5 models · tight, may need quant downgrade

Model	Params	Quant	VRAM est.	Context	Evidence	Note
Llama 3.3 70B Instruct	70B	Q4_K_M	44.7 GB	8,192	No measured row yet	Tight fit at Q4_K_M — only 3% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Qwen 3.6 35B-A3B (MTP)	35B	Q5_K_M	42.8 GB	8,192	No measured row yet	Tight fit at Q5_K_M — only 7% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Qwen 3 32B	32B	Q5_K_M	39.1 GB	8,192	No measured row yet	Tight fit at Q5_K_M — only 15% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Qwen 3.6 27B (MTP)	27B	Q8_0	43.6 GB	8,192	No measured row yet	Tight fit at Q8_0 — only 5% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Mistral Small 3 24B	24B	Q8_0	39.3 GB	8,192	No measured row yet	Tight fit at Q8_0 — only 15% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.

Not practical

15 models · oversize for this build

Model	Params	Quant	VRAM est.	Context	Evidence	Note
Llama 3.1 70B Instruct	70B	Q4_K_M	77 GB	8,192	No measured row yet	~77.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 67%. Drop quant or move to a larger build.
Llama 4 70B	70B	AWQ-INT4	77 GB	8,192	No measured row yet	~77.0 GB needed at AWQ-INT4 + 8,192 ctx — overshoots effective VRAM by 67%. Drop quant or move to a larger build.
Sarvam 105B	105B	Q4_K_M	113.2 GB	8,192	No measured row yet	~113.2 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 146%. Drop quant or move to a larger build.
Llama 4 Scout	109B	Q4_K_M	122.8 GB	8,192	No measured row yet	~122.8 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 167%. Drop quant or move to a larger build.
Qwen 3 235B-A22B	235B	Q4_K_M	266.6 GB	8,192	No measured row yet	~266.6 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 480%. Drop quant or move to a larger build.
DeepSeek Coder V2 236B	236B	Q4_K_M	258.7 GB	8,192	No measured row yet	~258.7 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 462%. Drop quant or move to a larger build.
DeepSeek V2.5 236B	236B	Q4_K_M	258.7 GB	8,192	No measured row yet	~258.7 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 462%. Drop quant or move to a larger build.
DeepSeek V4 Flash (284B MoE)	284B	Q4_K_M	312.1 GB	8,192	No measured row yet	~312.1 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 578%. Drop quant or move to a larger build.
MiniMax-M3	428B	Q3_K_M	433.1 GB	8,192	No measured row yet	~433.1 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 842%. Drop quant or move to a larger build.
DeepSeek V4	745B	AWQ-INT4	818.8 GB	8,192	No measured row yet	~818.8 GB needed at AWQ-INT4 + 8,192 ctx — overshoots effective VRAM by 1680%. Drop quant or move to a larger build.
GLM-5.2	753B	Q3_K_M	762 GB	8,192	No measured row yet	~762.0 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 1556%. Drop quant or move to a larger build.
Kimi K2.6	1000B	Q4_K_M	1130 GB	8,192	No measured row yet	~1130.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 2357%. Drop quant or move to a larger build.
Ring-2.6-1T	1000B	Q3_K_M	1011.9 GB	8,192	No measured row yet	~1011.9 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 2100%. Drop quant or move to a larger build.
Kimi K2.7-Code	1000B	Q3_K_M	1011.9 GB	8,192	No measured row yet	~1011.9 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 2100%. Drop quant or move to a larger build.
DeepSeek V4 Pro (1.6T MoE)	1600B	Q4_K_M	1766 GB	8,192	No measured row yet	~1766.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 3739%. Drop quant or move to a larger build.

Multi-GPU buying guide →

NVLink vs PCIe, tensor- vs pipeline-parallel, mixed-card honesty.

Hardware combinations →

Curated multi-GPU / cluster setups with effective-VRAM math.

Setup path-finder →

OS + runtime install commands for your stack.

Compatibility matrix →

Runtime × OS × hardware support truth table.

Shopping a full build instead of a single card?

If you're sizing a fresh AI build (not just a card to drop into an existing system), the build-budget walkthroughs cover the whole BOM honestly: AI PC build under $1,000 or AI PC build under $2,000 cover the realistic 2026 budget tiers.

Vertical-fit shopping? AI PC for students covers the budget + portability tradeoffs; AI PC for developers covers the coding workflow specifics; AI PC for small business covers the document-RAG / always-on machine.

Form-factor first? See best laptop for local AI, best Mac for local AI, best mini PC for local AI, or best used GPU for local AI.