RUNLOCALAIv38
->Will it run?Best GPUCompareTroubleshootGet startedLearnPulseModelsHardwareToolsBench
Run check
RUNLOCALAI

Independently operated catalog for local-AI hardware and software. Hand-written verdicts. Source-cited claims. Reproducible commands when we have them.

OP·Eruo Fredoline
DIR
  • Models
  • Hardware
  • Tools
  • Benchmarks
TOOLS
  • Will it run?
  • Compare hardware
  • Cost vs cloud
  • Choose my GPU
  • Prompting kits
  • Quick answers
REF
  • All buyer guides
  • Learn local AI
  • Methodology
  • Glossary
  • Errors KB
  • Trust
  • Suggest a feature
EDITOR
  • About
  • Author
  • How we make money
  • Editorial policy
  • Contact
LEGAL
  • Privacy
  • Terms
  • Sitemap
MAIL · MONTHLY DIGEST
Get monthly local AI changes
Monthly recap. No spam.
DISCLOSURE

Some links on this site are affiliate links (Amazon Associates and other first-class retailers). When you buy through them, we earn a small commission at no extra cost to you. Affiliate links do not influence our verdicts — there are cards we rate highly that we don't have affiliate relationships with, and cards that sell well that we refuse to recommend. How we make money →

© 2026 runlocalai.coIndependently operated
RUNLOCALAI · v38
← Back to Will-it-run

Custom build engine

Describe your build — any GPUs, CPU, RAM, OS, runtime, use case. We'll compute effective VRAM honestly, recommend a runtime, and tell you which models fit comfortably, which are borderline, and which aren't practical.

Total VRAM ≠ pooled VRAM. We never sum VRAM unless the silicon truly pools (Apple unified memory). We always explain why effective is lower than total.

Calculations follow the RunLocalAI Will-It-Run Framework: effective VRAM, model working set, runtime constraints, fit tiers, and measured-vs-estimated evidence labels.

Describe your build

Add GPUs, set CPU/RAM/OS, optionally pick a runtime + use case. URL updates as you change fields — share a build by copying the URL.

Build summary

Total VRAM
24 GB
Effective VRAM
~22 GB
range 20-22 GB
Topology
single gpu
none
Setup difficulty
beginner
speed penalty ~0%
Why effective VRAM is lower than total

Single NVIDIA RTX PRO 4000 Blackwell — 24 GB VRAM minus ~1.8 GB runtime/driver overhead = ~22 GB usable for weights + KV cache + activations. The remaining uncertainty band covers OS display use and background CUDA allocations.

Measured evidence on this hardware

Publicly inspectable measured rows for the selected hardware slug(s). Exact measured rows calibrate the fit table instead of leaving it as pure VRAM estimation.

No publicly inspectable benchmark rows are attached to this exact hardware yet. The engine will still calculate fit and runtime, but speed rows will remain estimated.

Coding agents — agentic tool-call burst

Workload-specific bottleneck. Where this kind of work actually breaks first, and what to budget for.

Bottleneck: kv cache

Coding agents emit 5-15 tool calls per task. Each call carries the full agent system prompt + context. KV-cache budget for that prompt × concurrent requests is the limit. The decode side is well-served by any modern card; the prefill side bottlenecks first.

Budget for
  • •32K context with KV-cache room to spare (~3-4 GB on 4090 AWQ-INT4)
  • •Prefix cache: prefer SGLang for >5 tool calls / task
  • •Decode latency: aim for >40 tok/s sustained

Recommended runtime

Best engine for this topology + skill level + use case.

vLLM
primary
involved

AWQ-INT4 path fits 32B-class models on a 24 GB card with concurrent users. The production-default for self-hosted coding agents and multi-user serving.

ExLlamaV2
alternative
involved

Single-stream throughput king on consumer NVIDIA. EXL2 4.65bpw on a 4090 hits the highest tok/s in this class.

WORKLOAD PROFILE
FITS
Muse Glimmer 30B @ Q4_K_M, 8K context on NVIDIA RTX PRO 4000 Blackwell
0 GB24 GBVRAM ceiling
Weights17 GB
KV cache0.7 GB
Activations0.9 GB
Runtime1.8 GB
Headroom3.6 GB
ESTIMATED DECODE RATE
28 tok/s
Bandwidth-derived estimate · efficiency 0.70. Real-world rates land within ±20% on well-tuned runtimes.
28 tokens per second02550100150

Models that fit your build

86 models considered (filtered by coding). Categorized by headroom at the recommended quant + a sensible context for your use case.

Comfortable
24 models · ≥15% headroom
ModelParamsQuantVRAM est.ContextEvidenceNote
Muse Glimmer 30B30BQ4_K_M18.6 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 16% headroom.
DeepSeek V3 Lite (16B MoE)16BQ4_K_M18 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 18% headroom.
DeepSeek Coder V2 Lite (16B)16BQ4_K_M18 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 18% headroom.
StarCoder 2 15B15BQ4_K_M17 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 23% headroom.
Qwen 2.5 14B Instruct14BQ5_K_M18 GB8,192No measured row yetFits cleanly at Q5_K_M + 8,192 ctx with 18% headroom.
Qwen 2.5 Coder 14B Instruct14BQ4_K_M15.8 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 28% headroom.
Qwen 3 14B14BQ4_K_M15.8 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 28% headroom.
Ministral 3 14B14BQ4_K_M16.6 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 25% headroom.
Mellum2 12B-A2.5B12BQ4_K_M14.5 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 34% headroom.
Gemma 4 12B12BQ4_K_M14 GB8,192No measured row yetFits cleanly at Q4_K_M + 8,192 ctx with 36% headroom.
Yi Coder 9B9BQ4_K_M10.2 GB8,192No measured row yetComfortable fit with 54% headroom — room to extend context or run alongside other workloads.
Qwen3.5 9B9BQ4_K_M11.4 GB8,192No measured row yetComfortable fit with 48% headroom — room to extend context or run alongside other workloads.
Ornith 1.0 9B9BQ4_K_M10.4 GB8,192No measured row yetComfortable fit with 53% headroom — room to extend context or run alongside other workloads.
Granite 4.1 8B Instruct9BQ8_014.2 GB8,192No measured row yetFits cleanly at Q8_0 + 8,192 ctx with 35% headroom.
OpenCoder 8B8BQ4_K_M13 GB16,384No measured row yetComfortable fit with 41% headroom — room to extend context or run alongside other workloads.
Qwen 3 8B8BQ8_016.6 GB16,384No measured row yetFits cleanly at Q8_0 + 16,384 ctx with 24% headroom.
DeepSeek R1 Distill Llama 8B8BQ4_K_M13 GB16,384No measured row yetComfortable fit with 41% headroom — room to extend context or run alongside other workloads.
Gervásio 8B PTPT8BQ4_K_M6.6 GB4,096No measured row yetComfortable fit with 70% headroom — room to extend context or run alongside other workloads.
Dolphin 3.0 8B8BQ4_K_M13.3 GB16,384No measured row yetFits cleanly at Q4_K_M + 16,384 ctx with 40% headroom.
EXAONE Deep 7.8B8BQ4_K_M12.3 GB16,384No measured row yetComfortable fit with 44% headroom — room to extend context or run alongside other workloads.
Qwen 2.5 7B Instruct7BQ8_09.5 GB16,384No measured row yetComfortable fit with 57% headroom — room to extend context or run alongside other workloads.
CodeGemma 7B7BQ4_K_M7.9 GB8,192No measured row yetComfortable fit with 64% headroom — room to extend context or run alongside other workloads.
Qwen 2.5 Coder 7B Instruct7BQ6_K13.6 GB16,384No measured row yetFits cleanly at Q6_K + 16,384 ctx with 38% headroom.
Codestral Mamba 7B7BQ4_K_M11.4 GB16,384No measured row yetComfortable fit with 48% headroom — room to extend context or run alongside other workloads.
Borderline
1 models · tight, may need quant downgrade
ModelParamsQuantVRAM est.ContextEvidenceNote
Llama 3.1 8B Instruct8BFP1619.1 GB16,384No measured row yetTight fit at FP16 — only 13% headroom. KV cache for longer context will OOM. Cap context tighter or drop one quant level.
Not practical
16 models · oversize for this build
ModelParamsQuantVRAM est.ContextEvidenceNote
GPT-OSS 20B21BMXFP425.2 GB8,192No measured row yet~25.2 GB needed at MXFP4 + 8,192 ctx — overshoots effective VRAM by 14%. Drop quant or move to a larger build.
Codestral 22B22BQ4_K_M24.7 GB8,192No measured row yet~24.7 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 12%. Drop quant or move to a larger build.
Mistral Small 3 24B24BQ4_K_M26.7 GB8,192No measured row yet~26.7 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 21%. Drop quant or move to a larger build.
Devstral Small 2 24B24BQ4_K_M26.7 GB8,192No measured row yet~26.7 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 21%. Drop quant or move to a larger build.
Gemma 4 26B-A4B26BQ4_K_M29.8 GB8,192No measured row yet~29.8 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 35%. Drop quant or move to a larger build.
Qwen 3.6 27B (MTP)27BQ3_K_M27.3 GB8,192No measured row yet~27.3 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 24%. Drop quant or move to a larger build.
Qwen3.5 27B27BQ4_K_M31.4 GB8,192No measured row yet~31.4 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 43%. Drop quant or move to a larger build.
Qwen3.6 27B27BQ4_K_M31.4 GB8,192No measured row yet~31.4 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 43%. Drop quant or move to a larger build.
Granite 4.1 30B Instruct29BQ3_K_M29.2 GB8,192No measured row yet~29.2 GB needed at Q3_K_M + 8,192 ctx — overshoots effective VRAM by 33%. Drop quant or move to a larger build.
Qwen 3 30B-A3B30BQ4_K_M33.9 GB8,192No measured row yet~33.9 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 54%. Drop quant or move to a larger build.
Sarvam 30B30BQ4_K_M24.8 GB4,096No measured row yet~24.8 GB needed at Q4_K_M + 4,096 ctx — overshoots effective VRAM by 13%. Drop quant or move to a larger build.
Granite 4.1 30B30BQ4_K_M32.9 GB8,192No measured row yet~32.9 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 49%. Drop quant or move to a larger build.
Qwen3 Coder 30B-A3B30BQ4_K_M35 GB8,192No measured row yet~35.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 59%. Drop quant or move to a larger build.
North Mini Code 1.030BQ4_K_M35 GB8,192No measured row yet~35.0 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 59%. Drop quant or move to a larger build.
Gemma 4 31B Dense31BQ4_K_M34.4 GB8,192No measured row yet~34.4 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 56%. Drop quant or move to a larger build.
GLM-4.7-Flash31BQ4_K_M35.5 GB8,192No measured row yet~35.5 GB needed at Q4_K_M + 8,192 ctx — overshoots effective VRAM by 61%. Drop quant or move to a larger build.

Related

Multi-GPU buying guide →

NVLink vs PCIe, tensor- vs pipeline-parallel, mixed-card honesty.

Hardware combinations →

Curated multi-GPU / cluster setups with effective-VRAM math.

Setup path-finder →

OS + runtime install commands for your stack.

Compatibility matrix →

Runtime × OS × hardware support truth table.

Shopping a full build instead of a single card?

If you're sizing a fresh AI build (not just a card to drop into an existing system), the build-budget walkthroughs cover the whole BOM honestly: AI PC build under $1,000 or AI PC build under $2,000 cover the realistic 2026 budget tiers.

Vertical-fit shopping? AI PC for students covers the budget + portability tradeoffs; AI PC for developers covers the coding workflow specifics; AI PC for small business covers the document-RAG / always-on machine.

Form-factor first? See best laptop for local AI, best Mac for local AI, best mini PC for local AI, or best used GPU for local AI.

See something off?Submit a benchmark·Report outdated·Suggest a correctionWe read every submission. Editorial review takes 1-7 days.