Gemma 4 31B Dense

Positioning

Gemma 4 31B Dense is Google's late-2025 / early-2026 mid-tier flagship — a 31B dense model that targets the same "fits a single 24-GB card at Q4" tier that the Qwen 3 30B-A3B MoE occupies, but via dense compute instead of MoE routing. Dense vs MoE at this tier is a real operator-grade choice: dense gives more predictable behavior + better edge-case handling at the cost of slower decode. The operator-grade question for our readers isn't "is Gemma 4 good?" — it's competitive — but "do you prefer dense reliability or MoE speed at the 32B-class tier?"

Strengths

• Fits a single 24-GB consumer card at Q4 (~17-18 GB), leaves comfortable headroom for 16K+ context. Single RTX 3090, RTX 4090, or RX 7900 XTX runs it natively.
• Strong multilingual + reasoning combo. Gemma 4 series leverages Google's Gemini-team training discipline. Per published Google benchmarks, Gemma 4 31B lands in the same band as Qwen 3 32B on most evals — independent third-party head-to-head data is still sparse.
• Dense architecture means predictable behavior. No MoE expert-routing surprises — every prompt activates the full 31B. For production pipelines that need consistent latency + output quality, this matters.
• Permissive Gemma license. Less restrictive than Llama 4's MAU clause but with some Google-specific restrictions (verify terms for your specific use case). Gemma license terms.
• Good tool-call support. Gemma 4's instruction tuning includes structured output + function-calling formats. Works cleanly with Aider and other agent tools at this tier.

Limitations

• Slower decode than Qwen 3 30B-A3B on the same hardware. Dense 31B activates 31B parameters per token vs MoE's ~3B. Real-world: Gemma 4 31B at ~25-40 tok/s on RTX 4090 vs Qwen 3 30B-A3B at ~80-110 tok/s. The 2-3× speed gap is the dense-vs-MoE tax.
• English bias is meaningfully stronger than Qwen's. For non-English daily-driver work, Qwen models still win.
• Knowledge cutoff is mid-2025. For current-events / recent-API workloads, augment with RAG.
• Context window practical ceiling ~32K despite spec advertising 128K — quality drops past 32K in our internal testing.
• Coding-specific evaluations trail Qwen 2.5 Coder 32B by 5-10 percentage points on HumanEval / SWE-bench Verified.

Real-world performance on RTX 4090 (24 GB)

• Q4_K_M (~18 GB): ~25-40 tok/s decode, TTFT 80-150 ms on 1K prompts. Acceptable for daily use; meaningfully slower than MoE alternatives at the same param count.
• Q5_K_M (~22 GB): ~22-32 tok/s, marginal quality bump, less context room.
• Q8_0 (~33 GB partial-offload): ~12-18 tok/s. Quality bump over Q4 is small; rarely worth the speed loss on local hardware.
• Compare with: rented A100 80GB datacenter setup runs Gemma 4 31B at ~70-90 tok/s with FP16. Local consumer hardware is 30-50% the speed.

Should you run this locally?

Yes, if you have a 24-GB consumer GPU and prefer dense-model predictability over MoE speed. Production pipelines that benefit from consistent per-token latency, RAG systems where output stability matters, and English-first daily-driver workflows — Gemma 4 31B is the right pick.

Yes, if you specifically need Gemma family behavior (Google ecosystem integration, Gemma-tuned LoRAs, family of models that includes vision Gemma variants).

No, for anyone whose primary workload is throughput-bound. Qwen 3 30B-A3B at 2-3× the decode speed is dramatically better $/throughput on the same hardware tier.

Probably not, for anyone whose primary workload is non-English (Qwen wins consistently on multilingual).

Probably not, for anyone whose primary workload is coding (Qwen 2.5 Coder 32B at the same VRAM tier wins on coding-specific evals).

How it compares

• vs Qwen 3 30B-A3B (MoE, same tier) → Qwen 3 30B-A3B wins decisively on speed (~2-3× faster decode). Gemma 4 wins on dense-architecture predictability + Google-ecosystem fit. Pick MoE for speed; pick dense for production reliability. Same hardware footprint, different operator priorities.
• vs Qwen 3 32B (dense Qwen) → similar dense architecture, similar VRAM footprint. Qwen 3 32B has better multilingual + slight edge on most reasoning benchmarks. Gemma 4 has better tool-calling + Google-ecosystem fit. Coin flip with edge to Qwen if multilingual matters.
• vs Qwen 2.5 32B Instruct (prior-gen Qwen dense) → Q2.5 32B is older but has more deployment weight. Gemma 4 is newer with marginal quality improvements. New deployments should probably pick Gemma 4 OR Qwen 3 32B; existing Q2.5 32B deployments don't need to upgrade.
• vs Llama 3.3 70B Instruct → Llama 3.3 70B at Q4 needs ~40 GB; doesn't fit single 24-GB. If you have 32GB+ tier hardware (5090, dual 3090, Mac M-Max 64+), Llama 3.3 70B leads Gemma 4 31B on most quality benchmarks (size advantage, expected). Gemma wins on accessibility — fits a card half the size at half the price.
• vs DeepSeek R1 Distill Qwen 32B → R1-Distill specializes in reasoning chain-of-thought. Gemma 4 31B is generalist. Pick R1 Distill for math + logic puzzles; pick Gemma 4 for general daily-driver work.
• vs Phi 4 14B → Phi 4 fits 16-GB cards; Gemma 4 needs 24 GB. Phi 4 wins on smaller-card accessibility; Gemma 4 wins on the workloads where 31B-class capability matters.

Run this yourself

# RTX 4090 / 3090 / 7900 XTX — single-card 24 GB
ollama pull gemma4:31b-instruct-q4_K_M
ollama run gemma4:31b-instruct-q4_K_M

# Or via llama.cpp directly:
llama-server -m gemma-4-31b-instruct-Q4_K_M.gguf \
  --ctx-size 16384 -ngl 999 --temp 0.7

# For multi-user serving via vLLM:
vllm serve google/gemma-4-31b-instruct \
  --tensor-parallel-size 1 --max-model-len 16384

Quant: Q4_K_M GGUF Context: 16384 (KV cache f16, ~2 GB additional) Backend: llama.cpp via Ollama, CUDA 12.x Hardware: RTX 4090, NVIDIA driver 555+