Gemma

Gemma 1 (February 2024) launched as Google's first open-weight LLM family with Gemma 2B and Gemma 7B. Architecture: dense decoder using the Gemini-derived transformer with multi-query attention (MQA, 1 KV head shared across all query heads), GeGLU activation, and RoPE positional embeddings with base frequency 10,000. The 256K vocabulary (SentencePiece tokenizer) was identical to Gemini's vocabulary, establishing a clear architectural lineage. Gemma 1 was trained on 6T tokens of primarily English web data, math, and code, using knowledge distillation from a larger Gemini teacher model. Gemma 2 (June 2024) introduced several architectural improvements. The 9B and 27B models adopted a hybrid attention mechanism: alternating layers of sliding window attention (window size 4096) and full global attention, interleaved in a repeating pattern. This was architecturally similar to [Mistral 7B](/families/mistral)'s sliding window + periodic global attention design. GeGLU activation was preserved. The 27B added GQA (grouped-query attention) with 16 query heads and 16 KV heads in the global attention layers. Training data expanded to 13T tokens with stronger emphasis on code (20%), math (15%), and multilingual (10%). Gemma 3 (March 2025) scaled to 4 model sizes (1B, 4B, 12B, 27B) with a unified architecture. The smaller variants (1B, 4B) used full global attention; the larger variants (12B, 27B) used the hybrid SWA/GQA design from Gemma 2. The key innovation was multimodal capability: Gemma 3 12B and 27B accepted visual inputs through a SigLIP vision encoder with 400M parameters, projecting visual tokens into the text embedding space. This was Google's first open-weight VLM, competing with [LLaVA](/families/llava) and [InternVL](/families/internvl). Training data reached 18T tokens, with the vision component trained on 2B image-text pairs. Gemma 4 (April 2026) introduced the 31B model as the current flagship. Architecture: hybrid SWA/GQA with expanded context to 256K tokens using RoPE with extended base frequency. The SigLIP vision encoder was upgraded to SigLIP-2 with 800M parameters for improved visual understanding. Knowledge distillation from a Gemini 2.5 Ultra teacher was key to Gemma 4's quality. Training data: 25T tokens. The design philosophy is "Gemini capability at deployable scale" — Gemma is Google's bridge between closed-source Gemini research and the open-weight ecosystem.

Gemma models are among the most parameter-efficient open-weight models, designed from inception for local deployment. Google's design brief for Gemma explicitly targets on-device and single-GPU inference. Gemma 1 2B at FP16 requires ~4 GB VRAM and runs on phone-class hardware. The 2B at Q4 (~1.2 GB) runs comfortably on a [Google Tensor G4](/hardware/google-tensor-g4) or any modern smartphone. On [MacBook Pro M4 Max](/hardware/macbook-pro-16-m4-max), Gemma 1 7B at Q4 achieves 80+ tokens/second via [MLX-LM](/tools/mlx-lm) — the fastest local 7B-class model in 2024. This extreme deployability established Gemma as the "compute floor" for useful LLMs. Gemma 2 9B at Q4 (~5 GB) maintains the accessibility of Gemma 1 7B at higher quality. On [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb), ~60 tokens/second via [Ollama](/tools/ollama). The 27B at Q4 (~16 GB) fits on a single [RTX 4090](/hardware/rtx-4090) 24 GB with room to spare for KV-cache and batching. On [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) 192 GB, Gemma 2 27B at Q4 achieves ~30 tokens/second — the quality-per-watt champion. At Q4, the 27B vision model (~17 GB total including vision encoder) fits on a single [RTX 4090](/hardware/rtx-4090) 24 GB. On [Apple M3 Ultra](/hardware/apple-m3-ultra), ~15 tokens/second from the vision pipeline (image encoding adds ~200ms latency, negligible at interactive speeds). Gemma 3 was the first VLM practically deployable on consumer hardware, predating good [LLaVA](/families/llava) local support by 6 months. Gemma 4 31B at Q4 (~18 GB) just fits on [RTX 4090](/hardware/rtx-4090) 24 GB with ~5 GB headroom for context. On [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) 192 GB, Gemma 4 31B at Q4 achieves ~25 tokens/second. The 256K context with hybrid SWA/GQA creates a VRAM profile for KV-cache that scales more gently than [Llama 3.3 70B](/families/llama) at extended context — Gemma 4 31B at 128K context uses ~12 GB KV-cache vs ~64 GB for Llama 3.3 70B. The hybrid attention design's local viability advantage becomes more significant at longer contexts. The trajectory: Gemma models have maintained a consistent VRAM-per-quality-point ratio that's ~30-40% better than [Llama](/families/llama) at equivalent capability levels. Gemma is the default choice for on-device deployment (smartphone, tablet, edge device) and for single-GPU deployments where every GB of VRAM counts.

Gemma 1 7B (February 2024) scored MMLU 64.3% (5-shot), exceeding [Llama 2 13B](/families/llama) (54.8%) and close to Llama 2 70B (68.9%). HumanEval 32.3% at 5-shot was competitive. GSM8K 46.4% at 8-shot was mid-pack. MMLU scores were surprisingly strong for a 7B model trained on 6T tokens — the knowledge distillation from Gemini teacher was clearly effective. The 2B scored MMLU 42.1% and HumanEval 20.1%, setting a new floor for "usable" LLM size. This was the parameter-efficiency breakthrough: Gemma 2 27B matched the MMLU of Llama 3 70B (79.5%) within 1.4 points. The 9B scored MMLU 70.1% and GSM8K 76.3%, exceeding [Mistral 7B](/families/mistral) and [Llama 3 8B](/families/llama). On IFEval, Gemma 2 27B scored 75.8% strict-prompt, competitive with GPT-4 early checkpoints. MATH 500: 51.2% for 27B, 42.3% for 9B. Gemma 3 27B (March 2025) added vision benchmarks: MMMU 55.3% (multimodal understanding), DocVQA 87.2% (document understanding), and ChartQA 78.4%. Text benchmarks: MMLU 85.2%, GSM8K 92.1%, HumanEval 71.5%, MATH 500 72.3%. On code benchmarks, Gemma 3 27B was competitive with [DeepSeek Coder V3](/families/deepseek) at the 27B scale — LiveCodeBench 41.5%. The vision capabilities made Gemma 3 an immediate competitor to [InternVL 2.5](/families/internvl) on document understanding and chart reading. Gemma 4 31B (April 2026) is the current benchmark flagship: MMLU 88.7%, GSM8K 94.5%, HumanEval 78.2%, MATH 500 82.1%. On vision benchmarks, MMMU 62.1%, DocVQA 90.3%, ChartQA 83.5%. On LiveCodeBench, 52.8% — within striking distance of [DeepSeek V3](/families/deepseek). On GPQA Diamond, 58.7% — competitive with GPT-4o. The benchmark trajectory for Gemma is the steepest improvement curve in open-weight: each generation closes ~5-7 MMLU points and ~10-15 MATH points vs the previous. Where Gemma wins vs peers: parameter efficiency (best capability-per-parameter in open-weight at sub-35B scale) and vision-language integration (the most capable open-weight VLM per byte of model size). Where Gemma trails: Chinese-language benchmarks (trails [Qwen](/families/qwen) significantly), long-form instruction following (IFEval scores trail [Llama 3.3](/families/llama)), and very long context retrieval (despite 256K context, retrieval accuracy at distant positions trails [Llama](/families/llama) due to SWA layer tradeoffs).

Gemma runtime support is strong but distinctly Google-centric. The close relationship between Gemma and Google's infrastructure means first-class support on Google Cloud and Vertex AI, with community-driven support on other platforms. [vLLM](/tools/vllm) added Gemma support in v0.4.0 for Gemma 1 and maintained first-class support through Gemma 4. The vLLM Gemma implementation handles the hybrid SWA/GQA attention pattern correctly — the sliding-window layers use paged attention with window masking. [Ollama](/tools/ollama) provides one-command Gemma deployment: `ollama run gemma:27b`, `ollama run gemma2:9b`. Gemma models are among the most popular on Ollama, particularly the 2B (for edge devices) and 9B (for laptops). On [RTX 4090](/hardware/rtx-4090), Gemma 2 27B via Ollama at Q4 achieves ~45 tokens/second. [llama.cpp](/tools/llama-cpp) supports all Gemma generations with GGUF format. Gemma's interleaved SWA/GQA architecture requires custom attention mask generation in llama.cpp, which was added in b2380+. The hybrid attention implementation in llama.cpp has moderate optimization but functional correctness. [TensorRT-LLM](/tools/tensorrt-llm) provides the most optimized Gemma inference path on NVIDIA hardware. The Gemma TRT plugin includes fused hybrid SWA/GQA attention kernels. [MLX-LM](/tools/mlx-lm) provides excellent Gemma support on Apple Silicon, a natural fit given Google's JAX/MLX architectural similarities. MLX-LM's Gemma support includes the vision encoder for Gemma 3+ multimodal variants, enabling local image understanding on Apple Silicon. The Gemma tokenizer uses SentencePiece with a 256K vocabulary, and the `add_bos_token` / `add_eos_token` settings must be configured correctly for instruction-following quality. The standard Gemma chat template uses `<start_of_turn>user` and `<start_of_turn>model` tokens. [Text Generation Inference](/tools/text-generation-inference) supports Gemma 2+ with flash-attention optimization for both SWA and global attention layers. TGI's Gemma support addresses the attention-pattern switching correctly, a common correctness bug in early Gemma implementations. TensorFlow Lite and MediaPipe support Gemma 1 and Gemma 2 for on-device deployment via Google's official tooling. This is unique to Gemma — no other major open-weight family has first-party mobile inference support. Gemma 2B runs on Android devices via MediaPipe at ~5 tokens/second on-device.

Gemma models quantize well at standard bit widths, benefiting from the GeGLU activation's smooth distributions and the knowledge-distillation training which produces well-behaved weight matrices. GGUF via [llama.cpp](/tools/llama-cpp) provides the broadest path. Gemma 2 27B at Q4_K_M (~16 GB) loses ~0.4% MMLU vs FP16, effectively lossless. At Q3_K_M (~12 GB), ~2.0% MMLU degradation. IQ-quant variants perform well: IQ4_XS on Gemma saves ~12% model size vs Q4_K_M at equivalent quality. Gemma 4 31B at Q4_K_M (~18 GB) loses ~0.5% MMLU. At Q3_K_L (~14 GB), ~2.2% degradation. The 256K context makes IQ-quant variants more valuable for Gemma 4 because the KV-cache savings at long context compound the model-size savings. IQ3_XXS (~13 GB) achieves Q3_K_M-quality at ~20% smaller filesize. GPTQ and AWQ support is mature for Gemma 2+. Gemma 2 27B AWQ 4-bit (~16 GB) is lossless on MMLU (<0.2% degradation). The AWQ/HQQ quantization formats handle Gemma's hybrid attention correctly — the activation patterns in SWA vs global attention layers differ, and AWQ's per-channel activation-aware scaling captures this distinction. GPTQ 4-bit provides equivalent quality to AWQ for Gemma. FP8 via [TensorRT-LLM](/tools/tensorrt-llm) and [vLLM](/tools/vllm) is the standard datacenter format. Gemma 4 31B FP8 (~31 GB) is effectively lossless (<0.1% degradation all benchmarks). FP8 KV-cache quantization saves ~2× KV-cache memory. A Gemma-specific quantization consideration: the SigLIP vision encoder in Gemma 3+ multimodal variants should generally not be quantized below FP16. Quantize the text LLM, keep the vision encoder at FP16. This ensures image understanding quality is preserved while the text model benefits from quantization savings. Google provides official Gemma quantization tooling through AI Edge Torch and the Gemma model garden. The official 4-bit quantization uses int4 with per-channel scaling specifically tuned for Gemma's activation distributions. On Tensor G4 and Snapdragon 8 Gen 3 mobile platforms, the official Google quantized Gemma 2B achieves ~12 tokens/second — optimized for mobile inference where llama.cpp is not available. Practical rule: Gemma models survive Q3_K_M for casual use (~2% loss); Q4_K_M is production quality (~0.5% loss). The smaller 2B and 9B models can be aggressively quantized (Q2_K) for extreme edge deployment where quality degradation is acceptable tradeoff for fitting on 1-2 GB devices.

Gemma's ecosystem is younger than [Llama](/families/llama) or [Qwen](/families/qwen) but has benefited from explosive growth driven by Google's backing and the Gemma Terms' commercial clarity. Derivative model volume is moderate (hundreds of fine-tunes) with a distinctive Google-ecosystem flavor. Key derivatives: CodeGemma (code-specialized Gemma 2, trained on 500B code tokens with FIM), GemmaScope (interpretability tooling from Google DeepMind), and various community fine-tunes for roleplaying and creative writing. The code-specific CodeGemma 7B is the canonical Gemma derivative for software development, scoring HumanEval 44.5% and MBPP 56.2%. License clarity is good with one notable quirk. The Gemma Terms of Use permit commercial use, redistribution, derivative creation, and model modification. There is no MAU threshold. Review the Gemma Terms for your specific deployment category. The Google ecosystem integration is Gemma's unique strength. Vertex AI provides one-click Gemma deployment. Google AI Studio offers free Gemma API access for prototyping. Kaggle hosts Gemma models and fine-tuning competitions. Android Studio integrates Gemma through MediaPipe for on-device AI. TensorFlow and JAX both support Gemma training. Training data transparency is moderate. Google discloses training data composition percentages and filtering methodologies but does not release the actual datasets. This is comparable to [Llama](/families/llama) and less transparent than [Aya](/families/aya). However, Google publishes detailed technical reports and model cards that are more comprehensive than most open-weight providers. The Kaggle and Google Colab ecosystem provides free-tier Gemma fine-tuning for experimentation. The T4 GPU on Colab free tier can QLoRA fine-tune Gemma 2 2B in ~4 hours, making Gemma the most accessible family for learning fine-tuning on free cloud resources. This educational advantage creates a growing community of developers whose first fine-tuning experience is on Gemma. Ecosystem gaps: Gemma's agentic and tool-use framework integration trails [Llama](/families/llama). Google's agentic frameworks (Project Mariner, Vertex AI Agent Builder) primarily target Gemini, not Gemma. Community agent frameworks (LangChain, CrewAI) have better Llama integration than Gemma integration. This gap is narrowing as Gemma 4's tool-use capability improves.

Gemma fine-tuning benefits from Google's infrastructure and official fine-tuning recipes. The combination of manageable model sizes (2B-31B) and Google-provided tooling makes Gemma fine-tuning accessible. LoRA and QLoRA support is first-class. HuggingFace PEFT supports Gemma with the `gemma` and `gemma2` architecture tags. For Gemma 2 9B, QLoRA rank 64 with 4-bit base model on [RTX 4090](/hardware/rtx-4090) uses ~20 GB VRAM and trains 10K samples in ~3.5 hours. For Gemma 4 31B, QLoRA rank 32 with 4-bit base on [RTX 4090](/hardware/rtx-4090) uses ~23 GB VRAM (tight) and trains 10K samples in ~10 hours. Full fine-tuning: Gemma 2 9B needs ~72 GB VRAM (1× [A100 80GB](/hardware/nvidia-a100-80gb-sxm)). Gemma 4 31B needs ~250 GB VRAM (4× [A100 80GB](/hardware/nvidia-a100-80gb-sxm) with ZeRO-3). Full fine-tuning at 31B is datacenter-scale. Google provides official fine-tuning infrastructure: Vertex AI fine-tuning jobs for Gemma, Google Colab free-tier T4 for Gemma 2B/7B QLoRA, and Kaggle notebooks with free GPU access. The official Google fine-tuning recipes are documented in the Gemma model garden with hyperparameter sweeps published for each model size. The dataset ecosystem includes all standard instruction datasets. Google provides a Gemma-specific chat template: `<start_of_turn>user {text}<end_of_turn> <start_of_turn>model `. Using the wrong template is the most common Gemma fine-tuning error — Llama templates produce ~20% instruction-following degradation on Gemma. Axolotl has pre-built Gemma configs with the correct chat template and LoRA targets. [Unsloth](/tools/unsloth) does not support Gemma as of mid-2026. The Axolotl configs correctly identify LoRA target modules for Gemma: the Q/K/V/O projections in global attention layers benefit most from adaptation. DPO alignment for Gemma is documented through Google's official alignment recipes. The Gemma Instruct models use RLHF with a reward model trained on preference data. Community DPO on Gemma 2 9B produces measurable preference alignment improvements comparable to [Mistral](/families/mistral) DPO results. Vision fine-tuning for Gemma 3+ multimodal variants is possible via HuggingFace PEFT with vision encoder LoRA. The SigLIP vision encoder can be partially fine-tuned (LoRA on projection layer only) while keeping the LLM frozen or fine-tuned, enabling domain-specific visual understanding fine-tunes.

Gemma's Terms of Use are broadly permissive for commercial use but include a "Prohibited Uses" section that enumerates restricted activities. The high-risk AI system restriction is the most operationally meaningful — if your deployment falls under EU AI Act high-risk categories, consult the Gemma Terms for required safeguards. On-device deployment via MediaPipe is the most unique Gemma deployment path. Google's official Gemma 2B MediaPipe integration runs on Android, iOS, and web (WebGPU). This enables offline, private LLM inference on smartphones. Latency on Pixel 9 (Tensor G4) for Gemma 2B: ~200ms time-to-first-token, ~8 tokens/second decode. The hybrid SWA/GQA attention has a context-quality tradeoff. Gemma 2+ uses alternating SWA and global attention layers. This produces measurable retrieval-quality degradation (2-4% on Needle-in-Haystack at 128K vs 32K). For RAG deployments on Gemma 4 at 256K context, place critical documents within the first 64K tokens. Chat template specificity is critical. Gemma uses `<start_of_turn>user` and `<start_of_turn>model` tokens, with `<end_of_turn>` terminating each turn. Using Llama or ChatML templates degrades IFEval scores by 15-20%. [Ollama](/tools/ollama) and [vLLM](/tools/vllm) auto-detect the correct template. Manual configurations should verify the `add_bos_token` setting. Vertex AI deployment for Gemma provides enterprise-grade features: private endpoints, IAM access control, Cloud Audit Logs. For enterprises already on Google Cloud, Vertex AI Gemma deployment eliminates infrastructure management. The vision encoder in Gemma 3+ multimodal variants has a specific preprocessing requirement: images must be resized to 896×896 pixels with the SigLIP preprocessing pipeline. Using standard CLIP preprocessing (224×224) produces unrecognized image formats and silent quality degradation. Fixed in vLLM 0.6.3. Verify version if deploying Gemma on vLLM. On [llama.cpp](/tools/llama-cpp) b2700-b2780, KV-cache offloading for Gemma's hybrid attention layers had a bug where SWA and global KV entries were stored at different offsets, causing garbled output at context boundaries. Fixed in b2781+. Gemma 4 31B's 256K context at FP16 KV-cache (~31 GB VRAM for KV-cache alone at 256K) is impractical on consumer hardware. The practical limit for consumer deployment at 256K is Q4_K_M on [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) or 4-bit KV-cache quantization which reduces KV-cache to ~8 GB.