MedGemma 27B

MedGemma 27B is Google's medical-domain fine-tune of Gemma 27B. Specialized for clinical text understanding, medical Q&A, and biomedical reasoning. Run at Q4_K_M via Ollama (ollama pull medgemma:27b) or llama.cpp with -ngl 999 -fa -c 4096. Q4_K_M file size ~15 GB on disk. Minimum VRAM: 12 GB — RTX 4070 (12GB) at Q4_K_M with KV offload for 4K context. RTX 4090 24GB: Q4_K_M comfortably at 16K+ context. Recommended: RTX 4090 24GB at Q4_K_M. Throughput: ~40-60 tok/s on RTX 4090 at Q4_K_M. Gemma architecture — well-supported in llama.cpp, Ollama, and Gemma-specific stacks. MedGemma is Google's medical AI offering — trained on medical literature, clinical notes, and biomedical datasets. Strong on: medical terminology, diagnostic reasoning, drug information, clinical summarization. Weak on: non-medical topics (catastrophic forgetting from domain fine-tuning), current medical guidelines (knowledge cutoff). Not FDA-approved — research model only. License: Gemma license (verify commercial terms). Use for: medical research, clinical decision support (with human review), biomedical literature analysis. Never for: autonomous clinical decisions. Context: Gemma's 8K (practical 4-8K on 12-16 GB). Medical contexts are typically shorter — less KV pressure.

Minimum: RTX 3060 12GB at Q3_K_M with KV offload. Recommended: RTX 4090 24GB at Q4_K_M (16K+ context). VRAM math: 27B dense, Q4_K_M ≈ 15 GB. KV cache at 8K: ~6 GB. Total: ~21 GB at 8K. RTX 4090 24GB: Q4 + 8-16K context — comfortable on-GPU. RTX 3080 10GB: Q3_K_M with KV offload. RTX 4080 16GB: Q4 + 4K context on-GPU. MacBook Pro M4 Pro 24GB+: Q4 at 12-25 tok/s. Cloud: A10 24GB at Q4_K_M. MedGemma is one of the largest medical-specific models that fits comfortably on consumer GPUs. AWQ-INT4 drops to ~13 GB. For clinical settings: the model's knowledge is frozen — pair with RAG on current medical literature (PubMed, UpToDate).

1. Medical hallucination. MedGemma generates plausible-sounding but incorrect medical information — drug dosages, contraindications, treatment protocols. Never use for autonomous clinical decisions. Always verify outputs against current medical guidelines. 2. Catastrophic forgetting. Domain fine-tuning on medical data severely degrades general knowledge. The model will hallucinate on non-medical topics worse than base Gemma 27B. 3. Knowledge cutoff. Medical knowledge is frozen at training time. New drugs, treatment guidelines, and research published after the cutoff aren't known. RAG on current literature is essential for clinical use. 4. Q3 terminology precision. Medical terminology at Q3 introduces dangerous errors — confusing similar drug names, dosages, or anatomical references. Use Q4_K_M minimum for medical tasks. Q8 or FP16 for clinical settings.

Ollama for quick-start. llama.cpp for production. vLLM for serving. Gemma architecture — well-supported. For clinical use: pair with a biomedical RAG pipeline (PubMedBERT embeddings + current literature vectors). MedGemma should augment, not replace, clinical expertise.

Mistake: Using MedGemma as a diagnostic tool without physician review. Fix: This is a research model, not a medical device. Always have a qualified physician review outputs. The model hallucinates drug names and dosages. Mistake: Trusting MedGemma for current treatment guidelines. Fix: Knowledge cutoff limits the model to pre-training data. Use RAG on UpToDate, PubMed, or current clinical guidelines for up-to-date information. Mistake: Using Q3 quantization for medical tasks. Fix: Q3 confuses medical terminology. Use Q4_K_M minimum. Q8 or FP16 for clinical settings where precision matters. Mistake: Expecting MedGemma to perform well on general knowledge tasks. Fix: Medical fine-tuning causes catastrophic forgetting of non-medical knowledge. Use base Gemma 27B for non-medical tasks.