Grouped-Query Attention (GQA)
Definition pending
We've cataloged "Grouped-Query Attention (GQA)" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
GQA (Grouped-Query Attention) is a memory optimization: instead of every attention head having its own key-value pair, heads share KV pairs in groups. This reduces KV-cache size by 2–8×. Llama 2 70B uses GQA with 8 KV heads (vs 64 query heads). For you, this means longer context fits in less VRAM.
Workflow example
When comparing models, check if they use GQA (most modern ones do: Llama 2+, Mistral, DeepSeek). A model with GQA can fit ~4× more context tokens in the same VRAM. If your use case needs 32K+ context, GQA is a hard requirement — without it, the KV cache alone for 32K tokens on a 70B model would eat ~8 GB of VRAM. Check model specs before downloading.