Aya

Architecture: dense decoder with GQA (4 KV heads), SwiGLU activation, and RoPE with 128K context via interpolation. The training data was unprecedented in breadth: 513M examples spanning 101 languages, with explicit language balancing to prevent English dominance. The dataset, Aya Dataset, was crowd-sourced from fluent speakers worldwide and was itself released openly (CC-BY-4.0) — a transparency standard that no other open-weight family has matched. Vocab size: 256K with a multilingual tokenizer designed for equal treatment of Latin, Cyrillic, Arabic, Devanagari, CJK, and Thai scripts. Aya 23 (May 2024) scaled to 8B and 35B variants with the same architecture and language coverage. Training data expanded to 1.2B examples. The key architectural change: dynamic NTK-aware RoPE scaling for improved long-context performance in non-English scripts where tokenization produces longer sequences. The 35B variant adopted GQA with 8 KV heads for improved throughput at scale. The architecture remained intentionally conservative — Cohere For AI's priority was language-coverage breadth, not architectural innovation. Aya Expanse (October 2024) represents the current generation: 8B and 32B variants with continued multilingual transformer architecture. Training data expanded further with emphasis on low-resource languages (the "long tail" of the 101-language distribution). The architectural highlight is the multilingual routing mechanism: certain transformer layers specialize in language-family-specific processing through learned language embeddings that modulate attention patterns. This "soft language routing" allows Aya Expanse to maintain quality across 101 languages without 101× the parameters. Training data: 2.5B examples with explicit balance constraints per language. The architectural evolution from Aya 101 → Aya 23 → Aya Expanse traces a language-coverage-first philosophy. Unlike [Command R](/families/command-r) (enterprise RAG specialization within the same organization) or [Llama](/families/llama) (general English-first capability), Aya optimizes for breadth and linguistic fairness. The soft language routing in Aya Expanse is the family's only architectural novelty — everything else is standard dense transformer optimized for data diversity rather than parameter efficiency. This makes Aya models architecturally boring (good for compatibility) and linguistically extraordinary (101 languages with balanced quality).

Aya models are locally-friendly at the 8B scale and challenging at the 32-35B scale, but their unique value proposition (101-language coverage) may justify hardware investment for specific multilingual use cases. Aya 23 8B at Q4 (~5 GB) runs on any modern GPU. On [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb), ~70 tokens/second via [Ollama](/tools/ollama). On [MacBook Pro M4 Max](/hardware/macbook-pro-16-m4-max) via [MLX-LM](/tools/mlx-lm), ~40 tokens/second. The 8B is deployable on laptop-class hardware with excellent throughput. For multilingual local assistants (Arabic, Hindi, Vietnamese, Swahili), Aya 23 8B is the most capable locally-deployable model. Aya Expanse 32B at Q4 (~18 GB) fits on [RTX 4090](/hardware/rtx-4090) 24 GB with ~5 GB headroom for context. On [Apple M3 Ultra](/hardware/apple-m3-ultra), ~20 tokens/second via [llama.cpp](/tools/llama-cpp) at Q4. Aya Expanse 32B provides the highest-quality locally-available multilingual capability; no other model at this hardware tier covers 101 languages. Aya 23 35B at Q4 (~20 GB) fits on [RTX 4090](/hardware/rtx-4090) 24 GB. The slightly larger size vs the 32B Expanse variant produces ~10% lower throughput but equivalent quality. For practical purposes, Aya Expanse 32B is the recommended local deployment over Aya 23 35B — the soft language routing in Expanse produces better low-resource-language quality at a smaller parameter count. The trajectory: Aya models have maintained consistent local viability. The 8B models are universally deployable. The 32-35B models require mid-range consumer GPUs ([RTX 4090](/hardware/rtx-4090)) or Apple Silicon with 32+ GB unified memory. This makes Aya the only family where the most capable model (Aya Expanse 32B) is deployable on a single consumer GPU. [Qwen](/families/qwen) covers Chinese/English well but has gaps in African, South Asian, and Eastern European languages. [Llama](/families/llama) covers major world languages but treats the long tail poorly. Aya's 101-language coverage at consumer-hardware scale is unique and may justify the hardware cost for users who need it. For hardware recommendations: [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb) for Aya Expanse 8B at Q4; [RTX 4090](/hardware/rtx-4090) 24 GB for Aya Expanse 32B at Q4; [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) for Aya Expanse 32B at Q4 with batch processing.

Aya models are benchmarked differently from other families — the primary metrics are multilingual coverage, cross-lingual consistency, and low-resource language quality rather than peak English performance. Aya 101 13B (February 2024) scored MMLU 62.1% (5-shot English), which was behind [Llama 2 13B](/families/llama) (54.8% but with English-specific training). On multilingual benchmarks: the Aya Evaluation Suite (covering 101 languages on 6 tasks) showed consistent quality across the top 40 languages (within 10% variation) and declining but still usable quality for the bottom 61 languages. The benchmark methodology was itself a contribution — no existing benchmark suite covered 101 languages at this depth, and the Aya team created evaluation datasets for previously unevaluated languages. Aya 23 35B (May 2024) scored MMLU 74.3% (5-shot English), competitive with [Command R 35B](/families/command-r) (72.3%). The multilingual benchmarks showed quality parity with single-language models for the top 20 languages (English, French, Spanish, Arabic, Chinese, etc.) and 10-25% degradation for the bottom 20 languages — a dramatic improvement over Aya 101's low-resource performance. The Aya 23 8B scored MMLU 65.2% on English and maintained above 50% quality for 80 of 101 languages. Aya Expanse 32B (October 2024) scores MMLU 79.8% (5-shot English), competitive with [Llama 3 70B](/families/llama) on English benchmarks despite 2.2× fewer parameters. GSM8K 75.3% (8-shot). HumanEval 52.1% (pass@1). The multilingual benchmarks are the story: quality consistency across 101 languages is within 15% of peak for 90 languages — unprecedented language-coverage breadth. On low-resource language benchmarks, Aya Expanse 32B exceeds [Qwen 3 32B](/families/qwen) by 20-50 points for languages that Qwen's training data barely covers (Swahili, Amharic, Yoruba, Quechua). The Aya Evaluation Suite (Aya Eval) introduced by the Aya team provides standardized multilingual benchmarks for 101 languages across classification, summarization, translation, and question-answering tasks. This benchmark set has become the reference standard for multilingual model evaluation in open-weight AI. Where Aya wins: multilingual coverage breadth (no competitor covers 101 languages), low-resource language quality (gap vs next-best model is >20 points for many languages), and balanced performance across language families. Aya's benchmark identity is multilingual fairness, not capability frontier.

Aya runtime support mirrors [Command R](/families/command-r)'s in scope — narrower than [Llama](/families/llama) but functional on major platforms. The architectural simplicity (standard dense transformer) means every engine supports Aya with minimal configuration. [vLLM](/tools/vllm) supports Aya models since v0.4.5 using the standard Llama-like architecture configuration. Aya Expanse 32B on [L40S](/hardware/nvidia-l40s) at Q4 achieves ~8,000 tokens/second at batch 64. vLLM's prefix caching is effective for Aya's multilingual workloads where system prompts in different languages share structural patterns. [Ollama](/tools/ollama) provides Aya deployment: `ollama run aya:8b`, `ollama run aya-expanse:32b`. Aya models are available through the Ollama library. On [RTX 4090](/hardware/rtx-4090), Aya Expanse 32B at Q4 achieves ~35 tokens/second. The 8B on any GPU achieves 70+ tokens/second. Ollama is the recommended local deployment path for Aya. [llama.cpp](/tools/llama-cpp) supports all Aya models via GGUF format. Aya Expanse 32B at Q4 on [Apple M3 Ultra](/hardware/apple-m3-ultra) with Metal achieves ~20 tokens/second. The 256K vocabulary with multilingual tokenization is handled correctly in llama.cpp since b2600+. [TensorRT-LLM](/tools/tensorrt-llm) supports Aya models for datacenter deployment. Aya Expanse 32B FP8 on [H100 SXM](/hardware/nvidia-h100-sxm) achieves ~10,000 tokens/second at batch 128. [MLX-LM](/tools/mlx-lm) provides good Aya support on Apple Silicon. Aya Expanse 32B at Q4 on [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) achieves ~22 tokens/second. The multilingual tokenizer works correctly in MLX, including for non-Latin scripts. [Text Generation Inference](/tools/text-generation-inference) supports Aya with standard HuggingFace pipeline. TGI 1.5+ handles the soft language routing in Aya Expanse correctly — earlier versions had a bug where the language embedding was not forwarded correctly. [Transformers](/tools/transformers) provides first-class Aya support. The `aya-23` and `aya-expanse` model cards are available on HuggingFace with full configuration. The Cohere For AI team provides official model weights and deployment documentation. Cohere's platform provides Aya deployment through the Cohere API for production multilingual applications. The API path is the recommended production deployment — Cohere's internal optimizations for the Aya architecture produce better throughput than self-hosted vLLM.

Aya models quantize similarly to standard dense transformers with minor multilingual-specific considerations. The 256K vocabulary tokenizer's token distribution becomes relevant at low bit widths. GGUF via [llama.cpp](/tools/llama-cpp) is the primary path. Aya Expanse 8B at Q4_K_M (~5 GB) loses ~0.3% MMLU vs FP16, effectively lossless. Aya Expanse 32B at Q4_K_M (~18 GB) loses ~0.5% MMLU. At Q3_K_M (~14 GB), ~2.0% degradation. At IQ3_XXS (~12 GB), ~2.5% degradation — the quality loss is slightly higher than [Llama](/families/llama) at equivalent size because multilingual weight matrices have more diverse activation patterns from the 256K vocabulary. A multilingual-specific quantization finding: at Q3 and below, quality degrades disproportionately for low-resource languages. The model weights that serve low-resource languages have less redundancy (there are fewer examples per language in training, making each weight increment matter more), so quantization noise hits low-resource language performance harder than high-resource language performance. At Q4_K_M, the degradation is uniform across languages (~0.5%). At Q3_K_M, English degrades ~1.5% while Swahili degrades ~3.5%. For deployments serving low-resource languages, stick to Q4_K_M or above. GPTQ and AWQ support is available. Aya Expanse 32B AWQ 4-bit (~18 GB) is lossless on English MMLU (<0.2% degradation). AWQ performs well across all languages at 4-bit. GPTQ provides equivalent quality. AWQ is the recommended format for vLLM production deployment. EXL2 is not tested for Aya models. The EXL2 kernel doesn't have specific Aya optimizations; standard dense transformer EXL2 configs should work but haven't been community-benchmarked. FP8 via [TensorRT-LLM](/tools/tensorrt-llm) and [vLLM](/tools/vllm) is effectively lossless (<0.1% all languages). FP8 KV-cache quantization is supported. For multilingual deployments, the soft language routing in Aya Expanse requires special attention: the language embedding layer that modulates attention patterns should not be quantized below FP16. Trivial size (~10M parameters) means the precision savings are negligible. Keep the language embedding at FP16 regardless of model weight quantization. Practical rule: Aya Expanse 8B handles Q3_K_M well (~1.5% loss); Aya Expanse 32B needs Q4_K_M for production (~0.5% loss), especially for low-resource language deployments. On [RTX 4090](/hardware/rtx-4090), Q4_K_M for the 32B fits comfortably in 24 GB with ~5 GB headroom.

Aya's ecosystem is research-oriented and transparency-driven, with a community centered on multilingual AI rather than commercial deployment. The ecosystem is smaller than [Llama](/families/llama) or [Qwen](/families/qwen) but uniquely deep in multilingual AI research. Derivative model volume is low (dozens of fine-tunes). The CC-BY-NC-4.0 license restricts commercial fine-tune distribution similarly to [Command R](/families/command-r). Community derivatives primarily focus on: language-specific fine-tunes (Arabic Aya, Hindi Aya, Vietnamese Aya — fine-tuned for specific language quality improvements), and domain-adapted multilingual models (medical Aya, legal Aya). The derivative culture is research-driven rather than deployment-driven. License clarity shows strengths and constraints. CC-BY-NC-4.0 permits research and non-commercial use, requires attribution, and prohibits commercial use without separate licensing. This is the same license as [Command R](/families/command-r), reflecting Cohere For AI's organizational approach. This creates an unusual dynamic: you can use the Aya Dataset to train your own model commercially, but you cannot deploy the pre-trained Aya model commercially. Transparency is Aya's ecosystem superpower. The Aya Dataset (513M examples → 2.5B examples) is fully released under CC-BY-4.0 with language labels, source attribution, and quality annotations. Aya's dataset transparency sets the gold standard for open-weight AI and enables research reproducibility that other families cannot match. The dataset has been used to train hundreds of research models beyond the official Aya releases. The multilingual AI research community centers on Aya. The Aya project collaborates with linguists, NLP researchers, and native speakers worldwide. The Aya Evaluation Suite has been adopted as a standard multilingual benchmark by the broader NLP community. The Aya Model License and Dataset have become reference points for multilingual AI ethics discussions — the Aya project's model of crowd-sourced, transparently-licensed data challenges the "scrape everything" approach of other families. Ecosystem gaps: commercial deployment ecosystem is minimal due to CC-BY-NC, fine-tuning tooling is standard but not optimized, and agentic/tool-use frameworks have limited Aya integration. The ecosystem serves research and non-commercial multilingual applications well but does not serve commercial production deployment.

Aya fine-tuning is accessible at the 8B scale and challenging at the 32B scale, with a research-focused tooling ecosystem. LoRA and QLoRA support is available through HuggingFace PEFT. For Aya Expanse 8B, QLoRA rank 64 with 4-bit base model on [RTX 4090](/hardware/rtx-4090) uses ~20 GB VRAM and trains 10K samples in ~3 hours. For Aya Expanse 32B, QLoRA rank 32 with 4-bit base on [RTX 4090](/hardware/rtx-4090) uses ~23 GB VRAM (tight) and trains 10K samples in ~10 hours. The 8B is the recommended fine-tuning target for most multilingual fine-tuning tasks. Full fine-tuning: Aya Expanse 8B needs ~64 GB VRAM (1× [A100 80GB](/hardware/nvidia-a100-80gb-sxm)). Aya Expanse 32B needs ~256 GB VRAM (4× [A100 80GB](/hardware/nvidia-a100-80gb-sxm) with ZeRO-3). Full fine-tuning at 32B is a research-lab-scale investment. The Aya Dataset enables a unique fine-tuning workflow: language-specific continued pre-training followed by instruction fine-tuning. Because the Aya Dataset includes pre-training data (not just instruction data), you can extend Aya's pre-training for specific languages before instruction-tuning. This "continued multilingual pre-training" approach produces language-specific fine-tunes that outperform simple instruction-fine-tuning on the target language by 5-15 points on quality benchmarks. Multilingual fine-tuning considerations: the soft language routing in Aya Expanse should be preserved during fine-tuning. If you fine-tune Aya Expanse on a single language, the soft routing adaptation may collapse toward that language, reducing quality in other languages. To maintain multilingual capability: fine-tune with a mixed dataset containing at least 20% examples from other languages. This preserves the soft routing's language-differentiation capability. DPO alignment for Aya is possible but less documented than for [Llama](/families/llama) or [Mistral](/families/mistral). The Aya models are not RLHF-tuned from the base; instruction-following comes from supervised fine-tuning. Adding DPO post-SFT may improve alignment but is not part of the standard Aya fine-tuning recipe. Axolotl has limited Aya support; Unsloth does not support Aya. The fine-tuning tooling ecosystem is research-grade rather than production-grade, reflecting Aya's research orientation.

Aya's CC-BY-NC-4.0 license is the primary deployment constraint, identical to [Command R](/families/command-r). Commercial use requires a separate license from Cohere. Research, personal, and non-profit use is permitted. Verify licensing with Cohere for any deployment at a for-profit organization. Language coverage breadth creates specific quality expectations. Users of low-resource languages should expect lower but still functional quality. The quality gradient is documented in the Aya Expanse model card; consult it for expected quality in your target language before committing to deployment. The Aya tokenizer's 256K vocabulary is optimized for multilingual fairness but creates token-efficiency tradeoffs. For English text, Aya uses ~20% more tokens than [Llama 3+](/families/llama)'s English-optimized tokenizer. For low-resource languages with non-Latin scripts, Aya uses ~30-50% fewer tokens than Llama — the multilingual tokenizer was designed for this. Soft language routing has a deployment implication: Aya Expanse's language embedding should be set to the primary language of the user interaction. For deployments serving multilingual users who mix languages in conversation, test Aya's quality on mixed-language inputs before deployment. Chat template accuracy: Aya uses a Cohere-style chat template with `<BOS_TOKEN><|START_OF_TURN_TOKEN|>USER<|START_OF_TURN_TOKEN|>CHATBOT` formatting. This is similar to but distinct from [Command R](/families/command-r)'s chat template. Using Llama or ChatML templates degrades instruction-following. Ollama and vLLM auto-detect the template from model metadata. The Aya Dataset's CC-BY-4.0 license is more permissive than the model weights' CC-BY-NC. This creates a deployment option: use the Aya Dataset to train your own model under any license you choose. If you need a commercially-deployable multilingual model, training a new model on the Aya Dataset (or mixing it with other data) under Apache 2.0 or MIT is a viable path that avoids the CC-BY-NC constraint on the pre-trained Aya weights. Long-context multilingual performance degrades more than English at extreme context. At 128K context, retrieval accuracy for low-resource languages drops 5-10% more than English because the attention mechanism, trained primarily on shorter multilingual sequences, doesn't generalize perfectly to extended multilingual contexts. Test at your target context length with representative multilingual data.