Command R

Command R (March 2024) launched as Cohere's first open-weight enterprise RAG model — a 35B dense transformer decoder with GQA (8 KV heads), SwiGLU activation, and RoPE with base frequency 500,000 for native 128K context. The training data emphasized enterprise knowledge work: structured documents, technical manuals, financial reports, and multilingual corporate communications. Cohere's enterprise DNA shaped the architecture toward reliable RAG rather than creative generation. Command R+ (April 2024) scaled to 104B parameters while maintaining the same architectural primitives: full global attention, GQA, SwiGLU, 128K context. The scaling was pure capacity expansion — same architecture, more layers, wider FFN, more attention heads. The 104B variant was trained on additional data emphasizing advanced reasoning and longer-context understanding. The consistent architecture between Command R 35B and Command R+ 104B meant inference engines supporting one automatically supported the other. Command R+ 08-2024 (August 2024) refined the 104B with improved RLHF alignment using Cohere's constitutional AI-inspired approach. Architecture unchanged; training methodology and data quality improved. This version specifically improved tool-use reliability and multilingual consistency — function-calling accuracy improved from ~82% to ~91% on the Berkeley Function Calling Leaderboard. Aya Expanse (October 2024) is a separate Cohere For AI model family with different architecture and licensing. While sharing Cohere organizational DNA, [Aya Expanse 32B](/families/aya) is architecturally distinct from Command R and should not be confused with it. The architectural evolution is intentionally conservative: Command R's full global attention + enterprise tokenizer + tool-use pre-training established a differentiated design at V1, and subsequent releases have refined rather than redesigned. This architectural stability simplifies production deployments — the serving configuration that works for Command R 35B works for Command R+ 104B with only resource scaling. The contrast with [Llama](/families/llama) (which added MoE in V4) and [DeepSeek](/families/deepseek) (which continuously evolves MLA) highlights Cohere's enterprise-focused conservatism.

Command R models present an unusual local viability profile: they are among the most locally-demanding models at their parameter count due to full global attention at 128K context, but their RAG specialization makes them uniquely suited for specific local use cases. Total ~150 GB for a production deployment at full 128K context with reasonable batch size. This is impractical on consumer hardware: [RTX 4090](/hardware/rtx-4090) 24 GB cannot fit the model, let alone the KV-cache. On 2× [RTX 4090](/hardware/rtx-4090) 24 GB with tensor parallelism, ~25 tokens/second at Q4 with 32K context (longer context exceeds VRAM). Command R+ 104B at FP16 requires ~210 GB VRAM plus ~16 GB per 128K sequence. At Q4 (~58 GB), the model fits on 3× [RTX 4090](/hardware/rtx-4090) with tensor parallelism but VRAM for context is severely limited. [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) 192 GB at Q4 with 32K context is the only consumer-accessible path — expect ~6 tokens/second. This is slow enough to be non-interactive but functional for batch RAG processing. The trajectory from Command R 35B → Command R+ 104B makes local deployment meaningfully harder. The 35B was already pushing consumer hardware limits; the 104B requires multi-GPU or Mac Ultra-class hardware. This trajectory reflects Command R's enterprise positioning — these models are designed for datacenter deployment on Cohere's cloud or enterprise self-hosted clusters, not personal hardware. However, the RAG specialization changes the local viability calculus. No other locally-deployable model at 35B scale provides this — [Llama 3.1](/families/llama) and [Qwen](/families/qwen) have KV-cache efficiency through GQA that helps but full global attention for RAG is Command R's unique advantage. If you need a local RAG specialist that reliably retrieves information from large documents, the hardware investment in Command R hardware may be justified by the retrieval quality. For hardware recommendations: [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) 192 GB is the practical floor for Command R 35B at usable context lengths. Cloud rental of [A100 80GB](/hardware/nvidia-a100-80gb-sxm) or [L40S](/hardware/nvidia-l40s) clusters is the standard path for Command R+ deployment. Consumer GPUs (< 32 GB VRAM) cannot practically deploy Command R models at production context lengths.

Command R 35B (March 2024) launched with a benchmark profile oriented toward enterprise use cases rather than academic benchmarks. MMLU 72.3% (5-shot) was competitive with [Llama 2 70B](/families/llama) (68.9%). GSM8K 64.2% (8-shot) was adequate but behind Llama 2 70B (56.8%). HumanEval 44.5% (pass@1) was moderate — code generation was not a priority. The standout metrics were in RAG-specific evaluations: multi-hop reasoning accuracy 82.1% (HotpotQA), document groundedness 94.3% (Cohere internal eval), and citation accuracy 87.2%. These RAG benchmarks were Command R's differentiated strength — no other open-weight model at 35B matched this retrieval quality. Command R+ 104B (April 2024) scored MMLU 80.5% (5-shot), GSM8K 79.8% (8-shot), HumanEval 52.8% — competitive with [Llama 3 70B](/families/llama) (79.5% MMLU). Tool-use benchmarks improved dramatically: Berkeley Function Calling Leaderboard 82.4% accuracy, Gorilla API-Bench 76.1%. Multilingual benchmarks: Arabic MMLU 68.3%, French MMLU 77.2%, Japanese MMLU 71.5% — strong multilingual coverage exceeding [Llama](/families/llama) on non-English. The 104B's benchmark profile was squarely at the Llama 3 70B capability level but with superior RAG and tool-use scores. Command R+ 08-2024 refined with MMLU 82.1%, GSM8K 82.5%, HumanEval 56.0%. The Berkeley Function Calling Leaderboard improvement to 91.3% was the standout — matching GPT-4 on function-calling accuracy. Multilingual benchmarks improved 2-4 points across all 10 supported languages. The 08-2024 release established Command R+ as the open-weight leader on enterprise function-calling workflows. The Command R benchmark identity: not chasing the capability frontier (which [DeepSeek](/families/deepseek) and [Qwen 3](/families/qwen) lead) but maximizing RAG/enterprise workflow quality. Where Command R wins: RAG retrieval accuracy at full context, function-calling reliability, multilingual corporate communications. Where Command R trails: general reasoning (behind Llama/Qwen/DeepSeek by 5-10 MMLU points at equivalent parameter scale), code generation (HumanEval scores are ~15-20 points behind [DeepSeek Coder V3](/families/deepseek)), pure chat capability (less polished conversation style than [Llama 3.1](/families/llama)). The benchmark profile reflects deliberate design tradeoffs: RAG and tool-use over general reasoning and chat.

Command R runtime support is narrower than [Llama](/families/llama) but deep where it matters for enterprise deployment. Cohere's enterprise focus means first-class support on platforms that enterprises use. [vLLM](/tools/vllm) added Command R support in v0.4.1. The full global attention at 128K memory profile requires vLLM's paged attention to manage the large KV-cache efficiently. vLLM's prefix caching is disproportionately effective for Command R because RAG workloads with shared document prefixes produce high cache-hit rates. [SGLang](/tools/sglang) added Command R support in v0.2.3. SGLang's RadixAttention tree cache is well-suited to Command R's RAG workloads — documents shared across multiple user queries benefit from prefix caching. Command R 35B on 2× [L40S](/hardware/nvidia-l40s) with SGLang achieves ~30 tokens/second decode for single requests. [Ollama](/tools/ollama) supports Command R 35B through the llama.cpp backend. The canonical invocation: `ollama run command-r:35b`. Full global attention at 128K requires significant RAM; Ollama's default context window is 8K for Command R to maintain interactive speeds. Users can extend via `/set parameter num_ctx 32768` but at reduced speed. Command R+ 104B on Ollama is functional but slow — not recommended for interactive use. [llama.cpp](/tools/llama-cpp) supports Command R through standard GGUF format. Command R 35B at Q4 on [Apple M3 Ultra](/hardware/apple-m3-ultra) with 32K context achieves ~12 tokens/second. At 128K context, throughput drops to ~3 tokens/second due to attention-compute growth. [TensorRT-LLM](/tools/tensorrt-llm) supports Command R with flash-attention fused kernels. Command R+ 104B FP8 on 8× [H100 SXM](/hardware/nvidia-h100-sxm) achieves ~12,000 tokens/second at batch 128, benefiting from TRT's optimized full-attention implementation. This is the highest-throughput production path. [Text Generation Inference](/tools/text-generation-inference) supports Command R with the HuggingFace pipeline. TGI's RAG-specific features (document chunking, prompt templating) are well-matched to Command R's design. TGI 1.4+ handles the full 128K global attention correctly. Cohere's own serving infrastructure is the reference implementation. The Cohere API uses proprietary optimizations for Command R that are not available in open-source engines. This is a unique pattern — most open-weight families perform best on community engines; Command R performs best on Cohere's infrastructure.

Command R quantization faces challenges from the full global attention architecture. The KV-cache memory is the dominant VRAM consumer at production context lengths, not the model weights, reducing the practical benefit of weight quantization. GGUF via [llama.cpp](/tools/llama-cpp) is the most tested path. Command R 35B at Q4_K_M (~20 GB) loses ~0.5% MMLU vs FP16 — standard degradation for this size. At Q3_K_M (~15 GB), ~2.2% MMLU degradation. The full attention mechanism means KV-cache memory is independent of weight quantization — 128K context at FP16 KV-cache consumes ~8 GB regardless of whether model weights are Q4 or FP16. At Q4 KV-cache, 128K context uses ~4 GB. This creates an unusual deployment dynamic: KV-cache quantization saves more VRAM than model weight quantization at long context. Command R+ 104B at Q4_K_M (~58 GB) loses ~0.6% MMLU. At Q3_K_M (~44 GB), ~2.5% degradation. Q3 is borderline usable; Q2_K (~35 GB) loses ~8% MMLU and is not recommended. The 104B's larger parameter count provides more quantization headroom than the 35B, making Q3_K_M slightly more viable for the larger model paradoxically. GPTQ and AWQ support is available for Command R. AWQ 4-bit (~20 GB for 35B, ~58 GB for 104B) achieves lossless quality on MMLU. The activation distributions in Command R's full-attention layers are well-behaved, making AWQ and GPTQ equally effective. Choose AWQ for vLLM compatibility. FP8 via [TensorRT-LLM](/tools/tensorrt-llm) and [vLLM](/tools/vllm) is the standard datacenter format. Command R 35B FP8 (~35 GB) and Command R+ 104B FP8 (~104 GB) are effectively lossless. FP8 KV-cache quantization saves ~2× KV-cache memory and is the most impactful optimization for Command R — at 128K context, FP8 KV-cache (~4 GB) vs FP16 KV-cache (~8 GB) for the 35B can be the difference between fitting in GPU memory or not. The recommendation for Command R quantization differs from other families: prioritize KV-cache quantization (FP8 or 4-bit KV-cache) over aggressive model weight quantization because at typical RAG context lengths (32K-128K), KV-cache memory dominates total VRAM usage. A Q4_K_M model with FP8 KV-cache is the practical sweet spot: ~24 GB total for 35B at 128K context. For [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) users, Q4_K_M with 4-bit KV-cache fits Command R 35B + 128K context in ~22 GB.

Command R's ecosystem is enterprise-oriented and Cohere-centric, with moderate community growth constrained by the CC-BY-NC-4.0 license. The ecosystem is smaller than [Llama](/families/llama) or [Qwen](/families/qwen) but with higher enterprise-adoption density. Derivative model volume is low (tens of fine-tunes). The CC-BY-NC license restricts commercial fine-tune distribution, discouraging the community fine-tuning culture that drives Llama/Qwen ecosystem growth. Major derivatives: few community fine-tunes exist; the primary derivatives are Cohere's own incremental releases (35B → + → + 08-2024). This is a deliberate strategy — Cohere wants to control the Command R quality curve rather than rely on community fine-tunes. License clarity is a significant ecosystem constraint. CC-BY-NC-4.0 permits research use, personal use, and non-commercial deployment but prohibits commercial use without a separate Cohere license. The CC-BY-NC license is the primary factor limiting Command R's community ecosystem growth. Cohere's enterprise ecosystem provides compensating depth. Cohere's platform (Dashboard, API, fine-tuning service) provides managed Command R deployment with enterprise SLAs, SOC 2 compliance, and data residency options. For enterprises that need RAG-specialized models and are already on Cohere's platform, Command R's enterprise ecosystem is more complete than self-hosting alternatives. Training data transparency is moderate. Cohere discloses training data sources at the category level (web, books, code, enterprise documents) but does not disclose dataset composition percentages or filtering methodology. This is comparable to [Llama](/families/llama). The RAG ecosystem around Command R is deep. Cohere's RAG pipeline (embedding models, reranker, chat model) is integrated with Command R's native tool-use and citation format. LangChain and LlamaIndex have Command R-specific RAG implementations. The Cohere toolkit (cohere-toolkit) provides a reference RAG implementation with document parsing, chunking, embedding, and retrieval optimized for Command R's full-attention context handling. Ecosystem gaps: general-purpose fine-tuning community is minimal; creative/roleplay derivatives are absent; non-English fine-tuning community is limited despite Command R's multilingual capability. The license restriction creates a hard ceiling on ecosystem growth that no amount of technical quality can overcome.

Command R fine-tuning is available through Cohere's platform and standard transformers tooling, with distinct paths for research (free, CC-BY-NC) and commercial (paid, Cohere platform). LoRA and QLoRA support is available through HuggingFace PEFT. The practical Command R 35B QLoRA hardware: [A6000](/hardware/nvidia-a40) 48 GB or [L40S](/hardware/nvidia-l40s) 48 GB, training 10K samples in ~12 hours. Command R+ 104B QLoRA requires ~72 GB VRAM — 2× [A100 80GB](/hardware/nvidia-a100-80gb-sxm) with ZeRO-3. Full fine-tuning is infeasible for individuals: Command R 35B needs ~280 GB VRAM (4× [A100 80GB](/hardware/nvidia-a100-80gb-sxm) with ZeRO-3). Command R+ 104B needs ~832 GB VRAM (12× [A100 80GB](/hardware/nvidia-a100-80gb-sxm)). Full fine-tuning is for well-funded enterprise teams. Cohere's platform provides the primary fine-tuning path. The Cohere Dashboard offers fine-tuning jobs with hyperparameter selection, dataset upload, and evaluation metrics. This is the recommended path for production fine-tunes because Cohere's internal fine-tuning infrastructure is optimized for Command R's architecture and produces better-quality fine-tunes than DIY HuggingFace PEFT. The tradeoff: Cohere platform fine-tuning is paid and the output model is hosted on Cohere; self-hosted fine-tune export requires commercial licensing. The dataset ecosystem for Command R emphasizes RAG-specific formats. Training data should use Command R's citation format (`<co: 0>` document reference tags) for RAG fine-tuning. The function-calling format uses specific JSON schemas that differ from OpenAI's function-calling format, requiring Command R-specific dataset preparation. Axolotl has basic Command R support; Unsloth does not support Command R. The fine-tuning ecosystem is thinner than for [Llama](/families/llama) due to the license restriction limiting community tooling investment. A Command R-specific fine-tuning consideration: the Cohere tokenizer's 256K vocabulary handles enterprise document formats (PDF text extraction artifacts, markup, structured data) better than BPE tokenizers. If you fine-tune Command R for a domain-specific document corpus, the tokenizer will naturally handle document formatting quirks that trip up other models' tokenizers. This is an underappreciated advantage of Command R for document-heavy fine-tuning.

[Command R](/families/command-r)'s CC-BY-NC-4.0 license is the single most important deployment consideration. Commercial use requires a separate license from Cohere. This means: any deployment that generates revenue (directly or indirectly) is non-compliant without a Cohere commercial license. Verify with Cohere's licensing team before production deployment. Full global attention at 128K context creates unbounded memory scaling. Each additional token of context adds O(L) computation and memory to every subsequent token generation. For interactive use, this means time-to-first-token grows linearly with context length: ~100ms at 8K, ~500ms at 32K, ~2s at 128K. Function-calling format compatibility requires attention. Command R uses a Cohere-specific function-calling schema that differs from OpenAI's widespread format. The Cohere format uses a preamble with `<TOOL_CALL>` and `<|tool_call|>` tokens. Applications using OpenAI-format function definitions must convert to Cohere format for Command R. Citation format for RAG is specific and must be followed. Command R expects document citations in `<co: N>` format where N is the document index. Misformatting the document prefix destroys citation accuracy, the primary reason to use Command R over alternatives. The correct prompt structure is documented in Cohere's RAG guide and must be followed precisely. Deployment on [vLLM](/tools/[vllm](/tools/vllm)) requires specific attention backend configuration. The full global attention at 128K must use vLLM's flash-attention backend (FA2 or FA3) — vLLM's default paged attention falls back to quadratic attention, which is correct but slow (10× slower at 128K). Ensure `--attention-backend flash-attn` is set. The enterprise-oriented training data creates a stylistic profile that may not suit all use cases. Command R models produce professional, factual, carefully-cited responses appropriate for enterprise communications. This can feel sterile or overly formal for casual chat applications. Multilingual quality degrades at extreme context lengths for non-English languages. While Command R supports 10 languages well at standard context, at 128K context the attention distribution becomes slightly biased toward English tokens (which dominate the training data), reducing non-English retrieval accuracy by 3-5% at 128K vs 32K. Test multilingual RAG accuracy at your target context length.