Yi

Yi-6B and Yi-34B (November 2023) launched as 01.AI's first open-weight models. Architecture: dense transformer decoder with GQA (Yi-34B: 8 KV heads; Yi-6B: 4 KV heads), SwiGLU activation, and RoPE positional embeddings. The 34B model was the first open-weight 30B-class model to compete with 70B-class models, demonstrating strong parameter efficiency from its bilingual training data composition. Yi-34B-200K (March 2024) extended the base Yi-34B to 200K context via NTK-aware RoPE scaling with adjusted base frequency. The 200K context was the longest available in any open-weight model at release (exceeding Llama 2's 4K and Qwen 1.5's 32K). This produced usable 200K attention but with measurable retrieval-quality degradation at extreme positions (>128K). Yi-VL-6B and Yi-VL-34B (January 2024) added vision capability through a CLIP ViT-L/14 vision encoder (428M parameters) with a two-layer MLP projector connecting vision embeddings to the language model. The vision encoder was frozen during training; only the projector was trained. This is architecturally identical to [LLaVA 1.5](/families/llava)'s vision-language design, reflecting the standard VLM architecture of early 2024. Yi-1.5 (May 2024) refined the Yi-6B, Yi-9B, and Yi-34B with improved training data (4.1T tokens) and better data filtering. Architecture unchanged. The 1.5 release added Yi-1.5-9B to the family. RLHF alignment via DPO was applied to the chat variants. Yi-Lightning (October 2024) was a shift to commercial closed-weight with an undisclosed architecture. It is referenced for completeness but is outside open-weight deployment scope. Yi-Coder-9B (August 2024) was a code-specialized 9B model based on the Yi architecture with 2.4T additional code tokens continued pretraining and FIM (fill-in-the-middle) training. The architecture was standard Yi-9B with code-specific weight adaptation. The architectural evolution traces a pattern: launch with efficient dense 6B/34B models, add vision capabilities and extended context through targeted post-training, refine with data quality improvements. 01.AI's strategy has been architectural conservatism focused on bilingual (English-Chinese) quality. The Yi family's architectures are standard dense [transformers](/tools/transformers) — the differentiation is in the bilingual training data and the 200K context extension technique.

Yi models are locally-friendly at all sizes, reflecting the dense transformer architecture and modest parameter counts. The family peaks at 34B, making the full range deployable on consumer hardware. Yi-6B at Q4 (~4 GB) runs on any GPU. On [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb), ~80 tokens/second via [Ollama](/tools/ollama). On [MacBook Pro M4 Max](/hardware/macbook-pro-16-m4-max), ~55 tokens/second via [MLX-LM](/tools/mlx-lm). The 6B is deployable on laptop-class hardware. Yi-9B at Q4 (~5.5 GB) fits easily on [RTX 3060 12GB](/hardware/rtx-3060-12gb). On [Apple M3 Ultra](/hardware/apple-m3-ultra), ~50 tokens/second. The 9B provides a quality upgrade over 6B at minimal additional VRAM cost. Yi-34B at Q4 (~19 GB) fits on [RTX 4090](/hardware/rtx-4090) 24 GB with ~4 GB headroom for context. On [Apple M3 Ultra](/hardware/apple-m3-ultra), ~22 tokens/second. The 34B at Q3_K_M (~15 GB) fits on [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb) with ~1 GB headroom, running ~18 tokens/second at ~2% quality degradation — acceptable for casual use. Yi-34B-200K at Q4 (~19 GB) has a KV-cache challenge at extended context: 200K context at Q4 KV-cache consumes ~12 GB of additional VRAM. Total VRAM at 200K context with Q4 model + Q4 KV-cache: ~31 GB. On [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) 192 GB, the full 200K context at Q4 is feasible with ~40 GB total usage and ~10 tokens/second. The 200K context is the model's defining feature and may justify the Mac Studio investment for users needing extremely long context at consumer price points. Yi-VL-34B at Q4 (~22 GB including vision encoder) fits on [RTX 4090](/hardware/rtx-4090) 24 GB. Vision encoding adds ~400ms latency, negligible for interactive use. The VL variant provides the most capable locally-deployable VLM for Chinese-English bilingual use — [Gemma 3 27B](/families/gemma) is the closest competitor locally. The trajectory: Yi has maintained consistent local viability at all sizes. The dense architecture and 34B ceiling mean Yi models were designed for consumer hardware from inception. For hardware recommendations: [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb) for Yi-9B/34B at Q3; [RTX 4090](/hardware/rtx-4090) for Yi-34B at Q4; [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) for Yi-34B-200K at full context.

Yi-34B (November 2023) debuted with MMLU 76.3% (5-shot), matching [Llama 2 70B](/families/llama) (68.9%) and exceeding it significantly — the first 34B model to beat 70B-class English benchmarks. GSM8K 68.2% (8-shot), HumanEval 42.5%. C-Eval (Chinese knowledge) 81.4%, CMMLU 82.8% — native Chinese benchmark scores that exceeded every non-Chinese model at release. The bilingual benchmark profile showed Yi-34B as a "70B-quality model at 34B parameters," delivered through bilingual training data that provided more language-specific parameter efficiency. Yi-34B-200K (March 2024) maintained the same benchmark scores at standard context and demonstrated 86.3% Needle-in-Haystack retrieval accuracy at 200K — strong long-context retrieval quality at a then-unprecedented context length. Retrieval quality degraded to ~72% at 200K but remained above 80% through 128K. Yi-1.5 34B (May 2024) scored MMLU 80.2%, GSM8K 82.1%, HumanEval 55.8% — matching [Llama 3 70B](/families/llama) on MMLU at 2× fewer parameters. C-Eval 86.2%, CMMLU 87.5%. The 1.5 update narrowed the gap to Llama 3 70B on English benchmarks while maintaining Chinese leadership. Yi-1.5 9B scored MMLU 69.8%, GSM8K 72.5%, HumanEval 48.2% — competitive with [Gemma 2 9B](/families/gemma) on general benchmarks with superior Chinese performance. C-Eval 75.3%, CMMLU 76.8%. Yi-Coder-9B scored HumanEval 58.2% and MBPP 62.5% at pass@1, competitive with dedicated code models at 9B scale. LiveCodeBench 32.1%. The code benchmarks showed Yi architecture adapted effectively to code specialization despite the bilingual training heritage. The Yi benchmark identity: bilingual English-Chinese capability at class-leading parameter efficiency for Chinese. Where Yi wins: Chinese-language benchmarks (C-Eval, CMMLU) matching Chinese-native models like [Qwen](/families/qwen) at smaller parameter counts, extreme context retrieval (200K at usable quality), and parameter efficiency for bilingual workloads. Where Yi trails: peak English capability (behind [Llama 3.1 70B](/families/llama) and [Qwen 3 32B](/families/qwen) by 5-8 MMLU points), multilingual beyond English-Chinese (behind [Aya](/families/aya)), and code generation (competitive but not class-leading). Yi's benchmarks reflect a targeted bilingual strategy rather than a general-capability push.

Yi runtime support mirrors standard [Llama](/families/llama)-like model support since the architecture is a standard dense transformer. [vLLM](/tools/vllm) supports Yi since v0.4.0. Yi-34B on [L40S](/hardware/nvidia-l40s) with vLLM achieves ~6,000 tokens/second at batch 64. The 200K context requires vLLM's long-context optimizations (paged attention with extended KV-cache management). vLLM handles the NTK-aware RoPE scaling correctly for Yi-34B-200K. [Ollama](/tools/ollama) provides one-command Yi deployment: `ollama run yi:34b`, `ollama run yi:9b`. Yi-34B at Q4 on [RTX 4090](/hardware/rtx-4090) via Ollama achieves ~35 tokens/second. Yi-34B-200K on Ollama with extended context requires `/set parameter num_ctx 131072` for 128K. [llama.cpp](/tools/llama-cpp) supports all Yi models through GGUF format. Yi-34B at Q4 on [Apple M3 Ultra](/hardware/apple-m3-ultra) with Metal achieves ~22 tokens/second. The 200K context in llama.cpp requires --rope-scale and --context-size flags set correctly. [MLX-LM](/tools/mlx-lm) provides good Yi support on Apple Silicon. Yi-34B at Q4 on [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) achieves ~25 tokens/second. [TensorRT-LLM](/tools/tensorrt-llm) supports Yi for datacenter deployment. Yi-34B FP8 on [H100 SXM](/hardware/nvidia-h100-sxm) achieves ~10,000 tokens/second at batch 128. [Text Generation Inference](/tools/text-generation-inference) supports Yi through standard HuggingFace pipeline. Yi-VL models require TGI's multimodal support (TGI 2.0+). [Transformers](/tools/transformers) supports Yi with the `yi` architecture tag. Yi-VL models use the `yi-vl` tag with LLaVA-style multimodal architecture. Yi-VL vision models require specific runtime support: the CLIP ViT-L vision encoder must be loaded alongside the LLM. [vLLM](/tools/vllm) supports Yi-VL since v0.5.0 with multimodal pipelines. [Ollama](/tools/ollama) supports Yi-VL through the llava model type (Yi-VL uses the same multimodal architecture as [LLaVA](/families/llava)).

Yi models quantize predictably well as standard dense transformers. The bilingual training data produces weight distributions similar to [Llama](/families/llama). GGUF via [llama.cpp](/tools/llama-cpp) is the standard path. Yi-34B at Q4_K_M (~19 GB) loses ~0.4% MMLU vs FP16. At Q3_K_M (~15 GB), ~2.0% MMLU degradation. The IQ-quant variants perform comparably to K-quant for Yi — the weight distributions lack the extreme outlier channels that make IQ-quant disproportionately beneficial for some families. Yi-6B at Q4_K_M (~4 GB) loses ~0.3% MMLU, effectively lossless. Yi-9B at Q4 (~5.5 GB) loses ~0.3%. The smaller models are particularly quantization-robust. GPTQ and AWQ support is available. Yi-34B AWQ 4-bit (~19 GB) is lossless on MMLU (<0.2% degradation). AWQ is the recommended format for vLLM production deployment. FP8 via [TensorRT-LLM](/tools/tensorrt-llm) and [vLLM](/tools/vllm) is effectively lossless (<0.1% degradation). FP8 KV-cache quantization saves ~2× KV-cache memory. Yi-34B-200K has a specific quantization consideration: at 200K context, KV-cache memory (even at 4-bit KV-cache) is ~12 GB, dwarfing the model weight memory of ~19 GB at Q4. Aggressive weight quantization (Q3, ~15 GB) saves only 4 GB while hurting quality, whereas KV-cache quantization (4-bit, saving ~6 GB vs FP16 KV-cache) provides better memory savings with less quality impact. For Yi-34B-200K deployments, prioritize KV-cache quantization over aggressive model weight quantization. Practical rule: Yi-6B/9B handle Q3_K_M well (~1.5% loss); Yi-34B needs Q4_K_M for production (~0.4% loss). For Yi-34B-200K, Q4_K_M model weights + Q4 KV-cache fits in ~31 GB — the practical VRAM target for full-context deployment.

Yi's ecosystem is moderate-sized but strategically important as one of the leading Chinese startup model families. The ecosystem reflects 01.AI's position as a well-funded Chinese AI startup competing with [Qwen](/families/qwen) (Alibaba) and [DeepSeek](/families/deepseek) (DeepSeek AI). Derivative model volume is moderate (hundreds of fine-tunes). Key derivatives: Yi-34B-Chat (01.AI's own chat fine-tune), various Chinese-language fine-tunes on ModelScope, Yi-VL (multimodal), and Yi-Coder (code). The Chinese ecosystem on ModelScope is active, with fine-tunes targeting Chinese enterprise use cases. The global HuggingFace ecosystem is less active for Yi than for [Qwen](/families/qwen) due to Qwen's broader multilingual coverage. License clarity is a major strength: Apache 2.0 covers all Yi open-weight models. Full commercial use, redistribution, modification, and fine-tune distribution are permitted without restriction beyond standard Apache 2.0 attribution and patent grant. This permissive licensing makes Yi one of the safest legal bets in open-weight AI, alongside [Qwen](/families/qwen) (Apache 2.0 for most variants) and [Phi](/families/phi) (MIT). The Apache 2.0 license enables enterprise adoption without license negotiation. Training data transparency is limited. 01.AI discloses training data composition percentages (English 50%, Chinese 30%, multilingual 20%) but does not release the actual datasets. This is comparable to [Llama](/families/llama) and [Qwen](/families/qwen). The Chinese ecosystem is Yi's primary community. ModelScope hosts hundreds of Yi fine-tunes. Chinese enterprise adoption of Yi for internal tools is growing, driven by Apache 2.0 licensing and strong Chinese-language capability at moderate parameter counts (34B vs Qwen 72B's larger requirement). The 34B parameter count is practical for Chinese enterprise self-hosting on 2-4 GPU nodes, while 72B+ models require larger clusters. Ecosystem gaps: global (non-Chinese) adoption trails [Qwen](/families/qwen) significantly; tool-use and agent frameworks have limited Yi integration; the shift of 01.AI's focus to Yi-Lightning (closed-weight commercial) may reduce future open-weight Yi investment. The ecosystem's dependence on a single startup's continued investment in open-weight releases is a sustainability risk — if 01.AI shifts fully to closed-weight, the Yi open-weight ecosystem would stagnate.

Yi fine-tuning is standard for dense transformer models, benefiting from Apache 2.0 licensing that permits fine-tune distribution. LoRA and QLoRA via HuggingFace PEFT is the standard path. For Yi-34B, QLoRA rank 32 with 4-bit base model on [RTX 4090](/hardware/rtx-4090) 24 GB uses ~22 GB VRAM (tight) and trains 10K samples in ~10 hours. For Yi-9B, QLoRA rank 64 on [RTX 4090](/hardware/rtx-4090) uses ~18 GB VRAM and trains in ~3 hours. For Yi-6B, QLoRA on [RTX 3060 12GB](/hardware/rtx-3060-12gb) uses ~10 GB VRAM and trains in ~2 hours. Full fine-tuning: Yi-34B needs ~270 GB VRAM (4× [A100 80GB](/hardware/nvidia-a100-80gb-sxm) with ZeRO-3). Yi-9B needs ~72 GB VRAM (1× [A100 80GB](/hardware/nvidia-a100-80gb-sxm)). Full fine-tuning at 34B is an enterprise investment. Bilingual fine-tuning considerations: Yi models trained on balanced English-Chinese data benefit from continued bilingual fine-tuning. Fine-tuning on English-only data can reduce Chinese quality by 5-10 points on C-Eval. To maintain bilingual capability, fine-tune with at least 20-30% Chinese data in the training mix. This bilingual preservation strategy is important for deployments serving both English and Chinese users. Axolotl supports Yi with correct chat template and LoRA configurations. [Unsloth](/tools/unsloth) does not support Yi. The fine-tuning tooling is adequate but not optimized — standard transformers + PEFT tooling without Yi-specific speed optimizations. The ModelScope fine-tuning ecosystem provides Yi-specific configs, datasets, and deployment scripts for Chinese-language enterprise fine-tuning. For Chinese-market deployments, ModelScope is the recommended resource for Yi fine-tuning. DPO alignment for Yi is possible through standard TRL pipelines. Yi-1.5 Chat models used DPO training; the recipe is not officially documented but community reproduction on Yi-34B produces 3-5 point instruction-following improvements.

Apache 2.0 license makes Yi deployment legally straightforward. No MAU threshold, no commercial-use restriction, no use-case enumeration — standard Apache 2.0 terms. The license permits redistribution, modification, and integration into proprietary products. This is the same license tier as [Qwen](/families/qwen) and more permissive than [Llama](/families/llama) (MAU clause) or [DeepSeek](/families/deepseek) (custom license). Chat template: Yi uses ChatML format with `<|im_start|>` and `<|im_end|>` tokens. Standard ChatML templates work correctly with Yi. This is the same format used by [Qwen](/families/qwen) and several other models, making Yi interoperable with ChatML-based tooling. Yi-34B-200K's 200K context requires specific VRAM provisioning. At 200K with FP16 KV-cache: ~24 GB KV-cache alone. At Q4 KV-cache: ~12 GB. The retrieval quality at extreme context positions (128K-200K) is functional but degraded (~72% Needle-in-Haystack). Test Yi-34B-200K retrieval quality on your specific document types at target context lengths before deployment. The Chinese-language training bias means Yi models may exhibit content moderation patterns aligned with Chinese regulations. Topics related to Taiwan, Tibet, Xinjiang, Tiananmen Square, and Falun Gong may trigger refusal patterns. This is similar to [Qwen](/families/qwen) and differs from [Llama](/families/llama)'s Western-aligned content moderation. Test Yi against your content policy for sensitive topics. Yi-VL deployment adds vision encoder requirements. The CLIP ViT-L encoder (~430M parameters, ~850 MB) must be loaded alongside the LLM. Image preprocessing: resize to 336×336 pixels, normalize with CLIP statistics. Multi-image inputs are supported in the Yi-VL chat template but each image adds ~400ms encoding latency on [RTX 4090](/hardware/rtx-4090). Yi's differentiator is Apache 2.0 licensing + strong Chinese capability at 34B. If Chinese-language quality is not required, other 32-35B models provide superior English and multilingual performance. 01.AI's organizational focus shift toward Yi-Lightning (closed-weight commercial) creates uncertainty for Yi open-weight's future. There is no publicly announced Yi-2 or Yi-34B successor. Deploying Yi in 2026 means committing to a model family without a confirmed future roadmap.