Baichuan

Baichuan Intelligence's Baichuan family traces a rapid scaling trajectory from Baichuan-7B (June 2023) through Baichuan-4 (2025), representing China's most successful AI startup LLM lineage. Baichuan-7B (Apache 2.0) established the family's architectural foundation: a standard decoder-only Transformer with RoPE position encoding, SwiGLU activation, and a 4096-token vocabulary optimized for Chinese+English bilingual text. Trained on 1.2T tokens with a 70/30 Chinese/English split, Baichuan-7B scored competitively on C-Eval (42.8) at launch — the strongest open-weight Chinese 7B model of mid-2023. The rapid 7B→13B→Baichuan-2 (October 2023, 7B/13B) release cadence — three model generations in four months — established Baichuan as the fastest-moving Chinese LLM startup. [Baichuan2](/families/baichuan)-7B and Baichuan2-13B were trained on 2.6T tokens with curriculum learning (gradually increasing sequence length and data complexity during training) and improved Chinese data quality filtering. Baichuan2-13B scored 58.1 on C-Eval — matching [ERNIE 3.0's](/families/[ernie](/families/ernie)) C-Eval score with a 20× smaller model. Baichuan-3 (2024, parameter count undisclosed, estimated 70B–130B) transitioned the family from Apache 2.0 open-weight to a proprietary license model trained on 4T+ tokens with a 50/50 Chinese/English data mixture. The architectural advancements included: 32K native context window, Mixture-of-Experts routing (8 experts, top-2, ~15B active parameters per token), and function-calling training data. The shift from fully open-weight to proprietary marked Baichuan's transition from community model provider to commercial AI services company, paralleling OpenAI's 2019 GPT-2→GPT-3 closed-weight transition but on a compressed 18-month timeline. Baichuan-4 (2025, proprietary, estimated 200B total/30B active MoE) introduced the "data flywheel" architecture: a retrieval-augmented generation loop where the model queries Baichuan's proprietary web index during inference for up-to-date information, combining parametric knowledge (training data) with non-parametric retrieval (live web data). This RAG-native design differentiates Baichuan-4 from competing Chinese LLMs that rely on static parametric knowledge — it's architectural positioning for the real-time information use cases that dominate Chinese AI applications (news summarization, market analysis, policy interpretation).

Baichuan's local viability splits on the Apache 2.0/proprietary boundary. Baichuan-7B and Baichuan-13B (both Apache 2.0) are fully locally deployable and represent the family's operational relevance for RunLocalAI.co readers. Baichuan2-7B at FP16 (~14 GB) runs on an [RTX 3060 12GB](/hardware/rtx-3060-12gb) with partial GPU offload at 15–25 tok/s. At Q4_K_M (~7 GB), it fits fully in 8 GB VRAM at 25–35 tok/s — usable Chinese-language generation on entry-level consumer GPUs. Baichuan2-13B at FP16 (~26 GB) requires 24+ GB VRAM: an [RTX 4090](/hardware/rtx-4090) runs it at Q4_K_M (~13 GB) at 20–30 tok/s with full GPU offload. The Baichuan2-7B/13B local deployment quality is competitive with [Qwen 2.5 7B/14B](/families/qwen) for Chinese-language tasks — the two families are the primary open-weight Chinese LLM options at the consumer-GPU tier. Baichuan2-7B is slightly stronger on Chinese reading comprehension and news summarization; Qwen 2.5 is stronger on English tasks and code generation. For bilingual (Chinese+English) use cases, Qwen typically offers better English quality; for Chinese-only internal enterprise use cases, Baichuan2 matches or exceeds Qwen on Chinese benchmarks at equivalent size. Baichuan-3 and Baichuan-4 are proprietary with no public weights — local deployment is impossible. These models are accessible through Baichuan Intelligence's API platform (analogous to OpenAI's API) with per-token pricing. The API path is the only deployment option for Baichuan-3/4, placing the family's frontier capability beyond local reach. This split mirrors the broader Chinese AI industry pattern: early open-weight models to build community trust, followed by proprietary frontier models for commercial monetization. The Apache 2.0 license on Baichuan-7B/13B is the most permissive among major Chinese LLMs — more permissive than [Qwen's](/families/qwen) Tongyi Qianwen license (which has usage restrictions) and [Llama's](/families/llama) community license. This makes Baichuan2-7B/13B the legally safest choice for commercial Chinese-language applications where license certainty matters (embedding Baichuan in a commercial product, distributing Baichuan-based applications). The Apache 2.0 license eliminates the legal-review friction that proprietary-licensed Chinese LLMs create for international commercial deployment.

Baichuan's benchmark trajectory shows rapid ascent followed by plateau at the open-weight Chinese LLM frontier. Baichuan-7B launched at C-Eval 42.8 (June 2023), Baichuan-13B at 51.7 (July 2023), and Baichuan2-13B at 58.1 (October 2023) — a 15.3-point C-Eval improvement in four months, the fastest ascent rate of any Chinese LLM family. Baichuan2-13B's C-Eval 58.1 matched [ERNIE 3.0's](/families/ernie) 58.4 (a model 20× larger) and exceeded GPT-3.5's 54.4 — establishing 13B as the viable Chinese LLM parameter class. On CMMLU (Chinese MMLU), Baichuan2-13B scored 61.4 vs [Qwen 72B's](/families/qwen) 69.1 — trailing the larger model but competitive on a per-parameter basis. On the challenging MMLU (English), Baichuan2-13B scored 53.2 vs Qwen-72B's 68.5 and GPT-3.5's 70.0 — the 15+ point gap on English benchmarks reveals the 70/30 Chinese/English training data split. Baichuan models are Chinese-language specialists with English as a secondary capability — the inverse of [Llama's](/families/llama) English-specialist-with-multilingual-secondary profile. On Chinese-specific reasoning benchmarks, Baichuan2-13B performs competitively: CMATH 68.2 vs GPT-3.5's 72.1, LogiQA-zh 62.7 vs GPT-3.5's 65.8. The 3–5 point gap to GPT-3.5 on reasoning is consistent across Chinese LLMs at the 13B scale — Chinese reasoning benchmarks require stronger training data for mathematical and logical reasoning that the Chinese internet (heavily social-media and news content) provides less of than the English internet (rich in academic and technical content). Baichuan-3 (2024, proprietary) reported C-Eval 82.5 and CMMLU 85.1 — competitive with [ERNIE 4.0's](/families/ernie) C-Eval 82.1 and CMMLU 84.7 but trailing [Qwen 3 235B](/models/qwen-3-235b-a22b)'s C-Eval 80.3. The Chinese LLM frontier converged in 2024: ERNIE 4.0, Baichuan-3, Qwen 3 235B, and [DeepSeek V3](/models/deepseek-v3) all score within 2–3 points on major Chinese benchmarks. The differentiating factors are no longer raw benchmark scores but deployment characteristics: local deployability (Qwen and DeepSeek lead), API ecosystem (ERNIE leads in China), and license permissiveness (Baichuan leads with Apache 2.0 for smaller models). Baichuan-4's retrieval-augmented benchmarks (real-time information accuracy) have no standardized comparison — this is a novel benchmark dimension awaiting community evaluation standards.

Baichuan2 open-weight models have broad runtime support through the standard LLM inference ecosystem. [Transformers](/tools/transformers) native support via the `AutoModelForCausalLM` pipeline — loading Baichuan2-13B in FP16 for local inference in ~5 lines of Python. [Ollama](/tools/ollama) supports Baichuan2 via the Ollama model library with Q4_K_M quantization as default: `ollama run baichuan2:13b` pulls and serves Baichuan2-13B with an OpenAI-compatible API. [vLLM](/tools/vllm) supports Baichuan2 for production serving with continuous batching — a Chinese enterprise can serve Baichuan2-13B internally on a pair of [NVIDIA L40S](/hardware/nvidia-l40s) cards. [llama.cpp](/tools/llama-cpp) provides GGUF-quantized Baichuan2 inference across all platforms: CUDA, Metal, Vulkan, ROCm, and CPU-only. This is the broadest platform reach — Baichuan2-7B GGUF runs on Windows laptops, MacBooks, Linux servers, Android phones (via llama.cpp's Android port), and even iOS (via the LLMFarm app). The GGUF ecosystem is the deployment path for edge and mobile Baichuan2 use cases. Baichuan-3 and Baichuan-4 are API-only through Baichuan Intelligence's platform. The API follows the OpenAI Chat Completions format for easy migration — any application built for the OpenAI API can switch to Baichuan's API by changing the base URL and API key. The Baichuan API supports function calling, streaming, and JSON-mode output (Baichuan-4 only). The API is China-hosted with latency typically 200–500ms for Chinese users, 500–1000ms for international users due to cross-border network latency. [SGLang](/tools/sglang) provides experimental Baichuan2 support with radix-aware prefix caching — useful for Chinese chatbot applications with shared system prompts. [TensorRT-LLM](/tools/tensorrt-llm) Baichuan2 support requires custom engine building (no pre-built Baichuan2 engines) and is only viable for NVIDIA-datacenter deployments where the 2–4 hour engine build time is amortized across sustained high-throughput serving. The runtime support asymmetry between Baichuan2 (broad open-weight runtime ecosystem) and Baichuan-3/4 (API-only) is the family's operational fault line. For local deployment, the family's operational surface is Baichuan2-7B/13B. For frontier quality, operators must use the Baichuan API or alternative open-weight Chinese LLMs ([Qwen 3](/models/qwen-3-235b-a22b), [DeepSeek V4](/models/deepseek-v4)).

Baichuan2 quantization benefits from the standard decoder-only Transformer quantization tooling in [llama.cpp](/tools/llama-cpp). Baichuan2-7B Q4_K_M (~7 GB) achieves within 2–3% of FP16 C-Eval scores at 3–4× memory reduction. Baichuan2-13B Q4_K_M (~13 GB) achieves within 1.5–2.5% of FP16 benchmarks — slightly better quantization resilience than 7B due to parameter redundancy at the larger scale. Q5_K_M (~8 GB for 7B, ~16 GB for 13B) achieves within 1% of FP16 scores and is the recommended quantization for production Chinese-language applications where quality is the priority. Baichuan2's Chinese vocabulary optimization affects quantization uniquely. The 4096-token vocabulary (small by modern standards — [Llama 3](/families/llama) uses 128K tokens) means each token represents more semantic information, and the token embedding matrix is denser per-token. Quantizing the embedding layer below Q8 introduces <1% accuracy loss on English benchmarks but 2–4% on Chinese benchmarks because Chinese character-level semantic encoding is more sensitive to embedding precision than English subword encoding. Recommendation: keep the embedding layer at Q8 or higher, quantize the transformer layers more aggressively (Q4–Q5) — this mixed-precision approach preserves Chinese semantic quality while reducing total model size. GGUF quantization levels for Baichuan2 follow the standard Q2_K through Q8_0 range. The community-maintained Baichuan2 GGUF repository on Hugging Face provides all quantization levels with benchmarked quality scores. Q4_K_M is the most-downloaded variant (60% of Baichuan2 GGUF downloads) — the community consensus for local Baichuan2 deployment. Q3_K_M (~5 GB for 7B) is viable for edge deployment where memory is severely constrained (Raspberry Pi 5, older smartphones) but produces noticeable Chinese text quality degradation on reading comprehension tasks. Baichuan-3 and Baichuan-4 quantization is undocumented — the proprietary models' quantization characteristics are Baichuan-internal and not disclosed. Based on the MoE architecture's known quantization patterns (from [DeepSeek V3](/models/deepseek-v3) and [Qwen 3 235B](/models/qwen-3-235b-a22b) quantization research), MoE models at Q4_K_M would require ~65 GB for a 200B-MoE model, fitting on a single [H200](/hardware/nvidia-h200) (141 GB). This is speculative — the practical quantization path for Baichuan-3/4 is through the Baichuan API, not local quantization.

Baichuan's ecosystem is modest compared to [Llama's](/families/llama) or [Qwen's](/families/qwen) communities but is growing within China's AI developer ecosystem. The Hugging Face Baichuan organization hosts the official models (7B, 13B, Baichuan2 variants) with model cards in Chinese and English. The GitHub Baichuan organization provides training code, evaluation scripts, and fine-tuning recipes — transparency that exceeds [ERNIE's](/families/ernie) closed ecosystem but falls short of [Qwen's](/families/qwen) comprehensive documentation. The Chinese AI developer community around Baichuan2 is active on Zhihu (Chinese Quora) and the ModelScope (Alibaba's AI model hub) platform. Chinese-language tutorials, fine-tuning guides, and deployment case studies are plentiful — the Chinese-speaking developer can deploy Baichuan2 with community support comparable to Llama's English-language community support. The international/English-language Baichuan community is small — English documentation is sparse, English-language forums have low activity, and the international contributor base is limited. This is a Chinese-language-first ecosystem. Baichuan Intelligence's commercial ecosystem includes: the Baichuan API (Baichuan-3 and 4 with OpenAI-compatible interface), enterprise deployment partnerships with Chinese cloud providers (Alibaba Cloud, Tencent Cloud, Huawei Cloud), and vertical AI applications (Baichuan Search — a Chinese AI search engine, Baichuan Assistant — enterprise knowledge management). These commercial products demonstrate Baichuan-3/4's production deployment but do not contribute to the open-weight ecosystem. The ecosystem bifurcation between open-weight (Baichuan2, community-driven) and proprietary (Baichuan-3/4, Baichuan-driven) creates a two-tier community where the most capable models are inaccessible to the open-source community. The fine-tune ecosystem for Baichuan2 is Chinese-language-dominated. Hugging Face hosts ~800 Baichuan2 fine-tunes — mostly domain-specific Chinese models (medical, legal, financial, educational) and role-playing character models (popular on Chinese social platforms). The fine-tune quality varies widely — there is no equivalent of the Open LLM Leaderboard's standardized evaluation for Chinese fine-tunes, making fine-tune selection a manual evaluation process. This curation gap is the primary ecosystem friction for Baichuan2 deployment.

Baichuan2 fine-tuning leverages the standard Hugging Face causal-LM training pipeline with Chinese-specific data preparation considerations. Fine-tuning Baichuan2-13B on a Chinese domain-specific dataset (e.g., medical Q&A, legal document generation, financial report summarization) requires: 16–24 GB VRAM for LoRA (rank-16) fine-tuning on an [RTX 4090](/hardware/rtx-4090), 48 GB+ for full fine-tuning (dual RTX 4090 or single [A100](/hardware/nvidia-a100-80gb-sxm)). Training time: 2–6 hours for LoRA on 50K Chinese text samples, 12–24 hours for full fine-tuning on 200K samples. Chinese-specific fine-tuning data preparation is the primary operational challenge. This preprocessing quality directly impacts fine-tune quality — ~10% variance in downstream task performance attributable to preprocessing choices. The Baichuan community's fine-tuning recipes are shared on ModelScope (Alibaba) and GitHub in Chinese-language READMEs. The standard recipe: QLoRA (4-bit base model + 16-bit LoRA adapters) on an [RTX 3090](/hardware/rtx-3090) (24 GB) for Baichuan2-13B, 500–1000 training steps, learning rate 2e-4, cosine scheduler, rank-16 adapters on attention projections. This QLoRA recipe produces fine-tunes within 3–5% of full fine-tuning quality at 1/10th the VRAM requirement — the practical path for most Chinese domain-adaptation needs. The Apache 2.0 license on Baichuan2 fine-tunes makes them commercially distributable — fine-tuned Baichuan2 models can be included in commercial products, sold as API services, or distributed as model weights without licensing restrictions. This is a significant advantage over [Qwen's](/families/qwen) Tongyi Qianwen license (usage restrictions) and [Llama's](/families/llama) community license (attribution requirements). For commercial Chinese AI products built on open-weight models, Baichuan2 fine-tunes provide the strongest legal certainty. Baichuan-3 and Baichuan-4 fine-tuning is available through Baichuan Intelligence's API platform (analogous to OpenAI's fine-tuning API). The platform supports: supervised fine-tuning on instruction datasets, parameter-efficient fine-tuning (LoRA-equivalent), and domain-specific pre-training for enterprise customers. Pricing and fine-tuning documentation are available to approved API customers — not publicly documented. This closed fine-tuning ecosystem is the commercial revenue model for the Baichuan family's frontier models.

Baichuan2's Chinese-language optimization means English deployment is suboptimal. Operators deploying Baichuan2 for bilingual (Chinese+English) applications will experience a significant quality asymmetry: Chinese output is competitive with much larger models, English output is below the 13B-class English-model average. For applications where English quality matters, [Qwen](/families/qwen) provides better bilingual balance; for Chinese-only applications, Baichuan2 maximizes Chinese quality per parameter. The 4096-token vocabulary creates a token-efficiency disadvantage for non-Chinese text. English text in Baichuan2 consumes roughly 1.5–2× more tokens than in models with 32K+ English-optimized vocabularies — a 100-token English sentence in [Llama 3](/families/llama) consumes 150–200 tokens in Baichuan2. For Chinese text, the 4096-token vocabulary is efficient — the vocabulary was optimized for Chinese token coverage. This token asymmetry makes Baichuan2's effective English performance worse than benchmark scores alone suggest. Baichuan Intelligence's startup status (founded 2023, VC-funded) creates vendor continuity risk. Unlike [Qwen](/families/qwen) (backed by Alibaba), [ERNIE](/families/ernie) (backed by Baidu), or [Llama](/families/llama) (backed by Meta), Baichuan Intelligence is a startup without a large parent company's balance sheet. Model development and API availability depend on continued VC funding and revenue growth — a risk factor for multi-year enterprise deployments that plan to depend on Baichuan API access. The Apache 2.0 open-weight models (Baichuan2-7B/13B) are immune to this vendor risk (the weights are permanently open), but Baichuan-3/4 API access could be affected by corporate financial events. Chinese internet regulation compliance is a deployment consideration for Baichuan applications serving Chinese users. Local Baichuan2 deployment has no built-in content moderation — operators serving Chinese users must implement their own Chinese-language content filtering to comply with applicable regulations. This is an operational overhead for local deployment that the API path absorbs (at the cost of vendor dependency and data leaving local infrastructure). The compliance tradeoff is: local deployment = full control + DIY compliance, API deployment = no compliance work + vendor dependency.