Baichuan

Start with Baichuan 2 13B at Q4_K_M via Ollama — fits on single RTX 3060 12GB at 8 GB VRAM. Baichuan 2 is a Chinese-English bilingual model built by Baichuan Intelligence with a custom 125K vocab tokenizer (Byte-Pair Encoding optimized for Chinese) — it processes Chinese text ~25% more token-efficiently than Llama models on Chinese-language content. The 13B scores C-Eval ~78%, MMLU ~72%, and is particularly strong on legal and medical Chinese benchmarks (Baichuan's training data emphasizes professional Chinese domains). For higher quality, Baichuan 2 53B Q4 (32 GB) requires 2× RTX 3090 or single A6000 48 GB. For consumer-level Chinese assistance, Baichuan 2 7B Q4 (~5 GB) runs on MacBook Pro M4 Max. Skip Baichuan 1 — v2 added doubled training data (2.6T tokens) and Chinese-optimized RLHF. Apache 2.0 license. However, Qwen 3 now outperforms Baichuan 2 on every Chinese benchmark — Baichuan's development cadence has slowed significantly.

For single-user local: Ollama + baichuan2:13b Q4_K_M on RTX 3060 12GB or Apple M3 via MLX-LM. Baichuan uses standard dense transformer with RoPE and GQA (v2) — any Llama-compatible engine works. For multi-user serving: vLLM 0.6.0+ with AWQ 4-bit on L4 24 GB for 13B variant. Baichuan's custom chat template uses <reserved_106> for user and <reserved_107> for assistant — this is non-standard and must be configured in the engine's chat template. For Chinese legal/medical deployments: Baichuan 2 53B FP16 on 2× A100 80GB with vLLM is the recommended professional stack — the domain-specific training provides measurable quality advantages on Chinese professional exams vs general-purpose bilingual models. For alternatives with active development, switch to Qwen 3 using identical hardware — Qwen 3 32B at Q4 requires the same VRAM as Baichuan 2 13B FP16 and outperforms it on all benchmarks.