Yi 1.5 34B

01.AI's 34B model. Solid bilingual EN/ZH performance, Apache 2.0.

License: Apache 2.0·Released May 12, 2024·Context: 16,384 tokens

BLK · VERDICT

Our verdict

OP · Eruo Fredoline|VERIFIED JUN 12, 2026

7.4/10

Performance evidence

Performance statements in this entry have not been tied to a published, configuration-matched run. Treat speed figures as unverified estimates, not validated measurements. Compare the full configuration of a published benchmark before relying on throughput or latency.

Positioning

A capable 34B-class generalist with good multilingual support, especially Chinese. Right pick for users who specifically want 01.AI's training distribution, or as a known-quantity baseline.

Strengths

34B fits 24 GB at Q4_K_M — full GPU on a 4090.
Apache 2.0 license — clean commercial terms.
Strong Chinese-English — better than Qwen 2.5 32B on Chinese-specific tasks.

Limitations

Beaten by Qwen 3 32B on most general benchmarks.
Long-context recall weaker than spec.
Knowledge cutoff dated — 2024-era data.

Historical performance claims (unverified) on RTX 4090

Q4_K_M (20.7 GB): 65–80 tok/s decode — full GPU
Q5_K_M (24.4 GB): partial offload, 22–30 tok/s
Q8_0 (37 GB): workstation territory

Should you run this locally?

Yes, for Chinese-English-specific work, or as an Apache-licensed alternative to Qwen. No, for general English work — Qwen 3 32B is stronger at the same VRAM.

How it compares

vs Qwen 3 32B → Qwen wins on general capability; Yi has cleaner license.
vs Mistral Small 3 24B → Mistral wins on instruction polish; Yi has slight edge on Chinese.
vs Llama 3.3 70B → Llama 3.3 70B is much smarter; Yi 34B is the full-GPU pick.

Run this yourself

ollama pull yi:34b-chat-v1.5-q4_K_M
ollama run yi:34b-chat-v1.5-q4_K_M

Example settings (not a measured run): Q4_K_M GGUF, 8192 ctx, full GPU on RTX 4090

›Why this rating

7.4/10 — 01.AI's 34B that fits 24 GB at Q4. Solid, but mostly eclipsed by Qwen 3 32B and Mistral Small 3 24B. Loses points by sitting in an awkward middle without a clear differentiator.

Overview

01.AI's 34B model. Solid bilingual EN/ZH performance, Apache 2.0.

How to run it

Yi 1.5 34B is 01.AI's dense 34B model. Check the Ollama Yi listing and select a tag that explicitly identifies the 1.5 model and quantization. The earlier ollama pull yi:34b command selects the default 34B package; verify its version before comparing results.

For llama.cpp, download a compatible Yi 1.5 34B GGUF. The earlier example flags were -ngl 999 -fa -c 8192; verify them against the installed release, start with shorter context if needed, and inspect GPU allocation. A 34B Q4_K_M weight budget is about 20.5 GB before KV cache and runtime, so a 16 GB card requires weight offload as well as any cache changes.

A 24 GB GPU can be a starting point with a checked artifact and context budget. It does not guarantee 16K or 32K context. Yi-34B-200K is a different variant; do not transfer its context limit to this entry. No measured throughput is linked for these example settings.

Yi 1.5 9B is a smaller alternative. For another model near this parameter count, see Qwen 3 32B.

Hardware guidance

For 34B total parameters, Q4_K_M weights are approximately 20.5 GB before KV cache and runtime. Check the downloaded artifact size. A 24 GB RTX 3090 or RTX 4090 leaves limited memory for context. A 32 GB RTX 5090 provides more headroom, but a fixed 32K context is not established here.

A 16 GB GPU cannot hold those Q4 weights, even with KV cache moved to RAM; weight offload or a smaller artifact is also necessary. System RAM must hold the offloaded working set and other applications. AWQ and GGUF sizes differ, so use the actual file and runtime allocation. On a Mac, reserve unified memory for macOS before evaluating fit. No matched timing evidence is linked for these configurations.

What breaks first

200K context variant confusion. Yi-34B-200K exists alongside Yi 1.5. The 200K variant uses different RoPE scaling — don't mix GGUF files. 2. Chat template. Yi uses a specific chat template. Standard Llama templates produce garbled Chinese and poorly structured English. Use Yi's template. 3. Q3 quality on Yi. Yi 34B at Q3 loses more factual accuracy than similarly-sized Llama models. The Yi architecture appears more quant-sensitive. Use Q4_K_M minimum. 4. 01.AI licensing. Yi's license has specific restrictions. Verify commercial use terms — 01.AI's license may differ from Apache/MIT. Check on huggingface.co/01-ai/Yi-1.5-34B.

Runtime recommendation

Ollama for quick-start (Yi is commonly available). llama.cpp for production. vLLM for serving. Yi architecture is well-supported — no special kernels needed. MLX-LM on Apple Silicon. Standard inference stack applies.

Common beginner mistakes

Mistake: Pulling yi:34b and getting the 200K variant. Fix: Verify the exact tag — Yi-34B-200K is a different model with different RoPE scaling. Check ollama show for model details. Mistake: Using Llama chat template. Fix: Yi has a specific template. Check hf tokenizer_config.json. Wrong template = broken Chinese and messy English outputs. Mistake: Expecting Yi 1.5 to match Llama 3.1 70B quality. Fix: 34B ≠ 70B. Yi 1.5 34B is strong for its size but doesn't match 70B-class models. It outperforms similarly-sized Llama models in Chinese, but English quality is comparable. Mistake: Ignoring Yi's commercial license restrictions. Fix: 01.AI's license may restrict commercial use differently than Meta's Llama license. Verify before production deployment.