Qwen

Qwen 1 (November 2023) launched as Alibaba's first open-weight LLM family — Qwen-7B, Qwen-14B, and Qwen-72B. Architecture: dense transformer decoder with pre-norm RMSNorm, SwiGLU activation, and RoPE position embeddings. Vocabulary of 152K tokens using tiktoken-based BPE tokenizer. The Qwen-1.5 refresh (February 2024) was architectural housekeeping — added GQA, extended context to 32K via YaRN interpolation, and standardized the vocabulary. Qwen-1.5-72B adopted the Llama-like GQA ratio of 8 KV heads, making it compatible with every [Llama](/families/llama)-targeted inference engine with minimal config changes. Qwen 2 (June 2024) was the architectural branching point. The 72B dense doubled context to 128K using YaRN with dynamic NTK scaling. Training data volume reached ~7T tokens. The MoE architecture introduced hash-based expert routing (different from [DeepSeek V3](/families/deepseek)'s learned routing) — each token is deterministically assigned to a subset of experts based on a hash function, eliminating the load-balancing loss term needed in learned MoE routing. This architectural decision makes Qwen MoE models easier to serve than [DeepSeek](/families/deepseek) MoE models because the expert assignment is deterministic and distribution is uniform by construction. Qwen 2.5 (September 2024) expanded the family with Qwen2.5-14B, Qwen2.5-32B, Qwen2.5-72B dense models and the Qwen2.5-Coder branch. The signature achievement was Qwen2.5-32B matching or exceeding [Llama 3.1 70B](/families/llama) on MMLU and code benchmarks while requiring half the parameters. This efficiency came from continued pretraining on 18T tokens with higher-quality data filtering (Qwen's data filtering pipeline is distinct in using teacher-model scoring for data quality assessment at web scale). The 235B MoE is the current architectural flagship — 235B total parameters with 22B active per token using top-8 expert routing. Shared expert architecture places 1 "shared expert" always activated plus 7 "routed experts" selected by hash. Qwen3 training data volume: ~36T tokens. The 235B MoE achieves frontier reasoning capability at ~12× lower per-token compute than dense 235B would require. Qwen3-Extension (Qwen3-5-235B, March 2026) extended the 235B architecture with enhanced tool-use projection layers and agentic finetuning.

Qwen's local viability trajectory is the most dramatic in open-weight AI — from "barely runnable by consumers" to "best-in-class local deployment efficiency" in 18 months. Qwen 1 72B at release was datacenter-grade. The 14B ran on a single [RTX 3090](/hardware/rtx-3090) 24 GB at Q4 but with visible quality degradation because Qwen 1's outlier channel distribution made uniform quantization more damaging than on [Llama](/families/llama). Qwen 1 was functionally a cloud model for Western consumers — the Chinese deployment ecosystem (ModelScope, Alibaba Cloud PAI) was the intended self-host target. Qwen 1.5 changed this trajectory by adopting GQA, making 72B Q4 (~40 GB) fit on 2× [RTX 4090](/hardware/rtx-4090) or a single [A6000](/hardware/nvidia-a40) 48 GB. The 32B at Q4 (~18 GB) fit on a single [RTX 4090](/hardware/rtx-4090). By Qwen 2.5, the 32B at Q4_K_M became the default recommendation for single-GPU local deployments — it fits comfortably on [RTX 5090](/hardware/rtx-5090) 32 GB and [Apple M3 Ultra](/hardware/apple-m3-ultra) Macs using [MLX-LM](/tools/mlx-lm). Qwen 3 235B-A22B MoE at FP16 requires ~470 GB total model memory, but only ~44 GB needs to be in high-speed memory for active expert weights per forward pass. On an [MI300X](/hardware/amd-mi300x) 192 GB, ~15 tokens/second at Q4. Above Q4, the model fits comfortably in GPU memory and expert offloading is unnecessary. The MoE architecture makes Qwen 3 235B the most locally-viable frontier-scale model in 2026 — it achieves [DeepSeek V4](/families/deepseek)-tier reasoning at consumer-hardware-friendly VRAM requirements. The trajectory from Qwen 1→3: getting dramatically more local-friendly over time. Qwen 3 8B runs on a [MacBook Pro M4 Max](/hardware/macbook-pro-16-m4-max) at 20+ tokens/second. The 32B at Q4 on [RTX 4090](/hardware/rtx-4090) is the standard local recommendation. The 235B MoE with expert offloading is the frontier-local ceiling. Qwen's local viability is now arguably better than [Llama](/families/llama) at the frontier scale because MoE expert offloading dramatically reduces GPU VRAM requirements vs dense [Llama 3.1 405B](/families/llama). However, a caveat: Qwen's Chinese-language training bias means models with < 10% Chinese representation in training data are more query-efficient for Western-language users. For bilingual/Chinese workflows, Qwen tokenizer efficiency is superior.

Qwen 1 (November 2023) entered with competitive scores: Qwen-72B scored 77.5% on 5-shot MMLU, 72.5% on C-Eval (Chinese knowledge benchmark), and 58.1% on HumanEval pass@1. The Chinese-English split was Qwen's differentiator — CMMLU 83.3% and C-Eval 72.5% at release exceeded every non-Chinese open-weight model by 20+ points on Chinese benchmarks. On English-only benchmarks, Qwen-72B was competitive with [Llama 2 70B](/families/llama) (68.9% MMLU) — a strong showing for a bilingual-first model. Qwen 1.5 72B improved MMLU to 81.5%, GSM8K 8-shot to 76.8%, and HumanEval to 63.4%. The standout was MATH 500: Qwen 1.5 72B scored 42.0% (5-shot), significantly above [Llama 3 70B](/families/llama) at release (34.2%). This math advantage would become Qwen's signature benchmark strength over the next two years. Qwen 1.5 110B Chat scored MMLU 82.3% and GSM8K 82.2% — the first time an open-weight model exceeded GPT-3.5-turbo comprehensively. Qwen 2 (June 2024) delivered the MoE efficiency breakthrough. Qwen2-72B scored MMLU 84.2%, GSM8K 89.5%, and HumanEval 79.3% — largely matching [Llama 3.1 70B](/families/llama). The MoE variant (57B-A14B) scored MMLU 79.0% with only 14B active parameters — approximately the quality of [Llama 3 70B](/families/llama) at 5× lower compute. On IFEval strict-prompt, Qwen2-72B scored 72.1% — instruction-following quality competitive with GPT-4 at release. Qwen 2.5 (September 2024) pushed MMLU to 86.5% for the 72B, GSM8K to 93.7%, and MATH 500 to 78.0%. The signature result was on LiveCodeBench: Qwen2.5-72B at 43.2% exceeded [DeepSeek Coder V3](/families/deepseek) at release (41.8%) — Qwen had caught DeepSeek on code benchmarks through sheer data scale. HumanEval+ (extended evaluations) at 77.4% for Qwen2.5-72B was within 2 points of GPT-4-Turbo. Qwen 3 235B-A22B (January 2026) represents Qwen's frontier benchmark entry. MMLU 90.1%, GSM8K 96.3%, MATH 500 90.5%, and HumanEval 86.2% — competitive with [DeepSeek V3](/families/deepseek) and GPT-4o on reasoning. On GPQA Diamond (graduate-level reasoning), Qwen 3 235B scores 58.3% vs GPT-4-Turbo's 54.1%. On AIME 2024 competition math, 52.4% — close to [DeepSeek V4](/families/deepseek) tier. Where Qwen wins vs [Llama](/families/llama): math, code, and multilingual benchmarks. Where Qwen trails: truthfulness and refusal-accuracy tradeoffs, where [Llama 3.3 70B](/families/llama) still holds the open-weight lead on TruthfulQA.

Qwen runtime support has matured from "secondary citizen requiring custom configs" at Qwen 1 launch to "first-class support" for Qwen 2.5+ in 2026. The inflection point was Qwen 1.5's adoption of GQA architecture, which made Qwen compatible with every Llama-targeted inference pipeline with trivial config changes. Qwen 3 235B MoE on vLLM requires the MoE expert parallelism feature (vLLM 0.6.5+) and benefits from EPLB (expert-parallel load balancing) at high concurrency. On 4× [H100 SXM](/hardware/nvidia-h100-sxm), vLLM achieves ~8,000 tokens/second at batch 64 for Qwen 3 235B FP8. [SGLang](/tools/sglang) provides Qwen MoE-optimized serving with the SGLang MoE backend (v0.2.2+). Qwen 3 235B on 2× [L40S](/hardware/nvidia-l40s) at Q4 with expert offloading achieves ~35 tokens/second decode — this is the recommended path for Qwen MoE on non-datacenter NVIDIA GPUs. [Ollama](/tools/ollama) support for Qwen landed with Ollama 0.1.32 (Qwen 1.5+) using the llama.cpp backend. The canonical Ollama invocation: `ollama run qwen3:32b`. GGUF quants are maintained by the Qwen team on HuggingFace for all major model sizes, with the bartowski community providing additional IQ-quant and K-quant variants. [llama.cpp](/tools/llama-cpp) rebuilt its Qwen architecture support in the GGUF v3 format (llama.cpp b2710+). Qwen 3 32B at Q4_K_M on [Apple M3 Ultra](/hardware/apple-m3-ultra) with Metal achieves ~15 tokens/second — very strong for a 32B dense model. Qwen 3 235B FP8 on 8× [H100 SXM](/hardware/nvidia-h100-sxm) achieves ~25,000 tokens/second at batch 256 — matching Llama 3 70B throughput despite 3.3× higher total parameter count, demonstrating the throughput advantage of MoE architecture on TensorRT. [MLX-LM](/tools/mlx-lm) supports Qwen 2.5+ with first-class Apple Silicon optimizations. This is the recommended Qwen local inference path for Mac users. [Text Generation Inference](/tools/text-generation-inference) supports Qwen 2+ with the same pipeline as Llama. The Qwen chat template is correctly handled in TGI 1.4+ (earlier versions had a trailing space bug in the assistant prefix). [Transformers](/tools/transformers) (HuggingFace) has first-class Qwen support. The Qwen models load with `AutoModelForCausalLM` with zero custom code since Qwen 1.5+. The `qwen2` architecture tag in the HuggingFace hub is stable and backward-compatible across Qwen 2 → 3 generations.

Qwen quantization robustness varies significantly by generation and model size. Qwen 1 (pre-GQA) was quantization-hostile: the MHA architecture produced outlier channels in the value projection that caused catastrophic degradation below 6-bit. Qwen 1.5+ with GQA solved this — the attention head grouping naturally smooths outlier activation patterns, making uniform quantization dramatically more effective. GGUF via [llama.cpp](/tools/llama-cpp) is the most tested path. Qwen 3 32B at Q4_K_M (~18 GB) loses ~0.4% MMLU accuracy vs FP16 — effectively lossless. At Q3_K_M (~14 GB), degradation is ~1.8% MMLU and ~2.5% GSM8K. Below Q3, quality drops steeply: Q2_K on Qwen 3 32B loses 8+ MMLU points, making it unusable for serious work. IQ3_XXS Qwen 3 32B (~12 GB) achieves quality roughly equivalent to Q3_K_M at ~15% smaller filesize. GPTQ and AWQ support is mature for Qwen 2.5+. Qwen 3 32B AWQ 4-bit (~18 GB) with groupsize 128 is lossless on MMLU (<0.2% degradation). For production Qwen deployments on [vLLM](/tools/vllm), AWQ 4-bit is the recommended format. EXL2 (ExLlamaV2) provides mixed-precision for Qwen 2.5+ at arbitrary 2.0-8.0 bpw. Qwen 3 32B at 3.5 bpw (~16 GB) is the practical minimum for acceptable quality (~1.2% MMLU degradation). At 4.0 bpw (~18 GB), quality is indistinguishable from GPTQ. FP8 via [TensorRT-LLM](/tools/tensorrt-llm) and [vLLM](/tools/vllm) is the recommended datacenter format for Qwen. Qwen 3 32B FP8 is lossless vs FP16 on all benchmarks. Qwen 3 235B MoE FP8 is similarly lossless — MoE models are surprisingly robust to FP8 quantization because the expert gating mechanism provides soft redundancy that absorbs small weight perturbations. A Qwen-specific quantization note: the hash-based MoE routing in Qwen 3 MoE is sensitive to aggressive quantization of the expert-router weights. The router is a small MLP (~50M parameters in Qwen 3 235B) that deterministically assigns each token to experts. Always keep the router weights at FP16 or FP8; never quantize Qwen MoE router weights below 8-bit. The practical sizing rule: Qwen 3 8B survives Q3_K_M for casual use (~1.5% loss); Qwen 3 32B needs Q4_K_M minimum for production (~0.4% loss); Qwen 3 235B MoE needs FP8 for expert weights + at minimum Q4 for offloaded experts (~0.8% loss).

Qwen's ecosystem is the second-largest in open-weight AI after [Llama](/families/llama), with exceptional growth in 2025-2026 driven by Apache 2.0 licensing and strong Chinese-English bilingual capability. The ecosystem splits into two overlapping communities: the Chinese ecosystem (ModelScope, Alibaba Cloud PAI, Tongyi Qianwen platform) and the global HuggingFace ecosystem. Both are substantive. Derivative model volume is in the thousands and growing fast. The Qwen-Coder series has become the dominant open-weight code model for Chinese-language codebases and the second choice globally after [DeepSeek Coder](/families/deepseek). License clarity is a Qwen ecosystem superpower. Apache 2.0 means Qwen can be used for any purpose including redistribution, modification, and integration into proprietary products with only attribution and patent grant requirements. This permissiveness is the primary factor driving enterprise Qwen adoption for self-hosted deployments in regulated industries where license risk assessment is mandatory. The training data transparency for Qwen is better than [Llama](/families/llama) but still incomplete. Alibaba publishes training data composition ratios (percentage by language, domain category breakdown) but does not release the actual datasets. This is more transparent than Llama's data disclosures but less transparent than [Aya](/families/aya)'s full dataset release. The Chinese-language ecosystem around Qwen is massive and growing: ModelScope hosts thousands of Qwen fine-tunes, Qwen-powered applications, and deployment templates. The integration with Alibaba Cloud services (PAI-EAS, DSW, PAI-DLC) provides enterprise-grade deployment infrastructure in China that Western open-weight families lack. Conversely, the Western deployment ecosystem (HuggingFace, vLLM, Ollama) is fully Qwen-compatible, making the ecosystem genuinely global rather than Chinese-only. Ecosystem gaps: Qwen's tool-use and agent frameworks are less mature than [Llama](/families/llama). The function-calling benchmarks and agent frameworks (LangChain, CrewAI, AutoGen) have better Llama integration than Qwen integration. Qwen's chat template differences cause subtle compatibility issues with agentic frameworks that assume Llama-style formatting. This gap is narrowing rapidly in 2026 as framework developers add explicit Qwen support.

Qwen fine-tuning is mature and well-tooled, with the Qwen team providing first-party fine-tuning recipes for every generation. The combination of Apache 2.0 licensing (enabling fine-tune distribution without restriction) and the Qwen team's detailed technical reports creates a well-documented fine-tuning ecosystem. LoRA and QLoRA support is first-class. HuggingFace PEFT supports Qwen with the `qwen2` architecture tag for all 1.5+ generations. Training time: ~8 hours for a 10K-sample dataset at sequence length 2048. Full fine-tuning requirements parallel [Llama](/families/llama). Qwen 3 8B full fine-tune requires ~64 GB VRAM (4× parameter count). Full fine-tuning Qwen 3 is effectively a datacenter workflow; for consumer hardware, QLoRA is the normal path. For English tasks, the standard datasets (OpenOrca, ShareGPT, UltraChat) work with Qwen with the standard caveat that Qwen's chat template differs from Llama's — the `<|im_start|>` and `<|im_end|>` tokens must be correctly formatted or fine-tune quality degrades sharply. Unsloth provides accelerated Qwen fine-tuning with 2× speedup vs vanilla HuggingFace transformers. Training Qwen 3 8B on [RTX 4090](/hardware/rtx-4090) takes ~1.5 hours for a 10K-sample set with Unsloth vs ~3 hours with vanilla transformers. Unsloth's Qwen acceleration is comparable to its Llama acceleration. Axolotl has Qwen support with pre-built configs for Qwen 2+ fine-tuning. The Axolotl Qwen configs include correct chat templates, tokenizer configs, and LoRA target modules for each Qwen generation. For teams using Axolotl's YAML-based config system, switching from Llama to Qwen is a config-file change. RLHF and DPO support is available via TRL with Qwen-tested examples. Community DPO recipes for Qwen 3 8B produce measurable preference-alignment improvements in ~4 GPU-hours on a single [RTX 4090](/hardware/rtx-4090). This impacts fine-tuning VRAM because attention-compute scales quadratically with sequence length. For English-only fine-tuning on Qwen, reduce max sequence length by ~15% vs a Llama fine-tune to maintain equivalent VRAM budget. For Chinese or bilingual workloads, Qwen's tokenizer is more efficient than Llama's, partially offsetting this effect. Merging Qwen fine-tunes is possible via MergeKit with Qwen architecture support. The Qwen merging community on ModelScope is growing but smaller than the Llama merging community on HuggingFace.

Qwen's Apache 2.0 license is the family's greatest deployment advantage and carries minimal restrictions: attribution in NOTICES file, patent grant, and standard Apache 2.0 disclaimer. There is no MAU threshold, no commercial-use restriction, no military-use prohibition beyond standard export controls. This makes Qwen the safest legal bet for enterprise self-hosted deployment among frontier-capability open-weight models in 2026. Chat template compatibility is the most common Qwen deployment bug. Qwen uses `<|im_start|>` / `<|im_end|>` delimiters and a specific role ordering (system → user → assistant). Using Llama-style chat templates on Qwen produces garbled instruction-following — the model appears confused because it's receiving formatting it wasn't trained on. Tokenizer specialization causes subtle degradation on English-only workloads. Qwen's 152K vocab emphasizes CJK code points; English text consumes ~15% more tokens than on [Llama 3+](/families/llama)'s 128K vocabulary. This is usually negligible but worth factoring into capacity planning for high-volume English-only production deployments. MoE deployment for [Qwen 3](/families/qwen) 235B has specific VRAM requirements. At FP8, the full model (235B total, 22B active) consumes ~240 GB total but only ~24 GB of fast memory for active experts. Censorship and refusal behavior require awareness. Qwen 3's RLHF alignment reflects Alibaba's content policies, which differ from Western content policies. Conversely, Qwen has weaker refusals on Western-regulated content categories (self-harm, violence depiction, controlled substance information) than Llama. For deployments serving global audiences, test Qwen's refusal patterns against your content policy before production. This reduces the throughput benefit of prefix caching by ~25-30% compared to dense models where prefix caching eliminates all computation for cached tokens. For high-concurrency Qwen MoE deployments, the throughput curve has a steeper falloff with prefix diversity than dense models — plan for this in capacity modeling. Finally, API-keys for Qwen Max/Plus (closed-weight Alibaba Cloud models) are NOT covered by the Apache 2.0 license for Qwen open-weight models. Deploying the open-weight Qwen locally is Apache 2.0; using Alibaba Cloud's Qwen API is a separate commercial agreement with usage quotas and data handling terms. Do not confuse the two — the open-weight Qwen model weights are Apache 2.0, the Qwen cloud service is not.