Phi

Architecture: dense decoder-only transformer with FlashAttention, rotary embeddings, and a 50K vocabulary. The small scale was the point — Phi-1 proved that data quality dominates data quantity for small-model capability. Phi-1 scored HumanEval 50.6% pass@1, exceeding many 7B-class models despite 5× fewer parameters. Phi-1.5 (September 2023) scaled to 1.3B with enhanced training data (30B tokens including synthetic textbooks and filtered web data). The architecture was unchanged; the improvement came from data mixture. Phi-1.5 reached competitive commonsense reasoning scores but remained Python-code-specialized from Phi-1's training composition. Phi-2 (December 2023) scaled to 2.7B parameters on 1.4T tokens of synthetic + filtered data. Architecture: dense transformer with GQA (2 KV heads), SwiGLU activation, and RoPE with base frequency 10,000. This was the first Phi model with production-usable general knowledge. Phi-2's 2.7B scored MMLU 56.7%, HumanEval 59.0% — matching [Mistral 7B](/families/mistral) code benchmarks at 2.6× fewer parameters. The training data expanded to 3.3T tokens of synthetic textbook-quality + carefully filtered web data. Phi-3-mini 3.8B scored MMLU 69.1% — exceeding [Llama 3 8B](/families/llama) at half the parameters. The architectural key was the LongRope mechanism for context extension, which was novel at release. Phi-3.5 (August 2024) refined Phi-3-mini and Phi-3-small with improved post-training (DPO, RLHF). Architecture unchanged. MoE versions shipped: Phi-3.5-MoE (42B total, 6.6B active) with 16 experts and top-2 routing. The MoE variant achieved MMLU 78.9% with only 6.6B active parameters — a dramatic parameter-efficiency improvement. Phi-4 (January 2025) represents the current flagship. Phi-4 14B dense, trained on 5T tokens of synthetic data generated by teacher models and filtered for quality. Architecture: dense transformer with GQA at 14B scale, context 128K via LongRope, and continued use of the "textbook-quality" synthetic data pipeline. MMLU 84.8% at 14B scale, competitive with [Llama 3.1 70B](/families/llama). The evolution from Phi-1 to Phi-4 traces a clear trajectory: from 1.3B to 14B parameters, from code-only to general capability, from research experiment to production-grade model, while maintaining the core thesis that data quality beats parameter scale. Phi-4 14B's competitive performance with 70B-class models validates this thesis at production scale.

Phi models are the most locally-friendly models at every capability tier, reflecting the design philosophy of maximizing capability-per-parameter. The smallest useful models in open-weight AI are Phi models. Phi-1 1.3B at FP16 (~2.6 GB) runs on any modern device. On a [MacBook Pro M4 Max](/hardware/macbook-pro-16-m4-max) via [MLX-LM](/tools/mlx-lm), 150+ tokens/second. On a phone, ~15 tokens/second. Phi-1.5 at Q4 (~0.8 GB) runs comfortably on a Raspberry Pi 5. This extreme deployability makes Phi the entry point for embedded/edge LLM deployment. Phi-2 2.7B at Q4 (~1.6 GB) runs on any GPU with 2+ GB VRAM. On [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb), ~120 tokens/second via [Ollama](/tools/ollama). The quality-to-VRAM ratio is unmatched at this size class. Phi-3-mini 3.8B at Q4 (~2.3 GB) is the practical minimum for production-grade small model deployment. On [RTX 3060 12GB](/hardware/rtx-3060-12gb), ~90 tokens/second. On [Apple M3 Ultra](/hardware/apple-m3-ultra) via [MLX-LM](/tools/mlx-lm), ~70 tokens/second. The 128K context is practically deployable at this size because the KV-cache for 128K at 3.8B 4-bit is only ~2 GB — fitting comfortably within GPU memory even on low-end cards. Phi-3.5-MoE (42B total, 6.6B active) is the locally-viable MoE champion. FP16 ~84 GB total but only ~14 GB active, fitting on [RTX 4090](/hardware/rtx-4090) 24 GB with expert offloading. At Q4, ~24 GB total and ~4 GB active, fitting on [RTX 3060 12GB](/hardware/rtx-3060-12gb) with offloading. This is the only MoE model deployable on a sub-12GB VRAM GPU with useful throughput. Phi-4 14B at Q4 (~8 GB) fits on any GPU with 10+ GB VRAM. On [RTX 4090](/hardware/rtx-4090) 24 GB, ~50 tokens/second via [Ollama](/tools/ollama). On [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra), ~35 tokens/second. The 14B at Q4 on [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb) uses ~9 GB (model + KV-cache) and achieves ~30 tokens/second — a $399 GPU running a model competitive with [Llama 3.1 70B](/families/llama) on reasoning. The trajectory: Phi models have progressively increased in capability at each size tier while maintaining best-in-class local deployability. The design philosophy deliberately targets local inference as the primary deployment mode.

Phi-1 (June 2023) scored HumanEval 50.6% pass@1 at 1.3B parameters — exceeding every 7B-class model of its era on code generation. MBPP 55.5%. The model was purpose-built for code, and code benchmarks reflected this specialization. General benchmarks were weak: MMLU was not meaningfully tested at this scale. Phi-2 (December 2023) scored MMLU 56.7% (5-shot), GSM8K 57.6% (8-shot), and HumanEval 59.0% (pass@1) at 2.7B. The MMLU score exceeded [Llama 2 13B](/families/llama) (54.8%) at 4.8× fewer parameters. HumanEval 59.0% matched [Llama 2 70B](/families/llama) Instruct (59.9% at pass@10). This was the first demonstration that a sub-3B model could compete with 70B-class models on code benchmarks. Phi-3-mini 3.8B (April 2024) scored MMLU 69.1% (5-shot), GSM8K 78.1% (8-shot), HumanEval 62.2%, and IFEval 53.7%. MMLU 69.1% exceeded [Llama 3 8B](/families/llama) (68.2%) at 2.1× fewer parameters. GSM8K 78.1% exceeded [Mistral 7B](/families/mistral) (46.1%) dramatically. The parameter-efficiency margins were the largest in open-weight AI. Phi-3-medium 14B scored MMLU 78.0%, GSM8K 85.6%, HumanEval 71.8% — matching [Llama 3 70B](/families/llama) MMLU at 5× fewer parameters. This was the family's "wake-up moment" for the broader community: a 14B model from Microsoft was competitive with Meta's 70B flagship. Phi-3.5-MoE (42B total, 6.6B active) scored MMLU 78.9%, GSM8K 88.9%, HumanEval 75.2% — matching [Llama 3.1 70B](/families/llama) on MMLU and exceeding it on code. The MoE design delivered 70B-class capability at 6.6B active parameters, a remarkable engineering achievement. Phi-4 14B (January 2025) scored MMLU 84.8%, GSM8K 91.2%, HumanEval 79.4%, MATH 500 70.1%, IFEval 72.3%. These numbers are within 3-5 points of [Llama 3.1 70B](/families/llama) and competitive with [Qwen 2.5 32B](/families/qwen). Phi-4's MATH 500 at 70.1% exceeded Llama 3.1 70B Instruct (65.4%). On GPQA Diamond, Phi-4 scored 49.2% — behind [DeepSeek V3](/families/deepseek) but ahead of every other sub-30B model. The Phi benchmark identity: the steepest capability-per-parameter curve in open-weight AI. Where Phi wins: parameter efficiency (best-in-class at every size), code benchmarks (competitive with models 5× larger), and math reasoning (Phi-4's MATH 500 exceeds most 70B models).

Phi models benefit from broad runtime support due to their architectural conservatism: standard dense transformers with GQA, SwiGLU, and RoPE. Every engine that supports [Llama](/families/llama) supports Phi with trivial configuration changes. [vLLM](/tools/vllm) added Phi support in v0.4.0 for Phi-2+ with the standard Llama architecture support. Phi-4 14B on [L40S](/hardware/nvidia-l40s) at Q4 achieves ~8,000 tokens/second at batch 64. The LongRope context extension in Phi-3+ requires vLLM 0.5.0+ for correct RoPE frequency scaling at 128K context. [Ollama](/tools/ollama) provides one-command Phi deployment: `ollama run phi3:mini`, `ollama run phi4:14b`. Phi models are heavily used on Ollama due to their small size. Phi-3-mini 3.8B on any modern GPU achieves 80+ tokens/second, making it the default recommendation for users with <= 8 GB VRAM. [llama.cpp](/tools/llama-cpp) supports all Phi generations with GGUF format. Phi-4 14B at Q4 on [Apple M3 Ultra](/hardware/apple-m3-ultra) with Metal achieves ~35 tokens/second. CPU-only inference on a high-end desktop: Phi-3-mini at Q4 achieves ~15 tokens/second — usable for batch processing. [TensorRT-LLM](/tools/tensorrt-llm) supports Phi-3+ for production serving. Phi-4 14B FP8 on [H100 SXM](/hardware/nvidia-h100-sxm) achieves ~12,000 tokens/second at batch 128. [MLX-LM](/tools/mlx-lm) provides excellent Phi support on Apple Silicon. Phi-4 14B at Q4 on [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) achieves ~38 tokens/second. MLX-LM's Phi support includes correct LongRope handling. Phi-3.5-MoE requires the `phi3` architecture tag with MoE configuration. The Phi tokenizer uses a 100K vocabulary with the Llama tokenizer backbone; chat templates use `<|user|>` and `<|assistant|>` tokens. [ExLlamaV2](/tools/exllamav2) supports Phi-3+ dense models with EXL2 quantization. Phi-4 14B at 4.0 bpw on [RTX 4090](/hardware/rtx-4090) achieves ~55 tokens/second. The MoE Phi-3.5 variant is not supported in ExLlamaV2. [Text Generation Inference](/tools/text-generation-inference) supports Phi-3+ with flash-attention. The LongRope context extension is correctly implemented in TGI 1.4+. ONNX Runtime and DirectML support Phi models for Windows deployment via the Olive optimization toolkit. This is the recommended path for Windows users without NVIDIA GPUs, supporting AMD, Intel Arc, and integrated GPUs.

Phi models are the most quantization-friendly models in open-weight AI at equivalent capability levels. The synthetic training data produces extremely well-behaved weight distributions with minimal outlier channels, making aggressive quantization unusually effective. GGUF via [llama.cpp](/tools/llama-cpp) provides the broadest path. At Q3_K_M (~1.8 GB), ~1.2% MMLU degradation. At Q2_K (~1.3 GB), ~3.5% MMLU degradation — still usable for casual applications. Phi models uniquely survive Q2_K quantization while remaining functional, a consequence of the well-regularized synthetic training data. Phi-4 14B at Q4_K_M (~8 GB) loses ~0.3% MMLU. At Q3_K_M (~6.5 GB), ~1.5% degradation. At IQ3_XXS (~5.5 GB), ~2.2% degradation — fitting on any 8 GB GPU at still-usable quality. Phi-3.5-MoE requires MoE-aware quantization. The 16 experts at Q4_K_M total ~24 GB, with ~4 GB active. IQ3_XXS on expert weights (~18 GB total, ~3 GB active) achieves Q3_K_M-quality and fits expert-resident on any 16 GB GPU. GPTQ and AWQ support is mature. Phi-4 14B AWQ 4-bit (~8 GB) is lossless on MMLU (<0.15% degradation). AWQ's per-channel scaling is slightly more effective for Phi than for other families — the synthetic-data training produces activation distributions with clean separation between high-magnitude and low-magnitude channels, which AWQ's scaling exploits well. EXL2 supports Phi-3+ dense models at arbitrary bitrates. Phi-4 14B at 3.5 bpw (~6.3 GB) loses ~1.8% MMLU — acceptable for VRAM-constrained deployment. At 4.5 bpw (~8.1 GB), quality is indistinguishable from FP16. FP8 via [TensorRT-LLM](/tools/tensorrt-llm) and [vLLM](/tools/vllm) is the standard format. Phi-4 14B FP8 is lossless (<0.1% degradation all benchmarks). FP8 KV-cache quantization saves ~2× KV-cache memory and is supported for Phi-3+ in vLLM. Microsoft provides official quantization tooling through Olive and ONNX Runtime: INT4 quantization for Phi models optimized for DirectML on Windows. The official Microsoft INT4 Phi-3-mini achieves <0.3% quality degradation on MMLU and runs on any DirectX 12 GPU. This is the recommended path for Windows production deployment. Practical rule: Phi models are the only family where Q3_K_M is genuinely production-viable at most sizes. Q2_K is usable for experimentation. The synthetic-data training produces unprecedented quantization robustness, making Phi the default choice for extreme VRAM-constrained deployments.

Phi's ecosystem is moderate in volume but high in quality, reflecting Microsoft's institutional backing and the family's unique position as the parameter-efficiency champion. The ecosystem is distinctly Microsoft-centric but with growing community adoption. Derivative model volume is moderate (hundreds of fine-tunes). The Microsoft-provided chat fine-tunes are the primary deployment models; community derivatives are secondary. License clarity is Phi's ecosystem superpower. MIT license covers all Phi-1 through [Phi-4](/families/phi) models. MIT is the most permissive major open-source license: no restrictions whatsoever beyond attribution and liability disclaimer. No MAU threshold, no commercial-use restriction, no use-case enumeration. For enterprises requiring the simplest possible license with zero compliance overhead, Phi is the safest choice. The Microsoft ecosystem integration is deep. Azure AI Studio provides one-click Phi deployment with managed endpoints. ONNX Runtime and Olive provide Windows-optimized inference. Windows Copilot Runtime integrates Phi models for on-device AI in Windows 11. Visual Studio Code extensions use Phi for local AI assistance. This Microsoft infrastructure depth provides enterprise-grade deployment tooling that community-centric families lack. Training data transparency is a Phi strength. Microsoft publishes detailed methodology for the synthetic data pipeline, including teacher model configurations, filtering criteria, and data mixture ratios. While the actual synthetic dataset is not released, the methodology is documented at a level that allows community reproduction. The educational ecosystem around Phi is growing. Microsoft Learn provides Phi fine-tuning tutorials. The small model sizes make Phi the default educational platform for learning LLM fine-tuning on constrained hardware. Phi-3-mini QLoRA fine-tuning on free Google Colab T4 GPU completes in ~2 hours for 10K samples. Ecosystem gaps: Phi's multilingual capability is limited by English-dominant synthetic training data. Tool-use and agent framework integration trails [Llama](/families/[llama](/families/llama)). The chat template (`<|user|>` / `<|assistant|>`) is less standardized than ChatML, causing integration friction with some frameworks.

Phi fine-tuning benefits from small model sizes and Microsoft-provided fine-tuning recipes. The low VRAM requirements make Phi fine-tuning the most accessible path to experimenting with custom LLMs. LoRA and QLoRA support is first-class. HuggingFace PEFT supports Phi with the `phi` and `phi3` architecture tags. For Phi-4 14B, QLoRA rank 32 with 4-bit base model on [RTX 4090](/hardware/rtx-4090) uses ~16 GB VRAM and trains 10K samples in ~5 hours. For Phi-3-mini 3.8B, QLoRA rank 64 on [RTX 3060 12GB](/hardware/rtx-3060-12gb) uses ~10 GB VRAM and trains 10K samples in ~2 hours. Full fine-tuning: Phi-4 14B needs ~112 GB VRAM (2× [A100 80GB](/hardware/nvidia-a100-80gb-sxm) with ZeRO-2). Phi-3-mini 3.8B needs ~32 GB VRAM (1× [RTX 4090](/hardware/rtx-4090) 24 GB is insufficient; use [A6000](/hardware/nvidia-a40) 48 GB or 2× [RTX 4090](/hardware/rtx-4090) with ZeRO-2). Full fine-tuning at 14B is accessible to individuals with 2-GPU setups. Microsoft provides official fine-tuning tooling: Azure AI Studio fine-tuning jobs, Olive fine-tuning optimization, and ONNX Runtime fine-tuning export. The Azure path provides managed infrastructure but adds cloud costs; the local path via HuggingFace PEFT is free and well-documented. The dataset ecosystem benefits from Phi's English strength and code capability. Standard datasets work well. This "synthetic fine-tuning" approach is well-documented for Phi and is a cost-effective way to create domain-specific models without manually curating training data. Axolotl supports Phi-3+ with correct chat template and LoRA configurations. [Unsloth](/tools/unsloth) does not support Phi as of mid-2026. The Axolotl Phi configs correctly handle the `<|user|>` / `<|assistant|>` chat template. DPO alignment for Phi is documented through Microsoft's Phi-3 post-training recipes. The Phi-3 Instruct models use DPO training with the UltraFeedback dataset. Community DPO on Phi-3-mini produces alignment improvements in ~4 GPU-hours on [RTX 4090](/hardware/rtx-4090). A Phi-specific fine-tuning consideration: Phi models trained on synthetic data tends to produce more concise outputs than web-trained models. Fine-tuning with very long examples (>2000 tokens) can sometimes conflict with Phi's synthetic-data pre-training tendency toward conciseness. Use shorter training examples or increase the number of epochs to compensate if your fine-tuning task requires long-form generation.

Phi's MIT license is the most permissive in open-weight AI. This makes Phi the lowest-legal-risk model for any deployment. Enterprises with strict license review processes should consider Phi as the default choice when license ambiguity would block deployment of other families. ONNX Runtime deployment is a Phi-specific advantage. Microsoft's Olive toolkit optimizes Phi models for ONNX Runtime with INT4 quantization and DirectML execution. For Windows enterprise deployments where GPU hardware is heterogeneous, Phi + ONNX Runtime is uniquely capable. Phi-3.5-MoE deployment has the same MoE serving considerations as other MoE families but at much smaller scale. [vLLM](/tools/vllm) MoE support works correctly for Phi-3.5-MoE at this scale. The router sensitivity to quantization follows standard MoE patterns — keep router weights at FP16. This conciseness bias is a design feature (fewer hallucinations, more efficient output) but may be unsuitable for creative writing or long-form generation tasks. If your deployment requires long-form content generation, [Llama](/families/llama) or [Mistral](/families/mistral) may produce more natural results. Chat template compatibility: Phi uses `<|user|>` and `<|assistant|>` tokens with `<|end|>` turn terminators. The system prompt is formatted as `<|system|>{text}<|end|>`. Using other chat templates causes significant quality degradation. [Ollama](/tools/ollama) and [vLLM](/tools/vllm) auto-detect the Phi template correctly from tokenizer_config.json since Phi-3+. A known Phi-3 behavior: at temperature 0, Phi-3-mini sometimes produces repetitive completions on certain prompts. Setting temperature to 0.1 or using top_p=0.9 resolves this. This is less common with Phi-4 but still observable on edge cases. Windows Copilot Runtime integration enables Phi deployment as a system-level AI service on Windows 11. This is unique to Phi — no other open-weight model has OS-level integration. The Copilot Runtime provides managed memory, power efficiency, and background execution. Phi-3-mini runs in the Copilot Runtime at ~10 tokens/second on laptop integrated GPUs with <5W power consumption. Non-English language quality degrades significantly on Phi models. The synthetic training data is >95% English, and non-English benchmarks are 10-30 points below English benchmarks depending on language. For multilingual deployments, use [Aya](/families/aya) or [Qwen](/families/qwen) instead. Phi is an English-first family by design.