DBRX

DBRX (March 2024) launched as Databricks' first open-weight model family, developed by the MosaicML team. The architecture is a fine-grained Mixture-of-Experts design: 132B total parameters with 36B active per token, using 16 experts per transformer layer with top-4 routing. Each expert is a standard FFN block with SwiGLU activation. Attention uses GQA (8 KV heads) with RoPE positional embeddings and 32K context. The fine-grained MoE design is the architectural signature: 16 experts × top-4 routing vs [Mixtral](/families/mistral)'s 8 experts × top-2 routing. More experts with higher active count produces more compute per token (36B active for DBRX vs 12.9B for Mixtral 8x7B), targeting quality over compute efficiency. The expert routing uses learned routing weights (similar to [DeepSeek](/families/deepseek)) with a load-balancing auxiliary loss. DBRX's training was notable for infrastructure transparency: trained on 12T tokens using MosaicML's Composer + StreamingDataset platforms, on 3,072 H100 GPUs over approximately 3 months. The training infrastructure cost (~$10M) and methodology were disclosed in detail, providing a reference point for the cost of training 132B-class MoE models. Databricks' MosaicML pretraining platform (Composer, StreamingDataset, LLM Foundry) is the training stack — DBRX serves as the reference model demonstrating MosaicML's capabilities. The DBRX Instruct variant added supervised fine-tuning and DPO alignment on top of the base model. Architecture unchanged; post-training methodology added instruction-following capability. The family is architecturally frozen after the initial release — no DBRX 2 or architectural evolution has shipped as of mid-2026. DBRX represents a single-generation MoE experiment from Databricks, testing the premise that fine-grained MoE with MosaicML's training infrastructure could produce competitive results. DBRX remains architecturally interesting for its fine-grained expert design (16 experts vs competitors' 8-256 range) but has been surpassed in capability. The architectural evolution (or lack thereof) tells a story: DBRX was a promising single-release MoE model that demonstrated Databricks' training infrastructure capabilities, but without follow-up releases, it's a snapshot of early-2024 MoE design rather than an evolving family.

DBRX models occupy an awkward position in local viability: the 132B total / 36B active MoE configuration requires more VRAM than most consumer hardware provides, while delivering capability that 2026 models achieve at smaller parameter counts. DBRX 132B MoE at FP16 requires ~264 GB total VRAM with ~72 GB active. This fits on: 4× [H100 SXM](/hardware/nvidia-h100-sxm), 2× [MI300X](/hardware/amd-mi300x), or [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) with aggressive offloading. Expert offloading on [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) 192 GB: the model at Q4 (~75 GB total, ~20 GB active) fits with ~120 GB for inactive experts in unified memory and ~20 GB in fast-access memory. Throughput: ~5-8 tokens/second at Q4 with expert offloading via [llama.cpp](/tools/llama-cpp). On consumer NVIDIA GPUs, DBRX deployment is constrained. Single [RTX 4090](/hardware/rtx-4090) 24 GB cannot fit the active experts (~20 GB) plus KV-cache at reasonable context. 2× [RTX 4090](/hardware/rtx-4090) with expert parallelism at Q4: ~35 GB active experts across 2 GPUs, leaving ~13 GB for context — limited to ~16K context. No single consumer GPU under $2,000 can practically serve DBRX at production context lengths. DBRX at Q3_K_M (~58 GB total, ~16 GB active) is the minimum for consumer deployment on 2× [RTX 4090](/hardware/rtx-4090) at 8K context (~10 tokens/second). The quality degradation at Q3 (~2.5% MMLU) makes this borderline for serious use. The trajectory is flat — no DBRX 2 with improved local viability has shipped. The model's 2024-era MoE design is less locally-viable than 2026 MoE competitors: [Qwen 3 235B-A22B](/families/qwen) (22B active, hash routing for deterministic expert selection simplifies offloading) and [DeepSeek V3](/families/deepseek) (37B active but MLA attention compresses KV-cache by 8×). DBRX's 16-expert top-4 routing means more active parameters per token (36B) than competitor MoE models at similar capability level in 2026. The practical case for local DBRX deployment in 2026 is narrow. If your hardware budget is 2× consumer GPUs, [Llama 3.3 70B](/families/llama) at Q4 is more practical. DBRX's local viability has been undercut by 2026 alternatives that achieve better capability-per-VRAM. The model remains functional for Databricks platform users who deploy via Databricks Model Serving rather than local self-hosted inference.

DBRX Instruct (March 2024) launched with competitive if not class-leading benchmarks. MMLU 73.7% (5-shot) was behind [Llama 3 70B](/families/llama) (79.5%) and [Mixtral 8x22B](/families/mistral) (77.8%). GSM8K 69.4% (8-shot) was adequate but behind Llama 3 70B (81.2%). HumanEval 56.3% (pass@1) was competitive with Llama 3 70B (54.9%). The fine-grained MoE design (16 experts × top-4) showed mixed benchmark results. On knowledge-heavy benchmarks (MMLU, ARC-Challenge), DBRX underperformed dense models at similar parameter scale, suggesting that MoE routing with many small experts reduces individual expert capacity. On reasoning benchmarks (GSM8K, HumanEval), the MoE performed competitively, suggesting reasoning benefits from the broader expert coverage. Programming benchmarks: HumanEval 56.3% at pass@1, MBPP 58.7%, MultiPL-E across various programming languages showed consistent but not exceptional code capability. DBRX was not positioned as a code model and the code benchmarks reflected general capability rather than specialization. Where DBRX benchmarked well: the Inflection-1 benchmark suite (Databricks' internal evaluation) showed competitive quality, Long-range context benchmarks (32K) showed stable quality across the full context due to the dense attention layers (not sliding window), and TruthfulQA MC2 62.1% was competitive (between [Llama 2 70B](/families/llama) at 65.1% and [Llama 2 13B](/families/llama) at 56.0%). The benchmark gap vs 2026 models is substantial: [DeepSeek V3/V4](/families/deepseek) exceed DBRX by 15-20 MMLU points and 25-30 MATH points. [Qwen 3 235B-A22B](/families/qwen) exceeds DBRX by ~16 MMLU points at fewer active parameters (22B vs 36B). DBRX's benchmarks reflect mid-2024 capability levels and have not been updated — there is no DBRX 2 to close the gap. Where DBRX continues to serve: Databricks platform customers who need an integrated model with their data lakehouse infrastructure benefit from DBRX's Databricks-native deployment, even if the model's benchmark quality is behind 2026 frontier. The benchmarks are adequate for internal enterprise workloads (document Q&A, SQL generation, data analysis) even if they trail for frontier research.

DBRX runtime support is narrow and Databricks-centric, reflecting the model's origin as a platform-specific release. Community engine support exists but is less mature than for major families. Databricks Model Serving is the reference deployment path. DBRX deployment via Databricks provides optimized serving infrastructure with automatic scaling, monitoring, and enterprise security. For Databricks platform customers, Model Serving eliminates the infrastructure complexity of self-hosting DBRX. The Databricks path includes DBRX-specific optimizations not available in community engines. [vLLM](/tools/vllm) supports DBRX MoE since v0.4.3. DBRX on 4× [H100 SXM](/hardware/nvidia-h100-sxm) with vLLM achieves ~4,000 tokens/second at batch 32. The 16-expert top-4 routing creates expert load balancing challenges in vLLM — the EPLB feature helps but the fine-grained expert design (many small experts with high active count) produces more load imbalance than [Mixtral](/families/mistral)'s 8-expert design. [llama.cpp](/tools/llama-cpp) supports DBRX MoE with expert offloading. DBRX at Q4 on [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) achieves ~6 tokens/second with expert offloading. The 16-expert top-4 routing adds overhead vs Mixtral's 8-expert top-2 — DBRX activates twice as many experts per token, doubling the expert weight loading cost. [Ollama](/tools/ollama) supports DBRX through the llama.cpp backend. `ollama run dbrx:instruct` launches the model. Performance on consumer hardware is limited by the 36B active parameter count. [TensorRT-LLM](/tools/tensorrt-llm) added DBRX support in 0.10.0. DBRX FP8 on 4× [H100 SXM](/hardware/nvidia-h100-sxm) achieves ~10,000 tokens/second at batch 128. TensorRT's expert-aware scheduling handles DBRX's fine-grained expert routing more efficiently than vLLM. [SGLang](/tools/sglang) has basic DBRX MoE support. Performance is comparable to vLLM. [Transformers](/tools/transformers) has DBRX support via the `dbrx` architecture tag. The model loads with standard AutoModelForCausalLM and requires ~130 GB for FP16 inference. The runtime support landscape is thin compared to [Llama](/families/llama) — DBRX's MoE design with 16 experts creates more serving complexity than Mixtral (8 experts) or Qwen 3 MoE (hash routing simplifies expert selection), and the model's community ecosystem hasn't motivated the optimization investment that larger families receive.

DBRX quantization follows standard MoE patterns with specific considerations for the fine-grained expert design. GGUF via [llama.cpp](/tools/llama-cpp) is the primary path. DBRX at Q4_K_M (~75 GB total) loses ~0.6% MMLU vs FP16. At Q3_K_M (~58 GB), ~2.5% MMLU degradation. The fine-grained expert design (many small experts) makes expert weights more sensitive to quantization than coarse-grained MoE — each expert has fewer parameters, so each weight matters more proportionally. At Q3, routing accuracy degrades ~3% (some tokens routed to suboptimal experts), adding to the weight-precision quality loss. The router must remain at FP16 (standard MoE constraint). DBRX's 16-expert router is ~40M parameters — negligible memory cost. AWQ and GPTQ support is limited for DBRX. Limited community testing means quality data is sparse. For production NVIDIA deployment, FP8 via TensorRT-LLM is the recommended format. FP8 via [TensorRT-LLM](/tools/tensorrt-llm) is effectively lossless (<0.2% degradation). This is the production format for DBRX on NVIDIA hardware. Practical rule: DBRX at Q4_K_M provides acceptable quality (~0.6% loss) for [Mac Studio M3 Ultra](/hardware/mac-studio-m3-ultra) deployment. Below Q4, quality degrades more rapidly than equivalent-sized dense models due to the fine-grained expert sensitivity. For serious use, Q4_K_M or FP8 are the quality floor.

DBRX's ecosystem is Databricks-centric and has low community momentum, reflecting the model's single-release status and the competitive MoE landscape of 2026. Derivative model volume is very low (single-digit fine-tunes). No major community derivatives have emerged from DBRX. The model serves primarily as a Databricks platform asset rather than a community fine-tuning base. The absence of DBRX 2 or architectural evolution has prevented ecosystem growth. License clarity: the DBRX Open Model License permits commercial use, redistribution, and modification. The license terms are standard for Databricks open models: broad commercial permissibility with standard restrictions (no illegal use, compliance with laws). The license is commercially-friendly and comparable to [Llama](/families/llama)'s Community License. The Databricks ecosystem integration is DBRX's unique value. DBRX is integrated with Databricks' data lakehouse platform: Unity Catalog model registry, Model Serving endpoints, Lakehouse Monitoring, and Databricks SQL AI functions. For organizations already on Databricks, DBRX provides a managed, secure, integrated model deployment that avoids the infrastructure overhead of self-hosting. No other model family has comparable lakehouse integration. MosaicML's training infrastructure story is DBRX's secondary ecosystem contribution. The MosaicML Composer + StreamingDataset + LLM Foundry stack demonstrated cost-effective training of 132B MoE models at ~$10M — a pricing benchmark that helped calibrate the industry's understanding of large-model training costs. Databricks acquired MosaicML for $1.3B in 2023, and DBRX served as the technology demonstration of the combined platform. Ecosystem gaps: community fine-tuning culture is absent; multilingual derivatives don't exist; non-Databricks deployment documentation is sparse; and the 2026 MoE competitive landscape ([DeepSeek V3/V4](/families/deepseek), [Qwen 3 MoE](/families/qwen)) has made DBRX's MoE design less relevant. DBRX's ecosystem value is primarily for existing Databricks customers.

DBRX fine-tuning is most practical through Databricks' platform, with limited community tooling for self-hosted fine-tuning. Databricks Model Fine-Tuning is the recommended path: managed infrastructure, automatic hyperparameter optimization, and integrated deployment. The Databricks fine-tuning service handles the MoE training complexity (expert-level parallelism, routing gradient computation, load-balancing loss). For Databricks customers, this is the turnkey DBRX fine-tuning path. LoRA and QLoRA via HuggingFace PEFT are technically possible but under-tested for DBRX's MoE architecture. The 16-expert fine-grained design makes LoRA placement decisions non-obvious — which experts and layers should LoRA adapters target? Community experimentation is minimal. For QLoRA on DBRX: ~60 GB VRAM minimum (1× [A100 80GB](/hardware/nvidia-a100-80gb-sxm) with ZeRO-2) using 4-bit base model. Training throughput is lower than equivalently-sized dense models due to MoE routing overhead backpropagation. Full fine-tuning: ~1 TB VRAM minimum (16× [H100 SXM](/hardware/nvidia-h100-sxm) with full parallelism). This is enterprise-DGX-scale and not accessible to individual developers. The fine-tuning ecosystem is the thinnest of any major open-weight family, reflecting DBRX's single-release status and the availability of superior fine-tuning targets in 2026 ([Llama 3.3 70B](/families/llama), [Qwen 3 32B](/families/qwen)). Unless you are a Databricks customer using Databricks Model Fine-Tuning, there is little reason to fine-tune DBRX over contemporary alternatives.

DBRX's DBRX Open Model License is commercially permissive but has been superseded by more permissively-licensed alternatives in 2026 ([Qwen](/families/qwen) Apache 2.0, [Mistral](/families/mistral) Apache 2.0 for open models, [Phi](/families/phi) MIT). The license itself is not problematic — it's broadly commercial — but the model's 2024-era capability level means the license isn't the limiting factor for deployment. Databricks Model Serving provides the most reliable DBRX deployment path. Self-hosting DBRX MoE requires significant debugging of expert routing, load balancing, and MoE-specific inference optimizations. Databricks' managed path abstracts this complexity. If self-hosting, expect 2-3× lower throughput than Databricks Model Serving due to Databricks' internal optimizations not available in open-source engines. The fine-grained MoE design has a known deployment characteristic: expert load imbalance worsens with batch diversity. The mitigation: use EPLB in vLLM or accept lower throughput at batch sizes where load naturally balances (batch 8-16). Chat template: DBRX uses a format with `<|im_start|>` and `<|im_end|>` tokens, similar to ChatML but with specific system prompt handling. The template is documented in the DBRX model card. DBRX's 32K context is a limitation for 2026 deployment. Most current models support 128K+ context. For RAG and long-document workflows, DBRX's 32K limit requires document chunking that more modern models avoid. Competitive displacement: DBRX in 2026 is outclassed by [DeepSeek V3](/families/deepseek) (better MoE, 4× context, higher quality) and [Qwen 3 235B-A22B](/families/qwen) (similar active params, higher quality, 4× context). The primary reason to deploy DBRX in 2026 is Databricks platform integration, not model capability. For new deployments not tied to Databricks infrastructure, contemporary alternatives are superior on every metric. DBRX deployment on [llama.cpp](/tools/llama-cpp) requires the MoE backend with expert offloading enabled. The 16-expert activation pattern means expert prefetching cannot predict which experts will be needed for the next token, so every token requires loading the expert weights from offloaded storage. This produces erratic latency (1-10s per token depending on cache-hit rate for expert weights). For interactive use, this latency variance is problematic.