Flux

Black Forest Labs released Flux Schnell (timestep-distilled, Apache 2.0) in August 2024 alongside Flux Dev (guidance-distilled, non-commercial license) and Flux Pro (API-only, closed). All three share the MMDIT (Multi-Modal Diffusion Transformer) backbone — a 12B-parameter transformer operating in a joint text-image latent space with dual-stream text/image conditioning via T5-XXL text encoder and a VAE adapted from [Stable Diffusion](/families/stable-diffusion)'s latent compression pipeline. Flux.1 (the first-generation suffix) uses flow matching with rectified flow trajectories instead of DDPM noise schedules, which cuts inference steps from 25–50 to 4–8 for Schnell and 25–50 for Dev while maintaining text rendering fidelity. The MMDIT architecture's defining contribution is text rendering accuracy. Unlike prior diffusion models that treat text as a global conditioning vector, MMDIT cross-attends individual text tokens directly into the diffusion transformer layers, enabling character-level rendering control. This is why Flux outputs readable text in signs, posters, and UI mockups where [Stable Diffusion 3](/families/stable-diffusion) and earlier models produce garbled letterforms. The tradeoff: MMDIT's 12B parameter count demands ~24 GB VRAM at FP16 for the Dev variant, roughly 3× the memory footprint of SDXL's 2.6B UNet. Flux Pro (closed-weight, API-only) adds classifier-free guidance scheduling and higher-resolution native generation (up to 2K). Flux.2 (2025) introduced Pro-2, Dev-2, and Schnell-2 with a refined MMDIT-v2 backbone that improved text rendering on small font sizes and added depth-conditioned inpainting. The architectural lineage from Flux.1 to Flux.2 is evolutionary rather than revolutionary — the core MMDIT design proved correct on the first iteration and subsequent releases refined training data quality, prompt alignment, and inference efficiency.

Flux Dev is the practical local generation workhorse. At FP16, it requires ~24 GB VRAM — runnable on an [RTX 4090](/hardware/rtx-4090) (24 GB, ~8–12 seconds per 1024×1024 image at 25 steps) or [RTX 5090](/hardware/rtx-5090) (32 GB, ~5–7s per image). Flux Schnell generates at 4 steps (~1.5–3s on an RTX 4090) with ~15% quality reduction vs Dev, making it viable for interactive workflows and batch processing. The FP8 quantized Flux Dev (via [ComfyUI](/tools/comfyui) with --fast load) fits in 16 GB VRAM at 3–5× the generation time due to on-the-fly dequantization overhead. Multi-GPU setups shift the viability curve significantly. Dual [RTX 3090](/hardware/rtx-3090) (2×24 GB = 48 GB pool) runs Flux Dev FP16 fully on GPU at ~10–15s/image with tensor parallelism via the ComfyUI multi-GPU nodes. An [NVIDIA L40S](/hardware/nvidia-l40s) at 48 GB runs Flux Dev FP16 at ~6–8s/image in a single-card datacenter form factor. The [Apple M3 Ultra](/hardware/apple-m3-ultra) with 192 GB unified memory runs Flux Dev via Core ML at ~40–60s/image — functional but 5–8× slower than an RTX 4090 due to the MPS backend's immature diffusion transformer optimization. Flux Schnell's Apache 2.0 license makes it the only local-first Flux variant suitable for commercial deployment without a Black Forest Labs API contract. Flux Dev's non-commercial license creates a local experimentation → cloud production pipeline pattern: iterate prompts and compositions locally on Dev, then switch to the Pro API for commercial output. Flux Pro is API-only and unavailable locally, occupying the same commercial-only tier as [DALL-E](/families/gpt) or [Imagen](/families/gemini). The practical ceiling: Flux Pro-2 at 2K resolution requires ~48 GB VRAM even at FP8, placing it in the [NVIDIA A100 80GB](/hardware/nvidia-a100-80gb-sxm) or [H200](/hardware/nvidia-h200) tier for local operation. The local viability trajectory is improving. The ComfyUI Flux ecosystem ships GGUF-quantized checkpoints at Q4_K_M (~11 GB) that load on a 16 GB GPU at 15–25s/image. This is barely interactive but functional for overnight batch rendering. Q8 quantized (~19 GB) achieves near-lossless quality at ~12–18s/image on 24 GB GPUs. The next 12–18 months should see Flash Attention 3 integration and torch.compile optimizations bringing Flux Dev to ~4–6s/image on consumer 24 GB GPUs — roughly matching current Pro API latency.

Flux entered the image generation benchmark landscape in mid-2024 at or above the state of the art across text rendering and aesthetic quality. On GenEval (text-to-image compositional benchmark), Flux Pro scored 0.69 vs DALL-E 3's 0.67 and [SD3](/families/stable-diffusion)'s 0.62, with the margin concentrated in text rendering (0.88 vs 0.72 for SD3) and positional reasoning (0.71 vs 0.58). On the DrawBench compositional split, Flux Dev matched or exceeded Flux Pro on 4 of 8 categories, suggesting the non-commercial variant retains most of the model's architectural advantages. Human preference benchmarks (PartiPrompts, HPS v2) placed Flux Pro at 28.7% win rate against Midjourney v6.1 in blind A/B testing — statistically tied within confidence intervals. Flux Dev scored 26.1% against the same Midjourney baseline, a 2.6-point gap that roughly corresponds to the Pro→Dev distillation loss. Flux Schnell at 4 steps scored 21.3% win rate — a 7.4-point drop from Pro but remarkable for a 4-step distilled model that runs interactively on consumer hardware. On T2I-CompBench (attribute binding, spatial relationships, and text rendering), Flux Pro led all open-weight competitors at 0.74 composite score, with the text rendering sub-score (0.91) establishing the family's core competency. The gap narrowed with Flux.2: the T2I-CompBench composite for Flux Dev-2 rose to 0.71 (from 0.67 for Dev-1), closing 60% of the Pro→Dev gap. On the MJHQ-30K aesthetic benchmark, Flux Pro-2 scored 6.7/10 vs MJ v6.1's 6.8/10 — within sampling noise of the closed-source leader. Flux consistently underperforms on style diversity benchmarks (StyleDrop, StyleBoost) compared to [Stable Diffusion](/families/stable-diffusion), which benefits from the massive LoRA ecosystem enabling style-conditioned generation. Flux's MMDIT architecture's tight text-image coupling produces photorealistic and graphical styles by default but struggles with abstract and watercolor styles without explicit fine-tuned adapters. This is a benchmark limitation (style diversity benchmarks favor models with LoRA ecosystems) rather than a fundamental architectural constraint.

[ComfyUI](/tools/comfyui) is the primary runtime for Flux local inference, with native MMDIT node support and the widest workflow library. The Flux Dev/Schnell loader node handles checkpoint loading, T5 text encoder loading, and VAE loading in a single step. ComfyUI's node-graph architecture enables complex Flux pipelines — ControlNet depth conditioning, IP-Adapter style transfer, regional prompting via attention masking, and multi-pass refinement — that are impractical in monolithic generation UIs. The ComfyUI Manager plugin ecosystem provides one-click Flux LoRA and ControlNet installation. [Diffusers](/tools/diffusers) supports Flux via the FluxPipeline class with both Dev and Schnell variants, using the same MMDITTransformer2DModel backend as the reference implementation. Diffusers' Flux support is research-optimized rather than throughput-optimized: it prioritizes PyTorch-native checkpoint loading and modularity over generation speed. Diffusers-generated Flux images at 25 steps on FP16 run ~30–40% slower than the equivalent ComfyUI workflow on identical hardware due to ComfyUI's aggressive attention caching and memory pooling. GGUF-quantized Flux checkpoints (via city96's Flux-GGUF nodes for ComfyUI) enable 16 GB GPU operation at Q4_K_M quality. The GGUF loader performs on-the-fly dequantization, trading ~2× generation time for ~2× memory reduction. This is the only practical local path for Flux on an [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb) or [RTX 3080 10GB](/hardware/rtx-3080-10gb) (Q4 with heavy CPU fallback). Flux has no official [Ollama](/tools/ollama), [vLLM](/tools/vllm), or [llama.cpp](/tools/llama-cpp) support — these runtimes are text-generation-optimized and lack the VAE + T5-XXL + MMDIT pipeline architecture required for diffusion image generation. This means Flux cannot be served via OpenAI-compatible APIs or batched like text models. For production Flux serving, the typical pattern is ComfyUI headless mode behind a REST API wrapper with GPU worker pooling via RunPod or modal.com serverless GPU infrastructure.

Flux quantization operates in two regimes: checkpoint-level quantization (GGUF for VRAM-constrained GPUs) and inference-level precision reduction (FP8/FP16 tradeoff). The GGUF quantization path, implemented through ComfyUI's Flux-GGUF loader, supports Q2_K through Q8_0 with the following VRAM/quality profile: Q4_K_M at ~11 GB with minimal quality loss (PSNR drop <0.5 dB vs FP16 on 512×512 crops), Q4_K_S at ~10 GB with slight texture blurring, Q5_K_M at ~13 GB as the VRAM/quality sweet spot for 16 GB GPUs, and Q8_0 at ~19 GB with near-lossless output. Q2_K and Q3_K variants produce visible blocking artifacts and should be avoided for production use. FP8 inference (via torch.float8_e4m3fn in the Diffusers backend) reduces the MMDIT transformer's memory footprint from 24 GB to ~16 GB while maintaining generation quality within 0.3% FID variance on COCO-30K. The VAE decoder and T5 text encoder typically remain at FP16 — they account for only ~2 GB of the total footprint and quantizing them below FP16 introduces latent-space decoding artifacts (posterization in flat color regions, banding in gradients). The practical recommendation: quantize the MMDIT backbone to FP8 or Q8, leave the VAE at FP16, and the T5 encoder at FP16 or BF16. Quantization uniquely impacts Flux's text rendering capability — its architectural signature feature. At Q4_K_M and below, small-font text (under 24pt equivalent at 1024×1024) shows increased character corruption rate (from ~3% at FP16 to ~12% at Q4_K_M as measured by OCR word error rate on generated text). This is because the MMDIT's cross-attention layers that perform character-level conditioning are among the most quantization-sensitive components. For workflows where text rendering is the primary requirement (UI mockups, poster generation, signage), FP16 or Q8 is strongly preferred over Q4 variants. The VAE's latent-space quantization via tiny-VAE or TAESD reduces VAE memory by ~60% at the cost of ~2% PSNR loss — a worthwhile tradeoff when the MMDIT backbone is already quantized. Combined MMDIT Q4 + tiny-VAE yields a ~10 GB total footprint, making Flux Dev runnable on 12 GB GPUs at the quality floor for non-text-critical applications.

Flux's ecosystem is younger but higher-quality than [Stable Diffusion's](/families/stable-diffusion) decade-deep tooling landscape. The core ComfyUI Flux node ecosystem covers: ControlNet (depth, canny, pose, inpainting), IP-Adapter (image-prompted style transfer), LoRA training and inference (via the Kohya-ss Flux trainer and ComfyUI LoRA loader), regional prompting (attention masking for multi-subject scenes), and video generation (AnimateDiff-Flux, early-stage). The LoRA ecosystem is roughly 1/20th the size of SDXL's — approximately 5,000 public Flux LoRAs vs 100,000+ for SDXL on CivitAI as of early 2026. However, Flux LoRAs are disproportionately high-quality: because training a Flux LoRA requires a 24 GB+ GPU (vs 8 GB for SDXL LoRAs), the creator pool skews toward experienced practitioners. Character LoRAs, art style LoRAs, and product visualization LoRAs are the three most common categories. The Kohya-ss SD3/Flux trainer supports rank-4 to rank-128 LoRA training with a ~6-hour training time on an [RTX 4090](/hardware/rtx-4090) for a 100-image dataset at rank-32. The ControlNet ecosystem is functional but sparse compared to SDXL. Flux depth ControlNet (trained by Black Forest Labs and community-refined) works reliably; canny-edge and pose ControlNets exist but show 10–20% lower adherence rates than their SDXL equivalents. Inpainting ControlNet (mask-guided regeneration) works at parity with SDXL. The ControlNet training community is constrained by Flux's 24 GB minimum — ControlNet training on Flux requires 48 GB+ VRAM, limiting contributors to the [NVIDIA L40S](/hardware/nvidia-l40s) and [A100](/hardware/nvidia-a100-80gb-sxm) tier. The ComfyUI Flux ecosystem is the fastest-growing segment. The Flux Dev→ComfyUI→CivitAI pipeline (download checkpoint, load in ComfyUI, browse workflow templates) is well-documented across YouTube tutorials and the ComfyUI subreddit. The tooling maturity gap vs SDXL is closing at roughly 20–30% per year as the ComfyUI community migrates to MMDIT-native workflows from the UNet-centric SDXL tooling.

Flux fine-tuning is GPU-intensive compared to SDXL. A full fine-tune of Flux Dev on a 1,000-image dataset requires ~48 GB VRAM (single [A100 80GB](/hardware/nvidia-a100-80gb-sxm) or dual [RTX 4090](/hardware/rtx-4090) with FSDP) and ~8–12 hours of training time at batch size 1 with gradient accumulation. This 8–10× higher resource requirement vs SDXL full fine-tunes (which run on a 12 GB GPU) constrains the fine-tuning community to cloud GPU providers or high-end workstation setups. LoRA is the practical fine-tuning method. Flux LoRA training via Kohya-ss (or the newer ai-toolkit Flux trainer) on an RTX 4090 achieves convergence in 4–8 hours for 50–150 images at rank-16 to rank-32. The Flux LoRA training recipe differs from SDXL's: Flux requires higher learning rates (1e-4 vs 5e-5 for SDXL), longer training (2,000–4,000 steps vs 1,000–2,000 for SDXL), and benefits from text-encoder training (not just the MMDIT backbone) for style LoRAs. The ai-toolkit Flux trainer supports Prodigy-optimized LoRA training with automatic learning rate scheduling — the current best-practice for Flux LoRAs. Dreambooth-style fine-tuning (full checkpoint fine-tuning on 5–20 images of a single subject) is possible on Flux Dev at 48 GB but the results are less reliable than SDXL's mature Dreambooth ecosystem. Flux's T5-XXL text encoder tends to overfit on small datasets, producing subject leakage where the subject concept bleeds into unrelated prompts. Mitigation: train the MMDIT backbone only (frozen T5 encoder) for Dreambooth on Flux, which produces ~80% of SDXL Dreambooth quality at 5× the VRAM cost. The community fine-tuning landscape clusters around: (1) character consistency LoRAs for illustration and game asset pipelines, (2) product photography LoRAs for e-commerce image generation, (3) architectural visualization LoRAs for real estate and interior design, and (4) NSFW/adult character LoRAs (which dominate CivitAI upload counts for Flux as they do for SDXL). The high hardware barrier means Flux fine-tunes skew professional — the hobbyist LoRA creator segment remains on SDXL.

Flux Dev's non-commercial license is the primary deployment constraint. The Black Forest Labs license permits non-commercial use, research, and personal projects but requires a commercial API agreement (via [bfl.ml](https://bfl.ml)) for any revenue-generating deployment. This creates a hard fork in deployment patterns: hobbyists and researchers use Flux Dev locally, while commercial users either use Flux Schnell (Apache 2.0) at reduced quality or pay the Pro API. There is no legal commercial local deployment path for Flux Dev without a BFL license. This is fundamentally different from [Stable Diffusion](/families/stable-diffusion) where SDXL and SD3.5 are permissively licensed for commercial use. VRAM demands are the secondary deployment constraint. Flux Dev at FP16 on a single 24 GB GPU operates at the memory ceiling — there is zero headroom for ControlNet, IP-Adapter, or video generation without quantization or multi-GPU. In production, the typical Flux deployment on an [RTX 4090](/hardware/rtx-4090) runs at 23+ GB VRAM utilization. Any VRAM spike (from a concurrent request, OS display compositor, or monitoring agent) triggers CUDA OOM. Mitigation: disable OS GPU acceleration for display output, run headless, and allocate a 1 GB VRAM buffer by quantizing to Q8. The T5-XXL text encoder (~5 GB at FP16) is often the hidden VRAM consumer. It loads alongside the MMDIT backbone and VAE, adding ~5 GB to the total footprint. The T5 encoder can be quantized to Q8 (saving ~2 GB) without measurable prompt understanding degradation. The CLIP-L text encoder (a secondary text encoder used for compatibility with SD workflows) adds another ~0.5 GB — disable it when using Flux-native workflows to reclaim VRAM. Flux inpainting and outpainting workflows have a 2× VRAM multiplication effect. Because Flux processes the full image (not just the masked region), a 1024×1024 inpainting operation with a 512×512 mask still generates a 1024×1024 latent — effectively doubling the per-image cost for partial regenerations. This makes Flux inpainting economically unviable at scale without batching, which further compounds VRAM requirements. The operational heuristic: use SDXL inpainting for bulk masking workflows and reserve Flux for single-image refinements where text quality matters.