Mixed Precision
Definition pending
We've cataloged "Mixed Precision" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
Mixed precision uses FP16 (or BF16) for most operations but keeps critical paths in FP32. During training, this means ~2× speed and half the memory with no accuracy loss. During inference, most frameworks already use mixed precision automatically — the model loads in FP16/quantized but attention softmax and layer norm run in FP32 for numerical stability.
Workflow example
Mixed precision is automatic in most frameworks (PyTorch AMP, llama.cpp, vLLM). You don't configure it explicitly for inference. For training: torch.cuda.amp.autocast() wraps your forward pass. If you see "loss scale" logs during training, mixed precision is active. If training loss suddenly becomes NaN, the loss scale dropped too far — increase the initial scale or switch to BF16 which doesn't need loss scaling.