BF16 (BFloat16)
Definition pending
We've cataloged "BF16 (BFloat16)" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
BF16 (Brain Float 16) uses 16 bits like FP16 but with the same 8-bit exponent range as FP32. This means it handles very large and very small numbers better than FP16, making training more stable. For inference, BF16 and FP16 are nearly identical speed — the difference is training stability. If your GPU supports BF16 (A100, H100, RTX 4090), use it for training.
Workflow example
Check BF16 support: nvidia-smi → check "Compute Capability". CC 8.0+ (A100, RTX 30/40 series) supports BF16 natively. In PyTorch: model.to(torch.bfloat16). For GGUF models, BF16 doesn't matter — quantized formats (Q4, Q5, Q8) are independent of training precision. Only relevant when you're converting PyTorch → GGUF or doing fine-tuning.