FP32
Definition pending
We've cataloged "FP32" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
FP32 (single precision, 32-bit float) is the training default. For inference, it's wasteful — uses 2× the memory of FP16 with no perceptible quality improvement on modern models. You only need FP32 if you're debugging numerical issues (NaNs, exploding activations) or training from scratch. Most operators never run FP32 inference.
Workflow example
If your model outputs NaNs, the first debugging step is to run one forward pass in FP32 to check if the issue is quantization-related. If FP32 also produces NaNs, the model weights are corrupted or the architecture has a bug. If FP32 is clean but FP16/quantized is broken, your quantization calibration was bad — re-quantize with representative calibration data.