FP8
Definition pending
We've cataloged "FP8" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
FP8 uses 8 bits for weights and activations during inference on H100/H200 GPUs (FP8 native support on Transformer Engine). It's 2× faster than FP16 with negligible quality loss for well-trained models. The catch: only H100/H200/B200 GPUs support it natively. On consumer GPUs (RTX 4090), FP8 is emulated and actually slower than FP16.
Workflow example
FP8 is a datacenter-only feature for now. If you're on H100s: enable FP8 in vLLM with --quantization fp8. Quality benchmarks show <1% accuracy loss vs FP16 for most models. If you're on consumer hardware, skip FP8 and go straight to INT4/INT8 quantized formats — they have broader support and better tooling (llama.cpp, AutoGPTQ).