INT8
Definition pending
We've cataloged "INT8" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
INT8 uses 8 bits per parameter — 1 byte per weight, half of FP16. Quality loss is near-zero (<0.1%) for most models. The memory savings vs FP16: 2×. For production deployments where quality absolutely cannot degrade, INT8 is the safest quantized format. Many serving frameworks (vLLM, TensorRT-LLM) support INT8 with optimized kernels.
Workflow example
Use INT8 when quality matters more than memory: (1) for coding models where a single wrong token breaks the program, use Q8_0 (GGUF) or INT8 (GPTQ/AWQ), (2) for math/reasoning tasks, same rationale, (3) for general chat where occasional awkward phrasing is acceptable, Q4 is fine. The quality difference between Q4 and Q8 diminishes for instruction-tuned models > 30B params.