Q4_0 Quantization

Q4_0 is the original llama.cpp 4-bit quantization: INT4 weights with one FP16 scale per 32-element block, no zero-point, no importance matrix. Each parameter takes ~4.5 bits.

Q4_0 has been superseded by Q4_K_M for almost all use cases. It's faster than K-quants on some old hardware (no per-row importance matrix to apply) but quality is worse — perplexity is typically 0.3–0.5 points above FP16 vs 0.1–0.2 for Q4_K_M.

Still seen in the wild because some early GGUF model releases shipped only Q4_0 and Q8_0. If you have a choice, prefer Q4_K_M.

An operator downloads a community fine-tune of Mistral 7B from 2023 and finds the repository only offers Q4_0 and Q8_0 GGUF files — no K-quants, because the release predates llama.cpp's importance-matrix quantization work. Rather than serving Q4_0 directly, they use llama.cpp's quantize tool to re-quantize from a higher-precision source (or find an F16 upload of the same fine-tune) into Q4_K_M, since Q4_0's lack of an importance matrix means it spends the same bit budget on every weight regardless of sensitivity, producing a visibly higher perplexity than K-quants at a nearly identical file size. The only scenario where they'd deliberately keep Q4_0 is deploying to legacy ARM devices where older llama.cpp builds lack optimized K-quant kernels — in that narrow case, Q4_0's simpler dequantization path can still edge out Q4_K_M on raw decode speed.