Temperature 0 (Greedy Sampling)

Temperature 0 disables sampling entirely — the model picks the highest-logit token at every step. Equivalent to greedy decoding with top_k=1. Gives reproducible output (subject to GPU non-determinism) but tends toward repetitive, lower-diversity completions.

Use temperature 0 for: code generation, structured output (JSON), evaluation, debugging quantization quality. Avoid for: creative writing, chat where variety matters.

A common bug: setting temperature to a tiny non-zero value (0.001) hoping for "almost deterministic." This is worse than temperature 0 — sampling still happens, output varies, but the variance is small enough to look stable until it suddenly doesn't.

A team building a JSON-extraction pipeline on Llama 3.1 8B initially sets temperature to 0.2, assuming "low but nonzero" is safer than greedy. In production, roughly 1 in 200 extractions returns malformed JSON — a stray sampled token breaks the schema. Switching to true temperature 0 (greedy) eliminates the failures entirely, because the highest-logit token at a JSON syntax boundary is almost always the syntactically valid one, and removing sampling removes the tail-risk draws that occasionally pick the second-best (invalid) token. The lesson generalizes: for structured output, evaluation harnesses, or any pipeline where output must be parseable, temperature 0 isn't just "more consistent" than 0.1-0.3, it removes an entire class of rare failures that only show up at scale. Creative-writing endpoints on the same runtime stay at 0.7-1.0 for variety.