Mirostat Sampling

Mirostat is a sampling algorithm that targets a fixed perplexity-like "surprise" level (tau) instead of a fixed top-p or top-k. The decoder dynamically tightens or loosens the candidate set at each step to keep the running entropy near tau.

Two variants: Mirostat v1 (the original) and Mirostat v2 (simplified, used by llama.cpp). Both expose tau (target surprise, typical 5.0) and eta (learning rate, typical 0.1).

Some users find Mirostat reduces repetition and produces more coherent long-form generations than top-p alone. Empirically the difference is task-dependent and usually small; top-p with a reasonable repeat penalty handles most cases. Worth knowing because it's a built-in option in llama.cpp/Ollama and shows up in older Reddit threads.

A user running long-form story generation on a 7B model through Ollama notices that top-p 0.9 sampling degrades into repetitive loops after roughly 800-1000 tokens — the model latches onto a phrase and recycles it. Switching to Mirostat v2 with tau=5.0, eta=0.1 keeps the perceived surprise level constant across the whole generation instead of letting the candidate distribution collapse, and the repetition loop stops recurring in that specific test. They also try tau=3.0 for a tighter, more focused output on a coding-assistant prompt and find it's harder to tune predictably than just lowering top-p and raising the repeat penalty — for short, structured completions Mirostat's dynamic tuning doesn't clearly outperform static sampling, so they keep it only in the long-form creative preset.