Monte Carlo Methods

Monte Carlo methods are a class of algorithms that use repeated random sampling to approximate numerical results. In local AI, they show up in sampling strategies during text generation: instead of always picking the most likely token, the model randomly samples from the probability distribution over the vocabulary. This introduces diversity in outputs. Operators encounter Monte Carlo methods when adjusting temperature or top-p sampling parameters in llama.cpp or Ollama — higher temperature increases randomness, making the sampling more 'Monte Carlo-like'.

Monte Carlo methods rely on the law of large numbers: as you draw more random samples, the average of those samples converges to the true value. In AI, they are used not only for text generation but also for Bayesian inference, reinforcement learning (e.g., Monte Carlo tree search in AlphaGo), and estimating model uncertainty. For local operators, the most direct encounter is in the sampling step of autoregressive generation. The model outputs a probability distribution over tokens; a Monte Carlo sample picks a token according to those probabilities. This is controlled by temperature (scaling logits) and top-p (nucleus sampling). Lower temperature makes the distribution sharper, reducing randomness; higher temperature flattens it, increasing diversity. Operators can tune these to balance creativity vs. coherence.

When running Llama 3.1 8B via llama-cli with --temp 0.8, the model uses Monte Carlo sampling: each token is drawn randomly from the probability distribution. At temperature 0.8, the distribution is moderately flattened, producing varied outputs. At temperature 0.0, the model always picks the most likely token (greedy decoding), which is deterministic. On an RTX 3090, both settings run at similar speed (~40 tok/s) because the sampling step is cheap relative to inference.

In Ollama, operators set temperature via the API: curl http://localhost:11434/api/generate -d '{"model": "llama3.1:8b", "prompt": "Hello", "options": {"temperature": 0.7}}'. The runtime applies Monte Carlo sampling to the logits. In LM Studio, the 'Temperature' slider in the UI controls the same mechanism. Operators can also set top_p (nucleus sampling) to limit the sampling pool to the top tokens covering a cumulative probability mass, reducing the chance of sampling low-probability tokens.