TL;DR
Temperature is the randomness dial. Low temperature makes the model pick the most likely next token almost every time (consistent, predictable); high temperature lets it sample more widely (varied, more creative, less reliable). It changes how the model chooses words — not how many it writes.
The idea
When an LLM writes, it builds the answer a little at a time, and at each step it has many options for what comes next — some much more likely to fit than others (you saw this idea at the end of Lesson 1). Temperature is the dial that decides how adventurously it picks among those options.
- At temperature 0, the model almost always takes the single most likely option. The same prompt produces nearly the same answer every time — predictable and reliable.
- At temperature 0.9, the model takes more chances on less likely options. The same prompt produces different answers each run — sometimes more creative, sometimes off the rails.
Analogy
Temperature is how loaded the dice are. At 0, the dice always land on the favorite — same result every roll. Crank it up and the underdogs start winning sometimes: more surprising, less predictable.
Example: matching temperature to the task
The right setting depends entirely on the job:
| Use case | Temperature | Why |
|---|---|---|
| Extracting data, classification, JSON output | ~0 | You want the same correct answer every time. |
| Support replies, summaries | ~0.3–0.5 | Mostly consistent, a little natural variation. |
| Brainstorming names, creative writing | ~0.7–1.0 | You want variety and surprise. |
A billing agent that must return a refund amount should run near 0 — you never want creativity in a dollar figure. A naming assistant should run hot, because ten different ideas beat the same idea ten times.
Try it yourself
The same prompt, run at three different temperatures. Drag the slider to flip between them. Notice that token usage stays roughly the same — temperature does not change how long the response is, only how it's sampled.
Common mistake
High temperature isn't "smarter" — it's just more random. If you're getting inconsistent or wrong answers on a task that has one right answer, turn the temperature down, don't up.
Key takeaways
Key takeaways
- Temperature controls randomness in how the next token is chosen.
- Low (
0) = deterministic and reliable; high (0.9) = varied and creative. - Match the setting to the task: facts run cold, creativity runs hot.
- Temperature changes how the model picks words, not how many.
Go deeper: top-p and other sampling knobs
Temperature isn't the only sampling control:
- Top-p (nucleus sampling). Instead of considering every token, the model keeps only the smallest set of tokens whose probabilities add up to p (say 0.9) and samples from those. It's another way to bound randomness; many setups tune temperature or top-p rather than both.
- Top-k. Similar idea — keep only the k most likely tokens before sampling.
- Determinism caveat. Even at temperature 0, results aren't always bit-for-bit identical across runs due to how computations are batched on hardware — but they'll be very close. For truly repeatable evals, pin every setting and expect near-determinism, not a guarantee.