How an LLM predicts the next token
A large language model generates text one token at a time. At each step it produces a probability for every possible next token, then picks one and repeats. That's the whole loop — there is no plan for the full sentence, just a very good guess at what comes next.
The 'temperature' setting controls how adventurous that pick is. Low temperature always takes the safest, highest-probability token; high temperature samples more surprising ones, trading reliability for creativity.
Turn the temperature — reshape the odds
The cat sat on the ___
Top token probability
73%
Balanced: usually sensible, with room to vary.
Free · runs entirely in your browser · nothing to install
How to use it
- Read the bar chart: each bar is a candidate next token and its probability.
- Lower the temperature and watch the model concentrate on the top choice.
- Raise it and watch probability spread out to riskier options.
What you'll take away
- Why LLMs are 'next-token predictors' at heart.
- What temperature does and when to turn it up or down.
- How sampling choices shape the personality of the output.
Want to actually build this?
This demo is one moment inside a full Math to Machine lesson — predict, build, and explain the concept, with an AI tutor that gives hints, not answers. The first five lessons are free.
FAQ
- How does an LLM decide what word comes next?
- It computes a probability distribution over all possible next tokens given everything so far, then samples one token from that distribution. It repeats this for every token it generates.
- What is temperature in an LLM?
- Temperature controls randomness in sampling. Near zero, the model almost always picks the single most likely token (focused, repetitive). Higher values flatten the distribution so less likely, more creative tokens get chosen.