How ChatGPT picks the next word
Take this if you use a chatbot most days and have no idea what happens between your question and its answer.
What you’ll be able to do
You can explain what a token is, why a model produces a distribution rather than an answer, and why a confident wrong answer is the expected behaviour rather than a bug.
What it assumes
You have used ChatGPT, Claude or Gemini at least once. No maths and no coding required.
- Price
- Free
- Length
- 1 week
- Sessions
- 4
- Effort
- ~2 h/week · 2 h total
- For
- Class 11–12 · College · Anyone
Adds it to My courses on your dashboard. No card, nothing to cancel.
What’s taught, in order
- 01
Text is not words
Running your own name, a Hindi sentence and a long number through a real tokenizer, and finding that the model does not see any of them the way you do. Most surprising model behaviour starts here.
- 02
Prediction as a distribution
The model does not choose a next word, it scores every possible one. You commit to your own top five before seeing the model's, which turns an abstraction into a game you can win or lose.
- 03
Temperature and the illusion of creativity
The same distribution, sampled two ways, produces a dull answer and a wild one. Seeing that creativity is a sampling setting rather than a property of the model is the session that changes how people prompt.
- 04
Why it bluffs
A model that always has a next-token distribution never has the option of silence. Following that one fact to its conclusion explains hallucination better than any list of tips.
What you build
Find three inputs that a real tokenizer handles badly — try your own name, a word in an Indian language, a long decimal number, an emoji. For each, record how many tokens it became and write one sentence predicting a task the model will therefore be bad at.
Check it yourself against this
- You have three inputs that each tokenize in a surprising way.
- For at least one, you predicted a downstream failure and then confirmed it in a chatbot.
- You can explain why counting letters in a word is hard for a model.
- You can state, in one sentence, what a token is without using the word word.
Then answer this
If a model only ever predicts the next token, why does a long answer of many tokens still stay on topic?
Two or three sentences, in your own words. If you cannot, the course is not finished — go back to the module it came from.
What this course does not do
This course explains what a language model does, not how it does it — attention, transformer blocks and training are all out of scope, and it will not teach you to build one. It also stays away from why a particular model gives a particular answer, which nobody can currently explain in full.
Where this leads
Both interactive demos this course is built around are free standalone pages, so you can break a tokenizer and play the next-token prediction game right now without signing up for anything.
FAQ
- Does this teach prompt engineering?
- Not directly, but it explains the mechanism that every prompting technique is exploiting, which tends to be more durable than a list of tricks. If you want the practical side, the Prompting as Engineering sprint and the free prompt patterns library both build on this.
- Is this specific to ChatGPT?
- No. Tokenization, next-token prediction and temperature are how Claude, Gemini, Llama and every other current large language model work. The tokenizer details differ between them, and the course points out where.
- I am in Class 9. Is this too advanced?
- It is pitched at Class 11 and up because the sampling discussion assumes you are comfortable with the idea of a probability, but there is no formal maths in it. If you are comfortable with percentages you will be fine.
Other courses
This is a standalone course. It does not add weeks to the 78-week curriculum or change your roadmap. If you want the whole path instead, the full programme is ₹399/month · ₹2,000/year — see what it covers.