Guessing What Comes Next
A huge part of intelligence is guessing what comes next. The simple game at the heart of today's chatbots.
A quick note on how we teach here. You won't find equations or numbers in this series, and that is on purpose. The goal is understanding, not arithmetic, so every idea is explained in plain words and simple pictures. When you want the math later, the rest of Internals Decoded is ready for you.
Welcome back. Last time we took a word, or even a whole idea, and gave it a spot in an imaginary space. Words that mean similar things end up close together, while unrelated words drift apart. That is a useful way for a machine to hold onto meaning. But holding meaning is only half the story. Today we put it to work, and meet the game at the heart of today's chatbots and writing assistants. At first it sounds laughably simple. Yet once you see it, you will recognize it everywhere.
A huge part of intelligence is guessing what comes next based on what came before. When you know the world well enough, the next step often feels obvious.
Finishing the sentence together
Think about a conversation with a close friend. You are talking about a long day and she says, "I could really go for a cup of..." Your mind has already leaped ahead. Coffee. Or tea. Maybe hot chocolate if the day was particularly cold. You did not read her thoughts. And you are not a magician. You have simply lived through enough tired afternoons, and heard enough sentences just like this one, to know what usually follows. Your brain is a giant collection of lived patterns. Give it the first part, and it will hand you back a short list of likely endings.
This everyday habit is not a little party trick. It is a deep form of understanding. To guess what comes next, you need to have a feeling for how the world works. You need to know what a cup is for. You need to know what people reach for when they are tired. A good guess tells you that the listener really got the setup.
The machine that plays the same game
A computer can learn to play this game too, on a scale far bigger than any one person. Imagine a machine that has read a whole library of everyday writing. Letters, stories, chat messages, recipe directions. It does not understand thirst or tiredness the way you do. But it has seen the phrase "I could really go for a cup of" followed by "coffee" so many times that the pattern is clear. It has also seen the other words that sometimes fill that spot, and how often each one turns up. And it can handle sentences it has never seen before, because it learned the patterns, not just the exact phrases.
Here is what happens when the machine reaches the blank. It takes in all the words so far, together, and works out what kind of word belongs next. Then it gives every word it knows a score for how well that word would fit. The map from last time helps here. If "coffee" fits, "tea" sits nearby on the map and scores well too, while "banana" sits far away and scores low.
Picture a very long scoreboard with every word the machine knows written down one side. Next to "coffee" and "tea" sit long bars. "Cocoa" has a shorter one. "Banana" and "bicycle" have bars so short you can barely see them. The words with the longest bars are the machine's likely guesses. Notice that the scores are about what fits next, not about what means the same. "Mug" means something close to "cup", but "I could really go for a cup of mug" makes no sense, so "mug" scores low here.
One word at a time
Once the machine has picked a word, it tacks that word onto the end and starts again. Suppose it has the words "I could really go for a cup of" and picks "coffee". Now it looks at the slightly longer phrase: "I could really go for a cup of coffee." What comes next? Maybe a period. Maybe "right", as in "right now". Maybe "and", to add a snack. It guesses again. And then again. Word by word, it builds whole paragraphs that sound like something a person might write.
There is no finished script stored anywhere for the whole reply. The words come out one at a time. That does not mean the machine has no sense of where it is going. Researchers at Anthropic, the company that makes the Claude assistant, looked inside one of their models while it wrote rhyming poetry, and wrote up what they saw in Tracing the thoughts of a large language model. They found it settled on the rhyming word before it wrote the line that ends with it. Some newer models also write rough working notes before they give their answer, which their makers call thinking. But even those notes come out the same way, one guessed word after another. From the outside, this can look like a mind at work. Underneath, it is a long, careful round of finish the sentence.
The one game underneath the noise
You may have heard people talking about large language models. The technology has felt sudden and overwhelming. But strip away the complexity and you find this small, stubborn process at the center. A machine sees the words so far. It picks a likely next word. It writes it down. It looks again.
People in the field call this next word prediction. You will also hear next token prediction. A token is a word or a piece of a word, and tokens get their own piece later in this series, Breaking Language Into Pieces. For now, it is enough to think of a token as a word.
Next word prediction sounds too simple to matter. And yet it is the engine inside the AI writing assistants, chatbots and question answerers that fill the news. It is not the whole story, though.
From sentence finisher to helpful assistant
A machine that has only practiced this game is a sentence finisher. Ask it a question and it may simply carry on the text the way a page on the internet might, sometimes with more questions instead of an answer. So the assistants you use get a second round of training. After the machine has practiced next word guessing on a mountain of writing, it practices on examples of good answers, and on ratings that say which of its replies people preferred. That teaches it to use the same guessing game to be helpful. The companies that build these assistants describe both rounds in their own guides, such as Anthropic's glossary. The game itself never changes. Every reply is still built one guessed word at a time.
Where this is heading
A big slice of intelligence is very good guessing about what comes next. A language model plays that game one word at a time: it reads everything so far, scores every word it knows, picks a likely one, and goes again. Nobody wrote rules for this skill. The machine got good at the game by practicing on that huge library, guess after guess, and how that practice works is a story the series tells a little later.
But "pick a likely word" raises a question we have been stepping around. What does it mean for one word to be more likely than another, and how does the machine choose among them? That is where we go next, and it is gentler than it sounds.
Test yourself
Finish this sentence: "The early bird catches the..." Now name two other words that could fit. Which would a machine score highest, and why? And what does the machine do once it has picked a word?
Answer: "Worm" would score highest. The machine has seen this saying end with "worm" again and again in its library of writing, so "worm" gets by far the longest bar on its scoreboard. Words like "bus" or "train" could fit a sentence about catching something, so they get some score, but much less. A word that means something close to "bird", like "robin", scores low, because the scores are about what fits next, not about what means the same. Once the machine has picked a word, it adds it to the end, reads everything again, and scores every word once more to choose the next one. It repeats this, one word at a time, until the reply is done.
Next: How Likely, Really. It explains what "likely" means to a machine, and why it does not always pick the word at the top of the scoreboard.
