Studying vs Taking the Test
A model has two lives: slow, costly learning, then fast, cheap answering. Training versus inference.
A quick note on how we teach here. You won't find equations or numbers in this series, and that is on purpose. The goal is understanding, not arithmetic, so every idea is explained in plain words and simple pictures. When you want the math later, the rest of Internals Decoded is ready for you.
Welcome back. In the last few pieces we put together a big idea: a machine can get better at guessing by adjusting tiny knobs, layer after layer, until it finds deep, hidden patterns. That process is slow and painstaking. It takes an enormous amount of work. But once the machine has done all that work, something wonderful happens. Asking it a question becomes far easier. It can answer in moments. And because its knobs no longer change, what it learned stays put. It keeps patterns, not a perfect copy of everything it read.
Here is the one idea for today: a model lives two very different lives. For now, "model" just means the machine we have been training; the next piece pins the word down properly. First it learns, slowly and expensively, from mountains of examples. Then it is used, quickly and cheaply, to answer one thing at a time. Those two activities are not the same at all. Think of studying for a big exam, then sitting the exam itself. The months of studying are where all the growth happens. The exam is just a moment where you apply what you already have. You do not learn anything new during the test. You simply use the knowledge that is already tucked away inside you.
The two lives of a model
Picture a student preparing for a history exam. For weeks and weeks she reads books, takes notes, makes flashcards, gets things wrong, corrects herself, and slowly builds a picture of the past inside her head. All that effort is exhausting. It burns energy. It takes time. And the whole point is to change what she knows.
Then the day of the exam arrives. She sits down, reads the first question, and answers it. She does not stop to open a textbook. She does not change her understanding. She just calls on the patterns already formed in her mind. The exam itself is quick. It costs her very little effort compared to all those weeks of study.
Notice what kind of studying pays off. A student who only memorized last year's answers does badly the moment the questions change. A good student learns patterns that work on questions she has never seen. A good machine has to do the same, which is why the people building one always check it on examples it never studied.
A learning machine lives these same two lives, but with knobs instead of brain cells. First it goes through a long, slow, expensive period where it sees example after example. With each example it nudges its knobs ever so slightly so that its guesses get a tiny bit better. This is the studying phase. Then, later, when someone asks it a question, it simply takes the input, runs it through the settled knobs, and produces an answer. No knobs change. No new learning happens. The answer comes out quickly, like a well rehearsed tune.
Learning is changing knobs
Remember the knobs, the parameters from The Knobs Inside? A model is full of them, sometimes an enormous number. During the studying phase, the machine spends all its time adjusting those knobs. Every time it sees a new example, it makes a guess, checks how far off it was, and gently turns the knobs so that next time it would have been a tiny bit closer. It does this over and over again, on more examples than you could count in a lifetime. That is why the studying phase takes so long and uses so much energy. The machine is doing the mental equivalent of reading every book in a library, taking notes, and rewriting its own understanding.
You can picture this as the foggy valley walk from Practice Makes It Smaller. The machine cannot see the lowest valley from where it stands. It can only feel the ground right beneath its feet and take a small step downhill. It repeats this step after step. Eventually, after all that work, the knobs settle into a shape that works well for the task. That carefully shaped set of knobs is what the machine will carry forward into its second life.
Answering is using what it already has
Once the studying is done, the model enters its second life. Someone hands it a new input and asks a question. The model does not learn from this. It does not update any knobs. It simply takes the input, passes it through the settled layers of knobs, and produces an answer. This is the exam. And because no learning is happening, it is fast.
When a chatbot seems to remember something you said earlier in a conversation, that is not learning either. The app hands the whole conversation so far back to the model with every new message, so the model can read it again, like a student who can look back at the earlier questions and answers on her own exam paper. Anthropic's guide to its Messages API says it plainly: you always send the full conversation history. Some companies may also use conversations, when your settings allow it, to help train a later version, as Anthropic explains for its Claude apps. But that is a new round of studying for a new version, not learning during the exam.
Imagine a trained piano player. When she first learned a difficult piece, she spent months practicing slowly, making mistakes, correcting them, building the muscle memory. But now, when someone asks her to play it at a concert, she does not practice. She just plays. Her fingers know the path. The same is true for a model after studying. The hard work already happened. Now it just uses what practice built.
This is why people can build a model at great expense and then share it with the world. The studying phase might cost a fortune in electricity and time. Once it is done, each single answer costs only a tiny amount next to the training. But a popular model answers a huge number of questions for a huge number of people, every day, and all those tiny amounts add up to a large bill of their own.
The names people use
Because these two lives are so different, the people who build these machines have given them two distinct names. They call the studying phase training. And they call the test taking phase, the moment when the model answers a question without learning anything new, inference.
When you hear that building a model cost a fortune while asking it one question costs very little, that is the difference between training and inference. Later, when you want the real costs and the machinery behind them, with numbers, Training vs Inference: Why Building Costs Millions and Asking Costs Cents in the series AI, From Zero picks this up.
Where this is heading
A model does not do the same thing its whole life. First it trains, slowly shaping its knobs on a mountain of examples. Then it infers, answering quickly with the knobs it already has. Training and inference. Two lives, one set of knobs.
And now we have all the pieces. We know about patterns, about turning meaning into places, about learning from mistakes, and about deep layers of tiny decision makers. It is time to put a proper name to the whole thing, the one word that sits at the center of every AI conversation. What do we actually mean when we say a model?
Test yourself
Early in a conversation, you tell a chatbot your name. Later in the same conversation, it uses your name. A friend says this proves the chatbot learned something about you. Using studying and the exam, explain what really happened.
Answer: The chatbot was in its exam phase, inference, the whole time. Its knobs did not change while it answered you, so it learned nothing in the training sense. It could use your name because the app hands it the whole conversation so far, including the message where you said your name, every time you send something new. It reads your name the way a student rereads an earlier question on her exam paper. Start a fresh conversation, and unless the app keeps a note of your name and hands that back too, the name is gone. Real learning, changing the knobs, only happens in a new round of training.
Next: What We Mean by a Model. It finally pins down the word model: the finished, trained thing that all this studying leaves behind.
