Studying vs Taking the Test

A quick note on how we teach here. You won't find equations or numbers in this series, and that is on purpose. The goal is understanding, not arithmetic, so every idea is explained in plain words and simple pictures. When you want the math later, the rest of Internals Decoded is ready for you.

Welcome back, friend. In the last few pieces we put together a big idea: a machine can get better at guessing by adjusting tiny knobs, layer after layer, until it finds deep, hidden patterns. That process is slow and painstaking. It takes an enormous amount of work. But once the machine has done all that work, something wonderful happens. Asking it a question becomes almost effortless. It can answer in the blink of an eye. And it never forgets what it learned.

Here is the one idea for today: a model lives two very different lives. First it learns, slowly and expensively, from mountains of examples. Then it is used, quickly and cheaply, to answer one thing at a time. Those two activities are not the same at all. Think of studying for a big exam, then sitting the exam itself. The months of studying are where all the growth happens. The exam is just a moment where you apply what you already have. You do not learn anything new during the test. You simply use the knowledge that is already tucked away inside you.

The two lives of a model

Picture a student preparing for a history exam. For weeks and weeks she reads books, takes notes, makes flashcards, gets things wrong, corrects herself, and slowly builds a picture of the past inside her head. All that effort is exhausting. It burns energy. It takes time. And the whole point is to change what she knows.

Then the day of the exam arrives. She sits down, reads the first question, and answers it. She does not stop to open a textbook. She does not change her understanding. She just calls on the patterns already formed in her mind. The exam itself is quick. It costs her very little effort compared to all those weeks of study.

A learning machine does exactly the same thing, but with knobs instead of brain cells. First it goes through a long, slow, expensive period where it sees example after example. With each example it nudges its knobs ever so slightly so that its guesses get a tiny bit better. This is the studying phase. Then, later, when someone asks it a question, it simply takes the input, runs it through the settled knobs, and produces an answer. No knobs change. No new learning happens. The answer just falls out, fast and cheap, like a well rehearsed tune.

Learning is changing knobs

Remember those knobs we first met back in part thirteen? A model is full of them, sometimes an enormous number. During the studying phase, the machine spends all its time adjusting those knobs. Every time it sees a new example, it makes a guess, checks how far off it was, and gently turns the knobs so that next time it would have been a tiny bit closer. It does this over and over again, on more examples than you could count in a lifetime. That is why the studying phase takes so long and uses so much energy. The machine is doing the mental equivalent of reading every book in a library, taking notes, and rewriting its own understanding.

You can think of this as a long, slow walk across a hilly landscape in thick fog. The machine cannot see the lowest valley from where it stands. It can only feel the ground right beneath its feet and take a small step downhill. It repeats this step after step, learning the shape of the land. Eventually, after all that work, the knobs settle into a shape that works well for the task. That carefully shaped set of knobs is what the machine will carry forward into its second life.

Answering is using what it already has

Once the studying is done, the model enters its second life. Someone hands it a new input and asks a question. The model does not learn from this. It does not update any knobs. It simply takes the input, passes it through the settled layers of knobs, and produces an answer. This is the exam. And because no learning is happening, it is fast. Very fast.

Imagine a trained piano player. When she first learned a difficult piece, she spent months practicing slowly, making mistakes, correcting them, building the muscle memory. But now, when someone asks her to play it at a concert, she does not practice. She just plays. Her fingers know the path. The answer flows out. The same is true for a model after studying. The hard work already happened. Now it is just replaying what it knows.

This is why people can build a model at great expense and then share it with the world. The studying phase might cost a fortune in electricity and time. But once it is done, that model can answer millions of questions for pennies. The heavy lifting is behind it.

The names people use

Because these two lives are so different, the people who build these machines have given them two distinct names. They call the studying phase training. And they call the test taking phase, the moment when the model answers a question without learning anything new, inference.

When you hear someone say that building a model cost a fortune but asking it a question costs almost nothing, they are talking about the difference between training and inference. You are now in on that secret.

Where this is heading

Today we saw that a model does not do the same thing its whole life. First it trains, slowly shaping its knobs on a mountain of examples. Then it infers, answering quickly and cheaply with the knobs it already has. Training and inference. Two lives, one set of understanding.

And now we have all the pieces. We know about patterns, about turning meaning into places, about learning from mistakes, and about deep layers of tiny decision makers. It is time to put a name to the whole thing, the one word that sits at the centre of every AI conversation. In the next piece we will finally talk about what that word really means. What do we actually mean when we say a model? I cannot wait to show you.