The Knobs Inside
Learning is just turning a vast bank of tiny knobs until the machine behaves. Meet parameters.
A quick note on how we teach here. You won't find equations or numbers in this series, and that is on purpose. The goal is understanding, not arithmetic, so every idea is explained in plain words and simple pictures. When you want the math later, the rest of Internals Decoded is ready for you.
Welcome back. We have spent some time now watching a machine learn from examples. We saw it make a guess, measure how far off it was, and then adjust itself a little. Last time we saw how it adjusts: backpropagation works out which way each knob should turn, and gradient descent takes a small step downhill toward a smaller loss. But we have been talking about those knobs as if they were one big thing. Today we open the lid and look inside. What is the machine actually turning when it learns?
Inside the machine sits a huge collection of tiny adjustable settings. Learning is nothing more than turning these settings, one small nudge at a time, until the whole thing behaves the way we want. When people talk about how "big" an AI is, they are usually counting these settings.
A giant mixing desk
Think about a music studio. Inside the recording room sits an enormous mixing desk, covered in knobs and sliders. There are so many you could not count them all just by looking. Each knob does something tiny on its own. One might brighten the sound of a cymbal. Another might add a little warmth to a voice. A slider might push a guitar further back in the mix. Alone, each change is barely noticeable. But together, set just right, all those tiny adjustments turn a rough, flat recording into a rich, beautiful song.
The craft of the sound engineer is patience. She listens, turns a knob a hair, listens again, nudges a slider, and keeps going until everything blends perfectly. No single knob holds the song. The song lives in the whole arrangement of them.
This is exactly what happens inside the machine we have been training. It has a vast mixing desk of its own. Every tiny setting is like one of those knobs. Training is the long, patient process of listening to the machine's output and nudging the knobs until the result sounds right.
Tiny nudges, big changes
Remember how training works. The machine sees an example, makes a guess, and then we show it the correct answer. It measures the gap between its guess and the truth. Then it works backward through its own mixing desk, the backward pass called backpropagation, and nudges every knob on the desk a tiny amount, some more than others. It does not turn them randomly. It turns each one in the direction that would have made its guess a little closer to the truth. Then it moves on to the next example and does it all over again.
Over many rounds of practice, some knobs end up moving a lot. Others barely move. The whole desk slowly settles into a shape that produces good guesses for all kinds of examples. The remarkable part is that no single knob holds a complete idea. You cannot point to one knob and say, "This one knows about cats," or "This one handles the color blue." The skill lives in the whole arrangement, the way a melody lives in all the notes together, not in any single note.
The name for these knobs
People who build these machines have a name for these tiny adjustable settings. They call them parameters. You will also hear them called weights, because most of them set how much weight, or importance, one signal gets as it passes from one part of the machine to the next. When a news headline brags about an AI with a huge number of parameters, it is bragging about how many of these tiny knobs the machine has to tune.
Those counts are enormous. Meta, the company behind Facebook, has openly shared a version of its Llama machine with many times more knobs than there are people on Earth, as Meta's announcement of that Llama shows. And every one of those knobs was set by practice, not by hand.
A mixing desk with more knobs can shape sound in more detailed ways, if you have the patience to set them all. In the same way, a machine with more parameters can learn more subtle patterns. But it also takes far more practice to get all those knobs just right, so bigger is not automatically better. In a well known paper called Training Compute-Optimal Large Language Models, a smaller machine that practiced on far more examples beat machines much bigger than itself.
So when you hear that an AI has a certain number of parameters, you know what is being counted: the knobs inside.
Where this is heading
Inside every trained machine sits a bank of parameters, set through patient practice. Learning is the craft of turning them until the whole thing behaves. But we have been talking about "the machine" as if it were one shapeless blob. It is not. These knobs are arranged in a particular shape inside, a structure that helps the machine find patterns. That arrangement is loosely borrowed from something you already know, something familiar and alive.
Test yourself
A news story says one AI is "bigger" than another. Bigger in what way? And does bigger automatically mean better?
Answer: Bigger here means it has more parameters, more knobs on its mixing desk. More knobs give it room to learn subtler patterns, the way a bigger desk can shape sound in finer detail. But bigger does not automatically mean better. Every one of those knobs has to be set through practice, so a bigger machine needs far more examples and far more training to set them well. A smaller machine that practiced well can beat a bigger one that did not. Size tells you how much room there is to learn, not how much was actually learned.
Next: Loosely Inspired by the Brain. It shows the shape those knobs are arranged in, and the one small idea that shape borrowed from the brain.
