Getting It Wrong on Purpose

A quick note on how we teach here. You won't find equations or numbers in this series, and that is on purpose. The goal is understanding, not arithmetic, so every idea is explained in plain words and simple pictures. When you want the math later, the rest of Internals Decoded is ready for you.

Welcome back. In the last piece, we watched a machine learn from examples, adjusting its guesses as it saw more and more of them. But we left one big question hanging in the air: how does the machine know how to adjust? The answer begins with something that might surprise you. It begins with getting things wrong.

The heart of this piece is simple. Mistakes are not a nuisance during learning. They are the teacher. Every wrong guess tells the machine how far off it was and in which direction, and that signal is what lets it improve.

A ball and a target

Imagine you are learning to throw a ball at a target. Maybe a bucket across the yard. You have never done it before. Your first throw sails wide to the left. You see it land, and instantly you know: that was too far left. Next throw, you aim a little more to the right. Still not perfect, maybe a bit short. But each miss gives you a clear signal. Without seeing where the ball landed, you could throw all day and never improve. The mistake itself is your guide.

Measuring the miss

A machine learning from examples does something very similar. It makes a guess. Then it looks at the correct answer from the example. It compares the two and asks: how far apart are they? That distance, that gap between what it guessed and what the right answer was, is a measure of how wrong it was. If the guess was close, the gap is tiny. If the guess was wildly off, the gap is large. This gap is not a failure. It is the most useful piece of information the machine has. It is the equivalent of seeing where the ball landed.

Small nudges and big corrections

Once the machine knows the size of its mistake, it can use that to decide how much to change its inner settings. Think back to the ball. If you missed by just a hair, you might only tweak your aim a tiny bit. If you missed by a mile, you would make a much bigger adjustment. The machine does the same. A small miss leads to a small nudge of its knobs and dials. A big miss leads to a bigger correction. This is why the machine needs to see many, many examples. Each one gives it a new chance to measure its mistake and adjust a little. Over time, those nudges add up, and the machine gets better and better at its task.

The whole point of training

When people talk about training a machine learning system, they often say the goal is to "reduce the loss" or "minimize the error." Those words can sound technical and scary. But now you already understand what they mean. The loss is simply the measure of how wrong the machine currently is. A high loss means it is making big mistakes. A low loss means it is getting things right most of the time. The entire training grind, all those example after example, is just a long, patient effort to push that loss down. To make the mistakes smaller and smaller, until they are as tiny as possible. The machine is not trying to be perfect in one giant leap. It is just trying to be a little less wrong with each step, guided by the mistakes it just made.

People who build these systems call this measure of wrongness the "loss" or the "error." When you hear them say that training "reduces the loss," they only mean the machine is getting things wrong less often. That is all.

Where this is heading

Knowing how wrong you are is one thing. Knowing how to change so you are less wrong next time is another. The machine has measured its mistake. Now it needs a clever, patient method to figure out exactly which knobs to turn and by how much, so that the next guess lands closer to the mark. That method is the subject of our next piece. And it is so beautiful to see in motion that we will point you to a wonderful video that makes it crystal clear. For now, just hold onto this: every mistake is a teacher. Without them, a machine could never learn at all.