Tiny Decision Makers in Layers

A quick note on how we teach here. You won't find equations or numbers in this series, and that is on purpose. The goal is understanding, not arithmetic, so every idea is explained in plain words and simple pictures. When you want the math later, the rest of Internals Decoded is ready for you.

Welcome back. Last time we saw that neural networks borrowed a picture from the brain: many simple parts passing signals to each other, joined by connections whose strength can change as the machine learns. It is not a copy of a brain. It is more like a rough sketch that gave people a place to start. Today we look at those simple parts up close, and at how they are arranged.

A neural network is built from many tiny parts, each making a very small decision, arranged in layers so that one layer's answers become the next layer's questions. Simple parts, stacked together, add up to something capable.

The tiny decision maker

Picture yourself deciding whether to go and see a film. One friend, whose taste you trust completely, says it is wonderful. Another friend, who loves everything, says it is good. A review you only half trust says it is dull. You do not count the opinions equally. You give each one a weight, depending on how much you trust it, put them all together, and end up keen, lukewarm, or not interested at all.

Each tiny part of a neural network makes a decision just like that. It receives signals from many parts before it. Some signals are strong. Some are weak. The part has its own set of knobs, like the ones we met a couple of pieces ago. Those knobs decide how much weight the part gives to each incoming signal, the way you decided how much to trust each friend.

The part mixes the signals together, guided by its knobs. Then it makes one small judgment: should it pass a strong signal forward, a weak one, or almost nothing at all? It does not think in words. It just does a little bit of arithmetic, the kind a pocket calculator could handle, and then decides how loudly to speak up to the parts that come after it.

People call each of these tiny decision makers a neuron. The name is a small nod to that loose brain idea we talked about last time. A real brain cell is far more complex, but the name stuck. In a machine, a neuron is just a very simple rule that takes what it is handed, weighs it, and passes on a result.

A row of decision makers

When you put a whole row of neurons side by side, all of them working on the same input at the same time, you get something called a layer. A layer is like a panel of judges, each one looking at the same thing but through its own set of knobs. In a network that looks at pictures, one neuron might respond to a certain kind of edge. Another might respond to a patch of color. Together, the layer paints a rough picture of what it sees.

The answers from one layer do not go straight out as the final answer. They become the input for the next layer, which weighs them with its own knobs and passes on its own answers. And so on, layer after layer, until the last layer gives the network's guess.

Building a picture, layer by layer

Here is what that stacking can do. The first layer in a network that looks at images might notice very simple things: a bright spot here, a dark line there. The second layer receives the first layer's rough notes. It can combine those notes into slightly more interesting things: a corner, a curve, a patch of texture. The third layer might piece those together into something that looks like an eye or a wheel. A layer after that might put eyes, ears and fur together into a face. Each layer's answers become the next layer's questions.

This is how meaning gets built up from almost nothing. No single neuron understands the whole picture. No single layer does either. But the stack of layers, each one building on what the last one handed it, can eventually recognize a cat, or a handwritten word, or a familiar voice.

This is the neural network we named last time: neurons, arranged in layers, passing signals forward. Simple parts, stacked, add up to something capable.

Where this is heading

A network is made of many tiny decision makers, each weighing its signals and passing on one small judgment, lined up in layers that build on each other. If a few layers can do useful work, what happens when you stack many, many layers deep? Something changes. The network can learn ideas that are far more complex and subtle. That depth is the reason for a phrase you have surely heard: deep learning.

Test yourself

A network that reads photos has many layers. Which layer would you expect to notice "a whisker", and which would notice "a cat face"? Why not the other way round?

Answer: An early layer would notice the whisker, and a later layer the cat face. A whisker is a simple thing, little more than a thin line, and the early layers are the ones that spot simple marks like lines and edges. A cat face is built out of many simpler pieces: eyes, ears, fur, whiskers. Each layer only works with what the layer before it handed over, so the face can only be put together once earlier layers have found those pieces. It cannot go the other way round, because a layer near the start has nothing to build a face from yet.

Next: Why Deeper Is Different. It stacks many more layers and shows why depth changes what a network can learn, which is where the phrase deep learning comes from.