Turning Meaning Into Places

A quick note on how we teach here. You won't find equations or numbers in this series, and that is on purpose. The goal is understanding, not arithmetic, so every idea is explained in plain words and simple pictures. When you want the math later, the rest of Internals Decoded is ready for you.

Welcome back. Last time, we saw that a pattern is just a repeating shape in the way a machine’s switches light up. Once a machine can spot a pattern, it can tell the difference between a cat and a cup, or between a happy sentence and an angry one. Today we ask a new question. How can a machine handle something as slippery as meaning when all it has to work with is on and off?

A machine handles meaning by turning each thing it knows into a place on a map. Similar things end up close to each other. Unalike things end up far apart. Closeness on that map becomes the machine’s only way of saying, “These two belong together.”

A supermarket for meaning

Think of the last time you walked into a large supermarket. You probably did not stop and read a manual. You just knew without thinking that milk would be somewhere near the cheese and the yogurt. Baking flour would be near the sugar and the chocolate chips. Nobody would put the toothpaste next to the oranges. The whole store is arranged so that things that go together live near each other. The arrangement itself tells you something about what those things are.

A library does the same. Travel guides sit with travel guides. Cookbooks share a shelf with other cookbooks. If you pulled a book at random and looked at its neighbors, you could guess its topic before even reading the title. That is because the shelf position carries meaning. The meaning is “this book is about food,” not because a librarian whispered that to the shelf, but because all the other food books are right there.

A machine builds exactly this kind of map, but inside its own world of switches. It takes every word, every picture, every sound it can learn about and gives each one a little spot on a giant imaginary map. The map is not flat like a supermarket floor. It is more like a huge, open field that stretches in many directions at once. You cannot draw it on a piece of paper because it has too many dimensions, but you can still picture the idea. Each thing sits somewhere.

How words find their place

Picture that huge empty field. The machine places the word “king” at some random spot. Then it places “queen” nearby, then “prince,” then “princess.” Soon the whole royal court is clustered in one corner. On the far edge of the field, you find “banana,” “apple,” and “grape” huddled together. In another area, you see “run,” “jump,” and “walk” all crowded close.

Why did the machine put them there? Not because it opened a dictionary. Not because someone told it that a king is a ruler. The machine has never understood anything the way you and I do. It only watched many, many sentences. It noticed that whenever the word “king” appeared, the word “queen” often appeared nearby, or in a very similar spot in the sentence. The same kind of other words surrounded them both. “Banana,” on the other hand, showed up with “smoothie” and “yellow” and “peel,” almost never with “throne” or “crown.” Over time, the machine nudged its little pins around the field until words that keep each other’s company ended up as neighbors. Closeness on this map became a stand in for similarity of meaning. The map turned into a map of meaning.

The machine never “knows” what a king is. But because it placed “king” near “queen” and far from “banana,” it can answer a question like “what is a king?” by saying, “It is like a queen, but slightly different, and definitely not a banana.” That might sound silly, but this simple sense of near and far unlocks a huge amount of power.

The names people give those places

Now you have a name for a spot on this map. People call that position an embedding. When you hear someone say that a word has been turned into an embedding, they just mean that the machine gave it a place on the map of meaning. The idea of a position itself, the little arrow that points from the center of the field to that exact spot, is called a vector. Both words, embedding and vector, are used constantly in AI. But now you know they simply mean: a place on the map.

Everything gets an embedding. A whole sentence can have an embedding. A photograph of a cat can have an embedding. In each case, the machine looks at the thing, decides where on the map it belongs, and plants it there. Things that are alike end up as neighbors.

Where this is heading

The machine can now capture meaning without ever understanding a single word. It does this by giving each thing a home on a map, a home that was chosen purely by watching what appears with what. Combine that idea with the patterns we talked about earlier, and the machine gains a startling new ability. It can look at a half finished sentence and guess which word is likely to come next. It can look at part of a picture and guess what should fill the rest. This simple trick, guessing what comes next, is the engine underneath so much of the AI you hear about today.

In the next piece, we will walk through exactly how a machine uses the map of meaning to make those guesses. You will see that predicting the next word is not magic. It is just a game of noticing which paths across the map are the most worn.