What an LLM Actually Is (and Why Predicting Words Works)

A large language model, or LLM, is a computer program that does one thing: given some text, it predicts what comes next, one small piece at a time. ChatGPT, Claude and Gemini are all built on LLMs. There is no database of facts inside, no list of grammar rules and no stored answers. There are billions of numbers, set by training, that turn the text so far into a guess about the next piece.

Everything you have seen these chatbots do starts with the pressure to guess the next word well. Then further training on example conversations and feedback shapes that guessing into a helpful assistant. This piece explains both halves, in plain words.

If you have just finished The Groundwork, welcome back. You already hold most of the words this piece uses: tokens, the map of meaning, the raffle, loss, the knobs and the model. Here they come together, and you meet two new ideas, the transformer and attention. If you are starting here instead, you need nothing else. Every idea is explained as we go.

Start with the autocomplete on your phone

Think about the word suggestions above your phone's keyboard. Type "I'm going to bake" and it might offer "cookies", "a cake" or "bread". A small model looks at what you have typed and ranks the likely next words.

An LLM plays the same game, with three big differences. It learned from a vast amount of text, far more than any person could read in many lifetimes. It takes in a long stretch of text at once, often a whole conversation or document, not just the last few words. And it is enormously bigger, which lets it pick up much deeper patterns.

When you ask a chatbot for a chocolate chip cookie recipe, it does not fetch a recipe from a database. It starts from your message and predicts the first piece of its reply, then the next, then the next. "Ingredients:" leads to "1 cup", which leads to "butter", and so on. The whole recipe rolls out one piece at a time, guided by patterns the model picked up from the many recipes, cooking blogs and forums in its training text. Some chat apps can also search the web and hand the model what they find, but the model still writes the answer the same way, one piece at a time.

What happens inside, in plain words

Here is the whole journey of your message, in three steps. Each step gets its own piece later in this series, so treat this as the short tour.

Text becomes tokens, and tokens become numbers. First, a tool called a tokenizer chops your message into tokens: whole common words, or pieces of rarer ones. Then each token gets a long list of numbers, its place on a map of meaning, where tokens used in similar ways sit close together. That list is called an embedding. Tokens: The Currency of AI and Embeddings: Meaning as Numbers go deeper.

A tall stack of layers reworks those numbers. The design inside most of today's LLMs is called the transformer. Researchers at Google introduced it in 2017 in a paper called Attention Is All You Need. A transformer is a stack of layers, often dozens of them. In each layer, every token looks back at the tokens before it and pulls in information from the ones that matter to it. In "The cake was dry because it baked too long", the token "it" pulls in information from "cake". This looking back is called attention, and Attention, Gently: How Models Decide What Matters gives it a full piece. After attention, each token's numbers are reworked on their own, and the result moves up to the next layer. Layer by layer, a token's numbers come to carry what it means in this particular sentence.

At the top, a probability for every possible next token. After the last layer, the model gives every token it knows a score, and the scores are turned into probabilities: shares that together make up the whole. Then one token is drawn, like a raffle in which likely tokens hold more tickets. It is added to the end of the text, and the whole process runs again for the next token. A reply of a few hundred words is that loop, run hundreds of times.

Diagram loading

All of this is driven by numbers called parameters: the knobs on the giant mixing desk from The Knobs Inside. They decide where each token starts on the map, how attention weighs one token against another, and how each layer reworks the numbers. Nobody sets them by hand. Training sets them, by having the model guess the next token across a huge pile of text and nudging every knob after each guess so that the real next token becomes a little more likely.

How big is big? Meta published unusually complete details for its Llama 3 models in The Llama 3 Herd of Models. Its largest model has 405 billion parameters and was trained on 15.6 trillion tokens of text.

One large model, by the numbers

405 billion

Parameters (knobs) set by training

15.6 trillion

Tokens of text it was trained on

128K

Tokens it can take in at once

For the curiousthe one formula behind next word prediction.

$$\begin{aligned} p(\text{text}) = {} & p(t_1) \times p(t_2 \mid t_1) \\ & \times p(t_3 \mid t_1, t_2) \times \cdots \end{aligned}$$

In words: the chance of a whole piece of text is the chance of its first token, times the chance of the second given the first, times the chance of the third given the first two, and so on. The bar reads "given". An LLM learns to estimate each of those "given everything so far" chances. Training pushes up the chance it gives to the real next token in its training text, and the loss measures how far it still falls short.

Why does predicting the next word work so well?

Guessing the next word sounds like a party trick. It turns out to be a demanding teacher, because to guess well across a vast library of text, a model has to pick up a lot about the world the text describes.

Try it yourself. To finish "The capital of France is", you need a fact. To finish "She dropped the glass, and it", you need a feel for what happens to dropped glass. To finish the last line of a mystery novel, "So the thief was", you need to have followed the whole plot. Every one of those is a next word guess. A model that keeps getting them right has, in a working sense, picked up facts, cause and effect, and how a story hangs together, because those are what make the next word predictable.

There is a neat way to see why. Predicting well and compressing well are two sides of one coin: anything that predicts text well can be turned into a program that squeezes text into less space, and the other way round. A Google DeepMind team showed in Language Modeling Is Compression that large language models make strong general compressors, even of image and audio data, though they were trained mostly on text. To squeeze a library small, you have to find the patterns that repeat across it. Finding those patterns is what learning means here.

The other half of the story is scale. In 2020, researchers at OpenAI found that a model's prediction error falls smoothly and predictably as you add parameters, training text and computing power (Scaling Laws for Neural Language Models). Two years later, a DeepMind study found that model size and training text should grow together: double the model, double the text. Its model Chinchilla had 70 billion parameters, was trained on 1.4 trillion tokens, about 20 tokens for every parameter, and beat much larger models, including DeepMind's own 280 billion parameter Gopher (Training Compute-Optimal Large Language Models).

Model makers now often train smaller models for far longer than that balance suggests, because a small model is cheaper to run every time someone uses it. Meta says it trained its smaller Llama 3 models "for much longer than is compute-optimal", and that they beat compute-optimal models at the same running cost (The Llama 3 Herd of Models). Even its 8 billion parameter model saw about 15 trillion tokens (Llama 3.1 model card).

Scale also brought a skill nobody trained for directly. Show a big model a few examples of a new task inside your message, then a new case, and it often continues the pattern correctly, even though none of its knobs changed. OpenAI's GPT-3 paper made this famous (Language Models are Few-Shot Learners). Researchers who look inside models have since found small circuits, called induction heads, that spot a pattern earlier in the text and continue it, and they argue these may drive much of this skill (In-context Learning and Induction Heads).

From predictor to assistant

A model trained only to predict the next word is called a base model, or a pretrained model. It is a brilliant text continuer, but not yet an assistant. Ask it a question and it may carry on the way a web page might, with more questions instead of an answer. As Anthropic's glossary puts it, pretrained models "are not inherently good at answering questions or following instructions". So the chatbots you use go through more training, in stages.

Pretraining. Next word prediction on a huge pile of text. This is where the model learns its language and most of what it knows. The LIMA study argues that "almost all knowledge in large language models is learned during pretraining" (LIMA: Less Is More for Alignment).

Instruction tuning. Training on examples of requests paired with good answers, often written by people. The model learns the shape of being helpful: answer the question, follow the requested format, stop when done.

Preference training. People compare two or more of the model's answers and pick the better one, and the model is trained toward the kinds of answers people prefer. The best known method is reinforcement learning from human feedback, or RLHF. OpenAI's InstructGPT study showed how much this matters: people preferred answers from a 1.3 billion parameter model trained this way over answers from the 175 billion parameter GPT-3, "despite having 100x fewer parameters" (Training language models to follow instructions with human feedback). Some of the judging can also be done by another AI model following a written list of principles, a method Anthropic introduced as Constitutional AI.

Reasoning training. Newer models are also trained with reinforcement learning on problems whose answers can be checked, such as math and code. They learn to write out working notes before the final answer, often called thinking, and to check and correct themselves along the way. DeepSeek described this openly for its R1 model (DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning). This is what a chat app means when it offers a thinking or reasoning mode.

The game underneath never changes. Every reply, the thinking included, is still built one token at a time. The extra training changes which continuations the model finds likely: a helpful answer instead of rambling, a careful step instead of a leap.

Three things that follow

Knowing what an LLM is explains three things that surprise many people.

It does not look things up. Unless the app gives it a search tool, the model answers from the patterns set during training. When those patterns are thin, it can produce something that sounds right and is not. Why Models Make Things Up explains why.

Its knowledge stops at a date. A model learns only from text collected before its training, so it knows nothing of what happened after. Model makers publish this date as the knowledge cutoff; Anthropic, for example, lists one for every current model on its models overview.

Talking to it does not change it. When you chat, the model's knobs stay exactly as training left them. The app sends the whole conversation back to the model with every new message, which is how it seems to remember what you said earlier. Anthropic's guide for developers says it directly: its API, the connection programs use to talk to the model, "is stateless, which means that you always send the full conversational history" (Anthropic's guide to its Messages API). Some companies may use your conversations to train later models, depending on your settings, as Anthropic explains for its Claude apps. The model you are talking to stays the same. Training vs Inference: Why Building Costs Millions and Asking Costs Cents shows why using a model is so different from building one.

Quick reference

Term	What it means
LLM	A very large model trained to predict the next token of text
Token	A piece of text the model reads and writes: a common word, or part of a rarer one
Parameters	The billions of numbers (knobs) that training sets; they hold what the model learned
Transformer	The layered design inside most of today's LLMs
Attention	How each token pulls in information from the tokens before it
Pretraining	The first round of training: next token prediction on a huge pile of text
Instruction tuning	Training on examples of requests and good answers
RLHF	Training toward the answers people prefer
Reasoning model	A model also trained to work through problems step by step before it answers
Knowledge cutoff	The date after which the model's training text has nothing
Inference	Using the trained model to answer; its knobs do not change

Frequently Asked Questions

Q: Does the model actually understand language? It depends on what you mean by understand, and researchers still argue about it. What is clear is that the model builds an internal picture of text that captures grammar, meaning and many facts well enough to write, summarize and solve many problems. It also makes mistakes no person would make, a good reminder that it learned from text, not from living in the world.

Q: How can it do arithmetic if it only predicts the next word? Its training text is full of worked sums, so it picks up the patterns of arithmetic. It does well on common cases and can slip on long or unusual numbers, because there is no calculator inside it. Models trained to reason step by step do much better on math, and developers can give a model a tool that runs a small program to do a sum exactly (Anthropic's code execution tool).

Q: Why do bigger models seem to suddenly gain new abilities? Researchers have seen some abilities jump from near zero to good once models pass a certain size, and called them emergent (Emergent Abilities of Large Language Models). Others showed that many of these jumps turn into smooth, steady improvement when the scoring changes from all or nothing to partial credit (Are Emergent Abilities of Large Language Models a Mirage?). The debate is still open: some abilities look sudden mainly because of how they are measured.

Q: Can an LLM memorize its training text? Yes, sometimes word for word. One study found that memorization grows with the size of the model, with how many times a passage was repeated in the training text, and with how much of the passage you give it as a prompt (Quantifying Memorization Across Neural Language Models). That raises privacy and copyright questions, and it is one reason training text is usually cleaned of repeated copies.

Test yourself

A colleague asks a chatbot for your company's refund policy. It answers at once with a clear, confident policy: thirty days, receipt required, store credit only. Your company has no such policy. Using what you now know about LLMs, explain what happened and how to get a trustworthy answer.

Answer: The model never saw your company's policy. Private documents are not part of the text it learned from, and it does not look anything up unless the app gives it a search tool. So it did what it always does: it predicted, token by token, what a refund policy usually looks like, and its training text is full of real policies with thirty days and receipts. Its later training taught it to answer helpfully and fluently, which is why the made up policy sounds so sure of itself. Confident wording tells you nothing about whether an answer is true. To get a trustworthy answer, paste the real policy into the conversation, or use a tool that searches your company's own documents and hands the right passage to the model. Then the model can base its answer on the text in front of it, and you can check the answer against that text.

Next: Tokens: The Currency of AI. It looks closely at the pieces a model reads and writes, and why they decide what you pay and how much fits in a conversation.

If you want this kind of breakdown every week, the real internals behind the tools you use, subscribe to Internals Decoded at internalsdecoded.com.

Sources

• Attention Is All You Need (Vaswani et al. 2017)
• The Llama 3 Herd of Models (Meta 2024)
• Llama 3.1 model card (Meta)
• Language Modeling Is Compression (Delétang et al. 2023)
• Scaling Laws for Neural Language Models (Kaplan et al. 2020)
• Training Compute-Optimal Large Language Models (Hoffmann et al. 2022)
• Language Models are Few-Shot Learners (Brown et al. 2020)
• In-context Learning and Induction Heads (Olsson et al. 2022)
• Glossary (Anthropic)
• LIMA: Less Is More for Alignment (Zhou et al. 2023)
• Training language models to follow instructions with human feedback (Ouyang et al. 2022)
• Constitutional AI: Harmlessness from AI Feedback (Bai et al. 2022)
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (DeepSeek-AI 2025)
• Models overview (Anthropic, read October 2026)
• Using the Messages API (Anthropic)
• Updates to Consumer Terms and Privacy Policy (Anthropic)
• Web search tool (Anthropic)
• Code execution tool (Anthropic)
• Emergent Abilities of Large Language Models (Wei et al. 2022)
• Are Emergent Abilities of Large Language Models a Mirage? (Schaeffer et al. 2023)
• Quantifying Memorization Across Neural Language Models (Carlini et al. 2022)