Few-Shot: Examples Beat Instructions

In Why Your Prompts Fail (and the Anatomy That Works) we built the skeleton of a prompt that behaves: role, task, constraints and format. That skeleton gives you structure. Now we add the muscle. A few well chosen examples often steer the model more reliably than more paragraphs of instruction, especially on tone, length and edge cases. Here is why, and how to pick the right examples for a customer support reply assistant.

The model does not always need correct labels to benefit from examples. A 2022 study found that on a range of classification and multiple choice tasks, randomly swapping the labels in the examples barely hurt performance, across 12 models including GPT-3 (Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?, Min et al.). What the examples supplied was the shape of the task: the set of possible labels, the kind of input to expect and the format of the output. Later work found that larger models do read the labels, and can even follow deliberately flipped ones (Larger language models do in-context learning differently, Wei et al. 2023), so you should still use correct labels. The lasting lesson is about shape. Once you see examples as a definition of the task's shape, you stop writing essays and start curating tiny datasets.

Why do examples beat instructions in practice?

Think of a prompt as a tiny training set that the model reads in one go. Instructions are like a syllabus. Examples are like worked problems. A syllabus tells you what to study. Worked problems show you exactly what a correct answer looks like. Most students, and most language models, learn a lot from the worked problems.

When you write "Extract the customer's sentiment and reply with a short apology if it is negative," the model has to settle a dozen ambiguous decisions. Should the apology be formal or casual? How short is short? What if the sentiment is mixed? You can answer all of those with more instruction text. Or you can show three example input and output pairs that encode every one of those choices at once. The model picks up the concrete patterns and reproduces them for the new input.

One line of research traces this to the training data. Chan et al. found that the ability to learn from examples in the prompt emerges when the training data is "bursty", with items appearing in clusters rather than spread evenly, a property natural language has (Data Distributional Properties Drive Emergent In-Context Learning in Transformers). Think of a block of product reviews, each followed by a star rating, or a run of code snippets, each followed by its output. A model trained on text like this learns that when a pattern repeats in the context, continuing it is a good bet.

That explains why examples were the stronger signal for base models, which are trained only to continue text and learned from writing where instructions are often not obeyed. Today's chat models are different. They are fine-tuned to follow instructions, often with human feedback, as you can read in Fine-Tuning, Explained: Teaching an Old Model New Tricks. In OpenAI's InstructGPT work, people preferred the outputs of a 1.3 billion parameter model tuned this way to those of the 175 billion parameter GPT-3, a model more than 100 times larger (Training language models to follow instructions with human feedback, Ouyang et al.). So instruction-tuned models follow clear instructions well, and OpenAI's advice for its reasoning models is to try a prompt without examples first and add a few only if the output needs them (OpenAI's reasoning best practices). Examples still earn their place by settling the details that instructions leave vague: tone, length, and what to do with the odd case.

Why examples beat instructions

Instructions only

• Tone and formality are unspecified
• Output length is undefined
• Handling of edge cases is unclear
• Model must guess from vague directions

With few-shot examples

• Tone is set by example replies
• Length matches the example replies
• Examples cover positive, negative, neutral
• Model copies the demonstrated pattern

How does the model actually use those examples?

Nobody fully knows how a model uses the examples in a prompt, but two research ideas give useful intuitions: induction heads, and in-context learning as a kind of gradient descent. You can skip this section and still write good few-shot prompts.

Induction heads are pairs of attention heads that implement a simple algorithm. If the model has seen the pattern [A][B] earlier in the context, and later sees [A] again, it predicts [B]. One head copies information about the previous token into the current position. The next head attends back to the earlier occurrence of [A] and pulls forward [B]. The researchers who described them present evidence, strong in small models and indirect in large ones, that these heads drive much of in-context learning (In-context Learning and Induction Heads, Olsson et al. 2022).

Diagram loading

When a prompt shows "Input: ... Output: POS" and then "Input: ... Output: NEG", induction heads can match the pattern that a label follows "Output:". When the final "Output:" appears, they attend back to the earlier label positions and pull a likely label into the prediction. No weights change; the model is matching patterns in its context. For free text replies like our support assistant's, the same copying works on style and structure rather than single labels: the greeting, the length, the closing offer. The more consistent your format, the easier it is for the model to pick up.

Induction heads explain simple copying, but models also handle more complex mappings. A second line of work shows that, in simplified settings, attention can mimic a step of gradient descent on the examples: transformers trained on small regression problems end up doing something very close to fitting a little model to the examples in the prompt, without changing their weights (Transformers learn in-context by gradient descent, von Oswald et al.; What learning algorithm is in-context learning? Investigations with linear models, Akyürek et al.). Dai et al. argue that real GPT models behave in a similar way (Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers). Whether large models trained on ordinary text actually do this is still debated: a study that tested the idea on LLaMA 7B found that in-context learning and gradient descent behave differently, and called the equivalence "an open hypothesis" (Do pretrained Transformers Learn In-Context by Gradient Descent?, Shen et al.).

Either way, the practical reading is the same as Min et al.'s. Examples tell the model the format, the label vocabulary and the kind of input to expect, and the model's existing knowledge fills in the rest. That is why random labels often worked in that 2022 study, and why larger models, which also read the labels, do better with correct ones.

What makes a good example for a customer support assistant?

Good examples are not necessarily the most representative ones. They are the ones that pin down the ambiguous parts of your task. For our support reply assistant, the instruction might say "write a polite, helpful reply." But what does polite mean? How long should the reply be? Should it include a greeting? A signature?

Three examples answer all of that at once.

Example 1 (positive sentiment): Input: "My order arrived a day early, thank you!" Output: "That is great to hear! We are glad your order arrived ahead of schedule. Let us know if you need anything else."

Example 2 (negative sentiment): Input: "The package was damaged when it arrived." Output: "We are sorry about the damaged package. Please send us a photo and we will ship a replacement right away."

Example 3 (neutral or mixed): Input: "The product works fine but the instructions were confusing." Output: "Thank you for the feedback. We are working on clearer instructions. If you have any questions about setup, just reply here."

These examples define the tone (friendly but not overly casual), the length (two to three sentences), the handling of different sentiment cases, and the presence of a closing offer. The model can now generalize to new inputs because it has seen the pattern.

This matches what the research found matters most: examples that show the range of possible outputs, the kind of input to expect and a consistent format (Min et al.). You do not need a statistically balanced sample. You need a handful of cases that leave no doubt about what the output should look like. And the choice of examples matters a lot. In automatic prompt optimization, where software searches for better prompts, choosing better examples often beat optimizing the instruction, and doing both was best (Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimization, Wan et al. 2024).

When do instructions still matter?

Instructions are not useless. They set the high level goal and can carry rules that examples alone might miss. For instance, you might add the instruction "Never promise a refund unless the customer explicitly asks for one." That rule is hard to show through examples unless you include one that demonstrates it.

For format and tone, examples usually carry more weight than prose. Instructions set the goal and the hard rules; examples show what success looks like. If you have a tight output format or tricky edge cases, spend your time curating examples rather than polishing prose.

The best results come from combining both: a short instruction block that defines the goal and any hard rules, then a few examples that show exactly what success looks like. The providers suggest the same layout. OpenAI's prompting guide puts identity, instructions, examples and context, in that order, in the developer message, with the real input in the user message (OpenAI's prompt engineering guide). Anthropic recommends wrapping each example in <example> tags, with several inside <examples>, so the model can tell them apart from instructions, and suggests three to five of them (Anthropic's prompting best practices).

Quick Reference

Property	Value
Starting point	Three to five examples (Anthropic's suggestion); for reasoning models, try none first (OpenAI)
Labels	Use correct labels; label errors matter more for larger models and new tasks
Primary signal from examples	Label space, input distribution, output format
Proposed mechanisms	Induction heads (copying patterns); in-context learning as gradient descent (shown in simplified settings, debated for real models)
Best placement in prompt	After the instructions, in a clearly marked examples section of the system or developer message; the real input goes in the user message
Example selection strategy	Cover all output classes and the edge cases that cause trouble

Frequently Asked Questions

Q: If random labels worked, why bother with correct labels at all? Random labels only worked, in that 2022 study, on classification tasks where the model already knew the answer and mainly needed the format. Larger models read the labels and can be misled by wrong ones (Wei et al. 2023), and for new or specialized tasks the model has to learn the mapping from your examples. Use correct labels.

Q: How many examples should I include? Start with three. More examples help up to a point, but each one uses context window space and adds latency. For most format driven tasks, three to five are enough to define the pattern. For reasoning models, OpenAI suggests trying a prompt without examples first.

Q: Should examples come before or after the instruction? Place the instructions first, then the examples, then the real input. This gives the model the high level goal early and then the concrete pattern to follow. If the prompt is very long, you can repeat a short instruction at the end, where models read it most reliably (see part 1).

Q: Can I use the same examples for different tasks? No. Examples are task specific. They define the output format and label space for a particular task. If you change the task, you need new examples that match the new output.

Q: How do I know if my examples are good enough? Run the prompt on a small test set of 10 to 20 inputs. If the outputs consistently follow the desired format and handle the edge cases correctly, your examples are working. If the model drifts, add an example that covers the failing case. Part 6, Evaluating Prompts: Beyond 'Looks Good to Me', shows how to turn this check into a repeatable test.

Test yourself

Your customer support assistant is supposed to reply in exactly three sentences, no more. You have given it two examples that are three sentences long, but the model sometimes produces four sentence replies. What is the most likely fix?

Answer: Add an example where the input is a long, complex complaint that would naturally invite a longer reply, but the output is still exactly three sentences. The model is not breaking your rule on purpose. Your two examples never showed it what to do with a long complaint, so it falls back on a common pattern in its training data, where longer complaints get longer replies. An example that keeps a long complaint to three sentences shows that the limit holds for every input, not just the easy ones. Make sure your examples also cover the other cases that tend to run long, such as mixed sentiment. You can also add a short instruction like "Replies must be exactly three sentences" at the very end of the prompt; instruction-tuned models follow clear rules well, and the example and the rule together usually settle it. Then rerun your test set to confirm.

Next: Chain of Thought: When Thinking Out Loud Helps. It covers when asking a model to reason step by step helps, when it hurts, and how the built-in thinking of reasoning models changed the advice.

If you want this kind of breakdown every week, how real systems actually work under the hood, subscribe to Internals Decoded at internalsdecoded.com.

Sources

• Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (Min et al. 2022)
• Larger language models do in-context learning differently (Wei et al. 2023)
• Data Distributional Properties Drive Emergent In-Context Learning in Transformers (Chan et al. 2022)
• Training language models to follow instructions with human feedback (Ouyang et al. 2022)
• Reasoning best practices (OpenAI, read October 2026)
• In-context Learning and Induction Heads (Olsson et al. 2022)
• Transformers learn in-context by gradient descent (von Oswald et al. 2022)
• What learning algorithm is in-context learning? Investigations with linear models (Akyürek et al. 2022)
• Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizers (Dai et al. 2022)
• Do pretrained Transformers Learn In-Context by Gradient Descent? (Shen et al. 2023)
• Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimization (Wan et al. 2024)
• Prompt engineering (OpenAI, read October 2026)
• Prompting best practices (Anthropic, read October 2026)