Few-Shot: Examples Beat Instructions

Last time we built the skeleton of a prompt that behaves: role, task, constraints, and format. That skeleton gives you structure. Now we add the muscle. A few well chosen examples will steer the model more reliably than any amount of prose instruction. Here is why that is true, and how to pick the right examples for a customer support reply assistant.

The surprising part is that the model does not need correct labels to benefit from examples. It often performs nearly as well when you attach the wrong labels to your demonstrations. The examples are not teaching the model a semantic mapping. They are showing it the shape of the task: the label space, the input distribution, and the output format. Once you understand that, you stop writing essays and start curating tiny datasets.

Why do examples beat instructions in practice?

Think of a prompt as a tiny training set that the model processes in one forward pass. Instructions are like a syllabus. Examples are like worked problems. A syllabus tells you what to study. Worked problems show you exactly what a correct answer looks like. Most students, and most language models, learn more from the worked problems.

When you write “Extract the customer’s sentiment and reply with a short apology if it is negative,” the model has to interpret a dozen ambiguous decisions. Should the apology be formal or casual? How short is short? What if the sentiment is mixed? You can answer all of those with more instruction text. Or you can show three example input-output pairs that implicitly encode every one of those choices. The model’s pattern completion machinery will latch onto the concrete patterns and reproduce them for the new input. source

This is not just a UX observation. It is a consequence of how transformers are pretrained. During training, the model sees countless sequences where several similar examples appear in a burst. A block of product reviews each followed by a star rating. A series of code snippets each followed by their output. The model learns that when a pattern repeats consistently in the context, continuing that pattern is the safest next-token prediction. source

Instructions, by contrast, are statistically weaker. In pretraining data, instructions are often followed by text that does not perfectly obey them. The model learns that instructions are noisy hints, not hard constraints. So when you give it a few-shot prompt, the examples speak louder than the instructions because they match the statistical regime the model was optimized for. source

Why examples beat instructions

Instructions only

• Tone and formality are unspecified
• Output length is undefined
• Handling of edge cases is unclear
• Model must guess from vague directions

With few shot examples

• Tone is set by example replies
• Length is constrained to 2 to 3 sentences
• Examples cover positive, negative, neutral
• Model copies the demonstrated pattern

How does the model actually use those examples?

The model does not read examples the way a human would. It runs a set of learned circuits that treat the prompt as a small dataset and compute an answer for the final query. Two mechanisms explain most of the behavior: induction heads and implicit gradient descent.

Induction heads are pairs of attention heads that implement a simple algorithm. If the model has seen the pattern [A][B] earlier in the context, and later sees [A] again, it predicts [B]. One head copies information about the previous token into the current position. The next head attends back to the earlier occurrence of [A] and pulls forward [B]. source

In a customer support prompt, when you show “Input: ... Output: POS” and then “Input: ... Output: NEG”, the induction heads learn that after “Output:” comes one of those labels. When the final “Output:” appears, they attend back to the earlier label positions and pull the most likely token into the prediction. The more consistent your format, the easier this circuit works. source

Induction heads explain simple pattern copying. But models also handle more complex mappings. A second line of work shows that transformer layers can implement gradient descent inside the forward pass. Given a sequence of input-output pairs, the attention and feedforward layers can compute an implicit loss and update a set of fast weights, all without changing the model’s permanent parameters. source

In this view, your few-shot examples are literally the training data for an inner optimizer. The model fits a simple function (often a linear mapping over learned features) to those examples and then applies it to the query. The instructions might set the prior or the learning rate, but the gradient signal comes from the examples. That is why even random labels often work: the model is not learning the semantic mapping. It is learning the output format and the label vocabulary from the examples, then using its own pretrained knowledge to fill in the correct answer. source

What makes a good example for a customer support assistant?

Good examples are not necessarily the most representative ones. They are the ones that pin down the ambiguous dimensions of your task. For our support reply assistant, the instruction might say “write a polite, helpful reply.” But what does polite mean? How long should the reply be? Should it include a greeting? A signature?

Three examples answer all of that at once.

Example 1 (positive sentiment): Input: “My order arrived a day early, thank you!” Output: “That is great to hear! We are glad your order arrived ahead of schedule. Let us know if you need anything else.”

Example 2 (negative sentiment): Input: “The package was damaged when it arrived.” Output: “We are sorry about the damaged package. Please send us a photo and we will ship a replacement right away.”

Example 3 (neutral or mixed): Input: “The product works fine but the instructions were confusing.” Output: “Thank you for the feedback. We are working on clearer instructions. If you have any questions about setup, just reply here.”

These examples define the tone (friendly but not overly casual), the length (two to three sentences), the handling of different sentiment cases, and the presence of a closing offer. The model can now generalize to new inputs because it has seen the pattern.

The research on exemplar selection confirms this. The most effective examples are those that cover the label space, expose the model to the input distribution, and establish a consistent format. source You do not need a statistically balanced sample. You need a handful of cases that leave no ambiguity about what the output should look like. When optimizing prompts, picking better examples usually yields larger gains than tweaking instruction wording. source

When do instructions still matter?

Instructions are not useless. They set the high-level objective and can encode safety constraints that examples alone might miss. For instance, you might add an instruction: “Never promise a refund unless the customer explicitly asks for one.” That rule is hard to encode purely through examples unless you include an example that demonstrates it.

But the power dynamic is clear. Instructions are the scaffold. Examples are the load bearing walls. If you have a tight output schema or tricky edge cases, invest your time in curating examples, not in polishing prose.

The best results come from combining both. Use a short instruction block to define the goal and any hard rules. Then provide three to five examples that show the model exactly what success looks like. This matches the API (application programming interface) design patterns recommended by providers: system message for durable instructions, user messages for examples and queries. source

Quick Reference

Property	Value
Minimum effective examples	3 for format-heavy tasks
Label correctness requirement	Not critical; random labels often work
Primary signal from examples	Label space, input distribution, output format
Key internal mechanism	Induction heads and implicit gradient descent
Best placement in prompt	After instructions, before query
Example selection strategy	Cover all output classes and edge cases

Frequently Asked Questions

Q: If random labels work, why bother with correct labels at all? Random labels work for tasks where the model already knows the answer from pretraining and only needs format guidance. For novel or domain specific tasks where the mapping is not in the training data, correct labels become important because the model must learn the mapping in context. When in doubt, use correct labels.

Q: How many examples should I include? Start with three. More examples improve performance up to a point, but each additional example consumes context window and adds latency. For most format driven tasks, three to five examples are enough to define the pattern. For complex reasoning tasks, you might need more.

Q: Should examples come before or after the instruction? Place instructions first, then examples, then the query. This gives the model the high level goal early and then the concrete pattern to follow. If the prompt is very long, you can repeat a short instruction at the end to combat recency effects.

Q: Can I use the same examples for different tasks? No. Examples are task specific. They define the output format and label space for a particular task. If you change the task, you need new examples that match the new output schema.

Q: How do I know if my examples are good enough? Run the prompt on a small test set of 10 to 20 inputs. If the outputs consistently follow the desired format and handle edge cases correctly, your examples are working. If the model drifts, add an example that covers the failing case.

Test yourself

Your customer support assistant is supposed to reply in exactly three sentences, no more. You have given it two examples that are three sentences long, but the model sometimes produces four sentence replies. What is the most likely fix?

Answer: Add an example where the input is a complex complaint that would naturally invite a longer reply, but the output is still exactly three sentences. The model is not violating your format rule out of malice. It is falling back on its pretraining distribution, where longer complaints often get longer replies. By showing an example that explicitly constrains a long complaint to three sentences, you teach the model that the three sentence rule is absolute, not a suggestion. If that does not work, you can also add a short instruction like “Replies must be exactly three sentences” at the very end of the prompt, but the example is the stronger signal.

Next time we will look at chain of thought: when and how to make the model show its work. The assistant will start reasoning through refund policies step by step, and you will see a different kind of in context learning kick in.

If you want this kind of breakdown every week, how real systems actually work under the hood, subscribe to Internals Decoded at internalsdecoded.com.

Sources

• Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (Min et al., 2022)
• Data Distributional Properties Drive Emergent In-Context Learning in Transformers (Chan et al., 2022)
• In-context Learning and Induction Heads (Elhage et al., 2022)
• Transformers learn in-context by gradient descent (von Oswald et al., 2023)
• What learning algorithm is in-context learning? Investigations with linear models (Akyürek et al., 2023)
• Teach Better or Show Smarter? On Instructions and Exemplars in Prompt Engineering (Ye et al., 2023)
• OpenAI Prompt Engineering Guide