Fine-Tuning, Explained: Teaching an Old Model New Tricks

Fine-tuning takes a pretrained model and gives it a second round of training on a small, task specific set of examples. The result is a lasting change in behavior: a general chatbot learns to write in your brand voice, an assistant starts drafting emails the way your team does, a medical support bot adopts the shorthand of clinicians. Unlike prompting, which wraps instructions around the model, fine-tuning changes the model's own weights, so the new behavior becomes its default.

But here is the paradox that catches many teams off guard: for most customization problems, you should not fine-tune at all. If the problem is missing knowledge, retrieval augmented generation (RAG) is usually the better tool. It is cheaper, easy to roll back, and it never touches the model's weights, so it cannot cause the worst side effect of fine-tuning, the loss of skills the model already had. When you truly need to change how the model speaks, not what it knows, fine-tuning is the tool for the job.

In Open vs Closed Models: What 'Open Source AI' Really Means, we compared the kinds of models you can use and the control each one gives you. Today we move from "which model" to "how do I make that model do exactly what I want?"

What actually happens inside the model when you fine-tune?

Imagine a chef who trained in every culinary tradition on earth. She knows how to sauté, braise, ferment and flambé, but she has never worked in your kitchen. Your restaurant serves a handful of signature dishes with very specific techniques. You could give her a checklist every morning (a prompt), but she might still slip back into her classical training. Fine-tuning is like a week long boot camp in your kitchen: you hand her the 40 dishes that define your menu, have her cook them over and over, and by Friday she no longer needs the checklist.

Technically, the model starts with the parameters it learned in pretraining on trillions of tokens. They already hold language, common sense and a lot of world knowledge. You then collect examples of the behavior you want, as pairs of input and ideal output. For a chatbot that drafts emails in your company's tone, that might be a couple of hundred pairs of rough notes and the polished emails written from them. During fine-tuning, the model reads each input and tries to predict the output token by token. The loss measures how far off it was, and backpropagation nudges the weights so its answers line up with your examples, the same loop you met in Getting It Wrong on Purpose and Practice Makes It Smaller.

Because the starting point is already a fluent model, the changes are small and the run is short. Full fine-tuning usually takes smaller steps than pretraining did (a smaller learning rate), so it adjusts the model without wrecking what it knows. After only a few hundred examples, the model can pick up a pattern, a wording style, a preferred format, the jargon of a field, and reproduce it reliably without being told.

Which fine-tuning methods exist, and when do you use them?

"Fine-tuning" covers several distinct techniques. The simplest is supervised fine-tuning (SFT). You give the model pairs of inputs and outputs and train it to produce the outputs. This works beautifully when the behavior you want can be shown in examples: "When you see a bullet list of facts, turn it into a three sentence summary in the voice of a friendly physician."

When the goal is to follow any instruction, not just one template, you use instruction tuning. It is SFT on a large mix of instructions and good responses, and it is the step that teaches a base model that "Summarize the following article:" is a request, not just text to continue. It can take surprisingly little: the LIMA study turned a 65 billion parameter base model into a capable assistant with just 1,000 carefully chosen examples (LIMA: Less Is More for Alignment), and OpenAI's InstructGPT work showed the full recipe of demonstrations followed by human feedback (Training language models to follow instructions with human feedback).

Full fine-tuning updates every weight. That gives the strongest adaptation but takes a lot of memory: a 7 billion parameter model in 16 bit precision needs 14 GB just to hold its weights, and training with the usual Adam optimizer needs room for gradients and the optimizer's own numbers too. The ZeRO paper counts about 16 bytes per parameter for this common setup (ZeRO: Memory Optimizations Toward Training Trillion Parameter Models), which comes to more than 100 GB for a 7 billion parameter model before counting anything else.

Parameter efficient fine-tuning avoids most of that. LoRA (low rank adaptation), the most popular method, freezes the base model and adds small trainable matrices that learn the change needed for the new task. Afterwards you can merge those small matrices into the base weights, so the tuned model runs exactly as fast as the original. LoRA usually trains well under 1% of the parameters. For GPT-3, its authors reported 10,000 times fewer trainable parameters and a third of the GPU memory compared with full fine-tuning, with quality on par or better on the tasks they tested (LoRA: Low-Rank Adaptation of Large Language Models). You still have to load the whole base model, though. Loading it in 4 bit form, the QLoRA approach, cuts that further: its authors fine-tuned a 65 billion parameter model on a single 48 GB GPU (QLoRA: Efficient Finetuning of Quantized LLMs). Hugging Face's PEFT library is a common way to do both.

A final category, preference tuning, trains for what people prefer rather than for exact output text. The best known pipeline is RLHF (reinforcement learning from human feedback): collect judgments of which answer is better, train a reward model on them, then use reinforcement learning to push the language model toward the preferred kind of answer. A simpler alternative, DPO (Direct Preference Optimization), learns from the same preference pairs with a plain classification style loss and no separate reward model; its authors found it matched or beat the reinforcement learning approach while being much simpler to run (Direct Preference Optimization: Your Language Model is Secretly a Reward Model). You use preference tuning when the problem is not "what answer" but "what makes an answer good": safety, politeness, refusing harmful requests.

When does fine-tuning actually help, and when does RAG beat it?

First, what RAG is, in three sentences. Retrieval augmented generation means searching your own documents for the passages that match a question, then putting the best matches into the prompt next to it. The model answers from the text in front of it, so you can update the documents at any time without retraining. Fine-Tuning vs RAG vs Prompting: Which Lever When compares all three options in more depth.

The choice between fine-tuning and RAG comes down to one question: are you trying to change what the model knows, or how it behaves?

If you need the model to use up to date facts, today's prices, this week's schedule, the contents of your private knowledge base, RAG is almost always the right answer. You leave the model untouched and put relevant documents into the prompt when a question comes in. You can update the documents without retraining, the model can point to its source, and there is no risk of degrading its general skills. A recipe chatbot is a perfect candidate: the model's language skills stay general, while each request pulls a fresh list of ingredients and steps from your database.

Fine-tuning shines when you need to change the model's style or format. Suppose your recipe bot must always suggest vegan substitutions and present ingredients in a table, however the user phrases the request. You could try a long system prompt, but the model might occasionally drift from the format, especially in long conversations. Fine-tuning on a few hundred carefully chosen examples of ideal answers can make that behavior the default. Likewise, if your team's chatbot must use a specific house voice, "concise, never start a sentence with 'I think', always use bullet points for steps", a small supervised fine-tuning run can make it consistent.

How many examples is enough? Less than you might think for style, and fine-tuning is the wrong tool for facts.

OpenAI's guide suggests "starting with 50 well-crafted demonstrations and evaluating the results", then adding more if needed (OpenAI's supervised fine-tuning guide). Treat these numbers as starting points, not rules.

A hybrid also works well: keep the knowledge in RAG and fine-tune only the style. A LoRA adapter trained on tone and format sits on top of the base model, while retrieval supplies the facts. RAG handles what is said; fine-tuning handles how it is said.

Why most teams never need fine-tuning

The hype around fine-tuning creates the impression that customizing a model means retraining it. In practice, good prompts and a solid RAG pipeline cover most customization needs. Fine-tuning brings burdens many teams underestimate.

First, data. You need a clean, well chosen set of examples that shows exactly the behavior you want. A few messy examples from a chat channel will not do, and preparing good ones often takes longer than building the retrieval pipeline.

Second, evaluation. A prompt change can be tested and rolled back in minutes. A fine-tuned model is a new artifact that must be checked not just on the new task but on everything the model used to do well. Did it get worse at long conversations? Is it more likely to invent numbers? Finding out needs a broad set of tests.

Third, cost and access. Even with LoRA, fine-tuning takes GPU time, experiments with settings, and the work of keeping and serving several model versions. And access to fine-tuning closed models is shrinking: as of October 2026, OpenAI is winding down its fine-tuning platform and no longer accepts new users (OpenAI's fine-tuning guide), Anthropic does not offer fine-tuning on its API (Anthropic's glossary), and Google offers tuning only for some Gemini models, through its Vertex AI platform (Gemini supervised tuning). Most fine-tuning today happens on open-weight models.

Fourth, fragility. Because fine-tuning changes the model itself, a small mistake in the training examples can introduce subtle problems. A model tuned to be "empathetic" might start apologizing in answer to plain factual questions, a drift you may only notice after many real conversations.

For most teams the pattern is: start with a good prompt, add RAG when you need facts, and keep fine-tuning for the narrow cases where style or format will not budge any other way. A recipe chatbot that fetches ingredients through RAG and keeps a friendly tone through its prompt often satisfies users completely.

What is catastrophic forgetting, and how do you manage it?

Every fine-tuning run risks eroding the model's general abilities. This is catastrophic forgetting: the model becomes so specialized that it loses skills it used to have, sometimes dramatically. You might fine-tune a model on medical notes and find it has become worse at everyday reasoning.

It is a real effect. One study of models from 1 to 7 billion parameters found forgetting in all of them during continued instruction tuning, and found it grew more severe as the models got bigger within that range (An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning).

Several strategies help. Rehearsal mixes a little general data into the fine-tuning examples, reminding the model of what it used to do. Regularization methods add a penalty for changing the weights that mattered most before. LoRA helps in its own way: because it leaves the original weights untouched, you can always remove the adapter and get the original model back, and in practice LoRA tends to forget less than full fine-tuning. It does not eliminate forgetting, and it also learns less: one careful comparison found LoRA "substantially underperforms" full fine-tuning on new programming and math skills while it "better maintains the base model's performance" on other tasks (LoRA Learns Less and Forgets Less).

So a safe workflow is to start with LoRA on a focused style dataset, test the model broadly after training, and only move to full fine-tuning if the task needs it and your tests show no losses elsewhere.

Quick Reference

Frequently Asked Questions

Q: Can I fine-tune a model to "know" new facts, like my company's product catalog? Usually it is the wrong tool. Models learn new facts through fine-tuning slowly and unreliably, and one study found that as they do learn them, they become more likely to invent things (Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?). Store your catalog in a retrieval index and let RAG feed the facts to the model when they are needed.

Q: How many examples do I really need for a tone and format change? Start small. OpenAI's guide suggests beginning with about 50 well made examples, checking the results, and adding more if the behavior is not yet consistent. The examples should vary in phrasing and cover the cases you care about: short requests, long multi step ones, odd inputs. More examples buy consistency, with diminishing returns.

Q: Is fine-tuning expensive compared to prompting? To set up, yes: you pay for GPU time and for preparing data. In daily use it can save money, because a model that already knows your format needs only a short prompt, so each request sends fewer input tokens.

Q: What is the cheapest way to get started? Use LoRA on a 4 bit base model, the QLoRA approach, with an open-weight model and a library such as Hugging Face PEFT. If a 65 billion parameter model fits on a single 48 GB GPU this way, a 7 billion parameter model fits on far smaller ones. Some cloud platforms will also run fine-tuning jobs for you; check which models they support, since offerings for closed models have been shrinking.

Q: Do I need special serving infrastructure for a fine-tuned model? No. Once you merge a LoRA adapter into the base weights, the tuned model has exactly the same shape as the original, so you serve it the same way, for example with vLLM, with no extra delay. If you want several task specific adapters on one base model, you can keep them separate and switch between them, which needs a serving system that supports loading adapters.

Test yourself

Your team runs a medical question answering chatbot for doctors. It uses RAG over a large clinical database and answers questions accurately. Doctors complain that the answers sound "like an encyclopedia", not like a colleague sharing a quick insight. They want medical shorthand, less hedging, and the most actionable information first. Better prompts helped a little, but the tone still feels stiff. Someone suggests fine-tuning. Would you fine-tune, and if so, how would you protect the factual reliability?

Answer: Yes. This is a tone and style problem, which is exactly where fine-tuning pays off, and the facts already come from RAG, which stays as it is. Train a LoRA adapter on a few hundred examples of ideal doctor to doctor answers, built in the same format the bot uses, with the retrieved passages in the input, so the model learns to keep using them. Because LoRA leaves the base weights untouched and tends to forget less, you also keep an easy way back: remove the adapter. To guard against drift, mix in some ordinary factual questions with correct answers, and test the tuned model on a held out set of factual questions before and after training. If accuracy drops or invented details appear, stop earlier, use fewer training steps or a smaller adapter, and test again.

Next: How to Actually Choose a Model. It brings the series together to help you pick the right model for what you actually need.

If you want this kind of breakdown every week, how real AI systems actually work under the hood, subscribe to Internals Decoded at internalsdecoded.com.

Sources

• LIMA: Less Is More for Alignment (Zhou et al. 2023)
• Training language models to follow instructions with human feedback (Ouyang et al. 2022)
• ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (Rajbhandari et al. 2019)
• LoRA: Low-Rank Adaptation of Large Language Models (Hu et al. 2021)
• QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers et al. 2023)
• PEFT documentation (Hugging Face)
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al. 2023)
• Supervised fine-tuning guide and Fine-tuning guide (OpenAI, read October 2026)
• Glossary (Anthropic)
• Supervised fine-tuning for Gemini models (Google Cloud, read October 2026)
• An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning (Luo et al. 2023)
• LoRA Learns Less and Forgets Less (Biderman et al. 2024)
• Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (Gekhman et al. 2024)