Landmark 6 stops to get here · leads to 6

Fine-Tuning

The process of further training a pre-trained model on a specific dataset to adapt it for a particular task or domain.

Your route here

6 stops · basics first
  1. Dataset ✓ understood

    A collection of data examples used for training, validating, or testing machine learning models.

  2. Training ✓ understood

    The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.

  3. Machine Learning ✓ understood

    Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

  4. Unsupervised Learning ✓ understood

    Learning from unlabeled data to discover hidden patterns, structures, or relationships without explicit target outputs.

  5. Self-Supervised Learning ✓ understood

    Training on unlabeled data by making the data supply its own answers: hide or alter part of an input and have the model predict or match it.

  6. Pre-training ✓ understood

    Training a model on a large dataset (often self-supervised) before fine-tuning on specific tasks, enabling transfer learning.

  7. Fine-Tuning · you are here ✓ understood

Picture it

  1. 01 Pre-trained base model General knowledge from broad data
  2. 02 Task-specific dataset Smaller, focused examples
  3. 03 Continue training Usually a lower learning rate
  4. 04 Evaluate on validation Check gains, watch for forgetting
  5. 05 Specialized model
Notice that training starts from existing weights, not scratch, so far less data and compute are needed.

Fine-tuning is taking a model that has already been trained and training it a little more on a smaller, focused dataset, so it gets good at a particular task, domain or style. The expensive general learning happened during pre-training; fine-tuning adjusts the result rather than starting over.

The payoff is efficiency. Because the model already knows a lot, it needs far fewer examples to learn something new. In the 2018 ULMFiT paper, a text classifier fine-tuned from a pre-trained language model on just 100 labelled examples matched one trained from scratch on 100 times as much data.

How it works

The diagram shows the loop. Start from the pre-trained weights, prepare a task-specific dataset, and continue training with ordinary gradient descent. Two details matter. The learning rate is usually set lower than in pre-training, so the updates refine the model instead of overwriting it. And you evaluate on a held-out set throughout, watching both for gains on the new task and for losses on what the model could already do.

The main kinds

  • Full fine-tuning updates every weight. It is the most flexible, but you end up storing a complete copy of the model for each task.
  • Parameter-efficient fine-tuning freezes the original weights and trains a small add-on. A widely used method, LoRA (2021), learns small low-rank matrices beside each weight layer. On GPT-3, it cut the trainable parameters by a factor of 10,000 and GPU memory use by a factor of three, while matching full fine-tuning on quality.
  • Instruction tuning fine-tunes on examples of requests paired with good responses, which turns a base model that only continues text into one that follows instructions.
  • Preference tuning, such as RLHF, goes a step further and trains on human judgments of which answer is better. OpenAI’s InstructGPT combined both steps, and people preferred answers from its 1.3-billion-parameter model to those from the original 175-billion-parameter GPT-3.

The catch

Training hard on a narrow dataset can make a model lose abilities it had before, a problem called catastrophic forgetting. Lower learning rates, fewer training steps and mixing some general data back in all reduce the risk.

Fine-tuning is also not always the right tool. It is good at teaching a format, a style or a skill. For supplying facts that change, retrieving them or putting them in the prompt is often simpler, and easier to keep current than retraining.

Where it sits

Explore nearby

In the research

All papers →

7 papers that build on Fine-Tuning ; showing 5, canon first.

Canon · 2018 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding It established “pre-train once, fine-tune everywhere”, and still powers much of search and classification. Canon · 2021 LoRA: Low-Rank Adaptation of Large Language Models It made adapting big models cheap enough for everyone, and is why fine-tunes are shared as small adapter files. Canon · 2022 Training Language Models to Follow Instructions with Human Feedback This recipe, RLHF, is what turned raw language models into assistants like ChatGPT. Frontier · Sep 2025 · 114 citations Fast-dLLM v2: Efficient Block-Diffusion LLM A cheap path to faster inference from models you already have. Frontier · Aug 2025 · 112 citations On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification A tiny, theory-backed change to the most common training step in the stack.

Sources

  1. Howard and Ruder, "Universal Language Model Fine-tuning for Text Classification" . 2018 (ULMFiT)
  2. Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models" . 2021
  3. Ouyang et al., "Training language models to follow instructions with human feedback" . 2022 (InstructGPT)