Adapting Models

Full fine-tuning: data, compute, risk

Every weight moves, for better and worse

Scroll

Every weight moves, for better and worse

Full fine-tuning keeps training a pretrained model on your own examples and lets every weight change. It is the most powerful way to adapt a model and the most expensive, and it is rarely the first thing to try. The page on fine-tuning versus RAG covers when it is worth it.

The cost is mostly memory. Training needs the weights, a gradient for every weight, and the optimizer's running statistics, which in a standard mixed-precision setup comes to about 16 bytes per parameter before activations. An 8-billion-parameter model that serves in 16 GB needs around 128 GB to train, spread across several accelerators. The quality of the data matters more than its volume: a few thousand clean, consistent examples of the behavior you want usually beat a large, noisy set. And every fine-tuned variant is a full copy of the model, which you have to store, serve, and redo when the base model is upgraded.

The main risk is forgetting. Because every weight moves, the model can get better at your task while losing general abilities it had, including some of its safety behavior; even fine-tuning on harmless data can weaken refusals. Mixing general examples into the training set, using a low learning rate and training for only a few passes all reduce the damage. None of them removes it, so evaluate on two sets after every run: one for the task, and one for the general and safety behavior you need to keep. LoRA avoids much of this by freezing the original weights and training a small add-on instead.

Training
WHAT FULL FINE-TUNING COSTS memory for an 8B model serving: 16 GB training: ~128 GB per parameter: 2 bytes weights + 2 gradients + 12 optimizer state, before activations skill · illustrative task skill general skill, with general data mixed in general skill training steps → Every weight moves, so the model can gain the task and lose general skills it had. Evaluate both after every run.
Training memory against serving memory, and task skill rising while general skill falls.