Learn · Intermediate

Model merging: combining two fine-tuned models by averaging their weights

Model merging combines two or more fine-tuned models into a single model by doing arithmetic on their weight values - averaging them, adding their differences, or interpolating along the path between them. It requires no training data, no gradient steps and no GPU cluster: a merge is a few minutes of elementwise arithmetic that produces one model, the same size as its parents, that can do several of their jobs. It works because the models being merged are nearly always fine-tunes of the same base, so they live close enough together in weight space that the space between them is not garbage.

That last sentence is the whole idea, and it was a genuine surprise when it turned out to be true.

The setup: what a fine-tune actually is

Start with a base model - call its weights W. Fine-tune it on medical text and you get W_med. Fine-tune the same base separately on legal documents and you get W_law. Both are large piles of numbers with the same shape, both are small perturbations of W, and neither knows the other exists. (If you need the background, see fine-tuning and LoRA.)

The obvious way to get one model that does both is to fine-tune on both datasets at once. That requires the data, the compute and, often, dealing with catastrophic forgetting as one task overwrites the other.

Merging asks a stranger question: what if you just averaged the two piles of numbers?

Model soups: averaging works

Wortsman and colleagues tested exactly this in 2022 with what they called model soups. Take many fine-tunes of one base model, made with different hyperparameters, and average all their weights. The averaged model was more accurate than any individual ingredient, and unlike an ensemble it costs nothing extra to run - you end up with one model, not five.

The intuition is that different fine-tuning runs land in slightly different spots at the bottom of the same broad basin in the loss landscape. Each has its own idiosyncratic errors. Averaging cancels the idiosyncrasies and keeps what they agree on - the same reason an average of several people's estimates often beats any single estimate.

Task vectors: skills as arithmetic

The next step is the one that makes merging feel almost illicit. Ilharco and colleagues defined a task vector as the difference between a fine-tuned model and its base: T_med = W_med - W. That subtraction isolates what learning the medical task changed.

Then they showed those vectors behave like vectors. Add two and you get a model with both skills: W + T_med + T_law. Negate one and you get a model that is measurably worse at that task - a crude form of machine unlearning. You can even complete analogies with them.

Think of the base model as a fully furnished house, and each task vector as a list of changes one tenant made. Apply two lists of changes and you get a house with both sets of modifications. Apply one in reverse and you undo that tenant's work. It works as long as the tenants mostly touched different rooms.

Where it breaks, and the fixes

That caveat is doing real work. When two task vectors want to move the same weight in opposite directions, naive addition averages the conflict into mush. This is called interference, and it is the central problem in merging.

TIES-Merging attacks it in three steps: trim each task vector to its largest-magnitude entries, since most of the change is concentrated in a small fraction of weights; elect a sign for each weight by seeing which direction has more total magnitude across the models; and merge only the entries agreeing with that elected sign. Contested weights get resolved rather than averaged into nothing.

DARE went further with a finding that sounds like a typo: you can randomly zero out 90% or more of a task vector's entries, rescale the survivors, and lose almost nothing. Fine-tuning updates are extremely redundant. Because pruned vectors collide less, DARE is often used as a preprocessing step before any other merge method.

SLERP - spherical linear interpolation - handles the two-model case differently, interpolating along the arc between two weight vectors rather than the straight line between them, which preserves the magnitude of the weights instead of shrinking it. This is not a toy: Mistral built its Shieldstral safety classifier by SLERP-merging three LoRA-tuned checkpoints with a base instruct model, and says the merge recovers capabilities the individual fine-tunes lost.

The deeper theoretical question is why fine-tunes are compatible at all, since two networks trained from scratch on the same data usually cannot be averaged - their hidden units end up in a different arbitrary order. Git Re-Basin showed you can often find a permutation of one network's units that makes it mergeable with another, suggesting many apparently distinct solutions are the same solution wearing a different arrangement.

Why it matters

Merging changed the economics of the open-weight model ecosystem. A hobbyist with no training budget can combine a model good at reasoning with one good at a specific language and ship the result the same afternoon. A large fraction of the community models on Hugging Face are merges, and it is now a standard step in production pipelines at labs that could afford to train instead.

It also reframes what a fine-tuned model is. Not a monolithic artifact, but a base plus a diff - and diffs, it turns out, can be stored, shared, pruned, negated and added together.

The honest limits

Merging only works between models sharing a common ancestor and identical architecture. It cannot add knowledge neither parent had. Results are unpredictable enough that practitioners merge, evaluate, adjust weights, and repeat - it remains partly empirical craft. And a merged model can inherit its parents' failure modes just as easily as their skills, which matters when one parent was safety-tuned and the other was not.

Key papers
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (Wortsman et al., 2022)
Editing Models with Task Arithmetic (Ilharco et al., 2022)
Git Re-Basin: Merging Models modulo Permutation Symmetries (Ainsworth, Hayase & Srinivasa, 2022)
TIES-Merging: Resolving Interference When Merging Models (Yadav et al., 2023)
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch (DARE, Yu et al., 2023)

Key questions

What is model merging?

Model merging combines two or more neural networks into one by doing arithmetic directly on their weight values - typically averaging them or adding up their differences from a shared starting point. No training data and no gradient steps are involved, so a merge takes minutes on a laptop rather than GPU-hours.

Why does averaging two models' weights produce something that works?

Because the models being merged are almost always fine-tunes of the same base model, so they sit in the same region of weight space and their differences are small. Averaging two unrelated networks trained from scratch produces noise, which is why merging is a fine-tuning technique rather than a general one.

How is merging different from distillation or an ensemble?

An ensemble runs every model at inference and combines the outputs, so it costs several times as much to run. Distillation trains a new student model on a teacher's behaviour, which costs training compute. Merging costs neither: it produces one model, the same size as the originals, at the price of some elementwise arithmetic.

Cite this

APA

Ground Truth. (2026, August 4). Model merging: combining two fine-tuned models by averaging their weights. Ground Truth. https://groundtruth.day/learn/model-merging.html

BibTeX

@misc{groundtruth:model-merging,
  title  = {Model merging: combining two fine-tuned models by averaging their weights},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {aug},
  url    = {https://groundtruth.day/learn/model-merging.html}
}

Topics: fundamentals · model-merging · fine-tuning · efficiency · open-weight-models · training