# TIES-Merging: Resolving Interference When Merging Models

merging fine-tuned models by trimming small changes, electing a sign, and averaging

Authors: Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, Mohit Bansal
Venue: Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 (2023)

## What this paper shows
TIES-Merging combines several independently fine-tuned models into one without retraining, by discarding small parameter changes and resolving sign conflicts before averaging.

## Claims, with scope
- TIES-Merging produces a single multitask model from several task-specific fine-tuned checkpoints with no additional training and no access to the original training data.
  Scope: Checkpoints must share the same architecture and the same pre-trained initialisation. Evaluated on T5-base, T5-large, ViT and IA3 adapters.
  Evidence: Section 4
- Two sources of interference degrade model merging: redundant parameter changes, and disagreement on a parameter's sign across the models being merged.
  Scope: Demonstrated for parameter-space merging of models fine-tuned from a shared initialisation; not a claim about models trained from scratch.
  Evidence: Section 3
- Trimming low-magnitude parameter changes, electing a single sign per parameter, and averaging only the agreeing values outperforms plain weight averaging and task arithmetic, with the gap widening as more models are merged.
  Scope: Up to 7 models in the reported experiments; same architecture and initialisation throughout.
  Evidence: Table 2, Figure 3

## Common misreadings
- It is not a training method. No gradient steps and no training data are required.
- It does not merge models with different architectures or different pre-trained initialisations.
- The gain is not from trimming alone; resolving sign disagreement is the component that matters most as the number of models grows.

## Terminology
- interference: Used narrowly here for two specific effects during parameter merging -- redundant parameter values, and sign disagreement across models -- not for task interference during multitask training.
- trim: Resetting the parameters that changed least during fine-tuning back to their pre-trained values, before any averaging.

## Links
- arXiv: https://arxiv.org/abs/2306.01708
- PDF: https://arxiv.org/pdf/2306.01708
- HTML: https://ar5iv.labs.arxiv.org/html/2306.01708
- Hugging Face: https://huggingface.co/papers/2306.01708
- alphaXiv: https://www.alphaxiv.org/abs/2306.01708
- DOI: https://doi.org/10.52202/075280-0310
- Semantic Scholar: https://www.semanticscholar.org/paper/259064039
- Publisher: http://papers.nips.cc/paper\_files/paper/2023/hash/1644c9af28ab7916874f6fd6228a9bcf-Abstract-Conference.html
- Code: https://github.com/prateeky2806/ties-merging

## How to cite
@inproceedings{DBLP:conf/nips/YadavTCRB23,
author       = {Prateek Yadav and
                  Derek Tam and
                  Leshem Choshen and
                  Colin A. Raffel and
                  Mohit Bansal},
  editor       = {Alice Oh and
                  Tristan Naumann and
                  Amir Globerson and
                  Kate Saenko and
                  Moritz Hardt and
                  Sergey Levine},
  title        = {TIES-Merging: Resolving Interference When Merging Models},
  booktitle    = {Advances in Neural Information Processing Systems 36: Annual Conference
                  on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans,
                  LA, USA, December 10 - 16, 2023},
  year         = {2023},
  url          = {http://papers.nips.cc/paper\_files/paper/2023/hash/1644c9af28ab7916874f6fd6228a9bcf-Abstract-Conference.html},
  timestamp    = {Fri, 01 Mar 2024 00:00:00 +0100},
  biburl       = {https://dblp.org/rec/conf/nips/YadavTCRB23.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}