# Cluster & Tune: Boost Cold Start Performance in Text Classification

Authors: Eyal Shnarch, Ariel Gera, Alon Halfon, Lena Dankin, Leshem Choshen, Ranit Aharonov, Noam Slonim
Venue: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), {ACL} 2022, Dublin, Ireland, May 22-27, 2022 (2022)

## Abstract
In real-world scenarios, a text classification task often begins with a cold start, when labeled data is scarce. In such cases, the common practice of fine-tuning pre-trained models, such as BERT, for a target classification task, is prone to produce poor performance. We suggest a method to boost the performance of such models by adding an intermediate unsupervised classification task, between the pre-training and fine-tuning phases. As such an intermediate task, we perform clustering and train the pre-trained model on predicting the cluster labels.We test this hypothesis on various data sets, and show that this additional classification phase can significantly improve performance, mainly for topical classification tasks, when the number of labeled instances available for fine-tuning is only a couple of dozen to a few hundred.

## Links
- arXiv: https://arxiv.org/abs/2203.10581
- PDF: https://arxiv.org/pdf/2203.10581
- HTML: https://ar5iv.labs.arxiv.org/html/2203.10581
- Hugging Face: https://huggingface.co/papers/2203.10581
- alphaXiv: https://www.alphaxiv.org/abs/2203.10581
- DOI: https://doi.org/10.18653/V1/2022.ACL-LONG.526
- ACL Anthology: https://aclanthology.org/2022.acl-long.526/
- Semantic Scholar: https://www.semanticscholar.org/paper/247594392
- Publisher: https://doi.org/10.18653/v1/2022.acl-long.526

## How to cite
@inproceedings{DBLP:conf/acl/ShnarchGHDCAS22,
author       = {Eyal Shnarch and
                  Ariel Gera and
                  Alon Halfon and
                  Lena Dankin and
                  Leshem Choshen and
                  Ranit Aharonov and
                  Noam Slonim},
  editor       = {Smaranda Muresan and
                  Preslav Nakov and
                  Aline Villavicencio},
  title        = {Cluster {\&} Tune: Boost Cold Start Performance in Text Classification},
  booktitle    = {Proceedings of the 60th Annual Meeting of the Association for Computational
                  Linguistics (Volume 1: Long Papers), {ACL} 2022, Dublin, Ireland,
                  May 22-27, 2022},
  pages        = {7639--7653},
  publisher    = {Association for Computational Linguistics},
  year         = {2022},
  url          = {https://doi.org/10.18653/v1/2022.acl-long.526},
  doi          = {10.18653/V1/2022.ACL-LONG.526},
  timestamp    = {Mon, 01 Aug 2022 01:00:00 +0200},
  biburl       = {https://dblp.org/rec/conf/acl/ShnarchGHDCAS22.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}