# Call for Papers - The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus

Authors: Alex Warstadt, Leshem Choshen, Aaron Mueller, Adina Williams, Ethan Wilcox, Chengxu Zhuang
Venue: CoRR (2023)

## Abstract
We present the call for papers for the BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus. This shared task is intended for participants with an interest in small scale language modeling, human language acquisition, low-resource NLP, and cognitive modeling. In partnership with CoNLL and CMCL, we provide a platform for approaches to pretraining with a limited-size corpus sourced from data inspired by the input to children. The task has three tracks, two of which restrict the training data to pre-released datasets of 10M and 100M words and are ded-icated to explorations of approaches such as architectural variations, self-supervised objec-tives, or curriculum learning. The final track only restricts the amount of text used, allowing innovation in the choice of the data, its domain, and even its modality (i.e., data from sources other than text is welcome). We will release a shared evaluation pipeline which scores models on a variety of benchmarks and tasks, in-cluding targeted syntactic evaluations and natural language understanding.

## Links
- arXiv: https://arxiv.org/abs/2301.11796
- PDF: https://arxiv.org/pdf/2301.11796
- HTML: https://ar5iv.labs.arxiv.org/html/2301.11796
- Hugging Face: https://huggingface.co/papers/2301.11796
- alphaXiv: https://www.alphaxiv.org/abs/2301.11796
- DOI: https://doi.org/10.48550/ARXIV.2301.11796
- Semantic Scholar: https://www.semanticscholar.org/paper/256358845
- Publisher: https://doi.org/10.48550/arXiv.2301.11796

## How to cite
@article{DBLP:journals/corr/abs-2301-11796,
  author       = {Alex Warstadt and
                  Leshem Choshen and
                  Aaron Mueller and
                  Adina Williams and
                  Ethan Wilcox and
                  Chengxu Zhuang},
  title        = {Call for Papers - The BabyLM Challenge: Sample-efficient pretraining
                  on a developmentally plausible corpus},
  journal      = {CoRR},
  volume       = {abs/2301.11796},
  year         = {2023},
  url          = {https://doi.org/10.48550/arXiv.2301.11796},
  doi          = {10.48550/ARXIV.2301.11796},
  eprinttype    = {arXiv},
  eprint       = {2301.11796},
  timestamp    = {Tue, 31 Jan 2023 00:00:00 +0100},
  biburl       = {https://dblp.org/rec/journals/corr/abs-2301-11796.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}