# [Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus

Authors: Leshem Choshen, Ryan Cotterell, Michael Y. Hu, Tal Linzen, Aaron Mueller, Candace Ross, Alex Warstadt, Ethan Wilcox, Adina Williams, Chengxu Zhuang
Venue: CoRR (2024)

## Abstract
After last year's successful BabyLM Challenge, the competition will be hosted again in 2024/2025. The overarching goals of the challenge remain the same; however, some of the competition rules will be different. The big changes for this year's competition are as follows: First, we replace the loose track with a paper track, which allows (for example) non-model-based submissions, novel cognitively-inspired benchmarks, or analysis techniques. Second, we are relaxing the rules around pretraining data, and will now allow participants to construct their own datasets provided they stay within the 100M-word or 10M-word budget. Third, we introduce a multimodal vision-and-language track, and will release a corpus of 50% text-only and 50% image-text multimodal data as a starting point for LM model training. The purpose of this CfP is to provide rules for this year's challenge, explain these rule changes and their rationale in greater detail, give a timeline of this year's competition, and provide answers to frequently asked questions from last year's challenge.

## Links
- arXiv: https://arxiv.org/abs/2404.06214
- PDF: https://arxiv.org/pdf/2404.06214
- HTML: https://arxiv.org/html/2404.06214
- Hugging Face: https://huggingface.co/papers/2404.06214
- alphaXiv: https://www.alphaxiv.org/abs/2404.06214
- DOI: https://doi.org/10.48550/ARXIV.2404.06214
- Semantic Scholar: https://www.semanticscholar.org/paper/269009968
- Publisher: https://doi.org/10.48550/arXiv.2404.06214
- Code: https://github.com/babylm/evaluation-pipeline-2024

## How to cite
@article{DBLP:journals/corr/abs-2404-06214,
  author       = {Leshem Choshen and
                  Ryan Cotterell and
                  Michael Y. Hu and
                  Tal Linzen and
                  Aaron Mueller and
                  Candace Ross and
                  Alex Warstadt and
                  Ethan Wilcox and
                  Adina Williams and
                  Chengxu Zhuang},
  title        = {[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining
                  on a developmentally plausible corpus},
  journal      = {CoRR},
  volume       = {abs/2404.06214},
  year         = {2024},
  url          = {https://doi.org/10.48550/arXiv.2404.06214},
  doi          = {10.48550/ARXIV.2404.06214},
  eprinttype    = {arXiv},
  eprint       = {2404.06214},
  timestamp    = {Wed, 15 May 2024 01:00:00 +0200},
  biburl       = {https://dblp.org/rec/journals/corr/abs-2404-06214.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}