# Genie: Achieving Human Parity in Content-Grounded Datasets Generation

Authors: Asaf Yehudai, Boaz Carmeli, Yosi Mass, Ofir Arviv, Nathaniel Mills, Assaf Toledo, Eyal Shnarch, Leshem Choshen
Venue: CoRR (2024)

## Abstract
The lack of high-quality data for content-grounded generation tasks has been identified as a major obstacle to advancing these tasks. To address this gap, we propose Genie, a novel method for automatically generating high-quality content-grounded data. It consists of three stages: (a) Content Preparation, (b) Generation: creating task-specific examples from the content (e.g., question-answer pairs or summaries). (c) Filtering mechanism aiming to ensure the quality and faithfulness of the generated data. We showcase this methodology by generating three large-scale synthetic data, making wishes, for Long-Form Question-Answering (LFQA), summarization, and information extraction. In a human evaluation, our generated data was found to be natural and of high quality. Furthermore, we compare models trained on our data with models trained on human-written data -- ELI5 and ASQA for LFQA and CNN-DailyMail for Summarization. We show that our models are on par with or outperforming models trained on human-generated data and consistently outperforming them in faithfulness. Finally, we applied our method to create LFQA data within the medical domain and compared a model trained on it with models trained on other domains.

## Links
- arXiv: https://arxiv.org/abs/2401.14367
- PDF: https://arxiv.org/pdf/2401.14367
- HTML: https://arxiv.org/html/2401.14367
- Hugging Face: https://huggingface.co/papers/2401.14367
- alphaXiv: https://www.alphaxiv.org/abs/2401.14367
- DOI: https://doi.org/10.48550/ARXIV.2401.14367
- Semantic Scholar: https://www.semanticscholar.org/paper/267211959
- Publisher: https://doi.org/10.48550/arXiv.2401.14367

## How to cite
@article{DBLP:journals/corr/abs-2401-14367,
  author       = {Asaf Yehudai and
                  Boaz Carmeli and
                  Yosi Mass and
                  Ofir Arviv and
                  Nathaniel Mills and
                  Assaf Toledo and
                  Eyal Shnarch and
                  Leshem Choshen},
  title        = {Genie: Achieving Human Parity in Content-Grounded Datasets Generation},
  journal      = {CoRR},
  volume       = {abs/2401.14367},
  year         = {2024},
  url          = {https://doi.org/10.48550/arXiv.2401.14367},
  doi          = {10.48550/ARXIV.2401.14367},
  eprinttype    = {arXiv},
  eprint       = {2401.14367},
  timestamp    = {Tue, 06 Feb 2024 00:00:00 +0100},
  biburl       = {https://dblp.org/rec/journals/corr/abs-2401-14367.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}