# Enhancing the Transformer Decoder with Transition-based Syntax

Authors: Leshem Choshen, Omri Abend
Venue: Proceedings of the 26th Conference on Computational Natural Language Learning, CoNLL 2022, Abu Dhabi, United Arab Emirates (Hybrid Event), December 7-8, 2022 (2022)

## Abstract
Notwithstanding recent advances, syntactic generalization remains a challenge for text decoders. While some studies showed gains from incorporating source-side symbolic syntactic and semantic structure into text generation Transformers, very little work addressed the decoding of such structure. We propose a general approach for tree decoding using a transition-based approach. Examining the challenging test case of incorporating Universal Dependencies syntax into machine translation, we present substantial improvements on test sets that focus on syntactic generalization, while presenting improved or comparable performance on standard MT benchmarks. Further qualitative analysis addresses cases where syntactic generalization in the vanilla Transformer decoder is inadequate and demonstrates the advantages afforded by integrating syntactic information.

## Links
- arXiv: https://arxiv.org/abs/2101.12640
- PDF: https://arxiv.org/pdf/2101.12640
- HTML: https://ar5iv.labs.arxiv.org/html/2101.12640
- Hugging Face: https://huggingface.co/papers/2101.12640
- alphaXiv: https://www.alphaxiv.org/abs/2101.12640
- DOI: https://doi.org/10.18653/V1/2022.CONLL-1.27
- ACL Anthology: https://aclanthology.org/2022.conll-1.27/
- Semantic Scholar: https://www.semanticscholar.org/paper/248810900
- Publisher: https://doi.org/10.18653/v1/2022.conll-1.27

## How to cite
@inproceedings{DBLP:conf/conll/ChoshenA22,
author       = {Leshem Choshen and
                  Omri Abend},
  editor       = {Antske Fokkens and
                  Vivek Srikumar},
  title        = {Enhancing the Transformer Decoder with Transition-based Syntax},
  booktitle    = {Proceedings of the 26th Conference on Computational Natural Language
                  Learning, CoNLL 2022, Abu Dhabi, United Arab Emirates (Hybrid Event),
                  December 7-8, 2022},
  pages        = {384--404},
  publisher    = {Association for Computational Linguistics},
  year         = {2022},
  url          = {https://doi.org/10.18653/v1/2022.conll-1.27},
  doi          = {10.18653/V1/2022.CONLL-1.27},
  timestamp    = {Fri, 12 Apr 2024 01:00:00 +0200},
  biburl       = {https://dblp.org/rec/conf/conll/ChoshenA22.bib},
  bibsource    = {dblp computer science bibliography, https://dblp.org}
}