We propose USim, a semantic measure for Grammatical Error Correction (that measures the semantic faithfulness of the output to the source, thereby complementing existing reference-less measures (RLMs) for measuring the output’s grammaticality. USim operates by comparing the semantic symbolic structure of the source and the correction, without relying on manually-curated references. Our experiments establish the validity of USim, by showing that the semantic structures can be consistently applied to ungrammatical text, that valid corrections obtain a high USim similarity score to the source, and that invalid corrections obtain a lower score.
@inproceedings{DBLP:conf/naacl/ChoshenA18,
author = {Leshem Choshen and
Omri Abend},
editor = {Marilyn A. Walker and
Heng Ji and
Amanda Stent},
title = {Reference-less Measure of Faithfulness for Grammatical Error Correction},
booktitle = {Proceedings of the 2018 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language Technologies,
NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short
Papers)},
pages = {124--129},
publisher = {Association for Computational Linguistics},
year = {2018},
url = {https://doi.org/10.18653/v1/n18-2020},
doi = {10.18653/V1/N18-2020},
timestamp = {Fri, 06 Aug 2021 01:00:00 +0200},
biburl = {https://dblp.org/rec/conf/naacl/ChoshenA18.bib},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
See the full reference list in the paper.