Elamur Research⌕
☰
home / papers / 2022 / SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser

SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser

Summary

Grammatical error correction (GEC) systems usually treat input as a flat token sequence, even though many errors, such as tense or number agreement, depend on syntactic relations. Adding syntax is difficult because standard dependency parsers are trained on clean treebanks and perform poorly on learner text. SynGEC addresses this with a purpose-built parser, GOPar. The authors extend the Stanford Dependencies scheme with three error labels so that one tree can encode both syntax and mistakes. They then parse the corrected side of parallel GEC data with an ordinary parser, align source and target with ERRANT, and project the trees onto the erroneous sentences. GOPar is trained on these automatically derived trees, so no manual annotation is needed. Its output is encoded by label-aware GCN layers placed on top of a Transformer encoder. On English (CoNLL-14, BEA-19) and Chinese (NLPCC-18, MuCGEC) benchmarks, SynGEC consistently beats strong baselines, including BART-enhanced ones. This shows that error-aware syntax helps GEC in a data-scarce setting and that the approach does not depend on a particular language.

Key points

  • The extended scheme builds on Stanford Dependencies v3.3.0 and adds the GED labels "S" (substituted), "R" (redundant) and "M" (missing). Word-order errors are modelled as a redundant error plus a missing one, and label conflicts are resolved with the priority S > R > M.

  • GOPar training data come from tree projection over parallel GEC corpora: target sentences are parsed, ERRANT extracts the edits, and trees are mapped to the source side using rules. High-quality training pairs are scarce, about 126K for English and 157K for Chinese, excluding the noisy Lang8 data.

  • Syntax is injected through DepGCN blocks between the Transformer encoder and decoder. GCN was preferred over GAT because it performed similarly but ran faster.

  • Chinese single-model F0.5 scores:

    DatasetSettingBaselineSynGEC
    NLPCC-18without PLM36.3137.91
    NLPCC-18with PLM44.6445.32
    MuCGECwithout PLM38.4940.58
    MuCGECwith PLM45.9346.51

  • Code and data are released at github.com/HillZhang1999/SynGEC. The authors name several limitations: noise from the off-the-shelf parser, coarse error labels, and use of source-side syntax only.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

39 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 37 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.emnlp-main.162 · arXiv 2210.12484

@inproceedings{zhang2022syngec,
  title = {{SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser}},
  author = {Zhang, Yue and Zhang, Bo and Li, Zhenghua and Bao, Zuyi and Li, Chen and Zhang, Min},
  booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.162},
  eprint = {2210.12484},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2022.emnlp-main.162}
}

References

78 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Roee Aharoni and Yoav Goldberg. 2017. Towards string-to-tree neural machine translation. In Proceedings of ACL (Short Papers), pages 132–140. find our card
  2. Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. 2019. Parallel iterative edit models for local sequence transduction. In Proceedings of EMNLP-IJCNLP, pages 4260–4270. find
  3. Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450. find our card
  4. Jasmijn Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Sima’an. 2017. Graph convolutional encoders for syntax-aware neural machine translation. In Proceedings of EMNLP, pages 1957–1967. find our card
  5. Samuel Bell, Helen Yannakoudakis, and Marek Rei. 2019. Context is key: Grammatical error detection with contextual word representations. In Proceedings of BEA@ACL, pages 103–115. find
  6. Yevgeni Berzak, Jessica Kenney, Carolyn Spadine, Jing Xian Wang, Lucia Lam, Keiko Sophie Mori, Sebastian Garza, and Boris Katz. 2016. Universal dependencies for learner English. In Proceedings of ACL, pages 737–746. find our card
  7. Christopher Bryant, Mariano Felice, Øistein E Andersen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Proceedings of BEA@ACL, pages 52–75. find our card
  8. Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic annotation and evaluation of error types for grammatical error correction. In Proceedings of ACL, pages 793–805. find our card
  9. Aoife Cahill. 2015. Parsing learner text: To shoehorn or not to shoehorn. In Proceedings of Linguistic Annotation Workshop, pages 144–147. find
  10. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: pretraining text encoders as discriminators rather than generators. In Proceedings of ICLR. find
  11. Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, and Guoping Hu. 2020. Revisiting pre-trained models for Chinese natural language processing. In Proceedings of EMNLP: findings, pages 657–668. find our card
  12. Daniel Dahlmeier and Hwee Tou Ng. 2012. Better evaluation for grammatical error correction. In Proceedings of NAACL-HLT, pages 568–572. find
  13. Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building a large annotated corpus of learner English: The nus corpus of learner English. In Proceedings of BEA@NAACL-HLT, pages 22–31. find
  14. Markus Dickinson and Marwa Ragheb. 2009. Dependency annotation for learner corpora. In Proceedings of International Workshop on Treebanks and Linguistic Theories, page 59. find
  15. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In Proceedings of ICLR. find our card
  16. Jennifer Foster. 2004. Parsing ungrammatical input: an evaluation procedure. In Proceedings of LREC. find our card
  17. Jennifer Foster, Joachim Wagner, and Josef Van Genabith. 2008. Adapting a wsj-trained parser to grammatically noisy text. In Proceedings of ACL (short), pages 221–224. find our card
  18. Roman Grundkiewicz, Christopher Bryant, and Mariano Felice. 2020. A crash course in automatic grammatical error correction. In Proceedings of COLING: Tutorial Abstracts, pages 33–38. find
  19. Jiatao Gu, Changhan Wang, and Jake Zhao Junbo. 2019. Levenshtein transformer. In Proceedings of NIPS, pages 11181–11191. find
  20. Homa B Hashemi and Rebecca Hwa. 2016. An evaluation of parser robustness for ungrammatical sentences. In Proceedings of EMNLP, pages 1765–1774. find our card
  21. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of CVPR, pages 770–778. find
  22. Matthew Honnibal and Mark Johnson. 2014. Joint incremental disfluency detection and dependency parsing. TACL, 2:131–142. find
  23. Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, and Kenneth Heafield. 2018. Approaching neural grammatical error correction as a low-resource machine translation task. In Proceedings of NAACL-HLT, pages 595–606. find
  24. Masahiro Kaneko and Mamoru Komachi. 2019. Multi-head multi-layer attention to deep language representations for grammatical error detection. Computing Research Repository, pages 883–891. find
  25. Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, and Kentaro Inui. 2020. Encoder-decoder models can benefit from pre-trained masked language models in grammatical error correction. In Proceedings of ACL, pages 4248–4254. find
  26. Satoru Katsumata and Mamoru Komachi. 2020. Stronger baselines for grammatical error correction using a pretrained encoder-decoder model. In Proceedings of AACL, pages 827–832. find
  27. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  28. Jared Lichtarge, Chris Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, and Simon Tong. 2019. Corpora generation for grammatical error correction. In Proceedings of NAACL-HLT, pages 3291–3301. find
  29. Mary Ann Marcinkiewicz. 1994. Building a large annotated corpus of English: The penn treebank. Using Large Corpora, page 273. find our card
  30. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Proceedings of NAACL-HLT. find our card
  31. Masato Mita, Shun Kiyono, Masahiro Kaneko, Jun Suzuki, and Kentaro Inui. 2020. A self-refinement strategy for noise reduction in grammatical error correction. In Proceedings of EMNLP (Findings), pages 267–280. find
  32. Ryo Nagata and Keisuke Sakaguchi. 2016. Phrase structure annotation and parsing for learner English. In Proceedings of ACL, pages 1837–1847. find
  33. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of ICML, pages 807–814. find
  34. Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. Ground truth for grammatical error correction metrics. In Proceedings of ACL (short), pages 588–593. find
  35. Courtney Napoles, Keisuke Sakaguchi, and Joel Tetreault. 2017. JFLEG: A fluency corpus and benchmark for grammatical error correction. In Proceedings of EACL, pages 229–234. find
  36. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of ICLR. find our card
  37. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In Proceedings of CoNLL: Shared Task, pages 1–14. find
  38. Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizumoto, and Kentaro Inui. 2019. An empirical study of incorporating pseudo data into grammatical error correction. In Proceedings of EMNLP-IJCNLP, pages 1236–1242. find
  39. Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. 2020. Gector–grammatical error correction: Tag, not rewrite. In Proceedings of BEA@ACL, pages 163– 170. find
  40. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of ACL, pages 7871–7880. find our card
  41. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT(Demo), pages 48–53. find our card
  42. Zuchao Li, Kevin Parnow, and Hai Zhao. 2022. Incorporating rich syntax information in grammatical error correction. Information Processing & Management, 59(3):102891. find
  43. Slav Petrov and Ryan McDonald. 2012. Overview of the 2012 shared task on parsing the web. In First Workshop on Syntactic Analysis of Non-Canonical Language (SANCL). find
  44. Jared Lichtarge, Chris Alberti, and Shankar Kumar. 2020. Data weighted training strategies for grammatical error correction. TACL, pages 634–646. find
  45. Eva Pettersson, Beáta Megyesi, and Joakim Nivre. 2012. Parsing the past-identification of verb constructions in historical text. In Proceedings of LaTeCH@EACL. find
  46. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research (JMLR), 21(140):1–67. find
  47. Marek Rei and Helen Yannakoudakis. 2016. Compositional sequence labeling models for error detection in learner writing. In Proceedings of ACL, pages 1181–1191. find
  48. Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In Proceedings of ACL-IJCNLP, pages 702–707. find
  49. Devendra Sachan, Yuhao Zhang, Peng Qi, and William L Hamilton. 2021. Do syntax trees help pre-trained transformers extract information? In Proceedings of EACL, pages 2647–2661. find
  50. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of ACL, pages 1715– 1725. find
  51. Yunfan Shao, Zhichao Geng, Yitao Liu, Junqi Dai, Fei Yang, Li Zhe, Hujun Bao, and Xipeng Qiu. 2021. CPT: A pre-trained unbalanced transformer for both Chinese language understanding and generation. arXiv preprint arXiv:2109.05729. find
  52. Felix Stahlberg and Shankar Kumar. 2021. Synthetic data generation for grammatical error correction with tagged corruption models. In Proceedings of BEA@EACL, pages 37–47. find
  53. Xin Sun, Tao Ge, Shuming Ma, Jingjing Li, Furu Wei, and Houfeng Wang. 2022. A unified strategy for multilingual grammatical error correction with pretrained cross-lingual language model. arXiv preprint arXiv:2201.10707. — [arXiv:2201.10707](https://arxiv.org/abs/2201.10707) find
  54. Zhaohong Wan and Xiaojun Wan. 2021. A syntax-guided grammatical error correction model with dependency tree correction. arXiv preprint arXiv:2111.03294. — [arXiv:2111.03294](https://arxiv.org/abs/2111.03294) find
  55. Xinyi Wang, Hieu Pham, Pengcheng Yin, and Graham Neubig. 2018. A tree-based decoder for neural machine translation. In Proceedings of EMNLP, pages 4772–4777. find
  56. Yu Wang, Yuelin Wang, Kai Dang, Jie Liu, and Zhuo Liu. 2021. A comprehensive survey of grammatical error correction. ACM Transactions on Intelligent Systems and Technology (TIST), pages 1–51. find
  57. Ronald J Williams and David Zipser. 1989. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1(2):270–280. find
  58. Sam Wiseman and Alexander M. Rush. 2016. Sequence-to-sequence learning as beam-search optimization. In Proceedings of EMNLP, pages 1296–1306. find
  59. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… find our card
  60. Qingrong Xia, Zhenghua Li, Min Zhang, Meishan Zhang, Guohong Fu, Rui Wang, and Luo Si. 2019. Syntax-aware neural semantic role labeling. In Proceedings of AAAI, pages 7305–7313. find
  61. Hang Yan, Xipeng Qiu, and Xuanjing Huang. 2020. A graph-based model for joint Chinese word segmentation and dependency parsing. TACL, pages 78–92. find
  62. Xin Sun, Tao Ge, Furu Wei, and Houfeng Wang. 2021. Instantaneous grammatical error correction with shallow aggressive decoding. In Proceedings of ACL-IJCNLP, pages 5937–5947. find
  63. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading ESOL texts. In Proceedings of ACL, pages 180–189. find
  64. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of ICCV, pages 2818–2826. find
  65. Michihiro Yasunaga, Jure Leskovec, and Percy Liang. 2021. Lm-critic: Language models for unsupervised grammatical error correction. In Proceedings of EMNLP, pages 7752–7763. find
  66. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NIPS, pages 5998–6008. find
  67. Zheng Yuan and Ted Briscoe. 2016. Grammatical error correction using neural machine translation. In Proceedings of NAACL-HLT, pages 380–386. find
  68. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of ICLR. find
  69. Zheng Yuan, Shiva Taslimipoor, Christopher Davis, and Christopher Bryant. 2021. Multi-class grammatical error detection for correction: A tale of two systems. In Proceedings of EMNLP, pages 8722–8736. find
  70. Joachim Wagner. 2012. Detecting grammatical errors with treebank-induced, probabilistic parsers. Ph.D. thesis, Dublin City University. find
  71. Baolin Zhang. 2009. Features and functions of the HSK dynamic composition corpus. International Chinese Language Education, 4:71–79. find
  72. Configuration Value Pre-training Transformer-base (w/o PLM) Base architecture Transformer-large (w/ PLM) Pretrained Language model BART-large (Lewis et al., 2020) Number of epochs 60 Devices 8 Tesla V100 GPU (32GB) Batch size per GPU 8096 tokens Adam (Kingma and Ba, 2014) Optimizer (β 1 “ 0.9, β 2 “… find
  73. Bo Zhang, Yue Zhang, Rui Wang, Zhenghua Li, and Min Zhang. 2020a. Syntax-aware opinion role labeling with dependency graph convolutional networks. In Proceedings of ACL, pages 3249–3258. find
  74. Meishan Zhang, Zhenghua Li, Guohong Fu, and Min Zhang. 2019. Syntax-enhanced neural machine translation with syntax-aware word representations. In Proceedings of NAACL-HLT, pages 1151–1161. find
  75. Yu Zhang, Zhenghua Li, and Min Zhang. 2020b. Efficient second-order treecrf for neural dependency parsing. In Proceedings of ACL, pages 3295–3305. find our card
  76. Yue Zhang, Zhenghua Li, Zuyi Bao, Jiacheng Li, Bo Zhang, Chen Li, Fei Huang, and Min Zhang. 2022. MuCGEC: a multi-reference multi-source evaluation dataset for Chinese grammatical error correction. In Proceedings of NAACL-HLT, pages 3118–3130. find
  77. Yuanyuan Zhao, Nan Jiang, Weiwei Sun, and Xiaojun Wan. 2018. Overview of the NLPCC 2018 shared task: Grammatical error correction. In CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC), pages 439–445. find
  78. Zewei Zhao and Houfeng Wang. 2020. MaskGEC: Improving neural grammatical error correction via dynamic masking. In Proceedings of AAAI, pages 1226– 1233. *Table 7: Hyper-parameter values used in our experiments.* Baseline SynGEC Baseline SynGEC find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/zhang-2022-syngec-syntax-enhanced-grammatical-error/