Elamur Research⌕
☰
home / papers / 2022 / Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction

Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2022.findings-emnlp.40.pdf

Summary

Most Chinese Grammatical Error Correction (CGEC) research faces two problems: there is little high-quality annotated training data, and standard test sets such as NLPCC and CGED contain errors from L2 learners. Learner errors tend to be short and obvious, while native speakers make subtler errors in otherwise fluent text. The authors address both problems. First, they introduce CLG, a method that adds synthetic errors to clean text using hand-designed rules. The rules are based on Chinese linguistics references and cover six error categories. Ambiguity and spelling errors are deliberately excluded. The clean source sentences come from public Chinese corpora. Second, they release NaCGEC, a human-annotated benchmark built entirely from real errors made by native speakers. In experiments with Transformer and GECToR-Chinese models, training on CLG data outperforms training on conventional CGEC data such as Lang8 and gives larger gains than generic data augmentation. NaCGEC proves hard both for models and for educated native speakers. Together, the two resources move CGEC evaluation closer to real-world use.

Key points

  • The rules target six kinds of mistakes: Improper Logicality, Structural Confusion, Missing Component, Redundant Component, Improper Word Order, and Improper Collocation. Ambiguity is dropped because correcting it requires context.
  • Clean source text comes from People's Daily, a Chinese MT dataset, and Chinese wiki corpora. In a manual check of 1,000 randomly sampled sentences, more than 97% were grammatical.
  • The tested models are a Transformer with about 54M parameters and GECToR-Chinese with about 108M parameters. The authors did not try larger models or ensembles.
  • Per-type F0.5 scores on NaCGEC show the weakest results on Improper Collocation (6.13), Missing Component (7.44), and Improper Word Order (13.20). The best result is on Structural Confusion (33.28).
  • Code and data are available at github.com/masr2000/CLG-CGEC. The authors note that the rules are specific to Chinese and do not transfer directly to other languages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

32 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 31 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.findings-emnlp.40 · arXiv 2210.10442

@inproceedings{ma2022linguistic,
  title = {{Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction}},
  author = {Ma, Shirong and Li, Yinghui and Sun, Rongyi and Zhou, Qingyu and Huang, Shulin and Zhang, Ding and Yangning, Li and Liu, Ruiyang and Li, Zhongli and Cao, Yunbo and Zheng, Haitao and Shen, Ying},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-emnlp.40},
  eprint = {2210.10442},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2022.findings-emnlp.40}
}

References

39 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xiliang Cui and Bao-lin Zhang. 2011. The principles for building the “international corpus of learner chinese”. Applied Linguistics, 2:100–108. find
  2. Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, and Guoping Hu. 2020. Revisiting pre-trained models for Chinese natural language processing. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 657–668, Online. Association for Computational Linguistics. find our card
  3. Daniel Dahlmeier and Hwee Tou Ng. 2012. Better evaluation for grammatical error correction. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 568–572, Montréal, Canada. Association for Computationa… find
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… find
  5. Huizhong Duan and Bo-June Paul Hsu. 2011. Online spelling correction for query completion. In Proceedings of the 20th International Conference on World Wide Web, WWW 2011, Hyderabad, India, March 28 - April 1, 2011, pages 117–126. ACM. find
  6. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. find
  7. Kai Fu, Jin Huang, and Yitao Duan. 2018. Youdao’s winning solution to the NLPCC-2018 task 2 challenge: A neural machine translation approach to chinese grammatical error correction. In Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Hohhot, China, Au… find
  8. Borong Huang and Xudong Liao. 2011. Modern Chinese (Updated Fifth Edition). Higher Education Press, Beijing, China. find
  9. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. find
  10. Marek Kubis, Zygmunt Vetulani, Mikolaj Wypych, and Tomasz Zietkiewicz. 2019. Open challenge for correcting errors of speech recognition systems. In Human Language Technology. Challenges for Computer Science and Linguistics - 9th Language and Technology Conference, LTC 2019, Poznan, Poland, May 17-19… find
  11. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), Williams College, Williamstown, MA, USA, June 28… find
  12. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics, pages 159–174. find
  13. VI Levenshtein. 1966. Binary codes capable of correcting deletions, insertions and reversals. In Soviet Physics Doklady, volume 10, page 707. find
  14. Jiquan Li, Junliang Guo, Yongxin Zhu, Xin Sheng, De-qiang Jiang, Bo Ren, and Linli Xu. 2022a. Sequence-to-action: Grammatical error correction with action guided sequence generation. CoRR, abs/2205.10884. — [arXiv:2205.10884](https://arxiv.org/abs/2205.10884) find
  15. Piji Li and Shuming Shi. 2021. Tail-to-tail non-autoregressive sequence prediction for Chinese grammatical error correction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1… find
  16. Yinghui Li, Qingyu Zhou, Yangning Li, Zhongli Li, Ruiyang Liu, Rongyi Sun, Zizhen Wang, Chao Li, Yunbo Cao, and Hai-Tao Zheng. 2022b. The past mistake is the future wisdom: Error-driven contrastive probability optimization for Chinese spell checking. In Findings of the Association for Computational… find
  17. Deng Liang, Chen Zheng, Lei Guo, Xin Cui, Xiuzhang Xiong, Hengqiao Rong, and Jinpeng Dong. 2020. BERT enhanced neural machine translation and sequence tagging model for Chinese grammatical error diagnosis. In Proceedings of the 6th Workshop on Natural Language Processing Techniques for Educational A… find
  18. Ilya Loshchilov and Frank Hutter. 2017. weight decay regularization in adam. abs/1711.05101. find
  19. Fixing CoRR, Eric Malmi, Sebastian Krause, Sascha Rothe, Daniil Mirylenka, and Aliaksei Severyn. 2019. Encode, tag, realize: High-precision text editing. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural La… find our card
  20. Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. 2020. GECToR – grammatical error correction: Tag, not rewrite. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 163–170, Seattle, WA, USA → Online. A… find
  21. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, a… find our card
  22. Gaoqi Rao, Qi Gong, Baolin Zhang, and Endong Xun. 2018. Overview of NLPTEA-2018 share task Chinese grammatical error diagnosis. In Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, pages 42–51, Melbourne, Australia. Association for Computational… find
  23. Gaoqi Rao, Erhong Yang, and Baolin Zhang. 2020. Overview of NLPTEA-2020 shared task for Chinese grammatical error diagnosis. In Proceedings of the 6th Workshop on Natural Language Processing Techniques for Educational Applications, pages 25–35, Suzhou, China. Association for Computational Linguistic… find
  24. Hongkai Ren, Liner Yang, and Endong Xun. 2018. A sequence to sequence learning for chinese grammatical error correction. In Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Hohhot, China, August 26-30, 2018, Proceedings, Part II, volume 11109 of Lectu… find
  25. Jingmin Shao. 2016. General Theory of Modern Chinese (Third Edition). Shanghai Educational Publishing House, Shanghai, China. find
  26. Maosong Sun, Xinxiong Chen, Kaixu Zhang, Zhipeng Guo, and Zhiyuan Liu. 2016. Thulac: An efficient lexical analyzer for chinese. find
  27. Zecheng Tang, Yixin Ji, Yibo Zhao, and Junhui Li. 2021. Chinese grammatical error correction enhanced by data augmentation from word and character levels. In Proceedings of the 20th Chinese National Conference on Computational Linguistics, Hohhot, China, pages 13–15. find
  28. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017,… find
  29. Chencheng Wang, Liner Yang, Yingying Wang, Yong-ping Du, and Erhong Yang. 2020a. Chinese grammatical error correction method based on transformer enhanced architecture. Journal of Chinese Information Processing, 34(6):106. find
  30. Dingmin Wang, Yan Song, Jing Li, Jialong Han, and Haisong Zhang. 2018. A hybrid approach to automatic corpus generation for Chinese spelling check. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2517–2527, Brussels, Belgium. Association for Computati… find
  31. Bright Xu. 2019. Nlp chinese corpus: Large scale chinese corpus for nlp. find
  32. Bao-lin Zhang and Xiliang Cui. 2013. Design concepts of “the construction and research of the interlanguage corpus of chinese from global learners”. Language Teaching and Linguistic Study, 5:27–34. find
  33. Yue Zhang, Zhenghua Li, Zuyi Bao, Jiacheng Li, Bo Zhang, Chen Li, Fei Huang, and Min Zhang. 2022. Mucgec: a multi-reference multi-source evaluation dataset for chinese grammatical error correction. CoRR, abs/2204.10994. find
  34. Yuanyuan Zhao, Nan Jiang, Weiwei Sun, and Xiaojun Wan. 2018. Overview of the NLPCC 2018 shared task: Grammatical error correction. In Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Hohhot, China, August 26-30, 2018, Proceedings, Part II, volume 1110… find
  35. Zewei Zhao and Houfeng Wang. 2020. Maskgec: Improving neural grammatical error correction via dynamic masking. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Sympos… find
  36. Junpei Zhou, Chen Li, Hengyou Liu, Zuyi Bao, Guang-wei Xu, and Linlin Li. 2018. Chinese grammatical error correction using statistical and neural models. In Natural Language Processing and Chinese Computing - 7th CCF International Conference, NLPCC 2018, Hohhot, China, August 26-30, 2018, Proceeding… find
  37. Wei Wang, Bin Bi, Ming Yan, Chen Wu, Jiangnan Xia, Zuyi Bao, Liwei Peng, and Luo Si. 2020b. Structbert: Incorporating language structures into pre-training for deep language understanding. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 202… find
  38. Yingying Wang, Cunliang Kong, Liner Yang, Yijun Wang, Xiaorong Lu, Renfen Hu, Shan He, Zhenghao Liu, Yun Chen, Erhong Yang, and Maosong Sun. 2021. YACLC: A chinese learner corpus with multi-dimensional annotation. CoRR, abs/2112.15043. — [arXiv:2112.15043](https://arxiv.org/abs/2112.15043) find
  39. Yu Wang, Yuelin Wang, Jie Liu, and Zhuo Liu. 2020c. A comprehensive survey of grammar error correction. CoRR, abs/2005.06600. — [arXiv:2005.06600](https://arxiv.org/abs/2005.06600) find

↑ back to top