Elamur Research⌕
☰
home / papers / 2014 / A Sentence Judgment System for Grammatical Error Detection

A Sentence Judgment System for Grammatical Error Detection

Summary

Learners of Chinese as a foreign language (CFL) have far fewer automatic grammar-checking tools than learners of English, even as MOOCs increase the need for large-scale assessment. The authors present a web-based demo system that decides whether a Chinese sentence contains grammatical errors. Each input is segmented, with unknown-word merging to handle OOV items, and POS-tagged. Two detectors then analyse it. The first applies 142 expert-written syntactic rules derived from Cheng's (1997) collection of 900 typical learner errors. The second is a statistical component. Bi- and tri-gram SRILM models trained on the Sinica corpus score 19,080 erroneous HSK sentences and their manual corrections, and normal densities fitted to these scores act as "correct" and "incorrect" models. A sentence is flagged when the incorrect model gives it higher probability. In tests, the rules proved precise but missed most errors, while the n-gram models showed the opposite pattern. Combining the two lets users trade recall against precision. Because the interface also displays the matched rules and n-gram frequencies, it can support autonomous learning.

Key points

  • The rule notation uses wildcards, exclusions, alternatives, mutual exclusivity (mx{W1 W2}) and a follows-after operator over POS tags.
  • The test set has 880 erroneous sentences from CSL learners at the NCKU Chinese Language Center, plus their 880 corrected versions.
  • Rule-based detection reached a precision of 0.857 but a recall of only 0.224, with a false positive rate of 0.038.
  • The 3-gram model outperformed the 2-gram model on every metric (F1 0.689 vs 0.638), but it wrongly flagged about 60% of correct sentences.
  • Combining rules with AND raised precision to 0.924 and lowered FPR to 0.007, though recall fell to 0.083. The OR combination reached a recall of 1.0. The demo itself flags a sentence only when both methods agree.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{lee2014sentence,
  title = {{A Sentence Judgment System for Grammatical Error Detection}},
  author = {Lee, Lung-Hao and Yu, Liang-Chih and Lee, Kuei-Ching and Tseng, Yuen-Hsien and Chang, Li-Ping and Chen, Hsin-Hsi},
  booktitle = {Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: System Demonstrations},
  year = {2014},
  url = {https://aclanthology.org/C14-2015/}
}

References

9 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Andreas Stolcke. 2002. SRILM — An extensible language modeling toolkit. Proceedings of ICSLP’02, pages 901-904. find
  2. Chi-Hsin Yu and Hsin-Hsi Chen. 2012. Detecting word ordering errors in Chinese sentences for learning Chinese as a foreign language. Proceedings of COLING’12, pages 3003-3018. find
  3. Christopher D. Manning and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. MIT Press. Cambridge, MA. find
  4. Chung-Hsien Wu, Chao-Hung Liu, Matthew Harris and Liang-Chih Yu. 2010. Sentence correction incorporating relative position and parse template language model. IEEE Transactions on Audio, Speech, and Language Processing, 18(6):1170-1181. find
  5. Keh-Jiann Chen and Wei-Yun Ma. 2002. Unknown word extraction for Chinese documents. Proceedings of COLING’02, pages 169-175. find
  6. M. Cheng. 1997. Analysis of 900 Common Erroneous Samples of Chinese Sentences - for Chinese Learners from English Speaking Countries (in Chinese). Beijing, CN: Sinolingua. find
  7. Ru-Ying Chang, Chung-Hsien Wu, and Philips K. Prasetyo. 2012. Error diagnosis of Chinese sentences using inductive learning algorithm and decomposition-based testing mechanism. ACM Transactions on Asian Language Information Processing, 11(1):Article 3. find
  8. Yu-Fang Tsai and Keh-Jiann Chen. 2004. Reliable and cost-effective pos-tagging. International Journal of Computational Linguistics and Chinese Language Processing, 9(1):83-96. find
  9. Recall 0.224 0.751 0.838 1.000 1.000 0.083 0.083 F1 0.356 0.638 0.689 0.667 0.668 0.153 0.153 False Positive Rate 0.038 0.603 0.595 1.000 0.993 0.007 0.007 find

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lee-2014-sentence-judgment-system-grammatical-error/