Elamur Research⌕
☰
home / papers / 2017 / Adapting Sequence Models for Sentence Correction

Adapting Sequence Models for Sentence Correction

Paper · 2017 · Allen Schmaltz, Yoon Kim, Alexander Rush et al.

Summary

The paper examines how to make neural sequence-to-sequence models competitive for sentence correction, which is the task of rewriting a possibly erroneous sentence into a fixed version. Earlier work suggested that phrase-based SMT with task-specific features was the strongest approach, but those comparisons rarely used identical training data. The authors run controlled experiments on the AESW 2016 corpus, which contains over a million sentences from academic writing corrected by professional editors. They compare word-level, CharCNN, and pure character-level encoder-decoders against a strong Moses-based SMT baseline. They also propose representing the target as the source with inline insertion and deletion tags (diffs), and optionally adding a journal-domain token. Character-level models that predict diffs performed best and clearly beat SMT. On CoNLL-2014, adjusting bias weights on the diff tags let a word model match prior neural systems while using much less data. The work shows that output representation and controlled comparisons matter as much as architecture choice.

Key points

  • AESW training data: 1,182,491 sentences, 460,901 of them with edits. Models are 2-layer LSTMs with 750 hidden units, built in OpenNMT. Code is released at github.com/allenschmaltz/grammar.
  • On AESW test, CHAR+BI reached 44.62 M2, compared with 38.31 for SMT+BLEU (about 6 M2 / 0.5 GLEU points higher). Adding domain information (CHAR+BI+DOM) raised this to 46.72 M2 and 91.39 GLEU.
  • The character model is also the smallest, at about 25M parameters versus about 144M for WORD+BI. It handles rare replacement errors far better than SMT or the word models.
  • On CoNLL-2013, tuning the diff-tag biases lifted WORD+BI from 4.60 to 33.14 F0.5.
  • On the CoNLL-2014 test set, WORD+BI trained on NUCLE and Lang-8 scored 41.37 M2. This comes within 0.2 of Ji et al. (2017), who used CLC, and outperforms the systems of both Yuan and Briscoe (2016) and Xie et al. (2016).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harvard UniversityCambridge · United States112 works · 42,023 citations🏢Meta (Israel)Tel Aviv · Israel38 works · 32,024 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Allen Schmaltz, Yoon Kim, Alexander Rush and Stuart Shieber. 2017. Adapting Sequence Models for Sentence Correction. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1298 · arXiv 1707.09067

@inproceedings{schmaltz2017adapting,
  title = {{Adapting Sequence Models for Sentence Correction}},
  author = {Allen Schmaltz, Yoon Kim, Alexander Rush and Stuart Shieber},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1298},
  eprint = {1707.09067},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d17-1298}
}

References

29 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shamil Chollampatt, Duc Tam Hoang, and Hwee Tou Ng. 2016a. Adapting grammatical error correction based on the native language of writers with neural network joint models. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1901–1911, Austin, Texas. Associ… find
  2. Shamil Chollampatt, Kaveh Taghipour, and Hwee Tou Ng. 2016b. Neural network translation models for grammatical error correction. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, pages 2768–2774. AAAI Press.
  3. Daniel Dahlmeier and Hwee Tou Ng. 2012. Better evaluation for grammatical error correction. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT ’12, pages 568–572, Stroudsburg, PA, USA. Associati… find
  4. Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building a large annotated corpus of learner english: The nus corpus of learner english. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 22–31, Atlanta, Georgia. Association for Computat…
  5. Robert Dale, Ilya Anisimoff, and George Narroway. 2012. find
  6. Hoo 2012: A report on the preposition and determiner error correction shared task. In Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, pages 54–62, Stroudsburg, PA, USA. Association for Computational Linguistics.
  7. Robert Dale and Adam Kilgarriff. 2011. Helping our own: The hoo 2011 pilot shared task. In Proceedings of the 13th European Workshop on Natural Language Generation, ENLG ’11, pages 242–249, Stroudsburg, PA, USA. Association for Computational Linguistics.
  8. Vidas Daudaravicius. 2016. Automated evaluation of scientific writing data set (version 1.2) [data file]. VTeX. find
  9. Vidas Daudaravicius, Rafael E. Banchs, Elena Volodina, and Courtney Napoles. 2016. A report on the automatic evaluation of scientific writing shared task. In Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, pages 53–62, San Diego, CA. Association for C… find
  10. Yvette Graham, Nitika Mathur, and Timothy Baldwin. 2014. Randomized significance tests in machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 266–274, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  11. Duc Tam Hoang, Shamil Chollampatt, and Hwee Tou Ng. 2016. Exploiting n-best hypotheses to improve an smt approach to grammatical error correction. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJ-CAI’16, pages 2803–2809. AAAI Press. find
  12. J. Ji, Q. Wang, K. Toutanova, Y. Gong, S. Truong, and J. Gao. 2017. A Nested Attention Neural Hybrid Model for Grammatical Error Correction. ArXiv eprints. find
  13. Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2016. Phrase-based machine translation is state-ofthe-art for automatic grammatical error correction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1546–1556, Austin, Texas. Association for Computationa… find
  14. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M. Rush. 2016. Character-Aware Neural Language Models. In Proceedings of AAAI.
  15. G. Klein, Y. Kim, Y. Deng, J. Senellart, and A. M. Rush. OpenNMT: Open-Source Toolkit for Neural Machine Translation. ArXiv e-prints.
  16. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of EMNLP 2004, pages 388–395, Barcelona, Spain. Association for Computational Linguistics.
  17. Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1412–1421, Lisbon, Portugal. Association for Computational Linguistics.
  18. Tomoya Mizumoto, Yuta Hayashibe, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. 2012. The effect of learner corpus size in grammatical error correction of ESL writings. In Proceedings of COLING 2012: Posters, pages 863–872, Mumbai, India. find
  19. The COLING 2012 Organizing Committee. find
  20. Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2016. GLEU without tuning. eprint arXiv:1605.02592 [cs.CL]. find
  21. Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pages 1– 14, Baltimore, M… find
  22. Hwee Tou Ng, Siew Mei Wu, Yuanbin Wu, Christian Hadiwinoto, and Joel Tetreault. 2013. The CoNLL-2013 shared task on grammatical error correction. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task, pages 1–12, Sofia, Bulgaria. Association for Computa… find
  23. Diane Nicholls. 2003. The cambridge learner corpus: Error coding and analysis for lexicography and ELT. In Proceedings of the Corpus Linguistics 2003 conference, pages 572–581. find
  24. Alla Rozovskaya and Dan Roth. 2016. Grammatical error correction: Machine translation and classifiers. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2205–2215, Berlin, Germany. Association for Computational Linguistics. find
  25. Allen Schmaltz, Yoon Kim, Alexander M. Rush, and Stuart Shieber. 2016. Sentence-level grammatical error identification as sequence-to-sequence correction. In Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, pages 242–251, San Diego, CA. Association for… find
  26. Andreas Stolcke. 2002. Srilm – an extensible language modeling toolkit. In Proc. Intl. Conf. on Spoken Language Processing, volume 2, pages 901–904, Denver. find
  27. Toshikazu Tajiri, Mamoru Komachi, and Yuji Matsumoto. 2012. Tense and aspect error correction for esl learners using global context. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 198–202, Jeju Island, Korea. Association for…
  28. Ziang Xie, Anand Avati, Naveen Arivazhagan, Dan Jurafsky, and Andrew Y. Ng. 2016. Neural language correction with character-based attention. CoRR, abs/1603.09727. find
  29. Zheng Yuan and Ted Briscoe. 2016. Grammatical error correction using neural machine translation. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 380–386, San Diego, California. Association for Co…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/schmaltz-2017-adapting-sequence-models-sentence-correction/