Elamur Research
☰
home / papers / 2012 / Sentence Simplification by Monolingual Machine Translation

Sentence Simplification by Monolingual Machine Translation

What it is about

The paper simplifies sentences with an off-the-shelf phrase-based statistical MT system (Moses), trained on aligned Wikipedia and Simple Wikipedia sentences. The one addition is a re-ranking of the ten best outputs by dissimilarity to the input, measured by Levenshtein distance, so that the system does not return the input unchanged. It is compared with a word-substitution baseline and with the Zhu et al. (2010) system and the ILP system of Woodsend and Lapata (2011). Human judges rate fluency and adequacy better than the other systems, and simplicity not significantly different from the best systems. The output is conservative: few edits, no sentence splits. The authors argue that the Flesch-Kincaid grade level should be used with caution, because it correlates poorly with human judgements, while BLEU correlates with fluency and adequacy.

Ideas

For Mova

Re-ranking the n-best by dissimilarity is a small, deterministic change in the translator's output stage. The phrase-based MT system is not for Mova. The n-best sweep (I2) and the caution about readability scores fit tools/translate/measure.rs. Relevant to the normal-mode scoring question and to open problem 3.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/P12-1107.pdf

Summary

The paper treats sentence simplification as translation from English into simpler English. The authors train a standard phrase-based MT system (Moses) on PWKP, a corpus of sentence pairs automatically aligned between English Wikipedia and Simple Wikipedia. Plain PBMT on such data tends to copy its input, so they add one step: among the 10 best decoder outputs, they select a candidate that departs from the source sentence. They call the resulting system PBMT-R. Human judges rated Simplicity, Fluency and Adequacy for five outputs: PBMT-R, a WordNet-based word-substitution baseline, the syntax-based Zhu system, Woodsend and Lapata's RevILP, and the original Simple Wikipedia sentences. PBMT-R simplified about as well as the stronger systems. It was judged significantly more fluent and more faithful to the meaning, while making only modest phrase-level edits. The authors also find that readability formulas such as Flesch-Kincaid can reward broken output. They therefore recommend reference-based metrics like BLEU over readability scores for evaluating simplification systems.

Key points

  • Data: PWKP (Zhu et al., 2010) has 108,016 aligned sentence pairs from 65,133 article pairs, aligned with sentence-level TF*IDF.
    • Simple Wikipedia sentences average 20.87 tokens versus 25.01 for Wikipedia, about 17% shorter.
    • Average word length barely differs (4.89 vs 5.06 characters).
  • Pipeline: Moses with GIZA++ word alignment and an SRILM n-gram language model trained only on the Simple Wikipedia side.
    • The re-ranking step operates on the 10-best list.
    • Levenshtein distance was used to study how varying n affects the output.
  • Baseline: replaces each noun, verb and adjective with the WordNet synonym the Simple Wikipedia language model scores highest. POS tags come from the Memory-Based Tagger trained on the Brown corpus.
  • Automatic metrics: RevILP and PBMT-R obtained the highest BLEU, while Zhu scored best on Flesch-Kincaid, mostly because it often splits sentences.
  • Correlations with human judgements:
    • Flesch-Kincaid correlated negatively with Simplicity (−0.45) but positively with Fluency and Adequacy.
    • BLEU correlated with Simplicity (0.42) and Fluency (0.26) but showed no significant link to Adequacy.

Interesting ideas

  • Dissimilarity re-ranking. A monolingual MT model trained on "near-copy" data tends to reproduce its input. Choosing among n-best outputs by how different they are from the source pushes the system to actually rewrite, without retraining. The same trick could serve paraphrasing, style transfer, or normalization for Ukrainian, where parallel data is also noisy and mostly identical on both sides.
  • Language model trained on the target register only. Training the LM on the simple side of the corpus makes it act as a style filter for both the MT system and the synonym-substitution baseline. This is a cheap way to steer any generator toward a desired register.
  • Simplicity is not the same as shortness. The method rephrases rather than deletes or splits, which kept meaning and grammaticality intact. This is a useful design principle for accessibility-oriented text tools, where dropped content is costly.
  • Distrust surface readability formulas. Randomly splitting sentences would improve Flesch-Kincaid while ruining the text. Any evaluation of simplification, or of readability-targeted generation, should pair reference-based metrics with human judgements of Fluency, Adequacy and Simplicity rather than rely on length-based scores.
  • Include the gold reference and a conservative baseline in human evaluation. Rating the original Simple Wikipedia sentences and a word-substitution baseline alongside the systems calibrates the scores. The gold sentences show what "good" looks like, and the cautious baseline shows how much credit a minimal-change system earns.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{wubben2012sentence,
  title = {{Sentence Simplification by Monolingual Machine Translation}},
  author = {Wubben, Sander and Bosch, Antal van den and Krahmer, Emiel},
  booktitle = {Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2012},
  url = {https://aclanthology.org/P12-1107/}
}

References

34 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In ACL ’05: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 597–604, Morristown, NJ, USA. Association for Computational Linguistics. find our card
  2. Chris Callison-Burch. 2008. Syntactic constraints on paraphrases extracted from parallel corpora. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’08, pages 196–205, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  3. Yvonne Canning, John Tait, Jackie Archibald, and Ros Crawley. 2000. Cohesive regeneration of syntactically simplified newspaper text. In Proceedings of RO-MAND 2000, Lausanne. find
  4. John Carroll, Guido Minnen, Yvonne Canning, Siobhan Devlin, and John Tait. 1998. Practical simplification of English newspaper text to assist aphasic readers. In AAAI-98 Workshop on Integrating Artificial Intelligence and Assistive Technology, Madison, Wisconsin. find
  5. John Carroll, Guido Minnen, Darren Pearce, Yvonne Canning, Siobhan Devlin, and John Tait. 1999. Simplifying text for language-impaired readers. In Proceedings of EACL’99, Bergen. ACL. find
  6. R. Chandrasekar and B. Srinivas. 1997. Automatic rules for text simplification. Knowledge-Based Systems, 10:183–190. find
  7. Raman Chandrasekar, Christine Doran, and Bangalore Srinivas. 1996. Motivations and methods for text simplification. In Proceedings of the Sixteenth International Conference on Computational Linguistics (COLING’96), pages 1041–1044. find
  8. David Chiang, Adam Lopez, Nitin Madnani, Christof Monz, Philip Resnik, and Michael Subotin. 2005. The hiero machine translation system: extensions, evaluation, and analysis. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT ’05, pages 779– 786, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  9. Will Coster and David Kauchak. 2011. Learning to simplify sentences using wikipedia. In Proceedings of the Workshop on Monolingual Text-To-Text Generation, pages 1–9, Portland, Oregon, June. Association for Computational Linguistics. find
  10. Walter Daelemans, Jakub Zavrel, Peter Berck, and Steven Gillis. 1996. MBT: A Memory-Based Part of Speech Tagger-Generator. In Proc. of Fourth Workshop on Very Large Corpora, pages 14–27. ACL SIGDAT. find
  11. Walter Daelemans, Anja Hothker, and Erik Tjong Kim Sang. 2004. Automatic sentence simplification for subtitling in dutch and english. In Proceedings of the 4th International Conference on Language Resources and Evaluation, pages 1045–1048. find
  12. George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second international conference on Human Language Technology Research, HLT ’02, pages 138–145, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. find
  13. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. The MIT Press, May. find
  14. Katja Filippova and Michael Strube. 2008. Sentence fusion via dependency graph compression. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 177–185, Honolulu, Hawaii, October. Association for Computational Linguistics. find
  15. Kentaro Inui, Atsushi Fujita, Tetsuro Takahashi, Ryu Iida, and Tomoya Iwakura. 2003. Text simplification for reading assistance: A project note. In Proceedings of the Second International Workshop on Paraphrasing, pages 9–16, Sapporo, Japan, July. Association for Computational Linguistics. find
  16. Kevin Knight and Daniel Marcu. 2000. Statistics-based summarization – step one: Sentence compression. In Proceedings of the 17th National Conference on Artificial Intelligence (AAAI), pages 703 – 710, Austin, Texas, USA, July 30 – August 3. find
  17. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris C. Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In ACL. The Association for Computer Linguistics. find
  18. V. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8):707–710. find
  19. Zhifei Li, Chris Callison-Burch, Chris Dyer, Juri Ganitkevitch, Sanjeev Khudanpur, Lane Schwartz, Wren N. G. Thornton, Jonathan Weese, and Omar F. Zaidan. 2009. Joshua: an open source toolkit for parsing-based machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 135–139, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  20. Nitin Madnani, Necip Fazil Ayan, Philip Resnik, and Bonnie J. Dorr. 2007. Using paraphrases for parameter tuning in statistical machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation, StatMT ’07, pages 120–127, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  21. Rani Nelken and Stuart M. Shieber. 2006. Towards robust context-sensitive sentence alignment for monolingual corpora. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL-06), Trento, Italy, 3–7 April. find
  22. Franz J. Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Comput. Linguist., 29(1):19–51, March. find
  23. Chris Quirk, Chris Brockett, and William Dolan. 2004. Monolingual machine translation for paraphrase generation. In Dekang Lin and Dekai Wu, editors, Proceedings of EMNLP 2004, pages 142–149, Barcelona, Spain, July. Association for Computational Linguistics. find our card
  24. Advaith Siddharthan. 2002. An architecture for a text simplification system. In Language Engineering Conference, page 64. IEEE Computer Society. find
  25. David A. Smith and Jason Eisner. 2006. Quasi-synchronous grammars: Alignment by soft projection of syntactic dependencies. In Proceedings of the HLT-NAACL Workshop on Statistical Machine Translation, pages 23–30, New York, June. find our card
  26. Andreas Stolcke. 2002. SRILM - An Extensible Language Modeling Toolkit. In In Proc. Int. Conf. on Spoken Language Processing, pages 901–904, Denver, Colorado. find
  27. D. Vickrey and D. Koller. 2008. Sentence simplification for semantic role labeling. In Proceedings of the 46th Meeting of the Association for Computational Linguistics: Human Language Technologies. find
  28. Willian Massami Watanabe, Arnaldo Candido Junior, Vincius Rodriguez de Uzłda, Renata Pontin de Mattos Fortes, Thiago Alexandre Salgueiro Pardo, and Sandra M. Alusio. 2009. Facilita: reading assistance for low-literacy readers. In Brad Mehlenbacher, Aristidis Protopsaltis, Ashley Williams, and Shaun Slattery, editors, SIGDOC, pages 29–36. ACM. find
  29. Kristian Woodsend and Mirella Lapata. 2011. Learning to simplify sentences with quasi-synchronous grammar and integer programming. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 409–420, Edinburgh, Scotland, UK., July. Association for Computational Linguistics. find our card
  30. Sander Wubben, Antal van den Bosch, and Emiel Krahmer. 2010. Paraphrase generation as monolingual translation: data and evaluation. In Proceedings of the 6th International Natural Language Generation Conference, INLG ’10, pages 203–207, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  31. Kenji Yamada and Kevin Knight. 2001. A syntax-based statistical translation model. In Proceedings of the 39th Annual Meeting on Association for Computational Linguistics, ACL ’01, pages 523–530, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  32. Mark Yatskar, Bo Pang, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2010. For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia. In Proceedings of the NAACL, pages 365–368. find
  33. Shiqi Zhao, Xiang Lan, Ting Liu, and Sheng Li. 2009. Application-driven statistical paraphrase generation. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2, ACL ’09, pages 834–842, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  34. Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010. A monolingual tree-based translation model for sentence simplification. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), pages 1353–1361, Beijing, China, August. Coling 2010 Organizing Committee. find our card