Elamur Research⌕
☰
home / papers / 2011 / Learning to Simplify Sentences Using Wikipedia

Learning to Simplify Sentences Using Wikipedia

Summary

The paper treats sentence simplification as monolingual translation, where ordinary English is "translated" into simpler English. The training data is a corpus of 137K sentence pairs, built by matching English Wikipedia articles with their Simple English Wikipedia counterparts. Unlike sentence compression, which mostly removes words, these pairs show every kind of edit: rewording, reordering, insertion and deletion. The authors therefore start from a phrase-based MT system, Moses, and add explicit phrasal deletion. They do this by changing the GIZA++ word alignments so that unaligned source words map to NULL. The result is a phrase table that can drop whole phrases. The new system, Moses+Del, was scored against human reference simplifications using BLEU, word-F1 and SSA. It significantly outperformed two compression systems (T3 and a Knight & Marcu implementation) and plain Moses. This matters because earlier data-driven work (Zhu et al., 2010) improved only target-side measures such as readability and perplexity, not agreement with human references.

Key points

  • Corpus construction:
    • Articles are paired by exact title match, after removing stubs, disambiguation pages, meta-pages and one-line articles.
    • Paragraphs are paired by TF-IDF cosine similarity above a threshold.
    • Sentences are aligned with dynamic programming over five operations: insert, delete, 1–1, 2–1 and 1–2.
    • Only sentence pairs with similarity above 0.5 are kept.
  • Size: the 137K-pair corpus is larger than earlier simplification data. It is orders of magnitude larger than compression corpora, which hold at most a few thousand pairs.
  • Motivation for deletion: 47% of sentence pairs contain deletions. Standard phrase-based MT normally forbids empty target phrases, so it cannot learn these directly.
  • Controlled comparison: when trained only on T3's data (29,700 training pairs plus 300 for tuning), Moses+Del reached BLEU 0.5952. That is lower than with the full corpus but still significantly above T3.
  • Oracle analysis: the best translation from 1000-best lists raised BLEU from 0.5987 to 0.6317 for Moses and from 0.6046 to 0.6421 for Moses+Del.

Interesting ideas

  • I1. Adding new operations by editing alignments. Instead of rewriting the decoder, the authors post-process word alignments by attaching unaligned source spans to NULL. Standard phrase extraction then learns deletion rules on its own. This trick could add other operations to existing pipelines at low cost.
  • I2. Mining parallel data from comparable resources. A "simple" edition of an encyclopedia works as a noisy parallel corpus once it passes through title, paragraph and sentence alignment. The same idea fits any language that has comparable versions of the same content, such as plain-language or learner editions.
  • I3. A similarity cutoff instead of a tuned skip penalty. The authors keep alignments above a fixed TF-IDF cosine value rather than tuning insertion and deletion penalties in the dynamic programming. They found this more intuitive, and it makes the precision/recall trade-off of the corpus easy to control.
  • I4. Measuring reranking headroom with an n-best oracle. Picking the best candidate from the n-best list against the reference shows how much a reranker could gain. A more flexible model also produces more varied candidate lists, which this measure exposes. It is a cheap check for any generation system.
  • I5. Comparing on the competitor's training set. Retraining the new system on the baseline's smaller data separates the benefit of the method from the benefit of more data. This makes the comparison more honest.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Topics: Machine translation

Cited by

78 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 69 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{coster2011learning,
  title = {{Learning to Simplify Sentences Using Wikipedia}},
  author = {Coster, Will and Kauchak, David},
  booktitle = {Proceedings of the Workshop on Monolingual Text-To-Text Generation},
  year = {2011},
  url = {https://aclanthology.org/W11-1601/}
}

References

33 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Regina Barzilay and Noemie Elhadad. 2003. Sentence alignment for monolingual comparable corpora. In Proceedings of EMNLP. find
  2. Yvonne Canning, John Tait, Jackie Archibald, and Ros Crawley. 2000. Cohesive generation of syntactically simplified newspaper text. In Proceedings of TSD. find
  3. John Carroll, Gido Minnen, Yvonne Canning, Siobhan Devlin, and John Tait. 1998. Practical simplification of English newspaper text to assist aphasic readers. In Proceedings of AAAI Workshop on Integrating AI and Assistive Technology. find
  4. Raman Chandrasekar and Bangalore Srinivas. 1997. Automatic induction of rules for text simplification. In Knowledge Based Systems. ↗
  5. Yu Chen, Martin Kay, and Andreas Eisele. 2009. Intersecting multilingual data for faster and better statistical translations. In Proceedings of HLT/NAACL. find
  6. David Chiang. 2010. Learning to translate with source and target syntax. In Proceedings of ACL. find our card
  7. James Clarke and Mirella Lapata. 2006. Models for sentence compression: A comparison across domains, training requirements and evaluation measures. In Proceedings of ACL. find
  8. Trevor Cohn and Mirella Lapata. 2009. Sentence compression as tree transduction. Journal of Artificial Intelligence Research. ↗
  9. Will Coster and David Kauchak. 2011. Simple English Wikipedia: A new simplification task. In Proceedings of ACL (Short Paper). find
  10. Lijun Feng. 2008. Text simplification: A survey. CUNY Technical Report. find
  11. Michel Galley and Kathleen McKeown. 2007. Lexicalized Markov grammars for sentence compression. In Proceedings of HLT/NAACL. ↗
  12. Ruifang Ge and Raymond Mooney. 2006. Discriminative reranking for semantic parsing. In Proceedings of COLING. find
  13. Dan Klein and Christopher Manning. 2003. Accurate unlexicalized parsing. In Proceedings of ACL. find our card
  14. Kevin Knight and Daniel Marcu. 2002. Summarization beyond sentence extraction: A probabilistic approach to sentence compression. Artificial Intelligence. ↗
  15. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of ACL. find
  16. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of EMNLP. find
  17. Ryan McDonald. 2006. Discriminative sentence compression with soft syntactic evidence. In Proceedings of EACL. find
  18. Makoto Miwa, Rune Saetre, Yusuke Miyao, and Jun’ichi Tsujii. 2010. Entity-focused sentence simplication for relation extraction. In Proceedings of COLING. find
  19. Courtney Napoles and Mark Dredze. 2010. Learning simple Wikipedia: A cogitation in ascertaining abecedarian language. In Proceedings of HLT/NAACL Workshop on Computation Linguistics and Writing. find
  20. Rani Nelken and Stuart Shieber. 2006. Towards robust context-sensitive sentence alignment for monolingual corpora. In Proceedings of AMTA. find
  21. Tadashi Nomoto. 2007. Discriminative sentence compression with conditional random fields. In Information Processing and Management. ↗
  22. Tadashi Nomoto. 2008. A generic sentence trimmer with CRFs. In Proceedings of HLT/NAACL. find
  23. Tadashi Nomoto. 2009. A comparison of model free versus model intensive approaches to sentence compression. In Proceedings of EMNLP. find
  24. F.J. Och and H. Ney. 2000. Improved statistical alignment models. In Proceedings of ACL. find our card
  25. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. ↗
  26. Franz Josef Och, Kenji Yamada, Stanford U, Alex Fraser, Daniel Gildea, and Viren Jain. 2004. A smorgasbord of features for statistical machine translation. In Proceedings of HLT/NAACL. find
  27. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of ACL. find our card
  28. Jenine Turner and Eugene Charniak. 2005. Supervised and unsupervised learning for sentence compression. In Proceedings of ACL. find
  29. David Vickrey and Daphne Koller. 2008. Sentence simplification for semantic role labeling. In Proceedings of ACL. find
  30. S. Wubben, A. van den Bosch, and E. Krahmer. 2010. Paraphrase generation as monolingual translation: Data and evaluation. In Proceedings of the International Workshop on Natural Language Generation. find
  31. Elif Yamangil and Rani Nelken. 2008. Mining Wikipedia revision histories for improving sentence compression. In ACL. find
  32. Mark Yatskar, Bo Pang, Critian Danescu-Niculescu-Mizil, and Lillian Lee. 2010. For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia. In Proceedings of HLT/NAACL (Short Paper). ↗
  33. Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010. A monolingual tree-based translation model for sentence simplification. In Proceedings of COLING. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/coster-2011-learning-simplify-sentences-wikipedia/