Elamur Research
☰
home / papers / 2017 / Sentence Simplification with Deep Reinforcement Learning

Sentence Simplification with Deep Reinforcement Learning

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D17-1062.pdf

Summary

Zhang and Lapata address sentence simplification: rewriting a sentence so it is easier to read while keeping its meaning. Standard attention-based encoder-decoder models tend to copy the input almost unchanged, because most target words in simplification corpora also appear in the source. The authors propose DRESS (Deep REinforcement Sentence Simplification), an LSTM encoder-decoder treated as an agent and trained with the REINFORCE algorithm. The reward combines three signals:

  • Simplicity: based on SARI, applied in both the normal and the reversed direction to reduce the effect of noisy single references.
  • Relevance: measures whether the meaning of the source is preserved.
  • Fluency: derived from a language model trained on simple sentences.

A variant, DRESS-LS, adds a lexical simplification component. The models were evaluated on Newsela, WikiSmall and WikiLarge with automatic metrics (BLEU, SARI, FKGL) and human ratings. The RL-trained models produce noticeably simpler output than a plain encoder-decoder and statistical baselines, and they remain fluent. The work shows that reinforcement learning is an effective way to build task-specific knowledge into neural text generation.

Key points

  • Copying dominates the data: about 73% of target words come from the source in Newsela, and about 83% in the Wikipedia-based datasets. This explains why a vanilla EncDecA mostly repeats its input.
  • The reward is a weighted sum λ_S·r_S + λ_R·r_R + λ_F·r_F. The simplicity term mixes SARI(X, Ŷ, Y) and reverse SARI with a weight β.
  • On the Newsela test set, DRESS shortens sentences to 14.2 tokens on average, compared with 23.1 for PBMT-R and 12.7 for the references. Its TER of 0.46 indicates substantial rewriting, mainly through deletions.
  • In human evaluation, DRESS-LS and DRESS were significantly simpler (p < 0.01) than EncDecA, PBMT-R and Hybrid on Newsela. On WikiLarge, which has 8 references per test sentence, DRESS-LS significantly beat all comparison systems on the combined rating.
  • Code and data are released at github.com/XingxingZhang/dress. Future directions include explicit sentence splitting and simplification of whole documents.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation

Cited by

236 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 225 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d17-1062

@inproceedings{zhang2017sentence,
  title = {{Sentence Simplification with Deep Reinforcement Learning}},
  author = {Zhang, Xingxing and Lapata, Mirella},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1062},
  url = {https://doi.org/10.18653/v1/d17-1062}
}

References

49 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sébastien Jean, Orhan Firat, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. Montreal neural machine translation systems for WMT15. In Proceedings of the 10th Workshop on Statistical Machine Translation, pages 134–140, Lisbon, Portugal. find
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR, San Diego, CA. find our card
  3. Nobuhiro Kaji, Daisuke Kawahara, Sadao Kurohashi, and Satoshi Sato. 2002. Verb paraphrase based on case frame alignment. In Proceedings of 40th ACL, pages 215–222, Philadelphia, PA. find
  4. Beata Beigman Klebanov, Kevin Knight, and Daniel Marcu. 2004. Text simplification for information-seeking applications. In Proceedings of ODBASE, volume 3290 of Lecture Notes in Computer Science, pages 735–747, Agia Napa, Cyprus. Springer. find
  5. David Kauchak. 2013. Improving text simplification language modeling using unsimplified text data. In Proceedings of the 51st ACL, pages 1537–1546, Sofia, Bulgaria. find
  6. J. Carroll, G. Minnen, D. Pearce, Y. Canning, S. Devlin, and J Tait. 1999. Simplifying text for language-impaired readers. In Proceedings of the 9th EACL, pages 269–270, Bergen, Norway. find
  7. J Peter Kincaid, Robert P Fishburne Jr, Richard L Rogers, and Brad S Chissom. 1975. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Technical report, DTIC Document. find
  8. R. Chandrasekar, C. Doran, and B. Srinivas. 1996. Motivations and methods for text simplification. In Proceedings of the 16th COLING, pages 1041–1044, Copenhagen, Denmark. find
  9. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  10. Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of NAACL: HLT, pages 93–98, San Diego, CA. find our card
  11. Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016. Deep reinforcement learning for dialogue generation. In Proceedings of the 2016 EMNLP, pages 1192–1202, Austin, TX. find our card
  12. James Curran, Stephen Clark, and Johan Bos. 2007. Linguistically motivated large-scale nlp with c&c and boxer. In Proceedings of the 45th ACL Companion Volume Proceedings of the Demo and Poster Sessions, pages 33–36, Prague, Czech Republic. find
  13. Wang Ling, Phil Blunsom, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, Fumin Wang, and Andrew Senior. 2016. Latent predictor networks for code generation. In Proceedings of the 54th ACL, pages 599–609, Berlin, Germany. find
  14. Andrew M Dai and Quoc V Le. 2015. Semi-supervised sequence learning. In Advances in Neural Information Processing Systems, pages 3079–3087. find
  15. Siobhan Devlin. 1999. Simplifying Natural Language for Aphasic Readers. Ph.D. thesis, University of Sunderland. find
  16. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 EMNLP, pages 1412–1421, Lisbon, Portugal. find
  17. Richard Evans, Constantin Or asan, and Iustin Dornescu. 2014. An evaluation of syntactic simplification rules for people with autism. In Proceedings of the 3rd Workshop on Predicting and Improving Text Readability for Target Reader Populations (PITR), pages 131–140, Gothenburg, Sweden. find
  18. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In ACL System Demonstrations, pages 55–60. find our card
  19. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The paraphrase database. In Proceedings of NAACL-HLT, pages 758–764, Atlanta, GA. find our card
  20. Karthik Narasimhan, Adam Yala, and Regina Barzilay. 2016. Improving information extraction by acquiring external evidence with reinforcement learning. In Proceedings of the 2016 EMNLP, pages 2355– 2365, Austin, TX. find
  21. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. ↗
  22. Shashi Narayan and Claire Gardent. 2014. Hybrid simplification using deep semantics and machine translation. In Proceedings of the 52nd ACL, pages 435– 445, Baltimore, MD. find
  23. Kentaro Inui, Atsushi Fujita, Tetsuro Takahashi, Ryu Iida, and Tomoya Iwakura. 2003. Text simplification for reading assistance: A project note. In Proceedings of the 2nd International Workshop on Paraphrasing, pages 9–16, Sapporo, Japan. find
  24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th ACL, pages 311–318, Philadelphia, PA. find our card
  25. Willian Massami Watanabe, Arnaldo Candido Junior, Vinı́cius Rodriguez de Uzẽda, Renata Pontin de Mattos Fortes, Thiago Alexandre Salgueiro Pardo, and Sandra Maria Alu´sio. 2009. Facilita: reading assistance for low-literacy readers. In Proceedings of the 27th ACM International Conference on Design of Communication, Bloomington, IN. find
  26. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. ICML (3), 28:1310–1318. find
  27. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In In Proceedings of the EMNLP 2014, volume 14, pages 1532–43, Doha, Qatar. find
  28. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229–256. find
  29. MarcAurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence level training with recurrent neural networks. In Proceedings of ICLR, San Juan, Puerto Rico. find our card
  30. Kristian Woodsend and Mirella Lapata. 2011. Learning to simplify sentences with quasi-synchronous grammar and integer programming. In Proceedings of the 2011 EMNLP, pages 409–420, Edinburgh, Scotland. find our card
  31. Luz Rello, Clara Bayarri, Azuki Górriz, Ricardo Baeza-Yates, Saurabh Gupta, Gaurang Kanvinde, Horacio Saggion, Stefan Bott, Roberto Carlini, and Vasile Topac. 2013. Dyswebxia 2.0!: More accessible text for people with dyslexia. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility, pages –, Brazil. find
  32. Kristian Woodsend and Mirella Lapata. 2014. Text rewriting improves semantic role labeling. Journal of Artificial Intelligence Research, 51:133–164. find
  33. Seonggi Ryang and Takeshi Abekawa. 2012. Framework of automatic text summarization using reinforcement learning. In Proceedings of the 2012 EMNLP-CoNLL, pages 256–265, Jeju Island, Korea. find
  34. Sander Wubben, Antal Van Den Bosch, and Emiel Krahmer. 2012. Sentence simplification by monolingual machine translation. In Proceedings of the 50th ACL, pages 1015–1024, Jeju Island, Korea. find our card
  35. Matthew Shardlow. 2014. A survey of automated text simplification. International Journal of Advanced Computer Science and Applications, pages 581– 701. Special Issue on Natural Language Processing. find
  36. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard S Zemel, and Yoshua Bengio. 2015a. Show, attend and tell: Neural image caption generation with visual attention. arXiv preprint arXiv:1502.03044, 2(3):5. ↗ our card
  37. Advaith Siddharthan. 2004. Syntactic simplification and text cohesion. in research on language and computation. Research on Language and Computation, 4(1):77–109. find
  38. Wei Xu, Chris Callison-Burch, and Courtney Napoles. 2015b. Problems in current text simplification research: New data can help. Transactions of the Association for Computational Linguistics, 3:283–297. find our card
  39. Advaith Siddharthan. 2014. A survey of research on text simplification. International Journal of Applied Linguistics, 165(2):259–298. find
  40. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing statistical machine translation for text simplification. Transactions of the Association for Computational Linguistics, 4:401–415. find our card
  41. David A Smith and Jason Eisner. 2006. Quasi-synchronous grammars: Alignment by soft projection of syntactic dependencies. In Proceedings of the NAACL 206 Workshop on Statistical Machine Translation, pages 23–30, New York City. find our card
  42. Kenji Yamada and Kevin Knight. 2001. A syntax-based statistical translation model. In Proceedings of the 39th ACL, pages 523–530, Toulouse, France. find our card
  43. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of association for machine translation in the Americas, volume 200. find our card
  44. Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329. ↗ our card
  45. Lucia Specia, Sujay Kumar Jauhar, and Rada Mihalcea. 2012. Semeval-2012 task 1: English lexical simplification. In In Proceedings of *SEM 2012, pages 347–355, Montréal, Canada. find
  46. Xingxing Zhang and Mirella Lapata. 2014. Chinese poetry generation with recurrent neural networks. In Proceedings of the 2014 EMNLP, pages 670–680, Doha, Qatar. find
  47. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112. find
  48. Zhemin Zhu, Delphine Bernhard, and Iryna Gurevych. 2010. A monolingual tree-based translation model for sentence simplification. In Proceedings of the 23rd COLING, pages 1353–1361, Beijing, China. find our card
  49. D. Vickrey and D. Koller. 2008. Sentence simplification for semantic role labeling. In Proceedings of ACL-08: HLT, pages 344–352, Columbus, OH. find