Elamur Research
☰
home / papers / 2017 / A Deep Reinforced Model for Abstractive Summarization

A Deep Reinforced Model for Abstractive Summarization

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1705.04304

Summary

Attention-based encoder-decoder summarizers work well on short texts. On long news articles with multi-sentence summaries, though, they tend to repeat phrases and lose coherence. The authors, from Salesforce Research, tackle this with changes to both the architecture and the training. The model uses a bi-directional LSTM encoder and an LSTM decoder. Intra-temporal attention lowers the weight of input tokens that earlier steps already focused on, and a new intra-decoder attention lets the decoder look back at what it has already produced. A pointer switch copies rare words and named entities from the input, and beam search blocks any repeated trigram. For training, the standard maximum-likelihood (teacher forcing) loss is mixed with a self-critical policy-gradient objective that rewards ROUGE. This reduces exposure bias while keeping the output fluent. The model was evaluated on CNN/Daily Mail and on the New York Times corpus, which the authors say is the first end-to-end abstractive model on that dataset. It scores 41.16 ROUGE-1 on CNN/Daily Mail, above the earlier state of the art, and human judges found its summaries more readable. The work shows that combining metric-driven RL with supervised training makes long-form abstractive summarization practical.

Key points

  • Main result: 41.16 ROUGE-1 on CNN/Daily Mail, the best reported at the time, and human evaluation also favoured its readability over other abstractive systems.
  • Hybrid objective: ML and RL losses are combined with weight γ = 0.9984. The RL part follows Rennie et al. (2016), using a greedily decoded output as the baseline. During ML training, the model's own previous prediction replaces the ground-truth input token 25% of the time.
  • NYT dataset setup: the data is split by publication date into 589,284 training, 32,736 validation and 32,739 test examples. After capping inputs at 800 tokens and outputs at 100, examples average 549 input and 40 output tokens. Pointer supervision comes from the Stanford NER tagger, which is needed only to build the data.
  • Architecture details: the encoder has a 200-d LSTM in each direction and the decoder is a 400-d LSTM. Input embeddings are 100-d and initialized with GloVe. Vocabularies are 150k tokens for input and 50k for output, and the embedding matrix is shared with the output projection.
  • Inference: beam search of width 5, with any repeated trigram blocked. This rule is based on the observation that reference summaries almost never repeat a trigram.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

489 papers of our library cite it (of the 140673 whose bibliographies we read); 40 of them have a page here, 449 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1705.04304 · arXiv 1705.04304

@misc{paulus2017deep,
  title = {{A Deep Reinforced Model for Abstractive Summarization}},
  author = {Paulus, Romain and Xiong, Caiming and Socher, Richard},
  year = {2017},
  doi = {10.48550/arxiv.1705.04304},
  eprint = {1705.04304},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1705.04304}
}

References

39 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. find our card
  2. Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, and Hui Jiang. Distraction-based neural networks for modeling documents. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), pp. 2754–2760, 2016. find
  3. Jianpeng Cheng, Li Dong, and Mirella Lapata. Long short-term memory-networks for machine reading. arXiv preprint arXiv:1601.06733, 2016. find our card
  4. Sumit Chopra, Michael Auli, Alexander M Rush, and SEAS Harvard. Abstractive sentence summarization with attentive recurrent neural networks. Proceedings of NAACL-HLT16, pp. 93–98, 2016. find our card
  5. Readability 6.76 4.18 7.04 Relevance 7.14 6.32 7.45 Bonnie Dorr, David Zajic, and Richard Schwartz. Hedge trimmer: A parse-and-trim approach to headline generation. In Proceedings of the HLT-NAACL 03 on Text summarization workshop-Volume 5, pp. 1–8. Association for Computational Linguistics, 2003. find
  6. Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein. Learning-based single-document summarization with compression and anaphoricity constraints. arXiv preprint arXiv:1603.08887, 2016. find
  7. Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. Pointing the unknown words. arXiv preprint arXiv:1603.08148, 2016. find
  8. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems, pp. 1693–1701, 2015. find our card
  9. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735–1780, 1997. find
  10. Kai Hong and Ani Nenkova. Improving the estimation of word importance for news multi-document summarization-extended technical report. 2014. find
  11. Kai Hong, Mitchell Marcus, and Ani Nenkova. System combination for multi-document summarization. In EMNLP, pp. 107–117, 2015. find
  12. Hakan Inan, Khashayar Khosravi, and Richard Socher. Tying word vectors and word classifiers: A loss framework for language modeling. Proceedings of the International Conference on Learning Representations, 2017. find
  13. Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. find
  14. Junyi Jessy Li, Kapil Thadani, and Amanda Stent. The role of discourse units in near-extractive summarization. In 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 137, 2016. find
  15. Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out: Proceedings of the ACL-04 workshop, volume 8. Barcelona, Spain, 2004. find
  16. Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023, 2016. find
  17. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. The stanford corenlp natural language processing toolkit. In ACL (System Demonstrations), pp. 55–60, 2014. find our card
  18. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. Proceedings of the International Conference on Learning Representations, 2017. find our card
  19. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pp. 3111–3119, 2013. find
  20. Ramesh Nallapati, Bowen Zhou, Çağlar Gülçehre, Bing Xiang, et al. Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023, 2016. find
  21. Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. Proceedings of the 31st AAAI conference, 2017. find our card
  22. Mohammad Norouzi, Samy Bengio, Navdeep Jaitly, Mike Schuster, Yonghui Wu, Dale Schuurmans, et al. Reward augmented maximum likelihood for neural structured prediction. In Advances In Neural Information Processing Systems, pp. 1723–1731, 2016. find
  23. Benjamin Nye and Ani Nenkova. Identification and characterization of newsworthy verbs in world news. In HLT-NAACL, pp. 1440–1445, 2015. find
  24. Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, volume 14, pp. 1532–1543, 2014. find
  25. Ofir Press and Lior Wolf. Using the output embedding to improve language models. arXiv preprint arXiv:1608.05859, 2016. find our card
  26. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. Sequence level training with recurrent neural networks. arXiv preprint arXiv:1511.06732, 2015. find our card
  27. Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. Self-critical sequence training for image captioning. arXiv preprint arXiv:1612.00563, 2016. find our card
  28. Alexander M Rush, Sumit Chopra, and Jason Weston. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685, 2015. find
  29. Evan Sandhaus. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia, 6(12):e26752, 2008. find
  30. Baskaran Sankaran, Haitao Mi, Yaser Al-Onaizan, and Abe Ittycheriah. Temporal attention model for neural machine translation. arXiv preprint arXiv:1608.02927, 2016. — [arXiv:1608.02927](https://arxiv.org/abs/1608.02927) find
  31. Abigail See, Peter J. Liu, and Christopher D. Manning. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1073–1083, July 2017. find
  32. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp. 3104–3112, 2014. find
  33. Arun Venkatraman, Martial Hebert, and J Andrew Bagnell. Improving multi-step prediction of learned time series models. In AAAI, pp. 3024–3030, 2015. find
  34. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In Advances in Neural Information Processing Systems, pp. 2692–2700, 2015. find
  35. Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229–256, 1992. find
  36. Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1(2):270–280, 1989. find
  37. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016. find
  38. Yinfei Yang and Ani Nenkova. Detecting information-dense texts in multiple news domains. In AAAI, pp. 1650–1656, 2014. find
  39. Wenyuan Zeng, Wenjie Luo, Sanja Fidler, and Raquel Urtasun. Efficient summarization with read-again and copy mechanism. arXiv preprint arXiv:1611.03382, 2016. — [arXiv:1611.03382](https://arxiv.org/abs/1611.03382) find