Elamur Research
☰
home / papers / 2016 / Neural Summarization by Extracting Sentences and Words

Neural Summarization by Extracting Sentences and Words

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P16-1046.pdf

Summary

Extractive summarizers have traditionally depended on hand-crafted cues such as sentence position, length and word frequency. Cheng and Lapata replace these cues with features learned by neural networks. Their single-document framework pairs a hierarchical document reader with an attention-based extractor. The reader builds sentence representations from words using convolution and then composes them into a document representation. The extractor uses attention directly to pick output units, in the spirit of Pointer Networks, rather than to feed a decoder. Two variants follow from this design: NN-SE, which labels sentences, and NN-WE, which generates summaries using only words found in the source document.

Neural training needs far more labeled data than earlier corpora offered, so the authors built large training sets automatically from DailyMail articles and their editor-written highlights. They evaluated on DUC 2002 and on DailyMail with ROUGE and human judges. NN-SE matched state-of-the-art systems that rely on linguistic constraints, and it did so without any linguistic annotation. This shows that summarization can be learned end-to-end when enough data is available.

Key points

  • Sentence-extraction dataset: A rule-based labeler matched document sentences to highlights. It used sentence position, unigram/bigram overlap and entity counts, with weights tuned on 9,000 manually labeled documents from Woodsend and Lapata (2010). It reached 85% accuracy on 216 held-out documents and was then applied to about 200K DailyMail documents, marking roughly 30% of sentences as summary-worthy.
  • Word-extraction dataset: This set has 170K DailyMail articles. A pair was kept only if all highlight words, after stemming, appear in the article. Out-of-vocabulary words could be swapped for an embedding-space neighbor found in the article; pairs with no such substitute were discarded.
  • Evaluation data: Systems were tested on the DUC 2002 corpus and on 500 DailyMail test articles. NN-SE achieved the best ROUGE scores on both. Scores on DailyMail were lower overall because its highlights are terse and heavily paraphrased.
  • Human evaluation: Judges on Amazon Mechanical Turk ranked summaries for informativeness and fluency. Human summaries were ranked 1st 27% of the time and NN-SE 22%, while ILP was most often ranked 2nd (38%).
  • Significance: NN-SE and ILP were significantly better (p < 0.01) than LEAD, NN-WE and NN-ABS. They did not differ significantly from each other or from the human gold standard.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

210 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 200 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-1046

@inproceedings{cheng2016neural,
  title = {{Neural Summarization by Extracting Sentences and Words}},
  author = {Cheng, Jianpeng and Lapata, Mirella},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1046},
  url = {https://doi.org/10.18653/v1/p16-1046}
}

References

36 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR 2015, San Diego, California. find our card
  3. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent convolutional neural networks for discourse compositionality. In Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality, pages 119–126, Sofia, Bulgaria. find our card
  4. Michele Banko, Vibhu O. Mittal, and Michael J. Witbrock. 2000. Headline generation based on statistical translation. In Proceedings of the 38th ACL, pages 318–325, Hong Kong. find
  5. Yoon Kim, Yacine Jernite, David Sontag, and Alexander M Rush. 2016. Character-aware neural language models. In Proceedings of the 30th AAAI, Phoenix, Arizon. to appear. find our card
  6. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems 28, pages 1171–1179. Curran Associates, Inc. find
  7. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 EMNLP, pages 1746–1751, Doha, Qatar. find our card
  8. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2014. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005. find our card
  9. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. find
  10. Hayato Kobayashi, Masaki Noguchi, and Taichi Yatsuka. 2015. Summarization based on embedding distributions. In Proceedings of the 2015 EMNLP, pages 1984–1989, Lisbon, Portugal. find
  11. Trevor Anthony Cohn and Mirella Lapata. 2009. Sentence compression as tree transduction. Journal of Artificial Intelligence Research, pages 637–674. find
  12. Conroy and O’Leary. 2001. Text summarization via hidden Markov models. In Proceedings of the 34th Annual ACL SIGIR, pages 406–407, New Oleans, Louisiana. find
  13. Julian Kupiec, Jan O. Pedersen, and Francine Chen. 1995. A trainable document summarizer. In Proceedings of the 18th Annual International ACM SI-GIR, pages 68–73, Seattle, Washington. find
  14. Güneş Erkan and Dragomir R. Radev. 2004. Lexpagerank: Prestige in multi-document text summarization. In Proceedings of the 2004 EMNLP, pages 365–371, Barcelona, Spain. find
  15. Rémi Lebret, Pedro O Pinheiro, and Ronan Collobert. 2015. Phrase-based image captioning. In Proceedings of the 32nd ICML, Lille, France. find
  16. Elena Filatova and Vasileios Hatzivassiloglou. 2004. Event-based extractive summarization. In Stan Szpakowicz Marie-Francine Moens, editor, Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 104–111, Barcelona, Spain. find
  17. Chin-Yew Lin and Eduard H. Hovy. 2003. Automatic evaluation of summaries using n-gram co-occurrence statistics. In Proceedings of HLT NAACL, pages 71–78, Edmonton, Canada. find
  18. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of the 54th ACL, Berlin, Germany. to appear. find our card
  19. Rada Mihalcea. 2005. Language independent extractive summarization. In Proceedings of the ACL Interactive Poster and Demonstration Sessions, pages 49–52, Ann Arbor, Michigan. find
  20. Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loic Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2015. On using monolingual corpora in neural machine translation. arXiv preprint arXiv:1503.03535. find our card
  21. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems 26, pages 3111–3119. Curran Associates, Inc. find
  22. Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. 2016. Pointing the unknown words. In Proceedings of the 54th ACL, Berlin, Germany. to appear. find
  23. Ani Nenkova, Lucy Vanderwende, and Kathleen McKeown. 2006. A compositional context sensitive multi-document summarizer: exploring the factors that influence summarization. In Proceedings of the 29th Annual ACM SIGIR, pages 573–580, Washington, Seattle. find
  24. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems 28, pages 1684– 1692. Curran Associates, Inc. find our card
  25. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st ACL, pages 160–167, Sapporo, Japan. find
  26. Daraksha Parveen, Hans-Martin Ramsl, and Michael Strube. 2015. Topical coherence for graph-based extractive summarization. In Proceedings of the 2015 EMNLP, pages 1949–1954, Lisbon, Portugal, September. find
  27. Dragomir Radev, Timothy Allison, Sasha Blair-Goldensohn, John Blitzer, Arda Celebi, Stanko Dimitrov, Elliott Drabek, Ali Hakim, Wai Lam, Danyu Liu, et al. 2004. Mead-a platform for multidocument multilingual text summarization. Technical report, Columbia University Academic Commons. find
  28. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 EMNLP, pages 379–389, Lisbon, Portugal. find
  29. Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27, pages 3104–3112. Curran Associates, Inc. find
  30. Krysta Svore, Lucy Vanderwende, and Christopher Burges. 2007. Enhancing single-document summarization by combining RankNet and third-party sources. In Proceedings of the 2007 EMNLP-CoNLL, pages 448–457, Prague, Czech Republic. find
  31. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Advances in Neural Information Processing Systems 28, pages 2674–2682. Curran Associates, Inc. find
  32. Xiaojun Wan. 2010. Towards a unified approach to simultaneous single-document and multi-document summarizations. In Proceedings of the 23rd COL-ING, pages 1137–1145. find
  33. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229–256. find
  34. Kristian Woodsend and Mirella Lapata. 2010. Automatic generation of story highlights. In Proceedings of the 48th ACL, pages 565–574, Uppsala, Sweden. find
  35. Dani Yogatama, Fei Liu, and Noah A. Smith. 2015. Extractive summarization by maximizing semantic volume. In Proceedings of the 2015 EMNLP, pages 1961–1966, Lisbon, Portugal. find
  36. Xingxing Zhang and Mirella Lapata. 2014. Chinese poetry generation with recurrent neural networks. In Proceedings of 2014 EMNLP, pages 670–680, Doha, Qatar. find