Elamur Research
☰
home / papers / 2016 / SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents

SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1611.04230

Summary

The paper addresses single-document extractive summarization: choosing which sentences of a news article belong in its summary. The authors present SummaRuNNer, which frames the task as sequence classification. A two-layer bidirectional GRU-RNN builds representations at the word level and then at the sentence level. A logistic layer then revisits each sentence in document order and decides whether to include it. This decision combines separate, inspectable terms for content, salience relative to the document, novelty against a running summary representation, and absolute and relative position. The training labels are derived greedily by picking sentences that maximize Rouge against human abstracts. The authors also propose an alternative abstractive training scheme, in which a temporary RNN decoder must regenerate the reference summary from the model's summary vector. This removes the need for sentence-level labels. Evaluated on the CNN/DailyMail corpus, the model matches or beats prior neural systems. On out-of-domain DUC 2002 it ties Cheng and Lapata (2016) but trails graph-based methods. The work matters because it shows a simple, interpretable neural extractor can compete with more complex models.

Key points

  • Architecture: SummaRuNNer has a word-level bidirectional GRU and a sentence-level bidirectional GRU. The document vector is an average-pooled, tanh-transformed representation, and each sentence is classified with interpretable score terms (content, salience, novelty, positional importance).
  • CNN/DailyMail results: On the full test set, using full-length F1 Rouge, SummaRuNNer reaches Rouge-2 of 16.2±0.2 and Rouge-L of 35.3±0.2. It significantly beats the abstractive state of the art and also beats Lead-3 on Rouge-1 and Rouge-2.
  • DUC 2002 results: Trained on DailyMail and tested on DUC 2002 (75-word recall), it scores 46.6 Rouge-1, 23.1 Rouge-2 and 43.03 Rouge-L. This is statistically on par with Cheng et al. '16 (47.4 / 23.0 / 41.2) but below TGRAPH and URANK.
  • Abstractive training: The SummaRuNNer-abs variant needs no extractive labels, but it lands a couple of Rouge points below extractive training (for example, 44.8 Rouge-1 on DUC 2002).
  • Interpretability: Because the scoring terms are separate, each selection decision can be visualized by feature, which helps explain outputs to users.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

333 papers of our library cite it (of the 140673 whose bibliographies we read); 17 of them have a page here, 316 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1611.04230 · arXiv 1611.04230

@misc{nallapati2016summarunner,
  title = {{SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents}},
  author = {Nallapati, Ramesh and Zhai, Feifei and Zhou, Bowen},
  year = {2016},
  doi = {10.48550/arxiv.1611.04230},
  eprint = {1611.04230},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1611.04230}
}

References

23 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bahdanau, D.; Cho, K.; and Bengio, Y. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. find our card
  2. Cao, Z.; Chen, C.; Li, W.; Li, S.; Wei, F.; and Zhou, M. 2015. Tgsum: Build tweet guided multi-document summarization dataset. CoRR abs/1511.08417. — [arXiv:1511.08417](https://arxiv.org/abs/1511.08417) find
  3. Cao, Z.; Li, W.; Li, S.; and Wei, F. 2016. Attsum: Joint learning of focusing and summarization with neural attention. arXiv preprint arXiv:1604.00125. find
  4. Carbonell, J., and Goldstein, J. 1998. The use of mmr, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, 335– 336. ACM. find
  5. Cheng, J., and Lapata, M. 2016. Neural summarization by extracting sentences and words. 54th Annual Meeting of the Association for Computational Linguistics. find our card
  6. Chung, J.; Gülçehre, Ç.; Cho, K.; and Bengio, Y. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. CoRR abs/1412.3555. find
  7. Ganesan, K.; Zhai, C.; and Han, J. 2010. Opinosis: a graph-based approach to abstractive summarization of highly redundant opinions. In Proceedings of the 23rd international conference on computational linguistics, 340–348. Association for Computational Linguistics. find
  8. Hermann, K. M.; Kociský, T.; Grefenstette, E.; Espeholt, L.; Kay, W.; Suleyman, M.; and Blunsom, P. 2015. Teaching machines to read and comprehend. CoRR abs/1506.03340. find our card
  9. Kageback, M.; Mogren, O.; Tahmasebi, N.; and Dubhashi, D. 2014. Extractive summarization using continuous vector space models. 31–39. find
  10. McDonald, R. 2007. A study of global inference algorithms in multi-document summarization. 557– 564. find
  11. Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and Dean, J. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, 3111– 3119. find
  12. Nallapati, R.; Zhou, B.; dos Santos, C.; Gulcehre, C.; and Xiang, B. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. The SIGNLL Conference on Computational Natural Language Learning. find
  13. Nallapati, R.; Zhou, B.; and Xiang, B. 2016. Sequence-to-sequence rnns for text summarization. International Conference on Learning Representations, Workshop track. find
  14. Parveen, D.; Ramsl, H.-M.; and Strube, M. 2015. Topical coherence for graph-based extractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 19491954. find
  15. Radev, D., and Erkan, G. 2004. Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research 457–479. find
  16. Rush, A. M.; Chopra, S.; and Weston, J. 2015. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685. find
  17. Shen, D.; Sun, J.-T.; Li, H.; Yang, Q.; and Chen, Z. 2007. Document summarization using conditional random fields. In Proceedings of IJCAI. find
  18. Socher, R.; Huang, E. H.; Pennin, J.; Manning, C. D.; and Ng, A. Y. 2011. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. 801–809. find
  19. Svore, K. M.; Vanderwende, L.; and Burges, C. J. 2007. Enhancing single-document summarization by combining ranknet and third-party sources. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, 448–457. find
  20. Wan, X. 2010. Towards a unified approach to simultaneous single-document and multidocument summarizations. In In Proceedings of the 23rd COLING, 11371145. find
  21. Woodsend, K., and Lapata, M. 2010. Automatic generation of story highlights. In In Proceedings of the 48th ACL, 565574. find
  22. Yin, W., and Pei, Y. 2015. Optimizing sentence modeling and selection for document summarization. In Proceedings of the 24th International Conference on Artificial Intelligence, 1383–1389. AAAI Press. find
  23. Zeiler, M. D. 2012. ADADELTA: an adaptive learning rate method. CoRR abs/1212.5701. find our card