Elamur Research⌕
☰
home / papers / 2019 / Scoring Sentence Singletons and Pairs for Abstractive Summarization

Scoring Sentence Singletons and Pairs for Abstractive Summarization

Paper · 2019 · Logan Lebanoff, Kaiqiang Song, Franck Dernoncourt et al.

Summary

When people write summaries, they usually form each summary sentence by shortening one source sentence or by combining material from two. Prior research handles these cases separately. Extractive methods select only individual sentences, and sentence fusion work assumes the sentences to merge are already given. Lebanoff et al. close this gap by ranking single sentences and sentence pairs in one shared scoring space. Every singleton and every pair from the input document(s) is encoded with BERT, which natively accepts one- or two-segment input. The model is fine-tuned to predict whether an instance contributes content to a reference summary sentence, with vector space models (VSM) tested as lighter alternatives. The top-ranked instances go to a pointer-generator network, which compresses singletons and fuses pairs to build the summary one sentence at a time. Experiments on DUC-04, CNN/Daily Mail and XSum yield results broadly comparable to earlier systems. They also show that abstractors degrade when many pairs are chosen, which points to fusion as a weakness of current encoder-decoder models. Because selection and generation are decoupled, the approach suits domains with limited training data.

Key points

  • Across DUC-04, CNN/Daily Mail and XSum, 60–85% of reference summary sentences draw on one or two source sentences. In all three datasets, the first summary sentence is the most likely to be a fusion.
  • The candidate set holds all singletons plus all pairs, N(N−1)/2 + N instances for N sentences. BERT's [CLS] representation is fine-tuned to classify these instances (BERT-SingPairMix), and GT-SingPairMix serves as the oracle upper bound.
  • BERT-SingPairMix selects pairs for 100% of instances on DUC-04, 76.90% on CNN/DM and 28.02% on XSum. The extractive output (BERT-Extr) beats the abstractive BERT-Abs-PG on DUC-04 and CNN/DM, while the reverse holds on XSum.
  • ROUGE (R-1, R-2, R-L, R-SU4) is computed with outputs capped at 100 words, or 40 words for XSum. XSum is the hardest dataset: its salient singletons appear anywhere in the document, and the two sentences of a pair often sit far apart.
  • Code and models are released at github.com/ucfnlp/summarization-sing-pair-mix.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢Adobe SystemsSan Jose · United States14 works · 3,493 citations🎓University of Central FloridaOrlando · United States8 works · 39,618 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Summarization · 159 works

Cite

Cite as Logan Lebanoff, Kaiqiang Song, Franck Dernoncourt, Doo Soon Kim, Seokhwan Kim, Walter Chang and Fei Liu. 2019. Scoring Sentence Singletons and Pairs for Abstractive Summarization. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1209

@inproceedings{lebanoff2019scoring,
  title = {{Scoring Sentence Singletons and Pairs for Abstractive Summarization}},
  author = {Logan Lebanoff, Kaiqiang Song, Franck Dernoncourt, Doo Soon Kim, Seokhwan Kim, Walter Chang and Fei Liu},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1209},
  url = {https://doi.org/10.18653/v1/p19-1209}
}

References

57 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Regina Barzilay and Kathleen R. McKeown. 2005. Sentence fusion for multidocument news summarization. Computational Linguistics, 31(3).
  2. Taylor Berg-Kirkpatrick, Dan Gillick, and Dan Klein. 2011. Jointly learning to extract and compress. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  3. Lidong Bing, Piji Li, Yi Liao, Wai Lam, Weiwei Guo, and Rebecca J. Passonneau. 2015. Abstractive multi-document summarization via phrase selection and merging. In Proceedings of ACL. find
  4. Ziqiang Cao, Wenjie Li, Sujian Li, and Furu Wei. 2018a. Retrieve, rerank and rewrite: Soft template based neural summarization. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  5. Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018b. Faithful to the original: Fact aware neural abstractive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). find
  6. Jaime Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR). find
  7. Giuseppe Carenini, Raymond Ng, and Adam Pauls. 2006. Multi-document summarization of evaluative text. In Proceedings of 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL). find
  8. Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. Deep communicating agents for abstractive summarization. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL).
  9. Danqi Chen, Jason Bolton, and Christopher D. Manning. 2016a. A thorough examination of the cnn/daily mail reading comprehension task. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL).
  10. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, and Hui Jiang. 2016b. Distraction-based neural networks for document summarization. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI). find
  11. Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  12. Victor Chenal and Jackie Chi Kit Cheung. 2016. Predicting sentential semantic compatibility for aggregation in text-to-text generation. In Proceedings of the International Conference on Computational Linguistics (COLING). find
  13. Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In Proceedings of ACL.
  14. Sangwoo Cho, Logan Lebanoff, Hassan Foroosh, and Fei Liu. 2019. Improving the similarity measure of determinantal point processes for extractive multi-document summarization. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  15. Hal Daumé III and Daniel Marcu. 2002. A noisy-channel model for document compression. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  16. Hal Daumé III and Daniel Marcu. 2004. Generic sentence fusion is an ill-defined summarization task. In Proceedings of ACL Workshop on Text Summarization Branches Out. find
  17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805.
  18. Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein. 2016. Learning-based single-document summarization with compression and anaphoricity constraints. In Proceedings of the Association for Computational Linguistics (ACL).
  19. Günes Erkan and Dragomir R. Radev. 2004. LexRank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research. find
  20. Katja Filippova. 2010. Multi-sentence compression: Finding shortest paths in word graphs. In Proceedings of the International Conference on Computational Linguistics (COLING). find
  21. Katja Filippova, Enrique Alfonseca, Carlos Colmenares, Lukasz Kaiser, and Oriol Vinyals. 2015. Sentence compression by deletion with lstms. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
  22. Dimitrios Galanis, Gerasimos Lampouras, and Ion Androutsopoulos. 2012. Extractive multi-document summarization with integer linear programming and support vector regression. In Proceedings of the International Conference on Computational Linguistics (COLING). find
  23. Kavita Ganesan, ChengXiang Zhai, and Jiawei Han. 2010. Opinosis: A graph-based approach to abstractive summarization of highly redundant opinions. In Proceedings of the International Conference on Computational Linguistics (COLING).
  24. Sebastian Gehrmann, Yuntian Deng, and Alexander M. Rush. 2018. Bottom-up abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
  25. Dan Gillick and Benoit Favre. 2009. A scalable global model for summarization. In Proceedings of the NAACL Workshop on Integer Linear Programming for Natural Langauge Processing.
  26. Han Guo, Ramakanth Pasunuru, and Mohit Bansal. 2018. Soft, layer-specific multi-task summarization with entailment and question generation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  27. Aria Haghighi and Lucy Vanderwende. 2009. Exploring content models for multi-document summarization. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL).
  28. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Proceedings of Neural Information Processing Systems (NIPS).
  29. Kai Hong and Ani Nenkova. 2014. Improving the estimation of word importance for news multi-document summarization. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL). find
  30. Wan-Ting Hsu, Chieh-Kai Lin, Ming-Ying Lee, Kerui Min, Jing Tang, and Min Sun. 2018. A unified model for extractive and abstractive summarization using inconsistency loss. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  31. Chris Kedzie, Kathleen McKeown, and Hal Daume III. 2018. Content selection in deep learning models of summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
  32. Alex Kulesza and Ben Taskar. 2011. Learning determinantal point processes. In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI). find
  33. Logan Lebanoff, Kaiqiang Song, and Fei Liu. 2018. Adapting the neural encoder-decoder framework from single to multi-document summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  34. Chen Li, Fei Liu, Fuliang Weng, and Yang Liu. 2013. Document summarization via guided sentence compression. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  35. Chen Li, Yang Liu, Fei Liu, Lin Zhao, and Fuliang Weng. 2014. Improving multi-document summarization by sentence compression based on expanded constituent parse tree. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP). find
  36. Piji Li, Wai Lam, Lidong Bing, and Zihao Wang. 2017. Deep recurrent generative decoder for abstractive text summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  37. Kexin Liao, Logan Lebanoff, and Fei Liu. 2018. Abstract meaning representation for multi-document summarization. In Proceedings of the International Conference on Computational Linguistics (COL-ING).
  38. Chin-Yew Lin. 2004. ROUGE: a package for automatic evaluation of summaries. In Proceedings of ACL Workshop on Text Summarization Branches Out.
  39. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press. find
  40. Andre F. T. Martins and Noah A. Smith. 2009. Summarization with a joint model for sentence extraction and compression. In Proceedings of the ACL Workshop on Integer Linear Programming for Natural Language Processing. find
  41. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of SIGNLL. find
  42. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don’t give me the details, just the summary! Topic-aware convolutional neural networks for extreme summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  43. Ani Nenkova and Kathleen McKeown. 2011. Automatic summarization. Foundations and Trends in Information Retrieval. find
  44. Paul Over and James Yen. 2004. An introduction to DUC-2004. National Institute of Standards and Technology. find
  45. Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
  46. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for sentence summarization. In Proceedings of EMNLP. find
  47. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  48. Chao Shen and Tao Li. 2010. Multi-document summarization via the minimum dominating set. In Proceedings of the International Conference on Computational Linguistics (COLING). find
  49. Kaiqiang Song, Lin Zhao, and Fei Liu. 2018. Structure-infused copy mechanisms for abstractive summarization. In Proceedings of the International Conference on Computational Linguistics (COL-ING). find
  50. Jiwei Tan, Xiaojun Wan, and Jianguo Xiao. 2017. Abstractive document summarization with a graph-based attentional neural model. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
  51. Kapil Thadani and Kathleen McKeown. 2013. Supervised sentence fusion with single-stage inference. In Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP). find
  52. Lucy Vanderwende, Hisami Suzuki, Chris Brockett, and Ani Nenkova. 2007. Beyond SumBasic: Task-focused summarization with sentence simplification and lexical expansion. Information Processing and Management, 43(6):1606–1618. find
  53. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. https://arxiv.org/abs/1706.03762. In Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS). ↗
  54. Lu Wang, Hema Raghavan, Vittorio Castelli, Radu Florian, and Claire Cardie. 2013. A sentence compression based framework to query-focused multi-document summarization. In Proceedings of ACL. find
  55. Kam-Fai Wong, Mingli Wu, and Wenjie Li. 2008. Extractive summarization using supervised and semi-supervised learning. In Proceedings of the International Conference on Computational Linguistics (COLING). find
  56. David Zajic, Bonnie J. Dorr, Jimmy Lin, and Richard Schwartz. 2007. Multi-candidate reduction: Sentence compression as a tool for document summarization tasks. Information Processing and Management. find
  57. Qingyu Zhou, Nan Yang, Furu Wei, and Ming Zhou. 2017. Selective encoding for abstractive sentence summarization. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lebanoff-2019-scoring-sentence-singletons-pairs-abstractive/