Elamur Research
☰
home / papers / 2021 / GSum: A General Framework for Guided Neural Abstractive Summarization

GSum: A General Framework for Guided Neural Abstractive Summarization

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/2021.naacl-main.384.pdf

Summary

Abstractive summarizers write fluent text, but they can introduce facts not found in the source and are hard to steer. Earlier work added guidance of one kind at a time, such as length tokens, keywords, relation triples, or retrieved templates. Those approaches were never compared directly. The authors present GSum, a single framework that conditions a pretrained encoder-decoder (BERT or BART) on both the document and an external guidance signal.

GSum uses two encoders, one for the document and one for the guidance. They share their word embeddings and lower layers. Each decoder layer first attends to the guidance and then to the full document. The authors test four signal types: highlighted sentences, keywords, (subject, relation, object) triples, and retrieved summaries. A key training choice is to build guidance with an oracle that also sees the reference summary; at test time, guidance comes from an automatic extractor or a user.

With highlighted sentences as guidance, GSum reaches state-of-the-art ROUGE on 4 of 6 benchmarks. Its outputs are judged more faithful, and changing the guidance lets users control what the summary covers.

Key points

  • On CNN/DM, sentence-guided GSum improves on the previous best system by 1.28 ROUGE-1, 0.79 ROUGE-2 and 1.13 ROUGE-L.
  • Oracle-based guidance during training is essential. Compared with training on automatically extracted guidance (both tested with automatic guidance), it raises ROUGE-1/2/L from 41.61/19.04/38.65 to 43.78/20.66/40.66.
  • In human faithfulness ratings, all four guided variants scored about 2.30–2.39, versus 2.117 for the BertAbs baseline (significant, p < 0.001, bootstrap).
  • The guidance types complement each other, so combining their outputs could yield further gains. Guided models also produce more novel words.
  • Code is released at github.com/neulab/guided_summarization.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

117 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 112 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.naacl-main.384

@inproceedings{dou2021gsum,
  title = {{GSum: A General Framework for Guided Neural Abstractive Summarization}},
  author = {Dou, Zi-Yi and Liu, Pengfei and Hayashi, Hiroaki and Jiang, Zhengbao and Neubig, Graham},
  booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2021},
  doi = {10.18653/v1/2021.naacl-main.384},
  url = {https://doi.org/10.18653/v1/2021.naacl-main.384}
}

References

36 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find our card
  2. Gabor Angeli, Melvin Jose Johnson Premkumar, and Christopher D Manning. 2015. Leveraging linguistic structure for open domain information extraction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find our card
  3. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS). find our card
  4. Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. find our card
  5. Kai Hong, Mitchell Marcus, and Ani Nenkova. 2015. System combination for multi-document summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  6. Ziqiang Cao, Wenjie Li, Sujian Li, and Furu Wei. 2018. Retrieve, rerank and rewrite: Soft template based neural summarization. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  7. Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find our card
  8. Sumit Chopra, Michael Auli, and Alexander M Rush. 2016. Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). find our card
  9. Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the Conference of the North American Chapter of the Association for Computational Lingu… find our card
  10. Hanqi Jin, Tianming Wang, and Xiaojun Wan. 2020. Semsum: Semantic dependency guided neural abstractive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). find
  11. Chris Kedzie, Kathleen McKeown, and Hal Daumé III. 2018. Content selection in deep learning models of summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  12. Yuta Kikuchi, Graham Neubig, Ryohei Sasano, Hiroya Takamura, and Manabu Okumura. 2016. Controlling output length in neural encoder-decoders. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  13. Byeongchang Kim, Hyunwoo Kim, and Gunhee Kim. 2019. Abstractive summarization of reddit posts with multi-level memory networks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  14. Mahnaz Koupaee and William Yang Wang. 2018. Wikihow: A large scale text summarization dataset. arXiv preprint. find
  15. Wojciech Kryściński, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Evaluating the factual consistency of abstractive text summarization. arXiv preprint. find
  16. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the Annual Meeting of the Associat… find our card
  17. Chenliang Li, Weiran Xu, Si Li, and Sheng Gao. 2018. Guiding generation for abstractive text summarization based on key information guide network. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  18. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out. find
  19. Peter J Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. 2018a. Generating wikipedia by summarizing long sequences. In Proceddings of the Intertional Conference on Learning Representatinos (ICLR). find our card
  20. Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018b. Ranking sentences for extractive summarization with reinforcement learning. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  21. Karolina Owczarzak and Hoa Trang Dang. 2010. Overview of the TAC 2010 summarization track. In Proceedings of the Text Analysis Conference (TAC). find
  22. Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In Proceddings of the Intertional Conference on Learning Representatinos (ICLR). find our card
  23. Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  24. Itsumi Saito, Kyosuke Nishida, Kosuke Nishida, and Junji Tomita. 2020. Abstractive summarization with combination of pre-trained sequence-to-sequence and saliency models. arXiv preprint. find
  25. Evan Sandhaus. 2008. The new york times annotated corpus. Linguistic Data Consortium. find
  26. Yang Liu and Mirella Lapata. 2019. Text summarization with pretrained encoders. In Proceedings of the Coference on Empirical Methods in Natural Language Processing (EMNLP). find
  27. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  28. Yizhu Liu, Zhiyi Luo, and Kenny Zhu. 2018b. Controlling length in abstractive summarization using a convolutional neural network. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS). find
  30. Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  31. Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, and Xuanjing Huang. 2020. Extractive summarization as text matching. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find our card
  32. Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. Summarunner: A recurrent neural network based sequence model for extractive summarization of documents. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). find our card
  33. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of the SIGNLL Conference on Computational Natural Language Learning (CoNLL). find
  34. Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018a. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  35. Qingyu Zhou, Nan Yang, Furu Wei, Shaohan Huang, Ming Zhou, and Tiejun Zhao. 2018. Neural document summarization by jointly learning to score and select sentences. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). find
  36. Chenguang Zhu, William Hinthorn, Ruochen Xu, Qingkai Zeng, Michael Zeng, Xuedong Huang, and Meng Jiang. 2020. Boosting factual correctness of abstractive summarization with knowledge graph. arXiv preprint. find