Elamur Research⌕
☰
home / papers / 2016 / Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints

Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints

Summary

The paper tackles single-document summarization. This task lacks the redundancy signal that multi-document systems rely on, and simple lead-based baselines are hard to beat. The authors train a discriminative, feature-rich extractive-compressive model on the New York Times Annotated Corpus, which contains roughly 100,000 articles paired with abstractive summaries.

Inference is an ILP that picks sub-sentential textual units under a word budget. Two kinds of constraints shape the output:

  • Grammaticality constraints combine RST-based discourse compressions with syntactic deletion patterns, encoded as "requires" links between units.
  • Anaphoricity constraints ensure that any kept pronoun either has its antecedent in the summary or is replaced by a full mention.

Weights are learned end-to-end with max-margin training against ROUGE-style losses. On the NYT50 test set, the full system clearly beats prefix baselines, a bigram-frequency method and a reimplementation of Yoshida et al.'s Tree Knapsack on ROUGE. Mechanical Turk ratings show its readability comes close to sentence extraction.

The work shows that learned content selection plus linguistically motivated constraints can raise informativeness without sacrificing fluency.

Key points

  • NYT50 results: the full model scores ROUGE-1 42.2 and ROUGE-2 25.9. By comparison, Tree Knapsack scores 34.7/19.6, first-k-words 35.7/21.6 and sentence extraction 38.8/23.5.
  • Ablations: removing syntactic compressions drops scores to 41.1/25.0, and removing discourse compressions drops them to 40.5/24.7. Removing anaphoricity slightly raises ROUGE (42.5/26.3) but increases unclear pronouns from 0.36 to 0.44.
  • Linguistic quality: human clarity scores are 7.52 for the full system and 6.38 for unconstrained EDU extraction, against 7.93 for sentence extraction.
  • Training cost: the model trained on 22,000 documents (those with fewer than 100 EDUs) for 10 passes in about 20 hours on one EC2 core.
  • RST Discourse Treebank: on this 20-document test set, the full model reaches ROUGE-1 26.3 versus 25.1 for Tree Knapsack, but the difference is not statistically significant.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis

Cited by

63 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 57 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p16-1188 · arXiv 1603.08887

@inproceedings{durrett2016learning,
  title = {{Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints}},
  author = {Durrett, Greg and Berg-Kirkpatrick, Taylor and Klein, Dan},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1188},
  eprint = {1603.08887},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p16-1188}
}

References

55 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. We also evaluate on the RST Discourse Treebank, of which 30 documents have abstractive summaries. Following Hirao et al. (2013), we use the gold EDU segmentation from the RST corpus but automatic RST trees. We break this into a 10-document development set and a 20-document test set. Table 2 shows th… find
  2. Miguel Almeida and Andre Martins. 2013. Fast and Robust Compressive Summarization with Dual Decomposition and Multi-Task Learning. In Proceedings of the Association for Computational Linguistics (ACL). find
  3. Regina Barzilay and Mirella Lapata. 2008. Modeling Local Coherence: An Entity-based Approach. Computational Linguistics, 34(1):1–34, March. find our card
  4. Regina Barzilay and Lillian Lee. 2004. Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  5. Regina Barzilay, Noemie Elhadad, and Kathleen R. McKeown. 2001. Sentence Ordering in Multidocu-ment Summarization. In Proceedings of the International Conference on Human Language Technology Research. find
  6. Taylor Berg-Kirkpatrick, Dan Gillick, and Dan Klein. 2011. Jointly Learning to Extract and Compress. In Proceedings of the Association for Computational Linguistics (ACL). find
  7. Jaime Carbonell and Jade Goldstein. 1998. The Use of MMR, Diversity-based Reranking for Reordering Documents and Producing Summaries. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. find
  8. Lynn Carlson, Daniel Marcu, and Mary Ellen Okurowski. 2001. Building a Discourse-tagged Corpus in the Framework of Rhetorical Structure Theory. In Proceedings of the Second SIGDIAL Workshop on Discourse and Dialogue. find our card
  9. Janara Christensen, Mausam, Stephen Soderland, and Oren Etzioni. 2013. Towards Coherent Multi-Document Summarization. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  10. James Clarke and Mirella Lapata. 2008. Global Inference for Sentence Compression an Integer Linear Programming Approach. Journal of Artificial Intelligence Research, 31(1):399–429, March. find
  11. James Clarke and Mirella Lapata. 2010. Discourse Constraints for Document Compression. Computational Linguistics, 36(3):411–441, September. find
  12. Hal Daumé, III and Daniel Marcu. 2002. A Noisy-Channel Model for Document Compression. In Proceedings of the Association for Computational Linguistics (ACL). find
  13. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research, 12:2121–2159, July. find
  14. Jesse Dunietz and Daniel Gillick. 2014. A New Entity Salience Task with Millions of Training Examples. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL). find our card
  15. Greg Durrett and Dan Klein. 2013. Easy Victories and Uphill Battles in Coreference Resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), October. find our card
  16. Greg Durrett and Dan Klein. 2014. A Joint Model for Entity Analysis: Coreference, Typing, and Linking. In Transactions of the Association for Computational Linguistics (TACL). find our card
  17. Dan Gillick and Benoit Favre. 2009. A Scalable Global Model for Summarization. In Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing. find
  18. Dan Gillick and Yang Liu. 2010. Non-Expert Evaluation of Summarization Systems is Risky. In Proceedings of the NAACL Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk. find
  19. H.P. Grice. 1975. Logic and Conversation. Syntax and Semantics 3: Speech Acts, pages 41–58. find
  20. Tsutomu Hirao, Yasuhisa Yoshida, Masaaki Nishino, Norihito Yasuda, and Masaaki Nagata. 2013. Single-Document Summarization as a Tree Knapsack Problem. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  21. Kai Hong and Ani Nenkova. 2014. Improving the Estimation of Word Importance for News Multi-Document Summarization. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL). find
  22. Shafiq Joty, Giuseppe Carenini, Raymond Ng, and Yashar Mehdad. 2013. Combining Intra- and Multi-sentential Rhetorical Parsing for Document-level Discourse Analysis. In Proceedings of the Association for Computational Linguistics (ACL). find
  23. Yuta Kikuchi, Tsutomu Hirao, Hiroya Takamura, Manabu Okumura, and Masaaki Nagata. 2014. Single Document Summarization based on Nested Tree Structure. In Proceedings of the Association for Computational Linguistics (ACL). find
  24. Jonathan K. Kummerfeld, Taylor Berg-Kirkpatrick, and Dan Klein. 2015. An Empirical Analysis of Optimization for Max-Margin NLP. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  25. Chen Li, Yang Liu, and Lin Zhao. 2015. Using External Resources and Joint Learning for Bigram Weighting in ILP-Based Multi-Document Summarization. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  26. Hui Lin and Jeff Bilmes. 2011. A Class of Submodular Functions for Document Summarization. In Proceedings of the Association for Computational Linguistics (ACL). find
  27. Barbara J. Grosz, Scott Weinstein, and Aravind K. Joshi. 1995. Centering: A Framework for Modeling the Local Coherence of Discourse. Computational Linguistics, 21(2):203–225, June. find our card
  28. Chin-Yew Lin and Eduard Hovy. 2003. Automatic Evaluation of Summaries Using N-gram Co-Occurrence Statistics. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL). find
  29. David Hall, Taylor Berg-Kirkpatrick, John Canny, and Dan Klein. 2014. Sparser, Better, Faster GPU Parsing. In Proceedings of the Association for Computational Linguistics (ACL). find
  30. Chin-Yew Lin. 2003. Improving Summarization Performance by Sentence Compression: A Pilot Study. In Proceedings of the International Workshop on Information Retrieval with Asian Languages. find
  31. Zhanying He, Chun Chen, Jiajun Bu, Can Wang, Lijun Zhang, Deng Cai, and Xiaofei He. 2012. Document Summarization Based on Data Reconstruction. In Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI). find
  32. Annie Louis and Ani Nenkova. 2012. A Coherence Model Based on Syntactic Patterns. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). find
  33. Hugo Hernault, Helmut Prendinger, David A. Duverle, Mitsuru Ishizuka, and Tim Paek. 2010. HILDA: A discourse parser using support vector machine classification. Dialogue and Discourse, 1:1–33. find our card
  34. Annie Louis, Aravind Joshi, and Ani Nenkova. 2010. Discourse Indicators for Content Selection in Summarization. In Proceedings of the SIGDIAL 2010 Conference. find
  35. Inderjeet Mani. 2001. AutomaticSummarization. John Benjamins Publishing. find
  36. William C. Mann and Sandra A. Thompson. 1988. Rhetorical Structure Theory: Toward a Functional Theory of Text Organization. Text, 8(3):243–281. find our card
  37. Daniel Marcu. 1998. Improving summarization through rhetorical parsing tuning. In Proceedings of the Workshop on Very Large Corpora. find
  38. Andre Martins and Noah A. Smith. 2009. Summarization with a Joint Model for Sentence Extraction and Compression. In Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing. find
  39. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online Large-margin Training of Dependency Parsers. In Proceedings of the Association for Computational Linguistics (ACL). find our card
  40. Ryan McDonald. 2006. Discriminative Sentence Compression With Soft Syntactic Evidence. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL). find
  41. Kathleen McKeown, Jacques Robin, and Karen Ku-kich. 1995. Generating Concise Natural Language Summaries. Information Processing and Management, 31(5):703–733, September. find
  42. Shamima Mithun and Leila Kosseim. 2011. Discourse Structures to Reduce Discourse Incoherence in Blog Summarization. In Proceedings of Recent Advances in Natural Language Processing. find
  43. Ani Nenkova and Kathleen McKeown. 2011. Automatic summarization. Foundations and Trends in Information Retrieval, 5(2?3):103–233. find
  44. Khanh Nguyen and Brendan O’Connor. 2015. Posterior Calibration and Exploratory Analysis for Natural Language Processing Models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  45. Daniele Pighin, Marco Cornolti, Enrique Alfonseca, and Katja Filippova. 2014. Modelling Events through Memory-based, Open-IE Patterns for Abstractive Summarization. In Proceedings of the Association for Computational Linguistics (ACL). find
  46. Nathan J. Ratliff, Andrew Bagnell, and Martin Zinke-vich. 2007. (Online) Subgradient Methods for Structured Prediction. In Proceedings of the International Conference on Artificial Intelligence and Statistics. find
  47. Evan Sandhaus. 2008. The New York Times Annotated Corpus. In Linguistic Data Consortium. find
  48. Noah A. Smith. 2011. Linguistic Structure Prediction. Morgan & Claypool Publishers, 1st edition. find
  49. Radu Soricut and Daniel Marcu. 2003. Sentence Level Discourse Parsing Using Syntactic and Lexical Information. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL). find our card
  50. Kapil Thadani and Kathleen McKeown. 2013. Supervised Sentence Fusion with Single-Stage Inference. In Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP). find
  51. Kristian Woodsend and Mirella Lapata. 2012. Multiple Aspect Summarization Using Integer Linear Programming. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). find
  52. Yasuhisa Yoshida, Jun Suzuki, Tsutomu Hirao, and Masaaki Nagata. 2014. Dependency-based Discourse Parser for Single-Document Summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). find our card
  53. Hitoshi Nishikawa, Kazuho Arita, Katsumi Tanaka, Tsutomu Hirao, Toshiro Makino, and Yoshihiro Matsuo. 2014. Learning to Generate Coherent Summary with Discriminative Hidden Semi-Markov Model. In Proceedings of the International Conference on Computational Linguistics (COLING). find
  54. Gerald Penn and Xiaodan Zhu. 2008. A Critical Reassessment of Evaluation Baselines for Speech Summarization. In Proceedings of the Association for Computational Linguistics (ACL). find
  55. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning Accurate, Compact, and Interpretable Tree Annotation. In Proceedings of the Conference on Computational Linguistics and the Association for Computational Linguistics (ACL-COLING). find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/durrett-2016-learning-based-single-document-summarization/