Elamur Research⌕
☰
home / papers / 2010 / Discourse Constraints for Document Compression

Discourse Constraints for Document Compression

Paper · 2010 · James Clarke and Mirella Lapata

Summary

Most sentence compression systems shorten each sentence on its own and ignore the surrounding text. Real applications such as summarization, however, need a shorter document that still reads coherently. Clarke and Lapata address this gap with a compression model that operates at the document level and accounts for local coherence. Documents are first annotated automatically with two kinds of discourse information: sentence centers based on Centering Theory, and lexical chains that connect related words across sentences. These annotations become additional constraints in an integer linear programming (ILP) model. The model extends Clarke and Lapata's (2008) word-deletion approach so that a whole document is compressed jointly, subject to both grammaticality and discourse constraints. Output is evaluated with a question-answering task that tests whether readers can still recover the key content. The discourse-aware model significantly outperforms discourse-agnostic state-of-the-art systems. This shows that discourse cues can be identified reliably in raw text and that they make compressed documents more informative and coherent.

Key points

  • Formulation: Compression is treated as word deletion. The ILP objective is redefined over the entire document rather than single sentences, discourse-specific constraints are added, and compression-rate constraints are made soft rather than hard.
  • Discourse representations: The model uses two complementary representations. Centering yields dense annotations, since it seeks a center in almost every sentence. Lexical chains capture broader document topics. The improvements come from combining the two; neither one alone is ideal.
  • Baselines and evaluation: The model is compared with a re-implementation of McDonald (2006) and with the sentence-level ILP of Clarke and Lapata (2008). Quality is judged with a Q&A evaluation paradigm.
  • Qualitative findings: In example outputs, the sentence-level ILP drops crucial facts and can lead readers to wrong inferences, while the discourse ILP keeps the core content.
  • Future directions: The discourse annotations could serve as features in discriminative models. Another proposed step is coupling compression with sentence extraction (cf. Martins & Smith 2009). Exact scores and corpus details appear in the omitted middle part of the provided text and are not reported here.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom359 works · 77,352 citations🎓University of Illinois Urbana-ChampaignUrbana · United States184 works · 69,372 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Discourse analysis · 466 works

Cite

Cite as James Clarke and Mirella Lapata. 2010. Discourse Constraints for Document Compression. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli_a_00004

@article{clarke2010discourse,
  title = {{Discourse Constraints for Document Compression}},
  author = {James Clarke and Mirella Lapata},
  journal = {Computational Linguistics},
  year = {2010},
  doi = {10.1162/coli\_a\_00004},
  url = {https://doi.org/10.1162/coli_a_00004}
}

References

68 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aho, A. V. and J. D. Ullman. 1969. Syntax directed translations and the pushdown assembler. Journal of Computer and System Sciences, 3:37–56. find
  2. Barzilay, R. and M. Elhadad. 1997. Using lexical chains for text summarization. In Proceedings of the ACL-97 Intelligent Scalable Text Summarization Workshop, pages 10–17, Madrid.
  3. Barzilay, Regina and Mirella Lapata. 2006. Aggregation via set partitioning for natural language generation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 359–366, New York, NY. find
  4. Barzilay, Regina and Mirella Lapata. 2008. Modeling local coherence: An entity-based approach. Computational Linguistics, 34(1):1–34.
  5. Boguraev, Branimir and Chris Kennedy. 1997. Salience-based content characterization of text documents. In Proceedings of the ACL’97/EACL’97 Workshop on Intelligent Scalable Text Summarization, pages 2–9, Madrid. find
  6. Brin, Sergey and Michael Page. 1998. Anatomy of a large-scale hypertextual Web search engine. In Proceedings of the 7th Conference on World Wide Web, pages 107–117, Brisbane. find
  7. Briscoe, E. J. and J. Carroll. 2002. Robust accurate statistical annotation of general text. In Proceedings of the 3rd International Conference on Language Resources and Evaluation (LREC–2002), pages 1499–1504, Las Palmas.
  8. Carlson, Lynn, John M. Conroy, Daniel Marcu, Dianne P. O’Leary, Mary E. Okurowski, and Anthony Taylor. 2001. An empirical study on the relation between abstracts, extracts, and the discourse structure of texts. In Proceedings of the DUC-2001 Workshop on Text Summarization, New Orleans, LA. find
  9. Chang, Ming-Wei, Lev Ratinov, and Dan Roth. 2007. Guiding semi-supervision with constraint-driven learning. In Proceedings of the 22nd International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 280–287, Prague. find
  10. Charniak, Eugene. 2000. A maximum-entropy-inspired parser. In Proceedings of the 1st North American Annual Meeting of the Association for Computational Linguistics, pages 132–139, Seattle, WA.
  11. Clarke, James. 2008. Global Inference for Sentence Compression: An Integer Linear Programming Approach. Ph.D. thesis, University of Edinburgh. find
  12. Clarke, James and Mirella Lapata. 2006. Models for sentence compression: A comparison across domains, training requirements and evaluation measures. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguisti… find
  13. Clarke, James and Mirella Lapata. 2008. Global inference for sentence compression: An integer linear programming approach. Journal of Artificial Intelligence Research, 31:399–429. find
  14. Cohn, Trevor and Mirella Lapata. 2009. Sentence compression as tree transduction. Journal of Artificial Intelligence Research, 34:637–674. find
  15. Corston-Oliver, Simon. 2001. Text compaction for display on very small screens. In Proceedings of the NAACL Workshop on Automatic Summarization, pages 89–98, Pittsburgh, PA. find
  16. Corston-Oliver, Simon H. 1998. Computing representations of the structure of written discourse. Technical Report MSR-TR-98-15, Microsoft Research, Redmond, WA. find
  17. Crammer, Koby and Yoram Singer. 2003. Ultraconservative online algorithms for multiclass problems. Journal of Machine Learning Research, 3:951–991. find
  18. Daumé III, Hal and Daniel Marcu. 2002. A noisy-channel model for document compression. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 449–456, Philadelphia, PA. find
  19. Denis, Pascal and Jason Baldridge. 2007. Joint determination of anaphoricity and coreference resolution using integer programming. In Proceedings of Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics, pages 236–243, Rochest…
  20. Dras, Mark. 1997. Reluctant paraphrase: Textual restructuring under an optimisation model. In Proceedings of the Fifth Biannual Meeting of the Pacific Association for Computational Linguistics, pages 98–104, Ohme. find
  21. Endres-Niggemeyer, Brigitte. 1998. Summarising Information. Springer, Berlin. find
  22. Fellbaum, Christiane, editor. 1998. WordNet: An Electronic Database. MIT Press, Cambridge, MA. find
  23. Galley, Michel and Kathleen McKeown. 2003. Improving word sense disambiguation in lexical chaining. In Proceedings of 18th International Joint Conference on Artificial Intelligence (IJCAI–03), pages 1486–1488, Acapulco, Mexico. find
  24. Galley, Michel and Kathleen McKeown. 2007. Lexicalized Markov grammars for sentence compression. In Proceedings of Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics, pages 180–187, Rochester, NY. find
  25. Grefenstette, Gregory. 1998. Producing Intelligent Telegraphic Text Reduction to Provide an Audio Scanning Service for the Blind. In Proceedings of the AAAI Symposium on Intelligent Text Summarization, pages 111–117, Stanford, CA. find
  26. Grosz, Barbara J., Scott Weinstein, and Aravind K. Joshi. 1995. Centering: a framework for modeling the local coherence of discourse. Computational Linguistics, 21(2):203–225.
  27. Halliday, M. A. K. and Ruqaiya Hasan. 1976. Cohesion in English. Longman, London. find
  28. Hirst, Graeme and David St-Onge. 1998. Lexical chains as representations of context for the detection and correction of malapropisms. In Christiane Fellbaum, editor, WordNet: An Electronic Database. MIT Press, Cambridge, MA, pages 305–332. find
  29. Hori, Chiori and Sadaoki Furui. 2004. Speech summarization: An approach through word extraction and a method for evaluation. IEICE Transactions on Information and Systems, E87-D(1):15–25, 1. find
  30. Jing, Hongyan. 2000. Sentence reduction for automatic text summarization. In Proceedings of the 6th conference on Applied Natural Language Processing, pages 310–315, Seattle, WA.
  31. Kibble, Rodger and Richard Power. 2004. Optimising referential coherence in text generation. Computational Linguistics, 30(4):401–416. find
  32. Knight, Kevin and Daniel Marcu. 2002. Summarization beyond sentence extraction: a probabilistic approach to sentence compression. Artificial Intelligence, 139(1):91–107. find
  33. Kupiec, Julian, Jan O. Pedersen, and Francine Chen. 1995. A trainable document summarizer. In Proceedings of SIGIR-95, pages 68–73, Seattle, WA. find
  34. Lin, Chin-Yew. 2003. Improving summarization performance by sentence compression—A pilot study. In Proceedings of the 6th International Workshop on Information Retrieval with Asian Languages, pages 1–8, Sapporo. find
  35. Lin, Dekang. 2001. LaTaT: Language and text analysis tools. In Proceedings of the first Human Language Technology Conference, pages 222–227, San Francisco, CA. find
  36. Mani, Inderjeet. 2001. Automatic Summarization. John Benjamins, Amsterdam. find
  37. Mani, Inderjeet, Thérèse Firmin, David House, Gary Klein, Beth Sundheim, and Lynette Hirschman. 2002. The TIPSTER SUMMAC Text Summarization Evaluation. Natural Language Engineering, 8:43–68. find
  38. Mani, Inderjeet, Barbara Gates, and Eric Bloedorn. 1999. Improving summaries by revising them. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 558–565, College Park, MD. find
  39. Mann, William C. and Sandra A. Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243–281.
  40. Marciniak, Tomasz and Michael Strube. 2005. Beyond the pipeline: Discrete optimization in NLP. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL–2005), pages 136–143, Ann Arbor, MI. find
  41. Marcu, Daniel. 2000. The Theory and Practice of Discourse Parsing and Summarization. The MIT Press, Cambridge, MA. find
  42. Martins, André and Noah A. Smith. 2009. Summarization with a joint model for sentence extraction and compression. In Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing, pages 1–9, Boulder, CO. find
  43. Martins, André, Noah Smith, and Eric Xing. 2009. Concise integer linear programming formulations for dependency parsing. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 342–350… find
  44. McDonald, Ryan. 2006. Discriminative sentence compression with soft syntactic constraints. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, pages 297–304, Trento. find
  45. Miltsakaki, Eleni and Karen Kukich. 2000. The role of centering theory’s rough-shift in the teaching and evaluation of writing skills. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 408–415, Hong Kong. find
  46. Morris, A., G. Kasper, and D. Adams. 1992. The effects and limitations of automated text condensing on reading comprehension performance. Information Systems Research, 3(1):17–35. find
  47. Morris, Jane and Graeme Hirst. 1991. Lexical cohesion computed by thesaural relations as an indicator of the structure of text. Computational Linguistics, 17(1):21–48.
  48. Nguyen, Minh Le, Akira Shimazu, Susumu Horiguchi, Tu Bao Ho, and Masaru Fukushi. 2004. Probabilistic sentence reduction using support vector machines. In Proceedings of the 20th International Conference on Computational Linguistics, pages 743–749, Geneva. find
  49. Olivers, S. H. and W. B. Dolan. 1999. Less is more; eliminating index terms from subordinate clauses. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 349–356, College Park, MD. find
  50. Ono, Kenji, Kazuo Sumita, and Seiji Miike. 1994. Abstract generation based on rhetorical structure extraction. In Proceedings of the 15th International Conference on Computational Linguistics, pages 344–348, Kyoto.
  51. Orăsan, Constantin. 2003. An evolutionary approach for improving the quality of automatic summaries. In ACL Workshop on Multilingual Summarization and Question Answering, pages 37–45, Sapporo, Japan. find
  52. Poesio, Massimo, Rosemary Stevenson, Barbara Di Eugenio, and Janet Hitzeman. 2004. Centering: a parametric theory and its instantiations. Computational Linguistics, 30(3):309–363.
  53. Press, William H., Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. 1992. Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, Cambridge, UK. find
  54. Punyakanok, Vasin, Dan Roth, Wen-tau Yih, and Dav Zimak. 2004. Semantic role labeling via integer linear programming inference. In Proceedings of the 20th International Conference on Computational Linguistics, pages 1346–1352, Geneva. find
  55. Riedel, Sebastian and James Clarke. 2006. Incremental integer linear programming for non-projective dependency parsing. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 129–137, Sydney. find
  56. Riezler, Stefan, Tracy H. King, Richard Crouch, and Annie Zaenen. 2003. Statistical sentence condensation using ambiguity packing and stochastic disambiguation methods for lexical-functional grammar. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the… find
  57. Roark, Brian. 2001. Probabilistic top–down parsing and language modeling. Computational Linguistics, 27(2):249–276.
  58. Roth, Dan and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the 8th Conference on Computational Natural Language Learning, pages 1–8, Boston, MA.
  59. Scott, Donia and Clarisse Sieckenius de Souza. 1990. Getting the message across in RST-based text generation. In Robert Dale, Chris Mellish, and Michael Zock, editors, Current Research in Natural Language Generation. Academic Press, New York, pages 47–73.
  60. Sjorochod’ko, E. F. 1972. Adaptive method for automatic abstracting and indexing. In Information Processing 71: Proceedings of the IFIP Congress 71, pages 1179–1182, Amsterdam. find
  61. Tetreault, Joel R. 2001. A corpus-based evaluation of centering and pronoun resolution. Computational Linguistics, 27(4):507–520.
  62. Teufel, Simone and Marc Moens. 2002. Summarizing scientific articles— Experiments with relevance and rhetorical status. Computational Linguistics, 28(4):409–446.
  63. Turner, Jenine and Eugene Charniak. 2005. Supervised and unsupervised learning for sentence compression. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 290–297, Ann Arbor, MI. find
  64. Vanderbei, Robert J. 2001. Linear Programming: Foundations and Extensions. Kluwer Academic Publishers, Boston, 2nd edition. find
  65. Walker, Marilyn, Aravind Joshi, and Ellen Prince. 1998. Centering in naturally occurring discourse: An overview. In Centering Theory in Discourse. Oxford University Press, Oxford, pages 1–28. find
  66. Winston, Wayne L. and Munirpallam Venkataramanan. 2003. Introduction to Mathematical Programming. Brooks/Cole, Independence, KY. find
  67. Wolf, Florian and Edward Gibson. 2004. Paragraph-, word-, and coherence-based approaches to sentence ranking: A comparison of algorithm and human performance. In Proceedings of the 42nd Meeting of the Association for Computational Linguistics, pages 383–390, Barcelona. find
  68. Zajic, David, Bonnie J. Dorr, Jimmy J. Lin, and Richard M. Schwartz. 2007. Multi-candidate reduction: Sentence compression as a tool for document summarization tasks. Information Processing and Management, 43(6):1549–1570. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/clarke-2010-discourse-constraints-document-compression/