Elamur Research⌕
☰
home / papers / 1998 / Generating Natural Language Summaries from Multiple On-Line Sources

Generating Natural Language Summaries from Multiple On-Line Sources

Paper · 1998 · Dragomir R. Radev and Kathleen R. McKeown

Summary

Radev and McKeown tackle information overload in online news. Statistical extractors of the time summarized one article at a time by lifting sentences, so they could not relate sources to each other, and the extracted sentences often read poorly together. The authors propose SUMMONS, a knowledge-based system that writes user-focused "briefings" from several articles about the same or related events, together with historical background. It works in the terrorism domain and takes as input templates produced by DARPA MUC information-extraction systems, recording slots such as perpetrators, victims and event type. A combiner merges templates across articles and over time to reveal agreements, generalizations, changes and contradictions. A sentence planner and a sentence generator then produce fluent English. Separately, SUMMONS harvests descriptions of people, places and organizations from past online news and stores them in its generation formalism, so later summaries can include them. The work matters because it was an early demonstration of symbolic multi-document summarization through language generation rather than extraction.

Key points

  • System: SUMMONS ("SUMMarizing Online NewS articles"). Its components include a combiner, a sentence planner, a sentence generator, a description categorizer and an extended summary generator.
  • Input and domain: It relies on MUC (MUC-4 terrorism) template outputs rather than raw text. Porting it to another domain would require new templates and new extraction rules.
  • Handling disagreement: When sources conflict, for example on casualty counts, SUMMONS reports both figures and attributes each to its source instead of choosing one.
  • Reusable resource: Entity descriptions are extracted from online newswire and reused in later summaries. This part is presented as independent of the domain.
  • Motivation and gaps in the extract: The paper cites more than 100 live newswire sources and more than 40 search services on the Web. The system-status and evaluation sections were missing from the extracted text, so no quantitative results are given here.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Dragomir R. Radev and Kathleen R. McKeown. 1998. Generating Natural Language Summaries from Multiple On-Line Sources. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@article{radev1998generating,
  title = {{Generating Natural Language Summaries from Multiple On-Line Sources}},
  author = {Dragomir R. Radev and Kathleen R. McKeown},
  journal = {Computational Linguistics},
  year = {1998},
  doi = {10.7916/d8z328hj},
  url = {https://doi.org/10.7916/d8z328hj}
}

References

75 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aberdeen, John, John Burger, Dennis Connolly, Susan Roberts, and Marc Vilain. 1992. MITRE-Bedford: Description of the ALEMBIC system as used for MUC-4. In Proceedings of the Fourth Message Understanding Conference (MUC-4), pages 215-222, McLean, VA, June. find
  2. Aho, Alfred, Shih-Fu Chang, Kathleen McKeown, Dragomir Radev, John Smith, and Kazi Zaman. 1997. Columbia digital news system: An environment for briefing and search over multimedia information. In Proceedings of ADL, Washington, DC, April. find
  3. Altavista. 1996. WWW site, URL: http://altavista.digital.com. ↗
  4. Aysuo, Damaris, Sean Boisen, Heidi Fox, Herb Gish, Robert Ingria, and Ralph Weischedel. 1992. BBN: Description of the PLUM system as used for MUC-4. In Proceedings of the Fourth Message Understanding Conference (MUC-4), pages 169-176, McLean, VA, June. find
  5. Barzilay, Regina and Michael Elhadad. 1997. Using lexical chains for text summarization. In Proceedings of the Workshop on Intelligent Scalable Text Summarization, pages 10-17, Madrid, Spain, August. Association for Computational Linguistics.
  6. Berners-Lee, Tim. 1992. World-Wide Web: The information universe. Electronic Networking, 2(1):52-58. find
  7. Boguraev, Branimir and Christopher Kennedy. 1997. Salience-based content characterization of text documents. In Proceedings of the Workshop on Intelligent Scalable Text Summarization, pages 2-9, Madrid, Spain, August. Association for Computational Linguistics. find
  8. Borko, H. 1975. Abstracting Concepts and Methods. Academic Press, New York. find
  9. Bourbeau, Laurent, Denis Carcagno, E. Goldberg, Richard Kittredge, and Alain Polgu~re. 1990. Bilingual generation of weather forecasts in an operations environment. In Hans Karlgren, editor, Proceedings of the 13th International Conference on Computational Linguistics (COLING-90), volume 3, pages 31… find
  10. Brandow, Ronald, Karl Mitze, and Lisa F. Rau. 1990. Automatic condensation of electronic publications by sentence selection. Information Processing and Management, 26:135–170. find
  11. Church, Kenneth W. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Proceedings of the Second Conference on Applied Natural Language Processing (ANLP-88), pages 136-143, Austin, TX, February. Association for Computational Linguistics.
  12. ClariNet. 1996. WWW site, URL: http://www.clari.net. ↗
  13. CNN Interactive. 1996. WWW site, URL: http://www.crm.com. ↗
  14. Coates-Stephens, Sam. 1991. Automatic lexical acquisition using within-text descriptions of proper nouns. In Proceedings of the Seventh Annual Conference of the UW Centre for the New OED and Text Research, pages 154-169. find
  15. Cowie, Jim, Louise Guthrie, Yorick Wilks, James Pustejovsk~ and Scott Waterman. 1992. CRL/NMSU and Brandeis: Description of the MucBruce system as used for MUC-4. In Proceedings of the Fourth Message Understanding Conference (MUC-4), pages 223-232, McLean, VI, June. find
  16. Cuts, Short. 1994. Science and Technology Section. Economist, 17:85-86, December. find
  17. Dalianis, Hercules and Edward Hovy. 1993. Aggregation in natural language generation. Proceedings of the 4th European Workshop on Natural Language Generation. find
  18. DejaNews. 1997. WWW site, URL: http://www.dejanews.com. ↗
  19. DeJong, G. F. 1979. Skimming Stories in Real Time: An Experiment in Integrated Understanding. Ph.D. thesis, Computer Science Department, Yale University. find
  20. Duford, Darrin. 1993. CREP: A regular expression-matching textual corpus tool. Technical Report CUCS-005-93, Columbia University. find
  21. Elhadad, Michael. 1991. FUF: The universal unifier — user manual, version 5.0. Technical Report CUCS-038-91, Columbia University. find
  22. Elhadad, Michael. 1993. Using Argumentation to Control Lexical Choice: A Unij~'cation-based Implementation. Ph.D. thesis, Computer Science Department, Columbia University. find
  23. Endres-Niggemeyer, Brigitte. 1993. An empirical process model of abstracting. In Workshop on Summarizing Text for Intelligent Communication, Dagstuhl, Germany, December. find
  24. Feiner, Steven and Kathleen McKeown. 1991. Automating the generation of coordinated multimedia explanations. IEEE Computer, 24(10):33-41, October. find
  25. Fisher, David, Stephen Soderland, Joseph McCarthy Fangfang Feng, and Wendy Lehnert. 1995. Description of the UMass system as used for MUC-6. In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 221-236. find
  26. Genesereth, Michael and Steven Ketchpel. 1994. Software agents. Communications of the ACM, 37(7):48-53, July. find
  27. Hahn, Udo. 1990. Topic parsing: accounting for text macro structures in full-text analysis. Information Processing and Management, 26:135–170. find
  28. Halliday, Michael and Ruqaiya Hasan. 1976. Cohesion in English. English Language Series. Longman, London. find
  29. Hov~ Eduard H. 1988. Planning coherent multisentential text. In Proceedings of the 26th Annual Meeting of the Association for Computational Linguistics, Buffalo, NY, June. Association for Computational Linguistics. find
  30. Hovy, Eduard and Chin Yew Lin. 1997. Automated text summarization in SUMMARIST. In Proceedings of the Workshop on Intelligent Scalable Text Summarization, pages 18-24, Madrid, Spain, August. Association for Computational Linguistics. find
  31. Iordanskaja, Lidija, M. Kim, Richard Kittredge, Benoit Lavoie, and Alain Polgu~re. 1994. Generation of extended bilingual statistical reports. In Proceedings of the 15th International Conference on Computational Linguistics (COLING-94), Kyoto, Japan. find
  32. Jing, Hongyan, Regina Barzilay, and Kathleen McKeown. 1998. Summarization evaluation methods: Experiments and analysis. In Symposium on Intelligent Text Summarization, Stanford, CA, March. find
  33. Kukich, Karen K. 1983. Design of a knowledge-based report generator. In Proceedings of the 21st Annual Meeting, pages 145-150, Cambridge, MA, June. Association for Computational Linguistics.
  34. Kukich, Karen, Rebecca Passonneau, Kathleen McKeown, Dragomir Radev, Vasileios Hatzivassiloglou, and Hongyan Jing. 1997. Software re-use and evolution in text generation applications. In ACL/EACL Workshop - From Research to Commercial Applications: Making NLP Technology Work in Practice, Madrid, Spa… find
  35. Kupiec, Julian M. 1993. MURAX: A robust linguistic approach for question answering using an on-line encyclopedia. In Proceedings, 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. find
  36. Kupiec, Julian M., Jan Pedersen, and Francine Chen. 1995. A trainable document summarizer. In Proceedings, 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 68-73, Seattle, WA, July. find
  37. Lehnert, Wendy, Joe McCarthy, Stephen Soderland, Ellen Riloff, Claire Cardie, Jonathan Peterson, and Fangfang Feng. 1993. UMass/Hughes: Description of the CIRCUS system used for MUC-5. In Proceedings of the Fifth Message Understanding Conference (MUC-5), pages 277-291, Baltimore, MD, August. find
  38. Luhn, Hans P. 1958. The automatic creation of literature abstracts. IBM Journal, pages 159-165. find
  39. Lycos, Inc. 1996. Home Page. WWW site, URL: http://www.lycos.com. ↗
  40. Mani, Inderjeet and Eric Bloedorn. 1997. Multi-document summarization by graph search and matching. In Proceedings of the Fourteenth National Conference on Artificial Intelligence (AAAI-97), pages 622-628, Providence, RI. American Association for Artificial Intelligence. find
  41. Mani, Inderjeet, Richard T. Macmillan, Susann Luperfoy, Elaine Lusher, and Sharon Laskowski. 1993. Indentifying unknown proper names in newswire text. In Proceedings of the Workshop on Acquisition of Lexical Knowledge from Text, pages 44–54, Columbus, OH, June. Special Interest Group on the Lexicon… find
  42. Marcu, Daniel. 1997. From discourse structures to text summaries. In Proceedings of the Workshop on Intelligent Scalable Text Summarization, pages 82-88, Madrid, Spain, August. Association for Computational Linguistics. find
  43. McDonald, David D. 1993. Internal and external evidence in the identification and semantic categorization of proper names. In Proceedings of the Workshop on Acquisition of Lexical Knowledge from Text, pages 32-43, Columbus, OH, June. Special Interest Group on the Lexicon of the Association for Compu… find
  44. McDonald, David D. and James D. Pustejovsky. 1986. Description-directed natural language generation. In Proceedings of the 9th IJCAL pages 799-805. IJCAI. find
  45. McKeown, Kathleen R. 1985. Text Generation: Using Discourse Strategies and Focus Constraints to Generate Natural Language Texts. Cambridge University Press, Cambridge, England. find
  46. McKeown, Kathleen R., Karen Kukich, and James Shaw. 1994a. Practical issues in automatic documentation generation. In Proceedings of the 4th Conference on Applied Natural Language Processing, Stuttgart, Germany, October. Association for Computational Linguistics. find
  47. McKeown, Kathleen R., Karen K. Kukich, and James Shaw. 1994b. Practical issues in automatic documentation generation. In Proceedings of the ACL Applied Natural Language Conference, Stuttgart, Germany, October. find
  48. McKeown, Kathleen R. and Dragomir Radev. 1995. Generating summaries of multiple news articles. In Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 74-82, Seattle, WA, July. find
  49. McKeown, Kathleen R., Jacques Robin, and Karen Kukich. 1995. Generating concise natural language summaries. Journal of Information Processing and Management, 31(5):703-733. find
  50. McKeown, Kathleen R., Jacques Robin, and Michael Tanenblatt. 1993. Tailoring lexical choice to the user's vocabulary in multimedia explanation generation. In Proceedings of the 31st Annual Meeting, Columbus, OH., June. Association for Computational Linguistics. find
  51. Miller, George A., Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine J. Miller. 1990. Introduction to WordNet: An on-line lexical database. International Journal of Lexicography (Special Issue), 3(4):235-312. find
  52. Mitra, Mandar, Amit Singhal, and Chris Buckley. 1997. Automatic text summarization by paragraph extraction. In Proceedings of the Workshop on Intelligent Scalable Text Summarization, pages 39-46, Madrid, Spain, August. Association for Computational Linguistics. find
  53. MUC, Message Understanding Conference. 1992. Proceedings of the Fourth Message Understanding Conference (MUC-4). DARPA Software and Intelligent Systems Technology Office. find
  54. NetSumrn. 1996. Home Page. WWW site, URL: http://www.labs.bt.com/innovate/informat/netsumm/index.htm. ↗
  55. New York Times. 1996. WWW site, URL: http://www.nytimes.com. ↗
  56. Paice, Chris. 1990. Constructing literature abstracts by computer: Techniques and prospects. Information Processing and Management, 26:171-186. find
  57. Paik, Woojin, Elizabeth D. Lidd~ Edmund Yu, and Mary McKenna. 1994. Interpretation of proper nouns for information retrieval. In Proceedings of the Human Language Technology Workshop, pages 309-313, Plainsboro, NJ, March. ARPA Software and Intelligent Systems Technology Office, Morgan Kaufmann, San… find
  58. Passormeau, Rebecca, Karen Kukich, Kathleen McKeown, Dragomir Radev, and Hongyan Jing. 1997. Summarizing Web traffic: A portability exercise. Technical Report CUCS-009-97, Columbia University, Department of Computer Science, New York. find
  59. Preston, Keith and Sandra Williams. 1994. Managing the information overload. Physics in Business, June. find
  60. Radev, Dragornir R. 1996. An architecture for distributed natural language summarization. In Proceedings of the 8th International Workshop on Natural Language Generation: Demonstrations and Posters, pages 45-48, Herstmonceux, England, June. find
  61. Radev, Dragomir R. and Kathleen R. McKeown. 1997. Building a generation knowledge source using internet-accessible newswire. In Proceedings of the 5th Conference on Applied Natural Language Processing, Washington, DC, April. find
  62. Rau, Lisa F. 1988. Conceptual information extraction and information retrieval from natural language input. In Proceedings, RAIO-88: Conference on User-Oriented, Content-Based, Text and Image Handling, pages 424-437, Cambridge, MA. find
  63. Rau, Lisa F., Ron Brandow, and Karl Mitze. 1994. Domain-independent summarization of news. In Summarizing Text for Intelligent Communication, pages 71-75, Dagstuhl, Germany. find
  64. Reuters News. 1996. WWW site, URL: http://www.yahoo.com/headlines/. ↗
  65. Robin, Jacques. 1994. Revision-Based Generation of Natural Language Summaries Providing Historical Background. Ph.D. thesis, Computer Science Department, Columbia University. find
  66. Robin, Jacques and Kathleen R. McKeown, 1993. Corpus analysis for revision-based generation of complex sentences. In Proceedings of the 11 th National Conference on Artificial Intelligence, Washington, DC, July. find
  67. Robin, Jacques and Kathleen R. McKeown. 1995. Empirically designing and evaluating a new revisior~-based model for summary generation. Artificial Intelligence Journal. In press. find
  68. R6sner, Michael. 1987. SEMTEX: A text generator for German. In Gerard Kempen, editor, Natural Language Generation: New Results in Artificial Intelligence, Psychology, and Linguistics. Martinus Ninjhoff Publishers. find
  69. Rothkegel, Annely. 1993. Abstracting from the perspective of text production. In Workshop on Summarizing Text for Intelligent Communication, Dagstuhl, Germany, December. find
  70. Shaw, James. 1995. Conciseness through aggregation in text generation. In Proceedings of the 33rd Association for Computational Linguistics Annual Meeting (Student Session), pages 329-331. find
  71. Spar& Jones, K. 1993. What might be in a summary? In Proceedings of Information Retrieval 93: Von der Modellierung zur Anwendung, pages 9-26, Universitatsverlag Knstanz. find
  72. Tait, John I. 1983. Automatic Summarsing of English Texts. Ph.D. thesis, University of Cambridge, Cambridge, England. find
  73. Wacholder, Nina, Yael Ravin, and Misook Choi. Disambiguation of proper names in text. In Proceedings of the Fifth Applied Natural Language Processing Conference, Washington DC. Association for Computational Linguistics. find
  74. Weischedel, Ralph, Damaris Ayuso, Sean Boisen, Heidi Fox, Robert Ingria, Tomoyoshi Matsukawa, Constantine Papageorgiou, Dawn McLaughlin, Masaichiro Kitagawa, Tsutomu Sakai, June Abe, Hiroto Hosihi, Yoichi Miyamoto, and Scott Miller. 1993. BBN: Description of the PLUM system as used for MUC-5. In Pro… find
  75. Young, S. R. and P. J. Hayes. 1985. Automatic classification and summarization of banking telexes. In Proceedings of the Second Conference on Artificial Intelligence Applications, pages 402-408. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/radev-1998-generating-natural-language-summaries-multiple/