Elamur Research⌕
☰
home / papers / 2000 / Multi-Document Summarization By Sentence Extraction

Multi-Document Summarization By Sentence Extraction

Paper · 2000 · Jade Goldstein, Vibhu Mittal, Jaime Carbonell et al.

Summary

The paper looks at how to summarize a whole set of related documents, such as hundreds of news stories returned by one search query. It argues that multi-document summarization is not just single-document summarization repeated. Related articles share a lot of background, news streams change over time, coreference across sources is harder, and the summary must be far shorter relative to the input. The authors propose an extraction-based system that selects sentences using fast, domain-independent statistical processing. It adds a metric that penalizes redundancy and rewards diversity among the chosen passages. The framework is modular, so it can be tuned to different genres, corpus properties and user needs. The paper lists requirements for such systems and several strategies for building summaries, and it shows a sample summary ordered by time. Evaluation was still being set up: 10-document sets with human gold-standard summaries and user ratings. The work matters because it frames redundancy control and extreme compression as the central problems in summarizing large document clusters.

Key points

  • The paper names four differences from single-document summarization: higher redundancy across documents, a temporal dimension where later reports may override earlier ones, much stronger compression, and harder coreference.
  • SUMMAC tested summaries at 10% compression. For 200-document clusters, the authors needed compression of roughly 1% or even 0.1%.
  • In an informal test, three people summarized 10 articles about the Microsoft Trial. Each produced a very different summary: one on courtroom details, one on the day's gist, and one on the trial's goals and outcome.
  • Six extraction strategies are listed, including common sections only, a centroid document, and centroid or latest document plus outliers. The proposed strategy combines common and unique sections with time weighting.
  • The planned evaluation has two parts:
    • Users sort sentences into clusters and build their own summaries, which serve as a gold standard.
    • Users rate system sentences as good, okay or bad, and suggest better replacements for weak ones.
    • Summarizer #6 is compared with #1, which takes common portions with no anti-redundancy, and with #3, a centroid-document summary.

Interesting ideas

  • I1. Treat redundancy as a first-class objective. When choosing passages, score them against what has already been selected, not only against the query. This is useful wherever outputs overlap heavily, such as search results, news aggregation or retrieval for RAG.
  • I2. Make the presentation order a configurable module. Passages can be ordered by document rank, by importance first, by topic cluster or along a time line. A deliberate ordering choice can change how readable the same set of sentences is.
  • I3. Centroid plus outliers. Summarize the most typical document, then add a little material from atypical ones. This is a cheap way to combine the core story with coverage of side information, and it carries over to clustering of any text collection.
  • I4. Ask raters for replacements, not just scores. When a sentence is judged okay or bad, the rater picks a better sentence from the source. This produces richer feedback that can also be used for training.
  • I5. Plan for update summaries and inconsistency detection. New summaries should account for earlier ones, and conflicting figures across sources should be flagged. Both matter for monitoring news, trade or other data streams over time.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States365 works · 158,048 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Summarization · 159 works

Cite

Cite as Jade Goldstein, Vibhu Mittal, Jaime Carbonell and Mark Kantrowitz. 2000. Multi-Document Summarization By Sentence Extraction. NAACL-ANLP 2000 Workshop: Automatic Summarization.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{goldstein2000multi,
  title = {{Multi-Document Summarization By Sentence Extraction}},
  author = {Jade Goldstein, Vibhu Mittal, Jaime Carbonell and Mark Kantrowitz},
  booktitle = {NAACL-ANLP 2000 Workshop: Automatic Summarization},
  year = {2000},
  doi = {10.3115/1117575.1117580},
  url = {https://doi.org/10.3115/1117575.1117580}
}

References

33 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. James Allan, Jaime Carbonell, George Doddington,, Jonathan Yamron, and Yiming Yang. 1998. Topic detection and tracking pilot study: Final report. In Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop. find
  2. Chinatsu Aone, M. E. Okurowski, J. Gorlinsky, and B. Larsen. 1997. A scalable summarization system using robust NLP. In Proceedings of the ACL'97/EACL'97 Workshop on Intelligent Scalable Text Summarization, pages 66-73, Madrid, Spain. find
  3. Breck Baldwin and Thomas S. Morton. 1998. Dynamic coreference-based summarization. In Proceedings of the Third Conference on Empirical Methods in Natural Language Processing (EMNLP-3), Granada, Spain, June.
  4. Regina Barzilay and Michael Elhadad. 1997. Using lexical chains for text summarization. In Proceedings of the ACL'97/EACL'97 Workshop on Intelligent Scalable Text Summarization, pages 10-17, Madrid, Spain.
  5. Branimir Boguraev and Chris Kennedy. 1997. Salience based content characterization of text documents. In Proceedings of the ACL'97/EACL'97 Workshop on Intelligent Scalable Text Summarization, pages 2-9,. Madrid, Spain. find
  6. Chris Buckley. 1985. Implementation of the SMART information retrieval system. Technical Report TR 85-686, Cornell University. find
  7. Jaime G. Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of SIGIR-98, Melbourne, Australia, August. find
  8. Jade Goldstein and Jaime Carbonell. 1998. The use of mmr and diversity-based reranking in document reranking and summarization. In Proceedings of the 14th Twente Workshop on Language Technology in Multimedia Information Retrieval, pages 152-166, Enschede, the Netherlands, December. find
  9. Jade Goldstein, Mark Kantrowitz, Vibhu O. Mittal, and • Jaime G. Carbonell. 1999. Summarizing Text Documents: Sentence Selection and Evaluation Metrics. Irf Proceedings of the 22nd International ACM SIGIR I i I I ! find
  10. I ! I I ! I I Conference on Research and Development in Information Retrieval (S1G1R-99), pages 121-128, Berkeley, CA. find
  11. Eduard Hovy and Chin-Yew Lin. 1997. Automated text summarization in SUMMARIST. In ACUEACL-97 Workshop on Intelligent Scalable Text Summarization, pages 18-24, Madrid, Spain, July. find
  12. Judith L. Klavans and James Shaw. 1995. Lexical semantics in summarization. In Proceedings of the First Annual Workshop of the IFIP Working Group FOR NLP and KR, Nantes, France, April. find
  13. Julian M. Kupiec, Jan Pedersen, and Francine Chen. 1995. A trainable document summarizer. In Proceedings of the 18th Annual Int. ACM/SIG1R Coaference on Research and Development in IR, pages 68-73, Seattle, WA, July. find
  14. P. H. Luhn. 1958. Automatic creation of literature abstracts. IBM Journal, pages 159-165. find
  15. Inderjeet Mani and Eric Bloedern. 1997. Multi-document summarization by graph search and merging. In Proceedings of AAA1-97, pages 622–628. AAAI. find
  16. Inderjeet Mani and Eric Bloedom. 1999. Summarizing similarities and differences among related documents. Information Retrieval, 1:35-67. find
  17. Daniel'Marcu. 1997. From discourse structures to text summaries. In Proceedings of the ACL'97/EACL'97 Workshop on Intelligent Scalable Text Summarization, pages 82-88, Madrid, Spain. find
  18. Kathleen McKeown, Jacques Robin, and Karen Kukich. 1995. Designing and evaluating a new revision-based model for summary generation. Info. Proc. and Management, 31 (5). find
  19. Kathleen McKeown, Judith Klavans, Vasileios Hatzivassiloglou, Regina Barzilay, and Eleazar Eskin. 1999. Towards Multidocument Summarization by Reformulation: Progress and Prospects. In Proceedings of AAAI-99, pages 453–460, Orlando, FL, July. find
  20. Mandar Mitra, Amit Singhal, and Chris Buckley. 1997. Automatic text summarization by paragraph extraction. In ACL/EACL-97 Workshop on Intelligent Scalable Text Summarization, pages 31-36, Madrid, Spain, July. find
  21. Chris D. Paice. 1990. Constructing literature abstracts by computer: Techniques and prospects. Info. Proc. and Management, 26:171-186. find
  22. Dragomir Radev and Kathy McKeown. 1998. Generating natural language summaries from multiple online sources. Compuutational Linguistics. find
  23. Gerald Salttm. 1970. Automatic processing of foreign language docuemnts. Journal of American Society for Information Sciences, 21:187-194. find
  24. Gerald Salton. 1989. Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley. find
  25. James Shaw. 1995. Conciseness through aggregation in text generation. In Proceedings of 33rd Association for Computational Linguistics, pages 329-331. find
  26. Gees C. Stein, Tomek Strzalkowski, and G. Bowden Wise. 1999. Summarizing Multiple Documents Using Text Extraction and Interactive Clustering. In Proceedings of PacLing-99: The Pacific Rim Conference on Computational Linguistics, pages 200-208, Waterloo, Canada. find
  27. Tomek Strzalkowski, Jin Wang, and Bowden Wise. 1998. A robust practical text summarization system. In AAAI Intelligent Text Summarization Workshop, pages 26-30, Stanford, CA, March. find
  28. J. I. Tait. 1983. Automatic Summarizing of English Texts. Ph.D. thesis, University of Cambridge, Cambridge, UK. find
  29. Simone Teufel and Marc Moens. 1997. Sentence extraction as a classification task. In ACL/EACL-97 Workshop on Intelligent Scalable Text Summarization, pages 58-65, Madrid, Spain, July. find
  30. TIPSTER. 1998a. Tipster text phase III 18-month workshop notes, May. find
  31. Fairfax, VA. TIPSTER. 1998b. Tipster text phase III 24-month workshop notes, October. find
  32. Baltimore, MD. Charles J. van Rijsbergen. 1979. Information Retrieval. Butterworths, London. Yiming Yang, Tom Pierce, and Jaime 13. find
  33. Carbonell. 1998. A study on retrospective and on-line event detection. In Proceedings of the 21th Ann lnt ACM SI-G1R Conference on Research and Development in Information Retrieval ( SIGIR'98), pages 28-36. :Yiming Yang, Jaime G. Carbonell, Ralf D. Brown, Tom Pierce, Brian T. Archibald, and Xin Liu.… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/goldstein-2000-multi-document-summarization-sentence-extraction/