Elamur Research⌕
☰
home / papers / 2004 / MEAD - A Platform for Multidocument Multilingual Text Summarization

MEAD - A Platform for Multidocument Multilingual Text Summarization

Summary

Research on extractive multidocument summarization lacked a shared, freely available toolkit where competing sentence-scoring methods and evaluation measures could be compared under the same conditions. This paper describes MEAD, an open-source environment (release 3.07) for summarizing clusters of English and Chinese documents. At the time of writing, more than 500 organizations had downloaded it. MEAD processes a cluster in four steps. It first converts the documents into its XML format, then computes per-sentence features as specified in a .meadrc configuration. A classifier merges those features into one score, and a reranker adjusts the scores for cross-sentence effects such as redundancy or chronology. Every data object (clusters, queries, feature tables, judgements, extracts) has its own DTD. The bundled MEADEval toolkit offers co-selection metrics, content-overlap metrics and an extrinsic retrieval-based measure. The paper reports no new benchmark scores. Its contribution is the infrastructure itself, together with the SummBank and CSTBank corpora and downstream systems such as NewsInEssence and WebInEssence. This infrastructure makes summarization experiments reproducible and easy to extend.

Key points

  • Sentence features: Centroid, SimWithFirst, Length (a cutoff), RealLength, Position, QueryOverlap, KeyWordMatch and LexPageRank (eigenvector centrality over a thresholded lexical similarity graph).
  • Classifiers and rerankers:
    • Classifiers: Default (a linear feature combination), Lead-based and Random baselines, and a Weka-based Decision-tree. The introduction also mentions trainable SVM and Maximum Entropy options.
    • Rerankers: Identity, Default (cosine-threshold redundancy removal), Time-based, Source-based, CST-based and MMR.
  • MEADEval: merged into MEAD in v3.07. It supports precision/recall, Kappa, Relative Utility, cosine, simple cosine, word overlap, longest common subsequence and BLEU. Relevance correlation (Spearman/Pearson) is available as an add-on.
  • SummBank:
    • It was built at the 2001 Johns Hopkins summer workshop.
    • It contains 40 English/Chinese news clusters and 360 human-written non-extractive multidocument summaries.
    • It also holds close to 2 million automatic and manual extracts.
  • CSTBank and use: CSTBank is a smaller corpus annotated for CST relations such as paraphrase, subsumption and elaboration. MEAD was used in DUC and TREC (QA and novelty detection) and runs under UNIX (Linux/Solaris/Cygwin).

Interesting ideas

  • I1. Pipeline split into features, classifier and reranker: Sentence-level scoring is kept separate from global decisions like removing redundancy. Each part can therefore be swapped or ablated on its own, which suits any extractive pipeline, including one for Ukrainian news.
  • I2. One schema per artifact type: Every data object (features, judgements, extracts, queries) has its own XML schema. This lets independent scripts and teams share intermediate results without custom glue code, which is a good pattern for a shared research library.
  • I3. Graded utility judgements: Several annotators give each sentence a graded utility score instead of a yes/no label. Relative Utility then credits near-miss choices and accounts for disagreement between annotators, which helps wherever a single gold extract is too strict.
  • I4. Extrinsic check via retrieval: Relevance correlation tests whether ranking documents by their summaries reproduces the ranking obtained from the full texts. This gives a cheap task-based quality signal that needs no reference summaries.
  • I5. Baselines inside the tool: Lead and random baselines ship as ordinary classifiers. Every experiment can then report them at no extra cost, which keeps claims of improvement honest.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

65 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 61 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{radev2004mead,
  title = {{MEAD - A Platform for Multidocument Multilingual Text Summarization}},
  author = {Radev, Dragomir and Allison, Timothy and Blair-Goldensohn, Sasha and Blitzer, John and Çelebi, Arda and Dimitrov, Stanko and Drabek, Elliott and Hakim, Ali and Lam, Wai and Liu, Danyu and Otterbacher, Jahna and Qi, Hong and Saggion, Horacio and Teufel, Simone and Topper, Michael and Winkel, Adam and Zhang, Zhu},
  booktitle = {Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC'04)},
  year = {2004},
  doi = {10.63317/2nevc4eu3swt},
  url = {https://doi.org/10.63317/2nevc4eu3swt}
}

References

13 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Carbonell, Jaime G. and Jade Goldstein, 1998. The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. In Alistair Moffat and Justin Zobel (eds.), Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval… find
  2. Otterbacher, Jahna C., Dragomir R. Radev, and Airong Luo, 2002. Revisions that improve cohesion in multi-document summaries: a preliminary study. In Proceedings of the Workshop on Automatic Summarization (including DUC 2002). Philadelphia: Association for Computational Linguistics. find
  3. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu, 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL). find our card
  4. Radev, Dragomir, 2000. A common theory of information fusion from multiple text sources, step one: Cross-document structure. In Proceedings, 1st ACL SIGDIAL Workshop on Discourse and Dialogue. Hong Kong. find
  5. Radev, Dragomir, Sasha Blair-Goldensohn, and Zhu Zhang, 2001a. Experiments in single and multi-document summarization using MEAD. In First Document Understanding Conference. New Orleans, LA. find
  6. Radev, Dragomir, Simone Teufel, Horacio Saggion, Wai Lam, John Blitzer, Arda Çelebi, Hong Qi, Elliott Drabek, and Danyu Liu, 2002. Evaluation of text summarization in a cross-lingual information retrieval framework. Technical report, Center for Language and Speech Processing, Johns Hopkins Universi… find
  7. Radev, Dragomir R., Sasha Blair-Goldensohn, Zhu Zhang, and Revathi Sundara Raghavan, 2001b. Interactive, domain-independent identification and summarization of topically related news articles. In 5th European Conference on Research and Advanced Technology for Digital Libraries. Darmstadt, Germany. find
  8. Radev, Dragomir R., Sasha Blair-Goldensohn, Zhu Zhang, and Revathi Sundara Raghavan, 2001c. Newsinessence: A system for domain-independent, real-time news clustering and multi-document summarization. In Human Language Technology Conference (Demo Session). San Diego, CA. find
  9. Radev, Dragomir R., Weiguo Fan, and Zhu Zhang, 2001d. Webinessence: A personalized web-based multi-document summarization and recommendation system. In NAACL Workshop on Automatic Summarization. Pittsburgh, PA. find
  10. Radev, Dragomir R., Hongyan Jing, Malgorzata Stys, and Daniel Tam, 2004. Centroid-based summarization of multiple documents. Information Processing and Management, in press. find
  11. Radev, Dragomir R., Jahna Otterbacher, Hong Qi, and Daniel Tam, 2003. Mead reducs: Michigan at duc 2003. In Proceedings of DUC 2003. Edmonton, AB, Canada. find
  12. Witten, Ian H. and Eibe Frank, 2000. Data Mining: Practical machine learning tools with Java implementations. San Francisco: Morgan Kaufmann. find
  13. Zhang, Zhu, Sasha Blair-Goldensohn, and Dragomir Radev, 2002. Towards CST-enhanced summarization. In Proceedings of the AAAI 2002 Conference. Edmonton, Alberta. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/radev-2004-mead-platform-multidocument-multilingual-text/