📄 Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies

Summary

The paper tackles generic summarization of news event clusters, meaning groups of articles from several agencies that describe one developing story. The authors build MEAD, which reuses cluster centroids from their CIDR topic detection and tracking system. A centroid is a pseudo-document of words with high Count*IDF across the whole cluster. MEAD scores each sentence by how much centroid vocabulary it carries, optionally adds a position (lead) bonus, and penalizes word overlap with higher-ranked sentences. It then extracts the top sentences for a chosen compression rate.

The authors also argue that precision/recall against a single ideal extract is a weak evaluation. Instead, they propose two judge-based notions:

• CBSU: each sentence gets a 0–10 relevance score for the cluster topic.
• CSIS: records when one sentence's information is contained in another's.

On a small corpus of 6 clusters, user studies showed high judge agreement on CBSU but much weaker agreement on CSIS. Against a multi-document Lead baseline, the centroid-plus-position variant came out ahead in most test conditions. This matters because it gives a cheap, query-independent way to summarize clusters and a reusable way to evaluate them at any length.

Key points

• The evaluation corpus has 558 sentences in 27 documents across 6 clusters, all produced by CIDR. Four clusters come from Usenet newsgroups and two from TDT-3 (topics 78 and 67). Cluster sizes range from 2 to 10 articles.
• Every cluster was summarized at 9 compression rates (10%–90%), giving 54 test cases.
• The pure centroid version of MEAD beat the Lead baseline in 29 of 54 cases. The lead+centroid version (w_c = 1, w_p = 1) beat it in 41 of 54. Cluster D favored MEAD at every compression rate.
• Redundancy is measured as R_s = 2 × overlapping words / (sum of both sentence lengths). A repeated word counts only min(m, n) times. Reranking is repeated until the extract stops changing.
• On CSIS, five judges fully agreed on 292 of 558 sentences, but 291 of those were "no subsumption" verdicts. Of 80 sentences involving subsumption, only 24 reached agreement from four or more judges.

Interesting ideas

• I1. Reuse clustering output as a summarization signal. The topic-detection centroid doubles as a profile of what the cluster is about. Any pipeline that already clusters texts can rank sentences against it at almost no extra cost.
• I2. Collect graded sentence utility instead of one gold extract. Judges score every sentence once, so summaries of any length can be evaluated offline without new annotation. This also avoids the fragility of picking a single "ideal" cut-off.
• I3. Treat redundancy as directional containment, not just similarity. Asking whether one sentence's content is contained in another's gives a sharper notion than plain similarity. Grouping mutual containment into equivalence classes could help deduplicate parallel reporting from several news sources.
• I4. Iterate the redundancy penalty to a fixed point. Subtracting an overlap penalty relative to better-ranked sentences, then re-ranking until the selection is stable, is a simple MMR-like heuristic. It is easy to plug into other extractive selectors.
• I5. Choose a strong baseline when random is already built in. Because chance-level selection was already part of the normalized score, the authors compared against a positional lead baseline instead. This keeps reported gains meaningful.