📄 Entity-Based Cross-Document Coreferencing Using the Vector Space Model

Summary

The paper addresses cross-document coreference: deciding whether mentions of the same name in different texts refer to one real-world entity. The focus is on people who share a name, a case that named-entity tools such as NetOwl and Textract cannot separate. The pipeline starts with CAMP, Penn's within-document coreference system. For each article, a SentenceExtractor gathers every sentence that mentions the chosen entity's coreference chain, which yields an entity-focused summary. VSM-Disambiguate then compares summaries pairwise using a standard tf-idf vector space model with cosine normalization, and merges pairs whose similarity exceeds a threshold. The test set is 197 New York Times articles matching a "John Smith" pattern. The authors also propose B-CUBED, a per-mention scoring method, because the MUC link-based scorer rewards bad merges too generously in this setting. The best F-measure is 84.6%. A same-name baseline gets only 23% precision under B-CUBED. The work matters for multi-document summarization, information fusion and tracking entities across collections.

Key points

• The data is 197 NYT articles from 1996–1997, all matching the regex /John.*?Smith/. They mention 35 distinct John Smiths: 24 appear in a single article, and the other 173 articles cover 11 people. No NYT data was used for training.
• The architecture is CAMP (a top system in MUC-6/MUC-7) → SentenceExtractor → VSM-Disambiguate. Similarity is the sum of products of tf × log(N/df) weights over shared stemmed, stop-word-filtered terms, normalized by vector length.
• The best B-CUBED F-measure is 84.6% when the term space is built from the extracted summaries only. It drops to 78.0% when the space is built from the full articles.
• Treating all John Smiths as one person scores 23% precision and 100% recall under B-CUBED. Under the MUC scorer the same baseline gets 83% precision.
• In a worked example, the MUC scorer gives 90% precision to two responses of clearly different quality. B-CUBED separates them, giving 58% and 76%.

Interesting ideas

• I1. Reuse within-document coreference to build entity profiles. Resolving pronouns and short name forms first lets you collect facts about an entity that would otherwise be missed, such as a job title stated only in a sentence with "he". This applies to any entity linking or profile-building task, including morphologically rich languages.
• I2. Query-focused summaries instead of whole documents. Comparing only the sentences about the target entity reduces noise from unrelated content. Here it gave a clear F-measure gain, which suggests that narrowing the context can beat richer input.
• I3. Turn the cross-document problem into a within-document one for evaluation. Treating each document as a mention inside a "meta document" lets existing coreference scorers run unchanged. This is a cheap way to bootstrap evaluation for new clustering-style tasks.
• I4. Score per element, not per link. B-CUBED computes precision and recall for each item and averages them. As a result, wrongly merging two large clusters costs more than attaching a small one. This is useful for evaluating any clustering, deduplication or record-linkage system where error sizes differ.
• I5. A deliberately ambiguous stress-test set. Selecting documents by a single very common name with a simple regex gives a hard and cheap benchmark for disambiguation. The same idea could be applied to frequent names or toponyms in other languages.