Elamur Research⌕
☰
home / papers / 2004 / Cross-Document Coreference on a Large Scale Corpus

Cross-Document Coreference on a Large Scale Corpus

Summary

The paper addresses cross-document coreference: deciding whether identical person names in different documents refer to the same individual. Earlier work was tested only on a small collection, so the authors built a much larger evaluation set, the "Person-x corpus". They drew documents from TREC volumes 1–3 using topic queries across 11 subject areas, tagged names with BBN's Identifinder, picked one person per document, and replaced that name with "person-x". The original names served as gold coreference chains. Each mention is represented by a 55-word context window (a "snippet"). Three ways of clustering snippets are compared: an incremental vector space approach approximating Bagga and Baldwin (1998), KL divergence, and agglomerative vector space. All are scored with B-CUBED. Agglomerative vector space gave the best F-measure on the original John Smith corpus and was far less sensitive to the choice of threshold. Its drawback is quadratic runtime. The study matters because it offers a scalable way to build test data and shows that results from small corpora may not carry over to large ones.

Key points

  • John Smith corpus: 197 New York Times articles mentioning 35 distinct John Smiths. Person-x corpus: 34,404 documents covering 14,767 real entities, roughly 46.7% of which appear only once.
  • On John Smith, agglomerative vector space reached a peak F of 88.2%. The incremental method scored 84.3%, close to Bagga and Baldwin's 84.6%, and a single-link variant scored 81.4%.
  • Evaluating separately within each genre lowered agglomerative F by 6 points to 77% and incremental F by about 5 points. KL divergence rose 3 points to 77%, matching agglomerative.
  • A 55-word snippet window maximized F-measure on the John Smith corpus and was used in all experiments.
  • Agglomerative clustering is O(n²) and took about 3× longer on Person-x. The authors consider this acceptable when the number of same-name mentions is small.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

26 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 21 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{gooi2004cross,
  title = {{Cross-Document Coreference on a Large Scale Corpus}},
  author = {Gooi, Chung Heong and Allan, James},
  booktitle = {Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004},
  year = {2004},
  doi = {10.21236/ada458579},
  url = {https://doi.org/10.21236/ada458579}
}

References

10 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Allan, James, ed. Topic Detection and Tracking: Event-based Information Organization, Kluwer Academic Publishers, 2002. find
  2. Allan, James and Raghavan, Hema. Entity Models: Construction and Application. Center for Intelligent Information Retrieval, Department of Computer Science, University of Massachusetts, 2002. find
  3. Bagga, Amit and Breck Baldwin. Algorithms for Scoring Coreference Chains. In Proceedings of the Linguistic Coreference Workshop at The First International Conference on Language Resources and Evaluation (LREC'98), pp563-566, 1998. find our card
  4. Bagga, Amit and Breck Baldwin. Entity-Based Cross-Document Coreferencing Using the Vector Space Model. Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics (COLING-ACL'98),pp.79-85, 1998. find
  5. Bagga, Amit and Breck Baldwin. Coreference as the Foundations for Link Analysis Over Free Text Databases. In Proceedings of the COLING-ACL'98 Content Visualization and Intermedia Representations Workshop (CVIR'98), pp. 19-24, 1998. find
  6. Bagga, Amit, and Biermann, Alan. A Methodology for Cross-Document Coreference. In Proceedings of the Fifth Joint Conference on Information Sciences (JCIS 2000), pp. 207-210, 2000. find
  7. Bagga, Amit and Baldwin, Breck. How Much Processing Is Required for Cross-Document Coreference? In Proceedings of the Linguistic Coreference Workshop (LREC'98), pp. 563-566, 1998. find
  8. BBN Technologies. Rough ‘n’ Ready™: Audio Indexing for Meetings and News. http://www.bbn.com/-speech/roughnready.html (2001) ↗
  9. Kibble, Rodger and Kees, van Deemter. Coreference Annotation: Whither? Proceedings of LREC2000, Athens, pp. 1281-1286, 2000. find
  10. Lee, Lillian. On the Effectiveness of the Skew Divergence for Statistical Language Analysis, Technical Report, Department of Computer Science, Cornell University, 2001. find

This page in Markdown → https://md.research.elamur.ai/free/papers/gooi-2004-cross-document-coreference-large-scale/