Elamur Research⌕
☰
home / papers / 1999 / What is coreference, and what should coreference annotation be?

What is coreference, and what should coreference annotation be?

Summary

This position paper asks which semantic relation corpora actually encode when they are annotated for "coreference." It focuses on the MUC-6/MUC-7 effort and its public Task Definition (TD). The authors use a strict definition: two NP occurrences corefer only if they denote the same entity. This makes coreference an equivalence relation, unlike anaphora, which is directional and depends on context.

They show that the TD asks annotators to link NPs that refer to nothing at all, such as quantified, most indefinite, and predicative NPs. Its handling of intensional, time-varying values, like a changing stock price, is unclear and even self-contradictory. The resulting relation, which the TD calls IDENT, therefore has no coherent rationale. The authors add that separating markable detection from linking is difficult in practice.

They propose limiting annotation to genuine coreference, which yields less data but more reliable data. This rethink should come before extensions to part/whole or class/instance links. Re-annotating a MUC-6 sample shows that little valid information would be lost.

Key points

  • The MUC TD sets four ranked goals: support the MUC task, reach 95% inter-annotator agreement, allow fast and cheap annotation, and serve linguists outside MUC. The authors argue that unclear definitions make these goals very hard to meet.
  • Three classes of non-referring NPs are still linked under IDENT: quantifying NPs ("Every man"), most indefinites, and predicative NPs. A substitution test can show that such NPs do not corefer with their anaphors.
  • The TD is inconsistent on intensionality. Section 1.3 links "the stock price" only to its most recent value. Section 6.4, however, links markables that the text equates at any point in time.
  • Appendix: a MUC-6 Wall Street Journal excerpt from MUC (1997) contains four non-singleton classes, with 7, 2, 2 and 3 NPs. Only the fourth class, the loss amounts, is not true coreference, and its links are questionable anyway.
  • The authors call for closer collaboration between the coreference annotation community and computational semantics. The work was linked to the GNOME project (EPSRC grant GR/L51126).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{deemter1999what,
  title = {{What is coreference, and what should coreference annotation be?}},
  author = {Deemter, Kees van and Kibble, Rodger},
  booktitle = {Coreference and Its Applications},
  year = {1999},
  doi = {10.3115/1608810.1608828},
  url = {https://doi.org/10.3115/1608810.1608828}
}

References

10 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. ABPI (1997). 1996-1997 ABPI Compendium of Patient Information Leaflets. Association of the British Pharmaceutical Industry. find
  2. BAGGA i . ET AL. (1999). Coreference and its Applications. Call for papers for workshop associated with ACL'99. See www. ca. duke. edu/~amit/ac199-wkshp, html MUC (1997). MUC-7 Coreference Task Definition, 13 July 1997. See www.muc.saic.com DAVIES S. ET AL. (1998). tating Coreference in Dialogues: p… find
  3. Anno-Pro-See www. cogsci, ed. ac. uk/-,~poesio/MATE/anno_manual. html DOWTY D. ET AL. (1981). DoTty, D., Wall, R. & Peters, S. Introduction to Montague Semantics. Dordrecht: Kluwer. find
  4. GAMUT L. T. F. (1991). Logic, Language and Meaning Volume 2. Chicago and London: University of Chicago Press. find
  5. HIRSCHMAN L., ROBINSON P., BURGER J. & VILAIN M. (1997). Automating Coreference: The Role of Annotated Training Data. In Proceedings of A A A I Spring Symposium on Applying Machine Learning to Discourse • Processing. find
  6. KAMP H. & REYLE U. (1993). From Discourse to Logic. Dordrecht: Kluwer Academic Publishers.• LYONS J. (1977). Semantics. Cambridge University Press. find
  7. Cambridge: POESIO, M., R. HENSCHEL, J. HITZEMAN, R. KIBBLE, S. MONTAGUE, AND K. VAN DEEMTER. (1999). Towards an Annotation Scheme for Noun Phrase Generation. To appear in H. Uszkoreit et al. (eds), EACL-99 Workshop on Linguistically Interpreted Corpora, Bergen. find
  8. POPESCU-BELIS A. (1998). How Corpora with Annotated Coreference Links Improve Reference Resolution. In R. ANTONIO ET AL., Eds., First Int. Conf. on Language Resources and Evaluation, p. 567-572. Granada: European Language Resources Association. find
  9. POPESCU-BELIS A. & ROBBA I. (1998). Three New Methods for Evaluating Reference Resolution. In First International Conference on Language Resources 8J Evaluation: Workshop on Linguistic Coreference. Granada: European Language Resources Association. find
  10. SIDNER C. (1983). Focusing in the comprehension of definite anaphora. InM.Brady and R.Berwick eds. Computational Models of Discourse. Cambridge, Mass:MIT Press. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/deemter-1999-what-is-coreference-what-should/