📄 Appendix F: MUC-7 Coreference Task Definition (version 3.0)

Summary

This document is the official annotation guideline for the coreference task at MUC-7. It explains how to mark noun phrases in a text that refer to the same entity. The scheme supports downstream information extraction, especially the Template Element and Scenario Template tasks, and also aims to produce a reusable corpus for discourse research. The authors rank four goals: usefulness for extraction, about 95% inter-annotator agreement, fast and cheap annotation, and research value. To keep agreement high, they annotate only the symmetric, transitive IDENT relation between markables (nouns, NPs, pronouns, dates, money and percentages). Set/subset, part/whole and clause-level links are left out. Annotation uses inline SGML COREF tags. The ID/REF attributes encode the links, MIN gives the head-based minimal span a system must match, and STATUS="OPT" marks optional links. When marking every value of a changing quantity would merge distinct values into one equivalence class, the guideline keeps the classes clean and accepts the lost links. Scoring gives the same result however the coreference chains are encoded.

Key points

• Only the IDENT relation is annotated. Because it is symmetric and transitive, the links form equivalence classes, and scoring does not depend on which REF pointers form a chain.
• The SGML attributes are ID, REF, TYPE (always IDENT), MIN (minimal head string for full credit) and STATUS=OPT (optional anaphor links).
• Collapsing chains are avoided. For example, in a sentence where a stock price falls from $4.02 to $3.85, only the latest value is linked to the price.
• Wh-question phrases and phrases whose antecedent is a clause are not annotated. In speech, disfluent or "erased" material is excluded.
• Appendix examples come from a MUC-6 Wall Street Journal article and a TRAINS dialogue transcript. Planned extensions include clause-level coreference, discontinuous or conjoined elements, a type vs. instance distinction, and set/subset and part/whole relations.