Elamur Research
☰
home / papers / 2001 / A Machine Learning Approach to Coreference Resolution of Noun Phrases

A Machine Learning Approach to Coreference Resolution of Noun Phrases

What it is about

The paper presents a learned, end-to-end coreference resolver for general noun phrases (not only pronouns), trained on a small annotated corpus. A pipeline of HMM-based modules first finds the markables: tokenisation, POS tagging, NP identification, named entities and nested NPs. Each pair of markables is described by 12 shallow features: distance, pronoun flags, string match, definiteness, demonstratives, number, semantic class, gender, both-proper-names, alias, and apposition. A C5 decision tree classifies pairs, and closest-first clustering builds the chains. On MUC-6 the system reaches recall 58.6%, precision 67.3% and F 62.6. On MUC-7 it reaches recall 56.1%, precision 65.5% and F 60.4. Only 6.5% of the 20,910 MUC-6 training pairs are positive. The three features alias, string match and apposition alone come within 2.3 F points (MUC-6) and 1 point (MUC-7) of the full system.

Ideas

For Mova

The code is coref/src/sieve.rs (the pairwise features), coref/src/mention.rs (markables) and coref/src/eval.rs. The chapters are languages/en/35-reference-and-coreference.md and, for number, languages/en/11-nouns.md. The measure is MUC F on the same dev and test splits, with the ablation above. Run every configuration twice and check that the results match exactly.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/J01-4004.pdf

Summary

The paper looks at coreference resolution in unrestricted text, meaning the task of deciding whether two noun phrases point to the same real-world entity. It covers all kinds of noun phrases, not only pronouns, and all entity types. The authors first find candidate "markables" with a pipeline of HMM-based modules: a part-of-speech tagger, noun phrase chunking, named entity recognition, and extraction of nested and possessive noun phrases. Each candidate pair is then described by 12 cheap features: sentence distance, whether each phrase is a pronoun, string match, whether the later phrase is definite or demonstrative, number agreement, and semantic class agreement based on WordNet. A decision-tree classifier learns from a small annotated corpus and decides whether two markables corefer. Tested on the standard MUC-6 and MUC-7 coreference data, the system scores about as well as the best rule-based systems. The authors say it is the first learning-based system to do so. This showed that a small amount of annotation can replace hand-written resolution rules.

Key points

  • Evaluated on the MUC-6 and MUC-7 coreference corpora. The reported F-measures are roughly 62.6% on MUC-6 and 60.4% on MUC-7. These figures are not in the supplied excerpt, so they should be checked against the full paper.
  • The markable pipeline found about 85% of the noun phrases in the gold coreference chains of 100 MUC-6 training documents.
  • Semantic classes (male, female, person, organization, location, date, time, money, percent, object) form a small ISA hierarchy. Each class is mapped to a WordNet synset, and every head noun is given its first, most frequent sense.
  • The method does far better than the unsupervised clustering of Cardie and Wagstaff (1999), which scored 53.6% F on MUC-6. Learning curves show that only a modest number of annotated documents is needed.

Topics: Coreference resolution

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.1162/089120101753342653

@article{soon2001machine,
  title = {{A Machine Learning Approach to Coreference Resolution of Noun Phrases}},
  author = {Soon, Wee Meng and Ng, Hwee Tou and Lim, Daniel Chung Yong},
  journal = {Computational Linguistics},
  year = {2001},
  doi = {10.1162/089120101753342653},
  url = {https://doi.org/10.1162/089120101753342653}
}

References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aone, Chinatsu and Scott William Bennett. 1995. Evaluating automated and manual acquisition of anaphora resolution strategies. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, 122-129. find
  2. Baldwin, Breck. 1997. CogNIAC: High precision coreference with limited knowledge and linguistic resources. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 38-45. find our card
  3. Bikel, Daniel M., Richard Schwartz, and Ralph M. Weischedel. 1999. An algorithm that learns what's in a name. Machine Learning, 34(1-3):211-231, February. find
  4. Cardie, Claire and Kiri Wagstaff. 1999. Noun phrase coreference as clustering. In Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 82-89. find
  5. Chinchor, Nancy A. 1998. Overview of MUC-7/MET-2. In Proceedings of the Seventh Message Understanding Conference (MUC-7). http://www.itl.nist.gov/iad/894.02/related_projects/muc/proceedings/muc_7_toc.html. ↗
  6. Church, Kenneth. 1988. A stochastic parts program and noun phrase parser for unrestricted text. In Proceedings of the Second Conference on Applied Natural Language Processing, pages 136-143. find
  7. Dietterich, Thomas G. 1998. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation, 10(7):1895-1924, October. find
  8. Fisher, David, Stephen Soderland, Joseph McCarthy, Fangfang Feng, and Wendy Lehnert. 1995. Description of the UMass system as used for MUC-6. In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 127-140. find
  9. Ge, Niyu, John Hale, and Eugene Charniak. 1998. A statistical approach to anaphora resolution. In Proceedings of the Sixth Workshop on Very Large Corpora, pages 161-170. find
  10. Kameyama, Megumi. 1997. Recognizing referential links: An information extraction perspective. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 46-53. find
  11. Kehler, Andrew. 1997. Probabilistic coreference in information extraction. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, pages 163-173. find
  12. Lappin, Shalom and Herbert J. Leass. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics, 20(4):535-561, December. find our card
  13. McCarthy, Joseph F. 1996. A Trainable Approach to Coreference Resolution for Information Extraction. Ph.D. thesis, University of Massachusetts Amherst, Department of Computer Science, September. find
  14. McCarthy, Joseph F. and Wendy Lehnert. 1995. Using decision trees for coreference resolution. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pages 1050-1055. find
  15. McEnery, A., I. Tanaka, and S. Botley. 1997. Corpus annotation and reference resolution. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 67-74. find
  16. Miller, George A. 1990. WordNet: An on-line lexical database. International Journal of Lexicography, 3(4):235-312. find
  17. Mitkov, Ruslan. 1997. Factors in anaphora resolution: They are not the only things that matter. A case study based on two different approaches. In Proceedings of the ACL Workshop on Operational Factors in Practical, Robust Anaphora Resolution for Unrestricted Texts, pages 14-21. find
  18. MUC-6. 1995. Coreference task definition (v2.3, 8 Sep 95). In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 335-344. find
  19. MUC-7. 1997. Coreference task definition (v3.0, 13 Jul 97). In Proceedings of the Seventh Message Understanding Conference (MUC-7). find
  20. Quinlan, John Ross. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco, CA. find our card
  21. Sundheim, Beth M. 1995. Overview of results of the MUC-6 evaluation. In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 13-31. find