Elamur Research⌕
☰
home / papers / 2006 / Coreference Resolution with and without Linguistic Knowledge

Coreference Resolution with and without Linguistic Knowledge

Summary

Statistical coreference resolution systems of the mid-2000s typically used sophisticated models but only 10–20 simple features. This paper asks whether a much richer, linguistically informed feature set helps. Uryupina builds a resolver with 351 automatically computed features covering surface similarity, syntax, semantics and discourse salience. The inputs are Charniak parser output, the C&C NER tagger and WordNet. The system is evaluated on MUC-7 using the pairwise (anaphor, candidate antecedent) set-up of Soon et al. (2001), and its reimplemented feature set serves as the baseline. Five learners are compared. With the extended features, most learners improve, mainly because recall rises, though usually at some cost in precision. SVM light benefits most, reaching an F-score of 65.4%, which the author reports as the best published MUC-7 result at the time. Learning curves show that the baseline features saturate after a few documents, while the rich set keeps improving. This suggests that the small training corpus, not the features, limits performance, and that more annotated data (e.g. ACE) could bring larger gains.

Key points

  • Feature set: 351 nominal features, expanded to 1096 binary features for SVM light and Maxent. They comprise 122 name-matching, 64 syntactic, 29 semantic (including four WordNet similarity measures) and 136 discourse/salience features.
  • Data: MUC-7, with 30 "dry-run" training documents (5049 markables) and 20 "formal" test documents (3369 markables). Training yields 34,601 instances, of which 1703 are positive and 32,898 negative.
  • Learners: RIPPER, SLIPPER, C4.5, SVM light and Maxent were tested.
    • SLIPPER collapsed to the single-chain baseline with both feature sets.
    • Maxent was the only learner whose precision and recall both rose.
  • Best result: SVM light with the extended features reached an F-score of 65.4% on MUC-7.
  • Learning curves: Training used 10, 15, 20, 25 and 30 documents. The extended set performs poorly with little data but improves steadily without converging, pointing to overfitting from too little data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

10 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 5 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{uryupina2006coreference,
  title = {{Coreference Resolution with and without Linguistic Knowledge}},
  author = {Uryupina, Olga},
  booktitle = {Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06)},
  year = {2006},
  url = {https://aclanthology.org/L06-1453/}
}

References

31 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chris Barker and Geoffrey K. Pullum. 1990. A theory of command relations. Linguistics and Philosophy, 13(1):1–34. find
  2. Alexander Budanitsky and Graeme Hirst. 2001. Semantic distance in wordnet: An experimental, application-oriented evaluation of five measures. In Proceedings of the NAACL Workshop on WordNet and Other Lexical Resources. find
  3. Claire Cardie and Kiri Wagstaff. 1999. Noun phrase coreference as clustering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 82–89. find our card
  4. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the 1st Meeting of the North American Chapter of the Association for Computational Linguistics, pages 132–139. find our card
  5. William W. Cohen. 1995. Fast effective rule induction. In Proceedings of the 12th International Conference on Machine Learning, pages 115–123. find
  6. Dan Cristea, Oana Postolache, and Ruslan Mitkov. 2002. Handling complex anaphora resolution cases. In Proceedings of the 4th Discourse Anaphora and Anaphor Resolution Colloquium. find
  7. James Curran and Stephen Clark. 2003a. Investigating gis and smoothing for maximum entropy taggers. In Proceedings of the 11th Meeting of the European Chapter of the Association for Computational Linguistics, pages 91–98. find
  8. James R. Curran and Stephen Clark. 2003b. Language independent NER using a maximum entropy tagger. In Proceedings of the Seventh Conference on Natural Language Learning, pages 164–167. find
  9. Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995. Centering: A framework for modelling the local coherence of discourse. Computational Linguistics, 21(2):203–226. find
  10. Sanda Harabagiu, Răzvan Bunescu, and Steven Maiorano. 2001. Text and knowledge mining for coreference resolution. In Proceedings of the 2nd Meeting of the North American Chapter of the Association for Computational Linguistics, pages 55–62. find
  11. Lynette Hirschman and Nancy Chinchor. 1997. MUC-7 coreference task definition. In Message Understanding Conference Proceedings. find our card
  12. Jerry Hobbs. 1978. Resolving pronoun references. Lingua, 44:311–338. find our card
  13. Thorsten Joachims. 1999. Making large-scale SVM learning practical. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT-Press. find
  14. Shalom Lapin and Herbert J. Leass. 1994. An algorithm for pronominal anaphora resolution. Computational LInguistics, 20(4):535–561. find our card
  15. Zhang Le, 2004. Maximum Entropy Modelling Toolkit for Python and C++. find
  16. Xiaoqiang Luo, Abe Ittycheriah, Hongyan Jing, Nanda Kambhatla, and Salim Roukos. 2004. A mention-synchronous coreference resolution algorithm based on the bell tree. In Proceedings of the 42nd Annual Meeting of the Association for Computationa Linguistics. find our card
  17. Andrew McCallum and Ben Wellner. 2003. Toward conditional models of identity uncertainty with application to proper noun coreference. In Proceedings of the IJCAI Workshop on Information Integration on the Web. find
  18. Joseph McCarthy and Wendy Lehnert. 1995. Using decision trees for coreference resolution. In Proceedings of the 14th International Conference on Artificial Intelligence, pages 1050–1055. find
  19. George Miller. 1990. Wordnet: An on-line lexical database. International Journal of Lexicography. find
  20. Vincent Ng and Claire Cardie. 2002a. Combining sample selection and error-driven pruning for machine learning of coreference rules. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find our card
  21. Vincent Ng and Claire Cardie. 2002b. Improving machine learning approaches to coreference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computationa Linguistics, pages 104–111. find our card
  22. J. Ross Quinlan. 1993. C4.5: Programs for Machine Learning. Morgan Kaufman. find our card
  23. Jeffrey C. Reynar and Adwait Ratnaparkhi. 1997. A maximum entropy approach to identifying sentence boundaries. In Proceedings of the Fifth Conference on Applied Natural Language Processing. find
  24. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics (Special Issue on Computational Anaphora Resolution), 27(4):521–544. find our card
  25. Michael Strube, Stefan Rapp, and Christof Müller. 2002. The influence of minimum edit distance on reference resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 312– 319. find
  26. Olga Uryupina. 2004a. Evaluating name-matching for coreference resolution. In Proceedings of the Language Resources and Evaluation Conference. find
  27. Olga Uryupina. 2004b. Linguistically motivated sample selection for coreference resolution. In Proceedings of the 4th Discourse Anaphora and Anaphor Resolution Colloquium. find
  28. Vladimir V. Vapnik. 1995. The Nature of Statistical Learning Theory. Springer. find
  29. Renata Vieira and Massimo Poesio. 2000. An empirically-based system for processing definite descriptions. Computational Linguistics, 26(4):539–593. find
  30. Renata Vieira. 1999. Applying inductive decision trees in resolution of definite NPs. In Proceedings of the Argentine Symposium on Artificial Intelligence. find
  31. Xiaofeng Yang, Jian Su, Guodong Zhou, and Chew-Lim Tan. 2004. Improving pronoun resolution by incorporating coreferential information of candidates. In Proceedings of the 42nd Annual Meeting of the Association for Computationa Linguistics. find our card

↑ back to top