Elamur Research⌕
☰
home / papers / 2015 / Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution

Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution

Summary

Coreference systems have to decide whether each mention starts a new entity or refers back to an earlier one. Most extracted mentions are non-anaphoric, and simple cues such as head, number or gender agreement often match pairs that are not actually coreferent. Earlier strong systems worked around this with linear models built on large, hand-designed or decision-tree-induced feature conjunctions. Wiseman, Rush, Shieber and Weston replace these with a non-linear mention-ranking neural network that takes only raw, unconjoined features. The network learns two separate hidden representations: one for anaphoricity (whether a mention has any antecedent) and one for scoring antecedent–mention pairs. Each representation is first pre-trained on its own subtask, and the full model is then trained on the complete ranking objective. Mentions are extracted with the Berkeley Coreference System. On the CoNLL 2012 English test set, the model achieved the best CoNLL score reported up to that point. It uses only local inference, so it is simpler than competing systems that rely on global or structured decoding.

Key points

  • The score combines an anaphoricity representation h_a and a pairwise representation h_p, each a tanh layer over raw features. They are joined either directly (g1) or through an extra hidden layer (g2). Both versions perform similarly, but their errors differ.
  • The model uses two raw feature sets: BASIC (the BCS features) and BASIC+, which adds number, gender, animacy, named-entity type, person and bilexical head features.
  • The model gains almost 2 CoNLL points over a comparable linear mention-ranking model and more than 1.5 points over the previous state-of-the-art system on CoNLL 2012 English.
  • BCS mention extraction recovers about 96.4% of truly anaphoric mentions in the training set, with roughly 3.5 non-anaphoric mentions for every anaphoric one.
  • Error analysis shows that pleonastic pronouns ("it", "you") cause more errors than nominal mentions that share no head with their antecedent. These pronouns are usually linked to another instance of the same pronoun. Because nearly 30% of anaphoric pronouns in the dev data belong to pronoun-only clusters, the authors doubt that a simple non-local rule against such clusters would help.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

75 papers of our library cite it (of the 140673 whose bibliographies we read); 35 of them have a page here, 40 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-1137

@inproceedings{wiseman2015learning,
  title = {{Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution}},
  author = {Wiseman, Sam and Rush, Alexander M. and Shieber, Stuart and Weston, Jason},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1137},
  url = {https://doi.org/10.3115/v1/p15-1137}
}

References

43 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amit Bagga and Breck Baldwin. 1998. Algorithms for Scoring Coreference Chains. In The first international conference on language resources and evaluation workshop on linguistics coreference, volume 1, pages 563–566. Citeseer. find our card
  2. Yonatan Belinkov, Tao Lei, Regina Barzilay, and Amir Globerson. 2014. Exploring Compositional Architectures and Word Vector Representations for Prepositional Phrase Attachment. Transactions of the Association for Computational Linguistics, 2:561–572. ↗ our card
  3. Eric Bengtson and Dan Roth. 2008. Understanding the Value of Features for Coreference Resolution. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 294–303. ACL. find our card
  4. Anders Björkelund and Richárd Farkas. 2012. Data-driven Multilingual Coreference Resolution using Resolver Stacking. In Joint Conference on EMNLP and CoNLL-Shared Task, pages 49–55. ACL. find our card
  5. Anders Björkelund and Jonas Kuhn. 2014. Learning structured perceptrons for coreference Resolution with Latent Antecedents and Non-local Features. ACL, Baltimore, MD, USA, June. find our card
  6. Kai-Wei Chang, Rajhans Samdani, Alla Rozovskaya, Mark Sammons, and Dan Roth. 2012. Illinois-coref: The UI System in the CoNLL-2012 Shared Task. In Joint Conference on EMNLP and CoNLL-Shared Task, pages 113–117. Association for Computational Linguistics. find
  7. Kai-Wei Chang, Rajhans Samdani, and Dan Roth. 2013. A Constrained Latent Variable Model for Coreference Resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 601–612. find our card
  8. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural Language Processing (almost) from Scratch. The Journal of Machine Learning Research, 12:2493–2537. find our card
  9. Aron Culotta, Michael Wick, Robert Hall, and Andrew McCallum. 2006. First-order Probabilistic Models for Coreference Resolution. NAACL-HLT. find our card
  10. Pascal Denis and Jason Baldridge. 2008. Specialized Models and Ranking for Coreference Resolution. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 660–669. ACL. find our card
  11. Pascal Denis, Jason Baldridge, et al. 2007. Joint Determination of Anaphoricity and Coreference Resolution using Integer Programming. In HLT-NAACL, pages 236–243. Citeseer. find our card
  12. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. The Journal of Machine Learning Research, 12:2121–2159. find
  13. Greg Durrett and Dan Klein. 2013. Easy Victories and Uphill Battles in Coreference Resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1971– 1982. find our card
  14. Greg Durrett and Dan Klein. 2014. A Joint Model for Entity Analysis: Coreference, Typing, and Linking. Transactions of the Association for Computational Linguistics, 2:477–490. ↗ our card
  15. Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, and Samy Bengio. 2010. Why does unsupervised pre-training help deep learning? The Journal of Machine Learning Research, 11:625–660. find
  16. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. Liblinear: A Library for Large Linear Classification. The Journal of Machine Learning Research, 9:1871–1874. ↗
  17. Eraldo Rezende Fernandes, Cı́cero Nogueira Dos Santos, and Ruy Luiz Milidiú. 2012. Latent Structure Perceptron with Feature Induction for Unrestricted Coreference Resolution. In Joint Conference on EMNLP and CoNLL-Shared Task, pages 41–48. Association for Computational Linguistics. find our card
  18. Aria Haghighi and Dan Klein. 2010. Coreference Resolution in a Modular, Entity-centered Model. In The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 385–393. Association for Computational Linguistics. find our card
  19. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. Ontonotes: the 90% Solution. In Proceedings of the human language technology conference of the NAACL, Companion Volume: Short Papers, pages 57–60. Association for Computational Linguistics. find our card
  20. Philipp Koehn. 2004. Statistical Significance Tests for Machine Translation Evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 388–395. Citeseer. find
  21. Jonathan K. Kummerfeld and Dan Klein. 2013. Error-driven Analysis of Challenges in Coreference Resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA, October. find our card
  22. Emmanuel Lassalle and Pascal Denis. 2013. Improving Pairwise Coreference Models through Feature Space Hierarchy Learning. In ACL 2013-Annual meeting of the Association for Computational Linguistics. find our card
  23. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s Multi-pass Sieve Coreference Resolution System at the CoNLL-2011 Shared Task. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 28–34. Association for Computational Linguistics. find our card
  24. Heeyoung Lee, Angel Chang, Yves Peirsman, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2013. Deterministic Coreference Resolution based on Entity-centric, Precision-ranked Rules. Computational Linguistics, 39(4):885–916. ↗ our card
  25. Xiaoqiang Luo, Sameer Pradhan, Marta Recasens, and Eduard Hovy. 2014. An Extension of BLANC to System Mentions. Proceedings of ACL, Baltimore, Maryland, June. find
  26. Xiaoqiang Luo. 2005. On Coreference Resolution Performance Metrics. In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 25–32. Association for Computational Linguistics. find our card
  27. Chao Ma, Janardhan Rao Doppa, J Walker Orr, Prashanth Mannem, Xiaoli Fern, Tom Dietterich, and Prasad Tadepalli. 2014. Prune-and-score: Learning for Greedy Coreference Resolution. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. find our card
  28. Andrew McCallum and Ben Wellner. 2003. Toward Conditional Models of Identity Uncertainty with Application to Proper Noun Coreference. Advances in Neural Information Processing Systems 17. find
  29. Yurii Nesterov. 1983. A Method of Solving a Convex Programming Problem with Convergence Rate O (1/k2). In Soviet Mathematics Doklady, volume 27, pages 372–376. find
  30. Vincent Ng and Claire Cardie. 2002. Identifying Anaphoric and Non-anaphoric Noun Phrases to Improve Coreference Resolution. In Proceedings of the 19th international conference on Computational linguistics-Volume 1, pages 1–7. Association for Computational Linguistics. find
  31. Vincent Ng. 2004. Learning Noun Phrase Anaphoricity to Improve Coreference Resolution: Issues in Representation and Optimization. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, page 151. Association for Computational Linguistics. find
  32. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL-Shared Task, pages 1–40. ACL. find our card
  33. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation. In Proceedings of the Association for Computational Linguistics. find our card
  34. Altaf Rahman and Vincent Ng. 2009. Supervised Models for Coreference Resolution. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2-Volume 2, pages 968–977. ACL. find our card
  35. Marta Recasens, Marie-Catherine de Marneffe, and Christopher Potts. 2013. The Life and Death of Discourse Entities: Identifying Singleton Mentions. In HLT-NAACL, pages 627–633. find
  36. Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Manning. 2011. Semi-supervised Recursive Autoencoders for Predicting Sentiment Distributions. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 151–161. ACL. find our card
  37. Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic Compositionality through Recursive Matrix-vector Spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201–1211. ACL. find
  38. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A Machine Learning Approach to Coreference Resolution of Noun Phrases. Computational Linguistics, 27(4):521–544. ↗ our card
  39. Veselin Stoyanov and Jason Eisner. 2012. Easy-first Coreference Resolution. In COLING, pages 2519– 2534. Citeseer. find our card
  40. Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. 2013. On the Importance of Initialization and Momentum in Deep Learning. In Proceedings of the 30th International Conference on Machine Learning, pages 1139–1147. find
  41. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A Model-theoretic Coreference Scoring Scheme. In Proceedings of the 6th conference on Message Understanding, pages 45–52. ACL. find our card
  42. Michael Wick, Sameer Singh, and Andrew McCallum. 2012. A Discriminative Hierarchical Model for Fast Coreference at Large Scale. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 379–388. Association for Computational Linguistics. find our card
  43. Chun-Nam John Yu and Thorsten Joachims. 2009. Learning Structural SVMs with Latent Variables. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 1169–1176. ACM. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wiseman-2015-learning-anaphoricity-antecedent-ranking-features/