Decentralized Entity-Level Modeling for Coreference Resolution

Summary

Coreference systems gain from entity-level information such as gender or semantic type, but exact inference over all possible partitions of mentions is intractable. Earlier entity-level systems therefore made greedy left-to-right or sieve-style decisions that cannot be revised later. Durrett, Hall and Klein propose a discriminative log-linear model built as a factor graph. Each mention chooses an antecedent or starts a new chain, and these choices are scored by rich pairwise features. Each mention also holds its own distribution over property values, and structural equality factors push linked mentions toward the same value. Entity information is thus spread across mentions without ever building explicit entities, and approximate inference is done with belief propagation. The authors test three kinds of properties on the CoNLL 2011 shared task data: noisy oracle properties, phi features, and unsupervised semantic clusters. The TRANSITIVE model matches or beats a pairwise baseline and a reimplemented left-to-right entity model (Rahman and Ng, 2009). It performs on par with the Stanford system, the winner of the shared task. This shows that global, entity-aware coreference can stay computationally tractable.

Key points

The architecture has three layers. Antecedent variables use unary pairwise factors (the BASIC model), mention-level property nodes carry priors, and equality factors take effect only when a mention links to that antecedent. A projection layer maps raw cluster values to "coreference-adapted" ones.
Pairwise features fall into five groups: anaphoricity, configuration (distance, c-command, appositives), string and head match, agreement (gender, number, animacy, NER), and speaker or discourse cues. Each is conjoined with mention types. WordNet features did not help.
The semantic clusters come from NAIVE BAYES and CONDITIONAL models over (noun head, role+governor) pairs. They were trained with 30 EM iterations on English Gigaword, using parses from the Berkeley Parser and SRL from Senna.
On development data averaged over ten training splits, noisy oracle properties raised the average F1 from 59.97 (BASIC) to 63.70 (TRANSITIVE).
On the CoNLL 2011 test set, TRANSITIVE reached an average F1 of 58.62, compared with 58.61 for Stanford and 57.97 for BASIC. It achieved the best MUC F1 at 63.73.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

18 papers of our library cite it (of the 140673 whose bibliographies we read); 14 of them have a page here, 4 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{durrett2013decentralized,
  title = {{Decentralized Entity-Level Modeling for Coreference Resolution}},
  author = {Durrett, Greg and Hall, David and Klein, Dan},
  booktitle = {Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2013},
  url = {https://aclanthology.org/P13-1012/}
}

References

42 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Amit Bagga and Breck Baldwin. 1998. Algorithms for Scoring Coreference Chains. In Proceedings of the Conference on Language Resources and Evaluation Workshop on Linguistics Coreference. find our card
Mohit Bansal and Dan Klein. 2012. Coreference Semantics from Web Features. In Proceedings of the Association for Computational Linguistics. find our card
Eric Bengtson and Dan Roth. 2008. Understanding the Value of Features for Coreference Resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find our card
Shane Bergsma and Dekang Lin. 2006. Bootstrapping Path-Based Pronoun Resolution. In Proceedings of the Conference on Computational Linguistics and the Association for Computational Linguistics. find our card
David Burkett and Dan Klein. 2012. Fast Inference in Phrase Extraction Models with Belief Propagation. In Proceedings of the North American Chapter of the Association for Computational Linguistics. find
Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural Language Processing (Almost) from Scratch. Journal of Machine Learning Research, 12:2493–2537, November. find our card
Ido Dagan and Alon Itai. 1990. Automatic Processing of Large Corpora for the Resolution of Anaphora References. In Proceedings of the Conference on Computational Linguistics - Volume 3. find our card
Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. 1977. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B, 39(1):1–38. find
Pascal Denis and Jason Baldridge. 2007. Joint Determination of Anaphoricity and Coreference Resolution using Integer Programming. In Proceedings of the North American Chapter of the Association for Computational Linguistics. find our card
Pascal Denis and Jason Baldridge. 2008. Specialized Models and Ranking for Coreference Resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find our card
John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research, 12:2121–2159, July. find
Eraldo Rezende Fernandes, Cı́cero Nogueira dos Santos, and Ruy Luiz Milidiú. 2012. Latent Structure Perceptron with Feature Induction for Unrestricted Coreference Resolution. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Proceedings and Conference on Computational… find our card
Jenny Rose Finkel and Christopher D. Manning. 2008. Enforcing Transitivity in Coreference Resolution. In Proceedings of the Association for Computational Linguistics: Short Papers. find our card
Kevin Gimpel and Noah A. Smith. 2010. Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions. In Proceedings of the North American Chapter for the Association for Computational Linguistics. find
David Graff, Junbo Kong, Ke Chen, and Kazuaki Maeda. 2007. English Gigaword Third Edition. Linguistic Data Consortium, Catalog Number LDC2007T07. find
Aria Haghighi and Dan Klein. 2010. Coreference Resolution in a Modular, Entity-Centered Model. In Proceedings of the North American Chapter of the Association for Computational Linguistics. find our card
Iris Hendrickx and Walter Daelemans, 2007. Adding Semantic Information: Unsupervised Clusters for Coreference Resolution. find
Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: the 90% solution. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Short Papers. find
Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2011. Stanford’s Multi-Pass Sieve Coreference Resolution System at the CoNLL-2011 Shared Task. In Proceedings of the Conference on Computational Natural Language Learning: Shared Task. find our card
Dong C. Liu and Jorge Nocedal. 1989. On the Limited Memory BFGS Method for Large Scale Optimization. Mathematical Programming, 45(3):503–528, December. find
Xiaoqiang Luo, Abe Ittycheriah, Hongyan Jing, Nanda Kambhatla, and Salim Roukos. 2004. A Mention-Synchronous Coreference Resolution Algorithm Based on the Bell Tree. In Proceedings of the Association for Computational Linguistics. find our card
Xiaoqiang Luo. 2005. On Coreference Resolution Performance Metrics. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find our card
Andrew McCallum and Ben Wellner. 2004. Conditional Models of Identity Uncertainty with Application to Noun Coreference. In Proceedings of Advances in Neural Information Processing Systems. find
George A. Miller. 1995. WordNet: A Lexical Database for English. Communications of the ACM, 38:39–41. find our card
Vincent Ng. 2007. Semantic class induction and coreference resolution. In Proceedings of the Association for Computational Linguistics. find our card
Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning Accurate, Compact, and Interpretable Tree Annotation. In Proceedings of the Conference on Computational Linguistics and the Association for Computational Linguistics. find our card
Simone Paolo Ponzetto and Michael Strube. 2006. Exploiting Semantic Role Labeling, WordNet and Wikipedia for Coreference Resolution. In Proceedings of the North American Chapter of the Association of Computational Linguistics. find our card
Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes. In Proceedings of the Conference on Computational Natural Language Learning: Shared Task. find our card
Karthik Raghunathan, Heeyoung Lee, Sudarshan Rangarajan, Nathanael Chambers, Mihai Surdeanu, Dan Jurafsky, and Christopher Manning. 2010. A Multi-Pass Sieve for Coreference Resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find our card
Altaf Rahman and Vincent Ng. 2009. Supervised Models for Coreference Resolution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find our card
Altaf Rahman and Vincent Ng. 2010. Inducing Fine-Grained Semantic Classes via Hierarchical and Collective Classification. In Proceedings of the International Conference on Computational Linguistics. find
Altaf Rahman and Vincent Ng. 2011. Narrowing the Modeling Gap: A Cluster-Ranking Approach to Coreference Resolution. Journal of Artificial Intelligence Research, 40(1):469–521, January. find
Mats Rooth, Stefan Riezler, Detlef Prescher, Glenn Carroll, and Franz Beil. 1999. Inducing a Semantically Annotated Lexicon via EM-Based Clustering. In Proceedings of the Association for Computational Linguistics. find
David A. Smith and Jason Eisner. 2008. Dependency Parsing by Belief Propagation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
Yang Song, Jing Jiang, Wayne Xin Zhao, Sujian Li, and Houfeng Wang. 2012. Joint Learning for Coreference Resolution with Markov Logic. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
Veselin Stoyanov and Jason Eisner. 2012. Easy-first Coreference Resolution. In Proceedings of the International Conference on Computational Linguistics. find our card
Veselin Stoyanov, Claire Cardie, Nathan Gilbert, Ellen Riloff, David Buttler, and David Hysom. 2010. find
Coreference Resolution with Reconcile. In Proceedings of the Association for Computational Linguistics: Short Papers. find
Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A Model-Theoretic Coreference Scoring Scheme. In Proceedings of the Conference on Message Understanding. find our card
Xiaofeng Yang, Jian Su, and Chew Lim Tan. 2005. Improving Pronoun Resolution Using Statistics-Based Semantic Compatibility Information. In Proceedings of the Association for Computational Linguistics. find our card
Xiaofeng Yang, Jian Su, Jun Lang, Chew L. Tan, Ting Liu, and Sheng Li. 2008. An Entity-Mention Model for Coreference Resolution with Inductive Logic Programming. In Proceedings of the Association for Computational Linguistics. find our card
Limin Yao, Aria Haghighi, Sebastian Riedel, and Andrew McCallum. 2011. Structured Relation Discovery Using Generative Models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/durrett-2013-decentralized-entity-level-modeling-coreference/

📄 Decentralized Entity-Level Modeling for Coreference Resolution