Elamur Research⌕
☰
home / papers / 2007 / First-Order Probabilistic Models for Coreference Resolution

First-Order Probabilistic Models for Coreference Resolution

Summary

Most machine-learned coreference systems judge noun phrase mentions two at a time. This means they cannot use evidence about a whole entity cluster, such as a ban on clusters made only of pronouns. Culotta, Wick and McCallum let a maximum-entropy model score entire sets of mentions. This gives a first-order probabilistic model with the expressiveness of Markov logic networks. The catch is that the number of candidate sets grows exponentially, so they add practical approximations:

  • they create cluster-level variables only when needed;
  • they use greedy agglomerative clustering at test time;
  • they train in an error-driven way, learning only from the first wrong merge in each training document;
  • they replace classification with a ranking objective, optimized with MIRA, so that a correct nearby cluster scores above the wrong one.

On the ACE coreference data, the full system cuts errors by 45% compared with a matched pairwise baseline. It also exceeds the best B-Cubed F1 previously reported on this dataset. The work shows that rich first-order features can be made tractable through sampling and approximate inference.

Key points

  • Training variants. Three first-order variants are compared with a Pairwise Model trained in the style of Soon et al. (2001):
    • First-Order Uniform, which samples clusters at random;
    • error-driven training;
    • First-Order MIRA, which combines error-driven updates with a ranking loss.
  • Main gain. On the ACE coreference dataset, error drops by 45% relative to the comparable pairwise approach.
  • Comparison with prior work. The best system reaches 79.3% B-Cubed F1, versus 69.3% for Ng (2005)'s meta-classifier. The authors note that the train-test splits may differ slightly.
  • Baseline check. The pairwise baseline performs roughly on par with Ng and Cardie (2002).
  • Contrast with LASO. Unlike LASO, each update uses just two examples: the highest-scoring wrong cluster and the highest-scoring correct one. This avoids penalizing coreferent subsets inside an incorrect merge.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

60 papers of our library cite it (of the 140673 whose bibliographies we read); 38 of them have a page here, 22 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{culotta2007first,
  title = {{First-Order Probabilistic Models for Coreference Resolution}},
  author = {Culotta, Aron and Wick, Michael and McCallum, Andrew},
  booktitle = {Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference},
  year = {2007},
  url = {https://aclanthology.org/N07-1011/}
}

References

29 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. B. Amit and B. Baldwin. 1998. Algorithms for scoring coreference chains. In Proceedings of the Seventh Message Understanding Conference (MUC7). find our card
  2. Razvan Bunescu and Raymond J. Mooney. 2004. Collective information extraction with relational markov networks. In ACL. find
  3. Y. Censor and S.A. Zenios. 1997. Parallel optimization : theory, algorithms, and applications. Oxford University Press. find
  4. Michael Collins and Brian Roark. 2004. Incremental parsing with the perceptron algorithm. In ACL. find our card
  5. Koby Crammer and Yoram Singer. 2003. Ultraconservative online algorithms for multiclass problems. JMLR, 3:951– 991. find
  6. Aron Culotta and Andrew McCallum. 2006. Tractable learning and inference with high-order representations. In ICML Workshop on Open Problems in Statistical Relational Learning, Pittsburgh, PA. find
  7. Hal Daumé III and Daniel Marcu. 2005a. A large-scale exploration of effective global features for a joint entity detection and tracking model. In HLT/EMNLP, Vancouver, Canada. find
  8. Hal Daumé III and Daniel Marcu. 2005b. Learning as search optimization: Approximate large margin methods for structured prediction. In ICML, Bonn, Germany. find
  9. Rodrigo de Salvo Braz, Eyal Amir, and Dan Roth. 2005. Lifted first-order probabilistic inference. In IJCAI, pages 1319– 1325. find
  10. Pascal Denis and Jason Baldridge. 2007. A ranking approach to pronoun resolution. In IJCAI. find
  11. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by gibbs sampling. In ACL, pages 363– 370. find
  12. H. Gaifman. 1964. Concerning measures in first order calculi. Israel J. Math, 2:1–18. find
  13. J. Y. Halpern. 1990. An analysis of first-order logics of probability. Artificial Intelligence, 46:311–350. find
  14. Xiaoqiang Luo, Abe Ittycheriah, Hongyan Jing, Nanda Kambhatla, and Salim Roukos. 2004. A mention-synchronous coreference resolution algorithm based on the Bell tree. In ACL, page 135. find our card
  15. A. McCallum and B. Wellner. 2003. Toward conditional models of identity uncertainty with application to proper noun coreference. In IJCAI Workshop on Information Integration on the Web. find
  16. Andrew McCallum and Ben Wellner. 2005. Conditional models of identity uncertainty with application to noun coreference. In Lawrence K. Saul, Yair Weiss, and Léon Bottou, editors, NIPS17. MIT Press, Cambridge, MA. find
  17. Brian Milch, Bhaskara Marthi, and Stuart Russell. 2004. BLOG: Relational modeling with unknown objects. In ICML 2004 Workshop on Statistical Relational Learning and Its Connections to Other Fields. find
  18. Brian Milch, Bhaskara Marthi, Stuart Russell, David Sontag, Daniel L. Ong, and Andrey Kolobov. 2005. BLOG: Probabilistic models with unknown objects. In IJCAI. find
  19. Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In ACL. find our card
  20. Vincent Ng. 2005. Machine learning for coreference resolution: From local classification to global ranking. In ACL. find our card
  21. Cristina Nicolae and Gabriel Nicolae. 2006. Bestcut: A graph algorithm for coreference resolution. In EMNLP, pages 275–283, Sydney, Australia, July. Association for Computational Linguistics. find our card
  22. Mark A. Paskin. 2002. Maximum entropy probabilistic logic. Technical Report UCB/CSD-01-1161, University of California, Berkeley. find
  23. D. Poole. 2003. First-order probabilistic inference. In IJCAI, pages 985–991, Acapulco, Mexico. Morgan Kaufman. find
  24. Matthew Richardson and Pedro Domingos. 2006. Markov logic networks. Machine Learning, 62:107–136. find
  25. D. Roth and W. Yih. 2004. A linear programming formulation for global inference in natural language tasks. In The 8th Conference on Compuational Natural Language Learning, May. find our card
  26. Parag Singla and Pedro Domingos. 2005. Discriminative training of markov logic networks. In AAAI, Pittsburgh, PA. find
  27. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Comput. Linguist., 27(4):521–544. find our card
  28. Charles Sutton and Andrew McCallum. 2004. Collective segmentation and labeling of distant entities in information extraction. Technical Report TR \# 04-49, University of Massachusetts, July. find
  29. Charles Sutton and Andrew McCallum. 2005. Piecewise training of undirected models. In 21st Conference on Uncertainty in Artificial Intelligence. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/culotta-2007-first-order-probabilistic-models-coreference/