Elamur Research⌕
☰
home / papers / 2015 / Solving Hard Coreference Problems

Solving Hard Coreference Problems

Summary

Coreference systems still struggle with pronouns, especially in Winograd-style sentences. In these, gender and number agreement offer no help, and picking the right antecedent depends on world knowledge. Peng, Khashabi and Roth propose Predicate Schemas, a knowledge representation organized around predicates:

  • Type 1 scores how plausible a mention is as the subject or object of a predicate.
  • Type 2 relates two predicates that share an argument, conditioned on the discourse connective between them.

The schemas are populated with statistics gathered without supervision from Gigaword, Wikipedia, web queries and polarity information. Earlier work fed such knowledge in as features, where sparsity weakens it. Here it is instead compiled at inference time into constraints inside an ILP-based mention-pair coreference model built on Chang et al. (2013). The constraints also capture interactions among several pronouns in one sentence.

The resulting system, KnowComb, beats both a general-purpose system and a dedicated classifier on hard pronoun cases. It stays at state-of-the-art level on ACE and OntoNotes. This shows that background knowledge can be added as constraints without degrading ordinary coreference.

Key points

  • On the Winograd dataset (1886 sentences, Rahman and Ng 2012), KnowComb reaches 76.76% precision. This compares with 53.26% for the general system of Chang et al. (2013) and 73.05% for the special-purpose Rahman and Ng classifier. On WinoCoref the gain is about 15%.
  • On standard benchmarks KnowComb roughly matches its base system IlliCons. Average scores are 78.97 vs 79.42 on ACE and 76.76 vs 77.05 on OntoNotes.
  • Type 1 and Type 2 schemas together cover 73% of Winograd instances: Cat1 is 16.8%, Cat2 is 56.2%, and the remaining 27.0% need richer context.
  • In the ablation on WinoCoref, Type 1 knowledge scores 90.26 on Cat1 instances versus 76.67 on all test data. Type 2 knowledge scores 83.38 on Cat2 versus 79.55 overall.
  • Code and data are released as Winocoref by the CogComp group (University of Illinois).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

38 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 26 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/n15-1082

@inproceedings{peng2015solving,
  title = {{Solving Hard Coreference Problems}},
  author = {Peng, Haoruo and Khashabi, Daniel and Roth, Dan},
  booktitle = {Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2015},
  doi = {10.3115/v1/n15-1082},
  url = {https://doi.org/10.3115/v1/n15-1082}
}

References

41 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. Bagga and B. Baldwin. 1998. Algorithms for scoring coreference chains. In The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference, pages 563–566. find our card
  2. N. Balasubramanian, S. Soderland, O. Etzioni, et al. 2012. Rel-grams: a probabilistic model of relations in text. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction, pages 101–105. Association for Computational Linguistics. find
  3. N. Balasubramanian, S. Soderland, Mausam, and O. Etzioni. 2013. Generating coherent event schemas at scale. In EMNLP, pages 1721–1731. find our card
  4. M. Bansal and D. Klein. 2012. Coreference semantics from web features. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), Jeju Island, South Korea, July. find our card
  5. E. Bengtson and D. Roth. 2008. Understanding the value of features for coreference resolution. In Proceedings of the Conference on Empirical Methods for Natural Language Processing (EMNLP), pages 294–303, Oct. find our card
  6. N. Chambers and D. Jurafsky. 2009. Unsupervised learning of narrative schemas and their participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, volume 2, pages 602–610. Associat… find
  7. K. Chang, R. Samdani, A. Rozovskaya, N. Rizzolo, M. Sammons, and D. Roth. 2011. Inference protocols for coreference resolution. In Proceedings of the Annual Conference on Computational Natural Language Learning (CoNLL), pages 40–44, Portland, Oregon, USA. Association for Computational Linguistics. find our card
  8. K. Chang, R. Samdani, and D. Roth. 2013. A constrained latent variable model for coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 601–612. Association for Computational Linguistics. find our card
  9. X. Cheng and D. Roth. 2013. Relational inference for wikification. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1787–1796. Association for Computational Linguistics. find
  10. D. Connolly, J. D. Burger, and D. S. Day. 1994. A machine learning approach to anaphoric reference. In Proceedings of the International Conference on New Methods in Language Processing (NeMLaP). ACL. find
  11. P. Denis and J. Baldridge. 2007. Joint determination of anaphoricity and coreference resolution using integer programming. In Proceedings of the Annual Meeting of the North American Association of Computational Linguistics (NAACL). find our card
  12. Y. Goldberg and M. Elhadad. 2010. An efficient algorithm for easy-first non-directional dependency parsing. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 742–750. Association for Computational Linguist… find
  13. J. R Hobbs. 1978. Resolving pronoun references. Lingua, 44(4):311–338. find our card
  14. H. Kobdani, H. Schuetze, M. Schiehlen, and H. Kamp. 2011. Bootstrapping coreference resolution using word associations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 783–792. Association for Computational Linguistics. find our card
  15. K. J. Kummerfeld and D. Klein. 2013. Error-driven analysis of challenges in coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 265–277. Association for Computational Linguistics. find our card
  16. S. Lappin and H. J. Leass. 1994. An algorithm for pronominal anaphora resolution. Computational linguistics, 20(4):535–561. find our card
  17. H. J. Levesque, E. Davis, and L. Morgenstern. 2011. The winograd schema challenge. In AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning. find
  18. X. Luo. 2005. On coreference resolution performance metrics. In Proceedings of the Conference on Empirical Methods for Natural Language Processing (EMNLP). find our card
  19. Mausam, M. Schmitz, R. Bart, S. Soderland, and O. Etzioni. 2012. Open language learning for information extraction. In Proceedings of Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). find
  20. V. Ng and C. Cardie. 2002a. Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution. In Proceedings of the 19th international conference on Computational linguistics-Volume 1, pages 1–7. Association for Computational Linguistics. find
  21. V. Ng and C. Cardie. 2002b. Improving machine learning approaches to coreference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. find our card
  22. V. Ng. 2004. Learning noun phrase anaphoricity to improve conference resolution: Issues in representation and optimization. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04). find our card
  23. NIST. 2004. The ACE evaluation plan. find
  24. S. Pradhan, L. Ramshaw, M. Marcus, M. Palmer, R. Weischedel, and N. Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In Proceedings of the Annual Conference on Computational Natural Language Learning (CoNLL). find our card
  25. S. Pradhan, A. Moschitti, N. Xue, O. Uryupina, and Y. Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of the Annual Conference on Computational Natural Language Learning (CoNLL). find our card
  26. V. Punyakanok and D. Roth. 2001. The use of classifiers in sequential inference. In The Conference on Advances in Neural Information Processing Systems (NIPS), pages 995–1001. MIT Press. find
  27. A. Rahman and V. Ng. 2011a. Coreference resolution with world knowledge. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 814–824. Association for Computational Linguistics. find our card
  28. A. Rahman and V. Ng. 2011b. Coreference resolution with world knowledge. In ACL, pages 814–824. find our card
  29. A. Rahman and V. Ng. 2012. Resolving complex cases of definite pronouns: the winograd schema challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 777–789. Association for Computational Linguistic… find our card
  30. L. Ratinov and D. Roth. 2012. Learning-based multi-sieve co-reference resolution with knowledge. In Proceedings of the Conference on Empirical Methods for Natural Language Processing (EMNLP). find our card
  31. Lev Ratinov, Dan Roth, Doug Downey, and Mike Anderson. 2011. Local and global algorithms for disambiguation to wikipedia. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1375–1384, Portland, Oregon, USA, June. Association… find
  32. D. Roth and W. Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Hwee Tou Ng and Ellen Riloff, editors, Proceedings of the Annual Conference on Computational Natural Language Learning (CoNLL), pages 1–8. Association for Computational Linguistics. find our card
  33. H. A. Schwartz and F. Gomez. 2009. Acquiring applicable common sense knowledge from the web. In Proceedings of the Workshop on Unsupervised and Minimally Supervised Learning of Lexical Semantics, UM-SLLS ’09, pages 1–9, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  34. W. M. Soon, D. C. Y. Lim, and H. T. Ng. 2001a. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, Volume 27, Number 4, December 2001. find our card
  35. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001b. A machine learning approach to coreference resolution of noun phrases. Computational linguistics, 27(4):521–544. find our card
  36. V. Stoyanov, C. Cardie, N. Gilbert, E. Riloff, D. Buttler, and D. Hysom. 2010. Coreference resolution with reconcile. In Proceedings of the ACL 2010 Conference Short Papers, pages 156–161. Association for Computational Linguistics. find our card
  37. N. Tandon, G. de Melo, and G. Weikum. 2014. Acquiring comparative commonsense knowledge from the web. In Proceedings of AAAI Conference on Artificial Intelligence. AAAI. find
  38. M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th conference on Message understanding. find our card
  39. T. Wilson, P. Hoffmann, S. Somasundaran, J. Kessler, J. Wiebe, Y. Choi, C. Cardie, E. Riloff, and S. Patwardhan. 2005. Opinionfinder: A system for subjectivity analysis. In Proceedings of HLT/EMNLP on interactive demonstrations, pages 34–35. Association for Computational Linguistics. find
  40. T. Winograd. 1972. Understanding natural language. Cognitive psychology, 3(1):1–191. find
  41. J. Zheng, L. Vilnis, S. Singh, J. D. Choi, and A. Mc-Callum. 2013. Dynamic knowledge-base alignment for coreference resolution. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning (CoNLL), pages 153–162, Sofia, Bulgaria, August. Association for Computational Lingu… find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/peng-2015-solving-hard-coreference-problems/