Elamur Research⌕
☰
home / papers / 2015 / Transferring Coreference Resolvers with Posterior Regularization

Transferring Coreference Resolvers with Posterior Regularization

Summary

Coreference resolvers usually depend on annotated corpora, which most languages lack. This paper builds resolvers for low-resource languages by transferring knowledge from an English system, using English→Spanish and English→Brazilian Portuguese as test cases. First, the Berkeley Coreference Resolution system labels the English side of the trilingual FAPESP parallel corpus. Next, the predicted mentions and chains are carried over to the target side through Berkeley aligner word alignments, which produces a noisy training set. A mention-ranking model with latent coreference trees is then trained on this projected data. Training uses a new softmax-margin version of posterior regularization (PR), which adds a task-specific cost to tune the precision/recall balance. It also uses a penalized formulation that sets the Lagrange multipliers directly, avoiding a saddle-point optimization. The approach outperforms delexicalized transfer with bilingual embeddings, direct bitext projection, and vanilla PR. Its largest gains are in recall. This matters because it yields a full end-to-end resolver without any target-language coreference annotation.

Key points

  • Data: The FAPESP corpus (Aziz and Specia, 2011) was used for projection. Evaluation used AnCora (SemEval 2010) for Spanish and Summ-It 3.0 for Portuguese (50 Folha de São Paulo documents). English OntoNotes (CoNLL 2011) served as the source training data, and TurboParser supplied POS tags and parses.
  • Projection filtering:
    • Alignments with confidence below 0.95 were dropped.
    • Mentions were projected with a maximal-span heuristic.
    • Documents were discarded if more than 15% of their mentions were unaligned.
    • About 73.8% of mentions survived for Portuguese and 69.9% for Spanish.
  • Mention recall (Spanish dev set): Softmax-margin PR reached 47.1%. This is roughly 5 points above direct projection, 10 above delexicalized transfer, and about 5 above vanilla PR. The higher mention recall carried over into recall gains on MUC, B³, and CEAF_e.
  • Possessive pronouns (Spanish): The share wrongly treated as non-anaphoric was:
    • 53.3% for the delexicalized model
    • 30.1% for direct projection
    • 4.9% for vanilla PR
    • 3.4% for softmax-margin PR

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

13 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 9 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-1138

@inproceedings{martins2015transferring,
  title = {{Transferring Coreference Resolvers with Posterior Regularization}},
  author = {Martins, André F. T.},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1138},
  url = {https://doi.org/10.3115/v1/p15-1138}
}

References

53 references, 32 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mariana S. C. Almeida, Cláudia Pinto, Helena Figueira, Pedro Mendes, and André F. T. Martins. 2015. Aligning opinions: Cross-lingual opinion mining with dependencies. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find
  2. Giuseppe Attardi, Stefano Dei Rossi, and Maria Simi. 2010. TANL-1: coreference resolution by parse analysis and similarity clustering. In Proc. of the International Workshop on Semantic Evaluation. find our card
  3. Wilker Aziz and Lucia Specia. 2011. Fully automatic compilation of a Portuguese-English parallel corpus for statistical machine translation. In STIL 2011. find
  4. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In Proc. of International Conference on Language Resources and Evaluation: Workshop on Linguistics Coreference. find our card
  5. Eric Bengtson and Dan Roth. 2008. Understanding the value of features for coreference resolution. In Proc. of Empirical Methods in Natural Language Processing. find our card
  6. Shane Bergsma and Dekang Lin. 2006. Bootstrapping path-based pronoun resolution. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find our card
  7. Sandra Collovini, Thiago Carbonel, Juliana Thiesen Fuchs, Jorge César Coelho, Lúcia Rino, and Renata Vieira. 2007. Summ-it: Um corpus anotado com informações discursivas visando a sumarização automática. In Workshop em Tecnologia da Informação e da Linguagem Humana. find
  8. Pascal Denis and Jason Baldridge. 2008. Specialized models and ranking for coreference resolution. In Proc. of Empirical Methods in Natural Language Processing. find our card
  9. Greg Durrett and Dan Klein. 2013. Easy victories and uphill battles in coreference resolution. In Proc. of Empirical Methods in Natural Language Processing. find our card
  10. Greg Durrett, David Hall, and Dan Klein. 2013. Decentralized entity-level modeling for coreference resolution. In Proc. of Annual Meeting of the Association for Computational Linguistics. find our card
  11. Eraldo Rezende Fernandes, Cı́cero Nogueira dos Santos, and Ruy Luiz Milidiú. 2012. Latent structure perceptron with feature induction for unrestricted coreference resolution. In Joint Conference on EMNLP and CoNLL-Shared Task, pages 41–48. find our card
  12. Kuzman Ganchev and Dipanjan Das. 2013. Cross-lingual discriminative learning of sequence models with posterior regularization. In Proc. of Empirical Methods in Natural Language Processing. find
  13. Kuzman Ganchev, João Graça, Jennifer Gillenwater, and Ben Taskar. 2010. Posterior regularization for structured latent variable models. Journal of Machine Learning Research, 11:2001–2049. find
  14. Kevin Gimpel and Noah A. Smith. 2010. Softmax-Margin CRFs: Training Log-Linear Models with Loss Functions. In NAACL. find
  15. Aria Haghighi and Dan Klein. 2007. Unsupervised coreference resolution in a nonparametric bayesian model. In Proc. of Annual Meeting of the Association for Computational Linguistics. find our card
  16. Aria Haghighi and Dan Klein. 2010. Coreference resolution in a modular, entity-centered model. In Proc. of Annual Conference of the North American Chapter of the Association for Computational Linguistics. find our card
  17. Sanda M Harabagiu and Steven J Maiorano. 2000. Multilingual coreference resolution. In Proc. of the Conference on Applied Natural Language Processing. find our card
  18. Christian Hardmeier, Jörg Tiedemann, and Joakim Nivre. 2013. Latent anaphora resolution for cross-lingual pronoun prediction. In Proc. of Empirical Methods in Natural Language Processing. find our card
  19. Karl Moritz Hermann and Phil Blunsom. 2014. Multilingual Models for Compositional Distributional Semantics. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find
  20. Rebecca Hwa, Philip Resnik, Amy Weinberg, Clara Cabezas, and Okan Kolak. 2005. Bootstrapping parsers via syntactic projection across parallel texts. Natural language engineering, 11(3):311–325. find
  21. Percy Liang, Ben Taskar, and Dan Klein. 2006. Alignment by agreement. In Proc. of North American Chapter of the Association of Computational Linguistics. find our card
  22. Percy Liang, Michael I Jordan, and Dan Klein. 2009. Learning from measurements in exponential families. In Proc. of International Conference on Machine Learning, pages 641–648. find
  23. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proc. of Empirical Methods in Natural Language Processing. Association for Computational Linguistics. find our card
  24. Gideon Mann and Andrew McCallum. 2010. Generalized expectation criteria for semi-supervised learning with weakly labeled data. Journal of Machine Learning Research, 11:955–984. find
  25. André F. T Martins, Noah A. Smith, Eric P. Xing, Pedro M. Q. Aguiar, and Mário A. T. Figueiredo. 2010. Turbo Parsers: Dependency Parsing by Approximate Variational Inference. In Proc. of Empirical Methods for Natural Language Processing. find our card
  26. André F. T Martins, Miguel B. Almeida, and Noah A. Smith. 2013. Turning on the turbo: Fast third-order non-projective turbo parsers. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find our card
  27. Ryan McDonald, Slav Petrov, and Keith Hall. 2011. Multi-source transfer of delexicalized dependency parsers. In Proc. of Empirical Methods in Natural Language Processing. find our card
  28. Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proc. of the Annual Meeting on Association for Computational Linguistics. find our card
  29. Vincent Ng. 2008. Unsupervised models for coreference resolution. In Proc. of Empirical Methods in Natural Language Processing. find our card
  30. Slav Petrov and Dan Klein. 2007. Improved inference for unlexicalized parsing. In Proc. of Annual Meeting of the North American Chapter of the Association for Computational Linguistics, pages 404–411. find
  31. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. A universal part-of-speech tagset. In Proc. of LREC. find
  32. Oana Postolache, Dan Cristea, and Constantin Orasan. 2006. Transferring coreference chains through word alignment. In Proc. of the International Conference on Language Resources and Evaluation. find our card
  33. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 Shared Task: Modeling multilingual unrestricted coreference in OntoNotes. In Proc. of the Conference on Computational Natural Language Learning: Shared Task. find our card
  34. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring coreference partitions of predicted mentions: A reference implementation. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find our card
  35. Karthik Raghunathan, Heeyoung Lee, Sudarshan Rangarajan, Nathanael Chambers, Mihai Surdeanu, Dan Jurafsky, and Christopher Manning. 2010. A multi-pass sieve for coreference resolution. In Proc. of Empirical Methods in Natural Language Processing. find our card
  36. Altaf Rahman and Vincent Ng. 2011. Narrowing the modeling gap: A cluster-ranking approach to coreference resolution. Journal of Artificial Intelligence Research, 40(1):469–521. find
  37. Altaf Rahman and Vincent Ng. 2012. Translation-based projection for multilingual coreference resolution. In Proc. of the Conference of the North American Chapter of the Association for Computational Linguistics. find our card
  38. Marta Recasens and Eduard Hovy. 2010. Coreference resolution across corpora: Languages, coding schemes, and preprocessing information. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find our card
  39. Marta Recasens and M Antònia Martı́. 2010. Ancora-co: Coreferentially annotated corpora for spanish and catalan. Language resources and evaluation, 44(4):315–345. find
  40. Marta Recasens, Lluı́s Màrquez, Emili Sapena, M Antònia Martı́, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. Semeval-2010 task 1: Coreference resolution in multiple languages. In Proc. of the International Workshop on Semantic Evaluation. find our card
  41. Noah A Smith and Jason Eisner. 2005. Contrastive estimation: Training log-linear models on unlabeled data. In Proc. of Annual Meeting on Association for Computational Linguistics. find
  42. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational linguistics, 27(4):521–544. find our card
  43. José Guilherme Camargo de Souza and Constantin Orăsan. 2011. Can projected chains in parallel corpora help coreference resolution? In Anaphora Processing and Applications, pages 59–69. Springer. find
  44. Veselin Stoyanov, Claire Cardie, Nathan Gilbert, Ellen Riloff, David Buttler, and David Hysom. 2010. Coreference resolution with reconcile. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find our card
  45. Oscar Täckström, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In Proc. of the North American Chapter of the Association for Computational Linguistics. find our card
  46. Oscar Täckström, Dipanjan Das, Slav Petrov, Ryan McDonald, and Joakim Nivre. 2013. Token and type constraints for cross-lingual part-of-speech tagging. Transactions of the Association for Computational Linguistics, 1:1–12. find
  47. Ivan Titov and Alexandre Klementiev. 2012. Cross-lingual induction of semantic roles. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find
  48. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proc. of the Annual Meeting of the Association for Computational Linguistics. find our card
  49. Yannick Versley, Simone Paolo Ponzetto, Massimo Poesio, Vladimir Eidelman, Alan Jern, Jason Smith, Xiaofeng Yang, and Alessandro Moschitti. 2008. Bart: A modular toolkit for coreference resolution. In Proc. of the Annual Meeting of the Association for Computational Linguistics: Demo Session. find our card
  50. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proc. of the Conference on Message Understanding, pages 45–52. Association for Computational Linguistics. find our card
  51. Mengqiu Wang and Chris Manning. 2014. Cross-lingual projected expectation regularization for weakly supervised learning. Transactions of the Association for Computational Linguistics, 2:55–66. find
  52. David Yarowsky, Grace Ngai, and Richard Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proc. of the First International Conference on Human Language Technology Research. find
  53. Daniel Zeman and Philip Resnik. 2008. Cross-language parser adaptation between related languages. In IJCNLP, pages 35–42. find

↑ back to top

This page in Markdown →