Elamur Research⌕
☰
home / papers / 2005 / An Expectation Maximization Approach to Pronoun Resolution

An Expectation Maximization Approach to Pronoun Resolution

Summary

The paper asks whether third-person pronouns can be linked to their antecedents without annotated training data. Cherry and Bergsma build a fully automatic pipeline. A dependency-parsed news-domain Question Answering corpus supplies candidate nouns from the current and previous sentence. Linguistic modules handle pleonastic, cataphoric and other special cases before learning. Expectation Maximization, modeled on the word-alignment approach of IBM Model 1, then treats the true antecedent as a hidden variable. It jointly estimates four lexicalized distributions:

  • gender/number affinity of a word,
  • the pronoun's syntactic context given the antecedent,
  • the prior likelihood of a word being an antecedent,
  • a distance ("jump") preference.

Seeding EM with a gender/number model collected from unambiguous cases helps. Reweighting the learned models with maximum entropy helps further. On a news test set with non-anaphoric pronouns removed, the unsupervised system comes within a small margin of a supervised SVM. The semi-supervised extension matches the SVM statistically. The work shows that unlabeled text alone can yield competitive pronoun resolution. It also shows that automatically induced gender/number statistics are an especially strong cue.

Key points

  • The candidate window (current plus preceding sentence) contained a correct antecedent in 97% of lists on a labeled development text.
  • Accuracy on all pronouns rose from 0.397 for the previous-noun baseline to 0.610 for uniform-start EM. The initializer alone reached 0.628, EM with the initializer 0.632, and the maxent extension 0.669, against an upper bound of 0.838.
  • The maxent weights favored the pronoun (gender/number) model heavily (λ = 0.931). The context, antecedent-prior and jump models received weights between 0.056 and 0.167.
  • The comparison SVM used 52 features and 1398 labeled pronouns and scored 0.714. EM with the initializer scored 0.664, a 7% relative drop. The maxent extension scored 0.708, which is not significantly different from the SVM (McNemar's test).
  • Sentences containing quotations were excluded from training because their candidate lists often lack the true antecedent (upper bound only 0.754 on these). Missing antecedents were mainly caused by:
    • pleonastic detection errors (31%),
    • non-noun referents (25%),
    • the limited window (21%).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

17 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 6 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{cherry2005expectation,
  title = {{An Expectation Maximization Approach to Pronoun Resolution}},
  author = {Cherry, Colin and Bergsma, Shane},
  booktitle = {Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)},
  year = {2005},
  doi = {10.3115/1706543.1706559},
  url = {https://doi.org/10.3115/1706543.1706559}
}

References

23 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David L. Bean and Ellen Riloff. 2004. Unsupervised learning of contextual role knowledge for coreference resolution. In HLT-NAACL, pages 297–304. find our card
  2. Adam L. Berger, Stephen A. Della Pietra, and Vincent J. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71. find our card
  3. Shane Bergsma. 2005a. Automatic acquisition of gender information for anaphora resolution. In Proceedings of the 18th Conference of the Canadian Society for Computational Intelligence (Canadian AI 2005), pages 342–353, Victoria, BC. find
  4. Shane Bergsma. 2005b. Corpus-based learning for pronominal anaphora resolution. Master’s thesis, Department of Computing Science, University of Alberta, Edmonton. http://www.cs.ualberta.ca/˜bergsma/Pubs/thesis.pdf. ↗
  5. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–312. find
  6. Claire Cardie and Kiri Wagstaff. 1999. Noun phrase coreference as clustering. In Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 82–89. find our card
  7. A. P. Dempster, N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1):1–38. find
  8. Niyu Ge, John Hale, and Eugene Charniak. 1998. A statistical approach to anaphora resolution. In Proceedings of the Sixth Workshop on Very Large Corpora, pages 161–171. find our card
  9. L. Haegeman. 1994. Introduction to Government & Binding theory: Second Edition. Basil Blackwell, Cambridge, UK. find
  10. Donald Hindle and Mats Rooth. 1993. Structural ambiguity and lexical relations. Computational Linguistics, 19(1):103– 120. find
  11. Harold Jeffreys, 1961. Theory of Probability, chapter 3.23. Oxford: Clarendon Press, 3rd edition. find
  12. Andrew Kehler. 1997. Probabilistic coreference in information extraction. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, pages 163–173. find our card
  13. Christopher Kennedy and Branimir Boguraev. 1996. Anaphora for everyone: Pronominal anaphora resolution without a parser. In Proceedings of the 16th Conference on Computational Linguistics, pages 113–118. find our card
  14. Shalom Lappin and Herbert J. Leass. 1994. An algorithm for pronominal anaphora resolution. Computational Linguistics, 20(4):535–561. find our card
  15. Dekang Lin. 1994. Principar - an efficient, broad-coverage, principle-based parser. In Proceedings of COLING-94, pages 42–48, Kyoto, Japan. find
  16. Robert Malouf. 2002. A comparison of algorithms for maximum entropy parameter estimation. In Proceedings of the Sixth Conference on Natural Language Learning (CoNLL-2002), pages 49–55. find our card
  17. Ruslan Mitkov, Richard Evans, and Constantin Orasan. 2002. A new, fully automatic version of Mitkov’s knowledge-poor pronoun resolution method. In Proceedings of the Third International Conference on Computational Linguistics and Intelligent Text Processing, pages 168–186. find
  18. Christoph Müller, Stefan Rapp, and Michael Strube. 2002. Applying co-training to reference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 352–359. find
  19. Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 104–111. find our card
  20. Vincent Ng and Claire Cardie. 2003. Weakly supervised natural language learning without redundant views. In HLT-NAACL 2003: Proceedings of the Main Conference, pages 173–180. find
  21. Franz J. Och and Hermann Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 295–302, Philadelphia, PA, July. find our card
  22. Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521– 544. find our card
  23. Ellen Vorhees. 2002. Overview of the TREC 2002 question answering track. In Proceedings of the Eleventh Text REtrieval Conference (TREC). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cherry-2005-expectation-maximization-approach-pronoun-resolution/