Elamur Research⌕
☰
home / papers / 2016 / Search Space Pruning: A Simple Solution for Better Coreference Resolvers

Search Space Pruning: A Simple Solution for Better Coreference Resolvers

Summary

Coreference resolvers perform far worse on automatically detected mentions than on gold mentions. With Cort's latent ranking model, the gap on CoNLL 2012 English is about 17 points. Moosavi and Strube attribute much of this to a large, unbalanced search space, since most noun-phrase mentions are singletons or non-referential. They propose a lightweight singleton detector that uses a small set of shallow features: lemmas, POS tags, context words, mention string, length, type, and whether the string or head recurs. These feed an anchored SVM with a degree-two polynomial kernel, trained only on predicted annotations from the CoNLL 2012 English data. Without hand-crafted linguistic features, it clearly beats the previous best singleton classifier, from Marneffe et al. The detector's output is then used in two ways: as a high-precision preprocessing filter for the Stanford rule-based system and Cort's pairwise model, and as an extra feature for Cort's latent ranking model. Every resolver improves significantly, and the latent ranking variant reached the best overall score then reported on the CoNLL 2012 English test set. Simple pruning thus offers a cheap, broadly applicable boost.

Key points

  • On the CoNLL 2012 English development set, the Surface model reaches 85.73% accuracy, compared with 78.0% for Marneffe et al. (2015). It does so with about 8.3K features instead of about 73K. Adding linguistic features changes results by only about 0.1 point.
  • A Confident setting keeps only SVM outputs at or beyond ±1. It gives precision around 94% for non-coreferent mentions, which makes aggressive preprocessing pruning safe.
  • Overall CoNLL scores on the test set:
    • Stanford rises from 54.97 to 56.83, beating its own singleton module at 55.59.
    • Cort pairwise rises from 59.45 to 60.46.
    • Cort latent ranking rises from 63.02 to 63.54, above Wiseman et al. (2015) at 63.39.
  • For the latent model, the singleton feature is trained with a two-fold split of the training data to avoid overfitting.
  • Error analysis shows that many precision errors are first mentions of entities. Spotting these requires context beyond the mention itself.
  • The singleton feature is released in the Cort source code (github.com/smartschat/cort).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

5 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 2 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1115

@inproceedings{moosavi2016search,
  title = {{Search Space Pruning: A Simple Solution for Better Coreference Resolvers}},
  author = {Moosavi, Nafise Sadat and Strube, Michael},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1115},
  url = {https://doi.org/10.18653/v1/n16-1115}
}

References

26 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In Proceedings of the 1st International Conference on Language Resources and Evaluation, Granada, Spain, 28–30 May 1998, pages 563–566. find our card
  2. Susan E. Brennan, Marilyn W. Friedman, and Carl J. Pollard. 1987. A centering approach to pronouns. In Proceedings of the 25th Annual Meeting of the Association for Computational Linguistics, Stanford, Cal., 6–9 July 1987, pages 155–162. find
  3. Kevin Clark and Christopher D. Manning. 2015. Entity-centric coreference resolution with model stacking. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Beijing, China, 26–31 July 2015, pages 1405–1415. find our card
  4. Pascal Denis and Jason Baldridge. 2007. Joint determination of anaphoricity and coreference resolution using integer programming. In Proceedings of Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics, Rochester, N.Y., 22–27… find our card
  5. Greg Durrett and Dan Klein. 2013. Easy victories and uphill battles in coreference resolution. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Wash., 18–21 October 2013, pages 1971–1982. find our card
  6. Yoav Goldberg and Michael Elhadad. 2007. SVM model tampering and anchored learning: A case study in Hebrew NP chunking. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, Prague, Czech Republic, 23–30 June 2007, pages 224–231. find our card
  7. Yoav Goldberg and Michael Elhadad. 2009. On the role of lexical features in sequence labeling. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Singapore, 6–7 August 2009, pages 1142–1151. find
  8. Barbara J. Grosz and Candace L. Sidner. 1986. Attention, intentions, and the structure of discourse. Computational Linguistics, 12(3):175–204. find our card
  9. Jerry R. Hobbs. 1978. Resolving pronominal references. Lingua, 44:311–338. find
  10. Emmanuel Lassalle and Pascal Denis. 2015. Joint anaphoricity detection and coreference resolution with constrained latent structures. In Proceedings of the 29th Conference on the Advancement of Artificial Intelligence, Austin, Texas, 25–30 January 2015, pages 2274–2280. find
  11. Heeyoung Lee, Angel Chang, Yves Peirsman, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2013. find
  12. Deterministic coreference resolution based on entity-centric, precision-ranked rules. Computational Linguistics, 39(4):885–916. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of the Human Language Technology Conference and the 2005 Conference on Empirical Methods… find our card
  13. Chao Ma, Janardhan Rao Doppa, J. Walker Orr, Prashanth Mannem, Xiaoli Fern, Tom Dietterich, and Prasad Tadepalli. 2014. Prune-and-score: Learning for greedy coreference resolution. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 25–29 October 2… find our card
  14. Marie-Catherine de Marneffe, Marta Recasens, and Christopher Potts. 2015. Modeling the lifespan of discourse entities with application to coreference resolution. Journal of Artificial Intelligent Research, 52:445–475. find
  15. Sebastian Martschat and Michael Strube. 2015. Latent structures for coreference resolution. Transactions of the Association for Computational Linguistics, 3:405– 418. find our card
  16. Nafise Sadat Moosavi and Michael Strube. 2014. Unsupervised coreference resolution by utilizing the most informative relations. In Proceedings of the 25th International Conference on Computational Linguistics, Dublin, Ireland, 23–29 August 2014, pages 644–655. find
  17. Vincent Ng and Claire Cardie. 2002. Identifying anaphoric and non-anaphoric noun phrases to improve coreference resolution. In Proceedings of the 19th International Conference on Computational Linguistics, Taipei, Taiwan, 24 August – 1 September 2002. find
  18. Vincent Ng. 2009. Graph-cut-based anaphoricity determination for coreference resolution. In Proceedings of Human Language Technologies 2009: The Conference of the North American Chapter of the Association for Computational Linguistics, Boulder, Col., 31 May – 5 June 2009, pages 575–583. find our card
  19. Vincent Ng. 2010. Supervised noun phrase coreference research: The first fifteen years. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, 11–16 July 2010, pages 1396–1411. find our card
  20. Eric W. Noreen. 1989. Computer Intensive Methods for Hypothesis Testing: An Introduction. find
  21. Wiley, New York, N.Y. Haoruo Peng, Kai-Wei Chang, and Dan Roth. 2015. A joint framework for coreference resolution and mention head detection. In Proceedings of the 19th Conference on Computational Natural Language Learning, Beijing, China, 30–31 July 2015, pages 12–21. find our card
  22. Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring coreference partitions of predicted mentions: A reference implementation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Bal… find our card
  23. Marta Recasens, Marie-Catherine de Marneffe, and Christopher Potts. 2013. The life and death of discourse entities: Identifying singleton mentions. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlan… find
  24. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the 6th Message Understanding Conference (MUC-6), pages 45–52, San Mateo, Cal. find our card
  25. Morgan Kaufmann. Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Beijing, Chin… find our card
  26. Guodong Zhou and Fang Kong. 2009. Global learning of noun phrase anaphoricity in coreference resolution via label propagation. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Singapore, 6–7 August 2009, pages 978–986. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/moosavi-2016-search-space-pruning-simple-solution/