Elamur Research⌕
☰
home / papers / 2005 / Machine Learning for Coreference Resolution: From Local Classification to Global Ranking

Machine Learning for Coreference Resolution: From Local Classification to Global Ranking

Summary

Most learning-based coreference resolvers decide pairwise whether two NPs corefer and then greedily cluster those decisions. This pipeline has three weaknesses: it depends on ad-hoc design choices, it explores only one partition, and it never directly optimizes clustering-level accuracy. Ng reframes the task as choosing among complete partitions. Many learning-based coreference systems are trained on part of the training data and each proposes a partition of a document's NPs. An SVM ranker, trained on a held-out subset, then picks the best candidate. The ranker uses global, partition-based features and is trained so that partitions scoring higher under the target coreference metric receive higher ranks.

The approach was evaluated on three ACE data sets with the MUC and B-CUBED scorers. It compared favorably with two state-of-the-art baselines that follow the standard pipeline (Soon et al.; Ng and Cardie), with gains of up to 4–7% under one of the two metrics. The work matters because it offers a flexible framework that can combine heterogeneous resolvers. It also reduces reliance on manual design decisions.

Key points

  • The candidate pool contains 54 coreference systems, built by combining:

    • three learners (C4.5, RIPPER, maximum entropy);
    • three training-instance creation methods (McCarthy and Lehnert's, Soon et al.'s, Ng and Cardie's);
    • two feature sets (Soon et al.'s 12 features, Ng and Cardie's 53);
    • three clustering algorithms (closest-first, best-first, aggressive-merge).

  • The ranker is trained with Joachims' (2002) SVM package on partitions generated for held-out texts.

  • B-CUBED F-measure on BNEWS / NPAPER / NWIRE:

    SystemBNEWSNPAPERNWIRE
    Ranking framework65.666.766.4
    Soon et al. baseline63.565.664.2
    Ng and Cardie baseline65.563.364.4
    Perfect ranker (upper bound)70.069.169.1

    The gains here are modest but consistent, which indicates robustness across data sets.

  • Under the MUC scorer, the same resolver performed best on all three test sets: C4.5 with McCarthy and Lehnert instances, Ng and Cardie features, and aggressive-merge clustering. Under B-CUBED, the best resolvers varied across test sets.

  • The two scorers rank candidates differently for two reasons. B-CUBED also credits correct non-coreference decisions, and it penalizes wrong merges of large clusters more heavily than MUC does.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

28 papers of our library cite it (of the 140673 whose bibliographies we read); 16 of them have a page here, 12 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1219840.1219860

@inproceedings{ng2005machine,
  title = {{Machine Learning for Coreference Resolution: From Local Classification to Global Ranking}},
  author = {Ng, Vincent},
  booktitle = {Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)},
  year = {2005},
  doi = {10.3115/1219840.1219860},
  url = {https://doi.org/10.3115/1219840.1219860}
}

References

26 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. C. Aone and S. W. Bennett. 1995. Evaluating automated and manual acquisition of anaphora resolution strategies. In Proc. of the ACL, pages 122–129. find our card
  2. A. Bagga and B. Baldwin. 1998. Entity-based cross-document coreferencing using the vector space model. In Proc. of COLING-ACL, pages 79–85. find
  3. N. Bansal, A. Blum, and S. Chawla. 2002. Correlation clustering. In Proc. of FOCS, pages 238–247. find
  4. D. Bean and E. Riloff. 2004. Unsupervised learning of contextual role knowledge for coreference resolution. In Proc. of HLT/NAACL, pages 297–304. find our card
  5. A. Berger, S. Della Pietra, and V. Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–71. find our card
  6. C. Cardie and K. Wagstaff. 1999. Noun phrase coreference as clustering. In Proc. of EMNLP/VLC, pages 82–89. find our card
  7. W. Cohen. 1995. Fast effective rule induction. In Proc. of ICML, pages 115–123. find
  8. M. Collins. 2002. Discriminative training methods for Hidden Markov Models: Theory and experiments with perceptron algorithms. In Proc. of EMNLP, pages 1–8. find our card
  9. S. Harabagiu, R. Bunescu, and S. Maiorano. 2001. Text and knowledge mining for coreference resolution. In Proc. of NAACL, pages 55–62. find
  10. R. Iida, K. Inui, H. Takamura, and Y. Matsumoto. 2003. Incorporating contextual cues in trainable models for coreference resolution. In Proc. of the EACL Workshop on The Computational Treatment of Anaphora. find
  11. T. Joachims. 2002. Optimizing search engines using clickthrough data. In Proc. of KDD, pages 133–142. find
  12. A. Kehler. 1997. Probabilistic coreference in information extraction. In Proc. of EMNLP, pages 163–173. find our card
  13. X. Luo, A. Ittycheriah, H. Jing, N. Kambhatla, and S. Roukos. 2004. A mention-synchronous coreference resolution algorithm based on the Bell tree. In Proc. of the ACL, pages 136–143. find our card
  14. Coreference System Feature Set Learner Ng and Cardie’s C4.5 Ng and Cardie’s C4.5 Ng and Cardie’s C4.5 Soon et al.’s RIPPER Ng and Cardie’s C4.5 Ng and Cardie’s MaxEnt Clustering Algorithm aggressive-merge aggressive-merge aggressive-merge closest-first aggressive-merge closest-first A. McCallum and… find
  15. J. McCarthy and W. Lehnert. 1995. Using decision trees for coreference resolution. In Proc. of the IJCAI, pages 1050–1055. find
  16. T. Morton. 2000. Coreference for NLP applications. In Proc. of the ACL. find
  17. C. Müller, S. Rapp, and M. Strube. 2002. Applying co-training to reference resolution. In Proc. of the ACL, pages 352–359. find
  18. V. Ng and C. Cardie. 2002a. Combining sample selection and error-driven pruning for machine learning of coreference rules. In Proc. of EMNLP, pages 55–62. find our card
  19. V. Ng and C. Cardie. 2002b. Improving machine learning approaches to coreference resolution. In Proc. of the ACL, pages 104–111. find our card
  20. J. R. Quinlan. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann. find our card
  21. W. M. Soon, H. T. Ng, and D. Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544. find our card
  22. M. Strube and C. Müller. 2003. A machine learning approach to pronoun resolution in spoken dialogue. In Proc. of the ACL, pages 168–175. find our card
  23. M. Strube, S. Rapp, and C. Müller. 2002. The influence of minimum edit distance on reference resolution. In Proc. of EMNLP, pages 312–319. find
  24. M. Vilain, J. Burger, J. Aberdeen, D. Connolly, and L. Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proc. of the Sixth Message Understanding Conference (MUC-6), pages 45–52. find our card
  25. X. Yang, G. D. Zhou, J. Su, and C. L. Tan. 2003. Coreference resolution using competitive learning approach. In Proc. of the ACL, pages 176–183. find
  26. D. Zelenko, C. Aone, and J. Tibbetts. 2004. Coreference resolution for information extraction. In Proc. of the ACL Workshop on Reference Resolution and its Applications, pages 9–16. find our card

↑ back to top

This page in Markdown →