Elamur Research⌕
☰
home / papers / 2011 / RelaxCor Participation in CoNLL Shared Task on Coreference Resolution

RelaxCor Participation in CoNLL Shared Task on Coreference Resolution

Summary

The paper reports how the RelaxCor system performed in the CoNLL-2011 shared task on intra-document English coreference resolution over the OntoNotes corpora. RelaxCor treats coreference as graph partitioning. Mentions are vertices, and each edge carries a weight equal to the summed weights of the pairwise constraints that fire for that mention pair. Relaxation labeling then iteratively assigns mentions to entities, so the classification and clustering steps happen jointly and consider the whole document. The constraints were learned automatically with C4.5 rules from more than a hundred binarized pair attributes, including a new speaker feature. The authors filtered negative training pairs by their distance to positive ones and tuned two parameters, balance and pruning, on development data. The system ran in the closed track with predicted preprocessing and its own rule-based mention detector. It reached an official test score of 55.99, the average of MUC, B3 and CEAFe. The results show the approach stays robust in a fully automatic setting and gains recall over the earlier version.

Key points

  • The mention detector reached 92.45% recall but only 27.34% precision on development data. Verbs were never extracted, and non-referential NPs were a frequent source of errors.
  • Official test F1 scores were 59.55 for MUC, 67.09 for B3, 41.32 for entity-based CEAF and 71.10 for BLANC, giving a 55.99 average. The development average was 56.61.
  • Negative pairs differing in more than d=5 attribute values from every positive example were dropped, which removed 22.8% of negatives. Contradictory pairs were also discarded.
  • The authors tuned balance (0–1, step 0.05) and pruning (2–14 neighbors, step 2) by grid search against the task's averaged metric.
  • In the closed track, the only external resources used were WordNet, the Bergsma and Lin (2006) gender/number data, and sense inventories.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 2 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{sapena2011relaxcor,
  title = {{RelaxCor Participation in CoNLL Shared Task on Coreference Resolution}},
  author = {Sapena, Emili and Padró, Lluís and Turmo, Jordi},
  booktitle = {Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task},
  year = {2011},
  url = {https://aclanthology.org/W11-1903/}
}

References

13 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In Proceedings of the Linguistic Coreference Workshop at LREC 98, pages 563– 566, Granada, Spain. find our card
  2. S. Bergsma and D. Lin. 2006. Bootstrapping path-based pronoun resolution. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 33–40. Association for Computational Linguistics. find our card
  3. Jie Cai and Michael Strube. 2010. Evaluation metrics for end-to-end coreference resolution systems. In Proceedings of SIGDIAL, pages 28–36, University of Tokyo, Japan. find our card
  4. M. Collins. 1999. Head-driven statistical models for natural language parsing. Ph.D. thesis, University of Pennsylvania. find
  5. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of the Joint Conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT-EMNLP 2005, pages 25–32, Vancouver, B.C., Canada. find our card
  6. G.A. Miller. 1995. WordNet: a lexical database for English. find our card
  7. Sameer Pradhan, Lance Ramshaw, Mitchell Marcus, Martha Palmer, Ralph Weischedel, and Nianwen Xue. 2011. Conll-2011 shared task: Modeling unrestricted coreference in ontonotes. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning (CoNLL 2011), Portland, Oregon, June. find our card
  8. J.R. Quinlan. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann. find our card
  9. Marta Recasens, Lluı́s Màrquez, Emili Sapena, M. Antònia Martı́, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. Semeval-2010 task 1: Coreference resolution in multiple languages. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 1–8, Uppsa… find our card
  10. Emili Sapena, Lluı́s Padró, and Jordi Turmo. 2010a. A Global Relaxation Labeling Approach to Coreference Resolution. In Proceedings of 23rd International Conference on Computational Linguistics, COLING, Beijing, China, August. find our card
  11. Emili Sapena, Lluı́s Padró, and Jordi Turmo. 2010b. RelaxCor: A Global Relaxation Labeling Approach to Coreference Resolution. In Proceedings of the ACL Workshop on Semantic Evaluations (SemEval-2010), Uppsala, Sweden, July. find our card
  12. W.M. Soon, H.T. Ng, and D.C.Y. Lim. 2001. A Machine Learning Approach to Coreference Resolution of Noun Phrases. Computational Linguistics, 27(4):521–544. find our card
  13. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of the Sixth Message Understanding Conference (MUC-6), pages 45–52. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sapena-2011-relaxcor-participation-conll-shared-task/