Elamur Research⌕
☰
home / papers / 2012 / Creating a Coreference Resolution System for Polish

Creating a Coreference Resolution System for Polish

Summary

This paper reports the first statistical approach to coreference resolution for Polish, a less-resourced language that previously had only rule-based experiments. The authors adapted BART (Beautiful Anaphora Resolution Toolkit), an open-source modular system originally built for English. Because BART's preprocessing and many of its feature extractors are tied to English, Polish texts were preprocessed outside the toolkit. Only 13 of BART's 64 feature extractors were kept, and a new Polish Language Plugin was written to map Polish tagset and morphological information into BART features. The test data came from the National Corpus of Polish. Mentions were chunked with Spejd, corrected by linguists, and stored in MMAX format. Under leave-one-out cross-validation, BART scored slightly below the authors' earlier rule-based system on every F1 measure, but the gap was small. The work shows that reusing an existing toolkit is a practical way to build a baseline for a new language. It also points out that language-specific tuning and larger annotated data are still needed.

Key points

  • Data: The data set has 15 texts of about 20 sentences each, with 5722 tokens, 1644 mentions and 1256 mention chains (singletons included). Zero anaphora was removed, unlike in the earlier rule-based test set.

  • Adaptation: The adaptation used 13 feature extractors covering gender and number agreement, head and string matching, mention type and sentence distance. The Polish Language Plugin was modelled on BART's German and Italian plugins.

  • F1 results (BART vs rule-based):

    • MUC: 61.38% vs 64.98%
    • B³: 88.21% vs 89.37%
    • CEAFM: 82.34% vs 83.94%
    • CEAFE: 85.40% vs 87.06%
    • BLANC: 78.43% vs 78.75%

    The paper's text says BART had higher precision on B³ and BLANC. The figures as reconstructed from the extracted table show the opposite: BART is higher on recall for these two metrics and lower on precision.

  • Scoring: BART natively supported only MUC, so the authors built a converter to the SemEval format to compute the other metrics.

  • Future work: The CORE project corpus already exceeds 20,000 words and 8,000 mentions. The authors also plan an extrinsic evaluation through summarization in the ATLAS project, comparing BART, the rule-based system, RARE and Reconcile.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{kopec2012creating,
  title = {{Creating a Coreference Resolution System for Polish}},
  author = {Kopeć, Mateusz and Ogrodniczuk, Maciej},
  booktitle = {Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)},
  year = {2012},
  doi = {10.63317/3s58utvtj3hu},
  url = {https://doi.org/10.63317/3s58utvtj3hu}
}

References

17 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In The First International Conference on Language Resources and Evaluation Workshop on Linguistics Coreference, pages 563–566. find our card
  2. Adam L. Berger, Stephen A. Della Pietra, and Vincent J. Della Pietra. 1996. A Maximum Entropy approach to Natural Language Processing. Computational Linguistics, 22:39–71. find our card
  3. Dan Cristea, Oana diana Postolache, Gabriela-Eugenia Dima, and Cătălina Barbu. 2002. AR-Engine – a framework for unrestricted coreference resolution. In Proceedings of the Third International Conference on Language Resources and Evaluation, LREC 2002, page 2000, Las Palmas, Canary Islands, Spain.… find
  4. Xiaoqiang Luo. 2005. On Coreference Resolution Performance Metrics. pages 25–32. find our card
  5. Ruslan Mitkov, Lamia Belguith, and Małgorzata Styś. 1998. Multilingual Robust Anaphora Resolution. In Proceedings of the Third International Conference on Empirical Methods in Natural Language Processing (EMNLP 1998), pages 7–16, Granada, Spain. find our card
  6. Christoph Müller and Michael Strube. 2006. Multi-level annotation of linguistic data with MMAX2. In Sabine Braun, Kurt Kohn, and Joybrato Mukherjee, editors, 3 Applied Technology for Language-Aided CMS co-funded by the European Commission under the Information and Communications Technologies (ICT)… find
  7. Corpus Technology and Language Pedagogy: New Resources, New Tools, New Methods, pages 197–214. Peter Lang, Frankfurt a.M., Germany. Maciej Ogrodniczuk and Mateusz Kopeć. 2011a. End-to-end coreference resolution baseline system for Polish. In Proceedings of the 5th Language & Technology Conference (… find
  8. Maciej Ogrodniczuk and Mateusz Kopeć. 2011b. Rule-based coreference resolution module for Polish. In Proceedings of the 8th Discourse Anaphora and Anaphor Resolution Colloquium (DAARC 2011), pages 191–200, Faro, Portugal. find
  9. Adam Przepiórkowski and Aleksander Buczyński. 2007. Spejd: Shallow parsing and disambiguation engine. In Proceedings of the 3rd Language & Technology Conference, Poznań. find
  10. Adam Przepiórkowski, Rafał L. Górski, Barbara Lewandowska-Tomaszczyk, and Marek Łaziński. 2008. Towards the National Corpus of Polish. In Proceedings of the Sixth International Conference on Language Resources and Evaluation, LREC 2008, Marrakech, Morocco. ELRA. find
  11. Marta Recasens and Eduard Hovy. 2010. BLANC: Implementing the Rand index for coreference evaluation. Natural Language Engineering, pages 1–26. find
  12. Marta Recasens, Eduard Hovy, and M. Antonia Marti. 2010a. A Typology of Near-Identity Relations for Coreference (NIDENT). In Nicoletta Calzolari, Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner, and Daniel Tapias, editors, Proceedings of the 7th Conference o… find our card
  13. Marta Recasens, Lluı́s Marquez, Mariona Taulé, M A Martı́, Véronique Hoste, Massimo Poesio, and Yannick Versley, 2010b. SemEval-2010 Task 1: Coreference Resolution in Multiple Languages, pages 70–75. Association for Computational Linguistics. find
  14. Veselin Stoyanov, Claire Cardie, Nathan Gilbert, Ellen Riloff, David Buttler, and David Hysom. 2010. Coreference resolution with reconcile. In Proceedings of the ACL 2010 Conference Short Papers, ACLShort ’10, pages 156–161, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  15. Yannick Versley, Simone Paolo Ponzetto, Massimo Poesio, Vladimir Eidelman, Alan Jern, Jason Smith, Xiaofeng Yang, and Alessandro Moschitti. 2008. BART: A modular toolkit for coreference resolution. In Association for Computational Linguistics (ACL) Demo Session. find our card
  16. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. November 1995. A Model-Theoretic Coreference Scoring Scheme. In Proceedings of the 6th Message Understanding Conference (MUC-6), pages 45–52. find our card
  17. Ian H. Witten, Eibe Frank, Len Trigg, Mark Hall, Geoffrey Holmes, and Sally Jo Cunningham. 1999. WEKA: Practical Machine Learning Tools and Techniques with Java Implementations. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/kopec-2012-creating-coreference-resolution-system-polish/