Elamur Research⌕
☰
home / papers / 2017 / Multi-source annotation projection of coreference chains: assessing strategies and testing opportunities

Multi-source annotation projection of coreference chains: assessing strategies and testing opportunities

Summary

Coreference resolvers trained only on annotations projected from one source language have stayed below 40 F1. The authors test whether combining projections from two source languages can produce better target-side coreference annotations. They use a small trilingual English-German-Russian parallel corpus of news and stories with gold coreference chains. German and Russian each serve as the target, and the other two languages act as sources. Because chain IDs do not match across languages, projected chains are treated as clusters and compared by mention overlap. They are then merged through concatenation-style or intersection-style strategies. The experiments run in two settings: one using word alignments alone, and one that also maps projected spans onto mentions extracted from dependency parses. Two-source projection beats single-source projection on Precision or Recall, though never on both together. Concatenating overlapping chains gives the best balance between the two, and intersection gives the highest Precision. Projections from non-English sources are weaker, which limits the gains. The study suggests multi-source transfer as a way to bootstrap coreference data for languages with few resources.

Key points

  • Data: The trilingual corpus from Grishina and Stede (2015) contains 413 parallel sentences, about 8–8.7k tokens per language, and roughly 1,026–1,077 markables per language. Sentences were aligned with LF Aligner and words with GIZA++.
  • Chain merging: Chains count as overlapping when their mention overlap reaches 0.5 (German) or 0.7 (Russian). Below these thresholds they are treated as disjoint.
  • Mention extraction: The second setting extracts mentions using the MATE parser for German and the MALT parser for Russian. The mapping strategy is adapted from Rahman and Ng (2012).
  • Best F1: The best scores are 57.0 for German and 59.0 for Russian. The u-con method improves over single-source projection from 55.0 to 57.0 for German and from 58.8 to 59.0 for Russian.
  • Error analysis: Noun phrases are projected less reliably than pronouns. Mention extraction helps German considerably but helps Russian only slightly. Recall is harder to raise than Precision.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-1506

@inproceedings{grishina2017multi,
  title = {{Multi-source annotation projection of coreference chains: assessing strategies and testing opportunities}},
  author = {Grishina, Yulia and Stede, Manfred},
  booktitle = {Proceedings of the 2nd Workshop on Coreference Resolution Beyond OntoNotes (CORBON 2017)},
  year = {2017},
  doi = {10.18653/v1/w17-1506},
  url = {https://doi.org/10.18653/v1/w17-1506}
}

References

20 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alan Akbik, Laura Chiticariu, Marina Danilevsky, Yunyao Li, Shivakumar Vaithyanathan, and Huaiyu Zhu. 2015. Generating high quality proposition banks for multilingual semantic role labeling. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Intern… find our card
  2. Maciej Ogrodniczuk. 2013. Translation-and projection-based unsupervised coreference resolution for Polish. In Language Processing and Intelligent Information Systems, pages 125–130. Springer. find
  3. Oana Postolache, Dan Cristea, and Constantin Orasan. 2006. Transferring coreference chains through word alignment. In Proceedings of 5th international conference on Language Resources and Evaluation (LREC). find our card
  4. Bernd Bohnet. 2010. Very high accuracy and fast dependency parsing is not a contradiction. In Proceedings of the 23rd international conference on computational linguistics, pages 89–97. Association for Computational Linguistics. find our card
  5. Sameer S. Pradhan and Nianwen Xue. 2009. Ontonotes: The 90% solution. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Tutorial Abstracts, pages 11–12, Boulder, Colorado, May. A… find our card
  6. Yulia Grishina and Manfred Stede. 2015. Knowledge-lean projection of coreference chains across languages. In Proceedings of the Eighth Workshop on Building and Using Comparable Corpora, pages 14–22, Beijing, China, July. Association for Computational Linguistics. find our card
  7. Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL - Shared Task, pages 1–40, Jeju Island, Korea, July. Association for Computational Lin… find our card
  8. Yulia Grishina and Manfred Stede, 2016. Parallel coreference annotation guidelines. Unpublished Manuscript 13 . find
  9. Anders Johannsen, Željko Agić, and Anders Søgaard. 2016. Joint part-of-speech and dependency projection from multiple sources. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 561–566, Berlin, Germany, August. Association fo… find
  10. Altaf Rahman and Vincent Ng. 2012. Translation-based projection for multilingual coreference resolution. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 720–730. Association for Computational Lin… find our card
  11. Ophélie Lacroix, Lauriane Aufrant, Guillaume Wisniewski, and François Yvon. 2016. Frustratingly easy cross-lingual transfer for transition-based dependency parsing. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… find
  12. Mohammad Sadegh Rasooli and Michael Collins. 2015. Density-driven cross-lingual transfer of dependency parsers. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 328–338. Association for Computational Linguistics. find
  13. Helmut Schmid. 1995. Probabilistic part-of-speech tagging using decision trees. In Proceedings of the ACL SIGDAT-Workshop. Association for Computational Linguistics. find
  14. André F. T. Martins. 2015. Transferring coreference resolvers with posterior regularization. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1427–1437,… find our card
  15. Serge Sharoff and Joakim Nivre. 2011. The proper place of men and machines in language technology: Processing russian without any linguistic knowledge. In Proc. find
  16. Dialogue 2011, Russian Conference on Computational Linguistics. find
  17. Joakim Nivre, Johan Hall, and Jens Nilsson. Maltparser: A data-driven parser-generator for dependency parsing. In Proceedings of 5th international conference on Language Resources and Evaluation (LREC). José Guilherme Camargo Souza and Constantin Orăsan. 2011. Can projected chains in parallel corp… find
  18. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
  19. Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In Proceedings of the 2016 Conference of the North American Chapter of the 13 Available at https://github.com/yuliagrishina/CORBON-2017-Shared-Task/blob/master/Parallel annotation guidel… ↗ our card
  20. David Yarowsky, Grace Ngai, and Richard Wicentowski. 2001. Inducing multilingual text analysis tools via robust projection across aligned corpora. In Proceedings of the first international conference on Human language technology research, pages 1– 8. Association for Computational Linguistics. find

↑ back to top

This page in Markdown →