Elamur Research⌕
☰
home / papers / 2014 / ParCor 1.0: A Parallel Pronoun-Coreference Corpus to Support Statistical MT

ParCor 1.0: A Parallel Pronoun-Coreference Corpus to Support Statistical MT

Summary

Statistical MT systems often mistranslate pronouns because they ignore what the pronoun refers to. They tend to fall back on the most frequent target form, for example rendering English "it" with a single fixed gender. To study and address this, the authors built ParCor 1.0, an English–German parallel corpus with manual pronoun-coreference annotation. It covers two genres: EU Bookshop publications (formal written text) and TED Talks from the IWSLT2013 test set (transcribed planned speech). Annotators labelled each pronoun's position and type, such as anaphoric, cataphoric, event, pleonastic, speaker or addressee reference, or generic. Where relevant, they also linked the pronoun to its antecedent, working in MMAX-2 with automatically pre-annotated input. The German side contains more pronouns overall, especially more pleonastic ones. The two genres also differ clearly: TED speakers rely heavily on first- and second-person forms. Inter-annotator agreement (Cohen's Kappa) was high. The resource can serve as a gold standard for evaluating pronoun translation, as data on cross-lingual pronoun differences, and potentially for anaphora resolution work. The authors describe it as the first parallel coreference corpus built specifically for SMT.

Key points

  • Data: 8 EU Bookshop documents and 11 TED Talks, all originally in English (one EU text has an unknown source language). The surrounding larger corpora, WIT3 and the OPUS EUBookshop corpus, can supply in-domain SMT training data and further languages.
  • Pronoun counts: TED has 2,424 (EN) and 3,006 (DE); EU Bookshop has 4,140 (EN) and 4,689 (DE).
  • Pleonastic pronouns are far more frequent in German: 223 vs 61 in TED and 391 vs 191 in EU Bookshop. Whether this is a systematic language difference or a translation effect remains open.
  • IAA via Cohen's Kappa ranged from 0.82 to 1.00 for EU Bookshop documents and from 0.75 to 0.98 for English TED Talks.
  • Planned next steps include expanding to more languages and genres, adding more automatic pre-annotation, and launching a shared task on pronoun translation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution

Cited by

30 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 22 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{guillou2014parcor,
  title = {{ParCor 1.0: A Parallel Pronoun-Coreference Corpus to Support Statistical MT}},
  author = {Guillou, Liane and Hardmeier, Christian and Smith, Aaron and Tiedemann, Jörg and Webber, Bonnie},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  url = {https://aclanthology.org/L14-1268/}
}

References

30 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bergsma, S. and Yarowsky, D. (2011). NADA: A Robust System for Non-Referential Pronoun Detection. In Proceedings of DAARC 2011, pages 12–23. find
  2. Broscheit, S., Ponzetto, S. P., Versley, Y., and Poesio, M. (2010). Extending BART to Provide a Coreference Resolution System for German. In Proceedings of LREC 2010. find our card
  3. Cartoni, B., Zufferey, S., Meyer, T., and Popescu-Belis, A. (2011). How Comparable Are Parallel Corpora? Measuring the Distribution of General Vocabulary and Connectives. In Proceedings of BUCC 2011, pages 78–86. find
  4. Cettolo, M., Girardi, C., and Federico, M. (2012). WIT 3 : Web Inventory of Transcribed and Translated Talks. In Proceedings of EAMT 2012, pages 261–268. find
  5. Chinchor, N. and Hirschman, L. (1998). MUC-7 Coreference Task Definition (Version 3.0). In Proceedings of MUC-7. find our card
  6. Cohen, J. (1960). A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement, 20(1). de Marneffe, M.-C., MacCartney, B., and Manning, C. D. (2006). Generating Typed Dependency Parses from Phrase Structure Parses. In Proceedings of LREC 2006, pages 449–454. de Souza, J.… find
  7. (EUB). EU Bookshop Online Archive. https://bookshop.europa.eu. Accessed: 2014-03-07. ↗
  8. Guillou, L. (2012). Improving Pronoun Translation for Statistical Machine Translation. In Proceedings of the Student Research Workshop, EACL 2012, pages 1–10. find
  9. Hajič, J., Hajičová, E., Panevová, J., Sgall, P., Bojar, O., Cinková, S., Fučı́ková, E., Mikulová, M., Pajas, P., Popelka, J., Semecký, J., Šindlerová, J., Štěpánek, J., Toman, J., Urešová, Z., and Žabokrtský, Z. (2012). Announcing Prague Czech-English Dependency Treebank 2.0. In P… find
  10. Halliday, M. A. and Hasan, R. (1976). Cohesion in English. Longman, London. find
  11. Hardmeier, C. and Federico, M. (2010). Modelling Pronominal Anaphora in Statistical Machine Translation. In Proceedings of the 7th International Workshop on Spoken Language Translation. find our card
  12. Hardmeier, C., Tiedemann, J., and Nivre, J. (2013). Latent Anaphora Resolution for Cross-Lingual Pronoun Prediction. In Proceedings of EMNLP. find our card
  13. Le Nagard, R. and Koehn, P. (2010). Aiding Pronoun Translation with Co-Reference Resolution. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 252–261. find our card
  14. Lee, H., Peirsman, Y., Chang, A., Chambers, N., Surdeanu, M., and Jurafsky, D. (2011). Stanford’s Multi-pass Sieve Coreference Resolution System at the CoNLL-2011 Shared Task. In Proceedings of CoNLL-2011, pages 28–34. find our card
  15. Marcus, M. P., Santorini, B., and Marcinkiewicz, M. A. (1993). Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
  16. Meyer, T. and Poláková, L. (2013). Machine Translation with Many Manually Labeled Discourse Connectives. In Proceedings of the 1st Workshop on Discourse in Machine Translation, ACL 2013, pages 43–50. find our card
  17. Mitkov, R. and Barbu, C. (2003). Using Bilingual Corpora to Improve Pronoun Resolution. Languages in Contrast, 4(2):201–211. find
  18. Müller, C. and Strube, M. (2006). Multi-Level Annotation of Linguistic Data with MMAX2. In Braun, S., Kohn, K., and Mukherjee, J., editors, Corpus Technology and Language Pedagogy: New Resources, New Tools, New Methods, pages 197–214. Peter Lang. find
  19. Naumann, K. and Möller, V. (2007). Manual for the Annotation of in-document Referential Relations. Technical report, Universität Tübingen Seminar für Sprachwissenschaft. find
  20. Novák, M. (2011). Utilization of Anaphora in Machine Translation. In Proceedings of Contributed Papers, Week of Doctoral Students 2011, pages 155–160. find
  21. Petrov, S., Barrett, L., Thibaux, R., and Klein, D. (2006). Learning Accurate, Compact, and Interpretable Tree Annotation. In Proceedings of ACL 2006, pages 433–440. find our card
  22. Popescu-Belis, A., Meyer, T., Liyanapathirana, J., Cartoni, B., and Zufferey, S. (2012). Discourse-level Annotation over EuroParl for Machine Translation. In Proceedings of LREC 2012, pages 2716–2720. find
  23. Postolache, O., Cristea, D., and Orăsan, C. (2006). Transferring Coreference Chains through Word Alignment. In Proceedings of LREC 2006, pages 889–892. find our card
  24. Pradhan, S., Ramshaw, L., Marcus, M., Palmer, M., Weischedel, R., and Xue, N. (2011). CoNLL-2011 Shared Task: Modeling Unrestricted Coreference in OntoNotes. In Proceedings of CoNLL 2011, pages 1–27. find our card
  25. Prasad, R., Dinesh, N., Lee, A., Miltsakaki, E., Robaldo, L., Joshi, A., and Webber, B. (2008). The Penn Discourse TreeBank 2.0. In Proceedings of LREC 2008. find our card
  26. (TED). IWSLT2013/WIT TED Talks. https://wit3.fbk.eu. Accessed: 2014-03-07. ↗
  27. Versley, Y., Beck, K., Hinrichs, E., and Telljohann, H. (2010). A Syntax-first Approach to High-quality Morphological Analysis and Lemma Disambiguation for the TüBa-D/Z Treebank. In Proceedings of TLT9. find
  28. Weischedel, R. and Brunstein, A. (2005). BBN Pronoun Coreference and Entity Type Corpus. find
  29. Weischedel, R., Palmer, M., Mitchell, M., Hovy, E., Pradhan, S., Ramshaw, L., Xue, N., Taylor, A., Kaufman, J., Franchini, M., El-Bachouti, M., Belvin, R., and Houston, A. (2011). OntoNotes version 4.0. find
  30. Žabokrtský, Z., Ptácěk, J., and Pajas, P. (2008). TectoMT: Highly Modular MT System with Tectogrammatics Used as Transfer Layer. In Proceedings of the Third Workshop on Statistical Machine Translation, ACL 2008, pages 167–170. find

↑ back to top