Elamur Research⌕
☰
home / papers / 2013 / Machine Translation with Many Manually Labeled Discourse Connectives

Machine Translation with Many Manually Labeled Discourse Connectives

Summary

Explicit discourse connectives such as while, as or meanwhile are hard for sentence-level SMT. One English form can signal several relations, and Czech may need a different connective, or even a different construction, for each one. Earlier English/French work used automatic classifiers that mislabel many instances. This paper instead uses gold-standard PDTB sense labels on the parallel PCEDT. That treebank pairs the Wall Street Journal texts with a human Czech translation, so manual relation labels for about 100 connective types can be fed straight into an EN→CZ system. The authors trained phrase-based Moses models in which each English connective is joined to its sense tag. One variant keeps the full tags and another keeps simplified tags; both are compared with an unlabeled baseline. Automatic scores were stable. For some ambiguous connectives, gains ranged from 4% to 60%, and the labels sometimes helped produce the right Czech syntax, such as a clause instead of a prepositional phrase. Manual analysis showed that labeling every connective is unnecessary. Only a handful of highly ambiguous ones (as, when, while, yet) really benefit.

Key points

  • Data: PCEDT 2.0 provides about 50k sentence-aligned EN/CZ sentences (WSJ sections 00–24). PDTB 2.0 contributes 18,459 manually annotated explicit connectives, with senses organized in a three-level hierarchy (4/16/23 classes) and up to 129 possible tags. Europarl EN/CZ was added to obtain enough training data.
  • Systems:
    • SYSTEM 1 uses the full PDTB tags, giving 63 observed sense tags.
    • SYSTEM 2 keeps only levels 1–2 and drops composite senses, giving 22 tags.
    • Both are compared with a BASELINE trained on the same text without labels.
  • Evaluation: Testing used PDTB section 23. BLEU was averaged over 5 MERT runs and checked with t-tests.
  • Manual comparison (SYSTEM 2 vs. BASELINE): On a sample of 630 labeled connectives, 8.2% of translations improved and 4.0% got worse, with the rest equal, once a recurring but–jenže case is excluded. For the connective as, SYSTEM 2 was better in 68% of 25 instances.
  • Future work: The authors plan factored translation models and want to select which connectives, and at what sense granularity, are worth labeling.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Discourse analysis · Machine translation

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 7 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{meyer2013machine,
  title = {{Machine Translation with Many Manually Labeled Discourse Connectives}},
  author = {Meyer, Thomas and Poláková, Lucie},
  booktitle = {Proceedings of the Workshop on Discourse in Machine Translation},
  year = {2013},
  url = {https://aclanthology.org/W13-3306/}
}

References

26 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Amal AlSaif. 2012. Human and Automatic Annotation of Discourse Relations for Arabic. Ph.D. thesis, University of Leeds. find
  2. Marine Carpuat and Michel Simard. 2012. The Trouble with SMT Consistency. In Proceedings of the 7th Workshop on Statistical Machine Translation (WMT), pages 442–449, Montreal, Canada. find
  3. Marine Carpuat and Dekai Wu. 2007. Improving Statistical Machine Translation using Word Sense Disambiguation. In Proceedings of Joint Conference on Empirical Methods in Natural Language Processing (EMNLP) and Computational Natural Language Learning (CoNLL), pages 61–72, Prague, Czech Republic. find
  4. Marine Carpuat. 2009. One Translation per Discourse. In Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions (SEW), pages 19–27, Singapore. find
  5. Yee Seng Chan, Hwee Tou Ng, and David Chiang. 2007. Word Sense Disambiguation Improves Statistical Machine Translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (ACL), pages 33–40, Prague, Czech Republic. find
  6. Pi-Chuan Chang, Dan Jurafsky, and Christopher D. Manning. 2009. Disambiguating ‘DE’ for Chinese-English Machine Translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation at the 12th Meeting of the European Chapter of the Association for Computational Linguistics (EACL), A… find
  7. Jonathan Clark, Chris Dyer, Alon Lavie, and Noah Smith. 2011. Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability. In Proceedings of ACL-HLT 2011 (46th Annual Meeting of the ACL: Human Language Technologies), Portland, OR. find
  8. Laurence Danlos, Diégo Antolinos-Basso, Chloé Braud, and Charlotte Roze. 2012. Vers le FDTB : French Discourse Tree Bank. In Actes de la conférence conjointe JEP-TALN-RECITAL 2012, volume 2: TALN, pages 471–478, Grenoble, France. find
  9. Vladimir Eidelman, Jordan Boyd-Graber, and Philip Resnik. 2012. Topic Models for Dynamic Translation Model Adaptation. In Proceedings of ACL 2012 (50th Annual Meeting of the Association for Computational Linguistics, pages 115–119, Jeju, Republic of Korea. find
  10. Jan Hajič, Eva Hajičová, Jarmila Panevová, Petr Sgall, Silvie Cinková, Eva Fučı́ková, Marie Mikulová, Petr Pajas, Jan Popelka, Jiřı́ Semecký, Jana Šindlerová, Jan Štěpánek, Josef Toman, Zdeňka Urešová, and Zdeněk Žabokrtský. 2011. Prague Czech-English Dependency Treebank 2.0. In… find
  11. Christian Hardmeier, Joakim Nivre, and Jörg Tiedemann. 2012. Document-Wide Decoding for Phrase-Based Statistical Machine Translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and Natural Language Learning (EMNLP-CoNLL), Jeju, Korea. find
  12. Christian Hardmeier. 2013. Discourse in Statistical Machine Translation. DISCOURS, 11:1–29. find
  13. Philipp Koehn and Hieu Hoang. 2007. Factored Translation Models. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing (EMNLP) and Computational Natural Language Learning (CONLL), pages 868– 876, Prague, Czech Republic. find our card
  14. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbs. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. In Proce… find
  15. Philipp Koehn. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Proceedings of MT Summit X, pages 79–86, Phuket, Thailand. find
  16. Jianjun Ma, Degen Huang, Haixia Liu, and Wenfeng Sheng. 2011. POS Tagging of English Particles for Machine Translation. In Proceedings of the Thirteenth Machine Translation Summit, pages 57–63, Xiamen, China. find
  17. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. find our card
  18. Thomas Meyer and Andrei Popescu-Belis. 2012. Using Sense-labeled Discourse Connectives for Statistical Machine Translation. In Proceedings of the EACL 2012 Joint Workshop on Exploiting Synergies between IR and MT, and Hybrid Approaches to MT (ESIRMT-HyTra), pages 129–138, Avignon, FR. find our card
  19. Lucie Mladová, Šárka Zikánová, Zuzanna Bedřichová, and Eva Hajičová. 2009. Towards a discourse corpus of Czech. In Proceedings of the Corpus Linguistics Conference, Liverpool, UK. find
  20. Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL), pages 160–167, Sapporo, Japan. find
  21. Marie-Paule Péry-Woodley, Nicholas Asher, Patrice Enjalbert, Farah Benamara, Myriam Bras, Cécile Fabre, Stéphane Ferrari, Lydia-Mai Ho-Dac, Anne Le Draoulec, Yann Mathet, Philippe Muller, Laurent Prévot, Josette Rebeyrolle, Ludovic Tanguy, Marianne Vergez-Couret, Laure Vieu, and Antoine Widlöch… find
  22. Andrei Popescu-Belis, Thomas Meyer, Jeevanthi Liyanapathirana, Bruno Cartoni, and Sandrine Zufferey. 2012. Discourse-level Annotation over Europarl for Machine Translation: Connectives and Pronouns. In Proceedings of the eighth international conference on Language Resources and Evaluation (LREC), Is… find
  23. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn Discourse Treebank 2.0. In Proceedings of 6th International Conference on Language Resources and Evaluation (LREC), pages 2961–2968, Marrakech, Morocco. find our card
  24. Manfred Stede. 2004. The Potsdam Commentary Corpus. In Proceedings of the ACL Workshop on Discourse Annotation, pages 96–102, Barcelona, Spain. find
  25. Andreas Stolcke, Jing Zheng, Wen Wang, and Victor Abrash. 2011. SRILM at Sixteen: Update and Outlook. In Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop, Waikoloa, Hawaii. find
  26. Yuping Zhou and Nianwen Xue. 2012. PDTB-style Discourse Annotation of Chinese Text. In Proceedings of the 50th Annual Meeting on Association for Computational Linguistics (ACL), Jeju Island, Korea. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/meyer-2013-machine-translation-many-manually-labeled/