Elamur Research
☰
home / papers / 2017 / Findings of the 2017 Conference on Machine Translation (WMT17)

Findings of the 2017 Conference on Machine Translation (WMT17)

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/W17-4717.pdf

Summary

WMT17 is an annual benchmarking campaign that tracks the state of machine translation and supplies common test sets, training corpora and human judgments. This overview covers three of its shared tasks. The news task covered 14 translation directions, each pairing English with one of seven languages: German, Czech, Russian, Chinese, Finnish, Turkish or Latvian. Latvian, a lower-resourced pair, and Chinese, run together with CWMT, were new this year. Professional translators built test sets of about 3,000 sentences per pair, half originally written on each side. Organizers also released parallel and monolingual training data, including the new EU Press Release corpus. In total, 31 institutions submitted 103 systems. These were ranked officially by human Direct Assessment of adequacy on a 100-point scale, gathered from researchers and Mechanical Turk workers. The quality estimation task asked for sentence-level effort scores and word- or phrase-level good/bad labels. The automatic post-editing (APE) task asked for corrections to the output of an unknown MT engine. In APE, neural approaches beat the baseline for every participant for the first time, yet still fall short of human post-edits. All data and judgments are public.

Key points

  • News task: 103 submissions from 31 institutions across 14 language directions. Test sets have roughly 3,000 sentences, except Latvian, which has 2,000 test plus 2,000 development sentences. Outputs can be compared interactively with MT-ComparEval at wmt.ufal.cz.
  • Training resources included Europarl, Common Crawl, the Yandex 1M Russian–English corpus, the United Nations corpus, CzEng v1.6 and News Commentary v12. New this year were the EU Press Release corpus (German, Finnish, Latvian) and Chinese corpora such as casict2015, neu2017 and datum2017.
  • Quality estimation and APE data came from two domains: English→German IT texts and German→English pharmaceutical texts. Quality estimation was split into sentence-level, word-level and phrase-level subtasks.
  • In APE (third edition), the top English→German system gained up to −4.9 TER and +7.6 BLEU over the baseline. The German→English gains were small (−0.26 TER, +0.28 BLEU) but statistically significant.
  • Human Direct Assessment in APE showed that all English→German systems beat the no-post-edit baseline. The German→English primary systems clustered with the baseline. TER and BLEU gave slightly different rankings, with DA agreeing somewhat more with BLEU.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cited by

290 papers of our library cite it (of the 140673 whose bibliographies we read); 32 of them have a page here, 258 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-4717

@inproceedings{bojar2017findings,
  title = {{Findings of the 2017 Conference on Machine Translation (WMT17)}},
  author = {Bojar, Ondřej and Chatterjee, Rajen and Federmann, Christian and Graham, Yvette and Haddow, Barry and Huang, Shujian and Huck, Matthias and Koehn, Philipp and Liu, Qun and Logacheva, Varvara and Monz, Christof and Negri, Matteo and Post, Matt and Rubino, Raphael and Specia, Lucia and Turchi, Marco},
  booktitle = {Proceedings of the Second Conference on Machine Translation},
  year = {2017},
  doi = {10.18653/v1/w17-4717},
  url = {https://doi.org/10.18653/v1/w17-4717}
}

References

109 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hervé Abdi. 2007. The bonferroni and šidák corrections for multiple comparisons. Encyclopedia of measurement and statistics, 3:103–107. find
  2. Eleftherios Avramidis. 2017a. Comparative quality estimation for machine translation: Observations on machine learning and features. The Prague Bulletin of Mathematical Linguistics (PBML), 108:307–318. find
  3. Eleftherios Avramidis. 2017b. Sentence-level quality estimation by predicting HTER as a multi-component metric. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  4. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. find our card
  5. Alexandre Berard, Laurent Besacier, and Olivier Pietquin. 2017. LIG-CRIStAL Submission for the WMT 2017 Automatic Post-Editing Task. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  6. Ergun Biçici. 2017. Predicting Translation Performance with Referential Translation Machines. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  7. Frédéric Blain, Varvara Logacheva, and Lucia Specia. 2016. Phrase level segmentation and labelling of machine translation errors. LREC16. find
  8. Frédéric Blain, Carolina Scarton, and Lucia Specia. 2017. Bilexical Embeddings for Quality Estimation. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  9. Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Aleš Tamchyna. 2014. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the… find our card
  10. Ondřej Bojar, Yvette Graham, and Amir Kamran. 2017a. Results of the WMT17 Metrics Shared Task. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find our card
  11. Ondřej Bojar, Jindřich Helcl, Tom Kocmi, Jindřich Libovický, and Tomáš Musil. 2017b. Results of the WMT17 Neural MT Training Task. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  12. Ondřej Bojar, Christian Buck, Chris Callison-Burch, Christian Federmann, Barry Haddow, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2013. Findings of the 2013 Workshop on Statistical Machine Translation. In Proceedings of the Eighth Workshop on Statistical Machine Transl… find our card
  13. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi,… find our card
  14. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, and Marco Turchi. 2015. Findings of the 2015 Workshop on Statistical Machine Translation. In Proc… find our card
  15. Ondřej Bojar, Ondřej Dušek, Tom Kocmi, Jindřich Libovický, Michal Novák, Martin Popel, Roman Sudarikov, and Dušan Variš. 2016b. CzEng 1.6: Enlarged Czech-English Parallel Corpus with Processing Tools Dockered. In Text, Speech and Dialogue: 19th International Conference, TSD 2016, Brno, Czech… find
  16. Franck Burlot, Pooyan Safari, Matthieu Labeau, Alexandre Allauzen, and François Yvon. 2017. LIMSI@WMT’17. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  17. Chris Callison-Burch, Cameron Shaw Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2007. (Meta-) evaluation of machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 136–158, Prague, Czech Methods for MT Automatic Post-Editing. In Proceedings… find
  18. Chris Callison-Burch, Cameron Shaw Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2008. Further meta-evaluation of machine translation. In Proceedings of the Third Workshop on Statistical Machine Translation, pages 70–106, Columbus, Ohio. Association for Computational Linguistics. find
  19. Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, and Omar Zaidan. 2010. Findings of the 2010 joint workshop on statistical machine translation and metrics for machine translation. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and Metri… find
  20. Chris Callison-Burch, Philipp Koehn, Christof Monz, Matt Post, Radu Soricut, and Lucia Specia. 2012. Findings of the 2012 workshop on statistical machine translation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 10–48, Montreal, Canada. Association for Computation… find our card
  21. Zhiming Chen, Yiming Tan, Chenlin Zhang, Qingyu Xiang, Lilin Zhang, Maoxi Li, and Mingwen WANG. 2017. Improving Machine Translation Quality Estimation with Neural Network Features. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Ling… find
  22. Yongchao Deng, Jungi Kim, Guillaume Klein, Catherine KOBUS, Natalia Segal, Christophe Servan, Bo Wang, Dakun Zhang, Josep Crego, and Jean Senellart. 2017. SYSTRAN Purely Neural MT Engines for WMT2017. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association fo… find
  23. Mattia Antonino Di Gangi, Nicola Bertoldi, and Marcello Federico. 2017. FBK’s Participation to the English-to-German News Translation Task of WMT 2017. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  24. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Josh Schroeder. 2009. Findings of the 2009 Workshop on Statistical Machine Translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 1–28, Athens, Greece. Association for Computational Linguistics. find
  25. Shuoyang Ding, Huda Khayrallah, Philipp Koehn, Matt Post, Gaurav Kumar, and Kevin Duh. 2017. The JHU Machine Translation Systems for WMT 2017. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  26. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Omar Zaidan. 2011. Findings of the 2011 workshop on statistical machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 22–64, Edinburgh, Scotland. Association for Computational Linguistics. find
  27. John Duchi and Yoram Singer. 2009. Efficient online and batch learning using forward backward splitting. Journal of Machine Learning Research, 10:2899– 2934. find
  28. Rajen Chatterjee, M. Amin Farajian, Matteo Negri, Marco Turchi, Ankit Srivastava, and Santanu Pal. 2017. Multi-source Neural Automatic Post-Editing: FBK’s participation in the WMT 2017 APE shared task. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association f… find
  29. Rajen Chatterjee, José G. C. de Souza, Matteo Negri, and Marco Turchi. 2016. The FBK Participation in the WMT 2016 Automatic Post-editing Shared Task. In Proceedings of the 11th Workshop on Statistical Machine Translation (WMT). find
  30. Rajen Chatterjee, Turchi Turchi, and Matteo Negri. 2015a. The FBK Participation in the WMT15 Automatic Post-editing Shared Task. In Proceedings of the 10th Workshop on Statistical Machine Translation (WMT). find
  31. Rajen Chatterjee, Marion Weller, Matteo Negri, and Marco Turchi. 2015b. Exploring the Planet of the APEs: a Comparative Study of State-of-the-art Melania Duma and Wolfgang Menzel. 2017. UHH Submission to the WMT17 Quality Estimation Shared Task. In Proceedings of the Second Conference on Machine Tra… find
  32. Desmond Elliott, Stella Frank, Loı̈c Barrault, Fethi Bougares, and Lucia Specia. 2017. Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computa… find
  33. Carlos Escolano, Marta R. Costa-jussà, and José A. R. Fonollosa. 2017. The TALP-UPC Neural Machine Translation System for German/Finnish-English Using the Inverse Direction Model in Rescoring. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Comp… find
  34. Christian Federmann. 2012. Appraise: an open-source toolkit for manual evaluation of mt output. The Prague Bulletin of Mathematical Linguistics, 98:25– 35. find
  35. Qin Gao and Stephan Vogel. 2008. Parallel Implementations of Word Alignment Tool. In Proceedings of the ACL 2008 Software Engineering, Testing, and Quality Assurance Workshop, pages 49–57, Columbus, Ohio. find
  36. Mercedes Garcı́a-Martı́nez, Ozan Caglayan, Walid Aransa, Adrien Bardet, Fethi Bougares, and Loı̈c Barrault. 2017. LIUM Machine Translation Systems for WMT17 News Translation Task. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Lingu… find
  37. Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. 2013. Continuous measurement scales in human evaluation of machine translation. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 33–41, Sofia, Bulgaria. Association for Computational… find
  38. Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. 2014. Is machine translation getting better over time? In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 443–451, Gothenburg, Sweden. Association for Computational L… find
  39. Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. 2016. Can machine translation systems be evaluated by the crowd alone. Natural Language Engineering, pages 1–28. find
  40. Yvette Graham, Nitika Mathur, and Timothy Baldwin. 2015. Accurate evaluation of segment-level machine translation metrics. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies, Denver, Colorado. find
  41. Ralf Herbrich, Tom Minka, and Thore Graepel. 2007. Trueskill TM : a bayesian skill rating system. In Advances in neural information processing systems, pages 569–576. find
  42. Teresa Herrmann, Jan Niehues, and Alex Waibel. 2013. Combining Word Reordering Methods on different Linguistic Abstraction Levels for Statistical Machine Translation. In Proceedings of the Seventh Workshop on Syntax, Semantics and Structure in Statistical Translation, Altanta, Georgia, USA. find
  43. Chris Hokamp. 2017. Ensembling Factored Neural Machine Translation Models for Automatic Post-Editing and Quality Estimation. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  44. Chester Holtz, Chuyang Ke, and Daniel Gildea. 2017. University of Rochester WMT 2017 NMT System Submission. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  45. Matthias Huck, Fabienne Braune, and Alexander Fraser. 2017. LMU Munich’s Neural Machine Translation Systems for News Articles and Health Information Texts. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  46. Arvi Hurskainen and Jörg Tiedemann. 2017. Rule-based Machine translation from English to Finnish. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  47. Stig-Arne Grönroos, Sami Virpioja, and Mikko Kurimo. 2017. Extending hybrid word-character neural machine translation with multi-task learning of morphological analysis. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  48. Antonio Jimeno Yepes, Aurelie Neveol, Mariana Neves, Karin Verspoor, Ondrej Bojar, Arthur Boyer, Cristian Grozea, Barry Haddow, Madeleine Kittner, Yvonne Lichtblau, Pavel Pecina, Roland Roller, Rudolf Rosa, Amy Siu, Philippe Thomas, and Saskia Trescher. 2017. Findings of the WMT 2017 Biomedical Tran… find
  49. Jeremy Gwinnup, Timothy Anderson, Grant Erdmann, Katherine Young, Michaeel Kazi, Elizabeth Salesky, Brian Thompson, and Jonathan Taylor. 2017. The AFRL-MITLL WMT17 Systems: Old, New, Borrowed, BLEU. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for… find
  50. Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2016. Log-linear combinations of monolingual and bilingual neural machine translation models for automatic post-editing. In Proceedings of the First Conference on Machine Translation, pages 751– 758, Berlin, Germany. Association for Computational Lingui… find
  51. Kenneth Heafield. 2011. KenLM: Faster and Smaller Language Model Queries. In Proceedings of the EMNLP 2011 Sixth Workshop on Statistical Machine Translation, pages 187–197, Edinburgh, Scotland, United Kingdom. find our card
  52. Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2017. An Exploration of Neural Sequence-to-Sequence Architecturesfor Automatic Post-Editing. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. find
  53. Hyun Kim and Jong-Hyeok Lee. 2016. Recurrent neural network based translation quality estimation. In Proceedings of the First Conference on Machine Translation, pages 787–792, Berlin, Germany. Association for Computational Linguistics. find
  54. Hyun Kim, Jong-Hyeok Lee, and Seung-Hoon Na. 2017. Predictor-Estimator using Multilevel Task Learning with Stack Propagation for Neural Quality Estimation. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  55. Philipp Koehn. 2004. Statistical Significance Tests for Machine Translation Evaluation. In Proceedings of EMNLP 2004, pages 388–395, Barcelona, Spain. find
  56. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. In Proc… find
  57. Pranava Swaroop Madhyastha, Xavier Carreras, and Ariadna Quattoni. 2014. Learning task-specific bilexical embeddings. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 161– 171, Dublin, Ireland. find
  58. André Martins, Marcin Junczys-Dowmunt, Fabio Kepler, Ramn Astudillo, Chris Hokamp, and Roman Grundkiewicz. 2017a. Pushing the limits of translation quality estimation. Transactions of the Association for Computational Linguistics, 5:205–218. find
  59. André F. T. Martins, Ramón Astudillo, Chris Hokamp, and Fabio Kepler. 2016. Unbabel’s participation in the wmt16 word-level translation quality estimation shared task. In Proceedings of the First Conference on Machine Translation, pages 806–811, Berlin, Germany. Association for Computational Lingu… find
  60. André F. T. Martins, Fabio Kepler, and Jose Monteiro. 2017b. Unbabel’s Participation in the WMT17 Translation Quality Estimation Shared Task. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  61. Philipp Koehn and Christof Monz. 2006. Manual and automatic evaluation of machine translation between european languages. In Proceedings on the Workshop on Statistical Machine Translation, pages 102– 121, New York City. Association for Computational Linguistics. find
  62. Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, ACL ’03, pages 160– 167, Sapporo, Japan. find
  63. Julia Kreutzer, Shigehiko Schamoni, and Stefan Riezler. 2015. Quality estimation from scratch (quetch): Deep learning for word-level translation quality estimation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 316–322, Lisbon, Portugal. Association for Computational… find
  64. Naoaki Okazaki. 2007. CRFsuite: a fast implementation of Conditional Random Fields. find
  65. Jason Lee, Kyunghyun Cho, and Thomas Hofmann. 2016. Fully character-level neural machine translation without explicit segmentation. arXiv preprint arXiv:1610.03017. find our card
  66. Jindr̆ich Libovický, Jindr̆ich Helcl, Marek Tlustý, Ondr̆ej Bojar, and Pavel Pecina. 2016. CUNI at Post-editing and Multimodal Translation Tasks. In Proceedings of the 11th Workshop on Statistical Machine Translation (WMT), Berlin, Germany. find
  67. Robert Östling, Yves Scherrer, Jörg Tiedemann, Gongbo Tang, and Tommi Nieminen. 2017. The Helsinki Neural Machine Translation System. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  68. Gustavo Paetzold and Lucia Specia. 2017. Feature-Enriched Character-Level Convolutions for Text Regression. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  69. Varvara Logacheva, Chris Hokamp, and Lucia Specia. 2016. MARMOT: A Toolkit for Translation Quality Estimation at the Word Level. In Proceedings of the 10th edition of the Language Resources and Evaluation Conference, Portorož, Slovenia. find
  70. Santanu Pal, Sudip Kumar Naskar, and Josef van Genabith. 2016a. Multi-engine and multi-alignment based automatic post-editing and its impact on translation productivity. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2559– 2570,… find
  71. The COLING 2016 Organizing Committee. find
  72. Ngoc Quang Luong, Laurent Besacier, and Benjamin Lecouteux. 2014. Lig system for word level qe task at wmt14. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 335–341, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  73. Santanu Pal, Sudip Kumar Naskar, Mihaela Vela, and Josef van Genabith. 2016b. A Neural Network based Approach to Automatic Post-Editing. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 281–286, Berlin, Germany. find
  74. Santanu Pal, Marcos Zampieri, and Josef van Genabith. 2016c. Usaar: An operation sequential model for automatic statistical post-editing. In Proceedings of the First Conference on Machine Translation, pages 759–763, Berlin, Germany. find
  75. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318, Philadelphia, Pennsylvania. find our card
  76. Raj Nath Patel and Sasikumar M. 2016. Translation quality estimation using recurrent neural network. In Proceedings of the First Conference on Machine Translation, pages 819–824, Berlin, Germany. Association for Computational Linguistics. find
  77. Jan-Thorsten Peter, Andreas Guta, Tamer Alkhouli, Parnia Bahar, Jan Rosendahl, Nick Rossenbach, Miguel Graça, and Hermann Ney. 2017a. The RWTH Aachen University English-German and German-English Machine Translation System for WMT 2017. In Proceedings of the Second Conference on Machine Translation,… find
  78. Jan-Thorsten Peter, Hermann Ney, Ondřej Bojar, Ngoc-Quan Pham, Jan Niehues, Alex Waibel, Franck Burlot, François Yvon, Mārcis Pinnis, Valters Sics, Joost Bastings, Miguel Rios, Wilker Aziz, Philip Williams, Frédéric Blain, and Lucia Specia. 2017b. The QT21 Combined Machine Translation System fo… find
  79. Ngoc-Quan Pham, Jan Niehues, Thanh-Le Ha, Eunah Cho, Matthias Sperber, and Alexander Waibel. 2017. The Karlsruhe Institute of Technology Systems for the News Translation Task in WMT 2017. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computation… find
  80. Mārcis Pinnis, Rihards Krišlauks, Toms Miks, Daiga Deksne, and Valters Šics. 2017. Tilde’s Machine Translation Systems for WMT 2017. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  81. Sylvain Raybaud, David Langlois, and Kamel Smali. 2011. this sentence is wrong. detecting errors in machine-translated sentences. Machine Translation, 25(1):1–34. find
  82. Matı̄ss Rikters, Chantal Amrhein, Maksym Del, and Mark Fishel. 2017. C-3MA: Tartu-Riga-Zurich Translation Systems for WMT17. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  83. Keisuke Sakaguchi, Matt Post, and Benjamin Van Durme. 2014. Efficient elicitation of annotations for human evaluation of machine translation. In Proceedings of the Workshop on Statistical Machine Translation, Baltimore, Maryland. Association for Computational Linguistics. find
  84. Carolina Scarton, Daniel Beck, Kashif Shah, Karin Sim Smith, and Lucia Specia. 2016. Word embeddings and discourse information for quality estimation. In Proceedings of the First Conference on Machine Translation, pages 831–837, Berlin, Germany. Association for Computational Linguistics. find
  85. Rico Sennrich, Alexandra Birch, Anna Currey, Ulrich Germann, Barry Haddow, Kenneth Heafield, Antonio Valerio Miceli Barone, and Philip Williams. 2017. The University of Edinburgh’s Neural MT Systems for WMT17. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Assoc… find
  86. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. find
  87. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Edinburgh neural machine translation systems for wmt 16. In Proceedings of the First Conference on Machine Translation, pages 371– 376, Berlin, Germany. Association for Computational Linguistics. find our card
  88. Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Minimum risk training for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1683–1692, Berlin, Germany. find our card
  89. Michel Simard, Cyril Goutte, and Pierre Isabelle. 2007. Statistical phrase-based post-editing. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 508–515, Rochester, New York. A… find
  90. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006a. A study of translation edit rate with targeted human annotation. In Proceedings of association for machine translation in the Americas, pages 223–231. find our card
  91. Matthew Snover, Bonnie J. Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006b. A study of translation edit rate with targeted human annotation. In 5th Conference of the Association for Machine Translation in the Americas (AMTA), Boston, Massachusetts. find our card
  92. Artem Sokolov, Julia Kreutzer, Kellen Sunderland, Pavel Danchenko, Witold Szymaniak, Hagen Fürstenau, and Stefan Riezler. 2017. A Shared Task on Bandit Learning for Machine Translation. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computationa… find
  93. Lucia Specia, Frédéric Blain, Kim Harris, Aljoscha Burchardt, Viviven Macketanz, Inguna Skadiņa, Marco Turchi, and Matteo Negri. 2017a. Translation quality and productivity: A study on rich morphology languages. In Proceedings of the 16th Machine Translation Summit, Nagoya, Japan. find
  94. Lucia Specia, Kim Harris, Aljoscha Burchardt, Marco Turchi, Matteo Negri, and Inguna Skadina. 2017b. Translation Quality and Productivity: A Study on Rich Morphology Languages. In Proceedings of the 16th Machine Translation Summit, Nagoya, Japan. find
  95. Lucia Specia, Gustavo Paetzold, and Carolina Scarton. 2015. Multi-level Translation Quality Prediction with QuEst++. In 53rd Annual Meeting of the Association for Computational Linguistics and Seventh International Joint Conference on Natural Language Processing of the Asian Federation of Natural La… find
  96. Lucia Specia, Kashif Shah, Jose G.C. de Souza, and Trevor Cohn. 2013. Quest - a translation quality estimation framework. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 79–84, Sofia, Bulgaria. find
  97. A. Stolcke. 2002. Srilm - an extensible language modeling toolkit. In ICSLP, pages 901–904, Denver, CO. find
  98. Roman Sudarikov, David Mareček, Tom Kocmi, Dušan Variš, and Ondřej Bojar. 2017. CUNI Submission in WMT17: Chimera Goes Neural. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  99. Roman Sudarikov, Martin Popel, Ondřej Bojar, Aljoscha Burchardt, and Ondřej Klejch. 2016. Using MT-ComparEval. In Translation Evaluation: From Fragmented Tools and Data Sets to an Integrated Ecosystem, pages 76–82. find
  100. Yiming Tan, Zhiming Chen, Liu Huang, Lilin Zhang, Maoxi Li, and Mingwen Wang. 2017a. Neural Post-Editing Based on Quality Estimation. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  101. Long Trieu, Trung-Tin Pham, and Le-Minh Nguyen. 2017. The JAIST Machine Translation Systems for WMT 17. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  102. Dušan Variš and Ondřej Bojar. 2017. CUNI System for WMT17 Automatic Post-Editing Task. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  103. Yuguang Wang, Shanbo Cheng, Liyang Jiang, Jiajun Yang, Wei Chen, Muze Li, Lin Shi, Yanfeng Wang, and Hongtao Yang. 2017. Sogou Neural Machine Translation Systems for WMT17. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  104. Krzysztof Wolk and Krzysztof Marasek. 2017. PJIIT’s systems for WMT 2017 Conference. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  105. Jia Xu, Yi Zong Kuang, Shondell Baijoo, Jacob Hyun Lee, Uman Shahzad, Mir Ahmed, Meredith Lancaster, and Chris Carlan. 2017. Hunter MT: A Course for Young Researchers in WMT17. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguist… find
  106. Alexander Yeh. 2000. More Accurate Tests for the Statistical Significance of Result Differences. In Coling-2000: the 18th Conference on Computational Linguistics, pages 947–953, Saarbrücken, Germany. find
  107. Jinchao Zhang, Peerachet Porkaew, Jiawei Hu, Qiuye Zhao, and Qun Liu. 2017. CASICT-DCU Neural Machine Translation Systems for WMT17. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find
  108. Yuan Zhang and David Weiss. 2016. Stack-propagation: Improved representation learning for syntax. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1557–1566, Berlin, Germany. Association for Computational Linguistics. find
  109. Zhixing Tan, Boli Wang, Jinming Hu, Yidong Chen, and xiaodong shi. 2017b. XMU Neural Machine Translation Systems for WMT 17. In Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark. Association for Computational Linguistics. find