Elamur Research
☰
home / papers / 2015 / Findings of the 2015 Workshop on Statistical Machine Translation

Findings of the 2015 Workshop on Statistical Machine Translation

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/W15-3001.pdf

Summary

This report presents the results of the WMT15 evaluation campaign, held with EMNLP 2015 in Lisbon. It measures the current state of machine translation and of tools that judge or repair MT output. The campaign had five tasks: news translation, metrics, tuning, quality estimation (QE) and a first, pilot automatic post-editing (APE) task. The metrics and tuning tasks are described in separate papers. The translation task covered English paired with Czech, French, German, Russian and Finnish, in both directions. Finnish took Hindi's place as the harder, less-resourced language. Organizers supplied training corpora and newly translated test sets. Systems were ranked through large-scale human judgments, aggregated with TrueSkill, and through automatic metrics. The APE pilot gave a clear lesson: no participant beat a baseline that simply leaves the MT output unchanged. Current statistical APE systems tend to over-correct. The campaign matters because all test data, system outputs and human judgments are released publicly, giving shared benchmarks for later MT and evaluation research.

Key points

  • The translation task received 68 systems from 24 institutions across ten directions. Seven anonymized external systems were added (1 commercial, 6 online) and counted as unconstrained.
  • Test sets had about 3000 sentences per pair: roughly 1500 originally in English and 1500 originally in the other language. The French–English set instead used reader comments from the Guardian and Le Monde (38 threads each).
  • Human ranking was streamlined by removing near-identical candidate outputs before annotation. TrueSkill produced the final system ordering.
  • QE drew 10 teams and 34 entries across three subtasks. These were sentence-level post-editing effort (English→Spanish), word-level good/bad labels (English→Spanish) and document-level Meteor prediction (English↔German).
  • The APE pilot attracted 4 teams and 7 entries on English→Spanish news, using data from Unbabel. Systems beat plain monolingual translation but not the "do nothing" baseline.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cited by

184 papers of our library cite it (of the 140673 whose bibliographies we read); 22 of them have a page here, 162 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-3001

@inproceedings{bojar2015findings,
  title = {{Findings of the 2015 Workshop on Statistical Machine Translation}},
  author = {Bojar, Ondřej and Chatterjee, Rajen and Federmann, Christian and Haddow, Barry and Huck, Matthias and Hokamp, Chris and Koehn, Philipp and Logacheva, Varvara and Monz, Christof and Negri, Matteo and Post, Matt and Scarton, Carolina and Specia, Lucia and Turchi, Marco},
  booktitle = {Proceedings of the Tenth Workshop on Statistical Machine Translation},
  year = {2015},
  doi = {10.18653/v1/w15-3001},
  url = {https://doi.org/10.18653/v1/w15-3001}
}

References

87 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Avramidis, E., Popović, M., and Burchardt, A. (2015). DFKI’s experimental hybrid MT system for WMT 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 66–73, Lisboa, Portugal. Association for Computational Linguistics. find
  2. Banerjee, S. and Lavie, A. (2005). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL 2005 Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization. find our card
  3. Béchara, H., Ma, Y., and van Genabith, J. (2011). Statistical Post-Editing for a Statistical MT System. In Proceedings of the 13th Machine Translation Summit, pages 308–315, Xiamen, China. find
  4. Beck, D., Shah, K., Cohn, T., and Specia, L. (2013a). SHEF-Lite: When less is more for translation quality estimation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 335–340, Sofia, Bulgaria. Association for Computational Linguistics. find
  5. Beck, D., Specia, L., and Cohn, T. (2013b). Reducing annotation effort for quality estimation via active learning. In 51st Annual Meeting of the Association for Computational Linguistics: Short Papers, ACL, pages 543–548, Sofia, Bulgaria. find
  6. Biçici, E. (2013). Referential translation machines for quality estimation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, Sofia, Bulgaria. find
  7. Biçici, E. and Way, A. (2014). Referential Translation Machines for Predicting Translation Quality. In Ninth Workshop on Statistical Machine Translation, pages 313–321, Baltimore, Maryland, USA. find
  8. Bicici, E., Liu, Q., and Way, A. (2015). Referential Translation Machines for Predicting Translation Quality and Related Statistics. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 304–308, Lisboa, Portugal. Association for Computational Linguistics. find
  9. Blain, F., Schwenk, H., and Senellart, J. (2012). Incremental adaptation using translation information and post-editing analysis. In International Workshop on Spoken Language Translation (IWSLT), pages 234–241, Hong-Kong (China). find
  10. Bojar, O., Buck, C., Callison-Burch, C., Federmann, C., Haddow, B., Koehn, P., Monz, C., Post, M., Soricut, R., and Specia, L. (2013). find
  11. Findings of the 2013 Workshop on Statistical Machine Translation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 1–42, Sofia, Bulgaria. Association for Computational Linguistics. find our card
  12. Bojar, O., Buck, C., Federmann, C., Haddow, B., Koehn, P., Leveling, J., Monz, C., Pecina, P., Post, M., Saint-Amand, H., Soricut, R., Specia, L., and Tamchyna, A. (2014). Findings of the 2014 workshop on statistical machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Tr… find our card
  13. Bojar, O. and Tamchyna, A. (2015). CUNI in WMT15: Chimera Strikes Again. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 79–83, Lisboa, Portugal. Association for Computational Linguistics. find
  14. Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., and Schroeder, J. (2007). (Meta-) evaluation of machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation (WMT07), Prague, Czech Republic. find
  15. Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., and Schroeder, J. (2008). Further meta-evaluation of machine translation. In Proceedings of the Third Workshop on Statistical Machine Translation (WMT08), Colmbus, Ohio. find
  16. Callison-Burch, C., Koehn, P., Monz, C., Peterson, K., Przybocki, M., and Zaidan, O. F. (2010). Findings of the 2010 joint workshop on statistical machine translation and metrics for machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation (WMT10), Uppsala, Swede… find
  17. Callison-Burch, C., Koehn, P., Monz, C., Post, M., Soricut, R., and Specia, L. (2012). Findings of the 2012 workshop on statistical machine translation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 10– 51, Montréal, Canada. Association for Computational Linguisti… find our card
  18. Callison-Burch, C., Koehn, P., Monz, C., and Schroeder, J. (2009). Findings of the 2009 workshop on statistical machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation (WMT09), Athens, Greece. find
  19. Callison-Burch, C., Koehn, P., Monz, C., and Zaidan, O. (2011). Findings of the 2011 workshop on statistical machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 22–64, Edinburgh, Scotland. find
  20. Camargo de Souza, J. G., González-Rubio, J., Buck, C., Turchi, M., and Negri, M. (2014). Fbk-upv-uedin participation in the wmt14 quality estimation shared-task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 322–328, Baltimore, Maryland, USA. Association for Computa… find
  21. Cap, F., Weller, M., Ramm, A., and Fraser, A. (2015). CimS - The CIS and IMS Joint Submission to WMT 2015 addressing morphological and syntactic differences in English to German SMT. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 84–91, Lisboa, Portugal. Association f… find
  22. Cettolo, M., Bertoldi, N., and Federico, M. (2014). The Repetition Rate of Text as a Predictor of the Effectiveness of Machine Translation Adaptation. In Proceedings of the 11th Biennial Conference of the Association for Machine Translation in the Americas (AMTA 2014), pages 166– 179, Vancouver, BC,… find
  23. Chatterjee, R., Turchi, M., and Negri, M. (2015a). The FBK Participation in the WMT15 Automatic Post-editing Shared Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 210–215, Lisboa, Portugal. Association for Computational Linguistics. find
  24. Chatterjee, R., Weller, M., Negri, M., and Turchi, M. (2015b). Exploring the Planet of the APEs: a Comparative Study of State-of-the-art Methods for MT Automatic Post-Editing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics), Beijing, China. find
  25. Cho, E., Ha, T.-L., Niehues, J., Herrmann, T., Mediani, M., Zhang, Y., and Waibel, A. (2015). The Karlsruhe Institute of Technology Translation Systems for the WMT 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 92–97, Lisboa, Portugal. Association for Computatio… find
  26. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurment, 20(1):37–46. find
  27. Dušek, O., Gomes, L., Novák, M., Popel, M., and Rosa, R. (2015). New Language Pairs in TectoMT. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 98– 104, Lisboa, Portugal. Association for Computational Linguistics. find
  28. Dyer, C., Lopez, A., Ganitkevitch, J., Weese, J., Ture, F., Blunsom, P., Setiawan, H., Eidelman, V., and Resnik, P. (2010). cdec: A decoder, alignment, and learning framework for finite-state and context-free translation models. In Proceedings of the Association for Computational Linguistics (ACL). find
  29. Esplà-Gomis, M., Sánchez-Martı́nez, F., and Forcada, M. (2015a). UAlacant word-level machine translation quality estimation system at WMT 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 309–315, Lisboa, Portugal. Association for Computational Linguistics. find
  30. Esplà-Gomis, M., Sánchez-Martnez, F., and Forcada, M. L. (2015b). Using on-line available sources of bilingual information for word-level machine translation quality estimation. In 18th Annual Conference of the European Associtation for Machine Translation, page 1926, Antalya, Turkey. find
  31. Federmann, C. (2012). Appraise: An Open-Source Toolkit for Manual Evaluation of Machine Translation Output. The Prague Bulletin of Mathematical Linguistics (PBML), 98:25– 35. find
  32. Gao, Q. and Vogel, S. (2008). Parallel Implementations of Word Alignment Tool. In Proceedings of the ACL 2008 Software Engineering, Testing, and Quality Assurance Workshop, pages 49–57, Columbus, Ohio. find
  33. Giménez, J. and Màrquez, L. (2010). Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation. The Prague Bulletin of Mathematical Linguistics, 94:77–86. find
  34. Graham, Y. (2015). Improving Evaluation of Machine Translation Quality Estimation. In 53rd Annual Meeting of the Association for Computational Linguistics and Seventh International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pages 1804–1813… find
  35. Grönroos, S.-A., Virpioja, S., and Kurimo, M. (2015). Tuning Phrase-Based Segmented Translation for a Morphologically Complex Target Language. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 105–111, Lisboa, Portugal. Association for Computational Linguistics. find
  36. Gwinnup, J., Anderson, T., Erdmann, G., Young, K., May, C., Kazi, M., Salesky, E., and Thompson, B. (2015). The AFRL-MITLL WMT15 System: Theres More than One Way to Decode It! In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 112– 119, Lisboa, Portugal. Association for C… find
  37. Ha, T.-L., Do, Q.-K., Cho, E., Niehues, J., Allauzen, A., Yvon, F., and Waibel, A. (2015). The KIT-LIMSI Translation System for WMT 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 120– 125, Lisboa, Portugal. Association for Computational Linguistics. find
  38. Haddow, B., Huck, M., Birch, A., Bogoychev, N., and Koehn, P. (2015). The Edinburgh/JHU Phrase-based Machine Translation Systems for WMT 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 126–133, Lisboa, Portugal. Association for Computational Linguistics. find
  39. Heafield, K. (2011). KenLM: faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187–197, Edinburgh, United Kingdom. Association for Computational Linguistics. find our card
  40. Jean, S., Firat, O., Cho, K., Memisevic, R., and Bengio, Y. (2015). Montreal Neural Machine Translation Systems for WMT15. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 134–140, Lisboa, Portugal. Association for Computational Linguistics. find
  41. Junczys-Dowmunt, M. and Szal, A. (2011). SyM-Giza++: Symmetrized Word Alignment Models for Statistical Machine Translation. In SIIS, volume 7053 of Lecture Notes in Computer Science, pages 379–390. Springer. find
  42. Koehn, P. (2004). Statistical Significance Tests for Machine Translation Evaluation. In Lin, D. and Wu, D., editors, Proceedings of EMNLP 2004, pages 388–395, Barcelona, Spain. find
  43. Koehn, P. (2005). Europarl: A Parallel Corpus for Statistical Machine Translation. In MT Summit X. find
  44. Koehn, P. (2010). Statistical Machine Translation. Cambridge University Press, New York, NY, USA, 1st edition. find
  45. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., and Herbst, E. (2007). Moses: Open source toolkit for statistical machine translation. In ACL 2007 Demonstrations, Prague, Czech Republic. find
  46. Koehn, P. and Monz, C. (2006). Manual and automatic evaluation of machine translation between European languages. In Proceedings of NAACL 2006 Workshop on Statistical Machine Translation, New York, New York. find
  47. Kolachina, P. and Ranta, A. (2015). GF Wide-coverage English-Finnish MT system for WMT 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 141– 144, Lisboa, Portugal. Association for Computational Linguistics. find
  48. Kreutzer, J., Schamoni, S., and Riezler, S. (2015). QUality Estimation from ScraTCH (QUETCH): Deep Learning for Word-level Translation Quality Estimation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 316–322, Lisboa, Portugal. Association for Computational Linguisti… find
  49. Landis, J. R. and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33:159–174. find
  50. Langlois, D. (2015). LORIA System for the WMT15 Quality Estimation Shared Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 323–329, Lisboa, Portugal. Association for Computational Linguistics. find
  51. Lavie, A. and Agarwal, A. (2007). Meteor: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, StatMT ’07, pages 228–231. find our card
  52. Liang, P., Taskar, B., and Klein, D. (2006). Alignment by Agreement. In HLTNAACL, New York. find
  53. Logacheva, V., Hokamp, C., and Specia, L. (2015). Data enhancement and selection strategies for the word-level Quality Estimation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 330–335, Lisboa, Portugal. Association for Computational Linguistics. find
  54. Luong, N. Q., Besacier, L., and Lecouteux, B. (2014). LIG System for Word Level QE task at WMT14. In Ninth Workshop on Statistical Machine Translation, pages 335–341, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  55. Luong, N. Q., Lecouteux, B., and Besacier, L. (2013). LIG system for WMT13 QE task: Investigating the usefulness of features in word confidence estimation for MT. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 384–389, Sofia, Bulgaria. Association for Computational L… find
  56. Marie, B., Allauzen, A., Burlot, F., Do, Q.-K., Ive, J., knyazeva, e., Labeau, M., Lavergne, T., Löser, K., Pécheux, N., and Yvon, F. (2015). LIMSI@WMT’15 : Translation Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 145–151, Lisboa, Portugal. Association for C… find
  57. Och, F. J. (2003). Minimum Error Rate Training in Statistical Machine Translation. In ACL03, pages 160–167, Sapporo, Japan. find
  58. Padr, L. and Stanilovsky, E. (2012). Freeling 3.0: Towards wider multilinguality. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. find
  59. Pal, S., Naskar, S., and Bandyopadhyay, S. (2013). A Hybrid Word Alignment Model for Phrase-Based Statistical Machine Translation. In Proceedings of the Second Workshop on Hybrid Approaches to Translation, pages 94–101, Sofia, Bulgaria. find
  60. Pal, S., Naskar, S., and van Genabith, J. (2015a). UdS-Sant: English–German Hybrid Machine Translation System. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 152–157, Lisboa, Portugal. Association for Computational Linguistics. find
  61. Pal, S., Vela, M., Naskar, S. K., and van Genabith, J. (2015b). USAAR-SAPE: An English– Spanish Statistical Automatic Post-Editing System. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 216– 221, Lisboa, Portugal. Association for Computational Linguistics. find
  62. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318, Morristown, NJ, USA. find our card
  63. Parton, K., Habash, N., McKeown, K., Iglesias, G., and de Gispert, A. (2012). Can Automatic Post-Editing Make MT More Meaningful? In Proceedings of the 16th Conference of the European Association for Machine Translation (EAMT), pages 111–118, Trento, Italy. find
  64. Peter, J.-T., Toutounchi, F., Wuebker, J., and Ney, H. (2015). The RWTH Aachen German-English Machine Translation System for WMT 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 158–163, Lisboa, Portugal. Association for Computational Linguistics. find
  65. Potet, M., Esperana-Rodier, E., Besacier, L., and Blanchon, H. (2012). Collection of a large database of french-english smt output corrections. In LREC, pages 4043–4048. European Language Resources Association (ELRA). find
  66. Quernheim, D. (2015). Exact Decoding with Multi Bottom-Up Tree Transducers. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 164–171, Lisboa, Portugal. Association for Computational Linguistics. find
  67. Raybaud, S., Langlois, D., and Smali, K. (2011). this sentence is wrong. detecting errors in machine-translated sentences. Machine Translation, 25(1):1–34. find
  68. Rubino, R., Pirinen, T., Esplà-Gomis, M., Ljubešić, N., Ortiz Rojas, S., Papavassiliou, V., Prokopidis, P., and Toral, A. (2015). Abu-MaTran at WMT 2015 Translation Task: Morphological Segmentation and Web Crawling. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 18… find
  69. Scarton, C., Tan, L., and Specia, L. (2015a). USHEF and USAAR-USHEF participation in the WMT15 QE shared task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 336–341, Lisboa, Portugal. Association for Computational Linguistics. find
  70. Scarton, C., Zampieri, M., Vela, M., van Genabith, J., and Specia, L. (2015b). Searching for Context: a Study on Document-Level Labels for Translation Quality Estimation. In The 18th Annual Conference of the European Association for Machine Translation, pages 121–128, Antalya, Turkey. find
  71. Schwartz, L., Bryce, B., Geigle, C., Massung, S., Liu, Y., Peng, H., Raja, V., Roy, S., and Upadhyay, S. (2015). The University of Illinois submission to the WMT 2015 Shared Translation Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 192– 198, Lisboa, Portugal. A… find
  72. Shah, K., Logacheva, V., Paetzold, G., Blain, F., Beck, D., Bougares, F., and Specia, L. (2015). SHEF-NN: Translation Quality Estimation with Neural Networks. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 342–347, Lisboa, Portugal. Association for Computational Lingu… find
  73. Shang, L., Cai, D., and Ji, D. (2015). Strategy-Based Technology for Estimating MT Quality. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 348–352, Lisboa, Portugal. Association for Computational Linguistics. find
  74. Simard, M., Goutte, C., and Isabelle, P. (2007). Statistical Phrase-Based Post-Editing. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT), pages 508–515, Rochester, New York. find
  75. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006a). A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of Association for Machine Translation in the Americas, pages 223– 231, Cambridge, Massachusetts, USA. find our card
  76. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006b). A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Biennial Conference of the Association for Machine Translation in the Americas (AMTA-2006), Cambridge, Massachusetts. find our card
  77. Specia, L., Paetzold, G., and Scarton, C. (2015). Multi-level Translation Quality Prediction with QuEst++. In 53rd Annual Meeting of the Association for Computational Linguistics and Seventh International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Pro… find
  78. Specia, L., Shah, K., de Souza, J. G., and Cohn, T. (2013). QuEst - A translation quality estimation framework. In 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL-2013, pages 79– 84, Sofia, Bulgaria. find
  79. Stanojević, M., Kamran, A., and Bojar, O. (2015a). Results of the WMT15 Tuning Shared Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 274– 281, Lisboa, Portugal. Association for Computational Linguistics. find
  80. Stanojević, M., Kamran, A., Koehn, P., and Bojar, O. (2015b). Results of the WMT15 Metrics Shared Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 256–273, Lisboa, Portugal. Association for Computational Linguistics. find
  81. Steele, D., Sim Smith, K., and Specia, L. (2015). Sheffield Systems for the Finnish-English WMT Translation Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 172–176, Lisboa, Portugal. Association for Computational Linguistics. find
  82. Tezcan, A., Hoste, V., Desmet, B., and Macken, L. (2015). UGENT-LT3 SCATE System for Machine Translation Quality Estimation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 353–360, Lisboa, Portugal. Association for Computational Linguistics. find
  83. Tiedemann, J., Ginter, F., and Kanerva, J. (2015). Morphological Segmentation and OPUS for Finnish-English Machine Translation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 177–183, Lisboa, Portugal. Association for Computational Linguistics. find
  84. Williams, P., Sennrich, R., Nadejde, M., Huck, M., and Koehn, P. (2015). Edinburgh’s Syntax-Based Systems at WMT 2015. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 199–209, Lisboa, Portugal. Association for Computational Linguistics. find
  85. Wisniewski, G., Pécheux, N., and Yvon, F. (2015). Why Predicting Post-Edition is so Hard? Failure Analysis of LIMSI Submission to the APE Shared Task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 222–227, Lisboa, Portugal. Association for Computational Linguistics. find
  86. Wisniewski, G., Singh, A. K., Segal, N., and Yvon, F. (2013). Design and Analysis of a Large Corpus of Post-edited Translations: Quality Estimation, Failure Analysis and the Variability of Post-edition. Machine Translation Summit, 14:117–124. find
  87. UEDIN - SYNTAX MONTREAL UEDIN - JHU CU - TECTO ONLINE - A ONLINE - B find