Elamur Research
☰
home / papers / 2014 / Findings of the 2014 Workshop on Statistical Machine Translation

Findings of the 2014 Workshop on Statistical Machine Translation

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/W14-3302.pdf

Summary

WMT14 is the ninth edition of an annual shared-task campaign that benchmarks machine translation and the methods used to evaluate it. It had four tasks: news translation, quality estimation, a new medical translation task, and a metrics task, which is reported in a separate paper. The news task covered ten directions between English and Czech, French, German, Hindi and Russian. Hindi replaced Spanish to add a distant, low-resource language pair. The organizers rebuilt the test sets so that every sentence was originally written in either English or the paired language, not in some third language. The goal was to reduce translationese artifacts. Participants received parallel and monolingual training data. The organizers then scored 143 submissions from 23 institutions, plus anonymized online and commercial systems, with automatic metrics. Participating researchers also ranked the outputs by hand, and a new ranking method reached reliable results with fewer judgments. In the medical task, the systems were also tested inside a cross-lingual information retrieval setup. The released data and human judgments became a widely reused MT benchmark.

Key points

  • 143 news-translation systems came from 23 institutions. The organizers added 4 commercial and 4 online systems in anonymized form and treated them as unconstrained.
  • Each English–X test set has about 3000 sentences: about 1500 written originally in English and 1500 in the other language. Hindi–English has 2507. Capita and Yandex did the professional translations.
  • Training resources included Europarl, News Commentary, Common Crawl, the UN corpus, the 10^9 French–English corpus, CzEng, the Yandex 1M Russian–English corpus, Wiki Headlines, and the new Hindi–English corpus (Bojar et al., 2014).
  • Quality estimation had four subtasks: 1–3 Likert scores, edit percentage, post-editing time, and word-level error labels. It drew 57 entries from 10 teams on English–Spanish and English–German data in both directions.
  • Medical task: UEDIN led summary translation (DE–EN, EN–CS, EN–FR). In the IR evaluation on CLEF 2013 eHealth, UEDIN's query translations retrieved nearly as well as the reference translations did.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Topics: Machine translation

Cited by

394 papers of our library cite it (of the 140673 whose bibliographies we read); 29 of them have a page here, 365 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-3302

@inproceedings{bojar2014findings,
  title = {{Findings of the 2014 Workshop on Statistical Machine Translation}},
  author = {Bojar, Ondřej and Buck, Christian and Federmann, Christian and Haddow, Barry and Koehn, Philipp and Leveling, Johannes and Monz, Christof and Pecina, Pavel and Post, Matt and Saint-Amand, Herve and Soricut, Radu and Specia, Lucia and Tamchyna, Aleš},
  booktitle = {Proceedings of the Ninth Workshop on Statistical Machine Translation},
  year = {2014},
  doi = {10.3115/v1/w14-3302},
  url = {https://doi.org/10.3115/v1/w14-3302}
}

References

92 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Avramidis, E. (2014). Efforts on machine learning over human-mediated translation edit rate. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  2. Beck, D., Shah, K., and Specia, L. (2014). Shef-lite 2.0: Sparse multi-task gaussian processes for translation quality estimation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  3. Biçici, E. (2013). Referential translation machines for quality estimation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, Sofia, Bulgaria. find
  4. Biçici, E., Liu, Q., and Way, A. (2014). Parallel FDA5 for fast deployment of accurate statistical machine translation systems. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, USA. Association for Computational Linguistics. find
  5. Bicici, E., Liu, Q., and Way, A. (2014). Parallel fda5 for fast deployment of accurate statistical machine translation systems. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  6. Bicici, E. and Way, A. (2014). Referential translation machines for predicting translation quality. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  7. Bojar, O., Buck, C., Callison-Burch, C., Federmann, C., Haddow, B., Koehn, P., Monz, C., Post, M., Soricut, R., and Specia, L. (2013). Findings of the 2013 Workshop on Statistical Machine Translation. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 1–42, Sofia, Bulgar… find our card
  8. Bojar, O., Diatka, V., Rychlý, P., Straňák, P., Tamchyna, A., and Zeman, D. (2014). Hindi-English and Hindi-only Corpus for Machine Translation. In Proceedings of the Ninth International Language Resources and Evaluation Conference, Reykjavik, Iceland. ELRA. find
  9. Bojar, O., Ercegovčević, M., Popel, M., and Zaidan, O. (2011). A grain of salt for the WMT manual evaluation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 1–11, Edinburgh, Scotland. Association for Computational Linguistics. find
  10. Borisov, A. and Galinskaya, I. (2014). Yandex school of data analysis russian-english machine translation system for wmt14. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  11. Bouayad-Agha, N., Scott, D. R., and Power, R. (2000). Integrating content and style in documents: A case study of patient information leaflets. Information Design Journal, 9(2– 3):161–176. find
  12. Buckley, C. and Voorhees, E. M. (2004). Retrieval evaluation with incomplete information. find
  13. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 25– 32, Sheffield, United Kingdom. Buitelaar, P., Sacaleanu, B., Špela Vintar, Steffen, D., Volk, M., Dejean, H., Gaussier, E., Widdows, D., Weiser, O., and Frederking, R… find
  14. Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., and Schroeder, J. (2007). (Meta-) evaluation of machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation (WMT07), Prague, Czech Republic. find
  15. Callison-Burch, C., Fordyce, C., Koehn, P., Monz, C., and Schroeder, J. (2008). Further meta-evaluation of machine translation. In Proceedings of the Third Workshop on Statistical Machine Translation (WMT08), Colmbus, Ohio. find
  16. Callison-Burch, C., Koehn, P., Monz, C., Peterson, K., Przybocki, M., and Zaidan, O. F. (2010). Findings of the 2010 joint workshop on statistical machine translation and metrics for machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation (WMT10), Uppsala, Swede… find
  17. Callison-Burch, C., Koehn, P., Monz, C., Post, M., Soricut, R., and Specia, L. (2012). Findings of the 2012 workshop on statistical machine translation. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 10– 51, Montréal, Canada. Association for Computational Linguisti… find our card
  18. Callison-Burch, C., Koehn, P., Monz, C., and Schroeder, J. (2009). Findings of the 2009 workshop on statistical machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation (WMT09), Athens, Greece. find
  19. Callison-Burch, C., Koehn, P., Monz, C., and Zaidan, O. (2011). Findings of the 2011 workshop on statistical machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 22–64, Edinburgh, Scotland. find
  20. Camargo de Souza, J. G., González-Rubio, J., Buck, C., Turchi, M., and Negri, M. (2014). Fbk-upv-uedin participation in the wmt14 quality estimation shared-task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguist… find
  21. Cap, F., Weller, M., Ramm, A., and Fraser, A. (2014). Cims – the cis and ims joint submission to wmt 2014 translating from english into german. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  22. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurment, 20(1):37–46. find
  23. Cohn, T. and Specia, L. (2013). Modelling annotator bias with multi-task gaussian processes: An application to machine translation quality estimation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, ACL-2013, pages 32–42, Sofia, Bulgaria. find
  24. Collins, M. (2002). Discriminative training methods for hidden markov models: theory and experiments with perceptron algorithms. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10, EMNLP ’02, pages 1–8, Stroudsburg, PA, USA. Association for Comput… find our card
  25. Costa-jussà, M. R., Gupta, P., Rosso, P., and Banchs, R. E. (2014). English-to-hindi system description for wmt 2014: Deep source-context features for moses. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  26. Do, Q. K., Herrmann, T., Niehues, J., Allauzen, A., Yvon, F., and Waibel, A. (2014). The kit-limsi translation system for wmt 2014. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  27. Dungarwal, P., Chatterjee, R., Mishra, A., Kunchukuttan, A., Shah, R., and Bhattacharyya, P. (2014). The iit bombay hindi-english translation system at wmt 2014. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguisti… find
  28. Durrani, N., Haddow, B., Koehn, P., and Heafield, K. (2014a). Edinburgh’s phrase-based machine translation systems for wmt-14. In Proceedings of the ACL 2014 Ninth Workshop of Statistical Machine Translation, Baltimore, USA. find
  29. Durrani, N., Haddow, B., Koehn, P., and Heafield, K. (2014b). Edinburghs phrase-based machine translation systems for wmt-14. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  30. Dušek, O., Hajič, J., Hlaváčová, J., Novák, M., Pecina, P., Rosa, R., Tamchyna, A., Urešová, Z., and Zeman, D. (2014). Machine translation of medical texts in the khresmoi project. In Proceedings of the ACL 2014 Ninth Workshop of Statistical Machine Translation, Baltimore, USA. find
  31. Federmann, C. (2012). Appraise: An Open-Source Toolkit for Manual Evaluation of Machine Translation Output. The Prague Bulletin of Mathematical Linguistics (PBML), 98:25– 35. find
  32. Foster, J. (2007). Treebanks gone bad: Parser evaluation and retraining using a treebank of ungrammatical sentences. International Journal on Document Analysis and Recognition, 10(3-4):129–145. find
  33. Freitag, M., Peitz, S., Wuebker, J., Ney, H., Huck, M., Sennrich, R., Durrani, N., Nadejde, M., Williams, P., Koehn, P., Herrmann, T., Cho, E., and Waibel, A. (2014). Eu-bridge mt: Combined machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryl… find
  34. Gamon, M., Aue, A., and Smets, M. (2005). Sentence-level MT evaluation without reference translations: beyond language modeling. In Proceedings of the Annual Conference of the European Association for Machine Translation, Budapest. find
  35. Geurts, P., Ernst, D., and Wehenkel, L. (2006). Extremely randomized trees. Machine Learning, 63(1):3–42. find
  36. Green, S., Cer, D., and Manning, C. (2014). Phrasal: A toolkit for new directions in statistical machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  37. Hardmeier, C., Stymne, S., Tiedemann, J., Smith, A., and Nivre, J. (2014). Anaphora models and reordering for phrase-based smt. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  38. Herbrich, R., Minka, T., and Graepel, T. (2006). TrueSkill TM : A Bayesian Skill Rating System. In Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, pages 569–576, Vancouver, British Columbia, Canada. MIT Press. find
  39. Herrmann, T., Mediani, M., Cho, E., Ha, T.-L., Niehues, J., Slawik, I., Zhang, Y., and Waibel, A. (2014). The karlsruhe institute of technology translation systems for the wmt 2014. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Com… find
  40. Hokamp, C., Calixto, I., Wagner, J., and Zhang, J. (2014). Target-centric features for translation quality estimation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  41. Hopkins, M. and May, J. (2013). Models of translation competitions. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1416–1424, Sofia, Bulgaria. find
  42. Järvelin, K. and Kekäläinen, J. (2002). Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems, 20(4):422–446. find
  43. Kim, J.-D., Ohta, T., Tateisi, Y., and Tsujii, J. (2003). GENIA corpus – a semantically annotated corpus for bio-textmining. Bioinformatics, 19(suppl 1):i180–i182. find
  44. Knox, C., Law, V., Jewison, T., Liu, P., Ly, S., Frolkis, A., Pon, A., Banco, K., Mak, C., Neveu, V., Djoumbou, Y., Eisner, R., Guo, A. C., and Wishart, D. S. (2011). DrugBank 3.0: a comprehensive resource for Omics research on drugs. Nucleic acids research, 39(suppl 1):D1035–D1041. find
  45. Koehn, P. (2012a). Simulating human judgment in machine translation evaluation campaigns. In International Workshop on Spoken Language Translation (IWSLT). find
  46. Koehn, P. (2012b). Simulating Human Judgment in Machine Translation Evaluation Campaigns. In Proceedings of the Ninth International Workshop on Spoken Language Translation, pages 179–184, Hong Kong, China. find
  47. Koehn, P. and Monz, C. (2006). Manual and automatic evaluation of machine translation between European languages. In Proceedings of NAACL 2006 Workshop on Statistical Machine Translation, New York, New York. find
  48. Koppel, M. and Ordan, N. (2011). Translationese and its dialects. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Techolo-gies, pages 1318–1326, Portland, Oregon. find
  49. Landis, J. R. and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33:159–174. find
  50. Leusch, G., Ueffing, N., and Ney, H. (2006). Cder: Efficient mt evaluation using block movements. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, pages 241–248, Trento, Italy. find
  51. Li, J., Kim, S.-J., Na, H., and Lee, J.-H. (2014a). Postech’s system description for medical text translation task. In Proceedings of the ACL 2014 Ninth Workshop of Statistical Machine Translation, Baltimore, USA. find
  52. Li, L., Wu, X., Vaillo, S. C., Xie, J., Way, A., and Liu, Q. (2014b). The dcu-ictcas mt system at wmt 2014 on german-english translation task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  53. Lopez, A. (2012). Putting Human Assessments of Machine Translation Systems in Order. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pages 1–9, Montréal, Canada. Association for Computational Linguistics. find
  54. Lu, Y., Wang, L., Wong, D. F., Chao, L. S., Wang, Y., and Oliveira, F. (2014). Domain adaptation for medical text translation using web resources. In Proceedings of the ACL 2014 Ninth Workshop of Statistical Machine Translation, Baltimore, USA. find
  55. Luong, N. Q., Besacier, L., and Lecouteux, B. (2014). Lig system for word level qe task at wmt14. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  56. Luong, N. Q., Lecouteux, B., and Besacier, L. (2013). LIG system for WMT13 QE task: Investigating the usefulness of features in word confidence estimation for MT. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 384–389, Sofia, Bulgaria. Association for Computational L… find
  57. Macháček, M. and Bojar, O. (2014). Results of the wmt14 metrics shared task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find our card
  58. Matthews, A., Ammar, W., Bhatia, A., Feely, W., Hanneman, G., Schlinger, E., Swayamdipta, S., Tsvetkov, Y., Lavie, A., and Dyer, C. (2014). The cmu machine translation systems at wmt 2014. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association… find
  59. Neidert, J., Schuster, S., Green, S., Heafield, K., and Manning, C. (2014). Stanford universitys submissions to the wmt 2014 translation task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  60. Okita, T., Vahid, A. H., Way, A., and Liu, Q. (2014). Dcu terminology translation system for medical query subtask at wmt14. In Proceedings of the ACL 2014 Ninth Workshop of Statistical Machine Translation, Baltimore, USA. find
  61. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, PA, USA. Association for Computational Linguistics. find our card
  62. Pécheux, N., Gong, L., Do, Q. K., Marie, B., Ivanishcheva, Y., Allauzen, A., Lavergne, T., Niehues, J., Max, A., and Yvon, Y. (2014). LIMSI @ WMT’14 Medical Translation Task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, USA. find
  63. Pecina, P., Dušek, O., Goeuriot, L., Hajič, J., Hlaváčová, J., Jones, G., Kelly, L., Leveling, J., Mareček, D., Novák, M., Popel, M., Rosa, R., Tamchyna, A., and Urešová, Z. (2014). Adaptation of machine translation for multilingual information retrieval in the medical domain. Artificial In… find
  64. Peitz, S., Wuebker, J., Freitag, M., and Ney, H. (2014). The rwth aachen german-english machine translation system for wmt 2014. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  65. Pouliquen, B. and Mazenc, C. (2011). COPPA, CLIR and TAPTA: three tools to assist in overcoming the patent barrier at WIPO. In Proceedings of the Thirteenth Machine Translation Summit, pages 24–30, Xiamen, China. Asia-Pacific Association for Machine Translation. find
  66. Powers, D. M. W. (2011). Evaluation: from precision, recall and f-measure to roc, informedness, markedness & correlation. Journal of Machine Learning Technologies. find
  67. Quernheim, D. and Cap, F. (2014). Large-scale exact decoding: The ims-ttt submission to wmt14. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  68. Rosse, C. and Mejino Jr., J. L. V. (2008). The foundational model of anatomy ontology. In Burger, A., Davidson, D., and Baldock, R., editors, Anatomy Ontologies for Bioinformatics, volume 6 of Computational Biology, pages 59– 117. Springer London. find
  69. Rubino, R., Toral, A., Sánchez-Cartagena, V. M., Ferrández-Tordera, J., Ortiz Rojas, S., Ramı́rez-Sánchez, G., Sánchez-Martı́nez, F., and Way, A. (2014). Abu-matran at wmt 2014 translation task: Two-step data selection and rbmt-style synthetic rules. In Proceedings of the Ninth Workshop on Stati… find
  70. Sakaguchi, K., Post, M., and Van Durme, B. (2014). Efficient elicitation of annotations for human evaluation of machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland. find
  71. Sánchez-Cartagena, V. M., Pérez-Ortiz, J. A., and Sánchez-Martı́nez, F. (2014). The ua-prompsit hybrid machine translation system for the 2014 workshop on statistical machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Associatio… find
  72. Scarton, C. and Specia, L. (2014). Exploring consensus in machine translation for quality estimation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  73. Schwartz, L., Anderson, T., Gwinnup, J., and Young, K. (2014). Machine translation and monolingual postediting: The afrl wmt-14 system. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  74. Seginer, Y. (2007). Learning Syntactic Structure. PhD thesis, University of Amsterdam. find
  75. Shah, K., Cohn, T., and Specia, L. (2013). An investigation on the effectiveness of features for translation quality estimation. In Proceedings of the Machine Translation Summit XIV, pages 167–174, Nice, France. find
  76. Shah, K. and Specia, L. (2014). Quality estimation for translation selection. In Proceedings of the 17th Annual Conference of the European Association for Machine Translation, Dubrovnik, Croatia. find
  77. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006). A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Biennial Conference of the Association for Machine Translation in the Americas (AMTA-2006), Cambridge, Massachusetts. find our card
  78. Souza, J. G. C. d., Espl-Gomis, M., Turchi, M., and Negri, M. (2013). Exploiting qualitative information from automatic word alignment for cross-lingual nlp tasks. In The 51st Annual Meeting of the Association for Computational Linguistics - Short Papers (ACL Short Papers 2013). find
  79. Specia, L., Shah, K., de Souza, J. G. C., and Cohn, T. (2013). QuEst - A Translation Quality Estimation Framework. In Proceedings of the 51th Conference of the Association for Computational Linguistics (ACL), Demo Session, Sofia, Bulgaria. find
  80. Tamchyna, A., Popel, M., Rosa, R., and Bojar, O. (2014). Cuni in wmt14: Chimera still awaits bellerophon. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  81. Tan, L. and Pal, S. (2014). Manawi: Using multi-word expressions and named entities to improve machine translation. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  82. Thompson, P., Iqbal, S., McNaught, J., and Ananiadou, S. (2009). Construction of an annotated corpus to support biomedical information extraction. BMC bioinformatics, 10(1):349. find
  83. Tiedemann, J. (2009). News from OPUS – a collection of multilingual parallel corpora with tools and interfaces. In Recent Advances in Natural Language Processing, volume 5, pages 237–248, Borovets, Bulgaria. John Benjamins. find
  84. Tillmann, C., Vogel, S., Ney, H., Zubiaga, A., and Sawaf, H. (1997). Accelerated DP based search for statistical translation. In Kokkinakis, G., Fakotakis, N., and Dermatas, E., editors, Proceedings of the Fifth European Conference on Speech Communication and Technology, pages 2667–2670, Rhodes, Gre… find
  85. U.S. National Library of Medicine (2009). UMLS reference manual. Metathesaurus. Bethesda, MD, USA. find
  86. Voorhees, E. M. and Harman, D. K., editors (2005). TREC: Experiment and evaluation in information retrieval, volume 63 of Digital libraries and electronic publishing series. MIT press Cambridge, Cambridge, MA, USA. find
  87. Wang, L., Lu, Y., Wong, D. F., Chao, L. S., Wang, Y., and Oliveira., F. (2014). Combining domain adaptation approaches for medical text translation. In Proceedings of the ACL 2014 Ninth Workshop of Statistical Machine Translation, Baltimore, USA. find
  88. Wäschle, K. and Riezler, S. (2012). Analyzing parallelism and domain similarities in the MAREC patent corpus. In Salampasis, M. and Larsen, B., editors, Multidisciplinary Information Retrieval, volume 7356 of Lecture Notes in Computer Science, pages 12–27. Springer Berlin Heidelberg. find
  89. Williams, P., Sennrich, R., Nadejde, M., Huck, M., Hasler, E., and Koehn, P. (2014). Edinburghs syntax-based systems at wmt 2014. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. find
  90. Wisniewski, G., Pécheux, N., Allauzen, A., and Yvon, F. (2014). Limsi submission for wmt’14 qe task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, Baltimore, Maryland, USA. Association for Computational Linguistics. wu, x., Haque, R., Okita, T., Arora, P., Way, A., and Li… find
  91. Zdeňka Urešová, Ondřej Dušek, J. H. and Pecina, P. (2014). Multilingual test sets for machine translation of search queries for cross-lingual information retrieval in the medical domain. In To appear in Proceedings of the Ninth International Conference on Language Resources and Evaluation, Reyk… find
  92. Zhang, J., Wu, X., Calixto, I., Vahid, A. H., Zhang, X., Way, A., and Liu, Q. (2014). Experiments in medical translation shared task at wmt 2014. In Proceedings of the ACL 2014 Ninth Workshop of Statistical Machine Translation, Baltimore, USA. find