Elamur Research⌕
☰
home / papers / 2022 / Findings of the IWSLT 2022 Evaluation Campaign

Findings of the IWSLT 2022 Evaluation Campaign

Summary

This paper is the official report on the shared tasks run before IWSLT 2022, the main yearly venue for spoken language translation research. The campaign had eight tasks: simultaneous, offline, speech-to-speech, low-resource, multilingual, dialect, formality-controlled and isometric speech translation. Together they drew 27 academic and industry teams. For each task, the organizers describe the goals, the released training and test data, the evaluation metrics, the submitted systems and the results.

Several tasks were new or substantially redesigned. English-to-German speech-to-speech translation appeared for the first time. The simultaneous track added Mandarin as a target and fed text systems with streaming ASR output instead of gold transcripts. Experimental human evaluation was also introduced, including an MQM-based assessment for English–Japanese.

The isometric task tested length-matched translation in a dubbing scenario. It showed that ordinary MT quality scores miss whether a translation allows a natural speaking rate. The report thus serves as a reference point for comparing approaches across many SLT settings, including low-resource and dialectal speech.

Key points

  • 27 teams took part. The simultaneous task drew 7 teams, its highest number so far, covering English→German, Japanese and Mandarin.
  • Simultaneous systems were submitted as Docker images and scored with SimulEval. They were ranked by BLEU within low, medium and high latency regimes defined by AL thresholds on MuST-C tst-COMMON (1000/2000/4000 for English–German).
  • Low-resource and dialect tasks covered Tamasheq→French and Tunisian Arabic→English. The Tunisian task had three data conditions, including the use of Modern Standard Arabic data.
  • Formality control covered English→German, Spanish, Hindi, Japanese, Italian and Russian. Italian and Russian were zero-shot, with no formality-annotated training data.
  • In the isometric English→French/German/Spanish dubbing evaluation, STRONG BASELINE ranked best. Smoothness and LC metrics explained why rankings differed from MT-only human judgments.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cited by

97 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 94 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.iwslt-1.10

@inproceedings{anastasopoulos2022findings,
  title = {{Findings of the IWSLT 2022 Evaluation Campaign}},
  author = {Anastasopoulos, Antonios and Barrault, Loïc and Bentivogli, Luisa and Boito, Marcely Zanon and Bojar, Ondřej and Cattoni, Roldano and Currey, Anna and Dinu, Georgiana and Duh, Kevin and Elbayad, Maha and Emmanuel, Clara and Estève, Yannick and Federico, Marcello and Federmann, Christian and Gahbiche, Souhir and Gong, Hongyu and Grundkiewicz, Roman and Haddow, Barry and Hsu, Benjamin and Javorský, Dávid and Kloudová, Vĕra and Lakew, Surafel and Ma, Xutai and Mathur, Prashant and McNamee, Paul and Murray, Kenton and Nǎdejde, Maria and Nakamura, Satoshi and Negri, Matteo and Niehues, Jan and Niu, Xing and Ortega, John and Pino, Juan and Salesky, Elizabeth and Shi, Jiatong and Sperber, Matthias and Stüker, Sebastian and Sudoh, Katsuhito and Turchi, Marco and Virkar, Yogesh and Waibel, Alexander and Wang, Changhan and Watanabe, Shinji},
  booktitle = {Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022)},
  year = {2022},
  doi = {10.18653/v1/2022.iwslt-1.10},
  url = {https://doi.org/10.18653/v1/2022.iwslt-1.10}
}

References

138 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondřej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta R. Costa-jussa, Cristina España-Bonet, Angela Fan, Christian Federmann, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christop… find
  2. Luisa Bentivogli, Mauro Cettolo, Marco Gaido, Alina Karakanta, Alberto Martinelli, and Marco Turchi Matteo Negri. 2021. Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, Ba… find
  3. Antonios Anastasopoulos, Ondřej Bojar, Jacob Bremerman, Roldano Cattoni, Maha Elbayad, Marcello Federico, Xutai Ma, Satoshi Nakamura, Matteo Negri, Jan Niehues, Juan Pino, Elizabeth Salesky, Sebastian Stüker, Katsuhito Sudoh, Marco Turchi, Alexander Waibel, Changhan Wang, and Matthew Wiesner. 2021… find
  4. FINDINGS OF THE IWSLT 2021 EVALUATION CAMPAIGN. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), pages 1–29, Bangkok, Thailand (online). Association for Computational Linguistics. find
  5. Aakash Bhatnagar, Nidhir Bhavsar, Muskaan Singh, and Petr Motlicek. 2022. Hierarchical Multi-task learning framework for Isometric-Speech Language Translation. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  6. Marcely Zanon Boito, Fethi Bougares, Florentin Barbier, Souhir Gahbiche, Loı̈c Barrault, Mickael Rouvier, and Yannick Estéve. 2022a. Speech resources in the tamasheq language. Language Resources and Evaluation Conference (LREC). find
  7. Ebrahim Ansari, Amittai Axelrod, Nguyen Bach, Ondrej Bojar, Roldano Cattoni, Fahim Dalvi, Nadir Durrani, Marcello Federico, Christian Federmann, Jiatao Gu, Fei Huang, Kevin Knight, Xutai Ma, Ajay Nagesh, Matteo Negri, Jan Niehues, Juan Pino, Elizabeth Salesky, Xing Shi, Sebastian Stüker, Marco Turc… find
  8. Marcely Zanon Boito, John Ortega, Hugo Riguidel, Antoine Laurent, Loı̈c Barrault, Fethi Bougares, Firas Chaabani, Ha Nguyen, Florentin Barbier, Souhir Gahbiche, and Yannick Estève. 2022b. ON-TRAC Consortium Systems for the IWSLT 2022 Dialect and Low-resource Speech Translation Tasks. In Proceedings… find
  9. Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In LREC. find our card
  10. Roldano Cattoni, Mattia Antonino Di Gangi, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2021. Must-c: A multilingual corpus for end-to-end speech translation. Computer Speech & Language, 66:101155. find
  11. Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, et al. 2021. XLS-R: Self-supervised cross-lingual speech representation learning at scale. arXiv preprint arXiv:2111.09296. find our card
  12. Mauro Cettolo, Marcello Federico, Luisa Bentivogli, Jan Niehues, Sebastian Stüker, K. Sudoh, K. Yoshino, and Christian Federmann. 2017. Overview of the IWSLT 2017 Evaluation Campaign. In Proceedings of the 14th International Workshop on Spoken Language Translation (IWSLT 2017), pages 2–14, Tokyo, J… find our card
  13. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020a. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural Information Processing Systems, 33:12449–12460. find our card
  14. Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012. WIT 3 : Web Inventory of Transcribed and Translated Talks. In Proceedings of the Annual Conference of the European Association for Machine Translation (EAMT), Trento, Italy. find
  15. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020b. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems, volume 33, pages 12449–12460. Curran Associates, Inc. find our card
  16. Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, Roldano Cattoni, and Marcello Federico. 2015. find
  17. The IWSLT 2015 Evaluation Campaign. In Proceedings of the 12th International Workshop on Spoken Language Translation (IWSLT 2015), Da Nang, Vietnam. find
  18. BBC. 2019. BBC Subtitle Guidelines. BBC © 2018 Version 1.1.8. find
  19. Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. 2013. Report on the 10th IWSLT Evaluation Campaign. In Proceedings of the Tenth International Workshop on Spoken Language Translation (IWSLT 2013), Heidelberg, Germany. find
  20. Benjamin Beilharz and Xin Sun. 2019. LibriVoxDeEn - A Corpus for German-to-English Speech Translation and Speech Recognition. find
  21. Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. 2014. Report on the 11th IWSLT Evaluation Campaign, IWSLT 2014. In Proceedings of the Eleventh International Workshop on Spoken Language Translation (IWSLT 2014), Lake Tahoe, USA. find our card
  22. Solène Evain, Ha Nguyen, Hang Le, Marcely Zanon Boito, Salima Mdhaffar, Sina Alisamir, Ziyi Tong, Natalia Tomashenko, Marco Dinarelli, Titouan Parcollet, et al. 2021. Task agnostic and task specific self-supervised learning from speech with LeBenchmark. In Thirty-fifth Conference on Neural Informat… find
  23. Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. 2016. find
  24. The IWSLT 2016 Evaluation Campaign. In Proceedings of the 13th International Workshop on Spoken Language Translation (IWSLT 2016), Seattle, USA. find
  25. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. find
  26. Colin Cherry and George Foster. 2019. Thinking slow about latency evaluation for simultaneous machine translation. arXiv preprint arXiv:1906.00048. find
  27. M. Federico, Y. Virkar, R. Enyedi, and R. Barra-Chicote. 2020a. Evaluating and optimizing prosodic alignment for automatic dubbing. In Proceedings of Interspeech, page 5. find
  28. Joon Son Chung, Arsha Nagrani, and Andrew Zisserman. 2018. VoxCeleb2: Deep Speaker Recognition. In Interspeech, pages 1086–1090. find
  29. Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. 2021. Unsupervised Cross-Lingual Representation Learning for Speech Recognition. In Proc. find
  30. Interspeech 2021, pages 2426–2430. find
  31. Marcello Federico, Luisa Bentivogli, Michael Paul, and Sebastian Stüker. 2011. Overview of the IWSLT 2011 Evaluation Campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 11–27, San Francisco, USA. find
  32. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association f… find
  33. Marcello Federico, Mauro Cettolo, Luisa Bentivogli, Michael Paul, and Sebastian Stüker. 2012. Overview of the IWSLT 2012 Evaluation Campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 11–27, Hong Kong, HK. find
  34. Marcello Federico, Robert Enyedi, Roberto Barra-Chicote, Ritwik Giri, Umut Isik, Arvindh Krishnaswamy, and Hassan Sawaf. 2020b. From Speech-to-Speech Translation to Automatic Dubbing. In Proc. of IWSLT, pages 257–264, Online. ACL. find
  35. Siddharth Dalmia, Brian Yan, Vikas Raunak, Florian Metze, and Shinji Watanabe. 2021. Searchable hidden intermediates for end-to-end models of decomposable sequence tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langu… find
  36. Christian Federmann. 2018. Appraise evaluation framework for machine translation. In Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pages 86–88, Santa Fe, New Mexico. Association for Computational Linguistics. find
  37. Mattia A. Di Gangi, Roldano Cattoni, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2019. MuST-C: a Multilingual Speech Translation Corpus. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (… find our card
  38. Weston Feely, Eva Hasler, and Adrià de Gispert. 2019. Controlling Japanese honorifics in English-to-Japanese neural machine translation. In Proceedings of the 6th Workshop on Asian Translation, pages 45–53, Hong Kong, China. Association for Computational Linguistics. find
  39. Matthias Eck and Chiori Hori. 2005. Overview of the IWSLT 2005 evaluation campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 1–22, Pittsburgh, PA. find
  40. Cameron Shaw Fordyce. 2007. Overview of the IWSLT 2007 evaluation campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 1–12, Trento, Italy. find
  41. Johanes Effendi, Yogesh Virkar, Roberto Barra-Chicote, and Marcello Federico. 2022. Duration modeling of neural tts for automatic dubbing. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8037–8041. find
  42. Markus Freitag, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. 2021a. Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9:1460–1474. find our card
  43. Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie, and Ondřej Bojar. 2021b. Results of the WMT21 metrics shared task: Evaluating metrics with expert-based human evaluations on TED and news domain. In Proceedings of the Sixth Conference on Machine Trans… find our card
  44. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770– 778. find
  45. François Hernandez, Vincent Nguyen, Sahar Ghannay, Natalia A. Tomashenko, and Yannick Estève. 2018. TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation. CoRR, abs/1805.04699. — [arXiv:1805.04699](https://arxiv.org/abs/1805.04699) find
  46. Ryo Fukuda, Yuka Ko, Yasumasa Kano, Kosuke Doi, Hirotaka Tokuyama, Sakriani Sakti, Katsuhito Sudoh, and Satoshi Nakamura. 2022. NAIST Simultaneous Speech-to-Text Translation System for IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  47. Oleksii Hrinchuk, Vahid Noroozi, Abhinav Khattar, Anton Peganov, Sandeep Subramanian, Somshubra Majumdar, and Oleksii Kuchaiev. 2022. NVIDIA NeMo Offline Speech Translation Systems for IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  48. Marco Gaido, Sara Papi, Dennis Fucci, Giuseppe Fiameni, Matteo Negri, and Marco Turchi. 2022. Efficient yet Competitive Speech Translation: FBK@IWSLT2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  49. Javier Iranzo-Sánchez, Jorge Civera Saiz, and Alfons Juan. 2021. Stream-level latency evaluation for simultaneous machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 664–670, Punta Cana, Dominican Republic. Association for Computational Linguistics. find
  50. Karthik Gopalakrishnan, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, and Dilek Hakkani-Tür. 2019. Topical-Chat: Towards knowledge-grounded open-domain conversations. In Proc. find
  51. Interspeech 2019, pages 1891–1895. find
  52. Javier Iranzo-Sánchez, Javier Jorge Cano, Alejandro Pérez-González de Martos, Adrián Giménez Pastor, Gonçal Garcés Dı́az-Munı́o, Pau Baquero-Arnal, Joan Albert Silvestre-Cerdá, Jorge Civera Saiz, Albert Sanchis, and Alfons Juan. 2022. MLLP-VRAIN UPV systems for the IWSLT 2022 Simultaneous Sp… find
  53. Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. 2013. Continuous measurement scales in human evaluation of machine translation. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 33–41, Sofia, Bulgaria. Association for Computational… find
  54. Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang. 2020. Conformer: Convolution-augmented transformer for speech recognition. In Proceedings of Interspeech 2020, 21st Annual Conference of the International… find our card
  55. Javier Iranzo-Sánchez, Joan Albert Silvestre-Cerdà, Javier Jorge, Nahuel Roselló, Adrià Giménez, Albert Sanchis, Jorge Civera, and Alfons Juan. 2020. Europarl-st: A multilingual corpus for speech translation of parliamentary debates. In Proc. of 45th Intl. Conf. on Acoustics, Speech, and Signal… find
  56. Bao Guo, Mengge Liu, Wen Zhang, Hexuan Chen, Chang Mu, Xiang Li, Jianwei Cui, Bin Wang, and Yuhang Guo. 2022a. The Xiaomi Text-to-Text Simultaneous Speech Translation System for IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  57. Won Jang, Dan Lim, Jaesam Yoon, Bongwan Kim, and Juntae Kim. 2021. UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation. In Interspeech, pages 2207–2211. find
  58. Dávid Javorský, Dominik Macháček, and Ondřej Bojar. 2022. Comprehension of subtitles from re-translating simultaneous speech translation. find
  59. Jiaxin Guo, Yinglu Li, Minghan Wang, Xiaosong Qiao, Yuxia Wang, Hengchao Shang, Chang Su, Yimeng Chen, Min Zhang, Shimin Tao, Hao Yang, and Ying Qin. 2022b. The HW-TSC’s Speech to Speech Translation System for IWSLT 2022 Evaluation. In Proceedings of the 19th International Conference on Spoken Langu… find
  60. Japan Translation Federation JTF. 2018. JTF Translation Quality Evaluation Guidelines, 1st Edition (in Japanese). find
  61. Jaehyeon Kim, Jungil Kong, and Juhee Son. 2021. Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In ICML. find
  62. Andrew Hayes and Klaus Krippendorff. 2007. Answering the call for a standard reliability measure Ali Can Kocabiyikoglu, Laurent Besacier, and Olivier Kraif. 2018. Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation. In Proceedings of LREC 201… find
  63. Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Alexandre Muzio, Saksham Singhal, Hany Hassan Awadalla, Xia Song, and Furu Wei. 2021. DeltaLM: Encoder-decoder pre-training for language generation and translation by augmenting pretrained multilingual encoders. arXiv. find
  64. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 conference on empirical methods in natural language processing, pages 388–395. find
  65. Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, and Juan Pino. 2020a. SIMULEVAL: An evaluation toolkit for simultaneous translation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 144–150, Online. Association for Com… find our card
  66. Surafel Lakew, Marcello Federico, Yue Wang, Cuong Hoang, Yogesh Virkar, Roberto Barra-Chicote, and Robert Enyedi. 2021a. Machine translation verbosity control for automatic dubbing. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). find
  67. Xutai Ma, Juan Pino, and Philipp Koehn. 2020b. SimulMT to SimulST: Adapting simultaneous text translation to end-to-end simultaneous speech translation. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint… find
  68. Surafel M Lakew, Yogesh Virkar, Prashant Mathur, and Marcello Federico. 2021b. Isometric mt: Neural machine translation for automatic dubbing. arXiv preprint arXiv:2112.08682. — [arXiv:2112.08682](https://arxiv.org/abs/2112.08682) find
  69. Surafel Melaku Lakew, Mattia Di Gangi, and Marcello Federico. 2019. Controlling the output length of neural machine translation. In Proc. IWSLT. find
  70. Dominik Macháček, Jonáš Kratochvı́l, Tereza Vojtěchová, and Ondřej Bojar. 2019. A speech test set of practice business presentations with additional relevant texts. In Statistical Language and Speech Processing, pages 151–161, Cham, Switzerland. Springer Nature Switzerland AG. find
  71. Yinglu Li, Minghan Wang, Jiaxin Guo, Xiaosong Qiao, Yuxia Wang, Daimeng Wei, Chang Su, Yimeng Chen, Min Zhang, Shimin Tao, Hao Yang, and Ying Qin. 2022a. The HW-TSC’s Offline Speech Translation System for IWSLT 2022 Evaluation. In Proceedings of the 19th International Conference on Spoken Language T… find
  72. Evgeny Matusov, Patrick Wilken, and Yota Georgakopoulou. 2019. Customizing neural machine translation for subtitling. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 82–93, Florence, Italy. Association for Computational Linguistics. find
  73. Zongyao Li, JiaXin Guo, Daimeng Wei, Hengchao Shang, Minghan Wang, Ting Zhu, Zhanglin Wu, Zhengzhe Yu, Xiaoyu Chen, Lizhi Lei, Hao Yang, and Ying Qin. 2022b. HW-TSC’s Participation in the IWSLT 2022 Isometric Spoken Language Translation. In Proceedings of the 19th International Conference on Spoken… find
  74. J. Niehues, R. Cattoni, S. Stüker, M. Negri, M. Turchi, T. Ha, E. Salesky, R. Sanabria, L. Barrault, L. Specia, and M. Federico. 2019. find
  75. The IWSLT 2019 Evaluation Campaign. In Proceedings of the 16th International Workshop on Spoken Language Translation (IWSLT 2019), Hong Kong, China. find
  76. Pierre Lison, Jörg Tiedemann, and Milen Kouylekov. 2018. OpenSubtitles2018: Statistical rescoring of sentence alignments in large, noisy parallel corpora. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Res… find our card
  77. Jan Niehues. 2020. Machine translation with unsupervised length-constraints. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (AMTA 2020), pages 21–35. find
  78. Jan Niehues, Roldano Cattoni, Sebastian Stüker, Mauro Cettolo, Marco Turchi, and Marcello Federico. 2018. find
  79. The IWSLT 2018 Evaluation Campaign. In Proceedings of the 15th International Workshop on Spoken Language Translation (IWSLT 2018), pages 2–6, Bruges, Belgium. find
  80. Dan Liu, Mengge Du, Xiaoxi Li, Ya Li, and Enhong Chen. 2021. Cross attention augmented transducer networks for simultaneous translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 39–55, Online and Punta Cana, Dominican Republic. Association for… find
  81. Xing Niu, Sudha Rao, and Marine Carpuat. 2018. Multi-task neural models for translating between styles within and across languages. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1008–1021, Santa Fe, New Mexico, USA. Association for Computational Linguistics. find
  82. Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726–742. find
  83. Maria Nădejde, Anna Currey, Benjamin Hsu, Xing Niu, Marcello Federico, and Georgiana Dinu. 2022. CoCoA-MT: A dataset and benchmark for Contrastive Controlled MT with application to formality. In Findings of the Association for Computational Linguistics: NAACL 2022, Seattle, USA. Association for Com… find
  84. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. find our card
  85. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Belgium, Brussels. Association for Computational Linguistics. find our card
  86. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. find our card
  87. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67. find
  88. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: An asr corpus based on public domain audio books. 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5206–5210. find
  89. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. Comet: A neural framework for mt evaluation. arXiv preprint arXiv:2009.09025. find our card
  90. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics. Association for Computational Linguistics. find our card
  91. Elijah Rippeth, Sweta Agrawal, and Marine Carpuat. 2022. Controlling Translation Formality Using Pre-trained Multilingual Language Models. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  92. Kyubyong Park and Thomas Mulc. 2019. Css10: A collection of single speaker speech datasets for 10 languages. Interspeech. find
  93. Anthony Rousseau, Paul Deléglise, and Yannick Esteve. 2014. Enhancing the ted-lium corpus with selected data for language modeling and more ted talks. In LREC. find
  94. Michael Paul. 2006. Overview of the IWSLT 2006 Evaluation Campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 1–15, Kyoto, Japan. find
  95. M. Rouvier, G. Dupuy, P. Gay, E. Khoury, T. Merlin, and S. Meignier. 2013. An Open-source State-ofthe-art Toolbox for Broadcast News Diarization. In Proceedings of the Interspeech. find
  96. Michael Paul. 2008. Overview of the IWSLT 2008 Evaluation Campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 1–17, Waikiki, Hawaii. find
  97. Ashutosh Saboo and Timo Baumann. 2019. Integration of Dubbing Constraints into Machine Translation. In Proc. of WMT, pages 94–101, Florence, Italy. ACL. find
  98. Michael Paul. 2009. Overview of the IWSLT 2009 Evaluation Campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 1–18, Tokyo, Japan. find
  99. Elizabeth Salesky, Julian Mäder, and Severin Klinger. 2021. Assessing evaluation metrics for speech-to-speech translation. In ASRU. find
  100. Michael Paul, Marcello Federico, and Sebastian Stüker. 2010. Overview of the IWSLT 2010 Evaluation Campaign. In Proceedings of the International Workshop on Spoken Language Translation, pages 3–27, Paris, France. find
  101. Ramon Sanabria, Ozan Caglayan, Shruti Palaskar, Desmond Elliott, Loı̈c Barrault, Lucia Specia, and Florian Metze. 2018. How2: a large-scale dataset for multimodal language understanding. In Proceedings of the Workshop on Visually Grounded Interaction and Language (ViGIL). NeurIPS. find
  102. Ngoc-Quan Pham, Tuan Nam Nguyen, Thai-Binh Nguyen, Danni Liu, Carlos Mullov, Jan Niehues, and Alexander Waibel. 2022. Efficient yet Competitive Speech Translation: FBK@IWSLT2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  103. Andrea Schioppa, David Vilar, Artem Sokolov, and Katja Filippova. 2021a. Controlling machine translation for multiple attributes with additive interventions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6676–6696, Online and Punta Cana, Dominican R… find
  104. Peter Polák, Ngoc-Quan Pham, Tuan Nam Nguyen, Danni Liu, Carlos Mullov, Jan Niehues, Ondřej Bojar, and Alexander Waibel. 2022. System for Simultaneous Speech Translation Task at IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  105. Andrea Schioppa, David Vilar, Artem Sokolov, and Katja Filippova. 2021b. Controlling machine translation for multiple attributes with additive interventions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6676–6696. find
  106. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of NIPS 2017. find
  107. Sebastian Vincent, Loı̈c Barrault, and Carolina Scarton. 2022. Controlling Formality in Low-Resource NMT with Domain Adaptation and Re-Ranking: SLT-CDT-UoS at IWSLT2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  108. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Controlling politeness in neural machine translation via side constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 35–40, San Diego… find our card
  109. Yogesh Virkar, Marcello Federico, Robert Enyedi, and Roberto Barra-Chicote. 2021. Improvements to Prosodic Alignment for Automatic Dubbing. In ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7543–7574. ISSN: 2379-190X. find
  110. Akshaya Vishnu Kudlu Shanbhogue, Ran Xue, Ching-Yun Chang, and Sarah Campbell. 2022. Amazon Alexa AI’s System for IWSLT 2022 Offline Speech Translation Shared Task. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  111. Yogesh Virkar, Marcello Federico, Robert Enyedi, and Barra-Chicote Roberto. 2022. Prosodic alignment for off-screen automatic dubbing. arXiv preprint arXiv:2204.02530. — [arXiv:2204.02530](https://arxiv.org/abs/2204.02530) find
  112. Gabriel Synnaeve, Qiantong Xu, Jacob Kahn, Edouard Grave, Tatiana Likhomanenko, Vineel Pratap, Anuroop Sriram, Vitaliy Liptchinsky, and Ronan Collobert. 2020. End-to-end asr: from supervised to semi-supervised learning with modern architectures. In ICML. find
  113. Aditi Viswanathan, Varden Wang, and Antonina Kononova. 2019. Controlling formality and style of machine translation output using AutoML. In SIM-Big, volume 1070 of Communications in Computer and Information Science, pages 306–313. Springer. find
  114. Sho Takase and Naoaki Okazaki. 2019. Positional Encoding to Control Output Sequence Length. Proc. of NAACL. find
  115. Changhan Wang, Juan Pino, Anne Wu, and Jiatao Gu. 2020a. Covost: A diverse multilingual speech-to-text translation corpus. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 4197–4203. find
  116. Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. Multilingual translation with extensible multilingual pretraining and finetuning. find
  117. Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. 2021. VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation. In Proceedings of the 59th Annual… find our card
  118. Amirhossein Tebbifakhr, Ruchit Agrawal, Matteo Negri, and Marco Turchi. 2018. Multi-source transformer with combined losses for automatic post editing. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 846– 852. find
  119. Jörg Tiedemann, Santhosh Thottingal, et al. 2020. Opus-mt–building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation. European Association for Machine Translation. find
  120. Changhan Wang, Yun Tang, Xutai Ma, Anne Wu, Dmytro Okhonko, and Juan Pino. 2020b. fairseq s2t: Fast speech-to-text modeling with fairseq. arXiv preprint arXiv:2010.05171. find
  121. Minghan Wang, Jiaxin GUO, Yinglu Li, Xiaosong Qiao, Yuxia Wang, Zongyao Li, Chang Su, Yimeng Chen, Min Zhang, Shimin Tao amd Hao Yang, and Ying Qin. 2022. The HW-TSC’s Simultaneous Speech Translation System for IWSLT 2022 Evaluation. In Proceedings of the 19th International Conference on Spoken Lang… find
  122. Ioannis Tsiamas, Gerard I. Gállego, Carlos Escolano, José A. R. Fonollosa, and Marta R. Costa-jussà. 2022a. Pretrained Speech Encoders and Efficient Fine-tuning Methods for Speech Translation: UPC at IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (I… find
  123. Patrick Wilken and Evgeny Matusov. 2022. AppTek’s Submission to the IWSLT 2022 Isometric Spoken Language Translation Task. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  124. Ioannis Tsiamas, Gerard I. Gállego, José A. R. Fonollosa, and Marta R. Costa-jussà. 2022b. Shas: Approaching optimal segmentation for end-to-end speech translation. find
  125. Lijun Wu, Juntao Li, Yue Wang, Qi Meng, Tao Qin, Wei Chen, Min Zhang, Tie-Yan Liu, et al. 2021. Rdrop: regularized dropout for neural networks. Advances in Neural Information Processing Systems, 34. find
  126. Ziqiang Zhang and Junyi Ao. 2022. The YiTrans Neural Speech Translation Systems for IWSLT 2022 Offline Shared Task. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  127. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934. find
  128. Qinpei Zhu, Renshou Wu, Guangfeng Liu, Xinyu Zhu, Xingyu Chen, Yang Zhou, Qingliang Miao, Rui Wang, and Kai Yu. 2022. The AISP-SJTU Simultaneous Translation System for IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  129. Brian Yan, Patrick Fernandes, Siddharth Dalmia, Jiatong Shi, Yifan Peng, Dan Berrebbi, Xinyi Wang, Graham Neubig, and Shinji Watanabe. 2022. find
  130. CMU’s IWSLT 2022 Dialect Speech Translation System. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  131. Jinyi Yang, Amir Hussein, Matthew Wiesner, and Sanjeev Khudanpur. 2022. find
  132. JHU IWSLT 2022 Dialect Speech Translation System Description. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  133. Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. Paws-x: A cross-lingual adversarial dataset for paraphrase identification. arXiv preprint arXiv:1908.11828. find our card
  134. Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom, and Chris Dyer. 2020. Better document-level machine translation with bayes’ rule. Transactions of the Association for Computational Linguistics, 8(0):346–360. find
  135. Daniel Zhang, Jiang Yu, Pragati Verma, Ashwinkumar Ganesan, and Sarah Campbell. 2022a. Improving Machine Translation Formality Control with Weakly-Labelled Data Augmentation and Post Editing Strategies. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  136. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675. find
  137. Weitai Zhang, Zhongyi Ye, Haitao Tang, Xiaoxi Li, Xinyuan Zhou, Jing Yang, Jianwei Cui, Dan Liu, Junhua Liu, and Lirong Dai. 2022b. The USTC-NELSLIP Offline Speech Translation Systems for IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find
  138. Yuhao Zhang, Canan Huang, Chen Xu, Xiaoqian Liu, Bei Li, Anxiang Ma, Tong Xiao, and Jingbo Zhu. 2022c. The NiuTrans’s Submission to the IWSLT22 English-to-Chinese Offline Speech Translation Task. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/anastasopoulos-2022-findings-iwslt-2022-evaluation-campaign/