Elamur Research⌕
☰
home / papers / 2026 / BSC's Submission to the Instruction Following Track of IWSLT 2026

BSC's Submission to the Instruction Following Track of IWSLT 2026

Summary

The paper presents the Barcelona Supercomputing Center's entry to the unconstrained Instruction Following track of IWSLT 2026. In this track, a single system processes English audio and follows prompts in English, German, Italian or Chinese. The tasks are ASR, speech translation (S2TT), spoken QA (SQA) and an undisclosed "surprisal" task.

The authors build an end-to-end Speech LLM. A speech encoder (SeamlessM4T v2 or mHuBERT) feeds a linear projection into the translation-specialised SalamandraTA-7b-instruct-WMT25. Training mixes speech and text data across ASR, translation, QA and general instruction following. The model uses Chain-of-Thought decoding: it first writes a transcription inside special "think" tokens and then produces the answer. Because of this design, text-only QA and translation data fit the same pattern, with the text context standing in for the transcript. Varied prompt templates reduce overfitting to specific instructions, and an external LLM post-edits outputs to fix formatting.

On the IWSLT 2025 test set, the system rivals the Phi4-Multimodal baseline on S2TT and beats it on several SQA languages. The SeamlessM4T variant achieved this despite incomplete training. The results suggest that transcribe-then-answer decoding is an effective way to reuse text resources in speech systems.

Key points

  • Architecture:
    • The SeamlessM4T v2 encoder is kept frozen, while mhubert-base-25hz is fully fine-tuned.
    • The backbone is SalamandraTA-7b-instruct-WMT25.
    • Training lasted a single epoch, using 32 H100 accelerators and a total batch size of 256 across all devices.
  • Data:
    • About 11.5k hours (3.95M samples) of English ASR from Common Voice 22.0, VoxPopuli, MLS and YODAS–Granary.
    • S2TT from EuroparlST and CoVoST2.
    • T2TT filtered with COMETKiwi.
    • SQA from LibriSQA, NMSQA (translated with Tower+ and answers rewritten with Gemma4 31B) and SQuAD 2.0, voiced with Kyutai TTS.
    • CMRC2018 for Chinese QA; Dolci-Instruct-SFT and Aya Dataset for instruction following.
  • IWSLT 2025 (MCIF) results, best variant (SeamlessM4T + post-editing):
    • ASR: WER 0.11, against 0.07 for Phi4-Multimodal.
    • S2TT: COMET of 0.78 for de, 0.81 for it and 0.80 for zh.
    • SQA: best BERTScore for de (0.42) and a tie for zh (0.37).
  • Effect of post-editing:
    • On the 2025 set, it raised SQA scores for it (0.36→0.41) and zh (0.34→0.37), mainly by correcting output formats.
    • On the 2026 benchmark, it lifted German surprisal-task format accuracy from 0.712 to 0.997.
    • However, it slightly lowered English QA scores.
  • Official IWSLT 2026 results: The PRIMARY (SeamlessM4T-based) submission scored best on SQA across all languages, even though it completed only about 0.8 of a training epoch.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.iwslt-1.19

@inproceedings{pareras2026bsc,
  title = {{BSC's Submission to the Instruction Following Track of IWSLT 2026}},
  author = {Pareras, Oriol and Llado, Joan and Buitrago, Pol and Casals-Salvador, Marc and Costa, Federico and Espana-Bonet, Cristina},
  booktitle = {Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.iwslt-1.19},
  url = {https://doi.org/10.18653/v1/2026.iwslt-1.19}
}

References

32 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, and 1 others. 2025. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras. arXiv preprint arXiv:2503.… find our card
  2. David Ifeoluwa Adelani, Victor Agostinelli, Antonios Anastasopoulos, Luisa Bentivogli, Ondřej Bojar, Sebastien Bratières, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, Lizhong Chen, Marcello Federico, Marco Gaido, Mahendra Gupta, HyoJung Han, Ali Hatami, David Javorský, Yejin Jeon, Marek Kaszteln… find
  3. Michael Hassid, Tal Remez, Tu Anh Nguyen, Itai Gat, Alexis CONNEAU, Felix Kreuk, Jade Copet, Alexandre Defossez, Gabriel Synnaeve, Emmanuel Dupoux, Roy Schwartz, and Yossi Adi. 2023. Textually pre-trained speech language models. In Advances in Neural Information Processing Systems, volume 36, pages… find
  4. Ke Hu, Zhehuai Chen, Chao-Han Huck Yang, Piotr Że-lasko, Oleksii Hrinchuk, Vitaly Lavrukhin, Jagadeesh Balam, and Boris Ginsburg. 2025. Chain-of-thought prompting for speech translation. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1… find
  5. J. Iranzo-Sánchez, J. A. Silvestre-Cerdà, J. Jorge, N. Roselló, A. Giménez, A. Sanchis, J. Civera, and A. Juan. 2020. Europarl-st: A multilingual corpus for speech translation of parliamentary debates. In ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (IC… find
  6. Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In Proceedings of the twelfth language resources and evaluation conference, pages 421… find our card
  7. Beomseok Lee, Marcely Zanon Boito, Laurent Besacier, and Ioan Calapodescu. 2025. NAVER LABS Europe submission to the instruction-following track. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 186–200, Vienna, Austria (in-person and online). As… find
  8. Giuseppe Attanasio, Sonal Sannigrahi, Ben Peters, and André F. T. Martins. 2025. Instituto de telecomunicações at IWSLT 2025: Aligning small-scale speech and language models for speech-to-text learning. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025),… find
  9. Guan-Ting Lin, Yung-Sung Chuang, Ho-Lam Chung, Shu wen Yang, Hsuan-Jui Chen, Shuyan Annie Dong, Shang-Wen Li, Abdelrahman Mohamed, Hung yi Lee, and Lin shan Lee. 2022. DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answering. In Interspeech 2022, pages 5165–5169. find
  10. Loïc Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, and 1 others. 2023. Seamless: Multilingual expressive and streaming speech translation. arXiv preprint arXiv:2312.05187. find our card
  11. Nam Luu and Ondřej Bojar. 2025. CUNI-NL@IWSLT 2025: End-to-end offline speech translation and instruction following with LLMs. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 282–288, Vienna, Austria (in-person and online). Association for Comp… find
  12. Yiming Cui, Ting Liu, Wanxiang Che, Li Xiao, Zhipeng Chen, Wentao Ma, Shijin Wang, and Guoping Hu. 2019. A span-extraction dataset for chinese machine reading comprehension. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conf… find
  13. Team Olmo, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Soldaini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, Pete Walsh, and 49 others. 20… find
  14. Javier Garcia Gilabert, Xixian Liao, Severino Da Dalt, Ella Bohman, Audrey Mash, Francesca De Luca Fornaciari, Irene Baucells, Joan Llop, Miguel Claramunt, Carlos Escolano, and Maite Melero. 2025. From SALAMANDRA to SALAMANDRATA: BSC submission for WMT25 general machine translation shared task. In P… find
  15. Sara Papi, Maike Züfle, Marco Gaido, Beatrice Savoldi, Danni Liu, Ioannis Douros, Luisa Bentivogli, and Jan Niehues. 2026. MCIF: Multimodal crosslingual instruction-following benchmark from scientific talks. In The Fourteenth International Conference on Learning Representations. find
  16. Oriol Pareras, Gerard I Gállego, Federico Costa, Cristina Espana-Bonet, and Javier Hernando. 2025. Revisiting direct speech-to-text translation with speech llms: Better scaling than cot prompting? arXiv preprint arXiv:2510.03093. — [arXiv:2510.03093](https://arxiv.org/abs/2510.03093) find
  17. Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. 2020. MLS: A Large-Scale Multilingual Dataset for Speech Research. In Interspeech 2020, pages 2757–2761. find our card
  18. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789, Melbourne, Australia. Association for Computational Ling… find our card
  19. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. find our card
  20. Nithin Rao Koluguri, Monica Sekoyan, George Zelen-froynd, Sasha Meister, Shuoyang Ding, Sofia Ko-standian, He Huang, Nikolay Karpov, Jagadeesh Balam, Vitaly Lavrukhin, Yifan Peng, Sara Papi, Marco Gaido, Alessio Brutti, and Boris Ginsburg. 2025. Granary: Speech Recognition and Translation Dataset in… find
  21. Ricardo Rei, Nuno M. Guerreiro, José Pombal, João Alves, Pedro Teixeirinha, Amin Farajian, and André F. T. Martins. 2025. Tower+: Bridging generality and translation specialization in multilingual llms. Preprint, arXiv:2506.17080. find
  22. Ricardo Rei, Marcos Treviso, Nuno M Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, José GC De Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, and 1 others. 2022. find
  23. Cometkiwi: Ist-unbabel 2022 submission for the quality estimation shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 634– 645. find our card
  24. Francesco Verdini, Pierfrancesco Melucci, Stefano Perna, Francesco Cariaggi, Marco Gaido, Sara Papi, Szymon Mazurek, Marek Kasztelnik, Luisa Bentivogli, Sebastien Bratières, Paolo Merialdo, and Simone Scardapane. 2025. How to Connect Speech Foundation Models and Large Language Models? What Matters a… find
  25. Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. 2021a. Voxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation. In Proceedings of the 59th Annual… find our card
  26. Changhan Wang, Anne Wu, Jiatao Gu, and Juan Pino. 2021b. CoVoST 2 and Massively Multilingual Speech Translation. In Interspeech 2021, pages 2247– 2251. find
  27. Neil Zeghidour, Eugene Kharitonov, Manu Orsini, Václav Volhejn, Gabriel de Marmiesse, Edouard Grave, Patrick Pérez, Laurent Mazaré, and Alexandre Défossez. 2025. Streaming sequence-to-sequence learning with delayed streams modeling. Technical report, Kyutai. find
  28. Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. Improving massively multilingual neural machine translation and zero-shot translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1628– 1639, Online. Association for Computational… find our card
  29. Zihan Zhao, Yiyang Jiang, Heyang Liu, Yu Wang, and Yanfeng Wang. 2025. Librisqa: A novel dataset and framework for spoken question answering with large language models. IEEE Transactions on Artificial Intelligence, 6(11):2884–2895. find
  30. Shivalika Singh, Freddie Vargus, Daniel D’souza, Börje F Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura O’Mahony, and 1 others. 2024. Aya dataset: An open-access collection for multilingual instruction tuning. In Proceedings of the 62nd Annual Meeti… find our card
  31. Jörg Tiedemann. 2020. The tatoeba translation challenge–realistic data sets for low resource and multilingual mt. In Proceedings of the fifth conference on machine translation, pages 1174–1182. find our card
  32. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/pareras-2026-bsc-s-submission-instruction-following/