Elamur Research⌕
☰
home / papers / 2026 / A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026

A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026

Summary

This CUNI submission to the IWSLT 2026 Simultaneous Speech Translation task turns an offline direct speech-to-text model into a simultaneous one without retraining. The authors pair Canary-1B-v2 with the AlignAtt policy, which uses decoder cross-attention to hold back tokens that attend to the newest audio frames. Silero VAD filters non-speech, as in the SimulStreaming framework. To make this work, they extended NVIDIA's NeMo toolkit to accept a forced decoder prefix and fixed a bug in cross-attention outputs under beam search. They tuned chunk size and attention-frame thresholds by grid search to meet the task's latency limits, targeting Czech→English, English→German and English→Italian.

On dev data, the system usually beats the organizers' Qwen3-based cascade, the existing Canary sliding-window implementation, and a Whisper-based SimulStreaming system. Gains are largest at high latency. The main exception is English→German at low latency, where BLEU and chrF trail the cascade. Because the model has only 1B parameters and covers 25 source and 25 target languages, it is a credible starting point for on-device simultaneous translation and a reference baseline for future work.

Key points

  • Development data and metrics:
    • English→German and English→Italian were developed on MCIF.
    • Czech→English used the IWSLT 2026 dev set, which consists of Czech Chamber of Deputies meetings.
    • Final candidates were chosen by XCOMET-XL.
    • Latency was measured with computationally unaware LongYAAL, with limits of under 2000 ms (low regime) and under 4000 ms (high regime).
  • English→German and English→Italian, high latency:
    • English→German reached 31.73 BLEU against 27.66 for the best organizers' baseline.
    • English→Italian reached 43.56 BLEU against 37.76.
    • The Canary sliding-window system scored 23.65 BLEU on English→German, about 8 points lower.
  • Czech→English versus SimulStreaming with Whisper:
    • High latency: 32.01 vs 24.20 BLEU.
    • Low latency: 27.78 vs 22.11 BLEU.
  • Implementation details:
    • Incoming audio arrives in chunks controlled by MinChunkSize.
    • A 30-second rolling audio buffer is paired with a buffer of already-committed target text, both trimmed together.
    • The code is integrated into github.com/ufal/SimulStreaming and was also ported to Simulstream for computationally aware evaluation.
  • Limitations and future work:
    • Combining a forced prefix with in-domain context in the decoder prompt made the model stop producing output.
    • Evaluating a quantized Canary in simultaneous mode is left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.iwslt-1.22

@inproceedings{ortega2026pocket,
  title = {{A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026}},
  author = {Ortega, Aziz Sharipov and Macháček, Dominik},
  booktitle = {Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.iwslt-1.22},
  url = {https://doi.org/10.18653/v1/2026.iwslt-1.22}
}

References

28 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Idris Abdulmumin, Victor Agostinelli, Tanel Alumäe, Antonios Anastasopoulos, Luisa Bentivogli, Ondřej Bojar, Claudia Borg, Fethi Bougares, Roldano Cattoni, Mauro Cettolo, Lizhong Chen, William Chen, Raj Dabre, Yannick Estève, Marcello Federico, Mark Fishel, Marco Gaido, Dávid Javorský, Marek Kaszte… find
  2. David Ifeoluwa Adelani, Victor Agostinelli, Antonios Anastasopoulos, Luisa Bentivogli, Ondřej Bojar, Sebastien Bratières, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, Lizhong Chen, Marcello Federico, Marco Gaido, Mahendra Gupta, HyoJung Han, Ali Hatami, David Javorský, Yejin Jeon, Marek Kaszteln… find
  3. Antonios Anastasopoulos and 1 others. 2026. Speech translation and metrics in 2026: Findings of the IWSLT campaign. In Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026). Association for Computational Linguistics. find
  4. Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, and Colin Raffel. 2019. Monotonic infinite lookback attention for simultaneous machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pa… find
  5. Nenad Banfic, David Fan, Kunal Vaishnavi, Sam Kemp, Sunghoon Choi, Rui Ren, Sayan Shaw, and Meng Tang. 2026. Pushing the limits of on-device streaming asr: A compact, high-accuracy english model for low-latency inference. find
  6. Seamless Communication, Loïc Barrault, Yu-An Chung, Mariano Cora Meglioli, David Dale, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, Christopher Klaiber, Pengwei Li, Daniel Licht, Jean Maillard, Alice Rakotoarison, Kaushik Ram Sadagopan, Guillaume Wenze… find our card
  7. Marco Gaido, Sara Papi, Mauro Cettolo, Matteo Negri, and Luisa Bentivogli. 2025. Simulstream: Open-source toolkit for evaluation and demonstration of streaming speech-to-text translation systems. Preprint, arXiv:2512.17648. find
  8. Nuno M. Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André F. T. Martins. 2024. xCOMET: Transparent machine translation evaluation through fine-grained error detection. Transactions of the Association for Computational Linguistics, 12:979–995. find our card
  9. Eric Harper, Somshubra Majumdar, Oleksii Kuchaiev, Li Jason, Yang Zhang, Evelina Bakhturina, Vahid Noroozi, Sandeep Subramanian, Koluguri Nithin, Huang Jocelyn, Fei Jia, Jagadeesh Balam, Xuesong Yang, Micha Livne, Yi Dong, Sean Naren, and Boris Ginsburg. NeMo: a toolkit for Conversational AI and Lar… find
  10. Roman Koshkin, Je Haesung, Lianbo Liu, Hao Shi, Meng Zhao, Yusuke Fujita, and Yui Sudo. 2026. Streaming translation and transcription through speech-to-text causal alignment. find
  11. Tom Labiausse, Laurent Mazaré, Edouard Grave, Patrick Pérez, Alexandre Défossez, and Neil Zeghidour. 2025. High-fidelity simultaneous speech-to-speech translation. Preprint, arXiv:2502.03382. — [arXiv:2502.03382](https://arxiv.org/abs/2502.03382) find
  12. Dominik Macháček and Peter Polák. 2025. Simultaneous translation with offline speech and LLM models in CUNI submission to IWSLT 2025. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 389–398, Vienna, Austria (in-person and online). Association f… find
  13. Jan Niehues and 1 others. 2018. find
  14. Low-Latency Neural Speech Translation. In Proc. Interspeech 2018. find
  15. Sara Papi, Marco Gaido, Matteo Negri, and Luisa Bentivogli. 2024. StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand… find
  16. Sara Papi, Marco Gaido, Matteo Negri, and Marco Turchi. 2022. Does simultaneous speech translation need simultaneous models? In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 141–153, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  17. Sara Papi, Peter Polák, Dominik Macháček, and Ondřej Bojar. 2025a. How “real” is your real-time simultaneous speech-to-text translation system? Transactions of the Association for Computational Linguistics, 13:281–313. find
  18. Sara Papi, Marco Turchi, and Matteo Negri. 2023. Alignatt: Using attention-based audio-translation alignments as a guide for simultaneous speech translation. Preprint, arXiv:2305.11408. — [arXiv:2305.11408](https://arxiv.org/abs/2305.11408) find
  19. Sara Papi, Maike Züfle, Marco Gaido, Beatrice Savoldi, Danni Liu, Ioannis Douros, Luisa Bentivogli, and Jan Niehues. 2025b. Mcif: Multimodal crosslingual instruction-following benchmark from scientific talks. Preprint, arXiv:2507.19634. — [arXiv:2507.19634](https://arxiv.org/abs/2507.19634) find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… find our card
  21. Peter Polák, Sara Papi, Luisa Bentivogli, and Ondřej Bojar. 2025. Better late than never: Meta-evaluation of latency metrics for simultaneous speech-to-text translation. Preprint, arXiv:2509.17349. find
  22. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. find our card
  23. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, pages 28492–28518. PMLR. find our card
  24. Monica Sekoyan, Nithin Rao Koluguri, Nune Tade-vosyan, Piotr Zelasko, Travis Bartley, Nikolay Karpov, Jagadeesh Balam, and Boris Ginsburg. 2025. Canary-1b-v2 & parakeet-tdt-0.6b-v3: Efficient and high-performance models for multilingual asr and ast. Preprint, arXiv:2509.14128. — [arXiv:2509.14128](https://arxiv.org/abs/2509.14128) find
  25. Sukanta Sen, Ondřej Bojar, and Barry Haddow. 2022. Simultaneous translation for unsegmented input: A sliding window approach. Preprint, arXiv:2210.09754. — [arXiv:2210.09754](https://arxiv.org/abs/2210.09754) find
  26. Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, Jin Xu, Jingren Zhou, and Junyang Lin. 2026. Qwen3-asr technical report. Preprint, arXiv:2601.21337. find
  27. Silero Team. 2024. Silero vad: pre-trained enterprise-grade voice activity detector (vad), number detector and language classifier. https://github.com/ snakers4/silero-vad. ↗
  28. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ortega-2026-pocket-offline-model-simultaneous-speech/