Elamur Research⌕
☰
home / papers / 2026 / Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech

Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech

Paper · 2026 · Siqi Ouyang, Shuoyang Ding, Oleksii Hrinchuk et al.

Summary

Simultaneous speech translation (SST) systems built on LLMs can handle unbounded, hours-long audio efficiently when framed as a multi-turn dialogue that reuses the KV cache. Training them, however, depends on synthetic read–write trajectories, and these are often flawed. The authors propose Hierarchical Policy Optimization (HPO), an RL post-training stage that corrects behaviours the model learned from such noisy SFT data.

HPO starts from an InfiniSST model trained on word-alignment-based synthetic data. It adapts GRPO with a reward that rewards speed only after quality is secured: if a sentence's quality falls below a threshold, its latency score is set to the worst value. Quality and latency rewards are group-normalized separately and then summed. Hypotheses are matched to reference sentences with SEGALE rather than mwersegmenter.

On ACL 60/60 and RealSI (English→Chinese/German/Japanese), HPO beats the strongest baseline by large margins and even exceeds offline translation quality. This shows that RL can make synthetic-data-trained streaming translators more practical.

Key points

  • Gains at about 1.5 s average latency: over +7 COMET, +1.25 MetricX and +4 BLEURT compared with the best baseline. Latency is measured with StreamLAAL.
  • Base system is InfiniSST, which had the best quality in the IWSLT 2025 low-latency track. It pairs a streaming speech encoder with an LLM and uses a sliding window plus Attention Sink for unbounded input. SFT data are built with SimAlign on segments of up to 67.2 s (60 chunks of 1.12 s).
  • Among the reward functions tested (MQM, VIP-LLM, MetricX, COMET), MetricX performed best and agreed most closely with human VIP-protocol judgments.
  • With mwersegmenter, the model learned to hack the reward by producing gibberish that still scored well. SEGALE, which uses spaCy and allows null alignments, prevents this.
  • The default hyperparameters gave the best results: quality threshold −5, maximum latency 10 s, and latency weight 0.5. The study is limited to one architecture and to English as the only source language.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🏢NvidiaSanta Clara · United States25 works · 5,058 citations🏢Nvidia (United Kingdom)Reading · United Kingdom18 works · 4,771 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.Machine translation · 1,087 works

Cite

Cite as Siqi Ouyang, Shuoyang Ding, Oleksii Hrinchuk, Vitaly Lavrukhin, Brian Yan, Boris Ginsburg and Lei Li. 2026. Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.80

@inproceedings{ouyang2026hierarchical,
  title = {{Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech}},
  author = {Siqi Ouyang, Shuoyang Ding, Oleksii Hrinchuk, Vitaly Lavrukhin, Brian Yan, Boris Ginsburg and Lei Li},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.80},
  url = {https://doi.org/10.18653/v1/2026.acl-long.80}
}

References

50 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Idris Abdulmumin, Victor Agostinelli, Tanel Alumäe, Antonios Anastasopoulos, Luisa Bentivogli, Ondřej Bojar, Claudia Borg, Fethi Bougares, Roldano Cattoni, Mauro Cettolo, Lizhong Chen, William Chen, Raj Dabre, Yannick Estève, Marcello Federico, Mark Fishel, Marco Gaido, Dávid Javorský, Marek Kaszte… find
  2. Ibrahim Said Ahmad, Antonios Anastasopoulos, Ondřej Bojar, Claudia Borg, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, William Chen, Qianqian Dong, Marcello Federico, Barry Haddow, Dávid Javorský, Mateusz Krubiński, Tsz Kin Lam, Xutai Ma, Prashant Mathur, Evgeny Matusov, Chandresh Maurya, John M… find
  3. FIND-INGS OF THE IWSLT 2024 EVALUATION CAM-PAIGN. In Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), pages 1–11, Bangkok, Thailand (in-person and online). Association for Computational Linguistics. find
  4. Ashkan Alinejad, Maryam Siahbani, and Anoop Sarkar. 2018. Prediction improves simultaneous neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3022–3027, Brussels, Belgium. Association for Computational Linguistics. find
  5. Shanbo Cheng, Yu Bao, Qian Cao, Luyang Huang, Liyan Kang, Zhicheng Liu, Yu Lu, Wenhao Zhu, Jingwen Chen, Zhichao Huang, Tao Li, Yifu Li, Huiying Lin, Sitong Liu, Ningxin Peng, Shuaijie She, Lu Xu, Nuo Xu, Sen Yang, and 7 others. 2025a. Seed-x: Building strong multilingual translation llm with 7b par… — [arXiv:2507.13618](https://arxiv.org/abs/2507.13618) find
  6. Shanbo Cheng, Yu Bao, Zhichao Huang, Yu Lu, Ningxin Peng, Lu Xu, Runsheng Yu, Rong Cao, Yujiao Du, Ting Han, Yuxiang Hu, Zeyang Li, Sitong Liu, Shengtao Ma, Shiguang Pan, Jiongchen Xiao, Nuo Xu, Meng Yang, Rong Ye, and 9 others. 2025b. Seed liveinterpret 2.0: End-to-end simultaneous speech-to-speech… — [arXiv:2507.17527](https://arxiv.org/abs/2507.17527) find
  7. Shanbo Cheng, Zhichao Huang, Tom Ko, Hang Li, Ningxin Peng, Lu Xu, and Qini Zhang. 2024a. Towards achieving human parity on end-to-end simultaneous speech translation via llm agent. Preprint, arXiv:2407.21646. find
  8. Shanbo Cheng, Zhichao Huang, Tom Ko, Hang Li, Ningxin Peng, Lu Xu, and Qini Zhang. 2024b. Towards achieving human parity on end-to-end simultaneous speech translation via llm agent. arXiv preprint arXiv:2407.21646. find
  9. David Dale and Marta R. Costa-jussà. 2024. BLASER 2.0: a metric for evaluation and quality estimation of massively multilingual speech and text translation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16075–16085, Miami, Florida, USA. Association for Computational… find
  10. Mattia A. Di Gangi, Roldano Cattoni, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2019. MuST-C: a Multilingual Speech Translation Corpus. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (…
  11. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. Language-agnostic BERT sentence embedding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878–891, Dublin, Ireland. Association for Computation…
  12. Biao Fu, Donglei Yu, Minpeng Liao, Chengxi Li, Yidong Chen, Kai Fan, and Xiaodong Shi. 2025. Efficient and adaptive simultaneous speech translation with fully unidirectional architecture. Preprint, arXiv:2504.11809. — [arXiv:2504.11809](https://arxiv.org/abs/2504.11809) find
  13. Alvin Grissom II, He He, Jordan Boyd-Graber, John Morgan, and Hal Daumé III. 2014. Don’t until the final verb wait: Reinforcement learning for simultaneous machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1342–1352, Doha, Q… find
  14. Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Victor O.K. Li. 2017. Learning to translate in real-time with neural machine translation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 1053–1062, Valencia, Spa…
  15. Nuno M. Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André F. T. Martins. 2024. xcomet: Transparent machine translation evaluation through fine-grained error detection. Transactions of the Association for Computational Linguistics, 12:979–995.
  16. Shoutao Guo, Xiang Li, Mengge Liu, Wei Chen, and Yang Feng. 2025. Streamuni: Achieving streaming speech translation with a unified large speech-language model. Preprint, arXiv:2507.07803. — [arXiv:2507.07803](https://arxiv.org/abs/2507.07803) find
  17. Julia Ive, Andy Mingren Li, Yishu Miao, Ozan Caglayan, Pranava Madhyastha, and Lucia Specia. 2021. Exploiting multimodal reinforcement learning for simultaneous machine translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main V… find
  18. Masoud Jalili Sabet, Philipp Dufter, François Yvon, and Hinrich Schütze. 2020. SimAlign: High quality word alignments without parallel training data using static and contextualized embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1627–1643, Online. Associat… find
  19. Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. 2024. MetricX-24: The Google submission to the WMT 2024 metrics shared task. In Proceedings of the Ninth Conference on Machine Translation, pages 492–504, Miami, Florida, USA. Association for Computational Linguistics.
  20. Tom Kocmi and Christian Federmann. 2023. GEMBA-MQM: Detecting translation quality error spans with GPT-4. In Proceedings of the Eighth Conference on Machine Translation, pages 768–775, Singapore. Association for Computational Linguistics.
  21. Roman Koshkin, Katsuhito Sudoh, and Satoshi Nakamura. 2024. TransLLaMa: LLM-based simultaneous translation system. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 461–476, Miami, Florida, USA. Association for Computational Linguistics. find
  22. Alon Lavie, Greg Hanneman, Sweta Agrawal, Diptesh Kanojia, Chi-Kiu Lo, Vilém Zouhar, Frederic Blain, Chrysoula Zerva, Eleftherios Avramidis, Sourabh Deoghare, Archchana Sindhujan, Jiayi Wang, David Ifeoluwa Adelani, Brian Thompson, Tom Kocmi, Markus Freitag, and Daniel Deutsch. 2025. Findings of the… find
  23. Xinjian Li, Shinnosuke Takamichi, Takaaki Saeki, William Chen, Sayaka Shiota, and Shinji Watanabe. 2023. Yodas: Youtube-oriented dataset for audio and speech. In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1–8. find
  24. Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, and Haifeng Wang. 2019. STACL: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework. In Proceedings of the…
  25. Xutai Ma, Juan Pino, and Philipp Koehn. 2020. SimulMT to SimulST: Adapting simultaneous text translation to end-to-end simultaneous speech translation. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint C…
  26. Mana Makinae, Yusuke Sakai, Hidetaka Kamigaito, and Taro Watanabe. 2024. Simul-MuST-C: Simultaneous multilingual speech translation corpus using large language model. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22185–22205, Miami, Florida, USA. As… find
  27. Evgeny Matusov, Gregor Leusch, Oliver Bender, and Hermann Ney. 2005. Evaluating machine translation output with automatic sentence segmentation. In Proceedings of the Second International Workshop on Spoken Language Translation, Pittsburgh, Pennsylvania, USA.
  28. Vahid Noroozi, Somshubra Majumdar, Ankur Kumar, Jagadeesh Balam, and Boris Ginsburg. 2024. Stateful conformer with cache-based inference for streaming automatic speech recognition. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 12041–12… find
  29. Siqi Ouyang, Xi Xu, and Lei Li. 2025a. find
  30. CMU’s IWSLT 2025 simultaneous speech translation system. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 309–314, Vienna, Austria (in-person and online). Association for Computational Linguistics. find
  31. Siqi Ouyang, Xi Xu, and Lei Li. 2025b. InfiniSST: Simultaneous translation of unbounded speech with large language model. In Findings of the Association for Computational Linguistics: ACL 2025, pages 3032–3046, Vienna, Austria. Association for Computational Linguistics. find
  32. Sara Papi, Marco Gaido, Matteo Negri, and Luisa Bentivogli. 2024. StreamAtt: Direct streaming speech-to-text translation with attention-based audio history selection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3692–3707,… find
  33. Sara Papi, Marco Gaido, Matteo Negri, and Marco Turchi. 2022. Over-generation cannot be rewarded: Length-adaptive average lagging for simultaneous speech translation. In Proceedings of the Third Workshop on Automatic Simultaneous Translation, pages 12–17, Online. Association for Computational Lingui… find
  34. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  35. José Pombal, Dongkeun Yoon, Patrick Fernandes, Ian Wu, Seungone Kim, Ricardo Rei, Graham Neubig, and Andre Martins. 2025. M-prometheus: A suite of open multilingual LLM judges. In Second Conference on Language Modeling. find
  36. Amy Pu, Hyung Won Chung, Ankur Parikh, Sebastian Gehrmann, and Thibault Sellam. 2021. Learning compact metrics for MT. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 751–762, Online and Punta Cana, Dominican Republic. Association for Computational Li…
  37. Ricardo Rei, Nuno M. Guerreiro, José Pombal, Daan van Stigt, Marcos Treviso, Luisa Coheur, José G. C. de Souza, and André Martins. 2023. Scaling up CometKiwi: Unbabel-IST 2023 submission for the quality estimation shared task. In Proceedings of the Eighth Conference on Machine Translation, pages 84…
  38. Yi Ren, Jinglin Liu, Xu Tan, Chen Zhang, Tao Qin, Zhou Zhao, and Tie-Yan Liu. 2020. SimulSpeech: End-to-end simultaneous speech to text translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3787– 3796, Online. Association for Computational Lin… find
  39. Elizabeth Salesky, Kareem Darwish, Mohamed Al-Badrashiny, Mona Diab, and Jan Niehues. 2023. Evaluating multilingual speech translation under realistic conditions with resegmentation and terminology. In Proceedings of the 20th International Conference on Spoken Language Translation (IWSLT 2023), page… find
  40. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Preprint, arXiv:2402.03300.
  41. Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. Roformer: Enhanced transformer with rotary position embedding. Neurocomput., 568(C).
  42. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  43. Brian Thompson and Philipp Koehn. 2019. Vecalign: Improved sentence alignment in linear time and space. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1342– 1348…
  44. Changhan Wang, Anne Wu, Jiatao Gu, and Juan Pino. 2021. Covost 2 and massively multilingual speech translation. In Interspeech 2021, pages 2247–2251. find
  45. Kuang-Da Wang, Shuoyang Ding, Chao-Han Huck Yang, Ping-Chun Hsieh, Wen-Chih Peng, Vitaly Lavrukhin, and Boris Ginsburg. 2025a. Extending automatic machine translation evaluation to book-length documents. Preprint, arXiv:2509.17249. find
  46. Minghan Wang, Thuy-Trang Vu, Yuxia Wang, Ehsan Shareghi, and Gholamreza Haffari. 2025b. Conversational SimulMT: Efficient simultaneous translation with large language models. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 93–105, Vienna, Austri… find
  47. Zecheng Wang. 2022. Simultaneous Machine Translation with Deep Reinforcement Learning. Ph.D. thesis, Master’s thesis, University of California. find
  48. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient streaming language models with attention sinks. In The Twelfth International Conference on Learning Representations. find
  49. Ting Xu, Zhichao Huang, Jiankai Sun, Shanbo Cheng, and Wai Lam. 2025. SeqPO-SiMT: Sequential policy optimization for simultaneous machine translation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16107–16123, Vienna, Austria. Association for Computational Linguistics… find
  50. Hao Zhang, Nianwen Si, Yaqi Chen, Wenlin Zhang, Xukui Yang, Dan Qu, and Xiaolin Jiao. 2023. Tuning large language model for end-to-end speech translation. Preprint, arXiv:2310.02050. — [arXiv:2310.02050](https://arxiv.org/abs/2310.02050) find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ouyang-2026-hierarchical-policy-optimization-simultaneous-translation/