Elamur Research⌕
☰
home / papers / 2026 / IIIT-BGP IWSLT 2026 Systems for Low-resource ST

IIIT-BGP IWSLT 2026 Systems for Low-resource ST

Summary

This system paper describes the authors' entries to the IWSLT 2026 low-resource track for Bhojpuri→Hindi speech translation. Bhojpuri has tens of millions of speakers but little transcribed or translated speech, inconsistent spelling, and heavy mixing with Hindi. The team built two kinds of systems. The end-to-end one takes a Bhojpuri Wav2Vec2 encoder from Vakyansh and feeds it into the decoder of NLLB-200 (distilled 600M), skipping NLLB's own text encoder. A newly trained adapter connects the two. It pools several encoder layers with learned weights, shortens the sequence with convolutions, and then refines it with a small Transformer. LoRA or partial unfreezing adapts the decoder. The cascaded system fine-tunes Whisper for ASR and NLLB-200 for MT, and then uses QE Fusion with COMET-Kiwi to combine beam candidates. On the dev set the end-to-end model clearly wins, but its score falls by about 20 BLEU on the test set. The cascade loses far less. The authors trace much of this gap to misaligned audio and labels in the official data and to an adapter that had no pretraining.

Key points

  • Data: about 22 h of official news speech (20 h train / 2 h dev / 0.5 h test), plus extra resources:
    • roughly 7 h of real and 4.2 h of synthetic Bhojpuri ASR audio;
    • a Kaggle parallel corpus of about 44.7k pairs;
    • back-translated text produced with nllb-200-1.3B.
  • Adapter design:
    • weighted pooling of Wav2Vec2 layers 6, 8, 10 and 12;
    • two stride-2 Conv1d layers that compress the sequence 4:1;
    • a refinement block of 2 Transformer layers with 8 heads.
  • Dev results:
    • The best end-to-end setup, LoRA-only, reached 32.77 BLEU and 53.43 chrF++.
    • In the cascade, QE Fusion raised scores over beam search. BLEU went from 15.96 to 16.09, and COMET from 0.5394 to 0.5569.
  • Test results:
    • CONTRASTIVE1 (end-to-end, more decoder layers unfrozen): 12.3 BLEU.
    • PRIMARY: 12.1 BLEU.
    • Cascade CONTRASTIVE2: 10.1 BLEU, but the best chrF++ at 39.
    • The cascade's dev-to-test drop was only about 6 BLEU.
  • Data quality: manual inspection turned up label–audio mismatches in both the training and dev splits. The authors suggest pretraining the adapter as a fix.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.iwslt-1.31

@inproceedings{singh2026iiit,
  title = {{IIIT-BGP IWSLT 2026 Systems for Low-resource ST}},
  author = {Singh, Kaustuk Pratap and Dipanshu and Singh, Vedant and Rishu, Kumar},
  booktitle = {Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.iwslt-1.31},
  url = {https://doi.org/10.18653/v1/2026.iwslt-1.31}
}

References

38 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ifeoluwa Adelani, Victor Agostinelli, Antonios Anastasopoulos, Luisa Bentivogli, Ondřej Bojar, Sebastien Bratières, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, Lizhong Chen, Marcello Federico, Marco Gaido, Mahendra Gupta, HyoJung Han, Ali मुख्यमंत्री ने कहा िक प्रदेश में गन्ना िकसानों को 3… find
  2. Marko Avila and Josep Crego. 2025. SYSTRAN @ IWSLT 2025 low-resource track. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 324–332, Vienna, Austria (in-person and online). Association for Computational Linguistics. find
  3. Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances in Neural Information Processing Systems. find our card
  4. Harveen Singh Chadha, Anirudh Gupta, Priyanshi Shah, Neeraj Chhimwal, Ankur Dhuriya, Rishabh Gaur, and Vivek Raghavan. 2022. Vakyansh: Asr toolkit for low resource indic languages. Preprint, arXiv:2203.16512. — [arXiv:2203.16512](https://arxiv.org/abs/2203.16512) find
  5. Marta R Costa-Jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, and 1 others. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. find
  6. Rishi Das. Hindi to bhojpuri. https://www.kaggle.com/datasets/rishi2003das/hindi-to-bhojpuri. Kaggle dataset. ↗
  7. Sandareka Fernando, Surangika Ranathunga, Sanath Jayasena, and Gihan Dias. 2016. Comprehensive part-of-speech tag set and SVM based POS tagger for Sinhala. In Proceedings of the 6th Workshop on South and Southeast Asian Natural Language Processing (WSSANLP2016), pages 173–182, Osaka, Japan. find
  8. The COLING 2016 Organizing Committee. find
  9. Markus Freitag, David Grangier, Qijun Tan, and Bowen Liang. 2022. High quality rather than high model probability: Minimum Bayes risk decoding with neural metrics. Transactions of the Association for Computational Linguistics, 10:811–825. find
  10. Gerard I. Gállego, Ioannis Tsiamas, Carlos Escolano, José A. R. Fonollosa, and Marta R. Costa-jussà. 2021. End-to-end speech translation with pre-trained models and adapters: UPC at IWSLT 2021. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), pages 110… find
  11. Dan Hendrycks and Kevin Gimpel. 2016. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415. find
  12. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3. find
  13. Pratik Joshi, Christain Barnes, Sebastin Santy, Simran Khanuja, Sanket Shah, Anirudh Srinivasan, Satwik Bhattamishra, Sunayana Sitaram, Monojit Choudhury, and Kalika Bali. 2019. Unsung challenges of building and deploying language technologies for low resource language communities. In Proceedings of… find
  14. Sakshi Joshi, Eldho Ittan George, Tahir Javed, Kaushal Bhogale, Nikhil Narasimhan, and Mitesh M. Khapra. 2025. Recognizing every voice: Towards inclusive asr for rural bhojpuri women. Preprint, arXiv:2506.09653. — [arXiv:2506.09653](https://arxiv.org/abs/2506.09653) find
  15. Tom Ko, Vijayaditya Peddinti, Daniel Povey, and Sanjeev Khudanpur. 2015. Audio augmentation for speech recognition. In INTERSPEECH, pages 3586–3590. find
  16. Ritesh Kumar, Siddharth Singh, Shyam Ratan, Mohit Raj, Sonal Sinha, Bornini Lahiri, Vivek Seshadri, Kalika Bali, and Atul Kr Ojha. 2022. Annotated speech corpus for low resource indian languages: Awadhi, bhojpuri, braj and magahi. arXiv preprint arXiv:2206.12931. — [arXiv:2206.12931](https://arxiv.org/abs/2206.12931) find
  17. Shankar Kumar and William Byrne. 2004. Minimum Bayes-risk decoding for statistical machine translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, pages 169–176, Boston, Massachusetts, USA… find our card
  18. Jan Niehues, Elizabeth Salesky, Marco Turchi, and Matteo Negri. 2021. Tutorial: End-to-end speech translation. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts, pages 10–13, online. Association for Computational Lingui… find
  19. Yuta Nishikawa and Satoshi Nakamura. 2023. Interconnection: Effective connection between pre-trained encoder and decoder for speech translation. arXiv preprint arXiv:2305.16897. — [arXiv:2305.16897](https://arxiv.org/abs/2305.16897) find
  20. Sara Papi, Javier Garcia Gilabert, Zachary Hopton, Vilém Zouhar, Carlos Escolano, Gerard I Gállego, Jorge Iranzo-Sánchez, Ahrii Kim, Dominik Macháček, Patricia Schmidtova, and 1 others. 2025. Hearing to translate: The effectiveness of speech modality integration into llms. arXiv preprint arXiv:2512.… find
  21. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. find our card
  22. Daniel S Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D Cubuk, and Quoc V Le. 2019. Specaugment: A simple data augmentation method for automatic speech recognition. arXiv preprint arXiv:1904.08779. find our card
  23. Ankita Pasad, Chung-Ming Chien, Shane Settle, and Karen Livescu. 2024. What do self-supervised speech models know about words? Transactions of the Association for Computational Linguistics, 12:372–391. find
  24. Ngoc-Quan Pham, Tuan Nam Nguyen, Thai-Binh Nguyen, Danni Liu, Carlos Mullov, Jan Niehues, and Alexander Waibel. 2022. Effective combination of pretrained models - KIT@IWSLT2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 190–197, Dublin, Ir… find
  25. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. find our card
  26. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. find
  27. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. arXiv preprint. find our card
  28. Rajesh Kumar Mundotiya, Manish Kumar Singh, Rahul Kapur, Swasti Mishra, and Anil Kumar Singh. 2021. Linguistic resources for bhojpuri, magahi, and Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on A… find our card
  29. Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, José G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and André F. T. Martins. 2022. find
  30. CometKiwi: IST-unbabel 2022 submission for the quality estimation shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 634–645, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. find our card
  31. Nicholas Ruiz and Marcello Federico. 2014. Assessing the impact of speech recognition errors on machine translation quality. In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas: MT Researchers Track, pages 261–274, Vancouver, Canada. Association for Machi… find
  32. Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In International conference on machine learning, pages 4596–4604. PMLR. find
  33. Matthias Sperber and Matthias Paulik. 2020. Speech translation and the end-to-end promise: Taking stock of where we are. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7409–7421, Online. Association for Computational Linguistics. find
  34. Ioannis Tsiamas, Gerard I. Gállego, Carlos Escolano, José A. R. Fonollosa, and Marta R. Costa-jussà. 2022. Pretrained speech encoders and efficient fine-tuning methods for speech translation: UPC at IWSLT 2022. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT… find
  35. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. find
  36. Giorgos Vernikos and Andrei Popescu-Belis. 2024. Don’t rank, combine! combining machine translation hypotheses using quality estimation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12087–12105, Bangkok, Thailand. Associati… find
  37. Ron J Weiss, Jan Chorowski, Navdeep Jaitly, Yonghui Wu, and Zhifeng Chen. 2017. Sequence-to-sequence models can directly translate foreign speech. arXiv preprint arXiv:1703.08581. — [arXiv:1703.08581](https://arxiv.org/abs/1703.08581) find
  38. Jinming Zhao, Hao Yang, Ehsan Shareghi, and Gholamreza Haffari. 2022. M-adapter: Modality adaptation for end-to-end speech-to-text translation. arXiv preprint arXiv:2207.00952. — [arXiv:2207.00952](https://arxiv.org/abs/2207.00952) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/singh-2026-iiit-bgp-iwslt-2026-systems/