Elamur Research⌕
☰
home / papers / 2026 / PUMA: Projected Universal Multilingual ASR for Low-Resource Settings. Application to Diverse African Languages

PUMA: Projected Universal Multilingual ASR for Low-Resource Settings. Application to Diverse African Languages

Paper · 2026 · Ilyes Oukid, Bilal Faye, Hanane Azzag et al.

Summary

Multilingual speech recognition systems often perform poorly on under-resourced, linguistically varied languages. Adding per-language adapters can help, but each new language then adds parameters. The authors propose PUMA, a compact multilingual ASR model that keeps all weights shared and signals the target language through a single learnable language token. This token is prepended to frozen wav2vec 2.0 acoustic features and passed through a small trainable Universal Language Projection (ULP) module. The token is then removed, and the result is added back to the features before a frozen wav2vec 2.0 encoder and a CTC-trained output head.

The model was tested on ten African languages and Arabic under three regimes: per-language, progressive (sequential with replay), and joint multilingual training. PUMA beat strong multilingual baselines and stayed close to much larger systems in a cross-corpus zero-shot evaluation, with roughly 30M trainable parameters. This matters for frugal ASR, where a new language should cost only one extra token.

Key points

  • Architecture. The ULP module consists of 4 shared Transformer blocks with RMSNorm. The feature extractor, feature projection and encoder are frozen components from wav2vec 2.0, and training uses CTC loss.
  • Parameter budget. Trainable parameters stay fixed at about 30M no matter how many languages are supported. By contrast, each MMS language adapter adds more than 2M parameters.
  • Zero-shot comparison. PUMA outperformed Whisper (small, medium, large), MMS and Seamless-M4T v2 Large. Omnilingual ASR CTC-1B scored slightly better, and Omni-LLM-7B achieved the best results overall.
  • Ablation on the language token. Removing the token raised word error rate in every language in Table 5 for which the extracted figures are legible. Examples: Fon rose from 0.207 to 0.296, Arabic from 0.384 to 0.505, and Yoruba from 0.245 to 0.297.
  • Stated limitation and future work. The authors note that zero-shot ability is limited by the lightweight design. They plan to explore multi-token language representations and Mixture-of-Experts approaches.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre National de la Recherche ScientifiqueParis · France168 works · 62,369 citations🔬Laboratoire d'Informatique de Paris-NordVilletaneuse · France9 works · 510 citations🎓Université Sorbonne Paris NordVilletaneuse · France8 works · 468 citations🎓Université de Versailles Saint-Quentin-en-YvelinesVersailles · France2 works

Also on the byline: Données et algorithmes pour une ville intelligente et durable.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 3Research subjects and languages the work is about.Multilingual & cross-lingual NLP · 828 worksSpeech & spoken language · 480 worksLow-resource languages · 267 works

Cite

Cite as Ilyes Oukid, Bilal Faye, Hanane Azzag, Mustapha Lebbah and Said Yacine Boulahia. 2026. PUMA: Projected Universal Multilingual ASR for Low-Resource Settings. Application to Diverse African Languages. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.17

@inproceedings{oukid2026puma,
  title = {{PUMA: Projected Universal Multilingual ASR for Low-Resource Settings. Application to Diverse African Languages}},
  author = {Ilyes Oukid, Bilal Faye, Hanane Azzag, Mustapha Lebbah and Said Yacine Boulahia},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.17},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.17}
}

References

44 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hidi Agili. 2025. Yoruba tts dataset. https://huggingface.co/datasets/Hidi-agili/ yoruba_tts_dataset. ↗
  2. Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, and 1 others. 2022. Xls-r: Self-supervised cross-lingual speech representation learning at scale. In Proc.
  3. Interspeech 2022, pages 2278–2282. processing tasks. IEEE/ACM Transactions on Audio, Speech, and Language Processing. find
  4. Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, and Michael Auli. 2021. Unsupervised speech recognition. Advances in Neural Information Processing Systems, 34:27826–27839. find
  5. Anjuli Kannan, Arindrima Datta, Tara N Sainath, Eugene Weinstein, Bhuvana Ramabhadran, Yonghui Wu, Ankur Bapna, Zhifeng Chen, and Seungji Lee. 2019. Large-scale multilingual speech recognition with a streaming end-to-end model. In Proc. find
  6. Interspeech 2019, pages 2130–2134. find
  7. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449–12460.
  8. Junwen Bai, Bo Li, Qiujia Li, Tara N Sainath, and Trevor Strohman. 2024. Efficient adapter finetuning for tail languages in streaming multilingual asr. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10841–10845. IEEE. find
  9. Yaqi Chen, Hao Zhang, Xukui Yang, Wenlin Zhang, and Dan Qu. 2024. Meta-adapter for self-supervised speech models: A solution to low-resource speech recognition challenges. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-… find
  10. Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. 2021. Unsupervised cross-lingual representation learning for speech recognition. In Proc. find
  11. Interspeech 2021, pages 2426–2430. find
  12. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. 2023. Fleurs: Few-shot learning evaluation of universal representations of speech. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 798–805. IEEE.
  13. Bo Li, Ruoming Pang, Tara N Sainath, Anmol Gulati, Yu Zhang, James Qin, Parisa Haghani, W Ronny Huang, Min Ma, and Junwen Bai. 2021. Scaling end-to-end models for large-scale multilingual asr. In 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1011–1018. IEEE. find
  14. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74–81. Association for Computational Linguistics.
  15. Alexander H Liu, Wei-Ning Hsu, Michael Auli, and Alexei Baevski. 2023. Towards end-to-end unsupervised speech recognition. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 221–228. IEEE. find
  16. William Mattingly. 2024. Xhosa merged audio dataset. https://huggingface.co/datasets/ wjbmattingly/xhosa_merged_audio. ↗
  17. Yahya Mohamed. 2024. Arabic audio rev3 9643 2021 dataset. https://huggingface.co/datasets/ Yahya-Mohamed/Arabic_Audio_Rev3_9643_ 2021_Dataset. ↗
  18. Theo Nwankwo. 2025. Igbo tts normalized dataset. https://huggingface.co/datasets/ Twelve2five/igbo_tts_normalized. ↗
  19. Yiga Gilbert. 2024. Alffa wolof asr dataset. https://huggingface.co/datasets/yigagilbert/ alffa-wolof-asr-dataset-19hr. ↗
  20. CLEAR Global. 2025. Hausa synthetic asr dataset (xtts). https://huggingface.co/datasets/CLEAR-Global/ Hausa-Synthetic-ASR-Dataset-XTTS. ↗
  21. Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. 2006. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pages 369–376. find
  22. ASR Omnilingual, Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang, Ife Adebara, Michael Auli, Can Balioglu, and 1 others. 2025. Omnilingual asr: Open-source multilingual speech recognition for 1600+ languages. arXiv preprint arXiv:2511.09690. — [arXiv:2511.09690](https://arxiv.org/abs/2511.09690) find
  23. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. Association for Computational Linguistics.
  24. Qing Hu, Yan Zhang, Xianlei Zhang, Zongyu Han, and Xiuxia Liang. 2024. Language fusion via adapters for low-resource speech recognition. Speech Communication, 158:103037. find
  25. Vineel Pratap, Anuroop Sriram, Paden Tomasello, Awni Hannun, Vitaliy Liptchinsky, Gabriel Synnaeve, and Ronan Collobert. 2020. Massively multilingual asr: 50 languages, 1 model, 1 billion parameters. arXiv preprint arXiv:2007.03001. — [arXiv:2007.03001](https://arxiv.org/abs/2007.03001) find
  26. Nakamasa Inoue, Shinta Otake, Takumi Hirose, Masanari Ohi, and Rei Kawakami. 2024. Elp-adapters: Parameter efficient adapter tuning for various speech Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, and…
  27. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492–28518. PMLR.
  28. Dima Rekesh, Nithin Rao Koluguri, Samuel Kriman, Somshubra Majumdar, Vahid Noroozi, He Huang, Oleksii Hrinchuk, Krishna Puvvada, Ankur Kumar, Jagadeesh Balam, and 1 others. 2023. Fast conformer with linearly scalable attention for efficient speech recognition. In 2023 IEEE Automatic Speech Recogniti… find
  29. Ahnaf Mozib Samin, Shekhar Nayak, Andrea De Marco, and Claudia Borg. 2025. Investigating adapters for parameter-efficient low-resource automatic speech recognition. In Proceedings of the 10th Workshop on Representation Learning for NLP (RepL4NLP-2025), pages 100–107. find
  30. Fabio Sicoli. 2024. Twi dataset. https://huggingface.co/datasets/fsicoli/twi. ↗
  31. Realtime Speech. 2025. Shona https://huggingface.co/datasets/ realtime-speech/shona1. dataset. ↗
  32. Yui Sudo, Muhammad Shakeel, Yosuke Fukumoto, Yifan Peng, and Shinji Watanabe. 2024. Contextualized automatic speech recognition with attention-based bias phrase boosted beam search. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10896–109… find
  33. Jonathan Suru. 2024. Fon tts dataset. https://huggingface.co/datasets/jonathansuru/ fon_tts. ↗
  34. SEAMLESS Communication Team. 2025. Joint speech and text machine translation for up to 100 languages. Nature, 637(8046):587–593. find
  35. Andros Tjandra, Nayan Singhal, David Zhang, Ozlem Kalinli, Abdelrahman Mohamed, Duc Le, and Michael L Seltzer. 2023. Massively multilingual asr on 70 languages: Tokenization, architecture, and generalization capabilities. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Sig… find
  36. Shubham Toshniwal, Tara N Sainath, Ron J Weiss, Bo Li, Pedro Moreno, Eugene Weinstein, and Kanishka Rao. 2018. Multilingual speech recognition with a single end-to-end model. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 4904–4908. IEEE. find
  37. TutlaytAI. 2024. Kabyle asr dataset. https://huggingface.co/datasets/TutlaytAI/ kabyle_asr. ↗
  38. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  39. Hongfei Xue, Kaixun Huang, Zhikai Zhou, Shen Huang, Shidong Shang, and Lei Xie. 2025. The tea-aslp system for multilingual conversational speech recognition and speech diarization in mlc-slm 2025 challenge. In Proc. find
  40. Hongfei Xue, Qijie Shao, Kaixun Huang, Peikun Chen, Jie Liu, and Lei Xie. 2024. Sshr: Leveraging self-supervised hierarchical representations for multilingual automatic speech recognition. In 2024 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE. find
  41. Zhongzhi Yu, Yang Zhang, Kaizhi Qian, Cheng Wan, Yonggan Fu, Yongan Zhang, and Yingyan Celine Lin. 2023. Master-asr: achieving multilingual scalability and low-resource adaptation in asr with modular learning. In International Conference on Machine Learning, pages 40475–40487. PMLR. find
  42. Biao Zhang and Rico Sennrich. 2019. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32. find
  43. Yu Zhang, Wei Han, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Bo Li, Vera Axelrod, Gary Wang, and 1 others. 2023. Google usm: Scaling automatic speech recognition beyond 100 languages. arXiv preprint arXiv:2303.01037. — [arXiv:2303.01037](https://arxiv.org/abs/2303.01037) find
  44. Jinming Zhao, Vineel Pratap, and Michael Auli. 2025. Scaling a simple approach to zero-shot speech recognition. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/oukid-2026-puma-projected-universal-multilingual-asr/