Elamur Research⌕
☰
home / papers / 2026 / Zero-Shot Context-Aware ASR for Diverse Arabic Varieties

Zero-Shot Context-Aware ASR for Diverse Arabic Varieties

Paper · 2026 · Bashar Talafha, Amin Abu Alhassan and Muhammad Abdul-Mageed

Summary

Multilingual speech recognizers such as Whisper handle Modern Standard Arabic (MSA) reasonably well, but their accuracy drops sharply on regional dialects and accented speech, where labeled data is scarce. The authors test whether recognition can improve with no retraining at all, by feeding the model extra information at inference time. For Whisper they compare three ways of supplying this context:

  • inserting a rough first-pass transcript, produced by SeamlessM4T, into the decoder prompt, with its word order shuffled or reversed so the model does not simply continue it;
  • prompting with a similar sentence retrieved from a text index;
  • prepending retrieved speech–text exemplars to the encoder and decoder, optionally with audio synthesized in the target speaker's voice.

For CTC models that cannot take prompts, such as Omnilingual ASR, they rerank the n-best list by choosing the hypothesis closest in text to outputs from external "proxy" recognizers. Across ten Arabic test conditions, word error rate falls noticeably, most strongly for MSA and less for dialects. This offers a cheap adaptation route for low-resource varieties.

Key points

  • Relative to the Whisper and SeamlessM4T baselines, the strongest context-aware variants bring average WER down by 22.29% for MSA, by 20.54% when MSA is spoken with an accent, and by 9.15% for dialectal Arabic (all relative reductions).
  • On Common Voice MSA, proxy-guided n-best selection with Omnilingual ASR lowers WER from 20.58% (top-1) to 17.36%, a 15.6% relative reduction. Combining several proxies reaches 17.34%, against an oracle of 13.49%.
  • Reversing or shuffling the words of the prompt reduces hallucinations in which Whisper continues the prompt text, and speaker-matched TTS exemplars make prefixing more stable.
  • Retrieval relies on character-level TF–IDF, and retrieval indices exclude evaluation transcripts.
  • The authors note several limitations: extra latency from the auxiliary ASR, the retrieval step and TTS; Whisper's limited prompt budget; and missing dialects such as Sudanese, Mauritanian and Iraqi.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of British ColumbiaVancouver · Canada104 works · 37,403 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksSignal Processing · 493 works

🧭 Topics 2Research subjects and languages the work is about.Semitic languages · 3,066 worksSpeech & spoken language · 519 works

Cite

Cite as Bashar Talafha, Amin Abu Alhassan and Muhammad Abdul-Mageed. 2026. Zero-Shot Context-Aware ASR for Diverse Arabic Varieties. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1296

@inproceedings{talafha2026zero,
  title = {{Zero-Shot Context-Aware ASR for Diverse Arabic Varieties}},
  author = {Bashar Talafha, Amin Abu Alhassan and Muhammad Abdul-Mageed},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1296},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1296}
}

References

29 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Hamdy Mubarak, Shammur Absar Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, and 1 others. 2023. Larabench: Benchmarking arabic ai with large language models. arXiv preprint arXiv:2305.14982. ↗
  2. Ahmed Ali, Peter Bell, Mark Gales, Kevin Kilgour, Pierre Lanchantin, Xunying Liu, Steve Renals, and 1 others. 2016a. The mgb-2 challenge: Arabic multi-dialect broadcast media recognition. In IEEE Spoken Language Technology Workshop (SLT), pages 499– 504.
  3. Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, and Yifan Zhang. 2016b. The mgb-2 challenge: Arabic multi-dialect broadcast media recognition. In 2016 IEEE Spoken Language Technology Workshop (SLT), pages 279– 284. IEEE.
  4. Ahmed Ali, Stephan Vogel, and Steve Renals. 2017. Speech recognition challenge in the wild: Arabic mgb-3. In 2017 IEEE Automatic Speech Recognition https://alliancecan.ca and Understanding Workshop (ASRU), pages 316– 322. IEEE. ↗
  5. Ahmed Ali and 1 others. 2019. The mgb-5 challenge: Arabic multi-dialect broadcast media recognition for youtube videos. In Proc. ASRU. find
  6. Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M Tyers, and Gregor Weber. 2019. Common voice: A massively-multilingual speech corpus. arXiv preprint arXiv:1912.06670. ↗
  7. Arun Babu and et al. 2021. Xls-r: Self-supervised cross-lingual speech representation learning at scale. In Proc. Interspeech.
  8. Loïc Barrault, Yu-An Chung, Mariano Cora Meglioli, David Dale, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, and 1 others. 2023. Seamlessm4t: massively multilingual & multimodal machine translation. arXiv preprint arXiv:2308.11596. ↗
  9. Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, and 1 others. 2024. Xtts: a massively multilingual zero-shot text-to-speech model. arXiv preprint arXiv:2406.04904. ↗
  10. Jian Cheng. 2024. Context-aware speech recognition using prompts for language learners. In Proc. Interspeech 2024, pages 4009–4013. find
  11. Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna. 2023. Fleurs: Few-shot learning evaluation of universal representations of speech. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 798–805. IEEE.
  12. Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. 2020. Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification. arXiv preprint arXiv:2005.07143. ↗
  13. Yuu Jinnai. 2025. Re-evaluating minimum bayes risk decoding for automatic speech recognition. arXiv preprint arXiv:2510.19471. ↗
  14. Daniel Jurafsky and James H. Martin. 2025. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, with Language Models, 3rd edition. Online manuscript released August 24, 2025. find
  15. Maya K Nachesa and Vlad Niculae. 2024. knn for whisper and its effect on bias and speaker adaptation. arXiv preprint arXiv:2410.18850. ↗
  16. ASR Omnilingual, Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang, Ife Adebara, Michael Auli, Can Balioglu, and 1 others. 2025. Omnilingual asr: Open-source multilingual speech recognition for 1600+ languages. arXiv preprint arXiv:2511.09690. ↗
  17. Vineel Pratap, Qiantong Xu, Tatiana Likhomanenko, and 1 others. 2023. Scaling speech technology to 1,000+ languages. arXiv preprint arXiv:2305.13574. ↗
  18. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492–28518. PMLR. ↗
  19. Monica Sekoyan, Nithin Rao Koluguri, Nune Tade-vosyan, Piotr Zelasko, Travis Bartley, Nikolay Karpov, Jagadeesh Balam, and Boris Ginsburg. 2025. Canary-1b-v2 & parakeet-tdt-0.6 b-v3: Efficient and high-performance models for multilingual asr and ast. arXiv preprint arXiv:2509.14128. ↗
  20. Peng Shen, Xugang Lu, and Hisashi Kawai. 2025. Retrieval-augmented speech recognition approach for domain challenges. arXiv preprint arXiv:2502.15264. ↗
  21. Jiwon Suh, Injae Na, and Woohwan Jung. 2024. Improving domain-specific asr with llm-generated contextual descriptions. In Proceedings of Interspeech 2024. ArXiv:2407.17874. ↗
  22. Bashar Talafha, Karima Kadaoui, Samar Mohamed Magdy, Mariem Habiboullah, Chafei Mohamed Chafei, Ahmed Oumar El-Shangiti, Hiba Zayed, Rahaf Alhamouri, Rwaa Assi, Aisha Alraeesi, and 1 others. 2024. Casablanca: Data and models for multidialectal arabic speech recognition. arXiv preprint arXiv:2410.04527. ↗
  23. Bashar Talafha, Abdul Waheed, and Muhammad Abdul-Mageed. 2023. N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition. In Proc. Interspeech.
  24. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, and 1 others. 2025. Fanar: An arabic-centric multimodal generative ai platform. arXiv preprint arXiv:2501.13944. ↗
  25. Abdul Waheed, Karima Kadaoui, and Muhammad Abdul-Mageed. 2024a. To distill or not to distill? on the robustness of robust knowledge distillation. arXiv preprint arXiv:2406.04512. ↗
  26. Abdul Waheed, Karima Kadaoui, Bhiksha Raj, and Muhammad Abdul-Mageed. 2024b. udistil-whisper: Label-free data filtering for knowledge distillation in low-data regimes. arXiv preprint arXiv:2407.01257. ↗
  27. Nasser Waheed and 1 others. 2024c. To distill or not to distill? on the robustness of robust knowledge distillation. arXiv preprint arXiv:2406.04512. ↗
  28. Siyin Wang, Chao-Han Yang, Ji Wu, and Chao Zhang. 2024a. Can whisper perform speech-based in-context learning? In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 13421–13425. IEEE. find
  29. Siyin Wang, Chao-Han Huck Yang, Ji Wu, and Chao Zhang. 2024b. Can Whisper Perform Speech-Based In-Context Learning? In Proc. ICASSP. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/talafha-2026-zero-shot-context-aware-asr/