Elamur Research⌕
☰
home / papers / 2026 / MENASpeechBank: A Reference Voice Bank with Persona-Conditioned Multi-Turn Conversations for AudioLLMs

MENASpeechBank: A Reference Voice Bank with Persona-Conditioned Multi-Turn Conversations for AudioLLMs

Summary

Speech-capable LLMs need aligned audio–text data that is conversational, varied in task, and broad in speakers and dialects. Such data is scarce, especially for MENA Arabic varieties, because recording real persona-rich dialogues is costly and licensing limits reuse. The authors build MenaSpeechBank, a curated bank of reference voices in English, MSA and several Arabic dialects. Around it they build a generation pipeline. Speaker metadata seeds personas, which are filled out with country-level World Values Survey attributes and simple heuristics. The personas are then matched to taxonomy-based scenarios by embedding similarity. An LLM role-plays each dialogue, with the persona as the user and a helpful assistant replying. The user turns are rendered as speech by voice cloning from the reference recordings.

Models are scored on synthetic and human-recorded conversations by an LLM judge, using APR as the main metric and ARS as a diagnostic. Strong closed models do best. Fine-tuning gives open-weight SpeechLLMs a large boost, and Arabic-centric ASR→LLM cascades are competitive. The resource opens controlled study of spoken multi-turn memory, personalization and robustness for an underserved region.

Key points

  • The voice bank has about 18K utterances from 124 speakers across several MENA countries, drawn from QASR, ADI-17 (Egyptian, North African, Gulf, Levantine) and in-house recordings.
  • Reference audio was filtered in three ways:
    • In-house clips were kept only if WER was at most 0.05, using Fanar (Aura) transcription for Arabic and Whisper-Small for English.
    • QASR clips were 5–8 s segments, 10 per speaker, checked for MSA with Tamyiz and then by hand.
    • From ADI-17, 56 clean single-speaker samples from five speakers were picked by hand.
  • Personas combine several sources:
    • speaker metadata;
    • WVS Wave 7 country aggregates, adjusted by age;
    • sampled names, cities, jobs, devices and AI-competence levels;
    • two AI use cases;
    • a perturbed OCEAN personality vector.
  • The 124 seed profiles were expanded and deduplicated with multilingual Sentence-BERT, dropping candidates above 0.80 cosine similarity. This left 469 personas.
  • The pipeline produced about 5K scenarios and about 417K role-play conversations, mostly in MSA because dialectal conversational TTS is limited.
  • The judge's rubric covers eight dimensions: safety, language, calibration, tracking of context across turns, coherence, specificity, completeness and relevance.

Interesting ideas

  • I1. Embedding-based deduplication. Generated candidates are compared by embedding similarity and discarded above a fixed threshold, which keeps a synthetic pool diverse instead of letting near-copies pile up.
  • I2. Round-trip agreement filter. A sample is kept only if an independent system (ASR) reproduces its reference text almost exactly, which is a cheap check that data and labels truly align.
  • I3. Fixed intent, varied surface. Robustness is probed by holding the semantic intent of a scenario constant while varying speaker, dialect and recording conditions, so any change in output points to sensitivity to form.
  • I4. Profile-bounded memory tests. Each dialogue rests on an explicit persona profile, so one can check whether a model reuses facts from earlier turns and whether it invents user facts not present in the profile.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Self-play and LLM-teacher items could be deduplicated the same way, ideally by comparing their frame or graph signatures rather than surface text, so silver data does not overweight one pattern.
  • I2 (back): The text → graph → text compressor could use round-trip fidelity as an admission gate, keeping only sentences whose regenerated text matches the original closely as trusted training or test material.
  • I3 (learn): Evaluation could pair each test item with paraphrase variants that should yield identical frames and answers, so a divergence acts as a built-in negative control.
  • I4 (answer): QA on tales could explicitly test whether answers stay within entities and states introduced earlier, and flag any claim the world of the text does not support.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Semitic languages

Cite

Cite as Zien Sheikh Ali, Hunzalah Hassan Bhatti, Rabindra Nath Nandi, Shammur Absar Chowdhury and Firoj Alam. 2026. MENASpeechBank: A Reference Voice Bank with Persona-Conditioned Multi-Turn Conversations for AudioLLMs. arXiv:2602.07036.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2602.07036 · arXiv 2602.07036

@misc{ali2026menaspeechbank,
  title = {{MENASpeechBank: A Reference Voice Bank with Persona-Conditioned Multi-Turn Conversations for AudioLLMs}},
  author = {Zien Sheikh Ali, Hunzalah Hassan Bhatti, Rabindra Nath Nandi, Shammur Absar Chowdhury and Firoj Alam},
  year = {2026},
  doi = {10.48550/arxiv.2602.07036},
  eprint = {2602.07036},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2602.07036}
}

References

42 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Badr M. Abdullah, Matthew Baas, Bernd Möbius, and Dietrich Klakow. 2025. Voice Conversion Improves Cross-Domain Robustness for Spoken Arabic Dialect Identification. In Interspeech 2025, pages 2790– 2794. our card
  2. Syeda Faiza Ahmed, Zien Sheikh Ali, Hunzalah Hassan Bhatti, Firoj Alam, and Shammur Absar Chowdhury. 2026. Multi-turn conversational ai from text to multimodal interaction: Data, models, evaluation, and open challenges. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing. find
  3. Shahad Al-Khalifa, Nadir Durrani, Hend Al-Khalifa, and Firoj Alam. 2025. The landscape of arabic large language models. Communications of the ACM, 68(10):54–61. find
  4. Firoj Alam, Md Arid Hasan, and Shammur Absar Chowdhury. 2025. SpokenNativQA: Multilingual Everyday Spoken Queries for LLMs. In Interspeech 2025, pages 2685–2689. Firoj Alam, Ali Ezzat Shahroor, Md. find
  5. Arid Hasan, Zien Sheikh Ali, Hunzalah Hassan Bhatti, Mohamed Bayan Kmainasi, Shammur Absar Chowdhury, Basel Mousi, Fahim Dalvi, Nadir Durrani, and Natasa Milic-Frayling. 2026. OASIS: A multilingual and multimodal dataset for culturally grounded spoken visual qa. arXiv preprint arXiv:2510.06371. ↗ our card
  6. Ahmed Ali, Shammur Chowdhury, Mohamed Afify, Wassim El-Hajj, Hazem Hajj, Mourad Abbas, Amir Hussein, Nada Ghneim, Mohammad Abushariah, and Assal Alqudah. 2021. Connecting arabs: bridging the gap in dialectal speech recognition. Commun, ACM, 64(4):124–129. find
  7. Zien Sheikh Ali, Hamdy Mubarak, Soon-Gyo Jung, Hunzalah Hassan Bhatti, Firoj Alam, and Shammur Absar Chowdhury. 2026. WASIL: In-the-wild arabic spoken interactions with LLMs. In Proceedings of Interspeech 2026, Sydney, Australia. our card
  8. M Saiful Bari, Yazeed Alnumay, Norah Alzahrani, Nouf Alotaibi, Hisham Alyahya, AlRashed AlRashed, Faisal Mirza, Shaykhah Alsubaie, Hassan Alahmed, Ghadah Alabduljabbar, Raghad Alkhathran, Yousef Almushayqih, Raneem Alnajim, Salman I Alsubaihi, Maryam Al Mansour, Saad Hassan, Majed Alrubaian, Ali Alammari, Zaki Alawami, and 7 others. 2025. Allam: Large language models for arabic and english. In International Conference on Learning Representations, volume 2025, pages 34179–34214. our card
  9. Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Ben Goodrich, Daniel Duckworth, Semih Yavuz, Amit Dubey, Kyu-Young Kim, and Andy Cedilnik. 2019. Taskmaster-1: Toward a realistic and diverse dialog dataset. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4516–4525, Hong Kong, China. Association for Computational Linguistics. our card
  10. Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Aljafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, and Julian Weber. 2024. XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model. In Interspeech 2024, pages 4978–4982. find
  11. Shammur Absar Chowdhury, Evgeny Stepanov, and Giuseppe Riccardi. 2016. Predicting user satisfaction from turn-taking in spoken conversations. In Proceedings of Interspeech. find
  12. Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. 2023. Qwen-Audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919. ↗ our card
  13. EVS/WVS. 2022. European values study and world values survey: Joint evs/wvs 2017-2022 dataset (joint evs/wvs). JD Systems Institute and WVSA. Dataset Version 3.0. 0. find
  14. Advait Gosai, Tyler Vuong, Utkarsh Tyagi, Steven Li, Wenjia You, Miheer Bavare, Arda Uçar, Zhongwang Fang, Brian Jang, Bing Liu, and 1 others. 2025. Audio MultiChallenge: A multi-turn evaluation of spoken dialogue systems on natural human interaction. arXiv preprint arXiv:2512.14865. ↗
  15. Xingang Guo, Utkarsh Tyagi, Advait Gosai, Paula Vergara, Jayeon Park, Ernesto Gabriel Hernández Montoya, Chen Bo Calvin Zhang, Bin Hu, Yunzhong He, Bing Liu, and 1 others. 2025. Beyond seeing: Evaluating multimodal llms on tool-enabled image perception, transformation, and reasoning. arXiv preprint arXiv:2510.12712. ↗
  16. Kilem Li Gwet. 2008. Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(1):29–48. find
  17. Christian Haerpfer, Ronald Inglehart, Alejandro Moreno, Christian Welzel, Kseniya Kizilova, Jaime Diez-Medrano, Marta Lagos, Pippa Norris, Eduard Ponarin, and Bi Puranen. 2022. World values survey: Round seven – country-pooled datafile version 6.0. Technical report, JD Systems Institute and WVSA Secretariat, Madrid, Spain and Vienna, Austria. Md. find
  18. Arid Hasan, Maram Hasanain, Fatema Ahmad, Sahinur Rahman Laskar, Sunaya Upadhyay, Vrunda N Sukhadia, Mucahid Kutlu, Shammur Absar Chowdhury, and Firoj Alam. 2025. NativQA: Multilingual culturally-aligned natural query for LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14886–14909, Vienna, Austria. Association for Computational Linguistics. find
  19. Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, Yi Ren, Yuexian Zou, Zhou Zhao, and Shinji Watanabe. 2024. AudioGPT: Understanding and generating speech, music, sound, and talking head. Proceedings of the AAAI Conference on Artificial Intelligence, 38(21):23802–23804. find
  20. Pegah Jandaghi, Xianghai Sheng, Xinyi Bai, Jay Pujara, and Hakim Sidahmed. 2024. Faithful persona-based conversational dataset generation with large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15245–15270, Bangkok, Thailand. Association for Computational Linguistics. find
  21. Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. 2023. Voicebox: Text-guided multilingual universal speech generation at scale. In Advances in Neural Information Processing Systems, volume 36, pages 14005–14034. Curran Associates, Inc. find
  22. Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016. A persona-based neural conversation model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 994–1003, Berlin, Germany. Association for Computational Linguistics. our card
  23. Gabriel Mittag, Babak Naderi, Assmaa Chehadi, and Sebastian Möller. 2021. NISQA: A deep cnn-self-attention model for multidimensional speech quality prediction with crowdsourced datasets. In Proc. Interspeech 2021, pages 2127–2131. find
  24. Hamdy Mubarak, Amir Hussein, Shammur Absar Chowdhury, and Ahmed Ali. 2021. QASR: QCRI aljazeera speech resource a large scale annotated Arabic speech corpus. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2274–2285, Online. Association for Computational Linguistics. find
  25. Eliya Nachmani, Alon Levkovitch, Roy Hirsch, Julian Salazar, Chulayuth Asawaroengchai, Soroosh Mariooryad, Ehud Rivlin, RJ Skerry-Ryan, and Michelle Tadmor Ramanovich. 2024. Spoken question answering and speech continuation using spectrogram-powered LLM. In The Twelfth International Conference on Learning Representations (ICLR). find
  26. Vahid Noroozi, Zhehuai Chen, Somshubra Majumdar, Steve Huang, Jagadeesh Balam, and Boris Ginsburg. 2024. Instruction data generation and unsupervised adaptation for speech language models. In Interspeech 2024, pages 4049–4053. find
  27. Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020. Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-20), pages 8689–8696. find
  28. Mirco Ravanelli, Titouan Parcollet, Peter Plantinga, Aku Rouhe, Samuele Cornell, Loren Lugosch, Cem Subakan, Nauman Dawalatabad, Abdelwahab Heba, Jianyuan Zhong, Ju-Chieh Chou, Sung-Lin Yeh, Szu-Wei Fu, Chien-Feng Liao, Elena Rastorgueva, François Grondin, William Aris, Hwidong Na, Yan Gao, and 2 others. 2021. SpeechBrain: A general-purpose speech toolkit. arXiv preprint arXiv:2106.04624. ↗
  29. Traian Rebedea, Makesh Sreedhar, Shaona Ghosh, Jiaqi Zeng, and Christopher Parisien. 2024. CantTalk-AboutThis: Aligning language models to stay on topic in dialogues. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12232–12252, Miami, Florida, USA. Association for Computational Linguistics. find
  30. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics. ↗ our card
  31. Suwon Shon, Ahmed Ali, Younes Samih, Hamdy Mubarak, and James Glass. 2020. ADI17: A fine-grained arabic dialect identification dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8244–8248. IEEE. our card
  32. E Stepanov, Benoit Favre, Firoj Alam, S Chowdhury, Karan Singla, Jeremy Trione, Frederic Béchet, and Giuseppe Riccardi. 2015. Automatic summarization of call-center conversations. In Proceedings of ASRU. find
  33. Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. 2024. SALMONN: Towards generic hearing abilities for large language models. Preprint, arXiv:2310.13289. ↗ our card
  34. FANAR Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Minhaj Ahmad, Abdulaziz Al-Homaid, Anas Al-Nuaimi, Enes Altinisik, Ehsaneddin Asgari, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Asim Ersoy, Masoomali Fatehkia, Mohammed Qusay Hashim, and 18 others. 2026. Fanar 2.0: Arabic generative ai stack. arXiv preprint arXiv:2603.16397. ↗
  35. Fanar Team, Ummar Abbas, Mohammad Shahmeer Ahmad, Firoj Alam, Enes Altinisik, Ehsannedin Asgari, Yazan Boshmaf, Sabri Boughorbel, Sanjay Chawla, Shammur Chowdhury, Fahim Dalvi, Kareem Darwish, Nadir Durrani, Mohamed Elfeky, Ahmed Elmagarmid, Mohamed Eltabakh, Masoomali Fatehkia, Anastasios Fragkopoulos, Maram Hasanain, and 23 others. 2025. Fanar: An arabic-centric multimodal generative ai platform. our card
  36. Bin Wang, Xunlong Zou, Geyu Lin, Shuo Sun, Zhuohan Liu, Wenyu Zhang, Zhengyuan Liu, AiTi Aw, and Nancy F. Chen. 2025a. AudioBench: A universal benchmark for audio large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4297–4316, Albuquerque, New Mexico. Association for Computational Linguistics. find
  37. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2025b. Neural codec language models are zero-shot text to speech synthesizers. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 33:705–718. our card
  38. Qiongqiong Wang, Hardik Bhupendra Sailor, Tianchi Liu, Wenyu Zhang, Muhammad Huzaifah, Nattadaporn Lertcheva, Shuo Sun, Nancy F. Chen, Jinyang Wu, and AiTi Aw. 2025c. Benchmarking contextual and paralinguistic reasoning in Speech-LLMs: A case study with in-the-wild data. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 14133–14148, Suzhou, China. Association for Computational Linguistics. find
  39. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-Instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13484–13508, Toronto, Canada. Association for Computational Linguistics. find
  40. Qian Yang, Jin Xu, Wenrui Liu, Yunfei Chu, Ziyue Jiang, Xiaohuan Zhou, Yichong Leng, Yuanjun Lv, Zhou Zhao, Chang Zhou, and Jingren Zhou. 2024. AIR-bench: Benchmarking large audio-language models via generative comprehension. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1979–1998, Bangkok, Thailand. Association for Computational Linguistics. find
  41. Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. 2023. SpeechGPT: Empowering large language models with intrinsic cross-modal conversational abilities. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15757–15773, Singapore. Association for Computational Linguistics. our card
  42. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics. our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ali-2026-menaspeechbank-reference-voice-bank-persona/