Elamur Research⌕
☰
home / papers / 2026 / Fine-tuning Whisper with Spontaneous Persian Speech (SPS)

Fine-tuning Whisper with Spontaneous Persian Speech (SPS)

Summary

Most Persian speech corpora, such as FARSDAT and Mozilla Common Voice, contain read speech. As a result, ASR systems handle informal Persian poorly, especially hesitations, overlapping turns and discourse markers. The authors address this gap with the Spontaneous Persian Speech (SPS) dataset, built from YouTube videos in which two or three young speakers with a Tehrani accent chat in places ranging from studios to cars and shops. They used a semi-automatic workflow:

  • OpenAI's Whisper produced the first transcripts as .srt files.
  • The pysrt library converted these to TextGrid format.
  • A native speaker corrected the timestamps, spelling and punctuation in Praat.

The resulting corpus has roughly 11 hours of audio. The authors tested off-the-shelf Whisper on a 70-minute portion and found very high error rates, often involving consonants with similar articulation. They attribute this to missing informal register in Whisper's training data. Fine-tuning the small and medium Whisper models on SPS substantially lowered WER. The work suggests that even a modest corpus targeted at one domain can meaningfully strengthen ASR for an under-resourced, diglossic language.

Key points

  • SPS contains 694 minutes of audio from 65 speakers (34 male, 31 female), totalling 526,585 tokens. Speakers are aged 20–30 and recorded in quiet and noisy environments.
  • Segments last 1.24–3.25 s and hold about 3–9 words, averaging 5.01 words per interval. Lexical diversity (TTR) is 0.352.
  • Baseline WER on 70 minutes of SPS was 88.72% for Whisper small, 83.10% for medium and 49.04% for large (large-v3). Typical errors include /d/→/t/ and /z/→/s/ substitutions and dropped words.
  • Once fine-tuned on SPS, the small and medium models each showed a marked drop in WER. The provided text omits the exact post-fine-tuning figures.
  • The dataset is for non-commercial research use only. It will not be shared publicly until YouTube channel owners give consent, and it would also need anonymization, since speakers mention their full names.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

 Institutions

🎓Université Paris CitéParis · France11 works · 1,368 citations🔬Approches Linguistiques Théoriques, Appliquées et Expérimentales : langues et cultures connectéesParis · France4 works · 64 citations

Cite

Cite as Behnoosh Namdarzadeh and Nicolas Ballier. 2026. Fine-tuning Whisper with Spontaneous Persian Speech (SPS). Proceedings of the SIGUL 2026 Joint Workshop with ELE, EURALI, and DCLRL: Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered Languages.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2ca2yoj8fzgd

@inproceedings{namdarzadeh2026fine,
  title = {{Fine-tuning Whisper with Spontaneous Persian Speech (SPS)}},
  author = {Behnoosh Namdarzadeh and Nicolas Ballier},
  booktitle = {Proceedings of the SIGUL 2026 Joint Workshop with ELE, EURALI, and DCLRL: Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered Languages},
  year = {2026},
  doi = {10.63317/2ca2yoj8fzgd},
  url = {https://doi.org/10.63317/2ca2yoj8fzgd}
}

References

34 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15166–15184, Miami, Florida, USA. Association for Computational Linguistics. find
  2. Martine Adda-Decker, Claude Barras, Gilles Adda, Patrick Paroubek, Philippe Boula de Mareüil, and Benoit Habert. 2008. Annotation and analysis of overlapping speech in political interviews. In LREC 2008, pages 3105–3111. find
  3. Nicolas Ballier, Behnoosh Namdarzadeh, Maria Zimina, and Jean-Baptiste Yunès. 2023. Translating dislocations or parentheticals: Investigating the role of prosodic boundaries for spoken language translation of French into English. In Proceedings of Machine Translation Summit XIX, Vol. 2: Users Track,… find
  4. Paul Boersma and David Weenink. 2025. Praat: doing phonetics by computer. https://www.praat.org. Computer program. ↗
  5. Ruixing Cihan, Zhang Xiangyu, Mehmet Emre, Veronica Lavanya, Rong Ethan, Emmanuel Sanjeev, and Leibny Paola Garcia. 2025. Casper: A large scale spontaneous speech dataset. arXiv preprint arXiv:2506.00267v1. — [arXiv:2506.00267](https://arxiv.org/abs/2506.00267) find
  6. Steven Coats. 2019. A Corpus of Regional American Language from YouTube. In Proceedings of the 4th Digital Humanities in the Nordic Countries Conference (DHN 2019), pages 79–91, Copenhagen, Denmark. find
  7. Abed Alhakim Freihat and Mourad Abbas, editors. 2021. Proceedings of the Second International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2021) co-located with ICNLSP 2021. Association for Computational Linguistics, Trento, Italy. find
  8. Dawit Ketema Gete, Bedru Yimam Ahmed, Tadesse Destaw Belay, Yohannes Ayana Ejigu, Sukairaj Hafiz Imam, Alemu Belay Tessema, Mohammed Oumer Adem, Tadesse Amare Belay, Robert Geislinger, Umma Aliyu Musa, Martin Semmann, Shamsuddeen Hassan Muhammad, Henning Schreiber, and Seid Muhie Yimam. 2025. Whispe… our card
  9. Roya Kabiri, Simin Karimi, and Mihai Surdeanu. 2022. Informal Persian Universal Dependency treebank. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 7096–7105, Marseille, France. European Language Resources Association. find
  10. William Labov. 1972. Sociolinguistic Patterns. University of Pennsylvania Press, Philadelphia. find
  11. V. I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8):707–710. find
  12. Song Li, Yongbin You, Xuezhi Wang, Zhengkun Tian, Ke Ding, and Guanglu Wan. 2024. Msr-86k: An evolving, multilingual corpus with 86,300 hours of transcribed audio for speech recognition research. pages 1245–1249. find
  13. Zoey Liu, Crystal Richardson, Richard Hatcher, and Emily Prud’hommeaux. 2022. Not always about you: Prioritizing community needs when developing endangered language technology. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3… find
  14. Yanir Marmor, Kinneret Misgav, and Yair Lifshitz. 2023. ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and Development. ArXiv2307.08720. our card
  15. Mihai Nadăș, Laura Dioșan, and Andreea Tomescu. 2025. Synthetic data generation using large language models: Advances in text and code. IEEE Access, pages 134615– 134633. find
  16. Behnoosh Namdarzadeh and Nicolas Ballier. 2022. The neural machine translation of dislocations. find
  17. Nuno M. Guerreiro, Duarte M. Alves, Jonas Waldendorf, Barry Haddow, Alexandra Birch, Pierre Colombo, and André F. T. Martins. 2023. Hallucinations in large multilingual translation models. Transactions of the Association for Computational Linguistics, 11:1500–1517. our card
  18. Behnoosh Namdarzadeh, Nicolas Ballier, Lichao Zhu, Guillaume Wisniewski, and Jean-Baptiste Yunès. 2022. Toward a test set of dislocations in Persian for neural machine translation. In Proceedings of the Third International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2022) co-locat… find
  19. Camille Harris, Chijioke Mgbahurike, Neha Kumar, and Diyi Yang. 2024. Modeling gender and dialect bias in automatic speech recognition. In Daniel Pawlowicz, Jule Weber, and Claudia Dukino. 2025. Effectiveness of Whisper’s Fine-Tuning for Domain-Specific Use Cases in the Industry. In Proceedings of t… find
  20. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine Mcleavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 28492–2… our card
  21. Ali Safaya and Engin Erzin. 2022. Experiments on Turkish ASR with Self-Supervised Speech Representation Learning. ArXiv2210.07323. — [arXiv:2210.07323](https://arxiv.org/abs/2210.07323) find
  22. Mohammad Azim Saraji, Abdullah Khalili, and Ahmad Hatam. 2025. PAZHVAK: a Word-Level Farsi Speech Corpus by University of Hormoz-gan. SN Computer Science, 6(7):865. find
  23. Nima Sedghiyeh, Sara Sadeghi, Reza Khodadadi, Farzin Kashani, Omid Aghdaei, Somayeh Rahimi, and Mohammad Sadegh Safari. 2025. PSRB: A Comprehensive Benchmark for Evaluating Persian ASR Systems. arXiv preprint arXiv:2505.21230. — [arXiv:2505.21230](https://arxiv.org/abs/2505.21230) find
  24. Najm Ul Sehar, Ayesha Khalid, Farah Adeeba, and Sarmad Hussain. 2025. Benchmarking Whisper for Low-Resource Speech Recognition: An N-Shot Evaluation on Pashto, Punjabi, and Urdu. In Proceedings of the First Workshop on Challenges in Processing South Asian Languages (CHiPSAL 2025), pages 202–207, Abu… find
  25. Jingyi Sun, Yaru Wu, Nicolas Audibert, and Martine Adda-Decker. 2024. Création d’un corpus parallèle de styles de parole en mandarin via l’auto-transcription et l’alignement forcé. In Actes de JEP-TALN-RÉCITAL 2024 (35� Journées d’Études sur la Parole), pages 291– 300, Toulouse, France. Actes des 35… find
  26. Nasrin Taghizadeh and Heshaam Faili. 2016. Automatic wordnet development for low-resource languages using cross-lingual WSD. Journal of Artificial Intelligence Research, 56:61–87. find
  27. Jean E Fox Tree and Josef C Schrock. 1999. Discourse markers in spontaneous speech: Oh what a difference an oh makes. Journal of Memory and Language, 40(2):280–295. find
  28. Jörgen Valk and Tanel Alumäe. 2021. Voxlingua107: a dataset for spoken language recognition. In 2021 IEEE Spoken Language Technology Workshop (SLT), pages 652–658. IEEE. find
  29. Marcely Zanon Boito, Laurent Besacier, Natalia Tomashenko, and Yannick Estève. 2022. A study of gender impact in self-supervised models for speech-to-text systems. In Proceedings of Interspeech 2022, pages 1278–1282, Incheon, Korea. \#\# 10. find
  30. Language Resource References Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. Common voice: A massively-multilingual speech corpus. In Proceedings of the 12th Conference on Language Resour… our card
  31. Team Omnilingual ASR, Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang, Ife Adebara, Michael Auli, Can Balioglu, Kevin Chan, Chierh Cheng, Joe Chuang, Caley Droof, Mark Duppenthaler, Paul-Ambroise Duquenne, Alexander Erben, Cynthia Gao, Gabriel Mejia Gonzalez,… find
  32. Mahmood Bijankhan, Javad Sheikhzadegan, and Mahmood R Roohani. 1994. FARSDAT- The speech database of Farsi spoken language. In Proceedings of the Australian Conference on Speech Science and Technology, volume 2, pages 826–830. find
  33. Mahmood Bijankhan, Javad Sheykhzadegan, Mahmood R Roohani, Rahman Zarrintare, Seyyed Z Ghasemi, and Mohammad E Ghasedi. 2003. Tfarsdat-the telephone Farsi speech database. In Proc. find
  34. Eurospeech 2003, pages 1525–1528. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/namdarzadeh-2026-fine-tuning-whisper-spontaneous-persian/