Elamur Research⌕
☰
home / papers / 2023 / ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpus

ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpus

Summary

Arabic speech synthesis lags behind English largely because there is little suitable training data. Freely available Arabic speech collections tend to be multi-speaker and casual, with uneven recording quality. Corpora built specifically for TTS are compact sets designed for phonetic coverage, too small for modern end-to-end neural models.

The authors build ClArTTS from a LibriVox audiobook of a Classical Arabic book read by one male speaker. They segment the long recordings at pauses and drop chapters whose speaking style diverges. Three annotators then transcribe every segment by hand with full diacritics, and the transcripts go through two rounds of checking. The result is about 12 hours of consistent single-speaker speech.

They compare the corpus statistically with two existing Arabic TTS datasets, ASC and BAC. They then train Glow-TTS and Grad-TTS baselines on ClArTTS and on ASC, using both character and phoneme inputs. According to subjective and objective evaluations, synthesis trained on ClArTTS sounds better than synthesis trained on ASC.

This matters because it gives the field an open, sufficiently large Arabic resource, released with a demo and a HiFi-GAN vocoder.

Key points

  • Source: the LibriVox recording of Kitab Adab al-Dunya w'al-Din by Abu al-Hasan al-Mawardi, roughly 16 hours in 20 chapters. Three chapters were removed for stylistic mismatch.

  • Final ClArTTS: 12 h 10 min, 10,334 utterances, a single male speaker, original sampling rate of 40,100 Hz kept.

  • Segmentation and annotation:

    • Praat intensity-based scripts proposed the pause boundaries, which were cross-checked with Kaldi energy-based VAD.
    • Transcripts are fully diacritized and carry tags: [B] background noise, [H] hesitation or stutter, [*] unclear speech, [O] human noise.
    • SNR was used to guide which segments to keep.

  • Corpus comparison:

    CorpusSentencesWordsUnique wordsUnique diphones
    ClArTTS10,33482,97027,870520
    ASC (about 3.4 h)1,913——682
    BAC202———

    The diphone frequencies of ClArTTS were also checked against the large diacritized text corpus Tashkeela.

  • Baselines and evaluation:

    • Models: Glow-TTS and Grad-TTS with a HiFi-GAN vocoder.
    • Inputs: raw characters versus phonemes from a rule-based grapheme-to-phoneme converter.
    • Tested on both Classical Arabic and Modern Standard Arabic.
    • Metrics: MOS for listener judgments, plus objective measures including PESQ, a speaker-similarity score, MCD, Lf0 RMSE and BAP.

Interesting ideas

  • I1. An automatic segmenter's output (Praat pauses) was not trusted alone but confirmed by a second, independent detector (Kaldi VAD) before humans saw it. Agreement between unrelated tools is a cheap quality filter. — For Mova (learn): silver UD trees could likewise be accepted only where witness parsers built on different principles agree, with disagreements routed to an LLM teacher tier.
  • I2. Noise, hesitation and unclear stretches were kept in the transcript as explicit tags rather than deleted. Downstream users can then decide how to handle them, and the data stays honest about what was actually said. — For Mova (tokens): the tokenizer could emit typed markers for disfluencies, OCR junk or unreadable spans, so later stages can explain why a span was skipped instead of silently losing it.
  • I3. The authors checked the corpus's unit distribution (diphones) against a much larger reference corpus (Tashkeela), exposing over- and under-represented units. — For Mova (learn): comparing the distribution of deprels or constructions in our silver treebank against a large reference treebank would reveal coverage holes before training gparse.
  • I4. They preferred stylistic consistency over raw size, discarding whole chapters that drifted in style after spot listening. They also cite evidence that random subsets can rival coverage-optimized selections. — For Mova (learn): when assembling training sets, removing register-divergent sources and testing a random-subset baseline against hand-curated selections would serve as a useful negative control.
  • I5. They ran a controlled comparison of raw characters versus rule-based phonemes as model input. This shows directly how much a hand-written normalization layer actually contributes. — For Mova (tokens): ablating the morphology and lemma layer against raw forms when training the tagger or parser would show whether each rule-based preprocessing step earns its place.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

17 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 14 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.00069 · arXiv 2303.00069

@misc{kulkarni2023clartts,
  title = {{ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpus}},
  author = {Kulkarni, Ajinkya and Kulkarni, Atharva and Shatnawi, Sara Abedalmonem Mohammad and Aldarmaki, Hanan},
  year = {2023},
  doi = {10.48550/arxiv.2303.00069},
  eprint = {2303.00069},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.00069}
}

References

25 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. J. Hunt and A. W. Black, “Unit selection in a concatenative speech synthesis system using a large speech database,” in 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings, vol. 1. IEEE, 1996, pp. 373–376. find
  2. H. Zen, T. Nose, J. Yamagishi, S. Sako, T. Masuko, A. W. Black, and K. Tokuda, “The hmm-based speech synthesis system (hts) version 2.0.” SSW, vol. 6, pp. 294–299, 2007. find
  3. Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. V. Le, Y. Agiomyrgiannakis, R. Clark, and R. Saurous, “Tacotron: Towards end-to-end speech synthesis,” in proceedings of INTER-SPEECH, 2017. find
  4. R. J. Weiss, R. Skerry-Ryan, E. Battenberg, S. Mariooryad, and D. P. Kingma, “Wave-tacotron: Spectrogram-free end-to-end text-to-speech synthesis,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 5679–5683. find
  5. N. Halabi, “Modern standard arabic phonetics for speech synthesis,” Ph.D. dissertation, UNIVERSITY OF SOUTHAMPTON, 2016. find
  6. A. Amrouche, A. Abed, K. Ferrat, K. N. Boubakeur, Y. Bentrcia, and L. Falek, “Balanced arabic corpus design for speech synthesis,” International Journal of Speech Technology, vol. 24, no. 3, pp. 747–759, 2021. find
  7. T. Lambert, N. Braunschweiler, and S. Buchholz, “How (not) to select your voice corpus: random selection vs. phonologically balanced.” in SSW. Citeseer, 2007, pp. 264–269. find
  8. E. Bakhturina, V. Lavrukhin, B. Ginsburg, and Y. Zhang, “Hi-fi multi-speaker english tts dataset,” in Interspeech, 2021. find
  9. R. Abdelmalek and Z. Mnasri, “High quality arabic text-to-speech synthesis using unit selection,” 2016 13th International Multi-Conference on Systems, Signals & Devices (SSD), pp. 1–5, 2016. find
  10. A. A. Shalaby, O. A. Dakkak, and N. Ghneim, “An arabic text to speech based on semi-syllable concatenation,” International Review on Computers and Software, vol. 11, pp. 1178–1186, 2016. find
  11. O. O. Khalifa, M. Z. Obaid, A. W. Naji, and J. I. Daoud, “A rule-based arabic text-to-speech system based on hybrid synthesis technique,” 2011. find
  12. N. Halabi, “Modern standard arabic phonetics for speech synthesis,” 2016. find
  13. M. Baali, T. Hayashi, H. Mubarak, S. Maiti, S. Watanabe, W. El-Hajj, and A. Ali, “Unsupervised data selection for tts: Using arabic broadcast news as a case study,” ArXiv, vol. abs/2301.09099, 2023. ↗ our card
  14. A. Abdelali, N. Durrani, C. Demiroğlu, F. Dalvi, H. Mubarak, and K. Darwish, “Natiq: An end-to-end text-to-speech system for arabic,” ArXiv, vol. abs/2206.07373, 2022. ↗
  15. J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerrv-Ryan, R. A. Saurous, Y. Agiomvrgiannakis, and Y. Wu, “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 4779–4783. find
  16. O. Zine and A. Meziane, “Novel approach for quality enhancement of arabic text to speech synthesis,” 2017 International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–6, 2017. find
  17. A. Amrouche, A. Abed, K. Ferrat, K. N. Boubakeur, Y. Bentrcia, and L. Falek, “Balanced arabic corpus design for speech synthesis,” International Journal of Speech Technology, vol. 24, pp. 747 – 759, 2021. find
  18. T. Zerrouki and A. Balla, “Tashkeela: Novel corpus of arabic vocalized texts, data for auto-diacritization systems,” Data in Brief, vol. 11, pp. 147 – 151, 2017. find
  19. V. Popov, I. Vovk, V. Gogoryan, T. Sadekova, and M. A. Kudinov, “Grad-TTS: A diffusion probabilistic model for text-to-speech,” proceedings of International Conference on Machine Learning (ICML), 2021. find
  20. J. Kim, S. Kim, J. Kong, and S. Yoon, “Glow-TTS: A generative Flow for text-to-speech via monotonic alignment search,” In proceedings of Conference on Neural Information Processing Systems (NIPS), 2020. find
  21. J. Kong, J. Kim, and J. Bae, “HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis,” in proceedings of Conference on Neural Information Processing Systems (NIPS), 2020. find
  22. M. D. Polkosky and J. R. Lewis, “Expanding the MOS: development and psychometric evaluation of the MOS-R and MOS-X,” International Journal of Speech Technology, vol. 6, pp. 161–182, 2003. find
  23. A. W. Rix, J. G. Beerends, M. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,” 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221), vol. 2, pp. 749–752 vol.2, 2001. find
  24. A. Kulkarni, V. Colotte, and D. Jouvet, “Analysis of expressivity transfer in non-autoregressive end-to-end multispeaker tts systems,” in Interspeech, 2022. find
  25. B. Desplanques, J. Thienpondt, and K. Demuynck, “Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,” in Interspeech, 2020. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kulkarni-2023-clartts-open-source-classical-arabic/