Elamur Research⌕
☰
home / papers / 2023 / Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Study

Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Study

Summary

The paper asks whether usable Arabic text-to-speech can be built without a studio corpus by reusing broadcast news collected for ASR. The authors start from Aljazeera recordings in MGB-2/QASR and keep only two anchor speakers, one male and one female. They then filter for clean segments in two ways: a linguist sorted segments into six classes, and three automatic MOS predictors (MOSNet, wv-MOS, DNSMOS) scored each segment, with a threshold of 4. An Arabic ASR system repairs faulty transcripts, and automatic diacritization adds the missing vowels. For modelling, Tacotron2 and Transformer-TTS are pre-trained on English LJSpeech and fine-tuned on about one hour of Arabic. These autoregressive models then provide durations to a FastSpeech2-based Conformer. The final system reaches 3.9% CER, against 1.3% for real recordings, and scores MOS 4.4 for intelligibility and 4.2 for naturalness. Listeners often recognized the actual broadcaster. The study shows that a small, carefully filtered subset of found data can be enough to build TTS for a language with few resources.

Key points

  • Source data: MGB-2 Aljazeera broadcasts from 2004 to 2015, covering more than 4,000 episodes across 19 programs, with audio at 16 kHz. The speaker metadata was messy; Barack Obama alone appeared under 9 name variants. It was cleaned through repeated automatic parsing followed by manual checks.
  • Manual labelling: Among the anchors' segments, the two biggest problems were bad recording quality (40%) and overlapping speech (33%). About one hour of clean audio per speaker survived.
  • Automatic selection: DNSMOS and wv-MOS agreed closely with the human "good" labels. A TTS trained on the DNSMOS selection reached 4.0% CER, compared with 3.9% for the hand-picked selection.
  • Diacritization: Adding vowels to the transcripts cut CER from 32.2% to 3.9%.
  • Training and release: The AR models start from weights pre-trained on LJSpeech (24 h). The token embedding layer is not transferred because the symbol sets differ, and guided attention loss is added. Audio is generated with a Parallel WaveGAN vocoder. The recipe is published in ESPnet as qasr_tts.

Interesting ideas

  • I1. Name the failure types first, then validate the filter: The authors defined explicit classes of bad data on a sample labelled by an expert, and only then checked an automatic filter against that sample. The filter's quality becomes measurable instead of assumed. — For Mova (learn): Classify silver-tree failures (e.g. wrong attachment, tokenization error, garbage input) on a hand-checked subset. Then measure how well witness-agreement filtering recovers the trees that humans judged good.
  • I2. A small clean set can beat a large noisy one: A cheap quality score with a fixed threshold produced a training subset that matched expert selection. This suggests aggressive filtering often pays off more than collecting more volume. — For Mova (learn): Train parsers only on silver sentences above a strict agreement threshold, and compare against the full silver pool to show how much noise costs.
  • I3. Repair good samples instead of discarding them: Segments with clean audio but faulty transcripts were corrected with a second model (ASR) rather than thrown away. This keeps scarce data that would otherwise be lost. — For Mova (learn): When witness parsers disagree on only one arc, apply self-induced repair rules to fix that arc. This keeps the rest of the sentence's tree rather than dropping it.
  • I4. Restore missing input information before modelling: Adding diacritics, which the text simply lacked, made a larger difference than any architecture choice. — For Mova (tokens): Normalize information that is often missing from input before tagging, such as dropped apostrophes, casing, and contractions. This keeps downstream errors from being blamed on the parser.
  • I5. Use a slow, robust model to train a fast one: The autoregressive model, which learns alignment reliably, provided durations to the faster non-autoregressive model, which could not learn alignment well from one hour of data. — For Mova (tree): The GPU transformer parser gparse could provide arc or label targets for training the faster perceptron parser on unlabelled text.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

5 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 3 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2301.09099 · arXiv 2301.09099

@misc{baali2023unsupervised,
  title = {{Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Study}},
  author = {Baali, Massa and Hayashi, Tomoki and Mubarak, Hamdy and Maiti, Soumi and Watanabe, Shinji and El-Hajj, Wassim and Ali, Ahmed},
  year = {2023},
  doi = {10.48550/arxiv.2301.09099},
  eprint = {2301.09099},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2301.09099}
}

References

31 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. H. Zen, V. Dang, R. Clark, Y. Zhang, Ron J Weiss, Y. Jia, Z. Chen, and Y. Wu, “LibriTTS: A corpus derived from Librispeech for text-to-speech,” Proc. Interspeech 2019. find our card
  2. Y. Ren, Y. Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Liu, “Fastspeech: Fast, robust and controllable text to speech,” Advances in neural information processing systems, 2019. find
  3. H. Tachibana, K. Uenoyama, and S. Aihara, “Efficiently trainable text-to-speech system based on deep convolutional networks with guided attention,” in ICASSP, 2018. find
  4. Je. Donahue, S. Dieleman, M. Bińkowski, E. Elsen, and K. Simonyan, “End-to-end adversarial text-to-speech,” International Conference on Learning Representations, 2020. find
  5. Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, et al., “Tacotron: Towards end-to-end speech synthesis,” Proc. Interspeech 2017. find
  6. Y. Ren, C. Hu, T. Qin, S. Zhao, Z. Zhao, and T. Liu, “Fastspeech 2: Fast and high-quality end-to-end text-to-speech,” International Conference on Learning Representations, 2020. find
  7. R. Sonobe, S. Takamichi, and H. Saruwatari, “JSUT corpus: free large-scale japanese speech corpus for end-to-end speech synthesis,” arXiv:1711.00354, 2017. ↗
  8. T. Hayashi, R. Yamamoto, K. Inoue, T. Yoshimura, S. Watanabe, T. Toda, K. Takeda, Y. Zhang, and X. Tan, “Espnet-tts: Unified, reproducible, and integratable open source end-to-end text-to-speech toolkit,” in ICASSP, 2020. find
  9. T. Tu, Y. Chen, C. Yeh, and H. Lee, “End-to-end text-to-speech for low-resource languages by cross-lingual transfer learning,” Proc. Interspeech 2019. find
  10. Y. Chung, Y. Wang, W. Hsu, Y. Zhang, and R. Skerry-Ryan, “Semi-supervised training for improving data efficiency in end-to-end speech synthesis,” in ICASSP, 2019. find
  11. J. Xu, X. Tan, Y. Ren, T. Qin, J. Li, S. Zhao, and T. Y. Liu, “Lrspeech: Extremely low-resource speech synthesis and recognition,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2020. find
  12. H. Zhang and Y. Lin, “Unsupervised learning for sequence-to-sequence text-to-speech for low-resource languages,” Proc. Interspeech 2020. find
  13. P. Gallegos, J. Williams, J. Rownicka, and S. King, “An unsupervised method to select a speaker subset from large multi-speaker speech synthesis datasets,” Interspeech, 2020. find
  14. H. Mubarak, A. Hussein, S. Chowdhury, and A. Ali, “QASR: QCRI Aljazeera speech resource–a large scale annotated Arabic speech corpus,” ACL, 2021. find
  15. P. Bell, M. Gales, T. Hain, J. Kilgour, P. Lanchantin, X. Liu, A. McParland, S. Renals, Os. Saz, M. Wester, et al., “The MGB challenge: Evaluating multi-genre Broadcast media recognition,” in ASRU, 2015. find
  16. A. Ali, P. Bell, J. Glass, Y. Messaoui, H. Mubarak, S. Renals, and Y. Zhang, “The MGB-2 challenge: Arabic multi-dialect broadcast media recognition,” in SLT, 2016. find our card
  17. C. K. Reddy, V. Gopal, and R. Cutler, “Dnsmos: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors,” in ICASSP 2021. find
  18. C. Lo, S. Fu, W. Huang, X. Wang, J. Yamagishi, Y. Tsao, and H. Wang, “Mosnet: Deep learning based objective assessment for voice conversion,” in Interspeech 2019. find
  19. P. Andreev, A. Alanov, O. Ivanov, and D. Vetrov, “Hifi++: a unified framework for neural vocoding, bandwidth extension and speech enhancement,” arXiv:2203.13086, 2022. ↗
  20. B. Naderi and R. Cutler, “An open source implementation of itu-t recommendation p. 808 with validation,” Proc. Interspeech 2020. find
  21. C. K. Reddy, V. Gopal, R. Cutler, E. Beyrami, R. Cheng, H. Dubey, S. Matusevych, R. Aichner, A. Aazami, S. Braun, et al., “The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results,” Proc. Interspeech 2020. find
  22. A. Hussein, S. Watanabe, and A. Ali, “Arabic speech recognition by end-to-end, modular systems and human,” Computer Speech & Language, 2021. find our card
  23. A. Abdelali, K. Darwish, N. Durrani, and H. Mubarak, “Farasa: A fast and furious segmenter for Arabic,” in The North American chapter of the association for computational linguistics: Demonstrations, 2016. find our card
  24. J. Shen, R. Pang, R. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerrv-Ryan, et al., “Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions,” in ICASSP, 2018. find
  25. N. Li, S. Liu, Y. Liu, S. Zhao, and M. Liu, “Neural speech synthesis with transformer network,” in AAAI, 2019. find
  26. J. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, and Y. Bengio, “Attention-based models for speech recognition,” NIPS, 2015. find our card
  27. K. Ito and L. Johnson, “The lj speech dataset,” https://keithito.com/LJ-Speech-Dataset, 2017. Online: ↗
  28. P. Guo, F. Boyer, X. Chang, T. Hayashi, Y. Higuchi, H. Inaguma, N. Kamo, C. Li, D. Garcia-Romero, J. Shi, et al., “Recent developments on ESPnet toolkit boosted by conformer,” ICASSP, 2021. find
  29. A. Łańcucki, “Fastpitch: Parallel text-to-speech with pitch prediction,” ICASSP, 2021. find
  30. N. Perraudin, P. Balazs, and P. L. Søndergaard, “A fast griffin-lim algorithm,” in 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. find
  31. R. Yamamoto, E. Song, and J. Kim, “Parallel wavegan: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram,” in ICASSP, 2020. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/baali-2023-unsupervised-data-selection-tts-using/