Elamur Research⌕
☰
home / papers / 2024 / GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speech

GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speech

Paper · 2024 · Wenbin Wang, Yang Song and Sanjay Jha

Summary

Zero-shot speaker adaptive TTS clones a voice from a few seconds of audio, but it still handles accented speakers poorly. The authors trace much of this to the training data: LibriTTS is dominated by US English, and VCTK has only 11 accents. Common Voice covers far more accents, but its clips are often noisy, band-limited or misread, and its speaker labels are incomplete and messy.

The authors build GLOBE from the English part of Common Voice 14.0 with a staged pipeline:

  • discard clips with low SNR, narrow real bandwidth or long internal pauses;
  • verify transcripts against Whisper output after normalizing both texts;
  • clean accent labels and fill missing accent, age and gender with trained classifiers;
  • trim edge silence and denoise.

The result is 535 hours of 24 kHz speech from 23,519 speakers covering 164 accents, with complete speaker metadata. A speaker adaptive TTS model trained on GLOBE gives better speaker similarity than models trained on other popular corpora, with comparable naturalness. This makes GLOBE an accent-diverse resource for fairer voice cloning.

Key points

  • GLOBE was filtered from Common Voice English, which has 3,347 hours and 88,904 speakers. It keeps 535 hours from 23,519 speakers, 164 accents and more than 50 countries. For comparison, VCTK has 44 hours and 109 speakers, and LibriTTS/LibriTTS-R have about 585 hours and 2,456 speakers.
  • The audio filters are:
    • SNR below 0 dB (WADA estimation) is removed;
    • actual bandwidth under 12 kHz is removed;
    • continuous internal silence over 930 ms (detected with Silero VAD) is removed.
  • The transcript check works as follows. Whisper transcribes each clip, and both texts are normalized to spoken form with NeMo WFST text normalization. Clips with a word-level edit distance above 1 are dropped. Clips whose single difference is a repeated word are also dropped.
  • Missing metadata is filled by three classifiers that share one architecture built on HuBERT and ECAPA-TDNN. They are trained on about 11,000 speakers with square-root sampling. Test accuracy is 97.22% for accent, 99.55% for age and 99.95% for gender.
  • Evaluation covers ground-truth samples and TTS outputs. The metrics are NMOS (five raters per sample, 95% confidence interval), UT-MOS predicted naturalness, WER from a Conformer ASR model, and speaker embedding cosine similarity (SMCS) with TitaNet-L.

Interesting ideas

  • I1. Use an independent recognizer as a witness for the given transcript, and treat certain kinds of disagreement as diagnostic rather than just counting errors: a lone repeated word signals a reader correcting themselves.
  • I2. Take filter thresholds from statistics of a trusted reference corpus instead of guessing; the 930 ms pause limit is the longest internal silence found in LibriTTS clean.
  • I3. Clean noisy categorical labels before using them: merge or drop categories with too few members, delete meaningless labels, then train a predictor on the cleaned data to fill gaps and test it on label-balanced held-out sets.
  • I4. Square-root sampling flattens a long-tailed label distribution during training without fully equalizing it, so rare classes still get learned.
  • I5. Normalize both sides to the same spoken form before computing edit distance, so harmless formatting differences do not count as mismatches.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when witness parsers disagree on silver sentences, classify the disagreement type, such as attachment of a repeated token or a coordination split, and reject sentences with known bad patterns even when the overall disagreement is small.
  • I2 (learn): set silver-treebank cutoffs, such as maximum dependency length or non-projective arcs per sentence, from the observed maxima in gold UD EWT.
  • I3 (world): fill missing Wikidata/WordNet attributes the same way, prune rare or junk classes first and report accuracy on a balanced test set, so that frequent classes do not inflate the score.
  • I4 (tree): sample training sentences by the square root of the frequency of their rarest deprel, so the perceptron parser and gparse see enough rare relations like vocative or dislocated.
  • I5 (back): in the text → graph → text compressor, canonicalize numbers, contractions and casing in both the original and the regenerated text before measuring round-trip fidelity.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓UNSW SydneySydney · Australia34 works · 1,509 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksSignal Processing · 452 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksSpeech & spoken language · 480 works

Cite

Cite as Wenbin Wang, Yang Song and Sanjay Jha. 2024. GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speech. arXiv:2406.14875.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2406.14875 · arXiv 2406.14875

@misc{wang2024globe,
  title = {{GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speech}},
  author = {Wenbin Wang, Yang Song and Sanjay Jha},
  year = {2024},
  doi = {10.48550/arxiv.2406.14875},
  eprint = {2406.14875},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2406.14875}
}

References

39 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Y. Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” in Proc. of ICLR, 2021. ↗
  2. V. Popov, I. Vovk, V. Gogoryan, T. Sadekova, and M. A. Kudinov, “Grad-tts: A diffusion probabilistic model for text-to-speech,” in Proc. of ICML, vol. 139, 2021, pp. 8599–8608. find
  3. J. Kim, J. Kong, and J. Son, “Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech,” in Proc. of ICML, vol. 139, 2021, pp. 5530–5540. ↗
  4. Z. Ye, W. Xue, X. Tan, J. Chen, Q. Liu, and Y. Guo, “Comospeech: One-step speech and singing voice synthesis via consistency model,” in Proc. of ACM MM, 2023, pp. 1831–1839. find
  5. X. Tan, T. Qin, F. K. Soong, and T. Liu, “A survey on neural speech synthesis,” CoRR, vol. abs/2106.15561, 2021. ↗
  6. M. Chen, X. Tan, B. Li, Y. Liu, T. Qin, S. Zhao, and T. Liu, “Adaspeech: Adaptive text to speech for custom voice,” in Proc. of ICLR, 2021. ↗
  7. E. Casanova, J. Weber, C. D. Shulby, A. C. Júnior, E. Gölge, and M. A. Ponti, “Yourtts: Towards zero-shot multi-speaker TTS and zero-shot voice conversion for everyone,” in Proc. of ICML, vol. 162, 2022, pp. 2709–2720. ↗
  8. W. Wang, Y. Song, and S. Jha, “Generalizable zero-shot speaker adaptive speech synthesis with disentangled representations,” in Proc. of INTERSPEECH, 2023, pp. 4454–4458. find
  9. Z. Jiang, Y. Ren, Z. Ye, J. Liu, C. Zhang, Q. Yang, S. Ji, R. Huang, C. Wang, X. Yin, Z. Ma, and Z. Zhao, “Mega-tts: Zero-shot text-to-speech at scale with intrinsic inductive bias,” CoRR, vol. abs/2306.03509, 2023. ↗
  10. W. Wang, Y. Song, and S. K. Jha, “USAT: A universal speaker-adaptive text-to-speech approach,” IEEE ACM Trans. Audio Speech Lang. Process., vol. 32, pp. 2590–2604, 2024. ↗
  11. C. Wang, S. Chen, Y. Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y. Liu, H. Wang, J. Li, L. He, S. Zhao, and F. Wei, “Neural codec language models are zero-shot text to speech synthesizers,” CoRR, vol. abs/2301.02111, 2023. ↗
  12. H. Zen, V. Dang, R. Clark, Y. Zhang, R. J. Weiss, Y. Jia, Z. Chen, and Y. Wu, “Libritts: A corpus derived from librispeech for text-to-speech,” in Proc. of INTERSPEECH, 2019, pp. 1526–1530. ↗
  13. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in Proc. of ICASSP, 2015, pp. 5206–5210. find
  14. V. Christophe, Y. Junichi, M. Kirsten, Y. Zhang, R. J. Weiss, Y. Jia, Z. Chen, and Y. Wu, “Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit,” in University of Edinburgh. The Centre for Speech Technology Research (CSTR), 2016. find
  15. R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Henretty, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” in Proc. of LREC, 2020, pp. 4218–4222. ↗
  16. S. Ogun, V. Colotte, and E. Vincent, “Can we use common voice to train a multi-speaker TTS system?” in Proc. of IEEE SLT, 2022, pp. 900–905. find
  17. Y. Koizumi, H. Zen, S. Karita, Y. Ding, K. Yatabe, N. Morioka, M. Bacchiani, Y. Zhang, W. Han, and A. Bapna, “Libritts-r: A restored multi-speaker text-to-speech corpus,” CoRR, vol. abs/2305.18802, 2023. ↗
  18. C. Kim and R. M. Stern, “Robust signal-to-noise ratio estimation based on waveform amplitude distribution analysis,” in Proc. of INTERSPEECH, 2008. find
  19. E. Bakhturina, V. Lavrukhin, B. Ginsburg, and Y. Zhang, “Hi-fi multi-speaker english TTS dataset,” in Proc. of INTERSPEECH, 2021, pp. 2776–2780. ↗
  20. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in Proc. of ICML, 2023. ↗
  21. Y. Zhang, E. Bakhturina, and B. Ginsburg, “Nemo (inverse) text normalization: From development to production,” in Proc. of INTERSPEECH, 2021, pp. 4857–4859. ↗
  22. W. Hsu, B. Bolte, Y. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE ACM Trans. Audio Speech Lang. Process., vol. 29, pp. 3451–3460, 2021. ↗
  23. B. Desplanques, J. Thienpondt, and K. Demuynck, “ECAPA-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification,” in Proc. of INTERSPEECH, 2020, pp. 3830–3834. find
  24. B. Kang, S. Xie, M. Rohrbach, Z. Yan, A. Gordo, J. Feng, and Y. Kalantidis, “Decoupling representation and classifier for long-tailed recognition,” in Proc. of ICLR, 2020. ↗
  25. D. Min, D. B. Lee, E. Yang, and S. J. Hwang, “Meta-stylespeech : Multi-speaker adaptive text-to-speech generation,” in Proc. of ICML, vol. 139, 2021, pp. 7748–7759. ↗
  26. S. Ö. Arik, J. Chen, K. Peng, W. Ping, and Y. Zhou, “Neural voice cloning with a few samples,” in Proc. of NeurIPS, 2018, pp. 10 040–10 050. find
  27. T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “UTMOS: utokyo-sarulab system for voicemos challenge 2022,” in Proc. of INTERSPEECH, 2022, pp. 4521– 4525. find
  28. S. Huang, C. Lin, D. Liu, Y. Chen, and H. Lee, “Meta-tts: Meta-learning for few-shot speaker adaptive text-to-speech,” IEEE ACM Trans. Audio Speech Lang. Process., vol. 30, pp. 1558– 1571, 2022. ↗
  29. J. Xue, Y. Deng, Y. Han, Y. Li, J. Sun, and J. Liang, “ECAPA-TDNN for multi-speaker text-to-speech synthesis,” in Proc. of ISCSLP, 2022, pp. 230–234. find
  30. W. Huang, E. Cooper, Y. Tsao, H. Wang, T. Toda, and J. Yamagishi, “The voicemos challenge 2022,” in Proc. of INTERSPEECH, 2022, pp. 4536–4540. find
  31. D. Rekesh, S. Kriman, S. Majumdar, V. Noroozi, H. Huang, O. Hrinchuk, A. Kumar, and B. Ginsburg, “Fast conformer with linearly scalable attention for efficient speech recognition,” CoRR, vol. abs/2305.05084, 2023. ↗
  32. N. R. Koluguri, T. Park, and B. Ginsburg, “Titanet: Neural model for speaker representation with 1d depth-wise separable convolutions and global context,” in Proc. of ICASSP, 2022. ↗
  33. A. Nagrani, J. S. Chung, and A. Zisserman, “Voxceleb: A large-scale speaker identification dataset,” in Proc. of Interspeech, 2017. find
  34. D. Friedman and A. B. Dieng, “The vendi score: A diversity evaluation metric for machine learning,” Transactions on Machine Learning Research, 2023. ↗
  35. R. Burgert, X. Li, A. Leite, K. Ranasinghe, and M. S. Ryoo, “Diffusion illusions: Hiding images in plain sight,” CoRR, vol. abs/2312.03817, 2023. ↗
  36. Y. Chen, B. Xu, Q. Wang, Y. Liu, and Z. Mao, “Benchmarking large language models on controllable generation under diversified instructions,” CoRR, vol. abs/2401.00690, 2024. ↗
  37. F. Wilcoxon, “Individual comparisons by ranking methods,” in Breakthroughs in Statistics: Methodology and Distribution, 1992, pp. 196–202. ↗
  38. M. Bernard and H. Titeux, “Phonemizer: Text to phones transcription for multiple languages in python,” Journal of Open Source Software, vol. 6, no. 68, p. 3958, 2021. ↗
  39. I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in Proc. of ICLR, 2019. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2024-globe-high-quality-english-corpus/