Elamur Research
☰
home / papers / 2020 / MLS: A Large-Scale Multilingual Dataset for Speech Research

MLS: A Large-Scale Multilingual Dataset for Speech Research

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive license to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/2012.03411

Summary

Open ASR benchmarks outside English tend to be small, fragmented, or restricted by their licenses. LibriSpeech became a standard largely because it is free, but it covers only English. The authors built Multilingual LibriSpeech (MLS), a read-speech corpus drawn from public-domain LibriVox audiobooks. It covers eight languages: roughly 44.5K hours of English and about 6K hours across German, Dutch, Spanish, French, Portuguese, Italian and Polish.

The labeling pipeline works in four steps:

  • In-house acoustic models cut long recordings into short clips.
  • The same models produce pseudo-labels for each clip.
  • Each pseudo-label is matched against the original book text to recover the true transcript.
  • Numbers, hyphens and apostrophes are cleaned with dedicated heuristics.

The release includes speaker-disjoint train/dev/test splits, n-gram language models and Transformer-based baseline recognizers for every language. Training on the English portion of MLS lowers WER on LibriSpeech's own test sets compared with training on LibriSpeech's 960 hours. MLS therefore gives the community a freely licensed resource for large-scale and multilingual ASR, with possible uses in TTS as well.

Key points

  • Segmentation and pseudo-labels: TDS models with ASG loss, trained on de-identified audio from public videos, split the recordings into 10–20 s segments at silences. They then generate pseudo-labels using 4-gram beam-search decoding.
  • Transcript retrieval: book text is divided into 1250-word windows with a 1000-word stride. The best window is found with TF-IDF similarity over bigrams, then refined with Smith-Waterman alignment. Candidates are discarded if their WER against the pseudo-label exceeds 40%.
  • Language models: 3-gram and 5-gram LMs were trained with KenLM and released together with their training text. Polish has a dev OOV rate of 13.39%, while English has only 0.18%.
  • Baselines: models were built in wav2letter++ with a 36-block Transformer encoder, CTC loss and SpecAugment. Decoding with the 5-gram LM lowers WER for every language except Polish. English test WER is 5.88 with the 5-gram LM.
  • Comparison with LibriSpeech: the same architecture with a 4-gram LM reaches 4.38 WER on LibriSpeech test-other when trained on MLS English. Training on LibriSpeech's 960 hours gives 5.6. The dataset is distributed through OpenSLR.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Cited by

151 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 142 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.21437/interspeech.2020-2826 · arXiv 2012.03411

@misc{pratap2020mls,
  title = {{MLS: A Large-Scale Multilingual Dataset for Speech Research}},
  author = {Pratap, Vineel and Xu, Qiantong and Sriram, Anuroop and Synnaeve, Gabriel and Collobert, Ronan},
  year = {2020},
  doi = {10.21437/interspeech.2020-2826},
  eprint = {2012.03411},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.21437/interspeech.2020-2826}
}

References

25 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An asr corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210. find
  2. M. P. Harper, “The iarpa babel program,” https://www.iarpa.gov/index.php/research-programs/babel. ↗
  3. Voxforge.org, “Free speech... recognition (linux, windows and mac) - voxforge.org,” http://www.voxforge.org/, accessed 06/25/2014. ↗
  4. R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” 2019. find our card
  5. I. Solak, “The M-AILABS Speech Dataset,” https://www.caito.de/2019/01/the-m-ailabs-speech-dataset/, 2019, [Online; accessed 19-April-2020]. ↗
  6. A. W. Black, “Cmu wilderness multilingual speech dataset,” in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 5971–5975. find
  7. A. Hannun, A. Lee, Q. Xu, and R. Collobert, “Sequence-to-sequence speech recognition with time-depth separable convolutions,” in INTERSPEECH, 2019. find
  8. R. Collobert, C. Puhrsch, and G. Synnaeve, “Wav2letter: an end-to-end convnet-based speech recognition system,” 2016. find
  9. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 369–376. find
  10. V. Liptchinsky, G. Synnaeve, and R. Collobert, “Letter-based speech recognition with gated convnets,” 2017. find
  11. V. Pratap, Q. Xu, J. Kahn, G. Avidov, T. Likhomanenko, A. Hannun, V. Liptchinsky, G. Synnaeve, and R. Collobert, “Scaling up online speech recognition using convnets,” 2020. find
  12. G. Synnaeve, Q. Xu, J. Kahn, T. Likhomanenko, E. Grave, V. Pratap, A. Sriram, V. Liptchinsky, and R. Collobert, “End-to-end asr: from supervised to semi-supervised learning with modern architectures,” 2019. find
  13. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015, pp. 5206–5210. find
  14. V. Manohar, D. Povey, and S. Khudanpur, “Jhu kaldi system for arabic mgb-3 asr challenge using diarization, audio-transcript alignment and transfer learning,” in 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2017, pp. 346–352. find
  15. T. Smith and M. Waterman, “Identification of common molecular subsequences,” Journal of Molecular Biology, vol. 147, no. 1, pp. 195 – 197, 1981. [Online]. Available: http://www.sciencedirect.com/science/article/pii/0022283681900875 ↗
  16. B. E. Boser, I. M. Guyon, and V. N. Vapnik, “A training algorithm for optimal margin classifiers,” in Proceedings of the Fifth Annual Workshop on Computational Learning Theory, ser. COLT ’92. New York, NY, USA: Association for Computing Machinery, 1992, p. 144–152. [Online]. Available: https://doi.o… — [doi:10.1145/130385.130401](https://doi.org/10.1145/130385.130401) ↗
  17. J. Kahn, M. Rivière, W. Zheng, E. Kharitonov, Q. Xu, P. E. Mazaré, J. Karadayi, V. Liptchinsky, R. Collobert, C. Fuegen, T. Likhomanenko, G. Synnaeve, A. Joulin, A. Mohamed, and E. Dupoux, “Libri-light: A benchmark for asr with limited or no supervision,” in ICASSP 2020 - 2020 IEEE International C… find
  18. K. Heafield, “Kenlm: Faster and smaller language model queries,” in Proceedings of the sixth workshop on statistical machine translation. Association for Computational Linguistics, 2011, pp. 187–197. find our card
  19. V. Pratap, A. Hannun, Q. Xu, J. Cai, J. Kahn, G. Synnaeve, V. Liptchinsky, and R. Collobert, “Wav2letter++: A fast open-source speech recognition system,” in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 6460–6464. find
  20. T. Likhomanenko, Q. Xu, V. Pratap, P. Tomasello, J. Kahn, G. Avidov, R. Collobert, and G. Synnaeve, “Rethinking evaluation in asr: Are our models robust enough?” arXiv preprint arXiv:2010.11745, 2020. — [arXiv:2010.11745](https://arxiv.org/abs/2010.11745) find
  21. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2017. find
  22. A. Fan, E. Grave, and A. Joulin, “Reducing transformer depth on demand with structured dropout,” in 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. [Online]. Available: https://openreview.net/forum?id=SylO2yStDr ↗ our card
  23. J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods for online learning and stochastic optimization.” Journal of machine learning research, vol. 12, no. 7, 2011. find
  24. D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, “Specaugment: A simple data augmentation method for automatic speech recognition,” Interspeech 2019, Sep 2019. [Online]. Available: http://dx.doi.org/10.21437/interspeech.2019-2680 ↗ our card
  25. H. Zen, R. Clark, R. J. Weiss, V. Dang, Y. Jia, Y. Wu, Y. Zhang, and Z. Chen, “Libritts: A corpus derived from librispeech for text-to-speech,” in Interspeech, 2019. [Online]. Available: https://arxiv.org/abs/1904.02882 ↗ our card