Elamur Research⌕
☰
home / papers / 2020 / AccentDB: A Database of Non-Native English Accents to Assist Neural Speech Recognition

AccentDB: A Database of Non-Native English Accents to Assist Neural Speech Recognition

Paper · 2020 · Afroz Ahamad, Ankit Anand and Pranesh Bhargava

Summary

Speech recognizers handle native English well but struggle with speakers whose English is shaped by another first language. The authors argue that accent research needs carefully designed data. They list what such a database requires: many speakers, sentence-length material, identical and phonetically balanced content across speakers, and meaningful sentences. They then build AccentDB to meet these requirements. Volunteers with strong Bangla, Malayalam, Odiya and Telugu accents read Harvard Sentences in a soundproof booth. The authors added synthetic recordings from Amazon Polly for four native accents (American, Australian, British, Welsh) and a metropolitan "Indian" accent. Because every accent reads the same sentences, the corpus is parallel.

On this data they examine how separable the accents are. They train accent classifiers, compare them with human-labelled classes, and test generalization on seen and unseen data. They also inspect attention scores to see which audio frames matter. Finally, they propose accent neutralization, which maps non-native speech toward native accents using autoencoders, and report preliminary results. The resource is intended to support ASR training, feature augmentation and acoustic transformation.

Key points

  • AccentDB covers 9 accents with 16,984 samples (about 19 h 49 min). The 4 self-collected non-native accents come from 8 speakers and total about 9 h 6 min. The 5 Amazon Polly accents total about 10 h 43 min.
  • Content comes from the Harvard Sentences (IEEE, 1969): 72 sets of 10 phonetically balanced sentences. At least the same 25 sets were recorded for all four non-native accents, and all 72 sets were compiled for the Polly accents.
  • Recordings were made in a professional soundproof booth with an Audio Technica AT2005USB microphone. Prompts were shown through a purpose-built web-app.
  • The authors criticize existing resources such as the Speech Accent Archive, FAE and CMU-Indic. Their problems include unverified self-reported accent labels, inconsistent recording conditions, and content that differs across samples.
  • For neutralization, the authors use a convolutional autoencoder whose skip connections pass a target-accent label vector into every layer. They also try a stacked LSTM autoencoder with skip connections.

Interesting ideas

  • I1. The authors write down explicit data requirements before collecting anything, such as sentence-length material, uniform content and balanced phonemes, and then audit existing corpora against that list. This makes dataset design choices checkable rather than ad hoc.
  • I2. Keeping the text identical across all accents isolates the one variable under study, so differences can only come from accent and not from content.
  • I3. The paper shows that self-assigned labels in crowdsourced data can be wrong, for example a speaker whose original accent faded abroad, and calls for independent supervision of labels.
  • I4. The authors note that isolated words behave differently from words in connected speech, so the data should match the target use, which here is sentences.
  • I5. The neutralization model injects a target-style label into every layer through skip connections, so the desired output condition is never lost deep in the network.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Before building a new silver or evaluation set, Mova could state a short requirement checklist and score candidate sources against it, so gaps are visible up front.
  • I2 (learn): An evaluation suite could present the same propositions in several phrasings, registers or UD dialects, so that changes in parser or frame output reveal sensitivity to form rather than meaning.
  • I3 (learn): This supports Mova's practice of agreeing silver data across several witness parsers instead of trusting any single source's labels, and of logging where each label came from.
  • I4 (tokens): Tagger and lemmatizer tests should run on words in running text rather than on word lists, because context-driven behaviour only appears there.
  • I5 (back): A text generator or graph-to-text compressor could carry the target register or dialect as a condition at each generation step, instead of only at the start.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of GroningenGroningen · Netherlands49 works · 4,843 citations

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPsychology · 798 worksExperimental and Cognitive Psychology · 463 worksSignal Processing · 213 works

🧭 Topics 2Research subjects and languages the work is about.English · 1,373 worksSpeech & spoken language · 233 works

Cite

Cite as Afroz Ahamad, Ankit Anand and Pranesh Bhargava. 2020. AccentDB: A Database of Non-Native English Accents to Assist Neural Speech Recognition. Proceedings of the Twelfth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ahamad2020accentdb,
  title = {{AccentDB: A Database of Non-Native English Accents to Assist Neural Speech Recognition}},
  author = {Afroz Ahamad, Ankit Anand and Pranesh Bhargava},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.659/}
}

References

29 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. ↗
  2. Bearman, A., Josund, K., and Fiore, G. (2017). Accent conversion using artificial neural networks. find
  3. Bird, J. J., Wanner, E., Ekárt, A., and Faria, D. R. (2019). Accent classification in human speech biometrics for native and non-native english speakers. find
  4. Chorowski, J. K., Bahdanau, D., Serdyuk, D., Cho, K., and Bengio, Y. (2015). Attention-based models for speech recognition. In Advances in neural information processing systems, pages 577–585. ↗
  5. Ge, Z., Tan, Y., and Ganapathiraju, A. (2015). Accent classification with phonetic vowel representation. In 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), pages 529–533. IEEE. find
  6. Gemmeke, J. F., Ellis, D. P. W., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., and Ritter, M. (2017). Audio set: An ontology and human-labeled dataset for audio events. In Proc. find
  7. IEEE ICASSP 2017, New Orleans, LA. find
  8. Guo, J., Xu, N., Li, L.-J., and Alwan, A. (2017). Attention based cldnns for short-duration acoustic scene classification. find
  9. Hernandez, S. P., Bulitko, V., Carleton, S., Ensslin, A., and Goorimoorthee, T. (2018). Deep learning for classification of speech accents in video games. In Joint Proceedings of the AIIDE 2018 Workshops 2018. find
  10. IEEE. (1969). Ieee recommended practice for speech quality measurements. IEEE No 297-1969, pages 1–24. find
  11. Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al. (2017). Googleâs multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5:339–351. find
  12. Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  13. Kitashov, F., Svitanko, E., and Dutta, D. (2018). Foreign english accent adjustment by learning phonetic patterns. CoRR, abs/1807.03625. ↗
  14. Ko, T., Peddinti, V., Povey, D., and Khudanpur, S. (2015). Audio augmentation for speech recognition. In INTER-SPEECH, pages 3586–3589. ISCA. find
  15. Kominek, J. and Black, A. (2004). The cmu arctic speech databases. SSW5-2004, 01. find
  16. Ladefoged, P. (1993.). A Course in phonetics. Harcourt,, Firt Worth :, 3rd ed. edition. find
  17. Lander, T. (2007). Cslu foreign accented english release 1.2 ldc2007s08. find
  18. Mazzoni, D. (1999). Audacity ® software is copyright © 1999-2019 audacity team. find
  19. Nagrani, A., Chung, J. S., and Zisserman, A. (2017). Voxceleb: a large-scale speaker identification dataset. CoRR. find
  20. Park, D. S., Chan, W., Zhang, Y., Chiu, C.-C., Zoph, B., Cubuk, E. D., and Le, Q. V. (2019). Specaugment: A simple data augmentation method for automatic speech recognition. arXiv preprint arXiv:1904.08779. ↗
  21. Saon, G., Kurata, G., Sercu, T., Audhkhasi, K., Thomas, S., Dimitriadis, D., Cui, X., Ramabhadran, B., Picheny, M., Lim, L., Roomi, B., and Hall, P. (2017). English conversational telephone speech recognition by humans and machines. CoRR, abs/1703.02136. ↗
  22. Teixeira, C., Trancoso, I., and Serralheiro, A. (1996). Accent identification. 01. find
  23. Tieleman, T. and Hinton, G. (2012). Lecture 6.5— RmsProp: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning. find
  24. Toda, T., Chen, L.-H., Saito, D., Villavicencio, F., Wester, M., Wu, Z., and Yamagishi, J. (2016). The voice conversion challenge 2016. find
  25. Viglino, T., Motlicek, P., and Cernak, M. (2019). End-to-end accented speech recognition. Proc. Interspeech 2019, pages 2140–2144. find
  26. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., and Manzagol, P.-A. (2010). Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11:3371– 3408, December. find
  27. Weinberger, S. and Kunath, S. (2011). The speech accent archive: Towards a typology of english accents. Language and Computers, 73, 12. ↗
  28. Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015). Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning, pages 2048–2057. ↗
  29. Yang, X., Audhkhasi, K., Rosenberg, A., Thomas, S., Ramabhadran, B., and Hasegawa-Johnson, M. (2018). Joint modeling of accents and acoustics for multi-accent speech recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ahamad-2020-accentdb-database-non-native-english/