Elamur Research⌕
☰
home / papers / 2023 / MPSA-DenseNet: A novel deep learning model for English accent classification

MPSA-DenseNet: A novel deep learning model for English accent classification

Paper · 2023 · Tianyu Song, Linh Thi Hoai Nguyen and Ton Viet Ta

Summary

The paper addresses automatic recognition of a speaker's English accent by country or region, rather than transcribing accented speech. The authors assembled 9117 recordings covering six varieties: three native (British, American, Scottish) and three non-native (Chinese, German, Indian). The recordings come from the Speech Accent Archive, the CSTR VCTK Corpus and their own collection. Each clip was converted to wav, resampled, fixed to six seconds, lightly noised and turned into MFCC feature maps. On a DenseNet-121 backbone they built three variants:

  • Multi-DenseNet adds speaker age and gender as auxiliary prediction tasks over a shared network.
  • PSA-DenseNet inserts a PSA attention module.
  • MPSA-DenseNet combines both.

All variants were trained with identical hyperparameters and compared with plain DenseNet-121 and EPSANet. The combined model scored best, at about 97% on F0.5 and validation accuracy, and each single addition already beat the baselines. The work suggests that cheap side labels and attention can noticeably sharpen a fine-grained audio classifier.

Key points

  • Data: 9117 clips, six accents, plus labels for five age bands and binary gender used only as auxiliary targets.

  • Preprocessing: FFmpeg conversion to wav, 44,100 Hz, two channels, trim or silence-pad to 6 s, Gaussian noise, pre-emphasis α = 0.97, and 64 MFCCs over 516 frames (hop 512).

  • Training: MXNet, Adam, learning rate 0.0001, 128 epochs, batch 16, softmax cross-entropy, with the same settings for every model.

  • Results on the test set (micro / macro F0.5) and validation accuracy:

    ModelMicro F0.5Macro F0.5Val. accuracyParameters
    MPSA-DenseNet97.2%96.4%97.5%19.64M
    Multi-DenseNet95.3%94.9%95.1%5.56M
    PSA-DenseNet94.8%94.2%93.8%19.92M
    EPSANet94.3%93.5%94.3%20.6M
    DenseNet-12193.6%92.7%93.9%7.98M

  • Error analysis: German was recognised perfectly, while Mandarin and American were the weakest classes but still above 94%. The most common mistake was Indian labelled as Mandarin (4.3%).

Interesting ideas

  • I1. Predicting easily available side attributes (age, gender) through separate heads on one shared trunk acts as a regulariser and boosted the main task without extra annotation effort.
  • I2. Each component was tested alone and in combination under one frozen hyperparameter set, with parameter counts reported next to scores, so gains can be attributed rather than tuned in.
  • I3. Micro and macro F-scores are reported together with a normalised confusion matrix, which exposes skewed classes and names the single most frequent confusion pair.
  • I4. Inputs are normalised to a fixed shape and mildly perturbed with noise so the model learns features that survive small distortions.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): gparse could share its encoder with light auxiliary heads such as UPOS, morphological features or distance-to-head, which would push the encoder toward syntax-relevant representations at almost no labelling cost.
  • I2 (learn): when adding a new feature or repair rule, run the base, each addition alone, and the full combination with identical settings, and log model size, to keep improvement claims honest.
  • I3 (learn): dependency-label and frame-role evaluations could routinely report macro scores and the top confused label pairs, pointing directly at which UD relations or VerbNet roles need targeted repair rules.
  • I4 (tokens): the tagger and lemmatizer could be trained on lightly corrupted copies of the text, with typos, casing changes and missing punctuation, to stay stable on messy school or web input.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Kyushu UniversityFukuoka · Japan4 works · 25 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSignal Processing · 452 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Tianyu Song, Linh Thi Hoai Nguyen and Ton Viet Ta. 2023. MPSA-DenseNet: A novel deep learning model for English accent classification. arXiv:2306.08798.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2306.08798 · arXiv 2306.08798

@misc{song2023mpsa,
  title = {{MPSA-DenseNet: A novel deep learning model for English accent classification}},
  author = {Tianyu Song, Linh Thi Hoai Nguyen and Ton Viet Ta},
  year = {2023},
  doi = {10.48550/arxiv.2306.08798},
  eprint = {2306.08798},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2306.08798}
}

References

25 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ittichaichareon, C., Suksri, S., & Yingthawornsuk, T. Speech recognition using MFCC. International conference on computer graphics, simulation and modeling 9 (2012). find
  2. Hossan, M. A., Memon, S., & Gregory, M. A. A novel approach for MFCC feature extraction. 2010 4th International Conference on Signal Processing and Communication Systems. 1-5 (2010). Accuracy (Validation set) Total Parameters 92.7% 93.5% 94.9% 94.2% 96.4% 93.9% 94.3% 95.1% 93.8% 97.5% 7.98M 20.6M 5.56M 19.92M 19.64M 10/11 find
  3. Aida-Zade,K.R., Ardil, C., & Rustamov, S.S. Investigation of combined use of MFCC and LPC features in speech recognition systems. World Academy of Science, Engineering and Technology, 19, 74-80 (2006). find
  4. Pedersen, C., & Diederich, J. Accent classification using support vector machines. 6th IEEE/ACIS International Conference on Computer and Information Science (ICIS 2007) 444-449 (2007). find
  5. Rizwan, M., & Anderson, D. V. A weighted accent classification using multiple words. Neurocomputing, 277, 120-128 (2018). find
  6. Tang, H., & Ghorbani, A. A. Accent classification using support vector machine and hidden markov model. Advances in Artificial Intelligence: 16th Conference of the Canadian Society for Computational Studies of Intelligence, AI 2003, Halifax, Canada, June 11–13, 2003, Proceedings 16. 629-631. Springer Berlin Heidelberg (2003). find
  7. Hou, J., Liu, Y., Zheng, T. F., Olsen, J., & Tian, J. Multi-layered features with SVM for Chinese accent identification. 2010 International Conference on Audio, Language and Image Processing. 25-30 (2010). find
  8. Angkititrakul, P., & Hansen, J. H. Advances in phone-based modeling for automatic accent classification. IEEE transactions on audio, speech, and language processing, 14(2), 634-646 (2006). find
  9. Kumpf, K., & King, R. W. Automatic accent classification of foreign accented Australian English speech. Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP’96 (Vol. 3, pp. 1740-1743). IEEE (1996). find
  10. Bird, J. J., Wanner, E., Ekárt, A., & Faria, D. R. Accent classification in human speech biometrics for native and non-native english speakers. Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments 554-560 (2019). find
  11. Jiao, Y., Tu, M., Berisha, V., & Liss, J. M. Accent Identification by Combining Deep Neural Networks and Recurrent Neural Networks Trained on Long and Short Term Features. Interspeech 2388-2392 (2016). find
  12. Chionh, K., Song, M., & Yin, Y. Application of Convolutional Neural Networks in Accent Identification. Project Report, Carnegie Mellon University, Pittsburgh, Pennsylvania (2018). find
  13. Al-Jumaili, Z., Bassiouny, T., Alanezi, A., Khan, W., Al-Jumeily, D., & Hussain, A. J. Classification of Spoken English Accents Using Deep Learning and Speech Analysis. Intelligent Computing Methodologies: 18th International Conference, ICIC 2022, Xi’an, China, August 7–11, 2022, Proceedings, Part III 277-287. Cham: Springer International Publishing (2022). find
  14. Gao, Q., Wu, H., Sun, Y., & Duan, Y. An end-to-end speech accent recognition method based on hybrid ctc/attention transformer asr. ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 7253-7257. IEEE (2021). find
  15. Zeng, Y., Mao, H., Peng, D., & Yi, Z. Spectrogram based multi-task audio classification. Multimedia Tools and Applications, 78, 3705-3722 (2019). find
  16. Weinberger, Steven. Speech Accent Archive. George Mason University. Retrieved from http://accent.gmu.edu (2015). ↗
  17. Yamagishi, Junichi; Veaux, Christophe; MacDonald, Kirsten. CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit (version 0.92), [sound]. University of Edinburgh. The Centre for Speech Technology Research (CSTR). https://doi.org/10.7488/ds/2645 (2019). ↗
  18. McFee, B., Raffel, C., Liang, D., Ellis, D. P., McVicar, M., Battenberg, E., & Nieto, O. librosa: Audio and music signal analysis in python. Proceedings of the 14th python in science conference. Vol. 8, pp. 18-25 (2015). find
  19. Tomar, Suramya. Converting video formats with FFmpeg. Linux journal, 2006 (146),10 (2006). find
  20. Brigham, E. O. The fast Fourier transform and its applications. Prentice-Hall, Inc. (1998). find
  21. Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition 4700-4708 (2017). find
  22. Caruana, R. Multitask learning. Machine learning, 28, 41-75 (1997). find
  23. Crawshaw, M. Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796 (2020). ↗
  24. Zhang, H., Zu, K., Lu, J., Zou, Y., & Meng, D. EPSANet: An efficient pyramid squeeze attention block on convolutional neural network. Proceedings of the Asian Conference on Computer Vision 1161-1177 (2022). find
  25. Huang, G., Liu, S., Van der Maaten, L., & Weinberger, K. Q. Condensenet: An efficient densenet using learned group convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition 2752-2761 (2018). 11/11 find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/song-2023-mpsa-densenet-novel-deep-learning/