Elamur Research⌕
☰
home / papers / 2017 / Pronunciation recognition of English phonemes /\textipa{@}/, /æ/, /\textipa{A}:/ and /\textipa{2}/ using Formants and Mel Frequency Cepstral Coefficients

Pronunciation recognition of English phonemes /\textipa{@}/, /æ/, /\textipa{A}:/ and /\textipa{2}/ using Formants and Mel Frequency Cepstral Coefficients

Paper · 2017 · Keith Y. Patarroyo and Vladimir Vargas-Calderon

Summary

The paper asks how well two classic acoustic feature sets can tell apart four English vowels that learners often confuse: /@/, /æ/, /A:/ and /2/. Spanish and Italian speakers, for example, tend to merge them into one /a/. The motivation is automatic pronunciation assessment for language teaching. The authors took recordings by native speakers from the Vocal Joystick Vowel Corpus. They used short and long, single-level-intonation files at quiet, normal or loud amplitude. For each file they removed silence, normalised amplitude, cut a 40 ms segment and applied a Hamming window. They then extracted the first two formants via Linear Predictive Coding and 13 MFCCs, and trained a Decision Tree Classifier on each feature set. Formants reached about 70% accuracy. MFCCs performed worse on the four confusable vowels (52–56%) but clearly better on distinct vowels (up to 88%). The authors conclude that these standard tools miss the fine distinctions human listeners make. That limits their use as objective pronunciation judges.

Key points

  • Data come from the Vocal Joystick Vowel Corpus (Bilmes et al., 2006). The file count is reported inconsistently: 748 in the abstract, 784 in the methods. After removing formant outliers beyond 1.5σ of each vowel's mean, 642 signals remained.
  • Features were F1 and F2 from LPC, and 13 MFCCs. The MFCCs were projected to 2D with PCA only for visualisation. The classifier in every case was a Decision Tree Classifier.
  • With the two formants, all four vowels were classified at about 70% accuracy. The measured formant means were compared against Hawkins et al., Hunter et al. and Deterding, showing large variation across studies.
  • With MFCCs, the four confusable vowels gave 52% (abstract) or 56% (body text). The body reports 71% after dropping /A:/, which overlaps with /2/, while the abstract says /@/ was dropped.
  • As a control with dissimilar vowels (/2/, /e/, /u/, /o/), MFCCs reached 73%, and 88% once /o/ (mixed with /u/) was removed.

Interesting ideas

  • I1. The authors check a feature set on a deliberately easy contrast set as well as the hard one, so a weak result can be blamed on the task rather than the pipeline.
  • I2. They project the features onto a plane to see which classes overlap, then remove the overlapping class and re-measure to confirm the confusion.
  • I3. They drop per-class outliers beyond 1.5 standard deviations before training, which removes mis-measured or mislabelled samples cheaply.
  • I4. They compare their own measured values against several published reference tables as a sanity check, and treat disagreement between sources as a sign of real population variation.
  • I5. They pair a few interpretable features (two formants) with a transparent classifier, giving a baseline whose decisions can be read directly.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Evaluation could pair each hard benchmark slice with an "easy" control slice, so that a drop in parser or reader accuracy can be read as a real difficulty effect and not a broken component.
  • I2 (tokens): Tagger errors could be diagnosed the same way: find which POS or morphology classes collapse into each other, then retest without one of them to localise the ambiguity.
  • I3 (learn): Silver data agreed by witness parsers could get a similar per-label statistical filter, for example on arc length or attachment-score distributions, to drop suspicious examples before training.
  • I4 (world): Compiled measures and values in the knowledge layer could be cross-checked against multiple sources, recording the spread instead of one number when sources disagree.
  • I5 (tree): A small, inspectable feature-plus-decision-rule baseline for attachment choices would give an explainable reference point against which gparse's gains can be measured and argued.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 7 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works Psychology · 798 works Experimental and Cognitive Psychology · 463 works Signal Processing · 213 works

🧭 Topics 1 English · 1,373 works

Cite

Cite as Keith Y. Patarroyo and Vladimir Vargas-Calderon. 2017. Pronunciation recognition of English phonemes /\textipa{@}/, /æ/, /\textipa{A}:/ and /\textipa{2}/ using Formants and Mel Frequency Cepstral Coefficients. arXiv:1702.07071.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1702.07071 · arXiv 1702.07071

@misc{patarroyo2017pronunciation,
  title = {{Pronunciation recognition of English phonemes /textipa@/, /æ/, /textipaA:/ and /textipa2/ using Formants and Mel Frequency Cepstral Coefficients}},
  author = {Keith Y. Patarroyo and Vladimir Vargas-Calderon},
  year = {2017},
  doi = {10.48550/arxiv.1702.07071},
  eprint = {1702.07071},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1702.07071}
}

References

27 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Baker, J., Li Deng, Glass, J., Khudanpur, S., Chin-hui Lee, Morgan, N., & O’Shaughnessy, D. (2009). Developments and directions in speech recognition and understanding, Part 1 [DSP Education]. IEEE Signal Processing Magazine, 26(3), 75-80. http://dx.doi.org/10.1109/msp.2009.932166 ↗
  2. Bellini, D. (n.d.). Expressive Digital Signal Processing (DSP) package for Python, 2016. Retrieved November 16, 2016, from https://github.com/danilobellini/ audiolazy ↗
  3. Bilmes, J., Wright, R., Xiao, L., Malkin, J. & Kilanski, K. 2006. http://melodi.ee.washington.edu/vj/index.html ↗
  4. Davis, S. & Mermelstein, P. (1980). Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions On Acoustics, Speech, And Signal Processing, 28(4), 357-366. http://dx.doi.org/10.1109/tassp.1980.1163420 ↗
  5. Deng, L. & O’Shaughnessy, D. (2003). Speech processing (1st ed., pp. 19-26). New York: Marcel Dekker. find
  6. Deterding, D. (2006). The North Wind versus a Wolf: short texts for the description and measurement of English pronunciation. Journal Of The International Phonetic Association, 36(02), 187. http://dx.doi.org/10.1017/ s0025100306002544 ↗
  7. Ghaffarvand Mokari, P. & Werner, S. (2016). Perceptual assimilation predicts acquisition of foreign language sounds: The case of Azerbaijani learners’ production and perception of Standard Southern British English vowels. Lingua, 185, 81-95. http://dx.doi.org/10.1016/j.lingua.2016.07.008 ↗
  8. Hawkins, S. & Midgley, J. (2005). Formant frequencies of RP monophthongs in four age groups of speakers. Journal Of The International Phonetic Association, 35(02), 183. http://dx.doi.org/10.1017/s0025100305002124 ↗
  9. Hillenbrand, J., Getty, L., Clark, M., & Wheeler, K. (1995). Acoustic characteristics of American English vowels. The Journal Of The Acoustical Society Of America, 97(5), 3099. http://dx.doi.org/10.1121/1.411872 ↗
  10. Hincks, R. (2003). Speech technologies for pronunciation feedback and evaluation. Recall, 15(01). http://dx.doi.org/10.1017/s0958344003000211 ↗
  11. Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., & Jailty, N. et al. (2012). Deep Neural Networks for Acoustic Modeling in Speech Recognition. Retrieved from http://static.googleusercontent.com/media/research.google.com/ en//pubs/archive/38131.pdf ↗
  12. Hunter, G. & Kebede, H. (2012). Formant frequencies of British English vowels produced by native speakers of Farsi. Société Française d’Acoustique. Retrieved from https://hal.archives-ouvertes.fr/hal-00810580/document ↗
  13. Lee, K. (1988). On large-vocabulary speaker-independent continuous speech recognition. Speech Communication, 7(4), 375-379. http://dx.doi.org/10. 1016/0167-6393(88)90053-2 ↗
  14. Lyons, J. Mel Frequency Cepstral Coefficient (MFCC) tutorial, Practical Cryptography. Retrieved October 15, 2016, from http://practicalcryptography.com/miscellaneous/machine-learning/guide-mel-frequency-cepstral-coefficients-mfccs/ ↗
  15. Martin, I. & Jackson, C. (2016). Pronunciation Training Facilitates the Learning and Retention of L2 Grammatical Structures. Foreign Language Annals, 49(4), 658-676. http://dx.doi.org/10.1111/flan.12224 ↗
  16. Mirzaei, K., Gowhary, H., Azizifar, A., & Esmaeili, Z. (2015). Comparing the Phonological Performance of Kurdish and Persian EFL Learners in Pronunciation of English Vowels. Procedia - Social And Behavioral Sciences, 199, 387-393. http://dx.doi.org/10.1016/j.sbspro.2015.07.523 ↗
  17. Neri, A., Mich, O., Gerosa, M., & Giuliani, D. (2008). The effectiveness of computer assisted pronunciation training for foreign language learning by children. Computer Assisted Language Learning, 21(5), 393-408. http://dx.doi.org/10. 1080/09588220802447651 ↗
  18. Oppenheim, J. & Magnasco, M. (2013). Human Time-Frequency Acuity Beats the Fourier Uncertainty Principle. Physical Review Letters, 110(4). http://dx.doi.org/10.1103/physrevlett.110.044301 ↗
  19. O’Grady, W. & Dobrovolosky, M. (1997). Contemporary Linguistics (3rd ed., pp. 40-87). Boston: Bedford/St. Martin’s. find
  20. Prica, B. & Ilić, S. (2010). Recognition of vowels in continuous speech by using formants. Facta Universitatis - Series: Electronics And Energetics, 23(3), 379-393. http://dx.doi.org/10.2298/fuee1003379p ↗
  21. Rasilo, H. & Räsänen, O. (2017). An online model for vowel imitation learning. Speech Communication, 86, 1-23. http://dx.doi.org/10.1016/j.specom. 2016.10.010 ↗
  22. Rosen, S. & Howell, P. (2011). Signals and systems for speech and hearing (2nd ed., p. 163). London: Academic Press. find
  23. Sethna, J. (2006). Statistical mechanics: Entropy, Order Parameters and Complexity (1st ed., p. 299). Oxford: Oxford University Press. find
  24. Tan, P., Steinbach, M., & Kumar, V. (2005). Introduction to Data Mining (1st ed., p. 150). Boston: Pearson. find
  25. Wei, S., Hu, G., Hu, Y., & Wang, R. (2009). A new method for mispronunciation detection using Support Vector Machine based on Pronunciation Space Models. Speech Communication, 51(10), 896-905. http://dx.doi.org/10.1016/ j.specom.2009.03.004 ↗
  26. Wellekens, C. (1998). Introduction to Speech Recognition Using Neural Networks. Bruges: ESANN. Retrieved from https://www.elen.ucl.ac.be/Proceedings/ esann/esannpdf/es1998-456.pdf ↗
  27. Zegers, P. (1998). Speech Recognition Using Neural Networks (Master of Science with a major in Electrical Engineering). University of Arizona. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/patarroyo-2017-pronunciation-recognition-english-phonemes-textipa/