Elamur Research⌕
☰
home / papers / 2022 / Automated detection of pronunciation errors in non-native English speech employing deep learning

Automated detection of pronunciation errors in non-native English speech employing deep learning

Paper · 2022 · Daniel Korzekwa

Summary

This doctoral dissertation, an "Implementation Doctorate" run jointly by Gdańsk University of Technology and Amazon TTS Research, addresses automatic pronunciation assessment inside Computer-Assisted Pronunciation Training (CAPT) for non-native (L2) English learners. Existing detectors are not very reliable, with roughly 60% precision at 40–80% recall. False alarms can discourage learners, and missed errors slow their progress.

The central difficulty is the scarcity of annotated mispronounced speech. The author therefore recasts error detection as the task of producing artificial L2-style mispronunciations, which supplies as much training data as needed. Three generation routes are studied: P2P, T2S and S2S. A second contribution is an end-to-end multi-task detector that avoids aligning canonical and recognized phoneme strings. Further chapters cover uncertainty modeling and lexical stress errors.

The best methods raise AUC from 0.528 to 0.749, a 41% gain over the prior state of the art. The detectors were deployed at Amazon to catch errors in synthetic speech, and the techniques were also applied to detecting and reconstructing dysarthric speech.

Key points

  • Baseline problem: earlier CAPT systems reach about 60% precision at 40–80% recall. The proposed methods lift AUC from 0.528 to 0.749, a relative improvement of 41%.
  • Errors are treated at several granularities: phoneme, word and sentence. Two example types are phoneme substitution (/eh/ spoken as /ey/ in "I said") and lexical stress placed on the wrong syllable (as in "remind").
  • Three ways of generating mispronounced speech are compared:
    • P2P, which perturbs phonemes;
    • T2S, which goes from text to speech;
    • S2S, which converts speech to speech.
  • One chapter models pronunciation uncertainty with a phoneme recognizer, a pronunciation model and an error detector. Another detects lexical stress errors using attention and augmentation with synthetic speech.
  • Outside L2 learning, the models check synthetic speech at Amazon to speed up TTS research. They were also extended to dysarthria detection, interpretable modeling of dysarthric patterns, and reconstruction of dysarthric speech.

Interesting ideas

  • I1. When labelled errors are rare, the author corrupts correct inputs in controlled ways to create unlimited labelled negatives, which turns a scarce-data detection task into a generation task.
  • I2. Rather than comparing output against one canonical form, the detector models how much variation correct native speech actually allows. It flags only deviations that fall outside that range.
  • I3. An error detector built for learners was reused as an automatic quality check on a generator (TTS), which cut down on manual listening during research.
  • I4. Feedback granularity is treated as a design choice: phoneme-level reports may be precise but too complex for a learner. Word-level output can be more useful, especially since false alarms carry a motivational cost.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Mova could produce deliberately broken sentences or trees with known faults (wrong attachment, swapped roles) to train and test its self-induced repair rules, with exact labels for each fault.
  • I2 (tree): disagreements among witness parsers could be checked against a learned spread of acceptable analyses, so that harmless UD dialect differences are not marked as parse errors.
  • I3 (back): Mova's own readers could re-parse text generated by the text → graph → text compressor and flag places where the meaning graph drifts from the original.
  • I4 (answer): explanations to students could be pitched at the word or clause level, and the system could favor precision over recall when flagging mistakes in a student's answer.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPsychology · 798 worksExperimental and Cognitive Psychology · 463 worksSignal Processing · 213 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Daniel Korzekwa. 2022. Automated detection of pronunciation errors in non-native English speech employing deep learning. arXiv:2209.06265.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2209.06265 · arXiv 2209.06265

@misc{korzekwa2022automated,
  title = {{Automated detection of pronunciation errors in non-native English speech employing deep learning}},
  author = {Daniel Korzekwa},
  year = {2022},
  doi = {10.48550/arxiv.2209.06265},
  eprint = {2209.06265},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2209.06265}
}

References

206 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. ## 1. Abdi, H. (2007). “Binomial distribution: Binomial and sign tests”. In: Encyclopedia of measurement and statistics 1. find
  2. Ahmad Khan, Z., P. Green, S. Creer, and S. Cunningham (2011). Reconstructing the voice of an individual following laryngectomy. DOI : 10.3109/07434618.2010.545078. ↗
  3. Ai, R. (2015). “Automatic pronunciation error detection and feedback generation for call applications”. In: International Conference on Learning and Collaboration Technologies. Springer, pp. 175–186. find
  4. Ali, J., R. Khan, N. Ahmad, and I. Maqsood (2012). “Random forests and decision trees”. In: International Journal of Computer Science Issues (IJCSI) 9.5, p. 272. find
  5. Alzheimersresearchuk (2015). “One in three people born in 2015 will develop dementia, new analysis shows”. In. find
  6. Arpabet, W. (2022). Arpabebet, accessed on June 2022. URL : https://en.wikipedia.org/wiki/ARPABET. ↗
  7. ASHA (2018). “The American Speech-Language-Hearing Association (ASHA) - Dysarthria”. In. 8. — (2022). “American Speech-Language-Hearing Association (ASHA), accessed on June 2022”. In: URL : https://www.asha.org. ↗
  8. Asrifan, A., C. T. Zita, K. Vargheese, T. Syamsu, and M. Amir (2020). “THE EFFECTS OF CALL (COMPUTER ASSISTED LANGUAGE LEARNING) TO-WARD THE STUDENTS’ENGLISH ACHIEVEMENT AND ATTITUDE”. In: Journal of advanced English studies 3.2, pp. 94–106. find
  9. Atwell, E., P. Howarth, and D. Souter (2003). “The ISLE corpus: Italian and German spoken learner’s English”. In: ICAME Journal: Intl. Computer Archive of Modern and Medieval English Journal 27, pp. 5–18. find
  10. Badenhorst, J. and F. De Wet (2017). “The limitations of data perturbation for ASR of learner data in under-resourced languages”. In: 2017 Pattern Recognition Association of South Africa and Robotics and Mechatronics (PRASA-RobMech). IEEE, pp. 44–49. find
  11. Bahdanau, D., J. Chorowski, D. Serdyuk, P. Brakel, and Y. Bengio (2016). “End-to-end attention-based large vocabulary speech recognition”. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp. 4945–4949. find
  12. Banovic, S., L. Zunic, and O. Sinanovic (2018). “Communication Difficulties as a Result of Dementia”. In: Materia Socio Medica 30.2, p. 221. ISSN : 1512-7680. DOI : 10.5455/msm.2018.30.221- 224. URL : https://www.ejmanager.com/fulltextpdf.php?mno=302643414. ↗
  13. Banovic, S., L. J. Zunic, and O. Sinanovic (2018). “Communication difficulties as a result of dementia”. In: Materia socio-medica 30.3, p. 221. ↗
  14. Beringer, G., D. Korzekwa, A. Sanchez, B. Wang, and J. Lorenzo-Trueba (2020). “Extending Goodness of Pronunciation to generate mispronunciationhypothe-ses for pronunciation assessment in L2-English”. In: Amazon Machine Learning Conference, Seattle. find
  15. Bilinski, P., T. Merritt, A. Ezzerg, K. Pokora, S. Cygert, K. Yanagisawa, R. Barra-Chicote, and D. Korzekwa (2022). “Creating New Voices using Normalizing Flows”. In: accepted to Interspeech 2022. find
  16. Bishop, C. M. (2006). “Pattern recognition”. In: Machine learning 128.9. 19. find
  17. Boersma, P. (2006). “Praat: doing phonetics by computer”. In: http://www.praat.org/. ↗
  18. Botchkarev, A. (2018). “Performance metrics (error measures) in machine learning regression, forecasting and prognostics: Properties and typology”. In: arXiv preprint arXiv:1809.03006. ↗
  19. Bowman, S. R., L. Vilnis, O. Vinyals, A. M. Dai, R. Józefowicz, and S. Bengio (2015). “Generating Sentences from a Continuous Space”. In: CoRR abs/1511.0. arXiv: 1511.06349. ↗
  20. Brady, M. C., H. Kelly, J. Godwin, P. Enderby, and P. Campbell (2016). “Speech and language therapy for aphasia following stroke”. In: Cochrane database of systematic reviews 6. ↗
  21. Carmichael, J., V. Wan, and P. Green (2008). “Combining neural network and rule-based systems for dysarthria diagnosis”. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. find
  22. Chaudhari, S., V. Mithal, G. Polatkan, and R. Ramanath (2021). “An attentive survey of attention models”. In: ACM Transactions on Intelligent Systems and Technology (TIST) 12.5, pp. 1–32. ↗
  23. Chen, J.-Y. and L. Wang (2010). “Automatic lexical stress detection for Chinese learners’ of English”. In: 2010 7th Intl. Symposium on Chinese Spoken Language Processing. IEEE, pp. 407–411. find
  24. Chen, N. and Q. He (2007). “Using nonlinear features in automatic English lexical stress detection”. In: 2007 Intl. Conference on Computational Intelligence and Security Workshops (CISW 2007). IEEE, pp. 328–332. ↗
  25. Chen, T., M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, and Z. Zhang (2015). “MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems”. In: CoRR abs/1512.01274. arXiv: 1512.01274. URL : http://arxiv.org/abs/1512.01274. ↗
  26. Cheng, S., Z. Liu, L. Li, Z. Tang, D. Wang, and T. F. Zheng (2020). “ASR-Free Pronunciation Assessment”. In: arXiv preprint arXiv:2005.11902. ↗
  27. Bengio (2014). “Learning Phrase Representations using {RNN} Encoder-Decoder for Statistical Machine Translation”. In: CoRR abs/1406.1. arXiv: 1406.1078. ↗
  28. Cho, K., B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio (2014). “Learning phrase representations using RNN encoder-decoder for statistical machine translation”. In: arXiv preprint arXiv:1406.1078. ↗
  29. Chorowski, J., D. Bahdanau, K. Cho, and Y. Bengio (2014). “End-to-end continuous speech recognition using attention-based recurrent NN: First results”. In: arXiv preprint arXiv:1412.1602. ↗
  30. Chorowski, J., R. J. Weiss, S. Bengio, and A. van den Oord (2019). “Unsupervised speech representation learning using wavenet autoencoders”. In: IEEE/ACM transactions on audio, speech, and language processing 27.12, pp. 2041–2053. ↗
  31. Chorowski, J. K., D. Bahdanau, D. Serdyuk, K. Cho, and Y. Bengio (2015). “Attention-based models for speech recognition”. In: Advances in neural information processing systems, pp. 577–585. ↗
  32. Cuny, M. L., M. Pallone, H. Piana, N. Boddaert, C. Sainte-Rose, L. Vaivre-Douret, P. Piolino, and S. Puget (2017). “Neuropsychological improvement after posterior fossa arachnoid cyst drainage”. In: Child’s Nervous System. ISSN : 14330350. DOI : 10.1007/s00381-016-3285-x.36. — (2017). “Neuropsychological improvement after posterior fossa arachnoid cyst drainage”. In: Child’s Nervous System 33.1, pp. 135–141. ↗
  33. Czyzewski, A., B. Kostek, P. Bratoszewski, J. Kotus, and M. Szykulski (2017). “An audio-visual corpus for multimodal automatic speech recognition”. In: Journal of Intelligent Information Systems 49.2, pp. 167–192. ↗
  34. Damianou, A. and N. D. Lawrence (2013). “Deep gaussian processes”. In: Artificial intelligence and statistics. PMLR, pp. 207–215. ↗
  35. Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge university press. ↗
  36. Denham, K. and A. Lobeck (2012). Linguistics for everyone: An introduction. Cengage Learning. find
  37. Doersch, C. (2016). Tutorial on Variational Autoencoders. find
  38. Drugman, T., P. Alku, A. Alwan, and B. Yegnanarayana (Sept. 2014). “Glottal Source Processing: from Analysis to Applications”. In: Computer Speech and Language 28. DOI : 10.1016/j.csl.2014.03.003. ↗
  39. Duan, R., T. Kawahara, M. Dantsuji, and H. Nanjo (2019). “Cross-lingual transfer learning of non-native acoustic modeling for pronunciation error detection and diagnosis”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 28, pp. 391–401. ↗
  40. Duvenaud, D. (2014). “The Kernel cookbook: Advice on covariance functions, accessed on June 2022”. In: URL : https://www.cs.toronto.edu/~duvenaud/cookbook/. ↗
  41. Eberhard, D. M., G. F. Simons, and C. D. Fennig (2021). “Ethnologue: Languages of the World. Twenty-fourth edition. Dallas”. In: URL : https://www.ethnologue.com/ethnoblog/gary-simons/welcome-24th-edition. ↗
  42. EF-Education-First (2020). EF English Proficiency Index. URL : https://www.ef.pl/assetscdn/WIBIwq6RdJvcD9bc8RMd/legacy/__/~/media/centralefcom/ epi/downloads/full-reports/v10/ef-epi-2020-english.pdf. ↗
  43. Eklund, V.-V. (2019). “Data Augmentation Techniques for Robust Audio Analysis”. MA thesis. Tampere University. find
  44. Elias, I., H. Zen, J. Shen, Y. Zhang, Y. Jia, R. Weiss, and Y. Wu (2020). “Parallel Tacotron: Non-Autoregressive and Controllable TTS”. In: arXiv preprint arXiv:2010.11439. ↗
  45. Erickson, N., J. Mueller, A. Shirkov, H. Zhang, P. Larroy, M. Li, and A. Smola (2020). “AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data”. In: arXiv preprint arXiv:2003.06505. ↗
  46. Ezzerg, A., A. Gabrys, B. Putrycz, D. Korzekwa, D. Saez-Trigueros, D. McHardy, K. Pokora, J. Lachowicz, J. Lorenzo-Trueba, and V. Klimkov (2021). “Enhancing audio quality for expressive Neural Text-to-Speech”. In: Proc. 11th ISCA Speech Synthesis Workshop (SSW 11), pp. 78–83. DOI : 10.21437/SSW.2021-14. ↗
  47. Falk, T. H., W. Y. Chan, and F. Shein (2012). “Characterization of atypical vocal source excitation, temporal dynamics and prosody for objective measurement of dysarthric word intelligibility”. In: Speech Communication. ISSN : 01676393. DOI : 10.1016/j.specom.2011.03.007. ↗
  48. Farrajota, L., C. Maruta, J. Maroco, I. P. Martins, M. Guerreiro, and A. De Mendonca (2012). “Speech therapy in primary progressive aphasia: a pilot study”. In: Dementia and geriatric cognitive disorders extra 2.1, pp. 321–331. ↗
  49. Fazel, A., W. Yang, Y. Liu, R. Barra-Chicote, Y. Meng, R. Maas, and J. Droppo (2021). “SynthASR: Unlocking Synthetic Data for Speech Recognition”. In: Proc. find
  50. Interspeech 2021, pp. 896–900. DOI : 10.21437/Interspeech.2021-1882. ↗
  51. Ferrer, L., H. Bratt, C. Richey, H. Franco, V. Abrash, and K. Precoda (2015). “Classification of lexical stress using spectral and prosodic features for computer-assisted language learning systems”. In: Speech Communication 69, pp. 31–45. 55. ↗
  52. Field, J. (2005). “Intelligibility and the listener: The role of lexical stress”. In: TESOL quarterly 39.3, pp. 399–423. DOI : 10.2307/3588487. ↗
  53. Fouz-González, J. (2015). “Trends and directions in computer-assisted pronunciation training”. In: Investigating English Pronunciation, pp. 314–342. ↗
  54. Franco, H., L. Neumeyer, Y. Kim, and O. Ronen (1997). “Automatic pronunciation scoring for language instruction”. In: 1997 IEEE international conference on acoustics, speech, and signal processing. Vol. 2. IEEE, pp. 1471–1474. find
  55. Fu, K., S. Gao, K. Wang, W. Li, X. Tian, and Z. Ma (2022). “Improving Non-native Word-level Pronunciation Scoring with Phone-level Mixup Data Augmentation and Multi-source Information”. In: arXiv preprint arXiv:2203.01826, submitted to INTERSPEECH 2022. DOI : 10.48550/ARXIV.2203.01826. ↗
  56. Fu, K., J. Lin, D. Ke, Y. Xie, J. Zhang, and B. Lin (2021). “A Full Text-Dependent End to End Mispronunciation Detection and Diagnosis with Easy Data Augmentation Techniques”. In: arXiv preprint arXiv:2104.08428. ↗
  57. Gabryś, A., Y. Jiao, V. Klimkov, D. Korzekwa, and R. Barra-Chicote (2021). “Improving the Expressiveness of Neural Vocoding with Non-Affine Normalizing Flows”. In: Proc. Interspeech 2021, pp. 1679–1683. DOI : 10.21437/Interspeech.2021-1555. ↗
  58. Garofolo, J. S., L. F. Lamel, W. M. Fisher, J. G. Fiscus, and D. S. Pallett (1993). “DARPA TIMIT acoustic-phonetic continous speech corpus CD-ROM. NIST speech disc 1-1.1”. In: STIN 93, p. 27403. find
  59. Gillespie, S., Y. Y. Logan, E. Moore, J. Laures-Gore, S. Russell, and R. Patel (2017). “Cross-database models for the classification of dysarthria presence”. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. DOI : 10.21437/Interspeech.2017-216. ↗
  60. Glorot, X. and Y. Bengio (2010). “Understanding the difficulty of training deep feedforward neural networks.” In: AISTATS. Ed. by Y. W. Teh and D. M. Titterington. Vol. 9. JMLR Proceedings. JMLR.org, pp. 249–256. find
  61. Golonka, E. M., A. R. Bowles, V. M. Frank, D. L. Richardson, and S. Freynik (2014). “Technologies for foreign language learning: A review of technology types and their effectiveness”. In: Computer assisted language learning 27.1, pp. 70– 105. find
  62. Gong, Y., Z. Chen, I.-H. Chu, P. Chang, and J. Glass (2022). “Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment”. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7262–7266. DOI : 10.1109/ find
  63. Goodfellow, I., Y. Bengio, and A. Courville (2016). Deep learning. MIT press. find
  64. Graves, A., A. Mohamed, and G. Hinton (2013). “Speech recognition with deep recurrent neural networks”. In: 2013 IEEE Intl. conference on acoustics, speech and signal processing. IEEE, pp. 6645–6649. ↗
  65. Graves, A. (2012). “Connectionist temporal classification”. In: Supervised Sequence Labelling with Recurrent Neural Networks. Springer, pp. 61–93. ↗
  66. Griffin, D. W. and J. S. Lim (1984). “Signal Estimation from Modified Short-Time Fourier Transform”. In: IEEE Transactions on Acoustics, Speech, and Signal Processing. ISSN : 00963518. DOI : 10.1109/TASSP.1984.1164317. ↗
  67. Wang, J. Cai, et al. (2018). “Recent advances in convolutional neural networks”. In: Pattern Recognition 77, pp. 354–377. find
  68. Guo, J. et al. (2020). “GluonCV and GluonNLP: Deep Learning in Computer Vision and Natural Language Processing.” In: Journal of Machine Learning Research 21.23, pp. 1–7. ↗
  69. Harrison, A. M., W.-K. Lo, X.-j. Qian, and H. Meng (2009). “Implementation of an extended recognition network for mispronunciation detection and diagnosis in computer-assisted pronunciation training”. In: Intl. Workshop on Speech and Language Technology in Education. find
  70. Heck, L. P., Y. Konig, M. K. Sönmez, and M. Weintraub (2000). “Robustness to telephone handset distortion in speaker recognition by discriminative feature design”. In: Speech Communication 31.2-3, pp. 181–192. DOI : 10.1016/S0167-6393(99)00077-1. ↗
  71. Hieke, A. E. (1984). “Linking as a marker of fluent speech”. In: Language and Speech 27.4, pp. 343–354. ↗
  72. Hines, A., E. Gillen, D. Kelly, J. Skoglund, A. Kokaram, and N. Harte (2015). “ViSQOLAudio: An objective audio quality metric for low bitrate codecs”. In: The Journal of the Acoustical Society of America 137.6, EL449–EL455. ↗
  73. Hossin, M. and M. N. Sulaiman (2015). “A review on evaluation metrics for data classification evaluations”. In: International journal of data mining & knowledge management process 5.2, p. 1. ↗
  74. Hsu, W.-N., Y. Zhang, and J. R. Glass (2017). “Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data”. In: CoRR abs/1709.0. arXiv: 1709.07902. ↗
  75. Hu, Z., Z. Yang, X. Liang, R. Salakhutdinov, and E. P. Xing (2017). “Controllable Text Generation”. In: CoRR abs/1703.0. arXiv: 1703.00955. ↗
  76. Huang, G., A. Gorin, J.-L. Gauvain, and L. Lamel (2016). “Machine translation based data augmentation for cantonese keyword spotting”. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 6020–6024. find
  77. Huang, W.-C., K. Kobayashi, Y.-H. Peng, C.-F. Liu, Y. Tsao, H.-M. Wang, and T. Toda (2021). “A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion”. In: arXiv preprint arXiv:2106.01415. ↗
  78. Huybrechts, G., T. Merritt, G. Comini, B. Perz, R. Shah, and J. Lorenzo-Trueba (2021). “Low-resource expressive text-to-speech using data augmentation”. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 6593–6597. find
  79. Jia, Y., R. J. Weiss, F. Biadsy, W. Macherey, M. Johnson, Z. Chen, and Y. Wu (2019). “Direct speech-to-speech translation with a sequence-to-sequence model”. In: arXiv preprint arXiv:1904.06037. ↗
  80. Jiao, Y., A. Gabryś, G. Tinchev, B. Putrycz, D. Korzekwa, and V. Klimkov (2021). find
  81. “Universal neural vocoding with parallel wavenet”. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 6044–6048. DOI : 10.1109/ICASSP39728.2021.9414444. ↗
  82. Johnston, A. B. and D. C. Burnett (2012). WebRTC: APIs and RTCWEB Protocols of the HTML5 Real-Time Web. USA: Digital Codex LLC. ISBN : 0985978805, 9780985978808. find
  83. Jordan, M. I., Z. Ghahramani, T. S. Jaakkola, and L. K. Saul (1999). “An introduction to variational methods for graphical models”. In: Machine learning 37.2, pp. 183–233. ↗
  84. Jung, Y.-J., S.-C. Rhee, et al. (2018). “Acoustic analysis of English lexical stress produced by Korean, Japanese and Taiwanese-Chinese speakers”. In: Phonetics and Speech Sciences 10.1, pp. 15–22. ↗
  85. Jurafsky, D. and J. H. Martin (2009). Speech and Language Processing (2Nd Edition). Upper Saddle River, NJ, USA: Prentice-Hall, Inc. ISBN : 0131873210. find
  86. Kim, H., M. Hasegawa-Johnson, A. Perlman, J. Gunderson, T. Huang, K. Watkin, and S. Frame (2008). “Dysarthric Speech Database for Universal Access Research”. In: INTERSPEECH. ISSN : 19909772. find
  87. Kobyzev, I., S. Prince, and M. Brubaker (2020). “Normalizing flows: An introduction and review of current methods”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. ↗
  88. Koller, D. and N. Friedman (2009). Probabilistic graphical models: principles and techniques. MIT press. find
  89. Komatsu, S. and M. Sasayama (2019). “Speech Error Detection depending on Linguistic Units”. In: Proceedings of the 2019 3rd International Conference on Natural Language Processing and Information Retrieval, pp. 75–79. find
  90. Kominek, J. and A. W. Black (2004). “The CMU Arctic speech databases”. In: Fifth ISCA workshop on speech synthesis. find
  91. Korzekwa, D., R. Barra-Chicote, B. Kostek, T. Drugman, and M. Lajszczak (2019). “Interpretable Deep Learning Model for the Detection and Reconstruction of Dysarthric Speech”. In: Proc. Interspeech 2019, pp. 3890–3894. DOI : 10.21437/ find
  92. Korzekwa, D., R. Barra-Chicote, S. Zaporowski, G. Beringer, J. Lorenzo-Trueba, A. Serafinowicz, J. Droppo, T. Drugman, and B. Kostek (2021). “Detection of Lexical Stress Errors in Non-Native (L2) English with Data Augmentation and Attention”. In: Proc. Interspeech 2021, pp. 3915–3919. DOI : 10.21437/Interspeech.2021-86. ↗
  93. Korzekwa, D. and B. Kostek (2019). “Deep learning model for automated assessment of lexical stress of non-native English speakers”. In: The Journal of the Acoustical Society of America 146.4, pp. 2956–2957. DOI : 10.1121/1.5137270. ↗
  94. Korzekwa, D., J. Lorenzo-Trueba, T. Drugman, S. Calamaro, and B. Kostek (2021). “Weakly-Supervised Word-Level Pronunciation Error Detection in Non-Native English Speech”. In: Proc. Interspeech 2021, pp. 4408–4412. DOI : 10.21437/
  95. Korzekwa, D., J. Lorenzo-Trueba, T. Drugman, and B. Kostek (2022). “Computer-assisted Pronunciation Training - Speech synthesis is almost all you need”. In: accepted for publication in Speech Communication Journal on June 17 ‘2022, in print. ↗
  96. Korzekwa, D., J. Lorenzo-Trueba, S. Zaporowski, S. Calamaro, T. Drugman, and B. Kostek (2021). “Mispronunciation Detection in Non-Native (L2) English with Uncertainty Modeling”. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 7738–7742. DOI : 10.1109/ICASSP39728.2021.9413953. ↗
  97. Koyuncu, E., P. Çam, N. Altınok, D. E. Çallı, T. Y. Duman, and N. Özgirgin (2016). “Speech and language therapy for aphasia following subacute stroke”. In: Neural Regeneration Research 11.10, p. 1591. 100. ↗
  98. Krishna, G. (2018). “Excitation Source Analysis of Dysarthric Speech for Early Stage Detection of Dysarthria”. In: WSPD. 101. find
  99. Kroll, J. F. and P. E. Dussias (2017). “The benefits of multilingualism to the personal and professional development of residents of the US”. In: Foreign Language Annals 50.2, pp. 248–259. 102. ↗
  100. Lake, B. M., R. Salakhutdinov, and J. B. Tenenbaum (2015). “Human-level concept learning through probabilistic program induction”. In: Science 350.6266, pp. 1332–1338. 103. ↗
  101. Lansford, K. L. and J. M. Liss (2014). “Vowel Acoustics in Dysarthria: Speech Disorder Diagnosis and Classification”. In: Journal of Speech Language and Hearing Research. ISSN : 1092-4388. DOI : 10.1044/1092- 4388(2013 /12- 0262). arXiv: NIHMS150003. 104. ↗
  102. Latorre, J., J. Lachowicz, J. Lorenzo-Trueba, T. Merritt, T. Drugman, S. Ronanki, and V. Klimkov (2019). “Effect of data reduction on sequence-to-sequence neural tts”. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 7075–7079. 105. find
  103. Latorre, J., J. Lachowicz, J. Lorenzo-Trueba, T. Merritt, T. Drugman, S. Ronanki, and K. Viacheslav (2018). “Effect of data reduction on sequence-to-sequence neural {TTS}”. In: CoRR abs/1811.0. arXiv: 1811.06315. 106. ↗
  104. LeCun, Y., Y. Bengio, and G. Hinton (2015). “Deep learning”. In: nature 521.7553, pp. 436–444. 107. find
  105. Lee, A. et al. (2016). “Language-independent methods for computer-assisted pronunciation training”. PhD thesis. Massachusetts Institute of Technology. 108. find
  106. Lee, A. and J. R. Glass (2013). “Pronunciation assessment via a comparison-based system”. In: SLaTE. 109. find
  107. Lee, Y.-G. and S.-Y. Kim (2008). “Introduction to statistics”. In: Yulgokbooks, Korea, pp. 342–351. 110. find
  108. Lepage, A. and M. G. Busà (2014). “Intelligibility of English L2: The effects of incorrect word stress placement and incorrect vowel reduction in the speech of French and Italian learners of English”. In: Proceedings of the International Symposium on the Acquisition of Second Language Speech Concordia Working Papers in Applied Linguistics. Vol. 5. 2014, pp. 387–400. 111. find
  109. Leung, W.-K., X. Liu, and H. Meng (2019). “CNN-RNN-CTC based end-to-end mispronunciation detection and diagnosis”. In: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 8132–8136. 112. find
  110. Levy, M. and G. Stockwell (2013). CALL dimensions: Options and issues in computer-assisted language learning. Routledge. 113. find
  111. Li, H., S. Huang, S. Wang, and B. Xu (2011). “Context-Dependent Duration Modeling with Backoff Strategy and Look-Up Tables for Pronunciation Assessment and Mispronunciation Detection”. In: INTERSPEECH 2011, 12th Annual Conference of the International Speech Communication Association, Florence, Italy, August 27-31, 2011. ISCA, pp. 1133–1136. 114. find
  112. Li, K., S. Mao, X. Li, Z. Wu, and H. Meng (2018). “Automatic lexical stress and pitch accent detection for L2 English speech using multi-distribution deep neural networks”. In: Speech Communication 96, pp. 28–36. 115. ↗
  113. Li, K., X. Qian, S. Kang, and H. Meng (2013). “Lexical stress detection for L2 find
  114. English speech using deep belief networks.” In: Interspeech, pp. 1811–1815. 116. find
  115. Li, K., X. Qian, and H. Meng (2016). “Mispronunciation detection and diagnosis in l2 English speech using multidistribution deep neural networks”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 25.1, pp. 193– 207. 117. ↗
  116. Lin, B. and L. Wang (2021). “Deep Feature Transfer Learning for Automatic Pronunciation Assessment”. In: Proc. Interspeech 2021, pp. 4438–4442. DOI : 10.21437/Interspeech.2021-931.118. ↗
  117. Lorenzo-Trueba, J., T. Drugman, J. Latorre, T. Merritt, B. Putrycz, R. Barra-Chicote, A. Moinet, and V. Aggarwal (2018). “Towards achieving robust universal neural vocoding”. In: arXiv preprint arXiv:1811.06292. 119. ↗
  118. Marcus, G. (2018). “Deep learning: A critical appraisal”. In: arXiv preprint arXiv:1801.00631. 120. ↗
  119. Mathieu, E., T. Rainforth, N. Siddharth, and Y. W. Teh (2018). “Disentangling Disentanglement in Variational Auto-Encoders”. In: arXiv: 1812.02833. 121. ↗
  120. McAuliffe, M., M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger (2017). “Montreal Forced Aligner: Trainable Text-Speech Alignment Using Kaldi.” In: Interspeech. Vol. 2017, pp. 498–502. 122. find
  121. Mehri Kamrood, A., M. Davoudi, S. Ghaniabadi, and S. M. R. Amirian (2019). “Diagnosing L2 learners’ development through online computerized dynamic assessment”. In: Computer Assisted Language Learning, pp. 1–30. 123. find
  122. Merritt, T., A. Ezzerg, P. Biliński, M. Proszewska, K. Pokora, R. Barra-Chicote, and D. Korzekwa (2022). “Text-Free Non-Parallel Many-To-Many Voice Conversion Using Normalising Flow”. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6782–6786. DOI : 10.1109/ICASSP43922.2022.9746368.124. ↗
  123. Merritt, T., B. Putrycz, A. Nadolski, T. Ye, D. Korzekwa, W. Dolecki, T. Drugman, V. Klimkov, A. Moinet, A. Breen, et al. (2018). “Comprehensive evaluation of statistical speech waveform synthesis”. In: 2018 IEEE Spoken Language Technology Workshop (SLT). IEEE, pp. 325–331. 125. find
  124. Meyes, R., M. Lu, C. W. de Puiseau, and T. Meisen (2019). “Ablation studies in artificial neural networks”. In: arXiv preprint arXiv:1901.08644. 126. Michel, J.-B., Y. K. Shen, A. P. Aiden, A. Veres, M. K. Gray, J. P. Pickett, D. ↗
  125. Hoiberg, D. Clancy, P. Norvig, J. Orwant, et al. (2011). “Quantitative analysis of culture using millions of digitized books”. In: science 331.6014, pp. 176–182. 127. find
  126. Minematsu, N. (2004). “Pronunciation assessment based upon the phonological distortions observed in language learners’ utterances”. In: INTERSPEECH. 128. find
  127. Minka, T. P. (2013). “Expectation propagation for approximate Bayesian inference”. In: arXiv preprint arXiv:1301.2294. 129. ↗
  128. Modeltalker (n.d.). www.modeltalker.com.130. Moon, T. K. (1996). “The expectation-maximization algorithm”. In: IEEE Signal processing magazine 13.6, pp. 47–60. DOI : 10.1109/79.543975.131. ↗
  129. Mu, Z., X. Yang, and Y. Dong (2021). “Review of end-to-end speech synthesis technology based on deep learning”. In: arXiv preprint arXiv:2104.09995. DOI : 10.48550/ARXIV.2104.09995.132. ↗
  130. Murphy, K. P. (2012). Machine learning: a probabilistic perspective. MIT press. 133. find
  131. Narendra, N. P. and P. Alku (2018). “Dysarthric Speech Classification Using Glottal Features Computed from Non-words, Words and Sentences”. In: Interspeech 2018, 19th Annual Conference of the International Speech Communication Association, Hyderabad, India, 2-6 September 2018. Ed. by B. Yegnanarayana. ISCA, pp. 3403–3407. DOI : 10.21437/Interspeech.2018-1059.134. ↗
  132. Needleman, S. B. and C. D. Wunsch (1970). “A general method applicable to the search for similarities in the amino acid sequence of two proteins”. In: Journal of molecular biology 48.3, pp. 443–453. 135. ↗
  133. Neri, A., O. Mich, M. Gerosa, and D. Giuliani (2008). “The effectiveness of computer assisted pronunciation training for foreign language learning by children”. In: Computer Assisted Language Learning 21.5, pp. 393–408. 136. ↗
  134. Nicolao, M., A. V. Beeston, and T. Hain (2015). “Automatic assessment of English learner pronunciation using discriminative classifiers”. In: 2015 IEEE Intl. Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 5351– 5355. 137. find
  135. Nicolao, M., H. Christensen, S. Cunningham, P. Green, and T. Hain (2016). “A framework for collecting realistic recordings of dysarthric speech - The home-Service corpus”. In: Proceedings of the 10th International Conference on Language Resources and Evaluation, LREC 2016. ISBN : 9782951740891. 138. find
  136. Oneata, D. and H. Cucu (2022). “Improving Multimodal Speech Recognition by Data Augmentation and Speech Representations”. In: arXiv preprint arXiv:2204.13206. 139. Oord, A., Y. Li, I. Babuschkin, K. Simonyan, O. Vinyals, K. Kavukcuoglu, G. ↗
  137. Driessche, E. Lockhart, L. Cobo, F. Stimberg, et al. (2018). “Parallel wavenet: Fast high-fidelity speech synthesis”. In: International conference on machine learning. PMLR, pp. 3918–3926. 140. find
  138. Ore, Ø. (2017). Cardano: The gambling scholar. Vol. 5063. Princeton University Press. 141. find
  139. Paleyes, A., M. Pullin, M. Mahsereci, N. Lawrence, and J. Gonzalez (2019). “Emulation of physical processes with emukit”. In: Second Workshop on Machine Learning and the Physical Sciences, NeurIPS. 142. find
  140. Panayotov, V., G. Chen, D. Povey, and S. Khudanpur (2015). “Librispeech: an asr corpus based on public domain audio books”. In: 2015 IEEE Intl. Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 5206–5210. 143. Patton, B., Y. Agiomyrgiannakis, M. Terry, K. Wilson, R. A. Saurous, and D. find
  141. Sculley (2016). “AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech”. In: arXiv preprint arXiv:1611.09207. 144. ↗
  142. Peng, L., K. Fu, B. Lin, D. Ke, and J. Zhan (2021). “A Study on Fine-Tuning wav2vec2.0 Model for the Task of Mispronunciation Detection and Diagnosis”. find
  143. In: Proc. Interspeech 2021, pp. 4448–4452. DOI : 10.21437/Interspeech.2021-1344.145. Piotrowska, M., A. Czyżewski, T. Ciszewski, G. Korvel, A. Kurowski, and B. ↗
  144. Kostek (2021). “Evaluation of aspiration problems in L2 English pronunciation employing machine learning”. In: The Journal of the Acoustical Society of America 150.1, pp. 120–132. 146. find
  145. Plantinga, P. and E. Fosler-Lussier (2019). “Towards Real-Time Mispronunciation Detection in Kids’ Speech”. In: 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, pp. 690–696. 147. find
  146. Porzuczek, A. and A. Rojczyk (2017). “English word stress in Polish learners speech production and metacompetence”. In: Research in Language 15.4, pp. 313– 323. 148. ↗
  147. Posner, M. I. and S. E. Petersen (1990). “The attention system of the human brain”. In: Annual review of neuroscience 13.1, pp. 25–42. 149. ↗
  148. Qian, X., H. Meng, and F. Soong (2010). “Capturing L2 segmental mispronunciations with joint-sequence models in computer-aided pronunciation training (CAPT)”. In: 2010 7th Intl. Symposium on Chinese Spoken Language Processing. IEEE, pp. 84–88. 150. find
  149. Rabiner, L. and R. Schafer (1978). Digital Processing of Speech Signals. Englewood Cliffs: Prentice Hall. 151. find
  150. Radzikowski, K., L. Wang, and O. Yoshie (2016). “Non-native English speakers’ speech correction, based on domain focused document”. In: Proceedings of the 18th International Conference on Information Integration and Web-based Applications and Services, pp. 276–281. 152. find
  151. Ramanathi, M. K., C. Yarra, and P. K. Ghosh (2019). “ASR Inspired Syllable Stress Detection for Pronunciation Evaluation Without Using a Supervised Classifier and Syllable Level Features.” In: INTERSPEECH, pp. 924–928. 153. find
  152. Ren, Y., Y. Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T.-Y. Liu (2019). “Fastspeech: Fast, robust and controllable text to speech”. In: arXiv preprint arXiv:1905.09263. 154. ↗
  153. Romana, A., J. Bandon, M. Perez, S. Gutierrez, R. Richter, A. Roberts, and E. M. Provost (2021). “Automatically Detecting Errors and Disfluencies in Read Speech to Predict Cognitive Impairment in People with Parkinson’s Disease”. In: 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021. International Speech Communication Association, pp. 156–160. 155. find
  154. Rosenberg, A. and B. Ramabhadran (2017). “Bias and Statistical Significance in Evaluating Speech Synthesis with Mean Opinion Scores.” In: Interspeech, pp. 3976–3980. 156. find
  155. Rosenblatt, F. (1960). “Perceptron simulation experiments”. In: Proceedings of the IRE 48.3, pp. 301–309. 157. ↗
  156. Ruan, Y., X. Wang, H. Liu, Z. Ou, Y. Gao, J. Cheng, and Y. Qian (2019). “An End-to-end Approach for Lexical Stress Detection based on Transformer”. In: arXiv preprint arXiv:1911.04862. 158. ↗
  157. Rudzicz, F., A. K. Namasivayam, and T. Wolff (2012). “The TORGO database of acoustic and articulatory speech from speakers with dysarthria”. In: Language Resources and Evaluation. ISSN : 1574020X. DOI : 10.1007/s10579-011-9145-0. arXiv: 0507464v2 [arXiv:astro-ph]. 159. ↗
  158. Särkkä, S. (2013). Bayesian filtering and smoothing. 3. Cambridge University Press. 160. ↗
  159. Sarria-Paja, M. and T. Falk (2012). “Automated Dysarthria Severity Classification for Improved Objective Intelligibility Assessment of Spastic Dysarthric Speech.” In: Interspeech. ISBN : 9781622767595. 161. find
  160. Series, B. (2014). “Method for the subjective assessment of intermediate quality level of audio systems”. In: International Telecommunication Union Radiocommunication Assembly. 162. find
  161. Shah, R., K. Pokora, A. Ezzerg, V. Klimkov, G. Huybrechts, B. Putrycz, D. Korzekwa, and T. Merritt (2021). “Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech”. In: Proc. 11th ISCA Speech Synthesis Workshop (SSW 11), pp. 96–101. DOI : 10.21437/SSW.2021-17.163. ↗
  162. Shahin, M. A., J. Epps, and B. Ahmed (2016). “Automatic Classification of Lexical Stress in English and Arabic Languages Using Deep Learning.” In: INTERSPEECH, pp. 175–179. 164. find
  163. Shattuck-Hufnagel, S., M. Ostendorf, and K. Ross (1994). “Stress shift and early pitch accent placement in lexical items in American English”. In: Journal of Phonetics 22.4, pp. 357–388. 165. ↗
  164. Skerry-Ryan, R. J., E. Battenberg, Y. Xiao, Y. Wang, D. Stanton, J. Shor, R. J. Weiss, R. Clark, and R. A. Saurous (2018). “Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron”. In: CoRR abs/1803.0. arXiv: 1803.09047. 166. ↗
  165. Skerry-Ryan, R., E. Battenberg, Y. Xiao, Y. Wang, D. Stanton, J. Shor, R. Weiss, R. Clark, and R. A. Saurous (2018). “Towards end-to-end prosody transfer for expressive speech synthesis with tacotron”. In: international conference on machine learning. PMLR, pp. 4693–4702. 167. ↗
  166. Sofaer, H. R., J. A. Hoeting, and C. S. Jarnevich (2019). “The area under the precision-recall curve as a performance metric for rare binary events”. In: Methods in Ecology and Evolution 10.4, pp. 565–577. 168. ↗
  167. Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov (2014). “Dropout: A Simple Way to Prevent Neural Networks from Overfitting”. find
  168. In: Journal of Machine Learning Research 15, pp. 1929–1958. 169. find
  169. Statista (2021). Most common languages used on the internet as of January 2020, by share of internet users. URL : https://www.statista.com/statistics/262946/share-of-the-most-common-languages-on-the-internet/.170. ↗
  170. Sudhakara, S., M. K. Ramanathi, C. Yarra, A. Das, and P. Ghosh (2019). “Noise robust goodness of pronunciation measures using teacher’s utterance”. In: SLaTE. 171. find
  171. Sudhakara, S., M. K. Ramanathi, C. Yarra, and P. K. Ghosh (2019). “An Improved Goodness of Pronunciation (GoP) Measure for Pronunciation Evaluation with DNN-HMM System Considering HMM Transition Probabilities.” In: INTER-SPEECH, pp. 954–958. 172. find
  172. Sutskever, I., O. Vinyals, and Q. V. Le (2014). “Sequence to sequence learning with neural networks”. In: Advances in neural information processing systems, pp. 3104–3112. 173. ↗
  173. Tejedor-García, C., D. Escudero, E. Cámara-Arenas, C. González-Ferreras, and V. Cardeñoso-Payo (2020). “Assessing pronunciation improvement in students of English using a controlled computer-assisted pronunciation tool”. In: IEEE Transactions on Learning Technologies. 174. find
  174. Todhunter, I. (2014). A history of the mathematical theory of probability. Cambridge University Press. 175. ↗
  175. Trujillo, F. (2006). “The production of speech sounds”. In: English Phonetics and Phonology. URL : https://www.ugr.es/~ftsaez/fonetica/production_speech. pdf. 176. ↗
  176. Tu, M., V. Berisha, and J. Liss (2017). “Interpretable Objective Assessment of Dysarthric Speech Based on Deep Neural Networks”. In: Interspeech 2017, 18th Annual Conference of the International Speech Communication Association, Stockholm, Sweden, August 20-24, 2017. Ed. by F. Lacerda. ISCA, pp. 1849–1853. DOI : 10.21437/Interspeech.2017.177. ↗
  177. UNESCO (2016). If you don’t understand, how can you learn? URL : https://en.unesco . org / news / 40 - don - t - access - education - language - they - understand. 178. University physics Volume 1 (2016). eng. ISBN : 1-938168-27-5. 179. ↗
  178. Valizada, A., S. Jafarova, E. Sultanov, and S. Rustamov (2021). “Development and Evaluation of Speech Synthesis System Based on Deep Learning Models”. In: Symmetry 13.5, p. 819. 180. ↗
  179. Van Den Oord, A., O. Vinyals, et al. (2017). “Neural discrete representation learning”. In: Advances in Neural Information Processing Systems 30, pp. 6306– 6315. 181. ↗
  180. Vásquez-Correa, J. C., T. Arias-Vergara, J. R. Orozco-Arroyave, and E. Nöth (2018). “A Multitask Learning Approach to Assess the Dysarthria Severity in Patients with Parkinson’s Disease”. In: Interspeech 2018, 19th Annual Conference of the International Speech Communication Association, Hyderabad, India, 2-6 September 2018. Ed. by B. Yegnanarayana. ISCA, pp. 456–460. DOI : 10.21437/Interspeech.2018-1988.182. ↗
  181. Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin (2017). “Attention Is All You Need”. In: CoRR abs/1706.0. arXiv: 1706.03762. 183. Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. ↗
  182. Kaiser, and I. Polosukhin (2017). “Attention is all you need”. In: Advances in neural information processing systems, pp. 5998–6008. 184.
  183. Wagner, P., J. Beskow, S. Betz, J. Edlund, J. Gustafson, G. Eje Henter, S. Le Maguer, Z. Malisz, É. Székely, C. Tånnander, et al. (2019). “Speech synthesis evaluation—state-of-the-art assessment and suggestion for a novel research program”. In: Proceedings of the 10th Speech Synthesis Workshop (SSW10). 185. find
  184. Wang, J., Y. Qin, Z. Peng, and T. Lee (2019). “Child Speech Disorder Detection with Siamese Recurrent Network Using Speech Attribute Features.” In: INTERSPEECH, pp. 3885–3889. 186. find
  185. Wang, Y., R. J. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio, Q. V. Le, Y. Agiomyrgiannakis, R. Clark, and R. A. Saurous (2017). “Tacotron: {A} Fully End-to-End Text-To-Speech Synthesis Model”. In: CoRR abs/1703.1. arXiv: 1703.10135. 187. Wang, Y., R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. ↗
  186. Xiao, Z. Chen, S. Bengio, et al. (2017). “Tacotron: Towards end-to-end speech synthesis”. In: arXiv preprint arXiv:1703.10135. 188. ↗
  187. Weber, D., S. Zaporowski, and D. Korzekwa (2020). “Constructing a Dataset of Speech Recordings with Lombard Effect”. In: 24th IEEE SPA. DOI : 10.23919/ find
  188. Welch, L. R. (2003). “Hidden Markov models and the Baum-Welch algorithm”. In: IEEE Information Theory Society Newsletter 53.4, pp. 10–13. URL : http://yanfenglu.net/documents/Baum-Welch_Algorithm.pdf. 190. ↗
  189. Williams, C. K. and C. E. Rasmussen (2006). Gaussian processes for machine learning. Vol. 2. 3. MIT press Cambridge, MA. DOI : 10.7551/mitpress/3206.001.0001.191. ↗
  190. Witt, S. M. and S. J. Young (2000). “Phone-level pronunciation scoring and assessment for interactive language learning”. In: Speech communication 30.2-3, pp. 95–108. 192. ↗
  191. Wong, S. C., A. Gatt, V. Stamatescu, and M. D. McDonnell (2016). “Understanding data augmentation for classification: when to warp?” In: 2016 international conference on digital image computing: techniques and applications (DICTA). IEEE, pp. 1–6. 193. find
  192. Woolson, R. (2007). “Wilcoxon signed-rank test”. In: Wiley encyclopedia of clinical trials, pp. 1–3. 194. ↗
  193. WorldEconomicForum (2018). Speaking more than one language can boost economic growth. URL : https://www.weforum.org/agenda/2018/02/speaking-more-languages-boost-economic-growth. 195. ↗
  194. Xiao, Y., F. K. Soong, and W. Hu (2018). “Paired phone-posteriors approach to esl pronunciation quality assessment”. In: bdl. Vol. 1. 782d, p. 3. 196. find
  195. Xu, X., Y. Kang, S. Cao, B. Lin, and L. Ma (2021). “Explore wav2vec 2.0 for Mispronunciation Detection”. In: Proc. Interspeech 2021, pp. 4428–4432. DOI : 10.21437/Interspeech.2021-777.197. ↗
  196. Yamagishi, J., C. Veaux, S. King, and S. Renals (2012). “Speech synthesis technologies for individuals with vocal disabilities: Voice banking and reconstruction”. In: Acoustical Science and Technology 33.1, pp. 1–5. DOI : 10.1250/ast.33.1.198. ↗
  197. Yan, B.-C. and B. Chen (2021). “End-to-End Mispronunciation Detection and Diagnosis From Raw Waveforms”. In: arXiv preprint arXiv:2103.03023. 199. ↗
  198. Yan, B.-C., S.-W. F. Jiang, F.-A. Chao, and B. Chen (2021). “Maximum F1-score training for end-to-end mispronunciation detection and diagnosis of L2 English speech”. In: arXiv preprint arXiv:2108.13816. 200. ↗
  199. Yan, B.-C., M.-C. Wu, H.-T. Hung, and B. Chen (2020). “An End-to-End Mispronunciation Detection System for L2 English Speech Leveraging Novel Anti-Phone Modeling”. In: Proc. Interspeech 2020, pp. 3032–3036. DOI : 10.21437/
  200. Zen, H., V. Dang, R. Clark, Y. Zhang, R. J. Weiss, Y. Jia, Z. Chen, and Y. Wu (2019). “LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech”. In: Proc. Interspeech 2019, pp. 1526–1530. DOI : 10.21437/Interspeech.2019-2441.202. ↗
  201. Zhang, D., A. Ganesan, S. Campbell, and D. Korzekwa (2022). “L2-GEN: A Neural Phoneme Paraphrasing Approach to L2 Speech Synthesis for Mispronunciation Diagnosis”. In: accepted to Interspeech 2022. 203. find
  202. Zhang, Y.-J., S. Pan, L. He, and Z.-H. Ling (2018). “Learning latent representations for style control and transfer in end-to-end speech synthesis”. In: CoRR abs/1812.0. arXiv: 1812.04342. 204. ↗
  203. Zhang, L., Z. Zhao, C. Ma, L. Shan, H. Sun, L. Jiang, S. Deng, and C. Gao (2020). “End-to-end automatic pronunciation error detection based on improved hybrid ctc/attention architecture”. In: Sensors 20.7, p. 1809. 205. ↗
  204. Zhang, Z., Y. Wang, and J. Yang (2021). “Text-conditioned Transformer for automatic pronunciation error detection”. In: Speech Communication 130, pp. 55– 63. 206. ↗
  205. Zhao, G., S. Sonsaat, A. O. Silpachai, I. Lucic, E. Chukharev-Hudilainen, J. Levis, and R. Gutierrez-Osuna (2018). “L2-ARCTIC: A non-native English speech corpus”. In: Perception Sensing Instrumentation Lab. 207. find
  206. Zhao, J., H. Yuan, J. Liu, and S. Xia (2011). “Automatic lexical stress detection using acoustic features for computer assisted language learning”. In: Proc. APSIPA ASC, pp. 247–251. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/korzekwa-2022-automated-detection-pronunciation-errors-non/